2024-04-26T21:27:53.046 INFO:root:teuthology version: 0.0.1.dev269+g9481b1d 2024-04-26T21:27:53.046 DEBUG:teuthology.run:Teuthology command: teuthology --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/fuse objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/no 5-workunit/suites/fsync-tester}} --owner scheduled_rishabh@teuthology --verbose --name rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi --archive /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675222 -- /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675222/orig.config.yaml 2024-04-26T21:27:53.104 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-26T21:27:53.191 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675222 branch: wip-rishabh-testing-20240426.111959 description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/fuse objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/no 5-workunit/suites/fsync-tester}} email: null first_in_suite: false job_id: '7675222' kernel: flavor: default kdb: true sha1: 806ec36b21129f113ce356c007dd59694d340b37 last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-rishabh-testing-20240426.111959 ceph: cephfs: max_mds: 5 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - slow metadata IO - SLOW_OPS - slow request sha1: bc612ffb9c1d5bc27536bc917a705262a36d4387 subvols: create: 2 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported dry_run: true install: ceph: flavor: default sha1: bc612ffb9c1d5bc27536bc917a705262a36d4387 kclient: client.0: mount_subvol_num: 1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-rishabh-testing-20240426.111959 sha1: bc612ffb9c1d5bc27536bc917a705262a36d4387 owner: scheduled_rishabh@teuthology priority: 50 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 816 sha1: bc612ffb9c1d5bc27536bc917a705262a36d4387 sleep_before_teardown: 0 subset: 166/512 suite: fs suite_branch: wip-rishabh-testing-20240426.111959 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_bc612ffb9c1d5bc27536bc917a705262a36d4387/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: bc612ffb9c1d5bc27536bc917a705262a36d4387 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi072.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOO9j6nfuSaYnYgJ4CCQeXI2ZQvnyqhAlAMdm7yDr4UqH+LXjC2p8GH3dBC5lgZJYuEdv6rVF0NJim3y6Z64kPI= smithi076.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKEiBHDhCJmqE4kC4apcz20271tkHnhasHtvmur2ihdLkaD1EJj5U06sZklkWMIkF0teFHAVTQtRLQlp3jcQo48= smithi164.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPcotv7qJm0okzUvZsfvRA2q1YGczSxK+UYcRcEqXGtb6XUBjOASSSrAKfRpTlQ8q/0m5ozS7wHmPnLHXIr/rn0= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - ceph-fuse: null - exec: mon.a: - ceph fs set cephfs balance_automate true - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - workunit: clients: all: - suites/fsync-tester.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: 9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17 timestamp: 2024-04-26_19:30:57 tube: smithi user: rishabh verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-26T21:27:53.191 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_bc612ffb9c1d5bc27536bc917a705262a36d4387/qa; will attempt to use it 2024-04-26T21:27:53.192 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_bc612ffb9c1d5bc27536bc917a705262a36d4387/qa/tasks 2024-04-26T21:27:53.192 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-26T21:27:53.193 INFO:teuthology.task.internal:Checking packages... 2024-04-26T21:27:53.217 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'bc612ffb9c1d5bc27536bc917a705262a36d4387' 2024-04-26T21:27:53.217 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-26T21:27:53.217 INFO:teuthology.packaging:ref: None 2024-04-26T21:27:53.217 INFO:teuthology.packaging:tag: None 2024-04-26T21:27:53.217 INFO:teuthology.packaging:branch: wip-rishabh-testing-20240426.111959 2024-04-26T21:27:53.217 INFO:teuthology.packaging:sha1: bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T21:27:53.217 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-rishabh-testing-20240426.111959 2024-04-26T21:27:53.473 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-3306.gbc612ffb 2024-04-26T21:27:53.475 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-26T21:27:53.483 INFO:teuthology.task.internal:no buildpackages task found 2024-04-26T21:27:53.483 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-26T21:27:53.492 INFO:teuthology.task.internal:Saving configuration 2024-04-26T21:27:53.512 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-26T21:27:53.566 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-26T21:27:53.589 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi072.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675222', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-26 21:22:20.657888', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOO9j6nfuSaYnYgJ4CCQeXI2ZQvnyqhAlAMdm7yDr4UqH+LXjC2p8GH3dBC5lgZJYuEdv6rVF0NJim3y6Z64kPI='} 2024-04-26T21:27:53.610 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi076.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675222', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-26 21:22:20.661625', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKEiBHDhCJmqE4kC4apcz20271tkHnhasHtvmur2ihdLkaD1EJj5U06sZklkWMIkF0teFHAVTQtRLQlp3jcQo48='} 2024-04-26T21:27:53.631 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi164.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675222', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-26 21:22:20.659947', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPcotv7qJm0okzUvZsfvRA2q1YGczSxK+UYcRcEqXGtb6XUBjOASSSrAKfRpTlQ8q/0m5ozS7wHmPnLHXIr/rn0='} 2024-04-26T21:27:53.631 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-26T21:27:53.638 INFO:teuthology.task.internal:roles: ubuntu@smithi072.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-04-26T21:27:53.638 INFO:teuthology.task.internal:roles: ubuntu@smithi076.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-04-26T21:27:53.638 INFO:teuthology.task.internal:roles: ubuntu@smithi164.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-04-26T21:27:53.638 INFO:teuthology.run_tasks:Running task console_log... 2024-04-26T21:27:53.731 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f22e7bb0160>, signals=[15]) 2024-04-26T21:27:53.732 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-26T21:27:53.737 INFO:teuthology.task.internal:Opening connections... 2024-04-26T21:27:53.737 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi072.front.sepia.ceph.com 2024-04-26T21:27:53.739 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T21:27:53.815 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi076.front.sepia.ceph.com 2024-04-26T21:27:53.816 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi076.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T21:27:53.889 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi164.front.sepia.ceph.com 2024-04-26T21:27:53.890 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi164.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T21:27:53.965 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-26T21:27:53.971 DEBUG:teuthology.orchestra.run.smithi072:> uname -m 2024-04-26T21:27:53.989 INFO:teuthology.orchestra.run.smithi072.stdout:x86_64 2024-04-26T21:27:53.989 DEBUG:teuthology.orchestra.run.smithi072:> cat /etc/os-release 2024-04-26T21:27:54.046 INFO:teuthology.orchestra.run.smithi072.stdout:NAME="CentOS Stream" 2024-04-26T21:27:54.046 INFO:teuthology.orchestra.run.smithi072.stdout:VERSION="9" 2024-04-26T21:27:54.046 INFO:teuthology.orchestra.run.smithi072.stdout:ID="centos" 2024-04-26T21:27:54.046 INFO:teuthology.orchestra.run.smithi072.stdout:ID_LIKE="rhel fedora" 2024-04-26T21:27:54.047 INFO:teuthology.orchestra.run.smithi072.stdout:VERSION_ID="9" 2024-04-26T21:27:54.047 INFO:teuthology.orchestra.run.smithi072.stdout:PLATFORM_ID="platform:el9" 2024-04-26T21:27:54.047 INFO:teuthology.orchestra.run.smithi072.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-26T21:27:54.047 INFO:teuthology.orchestra.run.smithi072.stdout:ANSI_COLOR="0;31" 2024-04-26T21:27:54.047 INFO:teuthology.orchestra.run.smithi072.stdout:LOGO="fedora-logo-icon" 2024-04-26T21:27:54.047 INFO:teuthology.orchestra.run.smithi072.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-26T21:27:54.047 INFO:teuthology.orchestra.run.smithi072.stdout:HOME_URL="https://centos.org/" 2024-04-26T21:27:54.047 INFO:teuthology.orchestra.run.smithi072.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-26T21:27:54.047 INFO:teuthology.orchestra.run.smithi072.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-26T21:27:54.047 INFO:teuthology.orchestra.run.smithi072.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-26T21:27:54.048 INFO:teuthology.lock.ops:Updating smithi072.front.sepia.ceph.com on lock server 2024-04-26T21:27:54.067 DEBUG:teuthology.orchestra.run.smithi076:> uname -m 2024-04-26T21:27:54.085 INFO:teuthology.orchestra.run.smithi076.stdout:x86_64 2024-04-26T21:27:54.085 DEBUG:teuthology.orchestra.run.smithi076:> cat /etc/os-release 2024-04-26T21:27:54.142 INFO:teuthology.orchestra.run.smithi076.stdout:NAME="CentOS Stream" 2024-04-26T21:27:54.142 INFO:teuthology.orchestra.run.smithi076.stdout:VERSION="9" 2024-04-26T21:27:54.142 INFO:teuthology.orchestra.run.smithi076.stdout:ID="centos" 2024-04-26T21:27:54.142 INFO:teuthology.orchestra.run.smithi076.stdout:ID_LIKE="rhel fedora" 2024-04-26T21:27:54.142 INFO:teuthology.orchestra.run.smithi076.stdout:VERSION_ID="9" 2024-04-26T21:27:54.142 INFO:teuthology.orchestra.run.smithi076.stdout:PLATFORM_ID="platform:el9" 2024-04-26T21:27:54.142 INFO:teuthology.orchestra.run.smithi076.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-26T21:27:54.142 INFO:teuthology.orchestra.run.smithi076.stdout:ANSI_COLOR="0;31" 2024-04-26T21:27:54.143 INFO:teuthology.orchestra.run.smithi076.stdout:LOGO="fedora-logo-icon" 2024-04-26T21:27:54.143 INFO:teuthology.orchestra.run.smithi076.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-26T21:27:54.143 INFO:teuthology.orchestra.run.smithi076.stdout:HOME_URL="https://centos.org/" 2024-04-26T21:27:54.143 INFO:teuthology.orchestra.run.smithi076.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-26T21:27:54.143 INFO:teuthology.orchestra.run.smithi076.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-26T21:27:54.143 INFO:teuthology.orchestra.run.smithi076.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-26T21:27:54.143 INFO:teuthology.lock.ops:Updating smithi076.front.sepia.ceph.com on lock server 2024-04-26T21:27:54.158 DEBUG:teuthology.orchestra.run.smithi164:> uname -m 2024-04-26T21:27:54.176 INFO:teuthology.orchestra.run.smithi164.stdout:x86_64 2024-04-26T21:27:54.176 DEBUG:teuthology.orchestra.run.smithi164:> cat /etc/os-release 2024-04-26T21:27:54.234 INFO:teuthology.orchestra.run.smithi164.stdout:NAME="CentOS Stream" 2024-04-26T21:27:54.234 INFO:teuthology.orchestra.run.smithi164.stdout:VERSION="9" 2024-04-26T21:27:54.234 INFO:teuthology.orchestra.run.smithi164.stdout:ID="centos" 2024-04-26T21:27:54.234 INFO:teuthology.orchestra.run.smithi164.stdout:ID_LIKE="rhel fedora" 2024-04-26T21:27:54.234 INFO:teuthology.orchestra.run.smithi164.stdout:VERSION_ID="9" 2024-04-26T21:27:54.234 INFO:teuthology.orchestra.run.smithi164.stdout:PLATFORM_ID="platform:el9" 2024-04-26T21:27:54.234 INFO:teuthology.orchestra.run.smithi164.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-26T21:27:54.234 INFO:teuthology.orchestra.run.smithi164.stdout:ANSI_COLOR="0;31" 2024-04-26T21:27:54.235 INFO:teuthology.orchestra.run.smithi164.stdout:LOGO="fedora-logo-icon" 2024-04-26T21:27:54.235 INFO:teuthology.orchestra.run.smithi164.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-26T21:27:54.235 INFO:teuthology.orchestra.run.smithi164.stdout:HOME_URL="https://centos.org/" 2024-04-26T21:27:54.235 INFO:teuthology.orchestra.run.smithi164.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-26T21:27:54.235 INFO:teuthology.orchestra.run.smithi164.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-26T21:27:54.235 INFO:teuthology.orchestra.run.smithi164.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-26T21:27:54.235 INFO:teuthology.lock.ops:Updating smithi164.front.sepia.ceph.com on lock server 2024-04-26T21:27:54.250 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-26T21:27:54.261 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-26T21:27:54.268 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-26T21:27:54.268 DEBUG:teuthology.orchestra.run.smithi072:> test '!' -e /home/ubuntu/cephtest 2024-04-26T21:27:54.270 DEBUG:teuthology.orchestra.run.smithi076:> test '!' -e /home/ubuntu/cephtest 2024-04-26T21:27:54.273 DEBUG:teuthology.orchestra.run.smithi164:> test '!' -e /home/ubuntu/cephtest 2024-04-26T21:27:54.290 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-26T21:27:54.296 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-26T21:27:54.297 DEBUG:teuthology.orchestra.run.smithi072:> test -z $(ls -A /var/lib/ceph) 2024-04-26T21:27:54.327 DEBUG:teuthology.orchestra.run.smithi076:> test -z $(ls -A /var/lib/ceph) 2024-04-26T21:27:54.329 DEBUG:teuthology.orchestra.run.smithi164:> test -z $(ls -A /var/lib/ceph) 2024-04-26T21:27:54.427 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-26T21:27:54.494 INFO:teuthology.run_tasks:Running task kernel... 2024-04-26T21:27:54.509 INFO:teuthology.task.kernel:normalize config orig: {'flavor': 'default', 'kdb': True, 'sha1': '806ec36b21129f113ce356c007dd59694d340b37'} 2024-04-26T21:27:54.510 INFO:teuthology.task.kernel:config {'mon.a': {'flavor': 'default', 'kdb': True, 'sha1': '806ec36b21129f113ce356c007dd59694d340b37'}, 'mon.b': {'flavor': 'default', 'kdb': True, 'sha1': '806ec36b21129f113ce356c007dd59694d340b37'}, 'mon.c': {'flavor': 'default', 'kdb': True, 'sha1': '806ec36b21129f113ce356c007dd59694d340b37'}}, timeout 300 2024-04-26T21:27:54.510 DEBUG:teuthology.orchestra.run.smithi072:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-26T21:27:54.511 DEBUG:teuthology.orchestra.run.smithi076:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-26T21:27:54.511 DEBUG:teuthology.orchestra.run.smithi164:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-26T21:27:54.527 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T21:27:54.527 DEBUG:teuthology.task.kernel:sha1 for mon.b is 806ec36b21129f113ce356c007dd59694d340b37 2024-04-26T21:27:54.527 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "806ec36b21129f113ce356c007dd59694d340b37"... 2024-04-26T21:27:54.528 DEBUG:teuthology.orchestra.run.smithi076:> uname -r 2024-04-26T21:27:54.528 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T21:27:54.529 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T21:27:54.529 DEBUG:teuthology.task.kernel:sha1 for mon.c is 806ec36b21129f113ce356c007dd59694d340b37 2024-04-26T21:27:54.529 INFO:teuthology.task.kernel:Checking kernel version of mon.c, want "806ec36b21129f113ce356c007dd59694d340b37"... 2024-04-26T21:27:54.529 DEBUG:teuthology.orchestra.run.smithi164:> uname -r 2024-04-26T21:27:54.530 DEBUG:teuthology.task.kernel:sha1 for mon.a is 806ec36b21129f113ce356c007dd59694d340b37 2024-04-26T21:27:54.530 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "806ec36b21129f113ce356c007dd59694d340b37"... 2024-04-26T21:27:54.530 DEBUG:teuthology.orchestra.run.smithi072:> uname -r 2024-04-26T21:27:54.584 INFO:teuthology.orchestra.run.smithi076.stdout:5.14.0-437.el9.x86_64 2024-04-26T21:27:54.584 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-437.el9.x86_64 vs 806ec36b21129f113ce356c007dd59694d340b37 2024-04-26T21:27:54.585 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-04-26T21:27:54.585 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=806ec36b21129f113ce356c007dd59694d340b37 2024-04-26T21:27:54.589 INFO:teuthology.orchestra.run.smithi164.stdout:5.14.0-437.el9.x86_64 2024-04-26T21:27:54.589 INFO:teuthology.orchestra.run.smithi072.stdout:5.14.0-437.el9.x86_64 2024-04-26T21:27:54.590 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-437.el9.x86_64 vs 806ec36b21129f113ce356c007dd59694d340b37 2024-04-26T21:27:54.590 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-04-26T21:27:54.590 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=806ec36b21129f113ce356c007dd59694d340b37 2024-04-26T21:27:54.591 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-437.el9.x86_64 vs 806ec36b21129f113ce356c007dd59694d340b37 2024-04-26T21:27:54.591 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-04-26T21:27:54.591 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=806ec36b21129f113ce356c007dd59694d340b37 2024-04-26T21:27:54.735 DEBUG:teuthology.orchestra.run.smithi164:> sudo yum upgrade -y linux-firmware 2024-04-26T21:27:54.797 DEBUG:teuthology.orchestra.run.smithi076:> sudo yum upgrade -y linux-firmware 2024-04-26T21:27:54.809 DEBUG:teuthology.orchestra.run.smithi072:> sudo yum upgrade -y linux-firmware 2024-04-26T21:27:57.083 INFO:teuthology.orchestra.run.smithi076.stdout:CentOS Stream 9 - BaseOS 12 MB/s | 8.0 MB 00:00 2024-04-26T21:27:57.772 INFO:teuthology.orchestra.run.smithi072.stdout:CentOS Stream 9 - BaseOS 13 MB/s | 8.0 MB 00:00 2024-04-26T21:27:59.011 INFO:teuthology.orchestra.run.smithi076.stdout:CentOS Stream 9 - AppStream 22 MB/s | 19 MB 00:00 2024-04-26T21:27:59.617 INFO:teuthology.orchestra.run.smithi072.stdout:CentOS Stream 9 - AppStream 25 MB/s | 19 MB 00:00 2024-04-26T21:28:03.137 INFO:teuthology.orchestra.run.smithi076.stdout:CentOS Stream 9 - CRB 12 MB/s | 6.2 MB 00:00 2024-04-26T21:28:03.821 INFO:teuthology.orchestra.run.smithi072.stdout:CentOS Stream 9 - CRB 9.3 MB/s | 6.2 MB 00:00 2024-04-26T21:28:04.540 INFO:teuthology.orchestra.run.smithi164.stdout:CentOS Stream 9 - BaseOS 1.0 MB/s | 8.0 MB 00:08 2024-04-26T21:28:05.022 INFO:teuthology.orchestra.run.smithi076.stdout:CentOS Stream 9 - Extras packages 31 kB/s | 16 kB 00:00 2024-04-26T21:28:05.462 INFO:teuthology.orchestra.run.smithi072.stdout:CentOS Stream 9 - Extras packages 34 kB/s | 16 kB 00:00 2024-04-26T21:28:07.007 INFO:teuthology.orchestra.run.smithi076.stdout:Extra Packages for Enterprise Linux 11 MB/s | 21 MB 00:01 2024-04-26T21:28:12.015 INFO:teuthology.orchestra.run.smithi076.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-04-26T21:28:12.204 INFO:teuthology.orchestra.run.smithi076.stdout:Last metadata expiration check: 0:00:01 ago on Fri 26 Apr 2024 09:28:11 PM UTC. 2024-04-26T21:28:14.036 INFO:teuthology.orchestra.run.smithi076.stdout:Dependencies resolved. 2024-04-26T21:28:14.037 INFO:teuthology.orchestra.run.smithi076.stdout:Nothing to do. 2024-04-26T21:28:14.037 INFO:teuthology.orchestra.run.smithi076.stdout:Complete! 2024-04-26T21:28:14.153 INFO:teuthology.task.kernel:Downloading kernel 806ec36b21129f113ce356c007dd59694d340b37 on mon.b... 2024-04-26T21:28:14.153 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=806ec36b21129f113ce356c007dd59694d340b37 2024-04-26T21:28:14.372 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ 2024-04-26T21:28:14.372 DEBUG:teuthology.orchestra.run.smithi076:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ --input-file=- 2024-04-26T21:28:14.375 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi076.front.sepia.ceph.com to complete... 2024-04-26T21:28:24.713 INFO:teuthology.orchestra.run.smithi164.stdout:CentOS Stream 9 - AppStream 1.0 MB/s | 19 MB 00:19 2024-04-26T21:28:25.101 INFO:teuthology.orchestra.run.smithi072.stdout:Extra Packages for Enterprise Linux 1.1 MB/s | 21 MB 00:19 2024-04-26T21:28:26.491 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26 21:28:26 URL:https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm [357367792/357367792] -> "/tmp/kernel.x86_64.rpm" [1] 2024-04-26T21:28:26.492 INFO:teuthology.orchestra.run.smithi076.stderr:FINISHED --2024-04-26 21:28:26-- 2024-04-26T21:28:26.492 INFO:teuthology.orchestra.run.smithi076.stderr:Total wall clock time: 12s 2024-04-26T21:28:26.492 INFO:teuthology.orchestra.run.smithi076.stderr:Downloaded: 1 files, 341M in 12s (28.8 MB/s) 2024-04-26T21:28:26.494 INFO:teuthology.task.kernel:Installing kernel 806ec36b21129f113ce356c007dd59694d340b37 on mon.b... 2024-04-26T21:28:26.494 DEBUG:teuthology.orchestra.run.smithi076:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-04-26T21:28:28.625 INFO:teuthology.orchestra.run.smithi076.stdout:Verifying... ######################################## 2024-04-26T21:28:28.859 INFO:teuthology.orchestra.run.smithi076.stdout:Preparing... ######################################## 2024-04-26T21:28:28.969 INFO:teuthology.orchestra.run.smithi076.stdout:Updating / installing... 2024-04-26T21:28:29.056 INFO:teuthology.orchestra.run.smithi164.stdout:CentOS Stream 9 - CRB 9.3 MB/s | 6.2 MB 00:00 2024-04-26T21:28:30.174 INFO:teuthology.orchestra.run.smithi072.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-04-26T21:28:30.715 INFO:teuthology.orchestra.run.smithi164.stdout:CentOS Stream 9 - Extras packages 54 kB/s | 16 kB 00:00 2024-04-26T21:28:32.219 INFO:teuthology.orchestra.run.smithi072.stdout:Dependencies resolved. 2024-04-26T21:28:32.220 INFO:teuthology.orchestra.run.smithi072.stdout:Nothing to do. 2024-04-26T21:28:32.220 INFO:teuthology.orchestra.run.smithi072.stdout:Complete! 2024-04-26T21:28:32.348 INFO:teuthology.task.kernel:Downloading kernel 806ec36b21129f113ce356c007dd59694d340b37 on mon.a... 2024-04-26T21:28:32.349 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=806ec36b21129f113ce356c007dd59694d340b37 2024-04-26T21:28:32.519 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ 2024-04-26T21:28:32.519 DEBUG:teuthology.orchestra.run.smithi072:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ --input-file=- 2024-04-26T21:28:32.520 INFO:teuthology.orchestra.run.smithi164.stdout:Extra Packages for Enterprise Linux 12 MB/s | 21 MB 00:01 2024-04-26T21:28:32.522 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi072.front.sepia.ceph.com to complete... 2024-04-26T21:28:36.920 INFO:teuthology.orchestra.run.smithi076.stdout:kernel-6.9.0_rc3_g806ec36b2112-1 ######################################## 2024-04-26T21:28:37.566 INFO:teuthology.orchestra.run.smithi164.stdout:lab-extras 38 kB/s | 1.7 kB 00:00 2024-04-26T21:28:39.524 INFO:teuthology.orchestra.run.smithi164.stdout:Dependencies resolved. 2024-04-26T21:28:39.525 INFO:teuthology.orchestra.run.smithi164.stdout:Nothing to do. 2024-04-26T21:28:39.525 INFO:teuthology.orchestra.run.smithi164.stdout:Complete! 2024-04-26T21:28:39.628 INFO:teuthology.task.kernel:Downloading kernel 806ec36b21129f113ce356c007dd59694d340b37 on mon.c... 2024-04-26T21:28:39.628 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=806ec36b21129f113ce356c007dd59694d340b37 2024-04-26T21:28:39.848 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ 2024-04-26T21:28:39.848 DEBUG:teuthology.orchestra.run.smithi164:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ --input-file=- 2024-04-26T21:28:39.851 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi164.front.sepia.ceph.com to complete... 2024-04-26T21:28:46.055 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26 21:28:46 URL:https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm [357367792/357367792] -> "/tmp/kernel.x86_64.rpm" [1] 2024-04-26T21:28:46.055 INFO:teuthology.orchestra.run.smithi072.stderr:FINISHED --2024-04-26 21:28:46-- 2024-04-26T21:28:46.055 INFO:teuthology.orchestra.run.smithi072.stderr:Total wall clock time: 14s 2024-04-26T21:28:46.055 INFO:teuthology.orchestra.run.smithi072.stderr:Downloaded: 1 files, 341M in 13s (25.8 MB/s) 2024-04-26T21:28:46.058 INFO:teuthology.task.kernel:Installing kernel 806ec36b21129f113ce356c007dd59694d340b37 on mon.a... 2024-04-26T21:28:46.059 DEBUG:teuthology.orchestra.run.smithi072:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-04-26T21:28:48.145 INFO:teuthology.orchestra.run.smithi072.stdout:Verifying... ######################################## 2024-04-26T21:28:48.300 INFO:teuthology.orchestra.run.smithi072.stdout:Preparing... ######################################## 2024-04-26T21:28:48.403 INFO:teuthology.orchestra.run.smithi072.stdout:Updating / installing... 2024-04-26T21:28:56.381 INFO:teuthology.orchestra.run.smithi072.stdout:kernel-6.9.0_rc3_g806ec36b2112-1 ######################################## 2024-04-26T21:29:09.060 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26 21:29:09 URL:https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm [357367792/357367792] -> "/tmp/kernel.x86_64.rpm" [1] 2024-04-26T21:29:09.061 INFO:teuthology.orchestra.run.smithi164.stderr:FINISHED --2024-04-26 21:29:09-- 2024-04-26T21:29:09.061 INFO:teuthology.orchestra.run.smithi164.stderr:Total wall clock time: 29s 2024-04-26T21:29:09.061 INFO:teuthology.orchestra.run.smithi164.stderr:Downloaded: 1 files, 341M in 29s (11.9 MB/s) 2024-04-26T21:29:09.064 INFO:teuthology.task.kernel:Installing kernel 806ec36b21129f113ce356c007dd59694d340b37 on mon.c... 2024-04-26T21:29:09.064 DEBUG:teuthology.orchestra.run.smithi164:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-04-26T21:29:11.189 INFO:teuthology.orchestra.run.smithi164.stdout:Verifying... ######################################## 2024-04-26T21:29:11.310 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing... ######################################## 2024-04-26T21:29:11.416 INFO:teuthology.orchestra.run.smithi164.stdout:Updating / installing... 2024-04-26T21:29:16.388 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi076.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-04-26T21:29:16.389 DEBUG:teuthology.orchestra.run.smithi076:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-04-26T21:29:16.465 INFO:teuthology.orchestra.run.smithi076.stdout:/boot/System.map-6.9.0-rc3-g806ec36b2112 2024-04-26T21:29:16.465 INFO:teuthology.orchestra.run.smithi076.stdout:/boot/config-6.9.0-rc3-g806ec36b2112 2024-04-26T21:29:16.465 INFO:teuthology.orchestra.run.smithi076.stdout:/boot/vmlinuz-6.9.0-rc3-g806ec36b2112 2024-04-26T21:29:16.465 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/System.map 2024-04-26T21:29:16.465 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/config 2024-04-26T21:29:16.465 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel 2024-04-26T21:29:16.465 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch 2024-04-26T21:29:16.465 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86 2024-04-26T21:29:16.466 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto 2024-04-26T21:29:16.466 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-04-26T21:29:16.466 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-04-26T21:29:16.466 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-04-26T21:29:16.466 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-04-26T21:29:16.466 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-04-26T21:29:16.466 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-04-26T21:29:16.466 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-04-26T21:29:16.466 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32c-intel.ko 2024-04-26T21:29:16.466 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-04-26T21:29:16.466 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-04-26T21:29:16.466 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-04-26T21:29:16.466 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-04-26T21:29:16.467 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx2.ko 2024-04-26T21:29:16.467 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-04-26T21:29:16.467 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-04-26T21:29:16.467 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-04-26T21:29:16.467 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-04-26T21:29:16.467 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-04-26T21:29:16.467 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-04-26T21:29:16.467 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-04-26T21:29:16.467 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel 2024-04-26T21:29:16.467 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu 2024-04-26T21:29:16.467 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce 2024-04-26T21:29:16.467 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-04-26T21:29:16.467 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm 2024-04-26T21:29:16.467 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-amd.ko 2024-04-26T21:29:16.468 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-intel.ko 2024-04-26T21:29:16.468 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm.ko 2024-04-26T21:29:16.468 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto 2024-04-26T21:29:16.468 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ansi_cprng.ko 2024-04-26T21:29:16.468 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/anubis.ko 2024-04-26T21:29:16.468 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/arc4.ko 2024-04-26T21:29:16.468 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx 2024-04-26T21:29:16.468 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_memcpy.ko 2024-04-26T21:29:16.468 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_pq.ko 2024-04-26T21:29:16.468 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_raid6_recov.ko 2024-04-26T21:29:16.468 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_tx.ko 2024-04-26T21:29:16.468 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_xor.ko 2024-04-26T21:29:16.468 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/raid6test.ko 2024-04-26T21:29:16.468 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authenc.ko 2024-04-26T21:29:16.469 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authencesn.ko 2024-04-26T21:29:16.469 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blake2b_generic.ko 2024-04-26T21:29:16.469 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_common.ko 2024-04-26T21:29:16.469 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_generic.ko 2024-04-26T21:29:16.469 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/camellia_generic.ko 2024-04-26T21:29:16.469 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast5_generic.ko 2024-04-26T21:29:16.469 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast6_generic.ko 2024-04-26T21:29:16.469 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast_common.ko 2024-04-26T21:29:16.469 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ccm.ko 2024-04-26T21:29:16.469 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cmac.ko 2024-04-26T21:29:16.469 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crc32_generic.ko 2024-04-26T21:29:16.469 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_engine.ko 2024-04-26T21:29:16.469 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_user.ko 2024-04-26T21:29:16.469 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cts.ko 2024-04-26T21:29:16.470 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/des_generic.ko 2024-04-26T21:29:16.470 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/dh_generic.ko 2024-04-26T21:29:16.470 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/echainiv.ko 2024-04-26T21:29:16.470 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/essiv.ko 2024-04-26T21:29:16.470 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/fcrypt.ko 2024-04-26T21:29:16.470 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/khazad.ko 2024-04-26T21:29:16.470 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4.ko 2024-04-26T21:29:16.470 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4hc.ko 2024-04-26T21:29:16.470 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/md4.ko 2024-04-26T21:29:16.470 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/michael_mic.ko 2024-04-26T21:29:16.470 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcbc.ko 2024-04-26T21:29:16.470 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcrypt.ko 2024-04-26T21:29:16.470 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/rmd160.ko 2024-04-26T21:29:16.470 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/seed.ko 2024-04-26T21:29:16.471 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/serpent_generic.ko 2024-04-26T21:29:16.471 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tcrypt.ko 2024-04-26T21:29:16.471 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tea.ko 2024-04-26T21:29:16.471 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_common.ko 2024-04-26T21:29:16.471 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_generic.ko 2024-04-26T21:29:16.471 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/vmac.ko 2024-04-26T21:29:16.471 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/wp512.ko 2024-04-26T21:29:16.471 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xcbc.ko 2024-04-26T21:29:16.471 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xor.ko 2024-04-26T21:29:16.471 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xxhash_generic.ko 2024-04-26T21:29:16.471 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers 2024-04-26T21:29:16.471 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi 2024-04-26T21:29:16.471 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_ipmi.ko 2024-04-26T21:29:16.472 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_pad.ko 2024-04-26T21:29:16.472 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/ec_sys.ko 2024-04-26T21:29:16.472 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/platform_profile.ko 2024-04-26T21:29:16.472 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbs.ko 2024-04-26T21:29:16.472 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbshc.ko 2024-04-26T21:29:16.472 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/video.ko 2024-04-26T21:29:16.472 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata 2024-04-26T21:29:16.472 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/acard-ahci.ko 2024-04-26T21:29:16.472 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ahci_platform.ko 2024-04-26T21:29:16.472 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ata_generic.ko 2024-04-26T21:29:16.472 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/libahci_platform.ko 2024-04-26T21:29:16.472 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_acpi.ko 2024-04-26T21:29:16.472 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ali.ko 2024-04-26T21:29:16.472 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_amd.ko 2024-04-26T21:29:16.472 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_artop.ko 2024-04-26T21:29:16.473 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atiixp.ko 2024-04-26T21:29:16.473 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atp867x.ko 2024-04-26T21:29:16.473 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd640.ko 2024-04-26T21:29:16.473 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd64x.ko 2024-04-26T21:29:16.473 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cypress.ko 2024-04-26T21:29:16.473 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_efar.ko 2024-04-26T21:29:16.473 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt366.ko 2024-04-26T21:29:16.473 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt37x.ko 2024-04-26T21:29:16.473 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x2n.ko 2024-04-26T21:29:16.473 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x3.ko 2024-04-26T21:29:16.473 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it8213.ko 2024-04-26T21:29:16.473 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it821x.ko 2024-04-26T21:29:16.473 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_jmicron.ko 2024-04-26T21:29:16.473 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_marvell.ko 2024-04-26T21:29:16.474 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_mpiix.ko 2024-04-26T21:29:16.474 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_netcell.ko 2024-04-26T21:29:16.474 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ninja32.ko 2024-04-26T21:29:16.474 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87410.ko 2024-04-26T21:29:16.474 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87415.ko 2024-04-26T21:29:16.474 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_oldpiix.ko 2024-04-26T21:29:16.474 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_opti.ko 2024-04-26T21:29:16.474 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_optidma.ko 2024-04-26T21:29:16.474 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pcmcia.ko 2024-04-26T21:29:16.474 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc2027x.ko 2024-04-26T21:29:16.474 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-04-26T21:29:16.474 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_piccolo.ko 2024-04-26T21:29:16.475 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_rdc.ko 2024-04-26T21:29:16.475 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sch.ko 2024-04-26T21:29:16.475 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_serverworks.ko 2024-04-26T21:29:16.475 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sil680.ko 2024-04-26T21:29:16.475 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sis.ko 2024-04-26T21:29:16.475 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sl82c105.ko 2024-04-26T21:29:16.475 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_triflex.ko 2024-04-26T21:29:16.475 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_via.ko 2024-04-26T21:29:16.475 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pdc_adma.ko 2024-04-26T21:29:16.475 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_inic162x.ko 2024-04-26T21:29:16.475 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_mv.ko 2024-04-26T21:29:16.475 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_nv.ko 2024-04-26T21:29:16.475 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_promise.ko 2024-04-26T21:29:16.475 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_qstor.ko 2024-04-26T21:29:16.476 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil.ko 2024-04-26T21:29:16.476 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil24.ko 2024-04-26T21:29:16.476 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sis.ko 2024-04-26T21:29:16.476 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_svw.ko 2024-04-26T21:29:16.476 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sx4.ko 2024-04-26T21:29:16.476 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_uli.ko 2024-04-26T21:29:16.476 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_via.ko 2024-04-26T21:29:16.476 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_vsc.ko 2024-04-26T21:29:16.476 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay 2024-04-26T21:29:16.476 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay/ks0108.ko 2024-04-26T21:29:16.476 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base 2024-04-26T21:29:16.476 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap 2024-04-26T21:29:16.476 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-i2c.ko 2024-04-26T21:29:16.476 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-mmio.ko 2024-04-26T21:29:16.476 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma 2024-04-26T21:29:16.477 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma/bcma.ko 2024-04-26T21:29:16.477 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block 2024-04-26T21:29:16.477 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe 2024-04-26T21:29:16.477 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe/aoe.ko 2024-04-26T21:29:16.477 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/brd.ko 2024-04-26T21:29:16.477 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd 2024-04-26T21:29:16.477 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd/drbd.ko 2024-04-26T21:29:16.477 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/floppy.ko 2024-04-26T21:29:16.477 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/loop.ko 2024-04-26T21:29:16.477 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx 2024-04-26T21:29:16.478 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-04-26T21:29:16.478 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/nbd.ko 2024-04-26T21:29:16.478 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk 2024-04-26T21:29:16.478 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk/null_blk.ko 2024-04-26T21:29:16.478 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/pktcdvd.ko 2024-04-26T21:29:16.478 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/rbd.ko 2024-04-26T21:29:16.478 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/virtio_blk.ko 2024-04-26T21:29:16.478 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback 2024-04-26T21:29:16.478 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-04-26T21:29:16.478 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkfront.ko 2024-04-26T21:29:16.478 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram 2024-04-26T21:29:16.478 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram/zram.ko 2024-04-26T21:29:16.478 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char 2024-04-26T21:29:16.478 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hangcheck-timer.ko 2024-04-26T21:29:16.479 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random 2024-04-26T21:29:16.479 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/amd-rng.ko 2024-04-26T21:29:16.479 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/intel-rng.ko 2024-04-26T21:29:16.479 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-04-26T21:29:16.479 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/via-rng.ko 2024-04-26T21:29:16.479 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/virtio-rng.ko 2024-04-26T21:29:16.479 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi 2024-04-26T21:29:16.479 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-04-26T21:29:16.479 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-04-26T21:29:16.479 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-04-26T21:29:16.479 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_si.ko 2024-04-26T21:29:16.479 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-04-26T21:29:16.479 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-04-26T21:29:16.479 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/lp.ko 2024-04-26T21:29:16.480 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave 2024-04-26T21:29:16.480 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave/mwave.ko 2024-04-26T21:29:16.480 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ppdev.ko 2024-04-26T21:29:16.480 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tlclk.ko 2024-04-26T21:29:16.480 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm 2024-04-26T21:29:16.480 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm.ko 2024-04-26T21:29:16.480 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_atmel.ko 2024-04-26T21:29:16.480 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_infineon.ko 2024-04-26T21:29:16.480 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_nsc.ko 2024-04-26T21:29:16.480 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis.ko 2024-04-26T21:29:16.480 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-04-26T21:29:16.480 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/uv_mmtimer.ko 2024-04-26T21:29:16.480 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/virtio_console.ko 2024-04-26T21:29:16.480 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq 2024-04-26T21:29:16.481 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-04-26T21:29:16.481 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-04-26T21:29:16.481 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/p4-clockmod.ko 2024-04-26T21:29:16.481 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-04-26T21:29:16.481 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/powernow-k8.ko 2024-04-26T21:29:16.481 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/speedstep-lib.ko 2024-04-26T21:29:16.481 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto 2024-04-26T21:29:16.481 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp 2024-04-26T21:29:16.481 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-04-26T21:29:16.481 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp.ko 2024-04-26T21:29:16.481 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel 2024-04-26T21:29:16.481 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat 2024-04-26T21:29:16.481 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common 2024-04-26T21:29:16.481 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-04-26T21:29:16.481 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-04-26T21:29:16.482 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-04-26T21:29:16.482 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-aes.ko 2024-04-26T21:29:16.482 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-sha.ko 2024-04-26T21:29:16.482 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio 2024-04-26T21:29:16.482 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-04-26T21:29:16.482 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca 2024-04-26T21:29:16.482 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca/dca.ko 2024-04-26T21:29:16.482 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq 2024-04-26T21:29:16.482 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-04-26T21:29:16.482 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma 2024-04-26T21:29:16.483 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw 2024-04-26T21:29:16.483 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw/dw_dmac.ko 2024-04-26T21:29:16.483 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat 2024-04-26T21:29:16.483 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat/ioatdma.ko 2024-04-26T21:29:16.483 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac 2024-04-26T21:29:16.483 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/amd64_edac.ko 2024-04-26T21:29:16.483 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/e752x_edac.ko 2024-04-26T21:29:16.483 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/edac_mce_amd.ko 2024-04-26T21:29:16.483 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3000_edac.ko 2024-04-26T21:29:16.483 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3200_edac.ko 2024-04-26T21:29:16.483 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5100_edac.ko 2024-04-26T21:29:16.483 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5400_edac.ko 2024-04-26T21:29:16.484 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7300_edac.ko 2024-04-26T21:29:16.484 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7core_edac.ko 2024-04-26T21:29:16.484 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i82975x_edac.ko 2024-04-26T21:29:16.484 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/ie31200_edac.ko 2024-04-26T21:29:16.484 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/sb_edac.ko 2024-04-26T21:29:16.484 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/x38_edac.ko 2024-04-26T21:29:16.484 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware 2024-04-26T21:29:16.484 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/edd.ko 2024-04-26T21:29:16.484 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/iscsi_ibft.ko 2024-04-26T21:29:16.484 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio 2024-04-26T21:29:16.484 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-ich.ko 2024-04-26T21:29:16.484 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-viperboard.ko 2024-04-26T21:29:16.484 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid 2024-04-26T21:29:16.484 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-appleir.ko 2024-04-26T21:29:16.484 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-aureal.ko 2024-04-26T21:29:16.485 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-axff.ko 2024-04-26T21:29:16.485 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-dr.ko 2024-04-26T21:29:16.485 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elecom.ko 2024-04-26T21:29:16.485 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elo.ko 2024-04-26T21:29:16.485 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-emsff.ko 2024-04-26T21:29:16.485 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gaff.ko 2024-04-26T21:29:16.485 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gt683r.ko 2024-04-26T21:29:16.485 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gyration.ko 2024-04-26T21:29:16.485 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-kbd.ko 2024-04-26T21:29:16.485 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-mouse.ko 2024-04-26T21:29:16.485 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtekff.ko 2024-04-26T21:29:16.485 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-hyperv.ko 2024-04-26T21:29:16.485 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-icade.ko 2024-04-26T21:29:16.485 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-keytouch.ko 2024-04-26T21:29:16.486 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-kye.ko 2024-04-26T21:29:16.486 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lcpower.ko 2024-04-26T21:29:16.486 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-led.ko 2024-04-26T21:29:16.486 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lenovo.ko 2024-04-26T21:29:16.486 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-dj.ko 2024-04-26T21:29:16.486 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-04-26T21:29:16.486 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-multitouch.ko 2024-04-26T21:29:16.486 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-ortek.ko 2024-04-26T21:29:16.486 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-penmount.ko 2024-04-26T21:29:16.486 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-petalynx.ko 2024-04-26T21:29:16.486 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-picolcd.ko 2024-04-26T21:29:16.486 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-pl.ko 2024-04-26T21:29:16.486 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-plantronics.ko 2024-04-26T21:29:16.487 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-primax.ko 2024-04-26T21:29:16.487 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-rmi.ko 2024-04-26T21:29:16.487 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-arvo.ko 2024-04-26T21:29:16.487 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-common.ko 2024-04-26T21:29:16.487 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-isku.ko 2024-04-26T21:29:16.487 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kone.ko 2024-04-26T21:29:16.487 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-04-26T21:29:16.487 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-konepure.ko 2024-04-26T21:29:16.487 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-04-26T21:29:16.487 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-lua.ko 2024-04-26T21:29:16.487 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-pyra.ko 2024-04-26T21:29:16.487 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-ryos.ko 2024-04-26T21:29:16.487 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-savu.ko 2024-04-26T21:29:16.487 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat.ko 2024-04-26T21:29:16.487 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-saitek.ko 2024-04-26T21:29:16.488 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-samsung.ko 2024-04-26T21:29:16.488 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sensor-hub.ko 2024-04-26T21:29:16.488 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sjoy.ko 2024-04-26T21:29:16.488 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sony.ko 2024-04-26T21:29:16.488 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-speedlink.ko 2024-04-26T21:29:16.488 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-steelseries.ko 2024-04-26T21:29:16.488 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sunplus.ko 2024-04-26T21:29:16.488 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-thrustmaster.ko 2024-04-26T21:29:16.488 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tivo.ko 2024-04-26T21:29:16.488 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tmff.ko 2024-04-26T21:29:16.488 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-topseed.ko 2024-04-26T21:29:16.488 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-twinhan.ko 2024-04-26T21:29:16.488 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-uclogic.ko 2024-04-26T21:29:16.489 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-waltop.ko 2024-04-26T21:29:16.489 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-wiimote.ko 2024-04-26T21:29:16.489 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-xinmo.ko 2024-04-26T21:29:16.489 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zpff.ko 2024-04-26T21:29:16.489 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zydacron.ko 2024-04-26T21:29:16.489 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/uhid.ko 2024-04-26T21:29:16.489 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/wacom.ko 2024-04-26T21:29:16.489 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv 2024-04-26T21:29:16.489 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_balloon.ko 2024-04-26T21:29:16.489 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_utils.ko 2024-04-26T21:29:16.489 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_vmbus.ko 2024-04-26T21:29:16.489 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon 2024-04-26T21:29:16.489 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru.ko 2024-04-26T21:29:16.489 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru3.ko 2024-04-26T21:29:16.489 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/acpi_power_meter.ko 2024-04-26T21:29:16.490 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7414.ko 2024-04-26T21:29:16.490 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7418.ko 2024-04-26T21:29:16.490 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adc128d818.ko 2024-04-26T21:29:16.490 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1025.ko 2024-04-26T21:29:16.490 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1026.ko 2024-04-26T21:29:16.490 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1029.ko 2024-04-26T21:29:16.490 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1031.ko 2024-04-26T21:29:16.490 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm9240.ko 2024-04-26T21:29:16.490 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ads7828.ko 2024-04-26T21:29:16.490 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7410.ko 2024-04-26T21:29:16.490 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7411.ko 2024-04-26T21:29:16.490 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7462.ko 2024-04-26T21:29:16.490 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7470.ko 2024-04-26T21:29:16.490 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7475.ko 2024-04-26T21:29:16.491 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7x10.ko 2024-04-26T21:29:16.491 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/amc6821.ko 2024-04-26T21:29:16.491 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/applesmc.ko 2024-04-26T21:29:16.491 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asb100.ko 2024-04-26T21:29:16.491 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asc7621.ko 2024-04-26T21:29:16.491 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asus_atk0110.ko 2024-04-26T21:29:16.491 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/atxp1.ko 2024-04-26T21:29:16.491 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/coretemp.ko 2024-04-26T21:29:16.491 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-04-26T21:29:16.491 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dme1737.ko 2024-04-26T21:29:16.491 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds1621.ko 2024-04-26T21:29:16.491 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds620.ko 2024-04-26T21:29:16.491 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc1403.ko 2024-04-26T21:29:16.492 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc6w201.ko 2024-04-26T21:29:16.492 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71805f.ko 2024-04-26T21:29:16.492 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71882fg.ko 2024-04-26T21:29:16.492 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f75375s.ko 2024-04-26T21:29:16.492 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fam15h_power.ko 2024-04-26T21:29:16.492 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fschmd.ko 2024-04-26T21:29:16.492 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g760a.ko 2024-04-26T21:29:16.492 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g762.ko 2024-04-26T21:29:16.492 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl518sm.ko 2024-04-26T21:29:16.492 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl520sm.ko 2024-04-26T21:29:16.492 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/hwmon-vid.ko 2024-04-26T21:29:16.492 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5500_temp.ko 2024-04-26T21:29:16.492 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5k_amb.ko 2024-04-26T21:29:16.492 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmaem.ko 2024-04-26T21:29:16.492 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmpex.ko 2024-04-26T21:29:16.492 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina209.ko 2024-04-26T21:29:16.493 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina2xx.ko 2024-04-26T21:29:16.493 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/it87.ko 2024-04-26T21:29:16.493 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k10temp.ko 2024-04-26T21:29:16.493 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k8temp.ko 2024-04-26T21:29:16.493 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lineage-pem.ko 2024-04-26T21:29:16.493 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm63.ko 2024-04-26T21:29:16.493 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm73.ko 2024-04-26T21:29:16.493 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm75.ko 2024-04-26T21:29:16.493 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm77.ko 2024-04-26T21:29:16.493 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm78.ko 2024-04-26T21:29:16.493 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm80.ko 2024-04-26T21:29:16.493 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm83.ko 2024-04-26T21:29:16.493 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm85.ko 2024-04-26T21:29:16.493 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm87.ko 2024-04-26T21:29:16.493 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm90.ko 2024-04-26T21:29:16.494 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm92.ko 2024-04-26T21:29:16.494 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm93.ko 2024-04-26T21:29:16.494 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95234.ko 2024-04-26T21:29:16.494 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95241.ko 2024-04-26T21:29:16.494 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95245.ko 2024-04-26T21:29:16.494 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc2945.ko 2024-04-26T21:29:16.494 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4151.ko 2024-04-26T21:29:16.494 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4215.ko 2024-04-26T21:29:16.494 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4222.ko 2024-04-26T21:29:16.494 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4245.ko 2024-04-26T21:29:16.494 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4260.ko 2024-04-26T21:29:16.494 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4261.ko 2024-04-26T21:29:16.494 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max16065.ko 2024-04-26T21:29:16.494 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1619.ko 2024-04-26T21:29:16.495 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1668.ko 2024-04-26T21:29:16.495 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max197.ko 2024-04-26T21:29:16.495 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6639.ko 2024-04-26T21:29:16.495 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6650.ko 2024-04-26T21:29:16.495 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6697.ko 2024-04-26T21:29:16.495 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/mcp3021.ko 2024-04-26T21:29:16.495 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6683.ko 2024-04-26T21:29:16.495 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775-core.ko 2024-04-26T21:29:16.495 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775.ko 2024-04-26T21:29:16.495 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct7802.ko 2024-04-26T21:29:16.495 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87360.ko 2024-04-26T21:29:16.495 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87427.ko 2024-04-26T21:29:16.495 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pcf8591.ko 2024-04-26T21:29:16.496 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus 2024-04-26T21:29:16.496 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-04-26T21:29:16.496 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-04-26T21:29:16.496 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-04-26T21:29:16.496 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max16064.ko 2024-04-26T21:29:16.496 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max34440.ko 2024-04-26T21:29:16.496 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max8688.ko 2024-04-26T21:29:16.496 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-04-26T21:29:16.496 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-04-26T21:29:16.496 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-04-26T21:29:16.496 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-04-26T21:29:16.496 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-04-26T21:29:16.496 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-04-26T21:29:16.496 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/powr1220.ko 2024-04-26T21:29:16.497 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5627.ko 2024-04-26T21:29:16.497 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5636.ko 2024-04-26T21:29:16.497 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch56xx-common.ko 2024-04-26T21:29:16.497 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht15.ko 2024-04-26T21:29:16.497 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht21.ko 2024-04-26T21:29:16.497 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/shtc1.ko 2024-04-26T21:29:16.497 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sis5595.ko 2024-04-26T21:29:16.497 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47b397.ko 2024-04-26T21:29:16.497 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m1.ko 2024-04-26T21:29:16.497 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m192.ko 2024-04-26T21:29:16.497 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/thmc50.ko 2024-04-26T21:29:16.497 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp102.ko 2024-04-26T21:29:16.497 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp103.ko 2024-04-26T21:29:16.498 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp401.ko 2024-04-26T21:29:16.498 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp421.ko 2024-04-26T21:29:16.498 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via-cputemp.ko 2024-04-26T21:29:16.498 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via686a.ko 2024-04-26T21:29:16.498 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt1211.ko 2024-04-26T21:29:16.498 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt8231.ko 2024-04-26T21:29:16.498 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627ehf.ko 2024-04-26T21:29:16.498 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627hf.ko 2024-04-26T21:29:16.498 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83781d.ko 2024-04-26T21:29:16.498 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83791d.ko 2024-04-26T21:29:16.498 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83792d.ko 2024-04-26T21:29:16.498 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83793.ko 2024-04-26T21:29:16.499 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83795.ko 2024-04-26T21:29:16.499 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l785ts.ko 2024-04-26T21:29:16.499 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l786ng.ko 2024-04-26T21:29:16.499 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c 2024-04-26T21:29:16.499 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos 2024-04-26T21:29:16.499 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-04-26T21:29:16.499 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-04-26T21:29:16.499 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses 2024-04-26T21:29:16.499 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-04-26T21:29:16.499 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-04-26T21:29:16.499 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-04-26T21:29:16.499 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-04-26T21:29:16.499 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-04-26T21:29:16.499 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-04-26T21:29:16.500 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-04-26T21:29:16.500 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-04-26T21:29:16.500 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-i801.ko 2024-04-26T21:29:16.500 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-isch.ko 2024-04-26T21:29:16.500 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-04-26T21:29:16.500 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-04-26T21:29:16.500 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-04-26T21:29:16.500 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-parport.ko 2024-04-26T21:29:16.500 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-04-26T21:29:16.500 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-04-26T21:29:16.500 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-04-26T21:29:16.500 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-04-26T21:29:16.500 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-04-26T21:29:16.500 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-04-26T21:29:16.501 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-via.ko 2024-04-26T21:29:16.501 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-04-26T21:29:16.501 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-04-26T21:29:16.501 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-dev.ko 2024-04-26T21:29:16.501 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-mux.ko 2024-04-26T21:29:16.501 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-04-26T21:29:16.501 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-smbus.ko 2024-04-26T21:29:16.501 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-stub.ko 2024-04-26T21:29:16.501 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband 2024-04-26T21:29:16.501 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core 2024-04-26T21:29:16.501 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_cm.ko 2024-04-26T21:29:16.501 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_core.ko 2024-04-26T21:29:16.501 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_umad.ko 2024-04-26T21:29:16.501 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-04-26T21:29:16.502 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/iw_cm.ko 2024-04-26T21:29:16.502 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_cm.ko 2024-04-26T21:29:16.502 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-04-26T21:29:16.502 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw 2024-04-26T21:29:16.502 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4 2024-04-26T21:29:16.502 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-04-26T21:29:16.502 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4 2024-04-26T21:29:16.502 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-04-26T21:29:16.502 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5 2024-04-26T21:29:16.502 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-04-26T21:29:16.502 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca 2024-04-26T21:29:16.502 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-04-26T21:29:16.502 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp 2024-04-26T21:29:16.503 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib 2024-04-26T21:29:16.503 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-04-26T21:29:16.503 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser 2024-04-26T21:29:16.503 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-04-26T21:29:16.503 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert 2024-04-26T21:29:16.503 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-04-26T21:29:16.503 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp 2024-04-26T21:29:16.503 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-04-26T21:29:16.503 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt 2024-04-26T21:29:16.503 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-04-26T21:29:16.503 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input 2024-04-26T21:29:16.503 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport 2024-04-26T21:29:16.503 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-04-26T21:29:16.503 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/fm801-gp.ko 2024-04-26T21:29:16.504 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/gameport.ko 2024-04-26T21:29:16.504 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/joydev.ko 2024-04-26T21:29:16.504 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard 2024-04-26T21:29:16.504 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/gpio_keys.ko 2024-04-26T21:29:16.504 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse 2024-04-26T21:29:16.504 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/appletouch.ko 2024-04-26T21:29:16.504 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/bcm5974.ko 2024-04-26T21:29:16.504 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/cyapatp.ko 2024-04-26T21:29:16.504 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/elan_i2c.ko 2024-04-26T21:29:16.504 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/sermouse.ko 2024-04-26T21:29:16.504 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-04-26T21:29:16.504 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_usb.ko 2024-04-26T21:29:16.504 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/vsxxxaa.ko 2024-04-26T21:29:16.504 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4 2024-04-26T21:29:16.504 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4/rmi_core.ko 2024-04-26T21:29:16.505 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio 2024-04-26T21:29:16.505 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/altera_ps2.ko 2024-04-26T21:29:16.505 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/arc_ps2.ko 2024-04-26T21:29:16.505 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-04-26T21:29:16.505 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/serio_raw.ko 2024-04-26T21:29:16.505 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/sparse-keymap.ko 2024-04-26T21:29:16.505 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds 2024-04-26T21:29:16.505 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-blinkm.ko 2024-04-26T21:29:16.505 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lm3530.ko 2024-04-26T21:29:16.505 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lp3944.ko 2024-04-26T21:29:16.505 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-ss4200.ko 2024-04-26T21:29:16.505 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger 2024-04-26T21:29:16.505 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-04-26T21:29:16.505 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-04-26T21:29:16.506 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-04-26T21:29:16.506 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-04-26T21:29:16.506 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-04-26T21:29:16.506 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-04-26T21:29:16.506 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-04-26T21:29:16.506 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-04-26T21:29:16.506 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-04-26T21:29:16.506 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md 2024-04-26T21:29:16.506 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache 2024-04-26T21:29:16.506 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache/bcache.ko 2024-04-26T21:29:16.506 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-bio-prison.ko 2024-04-26T21:29:16.506 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache-smq.ko 2024-04-26T21:29:16.506 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache.ko 2024-04-26T21:29:16.507 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-crypt.ko 2024-04-26T21:29:16.507 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-delay.ko 2024-04-26T21:29:16.507 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-flakey.ko 2024-04-26T21:29:16.507 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-log-userspace.ko 2024-04-26T21:29:16.507 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-multipath.ko 2024-04-26T21:29:16.507 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-queue-length.ko 2024-04-26T21:29:16.507 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-raid.ko 2024-04-26T21:29:16.507 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-round-robin.ko 2024-04-26T21:29:16.507 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-service-time.ko 2024-04-26T21:29:16.507 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-switch.ko 2024-04-26T21:29:16.507 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-thin-pool.ko 2024-04-26T21:29:16.507 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-verity.ko 2024-04-26T21:29:16.507 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data 2024-04-26T21:29:16.507 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-04-26T21:29:16.508 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid0.ko 2024-04-26T21:29:16.508 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid1.ko 2024-04-26T21:29:16.508 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid10.ko 2024-04-26T21:29:16.508 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid456.ko 2024-04-26T21:29:16.508 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message 2024-04-26T21:29:16.508 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion 2024-04-26T21:29:16.508 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptbase.ko 2024-04-26T21:29:16.508 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptctl.ko 2024-04-26T21:29:16.508 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptfc.ko 2024-04-26T21:29:16.508 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptlan.ko 2024-04-26T21:29:16.508 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptsas.ko 2024-04-26T21:29:16.508 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptscsih.ko 2024-04-26T21:29:16.508 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptspi.ko 2024-04-26T21:29:16.508 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd 2024-04-26T21:29:16.508 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_ich.ko 2024-04-26T21:29:16.509 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_sch.ko 2024-04-26T21:29:16.509 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/mfd-core.ko 2024-04-26T21:29:16.509 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/sm501.ko 2024-04-26T21:29:16.509 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/viperboard.ko 2024-04-26T21:29:16.509 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/vx855.ko 2024-04-26T21:29:16.509 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/wl1273-core.ko 2024-04-26T21:29:16.509 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc 2024-04-26T21:29:16.509 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl 2024-04-26T21:29:16.509 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-04-26T21:29:16.509 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds9802als.ko 2024-04-26T21:29:16.509 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds990x.ko 2024-04-26T21:29:16.509 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/bh1770glc.ko 2024-04-26T21:29:16.509 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710 2024-04-26T21:29:16.509 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710/cb710.ko 2024-04-26T21:29:16.509 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom 2024-04-26T21:29:16.510 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/at24.ko 2024-04-26T21:29:16.510 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-04-26T21:29:16.510 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/max6875.ko 2024-04-26T21:29:16.510 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/enclosure.ko 2024-04-26T21:29:16.510 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/hpilo.ko 2024-04-26T21:29:16.510 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29003.ko 2024-04-26T21:29:16.510 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29020.ko 2024-04-26T21:29:16.510 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d 2024-04-26T21:29:16.510 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-04-26T21:29:16.510 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-04-26T21:29:16.510 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei 2024-04-26T21:29:16.510 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-me.ko 2024-04-26T21:29:16.510 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-txe.ko 2024-04-26T21:29:16.510 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei.ko 2024-04-26T21:29:16.511 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru 2024-04-26T21:29:16.511 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru/gru.ko 2024-04-26T21:29:16.511 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp 2024-04-26T21:29:16.511 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xp.ko 2024-04-26T21:29:16.511 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpc.ko 2024-04-26T21:29:16.511 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-04-26T21:29:16.511 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_7xx1.ko 2024-04-26T21:29:16.511 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_core.ko 2024-04-26T21:29:16.511 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tsl2550.ko 2024-04-26T21:29:16.511 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_balloon.ko 2024-04-26T21:29:16.511 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci 2024-04-26T21:29:16.511 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-04-26T21:29:16.511 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net 2024-04-26T21:29:16.511 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding 2024-04-26T21:29:16.512 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding/bonding.ko 2024-04-26T21:29:16.512 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa 2024-04-26T21:29:16.512 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53 2024-04-26T21:29:16.512 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53/b53_common.ko 2024-04-26T21:29:16.512 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/bcm-sf2.ko 2024-04-26T21:29:16.512 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6060.ko 2024-04-26T21:29:16.512 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx 2024-04-26T21:29:16.512 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-04-26T21:29:16.512 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dummy.ko 2024-04-26T21:29:16.512 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/eql.ko 2024-04-26T21:29:16.512 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet 2024-04-26T21:29:16.512 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com 2024-04-26T21:29:16.512 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-04-26T21:29:16.512 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-04-26T21:29:16.513 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-04-26T21:29:16.513 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-04-26T21:29:16.513 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390 2024-04-26T21:29:16.513 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/8390.ko 2024-04-26T21:29:16.513 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-04-26T21:29:16.513 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-04-26T21:29:16.513 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-04-26T21:29:16.513 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec 2024-04-26T21:29:16.513 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-04-26T21:29:16.513 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere 2024-04-26T21:29:16.513 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere/et131x.ko 2024-04-26T21:29:16.513 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon 2024-04-26T21:29:16.513 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-04-26T21:29:16.513 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera 2024-04-26T21:29:16.514 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-04-26T21:29:16.514 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd 2024-04-26T21:29:16.514 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-04-26T21:29:16.514 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-04-26T21:29:16.514 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-04-26T21:29:16.514 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros 2024-04-26T21:29:16.514 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx 2024-04-26T21:29:16.514 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-04-26T21:29:16.514 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c 2024-04-26T21:29:16.514 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-04-26T21:29:16.514 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e 2024-04-26T21:29:16.514 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-04-26T21:29:16.514 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx 2024-04-26T21:29:16.515 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-04-26T21:29:16.515 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-04-26T21:29:16.515 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom 2024-04-26T21:29:16.515 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-04-26T21:29:16.515 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-04-26T21:29:16.515 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-04-26T21:29:16.515 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-04-26T21:29:16.515 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-04-26T21:29:16.515 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet 2024-04-26T21:29:16.515 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-04-26T21:29:16.515 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-04-26T21:29:16.515 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade 2024-04-26T21:29:16.515 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna 2024-04-26T21:29:16.516 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-04-26T21:29:16.516 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio 2024-04-26T21:29:16.516 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb 2024-04-26T21:29:16.516 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-04-26T21:29:16.516 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-04-26T21:29:16.516 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-04-26T21:29:16.516 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-04-26T21:29:16.516 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-04-26T21:29:16.516 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-04-26T21:29:16.516 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-04-26T21:29:16.516 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-04-26T21:29:16.516 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-04-26T21:29:16.516 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco 2024-04-26T21:29:16.516 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic 2024-04-26T21:29:16.516 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-04-26T21:29:16.517 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec 2024-04-26T21:29:16.517 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip 2024-04-26T21:29:16.517 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-04-26T21:29:16.517 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-04-26T21:29:16.517 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-04-26T21:29:16.517 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-04-26T21:29:16.517 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-04-26T21:29:16.517 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-04-26T21:29:16.517 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink 2024-04-26T21:29:16.517 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-04-26T21:29:16.517 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-04-26T21:29:16.517 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dnet.ko 2024-04-26T21:29:16.517 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex 2024-04-26T21:29:16.517 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet 2024-04-26T21:29:16.518 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-04-26T21:29:16.518 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ethoc.ko 2024-04-26T21:29:16.518 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/fealnx.ko 2024-04-26T21:29:16.518 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel 2024-04-26T21:29:16.518 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e100.ko 2024-04-26T21:29:16.518 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000 2024-04-26T21:29:16.518 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-04-26T21:29:16.518 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e 2024-04-26T21:29:16.518 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-04-26T21:29:16.518 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k 2024-04-26T21:29:16.518 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-04-26T21:29:16.518 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e 2024-04-26T21:29:16.518 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-04-26T21:29:16.519 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf 2024-04-26T21:29:16.519 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-04-26T21:29:16.519 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb 2024-04-26T21:29:16.519 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-04-26T21:29:16.519 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf 2024-04-26T21:29:16.519 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-04-26T21:29:16.519 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe 2024-04-26T21:29:16.519 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-04-26T21:29:16.519 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf 2024-04-26T21:29:16.519 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-04-26T21:29:16.519 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/jme.ko 2024-04-26T21:29:16.519 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell 2024-04-26T21:29:16.519 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-04-26T21:29:16.519 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/skge.ko 2024-04-26T21:29:16.519 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-04-26T21:29:16.520 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox 2024-04-26T21:29:16.520 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4 2024-04-26T21:29:16.520 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-04-26T21:29:16.520 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-04-26T21:29:16.520 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5 2024-04-26T21:29:16.520 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-04-26T21:29:16.520 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-04-26T21:29:16.520 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel 2024-04-26T21:29:16.520 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-04-26T21:29:16.520 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom 2024-04-26T21:29:16.520 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge 2024-04-26T21:29:16.520 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-04-26T21:29:16.520 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi 2024-04-26T21:29:16.520 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-04-26T21:29:16.520 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-04-26T21:29:16.521 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion 2024-04-26T21:29:16.521 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-04-26T21:29:16.521 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia 2024-04-26T21:29:16.521 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-04-26T21:29:16.521 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines 2024-04-26T21:29:16.521 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-04-26T21:29:16.521 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-04-26T21:29:16.521 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic 2024-04-26T21:29:16.521 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen 2024-04-26T21:29:16.521 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-04-26T21:29:16.521 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-04-26T21:29:16.521 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-04-26T21:29:16.521 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-04-26T21:29:16.521 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc 2024-04-26T21:29:16.521 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-04-26T21:29:16.522 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek 2024-04-26T21:29:16.522 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-04-26T21:29:16.522 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-04-26T21:29:16.522 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/atp.ko 2024-04-26T21:29:16.522 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-04-26T21:29:16.522 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker 2024-04-26T21:29:16.522 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-04-26T21:29:16.522 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc 2024-04-26T21:29:16.522 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-04-26T21:29:16.522 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan 2024-04-26T21:29:16.522 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-04-26T21:29:16.522 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis 2024-04-26T21:29:16.522 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis190.ko 2024-04-26T21:29:16.523 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis900.ko 2024-04-26T21:29:16.523 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc 2024-04-26T21:29:16.523 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-04-26T21:29:16.523 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-04-26T21:29:16.523 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-04-26T21:29:16.523 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-04-26T21:29:16.523 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro 2024-04-26T21:29:16.523 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac 2024-04-26T21:29:16.523 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-04-26T21:29:16.523 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-04-26T21:29:16.523 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun 2024-04-26T21:29:16.523 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/cassini.ko 2024-04-26T21:29:16.523 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/niu.ko 2024-04-26T21:29:16.524 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sungem.ko 2024-04-26T21:29:16.524 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-04-26T21:29:16.524 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti 2024-04-26T21:29:16.524 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-04-26T21:29:16.524 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti 2024-04-26T21:29:16.524 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti/tlan.ko 2024-04-26T21:29:16.524 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via 2024-04-26T21:29:16.524 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-04-26T21:29:16.524 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-04-26T21:29:16.524 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet 2024-04-26T21:29:16.524 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-04-26T21:29:16.524 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-04-26T21:29:16.524 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom 2024-04-26T21:29:16.525 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-04-26T21:29:16.525 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/geneve.ko 2024-04-26T21:29:16.525 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv 2024-04-26T21:29:16.525 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-04-26T21:29:16.525 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154 2024-04-26T21:29:16.525 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154/fakelb.ko 2024-04-26T21:29:16.525 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ifb.ko 2024-04-26T21:29:16.525 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan 2024-04-26T21:29:16.525 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan/ipvlan.ko 2024-04-26T21:29:16.525 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvlan.ko 2024-04-26T21:29:16.525 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvtap.ko 2024-04-26T21:29:16.525 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio 2024-04-26T21:29:16.525 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio.ko 2024-04-26T21:29:16.526 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-04-26T21:29:16.526 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-04-26T21:29:16.526 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-regmap.ko 2024-04-26T21:29:16.526 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mii.ko 2024-04-26T21:29:16.526 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/net_failover.ko 2024-04-26T21:29:16.526 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/netconsole.ko 2024-04-26T21:29:16.526 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/nlmon.ko 2024-04-26T21:29:16.526 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs 2024-04-26T21:29:16.526 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs-lynx.ko 2024-04-26T21:29:16.526 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-04-26T21:29:16.526 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy 2024-04-26T21:29:16.526 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/amd.ko 2024-04-26T21:29:16.526 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-04-26T21:29:16.526 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-04-26T21:29:16.527 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm7xxx.ko 2024-04-26T21:29:16.527 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm87xx.ko 2024-04-26T21:29:16.527 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/broadcom.ko 2024-04-26T21:29:16.527 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/cicada.ko 2024-04-26T21:29:16.527 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/davicom.ko 2024-04-26T21:29:16.527 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/dp83640.ko 2024-04-26T21:29:16.527 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/et1011c.ko 2024-04-26T21:29:16.527 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/icplus.ko 2024-04-26T21:29:16.527 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/lxt.ko 2024-04-26T21:29:16.527 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/marvell.ko 2024-04-26T21:29:16.527 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/micrel.ko 2024-04-26T21:29:16.527 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/national.ko 2024-04-26T21:29:16.527 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/phylink.ko 2024-04-26T21:29:16.527 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/qsemi.ko 2024-04-26T21:29:16.527 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/realtek.ko 2024-04-26T21:29:16.528 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/smsc.ko 2024-04-26T21:29:16.528 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/ste10Xp.ko 2024-04-26T21:29:16.528 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/vitesse.ko 2024-04-26T21:29:16.528 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp 2024-04-26T21:29:16.528 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/bsd_comp.ko 2024-04-26T21:29:16.528 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_async.ko 2024-04-26T21:29:16.528 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_deflate.ko 2024-04-26T21:29:16.528 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_generic.ko 2024-04-26T21:29:16.528 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_mppe.ko 2024-04-26T21:29:16.528 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_synctty.ko 2024-04-26T21:29:16.528 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppoe.ko 2024-04-26T21:29:16.528 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppox.ko 2024-04-26T21:29:16.528 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pptp.ko 2024-04-26T21:29:16.528 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip 2024-04-26T21:29:16.528 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slhc.ko 2024-04-26T21:29:16.529 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slip.ko 2024-04-26T21:29:16.529 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/sungem_phy.ko 2024-04-26T21:29:16.529 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tap.ko 2024-04-26T21:29:16.529 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team 2024-04-26T21:29:16.529 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team.ko 2024-04-26T21:29:16.529 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_activebackup.ko 2024-04-26T21:29:16.529 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_broadcast.ko 2024-04-26T21:29:16.529 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-04-26T21:29:16.529 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_random.ko 2024-04-26T21:29:16.529 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-04-26T21:29:16.529 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tun.ko 2024-04-26T21:29:16.529 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/veth.ko 2024-04-26T21:29:16.529 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/virtio_net.ko 2024-04-26T21:29:16.529 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3 2024-04-26T21:29:16.530 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-04-26T21:29:16.530 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan 2024-04-26T21:29:16.530 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan/vxlan.ko 2024-04-26T21:29:16.530 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback 2024-04-26T21:29:16.530 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback/xen-netback.ko 2024-04-26T21:29:16.530 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netfront.ko 2024-04-26T21:29:16.530 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb 2024-04-26T21:29:16.530 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb/ntb.ko 2024-04-26T21:29:16.530 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme 2024-04-26T21:29:16.530 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host 2024-04-26T21:29:16.530 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme-core.ko 2024-04-26T21:29:16.530 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme.ko 2024-04-26T21:29:16.530 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport 2024-04-26T21:29:16.530 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport.ko 2024-04-26T21:29:16.531 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_cs.ko 2024-04-26T21:29:16.531 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_pc.ko 2024-04-26T21:29:16.531 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_serial.ko 2024-04-26T21:29:16.531 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci 2024-04-26T21:29:16.531 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug 2024-04-26T21:29:16.531 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-04-26T21:29:16.531 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie 2024-04-26T21:29:16.531 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie/aer_inject.ko 2024-04-26T21:29:16.531 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/xen-pcifront.ko 2024-04-26T21:29:16.531 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia 2024-04-26T21:29:16.531 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/i82092.ko 2024-04-26T21:29:16.531 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/pd6729.ko 2024-04-26T21:29:16.531 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/yenta_socket.ko 2024-04-26T21:29:16.531 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl 2024-04-26T21:29:16.531 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel 2024-04-26T21:29:16.532 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-04-26T21:29:16.532 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform 2024-04-26T21:29:16.532 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86 2024-04-26T21:29:16.532 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acer-wmi.ko 2024-04-26T21:29:16.532 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acerhdf.ko 2024-04-26T21:29:16.532 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/apple-gmux.ko 2024-04-26T21:29:16.532 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-laptop.ko 2024-04-26T21:29:16.532 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-04-26T21:29:16.532 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-wmi.ko 2024-04-26T21:29:16.532 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/classmate-laptop.ko 2024-04-26T21:29:16.532 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-04-26T21:29:16.532 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-04-26T21:29:16.532 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-04-26T21:29:16.532 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-04-26T21:29:16.532 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/hdaps.ko 2024-04-26T21:29:16.533 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel 2024-04-26T21:29:16.533 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-04-26T21:29:16.533 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel_ips.ko 2024-04-26T21:29:16.533 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/msi-wmi.ko 2024-04-26T21:29:16.533 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/mxm-wmi.ko 2024-04-26T21:29:16.533 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-04-26T21:29:16.533 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-laptop.ko 2024-04-26T21:29:16.533 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-q10.ko 2024-04-26T21:29:16.533 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/topstar-laptop.ko 2024-04-26T21:29:16.533 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-04-26T21:29:16.533 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_haps.ko 2024-04-26T21:29:16.533 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi-bmof.ko 2024-04-26T21:29:16.533 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi.ko 2024-04-26T21:29:16.533 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap 2024-04-26T21:29:16.534 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_common.ko 2024-04-26T21:29:16.534 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_msr.ko 2024-04-26T21:29:16.534 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps 2024-04-26T21:29:16.534 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients 2024-04-26T21:29:16.534 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-gpio.ko 2024-04-26T21:29:16.534 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-ldisc.ko 2024-04-26T21:29:16.534 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps_parport.ko 2024-04-26T21:29:16.534 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/pps_core.ko 2024-04-26T21:29:16.534 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp 2024-04-26T21:29:16.534 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp.ko 2024-04-26T21:29:16.534 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp_kvm.ko 2024-04-26T21:29:16.534 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras 2024-04-26T21:29:16.534 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd 2024-04-26T21:29:16.535 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl 2024-04-26T21:29:16.535 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-04-26T21:29:16.535 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/fmpm.ko 2024-04-26T21:29:16.535 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc 2024-04-26T21:29:16.535 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-bq32k.ko 2024-04-26T21:29:16.535 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1286.ko 2024-04-26T21:29:16.535 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1307.ko 2024-04-26T21:29:16.535 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1374.ko 2024-04-26T21:29:16.535 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1511.ko 2024-04-26T21:29:16.535 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1553.ko 2024-04-26T21:29:16.535 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1672.ko 2024-04-26T21:29:16.535 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1742.ko 2024-04-26T21:29:16.535 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds2404.ko 2024-04-26T21:29:16.536 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds3232.ko 2024-04-26T21:29:16.536 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-em3027.ko 2024-04-26T21:29:16.536 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-fm3130.ko 2024-04-26T21:29:16.536 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl12022.ko 2024-04-26T21:29:16.536 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl1208.ko 2024-04-26T21:29:16.536 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m41t80.ko 2024-04-26T21:29:16.536 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t35.ko 2024-04-26T21:29:16.536 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t59.ko 2024-04-26T21:29:16.536 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-max6900.ko 2024-04-26T21:29:16.536 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-msm6242.ko 2024-04-26T21:29:16.536 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf2127.ko 2024-04-26T21:29:16.536 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf85063.ko 2024-04-26T21:29:16.536 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8523.ko 2024-04-26T21:29:16.536 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8563.ko 2024-04-26T21:29:16.536 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8583.ko 2024-04-26T21:29:16.536 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rp5c01.ko 2024-04-26T21:29:16.537 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rs5c372.ko 2024-04-26T21:29:16.537 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-04-26T21:29:16.537 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8025.ko 2024-04-26T21:29:16.537 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8581.ko 2024-04-26T21:29:16.537 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-04-26T21:29:16.537 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-x1205.ko 2024-04-26T21:29:16.537 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi 2024-04-26T21:29:16.537 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-9xxx.ko 2024-04-26T21:29:16.537 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-sas.ko 2024-04-26T21:29:16.537 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-xxxx.ko 2024-04-26T21:29:16.537 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/BusLogic.ko 2024-04-26T21:29:16.537 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/a100u2w.ko 2024-04-26T21:29:16.537 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid 2024-04-26T21:29:16.537 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid/aacraid.ko 2024-04-26T21:29:16.537 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/advansys.ko 2024-04-26T21:29:16.537 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx 2024-04-26T21:29:16.538 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-04-26T21:29:16.538 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-04-26T21:29:16.538 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/am53c974.ko 2024-04-26T21:29:16.538 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr 2024-04-26T21:29:16.538 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-04-26T21:29:16.538 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/atp870u.ko 2024-04-26T21:29:16.538 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi 2024-04-26T21:29:16.538 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-04-26T21:29:16.538 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa 2024-04-26T21:29:16.538 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa/bfa.ko 2024-04-26T21:29:16.538 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc 2024-04-26T21:29:16.538 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-04-26T21:29:16.538 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i 2024-04-26T21:29:16.538 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-04-26T21:29:16.538 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ch.ko 2024-04-26T21:29:16.539 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor 2024-04-26T21:29:16.539 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor/csiostor.ko 2024-04-26T21:29:16.539 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi 2024-04-26T21:29:16.539 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i 2024-04-26T21:29:16.539 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-04-26T21:29:16.539 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i 2024-04-26T21:29:16.539 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-04-26T21:29:16.539 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-04-26T21:29:16.539 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dc395x.ko 2024-04-26T21:29:16.539 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler 2024-04-26T21:29:16.539 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-04-26T21:29:16.539 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-04-26T21:29:16.539 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-04-26T21:29:16.539 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-04-26T21:29:16.539 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dmx3191d.ko 2024-04-26T21:29:16.540 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r 2024-04-26T21:29:16.540 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r/esas2r.ko 2024-04-26T21:29:16.540 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esp_scsi.ko 2024-04-26T21:29:16.540 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe 2024-04-26T21:29:16.540 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/fcoe.ko 2024-04-26T21:29:16.540 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-04-26T21:29:16.540 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic 2024-04-26T21:29:16.540 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic/fnic.ko 2024-04-26T21:29:16.540 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hpsa.ko 2024-04-26T21:29:16.540 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hptiop.ko 2024-04-26T21:29:16.540 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hv_storvsc.ko 2024-04-26T21:29:16.540 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/initio.ko 2024-04-26T21:29:16.540 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ipr.ko 2024-04-26T21:29:16.540 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ips.ko 2024-04-26T21:29:16.540 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci 2024-04-26T21:29:16.541 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci/isci.ko 2024-04-26T21:29:16.541 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-04-26T21:29:16.541 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_tcp.ko 2024-04-26T21:29:16.541 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc 2024-04-26T21:29:16.541 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc/libfc.ko 2024-04-26T21:29:16.541 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi.ko 2024-04-26T21:29:16.541 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi_tcp.ko 2024-04-26T21:29:16.541 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas 2024-04-26T21:29:16.541 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas/libsas.ko 2024-04-26T21:29:16.541 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc 2024-04-26T21:29:16.541 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc/lpfc.ko 2024-04-26T21:29:16.541 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid 2024-04-26T21:29:16.541 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid.ko 2024-04-26T21:29:16.541 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-04-26T21:29:16.541 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-04-26T21:29:16.542 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-04-26T21:29:16.542 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas 2024-04-26T21:29:16.542 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-04-26T21:29:16.542 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas 2024-04-26T21:29:16.542 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas/mvsas.ko 2024-04-26T21:29:16.542 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvumi.ko 2024-04-26T21:29:16.542 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001 2024-04-26T21:29:16.542 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-04-26T21:29:16.542 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pmcraid.ko 2024-04-26T21:29:16.542 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla1280.ko 2024-04-26T21:29:16.542 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx 2024-04-26T21:29:16.542 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-04-26T21:29:16.542 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-04-26T21:29:16.542 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx 2024-04-26T21:29:16.543 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-04-26T21:29:16.543 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/raid_class.ko 2024-04-26T21:29:16.543 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_debug.ko 2024-04-26T21:29:16.543 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_fc.ko 2024-04-26T21:29:16.543 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-04-26T21:29:16.543 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_sas.ko 2024-04-26T21:29:16.543 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_spi.ko 2024-04-26T21:29:16.543 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_srp.ko 2024-04-26T21:29:16.543 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ses.ko 2024-04-26T21:29:16.543 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/st.ko 2024-04-26T21:29:16.543 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/stex.ko 2024-04-26T21:29:16.543 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2 2024-04-26T21:29:16.543 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-04-26T21:29:16.543 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/virtio_scsi.ko 2024-04-26T21:29:16.543 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/vmw_pvscsi.ko 2024-04-26T21:29:16.543 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/wd719x.ko 2024-04-26T21:29:16.544 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb 2024-04-26T21:29:16.544 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb/ssb.ko 2024-04-26T21:29:16.544 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target 2024-04-26T21:29:16.544 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi 2024-04-26T21:29:16.544 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-04-26T21:29:16.544 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback 2024-04-26T21:29:16.544 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback/tcm_loop.ko 2024-04-26T21:29:16.544 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_file.ko 2024-04-26T21:29:16.544 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_iblock.ko 2024-04-26T21:29:16.544 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_mod.ko 2024-04-26T21:29:16.544 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_pscsi.ko 2024-04-26T21:29:16.544 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_user.ko 2024-04-26T21:29:16.544 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc 2024-04-26T21:29:16.544 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-04-26T21:29:16.544 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal 2024-04-26T21:29:16.545 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel 2024-04-26T21:29:16.545 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal 2024-04-26T21:29:16.545 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-04-26T21:29:16.545 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-04-26T21:29:16.545 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-04-26T21:29:16.545 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-04-26T21:29:16.545 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-04-26T21:29:16.545 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-04-26T21:29:16.545 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-04-26T21:29:16.545 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-04-26T21:29:16.545 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-04-26T21:29:16.545 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-04-26T21:29:16.545 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-04-26T21:29:16.545 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-04-26T21:29:16.546 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-04-26T21:29:16.546 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-04-26T21:29:16.546 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-04-26T21:29:16.546 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-04-26T21:29:16.546 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-04-26T21:29:16.546 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-04-26T21:29:16.546 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty 2024-04-26T21:29:16.546 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless 2024-04-26T21:29:16.546 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-04-26T21:29:16.546 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_gsm.ko 2024-04-26T21:29:16.546 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_hdlc.ko 2024-04-26T21:29:16.546 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/nozomi.ko 2024-04-26T21:29:16.547 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial 2024-04-26T21:29:16.547 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250 2024-04-26T21:29:16.547 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-04-26T21:29:16.547 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/arc_uart.ko 2024-04-26T21:29:16.547 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm 2024-04-26T21:29:16.547 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm/jsm.ko 2024-04-26T21:29:16.547 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs 2024-04-26T21:29:16.547 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core 2024-04-26T21:29:16.547 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core/ufshcd-core.ko 2024-04-26T21:29:16.547 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host 2024-04-26T21:29:16.547 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-04-26T21:29:16.547 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio 2024-04-26T21:29:16.547 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio.ko 2024-04-26T21:29:16.547 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_aec.ko 2024-04-26T21:29:16.548 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_cif.ko 2024-04-26T21:29:16.548 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_pci_generic.ko 2024-04-26T21:29:16.548 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_sercos3.ko 2024-04-26T21:29:16.548 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb 2024-04-26T21:29:16.548 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class 2024-04-26T21:29:16.548 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-acm.ko 2024-04-26T21:29:16.548 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-wdm.ko 2024-04-26T21:29:16.548 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usblp.ko 2024-04-26T21:29:16.548 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usbtmc.ko 2024-04-26T21:29:16.548 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host 2024-04-26T21:29:16.548 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/sl811-hcd.ko 2024-04-26T21:29:16.548 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image 2024-04-26T21:29:16.548 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/mdc800.ko 2024-04-26T21:29:16.548 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/microtek.ko 2024-04-26T21:29:16.548 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc 2024-04-26T21:29:16.549 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/adutux.ko 2024-04-26T21:29:16.549 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/appledisplay.ko 2024-04-26T21:29:16.549 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi26.ko 2024-04-26T21:29:16.549 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi62.ko 2024-04-26T21:29:16.549 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ezusb.ko 2024-04-26T21:29:16.549 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/idmouse.ko 2024-04-26T21:29:16.549 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/iowarrior.ko 2024-04-26T21:29:16.549 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/isight_firmware.ko 2024-04-26T21:29:16.549 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ldusb.ko 2024-04-26T21:29:16.549 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/legousbtower.ko 2024-04-26T21:29:16.549 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga 2024-04-26T21:29:16.549 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-04-26T21:29:16.549 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/trancevibrator.ko 2024-04-26T21:29:16.549 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usb3503.ko 2024-04-26T21:29:16.549 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usblcd.ko 2024-04-26T21:29:16.549 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usbsevseg.ko 2024-04-26T21:29:16.550 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/uss720.ko 2024-04-26T21:29:16.550 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/yurex.ko 2024-04-26T21:29:16.550 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy 2024-04-26T21:29:16.550 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy/phy-generic.ko 2024-04-26T21:29:16.550 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial 2024-04-26T21:29:16.550 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/aircable.ko 2024-04-26T21:29:16.550 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ark3116.ko 2024-04-26T21:29:16.550 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/belkin_sa.ko 2024-04-26T21:29:16.550 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ch341.ko 2024-04-26T21:29:16.550 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cp210x.ko 2024-04-26T21:29:16.550 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cyberjack.ko 2024-04-26T21:29:16.550 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cypress_m8.ko 2024-04-26T21:29:16.550 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/digi_acceleport.ko 2024-04-26T21:29:16.551 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/empeg.ko 2024-04-26T21:29:16.551 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ftdi_sio.ko 2024-04-26T21:29:16.551 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/garmin_gps.ko 2024-04-26T21:29:16.551 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_edgeport.ko 2024-04-26T21:29:16.551 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_ti.ko 2024-04-26T21:29:16.551 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipaq.ko 2024-04-26T21:29:16.551 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipw.ko 2024-04-26T21:29:16.551 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ir-usb.ko 2024-04-26T21:29:16.551 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-04-26T21:29:16.551 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan.ko 2024-04-26T21:29:16.551 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan_pda.ko 2024-04-26T21:29:16.551 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kl5kusb105.ko 2024-04-26T21:29:16.551 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kobil_sct.ko 2024-04-26T21:29:16.551 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mct_u232.ko 2024-04-26T21:29:16.551 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7720.ko 2024-04-26T21:29:16.552 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7840.ko 2024-04-26T21:29:16.552 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/navman.ko 2024-04-26T21:29:16.552 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/omninet.ko 2024-04-26T21:29:16.552 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/opticon.ko 2024-04-26T21:29:16.552 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/option.ko 2024-04-26T21:29:16.552 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/oti6858.ko 2024-04-26T21:29:16.552 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/pl2303.ko 2024-04-26T21:29:16.552 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcaux.ko 2024-04-26T21:29:16.552 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcserial.ko 2024-04-26T21:29:16.552 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/quatech2.ko 2024-04-26T21:29:16.552 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/safe_serial.ko 2024-04-26T21:29:16.552 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/sierra.ko 2024-04-26T21:29:16.552 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/spcp8x5.ko 2024-04-26T21:29:16.552 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ssu100.ko 2024-04-26T21:29:16.552 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/symbolserial.ko 2024-04-26T21:29:16.552 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-04-26T21:29:16.553 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-04-26T21:29:16.553 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_debug.ko 2024-04-26T21:29:16.553 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_wwan.ko 2024-04-26T21:29:16.553 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/visor.ko 2024-04-26T21:29:16.553 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/whiteheat.ko 2024-04-26T21:29:16.553 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/xsens_mt.ko 2024-04-26T21:29:16.553 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage 2024-04-26T21:29:16.553 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/uas.ko 2024-04-26T21:29:16.553 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-alauda.ko 2024-04-26T21:29:16.553 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-cypress.ko 2024-04-26T21:29:16.553 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-datafab.ko 2024-04-26T21:29:16.553 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-04-26T21:29:16.553 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-freecom.ko 2024-04-26T21:29:16.553 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-isd200.ko 2024-04-26T21:29:16.554 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-04-26T21:29:16.554 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-karma.ko 2024-04-26T21:29:16.554 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-onetouch.ko 2024-04-26T21:29:16.554 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-realtek.ko 2024-04-26T21:29:16.554 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr09.ko 2024-04-26T21:29:16.554 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr55.ko 2024-04-26T21:29:16.554 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-usbat.ko 2024-04-26T21:29:16.554 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/usb-storage.ko 2024-04-26T21:29:16.554 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip 2024-04-26T21:29:16.554 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-core.ko 2024-04-26T21:29:16.554 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-host.ko 2024-04-26T21:29:16.554 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-04-26T21:29:16.554 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio 2024-04-26T21:29:16.554 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci 2024-04-26T21:29:16.555 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-04-26T21:29:16.555 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci.ko 2024-04-26T21:29:16.555 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio.ko 2024-04-26T21:29:16.555 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-04-26T21:29:16.555 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost 2024-04-26T21:29:16.555 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost.ko 2024-04-26T21:29:16.555 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_iotlb.ko 2024-04-26T21:29:16.555 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_net.ko 2024-04-26T21:29:16.555 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_scsi.ko 2024-04-26T21:29:16.555 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video 2024-04-26T21:29:16.555 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight 2024-04-26T21:29:16.555 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/apple_bl.ko 2024-04-26T21:29:16.555 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/lcd.ko 2024-04-26T21:29:16.555 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/platform_lcd.ko 2024-04-26T21:29:16.556 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio 2024-04-26T21:29:16.556 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio.ko 2024-04-26T21:29:16.556 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_balloon.ko 2024-04-26T21:29:16.556 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_mmio.ko 2024-04-26T21:29:16.556 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci.ko 2024-04-26T21:29:16.556 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-04-26T21:29:16.556 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-04-26T21:29:16.556 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_ring.ko 2024-04-26T21:29:16.556 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1 2024-04-26T21:29:16.556 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters 2024-04-26T21:29:16.556 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2482.ko 2024-04-26T21:29:16.556 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2490.ko 2024-04-26T21:29:16.556 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves 2024-04-26T21:29:16.556 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-04-26T21:29:16.556 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-04-26T21:29:16.557 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-04-26T21:29:16.557 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-04-26T21:29:16.557 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-04-26T21:29:16.557 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-04-26T21:29:16.557 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-04-26T21:29:16.557 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-04-26T21:29:16.557 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-04-26T21:29:16.557 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_smem.ko 2024-04-26T21:29:16.557 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_therm.ko 2024-04-26T21:29:16.557 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/wire.ko 2024-04-26T21:29:16.557 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog 2024-04-26T21:29:16.557 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim1535_wdt.ko 2024-04-26T21:29:16.557 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim7101_wdt.ko 2024-04-26T21:29:16.557 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/f71808e_wdt.ko 2024-04-26T21:29:16.557 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/hpwdt.ko 2024-04-26T21:29:16.557 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/i6300esb.ko 2024-04-26T21:29:16.558 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-04-26T21:29:16.558 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_wdt.ko 2024-04-26T21:29:16.558 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ib700wdt.ko 2024-04-26T21:29:16.558 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ibmasr.ko 2024-04-26T21:29:16.558 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-04-26T21:29:16.558 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it8712f_wdt.ko 2024-04-26T21:29:16.558 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it87_wdt.ko 2024-04-26T21:29:16.558 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/machzwd.ko 2024-04-26T21:29:16.558 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/nv_tco.ko 2024-04-26T21:29:16.558 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_pci.ko 2024-04-26T21:29:16.558 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_usb.ko 2024-04-26T21:29:16.558 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-04-26T21:29:16.558 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sch311x_wdt.ko 2024-04-26T21:29:16.559 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/softdog.ko 2024-04-26T21:29:16.559 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sp5100_tco.ko 2024-04-26T21:29:16.559 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/via_wdt.ko 2024-04-26T21:29:16.559 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-04-26T21:29:16.559 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83877f_wdt.ko 2024-04-26T21:29:16.559 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83977f_wdt.ko 2024-04-26T21:29:16.559 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/wdt_pci.ko 2024-04-26T21:29:16.559 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/xen_wdt.ko 2024-04-26T21:29:16.559 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen 2024-04-26T21:29:16.559 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-acpi-processor.ko 2024-04-26T21:29:16.559 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-evtchn.ko 2024-04-26T21:29:16.559 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntalloc.ko 2024-04-26T21:29:16.559 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntdev.ko 2024-04-26T21:29:16.559 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback 2024-04-26T21:29:16.559 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-04-26T21:29:16.560 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-privcmd.ko 2024-04-26T21:29:16.560 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs 2024-04-26T21:29:16.560 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs/xenfs.ko 2024-04-26T21:29:16.560 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs 2024-04-26T21:29:16.560 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p 2024-04-26T21:29:16.560 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p/9p.ko 2024-04-26T21:29:16.560 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs 2024-04-26T21:29:16.560 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs/affs.ko 2024-04-26T21:29:16.560 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs 2024-04-26T21:29:16.560 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs/befs.ko 2024-04-26T21:29:16.560 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/binfmt_misc.ko 2024-04-26T21:29:16.560 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs 2024-04-26T21:29:16.560 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs/btrfs.ko 2024-04-26T21:29:16.560 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles 2024-04-26T21:29:16.560 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles/cachefiles.ko 2024-04-26T21:29:16.561 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph 2024-04-26T21:29:16.561 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph/ceph.ko 2024-04-26T21:29:16.561 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda 2024-04-26T21:29:16.561 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda/coda.ko 2024-04-26T21:29:16.561 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs 2024-04-26T21:29:16.561 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs/cramfs.ko 2024-04-26T21:29:16.561 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm 2024-04-26T21:29:16.561 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm/dlm.ko 2024-04-26T21:29:16.561 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs 2024-04-26T21:29:16.561 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs/ecryptfs.ko 2024-04-26T21:29:16.561 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs 2024-04-26T21:29:16.561 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs/f2fs.ko 2024-04-26T21:29:16.561 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat 2024-04-26T21:29:16.561 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/fat.ko 2024-04-26T21:29:16.561 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/msdos.ko 2024-04-26T21:29:16.562 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/vfat.ko 2024-04-26T21:29:16.562 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse 2024-04-26T21:29:16.562 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/cuse.ko 2024-04-26T21:29:16.562 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/fuse.ko 2024-04-26T21:29:16.562 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2 2024-04-26T21:29:16.562 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2/gfs2.ko 2024-04-26T21:29:16.562 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs 2024-04-26T21:29:16.562 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs/hfs.ko 2024-04-26T21:29:16.562 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus 2024-04-26T21:29:16.562 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus/hfsplus.ko 2024-04-26T21:29:16.562 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs 2024-04-26T21:29:16.562 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs/isofs.ko 2024-04-26T21:29:16.563 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs 2024-04-26T21:29:16.563 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs/jfs.ko 2024-04-26T21:29:16.563 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd 2024-04-26T21:29:16.563 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd/lockd.ko 2024-04-26T21:29:16.563 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix 2024-04-26T21:29:16.563 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix/minix.ko 2024-04-26T21:29:16.563 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs 2024-04-26T21:29:16.563 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs/netfs.ko 2024-04-26T21:29:16.563 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs 2024-04-26T21:29:16.563 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout 2024-04-26T21:29:16.563 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-04-26T21:29:16.563 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout 2024-04-26T21:29:16.563 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-04-26T21:29:16.563 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout 2024-04-26T21:29:16.564 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-04-26T21:29:16.564 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfs.ko 2024-04-26T21:29:16.564 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv3.ko 2024-04-26T21:29:16.564 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv4.ko 2024-04-26T21:29:16.564 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common 2024-04-26T21:29:16.564 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/grace.ko 2024-04-26T21:29:16.564 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/nfs_acl.ko 2024-04-26T21:29:16.564 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd 2024-04-26T21:29:16.564 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd/nfsd.ko 2024-04-26T21:29:16.564 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2 2024-04-26T21:29:16.564 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2/nilfs2.ko 2024-04-26T21:29:16.564 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls 2024-04-26T21:29:16.564 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-celtic.ko 2024-04-26T21:29:16.564 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-centeuro.ko 2024-04-26T21:29:16.565 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-croatian.ko 2024-04-26T21:29:16.565 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-cyrillic.ko 2024-04-26T21:29:16.565 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-gaelic.ko 2024-04-26T21:29:16.565 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-greek.ko 2024-04-26T21:29:16.565 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-iceland.ko 2024-04-26T21:29:16.565 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-inuit.ko 2024-04-26T21:29:16.565 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-roman.ko 2024-04-26T21:29:16.565 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-romanian.ko 2024-04-26T21:29:16.565 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-turkish.ko 2024-04-26T21:29:16.565 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1250.ko 2024-04-26T21:29:16.565 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1251.ko 2024-04-26T21:29:16.565 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1255.ko 2024-04-26T21:29:16.565 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp737.ko 2024-04-26T21:29:16.565 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp775.ko 2024-04-26T21:29:16.565 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp850.ko 2024-04-26T21:29:16.566 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp852.ko 2024-04-26T21:29:16.566 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp855.ko 2024-04-26T21:29:16.566 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp857.ko 2024-04-26T21:29:16.566 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp860.ko 2024-04-26T21:29:16.566 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp861.ko 2024-04-26T21:29:16.566 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp862.ko 2024-04-26T21:29:16.566 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp863.ko 2024-04-26T21:29:16.566 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp864.ko 2024-04-26T21:29:16.566 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp865.ko 2024-04-26T21:29:16.566 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp866.ko 2024-04-26T21:29:16.566 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp869.ko 2024-04-26T21:29:16.566 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp874.ko 2024-04-26T21:29:16.566 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp932.ko 2024-04-26T21:29:16.566 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp936.ko 2024-04-26T21:29:16.567 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp949.ko 2024-04-26T21:29:16.567 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp950.ko 2024-04-26T21:29:16.567 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_euc-jp.ko 2024-04-26T21:29:16.567 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-1.ko 2024-04-26T21:29:16.567 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-13.ko 2024-04-26T21:29:16.567 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-14.ko 2024-04-26T21:29:16.567 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-15.ko 2024-04-26T21:29:16.567 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-2.ko 2024-04-26T21:29:16.567 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-3.ko 2024-04-26T21:29:16.567 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-4.ko 2024-04-26T21:29:16.567 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-5.ko 2024-04-26T21:29:16.567 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-6.ko 2024-04-26T21:29:16.567 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-7.ko 2024-04-26T21:29:16.567 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-9.ko 2024-04-26T21:29:16.567 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-r.ko 2024-04-26T21:29:16.568 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-ru.ko 2024-04-26T21:29:16.568 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-u.ko 2024-04-26T21:29:16.568 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_ucs2_utils.ko 2024-04-26T21:29:16.568 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_utf8.ko 2024-04-26T21:29:16.568 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2 2024-04-26T21:29:16.568 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster 2024-04-26T21:29:16.568 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-04-26T21:29:16.568 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm 2024-04-26T21:29:16.568 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-04-26T21:29:16.568 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs 2024-04-26T21:29:16.568 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-04-26T21:29:16.568 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2.ko 2024-04-26T21:29:16.568 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-04-26T21:29:16.569 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-04-26T21:29:16.569 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-04-26T21:29:16.569 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs 2024-04-26T21:29:16.569 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs/overlay.ko 2024-04-26T21:29:16.569 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore 2024-04-26T21:29:16.569 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore/ramoops.ko 2024-04-26T21:29:16.569 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs 2024-04-26T21:29:16.569 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs/reiserfs.ko 2024-04-26T21:29:16.569 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs 2024-04-26T21:29:16.569 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs/romfs.ko 2024-04-26T21:29:16.569 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb 2024-04-26T21:29:16.569 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client 2024-04-26T21:29:16.569 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client/cifs.ko 2024-04-26T21:29:16.569 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common 2024-04-26T21:29:16.570 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_arc4.ko 2024-04-26T21:29:16.570 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_md4.ko 2024-04-26T21:29:16.570 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs 2024-04-26T21:29:16.570 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs/squashfs.ko 2024-04-26T21:29:16.570 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv 2024-04-26T21:29:16.570 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv/sysv.ko 2024-04-26T21:29:16.570 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf 2024-04-26T21:29:16.570 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf/udf.ko 2024-04-26T21:29:16.570 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs 2024-04-26T21:29:16.570 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs/ufs.ko 2024-04-26T21:29:16.570 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs 2024-04-26T21:29:16.570 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs/xfs.ko 2024-04-26T21:29:16.570 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel 2024-04-26T21:29:16.571 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu 2024-04-26T21:29:16.571 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu/rcutorture.ko 2024-04-26T21:29:16.571 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/torture.ko 2024-04-26T21:29:16.571 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace 2024-04-26T21:29:16.571 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-04-26T21:29:16.571 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib 2024-04-26T21:29:16.571 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/asn1_encoder.ko 2024-04-26T21:29:16.571 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc-itu-t.ko 2024-04-26T21:29:16.571 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc8.ko 2024-04-26T21:29:16.571 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto 2024-04-26T21:29:16.571 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libarc4.ko 2024-04-26T21:29:16.571 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libdes.ko 2024-04-26T21:29:16.571 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/libcrc32c.ko 2024-04-26T21:29:16.571 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lru_cache.ko 2024-04-26T21:29:16.572 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4 2024-04-26T21:29:16.572 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4_compress.ko 2024-04-26T21:29:16.572 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4hc_compress.ko 2024-04-26T21:29:16.572 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math 2024-04-26T21:29:16.572 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math/cordic.ko 2024-04-26T21:29:16.572 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6 2024-04-26T21:29:16.572 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6/raid6_pq.ko 2024-04-26T21:29:16.572 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon 2024-04-26T21:29:16.572 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon/reed_solomon.ko 2024-04-26T21:29:16.572 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_bm.ko 2024-04-26T21:29:16.572 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_fsm.ko 2024-04-26T21:29:16.572 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_kmp.ko 2024-04-26T21:29:16.572 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd 2024-04-26T21:29:16.572 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd/zstd_compress.ko 2024-04-26T21:29:16.573 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm 2024-04-26T21:29:16.573 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm/hwpoison-inject.ko 2024-04-26T21:29:16.573 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net 2024-04-26T21:29:16.573 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan 2024-04-26T21:29:16.573 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/6lowpan.ko 2024-04-26T21:29:16.573 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_dest.ko 2024-04-26T21:29:16.573 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_fragment.ko 2024-04-26T21:29:16.573 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_hop.ko 2024-04-26T21:29:16.573 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_ipv6.ko 2024-04-26T21:29:16.573 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_mobility.ko 2024-04-26T21:29:16.573 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_routing.ko 2024-04-26T21:29:16.573 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_udp.ko 2024-04-26T21:29:16.573 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802 2024-04-26T21:29:16.573 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/garp.ko 2024-04-26T21:29:16.574 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/mrp.ko 2024-04-26T21:29:16.574 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/p8022.ko 2024-04-26T21:29:16.574 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/psnap.ko 2024-04-26T21:29:16.574 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/stp.ko 2024-04-26T21:29:16.574 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q 2024-04-26T21:29:16.574 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q/8021q.ko 2024-04-26T21:29:16.574 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p 2024-04-26T21:29:16.574 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet.ko 2024-04-26T21:29:16.574 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_fd.ko 2024-04-26T21:29:16.574 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_rdma.ko 2024-04-26T21:29:16.574 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_virtio.ko 2024-04-26T21:29:16.574 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv 2024-04-26T21:29:16.574 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv/batman-adv.ko 2024-04-26T21:29:16.574 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge 2024-04-26T21:29:16.575 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/br_netfilter.ko 2024-04-26T21:29:16.575 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/bridge.ko 2024-04-26T21:29:16.575 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter 2024-04-26T21:29:16.575 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-04-26T21:29:16.575 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_among.ko 2024-04-26T21:29:16.575 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arp.ko 2024-04-26T21:29:16.575 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-04-26T21:29:16.575 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-04-26T21:29:16.575 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip.ko 2024-04-26T21:29:16.575 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-04-26T21:29:16.575 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_limit.ko 2024-04-26T21:29:16.575 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_log.ko 2024-04-26T21:29:16.575 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark.ko 2024-04-26T21:29:16.575 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-04-26T21:29:16.575 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-04-26T21:29:16.576 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-04-26T21:29:16.576 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-04-26T21:29:16.576 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_snat.ko 2024-04-26T21:29:16.576 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_stp.ko 2024-04-26T21:29:16.576 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-04-26T21:29:16.576 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-04-26T21:29:16.576 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-04-26T21:29:16.576 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-04-26T21:29:16.576 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtables.ko 2024-04-26T21:29:16.576 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-04-26T21:29:16.576 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-04-26T21:29:16.576 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-04-26T21:29:16.576 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph 2024-04-26T21:29:16.576 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph/libceph.ko 2024-04-26T21:29:16.576 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core 2024-04-26T21:29:16.576 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/failover.ko 2024-04-26T21:29:16.577 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/pktgen.ko 2024-04-26T21:29:16.577 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp 2024-04-26T21:29:16.577 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp.ko 2024-04-26T21:29:16.577 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_diag.ko 2024-04-26T21:29:16.577 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv4.ko 2024-04-26T21:29:16.577 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv6.ko 2024-04-26T21:29:16.577 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver 2024-04-26T21:29:16.577 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver/dns_resolver.ko 2024-04-26T21:29:16.577 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa 2024-04-26T21:29:16.577 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/dsa_core.ko 2024-04-26T21:29:16.577 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_brcm.ko 2024-04-26T21:29:16.577 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_dsa.ko 2024-04-26T21:29:16.577 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_none.ko 2024-04-26T21:29:16.577 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_trailer.ko 2024-04-26T21:29:16.577 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154 2024-04-26T21:29:16.577 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan 2024-04-26T21:29:16.578 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-04-26T21:29:16.578 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154.ko 2024-04-26T21:29:16.578 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154_socket.ko 2024-04-26T21:29:16.578 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4 2024-04-26T21:29:16.578 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ah4.ko 2024-04-26T21:29:16.578 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/esp4.ko 2024-04-26T21:29:16.578 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/fou.ko 2024-04-26T21:29:16.578 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/gre.ko 2024-04-26T21:29:16.578 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/inet_diag.ko 2024-04-26T21:29:16.578 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_gre.ko 2024-04-26T21:29:16.578 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_tunnel.ko 2024-04-26T21:29:16.578 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_vti.ko 2024-04-26T21:29:16.578 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipcomp.ko 2024-04-26T21:29:16.578 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipip.ko 2024-04-26T21:29:16.579 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter 2024-04-26T21:29:16.579 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arp_tables.ko 2024-04-26T21:29:16.579 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-04-26T21:29:16.579 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-04-26T21:29:16.579 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-04-26T21:29:16.579 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-04-26T21:29:16.579 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-04-26T21:29:16.579 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-04-26T21:29:16.579 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-04-26T21:29:16.579 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-04-26T21:29:16.579 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-04-26T21:29:16.579 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_security.ko 2024-04-26T21:29:16.579 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-04-26T21:29:16.579 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-04-26T21:29:16.580 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-04-26T21:29:16.580 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-04-26T21:29:16.580 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-04-26T21:29:16.580 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-04-26T21:29:16.580 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-04-26T21:29:16.580 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-04-26T21:29:16.580 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-04-26T21:29:16.580 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-04-26T21:29:16.580 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_bic.ko 2024-04-26T21:29:16.580 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_dctcp.ko 2024-04-26T21:29:16.580 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_diag.ko 2024-04-26T21:29:16.580 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_highspeed.ko 2024-04-26T21:29:16.580 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_htcp.ko 2024-04-26T21:29:16.581 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_hybla.ko 2024-04-26T21:29:16.581 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_illinois.ko 2024-04-26T21:29:16.581 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_lp.ko 2024-04-26T21:29:16.581 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_scalable.ko 2024-04-26T21:29:16.581 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_vegas.ko 2024-04-26T21:29:16.581 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_veno.ko 2024-04-26T21:29:16.581 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_westwood.ko 2024-04-26T21:29:16.581 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_yeah.ko 2024-04-26T21:29:16.581 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tunnel4.ko 2024-04-26T21:29:16.581 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_diag.ko 2024-04-26T21:29:16.581 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_tunnel.ko 2024-04-26T21:29:16.581 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/xfrm4_tunnel.ko 2024-04-26T21:29:16.581 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6 2024-04-26T21:29:16.581 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ah6.ko 2024-04-26T21:29:16.582 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/esp6.ko 2024-04-26T21:29:16.582 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/fou6.ko 2024-04-26T21:29:16.582 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_tunnel.ko 2024-04-26T21:29:16.582 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-04-26T21:29:16.582 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_vti.ko 2024-04-26T21:29:16.582 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ipcomp6.ko 2024-04-26T21:29:16.582 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter 2024-04-26T21:29:16.582 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-04-26T21:29:16.582 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-04-26T21:29:16.582 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-04-26T21:29:16.582 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-04-26T21:29:16.582 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-04-26T21:29:16.582 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-04-26T21:29:16.582 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-04-26T21:29:16.583 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-04-26T21:29:16.583 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-04-26T21:29:16.583 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-04-26T21:29:16.583 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-04-26T21:29:16.583 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-04-26T21:29:16.583 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-04-26T21:29:16.583 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-04-26T21:29:16.583 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-04-26T21:29:16.583 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-04-26T21:29:16.583 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-04-26T21:29:16.583 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-04-26T21:29:16.583 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-04-26T21:29:16.583 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-04-26T21:29:16.583 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-04-26T21:29:16.584 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-04-26T21:29:16.584 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-04-26T21:29:16.584 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-04-26T21:29:16.584 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/sit.ko 2024-04-26T21:29:16.584 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/tunnel6.ko 2024-04-26T21:29:16.584 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/xfrm6_tunnel.ko 2024-04-26T21:29:16.584 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key 2024-04-26T21:29:16.584 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key/af_key.ko 2024-04-26T21:29:16.584 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp 2024-04-26T21:29:16.584 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_core.ko 2024-04-26T21:29:16.584 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_debugfs.ko 2024-04-26T21:29:16.584 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_eth.ko 2024-04-26T21:29:16.585 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip.ko 2024-04-26T21:29:16.585 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip6.ko 2024-04-26T21:29:16.585 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_netlink.ko 2024-04-26T21:29:16.585 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ppp.ko 2024-04-26T21:29:16.585 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc 2024-04-26T21:29:16.585 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc/llc.ko 2024-04-26T21:29:16.585 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154 2024-04-26T21:29:16.585 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154/mac802154.ko 2024-04-26T21:29:16.585 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls 2024-04-26T21:29:16.585 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls/mpls_gso.ko 2024-04-26T21:29:16.585 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter 2024-04-26T21:29:16.585 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset 2024-04-26T21:29:16.585 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set.ko 2024-04-26T21:29:16.585 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-04-26T21:29:16.586 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-04-26T21:29:16.586 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-04-26T21:29:16.586 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-04-26T21:29:16.586 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-04-26T21:29:16.586 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-04-26T21:29:16.586 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-04-26T21:29:16.586 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-04-26T21:29:16.586 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-04-26T21:29:16.586 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-04-26T21:29:16.586 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-04-26T21:29:16.586 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-04-26T21:29:16.586 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-04-26T21:29:16.586 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-04-26T21:29:16.586 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-04-26T21:29:16.587 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs 2024-04-26T21:29:16.587 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs.ko 2024-04-26T21:29:16.587 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-04-26T21:29:16.587 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-04-26T21:29:16.587 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-04-26T21:29:16.587 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-04-26T21:29:16.587 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-04-26T21:29:16.587 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-04-26T21:29:16.587 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-04-26T21:29:16.587 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-04-26T21:29:16.587 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-04-26T21:29:16.587 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-04-26T21:29:16.587 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-04-26T21:29:16.587 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-04-26T21:29:16.587 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-04-26T21:29:16.588 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conncount.ko 2024-04-26T21:29:16.588 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack.ko 2024-04-26T21:29:16.588 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-04-26T21:29:16.588 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-04-26T21:29:16.588 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-04-26T21:29:16.588 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_h323.ko 2024-04-26T21:29:16.588 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_irc.ko 2024-04-26T21:29:16.588 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-04-26T21:29:16.588 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-04-26T21:29:16.588 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-04-26T21:29:16.588 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sane.ko 2024-04-26T21:29:16.588 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sip.ko 2024-04-26T21:29:16.588 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-04-26T21:29:16.588 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-04-26T21:29:16.588 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_dup_netdev.ko 2024-04-26T21:29:16.589 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table.ko 2024-04-26T21:29:16.589 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table_inet.ko 2024-04-26T21:29:16.589 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_log_syslog.ko 2024-04-26T21:29:16.589 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat.ko 2024-04-26T21:29:16.589 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_amanda.ko 2024-04-26T21:29:16.589 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_ftp.ko 2024-04-26T21:29:16.589 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_irc.ko 2024-04-26T21:29:16.589 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_sip.ko 2024-04-26T21:29:16.589 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_tftp.ko 2024-04-26T21:29:16.589 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_synproxy_core.ko 2024-04-26T21:29:16.589 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_tables.ko 2024-04-26T21:29:16.589 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink.ko 2024-04-26T21:29:16.589 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_acct.ko 2024-04-26T21:29:16.589 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_log.ko 2024-04-26T21:29:16.590 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_osf.ko 2024-04-26T21:29:16.590 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_queue.ko 2024-04-26T21:29:16.590 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_chain_nat.ko 2024-04-26T21:29:16.590 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_compat.ko 2024-04-26T21:29:16.590 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_ct.ko 2024-04-26T21:29:16.590 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_dup_netdev.ko 2024-04-26T21:29:16.590 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fib.ko 2024-04-26T21:29:16.590 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fwd_netdev.ko 2024-04-26T21:29:16.590 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_hash.ko 2024-04-26T21:29:16.590 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_limit.ko 2024-04-26T21:29:16.590 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_log.ko 2024-04-26T21:29:16.590 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_masq.ko 2024-04-26T21:29:16.590 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_nat.ko 2024-04-26T21:29:16.590 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_numgen.ko 2024-04-26T21:29:16.591 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_queue.ko 2024-04-26T21:29:16.591 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_quota.ko 2024-04-26T21:29:16.591 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_redir.ko 2024-04-26T21:29:16.591 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject.ko 2024-04-26T21:29:16.591 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject_inet.ko 2024-04-26T21:29:16.591 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_socket.ko 2024-04-26T21:29:16.591 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_tunnel.ko 2024-04-26T21:29:16.591 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_AUDIT.ko 2024-04-26T21:29:16.591 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CHECKSUM.ko 2024-04-26T21:29:16.591 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CLASSIFY.ko 2024-04-26T21:29:16.591 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-04-26T21:29:16.591 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CT.ko 2024-04-26T21:29:16.591 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_DSCP.ko 2024-04-26T21:29:16.591 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HL.ko 2024-04-26T21:29:16.591 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HMARK.ko 2024-04-26T21:29:16.592 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_IDLETIMER.ko 2024-04-26T21:29:16.592 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LED.ko 2024-04-26T21:29:16.592 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LOG.ko 2024-04-26T21:29:16.592 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_MASQUERADE.ko 2024-04-26T21:29:16.592 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NETMAP.ko 2024-04-26T21:29:16.592 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFLOG.ko 2024-04-26T21:29:16.592 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFQUEUE.ko 2024-04-26T21:29:16.592 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_RATEEST.ko 2024-04-26T21:29:16.592 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_REDIRECT.ko 2024-04-26T21:29:16.592 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_SECMARK.ko 2024-04-26T21:29:16.592 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPMSS.ko 2024-04-26T21:29:16.592 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-04-26T21:29:16.592 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TEE.ko 2024-04-26T21:29:16.592 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TPROXY.ko 2024-04-26T21:29:16.593 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TRACE.ko 2024-04-26T21:29:16.593 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_addrtype.ko 2024-04-26T21:29:16.593 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_bpf.ko 2024-04-26T21:29:16.593 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cgroup.ko 2024-04-26T21:29:16.593 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cluster.ko 2024-04-26T21:29:16.593 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_comment.ko 2024-04-26T21:29:16.593 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connbytes.ko 2024-04-26T21:29:16.593 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlabel.ko 2024-04-26T21:29:16.593 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlimit.ko 2024-04-26T21:29:16.593 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connmark.ko 2024-04-26T21:29:16.593 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_conntrack.ko 2024-04-26T21:29:16.593 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cpu.ko 2024-04-26T21:29:16.593 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dccp.ko 2024-04-26T21:29:16.593 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_devgroup.ko 2024-04-26T21:29:16.593 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dscp.ko 2024-04-26T21:29:16.594 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ecn.ko 2024-04-26T21:29:16.594 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_esp.ko 2024-04-26T21:29:16.594 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hashlimit.ko 2024-04-26T21:29:16.594 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_helper.ko 2024-04-26T21:29:16.594 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hl.ko 2024-04-26T21:29:16.594 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipcomp.ko 2024-04-26T21:29:16.594 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_iprange.ko 2024-04-26T21:29:16.594 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipvs.ko 2024-04-26T21:29:16.594 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_l2tp.ko 2024-04-26T21:29:16.594 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_length.ko 2024-04-26T21:29:16.594 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_limit.ko 2024-04-26T21:29:16.594 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mac.ko 2024-04-26T21:29:16.594 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mark.ko 2024-04-26T21:29:16.594 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_multiport.ko 2024-04-26T21:29:16.595 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nat.ko 2024-04-26T21:29:16.595 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nfacct.ko 2024-04-26T21:29:16.595 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_osf.ko 2024-04-26T21:29:16.595 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_owner.ko 2024-04-26T21:29:16.595 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_physdev.ko 2024-04-26T21:29:16.595 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_pkttype.ko 2024-04-26T21:29:16.595 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_policy.ko 2024-04-26T21:29:16.595 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_quota.ko 2024-04-26T21:29:16.595 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_rateest.ko 2024-04-26T21:29:16.595 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_realm.ko 2024-04-26T21:29:16.595 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_recent.ko 2024-04-26T21:29:16.595 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_sctp.ko 2024-04-26T21:29:16.595 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_set.ko 2024-04-26T21:29:16.595 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_socket.ko 2024-04-26T21:29:16.596 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_state.ko 2024-04-26T21:29:16.596 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_statistic.ko 2024-04-26T21:29:16.596 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_string.ko 2024-04-26T21:29:16.596 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_tcpmss.ko 2024-04-26T21:29:16.596 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_time.ko 2024-04-26T21:29:16.596 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_u32.ko 2024-04-26T21:29:16.596 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink 2024-04-26T21:29:16.596 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink/netlink_diag.ko 2024-04-26T21:29:16.596 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh 2024-04-26T21:29:16.596 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh/nsh.ko 2024-04-26T21:29:16.596 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch 2024-04-26T21:29:16.596 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/openvswitch.ko 2024-04-26T21:29:16.596 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-geneve.ko 2024-04-26T21:29:16.596 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-gre.ko 2024-04-26T21:29:16.597 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-vxlan.ko 2024-04-26T21:29:16.597 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet 2024-04-26T21:29:16.597 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet/af_packet_diag.ko 2024-04-26T21:29:16.597 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds 2024-04-26T21:29:16.597 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds.ko 2024-04-26T21:29:16.597 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_rdma.ko 2024-04-26T21:29:16.597 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_tcp.ko 2024-04-26T21:29:16.597 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched 2024-04-26T21:29:16.597 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_csum.ko 2024-04-26T21:29:16.597 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_gact.ko 2024-04-26T21:29:16.597 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_mirred.ko 2024-04-26T21:29:16.597 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_nat.ko 2024-04-26T21:29:16.597 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_pedit.ko 2024-04-26T21:29:16.597 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_police.ko 2024-04-26T21:29:16.597 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_simple.ko 2024-04-26T21:29:16.598 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_skbedit.ko 2024-04-26T21:29:16.598 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_vlan.ko 2024-04-26T21:29:16.598 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_basic.ko 2024-04-26T21:29:16.598 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_bpf.ko 2024-04-26T21:29:16.598 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_flow.ko 2024-04-26T21:29:16.598 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_fw.ko 2024-04-26T21:29:16.598 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_route.ko 2024-04-26T21:29:16.598 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_u32.ko 2024-04-26T21:29:16.598 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_cmp.ko 2024-04-26T21:29:16.598 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_ipset.ko 2024-04-26T21:29:16.598 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_meta.ko 2024-04-26T21:29:16.598 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_nbyte.ko 2024-04-26T21:29:16.598 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_text.ko 2024-04-26T21:29:16.598 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_u32.ko 2024-04-26T21:29:16.599 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_choke.ko 2024-04-26T21:29:16.599 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_codel.ko 2024-04-26T21:29:16.599 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_drr.ko 2024-04-26T21:29:16.599 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_fq.ko 2024-04-26T21:29:16.599 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_gred.ko 2024-04-26T21:29:16.599 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hfsc.ko 2024-04-26T21:29:16.599 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hhf.ko 2024-04-26T21:29:16.599 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_htb.ko 2024-04-26T21:29:16.599 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_ingress.ko 2024-04-26T21:29:16.599 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio.ko 2024-04-26T21:29:16.599 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio_lib.ko 2024-04-26T21:29:16.599 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_multiq.ko 2024-04-26T21:29:16.599 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_netem.ko 2024-04-26T21:29:16.599 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_pie.ko 2024-04-26T21:29:16.600 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_plug.ko 2024-04-26T21:29:16.600 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_prio.ko 2024-04-26T21:29:16.600 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_qfq.ko 2024-04-26T21:29:16.600 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_red.ko 2024-04-26T21:29:16.600 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfb.ko 2024-04-26T21:29:16.600 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfq.ko 2024-04-26T21:29:16.600 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_tbf.ko 2024-04-26T21:29:16.600 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_teql.ko 2024-04-26T21:29:16.600 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp 2024-04-26T21:29:16.600 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp.ko 2024-04-26T21:29:16.600 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp_diag.ko 2024-04-26T21:29:16.600 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc 2024-04-26T21:29:16.600 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss 2024-04-26T21:29:16.600 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-04-26T21:29:16.600 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-04-26T21:29:16.601 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/sunrpc.ko 2024-04-26T21:29:16.601 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma 2024-04-26T21:29:16.601 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-04-26T21:29:16.601 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc 2024-04-26T21:29:16.601 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc.ko 2024-04-26T21:29:16.601 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc_diag.ko 2024-04-26T21:29:16.601 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix 2024-04-26T21:29:16.601 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix/unix_diag.ko 2024-04-26T21:29:16.601 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock 2024-04-26T21:29:16.601 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-04-26T21:29:16.601 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-04-26T21:29:16.601 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock.ko 2024-04-26T21:29:16.601 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_diag.ko 2024-04-26T21:29:16.601 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_loopback.ko 2024-04-26T21:29:16.602 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm 2024-04-26T21:29:16.602 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm/xfrm_ipcomp.ko 2024-04-26T21:29:16.602 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security 2024-04-26T21:29:16.602 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys 2024-04-26T21:29:16.602 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys 2024-04-26T21:29:16.602 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-04-26T21:29:16.602 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys 2024-04-26T21:29:16.602 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys/trusted.ko 2024-04-26T21:29:16.602 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias 2024-04-26T21:29:16.602 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias.bin 2024-04-26T21:29:16.602 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin 2024-04-26T21:29:16.602 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.alias.bin 2024-04-26T21:29:16.602 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.bin 2024-04-26T21:29:16.602 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.modinfo 2024-04-26T21:29:16.603 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep 2024-04-26T21:29:16.603 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep.bin 2024-04-26T21:29:16.603 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.devname 2024-04-26T21:29:16.603 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.order 2024-04-26T21:29:16.603 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.softdep 2024-04-26T21:29:16.603 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols 2024-04-26T21:29:16.603 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols.bin 2024-04-26T21:29:16.603 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-04-26T21:29:16.607 DEBUG:teuthology.task.kernel:get_image_version: 6.9.0-rc3-g806ec36b2112 2024-04-26T21:29:16.607 DEBUG:teuthology.orchestra.run.smithi076:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-04-26T21:29:16.629 INFO:teuthology.orchestra.run.smithi076.stdout:postinstall scriptlet (using /bin/sh): 2024-04-26T21:29:16.629 INFO:teuthology.orchestra.run.smithi076.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-26T21:29:16.629 INFO:teuthology.orchestra.run.smithi076.stdout: /usr/bin/kernel-install add 6.9.0-rc3-g806ec36b2112 /lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-04-26T21:29:16.629 INFO:teuthology.orchestra.run.smithi076.stdout:fi 2024-04-26T21:29:16.629 INFO:teuthology.orchestra.run.smithi076.stdout:for file in vmlinuz System.map config; do 2024-04-26T21:29:16.629 INFO:teuthology.orchestra.run.smithi076.stdout: if ! cmp --silent "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112"; then 2024-04-26T21:29:16.629 INFO:teuthology.orchestra.run.smithi076.stdout: cp "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112" 2024-04-26T21:29:16.629 INFO:teuthology.orchestra.run.smithi076.stdout: fi 2024-04-26T21:29:16.629 INFO:teuthology.orchestra.run.smithi076.stdout:done 2024-04-26T21:29:16.630 INFO:teuthology.orchestra.run.smithi076.stdout:if [ -d "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" ] && \ 2024-04-26T21:29:16.630 INFO:teuthology.orchestra.run.smithi076.stdout: ! diff -rq "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" >/dev/null 2>&1; then 2024-04-26T21:29:16.630 INFO:teuthology.orchestra.run.smithi076.stdout: rm -rf "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-04-26T21:29:16.630 INFO:teuthology.orchestra.run.smithi076.stdout: cp -r "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-04-26T21:29:16.630 INFO:teuthology.orchestra.run.smithi076.stdout:fi 2024-04-26T21:29:16.630 INFO:teuthology.orchestra.run.smithi076.stdout:if [ ! -e "/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep" ]; then 2024-04-26T21:29:16.630 INFO:teuthology.orchestra.run.smithi076.stdout: /usr/sbin/depmod 6.9.0-rc3-g806ec36b2112 2024-04-26T21:29:16.630 INFO:teuthology.orchestra.run.smithi076.stdout:fi 2024-04-26T21:29:16.630 INFO:teuthology.orchestra.run.smithi076.stdout:preuninstall scriptlet (using /bin/sh): 2024-04-26T21:29:16.630 INFO:teuthology.orchestra.run.smithi076.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-26T21:29:16.630 INFO:teuthology.orchestra.run.smithi076.stdout:kernel-install remove 6.9.0-rc3-g806ec36b2112 2024-04-26T21:29:16.630 INFO:teuthology.orchestra.run.smithi076.stdout:fi 2024-04-26T21:29:16.631 DEBUG:teuthology.orchestra.run.smithi076:> sudo rpm -qi grub2-tools 2024-04-26T21:29:16.715 INFO:teuthology.orchestra.run.smithi076.stdout:Name : grub2-tools 2024-04-26T21:29:16.715 INFO:teuthology.orchestra.run.smithi076.stdout:Epoch : 1 2024-04-26T21:29:16.715 INFO:teuthology.orchestra.run.smithi076.stdout:Version : 2.06 2024-04-26T21:29:16.715 INFO:teuthology.orchestra.run.smithi076.stdout:Release : 68.el9 2024-04-26T21:29:16.715 INFO:teuthology.orchestra.run.smithi076.stdout:Architecture: x86_64 2024-04-26T21:29:16.715 INFO:teuthology.orchestra.run.smithi076.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-04-26T21:29:16.715 INFO:teuthology.orchestra.run.smithi076.stdout:Group : Unspecified 2024-04-26T21:29:16.715 INFO:teuthology.orchestra.run.smithi076.stdout:Size : 8264254 2024-04-26T21:29:16.715 INFO:teuthology.orchestra.run.smithi076.stdout:License : GPLv3+ 2024-04-26T21:29:16.715 INFO:teuthology.orchestra.run.smithi076.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-04-26T21:29:16.715 INFO:teuthology.orchestra.run.smithi076.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-04-26T21:29:16.716 INFO:teuthology.orchestra.run.smithi076.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-04-26T21:29:16.716 INFO:teuthology.orchestra.run.smithi076.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-04-26T21:29:16.716 INFO:teuthology.orchestra.run.smithi076.stdout:Packager : builder@centos.org 2024-04-26T21:29:16.716 INFO:teuthology.orchestra.run.smithi076.stdout:Vendor : CentOS 2024-04-26T21:29:16.716 INFO:teuthology.orchestra.run.smithi076.stdout:URL : http://www.gnu.org/software/grub/ 2024-04-26T21:29:16.716 INFO:teuthology.orchestra.run.smithi076.stdout:Summary : Support tools for GRUB. 2024-04-26T21:29:16.716 INFO:teuthology.orchestra.run.smithi076.stdout:Description : 2024-04-26T21:29:16.716 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-26T21:29:16.716 INFO:teuthology.orchestra.run.smithi076.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-04-26T21:29:16.716 INFO:teuthology.orchestra.run.smithi076.stdout:customizable bootloader with modular architecture. It supports a rich 2024-04-26T21:29:16.716 INFO:teuthology.orchestra.run.smithi076.stdout:variety of kernel formats, file systems, computer architectures and 2024-04-26T21:29:16.716 INFO:teuthology.orchestra.run.smithi076.stdout:hardware devices. 2024-04-26T21:29:16.716 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-26T21:29:16.716 INFO:teuthology.orchestra.run.smithi076.stdout:This subpackage provides tools for support of all platforms. 2024-04-26T21:29:16.717 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-04-26T21:29:16.717 INFO:teuthology.task.kernel:Updating grub on smithi076 to boot 6.9.0-rc3-g806ec36b2112 2024-04-26T21:29:16.717 DEBUG:teuthology.orchestra.run.smithi076:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-04-26T21:29:17.294 INFO:teuthology.orchestra.run.smithi076.stderr:Generating grub configuration file ... 2024-04-26T21:29:18.577 INFO:teuthology.orchestra.run.smithi076.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-04-26T21:29:18.601 INFO:teuthology.orchestra.run.smithi076.stderr:done 2024-04-26T21:29:18.604 DEBUG:teuthology.orchestra.run.smithi076:> mktemp 2024-04-26T21:29:18.646 INFO:teuthology.orchestra.run.smithi076.stdout:/tmp/tmp.ZPpPXxISsG 2024-04-26T21:29:18.647 DEBUG:teuthology.orchestra.run.smithi076:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.ZPpPXxISsG 2024-04-26T21:29:18.716 DEBUG:teuthology.orchestra.run.smithi076:> sudo chmod 0666 /tmp/tmp.ZPpPXxISsG 2024-04-26T21:29:18.860 DEBUG:teuthology.orchestra.remote:smithi076:/tmp/tmp.ZPpPXxISsG is 6KB 2024-04-26T21:29:18.910 DEBUG:teuthology.orchestra.run.smithi076:> rm -fr /tmp/tmp.ZPpPXxISsG 2024-04-26T21:29:18.924 DEBUG:teuthology.orchestra.run.smithi076:> sudo /bin/ls /boot/loader/entries || true 2024-04-26T21:29:18.991 INFO:teuthology.orchestra.run.smithi076.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-04-26T21:29:18.991 INFO:teuthology.orchestra.run.smithi076.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-04-26T21:29:18.991 INFO:teuthology.orchestra.run.smithi076.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-04-26T21:29:18.991 INFO:teuthology.orchestra.run.smithi076.stdout:7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112.conf 2024-04-26T21:29:18.992 DEBUG:teuthology.orchestra.run.smithi076:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112 2024-04-26T21:29:19.165 DEBUG:teuthology.orchestra.run.smithi076:> sudo shutdown -r now 2024-04-26T21:29:19.412 INFO:teuthology.orchestra.run.smithi164.stdout:kernel-6.9.0_rc3_g806ec36b2112-1 ######################################## 2024-04-26T21:29:49.169 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-04-26T21:29:49.169 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi076.front.sepia.ceph.com' 2024-04-26T21:29:49.170 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi076.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T21:29:52.576 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi072.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-04-26T21:29:52.577 DEBUG:teuthology.orchestra.run.smithi072:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-04-26T21:29:52.674 INFO:teuthology.orchestra.run.smithi072.stdout:/boot/System.map-6.9.0-rc3-g806ec36b2112 2024-04-26T21:29:52.674 INFO:teuthology.orchestra.run.smithi072.stdout:/boot/config-6.9.0-rc3-g806ec36b2112 2024-04-26T21:29:52.674 INFO:teuthology.orchestra.run.smithi072.stdout:/boot/vmlinuz-6.9.0-rc3-g806ec36b2112 2024-04-26T21:29:52.674 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/System.map 2024-04-26T21:29:52.674 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/config 2024-04-26T21:29:52.674 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel 2024-04-26T21:29:52.674 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch 2024-04-26T21:29:52.674 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86 2024-04-26T21:29:52.674 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto 2024-04-26T21:29:52.674 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-04-26T21:29:52.675 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-04-26T21:29:52.675 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-04-26T21:29:52.675 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-04-26T21:29:52.675 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-04-26T21:29:52.675 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-04-26T21:29:52.675 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-04-26T21:29:52.675 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32c-intel.ko 2024-04-26T21:29:52.675 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-04-26T21:29:52.675 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-04-26T21:29:52.675 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-04-26T21:29:52.675 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-04-26T21:29:52.675 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx2.ko 2024-04-26T21:29:52.675 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-04-26T21:29:52.675 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-04-26T21:29:52.676 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-04-26T21:29:52.676 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-04-26T21:29:52.676 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-04-26T21:29:52.676 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-04-26T21:29:52.676 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-04-26T21:29:52.676 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel 2024-04-26T21:29:52.676 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu 2024-04-26T21:29:52.676 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce 2024-04-26T21:29:52.676 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-04-26T21:29:52.676 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm 2024-04-26T21:29:52.676 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-amd.ko 2024-04-26T21:29:52.676 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-intel.ko 2024-04-26T21:29:52.676 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm.ko 2024-04-26T21:29:52.676 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto 2024-04-26T21:29:52.676 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ansi_cprng.ko 2024-04-26T21:29:52.677 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/anubis.ko 2024-04-26T21:29:52.677 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/arc4.ko 2024-04-26T21:29:52.677 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx 2024-04-26T21:29:52.677 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_memcpy.ko 2024-04-26T21:29:52.677 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_pq.ko 2024-04-26T21:29:52.677 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_raid6_recov.ko 2024-04-26T21:29:52.677 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_tx.ko 2024-04-26T21:29:52.677 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_xor.ko 2024-04-26T21:29:52.677 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/raid6test.ko 2024-04-26T21:29:52.677 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authenc.ko 2024-04-26T21:29:52.677 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authencesn.ko 2024-04-26T21:29:52.677 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blake2b_generic.ko 2024-04-26T21:29:52.677 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_common.ko 2024-04-26T21:29:52.677 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_generic.ko 2024-04-26T21:29:52.678 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/camellia_generic.ko 2024-04-26T21:29:52.678 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast5_generic.ko 2024-04-26T21:29:52.678 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast6_generic.ko 2024-04-26T21:29:52.678 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast_common.ko 2024-04-26T21:29:52.678 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ccm.ko 2024-04-26T21:29:52.678 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cmac.ko 2024-04-26T21:29:52.678 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crc32_generic.ko 2024-04-26T21:29:52.678 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_engine.ko 2024-04-26T21:29:52.678 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_user.ko 2024-04-26T21:29:52.678 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cts.ko 2024-04-26T21:29:52.678 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/des_generic.ko 2024-04-26T21:29:52.678 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/dh_generic.ko 2024-04-26T21:29:52.678 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/echainiv.ko 2024-04-26T21:29:52.678 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/essiv.ko 2024-04-26T21:29:52.679 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/fcrypt.ko 2024-04-26T21:29:52.679 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/khazad.ko 2024-04-26T21:29:52.679 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4.ko 2024-04-26T21:29:52.679 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4hc.ko 2024-04-26T21:29:52.679 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/md4.ko 2024-04-26T21:29:52.679 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/michael_mic.ko 2024-04-26T21:29:52.679 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcbc.ko 2024-04-26T21:29:52.679 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcrypt.ko 2024-04-26T21:29:52.679 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/rmd160.ko 2024-04-26T21:29:52.679 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/seed.ko 2024-04-26T21:29:52.679 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/serpent_generic.ko 2024-04-26T21:29:52.679 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tcrypt.ko 2024-04-26T21:29:52.679 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tea.ko 2024-04-26T21:29:52.679 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_common.ko 2024-04-26T21:29:52.680 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_generic.ko 2024-04-26T21:29:52.680 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/vmac.ko 2024-04-26T21:29:52.680 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/wp512.ko 2024-04-26T21:29:52.680 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xcbc.ko 2024-04-26T21:29:52.680 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xor.ko 2024-04-26T21:29:52.680 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xxhash_generic.ko 2024-04-26T21:29:52.680 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers 2024-04-26T21:29:52.680 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi 2024-04-26T21:29:52.680 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_ipmi.ko 2024-04-26T21:29:52.680 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_pad.ko 2024-04-26T21:29:52.680 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/ec_sys.ko 2024-04-26T21:29:52.680 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/platform_profile.ko 2024-04-26T21:29:52.680 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbs.ko 2024-04-26T21:29:52.680 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbshc.ko 2024-04-26T21:29:52.681 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/video.ko 2024-04-26T21:29:52.681 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata 2024-04-26T21:29:52.681 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/acard-ahci.ko 2024-04-26T21:29:52.681 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ahci_platform.ko 2024-04-26T21:29:52.681 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ata_generic.ko 2024-04-26T21:29:52.681 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/libahci_platform.ko 2024-04-26T21:29:52.681 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_acpi.ko 2024-04-26T21:29:52.681 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ali.ko 2024-04-26T21:29:52.681 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_amd.ko 2024-04-26T21:29:52.681 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_artop.ko 2024-04-26T21:29:52.681 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atiixp.ko 2024-04-26T21:29:52.681 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atp867x.ko 2024-04-26T21:29:52.681 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd640.ko 2024-04-26T21:29:52.681 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd64x.ko 2024-04-26T21:29:52.682 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cypress.ko 2024-04-26T21:29:52.682 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_efar.ko 2024-04-26T21:29:52.682 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt366.ko 2024-04-26T21:29:52.682 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt37x.ko 2024-04-26T21:29:52.682 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x2n.ko 2024-04-26T21:29:52.682 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x3.ko 2024-04-26T21:29:52.682 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it8213.ko 2024-04-26T21:29:52.682 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it821x.ko 2024-04-26T21:29:52.682 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_jmicron.ko 2024-04-26T21:29:52.682 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_marvell.ko 2024-04-26T21:29:52.682 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_mpiix.ko 2024-04-26T21:29:52.682 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_netcell.ko 2024-04-26T21:29:52.682 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ninja32.ko 2024-04-26T21:29:52.682 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87410.ko 2024-04-26T21:29:52.683 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87415.ko 2024-04-26T21:29:52.683 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_oldpiix.ko 2024-04-26T21:29:52.683 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_opti.ko 2024-04-26T21:29:52.683 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_optidma.ko 2024-04-26T21:29:52.683 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pcmcia.ko 2024-04-26T21:29:52.683 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc2027x.ko 2024-04-26T21:29:52.683 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-04-26T21:29:52.683 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_piccolo.ko 2024-04-26T21:29:52.683 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_rdc.ko 2024-04-26T21:29:52.683 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sch.ko 2024-04-26T21:29:52.683 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_serverworks.ko 2024-04-26T21:29:52.683 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sil680.ko 2024-04-26T21:29:52.683 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sis.ko 2024-04-26T21:29:52.683 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sl82c105.ko 2024-04-26T21:29:52.683 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_triflex.ko 2024-04-26T21:29:52.683 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_via.ko 2024-04-26T21:29:52.684 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pdc_adma.ko 2024-04-26T21:29:52.684 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_inic162x.ko 2024-04-26T21:29:52.684 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_mv.ko 2024-04-26T21:29:52.684 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_nv.ko 2024-04-26T21:29:52.684 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_promise.ko 2024-04-26T21:29:52.684 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_qstor.ko 2024-04-26T21:29:52.684 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil.ko 2024-04-26T21:29:52.684 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil24.ko 2024-04-26T21:29:52.684 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sis.ko 2024-04-26T21:29:52.684 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_svw.ko 2024-04-26T21:29:52.684 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sx4.ko 2024-04-26T21:29:52.684 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_uli.ko 2024-04-26T21:29:52.684 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_via.ko 2024-04-26T21:29:52.684 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_vsc.ko 2024-04-26T21:29:52.684 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay 2024-04-26T21:29:52.685 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay/ks0108.ko 2024-04-26T21:29:52.685 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base 2024-04-26T21:29:52.685 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap 2024-04-26T21:29:52.685 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-i2c.ko 2024-04-26T21:29:52.685 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-mmio.ko 2024-04-26T21:29:52.685 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma 2024-04-26T21:29:52.685 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma/bcma.ko 2024-04-26T21:29:52.685 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block 2024-04-26T21:29:52.685 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe 2024-04-26T21:29:52.685 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe/aoe.ko 2024-04-26T21:29:52.685 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/brd.ko 2024-04-26T21:29:52.685 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd 2024-04-26T21:29:52.685 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd/drbd.ko 2024-04-26T21:29:52.685 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/floppy.ko 2024-04-26T21:29:52.685 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/loop.ko 2024-04-26T21:29:52.686 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx 2024-04-26T21:29:52.686 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-04-26T21:29:52.686 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/nbd.ko 2024-04-26T21:29:52.686 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk 2024-04-26T21:29:52.686 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk/null_blk.ko 2024-04-26T21:29:52.686 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/pktcdvd.ko 2024-04-26T21:29:52.686 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/rbd.ko 2024-04-26T21:29:52.686 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/virtio_blk.ko 2024-04-26T21:29:52.686 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback 2024-04-26T21:29:52.686 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-04-26T21:29:52.686 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkfront.ko 2024-04-26T21:29:52.686 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram 2024-04-26T21:29:52.686 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram/zram.ko 2024-04-26T21:29:52.687 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char 2024-04-26T21:29:52.687 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hangcheck-timer.ko 2024-04-26T21:29:52.687 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random 2024-04-26T21:29:52.687 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/amd-rng.ko 2024-04-26T21:29:52.687 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/intel-rng.ko 2024-04-26T21:29:52.687 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-04-26T21:29:52.687 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/via-rng.ko 2024-04-26T21:29:52.687 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/virtio-rng.ko 2024-04-26T21:29:52.687 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi 2024-04-26T21:29:52.687 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-04-26T21:29:52.687 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-04-26T21:29:52.687 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-04-26T21:29:52.687 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_si.ko 2024-04-26T21:29:52.687 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-04-26T21:29:52.687 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-04-26T21:29:52.687 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/lp.ko 2024-04-26T21:29:52.687 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave 2024-04-26T21:29:52.688 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave/mwave.ko 2024-04-26T21:29:52.688 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ppdev.ko 2024-04-26T21:29:52.688 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tlclk.ko 2024-04-26T21:29:52.688 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm 2024-04-26T21:29:52.688 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm.ko 2024-04-26T21:29:52.688 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_atmel.ko 2024-04-26T21:29:52.688 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_infineon.ko 2024-04-26T21:29:52.688 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_nsc.ko 2024-04-26T21:29:52.688 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis.ko 2024-04-26T21:29:52.688 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-04-26T21:29:52.688 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/uv_mmtimer.ko 2024-04-26T21:29:52.688 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/virtio_console.ko 2024-04-26T21:29:52.688 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq 2024-04-26T21:29:52.688 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-04-26T21:29:52.688 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-04-26T21:29:52.689 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/p4-clockmod.ko 2024-04-26T21:29:52.689 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-04-26T21:29:52.689 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/powernow-k8.ko 2024-04-26T21:29:52.689 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/speedstep-lib.ko 2024-04-26T21:29:52.689 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto 2024-04-26T21:29:52.689 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp 2024-04-26T21:29:52.689 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-04-26T21:29:52.689 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp.ko 2024-04-26T21:29:52.689 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel 2024-04-26T21:29:52.689 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat 2024-04-26T21:29:52.689 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common 2024-04-26T21:29:52.689 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-04-26T21:29:52.689 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-04-26T21:29:52.689 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-04-26T21:29:52.689 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-aes.ko 2024-04-26T21:29:52.689 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-sha.ko 2024-04-26T21:29:52.689 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio 2024-04-26T21:29:52.690 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-04-26T21:29:52.690 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca 2024-04-26T21:29:52.690 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca/dca.ko 2024-04-26T21:29:52.690 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq 2024-04-26T21:29:52.690 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-04-26T21:29:52.690 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma 2024-04-26T21:29:52.690 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw 2024-04-26T21:29:52.690 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw/dw_dmac.ko 2024-04-26T21:29:52.690 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat 2024-04-26T21:29:52.690 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat/ioatdma.ko 2024-04-26T21:29:52.690 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac 2024-04-26T21:29:52.690 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/amd64_edac.ko 2024-04-26T21:29:52.690 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/e752x_edac.ko 2024-04-26T21:29:52.690 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/edac_mce_amd.ko 2024-04-26T21:29:52.691 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3000_edac.ko 2024-04-26T21:29:52.691 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3200_edac.ko 2024-04-26T21:29:52.691 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5100_edac.ko 2024-04-26T21:29:52.691 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5400_edac.ko 2024-04-26T21:29:52.691 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7300_edac.ko 2024-04-26T21:29:52.691 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7core_edac.ko 2024-04-26T21:29:52.691 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i82975x_edac.ko 2024-04-26T21:29:52.691 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/ie31200_edac.ko 2024-04-26T21:29:52.691 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/sb_edac.ko 2024-04-26T21:29:52.691 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/x38_edac.ko 2024-04-26T21:29:52.691 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware 2024-04-26T21:29:52.691 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/edd.ko 2024-04-26T21:29:52.691 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/iscsi_ibft.ko 2024-04-26T21:29:52.691 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio 2024-04-26T21:29:52.692 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-ich.ko 2024-04-26T21:29:52.692 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-viperboard.ko 2024-04-26T21:29:52.692 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid 2024-04-26T21:29:52.692 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-appleir.ko 2024-04-26T21:29:52.692 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-aureal.ko 2024-04-26T21:29:52.692 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-axff.ko 2024-04-26T21:29:52.692 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-dr.ko 2024-04-26T21:29:52.692 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elecom.ko 2024-04-26T21:29:52.692 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elo.ko 2024-04-26T21:29:52.692 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-emsff.ko 2024-04-26T21:29:52.692 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gaff.ko 2024-04-26T21:29:52.692 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gt683r.ko 2024-04-26T21:29:52.692 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gyration.ko 2024-04-26T21:29:52.692 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-kbd.ko 2024-04-26T21:29:52.692 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-mouse.ko 2024-04-26T21:29:52.693 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtekff.ko 2024-04-26T21:29:52.693 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-hyperv.ko 2024-04-26T21:29:52.693 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-icade.ko 2024-04-26T21:29:52.693 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-keytouch.ko 2024-04-26T21:29:52.693 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-kye.ko 2024-04-26T21:29:52.693 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lcpower.ko 2024-04-26T21:29:52.693 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-led.ko 2024-04-26T21:29:52.693 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lenovo.ko 2024-04-26T21:29:52.693 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-dj.ko 2024-04-26T21:29:52.693 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-04-26T21:29:52.693 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-multitouch.ko 2024-04-26T21:29:52.693 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-ortek.ko 2024-04-26T21:29:52.693 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-penmount.ko 2024-04-26T21:29:52.693 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-petalynx.ko 2024-04-26T21:29:52.693 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-picolcd.ko 2024-04-26T21:29:52.694 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-pl.ko 2024-04-26T21:29:52.694 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-plantronics.ko 2024-04-26T21:29:52.694 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-primax.ko 2024-04-26T21:29:52.694 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-rmi.ko 2024-04-26T21:29:52.694 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-arvo.ko 2024-04-26T21:29:52.694 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-common.ko 2024-04-26T21:29:52.694 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-isku.ko 2024-04-26T21:29:52.694 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kone.ko 2024-04-26T21:29:52.694 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-04-26T21:29:52.694 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-konepure.ko 2024-04-26T21:29:52.694 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-04-26T21:29:52.694 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-lua.ko 2024-04-26T21:29:52.694 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-pyra.ko 2024-04-26T21:29:52.695 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-ryos.ko 2024-04-26T21:29:52.695 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-savu.ko 2024-04-26T21:29:52.695 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat.ko 2024-04-26T21:29:52.695 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-saitek.ko 2024-04-26T21:29:52.695 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-samsung.ko 2024-04-26T21:29:52.695 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sensor-hub.ko 2024-04-26T21:29:52.695 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sjoy.ko 2024-04-26T21:29:52.695 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sony.ko 2024-04-26T21:29:52.695 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-speedlink.ko 2024-04-26T21:29:52.695 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-steelseries.ko 2024-04-26T21:29:52.695 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sunplus.ko 2024-04-26T21:29:52.695 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-thrustmaster.ko 2024-04-26T21:29:52.695 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tivo.ko 2024-04-26T21:29:52.695 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tmff.ko 2024-04-26T21:29:52.695 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-topseed.ko 2024-04-26T21:29:52.696 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-twinhan.ko 2024-04-26T21:29:52.696 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-uclogic.ko 2024-04-26T21:29:52.696 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-waltop.ko 2024-04-26T21:29:52.696 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-wiimote.ko 2024-04-26T21:29:52.696 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-xinmo.ko 2024-04-26T21:29:52.696 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zpff.ko 2024-04-26T21:29:52.696 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zydacron.ko 2024-04-26T21:29:52.696 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/uhid.ko 2024-04-26T21:29:52.696 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/wacom.ko 2024-04-26T21:29:52.696 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv 2024-04-26T21:29:52.696 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_balloon.ko 2024-04-26T21:29:52.696 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_utils.ko 2024-04-26T21:29:52.696 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_vmbus.ko 2024-04-26T21:29:52.696 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon 2024-04-26T21:29:52.696 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru.ko 2024-04-26T21:29:52.697 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru3.ko 2024-04-26T21:29:52.697 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/acpi_power_meter.ko 2024-04-26T21:29:52.697 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7414.ko 2024-04-26T21:29:52.697 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7418.ko 2024-04-26T21:29:52.697 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adc128d818.ko 2024-04-26T21:29:52.697 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1025.ko 2024-04-26T21:29:52.697 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1026.ko 2024-04-26T21:29:52.697 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1029.ko 2024-04-26T21:29:52.697 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1031.ko 2024-04-26T21:29:52.697 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm9240.ko 2024-04-26T21:29:52.697 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ads7828.ko 2024-04-26T21:29:52.697 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7410.ko 2024-04-26T21:29:52.697 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7411.ko 2024-04-26T21:29:52.697 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7462.ko 2024-04-26T21:29:52.698 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7470.ko 2024-04-26T21:29:52.698 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7475.ko 2024-04-26T21:29:52.698 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7x10.ko 2024-04-26T21:29:52.698 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/amc6821.ko 2024-04-26T21:29:52.698 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/applesmc.ko 2024-04-26T21:29:52.698 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asb100.ko 2024-04-26T21:29:52.698 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asc7621.ko 2024-04-26T21:29:52.698 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asus_atk0110.ko 2024-04-26T21:29:52.698 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/atxp1.ko 2024-04-26T21:29:52.698 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/coretemp.ko 2024-04-26T21:29:52.698 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-04-26T21:29:52.698 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dme1737.ko 2024-04-26T21:29:52.698 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds1621.ko 2024-04-26T21:29:52.699 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds620.ko 2024-04-26T21:29:52.699 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc1403.ko 2024-04-26T21:29:52.699 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc6w201.ko 2024-04-26T21:29:52.699 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71805f.ko 2024-04-26T21:29:52.699 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71882fg.ko 2024-04-26T21:29:52.699 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f75375s.ko 2024-04-26T21:29:52.699 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fam15h_power.ko 2024-04-26T21:29:52.699 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fschmd.ko 2024-04-26T21:29:52.699 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g760a.ko 2024-04-26T21:29:52.699 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g762.ko 2024-04-26T21:29:52.699 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl518sm.ko 2024-04-26T21:29:52.699 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl520sm.ko 2024-04-26T21:29:52.699 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/hwmon-vid.ko 2024-04-26T21:29:52.699 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5500_temp.ko 2024-04-26T21:29:52.699 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5k_amb.ko 2024-04-26T21:29:52.699 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmaem.ko 2024-04-26T21:29:52.700 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmpex.ko 2024-04-26T21:29:52.700 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina209.ko 2024-04-26T21:29:52.700 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina2xx.ko 2024-04-26T21:29:52.700 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/it87.ko 2024-04-26T21:29:52.700 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k10temp.ko 2024-04-26T21:29:52.700 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k8temp.ko 2024-04-26T21:29:52.700 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lineage-pem.ko 2024-04-26T21:29:52.700 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm63.ko 2024-04-26T21:29:52.700 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm73.ko 2024-04-26T21:29:52.700 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm75.ko 2024-04-26T21:29:52.700 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm77.ko 2024-04-26T21:29:52.700 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm78.ko 2024-04-26T21:29:52.700 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm80.ko 2024-04-26T21:29:52.700 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm83.ko 2024-04-26T21:29:52.700 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm85.ko 2024-04-26T21:29:52.700 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm87.ko 2024-04-26T21:29:52.700 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm90.ko 2024-04-26T21:29:52.701 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm92.ko 2024-04-26T21:29:52.701 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm93.ko 2024-04-26T21:29:52.701 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95234.ko 2024-04-26T21:29:52.701 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95241.ko 2024-04-26T21:29:52.701 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95245.ko 2024-04-26T21:29:52.701 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc2945.ko 2024-04-26T21:29:52.701 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4151.ko 2024-04-26T21:29:52.701 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4215.ko 2024-04-26T21:29:52.701 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4222.ko 2024-04-26T21:29:52.701 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4245.ko 2024-04-26T21:29:52.701 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4260.ko 2024-04-26T21:29:52.701 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4261.ko 2024-04-26T21:29:52.701 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max16065.ko 2024-04-26T21:29:52.701 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1619.ko 2024-04-26T21:29:52.701 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1668.ko 2024-04-26T21:29:52.701 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max197.ko 2024-04-26T21:29:52.702 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6639.ko 2024-04-26T21:29:52.702 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6650.ko 2024-04-26T21:29:52.702 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6697.ko 2024-04-26T21:29:52.702 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/mcp3021.ko 2024-04-26T21:29:52.702 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6683.ko 2024-04-26T21:29:52.702 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775-core.ko 2024-04-26T21:29:52.702 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775.ko 2024-04-26T21:29:52.702 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct7802.ko 2024-04-26T21:29:52.702 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87360.ko 2024-04-26T21:29:52.702 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87427.ko 2024-04-26T21:29:52.702 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pcf8591.ko 2024-04-26T21:29:52.702 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus 2024-04-26T21:29:52.702 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-04-26T21:29:52.702 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-04-26T21:29:52.703 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-04-26T21:29:52.703 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max16064.ko 2024-04-26T21:29:52.703 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max34440.ko 2024-04-26T21:29:52.703 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max8688.ko 2024-04-26T21:29:52.703 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-04-26T21:29:52.703 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-04-26T21:29:52.703 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-04-26T21:29:52.703 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-04-26T21:29:52.703 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-04-26T21:29:52.703 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-04-26T21:29:52.703 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/powr1220.ko 2024-04-26T21:29:52.703 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5627.ko 2024-04-26T21:29:52.703 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5636.ko 2024-04-26T21:29:52.703 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch56xx-common.ko 2024-04-26T21:29:52.703 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht15.ko 2024-04-26T21:29:52.703 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht21.ko 2024-04-26T21:29:52.704 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/shtc1.ko 2024-04-26T21:29:52.704 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sis5595.ko 2024-04-26T21:29:52.704 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47b397.ko 2024-04-26T21:29:52.704 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m1.ko 2024-04-26T21:29:52.704 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m192.ko 2024-04-26T21:29:52.704 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/thmc50.ko 2024-04-26T21:29:52.704 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp102.ko 2024-04-26T21:29:52.704 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp103.ko 2024-04-26T21:29:52.704 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp401.ko 2024-04-26T21:29:52.704 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp421.ko 2024-04-26T21:29:52.704 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via-cputemp.ko 2024-04-26T21:29:52.704 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via686a.ko 2024-04-26T21:29:52.704 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt1211.ko 2024-04-26T21:29:52.704 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt8231.ko 2024-04-26T21:29:52.704 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627ehf.ko 2024-04-26T21:29:52.704 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627hf.ko 2024-04-26T21:29:52.705 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83781d.ko 2024-04-26T21:29:52.705 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83791d.ko 2024-04-26T21:29:52.705 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83792d.ko 2024-04-26T21:29:52.705 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83793.ko 2024-04-26T21:29:52.705 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83795.ko 2024-04-26T21:29:52.705 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l785ts.ko 2024-04-26T21:29:52.705 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l786ng.ko 2024-04-26T21:29:52.705 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c 2024-04-26T21:29:52.705 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos 2024-04-26T21:29:52.705 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-04-26T21:29:52.705 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-04-26T21:29:52.705 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses 2024-04-26T21:29:52.705 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-04-26T21:29:52.705 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-04-26T21:29:52.705 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-04-26T21:29:52.705 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-04-26T21:29:52.706 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-04-26T21:29:52.706 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-04-26T21:29:52.706 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-04-26T21:29:52.706 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-04-26T21:29:52.706 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-i801.ko 2024-04-26T21:29:52.706 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-isch.ko 2024-04-26T21:29:52.706 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-04-26T21:29:52.706 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-04-26T21:29:52.706 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-04-26T21:29:52.706 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-parport.ko 2024-04-26T21:29:52.706 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-04-26T21:29:52.706 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-04-26T21:29:52.706 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-04-26T21:29:52.707 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-04-26T21:29:52.707 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-04-26T21:29:52.707 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-04-26T21:29:52.707 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-via.ko 2024-04-26T21:29:52.707 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-04-26T21:29:52.707 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-04-26T21:29:52.707 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-dev.ko 2024-04-26T21:29:52.707 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-mux.ko 2024-04-26T21:29:52.707 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-04-26T21:29:52.707 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-smbus.ko 2024-04-26T21:29:52.707 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-stub.ko 2024-04-26T21:29:52.707 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband 2024-04-26T21:29:52.707 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core 2024-04-26T21:29:52.707 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_cm.ko 2024-04-26T21:29:52.707 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_core.ko 2024-04-26T21:29:52.707 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_umad.ko 2024-04-26T21:29:52.707 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-04-26T21:29:52.708 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/iw_cm.ko 2024-04-26T21:29:52.708 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_cm.ko 2024-04-26T21:29:52.708 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-04-26T21:29:52.708 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw 2024-04-26T21:29:52.708 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4 2024-04-26T21:29:52.708 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-04-26T21:29:52.708 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4 2024-04-26T21:29:52.708 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-04-26T21:29:52.708 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5 2024-04-26T21:29:52.708 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-04-26T21:29:52.708 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca 2024-04-26T21:29:52.708 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-04-26T21:29:52.708 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp 2024-04-26T21:29:52.708 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib 2024-04-26T21:29:52.708 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-04-26T21:29:52.708 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser 2024-04-26T21:29:52.709 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-04-26T21:29:52.709 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert 2024-04-26T21:29:52.709 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-04-26T21:29:52.709 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp 2024-04-26T21:29:52.709 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-04-26T21:29:52.709 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt 2024-04-26T21:29:52.709 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-04-26T21:29:52.709 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input 2024-04-26T21:29:52.709 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport 2024-04-26T21:29:52.709 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-04-26T21:29:52.709 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/fm801-gp.ko 2024-04-26T21:29:52.709 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/gameport.ko 2024-04-26T21:29:52.709 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/joydev.ko 2024-04-26T21:29:52.709 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard 2024-04-26T21:29:52.710 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/gpio_keys.ko 2024-04-26T21:29:52.710 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse 2024-04-26T21:29:52.710 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/appletouch.ko 2024-04-26T21:29:52.710 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/bcm5974.ko 2024-04-26T21:29:52.710 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/cyapatp.ko 2024-04-26T21:29:52.710 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/elan_i2c.ko 2024-04-26T21:29:52.710 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/sermouse.ko 2024-04-26T21:29:52.710 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-04-26T21:29:52.710 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_usb.ko 2024-04-26T21:29:52.710 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/vsxxxaa.ko 2024-04-26T21:29:52.710 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4 2024-04-26T21:29:52.710 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4/rmi_core.ko 2024-04-26T21:29:52.710 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio 2024-04-26T21:29:52.710 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/altera_ps2.ko 2024-04-26T21:29:52.711 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/arc_ps2.ko 2024-04-26T21:29:52.711 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-04-26T21:29:52.711 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/serio_raw.ko 2024-04-26T21:29:52.711 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/sparse-keymap.ko 2024-04-26T21:29:52.711 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds 2024-04-26T21:29:52.711 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-blinkm.ko 2024-04-26T21:29:52.711 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lm3530.ko 2024-04-26T21:29:52.711 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lp3944.ko 2024-04-26T21:29:52.711 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-ss4200.ko 2024-04-26T21:29:52.711 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger 2024-04-26T21:29:52.711 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-04-26T21:29:52.711 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-04-26T21:29:52.711 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-04-26T21:29:52.711 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-04-26T21:29:52.711 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-04-26T21:29:52.711 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-04-26T21:29:52.712 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-04-26T21:29:52.712 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-04-26T21:29:52.712 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-04-26T21:29:52.712 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md 2024-04-26T21:29:52.712 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache 2024-04-26T21:29:52.712 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache/bcache.ko 2024-04-26T21:29:52.712 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-bio-prison.ko 2024-04-26T21:29:52.712 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache-smq.ko 2024-04-26T21:29:52.712 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache.ko 2024-04-26T21:29:52.712 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-crypt.ko 2024-04-26T21:29:52.712 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-delay.ko 2024-04-26T21:29:52.712 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-flakey.ko 2024-04-26T21:29:52.712 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-log-userspace.ko 2024-04-26T21:29:52.712 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-multipath.ko 2024-04-26T21:29:52.712 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-queue-length.ko 2024-04-26T21:29:52.713 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-raid.ko 2024-04-26T21:29:52.713 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-round-robin.ko 2024-04-26T21:29:52.713 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-service-time.ko 2024-04-26T21:29:52.713 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-switch.ko 2024-04-26T21:29:52.713 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-thin-pool.ko 2024-04-26T21:29:52.713 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-verity.ko 2024-04-26T21:29:52.713 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data 2024-04-26T21:29:52.713 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-04-26T21:29:52.713 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid0.ko 2024-04-26T21:29:52.713 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid1.ko 2024-04-26T21:29:52.713 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid10.ko 2024-04-26T21:29:52.713 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid456.ko 2024-04-26T21:29:52.713 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message 2024-04-26T21:29:52.713 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion 2024-04-26T21:29:52.713 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptbase.ko 2024-04-26T21:29:52.714 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptctl.ko 2024-04-26T21:29:52.714 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptfc.ko 2024-04-26T21:29:52.714 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptlan.ko 2024-04-26T21:29:52.714 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptsas.ko 2024-04-26T21:29:52.714 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptscsih.ko 2024-04-26T21:29:52.714 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptspi.ko 2024-04-26T21:29:52.714 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd 2024-04-26T21:29:52.714 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_ich.ko 2024-04-26T21:29:52.714 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_sch.ko 2024-04-26T21:29:52.714 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/mfd-core.ko 2024-04-26T21:29:52.714 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/sm501.ko 2024-04-26T21:29:52.714 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/viperboard.ko 2024-04-26T21:29:52.714 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/vx855.ko 2024-04-26T21:29:52.714 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/wl1273-core.ko 2024-04-26T21:29:52.715 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc 2024-04-26T21:29:52.715 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl 2024-04-26T21:29:52.715 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-04-26T21:29:52.715 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds9802als.ko 2024-04-26T21:29:52.715 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds990x.ko 2024-04-26T21:29:52.715 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/bh1770glc.ko 2024-04-26T21:29:52.715 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710 2024-04-26T21:29:52.715 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710/cb710.ko 2024-04-26T21:29:52.715 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom 2024-04-26T21:29:52.715 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/at24.ko 2024-04-26T21:29:52.715 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-04-26T21:29:52.715 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/max6875.ko 2024-04-26T21:29:52.715 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/enclosure.ko 2024-04-26T21:29:52.715 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/hpilo.ko 2024-04-26T21:29:52.715 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29003.ko 2024-04-26T21:29:52.716 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29020.ko 2024-04-26T21:29:52.716 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d 2024-04-26T21:29:52.716 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-04-26T21:29:52.716 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-04-26T21:29:52.716 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei 2024-04-26T21:29:52.716 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-me.ko 2024-04-26T21:29:52.716 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-txe.ko 2024-04-26T21:29:52.716 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei.ko 2024-04-26T21:29:52.716 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru 2024-04-26T21:29:52.716 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru/gru.ko 2024-04-26T21:29:52.716 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp 2024-04-26T21:29:52.716 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xp.ko 2024-04-26T21:29:52.716 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpc.ko 2024-04-26T21:29:52.716 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-04-26T21:29:52.716 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_7xx1.ko 2024-04-26T21:29:52.717 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_core.ko 2024-04-26T21:29:52.717 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tsl2550.ko 2024-04-26T21:29:52.717 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_balloon.ko 2024-04-26T21:29:52.717 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci 2024-04-26T21:29:52.717 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-04-26T21:29:52.717 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net 2024-04-26T21:29:52.717 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding 2024-04-26T21:29:52.717 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding/bonding.ko 2024-04-26T21:29:52.717 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa 2024-04-26T21:29:52.717 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53 2024-04-26T21:29:52.717 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53/b53_common.ko 2024-04-26T21:29:52.717 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/bcm-sf2.ko 2024-04-26T21:29:52.717 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6060.ko 2024-04-26T21:29:52.717 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx 2024-04-26T21:29:52.717 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-04-26T21:29:52.717 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dummy.ko 2024-04-26T21:29:52.718 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/eql.ko 2024-04-26T21:29:52.718 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet 2024-04-26T21:29:52.718 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com 2024-04-26T21:29:52.718 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-04-26T21:29:52.718 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-04-26T21:29:52.718 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-04-26T21:29:52.718 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-04-26T21:29:52.718 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390 2024-04-26T21:29:52.718 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/8390.ko 2024-04-26T21:29:52.718 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-04-26T21:29:52.718 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-04-26T21:29:52.718 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-04-26T21:29:52.718 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec 2024-04-26T21:29:52.718 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-04-26T21:29:52.719 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere 2024-04-26T21:29:52.719 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere/et131x.ko 2024-04-26T21:29:52.719 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon 2024-04-26T21:29:52.719 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-04-26T21:29:52.719 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera 2024-04-26T21:29:52.719 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-04-26T21:29:52.719 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd 2024-04-26T21:29:52.719 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-04-26T21:29:52.719 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-04-26T21:29:52.719 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-04-26T21:29:52.719 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros 2024-04-26T21:29:52.719 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx 2024-04-26T21:29:52.719 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-04-26T21:29:52.719 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c 2024-04-26T21:29:52.719 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-04-26T21:29:52.719 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e 2024-04-26T21:29:52.720 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-04-26T21:29:52.720 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx 2024-04-26T21:29:52.720 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-04-26T21:29:52.720 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-04-26T21:29:52.720 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom 2024-04-26T21:29:52.720 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-04-26T21:29:52.720 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-04-26T21:29:52.720 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-04-26T21:29:52.720 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-04-26T21:29:52.720 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-04-26T21:29:52.720 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet 2024-04-26T21:29:52.720 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-04-26T21:29:52.720 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-04-26T21:29:52.720 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade 2024-04-26T21:29:52.720 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna 2024-04-26T21:29:52.721 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-04-26T21:29:52.721 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio 2024-04-26T21:29:52.721 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb 2024-04-26T21:29:52.721 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-04-26T21:29:52.721 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-04-26T21:29:52.721 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-04-26T21:29:52.721 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-04-26T21:29:52.721 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-04-26T21:29:52.721 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-04-26T21:29:52.721 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-04-26T21:29:52.721 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-04-26T21:29:52.721 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-04-26T21:29:52.721 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco 2024-04-26T21:29:52.721 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic 2024-04-26T21:29:52.721 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-04-26T21:29:52.722 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec 2024-04-26T21:29:52.722 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip 2024-04-26T21:29:52.722 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-04-26T21:29:52.722 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-04-26T21:29:52.722 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-04-26T21:29:52.722 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-04-26T21:29:52.722 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-04-26T21:29:52.722 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-04-26T21:29:52.722 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink 2024-04-26T21:29:52.722 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-04-26T21:29:52.722 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-04-26T21:29:52.722 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dnet.ko 2024-04-26T21:29:52.722 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex 2024-04-26T21:29:52.722 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet 2024-04-26T21:29:52.723 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-04-26T21:29:52.723 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ethoc.ko 2024-04-26T21:29:52.723 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/fealnx.ko 2024-04-26T21:29:52.723 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel 2024-04-26T21:29:52.723 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e100.ko 2024-04-26T21:29:52.723 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000 2024-04-26T21:29:52.723 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-04-26T21:29:52.723 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e 2024-04-26T21:29:52.723 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-04-26T21:29:52.723 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k 2024-04-26T21:29:52.723 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-04-26T21:29:52.723 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e 2024-04-26T21:29:52.723 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-04-26T21:29:52.723 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf 2024-04-26T21:29:52.724 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-04-26T21:29:52.724 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb 2024-04-26T21:29:52.724 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-04-26T21:29:52.724 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf 2024-04-26T21:29:52.724 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-04-26T21:29:52.724 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe 2024-04-26T21:29:52.724 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-04-26T21:29:52.724 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf 2024-04-26T21:29:52.724 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-04-26T21:29:52.724 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/jme.ko 2024-04-26T21:29:52.724 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell 2024-04-26T21:29:52.724 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-04-26T21:29:52.724 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/skge.ko 2024-04-26T21:29:52.724 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-04-26T21:29:52.724 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox 2024-04-26T21:29:52.725 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4 2024-04-26T21:29:52.725 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-04-26T21:29:52.725 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-04-26T21:29:52.725 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5 2024-04-26T21:29:52.725 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-04-26T21:29:52.725 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-04-26T21:29:52.725 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel 2024-04-26T21:29:52.725 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-04-26T21:29:52.725 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom 2024-04-26T21:29:52.725 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge 2024-04-26T21:29:52.725 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-04-26T21:29:52.725 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi 2024-04-26T21:29:52.725 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-04-26T21:29:52.725 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-04-26T21:29:52.726 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion 2024-04-26T21:29:52.726 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-04-26T21:29:52.726 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia 2024-04-26T21:29:52.726 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-04-26T21:29:52.726 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines 2024-04-26T21:29:52.726 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-04-26T21:29:52.726 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-04-26T21:29:52.726 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic 2024-04-26T21:29:52.726 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen 2024-04-26T21:29:52.726 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-04-26T21:29:52.726 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-04-26T21:29:52.726 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-04-26T21:29:52.726 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-04-26T21:29:52.726 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc 2024-04-26T21:29:52.727 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-04-26T21:29:52.727 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek 2024-04-26T21:29:52.727 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-04-26T21:29:52.727 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-04-26T21:29:52.727 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/atp.ko 2024-04-26T21:29:52.727 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-04-26T21:29:52.727 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker 2024-04-26T21:29:52.727 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-04-26T21:29:52.727 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc 2024-04-26T21:29:52.727 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-04-26T21:29:52.727 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan 2024-04-26T21:29:52.727 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-04-26T21:29:52.727 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis 2024-04-26T21:29:52.727 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis190.ko 2024-04-26T21:29:52.728 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis900.ko 2024-04-26T21:29:52.728 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc 2024-04-26T21:29:52.728 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-04-26T21:29:52.728 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-04-26T21:29:52.728 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-04-26T21:29:52.728 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-04-26T21:29:52.728 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro 2024-04-26T21:29:52.728 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac 2024-04-26T21:29:52.728 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-04-26T21:29:52.728 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-04-26T21:29:52.728 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun 2024-04-26T21:29:52.728 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/cassini.ko 2024-04-26T21:29:52.728 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/niu.ko 2024-04-26T21:29:52.728 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sungem.ko 2024-04-26T21:29:52.728 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-04-26T21:29:52.729 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti 2024-04-26T21:29:52.729 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-04-26T21:29:52.729 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti 2024-04-26T21:29:52.729 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti/tlan.ko 2024-04-26T21:29:52.729 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via 2024-04-26T21:29:52.729 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-04-26T21:29:52.729 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-04-26T21:29:52.729 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet 2024-04-26T21:29:52.729 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-04-26T21:29:52.729 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-04-26T21:29:52.729 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom 2024-04-26T21:29:52.729 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-04-26T21:29:52.729 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/geneve.ko 2024-04-26T21:29:52.729 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv 2024-04-26T21:29:52.730 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-04-26T21:29:52.730 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154 2024-04-26T21:29:52.730 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154/fakelb.ko 2024-04-26T21:29:52.730 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ifb.ko 2024-04-26T21:29:52.730 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan 2024-04-26T21:29:52.730 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan/ipvlan.ko 2024-04-26T21:29:52.730 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvlan.ko 2024-04-26T21:29:52.730 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvtap.ko 2024-04-26T21:29:52.730 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio 2024-04-26T21:29:52.730 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio.ko 2024-04-26T21:29:52.730 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-04-26T21:29:52.730 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-04-26T21:29:52.730 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-regmap.ko 2024-04-26T21:29:52.730 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mii.ko 2024-04-26T21:29:52.731 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/net_failover.ko 2024-04-26T21:29:52.731 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/netconsole.ko 2024-04-26T21:29:52.731 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/nlmon.ko 2024-04-26T21:29:52.731 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs 2024-04-26T21:29:52.731 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs-lynx.ko 2024-04-26T21:29:52.731 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-04-26T21:29:52.731 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy 2024-04-26T21:29:52.731 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/amd.ko 2024-04-26T21:29:52.731 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-04-26T21:29:52.731 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-04-26T21:29:52.731 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm7xxx.ko 2024-04-26T21:29:52.731 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm87xx.ko 2024-04-26T21:29:52.731 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/broadcom.ko 2024-04-26T21:29:52.731 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/cicada.ko 2024-04-26T21:29:52.731 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/davicom.ko 2024-04-26T21:29:52.732 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/dp83640.ko 2024-04-26T21:29:52.732 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/et1011c.ko 2024-04-26T21:29:52.732 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/icplus.ko 2024-04-26T21:29:52.732 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/lxt.ko 2024-04-26T21:29:52.732 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/marvell.ko 2024-04-26T21:29:52.732 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/micrel.ko 2024-04-26T21:29:52.732 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/national.ko 2024-04-26T21:29:52.732 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/phylink.ko 2024-04-26T21:29:52.732 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/qsemi.ko 2024-04-26T21:29:52.732 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/realtek.ko 2024-04-26T21:29:52.732 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/smsc.ko 2024-04-26T21:29:52.732 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/ste10Xp.ko 2024-04-26T21:29:52.732 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/vitesse.ko 2024-04-26T21:29:52.732 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp 2024-04-26T21:29:52.732 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/bsd_comp.ko 2024-04-26T21:29:52.732 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_async.ko 2024-04-26T21:29:52.733 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_deflate.ko 2024-04-26T21:29:52.733 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_generic.ko 2024-04-26T21:29:52.733 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_mppe.ko 2024-04-26T21:29:52.733 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_synctty.ko 2024-04-26T21:29:52.733 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppoe.ko 2024-04-26T21:29:52.733 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppox.ko 2024-04-26T21:29:52.733 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pptp.ko 2024-04-26T21:29:52.733 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip 2024-04-26T21:29:52.733 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slhc.ko 2024-04-26T21:29:52.733 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slip.ko 2024-04-26T21:29:52.733 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/sungem_phy.ko 2024-04-26T21:29:52.733 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tap.ko 2024-04-26T21:29:52.733 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team 2024-04-26T21:29:52.733 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team.ko 2024-04-26T21:29:52.733 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_activebackup.ko 2024-04-26T21:29:52.734 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_broadcast.ko 2024-04-26T21:29:52.734 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-04-26T21:29:52.734 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_random.ko 2024-04-26T21:29:52.734 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-04-26T21:29:52.734 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tun.ko 2024-04-26T21:29:52.734 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/veth.ko 2024-04-26T21:29:52.734 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/virtio_net.ko 2024-04-26T21:29:52.734 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3 2024-04-26T21:29:52.734 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-04-26T21:29:52.734 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan 2024-04-26T21:29:52.734 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan/vxlan.ko 2024-04-26T21:29:52.734 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback 2024-04-26T21:29:52.734 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback/xen-netback.ko 2024-04-26T21:29:52.734 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netfront.ko 2024-04-26T21:29:52.735 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb 2024-04-26T21:29:52.735 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb/ntb.ko 2024-04-26T21:29:52.735 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme 2024-04-26T21:29:52.735 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host 2024-04-26T21:29:52.735 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme-core.ko 2024-04-26T21:29:52.735 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme.ko 2024-04-26T21:29:52.735 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport 2024-04-26T21:29:52.735 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport.ko 2024-04-26T21:29:52.735 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_cs.ko 2024-04-26T21:29:52.735 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_pc.ko 2024-04-26T21:29:52.735 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_serial.ko 2024-04-26T21:29:52.735 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci 2024-04-26T21:29:52.735 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug 2024-04-26T21:29:52.735 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-04-26T21:29:52.735 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie 2024-04-26T21:29:52.735 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie/aer_inject.ko 2024-04-26T21:29:52.735 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/xen-pcifront.ko 2024-04-26T21:29:52.736 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia 2024-04-26T21:29:52.736 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/i82092.ko 2024-04-26T21:29:52.736 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/pd6729.ko 2024-04-26T21:29:52.736 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/yenta_socket.ko 2024-04-26T21:29:52.736 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl 2024-04-26T21:29:52.736 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel 2024-04-26T21:29:52.736 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-04-26T21:29:52.736 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform 2024-04-26T21:29:52.736 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86 2024-04-26T21:29:52.736 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acer-wmi.ko 2024-04-26T21:29:52.736 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acerhdf.ko 2024-04-26T21:29:52.736 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/apple-gmux.ko 2024-04-26T21:29:52.736 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-laptop.ko 2024-04-26T21:29:52.736 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-04-26T21:29:52.736 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-wmi.ko 2024-04-26T21:29:52.736 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/classmate-laptop.ko 2024-04-26T21:29:52.737 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-04-26T21:29:52.737 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-04-26T21:29:52.737 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-04-26T21:29:52.737 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-04-26T21:29:52.737 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/hdaps.ko 2024-04-26T21:29:52.737 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel 2024-04-26T21:29:52.737 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-04-26T21:29:52.737 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel_ips.ko 2024-04-26T21:29:52.737 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/msi-wmi.ko 2024-04-26T21:29:52.737 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/mxm-wmi.ko 2024-04-26T21:29:52.737 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-04-26T21:29:52.737 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-laptop.ko 2024-04-26T21:29:52.737 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-q10.ko 2024-04-26T21:29:52.737 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/topstar-laptop.ko 2024-04-26T21:29:52.737 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-04-26T21:29:52.737 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_haps.ko 2024-04-26T21:29:52.738 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi-bmof.ko 2024-04-26T21:29:52.738 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi.ko 2024-04-26T21:29:52.738 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap 2024-04-26T21:29:52.738 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_common.ko 2024-04-26T21:29:52.738 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_msr.ko 2024-04-26T21:29:52.738 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps 2024-04-26T21:29:52.738 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients 2024-04-26T21:29:52.738 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-gpio.ko 2024-04-26T21:29:52.738 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-ldisc.ko 2024-04-26T21:29:52.738 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps_parport.ko 2024-04-26T21:29:52.738 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/pps_core.ko 2024-04-26T21:29:52.738 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp 2024-04-26T21:29:52.738 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp.ko 2024-04-26T21:29:52.738 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp_kvm.ko 2024-04-26T21:29:52.739 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras 2024-04-26T21:29:52.739 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd 2024-04-26T21:29:52.739 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl 2024-04-26T21:29:52.739 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-04-26T21:29:52.739 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/fmpm.ko 2024-04-26T21:29:52.739 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc 2024-04-26T21:29:52.739 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-bq32k.ko 2024-04-26T21:29:52.739 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1286.ko 2024-04-26T21:29:52.739 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1307.ko 2024-04-26T21:29:52.739 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1374.ko 2024-04-26T21:29:52.739 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1511.ko 2024-04-26T21:29:52.739 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1553.ko 2024-04-26T21:29:52.739 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1672.ko 2024-04-26T21:29:52.739 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1742.ko 2024-04-26T21:29:52.739 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds2404.ko 2024-04-26T21:29:52.740 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds3232.ko 2024-04-26T21:29:52.740 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-em3027.ko 2024-04-26T21:29:52.740 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-fm3130.ko 2024-04-26T21:29:52.740 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl12022.ko 2024-04-26T21:29:52.740 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl1208.ko 2024-04-26T21:29:52.740 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m41t80.ko 2024-04-26T21:29:52.740 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t35.ko 2024-04-26T21:29:52.740 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t59.ko 2024-04-26T21:29:52.740 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-max6900.ko 2024-04-26T21:29:52.740 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-msm6242.ko 2024-04-26T21:29:52.740 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf2127.ko 2024-04-26T21:29:52.740 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf85063.ko 2024-04-26T21:29:52.740 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8523.ko 2024-04-26T21:29:52.740 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8563.ko 2024-04-26T21:29:52.740 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8583.ko 2024-04-26T21:29:52.740 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rp5c01.ko 2024-04-26T21:29:52.740 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rs5c372.ko 2024-04-26T21:29:52.741 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-04-26T21:29:52.741 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8025.ko 2024-04-26T21:29:52.741 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8581.ko 2024-04-26T21:29:52.741 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-04-26T21:29:52.741 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-x1205.ko 2024-04-26T21:29:52.741 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi 2024-04-26T21:29:52.741 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-9xxx.ko 2024-04-26T21:29:52.741 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-sas.ko 2024-04-26T21:29:52.741 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-xxxx.ko 2024-04-26T21:29:52.741 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/BusLogic.ko 2024-04-26T21:29:52.741 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/a100u2w.ko 2024-04-26T21:29:52.741 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid 2024-04-26T21:29:52.741 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid/aacraid.ko 2024-04-26T21:29:52.741 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/advansys.ko 2024-04-26T21:29:52.741 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx 2024-04-26T21:29:52.741 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-04-26T21:29:52.741 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-04-26T21:29:52.742 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/am53c974.ko 2024-04-26T21:29:52.742 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr 2024-04-26T21:29:52.742 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-04-26T21:29:52.742 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/atp870u.ko 2024-04-26T21:29:52.742 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi 2024-04-26T21:29:52.742 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-04-26T21:29:52.742 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa 2024-04-26T21:29:52.742 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa/bfa.ko 2024-04-26T21:29:52.742 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc 2024-04-26T21:29:52.742 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-04-26T21:29:52.742 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i 2024-04-26T21:29:52.742 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-04-26T21:29:52.742 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ch.ko 2024-04-26T21:29:52.742 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor 2024-04-26T21:29:52.742 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor/csiostor.ko 2024-04-26T21:29:52.743 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi 2024-04-26T21:29:52.743 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i 2024-04-26T21:29:52.743 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-04-26T21:29:52.743 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i 2024-04-26T21:29:52.743 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-04-26T21:29:52.743 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-04-26T21:29:52.743 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dc395x.ko 2024-04-26T21:29:52.743 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler 2024-04-26T21:29:52.743 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-04-26T21:29:52.743 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-04-26T21:29:52.743 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-04-26T21:29:52.743 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-04-26T21:29:52.743 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dmx3191d.ko 2024-04-26T21:29:52.743 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r 2024-04-26T21:29:52.743 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r/esas2r.ko 2024-04-26T21:29:52.744 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esp_scsi.ko 2024-04-26T21:29:52.744 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe 2024-04-26T21:29:52.744 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/fcoe.ko 2024-04-26T21:29:52.744 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-04-26T21:29:52.744 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic 2024-04-26T21:29:52.744 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic/fnic.ko 2024-04-26T21:29:52.744 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hpsa.ko 2024-04-26T21:29:52.744 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hptiop.ko 2024-04-26T21:29:52.744 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hv_storvsc.ko 2024-04-26T21:29:52.744 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/initio.ko 2024-04-26T21:29:52.744 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ipr.ko 2024-04-26T21:29:52.744 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ips.ko 2024-04-26T21:29:52.744 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci 2024-04-26T21:29:52.744 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci/isci.ko 2024-04-26T21:29:52.744 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-04-26T21:29:52.744 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_tcp.ko 2024-04-26T21:29:52.745 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc 2024-04-26T21:29:52.745 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc/libfc.ko 2024-04-26T21:29:52.745 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi.ko 2024-04-26T21:29:52.745 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi_tcp.ko 2024-04-26T21:29:52.745 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas 2024-04-26T21:29:52.745 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas/libsas.ko 2024-04-26T21:29:52.745 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc 2024-04-26T21:29:52.745 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc/lpfc.ko 2024-04-26T21:29:52.745 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid 2024-04-26T21:29:52.745 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid.ko 2024-04-26T21:29:52.745 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-04-26T21:29:52.745 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-04-26T21:29:52.745 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-04-26T21:29:52.745 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas 2024-04-26T21:29:52.745 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-04-26T21:29:52.745 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas 2024-04-26T21:29:52.746 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas/mvsas.ko 2024-04-26T21:29:52.746 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvumi.ko 2024-04-26T21:29:52.746 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001 2024-04-26T21:29:52.746 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-04-26T21:29:52.746 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pmcraid.ko 2024-04-26T21:29:52.746 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla1280.ko 2024-04-26T21:29:52.746 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx 2024-04-26T21:29:52.746 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-04-26T21:29:52.746 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-04-26T21:29:52.746 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx 2024-04-26T21:29:52.746 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-04-26T21:29:52.746 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/raid_class.ko 2024-04-26T21:29:52.746 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_debug.ko 2024-04-26T21:29:52.746 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_fc.ko 2024-04-26T21:29:52.746 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-04-26T21:29:52.746 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_sas.ko 2024-04-26T21:29:52.747 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_spi.ko 2024-04-26T21:29:52.747 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_srp.ko 2024-04-26T21:29:52.747 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ses.ko 2024-04-26T21:29:52.747 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/st.ko 2024-04-26T21:29:52.747 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/stex.ko 2024-04-26T21:29:52.747 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2 2024-04-26T21:29:52.747 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-04-26T21:29:52.747 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/virtio_scsi.ko 2024-04-26T21:29:52.747 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/vmw_pvscsi.ko 2024-04-26T21:29:52.747 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/wd719x.ko 2024-04-26T21:29:52.747 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb 2024-04-26T21:29:52.747 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb/ssb.ko 2024-04-26T21:29:52.747 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target 2024-04-26T21:29:52.747 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi 2024-04-26T21:29:52.747 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-04-26T21:29:52.747 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback 2024-04-26T21:29:52.748 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback/tcm_loop.ko 2024-04-26T21:29:52.748 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_file.ko 2024-04-26T21:29:52.748 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_iblock.ko 2024-04-26T21:29:52.748 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_mod.ko 2024-04-26T21:29:52.748 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_pscsi.ko 2024-04-26T21:29:52.748 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_user.ko 2024-04-26T21:29:52.748 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc 2024-04-26T21:29:52.748 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-04-26T21:29:52.748 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal 2024-04-26T21:29:52.748 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel 2024-04-26T21:29:52.748 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal 2024-04-26T21:29:52.748 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-04-26T21:29:52.748 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-04-26T21:29:52.748 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-04-26T21:29:52.748 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-04-26T21:29:52.748 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-04-26T21:29:52.748 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-04-26T21:29:52.749 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-04-26T21:29:52.749 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-04-26T21:29:52.749 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-04-26T21:29:52.749 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-04-26T21:29:52.749 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-04-26T21:29:52.749 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-04-26T21:29:52.749 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-04-26T21:29:52.749 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-04-26T21:29:52.749 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-04-26T21:29:52.749 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-04-26T21:29:52.749 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-04-26T21:29:52.749 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-04-26T21:29:52.749 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty 2024-04-26T21:29:52.749 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless 2024-04-26T21:29:52.749 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-04-26T21:29:52.749 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_gsm.ko 2024-04-26T21:29:52.750 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_hdlc.ko 2024-04-26T21:29:52.750 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/nozomi.ko 2024-04-26T21:29:52.750 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial 2024-04-26T21:29:52.750 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250 2024-04-26T21:29:52.750 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-04-26T21:29:52.750 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/arc_uart.ko 2024-04-26T21:29:52.750 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm 2024-04-26T21:29:52.750 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm/jsm.ko 2024-04-26T21:29:52.750 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs 2024-04-26T21:29:52.750 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core 2024-04-26T21:29:52.750 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core/ufshcd-core.ko 2024-04-26T21:29:52.750 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host 2024-04-26T21:29:52.750 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-04-26T21:29:52.750 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio 2024-04-26T21:29:52.750 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio.ko 2024-04-26T21:29:52.751 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_aec.ko 2024-04-26T21:29:52.751 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_cif.ko 2024-04-26T21:29:52.751 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_pci_generic.ko 2024-04-26T21:29:52.751 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_sercos3.ko 2024-04-26T21:29:52.751 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb 2024-04-26T21:29:52.751 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class 2024-04-26T21:29:52.751 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-acm.ko 2024-04-26T21:29:52.751 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-wdm.ko 2024-04-26T21:29:52.751 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usblp.ko 2024-04-26T21:29:52.751 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usbtmc.ko 2024-04-26T21:29:52.751 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host 2024-04-26T21:29:52.751 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/sl811-hcd.ko 2024-04-26T21:29:52.751 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image 2024-04-26T21:29:52.751 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/mdc800.ko 2024-04-26T21:29:52.751 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/microtek.ko 2024-04-26T21:29:52.751 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc 2024-04-26T21:29:52.752 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/adutux.ko 2024-04-26T21:29:52.752 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/appledisplay.ko 2024-04-26T21:29:52.752 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi26.ko 2024-04-26T21:29:52.752 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi62.ko 2024-04-26T21:29:52.752 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ezusb.ko 2024-04-26T21:29:52.752 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/idmouse.ko 2024-04-26T21:29:52.752 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/iowarrior.ko 2024-04-26T21:29:52.752 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/isight_firmware.ko 2024-04-26T21:29:52.752 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ldusb.ko 2024-04-26T21:29:52.752 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/legousbtower.ko 2024-04-26T21:29:52.752 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga 2024-04-26T21:29:52.752 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-04-26T21:29:52.752 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/trancevibrator.ko 2024-04-26T21:29:52.752 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usb3503.ko 2024-04-26T21:29:52.752 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usblcd.ko 2024-04-26T21:29:52.752 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usbsevseg.ko 2024-04-26T21:29:52.753 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/uss720.ko 2024-04-26T21:29:52.753 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/yurex.ko 2024-04-26T21:29:52.753 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy 2024-04-26T21:29:52.753 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy/phy-generic.ko 2024-04-26T21:29:52.753 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial 2024-04-26T21:29:52.753 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/aircable.ko 2024-04-26T21:29:52.753 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ark3116.ko 2024-04-26T21:29:52.753 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/belkin_sa.ko 2024-04-26T21:29:52.753 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ch341.ko 2024-04-26T21:29:52.753 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cp210x.ko 2024-04-26T21:29:52.753 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cyberjack.ko 2024-04-26T21:29:52.753 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cypress_m8.ko 2024-04-26T21:29:52.753 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/digi_acceleport.ko 2024-04-26T21:29:52.753 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/empeg.ko 2024-04-26T21:29:52.753 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ftdi_sio.ko 2024-04-26T21:29:52.753 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/garmin_gps.ko 2024-04-26T21:29:52.753 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_edgeport.ko 2024-04-26T21:29:52.754 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_ti.ko 2024-04-26T21:29:52.754 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipaq.ko 2024-04-26T21:29:52.754 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipw.ko 2024-04-26T21:29:52.754 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ir-usb.ko 2024-04-26T21:29:52.754 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-04-26T21:29:52.754 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan.ko 2024-04-26T21:29:52.754 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan_pda.ko 2024-04-26T21:29:52.754 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kl5kusb105.ko 2024-04-26T21:29:52.754 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kobil_sct.ko 2024-04-26T21:29:52.754 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mct_u232.ko 2024-04-26T21:29:52.754 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7720.ko 2024-04-26T21:29:52.754 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7840.ko 2024-04-26T21:29:52.754 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/navman.ko 2024-04-26T21:29:52.754 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/omninet.ko 2024-04-26T21:29:52.755 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/opticon.ko 2024-04-26T21:29:52.755 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/option.ko 2024-04-26T21:29:52.755 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/oti6858.ko 2024-04-26T21:29:52.755 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/pl2303.ko 2024-04-26T21:29:52.755 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcaux.ko 2024-04-26T21:29:52.755 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcserial.ko 2024-04-26T21:29:52.755 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/quatech2.ko 2024-04-26T21:29:52.755 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/safe_serial.ko 2024-04-26T21:29:52.755 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/sierra.ko 2024-04-26T21:29:52.755 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/spcp8x5.ko 2024-04-26T21:29:52.755 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ssu100.ko 2024-04-26T21:29:52.755 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/symbolserial.ko 2024-04-26T21:29:52.755 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-04-26T21:29:52.755 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-04-26T21:29:52.755 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_debug.ko 2024-04-26T21:29:52.755 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_wwan.ko 2024-04-26T21:29:52.756 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/visor.ko 2024-04-26T21:29:52.756 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/whiteheat.ko 2024-04-26T21:29:52.756 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/xsens_mt.ko 2024-04-26T21:29:52.756 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage 2024-04-26T21:29:52.756 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/uas.ko 2024-04-26T21:29:52.756 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-alauda.ko 2024-04-26T21:29:52.756 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-cypress.ko 2024-04-26T21:29:52.756 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-datafab.ko 2024-04-26T21:29:52.756 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-04-26T21:29:52.756 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-freecom.ko 2024-04-26T21:29:52.756 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-isd200.ko 2024-04-26T21:29:52.756 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-04-26T21:29:52.756 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-karma.ko 2024-04-26T21:29:52.756 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-onetouch.ko 2024-04-26T21:29:52.756 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-realtek.ko 2024-04-26T21:29:52.756 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr09.ko 2024-04-26T21:29:52.757 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr55.ko 2024-04-26T21:29:52.757 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-usbat.ko 2024-04-26T21:29:52.757 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/usb-storage.ko 2024-04-26T21:29:52.757 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip 2024-04-26T21:29:52.757 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-core.ko 2024-04-26T21:29:52.757 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-host.ko 2024-04-26T21:29:52.757 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-04-26T21:29:52.757 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio 2024-04-26T21:29:52.757 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci 2024-04-26T21:29:52.757 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-04-26T21:29:52.757 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci.ko 2024-04-26T21:29:52.757 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio.ko 2024-04-26T21:29:52.757 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-04-26T21:29:52.757 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost 2024-04-26T21:29:52.757 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost.ko 2024-04-26T21:29:52.758 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_iotlb.ko 2024-04-26T21:29:52.758 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_net.ko 2024-04-26T21:29:52.758 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_scsi.ko 2024-04-26T21:29:52.758 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video 2024-04-26T21:29:52.758 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight 2024-04-26T21:29:52.758 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/apple_bl.ko 2024-04-26T21:29:52.758 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/lcd.ko 2024-04-26T21:29:52.758 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/platform_lcd.ko 2024-04-26T21:29:52.758 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio 2024-04-26T21:29:52.758 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio.ko 2024-04-26T21:29:52.758 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_balloon.ko 2024-04-26T21:29:52.758 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_mmio.ko 2024-04-26T21:29:52.758 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci.ko 2024-04-26T21:29:52.758 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-04-26T21:29:52.759 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-04-26T21:29:52.759 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_ring.ko 2024-04-26T21:29:52.759 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1 2024-04-26T21:29:52.759 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters 2024-04-26T21:29:52.759 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2482.ko 2024-04-26T21:29:52.759 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2490.ko 2024-04-26T21:29:52.759 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves 2024-04-26T21:29:52.759 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-04-26T21:29:52.759 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-04-26T21:29:52.759 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-04-26T21:29:52.759 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-04-26T21:29:52.759 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-04-26T21:29:52.759 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-04-26T21:29:52.759 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-04-26T21:29:52.760 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-04-26T21:29:52.760 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-04-26T21:29:52.760 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_smem.ko 2024-04-26T21:29:52.760 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_therm.ko 2024-04-26T21:29:52.760 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/wire.ko 2024-04-26T21:29:52.760 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog 2024-04-26T21:29:52.760 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim1535_wdt.ko 2024-04-26T21:29:52.760 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim7101_wdt.ko 2024-04-26T21:29:52.760 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/f71808e_wdt.ko 2024-04-26T21:29:52.760 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/hpwdt.ko 2024-04-26T21:29:52.760 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/i6300esb.ko 2024-04-26T21:29:52.760 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-04-26T21:29:52.760 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_wdt.ko 2024-04-26T21:29:52.760 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ib700wdt.ko 2024-04-26T21:29:52.760 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ibmasr.ko 2024-04-26T21:29:52.761 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-04-26T21:29:52.761 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it8712f_wdt.ko 2024-04-26T21:29:52.761 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it87_wdt.ko 2024-04-26T21:29:52.761 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/machzwd.ko 2024-04-26T21:29:52.761 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/nv_tco.ko 2024-04-26T21:29:52.761 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_pci.ko 2024-04-26T21:29:52.761 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_usb.ko 2024-04-26T21:29:52.761 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-04-26T21:29:52.761 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sch311x_wdt.ko 2024-04-26T21:29:52.761 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/softdog.ko 2024-04-26T21:29:52.761 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sp5100_tco.ko 2024-04-26T21:29:52.761 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/via_wdt.ko 2024-04-26T21:29:52.761 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-04-26T21:29:52.761 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83877f_wdt.ko 2024-04-26T21:29:52.761 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83977f_wdt.ko 2024-04-26T21:29:52.762 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/wdt_pci.ko 2024-04-26T21:29:52.762 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/xen_wdt.ko 2024-04-26T21:29:52.762 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen 2024-04-26T21:29:52.762 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-acpi-processor.ko 2024-04-26T21:29:52.762 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-evtchn.ko 2024-04-26T21:29:52.762 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntalloc.ko 2024-04-26T21:29:52.762 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntdev.ko 2024-04-26T21:29:52.762 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback 2024-04-26T21:29:52.762 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-04-26T21:29:52.762 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-privcmd.ko 2024-04-26T21:29:52.762 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs 2024-04-26T21:29:52.762 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs/xenfs.ko 2024-04-26T21:29:52.762 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs 2024-04-26T21:29:52.762 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p 2024-04-26T21:29:52.763 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p/9p.ko 2024-04-26T21:29:52.763 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs 2024-04-26T21:29:52.763 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs/affs.ko 2024-04-26T21:29:52.763 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs 2024-04-26T21:29:52.763 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs/befs.ko 2024-04-26T21:29:52.763 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/binfmt_misc.ko 2024-04-26T21:29:52.763 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs 2024-04-26T21:29:52.763 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs/btrfs.ko 2024-04-26T21:29:52.763 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles 2024-04-26T21:29:52.763 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles/cachefiles.ko 2024-04-26T21:29:52.763 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph 2024-04-26T21:29:52.763 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph/ceph.ko 2024-04-26T21:29:52.763 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda 2024-04-26T21:29:52.763 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda/coda.ko 2024-04-26T21:29:52.763 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs 2024-04-26T21:29:52.763 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs/cramfs.ko 2024-04-26T21:29:52.764 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm 2024-04-26T21:29:52.764 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm/dlm.ko 2024-04-26T21:29:52.764 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs 2024-04-26T21:29:52.764 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs/ecryptfs.ko 2024-04-26T21:29:52.764 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs 2024-04-26T21:29:52.764 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs/f2fs.ko 2024-04-26T21:29:52.764 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat 2024-04-26T21:29:52.764 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/fat.ko 2024-04-26T21:29:52.764 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/msdos.ko 2024-04-26T21:29:52.764 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/vfat.ko 2024-04-26T21:29:52.764 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse 2024-04-26T21:29:52.764 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/cuse.ko 2024-04-26T21:29:52.764 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/fuse.ko 2024-04-26T21:29:52.764 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2 2024-04-26T21:29:52.764 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2/gfs2.ko 2024-04-26T21:29:52.765 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs 2024-04-26T21:29:52.765 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs/hfs.ko 2024-04-26T21:29:52.765 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus 2024-04-26T21:29:52.765 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus/hfsplus.ko 2024-04-26T21:29:52.765 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs 2024-04-26T21:29:52.765 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs/isofs.ko 2024-04-26T21:29:52.765 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs 2024-04-26T21:29:52.765 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs/jfs.ko 2024-04-26T21:29:52.765 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd 2024-04-26T21:29:52.765 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd/lockd.ko 2024-04-26T21:29:52.765 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix 2024-04-26T21:29:52.765 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix/minix.ko 2024-04-26T21:29:52.765 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs 2024-04-26T21:29:52.765 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs/netfs.ko 2024-04-26T21:29:52.765 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs 2024-04-26T21:29:52.766 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout 2024-04-26T21:29:52.766 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-04-26T21:29:52.766 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout 2024-04-26T21:29:52.766 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-04-26T21:29:52.766 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout 2024-04-26T21:29:52.766 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-04-26T21:29:52.766 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfs.ko 2024-04-26T21:29:52.766 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv3.ko 2024-04-26T21:29:52.766 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv4.ko 2024-04-26T21:29:52.766 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common 2024-04-26T21:29:52.766 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/grace.ko 2024-04-26T21:29:52.766 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/nfs_acl.ko 2024-04-26T21:29:52.766 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd 2024-04-26T21:29:52.767 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd/nfsd.ko 2024-04-26T21:29:52.767 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2 2024-04-26T21:29:52.767 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2/nilfs2.ko 2024-04-26T21:29:52.767 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls 2024-04-26T21:29:52.767 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-celtic.ko 2024-04-26T21:29:52.767 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-centeuro.ko 2024-04-26T21:29:52.767 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-croatian.ko 2024-04-26T21:29:52.767 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-cyrillic.ko 2024-04-26T21:29:52.767 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-gaelic.ko 2024-04-26T21:29:52.767 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-greek.ko 2024-04-26T21:29:52.767 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-iceland.ko 2024-04-26T21:29:52.767 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-inuit.ko 2024-04-26T21:29:52.767 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-roman.ko 2024-04-26T21:29:52.767 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-romanian.ko 2024-04-26T21:29:52.767 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-turkish.ko 2024-04-26T21:29:52.767 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1250.ko 2024-04-26T21:29:52.768 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1251.ko 2024-04-26T21:29:52.768 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1255.ko 2024-04-26T21:29:52.768 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp737.ko 2024-04-26T21:29:52.768 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp775.ko 2024-04-26T21:29:52.768 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp850.ko 2024-04-26T21:29:52.768 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp852.ko 2024-04-26T21:29:52.768 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp855.ko 2024-04-26T21:29:52.768 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp857.ko 2024-04-26T21:29:52.768 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp860.ko 2024-04-26T21:29:52.768 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp861.ko 2024-04-26T21:29:52.768 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp862.ko 2024-04-26T21:29:52.768 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp863.ko 2024-04-26T21:29:52.768 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp864.ko 2024-04-26T21:29:52.768 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp865.ko 2024-04-26T21:29:52.768 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp866.ko 2024-04-26T21:29:52.768 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp869.ko 2024-04-26T21:29:52.769 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp874.ko 2024-04-26T21:29:52.769 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp932.ko 2024-04-26T21:29:52.769 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp936.ko 2024-04-26T21:29:52.769 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp949.ko 2024-04-26T21:29:52.769 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp950.ko 2024-04-26T21:29:52.769 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_euc-jp.ko 2024-04-26T21:29:52.769 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-1.ko 2024-04-26T21:29:52.769 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-13.ko 2024-04-26T21:29:52.769 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-14.ko 2024-04-26T21:29:52.769 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-15.ko 2024-04-26T21:29:52.769 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-2.ko 2024-04-26T21:29:52.769 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-3.ko 2024-04-26T21:29:52.769 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-4.ko 2024-04-26T21:29:52.769 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-5.ko 2024-04-26T21:29:52.769 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-6.ko 2024-04-26T21:29:52.769 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-7.ko 2024-04-26T21:29:52.769 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-9.ko 2024-04-26T21:29:52.770 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-r.ko 2024-04-26T21:29:52.770 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-ru.ko 2024-04-26T21:29:52.770 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-u.ko 2024-04-26T21:29:52.770 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_ucs2_utils.ko 2024-04-26T21:29:52.770 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_utf8.ko 2024-04-26T21:29:52.770 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2 2024-04-26T21:29:52.770 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster 2024-04-26T21:29:52.770 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-04-26T21:29:52.770 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm 2024-04-26T21:29:52.770 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-04-26T21:29:52.770 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs 2024-04-26T21:29:52.770 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-04-26T21:29:52.770 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2.ko 2024-04-26T21:29:52.770 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-04-26T21:29:52.771 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-04-26T21:29:52.771 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-04-26T21:29:52.771 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs 2024-04-26T21:29:52.771 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs/overlay.ko 2024-04-26T21:29:52.771 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore 2024-04-26T21:29:52.771 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore/ramoops.ko 2024-04-26T21:29:52.771 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs 2024-04-26T21:29:52.771 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs/reiserfs.ko 2024-04-26T21:29:52.771 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs 2024-04-26T21:29:52.771 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs/romfs.ko 2024-04-26T21:29:52.771 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb 2024-04-26T21:29:52.771 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client 2024-04-26T21:29:52.771 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client/cifs.ko 2024-04-26T21:29:52.771 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common 2024-04-26T21:29:52.771 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_arc4.ko 2024-04-26T21:29:52.772 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_md4.ko 2024-04-26T21:29:52.772 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs 2024-04-26T21:29:52.772 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs/squashfs.ko 2024-04-26T21:29:52.772 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv 2024-04-26T21:29:52.772 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv/sysv.ko 2024-04-26T21:29:52.772 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf 2024-04-26T21:29:52.772 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf/udf.ko 2024-04-26T21:29:52.772 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs 2024-04-26T21:29:52.772 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs/ufs.ko 2024-04-26T21:29:52.772 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs 2024-04-26T21:29:52.772 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs/xfs.ko 2024-04-26T21:29:52.772 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel 2024-04-26T21:29:52.772 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu 2024-04-26T21:29:52.772 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu/rcutorture.ko 2024-04-26T21:29:52.772 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/torture.ko 2024-04-26T21:29:52.772 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace 2024-04-26T21:29:52.773 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-04-26T21:29:52.773 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib 2024-04-26T21:29:52.773 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/asn1_encoder.ko 2024-04-26T21:29:52.773 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc-itu-t.ko 2024-04-26T21:29:52.773 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc8.ko 2024-04-26T21:29:52.773 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto 2024-04-26T21:29:52.773 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libarc4.ko 2024-04-26T21:29:52.773 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libdes.ko 2024-04-26T21:29:52.773 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/libcrc32c.ko 2024-04-26T21:29:52.773 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lru_cache.ko 2024-04-26T21:29:52.773 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4 2024-04-26T21:29:52.773 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4_compress.ko 2024-04-26T21:29:52.773 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4hc_compress.ko 2024-04-26T21:29:52.773 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math 2024-04-26T21:29:52.773 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math/cordic.ko 2024-04-26T21:29:52.773 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6 2024-04-26T21:29:52.774 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6/raid6_pq.ko 2024-04-26T21:29:52.774 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon 2024-04-26T21:29:52.774 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon/reed_solomon.ko 2024-04-26T21:29:52.774 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_bm.ko 2024-04-26T21:29:52.774 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_fsm.ko 2024-04-26T21:29:52.774 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_kmp.ko 2024-04-26T21:29:52.774 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd 2024-04-26T21:29:52.774 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd/zstd_compress.ko 2024-04-26T21:29:52.774 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm 2024-04-26T21:29:52.774 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm/hwpoison-inject.ko 2024-04-26T21:29:52.774 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net 2024-04-26T21:29:52.774 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan 2024-04-26T21:29:52.774 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/6lowpan.ko 2024-04-26T21:29:52.775 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_dest.ko 2024-04-26T21:29:52.775 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_fragment.ko 2024-04-26T21:29:52.775 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_hop.ko 2024-04-26T21:29:52.775 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_ipv6.ko 2024-04-26T21:29:52.775 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_mobility.ko 2024-04-26T21:29:52.775 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_routing.ko 2024-04-26T21:29:52.775 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_udp.ko 2024-04-26T21:29:52.775 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802 2024-04-26T21:29:52.775 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/garp.ko 2024-04-26T21:29:52.775 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/mrp.ko 2024-04-26T21:29:52.775 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/p8022.ko 2024-04-26T21:29:52.775 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/psnap.ko 2024-04-26T21:29:52.775 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/stp.ko 2024-04-26T21:29:52.775 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q 2024-04-26T21:29:52.775 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q/8021q.ko 2024-04-26T21:29:52.776 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p 2024-04-26T21:29:52.776 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet.ko 2024-04-26T21:29:52.776 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_fd.ko 2024-04-26T21:29:52.776 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_rdma.ko 2024-04-26T21:29:52.776 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_virtio.ko 2024-04-26T21:29:52.776 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv 2024-04-26T21:29:52.776 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv/batman-adv.ko 2024-04-26T21:29:52.776 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge 2024-04-26T21:29:52.776 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/br_netfilter.ko 2024-04-26T21:29:52.776 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/bridge.ko 2024-04-26T21:29:52.776 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter 2024-04-26T21:29:52.776 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-04-26T21:29:52.776 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_among.ko 2024-04-26T21:29:52.776 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arp.ko 2024-04-26T21:29:52.777 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-04-26T21:29:52.777 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-04-26T21:29:52.777 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip.ko 2024-04-26T21:29:52.777 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-04-26T21:29:52.777 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_limit.ko 2024-04-26T21:29:52.777 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_log.ko 2024-04-26T21:29:52.777 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark.ko 2024-04-26T21:29:52.777 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-04-26T21:29:52.777 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-04-26T21:29:52.777 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-04-26T21:29:52.777 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-04-26T21:29:52.777 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_snat.ko 2024-04-26T21:29:52.777 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_stp.ko 2024-04-26T21:29:52.777 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-04-26T21:29:52.777 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-04-26T21:29:52.778 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-04-26T21:29:52.778 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-04-26T21:29:52.778 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtables.ko 2024-04-26T21:29:52.778 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-04-26T21:29:52.778 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-04-26T21:29:52.778 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-04-26T21:29:52.778 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph 2024-04-26T21:29:52.778 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph/libceph.ko 2024-04-26T21:29:52.778 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core 2024-04-26T21:29:52.778 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/failover.ko 2024-04-26T21:29:52.778 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/pktgen.ko 2024-04-26T21:29:52.778 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp 2024-04-26T21:29:52.778 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp.ko 2024-04-26T21:29:52.778 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_diag.ko 2024-04-26T21:29:52.779 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv4.ko 2024-04-26T21:29:52.779 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv6.ko 2024-04-26T21:29:52.779 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver 2024-04-26T21:29:52.779 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver/dns_resolver.ko 2024-04-26T21:29:52.779 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa 2024-04-26T21:29:52.779 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/dsa_core.ko 2024-04-26T21:29:52.779 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_brcm.ko 2024-04-26T21:29:52.779 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_dsa.ko 2024-04-26T21:29:52.779 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_none.ko 2024-04-26T21:29:52.779 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_trailer.ko 2024-04-26T21:29:52.779 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154 2024-04-26T21:29:52.779 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan 2024-04-26T21:29:52.779 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-04-26T21:29:52.779 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154.ko 2024-04-26T21:29:52.779 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154_socket.ko 2024-04-26T21:29:52.780 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4 2024-04-26T21:29:52.780 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ah4.ko 2024-04-26T21:29:52.780 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/esp4.ko 2024-04-26T21:29:52.780 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/fou.ko 2024-04-26T21:29:52.780 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/gre.ko 2024-04-26T21:29:52.780 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/inet_diag.ko 2024-04-26T21:29:52.780 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_gre.ko 2024-04-26T21:29:52.780 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_tunnel.ko 2024-04-26T21:29:52.780 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_vti.ko 2024-04-26T21:29:52.780 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipcomp.ko 2024-04-26T21:29:52.780 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipip.ko 2024-04-26T21:29:52.780 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter 2024-04-26T21:29:52.780 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arp_tables.ko 2024-04-26T21:29:52.780 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-04-26T21:29:52.781 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-04-26T21:29:52.781 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-04-26T21:29:52.781 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-04-26T21:29:52.781 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-04-26T21:29:52.781 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-04-26T21:29:52.781 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-04-26T21:29:52.781 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-04-26T21:29:52.781 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-04-26T21:29:52.781 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_security.ko 2024-04-26T21:29:52.781 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-04-26T21:29:52.781 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-04-26T21:29:52.781 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-04-26T21:29:52.781 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-04-26T21:29:52.781 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-04-26T21:29:52.781 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-04-26T21:29:52.782 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-04-26T21:29:52.782 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-04-26T21:29:52.782 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-04-26T21:29:52.782 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-04-26T21:29:52.782 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_bic.ko 2024-04-26T21:29:52.782 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_dctcp.ko 2024-04-26T21:29:52.782 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_diag.ko 2024-04-26T21:29:52.782 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_highspeed.ko 2024-04-26T21:29:52.782 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_htcp.ko 2024-04-26T21:29:52.782 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_hybla.ko 2024-04-26T21:29:52.782 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_illinois.ko 2024-04-26T21:29:52.782 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_lp.ko 2024-04-26T21:29:52.782 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_scalable.ko 2024-04-26T21:29:52.782 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_vegas.ko 2024-04-26T21:29:52.782 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_veno.ko 2024-04-26T21:29:52.783 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_westwood.ko 2024-04-26T21:29:52.783 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_yeah.ko 2024-04-26T21:29:52.783 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tunnel4.ko 2024-04-26T21:29:52.783 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_diag.ko 2024-04-26T21:29:52.783 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_tunnel.ko 2024-04-26T21:29:52.783 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/xfrm4_tunnel.ko 2024-04-26T21:29:52.783 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6 2024-04-26T21:29:52.783 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ah6.ko 2024-04-26T21:29:52.783 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/esp6.ko 2024-04-26T21:29:52.783 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/fou6.ko 2024-04-26T21:29:52.783 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_tunnel.ko 2024-04-26T21:29:52.783 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-04-26T21:29:52.783 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_vti.ko 2024-04-26T21:29:52.783 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ipcomp6.ko 2024-04-26T21:29:52.783 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter 2024-04-26T21:29:52.783 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-04-26T21:29:52.784 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-04-26T21:29:52.784 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-04-26T21:29:52.784 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-04-26T21:29:52.784 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-04-26T21:29:52.784 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-04-26T21:29:52.784 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-04-26T21:29:52.784 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-04-26T21:29:52.784 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-04-26T21:29:52.784 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-04-26T21:29:52.784 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-04-26T21:29:52.784 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-04-26T21:29:52.784 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-04-26T21:29:52.784 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-04-26T21:29:52.784 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-04-26T21:29:52.785 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-04-26T21:29:52.785 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-04-26T21:29:52.785 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-04-26T21:29:52.785 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-04-26T21:29:52.785 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-04-26T21:29:52.785 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-04-26T21:29:52.785 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-04-26T21:29:52.785 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-04-26T21:29:52.785 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-04-26T21:29:52.785 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/sit.ko 2024-04-26T21:29:52.785 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/tunnel6.ko 2024-04-26T21:29:52.785 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/xfrm6_tunnel.ko 2024-04-26T21:29:52.785 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key 2024-04-26T21:29:52.785 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key/af_key.ko 2024-04-26T21:29:52.786 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp 2024-04-26T21:29:52.786 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_core.ko 2024-04-26T21:29:52.786 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_debugfs.ko 2024-04-26T21:29:52.786 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_eth.ko 2024-04-26T21:29:52.786 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip.ko 2024-04-26T21:29:52.786 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip6.ko 2024-04-26T21:29:52.786 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_netlink.ko 2024-04-26T21:29:52.786 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ppp.ko 2024-04-26T21:29:52.786 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc 2024-04-26T21:29:52.786 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc/llc.ko 2024-04-26T21:29:52.786 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154 2024-04-26T21:29:52.786 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154/mac802154.ko 2024-04-26T21:29:52.786 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls 2024-04-26T21:29:52.787 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls/mpls_gso.ko 2024-04-26T21:29:52.787 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter 2024-04-26T21:29:52.787 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset 2024-04-26T21:29:52.787 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set.ko 2024-04-26T21:29:52.787 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-04-26T21:29:52.787 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-04-26T21:29:52.787 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-04-26T21:29:52.787 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-04-26T21:29:52.787 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-04-26T21:29:52.787 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-04-26T21:29:52.787 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-04-26T21:29:52.787 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-04-26T21:29:52.787 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-04-26T21:29:52.787 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-04-26T21:29:52.787 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-04-26T21:29:52.787 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-04-26T21:29:52.788 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-04-26T21:29:52.788 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-04-26T21:29:52.788 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-04-26T21:29:52.788 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs 2024-04-26T21:29:52.788 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs.ko 2024-04-26T21:29:52.788 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-04-26T21:29:52.788 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-04-26T21:29:52.788 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-04-26T21:29:52.788 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-04-26T21:29:52.788 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-04-26T21:29:52.788 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-04-26T21:29:52.788 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-04-26T21:29:52.788 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-04-26T21:29:52.788 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-04-26T21:29:52.788 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-04-26T21:29:52.788 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-04-26T21:29:52.789 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-04-26T21:29:52.789 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-04-26T21:29:52.789 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conncount.ko 2024-04-26T21:29:52.789 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack.ko 2024-04-26T21:29:52.789 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-04-26T21:29:52.789 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-04-26T21:29:52.789 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-04-26T21:29:52.789 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_h323.ko 2024-04-26T21:29:52.789 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_irc.ko 2024-04-26T21:29:52.789 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-04-26T21:29:52.789 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-04-26T21:29:52.789 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-04-26T21:29:52.789 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sane.ko 2024-04-26T21:29:52.789 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sip.ko 2024-04-26T21:29:52.789 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-04-26T21:29:52.789 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-04-26T21:29:52.790 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_dup_netdev.ko 2024-04-26T21:29:52.790 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table.ko 2024-04-26T21:29:52.790 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table_inet.ko 2024-04-26T21:29:52.790 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_log_syslog.ko 2024-04-26T21:29:52.790 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat.ko 2024-04-26T21:29:52.790 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_amanda.ko 2024-04-26T21:29:52.790 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_ftp.ko 2024-04-26T21:29:52.790 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_irc.ko 2024-04-26T21:29:52.790 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_sip.ko 2024-04-26T21:29:52.790 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_tftp.ko 2024-04-26T21:29:52.790 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_synproxy_core.ko 2024-04-26T21:29:52.790 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_tables.ko 2024-04-26T21:29:52.790 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink.ko 2024-04-26T21:29:52.790 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_acct.ko 2024-04-26T21:29:52.790 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_log.ko 2024-04-26T21:29:52.791 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_osf.ko 2024-04-26T21:29:52.791 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_queue.ko 2024-04-26T21:29:52.791 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_chain_nat.ko 2024-04-26T21:29:52.791 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_compat.ko 2024-04-26T21:29:52.791 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_ct.ko 2024-04-26T21:29:52.791 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_dup_netdev.ko 2024-04-26T21:29:52.791 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fib.ko 2024-04-26T21:29:52.791 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fwd_netdev.ko 2024-04-26T21:29:52.791 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_hash.ko 2024-04-26T21:29:52.791 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_limit.ko 2024-04-26T21:29:52.791 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_log.ko 2024-04-26T21:29:52.791 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_masq.ko 2024-04-26T21:29:52.791 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_nat.ko 2024-04-26T21:29:52.791 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_numgen.ko 2024-04-26T21:29:52.791 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_queue.ko 2024-04-26T21:29:52.792 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_quota.ko 2024-04-26T21:29:52.792 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_redir.ko 2024-04-26T21:29:52.792 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject.ko 2024-04-26T21:29:52.792 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject_inet.ko 2024-04-26T21:29:52.792 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_socket.ko 2024-04-26T21:29:52.792 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_tunnel.ko 2024-04-26T21:29:52.792 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_AUDIT.ko 2024-04-26T21:29:52.792 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CHECKSUM.ko 2024-04-26T21:29:52.792 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CLASSIFY.ko 2024-04-26T21:29:52.792 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-04-26T21:29:52.792 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CT.ko 2024-04-26T21:29:52.792 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_DSCP.ko 2024-04-26T21:29:52.792 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HL.ko 2024-04-26T21:29:52.792 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HMARK.ko 2024-04-26T21:29:52.792 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_IDLETIMER.ko 2024-04-26T21:29:52.793 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LED.ko 2024-04-26T21:29:52.793 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LOG.ko 2024-04-26T21:29:52.793 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_MASQUERADE.ko 2024-04-26T21:29:52.793 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NETMAP.ko 2024-04-26T21:29:52.793 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFLOG.ko 2024-04-26T21:29:52.793 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFQUEUE.ko 2024-04-26T21:29:52.793 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_RATEEST.ko 2024-04-26T21:29:52.793 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_REDIRECT.ko 2024-04-26T21:29:52.793 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_SECMARK.ko 2024-04-26T21:29:52.793 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPMSS.ko 2024-04-26T21:29:52.793 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-04-26T21:29:52.793 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TEE.ko 2024-04-26T21:29:52.793 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TPROXY.ko 2024-04-26T21:29:52.793 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TRACE.ko 2024-04-26T21:29:52.793 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_addrtype.ko 2024-04-26T21:29:52.794 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_bpf.ko 2024-04-26T21:29:52.794 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cgroup.ko 2024-04-26T21:29:52.794 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cluster.ko 2024-04-26T21:29:52.794 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_comment.ko 2024-04-26T21:29:52.794 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connbytes.ko 2024-04-26T21:29:52.794 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlabel.ko 2024-04-26T21:29:52.794 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlimit.ko 2024-04-26T21:29:52.794 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connmark.ko 2024-04-26T21:29:52.794 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_conntrack.ko 2024-04-26T21:29:52.794 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cpu.ko 2024-04-26T21:29:52.794 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dccp.ko 2024-04-26T21:29:52.794 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_devgroup.ko 2024-04-26T21:29:52.794 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dscp.ko 2024-04-26T21:29:52.794 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ecn.ko 2024-04-26T21:29:52.794 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_esp.ko 2024-04-26T21:29:52.795 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hashlimit.ko 2024-04-26T21:29:52.795 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_helper.ko 2024-04-26T21:29:52.795 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hl.ko 2024-04-26T21:29:52.795 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipcomp.ko 2024-04-26T21:29:52.795 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_iprange.ko 2024-04-26T21:29:52.795 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipvs.ko 2024-04-26T21:29:52.795 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_l2tp.ko 2024-04-26T21:29:52.795 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_length.ko 2024-04-26T21:29:52.795 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_limit.ko 2024-04-26T21:29:52.795 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mac.ko 2024-04-26T21:29:52.795 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mark.ko 2024-04-26T21:29:52.795 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_multiport.ko 2024-04-26T21:29:52.795 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nat.ko 2024-04-26T21:29:52.795 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nfacct.ko 2024-04-26T21:29:52.796 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_osf.ko 2024-04-26T21:29:52.796 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_owner.ko 2024-04-26T21:29:52.796 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_physdev.ko 2024-04-26T21:29:52.796 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_pkttype.ko 2024-04-26T21:29:52.796 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_policy.ko 2024-04-26T21:29:52.796 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_quota.ko 2024-04-26T21:29:52.796 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_rateest.ko 2024-04-26T21:29:52.796 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_realm.ko 2024-04-26T21:29:52.796 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_recent.ko 2024-04-26T21:29:52.796 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_sctp.ko 2024-04-26T21:29:52.796 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_set.ko 2024-04-26T21:29:52.796 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_socket.ko 2024-04-26T21:29:52.796 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_state.ko 2024-04-26T21:29:52.796 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_statistic.ko 2024-04-26T21:29:52.796 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_string.ko 2024-04-26T21:29:52.796 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_tcpmss.ko 2024-04-26T21:29:52.797 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_time.ko 2024-04-26T21:29:52.797 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_u32.ko 2024-04-26T21:29:52.797 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink 2024-04-26T21:29:52.797 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink/netlink_diag.ko 2024-04-26T21:29:52.797 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh 2024-04-26T21:29:52.797 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh/nsh.ko 2024-04-26T21:29:52.797 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch 2024-04-26T21:29:52.797 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/openvswitch.ko 2024-04-26T21:29:52.797 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-geneve.ko 2024-04-26T21:29:52.797 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-gre.ko 2024-04-26T21:29:52.797 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-vxlan.ko 2024-04-26T21:29:52.797 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet 2024-04-26T21:29:52.797 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet/af_packet_diag.ko 2024-04-26T21:29:52.797 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds 2024-04-26T21:29:52.797 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds.ko 2024-04-26T21:29:52.798 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_rdma.ko 2024-04-26T21:29:52.798 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_tcp.ko 2024-04-26T21:29:52.798 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched 2024-04-26T21:29:52.798 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_csum.ko 2024-04-26T21:29:52.798 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_gact.ko 2024-04-26T21:29:52.798 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_mirred.ko 2024-04-26T21:29:52.798 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_nat.ko 2024-04-26T21:29:52.798 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_pedit.ko 2024-04-26T21:29:52.798 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_police.ko 2024-04-26T21:29:52.798 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_simple.ko 2024-04-26T21:29:52.798 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_skbedit.ko 2024-04-26T21:29:52.798 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_vlan.ko 2024-04-26T21:29:52.798 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_basic.ko 2024-04-26T21:29:52.798 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_bpf.ko 2024-04-26T21:29:52.799 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_flow.ko 2024-04-26T21:29:52.799 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_fw.ko 2024-04-26T21:29:52.799 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_route.ko 2024-04-26T21:29:52.799 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_u32.ko 2024-04-26T21:29:52.799 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_cmp.ko 2024-04-26T21:29:52.799 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_ipset.ko 2024-04-26T21:29:52.799 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_meta.ko 2024-04-26T21:29:52.799 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_nbyte.ko 2024-04-26T21:29:52.799 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_text.ko 2024-04-26T21:29:52.799 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_u32.ko 2024-04-26T21:29:52.799 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_choke.ko 2024-04-26T21:29:52.799 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_codel.ko 2024-04-26T21:29:52.799 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_drr.ko 2024-04-26T21:29:52.799 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_fq.ko 2024-04-26T21:29:52.799 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_gred.ko 2024-04-26T21:29:52.799 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hfsc.ko 2024-04-26T21:29:52.799 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hhf.ko 2024-04-26T21:29:52.800 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_htb.ko 2024-04-26T21:29:52.800 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_ingress.ko 2024-04-26T21:29:52.800 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio.ko 2024-04-26T21:29:52.800 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio_lib.ko 2024-04-26T21:29:52.800 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_multiq.ko 2024-04-26T21:29:52.800 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_netem.ko 2024-04-26T21:29:52.800 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_pie.ko 2024-04-26T21:29:52.800 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_plug.ko 2024-04-26T21:29:52.800 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_prio.ko 2024-04-26T21:29:52.800 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_qfq.ko 2024-04-26T21:29:52.800 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_red.ko 2024-04-26T21:29:52.800 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfb.ko 2024-04-26T21:29:52.800 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfq.ko 2024-04-26T21:29:52.800 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_tbf.ko 2024-04-26T21:29:52.801 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_teql.ko 2024-04-26T21:29:52.801 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp 2024-04-26T21:29:52.801 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp.ko 2024-04-26T21:29:52.801 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp_diag.ko 2024-04-26T21:29:52.801 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc 2024-04-26T21:29:52.801 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss 2024-04-26T21:29:52.801 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-04-26T21:29:52.801 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-04-26T21:29:52.801 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/sunrpc.ko 2024-04-26T21:29:52.801 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma 2024-04-26T21:29:52.801 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-04-26T21:29:52.801 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc 2024-04-26T21:29:52.801 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc.ko 2024-04-26T21:29:52.801 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc_diag.ko 2024-04-26T21:29:52.801 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix 2024-04-26T21:29:52.801 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix/unix_diag.ko 2024-04-26T21:29:52.802 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock 2024-04-26T21:29:52.802 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-04-26T21:29:52.802 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-04-26T21:29:52.802 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock.ko 2024-04-26T21:29:52.802 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_diag.ko 2024-04-26T21:29:52.802 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_loopback.ko 2024-04-26T21:29:52.802 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm 2024-04-26T21:29:52.802 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm/xfrm_ipcomp.ko 2024-04-26T21:29:52.802 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security 2024-04-26T21:29:52.802 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys 2024-04-26T21:29:52.802 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys 2024-04-26T21:29:52.802 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-04-26T21:29:52.802 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys 2024-04-26T21:29:52.802 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys/trusted.ko 2024-04-26T21:29:52.803 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias 2024-04-26T21:29:52.803 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias.bin 2024-04-26T21:29:52.803 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin 2024-04-26T21:29:52.803 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.alias.bin 2024-04-26T21:29:52.803 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.bin 2024-04-26T21:29:52.803 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.modinfo 2024-04-26T21:29:52.803 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep 2024-04-26T21:29:52.803 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep.bin 2024-04-26T21:29:52.803 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.devname 2024-04-26T21:29:52.803 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.order 2024-04-26T21:29:52.803 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.softdep 2024-04-26T21:29:52.803 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols 2024-04-26T21:29:52.803 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols.bin 2024-04-26T21:29:52.803 INFO:teuthology.orchestra.run.smithi072.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-04-26T21:29:52.807 DEBUG:teuthology.task.kernel:get_image_version: 6.9.0-rc3-g806ec36b2112 2024-04-26T21:29:52.807 DEBUG:teuthology.orchestra.run.smithi072:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-04-26T21:29:52.831 INFO:teuthology.orchestra.run.smithi072.stdout:postinstall scriptlet (using /bin/sh): 2024-04-26T21:29:52.831 INFO:teuthology.orchestra.run.smithi072.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-26T21:29:52.831 INFO:teuthology.orchestra.run.smithi072.stdout: /usr/bin/kernel-install add 6.9.0-rc3-g806ec36b2112 /lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-04-26T21:29:52.831 INFO:teuthology.orchestra.run.smithi072.stdout:fi 2024-04-26T21:29:52.831 INFO:teuthology.orchestra.run.smithi072.stdout:for file in vmlinuz System.map config; do 2024-04-26T21:29:52.831 INFO:teuthology.orchestra.run.smithi072.stdout: if ! cmp --silent "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112"; then 2024-04-26T21:29:52.831 INFO:teuthology.orchestra.run.smithi072.stdout: cp "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112" 2024-04-26T21:29:52.831 INFO:teuthology.orchestra.run.smithi072.stdout: fi 2024-04-26T21:29:52.831 INFO:teuthology.orchestra.run.smithi072.stdout:done 2024-04-26T21:29:52.831 INFO:teuthology.orchestra.run.smithi072.stdout:if [ -d "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" ] && \ 2024-04-26T21:29:52.831 INFO:teuthology.orchestra.run.smithi072.stdout: ! diff -rq "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" >/dev/null 2>&1; then 2024-04-26T21:29:52.831 INFO:teuthology.orchestra.run.smithi072.stdout: rm -rf "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-04-26T21:29:52.832 INFO:teuthology.orchestra.run.smithi072.stdout: cp -r "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-04-26T21:29:52.832 INFO:teuthology.orchestra.run.smithi072.stdout:fi 2024-04-26T21:29:52.832 INFO:teuthology.orchestra.run.smithi072.stdout:if [ ! -e "/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep" ]; then 2024-04-26T21:29:52.832 INFO:teuthology.orchestra.run.smithi072.stdout: /usr/sbin/depmod 6.9.0-rc3-g806ec36b2112 2024-04-26T21:29:52.832 INFO:teuthology.orchestra.run.smithi072.stdout:fi 2024-04-26T21:29:52.832 INFO:teuthology.orchestra.run.smithi072.stdout:preuninstall scriptlet (using /bin/sh): 2024-04-26T21:29:52.832 INFO:teuthology.orchestra.run.smithi072.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-26T21:29:52.832 INFO:teuthology.orchestra.run.smithi072.stdout:kernel-install remove 6.9.0-rc3-g806ec36b2112 2024-04-26T21:29:52.832 INFO:teuthology.orchestra.run.smithi072.stdout:fi 2024-04-26T21:29:52.833 DEBUG:teuthology.orchestra.run.smithi072:> sudo rpm -qi grub2-tools 2024-04-26T21:29:52.904 INFO:teuthology.orchestra.run.smithi072.stdout:Name : grub2-tools 2024-04-26T21:29:52.905 INFO:teuthology.orchestra.run.smithi072.stdout:Epoch : 1 2024-04-26T21:29:52.905 INFO:teuthology.orchestra.run.smithi072.stdout:Version : 2.06 2024-04-26T21:29:52.905 INFO:teuthology.orchestra.run.smithi072.stdout:Release : 68.el9 2024-04-26T21:29:52.905 INFO:teuthology.orchestra.run.smithi072.stdout:Architecture: x86_64 2024-04-26T21:29:52.905 INFO:teuthology.orchestra.run.smithi072.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-04-26T21:29:52.905 INFO:teuthology.orchestra.run.smithi072.stdout:Group : Unspecified 2024-04-26T21:29:52.905 INFO:teuthology.orchestra.run.smithi072.stdout:Size : 8264254 2024-04-26T21:29:52.905 INFO:teuthology.orchestra.run.smithi072.stdout:License : GPLv3+ 2024-04-26T21:29:52.905 INFO:teuthology.orchestra.run.smithi072.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-04-26T21:29:52.905 INFO:teuthology.orchestra.run.smithi072.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-04-26T21:29:52.905 INFO:teuthology.orchestra.run.smithi072.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-04-26T21:29:52.905 INFO:teuthology.orchestra.run.smithi072.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-04-26T21:29:52.905 INFO:teuthology.orchestra.run.smithi072.stdout:Packager : builder@centos.org 2024-04-26T21:29:52.905 INFO:teuthology.orchestra.run.smithi072.stdout:Vendor : CentOS 2024-04-26T21:29:52.906 INFO:teuthology.orchestra.run.smithi072.stdout:URL : http://www.gnu.org/software/grub/ 2024-04-26T21:29:52.906 INFO:teuthology.orchestra.run.smithi072.stdout:Summary : Support tools for GRUB. 2024-04-26T21:29:52.906 INFO:teuthology.orchestra.run.smithi072.stdout:Description : 2024-04-26T21:29:52.906 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T21:29:52.906 INFO:teuthology.orchestra.run.smithi072.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-04-26T21:29:52.906 INFO:teuthology.orchestra.run.smithi072.stdout:customizable bootloader with modular architecture. It supports a rich 2024-04-26T21:29:52.906 INFO:teuthology.orchestra.run.smithi072.stdout:variety of kernel formats, file systems, computer architectures and 2024-04-26T21:29:52.906 INFO:teuthology.orchestra.run.smithi072.stdout:hardware devices. 2024-04-26T21:29:52.906 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T21:29:52.906 INFO:teuthology.orchestra.run.smithi072.stdout:This subpackage provides tools for support of all platforms. 2024-04-26T21:29:52.907 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-04-26T21:29:52.907 INFO:teuthology.task.kernel:Updating grub on smithi072 to boot 6.9.0-rc3-g806ec36b2112 2024-04-26T21:29:52.907 DEBUG:teuthology.orchestra.run.smithi072:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-04-26T21:29:53.561 INFO:teuthology.orchestra.run.smithi072.stderr:Generating grub configuration file ... 2024-04-26T21:29:55.132 INFO:teuthology.orchestra.run.smithi072.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-04-26T21:29:55.154 INFO:teuthology.orchestra.run.smithi072.stderr:done 2024-04-26T21:29:55.156 DEBUG:teuthology.orchestra.run.smithi072:> mktemp 2024-04-26T21:29:55.184 INFO:teuthology.orchestra.run.smithi072.stdout:/tmp/tmp.BOuaiTmrU5 2024-04-26T21:29:55.184 DEBUG:teuthology.orchestra.run.smithi072:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.BOuaiTmrU5 2024-04-26T21:29:55.260 DEBUG:teuthology.orchestra.run.smithi072:> sudo chmod 0666 /tmp/tmp.BOuaiTmrU5 2024-04-26T21:29:55.397 DEBUG:teuthology.orchestra.remote:smithi072:/tmp/tmp.BOuaiTmrU5 is 6KB 2024-04-26T21:29:55.446 DEBUG:teuthology.orchestra.run.smithi072:> rm -fr /tmp/tmp.BOuaiTmrU5 2024-04-26T21:29:55.461 DEBUG:teuthology.orchestra.run.smithi072:> sudo /bin/ls /boot/loader/entries || true 2024-04-26T21:29:55.522 INFO:teuthology.orchestra.run.smithi072.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-04-26T21:29:55.522 INFO:teuthology.orchestra.run.smithi072.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-04-26T21:29:55.522 INFO:teuthology.orchestra.run.smithi072.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-04-26T21:29:55.522 INFO:teuthology.orchestra.run.smithi072.stdout:7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112.conf 2024-04-26T21:29:55.523 DEBUG:teuthology.orchestra.run.smithi072:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112 2024-04-26T21:29:55.671 DEBUG:teuthology.orchestra.run.smithi072:> sudo shutdown -r now 2024-04-26T21:29:56.382 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi164.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-04-26T21:29:56.382 DEBUG:teuthology.orchestra.run.smithi164:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-04-26T21:29:56.457 INFO:teuthology.orchestra.run.smithi164.stdout:/boot/System.map-6.9.0-rc3-g806ec36b2112 2024-04-26T21:29:56.457 INFO:teuthology.orchestra.run.smithi164.stdout:/boot/config-6.9.0-rc3-g806ec36b2112 2024-04-26T21:29:56.457 INFO:teuthology.orchestra.run.smithi164.stdout:/boot/vmlinuz-6.9.0-rc3-g806ec36b2112 2024-04-26T21:29:56.457 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/System.map 2024-04-26T21:29:56.457 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/config 2024-04-26T21:29:56.458 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel 2024-04-26T21:29:56.458 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch 2024-04-26T21:29:56.458 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86 2024-04-26T21:29:56.458 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto 2024-04-26T21:29:56.458 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-04-26T21:29:56.458 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-04-26T21:29:56.458 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-04-26T21:29:56.458 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-04-26T21:29:56.458 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-04-26T21:29:56.458 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-04-26T21:29:56.458 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-04-26T21:29:56.458 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32c-intel.ko 2024-04-26T21:29:56.458 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-04-26T21:29:56.458 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-04-26T21:29:56.459 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-04-26T21:29:56.459 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-04-26T21:29:56.459 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx2.ko 2024-04-26T21:29:56.459 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-04-26T21:29:56.459 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-04-26T21:29:56.459 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-04-26T21:29:56.459 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-04-26T21:29:56.459 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-04-26T21:29:56.459 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-04-26T21:29:56.459 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-04-26T21:29:56.459 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel 2024-04-26T21:29:56.459 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu 2024-04-26T21:29:56.459 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce 2024-04-26T21:29:56.460 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-04-26T21:29:56.460 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm 2024-04-26T21:29:56.460 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-amd.ko 2024-04-26T21:29:56.460 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-intel.ko 2024-04-26T21:29:56.460 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm.ko 2024-04-26T21:29:56.460 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto 2024-04-26T21:29:56.460 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ansi_cprng.ko 2024-04-26T21:29:56.460 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/anubis.ko 2024-04-26T21:29:56.460 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/arc4.ko 2024-04-26T21:29:56.460 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx 2024-04-26T21:29:56.460 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_memcpy.ko 2024-04-26T21:29:56.460 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_pq.ko 2024-04-26T21:29:56.460 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_raid6_recov.ko 2024-04-26T21:29:56.460 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_tx.ko 2024-04-26T21:29:56.461 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_xor.ko 2024-04-26T21:29:56.461 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/raid6test.ko 2024-04-26T21:29:56.461 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authenc.ko 2024-04-26T21:29:56.461 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authencesn.ko 2024-04-26T21:29:56.461 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blake2b_generic.ko 2024-04-26T21:29:56.461 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_common.ko 2024-04-26T21:29:56.461 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_generic.ko 2024-04-26T21:29:56.461 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/camellia_generic.ko 2024-04-26T21:29:56.461 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast5_generic.ko 2024-04-26T21:29:56.462 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast6_generic.ko 2024-04-26T21:29:56.462 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast_common.ko 2024-04-26T21:29:56.462 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ccm.ko 2024-04-26T21:29:56.462 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cmac.ko 2024-04-26T21:29:56.462 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crc32_generic.ko 2024-04-26T21:29:56.462 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_engine.ko 2024-04-26T21:29:56.462 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_user.ko 2024-04-26T21:29:56.462 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cts.ko 2024-04-26T21:29:56.462 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/des_generic.ko 2024-04-26T21:29:56.462 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/dh_generic.ko 2024-04-26T21:29:56.462 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/echainiv.ko 2024-04-26T21:29:56.463 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/essiv.ko 2024-04-26T21:29:56.463 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/fcrypt.ko 2024-04-26T21:29:56.463 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/khazad.ko 2024-04-26T21:29:56.463 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4.ko 2024-04-26T21:29:56.463 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4hc.ko 2024-04-26T21:29:56.463 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/md4.ko 2024-04-26T21:29:56.463 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/michael_mic.ko 2024-04-26T21:29:56.463 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcbc.ko 2024-04-26T21:29:56.463 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcrypt.ko 2024-04-26T21:29:56.463 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/rmd160.ko 2024-04-26T21:29:56.463 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/seed.ko 2024-04-26T21:29:56.463 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/serpent_generic.ko 2024-04-26T21:29:56.463 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tcrypt.ko 2024-04-26T21:29:56.463 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tea.ko 2024-04-26T21:29:56.464 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_common.ko 2024-04-26T21:29:56.464 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_generic.ko 2024-04-26T21:29:56.464 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/vmac.ko 2024-04-26T21:29:56.464 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/wp512.ko 2024-04-26T21:29:56.464 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xcbc.ko 2024-04-26T21:29:56.464 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xor.ko 2024-04-26T21:29:56.464 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xxhash_generic.ko 2024-04-26T21:29:56.464 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers 2024-04-26T21:29:56.464 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi 2024-04-26T21:29:56.464 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_ipmi.ko 2024-04-26T21:29:56.464 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_pad.ko 2024-04-26T21:29:56.464 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/ec_sys.ko 2024-04-26T21:29:56.464 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/platform_profile.ko 2024-04-26T21:29:56.464 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbs.ko 2024-04-26T21:29:56.465 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbshc.ko 2024-04-26T21:29:56.465 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/video.ko 2024-04-26T21:29:56.465 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata 2024-04-26T21:29:56.465 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/acard-ahci.ko 2024-04-26T21:29:56.465 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ahci_platform.ko 2024-04-26T21:29:56.465 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ata_generic.ko 2024-04-26T21:29:56.465 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/libahci_platform.ko 2024-04-26T21:29:56.465 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_acpi.ko 2024-04-26T21:29:56.465 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ali.ko 2024-04-26T21:29:56.465 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_amd.ko 2024-04-26T21:29:56.465 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_artop.ko 2024-04-26T21:29:56.465 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atiixp.ko 2024-04-26T21:29:56.465 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atp867x.ko 2024-04-26T21:29:56.465 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd640.ko 2024-04-26T21:29:56.466 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd64x.ko 2024-04-26T21:29:56.466 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cypress.ko 2024-04-26T21:29:56.466 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_efar.ko 2024-04-26T21:29:56.466 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt366.ko 2024-04-26T21:29:56.466 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt37x.ko 2024-04-26T21:29:56.466 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x2n.ko 2024-04-26T21:29:56.466 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x3.ko 2024-04-26T21:29:56.466 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it8213.ko 2024-04-26T21:29:56.466 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it821x.ko 2024-04-26T21:29:56.466 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_jmicron.ko 2024-04-26T21:29:56.466 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_marvell.ko 2024-04-26T21:29:56.466 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_mpiix.ko 2024-04-26T21:29:56.466 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_netcell.ko 2024-04-26T21:29:56.466 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ninja32.ko 2024-04-26T21:29:56.467 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87410.ko 2024-04-26T21:29:56.467 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87415.ko 2024-04-26T21:29:56.467 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_oldpiix.ko 2024-04-26T21:29:56.467 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_opti.ko 2024-04-26T21:29:56.467 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_optidma.ko 2024-04-26T21:29:56.467 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pcmcia.ko 2024-04-26T21:29:56.467 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc2027x.ko 2024-04-26T21:29:56.467 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-04-26T21:29:56.467 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_piccolo.ko 2024-04-26T21:29:56.467 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_rdc.ko 2024-04-26T21:29:56.467 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sch.ko 2024-04-26T21:29:56.467 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_serverworks.ko 2024-04-26T21:29:56.467 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sil680.ko 2024-04-26T21:29:56.468 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sis.ko 2024-04-26T21:29:56.468 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sl82c105.ko 2024-04-26T21:29:56.468 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_triflex.ko 2024-04-26T21:29:56.468 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_via.ko 2024-04-26T21:29:56.468 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pdc_adma.ko 2024-04-26T21:29:56.468 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_inic162x.ko 2024-04-26T21:29:56.468 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_mv.ko 2024-04-26T21:29:56.468 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_nv.ko 2024-04-26T21:29:56.468 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_promise.ko 2024-04-26T21:29:56.468 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_qstor.ko 2024-04-26T21:29:56.468 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil.ko 2024-04-26T21:29:56.468 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil24.ko 2024-04-26T21:29:56.469 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sis.ko 2024-04-26T21:29:56.469 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_svw.ko 2024-04-26T21:29:56.469 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sx4.ko 2024-04-26T21:29:56.469 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_uli.ko 2024-04-26T21:29:56.469 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_via.ko 2024-04-26T21:29:56.469 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_vsc.ko 2024-04-26T21:29:56.469 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay 2024-04-26T21:29:56.469 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay/ks0108.ko 2024-04-26T21:29:56.469 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base 2024-04-26T21:29:56.469 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap 2024-04-26T21:29:56.469 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-i2c.ko 2024-04-26T21:29:56.469 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-mmio.ko 2024-04-26T21:29:56.470 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma 2024-04-26T21:29:56.470 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma/bcma.ko 2024-04-26T21:29:56.470 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block 2024-04-26T21:29:56.470 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe 2024-04-26T21:29:56.470 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe/aoe.ko 2024-04-26T21:29:56.470 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/brd.ko 2024-04-26T21:29:56.470 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd 2024-04-26T21:29:56.470 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd/drbd.ko 2024-04-26T21:29:56.470 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/floppy.ko 2024-04-26T21:29:56.470 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/loop.ko 2024-04-26T21:29:56.470 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx 2024-04-26T21:29:56.471 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-04-26T21:29:56.471 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/nbd.ko 2024-04-26T21:29:56.471 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk 2024-04-26T21:29:56.471 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk/null_blk.ko 2024-04-26T21:29:56.471 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/pktcdvd.ko 2024-04-26T21:29:56.471 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/rbd.ko 2024-04-26T21:29:56.471 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/virtio_blk.ko 2024-04-26T21:29:56.471 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback 2024-04-26T21:29:56.471 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-04-26T21:29:56.471 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkfront.ko 2024-04-26T21:29:56.471 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram 2024-04-26T21:29:56.471 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram/zram.ko 2024-04-26T21:29:56.471 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char 2024-04-26T21:29:56.471 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hangcheck-timer.ko 2024-04-26T21:29:56.471 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random 2024-04-26T21:29:56.472 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/amd-rng.ko 2024-04-26T21:29:56.472 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/intel-rng.ko 2024-04-26T21:29:56.472 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-04-26T21:29:56.472 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/via-rng.ko 2024-04-26T21:29:56.472 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/virtio-rng.ko 2024-04-26T21:29:56.472 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi 2024-04-26T21:29:56.472 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-04-26T21:29:56.472 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-04-26T21:29:56.472 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-04-26T21:29:56.472 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_si.ko 2024-04-26T21:29:56.472 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-04-26T21:29:56.472 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-04-26T21:29:56.472 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/lp.ko 2024-04-26T21:29:56.472 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave 2024-04-26T21:29:56.472 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave/mwave.ko 2024-04-26T21:29:56.473 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ppdev.ko 2024-04-26T21:29:56.473 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tlclk.ko 2024-04-26T21:29:56.473 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm 2024-04-26T21:29:56.473 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm.ko 2024-04-26T21:29:56.473 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_atmel.ko 2024-04-26T21:29:56.473 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_infineon.ko 2024-04-26T21:29:56.473 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_nsc.ko 2024-04-26T21:29:56.473 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis.ko 2024-04-26T21:29:56.473 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-04-26T21:29:56.473 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/uv_mmtimer.ko 2024-04-26T21:29:56.473 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/virtio_console.ko 2024-04-26T21:29:56.473 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq 2024-04-26T21:29:56.473 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-04-26T21:29:56.473 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-04-26T21:29:56.473 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/p4-clockmod.ko 2024-04-26T21:29:56.473 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-04-26T21:29:56.474 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/powernow-k8.ko 2024-04-26T21:29:56.474 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/speedstep-lib.ko 2024-04-26T21:29:56.474 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto 2024-04-26T21:29:56.474 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp 2024-04-26T21:29:56.474 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-04-26T21:29:56.474 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp.ko 2024-04-26T21:29:56.474 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel 2024-04-26T21:29:56.474 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat 2024-04-26T21:29:56.474 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common 2024-04-26T21:29:56.474 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-04-26T21:29:56.474 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-04-26T21:29:56.474 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-04-26T21:29:56.474 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-aes.ko 2024-04-26T21:29:56.474 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-sha.ko 2024-04-26T21:29:56.474 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio 2024-04-26T21:29:56.475 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-04-26T21:29:56.475 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca 2024-04-26T21:29:56.475 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca/dca.ko 2024-04-26T21:29:56.475 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq 2024-04-26T21:29:56.475 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-04-26T21:29:56.475 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma 2024-04-26T21:29:56.475 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw 2024-04-26T21:29:56.475 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw/dw_dmac.ko 2024-04-26T21:29:56.475 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat 2024-04-26T21:29:56.475 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat/ioatdma.ko 2024-04-26T21:29:56.475 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac 2024-04-26T21:29:56.475 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/amd64_edac.ko 2024-04-26T21:29:56.475 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/e752x_edac.ko 2024-04-26T21:29:56.475 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/edac_mce_amd.ko 2024-04-26T21:29:56.475 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3000_edac.ko 2024-04-26T21:29:56.475 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3200_edac.ko 2024-04-26T21:29:56.476 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5100_edac.ko 2024-04-26T21:29:56.476 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5400_edac.ko 2024-04-26T21:29:56.476 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7300_edac.ko 2024-04-26T21:29:56.476 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7core_edac.ko 2024-04-26T21:29:56.476 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i82975x_edac.ko 2024-04-26T21:29:56.476 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/ie31200_edac.ko 2024-04-26T21:29:56.476 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/sb_edac.ko 2024-04-26T21:29:56.476 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/x38_edac.ko 2024-04-26T21:29:56.476 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware 2024-04-26T21:29:56.476 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/edd.ko 2024-04-26T21:29:56.476 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/iscsi_ibft.ko 2024-04-26T21:29:56.476 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio 2024-04-26T21:29:56.476 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-ich.ko 2024-04-26T21:29:56.476 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-viperboard.ko 2024-04-26T21:29:56.476 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid 2024-04-26T21:29:56.477 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-appleir.ko 2024-04-26T21:29:56.477 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-aureal.ko 2024-04-26T21:29:56.477 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-axff.ko 2024-04-26T21:29:56.477 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-dr.ko 2024-04-26T21:29:56.477 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elecom.ko 2024-04-26T21:29:56.477 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elo.ko 2024-04-26T21:29:56.477 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-emsff.ko 2024-04-26T21:29:56.477 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gaff.ko 2024-04-26T21:29:56.477 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gt683r.ko 2024-04-26T21:29:56.477 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gyration.ko 2024-04-26T21:29:56.477 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-kbd.ko 2024-04-26T21:29:56.477 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-mouse.ko 2024-04-26T21:29:56.477 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtekff.ko 2024-04-26T21:29:56.477 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-hyperv.ko 2024-04-26T21:29:56.478 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-icade.ko 2024-04-26T21:29:56.478 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-keytouch.ko 2024-04-26T21:29:56.478 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-kye.ko 2024-04-26T21:29:56.478 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lcpower.ko 2024-04-26T21:29:56.478 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-led.ko 2024-04-26T21:29:56.478 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lenovo.ko 2024-04-26T21:29:56.478 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-dj.ko 2024-04-26T21:29:56.478 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-04-26T21:29:56.478 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-multitouch.ko 2024-04-26T21:29:56.478 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-ortek.ko 2024-04-26T21:29:56.478 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-penmount.ko 2024-04-26T21:29:56.478 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-petalynx.ko 2024-04-26T21:29:56.478 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-picolcd.ko 2024-04-26T21:29:56.478 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-pl.ko 2024-04-26T21:29:56.478 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-plantronics.ko 2024-04-26T21:29:56.479 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-primax.ko 2024-04-26T21:29:56.479 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-rmi.ko 2024-04-26T21:29:56.479 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-arvo.ko 2024-04-26T21:29:56.479 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-common.ko 2024-04-26T21:29:56.479 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-isku.ko 2024-04-26T21:29:56.479 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kone.ko 2024-04-26T21:29:56.479 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-04-26T21:29:56.479 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-konepure.ko 2024-04-26T21:29:56.479 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-04-26T21:29:56.479 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-lua.ko 2024-04-26T21:29:56.479 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-pyra.ko 2024-04-26T21:29:56.479 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-ryos.ko 2024-04-26T21:29:56.479 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-savu.ko 2024-04-26T21:29:56.479 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat.ko 2024-04-26T21:29:56.479 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-saitek.ko 2024-04-26T21:29:56.479 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-samsung.ko 2024-04-26T21:29:56.480 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sensor-hub.ko 2024-04-26T21:29:56.480 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sjoy.ko 2024-04-26T21:29:56.480 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sony.ko 2024-04-26T21:29:56.480 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-speedlink.ko 2024-04-26T21:29:56.480 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-steelseries.ko 2024-04-26T21:29:56.480 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sunplus.ko 2024-04-26T21:29:56.480 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-thrustmaster.ko 2024-04-26T21:29:56.480 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tivo.ko 2024-04-26T21:29:56.480 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tmff.ko 2024-04-26T21:29:56.480 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-topseed.ko 2024-04-26T21:29:56.480 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-twinhan.ko 2024-04-26T21:29:56.480 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-uclogic.ko 2024-04-26T21:29:56.480 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-waltop.ko 2024-04-26T21:29:56.480 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-wiimote.ko 2024-04-26T21:29:56.480 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-xinmo.ko 2024-04-26T21:29:56.480 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zpff.ko 2024-04-26T21:29:56.480 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zydacron.ko 2024-04-26T21:29:56.481 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/uhid.ko 2024-04-26T21:29:56.481 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/wacom.ko 2024-04-26T21:29:56.481 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv 2024-04-26T21:29:56.481 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_balloon.ko 2024-04-26T21:29:56.481 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_utils.ko 2024-04-26T21:29:56.481 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_vmbus.ko 2024-04-26T21:29:56.481 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon 2024-04-26T21:29:56.481 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru.ko 2024-04-26T21:29:56.481 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru3.ko 2024-04-26T21:29:56.481 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/acpi_power_meter.ko 2024-04-26T21:29:56.481 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7414.ko 2024-04-26T21:29:56.481 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7418.ko 2024-04-26T21:29:56.481 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adc128d818.ko 2024-04-26T21:29:56.481 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1025.ko 2024-04-26T21:29:56.481 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1026.ko 2024-04-26T21:29:56.481 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1029.ko 2024-04-26T21:29:56.482 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1031.ko 2024-04-26T21:29:56.482 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm9240.ko 2024-04-26T21:29:56.482 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ads7828.ko 2024-04-26T21:29:56.482 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7410.ko 2024-04-26T21:29:56.482 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7411.ko 2024-04-26T21:29:56.482 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7462.ko 2024-04-26T21:29:56.482 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7470.ko 2024-04-26T21:29:56.482 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7475.ko 2024-04-26T21:29:56.482 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7x10.ko 2024-04-26T21:29:56.482 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/amc6821.ko 2024-04-26T21:29:56.482 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/applesmc.ko 2024-04-26T21:29:56.482 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asb100.ko 2024-04-26T21:29:56.482 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asc7621.ko 2024-04-26T21:29:56.482 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asus_atk0110.ko 2024-04-26T21:29:56.482 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/atxp1.ko 2024-04-26T21:29:56.483 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/coretemp.ko 2024-04-26T21:29:56.483 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-04-26T21:29:56.483 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dme1737.ko 2024-04-26T21:29:56.483 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds1621.ko 2024-04-26T21:29:56.483 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds620.ko 2024-04-26T21:29:56.483 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc1403.ko 2024-04-26T21:29:56.483 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc6w201.ko 2024-04-26T21:29:56.483 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71805f.ko 2024-04-26T21:29:56.483 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71882fg.ko 2024-04-26T21:29:56.483 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f75375s.ko 2024-04-26T21:29:56.483 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fam15h_power.ko 2024-04-26T21:29:56.483 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fschmd.ko 2024-04-26T21:29:56.483 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g760a.ko 2024-04-26T21:29:56.484 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g762.ko 2024-04-26T21:29:56.484 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl518sm.ko 2024-04-26T21:29:56.484 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl520sm.ko 2024-04-26T21:29:56.484 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/hwmon-vid.ko 2024-04-26T21:29:56.484 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5500_temp.ko 2024-04-26T21:29:56.484 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5k_amb.ko 2024-04-26T21:29:56.484 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmaem.ko 2024-04-26T21:29:56.484 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmpex.ko 2024-04-26T21:29:56.484 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina209.ko 2024-04-26T21:29:56.484 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina2xx.ko 2024-04-26T21:29:56.484 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/it87.ko 2024-04-26T21:29:56.484 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k10temp.ko 2024-04-26T21:29:56.484 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k8temp.ko 2024-04-26T21:29:56.484 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lineage-pem.ko 2024-04-26T21:29:56.484 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm63.ko 2024-04-26T21:29:56.485 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm73.ko 2024-04-26T21:29:56.485 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm75.ko 2024-04-26T21:29:56.485 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm77.ko 2024-04-26T21:29:56.485 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm78.ko 2024-04-26T21:29:56.485 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm80.ko 2024-04-26T21:29:56.485 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm83.ko 2024-04-26T21:29:56.485 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm85.ko 2024-04-26T21:29:56.485 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm87.ko 2024-04-26T21:29:56.485 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm90.ko 2024-04-26T21:29:56.485 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm92.ko 2024-04-26T21:29:56.485 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm93.ko 2024-04-26T21:29:56.485 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95234.ko 2024-04-26T21:29:56.485 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95241.ko 2024-04-26T21:29:56.485 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95245.ko 2024-04-26T21:29:56.485 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc2945.ko 2024-04-26T21:29:56.485 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4151.ko 2024-04-26T21:29:56.486 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4215.ko 2024-04-26T21:29:56.486 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4222.ko 2024-04-26T21:29:56.486 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4245.ko 2024-04-26T21:29:56.486 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4260.ko 2024-04-26T21:29:56.486 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4261.ko 2024-04-26T21:29:56.486 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max16065.ko 2024-04-26T21:29:56.486 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1619.ko 2024-04-26T21:29:56.486 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1668.ko 2024-04-26T21:29:56.486 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max197.ko 2024-04-26T21:29:56.486 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6639.ko 2024-04-26T21:29:56.486 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6650.ko 2024-04-26T21:29:56.486 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6697.ko 2024-04-26T21:29:56.486 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/mcp3021.ko 2024-04-26T21:29:56.487 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6683.ko 2024-04-26T21:29:56.487 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775-core.ko 2024-04-26T21:29:56.487 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775.ko 2024-04-26T21:29:56.487 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct7802.ko 2024-04-26T21:29:56.487 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87360.ko 2024-04-26T21:29:56.487 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87427.ko 2024-04-26T21:29:56.487 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pcf8591.ko 2024-04-26T21:29:56.487 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus 2024-04-26T21:29:56.487 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-04-26T21:29:56.487 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-04-26T21:29:56.487 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-04-26T21:29:56.487 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max16064.ko 2024-04-26T21:29:56.487 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max34440.ko 2024-04-26T21:29:56.487 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max8688.ko 2024-04-26T21:29:56.487 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-04-26T21:29:56.487 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-04-26T21:29:56.488 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-04-26T21:29:56.488 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-04-26T21:29:56.488 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-04-26T21:29:56.488 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-04-26T21:29:56.488 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/powr1220.ko 2024-04-26T21:29:56.488 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5627.ko 2024-04-26T21:29:56.488 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5636.ko 2024-04-26T21:29:56.488 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch56xx-common.ko 2024-04-26T21:29:56.488 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht15.ko 2024-04-26T21:29:56.488 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht21.ko 2024-04-26T21:29:56.488 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/shtc1.ko 2024-04-26T21:29:56.488 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sis5595.ko 2024-04-26T21:29:56.488 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47b397.ko 2024-04-26T21:29:56.488 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m1.ko 2024-04-26T21:29:56.489 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m192.ko 2024-04-26T21:29:56.489 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/thmc50.ko 2024-04-26T21:29:56.489 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp102.ko 2024-04-26T21:29:56.489 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp103.ko 2024-04-26T21:29:56.489 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp401.ko 2024-04-26T21:29:56.489 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp421.ko 2024-04-26T21:29:56.489 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via-cputemp.ko 2024-04-26T21:29:56.489 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via686a.ko 2024-04-26T21:29:56.489 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt1211.ko 2024-04-26T21:29:56.489 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt8231.ko 2024-04-26T21:29:56.489 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627ehf.ko 2024-04-26T21:29:56.489 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627hf.ko 2024-04-26T21:29:56.489 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83781d.ko 2024-04-26T21:29:56.489 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83791d.ko 2024-04-26T21:29:56.489 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83792d.ko 2024-04-26T21:29:56.490 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83793.ko 2024-04-26T21:29:56.490 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83795.ko 2024-04-26T21:29:56.490 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l785ts.ko 2024-04-26T21:29:56.490 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l786ng.ko 2024-04-26T21:29:56.490 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c 2024-04-26T21:29:56.490 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos 2024-04-26T21:29:56.490 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-04-26T21:29:56.490 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-04-26T21:29:56.490 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses 2024-04-26T21:29:56.490 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-04-26T21:29:56.490 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-04-26T21:29:56.490 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-04-26T21:29:56.490 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-04-26T21:29:56.490 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-04-26T21:29:56.491 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-04-26T21:29:56.491 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-04-26T21:29:56.491 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-04-26T21:29:56.491 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-i801.ko 2024-04-26T21:29:56.491 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-isch.ko 2024-04-26T21:29:56.491 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-04-26T21:29:56.491 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-04-26T21:29:56.491 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-04-26T21:29:56.491 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-parport.ko 2024-04-26T21:29:56.491 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-04-26T21:29:56.491 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-04-26T21:29:56.491 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-04-26T21:29:56.491 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-04-26T21:29:56.491 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-04-26T21:29:56.491 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-04-26T21:29:56.491 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-via.ko 2024-04-26T21:29:56.492 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-04-26T21:29:56.492 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-04-26T21:29:56.492 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-dev.ko 2024-04-26T21:29:56.492 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-mux.ko 2024-04-26T21:29:56.492 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-04-26T21:29:56.492 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-smbus.ko 2024-04-26T21:29:56.492 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-stub.ko 2024-04-26T21:29:56.492 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband 2024-04-26T21:29:56.492 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core 2024-04-26T21:29:56.492 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_cm.ko 2024-04-26T21:29:56.492 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_core.ko 2024-04-26T21:29:56.492 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_umad.ko 2024-04-26T21:29:56.492 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-04-26T21:29:56.492 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/iw_cm.ko 2024-04-26T21:29:56.493 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_cm.ko 2024-04-26T21:29:56.493 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-04-26T21:29:56.493 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw 2024-04-26T21:29:56.493 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4 2024-04-26T21:29:56.493 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-04-26T21:29:56.493 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4 2024-04-26T21:29:56.493 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-04-26T21:29:56.493 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5 2024-04-26T21:29:56.493 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-04-26T21:29:56.493 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca 2024-04-26T21:29:56.493 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-04-26T21:29:56.493 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp 2024-04-26T21:29:56.493 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib 2024-04-26T21:29:56.493 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-04-26T21:29:56.494 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser 2024-04-26T21:29:56.494 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-04-26T21:29:56.494 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert 2024-04-26T21:29:56.494 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-04-26T21:29:56.494 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp 2024-04-26T21:29:56.494 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-04-26T21:29:56.494 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt 2024-04-26T21:29:56.494 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-04-26T21:29:56.494 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input 2024-04-26T21:29:56.494 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport 2024-04-26T21:29:56.494 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-04-26T21:29:56.494 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/fm801-gp.ko 2024-04-26T21:29:56.494 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/gameport.ko 2024-04-26T21:29:56.495 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/joydev.ko 2024-04-26T21:29:56.495 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard 2024-04-26T21:29:56.495 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/gpio_keys.ko 2024-04-26T21:29:56.495 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse 2024-04-26T21:29:56.495 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/appletouch.ko 2024-04-26T21:29:56.495 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/bcm5974.ko 2024-04-26T21:29:56.495 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/cyapatp.ko 2024-04-26T21:29:56.495 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/elan_i2c.ko 2024-04-26T21:29:56.495 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/sermouse.ko 2024-04-26T21:29:56.495 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-04-26T21:29:56.495 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_usb.ko 2024-04-26T21:29:56.495 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/vsxxxaa.ko 2024-04-26T21:29:56.495 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4 2024-04-26T21:29:56.495 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4/rmi_core.ko 2024-04-26T21:29:56.495 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio 2024-04-26T21:29:56.496 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/altera_ps2.ko 2024-04-26T21:29:56.496 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/arc_ps2.ko 2024-04-26T21:29:56.496 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-04-26T21:29:56.496 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/serio_raw.ko 2024-04-26T21:29:56.496 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/sparse-keymap.ko 2024-04-26T21:29:56.496 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds 2024-04-26T21:29:56.496 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-blinkm.ko 2024-04-26T21:29:56.496 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lm3530.ko 2024-04-26T21:29:56.496 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lp3944.ko 2024-04-26T21:29:56.496 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-ss4200.ko 2024-04-26T21:29:56.496 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger 2024-04-26T21:29:56.496 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-04-26T21:29:56.496 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-04-26T21:29:56.496 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-04-26T21:29:56.496 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-04-26T21:29:56.497 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-04-26T21:29:56.497 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-04-26T21:29:56.497 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-04-26T21:29:56.497 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-04-26T21:29:56.497 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-04-26T21:29:56.497 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md 2024-04-26T21:29:56.497 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache 2024-04-26T21:29:56.497 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache/bcache.ko 2024-04-26T21:29:56.497 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-bio-prison.ko 2024-04-26T21:29:56.497 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache-smq.ko 2024-04-26T21:29:56.497 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache.ko 2024-04-26T21:29:56.497 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-crypt.ko 2024-04-26T21:29:56.497 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-delay.ko 2024-04-26T21:29:56.497 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-flakey.ko 2024-04-26T21:29:56.497 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-log-userspace.ko 2024-04-26T21:29:56.497 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-multipath.ko 2024-04-26T21:29:56.498 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-queue-length.ko 2024-04-26T21:29:56.498 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-raid.ko 2024-04-26T21:29:56.498 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-round-robin.ko 2024-04-26T21:29:56.498 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-service-time.ko 2024-04-26T21:29:56.498 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-switch.ko 2024-04-26T21:29:56.498 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-thin-pool.ko 2024-04-26T21:29:56.498 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-verity.ko 2024-04-26T21:29:56.498 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data 2024-04-26T21:29:56.498 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-04-26T21:29:56.498 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid0.ko 2024-04-26T21:29:56.498 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid1.ko 2024-04-26T21:29:56.498 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid10.ko 2024-04-26T21:29:56.498 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid456.ko 2024-04-26T21:29:56.498 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message 2024-04-26T21:29:56.499 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion 2024-04-26T21:29:56.499 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptbase.ko 2024-04-26T21:29:56.499 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptctl.ko 2024-04-26T21:29:56.499 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptfc.ko 2024-04-26T21:29:56.499 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptlan.ko 2024-04-26T21:29:56.499 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptsas.ko 2024-04-26T21:29:56.499 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptscsih.ko 2024-04-26T21:29:56.499 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptspi.ko 2024-04-26T21:29:56.499 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd 2024-04-26T21:29:56.499 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_ich.ko 2024-04-26T21:29:56.499 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_sch.ko 2024-04-26T21:29:56.499 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/mfd-core.ko 2024-04-26T21:29:56.499 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/sm501.ko 2024-04-26T21:29:56.499 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/viperboard.ko 2024-04-26T21:29:56.499 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/vx855.ko 2024-04-26T21:29:56.499 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/wl1273-core.ko 2024-04-26T21:29:56.500 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc 2024-04-26T21:29:56.500 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl 2024-04-26T21:29:56.500 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-04-26T21:29:56.500 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds9802als.ko 2024-04-26T21:29:56.500 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds990x.ko 2024-04-26T21:29:56.500 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/bh1770glc.ko 2024-04-26T21:29:56.500 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710 2024-04-26T21:29:56.500 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710/cb710.ko 2024-04-26T21:29:56.500 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom 2024-04-26T21:29:56.500 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/at24.ko 2024-04-26T21:29:56.500 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-04-26T21:29:56.500 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/max6875.ko 2024-04-26T21:29:56.500 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/enclosure.ko 2024-04-26T21:29:56.500 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/hpilo.ko 2024-04-26T21:29:56.501 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29003.ko 2024-04-26T21:29:56.501 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29020.ko 2024-04-26T21:29:56.501 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d 2024-04-26T21:29:56.501 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-04-26T21:29:56.501 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-04-26T21:29:56.501 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei 2024-04-26T21:29:56.501 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-me.ko 2024-04-26T21:29:56.501 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-txe.ko 2024-04-26T21:29:56.501 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei.ko 2024-04-26T21:29:56.501 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru 2024-04-26T21:29:56.501 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru/gru.ko 2024-04-26T21:29:56.501 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp 2024-04-26T21:29:56.501 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xp.ko 2024-04-26T21:29:56.502 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpc.ko 2024-04-26T21:29:56.502 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-04-26T21:29:56.502 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_7xx1.ko 2024-04-26T21:29:56.502 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_core.ko 2024-04-26T21:29:56.502 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tsl2550.ko 2024-04-26T21:29:56.502 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_balloon.ko 2024-04-26T21:29:56.502 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci 2024-04-26T21:29:56.502 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-04-26T21:29:56.502 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net 2024-04-26T21:29:56.502 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding 2024-04-26T21:29:56.502 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding/bonding.ko 2024-04-26T21:29:56.502 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa 2024-04-26T21:29:56.502 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53 2024-04-26T21:29:56.503 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53/b53_common.ko 2024-04-26T21:29:56.503 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/bcm-sf2.ko 2024-04-26T21:29:56.503 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6060.ko 2024-04-26T21:29:56.503 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx 2024-04-26T21:29:56.503 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-04-26T21:29:56.503 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dummy.ko 2024-04-26T21:29:56.503 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/eql.ko 2024-04-26T21:29:56.503 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet 2024-04-26T21:29:56.503 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com 2024-04-26T21:29:56.503 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-04-26T21:29:56.503 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-04-26T21:29:56.503 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-04-26T21:29:56.503 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-04-26T21:29:56.503 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390 2024-04-26T21:29:56.504 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/8390.ko 2024-04-26T21:29:56.504 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-04-26T21:29:56.504 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-04-26T21:29:56.504 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-04-26T21:29:56.504 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec 2024-04-26T21:29:56.504 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-04-26T21:29:56.504 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere 2024-04-26T21:29:56.504 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere/et131x.ko 2024-04-26T21:29:56.504 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon 2024-04-26T21:29:56.504 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-04-26T21:29:56.504 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera 2024-04-26T21:29:56.504 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-04-26T21:29:56.504 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd 2024-04-26T21:29:56.504 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-04-26T21:29:56.504 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-04-26T21:29:56.505 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-04-26T21:29:56.505 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros 2024-04-26T21:29:56.505 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx 2024-04-26T21:29:56.505 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-04-26T21:29:56.505 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c 2024-04-26T21:29:56.505 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-04-26T21:29:56.505 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e 2024-04-26T21:29:56.505 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-04-26T21:29:56.505 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx 2024-04-26T21:29:56.505 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-04-26T21:29:56.505 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-04-26T21:29:56.505 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom 2024-04-26T21:29:56.505 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-04-26T21:29:56.505 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-04-26T21:29:56.506 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-04-26T21:29:56.506 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-04-26T21:29:56.506 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-04-26T21:29:56.506 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet 2024-04-26T21:29:56.506 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-04-26T21:29:56.506 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-04-26T21:29:56.506 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade 2024-04-26T21:29:56.506 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna 2024-04-26T21:29:56.506 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-04-26T21:29:56.506 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio 2024-04-26T21:29:56.506 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb 2024-04-26T21:29:56.507 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-04-26T21:29:56.507 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-04-26T21:29:56.507 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-04-26T21:29:56.507 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-04-26T21:29:56.507 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-04-26T21:29:56.507 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-04-26T21:29:56.507 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-04-26T21:29:56.507 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-04-26T21:29:56.507 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-04-26T21:29:56.507 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco 2024-04-26T21:29:56.507 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic 2024-04-26T21:29:56.507 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-04-26T21:29:56.507 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec 2024-04-26T21:29:56.508 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip 2024-04-26T21:29:56.508 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-04-26T21:29:56.508 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-04-26T21:29:56.508 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-04-26T21:29:56.508 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-04-26T21:29:56.508 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-04-26T21:29:56.508 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-04-26T21:29:56.508 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink 2024-04-26T21:29:56.508 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-04-26T21:29:56.508 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-04-26T21:29:56.508 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dnet.ko 2024-04-26T21:29:56.508 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex 2024-04-26T21:29:56.508 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet 2024-04-26T21:29:56.508 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-04-26T21:29:56.508 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ethoc.ko 2024-04-26T21:29:56.509 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/fealnx.ko 2024-04-26T21:29:56.509 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel 2024-04-26T21:29:56.509 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e100.ko 2024-04-26T21:29:56.509 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000 2024-04-26T21:29:56.509 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-04-26T21:29:56.509 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e 2024-04-26T21:29:56.509 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-04-26T21:29:56.509 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k 2024-04-26T21:29:56.509 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-04-26T21:29:56.509 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e 2024-04-26T21:29:56.509 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-04-26T21:29:56.509 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf 2024-04-26T21:29:56.509 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-04-26T21:29:56.509 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb 2024-04-26T21:29:56.509 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-04-26T21:29:56.510 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf 2024-04-26T21:29:56.510 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-04-26T21:29:56.510 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe 2024-04-26T21:29:56.510 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-04-26T21:29:56.510 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf 2024-04-26T21:29:56.510 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-04-26T21:29:56.510 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/jme.ko 2024-04-26T21:29:56.510 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell 2024-04-26T21:29:56.510 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-04-26T21:29:56.510 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/skge.ko 2024-04-26T21:29:56.510 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-04-26T21:29:56.510 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox 2024-04-26T21:29:56.510 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4 2024-04-26T21:29:56.510 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-04-26T21:29:56.511 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-04-26T21:29:56.511 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5 2024-04-26T21:29:56.511 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-04-26T21:29:56.511 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-04-26T21:29:56.511 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel 2024-04-26T21:29:56.511 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-04-26T21:29:56.511 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom 2024-04-26T21:29:56.511 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge 2024-04-26T21:29:56.511 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-04-26T21:29:56.511 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi 2024-04-26T21:29:56.511 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-04-26T21:29:56.511 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-04-26T21:29:56.511 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion 2024-04-26T21:29:56.511 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-04-26T21:29:56.512 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia 2024-04-26T21:29:56.512 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-04-26T21:29:56.512 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines 2024-04-26T21:29:56.512 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-04-26T21:29:56.512 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-04-26T21:29:56.512 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic 2024-04-26T21:29:56.512 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen 2024-04-26T21:29:56.512 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-04-26T21:29:56.512 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-04-26T21:29:56.512 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-04-26T21:29:56.512 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-04-26T21:29:56.512 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc 2024-04-26T21:29:56.512 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-04-26T21:29:56.512 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek 2024-04-26T21:29:56.512 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-04-26T21:29:56.512 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-04-26T21:29:56.513 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/atp.ko 2024-04-26T21:29:56.513 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-04-26T21:29:56.513 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker 2024-04-26T21:29:56.513 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-04-26T21:29:56.513 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc 2024-04-26T21:29:56.513 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-04-26T21:29:56.513 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan 2024-04-26T21:29:56.513 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-04-26T21:29:56.513 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis 2024-04-26T21:29:56.513 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis190.ko 2024-04-26T21:29:56.513 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis900.ko 2024-04-26T21:29:56.513 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc 2024-04-26T21:29:56.513 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-04-26T21:29:56.513 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-04-26T21:29:56.513 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-04-26T21:29:56.514 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-04-26T21:29:56.514 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro 2024-04-26T21:29:56.514 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac 2024-04-26T21:29:56.514 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-04-26T21:29:56.514 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-04-26T21:29:56.514 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun 2024-04-26T21:29:56.514 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/cassini.ko 2024-04-26T21:29:56.514 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/niu.ko 2024-04-26T21:29:56.514 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sungem.ko 2024-04-26T21:29:56.514 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-04-26T21:29:56.514 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti 2024-04-26T21:29:56.514 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-04-26T21:29:56.514 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti 2024-04-26T21:29:56.514 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti/tlan.ko 2024-04-26T21:29:56.514 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via 2024-04-26T21:29:56.515 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-04-26T21:29:56.515 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-04-26T21:29:56.515 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet 2024-04-26T21:29:56.515 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-04-26T21:29:56.515 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-04-26T21:29:56.515 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom 2024-04-26T21:29:56.515 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-04-26T21:29:56.515 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/geneve.ko 2024-04-26T21:29:56.515 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv 2024-04-26T21:29:56.515 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-04-26T21:29:56.515 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154 2024-04-26T21:29:56.515 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154/fakelb.ko 2024-04-26T21:29:56.515 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ifb.ko 2024-04-26T21:29:56.515 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan 2024-04-26T21:29:56.515 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan/ipvlan.ko 2024-04-26T21:29:56.516 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvlan.ko 2024-04-26T21:29:56.516 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvtap.ko 2024-04-26T21:29:56.516 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio 2024-04-26T21:29:56.516 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio.ko 2024-04-26T21:29:56.516 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-04-26T21:29:56.516 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-04-26T21:29:56.516 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-regmap.ko 2024-04-26T21:29:56.516 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mii.ko 2024-04-26T21:29:56.516 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/net_failover.ko 2024-04-26T21:29:56.516 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/netconsole.ko 2024-04-26T21:29:56.516 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/nlmon.ko 2024-04-26T21:29:56.516 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs 2024-04-26T21:29:56.516 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs-lynx.ko 2024-04-26T21:29:56.516 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-04-26T21:29:56.516 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy 2024-04-26T21:29:56.516 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/amd.ko 2024-04-26T21:29:56.517 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-04-26T21:29:56.517 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-04-26T21:29:56.517 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm7xxx.ko 2024-04-26T21:29:56.517 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm87xx.ko 2024-04-26T21:29:56.517 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/broadcom.ko 2024-04-26T21:29:56.517 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/cicada.ko 2024-04-26T21:29:56.517 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/davicom.ko 2024-04-26T21:29:56.517 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/dp83640.ko 2024-04-26T21:29:56.517 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/et1011c.ko 2024-04-26T21:29:56.517 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/icplus.ko 2024-04-26T21:29:56.517 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/lxt.ko 2024-04-26T21:29:56.517 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/marvell.ko 2024-04-26T21:29:56.517 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/micrel.ko 2024-04-26T21:29:56.517 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/national.ko 2024-04-26T21:29:56.517 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/phylink.ko 2024-04-26T21:29:56.517 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/qsemi.ko 2024-04-26T21:29:56.518 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/realtek.ko 2024-04-26T21:29:56.518 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/smsc.ko 2024-04-26T21:29:56.518 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/ste10Xp.ko 2024-04-26T21:29:56.518 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/vitesse.ko 2024-04-26T21:29:56.518 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp 2024-04-26T21:29:56.518 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/bsd_comp.ko 2024-04-26T21:29:56.518 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_async.ko 2024-04-26T21:29:56.518 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_deflate.ko 2024-04-26T21:29:56.518 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_generic.ko 2024-04-26T21:29:56.518 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_mppe.ko 2024-04-26T21:29:56.518 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_synctty.ko 2024-04-26T21:29:56.518 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppoe.ko 2024-04-26T21:29:56.518 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppox.ko 2024-04-26T21:29:56.519 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pptp.ko 2024-04-26T21:29:56.519 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip 2024-04-26T21:29:56.519 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slhc.ko 2024-04-26T21:29:56.519 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slip.ko 2024-04-26T21:29:56.519 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/sungem_phy.ko 2024-04-26T21:29:56.519 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tap.ko 2024-04-26T21:29:56.519 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team 2024-04-26T21:29:56.519 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team.ko 2024-04-26T21:29:56.519 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_activebackup.ko 2024-04-26T21:29:56.519 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_broadcast.ko 2024-04-26T21:29:56.519 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-04-26T21:29:56.519 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_random.ko 2024-04-26T21:29:56.519 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-04-26T21:29:56.519 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tun.ko 2024-04-26T21:29:56.520 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/veth.ko 2024-04-26T21:29:56.520 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/virtio_net.ko 2024-04-26T21:29:56.520 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3 2024-04-26T21:29:56.520 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-04-26T21:29:56.520 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan 2024-04-26T21:29:56.520 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan/vxlan.ko 2024-04-26T21:29:56.520 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback 2024-04-26T21:29:56.520 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback/xen-netback.ko 2024-04-26T21:29:56.520 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netfront.ko 2024-04-26T21:29:56.520 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb 2024-04-26T21:29:56.520 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb/ntb.ko 2024-04-26T21:29:56.520 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme 2024-04-26T21:29:56.520 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host 2024-04-26T21:29:56.520 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme-core.ko 2024-04-26T21:29:56.520 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme.ko 2024-04-26T21:29:56.521 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport 2024-04-26T21:29:56.521 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport.ko 2024-04-26T21:29:56.521 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_cs.ko 2024-04-26T21:29:56.521 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_pc.ko 2024-04-26T21:29:56.521 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_serial.ko 2024-04-26T21:29:56.521 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci 2024-04-26T21:29:56.521 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug 2024-04-26T21:29:56.521 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-04-26T21:29:56.521 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie 2024-04-26T21:29:56.521 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie/aer_inject.ko 2024-04-26T21:29:56.521 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/xen-pcifront.ko 2024-04-26T21:29:56.521 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia 2024-04-26T21:29:56.521 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/i82092.ko 2024-04-26T21:29:56.521 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/pd6729.ko 2024-04-26T21:29:56.521 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/yenta_socket.ko 2024-04-26T21:29:56.522 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl 2024-04-26T21:29:56.522 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel 2024-04-26T21:29:56.522 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-04-26T21:29:56.522 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform 2024-04-26T21:29:56.522 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86 2024-04-26T21:29:56.522 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acer-wmi.ko 2024-04-26T21:29:56.522 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acerhdf.ko 2024-04-26T21:29:56.522 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/apple-gmux.ko 2024-04-26T21:29:56.522 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-laptop.ko 2024-04-26T21:29:56.522 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-04-26T21:29:56.522 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-wmi.ko 2024-04-26T21:29:56.522 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/classmate-laptop.ko 2024-04-26T21:29:56.522 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-04-26T21:29:56.523 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-04-26T21:29:56.523 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-04-26T21:29:56.523 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-04-26T21:29:56.523 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/hdaps.ko 2024-04-26T21:29:56.523 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel 2024-04-26T21:29:56.523 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-04-26T21:29:56.523 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel_ips.ko 2024-04-26T21:29:56.523 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/msi-wmi.ko 2024-04-26T21:29:56.523 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/mxm-wmi.ko 2024-04-26T21:29:56.523 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-04-26T21:29:56.523 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-laptop.ko 2024-04-26T21:29:56.523 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-q10.ko 2024-04-26T21:29:56.523 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/topstar-laptop.ko 2024-04-26T21:29:56.524 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-04-26T21:29:56.524 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_haps.ko 2024-04-26T21:29:56.524 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi-bmof.ko 2024-04-26T21:29:56.524 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi.ko 2024-04-26T21:29:56.524 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap 2024-04-26T21:29:56.524 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_common.ko 2024-04-26T21:29:56.524 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_msr.ko 2024-04-26T21:29:56.524 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps 2024-04-26T21:29:56.524 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients 2024-04-26T21:29:56.524 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-gpio.ko 2024-04-26T21:29:56.524 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-ldisc.ko 2024-04-26T21:29:56.524 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps_parport.ko 2024-04-26T21:29:56.524 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/pps_core.ko 2024-04-26T21:29:56.524 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp 2024-04-26T21:29:56.524 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp.ko 2024-04-26T21:29:56.525 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp_kvm.ko 2024-04-26T21:29:56.525 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras 2024-04-26T21:29:56.525 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd 2024-04-26T21:29:56.525 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl 2024-04-26T21:29:56.525 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-04-26T21:29:56.525 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/fmpm.ko 2024-04-26T21:29:56.525 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc 2024-04-26T21:29:56.525 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-bq32k.ko 2024-04-26T21:29:56.525 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1286.ko 2024-04-26T21:29:56.525 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1307.ko 2024-04-26T21:29:56.525 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1374.ko 2024-04-26T21:29:56.525 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1511.ko 2024-04-26T21:29:56.526 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1553.ko 2024-04-26T21:29:56.526 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1672.ko 2024-04-26T21:29:56.526 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1742.ko 2024-04-26T21:29:56.526 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds2404.ko 2024-04-26T21:29:56.526 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds3232.ko 2024-04-26T21:29:56.526 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-em3027.ko 2024-04-26T21:29:56.526 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-fm3130.ko 2024-04-26T21:29:56.526 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl12022.ko 2024-04-26T21:29:56.526 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl1208.ko 2024-04-26T21:29:56.526 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m41t80.ko 2024-04-26T21:29:56.526 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t35.ko 2024-04-26T21:29:56.526 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t59.ko 2024-04-26T21:29:56.527 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-max6900.ko 2024-04-26T21:29:56.527 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-msm6242.ko 2024-04-26T21:29:56.527 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf2127.ko 2024-04-26T21:29:56.527 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf85063.ko 2024-04-26T21:29:56.527 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8523.ko 2024-04-26T21:29:56.527 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8563.ko 2024-04-26T21:29:56.527 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8583.ko 2024-04-26T21:29:56.527 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rp5c01.ko 2024-04-26T21:29:56.527 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rs5c372.ko 2024-04-26T21:29:56.527 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-04-26T21:29:56.527 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8025.ko 2024-04-26T21:29:56.527 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8581.ko 2024-04-26T21:29:56.527 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-04-26T21:29:56.528 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-x1205.ko 2024-04-26T21:29:56.528 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi 2024-04-26T21:29:56.528 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-9xxx.ko 2024-04-26T21:29:56.528 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-sas.ko 2024-04-26T21:29:56.528 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-xxxx.ko 2024-04-26T21:29:56.528 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/BusLogic.ko 2024-04-26T21:29:56.528 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/a100u2w.ko 2024-04-26T21:29:56.528 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid 2024-04-26T21:29:56.528 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid/aacraid.ko 2024-04-26T21:29:56.528 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/advansys.ko 2024-04-26T21:29:56.528 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx 2024-04-26T21:29:56.528 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-04-26T21:29:56.528 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-04-26T21:29:56.528 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/am53c974.ko 2024-04-26T21:29:56.528 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr 2024-04-26T21:29:56.529 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-04-26T21:29:56.529 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/atp870u.ko 2024-04-26T21:29:56.529 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi 2024-04-26T21:29:56.529 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-04-26T21:29:56.529 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa 2024-04-26T21:29:56.529 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa/bfa.ko 2024-04-26T21:29:56.529 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc 2024-04-26T21:29:56.529 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-04-26T21:29:56.529 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i 2024-04-26T21:29:56.529 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-04-26T21:29:56.529 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ch.ko 2024-04-26T21:29:56.529 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor 2024-04-26T21:29:56.529 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor/csiostor.ko 2024-04-26T21:29:56.529 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi 2024-04-26T21:29:56.529 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i 2024-04-26T21:29:56.529 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-04-26T21:29:56.530 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i 2024-04-26T21:29:56.530 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-04-26T21:29:56.530 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-04-26T21:29:56.530 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dc395x.ko 2024-04-26T21:29:56.530 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler 2024-04-26T21:29:56.530 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-04-26T21:29:56.530 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-04-26T21:29:56.530 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-04-26T21:29:56.530 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-04-26T21:29:56.530 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dmx3191d.ko 2024-04-26T21:29:56.530 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r 2024-04-26T21:29:56.530 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r/esas2r.ko 2024-04-26T21:29:56.530 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esp_scsi.ko 2024-04-26T21:29:56.530 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe 2024-04-26T21:29:56.530 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/fcoe.ko 2024-04-26T21:29:56.531 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-04-26T21:29:56.531 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic 2024-04-26T21:29:56.531 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic/fnic.ko 2024-04-26T21:29:56.531 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hpsa.ko 2024-04-26T21:29:56.531 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hptiop.ko 2024-04-26T21:29:56.531 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hv_storvsc.ko 2024-04-26T21:29:56.531 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/initio.ko 2024-04-26T21:29:56.531 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ipr.ko 2024-04-26T21:29:56.531 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ips.ko 2024-04-26T21:29:56.531 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci 2024-04-26T21:29:56.531 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci/isci.ko 2024-04-26T21:29:56.531 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-04-26T21:29:56.531 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_tcp.ko 2024-04-26T21:29:56.531 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc 2024-04-26T21:29:56.532 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc/libfc.ko 2024-04-26T21:29:56.532 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi.ko 2024-04-26T21:29:56.532 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi_tcp.ko 2024-04-26T21:29:56.532 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas 2024-04-26T21:29:56.532 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas/libsas.ko 2024-04-26T21:29:56.532 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc 2024-04-26T21:29:56.532 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc/lpfc.ko 2024-04-26T21:29:56.532 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid 2024-04-26T21:29:56.532 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid.ko 2024-04-26T21:29:56.532 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-04-26T21:29:56.532 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-04-26T21:29:56.532 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-04-26T21:29:56.532 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas 2024-04-26T21:29:56.532 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-04-26T21:29:56.532 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas 2024-04-26T21:29:56.533 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas/mvsas.ko 2024-04-26T21:29:56.533 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvumi.ko 2024-04-26T21:29:56.533 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001 2024-04-26T21:29:56.533 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-04-26T21:29:56.533 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pmcraid.ko 2024-04-26T21:29:56.533 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla1280.ko 2024-04-26T21:29:56.533 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx 2024-04-26T21:29:56.533 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-04-26T21:29:56.533 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-04-26T21:29:56.533 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx 2024-04-26T21:29:56.533 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-04-26T21:29:56.533 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/raid_class.ko 2024-04-26T21:29:56.533 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_debug.ko 2024-04-26T21:29:56.533 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_fc.ko 2024-04-26T21:29:56.533 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-04-26T21:29:56.533 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_sas.ko 2024-04-26T21:29:56.534 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_spi.ko 2024-04-26T21:29:56.534 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_srp.ko 2024-04-26T21:29:56.534 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ses.ko 2024-04-26T21:29:56.534 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/st.ko 2024-04-26T21:29:56.534 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/stex.ko 2024-04-26T21:29:56.534 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2 2024-04-26T21:29:56.534 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-04-26T21:29:56.534 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/virtio_scsi.ko 2024-04-26T21:29:56.534 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/vmw_pvscsi.ko 2024-04-26T21:29:56.534 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/wd719x.ko 2024-04-26T21:29:56.534 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb 2024-04-26T21:29:56.534 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb/ssb.ko 2024-04-26T21:29:56.534 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target 2024-04-26T21:29:56.534 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi 2024-04-26T21:29:56.534 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-04-26T21:29:56.535 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback 2024-04-26T21:29:56.535 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback/tcm_loop.ko 2024-04-26T21:29:56.535 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_file.ko 2024-04-26T21:29:56.535 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_iblock.ko 2024-04-26T21:29:56.535 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_mod.ko 2024-04-26T21:29:56.535 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_pscsi.ko 2024-04-26T21:29:56.535 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_user.ko 2024-04-26T21:29:56.535 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc 2024-04-26T21:29:56.535 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-04-26T21:29:56.535 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal 2024-04-26T21:29:56.535 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel 2024-04-26T21:29:56.535 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal 2024-04-26T21:29:56.535 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-04-26T21:29:56.535 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-04-26T21:29:56.535 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-04-26T21:29:56.536 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-04-26T21:29:56.536 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-04-26T21:29:56.536 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-04-26T21:29:56.536 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-04-26T21:29:56.536 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-04-26T21:29:56.536 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-04-26T21:29:56.536 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-04-26T21:29:56.536 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-04-26T21:29:56.536 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-04-26T21:29:56.536 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-04-26T21:29:56.536 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-04-26T21:29:56.536 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-04-26T21:29:56.536 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-04-26T21:29:56.537 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-04-26T21:29:56.537 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-04-26T21:29:56.537 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty 2024-04-26T21:29:56.537 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless 2024-04-26T21:29:56.537 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-04-26T21:29:56.537 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_gsm.ko 2024-04-26T21:29:56.537 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_hdlc.ko 2024-04-26T21:29:56.537 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/nozomi.ko 2024-04-26T21:29:56.537 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial 2024-04-26T21:29:56.537 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250 2024-04-26T21:29:56.537 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-04-26T21:29:56.537 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/arc_uart.ko 2024-04-26T21:29:56.537 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm 2024-04-26T21:29:56.537 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm/jsm.ko 2024-04-26T21:29:56.537 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs 2024-04-26T21:29:56.538 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core 2024-04-26T21:29:56.538 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core/ufshcd-core.ko 2024-04-26T21:29:56.538 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host 2024-04-26T21:29:56.538 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-04-26T21:29:56.538 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio 2024-04-26T21:29:56.538 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio.ko 2024-04-26T21:29:56.538 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_aec.ko 2024-04-26T21:29:56.538 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_cif.ko 2024-04-26T21:29:56.538 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_pci_generic.ko 2024-04-26T21:29:56.538 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_sercos3.ko 2024-04-26T21:29:56.538 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb 2024-04-26T21:29:56.538 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class 2024-04-26T21:29:56.538 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-acm.ko 2024-04-26T21:29:56.538 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-wdm.ko 2024-04-26T21:29:56.538 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usblp.ko 2024-04-26T21:29:56.539 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usbtmc.ko 2024-04-26T21:29:56.539 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host 2024-04-26T21:29:56.539 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/sl811-hcd.ko 2024-04-26T21:29:56.539 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image 2024-04-26T21:29:56.539 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/mdc800.ko 2024-04-26T21:29:56.539 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/microtek.ko 2024-04-26T21:29:56.539 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc 2024-04-26T21:29:56.539 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/adutux.ko 2024-04-26T21:29:56.539 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/appledisplay.ko 2024-04-26T21:29:56.539 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi26.ko 2024-04-26T21:29:56.539 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi62.ko 2024-04-26T21:29:56.539 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ezusb.ko 2024-04-26T21:29:56.539 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/idmouse.ko 2024-04-26T21:29:56.539 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/iowarrior.ko 2024-04-26T21:29:56.539 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/isight_firmware.ko 2024-04-26T21:29:56.539 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ldusb.ko 2024-04-26T21:29:56.540 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/legousbtower.ko 2024-04-26T21:29:56.540 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga 2024-04-26T21:29:56.540 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-04-26T21:29:56.540 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/trancevibrator.ko 2024-04-26T21:29:56.540 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usb3503.ko 2024-04-26T21:29:56.540 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usblcd.ko 2024-04-26T21:29:56.540 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usbsevseg.ko 2024-04-26T21:29:56.540 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/uss720.ko 2024-04-26T21:29:56.540 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/yurex.ko 2024-04-26T21:29:56.540 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy 2024-04-26T21:29:56.540 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy/phy-generic.ko 2024-04-26T21:29:56.540 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial 2024-04-26T21:29:56.540 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/aircable.ko 2024-04-26T21:29:56.540 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ark3116.ko 2024-04-26T21:29:56.540 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/belkin_sa.ko 2024-04-26T21:29:56.540 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ch341.ko 2024-04-26T21:29:56.541 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cp210x.ko 2024-04-26T21:29:56.541 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cyberjack.ko 2024-04-26T21:29:56.541 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cypress_m8.ko 2024-04-26T21:29:56.541 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/digi_acceleport.ko 2024-04-26T21:29:56.541 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/empeg.ko 2024-04-26T21:29:56.541 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ftdi_sio.ko 2024-04-26T21:29:56.541 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/garmin_gps.ko 2024-04-26T21:29:56.541 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_edgeport.ko 2024-04-26T21:29:56.541 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_ti.ko 2024-04-26T21:29:56.541 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipaq.ko 2024-04-26T21:29:56.541 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipw.ko 2024-04-26T21:29:56.541 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ir-usb.ko 2024-04-26T21:29:56.541 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-04-26T21:29:56.541 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan.ko 2024-04-26T21:29:56.542 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan_pda.ko 2024-04-26T21:29:56.542 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kl5kusb105.ko 2024-04-26T21:29:56.542 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kobil_sct.ko 2024-04-26T21:29:56.542 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mct_u232.ko 2024-04-26T21:29:56.542 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7720.ko 2024-04-26T21:29:56.542 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7840.ko 2024-04-26T21:29:56.542 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/navman.ko 2024-04-26T21:29:56.542 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/omninet.ko 2024-04-26T21:29:56.542 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/opticon.ko 2024-04-26T21:29:56.542 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/option.ko 2024-04-26T21:29:56.542 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/oti6858.ko 2024-04-26T21:29:56.542 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/pl2303.ko 2024-04-26T21:29:56.542 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcaux.ko 2024-04-26T21:29:56.543 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcserial.ko 2024-04-26T21:29:56.543 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/quatech2.ko 2024-04-26T21:29:56.543 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/safe_serial.ko 2024-04-26T21:29:56.543 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/sierra.ko 2024-04-26T21:29:56.543 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/spcp8x5.ko 2024-04-26T21:29:56.543 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ssu100.ko 2024-04-26T21:29:56.543 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/symbolserial.ko 2024-04-26T21:29:56.543 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-04-26T21:29:56.543 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-04-26T21:29:56.543 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_debug.ko 2024-04-26T21:29:56.543 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_wwan.ko 2024-04-26T21:29:56.543 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/visor.ko 2024-04-26T21:29:56.543 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/whiteheat.ko 2024-04-26T21:29:56.543 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/xsens_mt.ko 2024-04-26T21:29:56.543 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage 2024-04-26T21:29:56.543 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/uas.ko 2024-04-26T21:29:56.544 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-alauda.ko 2024-04-26T21:29:56.544 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-cypress.ko 2024-04-26T21:29:56.544 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-datafab.ko 2024-04-26T21:29:56.544 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-04-26T21:29:56.544 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-freecom.ko 2024-04-26T21:29:56.544 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-isd200.ko 2024-04-26T21:29:56.544 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-04-26T21:29:56.544 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-karma.ko 2024-04-26T21:29:56.544 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-onetouch.ko 2024-04-26T21:29:56.544 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-realtek.ko 2024-04-26T21:29:56.544 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr09.ko 2024-04-26T21:29:56.544 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr55.ko 2024-04-26T21:29:56.545 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-usbat.ko 2024-04-26T21:29:56.545 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/usb-storage.ko 2024-04-26T21:29:56.545 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip 2024-04-26T21:29:56.545 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-core.ko 2024-04-26T21:29:56.545 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-host.ko 2024-04-26T21:29:56.545 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-04-26T21:29:56.545 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio 2024-04-26T21:29:56.545 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci 2024-04-26T21:29:56.545 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-04-26T21:29:56.545 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci.ko 2024-04-26T21:29:56.545 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio.ko 2024-04-26T21:29:56.545 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-04-26T21:29:56.545 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost 2024-04-26T21:29:56.546 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost.ko 2024-04-26T21:29:56.546 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_iotlb.ko 2024-04-26T21:29:56.546 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_net.ko 2024-04-26T21:29:56.546 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_scsi.ko 2024-04-26T21:29:56.546 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video 2024-04-26T21:29:56.546 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight 2024-04-26T21:29:56.546 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/apple_bl.ko 2024-04-26T21:29:56.546 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/lcd.ko 2024-04-26T21:29:56.546 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/platform_lcd.ko 2024-04-26T21:29:56.546 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio 2024-04-26T21:29:56.546 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio.ko 2024-04-26T21:29:56.546 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_balloon.ko 2024-04-26T21:29:56.547 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_mmio.ko 2024-04-26T21:29:56.547 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci.ko 2024-04-26T21:29:56.547 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-04-26T21:29:56.547 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-04-26T21:29:56.547 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_ring.ko 2024-04-26T21:29:56.547 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1 2024-04-26T21:29:56.547 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters 2024-04-26T21:29:56.547 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2482.ko 2024-04-26T21:29:56.547 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2490.ko 2024-04-26T21:29:56.547 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves 2024-04-26T21:29:56.547 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-04-26T21:29:56.547 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-04-26T21:29:56.547 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-04-26T21:29:56.547 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-04-26T21:29:56.547 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-04-26T21:29:56.548 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-04-26T21:29:56.548 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-04-26T21:29:56.548 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-04-26T21:29:56.548 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-04-26T21:29:56.548 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_smem.ko 2024-04-26T21:29:56.548 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_therm.ko 2024-04-26T21:29:56.548 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/wire.ko 2024-04-26T21:29:56.548 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog 2024-04-26T21:29:56.548 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim1535_wdt.ko 2024-04-26T21:29:56.548 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim7101_wdt.ko 2024-04-26T21:29:56.548 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/f71808e_wdt.ko 2024-04-26T21:29:56.548 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/hpwdt.ko 2024-04-26T21:29:56.548 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/i6300esb.ko 2024-04-26T21:29:56.548 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-04-26T21:29:56.548 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_wdt.ko 2024-04-26T21:29:56.548 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ib700wdt.ko 2024-04-26T21:29:56.549 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ibmasr.ko 2024-04-26T21:29:56.549 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-04-26T21:29:56.549 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it8712f_wdt.ko 2024-04-26T21:29:56.549 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it87_wdt.ko 2024-04-26T21:29:56.549 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/machzwd.ko 2024-04-26T21:29:56.549 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/nv_tco.ko 2024-04-26T21:29:56.549 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_pci.ko 2024-04-26T21:29:56.549 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_usb.ko 2024-04-26T21:29:56.549 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-04-26T21:29:56.549 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sch311x_wdt.ko 2024-04-26T21:29:56.549 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/softdog.ko 2024-04-26T21:29:56.549 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sp5100_tco.ko 2024-04-26T21:29:56.549 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/via_wdt.ko 2024-04-26T21:29:56.549 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-04-26T21:29:56.549 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83877f_wdt.ko 2024-04-26T21:29:56.550 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83977f_wdt.ko 2024-04-26T21:29:56.550 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/wdt_pci.ko 2024-04-26T21:29:56.550 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/xen_wdt.ko 2024-04-26T21:29:56.550 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen 2024-04-26T21:29:56.550 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-acpi-processor.ko 2024-04-26T21:29:56.550 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-evtchn.ko 2024-04-26T21:29:56.550 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntalloc.ko 2024-04-26T21:29:56.550 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntdev.ko 2024-04-26T21:29:56.550 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback 2024-04-26T21:29:56.550 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-04-26T21:29:56.550 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-privcmd.ko 2024-04-26T21:29:56.550 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs 2024-04-26T21:29:56.550 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs/xenfs.ko 2024-04-26T21:29:56.550 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs 2024-04-26T21:29:56.551 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p 2024-04-26T21:29:56.551 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p/9p.ko 2024-04-26T21:29:56.551 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs 2024-04-26T21:29:56.551 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs/affs.ko 2024-04-26T21:29:56.551 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs 2024-04-26T21:29:56.551 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs/befs.ko 2024-04-26T21:29:56.551 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/binfmt_misc.ko 2024-04-26T21:29:56.551 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs 2024-04-26T21:29:56.551 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs/btrfs.ko 2024-04-26T21:29:56.551 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles 2024-04-26T21:29:56.551 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles/cachefiles.ko 2024-04-26T21:29:56.551 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph 2024-04-26T21:29:56.551 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph/ceph.ko 2024-04-26T21:29:56.551 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda 2024-04-26T21:29:56.551 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda/coda.ko 2024-04-26T21:29:56.551 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs 2024-04-26T21:29:56.552 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs/cramfs.ko 2024-04-26T21:29:56.552 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm 2024-04-26T21:29:56.552 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm/dlm.ko 2024-04-26T21:29:56.552 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs 2024-04-26T21:29:56.552 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs/ecryptfs.ko 2024-04-26T21:29:56.552 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs 2024-04-26T21:29:56.552 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs/f2fs.ko 2024-04-26T21:29:56.552 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat 2024-04-26T21:29:56.552 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/fat.ko 2024-04-26T21:29:56.552 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/msdos.ko 2024-04-26T21:29:56.552 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/vfat.ko 2024-04-26T21:29:56.552 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse 2024-04-26T21:29:56.552 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/cuse.ko 2024-04-26T21:29:56.552 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/fuse.ko 2024-04-26T21:29:56.552 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2 2024-04-26T21:29:56.552 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2/gfs2.ko 2024-04-26T21:29:56.553 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs 2024-04-26T21:29:56.553 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs/hfs.ko 2024-04-26T21:29:56.553 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus 2024-04-26T21:29:56.553 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus/hfsplus.ko 2024-04-26T21:29:56.553 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs 2024-04-26T21:29:56.553 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs/isofs.ko 2024-04-26T21:29:56.553 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs 2024-04-26T21:29:56.553 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs/jfs.ko 2024-04-26T21:29:56.553 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd 2024-04-26T21:29:56.553 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd/lockd.ko 2024-04-26T21:29:56.553 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix 2024-04-26T21:29:56.553 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix/minix.ko 2024-04-26T21:29:56.553 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs 2024-04-26T21:29:56.553 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs/netfs.ko 2024-04-26T21:29:56.554 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs 2024-04-26T21:29:56.554 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout 2024-04-26T21:29:56.554 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-04-26T21:29:56.554 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout 2024-04-26T21:29:56.554 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-04-26T21:29:56.554 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout 2024-04-26T21:29:56.554 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-04-26T21:29:56.554 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfs.ko 2024-04-26T21:29:56.554 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv3.ko 2024-04-26T21:29:56.554 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv4.ko 2024-04-26T21:29:56.554 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common 2024-04-26T21:29:56.554 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/grace.ko 2024-04-26T21:29:56.554 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/nfs_acl.ko 2024-04-26T21:29:56.554 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd 2024-04-26T21:29:56.555 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd/nfsd.ko 2024-04-26T21:29:56.555 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2 2024-04-26T21:29:56.555 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2/nilfs2.ko 2024-04-26T21:29:56.555 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls 2024-04-26T21:29:56.555 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-celtic.ko 2024-04-26T21:29:56.555 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-centeuro.ko 2024-04-26T21:29:56.555 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-croatian.ko 2024-04-26T21:29:56.555 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-cyrillic.ko 2024-04-26T21:29:56.555 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-gaelic.ko 2024-04-26T21:29:56.555 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-greek.ko 2024-04-26T21:29:56.555 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-iceland.ko 2024-04-26T21:29:56.555 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-inuit.ko 2024-04-26T21:29:56.555 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-roman.ko 2024-04-26T21:29:56.556 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-romanian.ko 2024-04-26T21:29:56.556 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-turkish.ko 2024-04-26T21:29:56.556 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1250.ko 2024-04-26T21:29:56.556 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1251.ko 2024-04-26T21:29:56.556 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1255.ko 2024-04-26T21:29:56.556 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp737.ko 2024-04-26T21:29:56.556 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp775.ko 2024-04-26T21:29:56.556 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp850.ko 2024-04-26T21:29:56.556 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp852.ko 2024-04-26T21:29:56.556 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp855.ko 2024-04-26T21:29:56.556 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp857.ko 2024-04-26T21:29:56.556 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp860.ko 2024-04-26T21:29:56.556 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp861.ko 2024-04-26T21:29:56.556 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp862.ko 2024-04-26T21:29:56.557 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp863.ko 2024-04-26T21:29:56.557 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp864.ko 2024-04-26T21:29:56.557 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp865.ko 2024-04-26T21:29:56.557 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp866.ko 2024-04-26T21:29:56.557 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp869.ko 2024-04-26T21:29:56.557 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp874.ko 2024-04-26T21:29:56.557 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp932.ko 2024-04-26T21:29:56.557 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp936.ko 2024-04-26T21:29:56.557 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp949.ko 2024-04-26T21:29:56.557 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp950.ko 2024-04-26T21:29:56.557 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_euc-jp.ko 2024-04-26T21:29:56.557 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-1.ko 2024-04-26T21:29:56.557 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-13.ko 2024-04-26T21:29:56.558 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-14.ko 2024-04-26T21:29:56.558 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-15.ko 2024-04-26T21:29:56.558 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-2.ko 2024-04-26T21:29:56.558 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-3.ko 2024-04-26T21:29:56.558 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-4.ko 2024-04-26T21:29:56.558 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-5.ko 2024-04-26T21:29:56.558 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-6.ko 2024-04-26T21:29:56.558 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-7.ko 2024-04-26T21:29:56.558 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-9.ko 2024-04-26T21:29:56.558 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-r.ko 2024-04-26T21:29:56.558 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-ru.ko 2024-04-26T21:29:56.558 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-u.ko 2024-04-26T21:29:56.558 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_ucs2_utils.ko 2024-04-26T21:29:56.559 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_utf8.ko 2024-04-26T21:29:56.559 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2 2024-04-26T21:29:56.559 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster 2024-04-26T21:29:56.559 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-04-26T21:29:56.559 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm 2024-04-26T21:29:56.559 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-04-26T21:29:56.559 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs 2024-04-26T21:29:56.559 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-04-26T21:29:56.559 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2.ko 2024-04-26T21:29:56.559 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-04-26T21:29:56.559 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-04-26T21:29:56.559 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-04-26T21:29:56.559 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs 2024-04-26T21:29:56.559 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs/overlay.ko 2024-04-26T21:29:56.560 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore 2024-04-26T21:29:56.560 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore/ramoops.ko 2024-04-26T21:29:56.560 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs 2024-04-26T21:29:56.560 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs/reiserfs.ko 2024-04-26T21:29:56.560 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs 2024-04-26T21:29:56.560 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs/romfs.ko 2024-04-26T21:29:56.560 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb 2024-04-26T21:29:56.560 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client 2024-04-26T21:29:56.560 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client/cifs.ko 2024-04-26T21:29:56.560 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common 2024-04-26T21:29:56.560 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_arc4.ko 2024-04-26T21:29:56.560 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_md4.ko 2024-04-26T21:29:56.560 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs 2024-04-26T21:29:56.561 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs/squashfs.ko 2024-04-26T21:29:56.561 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv 2024-04-26T21:29:56.561 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv/sysv.ko 2024-04-26T21:29:56.561 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf 2024-04-26T21:29:56.561 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf/udf.ko 2024-04-26T21:29:56.561 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs 2024-04-26T21:29:56.561 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs/ufs.ko 2024-04-26T21:29:56.561 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs 2024-04-26T21:29:56.561 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs/xfs.ko 2024-04-26T21:29:56.561 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel 2024-04-26T21:29:56.561 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu 2024-04-26T21:29:56.561 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu/rcutorture.ko 2024-04-26T21:29:56.561 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/torture.ko 2024-04-26T21:29:56.562 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace 2024-04-26T21:29:56.562 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-04-26T21:29:56.562 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib 2024-04-26T21:29:56.562 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/asn1_encoder.ko 2024-04-26T21:29:56.562 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc-itu-t.ko 2024-04-26T21:29:56.562 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc8.ko 2024-04-26T21:29:56.562 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto 2024-04-26T21:29:56.562 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libarc4.ko 2024-04-26T21:29:56.562 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libdes.ko 2024-04-26T21:29:56.562 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/libcrc32c.ko 2024-04-26T21:29:56.562 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lru_cache.ko 2024-04-26T21:29:56.562 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4 2024-04-26T21:29:56.562 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4_compress.ko 2024-04-26T21:29:56.563 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4hc_compress.ko 2024-04-26T21:29:56.563 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math 2024-04-26T21:29:56.563 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math/cordic.ko 2024-04-26T21:29:56.563 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6 2024-04-26T21:29:56.563 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6/raid6_pq.ko 2024-04-26T21:29:56.563 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon 2024-04-26T21:29:56.563 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon/reed_solomon.ko 2024-04-26T21:29:56.563 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_bm.ko 2024-04-26T21:29:56.563 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_fsm.ko 2024-04-26T21:29:56.563 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_kmp.ko 2024-04-26T21:29:56.563 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd 2024-04-26T21:29:56.563 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd/zstd_compress.ko 2024-04-26T21:29:56.563 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm 2024-04-26T21:29:56.564 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm/hwpoison-inject.ko 2024-04-26T21:29:56.564 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net 2024-04-26T21:29:56.564 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan 2024-04-26T21:29:56.564 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/6lowpan.ko 2024-04-26T21:29:56.564 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_dest.ko 2024-04-26T21:29:56.564 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_fragment.ko 2024-04-26T21:29:56.564 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_hop.ko 2024-04-26T21:29:56.564 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_ipv6.ko 2024-04-26T21:29:56.564 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_mobility.ko 2024-04-26T21:29:56.564 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_routing.ko 2024-04-26T21:29:56.565 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_udp.ko 2024-04-26T21:29:56.565 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802 2024-04-26T21:29:56.565 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/garp.ko 2024-04-26T21:29:56.565 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/mrp.ko 2024-04-26T21:29:56.565 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/p8022.ko 2024-04-26T21:29:56.565 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/psnap.ko 2024-04-26T21:29:56.565 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/stp.ko 2024-04-26T21:29:56.565 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q 2024-04-26T21:29:56.565 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q/8021q.ko 2024-04-26T21:29:56.565 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p 2024-04-26T21:29:56.565 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet.ko 2024-04-26T21:29:56.565 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_fd.ko 2024-04-26T21:29:56.565 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_rdma.ko 2024-04-26T21:29:56.565 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_virtio.ko 2024-04-26T21:29:56.566 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv 2024-04-26T21:29:56.566 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv/batman-adv.ko 2024-04-26T21:29:56.566 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge 2024-04-26T21:29:56.566 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/br_netfilter.ko 2024-04-26T21:29:56.566 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/bridge.ko 2024-04-26T21:29:56.566 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter 2024-04-26T21:29:56.566 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-04-26T21:29:56.566 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_among.ko 2024-04-26T21:29:56.566 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arp.ko 2024-04-26T21:29:56.566 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-04-26T21:29:56.566 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-04-26T21:29:56.566 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip.ko 2024-04-26T21:29:56.566 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-04-26T21:29:56.567 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_limit.ko 2024-04-26T21:29:56.567 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_log.ko 2024-04-26T21:29:56.567 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark.ko 2024-04-26T21:29:56.567 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-04-26T21:29:56.567 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-04-26T21:29:56.567 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-04-26T21:29:56.567 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-04-26T21:29:56.567 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_snat.ko 2024-04-26T21:29:56.567 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_stp.ko 2024-04-26T21:29:56.567 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-04-26T21:29:56.567 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-04-26T21:29:56.567 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-04-26T21:29:56.567 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-04-26T21:29:56.567 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtables.ko 2024-04-26T21:29:56.568 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-04-26T21:29:56.568 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-04-26T21:29:56.568 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-04-26T21:29:56.568 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph 2024-04-26T21:29:56.568 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph/libceph.ko 2024-04-26T21:29:56.568 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core 2024-04-26T21:29:56.568 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/failover.ko 2024-04-26T21:29:56.568 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/pktgen.ko 2024-04-26T21:29:56.568 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp 2024-04-26T21:29:56.568 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp.ko 2024-04-26T21:29:56.568 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_diag.ko 2024-04-26T21:29:56.568 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv4.ko 2024-04-26T21:29:56.568 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv6.ko 2024-04-26T21:29:56.568 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver 2024-04-26T21:29:56.568 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver/dns_resolver.ko 2024-04-26T21:29:56.569 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa 2024-04-26T21:29:56.569 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/dsa_core.ko 2024-04-26T21:29:56.569 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_brcm.ko 2024-04-26T21:29:56.569 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_dsa.ko 2024-04-26T21:29:56.569 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_none.ko 2024-04-26T21:29:56.569 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_trailer.ko 2024-04-26T21:29:56.569 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154 2024-04-26T21:29:56.569 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan 2024-04-26T21:29:56.569 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-04-26T21:29:56.569 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154.ko 2024-04-26T21:29:56.569 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154_socket.ko 2024-04-26T21:29:56.569 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4 2024-04-26T21:29:56.569 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ah4.ko 2024-04-26T21:29:56.569 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/esp4.ko 2024-04-26T21:29:56.570 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/fou.ko 2024-04-26T21:29:56.570 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/gre.ko 2024-04-26T21:29:56.570 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/inet_diag.ko 2024-04-26T21:29:56.570 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_gre.ko 2024-04-26T21:29:56.570 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_tunnel.ko 2024-04-26T21:29:56.570 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_vti.ko 2024-04-26T21:29:56.570 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipcomp.ko 2024-04-26T21:29:56.570 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipip.ko 2024-04-26T21:29:56.570 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter 2024-04-26T21:29:56.570 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arp_tables.ko 2024-04-26T21:29:56.570 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-04-26T21:29:56.570 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-04-26T21:29:56.571 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-04-26T21:29:56.571 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-04-26T21:29:56.571 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-04-26T21:29:56.571 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-04-26T21:29:56.571 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-04-26T21:29:56.571 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-04-26T21:29:56.571 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-04-26T21:29:56.571 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_security.ko 2024-04-26T21:29:56.571 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-04-26T21:29:56.571 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-04-26T21:29:56.571 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-04-26T21:29:56.571 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-04-26T21:29:56.571 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-04-26T21:29:56.571 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-04-26T21:29:56.572 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-04-26T21:29:56.572 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-04-26T21:29:56.572 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-04-26T21:29:56.572 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-04-26T21:29:56.572 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_bic.ko 2024-04-26T21:29:56.572 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_dctcp.ko 2024-04-26T21:29:56.572 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_diag.ko 2024-04-26T21:29:56.572 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_highspeed.ko 2024-04-26T21:29:56.572 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_htcp.ko 2024-04-26T21:29:56.572 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_hybla.ko 2024-04-26T21:29:56.572 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_illinois.ko 2024-04-26T21:29:56.572 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_lp.ko 2024-04-26T21:29:56.572 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_scalable.ko 2024-04-26T21:29:56.572 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_vegas.ko 2024-04-26T21:29:56.573 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_veno.ko 2024-04-26T21:29:56.573 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_westwood.ko 2024-04-26T21:29:56.573 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_yeah.ko 2024-04-26T21:29:56.573 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tunnel4.ko 2024-04-26T21:29:56.573 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_diag.ko 2024-04-26T21:29:56.573 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_tunnel.ko 2024-04-26T21:29:56.573 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/xfrm4_tunnel.ko 2024-04-26T21:29:56.573 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6 2024-04-26T21:29:56.573 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ah6.ko 2024-04-26T21:29:56.573 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/esp6.ko 2024-04-26T21:29:56.573 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/fou6.ko 2024-04-26T21:29:56.573 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_tunnel.ko 2024-04-26T21:29:56.573 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-04-26T21:29:56.573 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_vti.ko 2024-04-26T21:29:56.573 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ipcomp6.ko 2024-04-26T21:29:56.574 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter 2024-04-26T21:29:56.574 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-04-26T21:29:56.574 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-04-26T21:29:56.574 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-04-26T21:29:56.574 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-04-26T21:29:56.574 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-04-26T21:29:56.574 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-04-26T21:29:56.574 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-04-26T21:29:56.574 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-04-26T21:29:56.574 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-04-26T21:29:56.574 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-04-26T21:29:56.574 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-04-26T21:29:56.574 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-04-26T21:29:56.575 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-04-26T21:29:56.575 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-04-26T21:29:56.575 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-04-26T21:29:56.575 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-04-26T21:29:56.575 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-04-26T21:29:56.575 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-04-26T21:29:56.575 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-04-26T21:29:56.575 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-04-26T21:29:56.575 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-04-26T21:29:56.575 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-04-26T21:29:56.575 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-04-26T21:29:56.575 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-04-26T21:29:56.575 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/sit.ko 2024-04-26T21:29:56.575 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/tunnel6.ko 2024-04-26T21:29:56.576 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/xfrm6_tunnel.ko 2024-04-26T21:29:56.576 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key 2024-04-26T21:29:56.576 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key/af_key.ko 2024-04-26T21:29:56.576 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp 2024-04-26T21:29:56.576 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_core.ko 2024-04-26T21:29:56.576 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_debugfs.ko 2024-04-26T21:29:56.576 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_eth.ko 2024-04-26T21:29:56.576 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip.ko 2024-04-26T21:29:56.576 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip6.ko 2024-04-26T21:29:56.576 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_netlink.ko 2024-04-26T21:29:56.576 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ppp.ko 2024-04-26T21:29:56.576 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc 2024-04-26T21:29:56.576 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc/llc.ko 2024-04-26T21:29:56.576 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154 2024-04-26T21:29:56.577 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154/mac802154.ko 2024-04-26T21:29:56.577 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls 2024-04-26T21:29:56.577 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls/mpls_gso.ko 2024-04-26T21:29:56.577 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter 2024-04-26T21:29:56.577 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset 2024-04-26T21:29:56.577 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set.ko 2024-04-26T21:29:56.577 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-04-26T21:29:56.577 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-04-26T21:29:56.577 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-04-26T21:29:56.577 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-04-26T21:29:56.577 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-04-26T21:29:56.577 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-04-26T21:29:56.577 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-04-26T21:29:56.577 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-04-26T21:29:56.577 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-04-26T21:29:56.578 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-04-26T21:29:56.578 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-04-26T21:29:56.578 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-04-26T21:29:56.578 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-04-26T21:29:56.578 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-04-26T21:29:56.578 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-04-26T21:29:56.578 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs 2024-04-26T21:29:56.578 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs.ko 2024-04-26T21:29:56.578 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-04-26T21:29:56.578 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-04-26T21:29:56.578 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-04-26T21:29:56.578 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-04-26T21:29:56.578 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-04-26T21:29:56.579 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-04-26T21:29:56.579 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-04-26T21:29:56.579 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-04-26T21:29:56.579 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-04-26T21:29:56.579 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-04-26T21:29:56.579 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-04-26T21:29:56.579 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-04-26T21:29:56.579 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-04-26T21:29:56.579 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conncount.ko 2024-04-26T21:29:56.579 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack.ko 2024-04-26T21:29:56.579 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-04-26T21:29:56.579 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-04-26T21:29:56.579 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-04-26T21:29:56.579 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_h323.ko 2024-04-26T21:29:56.580 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_irc.ko 2024-04-26T21:29:56.580 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-04-26T21:29:56.580 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-04-26T21:29:56.580 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-04-26T21:29:56.580 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sane.ko 2024-04-26T21:29:56.580 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sip.ko 2024-04-26T21:29:56.580 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-04-26T21:29:56.580 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-04-26T21:29:56.580 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_dup_netdev.ko 2024-04-26T21:29:56.580 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table.ko 2024-04-26T21:29:56.580 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table_inet.ko 2024-04-26T21:29:56.580 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_log_syslog.ko 2024-04-26T21:29:56.580 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat.ko 2024-04-26T21:29:56.580 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_amanda.ko 2024-04-26T21:29:56.581 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_ftp.ko 2024-04-26T21:29:56.581 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_irc.ko 2024-04-26T21:29:56.581 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_sip.ko 2024-04-26T21:29:56.581 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_tftp.ko 2024-04-26T21:29:56.581 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_synproxy_core.ko 2024-04-26T21:29:56.581 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_tables.ko 2024-04-26T21:29:56.581 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink.ko 2024-04-26T21:29:56.581 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_acct.ko 2024-04-26T21:29:56.581 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_log.ko 2024-04-26T21:29:56.581 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_osf.ko 2024-04-26T21:29:56.581 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_queue.ko 2024-04-26T21:29:56.581 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_chain_nat.ko 2024-04-26T21:29:56.581 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_compat.ko 2024-04-26T21:29:56.581 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_ct.ko 2024-04-26T21:29:56.582 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_dup_netdev.ko 2024-04-26T21:29:56.582 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fib.ko 2024-04-26T21:29:56.582 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fwd_netdev.ko 2024-04-26T21:29:56.582 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_hash.ko 2024-04-26T21:29:56.582 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_limit.ko 2024-04-26T21:29:56.582 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_log.ko 2024-04-26T21:29:56.582 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_masq.ko 2024-04-26T21:29:56.582 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_nat.ko 2024-04-26T21:29:56.582 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_numgen.ko 2024-04-26T21:29:56.582 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_queue.ko 2024-04-26T21:29:56.582 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_quota.ko 2024-04-26T21:29:56.582 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_redir.ko 2024-04-26T21:29:56.582 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject.ko 2024-04-26T21:29:56.583 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject_inet.ko 2024-04-26T21:29:56.583 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_socket.ko 2024-04-26T21:29:56.583 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_tunnel.ko 2024-04-26T21:29:56.583 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_AUDIT.ko 2024-04-26T21:29:56.583 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CHECKSUM.ko 2024-04-26T21:29:56.583 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CLASSIFY.ko 2024-04-26T21:29:56.583 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-04-26T21:29:56.583 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CT.ko 2024-04-26T21:29:56.583 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_DSCP.ko 2024-04-26T21:29:56.583 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HL.ko 2024-04-26T21:29:56.583 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HMARK.ko 2024-04-26T21:29:56.583 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_IDLETIMER.ko 2024-04-26T21:29:56.583 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LED.ko 2024-04-26T21:29:56.583 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LOG.ko 2024-04-26T21:29:56.583 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_MASQUERADE.ko 2024-04-26T21:29:56.583 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NETMAP.ko 2024-04-26T21:29:56.584 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFLOG.ko 2024-04-26T21:29:56.584 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFQUEUE.ko 2024-04-26T21:29:56.584 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_RATEEST.ko 2024-04-26T21:29:56.584 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_REDIRECT.ko 2024-04-26T21:29:56.584 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_SECMARK.ko 2024-04-26T21:29:56.584 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPMSS.ko 2024-04-26T21:29:56.584 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-04-26T21:29:56.584 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TEE.ko 2024-04-26T21:29:56.584 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TPROXY.ko 2024-04-26T21:29:56.584 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TRACE.ko 2024-04-26T21:29:56.584 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_addrtype.ko 2024-04-26T21:29:56.584 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_bpf.ko 2024-04-26T21:29:56.584 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cgroup.ko 2024-04-26T21:29:56.584 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cluster.ko 2024-04-26T21:29:56.584 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_comment.ko 2024-04-26T21:29:56.585 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connbytes.ko 2024-04-26T21:29:56.585 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlabel.ko 2024-04-26T21:29:56.585 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlimit.ko 2024-04-26T21:29:56.585 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connmark.ko 2024-04-26T21:29:56.585 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_conntrack.ko 2024-04-26T21:29:56.585 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cpu.ko 2024-04-26T21:29:56.585 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dccp.ko 2024-04-26T21:29:56.585 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_devgroup.ko 2024-04-26T21:29:56.585 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dscp.ko 2024-04-26T21:29:56.585 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ecn.ko 2024-04-26T21:29:56.585 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_esp.ko 2024-04-26T21:29:56.585 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hashlimit.ko 2024-04-26T21:29:56.585 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_helper.ko 2024-04-26T21:29:56.585 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hl.ko 2024-04-26T21:29:56.585 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipcomp.ko 2024-04-26T21:29:56.586 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_iprange.ko 2024-04-26T21:29:56.586 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipvs.ko 2024-04-26T21:29:56.586 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_l2tp.ko 2024-04-26T21:29:56.586 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_length.ko 2024-04-26T21:29:56.586 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_limit.ko 2024-04-26T21:29:56.586 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mac.ko 2024-04-26T21:29:56.586 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mark.ko 2024-04-26T21:29:56.586 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_multiport.ko 2024-04-26T21:29:56.586 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nat.ko 2024-04-26T21:29:56.586 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nfacct.ko 2024-04-26T21:29:56.586 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_osf.ko 2024-04-26T21:29:56.586 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_owner.ko 2024-04-26T21:29:56.586 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_physdev.ko 2024-04-26T21:29:56.586 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_pkttype.ko 2024-04-26T21:29:56.587 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_policy.ko 2024-04-26T21:29:56.587 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_quota.ko 2024-04-26T21:29:56.587 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_rateest.ko 2024-04-26T21:29:56.587 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_realm.ko 2024-04-26T21:29:56.587 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_recent.ko 2024-04-26T21:29:56.587 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_sctp.ko 2024-04-26T21:29:56.587 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_set.ko 2024-04-26T21:29:56.587 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_socket.ko 2024-04-26T21:29:56.587 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_state.ko 2024-04-26T21:29:56.587 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_statistic.ko 2024-04-26T21:29:56.587 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_string.ko 2024-04-26T21:29:56.587 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_tcpmss.ko 2024-04-26T21:29:56.587 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_time.ko 2024-04-26T21:29:56.587 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_u32.ko 2024-04-26T21:29:56.587 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink 2024-04-26T21:29:56.588 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink/netlink_diag.ko 2024-04-26T21:29:56.588 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh 2024-04-26T21:29:56.588 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh/nsh.ko 2024-04-26T21:29:56.588 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch 2024-04-26T21:29:56.588 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/openvswitch.ko 2024-04-26T21:29:56.588 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-geneve.ko 2024-04-26T21:29:56.588 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-gre.ko 2024-04-26T21:29:56.588 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-vxlan.ko 2024-04-26T21:29:56.588 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet 2024-04-26T21:29:56.588 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet/af_packet_diag.ko 2024-04-26T21:29:56.588 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds 2024-04-26T21:29:56.588 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds.ko 2024-04-26T21:29:56.589 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_rdma.ko 2024-04-26T21:29:56.589 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_tcp.ko 2024-04-26T21:29:56.589 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched 2024-04-26T21:29:56.589 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_csum.ko 2024-04-26T21:29:56.589 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_gact.ko 2024-04-26T21:29:56.589 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_mirred.ko 2024-04-26T21:29:56.589 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_nat.ko 2024-04-26T21:29:56.589 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_pedit.ko 2024-04-26T21:29:56.589 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_police.ko 2024-04-26T21:29:56.589 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_simple.ko 2024-04-26T21:29:56.589 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_skbedit.ko 2024-04-26T21:29:56.589 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_vlan.ko 2024-04-26T21:29:56.589 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_basic.ko 2024-04-26T21:29:56.589 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_bpf.ko 2024-04-26T21:29:56.589 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_flow.ko 2024-04-26T21:29:56.590 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_fw.ko 2024-04-26T21:29:56.590 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_route.ko 2024-04-26T21:29:56.590 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_u32.ko 2024-04-26T21:29:56.590 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_cmp.ko 2024-04-26T21:29:56.590 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_ipset.ko 2024-04-26T21:29:56.590 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_meta.ko 2024-04-26T21:29:56.590 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_nbyte.ko 2024-04-26T21:29:56.590 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_text.ko 2024-04-26T21:29:56.590 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_u32.ko 2024-04-26T21:29:56.590 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_choke.ko 2024-04-26T21:29:56.590 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_codel.ko 2024-04-26T21:29:56.590 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_drr.ko 2024-04-26T21:29:56.590 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_fq.ko 2024-04-26T21:29:56.590 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_gred.ko 2024-04-26T21:29:56.591 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hfsc.ko 2024-04-26T21:29:56.591 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hhf.ko 2024-04-26T21:29:56.591 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_htb.ko 2024-04-26T21:29:56.591 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_ingress.ko 2024-04-26T21:29:56.591 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio.ko 2024-04-26T21:29:56.591 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio_lib.ko 2024-04-26T21:29:56.591 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_multiq.ko 2024-04-26T21:29:56.591 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_netem.ko 2024-04-26T21:29:56.591 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_pie.ko 2024-04-26T21:29:56.591 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_plug.ko 2024-04-26T21:29:56.591 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_prio.ko 2024-04-26T21:29:56.591 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_qfq.ko 2024-04-26T21:29:56.591 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_red.ko 2024-04-26T21:29:56.591 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfb.ko 2024-04-26T21:29:56.591 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfq.ko 2024-04-26T21:29:56.591 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_tbf.ko 2024-04-26T21:29:56.592 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_teql.ko 2024-04-26T21:29:56.592 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp 2024-04-26T21:29:56.592 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp.ko 2024-04-26T21:29:56.592 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp_diag.ko 2024-04-26T21:29:56.592 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc 2024-04-26T21:29:56.592 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss 2024-04-26T21:29:56.592 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-04-26T21:29:56.592 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-04-26T21:29:56.592 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/sunrpc.ko 2024-04-26T21:29:56.592 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma 2024-04-26T21:29:56.592 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-04-26T21:29:56.592 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc 2024-04-26T21:29:56.592 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc.ko 2024-04-26T21:29:56.592 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc_diag.ko 2024-04-26T21:29:56.592 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix 2024-04-26T21:29:56.593 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix/unix_diag.ko 2024-04-26T21:29:56.593 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock 2024-04-26T21:29:56.593 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-04-26T21:29:56.593 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-04-26T21:29:56.593 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock.ko 2024-04-26T21:29:56.593 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_diag.ko 2024-04-26T21:29:56.593 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_loopback.ko 2024-04-26T21:29:56.593 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm 2024-04-26T21:29:56.593 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm/xfrm_ipcomp.ko 2024-04-26T21:29:56.593 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security 2024-04-26T21:29:56.593 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys 2024-04-26T21:29:56.593 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys 2024-04-26T21:29:56.593 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-04-26T21:29:56.593 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys 2024-04-26T21:29:56.593 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys/trusted.ko 2024-04-26T21:29:56.593 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias 2024-04-26T21:29:56.594 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias.bin 2024-04-26T21:29:56.594 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin 2024-04-26T21:29:56.594 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.alias.bin 2024-04-26T21:29:56.594 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.bin 2024-04-26T21:29:56.594 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.modinfo 2024-04-26T21:29:56.594 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep 2024-04-26T21:29:56.594 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep.bin 2024-04-26T21:29:56.594 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.devname 2024-04-26T21:29:56.594 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.order 2024-04-26T21:29:56.594 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.softdep 2024-04-26T21:29:56.594 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols 2024-04-26T21:29:56.594 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols.bin 2024-04-26T21:29:56.594 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-04-26T21:29:56.598 DEBUG:teuthology.task.kernel:get_image_version: 6.9.0-rc3-g806ec36b2112 2024-04-26T21:29:56.598 DEBUG:teuthology.orchestra.run.smithi164:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-04-26T21:29:56.621 INFO:teuthology.orchestra.run.smithi164.stdout:postinstall scriptlet (using /bin/sh): 2024-04-26T21:29:56.621 INFO:teuthology.orchestra.run.smithi164.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-26T21:29:56.621 INFO:teuthology.orchestra.run.smithi164.stdout: /usr/bin/kernel-install add 6.9.0-rc3-g806ec36b2112 /lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-04-26T21:29:56.621 INFO:teuthology.orchestra.run.smithi164.stdout:fi 2024-04-26T21:29:56.621 INFO:teuthology.orchestra.run.smithi164.stdout:for file in vmlinuz System.map config; do 2024-04-26T21:29:56.622 INFO:teuthology.orchestra.run.smithi164.stdout: if ! cmp --silent "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112"; then 2024-04-26T21:29:56.622 INFO:teuthology.orchestra.run.smithi164.stdout: cp "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112" 2024-04-26T21:29:56.622 INFO:teuthology.orchestra.run.smithi164.stdout: fi 2024-04-26T21:29:56.622 INFO:teuthology.orchestra.run.smithi164.stdout:done 2024-04-26T21:29:56.622 INFO:teuthology.orchestra.run.smithi164.stdout:if [ -d "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" ] && \ 2024-04-26T21:29:56.622 INFO:teuthology.orchestra.run.smithi164.stdout: ! diff -rq "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" >/dev/null 2>&1; then 2024-04-26T21:29:56.622 INFO:teuthology.orchestra.run.smithi164.stdout: rm -rf "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-04-26T21:29:56.622 INFO:teuthology.orchestra.run.smithi164.stdout: cp -r "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-04-26T21:29:56.622 INFO:teuthology.orchestra.run.smithi164.stdout:fi 2024-04-26T21:29:56.622 INFO:teuthology.orchestra.run.smithi164.stdout:if [ ! -e "/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep" ]; then 2024-04-26T21:29:56.622 INFO:teuthology.orchestra.run.smithi164.stdout: /usr/sbin/depmod 6.9.0-rc3-g806ec36b2112 2024-04-26T21:29:56.622 INFO:teuthology.orchestra.run.smithi164.stdout:fi 2024-04-26T21:29:56.622 INFO:teuthology.orchestra.run.smithi164.stdout:preuninstall scriptlet (using /bin/sh): 2024-04-26T21:29:56.623 INFO:teuthology.orchestra.run.smithi164.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-26T21:29:56.623 INFO:teuthology.orchestra.run.smithi164.stdout:kernel-install remove 6.9.0-rc3-g806ec36b2112 2024-04-26T21:29:56.623 INFO:teuthology.orchestra.run.smithi164.stdout:fi 2024-04-26T21:29:56.624 DEBUG:teuthology.orchestra.run.smithi164:> sudo rpm -qi grub2-tools 2024-04-26T21:29:56.695 INFO:teuthology.orchestra.run.smithi164.stdout:Name : grub2-tools 2024-04-26T21:29:56.695 INFO:teuthology.orchestra.run.smithi164.stdout:Epoch : 1 2024-04-26T21:29:56.696 INFO:teuthology.orchestra.run.smithi164.stdout:Version : 2.06 2024-04-26T21:29:56.696 INFO:teuthology.orchestra.run.smithi164.stdout:Release : 68.el9 2024-04-26T21:29:56.696 INFO:teuthology.orchestra.run.smithi164.stdout:Architecture: x86_64 2024-04-26T21:29:56.696 INFO:teuthology.orchestra.run.smithi164.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-04-26T21:29:56.696 INFO:teuthology.orchestra.run.smithi164.stdout:Group : Unspecified 2024-04-26T21:29:56.696 INFO:teuthology.orchestra.run.smithi164.stdout:Size : 8264254 2024-04-26T21:29:56.696 INFO:teuthology.orchestra.run.smithi164.stdout:License : GPLv3+ 2024-04-26T21:29:56.696 INFO:teuthology.orchestra.run.smithi164.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-04-26T21:29:56.696 INFO:teuthology.orchestra.run.smithi164.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-04-26T21:29:56.696 INFO:teuthology.orchestra.run.smithi164.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-04-26T21:29:56.696 INFO:teuthology.orchestra.run.smithi164.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-04-26T21:29:56.696 INFO:teuthology.orchestra.run.smithi164.stdout:Packager : builder@centos.org 2024-04-26T21:29:56.696 INFO:teuthology.orchestra.run.smithi164.stdout:Vendor : CentOS 2024-04-26T21:29:56.696 INFO:teuthology.orchestra.run.smithi164.stdout:URL : http://www.gnu.org/software/grub/ 2024-04-26T21:29:56.696 INFO:teuthology.orchestra.run.smithi164.stdout:Summary : Support tools for GRUB. 2024-04-26T21:29:56.697 INFO:teuthology.orchestra.run.smithi164.stdout:Description : 2024-04-26T21:29:56.697 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-26T21:29:56.697 INFO:teuthology.orchestra.run.smithi164.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-04-26T21:29:56.697 INFO:teuthology.orchestra.run.smithi164.stdout:customizable bootloader with modular architecture. It supports a rich 2024-04-26T21:29:56.697 INFO:teuthology.orchestra.run.smithi164.stdout:variety of kernel formats, file systems, computer architectures and 2024-04-26T21:29:56.697 INFO:teuthology.orchestra.run.smithi164.stdout:hardware devices. 2024-04-26T21:29:56.697 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-26T21:29:56.697 INFO:teuthology.orchestra.run.smithi164.stdout:This subpackage provides tools for support of all platforms. 2024-04-26T21:29:56.698 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-04-26T21:29:56.698 INFO:teuthology.task.kernel:Updating grub on smithi164 to boot 6.9.0-rc3-g806ec36b2112 2024-04-26T21:29:56.698 DEBUG:teuthology.orchestra.run.smithi164:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-04-26T21:29:57.259 INFO:teuthology.orchestra.run.smithi164.stderr:Generating grub configuration file ... 2024-04-26T21:29:58.689 INFO:teuthology.orchestra.run.smithi164.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-04-26T21:29:58.716 INFO:teuthology.orchestra.run.smithi164.stderr:done 2024-04-26T21:29:58.719 DEBUG:teuthology.orchestra.run.smithi164:> mktemp 2024-04-26T21:29:58.756 INFO:teuthology.orchestra.run.smithi164.stdout:/tmp/tmp.cYvvuIwFoF 2024-04-26T21:29:58.756 DEBUG:teuthology.orchestra.run.smithi164:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.cYvvuIwFoF 2024-04-26T21:29:58.825 DEBUG:teuthology.orchestra.run.smithi164:> sudo chmod 0666 /tmp/tmp.cYvvuIwFoF 2024-04-26T21:29:58.960 DEBUG:teuthology.orchestra.remote:smithi164:/tmp/tmp.cYvvuIwFoF is 6KB 2024-04-26T21:29:59.008 DEBUG:teuthology.orchestra.run.smithi164:> rm -fr /tmp/tmp.cYvvuIwFoF 2024-04-26T21:29:59.023 DEBUG:teuthology.orchestra.run.smithi164:> sudo /bin/ls /boot/loader/entries || true 2024-04-26T21:29:59.084 INFO:teuthology.orchestra.run.smithi164.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-04-26T21:29:59.084 INFO:teuthology.orchestra.run.smithi164.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-04-26T21:29:59.084 INFO:teuthology.orchestra.run.smithi164.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-04-26T21:29:59.084 INFO:teuthology.orchestra.run.smithi164.stdout:7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112.conf 2024-04-26T21:29:59.085 DEBUG:teuthology.orchestra.run.smithi164:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112 2024-04-26T21:29:59.240 DEBUG:teuthology.orchestra.run.smithi164:> sudo shutdown -r now 2024-04-26T21:30:07.518 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.76 2024-04-26T21:30:16.522 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi076.front.sepia.ceph.com' 2024-04-26T21:30:16.523 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi076.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T21:30:19.582 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.76 2024-04-26T21:30:25.680 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-04-26T21:30:25.680 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi072.front.sepia.ceph.com' 2024-04-26T21:30:25.681 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T21:30:29.246 INFO:teuthology.task.kernel:Checking client mon.c for new kernel version... 2024-04-26T21:30:29.246 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi164.front.sepia.ceph.com' 2024-04-26T21:30:29.247 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi164.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T21:30:31.585 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi076.front.sepia.ceph.com' 2024-04-26T21:30:31.585 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi076.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T21:30:44.130 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-04-26T21:30:47.710 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.164 2024-04-26T21:30:53.134 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi072.front.sepia.ceph.com' 2024-04-26T21:30:53.135 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T21:30:56.190 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-04-26T21:30:56.713 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi164.front.sepia.ceph.com' 2024-04-26T21:30:56.726 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi164.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T21:30:59.807 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.164 2024-04-26T21:31:08.197 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi072.front.sepia.ceph.com' 2024-04-26T21:31:08.198 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T21:31:11.810 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi164.front.sepia.ceph.com' 2024-04-26T21:31:11.811 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi164.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T21:31:30.207 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.164 2024-04-26T21:31:31.589 DEBUG:teuthology.orchestra.remote:timed out 2024-04-26T21:31:45.218 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi164.front.sepia.ceph.com' 2024-04-26T21:31:45.219 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi164.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T21:31:46.591 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi076.front.sepia.ceph.com' 2024-04-26T21:31:46.592 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi076.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T21:31:46.981 DEBUG:teuthology.orchestra.run.smithi076:> true 2024-04-26T21:31:47.607 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi076.front.sepia.ceph.com' 2024-04-26T21:31:47.607 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "6.9.0-rc3-g806ec36b2112"... 2024-04-26T21:31:47.608 DEBUG:teuthology.orchestra.run.smithi076:> uname -r 2024-04-26T21:31:47.655 INFO:teuthology.orchestra.run.smithi076.stdout:6.9.0-rc3-g806ec36b2112 2024-04-26T21:31:47.655 DEBUG:teuthology.task.kernel:current kernel version is 6.9.0-rc3-g806ec36b2112 vs 6.9.0-rc3-g806ec36b2112 2024-04-26T21:31:47.655 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-26T21:31:47.655 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-04-26T21:31:48.290 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.164 2024-04-26T21:31:48.657 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-04-26T21:31:48.657 DEBUG:teuthology.orchestra.run.smithi076:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-26T21:31:48.775 INFO:teuthology.orchestra.run.smithi076.stdout:ttyS1 2024-04-26T21:31:48.833 DEBUG:teuthology.parallel:result is None 2024-04-26T21:32:06.292 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi164.front.sepia.ceph.com' 2024-04-26T21:32:06.292 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi164.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T21:32:06.657 DEBUG:teuthology.orchestra.run.smithi164:> true 2024-04-26T21:32:07.323 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi164.front.sepia.ceph.com' 2024-04-26T21:32:07.323 INFO:teuthology.task.kernel:Checking kernel version of mon.c, want "6.9.0-rc3-g806ec36b2112"... 2024-04-26T21:32:07.323 DEBUG:teuthology.orchestra.run.smithi164:> uname -r 2024-04-26T21:32:07.361 INFO:teuthology.orchestra.run.smithi164.stdout:6.9.0-rc3-g806ec36b2112 2024-04-26T21:32:07.361 DEBUG:teuthology.task.kernel:current kernel version is 6.9.0-rc3-g806ec36b2112 vs 6.9.0-rc3-g806ec36b2112 2024-04-26T21:32:07.361 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-26T21:32:07.361 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-04-26T21:32:08.200 DEBUG:teuthology.orchestra.remote:timed out 2024-04-26T21:32:08.362 INFO:teuthology.task.kernel:Enabling kdb on mon.c... 2024-04-26T21:32:08.362 DEBUG:teuthology.orchestra.run.smithi164:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-26T21:32:08.556 INFO:teuthology.orchestra.run.smithi164.stdout:ttyS1 2024-04-26T21:32:08.629 DEBUG:teuthology.parallel:result is None 2024-04-26T21:32:23.201 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi072.front.sepia.ceph.com' 2024-04-26T21:32:23.202 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T21:32:23.647 DEBUG:teuthology.orchestra.run.smithi072:> true 2024-04-26T21:32:24.334 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi072.front.sepia.ceph.com' 2024-04-26T21:32:24.335 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "6.9.0-rc3-g806ec36b2112"... 2024-04-26T21:32:24.335 DEBUG:teuthology.orchestra.run.smithi072:> uname -r 2024-04-26T21:32:24.385 INFO:teuthology.orchestra.run.smithi072.stdout:6.9.0-rc3-g806ec36b2112 2024-04-26T21:32:24.385 DEBUG:teuthology.task.kernel:current kernel version is 6.9.0-rc3-g806ec36b2112 vs 6.9.0-rc3-g806ec36b2112 2024-04-26T21:32:24.385 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-26T21:32:24.385 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-04-26T21:32:25.387 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-04-26T21:32:25.387 DEBUG:teuthology.orchestra.run.smithi072:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-26T21:32:25.564 INFO:teuthology.orchestra.run.smithi072.stdout:ttyS1 2024-04-26T21:32:25.618 DEBUG:teuthology.parallel:result is None 2024-04-26T21:32:25.619 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-26T21:32:25.625 INFO:teuthology.task.internal:Creating test directory... 2024-04-26T21:32:25.625 DEBUG:teuthology.orchestra.run.smithi072:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-26T21:32:25.633 DEBUG:teuthology.orchestra.run.smithi076:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-26T21:32:25.641 DEBUG:teuthology.orchestra.run.smithi164:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-26T21:32:25.698 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-26T21:32:25.705 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-26T21:32:25.712 INFO:teuthology.task.internal:Creating archive directory... 2024-04-26T21:32:25.712 DEBUG:teuthology.orchestra.run.smithi072:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-26T21:32:25.717 DEBUG:teuthology.orchestra.run.smithi076:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-26T21:32:25.724 DEBUG:teuthology.orchestra.run.smithi164:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-26T21:32:25.793 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-26T21:32:25.799 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-26T21:32:25.800 DEBUG:teuthology.orchestra.run.smithi072:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-26T21:32:25.804 DEBUG:teuthology.orchestra.run.smithi076:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-26T21:32:25.811 DEBUG:teuthology.orchestra.run.smithi164:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-26T21:32:25.888 INFO:teuthology.orchestra.run.smithi072.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-26T21:32:25.902 INFO:teuthology.orchestra.run.smithi076.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-26T21:32:25.909 INFO:teuthology.orchestra.run.smithi164.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-26T21:32:25.918 INFO:teuthology.orchestra.run.smithi072.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-26T21:32:25.936 INFO:teuthology.orchestra.run.smithi076.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-26T21:32:25.941 INFO:teuthology.orchestra.run.smithi164.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-26T21:32:25.945 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-26T21:32:25.952 INFO:teuthology.task.internal:Configuring sudo... 2024-04-26T21:32:25.952 DEBUG:teuthology.orchestra.run.smithi072:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-26T21:32:25.959 DEBUG:teuthology.orchestra.run.smithi076:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-26T21:32:25.964 DEBUG:teuthology.orchestra.run.smithi164:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-26T21:32:26.045 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-26T21:32:26.054 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-26T21:32:26.054 DEBUG:teuthology.orchestra.run.smithi072:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-26T21:32:26.062 DEBUG:teuthology.orchestra.run.smithi076:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-26T21:32:26.067 DEBUG:teuthology.orchestra.run.smithi164:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-26T21:32:26.118 DEBUG:teuthology.orchestra.run.smithi072:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-26T21:32:26.192 DEBUG:teuthology.orchestra.run.smithi072:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-26T21:32:26.285 DEBUG:teuthology.orchestra.run.smithi072:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-26T21:32:26.370 DEBUG:teuthology.orchestra.run.smithi072:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-26T21:32:26.455 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2024-04-26T21:32:26.455 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-26T21:32:26.539 DEBUG:teuthology.orchestra.run.smithi076:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-26T21:32:26.625 DEBUG:teuthology.orchestra.run.smithi076:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-26T21:32:26.710 DEBUG:teuthology.orchestra.run.smithi076:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-26T21:32:26.791 DEBUG:teuthology.orchestra.run.smithi076:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-26T21:32:26.872 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-04-26T21:32:26.872 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-26T21:32:26.952 DEBUG:teuthology.orchestra.run.smithi164:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-26T21:32:27.036 DEBUG:teuthology.orchestra.run.smithi164:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-26T21:32:27.138 DEBUG:teuthology.orchestra.run.smithi164:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-26T21:32:27.222 DEBUG:teuthology.orchestra.run.smithi164:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-26T21:32:27.310 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-04-26T21:32:27.310 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-26T21:32:27.395 DEBUG:teuthology.orchestra.run.smithi072:> sudo service rsyslog restart 2024-04-26T21:32:27.402 DEBUG:teuthology.orchestra.run.smithi076:> sudo service rsyslog restart 2024-04-26T21:32:27.409 DEBUG:teuthology.orchestra.run.smithi164:> sudo service rsyslog restart 2024-04-26T21:32:27.508 INFO:teuthology.orchestra.run.smithi072.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-26T21:32:27.517 INFO:teuthology.orchestra.run.smithi076.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-26T21:32:27.562 INFO:teuthology.orchestra.run.smithi164.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-26T21:32:27.904 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-26T21:32:27.912 INFO:teuthology.task.internal:Starting timer... 2024-04-26T21:32:27.912 INFO:teuthology.run_tasks:Running task pcp... 2024-04-26T21:32:27.927 INFO:teuthology.run_tasks:Running task selinux... 2024-04-26T21:32:27.935 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-04-26T21:32:27.935 DEBUG:teuthology.orchestra.run.smithi072:> sudo service auditd rotate 2024-04-26T21:32:28.039 INFO:teuthology.orchestra.run.smithi072.stdout:Rotating logs: 2024-04-26T21:32:28.043 DEBUG:teuthology.orchestra.run.smithi076:> sudo service auditd rotate 2024-04-26T21:32:28.197 INFO:teuthology.orchestra.run.smithi076.stdout:Rotating logs: 2024-04-26T21:32:28.202 DEBUG:teuthology.orchestra.run.smithi164:> sudo service auditd rotate 2024-04-26T21:32:28.333 INFO:teuthology.orchestra.run.smithi164.stdout:Rotating logs: 2024-04-26T21:32:28.338 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-26T21:32:28.338 DEBUG:teuthology.orchestra.run.smithi072:> /usr/sbin/getenforce 2024-04-26T21:32:28.393 INFO:teuthology.orchestra.run.smithi072.stdout:Permissive 2024-04-26T21:32:28.393 DEBUG:teuthology.orchestra.run.smithi076:> /usr/sbin/getenforce 2024-04-26T21:32:28.451 INFO:teuthology.orchestra.run.smithi076.stdout:Permissive 2024-04-26T21:32:28.452 DEBUG:teuthology.orchestra.run.smithi164:> /usr/sbin/getenforce 2024-04-26T21:32:28.503 INFO:teuthology.orchestra.run.smithi164.stdout:Permissive 2024-04-26T21:32:28.503 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi072.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi076.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi164.front.sepia.ceph.com': 'permissive'} 2024-04-26T21:32:28.503 DEBUG:teuthology.orchestra.run.smithi072:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-26T21:32:28.580 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T21:32:28.580 DEBUG:teuthology.orchestra.run.smithi076:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-26T21:32:28.659 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T21:32:28.660 DEBUG:teuthology.orchestra.run.smithi164:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-26T21:32:28.737 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T21:32:28.737 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-26T21:32:28.737 DEBUG:teuthology.orchestra.run.smithi072:> sudo /usr/sbin/setenforce permissive 2024-04-26T21:32:28.816 DEBUG:teuthology.orchestra.run.smithi076:> sudo /usr/sbin/setenforce permissive 2024-04-26T21:32:28.888 DEBUG:teuthology.orchestra.run.smithi164:> sudo /usr/sbin/setenforce permissive 2024-04-26T21:32:28.973 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-26T21:32:28.988 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-04-26T21:32:28.996 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-04-26T21:32:29.147 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-26T21:32:29.162 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-26T21:32:29.163 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi072.front.sepia.ceph.com,smithi076.front.sepia.ceph.com,smithi164.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-26T21:37:37.737 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi072.front.sepia.ceph.com'), Remote(name='ubuntu@smithi076.front.sepia.ceph.com'), Remote(name='ubuntu@smithi164.front.sepia.ceph.com')] 2024-04-26T21:37:37.738 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi072.front.sepia.ceph.com' 2024-04-26T21:37:37.740 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T21:37:37.853 DEBUG:teuthology.orchestra.run.smithi072:> true 2024-04-26T21:37:38.008 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi072.front.sepia.ceph.com' 2024-04-26T21:37:38.008 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi076.front.sepia.ceph.com' 2024-04-26T21:37:38.008 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi076.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T21:37:38.117 DEBUG:teuthology.orchestra.run.smithi076:> true 2024-04-26T21:37:38.276 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi076.front.sepia.ceph.com' 2024-04-26T21:37:38.276 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi164.front.sepia.ceph.com' 2024-04-26T21:37:38.277 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi164.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T21:37:38.382 DEBUG:teuthology.orchestra.run.smithi164:> true 2024-04-26T21:37:38.531 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi164.front.sepia.ceph.com' 2024-04-26T21:37:38.532 INFO:teuthology.run_tasks:Running task clock... 2024-04-26T21:37:38.543 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-26T21:37:38.544 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-26T21:37:38.544 DEBUG:teuthology.orchestra.run.smithi072:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-26T21:37:38.552 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-26T21:37:38.552 DEBUG:teuthology.orchestra.run.smithi076:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-26T21:37:38.559 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-26T21:37:38.559 DEBUG:teuthology.orchestra.run.smithi164:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-26T21:37:38.638 INFO:teuthology.orchestra.run.smithi072.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-26T21:37:38.648 INFO:teuthology.orchestra.run.smithi076.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-26T21:37:38.654 INFO:teuthology.orchestra.run.smithi164.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-26T21:37:38.679 INFO:teuthology.orchestra.run.smithi072.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-26T21:37:38.690 INFO:teuthology.orchestra.run.smithi076.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-26T21:37:38.695 INFO:teuthology.orchestra.run.smithi164.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-26T21:37:38.751 INFO:teuthology.orchestra.run.smithi072.stderr:sudo: ntpd: command not found 2024-04-26T21:37:38.760 INFO:teuthology.orchestra.run.smithi076.stderr:sudo: ntpd: command not found 2024-04-26T21:37:38.770 INFO:teuthology.orchestra.run.smithi164.stderr:sudo: ntpd: command not found 2024-04-26T21:37:38.780 INFO:teuthology.orchestra.run.smithi072.stdout:506 Cannot talk to daemon 2024-04-26T21:37:38.789 INFO:teuthology.orchestra.run.smithi076.stdout:506 Cannot talk to daemon 2024-04-26T21:37:38.799 INFO:teuthology.orchestra.run.smithi164.stdout:506 Cannot talk to daemon 2024-04-26T21:37:38.821 INFO:teuthology.orchestra.run.smithi072.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-26T21:37:38.828 INFO:teuthology.orchestra.run.smithi076.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-26T21:37:38.841 INFO:teuthology.orchestra.run.smithi164.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-26T21:37:38.864 INFO:teuthology.orchestra.run.smithi072.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-26T21:37:38.869 INFO:teuthology.orchestra.run.smithi076.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-26T21:37:38.883 INFO:teuthology.orchestra.run.smithi164.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-26T21:37:38.995 INFO:teuthology.orchestra.run.smithi076.stderr:bash: line 1: ntpq: command not found 2024-04-26T21:37:39.004 INFO:teuthology.orchestra.run.smithi076.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-26T21:37:39.005 INFO:teuthology.orchestra.run.smithi076.stdout:=============================================================================== 2024-04-26T21:37:39.005 INFO:teuthology.orchestra.run.smithi076.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T21:37:39.005 INFO:teuthology.orchestra.run.smithi076.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T21:37:39.005 INFO:teuthology.orchestra.run.smithi076.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T21:37:39.005 INFO:teuthology.orchestra.run.smithi076.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T21:37:39.011 INFO:teuthology.orchestra.run.smithi164.stderr:bash: line 1: ntpq: command not found 2024-04-26T21:37:39.011 INFO:teuthology.orchestra.run.smithi072.stderr:bash: line 1: ntpq: command not found 2024-04-26T21:37:39.018 INFO:teuthology.orchestra.run.smithi164.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-26T21:37:39.019 INFO:teuthology.orchestra.run.smithi164.stdout:=============================================================================== 2024-04-26T21:37:39.019 INFO:teuthology.orchestra.run.smithi164.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T21:37:39.019 INFO:teuthology.orchestra.run.smithi164.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T21:37:39.019 INFO:teuthology.orchestra.run.smithi164.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T21:37:39.019 INFO:teuthology.orchestra.run.smithi164.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T21:37:39.021 INFO:teuthology.orchestra.run.smithi072.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-26T21:37:39.021 INFO:teuthology.orchestra.run.smithi072.stdout:=============================================================================== 2024-04-26T21:37:39.021 INFO:teuthology.orchestra.run.smithi072.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T21:37:39.021 INFO:teuthology.orchestra.run.smithi072.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T21:37:39.021 INFO:teuthology.orchestra.run.smithi072.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T21:37:39.021 INFO:teuthology.orchestra.run.smithi072.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T21:37:39.022 INFO:teuthology.run_tasks:Running task install... 2024-04-26T21:37:39.031 DEBUG:teuthology.task.install:project ceph 2024-04-26T21:37:39.032 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387'}} 2024-04-26T21:37:39.032 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387'} 2024-04-26T21:37:39.032 INFO:teuthology.task.install:Using flavor: default 2024-04-26T21:37:39.038 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-26T21:37:39.038 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-04-26T21:37:39.039 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387', 'tag': None, 'wait_for_package': False} 2024-04-26T21:37:39.039 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T21:37:39.041 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387', 'tag': None, 'wait_for_package': False} 2024-04-26T21:37:39.042 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T21:37:39.043 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387', 'tag': None, 'wait_for_package': False} 2024-04-26T21:37:39.043 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T21:37:39.264 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/ 2024-04-26T21:37:39.264 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3306.gbc612ffb 2024-04-26T21:37:39.269 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/ 2024-04-26T21:37:39.269 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3306.gbc612ffb 2024-04-26T21:37:39.273 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/ 2024-04-26T21:37:39.274 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3306.gbc612ffb 2024-04-26T21:37:39.466 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-26T21:37:39.466 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-04-26T21:37:39.466 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-26T21:37:39.470 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-26T21:37:39.470 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-04-26T21:37:39.470 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-26T21:37:39.471 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-26T21:37:39.471 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2024-04-26T21:37:39.471 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-26T21:37:39.554 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-26T21:37:39.554 DEBUG:teuthology.orchestra.run.smithi076:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/bc612ffb9c1d5bc27536bc917a705262a36d4387/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-26T21:37:39.558 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-26T21:37:39.558 DEBUG:teuthology.orchestra.run.smithi164:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/bc612ffb9c1d5bc27536bc917a705262a36d4387/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-26T21:37:39.560 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-26T21:37:39.560 DEBUG:teuthology.orchestra.run.smithi072:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/bc612ffb9c1d5bc27536bc917a705262a36d4387/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-26T21:37:39.685 DEBUG:teuthology.orchestra.run.smithi076:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-26T21:37:39.686 DEBUG:teuthology.orchestra.run.smithi072:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-26T21:37:39.687 DEBUG:teuthology.orchestra.run.smithi164:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-26T21:37:39.796 DEBUG:teuthology.orchestra.run.smithi072:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-26T21:37:39.798 DEBUG:teuthology.orchestra.run.smithi164:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-26T21:37:39.800 DEBUG:teuthology.orchestra.run.smithi076:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-26T21:37:39.866 INFO:teuthology.orchestra.run.smithi076.stdout:check_obsoletes = 1 2024-04-26T21:37:39.869 DEBUG:teuthology.orchestra.run.smithi076:> sudo yum clean all 2024-04-26T21:37:39.872 INFO:teuthology.orchestra.run.smithi164.stdout:check_obsoletes = 1 2024-04-26T21:37:39.875 INFO:teuthology.orchestra.run.smithi072.stdout:check_obsoletes = 1 2024-04-26T21:37:39.877 DEBUG:teuthology.orchestra.run.smithi164:> sudo yum clean all 2024-04-26T21:37:39.878 DEBUG:teuthology.orchestra.run.smithi072:> sudo yum clean all 2024-04-26T21:37:40.178 INFO:teuthology.orchestra.run.smithi076.stdout:48 files removed 2024-04-26T21:37:40.199 INFO:teuthology.orchestra.run.smithi072.stdout:48 files removed 2024-04-26T21:37:40.202 INFO:teuthology.orchestra.run.smithi164.stdout:48 files removed 2024-04-26T21:37:40.216 DEBUG:teuthology.orchestra.run.smithi076:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-26T21:37:40.238 DEBUG:teuthology.orchestra.run.smithi072:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-26T21:37:40.240 DEBUG:teuthology.orchestra.run.smithi164:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-26T21:37:41.127 INFO:teuthology.orchestra.run.smithi072.stdout:ceph packages for x86_64 184 kB/s | 84 kB 00:00 2024-04-26T21:37:41.159 INFO:teuthology.orchestra.run.smithi164.stdout:ceph packages for x86_64 175 kB/s | 84 kB 00:00 2024-04-26T21:37:41.169 INFO:teuthology.orchestra.run.smithi076.stdout:ceph packages for x86_64 165 kB/s | 84 kB 00:00 2024-04-26T21:37:41.603 INFO:teuthology.orchestra.run.smithi072.stdout:ceph noarch packages 31 kB/s | 12 kB 00:00 2024-04-26T21:37:41.684 INFO:teuthology.orchestra.run.smithi164.stdout:ceph noarch packages 27 kB/s | 12 kB 00:00 2024-04-26T21:37:41.779 INFO:teuthology.orchestra.run.smithi076.stdout:ceph noarch packages 23 kB/s | 12 kB 00:00 2024-04-26T21:37:42.104 INFO:teuthology.orchestra.run.smithi072.stdout:ceph source packages 4.5 kB/s | 1.9 kB 00:00 2024-04-26T21:37:42.202 INFO:teuthology.orchestra.run.smithi164.stdout:ceph source packages 4.4 kB/s | 1.9 kB 00:00 2024-04-26T21:37:42.455 INFO:teuthology.orchestra.run.smithi076.stdout:ceph source packages 3.2 kB/s | 1.9 kB 00:00 2024-04-26T21:37:43.117 INFO:teuthology.orchestra.run.smithi164.stdout:CentOS Stream 9 - BaseOS 9.7 MB/s | 8.0 MB 00:00 2024-04-26T21:37:43.179 INFO:teuthology.orchestra.run.smithi072.stdout:CentOS Stream 9 - BaseOS 8.1 MB/s | 8.0 MB 00:00 2024-04-26T21:37:43.503 INFO:teuthology.orchestra.run.smithi076.stdout:CentOS Stream 9 - BaseOS 8.4 MB/s | 8.0 MB 00:00 2024-04-26T21:37:45.126 INFO:teuthology.orchestra.run.smithi164.stdout:CentOS Stream 9 - AppStream 24 MB/s | 19 MB 00:00 2024-04-26T21:37:45.830 INFO:teuthology.orchestra.run.smithi076.stdout:CentOS Stream 9 - AppStream 17 MB/s | 19 MB 00:01 2024-04-26T21:37:49.877 INFO:teuthology.orchestra.run.smithi164.stdout:CentOS Stream 9 - CRB 7.4 MB/s | 6.2 MB 00:00 2024-04-26T21:37:50.745 INFO:teuthology.orchestra.run.smithi076.stdout:CentOS Stream 9 - CRB 6.2 MB/s | 6.2 MB 00:00 2024-04-26T21:37:52.310 INFO:teuthology.orchestra.run.smithi164.stdout:CentOS Stream 9 - Extras packages 15 kB/s | 16 kB 00:01 2024-04-26T21:37:52.684 INFO:teuthology.orchestra.run.smithi076.stdout:CentOS Stream 9 - Extras packages 28 kB/s | 16 kB 00:00 2024-04-26T21:37:54.448 INFO:teuthology.orchestra.run.smithi076.stdout:Extra Packages for Enterprise Linux 13 MB/s | 21 MB 00:01 2024-04-26T21:37:57.412 INFO:teuthology.orchestra.run.smithi164.stdout:Extra Packages for Enterprise Linux 4.3 MB/s | 21 MB 00:04 2024-04-26T21:38:00.351 INFO:teuthology.orchestra.run.smithi076.stdout:lab-extras 27 kB/s | 1.7 kB 00:00 2024-04-26T21:38:02.735 INFO:teuthology.orchestra.run.smithi076.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-26T21:38:02.737 INFO:teuthology.orchestra.run.smithi076.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-26T21:38:02.757 INFO:teuthology.orchestra.run.smithi076.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-26T21:38:02.758 INFO:teuthology.orchestra.run.smithi076.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-26T21:38:02.759 INFO:teuthology.orchestra.run.smithi076.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-26T21:38:02.761 INFO:teuthology.orchestra.run.smithi076.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-26T21:38:02.763 INFO:teuthology.orchestra.run.smithi076.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-26T21:38:02.766 INFO:teuthology.orchestra.run.smithi076.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-26T21:38:02.767 INFO:teuthology.orchestra.run.smithi076.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-26T21:38:02.838 INFO:teuthology.orchestra.run.smithi076.stdout:Dependencies resolved. 2024-04-26T21:38:02.856 INFO:teuthology.orchestra.run.smithi076.stdout:======================================================================================= 2024-04-26T21:38:02.856 INFO:teuthology.orchestra.run.smithi076.stdout: Package Arch Version Repository Size 2024-04-26T21:38:02.856 INFO:teuthology.orchestra.run.smithi076.stdout:======================================================================================= 2024-04-26T21:38:02.856 INFO:teuthology.orchestra.run.smithi076.stdout:Installing: 2024-04-26T21:38:02.856 INFO:teuthology.orchestra.run.smithi076.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-26T21:38:02.857 INFO:teuthology.orchestra.run.smithi076.stdout: ceph x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 6.6 k 2024-04-26T21:38:02.857 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-base x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 5.4 M 2024-04-26T21:38:02.857 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-fuse x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 889 k 2024-04-26T21:38:02.857 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 145 k 2024-04-26T21:38:02.857 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mgr x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 1.5 M 2024-04-26T21:38:02.857 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 142 k 2024-04-26T21:38:02.857 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 3.5 M 2024-04-26T21:38:02.857 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 7.4 M 2024-04-26T21:38:02.857 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mgr-rook noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 49 k 2024-04-26T21:38:02.857 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-radosgw x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 11 M 2024-04-26T21:38:02.857 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-test x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 49 M 2024-04-26T21:38:02.857 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-volume noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 273 k 2024-04-26T21:38:02.857 INFO:teuthology.orchestra.run.smithi076.stdout: cephadm noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 762 k 2024-04-26T21:38:02.857 INFO:teuthology.orchestra.run.smithi076.stdout: cephfs-mirror x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 225 k 2024-04-26T21:38:02.858 INFO:teuthology.orchestra.run.smithi076.stdout: cephfs-shell x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 27 k 2024-04-26T21:38:02.858 INFO:teuthology.orchestra.run.smithi076.stdout: cephfs-top noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 18 k 2024-04-26T21:38:02.858 INFO:teuthology.orchestra.run.smithi076.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-26T21:38:02.858 INFO:teuthology.orchestra.run.smithi076.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-26T21:38:02.858 INFO:teuthology.orchestra.run.smithi076.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-26T21:38:02.858 INFO:teuthology.orchestra.run.smithi076.stdout: libcephfs-devel x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 32 k 2024-04-26T21:38:02.858 INFO:teuthology.orchestra.run.smithi076.stdout: libcephfs2 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 747 k 2024-04-26T21:38:02.858 INFO:teuthology.orchestra.run.smithi076.stdout: librados-devel x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 127 k 2024-04-26T21:38:02.858 INFO:teuthology.orchestra.run.smithi076.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-26T21:38:02.858 INFO:teuthology.orchestra.run.smithi076.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-26T21:38:02.858 INFO:teuthology.orchestra.run.smithi076.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-26T21:38:02.859 INFO:teuthology.orchestra.run.smithi076.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-26T21:38:02.859 INFO:teuthology.orchestra.run.smithi076.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-26T21:38:02.859 INFO:teuthology.orchestra.run.smithi076.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-26T21:38:02.859 INFO:teuthology.orchestra.run.smithi076.stdout: python3-cephfs x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 161 k 2024-04-26T21:38:02.859 INFO:teuthology.orchestra.run.smithi076.stdout: python3-rados x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 321 k 2024-04-26T21:38:02.859 INFO:teuthology.orchestra.run.smithi076.stdout: python3-rbd x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 298 k 2024-04-26T21:38:02.859 INFO:teuthology.orchestra.run.smithi076.stdout: python3-rgw x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 99 k 2024-04-26T21:38:02.859 INFO:teuthology.orchestra.run.smithi076.stdout: rbd-fuse x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 85 k 2024-04-26T21:38:02.859 INFO:teuthology.orchestra.run.smithi076.stdout: rbd-mirror x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 3.1 M 2024-04-26T21:38:02.859 INFO:teuthology.orchestra.run.smithi076.stdout: rbd-nbd x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 171 k 2024-04-26T21:38:02.859 INFO:teuthology.orchestra.run.smithi076.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-26T21:38:02.859 INFO:teuthology.orchestra.run.smithi076.stdout:Upgrading: 2024-04-26T21:38:02.859 INFO:teuthology.orchestra.run.smithi076.stdout: librados2 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 3.4 M 2024-04-26T21:38:02.860 INFO:teuthology.orchestra.run.smithi076.stdout: librbd1 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 3.1 M 2024-04-26T21:38:02.860 INFO:teuthology.orchestra.run.smithi076.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-26T21:38:02.860 INFO:teuthology.orchestra.run.smithi076.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-26T21:38:02.860 INFO:teuthology.orchestra.run.smithi076.stdout:Installing dependencies: 2024-04-26T21:38:02.860 INFO:teuthology.orchestra.run.smithi076.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-26T21:38:02.860 INFO:teuthology.orchestra.run.smithi076.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-26T21:38:02.860 INFO:teuthology.orchestra.run.smithi076.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-26T21:38:02.860 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-common x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 21 M 2024-04-26T21:38:02.860 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 34 k 2024-04-26T21:38:02.860 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mds x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 2.4 M 2024-04-26T21:38:02.860 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 249 k 2024-04-26T21:38:02.860 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mon x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 4.7 M 2024-04-26T21:38:02.860 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-osd x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 17 M 2024-04-26T21:38:02.861 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 16 k 2024-04-26T21:38:02.861 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-selinux x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 25 k 2024-04-26T21:38:02.861 INFO:teuthology.orchestra.run.smithi076.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-26T21:38:02.861 INFO:teuthology.orchestra.run.smithi076.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-26T21:38:02.861 INFO:teuthology.orchestra.run.smithi076.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-26T21:38:02.861 INFO:teuthology.orchestra.run.smithi076.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-26T21:38:02.861 INFO:teuthology.orchestra.run.smithi076.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-26T21:38:02.861 INFO:teuthology.orchestra.run.smithi076.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-26T21:38:02.861 INFO:teuthology.orchestra.run.smithi076.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-04-26T21:38:02.861 INFO:teuthology.orchestra.run.smithi076.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-04-26T21:38:02.861 INFO:teuthology.orchestra.run.smithi076.stdout: libcephsqlite x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 163 k 2024-04-26T21:38:02.861 INFO:teuthology.orchestra.run.smithi076.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-26T21:38:02.861 INFO:teuthology.orchestra.run.smithi076.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-26T21:38:02.861 INFO:teuthology.orchestra.run.smithi076.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-26T21:38:02.862 INFO:teuthology.orchestra.run.smithi076.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-26T21:38:02.862 INFO:teuthology.orchestra.run.smithi076.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-26T21:38:02.862 INFO:teuthology.orchestra.run.smithi076.stdout: libradosstriper1 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 501 k 2024-04-26T21:38:02.862 INFO:teuthology.orchestra.run.smithi076.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-26T21:38:02.862 INFO:teuthology.orchestra.run.smithi076.stdout: librgw2 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 5.4 M 2024-04-26T21:38:02.862 INFO:teuthology.orchestra.run.smithi076.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-26T21:38:02.862 INFO:teuthology.orchestra.run.smithi076.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-26T21:38:02.862 INFO:teuthology.orchestra.run.smithi076.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-26T21:38:02.862 INFO:teuthology.orchestra.run.smithi076.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-26T21:38:02.862 INFO:teuthology.orchestra.run.smithi076.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-26T21:38:02.862 INFO:teuthology.orchestra.run.smithi076.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-26T21:38:02.862 INFO:teuthology.orchestra.run.smithi076.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-26T21:38:02.862 INFO:teuthology.orchestra.run.smithi076.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-26T21:38:02.863 INFO:teuthology.orchestra.run.smithi076.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-04-26T21:38:02.863 INFO:teuthology.orchestra.run.smithi076.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-26T21:38:02.863 INFO:teuthology.orchestra.run.smithi076.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-26T21:38:02.863 INFO:teuthology.orchestra.run.smithi076.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-26T21:38:02.863 INFO:teuthology.orchestra.run.smithi076.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-26T21:38:02.863 INFO:teuthology.orchestra.run.smithi076.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-26T21:38:02.863 INFO:teuthology.orchestra.run.smithi076.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-26T21:38:02.863 INFO:teuthology.orchestra.run.smithi076.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-26T21:38:02.863 INFO:teuthology.orchestra.run.smithi076.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-26T21:38:02.863 INFO:teuthology.orchestra.run.smithi076.stdout: python3-ceph-argparse x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 45 k 2024-04-26T21:38:02.863 INFO:teuthology.orchestra.run.smithi076.stdout: python3-ceph-common x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 135 k 2024-04-26T21:38:02.863 INFO:teuthology.orchestra.run.smithi076.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-26T21:38:02.863 INFO:teuthology.orchestra.run.smithi076.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-26T21:38:02.864 INFO:teuthology.orchestra.run.smithi076.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-26T21:38:02.864 INFO:teuthology.orchestra.run.smithi076.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-26T21:38:02.864 INFO:teuthology.orchestra.run.smithi076.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-26T21:38:02.864 INFO:teuthology.orchestra.run.smithi076.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-26T21:38:02.864 INFO:teuthology.orchestra.run.smithi076.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-26T21:38:02.864 INFO:teuthology.orchestra.run.smithi076.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-26T21:38:02.864 INFO:teuthology.orchestra.run.smithi076.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-26T21:38:02.864 INFO:teuthology.orchestra.run.smithi076.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-26T21:38:02.864 INFO:teuthology.orchestra.run.smithi076.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-26T21:38:02.864 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-26T21:38:02.864 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-26T21:38:02.864 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-26T21:38:02.864 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-26T21:38:02.864 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-26T21:38:02.865 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-26T21:38:02.865 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-26T21:38:02.865 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-26T21:38:02.865 INFO:teuthology.orchestra.run.smithi076.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-26T21:38:02.865 INFO:teuthology.orchestra.run.smithi076.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-26T21:38:02.865 INFO:teuthology.orchestra.run.smithi076.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-26T21:38:02.865 INFO:teuthology.orchestra.run.smithi076.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-26T21:38:02.865 INFO:teuthology.orchestra.run.smithi076.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-26T21:38:02.865 INFO:teuthology.orchestra.run.smithi076.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-26T21:38:02.865 INFO:teuthology.orchestra.run.smithi076.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-26T21:38:02.865 INFO:teuthology.orchestra.run.smithi076.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-26T21:38:02.865 INFO:teuthology.orchestra.run.smithi076.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-26T21:38:02.866 INFO:teuthology.orchestra.run.smithi076.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-26T21:38:02.866 INFO:teuthology.orchestra.run.smithi076.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-26T21:38:02.866 INFO:teuthology.orchestra.run.smithi076.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-26T21:38:02.866 INFO:teuthology.orchestra.run.smithi076.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-26T21:38:02.866 INFO:teuthology.orchestra.run.smithi076.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-26T21:38:02.866 INFO:teuthology.orchestra.run.smithi076.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-26T21:38:02.866 INFO:teuthology.orchestra.run.smithi076.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-26T21:38:02.866 INFO:teuthology.orchestra.run.smithi076.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-26T21:38:02.866 INFO:teuthology.orchestra.run.smithi076.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-26T21:38:02.866 INFO:teuthology.orchestra.run.smithi076.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-26T21:38:02.866 INFO:teuthology.orchestra.run.smithi076.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-26T21:38:02.866 INFO:teuthology.orchestra.run.smithi076.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-26T21:38:02.866 INFO:teuthology.orchestra.run.smithi076.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-26T21:38:02.867 INFO:teuthology.orchestra.run.smithi076.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-26T21:38:02.867 INFO:teuthology.orchestra.run.smithi076.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-26T21:38:02.867 INFO:teuthology.orchestra.run.smithi076.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-26T21:38:02.867 INFO:teuthology.orchestra.run.smithi076.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-26T21:38:02.867 INFO:teuthology.orchestra.run.smithi076.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-26T21:38:02.867 INFO:teuthology.orchestra.run.smithi076.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-26T21:38:02.867 INFO:teuthology.orchestra.run.smithi076.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-26T21:38:02.867 INFO:teuthology.orchestra.run.smithi076.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-26T21:38:02.867 INFO:teuthology.orchestra.run.smithi076.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-26T21:38:02.867 INFO:teuthology.orchestra.run.smithi076.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-26T21:38:02.867 INFO:teuthology.orchestra.run.smithi076.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-26T21:38:02.868 INFO:teuthology.orchestra.run.smithi076.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-26T21:38:02.868 INFO:teuthology.orchestra.run.smithi076.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-26T21:38:02.868 INFO:teuthology.orchestra.run.smithi076.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-26T21:38:02.868 INFO:teuthology.orchestra.run.smithi076.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-26T21:38:02.868 INFO:teuthology.orchestra.run.smithi076.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-26T21:38:02.868 INFO:teuthology.orchestra.run.smithi076.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-26T21:38:02.868 INFO:teuthology.orchestra.run.smithi076.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-26T21:38:02.869 INFO:teuthology.orchestra.run.smithi076.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-26T21:38:02.869 INFO:teuthology.orchestra.run.smithi076.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-26T21:38:02.869 INFO:teuthology.orchestra.run.smithi076.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-26T21:38:02.869 INFO:teuthology.orchestra.run.smithi076.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-26T21:38:02.869 INFO:teuthology.orchestra.run.smithi076.stdout:Installing weak dependencies: 2024-04-26T21:38:02.869 INFO:teuthology.orchestra.run.smithi076.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-26T21:38:02.869 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-26T21:38:02.870 INFO:teuthology.orchestra.run.smithi076.stdout:Transaction Summary 2024-04-26T21:38:02.870 INFO:teuthology.orchestra.run.smithi076.stdout:======================================================================================= 2024-04-26T21:38:02.870 INFO:teuthology.orchestra.run.smithi076.stdout:Install 148 Packages 2024-04-26T21:38:02.870 INFO:teuthology.orchestra.run.smithi076.stdout:Upgrade 4 Packages 2024-04-26T21:38:02.870 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-26T21:38:02.871 INFO:teuthology.orchestra.run.smithi076.stdout:Total download size: 222 M 2024-04-26T21:38:02.871 INFO:teuthology.orchestra.run.smithi076.stdout:Downloading Packages: 2024-04-26T21:38:03.353 INFO:teuthology.orchestra.run.smithi164.stdout:lab-extras 17 kB/s | 1.7 kB 00:00 2024-04-26T21:38:03.356 INFO:teuthology.orchestra.run.smithi072.stdout:CentOS Stream 9 - AppStream 1.0 MB/s | 19 MB 00:18 2024-04-26T21:38:04.261 INFO:teuthology.orchestra.run.smithi076.stdout:(1/152): ceph-19.0.0-3306.gbc612ffb.el9.x86_64. 18 kB/s | 6.6 kB 00:00 2024-04-26T21:38:04.636 INFO:teuthology.orchestra.run.smithi076.stdout:(2/152): ceph-fuse-19.0.0-3306.gbc612ffb.el9.x8 2.3 MB/s | 889 kB 00:00 2024-04-26T21:38:04.737 INFO:teuthology.orchestra.run.smithi076.stdout:(3/152): ceph-immutable-object-cache-19.0.0-330 1.4 MB/s | 145 kB 00:00 2024-04-26T21:38:04.854 INFO:teuthology.orchestra.run.smithi076.stdout:(4/152): ceph-base-19.0.0-3306.gbc612ffb.el9.x8 5.6 MB/s | 5.4 MB 00:00 2024-04-26T21:38:05.063 INFO:teuthology.orchestra.run.smithi076.stdout:(5/152): ceph-mds-19.0.0-3306.gbc612ffb.el9.x86 7.3 MB/s | 2.4 MB 00:00 2024-04-26T21:38:05.121 INFO:teuthology.orchestra.run.smithi076.stdout:(6/152): ceph-mgr-19.0.0-3306.gbc612ffb.el9.x86 5.7 MB/s | 1.5 MB 00:00 2024-04-26T21:38:05.630 INFO:teuthology.orchestra.run.smithi076.stdout:(7/152): ceph-mon-19.0.0-3306.gbc612ffb.el9.x86 8.3 MB/s | 4.7 MB 00:00 2024-04-26T21:38:05.758 INFO:teuthology.orchestra.run.smithi164.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-26T21:38:05.760 INFO:teuthology.orchestra.run.smithi164.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-26T21:38:05.780 INFO:teuthology.orchestra.run.smithi164.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-26T21:38:05.782 INFO:teuthology.orchestra.run.smithi164.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-26T21:38:05.783 INFO:teuthology.orchestra.run.smithi164.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-26T21:38:05.784 INFO:teuthology.orchestra.run.smithi164.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-26T21:38:05.787 INFO:teuthology.orchestra.run.smithi164.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-26T21:38:05.790 INFO:teuthology.orchestra.run.smithi164.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-26T21:38:05.791 INFO:teuthology.orchestra.run.smithi164.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-26T21:38:05.863 INFO:teuthology.orchestra.run.smithi164.stdout:Dependencies resolved. 2024-04-26T21:38:05.885 INFO:teuthology.orchestra.run.smithi164.stdout:======================================================================================= 2024-04-26T21:38:05.885 INFO:teuthology.orchestra.run.smithi164.stdout: Package Arch Version Repository Size 2024-04-26T21:38:05.886 INFO:teuthology.orchestra.run.smithi164.stdout:======================================================================================= 2024-04-26T21:38:05.886 INFO:teuthology.orchestra.run.smithi164.stdout:Installing: 2024-04-26T21:38:05.886 INFO:teuthology.orchestra.run.smithi164.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-26T21:38:05.886 INFO:teuthology.orchestra.run.smithi164.stdout: ceph x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 6.6 k 2024-04-26T21:38:05.886 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-base x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 5.4 M 2024-04-26T21:38:05.886 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-fuse x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 889 k 2024-04-26T21:38:05.886 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 145 k 2024-04-26T21:38:05.886 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 1.5 M 2024-04-26T21:38:05.886 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 142 k 2024-04-26T21:38:05.886 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 3.5 M 2024-04-26T21:38:05.886 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 7.4 M 2024-04-26T21:38:05.886 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-rook noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 49 k 2024-04-26T21:38:05.886 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-radosgw x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 11 M 2024-04-26T21:38:05.887 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-test x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 49 M 2024-04-26T21:38:05.887 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-volume noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 273 k 2024-04-26T21:38:05.887 INFO:teuthology.orchestra.run.smithi164.stdout: cephadm noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 762 k 2024-04-26T21:38:05.887 INFO:teuthology.orchestra.run.smithi164.stdout: cephfs-mirror x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 225 k 2024-04-26T21:38:05.887 INFO:teuthology.orchestra.run.smithi164.stdout: cephfs-shell x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 27 k 2024-04-26T21:38:05.887 INFO:teuthology.orchestra.run.smithi164.stdout: cephfs-top noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 18 k 2024-04-26T21:38:05.887 INFO:teuthology.orchestra.run.smithi164.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-26T21:38:05.887 INFO:teuthology.orchestra.run.smithi164.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-26T21:38:05.887 INFO:teuthology.orchestra.run.smithi164.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-26T21:38:05.887 INFO:teuthology.orchestra.run.smithi164.stdout: libcephfs-devel x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 32 k 2024-04-26T21:38:05.887 INFO:teuthology.orchestra.run.smithi164.stdout: libcephfs2 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 747 k 2024-04-26T21:38:05.887 INFO:teuthology.orchestra.run.smithi164.stdout: librados-devel x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 127 k 2024-04-26T21:38:05.888 INFO:teuthology.orchestra.run.smithi164.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-26T21:38:05.888 INFO:teuthology.orchestra.run.smithi164.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-26T21:38:05.888 INFO:teuthology.orchestra.run.smithi164.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-26T21:38:05.888 INFO:teuthology.orchestra.run.smithi164.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-26T21:38:05.888 INFO:teuthology.orchestra.run.smithi164.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-26T21:38:05.888 INFO:teuthology.orchestra.run.smithi164.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-26T21:38:05.888 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cephfs x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 161 k 2024-04-26T21:38:05.888 INFO:teuthology.orchestra.run.smithi164.stdout: python3-rados x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 321 k 2024-04-26T21:38:05.888 INFO:teuthology.orchestra.run.smithi164.stdout: python3-rbd x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 298 k 2024-04-26T21:38:05.888 INFO:teuthology.orchestra.run.smithi164.stdout: python3-rgw x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 99 k 2024-04-26T21:38:05.888 INFO:teuthology.orchestra.run.smithi164.stdout: rbd-fuse x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 85 k 2024-04-26T21:38:05.888 INFO:teuthology.orchestra.run.smithi164.stdout: rbd-mirror x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 3.1 M 2024-04-26T21:38:05.888 INFO:teuthology.orchestra.run.smithi164.stdout: rbd-nbd x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 171 k 2024-04-26T21:38:05.888 INFO:teuthology.orchestra.run.smithi164.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-26T21:38:05.889 INFO:teuthology.orchestra.run.smithi164.stdout:Upgrading: 2024-04-26T21:38:05.889 INFO:teuthology.orchestra.run.smithi164.stdout: librados2 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 3.4 M 2024-04-26T21:38:05.889 INFO:teuthology.orchestra.run.smithi164.stdout: librbd1 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 3.1 M 2024-04-26T21:38:05.889 INFO:teuthology.orchestra.run.smithi164.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-26T21:38:05.889 INFO:teuthology.orchestra.run.smithi164.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-26T21:38:05.889 INFO:teuthology.orchestra.run.smithi164.stdout:Installing dependencies: 2024-04-26T21:38:05.889 INFO:teuthology.orchestra.run.smithi164.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-26T21:38:05.889 INFO:teuthology.orchestra.run.smithi164.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-26T21:38:05.889 INFO:teuthology.orchestra.run.smithi164.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-26T21:38:05.889 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-common x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 21 M 2024-04-26T21:38:05.889 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 34 k 2024-04-26T21:38:05.889 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mds x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 2.4 M 2024-04-26T21:38:05.889 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 249 k 2024-04-26T21:38:05.890 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mon x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 4.7 M 2024-04-26T21:38:05.890 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-osd x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 17 M 2024-04-26T21:38:05.890 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 16 k 2024-04-26T21:38:05.890 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-selinux x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 25 k 2024-04-26T21:38:05.890 INFO:teuthology.orchestra.run.smithi164.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-26T21:38:05.890 INFO:teuthology.orchestra.run.smithi164.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-26T21:38:05.890 INFO:teuthology.orchestra.run.smithi164.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-26T21:38:05.890 INFO:teuthology.orchestra.run.smithi164.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-26T21:38:05.890 INFO:teuthology.orchestra.run.smithi164.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-26T21:38:05.890 INFO:teuthology.orchestra.run.smithi164.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-26T21:38:05.890 INFO:teuthology.orchestra.run.smithi164.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-04-26T21:38:05.890 INFO:teuthology.orchestra.run.smithi164.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-04-26T21:38:05.890 INFO:teuthology.orchestra.run.smithi164.stdout: libcephsqlite x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 163 k 2024-04-26T21:38:05.891 INFO:teuthology.orchestra.run.smithi164.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-26T21:38:05.891 INFO:teuthology.orchestra.run.smithi164.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-26T21:38:05.891 INFO:teuthology.orchestra.run.smithi164.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-26T21:38:05.891 INFO:teuthology.orchestra.run.smithi164.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-26T21:38:05.891 INFO:teuthology.orchestra.run.smithi164.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-26T21:38:05.891 INFO:teuthology.orchestra.run.smithi164.stdout: libradosstriper1 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 501 k 2024-04-26T21:38:05.891 INFO:teuthology.orchestra.run.smithi164.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-26T21:38:05.891 INFO:teuthology.orchestra.run.smithi164.stdout: librgw2 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 5.4 M 2024-04-26T21:38:05.891 INFO:teuthology.orchestra.run.smithi164.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-26T21:38:05.891 INFO:teuthology.orchestra.run.smithi164.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-26T21:38:05.891 INFO:teuthology.orchestra.run.smithi164.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-26T21:38:05.891 INFO:teuthology.orchestra.run.smithi164.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-26T21:38:05.891 INFO:teuthology.orchestra.run.smithi164.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-26T21:38:05.891 INFO:teuthology.orchestra.run.smithi164.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-26T21:38:05.892 INFO:teuthology.orchestra.run.smithi164.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-26T21:38:05.892 INFO:teuthology.orchestra.run.smithi164.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-26T21:38:05.892 INFO:teuthology.orchestra.run.smithi164.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-04-26T21:38:05.892 INFO:teuthology.orchestra.run.smithi164.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-26T21:38:05.892 INFO:teuthology.orchestra.run.smithi164.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-26T21:38:05.892 INFO:teuthology.orchestra.run.smithi164.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-26T21:38:05.892 INFO:teuthology.orchestra.run.smithi164.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-26T21:38:05.892 INFO:teuthology.orchestra.run.smithi164.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-26T21:38:05.892 INFO:teuthology.orchestra.run.smithi164.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-26T21:38:05.892 INFO:teuthology.orchestra.run.smithi164.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-26T21:38:05.892 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-26T21:38:05.892 INFO:teuthology.orchestra.run.smithi164.stdout: python3-ceph-argparse x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 45 k 2024-04-26T21:38:05.892 INFO:teuthology.orchestra.run.smithi164.stdout: python3-ceph-common x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 135 k 2024-04-26T21:38:05.893 INFO:teuthology.orchestra.run.smithi164.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-26T21:38:05.893 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-26T21:38:05.893 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-26T21:38:05.893 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-26T21:38:05.893 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-26T21:38:05.893 INFO:teuthology.orchestra.run.smithi164.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-26T21:38:05.893 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-26T21:38:05.893 INFO:teuthology.orchestra.run.smithi164.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-26T21:38:05.893 INFO:teuthology.orchestra.run.smithi164.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-26T21:38:05.893 INFO:teuthology.orchestra.run.smithi164.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-26T21:38:05.893 INFO:teuthology.orchestra.run.smithi164.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-26T21:38:05.893 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-26T21:38:05.893 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-26T21:38:05.894 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-26T21:38:05.894 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-26T21:38:05.894 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-26T21:38:05.894 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-26T21:38:05.894 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-26T21:38:05.894 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-26T21:38:05.894 INFO:teuthology.orchestra.run.smithi164.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-26T21:38:05.895 INFO:teuthology.orchestra.run.smithi164.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-26T21:38:05.895 INFO:teuthology.orchestra.run.smithi164.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-26T21:38:05.895 INFO:teuthology.orchestra.run.smithi164.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-26T21:38:05.895 INFO:teuthology.orchestra.run.smithi164.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-26T21:38:05.895 INFO:teuthology.orchestra.run.smithi164.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-26T21:38:05.895 INFO:teuthology.orchestra.run.smithi164.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-26T21:38:05.895 INFO:teuthology.orchestra.run.smithi164.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-26T21:38:05.895 INFO:teuthology.orchestra.run.smithi164.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-26T21:38:05.895 INFO:teuthology.orchestra.run.smithi164.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-26T21:38:05.895 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-26T21:38:05.895 INFO:teuthology.orchestra.run.smithi164.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-26T21:38:05.895 INFO:teuthology.orchestra.run.smithi164.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-26T21:38:05.895 INFO:teuthology.orchestra.run.smithi164.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-26T21:38:05.895 INFO:teuthology.orchestra.run.smithi164.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-26T21:38:05.896 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-26T21:38:05.896 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-26T21:38:05.896 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-26T21:38:05.896 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-26T21:38:05.896 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-26T21:38:05.896 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-26T21:38:05.896 INFO:teuthology.orchestra.run.smithi164.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-26T21:38:05.896 INFO:teuthology.orchestra.run.smithi164.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-26T21:38:05.896 INFO:teuthology.orchestra.run.smithi164.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-26T21:38:05.896 INFO:teuthology.orchestra.run.smithi164.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-26T21:38:05.896 INFO:teuthology.orchestra.run.smithi164.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-26T21:38:05.896 INFO:teuthology.orchestra.run.smithi164.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-26T21:38:05.896 INFO:teuthology.orchestra.run.smithi164.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-26T21:38:05.896 INFO:teuthology.orchestra.run.smithi164.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-26T21:38:05.897 INFO:teuthology.orchestra.run.smithi164.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-26T21:38:05.897 INFO:teuthology.orchestra.run.smithi164.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-26T21:38:05.897 INFO:teuthology.orchestra.run.smithi164.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-26T21:38:05.897 INFO:teuthology.orchestra.run.smithi164.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-26T21:38:05.897 INFO:teuthology.orchestra.run.smithi164.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-26T21:38:05.897 INFO:teuthology.orchestra.run.smithi164.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-26T21:38:05.897 INFO:teuthology.orchestra.run.smithi164.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-26T21:38:05.897 INFO:teuthology.orchestra.run.smithi164.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-26T21:38:05.897 INFO:teuthology.orchestra.run.smithi164.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-26T21:38:05.897 INFO:teuthology.orchestra.run.smithi164.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-26T21:38:05.897 INFO:teuthology.orchestra.run.smithi164.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-26T21:38:05.897 INFO:teuthology.orchestra.run.smithi164.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-26T21:38:05.897 INFO:teuthology.orchestra.run.smithi164.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-26T21:38:05.898 INFO:teuthology.orchestra.run.smithi164.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-26T21:38:05.898 INFO:teuthology.orchestra.run.smithi164.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-26T21:38:05.898 INFO:teuthology.orchestra.run.smithi164.stdout:Installing weak dependencies: 2024-04-26T21:38:05.898 INFO:teuthology.orchestra.run.smithi164.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-26T21:38:05.898 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-26T21:38:05.898 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction Summary 2024-04-26T21:38:05.898 INFO:teuthology.orchestra.run.smithi164.stdout:======================================================================================= 2024-04-26T21:38:05.898 INFO:teuthology.orchestra.run.smithi164.stdout:Install 148 Packages 2024-04-26T21:38:05.898 INFO:teuthology.orchestra.run.smithi164.stdout:Upgrade 4 Packages 2024-04-26T21:38:05.898 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-26T21:38:05.899 INFO:teuthology.orchestra.run.smithi164.stdout:Total download size: 222 M 2024-04-26T21:38:05.899 INFO:teuthology.orchestra.run.smithi164.stdout:Downloading Packages: 2024-04-26T21:38:06.814 INFO:teuthology.orchestra.run.smithi076.stdout:(8/152): ceph-osd-19.0.0-3306.gbc612ffb.el9.x86 10 MB/s | 17 MB 00:01 2024-04-26T21:38:06.930 INFO:teuthology.orchestra.run.smithi076.stdout:(9/152): ceph-selinux-19.0.0-3306.gbc612ffb.el9 217 kB/s | 25 kB 00:00 2024-04-26T21:38:06.988 INFO:teuthology.orchestra.run.smithi164.stdout:(1/152): ceph-19.0.0-3306.gbc612ffb.el9.x86_64. 26 kB/s | 6.6 kB 00:00 2024-04-26T21:38:07.223 INFO:teuthology.orchestra.run.smithi076.stdout:(10/152): ceph-common-19.0.0-3306.gbc612ffb.el9 6.4 MB/s | 21 MB 00:03 2024-04-26T21:38:07.283 INFO:teuthology.orchestra.run.smithi164.stdout:(2/152): ceph-fuse-19.0.0-3306.gbc612ffb.el9.x8 3.0 MB/s | 889 kB 00:00 2024-04-26T21:38:07.397 INFO:teuthology.orchestra.run.smithi164.stdout:(3/152): ceph-immutable-object-cache-19.0.0-330 1.3 MB/s | 145 kB 00:00 2024-04-26T21:38:07.406 INFO:teuthology.orchestra.run.smithi076.stdout:(11/152): ceph-radosgw-19.0.0-3306.gbc612ffb.el 6.0 MB/s | 11 MB 00:01 2024-04-26T21:38:07.440 INFO:teuthology.orchestra.run.smithi076.stdout:(12/152): cephfs-mirror-19.0.0-3306.gbc612ffb.e 1.0 MB/s | 225 kB 00:00 2024-04-26T21:38:07.532 INFO:teuthology.orchestra.run.smithi076.stdout:(13/152): cephfs-shell-19.0.0-3306.gbc612ffb.el 217 kB/s | 27 kB 00:00 2024-04-26T21:38:07.555 INFO:teuthology.orchestra.run.smithi164.stdout:(4/152): ceph-base-19.0.0-3306.gbc612ffb.el9.x8 6.5 MB/s | 5.4 MB 00:00 2024-04-26T21:38:07.582 INFO:teuthology.orchestra.run.smithi076.stdout:(14/152): libcephfs-devel-19.0.0-3306.gbc612ffb 224 kB/s | 32 kB 00:00 2024-04-26T21:38:07.681 INFO:teuthology.orchestra.run.smithi164.stdout:(5/152): ceph-mds-19.0.0-3306.gbc612ffb.el9.x86 8.4 MB/s | 2.4 MB 00:00 2024-04-26T21:38:07.707 INFO:teuthology.orchestra.run.smithi076.stdout:(15/152): libcephfs2-19.0.0-3306.gbc612ffb.el9. 4.2 MB/s | 747 kB 00:00 2024-04-26T21:38:07.731 INFO:teuthology.orchestra.run.smithi164.stdout:(6/152): ceph-mgr-19.0.0-3306.gbc612ffb.el9.x86 8.7 MB/s | 1.5 MB 00:00 2024-04-26T21:38:07.741 INFO:teuthology.orchestra.run.smithi076.stdout:(16/152): libcephsqlite-19.0.0-3306.gbc612ffb.e 1.0 MB/s | 163 kB 00:00 2024-04-26T21:38:07.825 INFO:teuthology.orchestra.run.smithi076.stdout:(17/152): librados-devel-19.0.0-3306.gbc612ffb. 1.1 MB/s | 127 kB 00:00 2024-04-26T21:38:08.231 INFO:teuthology.orchestra.run.smithi164.stdout:(7/152): ceph-mon-19.0.0-3306.gbc612ffb.el9.x86 8.6 MB/s | 4.7 MB 00:00 2024-04-26T21:38:08.275 INFO:teuthology.orchestra.run.smithi076.stdout:(18/152): libradosstriper1-19.0.0-3306.gbc612ff 939 kB/s | 501 kB 00:00 2024-04-26T21:38:08.917 INFO:teuthology.orchestra.run.smithi076.stdout:(19/152): python3-ceph-argparse-19.0.0-3306.gbc 70 kB/s | 45 kB 00:00 2024-04-26T21:38:09.025 INFO:teuthology.orchestra.run.smithi076.stdout:(20/152): python3-ceph-common-19.0.0-3306.gbc61 1.2 MB/s | 135 kB 00:00 2024-04-26T21:38:09.143 INFO:teuthology.orchestra.run.smithi076.stdout:(21/152): python3-cephfs-19.0.0-3306.gbc612ffb. 1.4 MB/s | 161 kB 00:00 2024-04-26T21:38:09.403 INFO:teuthology.orchestra.run.smithi076.stdout:(22/152): librgw2-19.0.0-3306.gbc612ffb.el9.x86 3.4 MB/s | 5.4 MB 00:01 2024-04-26T21:38:09.437 INFO:teuthology.orchestra.run.smithi076.stdout:(23/152): python3-rados-19.0.0-3306.gbc612ffb.e 1.1 MB/s | 321 kB 00:00 2024-04-26T21:38:09.546 INFO:teuthology.orchestra.run.smithi076.stdout:(24/152): python3-rbd-19.0.0-3306.gbc612ffb.el9 2.1 MB/s | 298 kB 00:00 2024-04-26T21:38:09.579 INFO:teuthology.orchestra.run.smithi076.stdout:(25/152): python3-rgw-19.0.0-3306.gbc612ffb.el9 703 kB/s | 99 kB 00:00 2024-04-26T21:38:09.646 INFO:teuthology.orchestra.run.smithi076.stdout:(26/152): rbd-fuse-19.0.0-3306.gbc612ffb.el9.x8 858 kB/s | 85 kB 00:00 2024-04-26T21:38:09.730 INFO:teuthology.orchestra.run.smithi076.stdout:(27/152): rbd-nbd-19.0.0-3306.gbc612ffb.el9.x86 2.0 MB/s | 171 kB 00:00 2024-04-26T21:38:09.813 INFO:teuthology.orchestra.run.smithi076.stdout:(28/152): ceph-grafana-dashboards-19.0.0-3306.g 414 kB/s | 34 kB 00:00 2024-04-26T21:38:09.831 INFO:teuthology.orchestra.run.smithi164.stdout:(8/152): ceph-osd-19.0.0-3306.gbc612ffb.el9.x86 8.0 MB/s | 17 MB 00:02 2024-04-26T21:38:09.898 INFO:teuthology.orchestra.run.smithi164.stdout:(9/152): ceph-selinux-19.0.0-3306.gbc612ffb.el9 384 kB/s | 25 kB 00:00 2024-04-26T21:38:09.905 INFO:teuthology.orchestra.run.smithi076.stdout:(29/152): ceph-mgr-cephadm-19.0.0-3306.gbc612ff 1.5 MB/s | 142 kB 00:00 2024-04-26T21:38:10.157 INFO:teuthology.orchestra.run.smithi164.stdout:(10/152): ceph-radosgw-19.0.0-3306.gbc612ffb.el 5.5 MB/s | 11 MB 00:01 2024-04-26T21:38:10.274 INFO:teuthology.orchestra.run.smithi164.stdout:(11/152): cephfs-mirror-19.0.0-3306.gbc612ffb.e 1.9 MB/s | 225 kB 00:00 2024-04-26T21:38:10.389 INFO:teuthology.orchestra.run.smithi164.stdout:(12/152): cephfs-shell-19.0.0-3306.gbc612ffb.el 238 kB/s | 27 kB 00:00 2024-04-26T21:38:10.456 INFO:teuthology.orchestra.run.smithi076.stdout:(30/152): rbd-mirror-19.0.0-3306.gbc612ffb.el9. 3.5 MB/s | 3.1 MB 00:00 2024-04-26T21:38:10.474 INFO:teuthology.orchestra.run.smithi164.stdout:(13/152): libcephfs-devel-19.0.0-3306.gbc612ffb 377 kB/s | 32 kB 00:00 2024-04-26T21:38:10.767 INFO:teuthology.orchestra.run.smithi164.stdout:(14/152): libcephfs2-19.0.0-3306.gbc612ffb.el9. 2.5 MB/s | 747 kB 00:00 2024-04-26T21:38:10.842 INFO:teuthology.orchestra.run.smithi164.stdout:(15/152): libcephsqlite-19.0.0-3306.gbc612ffb.e 2.2 MB/s | 163 kB 00:00 2024-04-26T21:38:10.865 INFO:teuthology.orchestra.run.smithi076.stdout:(31/152): ceph-mgr-dashboard-19.0.0-3306.gbc612 3.7 MB/s | 3.5 MB 00:00 2024-04-26T21:38:10.900 INFO:teuthology.orchestra.run.smithi164.stdout:(16/152): librados-devel-19.0.0-3306.gbc612ffb. 2.2 MB/s | 127 kB 00:00 2024-04-26T21:38:10.973 INFO:teuthology.orchestra.run.smithi076.stdout:(32/152): ceph-mgr-modules-core-19.0.0-3306.gbc 2.3 MB/s | 249 kB 00:00 2024-04-26T21:38:11.034 INFO:teuthology.orchestra.run.smithi164.stdout:(17/152): libradosstriper1-19.0.0-3306.gbc612ff 3.7 MB/s | 501 kB 00:00 2024-04-26T21:38:11.082 INFO:teuthology.orchestra.run.smithi076.stdout:(33/152): ceph-mgr-rook-19.0.0-3306.gbc612ffb.e 458 kB/s | 49 kB 00:00 2024-04-26T21:38:11.207 INFO:teuthology.orchestra.run.smithi076.stdout:(34/152): ceph-prometheus-alerts-19.0.0-3306.gb 132 kB/s | 16 kB 00:00 2024-04-26T21:38:11.316 INFO:teuthology.orchestra.run.smithi076.stdout:(35/152): ceph-volume-19.0.0-3306.gbc612ffb.el9 2.5 MB/s | 273 kB 00:00 2024-04-26T21:38:11.591 INFO:teuthology.orchestra.run.smithi076.stdout:(36/152): cephadm-19.0.0-3306.gbc612ffb.el9.noa 2.7 MB/s | 762 kB 00:00 2024-04-26T21:38:11.717 INFO:teuthology.orchestra.run.smithi076.stdout:(37/152): cephfs-top-19.0.0-3306.gbc612ffb.el9. 147 kB/s | 18 kB 00:00 2024-04-26T21:38:11.875 INFO:teuthology.orchestra.run.smithi076.stdout:(38/152): fuse-2.9.9-15.el9.x86_64.rpm 510 kB/s | 80 kB 00:00 2024-04-26T21:38:12.042 INFO:teuthology.orchestra.run.smithi076.stdout:(39/152): libgfortran-11.4.1-3.el9.x86_64.rpm 4.7 MB/s | 803 kB 00:00 2024-04-26T21:38:12.101 INFO:teuthology.orchestra.run.smithi076.stdout:(40/152): libquadmath-11.4.1-3.el9.x86_64.rpm 3.3 MB/s | 195 kB 00:00 2024-04-26T21:38:12.168 INFO:teuthology.orchestra.run.smithi076.stdout:(41/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.8 MB/s | 253 kB 00:00 2024-04-26T21:38:12.192 INFO:teuthology.orchestra.run.smithi164.stdout:(18/152): librgw2-19.0.0-3306.gbc612ffb.el9.x86 4.6 MB/s | 5.4 MB 00:01 2024-04-26T21:38:12.276 INFO:teuthology.orchestra.run.smithi164.stdout:(19/152): python3-ceph-argparse-19.0.0-3306.gbc 548 kB/s | 45 kB 00:00 2024-04-26T21:38:12.318 INFO:teuthology.orchestra.run.smithi076.stdout:(42/152): python3-cryptography-36.0.1-4.el9.x86 8.2 MB/s | 1.2 MB 00:00 2024-04-26T21:38:12.376 INFO:teuthology.orchestra.run.smithi164.stdout:(20/152): python3-ceph-common-19.0.0-3306.gbc61 1.3 MB/s | 135 kB 00:00 2024-04-26T21:38:12.435 INFO:teuthology.orchestra.run.smithi164.stdout:(21/152): python3-cephfs-19.0.0-3306.gbc612ffb. 2.8 MB/s | 161 kB 00:00 2024-04-26T21:38:12.460 INFO:teuthology.orchestra.run.smithi076.stdout:(43/152): ceph-mgr-diskprediction-local-19.0.0- 3.7 MB/s | 7.4 MB 00:02 2024-04-26T21:38:12.494 INFO:teuthology.orchestra.run.smithi076.stdout:(44/152): python3-ply-3.11-14.el9.noarch.rpm 610 kB/s | 106 kB 00:00 2024-04-26T21:38:12.552 INFO:teuthology.orchestra.run.smithi164.stdout:(22/152): python3-rados-19.0.0-3306.gbc612ffb.e 2.7 MB/s | 321 kB 00:00 2024-04-26T21:38:12.553 INFO:teuthology.orchestra.run.smithi076.stdout:(45/152): python3-requests-2.25.1-8.el9.noarch. 2.1 MB/s | 125 kB 00:00 2024-04-26T21:38:12.620 INFO:teuthology.orchestra.run.smithi076.stdout:(46/152): python3-urllib3-1.26.5-5.el9.noarch.r 3.2 MB/s | 215 kB 00:00 2024-04-26T21:38:12.644 INFO:teuthology.orchestra.run.smithi164.stdout:(23/152): python3-rbd-19.0.0-3306.gbc612ffb.el9 3.2 MB/s | 298 kB 00:00 2024-04-26T21:38:12.654 INFO:teuthology.orchestra.run.smithi076.stdout:(47/152): python3-pycparser-2.20-6.el9.noarch.r 705 kB/s | 135 kB 00:00 2024-04-26T21:38:12.702 INFO:teuthology.orchestra.run.smithi164.stdout:(24/152): python3-rgw-19.0.0-3306.gbc612ffb.el9 1.7 MB/s | 99 kB 00:00 2024-04-26T21:38:12.769 INFO:teuthology.orchestra.run.smithi164.stdout:(25/152): rbd-fuse-19.0.0-3306.gbc612ffb.el9.x8 1.3 MB/s | 85 kB 00:00 2024-04-26T21:38:12.921 INFO:teuthology.orchestra.run.smithi076.stdout:(48/152): automake-1.16.2-8.el9.noarch.rpm 2.2 MB/s | 689 kB 00:00 2024-04-26T21:38:13.253 INFO:teuthology.orchestra.run.smithi164.stdout:(26/152): rbd-mirror-19.0.0-3306.gbc612ffb.el9. 6.4 MB/s | 3.1 MB 00:00 2024-04-26T21:38:13.320 INFO:teuthology.orchestra.run.smithi164.stdout:(27/152): rbd-nbd-19.0.0-3306.gbc612ffb.el9.x86 2.5 MB/s | 171 kB 00:00 2024-04-26T21:38:13.404 INFO:teuthology.orchestra.run.smithi164.stdout:(28/152): ceph-grafana-dashboards-19.0.0-3306.g 414 kB/s | 34 kB 00:00 2024-04-26T21:38:13.446 INFO:teuthology.orchestra.run.smithi076.stdout:(49/152): ceph-test-19.0.0-3306.gbc612ffb.el9.x 7.5 MB/s | 49 MB 00:06 2024-04-26T21:38:13.462 INFO:teuthology.orchestra.run.smithi164.stdout:(29/152): ceph-mgr-cephadm-19.0.0-3306.gbc612ff 2.4 MB/s | 142 kB 00:00 2024-04-26T21:38:13.480 INFO:teuthology.orchestra.run.smithi076.stdout:(50/152): boost-program-options-1.75.0-8.el9.x8 192 kB/s | 107 kB 00:00 2024-04-26T21:38:13.522 INFO:teuthology.orchestra.run.smithi076.stdout:(51/152): bison-3.7.4-5.el9.x86_64.rpm 1.1 MB/s | 943 kB 00:00 2024-04-26T21:38:13.589 INFO:teuthology.orchestra.run.smithi076.stdout:(52/152): flex-2.6.4-9.el9.x86_64.rpm 2.8 MB/s | 313 kB 00:00 2024-04-26T21:38:13.631 INFO:teuthology.orchestra.run.smithi076.stdout:(53/152): flexiblas-3.0.4-8.el9.x86_64.rpm 303 kB/s | 33 kB 00:00 2024-04-26T21:38:13.673 INFO:teuthology.orchestra.run.smithi076.stdout:(54/152): elfutils-libelf-devel-0.190-2.el9.x86 95 kB/s | 21 kB 00:00 2024-04-26T21:38:13.723 INFO:teuthology.orchestra.run.smithi076.stdout:(55/152): flexiblas-openblas-openmp-3.0.4-8.el9 199 kB/s | 18 kB 00:00 2024-04-26T21:38:13.790 INFO:teuthology.orchestra.run.smithi076.stdout:(56/152): libaio-devel-0.3.111-13.el9.x86_64.rp 96 kB/s | 11 kB 00:00 2024-04-26T21:38:13.824 INFO:teuthology.orchestra.run.smithi076.stdout:(57/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.6 MB/s | 160 kB 00:00 2024-04-26T21:38:13.932 INFO:teuthology.orchestra.run.smithi076.stdout:(58/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 8.7 MB/s | 3.0 MB 00:00 2024-04-26T21:38:13.963 INFO:teuthology.orchestra.run.smithi164.stdout:(30/152): ceph-mgr-dashboard-19.0.0-3306.gbc612 7.1 MB/s | 3.5 MB 00:00 2024-04-26T21:38:13.974 INFO:teuthology.orchestra.run.smithi076.stdout:(59/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 247 kB/s | 45 kB 00:00 2024-04-26T21:38:14.033 INFO:teuthology.orchestra.run.smithi076.stdout:(60/152): libtool-2.4.6-45.el9.x86_64.rpm 5.7 MB/s | 582 kB 00:00 2024-04-26T21:38:14.058 INFO:teuthology.orchestra.run.smithi076.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 307 kB/s | 25 kB 00:00 2024-04-26T21:38:14.092 INFO:teuthology.orchestra.run.smithi076.stdout:(62/152): librdkafka-1.6.1-102.el9.x86_64.rpm 2.4 MB/s | 662 kB 00:00 2024-04-26T21:38:14.117 INFO:teuthology.orchestra.run.smithi076.stdout:(63/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 571 kB/s | 47 kB 00:00 2024-04-26T21:38:14.184 INFO:teuthology.orchestra.run.smithi076.stdout:(64/152): lua-5.4.4-4.el9.x86_64.rpm 2.0 MB/s | 188 kB 00:00 2024-04-26T21:38:14.210 INFO:teuthology.orchestra.run.smithi076.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 386 kB/s | 35 kB 00:00 2024-04-26T21:38:14.252 INFO:teuthology.orchestra.run.smithi076.stdout:(66/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 1.5 MB/s | 292 kB 00:00 2024-04-26T21:38:14.410 INFO:teuthology.orchestra.run.smithi076.stdout:(67/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 22 MB/s | 4.4 MB 00:00 2024-04-26T21:38:14.511 INFO:teuthology.orchestra.run.smithi076.stdout:(68/152): openblas-openmp-0.3.21-2.el9.x86_64.r 15 MB/s | 4.7 MB 00:00 2024-04-26T21:38:14.534 INFO:teuthology.orchestra.run.smithi072.stdout:CentOS Stream 9 - CRB 866 kB/s | 6.2 MB 00:07 2024-04-26T21:38:14.547 INFO:teuthology.orchestra.run.smithi164.stdout:(31/152): ceph-test-19.0.0-3306.gbc612ffb.el9.x 11 MB/s | 49 MB 00:04 2024-04-26T21:38:14.612 INFO:teuthology.orchestra.run.smithi076.stdout:(69/152): postgresql-13.14-1.el9.x86_64.rpm 4.4 MB/s | 1.6 MB 00:00 2024-04-26T21:38:14.630 INFO:teuthology.orchestra.run.smithi164.stdout:(32/152): ceph-mgr-modules-core-19.0.0-3306.gbc 3.0 MB/s | 249 kB 00:00 2024-04-26T21:38:14.645 INFO:teuthology.orchestra.run.smithi076.stdout:(70/152): postgresql-private-libs-13.14-1.el9.x 1.0 MB/s | 133 kB 00:00 2024-04-26T21:38:14.679 INFO:teuthology.orchestra.run.smithi076.stdout:(71/152): postgresql-contrib-13.14-1.el9.x86_64 3.3 MB/s | 911 kB 00:00 2024-04-26T21:38:14.743 INFO:teuthology.orchestra.run.smithi164.stdout:(33/152): ceph-mgr-rook-19.0.0-3306.gbc612ffb.e 440 kB/s | 49 kB 00:00 2024-04-26T21:38:14.770 INFO:teuthology.orchestra.run.smithi076.stdout:(72/152): protobuf-3.14.0-13.el9.x86_64.rpm 8.1 MB/s | 1.0 MB 00:00 2024-04-26T21:38:14.797 INFO:teuthology.orchestra.run.smithi164.stdout:(34/152): ceph-prometheus-alerts-19.0.0-3306.gb 313 kB/s | 16 kB 00:00 2024-04-26T21:38:14.812 INFO:teuthology.orchestra.run.smithi076.stdout:(73/152): python3-attrs-20.3.0-7.el9.noarch.rpm 659 kB/s | 87 kB 00:00 2024-04-26T21:38:14.864 INFO:teuthology.orchestra.run.smithi164.stdout:(35/152): ceph-volume-19.0.0-3306.gbc612ffb.el9 4.1 MB/s | 273 kB 00:00 2024-04-26T21:38:14.888 INFO:teuthology.orchestra.run.smithi076.stdout:(74/152): python3-devel-3.9.18-3.el9.x86_64.rpm 3.2 MB/s | 244 kB 00:00 2024-04-26T21:38:14.973 INFO:teuthology.orchestra.run.smithi164.stdout:(36/152): cephadm-19.0.0-3306.gbc612ffb.el9.noa 6.9 MB/s | 762 kB 00:00 2024-04-26T21:38:14.985 INFO:teuthology.orchestra.run.smithi076.stdout:(75/152): python3-jinja2-2.11.3-5.el9.noarch.rp 2.5 MB/s | 248 kB 00:00 2024-04-26T21:38:15.097 INFO:teuthology.orchestra.run.smithi076.stdout:(76/152): python3-babel-2.9.1-2.el9.noarch.rpm 18 MB/s | 6.0 MB 00:00 2024-04-26T21:38:15.098 INFO:teuthology.orchestra.run.smithi164.stdout:(37/152): ceph-mgr-diskprediction-local-19.0.0- 6.5 MB/s | 7.4 MB 00:01 2024-04-26T21:38:15.122 INFO:teuthology.orchestra.run.smithi076.stdout:(77/152): python3-jsonpatch-1.21-16.el9.noarch. 188 kB/s | 26 kB 00:00 2024-04-26T21:38:15.123 INFO:teuthology.orchestra.run.smithi164.stdout:(38/152): cephfs-top-19.0.0-3306.gbc612ffb.el9. 122 kB/s | 18 kB 00:00 2024-04-26T21:38:15.164 INFO:teuthology.orchestra.run.smithi076.stdout:(78/152): python3-jsonpointer-2.0-4.el9.noarch. 290 kB/s | 19 kB 00:00 2024-04-26T21:38:15.189 INFO:teuthology.orchestra.run.smithi076.stdout:(79/152): python3-mako-1.1.4-6.el9.noarch.rpm 2.5 MB/s | 172 kB 00:00 2024-04-26T21:38:15.298 INFO:teuthology.orchestra.run.smithi076.stdout:(80/152): postgresql-server-13.14-1.el9.x86_64. 8.4 MB/s | 5.8 MB 00:00 2024-04-26T21:38:15.323 INFO:teuthology.orchestra.run.smithi076.stdout:(81/152): python3-markupsafe-1.1.1-12.el9.x86_6 220 kB/s | 35 kB 00:00 2024-04-26T21:38:15.399 INFO:teuthology.orchestra.run.smithi164.stdout:(39/152): ceph-common-19.0.0-3306.gbc612ffb.el9 2.4 MB/s | 21 MB 00:08 2024-04-26T21:38:15.407 INFO:teuthology.orchestra.run.smithi076.stdout:(82/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.3 MB/s | 250 kB 00:00 2024-04-26T21:38:15.441 INFO:teuthology.orchestra.run.smithi076.stdout:(83/152): python3-oauthlib-3.1.1-5.el9.noarch.r 1.9 MB/s | 222 kB 00:00 2024-04-26T21:38:15.475 INFO:teuthology.orchestra.run.smithi076.stdout:(84/152): python3-packaging-20.9-5.el9.noarch.r 1.1 MB/s | 77 kB 00:00 2024-04-26T21:38:15.509 INFO:teuthology.orchestra.run.smithi076.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 637 kB/s | 42 kB 00:00 2024-04-26T21:38:15.559 INFO:teuthology.orchestra.run.smithi076.stdout:(86/152): python3-protobuf-3.14.0-13.el9.noarch 3.2 MB/s | 269 kB 00:00 2024-04-26T21:38:15.593 INFO:teuthology.orchestra.run.smithi076.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.9 MB/s | 159 kB 00:00 2024-04-26T21:38:15.701 INFO:teuthology.orchestra.run.smithi076.stdout:(88/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 10 MB/s | 5.1 MB 00:00 2024-04-26T21:38:15.716 INFO:teuthology.orchestra.run.smithi164.stdout:(40/152): fuse-2.9.9-15.el9.x86_64.rpm 130 kB/s | 80 kB 00:00 2024-04-26T21:38:15.726 INFO:teuthology.orchestra.run.smithi076.stdout:(89/152): python3-pytz-2021.1-5.el9.noarch.rpm 383 kB/s | 51 kB 00:00 2024-04-26T21:38:15.760 INFO:teuthology.orchestra.run.smithi076.stdout:(90/152): python3-pyasn1-modules-0.4.8-6.el9.no 1.4 MB/s | 279 kB 00:00 2024-04-26T21:38:15.774 INFO:teuthology.orchestra.run.smithi164.stdout:(41/152): libgfortran-11.4.1-3.el9.x86_64.rpm 1.2 MB/s | 803 kB 00:00 2024-04-26T21:38:15.807 INFO:teuthology.orchestra.run.smithi164.stdout:(42/152): libquadmath-11.4.1-3.el9.x86_64.rpm 479 kB/s | 195 kB 00:00 2024-04-26T21:38:15.829 INFO:teuthology.orchestra.run.smithi076.stdout:(91/152): python3-requests-oauthlib-1.3.0-12.el 422 kB/s | 54 kB 00:00 2024-04-26T21:38:15.861 INFO:teuthology.orchestra.run.smithi076.stdout:(92/152): python3-toml-0.10.2-6.el9.noarch.rpm 418 kB/s | 42 kB 00:00 2024-04-26T21:38:15.866 INFO:teuthology.orchestra.run.smithi164.stdout:(43/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.7 MB/s | 253 kB 00:00 2024-04-26T21:38:15.895 INFO:teuthology.orchestra.run.smithi076.stdout:(93/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 688 kB/s | 44 kB 00:00 2024-04-26T21:38:15.900 INFO:teuthology.orchestra.run.smithi164.stdout:(44/152): python3-ply-3.11-14.el9.noarch.rpm 1.1 MB/s | 106 kB 00:00 2024-04-26T21:38:15.928 INFO:teuthology.orchestra.run.smithi076.stdout:(94/152): qatlib-24.02.0-1.el9.x86_64.rpm 3.2 MB/s | 221 kB 00:00 2024-04-26T21:38:15.958 INFO:teuthology.orchestra.run.smithi164.stdout:(45/152): python3-pycparser-2.20-6.el9.noarch.r 1.5 MB/s | 135 kB 00:00 2024-04-26T21:38:15.962 INFO:teuthology.orchestra.run.smithi076.stdout:(95/152): qatlib-service-24.02.0-1.el9.x86_64.r 541 kB/s | 36 kB 00:00 2024-04-26T21:38:15.996 INFO:teuthology.orchestra.run.smithi076.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 715 kB/s | 47 kB 00:00 2024-04-26T21:38:16.008 INFO:teuthology.orchestra.run.smithi164.stdout:(46/152): python3-cryptography-36.0.1-4.el9.x86 5.3 MB/s | 1.2 MB 00:00 2024-04-26T21:38:16.034 INFO:teuthology.orchestra.run.smithi164.stdout:(47/152): python3-requests-2.25.1-8.el9.noarch. 939 kB/s | 125 kB 00:00 2024-04-26T21:38:16.038 INFO:teuthology.orchestra.run.smithi076.stdout:(97/152): socat-1.7.4.1-5.el9.x86_64.rpm 4.0 MB/s | 305 kB 00:00 2024-04-26T21:38:16.067 INFO:teuthology.orchestra.run.smithi164.stdout:(48/152): python3-urllib3-1.26.5-5.el9.noarch.r 1.9 MB/s | 215 kB 00:00 2024-04-26T21:38:16.080 INFO:teuthology.orchestra.run.smithi076.stdout:(98/152): uuid-1.6.2-55.el9.x86_64.rpm 697 kB/s | 58 kB 00:00 2024-04-26T21:38:16.122 INFO:teuthology.orchestra.run.smithi076.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.1 MB/s | 94 kB 00:00 2024-04-26T21:38:16.147 INFO:teuthology.orchestra.run.smithi076.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 960 kB/s | 64 kB 00:00 2024-04-26T21:38:16.167 INFO:teuthology.orchestra.run.smithi164.stdout:(49/152): automake-1.16.2-8.el9.noarch.rpm 4.3 MB/s | 689 kB 00:00 2024-04-26T21:38:16.189 INFO:teuthology.orchestra.run.smithi076.stdout:(101/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 677 kB/s | 45 kB 00:00 2024-04-26T21:38:16.193 INFO:teuthology.orchestra.run.smithi164.stdout:(50/152): boost-program-options-1.75.0-8.el9.x8 860 kB/s | 107 kB 00:00 2024-04-26T21:38:16.264 INFO:teuthology.orchestra.run.smithi076.stdout:(102/152): lua-devel-5.4.4-4.el9.x86_64.rpm 192 kB/s | 22 kB 00:00 2024-04-26T21:38:16.335 INFO:teuthology.orchestra.run.smithi164.stdout:(51/152): bison-3.7.4-5.el9.x86_64.rpm 3.1 MB/s | 943 kB 00:00 2024-04-26T21:38:16.360 INFO:teuthology.orchestra.run.smithi164.stdout:(52/152): elfutils-libelf-devel-0.190-2.el9.x86 112 kB/s | 21 kB 00:00 2024-04-26T21:38:16.403 INFO:teuthology.orchestra.run.smithi072.stdout:CentOS Stream 9 - Extras packages 31 kB/s | 16 kB 00:00 2024-04-26T21:38:16.427 INFO:teuthology.orchestra.run.smithi164.stdout:(53/152): flex-2.6.4-9.el9.x86_64.rpm 1.3 MB/s | 313 kB 00:00 2024-04-26T21:38:16.444 INFO:teuthology.orchestra.run.smithi164.stdout:(54/152): flexiblas-3.0.4-8.el9.x86_64.rpm 303 kB/s | 33 kB 00:00 2024-04-26T21:38:16.486 INFO:teuthology.orchestra.run.smithi164.stdout:(55/152): libaio-devel-0.3.111-13.el9.x86_64.rp 274 kB/s | 11 kB 00:00 2024-04-26T21:38:16.511 INFO:teuthology.orchestra.run.smithi164.stdout:(56/152): flexiblas-openblas-openmp-3.0.4-8.el9 220 kB/s | 18 kB 00:00 2024-04-26T21:38:16.570 INFO:teuthology.orchestra.run.smithi164.stdout:(57/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 785 kB/s | 45 kB 00:00 2024-04-26T21:38:16.606 INFO:teuthology.orchestra.run.smithi076.stdout:(103/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 18 MB/s | 16 MB 00:00 2024-04-26T21:38:16.673 INFO:teuthology.orchestra.run.smithi076.stdout:(104/152): protobuf-compiler-3.14.0-13.el9.x86_ 1.7 MB/s | 863 kB 00:00 2024-04-26T21:38:16.679 INFO:teuthology.orchestra.run.smithi164.stdout:(58/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 9.4 MB/s | 3.0 MB 00:00 2024-04-26T21:38:16.712 INFO:teuthology.orchestra.run.smithi164.stdout:(59/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 711 kB/s | 160 kB 00:00 2024-04-26T21:38:16.762 INFO:teuthology.orchestra.run.smithi164.stdout:(60/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 520 kB/s | 25 kB 00:00 2024-04-26T21:38:16.790 INFO:teuthology.orchestra.run.smithi076.stdout:(105/152): abseil-cpp-20211102.0-3.el9.x86_64.r 1.0 MB/s | 548 kB 00:00 2024-04-26T21:38:16.804 INFO:teuthology.orchestra.run.smithi164.stdout:(61/152): librdkafka-1.6.1-102.el9.x86_64.rpm 2.8 MB/s | 662 kB 00:00 2024-04-26T21:38:16.824 INFO:teuthology.orchestra.run.smithi076.stdout:(106/152): gperftools-libs-2.9.1-2.el9.x86_64.r 1.4 MB/s | 309 kB 00:00 2024-04-26T21:38:16.838 INFO:teuthology.orchestra.run.smithi164.stdout:(62/152): libtool-2.4.6-45.el9.x86_64.rpm 3.6 MB/s | 582 kB 00:00 2024-04-26T21:38:16.849 INFO:teuthology.orchestra.run.smithi076.stdout:(107/152): grpc-data-1.46.7-10.el9.noarch.rpm 112 kB/s | 19 kB 00:00 2024-04-26T21:38:16.863 INFO:teuthology.orchestra.run.smithi164.stdout:(63/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 476 kB/s | 47 kB 00:00 2024-04-26T21:38:16.922 INFO:teuthology.orchestra.run.smithi164.stdout:(64/152): lua-5.4.4-4.el9.x86_64.rpm 2.2 MB/s | 188 kB 00:00 2024-04-26T21:38:16.947 INFO:teuthology.orchestra.run.smithi164.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 426 kB/s | 35 kB 00:00 2024-04-26T21:38:16.980 INFO:teuthology.orchestra.run.smithi164.stdout:(66/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 1.6 MB/s | 292 kB 00:00 2024-04-26T21:38:17.131 INFO:teuthology.orchestra.run.smithi164.stdout:(67/152): postgresql-13.14-1.el9.x86_64.rpm 11 MB/s | 1.6 MB 00:00 2024-04-26T21:38:17.248 INFO:teuthology.orchestra.run.smithi164.stdout:(68/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 15 MB/s | 4.4 MB 00:00 2024-04-26T21:38:17.340 INFO:teuthology.orchestra.run.smithi164.stdout:(69/152): openblas-openmp-0.3.21-2.el9.x86_64.r 11 MB/s | 4.7 MB 00:00 2024-04-26T21:38:17.349 INFO:teuthology.orchestra.run.smithi076.stdout:(108/152): libarrow-doc-9.0.0-11.el9.noarch.rpm 52 kB/s | 26 kB 00:00 2024-04-26T21:38:17.375 INFO:teuthology.orchestra.run.smithi076.stdout:(109/152): liboath-2.6.7-2.el9.x86_64.rpm 93 kB/s | 49 kB 00:00 2024-04-26T21:38:17.407 INFO:teuthology.orchestra.run.smithi164.stdout:(70/152): postgresql-contrib-13.14-1.el9.x86_64 3.2 MB/s | 911 kB 00:00 2024-04-26T21:38:17.440 INFO:teuthology.orchestra.run.smithi164.stdout:(71/152): postgresql-private-libs-13.14-1.el9.x 696 kB/s | 133 kB 00:00 2024-04-26T21:38:17.508 INFO:teuthology.orchestra.run.smithi076.stdout:(110/152): libarrow-9.0.0-11.el9.x86_64.rpm 6.2 MB/s | 4.4 MB 00:00 2024-04-26T21:38:17.533 INFO:teuthology.orchestra.run.smithi076.stdout:(111/152): libunwind-1.6.2-1.el9.x86_64.rpm 461 kB/s | 67 kB 00:00 2024-04-26T21:38:17.549 INFO:teuthology.orchestra.run.smithi164.stdout:(72/152): python3-attrs-20.3.0-7.el9.noarch.rpm 813 kB/s | 87 kB 00:00 2024-04-26T21:38:17.559 INFO:teuthology.orchestra.run.smithi076.stdout:(112/152): luarocks-3.9.2-1.el9.noarch.rpm 884 kB/s | 151 kB 00:00 2024-04-26T21:38:17.632 INFO:teuthology.orchestra.run.smithi164.stdout:(73/152): protobuf-3.14.0-13.el9.x86_64.rpm 4.5 MB/s | 1.0 MB 00:00 2024-04-26T21:38:17.741 INFO:teuthology.orchestra.run.smithi164.stdout:(74/152): python3-devel-3.9.18-3.el9.x86_64.rpm 2.2 MB/s | 244 kB 00:00 2024-04-26T21:38:17.784 INFO:teuthology.orchestra.run.smithi076.stdout:(113/152): parquet-libs-9.0.0-11.el9.x86_64.rpm 3.0 MB/s | 839 kB 00:00 2024-04-26T21:38:17.798 INFO:teuthology.orchestra.run.smithi072.stdout:Extra Packages for Enterprise Linux 17 MB/s | 21 MB 00:01 2024-04-26T21:38:17.843 INFO:teuthology.orchestra.run.smithi076.stdout:(114/152): python3-asyncssh-2.13.2-5.el9.noarch 1.9 MB/s | 548 kB 00:00 2024-04-26T21:38:17.866 INFO:teuthology.orchestra.run.smithi164.stdout:(75/152): python3-jinja2-2.11.3-5.el9.noarch.rp 2.0 MB/s | 248 kB 00:00 2024-04-26T21:38:17.893 INFO:teuthology.orchestra.run.smithi076.stdout:(115/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 130 kB/s | 43 kB 00:00 2024-04-26T21:38:17.918 INFO:teuthology.orchestra.run.smithi076.stdout:(116/152): python3-cachetools-4.2.4-1.el9.noarc 241 kB/s | 32 kB 00:00 2024-04-26T21:38:17.944 INFO:teuthology.orchestra.run.smithi076.stdout:(117/152): python3-certifi-2021.10.8-2.el9.noar 151 kB/s | 15 kB 00:00 2024-04-26T21:38:17.975 INFO:teuthology.orchestra.run.smithi164.stdout:(76/152): postgresql-server-13.14-1.el9.x86_64. 9.1 MB/s | 5.8 MB 00:00 2024-04-26T21:38:17.986 INFO:teuthology.orchestra.run.smithi076.stdout:(118/152): python3-cheroot-8.6.0-4.el9.noarch.r 1.8 MB/s | 172 kB 00:00 2024-04-26T21:38:18.019 INFO:teuthology.orchestra.run.smithi076.stdout:(119/152): python3-cherrypy-18.6.1-2.el9.noarch 4.7 MB/s | 358 kB 00:00 2024-04-26T21:38:18.053 INFO:teuthology.orchestra.run.smithi076.stdout:(120/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.7 MB/s | 303 kB 00:00 2024-04-26T21:38:18.078 INFO:teuthology.orchestra.run.smithi076.stdout:(121/152): python3-colorama-0.4.4-12.el9.noarch 399 kB/s | 35 kB 00:00 2024-04-26T21:38:18.112 INFO:teuthology.orchestra.run.smithi076.stdout:(122/152): python3-google-auth-2.29.0-1.el9.noa 2.3 MB/s | 206 kB 00:00 2024-04-26T21:38:18.117 INFO:teuthology.orchestra.run.smithi164.stdout:(77/152): python3-babel-2.9.1-2.el9.noarch.rpm 11 MB/s | 6.0 MB 00:00 2024-04-26T21:38:18.142 INFO:teuthology.orchestra.run.smithi164.stdout:(78/152): python3-jsonpatch-1.21-16.el9.noarch. 93 kB/s | 26 kB 00:00 2024-04-26T21:38:18.167 INFO:teuthology.orchestra.run.smithi164.stdout:(79/152): python3-jsonpointer-2.0-4.el9.noarch. 100 kB/s | 19 kB 00:00 2024-04-26T21:38:18.217 INFO:teuthology.orchestra.run.smithi164.stdout:(80/152): python3-markupsafe-1.1.1-12.el9.x86_6 712 kB/s | 35 kB 00:00 2024-04-26T21:38:18.251 INFO:teuthology.orchestra.run.smithi164.stdout:(81/152): python3-mako-1.1.4-6.el9.noarch.rpm 1.3 MB/s | 172 kB 00:00 2024-04-26T21:38:18.351 INFO:teuthology.orchestra.run.smithi164.stdout:(82/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.8 MB/s | 250 kB 00:00 2024-04-26T21:38:18.396 INFO:teuthology.orchestra.run.smithi076.stdout:(123/152): python3-grpcio-1.46.7-10.el9.x86_64. 6.0 MB/s | 2.0 MB 00:00 2024-04-26T21:38:18.410 INFO:teuthology.orchestra.run.smithi164.stdout:(83/152): python3-oauthlib-3.1.1-5.el9.noarch.r 1.4 MB/s | 222 kB 00:00 2024-04-26T21:38:18.421 INFO:teuthology.orchestra.run.smithi076.stdout:(124/152): python3-grpcio-tools-1.46.7-10.el9.x 425 kB/s | 144 kB 00:00 2024-04-26T21:38:18.446 INFO:teuthology.orchestra.run.smithi076.stdout:(125/152): python3-jaraco-8.2.1-3.el9.noarch.rp 32 kB/s | 11 kB 00:00 2024-04-26T21:38:18.468 INFO:teuthology.orchestra.run.smithi164.stdout:(84/152): python3-packaging-20.9-5.el9.noarch.r 667 kB/s | 77 kB 00:00 2024-04-26T21:38:18.471 INFO:teuthology.orchestra.run.smithi076.stdout:(126/152): python3-jaraco-classes-3.2.1-5.el9.n 241 kB/s | 18 kB 00:00 2024-04-26T21:38:18.493 INFO:teuthology.orchestra.run.smithi164.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 514 kB/s | 42 kB 00:00 2024-04-26T21:38:18.514 INFO:teuthology.orchestra.run.smithi076.stdout:(127/152): python3-jaraco-collections-3.0.0-8.e 352 kB/s | 23 kB 00:00 2024-04-26T21:38:18.539 INFO:teuthology.orchestra.run.smithi076.stdout:(128/152): python3-jaraco-functools-3.5.0-2.el9 214 kB/s | 19 kB 00:00 2024-04-26T21:38:18.564 INFO:teuthology.orchestra.run.smithi076.stdout:(129/152): python3-jaraco-text-3.2.0-6.el9.noar 214 kB/s | 20 kB 00:00 2024-04-26T21:38:18.602 INFO:teuthology.orchestra.run.smithi164.stdout:(86/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 12 MB/s | 5.1 MB 00:00 2024-04-26T21:38:18.644 INFO:teuthology.orchestra.run.smithi164.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.0 MB/s | 159 kB 00:00 2024-04-26T21:38:18.678 INFO:teuthology.orchestra.run.smithi164.stdout:(88/152): python3-protobuf-3.14.0-13.el9.noarch 1.3 MB/s | 269 kB 00:00 2024-04-26T21:38:18.762 INFO:teuthology.orchestra.run.smithi164.stdout:(89/152): python3-pyasn1-modules-0.4.8-6.el9.no 1.7 MB/s | 279 kB 00:00 2024-04-26T21:38:18.787 INFO:teuthology.orchestra.run.smithi164.stdout:(90/152): python3-pytz-2021.1-5.el9.noarch.rpm 360 kB/s | 51 kB 00:00 2024-04-26T21:38:18.789 INFO:teuthology.orchestra.run.smithi076.stdout:(130/152): python3-kubernetes-26.1.0-2.el9.noar 3.7 MB/s | 1.0 MB 00:00 2024-04-26T21:38:18.812 INFO:teuthology.orchestra.run.smithi164.stdout:(91/152): python3-requests-oauthlib-1.3.0-12.el 403 kB/s | 54 kB 00:00 2024-04-26T21:38:18.815 INFO:teuthology.orchestra.run.smithi076.stdout:(131/152): python3-logutils-0.3.5-21.el9.noarch 168 kB/s | 46 kB 00:00 2024-04-26T21:38:18.849 INFO:teuthology.orchestra.run.smithi076.stdout:(132/152): python3-more-itertools-8.12.0-2.el9. 278 kB/s | 79 kB 00:00 2024-04-26T21:38:18.874 INFO:teuthology.orchestra.run.smithi076.stdout:(133/152): python3-natsort-7.1.1-5.el9.noarch.r 696 kB/s | 58 kB 00:00 2024-04-26T21:38:18.879 INFO:teuthology.orchestra.run.smithi164.stdout:(92/152): python3-toml-0.10.2-6.el9.noarch.rpm 458 kB/s | 42 kB 00:00 2024-04-26T21:38:18.913 INFO:teuthology.orchestra.run.smithi164.stdout:(93/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 444 kB/s | 44 kB 00:00 2024-04-26T21:38:18.916 INFO:teuthology.orchestra.run.smithi076.stdout:(134/152): python3-pecan-1.4.2-3.el9.noarch.rpm 2.7 MB/s | 272 kB 00:00 2024-04-26T21:38:18.941 INFO:teuthology.orchestra.run.smithi076.stdout:(135/152): python3-portend-3.1.0-2.el9.noarch.r 179 kB/s | 16 kB 00:00 2024-04-26T21:38:18.975 INFO:teuthology.orchestra.run.smithi076.stdout:(136/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 904 kB/s | 90 kB 00:00 2024-04-26T21:38:19.000 INFO:teuthology.orchestra.run.smithi076.stdout:(137/152): python3-pyperclip-1.8.2-2.el9.noarch 344 kB/s | 28 kB 00:00 2024-04-26T21:38:19.005 INFO:teuthology.orchestra.run.smithi164.stdout:(94/152): qatlib-24.02.0-1.el9.x86_64.rpm 1.7 MB/s | 221 kB 00:00 2024-04-26T21:38:19.025 INFO:teuthology.orchestra.run.smithi076.stdout:(138/152): python3-repoze-lru-0.7-16.el9.noarch 629 kB/s | 31 kB 00:00 2024-04-26T21:38:19.030 INFO:teuthology.orchestra.run.smithi164.stdout:(95/152): qatlib-service-24.02.0-1.el9.x86_64.r 309 kB/s | 36 kB 00:00 2024-04-26T21:38:19.059 INFO:teuthology.orchestra.run.smithi076.stdout:(139/152): python3-routes-2.5.1-5.el9.noarch.rp 2.2 MB/s | 188 kB 00:00 2024-04-26T21:38:19.118 INFO:teuthology.orchestra.run.smithi076.stdout:(140/152): python3-rsa-4.9-2.el9.noarch.rpm 508 kB/s | 59 kB 00:00 2024-04-26T21:38:19.130 INFO:teuthology.orchestra.run.smithi164.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 382 kB/s | 47 kB 00:00 2024-04-26T21:38:19.143 INFO:teuthology.orchestra.run.smithi076.stdout:(141/152): python3-tempora-5.0.0-2.el9.noarch.r 312 kB/s | 36 kB 00:00 2024-04-26T21:38:19.168 INFO:teuthology.orchestra.run.smithi076.stdout:(142/152): python3-typing-extensions-4.4.0-2.el 476 kB/s | 51 kB 00:00 2024-04-26T21:38:19.189 INFO:teuthology.orchestra.run.smithi164.stdout:(97/152): socat-1.7.4.1-5.el9.x86_64.rpm 1.9 MB/s | 305 kB 00:00 2024-04-26T21:38:19.227 INFO:teuthology.orchestra.run.smithi076.stdout:(143/152): python3-webob-1.8.7-6.el9.noarch.rpm 2.7 MB/s | 230 kB 00:00 2024-04-26T21:38:19.239 INFO:teuthology.orchestra.run.smithi164.stdout:(98/152): uuid-1.6.2-55.el9.x86_64.rpm 536 kB/s | 58 kB 00:00 2024-04-26T21:38:19.260 INFO:teuthology.orchestra.run.smithi076.stdout:(144/152): python3-websocket-client-1.2.3-2.el9 777 kB/s | 90 kB 00:00 2024-04-26T21:38:19.294 INFO:teuthology.orchestra.run.smithi076.stdout:(145/152): python3-werkzeug-2.0.3-3.el9.1.noarc 3.3 MB/s | 427 kB 00:00 2024-04-26T21:38:19.297 INFO:teuthology.orchestra.run.smithi164.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 870 kB/s | 94 kB 00:00 2024-04-26T21:38:19.319 INFO:teuthology.orchestra.run.smithi076.stdout:(146/152): python3-zc-lockfile-2.0-10.el9.noarc 231 kB/s | 20 kB 00:00 2024-04-26T21:38:19.323 INFO:teuthology.orchestra.run.smithi164.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 770 kB/s | 64 kB 00:00 2024-04-26T21:38:19.373 INFO:teuthology.orchestra.run.smithi164.stdout:(101/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 602 kB/s | 45 kB 00:00 2024-04-26T21:38:19.428 INFO:teuthology.orchestra.run.smithi076.stdout:(147/152): re2-20211101-3.el9.x86_64.rpm 1.4 MB/s | 192 kB 00:00 2024-04-26T21:38:19.624 INFO:teuthology.orchestra.run.smithi164.stdout:(102/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 19 MB/s | 16 MB 00:00 2024-04-26T21:38:19.704 INFO:teuthology.orchestra.run.smithi076.stdout:(148/152): thrift-0.15.0-2.el9.x86_64.rpm 3.9 MB/s | 1.6 MB 00:00 2024-04-26T21:38:20.004 INFO:teuthology.orchestra.run.smithi076.stdout:(149/152): openssl-3.2.1-1.el9.x86_64.rpm 4.5 MB/s | 1.3 MB 00:00 2024-04-26T21:38:20.057 INFO:teuthology.orchestra.run.smithi164.stdout:(103/152): abseil-cpp-20211102.0-3.el9.x86_64.r 1.2 MB/s | 548 kB 00:00 2024-04-26T21:38:20.082 INFO:teuthology.orchestra.run.smithi164.stdout:(104/152): lua-devel-5.4.4-4.el9.x86_64.rpm 29 kB/s | 22 kB 00:00 2024-04-26T21:38:20.164 INFO:teuthology.orchestra.run.smithi164.stdout:(105/152): gperftools-libs-2.9.1-2.el9.x86_64.r 2.9 MB/s | 309 kB 00:00 2024-04-26T21:38:20.283 INFO:teuthology.orchestra.run.smithi164.stdout:(106/152): grpc-data-1.46.7-10.el9.noarch.rpm 97 kB/s | 19 kB 00:00 2024-04-26T21:38:20.288 INFO:teuthology.orchestra.run.smithi076.stdout:(150/152): librbd1-19.0.0-3306.gbc612ffb.el9.x8 3.7 MB/s | 3.1 MB 00:00 2024-04-26T21:38:20.350 INFO:teuthology.orchestra.run.smithi164.stdout:(107/152): libarrow-doc-9.0.0-11.el9.noarch.rpm 395 kB/s | 26 kB 00:00 2024-04-26T21:38:20.389 INFO:teuthology.orchestra.run.smithi076.stdout:(151/152): librados2-19.0.0-3306.gbc612ffb.el9. 3.2 MB/s | 3.4 MB 00:01 2024-04-26T21:38:20.451 INFO:teuthology.orchestra.run.smithi164.stdout:(108/152): libarrow-9.0.0-11.el9.x86_64.rpm 15 MB/s | 4.4 MB 00:00 2024-04-26T21:38:20.463 INFO:teuthology.orchestra.run.smithi076.stdout:(152/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 5.2 MB/s | 2.4 MB 00:00 2024-04-26T21:38:20.471 INFO:teuthology.orchestra.run.smithi076.stdout:-------------------------------------------------------------------------------- 2024-04-26T21:38:20.472 INFO:teuthology.orchestra.run.smithi076.stdout:Total 13 MB/s | 222 MB 00:17 2024-04-26T21:38:20.476 INFO:teuthology.orchestra.run.smithi164.stdout:(109/152): liboath-2.6.7-2.el9.x86_64.rpm 390 kB/s | 49 kB 00:00 2024-04-26T21:38:20.526 INFO:teuthology.orchestra.run.smithi164.stdout:(110/152): libunwind-1.6.2-1.el9.x86_64.rpm 908 kB/s | 67 kB 00:00 2024-04-26T21:38:20.584 INFO:teuthology.orchestra.run.smithi164.stdout:(111/152): luarocks-3.9.2-1.el9.noarch.rpm 1.4 MB/s | 151 kB 00:00 2024-04-26T21:38:20.643 INFO:teuthology.orchestra.run.smithi164.stdout:(112/152): parquet-libs-9.0.0-11.el9.x86_64.rpm 7.0 MB/s | 839 kB 00:00 2024-04-26T21:38:20.710 INFO:teuthology.orchestra.run.smithi164.stdout:(113/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 659 kB/s | 43 kB 00:00 2024-04-26T21:38:20.777 INFO:teuthology.orchestra.run.smithi164.stdout:(114/152): python3-cachetools-4.2.4-1.el9.noarc 488 kB/s | 32 kB 00:00 2024-04-26T21:38:20.810 INFO:teuthology.orchestra.run.smithi164.stdout:(115/152): python3-asyncssh-2.13.2-5.el9.noarch 2.4 MB/s | 548 kB 00:00 2024-04-26T21:38:20.863 INFO:teuthology.orchestra.run.smithi164.stdout:(116/152): python3-certifi-2021.10.8-2.el9.noar 176 kB/s | 15 kB 00:00 2024-04-26T21:38:20.903 INFO:teuthology.orchestra.run.smithi164.stdout:(117/152): python3-cheroot-8.6.0-4.el9.noarch.r 1.8 MB/s | 172 kB 00:00 2024-04-26T21:38:20.938 INFO:teuthology.orchestra.run.smithi164.stdout:(118/152): python3-cherrypy-18.6.1-2.el9.noarch 4.7 MB/s | 358 kB 00:00 2024-04-26T21:38:20.997 INFO:teuthology.orchestra.run.smithi164.stdout:(119/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.2 MB/s | 303 kB 00:00 2024-04-26T21:38:21.014 INFO:teuthology.orchestra.run.smithi164.stdout:(120/152): python3-colorama-0.4.4-12.el9.noarch 473 kB/s | 35 kB 00:00 2024-04-26T21:38:21.073 INFO:teuthology.orchestra.run.smithi164.stdout:(121/152): python3-google-auth-2.29.0-1.el9.noa 2.7 MB/s | 206 kB 00:00 2024-04-26T21:38:21.140 INFO:teuthology.orchestra.run.smithi164.stdout:(122/152): protobuf-compiler-3.14.0-13.el9.x86_ 489 kB/s | 863 kB 00:01 2024-04-26T21:38:21.173 INFO:teuthology.orchestra.run.smithi164.stdout:(123/152): python3-grpcio-tools-1.46.7-10.el9.x 1.5 MB/s | 144 kB 00:00 2024-04-26T21:38:21.257 INFO:teuthology.orchestra.run.smithi164.stdout:(124/152): python3-grpcio-1.46.7-10.el9.x86_64. 8.4 MB/s | 2.0 MB 00:00 2024-04-26T21:38:21.282 INFO:teuthology.orchestra.run.smithi164.stdout:(125/152): python3-jaraco-classes-3.2.1-5.el9.n 164 kB/s | 18 kB 00:00 2024-04-26T21:38:21.324 INFO:teuthology.orchestra.run.smithi164.stdout:(126/152): python3-jaraco-8.2.1-3.el9.noarch.rp 59 kB/s | 11 kB 00:00 2024-04-26T21:38:21.349 INFO:teuthology.orchestra.run.smithi164.stdout:(127/152): python3-jaraco-collections-3.0.0-8.e 253 kB/s | 23 kB 00:00 2024-04-26T21:38:21.374 INFO:teuthology.orchestra.run.smithi164.stdout:(128/152): python3-jaraco-functools-3.5.0-2.el9 212 kB/s | 19 kB 00:00 2024-04-26T21:38:21.409 INFO:teuthology.orchestra.run.smithi164.stdout:(129/152): python3-jaraco-text-3.2.0-6.el9.noar 237 kB/s | 20 kB 00:00 2024-04-26T21:38:21.467 INFO:teuthology.orchestra.run.smithi164.stdout:(130/152): python3-more-itertools-8.12.0-2.el9. 1.3 MB/s | 79 kB 00:00 2024-04-26T21:38:21.492 INFO:teuthology.orchestra.run.smithi164.stdout:(131/152): python3-logutils-0.3.5-21.el9.noarch 397 kB/s | 46 kB 00:00 2024-04-26T21:38:21.561 INFO:teuthology.orchestra.run.smithi164.stdout:(132/152): python3-kubernetes-26.1.0-2.el9.noar 4.8 MB/s | 1.0 MB 00:00 2024-04-26T21:38:21.592 INFO:teuthology.orchestra.run.smithi164.stdout:(133/152): python3-natsort-7.1.1-5.el9.noarch.r 462 kB/s | 58 kB 00:00 2024-04-26T21:38:21.626 INFO:teuthology.orchestra.run.smithi164.stdout:(134/152): python3-portend-3.1.0-2.el9.noarch.r 256 kB/s | 16 kB 00:00 2024-04-26T21:38:21.660 INFO:teuthology.orchestra.run.smithi164.stdout:(135/152): python3-pecan-1.4.2-3.el9.noarch.rpm 1.6 MB/s | 272 kB 00:00 2024-04-26T21:38:21.685 INFO:teuthology.orchestra.run.smithi164.stdout:(136/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 983 kB/s | 90 kB 00:00 2024-04-26T21:38:21.710 INFO:teuthology.orchestra.run.smithi164.stdout:(137/152): python3-pyperclip-1.8.2-2.el9.noarch 339 kB/s | 28 kB 00:00 2024-04-26T21:38:21.735 INFO:teuthology.orchestra.run.smithi164.stdout:(138/152): python3-repoze-lru-0.7-16.el9.noarch 411 kB/s | 31 kB 00:00 2024-04-26T21:38:21.777 INFO:teuthology.orchestra.run.smithi164.stdout:(139/152): python3-routes-2.5.1-5.el9.noarch.rp 2.0 MB/s | 188 kB 00:00 2024-04-26T21:38:21.803 INFO:teuthology.orchestra.run.smithi164.stdout:(140/152): python3-tempora-5.0.0-2.el9.noarch.r 540 kB/s | 36 kB 00:00 2024-04-26T21:38:21.821 INFO:teuthology.orchestra.run.smithi164.stdout:(141/152): python3-rsa-4.9-2.el9.noarch.rpm 546 kB/s | 59 kB 00:00 2024-04-26T21:38:21.853 INFO:teuthology.orchestra.run.smithi164.stdout:(142/152): python3-typing-extensions-4.4.0-2.el 683 kB/s | 51 kB 00:00 2024-04-26T21:38:21.886 INFO:teuthology.orchestra.run.smithi164.stdout:(143/152): python3-webob-1.8.7-6.el9.noarch.rpm 2.7 MB/s | 230 kB 00:00 2024-04-26T21:38:21.937 INFO:teuthology.orchestra.run.smithi164.stdout:(144/152): python3-websocket-client-1.2.3-2.el9 768 kB/s | 90 kB 00:00 2024-04-26T21:38:21.954 INFO:teuthology.orchestra.run.smithi164.stdout:(145/152): python3-zc-lockfile-2.0-10.el9.noarc 301 kB/s | 20 kB 00:00 2024-04-26T21:38:21.987 INFO:teuthology.orchestra.run.smithi164.stdout:(146/152): python3-werkzeug-2.0.3-3.el9.1.noarc 3.1 MB/s | 427 kB 00:00 2024-04-26T21:38:22.012 INFO:teuthology.orchestra.run.smithi164.stdout:(147/152): re2-20211101-3.el9.x86_64.rpm 2.5 MB/s | 192 kB 00:00 2024-04-26T21:38:22.096 INFO:teuthology.orchestra.run.smithi164.stdout:(148/152): thrift-0.15.0-2.el9.x86_64.rpm 11 MB/s | 1.6 MB 00:00 2024-04-26T21:38:22.364 INFO:teuthology.orchestra.run.smithi164.stdout:(149/152): librbd1-19.0.0-3306.gbc612ffb.el9.x8 9.0 MB/s | 3.1 MB 00:00 2024-04-26T21:38:22.539 INFO:teuthology.orchestra.run.smithi164.stdout:(150/152): librados2-19.0.0-3306.gbc612ffb.el9. 6.2 MB/s | 3.4 MB 00:00 2024-04-26T21:38:22.629 INFO:teuthology.orchestra.run.smithi164.stdout:(151/152): openssl-3.2.1-1.el9.x86_64.rpm 2.5 MB/s | 1.3 MB 00:00 2024-04-26T21:38:22.822 INFO:teuthology.orchestra.run.smithi164.stdout:(152/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 5.2 MB/s | 2.4 MB 00:00 2024-04-26T21:38:22.832 INFO:teuthology.orchestra.run.smithi164.stdout:-------------------------------------------------------------------------------- 2024-04-26T21:38:22.833 INFO:teuthology.orchestra.run.smithi164.stdout:Total 13 MB/s | 222 MB 00:16 2024-04-26T21:38:23.310 INFO:teuthology.orchestra.run.smithi076.stdout:Running transaction check 2024-04-26T21:38:23.557 INFO:teuthology.orchestra.run.smithi076.stdout:Transaction check succeeded. 2024-04-26T21:38:23.557 INFO:teuthology.orchestra.run.smithi076.stdout:Running transaction test 2024-04-26T21:38:23.715 INFO:teuthology.orchestra.run.smithi072.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-04-26T21:38:25.672 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction check 2024-04-26T21:38:25.919 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction check succeeded. 2024-04-26T21:38:25.919 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction test 2024-04-26T21:38:26.184 INFO:teuthology.orchestra.run.smithi072.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-26T21:38:26.186 INFO:teuthology.orchestra.run.smithi072.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-26T21:38:26.206 INFO:teuthology.orchestra.run.smithi072.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-26T21:38:26.207 INFO:teuthology.orchestra.run.smithi072.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-26T21:38:26.209 INFO:teuthology.orchestra.run.smithi072.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-26T21:38:26.210 INFO:teuthology.orchestra.run.smithi072.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-26T21:38:26.212 INFO:teuthology.orchestra.run.smithi072.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-26T21:38:26.216 INFO:teuthology.orchestra.run.smithi072.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-26T21:38:26.217 INFO:teuthology.orchestra.run.smithi072.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-26T21:38:26.286 INFO:teuthology.orchestra.run.smithi072.stdout:Dependencies resolved. 2024-04-26T21:38:26.318 INFO:teuthology.orchestra.run.smithi072.stdout:======================================================================================= 2024-04-26T21:38:26.318 INFO:teuthology.orchestra.run.smithi072.stdout: Package Arch Version Repository Size 2024-04-26T21:38:26.318 INFO:teuthology.orchestra.run.smithi072.stdout:======================================================================================= 2024-04-26T21:38:26.318 INFO:teuthology.orchestra.run.smithi072.stdout:Installing: 2024-04-26T21:38:26.318 INFO:teuthology.orchestra.run.smithi072.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-26T21:38:26.318 INFO:teuthology.orchestra.run.smithi072.stdout: ceph x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 6.6 k 2024-04-26T21:38:26.318 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-base x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 5.4 M 2024-04-26T21:38:26.319 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-fuse x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 889 k 2024-04-26T21:38:26.319 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 145 k 2024-04-26T21:38:26.319 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mgr x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 1.5 M 2024-04-26T21:38:26.319 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 142 k 2024-04-26T21:38:26.319 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 3.5 M 2024-04-26T21:38:26.319 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 7.4 M 2024-04-26T21:38:26.319 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mgr-rook noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 49 k 2024-04-26T21:38:26.319 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-radosgw x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 11 M 2024-04-26T21:38:26.319 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-test x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 49 M 2024-04-26T21:38:26.319 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-volume noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 273 k 2024-04-26T21:38:26.319 INFO:teuthology.orchestra.run.smithi072.stdout: cephadm noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 762 k 2024-04-26T21:38:26.319 INFO:teuthology.orchestra.run.smithi072.stdout: cephfs-mirror x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 225 k 2024-04-26T21:38:26.319 INFO:teuthology.orchestra.run.smithi072.stdout: cephfs-shell x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 27 k 2024-04-26T21:38:26.320 INFO:teuthology.orchestra.run.smithi072.stdout: cephfs-top noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 18 k 2024-04-26T21:38:26.320 INFO:teuthology.orchestra.run.smithi072.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-26T21:38:26.320 INFO:teuthology.orchestra.run.smithi072.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-26T21:38:26.320 INFO:teuthology.orchestra.run.smithi072.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-26T21:38:26.320 INFO:teuthology.orchestra.run.smithi072.stdout: libcephfs-devel x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 32 k 2024-04-26T21:38:26.320 INFO:teuthology.orchestra.run.smithi072.stdout: libcephfs2 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 747 k 2024-04-26T21:38:26.320 INFO:teuthology.orchestra.run.smithi072.stdout: librados-devel x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 127 k 2024-04-26T21:38:26.320 INFO:teuthology.orchestra.run.smithi072.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-26T21:38:26.320 INFO:teuthology.orchestra.run.smithi072.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-26T21:38:26.320 INFO:teuthology.orchestra.run.smithi072.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-26T21:38:26.320 INFO:teuthology.orchestra.run.smithi072.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-26T21:38:26.320 INFO:teuthology.orchestra.run.smithi072.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-26T21:38:26.320 INFO:teuthology.orchestra.run.smithi072.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-26T21:38:26.321 INFO:teuthology.orchestra.run.smithi072.stdout: python3-cephfs x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 161 k 2024-04-26T21:38:26.321 INFO:teuthology.orchestra.run.smithi072.stdout: python3-rados x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 321 k 2024-04-26T21:38:26.321 INFO:teuthology.orchestra.run.smithi072.stdout: python3-rbd x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 298 k 2024-04-26T21:38:26.321 INFO:teuthology.orchestra.run.smithi072.stdout: python3-rgw x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 99 k 2024-04-26T21:38:26.321 INFO:teuthology.orchestra.run.smithi072.stdout: rbd-fuse x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 85 k 2024-04-26T21:38:26.321 INFO:teuthology.orchestra.run.smithi072.stdout: rbd-mirror x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 3.1 M 2024-04-26T21:38:26.321 INFO:teuthology.orchestra.run.smithi072.stdout: rbd-nbd x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 171 k 2024-04-26T21:38:26.321 INFO:teuthology.orchestra.run.smithi072.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-26T21:38:26.321 INFO:teuthology.orchestra.run.smithi072.stdout:Upgrading: 2024-04-26T21:38:26.321 INFO:teuthology.orchestra.run.smithi072.stdout: librados2 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 3.4 M 2024-04-26T21:38:26.321 INFO:teuthology.orchestra.run.smithi072.stdout: librbd1 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 3.1 M 2024-04-26T21:38:26.321 INFO:teuthology.orchestra.run.smithi072.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-26T21:38:26.321 INFO:teuthology.orchestra.run.smithi072.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-26T21:38:26.321 INFO:teuthology.orchestra.run.smithi072.stdout:Installing dependencies: 2024-04-26T21:38:26.322 INFO:teuthology.orchestra.run.smithi072.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-26T21:38:26.322 INFO:teuthology.orchestra.run.smithi072.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-26T21:38:26.322 INFO:teuthology.orchestra.run.smithi072.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-26T21:38:26.322 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-common x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 21 M 2024-04-26T21:38:26.322 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 34 k 2024-04-26T21:38:26.322 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mds x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 2.4 M 2024-04-26T21:38:26.322 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 249 k 2024-04-26T21:38:26.322 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mon x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 4.7 M 2024-04-26T21:38:26.322 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-osd x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 17 M 2024-04-26T21:38:26.322 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 16 k 2024-04-26T21:38:26.322 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-selinux x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 25 k 2024-04-26T21:38:26.322 INFO:teuthology.orchestra.run.smithi072.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-26T21:38:26.323 INFO:teuthology.orchestra.run.smithi072.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-26T21:38:26.323 INFO:teuthology.orchestra.run.smithi072.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-26T21:38:26.323 INFO:teuthology.orchestra.run.smithi072.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-26T21:38:26.323 INFO:teuthology.orchestra.run.smithi072.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-26T21:38:26.323 INFO:teuthology.orchestra.run.smithi072.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-26T21:38:26.323 INFO:teuthology.orchestra.run.smithi072.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-04-26T21:38:26.323 INFO:teuthology.orchestra.run.smithi072.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-04-26T21:38:26.323 INFO:teuthology.orchestra.run.smithi072.stdout: libcephsqlite x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 163 k 2024-04-26T21:38:26.323 INFO:teuthology.orchestra.run.smithi072.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-26T21:38:26.323 INFO:teuthology.orchestra.run.smithi072.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-26T21:38:26.323 INFO:teuthology.orchestra.run.smithi072.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-26T21:38:26.323 INFO:teuthology.orchestra.run.smithi072.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-26T21:38:26.323 INFO:teuthology.orchestra.run.smithi072.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-26T21:38:26.323 INFO:teuthology.orchestra.run.smithi072.stdout: libradosstriper1 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 501 k 2024-04-26T21:38:26.324 INFO:teuthology.orchestra.run.smithi072.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-26T21:38:26.324 INFO:teuthology.orchestra.run.smithi072.stdout: librgw2 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 5.4 M 2024-04-26T21:38:26.324 INFO:teuthology.orchestra.run.smithi072.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-26T21:38:26.324 INFO:teuthology.orchestra.run.smithi072.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-26T21:38:26.324 INFO:teuthology.orchestra.run.smithi072.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-26T21:38:26.324 INFO:teuthology.orchestra.run.smithi072.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-26T21:38:26.324 INFO:teuthology.orchestra.run.smithi072.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-26T21:38:26.324 INFO:teuthology.orchestra.run.smithi072.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-26T21:38:26.324 INFO:teuthology.orchestra.run.smithi072.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-26T21:38:26.324 INFO:teuthology.orchestra.run.smithi072.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-26T21:38:26.324 INFO:teuthology.orchestra.run.smithi072.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-04-26T21:38:26.324 INFO:teuthology.orchestra.run.smithi072.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-26T21:38:26.324 INFO:teuthology.orchestra.run.smithi072.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-26T21:38:26.324 INFO:teuthology.orchestra.run.smithi072.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-26T21:38:26.325 INFO:teuthology.orchestra.run.smithi072.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-26T21:38:26.325 INFO:teuthology.orchestra.run.smithi072.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-26T21:38:26.325 INFO:teuthology.orchestra.run.smithi072.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-26T21:38:26.325 INFO:teuthology.orchestra.run.smithi072.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-26T21:38:26.325 INFO:teuthology.orchestra.run.smithi072.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-26T21:38:26.325 INFO:teuthology.orchestra.run.smithi072.stdout: python3-ceph-argparse x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 45 k 2024-04-26T21:38:26.325 INFO:teuthology.orchestra.run.smithi072.stdout: python3-ceph-common x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 135 k 2024-04-26T21:38:26.325 INFO:teuthology.orchestra.run.smithi072.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-26T21:38:26.325 INFO:teuthology.orchestra.run.smithi072.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-26T21:38:26.325 INFO:teuthology.orchestra.run.smithi072.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-26T21:38:26.325 INFO:teuthology.orchestra.run.smithi072.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-26T21:38:26.325 INFO:teuthology.orchestra.run.smithi072.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-26T21:38:26.325 INFO:teuthology.orchestra.run.smithi072.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-26T21:38:26.326 INFO:teuthology.orchestra.run.smithi072.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-26T21:38:26.326 INFO:teuthology.orchestra.run.smithi072.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-26T21:38:26.326 INFO:teuthology.orchestra.run.smithi072.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-26T21:38:26.326 INFO:teuthology.orchestra.run.smithi072.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-26T21:38:26.326 INFO:teuthology.orchestra.run.smithi072.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-26T21:38:26.326 INFO:teuthology.orchestra.run.smithi072.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-26T21:38:26.326 INFO:teuthology.orchestra.run.smithi072.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-26T21:38:26.326 INFO:teuthology.orchestra.run.smithi072.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-26T21:38:26.326 INFO:teuthology.orchestra.run.smithi072.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-26T21:38:26.326 INFO:teuthology.orchestra.run.smithi072.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-26T21:38:26.326 INFO:teuthology.orchestra.run.smithi072.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-26T21:38:26.326 INFO:teuthology.orchestra.run.smithi072.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-26T21:38:26.327 INFO:teuthology.orchestra.run.smithi072.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-26T21:38:26.327 INFO:teuthology.orchestra.run.smithi072.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-26T21:38:26.327 INFO:teuthology.orchestra.run.smithi072.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-26T21:38:26.327 INFO:teuthology.orchestra.run.smithi072.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-26T21:38:26.327 INFO:teuthology.orchestra.run.smithi072.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-26T21:38:26.327 INFO:teuthology.orchestra.run.smithi072.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-26T21:38:26.327 INFO:teuthology.orchestra.run.smithi072.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-26T21:38:26.327 INFO:teuthology.orchestra.run.smithi072.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-26T21:38:26.327 INFO:teuthology.orchestra.run.smithi072.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-26T21:38:26.327 INFO:teuthology.orchestra.run.smithi072.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-26T21:38:26.329 INFO:teuthology.orchestra.run.smithi072.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-26T21:38:26.329 INFO:teuthology.orchestra.run.smithi072.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-26T21:38:26.329 INFO:teuthology.orchestra.run.smithi072.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-26T21:38:26.329 INFO:teuthology.orchestra.run.smithi072.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-26T21:38:26.329 INFO:teuthology.orchestra.run.smithi072.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-26T21:38:26.329 INFO:teuthology.orchestra.run.smithi072.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-26T21:38:26.329 INFO:teuthology.orchestra.run.smithi072.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-26T21:38:26.329 INFO:teuthology.orchestra.run.smithi072.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-26T21:38:26.329 INFO:teuthology.orchestra.run.smithi072.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-26T21:38:26.330 INFO:teuthology.orchestra.run.smithi072.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-26T21:38:26.330 INFO:teuthology.orchestra.run.smithi072.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-26T21:38:26.330 INFO:teuthology.orchestra.run.smithi072.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-26T21:38:26.330 INFO:teuthology.orchestra.run.smithi072.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-26T21:38:26.330 INFO:teuthology.orchestra.run.smithi072.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-26T21:38:26.330 INFO:teuthology.orchestra.run.smithi072.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-26T21:38:26.330 INFO:teuthology.orchestra.run.smithi072.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-26T21:38:26.330 INFO:teuthology.orchestra.run.smithi072.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-26T21:38:26.330 INFO:teuthology.orchestra.run.smithi072.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-26T21:38:26.330 INFO:teuthology.orchestra.run.smithi072.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-26T21:38:26.330 INFO:teuthology.orchestra.run.smithi072.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-26T21:38:26.330 INFO:teuthology.orchestra.run.smithi072.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-26T21:38:26.331 INFO:teuthology.orchestra.run.smithi072.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-26T21:38:26.331 INFO:teuthology.orchestra.run.smithi072.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-26T21:38:26.331 INFO:teuthology.orchestra.run.smithi072.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-26T21:38:26.331 INFO:teuthology.orchestra.run.smithi072.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-26T21:38:26.331 INFO:teuthology.orchestra.run.smithi072.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-26T21:38:26.331 INFO:teuthology.orchestra.run.smithi072.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-26T21:38:26.331 INFO:teuthology.orchestra.run.smithi072.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-26T21:38:26.331 INFO:teuthology.orchestra.run.smithi072.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-26T21:38:26.331 INFO:teuthology.orchestra.run.smithi072.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-26T21:38:26.331 INFO:teuthology.orchestra.run.smithi072.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-26T21:38:26.331 INFO:teuthology.orchestra.run.smithi072.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-26T21:38:26.331 INFO:teuthology.orchestra.run.smithi072.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-26T21:38:26.331 INFO:teuthology.orchestra.run.smithi072.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-26T21:38:26.331 INFO:teuthology.orchestra.run.smithi072.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-26T21:38:26.332 INFO:teuthology.orchestra.run.smithi072.stdout:Installing weak dependencies: 2024-04-26T21:38:26.332 INFO:teuthology.orchestra.run.smithi072.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-26T21:38:26.332 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T21:38:26.332 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction Summary 2024-04-26T21:38:26.332 INFO:teuthology.orchestra.run.smithi072.stdout:======================================================================================= 2024-04-26T21:38:26.332 INFO:teuthology.orchestra.run.smithi072.stdout:Install 148 Packages 2024-04-26T21:38:26.332 INFO:teuthology.orchestra.run.smithi072.stdout:Upgrade 4 Packages 2024-04-26T21:38:26.332 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T21:38:26.333 INFO:teuthology.orchestra.run.smithi072.stdout:Total download size: 222 M 2024-04-26T21:38:26.333 INFO:teuthology.orchestra.run.smithi072.stdout:Downloading Packages: 2024-04-26T21:38:26.368 INFO:teuthology.orchestra.run.smithi076.stdout:Transaction test succeeded. 2024-04-26T21:38:26.369 INFO:teuthology.orchestra.run.smithi076.stdout:Running transaction 2024-04-26T21:38:27.395 INFO:teuthology.orchestra.run.smithi072.stdout:(1/152): ceph-19.0.0-3306.gbc612ffb.el9.x86_64. 25 kB/s | 6.6 kB 00:00 2024-04-26T21:38:27.688 INFO:teuthology.orchestra.run.smithi072.stdout:(2/152): ceph-fuse-19.0.0-3306.gbc612ffb.el9.x8 3.0 MB/s | 889 kB 00:00 2024-04-26T21:38:27.771 INFO:teuthology.orchestra.run.smithi072.stdout:(3/152): ceph-immutable-object-cache-19.0.0-330 1.7 MB/s | 145 kB 00:00 2024-04-26T21:38:28.214 INFO:teuthology.orchestra.run.smithi072.stdout:(4/152): ceph-mds-19.0.0-3306.gbc612ffb.el9.x86 5.4 MB/s | 2.4 MB 00:00 2024-04-26T21:38:28.481 INFO:teuthology.orchestra.run.smithi072.stdout:(5/152): ceph-mgr-19.0.0-3306.gbc612ffb.el9.x86 5.7 MB/s | 1.5 MB 00:00 2024-04-26T21:38:28.662 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction test succeeded. 2024-04-26T21:38:28.663 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction 2024-04-26T21:38:28.865 INFO:teuthology.orchestra.run.smithi072.stdout:(6/152): ceph-common-19.0.0-3306.gbc612ffb.el9. 12 MB/s | 21 MB 00:01 2024-04-26T21:38:29.223 INFO:teuthology.orchestra.run.smithi072.stdout:(7/152): ceph-mon-19.0.0-3306.gbc612ffb.el9.x86 6.3 MB/s | 4.7 MB 00:00 2024-04-26T21:38:29.907 INFO:teuthology.orchestra.run.smithi072.stdout:(8/152): ceph-base-19.0.0-3306.gbc612ffb.el9.x8 1.9 MB/s | 5.4 MB 00:02 2024-04-26T21:38:30.275 INFO:teuthology.orchestra.run.smithi072.stdout:(9/152): ceph-osd-19.0.0-3306.gbc612ffb.el9.x86 12 MB/s | 17 MB 00:01 2024-04-26T21:38:30.302 INFO:teuthology.orchestra.run.smithi072.stdout:(10/152): ceph-selinux-19.0.0-3306.gbc612ffb.el 64 kB/s | 25 kB 00:00 2024-04-26T21:38:30.469 INFO:teuthology.orchestra.run.smithi072.stdout:(11/152): cephfs-mirror-19.0.0-3306.gbc612ffb.e 1.3 MB/s | 225 kB 00:00 2024-04-26T21:38:30.577 INFO:teuthology.orchestra.run.smithi072.stdout:(12/152): cephfs-shell-19.0.0-3306.gbc612ffb.el 251 kB/s | 27 kB 00:00 2024-04-26T21:38:30.695 INFO:teuthology.orchestra.run.smithi072.stdout:(13/152): libcephfs-devel-19.0.0-3306.gbc612ffb 273 kB/s | 32 kB 00:00 2024-04-26T21:38:30.962 INFO:teuthology.orchestra.run.smithi072.stdout:(14/152): ceph-radosgw-19.0.0-3306.gbc612ffb.el 6.1 MB/s | 11 MB 00:01 2024-04-26T21:38:31.020 INFO:teuthology.orchestra.run.smithi072.stdout:(15/152): libcephfs2-19.0.0-3306.gbc612ffb.el9. 2.2 MB/s | 747 kB 00:00 2024-04-26T21:38:31.055 INFO:teuthology.orchestra.run.smithi072.stdout:(16/152): libcephsqlite-19.0.0-3306.gbc612ffb.e 1.7 MB/s | 163 kB 00:00 2024-04-26T21:38:31.088 INFO:teuthology.orchestra.run.smithi072.stdout:(17/152): librados-devel-19.0.0-3306.gbc612ffb. 1.9 MB/s | 127 kB 00:00 2024-04-26T21:38:31.147 INFO:teuthology.orchestra.run.smithi072.stdout:(18/152): libradosstriper1-19.0.0-3306.gbc612ff 5.3 MB/s | 501 kB 00:00 2024-04-26T21:38:31.206 INFO:teuthology.orchestra.run.smithi072.stdout:(19/152): python3-ceph-argparse-19.0.0-3306.gbc 784 kB/s | 45 kB 00:00 2024-04-26T21:38:31.282 INFO:teuthology.orchestra.run.smithi072.stdout:(20/152): python3-ceph-common-19.0.0-3306.gbc61 1.8 MB/s | 135 kB 00:00 2024-04-26T21:38:31.374 INFO:teuthology.orchestra.run.smithi072.stdout:(21/152): python3-cephfs-19.0.0-3306.gbc612ffb. 1.7 MB/s | 161 kB 00:00 2024-04-26T21:38:31.483 INFO:teuthology.orchestra.run.smithi072.stdout:(22/152): python3-rados-19.0.0-3306.gbc612ffb.e 2.9 MB/s | 321 kB 00:00 2024-04-26T21:38:31.583 INFO:teuthology.orchestra.run.smithi072.stdout:(23/152): python3-rbd-19.0.0-3306.gbc612ffb.el9 2.9 MB/s | 298 kB 00:00 2024-04-26T21:38:31.676 INFO:teuthology.orchestra.run.smithi072.stdout:(24/152): python3-rgw-19.0.0-3306.gbc612ffb.el9 1.1 MB/s | 99 kB 00:00 2024-04-26T21:38:31.768 INFO:teuthology.orchestra.run.smithi072.stdout:(25/152): rbd-fuse-19.0.0-3306.gbc612ffb.el9.x8 937 kB/s | 85 kB 00:00 2024-04-26T21:38:32.362 INFO:teuthology.orchestra.run.smithi072.stdout:(26/152): librgw2-19.0.0-3306.gbc612ffb.el9.x86 4.2 MB/s | 5.4 MB 00:01 2024-04-26T21:38:32.470 INFO:teuthology.orchestra.run.smithi072.stdout:(27/152): rbd-mirror-19.0.0-3306.gbc612ffb.el9. 4.4 MB/s | 3.1 MB 00:00 2024-04-26T21:38:32.504 INFO:teuthology.orchestra.run.smithi072.stdout:(28/152): rbd-nbd-19.0.0-3306.gbc612ffb.el9.x86 1.2 MB/s | 171 kB 00:00 2024-04-26T21:38:32.530 INFO:teuthology.orchestra.run.smithi072.stdout:(29/152): ceph-grafana-dashboards-19.0.0-3306.g 588 kB/s | 34 kB 00:00 2024-04-26T21:38:32.581 INFO:teuthology.orchestra.run.smithi072.stdout:(30/152): ceph-mgr-cephadm-19.0.0-3306.gbc612ff 1.8 MB/s | 142 kB 00:00 2024-04-26T21:38:32.900 INFO:teuthology.orchestra.run.smithi076.stdout: Preparing : 1/1 2024-04-26T21:38:33.021 INFO:teuthology.orchestra.run.smithi076.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-26T21:38:33.110 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-26T21:38:33.248 INFO:teuthology.orchestra.run.smithi072.stdout:(31/152): ceph-mgr-dashboard-19.0.0-3306.gbc612 4.9 MB/s | 3.5 MB 00:00 2024-04-26T21:38:33.339 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-26T21:38:33.503 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-26T21:38:33.550 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-26T21:38:33.682 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-26T21:38:34.101 INFO:teuthology.orchestra.run.smithi072.stdout:(32/152): ceph-test-19.0.0-3306.gbc612ffb.el9.x 13 MB/s | 49 MB 00:03 2024-04-26T21:38:34.134 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-26T21:38:34.140 INFO:teuthology.orchestra.run.smithi072.stdout:(33/152): ceph-mgr-modules-core-19.0.0-3306.gbc 279 kB/s | 249 kB 00:00 2024-04-26T21:38:34.166 INFO:teuthology.orchestra.run.smithi072.stdout:(34/152): ceph-mgr-rook-19.0.0-3306.gbc612ffb.e 750 kB/s | 49 kB 00:00 2024-04-26T21:38:34.208 INFO:teuthology.orchestra.run.smithi072.stdout:(35/152): ceph-prometheus-alerts-19.0.0-3306.gb 248 kB/s | 16 kB 00:00 2024-04-26T21:38:34.246 INFO:teuthology.orchestra.run.smithi076.stdout: Upgrading : librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 8/156 2024-04-26T21:38:34.401 INFO:teuthology.orchestra.run.smithi072.stdout:(36/152): ceph-mgr-diskprediction-local-19.0.0- 4.1 MB/s | 7.4 MB 00:01 2024-04-26T21:38:34.413 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 8/156 2024-04-26T21:38:34.434 INFO:teuthology.orchestra.run.smithi072.stdout:(37/152): ceph-volume-19.0.0-3306.gbc612ffb.el9 1.0 MB/s | 273 kB 00:00 2024-04-26T21:38:34.438 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 9/156 2024-04-26T21:38:34.493 INFO:teuthology.orchestra.run.smithi072.stdout:(38/152): cephfs-top-19.0.0-3306.gbc612ffb.el9. 200 kB/s | 18 kB 00:00 2024-04-26T21:38:34.535 INFO:teuthology.orchestra.run.smithi072.stdout:(39/152): cephadm-19.0.0-3306.gbc612ffb.el9.noa 2.3 MB/s | 762 kB 00:00 2024-04-26T21:38:34.562 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 9/156 2024-04-26T21:38:34.569 INFO:teuthology.orchestra.run.smithi072.stdout:(40/152): fuse-2.9.9-15.el9.x86_64.rpm 601 kB/s | 80 kB 00:00 2024-04-26T21:38:34.596 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-rados-2:19.0.0-3306.gbc612ffb.el9.x86_64 10/156 2024-04-26T21:38:34.663 INFO:teuthology.orchestra.run.smithi072.stdout:(41/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.7 MB/s | 253 kB 00:00 2024-04-26T21:38:34.747 INFO:teuthology.orchestra.run.smithi072.stdout:(42/152): libgfortran-11.4.1-3.el9.x86_64.rpm 3.1 MB/s | 803 kB 00:00 2024-04-26T21:38:34.781 INFO:teuthology.orchestra.run.smithi072.stdout:(43/152): libquadmath-11.4.1-3.el9.x86_64.rpm 800 kB/s | 195 kB 00:00 2024-04-26T21:38:34.814 INFO:teuthology.orchestra.run.smithi072.stdout:(44/152): python3-ply-3.11-14.el9.noarch.rpm 1.6 MB/s | 106 kB 00:00 2024-04-26T21:38:34.848 INFO:teuthology.orchestra.run.smithi072.stdout:(45/152): python3-pycparser-2.20-6.el9.noarch.r 2.0 MB/s | 135 kB 00:00 2024-04-26T21:38:34.874 INFO:teuthology.orchestra.run.smithi072.stdout:(46/152): python3-requests-2.25.1-8.el9.noarch. 2.1 MB/s | 125 kB 00:00 2024-04-26T21:38:34.907 INFO:teuthology.orchestra.run.smithi072.stdout:(47/152): python3-urllib3-1.26.5-5.el9.noarch.r 3.6 MB/s | 215 kB 00:00 2024-04-26T21:38:34.965 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-26T21:38:34.966 INFO:teuthology.orchestra.run.smithi072.stdout:(48/152): python3-cryptography-36.0.1-4.el9.x86 4.1 MB/s | 1.2 MB 00:00 2024-04-26T21:38:35.027 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-26T21:38:35.052 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 13/156 2024-04-26T21:38:35.198 INFO:teuthology.orchestra.run.smithi164.stdout: Preparing : 1/1 2024-04-26T21:38:35.218 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 13/156 2024-04-26T21:38:35.245 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86 14/156 2024-04-26T21:38:35.296 INFO:teuthology.orchestra.run.smithi164.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-26T21:38:35.370 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86 14/156 2024-04-26T21:38:35.379 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-26T21:38:35.396 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-26T21:38:35.476 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-26T21:38:35.638 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-26T21:38:35.700 INFO:teuthology.orchestra.run.smithi072.stdout:(49/152): boost-program-options-1.75.0-8.el9.x8 146 kB/s | 107 kB 00:00 2024-04-26T21:38:35.757 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-26T21:38:35.798 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-26T21:38:35.814 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-26T21:38:35.849 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-26T21:38:36.011 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-26T21:38:36.024 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-26T21:38:36.141 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 18/156 2024-04-26T21:38:36.181 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-26T21:38:36.237 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-26T21:38:36.259 INFO:teuthology.orchestra.run.smithi072.stdout:(50/152): elfutils-libelf-devel-0.190-2.el9.x86 39 kB/s | 21 kB 00:00 2024-04-26T21:38:36.301 INFO:teuthology.orchestra.run.smithi072.stdout:(51/152): automake-1.16.2-8.el9.noarch.rpm 483 kB/s | 689 kB 00:01 2024-04-26T21:38:36.340 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-26T21:38:36.427 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-26T21:38:36.451 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-26T21:38:36.520 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-26T21:38:36.544 INFO:teuthology.orchestra.run.smithi164.stdout: Upgrading : librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 8/156 2024-04-26T21:38:36.586 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-26T21:38:36.627 INFO:teuthology.orchestra.run.smithi072.stdout:(52/152): bison-3.7.4-5.el9.x86_64.rpm 549 kB/s | 943 kB 00:01 2024-04-26T21:38:36.640 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-26T21:38:36.771 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 8/156 2024-04-26T21:38:36.787 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-26T21:38:36.804 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 9/156 2024-04-26T21:38:36.886 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-26T21:38:36.937 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 9/156 2024-04-26T21:38:36.944 INFO:teuthology.orchestra.run.smithi072.stdout:(53/152): flexiblas-3.0.4-8.el9.x86_64.rpm 51 kB/s | 33 kB 00:00 2024-04-26T21:38:36.950 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-26T21:38:37.005 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-rados-2:19.0.0-3306.gbc612ffb.el9.x86_64 10/156 2024-04-26T21:38:37.036 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-26T21:38:37.177 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-26T21:38:37.245 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-ceph-common-2:19.0.0-3306.gbc612ffb.el9. 31/156 2024-04-26T21:38:37.311 INFO:teuthology.orchestra.run.smithi072.stdout:(54/152): flex-2.6.4-9.el9.x86_64.rpm 299 kB/s | 313 kB 00:01 2024-04-26T21:38:37.324 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-ceph-argparse-2:19.0.0-3306.gbc612ffb.el 32/156 2024-04-26T21:38:37.385 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-26T21:38:37.388 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-cephfs-2:19.0.0-3306.gbc612ffb.el9.x86_6 33/156 2024-04-26T21:38:37.453 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-26T21:38:37.483 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 13/156 2024-04-26T21:38:37.512 INFO:teuthology.orchestra.run.smithi072.stdout:(55/152): flexiblas-openblas-openmp-3.0.4-8.el9 32 kB/s | 18 kB 00:00 2024-04-26T21:38:37.544 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-26T21:38:37.632 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-26T21:38:37.639 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 13/156 2024-04-26T21:38:37.693 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86 14/156 2024-04-26T21:38:37.760 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-26T21:38:37.794 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86 14/156 2024-04-26T21:38:37.827 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-26T21:38:37.847 INFO:teuthology.orchestra.run.smithi072.stdout:(56/152): libaio-devel-0.3.111-13.el9.x86_64.rp 21 kB/s | 11 kB 00:00 2024-04-26T21:38:37.893 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-26T21:38:37.958 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-26T21:38:38.042 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-26T21:38:38.198 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-26T21:38:38.260 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-26T21:38:38.315 INFO:teuthology.orchestra.run.smithi072.stdout:(57/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 200 kB/s | 160 kB 00:00 2024-04-26T21:38:38.446 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-26T21:38:38.499 INFO:teuthology.orchestra.run.smithi072.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 70 kB/s | 45 kB 00:00 2024-04-26T21:38:38.580 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 18/156 2024-04-26T21:38:38.628 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-26T21:38:38.673 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-26T21:38:38.757 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-26T21:38:38.791 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-26T21:38:38.847 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-26T21:38:38.880 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-26T21:38:38.911 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-26T21:38:38.934 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-26T21:38:38.960 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-26T21:38:38.998 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-26T21:38:39.000 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-26T21:38:39.074 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : librados-devel-2:19.0.0-3306.gbc612ffb.el9.x86_6 43/156 2024-04-26T21:38:39.111 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-26T21:38:39.177 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-26T21:38:39.191 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-26T21:38:39.222 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-26T21:38:39.276 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-26T21:38:39.381 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-26T21:38:39.439 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-ceph-common-2:19.0.0-3306.gbc612ffb.el9. 31/156 2024-04-26T21:38:39.490 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-ceph-argparse-2:19.0.0-3306.gbc612ffb.el 32/156 2024-04-26T21:38:39.512 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-26T21:38:39.521 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-cephfs-2:19.0.0-3306.gbc612ffb.el9.x86_6 33/156 2024-04-26T21:38:39.598 INFO:teuthology.orchestra.run.smithi076.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-26T21:38:39.639 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-26T21:38:39.681 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-26T21:38:39.683 INFO:teuthology.orchestra.run.smithi072.stdout:(59/152): librdkafka-1.6.1-102.el9.x86_64.rpm 485 kB/s | 662 kB 00:01 2024-04-26T21:38:39.701 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-26T21:38:39.778 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-26T21:38:39.814 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-26T21:38:39.816 INFO:teuthology.orchestra.run.smithi072.stdout:(60/152): libtool-2.4.6-45.el9.x86_64.rpm 442 kB/s | 582 kB 00:01 2024-04-26T21:38:39.842 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-26T21:38:39.852 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-26T21:38:39.918 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-26T21:38:39.961 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-26T21:38:40.017 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-26T21:38:40.049 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-26T21:38:40.102 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-26T21:38:40.118 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-26T21:38:40.327 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-26T21:38:40.376 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-26T21:38:40.417 INFO:teuthology.orchestra.run.smithi072.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 35 kB/s | 25 kB 00:00 2024-04-26T21:38:40.430 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-26T21:38:40.488 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-26T21:38:40.492 INFO:teuthology.orchestra.run.smithi072.stdout:(62/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 70 kB/s | 47 kB 00:00 2024-04-26T21:38:40.617 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-26T21:38:40.656 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-26T21:38:40.703 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-26T21:38:40.737 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-26T21:38:40.770 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 63/156 2024-04-26T21:38:40.796 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-26T21:38:40.834 INFO:teuthology.orchestra.run.smithi072.stdout:(63/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 728 kB/s | 3.0 MB 00:04 2024-04-26T21:38:40.891 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-26T21:38:40.935 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-26T21:38:40.987 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-26T21:38:40.989 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-26T21:38:41.031 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : librados-devel-2:19.0.0-3306.gbc612ffb.el9.x86_6 43/156 2024-04-26T21:38:41.091 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-26T21:38:41.167 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-26T21:38:41.237 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-26T21:38:41.286 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-26T21:38:41.322 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-26T21:38:41.342 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-26T21:38:41.412 INFO:teuthology.orchestra.run.smithi164.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-26T21:38:41.412 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-26T21:38:41.443 INFO:teuthology.orchestra.run.smithi072.stdout:(64/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 285 kB/s | 292 kB 00:01 2024-04-26T21:38:41.468 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-26T21:38:41.481 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-26T21:38:41.485 INFO:teuthology.orchestra.run.smithi072.stdout:(65/152): lua-5.4.4-4.el9.x86_64.rpm 190 kB/s | 188 kB 00:00 2024-04-26T21:38:41.517 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-26T21:38:41.553 INFO:teuthology.orchestra.run.smithi072.stdout:(66/152): openblas-0.3.21-2.el9.x86_64.rpm 54 kB/s | 35 kB 00:00 2024-04-26T21:38:41.563 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-26T21:38:41.592 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-26T21:38:41.612 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-26T21:38:41.628 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-26T21:38:41.684 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-26T21:38:41.713 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-26T21:38:41.752 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-26T21:38:41.761 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-26T21:38:41.803 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-26T21:38:41.889 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-26T21:38:41.903 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-26T21:38:41.957 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-26T21:38:41.961 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-26T21:38:42.009 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-26T21:38:42.104 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-26T21:38:42.165 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-26T21:38:42.233 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-26T21:38:42.255 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-26T21:38:42.270 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-26T21:38:42.323 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-26T21:38:42.326 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-26T21:38:42.456 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-26T21:38:42.503 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-26T21:38:42.584 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-26T21:38:42.627 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-26T21:38:42.660 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 63/156 2024-04-26T21:38:42.876 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-26T21:38:43.068 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-26T21:38:43.116 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-26T21:38:43.158 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-26T21:38:43.210 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-26T21:38:43.282 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-26T21:38:43.303 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-26T21:38:43.339 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-26T21:38:43.373 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-26T21:38:43.374 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-26T21:38:43.421 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-26T21:38:43.475 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-26T21:38:43.495 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-26T21:38:43.525 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-26T21:38:43.559 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-26T21:38:43.634 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-26T21:38:43.666 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-26T21:38:43.698 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-26T21:38:43.708 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-26T21:38:43.843 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-26T21:38:43.913 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-26T21:38:43.933 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-26T21:38:43.971 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-26T21:38:43.972 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-26T21:38:44.045 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-26T21:38:44.062 INFO:teuthology.orchestra.run.smithi072.stdout:(67/152): postgresql-13.14-1.el9.x86_64.rpm 648 kB/s | 1.6 MB 00:02 2024-04-26T21:38:44.081 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-26T21:38:44.198 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-26T21:38:44.267 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-26T21:38:44.900 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-26T21:38:45.055 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-26T21:38:45.242 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-26T21:38:45.317 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-26T21:38:45.358 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-26T21:38:45.484 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-26T21:38:45.552 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-26T21:38:45.653 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-26T21:38:45.679 INFO:teuthology.orchestra.run.smithi072.stdout:(68/152): postgresql-contrib-13.14-1.el9.x86_64 564 kB/s | 911 kB 00:01 2024-04-26T21:38:45.692 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-26T21:38:45.918 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-26T21:38:45.947 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-26T21:38:46.054 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-26T21:38:46.431 INFO:teuthology.orchestra.run.smithi072.stdout:(69/152): postgresql-private-libs-13.14-1.el9.x 178 kB/s | 133 kB 00:00 2024-04-26T21:38:46.667 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-26T21:38:46.775 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-26T21:38:46.824 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-26T21:38:46.869 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-26T21:38:46.880 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-26T21:38:47.027 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-26T21:38:47.198 INFO:teuthology.orchestra.run.smithi072.stdout:(70/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 793 kB/s | 4.4 MB 00:05 2024-04-26T21:38:47.283 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-26T21:38:47.354 INFO:teuthology.orchestra.run.smithi076.stdout: Upgrading : librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 99/156 2024-04-26T21:38:47.529 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 99/156 2024-04-26T21:38:47.561 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-rbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 100/156 2024-04-26T21:38:47.619 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-26T21:38:47.626 INFO:teuthology.orchestra.run.smithi072.stdout:(71/152): openblas-openmp-0.3.21-2.el9.x86_64.r 785 kB/s | 4.7 MB 00:06 2024-04-26T21:38:48.202 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 102/156 2024-04-26T21:38:48.232 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 103/156 2024-04-26T21:38:48.316 INFO:teuthology.orchestra.run.smithi072.stdout:(72/152): python3-attrs-20.3.0-7.el9.noarch.rpm 127 kB/s | 87 kB 00:00 2024-04-26T21:38:48.360 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 103/156 2024-04-26T21:38:48.385 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-rgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 104/156 2024-04-26T21:38:48.647 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-26T21:38:48.745 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-26T21:38:48.821 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-26T21:38:48.866 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-26T21:38:49.091 INFO:teuthology.orchestra.run.smithi072.stdout:(73/152): protobuf-3.14.0-13.el9.x86_64.rpm 546 kB/s | 1.0 MB 00:01 2024-04-26T21:38:49.262 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-26T21:38:49.299 INFO:teuthology.orchestra.run.smithi164.stdout: Upgrading : librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 99/156 2024-04-26T21:38:49.412 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 99/156 2024-04-26T21:38:49.447 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-rbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 100/156 2024-04-26T21:38:49.533 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-26T21:38:49.934 INFO:teuthology.orchestra.run.smithi072.stdout:(74/152): python3-devel-3.9.18-3.el9.x86_64.rpm 290 kB/s | 244 kB 00:00 2024-04-26T21:38:50.094 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 102/156 2024-04-26T21:38:50.127 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 103/156 2024-04-26T21:38:50.244 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 103/156 2024-04-26T21:38:50.270 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-rgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 104/156 2024-04-26T21:38:50.809 INFO:teuthology.orchestra.run.smithi072.stdout:(75/152): python3-jinja2-2.11.3-5.el9.noarch.rp 284 kB/s | 248 kB 00:00 2024-04-26T21:38:51.141 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 105/156 2024-04-26T21:38:51.182 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 105/156 2024-04-26T21:38:51.527 INFO:teuthology.orchestra.run.smithi072.stdout:(76/152): python3-jsonpatch-1.21-16.el9.noarch. 36 kB/s | 26 kB 00:00 2024-04-26T21:38:51.833 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 105/156 2024-04-26T21:38:51.928 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 106/156 2024-04-26T21:38:52.102 INFO:teuthology.orchestra.run.smithi072.stdout:(77/152): python3-jsonpointer-2.0-4.el9.noarch. 33 kB/s | 19 kB 00:00 2024-04-26T21:38:52.154 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 106/156 2024-04-26T21:38:52.155 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-26T21:38:52.155 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-26T21:38:52.155 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-26T21:38:52.196 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 107/156 2024-04-26T21:38:52.869 INFO:teuthology.orchestra.run.smithi072.stdout:(78/152): python3-mako-1.1.4-6.el9.noarch.rpm 225 kB/s | 172 kB 00:00 2024-04-26T21:38:52.978 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 105/156 2024-04-26T21:38:53.025 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 105/156 2024-04-26T21:38:53.495 INFO:teuthology.orchestra.run.smithi072.stdout:(79/152): python3-markupsafe-1.1.1-12.el9.x86_6 56 kB/s | 35 kB 00:00 2024-04-26T21:38:53.677 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 105/156 2024-04-26T21:38:53.720 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 106/156 2024-04-26T21:38:53.904 INFO:teuthology.orchestra.run.smithi072.stdout:(80/152): postgresql-server-13.14-1.el9.x86_64. 792 kB/s | 5.8 MB 00:07 2024-04-26T21:38:53.928 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 106/156 2024-04-26T21:38:53.928 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-26T21:38:53.928 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-26T21:38:53.928 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-26T21:38:53.980 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 107/156 2024-04-26T21:38:54.813 INFO:teuthology.orchestra.run.smithi072.stdout:(81/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 277 kB/s | 250 kB 00:00 2024-04-26T21:38:55.621 INFO:teuthology.orchestra.run.smithi072.stdout:(82/152): python3-oauthlib-3.1.1-5.el9.noarch.r 275 kB/s | 222 kB 00:00 2024-04-26T21:38:56.023 INFO:teuthology.orchestra.run.smithi072.stdout:(83/152): python3-babel-2.9.1-2.el9.noarch.rpm 792 kB/s | 6.0 MB 00:07 2024-04-26T21:38:56.399 INFO:teuthology.orchestra.run.smithi072.stdout:(84/152): python3-packaging-20.9-5.el9.noarch.r 99 kB/s | 77 kB 00:00 2024-04-26T21:38:56.708 INFO:teuthology.orchestra.run.smithi072.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 62 kB/s | 42 kB 00:00 2024-04-26T21:38:57.417 INFO:teuthology.orchestra.run.smithi072.stdout:(86/152): python3-protobuf-3.14.0-13.el9.noarch 266 kB/s | 269 kB 00:01 2024-04-26T21:38:57.492 INFO:teuthology.orchestra.run.smithi072.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 203 kB/s | 159 kB 00:00 2024-04-26T21:38:58.201 INFO:teuthology.orchestra.run.smithi072.stdout:(88/152): python3-pytz-2021.1-5.el9.noarch.rpm 72 kB/s | 51 kB 00:00 2024-04-26T21:38:58.501 INFO:teuthology.orchestra.run.smithi072.stdout:(89/152): python3-pyasn1-modules-0.4.8-6.el9.no 257 kB/s | 279 kB 00:01 2024-04-26T21:38:58.827 INFO:teuthology.orchestra.run.smithi072.stdout:(90/152): python3-requests-oauthlib-1.3.0-12.el 86 kB/s | 54 kB 00:00 2024-04-26T21:38:59.578 INFO:teuthology.orchestra.run.smithi072.stdout:(91/152): python3-toml-0.10.2-6.el9.noarch.rpm 56 kB/s | 42 kB 00:00 2024-04-26T21:39:00.154 INFO:teuthology.orchestra.run.smithi072.stdout:(92/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 784 kB/s | 5.1 MB 00:06 2024-04-26T21:39:00.212 INFO:teuthology.orchestra.run.smithi072.stdout:(93/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 70 kB/s | 44 kB 00:00 2024-04-26T21:39:00.829 INFO:teuthology.orchestra.run.smithi072.stdout:(94/152): qatlib-service-24.02.0-1.el9.x86_64.r 58 kB/s | 36 kB 00:00 2024-04-26T21:39:01.021 INFO:teuthology.orchestra.run.smithi072.stdout:(95/152): qatlib-24.02.0-1.el9.x86_64.rpm 255 kB/s | 221 kB 00:00 2024-04-26T21:39:01.472 INFO:teuthology.orchestra.run.smithi072.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 74 kB/s | 47 kB 00:00 2024-04-26T21:39:02.025 INFO:teuthology.orchestra.run.smithi072.stdout:(97/152): socat-1.7.4.1-5.el9.x86_64.rpm 304 kB/s | 305 kB 00:01 2024-04-26T21:39:02.117 INFO:teuthology.orchestra.run.smithi072.stdout:(98/152): uuid-1.6.2-55.el9.x86_64.rpm 90 kB/s | 58 kB 00:00 2024-04-26T21:39:02.734 INFO:teuthology.orchestra.run.smithi072.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 132 kB/s | 94 kB 00:00 2024-04-26T21:39:02.835 INFO:teuthology.orchestra.run.smithi072.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 89 kB/s | 64 kB 00:00 2024-04-26T21:39:03.110 INFO:teuthology.orchestra.run.smithi072.stdout:(101/152): lua-devel-5.4.4-4.el9.x86_64.rpm 81 kB/s | 22 kB 00:00 2024-04-26T21:39:03.403 INFO:teuthology.orchestra.run.smithi072.stdout:(102/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 67 kB/s | 45 kB 00:00 2024-04-26T21:39:03.586 INFO:teuthology.orchestra.run.smithi072.stdout:(103/152): protobuf-compiler-3.14.0-13.el9.x86_ 1.8 MB/s | 863 kB 00:00 2024-04-26T21:39:03.979 INFO:teuthology.orchestra.run.smithi072.stdout:(104/152): abseil-cpp-20211102.0-3.el9.x86_64.r 955 kB/s | 548 kB 00:00 2024-04-26T21:39:04.054 INFO:teuthology.orchestra.run.smithi072.stdout:(105/152): grpc-data-1.46.7-10.el9.noarch.rpm 261 kB/s | 19 kB 00:00 2024-04-26T21:39:04.088 INFO:teuthology.orchestra.run.smithi072.stdout:(106/152): gperftools-libs-2.9.1-2.el9.x86_64.r 620 kB/s | 309 kB 00:00 2024-04-26T21:39:04.214 INFO:teuthology.orchestra.run.smithi072.stdout:(107/152): libarrow-doc-9.0.0-11.el9.noarch.rpm 208 kB/s | 26 kB 00:00 2024-04-26T21:39:04.298 INFO:teuthology.orchestra.run.smithi072.stdout:(108/152): liboath-2.6.7-2.el9.x86_64.rpm 586 kB/s | 49 kB 00:00 2024-04-26T21:39:04.439 INFO:teuthology.orchestra.run.smithi072.stdout:(109/152): libarrow-9.0.0-11.el9.x86_64.rpm 11 MB/s | 4.4 MB 00:00 2024-04-26T21:39:04.475 INFO:teuthology.orchestra.run.smithi072.stdout:(110/152): libunwind-1.6.2-1.el9.x86_64.rpm 382 kB/s | 67 kB 00:00 2024-04-26T21:39:04.559 INFO:teuthology.orchestra.run.smithi072.stdout:(111/152): luarocks-3.9.2-1.el9.noarch.rpm 1.2 MB/s | 151 kB 00:00 2024-04-26T21:39:04.660 INFO:teuthology.orchestra.run.smithi072.stdout:(112/152): python3-asyncssh-2.13.2-5.el9.noarch 5.4 MB/s | 548 kB 00:00 2024-04-26T21:39:04.710 INFO:teuthology.orchestra.run.smithi072.stdout:(113/152): parquet-libs-9.0.0-11.el9.x86_64.rpm 3.6 MB/s | 839 kB 00:00 2024-04-26T21:39:04.736 INFO:teuthology.orchestra.run.smithi072.stdout:(114/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 580 kB/s | 43 kB 00:00 2024-04-26T21:39:04.786 INFO:teuthology.orchestra.run.smithi072.stdout:(115/152): python3-cachetools-4.2.4-1.el9.noarc 428 kB/s | 32 kB 00:00 2024-04-26T21:39:04.845 INFO:teuthology.orchestra.run.smithi072.stdout:(116/152): python3-certifi-2021.10.8-2.el9.noar 138 kB/s | 15 kB 00:00 2024-04-26T21:39:04.879 INFO:teuthology.orchestra.run.smithi072.stdout:(117/152): python3-cheroot-8.6.0-4.el9.noarch.r 1.8 MB/s | 172 kB 00:00 2024-04-26T21:39:04.947 INFO:teuthology.orchestra.run.smithi072.stdout:(118/152): python3-cherrypy-18.6.1-2.el9.noarch 3.5 MB/s | 358 kB 00:00 2024-04-26T21:39:04.989 INFO:teuthology.orchestra.run.smithi072.stdout:(119/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.7 MB/s | 303 kB 00:00 2024-04-26T21:39:05.022 INFO:teuthology.orchestra.run.smithi072.stdout:(120/152): python3-colorama-0.4.4-12.el9.noarch 471 kB/s | 35 kB 00:00 2024-04-26T21:39:05.073 INFO:teuthology.orchestra.run.smithi072.stdout:(121/152): python3-google-auth-2.29.0-1.el9.noa 2.4 MB/s | 206 kB 00:00 2024-04-26T21:39:05.157 INFO:teuthology.orchestra.run.smithi072.stdout:(122/152): python3-grpcio-tools-1.46.7-10.el9.x 1.7 MB/s | 144 kB 00:00 2024-04-26T21:39:05.230 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 107/156 2024-04-26T21:39:05.230 INFO:teuthology.orchestra.run.smithi076.stdout:skipping the directory /sys 2024-04-26T21:39:05.230 INFO:teuthology.orchestra.run.smithi076.stdout:skipping the directory /proc 2024-04-26T21:39:05.230 INFO:teuthology.orchestra.run.smithi076.stdout:skipping the directory /mnt 2024-04-26T21:39:05.230 INFO:teuthology.orchestra.run.smithi076.stdout:skipping the directory /var/tmp 2024-04-26T21:39:05.230 INFO:teuthology.orchestra.run.smithi076.stdout:skipping the directory /home 2024-04-26T21:39:05.230 INFO:teuthology.orchestra.run.smithi076.stdout:skipping the directory /root 2024-04-26T21:39:05.230 INFO:teuthology.orchestra.run.smithi076.stdout:skipping the directory /tmp 2024-04-26T21:39:05.230 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-26T21:39:05.232 INFO:teuthology.orchestra.run.smithi072.stdout:(123/152): python3-grpcio-1.46.7-10.el9.x86_64. 9.8 MB/s | 2.0 MB 00:00 2024-04-26T21:39:05.258 INFO:teuthology.orchestra.run.smithi072.stdout:(124/152): python3-jaraco-8.2.1-3.el9.noarch.rp 107 kB/s | 11 kB 00:00 2024-04-26T21:39:05.310 INFO:teuthology.orchestra.run.smithi072.stdout:(125/152): python3-jaraco-classes-3.2.1-5.el9.n 231 kB/s | 18 kB 00:00 2024-04-26T21:39:05.335 INFO:teuthology.orchestra.run.smithi072.stdout:(126/152): python3-jaraco-collections-3.0.0-8.e 302 kB/s | 23 kB 00:00 2024-04-26T21:39:05.386 INFO:teuthology.orchestra.run.smithi072.stdout:(127/152): python3-jaraco-functools-3.5.0-2.el9 260 kB/s | 19 kB 00:00 2024-04-26T21:39:05.412 INFO:teuthology.orchestra.run.smithi072.stdout:(128/152): python3-jaraco-text-3.2.0-6.el9.noar 261 kB/s | 20 kB 00:00 2024-04-26T21:39:05.494 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 108/156 2024-04-26T21:39:05.520 INFO:teuthology.orchestra.run.smithi072.stdout:(129/152): python3-kubernetes-26.1.0-2.el9.noar 7.7 MB/s | 1.0 MB 00:00 2024-04-26T21:39:05.541 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 108/156 2024-04-26T21:39:05.541 INFO:teuthology.orchestra.run.smithi076.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:39:05.541 INFO:teuthology.orchestra.run.smithi076.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-26T21:39:05.541 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-26T21:39:05.546 INFO:teuthology.orchestra.run.smithi072.stdout:(130/152): python3-logutils-0.3.5-21.el9.noarch 347 kB/s | 46 kB 00:00 2024-04-26T21:39:05.597 INFO:teuthology.orchestra.run.smithi072.stdout:(131/152): python3-more-itertools-8.12.0-2.el9. 1.0 MB/s | 79 kB 00:00 2024-04-26T21:39:05.622 INFO:teuthology.orchestra.run.smithi072.stdout:(132/152): python3-natsort-7.1.1-5.el9.noarch.r 767 kB/s | 58 kB 00:00 2024-04-26T21:39:05.690 INFO:teuthology.orchestra.run.smithi072.stdout:(133/152): python3-pecan-1.4.2-3.el9.noarch.rpm 2.9 MB/s | 272 kB 00:00 2024-04-26T21:39:05.715 INFO:teuthology.orchestra.run.smithi072.stdout:(134/152): python3-portend-3.1.0-2.el9.noarch.r 178 kB/s | 16 kB 00:00 2024-04-26T21:39:05.774 INFO:teuthology.orchestra.run.smithi072.stdout:(135/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.1 MB/s | 90 kB 00:00 2024-04-26T21:39:05.799 INFO:teuthology.orchestra.run.smithi072.stdout:(136/152): python3-pyperclip-1.8.2-2.el9.noarch 338 kB/s | 28 kB 00:00 2024-04-26T21:39:05.859 INFO:teuthology.orchestra.run.smithi072.stdout:(137/152): python3-repoze-lru-0.7-16.el9.noarch 368 kB/s | 31 kB 00:00 2024-04-26T21:39:05.892 INFO:teuthology.orchestra.run.smithi072.stdout:(138/152): python3-routes-2.5.1-5.el9.noarch.rp 2.0 MB/s | 188 kB 00:00 2024-04-26T21:39:05.935 INFO:teuthology.orchestra.run.smithi072.stdout:(139/152): python3-rsa-4.9-2.el9.noarch.rpm 788 kB/s | 59 kB 00:00 2024-04-26T21:39:05.970 INFO:teuthology.orchestra.run.smithi072.stdout:(140/152): python3-tempora-5.0.0-2.el9.noarch.r 465 kB/s | 36 kB 00:00 2024-04-26T21:39:06.012 INFO:teuthology.orchestra.run.smithi072.stdout:(141/152): python3-typing-extensions-4.4.0-2.el 664 kB/s | 51 kB 00:00 2024-04-26T21:39:06.054 INFO:teuthology.orchestra.run.smithi072.stdout:(142/152): python3-webob-1.8.7-6.el9.noarch.rpm 2.7 MB/s | 230 kB 00:00 2024-04-26T21:39:06.122 INFO:teuthology.orchestra.run.smithi072.stdout:(143/152): python3-websocket-client-1.2.3-2.el9 825 kB/s | 90 kB 00:00 2024-04-26T21:39:06.164 INFO:teuthology.orchestra.run.smithi072.stdout:(144/152): python3-werkzeug-2.0.3-3.el9.1.noarc 3.8 MB/s | 427 kB 00:00 2024-04-26T21:39:06.197 INFO:teuthology.orchestra.run.smithi072.stdout:(145/152): python3-zc-lockfile-2.0-10.el9.noarc 266 kB/s | 20 kB 00:00 2024-04-26T21:39:06.248 INFO:teuthology.orchestra.run.smithi072.stdout:(146/152): re2-20211101-3.el9.x86_64.rpm 2.2 MB/s | 192 kB 00:00 2024-04-26T21:39:06.332 INFO:teuthology.orchestra.run.smithi072.stdout:(147/152): thrift-0.15.0-2.el9.x86_64.rpm 12 MB/s | 1.6 MB 00:00 2024-04-26T21:39:06.702 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 107/156 2024-04-26T21:39:06.703 INFO:teuthology.orchestra.run.smithi164.stdout:skipping the directory /sys 2024-04-26T21:39:06.703 INFO:teuthology.orchestra.run.smithi164.stdout:skipping the directory /proc 2024-04-26T21:39:06.703 INFO:teuthology.orchestra.run.smithi164.stdout:skipping the directory /mnt 2024-04-26T21:39:06.703 INFO:teuthology.orchestra.run.smithi164.stdout:skipping the directory /var/tmp 2024-04-26T21:39:06.703 INFO:teuthology.orchestra.run.smithi164.stdout:skipping the directory /home 2024-04-26T21:39:06.703 INFO:teuthology.orchestra.run.smithi164.stdout:skipping the directory /root 2024-04-26T21:39:06.703 INFO:teuthology.orchestra.run.smithi164.stdout:skipping the directory /tmp 2024-04-26T21:39:06.703 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-26T21:39:06.926 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 108/156 2024-04-26T21:39:06.973 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 108/156 2024-04-26T21:39:06.974 INFO:teuthology.orchestra.run.smithi164.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:39:06.974 INFO:teuthology.orchestra.run.smithi164.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-26T21:39:06.974 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-26T21:39:07.212 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 109/156 2024-04-26T21:39:07.241 INFO:teuthology.orchestra.run.smithi072.stdout:(148/152): librados2-19.0.0-3306.gbc612ffb.el9. 3.4 MB/s | 3.4 MB 00:00 2024-04-26T21:39:07.293 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 109/156 2024-04-26T21:39:07.294 INFO:teuthology.orchestra.run.smithi076.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:39:07.294 INFO:teuthology.orchestra.run.smithi076.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-26T21:39:07.294 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-26T21:39:07.294 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-26T21:39:07.294 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-26T21:39:07.467 INFO:teuthology.orchestra.run.smithi072.stdout:(149/152): librbd1-19.0.0-3306.gbc612ffb.el9.x8 2.8 MB/s | 3.1 MB 00:01 2024-04-26T21:39:07.525 INFO:teuthology.orchestra.run.smithi072.stdout:(150/152): openssl-3.2.1-1.el9.x86_64.rpm 4.8 MB/s | 1.3 MB 00:00 2024-04-26T21:39:07.578 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 110/156 2024-04-26T21:39:07.647 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 110/156 2024-04-26T21:39:07.647 INFO:teuthology.orchestra.run.smithi076.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:39:07.647 INFO:teuthology.orchestra.run.smithi076.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-26T21:39:07.647 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-26T21:39:07.647 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-26T21:39:07.648 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-26T21:39:07.726 INFO:teuthology.orchestra.run.smithi072.stdout:(151/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 9.3 MB/s | 2.4 MB 00:00 2024-04-26T21:39:08.138 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 111/156 2024-04-26T21:39:08.208 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 111/156 2024-04-26T21:39:08.208 INFO:teuthology.orchestra.run.smithi076.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:39:08.208 INFO:teuthology.orchestra.run.smithi076.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-26T21:39:08.208 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-26T21:39:08.208 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-26T21:39:08.208 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-26T21:39:08.339 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-26T21:39:08.432 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-26T21:39:08.512 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-26T21:39:08.595 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 109/156 2024-04-26T21:39:08.673 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 109/156 2024-04-26T21:39:08.673 INFO:teuthology.orchestra.run.smithi164.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:39:08.673 INFO:teuthology.orchestra.run.smithi164.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-26T21:39:08.673 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-26T21:39:08.673 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-26T21:39:08.674 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-26T21:39:08.745 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-26T21:39:08.821 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-26T21:39:08.912 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-26T21:39:08.928 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 110/156 2024-04-26T21:39:08.997 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 110/156 2024-04-26T21:39:08.997 INFO:teuthology.orchestra.run.smithi164.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:39:08.997 INFO:teuthology.orchestra.run.smithi164.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-26T21:39:08.997 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-26T21:39:08.997 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-26T21:39:08.997 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-26T21:39:09.023 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-26T21:39:09.188 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-26T21:39:09.261 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-26T21:39:09.303 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-26T21:39:09.343 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-26T21:39:09.479 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 111/156 2024-04-26T21:39:09.548 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 111/156 2024-04-26T21:39:09.548 INFO:teuthology.orchestra.run.smithi164.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:39:09.548 INFO:teuthology.orchestra.run.smithi164.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-26T21:39:09.548 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-26T21:39:09.548 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-26T21:39:09.548 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-26T21:39:09.679 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-26T21:39:09.742 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-26T21:39:09.816 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-26T21:39:09.930 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 123/156 2024-04-26T21:39:09.981 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 123/156 2024-04-26T21:39:10.041 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3306.gbc612ffb.e 124/156 2024-04-26T21:39:10.082 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-26T21:39:10.149 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-26T21:39:10.157 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3306.gbc612ffb. 125/156 2024-04-26T21:39:10.191 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noa 126/156 2024-04-26T21:39:10.235 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-26T21:39:10.344 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-26T21:39:10.509 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-26T21:39:10.596 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-26T21:39:10.640 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-26T21:39:10.673 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-26T21:39:11.285 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 123/156 2024-04-26T21:39:11.334 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 123/156 2024-04-26T21:39:11.389 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3306.gbc612ffb.e 124/156 2024-04-26T21:39:11.465 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noa 126/156 2024-04-26T21:39:11.500 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3306.gbc612ffb. 125/156 2024-04-26T21:39:11.518 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.n 127/156 2024-04-26T21:39:11.536 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noa 126/156 2024-04-26T21:39:12.618 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.n 127/156 2024-04-26T21:39:12.651 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc6 128/156 2024-04-26T21:39:12.799 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc6 128/156 2024-04-26T21:39:12.859 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noa 126/156 2024-04-26T21:39:12.905 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.n 127/156 2024-04-26T21:39:12.974 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3306.gbc612ffb.el 129/156 2024-04-26T21:39:13.004 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 130/156 2024-04-26T21:39:13.073 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 130/156 2024-04-26T21:39:13.074 INFO:teuthology.orchestra.run.smithi076.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:39:13.074 INFO:teuthology.orchestra.run.smithi076.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-26T21:39:13.074 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-26T21:39:13.074 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-26T21:39:13.074 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-26T21:39:13.138 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 131/156 2024-04-26T21:39:13.177 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 131/156 2024-04-26T21:39:13.288 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-2:19.0.0-3306.gbc612ffb.el9.x86_64 132/156 2024-04-26T21:39:13.358 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 133/156 2024-04-26T21:39:14.012 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.n 127/156 2024-04-26T21:39:14.055 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc6 128/156 2024-04-26T21:39:14.200 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc6 128/156 2024-04-26T21:39:14.350 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-26T21:39:14.369 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3306.gbc612ffb.el 129/156 2024-04-26T21:39:14.400 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 135/156 2024-04-26T21:39:14.406 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 130/156 2024-04-26T21:39:14.468 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 135/156 2024-04-26T21:39:14.468 INFO:teuthology.orchestra.run.smithi076.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:39:14.468 INFO:teuthology.orchestra.run.smithi076.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-26T21:39:14.468 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-26T21:39:14.468 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-26T21:39:14.468 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-26T21:39:14.476 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 130/156 2024-04-26T21:39:14.476 INFO:teuthology.orchestra.run.smithi164.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:39:14.477 INFO:teuthology.orchestra.run.smithi164.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-26T21:39:14.477 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-26T21:39:14.477 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-26T21:39:14.477 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-26T21:39:14.541 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 131/156 2024-04-26T21:39:14.542 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3306.gbc612 136/156 2024-04-26T21:39:14.577 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 131/156 2024-04-26T21:39:14.610 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3306.gbc612 136/156 2024-04-26T21:39:14.610 INFO:teuthology.orchestra.run.smithi076.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:39:14.610 INFO:teuthology.orchestra.run.smithi076.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-26T21:39:14.610 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-26T21:39:14.683 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-2:19.0.0-3306.gbc612ffb.el9.x86_64 132/156 2024-04-26T21:39:14.684 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 137/156 2024-04-26T21:39:14.760 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 137/156 2024-04-26T21:39:14.760 INFO:teuthology.orchestra.run.smithi076.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:39:14.760 INFO:teuthology.orchestra.run.smithi076.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-26T21:39:14.760 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-26T21:39:14.760 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-26T21:39:14.760 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-26T21:39:14.780 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 133/156 2024-04-26T21:39:15.127 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 138/156 2024-04-26T21:39:15.198 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 138/156 2024-04-26T21:39:15.198 INFO:teuthology.orchestra.run.smithi076.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:39:15.198 INFO:teuthology.orchestra.run.smithi076.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-26T21:39:15.198 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-26T21:39:15.199 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-26T21:39:15.199 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-26T21:39:15.795 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-26T21:39:15.835 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 135/156 2024-04-26T21:39:15.902 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 135/156 2024-04-26T21:39:15.902 INFO:teuthology.orchestra.run.smithi164.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:39:15.903 INFO:teuthology.orchestra.run.smithi164.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-26T21:39:15.903 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-26T21:39:15.903 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-26T21:39:15.903 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-26T21:39:15.953 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3306.gbc612 136/156 2024-04-26T21:39:16.018 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3306.gbc612 136/156 2024-04-26T21:39:16.019 INFO:teuthology.orchestra.run.smithi164.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:39:16.019 INFO:teuthology.orchestra.run.smithi164.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-26T21:39:16.019 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-26T21:39:16.136 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 137/156 2024-04-26T21:39:16.212 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 137/156 2024-04-26T21:39:16.212 INFO:teuthology.orchestra.run.smithi164.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:39:16.212 INFO:teuthology.orchestra.run.smithi164.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-26T21:39:16.212 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-26T21:39:16.212 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-26T21:39:16.212 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-26T21:39:16.562 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 138/156 2024-04-26T21:39:16.628 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 138/156 2024-04-26T21:39:16.628 INFO:teuthology.orchestra.run.smithi164.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:39:16.628 INFO:teuthology.orchestra.run.smithi164.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-26T21:39:16.628 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-26T21:39:16.628 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-26T21:39:16.628 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-26T21:39:17.959 INFO:teuthology.orchestra.run.smithi072.stdout:(152/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 855 kB/s | 16 MB 00:19 2024-04-26T21:39:17.972 INFO:teuthology.orchestra.run.smithi072.stdout:-------------------------------------------------------------------------------- 2024-04-26T21:39:17.973 INFO:teuthology.orchestra.run.smithi072.stdout:Total 4.3 MB/s | 222 MB 00:51 2024-04-26T21:39:20.232 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-test-2:19.0.0-3306.gbc612ffb.el9.x86_64 139/156 2024-04-26T21:39:20.314 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : rbd-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 140/156 2024-04-26T21:39:20.410 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : rbd-nbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 141/156 2024-04-26T21:39:20.462 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-26T21:39:20.510 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-26T21:39:20.786 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction check 2024-04-26T21:39:20.845 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : cephfs-shell-2:19.0.0-3306.gbc612ffb.el9.x86_64 144/156 2024-04-26T21:39:20.903 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-26T21:39:20.931 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : libcephfs-devel-2:19.0.0-3306.gbc612ffb.el9.x86_ 146/156 2024-04-26T21:39:21.026 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction check succeeded. 2024-04-26T21:39:21.026 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction test 2024-04-26T21:39:21.697 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-test-2:19.0.0-3306.gbc612ffb.el9.x86_64 139/156 2024-04-26T21:39:21.755 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : rbd-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 140/156 2024-04-26T21:39:21.858 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : rbd-nbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 141/156 2024-04-26T21:39:21.901 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-26T21:39:21.950 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-26T21:39:22.024 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T21:39:22.080 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T21:39:22.142 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T21:39:22.295 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : cephfs-shell-2:19.0.0-3306.gbc612ffb.el9.x86_64 144/156 2024-04-26T21:39:22.374 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-26T21:39:22.402 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libcephfs-devel-2:19.0.0-3306.gbc612ffb.el9.x86_ 146/156 2024-04-26T21:39:23.216 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T21:39:23.267 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T21:39:23.330 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T21:39:23.476 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : cephfs-top-2:19.0.0-3306.gbc612ffb.el9.noarch 148/156 2024-04-26T21:39:23.597 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-26T21:39:23.674 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-26T21:39:23.839 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-26T21:39:23.887 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-26T21:39:23.887 INFO:teuthology.orchestra.run.smithi076.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-26T21:39:23.938 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction test succeeded. 2024-04-26T21:39:23.939 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction 2024-04-26T21:39:24.030 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-26T21:39:24.069 INFO:teuthology.orchestra.run.smithi076.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-26T21:39:24.069 INFO:teuthology.orchestra.run.smithi076.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-26T21:39:24.210 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-26T21:39:24.339 INFO:teuthology.orchestra.run.smithi076.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T21:39:24.631 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T21:39:24.631 INFO:teuthology.orchestra.run.smithi076.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-04-26T21:39:24.631 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-26T21:39:24.718 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : cephfs-top-2:19.0.0-3306.gbc612ffb.el9.noarch 148/156 2024-04-26T21:39:24.824 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-26T21:39:24.906 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-26T21:39:25.060 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-26T21:39:25.128 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-26T21:39:25.128 INFO:teuthology.orchestra.run.smithi164.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-26T21:39:25.255 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-26T21:39:25.303 INFO:teuthology.orchestra.run.smithi164.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-26T21:39:25.303 INFO:teuthology.orchestra.run.smithi164.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-26T21:39:25.444 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-26T21:39:25.594 INFO:teuthology.orchestra.run.smithi164.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T21:39:25.872 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T21:39:25.873 INFO:teuthology.orchestra.run.smithi164.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-04-26T21:39:25.873 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-26T21:39:30.388 INFO:teuthology.orchestra.run.smithi072.stdout: Preparing : 1/1 2024-04-26T21:39:30.604 INFO:teuthology.orchestra.run.smithi072.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-26T21:39:30.706 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-26T21:39:30.716 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-2:19.0.0-3306.gbc612ffb.el9.x86_64 1/156 2024-04-26T21:39:30.716 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 2/156 2024-04-26T21:39:30.716 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 3/156 2024-04-26T21:39:30.716 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 4/156 2024-04-26T21:39:30.716 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3306.gbc612 5/156 2024-04-26T21:39:30.716 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 6/156 2024-04-26T21:39:30.716 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 7/156 2024-04-26T21:39:30.716 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 8/156 2024-04-26T21:39:30.716 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 9/156 2024-04-26T21:39:30.716 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 10/156 2024-04-26T21:39:30.716 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 11/156 2024-04-26T21:39:30.716 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-test-2:19.0.0-3306.gbc612ffb.el9.x86_64 12/156 2024-04-26T21:39:30.717 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 13/156 2024-04-26T21:39:30.717 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : cephfs-shell-2:19.0.0-3306.gbc612ffb.el9.x86_64 14/156 2024-04-26T21:39:30.717 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libcephfs-devel-2:19.0.0-3306.gbc612ffb.el9.x86_ 15/156 2024-04-26T21:39:30.717 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 16/156 2024-04-26T21:39:30.717 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 17/156 2024-04-26T21:39:30.717 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : librados-devel-2:19.0.0-3306.gbc612ffb.el9.x86_6 18/156 2024-04-26T21:39:30.717 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86 19/156 2024-04-26T21:39:30.717 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 20/156 2024-04-26T21:39:30.717 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3306.gbc612ffb.el 21/156 2024-04-26T21:39:30.717 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-ceph-common-2:19.0.0-3306.gbc612ffb.el9. 22/156 2024-04-26T21:39:30.717 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-cephfs-2:19.0.0-3306.gbc612ffb.el9.x86_6 23/156 2024-04-26T21:39:30.717 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-rados-2:19.0.0-3306.gbc612ffb.el9.x86_64 24/156 2024-04-26T21:39:30.718 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-rbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 25/156 2024-04-26T21:39:30.718 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-rgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 26/156 2024-04-26T21:39:30.718 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : rbd-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 27/156 2024-04-26T21:39:30.718 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 28/156 2024-04-26T21:39:30.718 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : rbd-nbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 29/156 2024-04-26T21:39:30.718 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3306.gbc612ffb. 30/156 2024-04-26T21:39:30.718 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noa 31/156 2024-04-26T21:39:30.718 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.n 32/156 2024-04-26T21:39:30.718 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc6 33/156 2024-04-26T21:39:30.718 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3306.gbc612ffb.el 34/156 2024-04-26T21:39:30.718 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 35/156 2024-04-26T21:39:30.718 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3306.gbc612ffb.e 36/156 2024-04-26T21:39:30.718 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 37/156 2024-04-26T21:39:30.719 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 38/156 2024-04-26T21:39:30.719 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : cephfs-top-2:19.0.0-3306.gbc612ffb.el9.noarch 39/156 2024-04-26T21:39:30.719 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-26T21:39:30.719 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-26T21:39:30.719 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-26T21:39:30.719 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-26T21:39:30.719 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-26T21:39:30.719 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-26T21:39:30.719 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-26T21:39:30.719 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-26T21:39:30.719 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-26T21:39:30.719 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-26T21:39:30.719 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-26T21:39:30.720 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-26T21:39:30.720 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-26T21:39:30.720 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-26T21:39:30.720 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-26T21:39:30.720 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-26T21:39:30.720 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-26T21:39:30.720 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-26T21:39:30.720 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-26T21:39:30.720 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-26T21:39:30.720 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-26T21:39:30.720 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-26T21:39:30.720 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-26T21:39:30.720 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-26T21:39:30.720 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-26T21:39:30.721 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-26T21:39:30.721 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-26T21:39:30.721 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-26T21:39:30.721 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-26T21:39:30.721 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-26T21:39:30.721 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-26T21:39:30.721 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-26T21:39:30.721 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-26T21:39:30.721 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-26T21:39:30.721 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-26T21:39:30.721 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-26T21:39:30.721 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-26T21:39:30.721 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-26T21:39:30.721 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-26T21:39:30.722 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-26T21:39:30.722 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-26T21:39:30.722 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-26T21:39:30.722 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-26T21:39:30.722 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-26T21:39:30.722 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-26T21:39:30.722 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-26T21:39:30.722 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-26T21:39:30.722 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-26T21:39:30.722 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-26T21:39:30.722 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-26T21:39:30.722 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-26T21:39:30.722 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-26T21:39:30.723 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-26T21:39:30.723 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-26T21:39:30.723 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-26T21:39:30.723 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-26T21:39:30.723 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-26T21:39:30.723 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-26T21:39:30.723 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-26T21:39:30.723 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-26T21:39:30.723 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-26T21:39:30.723 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-26T21:39:30.723 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-26T21:39:30.723 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-26T21:39:30.723 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-26T21:39:30.724 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-26T21:39:30.724 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-26T21:39:30.724 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-26T21:39:30.724 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/156 2024-04-26T21:39:30.724 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/156 2024-04-26T21:39:30.724 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-26T21:39:30.724 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-26T21:39:30.724 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-26T21:39:30.724 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/156 2024-04-26T21:39:30.724 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-26T21:39:30.724 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-26T21:39:30.724 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-26T21:39:30.724 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-26T21:39:30.724 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-26T21:39:30.725 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-26T21:39:30.725 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-26T21:39:30.725 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-26T21:39:30.725 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-26T21:39:30.725 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-26T21:39:30.725 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-26T21:39:30.725 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-26T21:39:30.725 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-26T21:39:30.725 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-26T21:39:30.725 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-26T21:39:30.725 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-26T21:39:30.725 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-26T21:39:30.726 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-26T21:39:30.726 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-26T21:39:30.726 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-26T21:39:30.726 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-26T21:39:30.726 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-26T21:39:30.726 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-26T21:39:30.726 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-26T21:39:30.726 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-26T21:39:30.726 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-26T21:39:30.726 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-26T21:39:30.726 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-26T21:39:30.726 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-26T21:39:30.726 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-26T21:39:30.727 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-26T21:39:30.727 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-26T21:39:30.727 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-26T21:39:30.727 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-26T21:39:30.727 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-26T21:39:30.727 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 149/156 2024-04-26T21:39:30.727 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-26T21:39:30.727 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 151/156 2024-04-26T21:39:30.727 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-26T21:39:30.727 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-26T21:39:30.727 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-26T21:39:30.727 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-26T21:39:30.836 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-26T21:39:30.897 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-26T21:39:30.933 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-26T21:39:31.087 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-26T21:39:31.520 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-26T21:39:31.577 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-2:19.0.0-3306.gbc612ffb.el9.x86_64 1/156 2024-04-26T21:39:31.578 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 2/156 2024-04-26T21:39:31.578 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 3/156 2024-04-26T21:39:31.578 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 4/156 2024-04-26T21:39:31.578 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3306.gbc612 5/156 2024-04-26T21:39:31.578 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 6/156 2024-04-26T21:39:31.578 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 7/156 2024-04-26T21:39:31.578 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 8/156 2024-04-26T21:39:31.578 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 9/156 2024-04-26T21:39:31.578 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 10/156 2024-04-26T21:39:31.578 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 11/156 2024-04-26T21:39:31.578 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-test-2:19.0.0-3306.gbc612ffb.el9.x86_64 12/156 2024-04-26T21:39:31.578 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 13/156 2024-04-26T21:39:31.578 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : cephfs-shell-2:19.0.0-3306.gbc612ffb.el9.x86_64 14/156 2024-04-26T21:39:31.579 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libcephfs-devel-2:19.0.0-3306.gbc612ffb.el9.x86_ 15/156 2024-04-26T21:39:31.579 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 16/156 2024-04-26T21:39:31.579 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 17/156 2024-04-26T21:39:31.579 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : librados-devel-2:19.0.0-3306.gbc612ffb.el9.x86_6 18/156 2024-04-26T21:39:31.579 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86 19/156 2024-04-26T21:39:31.579 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 20/156 2024-04-26T21:39:31.579 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3306.gbc612ffb.el 21/156 2024-04-26T21:39:31.579 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-ceph-common-2:19.0.0-3306.gbc612ffb.el9. 22/156 2024-04-26T21:39:31.579 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-cephfs-2:19.0.0-3306.gbc612ffb.el9.x86_6 23/156 2024-04-26T21:39:31.580 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-rados-2:19.0.0-3306.gbc612ffb.el9.x86_64 24/156 2024-04-26T21:39:31.580 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-rbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 25/156 2024-04-26T21:39:31.580 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-rgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 26/156 2024-04-26T21:39:31.580 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : rbd-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 27/156 2024-04-26T21:39:31.580 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 28/156 2024-04-26T21:39:31.593 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : rbd-nbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 29/156 2024-04-26T21:39:31.593 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3306.gbc612ffb. 30/156 2024-04-26T21:39:31.593 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noa 31/156 2024-04-26T21:39:31.593 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.n 32/156 2024-04-26T21:39:31.594 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc6 33/156 2024-04-26T21:39:31.594 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3306.gbc612ffb.el 34/156 2024-04-26T21:39:31.594 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 35/156 2024-04-26T21:39:31.594 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3306.gbc612ffb.e 36/156 2024-04-26T21:39:31.594 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 37/156 2024-04-26T21:39:31.594 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 38/156 2024-04-26T21:39:31.594 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : cephfs-top-2:19.0.0-3306.gbc612ffb.el9.noarch 39/156 2024-04-26T21:39:31.594 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-26T21:39:31.594 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-26T21:39:31.594 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-26T21:39:31.594 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-26T21:39:31.594 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-26T21:39:31.594 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-26T21:39:31.595 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-26T21:39:31.595 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-26T21:39:31.595 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-26T21:39:31.595 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-26T21:39:31.595 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-26T21:39:31.595 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-26T21:39:31.595 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-26T21:39:31.595 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-26T21:39:31.595 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-26T21:39:31.595 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-26T21:39:31.595 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-26T21:39:31.595 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-26T21:39:31.596 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-26T21:39:31.596 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-26T21:39:31.596 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-26T21:39:31.596 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-26T21:39:31.596 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-26T21:39:31.596 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-26T21:39:31.596 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-26T21:39:31.596 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-26T21:39:31.596 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-26T21:39:31.596 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-26T21:39:31.596 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-26T21:39:31.597 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-26T21:39:31.597 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-26T21:39:31.597 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-26T21:39:31.597 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-26T21:39:31.597 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-26T21:39:31.597 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-26T21:39:31.597 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-26T21:39:31.597 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-26T21:39:31.597 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-26T21:39:31.597 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-26T21:39:31.597 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-26T21:39:31.597 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-26T21:39:31.597 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-26T21:39:31.597 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-26T21:39:31.598 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-26T21:39:31.598 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-26T21:39:31.598 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-26T21:39:31.598 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-26T21:39:31.598 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-26T21:39:31.598 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-26T21:39:31.598 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-26T21:39:31.598 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-26T21:39:31.598 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-26T21:39:31.598 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-26T21:39:31.598 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-26T21:39:31.598 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-26T21:39:31.598 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-26T21:39:31.599 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-26T21:39:31.599 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-26T21:39:31.599 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-26T21:39:31.599 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-26T21:39:31.599 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-26T21:39:31.599 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-26T21:39:31.599 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-26T21:39:31.599 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-26T21:39:31.599 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-26T21:39:31.599 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-26T21:39:31.599 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-26T21:39:31.599 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-26T21:39:31.599 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/156 2024-04-26T21:39:31.600 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/156 2024-04-26T21:39:31.600 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-26T21:39:31.600 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-26T21:39:31.600 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-26T21:39:31.600 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/156 2024-04-26T21:39:31.600 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-26T21:39:31.600 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-26T21:39:31.600 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-26T21:39:31.601 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-26T21:39:31.601 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-26T21:39:31.601 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-26T21:39:31.601 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-26T21:39:31.601 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-26T21:39:31.601 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-26T21:39:31.601 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-26T21:39:31.601 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-26T21:39:31.602 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-26T21:39:31.602 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-26T21:39:31.602 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-26T21:39:31.602 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-26T21:39:31.602 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-26T21:39:31.602 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-26T21:39:31.602 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-26T21:39:31.603 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-26T21:39:31.603 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-26T21:39:31.603 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-26T21:39:31.603 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-26T21:39:31.603 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-26T21:39:31.603 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-26T21:39:31.603 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-26T21:39:31.603 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-26T21:39:31.603 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-26T21:39:31.603 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-26T21:39:31.603 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-26T21:39:31.603 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-26T21:39:31.604 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-26T21:39:31.604 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-26T21:39:31.604 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-26T21:39:31.604 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-26T21:39:31.604 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-26T21:39:31.604 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 149/156 2024-04-26T21:39:31.604 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-26T21:39:31.604 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 151/156 2024-04-26T21:39:31.604 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-26T21:39:31.604 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-26T21:39:31.604 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-26T21:39:31.604 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-26T21:39:31.725 INFO:teuthology.orchestra.run.smithi072.stdout: Upgrading : librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 8/156 2024-04-26T21:39:31.902 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 8/156 2024-04-26T21:39:31.935 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 9/156 2024-04-26T21:39:32.064 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 9/156 2024-04-26T21:39:32.112 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-rados-2:19.0.0-3306.gbc612ffb.el9.x86_64 10/156 2024-04-26T21:39:32.500 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-26T21:39:32.562 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-26T21:39:32.595 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 13/156 2024-04-26T21:39:32.758 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 13/156 2024-04-26T21:39:32.790 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86 14/156 2024-04-26T21:39:32.886 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86 14/156 2024-04-26T21:39:32.917 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-26T21:39:32.992 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-26T21:39:33.133 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T21:39:33.133 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-26T21:39:33.133 INFO:teuthology.orchestra.run.smithi076.stdout:Upgraded: 2024-04-26T21:39:33.133 INFO:teuthology.orchestra.run.smithi076.stdout: librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:39:33.133 INFO:teuthology.orchestra.run.smithi076.stdout: librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:39:33.133 INFO:teuthology.orchestra.run.smithi076.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-26T21:39:33.133 INFO:teuthology.orchestra.run.smithi076.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-26T21:39:33.134 INFO:teuthology.orchestra.run.smithi076.stdout:Installed: 2024-04-26T21:39:33.134 INFO:teuthology.orchestra.run.smithi076.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-26T21:39:33.134 INFO:teuthology.orchestra.run.smithi076.stdout: automake-1.16.2-8.el9.noarch 2024-04-26T21:39:33.134 INFO:teuthology.orchestra.run.smithi076.stdout: bison-3.7.4-5.el9.x86_64 2024-04-26T21:39:33.134 INFO:teuthology.orchestra.run.smithi076.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-26T21:39:33.134 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:39:33.134 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:39:33.134 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:39:33.134 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:39:33.134 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-grafana-dashboards-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:39:33.134 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-immutable-object-cache-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:39:33.134 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:39:33.135 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:39:33.135 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:39:33.135 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:39:33.135 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:39:33.135 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mgr-modules-core-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:39:33.135 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:39:33.135 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:39:33.135 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:39:33.135 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-prometheus-alerts-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:39:33.135 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:39:33.135 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:39:33.135 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-test-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:39:33.136 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:39:33.136 INFO:teuthology.orchestra.run.smithi076.stdout: cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:39:33.136 INFO:teuthology.orchestra.run.smithi076.stdout: cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:39:33.136 INFO:teuthology.orchestra.run.smithi076.stdout: cephfs-shell-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:39:33.136 INFO:teuthology.orchestra.run.smithi076.stdout: cephfs-top-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:39:33.136 INFO:teuthology.orchestra.run.smithi076.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-26T21:39:33.136 INFO:teuthology.orchestra.run.smithi076.stdout: flex-2.6.4-9.el9.x86_64 2024-04-26T21:39:33.136 INFO:teuthology.orchestra.run.smithi076.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-26T21:39:33.136 INFO:teuthology.orchestra.run.smithi076.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-26T21:39:33.136 INFO:teuthology.orchestra.run.smithi076.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-26T21:39:33.137 INFO:teuthology.orchestra.run.smithi076.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-26T21:39:33.137 INFO:teuthology.orchestra.run.smithi076.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-26T21:39:33.137 INFO:teuthology.orchestra.run.smithi076.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-26T21:39:33.137 INFO:teuthology.orchestra.run.smithi076.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-26T21:39:33.137 INFO:teuthology.orchestra.run.smithi076.stdout: libarrow-9.0.0-11.el9.x86_64 2024-04-26T21:39:33.137 INFO:teuthology.orchestra.run.smithi076.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-04-26T21:39:33.137 INFO:teuthology.orchestra.run.smithi076.stdout: libcephfs-devel-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:39:33.137 INFO:teuthology.orchestra.run.smithi076.stdout: libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:39:33.137 INFO:teuthology.orchestra.run.smithi076.stdout: libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:39:33.137 INFO:teuthology.orchestra.run.smithi076.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-26T21:39:33.137 INFO:teuthology.orchestra.run.smithi076.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-26T21:39:33.137 INFO:teuthology.orchestra.run.smithi076.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-26T21:39:33.138 INFO:teuthology.orchestra.run.smithi076.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-26T21:39:33.138 INFO:teuthology.orchestra.run.smithi076.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-26T21:39:33.138 INFO:teuthology.orchestra.run.smithi076.stdout: librados-devel-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:39:33.138 INFO:teuthology.orchestra.run.smithi076.stdout: libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:39:33.138 INFO:teuthology.orchestra.run.smithi076.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-26T21:39:33.138 INFO:teuthology.orchestra.run.smithi076.stdout: librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:39:33.138 INFO:teuthology.orchestra.run.smithi076.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-26T21:39:33.138 INFO:teuthology.orchestra.run.smithi076.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-26T21:39:33.138 INFO:teuthology.orchestra.run.smithi076.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-26T21:39:33.138 INFO:teuthology.orchestra.run.smithi076.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-26T21:39:33.138 INFO:teuthology.orchestra.run.smithi076.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-26T21:39:33.138 INFO:teuthology.orchestra.run.smithi076.stdout: lua-5.4.4-4.el9.x86_64 2024-04-26T21:39:33.139 INFO:teuthology.orchestra.run.smithi076.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-26T21:39:33.139 INFO:teuthology.orchestra.run.smithi076.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-26T21:39:33.139 INFO:teuthology.orchestra.run.smithi076.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-26T21:39:33.139 INFO:teuthology.orchestra.run.smithi076.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-26T21:39:33.139 INFO:teuthology.orchestra.run.smithi076.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-26T21:39:33.139 INFO:teuthology.orchestra.run.smithi076.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-04-26T21:39:33.139 INFO:teuthology.orchestra.run.smithi076.stdout: postgresql-13.14-1.el9.x86_64 2024-04-26T21:39:33.139 INFO:teuthology.orchestra.run.smithi076.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-26T21:39:33.139 INFO:teuthology.orchestra.run.smithi076.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-26T21:39:33.139 INFO:teuthology.orchestra.run.smithi076.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-26T21:39:33.139 INFO:teuthology.orchestra.run.smithi076.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-26T21:39:33.139 INFO:teuthology.orchestra.run.smithi076.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-26T21:39:33.139 INFO:teuthology.orchestra.run.smithi076.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-26T21:39:33.139 INFO:teuthology.orchestra.run.smithi076.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-26T21:39:33.140 INFO:teuthology.orchestra.run.smithi076.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-26T21:39:33.140 INFO:teuthology.orchestra.run.smithi076.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-26T21:39:33.140 INFO:teuthology.orchestra.run.smithi076.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-26T21:39:33.140 INFO:teuthology.orchestra.run.smithi076.stdout: python3-ceph-argparse-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:39:33.140 INFO:teuthology.orchestra.run.smithi076.stdout: python3-ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:39:33.140 INFO:teuthology.orchestra.run.smithi076.stdout: python3-cephfs-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:39:33.140 INFO:teuthology.orchestra.run.smithi076.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-26T21:39:33.140 INFO:teuthology.orchestra.run.smithi076.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-26T21:39:33.140 INFO:teuthology.orchestra.run.smithi076.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-26T21:39:33.140 INFO:teuthology.orchestra.run.smithi076.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-26T21:39:33.140 INFO:teuthology.orchestra.run.smithi076.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-26T21:39:33.140 INFO:teuthology.orchestra.run.smithi076.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-26T21:39:33.140 INFO:teuthology.orchestra.run.smithi076.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-26T21:39:33.140 INFO:teuthology.orchestra.run.smithi076.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-26T21:39:33.141 INFO:teuthology.orchestra.run.smithi076.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-26T21:39:33.141 INFO:teuthology.orchestra.run.smithi076.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-26T21:39:33.141 INFO:teuthology.orchestra.run.smithi076.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-26T21:39:33.141 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-26T21:39:33.141 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-26T21:39:33.141 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-26T21:39:33.141 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-26T21:39:33.141 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-26T21:39:33.141 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-26T21:39:33.141 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-26T21:39:33.141 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-26T21:39:33.141 INFO:teuthology.orchestra.run.smithi076.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-26T21:39:33.141 INFO:teuthology.orchestra.run.smithi076.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-26T21:39:33.141 INFO:teuthology.orchestra.run.smithi076.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-26T21:39:33.142 INFO:teuthology.orchestra.run.smithi076.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-26T21:39:33.142 INFO:teuthology.orchestra.run.smithi076.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-26T21:39:33.142 INFO:teuthology.orchestra.run.smithi076.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-26T21:39:33.142 INFO:teuthology.orchestra.run.smithi076.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-26T21:39:33.142 INFO:teuthology.orchestra.run.smithi076.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-26T21:39:33.142 INFO:teuthology.orchestra.run.smithi076.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-26T21:39:33.142 INFO:teuthology.orchestra.run.smithi076.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-26T21:39:33.142 INFO:teuthology.orchestra.run.smithi076.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-26T21:39:33.142 INFO:teuthology.orchestra.run.smithi076.stdout: python3-ply-3.11-14.el9.noarch 2024-04-26T21:39:33.142 INFO:teuthology.orchestra.run.smithi076.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-26T21:39:33.142 INFO:teuthology.orchestra.run.smithi076.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-26T21:39:33.143 INFO:teuthology.orchestra.run.smithi076.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-26T21:39:33.143 INFO:teuthology.orchestra.run.smithi076.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-26T21:39:33.143 INFO:teuthology.orchestra.run.smithi076.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-26T21:39:33.143 INFO:teuthology.orchestra.run.smithi076.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-26T21:39:33.143 INFO:teuthology.orchestra.run.smithi076.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-26T21:39:33.143 INFO:teuthology.orchestra.run.smithi076.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-26T21:39:33.143 INFO:teuthology.orchestra.run.smithi076.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-26T21:39:33.143 INFO:teuthology.orchestra.run.smithi076.stdout: python3-rados-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:39:33.143 INFO:teuthology.orchestra.run.smithi076.stdout: python3-rbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:39:33.143 INFO:teuthology.orchestra.run.smithi076.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-26T21:39:33.143 INFO:teuthology.orchestra.run.smithi076.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-26T21:39:33.143 INFO:teuthology.orchestra.run.smithi076.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-26T21:39:33.143 INFO:teuthology.orchestra.run.smithi076.stdout: python3-rgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:39:33.143 INFO:teuthology.orchestra.run.smithi076.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-26T21:39:33.144 INFO:teuthology.orchestra.run.smithi076.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-26T21:39:33.144 INFO:teuthology.orchestra.run.smithi076.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-26T21:39:33.144 INFO:teuthology.orchestra.run.smithi076.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-26T21:39:33.144 INFO:teuthology.orchestra.run.smithi076.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-26T21:39:33.144 INFO:teuthology.orchestra.run.smithi076.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-26T21:39:33.144 INFO:teuthology.orchestra.run.smithi076.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-26T21:39:33.144 INFO:teuthology.orchestra.run.smithi076.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-26T21:39:33.144 INFO:teuthology.orchestra.run.smithi076.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-26T21:39:33.144 INFO:teuthology.orchestra.run.smithi076.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-26T21:39:33.144 INFO:teuthology.orchestra.run.smithi076.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-26T21:39:33.144 INFO:teuthology.orchestra.run.smithi076.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-26T21:39:33.144 INFO:teuthology.orchestra.run.smithi076.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-26T21:39:33.144 INFO:teuthology.orchestra.run.smithi076.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-26T21:39:33.144 INFO:teuthology.orchestra.run.smithi076.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-26T21:39:33.145 INFO:teuthology.orchestra.run.smithi076.stdout: rbd-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:39:33.145 INFO:teuthology.orchestra.run.smithi076.stdout: rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:39:33.145 INFO:teuthology.orchestra.run.smithi076.stdout: rbd-nbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:39:33.145 INFO:teuthology.orchestra.run.smithi076.stdout: re2-1:20211101-3.el9.x86_64 2024-04-26T21:39:33.145 INFO:teuthology.orchestra.run.smithi076.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-26T21:39:33.145 INFO:teuthology.orchestra.run.smithi076.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-26T21:39:33.145 INFO:teuthology.orchestra.run.smithi076.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-26T21:39:33.145 INFO:teuthology.orchestra.run.smithi076.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-26T21:39:33.145 INFO:teuthology.orchestra.run.smithi076.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-26T21:39:33.146 INFO:teuthology.orchestra.run.smithi076.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-26T21:39:33.146 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-26T21:39:33.146 INFO:teuthology.orchestra.run.smithi076.stdout:Complete! 2024-04-26T21:39:33.326 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-26T21:39:33.386 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-26T21:39:33.584 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-26T21:39:33.691 DEBUG:teuthology.parallel:result is None 2024-04-26T21:39:33.742 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 18/156 2024-04-26T21:39:33.789 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-26T21:39:33.838 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-26T21:39:33.942 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-26T21:39:33.995 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-26T21:39:34.060 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-26T21:39:34.100 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T21:39:34.100 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-26T21:39:34.101 INFO:teuthology.orchestra.run.smithi164.stdout:Upgraded: 2024-04-26T21:39:34.101 INFO:teuthology.orchestra.run.smithi164.stdout: librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:39:34.101 INFO:teuthology.orchestra.run.smithi164.stdout: librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:39:34.101 INFO:teuthology.orchestra.run.smithi164.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-26T21:39:34.101 INFO:teuthology.orchestra.run.smithi164.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-26T21:39:34.101 INFO:teuthology.orchestra.run.smithi164.stdout:Installed: 2024-04-26T21:39:34.101 INFO:teuthology.orchestra.run.smithi164.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-26T21:39:34.101 INFO:teuthology.orchestra.run.smithi164.stdout: automake-1.16.2-8.el9.noarch 2024-04-26T21:39:34.101 INFO:teuthology.orchestra.run.smithi164.stdout: bison-3.7.4-5.el9.x86_64 2024-04-26T21:39:34.101 INFO:teuthology.orchestra.run.smithi164.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-26T21:39:34.101 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:39:34.101 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:39:34.101 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:39:34.102 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:39:34.102 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-grafana-dashboards-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:39:34.102 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-immutable-object-cache-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:39:34.102 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:39:34.102 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:39:34.102 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:39:34.102 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:39:34.102 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:39:34.102 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-modules-core-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:39:34.102 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:39:34.102 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:39:34.102 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:39:34.102 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-prometheus-alerts-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:39:34.103 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:39:34.103 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:39:34.103 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-test-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:39:34.103 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:39:34.103 INFO:teuthology.orchestra.run.smithi164.stdout: cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:39:34.103 INFO:teuthology.orchestra.run.smithi164.stdout: cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:39:34.103 INFO:teuthology.orchestra.run.smithi164.stdout: cephfs-shell-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:39:34.103 INFO:teuthology.orchestra.run.smithi164.stdout: cephfs-top-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:39:34.103 INFO:teuthology.orchestra.run.smithi164.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-26T21:39:34.103 INFO:teuthology.orchestra.run.smithi164.stdout: flex-2.6.4-9.el9.x86_64 2024-04-26T21:39:34.103 INFO:teuthology.orchestra.run.smithi164.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-26T21:39:34.103 INFO:teuthology.orchestra.run.smithi164.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-26T21:39:34.103 INFO:teuthology.orchestra.run.smithi164.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-26T21:39:34.104 INFO:teuthology.orchestra.run.smithi164.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-26T21:39:34.104 INFO:teuthology.orchestra.run.smithi164.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-26T21:39:34.104 INFO:teuthology.orchestra.run.smithi164.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-26T21:39:34.104 INFO:teuthology.orchestra.run.smithi164.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-26T21:39:34.104 INFO:teuthology.orchestra.run.smithi164.stdout: libarrow-9.0.0-11.el9.x86_64 2024-04-26T21:39:34.104 INFO:teuthology.orchestra.run.smithi164.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-04-26T21:39:34.104 INFO:teuthology.orchestra.run.smithi164.stdout: libcephfs-devel-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:39:34.104 INFO:teuthology.orchestra.run.smithi164.stdout: libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:39:34.105 INFO:teuthology.orchestra.run.smithi164.stdout: libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:39:34.105 INFO:teuthology.orchestra.run.smithi164.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-26T21:39:34.105 INFO:teuthology.orchestra.run.smithi164.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-26T21:39:34.105 INFO:teuthology.orchestra.run.smithi164.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-26T21:39:34.105 INFO:teuthology.orchestra.run.smithi164.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-26T21:39:34.105 INFO:teuthology.orchestra.run.smithi164.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-26T21:39:34.105 INFO:teuthology.orchestra.run.smithi164.stdout: librados-devel-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:39:34.105 INFO:teuthology.orchestra.run.smithi164.stdout: libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:39:34.105 INFO:teuthology.orchestra.run.smithi164.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-26T21:39:34.105 INFO:teuthology.orchestra.run.smithi164.stdout: librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:39:34.106 INFO:teuthology.orchestra.run.smithi164.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-26T21:39:34.106 INFO:teuthology.orchestra.run.smithi164.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-26T21:39:34.106 INFO:teuthology.orchestra.run.smithi164.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-26T21:39:34.106 INFO:teuthology.orchestra.run.smithi164.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-26T21:39:34.106 INFO:teuthology.orchestra.run.smithi164.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-26T21:39:34.106 INFO:teuthology.orchestra.run.smithi164.stdout: lua-5.4.4-4.el9.x86_64 2024-04-26T21:39:34.106 INFO:teuthology.orchestra.run.smithi164.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-26T21:39:34.106 INFO:teuthology.orchestra.run.smithi164.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-26T21:39:34.106 INFO:teuthology.orchestra.run.smithi164.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-26T21:39:34.106 INFO:teuthology.orchestra.run.smithi164.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-26T21:39:34.107 INFO:teuthology.orchestra.run.smithi164.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-26T21:39:34.107 INFO:teuthology.orchestra.run.smithi164.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-04-26T21:39:34.107 INFO:teuthology.orchestra.run.smithi164.stdout: postgresql-13.14-1.el9.x86_64 2024-04-26T21:39:34.107 INFO:teuthology.orchestra.run.smithi164.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-26T21:39:34.107 INFO:teuthology.orchestra.run.smithi164.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-26T21:39:34.107 INFO:teuthology.orchestra.run.smithi164.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-26T21:39:34.107 INFO:teuthology.orchestra.run.smithi164.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-26T21:39:34.107 INFO:teuthology.orchestra.run.smithi164.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-26T21:39:34.107 INFO:teuthology.orchestra.run.smithi164.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-26T21:39:34.107 INFO:teuthology.orchestra.run.smithi164.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-26T21:39:34.107 INFO:teuthology.orchestra.run.smithi164.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-26T21:39:34.107 INFO:teuthology.orchestra.run.smithi164.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-26T21:39:34.107 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-26T21:39:34.107 INFO:teuthology.orchestra.run.smithi164.stdout: python3-ceph-argparse-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:39:34.108 INFO:teuthology.orchestra.run.smithi164.stdout: python3-ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:39:34.108 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cephfs-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:39:34.108 INFO:teuthology.orchestra.run.smithi164.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-26T21:39:34.108 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-26T21:39:34.108 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-26T21:39:34.108 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-26T21:39:34.108 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-26T21:39:34.108 INFO:teuthology.orchestra.run.smithi164.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-26T21:39:34.108 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-26T21:39:34.108 INFO:teuthology.orchestra.run.smithi164.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-26T21:39:34.108 INFO:teuthology.orchestra.run.smithi164.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-26T21:39:34.108 INFO:teuthology.orchestra.run.smithi164.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-26T21:39:34.108 INFO:teuthology.orchestra.run.smithi164.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-26T21:39:34.108 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-26T21:39:34.109 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-26T21:39:34.109 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-26T21:39:34.109 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-26T21:39:34.109 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-26T21:39:34.109 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-26T21:39:34.109 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-26T21:39:34.109 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-26T21:39:34.109 INFO:teuthology.orchestra.run.smithi164.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-26T21:39:34.109 INFO:teuthology.orchestra.run.smithi164.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-26T21:39:34.109 INFO:teuthology.orchestra.run.smithi164.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-26T21:39:34.109 INFO:teuthology.orchestra.run.smithi164.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-26T21:39:34.110 INFO:teuthology.orchestra.run.smithi164.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-26T21:39:34.110 INFO:teuthology.orchestra.run.smithi164.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-26T21:39:34.110 INFO:teuthology.orchestra.run.smithi164.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-26T21:39:34.110 INFO:teuthology.orchestra.run.smithi164.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-26T21:39:34.110 INFO:teuthology.orchestra.run.smithi164.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-26T21:39:34.110 INFO:teuthology.orchestra.run.smithi164.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-26T21:39:34.110 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-26T21:39:34.110 INFO:teuthology.orchestra.run.smithi164.stdout: python3-ply-3.11-14.el9.noarch 2024-04-26T21:39:34.110 INFO:teuthology.orchestra.run.smithi164.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-26T21:39:34.110 INFO:teuthology.orchestra.run.smithi164.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-26T21:39:34.110 INFO:teuthology.orchestra.run.smithi164.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-26T21:39:34.111 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-26T21:39:34.111 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-26T21:39:34.111 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-26T21:39:34.111 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-26T21:39:34.111 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-26T21:39:34.111 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-26T21:39:34.111 INFO:teuthology.orchestra.run.smithi164.stdout: python3-rados-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:39:34.111 INFO:teuthology.orchestra.run.smithi164.stdout: python3-rbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:39:34.111 INFO:teuthology.orchestra.run.smithi164.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-26T21:39:34.111 INFO:teuthology.orchestra.run.smithi164.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-26T21:39:34.111 INFO:teuthology.orchestra.run.smithi164.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-26T21:39:34.111 INFO:teuthology.orchestra.run.smithi164.stdout: python3-rgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:39:34.111 INFO:teuthology.orchestra.run.smithi164.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-26T21:39:34.111 INFO:teuthology.orchestra.run.smithi164.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-26T21:39:34.112 INFO:teuthology.orchestra.run.smithi164.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-26T21:39:34.112 INFO:teuthology.orchestra.run.smithi164.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-26T21:39:34.112 INFO:teuthology.orchestra.run.smithi164.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-26T21:39:34.112 INFO:teuthology.orchestra.run.smithi164.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-26T21:39:34.112 INFO:teuthology.orchestra.run.smithi164.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-26T21:39:34.112 INFO:teuthology.orchestra.run.smithi164.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-26T21:39:34.112 INFO:teuthology.orchestra.run.smithi164.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-26T21:39:34.112 INFO:teuthology.orchestra.run.smithi164.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-26T21:39:34.112 INFO:teuthology.orchestra.run.smithi164.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-26T21:39:34.113 INFO:teuthology.orchestra.run.smithi164.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-26T21:39:34.113 INFO:teuthology.orchestra.run.smithi164.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-26T21:39:34.113 INFO:teuthology.orchestra.run.smithi164.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-26T21:39:34.113 INFO:teuthology.orchestra.run.smithi164.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-26T21:39:34.113 INFO:teuthology.orchestra.run.smithi164.stdout: rbd-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:39:34.113 INFO:teuthology.orchestra.run.smithi164.stdout: rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:39:34.113 INFO:teuthology.orchestra.run.smithi164.stdout: rbd-nbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:39:34.113 INFO:teuthology.orchestra.run.smithi164.stdout: re2-1:20211101-3.el9.x86_64 2024-04-26T21:39:34.113 INFO:teuthology.orchestra.run.smithi164.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-26T21:39:34.113 INFO:teuthology.orchestra.run.smithi164.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-26T21:39:34.113 INFO:teuthology.orchestra.run.smithi164.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-26T21:39:34.113 INFO:teuthology.orchestra.run.smithi164.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-26T21:39:34.114 INFO:teuthology.orchestra.run.smithi164.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-26T21:39:34.114 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-26T21:39:34.115 INFO:teuthology.orchestra.run.smithi164.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-26T21:39:34.115 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-26T21:39:34.115 INFO:teuthology.orchestra.run.smithi164.stdout:Complete! 2024-04-26T21:39:34.151 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-26T21:39:34.267 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-26T21:39:34.350 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-26T21:39:34.388 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-26T21:39:34.457 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-26T21:39:34.576 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-26T21:39:34.611 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-ceph-common-2:19.0.0-3306.gbc612ffb.el9. 31/156 2024-04-26T21:39:34.664 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-ceph-argparse-2:19.0.0-3306.gbc612ffb.el 32/156 2024-04-26T21:39:34.702 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-cephfs-2:19.0.0-3306.gbc612ffb.el9.x86_6 33/156 2024-04-26T21:39:34.716 DEBUG:teuthology.parallel:result is None 2024-04-26T21:39:34.821 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-26T21:39:34.883 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-26T21:39:35.010 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-26T21:39:35.196 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-26T21:39:35.263 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-26T21:39:35.966 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-26T21:39:36.079 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-26T21:39:36.116 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-26T21:39:36.164 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-26T21:39:36.230 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : librados-devel-2:19.0.0-3306.gbc612ffb.el9.x86_6 43/156 2024-04-26T21:39:36.301 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-26T21:39:36.574 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-26T21:39:36.634 INFO:teuthology.orchestra.run.smithi072.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-26T21:39:36.716 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-26T21:39:36.818 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-26T21:39:36.870 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-26T21:39:36.919 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-26T21:39:37.000 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-26T21:39:37.046 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-26T21:39:37.108 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-26T21:39:37.177 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-26T21:39:37.384 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-26T21:39:37.446 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-26T21:39:37.502 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-26T21:39:37.555 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-26T21:39:37.700 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-26T21:39:37.746 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-26T21:39:37.794 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-26T21:39:37.837 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-26T21:39:37.870 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 63/156 2024-04-26T21:39:38.052 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-26T21:39:38.268 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-26T21:39:38.317 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-26T21:39:38.358 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-26T21:39:38.404 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-26T21:39:38.472 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-26T21:39:38.535 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-26T21:39:38.584 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-26T21:39:38.637 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-26T21:39:38.690 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-26T21:39:38.801 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-26T21:39:38.848 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-26T21:39:39.001 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-26T21:39:39.063 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-26T21:39:39.115 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-26T21:39:39.189 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-26T21:39:39.350 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-26T21:39:39.764 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-26T21:39:40.764 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-26T21:39:40.828 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-26T21:39:40.896 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-26T21:39:41.042 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-26T21:39:41.116 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-26T21:39:41.232 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-26T21:39:41.276 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-26T21:39:41.510 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-26T21:39:41.549 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-26T21:39:41.703 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-26T21:39:42.610 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-26T21:39:42.770 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-26T21:39:44.371 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-26T21:39:44.495 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-26T21:39:44.545 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-26T21:39:44.592 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-26T21:39:44.993 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-26T21:39:45.046 INFO:teuthology.orchestra.run.smithi072.stdout: Upgrading : librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 99/156 2024-04-26T21:39:45.169 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 99/156 2024-04-26T21:39:45.211 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-rbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 100/156 2024-04-26T21:39:45.272 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-26T21:39:45.814 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 102/156 2024-04-26T21:39:45.842 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 103/156 2024-04-26T21:39:45.944 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 103/156 2024-04-26T21:39:46.014 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-rgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 104/156 2024-04-26T21:39:48.708 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 105/156 2024-04-26T21:39:48.758 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 105/156 2024-04-26T21:39:49.399 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 105/156 2024-04-26T21:39:49.458 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 106/156 2024-04-26T21:39:49.630 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 106/156 2024-04-26T21:39:49.631 INFO:teuthology.orchestra.run.smithi072.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-26T21:39:49.631 INFO:teuthology.orchestra.run.smithi072.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-26T21:39:49.631 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T21:39:49.676 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 107/156 2024-04-26T21:40:06.492 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 107/156 2024-04-26T21:40:06.493 INFO:teuthology.orchestra.run.smithi072.stdout:skipping the directory /sys 2024-04-26T21:40:06.493 INFO:teuthology.orchestra.run.smithi072.stdout:skipping the directory /proc 2024-04-26T21:40:06.493 INFO:teuthology.orchestra.run.smithi072.stdout:skipping the directory /mnt 2024-04-26T21:40:06.493 INFO:teuthology.orchestra.run.smithi072.stdout:skipping the directory /var/tmp 2024-04-26T21:40:06.493 INFO:teuthology.orchestra.run.smithi072.stdout:skipping the directory /home 2024-04-26T21:40:06.493 INFO:teuthology.orchestra.run.smithi072.stdout:skipping the directory /root 2024-04-26T21:40:06.493 INFO:teuthology.orchestra.run.smithi072.stdout:skipping the directory /tmp 2024-04-26T21:40:06.493 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T21:40:06.749 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 108/156 2024-04-26T21:40:06.795 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 108/156 2024-04-26T21:40:06.795 INFO:teuthology.orchestra.run.smithi072.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:40:06.795 INFO:teuthology.orchestra.run.smithi072.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-26T21:40:06.796 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T21:40:08.421 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 109/156 2024-04-26T21:40:08.498 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 109/156 2024-04-26T21:40:08.498 INFO:teuthology.orchestra.run.smithi072.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:40:08.498 INFO:teuthology.orchestra.run.smithi072.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-26T21:40:08.498 INFO:teuthology.orchestra.run.smithi072.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-26T21:40:08.499 INFO:teuthology.orchestra.run.smithi072.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-26T21:40:08.499 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T21:40:08.764 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 110/156 2024-04-26T21:40:08.853 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 110/156 2024-04-26T21:40:08.854 INFO:teuthology.orchestra.run.smithi072.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:40:08.854 INFO:teuthology.orchestra.run.smithi072.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-26T21:40:08.854 INFO:teuthology.orchestra.run.smithi072.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-26T21:40:08.854 INFO:teuthology.orchestra.run.smithi072.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-26T21:40:08.854 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T21:40:09.357 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 111/156 2024-04-26T21:40:09.450 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 111/156 2024-04-26T21:40:09.450 INFO:teuthology.orchestra.run.smithi072.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:40:09.450 INFO:teuthology.orchestra.run.smithi072.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-26T21:40:09.450 INFO:teuthology.orchestra.run.smithi072.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-26T21:40:09.450 INFO:teuthology.orchestra.run.smithi072.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-26T21:40:09.450 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T21:40:09.575 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-26T21:40:09.667 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-26T21:40:09.750 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-26T21:40:09.988 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-26T21:40:10.060 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-26T21:40:10.147 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-26T21:40:10.255 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-26T21:40:10.426 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-26T21:40:10.491 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-26T21:40:10.530 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-26T21:40:10.574 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-26T21:40:11.250 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 123/156 2024-04-26T21:40:11.290 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 123/156 2024-04-26T21:40:11.349 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3306.gbc612ffb.e 124/156 2024-04-26T21:40:11.457 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3306.gbc612ffb. 125/156 2024-04-26T21:40:11.535 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noa 126/156 2024-04-26T21:40:12.866 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noa 126/156 2024-04-26T21:40:12.914 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.n 127/156 2024-04-26T21:40:14.009 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.n 127/156 2024-04-26T21:40:14.039 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc6 128/156 2024-04-26T21:40:14.186 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc6 128/156 2024-04-26T21:40:14.365 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3306.gbc612ffb.el 129/156 2024-04-26T21:40:14.395 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 130/156 2024-04-26T21:40:14.485 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 130/156 2024-04-26T21:40:14.485 INFO:teuthology.orchestra.run.smithi072.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:40:14.485 INFO:teuthology.orchestra.run.smithi072.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-26T21:40:14.485 INFO:teuthology.orchestra.run.smithi072.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-26T21:40:14.485 INFO:teuthology.orchestra.run.smithi072.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-26T21:40:14.485 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T21:40:14.547 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 131/156 2024-04-26T21:40:14.584 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 131/156 2024-04-26T21:40:14.696 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-2:19.0.0-3306.gbc612ffb.el9.x86_64 132/156 2024-04-26T21:40:14.776 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 133/156 2024-04-26T21:40:15.778 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-26T21:40:15.820 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 135/156 2024-04-26T21:40:15.914 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 135/156 2024-04-26T21:40:15.914 INFO:teuthology.orchestra.run.smithi072.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:40:15.914 INFO:teuthology.orchestra.run.smithi072.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-26T21:40:15.914 INFO:teuthology.orchestra.run.smithi072.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-26T21:40:15.914 INFO:teuthology.orchestra.run.smithi072.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-26T21:40:15.914 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T21:40:15.963 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3306.gbc612 136/156 2024-04-26T21:40:16.059 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3306.gbc612 136/156 2024-04-26T21:40:16.059 INFO:teuthology.orchestra.run.smithi072.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:40:16.059 INFO:teuthology.orchestra.run.smithi072.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-26T21:40:16.059 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T21:40:16.147 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 137/156 2024-04-26T21:40:16.224 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 137/156 2024-04-26T21:40:16.224 INFO:teuthology.orchestra.run.smithi072.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:40:16.224 INFO:teuthology.orchestra.run.smithi072.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-26T21:40:16.224 INFO:teuthology.orchestra.run.smithi072.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-26T21:40:16.224 INFO:teuthology.orchestra.run.smithi072.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-26T21:40:16.224 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T21:40:16.607 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 138/156 2024-04-26T21:40:16.694 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 138/156 2024-04-26T21:40:16.694 INFO:teuthology.orchestra.run.smithi072.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:40:16.695 INFO:teuthology.orchestra.run.smithi072.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-26T21:40:16.695 INFO:teuthology.orchestra.run.smithi072.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-26T21:40:16.695 INFO:teuthology.orchestra.run.smithi072.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-26T21:40:16.695 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T21:40:21.729 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-test-2:19.0.0-3306.gbc612ffb.el9.x86_64 139/156 2024-04-26T21:40:21.811 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : rbd-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 140/156 2024-04-26T21:40:21.899 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : rbd-nbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 141/156 2024-04-26T21:40:21.940 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-26T21:40:21.991 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-26T21:40:22.312 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : cephfs-shell-2:19.0.0-3306.gbc612ffb.el9.x86_64 144/156 2024-04-26T21:40:22.423 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-26T21:40:22.460 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : libcephfs-devel-2:19.0.0-3306.gbc612ffb.el9.x86_ 146/156 2024-04-26T21:40:23.353 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T21:40:23.408 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T21:40:23.476 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T21:40:24.838 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : cephfs-top-2:19.0.0-3306.gbc612ffb.el9.noarch 148/156 2024-04-26T21:40:25.005 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-26T21:40:25.076 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-26T21:40:25.245 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-26T21:40:25.290 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-26T21:40:25.290 INFO:teuthology.orchestra.run.smithi072.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-26T21:40:25.417 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-26T21:40:25.466 INFO:teuthology.orchestra.run.smithi072.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-26T21:40:25.466 INFO:teuthology.orchestra.run.smithi072.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-26T21:40:25.640 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-26T21:40:25.774 INFO:teuthology.orchestra.run.smithi072.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T21:40:26.058 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T21:40:26.058 INFO:teuthology.orchestra.run.smithi072.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-04-26T21:40:26.059 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T21:40:36.033 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-2:19.0.0-3306.gbc612ffb.el9.x86_64 1/156 2024-04-26T21:40:36.033 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 2/156 2024-04-26T21:40:36.033 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 3/156 2024-04-26T21:40:36.033 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 4/156 2024-04-26T21:40:36.033 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3306.gbc612 5/156 2024-04-26T21:40:36.033 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 6/156 2024-04-26T21:40:36.033 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 7/156 2024-04-26T21:40:36.033 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 8/156 2024-04-26T21:40:36.033 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 9/156 2024-04-26T21:40:36.034 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 10/156 2024-04-26T21:40:36.034 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 11/156 2024-04-26T21:40:36.034 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-test-2:19.0.0-3306.gbc612ffb.el9.x86_64 12/156 2024-04-26T21:40:36.034 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 13/156 2024-04-26T21:40:36.034 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : cephfs-shell-2:19.0.0-3306.gbc612ffb.el9.x86_64 14/156 2024-04-26T21:40:36.034 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : libcephfs-devel-2:19.0.0-3306.gbc612ffb.el9.x86_ 15/156 2024-04-26T21:40:36.034 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 16/156 2024-04-26T21:40:36.034 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 17/156 2024-04-26T21:40:36.034 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : librados-devel-2:19.0.0-3306.gbc612ffb.el9.x86_6 18/156 2024-04-26T21:40:36.034 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86 19/156 2024-04-26T21:40:36.034 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 20/156 2024-04-26T21:40:36.034 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3306.gbc612ffb.el 21/156 2024-04-26T21:40:36.035 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-ceph-common-2:19.0.0-3306.gbc612ffb.el9. 22/156 2024-04-26T21:40:36.035 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-cephfs-2:19.0.0-3306.gbc612ffb.el9.x86_6 23/156 2024-04-26T21:40:36.035 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-rados-2:19.0.0-3306.gbc612ffb.el9.x86_64 24/156 2024-04-26T21:40:36.035 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-rbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 25/156 2024-04-26T21:40:36.035 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-rgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 26/156 2024-04-26T21:40:36.035 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : rbd-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 27/156 2024-04-26T21:40:36.035 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 28/156 2024-04-26T21:40:36.035 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : rbd-nbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 29/156 2024-04-26T21:40:36.048 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3306.gbc612ffb. 30/156 2024-04-26T21:40:36.048 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noa 31/156 2024-04-26T21:40:36.048 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.n 32/156 2024-04-26T21:40:36.048 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc6 33/156 2024-04-26T21:40:36.048 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3306.gbc612ffb.el 34/156 2024-04-26T21:40:36.048 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 35/156 2024-04-26T21:40:36.048 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3306.gbc612ffb.e 36/156 2024-04-26T21:40:36.048 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 37/156 2024-04-26T21:40:36.049 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 38/156 2024-04-26T21:40:36.049 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : cephfs-top-2:19.0.0-3306.gbc612ffb.el9.noarch 39/156 2024-04-26T21:40:36.049 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-26T21:40:36.049 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-26T21:40:36.049 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-26T21:40:36.049 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-26T21:40:36.049 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-26T21:40:36.049 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-26T21:40:36.049 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-26T21:40:36.049 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-26T21:40:36.049 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-26T21:40:36.049 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-26T21:40:36.049 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-26T21:40:36.050 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-26T21:40:36.050 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-26T21:40:36.050 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-26T21:40:36.050 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-26T21:40:36.050 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-26T21:40:36.050 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-26T21:40:36.050 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-26T21:40:36.050 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-26T21:40:36.050 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-26T21:40:36.050 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-26T21:40:36.050 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-26T21:40:36.050 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-26T21:40:36.051 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-26T21:40:36.051 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-26T21:40:36.051 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-26T21:40:36.051 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-26T21:40:36.051 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-26T21:40:36.051 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-26T21:40:36.051 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-26T21:40:36.051 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-26T21:40:36.051 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-26T21:40:36.051 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-26T21:40:36.051 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-26T21:40:36.051 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-26T21:40:36.051 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-26T21:40:36.052 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-26T21:40:36.052 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-26T21:40:36.052 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-26T21:40:36.052 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-26T21:40:36.052 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-26T21:40:36.052 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-26T21:40:36.052 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-26T21:40:36.052 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-26T21:40:36.052 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-26T21:40:36.052 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-26T21:40:36.052 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-26T21:40:36.052 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-26T21:40:36.052 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-26T21:40:36.053 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-26T21:40:36.053 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-26T21:40:36.053 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-26T21:40:36.053 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-26T21:40:36.053 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-26T21:40:36.053 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-26T21:40:36.053 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-26T21:40:36.053 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-26T21:40:36.053 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-26T21:40:36.053 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-26T21:40:36.053 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-26T21:40:36.053 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-26T21:40:36.053 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-26T21:40:36.054 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-26T21:40:36.054 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-26T21:40:36.054 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-26T21:40:36.054 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-26T21:40:36.054 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-26T21:40:36.054 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-26T21:40:36.054 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/156 2024-04-26T21:40:36.054 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/156 2024-04-26T21:40:36.054 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-26T21:40:36.054 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-26T21:40:36.054 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-26T21:40:36.054 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/156 2024-04-26T21:40:36.054 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-26T21:40:36.055 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-26T21:40:36.055 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-26T21:40:36.055 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-26T21:40:36.055 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-26T21:40:36.055 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-26T21:40:36.055 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-26T21:40:36.055 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-26T21:40:36.055 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-26T21:40:36.055 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-26T21:40:36.055 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-26T21:40:36.055 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-26T21:40:36.055 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-26T21:40:36.055 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-26T21:40:36.055 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-26T21:40:36.056 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-26T21:40:36.056 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-26T21:40:36.056 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-26T21:40:36.056 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-26T21:40:36.056 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-26T21:40:36.056 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-26T21:40:36.056 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-26T21:40:36.056 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-26T21:40:36.056 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-26T21:40:36.056 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-26T21:40:36.056 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-26T21:40:36.056 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-26T21:40:36.056 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-26T21:40:36.057 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-26T21:40:36.057 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-26T21:40:36.057 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-26T21:40:36.057 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-26T21:40:36.057 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-26T21:40:36.057 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-26T21:40:36.057 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-26T21:40:36.057 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 149/156 2024-04-26T21:40:36.057 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-26T21:40:36.057 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 151/156 2024-04-26T21:40:36.057 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-26T21:40:36.057 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-26T21:40:36.057 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-26T21:40:36.058 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-26T21:40:38.540 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T21:40:38.540 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T21:40:38.540 INFO:teuthology.orchestra.run.smithi072.stdout:Upgraded: 2024-04-26T21:40:38.540 INFO:teuthology.orchestra.run.smithi072.stdout: librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:40:38.541 INFO:teuthology.orchestra.run.smithi072.stdout: librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:40:38.541 INFO:teuthology.orchestra.run.smithi072.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-26T21:40:38.541 INFO:teuthology.orchestra.run.smithi072.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-26T21:40:38.541 INFO:teuthology.orchestra.run.smithi072.stdout:Installed: 2024-04-26T21:40:38.541 INFO:teuthology.orchestra.run.smithi072.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-26T21:40:38.541 INFO:teuthology.orchestra.run.smithi072.stdout: automake-1.16.2-8.el9.noarch 2024-04-26T21:40:38.541 INFO:teuthology.orchestra.run.smithi072.stdout: bison-3.7.4-5.el9.x86_64 2024-04-26T21:40:38.541 INFO:teuthology.orchestra.run.smithi072.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-26T21:40:38.541 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:40:38.541 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:40:38.541 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:40:38.541 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:40:38.542 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-grafana-dashboards-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:40:38.542 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-immutable-object-cache-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:40:38.542 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:40:38.542 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:40:38.542 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:40:38.542 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:40:38.542 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:40:38.542 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mgr-modules-core-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:40:38.542 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:40:38.542 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:40:38.542 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:40:38.542 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-prometheus-alerts-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:40:38.543 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:40:38.543 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:40:38.543 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-test-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:40:38.543 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:40:38.543 INFO:teuthology.orchestra.run.smithi072.stdout: cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:40:38.543 INFO:teuthology.orchestra.run.smithi072.stdout: cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:40:38.543 INFO:teuthology.orchestra.run.smithi072.stdout: cephfs-shell-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:40:38.543 INFO:teuthology.orchestra.run.smithi072.stdout: cephfs-top-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:40:38.543 INFO:teuthology.orchestra.run.smithi072.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-26T21:40:38.543 INFO:teuthology.orchestra.run.smithi072.stdout: flex-2.6.4-9.el9.x86_64 2024-04-26T21:40:38.543 INFO:teuthology.orchestra.run.smithi072.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-26T21:40:38.543 INFO:teuthology.orchestra.run.smithi072.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-26T21:40:38.543 INFO:teuthology.orchestra.run.smithi072.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-26T21:40:38.544 INFO:teuthology.orchestra.run.smithi072.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-26T21:40:38.544 INFO:teuthology.orchestra.run.smithi072.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-26T21:40:38.544 INFO:teuthology.orchestra.run.smithi072.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-26T21:40:38.544 INFO:teuthology.orchestra.run.smithi072.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-26T21:40:38.544 INFO:teuthology.orchestra.run.smithi072.stdout: libarrow-9.0.0-11.el9.x86_64 2024-04-26T21:40:38.544 INFO:teuthology.orchestra.run.smithi072.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-04-26T21:40:38.544 INFO:teuthology.orchestra.run.smithi072.stdout: libcephfs-devel-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:40:38.544 INFO:teuthology.orchestra.run.smithi072.stdout: libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:40:38.544 INFO:teuthology.orchestra.run.smithi072.stdout: libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:40:38.544 INFO:teuthology.orchestra.run.smithi072.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-26T21:40:38.544 INFO:teuthology.orchestra.run.smithi072.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-26T21:40:38.544 INFO:teuthology.orchestra.run.smithi072.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-26T21:40:38.544 INFO:teuthology.orchestra.run.smithi072.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-26T21:40:38.544 INFO:teuthology.orchestra.run.smithi072.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-26T21:40:38.545 INFO:teuthology.orchestra.run.smithi072.stdout: librados-devel-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:40:38.545 INFO:teuthology.orchestra.run.smithi072.stdout: libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:40:38.545 INFO:teuthology.orchestra.run.smithi072.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-26T21:40:38.545 INFO:teuthology.orchestra.run.smithi072.stdout: librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:40:38.545 INFO:teuthology.orchestra.run.smithi072.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-26T21:40:38.545 INFO:teuthology.orchestra.run.smithi072.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-26T21:40:38.545 INFO:teuthology.orchestra.run.smithi072.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-26T21:40:38.545 INFO:teuthology.orchestra.run.smithi072.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-26T21:40:38.545 INFO:teuthology.orchestra.run.smithi072.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-26T21:40:38.545 INFO:teuthology.orchestra.run.smithi072.stdout: lua-5.4.4-4.el9.x86_64 2024-04-26T21:40:38.545 INFO:teuthology.orchestra.run.smithi072.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-26T21:40:38.545 INFO:teuthology.orchestra.run.smithi072.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-26T21:40:38.545 INFO:teuthology.orchestra.run.smithi072.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-26T21:40:38.546 INFO:teuthology.orchestra.run.smithi072.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-26T21:40:38.546 INFO:teuthology.orchestra.run.smithi072.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-26T21:40:38.546 INFO:teuthology.orchestra.run.smithi072.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-04-26T21:40:38.546 INFO:teuthology.orchestra.run.smithi072.stdout: postgresql-13.14-1.el9.x86_64 2024-04-26T21:40:38.546 INFO:teuthology.orchestra.run.smithi072.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-26T21:40:38.546 INFO:teuthology.orchestra.run.smithi072.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-26T21:40:38.546 INFO:teuthology.orchestra.run.smithi072.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-26T21:40:38.546 INFO:teuthology.orchestra.run.smithi072.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-26T21:40:38.546 INFO:teuthology.orchestra.run.smithi072.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-26T21:40:38.546 INFO:teuthology.orchestra.run.smithi072.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-26T21:40:38.546 INFO:teuthology.orchestra.run.smithi072.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-26T21:40:38.546 INFO:teuthology.orchestra.run.smithi072.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-26T21:40:38.546 INFO:teuthology.orchestra.run.smithi072.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-26T21:40:38.547 INFO:teuthology.orchestra.run.smithi072.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-26T21:40:38.547 INFO:teuthology.orchestra.run.smithi072.stdout: python3-ceph-argparse-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:40:38.547 INFO:teuthology.orchestra.run.smithi072.stdout: python3-ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:40:38.547 INFO:teuthology.orchestra.run.smithi072.stdout: python3-cephfs-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:40:38.547 INFO:teuthology.orchestra.run.smithi072.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-26T21:40:38.547 INFO:teuthology.orchestra.run.smithi072.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-26T21:40:38.547 INFO:teuthology.orchestra.run.smithi072.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-26T21:40:38.547 INFO:teuthology.orchestra.run.smithi072.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-26T21:40:38.547 INFO:teuthology.orchestra.run.smithi072.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-26T21:40:38.547 INFO:teuthology.orchestra.run.smithi072.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-26T21:40:38.547 INFO:teuthology.orchestra.run.smithi072.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-26T21:40:38.547 INFO:teuthology.orchestra.run.smithi072.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-26T21:40:38.547 INFO:teuthology.orchestra.run.smithi072.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-26T21:40:38.548 INFO:teuthology.orchestra.run.smithi072.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-26T21:40:38.548 INFO:teuthology.orchestra.run.smithi072.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-26T21:40:38.548 INFO:teuthology.orchestra.run.smithi072.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-26T21:40:38.548 INFO:teuthology.orchestra.run.smithi072.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-26T21:40:38.548 INFO:teuthology.orchestra.run.smithi072.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-26T21:40:38.548 INFO:teuthology.orchestra.run.smithi072.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-26T21:40:38.548 INFO:teuthology.orchestra.run.smithi072.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-26T21:40:38.548 INFO:teuthology.orchestra.run.smithi072.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-26T21:40:38.548 INFO:teuthology.orchestra.run.smithi072.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-26T21:40:38.548 INFO:teuthology.orchestra.run.smithi072.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-26T21:40:38.548 INFO:teuthology.orchestra.run.smithi072.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-26T21:40:38.548 INFO:teuthology.orchestra.run.smithi072.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-26T21:40:38.548 INFO:teuthology.orchestra.run.smithi072.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-26T21:40:38.548 INFO:teuthology.orchestra.run.smithi072.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-26T21:40:38.549 INFO:teuthology.orchestra.run.smithi072.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-26T21:40:38.549 INFO:teuthology.orchestra.run.smithi072.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-26T21:40:38.549 INFO:teuthology.orchestra.run.smithi072.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-26T21:40:38.549 INFO:teuthology.orchestra.run.smithi072.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-26T21:40:38.549 INFO:teuthology.orchestra.run.smithi072.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-26T21:40:38.549 INFO:teuthology.orchestra.run.smithi072.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-26T21:40:38.549 INFO:teuthology.orchestra.run.smithi072.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-26T21:40:38.549 INFO:teuthology.orchestra.run.smithi072.stdout: python3-ply-3.11-14.el9.noarch 2024-04-26T21:40:38.549 INFO:teuthology.orchestra.run.smithi072.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-26T21:40:38.549 INFO:teuthology.orchestra.run.smithi072.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-26T21:40:38.549 INFO:teuthology.orchestra.run.smithi072.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-26T21:40:38.549 INFO:teuthology.orchestra.run.smithi072.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-26T21:40:38.549 INFO:teuthology.orchestra.run.smithi072.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-26T21:40:38.550 INFO:teuthology.orchestra.run.smithi072.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-26T21:40:38.550 INFO:teuthology.orchestra.run.smithi072.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-26T21:40:38.550 INFO:teuthology.orchestra.run.smithi072.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-26T21:40:38.550 INFO:teuthology.orchestra.run.smithi072.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-26T21:40:38.550 INFO:teuthology.orchestra.run.smithi072.stdout: python3-rados-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:40:38.550 INFO:teuthology.orchestra.run.smithi072.stdout: python3-rbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:40:38.550 INFO:teuthology.orchestra.run.smithi072.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-26T21:40:38.550 INFO:teuthology.orchestra.run.smithi072.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-26T21:40:38.550 INFO:teuthology.orchestra.run.smithi072.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-26T21:40:38.550 INFO:teuthology.orchestra.run.smithi072.stdout: python3-rgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:40:38.550 INFO:teuthology.orchestra.run.smithi072.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-26T21:40:38.550 INFO:teuthology.orchestra.run.smithi072.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-26T21:40:38.550 INFO:teuthology.orchestra.run.smithi072.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-26T21:40:38.551 INFO:teuthology.orchestra.run.smithi072.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-26T21:40:38.551 INFO:teuthology.orchestra.run.smithi072.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-26T21:40:38.551 INFO:teuthology.orchestra.run.smithi072.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-26T21:40:38.551 INFO:teuthology.orchestra.run.smithi072.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-26T21:40:38.551 INFO:teuthology.orchestra.run.smithi072.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-26T21:40:38.551 INFO:teuthology.orchestra.run.smithi072.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-26T21:40:38.551 INFO:teuthology.orchestra.run.smithi072.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-26T21:40:38.551 INFO:teuthology.orchestra.run.smithi072.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-26T21:40:38.551 INFO:teuthology.orchestra.run.smithi072.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-26T21:40:38.551 INFO:teuthology.orchestra.run.smithi072.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-26T21:40:38.551 INFO:teuthology.orchestra.run.smithi072.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-26T21:40:38.551 INFO:teuthology.orchestra.run.smithi072.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-26T21:40:38.551 INFO:teuthology.orchestra.run.smithi072.stdout: rbd-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:40:38.551 INFO:teuthology.orchestra.run.smithi072.stdout: rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:40:38.552 INFO:teuthology.orchestra.run.smithi072.stdout: rbd-nbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:40:38.552 INFO:teuthology.orchestra.run.smithi072.stdout: re2-1:20211101-3.el9.x86_64 2024-04-26T21:40:38.552 INFO:teuthology.orchestra.run.smithi072.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-26T21:40:38.552 INFO:teuthology.orchestra.run.smithi072.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-26T21:40:38.552 INFO:teuthology.orchestra.run.smithi072.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-26T21:40:38.552 INFO:teuthology.orchestra.run.smithi072.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-26T21:40:38.552 INFO:teuthology.orchestra.run.smithi072.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-26T21:40:38.552 INFO:teuthology.orchestra.run.smithi072.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-26T21:40:38.553 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T21:40:38.553 INFO:teuthology.orchestra.run.smithi072.stdout:Complete! 2024-04-26T21:40:39.368 DEBUG:teuthology.parallel:result is None 2024-04-26T21:40:39.368 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T21:40:39.509 DEBUG:teuthology.orchestra.run.smithi072:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-26T21:40:39.565 INFO:teuthology.orchestra.run.smithi072.stdout:19.0.0-3306.gbc612ffb.el9 2024-04-26T21:40:39.566 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3306.gbc612ffb.el9 2024-04-26T21:40:39.566 INFO:teuthology.task.install:The correct ceph version 19.0.0-3306.gbc612ffb is installed. 2024-04-26T21:40:39.568 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T21:40:39.788 DEBUG:teuthology.orchestra.run.smithi076:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-26T21:40:39.862 INFO:teuthology.orchestra.run.smithi076.stdout:19.0.0-3306.gbc612ffb.el9 2024-04-26T21:40:39.863 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3306.gbc612ffb.el9 2024-04-26T21:40:39.863 INFO:teuthology.task.install:The correct ceph version 19.0.0-3306.gbc612ffb is installed. 2024-04-26T21:40:39.865 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T21:40:40.001 DEBUG:teuthology.orchestra.run.smithi164:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-26T21:40:40.076 INFO:teuthology.orchestra.run.smithi164.stdout:19.0.0-3306.gbc612ffb.el9 2024-04-26T21:40:40.077 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3306.gbc612ffb.el9 2024-04-26T21:40:40.077 INFO:teuthology.task.install:The correct ceph version 19.0.0-3306.gbc612ffb is installed. 2024-04-26T21:40:40.079 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-26T21:40:40.079 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2024-04-26T21:40:40.079 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-26T21:40:40.146 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-04-26T21:40:40.146 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-26T21:40:40.238 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-04-26T21:40:40.238 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-26T21:40:40.330 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-26T21:40:40.331 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2024-04-26T21:40:40.331 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd of=/usr/bin/daemon-helper 2024-04-26T21:40:40.391 DEBUG:teuthology.orchestra.run.smithi072:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-26T21:40:40.493 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-04-26T21:40:40.493 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd of=/usr/bin/daemon-helper 2024-04-26T21:40:40.584 DEBUG:teuthology.orchestra.run.smithi076:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-26T21:40:40.712 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-04-26T21:40:40.712 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd of=/usr/bin/daemon-helper 2024-04-26T21:40:40.801 DEBUG:teuthology.orchestra.run.smithi164:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-26T21:40:40.931 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-26T21:40:40.931 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2024-04-26T21:40:40.931 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-26T21:40:40.994 DEBUG:teuthology.orchestra.run.smithi072:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-26T21:40:41.096 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-04-26T21:40:41.097 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-26T21:40:41.185 DEBUG:teuthology.orchestra.run.smithi076:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-26T21:40:41.309 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-04-26T21:40:41.309 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-26T21:40:41.399 DEBUG:teuthology.orchestra.run.smithi164:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-26T21:40:41.527 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-26T21:40:41.527 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2024-04-26T21:40:41.527 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd of=/usr/bin/stdin-killer 2024-04-26T21:40:41.594 DEBUG:teuthology.orchestra.run.smithi072:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-26T21:40:41.693 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-04-26T21:40:41.693 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd of=/usr/bin/stdin-killer 2024-04-26T21:40:41.783 DEBUG:teuthology.orchestra.run.smithi076:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-26T21:40:41.866 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-04-26T21:40:41.866 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd of=/usr/bin/stdin-killer 2024-04-26T21:40:41.955 DEBUG:teuthology.orchestra.run.smithi164:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-26T21:40:42.039 INFO:teuthology.run_tasks:Running task cephadm... 2024-04-26T21:40:42.159 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387', 'subvols': {'create': 2}} 2024-04-26T21:40:42.159 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T21:40:42.159 INFO:tasks.cephadm:Cluster fsid is a1780950-0415-11ef-bc93-c7b262605968 2024-04-26T21:40:42.159 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-04-26T21:40:42.160 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.72', 'mon.b': '172.21.15.76', 'mon.c': '172.21.15.164'} 2024-04-26T21:40:42.160 INFO:tasks.cephadm:First mon is mon.a on smithi072 2024-04-26T21:40:42.160 INFO:tasks.cephadm:First mgr is x 2024-04-26T21:40:42.160 INFO:tasks.cephadm:Normalizing hostnames... 2024-04-26T21:40:42.160 DEBUG:teuthology.orchestra.run.smithi072:> sudo hostname $(hostname -s) 2024-04-26T21:40:42.238 DEBUG:teuthology.orchestra.run.smithi076:> sudo hostname $(hostname -s) 2024-04-26T21:40:42.328 DEBUG:teuthology.orchestra.run.smithi164:> sudo hostname $(hostname -s) 2024-04-26T21:40:42.422 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-04-26T21:40:42.422 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T21:40:42.562 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/78918/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-3306-gbc612ffb', 'node_name': '172.21.2.14+braggi14', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-3306.gbc612ffb'}, 'url': 'https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-04-26 13:02:42.256491', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-rishabh-testing-20240426.111959', 'chacra_url': 'https://1.chacra.ceph.com/repos/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-04-26T21:40:42.780 INFO:tasks.util.chacra:got chacra host 1.chacra.ceph.com, ref wip-rishabh-testing-20240426.111959, sha1 bc612ffb9c1d5bc27536bc917a705262a36d4387 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T21:40:42.782 INFO:tasks.cephadm:Discovered cachra url: https://1.chacra.ceph.com/binaries/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/x86_64/flavors/default/cephadm 2024-04-26T21:40:42.782 DEBUG:teuthology.orchestra.run.smithi072:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-26T21:40:43.392 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r--. 1 ubuntu ubuntu 781514 Apr 26 21:40 /home/ubuntu/cephtest/cephadm 2024-04-26T21:40:43.393 DEBUG:teuthology.orchestra.run.smithi076:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-26T21:40:44.138 INFO:teuthology.orchestra.run.smithi076.stdout:-rw-r--r--. 1 ubuntu ubuntu 781514 Apr 26 21:40 /home/ubuntu/cephtest/cephadm 2024-04-26T21:40:44.138 DEBUG:teuthology.orchestra.run.smithi164:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-26T21:40:46.153 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r--. 1 ubuntu ubuntu 781514 Apr 26 21:40 /home/ubuntu/cephtest/cephadm 2024-04-26T21:40:46.154 DEBUG:teuthology.orchestra.run.smithi072:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-26T21:40:46.214 DEBUG:teuthology.orchestra.run.smithi076:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-26T21:40:46.277 DEBUG:teuthology.orchestra.run.smithi164:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-26T21:40:46.352 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 on all hosts... 2024-04-26T21:40:46.352 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 pull 2024-04-26T21:40:46.357 DEBUG:teuthology.orchestra.run.smithi076:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 pull 2024-04-26T21:40:46.364 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 pull 2024-04-26T21:40:48.278 INFO:teuthology.orchestra.run.smithi076.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387... 2024-04-26T21:40:48.456 INFO:teuthology.orchestra.run.smithi164.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387... 2024-04-26T21:40:51.719 INFO:teuthology.orchestra.run.smithi072.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387... 2024-04-26T21:41:24.303 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-04-26T21:41:24.304 INFO:teuthology.orchestra.run.smithi164.stdout: "ceph_version": "ceph version 19.0.0-3306-gbc612ffb (bc612ffb9c1d5bc27536bc917a705262a36d4387) squid (dev)", 2024-04-26T21:41:24.304 INFO:teuthology.orchestra.run.smithi164.stdout: "image_id": "55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07", 2024-04-26T21:41:24.304 INFO:teuthology.orchestra.run.smithi164.stdout: "repo_digests": [ 2024-04-26T21:41:24.304 INFO:teuthology.orchestra.run.smithi164.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2" 2024-04-26T21:41:24.304 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-04-26T21:41:24.304 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-04-26T21:41:28.464 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-04-26T21:41:28.464 INFO:teuthology.orchestra.run.smithi076.stdout: "ceph_version": "ceph version 19.0.0-3306-gbc612ffb (bc612ffb9c1d5bc27536bc917a705262a36d4387) squid (dev)", 2024-04-26T21:41:28.464 INFO:teuthology.orchestra.run.smithi076.stdout: "image_id": "55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07", 2024-04-26T21:41:28.464 INFO:teuthology.orchestra.run.smithi076.stdout: "repo_digests": [ 2024-04-26T21:41:28.465 INFO:teuthology.orchestra.run.smithi076.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2" 2024-04-26T21:41:28.465 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-04-26T21:41:28.465 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-04-26T21:41:28.914 INFO:teuthology.orchestra.run.smithi072.stdout:{ 2024-04-26T21:41:28.914 INFO:teuthology.orchestra.run.smithi072.stdout: "ceph_version": "ceph version 19.0.0-3306-gbc612ffb (bc612ffb9c1d5bc27536bc917a705262a36d4387) squid (dev)", 2024-04-26T21:41:28.915 INFO:teuthology.orchestra.run.smithi072.stdout: "image_id": "55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07", 2024-04-26T21:41:28.915 INFO:teuthology.orchestra.run.smithi072.stdout: "repo_digests": [ 2024-04-26T21:41:28.915 INFO:teuthology.orchestra.run.smithi072.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2" 2024-04-26T21:41:28.915 INFO:teuthology.orchestra.run.smithi072.stdout: ] 2024-04-26T21:41:28.915 INFO:teuthology.orchestra.run.smithi072.stdout:} 2024-04-26T21:41:28.963 DEBUG:teuthology.orchestra.run.smithi072:> sudo mkdir -p /etc/ceph 2024-04-26T21:41:29.037 DEBUG:teuthology.orchestra.run.smithi076:> sudo mkdir -p /etc/ceph 2024-04-26T21:41:29.124 DEBUG:teuthology.orchestra.run.smithi164:> sudo mkdir -p /etc/ceph 2024-04-26T21:41:29.217 DEBUG:teuthology.orchestra.run.smithi072:> sudo chmod 777 /etc/ceph 2024-04-26T21:41:29.293 DEBUG:teuthology.orchestra.run.smithi076:> sudo chmod 777 /etc/ceph 2024-04-26T21:41:29.379 DEBUG:teuthology.orchestra.run.smithi164:> sudo chmod 777 /etc/ceph 2024-04-26T21:41:29.468 INFO:tasks.cephadm:Writing seed config... 2024-04-26T21:41:29.469 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-04-26T21:41:29.469 INFO:tasks.cephadm: override: [client] debug client = 20 2024-04-26T21:41:29.469 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-04-26T21:41:29.469 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-04-26T21:41:29.469 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-04-26T21:41:29.469 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-04-26T21:41:29.470 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-04-26T21:41:29.470 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-04-26T21:41:29.470 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-04-26T21:41:29.470 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-04-26T21:41:29.470 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-04-26T21:41:29.470 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-04-26T21:41:29.470 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-04-26T21:41:29.470 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-04-26T21:41:29.470 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-04-26T21:41:29.470 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-04-26T21:41:29.470 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-04-26T21:41:29.470 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-04-26T21:41:29.470 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-04-26T21:41:29.471 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-04-26T21:41:29.471 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-04-26T21:41:29.471 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-04-26T21:41:29.471 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-04-26T21:41:29.471 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-04-26T21:41:29.471 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-04-26T21:41:29.471 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-04-26T21:41:29.471 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-04-26T21:41:29.471 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-04-26T21:41:29.471 INFO:tasks.cephadm: override: [osd] bdev async discard = True 2024-04-26T21:41:29.471 INFO:tasks.cephadm: override: [osd] bdev enable discard = True 2024-04-26T21:41:29.471 INFO:tasks.cephadm: override: [osd] bluestore allocator = bitmap 2024-04-26T21:41:29.471 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-04-26T21:41:29.471 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-04-26T21:41:29.472 INFO:tasks.cephadm: override: [osd] debug bluefs = 1/20 2024-04-26T21:41:29.472 INFO:tasks.cephadm: override: [osd] debug bluestore = 1/20 2024-04-26T21:41:29.472 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-04-26T21:41:29.472 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-04-26T21:41:29.472 INFO:tasks.cephadm: override: [osd] debug rocksdb = 4/10 2024-04-26T21:41:29.472 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-04-26T21:41:29.472 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-04-26T21:41:29.472 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-04-26T21:41:29.472 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-04-26T21:41:29.472 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-04-26T21:41:29.472 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-04-26T21:41:29.472 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-04-26T21:41:29.472 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10 2024-04-26T21:41:29.473 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2024-04-26T21:41:29.473 DEBUG:teuthology.orchestra.run.smithi072:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-04-26T21:41:29.526 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = a1780950-0415-11ef-bc93-c7b262605968 [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bdev async discard = True bdev enable discard = True bluestore allocator = bitmap bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 1/20 debug bluestore = 1/20 debug ms = 1 debug osd = 20 debug rocksdb = 4/10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-04-26T21:41:29.526 DEBUG:teuthology.orchestra.run.smithi072:mon.a> sudo journalctl -f -n 0 -u ceph-a1780950-0415-11ef-bc93-c7b262605968@mon.a.service 2024-04-26T21:41:29.575 DEBUG:teuthology.orchestra.run.smithi072:mgr.x> sudo journalctl -f -n 0 -u ceph-a1780950-0415-11ef-bc93-c7b262605968@mgr.x.service 2024-04-26T21:41:29.620 INFO:tasks.cephadm:Bootstrapping... 2024-04-26T21:41:29.620 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 -v bootstrap --fsid a1780950-0415-11ef-bc93-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.72 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-04-26T21:41:29.882 INFO:teuthology.orchestra.run.smithi072.stdout:-------------------------------------------------------------------------------- 2024-04-26T21:41:29.882 INFO:teuthology.orchestra.run.smithi072.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387', '-v', 'bootstrap', '--fsid', 'a1780950-0415-11ef-bc93-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.72', '--skip-admin-label'] 2024-04-26T21:41:29.883 INFO:teuthology.orchestra.run.smithi072.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-04-26T21:41:29.883 INFO:teuthology.orchestra.run.smithi072.stdout:Verifying podman|docker is present... 2024-04-26T21:41:29.937 INFO:teuthology.orchestra.run.smithi072.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-26T21:41:29.937 INFO:teuthology.orchestra.run.smithi072.stdout:Verifying lvm2 is present... 2024-04-26T21:41:29.937 INFO:teuthology.orchestra.run.smithi072.stdout:Verifying time synchronization is in place... 2024-04-26T21:41:29.954 INFO:teuthology.orchestra.run.smithi072.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-26T21:41:29.954 INFO:teuthology.orchestra.run.smithi072.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-26T21:41:29.971 INFO:teuthology.orchestra.run.smithi072.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-26T21:41:29.971 INFO:teuthology.orchestra.run.smithi072.stdout:systemctl: stdout inactive 2024-04-26T21:41:29.991 INFO:teuthology.orchestra.run.smithi072.stdout:systemctl: stdout enabled 2024-04-26T21:41:30.008 INFO:teuthology.orchestra.run.smithi072.stdout:systemctl: stdout active 2024-04-26T21:41:30.008 INFO:teuthology.orchestra.run.smithi072.stdout:Unit chronyd.service is enabled and running 2024-04-26T21:41:30.008 INFO:teuthology.orchestra.run.smithi072.stdout:Repeating the final host check... 2024-04-26T21:41:30.058 INFO:teuthology.orchestra.run.smithi072.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-26T21:41:30.058 INFO:teuthology.orchestra.run.smithi072.stdout:podman (/bin/podman) version 4.9.4 is present 2024-04-26T21:41:30.058 INFO:teuthology.orchestra.run.smithi072.stdout:systemctl is present 2024-04-26T21:41:30.058 INFO:teuthology.orchestra.run.smithi072.stdout:lvcreate is present 2024-04-26T21:41:30.072 INFO:teuthology.orchestra.run.smithi072.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-26T21:41:30.072 INFO:teuthology.orchestra.run.smithi072.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-26T21:41:30.088 INFO:teuthology.orchestra.run.smithi072.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-26T21:41:30.088 INFO:teuthology.orchestra.run.smithi072.stdout:systemctl: stdout inactive 2024-04-26T21:41:30.107 INFO:teuthology.orchestra.run.smithi072.stdout:systemctl: stdout enabled 2024-04-26T21:41:30.125 INFO:teuthology.orchestra.run.smithi072.stdout:systemctl: stdout active 2024-04-26T21:41:30.125 INFO:teuthology.orchestra.run.smithi072.stdout:Unit chronyd.service is enabled and running 2024-04-26T21:41:30.125 INFO:teuthology.orchestra.run.smithi072.stdout:Host looks OK 2024-04-26T21:41:30.125 INFO:teuthology.orchestra.run.smithi072.stdout:Cluster fsid: a1780950-0415-11ef-bc93-c7b262605968 2024-04-26T21:41:30.125 INFO:teuthology.orchestra.run.smithi072.stdout:Acquiring lock 140450909413328 on /run/cephadm/a1780950-0415-11ef-bc93-c7b262605968.lock 2024-04-26T21:41:30.125 INFO:teuthology.orchestra.run.smithi072.stdout:Lock 140450909413328 acquired on /run/cephadm/a1780950-0415-11ef-bc93-c7b262605968.lock 2024-04-26T21:41:30.125 INFO:teuthology.orchestra.run.smithi072.stdout:Verifying IP 172.21.15.72 port 3300 ... 2024-04-26T21:41:30.126 INFO:teuthology.orchestra.run.smithi072.stdout:Verifying IP 172.21.15.72 port 6789 ... 2024-04-26T21:41:30.126 INFO:teuthology.orchestra.run.smithi072.stdout:Base mon IP(s) is [172.21.15.72:3300, 172.21.15.72:6789], mon addrv is [v2:172.21.15.72:3300,v1:172.21.15.72:6789] 2024-04-26T21:41:30.134 INFO:teuthology.orchestra.run.smithi072.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.72 metric 100 2024-04-26T21:41:30.134 INFO:teuthology.orchestra.run.smithi072.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.72 metric 100 2024-04-26T21:41:30.141 INFO:teuthology.orchestra.run.smithi072.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-04-26T21:41:30.141 INFO:teuthology.orchestra.run.smithi072.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 256 pref medium 2024-04-26T21:41:30.141 INFO:teuthology.orchestra.run.smithi072.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 1024 expires 1791sec hoplimit 64 pref medium 2024-04-26T21:41:30.148 INFO:teuthology.orchestra.run.smithi072.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-04-26T21:41:30.148 INFO:teuthology.orchestra.run.smithi072.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-04-26T21:41:30.148 INFO:teuthology.orchestra.run.smithi072.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-26T21:41:30.148 INFO:teuthology.orchestra.run.smithi072.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-04-26T21:41:30.148 INFO:teuthology.orchestra.run.smithi072.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe8f:b765/64 scope link 2024-04-26T21:41:30.148 INFO:teuthology.orchestra.run.smithi072.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-26T21:41:30.149 INFO:teuthology.orchestra.run.smithi072.stdout:Mon IP `172.21.15.72` is in CIDR network `172.21.0.0/20` 2024-04-26T21:41:30.149 INFO:teuthology.orchestra.run.smithi072.stdout:Mon IP `172.21.15.72` is in CIDR network `172.21.0.0/20` 2024-04-26T21:41:30.149 INFO:teuthology.orchestra.run.smithi072.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-04-26T21:41:30.150 INFO:teuthology.orchestra.run.smithi072.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-04-26T21:41:30.151 INFO:teuthology.orchestra.run.smithi072.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387... 2024-04-26T21:41:48.591 INFO:teuthology.orchestra.run.smithi072.stdout:/bin/podman: stdout 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 2024-04-26T21:41:48.591 INFO:teuthology.orchestra.run.smithi072.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387... 2024-04-26T21:41:48.591 INFO:teuthology.orchestra.run.smithi072.stdout:/bin/podman: stderr Getting image source signatures 2024-04-26T21:41:48.591 INFO:teuthology.orchestra.run.smithi072.stdout:/bin/podman: stderr Copying blob sha256:f340d36a00bce3feed0ad509aeafc9092c1c625bf553b8a613c380a79149b87e 2024-04-26T21:41:48.591 INFO:teuthology.orchestra.run.smithi072.stdout:/bin/podman: stderr Copying blob sha256:3d665f4c032644be9cae748af04bee69f2bfe96cdc3616dabe0ab17899633ab5 2024-04-26T21:41:48.591 INFO:teuthology.orchestra.run.smithi072.stdout:/bin/podman: stderr Copying config sha256:55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 2024-04-26T21:41:48.591 INFO:teuthology.orchestra.run.smithi072.stdout:/bin/podman: stderr Writing manifest to image destination 2024-04-26T21:41:49.834 INFO:teuthology.orchestra.run.smithi072.stdout:ceph: stdout ceph version 19.0.0-3306-gbc612ffb (bc612ffb9c1d5bc27536bc917a705262a36d4387) squid (dev) 2024-04-26T21:41:49.834 INFO:teuthology.orchestra.run.smithi072.stdout:Ceph version: ceph version 19.0.0-3306-gbc612ffb (bc612ffb9c1d5bc27536bc917a705262a36d4387) squid (dev) 2024-04-26T21:41:49.834 INFO:teuthology.orchestra.run.smithi072.stdout:Extracting ceph user uid/gid from container image... 2024-04-26T21:41:51.086 INFO:teuthology.orchestra.run.smithi072.stdout:stat: stdout 167 167 2024-04-26T21:41:51.087 INFO:teuthology.orchestra.run.smithi072.stdout:Creating initial keys... 2024-04-26T21:41:52.242 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph-authtool: stdout AQCfHyxmCMgNIhAAuZ0jrDEkEad7EC7SH4HqSA== 2024-04-26T21:41:53.393 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph-authtool: stdout AQCgHyxmj5CEKRAA3LlwE5vF/cDMnKGuqj6m6Q== 2024-04-26T21:41:56.527 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph-authtool: stdout AQChHyxmsucCNBAAbWgrh3V6HTEGHJXUw4QrJg== 2024-04-26T21:41:56.530 INFO:teuthology.orchestra.run.smithi072.stdout:Creating initial monmap... 2024-04-26T21:41:57.788 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-26T21:41:57.789 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-04-26T21:41:57.789 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to a1780950-0415-11ef-bc93-c7b262605968 2024-04-26T21:41:57.790 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-26T21:41:57.790 INFO:teuthology.orchestra.run.smithi072.stdout:monmaptool for a [v2:172.21.15.72:3300,v1:172.21.15.72:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-26T21:41:57.790 INFO:teuthology.orchestra.run.smithi072.stdout:setting min_mon_release = quincy 2024-04-26T21:41:57.790 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/monmaptool: set fsid to a1780950-0415-11ef-bc93-c7b262605968 2024-04-26T21:41:57.790 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-26T21:41:57.790 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T21:41:57.790 INFO:teuthology.orchestra.run.smithi072.stdout:Creating mon... 2024-04-26T21:41:58.903 INFO:teuthology.orchestra.run.smithi072.stdout:create mon.a on 2024-04-26T21:41:59.299 INFO:teuthology.orchestra.run.smithi072.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-04-26T21:41:59.551 INFO:teuthology.orchestra.run.smithi072.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-04-26T21:41:59.891 INFO:teuthology.orchestra.run.smithi072.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-a1780950-0415-11ef-bc93-c7b262605968.target → /etc/systemd/system/ceph-a1780950-0415-11ef-bc93-c7b262605968.target. 2024-04-26T21:41:59.892 INFO:teuthology.orchestra.run.smithi072.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-a1780950-0415-11ef-bc93-c7b262605968.target → /etc/systemd/system/ceph-a1780950-0415-11ef-bc93-c7b262605968.target. 2024-04-26T21:42:00.264 INFO:teuthology.orchestra.run.smithi072.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-a1780950-0415-11ef-bc93-c7b262605968@mon.a 2024-04-26T21:42:00.264 INFO:teuthology.orchestra.run.smithi072.stdout:systemctl: stderr Failed to reset failed state of unit ceph-a1780950-0415-11ef-bc93-c7b262605968@mon.a.service: Unit ceph-a1780950-0415-11ef-bc93-c7b262605968@mon.a.service not loaded. 2024-04-26T21:42:00.731 INFO:teuthology.orchestra.run.smithi072.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-a1780950-0415-11ef-bc93-c7b262605968.target.wants/ceph-a1780950-0415-11ef-bc93-c7b262605968@mon.a.service → /etc/systemd/system/ceph-a1780950-0415-11ef-bc93-c7b262605968@.service. 2024-04-26T21:42:01.420 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:01 smithi072 podman[27753]: 2024-04-26 21:42:01.179944114 +0000 UTC m=+0.195526399 container create 637fbb3f46041834882ebe14419bd43a1149afceb63349221ba609351f1d9d92 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-mon-a, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-04-26T21:42:01.420 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:01 smithi072 podman[27753]: 2024-04-26 21:42:01.418188733 +0000 UTC m=+0.433771025 container init 637fbb3f46041834882ebe14419bd43a1149afceb63349221ba609351f1d9d92 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, io.buildah.version=1.33.5) 2024-04-26T21:42:01.539 INFO:teuthology.orchestra.run.smithi072.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-26T21:42:01.539 INFO:teuthology.orchestra.run.smithi072.stdout:systemctl: stdout disabled 2024-04-26T21:42:01.554 INFO:teuthology.orchestra.run.smithi072.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-26T21:42:01.555 INFO:teuthology.orchestra.run.smithi072.stdout:systemctl: stdout inactive 2024-04-26T21:42:01.555 INFO:teuthology.orchestra.run.smithi072.stdout:firewalld.service is not enabled 2024-04-26T21:42:01.555 INFO:teuthology.orchestra.run.smithi072.stdout:Not possible to enable service . firewalld.service is not available 2024-04-26T21:42:01.556 INFO:teuthology.orchestra.run.smithi072.stdout:Waiting for mon to start... 2024-04-26T21:42:01.556 INFO:teuthology.orchestra.run.smithi072.stdout:Waiting for mon... 2024-04-26T21:42:01.705 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:01 smithi072 podman[27753]: 2024-04-26 21:42:01.428967626 +0000 UTC m=+0.444549879 container start 637fbb3f46041834882ebe14419bd43a1149afceb63349221ba609351f1d9d92 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-mon-a, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-04-26T21:42:01.705 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:01 smithi072 ceph-mon[27772]: mkfs a1780950-0415-11ef-bc93-c7b262605968 2024-04-26T21:42:01.705 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:01 smithi072 bash[27753]: 637fbb3f46041834882ebe14419bd43a1149afceb63349221ba609351f1d9d92 2024-04-26T21:42:01.705 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:01 smithi072 ceph-mon[27772]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-26T21:42:01.705 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:01 smithi072 systemd[1]: Started Ceph mon.a for a1780950-0415-11ef-bc93-c7b262605968. 2024-04-26T21:42:02.572 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:02 smithi072 ceph-mon[27772]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-26T21:42:02.572 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:02 smithi072 ceph-mon[27772]: monmap epoch 1 2024-04-26T21:42:02.572 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:02 smithi072 ceph-mon[27772]: fsid a1780950-0415-11ef-bc93-c7b262605968 2024-04-26T21:42:02.572 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:02 smithi072 ceph-mon[27772]: last_changed 2024-04-26T21:41:57.024587+0000 2024-04-26T21:42:02.572 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:02 smithi072 ceph-mon[27772]: created 2024-04-26T21:41:57.024587+0000 2024-04-26T21:42:02.573 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:02 smithi072 ceph-mon[27772]: min_mon_release 19 (squid) 2024-04-26T21:42:02.573 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:02 smithi072 ceph-mon[27772]: election_strategy: 1 2024-04-26T21:42:02.573 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:02 smithi072 ceph-mon[27772]: 0: [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] mon.a 2024-04-26T21:42:02.573 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:02 smithi072 ceph-mon[27772]: fsmap 2024-04-26T21:42:02.573 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:02 smithi072 ceph-mon[27772]: osdmap e1: 0 total, 0 up, 0 in 2024-04-26T21:42:02.573 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:02 smithi072 ceph-mon[27772]: mgrmap e1: no daemons active 2024-04-26T21:42:02.573 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:02 smithi072 ceph-mon[27772]: from='client.? 172.21.15.72:0/2411118911' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-04-26T21:42:02.926 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout cluster: 2024-04-26T21:42:02.927 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout id: a1780950-0415-11ef-bc93-c7b262605968 2024-04-26T21:42:02.927 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-04-26T21:42:02.927 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout 2024-04-26T21:42:02.928 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout services: 2024-04-26T21:42:02.928 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.776971s) 2024-04-26T21:42:02.928 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-04-26T21:42:02.929 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-04-26T21:42:02.929 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout 2024-04-26T21:42:02.929 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout data: 2024-04-26T21:42:02.929 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-04-26T21:42:02.929 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-04-26T21:42:02.929 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-04-26T21:42:02.929 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout pgs: 2024-04-26T21:42:02.929 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout 2024-04-26T21:42:02.935 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:02.188+0000 7f7d1b000700 1 Processor -- start 2024-04-26T21:42:02.935 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:02.188+0000 7f7d1b000700 1 -- start start 2024-04-26T21:42:02.935 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:02.188+0000 7f7d1b000700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7d1410ac80 0x7f7d1410b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:02.935 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:02.188+0000 7f7d1b000700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f7d1410b670 con 0x7f7d1410ac80 2024-04-26T21:42:02.935 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:02.189+0000 7f7d19c00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7d1410ac80 0x7f7d1410b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:02.935 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:02.190+0000 7f7d19c00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7d1410ac80 0x7f7d1410b0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:60654/0 (socket says 172.21.15.72:60654) 2024-04-26T21:42:02.935 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:02.190+0000 7f7d19c00700 1 -- 172.21.15.72:0/1028638899 learned_addr learned my addr 172.21.15.72:0/1028638899 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:42:02.935 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:02.190+0000 7f7d19c00700 1 -- 172.21.15.72:0/1028638899 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d1410be80 con 0x7f7d1410ac80 2024-04-26T21:42:02.935 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:02.190+0000 7f7d19c00700 1 --2- 172.21.15.72:0/1028638899 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7d1410ac80 0x7f7d1410b0a0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f7d080098a0 tx=0x7f7d08009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=653b456517a66e47 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:02.935 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:02.191+0000 7f7d13e00700 1 -- 172.21.15.72:0/1028638899 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7d08004160 con 0x7f7d1410ac80 2024-04-26T21:42:02.936 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:02.191+0000 7f7d13e00700 1 -- 172.21.15.72:0/1028638899 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d08004300 con 0x7f7d1410ac80 2024-04-26T21:42:02.936 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:02.191+0000 7f7d13e00700 1 -- 172.21.15.72:0/1028638899 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7d080036a0 con 0x7f7d1410ac80 2024-04-26T21:42:02.936 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:02.191+0000 7f7d1b000700 1 -- 172.21.15.72:0/1028638899 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7d1410ac80 msgr2=0x7f7d1410b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:02.936 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:02.191+0000 7f7d1b000700 1 --2- 172.21.15.72:0/1028638899 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7d1410ac80 0x7f7d1410b0a0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f7d080098a0 tx=0x7f7d08009bb0 comp rx=0 tx=0).stop 2024-04-26T21:42:02.936 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:02.191+0000 7f7d1b000700 1 -- 172.21.15.72:0/1028638899 shutdown_connections 2024-04-26T21:42:02.936 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:02.191+0000 7f7d1b000700 1 --2- 172.21.15.72:0/1028638899 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7d1410ac80 0x7f7d1410b0a0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:02.936 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:02.191+0000 7f7d1b000700 1 -- 172.21.15.72:0/1028638899 >> 172.21.15.72:0/1028638899 conn(0x7f7d1407e040 msgr2=0x7f7d1407e470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:02.936 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:02.192+0000 7f7d1b000700 1 -- 172.21.15.72:0/1028638899 shutdown_connections 2024-04-26T21:42:02.936 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:02.192+0000 7f7d1b000700 1 -- 172.21.15.72:0/1028638899 wait complete. 2024-04-26T21:42:02.936 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:02.193+0000 7f7d1b000700 1 Processor -- start 2024-04-26T21:42:02.936 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:02.193+0000 7f7d1b000700 1 -- start start 2024-04-26T21:42:02.937 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:02.193+0000 7f7d1b000700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7d1410ac80 0x7f7d1419e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:02.937 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:02.193+0000 7f7d1b000700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f7d1410c3b0 con 0x7f7d1410ac80 2024-04-26T21:42:02.937 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:02.193+0000 7f7d19c00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7d1410ac80 0x7f7d1419e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:02.937 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:02.193+0000 7f7d19c00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7d1410ac80 0x7f7d1419e3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:60656/0 (socket says 172.21.15.72:60656) 2024-04-26T21:42:02.937 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:02.193+0000 7f7d19c00700 1 -- 172.21.15.72:0/2411118911 learned_addr learned my addr 172.21.15.72:0/2411118911 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:42:02.937 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:02.194+0000 7f7d19c00700 1 -- 172.21.15.72:0/2411118911 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d1419e8f0 con 0x7f7d1410ac80 2024-04-26T21:42:02.937 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:02.194+0000 7f7d19c00700 1 --2- 172.21.15.72:0/2411118911 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7d1410ac80 0x7f7d1419e3b0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f7d1410b910 tx=0x7f7d08004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:02.937 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:02.194+0000 7f7d12000700 1 -- 172.21.15.72:0/2411118911 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7d08003db0 con 0x7f7d1410ac80 2024-04-26T21:42:02.937 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:02.194+0000 7f7d12000700 1 -- 172.21.15.72:0/2411118911 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d08011630 con 0x7f7d1410ac80 2024-04-26T21:42:02.937 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:02.194+0000 7f7d1b000700 1 -- 172.21.15.72:0/2411118911 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d1419eb80 con 0x7f7d1410ac80 2024-04-26T21:42:02.937 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:02.194+0000 7f7d12000700 1 -- 172.21.15.72:0/2411118911 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7d08011930 con 0x7f7d1410ac80 2024-04-26T21:42:02.938 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:02.195+0000 7f7d1b000700 1 -- 172.21.15.72:0/2411118911 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d141a1870 con 0x7f7d1410ac80 2024-04-26T21:42:02.938 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:02.195+0000 7f7d12000700 1 -- 172.21.15.72:0/2411118911 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f7d08011ad0 con 0x7f7d1410ac80 2024-04-26T21:42:02.938 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:02.195+0000 7f7d12000700 1 -- 172.21.15.72:0/2411118911 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f7d0801b450 con 0x7f7d1410ac80 2024-04-26T21:42:02.938 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:02.196+0000 7f7d1b000700 1 -- 172.21.15.72:0/2411118911 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d1406a520 con 0x7f7d1410ac80 2024-04-26T21:42:02.938 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:02.199+0000 7f7d12000700 1 -- 172.21.15.72:0/2411118911 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f7d08004260 con 0x7f7d1410ac80 2024-04-26T21:42:02.938 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:02.278+0000 7f7d1b000700 1 -- 172.21.15.72:0/2411118911 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f7d1402e680 con 0x7f7d1410ac80 2024-04-26T21:42:02.938 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:02.279+0000 7f7d12000700 1 -- 172.21.15.72:0/2411118911 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7f7d08025480 con 0x7f7d1410ac80 2024-04-26T21:42:02.938 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:02.280+0000 7f7d1b000700 1 -- 172.21.15.72:0/2411118911 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7d1410ac80 msgr2=0x7f7d1419e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:02.938 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:02.280+0000 7f7d1b000700 1 --2- 172.21.15.72:0/2411118911 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7d1410ac80 0x7f7d1419e3b0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f7d1410b910 tx=0x7f7d08004750 comp rx=0 tx=0).stop 2024-04-26T21:42:02.938 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:02.280+0000 7f7d1b000700 1 -- 172.21.15.72:0/2411118911 shutdown_connections 2024-04-26T21:42:02.938 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:02.280+0000 7f7d1b000700 1 --2- 172.21.15.72:0/2411118911 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7d1410ac80 0x7f7d1419e3b0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:02.938 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:02.280+0000 7f7d1b000700 1 -- 172.21.15.72:0/2411118911 >> 172.21.15.72:0/2411118911 conn(0x7f7d1407e040 msgr2=0x7f7d14195020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:02.938 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:02.280+0000 7f7d1b000700 1 -- 172.21.15.72:0/2411118911 shutdown_connections 2024-04-26T21:42:02.938 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:02.280+0000 7f7d1b000700 1 -- 172.21.15.72:0/2411118911 wait complete. 2024-04-26T21:42:02.939 INFO:teuthology.orchestra.run.smithi072.stdout:mon is available 2024-04-26T21:42:02.939 INFO:teuthology.orchestra.run.smithi072.stdout:Assimilating anything we can from ceph.conf... 2024-04-26T21:42:03.822 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:03 smithi072 ceph-mon[27772]: from='client.? 172.21.15.72:0/3944971354' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-26T21:42:03.822 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:03 smithi072 ceph-mon[27772]: from='client.? 172.21.15.72:0/3944971354' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-04-26T21:42:04.353 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout 2024-04-26T21:42:04.353 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout [global] 2024-04-26T21:42:04.353 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout fsid = a1780950-0415-11ef-bc93-c7b262605968 2024-04-26T21:42:04.353 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-04-26T21:42:04.353 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.72:3300,v1:172.21.15.72:6789] 2024-04-26T21:42:04.354 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-26T21:42:04.354 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-26T21:42:04.354 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-26T21:42:04.354 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-26T21:42:04.354 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout 2024-04-26T21:42:04.354 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-26T21:42:04.354 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-26T21:42:04.354 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout 2024-04-26T21:42:04.354 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout [osd] 2024-04-26T21:42:04.354 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout bdev_async_discard = True 2024-04-26T21:42:04.354 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-26T21:42:04.354 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-26T21:42:04.354 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-26T21:42:04.355 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-26T21:42:04.355 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-26T21:42:04.355 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:03.545+0000 7fa87c800700 1 Processor -- start 2024-04-26T21:42:04.355 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:03.545+0000 7fa87c800700 1 -- start start 2024-04-26T21:42:04.355 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:03.545+0000 7fa87c800700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa87407ee70 0x7fa87407d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:04.355 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:03.546+0000 7fa87c800700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fa87407d8a0 con 0x7fa87407ee70 2024-04-26T21:42:04.355 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:03.546+0000 7fa879e00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa87407ee70 0x7fa87407d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:04.355 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:03.546+0000 7fa879e00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa87407ee70 0x7fa87407d2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:60670/0 (socket says 172.21.15.72:60670) 2024-04-26T21:42:04.355 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:03.546+0000 7fa879e00700 1 -- 172.21.15.72:0/2822274683 learned_addr learned my addr 172.21.15.72:0/2822274683 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:42:04.355 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:03.547+0000 7fa879e00700 1 -- 172.21.15.72:0/2822274683 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa87407da20 con 0x7fa87407ee70 2024-04-26T21:42:04.355 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:03.547+0000 7fa879e00700 1 --2- 172.21.15.72:0/2822274683 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa87407ee70 0x7fa87407d2d0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fa8680098a0 tx=0x7fa868009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f05a0bd90e1b4067 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:04.355 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:03.548+0000 7fa878a00700 1 -- 172.21.15.72:0/2822274683 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa868004160 con 0x7fa87407ee70 2024-04-26T21:42:04.355 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:03.548+0000 7fa878a00700 1 -- 172.21.15.72:0/2822274683 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa868004300 con 0x7fa87407ee70 2024-04-26T21:42:04.355 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:03.548+0000 7fa878a00700 1 -- 172.21.15.72:0/2822274683 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa8680036a0 con 0x7fa87407ee70 2024-04-26T21:42:04.356 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:03.549+0000 7fa87c800700 1 -- 172.21.15.72:0/2822274683 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa87407ee70 msgr2=0x7fa87407d2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:04.356 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:03.549+0000 7fa87c800700 1 --2- 172.21.15.72:0/2822274683 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa87407ee70 0x7fa87407d2d0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fa8680098a0 tx=0x7fa868009bb0 comp rx=0 tx=0).stop 2024-04-26T21:42:04.356 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:03.549+0000 7fa87c800700 1 -- 172.21.15.72:0/2822274683 shutdown_connections 2024-04-26T21:42:04.356 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:03.549+0000 7fa87c800700 1 --2- 172.21.15.72:0/2822274683 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa87407ee70 0x7fa87407d2d0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:04.356 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:03.549+0000 7fa87c800700 1 -- 172.21.15.72:0/2822274683 >> 172.21.15.72:0/2822274683 conn(0x7fa874104240 msgr2=0x7fa874106660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:04.356 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:03.549+0000 7fa87c800700 1 -- 172.21.15.72:0/2822274683 shutdown_connections 2024-04-26T21:42:04.356 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:03.549+0000 7fa87c800700 1 -- 172.21.15.72:0/2822274683 wait complete. 2024-04-26T21:42:04.356 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:03.550+0000 7fa87c800700 1 Processor -- start 2024-04-26T21:42:04.356 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:03.550+0000 7fa87c800700 1 -- start start 2024-04-26T21:42:04.356 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:03.551+0000 7fa87c800700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa87407ee70 0x7fa87419e550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:04.356 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:03.551+0000 7fa87c800700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fa87407df50 con 0x7fa87407ee70 2024-04-26T21:42:04.356 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:03.551+0000 7fa879e00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa87407ee70 0x7fa87419e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:04.356 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:03.551+0000 7fa879e00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa87407ee70 0x7fa87419e550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:60678/0 (socket says 172.21.15.72:60678) 2024-04-26T21:42:04.356 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:03.551+0000 7fa879e00700 1 -- 172.21.15.72:0/3944971354 learned_addr learned my addr 172.21.15.72:0/3944971354 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:42:04.357 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:03.551+0000 7fa879e00700 1 -- 172.21.15.72:0/3944971354 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa87419ea90 con 0x7fa87407ee70 2024-04-26T21:42:04.357 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:03.551+0000 7fa879e00700 1 --2- 172.21.15.72:0/3944971354 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa87407ee70 0x7fa87419e550 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fa868000c00 tx=0x7fa868004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:04.357 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:03.552+0000 7fa86ea00700 1 -- 172.21.15.72:0/3944971354 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa868003db0 con 0x7fa87407ee70 2024-04-26T21:42:04.357 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:03.552+0000 7fa86ea00700 1 -- 172.21.15.72:0/3944971354 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa868011630 con 0x7fa87407ee70 2024-04-26T21:42:04.357 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:03.552+0000 7fa87c800700 1 -- 172.21.15.72:0/3944971354 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa87419ed20 con 0x7fa87407ee70 2024-04-26T21:42:04.357 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:03.552+0000 7fa86ea00700 1 -- 172.21.15.72:0/3944971354 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa868011950 con 0x7fa87407ee70 2024-04-26T21:42:04.357 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:03.552+0000 7fa87c800700 1 -- 172.21.15.72:0/3944971354 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8741a1a10 con 0x7fa87407ee70 2024-04-26T21:42:04.357 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:03.553+0000 7fa86ea00700 1 -- 172.21.15.72:0/3944971354 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fa868011af0 con 0x7fa87407ee70 2024-04-26T21:42:04.357 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:03.553+0000 7fa86ea00700 1 -- 172.21.15.72:0/3944971354 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fa86801c450 con 0x7fa87407ee70 2024-04-26T21:42:04.357 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:03.553+0000 7fa87c800700 1 -- 172.21.15.72:0/3944971354 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa874064a80 con 0x7fa87407ee70 2024-04-26T21:42:04.357 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:03.556+0000 7fa86ea00700 1 -- 172.21.15.72:0/3944971354 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fa868033080 con 0x7fa87407ee70 2024-04-26T21:42:04.357 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:03.633+0000 7fa87c800700 1 -- 172.21.15.72:0/3944971354 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7fa87402e650 con 0x7fa87407ee70 2024-04-26T21:42:04.357 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:03.640+0000 7fa86ea00700 1 -- 172.21.15.72:0/3944971354 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+492 (secure 0 0 0) 0x7fa868004260 con 0x7fa87407ee70 2024-04-26T21:42:04.357 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:03.642+0000 7fa86ea00700 1 -- 172.21.15.72:0/3944971354 <== mon.0 v2:172.21.15.72:3300/0 8 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fa86801c6a0 con 0x7fa87407ee70 2024-04-26T21:42:04.358 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:03.643+0000 7fa87c800700 1 -- 172.21.15.72:0/3944971354 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa87407ee70 msgr2=0x7fa87419e550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:04.358 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:03.643+0000 7fa87c800700 1 --2- 172.21.15.72:0/3944971354 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa87407ee70 0x7fa87419e550 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fa868000c00 tx=0x7fa868004750 comp rx=0 tx=0).stop 2024-04-26T21:42:04.358 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:03.643+0000 7fa87c800700 1 -- 172.21.15.72:0/3944971354 shutdown_connections 2024-04-26T21:42:04.358 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:03.643+0000 7fa87c800700 1 --2- 172.21.15.72:0/3944971354 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa87407ee70 0x7fa87419e550 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:04.358 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:03.643+0000 7fa87c800700 1 -- 172.21.15.72:0/3944971354 >> 172.21.15.72:0/3944971354 conn(0x7fa874104240 msgr2=0x7fa874106540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:04.358 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:03.643+0000 7fa87c800700 1 -- 172.21.15.72:0/3944971354 shutdown_connections 2024-04-26T21:42:04.358 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:03.643+0000 7fa87c800700 1 -- 172.21.15.72:0/3944971354 wait complete. 2024-04-26T21:42:04.358 INFO:teuthology.orchestra.run.smithi072.stdout:Generating new minimal ceph.conf... 2024-04-26T21:42:05.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:05 smithi072 ceph-mon[27772]: from='client.? 172.21.15.72:0/1307969234' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:42:05.958 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:04.943+0000 7fe68d600700 1 Processor -- start 2024-04-26T21:42:05.963 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:04.943+0000 7fe68d600700 1 -- start start 2024-04-26T21:42:05.963 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:04.944+0000 7fe68d600700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe688107510 0x7fe688107930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:05.963 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:04.944+0000 7fe68d600700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fe6880646c0 con 0x7fe688107510 2024-04-26T21:42:05.963 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:04.944+0000 7fe686a00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe688107510 0x7fe688107930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:05.964 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:04.944+0000 7fe686a00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe688107510 0x7fe688107930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:60690/0 (socket says 172.21.15.72:60690) 2024-04-26T21:42:05.964 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:04.944+0000 7fe686a00700 1 -- 172.21.15.72:0/2736435542 learned_addr learned my addr 172.21.15.72:0/2736435542 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:42:05.964 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:04.945+0000 7fe686a00700 1 -- 172.21.15.72:0/2736435542 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe688107e70 con 0x7fe688107510 2024-04-26T21:42:05.964 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:04.945+0000 7fe686a00700 1 --2- 172.21.15.72:0/2736435542 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe688107510 0x7fe688107930 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fe678009e80 tx=0x7fe67800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90f0ffc3388d1795 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:05.964 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:04.945+0000 7fe686000700 1 -- 172.21.15.72:0/2736435542 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe678017070 con 0x7fe688107510 2024-04-26T21:42:05.964 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:04.945+0000 7fe686000700 1 -- 172.21.15.72:0/2736435542 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fe67800b8c0 con 0x7fe688107510 2024-04-26T21:42:05.964 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:04.945+0000 7fe686000700 1 -- 172.21.15.72:0/2736435542 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe678004420 con 0x7fe688107510 2024-04-26T21:42:05.964 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:04.946+0000 7fe68d600700 1 -- 172.21.15.72:0/2736435542 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe688107510 msgr2=0x7fe688107930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:05.964 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:04.946+0000 7fe68d600700 1 --2- 172.21.15.72:0/2736435542 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe688107510 0x7fe688107930 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fe678009e80 tx=0x7fe67800b330 comp rx=0 tx=0).stop 2024-04-26T21:42:05.964 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:04.946+0000 7fe68d600700 1 -- 172.21.15.72:0/2736435542 shutdown_connections 2024-04-26T21:42:05.964 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:04.946+0000 7fe68d600700 1 --2- 172.21.15.72:0/2736435542 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe688107510 0x7fe688107930 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:05.964 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:04.946+0000 7fe68d600700 1 -- 172.21.15.72:0/2736435542 >> 172.21.15.72:0/2736435542 conn(0x7fe6881030d0 msgr2=0x7fe688105510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:05.964 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:04.946+0000 7fe68d600700 1 -- 172.21.15.72:0/2736435542 shutdown_connections 2024-04-26T21:42:05.965 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:04.946+0000 7fe68d600700 1 -- 172.21.15.72:0/2736435542 wait complete. 2024-04-26T21:42:05.965 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:04.947+0000 7fe68d600700 1 Processor -- start 2024-04-26T21:42:05.965 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:04.947+0000 7fe68d600700 1 -- start start 2024-04-26T21:42:05.965 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:04.948+0000 7fe68d600700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe688107510 0x7fe68819ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:05.965 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:04.948+0000 7fe68d600700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fe688108e40 con 0x7fe688107510 2024-04-26T21:42:05.965 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:04.948+0000 7fe686a00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe688107510 0x7fe68819ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:05.965 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:04.948+0000 7fe686a00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe688107510 0x7fe68819ecf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:60702/0 (socket says 172.21.15.72:60702) 2024-04-26T21:42:05.965 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:04.948+0000 7fe686a00700 1 -- 172.21.15.72:0/1307969234 learned_addr learned my addr 172.21.15.72:0/1307969234 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:42:05.965 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:04.948+0000 7fe686a00700 1 -- 172.21.15.72:0/1307969234 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe68819f230 con 0x7fe688107510 2024-04-26T21:42:05.965 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:04.949+0000 7fe686a00700 1 --2- 172.21.15.72:0/1307969234 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe688107510 0x7fe68819ecf0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fe678004000 tx=0x7fe678003ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:05.965 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:04.949+0000 7fe67fe00700 1 -- 172.21.15.72:0/1307969234 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe678020020 con 0x7fe688107510 2024-04-26T21:42:05.965 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:04.949+0000 7fe67fe00700 1 -- 172.21.15.72:0/1307969234 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fe6780128a0 con 0x7fe688107510 2024-04-26T21:42:05.965 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:04.949+0000 7fe67fe00700 1 -- 172.21.15.72:0/1307969234 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe678017040 con 0x7fe688107510 2024-04-26T21:42:05.965 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:04.949+0000 7fe68d600700 1 -- 172.21.15.72:0/1307969234 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe68819f4c0 con 0x7fe688107510 2024-04-26T21:42:05.966 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:04.949+0000 7fe68d600700 1 -- 172.21.15.72:0/1307969234 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6881a21a0 con 0x7fe688107510 2024-04-26T21:42:05.966 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:04.949+0000 7fe67fe00700 1 -- 172.21.15.72:0/1307969234 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fe678023020 con 0x7fe688107510 2024-04-26T21:42:05.966 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:04.949+0000 7fe67fe00700 1 -- 172.21.15.72:0/1307969234 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fe67800f050 con 0x7fe688107510 2024-04-26T21:42:05.966 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:04.950+0000 7fe68d600700 1 -- 172.21.15.72:0/1307969234 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe688064a80 con 0x7fe688107510 2024-04-26T21:42:05.966 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:04.953+0000 7fe67fe00700 1 -- 172.21.15.72:0/1307969234 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fe67802a020 con 0x7fe688107510 2024-04-26T21:42:05.966 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:05.030+0000 7fe68d600700 1 -- 172.21.15.72:0/1307969234 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7fe68802e720 con 0x7fe688107510 2024-04-26T21:42:05.966 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:05.030+0000 7fe67fe00700 1 -- 172.21.15.72:0/1307969234 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+175 (secure 0 0 0) 0x7fe67800b860 con 0x7fe688107510 2024-04-26T21:42:05.966 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:05.032+0000 7fe68d600700 1 -- 172.21.15.72:0/1307969234 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe688107510 msgr2=0x7fe68819ecf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:05.966 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:05.032+0000 7fe68d600700 1 --2- 172.21.15.72:0/1307969234 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe688107510 0x7fe68819ecf0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fe678004000 tx=0x7fe678003ad0 comp rx=0 tx=0).stop 2024-04-26T21:42:05.966 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:05.032+0000 7fe68d600700 1 -- 172.21.15.72:0/1307969234 shutdown_connections 2024-04-26T21:42:05.966 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:05.032+0000 7fe68d600700 1 --2- 172.21.15.72:0/1307969234 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe688107510 0x7fe68819ecf0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:05.966 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:05.032+0000 7fe68d600700 1 -- 172.21.15.72:0/1307969234 >> 172.21.15.72:0/1307969234 conn(0x7fe6881030d0 msgr2=0x7fe688195930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:05.966 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:05.032+0000 7fe68d600700 1 -- 172.21.15.72:0/1307969234 shutdown_connections 2024-04-26T21:42:05.966 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:05.032+0000 7fe68d600700 1 -- 172.21.15.72:0/1307969234 wait complete. 2024-04-26T21:42:05.967 INFO:teuthology.orchestra.run.smithi072.stdout:Restarting the monitor... 2024-04-26T21:42:06.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:06 smithi072 systemd[1]: Stopping Ceph mon.a for a1780950-0415-11ef-bc93-c7b262605968... 2024-04-26T21:42:06.752 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:06 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-mon-a[27768]: 2024-04-26T21:42:06.434+0000 7f6389200700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T21:42:06.753 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:06 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-mon-a[27768]: 2024-04-26T21:42:06.434+0000 7f6389200700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-04-26T21:42:06.753 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:06 smithi072 podman[28011]: 2024-04-26 21:42:06.489508047 +0000 UTC m=+0.173283727 container died 637fbb3f46041834882ebe14419bd43a1149afceb63349221ba609351f1d9d92 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-mon-a, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-04-26T21:42:07.067 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:06 smithi072 podman[28011]: 2024-04-26 21:42:06.889568235 +0000 UTC m=+0.573343921 container cleanup 637fbb3f46041834882ebe14419bd43a1149afceb63349221ba609351f1d9d92 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-04-26T21:42:07.067 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:06 smithi072 bash[28011]: ceph-a1780950-0415-11ef-bc93-c7b262605968-mon-a 2024-04-26T21:42:07.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:07 smithi072 podman[28023]: 2024-04-26 21:42:07.066300147 +0000 UTC m=+0.561277737 container remove 637fbb3f46041834882ebe14419bd43a1149afceb63349221ba609351f1d9d92 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-mon-a, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240424, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff) 2024-04-26T21:42:07.822 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:07 smithi072 systemd[1]: ceph-a1780950-0415-11ef-bc93-c7b262605968@mon.a.service: Deactivated successfully. 2024-04-26T21:42:07.822 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:07 smithi072 systemd[1]: Stopped Ceph mon.a for a1780950-0415-11ef-bc93-c7b262605968. 2024-04-26T21:42:07.822 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:07 smithi072 systemd[1]: ceph-a1780950-0415-11ef-bc93-c7b262605968@mon.a.service: Consumed 1.905s CPU time. 2024-04-26T21:42:07.822 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:07 smithi072 systemd[1]: Starting Ceph mon.a for a1780950-0415-11ef-bc93-c7b262605968... 2024-04-26T21:42:08.225 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:07 smithi072 podman[28119]: 2024-04-26 21:42:07.869763136 +0000 UTC m=+0.032638201 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T21:42:08.226 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:07 smithi072 podman[28119]: 2024-04-26 21:42:07.974434307 +0000 UTC m=+0.137309365 container create bf7f10eb1c0959b58dac995a0b937d5e3e743d459deadcea0d4e15c595fa54bf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-mon-a, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, ceph=True) 2024-04-26T21:42:08.226 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 podman[28119]: 2024-04-26 21:42:08.214733575 +0000 UTC m=+0.377608703 container init bf7f10eb1c0959b58dac995a0b937d5e3e743d459deadcea0d4e15c595fa54bf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-mon-a, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424) 2024-04-26T21:42:08.226 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 podman[28119]: 2024-04-26 21:42:08.22465091 +0000 UTC m=+0.387526004 container start bf7f10eb1c0959b58dac995a0b937d5e3e743d459deadcea0d4e15c595fa54bf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-mon-a, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff) 2024-04-26T21:42:08.305 INFO:teuthology.orchestra.run.smithi072.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-04-26T21:42:08.575 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: set uid:gid to 167:167 (ceph:ceph) 2024-04-26T21:42:08.575 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: ceph version 19.0.0-3306-gbc612ffb (bc612ffb9c1d5bc27536bc917a705262a36d4387) squid (dev), process ceph-mon, pid 2 2024-04-26T21:42:08.575 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: pidfile_write: ignore empty --pid-file 2024-04-26T21:42:08.575 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: load: jerasure load: lrc 2024-04-26T21:42:08.575 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: RocksDB version: 7.9.2 2024-04-26T21:42:08.575 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: Git sha 0 2024-04-26T21:42:08.575 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: Compile date 2024-04-26 11:39:23 2024-04-26T21:42:08.575 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: DB SUMMARY 2024-04-26T21:42:08.575 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: DB Session ID: ZQYF5C8KAH77S5C7R2UY 2024-04-26T21:42:08.575 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: CURRENT file: CURRENT 2024-04-26T21:42:08.576 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: IDENTITY file: IDENTITY 2024-04-26T21:42:08.576 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-04-26T21:42:08.576 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-04-26T21:42:08.576 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 103878 ; 2024-04-26T21:42:08.576 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: Options.error_if_exists: 0 2024-04-26T21:42:08.576 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: Options.create_if_missing: 0 2024-04-26T21:42:08.576 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: Options.paranoid_checks: 1 2024-04-26T21:42:08.576 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-26T21:42:08.576 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-26T21:42:08.576 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-26T21:42:08.576 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: Options.env: 0x561c7916acc0 2024-04-26T21:42:08.576 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: Options.fs: PosixFileSystem 2024-04-26T21:42:08.576 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: Options.info_log: 0x561c921dddc0 2024-04-26T21:42:08.577 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: Options.max_file_opening_threads: 16 2024-04-26T21:42:08.577 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: Options.statistics: (nil) 2024-04-26T21:42:08.577 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: Options.use_fsync: 0 2024-04-26T21:42:08.577 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: Options.max_log_file_size: 0 2024-04-26T21:42:08.577 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-26T21:42:08.577 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-26T21:42:08.577 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: Options.keep_log_file_num: 1000 2024-04-26T21:42:08.577 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: Options.recycle_log_file_num: 0 2024-04-26T21:42:08.577 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: Options.allow_fallocate: 1 2024-04-26T21:42:08.577 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: Options.allow_mmap_reads: 0 2024-04-26T21:42:08.577 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: Options.allow_mmap_writes: 0 2024-04-26T21:42:08.577 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: Options.use_direct_reads: 0 2024-04-26T21:42:08.577 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-26T21:42:08.578 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: Options.create_missing_column_families: 0 2024-04-26T21:42:08.578 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: Options.db_log_dir: 2024-04-26T21:42:08.578 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: Options.wal_dir: 2024-04-26T21:42:08.578 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-26T21:42:08.578 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-26T21:42:08.578 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-26T21:42:08.578 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-26T21:42:08.578 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-26T21:42:08.578 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-26T21:42:08.578 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: Options.advise_random_on_open: 1 2024-04-26T21:42:08.579 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: Options.db_write_buffer_size: 0 2024-04-26T21:42:08.579 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: Options.write_buffer_manager: 0x561c921af0e0 2024-04-26T21:42:08.579 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-26T21:42:08.579 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-26T21:42:08.579 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-26T21:42:08.579 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: Options.rate_limiter: (nil) 2024-04-26T21:42:08.579 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-26T21:42:08.579 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: Options.wal_recovery_mode: 2 2024-04-26T21:42:08.579 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: Options.enable_thread_tracking: 0 2024-04-26T21:42:08.579 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: Options.enable_pipelined_write: 0 2024-04-26T21:42:08.579 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: Options.unordered_write: 0 2024-04-26T21:42:08.579 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-26T21:42:08.579 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-26T21:42:08.579 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-26T21:42:08.579 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-26T21:42:08.580 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: Options.row_cache: None 2024-04-26T21:42:08.580 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: Options.wal_filter: None 2024-04-26T21:42:08.580 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-26T21:42:08.580 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: Options.allow_ingest_behind: 0 2024-04-26T21:42:08.580 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: Options.two_write_queues: 0 2024-04-26T21:42:08.580 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: Options.manual_wal_flush: 0 2024-04-26T21:42:08.580 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: Options.wal_compression: 0 2024-04-26T21:42:08.580 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: Options.atomic_flush: 0 2024-04-26T21:42:08.580 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-26T21:42:08.580 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-26T21:42:08.580 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-26T21:42:08.580 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: Options.log_readahead_size: 0 2024-04-26T21:42:08.580 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-26T21:42:08.580 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: Options.best_efforts_recovery: 0 2024-04-26T21:42:08.580 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-26T21:42:08.581 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-26T21:42:08.581 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: Options.allow_data_in_errors: 0 2024-04-26T21:42:08.581 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: Options.db_host_id: __hostname__ 2024-04-26T21:42:08.581 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-26T21:42:08.581 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: Options.max_background_jobs: 2 2024-04-26T21:42:08.581 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: Options.max_background_compactions: -1 2024-04-26T21:42:08.581 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: Options.max_subcompactions: 1 2024-04-26T21:42:08.582 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-26T21:42:08.582 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-26T21:42:08.582 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-26T21:42:08.582 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: Options.max_total_wal_size: 0 2024-04-26T21:42:08.582 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-26T21:42:08.582 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-26T21:42:08.582 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-26T21:42:08.582 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-26T21:42:08.582 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: Options.max_open_files: -1 2024-04-26T21:42:08.582 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: Options.bytes_per_sync: 0 2024-04-26T21:42:08.582 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-26T21:42:08.582 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-26T21:42:08.582 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: Options.compaction_readahead_size: 0 2024-04-26T21:42:08.582 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: Options.max_background_flushes: -1 2024-04-26T21:42:08.583 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: Compression algorithms supported: 2024-04-26T21:42:08.583 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-26T21:42:08.583 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: kZSTD supported: 0 2024-04-26T21:42:08.583 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: kXpressCompression supported: 0 2024-04-26T21:42:08.583 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: kLZ4HCCompression supported: 1 2024-04-26T21:42:08.583 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: kZlibCompression supported: 1 2024-04-26T21:42:08.583 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: kSnappyCompression supported: 1 2024-04-26T21:42:08.583 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: kLZ4Compression supported: 1 2024-04-26T21:42:08.583 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: kBZip2Compression supported: 0 2024-04-26T21:42:08.583 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-26T21:42:08.583 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-26T21:42:08.583 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-04-26T21:42:08.583 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-26T21:42:08.583 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-26T21:42:08.584 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: Options.merge_operator: 2024-04-26T21:42:08.584 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: Options.compaction_filter: None 2024-04-26T21:42:08.584 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: Options.compaction_filter_factory: None 2024-04-26T21:42:08.584 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: Options.sst_partitioner_factory: None 2024-04-26T21:42:08.584 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-26T21:42:08.584 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-26T21:42:08.584 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x561c921ddf80) 2024-04-26T21:42:08.584 INFO:journalctl@ceph.mon.a.smithi072.stdout: cache_index_and_filter_blocks: 1 2024-04-26T21:42:08.584 INFO:journalctl@ceph.mon.a.smithi072.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-26T21:42:08.584 INFO:journalctl@ceph.mon.a.smithi072.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-26T21:42:08.584 INFO:journalctl@ceph.mon.a.smithi072.stdout: pin_top_level_index_and_filter: 1 2024-04-26T21:42:08.584 INFO:journalctl@ceph.mon.a.smithi072.stdout: index_type: 0 2024-04-26T21:42:08.584 INFO:journalctl@ceph.mon.a.smithi072.stdout: data_block_index_type: 0 2024-04-26T21:42:08.585 INFO:journalctl@ceph.mon.a.smithi072.stdout: index_shortening: 1 2024-04-26T21:42:08.585 INFO:journalctl@ceph.mon.a.smithi072.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-26T21:42:08.585 INFO:journalctl@ceph.mon.a.smithi072.stdout: checksum: 4 2024-04-26T21:42:08.585 INFO:journalctl@ceph.mon.a.smithi072.stdout: no_block_cache: 0 2024-04-26T21:42:08.585 INFO:journalctl@ceph.mon.a.smithi072.stdout: block_cache: 0x561c921e11f0 2024-04-26T21:42:08.585 INFO:journalctl@ceph.mon.a.smithi072.stdout: block_cache_name: BinnedLRUCache 2024-04-26T21:42:08.585 INFO:journalctl@ceph.mon.a.smithi072.stdout: block_cache_options: 2024-04-26T21:42:08.585 INFO:journalctl@ceph.mon.a.smithi072.stdout: capacity : 536870912 2024-04-26T21:42:08.585 INFO:journalctl@ceph.mon.a.smithi072.stdout: num_shard_bits : 4 2024-04-26T21:42:08.585 INFO:journalctl@ceph.mon.a.smithi072.stdout: strict_capacity_limit : 0 2024-04-26T21:42:08.585 INFO:journalctl@ceph.mon.a.smithi072.stdout: high_pri_pool_ratio: 0.000 2024-04-26T21:42:08.585 INFO:journalctl@ceph.mon.a.smithi072.stdout: block_cache_compressed: (nil) 2024-04-26T21:42:08.585 INFO:journalctl@ceph.mon.a.smithi072.stdout: persistent_cache: (nil) 2024-04-26T21:42:08.585 INFO:journalctl@ceph.mon.a.smithi072.stdout: block_size: 4096 2024-04-26T21:42:08.586 INFO:journalctl@ceph.mon.a.smithi072.stdout: block_size_deviation: 10 2024-04-26T21:42:08.586 INFO:journalctl@ceph.mon.a.smithi072.stdout: block_restart_interval: 16 2024-04-26T21:42:08.586 INFO:journalctl@ceph.mon.a.smithi072.stdout: index_block_restart_interval: 1 2024-04-26T21:42:08.586 INFO:journalctl@ceph.mon.a.smithi072.stdout: metadata_block_size: 4096 2024-04-26T21:42:08.586 INFO:journalctl@ceph.mon.a.smithi072.stdout: partition_filters: 0 2024-04-26T21:42:08.586 INFO:journalctl@ceph.mon.a.smithi072.stdout: use_delta_encoding: 1 2024-04-26T21:42:08.586 INFO:journalctl@ceph.mon.a.smithi072.stdout: filter_policy: bloomfilter 2024-04-26T21:42:08.586 INFO:journalctl@ceph.mon.a.smithi072.stdout: whole_key_filtering: 1 2024-04-26T21:42:08.587 INFO:journalctl@ceph.mon.a.smithi072.stdout: verify_compression: 0 2024-04-26T21:42:08.587 INFO:journalctl@ceph.mon.a.smithi072.stdout: read_amp_bytes_per_bit: 0 2024-04-26T21:42:08.587 INFO:journalctl@ceph.mon.a.smithi072.stdout: format_version: 5 2024-04-26T21:42:08.587 INFO:journalctl@ceph.mon.a.smithi072.stdout: enable_index_compression: 1 2024-04-26T21:42:08.587 INFO:journalctl@ceph.mon.a.smithi072.stdout: block_align: 0 2024-04-26T21:42:08.587 INFO:journalctl@ceph.mon.a.smithi072.stdout: max_auto_readahead_size: 262144 2024-04-26T21:42:08.587 INFO:journalctl@ceph.mon.a.smithi072.stdout: prepopulate_block_cache: 0 2024-04-26T21:42:08.587 INFO:journalctl@ceph.mon.a.smithi072.stdout: initial_auto_readahead_size: 8192 2024-04-26T21:42:08.587 INFO:journalctl@ceph.mon.a.smithi072.stdout: num_file_reads_for_auto_readahead: 2 2024-04-26T21:42:08.587 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: Options.write_buffer_size: 33554432 2024-04-26T21:42:08.587 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: Options.max_write_buffer_number: 2 2024-04-26T21:42:08.587 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: Options.compression: NoCompression 2024-04-26T21:42:08.587 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: Options.bottommost_compression: Disabled 2024-04-26T21:42:08.587 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: Options.prefix_extractor: nullptr 2024-04-26T21:42:08.588 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-26T21:42:08.588 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: Options.num_levels: 7 2024-04-26T21:42:08.588 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-26T21:42:08.588 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-26T21:42:08.588 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-26T21:42:08.588 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-26T21:42:08.588 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-26T21:42:08.588 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-26T21:42:08.588 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-26T21:42:08.588 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-26T21:42:08.589 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-26T21:42:08.589 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-26T21:42:08.589 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-26T21:42:08.589 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-26T21:42:08.589 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-26T21:42:08.589 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: Options.compression_opts.level: 32767 2024-04-26T21:42:08.590 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: Options.compression_opts.strategy: 0 2024-04-26T21:42:08.590 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-26T21:42:08.590 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-26T21:42:08.590 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-26T21:42:08.590 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-26T21:42:08.590 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: Options.compression_opts.enabled: false 2024-04-26T21:42:08.590 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-26T21:42:08.590 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-26T21:42:08.590 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-26T21:42:08.590 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-26T21:42:08.590 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: Options.target_file_size_base: 67108864 2024-04-26T21:42:08.590 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-26T21:42:08.590 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-26T21:42:08.591 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-26T21:42:08.591 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-26T21:42:08.591 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-26T21:42:08.591 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-26T21:42:08.591 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-26T21:42:08.591 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-26T21:42:08.591 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-26T21:42:08.591 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-26T21:42:08.592 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-26T21:42:08.592 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-26T21:42:08.592 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-26T21:42:08.592 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-26T21:42:08.592 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: Options.arena_block_size: 1048576 2024-04-26T21:42:08.592 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-26T21:42:08.592 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-26T21:42:08.592 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: Options.disable_auto_compactions: 0 2024-04-26T21:42:08.592 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-26T21:42:08.592 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-26T21:42:08.592 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-26T21:42:08.592 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-26T21:42:08.592 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-26T21:42:08.592 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-26T21:42:08.593 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-26T21:42:08.593 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-26T21:42:08.593 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-26T21:42:08.593 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-26T21:42:08.593 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-04-26T21:42:08.593 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: Options.inplace_update_support: 0 2024-04-26T21:42:08.593 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-26T21:42:08.593 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-26T21:42:08.593 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-26T21:42:08.593 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-26T21:42:08.593 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: Options.bloom_locality: 0 2024-04-26T21:42:08.594 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: Options.max_successive_merges: 0 2024-04-26T21:42:08.594 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-26T21:42:08.594 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: Options.paranoid_file_checks: 0 2024-04-26T21:42:08.594 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: Options.force_consistency_checks: 1 2024-04-26T21:42:08.594 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: Options.report_bg_io_stats: 0 2024-04-26T21:42:08.594 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: Options.ttl: 2592000 2024-04-26T21:42:08.594 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-26T21:42:08.594 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-26T21:42:08.594 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-26T21:42:08.594 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: Options.enable_blob_files: false 2024-04-26T21:42:08.595 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: Options.min_blob_size: 0 2024-04-26T21:42:08.595 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: Options.blob_file_size: 268435456 2024-04-26T21:42:08.595 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-26T21:42:08.595 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-26T21:42:08.595 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-26T21:42:08.595 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-26T21:42:08.595 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-26T21:42:08.595 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: Options.blob_file_starting_level: 0 2024-04-26T21:42:08.595 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-26T21:42:08.595 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-04-26T21:42:08.595 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-04-26T21:42:08.595 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: ec0d1fb1-5ab7-4a50-9a7d-3457ea06affa 2024-04-26T21:42:08.595 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714167728281589, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-04-26T21:42:08.595 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-04-26T21:42:08.596 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714167728283592, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 99151, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 322, "table_properties": {"data_size": 97110, "index_size": 297, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 709, "raw_key_size": 13885, "raw_average_key_size": 49, "raw_value_size": 89478, "raw_average_value_size": 318, "num_data_blocks": 12, "num_entries": 281, "num_filter_entries": 281, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1714167728, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "ec0d1fb1-5ab7-4a50-9a7d-3457ea06affa", "db_session_id": "ZQYF5C8KAH77S5C7R2UY", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-04-26T21:42:08.596 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714167728283756, "job": 1, "event": "recovery_finished"} 2024-04-26T21:42:08.596 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-04-26T21:42:08.596 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-26T21:42:08.596 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x561c922fe000 2024-04-26T21:42:08.596 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: DB pointer 0x561c922ea000 2024-04-26T21:42:08.596 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-26T21:42:08.596 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-26T21:42:08.596 INFO:journalctl@ceph.mon.a.smithi072.stdout: ** DB Stats ** 2024-04-26T21:42:08.596 INFO:journalctl@ceph.mon.a.smithi072.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-26T21:42:08.596 INFO:journalctl@ceph.mon.a.smithi072.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-26T21:42:08.596 INFO:journalctl@ceph.mon.a.smithi072.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-26T21:42:08.597 INFO:journalctl@ceph.mon.a.smithi072.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-26T21:42:08.597 INFO:journalctl@ceph.mon.a.smithi072.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-26T21:42:08.597 INFO:journalctl@ceph.mon.a.smithi072.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-26T21:42:08.597 INFO:journalctl@ceph.mon.a.smithi072.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-26T21:42:08.597 INFO:journalctl@ceph.mon.a.smithi072.stdout: 2024-04-26T21:42:08.597 INFO:journalctl@ceph.mon.a.smithi072.stdout: ** Compaction Stats [default] ** 2024-04-26T21:42:08.597 INFO:journalctl@ceph.mon.a.smithi072.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-26T21:42:08.597 INFO:journalctl@ceph.mon.a.smithi072.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-26T21:42:08.597 INFO:journalctl@ceph.mon.a.smithi072.stdout: L0 2/0 98.69 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 75.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-26T21:42:08.597 INFO:journalctl@ceph.mon.a.smithi072.stdout: Sum 2/0 98.69 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 75.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-26T21:42:08.597 INFO:journalctl@ceph.mon.a.smithi072.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 75.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-26T21:42:08.597 INFO:journalctl@ceph.mon.a.smithi072.stdout: 2024-04-26T21:42:08.597 INFO:journalctl@ceph.mon.a.smithi072.stdout: ** Compaction Stats [default] ** 2024-04-26T21:42:08.597 INFO:journalctl@ceph.mon.a.smithi072.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-26T21:42:08.598 INFO:journalctl@ceph.mon.a.smithi072.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-26T21:42:08.598 INFO:journalctl@ceph.mon.a.smithi072.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 75.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-26T21:42:08.598 INFO:journalctl@ceph.mon.a.smithi072.stdout: 2024-04-26T21:42:08.598 INFO:journalctl@ceph.mon.a.smithi072.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-26T21:42:08.598 INFO:journalctl@ceph.mon.a.smithi072.stdout: 2024-04-26T21:42:08.598 INFO:journalctl@ceph.mon.a.smithi072.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-26T21:42:08.598 INFO:journalctl@ceph.mon.a.smithi072.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-26T21:42:08.598 INFO:journalctl@ceph.mon.a.smithi072.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-26T21:42:08.598 INFO:journalctl@ceph.mon.a.smithi072.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-26T21:42:08.598 INFO:journalctl@ceph.mon.a.smithi072.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-26T21:42:08.598 INFO:journalctl@ceph.mon.a.smithi072.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-26T21:42:08.598 INFO:journalctl@ceph.mon.a.smithi072.stdout: Cumulative compaction: 0.00 GB write, 13.87 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-26T21:42:08.598 INFO:journalctl@ceph.mon.a.smithi072.stdout: Interval compaction: 0.00 GB write, 13.87 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-26T21:42:08.598 INFO:journalctl@ceph.mon.a.smithi072.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-26T21:42:08.598 INFO:journalctl@ceph.mon.a.smithi072.stdout: Block cache BinnedLRUCache@0x561c921e11f0#2 capacity: 512.00 MB usage: 1.39 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.2e-05 secs_since: 0 2024-04-26T21:42:08.599 INFO:journalctl@ceph.mon.a.smithi072.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.89 KB,0.000169873%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-04-26T21:42:08.599 INFO:journalctl@ceph.mon.a.smithi072.stdout: 2024-04-26T21:42:08.599 INFO:journalctl@ceph.mon.a.smithi072.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-26T21:42:08.599 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: starting mon.a rank 0 at public addrs [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] at bind addrs [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid a1780950-0415-11ef-bc93-c7b262605968 2024-04-26T21:42:08.599 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: mon.a@-1(???) e1 preinit fsid a1780950-0415-11ef-bc93-c7b262605968 2024-04-26T21:42:08.599 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: mon.a@-1(???).mds e1 new map 2024-04-26T21:42:08.599 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: mon.a@-1(???).mds e1 print_map 2024-04-26T21:42:08.599 INFO:journalctl@ceph.mon.a.smithi072.stdout: e1 2024-04-26T21:42:08.599 INFO:journalctl@ceph.mon.a.smithi072.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-26T21:42:08.599 INFO:journalctl@ceph.mon.a.smithi072.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-26T21:42:08.599 INFO:journalctl@ceph.mon.a.smithi072.stdout: legacy client fscid: -1 2024-04-26T21:42:08.599 INFO:journalctl@ceph.mon.a.smithi072.stdout: 2024-04-26T21:42:08.599 INFO:journalctl@ceph.mon.a.smithi072.stdout: No filesystems configured 2024-04-26T21:42:08.599 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-26T21:42:08.599 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-26T21:42:08.600 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-26T21:42:08.600 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-26T21:42:08.600 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-04-26T21:42:08.600 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 bash[28119]: bf7f10eb1c0959b58dac995a0b937d5e3e743d459deadcea0d4e15c595fa54bf 2024-04-26T21:42:08.600 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 systemd[1]: Started Ceph mon.a for a1780950-0415-11ef-bc93-c7b262605968. 2024-04-26T21:42:08.600 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-26T21:42:08.600 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: monmap epoch 1 2024-04-26T21:42:08.600 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: fsid a1780950-0415-11ef-bc93-c7b262605968 2024-04-26T21:42:08.600 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: last_changed 2024-04-26T21:41:57.024587+0000 2024-04-26T21:42:08.600 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: created 2024-04-26T21:41:57.024587+0000 2024-04-26T21:42:08.600 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: min_mon_release 19 (squid) 2024-04-26T21:42:08.600 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: election_strategy: 1 2024-04-26T21:42:08.600 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: 0: [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] mon.a 2024-04-26T21:42:08.600 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: fsmap 2024-04-26T21:42:08.601 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: osdmap e1: 0 total, 0 up, 0 in 2024-04-26T21:42:08.601 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:08 smithi072 ceph-mon[28138]: mgrmap e1: no daemons active 2024-04-26T21:42:09.632 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:08.895+0000 7fca56400700 1 Processor -- start 2024-04-26T21:42:09.632 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:08.895+0000 7fca56400700 1 -- start start 2024-04-26T21:42:09.632 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:08.896+0000 7fca56400700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fca5010fc60 0x7fca50112050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:09.632 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:08.896+0000 7fca56400700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fca500834c0 con 0x7fca5010fc60 2024-04-26T21:42:09.632 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:08.897+0000 7fca4f400700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fca5010fc60 0x7fca50112050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:09.632 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:08.897+0000 7fca4f400700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fca5010fc60 0x7fca50112050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:60708/0 (socket says 172.21.15.72:60708) 2024-04-26T21:42:09.632 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:08.897+0000 7fca4f400700 1 -- 172.21.15.72:0/449968980 learned_addr learned my addr 172.21.15.72:0/449968980 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:42:09.632 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:08.897+0000 7fca4f400700 1 -- 172.21.15.72:0/449968980 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca50112590 con 0x7fca5010fc60 2024-04-26T21:42:09.632 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:08.898+0000 7fca4f400700 1 --2- 172.21.15.72:0/449968980 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fca5010fc60 0x7fca50112050 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fca400098a0 tx=0x7fca40009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90447365bad8ab5d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:09.632 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:08.898+0000 7fca4e000700 1 -- 172.21.15.72:0/449968980 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fca40004160 con 0x7fca5010fc60 2024-04-26T21:42:09.633 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:08.898+0000 7fca4e000700 1 -- 172.21.15.72:0/449968980 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fca40004300 con 0x7fca5010fc60 2024-04-26T21:42:09.633 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:08.898+0000 7fca4e000700 1 -- 172.21.15.72:0/449968980 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fca400119e0 con 0x7fca5010fc60 2024-04-26T21:42:09.633 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:08.899+0000 7fca56400700 1 -- 172.21.15.72:0/449968980 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fca5010fc60 msgr2=0x7fca50112050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:09.633 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:08.899+0000 7fca56400700 1 --2- 172.21.15.72:0/449968980 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fca5010fc60 0x7fca50112050 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fca400098a0 tx=0x7fca40009bb0 comp rx=0 tx=0).stop 2024-04-26T21:42:09.633 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:08.899+0000 7fca56400700 1 -- 172.21.15.72:0/449968980 shutdown_connections 2024-04-26T21:42:09.633 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:08.899+0000 7fca56400700 1 --2- 172.21.15.72:0/449968980 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fca5010fc60 0x7fca50112050 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:09.633 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:08.899+0000 7fca56400700 1 -- 172.21.15.72:0/449968980 >> 172.21.15.72:0/449968980 conn(0x7fca5007e220 msgr2=0x7fca50080660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:09.633 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:08.899+0000 7fca56400700 1 -- 172.21.15.72:0/449968980 shutdown_connections 2024-04-26T21:42:09.633 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:08.899+0000 7fca56400700 1 -- 172.21.15.72:0/449968980 wait complete. 2024-04-26T21:42:09.633 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:08.900+0000 7fca56400700 1 Processor -- start 2024-04-26T21:42:09.633 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:08.900+0000 7fca56400700 1 -- start start 2024-04-26T21:42:09.633 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:08.901+0000 7fca56400700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fca5010fc60 0x7fca501a30e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:09.634 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:08.901+0000 7fca56400700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fca50113560 con 0x7fca5010fc60 2024-04-26T21:42:09.634 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:08.901+0000 7fca4f400700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fca5010fc60 0x7fca501a30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:09.634 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:08.901+0000 7fca4f400700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fca5010fc60 0x7fca501a30e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:60716/0 (socket says 172.21.15.72:60716) 2024-04-26T21:42:09.634 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:08.901+0000 7fca4f400700 1 -- 172.21.15.72:0/1191031455 learned_addr learned my addr 172.21.15.72:0/1191031455 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:42:09.634 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:08.901+0000 7fca4f400700 1 -- 172.21.15.72:0/1191031455 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca501a3620 con 0x7fca5010fc60 2024-04-26T21:42:09.634 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:08.902+0000 7fca4f400700 1 --2- 172.21.15.72:0/1191031455 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fca5010fc60 0x7fca501a30e0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fca400116a0 tx=0x7fca400046a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:09.634 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:08.902+0000 7fca47e00700 1 -- 172.21.15.72:0/1191031455 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fca40003b70 con 0x7fca5010fc60 2024-04-26T21:42:09.634 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:08.902+0000 7fca47e00700 1 -- 172.21.15.72:0/1191031455 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fca4001a8a0 con 0x7fca5010fc60 2024-04-26T21:42:09.634 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:08.902+0000 7fca56400700 1 -- 172.21.15.72:0/1191031455 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca501a38b0 con 0x7fca5010fc60 2024-04-26T21:42:09.634 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:08.902+0000 7fca47e00700 1 -- 172.21.15.72:0/1191031455 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fca40024890 con 0x7fca5010fc60 2024-04-26T21:42:09.634 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:08.902+0000 7fca56400700 1 -- 172.21.15.72:0/1191031455 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca501a65a0 con 0x7fca5010fc60 2024-04-26T21:42:09.635 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:08.903+0000 7fca47e00700 1 -- 172.21.15.72:0/1191031455 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fca40022020 con 0x7fca5010fc60 2024-04-26T21:42:09.635 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:08.903+0000 7fca47e00700 1 -- 172.21.15.72:0/1191031455 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fca400196c0 con 0x7fca5010fc60 2024-04-26T21:42:09.635 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:08.903+0000 7fca56400700 1 -- 172.21.15.72:0/1191031455 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca50064a80 con 0x7fca5010fc60 2024-04-26T21:42:09.635 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:08.907+0000 7fca47e00700 1 -- 172.21.15.72:0/1191031455 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fca4001aa50 con 0x7fca5010fc60 2024-04-26T21:42:09.635 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:08.982+0000 7fca56400700 1 -- 172.21.15.72:0/1191031455 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7fca501a6770 con 0x7fca5010fc60 2024-04-26T21:42:09.635 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:08.985+0000 7fca47e00700 1 -- 172.21.15.72:0/1191031455 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7fca4001a430 con 0x7fca5010fc60 2024-04-26T21:42:09.635 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:08.986+0000 7fca56400700 1 -- 172.21.15.72:0/1191031455 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fca5010fc60 msgr2=0x7fca501a30e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:09.635 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:08.986+0000 7fca56400700 1 --2- 172.21.15.72:0/1191031455 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fca5010fc60 0x7fca501a30e0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fca400116a0 tx=0x7fca400046a0 comp rx=0 tx=0).stop 2024-04-26T21:42:09.635 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:08.986+0000 7fca56400700 1 -- 172.21.15.72:0/1191031455 shutdown_connections 2024-04-26T21:42:09.635 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:08.986+0000 7fca56400700 1 --2- 172.21.15.72:0/1191031455 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fca5010fc60 0x7fca501a30e0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:09.635 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:08.986+0000 7fca56400700 1 -- 172.21.15.72:0/1191031455 >> 172.21.15.72:0/1191031455 conn(0x7fca5007e220 msgr2=0x7fca5007ecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:09.636 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:08.986+0000 7fca56400700 1 -- 172.21.15.72:0/1191031455 shutdown_connections 2024-04-26T21:42:09.636 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:08.986+0000 7fca56400700 1 -- 172.21.15.72:0/1191031455 wait complete. 2024-04-26T21:42:09.636 INFO:teuthology.orchestra.run.smithi072.stdout:Wrote config to /etc/ceph/ceph.conf 2024-04-26T21:42:09.650 INFO:teuthology.orchestra.run.smithi072.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-04-26T21:42:09.651 INFO:teuthology.orchestra.run.smithi072.stdout:Creating mgr... 2024-04-26T21:42:09.652 INFO:teuthology.orchestra.run.smithi072.stdout:Verifying port 0.0.0.0:9283 ... 2024-04-26T21:42:09.652 INFO:teuthology.orchestra.run.smithi072.stdout:Verifying port 0.0.0.0:8765 ... 2024-04-26T21:42:10.101 INFO:teuthology.orchestra.run.smithi072.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-a1780950-0415-11ef-bc93-c7b262605968@mgr.x 2024-04-26T21:42:10.101 INFO:teuthology.orchestra.run.smithi072.stdout:systemctl: stderr Failed to reset failed state of unit ceph-a1780950-0415-11ef-bc93-c7b262605968@mgr.x.service: Unit ceph-a1780950-0415-11ef-bc93-c7b262605968@mgr.x.service not loaded. 2024-04-26T21:42:10.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:09 smithi072 ceph-mon[28138]: from='client.? 172.21.15.72:0/1191031455' entity='client.admin' 2024-04-26T21:42:10.356 INFO:teuthology.orchestra.run.smithi072.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-a1780950-0415-11ef-bc93-c7b262605968.target.wants/ceph-a1780950-0415-11ef-bc93-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-a1780950-0415-11ef-bc93-c7b262605968@.service. 2024-04-26T21:42:10.760 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 26 21:42:10 smithi072 systemd[1]: Starting Ceph mgr.x for a1780950-0415-11ef-bc93-c7b262605968... 2024-04-26T21:42:11.021 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 26 21:42:10 smithi072 podman[28347]: 2024-04-26 21:42:10.665395478 +0000 UTC m=+0.037718629 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T21:42:11.021 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 26 21:42:10 smithi072 podman[28347]: 2024-04-26 21:42:10.782317322 +0000 UTC m=+0.154640495 container create dd3246b2bcf10b1d2d10f3c6483838fea328137bfe9dd2dcfd4189150d4fa274 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-mgr-x, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0) 2024-04-26T21:42:11.170 INFO:teuthology.orchestra.run.smithi072.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-26T21:42:11.170 INFO:teuthology.orchestra.run.smithi072.stdout:systemctl: stdout disabled 2024-04-26T21:42:11.188 INFO:teuthology.orchestra.run.smithi072.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-26T21:42:11.189 INFO:teuthology.orchestra.run.smithi072.stdout:systemctl: stdout inactive 2024-04-26T21:42:11.189 INFO:teuthology.orchestra.run.smithi072.stdout:firewalld.service is not enabled 2024-04-26T21:42:11.189 INFO:teuthology.orchestra.run.smithi072.stdout:Not possible to enable service . firewalld.service is not available 2024-04-26T21:42:11.212 INFO:teuthology.orchestra.run.smithi072.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-26T21:42:11.212 INFO:teuthology.orchestra.run.smithi072.stdout:systemctl: stdout disabled 2024-04-26T21:42:11.235 INFO:teuthology.orchestra.run.smithi072.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-26T21:42:11.235 INFO:teuthology.orchestra.run.smithi072.stdout:systemctl: stdout inactive 2024-04-26T21:42:11.235 INFO:teuthology.orchestra.run.smithi072.stdout:firewalld.service is not enabled 2024-04-26T21:42:11.235 INFO:teuthology.orchestra.run.smithi072.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-04-26T21:42:11.235 INFO:teuthology.orchestra.run.smithi072.stdout:Waiting for mgr to start... 2024-04-26T21:42:11.235 INFO:teuthology.orchestra.run.smithi072.stdout:Waiting for mgr... 2024-04-26T21:42:11.296 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 26 21:42:11 smithi072 podman[28347]: 2024-04-26 21:42:11.04581863 +0000 UTC m=+0.418141810 container init dd3246b2bcf10b1d2d10f3c6483838fea328137bfe9dd2dcfd4189150d4fa274 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-mgr-x, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-04-26T21:42:11.296 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 26 21:42:11 smithi072 podman[28347]: 2024-04-26 21:42:11.055223112 +0000 UTC m=+0.427546262 container start dd3246b2bcf10b1d2d10f3c6483838fea328137bfe9dd2dcfd4189150d4fa274 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-mgr-x, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux ) 2024-04-26T21:42:11.297 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 26 21:42:11 smithi072 bash[28347]: dd3246b2bcf10b1d2d10f3c6483838fea328137bfe9dd2dcfd4189150d4fa274 2024-04-26T21:42:11.297 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 26 21:42:11 smithi072 systemd[1]: Started Ceph mgr.x for a1780950-0415-11ef-bc93-c7b262605968. 2024-04-26T21:42:11.570 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 26 21:42:11 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-mgr-x[28363]: 2024-04-26T21:42:11.295+0000 7febed244200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-26T21:42:11.570 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 26 21:42:11 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-mgr-x[28363]: 2024-04-26T21:42:11.541+0000 7febed244200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-26T21:42:11.822 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 26 21:42:11 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-mgr-x[28363]: 2024-04-26T21:42:11.675+0000 7febed244200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-26T21:42:12.090 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:11 smithi072 ceph-mon[28138]: from='client.? 172.21.15.72:0/1133769029' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-26T21:42:12.340 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 26 21:42:12 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-mgr-x[28363]: 2024-04-26T21:42:12.088+0000 7febed244200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-26T21:42:12.340 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 26 21:42:12 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-mgr-x[28363]: 2024-04-26T21:42:12.205+0000 7febed244200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-26T21:42:12.671 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout 2024-04-26T21:42:12.672 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout { 2024-04-26T21:42:12.672 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "fsid": "a1780950-0415-11ef-bc93-c7b262605968", 2024-04-26T21:42:12.672 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "health": { 2024-04-26T21:42:12.672 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-26T21:42:12.674 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-26T21:42:12.674 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-26T21:42:12.674 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout }, 2024-04-26T21:42:12.674 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-26T21:42:12.674 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-26T21:42:12.674 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout 0 2024-04-26T21:42:12.674 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout ], 2024-04-26T21:42:12.674 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-26T21:42:12.674 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "a" 2024-04-26T21:42:12.675 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout ], 2024-04-26T21:42:12.675 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "quorum_age": 3, 2024-04-26T21:42:12.676 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-26T21:42:12.676 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T21:42:12.676 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-26T21:42:12.676 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-26T21:42:12.677 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout }, 2024-04-26T21:42:12.677 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-26T21:42:12.677 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T21:42:12.677 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-26T21:42:12.677 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-26T21:42:12.677 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-26T21:42:12.686 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-26T21:42:12.686 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-26T21:42:12.686 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-26T21:42:12.686 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout }, 2024-04-26T21:42:12.687 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-26T21:42:12.687 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-26T21:42:12.687 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-26T21:42:12.687 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-26T21:42:12.687 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-26T21:42:12.687 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-26T21:42:12.687 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-26T21:42:12.687 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-26T21:42:12.687 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-26T21:42:12.687 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout }, 2024-04-26T21:42:12.687 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-26T21:42:12.687 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T21:42:12.687 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-26T21:42:12.688 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-26T21:42:12.688 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout }, 2024-04-26T21:42:12.688 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-26T21:42:12.688 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-26T21:42:12.688 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-26T21:42:12.688 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-26T21:42:12.688 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-26T21:42:12.688 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-26T21:42:12.688 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "restful" 2024-04-26T21:42:12.688 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout ], 2024-04-26T21:42:12.688 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-26T21:42:12.688 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout }, 2024-04-26T21:42:12.688 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-26T21:42:12.688 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T21:42:12.689 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "modified": "2024-04-26T21:42:01.504728+0000", 2024-04-26T21:42:12.689 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-26T21:42:12.689 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout }, 2024-04-26T21:42:12.689 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-26T21:42:12.689 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout } 2024-04-26T21:42:12.689 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:11.825+0000 7fef38400700 1 Processor -- start 2024-04-26T21:42:12.689 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:11.825+0000 7fef38400700 1 -- start start 2024-04-26T21:42:12.689 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:11.825+0000 7fef38400700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fef3007dd00 0x7fef3007c160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:12.689 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:11.825+0000 7fef38400700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fef3007c730 con 0x7fef3007dd00 2024-04-26T21:42:12.689 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:11.826+0000 7fef35a00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fef3007dd00 0x7fef3007c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:12.689 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:11.826+0000 7fef35a00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fef3007dd00 0x7fef3007c160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:37360/0 (socket says 172.21.15.72:37360) 2024-04-26T21:42:12.689 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:11.826+0000 7fef35a00700 1 -- 172.21.15.72:0/3952073064 learned_addr learned my addr 172.21.15.72:0/3952073064 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:42:12.689 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:11.826+0000 7fef35a00700 1 -- 172.21.15.72:0/3952073064 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef3007c8b0 con 0x7fef3007dd00 2024-04-26T21:42:12.689 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:11.827+0000 7fef35a00700 1 --2- 172.21.15.72:0/3952073064 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fef3007dd00 0x7fef3007c160 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fef240098a0 tx=0x7fef24009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=174b6ffa561e780 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:12.690 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:11.827+0000 7fef2fe00700 1 -- 172.21.15.72:0/3952073064 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fef24004160 con 0x7fef3007dd00 2024-04-26T21:42:12.690 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:11.827+0000 7fef2fe00700 1 -- 172.21.15.72:0/3952073064 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fef24004300 con 0x7fef3007dd00 2024-04-26T21:42:12.690 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:11.827+0000 7fef2fe00700 1 -- 172.21.15.72:0/3952073064 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fef240119e0 con 0x7fef3007dd00 2024-04-26T21:42:12.690 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:11.828+0000 7fef38400700 1 -- 172.21.15.72:0/3952073064 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fef3007dd00 msgr2=0x7fef3007c160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:12.690 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:11.828+0000 7fef38400700 1 --2- 172.21.15.72:0/3952073064 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fef3007dd00 0x7fef3007c160 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fef240098a0 tx=0x7fef24009bb0 comp rx=0 tx=0).stop 2024-04-26T21:42:12.690 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:11.828+0000 7fef38400700 1 -- 172.21.15.72:0/3952073064 shutdown_connections 2024-04-26T21:42:12.690 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:11.828+0000 7fef38400700 1 --2- 172.21.15.72:0/3952073064 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fef3007dd00 0x7fef3007c160 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:12.690 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:11.828+0000 7fef38400700 1 -- 172.21.15.72:0/3952073064 >> 172.21.15.72:0/3952073064 conn(0x7fef30104230 msgr2=0x7fef301066b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:12.690 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:11.828+0000 7fef38400700 1 -- 172.21.15.72:0/3952073064 shutdown_connections 2024-04-26T21:42:12.690 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:11.828+0000 7fef38400700 1 -- 172.21.15.72:0/3952073064 wait complete. 2024-04-26T21:42:12.690 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:11.829+0000 7fef38400700 1 Processor -- start 2024-04-26T21:42:12.690 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:11.830+0000 7fef38400700 1 -- start start 2024-04-26T21:42:12.690 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:11.830+0000 7fef38400700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fef3007dd00 0x7fef3019ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:12.691 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:11.830+0000 7fef38400700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fef3010f1e0 con 0x7fef3007dd00 2024-04-26T21:42:12.691 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:11.830+0000 7fef35a00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fef3007dd00 0x7fef3019ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:12.691 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:11.830+0000 7fef35a00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fef3007dd00 0x7fef3019ece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:37376/0 (socket says 172.21.15.72:37376) 2024-04-26T21:42:12.691 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:11.830+0000 7fef35a00700 1 -- 172.21.15.72:0/1133769029 learned_addr learned my addr 172.21.15.72:0/1133769029 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:42:12.691 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:11.831+0000 7fef35a00700 1 -- 172.21.15.72:0/1133769029 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef3019f220 con 0x7fef3007dd00 2024-04-26T21:42:12.691 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:11.831+0000 7fef35a00700 1 --2- 172.21.15.72:0/1133769029 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fef3007dd00 0x7fef3019ece0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fef24009130 tx=0x7fef24011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:12.691 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:11.831+0000 7fef2e000700 1 -- 172.21.15.72:0/1133769029 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fef24003b20 con 0x7fef3007dd00 2024-04-26T21:42:12.691 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:11.831+0000 7fef2e000700 1 -- 172.21.15.72:0/1133769029 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fef2401a8a0 con 0x7fef3007dd00 2024-04-26T21:42:12.691 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:11.831+0000 7fef38400700 1 -- 172.21.15.72:0/1133769029 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef3019f4b0 con 0x7fef3007dd00 2024-04-26T21:42:12.691 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:11.831+0000 7fef2e000700 1 -- 172.21.15.72:0/1133769029 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fef24024810 con 0x7fef3007dd00 2024-04-26T21:42:12.691 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:11.831+0000 7fef38400700 1 -- 172.21.15.72:0/1133769029 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef301a2190 con 0x7fef3007dd00 2024-04-26T21:42:12.691 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:11.832+0000 7fef2e000700 1 -- 172.21.15.72:0/1133769029 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fef24022020 con 0x7fef3007dd00 2024-04-26T21:42:12.691 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:11.832+0000 7fef2e000700 1 -- 172.21.15.72:0/1133769029 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fef2402dd90 con 0x7fef3007dd00 2024-04-26T21:42:12.691 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:11.832+0000 7fef38400700 1 -- 172.21.15.72:0/1133769029 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef30064a80 con 0x7fef3007dd00 2024-04-26T21:42:12.691 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:11.835+0000 7fef2e000700 1 -- 172.21.15.72:0/1133769029 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fef24043ec0 con 0x7fef3007dd00 2024-04-26T21:42:12.692 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:11.916+0000 7fef38400700 1 -- 172.21.15.72:0/1133769029 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fef301a2360 con 0x7fef3007dd00 2024-04-26T21:42:12.692 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:11.916+0000 7fef2e000700 1 -- 172.21.15.72:0/1133769029 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7fef24032020 con 0x7fef3007dd00 2024-04-26T21:42:12.692 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:11.918+0000 7fef38400700 1 -- 172.21.15.72:0/1133769029 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fef3007dd00 msgr2=0x7fef3019ece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:12.692 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:11.918+0000 7fef38400700 1 --2- 172.21.15.72:0/1133769029 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fef3007dd00 0x7fef3019ece0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fef24009130 tx=0x7fef24011a60 comp rx=0 tx=0).stop 2024-04-26T21:42:12.692 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:11.918+0000 7fef38400700 1 -- 172.21.15.72:0/1133769029 shutdown_connections 2024-04-26T21:42:12.692 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:11.918+0000 7fef38400700 1 --2- 172.21.15.72:0/1133769029 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fef3007dd00 0x7fef3019ece0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:12.692 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:11.918+0000 7fef38400700 1 -- 172.21.15.72:0/1133769029 >> 172.21.15.72:0/1133769029 conn(0x7fef30104230 msgr2=0x7fef30104d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:12.692 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:11.918+0000 7fef38400700 1 -- 172.21.15.72:0/1133769029 shutdown_connections 2024-04-26T21:42:12.692 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:11.918+0000 7fef38400700 1 -- 172.21.15.72:0/1133769029 wait complete. 2024-04-26T21:42:12.692 INFO:teuthology.orchestra.run.smithi072.stdout:mgr not available, waiting (1/15)... 2024-04-26T21:42:14.072 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 26 21:42:13 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-mgr-x[28363]: 2024-04-26T21:42:13.788+0000 7febed244200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-26T21:42:14.072 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 26 21:42:14 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-mgr-x[28363]: 2024-04-26T21:42:14.014+0000 7febed244200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-26T21:42:14.572 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 26 21:42:14 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-mgr-x[28363]: 2024-04-26T21:42:14.228+0000 7febed244200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-26T21:42:14.969 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 26 21:42:14 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-mgr-x[28363]: 2024-04-26T21:42:14.602+0000 7febed244200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-26T21:42:14.969 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 26 21:42:14 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-mgr-x[28363]: 2024-04-26T21:42:14.735+0000 7febed244200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-26T21:42:15.249 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 26 21:42:15 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-mgr-x[28363]: 2024-04-26T21:42:15.002+0000 7febed244200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-26T21:42:15.572 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:15 smithi072 ceph-mon[28138]: from='client.? 172.21.15.72:0/3033473458' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-26T21:42:15.572 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 26 21:42:15 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-mgr-x[28363]: 2024-04-26T21:42:15.260+0000 7febed244200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-26T21:42:16.081 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout 2024-04-26T21:42:16.082 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout { 2024-04-26T21:42:16.083 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "fsid": "a1780950-0415-11ef-bc93-c7b262605968", 2024-04-26T21:42:16.083 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "health": { 2024-04-26T21:42:16.083 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-26T21:42:16.083 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-26T21:42:16.083 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-26T21:42:16.083 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout }, 2024-04-26T21:42:16.085 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-26T21:42:16.085 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-26T21:42:16.085 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout 0 2024-04-26T21:42:16.085 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout ], 2024-04-26T21:42:16.086 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-26T21:42:16.086 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "a" 2024-04-26T21:42:16.086 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout ], 2024-04-26T21:42:16.086 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "quorum_age": 7, 2024-04-26T21:42:16.086 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-26T21:42:16.086 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T21:42:16.086 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-26T21:42:16.086 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-26T21:42:16.086 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout }, 2024-04-26T21:42:16.086 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-26T21:42:16.086 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T21:42:16.086 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-26T21:42:16.087 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-26T21:42:16.087 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-26T21:42:16.095 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-26T21:42:16.095 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-26T21:42:16.095 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-26T21:42:16.095 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout }, 2024-04-26T21:42:16.095 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-26T21:42:16.096 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-26T21:42:16.096 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-26T21:42:16.096 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-26T21:42:16.096 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-26T21:42:16.096 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-26T21:42:16.096 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-26T21:42:16.096 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-26T21:42:16.096 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-26T21:42:16.096 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout }, 2024-04-26T21:42:16.096 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-26T21:42:16.096 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T21:42:16.096 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-26T21:42:16.096 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-26T21:42:16.096 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout }, 2024-04-26T21:42:16.096 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-26T21:42:16.097 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-26T21:42:16.097 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-26T21:42:16.097 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-26T21:42:16.097 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-26T21:42:16.097 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-26T21:42:16.097 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "restful" 2024-04-26T21:42:16.097 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout ], 2024-04-26T21:42:16.097 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-26T21:42:16.097 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout }, 2024-04-26T21:42:16.097 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-26T21:42:16.097 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T21:42:16.097 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "modified": "2024-04-26T21:42:01.504728+0000", 2024-04-26T21:42:16.097 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-26T21:42:16.098 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout }, 2024-04-26T21:42:16.098 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-26T21:42:16.098 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout } 2024-04-26T21:42:16.098 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:15.276+0000 7f1583600700 1 Processor -- start 2024-04-26T21:42:16.098 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:15.276+0000 7f1583600700 1 -- start start 2024-04-26T21:42:16.098 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:15.277+0000 7f1583600700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f157c07dd00 0x7f157c07c160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:16.098 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:15.277+0000 7f1583600700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f157c07c730 con 0x7f157c07dd00 2024-04-26T21:42:16.098 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:15.277+0000 7f1580c00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f157c07dd00 0x7f157c07c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:16.098 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:15.277+0000 7f1580c00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f157c07dd00 0x7f157c07c160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:37384/0 (socket says 172.21.15.72:37384) 2024-04-26T21:42:16.098 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:15.277+0000 7f1580c00700 1 -- 172.21.15.72:0/1631989988 learned_addr learned my addr 172.21.15.72:0/1631989988 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:42:16.098 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:15.278+0000 7f1580c00700 1 -- 172.21.15.72:0/1631989988 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f157c07c8b0 con 0x7f157c07dd00 2024-04-26T21:42:16.098 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:15.278+0000 7f1580c00700 1 --2- 172.21.15.72:0/1631989988 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f157c07dd00 0x7f157c07c160 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f15700098a0 tx=0x7f1570009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=177bfbfdbbca2b2a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:16.099 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:15.278+0000 7f157b400700 1 -- 172.21.15.72:0/1631989988 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1570004160 con 0x7f157c07dd00 2024-04-26T21:42:16.099 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:15.278+0000 7f157b400700 1 -- 172.21.15.72:0/1631989988 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f1570004300 con 0x7f157c07dd00 2024-04-26T21:42:16.099 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:15.278+0000 7f157b400700 1 -- 172.21.15.72:0/1631989988 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f15700119e0 con 0x7f157c07dd00 2024-04-26T21:42:16.099 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:15.279+0000 7f1583600700 1 -- 172.21.15.72:0/1631989988 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f157c07dd00 msgr2=0x7f157c07c160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:16.099 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:15.279+0000 7f1583600700 1 --2- 172.21.15.72:0/1631989988 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f157c07dd00 0x7f157c07c160 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f15700098a0 tx=0x7f1570009bb0 comp rx=0 tx=0).stop 2024-04-26T21:42:16.099 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:15.279+0000 7f1583600700 1 -- 172.21.15.72:0/1631989988 shutdown_connections 2024-04-26T21:42:16.099 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:15.279+0000 7f1583600700 1 --2- 172.21.15.72:0/1631989988 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f157c07dd00 0x7f157c07c160 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:16.099 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:15.279+0000 7f1583600700 1 -- 172.21.15.72:0/1631989988 >> 172.21.15.72:0/1631989988 conn(0x7f157c104230 msgr2=0x7f157c1066b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:16.099 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:15.279+0000 7f1583600700 1 -- 172.21.15.72:0/1631989988 shutdown_connections 2024-04-26T21:42:16.099 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:15.279+0000 7f1583600700 1 -- 172.21.15.72:0/1631989988 wait complete. 2024-04-26T21:42:16.099 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:15.280+0000 7f1583600700 1 Processor -- start 2024-04-26T21:42:16.099 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:15.280+0000 7f1583600700 1 -- start start 2024-04-26T21:42:16.099 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:15.281+0000 7f1583600700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f157c07dd00 0x7f157c19ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:16.099 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:15.281+0000 7f1583600700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f157c10f1e0 con 0x7f157c07dd00 2024-04-26T21:42:16.100 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:15.281+0000 7f1580c00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f157c07dd00 0x7f157c19ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:16.100 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:15.281+0000 7f1580c00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f157c07dd00 0x7f157c19ece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:37388/0 (socket says 172.21.15.72:37388) 2024-04-26T21:42:16.100 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:15.281+0000 7f1580c00700 1 -- 172.21.15.72:0/3033473458 learned_addr learned my addr 172.21.15.72:0/3033473458 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:42:16.100 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:15.281+0000 7f1580c00700 1 -- 172.21.15.72:0/3033473458 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f157c19f220 con 0x7f157c07dd00 2024-04-26T21:42:16.100 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:15.282+0000 7f1580c00700 1 --2- 172.21.15.72:0/3033473458 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f157c07dd00 0x7f157c19ece0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1570009130 tx=0x7f1570011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:16.100 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:15.282+0000 7f1579600700 1 -- 172.21.15.72:0/3033473458 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1570003b20 con 0x7f157c07dd00 2024-04-26T21:42:16.100 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:15.282+0000 7f1579600700 1 -- 172.21.15.72:0/3033473458 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f157001a8a0 con 0x7f157c07dd00 2024-04-26T21:42:16.100 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:15.282+0000 7f1579600700 1 -- 172.21.15.72:0/3033473458 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1570024810 con 0x7f157c07dd00 2024-04-26T21:42:16.100 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:15.282+0000 7f1583600700 1 -- 172.21.15.72:0/3033473458 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f157c19f4b0 con 0x7f157c07dd00 2024-04-26T21:42:16.100 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:15.282+0000 7f1583600700 1 -- 172.21.15.72:0/3033473458 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f157c1a2190 con 0x7f157c07dd00 2024-04-26T21:42:16.100 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:15.282+0000 7f1579600700 1 -- 172.21.15.72:0/3033473458 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f1570022020 con 0x7f157c07dd00 2024-04-26T21:42:16.100 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:15.282+0000 7f1579600700 1 -- 172.21.15.72:0/3033473458 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f15700193c0 con 0x7f157c07dd00 2024-04-26T21:42:16.100 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:15.283+0000 7f1583600700 1 -- 172.21.15.72:0/3033473458 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f157c064a80 con 0x7f157c07dd00 2024-04-26T21:42:16.100 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:15.286+0000 7f1579600700 1 -- 172.21.15.72:0/3033473458 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f15700094f0 con 0x7f157c07dd00 2024-04-26T21:42:16.101 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:15.366+0000 7f1583600700 1 -- 172.21.15.72:0/3033473458 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f157c1a2360 con 0x7f157c07dd00 2024-04-26T21:42:16.101 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:15.367+0000 7f1579600700 1 -- 172.21.15.72:0/3033473458 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f157001ad70 con 0x7f157c07dd00 2024-04-26T21:42:16.101 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:15.368+0000 7f1583600700 1 -- 172.21.15.72:0/3033473458 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f157c07dd00 msgr2=0x7f157c19ece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:16.101 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:15.368+0000 7f1583600700 1 --2- 172.21.15.72:0/3033473458 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f157c07dd00 0x7f157c19ece0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1570009130 tx=0x7f1570011a60 comp rx=0 tx=0).stop 2024-04-26T21:42:16.101 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:15.368+0000 7f1583600700 1 -- 172.21.15.72:0/3033473458 shutdown_connections 2024-04-26T21:42:16.101 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:15.368+0000 7f1583600700 1 --2- 172.21.15.72:0/3033473458 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f157c07dd00 0x7f157c19ece0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:16.101 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:15.368+0000 7f1583600700 1 -- 172.21.15.72:0/3033473458 >> 172.21.15.72:0/3033473458 conn(0x7f157c104230 msgr2=0x7f157c104d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:16.101 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:15.368+0000 7f1583600700 1 -- 172.21.15.72:0/3033473458 shutdown_connections 2024-04-26T21:42:16.101 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:15.368+0000 7f1583600700 1 -- 172.21.15.72:0/3033473458 wait complete. 2024-04-26T21:42:16.102 INFO:teuthology.orchestra.run.smithi072.stdout:mgr not available, waiting (2/15)... 2024-04-26T21:42:16.322 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 26 21:42:16 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-mgr-x[28363]: 2024-04-26T21:42:16.067+0000 7febed244200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-26T21:42:17.014 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 26 21:42:16 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-mgr-x[28363]: 2024-04-26T21:42:16.619+0000 7febed244200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-26T21:42:17.014 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 26 21:42:16 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-mgr-x[28363]: 2024-04-26T21:42:16.748+0000 7febed244200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-26T21:42:17.014 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 26 21:42:17 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-mgr-x[28363]: 2024-04-26T21:42:17.012+0000 7febed244200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-26T21:42:18.072 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 26 21:42:17 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-mgr-x[28363]: 2024-04-26T21:42:17.653+0000 7febed244200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-26T21:42:18.072 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 26 21:42:17 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-mgr-x[28363]: 2024-04-26T21:42:17.776+0000 7febed244200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-26T21:42:18.410 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 26 21:42:18 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-mgr-x[28363]: 2024-04-26T21:42:18.098+0000 7febed244200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-26T21:42:18.411 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 26 21:42:18 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-mgr-x[28363]: 2024-04-26T21:42:18.280+0000 7febed244200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-26T21:42:18.411 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 26 21:42:18 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-mgr-x[28363]: 2024-04-26T21:42:18.409+0000 7febed244200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-26T21:42:18.680 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 26 21:42:18 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-mgr-x[28363]: 2024-04-26T21:42:18.562+0000 7febed244200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-26T21:42:18.946 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:18 smithi072 ceph-mon[28138]: from='client.? 172.21.15.72:0/333074971' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-26T21:42:19.244 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 26 21:42:18 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-mgr-x[28363]: 2024-04-26T21:42:18.945+0000 7febed244200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-26T21:42:19.532 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout 2024-04-26T21:42:19.532 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout { 2024-04-26T21:42:19.532 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "fsid": "a1780950-0415-11ef-bc93-c7b262605968", 2024-04-26T21:42:19.532 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "health": { 2024-04-26T21:42:19.532 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-26T21:42:19.532 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-26T21:42:19.532 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-26T21:42:19.532 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout }, 2024-04-26T21:42:19.532 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-26T21:42:19.533 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-26T21:42:19.533 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout 0 2024-04-26T21:42:19.533 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout ], 2024-04-26T21:42:19.533 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-26T21:42:19.533 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "a" 2024-04-26T21:42:19.533 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout ], 2024-04-26T21:42:19.533 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "quorum_age": 10, 2024-04-26T21:42:19.533 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-26T21:42:19.533 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T21:42:19.533 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-26T21:42:19.533 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-26T21:42:19.533 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout }, 2024-04-26T21:42:19.533 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-26T21:42:19.533 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T21:42:19.533 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-26T21:42:19.534 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-26T21:42:19.534 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-26T21:42:19.542 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-26T21:42:19.542 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-26T21:42:19.542 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-26T21:42:19.542 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout }, 2024-04-26T21:42:19.542 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-26T21:42:19.542 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-26T21:42:19.542 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-26T21:42:19.542 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-26T21:42:19.543 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-26T21:42:19.543 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-26T21:42:19.543 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-26T21:42:19.543 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-26T21:42:19.543 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-26T21:42:19.543 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout }, 2024-04-26T21:42:19.543 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-26T21:42:19.543 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T21:42:19.543 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-26T21:42:19.543 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-26T21:42:19.543 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout }, 2024-04-26T21:42:19.543 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-26T21:42:19.543 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-26T21:42:19.543 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-26T21:42:19.544 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-26T21:42:19.544 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-26T21:42:19.544 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-26T21:42:19.544 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "restful" 2024-04-26T21:42:19.544 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout ], 2024-04-26T21:42:19.544 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-26T21:42:19.544 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout }, 2024-04-26T21:42:19.544 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-26T21:42:19.544 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T21:42:19.544 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "modified": "2024-04-26T21:42:01.504728+0000", 2024-04-26T21:42:19.544 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-26T21:42:19.544 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout }, 2024-04-26T21:42:19.544 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-26T21:42:19.545 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout } 2024-04-26T21:42:19.545 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:18.731+0000 7f0605600700 1 Processor -- start 2024-04-26T21:42:19.545 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:18.732+0000 7f0605600700 1 -- start start 2024-04-26T21:42:19.545 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:18.732+0000 7f0605600700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0600108c80 0x7f06001090a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:19.545 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:18.732+0000 7f0605600700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f0600109670 con 0x7f0600108c80 2024-04-26T21:42:19.545 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:18.732+0000 7f05fea00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0600108c80 0x7f06001090a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:19.545 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:18.732+0000 7f05fea00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0600108c80 0x7f06001090a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:37400/0 (socket says 172.21.15.72:37400) 2024-04-26T21:42:19.545 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:18.732+0000 7f05fea00700 1 -- 172.21.15.72:0/3627571968 learned_addr learned my addr 172.21.15.72:0/3627571968 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:42:19.545 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:18.733+0000 7f05fea00700 1 -- 172.21.15.72:0/3627571968 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0600109e80 con 0x7f0600108c80 2024-04-26T21:42:19.545 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:18.733+0000 7f05fea00700 1 --2- 172.21.15.72:0/3627571968 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0600108c80 0x7f06001090a0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f05f00098a0 tx=0x7f05f0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15b534568029bc6c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:19.545 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:18.733+0000 7f05fd600700 1 -- 172.21.15.72:0/3627571968 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f05f0004160 con 0x7f0600108c80 2024-04-26T21:42:19.545 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:18.734+0000 7f05fd600700 1 -- 172.21.15.72:0/3627571968 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f05f0004300 con 0x7f0600108c80 2024-04-26T21:42:19.545 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:18.734+0000 7f05fd600700 1 -- 172.21.15.72:0/3627571968 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f05f00119e0 con 0x7f0600108c80 2024-04-26T21:42:19.546 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:18.734+0000 7f0605600700 1 -- 172.21.15.72:0/3627571968 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0600108c80 msgr2=0x7f06001090a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:19.546 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:18.734+0000 7f0605600700 1 --2- 172.21.15.72:0/3627571968 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0600108c80 0x7f06001090a0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f05f00098a0 tx=0x7f05f0009bb0 comp rx=0 tx=0).stop 2024-04-26T21:42:19.546 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:18.735+0000 7f0605600700 1 -- 172.21.15.72:0/3627571968 shutdown_connections 2024-04-26T21:42:19.546 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:18.735+0000 7f0605600700 1 --2- 172.21.15.72:0/3627571968 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0600108c80 0x7f06001090a0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:19.546 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:18.735+0000 7f0605600700 1 -- 172.21.15.72:0/3627571968 >> 172.21.15.72:0/3627571968 conn(0x7f06001041e0 msgr2=0x7f0600106660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:19.546 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:18.735+0000 7f0605600700 1 -- 172.21.15.72:0/3627571968 shutdown_connections 2024-04-26T21:42:19.546 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:18.735+0000 7f0605600700 1 -- 172.21.15.72:0/3627571968 wait complete. 2024-04-26T21:42:19.546 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:18.736+0000 7f0605600700 1 Processor -- start 2024-04-26T21:42:19.546 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:18.736+0000 7f0605600700 1 -- start start 2024-04-26T21:42:19.546 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:18.736+0000 7f0605600700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0600108c80 0x7f060019cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:19.546 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:18.736+0000 7f0605600700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f060010d0b0 con 0x7f0600108c80 2024-04-26T21:42:19.546 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:18.736+0000 7f05fea00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0600108c80 0x7f060019cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:19.546 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:18.736+0000 7f05fea00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0600108c80 0x7f060019cb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:37402/0 (socket says 172.21.15.72:37402) 2024-04-26T21:42:19.547 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:18.736+0000 7f05fea00700 1 -- 172.21.15.72:0/333074971 learned_addr learned my addr 172.21.15.72:0/333074971 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:42:19.547 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:18.737+0000 7f05fea00700 1 -- 172.21.15.72:0/333074971 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f060019d070 con 0x7f0600108c80 2024-04-26T21:42:19.547 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:18.737+0000 7f05fea00700 1 --2- 172.21.15.72:0/333074971 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0600108c80 0x7f060019cb30 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f05f00116a0 tx=0x7f05f0011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:19.547 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:18.737+0000 7f05f7400700 1 -- 172.21.15.72:0/333074971 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f05f00039f0 con 0x7f0600108c80 2024-04-26T21:42:19.547 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:18.737+0000 7f05f7400700 1 -- 172.21.15.72:0/333074971 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f05f0003b90 con 0x7f0600108c80 2024-04-26T21:42:19.547 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:18.737+0000 7f0605600700 1 -- 172.21.15.72:0/333074971 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f060019d300 con 0x7f0600108c80 2024-04-26T21:42:19.547 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:18.737+0000 7f05f7400700 1 -- 172.21.15.72:0/333074971 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f05f001a600 con 0x7f0600108c80 2024-04-26T21:42:19.547 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:18.738+0000 7f0605600700 1 -- 172.21.15.72:0/333074971 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f060019fff0 con 0x7f0600108c80 2024-04-26T21:42:19.547 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:18.738+0000 7f05f7400700 1 -- 172.21.15.72:0/333074971 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f05f001ac80 con 0x7f0600108c80 2024-04-26T21:42:19.547 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:18.738+0000 7f05f7400700 1 -- 172.21.15.72:0/333074971 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f05f0019700 con 0x7f0600108c80 2024-04-26T21:42:19.547 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:18.739+0000 7f0605600700 1 -- 172.21.15.72:0/333074971 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0600064a80 con 0x7f0600108c80 2024-04-26T21:42:19.547 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:18.743+0000 7f05f7400700 1 -- 172.21.15.72:0/333074971 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f05f0024a50 con 0x7f0600108c80 2024-04-26T21:42:19.547 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:18.824+0000 7f0605600700 1 -- 172.21.15.72:0/333074971 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f06001a0540 con 0x7f0600108c80 2024-04-26T21:42:19.548 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:18.824+0000 7f05f7400700 1 -- 172.21.15.72:0/333074971 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1240 (secure 0 0 0) 0x7f05f0024430 con 0x7f0600108c80 2024-04-26T21:42:19.548 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:18.826+0000 7f0605600700 1 -- 172.21.15.72:0/333074971 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0600108c80 msgr2=0x7f060019cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:19.548 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:18.826+0000 7f0605600700 1 --2- 172.21.15.72:0/333074971 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0600108c80 0x7f060019cb30 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f05f00116a0 tx=0x7f05f0011a60 comp rx=0 tx=0).stop 2024-04-26T21:42:19.548 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:18.826+0000 7f0605600700 1 -- 172.21.15.72:0/333074971 shutdown_connections 2024-04-26T21:42:19.548 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:18.826+0000 7f0605600700 1 --2- 172.21.15.72:0/333074971 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0600108c80 0x7f060019cb30 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:19.548 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:18.826+0000 7f0605600700 1 -- 172.21.15.72:0/333074971 >> 172.21.15.72:0/333074971 conn(0x7f06001041e0 msgr2=0x7f0600104d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:19.548 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:18.826+0000 7f0605600700 1 -- 172.21.15.72:0/333074971 shutdown_connections 2024-04-26T21:42:19.548 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:18.826+0000 7f0605600700 1 -- 172.21.15.72:0/333074971 wait complete. 2024-04-26T21:42:19.548 INFO:teuthology.orchestra.run.smithi072.stdout:mgr not available, waiting (3/15)... 2024-04-26T21:42:19.822 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 26 21:42:19 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-mgr-x[28363]: 2024-04-26T21:42:19.754+0000 7febed244200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-26T21:42:20.322 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 26 21:42:20 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-mgr-x[28363]: 2024-04-26T21:42:20.037+0000 7febed244200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-26T21:42:20.323 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:20 smithi072 ceph-mon[28138]: Activating manager daemon x 2024-04-26T21:42:20.323 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:20 smithi072 ceph-mon[28138]: mgrmap e2: x(active, starting, since 0.0120834s) 2024-04-26T21:42:20.323 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:20 smithi072 ceph-mon[28138]: from='mgr.14100 172.21.15.72:0/2064472021' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-26T21:42:20.323 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:20 smithi072 ceph-mon[28138]: from='mgr.14100 172.21.15.72:0/2064472021' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-26T21:42:20.323 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:20 smithi072 ceph-mon[28138]: from='mgr.14100 172.21.15.72:0/2064472021' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-26T21:42:20.323 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:20 smithi072 ceph-mon[28138]: from='mgr.14100 172.21.15.72:0/2064472021' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T21:42:20.323 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:20 smithi072 ceph-mon[28138]: from='mgr.14100 172.21.15.72:0/2064472021' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-26T21:42:20.323 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:20 smithi072 ceph-mon[28138]: Manager daemon x is now available 2024-04-26T21:42:20.323 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:20 smithi072 ceph-mon[28138]: from='mgr.14100 172.21.15.72:0/2064472021' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-26T21:42:20.324 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:20 smithi072 ceph-mon[28138]: from='mgr.14100 172.21.15.72:0/2064472021' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-26T21:42:21.572 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:21 smithi072 ceph-mon[28138]: from='mgr.14100 172.21.15.72:0/2064472021' entity='mgr.x' 2024-04-26T21:42:21.572 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:21 smithi072 ceph-mon[28138]: from='mgr.14100 172.21.15.72:0/2064472021' entity='mgr.x' 2024-04-26T21:42:21.572 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:21 smithi072 ceph-mon[28138]: from='mgr.14100 172.21.15.72:0/2064472021' entity='mgr.x' 2024-04-26T21:42:21.572 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:21 smithi072 ceph-mon[28138]: mgrmap e3: x(active, since 1.02307s) 2024-04-26T21:42:23.202 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:23 smithi072 ceph-mon[28138]: mgrmap e4: x(active, since 2s) 2024-04-26T21:42:23.203 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:23 smithi072 ceph-mon[28138]: from='client.? 172.21.15.72:0/3806264484' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-26T21:42:23.226 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout 2024-04-26T21:42:23.226 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout { 2024-04-26T21:42:23.226 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "fsid": "a1780950-0415-11ef-bc93-c7b262605968", 2024-04-26T21:42:23.226 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "health": { 2024-04-26T21:42:23.226 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-26T21:42:23.226 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-26T21:42:23.226 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-26T21:42:23.227 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout }, 2024-04-26T21:42:23.227 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-26T21:42:23.227 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-26T21:42:23.227 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout 0 2024-04-26T21:42:23.227 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout ], 2024-04-26T21:42:23.227 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-26T21:42:23.227 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "a" 2024-04-26T21:42:23.227 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout ], 2024-04-26T21:42:23.227 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "quorum_age": 14, 2024-04-26T21:42:23.227 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-26T21:42:23.227 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T21:42:23.227 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-26T21:42:23.227 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-26T21:42:23.227 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout }, 2024-04-26T21:42:23.228 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-26T21:42:23.228 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T21:42:23.228 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-26T21:42:23.228 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-26T21:42:23.228 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-26T21:42:23.228 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-26T21:42:23.228 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-26T21:42:23.228 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-26T21:42:23.228 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout }, 2024-04-26T21:42:23.228 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-26T21:42:23.228 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-26T21:42:23.228 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-26T21:42:23.228 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-26T21:42:23.228 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-26T21:42:23.229 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-26T21:42:23.229 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-26T21:42:23.229 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-26T21:42:23.229 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-26T21:42:23.229 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout }, 2024-04-26T21:42:23.229 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-26T21:42:23.229 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T21:42:23.229 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-26T21:42:23.229 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-26T21:42:23.229 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout }, 2024-04-26T21:42:23.229 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-26T21:42:23.229 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-26T21:42:23.229 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-26T21:42:23.229 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-26T21:42:23.230 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-26T21:42:23.230 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-26T21:42:23.230 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "restful" 2024-04-26T21:42:23.230 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout ], 2024-04-26T21:42:23.230 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-26T21:42:23.230 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout }, 2024-04-26T21:42:23.230 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-26T21:42:23.230 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T21:42:23.230 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "modified": "2024-04-26T21:42:01.504728+0000", 2024-04-26T21:42:23.230 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-26T21:42:23.230 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout }, 2024-04-26T21:42:23.230 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-26T21:42:23.230 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout } 2024-04-26T21:42:23.231 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:22.151+0000 7f31c0200700 1 Processor -- start 2024-04-26T21:42:23.231 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:22.151+0000 7f31c0200700 1 -- start start 2024-04-26T21:42:23.231 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:22.151+0000 7f31c0200700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f31b810fc60 0x7f31b8112050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:23.231 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:22.151+0000 7f31c0200700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f31b80834c0 con 0x7f31b810fc60 2024-04-26T21:42:23.231 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:22.152+0000 7f31bd800700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f31b810fc60 0x7f31b8112050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:23.231 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:22.152+0000 7f31bd800700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f31b810fc60 0x7f31b8112050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:34908/0 (socket says 172.21.15.72:34908) 2024-04-26T21:42:23.231 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:22.152+0000 7f31bd800700 1 -- 172.21.15.72:0/1153890679 learned_addr learned my addr 172.21.15.72:0/1153890679 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:42:23.231 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:22.152+0000 7f31bd800700 1 -- 172.21.15.72:0/1153890679 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31b8112590 con 0x7f31b810fc60 2024-04-26T21:42:23.231 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:22.153+0000 7f31bd800700 1 --2- 172.21.15.72:0/1153890679 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f31b810fc60 0x7f31b8112050 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f31ac0098a0 tx=0x7f31ac009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92ca9ddd1d5c1aff server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:23.231 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:22.153+0000 7f31b7e00700 1 -- 172.21.15.72:0/1153890679 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f31ac004160 con 0x7f31b810fc60 2024-04-26T21:42:23.231 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:22.153+0000 7f31b7e00700 1 -- 172.21.15.72:0/1153890679 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f31ac004300 con 0x7f31b810fc60 2024-04-26T21:42:23.231 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:22.153+0000 7f31b7e00700 1 -- 172.21.15.72:0/1153890679 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f31ac0119e0 con 0x7f31b810fc60 2024-04-26T21:42:23.231 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:22.154+0000 7f31c0200700 1 -- 172.21.15.72:0/1153890679 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f31b810fc60 msgr2=0x7f31b8112050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:23.231 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:22.154+0000 7f31c0200700 1 --2- 172.21.15.72:0/1153890679 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f31b810fc60 0x7f31b8112050 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f31ac0098a0 tx=0x7f31ac009bb0 comp rx=0 tx=0).stop 2024-04-26T21:42:23.232 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:22.154+0000 7f31c0200700 1 -- 172.21.15.72:0/1153890679 shutdown_connections 2024-04-26T21:42:23.232 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:22.154+0000 7f31c0200700 1 --2- 172.21.15.72:0/1153890679 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f31b810fc60 0x7f31b8112050 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:23.232 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:22.154+0000 7f31c0200700 1 -- 172.21.15.72:0/1153890679 >> 172.21.15.72:0/1153890679 conn(0x7f31b807e220 msgr2=0x7f31b8080660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:23.232 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:22.154+0000 7f31c0200700 1 -- 172.21.15.72:0/1153890679 shutdown_connections 2024-04-26T21:42:23.232 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:22.154+0000 7f31c0200700 1 -- 172.21.15.72:0/1153890679 wait complete. 2024-04-26T21:42:23.232 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:22.155+0000 7f31c0200700 1 Processor -- start 2024-04-26T21:42:23.232 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:22.155+0000 7f31c0200700 1 -- start start 2024-04-26T21:42:23.232 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:22.155+0000 7f31c0200700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f31b810fc60 0x7f31b81a30e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:23.232 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:22.155+0000 7f31c0200700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f31b8113560 con 0x7f31b810fc60 2024-04-26T21:42:23.232 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:22.156+0000 7f31bd800700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f31b810fc60 0x7f31b81a30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:23.232 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:22.156+0000 7f31bd800700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f31b810fc60 0x7f31b81a30e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:34918/0 (socket says 172.21.15.72:34918) 2024-04-26T21:42:23.232 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:22.156+0000 7f31bd800700 1 -- 172.21.15.72:0/3806264484 learned_addr learned my addr 172.21.15.72:0/3806264484 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:42:23.232 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:22.156+0000 7f31bd800700 1 -- 172.21.15.72:0/3806264484 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31b81a3620 con 0x7f31b810fc60 2024-04-26T21:42:23.233 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:22.156+0000 7f31bd800700 1 --2- 172.21.15.72:0/3806264484 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f31b810fc60 0x7f31b81a30e0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f31ac0119e0 tx=0x7f31ac0046a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:23.233 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:22.157+0000 7f31b6000700 1 -- 172.21.15.72:0/3806264484 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f31ac003bf0 con 0x7f31b810fc60 2024-04-26T21:42:23.233 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:22.157+0000 7f31b6000700 1 -- 172.21.15.72:0/3806264484 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f31ac01a8a0 con 0x7f31b810fc60 2024-04-26T21:42:23.233 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:22.157+0000 7f31c0200700 1 -- 172.21.15.72:0/3806264484 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31b81a38b0 con 0x7f31b810fc60 2024-04-26T21:42:23.233 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:22.157+0000 7f31b6000700 1 -- 172.21.15.72:0/3806264484 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f31ac0248c0 con 0x7f31b810fc60 2024-04-26T21:42:23.233 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:22.157+0000 7f31c0200700 1 -- 172.21.15.72:0/3806264484 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31b81a65a0 con 0x7f31b810fc60 2024-04-26T21:42:23.233 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:22.158+0000 7f31b6000700 1 -- 172.21.15.72:0/3806264484 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7f31ac022020 con 0x7f31b810fc60 2024-04-26T21:42:23.233 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:22.158+0000 7f31c0200700 1 -- 172.21.15.72:0/3806264484 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31b8064a80 con 0x7f31b810fc60 2024-04-26T21:42:23.233 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:22.158+0000 7f31b6000700 1 --2- 172.21.15.72:0/3806264484 >> [v2:172.21.15.72:6800/1012617371,v1:172.21.15.72:6801/1012617371] conn(0x7f31a00395b0 0x7f31a003ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:23.233 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:22.159+0000 7f31b6000700 1 -- 172.21.15.72:0/3806264484 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f31ac04d620 con 0x7f31b810fc60 2024-04-26T21:42:23.233 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:22.159+0000 7f31bce00700 1 --2- 172.21.15.72:0/3806264484 >> [v2:172.21.15.72:6800/1012617371,v1:172.21.15.72:6801/1012617371] conn(0x7f31a00395b0 0x7f31a003ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:23.233 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:22.160+0000 7f31bce00700 1 --2- 172.21.15.72:0/3806264484 >> [v2:172.21.15.72:6800/1012617371,v1:172.21.15.72:6801/1012617371] conn(0x7f31a00395b0 0x7f31a003ba70 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f31a4006fd0 tx=0x7f31a4006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:23.233 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:22.165+0000 7f31b6000700 1 -- 172.21.15.72:0/3806264484 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f31ac01a4e0 con 0x7f31b810fc60 2024-04-26T21:42:23.234 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:22.481+0000 7f31c0200700 1 -- 172.21.15.72:0/3806264484 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f31b81a6770 con 0x7f31b810fc60 2024-04-26T21:42:23.234 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:22.483+0000 7f31b6000700 1 -- 172.21.15.72:0/3806264484 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f31b81a6770 con 0x7f31b810fc60 2024-04-26T21:42:23.234 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:22.485+0000 7f31c0200700 1 -- 172.21.15.72:0/3806264484 >> [v2:172.21.15.72:6800/1012617371,v1:172.21.15.72:6801/1012617371] conn(0x7f31a00395b0 msgr2=0x7f31a003ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:23.234 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:22.485+0000 7f31c0200700 1 --2- 172.21.15.72:0/3806264484 >> [v2:172.21.15.72:6800/1012617371,v1:172.21.15.72:6801/1012617371] conn(0x7f31a00395b0 0x7f31a003ba70 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f31a4006fd0 tx=0x7f31a4006e40 comp rx=0 tx=0).stop 2024-04-26T21:42:23.234 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:22.485+0000 7f31c0200700 1 -- 172.21.15.72:0/3806264484 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f31b810fc60 msgr2=0x7f31b81a30e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:23.234 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:22.485+0000 7f31c0200700 1 --2- 172.21.15.72:0/3806264484 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f31b810fc60 0x7f31b81a30e0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f31ac0119e0 tx=0x7f31ac0046a0 comp rx=0 tx=0).stop 2024-04-26T21:42:23.234 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:22.486+0000 7f31c0200700 1 -- 172.21.15.72:0/3806264484 shutdown_connections 2024-04-26T21:42:23.234 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:22.486+0000 7f31c0200700 1 --2- 172.21.15.72:0/3806264484 >> [v2:172.21.15.72:6800/1012617371,v1:172.21.15.72:6801/1012617371] conn(0x7f31a00395b0 0x7f31a003ba70 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:23.234 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:22.486+0000 7f31c0200700 1 --2- 172.21.15.72:0/3806264484 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f31b810fc60 0x7f31b81a30e0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:23.234 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:22.486+0000 7f31c0200700 1 -- 172.21.15.72:0/3806264484 >> 172.21.15.72:0/3806264484 conn(0x7f31b807e220 msgr2=0x7f31b807ecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:23.234 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:22.486+0000 7f31c0200700 1 -- 172.21.15.72:0/3806264484 shutdown_connections 2024-04-26T21:42:23.234 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:22.486+0000 7f31c0200700 1 -- 172.21.15.72:0/3806264484 wait complete. 2024-04-26T21:42:23.234 INFO:teuthology.orchestra.run.smithi072.stdout:mgr is available 2024-04-26T21:42:24.476 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:24 smithi072 ceph-mon[28138]: from='client.? 172.21.15.72:0/3154793957' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-26T21:42:24.846 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout 2024-04-26T21:42:24.846 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout [global] 2024-04-26T21:42:24.846 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout fsid = a1780950-0415-11ef-bc93-c7b262605968 2024-04-26T21:42:24.846 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-04-26T21:42:24.846 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-26T21:42:24.846 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-26T21:42:24.846 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-26T21:42:24.846 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-26T21:42:24.846 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout 2024-04-26T21:42:24.846 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-26T21:42:24.846 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-26T21:42:24.847 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout 2024-04-26T21:42:24.847 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout [osd] 2024-04-26T21:42:24.847 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout bdev_async_discard = True 2024-04-26T21:42:24.847 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-26T21:42:24.847 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-26T21:42:24.847 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-26T21:42:24.847 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-26T21:42:24.847 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-26T21:42:24.847 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:23.821+0000 7f9002e00700 1 Processor -- start 2024-04-26T21:42:24.847 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:23.822+0000 7f9002e00700 1 -- start start 2024-04-26T21:42:24.847 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:23.822+0000 7f9002e00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8ffc109700 0x7f8ffc10baf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:24.847 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:23.822+0000 7f9002e00700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f8ffc0646e0 con 0x7f8ffc109700 2024-04-26T21:42:24.847 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:23.822+0000 7f8ffbe00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8ffc109700 0x7f8ffc10baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:24.848 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:23.822+0000 7f8ffbe00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8ffc109700 0x7f8ffc10baf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:34924/0 (socket says 172.21.15.72:34924) 2024-04-26T21:42:24.848 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:23.822+0000 7f8ffbe00700 1 -- 172.21.15.72:0/2232947911 learned_addr learned my addr 172.21.15.72:0/2232947911 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:42:24.848 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:23.823+0000 7f8ffbe00700 1 -- 172.21.15.72:0/2232947911 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ffc10c030 con 0x7f8ffc109700 2024-04-26T21:42:24.848 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:23.823+0000 7f8ffbe00700 1 --2- 172.21.15.72:0/2232947911 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8ffc109700 0x7f8ffc10baf0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f8fec0098a0 tx=0x7f8fec009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38efa94118e3ed7d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:24.848 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:23.824+0000 7f8ffb400700 1 -- 172.21.15.72:0/2232947911 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8fec004160 con 0x7f8ffc109700 2024-04-26T21:42:24.848 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:23.824+0000 7f8ffb400700 1 -- 172.21.15.72:0/2232947911 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f8fec004300 con 0x7f8ffc109700 2024-04-26T21:42:24.848 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:23.824+0000 7f8ffb400700 1 -- 172.21.15.72:0/2232947911 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8fec0119e0 con 0x7f8ffc109700 2024-04-26T21:42:24.848 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:23.824+0000 7f9002e00700 1 -- 172.21.15.72:0/2232947911 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8ffc109700 msgr2=0x7f8ffc10baf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:24.848 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:23.824+0000 7f9002e00700 1 --2- 172.21.15.72:0/2232947911 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8ffc109700 0x7f8ffc10baf0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f8fec0098a0 tx=0x7f8fec009bb0 comp rx=0 tx=0).stop 2024-04-26T21:42:24.848 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:23.824+0000 7f9002e00700 1 -- 172.21.15.72:0/2232947911 shutdown_connections 2024-04-26T21:42:24.848 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:23.824+0000 7f9002e00700 1 --2- 172.21.15.72:0/2232947911 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8ffc109700 0x7f8ffc10baf0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:24.848 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:23.824+0000 7f9002e00700 1 -- 172.21.15.72:0/2232947911 >> 172.21.15.72:0/2232947911 conn(0x7f8ffc1030d0 msgr2=0x7f8ffc105510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:24.848 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:23.825+0000 7f9002e00700 1 -- 172.21.15.72:0/2232947911 shutdown_connections 2024-04-26T21:42:24.848 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:23.825+0000 7f9002e00700 1 -- 172.21.15.72:0/2232947911 wait complete. 2024-04-26T21:42:24.848 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:23.825+0000 7f9002e00700 1 Processor -- start 2024-04-26T21:42:24.849 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:23.826+0000 7f9002e00700 1 -- start start 2024-04-26T21:42:24.849 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:23.826+0000 7f9002e00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8ffc109700 0x7f8ffc19ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:24.849 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:23.826+0000 7f9002e00700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f8ffc10cf40 con 0x7f8ffc109700 2024-04-26T21:42:24.849 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:23.826+0000 7f8ffbe00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8ffc109700 0x7f8ffc19ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:24.849 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:23.826+0000 7f8ffbe00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8ffc109700 0x7f8ffc19ecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:34928/0 (socket says 172.21.15.72:34928) 2024-04-26T21:42:24.849 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:23.826+0000 7f8ffbe00700 1 -- 172.21.15.72:0/3154793957 learned_addr learned my addr 172.21.15.72:0/3154793957 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:42:24.849 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:23.827+0000 7f8ffbe00700 1 -- 172.21.15.72:0/3154793957 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ffc19f1f0 con 0x7f8ffc109700 2024-04-26T21:42:24.849 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:23.827+0000 7f8ffbe00700 1 --2- 172.21.15.72:0/3154793957 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8ffc109700 0x7f8ffc19ecb0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f8fec009130 tx=0x7f8fec011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:24.849 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:23.827+0000 7f8ff9600700 1 -- 172.21.15.72:0/3154793957 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8fec003b20 con 0x7f8ffc109700 2024-04-26T21:42:24.849 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:23.827+0000 7f8ff9600700 1 -- 172.21.15.72:0/3154793957 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f8fec01a8a0 con 0x7f8ffc109700 2024-04-26T21:42:24.849 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:23.827+0000 7f8ff9600700 1 -- 172.21.15.72:0/3154793957 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8fec024810 con 0x7f8ffc109700 2024-04-26T21:42:24.849 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:23.827+0000 7f9002e00700 1 -- 172.21.15.72:0/3154793957 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ffc19f480 con 0x7f8ffc109700 2024-04-26T21:42:24.849 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:23.827+0000 7f9002e00700 1 -- 172.21.15.72:0/3154793957 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ffc1a2170 con 0x7f8ffc109700 2024-04-26T21:42:24.850 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:23.828+0000 7f8ff9600700 1 -- 172.21.15.72:0/3154793957 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7f8fec022020 con 0x7f8ffc109700 2024-04-26T21:42:24.850 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:23.828+0000 7f9002e00700 1 -- 172.21.15.72:0/3154793957 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ffc064a80 con 0x7f8ffc109700 2024-04-26T21:42:24.850 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:23.829+0000 7f8ff9600700 1 --2- 172.21.15.72:0/3154793957 >> [v2:172.21.15.72:6800/1012617371,v1:172.21.15.72:6801/1012617371] conn(0x7f8fe00395b0 0x7f8fe003ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:24.850 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:23.829+0000 7f8ff9600700 1 -- 172.21.15.72:0/3154793957 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f8fec04d720 con 0x7f8ffc109700 2024-04-26T21:42:24.850 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:23.830+0000 7f8ff3e00700 1 --2- 172.21.15.72:0/3154793957 >> [v2:172.21.15.72:6800/1012617371,v1:172.21.15.72:6801/1012617371] conn(0x7f8fe00395b0 0x7f8fe003ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:24.850 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:23.830+0000 7f8ff3e00700 1 --2- 172.21.15.72:0/3154793957 >> [v2:172.21.15.72:6800/1012617371,v1:172.21.15.72:6801/1012617371] conn(0x7f8fe00395b0 0x7f8fe003ba70 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f8fe4006fd0 tx=0x7f8fe4006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:24.850 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:23.835+0000 7f8ff9600700 1 -- 172.21.15.72:0/3154793957 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f8fec01aa50 con 0x7f8ffc109700 2024-04-26T21:42:24.850 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:24.057+0000 7f9002e00700 1 -- 172.21.15.72:0/3154793957 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f8ffc1a2340 con 0x7f8ffc109700 2024-04-26T21:42:24.850 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:24.060+0000 7f8ff9600700 1 -- 172.21.15.72:0/3154793957 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+436 (secure 0 0 0) 0x7f8ffc1a2340 con 0x7f8ffc109700 2024-04-26T21:42:24.850 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:24.063+0000 7f9002e00700 1 -- 172.21.15.72:0/3154793957 >> [v2:172.21.15.72:6800/1012617371,v1:172.21.15.72:6801/1012617371] conn(0x7f8fe00395b0 msgr2=0x7f8fe003ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:24.850 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:24.063+0000 7f9002e00700 1 --2- 172.21.15.72:0/3154793957 >> [v2:172.21.15.72:6800/1012617371,v1:172.21.15.72:6801/1012617371] conn(0x7f8fe00395b0 0x7f8fe003ba70 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f8fe4006fd0 tx=0x7f8fe4006e40 comp rx=0 tx=0).stop 2024-04-26T21:42:24.850 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:24.063+0000 7f9002e00700 1 -- 172.21.15.72:0/3154793957 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8ffc109700 msgr2=0x7f8ffc19ecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:24.850 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:24.063+0000 7f9002e00700 1 --2- 172.21.15.72:0/3154793957 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8ffc109700 0x7f8ffc19ecb0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f8fec009130 tx=0x7f8fec011a60 comp rx=0 tx=0).stop 2024-04-26T21:42:24.851 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:24.063+0000 7f9002e00700 1 -- 172.21.15.72:0/3154793957 shutdown_connections 2024-04-26T21:42:24.851 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:24.063+0000 7f9002e00700 1 --2- 172.21.15.72:0/3154793957 >> [v2:172.21.15.72:6800/1012617371,v1:172.21.15.72:6801/1012617371] conn(0x7f8fe00395b0 0x7f8fe003ba70 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:24.851 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:24.063+0000 7f9002e00700 1 --2- 172.21.15.72:0/3154793957 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8ffc109700 0x7f8ffc19ecb0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:24.851 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:24.063+0000 7f9002e00700 1 -- 172.21.15.72:0/3154793957 >> 172.21.15.72:0/3154793957 conn(0x7f8ffc1030d0 msgr2=0x7f8ffc103b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:24.851 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:24.063+0000 7f9002e00700 1 -- 172.21.15.72:0/3154793957 shutdown_connections 2024-04-26T21:42:24.851 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:24.063+0000 7f9002e00700 1 -- 172.21.15.72:0/3154793957 wait complete. 2024-04-26T21:42:24.851 INFO:teuthology.orchestra.run.smithi072.stdout:Enabling cephadm module... 2024-04-26T21:42:26.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:25 smithi072 ceph-mon[28138]: from='client.? 172.21.15.72:0/1685638640' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-04-26T21:42:26.072 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 26 21:42:25 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-mgr-x[28363]: ignoring --setuser ceph since I am not root 2024-04-26T21:42:26.072 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 26 21:42:25 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-mgr-x[28363]: ignoring --setgroup ceph since I am not root 2024-04-26T21:42:26.430 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 26 21:42:26 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-mgr-x[28363]: 2024-04-26T21:42:26.080+0000 7fe51c9b3200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-26T21:42:26.430 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 26 21:42:26 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-mgr-x[28363]: 2024-04-26T21:42:26.318+0000 7fe51c9b3200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-26T21:42:26.782 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 26 21:42:26 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-mgr-x[28363]: 2024-04-26T21:42:26.454+0000 7fe51c9b3200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-26T21:42:26.794 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:25.446+0000 7ff351600700 1 Processor -- start 2024-04-26T21:42:26.794 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:25.447+0000 7ff351600700 1 -- start start 2024-04-26T21:42:26.794 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:25.447+0000 7ff351600700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff34c107b30 0x7ff34c107f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:26.794 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:25.447+0000 7ff351600700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7ff34c108520 con 0x7ff34c107b30 2024-04-26T21:42:26.794 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:25.447+0000 7ff34aa00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff34c107b30 0x7ff34c107f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:26.794 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:25.448+0000 7ff34aa00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff34c107b30 0x7ff34c107f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:34940/0 (socket says 172.21.15.72:34940) 2024-04-26T21:42:26.794 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:25.448+0000 7ff34aa00700 1 -- 172.21.15.72:0/3594458917 learned_addr learned my addr 172.21.15.72:0/3594458917 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:42:26.795 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:25.448+0000 7ff34aa00700 1 -- 172.21.15.72:0/3594458917 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff34c108d30 con 0x7ff34c107b30 2024-04-26T21:42:26.795 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:25.448+0000 7ff34aa00700 1 --2- 172.21.15.72:0/3594458917 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff34c107b30 0x7ff34c107f50 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7ff33c0098a0 tx=0x7ff33c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f22c452f2836d721 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:26.795 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:25.449+0000 7ff349600700 1 -- 172.21.15.72:0/3594458917 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff33c004160 con 0x7ff34c107b30 2024-04-26T21:42:26.795 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:25.449+0000 7ff349600700 1 -- 172.21.15.72:0/3594458917 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7ff33c004300 con 0x7ff34c107b30 2024-04-26T21:42:26.795 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:25.449+0000 7ff349600700 1 -- 172.21.15.72:0/3594458917 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff33c0119e0 con 0x7ff34c107b30 2024-04-26T21:42:26.795 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:25.449+0000 7ff351600700 1 -- 172.21.15.72:0/3594458917 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff34c107b30 msgr2=0x7ff34c107f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:26.795 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:25.449+0000 7ff351600700 1 --2- 172.21.15.72:0/3594458917 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff34c107b30 0x7ff34c107f50 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7ff33c0098a0 tx=0x7ff33c009bb0 comp rx=0 tx=0).stop 2024-04-26T21:42:26.795 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:25.450+0000 7ff351600700 1 -- 172.21.15.72:0/3594458917 shutdown_connections 2024-04-26T21:42:26.795 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:25.450+0000 7ff351600700 1 --2- 172.21.15.72:0/3594458917 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff34c107b30 0x7ff34c107f50 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:26.795 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:25.450+0000 7ff351600700 1 -- 172.21.15.72:0/3594458917 >> 172.21.15.72:0/3594458917 conn(0x7ff34c1030d0 msgr2=0x7ff34c105510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:26.795 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:25.450+0000 7ff351600700 1 -- 172.21.15.72:0/3594458917 shutdown_connections 2024-04-26T21:42:26.795 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:25.450+0000 7ff351600700 1 -- 172.21.15.72:0/3594458917 wait complete. 2024-04-26T21:42:26.795 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:25.451+0000 7ff351600700 1 Processor -- start 2024-04-26T21:42:26.795 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:25.451+0000 7ff351600700 1 -- start start 2024-04-26T21:42:26.795 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:25.451+0000 7ff351600700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff34c107b30 0x7ff34c19ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:26.796 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:25.451+0000 7ff351600700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7ff34c10bf60 con 0x7ff34c107b30 2024-04-26T21:42:26.796 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:25.451+0000 7ff34aa00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff34c107b30 0x7ff34c19ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:26.796 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:25.451+0000 7ff34aa00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff34c107b30 0x7ff34c19ecf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:34956/0 (socket says 172.21.15.72:34956) 2024-04-26T21:42:26.796 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:25.451+0000 7ff34aa00700 1 -- 172.21.15.72:0/1685638640 learned_addr learned my addr 172.21.15.72:0/1685638640 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:42:26.796 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:25.452+0000 7ff34aa00700 1 -- 172.21.15.72:0/1685638640 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff34c19f230 con 0x7ff34c107b30 2024-04-26T21:42:26.796 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:25.452+0000 7ff34aa00700 1 --2- 172.21.15.72:0/1685638640 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff34c107b30 0x7ff34c19ecf0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7ff33c0119b0 tx=0x7ff33c011a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:26.796 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:25.452+0000 7ff343400700 1 -- 172.21.15.72:0/1685638640 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff33c003af0 con 0x7ff34c107b30 2024-04-26T21:42:26.796 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:25.452+0000 7ff343400700 1 -- 172.21.15.72:0/1685638640 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7ff33c01a8a0 con 0x7ff34c107b30 2024-04-26T21:42:26.796 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:25.452+0000 7ff351600700 1 -- 172.21.15.72:0/1685638640 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff34c19f4c0 con 0x7ff34c107b30 2024-04-26T21:42:26.796 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:25.452+0000 7ff343400700 1 -- 172.21.15.72:0/1685638640 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff33c024830 con 0x7ff34c107b30 2024-04-26T21:42:26.796 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:25.453+0000 7ff351600700 1 -- 172.21.15.72:0/1685638640 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff34c1a21a0 con 0x7ff34c107b30 2024-04-26T21:42:26.796 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:25.454+0000 7ff343400700 1 -- 172.21.15.72:0/1685638640 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7ff33c022020 con 0x7ff34c107b30 2024-04-26T21:42:26.796 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:25.454+0000 7ff351600700 1 -- 172.21.15.72:0/1685638640 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff34c064a80 con 0x7ff34c107b30 2024-04-26T21:42:26.796 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:25.454+0000 7ff343400700 1 --2- 172.21.15.72:0/1685638640 >> [v2:172.21.15.72:6800/1012617371,v1:172.21.15.72:6801/1012617371] conn(0x7ff3300395b0 0x7ff33003ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:26.796 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:25.455+0000 7ff343400700 1 -- 172.21.15.72:0/1685638640 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7ff33c04d610 con 0x7ff34c107b30 2024-04-26T21:42:26.797 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:25.455+0000 7ff34a000700 1 --2- 172.21.15.72:0/1685638640 >> [v2:172.21.15.72:6800/1012617371,v1:172.21.15.72:6801/1012617371] conn(0x7ff3300395b0 0x7ff33003ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:26.797 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:25.455+0000 7ff34a000700 1 --2- 172.21.15.72:0/1685638640 >> [v2:172.21.15.72:6800/1012617371,v1:172.21.15.72:6801/1012617371] conn(0x7ff3300395b0 0x7ff33003ba70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff338006fd0 tx=0x7ff338006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:26.797 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:25.460+0000 7ff343400700 1 -- 172.21.15.72:0/1685638640 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7ff33c00be50 con 0x7ff34c107b30 2024-04-26T21:42:26.797 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:25.747+0000 7ff351600700 1 -- 172.21.15.72:0/1685638640 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7ff34c0008d0 con 0x7ff34c107b30 2024-04-26T21:42:26.797 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:25.801+0000 7ff343400700 1 -- 172.21.15.72:0/1685638640 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7ff33c02ad60 con 0x7ff34c107b30 2024-04-26T21:42:26.797 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:25.805+0000 7ff343400700 1 -- 172.21.15.72:0/1685638640 <== mon.0 v2:172.21.15.72:3300/0 8 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7ff33c00b3e0 con 0x7ff34c107b30 2024-04-26T21:42:26.797 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:25.807+0000 7ff351600700 1 -- 172.21.15.72:0/1685638640 >> [v2:172.21.15.72:6800/1012617371,v1:172.21.15.72:6801/1012617371] conn(0x7ff3300395b0 msgr2=0x7ff33003ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:26.797 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:25.807+0000 7ff351600700 1 --2- 172.21.15.72:0/1685638640 >> [v2:172.21.15.72:6800/1012617371,v1:172.21.15.72:6801/1012617371] conn(0x7ff3300395b0 0x7ff33003ba70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff338006fd0 tx=0x7ff338006e40 comp rx=0 tx=0).stop 2024-04-26T21:42:26.797 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:25.807+0000 7ff351600700 1 -- 172.21.15.72:0/1685638640 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff34c107b30 msgr2=0x7ff34c19ecf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:26.797 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:25.807+0000 7ff351600700 1 --2- 172.21.15.72:0/1685638640 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff34c107b30 0x7ff34c19ecf0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7ff33c0119b0 tx=0x7ff33c011a90 comp rx=0 tx=0).stop 2024-04-26T21:42:26.797 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:25.807+0000 7ff351600700 1 -- 172.21.15.72:0/1685638640 shutdown_connections 2024-04-26T21:42:26.797 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:25.807+0000 7ff351600700 1 --2- 172.21.15.72:0/1685638640 >> [v2:172.21.15.72:6800/1012617371,v1:172.21.15.72:6801/1012617371] conn(0x7ff3300395b0 0x7ff33003ba70 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:26.797 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:25.807+0000 7ff351600700 1 --2- 172.21.15.72:0/1685638640 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff34c107b30 0x7ff34c19ecf0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:26.797 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:25.807+0000 7ff351600700 1 -- 172.21.15.72:0/1685638640 >> 172.21.15.72:0/1685638640 conn(0x7ff34c1030d0 msgr2=0x7ff34c103f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:26.797 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:25.807+0000 7ff351600700 1 -- 172.21.15.72:0/1685638640 shutdown_connections 2024-04-26T21:42:26.797 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:25.807+0000 7ff351600700 1 -- 172.21.15.72:0/1685638640 wait complete. 2024-04-26T21:42:27.072 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 26 21:42:26 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-mgr-x[28363]: 2024-04-26T21:42:26.848+0000 7fe51c9b3200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-26T21:42:27.072 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 26 21:42:26 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-mgr-x[28363]: 2024-04-26T21:42:26.960+0000 7fe51c9b3200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-26T21:42:27.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:26 smithi072 ceph-mon[28138]: from='client.? 172.21.15.72:0/1685638640' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-04-26T21:42:27.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:26 smithi072 ceph-mon[28138]: mgrmap e5: x(active, since 5s) 2024-04-26T21:42:28.069 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:27 smithi072 ceph-mon[28138]: from='client.? 172.21.15.72:0/1647619306' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-26T21:42:28.375 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout { 2024-04-26T21:42:28.375 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-04-26T21:42:28.375 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-26T21:42:28.375 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-04-26T21:42:28.375 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-26T21:42:28.376 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout } 2024-04-26T21:42:28.376 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:27.373+0000 7f9471e00700 1 Processor -- start 2024-04-26T21:42:28.376 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:27.374+0000 7f9471e00700 1 -- start start 2024-04-26T21:42:28.376 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:27.374+0000 7f9471e00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f946c07dd40 0x7f946c07c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:28.376 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:27.374+0000 7f9471e00700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f946c07c770 con 0x7f946c07dd40 2024-04-26T21:42:28.376 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:27.375+0000 7f946aa00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f946c07dd40 0x7f946c07c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:28.376 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:27.375+0000 7f946aa00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f946c07dd40 0x7f946c07c1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:34972/0 (socket says 172.21.15.72:34972) 2024-04-26T21:42:28.376 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:27.375+0000 7f946aa00700 1 -- 172.21.15.72:0/4041243802 learned_addr learned my addr 172.21.15.72:0/4041243802 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:42:28.376 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:27.375+0000 7f946aa00700 1 -- 172.21.15.72:0/4041243802 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f946c07c8f0 con 0x7f946c07dd40 2024-04-26T21:42:28.376 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:27.376+0000 7f946aa00700 1 --2- 172.21.15.72:0/4041243802 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f946c07dd40 0x7f946c07c1a0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f945c0098a0 tx=0x7f945c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f6bcae67dfe6513 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:28.376 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:27.376+0000 7f9469600700 1 -- 172.21.15.72:0/4041243802 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f945c004160 con 0x7f946c07dd40 2024-04-26T21:42:28.376 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:27.376+0000 7f9469600700 1 -- 172.21.15.72:0/4041243802 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f945c004300 con 0x7f946c07dd40 2024-04-26T21:42:28.376 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:27.376+0000 7f9469600700 1 -- 172.21.15.72:0/4041243802 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f945c0119e0 con 0x7f946c07dd40 2024-04-26T21:42:28.377 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:27.377+0000 7f9471e00700 1 -- 172.21.15.72:0/4041243802 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f946c07dd40 msgr2=0x7f946c07c1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:28.377 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:27.377+0000 7f9471e00700 1 --2- 172.21.15.72:0/4041243802 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f946c07dd40 0x7f946c07c1a0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f945c0098a0 tx=0x7f945c009bb0 comp rx=0 tx=0).stop 2024-04-26T21:42:28.377 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:27.377+0000 7f9471e00700 1 -- 172.21.15.72:0/4041243802 shutdown_connections 2024-04-26T21:42:28.377 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:27.377+0000 7f9471e00700 1 --2- 172.21.15.72:0/4041243802 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f946c07dd40 0x7f946c07c1a0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:28.377 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:27.377+0000 7f9471e00700 1 -- 172.21.15.72:0/4041243802 >> 172.21.15.72:0/4041243802 conn(0x7f946c1041c0 msgr2=0x7f946c106640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:28.377 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:27.377+0000 7f9471e00700 1 -- 172.21.15.72:0/4041243802 shutdown_connections 2024-04-26T21:42:28.377 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:27.377+0000 7f9471e00700 1 -- 172.21.15.72:0/4041243802 wait complete. 2024-04-26T21:42:28.377 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:27.378+0000 7f9471e00700 1 Processor -- start 2024-04-26T21:42:28.377 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:27.379+0000 7f9471e00700 1 -- start start 2024-04-26T21:42:28.377 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:27.379+0000 7f9471e00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f946c07dd40 0x7f946c19ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:28.377 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:27.379+0000 7f9471e00700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f946c10f190 con 0x7f946c07dd40 2024-04-26T21:42:28.377 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:27.379+0000 7f946aa00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f946c07dd40 0x7f946c19ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:28.378 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:27.379+0000 7f946aa00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f946c07dd40 0x7f946c19ed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:34986/0 (socket says 172.21.15.72:34986) 2024-04-26T21:42:28.378 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:27.379+0000 7f946aa00700 1 -- 172.21.15.72:0/1647619306 learned_addr learned my addr 172.21.15.72:0/1647619306 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:42:28.378 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:27.380+0000 7f946aa00700 1 -- 172.21.15.72:0/1647619306 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f946c19f270 con 0x7f946c07dd40 2024-04-26T21:42:28.378 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:27.380+0000 7f946aa00700 1 --2- 172.21.15.72:0/1647619306 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f946c07dd40 0x7f946c19ed30 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f945c0119b0 tx=0x7f945c011a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:28.378 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:27.380+0000 7f9470a00700 1 -- 172.21.15.72:0/1647619306 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f945c003af0 con 0x7f946c07dd40 2024-04-26T21:42:28.378 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:27.380+0000 7f9470a00700 1 -- 172.21.15.72:0/1647619306 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f945c01a8a0 con 0x7f946c07dd40 2024-04-26T21:42:28.378 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:27.380+0000 7f9470a00700 1 -- 172.21.15.72:0/1647619306 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f945c024830 con 0x7f946c07dd40 2024-04-26T21:42:28.378 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:27.380+0000 7f9471e00700 1 -- 172.21.15.72:0/1647619306 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f946c19f500 con 0x7f946c07dd40 2024-04-26T21:42:28.378 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:27.381+0000 7f9471e00700 1 -- 172.21.15.72:0/1647619306 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f946c1a21f0 con 0x7f946c07dd40 2024-04-26T21:42:28.378 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:27.382+0000 7f9470a00700 1 -- 172.21.15.72:0/1647619306 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7f945c022020 con 0x7f946c07dd40 2024-04-26T21:42:28.378 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:27.382+0000 7f9471e00700 1 -- 172.21.15.72:0/1647619306 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f946c064a80 con 0x7f946c07dd40 2024-04-26T21:42:28.378 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:27.382+0000 7f9470a00700 1 --2- 172.21.15.72:0/1647619306 >> [v2:172.21.15.72:6800/1012617371,v1:172.21.15.72:6801/1012617371] conn(0x7f9450039600 0x7f945003bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:28.378 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:27.382+0000 7f9470a00700 1 -- 172.21.15.72:0/1647619306 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f945c04d5b0 con 0x7f946c07dd40 2024-04-26T21:42:28.379 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:27.382+0000 7f946a000700 1 -- 172.21.15.72:0/1647619306 >> [v2:172.21.15.72:6800/1012617371,v1:172.21.15.72:6801/1012617371] conn(0x7f9450039600 msgr2=0x7f945003bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.72:6800/1012617371 2024-04-26T21:42:28.379 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:27.383+0000 7f946a000700 1 --2- 172.21.15.72:0/1647619306 >> [v2:172.21.15.72:6800/1012617371,v1:172.21.15.72:6801/1012617371] conn(0x7f9450039600 0x7f945003bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T21:42:28.379 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:27.388+0000 7f9470a00700 1 -- 172.21.15.72:0/1647619306 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f945c00f2e0 con 0x7f946c07dd40 2024-04-26T21:42:28.379 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:27.584+0000 7f946a000700 1 -- 172.21.15.72:0/1647619306 >> [v2:172.21.15.72:6800/1012617371,v1:172.21.15.72:6801/1012617371] conn(0x7f9450039600 msgr2=0x7f945003bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.72:6800/1012617371 2024-04-26T21:42:28.379 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:27.584+0000 7f946a000700 1 --2- 172.21.15.72:0/1647619306 >> [v2:172.21.15.72:6800/1012617371,v1:172.21.15.72:6801/1012617371] conn(0x7f9450039600 0x7f945003bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-26T21:42:28.379 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:27.729+0000 7f9471e00700 1 -- 172.21.15.72:0/1647619306 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f946c1a2460 con 0x7f946c07dd40 2024-04-26T21:42:28.379 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:27.730+0000 7f9470a00700 1 -- 172.21.15.72:0/1647619306 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7f945c04d310 con 0x7f946c07dd40 2024-04-26T21:42:28.379 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:27.733+0000 7f9471e00700 1 -- 172.21.15.72:0/1647619306 >> [v2:172.21.15.72:6800/1012617371,v1:172.21.15.72:6801/1012617371] conn(0x7f9450039600 msgr2=0x7f945003bac0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T21:42:28.379 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:27.733+0000 7f9471e00700 1 --2- 172.21.15.72:0/1647619306 >> [v2:172.21.15.72:6800/1012617371,v1:172.21.15.72:6801/1012617371] conn(0x7f9450039600 0x7f945003bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:28.379 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:27.733+0000 7f9471e00700 1 -- 172.21.15.72:0/1647619306 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f946c07dd40 msgr2=0x7f946c19ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:28.379 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:27.733+0000 7f9471e00700 1 --2- 172.21.15.72:0/1647619306 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f946c07dd40 0x7f946c19ed30 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f945c0119b0 tx=0x7f945c011a90 comp rx=0 tx=0).stop 2024-04-26T21:42:28.379 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:27.733+0000 7f9471e00700 1 -- 172.21.15.72:0/1647619306 shutdown_connections 2024-04-26T21:42:28.379 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:27.733+0000 7f9471e00700 1 --2- 172.21.15.72:0/1647619306 >> [v2:172.21.15.72:6800/1012617371,v1:172.21.15.72:6801/1012617371] conn(0x7f9450039600 0x7f945003bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:28.379 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:27.733+0000 7f9471e00700 1 --2- 172.21.15.72:0/1647619306 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f946c07dd40 0x7f946c19ed30 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:28.380 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:27.733+0000 7f9471e00700 1 -- 172.21.15.72:0/1647619306 >> 172.21.15.72:0/1647619306 conn(0x7f946c1041c0 msgr2=0x7f946c104cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:28.380 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:27.733+0000 7f9471e00700 1 -- 172.21.15.72:0/1647619306 shutdown_connections 2024-04-26T21:42:28.380 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:27.734+0000 7f9471e00700 1 -- 172.21.15.72:0/1647619306 wait complete. 2024-04-26T21:42:28.380 INFO:teuthology.orchestra.run.smithi072.stdout:Waiting for the mgr to restart... 2024-04-26T21:42:28.380 INFO:teuthology.orchestra.run.smithi072.stdout:Waiting for mgr epoch 5... 2024-04-26T21:42:28.680 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 26 21:42:28 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-mgr-x[28363]: 2024-04-26T21:42:28.598+0000 7fe51c9b3200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-26T21:42:28.961 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 26 21:42:28 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-mgr-x[28363]: 2024-04-26T21:42:28.842+0000 7fe51c9b3200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-26T21:42:29.218 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 26 21:42:29 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-mgr-x[28363]: 2024-04-26T21:42:29.063+0000 7fe51c9b3200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-26T21:42:29.555 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 26 21:42:29 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-mgr-x[28363]: 2024-04-26T21:42:29.433+0000 7fe51c9b3200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-26T21:42:29.805 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 26 21:42:29 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-mgr-x[28363]: 2024-04-26T21:42:29.553+0000 7fe51c9b3200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-26T21:42:30.072 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 26 21:42:29 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-mgr-x[28363]: 2024-04-26T21:42:29.804+0000 7fe51c9b3200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-26T21:42:30.072 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 26 21:42:30 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-mgr-x[28363]: 2024-04-26T21:42:30.053+0000 7fe51c9b3200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-26T21:42:31.322 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 26 21:42:30 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-mgr-x[28363]: 2024-04-26T21:42:30.833+0000 7fe51c9b3200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-26T21:42:31.764 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 26 21:42:31 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-mgr-x[28363]: 2024-04-26T21:42:31.375+0000 7fe51c9b3200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-26T21:42:31.764 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 26 21:42:31 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-mgr-x[28363]: 2024-04-26T21:42:31.501+0000 7fe51c9b3200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-26T21:42:31.764 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 26 21:42:31 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-mgr-x[28363]: 2024-04-26T21:42:31.762+0000 7fe51c9b3200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-26T21:42:32.822 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 26 21:42:32 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-mgr-x[28363]: 2024-04-26T21:42:32.399+0000 7fe51c9b3200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-26T21:42:32.822 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 26 21:42:32 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-mgr-x[28363]: 2024-04-26T21:42:32.520+0000 7fe51c9b3200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-26T21:42:33.135 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 26 21:42:32 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-mgr-x[28363]: 2024-04-26T21:42:32.835+0000 7fe51c9b3200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-26T21:42:33.135 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 26 21:42:33 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-mgr-x[28363]: 2024-04-26T21:42:33.010+0000 7fe51c9b3200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-26T21:42:33.135 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 26 21:42:33 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-mgr-x[28363]: 2024-04-26T21:42:33.133+0000 7fe51c9b3200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-26T21:42:33.572 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 26 21:42:33 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-mgr-x[28363]: 2024-04-26T21:42:33.281+0000 7fe51c9b3200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-26T21:42:34.072 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 26 21:42:33 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-mgr-x[28363]: 2024-04-26T21:42:33.662+0000 7fe51c9b3200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-26T21:42:34.681 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 26 21:42:34 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-mgr-x[28363]: 2024-04-26T21:42:34.402+0000 7fe51c9b3200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-26T21:42:35.072 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 26 21:42:34 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-mgr-x[28363]: 2024-04-26T21:42:34.680+0000 7fe51c9b3200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-26T21:42:35.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:34 smithi072 ceph-mon[28138]: Active manager daemon x restarted 2024-04-26T21:42:35.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:34 smithi072 ceph-mon[28138]: Activating manager daemon x 2024-04-26T21:42:35.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:34 smithi072 ceph-mon[28138]: osdmap e2: 0 total, 0 up, 0 in 2024-04-26T21:42:35.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:34 smithi072 ceph-mon[28138]: mgrmap e6: x(active, starting, since 0.00979534s) 2024-04-26T21:42:35.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:34 smithi072 ceph-mon[28138]: from='mgr.14120 172.21.15.72:0/474756945' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T21:42:35.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:34 smithi072 ceph-mon[28138]: from='mgr.14120 172.21.15.72:0/474756945' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-26T21:42:35.073 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:34 smithi072 ceph-mon[28138]: from='mgr.14120 172.21.15.72:0/474756945' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-26T21:42:35.073 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:34 smithi072 ceph-mon[28138]: from='mgr.14120 172.21.15.72:0/474756945' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-26T21:42:35.073 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:34 smithi072 ceph-mon[28138]: from='mgr.14120 172.21.15.72:0/474756945' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-26T21:42:35.073 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:34 smithi072 ceph-mon[28138]: Manager daemon x is now available 2024-04-26T21:42:35.073 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:34 smithi072 ceph-mon[28138]: from='mgr.14120 172.21.15.72:0/474756945' entity='mgr.x' 2024-04-26T21:42:35.073 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:34 smithi072 ceph-mon[28138]: from='mgr.14120 172.21.15.72:0/474756945' entity='mgr.x' 2024-04-26T21:42:36.039 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:35 smithi072 ceph-mon[28138]: Found migration_current of "None". Setting to last migration. 2024-04-26T21:42:36.039 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:35 smithi072 ceph-mon[28138]: from='mgr.14120 172.21.15.72:0/474756945' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:42:36.039 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:35 smithi072 ceph-mon[28138]: from='mgr.14120 172.21.15.72:0/474756945' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:42:36.039 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:35 smithi072 ceph-mon[28138]: from='mgr.14120 172.21.15.72:0/474756945' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-26T21:42:36.039 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:35 smithi072 ceph-mon[28138]: from='mgr.14120 172.21.15.72:0/474756945' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-26T21:42:36.039 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:35 smithi072 ceph-mon[28138]: from='mgr.14120 172.21.15.72:0/474756945' entity='mgr.x' 2024-04-26T21:42:36.039 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:35 smithi072 ceph-mon[28138]: from='mgr.14120 172.21.15.72:0/474756945' entity='mgr.x' 2024-04-26T21:42:36.039 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:35 smithi072 ceph-mon[28138]: from='mgr.14120 172.21.15.72:0/474756945' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:42:36.040 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:35 smithi072 ceph-mon[28138]: mgrmap e7: x(active, since 1.01697s) 2024-04-26T21:42:36.421 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout { 2024-04-26T21:42:36.421 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-04-26T21:42:36.421 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-26T21:42:36.421 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout } 2024-04-26T21:42:36.421 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:29.008+0000 7f5fbc400700 1 Processor -- start 2024-04-26T21:42:36.421 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:29.008+0000 7f5fbc400700 1 -- start start 2024-04-26T21:42:36.421 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:29.009+0000 7f5fbc400700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5fb407dd40 0x7f5fb407c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:36.421 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:29.009+0000 7f5fbc400700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f5fb407c770 con 0x7f5fb407dd40 2024-04-26T21:42:36.421 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:29.009+0000 7f5fb9a00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5fb407dd40 0x7f5fb407c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:36.422 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:29.009+0000 7f5fb9a00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5fb407dd40 0x7f5fb407c1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:35002/0 (socket says 172.21.15.72:35002) 2024-04-26T21:42:36.422 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:29.009+0000 7f5fb9a00700 1 -- 172.21.15.72:0/3539288485 learned_addr learned my addr 172.21.15.72:0/3539288485 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:42:36.422 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:29.010+0000 7f5fb9a00700 1 -- 172.21.15.72:0/3539288485 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fb407c8f0 con 0x7f5fb407dd40 2024-04-26T21:42:36.422 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:29.010+0000 7f5fb9a00700 1 --2- 172.21.15.72:0/3539288485 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5fb407dd40 0x7f5fb407c1a0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f5fa80098a0 tx=0x7f5fa8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c850946b23516f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:36.422 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:29.010+0000 7f5fb3e00700 1 -- 172.21.15.72:0/3539288485 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5fa8004160 con 0x7f5fb407dd40 2024-04-26T21:42:36.422 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:29.010+0000 7f5fb3e00700 1 -- 172.21.15.72:0/3539288485 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f5fa8004300 con 0x7f5fb407dd40 2024-04-26T21:42:36.422 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:29.010+0000 7f5fb3e00700 1 -- 172.21.15.72:0/3539288485 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5fa80119e0 con 0x7f5fb407dd40 2024-04-26T21:42:36.422 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:29.011+0000 7f5fbc400700 1 -- 172.21.15.72:0/3539288485 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5fb407dd40 msgr2=0x7f5fb407c1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:36.422 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:29.011+0000 7f5fbc400700 1 --2- 172.21.15.72:0/3539288485 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5fb407dd40 0x7f5fb407c1a0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f5fa80098a0 tx=0x7f5fa8009bb0 comp rx=0 tx=0).stop 2024-04-26T21:42:36.422 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:29.011+0000 7f5fbc400700 1 -- 172.21.15.72:0/3539288485 shutdown_connections 2024-04-26T21:42:36.422 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:29.011+0000 7f5fbc400700 1 --2- 172.21.15.72:0/3539288485 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5fb407dd40 0x7f5fb407c1a0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:36.422 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:29.011+0000 7f5fbc400700 1 -- 172.21.15.72:0/3539288485 >> 172.21.15.72:0/3539288485 conn(0x7f5fb4104260 msgr2=0x7f5fb4106680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:36.423 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:29.011+0000 7f5fbc400700 1 -- 172.21.15.72:0/3539288485 shutdown_connections 2024-04-26T21:42:36.423 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:29.011+0000 7f5fbc400700 1 -- 172.21.15.72:0/3539288485 wait complete. 2024-04-26T21:42:36.423 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:29.012+0000 7f5fbc400700 1 Processor -- start 2024-04-26T21:42:36.423 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:29.012+0000 7f5fbc400700 1 -- start start 2024-04-26T21:42:36.423 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:29.013+0000 7f5fbc400700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5fb407dd40 0x7f5fb419ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:36.423 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:29.013+0000 7f5fbc400700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f5fb410f1b0 con 0x7f5fb407dd40 2024-04-26T21:42:36.423 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:29.013+0000 7f5fb9a00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5fb407dd40 0x7f5fb419ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:36.423 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:29.013+0000 7f5fb9a00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5fb407dd40 0x7f5fb419ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:35012/0 (socket says 172.21.15.72:35012) 2024-04-26T21:42:36.423 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:29.013+0000 7f5fb9a00700 1 -- 172.21.15.72:0/1108024460 learned_addr learned my addr 172.21.15.72:0/1108024460 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:42:36.423 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:29.013+0000 7f5fb9a00700 1 -- 172.21.15.72:0/1108024460 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fb419f2a0 con 0x7f5fb407dd40 2024-04-26T21:42:36.423 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:29.014+0000 7f5fb9a00700 1 --2- 172.21.15.72:0/1108024460 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5fb407dd40 0x7f5fb419ed60 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f5fa8006b20 tx=0x7f5fa8011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:36.423 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:29.014+0000 7f5fb2000700 1 -- 172.21.15.72:0/1108024460 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5fa8003a60 con 0x7f5fb407dd40 2024-04-26T21:42:36.423 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:29.014+0000 7f5fb2000700 1 -- 172.21.15.72:0/1108024460 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f5fa8003c00 con 0x7f5fb407dd40 2024-04-26T21:42:36.423 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:29.014+0000 7f5fb2000700 1 -- 172.21.15.72:0/1108024460 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5fa8024780 con 0x7f5fb407dd40 2024-04-26T21:42:36.424 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:29.014+0000 7f5fbc400700 1 -- 172.21.15.72:0/1108024460 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fb419f530 con 0x7f5fb407dd40 2024-04-26T21:42:36.424 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:29.014+0000 7f5fbc400700 1 -- 172.21.15.72:0/1108024460 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fb41a2220 con 0x7f5fb407dd40 2024-04-26T21:42:36.424 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:29.015+0000 7f5fb2000700 1 -- 172.21.15.72:0/1108024460 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7f5fa8022020 con 0x7f5fb407dd40 2024-04-26T21:42:36.424 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:29.016+0000 7f5fb2000700 1 --2- 172.21.15.72:0/1108024460 >> [v2:172.21.15.72:6800/1012617371,v1:172.21.15.72:6801/1012617371] conn(0x7f5f9c039600 0x7f5f9c03bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:36.424 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:29.016+0000 7f5fb2000700 1 -- 172.21.15.72:0/1108024460 --> [v2:172.21.15.72:6800/1012617371,v1:172.21.15.72:6801/1012617371] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f5f9c03c1d0 con 0x7f5f9c039600 2024-04-26T21:42:36.424 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:29.016+0000 7f5fb2000700 1 -- 172.21.15.72:0/1108024460 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f5fa804d760 con 0x7f5fb407dd40 2024-04-26T21:42:36.424 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:29.016+0000 7f5fb9000700 1 -- 172.21.15.72:0/1108024460 >> [v2:172.21.15.72:6800/1012617371,v1:172.21.15.72:6801/1012617371] conn(0x7f5f9c039600 msgr2=0x7f5f9c03bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.72:6800/1012617371 2024-04-26T21:42:36.424 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:29.016+0000 7f5fb9000700 1 --2- 172.21.15.72:0/1108024460 >> [v2:172.21.15.72:6800/1012617371,v1:172.21.15.72:6801/1012617371] conn(0x7f5f9c039600 0x7f5f9c03bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T21:42:36.424 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:29.217+0000 7f5fb9000700 1 -- 172.21.15.72:0/1108024460 >> [v2:172.21.15.72:6800/1012617371,v1:172.21.15.72:6801/1012617371] conn(0x7f5f9c039600 msgr2=0x7f5f9c03bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.72:6800/1012617371 2024-04-26T21:42:36.424 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:29.217+0000 7f5fb9000700 1 --2- 172.21.15.72:0/1108024460 >> [v2:172.21.15.72:6800/1012617371,v1:172.21.15.72:6801/1012617371] conn(0x7f5f9c039600 0x7f5f9c03bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-26T21:42:36.424 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:29.618+0000 7f5fb9000700 1 -- 172.21.15.72:0/1108024460 >> [v2:172.21.15.72:6800/1012617371,v1:172.21.15.72:6801/1012617371] conn(0x7f5f9c039600 msgr2=0x7f5f9c03bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.72:6800/1012617371 2024-04-26T21:42:36.424 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:29.618+0000 7f5fb9000700 1 --2- 172.21.15.72:0/1108024460 >> [v2:172.21.15.72:6800/1012617371,v1:172.21.15.72:6801/1012617371] conn(0x7f5f9c039600 0x7f5f9c03bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-26T21:42:36.425 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:30.419+0000 7f5fb9000700 1 -- 172.21.15.72:0/1108024460 >> [v2:172.21.15.72:6800/1012617371,v1:172.21.15.72:6801/1012617371] conn(0x7f5f9c039600 msgr2=0x7f5f9c03bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.72:6800/1012617371 2024-04-26T21:42:36.425 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:30.419+0000 7f5fb9000700 1 --2- 172.21.15.72:0/1108024460 >> [v2:172.21.15.72:6800/1012617371,v1:172.21.15.72:6801/1012617371] conn(0x7f5f9c039600 0x7f5f9c03bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-26T21:42:36.425 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:32.021+0000 7f5fb9000700 1 -- 172.21.15.72:0/1108024460 >> [v2:172.21.15.72:6800/1012617371,v1:172.21.15.72:6801/1012617371] conn(0x7f5f9c039600 msgr2=0x7f5f9c03bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.72:6800/1012617371 2024-04-26T21:42:36.425 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:32.021+0000 7f5fb9000700 1 --2- 172.21.15.72:0/1108024460 >> [v2:172.21.15.72:6800/1012617371,v1:172.21.15.72:6801/1012617371] conn(0x7f5f9c039600 0x7f5f9c03bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-26T21:42:36.425 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:34.690+0000 7f5fb2000700 1 -- 172.21.15.72:0/1108024460 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mgrmap(e 6) ==== 46325+0+0 (secure 0 0 0) 0x7f5fa801aba0 con 0x7f5fb407dd40 2024-04-26T21:42:36.425 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:34.691+0000 7f5fb2000700 1 -- 172.21.15.72:0/1108024460 >> [v2:172.21.15.72:6800/1012617371,v1:172.21.15.72:6801/1012617371] conn(0x7f5f9c039600 msgr2=0x7f5f9c03bac0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T21:42:36.425 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:34.691+0000 7f5fb2000700 1 --2- 172.21.15.72:0/1108024460 >> [v2:172.21.15.72:6800/1012617371,v1:172.21.15.72:6801/1012617371] conn(0x7f5f9c039600 0x7f5f9c03bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:36.425 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:35.694+0000 7f5fb2000700 1 -- 172.21.15.72:0/1108024460 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mgrmap(e 7) ==== 46452+0+0 (secure 0 0 0) 0x7f5fa804f240 con 0x7f5fb407dd40 2024-04-26T21:42:36.425 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:35.695+0000 7f5fb2000700 1 --2- 172.21.15.72:0/1108024460 >> [v2:172.21.15.72:6800/4010358672,v1:172.21.15.72:6801/4010358672] conn(0x7f5f9c03cf40 0x7f5f9c03f330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:36.425 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:35.695+0000 7f5fb2000700 1 -- 172.21.15.72:0/1108024460 --> [v2:172.21.15.72:6800/4010358672,v1:172.21.15.72:6801/4010358672] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f5f9c03c1d0 con 0x7f5f9c03cf40 2024-04-26T21:42:36.425 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:35.697+0000 7f5fb9000700 1 --2- 172.21.15.72:0/1108024460 >> [v2:172.21.15.72:6800/4010358672,v1:172.21.15.72:6801/4010358672] conn(0x7f5f9c03cf40 0x7f5f9c03f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:36.425 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:35.700+0000 7f5fb9000700 1 --2- 172.21.15.72:0/1108024460 >> [v2:172.21.15.72:6800/4010358672,v1:172.21.15.72:6801/4010358672] conn(0x7f5f9c03cf40 0x7f5f9c03f330 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f5fa0003a10 tx=0x7f5fa00092b0 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:36.425 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:35.703+0000 7f5fb2000700 1 -- 172.21.15.72:0/1108024460 <== mgr.14120 v2:172.21.15.72:6800/4010358672 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7f5f9c03c1d0 con 0x7f5f9c03cf40 2024-04-26T21:42:36.426 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:35.713+0000 7f5fbc400700 1 -- 172.21.15.72:0/1108024460 --> [v2:172.21.15.72:6800/4010358672,v1:172.21.15.72:6801/4010358672] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f5fb406a520 con 0x7f5f9c03cf40 2024-04-26T21:42:36.426 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:35.714+0000 7f5fb2000700 1 -- 172.21.15.72:0/1108024460 <== mgr.14120 v2:172.21.15.72:6800/4010358672 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7f5fb406a520 con 0x7f5f9c03cf40 2024-04-26T21:42:36.426 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:35.714+0000 7f5fbc400700 1 -- 172.21.15.72:0/1108024460 >> [v2:172.21.15.72:6800/4010358672,v1:172.21.15.72:6801/4010358672] conn(0x7f5f9c03cf40 msgr2=0x7f5f9c03f330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:36.426 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:35.714+0000 7f5fbc400700 1 --2- 172.21.15.72:0/1108024460 >> [v2:172.21.15.72:6800/4010358672,v1:172.21.15.72:6801/4010358672] conn(0x7f5f9c03cf40 0x7f5f9c03f330 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f5fa0003a10 tx=0x7f5fa00092b0 comp rx=0 tx=0).stop 2024-04-26T21:42:36.426 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:35.714+0000 7f5fbc400700 1 -- 172.21.15.72:0/1108024460 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5fb407dd40 msgr2=0x7f5fb419ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:36.426 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:35.714+0000 7f5fbc400700 1 --2- 172.21.15.72:0/1108024460 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5fb407dd40 0x7f5fb419ed60 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f5fa8006b20 tx=0x7f5fa8011a60 comp rx=0 tx=0).stop 2024-04-26T21:42:36.426 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:35.714+0000 7f5fbc400700 1 -- 172.21.15.72:0/1108024460 shutdown_connections 2024-04-26T21:42:36.426 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:35.714+0000 7f5fbc400700 1 --2- 172.21.15.72:0/1108024460 >> [v2:172.21.15.72:6800/4010358672,v1:172.21.15.72:6801/4010358672] conn(0x7f5f9c03cf40 0x7f5f9c03f330 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:36.426 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:35.714+0000 7f5fbc400700 1 --2- 172.21.15.72:0/1108024460 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5fb407dd40 0x7f5fb419ed60 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:36.426 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:35.714+0000 7f5fbc400700 1 --2- 172.21.15.72:0/1108024460 >> [v2:172.21.15.72:6800/1012617371,v1:172.21.15.72:6801/1012617371] conn(0x7f5f9c039600 0x7f5f9c03bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:36.426 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:35.714+0000 7f5fbc400700 1 -- 172.21.15.72:0/1108024460 >> 172.21.15.72:0/1108024460 conn(0x7f5fb4104260 msgr2=0x7f5fb4104d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:36.426 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:35.715+0000 7f5fbc400700 1 -- 172.21.15.72:0/1108024460 shutdown_connections 2024-04-26T21:42:36.426 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:35.715+0000 7f5fbc400700 1 -- 172.21.15.72:0/1108024460 wait complete. 2024-04-26T21:42:36.427 INFO:teuthology.orchestra.run.smithi072.stdout:mgr epoch 5 is available 2024-04-26T21:42:36.427 INFO:teuthology.orchestra.run.smithi072.stdout:Setting orchestrator backend to cephadm... 2024-04-26T21:42:37.010 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:36 smithi072 ceph-mon[28138]: [26/Apr/2024:21:42:35] ENGINE Bus STARTING 2024-04-26T21:42:37.010 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:36 smithi072 ceph-mon[28138]: [26/Apr/2024:21:42:35] ENGINE Serving on https://172.21.15.72:7150 2024-04-26T21:42:37.010 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:36 smithi072 ceph-mon[28138]: [26/Apr/2024:21:42:35] ENGINE Serving on http://172.21.15.72:8765 2024-04-26T21:42:37.010 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:36 smithi072 ceph-mon[28138]: [26/Apr/2024:21:42:35] ENGINE Bus STARTED 2024-04-26T21:42:37.011 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:36 smithi072 ceph-mon[28138]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-26T21:42:37.011 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:36 smithi072 ceph-mon[28138]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-26T21:42:37.961 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:37 smithi072 ceph-mon[28138]: mgrmap e8: x(active, since 2s) 2024-04-26T21:42:37.961 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:37 smithi072 ceph-mon[28138]: from='mgr.14120 172.21.15.72:0/474756945' entity='mgr.x' 2024-04-26T21:42:37.962 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:37 smithi072 ceph-mon[28138]: from='mgr.14120 172.21.15.72:0/474756945' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:42:37.974 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:37.018+0000 7f8009200700 1 Processor -- start 2024-04-26T21:42:37.974 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:37.018+0000 7f8009200700 1 -- start start 2024-04-26T21:42:37.974 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:37.019+0000 7f8009200700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8004109760 0x7f800410bb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:37.975 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:37.019+0000 7f8009200700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f8004064680 con 0x7f8004109760 2024-04-26T21:42:37.975 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:37.019+0000 7f8001e00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8004109760 0x7f800410bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:37.975 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:37.019+0000 7f8001e00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8004109760 0x7f800410bb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:40358/0 (socket says 172.21.15.72:40358) 2024-04-26T21:42:37.975 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:37.019+0000 7f8001e00700 1 -- 172.21.15.72:0/2168701800 learned_addr learned my addr 172.21.15.72:0/2168701800 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:42:37.975 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:37.019+0000 7f8001e00700 1 -- 172.21.15.72:0/2168701800 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f800410c090 con 0x7f8004109760 2024-04-26T21:42:37.975 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:37.020+0000 7f8001e00700 1 --2- 172.21.15.72:0/2168701800 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8004109760 0x7f800410bb50 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f7ff40098a0 tx=0x7f7ff4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71c41b49bca87e55 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:37.975 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:37.020+0000 7f8000a00700 1 -- 172.21.15.72:0/2168701800 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7ff4004160 con 0x7f8004109760 2024-04-26T21:42:37.975 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:37.020+0000 7f8000a00700 1 -- 172.21.15.72:0/2168701800 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f7ff4004300 con 0x7f8004109760 2024-04-26T21:42:37.975 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:37.020+0000 7f8000a00700 1 -- 172.21.15.72:0/2168701800 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7ff40119e0 con 0x7f8004109760 2024-04-26T21:42:37.975 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:37.021+0000 7f8009200700 1 -- 172.21.15.72:0/2168701800 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8004109760 msgr2=0x7f800410bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:37.975 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:37.021+0000 7f8009200700 1 --2- 172.21.15.72:0/2168701800 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8004109760 0x7f800410bb50 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f7ff40098a0 tx=0x7f7ff4009bb0 comp rx=0 tx=0).stop 2024-04-26T21:42:37.975 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:37.021+0000 7f8009200700 1 -- 172.21.15.72:0/2168701800 shutdown_connections 2024-04-26T21:42:37.975 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:37.021+0000 7f8009200700 1 --2- 172.21.15.72:0/2168701800 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8004109760 0x7f800410bb50 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:37.975 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:37.021+0000 7f8009200700 1 -- 172.21.15.72:0/2168701800 >> 172.21.15.72:0/2168701800 conn(0x7f80041030d0 msgr2=0x7f8004105510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:37.975 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:37.021+0000 7f8009200700 1 -- 172.21.15.72:0/2168701800 shutdown_connections 2024-04-26T21:42:37.976 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:37.021+0000 7f8009200700 1 -- 172.21.15.72:0/2168701800 wait complete. 2024-04-26T21:42:37.976 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:37.022+0000 7f8009200700 1 Processor -- start 2024-04-26T21:42:37.976 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:37.022+0000 7f8009200700 1 -- start start 2024-04-26T21:42:37.976 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:37.022+0000 7f8009200700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8004109760 0x7f8004108fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:37.976 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:37.022+0000 7f8009200700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f800410cf40 con 0x7f8004109760 2024-04-26T21:42:37.976 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:37.023+0000 7f8001e00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8004109760 0x7f8004108fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:37.976 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:37.023+0000 7f8001e00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8004109760 0x7f8004108fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:40374/0 (socket says 172.21.15.72:40374) 2024-04-26T21:42:37.976 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:37.023+0000 7f8001e00700 1 -- 172.21.15.72:0/4294685289 learned_addr learned my addr 172.21.15.72:0/4294685289 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:42:37.976 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:37.023+0000 7f8001e00700 1 -- 172.21.15.72:0/4294685289 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8004109520 con 0x7f8004109760 2024-04-26T21:42:37.976 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:37.023+0000 7f8001e00700 1 --2- 172.21.15.72:0/4294685289 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8004109760 0x7f8004108fe0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f7ff4009130 tx=0x7f7ff4011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:37.976 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:37.024+0000 7f7ffaa00700 1 -- 172.21.15.72:0/4294685289 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7ff4003b20 con 0x7f8004109760 2024-04-26T21:42:37.976 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:37.024+0000 7f7ffaa00700 1 -- 172.21.15.72:0/4294685289 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f7ff401a8a0 con 0x7f8004109760 2024-04-26T21:42:37.976 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:37.024+0000 7f7ffaa00700 1 -- 172.21.15.72:0/4294685289 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7ff4024810 con 0x7f8004109760 2024-04-26T21:42:37.976 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:37.024+0000 7f8009200700 1 -- 172.21.15.72:0/4294685289 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8004107710 con 0x7f8004109760 2024-04-26T21:42:37.976 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:37.024+0000 7f8009200700 1 -- 172.21.15.72:0/4294685289 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8004107c50 con 0x7f8004109760 2024-04-26T21:42:37.977 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:37.025+0000 7f7ffaa00700 1 -- 172.21.15.72:0/4294685289 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f7ff4022020 con 0x7f8004109760 2024-04-26T21:42:37.977 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:37.025+0000 7f8009200700 1 -- 172.21.15.72:0/4294685289 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8004064a80 con 0x7f8004109760 2024-04-26T21:42:37.977 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:37.025+0000 7f7ffaa00700 1 --2- 172.21.15.72:0/4294685289 >> [v2:172.21.15.72:6800/4010358672,v1:172.21.15.72:6801/4010358672] conn(0x7f7fe8039660 0x7f7fe803bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:37.977 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:37.025+0000 7f7ffaa00700 1 -- 172.21.15.72:0/4294685289 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f7ff404da00 con 0x7f8004109760 2024-04-26T21:42:37.977 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:37.026+0000 7f8001400700 1 --2- 172.21.15.72:0/4294685289 >> [v2:172.21.15.72:6800/4010358672,v1:172.21.15.72:6801/4010358672] conn(0x7f7fe8039660 0x7f7fe803bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:37.977 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:37.026+0000 7f8001400700 1 --2- 172.21.15.72:0/4294685289 >> [v2:172.21.15.72:6800/4010358672,v1:172.21.15.72:6801/4010358672] conn(0x7f7fe8039660 0x7f7fe803bb20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f7ff0006fd0 tx=0x7f7ff0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:37.977 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:37.032+0000 7f7ffaa00700 1 -- 172.21.15.72:0/4294685289 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f7ff400b3e0 con 0x7f8004109760 2024-04-26T21:42:37.977 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:37.274+0000 7f8009200700 1 -- 172.21.15.72:0/4294685289 --> [v2:172.21.15.72:6800/4010358672,v1:172.21.15.72:6801/4010358672] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7f8004104320 con 0x7f7fe8039660 2024-04-26T21:42:37.977 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:37.286+0000 7f7ffaa00700 1 -- 172.21.15.72:0/4294685289 <== mgr.14120 v2:172.21.15.72:6800/4010358672 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f8004104320 con 0x7f7fe8039660 2024-04-26T21:42:37.977 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:37.289+0000 7f8009200700 1 -- 172.21.15.72:0/4294685289 >> [v2:172.21.15.72:6800/4010358672,v1:172.21.15.72:6801/4010358672] conn(0x7f7fe8039660 msgr2=0x7f7fe803bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:37.977 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:37.289+0000 7f8009200700 1 --2- 172.21.15.72:0/4294685289 >> [v2:172.21.15.72:6800/4010358672,v1:172.21.15.72:6801/4010358672] conn(0x7f7fe8039660 0x7f7fe803bb20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f7ff0006fd0 tx=0x7f7ff0006e40 comp rx=0 tx=0).stop 2024-04-26T21:42:37.977 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:37.289+0000 7f8009200700 1 -- 172.21.15.72:0/4294685289 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8004109760 msgr2=0x7f8004108fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:37.977 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:37.289+0000 7f8009200700 1 --2- 172.21.15.72:0/4294685289 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8004109760 0x7f8004108fe0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f7ff4009130 tx=0x7f7ff4011a60 comp rx=0 tx=0).stop 2024-04-26T21:42:37.977 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:37.290+0000 7f8009200700 1 -- 172.21.15.72:0/4294685289 shutdown_connections 2024-04-26T21:42:37.978 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:37.290+0000 7f8009200700 1 --2- 172.21.15.72:0/4294685289 >> [v2:172.21.15.72:6800/4010358672,v1:172.21.15.72:6801/4010358672] conn(0x7f7fe8039660 0x7f7fe803bb20 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:37.978 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:37.290+0000 7f8009200700 1 --2- 172.21.15.72:0/4294685289 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8004109760 0x7f8004108fe0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:37.978 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:37.290+0000 7f8009200700 1 -- 172.21.15.72:0/4294685289 >> 172.21.15.72:0/4294685289 conn(0x7f80041030d0 msgr2=0x7f8004103c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:37.978 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:37.290+0000 7f8009200700 1 -- 172.21.15.72:0/4294685289 shutdown_connections 2024-04-26T21:42:37.978 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:37.290+0000 7f8009200700 1 -- 172.21.15.72:0/4294685289 wait complete. 2024-04-26T21:42:38.895 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:38 smithi072 ceph-mon[28138]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:42:39.584 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout value unchanged 2024-04-26T21:42:39.584 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:38.628+0000 7f1968e00700 1 Processor -- start 2024-04-26T21:42:39.584 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:38.628+0000 7f1968e00700 1 -- start start 2024-04-26T21:42:39.585 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:38.628+0000 7f1968e00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f196410ae80 0x7f196410b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:39.587 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:38.629+0000 7f1968e00700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f196410b7e0 con 0x7f196410ae80 2024-04-26T21:42:39.587 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:38.629+0000 7f1961e00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f196410ae80 0x7f196410b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:39.587 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:38.629+0000 7f1961e00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f196410ae80 0x7f196410b2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:40384/0 (socket says 172.21.15.72:40384) 2024-04-26T21:42:39.587 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:38.629+0000 7f1961e00700 1 -- 172.21.15.72:0/1673279209 learned_addr learned my addr 172.21.15.72:0/1673279209 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:42:39.587 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:38.630+0000 7f1961e00700 1 -- 172.21.15.72:0/1673279209 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f196410bfe0 con 0x7f196410ae80 2024-04-26T21:42:39.587 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:38.630+0000 7f1961e00700 1 --2- 172.21.15.72:0/1673279209 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f196410ae80 0x7f196410b2a0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f19540098a0 tx=0x7f1954009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2074fa7b3e38750 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:39.587 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:38.630+0000 7f1960a00700 1 -- 172.21.15.72:0/1673279209 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1954004160 con 0x7f196410ae80 2024-04-26T21:42:39.587 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:38.630+0000 7f1960a00700 1 -- 172.21.15.72:0/1673279209 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f1954004300 con 0x7f196410ae80 2024-04-26T21:42:39.587 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:38.630+0000 7f1960a00700 1 -- 172.21.15.72:0/1673279209 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f19540119e0 con 0x7f196410ae80 2024-04-26T21:42:39.588 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:38.631+0000 7f1968e00700 1 -- 172.21.15.72:0/1673279209 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f196410ae80 msgr2=0x7f196410b2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:39.588 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:38.631+0000 7f1968e00700 1 --2- 172.21.15.72:0/1673279209 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f196410ae80 0x7f196410b2a0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f19540098a0 tx=0x7f1954009bb0 comp rx=0 tx=0).stop 2024-04-26T21:42:39.588 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:38.631+0000 7f1968e00700 1 -- 172.21.15.72:0/1673279209 shutdown_connections 2024-04-26T21:42:39.588 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:38.631+0000 7f1968e00700 1 --2- 172.21.15.72:0/1673279209 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f196410ae80 0x7f196410b2a0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:39.588 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:38.631+0000 7f1968e00700 1 -- 172.21.15.72:0/1673279209 >> 172.21.15.72:0/1673279209 conn(0x7f196407e270 msgr2=0x7f196407e680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:39.588 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:38.631+0000 7f1968e00700 1 -- 172.21.15.72:0/1673279209 shutdown_connections 2024-04-26T21:42:39.588 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:38.631+0000 7f1968e00700 1 -- 172.21.15.72:0/1673279209 wait complete. 2024-04-26T21:42:39.592 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:38.632+0000 7f1968e00700 1 Processor -- start 2024-04-26T21:42:39.592 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:38.632+0000 7f1968e00700 1 -- start start 2024-04-26T21:42:39.592 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:38.633+0000 7f1968e00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f196410ae80 0x7f196419ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:39.592 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:38.633+0000 7f1968e00700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f196410f2a0 con 0x7f196410ae80 2024-04-26T21:42:39.592 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:38.633+0000 7f1961e00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f196410ae80 0x7f196419ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:39.592 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:38.633+0000 7f1961e00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f196410ae80 0x7f196419ec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:40390/0 (socket says 172.21.15.72:40390) 2024-04-26T21:42:39.592 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:38.633+0000 7f1961e00700 1 -- 172.21.15.72:0/3231137213 learned_addr learned my addr 172.21.15.72:0/3231137213 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:42:39.592 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:38.633+0000 7f1961e00700 1 -- 172.21.15.72:0/3231137213 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f196419f1d0 con 0x7f196410ae80 2024-04-26T21:42:39.592 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:38.634+0000 7f1961e00700 1 --2- 172.21.15.72:0/3231137213 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f196410ae80 0x7f196419ec90 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f1954009130 tx=0x7f1954011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:39.592 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:38.634+0000 7f195aa00700 1 -- 172.21.15.72:0/3231137213 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1954003b20 con 0x7f196410ae80 2024-04-26T21:42:39.592 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:38.634+0000 7f195aa00700 1 -- 172.21.15.72:0/3231137213 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f195401a8a0 con 0x7f196410ae80 2024-04-26T21:42:39.593 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:38.634+0000 7f1968e00700 1 -- 172.21.15.72:0/3231137213 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f196419f460 con 0x7f196410ae80 2024-04-26T21:42:39.593 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:38.634+0000 7f195aa00700 1 -- 172.21.15.72:0/3231137213 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1954024810 con 0x7f196410ae80 2024-04-26T21:42:39.593 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:38.634+0000 7f1968e00700 1 -- 172.21.15.72:0/3231137213 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19641a2150 con 0x7f196410ae80 2024-04-26T21:42:39.593 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:38.635+0000 7f195aa00700 1 -- 172.21.15.72:0/3231137213 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f1954022020 con 0x7f196410ae80 2024-04-26T21:42:39.593 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:38.635+0000 7f1968e00700 1 -- 172.21.15.72:0/3231137213 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f196406a520 con 0x7f196410ae80 2024-04-26T21:42:39.593 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:38.636+0000 7f195aa00700 1 --2- 172.21.15.72:0/3231137213 >> [v2:172.21.15.72:6800/4010358672,v1:172.21.15.72:6801/4010358672] conn(0x7f19480395b0 0x7f194803ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:39.593 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:38.636+0000 7f195aa00700 1 -- 172.21.15.72:0/3231137213 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f195404da40 con 0x7f196410ae80 2024-04-26T21:42:39.593 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:38.636+0000 7f1961400700 1 --2- 172.21.15.72:0/3231137213 >> [v2:172.21.15.72:6800/4010358672,v1:172.21.15.72:6801/4010358672] conn(0x7f19480395b0 0x7f194803ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:39.593 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:38.637+0000 7f1961400700 1 --2- 172.21.15.72:0/3231137213 >> [v2:172.21.15.72:6800/4010358672,v1:172.21.15.72:6801/4010358672] conn(0x7f19480395b0 0x7f194803ba70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1950006fd0 tx=0x7f1950006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:39.593 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:38.642+0000 7f195aa00700 1 -- 172.21.15.72:0/3231137213 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f195400b3e0 con 0x7f196410ae80 2024-04-26T21:42:39.593 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:38.872+0000 7f1968e00700 1 -- 172.21.15.72:0/3231137213 --> [v2:172.21.15.72:6800/4010358672,v1:172.21.15.72:6801/4010358672] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7f1964108a80 con 0x7f19480395b0 2024-04-26T21:42:39.593 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:38.874+0000 7f195aa00700 1 -- 172.21.15.72:0/3231137213 <== mgr.14120 v2:172.21.15.72:6800/4010358672 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7f1964108a80 con 0x7f19480395b0 2024-04-26T21:42:39.593 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:38.877+0000 7f1968e00700 1 -- 172.21.15.72:0/3231137213 >> [v2:172.21.15.72:6800/4010358672,v1:172.21.15.72:6801/4010358672] conn(0x7f19480395b0 msgr2=0x7f194803ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:39.594 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:38.877+0000 7f1968e00700 1 --2- 172.21.15.72:0/3231137213 >> [v2:172.21.15.72:6800/4010358672,v1:172.21.15.72:6801/4010358672] conn(0x7f19480395b0 0x7f194803ba70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1950006fd0 tx=0x7f1950006e40 comp rx=0 tx=0).stop 2024-04-26T21:42:39.594 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:38.877+0000 7f1968e00700 1 -- 172.21.15.72:0/3231137213 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f196410ae80 msgr2=0x7f196419ec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:39.594 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:38.877+0000 7f1968e00700 1 --2- 172.21.15.72:0/3231137213 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f196410ae80 0x7f196419ec90 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f1954009130 tx=0x7f1954011a60 comp rx=0 tx=0).stop 2024-04-26T21:42:39.594 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:38.877+0000 7f1968e00700 1 -- 172.21.15.72:0/3231137213 shutdown_connections 2024-04-26T21:42:39.594 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:38.877+0000 7f1968e00700 1 --2- 172.21.15.72:0/3231137213 >> [v2:172.21.15.72:6800/4010358672,v1:172.21.15.72:6801/4010358672] conn(0x7f19480395b0 0x7f194803ba70 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:39.594 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:38.877+0000 7f1968e00700 1 --2- 172.21.15.72:0/3231137213 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f196410ae80 0x7f196419ec90 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:39.594 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:38.877+0000 7f1968e00700 1 -- 172.21.15.72:0/3231137213 >> 172.21.15.72:0/3231137213 conn(0x7f196407e270 msgr2=0x7f1964108370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:39.594 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:38.877+0000 7f1968e00700 1 -- 172.21.15.72:0/3231137213 shutdown_connections 2024-04-26T21:42:39.594 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:38.877+0000 7f1968e00700 1 -- 172.21.15.72:0/3231137213 wait complete. 2024-04-26T21:42:39.594 INFO:teuthology.orchestra.run.smithi072.stdout:Generating ssh key... 2024-04-26T21:42:39.885 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:39 smithi072 ceph-mon[28138]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:42:41.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:40 smithi072 ceph-mon[28138]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:42:41.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:40 smithi072 ceph-mon[28138]: Generating ssh key... 2024-04-26T21:42:41.323 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 26 21:42:41 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-mgr-x[28363]: Generating public/private rsa key pair. 2024-04-26T21:42:41.323 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 26 21:42:41 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-mgr-x[28363]: Your identification has been saved in /tmp/tmpsb25_rqm/key. 2024-04-26T21:42:41.323 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 26 21:42:41 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-mgr-x[28363]: Your public key has been saved in /tmp/tmpsb25_rqm/key.pub. 2024-04-26T21:42:41.323 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 26 21:42:41 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-mgr-x[28363]: The key fingerprint is: 2024-04-26T21:42:41.324 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 26 21:42:41 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-mgr-x[28363]: SHA256:oFs7Ls0rwECYs2FgmL1MuW2EsZNxUAPXdKqNkIV08vM ceph-a1780950-0415-11ef-bc93-c7b262605968 2024-04-26T21:42:41.324 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 26 21:42:41 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-mgr-x[28363]: The key's randomart image is: 2024-04-26T21:42:41.324 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 26 21:42:41 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-mgr-x[28363]: +---[RSA 3072]----+ 2024-04-26T21:42:41.324 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 26 21:42:41 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-mgr-x[28363]: |=*BOBo. . | 2024-04-26T21:42:41.324 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 26 21:42:41 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-mgr-x[28363]: |Oo*%...o | 2024-04-26T21:42:41.324 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 26 21:42:41 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-mgr-x[28363]: |o**=o o | 2024-04-26T21:42:41.324 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 26 21:42:41 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-mgr-x[28363]: |o +ooB . | 2024-04-26T21:42:41.324 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 26 21:42:41 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-mgr-x[28363]: | o .+ E S | 2024-04-26T21:42:41.324 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 26 21:42:41 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-mgr-x[28363]: | o o . | 2024-04-26T21:42:41.324 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 26 21:42:41 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-mgr-x[28363]: | ..oo | 2024-04-26T21:42:41.324 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 26 21:42:41 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-mgr-x[28363]: | o.o. | 2024-04-26T21:42:41.325 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 26 21:42:41 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-mgr-x[28363]: | oo. | 2024-04-26T21:42:41.325 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 26 21:42:41 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-mgr-x[28363]: +----[SHA256]-----+ 2024-04-26T21:42:41.809 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:40.194+0000 7fa768200700 1 Processor -- start 2024-04-26T21:42:41.809 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:40.194+0000 7fa768200700 1 -- start start 2024-04-26T21:42:41.809 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:40.194+0000 7fa768200700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa76007ee70 0x7fa76007d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:41.809 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:40.194+0000 7fa768200700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fa76007d8a0 con 0x7fa76007ee70 2024-04-26T21:42:41.813 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:40.195+0000 7fa765800700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa76007ee70 0x7fa76007d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:41.813 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:40.195+0000 7fa765800700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa76007ee70 0x7fa76007d2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:40398/0 (socket says 172.21.15.72:40398) 2024-04-26T21:42:41.813 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:40.195+0000 7fa765800700 1 -- 172.21.15.72:0/1629408176 learned_addr learned my addr 172.21.15.72:0/1629408176 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:42:41.813 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:40.195+0000 7fa765800700 1 -- 172.21.15.72:0/1629408176 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa76007da20 con 0x7fa76007ee70 2024-04-26T21:42:41.813 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:40.196+0000 7fa765800700 1 --2- 172.21.15.72:0/1629408176 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa76007ee70 0x7fa76007d2d0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fa7540098a0 tx=0x7fa754009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a28154bb27c0e2ef server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:41.813 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:40.196+0000 7fa75fe00700 1 -- 172.21.15.72:0/1629408176 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa754004160 con 0x7fa76007ee70 2024-04-26T21:42:41.813 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:40.196+0000 7fa75fe00700 1 -- 172.21.15.72:0/1629408176 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fa754004300 con 0x7fa76007ee70 2024-04-26T21:42:41.813 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:40.196+0000 7fa75fe00700 1 -- 172.21.15.72:0/1629408176 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa7540119e0 con 0x7fa76007ee70 2024-04-26T21:42:41.813 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:40.197+0000 7fa768200700 1 -- 172.21.15.72:0/1629408176 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa76007ee70 msgr2=0x7fa76007d2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:41.814 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:40.197+0000 7fa768200700 1 --2- 172.21.15.72:0/1629408176 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa76007ee70 0x7fa76007d2d0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fa7540098a0 tx=0x7fa754009bb0 comp rx=0 tx=0).stop 2024-04-26T21:42:41.814 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:40.197+0000 7fa768200700 1 -- 172.21.15.72:0/1629408176 shutdown_connections 2024-04-26T21:42:41.814 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:40.197+0000 7fa768200700 1 --2- 172.21.15.72:0/1629408176 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa76007ee70 0x7fa76007d2d0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:41.814 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:40.197+0000 7fa768200700 1 -- 172.21.15.72:0/1629408176 >> 172.21.15.72:0/1629408176 conn(0x7fa760104240 msgr2=0x7fa760106660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:41.814 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:40.197+0000 7fa768200700 1 -- 172.21.15.72:0/1629408176 shutdown_connections 2024-04-26T21:42:41.814 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:40.197+0000 7fa768200700 1 -- 172.21.15.72:0/1629408176 wait complete. 2024-04-26T21:42:41.814 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:40.198+0000 7fa768200700 1 Processor -- start 2024-04-26T21:42:41.814 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:40.198+0000 7fa768200700 1 -- start start 2024-04-26T21:42:41.814 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:40.198+0000 7fa768200700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa76007ee70 0x7fa7601a3150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:41.814 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:40.198+0000 7fa768200700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fa76010adb0 con 0x7fa76007ee70 2024-04-26T21:42:41.815 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:40.198+0000 7fa765800700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa76007ee70 0x7fa7601a3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:41.815 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:40.199+0000 7fa765800700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa76007ee70 0x7fa7601a3150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:40414/0 (socket says 172.21.15.72:40414) 2024-04-26T21:42:41.815 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:40.199+0000 7fa765800700 1 -- 172.21.15.72:0/3177463149 learned_addr learned my addr 172.21.15.72:0/3177463149 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:42:41.815 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:40.199+0000 7fa765800700 1 -- 172.21.15.72:0/3177463149 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7601a3690 con 0x7fa76007ee70 2024-04-26T21:42:41.815 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:40.199+0000 7fa765800700 1 --2- 172.21.15.72:0/3177463149 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa76007ee70 0x7fa7601a3150 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fa754006b20 tx=0x7fa754011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:41.815 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:40.199+0000 7fa75e000700 1 -- 172.21.15.72:0/3177463149 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa754003a60 con 0x7fa76007ee70 2024-04-26T21:42:41.815 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:40.199+0000 7fa75e000700 1 -- 172.21.15.72:0/3177463149 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fa754003c00 con 0x7fa76007ee70 2024-04-26T21:42:41.815 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:40.199+0000 7fa768200700 1 -- 172.21.15.72:0/3177463149 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7601a3920 con 0x7fa76007ee70 2024-04-26T21:42:41.815 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:40.199+0000 7fa75e000700 1 -- 172.21.15.72:0/3177463149 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa754024780 con 0x7fa76007ee70 2024-04-26T21:42:41.815 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:40.200+0000 7fa768200700 1 -- 172.21.15.72:0/3177463149 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7601a6610 con 0x7fa76007ee70 2024-04-26T21:42:41.815 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:40.201+0000 7fa75e000700 1 -- 172.21.15.72:0/3177463149 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7fa754022020 con 0x7fa76007ee70 2024-04-26T21:42:41.815 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:40.201+0000 7fa768200700 1 -- 172.21.15.72:0/3177463149 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa760064a80 con 0x7fa76007ee70 2024-04-26T21:42:41.815 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:40.201+0000 7fa75e000700 1 --2- 172.21.15.72:0/3177463149 >> [v2:172.21.15.72:6800/4010358672,v1:172.21.15.72:6801/4010358672] conn(0x7fa7480395b0 0x7fa74803ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:41.815 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:40.201+0000 7fa75e000700 1 -- 172.21.15.72:0/3177463149 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fa75404dbb0 con 0x7fa76007ee70 2024-04-26T21:42:41.816 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:40.201+0000 7fa764e00700 1 --2- 172.21.15.72:0/3177463149 >> [v2:172.21.15.72:6800/4010358672,v1:172.21.15.72:6801/4010358672] conn(0x7fa7480395b0 0x7fa74803ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:41.816 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:40.202+0000 7fa764e00700 1 --2- 172.21.15.72:0/3177463149 >> [v2:172.21.15.72:6800/4010358672,v1:172.21.15.72:6801/4010358672] conn(0x7fa7480395b0 0x7fa74803ba70 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fa74c006fd0 tx=0x7fa74c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:41.816 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:40.206+0000 7fa75e000700 1 -- 172.21.15.72:0/3177463149 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fa75404d480 con 0x7fa76007ee70 2024-04-26T21:42:41.816 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:40.434+0000 7fa768200700 1 -- 172.21.15.72:0/3177463149 --> [v2:172.21.15.72:6800/4010358672,v1:172.21.15.72:6801/4010358672] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7fa7600008d0 con 0x7fa7480395b0 2024-04-26T21:42:41.816 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:41.128+0000 7fa75e000700 1 -- 172.21.15.72:0/3177463149 <== mgr.14120 v2:172.21.15.72:6800/4010358672 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fa7600008d0 con 0x7fa7480395b0 2024-04-26T21:42:41.816 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:41.131+0000 7fa768200700 1 -- 172.21.15.72:0/3177463149 >> [v2:172.21.15.72:6800/4010358672,v1:172.21.15.72:6801/4010358672] conn(0x7fa7480395b0 msgr2=0x7fa74803ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:41.816 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:41.131+0000 7fa768200700 1 --2- 172.21.15.72:0/3177463149 >> [v2:172.21.15.72:6800/4010358672,v1:172.21.15.72:6801/4010358672] conn(0x7fa7480395b0 0x7fa74803ba70 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fa74c006fd0 tx=0x7fa74c006e40 comp rx=0 tx=0).stop 2024-04-26T21:42:41.816 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:41.131+0000 7fa768200700 1 -- 172.21.15.72:0/3177463149 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa76007ee70 msgr2=0x7fa7601a3150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:41.816 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:41.131+0000 7fa768200700 1 --2- 172.21.15.72:0/3177463149 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa76007ee70 0x7fa7601a3150 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fa754006b20 tx=0x7fa754011a60 comp rx=0 tx=0).stop 2024-04-26T21:42:41.816 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:41.131+0000 7fa768200700 1 -- 172.21.15.72:0/3177463149 shutdown_connections 2024-04-26T21:42:41.816 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:41.131+0000 7fa768200700 1 --2- 172.21.15.72:0/3177463149 >> [v2:172.21.15.72:6800/4010358672,v1:172.21.15.72:6801/4010358672] conn(0x7fa7480395b0 0x7fa74803ba70 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:41.816 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:41.131+0000 7fa768200700 1 --2- 172.21.15.72:0/3177463149 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa76007ee70 0x7fa7601a3150 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:41.816 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:41.131+0000 7fa768200700 1 -- 172.21.15.72:0/3177463149 >> 172.21.15.72:0/3177463149 conn(0x7fa760104240 msgr2=0x7fa760106630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:41.816 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:41.131+0000 7fa768200700 1 -- 172.21.15.72:0/3177463149 shutdown_connections 2024-04-26T21:42:41.817 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:41.131+0000 7fa768200700 1 -- 172.21.15.72:0/3177463149 wait complete. 2024-04-26T21:42:42.443 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:42 smithi072 ceph-mon[28138]: from='mgr.14120 172.21.15.72:0/474756945' entity='mgr.x' 2024-04-26T21:42:42.443 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:42 smithi072 ceph-mon[28138]: from='mgr.14120 172.21.15.72:0/474756945' entity='mgr.x' 2024-04-26T21:42:43.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:43 smithi072 ceph-mon[28138]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:42:43.443 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDBwVwmjYjtJrIQXFLnz1GGuT/um9Uys5TnoD6QESzyxrCzFZmQK7I+tbp7X2qEDf1XFODVrovfs8F+KS1Ycbr2J4pORAZLRE514yl2rW2AXDqx9k5Mkw+nos14kJdoW/ulRAf0eTR6kyIAsod6K+M6t0pCohYbP75FcL2ZEdsSK/K2LMRK/lXKrGupuGrg+sFjfYrYyTIOQOHL7mum3FDQYygC1xZtj+Khcv1Yf8KBhZU7PaTDsO6OiThdJMJfePMQE+3IcqczRjWFJ1e+ik0r5tB0JqWjpxiRcG0gxZs48yZNzhJFU6fxqkFeaSM5NtHfBE0eVuNpYUw7qgLjK8pqhpaKjewI/oWHqs821p/k31G+Cp0quEZJQHvmKVl5pUHUoef3F3c+XGGYoRnOZy7KxSt5FOh6CbNQYqLz4i51vxlGS+4MOc5odatBBFIuJh0HhtdyVxknqemAR9e9o/JuU39xn+wDD1G+7VB7U3CkrHt2NI+ExbwDK70ginPIr2M= ceph-a1780950-0415-11ef-bc93-c7b262605968 2024-04-26T21:42:43.443 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:42.503+0000 7f389d800700 1 Processor -- start 2024-04-26T21:42:43.443 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:42.503+0000 7f389d800700 1 -- start start 2024-04-26T21:42:43.443 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:42.503+0000 7f389d800700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f389810fc60 0x7f3898112050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:43.443 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:42.504+0000 7f389d800700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f38980834c0 con 0x7f389810fc60 2024-04-26T21:42:43.443 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:42.504+0000 7f3896a00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f389810fc60 0x7f3898112050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:43.444 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:42.504+0000 7f3896a00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f389810fc60 0x7f3898112050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:36360/0 (socket says 172.21.15.72:36360) 2024-04-26T21:42:43.444 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:42.504+0000 7f3896a00700 1 -- 172.21.15.72:0/2762878443 learned_addr learned my addr 172.21.15.72:0/2762878443 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:42:43.449 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:42.505+0000 7f3896a00700 1 -- 172.21.15.72:0/2762878443 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3898112590 con 0x7f389810fc60 2024-04-26T21:42:43.449 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:42.505+0000 7f3896a00700 1 --2- 172.21.15.72:0/2762878443 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f389810fc60 0x7f3898112050 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f38880098a0 tx=0x7f3888009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7fc2071b37f8776 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:43.449 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:42.506+0000 7f3895600700 1 -- 172.21.15.72:0/2762878443 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3888004160 con 0x7f389810fc60 2024-04-26T21:42:43.449 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:42.506+0000 7f3895600700 1 -- 172.21.15.72:0/2762878443 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f3888004300 con 0x7f389810fc60 2024-04-26T21:42:43.449 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:42.506+0000 7f3895600700 1 -- 172.21.15.72:0/2762878443 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f38880119e0 con 0x7f389810fc60 2024-04-26T21:42:43.449 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:42.507+0000 7f389d800700 1 -- 172.21.15.72:0/2762878443 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f389810fc60 msgr2=0x7f3898112050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:43.449 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:42.507+0000 7f389d800700 1 --2- 172.21.15.72:0/2762878443 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f389810fc60 0x7f3898112050 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f38880098a0 tx=0x7f3888009bb0 comp rx=0 tx=0).stop 2024-04-26T21:42:43.449 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:42.507+0000 7f389d800700 1 -- 172.21.15.72:0/2762878443 shutdown_connections 2024-04-26T21:42:43.449 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:42.507+0000 7f389d800700 1 --2- 172.21.15.72:0/2762878443 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f389810fc60 0x7f3898112050 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:43.449 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:42.507+0000 7f389d800700 1 -- 172.21.15.72:0/2762878443 >> 172.21.15.72:0/2762878443 conn(0x7f389807e220 msgr2=0x7f3898080660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:43.449 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:42.507+0000 7f389d800700 1 -- 172.21.15.72:0/2762878443 shutdown_connections 2024-04-26T21:42:43.449 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:42.507+0000 7f389d800700 1 -- 172.21.15.72:0/2762878443 wait complete. 2024-04-26T21:42:43.450 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:42.508+0000 7f389d800700 1 Processor -- start 2024-04-26T21:42:43.450 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:42.508+0000 7f389d800700 1 -- start start 2024-04-26T21:42:43.450 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:42.509+0000 7f389d800700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f389810fc60 0x7f38981a30e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:43.450 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:42.509+0000 7f389d800700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f3898113560 con 0x7f389810fc60 2024-04-26T21:42:43.450 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:42.509+0000 7f3896a00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f389810fc60 0x7f38981a30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:43.450 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:42.509+0000 7f3896a00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f389810fc60 0x7f38981a30e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:36366/0 (socket says 172.21.15.72:36366) 2024-04-26T21:42:43.450 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:42.509+0000 7f3896a00700 1 -- 172.21.15.72:0/2236524508 learned_addr learned my addr 172.21.15.72:0/2236524508 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:42:43.450 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:42.509+0000 7f3896a00700 1 -- 172.21.15.72:0/2236524508 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38981a3620 con 0x7f389810fc60 2024-04-26T21:42:43.450 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:42.510+0000 7f3896a00700 1 --2- 172.21.15.72:0/2236524508 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f389810fc60 0x7f38981a30e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f38880116a0 tx=0x7f38880046a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:43.450 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:42.510+0000 7f388f400700 1 -- 172.21.15.72:0/2236524508 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3888003b70 con 0x7f389810fc60 2024-04-26T21:42:43.450 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:42.510+0000 7f388f400700 1 -- 172.21.15.72:0/2236524508 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f388801a8a0 con 0x7f389810fc60 2024-04-26T21:42:43.450 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:42.510+0000 7f389d800700 1 -- 172.21.15.72:0/2236524508 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38981a38b0 con 0x7f389810fc60 2024-04-26T21:42:43.450 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:42.510+0000 7f388f400700 1 -- 172.21.15.72:0/2236524508 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3888024890 con 0x7f389810fc60 2024-04-26T21:42:43.451 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:42.510+0000 7f389d800700 1 -- 172.21.15.72:0/2236524508 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38981a65a0 con 0x7f389810fc60 2024-04-26T21:42:43.451 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:42.511+0000 7f388f400700 1 -- 172.21.15.72:0/2236524508 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f3888022020 con 0x7f389810fc60 2024-04-26T21:42:43.451 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:42.511+0000 7f389d800700 1 -- 172.21.15.72:0/2236524508 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3898064a80 con 0x7f389810fc60 2024-04-26T21:42:43.451 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:42.512+0000 7f388f400700 1 --2- 172.21.15.72:0/2236524508 >> [v2:172.21.15.72:6800/4010358672,v1:172.21.15.72:6801/4010358672] conn(0x7f387c039600 0x7f387c03bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:43.451 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:42.512+0000 7f388f400700 1 -- 172.21.15.72:0/2236524508 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f388804dd40 con 0x7f389810fc60 2024-04-26T21:42:43.451 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:42.512+0000 7f3896000700 1 --2- 172.21.15.72:0/2236524508 >> [v2:172.21.15.72:6800/4010358672,v1:172.21.15.72:6801/4010358672] conn(0x7f387c039600 0x7f387c03bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:43.451 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:42.513+0000 7f3896000700 1 --2- 172.21.15.72:0/2236524508 >> [v2:172.21.15.72:6800/4010358672,v1:172.21.15.72:6801/4010358672] conn(0x7f387c039600 0x7f387c03bac0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f3884006fd0 tx=0x7f3884006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:43.451 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:42.517+0000 7f388f400700 1 -- 172.21.15.72:0/2236524508 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f3888054080 con 0x7f389810fc60 2024-04-26T21:42:43.451 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:42.742+0000 7f389d800700 1 -- 172.21.15.72:0/2236524508 --> [v2:172.21.15.72:6800/4010358672,v1:172.21.15.72:6801/4010358672] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7f389807f3d0 con 0x7f387c039600 2024-04-26T21:42:43.451 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:42.743+0000 7f388f400700 1 -- 172.21.15.72:0/2236524508 <== mgr.14120 v2:172.21.15.72:6800/4010358672 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7f389807f3d0 con 0x7f387c039600 2024-04-26T21:42:43.451 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:42.745+0000 7f389d800700 1 -- 172.21.15.72:0/2236524508 >> [v2:172.21.15.72:6800/4010358672,v1:172.21.15.72:6801/4010358672] conn(0x7f387c039600 msgr2=0x7f387c03bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:43.451 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:42.745+0000 7f389d800700 1 --2- 172.21.15.72:0/2236524508 >> [v2:172.21.15.72:6800/4010358672,v1:172.21.15.72:6801/4010358672] conn(0x7f387c039600 0x7f387c03bac0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f3884006fd0 tx=0x7f3884006e40 comp rx=0 tx=0).stop 2024-04-26T21:42:43.451 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:42.745+0000 7f389d800700 1 -- 172.21.15.72:0/2236524508 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f389810fc60 msgr2=0x7f38981a30e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:43.451 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:42.745+0000 7f389d800700 1 --2- 172.21.15.72:0/2236524508 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f389810fc60 0x7f38981a30e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f38880116a0 tx=0x7f38880046a0 comp rx=0 tx=0).stop 2024-04-26T21:42:43.451 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:42.746+0000 7f389d800700 1 -- 172.21.15.72:0/2236524508 shutdown_connections 2024-04-26T21:42:43.451 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:42.746+0000 7f389d800700 1 --2- 172.21.15.72:0/2236524508 >> [v2:172.21.15.72:6800/4010358672,v1:172.21.15.72:6801/4010358672] conn(0x7f387c039600 0x7f387c03bac0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:43.452 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:42.746+0000 7f389d800700 1 --2- 172.21.15.72:0/2236524508 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f389810fc60 0x7f38981a30e0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:43.452 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:42.746+0000 7f389d800700 1 -- 172.21.15.72:0/2236524508 >> 172.21.15.72:0/2236524508 conn(0x7f389807e220 msgr2=0x7f389807ecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:43.452 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:42.746+0000 7f389d800700 1 -- 172.21.15.72:0/2236524508 shutdown_connections 2024-04-26T21:42:43.452 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:42.746+0000 7f389d800700 1 -- 172.21.15.72:0/2236524508 wait complete. 2024-04-26T21:42:43.452 INFO:teuthology.orchestra.run.smithi072.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-04-26T21:42:43.452 INFO:teuthology.orchestra.run.smithi072.stdout:Adding key to root@localhost authorized_keys... 2024-04-26T21:42:43.453 INFO:teuthology.orchestra.run.smithi072.stdout:Adding host smithi072... 2024-04-26T21:42:45.219 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:44 smithi072 ceph-mon[28138]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi072", "addr": "172.21.15.72", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:42:46.049 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:45 smithi072 ceph-mon[28138]: Deploying cephadm binary to smithi072 2024-04-26T21:42:51.020 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout Added host 'smithi072' with addr '172.21.15.72' 2024-04-26T21:42:51.020 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:44.022+0000 7f2efbc00700 1 Processor -- start 2024-04-26T21:42:51.020 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:44.022+0000 7f2efbc00700 1 -- start start 2024-04-26T21:42:51.021 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:44.022+0000 7f2efbc00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2ef410ae80 0x7f2ef410b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:51.021 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:44.022+0000 7f2efbc00700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f2ef410b7e0 con 0x7f2ef410ae80 2024-04-26T21:42:51.021 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:44.023+0000 7f2ef9200700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2ef410ae80 0x7f2ef410b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:51.021 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:44.023+0000 7f2ef9200700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2ef410ae80 0x7f2ef410b2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:36368/0 (socket says 172.21.15.72:36368) 2024-04-26T21:42:51.022 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:44.023+0000 7f2ef9200700 1 -- 172.21.15.72:0/3697702690 learned_addr learned my addr 172.21.15.72:0/3697702690 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:42:51.022 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:44.023+0000 7f2ef9200700 1 -- 172.21.15.72:0/3697702690 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ef410bfe0 con 0x7f2ef410ae80 2024-04-26T21:42:51.022 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:44.024+0000 7f2ef9200700 1 --2- 172.21.15.72:0/3697702690 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2ef410ae80 0x7f2ef410b2a0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f2ee40098a0 tx=0x7f2ee4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f3160f488e53089 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:51.022 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:44.024+0000 7f2ef3e00700 1 -- 172.21.15.72:0/3697702690 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2ee4004160 con 0x7f2ef410ae80 2024-04-26T21:42:51.022 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:44.024+0000 7f2ef3e00700 1 -- 172.21.15.72:0/3697702690 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f2ee4004300 con 0x7f2ef410ae80 2024-04-26T21:42:51.022 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:44.024+0000 7f2ef3e00700 1 -- 172.21.15.72:0/3697702690 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2ee40119e0 con 0x7f2ef410ae80 2024-04-26T21:42:51.025 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:44.025+0000 7f2efbc00700 1 -- 172.21.15.72:0/3697702690 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2ef410ae80 msgr2=0x7f2ef410b2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:51.025 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:44.025+0000 7f2efbc00700 1 --2- 172.21.15.72:0/3697702690 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2ef410ae80 0x7f2ef410b2a0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f2ee40098a0 tx=0x7f2ee4009bb0 comp rx=0 tx=0).stop 2024-04-26T21:42:51.025 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:44.025+0000 7f2efbc00700 1 -- 172.21.15.72:0/3697702690 shutdown_connections 2024-04-26T21:42:51.025 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:44.025+0000 7f2efbc00700 1 --2- 172.21.15.72:0/3697702690 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2ef410ae80 0x7f2ef410b2a0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:51.025 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:44.025+0000 7f2efbc00700 1 -- 172.21.15.72:0/3697702690 >> 172.21.15.72:0/3697702690 conn(0x7f2ef407e270 msgr2=0x7f2ef407e680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:51.025 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:44.025+0000 7f2efbc00700 1 -- 172.21.15.72:0/3697702690 shutdown_connections 2024-04-26T21:42:51.025 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:44.025+0000 7f2efbc00700 1 -- 172.21.15.72:0/3697702690 wait complete. 2024-04-26T21:42:51.026 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:44.026+0000 7f2efbc00700 1 Processor -- start 2024-04-26T21:42:51.026 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:44.026+0000 7f2efbc00700 1 -- start start 2024-04-26T21:42:51.026 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:44.026+0000 7f2efbc00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2ef410ae80 0x7f2ef419ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:51.026 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:44.026+0000 7f2efbc00700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f2ef410f2a0 con 0x7f2ef410ae80 2024-04-26T21:42:51.026 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:44.027+0000 7f2ef9200700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2ef410ae80 0x7f2ef419ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:51.026 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:44.027+0000 7f2ef9200700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2ef410ae80 0x7f2ef419ed10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:36374/0 (socket says 172.21.15.72:36374) 2024-04-26T21:42:51.026 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:44.027+0000 7f2ef9200700 1 -- 172.21.15.72:0/4134462514 learned_addr learned my addr 172.21.15.72:0/4134462514 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:42:51.026 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:44.027+0000 7f2ef9200700 1 -- 172.21.15.72:0/4134462514 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ef419f250 con 0x7f2ef410ae80 2024-04-26T21:42:51.026 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:44.027+0000 7f2ef9200700 1 --2- 172.21.15.72:0/4134462514 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2ef410ae80 0x7f2ef419ed10 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f2ee40116a0 tx=0x7f2ee40046a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:51.026 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:44.028+0000 7f2ef2000700 1 -- 172.21.15.72:0/4134462514 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2ee4003b70 con 0x7f2ef410ae80 2024-04-26T21:42:51.026 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:44.028+0000 7f2ef2000700 1 -- 172.21.15.72:0/4134462514 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f2ee401a8a0 con 0x7f2ef410ae80 2024-04-26T21:42:51.026 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:44.028+0000 7f2ef2000700 1 -- 172.21.15.72:0/4134462514 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2ee4024890 con 0x7f2ef410ae80 2024-04-26T21:42:51.026 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:44.028+0000 7f2efbc00700 1 -- 172.21.15.72:0/4134462514 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ef419f4e0 con 0x7f2ef410ae80 2024-04-26T21:42:51.026 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:44.028+0000 7f2efbc00700 1 -- 172.21.15.72:0/4134462514 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ef41a21c0 con 0x7f2ef410ae80 2024-04-26T21:42:51.027 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:44.029+0000 7f2efbc00700 1 -- 172.21.15.72:0/4134462514 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ef4064a80 con 0x7f2ef410ae80 2024-04-26T21:42:51.027 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:44.029+0000 7f2ef2000700 1 -- 172.21.15.72:0/4134462514 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f2ee4022020 con 0x7f2ef410ae80 2024-04-26T21:42:51.027 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:44.030+0000 7f2ef2000700 1 --2- 172.21.15.72:0/4134462514 >> [v2:172.21.15.72:6800/4010358672,v1:172.21.15.72:6801/4010358672] conn(0x7f2ed80395b0 0x7f2ed803ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:51.027 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:44.030+0000 7f2ef2000700 1 -- 172.21.15.72:0/4134462514 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f2ee404dbe0 con 0x7f2ef410ae80 2024-04-26T21:42:51.027 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:44.030+0000 7f2eebe00700 1 --2- 172.21.15.72:0/4134462514 >> [v2:172.21.15.72:6800/4010358672,v1:172.21.15.72:6801/4010358672] conn(0x7f2ed80395b0 0x7f2ed803ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:51.027 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:44.031+0000 7f2eebe00700 1 --2- 172.21.15.72:0/4134462514 >> [v2:172.21.15.72:6800/4010358672,v1:172.21.15.72:6801/4010358672] conn(0x7f2ed80395b0 0x7f2ed803ba70 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f2edc006fd0 tx=0x7f2edc006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:51.027 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:44.035+0000 7f2ef2000700 1 -- 172.21.15.72:0/4134462514 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f2ee402c3a0 con 0x7f2ef410ae80 2024-04-26T21:42:51.027 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:44.262+0000 7f2efbc00700 1 -- 172.21.15.72:0/4134462514 --> [v2:172.21.15.72:6800/4010358672,v1:172.21.15.72:6801/4010358672] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi072", "addr": "172.21.15.72", "target": ["mon-mgr", ""]}) -- 0x7f2ef4108ad0 con 0x7f2ed80395b0 2024-04-26T21:42:51.028 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:50.264+0000 7f2ef2000700 1 -- 172.21.15.72:0/4134462514 <== mgr.14120 v2:172.21.15.72:6800/4010358672 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7f2ef4108ad0 con 0x7f2ed80395b0 2024-04-26T21:42:51.029 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:50.269+0000 7f2efbc00700 1 -- 172.21.15.72:0/4134462514 >> [v2:172.21.15.72:6800/4010358672,v1:172.21.15.72:6801/4010358672] conn(0x7f2ed80395b0 msgr2=0x7f2ed803ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:51.029 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:50.269+0000 7f2efbc00700 1 --2- 172.21.15.72:0/4134462514 >> [v2:172.21.15.72:6800/4010358672,v1:172.21.15.72:6801/4010358672] conn(0x7f2ed80395b0 0x7f2ed803ba70 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f2edc006fd0 tx=0x7f2edc006e40 comp rx=0 tx=0).stop 2024-04-26T21:42:51.029 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:50.269+0000 7f2efbc00700 1 -- 172.21.15.72:0/4134462514 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2ef410ae80 msgr2=0x7f2ef419ed10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:51.029 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:50.269+0000 7f2efbc00700 1 --2- 172.21.15.72:0/4134462514 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2ef410ae80 0x7f2ef419ed10 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f2ee40116a0 tx=0x7f2ee40046a0 comp rx=0 tx=0).stop 2024-04-26T21:42:51.029 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:50.269+0000 7f2efbc00700 1 -- 172.21.15.72:0/4134462514 shutdown_connections 2024-04-26T21:42:51.029 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:50.269+0000 7f2efbc00700 1 --2- 172.21.15.72:0/4134462514 >> [v2:172.21.15.72:6800/4010358672,v1:172.21.15.72:6801/4010358672] conn(0x7f2ed80395b0 0x7f2ed803ba70 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:51.029 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:50.269+0000 7f2efbc00700 1 --2- 172.21.15.72:0/4134462514 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2ef410ae80 0x7f2ef419ed10 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:51.029 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:50.269+0000 7f2efbc00700 1 -- 172.21.15.72:0/4134462514 >> 172.21.15.72:0/4134462514 conn(0x7f2ef407e270 msgr2=0x7f2ef41083c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:51.029 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:50.269+0000 7f2efbc00700 1 -- 172.21.15.72:0/4134462514 shutdown_connections 2024-04-26T21:42:51.029 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:50.269+0000 7f2efbc00700 1 -- 172.21.15.72:0/4134462514 wait complete. 2024-04-26T21:42:51.029 INFO:teuthology.orchestra.run.smithi072.stdout:Deploying unmanaged mon service... 2024-04-26T21:42:51.525 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:51 smithi072 ceph-mon[28138]: from='mgr.14120 172.21.15.72:0/474756945' entity='mgr.x' 2024-04-26T21:42:51.525 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:51 smithi072 ceph-mon[28138]: Added host smithi072 2024-04-26T21:42:51.525 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:51 smithi072 ceph-mon[28138]: from='mgr.14120 172.21.15.72:0/474756945' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:42:53.164 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-04-26T21:42:53.164 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:51.684+0000 7fa5e2400700 1 Processor -- start 2024-04-26T21:42:53.164 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:51.684+0000 7fa5e2400700 1 -- start start 2024-04-26T21:42:53.164 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:51.685+0000 7fa5e2400700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa5dc108c60 0x7fa5dc109080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:53.164 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:51.685+0000 7fa5e2400700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fa5dc109650 con 0x7fa5dc108c60 2024-04-26T21:42:53.165 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:51.685+0000 7fa5db400700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa5dc108c60 0x7fa5dc109080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:53.165 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:51.685+0000 7fa5db400700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa5dc108c60 0x7fa5dc109080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:37244/0 (socket says 172.21.15.72:37244) 2024-04-26T21:42:53.165 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:51.685+0000 7fa5db400700 1 -- 172.21.15.72:0/2139512753 learned_addr learned my addr 172.21.15.72:0/2139512753 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:42:53.165 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:51.685+0000 7fa5db400700 1 -- 172.21.15.72:0/2139512753 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5dc109e60 con 0x7fa5dc108c60 2024-04-26T21:42:53.165 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:51.686+0000 7fa5db400700 1 --2- 172.21.15.72:0/2139512753 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa5dc108c60 0x7fa5dc109080 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fa5cc0098a0 tx=0x7fa5cc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ef0e53bb6f058bc server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:53.165 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:51.686+0000 7fa5da000700 1 -- 172.21.15.72:0/2139512753 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa5cc004160 con 0x7fa5dc108c60 2024-04-26T21:42:53.165 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:51.686+0000 7fa5da000700 1 -- 172.21.15.72:0/2139512753 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fa5cc004300 con 0x7fa5dc108c60 2024-04-26T21:42:53.165 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:51.686+0000 7fa5da000700 1 -- 172.21.15.72:0/2139512753 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa5cc0119e0 con 0x7fa5dc108c60 2024-04-26T21:42:53.165 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:51.687+0000 7fa5e2400700 1 -- 172.21.15.72:0/2139512753 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa5dc108c60 msgr2=0x7fa5dc109080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:53.165 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:51.687+0000 7fa5e2400700 1 --2- 172.21.15.72:0/2139512753 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa5dc108c60 0x7fa5dc109080 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fa5cc0098a0 tx=0x7fa5cc009bb0 comp rx=0 tx=0).stop 2024-04-26T21:42:53.166 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:51.687+0000 7fa5e2400700 1 -- 172.21.15.72:0/2139512753 shutdown_connections 2024-04-26T21:42:53.166 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:51.687+0000 7fa5e2400700 1 --2- 172.21.15.72:0/2139512753 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa5dc108c60 0x7fa5dc109080 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:53.166 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:51.687+0000 7fa5e2400700 1 -- 172.21.15.72:0/2139512753 >> 172.21.15.72:0/2139512753 conn(0x7fa5dc104240 msgr2=0x7fa5dc106660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:53.166 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:51.687+0000 7fa5e2400700 1 -- 172.21.15.72:0/2139512753 shutdown_connections 2024-04-26T21:42:53.166 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:51.687+0000 7fa5e2400700 1 -- 172.21.15.72:0/2139512753 wait complete. 2024-04-26T21:42:53.166 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:51.688+0000 7fa5e2400700 1 Processor -- start 2024-04-26T21:42:53.166 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:51.688+0000 7fa5e2400700 1 -- start start 2024-04-26T21:42:53.166 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:51.688+0000 7fa5e2400700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa5dc108c60 0x7fa5dc19ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:53.166 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:51.688+0000 7fa5e2400700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fa5dc10d090 con 0x7fa5dc108c60 2024-04-26T21:42:53.166 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:51.689+0000 7fa5db400700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa5dc108c60 0x7fa5dc19ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:53.166 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:51.689+0000 7fa5db400700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa5dc108c60 0x7fa5dc19ecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:37260/0 (socket says 172.21.15.72:37260) 2024-04-26T21:42:53.166 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:51.689+0000 7fa5db400700 1 -- 172.21.15.72:0/551461813 learned_addr learned my addr 172.21.15.72:0/551461813 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:42:53.166 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:51.689+0000 7fa5db400700 1 -- 172.21.15.72:0/551461813 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5dc19f200 con 0x7fa5dc108c60 2024-04-26T21:42:53.167 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:51.689+0000 7fa5db400700 1 --2- 172.21.15.72:0/551461813 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa5dc108c60 0x7fa5dc19ecc0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fa5cc0119b0 tx=0x7fa5cc011a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:53.167 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:51.689+0000 7fa5d3e00700 1 -- 172.21.15.72:0/551461813 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa5cc003af0 con 0x7fa5dc108c60 2024-04-26T21:42:53.167 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:51.689+0000 7fa5d3e00700 1 -- 172.21.15.72:0/551461813 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fa5cc01a8a0 con 0x7fa5dc108c60 2024-04-26T21:42:53.167 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:51.690+0000 7fa5e2400700 1 -- 172.21.15.72:0/551461813 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5dc19f490 con 0x7fa5dc108c60 2024-04-26T21:42:53.167 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:51.690+0000 7fa5d3e00700 1 -- 172.21.15.72:0/551461813 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa5cc024830 con 0x7fa5dc108c60 2024-04-26T21:42:53.167 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:51.690+0000 7fa5e2400700 1 -- 172.21.15.72:0/551461813 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5dc1a2180 con 0x7fa5dc108c60 2024-04-26T21:42:53.167 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:51.691+0000 7fa5e2400700 1 -- 172.21.15.72:0/551461813 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5dc064a80 con 0x7fa5dc108c60 2024-04-26T21:42:53.167 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:51.691+0000 7fa5d3e00700 1 -- 172.21.15.72:0/551461813 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7fa5cc022020 con 0x7fa5dc108c60 2024-04-26T21:42:53.167 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:51.691+0000 7fa5d3e00700 1 --2- 172.21.15.72:0/551461813 >> [v2:172.21.15.72:6800/4010358672,v1:172.21.15.72:6801/4010358672] conn(0x7fa5c00395b0 0x7fa5c003ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:53.167 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:51.691+0000 7fa5d3e00700 1 -- 172.21.15.72:0/551461813 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fa5cc04da30 con 0x7fa5dc108c60 2024-04-26T21:42:53.167 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:51.692+0000 7fa5daa00700 1 --2- 172.21.15.72:0/551461813 >> [v2:172.21.15.72:6800/4010358672,v1:172.21.15.72:6801/4010358672] conn(0x7fa5c00395b0 0x7fa5c003ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:53.167 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:51.692+0000 7fa5daa00700 1 --2- 172.21.15.72:0/551461813 >> [v2:172.21.15.72:6800/4010358672,v1:172.21.15.72:6801/4010358672] conn(0x7fa5c00395b0 0x7fa5c003ba70 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fa5c8006fd0 tx=0x7fa5c8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:53.167 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:51.697+0000 7fa5d3e00700 1 -- 172.21.15.72:0/551461813 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fa5cc02c3a0 con 0x7fa5dc108c60 2024-04-26T21:42:53.167 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:51.960+0000 7fa5e2400700 1 -- 172.21.15.72:0/551461813 --> [v2:172.21.15.72:6800/4010358672,v1:172.21.15.72:6801/4010358672] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7fa5dc105370 con 0x7fa5c00395b0 2024-04-26T21:42:53.168 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:51.968+0000 7fa5d3e00700 1 -- 172.21.15.72:0/551461813 <== mgr.14120 v2:172.21.15.72:6800/4010358672 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fa5dc105370 con 0x7fa5c00395b0 2024-04-26T21:42:53.168 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:51.972+0000 7fa5e2400700 1 -- 172.21.15.72:0/551461813 >> [v2:172.21.15.72:6800/4010358672,v1:172.21.15.72:6801/4010358672] conn(0x7fa5c00395b0 msgr2=0x7fa5c003ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:53.168 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:51.972+0000 7fa5e2400700 1 --2- 172.21.15.72:0/551461813 >> [v2:172.21.15.72:6800/4010358672,v1:172.21.15.72:6801/4010358672] conn(0x7fa5c00395b0 0x7fa5c003ba70 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fa5c8006fd0 tx=0x7fa5c8006e40 comp rx=0 tx=0).stop 2024-04-26T21:42:53.168 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:51.972+0000 7fa5e2400700 1 -- 172.21.15.72:0/551461813 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa5dc108c60 msgr2=0x7fa5dc19ecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:53.168 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:51.972+0000 7fa5e2400700 1 --2- 172.21.15.72:0/551461813 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa5dc108c60 0x7fa5dc19ecc0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fa5cc0119b0 tx=0x7fa5cc011a90 comp rx=0 tx=0).stop 2024-04-26T21:42:53.168 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:51.972+0000 7fa5e2400700 1 -- 172.21.15.72:0/551461813 shutdown_connections 2024-04-26T21:42:53.168 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:51.972+0000 7fa5e2400700 1 --2- 172.21.15.72:0/551461813 >> [v2:172.21.15.72:6800/4010358672,v1:172.21.15.72:6801/4010358672] conn(0x7fa5c00395b0 0x7fa5c003ba70 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:53.168 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:51.972+0000 7fa5e2400700 1 --2- 172.21.15.72:0/551461813 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa5dc108c60 0x7fa5dc19ecc0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:53.168 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:51.972+0000 7fa5e2400700 1 -- 172.21.15.72:0/551461813 >> 172.21.15.72:0/551461813 conn(0x7fa5dc104240 msgr2=0x7fa5dc104c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:53.168 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:51.972+0000 7fa5e2400700 1 -- 172.21.15.72:0/551461813 shutdown_connections 2024-04-26T21:42:53.168 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:51.973+0000 7fa5e2400700 1 -- 172.21.15.72:0/551461813 wait complete. 2024-04-26T21:42:53.168 INFO:teuthology.orchestra.run.smithi072.stdout:Deploying unmanaged mgr service... 2024-04-26T21:42:53.314 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:52 smithi072 ceph-mon[28138]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:42:53.314 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:52 smithi072 ceph-mon[28138]: Saving service mon spec with placement count:5 2024-04-26T21:42:53.315 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:52 smithi072 ceph-mon[28138]: from='mgr.14120 172.21.15.72:0/474756945' entity='mgr.x' 2024-04-26T21:42:55.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:54 smithi072 ceph-mon[28138]: from='mgr.14120 172.21.15.72:0/474756945' entity='mgr.x' 2024-04-26T21:42:55.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:54 smithi072 ceph-mon[28138]: from='mgr.14120 172.21.15.72:0/474756945' entity='mgr.x' 2024-04-26T21:42:55.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:54 smithi072 ceph-mon[28138]: from='mgr.14120 172.21.15.72:0/474756945' entity='mgr.x' 2024-04-26T21:42:55.129 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-04-26T21:42:55.130 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:54.198+0000 7f10aaa00700 1 Processor -- start 2024-04-26T21:42:55.130 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:54.198+0000 7f10aaa00700 1 -- start start 2024-04-26T21:42:55.130 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:54.198+0000 7f10aaa00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f10a4108c80 0x7f10a41090a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:55.130 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:54.198+0000 7f10aaa00700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f10a4109670 con 0x7f10a4108c80 2024-04-26T21:42:55.130 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:54.199+0000 7f10a3e00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f10a4108c80 0x7f10a41090a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:55.130 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:54.199+0000 7f10a3e00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f10a4108c80 0x7f10a41090a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:37264/0 (socket says 172.21.15.72:37264) 2024-04-26T21:42:55.130 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:54.199+0000 7f10a3e00700 1 -- 172.21.15.72:0/3425177926 learned_addr learned my addr 172.21.15.72:0/3425177926 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:42:55.135 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:54.199+0000 7f10a3e00700 1 -- 172.21.15.72:0/3425177926 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10a4109e80 con 0x7f10a4108c80 2024-04-26T21:42:55.135 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:54.200+0000 7f10a3e00700 1 --2- 172.21.15.72:0/3425177926 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f10a4108c80 0x7f10a41090a0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f10940098a0 tx=0x7f1094009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67e08aadede0d318 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:55.135 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:54.200+0000 7f10a2a00700 1 -- 172.21.15.72:0/3425177926 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1094004160 con 0x7f10a4108c80 2024-04-26T21:42:55.135 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:54.200+0000 7f10a2a00700 1 -- 172.21.15.72:0/3425177926 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1094004300 con 0x7f10a4108c80 2024-04-26T21:42:55.135 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:54.200+0000 7f10a2a00700 1 -- 172.21.15.72:0/3425177926 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1094011960 con 0x7f10a4108c80 2024-04-26T21:42:55.135 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:54.201+0000 7f10aaa00700 1 -- 172.21.15.72:0/3425177926 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f10a4108c80 msgr2=0x7f10a41090a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:55.136 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:54.201+0000 7f10aaa00700 1 --2- 172.21.15.72:0/3425177926 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f10a4108c80 0x7f10a41090a0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f10940098a0 tx=0x7f1094009bb0 comp rx=0 tx=0).stop 2024-04-26T21:42:55.136 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:54.201+0000 7f10aaa00700 1 -- 172.21.15.72:0/3425177926 shutdown_connections 2024-04-26T21:42:55.136 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:54.201+0000 7f10aaa00700 1 --2- 172.21.15.72:0/3425177926 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f10a4108c80 0x7f10a41090a0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:55.136 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:54.201+0000 7f10aaa00700 1 -- 172.21.15.72:0/3425177926 >> 172.21.15.72:0/3425177926 conn(0x7f10a4104240 msgr2=0x7f10a4106660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:55.136 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:54.201+0000 7f10aaa00700 1 -- 172.21.15.72:0/3425177926 shutdown_connections 2024-04-26T21:42:55.136 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:54.201+0000 7f10aaa00700 1 -- 172.21.15.72:0/3425177926 wait complete. 2024-04-26T21:42:55.136 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:54.202+0000 7f10aaa00700 1 Processor -- start 2024-04-26T21:42:55.136 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:54.202+0000 7f10aaa00700 1 -- start start 2024-04-26T21:42:55.136 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:54.203+0000 7f10aaa00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f10a4108c80 0x7f10a419ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:55.136 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:54.203+0000 7f10aaa00700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f10a410d0b0 con 0x7f10a4108c80 2024-04-26T21:42:55.136 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:54.203+0000 7f10a3e00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f10a4108c80 0x7f10a419ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:55.136 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:54.203+0000 7f10a3e00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f10a4108c80 0x7f10a419ed50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:37266/0 (socket says 172.21.15.72:37266) 2024-04-26T21:42:55.136 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:54.203+0000 7f10a3e00700 1 -- 172.21.15.72:0/2182843784 learned_addr learned my addr 172.21.15.72:0/2182843784 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:42:55.137 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:54.203+0000 7f10a3e00700 1 -- 172.21.15.72:0/2182843784 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10a419f290 con 0x7f10a4108c80 2024-04-26T21:42:55.137 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:54.203+0000 7f10a3e00700 1 --2- 172.21.15.72:0/2182843784 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f10a4108c80 0x7f10a419ed50 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f1094011670 tx=0x7f1094011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:55.137 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:54.204+0000 7f10a0c00700 1 -- 172.21.15.72:0/2182843784 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1094003a60 con 0x7f10a4108c80 2024-04-26T21:42:55.137 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:54.204+0000 7f10a0c00700 1 -- 172.21.15.72:0/2182843784 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f109401b8d0 con 0x7f10a4108c80 2024-04-26T21:42:55.137 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:54.204+0000 7f10aaa00700 1 -- 172.21.15.72:0/2182843784 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10a419f520 con 0x7f10a4108c80 2024-04-26T21:42:55.137 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:54.204+0000 7f10a0c00700 1 -- 172.21.15.72:0/2182843784 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1094023790 con 0x7f10a4108c80 2024-04-26T21:42:55.137 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:54.204+0000 7f10aaa00700 1 -- 172.21.15.72:0/2182843784 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10a41a2210 con 0x7f10a4108c80 2024-04-26T21:42:55.137 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:54.205+0000 7f10aaa00700 1 -- 172.21.15.72:0/2182843784 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10a410d230 con 0x7f10a4108c80 2024-04-26T21:42:55.137 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:54.205+0000 7f10a0c00700 1 -- 172.21.15.72:0/2182843784 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f109401b440 con 0x7f10a4108c80 2024-04-26T21:42:55.137 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:54.206+0000 7f10a0c00700 1 --2- 172.21.15.72:0/2182843784 >> [v2:172.21.15.72:6800/4010358672,v1:172.21.15.72:6801/4010358672] conn(0x7f10880395b0 0x7f108803ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:55.137 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:54.206+0000 7f10a0c00700 1 -- 172.21.15.72:0/2182843784 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f109404e050 con 0x7f10a4108c80 2024-04-26T21:42:55.137 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:54.206+0000 7f10a3400700 1 --2- 172.21.15.72:0/2182843784 >> [v2:172.21.15.72:6800/4010358672,v1:172.21.15.72:6801/4010358672] conn(0x7f10880395b0 0x7f108803ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:55.137 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:54.207+0000 7f10a3400700 1 --2- 172.21.15.72:0/2182843784 >> [v2:172.21.15.72:6800/4010358672,v1:172.21.15.72:6801/4010358672] conn(0x7f10880395b0 0x7f108803ba70 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f1090006fd0 tx=0x7f1090006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:55.137 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:54.217+0000 7f10a0c00700 1 -- 172.21.15.72:0/2182843784 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f109401f030 con 0x7f10a4108c80 2024-04-26T21:42:55.138 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:54.456+0000 7f10aaa00700 1 -- 172.21.15.72:0/2182843784 --> [v2:172.21.15.72:6800/4010358672,v1:172.21.15.72:6801/4010358672] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f10a4105340 con 0x7f10880395b0 2024-04-26T21:42:55.138 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:54.466+0000 7f10a0c00700 1 -- 172.21.15.72:0/2182843784 <== mgr.14120 v2:172.21.15.72:6800/4010358672 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f10a4105340 con 0x7f10880395b0 2024-04-26T21:42:55.138 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:54.469+0000 7f10aaa00700 1 -- 172.21.15.72:0/2182843784 >> [v2:172.21.15.72:6800/4010358672,v1:172.21.15.72:6801/4010358672] conn(0x7f10880395b0 msgr2=0x7f108803ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:55.138 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:54.469+0000 7f10aaa00700 1 --2- 172.21.15.72:0/2182843784 >> [v2:172.21.15.72:6800/4010358672,v1:172.21.15.72:6801/4010358672] conn(0x7f10880395b0 0x7f108803ba70 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f1090006fd0 tx=0x7f1090006e40 comp rx=0 tx=0).stop 2024-04-26T21:42:55.138 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:54.469+0000 7f10aaa00700 1 -- 172.21.15.72:0/2182843784 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f10a4108c80 msgr2=0x7f10a419ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:55.138 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:54.469+0000 7f10aaa00700 1 --2- 172.21.15.72:0/2182843784 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f10a4108c80 0x7f10a419ed50 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f1094011670 tx=0x7f1094011a30 comp rx=0 tx=0).stop 2024-04-26T21:42:55.138 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:54.469+0000 7f10aaa00700 1 -- 172.21.15.72:0/2182843784 shutdown_connections 2024-04-26T21:42:55.138 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:54.469+0000 7f10aaa00700 1 --2- 172.21.15.72:0/2182843784 >> [v2:172.21.15.72:6800/4010358672,v1:172.21.15.72:6801/4010358672] conn(0x7f10880395b0 0x7f108803ba70 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:55.138 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:54.469+0000 7f10aaa00700 1 --2- 172.21.15.72:0/2182843784 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f10a4108c80 0x7f10a419ed50 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:55.138 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:54.469+0000 7f10aaa00700 1 -- 172.21.15.72:0/2182843784 >> 172.21.15.72:0/2182843784 conn(0x7f10a4104240 msgr2=0x7f10a4104c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:55.138 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:54.469+0000 7f10aaa00700 1 -- 172.21.15.72:0/2182843784 shutdown_connections 2024-04-26T21:42:55.138 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:54.470+0000 7f10aaa00700 1 -- 172.21.15.72:0/2182843784 wait complete. 2024-04-26T21:42:55.979 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:55 smithi072 ceph-mon[28138]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:42:55.979 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:55 smithi072 ceph-mon[28138]: Saving service mgr spec with placement count:2 2024-04-26T21:42:55.980 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:55 smithi072 ceph-mon[28138]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:42:56.894 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:55.730+0000 7f3f34a00700 1 Processor -- start 2024-04-26T21:42:56.894 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:55.730+0000 7f3f34a00700 1 -- start start 2024-04-26T21:42:56.894 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:55.730+0000 7f3f34a00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3f3007dd40 0x7f3f3007c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:56.894 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:55.730+0000 7f3f34a00700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f3f3007c770 con 0x7f3f3007dd40 2024-04-26T21:42:56.894 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:55.731+0000 7f3f2de00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3f3007dd40 0x7f3f3007c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:56.894 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:55.731+0000 7f3f2de00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3f3007dd40 0x7f3f3007c1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:37278/0 (socket says 172.21.15.72:37278) 2024-04-26T21:42:56.895 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:55.731+0000 7f3f2de00700 1 -- 172.21.15.72:0/1860766064 learned_addr learned my addr 172.21.15.72:0/1860766064 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:42:56.895 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:55.731+0000 7f3f2de00700 1 -- 172.21.15.72:0/1860766064 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f3007c8f0 con 0x7f3f3007dd40 2024-04-26T21:42:56.895 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:55.731+0000 7f3f2de00700 1 --2- 172.21.15.72:0/1860766064 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3f3007dd40 0x7f3f3007c1a0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f3f200098a0 tx=0x7f3f20009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d19ebb0ed1a7623 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:56.895 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:55.732+0000 7f3f2ca00700 1 -- 172.21.15.72:0/1860766064 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3f20004160 con 0x7f3f3007dd40 2024-04-26T21:42:56.895 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:55.732+0000 7f3f2ca00700 1 -- 172.21.15.72:0/1860766064 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3f20004300 con 0x7f3f3007dd40 2024-04-26T21:42:56.895 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:55.732+0000 7f3f2ca00700 1 -- 172.21.15.72:0/1860766064 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3f20011960 con 0x7f3f3007dd40 2024-04-26T21:42:56.895 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:55.732+0000 7f3f34a00700 1 -- 172.21.15.72:0/1860766064 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3f3007dd40 msgr2=0x7f3f3007c1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:56.895 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:55.732+0000 7f3f34a00700 1 --2- 172.21.15.72:0/1860766064 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3f3007dd40 0x7f3f3007c1a0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f3f200098a0 tx=0x7f3f20009bb0 comp rx=0 tx=0).stop 2024-04-26T21:42:56.895 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:55.733+0000 7f3f34a00700 1 -- 172.21.15.72:0/1860766064 shutdown_connections 2024-04-26T21:42:56.895 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:55.733+0000 7f3f34a00700 1 --2- 172.21.15.72:0/1860766064 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3f3007dd40 0x7f3f3007c1a0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:56.895 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:55.733+0000 7f3f34a00700 1 -- 172.21.15.72:0/1860766064 >> 172.21.15.72:0/1860766064 conn(0x7f3f301041c0 msgr2=0x7f3f30106640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:56.895 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:55.733+0000 7f3f34a00700 1 -- 172.21.15.72:0/1860766064 shutdown_connections 2024-04-26T21:42:56.895 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:55.733+0000 7f3f34a00700 1 -- 172.21.15.72:0/1860766064 wait complete. 2024-04-26T21:42:56.895 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:55.734+0000 7f3f34a00700 1 Processor -- start 2024-04-26T21:42:56.895 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:55.734+0000 7f3f34a00700 1 -- start start 2024-04-26T21:42:56.895 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:55.734+0000 7f3f34a00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3f3007dd40 0x7f3f3019ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:56.896 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:55.734+0000 7f3f34a00700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f3f3010f1e0 con 0x7f3f3007dd40 2024-04-26T21:42:56.896 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:55.734+0000 7f3f2de00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3f3007dd40 0x7f3f3019ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:56.896 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:55.734+0000 7f3f2de00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3f3007dd40 0x7f3f3019ed10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:37284/0 (socket says 172.21.15.72:37284) 2024-04-26T21:42:56.896 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:55.734+0000 7f3f2de00700 1 -- 172.21.15.72:0/2113604924 learned_addr learned my addr 172.21.15.72:0/2113604924 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:42:56.896 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:55.735+0000 7f3f2de00700 1 -- 172.21.15.72:0/2113604924 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f3019f250 con 0x7f3f3007dd40 2024-04-26T21:42:56.896 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:55.735+0000 7f3f2de00700 1 --2- 172.21.15.72:0/2113604924 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3f3007dd40 0x7f3f3019ed10 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f3f20011670 tx=0x7f3f20011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:56.896 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:55.735+0000 7f3f26a00700 1 -- 172.21.15.72:0/2113604924 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3f20003a60 con 0x7f3f3007dd40 2024-04-26T21:42:56.896 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:55.735+0000 7f3f26a00700 1 -- 172.21.15.72:0/2113604924 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3f2001b8d0 con 0x7f3f3007dd40 2024-04-26T21:42:56.896 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:55.735+0000 7f3f26a00700 1 -- 172.21.15.72:0/2113604924 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3f20023790 con 0x7f3f3007dd40 2024-04-26T21:42:56.896 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:55.735+0000 7f3f34a00700 1 -- 172.21.15.72:0/2113604924 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f3019f4e0 con 0x7f3f3007dd40 2024-04-26T21:42:56.896 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:55.736+0000 7f3f34a00700 1 -- 172.21.15.72:0/2113604924 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f301a21c0 con 0x7f3f3007dd40 2024-04-26T21:42:56.896 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:55.737+0000 7f3f26a00700 1 -- 172.21.15.72:0/2113604924 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f3f2001b440 con 0x7f3f3007dd40 2024-04-26T21:42:56.896 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:55.737+0000 7f3f1be00700 1 -- 172.21.15.72:0/2113604924 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f30064a80 con 0x7f3f3007dd40 2024-04-26T21:42:56.896 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:55.737+0000 7f3f26a00700 1 --2- 172.21.15.72:0/2113604924 >> [v2:172.21.15.72:6800/4010358672,v1:172.21.15.72:6801/4010358672] conn(0x7f3f14039600 0x7f3f1403bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:56.896 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:55.737+0000 7f3f26a00700 1 -- 172.21.15.72:0/2113604924 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f3f2004dfe0 con 0x7f3f3007dd40 2024-04-26T21:42:56.897 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:55.737+0000 7f3f2d400700 1 --2- 172.21.15.72:0/2113604924 >> [v2:172.21.15.72:6800/4010358672,v1:172.21.15.72:6801/4010358672] conn(0x7f3f14039600 0x7f3f1403bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:56.897 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:55.738+0000 7f3f2d400700 1 --2- 172.21.15.72:0/2113604924 >> [v2:172.21.15.72:6800/4010358672,v1:172.21.15.72:6801/4010358672] conn(0x7f3f14039600 0x7f3f1403bac0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f3f1c006fd0 tx=0x7f3f1c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:56.897 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:55.742+0000 7f3f26a00700 1 -- 172.21.15.72:0/2113604924 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f3f2001f030 con 0x7f3f3007dd40 2024-04-26T21:42:56.897 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:55.977+0000 7f3f1be00700 1 -- 172.21.15.72:0/2113604924 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7f3f30075e40 con 0x7f3f3007dd40 2024-04-26T21:42:56.897 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:55.983+0000 7f3f26a00700 1 -- 172.21.15.72:0/2113604924 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7f3f200292a0 con 0x7f3f3007dd40 2024-04-26T21:42:56.897 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:55.985+0000 7f3f1be00700 1 -- 172.21.15.72:0/2113604924 >> [v2:172.21.15.72:6800/4010358672,v1:172.21.15.72:6801/4010358672] conn(0x7f3f14039600 msgr2=0x7f3f1403bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:56.897 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:55.985+0000 7f3f1be00700 1 --2- 172.21.15.72:0/2113604924 >> [v2:172.21.15.72:6800/4010358672,v1:172.21.15.72:6801/4010358672] conn(0x7f3f14039600 0x7f3f1403bac0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f3f1c006fd0 tx=0x7f3f1c006e40 comp rx=0 tx=0).stop 2024-04-26T21:42:56.897 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:55.985+0000 7f3f1be00700 1 -- 172.21.15.72:0/2113604924 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3f3007dd40 msgr2=0x7f3f3019ed10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:56.897 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:55.985+0000 7f3f1be00700 1 --2- 172.21.15.72:0/2113604924 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3f3007dd40 0x7f3f3019ed10 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f3f20011670 tx=0x7f3f20011a30 comp rx=0 tx=0).stop 2024-04-26T21:42:56.897 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:55.985+0000 7f3f1be00700 1 -- 172.21.15.72:0/2113604924 shutdown_connections 2024-04-26T21:42:56.897 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:55.985+0000 7f3f1be00700 1 --2- 172.21.15.72:0/2113604924 >> [v2:172.21.15.72:6800/4010358672,v1:172.21.15.72:6801/4010358672] conn(0x7f3f14039600 0x7f3f1403bac0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:56.897 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:55.985+0000 7f3f1be00700 1 --2- 172.21.15.72:0/2113604924 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3f3007dd40 0x7f3f3019ed10 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:56.897 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:55.985+0000 7f3f1be00700 1 -- 172.21.15.72:0/2113604924 >> 172.21.15.72:0/2113604924 conn(0x7f3f301041c0 msgr2=0x7f3f30104cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:56.897 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:55.987+0000 7f3f1be00700 1 -- 172.21.15.72:0/2113604924 shutdown_connections 2024-04-26T21:42:56.897 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:55.987+0000 7f3f1be00700 1 -- 172.21.15.72:0/2113604924 wait complete. 2024-04-26T21:42:57.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:56 smithi072 ceph-mon[28138]: from='client.? 172.21.15.72:0/2113604924' entity='client.admin' 2024-04-26T21:42:57.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:56 smithi072 ceph-mon[28138]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:42:58.899 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:57.918+0000 7f3ea9a00700 1 Processor -- start 2024-04-26T21:42:58.899 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:57.918+0000 7f3ea9a00700 1 -- start start 2024-04-26T21:42:58.899 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:57.918+0000 7f3ea9a00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3ea4108c60 0x7f3ea4109080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:58.899 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:57.918+0000 7f3ea9a00700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f3ea4109650 con 0x7f3ea4108c60 2024-04-26T21:42:58.899 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:57.919+0000 7f3ea2a00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3ea4108c60 0x7f3ea4109080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:58.899 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:57.919+0000 7f3ea2a00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3ea4108c60 0x7f3ea4109080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:37296/0 (socket says 172.21.15.72:37296) 2024-04-26T21:42:58.899 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:57.919+0000 7f3ea2a00700 1 -- 172.21.15.72:0/3908065366 learned_addr learned my addr 172.21.15.72:0/3908065366 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:42:58.899 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:57.920+0000 7f3ea2a00700 1 -- 172.21.15.72:0/3908065366 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ea4109e60 con 0x7f3ea4108c60 2024-04-26T21:42:58.899 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:57.920+0000 7f3ea2a00700 1 --2- 172.21.15.72:0/3908065366 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3ea4108c60 0x7f3ea4109080 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f3e940098a0 tx=0x7f3e94009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0c0ed1d111dc800 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:58.899 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:57.920+0000 7f3ea1600700 1 -- 172.21.15.72:0/3908065366 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3e94004160 con 0x7f3ea4108c60 2024-04-26T21:42:58.900 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:57.920+0000 7f3ea1600700 1 -- 172.21.15.72:0/3908065366 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3e94004300 con 0x7f3ea4108c60 2024-04-26T21:42:58.900 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:57.920+0000 7f3ea1600700 1 -- 172.21.15.72:0/3908065366 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3e94011960 con 0x7f3ea4108c60 2024-04-26T21:42:58.900 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:57.921+0000 7f3ea9a00700 1 -- 172.21.15.72:0/3908065366 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3ea4108c60 msgr2=0x7f3ea4109080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:58.900 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:57.921+0000 7f3ea9a00700 1 --2- 172.21.15.72:0/3908065366 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3ea4108c60 0x7f3ea4109080 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f3e940098a0 tx=0x7f3e94009bb0 comp rx=0 tx=0).stop 2024-04-26T21:42:58.900 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:57.921+0000 7f3ea9a00700 1 -- 172.21.15.72:0/3908065366 shutdown_connections 2024-04-26T21:42:58.900 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:57.921+0000 7f3ea9a00700 1 --2- 172.21.15.72:0/3908065366 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3ea4108c60 0x7f3ea4109080 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:58.900 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:57.921+0000 7f3ea9a00700 1 -- 172.21.15.72:0/3908065366 >> 172.21.15.72:0/3908065366 conn(0x7f3ea4104240 msgr2=0x7f3ea4106660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:58.900 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:57.921+0000 7f3ea9a00700 1 -- 172.21.15.72:0/3908065366 shutdown_connections 2024-04-26T21:42:58.900 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:57.921+0000 7f3ea9a00700 1 -- 172.21.15.72:0/3908065366 wait complete. 2024-04-26T21:42:58.900 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:57.923+0000 7f3ea9a00700 1 Processor -- start 2024-04-26T21:42:58.900 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:57.923+0000 7f3ea9a00700 1 -- start start 2024-04-26T21:42:58.900 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:57.923+0000 7f3ea9a00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3ea4108c60 0x7f3ea419cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:58.900 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:57.924+0000 7f3ea9a00700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f3ea410d090 con 0x7f3ea4108c60 2024-04-26T21:42:58.901 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:57.924+0000 7f3ea2a00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3ea4108c60 0x7f3ea419cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:58.901 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:57.924+0000 7f3ea2a00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3ea4108c60 0x7f3ea419cb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:37306/0 (socket says 172.21.15.72:37306) 2024-04-26T21:42:58.901 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:57.924+0000 7f3ea2a00700 1 -- 172.21.15.72:0/3702308861 learned_addr learned my addr 172.21.15.72:0/3702308861 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:42:58.901 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:57.924+0000 7f3ea2a00700 1 -- 172.21.15.72:0/3702308861 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ea419d0b0 con 0x7f3ea4108c60 2024-04-26T21:42:58.901 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:57.925+0000 7f3ea2a00700 1 --2- 172.21.15.72:0/3702308861 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3ea4108c60 0x7f3ea419cb70 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f3e94011670 tx=0x7f3e94011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:58.901 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:57.925+0000 7f3e9b400700 1 -- 172.21.15.72:0/3702308861 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3e9401f070 con 0x7f3ea4108c60 2024-04-26T21:42:58.901 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:57.925+0000 7f3e9b400700 1 -- 172.21.15.72:0/3702308861 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3e94003b20 con 0x7f3ea4108c60 2024-04-26T21:42:58.901 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:57.925+0000 7f3e9b400700 1 -- 172.21.15.72:0/3702308861 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3e94019580 con 0x7f3ea4108c60 2024-04-26T21:42:58.901 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:57.925+0000 7f3ea9a00700 1 -- 172.21.15.72:0/3702308861 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ea419d340 con 0x7f3ea4108c60 2024-04-26T21:42:58.901 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:57.925+0000 7f3ea9a00700 1 -- 172.21.15.72:0/3702308861 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ea41a0030 con 0x7f3ea4108c60 2024-04-26T21:42:58.901 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:57.926+0000 7f3e9b400700 1 -- 172.21.15.72:0/3702308861 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f3e94022070 con 0x7f3ea4108c60 2024-04-26T21:42:58.901 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:57.927+0000 7f3ea9a00700 1 -- 172.21.15.72:0/3702308861 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ea410d210 con 0x7f3ea4108c60 2024-04-26T21:42:58.901 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:57.927+0000 7f3e9b400700 1 --2- 172.21.15.72:0/3702308861 >> [v2:172.21.15.72:6800/4010358672,v1:172.21.15.72:6801/4010358672] conn(0x7f3e88039600 0x7f3e8803bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:58.901 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:57.927+0000 7f3e9b400700 1 -- 172.21.15.72:0/3702308861 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f3e9404db00 con 0x7f3ea4108c60 2024-04-26T21:42:58.902 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:57.927+0000 7f3ea2000700 1 --2- 172.21.15.72:0/3702308861 >> [v2:172.21.15.72:6800/4010358672,v1:172.21.15.72:6801/4010358672] conn(0x7f3e88039600 0x7f3e8803bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:58.902 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:57.928+0000 7f3ea2000700 1 --2- 172.21.15.72:0/3702308861 >> [v2:172.21.15.72:6800/4010358672,v1:172.21.15.72:6801/4010358672] conn(0x7f3e88039600 0x7f3e8803bac0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f3e90006fd0 tx=0x7f3e90006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:58.902 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:57.932+0000 7f3e9b400700 1 -- 172.21.15.72:0/3702308861 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f3e9404fe20 con 0x7f3ea4108c60 2024-04-26T21:42:58.902 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:58.184+0000 7f3ea9a00700 1 -- 172.21.15.72:0/3702308861 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7f3ea4064a80 con 0x7f3ea4108c60 2024-04-26T21:42:58.902 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:58.190+0000 7f3e9b400700 1 -- 172.21.15.72:0/3702308861 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7f3e9404d450 con 0x7f3ea4108c60 2024-04-26T21:42:58.902 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:58.194+0000 7f3ea9a00700 1 -- 172.21.15.72:0/3702308861 >> [v2:172.21.15.72:6800/4010358672,v1:172.21.15.72:6801/4010358672] conn(0x7f3e88039600 msgr2=0x7f3e8803bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:58.902 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:58.194+0000 7f3ea9a00700 1 --2- 172.21.15.72:0/3702308861 >> [v2:172.21.15.72:6800/4010358672,v1:172.21.15.72:6801/4010358672] conn(0x7f3e88039600 0x7f3e8803bac0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f3e90006fd0 tx=0x7f3e90006e40 comp rx=0 tx=0).stop 2024-04-26T21:42:58.902 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:58.194+0000 7f3ea9a00700 1 -- 172.21.15.72:0/3702308861 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3ea4108c60 msgr2=0x7f3ea419cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:58.902 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:58.194+0000 7f3ea9a00700 1 --2- 172.21.15.72:0/3702308861 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3ea4108c60 0x7f3ea419cb70 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f3e94011670 tx=0x7f3e94011a30 comp rx=0 tx=0).stop 2024-04-26T21:42:58.902 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:58.194+0000 7f3ea9a00700 1 -- 172.21.15.72:0/3702308861 shutdown_connections 2024-04-26T21:42:58.902 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:58.194+0000 7f3ea9a00700 1 --2- 172.21.15.72:0/3702308861 >> [v2:172.21.15.72:6800/4010358672,v1:172.21.15.72:6801/4010358672] conn(0x7f3e88039600 0x7f3e8803bac0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:58.902 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:58.194+0000 7f3ea9a00700 1 --2- 172.21.15.72:0/3702308861 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3ea4108c60 0x7f3ea419cb70 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:58.903 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:58.194+0000 7f3ea9a00700 1 -- 172.21.15.72:0/3702308861 >> 172.21.15.72:0/3702308861 conn(0x7f3ea4104240 msgr2=0x7f3ea4104da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:58.903 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:58.194+0000 7f3ea9a00700 1 -- 172.21.15.72:0/3702308861 shutdown_connections 2024-04-26T21:42:58.903 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:58.194+0000 7f3ea9a00700 1 -- 172.21.15.72:0/3702308861 wait complete. 2024-04-26T21:42:58.903 INFO:teuthology.orchestra.run.smithi072.stdout:Enabling the dashboard module... 2024-04-26T21:42:59.046 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:58 smithi072 ceph-mon[28138]: from='mgr.14120 172.21.15.72:0/474756945' entity='mgr.x' 2024-04-26T21:42:59.046 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:58 smithi072 ceph-mon[28138]: from='client.? 172.21.15.72:0/3702308861' entity='client.admin' 2024-04-26T21:43:00.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:59 smithi072 ceph-mon[28138]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:43:00.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:42:59 smithi072 ceph-mon[28138]: from='mgr.14120 172.21.15.72:0/474756945' entity='mgr.x' 2024-04-26T21:43:00.814 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:43:00 smithi072 ceph-mon[28138]: from='client.? 172.21.15.72:0/678682754' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-04-26T21:43:01.072 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 26 21:43:00 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-mgr-x[28363]: ignoring --setuser ceph since I am not root 2024-04-26T21:43:01.072 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 26 21:43:00 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-mgr-x[28363]: ignoring --setgroup ceph since I am not root 2024-04-26T21:43:01.072 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 26 21:43:01 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-mgr-x[28363]: 2024-04-26T21:43:01.020+0000 7fb1b57d4200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-26T21:43:01.543 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 26 21:43:01 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-mgr-x[28363]: 2024-04-26T21:43:01.264+0000 7fb1b57d4200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-26T21:43:01.543 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 26 21:43:01 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-mgr-x[28363]: 2024-04-26T21:43:01.394+0000 7fb1b57d4200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-26T21:43:01.822 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 26 21:43:01 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-mgr-x[28363]: 2024-04-26T21:43:01.791+0000 7fb1b57d4200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-26T21:43:01.822 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:43:01 smithi072 ceph-mon[28138]: from='client.? 172.21.15.72:0/678682754' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-04-26T21:43:01.822 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:43:01 smithi072 ceph-mon[28138]: mgrmap e9: x(active, since 26s) 2024-04-26T21:43:02.301 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 26 21:43:01 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-mgr-x[28363]: 2024-04-26T21:43:01.908+0000 7fb1b57d4200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-26T21:43:02.315 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:59.581+0000 7f293ac00700 1 Processor -- start 2024-04-26T21:43:02.315 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:59.581+0000 7f293ac00700 1 -- start start 2024-04-26T21:43:02.315 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:59.581+0000 7f293ac00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2934108cd0 0x7f29341090f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:02.315 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:59.581+0000 7f293ac00700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f29341096c0 con 0x7f2934108cd0 2024-04-26T21:43:02.315 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:59.582+0000 7f2933e00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2934108cd0 0x7f29341090f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:02.315 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:59.582+0000 7f2933e00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2934108cd0 0x7f29341090f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:37322/0 (socket says 172.21.15.72:37322) 2024-04-26T21:43:02.315 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:59.582+0000 7f2933e00700 1 -- 172.21.15.72:0/3468423086 learned_addr learned my addr 172.21.15.72:0/3468423086 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:43:02.315 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:59.583+0000 7f2933e00700 1 -- 172.21.15.72:0/3468423086 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2934109ed0 con 0x7f2934108cd0 2024-04-26T21:43:02.315 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:59.583+0000 7f2933e00700 1 --2- 172.21.15.72:0/3468423086 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2934108cd0 0x7f29341090f0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f29240098a0 tx=0x7f2924009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=145a0e8ff78014f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:02.316 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:59.583+0000 7f2932a00700 1 -- 172.21.15.72:0/3468423086 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2924004160 con 0x7f2934108cd0 2024-04-26T21:43:02.316 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:59.583+0000 7f2932a00700 1 -- 172.21.15.72:0/3468423086 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2924004300 con 0x7f2934108cd0 2024-04-26T21:43:02.316 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:59.583+0000 7f2932a00700 1 -- 172.21.15.72:0/3468423086 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2924011960 con 0x7f2934108cd0 2024-04-26T21:43:02.316 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:59.584+0000 7f293ac00700 1 -- 172.21.15.72:0/3468423086 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2934108cd0 msgr2=0x7f29341090f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:02.316 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:59.584+0000 7f293ac00700 1 --2- 172.21.15.72:0/3468423086 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2934108cd0 0x7f29341090f0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f29240098a0 tx=0x7f2924009bb0 comp rx=0 tx=0).stop 2024-04-26T21:43:02.316 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:59.584+0000 7f293ac00700 1 -- 172.21.15.72:0/3468423086 shutdown_connections 2024-04-26T21:43:02.316 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:59.584+0000 7f293ac00700 1 --2- 172.21.15.72:0/3468423086 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2934108cd0 0x7f29341090f0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:02.316 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:59.584+0000 7f293ac00700 1 -- 172.21.15.72:0/3468423086 >> 172.21.15.72:0/3468423086 conn(0x7f2934104230 msgr2=0x7f29341066b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:02.316 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:59.585+0000 7f293ac00700 1 -- 172.21.15.72:0/3468423086 shutdown_connections 2024-04-26T21:43:02.316 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:59.585+0000 7f293ac00700 1 -- 172.21.15.72:0/3468423086 wait complete. 2024-04-26T21:43:02.316 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:59.586+0000 7f293ac00700 1 Processor -- start 2024-04-26T21:43:02.316 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:59.587+0000 7f293ac00700 1 -- start start 2024-04-26T21:43:02.316 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:59.587+0000 7f293ac00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f293419efa0 0x7f293419f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:02.316 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:59.587+0000 7f293ac00700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f293410d100 con 0x7f293419efa0 2024-04-26T21:43:02.317 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:59.587+0000 7f2933e00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f293419efa0 0x7f293419f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:02.317 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:59.587+0000 7f2933e00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f293419efa0 0x7f293419f3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:37338/0 (socket says 172.21.15.72:37338) 2024-04-26T21:43:02.317 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:59.587+0000 7f2933e00700 1 -- 172.21.15.72:0/678682754 learned_addr learned my addr 172.21.15.72:0/678682754 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:43:02.317 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:59.588+0000 7f2933e00700 1 -- 172.21.15.72:0/678682754 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f293419f900 con 0x7f293419efa0 2024-04-26T21:43:02.317 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:59.588+0000 7f2933e00700 1 --2- 172.21.15.72:0/678682754 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f293419efa0 0x7f293419f3c0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f2934109d30 tx=0x7f2924012f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:02.317 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:59.589+0000 7f2930c00700 1 -- 172.21.15.72:0/678682754 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2924011900 con 0x7f293419efa0 2024-04-26T21:43:02.317 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:59.589+0000 7f2930c00700 1 -- 172.21.15.72:0/678682754 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2924003b30 con 0x7f293419efa0 2024-04-26T21:43:02.317 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:59.589+0000 7f2930c00700 1 -- 172.21.15.72:0/678682754 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f292401bc40 con 0x7f293419efa0 2024-04-26T21:43:02.317 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:59.589+0000 7f293ac00700 1 -- 172.21.15.72:0/678682754 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f293419fb90 con 0x7f293419efa0 2024-04-26T21:43:02.317 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:59.589+0000 7f293ac00700 1 -- 172.21.15.72:0/678682754 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29341a2670 con 0x7f293419efa0 2024-04-26T21:43:02.317 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:59.590+0000 7f2930c00700 1 -- 172.21.15.72:0/678682754 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f292401c450 con 0x7f293419efa0 2024-04-26T21:43:02.317 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:59.591+0000 7f2930c00700 1 --2- 172.21.15.72:0/678682754 >> [v2:172.21.15.72:6800/4010358672,v1:172.21.15.72:6801/4010358672] conn(0x7f2918039610 0x7f291803bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:02.317 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:59.591+0000 7f293ac00700 1 -- 172.21.15.72:0/678682754 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2934064a80 con 0x7f293419efa0 2024-04-26T21:43:02.318 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:59.591+0000 7f2930c00700 1 -- 172.21.15.72:0/678682754 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f292404dbd0 con 0x7f293419efa0 2024-04-26T21:43:02.318 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:59.591+0000 7f2933400700 1 --2- 172.21.15.72:0/678682754 >> [v2:172.21.15.72:6800/4010358672,v1:172.21.15.72:6801/4010358672] conn(0x7f2918039610 0x7f291803bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:02.318 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:59.592+0000 7f2933400700 1 --2- 172.21.15.72:0/678682754 >> [v2:172.21.15.72:6800/4010358672,v1:172.21.15.72:6801/4010358672] conn(0x7f2918039610 0x7f291803bad0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f2920006fd0 tx=0x7f2920006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:02.318 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:59.598+0000 7f2930c00700 1 -- 172.21.15.72:0/678682754 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f2924018350 con 0x7f293419efa0 2024-04-26T21:43:02.318 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:59.891+0000 7f293ac00700 1 -- 172.21.15.72:0/678682754 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7f29341a2e90 con 0x7f293419efa0 2024-04-26T21:43:02.318 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:00.721+0000 7f2930c00700 1 -- 172.21.15.72:0/678682754 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7f292404d930 con 0x7f293419efa0 2024-04-26T21:43:02.318 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:00.726+0000 7f2930c00700 1 -- 172.21.15.72:0/678682754 <== mon.0 v2:172.21.15.72:3300/0 8 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7f292401b4e0 con 0x7f293419efa0 2024-04-26T21:43:02.318 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:00.728+0000 7f293ac00700 1 -- 172.21.15.72:0/678682754 >> [v2:172.21.15.72:6800/4010358672,v1:172.21.15.72:6801/4010358672] conn(0x7f2918039610 msgr2=0x7f291803bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:02.318 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:00.728+0000 7f293ac00700 1 --2- 172.21.15.72:0/678682754 >> [v2:172.21.15.72:6800/4010358672,v1:172.21.15.72:6801/4010358672] conn(0x7f2918039610 0x7f291803bad0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f2920006fd0 tx=0x7f2920006e40 comp rx=0 tx=0).stop 2024-04-26T21:43:02.318 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:00.728+0000 7f293ac00700 1 -- 172.21.15.72:0/678682754 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f293419efa0 msgr2=0x7f293419f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:02.319 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:00.728+0000 7f293ac00700 1 --2- 172.21.15.72:0/678682754 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f293419efa0 0x7f293419f3c0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f2934109d30 tx=0x7f2924012f50 comp rx=0 tx=0).stop 2024-04-26T21:43:02.319 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:00.728+0000 7f293ac00700 1 -- 172.21.15.72:0/678682754 shutdown_connections 2024-04-26T21:43:02.319 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:00.728+0000 7f293ac00700 1 --2- 172.21.15.72:0/678682754 >> [v2:172.21.15.72:6800/4010358672,v1:172.21.15.72:6801/4010358672] conn(0x7f2918039610 0x7f291803bad0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:02.319 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:00.728+0000 7f293ac00700 1 --2- 172.21.15.72:0/678682754 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f293419efa0 0x7f293419f3c0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:02.319 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:00.728+0000 7f293ac00700 1 -- 172.21.15.72:0/678682754 >> 172.21.15.72:0/678682754 conn(0x7f2934104230 msgr2=0x7f2934104df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:02.319 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:00.728+0000 7f293ac00700 1 -- 172.21.15.72:0/678682754 shutdown_connections 2024-04-26T21:43:02.319 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:00.728+0000 7f293ac00700 1 -- 172.21.15.72:0/678682754 wait complete. 2024-04-26T21:43:03.470 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:43:03 smithi072 ceph-mon[28138]: from='client.? 172.21.15.72:0/2637460335' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-26T21:43:03.763 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 26 21:43:03 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-mgr-x[28363]: 2024-04-26T21:43:03.468+0000 7fb1b57d4200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-26T21:43:03.763 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 26 21:43:03 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-mgr-x[28363]: 2024-04-26T21:43:03.706+0000 7fb1b57d4200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-26T21:43:04.040 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 26 21:43:03 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-mgr-x[28363]: 2024-04-26T21:43:03.921+0000 7fb1b57d4200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-26T21:43:04.054 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout { 2024-04-26T21:43:04.054 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-04-26T21:43:04.054 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-26T21:43:04.054 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-04-26T21:43:04.054 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-26T21:43:04.054 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout } 2024-04-26T21:43:04.054 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:02.913+0000 7f0748a00700 1 Processor -- start 2024-04-26T21:43:04.054 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:02.913+0000 7f0748a00700 1 -- start start 2024-04-26T21:43:04.055 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:02.913+0000 7f0748a00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f074407dd00 0x7f074407c160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:04.055 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:02.913+0000 7f0748a00700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f074407c730 con 0x7f074407dd00 2024-04-26T21:43:04.055 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:02.914+0000 7f0741e00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f074407dd00 0x7f074407c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:04.055 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:02.914+0000 7f0741e00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f074407dd00 0x7f074407c160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:37154/0 (socket says 172.21.15.72:37154) 2024-04-26T21:43:04.055 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:02.914+0000 7f0741e00700 1 -- 172.21.15.72:0/1933280493 learned_addr learned my addr 172.21.15.72:0/1933280493 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:43:04.055 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:02.914+0000 7f0741e00700 1 -- 172.21.15.72:0/1933280493 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f074407c8b0 con 0x7f074407dd00 2024-04-26T21:43:04.055 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:02.914+0000 7f0741e00700 1 --2- 172.21.15.72:0/1933280493 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f074407dd00 0x7f074407c160 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f07300098a0 tx=0x7f0730009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=912a02bf758f2a4d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:04.055 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:02.915+0000 7f0740a00700 1 -- 172.21.15.72:0/1933280493 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0730004160 con 0x7f074407dd00 2024-04-26T21:43:04.055 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:02.915+0000 7f0740a00700 1 -- 172.21.15.72:0/1933280493 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0730004300 con 0x7f074407dd00 2024-04-26T21:43:04.055 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:02.915+0000 7f0740a00700 1 -- 172.21.15.72:0/1933280493 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0730011960 con 0x7f074407dd00 2024-04-26T21:43:04.055 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:02.916+0000 7f0748a00700 1 -- 172.21.15.72:0/1933280493 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f074407dd00 msgr2=0x7f074407c160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:04.055 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:02.916+0000 7f0748a00700 1 --2- 172.21.15.72:0/1933280493 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f074407dd00 0x7f074407c160 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f07300098a0 tx=0x7f0730009bb0 comp rx=0 tx=0).stop 2024-04-26T21:43:04.055 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:02.916+0000 7f0748a00700 1 -- 172.21.15.72:0/1933280493 shutdown_connections 2024-04-26T21:43:04.055 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:02.916+0000 7f0748a00700 1 --2- 172.21.15.72:0/1933280493 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f074407dd00 0x7f074407c160 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:04.056 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:02.916+0000 7f0748a00700 1 -- 172.21.15.72:0/1933280493 >> 172.21.15.72:0/1933280493 conn(0x7f0744104230 msgr2=0x7f07441066b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:04.056 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:02.916+0000 7f0748a00700 1 -- 172.21.15.72:0/1933280493 shutdown_connections 2024-04-26T21:43:04.056 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:02.916+0000 7f0748a00700 1 -- 172.21.15.72:0/1933280493 wait complete. 2024-04-26T21:43:04.056 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:02.917+0000 7f0748a00700 1 Processor -- start 2024-04-26T21:43:04.056 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:02.917+0000 7f0748a00700 1 -- start start 2024-04-26T21:43:04.056 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:02.917+0000 7f0748a00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f074407dd00 0x7f074419ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:04.056 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:02.917+0000 7f0748a00700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f074410f290 con 0x7f074407dd00 2024-04-26T21:43:04.056 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:02.918+0000 7f0741e00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f074407dd00 0x7f074419ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:04.056 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:02.918+0000 7f0741e00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f074407dd00 0x7f074419ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:37160/0 (socket says 172.21.15.72:37160) 2024-04-26T21:43:04.056 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:02.918+0000 7f0741e00700 1 -- 172.21.15.72:0/2637460335 learned_addr learned my addr 172.21.15.72:0/2637460335 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:43:04.056 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:02.918+0000 7f0741e00700 1 -- 172.21.15.72:0/2637460335 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f074419f240 con 0x7f074407dd00 2024-04-26T21:43:04.056 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:02.918+0000 7f0741e00700 1 --2- 172.21.15.72:0/2637460335 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f074407dd00 0x7f074419ed00 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f0730011670 tx=0x7f0730011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:04.056 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:02.919+0000 7f073aa00700 1 -- 172.21.15.72:0/2637460335 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0730003a60 con 0x7f074407dd00 2024-04-26T21:43:04.056 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:02.919+0000 7f073aa00700 1 -- 172.21.15.72:0/2637460335 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f073001b8d0 con 0x7f074407dd00 2024-04-26T21:43:04.056 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:02.919+0000 7f073aa00700 1 -- 172.21.15.72:0/2637460335 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0730023790 con 0x7f074407dd00 2024-04-26T21:43:04.057 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:02.919+0000 7f0748a00700 1 -- 172.21.15.72:0/2637460335 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f074419f4d0 con 0x7f074407dd00 2024-04-26T21:43:04.057 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:02.919+0000 7f0748a00700 1 -- 172.21.15.72:0/2637460335 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07441a21b0 con 0x7f074407dd00 2024-04-26T21:43:04.057 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:02.920+0000 7f073aa00700 1 -- 172.21.15.72:0/2637460335 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7f073001b440 con 0x7f074407dd00 2024-04-26T21:43:04.057 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:02.920+0000 7f0748a00700 1 -- 172.21.15.72:0/2637460335 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0744064a80 con 0x7f074407dd00 2024-04-26T21:43:04.057 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:02.920+0000 7f073aa00700 1 --2- 172.21.15.72:0/2637460335 >> [v2:172.21.15.72:6800/4010358672,v1:172.21.15.72:6801/4010358672] conn(0x7f0728039660 0x7f072803bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:04.057 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:02.921+0000 7f073aa00700 1 -- 172.21.15.72:0/2637460335 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f073004e130 con 0x7f074407dd00 2024-04-26T21:43:04.057 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:02.921+0000 7f0741400700 1 -- 172.21.15.72:0/2637460335 >> [v2:172.21.15.72:6800/4010358672,v1:172.21.15.72:6801/4010358672] conn(0x7f0728039660 msgr2=0x7f072803bb20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.72:6800/4010358672 2024-04-26T21:43:04.057 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:02.921+0000 7f0741400700 1 --2- 172.21.15.72:0/2637460335 >> [v2:172.21.15.72:6800/4010358672,v1:172.21.15.72:6801/4010358672] conn(0x7f0728039660 0x7f072803bb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T21:43:04.057 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:02.926+0000 7f073aa00700 1 -- 172.21.15.72:0/2637460335 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f073001f030 con 0x7f074407dd00 2024-04-26T21:43:04.057 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:03.122+0000 7f0741400700 1 -- 172.21.15.72:0/2637460335 >> [v2:172.21.15.72:6800/4010358672,v1:172.21.15.72:6801/4010358672] conn(0x7f0728039660 msgr2=0x7f072803bb20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.72:6800/4010358672 2024-04-26T21:43:04.057 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:03.122+0000 7f0741400700 1 --2- 172.21.15.72:0/2637460335 >> [v2:172.21.15.72:6800/4010358672,v1:172.21.15.72:6801/4010358672] conn(0x7f0728039660 0x7f072803bb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-26T21:43:04.057 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:03.211+0000 7f0748a00700 1 -- 172.21.15.72:0/2637460335 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f07441a2760 con 0x7f074407dd00 2024-04-26T21:43:04.057 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:03.212+0000 7f073aa00700 1 -- 172.21.15.72:0/2637460335 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7f07300239f0 con 0x7f074407dd00 2024-04-26T21:43:04.057 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:03.214+0000 7f0748a00700 1 -- 172.21.15.72:0/2637460335 >> [v2:172.21.15.72:6800/4010358672,v1:172.21.15.72:6801/4010358672] conn(0x7f0728039660 msgr2=0x7f072803bb20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T21:43:04.057 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:03.214+0000 7f0748a00700 1 --2- 172.21.15.72:0/2637460335 >> [v2:172.21.15.72:6800/4010358672,v1:172.21.15.72:6801/4010358672] conn(0x7f0728039660 0x7f072803bb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:04.058 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:03.214+0000 7f0748a00700 1 -- 172.21.15.72:0/2637460335 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f074407dd00 msgr2=0x7f074419ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:04.058 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:03.214+0000 7f0748a00700 1 --2- 172.21.15.72:0/2637460335 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f074407dd00 0x7f074419ed00 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f0730011670 tx=0x7f0730011a30 comp rx=0 tx=0).stop 2024-04-26T21:43:04.058 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:03.215+0000 7f0748a00700 1 -- 172.21.15.72:0/2637460335 shutdown_connections 2024-04-26T21:43:04.058 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:03.215+0000 7f0748a00700 1 --2- 172.21.15.72:0/2637460335 >> [v2:172.21.15.72:6800/4010358672,v1:172.21.15.72:6801/4010358672] conn(0x7f0728039660 0x7f072803bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:04.058 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:03.215+0000 7f0748a00700 1 --2- 172.21.15.72:0/2637460335 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f074407dd00 0x7f074419ed00 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:04.058 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:03.215+0000 7f0748a00700 1 -- 172.21.15.72:0/2637460335 >> 172.21.15.72:0/2637460335 conn(0x7f0744104230 msgr2=0x7f0744104d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:04.058 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:03.215+0000 7f0748a00700 1 -- 172.21.15.72:0/2637460335 shutdown_connections 2024-04-26T21:43:04.058 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:03.215+0000 7f0748a00700 1 -- 172.21.15.72:0/2637460335 wait complete. 2024-04-26T21:43:04.058 INFO:teuthology.orchestra.run.smithi072.stdout:Waiting for the mgr to restart... 2024-04-26T21:43:04.058 INFO:teuthology.orchestra.run.smithi072.stdout:Waiting for mgr epoch 9... 2024-04-26T21:43:04.572 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 26 21:43:04 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-mgr-x[28363]: 2024-04-26T21:43:04.309+0000 7fb1b57d4200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-26T21:43:04.572 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 26 21:43:04 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-mgr-x[28363]: 2024-04-26T21:43:04.439+0000 7fb1b57d4200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-26T21:43:04.877 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 26 21:43:04 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-mgr-x[28363]: 2024-04-26T21:43:04.702+0000 7fb1b57d4200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-26T21:43:05.278 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 26 21:43:04 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-mgr-x[28363]: 2024-04-26T21:43:04.955+0000 7fb1b57d4200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-26T21:43:06.072 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 26 21:43:05 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-mgr-x[28363]: 2024-04-26T21:43:05.731+0000 7fb1b57d4200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-26T21:43:06.404 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 26 21:43:06 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-mgr-x[28363]: 2024-04-26T21:43:06.276+0000 7fb1b57d4200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-26T21:43:06.404 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 26 21:43:06 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-mgr-x[28363]: 2024-04-26T21:43:06.402+0000 7fb1b57d4200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-26T21:43:07.072 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 26 21:43:06 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-mgr-x[28363]: 2024-04-26T21:43:06.666+0000 7fb1b57d4200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-26T21:43:07.572 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 26 21:43:07 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-mgr-x[28363]: 2024-04-26T21:43:07.301+0000 7fb1b57d4200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-26T21:43:07.572 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 26 21:43:07 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-mgr-x[28363]: 2024-04-26T21:43:07.424+0000 7fb1b57d4200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-26T21:43:08.048 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 26 21:43:07 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-mgr-x[28363]: 2024-04-26T21:43:07.742+0000 7fb1b57d4200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-26T21:43:08.048 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 26 21:43:07 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-mgr-x[28363]: 2024-04-26T21:43:07.919+0000 7fb1b57d4200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-26T21:43:08.048 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 26 21:43:08 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-mgr-x[28363]: 2024-04-26T21:43:08.046+0000 7fb1b57d4200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-26T21:43:08.322 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 26 21:43:08 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-mgr-x[28363]: 2024-04-26T21:43:08.196+0000 7fb1b57d4200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-26T21:43:08.822 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 26 21:43:08 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-mgr-x[28363]: 2024-04-26T21:43:08.562+0000 7fb1b57d4200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-26T21:43:09.572 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 26 21:43:09 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-mgr-x[28363]: 2024-04-26T21:43:09.314+0000 7fb1b57d4200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-26T21:43:09.905 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 26 21:43:09 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-mgr-x[28363]: 2024-04-26T21:43:09.598+0000 7fb1b57d4200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-26T21:43:09.906 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:43:09 smithi072 ceph-mon[28138]: Active manager daemon x restarted 2024-04-26T21:43:09.906 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:43:09 smithi072 ceph-mon[28138]: Activating manager daemon x 2024-04-26T21:43:09.906 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:43:09 smithi072 ceph-mon[28138]: osdmap e3: 0 total, 0 up, 0 in 2024-04-26T21:43:09.906 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:43:09 smithi072 ceph-mon[28138]: mgrmap e10: x(active, starting, since 0.00944299s) 2024-04-26T21:43:09.906 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:43:09 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T21:43:09.906 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:43:09 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-26T21:43:09.906 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:43:09 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-26T21:43:09.906 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:43:09 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-26T21:43:09.906 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:43:09 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-26T21:43:09.906 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:43:09 smithi072 ceph-mon[28138]: Manager daemon x is now available 2024-04-26T21:43:11.042 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:43:10 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:43:11.042 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:43:10 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-26T21:43:11.042 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:43:10 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-26T21:43:11.042 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:43:10 smithi072 ceph-mon[28138]: [26/Apr/2024:21:43:09] ENGINE Bus STARTING 2024-04-26T21:43:11.043 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:43:10 smithi072 ceph-mon[28138]: mgrmap e11: x(active, since 1.01366s) 2024-04-26T21:43:11.585 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout { 2024-04-26T21:43:11.585 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-04-26T21:43:11.586 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-26T21:43:11.586 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout } 2024-04-26T21:43:11.586 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:04.667+0000 7fa880200700 1 Processor -- start 2024-04-26T21:43:11.586 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:04.667+0000 7fa880200700 1 -- start start 2024-04-26T21:43:11.586 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:04.667+0000 7fa880200700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa878108c80 0x7fa8781090a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:11.586 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:04.667+0000 7fa880200700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fa878109670 con 0x7fa878108c80 2024-04-26T21:43:11.586 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:04.668+0000 7fa87d800700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa878108c80 0x7fa8781090a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:11.586 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:04.668+0000 7fa87d800700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa878108c80 0x7fa8781090a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:37174/0 (socket says 172.21.15.72:37174) 2024-04-26T21:43:11.586 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:04.668+0000 7fa87d800700 1 -- 172.21.15.72:0/1993675295 learned_addr learned my addr 172.21.15.72:0/1993675295 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:43:11.586 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:04.668+0000 7fa87d800700 1 -- 172.21.15.72:0/1993675295 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa878109e80 con 0x7fa878108c80 2024-04-26T21:43:11.586 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:04.669+0000 7fa87d800700 1 --2- 172.21.15.72:0/1993675295 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa878108c80 0x7fa8781090a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa86c0098a0 tx=0x7fa86c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=848e2f42134ac297 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:11.587 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:04.669+0000 7fa877e00700 1 -- 172.21.15.72:0/1993675295 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa86c004160 con 0x7fa878108c80 2024-04-26T21:43:11.587 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:04.669+0000 7fa877e00700 1 -- 172.21.15.72:0/1993675295 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa86c004300 con 0x7fa878108c80 2024-04-26T21:43:11.587 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:04.669+0000 7fa877e00700 1 -- 172.21.15.72:0/1993675295 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa86c011960 con 0x7fa878108c80 2024-04-26T21:43:11.587 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:04.670+0000 7fa880200700 1 -- 172.21.15.72:0/1993675295 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa878108c80 msgr2=0x7fa8781090a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:11.587 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:04.670+0000 7fa880200700 1 --2- 172.21.15.72:0/1993675295 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa878108c80 0x7fa8781090a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa86c0098a0 tx=0x7fa86c009bb0 comp rx=0 tx=0).stop 2024-04-26T21:43:11.587 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:04.670+0000 7fa880200700 1 -- 172.21.15.72:0/1993675295 shutdown_connections 2024-04-26T21:43:11.587 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:04.670+0000 7fa880200700 1 --2- 172.21.15.72:0/1993675295 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa878108c80 0x7fa8781090a0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:11.587 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:04.670+0000 7fa880200700 1 -- 172.21.15.72:0/1993675295 >> 172.21.15.72:0/1993675295 conn(0x7fa878104260 msgr2=0x7fa878106680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:11.587 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:04.670+0000 7fa880200700 1 -- 172.21.15.72:0/1993675295 shutdown_connections 2024-04-26T21:43:11.587 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:04.670+0000 7fa880200700 1 -- 172.21.15.72:0/1993675295 wait complete. 2024-04-26T21:43:11.587 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:04.671+0000 7fa880200700 1 Processor -- start 2024-04-26T21:43:11.587 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:04.671+0000 7fa880200700 1 -- start start 2024-04-26T21:43:11.587 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:04.671+0000 7fa880200700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa878108c80 0x7fa87807d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:11.587 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:04.671+0000 7fa880200700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fa87810d0b0 con 0x7fa878108c80 2024-04-26T21:43:11.588 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:04.671+0000 7fa87d800700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa878108c80 0x7fa87807d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:11.588 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:04.672+0000 7fa87d800700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa878108c80 0x7fa87807d9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:37176/0 (socket says 172.21.15.72:37176) 2024-04-26T21:43:11.588 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:04.672+0000 7fa87d800700 1 -- 172.21.15.72:0/685249961 learned_addr learned my addr 172.21.15.72:0/685249961 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:43:11.588 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:04.672+0000 7fa87d800700 1 -- 172.21.15.72:0/685249961 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa87807df30 con 0x7fa878108c80 2024-04-26T21:43:11.588 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:04.672+0000 7fa87d800700 1 --2- 172.21.15.72:0/685249961 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa878108c80 0x7fa87807d9f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa86c011670 tx=0x7fa86c011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:11.588 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:04.672+0000 7fa876000700 1 -- 172.21.15.72:0/685249961 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa86c01f070 con 0x7fa878108c80 2024-04-26T21:43:11.588 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:04.672+0000 7fa876000700 1 -- 172.21.15.72:0/685249961 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa86c003b20 con 0x7fa878108c80 2024-04-26T21:43:11.588 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:04.672+0000 7fa876000700 1 -- 172.21.15.72:0/685249961 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa86c019580 con 0x7fa878108c80 2024-04-26T21:43:11.588 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:04.673+0000 7fa880200700 1 -- 172.21.15.72:0/685249961 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa87807c120 con 0x7fa878108c80 2024-04-26T21:43:11.588 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:04.673+0000 7fa880200700 1 -- 172.21.15.72:0/685249961 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa87807c660 con 0x7fa878108c80 2024-04-26T21:43:11.588 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:04.674+0000 7fa876000700 1 -- 172.21.15.72:0/685249961 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7fa86c022070 con 0x7fa878108c80 2024-04-26T21:43:11.588 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:04.675+0000 7fa876000700 1 --2- 172.21.15.72:0/685249961 >> [v2:172.21.15.72:6800/4010358672,v1:172.21.15.72:6801/4010358672] conn(0x7fa860039660 0x7fa86003bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:11.588 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:04.675+0000 7fa876000700 1 -- 172.21.15.72:0/685249961 --> [v2:172.21.15.72:6800/4010358672,v1:172.21.15.72:6801/4010358672] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fa86003c230 con 0x7fa860039660 2024-04-26T21:43:11.588 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:04.675+0000 7fa876000700 1 -- 172.21.15.72:0/685249961 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fa86c04d7c0 con 0x7fa878108c80 2024-04-26T21:43:11.589 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:04.675+0000 7fa87ce00700 1 -- 172.21.15.72:0/685249961 >> [v2:172.21.15.72:6800/4010358672,v1:172.21.15.72:6801/4010358672] conn(0x7fa860039660 msgr2=0x7fa86003bb20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.72:6800/4010358672 2024-04-26T21:43:11.589 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:04.675+0000 7fa87ce00700 1 --2- 172.21.15.72:0/685249961 >> [v2:172.21.15.72:6800/4010358672,v1:172.21.15.72:6801/4010358672] conn(0x7fa860039660 0x7fa86003bb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T21:43:11.589 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:04.876+0000 7fa87ce00700 1 -- 172.21.15.72:0/685249961 >> [v2:172.21.15.72:6800/4010358672,v1:172.21.15.72:6801/4010358672] conn(0x7fa860039660 msgr2=0x7fa86003bb20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.72:6800/4010358672 2024-04-26T21:43:11.589 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:04.876+0000 7fa87ce00700 1 --2- 172.21.15.72:0/685249961 >> [v2:172.21.15.72:6800/4010358672,v1:172.21.15.72:6801/4010358672] conn(0x7fa860039660 0x7fa86003bb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-26T21:43:11.589 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:05.277+0000 7fa87ce00700 1 -- 172.21.15.72:0/685249961 >> [v2:172.21.15.72:6800/4010358672,v1:172.21.15.72:6801/4010358672] conn(0x7fa860039660 msgr2=0x7fa86003bb20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.72:6800/4010358672 2024-04-26T21:43:11.589 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:05.277+0000 7fa87ce00700 1 --2- 172.21.15.72:0/685249961 >> [v2:172.21.15.72:6800/4010358672,v1:172.21.15.72:6801/4010358672] conn(0x7fa860039660 0x7fa86003bb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-26T21:43:11.589 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:06.078+0000 7fa87ce00700 1 -- 172.21.15.72:0/685249961 >> [v2:172.21.15.72:6800/4010358672,v1:172.21.15.72:6801/4010358672] conn(0x7fa860039660 msgr2=0x7fa86003bb20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.72:6800/4010358672 2024-04-26T21:43:11.589 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:06.078+0000 7fa87ce00700 1 --2- 172.21.15.72:0/685249961 >> [v2:172.21.15.72:6800/4010358672,v1:172.21.15.72:6801/4010358672] conn(0x7fa860039660 0x7fa86003bb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-26T21:43:11.589 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:07.680+0000 7fa87ce00700 1 -- 172.21.15.72:0/685249961 >> [v2:172.21.15.72:6800/4010358672,v1:172.21.15.72:6801/4010358672] conn(0x7fa860039660 msgr2=0x7fa86003bb20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.72:6800/4010358672 2024-04-26T21:43:11.589 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:07.680+0000 7fa87ce00700 1 --2- 172.21.15.72:0/685249961 >> [v2:172.21.15.72:6800/4010358672,v1:172.21.15.72:6801/4010358672] conn(0x7fa860039660 0x7fa86003bb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-26T21:43:11.589 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:09.609+0000 7fa876000700 1 -- 172.21.15.72:0/685249961 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mgrmap(e 10) ==== 46338+0+0 (secure 0 0 0) 0x7fa86c01b580 con 0x7fa878108c80 2024-04-26T21:43:11.589 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:09.610+0000 7fa876000700 1 -- 172.21.15.72:0/685249961 >> [v2:172.21.15.72:6800/4010358672,v1:172.21.15.72:6801/4010358672] conn(0x7fa860039660 msgr2=0x7fa86003bb20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T21:43:11.589 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:09.610+0000 7fa876000700 1 --2- 172.21.15.72:0/685249961 >> [v2:172.21.15.72:6800/4010358672,v1:172.21.15.72:6801/4010358672] conn(0x7fa860039660 0x7fa86003bb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:11.589 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:10.611+0000 7fa876000700 1 -- 172.21.15.72:0/685249961 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mgrmap(e 11) ==== 46465+0+0 (secure 0 0 0) 0x7fa86c00b3e0 con 0x7fa878108c80 2024-04-26T21:43:11.589 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:10.612+0000 7fa876000700 1 --2- 172.21.15.72:0/685249961 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fa86003cff0 0x7fa86003f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:11.590 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:10.612+0000 7fa876000700 1 -- 172.21.15.72:0/685249961 --> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fa86003c230 con 0x7fa86003cff0 2024-04-26T21:43:11.590 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:10.612+0000 7fa87ce00700 1 --2- 172.21.15.72:0/685249961 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fa86003cff0 0x7fa86003f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:11.590 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:10.614+0000 7fa87ce00700 1 --2- 172.21.15.72:0/685249961 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fa86003cff0 0x7fa86003f3e0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fa864003a10 tx=0x7fa864009270 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:11.590 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:10.614+0000 7fa876000700 1 -- 172.21.15.72:0/685249961 <== mgr.14152 v2:172.21.15.72:6800/588150172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7fa86003c230 con 0x7fa86003cff0 2024-04-26T21:43:11.590 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:10.624+0000 7fa880200700 1 -- 172.21.15.72:0/685249961 --> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7fa87810d230 con 0x7fa86003cff0 2024-04-26T21:43:11.590 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:10.624+0000 7fa876000700 1 -- 172.21.15.72:0/685249961 <== mgr.14152 v2:172.21.15.72:6800/588150172 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7fa87810d230 con 0x7fa86003cff0 2024-04-26T21:43:11.590 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:10.625+0000 7fa880200700 1 -- 172.21.15.72:0/685249961 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fa86003cff0 msgr2=0x7fa86003f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:11.590 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:10.625+0000 7fa880200700 1 --2- 172.21.15.72:0/685249961 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fa86003cff0 0x7fa86003f3e0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fa864003a10 tx=0x7fa864009270 comp rx=0 tx=0).stop 2024-04-26T21:43:11.590 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:10.625+0000 7fa880200700 1 -- 172.21.15.72:0/685249961 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa878108c80 msgr2=0x7fa87807d9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:11.590 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:10.625+0000 7fa880200700 1 --2- 172.21.15.72:0/685249961 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa878108c80 0x7fa87807d9f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa86c011670 tx=0x7fa86c011a30 comp rx=0 tx=0).stop 2024-04-26T21:43:11.590 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:10.625+0000 7fa880200700 1 -- 172.21.15.72:0/685249961 shutdown_connections 2024-04-26T21:43:11.590 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:10.625+0000 7fa880200700 1 --2- 172.21.15.72:0/685249961 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fa86003cff0 0x7fa86003f3e0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:11.591 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:10.625+0000 7fa880200700 1 --2- 172.21.15.72:0/685249961 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa878108c80 0x7fa87807d9f0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:11.591 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:10.625+0000 7fa880200700 1 --2- 172.21.15.72:0/685249961 >> [v2:172.21.15.72:6800/4010358672,v1:172.21.15.72:6801/4010358672] conn(0x7fa860039660 0x7fa86003bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:11.591 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:10.625+0000 7fa880200700 1 -- 172.21.15.72:0/685249961 >> 172.21.15.72:0/685249961 conn(0x7fa878104260 msgr2=0x7fa878104d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:11.591 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:10.625+0000 7fa880200700 1 -- 172.21.15.72:0/685249961 shutdown_connections 2024-04-26T21:43:11.591 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:10.625+0000 7fa880200700 1 -- 172.21.15.72:0/685249961 wait complete. 2024-04-26T21:43:11.591 INFO:teuthology.orchestra.run.smithi072.stdout:mgr epoch 9 is available 2024-04-26T21:43:11.591 INFO:teuthology.orchestra.run.smithi072.stdout:Generating a dashboard self-signed certificate... 2024-04-26T21:43:11.822 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:43:11 smithi072 ceph-mon[28138]: [26/Apr/2024:21:43:09] ENGINE Serving on https://172.21.15.72:7150 2024-04-26T21:43:11.822 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:43:11 smithi072 ceph-mon[28138]: [26/Apr/2024:21:43:10] ENGINE Serving on http://172.21.15.72:8765 2024-04-26T21:43:11.822 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:43:11 smithi072 ceph-mon[28138]: [26/Apr/2024:21:43:10] ENGINE Bus STARTED 2024-04-26T21:43:11.822 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:43:11 smithi072 ceph-mon[28138]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-26T21:43:11.822 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:43:11 smithi072 ceph-mon[28138]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-26T21:43:12.822 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:43:12 smithi072 ceph-mon[28138]: mgrmap e12: x(active, since 2s) 2024-04-26T21:43:12.822 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:43:12 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:43:13.800 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-04-26T21:43:13.800 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:12.635+0000 7f963e800700 1 Processor -- start 2024-04-26T21:43:13.800 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:12.635+0000 7f963e800700 1 -- start start 2024-04-26T21:43:13.800 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:12.635+0000 7f963e800700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9638108cd0 0x7f96381090f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:13.800 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:12.635+0000 7f963e800700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f96381096c0 con 0x7f9638108cd0 2024-04-26T21:43:13.800 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:12.636+0000 7f9637400700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9638108cd0 0x7f96381090f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:13.800 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:12.636+0000 7f9637400700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9638108cd0 0x7f96381090f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:39220/0 (socket says 172.21.15.72:39220) 2024-04-26T21:43:13.800 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:12.636+0000 7f9637400700 1 -- 172.21.15.72:0/489157134 learned_addr learned my addr 172.21.15.72:0/489157134 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:43:13.800 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:12.636+0000 7f9637400700 1 -- 172.21.15.72:0/489157134 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9638109ed0 con 0x7f9638108cd0 2024-04-26T21:43:13.800 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:12.637+0000 7f9637400700 1 --2- 172.21.15.72:0/489157134 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9638108cd0 0x7f96381090f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f96280098a0 tx=0x7f9628009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a623e801661c4ae5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:13.800 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:12.637+0000 7f9636000700 1 -- 172.21.15.72:0/489157134 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9628004160 con 0x7f9638108cd0 2024-04-26T21:43:13.801 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:12.637+0000 7f9636000700 1 -- 172.21.15.72:0/489157134 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9628004300 con 0x7f9638108cd0 2024-04-26T21:43:13.801 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:12.637+0000 7f9636000700 1 -- 172.21.15.72:0/489157134 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9628011960 con 0x7f9638108cd0 2024-04-26T21:43:13.801 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:12.638+0000 7f963e800700 1 -- 172.21.15.72:0/489157134 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9638108cd0 msgr2=0x7f96381090f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:13.801 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:12.638+0000 7f963e800700 1 --2- 172.21.15.72:0/489157134 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9638108cd0 0x7f96381090f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f96280098a0 tx=0x7f9628009bb0 comp rx=0 tx=0).stop 2024-04-26T21:43:13.801 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:12.638+0000 7f963e800700 1 -- 172.21.15.72:0/489157134 shutdown_connections 2024-04-26T21:43:13.801 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:12.638+0000 7f963e800700 1 --2- 172.21.15.72:0/489157134 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9638108cd0 0x7f96381090f0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:13.801 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:12.638+0000 7f963e800700 1 -- 172.21.15.72:0/489157134 >> 172.21.15.72:0/489157134 conn(0x7f9638104230 msgr2=0x7f96381066b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:13.801 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:12.638+0000 7f963e800700 1 -- 172.21.15.72:0/489157134 shutdown_connections 2024-04-26T21:43:13.801 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:12.638+0000 7f963e800700 1 -- 172.21.15.72:0/489157134 wait complete. 2024-04-26T21:43:13.801 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:12.640+0000 7f963e800700 1 Processor -- start 2024-04-26T21:43:13.801 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:12.640+0000 7f963e800700 1 -- start start 2024-04-26T21:43:13.801 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:12.640+0000 7f963e800700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9638108cd0 0x7f963807d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:13.801 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:12.640+0000 7f963e800700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f963810d100 con 0x7f9638108cd0 2024-04-26T21:43:13.802 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:12.640+0000 7f9637400700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9638108cd0 0x7f963807d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:13.802 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:12.641+0000 7f9637400700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9638108cd0 0x7f963807d9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:39234/0 (socket says 172.21.15.72:39234) 2024-04-26T21:43:13.802 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:12.641+0000 7f9637400700 1 -- 172.21.15.72:0/706218033 learned_addr learned my addr 172.21.15.72:0/706218033 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:43:13.802 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:12.641+0000 7f9637400700 1 -- 172.21.15.72:0/706218033 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f963807df30 con 0x7f9638108cd0 2024-04-26T21:43:13.802 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:12.641+0000 7f9637400700 1 --2- 172.21.15.72:0/706218033 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9638108cd0 0x7f963807d9f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f9628011670 tx=0x7f9628011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:13.802 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:12.641+0000 7f962fe00700 1 -- 172.21.15.72:0/706218033 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9628003a60 con 0x7f9638108cd0 2024-04-26T21:43:13.802 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:12.641+0000 7f962fe00700 1 -- 172.21.15.72:0/706218033 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f962801b8d0 con 0x7f9638108cd0 2024-04-26T21:43:13.802 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:12.641+0000 7f963e800700 1 -- 172.21.15.72:0/706218033 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f963807c120 con 0x7f9638108cd0 2024-04-26T21:43:13.802 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:12.642+0000 7f963e800700 1 -- 172.21.15.72:0/706218033 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f963807c660 con 0x7f9638108cd0 2024-04-26T21:43:13.802 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:12.642+0000 7f962fe00700 1 -- 172.21.15.72:0/706218033 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9628023790 con 0x7f9638108cd0 2024-04-26T21:43:13.802 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:12.643+0000 7f962fe00700 1 -- 172.21.15.72:0/706218033 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 12) ==== 46571+0+0 (secure 0 0 0) 0x7f9628023ab0 con 0x7f9638108cd0 2024-04-26T21:43:13.802 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:12.643+0000 7f962fe00700 1 --2- 172.21.15.72:0/706218033 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f961c039600 0x7f961c03bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:13.802 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:12.643+0000 7f962fe00700 1 -- 172.21.15.72:0/706218033 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f962804e1f0 con 0x7f9638108cd0 2024-04-26T21:43:13.803 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:12.643+0000 7f963e800700 1 -- 172.21.15.72:0/706218033 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9638064a80 con 0x7f9638108cd0 2024-04-26T21:43:13.803 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:12.644+0000 7f9636a00700 1 --2- 172.21.15.72:0/706218033 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f961c039600 0x7f961c03bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:13.803 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:12.644+0000 7f9636a00700 1 --2- 172.21.15.72:0/706218033 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f961c039600 0x7f961c03bac0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f9624006fd0 tx=0x7f9624006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:13.803 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:12.649+0000 7f962fe00700 1 -- 172.21.15.72:0/706218033 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f962801f030 con 0x7f9638108cd0 2024-04-26T21:43:13.803 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:12.909+0000 7f963e800700 1 -- 172.21.15.72:0/706218033 --> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7f963807cf90 con 0x7f961c039600 2024-04-26T21:43:13.803 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:13.074+0000 7f962fe00700 1 -- 172.21.15.72:0/706218033 <== mgr.14152 v2:172.21.15.72:6800/588150172 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7f963807cf90 con 0x7f961c039600 2024-04-26T21:43:13.803 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:13.077+0000 7f963e800700 1 -- 172.21.15.72:0/706218033 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f961c039600 msgr2=0x7f961c03bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:13.803 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:13.077+0000 7f963e800700 1 --2- 172.21.15.72:0/706218033 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f961c039600 0x7f961c03bac0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f9624006fd0 tx=0x7f9624006e40 comp rx=0 tx=0).stop 2024-04-26T21:43:13.803 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:13.077+0000 7f963e800700 1 -- 172.21.15.72:0/706218033 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9638108cd0 msgr2=0x7f963807d9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:13.803 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:13.077+0000 7f963e800700 1 --2- 172.21.15.72:0/706218033 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9638108cd0 0x7f963807d9f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f9628011670 tx=0x7f9628011a30 comp rx=0 tx=0).stop 2024-04-26T21:43:13.803 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:13.077+0000 7f963e800700 1 -- 172.21.15.72:0/706218033 shutdown_connections 2024-04-26T21:43:13.803 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:13.077+0000 7f963e800700 1 --2- 172.21.15.72:0/706218033 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f961c039600 0x7f961c03bac0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:13.803 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:13.077+0000 7f963e800700 1 --2- 172.21.15.72:0/706218033 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9638108cd0 0x7f963807d9f0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:13.804 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:13.077+0000 7f963e800700 1 -- 172.21.15.72:0/706218033 >> 172.21.15.72:0/706218033 conn(0x7f9638104230 msgr2=0x7f9638104e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:13.804 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:13.078+0000 7f963e800700 1 -- 172.21.15.72:0/706218033 shutdown_connections 2024-04-26T21:43:13.804 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:13.078+0000 7f963e800700 1 -- 172.21.15.72:0/706218033 wait complete. 2024-04-26T21:43:13.804 INFO:teuthology.orchestra.run.smithi072.stdout:Creating initial admin user... 2024-04-26T21:43:14.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:43:14 smithi072 ceph-mon[28138]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:43:14.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:43:14 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:43:14.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:43:14 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:43:15.572 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:43:15 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:43:15.572 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:43:15 smithi072 ceph-mon[28138]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:43:15.572 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:43:15 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:43:15.617 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$7Yt.ByRQaAm19eGf55LQ6uk60mNlsCJ0tC1g1IQxO/AHxGZf0UT.a", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1714167794, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-04-26T21:43:15.617 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:14.417+0000 7f6778c00700 1 Processor -- start 2024-04-26T21:43:15.617 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:14.418+0000 7f6778c00700 1 -- start start 2024-04-26T21:43:15.617 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:14.418+0000 7f6778c00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f677407dd00 0x7f677407c160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:15.617 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:14.418+0000 7f6778c00700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f677407c730 con 0x7f677407dd00 2024-04-26T21:43:15.617 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:14.418+0000 7f6771e00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f677407dd00 0x7f677407c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:15.617 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:14.419+0000 7f6771e00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f677407dd00 0x7f677407c160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:39242/0 (socket says 172.21.15.72:39242) 2024-04-26T21:43:15.617 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:14.419+0000 7f6771e00700 1 -- 172.21.15.72:0/727654622 learned_addr learned my addr 172.21.15.72:0/727654622 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:43:15.618 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:14.419+0000 7f6771e00700 1 -- 172.21.15.72:0/727654622 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f677407c8b0 con 0x7f677407dd00 2024-04-26T21:43:15.618 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:14.419+0000 7f6771e00700 1 --2- 172.21.15.72:0/727654622 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f677407dd00 0x7f677407c160 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f67640098a0 tx=0x7f6764009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f11f57f8bd393a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:15.618 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:14.420+0000 7f6770a00700 1 -- 172.21.15.72:0/727654622 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6764004160 con 0x7f677407dd00 2024-04-26T21:43:15.618 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:14.420+0000 7f6770a00700 1 -- 172.21.15.72:0/727654622 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6764004300 con 0x7f677407dd00 2024-04-26T21:43:15.618 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:14.420+0000 7f6770a00700 1 -- 172.21.15.72:0/727654622 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6764011960 con 0x7f677407dd00 2024-04-26T21:43:15.618 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:14.420+0000 7f6778c00700 1 -- 172.21.15.72:0/727654622 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f677407dd00 msgr2=0x7f677407c160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:15.618 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:14.420+0000 7f6778c00700 1 --2- 172.21.15.72:0/727654622 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f677407dd00 0x7f677407c160 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f67640098a0 tx=0x7f6764009bb0 comp rx=0 tx=0).stop 2024-04-26T21:43:15.618 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:14.421+0000 7f6778c00700 1 -- 172.21.15.72:0/727654622 shutdown_connections 2024-04-26T21:43:15.618 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:14.421+0000 7f6778c00700 1 --2- 172.21.15.72:0/727654622 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f677407dd00 0x7f677407c160 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:15.618 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:14.421+0000 7f6778c00700 1 -- 172.21.15.72:0/727654622 >> 172.21.15.72:0/727654622 conn(0x7f67741041c0 msgr2=0x7f6774106640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:15.618 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:14.421+0000 7f6778c00700 1 -- 172.21.15.72:0/727654622 shutdown_connections 2024-04-26T21:43:15.619 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:14.421+0000 7f6778c00700 1 -- 172.21.15.72:0/727654622 wait complete. 2024-04-26T21:43:15.619 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:14.422+0000 7f6778c00700 1 Processor -- start 2024-04-26T21:43:15.619 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:14.422+0000 7f6778c00700 1 -- start start 2024-04-26T21:43:15.619 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:14.423+0000 7f6778c00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f677407dd00 0x7f677419ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:15.619 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:14.423+0000 7f6778c00700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f677410f220 con 0x7f677407dd00 2024-04-26T21:43:15.619 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:14.423+0000 7f6771e00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f677407dd00 0x7f677419ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:15.619 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:14.423+0000 7f6771e00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f677407dd00 0x7f677419ecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:39252/0 (socket says 172.21.15.72:39252) 2024-04-26T21:43:15.619 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:14.423+0000 7f6771e00700 1 -- 172.21.15.72:0/3273926015 learned_addr learned my addr 172.21.15.72:0/3273926015 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:43:15.619 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:14.423+0000 7f6771e00700 1 -- 172.21.15.72:0/3273926015 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f677419f200 con 0x7f677407dd00 2024-04-26T21:43:15.619 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:14.424+0000 7f6771e00700 1 --2- 172.21.15.72:0/3273926015 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f677407dd00 0x7f677419ecc0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f6764011670 tx=0x7f6764011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:15.619 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:14.424+0000 7f676aa00700 1 -- 172.21.15.72:0/3273926015 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6764003a60 con 0x7f677407dd00 2024-04-26T21:43:15.619 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:14.424+0000 7f676aa00700 1 -- 172.21.15.72:0/3273926015 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f676401b8d0 con 0x7f677407dd00 2024-04-26T21:43:15.620 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:14.424+0000 7f6778c00700 1 -- 172.21.15.72:0/3273926015 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f677419f490 con 0x7f677407dd00 2024-04-26T21:43:15.620 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:14.424+0000 7f676aa00700 1 -- 172.21.15.72:0/3273926015 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6764023790 con 0x7f677407dd00 2024-04-26T21:43:15.620 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:14.424+0000 7f6778c00700 1 -- 172.21.15.72:0/3273926015 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67741a2180 con 0x7f677407dd00 2024-04-26T21:43:15.620 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:14.426+0000 7f676aa00700 1 -- 172.21.15.72:0/3273926015 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 12) ==== 46571+0+0 (secure 0 0 0) 0x7f676401b440 con 0x7f677407dd00 2024-04-26T21:43:15.620 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:14.426+0000 7f6778c00700 1 -- 172.21.15.72:0/3273926015 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6774064a80 con 0x7f677407dd00 2024-04-26T21:43:15.620 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:14.426+0000 7f676aa00700 1 --2- 172.21.15.72:0/3273926015 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f6758039660 0x7f675803bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:15.620 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:14.426+0000 7f676aa00700 1 -- 172.21.15.72:0/3273926015 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f676404e160 con 0x7f677407dd00 2024-04-26T21:43:15.620 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:14.426+0000 7f6771400700 1 --2- 172.21.15.72:0/3273926015 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f6758039660 0x7f675803bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:15.620 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:14.427+0000 7f6771400700 1 --2- 172.21.15.72:0/3273926015 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f6758039660 0x7f675803bb20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f6760006fd0 tx=0x7f6760006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:15.620 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:14.433+0000 7f676aa00700 1 -- 172.21.15.72:0/3273926015 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f676401f030 con 0x7f677407dd00 2024-04-26T21:43:15.620 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:14.678+0000 7f6778c00700 1 -- 172.21.15.72:0/3273926015 --> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7f677419f970 con 0x7f6758039660 2024-04-26T21:43:15.620 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:14.935+0000 7f676aa00700 1 -- 172.21.15.72:0/3273926015 <== mgr.14152 v2:172.21.15.72:6800/588150172 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7f677419f970 con 0x7f6758039660 2024-04-26T21:43:15.621 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:14.938+0000 7f6778c00700 1 -- 172.21.15.72:0/3273926015 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f6758039660 msgr2=0x7f675803bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:15.621 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:14.938+0000 7f6778c00700 1 --2- 172.21.15.72:0/3273926015 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f6758039660 0x7f675803bb20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f6760006fd0 tx=0x7f6760006e40 comp rx=0 tx=0).stop 2024-04-26T21:43:15.621 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:14.938+0000 7f6778c00700 1 -- 172.21.15.72:0/3273926015 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f677407dd00 msgr2=0x7f677419ecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:15.621 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:14.938+0000 7f6778c00700 1 --2- 172.21.15.72:0/3273926015 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f677407dd00 0x7f677419ecc0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f6764011670 tx=0x7f6764011a30 comp rx=0 tx=0).stop 2024-04-26T21:43:15.621 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:14.939+0000 7f6778c00700 1 -- 172.21.15.72:0/3273926015 shutdown_connections 2024-04-26T21:43:15.621 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:14.939+0000 7f6778c00700 1 --2- 172.21.15.72:0/3273926015 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f6758039660 0x7f675803bb20 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:15.621 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:14.939+0000 7f6778c00700 1 --2- 172.21.15.72:0/3273926015 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f677407dd00 0x7f677419ecc0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:15.621 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:14.939+0000 7f6778c00700 1 -- 172.21.15.72:0/3273926015 >> 172.21.15.72:0/3273926015 conn(0x7f67741041c0 msgr2=0x7f6774104cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:15.621 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:14.939+0000 7f6778c00700 1 -- 172.21.15.72:0/3273926015 shutdown_connections 2024-04-26T21:43:15.621 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:14.939+0000 7f6778c00700 1 -- 172.21.15.72:0/3273926015 wait complete. 2024-04-26T21:43:15.621 INFO:teuthology.orchestra.run.smithi072.stdout:Fetching dashboard port number... 2024-04-26T21:43:17.549 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:43:17 smithi072 ceph-mon[28138]: mgrmap e13: x(active, since 6s) 2024-04-26T21:43:17.549 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:43:17 smithi072 ceph-mon[28138]: from='client.? 172.21.15.72:0/1212095934' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-04-26T21:43:18.060 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stdout 8443 2024-04-26T21:43:18.060 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:16.219+0000 7f52fe400700 1 Processor -- start 2024-04-26T21:43:18.060 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:16.219+0000 7f52fe400700 1 -- start start 2024-04-26T21:43:18.060 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:16.219+0000 7f52fe400700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f52f8108cd0 0x7f52f81090f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:18.061 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:16.219+0000 7f52fe400700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f52f81096c0 con 0x7f52f8108cd0 2024-04-26T21:43:18.061 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:16.220+0000 7f52f7400700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f52f8108cd0 0x7f52f81090f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:18.061 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:16.220+0000 7f52f7400700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f52f8108cd0 0x7f52f81090f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:39264/0 (socket says 172.21.15.72:39264) 2024-04-26T21:43:18.061 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:16.220+0000 7f52f7400700 1 -- 172.21.15.72:0/2531881830 learned_addr learned my addr 172.21.15.72:0/2531881830 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:43:18.061 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:16.220+0000 7f52f7400700 1 -- 172.21.15.72:0/2531881830 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52f8109ed0 con 0x7f52f8108cd0 2024-04-26T21:43:18.061 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:16.221+0000 7f52f7400700 1 --2- 172.21.15.72:0/2531881830 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f52f8108cd0 0x7f52f81090f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f52e80098a0 tx=0x7f52e8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9608c141d9a18d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:18.061 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:16.221+0000 7f52f6000700 1 -- 172.21.15.72:0/2531881830 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f52e8004160 con 0x7f52f8108cd0 2024-04-26T21:43:18.061 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:16.221+0000 7f52f6000700 1 -- 172.21.15.72:0/2531881830 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f52e8004300 con 0x7f52f8108cd0 2024-04-26T21:43:18.061 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:16.221+0000 7f52f6000700 1 -- 172.21.15.72:0/2531881830 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f52e8011960 con 0x7f52f8108cd0 2024-04-26T21:43:18.061 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:16.222+0000 7f52fe400700 1 -- 172.21.15.72:0/2531881830 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f52f8108cd0 msgr2=0x7f52f81090f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:18.062 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:16.222+0000 7f52fe400700 1 --2- 172.21.15.72:0/2531881830 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f52f8108cd0 0x7f52f81090f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f52e80098a0 tx=0x7f52e8009bb0 comp rx=0 tx=0).stop 2024-04-26T21:43:18.062 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:16.222+0000 7f52fe400700 1 -- 172.21.15.72:0/2531881830 shutdown_connections 2024-04-26T21:43:18.062 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:16.222+0000 7f52fe400700 1 --2- 172.21.15.72:0/2531881830 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f52f8108cd0 0x7f52f81090f0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:18.062 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:16.222+0000 7f52fe400700 1 -- 172.21.15.72:0/2531881830 >> 172.21.15.72:0/2531881830 conn(0x7f52f8104230 msgr2=0x7f52f81066b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:18.062 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:16.222+0000 7f52fe400700 1 -- 172.21.15.72:0/2531881830 shutdown_connections 2024-04-26T21:43:18.062 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:16.222+0000 7f52fe400700 1 -- 172.21.15.72:0/2531881830 wait complete. 2024-04-26T21:43:18.062 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:16.223+0000 7f52fe400700 1 Processor -- start 2024-04-26T21:43:18.062 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:16.223+0000 7f52fe400700 1 -- start start 2024-04-26T21:43:18.062 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:16.223+0000 7f52fe400700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f52f8108cd0 0x7f52f819ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:18.062 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:16.223+0000 7f52fe400700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f52f810d100 con 0x7f52f8108cd0 2024-04-26T21:43:18.062 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:16.224+0000 7f52f7400700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f52f8108cd0 0x7f52f819ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:18.063 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:16.224+0000 7f52f7400700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f52f8108cd0 0x7f52f819ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:39274/0 (socket says 172.21.15.72:39274) 2024-04-26T21:43:18.063 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:16.224+0000 7f52f7400700 1 -- 172.21.15.72:0/1212095934 learned_addr learned my addr 172.21.15.72:0/1212095934 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:43:18.063 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:16.224+0000 7f52f7400700 1 -- 172.21.15.72:0/1212095934 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52f819f2a0 con 0x7f52f8108cd0 2024-04-26T21:43:18.063 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:16.224+0000 7f52f7400700 1 --2- 172.21.15.72:0/1212095934 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f52f8108cd0 0x7f52f819ed60 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f52e8011670 tx=0x7f52e8011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:18.063 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:16.225+0000 7f52efe00700 1 -- 172.21.15.72:0/1212095934 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f52e8003a60 con 0x7f52f8108cd0 2024-04-26T21:43:18.063 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:16.225+0000 7f52efe00700 1 -- 172.21.15.72:0/1212095934 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f52e801b8d0 con 0x7f52f8108cd0 2024-04-26T21:43:18.063 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:16.225+0000 7f52efe00700 1 -- 172.21.15.72:0/1212095934 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f52e8023790 con 0x7f52f8108cd0 2024-04-26T21:43:18.063 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:16.225+0000 7f52fe400700 1 -- 172.21.15.72:0/1212095934 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52f819f530 con 0x7f52f8108cd0 2024-04-26T21:43:18.063 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:16.225+0000 7f52fe400700 1 -- 172.21.15.72:0/1212095934 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52f81a2220 con 0x7f52f8108cd0 2024-04-26T21:43:18.063 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:16.226+0000 7f52efe00700 1 -- 172.21.15.72:0/1212095934 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 12) ==== 46571+0+0 (secure 0 0 0) 0x7f52e8023a40 con 0x7f52f8108cd0 2024-04-26T21:43:18.063 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:16.226+0000 7f52fe400700 1 -- 172.21.15.72:0/1212095934 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52f8064a80 con 0x7f52f8108cd0 2024-04-26T21:43:18.063 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:16.226+0000 7f52efe00700 1 --2- 172.21.15.72:0/1212095934 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f52dc039660 0x7f52dc03bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:18.064 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:16.226+0000 7f52efe00700 1 -- 172.21.15.72:0/1212095934 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f52e804e2b0 con 0x7f52f8108cd0 2024-04-26T21:43:18.064 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:16.227+0000 7f52f6a00700 1 --2- 172.21.15.72:0/1212095934 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f52dc039660 0x7f52dc03bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:18.064 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:16.227+0000 7f52f6a00700 1 --2- 172.21.15.72:0/1212095934 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f52dc039660 0x7f52dc03bb20 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f52e4006fd0 tx=0x7f52e4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:18.064 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:16.232+0000 7f52efe00700 1 -- 172.21.15.72:0/1212095934 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f52e801f030 con 0x7f52f8108cd0 2024-04-26T21:43:18.064 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:16.358+0000 7f52efe00700 1 -- 172.21.15.72:0/1212095934 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f52e802c910 con 0x7f52f8108cd0 2024-04-26T21:43:18.064 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:16.466+0000 7f52fe400700 1 -- 172.21.15.72:0/1212095934 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7f52f81a27d0 con 0x7f52f8108cd0 2024-04-26T21:43:18.064 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:16.466+0000 7f52efe00700 1 -- 172.21.15.72:0/1212095934 <== mon.0 v2:172.21.15.72:3300/0 8 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7f52e801bda0 con 0x7f52f8108cd0 2024-04-26T21:43:18.064 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:16.469+0000 7f52fe400700 1 -- 172.21.15.72:0/1212095934 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f52dc039660 msgr2=0x7f52dc03bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:18.064 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:16.469+0000 7f52fe400700 1 --2- 172.21.15.72:0/1212095934 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f52dc039660 0x7f52dc03bb20 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f52e4006fd0 tx=0x7f52e4006e40 comp rx=0 tx=0).stop 2024-04-26T21:43:18.064 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:16.469+0000 7f52fe400700 1 -- 172.21.15.72:0/1212095934 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f52f8108cd0 msgr2=0x7f52f819ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:18.064 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:16.469+0000 7f52fe400700 1 --2- 172.21.15.72:0/1212095934 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f52f8108cd0 0x7f52f819ed60 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f52e8011670 tx=0x7f52e8011a30 comp rx=0 tx=0).stop 2024-04-26T21:43:18.065 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:16.469+0000 7f52fe400700 1 -- 172.21.15.72:0/1212095934 shutdown_connections 2024-04-26T21:43:18.065 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:16.469+0000 7f52fe400700 1 --2- 172.21.15.72:0/1212095934 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f52dc039660 0x7f52dc03bb20 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:18.065 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:16.469+0000 7f52fe400700 1 --2- 172.21.15.72:0/1212095934 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f52f8108cd0 0x7f52f819ed60 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:18.065 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:16.469+0000 7f52fe400700 1 -- 172.21.15.72:0/1212095934 >> 172.21.15.72:0/1212095934 conn(0x7f52f8104230 msgr2=0x7f52f8104d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:18.065 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:16.469+0000 7f52fe400700 1 -- 172.21.15.72:0/1212095934 shutdown_connections 2024-04-26T21:43:18.065 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:16.469+0000 7f52fe400700 1 -- 172.21.15.72:0/1212095934 wait complete. 2024-04-26T21:43:18.085 INFO:teuthology.orchestra.run.smithi072.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-26T21:43:18.085 INFO:teuthology.orchestra.run.smithi072.stdout:systemctl: stdout disabled 2024-04-26T21:43:18.107 INFO:teuthology.orchestra.run.smithi072.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-26T21:43:18.107 INFO:teuthology.orchestra.run.smithi072.stdout:systemctl: stdout inactive 2024-04-26T21:43:18.108 INFO:teuthology.orchestra.run.smithi072.stdout:firewalld.service is not enabled 2024-04-26T21:43:18.108 INFO:teuthology.orchestra.run.smithi072.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-04-26T21:43:18.114 INFO:teuthology.orchestra.run.smithi072.stdout:Ceph Dashboard is now available at: 2024-04-26T21:43:18.115 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T21:43:18.115 INFO:teuthology.orchestra.run.smithi072.stdout: URL: https://smithi072.front.sepia.ceph.com:8443/ 2024-04-26T21:43:18.115 INFO:teuthology.orchestra.run.smithi072.stdout: User: admin 2024-04-26T21:43:18.115 INFO:teuthology.orchestra.run.smithi072.stdout: Password: kcwjwumpjw 2024-04-26T21:43:18.115 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T21:43:18.116 INFO:teuthology.orchestra.run.smithi072.stdout:Saving cluster configuration to /var/lib/ceph/a1780950-0415-11ef-bc93-c7b262605968/config directory 2024-04-26T21:43:18.118 INFO:teuthology.orchestra.run.smithi072.stdout:Enabling autotune for osd_memory_target 2024-04-26T21:43:19.712 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:18.736+0000 7f2e86c00700 1 Processor -- start 2024-04-26T21:43:19.712 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:18.736+0000 7f2e86c00700 1 -- start start 2024-04-26T21:43:19.712 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:18.736+0000 7f2e86c00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2e80108c80 0x7f2e801090a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:19.712 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:18.736+0000 7f2e86c00700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f2e80109670 con 0x7f2e80108c80 2024-04-26T21:43:19.712 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:18.737+0000 7f2e7fe00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2e80108c80 0x7f2e801090a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:19.712 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:18.737+0000 7f2e7fe00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2e80108c80 0x7f2e801090a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:39282/0 (socket says 172.21.15.72:39282) 2024-04-26T21:43:19.712 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:18.737+0000 7f2e7fe00700 1 -- 172.21.15.72:0/2434887841 learned_addr learned my addr 172.21.15.72:0/2434887841 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:43:19.712 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:18.737+0000 7f2e7fe00700 1 -- 172.21.15.72:0/2434887841 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e80109e80 con 0x7f2e80108c80 2024-04-26T21:43:19.712 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:18.737+0000 7f2e7fe00700 1 --2- 172.21.15.72:0/2434887841 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2e80108c80 0x7f2e801090a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f2e700098a0 tx=0x7f2e70009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4439de4d557d793 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:19.712 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:18.738+0000 7f2e7ea00700 1 -- 172.21.15.72:0/2434887841 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2e70004160 con 0x7f2e80108c80 2024-04-26T21:43:19.717 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:18.738+0000 7f2e7ea00700 1 -- 172.21.15.72:0/2434887841 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2e70004300 con 0x7f2e80108c80 2024-04-26T21:43:19.717 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:18.738+0000 7f2e7ea00700 1 -- 172.21.15.72:0/2434887841 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2e70011960 con 0x7f2e80108c80 2024-04-26T21:43:19.717 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:18.738+0000 7f2e86c00700 1 -- 172.21.15.72:0/2434887841 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2e80108c80 msgr2=0x7f2e801090a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:19.717 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:18.738+0000 7f2e86c00700 1 --2- 172.21.15.72:0/2434887841 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2e80108c80 0x7f2e801090a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f2e700098a0 tx=0x7f2e70009bb0 comp rx=0 tx=0).stop 2024-04-26T21:43:19.717 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:18.738+0000 7f2e86c00700 1 -- 172.21.15.72:0/2434887841 shutdown_connections 2024-04-26T21:43:19.717 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:18.739+0000 7f2e86c00700 1 --2- 172.21.15.72:0/2434887841 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2e80108c80 0x7f2e801090a0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:19.718 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:18.739+0000 7f2e86c00700 1 -- 172.21.15.72:0/2434887841 >> 172.21.15.72:0/2434887841 conn(0x7f2e80104240 msgr2=0x7f2e80106660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:19.718 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:18.739+0000 7f2e86c00700 1 -- 172.21.15.72:0/2434887841 shutdown_connections 2024-04-26T21:43:19.718 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:18.739+0000 7f2e86c00700 1 -- 172.21.15.72:0/2434887841 wait complete. 2024-04-26T21:43:19.718 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:18.740+0000 7f2e86c00700 1 Processor -- start 2024-04-26T21:43:19.718 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:18.740+0000 7f2e86c00700 1 -- start start 2024-04-26T21:43:19.718 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:18.740+0000 7f2e86c00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2e80108c80 0x7f2e8019cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:19.718 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:18.740+0000 7f2e86c00700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f2e8010d0b0 con 0x7f2e80108c80 2024-04-26T21:43:19.718 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:18.740+0000 7f2e7fe00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2e80108c80 0x7f2e8019cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:19.718 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:18.740+0000 7f2e7fe00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2e80108c80 0x7f2e8019cb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:39284/0 (socket says 172.21.15.72:39284) 2024-04-26T21:43:19.718 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:18.740+0000 7f2e7fe00700 1 -- 172.21.15.72:0/587323912 learned_addr learned my addr 172.21.15.72:0/587323912 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:43:19.718 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:18.741+0000 7f2e7fe00700 1 -- 172.21.15.72:0/587323912 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e8019d070 con 0x7f2e80108c80 2024-04-26T21:43:19.719 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:18.741+0000 7f2e7fe00700 1 --2- 172.21.15.72:0/587323912 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2e80108c80 0x7f2e8019cb30 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f2e70011670 tx=0x7f2e70011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:19.719 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:18.741+0000 7f2e7cc00700 1 -- 172.21.15.72:0/587323912 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2e7001f070 con 0x7f2e80108c80 2024-04-26T21:43:19.719 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:18.741+0000 7f2e7cc00700 1 -- 172.21.15.72:0/587323912 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2e70003b20 con 0x7f2e80108c80 2024-04-26T21:43:19.719 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:18.741+0000 7f2e7cc00700 1 -- 172.21.15.72:0/587323912 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2e70019580 con 0x7f2e80108c80 2024-04-26T21:43:19.719 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:18.741+0000 7f2e86c00700 1 -- 172.21.15.72:0/587323912 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e8019d300 con 0x7f2e80108c80 2024-04-26T21:43:19.719 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:18.742+0000 7f2e86c00700 1 -- 172.21.15.72:0/587323912 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e8019fff0 con 0x7f2e80108c80 2024-04-26T21:43:19.719 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:18.743+0000 7f2e86c00700 1 -- 172.21.15.72:0/587323912 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e8010d230 con 0x7f2e80108c80 2024-04-26T21:43:19.719 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:18.743+0000 7f2e7cc00700 1 -- 172.21.15.72:0/587323912 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f2e70022070 con 0x7f2e80108c80 2024-04-26T21:43:19.719 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:18.743+0000 7f2e7cc00700 1 --2- 172.21.15.72:0/587323912 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f2e64039700 0x7f2e6403bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:19.719 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:18.743+0000 7f2e7cc00700 1 -- 172.21.15.72:0/587323912 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f2e7004dd30 con 0x7f2e80108c80 2024-04-26T21:43:19.719 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:18.744+0000 7f2e7f400700 1 --2- 172.21.15.72:0/587323912 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f2e64039700 0x7f2e6403bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:19.719 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:18.744+0000 7f2e7f400700 1 --2- 172.21.15.72:0/587323912 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f2e64039700 0x7f2e6403bbc0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f2e6c006fd0 tx=0x7f2e6c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:19.719 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:18.749+0000 7f2e7cc00700 1 -- 172.21.15.72:0/587323912 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f2e70018350 con 0x7f2e80108c80 2024-04-26T21:43:19.719 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:18.974+0000 7f2e86c00700 1 -- 172.21.15.72:0/587323912 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) -- 0x7f2e80064a80 con 0x7f2e80108c80 2024-04-26T21:43:19.719 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:18.975+0000 7f2e7cc00700 1 -- 172.21.15.72:0/587323912 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) ==== 127+0+0 (secure 0 0 0) 0x7f2e7001b440 con 0x7f2e80108c80 2024-04-26T21:43:19.720 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:18.978+0000 7f2e86c00700 1 -- 172.21.15.72:0/587323912 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f2e64039700 msgr2=0x7f2e6403bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:19.720 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:18.978+0000 7f2e86c00700 1 --2- 172.21.15.72:0/587323912 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f2e64039700 0x7f2e6403bbc0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f2e6c006fd0 tx=0x7f2e6c006e40 comp rx=0 tx=0).stop 2024-04-26T21:43:19.720 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:18.978+0000 7f2e86c00700 1 -- 172.21.15.72:0/587323912 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2e80108c80 msgr2=0x7f2e8019cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:19.720 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:18.978+0000 7f2e86c00700 1 --2- 172.21.15.72:0/587323912 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2e80108c80 0x7f2e8019cb30 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f2e70011670 tx=0x7f2e70011a30 comp rx=0 tx=0).stop 2024-04-26T21:43:19.720 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:18.978+0000 7f2e86c00700 1 -- 172.21.15.72:0/587323912 shutdown_connections 2024-04-26T21:43:19.720 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:18.978+0000 7f2e86c00700 1 --2- 172.21.15.72:0/587323912 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f2e64039700 0x7f2e6403bbc0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:19.720 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:18.978+0000 7f2e86c00700 1 --2- 172.21.15.72:0/587323912 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2e80108c80 0x7f2e8019cb30 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:19.720 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:18.978+0000 7f2e86c00700 1 -- 172.21.15.72:0/587323912 >> 172.21.15.72:0/587323912 conn(0x7f2e80104240 msgr2=0x7f2e80104d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:19.720 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:18.978+0000 7f2e86c00700 1 -- 172.21.15.72:0/587323912 shutdown_connections 2024-04-26T21:43:19.720 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:18.978+0000 7f2e86c00700 1 -- 172.21.15.72:0/587323912 wait complete. 2024-04-26T21:43:21.987 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:43:21 smithi072 ceph-mon[28138]: from='client.? 172.21.15.72:0/1608226534' entity='client.admin' 2024-04-26T21:43:21.996 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:20.329+0000 7f7252800700 1 Processor -- start 2024-04-26T21:43:21.997 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:20.329+0000 7f7252800700 1 -- start start 2024-04-26T21:43:21.997 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:20.329+0000 7f7252800700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f724c10aca0 0x7f724c10b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:21.997 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:20.330+0000 7f7252800700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f724c10b690 con 0x7f724c10aca0 2024-04-26T21:43:21.997 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:20.330+0000 7f7251400700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f724c10aca0 0x7f724c10b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:21.998 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:20.330+0000 7f7251400700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f724c10aca0 0x7f724c10b0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:39298/0 (socket says 172.21.15.72:39298) 2024-04-26T21:43:21.998 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:20.330+0000 7f7251400700 1 -- 172.21.15.72:0/2517088991 learned_addr learned my addr 172.21.15.72:0/2517088991 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:43:21.998 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:20.330+0000 7f7251400700 1 -- 172.21.15.72:0/2517088991 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f724c10bea0 con 0x7f724c10aca0 2024-04-26T21:43:21.999 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:20.331+0000 7f7251400700 1 --2- 172.21.15.72:0/2517088991 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f724c10aca0 0x7f724c10b0c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f7240009ab0 tx=0x7f7240009dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3be322250a218e6c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:21.999 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:20.331+0000 7f724be00700 1 -- 172.21.15.72:0/2517088991 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7240004160 con 0x7f724c10aca0 2024-04-26T21:43:21.999 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:20.331+0000 7f724be00700 1 -- 172.21.15.72:0/2517088991 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f724000bc70 con 0x7f724c10aca0 2024-04-26T21:43:21.999 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:20.331+0000 7f724be00700 1 -- 172.21.15.72:0/2517088991 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7240003b70 con 0x7f724c10aca0 2024-04-26T21:43:22.000 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:20.331+0000 7f7252800700 1 -- 172.21.15.72:0/2517088991 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f724c10aca0 msgr2=0x7f724c10b0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:22.001 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:20.331+0000 7f7252800700 1 --2- 172.21.15.72:0/2517088991 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f724c10aca0 0x7f724c10b0c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f7240009ab0 tx=0x7f7240009dc0 comp rx=0 tx=0).stop 2024-04-26T21:43:22.001 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:20.332+0000 7f7252800700 1 -- 172.21.15.72:0/2517088991 shutdown_connections 2024-04-26T21:43:22.001 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:20.332+0000 7f7252800700 1 --2- 172.21.15.72:0/2517088991 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f724c10aca0 0x7f724c10b0c0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:22.001 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:20.332+0000 7f7252800700 1 -- 172.21.15.72:0/2517088991 >> 172.21.15.72:0/2517088991 conn(0x7f724c07e090 msgr2=0x7f724c07e4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:22.001 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:20.332+0000 7f7252800700 1 -- 172.21.15.72:0/2517088991 shutdown_connections 2024-04-26T21:43:22.001 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:20.332+0000 7f7252800700 1 -- 172.21.15.72:0/2517088991 wait complete. 2024-04-26T21:43:22.001 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:20.333+0000 7f7252800700 1 Processor -- start 2024-04-26T21:43:22.001 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:20.333+0000 7f7252800700 1 -- start start 2024-04-26T21:43:22.001 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:20.333+0000 7f7252800700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f724c10aca0 0x7f724c19ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:22.001 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:20.333+0000 7f7252800700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f724c10f0d0 con 0x7f724c10aca0 2024-04-26T21:43:22.001 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:20.334+0000 7f7251400700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f724c10aca0 0x7f724c19ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:22.001 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:20.334+0000 7f7251400700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f724c10aca0 0x7f724c19ec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:39304/0 (socket says 172.21.15.72:39304) 2024-04-26T21:43:22.001 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:20.334+0000 7f7251400700 1 -- 172.21.15.72:0/1608226534 learned_addr learned my addr 172.21.15.72:0/1608226534 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:43:22.002 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:20.334+0000 7f7251400700 1 -- 172.21.15.72:0/1608226534 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f724c19f140 con 0x7f724c10aca0 2024-04-26T21:43:22.005 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:20.334+0000 7f7251400700 1 --2- 172.21.15.72:0/1608226534 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f724c10aca0 0x7f724c19ec00 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f72400038b0 tx=0x7f724000b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:22.005 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:20.335+0000 7f724a000700 1 -- 172.21.15.72:0/1608226534 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7240003ff0 con 0x7f724c10aca0 2024-04-26T21:43:22.005 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:20.335+0000 7f724a000700 1 -- 172.21.15.72:0/1608226534 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f724001a430 con 0x7f724c10aca0 2024-04-26T21:43:22.005 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:20.335+0000 7f7252800700 1 -- 172.21.15.72:0/1608226534 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f724c19f3d0 con 0x7f724c10aca0 2024-04-26T21:43:22.005 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:20.335+0000 7f724a000700 1 -- 172.21.15.72:0/1608226534 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f72400115b0 con 0x7f724c10aca0 2024-04-26T21:43:22.005 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:20.335+0000 7f7252800700 1 -- 172.21.15.72:0/1608226534 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f724c1a20c0 con 0x7f724c10aca0 2024-04-26T21:43:22.005 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:20.336+0000 7f7252800700 1 -- 172.21.15.72:0/1608226534 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f724c10f250 con 0x7f724c10aca0 2024-04-26T21:43:22.005 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:20.336+0000 7f724a000700 1 -- 172.21.15.72:0/1608226534 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f724001a8e0 con 0x7f724c10aca0 2024-04-26T21:43:22.005 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:20.337+0000 7f724a000700 1 --2- 172.21.15.72:0/1608226534 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f72340396a0 0x7f723403bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:22.006 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:20.337+0000 7f724a000700 1 -- 172.21.15.72:0/1608226534 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f724004db70 con 0x7f724c10aca0 2024-04-26T21:43:22.006 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:20.337+0000 7f7250a00700 1 --2- 172.21.15.72:0/1608226534 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f72340396a0 0x7f723403bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:22.006 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:20.338+0000 7f7250a00700 1 --2- 172.21.15.72:0/1608226534 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f72340396a0 0x7f723403bb60 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f7238006fd0 tx=0x7f7238006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:22.006 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:20.342+0000 7f724a000700 1 -- 172.21.15.72:0/1608226534 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f724001a690 con 0x7f724c10aca0 2024-04-26T21:43:22.006 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:20.682+0000 7f7252800700 1 -- 172.21.15.72:0/1608226534 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7f724c1a2290 con 0x7f724c10aca0 2024-04-26T21:43:22.006 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:20.688+0000 7f724a000700 1 -- 172.21.15.72:0/1608226534 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) ==== 153+0+0 (secure 0 0 0) 0x7f724000fda0 con 0x7f724c10aca0 2024-04-26T21:43:22.006 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:20.690+0000 7f7252800700 1 -- 172.21.15.72:0/1608226534 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f72340396a0 msgr2=0x7f723403bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:22.006 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:20.690+0000 7f7252800700 1 --2- 172.21.15.72:0/1608226534 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f72340396a0 0x7f723403bb60 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f7238006fd0 tx=0x7f7238006e40 comp rx=0 tx=0).stop 2024-04-26T21:43:22.006 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:20.691+0000 7f7252800700 1 -- 172.21.15.72:0/1608226534 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f724c10aca0 msgr2=0x7f724c19ec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:22.006 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:20.691+0000 7f7252800700 1 --2- 172.21.15.72:0/1608226534 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f724c10aca0 0x7f724c19ec00 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f72400038b0 tx=0x7f724000b890 comp rx=0 tx=0).stop 2024-04-26T21:43:22.006 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:20.691+0000 7f7252800700 1 -- 172.21.15.72:0/1608226534 shutdown_connections 2024-04-26T21:43:22.006 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:20.691+0000 7f7252800700 1 --2- 172.21.15.72:0/1608226534 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f72340396a0 0x7f723403bb60 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:22.007 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:20.691+0000 7f7252800700 1 --2- 172.21.15.72:0/1608226534 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f724c10aca0 0x7f724c19ec00 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:22.007 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:20.691+0000 7f7252800700 1 -- 172.21.15.72:0/1608226534 >> 172.21.15.72:0/1608226534 conn(0x7f724c07e090 msgr2=0x7f724c108490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:22.007 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:20.691+0000 7f7252800700 1 -- 172.21.15.72:0/1608226534 shutdown_connections 2024-04-26T21:43:22.007 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:20.691+0000 7f7252800700 1 -- 172.21.15.72:0/1608226534 wait complete. 2024-04-26T21:43:22.007 INFO:teuthology.orchestra.run.smithi072.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-04-26T21:43:22.007 INFO:teuthology.orchestra.run.smithi072.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-04-26T21:43:22.007 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T21:43:22.007 INFO:teuthology.orchestra.run.smithi072.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid a1780950-0415-11ef-bc93-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-04-26T21:43:22.007 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T21:43:22.007 INFO:teuthology.orchestra.run.smithi072.stdout:Or, if you are only running a single cluster on this host: 2024-04-26T21:43:22.007 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T21:43:22.007 INFO:teuthology.orchestra.run.smithi072.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-04-26T21:43:22.007 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T21:43:22.007 INFO:teuthology.orchestra.run.smithi072.stdout:Please consider enabling telemetry to help improve Ceph: 2024-04-26T21:43:22.008 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T21:43:22.008 INFO:teuthology.orchestra.run.smithi072.stdout: ceph telemetry on 2024-04-26T21:43:22.008 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T21:43:22.008 INFO:teuthology.orchestra.run.smithi072.stdout:For more information see: 2024-04-26T21:43:22.008 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T21:43:22.008 INFO:teuthology.orchestra.run.smithi072.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-04-26T21:43:22.008 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T21:43:22.008 INFO:teuthology.orchestra.run.smithi072.stdout:Bootstrap complete. 2024-04-26T21:43:22.115 INFO:tasks.cephadm:Fetching config... 2024-04-26T21:43:22.115 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2024-04-26T21:43:22.115 DEBUG:teuthology.orchestra.run.smithi072:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-04-26T21:43:22.208 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-04-26T21:43:22.208 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2024-04-26T21:43:22.208 DEBUG:teuthology.orchestra.run.smithi072:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-04-26T21:43:22.249 INFO:tasks.cephadm:Fetching mon keyring... 2024-04-26T21:43:22.249 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2024-04-26T21:43:22.249 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd if=/var/lib/ceph/a1780950-0415-11ef-bc93-c7b262605968/mon.a/keyring of=/dev/stdout 2024-04-26T21:43:22.366 INFO:tasks.cephadm:Fetching pub ssh key... 2024-04-26T21:43:22.366 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2024-04-26T21:43:22.366 DEBUG:teuthology.orchestra.run.smithi072:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-04-26T21:43:22.462 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-04-26T21:43:22.463 DEBUG:teuthology.orchestra.run.smithi072:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDBwVwmjYjtJrIQXFLnz1GGuT/um9Uys5TnoD6QESzyxrCzFZmQK7I+tbp7X2qEDf1XFODVrovfs8F+KS1Ycbr2J4pORAZLRE514yl2rW2AXDqx9k5Mkw+nos14kJdoW/ulRAf0eTR6kyIAsod6K+M6t0pCohYbP75FcL2ZEdsSK/K2LMRK/lXKrGupuGrg+sFjfYrYyTIOQOHL7mum3FDQYygC1xZtj+Khcv1Yf8KBhZU7PaTDsO6OiThdJMJfePMQE+3IcqczRjWFJ1e+ik0r5tB0JqWjpxiRcG0gxZs48yZNzhJFU6fxqkFeaSM5NtHfBE0eVuNpYUw7qgLjK8pqhpaKjewI/oWHqs821p/k31G+Cp0quEZJQHvmKVl5pUHUoef3F3c+XGGYoRnOZy7KxSt5FOh6CbNQYqLz4i51vxlGS+4MOc5odatBBFIuJh0HhtdyVxknqemAR9e9o/JuU39xn+wDD1G+7VB7U3CkrHt2NI+ExbwDK70ginPIr2M= ceph-a1780950-0415-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-26T21:43:22.574 INFO:teuthology.orchestra.run.smithi072.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDBwVwmjYjtJrIQXFLnz1GGuT/um9Uys5TnoD6QESzyxrCzFZmQK7I+tbp7X2qEDf1XFODVrovfs8F+KS1Ycbr2J4pORAZLRE514yl2rW2AXDqx9k5Mkw+nos14kJdoW/ulRAf0eTR6kyIAsod6K+M6t0pCohYbP75FcL2ZEdsSK/K2LMRK/lXKrGupuGrg+sFjfYrYyTIOQOHL7mum3FDQYygC1xZtj+Khcv1Yf8KBhZU7PaTDsO6OiThdJMJfePMQE+3IcqczRjWFJ1e+ik0r5tB0JqWjpxiRcG0gxZs48yZNzhJFU6fxqkFeaSM5NtHfBE0eVuNpYUw7qgLjK8pqhpaKjewI/oWHqs821p/k31G+Cp0quEZJQHvmKVl5pUHUoef3F3c+XGGYoRnOZy7KxSt5FOh6CbNQYqLz4i51vxlGS+4MOc5odatBBFIuJh0HhtdyVxknqemAR9e9o/JuU39xn+wDD1G+7VB7U3CkrHt2NI+ExbwDK70ginPIr2M= ceph-a1780950-0415-11ef-bc93-c7b262605968 2024-04-26T21:43:22.628 DEBUG:teuthology.orchestra.run.smithi076:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDBwVwmjYjtJrIQXFLnz1GGuT/um9Uys5TnoD6QESzyxrCzFZmQK7I+tbp7X2qEDf1XFODVrovfs8F+KS1Ycbr2J4pORAZLRE514yl2rW2AXDqx9k5Mkw+nos14kJdoW/ulRAf0eTR6kyIAsod6K+M6t0pCohYbP75FcL2ZEdsSK/K2LMRK/lXKrGupuGrg+sFjfYrYyTIOQOHL7mum3FDQYygC1xZtj+Khcv1Yf8KBhZU7PaTDsO6OiThdJMJfePMQE+3IcqczRjWFJ1e+ik0r5tB0JqWjpxiRcG0gxZs48yZNzhJFU6fxqkFeaSM5NtHfBE0eVuNpYUw7qgLjK8pqhpaKjewI/oWHqs821p/k31G+Cp0quEZJQHvmKVl5pUHUoef3F3c+XGGYoRnOZy7KxSt5FOh6CbNQYqLz4i51vxlGS+4MOc5odatBBFIuJh0HhtdyVxknqemAR9e9o/JuU39xn+wDD1G+7VB7U3CkrHt2NI+ExbwDK70ginPIr2M= ceph-a1780950-0415-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-26T21:43:22.758 INFO:teuthology.orchestra.run.smithi076.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDBwVwmjYjtJrIQXFLnz1GGuT/um9Uys5TnoD6QESzyxrCzFZmQK7I+tbp7X2qEDf1XFODVrovfs8F+KS1Ycbr2J4pORAZLRE514yl2rW2AXDqx9k5Mkw+nos14kJdoW/ulRAf0eTR6kyIAsod6K+M6t0pCohYbP75FcL2ZEdsSK/K2LMRK/lXKrGupuGrg+sFjfYrYyTIOQOHL7mum3FDQYygC1xZtj+Khcv1Yf8KBhZU7PaTDsO6OiThdJMJfePMQE+3IcqczRjWFJ1e+ik0r5tB0JqWjpxiRcG0gxZs48yZNzhJFU6fxqkFeaSM5NtHfBE0eVuNpYUw7qgLjK8pqhpaKjewI/oWHqs821p/k31G+Cp0quEZJQHvmKVl5pUHUoef3F3c+XGGYoRnOZy7KxSt5FOh6CbNQYqLz4i51vxlGS+4MOc5odatBBFIuJh0HhtdyVxknqemAR9e9o/JuU39xn+wDD1G+7VB7U3CkrHt2NI+ExbwDK70ginPIr2M= ceph-a1780950-0415-11ef-bc93-c7b262605968 2024-04-26T21:43:22.818 DEBUG:teuthology.orchestra.run.smithi164:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDBwVwmjYjtJrIQXFLnz1GGuT/um9Uys5TnoD6QESzyxrCzFZmQK7I+tbp7X2qEDf1XFODVrovfs8F+KS1Ycbr2J4pORAZLRE514yl2rW2AXDqx9k5Mkw+nos14kJdoW/ulRAf0eTR6kyIAsod6K+M6t0pCohYbP75FcL2ZEdsSK/K2LMRK/lXKrGupuGrg+sFjfYrYyTIOQOHL7mum3FDQYygC1xZtj+Khcv1Yf8KBhZU7PaTDsO6OiThdJMJfePMQE+3IcqczRjWFJ1e+ik0r5tB0JqWjpxiRcG0gxZs48yZNzhJFU6fxqkFeaSM5NtHfBE0eVuNpYUw7qgLjK8pqhpaKjewI/oWHqs821p/k31G+Cp0quEZJQHvmKVl5pUHUoef3F3c+XGGYoRnOZy7KxSt5FOh6CbNQYqLz4i51vxlGS+4MOc5odatBBFIuJh0HhtdyVxknqemAR9e9o/JuU39xn+wDD1G+7VB7U3CkrHt2NI+ExbwDK70ginPIr2M= ceph-a1780950-0415-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-26T21:43:22.946 INFO:teuthology.orchestra.run.smithi164.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDBwVwmjYjtJrIQXFLnz1GGuT/um9Uys5TnoD6QESzyxrCzFZmQK7I+tbp7X2qEDf1XFODVrovfs8F+KS1Ycbr2J4pORAZLRE514yl2rW2AXDqx9k5Mkw+nos14kJdoW/ulRAf0eTR6kyIAsod6K+M6t0pCohYbP75FcL2ZEdsSK/K2LMRK/lXKrGupuGrg+sFjfYrYyTIOQOHL7mum3FDQYygC1xZtj+Khcv1Yf8KBhZU7PaTDsO6OiThdJMJfePMQE+3IcqczRjWFJ1e+ik0r5tB0JqWjpxiRcG0gxZs48yZNzhJFU6fxqkFeaSM5NtHfBE0eVuNpYUw7qgLjK8pqhpaKjewI/oWHqs821p/k31G+Cp0quEZJQHvmKVl5pUHUoef3F3c+XGGYoRnOZy7KxSt5FOh6CbNQYqLz4i51vxlGS+4MOc5odatBBFIuJh0HhtdyVxknqemAR9e9o/JuU39xn+wDD1G+7VB7U3CkrHt2NI+ExbwDK70ginPIr2M= ceph-a1780950-0415-11ef-bc93-c7b262605968 2024-04-26T21:43:23.001 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a1780950-0415-11ef-bc93-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-04-26T21:43:25.773 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/a1780950-0415-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:43:28.853 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:28.851+0000 7f5d84c00700 1 -- 172.21.15.72:0/3788076560 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5d80077450 msgr2=0x7f5d800758b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:28.853 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:28.851+0000 7f5d84c00700 1 --2- 172.21.15.72:0/3788076560 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5d80077450 0x7f5d800758b0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f5d70009910 tx=0x7f5d70009c20 comp rx=0 tx=0).stop 2024-04-26T21:43:28.853 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:28.851+0000 7f5d84c00700 1 -- 172.21.15.72:0/3788076560 shutdown_connections 2024-04-26T21:43:28.853 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:28.851+0000 7f5d84c00700 1 --2- 172.21.15.72:0/3788076560 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5d80077450 0x7f5d800758b0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:28.853 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:28.851+0000 7f5d84c00700 1 -- 172.21.15.72:0/3788076560 >> 172.21.15.72:0/3788076560 conn(0x7f5d800fe420 msgr2=0x7f5d80100860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:28.853 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:28.851+0000 7f5d84c00700 1 -- 172.21.15.72:0/3788076560 shutdown_connections 2024-04-26T21:43:28.853 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:28.851+0000 7f5d84c00700 1 -- 172.21.15.72:0/3788076560 wait complete. 2024-04-26T21:43:28.854 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:28.852+0000 7f5d84c00700 1 Processor -- start 2024-04-26T21:43:28.854 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:28.853+0000 7f5d84c00700 1 -- start start 2024-04-26T21:43:28.854 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:28.853+0000 7f5d84c00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5d80077450 0x7f5d8019e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:28.854 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:28.853+0000 7f5d84c00700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f5d80106a60 con 0x7f5d80077450 2024-04-26T21:43:28.855 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:28.853+0000 7f5d77e00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5d80077450 0x7f5d8019e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:28.855 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:28.853+0000 7f5d77e00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5d80077450 0x7f5d8019e9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:50526/0 (socket says 172.21.15.72:50526) 2024-04-26T21:43:28.855 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:28.853+0000 7f5d77e00700 1 -- 172.21.15.72:0/3427022756 learned_addr learned my addr 172.21.15.72:0/3427022756 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:43:28.855 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:28.854+0000 7f5d77e00700 1 -- 172.21.15.72:0/3427022756 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d8019ef30 con 0x7f5d80077450 2024-04-26T21:43:28.855 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:28.854+0000 7f5d77e00700 1 --2- 172.21.15.72:0/3427022756 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5d80077450 0x7f5d8019e9f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f5d70005790 tx=0x7f5d700057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:28.855 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:28.854+0000 7f5d76a00700 1 -- 172.21.15.72:0/3427022756 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5d70027070 con 0x7f5d80077450 2024-04-26T21:43:28.855 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:28.854+0000 7f5d76a00700 1 -- 172.21.15.72:0/3427022756 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5d7000bdf0 con 0x7f5d80077450 2024-04-26T21:43:28.855 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:28.854+0000 7f5d84c00700 1 -- 172.21.15.72:0/3427022756 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d8019f1c0 con 0x7f5d80077450 2024-04-26T21:43:28.856 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:28.854+0000 7f5d76a00700 1 -- 172.21.15.72:0/3427022756 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5d7000fe50 con 0x7f5d80077450 2024-04-26T21:43:28.856 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:28.855+0000 7f5d84c00700 1 -- 172.21.15.72:0/3427022756 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d801a1eb0 con 0x7f5d80077450 2024-04-26T21:43:28.856 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:28.856+0000 7f5d84c00700 1 -- 172.21.15.72:0/3427022756 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d8010f9f0 con 0x7f5d80077450 2024-04-26T21:43:28.856 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:28.856+0000 7f5d76a00700 1 -- 172.21.15.72:0/3427022756 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f5d7002a070 con 0x7f5d80077450 2024-04-26T21:43:28.857 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:28.856+0000 7f5d76a00700 1 --2- 172.21.15.72:0/3427022756 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f5d640397f0 0x7f5d6403bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:28.857 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:28.857+0000 7f5d76a00700 1 -- 172.21.15.72:0/3427022756 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f5d7004daa0 con 0x7f5d80077450 2024-04-26T21:43:28.858 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:28.857+0000 7f5d7de00700 1 --2- 172.21.15.72:0/3427022756 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f5d640397f0 0x7f5d6403bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:28.858 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:28.858+0000 7f5d7de00700 1 --2- 172.21.15.72:0/3427022756 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f5d640397f0 0x7f5d6403bcb0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f5d68006fd0 tx=0x7f5d68006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:28.866 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:28.863+0000 7f5d76a00700 1 -- 172.21.15.72:0/3427022756 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f5d70013070 con 0x7f5d80077450 2024-04-26T21:43:29.092 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:29.091+0000 7f5d84c00700 1 -- 172.21.15.72:0/3427022756 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7f5d8002e5a0 con 0x7f5d80077450 2024-04-26T21:43:29.095 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:29.095+0000 7f5d76a00700 1 -- 172.21.15.72:0/3427022756 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7f5d700292c0 con 0x7f5d80077450 2024-04-26T21:43:29.099 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:29.098+0000 7f5d84c00700 1 -- 172.21.15.72:0/3427022756 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f5d640397f0 msgr2=0x7f5d6403bcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:29.099 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:29.098+0000 7f5d84c00700 1 --2- 172.21.15.72:0/3427022756 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f5d640397f0 0x7f5d6403bcb0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f5d68006fd0 tx=0x7f5d68006e40 comp rx=0 tx=0).stop 2024-04-26T21:43:29.099 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:29.098+0000 7f5d84c00700 1 -- 172.21.15.72:0/3427022756 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5d80077450 msgr2=0x7f5d8019e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:29.099 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:29.098+0000 7f5d84c00700 1 --2- 172.21.15.72:0/3427022756 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5d80077450 0x7f5d8019e9f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f5d70005790 tx=0x7f5d700057c0 comp rx=0 tx=0).stop 2024-04-26T21:43:29.099 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:29.098+0000 7f5d84c00700 1 -- 172.21.15.72:0/3427022756 shutdown_connections 2024-04-26T21:43:29.099 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:29.098+0000 7f5d84c00700 1 --2- 172.21.15.72:0/3427022756 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f5d640397f0 0x7f5d6403bcb0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:29.099 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:29.098+0000 7f5d84c00700 1 --2- 172.21.15.72:0/3427022756 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5d80077450 0x7f5d8019e9f0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:29.100 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:29.098+0000 7f5d84c00700 1 -- 172.21.15.72:0/3427022756 >> 172.21.15.72:0/3427022756 conn(0x7f5d800fe420 msgr2=0x7f5d800fef90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:29.100 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:29.098+0000 7f5d84c00700 1 -- 172.21.15.72:0/3427022756 shutdown_connections 2024-04-26T21:43:29.100 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:29.099+0000 7f5d84c00700 1 -- 172.21.15.72:0/3427022756 wait complete. 2024-04-26T21:43:29.467 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:43:29 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:43:29.467 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:43:29 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:43:29.467 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:43:29 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi072", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:43:29.467 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:43:29 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:43:29.467 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:43:29 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:43:29.467 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:43:29 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:43:29.467 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:43:29 smithi072 ceph-mon[28138]: from='client.? 172.21.15.72:0/3427022756' entity='client.admin' 2024-04-26T21:43:29.895 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-04-26T21:43:29.896 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a1780950-0415-11ef-bc93-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-04-26T21:43:30.296 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/a1780950-0415-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:43:30.379 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:43:30 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:43:30.379 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:43:30 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:43:30.379 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:43:30 smithi072 ceph-mon[28138]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:43:32.760 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:43:32 smithi072 ceph-mon[28138]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:43:32.933 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:32.932+0000 7fdaa3200700 1 -- 172.21.15.72:0/2774981761 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fda9c1050d0 msgr2=0x7fda9c1054f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:32.933 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:32.932+0000 7fdaa3200700 1 --2- 172.21.15.72:0/2774981761 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fda9c1050d0 0x7fda9c1054f0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fda8c009910 tx=0x7fda8c009c20 comp rx=0 tx=0).stop 2024-04-26T21:43:32.933 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:32.932+0000 7fdaa3200700 1 -- 172.21.15.72:0/2774981761 shutdown_connections 2024-04-26T21:43:32.933 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:32.932+0000 7fdaa3200700 1 --2- 172.21.15.72:0/2774981761 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fda9c1050d0 0x7fda9c1054f0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:32.933 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:32.932+0000 7fdaa3200700 1 -- 172.21.15.72:0/2774981761 >> 172.21.15.72:0/2774981761 conn(0x7fda9c100650 msgr2=0x7fda9c102ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:32.934 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:32.932+0000 7fdaa3200700 1 -- 172.21.15.72:0/2774981761 shutdown_connections 2024-04-26T21:43:32.934 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:32.932+0000 7fdaa3200700 1 -- 172.21.15.72:0/2774981761 wait complete. 2024-04-26T21:43:32.934 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:32.934+0000 7fdaa3200700 1 Processor -- start 2024-04-26T21:43:32.934 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:32.934+0000 7fdaa3200700 1 -- start start 2024-04-26T21:43:32.935 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:32.934+0000 7fdaa3200700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fda9c1050d0 0x7fda9c19a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:32.935 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:32.934+0000 7fdaa3200700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fda9c10aec0 con 0x7fda9c1050d0 2024-04-26T21:43:32.935 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:32.935+0000 7fda9be00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fda9c1050d0 0x7fda9c19a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:32.936 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:32.935+0000 7fda9be00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fda9c1050d0 0x7fda9c19a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:42800/0 (socket says 172.21.15.72:42800) 2024-04-26T21:43:32.936 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:32.935+0000 7fda9be00700 1 -- 172.21.15.72:0/2350496118 learned_addr learned my addr 172.21.15.72:0/2350496118 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:43:32.936 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:32.935+0000 7fda9be00700 1 -- 172.21.15.72:0/2350496118 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda9c19ab70 con 0x7fda9c1050d0 2024-04-26T21:43:32.936 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:32.935+0000 7fda9be00700 1 --2- 172.21.15.72:0/2350496118 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fda9c1050d0 0x7fda9c19a630 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fda8c005120 tx=0x7fda8c005810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:32.936 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:32.935+0000 7fda98c00700 1 -- 172.21.15.72:0/2350496118 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fda8c027070 con 0x7fda9c1050d0 2024-04-26T21:43:32.937 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:32.936+0000 7fda98c00700 1 -- 172.21.15.72:0/2350496118 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fda8c00f460 con 0x7fda9c1050d0 2024-04-26T21:43:32.937 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:32.936+0000 7fda98c00700 1 -- 172.21.15.72:0/2350496118 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fda8c017440 con 0x7fda9c1050d0 2024-04-26T21:43:32.937 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:32.936+0000 7fdaa3200700 1 -- 172.21.15.72:0/2350496118 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda9c19ae00 con 0x7fda9c1050d0 2024-04-26T21:43:32.937 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:32.936+0000 7fdaa3200700 1 -- 172.21.15.72:0/2350496118 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda9c19daf0 con 0x7fda9c1050d0 2024-04-26T21:43:32.937 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:32.937+0000 7fda98c00700 1 -- 172.21.15.72:0/2350496118 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fda8c02a070 con 0x7fda9c1050d0 2024-04-26T21:43:32.937 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:32.937+0000 7fdaa3200700 1 -- 172.21.15.72:0/2350496118 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda9c10b640 con 0x7fda9c1050d0 2024-04-26T21:43:32.938 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:32.937+0000 7fda98c00700 1 --2- 172.21.15.72:0/2350496118 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fda80039650 0x7fda8003bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:32.938 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:32.937+0000 7fda98c00700 1 -- 172.21.15.72:0/2350496118 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fda8c04d950 con 0x7fda9c1050d0 2024-04-26T21:43:32.938 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:32.938+0000 7fda9b400700 1 --2- 172.21.15.72:0/2350496118 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fda80039650 0x7fda8003bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:32.938 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:32.938+0000 7fda9b400700 1 --2- 172.21.15.72:0/2350496118 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fda80039650 0x7fda8003bb10 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fda88006fd0 tx=0x7fda88006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:32.943 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:32.943+0000 7fda98c00700 1 -- 172.21.15.72:0/2350496118 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fda8c013070 con 0x7fda9c1050d0 2024-04-26T21:43:33.173 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:33.172+0000 7fdaa3200700 1 -- 172.21.15.72:0/2350496118 --> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7fda9c02e5a0 con 0x7fda80039650 2024-04-26T21:43:33.183 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:33.182+0000 7fda98c00700 1 -- 172.21.15.72:0/2350496118 <== mgr.14152 v2:172.21.15.72:6800/588150172 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fda9c02e5a0 con 0x7fda80039650 2024-04-26T21:43:33.186 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:33.185+0000 7fdaa3200700 1 -- 172.21.15.72:0/2350496118 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fda80039650 msgr2=0x7fda8003bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:33.186 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:33.185+0000 7fdaa3200700 1 --2- 172.21.15.72:0/2350496118 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fda80039650 0x7fda8003bb10 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fda88006fd0 tx=0x7fda88006e40 comp rx=0 tx=0).stop 2024-04-26T21:43:33.186 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:33.185+0000 7fdaa3200700 1 -- 172.21.15.72:0/2350496118 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fda9c1050d0 msgr2=0x7fda9c19a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:33.186 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:33.185+0000 7fdaa3200700 1 --2- 172.21.15.72:0/2350496118 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fda9c1050d0 0x7fda9c19a630 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fda8c005120 tx=0x7fda8c005810 comp rx=0 tx=0).stop 2024-04-26T21:43:33.186 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:33.185+0000 7fdaa3200700 1 -- 172.21.15.72:0/2350496118 shutdown_connections 2024-04-26T21:43:33.186 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:33.185+0000 7fdaa3200700 1 --2- 172.21.15.72:0/2350496118 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fda80039650 0x7fda8003bb10 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:33.186 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:33.185+0000 7fdaa3200700 1 --2- 172.21.15.72:0/2350496118 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fda9c1050d0 0x7fda9c19a630 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:33.187 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:33.185+0000 7fdaa3200700 1 -- 172.21.15.72:0/2350496118 >> 172.21.15.72:0/2350496118 conn(0x7fda9c100650 msgr2=0x7fda9c1011f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:33.187 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:33.186+0000 7fdaa3200700 1 -- 172.21.15.72:0/2350496118 shutdown_connections 2024-04-26T21:43:33.187 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:33.186+0000 7fdaa3200700 1 -- 172.21.15.72:0/2350496118 wait complete. 2024-04-26T21:43:33.923 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi076 2024-04-26T21:43:33.924 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-04-26T21:43:33.924 DEBUG:teuthology.orchestra.run.smithi076:> dd of=/etc/ceph/ceph.conf 2024-04-26T21:43:33.980 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-04-26T21:43:33.980 DEBUG:teuthology.orchestra.run.smithi076:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-26T21:43:34.076 INFO:tasks.cephadm:Adding host smithi076 to orchestrator... 2024-04-26T21:43:34.076 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a1780950-0415-11ef-bc93-c7b262605968 -- ceph orch host add smithi076 2024-04-26T21:43:34.477 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/a1780950-0415-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:43:34.572 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:43:34 smithi072 ceph-mon[28138]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:43:34.572 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:43:34 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:43:34.572 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:43:34 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:43:34.572 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:43:34 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:43:34.572 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:43:34 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:43:34.572 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:43:34 smithi072 ceph-mon[28138]: Updating smithi072:/etc/ceph/ceph.conf 2024-04-26T21:43:34.572 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:43:34 smithi072 ceph-mon[28138]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:43:34.572 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:43:34 smithi072 ceph-mon[28138]: Updating smithi072:/var/lib/ceph/a1780950-0415-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T21:43:35.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:43:35 smithi072 ceph-mon[28138]: Updating smithi072:/etc/ceph/ceph.client.admin.keyring 2024-04-26T21:43:36.610 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:36.609+0000 7f2e85200700 1 -- 172.21.15.72:0/4168271782 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2e80075850 msgr2=0x7f2e80075c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:36.610 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:36.609+0000 7f2e85200700 1 --2- 172.21.15.72:0/4168271782 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2e80075850 0x7f2e80075c30 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f2e70009910 tx=0x7f2e70009c20 comp rx=0 tx=0).stop 2024-04-26T21:43:36.610 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:36.609+0000 7f2e85200700 1 -- 172.21.15.72:0/4168271782 shutdown_connections 2024-04-26T21:43:36.610 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:36.609+0000 7f2e85200700 1 --2- 172.21.15.72:0/4168271782 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2e80075850 0x7f2e80075c30 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:36.610 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:36.609+0000 7f2e85200700 1 -- 172.21.15.72:0/4168271782 >> 172.21.15.72:0/4168271782 conn(0x7f2e800fe960 msgr2=0x7f2e80100d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:36.610 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:36.609+0000 7f2e85200700 1 -- 172.21.15.72:0/4168271782 shutdown_connections 2024-04-26T21:43:36.610 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:36.609+0000 7f2e85200700 1 -- 172.21.15.72:0/4168271782 wait complete. 2024-04-26T21:43:36.611 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:36.610+0000 7f2e85200700 1 Processor -- start 2024-04-26T21:43:36.611 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:36.610+0000 7f2e85200700 1 -- start start 2024-04-26T21:43:36.611 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:36.611+0000 7f2e85200700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2e80075850 0x7f2e801a06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:36.612 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:36.611+0000 7f2e85200700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f2e8010f440 con 0x7f2e80075850 2024-04-26T21:43:36.612 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:36.611+0000 7f2e7de00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2e80075850 0x7f2e801a06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:36.612 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:36.611+0000 7f2e7de00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2e80075850 0x7f2e801a06b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:42824/0 (socket says 172.21.15.72:42824) 2024-04-26T21:43:36.612 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:36.611+0000 7f2e7de00700 1 -- 172.21.15.72:0/3318588194 learned_addr learned my addr 172.21.15.72:0/3318588194 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:43:36.613 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:36.612+0000 7f2e7de00700 1 -- 172.21.15.72:0/3318588194 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e801a0bf0 con 0x7f2e80075850 2024-04-26T21:43:36.613 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:36.612+0000 7f2e7de00700 1 --2- 172.21.15.72:0/3318588194 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2e80075850 0x7f2e801a06b0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f2e70006010 tx=0x7f2e70005710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:36.613 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:36.612+0000 7f2e76a00700 1 -- 172.21.15.72:0/3318588194 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2e70026070 con 0x7f2e80075850 2024-04-26T21:43:36.613 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:36.612+0000 7f2e76a00700 1 -- 172.21.15.72:0/3318588194 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2e7000b750 con 0x7f2e80075850 2024-04-26T21:43:36.613 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:36.612+0000 7f2e76a00700 1 -- 172.21.15.72:0/3318588194 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2e700038a0 con 0x7f2e80075850 2024-04-26T21:43:36.613 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:36.612+0000 7f2e85200700 1 -- 172.21.15.72:0/3318588194 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e801a0e80 con 0x7f2e80075850 2024-04-26T21:43:36.613 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:36.613+0000 7f2e85200700 1 -- 172.21.15.72:0/3318588194 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e8019a830 con 0x7f2e80075850 2024-04-26T21:43:36.614 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:36.614+0000 7f2e85200700 1 -- 172.21.15.72:0/3318588194 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e80076f00 con 0x7f2e80075850 2024-04-26T21:43:36.614 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:36.614+0000 7f2e76a00700 1 -- 172.21.15.72:0/3318588194 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f2e70029070 con 0x7f2e80075850 2024-04-26T21:43:36.615 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:36.614+0000 7f2e76a00700 1 --2- 172.21.15.72:0/3318588194 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f2e640396b0 0x7f2e6403bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:36.615 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:36.614+0000 7f2e76a00700 1 -- 172.21.15.72:0/3318588194 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f2e7001e070 con 0x7f2e80075850 2024-04-26T21:43:36.615 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:36.615+0000 7f2e7d400700 1 --2- 172.21.15.72:0/3318588194 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f2e640396b0 0x7f2e6403bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:36.616 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:36.615+0000 7f2e7d400700 1 --2- 172.21.15.72:0/3318588194 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f2e640396b0 0x7f2e6403bb70 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f2e6c006fd0 tx=0x7f2e6c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:36.620 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:36.620+0000 7f2e76a00700 1 -- 172.21.15.72:0/3318588194 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f2e70013070 con 0x7f2e80075850 2024-04-26T21:43:36.854 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:36.853+0000 7f2e85200700 1 -- 172.21.15.72:0/3318588194 --> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi076", "target": ["mon-mgr", ""]}) -- 0x7f2e8019bd60 con 0x7f2e640396b0 2024-04-26T21:43:37.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:43:36 smithi072 ceph-mon[28138]: Updating smithi072:/var/lib/ceph/a1780950-0415-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-26T21:43:37.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:43:36 smithi072 ceph-mon[28138]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:43:37.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:43:36 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:43:37.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:43:36 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:43:37.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:43:36 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:43:38.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:43:37 smithi072 ceph-mon[28138]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi076", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:43:39.074 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:43:38 smithi072 ceph-mon[28138]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:43:40.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:43:39 smithi072 ceph-mon[28138]: Deploying cephadm binary to smithi076 2024-04-26T21:43:41.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:43:40 smithi072 ceph-mon[28138]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:43:43.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:43:42 smithi072 ceph-mon[28138]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:43:43.077 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:43.077+0000 7f2e76a00700 1 -- 172.21.15.72:0/3318588194 <== mgr.14152 v2:172.21.15.72:6800/588150172 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7f2e8019bd60 con 0x7f2e640396b0 2024-04-26T21:43:43.078 INFO:teuthology.orchestra.run.smithi072.stdout:Added host 'smithi076' with addr '172.21.15.76' 2024-04-26T21:43:43.081 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:43.080+0000 7f2e85200700 1 -- 172.21.15.72:0/3318588194 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f2e640396b0 msgr2=0x7f2e6403bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:43.081 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:43.080+0000 7f2e85200700 1 --2- 172.21.15.72:0/3318588194 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f2e640396b0 0x7f2e6403bb70 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f2e6c006fd0 tx=0x7f2e6c006e40 comp rx=0 tx=0).stop 2024-04-26T21:43:43.082 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:43.080+0000 7f2e85200700 1 -- 172.21.15.72:0/3318588194 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2e80075850 msgr2=0x7f2e801a06b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:43.082 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:43.080+0000 7f2e85200700 1 --2- 172.21.15.72:0/3318588194 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2e80075850 0x7f2e801a06b0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f2e70006010 tx=0x7f2e70005710 comp rx=0 tx=0).stop 2024-04-26T21:43:43.082 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:43.080+0000 7f2e85200700 1 -- 172.21.15.72:0/3318588194 shutdown_connections 2024-04-26T21:43:43.082 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:43.080+0000 7f2e85200700 1 --2- 172.21.15.72:0/3318588194 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f2e640396b0 0x7f2e6403bb70 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:43.082 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:43.080+0000 7f2e85200700 1 --2- 172.21.15.72:0/3318588194 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2e80075850 0x7f2e801a06b0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:43.082 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:43.080+0000 7f2e85200700 1 -- 172.21.15.72:0/3318588194 >> 172.21.15.72:0/3318588194 conn(0x7f2e800fe960 msgr2=0x7f2e801090e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:43.082 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:43.081+0000 7f2e85200700 1 -- 172.21.15.72:0/3318588194 shutdown_connections 2024-04-26T21:43:43.082 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:43.081+0000 7f2e85200700 1 -- 172.21.15.72:0/3318588194 wait complete. 2024-04-26T21:43:43.797 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a1780950-0415-11ef-bc93-c7b262605968 -- ceph orch host ls --format=json 2024-04-26T21:43:44.228 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/a1780950-0415-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:43:44.380 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:43:44 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:43:44.381 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:43:44 smithi072 ceph-mon[28138]: Added host smithi076 2024-04-26T21:43:44.381 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:43:44 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:43:44.381 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:43:44 smithi072 ceph-mon[28138]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:43:44.381 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:43:44 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:43:46.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:43:45 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:43:46.259 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:46.256+0000 7fa629800700 1 -- 172.21.15.72:0/1503090607 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa624103090 msgr2=0x7fa624103470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:46.259 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:46.256+0000 7fa629800700 1 --2- 172.21.15.72:0/1503090607 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa624103090 0x7fa624103470 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fa614009910 tx=0x7fa614009c20 comp rx=0 tx=0).stop 2024-04-26T21:43:46.259 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:46.257+0000 7fa629800700 1 -- 172.21.15.72:0/1503090607 shutdown_connections 2024-04-26T21:43:46.259 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:46.257+0000 7fa629800700 1 --2- 172.21.15.72:0/1503090607 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa624103090 0x7fa624103470 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:46.259 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:46.257+0000 7fa629800700 1 -- 172.21.15.72:0/1503090607 >> 172.21.15.72:0/1503090607 conn(0x7fa6240fe960 msgr2=0x7fa624100d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:46.259 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:46.257+0000 7fa629800700 1 -- 172.21.15.72:0/1503090607 shutdown_connections 2024-04-26T21:43:46.259 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:46.257+0000 7fa629800700 1 -- 172.21.15.72:0/1503090607 wait complete. 2024-04-26T21:43:46.260 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:46.258+0000 7fa629800700 1 Processor -- start 2024-04-26T21:43:46.260 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:46.258+0000 7fa629800700 1 -- start start 2024-04-26T21:43:46.260 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:46.259+0000 7fa629800700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa624103090 0x7fa62419e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:46.260 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:46.259+0000 7fa629800700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fa62410d1c0 con 0x7fa624103090 2024-04-26T21:43:46.260 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:46.259+0000 7fa622a00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa624103090 0x7fa62419e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:46.260 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:46.259+0000 7fa622a00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa624103090 0x7fa62419e5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:34010/0 (socket says 172.21.15.72:34010) 2024-04-26T21:43:46.261 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:46.259+0000 7fa622a00700 1 -- 172.21.15.72:0/2413228525 learned_addr learned my addr 172.21.15.72:0/2413228525 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:43:46.261 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:46.259+0000 7fa622a00700 1 -- 172.21.15.72:0/2413228525 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa62419eb20 con 0x7fa624103090 2024-04-26T21:43:46.261 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:46.260+0000 7fa622a00700 1 --2- 172.21.15.72:0/2413228525 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa624103090 0x7fa62419e5e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fa614006010 tx=0x7fa614005710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:46.262 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:46.260+0000 7fa61be00700 1 -- 172.21.15.72:0/2413228525 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa614026070 con 0x7fa624103090 2024-04-26T21:43:46.262 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:46.260+0000 7fa61be00700 1 -- 172.21.15.72:0/2413228525 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa61400b6f0 con 0x7fa624103090 2024-04-26T21:43:46.262 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:46.260+0000 7fa629800700 1 -- 172.21.15.72:0/2413228525 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa62419ee10 con 0x7fa624103090 2024-04-26T21:43:46.262 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:46.260+0000 7fa61be00700 1 -- 172.21.15.72:0/2413228525 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa614003810 con 0x7fa624103090 2024-04-26T21:43:46.262 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:46.260+0000 7fa629800700 1 -- 172.21.15.72:0/2413228525 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa624198880 con 0x7fa624103090 2024-04-26T21:43:46.263 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:46.261+0000 7fa61be00700 1 -- 172.21.15.72:0/2413228525 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fa614029070 con 0x7fa624103090 2024-04-26T21:43:46.263 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:46.261+0000 7fa629800700 1 -- 172.21.15.72:0/2413228525 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa624104570 con 0x7fa624103090 2024-04-26T21:43:46.263 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:46.262+0000 7fa61be00700 1 --2- 172.21.15.72:0/2413228525 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fa6080396b0 0x7fa60803bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:46.263 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:46.262+0000 7fa61be00700 1 -- 172.21.15.72:0/2413228525 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fa61401e070 con 0x7fa624103090 2024-04-26T21:43:46.264 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:46.262+0000 7fa61a000700 1 --2- 172.21.15.72:0/2413228525 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fa6080396b0 0x7fa60803bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:46.264 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:46.263+0000 7fa61a000700 1 --2- 172.21.15.72:0/2413228525 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fa6080396b0 0x7fa60803bb70 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fa60c006fd0 tx=0x7fa60c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:46.269 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:46.268+0000 7fa61be00700 1 -- 172.21.15.72:0/2413228525 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fa614013070 con 0x7fa624103090 2024-04-26T21:43:46.500 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:46.498+0000 7fa629800700 1 -- 172.21.15.72:0/2413228525 --> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa624199be0 con 0x7fa6080396b0 2024-04-26T21:43:46.503 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:46.501+0000 7fa61be00700 1 -- 172.21.15.72:0/2413228525 <== mgr.14152 v2:172.21.15.72:6800/588150172 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+159 (secure 0 0 0) 0x7fa624199be0 con 0x7fa6080396b0 2024-04-26T21:43:46.503 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T21:43:46.503 INFO:teuthology.orchestra.run.smithi072.stdout:[{"addr": "172.21.15.72", "hostname": "smithi072", "labels": [], "status": ""}, {"addr": "172.21.15.76", "hostname": "smithi076", "labels": [], "status": ""}] 2024-04-26T21:43:46.505 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:46.504+0000 7fa629800700 1 -- 172.21.15.72:0/2413228525 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fa6080396b0 msgr2=0x7fa60803bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:46.505 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:46.504+0000 7fa629800700 1 --2- 172.21.15.72:0/2413228525 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fa6080396b0 0x7fa60803bb70 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fa60c006fd0 tx=0x7fa60c006e40 comp rx=0 tx=0).stop 2024-04-26T21:43:46.505 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:46.504+0000 7fa629800700 1 -- 172.21.15.72:0/2413228525 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa624103090 msgr2=0x7fa62419e5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:46.506 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:46.504+0000 7fa629800700 1 --2- 172.21.15.72:0/2413228525 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa624103090 0x7fa62419e5e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fa614006010 tx=0x7fa614005710 comp rx=0 tx=0).stop 2024-04-26T21:43:46.506 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:46.504+0000 7fa629800700 1 -- 172.21.15.72:0/2413228525 shutdown_connections 2024-04-26T21:43:46.506 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:46.504+0000 7fa629800700 1 --2- 172.21.15.72:0/2413228525 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fa6080396b0 0x7fa60803bb70 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:46.506 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:46.504+0000 7fa629800700 1 --2- 172.21.15.72:0/2413228525 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa624103090 0x7fa62419e5e0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:46.506 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:46.504+0000 7fa629800700 1 -- 172.21.15.72:0/2413228525 >> 172.21.15.72:0/2413228525 conn(0x7fa6240fe960 msgr2=0x7fa62410ac30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:46.506 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:46.504+0000 7fa629800700 1 -- 172.21.15.72:0/2413228525 shutdown_connections 2024-04-26T21:43:46.506 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:46.504+0000 7fa629800700 1 -- 172.21.15.72:0/2413228525 wait complete. 2024-04-26T21:43:47.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:43:46 smithi072 ceph-mon[28138]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:43:47.270 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi164 2024-04-26T21:43:47.271 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-04-26T21:43:47.271 DEBUG:teuthology.orchestra.run.smithi164:> dd of=/etc/ceph/ceph.conf 2024-04-26T21:43:47.329 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-04-26T21:43:47.329 DEBUG:teuthology.orchestra.run.smithi164:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-26T21:43:47.426 INFO:tasks.cephadm:Adding host smithi164 to orchestrator... 2024-04-26T21:43:47.426 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a1780950-0415-11ef-bc93-c7b262605968 -- ceph orch host add smithi164 2024-04-26T21:43:47.860 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/a1780950-0415-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:43:48.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:43:47 smithi072 ceph-mon[28138]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T21:43:48.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:43:47 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:43:49.037 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:43:48 smithi072 ceph-mon[28138]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:43:49.901 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:49.900+0000 7f9a76a00700 1 -- 172.21.15.72:0/1647909075 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9a700757f0 msgr2=0x7f9a70075bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:49.901 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:49.900+0000 7f9a76a00700 1 --2- 172.21.15.72:0/1647909075 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9a700757f0 0x7f9a70075bd0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f9a64009910 tx=0x7f9a64009c20 comp rx=0 tx=0).stop 2024-04-26T21:43:49.901 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:49.900+0000 7f9a76a00700 1 -- 172.21.15.72:0/1647909075 shutdown_connections 2024-04-26T21:43:49.901 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:49.900+0000 7f9a76a00700 1 --2- 172.21.15.72:0/1647909075 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9a700757f0 0x7f9a70075bd0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:49.902 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:49.900+0000 7f9a76a00700 1 -- 172.21.15.72:0/1647909075 >> 172.21.15.72:0/1647909075 conn(0x7f9a700fe7c0 msgr2=0x7f9a70100be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:49.902 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:49.900+0000 7f9a76a00700 1 -- 172.21.15.72:0/1647909075 shutdown_connections 2024-04-26T21:43:49.902 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:49.900+0000 7f9a76a00700 1 -- 172.21.15.72:0/1647909075 wait complete. 2024-04-26T21:43:49.902 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:49.901+0000 7f9a76a00700 1 Processor -- start 2024-04-26T21:43:49.902 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:49.902+0000 7f9a76a00700 1 -- start start 2024-04-26T21:43:49.902 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:49.902+0000 7f9a76a00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9a700757f0 0x7f9a701a05b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:49.903 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:49.902+0000 7f9a76a00700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f9a7010f290 con 0x7f9a700757f0 2024-04-26T21:43:49.903 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:49.902+0000 7f9a75600700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9a700757f0 0x7f9a701a05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:49.903 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:49.902+0000 7f9a75600700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9a700757f0 0x7f9a701a05b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:34028/0 (socket says 172.21.15.72:34028) 2024-04-26T21:43:49.903 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:49.902+0000 7f9a75600700 1 -- 172.21.15.72:0/492663559 learned_addr learned my addr 172.21.15.72:0/492663559 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:43:49.904 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:49.903+0000 7f9a75600700 1 -- 172.21.15.72:0/492663559 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a701a0af0 con 0x7f9a700757f0 2024-04-26T21:43:49.904 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:49.903+0000 7f9a75600700 1 --2- 172.21.15.72:0/492663559 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9a700757f0 0x7f9a701a05b0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f9a64006010 tx=0x7f9a64005710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:49.904 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:49.903+0000 7f9a6e000700 1 -- 172.21.15.72:0/492663559 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9a64026070 con 0x7f9a700757f0 2024-04-26T21:43:49.904 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:49.903+0000 7f9a6e000700 1 -- 172.21.15.72:0/492663559 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9a6400b750 con 0x7f9a700757f0 2024-04-26T21:43:49.904 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:49.903+0000 7f9a76a00700 1 -- 172.21.15.72:0/492663559 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a701a0d80 con 0x7f9a700757f0 2024-04-26T21:43:49.904 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:49.903+0000 7f9a6e000700 1 -- 172.21.15.72:0/492663559 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9a640038a0 con 0x7f9a700757f0 2024-04-26T21:43:49.905 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:49.904+0000 7f9a76a00700 1 -- 172.21.15.72:0/492663559 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a7019a680 con 0x7f9a700757f0 2024-04-26T21:43:49.905 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:49.905+0000 7f9a6e000700 1 -- 172.21.15.72:0/492663559 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f9a64029070 con 0x7f9a700757f0 2024-04-26T21:43:49.905 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:49.905+0000 7f9a76a00700 1 -- 172.21.15.72:0/492663559 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a70076ea0 con 0x7f9a700757f0 2024-04-26T21:43:49.906 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:49.905+0000 7f9a6e000700 1 --2- 172.21.15.72:0/492663559 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f9a580396b0 0x7f9a5803bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:49.906 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:49.905+0000 7f9a6e000700 1 -- 172.21.15.72:0/492663559 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f9a6401e070 con 0x7f9a700757f0 2024-04-26T21:43:49.906 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:49.906+0000 7f9a74c00700 1 --2- 172.21.15.72:0/492663559 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f9a580396b0 0x7f9a5803bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:49.907 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:49.906+0000 7f9a74c00700 1 --2- 172.21.15.72:0/492663559 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f9a580396b0 0x7f9a5803bb70 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f9a5c006fd0 tx=0x7f9a5c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:49.911 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:49.911+0000 7f9a6e000700 1 -- 172.21.15.72:0/492663559 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f9a64013070 con 0x7f9a700757f0 2024-04-26T21:43:50.141 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:50.140+0000 7f9a76a00700 1 -- 172.21.15.72:0/492663559 --> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi164", "target": ["mon-mgr", ""]}) -- 0x7f9a7019bbb0 con 0x7f9a580396b0 2024-04-26T21:43:51.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:43:50 smithi072 ceph-mon[28138]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:43:52.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:43:51 smithi072 ceph-mon[28138]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi164", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:43:53.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:43:52 smithi072 ceph-mon[28138]: Deploying cephadm binary to smithi164 2024-04-26T21:43:53.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:43:52 smithi072 ceph-mon[28138]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:43:53.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:43:52 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:43:53.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:43:52 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:43:53.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:43:52 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:43:53.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:43:52 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:43:53.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:43:52 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi076", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:43:53.073 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:43:52 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:43:53.073 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:43:52 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:43:54.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:43:53 smithi072 ceph-mon[28138]: Updating smithi076:/etc/ceph/ceph.conf 2024-04-26T21:43:55.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:43:54 smithi072 ceph-mon[28138]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:43:56.310 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:56.309+0000 7f9a6e000700 1 -- 172.21.15.72:0/492663559 <== mgr.14152 v2:172.21.15.72:6800/588150172 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f9a7019bbb0 con 0x7f9a580396b0 2024-04-26T21:43:56.311 INFO:teuthology.orchestra.run.smithi072.stdout:Added host 'smithi164' with addr '172.21.15.164' 2024-04-26T21:43:56.314 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:56.314+0000 7f9a76a00700 1 -- 172.21.15.72:0/492663559 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f9a580396b0 msgr2=0x7f9a5803bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:56.314 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:56.314+0000 7f9a76a00700 1 --2- 172.21.15.72:0/492663559 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f9a580396b0 0x7f9a5803bb70 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f9a5c006fd0 tx=0x7f9a5c006e40 comp rx=0 tx=0).stop 2024-04-26T21:43:56.315 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:56.314+0000 7f9a76a00700 1 -- 172.21.15.72:0/492663559 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9a700757f0 msgr2=0x7f9a701a05b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:56.315 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:56.314+0000 7f9a76a00700 1 --2- 172.21.15.72:0/492663559 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9a700757f0 0x7f9a701a05b0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f9a64006010 tx=0x7f9a64005710 comp rx=0 tx=0).stop 2024-04-26T21:43:56.315 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:56.314+0000 7f9a76a00700 1 -- 172.21.15.72:0/492663559 shutdown_connections 2024-04-26T21:43:56.315 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:56.314+0000 7f9a76a00700 1 --2- 172.21.15.72:0/492663559 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f9a580396b0 0x7f9a5803bb70 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:56.315 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:56.314+0000 7f9a76a00700 1 --2- 172.21.15.72:0/492663559 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9a700757f0 0x7f9a701a05b0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:56.315 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:56.314+0000 7f9a76a00700 1 -- 172.21.15.72:0/492663559 >> 172.21.15.72:0/492663559 conn(0x7f9a700fe7c0 msgr2=0x7f9a70108fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:56.315 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:56.314+0000 7f9a76a00700 1 -- 172.21.15.72:0/492663559 shutdown_connections 2024-04-26T21:43:56.316 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:56.314+0000 7f9a76a00700 1 -- 172.21.15.72:0/492663559 wait complete. 2024-04-26T21:43:56.962 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:43:56 smithi072 ceph-mon[28138]: Updating smithi076:/var/lib/ceph/a1780950-0415-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T21:43:56.962 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:43:56 smithi072 ceph-mon[28138]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:43:56.962 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:43:56 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:43:57.009 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a1780950-0415-11ef-bc93-c7b262605968 -- ceph orch host ls --format=json 2024-04-26T21:43:57.432 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/a1780950-0415-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:43:57.915 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:43:57 smithi072 ceph-mon[28138]: Updating smithi076:/etc/ceph/ceph.client.admin.keyring 2024-04-26T21:43:57.915 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:43:57 smithi072 ceph-mon[28138]: Added host smithi164 2024-04-26T21:43:57.915 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:43:57 smithi072 ceph-mon[28138]: Updating smithi076:/var/lib/ceph/a1780950-0415-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-26T21:43:57.915 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:43:57 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:43:57.916 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:43:57 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:43:57.916 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:43:57 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:43:58.935 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:43:58 smithi072 ceph-mon[28138]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:43:58.936 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:43:58 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:43:59.441 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:59.439+0000 7f0801400700 1 -- 172.21.15.72:0/3906026537 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f07fc103ff0 msgr2=0x7f07fc1043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:59.441 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:59.439+0000 7f0801400700 1 --2- 172.21.15.72:0/3906026537 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f07fc103ff0 0x7f07fc1043d0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f07ec009910 tx=0x7f07ec009c20 comp rx=0 tx=0).stop 2024-04-26T21:43:59.441 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:59.439+0000 7f0801400700 1 -- 172.21.15.72:0/3906026537 shutdown_connections 2024-04-26T21:43:59.441 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:59.439+0000 7f0801400700 1 --2- 172.21.15.72:0/3906026537 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f07fc103ff0 0x7f07fc1043d0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:59.441 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:59.439+0000 7f0801400700 1 -- 172.21.15.72:0/3906026537 >> 172.21.15.72:0/3906026537 conn(0x7f07fc0fd820 msgr2=0x7f07fc0ffc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:59.441 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:59.440+0000 7f0801400700 1 -- 172.21.15.72:0/3906026537 shutdown_connections 2024-04-26T21:43:59.441 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:59.440+0000 7f0801400700 1 -- 172.21.15.72:0/3906026537 wait complete. 2024-04-26T21:43:59.442 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:59.441+0000 7f0801400700 1 Processor -- start 2024-04-26T21:43:59.442 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:59.441+0000 7f0801400700 1 -- start start 2024-04-26T21:43:59.442 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:59.441+0000 7f0801400700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f07fc103ff0 0x7f07fc1a06e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:59.442 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:59.441+0000 7f0801400700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f07fc10d240 con 0x7f07fc103ff0 2024-04-26T21:43:59.442 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:59.442+0000 7f07fa800700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f07fc103ff0 0x7f07fc1a06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:59.442 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:59.442+0000 7f07fa800700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f07fc103ff0 0x7f07fc1a06e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:38508/0 (socket says 172.21.15.72:38508) 2024-04-26T21:43:59.442 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:59.442+0000 7f07fa800700 1 -- 172.21.15.72:0/4133843769 learned_addr learned my addr 172.21.15.72:0/4133843769 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:43:59.443 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:59.442+0000 7f07fa800700 1 -- 172.21.15.72:0/4133843769 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07fc1a0c20 con 0x7f07fc103ff0 2024-04-26T21:43:59.443 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:59.442+0000 7f07fa800700 1 --2- 172.21.15.72:0/4133843769 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f07fc103ff0 0x7f07fc1a06e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f07ec005060 tx=0x7f07ec005710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:59.443 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:59.443+0000 7f07f3400700 1 -- 172.21.15.72:0/4133843769 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f07ec026070 con 0x7f07fc103ff0 2024-04-26T21:43:59.443 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:59.443+0000 7f07f3400700 1 -- 172.21.15.72:0/4133843769 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f07ec00b750 con 0x7f07fc103ff0 2024-04-26T21:43:59.443 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:59.443+0000 7f07f3400700 1 -- 172.21.15.72:0/4133843769 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f07ec003850 con 0x7f07fc103ff0 2024-04-26T21:43:59.443 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:59.443+0000 7f0801400700 1 -- 172.21.15.72:0/4133843769 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07fc1a0eb0 con 0x7f07fc103ff0 2024-04-26T21:43:59.444 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:59.443+0000 7f0801400700 1 -- 172.21.15.72:0/4133843769 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07fc19a8a0 con 0x7f07fc103ff0 2024-04-26T21:43:59.444 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:59.444+0000 7f07f3400700 1 -- 172.21.15.72:0/4133843769 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f07ec029070 con 0x7f07fc103ff0 2024-04-26T21:43:59.445 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:59.444+0000 7f07f3400700 1 --2- 172.21.15.72:0/4133843769 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f07e00396b0 0x7f07e003bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:59.445 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:59.444+0000 7f07f3400700 1 -- 172.21.15.72:0/4133843769 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f07ec01e070 con 0x7f07fc103ff0 2024-04-26T21:43:59.445 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:59.444+0000 7f0801400700 1 -- 172.21.15.72:0/4133843769 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07fc06bb70 con 0x7f07fc103ff0 2024-04-26T21:43:59.445 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:59.445+0000 7f07f9e00700 1 --2- 172.21.15.72:0/4133843769 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f07e00396b0 0x7f07e003bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:59.445 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:59.445+0000 7f07f9e00700 1 --2- 172.21.15.72:0/4133843769 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f07e00396b0 0x7f07e003bb70 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f07e8006fd0 tx=0x7f07e8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:59.451 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:59.451+0000 7f07f3400700 1 -- 172.21.15.72:0/4133843769 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f07ec030020 con 0x7f07fc103ff0 2024-04-26T21:43:59.689 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:59.688+0000 7f0801400700 1 -- 172.21.15.72:0/4133843769 --> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f07fc19bdd0 con 0x7f07e00396b0 2024-04-26T21:43:59.691 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:59.690+0000 7f07f3400700 1 -- 172.21.15.72:0/4133843769 <== mgr.14152 v2:172.21.15.72:6800/588150172 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+239 (secure 0 0 0) 0x7f07fc19bdd0 con 0x7f07e00396b0 2024-04-26T21:43:59.691 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T21:43:59.691 INFO:teuthology.orchestra.run.smithi072.stdout:[{"addr": "172.21.15.72", "hostname": "smithi072", "labels": [], "status": ""}, {"addr": "172.21.15.76", "hostname": "smithi076", "labels": [], "status": ""}, {"addr": "172.21.15.164", "hostname": "smithi164", "labels": [], "status": ""}] 2024-04-26T21:43:59.694 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:59.693+0000 7f0801400700 1 -- 172.21.15.72:0/4133843769 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f07e00396b0 msgr2=0x7f07e003bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:59.694 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:59.693+0000 7f0801400700 1 --2- 172.21.15.72:0/4133843769 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f07e00396b0 0x7f07e003bb70 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f07e8006fd0 tx=0x7f07e8006e40 comp rx=0 tx=0).stop 2024-04-26T21:43:59.694 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:59.693+0000 7f0801400700 1 -- 172.21.15.72:0/4133843769 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f07fc103ff0 msgr2=0x7f07fc1a06e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:59.694 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:59.693+0000 7f0801400700 1 --2- 172.21.15.72:0/4133843769 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f07fc103ff0 0x7f07fc1a06e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f07ec005060 tx=0x7f07ec005710 comp rx=0 tx=0).stop 2024-04-26T21:43:59.695 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:59.693+0000 7f0801400700 1 -- 172.21.15.72:0/4133843769 shutdown_connections 2024-04-26T21:43:59.695 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:59.693+0000 7f0801400700 1 --2- 172.21.15.72:0/4133843769 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f07e00396b0 0x7f07e003bb70 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:59.695 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:59.693+0000 7f0801400700 1 --2- 172.21.15.72:0/4133843769 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f07fc103ff0 0x7f07fc1a06e0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:59.695 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:59.693+0000 7f0801400700 1 -- 172.21.15.72:0/4133843769 >> 172.21.15.72:0/4133843769 conn(0x7f07fc0fd820 msgr2=0x7f07fc109e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:59.695 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:59.693+0000 7f0801400700 1 -- 172.21.15.72:0/4133843769 shutdown_connections 2024-04-26T21:43:59.695 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:43:59.693+0000 7f0801400700 1 -- 172.21.15.72:0/4133843769 wait complete. 2024-04-26T21:44:00.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:43:59 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:00.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:43:59 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:00.430 INFO:tasks.cephadm:Setting crush tunables to default 2024-04-26T21:44:00.431 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a1780950-0415-11ef-bc93-c7b262605968 -- ceph osd crush tunables default 2024-04-26T21:44:00.858 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/a1780950-0415-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:44:01.187 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:00 smithi072 ceph-mon[28138]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:44:01.187 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:00 smithi072 ceph-mon[28138]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T21:44:02.762 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:02 smithi072 ceph-mon[28138]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:44:02.762 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:02 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:02.930 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:44:02.929+0000 7f2effe00700 1 -- 172.21.15.72:0/1784540990 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2f00075850 msgr2=0x7f2f00075c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:02.930 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:44:02.929+0000 7f2effe00700 1 --2- 172.21.15.72:0/1784540990 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2f00075850 0x7f2f00075c30 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f2ef0009910 tx=0x7f2ef0009c20 comp rx=0 tx=0).stop 2024-04-26T21:44:02.930 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:44:02.929+0000 7f2effe00700 1 -- 172.21.15.72:0/1784540990 shutdown_connections 2024-04-26T21:44:02.930 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:44:02.929+0000 7f2effe00700 1 --2- 172.21.15.72:0/1784540990 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2f00075850 0x7f2f00075c30 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:02.931 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:44:02.929+0000 7f2effe00700 1 -- 172.21.15.72:0/1784540990 >> 172.21.15.72:0/1784540990 conn(0x7f2f000fe780 msgr2=0x7f2f00100ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:02.931 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:44:02.929+0000 7f2effe00700 1 -- 172.21.15.72:0/1784540990 shutdown_connections 2024-04-26T21:44:02.931 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:44:02.929+0000 7f2effe00700 1 -- 172.21.15.72:0/1784540990 wait complete. 2024-04-26T21:44:02.931 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:44:02.930+0000 7f2effe00700 1 Processor -- start 2024-04-26T21:44:02.932 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:44:02.931+0000 7f2effe00700 1 -- start start 2024-04-26T21:44:02.932 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:44:02.931+0000 7f2effe00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2f00075850 0x7f2f001a05c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:02.932 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:44:02.931+0000 7f2effe00700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f2f0010f280 con 0x7f2f00075850 2024-04-26T21:44:02.932 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:44:02.931+0000 7f2efea00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2f00075850 0x7f2f001a05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:02.932 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:44:02.931+0000 7f2efea00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2f00075850 0x7f2f001a05c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:47136/0 (socket says 172.21.15.72:47136) 2024-04-26T21:44:02.932 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:44:02.931+0000 7f2efea00700 1 -- 172.21.15.72:0/3022658462 learned_addr learned my addr 172.21.15.72:0/3022658462 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:44:02.933 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:44:02.932+0000 7f2efea00700 1 -- 172.21.15.72:0/3022658462 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f001a0b00 con 0x7f2f00075850 2024-04-26T21:44:02.933 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:44:02.932+0000 7f2efea00700 1 --2- 172.21.15.72:0/3022658462 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2f00075850 0x7f2f001a05c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f2ef0006010 tx=0x7f2ef0005710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:02.933 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:44:02.932+0000 7f2ef7e00700 1 -- 172.21.15.72:0/3022658462 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2ef0026070 con 0x7f2f00075850 2024-04-26T21:44:02.933 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:44:02.932+0000 7f2ef7e00700 1 -- 172.21.15.72:0/3022658462 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2ef000b750 con 0x7f2f00075850 2024-04-26T21:44:02.933 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:44:02.932+0000 7f2ef7e00700 1 -- 172.21.15.72:0/3022658462 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2ef00038a0 con 0x7f2f00075850 2024-04-26T21:44:02.933 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:44:02.932+0000 7f2effe00700 1 -- 172.21.15.72:0/3022658462 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f001a0d90 con 0x7f2f00075850 2024-04-26T21:44:02.933 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:44:02.933+0000 7f2effe00700 1 -- 172.21.15.72:0/3022658462 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f0019a690 con 0x7f2f00075850 2024-04-26T21:44:02.934 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:44:02.934+0000 7f2ef7e00700 1 -- 172.21.15.72:0/3022658462 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f2ef0029070 con 0x7f2f00075850 2024-04-26T21:44:02.934 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:44:02.934+0000 7f2effe00700 1 -- 172.21.15.72:0/3022658462 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f00076f00 con 0x7f2f00075850 2024-04-26T21:44:02.935 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:44:02.934+0000 7f2ef7e00700 1 --2- 172.21.15.72:0/3022658462 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f2ee40396b0 0x7f2ee403bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:02.935 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:44:02.934+0000 7f2ef7e00700 1 -- 172.21.15.72:0/3022658462 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f2ef001e070 con 0x7f2f00075850 2024-04-26T21:44:02.935 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:44:02.934+0000 7f2ef6000700 1 --2- 172.21.15.72:0/3022658462 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f2ee40396b0 0x7f2ee403bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:02.935 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:44:02.935+0000 7f2ef6000700 1 --2- 172.21.15.72:0/3022658462 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f2ee40396b0 0x7f2ee403bb70 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f2ee8006fd0 tx=0x7f2ee8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:02.940 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:44:02.940+0000 7f2ef7e00700 1 -- 172.21.15.72:0/3022658462 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f2ef0013070 con 0x7f2f00075850 2024-04-26T21:44:03.170 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:44:03.169+0000 7f2effe00700 1 -- 172.21.15.72:0/3022658462 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f2f0019bbc0 con 0x7f2f00075850 2024-04-26T21:44:03.687 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:44:03.686+0000 7f2ef7e00700 1 -- 172.21.15.72:0/3022658462 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7f2ef0003a80 con 0x7f2f00075850 2024-04-26T21:44:03.691 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:44:03.690+0000 7f2effe00700 1 -- 172.21.15.72:0/3022658462 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f2ee40396b0 msgr2=0x7f2ee403bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:03.691 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:44:03.690+0000 7f2effe00700 1 --2- 172.21.15.72:0/3022658462 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f2ee40396b0 0x7f2ee403bb70 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f2ee8006fd0 tx=0x7f2ee8006e40 comp rx=0 tx=0).stop 2024-04-26T21:44:03.691 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:44:03.690+0000 7f2effe00700 1 -- 172.21.15.72:0/3022658462 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2f00075850 msgr2=0x7f2f001a05c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:03.691 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:44:03.690+0000 7f2effe00700 1 --2- 172.21.15.72:0/3022658462 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2f00075850 0x7f2f001a05c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f2ef0006010 tx=0x7f2ef0005710 comp rx=0 tx=0).stop 2024-04-26T21:44:03.691 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:44:03.690+0000 7f2effe00700 1 -- 172.21.15.72:0/3022658462 shutdown_connections 2024-04-26T21:44:03.691 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:44:03.690+0000 7f2effe00700 1 --2- 172.21.15.72:0/3022658462 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f2ee40396b0 0x7f2ee403bb70 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:03.691 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:44:03.690+0000 7f2effe00700 1 --2- 172.21.15.72:0/3022658462 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2f00075850 0x7f2f001a05c0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:03.691 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:44:03.690+0000 7f2effe00700 1 -- 172.21.15.72:0/3022658462 >> 172.21.15.72:0/3022658462 conn(0x7f2f000fe780 msgr2=0x7f2f00108f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:03.692 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:44:03.690+0000 7f2effe00700 1 -- 172.21.15.72:0/3022658462 shutdown_connections 2024-04-26T21:44:03.692 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:44:03.690+0000 7f2effe00700 1 -- 172.21.15.72:0/3022658462 wait complete. 2024-04-26T21:44:03.692 INFO:teuthology.orchestra.run.smithi072.stderr:adjusted tunables profile to default 2024-04-26T21:44:04.063 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:03 smithi072 ceph-mon[28138]: from='client.? 172.21.15.72:0/3022658462' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-26T21:44:04.489 INFO:tasks.cephadm:Adding mon.a on smithi072 2024-04-26T21:44:04.489 INFO:tasks.cephadm:Adding mon.b on smithi076 2024-04-26T21:44:04.489 INFO:tasks.cephadm:Adding mon.c on smithi164 2024-04-26T21:44:04.489 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a1780950-0415-11ef-bc93-c7b262605968 -- ceph orch apply mon '3;smithi072:172.21.15.72=a;smithi076:172.21.15.76=b;smithi164:172.21.15.164=c' 2024-04-26T21:44:04.937 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-26T21:44:05.043 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-26T21:44:05.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:04 smithi072 ceph-mon[28138]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:44:05.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:04 smithi072 ceph-mon[28138]: from='client.? 172.21.15.72:0/3022658462' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-26T21:44:05.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:04 smithi072 ceph-mon[28138]: osdmap e4: 0 total, 0 up, 0 in 2024-04-26T21:44:07.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:06 smithi072 ceph-mon[28138]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:44:07.740 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:07.738+0000 7fd0b1c00700 1 -- 172.21.15.164:0/1860485116 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd0ac1039b0 msgr2=0x7fd0ac105da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:07.740 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:07.738+0000 7fd0b1c00700 1 --2- 172.21.15.164:0/1860485116 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd0ac1039b0 0x7fd0ac105da0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fd09c009910 tx=0x7fd09c009c20 comp rx=0 tx=0).stop 2024-04-26T21:44:07.740 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:07.738+0000 7fd0b1c00700 1 -- 172.21.15.164:0/1860485116 shutdown_connections 2024-04-26T21:44:07.740 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:07.738+0000 7fd0b1c00700 1 --2- 172.21.15.164:0/1860485116 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd0ac1039b0 0x7fd0ac105da0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:07.740 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:07.738+0000 7fd0b1c00700 1 -- 172.21.15.164:0/1860485116 >> 172.21.15.164:0/1860485116 conn(0x7fd0ac0fd320 msgr2=0x7fd0ac0ff760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:07.740 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:07.738+0000 7fd0b1c00700 1 -- 172.21.15.164:0/1860485116 shutdown_connections 2024-04-26T21:44:07.740 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:07.738+0000 7fd0b1c00700 1 -- 172.21.15.164:0/1860485116 wait complete. 2024-04-26T21:44:07.741 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:07.739+0000 7fd0b1c00700 1 Processor -- start 2024-04-26T21:44:07.741 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:07.740+0000 7fd0b1c00700 1 -- start start 2024-04-26T21:44:07.741 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:07.740+0000 7fd0b1c00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd0ac1039b0 0x7fd0ac19a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:07.741 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:07.740+0000 7fd0b1c00700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fd0ac108c70 con 0x7fd0ac1039b0 2024-04-26T21:44:07.742 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:07.741+0000 7fd0aaa00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd0ac1039b0 0x7fd0ac19a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:07.742 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:07.741+0000 7fd0aaa00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd0ac1039b0 0x7fd0ac19a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.164:33136/0 (socket says 172.21.15.164:33136) 2024-04-26T21:44:07.742 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:07.741+0000 7fd0aaa00700 1 -- 172.21.15.164:0/386899114 learned_addr learned my addr 172.21.15.164:0/386899114 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-04-26T21:44:07.742 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:07.741+0000 7fd0aaa00700 1 -- 172.21.15.164:0/386899114 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0ac19ab70 con 0x7fd0ac1039b0 2024-04-26T21:44:07.743 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:07.742+0000 7fd0aaa00700 1 --2- 172.21.15.164:0/386899114 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd0ac1039b0 0x7fd0ac19a630 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fd09c000c00 tx=0x7fd09c005810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:07.744 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:07.742+0000 7fd0a3400700 1 -- 172.21.15.164:0/386899114 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd09c027070 con 0x7fd0ac1039b0 2024-04-26T21:44:07.744 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:07.743+0000 7fd0b1c00700 1 -- 172.21.15.164:0/386899114 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0ac19ae00 con 0x7fd0ac1039b0 2024-04-26T21:44:07.744 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:07.743+0000 7fd0a3400700 1 -- 172.21.15.164:0/386899114 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd09c00f460 con 0x7fd0ac1039b0 2024-04-26T21:44:07.744 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:07.743+0000 7fd0a3400700 1 -- 172.21.15.164:0/386899114 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd09c017440 con 0x7fd0ac1039b0 2024-04-26T21:44:07.744 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:07.743+0000 7fd0b1c00700 1 -- 172.21.15.164:0/386899114 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0ac19daf0 con 0x7fd0ac1039b0 2024-04-26T21:44:07.745 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:07.744+0000 7fd0b1c00700 1 -- 172.21.15.164:0/386899114 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0ac1093f0 con 0x7fd0ac1039b0 2024-04-26T21:44:07.745 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:07.744+0000 7fd0a3400700 1 -- 172.21.15.164:0/386899114 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fd09c02a070 con 0x7fd0ac1039b0 2024-04-26T21:44:07.746 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:07.745+0000 7fd0a3400700 1 --2- 172.21.15.164:0/386899114 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fd0900396b0 0x7fd09003bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:07.746 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:07.745+0000 7fd0a3400700 1 -- 172.21.15.164:0/386899114 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fd09c04d9b0 con 0x7fd0ac1039b0 2024-04-26T21:44:07.746 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:07.746+0000 7fd0aa000700 1 --2- 172.21.15.164:0/386899114 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fd0900396b0 0x7fd09003bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:07.748 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:07.747+0000 7fd0aa000700 1 --2- 172.21.15.164:0/386899114 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fd0900396b0 0x7fd09003bb70 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fd098006fd0 tx=0x7fd098006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:07.762 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:07.761+0000 7fd0a3400700 1 -- 172.21.15.164:0/386899114 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fd09c013070 con 0x7fd0ac1039b0 2024-04-26T21:44:08.018 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:08.017+0000 7fd0b1c00700 1 -- 172.21.15.164:0/386899114 --> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi072:172.21.15.72=a;smithi076:172.21.15.76=b;smithi164:172.21.15.164=c", "target": ["mon-mgr", ""]}) -- 0x7fd0ac02e720 con 0x7fd0900396b0 2024-04-26T21:44:08.035 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:08.034+0000 7fd0a3400700 1 -- 172.21.15.164:0/386899114 <== mgr.14152 v2:172.21.15.72:6800/588150172 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fd0ac02e720 con 0x7fd0900396b0 2024-04-26T21:44:08.035 INFO:teuthology.orchestra.run.smithi164.stdout:Scheduled mon update... 2024-04-26T21:44:08.038 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:08.037+0000 7fd0b1c00700 1 -- 172.21.15.164:0/386899114 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fd0900396b0 msgr2=0x7fd09003bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:08.038 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:08.037+0000 7fd0b1c00700 1 --2- 172.21.15.164:0/386899114 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fd0900396b0 0x7fd09003bb70 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fd098006fd0 tx=0x7fd098006e40 comp rx=0 tx=0).stop 2024-04-26T21:44:08.038 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:08.037+0000 7fd0b1c00700 1 -- 172.21.15.164:0/386899114 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd0ac1039b0 msgr2=0x7fd0ac19a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:08.038 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:08.037+0000 7fd0b1c00700 1 --2- 172.21.15.164:0/386899114 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd0ac1039b0 0x7fd0ac19a630 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fd09c000c00 tx=0x7fd09c005810 comp rx=0 tx=0).stop 2024-04-26T21:44:08.038 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:08.037+0000 7fd0b1c00700 1 -- 172.21.15.164:0/386899114 shutdown_connections 2024-04-26T21:44:08.038 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:08.037+0000 7fd0b1c00700 1 --2- 172.21.15.164:0/386899114 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fd0900396b0 0x7fd09003bb70 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:08.038 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:08.037+0000 7fd0b1c00700 1 --2- 172.21.15.164:0/386899114 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd0ac1039b0 0x7fd0ac19a630 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:08.039 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:08.037+0000 7fd0b1c00700 1 -- 172.21.15.164:0/386899114 >> 172.21.15.164:0/386899114 conn(0x7fd0ac0fd320 msgr2=0x7fd0ac0ff760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:08.039 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:08.037+0000 7fd0b1c00700 1 -- 172.21.15.164:0/386899114 shutdown_connections 2024-04-26T21:44:08.039 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:08.037+0000 7fd0b1c00700 1 -- 172.21.15.164:0/386899114 wait complete. 2024-04-26T21:44:08.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:07 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:08.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:07 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:08.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:07 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:08.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:07 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:08.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:07 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi164", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:44:08.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:07 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:44:08.073 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:07 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:44:08.073 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:07 smithi072 ceph-mon[28138]: Updating smithi164:/etc/ceph/ceph.conf 2024-04-26T21:44:08.802 DEBUG:teuthology.orchestra.run.smithi076:mon.b> sudo journalctl -f -n 0 -u ceph-a1780950-0415-11ef-bc93-c7b262605968@mon.b.service 2024-04-26T21:44:08.810 DEBUG:teuthology.orchestra.run.smithi164:mon.c> sudo journalctl -f -n 0 -u ceph-a1780950-0415-11ef-bc93-c7b262605968@mon.c.service 2024-04-26T21:44:08.816 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-26T21:44:08.817 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a1780950-0415-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-26T21:44:09.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:08 smithi072 ceph-mon[28138]: Updating smithi164:/var/lib/ceph/a1780950-0415-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T21:44:09.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:08 smithi072 ceph-mon[28138]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:44:09.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:08 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:09.241 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/a1780950-0415-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T21:44:10.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:09 smithi072 ceph-mon[28138]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi072:172.21.15.72=a;smithi076:172.21.15.76=b;smithi164:172.21.15.164=c", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:44:10.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:09 smithi072 ceph-mon[28138]: Saving service mon spec with placement smithi072:172.21.15.72=a;smithi076:172.21.15.76=b;smithi164:172.21.15.164=c;count:3 2024-04-26T21:44:10.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:09 smithi072 ceph-mon[28138]: Updating smithi164:/etc/ceph/ceph.client.admin.keyring 2024-04-26T21:44:10.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:09 smithi072 ceph-mon[28138]: Updating smithi164:/var/lib/ceph/a1780950-0415-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-26T21:44:10.073 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:09 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:10.073 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:09 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:10.073 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:09 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:10.073 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:09 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T21:44:10.073 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:09 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:44:11.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:10 smithi072 ceph-mon[28138]: Deploying daemon mon.c on smithi164 2024-04-26T21:44:11.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:10 smithi072 ceph-mon[28138]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:44:11.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:11.569+0000 7ff4b9800700 1 -- 172.21.15.164:0/3650044155 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff4b410e3d0 msgr2=0x7ff4b410e7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:11.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:11.569+0000 7ff4b9800700 1 --2- 172.21.15.164:0/3650044155 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff4b410e3d0 0x7ff4b410e7b0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7ff4a4009910 tx=0x7ff4a4009c20 comp rx=0 tx=0).stop 2024-04-26T21:44:11.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:11.569+0000 7ff4b9800700 1 -- 172.21.15.164:0/3650044155 shutdown_connections 2024-04-26T21:44:11.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:11.569+0000 7ff4b9800700 1 --2- 172.21.15.164:0/3650044155 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff4b410e3d0 0x7ff4b410e7b0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:11.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:11.569+0000 7ff4b9800700 1 -- 172.21.15.164:0/3650044155 >> 172.21.15.164:0/3650044155 conn(0x7ff4b4078bc0 msgr2=0x7ff4b407afe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:11.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:11.569+0000 7ff4b9800700 1 -- 172.21.15.164:0/3650044155 shutdown_connections 2024-04-26T21:44:11.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:11.570+0000 7ff4b9800700 1 -- 172.21.15.164:0/3650044155 wait complete. 2024-04-26T21:44:11.573 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:11.571+0000 7ff4b9800700 1 Processor -- start 2024-04-26T21:44:11.573 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:11.571+0000 7ff4b9800700 1 -- start start 2024-04-26T21:44:11.574 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:11.571+0000 7ff4b9800700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff4b410e3d0 0x7ff4b419edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:11.574 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:11.571+0000 7ff4b9800700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7ff4b41137f0 con 0x7ff4b410e3d0 2024-04-26T21:44:11.574 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:11.572+0000 7ff4b2a00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff4b410e3d0 0x7ff4b419edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:11.574 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:11.572+0000 7ff4b2a00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff4b410e3d0 0x7ff4b419edb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.164:56258/0 (socket says 172.21.15.164:56258) 2024-04-26T21:44:11.574 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:11.572+0000 7ff4b2a00700 1 -- 172.21.15.164:0/3741886131 learned_addr learned my addr 172.21.15.164:0/3741886131 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-04-26T21:44:11.574 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:11.573+0000 7ff4b2a00700 1 -- 172.21.15.164:0/3741886131 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4b419f2f0 con 0x7ff4b410e3d0 2024-04-26T21:44:11.575 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:11.573+0000 7ff4b2a00700 1 --2- 172.21.15.164:0/3741886131 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff4b410e3d0 0x7ff4b419edb0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ff4a4005060 tx=0x7ff4a4005710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:11.575 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:11.573+0000 7ff4ab400700 1 -- 172.21.15.164:0/3741886131 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff4a4026070 con 0x7ff4b410e3d0 2024-04-26T21:44:11.575 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:11.574+0000 7ff4b9800700 1 -- 172.21.15.164:0/3741886131 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4b419f5e0 con 0x7ff4b410e3d0 2024-04-26T21:44:11.575 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:11.574+0000 7ff4b9800700 1 -- 172.21.15.164:0/3741886131 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4b41a3680 con 0x7ff4b410e3d0 2024-04-26T21:44:11.575 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:11.574+0000 7ff4ab400700 1 -- 172.21.15.164:0/3741886131 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff4a400b6f0 con 0x7ff4b410e3d0 2024-04-26T21:44:11.575 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:11.574+0000 7ff4ab400700 1 -- 172.21.15.164:0/3741886131 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff4a4003810 con 0x7ff4b410e3d0 2024-04-26T21:44:11.576 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:11.575+0000 7ff4b9800700 1 -- 172.21.15.164:0/3741886131 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff490005330 con 0x7ff4b410e3d0 2024-04-26T21:44:11.576 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:11.575+0000 7ff4ab400700 1 -- 172.21.15.164:0/3741886131 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7ff4a4029070 con 0x7ff4b410e3d0 2024-04-26T21:44:11.577 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:11.576+0000 7ff4ab400700 1 --2- 172.21.15.164:0/3741886131 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7ff498039650 0x7ff49803bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:11.577 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:11.576+0000 7ff4ab400700 1 -- 172.21.15.164:0/3741886131 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7ff4a401e070 con 0x7ff4b410e3d0 2024-04-26T21:44:11.577 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:11.576+0000 7ff4b2000700 1 --2- 172.21.15.164:0/3741886131 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7ff498039650 0x7ff49803bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:11.578 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:11.577+0000 7ff4b2000700 1 --2- 172.21.15.164:0/3741886131 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7ff498039650 0x7ff49803bb10 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7ff49c006fd0 tx=0x7ff49c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:11.593 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:11.592+0000 7ff4ab400700 1 -- 172.21.15.164:0/3741886131 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7ff4a4013070 con 0x7ff4b410e3d0 2024-04-26T21:44:11.919 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:11.918+0000 7ff4b9800700 1 -- 172.21.15.164:0/3741886131 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7ff490005620 con 0x7ff4b410e3d0 2024-04-26T21:44:11.920 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:11.919+0000 7ff4ab400700 1 -- 172.21.15.164:0/3741886131 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+745 (secure 0 0 0) 0x7ff4a4015530 con 0x7ff4b410e3d0 2024-04-26T21:44:11.920 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-26T21:44:11.921 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":1,"fsid":"a1780950-0415-11ef-bc93-c7b262605968","modified":"2024-04-26T21:41:57.024587Z","created":"2024-04-26T21:41:57.024587Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:3300","nonce":0},{"type":"v1","addr":"172.21.15.72:6789","nonce":0}]},"addr":"172.21.15.72:6789/0","public_addr":"172.21.15.72:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-26T21:44:11.924 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:11.922+0000 7ff4b9800700 1 -- 172.21.15.164:0/3741886131 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7ff498039650 msgr2=0x7ff49803bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:11.924 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:11.922+0000 7ff4b9800700 1 --2- 172.21.15.164:0/3741886131 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7ff498039650 0x7ff49803bb10 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7ff49c006fd0 tx=0x7ff49c006e40 comp rx=0 tx=0).stop 2024-04-26T21:44:11.924 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:11.922+0000 7ff4b9800700 1 -- 172.21.15.164:0/3741886131 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff4b410e3d0 msgr2=0x7ff4b419edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:11.924 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:11.922+0000 7ff4b9800700 1 --2- 172.21.15.164:0/3741886131 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff4b410e3d0 0x7ff4b419edb0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ff4a4005060 tx=0x7ff4a4005710 comp rx=0 tx=0).stop 2024-04-26T21:44:11.924 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:11.922+0000 7ff4b9800700 1 -- 172.21.15.164:0/3741886131 shutdown_connections 2024-04-26T21:44:11.925 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:11.922+0000 7ff4b9800700 1 --2- 172.21.15.164:0/3741886131 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7ff498039650 0x7ff49803bb10 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:11.925 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:11.922+0000 7ff4b9800700 1 --2- 172.21.15.164:0/3741886131 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff4b410e3d0 0x7ff4b419edb0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:11.925 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:11.922+0000 7ff4b9800700 1 -- 172.21.15.164:0/3741886131 >> 172.21.15.164:0/3741886131 conn(0x7ff4b4078bc0 msgr2=0x7ff4b4110fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:11.925 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:11.922+0000 7ff4b9800700 1 -- 172.21.15.164:0/3741886131 shutdown_connections 2024-04-26T21:44:11.925 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:11.922+0000 7ff4b9800700 1 -- 172.21.15.164:0/3741886131 wait complete. 2024-04-26T21:44:11.925 INFO:teuthology.orchestra.run.smithi164.stderr:dumped monmap epoch 1 2024-04-26T21:44:13.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:12 smithi072 ceph-mon[28138]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:44:13.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:12 smithi072 ceph-mon[28138]: from='client.? 172.21.15.164:0/3741886131' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-26T21:44:14.785 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-26T21:44:14.786 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a1780950-0415-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-26T21:44:15.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:14 smithi072 ceph-mon[28138]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:44:15.277 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/a1780950-0415-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T21:44:16.263 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:16 smithi164 systemd[1]: Starting Ceph mon.c for a1780950-0415-11ef-bc93-c7b262605968... 2024-04-26T21:44:16.860 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:16 smithi164 podman[31873]: 2024-04-26 21:44:16.4854676 +0000 UTC m=+0.169832271 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-26T21:44:16.860 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:16 smithi164 podman[31873]: 2024-04-26 21:44:16.702623269 +0000 UTC m=+0.386988012 container create acf9ab0946806dac3b7a628e7e123c99f2530d3e593a61757b9b660f139eaf31 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-mon-c, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-04-26T21:44:17.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:16 smithi072 ceph-mon[28138]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:44:17.442 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 podman[31873]: 2024-04-26 21:44:17.223862623 +0000 UTC m=+0.908227363 container init acf9ab0946806dac3b7a628e7e123c99f2530d3e593a61757b9b660f139eaf31 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-mon-c, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240424, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T21:44:17.442 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 podman[31873]: 2024-04-26 21:44:17.235955282 +0000 UTC m=+0.920319969 container start acf9ab0946806dac3b7a628e7e123c99f2530d3e593a61757b9b660f139eaf31 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-mon-c, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T21:44:17.442 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: set uid:gid to 167:167 (ceph:ceph) 2024-04-26T21:44:17.442 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: ceph version 19.0.0-3306-gbc612ffb (bc612ffb9c1d5bc27536bc917a705262a36d4387) squid (dev), process ceph-mon, pid 2 2024-04-26T21:44:17.442 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: pidfile_write: ignore empty --pid-file 2024-04-26T21:44:17.442 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: load: jerasure load: lrc 2024-04-26T21:44:17.442 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: RocksDB version: 7.9.2 2024-04-26T21:44:17.442 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: Git sha 0 2024-04-26T21:44:17.442 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: Compile date 2024-04-26 11:39:23 2024-04-26T21:44:17.442 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: DB SUMMARY 2024-04-26T21:44:17.443 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: DB Session ID: 50M457RDXER2S4FHNRCP 2024-04-26T21:44:17.443 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: CURRENT file: CURRENT 2024-04-26T21:44:17.443 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: IDENTITY file: IDENTITY 2024-04-26T21:44:17.443 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-04-26T21:44:17.443 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-04-26T21:44:17.443 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2024-04-26T21:44:17.443 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: Options.error_if_exists: 0 2024-04-26T21:44:17.443 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: Options.create_if_missing: 0 2024-04-26T21:44:17.443 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: Options.paranoid_checks: 1 2024-04-26T21:44:17.443 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-26T21:44:17.443 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-26T21:44:17.443 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-26T21:44:17.443 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: Options.env: 0x55d8fc93bcc0 2024-04-26T21:44:17.443 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: Options.fs: PosixFileSystem 2024-04-26T21:44:17.444 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: Options.info_log: 0x55d929687dc0 2024-04-26T21:44:17.444 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: Options.max_file_opening_threads: 16 2024-04-26T21:44:17.444 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: Options.statistics: (nil) 2024-04-26T21:44:17.444 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: Options.use_fsync: 0 2024-04-26T21:44:17.444 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: Options.max_log_file_size: 0 2024-04-26T21:44:17.444 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-26T21:44:17.445 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-26T21:44:17.445 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: Options.keep_log_file_num: 1000 2024-04-26T21:44:17.445 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: Options.recycle_log_file_num: 0 2024-04-26T21:44:17.445 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: Options.allow_fallocate: 1 2024-04-26T21:44:17.445 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: Options.allow_mmap_reads: 0 2024-04-26T21:44:17.445 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: Options.allow_mmap_writes: 0 2024-04-26T21:44:17.445 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: Options.use_direct_reads: 0 2024-04-26T21:44:17.445 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-26T21:44:17.445 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: Options.create_missing_column_families: 0 2024-04-26T21:44:17.445 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: Options.db_log_dir: 2024-04-26T21:44:17.445 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: Options.wal_dir: 2024-04-26T21:44:17.445 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-26T21:44:17.445 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-26T21:44:17.446 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-26T21:44:17.446 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-26T21:44:17.446 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-26T21:44:17.446 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-26T21:44:17.446 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: Options.advise_random_on_open: 1 2024-04-26T21:44:17.446 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: Options.db_write_buffer_size: 0 2024-04-26T21:44:17.446 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: Options.write_buffer_manager: 0x55d9296590e0 2024-04-26T21:44:17.446 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-26T21:44:17.446 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-26T21:44:17.446 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-26T21:44:17.446 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: Options.rate_limiter: (nil) 2024-04-26T21:44:17.446 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-26T21:44:17.446 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: Options.wal_recovery_mode: 2 2024-04-26T21:44:17.447 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: Options.enable_thread_tracking: 0 2024-04-26T21:44:17.447 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: Options.enable_pipelined_write: 0 2024-04-26T21:44:17.447 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: Options.unordered_write: 0 2024-04-26T21:44:17.447 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-26T21:44:17.447 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-26T21:44:17.447 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-26T21:44:17.447 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-26T21:44:17.447 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: Options.row_cache: None 2024-04-26T21:44:17.447 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: Options.wal_filter: None 2024-04-26T21:44:17.447 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-26T21:44:17.447 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: Options.allow_ingest_behind: 0 2024-04-26T21:44:17.447 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: Options.two_write_queues: 0 2024-04-26T21:44:17.448 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: Options.manual_wal_flush: 0 2024-04-26T21:44:17.448 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: Options.wal_compression: 0 2024-04-26T21:44:17.448 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: Options.atomic_flush: 0 2024-04-26T21:44:17.448 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-26T21:44:17.448 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-26T21:44:17.448 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-26T21:44:17.448 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: Options.log_readahead_size: 0 2024-04-26T21:44:17.448 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-26T21:44:17.448 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: Options.best_efforts_recovery: 0 2024-04-26T21:44:17.448 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-26T21:44:17.448 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-26T21:44:17.448 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: Options.allow_data_in_errors: 0 2024-04-26T21:44:17.448 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: Options.db_host_id: __hostname__ 2024-04-26T21:44:17.449 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-26T21:44:17.449 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: Options.max_background_jobs: 2 2024-04-26T21:44:17.449 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: Options.max_background_compactions: -1 2024-04-26T21:44:17.449 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: Options.max_subcompactions: 1 2024-04-26T21:44:17.449 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-26T21:44:17.449 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-26T21:44:17.449 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-26T21:44:17.449 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: Options.max_total_wal_size: 0 2024-04-26T21:44:17.449 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-26T21:44:17.449 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-26T21:44:17.449 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-26T21:44:17.449 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-26T21:44:17.449 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: Options.max_open_files: -1 2024-04-26T21:44:17.449 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: Options.bytes_per_sync: 0 2024-04-26T21:44:17.450 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-26T21:44:17.450 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-26T21:44:17.450 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: Options.compaction_readahead_size: 0 2024-04-26T21:44:17.450 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: Options.max_background_flushes: -1 2024-04-26T21:44:17.450 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: Compression algorithms supported: 2024-04-26T21:44:17.450 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-26T21:44:17.450 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: kZSTD supported: 0 2024-04-26T21:44:17.450 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: kXpressCompression supported: 0 2024-04-26T21:44:17.450 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: kLZ4HCCompression supported: 1 2024-04-26T21:44:17.450 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: kZlibCompression supported: 1 2024-04-26T21:44:17.450 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: kSnappyCompression supported: 1 2024-04-26T21:44:17.451 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: kLZ4Compression supported: 1 2024-04-26T21:44:17.451 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: kBZip2Compression supported: 0 2024-04-26T21:44:17.451 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-26T21:44:17.452 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-26T21:44:17.452 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-04-26T21:44:17.452 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-26T21:44:17.452 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-26T21:44:17.452 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: Options.merge_operator: 2024-04-26T21:44:17.452 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: Options.compaction_filter: None 2024-04-26T21:44:17.452 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: Options.compaction_filter_factory: None 2024-04-26T21:44:17.452 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: Options.sst_partitioner_factory: None 2024-04-26T21:44:17.452 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-26T21:44:17.452 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-26T21:44:17.452 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55d929687f80) 2024-04-26T21:44:17.452 INFO:journalctl@ceph.mon.c.smithi164.stdout: cache_index_and_filter_blocks: 1 2024-04-26T21:44:17.452 INFO:journalctl@ceph.mon.c.smithi164.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-26T21:44:17.453 INFO:journalctl@ceph.mon.c.smithi164.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-26T21:44:17.453 INFO:journalctl@ceph.mon.c.smithi164.stdout: pin_top_level_index_and_filter: 1 2024-04-26T21:44:17.453 INFO:journalctl@ceph.mon.c.smithi164.stdout: index_type: 0 2024-04-26T21:44:17.453 INFO:journalctl@ceph.mon.c.smithi164.stdout: data_block_index_type: 0 2024-04-26T21:44:17.453 INFO:journalctl@ceph.mon.c.smithi164.stdout: index_shortening: 1 2024-04-26T21:44:17.453 INFO:journalctl@ceph.mon.c.smithi164.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-26T21:44:17.453 INFO:journalctl@ceph.mon.c.smithi164.stdout: checksum: 4 2024-04-26T21:44:17.453 INFO:journalctl@ceph.mon.c.smithi164.stdout: no_block_cache: 0 2024-04-26T21:44:17.453 INFO:journalctl@ceph.mon.c.smithi164.stdout: block_cache: 0x55d92968b1f0 2024-04-26T21:44:17.453 INFO:journalctl@ceph.mon.c.smithi164.stdout: block_cache_name: BinnedLRUCache 2024-04-26T21:44:17.453 INFO:journalctl@ceph.mon.c.smithi164.stdout: block_cache_options: 2024-04-26T21:44:17.453 INFO:journalctl@ceph.mon.c.smithi164.stdout: capacity : 536870912 2024-04-26T21:44:17.453 INFO:journalctl@ceph.mon.c.smithi164.stdout: num_shard_bits : 4 2024-04-26T21:44:17.453 INFO:journalctl@ceph.mon.c.smithi164.stdout: strict_capacity_limit : 0 2024-04-26T21:44:17.454 INFO:journalctl@ceph.mon.c.smithi164.stdout: high_pri_pool_ratio: 0.000 2024-04-26T21:44:17.454 INFO:journalctl@ceph.mon.c.smithi164.stdout: block_cache_compressed: (nil) 2024-04-26T21:44:17.454 INFO:journalctl@ceph.mon.c.smithi164.stdout: persistent_cache: (nil) 2024-04-26T21:44:17.454 INFO:journalctl@ceph.mon.c.smithi164.stdout: block_size: 4096 2024-04-26T21:44:17.454 INFO:journalctl@ceph.mon.c.smithi164.stdout: block_size_deviation: 10 2024-04-26T21:44:17.454 INFO:journalctl@ceph.mon.c.smithi164.stdout: block_restart_interval: 16 2024-04-26T21:44:17.454 INFO:journalctl@ceph.mon.c.smithi164.stdout: index_block_restart_interval: 1 2024-04-26T21:44:17.454 INFO:journalctl@ceph.mon.c.smithi164.stdout: metadata_block_size: 4096 2024-04-26T21:44:17.454 INFO:journalctl@ceph.mon.c.smithi164.stdout: partition_filters: 0 2024-04-26T21:44:17.454 INFO:journalctl@ceph.mon.c.smithi164.stdout: use_delta_encoding: 1 2024-04-26T21:44:17.454 INFO:journalctl@ceph.mon.c.smithi164.stdout: filter_policy: bloomfilter 2024-04-26T21:44:17.454 INFO:journalctl@ceph.mon.c.smithi164.stdout: whole_key_filtering: 1 2024-04-26T21:44:17.454 INFO:journalctl@ceph.mon.c.smithi164.stdout: verify_compression: 0 2024-04-26T21:44:17.455 INFO:journalctl@ceph.mon.c.smithi164.stdout: read_amp_bytes_per_bit: 0 2024-04-26T21:44:17.455 INFO:journalctl@ceph.mon.c.smithi164.stdout: format_version: 5 2024-04-26T21:44:17.455 INFO:journalctl@ceph.mon.c.smithi164.stdout: enable_index_compression: 1 2024-04-26T21:44:17.455 INFO:journalctl@ceph.mon.c.smithi164.stdout: block_align: 0 2024-04-26T21:44:17.455 INFO:journalctl@ceph.mon.c.smithi164.stdout: max_auto_readahead_size: 262144 2024-04-26T21:44:17.455 INFO:journalctl@ceph.mon.c.smithi164.stdout: prepopulate_block_cache: 0 2024-04-26T21:44:17.455 INFO:journalctl@ceph.mon.c.smithi164.stdout: initial_auto_readahead_size: 8192 2024-04-26T21:44:17.455 INFO:journalctl@ceph.mon.c.smithi164.stdout: num_file_reads_for_auto_readahead: 2 2024-04-26T21:44:17.455 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: Options.write_buffer_size: 33554432 2024-04-26T21:44:17.455 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: Options.max_write_buffer_number: 2 2024-04-26T21:44:17.455 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: Options.compression: NoCompression 2024-04-26T21:44:17.455 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: Options.bottommost_compression: Disabled 2024-04-26T21:44:17.455 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: Options.prefix_extractor: nullptr 2024-04-26T21:44:17.455 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-26T21:44:17.456 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: Options.num_levels: 7 2024-04-26T21:44:17.456 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-26T21:44:17.456 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-26T21:44:17.456 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-26T21:44:17.456 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-26T21:44:17.456 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-26T21:44:17.456 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-26T21:44:17.456 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-26T21:44:17.456 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-26T21:44:17.456 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-26T21:44:17.456 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-26T21:44:17.456 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-26T21:44:17.456 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-26T21:44:17.457 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-26T21:44:17.457 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: Options.compression_opts.level: 32767 2024-04-26T21:44:17.457 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: Options.compression_opts.strategy: 0 2024-04-26T21:44:17.457 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-26T21:44:17.457 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-26T21:44:17.457 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-26T21:44:17.457 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-26T21:44:17.457 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: Options.compression_opts.enabled: false 2024-04-26T21:44:17.457 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-26T21:44:17.457 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-26T21:44:17.457 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-26T21:44:17.457 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-26T21:44:17.457 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: Options.target_file_size_base: 67108864 2024-04-26T21:44:17.457 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-26T21:44:17.458 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-26T21:44:17.458 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-26T21:44:17.458 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-26T21:44:17.458 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-26T21:44:17.458 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-26T21:44:17.458 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-26T21:44:17.458 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-26T21:44:17.458 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-26T21:44:17.458 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-26T21:44:17.458 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-26T21:44:17.458 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-26T21:44:17.458 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-26T21:44:17.458 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-26T21:44:17.458 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: Options.arena_block_size: 1048576 2024-04-26T21:44:17.459 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-26T21:44:17.459 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-26T21:44:17.459 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: Options.disable_auto_compactions: 0 2024-04-26T21:44:17.459 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-26T21:44:17.459 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-26T21:44:17.459 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-26T21:44:17.459 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-26T21:44:17.459 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-26T21:44:17.459 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-26T21:44:17.459 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-26T21:44:17.459 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-26T21:44:17.459 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-26T21:44:17.459 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-26T21:44:17.460 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-04-26T21:44:17.460 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: Options.inplace_update_support: 0 2024-04-26T21:44:17.460 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-26T21:44:17.460 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-26T21:44:17.460 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-26T21:44:17.460 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-26T21:44:17.460 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: Options.bloom_locality: 0 2024-04-26T21:44:17.460 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: Options.max_successive_merges: 0 2024-04-26T21:44:17.460 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-26T21:44:17.460 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: Options.paranoid_file_checks: 0 2024-04-26T21:44:17.460 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: Options.force_consistency_checks: 1 2024-04-26T21:44:17.460 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: Options.report_bg_io_stats: 0 2024-04-26T21:44:17.460 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: Options.ttl: 2592000 2024-04-26T21:44:17.460 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-26T21:44:17.461 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-26T21:44:17.461 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-26T21:44:17.461 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: Options.enable_blob_files: false 2024-04-26T21:44:17.461 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: Options.min_blob_size: 0 2024-04-26T21:44:17.461 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: Options.blob_file_size: 268435456 2024-04-26T21:44:17.461 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-26T21:44:17.461 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-26T21:44:17.461 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-26T21:44:17.461 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-26T21:44:17.461 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-26T21:44:17.461 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: Options.blob_file_starting_level: 0 2024-04-26T21:44:17.461 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-26T21:44:17.461 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-04-26T21:44:17.461 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-04-26T21:44:17.462 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 2726be1e-a24f-4479-8534-40fd9c3894cb 2024-04-26T21:44:17.462 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714167857294483, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-04-26T21:44:17.462 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-04-26T21:44:17.462 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714167857295393, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1648, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1714167857, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "2726be1e-a24f-4479-8534-40fd9c3894cb", "db_session_id": "50M457RDXER2S4FHNRCP", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-04-26T21:44:17.462 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714167857295669, "job": 1, "event": "recovery_finished"} 2024-04-26T21:44:17.462 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-04-26T21:44:17.462 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-26T21:44:17.462 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55d9297a8000 2024-04-26T21:44:17.462 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: DB pointer 0x55d929794000 2024-04-26T21:44:17.462 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-26T21:44:17.462 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-26T21:44:17.462 INFO:journalctl@ceph.mon.c.smithi164.stdout: ** DB Stats ** 2024-04-26T21:44:17.462 INFO:journalctl@ceph.mon.c.smithi164.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-26T21:44:17.463 INFO:journalctl@ceph.mon.c.smithi164.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-26T21:44:17.463 INFO:journalctl@ceph.mon.c.smithi164.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-26T21:44:17.463 INFO:journalctl@ceph.mon.c.smithi164.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-26T21:44:17.463 INFO:journalctl@ceph.mon.c.smithi164.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-26T21:44:17.463 INFO:journalctl@ceph.mon.c.smithi164.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-26T21:44:17.463 INFO:journalctl@ceph.mon.c.smithi164.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-26T21:44:17.463 INFO:journalctl@ceph.mon.c.smithi164.stdout: 2024-04-26T21:44:17.463 INFO:journalctl@ceph.mon.c.smithi164.stdout: ** Compaction Stats [default] ** 2024-04-26T21:44:17.463 INFO:journalctl@ceph.mon.c.smithi164.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-26T21:44:17.463 INFO:journalctl@ceph.mon.c.smithi164.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-26T21:44:17.463 INFO:journalctl@ceph.mon.c.smithi164.stdout: L0 1/0 1.61 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-26T21:44:17.463 INFO:journalctl@ceph.mon.c.smithi164.stdout: Sum 1/0 1.61 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-26T21:44:17.463 INFO:journalctl@ceph.mon.c.smithi164.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-26T21:44:17.464 INFO:journalctl@ceph.mon.c.smithi164.stdout: 2024-04-26T21:44:17.464 INFO:journalctl@ceph.mon.c.smithi164.stdout: ** Compaction Stats [default] ** 2024-04-26T21:44:17.464 INFO:journalctl@ceph.mon.c.smithi164.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-26T21:44:17.464 INFO:journalctl@ceph.mon.c.smithi164.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-26T21:44:17.464 INFO:journalctl@ceph.mon.c.smithi164.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-26T21:44:17.464 INFO:journalctl@ceph.mon.c.smithi164.stdout: 2024-04-26T21:44:17.464 INFO:journalctl@ceph.mon.c.smithi164.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-26T21:44:17.464 INFO:journalctl@ceph.mon.c.smithi164.stdout: 2024-04-26T21:44:17.464 INFO:journalctl@ceph.mon.c.smithi164.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-26T21:44:17.464 INFO:journalctl@ceph.mon.c.smithi164.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-26T21:44:17.464 INFO:journalctl@ceph.mon.c.smithi164.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-26T21:44:17.464 INFO:journalctl@ceph.mon.c.smithi164.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-26T21:44:17.464 INFO:journalctl@ceph.mon.c.smithi164.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-26T21:44:17.464 INFO:journalctl@ceph.mon.c.smithi164.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-26T21:44:17.465 INFO:journalctl@ceph.mon.c.smithi164.stdout: Cumulative compaction: 0.00 GB write, 0.28 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-26T21:44:17.465 INFO:journalctl@ceph.mon.c.smithi164.stdout: Interval compaction: 0.00 GB write, 0.28 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-26T21:44:17.465 INFO:journalctl@ceph.mon.c.smithi164.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-26T21:44:17.465 INFO:journalctl@ceph.mon.c.smithi164.stdout: Block cache BinnedLRUCache@0x55d92968b1f0#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.2e-05 secs_since: 0 2024-04-26T21:44:17.465 INFO:journalctl@ceph.mon.c.smithi164.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-04-26T21:44:17.465 INFO:journalctl@ceph.mon.c.smithi164.stdout: 2024-04-26T21:44:17.465 INFO:journalctl@ceph.mon.c.smithi164.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-26T21:44:17.465 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-04-26T21:44:17.465 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: using public_addr v2:172.21.15.164:0/0 -> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] 2024-04-26T21:44:17.465 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: starting mon.c rank -1 at public addrs [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] at bind addrs [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid a1780950-0415-11ef-bc93-c7b262605968 2024-04-26T21:44:17.465 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: mon.c@-1(???) e0 preinit fsid a1780950-0415-11ef-bc93-c7b262605968 2024-04-26T21:44:17.465 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: mon.c@-1(synchronizing).mds e1 new map 2024-04-26T21:44:17.465 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: mon.c@-1(synchronizing).mds e1 print_map 2024-04-26T21:44:17.465 INFO:journalctl@ceph.mon.c.smithi164.stdout: e1 2024-04-26T21:44:17.466 INFO:journalctl@ceph.mon.c.smithi164.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-26T21:44:17.466 INFO:journalctl@ceph.mon.c.smithi164.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-26T21:44:17.466 INFO:journalctl@ceph.mon.c.smithi164.stdout: legacy client fscid: -1 2024-04-26T21:44:17.466 INFO:journalctl@ceph.mon.c.smithi164.stdout: 2024-04-26T21:44:17.466 INFO:journalctl@ceph.mon.c.smithi164.stdout: No filesystems configured 2024-04-26T21:44:17.466 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-04-26T21:44:17.466 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-04-26T21:44:17.466 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-04-26T21:44:17.466 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-04-26T21:44:17.466 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-04-26T21:44:17.466 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-04-26T21:44:17.466 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-26T21:44:17.466 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-26T21:44:17.466 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-26T21:44:17.467 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-26T21:44:17.467 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: Updating smithi072:/var/lib/ceph/a1780950-0415-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-26T21:44:17.467 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:44:17.467 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:17.467 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:17.467 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:17.468 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi076", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:44:17.468 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:44:17.468 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: Deploying cephadm binary to smithi076 2024-04-26T21:44:17.468 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:44:17.468 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:44:17.468 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:17.468 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: Added host smithi076 2024-04-26T21:44:17.468 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:44:17.468 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:44:17.468 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:17.468 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:17.468 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:44:17.468 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T21:44:17.469 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:17.469 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:44:17.469 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:44:17.469 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi164", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:44:17.469 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: Deploying cephadm binary to smithi164 2024-04-26T21:44:17.469 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:44:17.469 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:17.469 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:17.469 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:17.469 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:17.469 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi076", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:44:17.469 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:44:17.469 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:44:17.470 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: Updating smithi076:/etc/ceph/ceph.conf 2024-04-26T21:44:17.470 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:44:17.470 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: Updating smithi076:/var/lib/ceph/a1780950-0415-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T21:44:17.470 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:44:17.470 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:17.470 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: Updating smithi076:/etc/ceph/ceph.client.admin.keyring 2024-04-26T21:44:17.470 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: Added host smithi164 2024-04-26T21:44:17.470 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: Updating smithi076:/var/lib/ceph/a1780950-0415-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-26T21:44:17.470 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:17.470 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:17.470 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:17.470 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:44:17.470 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:44:17.471 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:17.471 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:17.471 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:44:17.471 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T21:44:17.471 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:44:17.471 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:17.471 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: from='client.? 172.21.15.72:0/3022658462' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-26T21:44:17.471 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:44:17.471 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: from='client.? 172.21.15.72:0/3022658462' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-26T21:44:17.471 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: osdmap e4: 0 total, 0 up, 0 in 2024-04-26T21:44:17.471 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:44:17.471 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:17.471 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:17.471 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:17.472 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:17.472 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi164", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:44:17.472 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:44:17.472 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:44:17.472 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: Updating smithi164:/etc/ceph/ceph.conf 2024-04-26T21:44:17.472 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: Updating smithi164:/var/lib/ceph/a1780950-0415-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T21:44:17.472 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:44:17.472 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:17.472 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi072:172.21.15.72=a;smithi076:172.21.15.76=b;smithi164:172.21.15.164=c", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:44:17.472 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: Saving service mon spec with placement smithi072:172.21.15.72=a;smithi076:172.21.15.76=b;smithi164:172.21.15.164=c;count:3 2024-04-26T21:44:17.472 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: Updating smithi164:/etc/ceph/ceph.client.admin.keyring 2024-04-26T21:44:17.472 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: Updating smithi164:/var/lib/ceph/a1780950-0415-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-26T21:44:17.472 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:17.472 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:17.473 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:17.473 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T21:44:17.473 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:44:17.473 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: Deploying daemon mon.c on smithi164 2024-04-26T21:44:17.473 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:44:17.473 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:44:17.473 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: from='client.? 172.21.15.164:0/3741886131' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-26T21:44:17.473 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:44:17.473 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:44:17.473 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 ceph-mon[31916]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-04-26T21:44:17.940 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 bash[31873]: acf9ab0946806dac3b7a628e7e123c99f2530d3e593a61757b9b660f139eaf31 2024-04-26T21:44:17.941 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:17 smithi164 systemd[1]: Started Ceph mon.c for a1780950-0415-11ef-bc93-c7b262605968. 2024-04-26T21:44:22.692 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:22 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T21:44:22.692 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:22 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T21:44:22.692 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:22 smithi164 ceph-mon[31916]: mon.a calling monitor election 2024-04-26T21:44:22.692 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:22 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T21:44:22.692 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:22 smithi164 ceph-mon[31916]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:44:22.692 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:22 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T21:44:22.692 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:22 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T21:44:22.692 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:22 smithi164 ceph-mon[31916]: mon.c calling monitor election 2024-04-26T21:44:22.692 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:22 smithi164 ceph-mon[31916]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:44:22.692 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:22 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T21:44:22.692 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:22 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T21:44:22.693 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:22 smithi164 ceph-mon[31916]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:44:22.693 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:22 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T21:44:22.693 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:22 smithi164 ceph-mon[31916]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-26T21:44:22.693 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:22 smithi164 ceph-mon[31916]: monmap epoch 2 2024-04-26T21:44:22.693 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:22 smithi164 ceph-mon[31916]: fsid a1780950-0415-11ef-bc93-c7b262605968 2024-04-26T21:44:22.693 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:22 smithi164 ceph-mon[31916]: last_changed 2024-04-26T21:44:17.350038+0000 2024-04-26T21:44:22.693 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:22 smithi164 ceph-mon[31916]: created 2024-04-26T21:41:57.024587+0000 2024-04-26T21:44:22.693 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:22 smithi164 ceph-mon[31916]: min_mon_release 19 (squid) 2024-04-26T21:44:22.693 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:22 smithi164 ceph-mon[31916]: election_strategy: 1 2024-04-26T21:44:22.693 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:22 smithi164 ceph-mon[31916]: 0: [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] mon.a 2024-04-26T21:44:22.693 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:22 smithi164 ceph-mon[31916]: 1: [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] mon.c 2024-04-26T21:44:22.693 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:22 smithi164 ceph-mon[31916]: fsmap 2024-04-26T21:44:22.693 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:22 smithi164 ceph-mon[31916]: osdmap e4: 0 total, 0 up, 0 in 2024-04-26T21:44:22.694 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:22 smithi164 ceph-mon[31916]: mgrmap e13: x(active, since 72s) 2024-04-26T21:44:22.694 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:22 smithi164 ceph-mon[31916]: overall HEALTH_OK 2024-04-26T21:44:22.694 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:22 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:22.694 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:22 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:22.694 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:22 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:22.694 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:22 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T21:44:22.694 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:22 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:44:22.823 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:22 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T21:44:22.823 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:22 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T21:44:22.823 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:22 smithi072 ceph-mon[28138]: mon.a calling monitor election 2024-04-26T21:44:22.823 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:22 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T21:44:22.823 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:22 smithi072 ceph-mon[28138]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:44:22.823 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:22 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T21:44:22.823 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:22 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T21:44:22.823 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:22 smithi072 ceph-mon[28138]: mon.c calling monitor election 2024-04-26T21:44:22.824 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:22 smithi072 ceph-mon[28138]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:44:22.824 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:22 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T21:44:22.824 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:22 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T21:44:22.824 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:22 smithi072 ceph-mon[28138]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:44:22.824 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:22 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T21:44:22.824 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:22 smithi072 ceph-mon[28138]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-26T21:44:22.824 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:22 smithi072 ceph-mon[28138]: monmap epoch 2 2024-04-26T21:44:22.824 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:22 smithi072 ceph-mon[28138]: fsid a1780950-0415-11ef-bc93-c7b262605968 2024-04-26T21:44:22.824 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:22 smithi072 ceph-mon[28138]: last_changed 2024-04-26T21:44:17.350038+0000 2024-04-26T21:44:22.824 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:22 smithi072 ceph-mon[28138]: created 2024-04-26T21:41:57.024587+0000 2024-04-26T21:44:22.824 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:22 smithi072 ceph-mon[28138]: min_mon_release 19 (squid) 2024-04-26T21:44:22.824 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:22 smithi072 ceph-mon[28138]: election_strategy: 1 2024-04-26T21:44:22.824 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:22 smithi072 ceph-mon[28138]: 0: [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] mon.a 2024-04-26T21:44:22.824 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:22 smithi072 ceph-mon[28138]: 1: [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] mon.c 2024-04-26T21:44:22.825 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:22 smithi072 ceph-mon[28138]: fsmap 2024-04-26T21:44:22.825 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:22 smithi072 ceph-mon[28138]: osdmap e4: 0 total, 0 up, 0 in 2024-04-26T21:44:22.825 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:22 smithi072 ceph-mon[28138]: mgrmap e13: x(active, since 72s) 2024-04-26T21:44:22.825 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:22 smithi072 ceph-mon[28138]: overall HEALTH_OK 2024-04-26T21:44:22.825 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:22 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:22.825 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:22 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:22.825 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:22 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:22.825 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:22 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T21:44:22.825 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:22 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:44:23.481 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:23.478+0000 7f5817200700 1 -- 172.21.15.164:0/3873070208 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f57f4005c90 msgr2=0x7f57f4008120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:23.481 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:23.478+0000 7f5817200700 1 --2- 172.21.15.164:0/3873070208 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f57f4005c90 0x7f57f4008120 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f58101032a0 tx=0x7f580000ff20 comp rx=0 tx=0).stop 2024-04-26T21:44:23.481 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:23.479+0000 7f5817200700 1 -- 172.21.15.164:0/3873070208 shutdown_connections 2024-04-26T21:44:23.481 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:23.479+0000 7f5817200700 1 --2- 172.21.15.164:0/3873070208 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f57f4005c90 0x7f57f4008120 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:23.482 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:23.479+0000 7f5817200700 1 -- 172.21.15.164:0/3873070208 >> 172.21.15.164:0/3873070208 conn(0x7f58100fd900 msgr2=0x7f58100ffd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:23.482 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:23.479+0000 7f5817200700 1 -- 172.21.15.164:0/3873070208 shutdown_connections 2024-04-26T21:44:23.482 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:23.479+0000 7f5817200700 1 -- 172.21.15.164:0/3873070208 wait complete. 2024-04-26T21:44:23.485 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:23.481+0000 7f5817200700 1 Processor -- start 2024-04-26T21:44:23.485 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:23.481+0000 7f5817200700 1 -- start start 2024-04-26T21:44:23.486 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:23.482+0000 7f5817200700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f57f4005c90 0x7f5810103b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:23.486 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:23.482+0000 7f5817200700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5810104050 0x7f58101044d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:23.486 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:23.482+0000 7f5817200700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f5810109bf0 con 0x7f5810104050 2024-04-26T21:44:23.486 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:23.482+0000 7f5817200700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f5810109a70 con 0x7f57f4005c90 2024-04-26T21:44:23.487 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:23.483+0000 7f580fe00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f57f4005c90 0x7f5810103b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:23.487 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:23.483+0000 7f580fe00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f57f4005c90 0x7f5810103b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:48862/0 (socket says 172.21.15.164:48862) 2024-04-26T21:44:23.487 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:23.483+0000 7f580fe00700 1 -- 172.21.15.164:0/141185299 learned_addr learned my addr 172.21.15.164:0/141185299 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-04-26T21:44:23.487 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:23.483+0000 7f580f400700 1 --2- 172.21.15.164:0/141185299 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5810104050 0x7f58101044d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:23.487 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:23.483+0000 7f580fe00700 1 -- 172.21.15.164:0/141185299 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f57f4005c90 msgr2=0x7f5810103b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-04-26T21:44:23.487 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:23.483+0000 7f580fe00700 1 -- 172.21.15.164:0/141185299 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f57f4005c90 msgr2=0x7f5810103b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-26T21:44:23.487 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:23.483+0000 7f580fe00700 1 --2- 172.21.15.164:0/141185299 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f57f4005c90 0x7f5810103b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-26T21:44:23.487 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:23.483+0000 7f580fe00700 1 --2- 172.21.15.164:0/141185299 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f57f4005c90 0x7f5810103b10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T21:44:23.487 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:23.484+0000 7f580f400700 1 -- 172.21.15.164:0/141185299 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f57f4005c90 msgr2=0x7f5810103b10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T21:44:23.487 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:23.484+0000 7f580f400700 1 --2- 172.21.15.164:0/141185299 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f57f4005c90 0x7f5810103b10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:23.487 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:23.484+0000 7f580f400700 1 -- 172.21.15.164:0/141185299 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5810105a30 con 0x7f5810104050 2024-04-26T21:44:23.487 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:23.484+0000 7f580f400700 1 --2- 172.21.15.164:0/141185299 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5810104050 0x7f58101044d0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f57fc00b4d0 tx=0x7f57fc00b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:23.488 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:23.485+0000 7f580cc00700 1 -- 172.21.15.164:0/141185299 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f57fc011630 con 0x7f5810104050 2024-04-26T21:44:23.488 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:23.485+0000 7f5817200700 1 -- 172.21.15.164:0/141185299 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58101a4ca0 con 0x7f5810104050 2024-04-26T21:44:23.488 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:23.485+0000 7f580cc00700 1 -- 172.21.15.164:0/141185299 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f57fc011c60 con 0x7f5810104050 2024-04-26T21:44:23.488 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:23.485+0000 7f580cc00700 1 -- 172.21.15.164:0/141185299 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f57fc010c90 con 0x7f5810104050 2024-04-26T21:44:23.488 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:23.485+0000 7f5817200700 1 -- 172.21.15.164:0/141185299 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58101a51e0 con 0x7f5810104050 2024-04-26T21:44:23.489 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:23.486+0000 7f5817200700 1 -- 172.21.15.164:0/141185299 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5810103550 con 0x7f5810104050 2024-04-26T21:44:23.489 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:23.487+0000 7f580cc00700 1 -- 172.21.15.164:0/141185299 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f57fc018020 con 0x7f5810104050 2024-04-26T21:44:23.489 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:23.487+0000 7f580cc00700 1 --2- 172.21.15.164:0/141185299 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f57f0039440 0x7f57f003b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:23.489 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:23.487+0000 7f580cc00700 1 -- 172.21.15.164:0/141185299 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f57fc04c180 con 0x7f5810104050 2024-04-26T21:44:23.489 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:23.488+0000 7f580fe00700 1 --2- 172.21.15.164:0/141185299 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f57f0039440 0x7f57f003b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:23.489 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:23.489+0000 7f580fe00700 1 --2- 172.21.15.164:0/141185299 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f57f0039440 0x7f57f003b900 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f580000fad0 tx=0x7f5800014040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:23.506 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:23.505+0000 7f580cc00700 1 -- 172.21.15.164:0/141185299 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f57fc01f020 con 0x7f5810104050 2024-04-26T21:44:23.822 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:23 smithi072 ceph-mon[28138]: Deploying daemon mon.b on smithi076 2024-04-26T21:44:23.822 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:23 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T21:44:23.822 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:23.821+0000 7f5817200700 1 -- 172.21.15.164:0/141185299 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f58101a5560 con 0x7f5810104050 2024-04-26T21:44:23.823 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:23 smithi164 ceph-mon[31916]: Deploying daemon mon.b on smithi076 2024-04-26T21:44:23.823 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:23 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T21:44:23.823 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:23.823+0000 7f580cc00700 1 -- 172.21.15.164:0/141185299 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7f57fc014170 con 0x7f5810104050 2024-04-26T21:44:23.823 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-26T21:44:23.824 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":2,"fsid":"a1780950-0415-11ef-bc93-c7b262605968","modified":"2024-04-26T21:44:17.350038Z","created":"2024-04-26T21:41:57.024587Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:3300","nonce":0},{"type":"v1","addr":"172.21.15.72:6789","nonce":0}]},"addr":"172.21.15.72:6789/0","public_addr":"172.21.15.72:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:3300","nonce":0},{"type":"v1","addr":"172.21.15.164:6789","nonce":0}]},"addr":"172.21.15.164:6789/0","public_addr":"172.21.15.164:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-04-26T21:44:23.826 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:23.826+0000 7f5817200700 1 -- 172.21.15.164:0/141185299 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f57f0039440 msgr2=0x7f57f003b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:23.827 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:23.826+0000 7f5817200700 1 --2- 172.21.15.164:0/141185299 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f57f0039440 0x7f57f003b900 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f580000fad0 tx=0x7f5800014040 comp rx=0 tx=0).stop 2024-04-26T21:44:23.827 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:23.826+0000 7f5817200700 1 -- 172.21.15.164:0/141185299 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5810104050 msgr2=0x7f58101044d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:23.827 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:23.826+0000 7f5817200700 1 --2- 172.21.15.164:0/141185299 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5810104050 0x7f58101044d0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f57fc00b4d0 tx=0x7f57fc00b890 comp rx=0 tx=0).stop 2024-04-26T21:44:23.827 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:23.826+0000 7f5817200700 1 -- 172.21.15.164:0/141185299 shutdown_connections 2024-04-26T21:44:23.827 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:23.826+0000 7f5817200700 1 --2- 172.21.15.164:0/141185299 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f57f0039440 0x7f57f003b900 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:23.827 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:23.826+0000 7f5817200700 1 --2- 172.21.15.164:0/141185299 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f57f4005c90 0x7f5810103b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:23.827 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:23.826+0000 7f5817200700 1 --2- 172.21.15.164:0/141185299 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5810104050 0x7f58101044d0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:23.827 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:23.826+0000 7f5817200700 1 -- 172.21.15.164:0/141185299 >> 172.21.15.164:0/141185299 conn(0x7f58100fd900 msgr2=0x7f58100fe440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:23.827 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:23.826+0000 7f5817200700 1 -- 172.21.15.164:0/141185299 shutdown_connections 2024-04-26T21:44:23.827 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:23.826+0000 7f5817200700 1 -- 172.21.15.164:0/141185299 wait complete. 2024-04-26T21:44:23.828 INFO:teuthology.orchestra.run.smithi164.stderr:dumped monmap epoch 2 2024-04-26T21:44:24.555 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:24 smithi164 ceph-mon[31916]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:44:24.555 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:24 smithi164 ceph-mon[31916]: from='client.? 172.21.15.164:0/141185299' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-26T21:44:24.822 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:24 smithi072 ceph-mon[28138]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:44:24.822 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:24 smithi072 ceph-mon[28138]: from='client.? 172.21.15.164:0/141185299' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-26T21:44:25.618 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-26T21:44:25.618 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a1780950-0415-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-26T21:44:26.060 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/a1780950-0415-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T21:44:26.941 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:26 smithi164 ceph-mon[31916]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:44:27.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:26 smithi072 ceph-mon[28138]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:44:28.066 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:27 smithi076 podman[31804]: 2024-04-26 21:44:27.830427998 +0000 UTC m=+0.156867481 container create 3c5a685d42be72f303a45f78f0af0fd7e7589cd7a8b21861f4f61ff00a8660e8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-mon-b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff) 2024-04-26T21:44:28.086 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:28.086+0000 7f9f54e00700 1 -- 172.21.15.164:0/216640577 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9f50102fc0 msgr2=0x7f9f501033a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:28.086 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:28.086+0000 7f9f54e00700 1 --2- 172.21.15.164:0/216640577 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9f50102fc0 0x7f9f501033a0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f9f40009910 tx=0x7f9f40009c20 comp rx=0 tx=0).stop 2024-04-26T21:44:28.086 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:28.086+0000 7f9f54e00700 1 -- 172.21.15.164:0/216640577 shutdown_connections 2024-04-26T21:44:28.086 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:28.086+0000 7f9f54e00700 1 --2- 172.21.15.164:0/216640577 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9f50102fc0 0x7f9f501033a0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:28.086 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:28.086+0000 7f9f54e00700 1 -- 172.21.15.164:0/216640577 >> 172.21.15.164:0/216640577 conn(0x7f9f500fe890 msgr2=0x7f9f50100cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:28.086 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:28.086+0000 7f9f54e00700 1 -- 172.21.15.164:0/216640577 shutdown_connections 2024-04-26T21:44:28.087 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:28.086+0000 7f9f54e00700 1 -- 172.21.15.164:0/216640577 wait complete. 2024-04-26T21:44:28.087 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:28.087+0000 7f9f54e00700 1 Processor -- start 2024-04-26T21:44:28.087 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:28.087+0000 7f9f54e00700 1 -- start start 2024-04-26T21:44:28.088 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:28.088+0000 7f9f54e00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9f50102fc0 0x7f9f5019e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:28.088 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:28.088+0000 7f9f54e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9f5019ebc0 0x7f9f50198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:28.088 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:28.088+0000 7f9f54e00700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f9f5010d0f0 con 0x7f9f50102fc0 2024-04-26T21:44:28.088 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:28.088+0000 7f9f54e00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f9f5010cf70 con 0x7f9f5019ebc0 2024-04-26T21:44:28.089 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:28.088+0000 7f9f4f400700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9f50102fc0 0x7f9f5019e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:28.089 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:28.088+0000 7f9f4ea00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9f5019ebc0 0x7f9f50198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:28.089 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:28.088+0000 7f9f4ea00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9f5019ebc0 0x7f9f50198750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:38208/0 (socket says 172.21.15.164:38208) 2024-04-26T21:44:28.089 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:28.088+0000 7f9f4ea00700 1 -- 172.21.15.164:0/2707820133 learned_addr learned my addr 172.21.15.164:0/2707820133 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-04-26T21:44:28.089 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:28.089+0000 7f9f4ea00700 1 -- 172.21.15.164:0/2707820133 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9f50102fc0 msgr2=0x7f9f5019e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:28.089 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:28.089+0000 7f9f4ea00700 1 --2- 172.21.15.164:0/2707820133 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9f50102fc0 0x7f9f5019e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:28.089 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:28.089+0000 7f9f4ea00700 1 -- 172.21.15.164:0/2707820133 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f5019f140 con 0x7f9f5019ebc0 2024-04-26T21:44:28.090 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:28.089+0000 7f9f4ea00700 1 --2- 172.21.15.164:0/2707820133 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9f5019ebc0 0x7f9f50198750 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f9f3c00b700 tx=0x7f9f3c00ba10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:28.090 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:28.090+0000 7f9f47e00700 1 -- 172.21.15.164:0/2707820133 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f9f3c011630 con 0x7f9f5019ebc0 2024-04-26T21:44:28.090 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:28.090+0000 7f9f47e00700 1 -- 172.21.15.164:0/2707820133 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9f3c011c60 con 0x7f9f5019ebc0 2024-04-26T21:44:28.090 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:28.090+0000 7f9f47e00700 1 -- 172.21.15.164:0/2707820133 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f9f3c010b80 con 0x7f9f5019ebc0 2024-04-26T21:44:28.090 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:28.090+0000 7f9f54e00700 1 -- 172.21.15.164:0/2707820133 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f50198cf0 con 0x7f9f5019ebc0 2024-04-26T21:44:28.090 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:28.090+0000 7f9f54e00700 1 -- 172.21.15.164:0/2707820133 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f50199280 con 0x7f9f5019ebc0 2024-04-26T21:44:28.091 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:28.091+0000 7f9f54e00700 1 -- 172.21.15.164:0/2707820133 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f501044a0 con 0x7f9f5019ebc0 2024-04-26T21:44:28.091 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:28.091+0000 7f9f47e00700 1 -- 172.21.15.164:0/2707820133 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f9f3c018020 con 0x7f9f5019ebc0 2024-04-26T21:44:28.092 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:28.092+0000 7f9f47e00700 1 --2- 172.21.15.164:0/2707820133 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f9f34039440 0x7f9f3403b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:28.092 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:28.092+0000 7f9f47e00700 1 -- 172.21.15.164:0/2707820133 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f9f3c04bba0 con 0x7f9f5019ebc0 2024-04-26T21:44:28.092 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:28.092+0000 7f9f4f400700 1 --2- 172.21.15.164:0/2707820133 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f9f34039440 0x7f9f3403b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:28.093 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:28.093+0000 7f9f4f400700 1 --2- 172.21.15.164:0/2707820133 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f9f34039440 0x7f9f3403b900 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f9f40009750 tx=0x7f9f40005a90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:28.098 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:28.098+0000 7f9f47e00700 1 -- 172.21.15.164:0/2707820133 <== mon.1 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f9f3c01f070 con 0x7f9f5019ebc0 2024-04-26T21:44:28.199 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:28.198+0000 7f9f47e00700 1 -- 172.21.15.164:0/2707820133 <== mon.1 v2:172.21.15.164:3300/0 7 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f3c018900 con 0x7f9f5019ebc0 2024-04-26T21:44:28.414 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:28.414+0000 7f9f54e00700 1 -- 172.21.15.164:0/2707820133 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f9f50199d20 con 0x7f9f5019ebc0 2024-04-26T21:44:28.566 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 podman[31804]: 2024-04-26 21:44:28.075690647 +0000 UTC m=+0.402130136 container init 3c5a685d42be72f303a45f78f0af0fd7e7589cd7a8b21861f4f61ff00a8660e8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-mon-b, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-04-26T21:44:28.567 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 podman[31804]: 2024-04-26 21:44:28.085591539 +0000 UTC m=+0.412031029 container start 3c5a685d42be72f303a45f78f0af0fd7e7589cd7a8b21861f4f61ff00a8660e8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-mon-b, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-04-26T21:44:28.567 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: set uid:gid to 167:167 (ceph:ceph) 2024-04-26T21:44:28.567 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: ceph version 19.0.0-3306-gbc612ffb (bc612ffb9c1d5bc27536bc917a705262a36d4387) squid (dev), process ceph-mon, pid 2 2024-04-26T21:44:28.567 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: pidfile_write: ignore empty --pid-file 2024-04-26T21:44:28.567 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: load: jerasure load: lrc 2024-04-26T21:44:28.567 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: RocksDB version: 7.9.2 2024-04-26T21:44:28.567 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: Git sha 0 2024-04-26T21:44:28.567 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: Compile date 2024-04-26 11:39:23 2024-04-26T21:44:28.567 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: DB SUMMARY 2024-04-26T21:44:28.567 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: DB Session ID: YRN9GF764UUNHQM2Q9P4 2024-04-26T21:44:28.567 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: CURRENT file: CURRENT 2024-04-26T21:44:28.567 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: IDENTITY file: IDENTITY 2024-04-26T21:44:28.568 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-04-26T21:44:28.568 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2024-04-26T21:44:28.568 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 636 ; 2024-04-26T21:44:28.568 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: Options.error_if_exists: 0 2024-04-26T21:44:28.568 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: Options.create_if_missing: 0 2024-04-26T21:44:28.568 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: Options.paranoid_checks: 1 2024-04-26T21:44:28.568 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-26T21:44:28.568 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-26T21:44:28.568 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-26T21:44:28.568 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: Options.env: 0x55a9bba05cc0 2024-04-26T21:44:28.568 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: Options.fs: PosixFileSystem 2024-04-26T21:44:28.568 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: Options.info_log: 0x55a9f0313dc0 2024-04-26T21:44:28.568 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: Options.max_file_opening_threads: 16 2024-04-26T21:44:28.569 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: Options.statistics: (nil) 2024-04-26T21:44:28.569 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: Options.use_fsync: 0 2024-04-26T21:44:28.569 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: Options.max_log_file_size: 0 2024-04-26T21:44:28.569 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-26T21:44:28.569 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-26T21:44:28.569 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: Options.keep_log_file_num: 1000 2024-04-26T21:44:28.569 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: Options.recycle_log_file_num: 0 2024-04-26T21:44:28.569 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: Options.allow_fallocate: 1 2024-04-26T21:44:28.569 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: Options.allow_mmap_reads: 0 2024-04-26T21:44:28.569 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: Options.allow_mmap_writes: 0 2024-04-26T21:44:28.569 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: Options.use_direct_reads: 0 2024-04-26T21:44:28.570 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-26T21:44:28.570 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: Options.create_missing_column_families: 0 2024-04-26T21:44:28.570 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: Options.db_log_dir: 2024-04-26T21:44:28.570 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: Options.wal_dir: 2024-04-26T21:44:28.570 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-26T21:44:28.570 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-26T21:44:28.570 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-26T21:44:28.570 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-26T21:44:28.570 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-26T21:44:28.570 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-26T21:44:28.570 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: Options.advise_random_on_open: 1 2024-04-26T21:44:28.570 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: Options.db_write_buffer_size: 0 2024-04-26T21:44:28.570 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: Options.write_buffer_manager: 0x55a9f02e50e0 2024-04-26T21:44:28.571 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-26T21:44:28.571 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-26T21:44:28.571 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-26T21:44:28.571 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: Options.rate_limiter: (nil) 2024-04-26T21:44:28.571 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-26T21:44:28.571 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: Options.wal_recovery_mode: 2 2024-04-26T21:44:28.571 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: Options.enable_thread_tracking: 0 2024-04-26T21:44:28.571 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: Options.enable_pipelined_write: 0 2024-04-26T21:44:28.571 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: Options.unordered_write: 0 2024-04-26T21:44:28.571 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-26T21:44:28.571 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-26T21:44:28.571 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-26T21:44:28.571 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-26T21:44:28.571 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: Options.row_cache: None 2024-04-26T21:44:28.572 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: Options.wal_filter: None 2024-04-26T21:44:28.572 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-26T21:44:28.572 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: Options.allow_ingest_behind: 0 2024-04-26T21:44:28.572 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: Options.two_write_queues: 0 2024-04-26T21:44:28.572 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: Options.manual_wal_flush: 0 2024-04-26T21:44:28.572 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: Options.wal_compression: 0 2024-04-26T21:44:28.572 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: Options.atomic_flush: 0 2024-04-26T21:44:28.572 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-26T21:44:28.573 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-26T21:44:28.573 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-26T21:44:28.573 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: Options.log_readahead_size: 0 2024-04-26T21:44:28.573 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-26T21:44:28.573 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: Options.best_efforts_recovery: 0 2024-04-26T21:44:28.573 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-26T21:44:28.573 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-26T21:44:28.573 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: Options.allow_data_in_errors: 0 2024-04-26T21:44:28.573 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: Options.db_host_id: __hostname__ 2024-04-26T21:44:28.573 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-26T21:44:28.573 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: Options.max_background_jobs: 2 2024-04-26T21:44:28.573 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: Options.max_background_compactions: -1 2024-04-26T21:44:28.573 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: Options.max_subcompactions: 1 2024-04-26T21:44:28.574 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-26T21:44:28.574 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-26T21:44:28.574 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-26T21:44:28.574 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: Options.max_total_wal_size: 0 2024-04-26T21:44:28.574 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-26T21:44:28.574 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-26T21:44:28.574 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-26T21:44:28.574 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-26T21:44:28.574 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: Options.max_open_files: -1 2024-04-26T21:44:28.574 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: Options.bytes_per_sync: 0 2024-04-26T21:44:28.574 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-26T21:44:28.574 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-26T21:44:28.574 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: Options.compaction_readahead_size: 0 2024-04-26T21:44:28.575 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: Options.max_background_flushes: -1 2024-04-26T21:44:28.575 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: Compression algorithms supported: 2024-04-26T21:44:28.575 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-26T21:44:28.575 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: kZSTD supported: 0 2024-04-26T21:44:28.575 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: kXpressCompression supported: 0 2024-04-26T21:44:28.575 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: kLZ4HCCompression supported: 1 2024-04-26T21:44:28.575 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: kZlibCompression supported: 1 2024-04-26T21:44:28.575 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: kSnappyCompression supported: 1 2024-04-26T21:44:28.575 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: kLZ4Compression supported: 1 2024-04-26T21:44:28.576 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: kBZip2Compression supported: 0 2024-04-26T21:44:28.576 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-26T21:44:28.576 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-26T21:44:28.576 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2024-04-26T21:44:28.576 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-26T21:44:28.576 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-26T21:44:28.576 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: Options.merge_operator: 2024-04-26T21:44:28.576 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: Options.compaction_filter: None 2024-04-26T21:44:28.576 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: Options.compaction_filter_factory: None 2024-04-26T21:44:28.576 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: Options.sst_partitioner_factory: None 2024-04-26T21:44:28.576 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-26T21:44:28.576 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-26T21:44:28.576 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55a9f0313f80) 2024-04-26T21:44:28.576 INFO:journalctl@ceph.mon.b.smithi076.stdout: cache_index_and_filter_blocks: 1 2024-04-26T21:44:28.577 INFO:journalctl@ceph.mon.b.smithi076.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-26T21:44:28.577 INFO:journalctl@ceph.mon.b.smithi076.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-26T21:44:28.577 INFO:journalctl@ceph.mon.b.smithi076.stdout: pin_top_level_index_and_filter: 1 2024-04-26T21:44:28.577 INFO:journalctl@ceph.mon.b.smithi076.stdout: index_type: 0 2024-04-26T21:44:28.577 INFO:journalctl@ceph.mon.b.smithi076.stdout: data_block_index_type: 0 2024-04-26T21:44:28.577 INFO:journalctl@ceph.mon.b.smithi076.stdout: index_shortening: 1 2024-04-26T21:44:28.577 INFO:journalctl@ceph.mon.b.smithi076.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-26T21:44:28.577 INFO:journalctl@ceph.mon.b.smithi076.stdout: checksum: 4 2024-04-26T21:44:28.577 INFO:journalctl@ceph.mon.b.smithi076.stdout: no_block_cache: 0 2024-04-26T21:44:28.577 INFO:journalctl@ceph.mon.b.smithi076.stdout: block_cache: 0x55a9f03171f0 2024-04-26T21:44:28.577 INFO:journalctl@ceph.mon.b.smithi076.stdout: block_cache_name: BinnedLRUCache 2024-04-26T21:44:28.577 INFO:journalctl@ceph.mon.b.smithi076.stdout: block_cache_options: 2024-04-26T21:44:28.577 INFO:journalctl@ceph.mon.b.smithi076.stdout: capacity : 536870912 2024-04-26T21:44:28.577 INFO:journalctl@ceph.mon.b.smithi076.stdout: num_shard_bits : 4 2024-04-26T21:44:28.578 INFO:journalctl@ceph.mon.b.smithi076.stdout: strict_capacity_limit : 0 2024-04-26T21:44:28.578 INFO:journalctl@ceph.mon.b.smithi076.stdout: high_pri_pool_ratio: 0.000 2024-04-26T21:44:28.578 INFO:journalctl@ceph.mon.b.smithi076.stdout: block_cache_compressed: (nil) 2024-04-26T21:44:28.578 INFO:journalctl@ceph.mon.b.smithi076.stdout: persistent_cache: (nil) 2024-04-26T21:44:28.578 INFO:journalctl@ceph.mon.b.smithi076.stdout: block_size: 4096 2024-04-26T21:44:28.578 INFO:journalctl@ceph.mon.b.smithi076.stdout: block_size_deviation: 10 2024-04-26T21:44:28.578 INFO:journalctl@ceph.mon.b.smithi076.stdout: block_restart_interval: 16 2024-04-26T21:44:28.578 INFO:journalctl@ceph.mon.b.smithi076.stdout: index_block_restart_interval: 1 2024-04-26T21:44:28.578 INFO:journalctl@ceph.mon.b.smithi076.stdout: metadata_block_size: 4096 2024-04-26T21:44:28.578 INFO:journalctl@ceph.mon.b.smithi076.stdout: partition_filters: 0 2024-04-26T21:44:28.578 INFO:journalctl@ceph.mon.b.smithi076.stdout: use_delta_encoding: 1 2024-04-26T21:44:28.578 INFO:journalctl@ceph.mon.b.smithi076.stdout: filter_policy: bloomfilter 2024-04-26T21:44:28.578 INFO:journalctl@ceph.mon.b.smithi076.stdout: whole_key_filtering: 1 2024-04-26T21:44:28.578 INFO:journalctl@ceph.mon.b.smithi076.stdout: verify_compression: 0 2024-04-26T21:44:28.579 INFO:journalctl@ceph.mon.b.smithi076.stdout: read_amp_bytes_per_bit: 0 2024-04-26T21:44:28.579 INFO:journalctl@ceph.mon.b.smithi076.stdout: format_version: 5 2024-04-26T21:44:28.579 INFO:journalctl@ceph.mon.b.smithi076.stdout: enable_index_compression: 1 2024-04-26T21:44:28.579 INFO:journalctl@ceph.mon.b.smithi076.stdout: block_align: 0 2024-04-26T21:44:28.579 INFO:journalctl@ceph.mon.b.smithi076.stdout: max_auto_readahead_size: 262144 2024-04-26T21:44:28.579 INFO:journalctl@ceph.mon.b.smithi076.stdout: prepopulate_block_cache: 0 2024-04-26T21:44:28.579 INFO:journalctl@ceph.mon.b.smithi076.stdout: initial_auto_readahead_size: 8192 2024-04-26T21:44:28.580 INFO:journalctl@ceph.mon.b.smithi076.stdout: num_file_reads_for_auto_readahead: 2 2024-04-26T21:44:28.580 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: Options.write_buffer_size: 33554432 2024-04-26T21:44:28.580 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: Options.max_write_buffer_number: 2 2024-04-26T21:44:28.580 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: Options.compression: NoCompression 2024-04-26T21:44:28.581 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: Options.bottommost_compression: Disabled 2024-04-26T21:44:28.581 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: Options.prefix_extractor: nullptr 2024-04-26T21:44:28.581 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-26T21:44:28.581 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: Options.num_levels: 7 2024-04-26T21:44:28.581 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-26T21:44:28.581 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-26T21:44:28.581 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-26T21:44:28.581 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-26T21:44:28.581 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-26T21:44:28.581 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-26T21:44:28.581 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-26T21:44:28.581 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-26T21:44:28.581 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-26T21:44:28.581 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-26T21:44:28.582 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-26T21:44:28.582 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-26T21:44:28.582 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-26T21:44:28.582 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: Options.compression_opts.level: 32767 2024-04-26T21:44:28.582 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: Options.compression_opts.strategy: 0 2024-04-26T21:44:28.582 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-26T21:44:28.582 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-26T21:44:28.582 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-26T21:44:28.582 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-26T21:44:28.582 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: Options.compression_opts.enabled: false 2024-04-26T21:44:28.582 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-26T21:44:28.582 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-26T21:44:28.582 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-26T21:44:28.582 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-26T21:44:28.583 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: Options.target_file_size_base: 67108864 2024-04-26T21:44:28.583 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-26T21:44:28.583 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-26T21:44:28.583 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-26T21:44:28.583 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-26T21:44:28.583 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-26T21:44:28.583 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-26T21:44:28.583 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-26T21:44:28.583 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-26T21:44:28.583 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-26T21:44:28.583 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-26T21:44:28.583 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-26T21:44:28.583 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-26T21:44:28.583 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-26T21:44:28.584 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-26T21:44:28.584 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: Options.arena_block_size: 1048576 2024-04-26T21:44:28.584 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-26T21:44:28.584 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-26T21:44:28.584 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: Options.disable_auto_compactions: 0 2024-04-26T21:44:28.584 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-26T21:44:28.584 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-26T21:44:28.584 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-26T21:44:28.584 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-26T21:44:28.584 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-26T21:44:28.584 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-26T21:44:28.584 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-26T21:44:28.584 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-26T21:44:28.585 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-26T21:44:28.585 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-26T21:44:28.585 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-04-26T21:44:28.585 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: Options.inplace_update_support: 0 2024-04-26T21:44:28.585 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-26T21:44:28.585 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-26T21:44:28.585 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-26T21:44:28.585 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-26T21:44:28.585 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: Options.bloom_locality: 0 2024-04-26T21:44:28.585 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: Options.max_successive_merges: 0 2024-04-26T21:44:28.585 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-26T21:44:28.585 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: Options.paranoid_file_checks: 0 2024-04-26T21:44:28.585 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: Options.force_consistency_checks: 1 2024-04-26T21:44:28.585 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: Options.report_bg_io_stats: 0 2024-04-26T21:44:28.586 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: Options.ttl: 2592000 2024-04-26T21:44:28.586 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-26T21:44:28.586 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-26T21:44:28.586 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-26T21:44:28.586 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: Options.enable_blob_files: false 2024-04-26T21:44:28.586 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: Options.min_blob_size: 0 2024-04-26T21:44:28.586 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: Options.blob_file_size: 268435456 2024-04-26T21:44:28.586 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-26T21:44:28.586 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-26T21:44:28.586 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-26T21:44:28.586 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-26T21:44:28.586 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-26T21:44:28.586 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: Options.blob_file_starting_level: 0 2024-04-26T21:44:28.587 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-26T21:44:28.587 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-04-26T21:44:28.587 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-04-26T21:44:28.587 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 97b6ccad-9f7c-4409-8e59-480a33aefe7c 2024-04-26T21:44:28.587 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714167868146885, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-04-26T21:44:28.587 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-04-26T21:44:28.587 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714167868147805, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1773, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1714167868, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "97b6ccad-9f7c-4409-8e59-480a33aefe7c", "db_session_id": "YRN9GF764UUNHQM2Q9P4", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-04-26T21:44:28.587 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714167868148115, "job": 1, "event": "recovery_finished"} 2024-04-26T21:44:28.587 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-04-26T21:44:28.587 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-26T21:44:28.587 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55a9f0434000 2024-04-26T21:44:28.587 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: DB pointer 0x55a9f0420000 2024-04-26T21:44:28.587 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-26T21:44:28.588 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-26T21:44:28.588 INFO:journalctl@ceph.mon.b.smithi076.stdout: ** DB Stats ** 2024-04-26T21:44:28.588 INFO:journalctl@ceph.mon.b.smithi076.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-26T21:44:28.588 INFO:journalctl@ceph.mon.b.smithi076.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-26T21:44:28.588 INFO:journalctl@ceph.mon.b.smithi076.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-26T21:44:28.588 INFO:journalctl@ceph.mon.b.smithi076.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-26T21:44:28.588 INFO:journalctl@ceph.mon.b.smithi076.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-26T21:44:28.588 INFO:journalctl@ceph.mon.b.smithi076.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-26T21:44:28.588 INFO:journalctl@ceph.mon.b.smithi076.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-26T21:44:28.588 INFO:journalctl@ceph.mon.b.smithi076.stdout: 2024-04-26T21:44:28.588 INFO:journalctl@ceph.mon.b.smithi076.stdout: ** Compaction Stats [default] ** 2024-04-26T21:44:28.588 INFO:journalctl@ceph.mon.b.smithi076.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-26T21:44:28.588 INFO:journalctl@ceph.mon.b.smithi076.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-26T21:44:28.588 INFO:journalctl@ceph.mon.b.smithi076.stdout: L0 1/0 1.73 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-26T21:44:28.589 INFO:journalctl@ceph.mon.b.smithi076.stdout: Sum 1/0 1.73 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-26T21:44:28.589 INFO:journalctl@ceph.mon.b.smithi076.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-26T21:44:28.589 INFO:journalctl@ceph.mon.b.smithi076.stdout: 2024-04-26T21:44:28.589 INFO:journalctl@ceph.mon.b.smithi076.stdout: ** Compaction Stats [default] ** 2024-04-26T21:44:28.589 INFO:journalctl@ceph.mon.b.smithi076.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-26T21:44:28.589 INFO:journalctl@ceph.mon.b.smithi076.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-26T21:44:28.589 INFO:journalctl@ceph.mon.b.smithi076.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-26T21:44:28.589 INFO:journalctl@ceph.mon.b.smithi076.stdout: 2024-04-26T21:44:28.589 INFO:journalctl@ceph.mon.b.smithi076.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-26T21:44:28.589 INFO:journalctl@ceph.mon.b.smithi076.stdout: 2024-04-26T21:44:28.589 INFO:journalctl@ceph.mon.b.smithi076.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-26T21:44:28.589 INFO:journalctl@ceph.mon.b.smithi076.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-26T21:44:28.589 INFO:journalctl@ceph.mon.b.smithi076.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-26T21:44:28.589 INFO:journalctl@ceph.mon.b.smithi076.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-26T21:44:28.589 INFO:journalctl@ceph.mon.b.smithi076.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-26T21:44:28.590 INFO:journalctl@ceph.mon.b.smithi076.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-26T21:44:28.590 INFO:journalctl@ceph.mon.b.smithi076.stdout: Cumulative compaction: 0.00 GB write, 0.30 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-26T21:44:28.590 INFO:journalctl@ceph.mon.b.smithi076.stdout: Interval compaction: 0.00 GB write, 0.30 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-26T21:44:28.590 INFO:journalctl@ceph.mon.b.smithi076.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-26T21:44:28.590 INFO:journalctl@ceph.mon.b.smithi076.stdout: Block cache BinnedLRUCache@0x55a9f03171f0#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.1e-05 secs_since: 0 2024-04-26T21:44:28.590 INFO:journalctl@ceph.mon.b.smithi076.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-04-26T21:44:28.590 INFO:journalctl@ceph.mon.b.smithi076.stdout: 2024-04-26T21:44:28.590 INFO:journalctl@ceph.mon.b.smithi076.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-26T21:44:28.590 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: mon.b does not exist in monmap, will attempt to join an existing cluster 2024-04-26T21:44:28.590 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: using public_addr v2:172.21.15.76:0/0 -> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] 2024-04-26T21:44:28.590 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: starting mon.b rank -1 at public addrs [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] at bind addrs [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid a1780950-0415-11ef-bc93-c7b262605968 2024-04-26T21:44:28.591 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: mon.b@-1(???) e0 preinit fsid a1780950-0415-11ef-bc93-c7b262605968 2024-04-26T21:44:28.591 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 bash[31804]: 3c5a685d42be72f303a45f78f0af0fd7e7589cd7a8b21861f4f61ff00a8660e8 2024-04-26T21:44:28.591 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 systemd[1]: Started Ceph mon.b for a1780950-0415-11ef-bc93-c7b262605968. 2024-04-26T21:44:28.591 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: mon.b@-1(synchronizing).mds e1 new map 2024-04-26T21:44:28.591 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: mon.b@-1(synchronizing).mds e1 print_map 2024-04-26T21:44:28.591 INFO:journalctl@ceph.mon.b.smithi076.stdout: e1 2024-04-26T21:44:28.591 INFO:journalctl@ceph.mon.b.smithi076.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-26T21:44:28.591 INFO:journalctl@ceph.mon.b.smithi076.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-26T21:44:28.591 INFO:journalctl@ceph.mon.b.smithi076.stdout: legacy client fscid: -1 2024-04-26T21:44:28.591 INFO:journalctl@ceph.mon.b.smithi076.stdout: 2024-04-26T21:44:28.591 INFO:journalctl@ceph.mon.b.smithi076.stdout: No filesystems configured 2024-04-26T21:44:28.591 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-04-26T21:44:28.591 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-04-26T21:44:28.592 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-04-26T21:44:28.592 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-04-26T21:44:28.592 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-04-26T21:44:28.592 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-04-26T21:44:28.592 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-26T21:44:28.592 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-26T21:44:28.592 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-26T21:44:28.592 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-26T21:44:28.592 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: Updating smithi072:/var/lib/ceph/a1780950-0415-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-26T21:44:28.592 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:44:28.592 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:28.592 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:28.592 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:28.592 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi076", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:44:28.592 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:44:28.593 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: Deploying cephadm binary to smithi076 2024-04-26T21:44:28.593 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:44:28.593 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:44:28.593 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:28.593 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: Added host smithi076 2024-04-26T21:44:28.593 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:44:28.593 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:44:28.593 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:28.593 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:28.593 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:44:28.593 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T21:44:28.593 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:28.593 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:44:28.593 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:44:28.594 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi164", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:44:28.594 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: Deploying cephadm binary to smithi164 2024-04-26T21:44:28.594 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:44:28.594 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:28.594 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:28.594 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:28.594 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:28.594 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi076", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:44:28.594 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:44:28.594 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:44:28.594 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: Updating smithi076:/etc/ceph/ceph.conf 2024-04-26T21:44:28.594 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:44:28.594 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: Updating smithi076:/var/lib/ceph/a1780950-0415-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T21:44:28.594 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:44:28.595 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:28.595 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: Updating smithi076:/etc/ceph/ceph.client.admin.keyring 2024-04-26T21:44:28.595 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: Added host smithi164 2024-04-26T21:44:28.595 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: Updating smithi076:/var/lib/ceph/a1780950-0415-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-26T21:44:28.595 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:28.595 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:28.595 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:28.595 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:44:28.595 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:44:28.595 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:28.595 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:28.595 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:44:28.595 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T21:44:28.595 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:44:28.596 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:28.596 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: from='client.? 172.21.15.72:0/3022658462' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-26T21:44:28.596 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:44:28.596 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: from='client.? 172.21.15.72:0/3022658462' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-26T21:44:28.596 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: osdmap e4: 0 total, 0 up, 0 in 2024-04-26T21:44:28.596 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:44:28.596 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:28.596 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:28.596 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:28.596 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:28.596 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi164", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:44:28.596 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:44:28.596 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:44:28.596 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: Updating smithi164:/etc/ceph/ceph.conf 2024-04-26T21:44:28.596 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: Updating smithi164:/var/lib/ceph/a1780950-0415-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T21:44:28.596 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:44:28.597 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:28.597 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi072:172.21.15.72=a;smithi076:172.21.15.76=b;smithi164:172.21.15.164=c", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:44:28.597 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: Saving service mon spec with placement smithi072:172.21.15.72=a;smithi076:172.21.15.76=b;smithi164:172.21.15.164=c;count:3 2024-04-26T21:44:28.597 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: Updating smithi164:/etc/ceph/ceph.client.admin.keyring 2024-04-26T21:44:28.597 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: Updating smithi164:/var/lib/ceph/a1780950-0415-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-26T21:44:28.597 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:28.598 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:28.598 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:28.598 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T21:44:28.598 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:44:28.598 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: Deploying daemon mon.c on smithi164 2024-04-26T21:44:28.598 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:44:28.598 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:44:28.598 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: from='client.? 172.21.15.164:0/3741886131' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-26T21:44:28.598 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:44:28.598 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:44:28.598 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T21:44:28.598 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T21:44:28.598 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: mon.a calling monitor election 2024-04-26T21:44:28.598 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T21:44:28.599 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:44:28.599 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T21:44:28.599 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T21:44:28.599 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: mon.c calling monitor election 2024-04-26T21:44:28.599 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:44:28.599 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T21:44:28.599 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T21:44:28.599 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:44:28.599 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T21:44:28.599 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-26T21:44:28.599 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: monmap epoch 2 2024-04-26T21:44:28.599 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: fsid a1780950-0415-11ef-bc93-c7b262605968 2024-04-26T21:44:28.599 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: last_changed 2024-04-26T21:44:17.350038+0000 2024-04-26T21:44:28.599 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: created 2024-04-26T21:41:57.024587+0000 2024-04-26T21:44:28.600 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: min_mon_release 19 (squid) 2024-04-26T21:44:28.600 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: election_strategy: 1 2024-04-26T21:44:28.600 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: 0: [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] mon.a 2024-04-26T21:44:28.600 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: 1: [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] mon.c 2024-04-26T21:44:28.600 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: fsmap 2024-04-26T21:44:28.600 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: osdmap e4: 0 total, 0 up, 0 in 2024-04-26T21:44:28.600 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: mgrmap e13: x(active, since 72s) 2024-04-26T21:44:28.600 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: overall HEALTH_OK 2024-04-26T21:44:28.600 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:28.600 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:28.600 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:28.600 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T21:44:28.600 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:44:28.600 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: Deploying daemon mon.b on smithi076 2024-04-26T21:44:28.600 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T21:44:28.600 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:44:28.601 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: from='client.? 172.21.15.164:0/141185299' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-26T21:44:28.601 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:44:28.601 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:28 smithi076 ceph-mon[31823]: mon.b@-1(synchronizing).paxosservice(auth 1..4) refresh upgraded, format 0 -> 3 2024-04-26T21:44:33.219 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:33.219+0000 7f9f47e00700 1 -- 172.21.15.164:0/2707820133 <== mon.1 v2:172.21.15.164:3300/0 8 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1281 (secure 0 0 0) 0x7f9f3c03e520 con 0x7f9f5019ebc0 2024-04-26T21:44:33.220 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-26T21:44:33.220 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":3,"fsid":"a1780950-0415-11ef-bc93-c7b262605968","modified":"2024-04-26T21:44:28.195062Z","created":"2024-04-26T21:41:57.024587Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:3300","nonce":0},{"type":"v1","addr":"172.21.15.72:6789","nonce":0}]},"addr":"172.21.15.72:6789/0","public_addr":"172.21.15.72:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:3300","nonce":0},{"type":"v1","addr":"172.21.15.164:6789","nonce":0}]},"addr":"172.21.15.164:6789/0","public_addr":"172.21.15.164:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:3300","nonce":0},{"type":"v1","addr":"172.21.15.76:6789","nonce":0}]},"addr":"172.21.15.76:6789/0","public_addr":"172.21.15.76:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2024-04-26T21:44:33.223 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:33.222+0000 7f9f54e00700 1 -- 172.21.15.164:0/2707820133 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f9f34039440 msgr2=0x7f9f3403b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:33.223 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:33.222+0000 7f9f54e00700 1 --2- 172.21.15.164:0/2707820133 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f9f34039440 0x7f9f3403b900 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f9f40009750 tx=0x7f9f40005a90 comp rx=0 tx=0).stop 2024-04-26T21:44:33.223 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:33.223+0000 7f9f54e00700 1 -- 172.21.15.164:0/2707820133 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9f5019ebc0 msgr2=0x7f9f50198750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:33.224 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:33.223+0000 7f9f54e00700 1 --2- 172.21.15.164:0/2707820133 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9f5019ebc0 0x7f9f50198750 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f9f3c00b700 tx=0x7f9f3c00ba10 comp rx=0 tx=0).stop 2024-04-26T21:44:33.224 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:33.223+0000 7f9f54e00700 1 -- 172.21.15.164:0/2707820133 shutdown_connections 2024-04-26T21:44:33.224 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:33.223+0000 7f9f54e00700 1 --2- 172.21.15.164:0/2707820133 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f9f34039440 0x7f9f3403b900 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:33.224 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:33.223+0000 7f9f54e00700 1 --2- 172.21.15.164:0/2707820133 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9f50102fc0 0x7f9f5019e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:33.224 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:33.223+0000 7f9f54e00700 1 --2- 172.21.15.164:0/2707820133 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9f5019ebc0 0x7f9f50198750 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:33.224 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:33.223+0000 7f9f54e00700 1 -- 172.21.15.164:0/2707820133 >> 172.21.15.164:0/2707820133 conn(0x7f9f500fe890 msgr2=0x7f9f5010ab30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:33.224 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:33.223+0000 7f9f54e00700 1 -- 172.21.15.164:0/2707820133 shutdown_connections 2024-04-26T21:44:33.224 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:33.223+0000 7f9f54e00700 1 -- 172.21.15.164:0/2707820133 wait complete. 2024-04-26T21:44:33.225 INFO:teuthology.orchestra.run.smithi164.stderr:dumped monmap epoch 3 2024-04-26T21:44:33.565 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:33 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T21:44:33.565 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:33 smithi076 ceph-mon[31823]: mon.a calling monitor election 2024-04-26T21:44:33.566 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:33 smithi076 ceph-mon[31823]: mon.c calling monitor election 2024-04-26T21:44:33.566 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:33 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T21:44:33.566 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:33 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T21:44:33.566 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:33 smithi076 ceph-mon[31823]: from='client.? 172.21.15.164:0/2707820133' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-26T21:44:33.566 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:33 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T21:44:33.566 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:33 smithi076 ceph-mon[31823]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:44:33.566 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:33 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T21:44:33.566 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:33 smithi076 ceph-mon[31823]: mon.b calling monitor election 2024-04-26T21:44:33.566 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:33 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T21:44:33.566 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:33 smithi076 ceph-mon[31823]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:44:33.566 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:33 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T21:44:33.566 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:33 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T21:44:33.567 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:33 smithi076 ceph-mon[31823]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-26T21:44:33.567 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:33 smithi076 ceph-mon[31823]: monmap epoch 3 2024-04-26T21:44:33.567 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:33 smithi076 ceph-mon[31823]: fsid a1780950-0415-11ef-bc93-c7b262605968 2024-04-26T21:44:33.567 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:33 smithi076 ceph-mon[31823]: last_changed 2024-04-26T21:44:28.195062+0000 2024-04-26T21:44:33.567 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:33 smithi076 ceph-mon[31823]: created 2024-04-26T21:41:57.024587+0000 2024-04-26T21:44:33.567 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:33 smithi076 ceph-mon[31823]: min_mon_release 19 (squid) 2024-04-26T21:44:33.567 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:33 smithi076 ceph-mon[31823]: election_strategy: 1 2024-04-26T21:44:33.567 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:33 smithi076 ceph-mon[31823]: 0: [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] mon.a 2024-04-26T21:44:33.567 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:33 smithi076 ceph-mon[31823]: 1: [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] mon.c 2024-04-26T21:44:33.567 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:33 smithi076 ceph-mon[31823]: 2: [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] mon.b 2024-04-26T21:44:33.567 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:33 smithi076 ceph-mon[31823]: fsmap 2024-04-26T21:44:33.567 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:33 smithi076 ceph-mon[31823]: osdmap e4: 0 total, 0 up, 0 in 2024-04-26T21:44:33.567 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:33 smithi076 ceph-mon[31823]: mgrmap e13: x(active, since 83s) 2024-04-26T21:44:33.567 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:33 smithi076 ceph-mon[31823]: overall HEALTH_OK 2024-04-26T21:44:33.568 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:33 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:33.568 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:33 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:33.573 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:33 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T21:44:33.573 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:33 smithi072 ceph-mon[28138]: mon.a calling monitor election 2024-04-26T21:44:33.573 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:33 smithi072 ceph-mon[28138]: mon.c calling monitor election 2024-04-26T21:44:33.573 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:33 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T21:44:33.573 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:33 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T21:44:33.573 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:33 smithi072 ceph-mon[28138]: from='client.? 172.21.15.164:0/2707820133' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-26T21:44:33.573 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:33 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T21:44:33.573 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:33 smithi072 ceph-mon[28138]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:44:33.573 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:33 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T21:44:33.573 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:33 smithi072 ceph-mon[28138]: mon.b calling monitor election 2024-04-26T21:44:33.573 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:33 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T21:44:33.574 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:33 smithi072 ceph-mon[28138]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:44:33.574 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:33 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T21:44:33.574 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:33 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T21:44:33.574 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:33 smithi072 ceph-mon[28138]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-26T21:44:33.574 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:33 smithi072 ceph-mon[28138]: monmap epoch 3 2024-04-26T21:44:33.574 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:33 smithi072 ceph-mon[28138]: fsid a1780950-0415-11ef-bc93-c7b262605968 2024-04-26T21:44:33.574 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:33 smithi072 ceph-mon[28138]: last_changed 2024-04-26T21:44:28.195062+0000 2024-04-26T21:44:33.574 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:33 smithi072 ceph-mon[28138]: created 2024-04-26T21:41:57.024587+0000 2024-04-26T21:44:33.574 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:33 smithi072 ceph-mon[28138]: min_mon_release 19 (squid) 2024-04-26T21:44:33.574 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:33 smithi072 ceph-mon[28138]: election_strategy: 1 2024-04-26T21:44:33.574 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:33 smithi072 ceph-mon[28138]: 0: [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] mon.a 2024-04-26T21:44:33.574 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:33 smithi072 ceph-mon[28138]: 1: [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] mon.c 2024-04-26T21:44:33.574 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:33 smithi072 ceph-mon[28138]: 2: [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] mon.b 2024-04-26T21:44:33.575 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:33 smithi072 ceph-mon[28138]: fsmap 2024-04-26T21:44:33.575 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:33 smithi072 ceph-mon[28138]: osdmap e4: 0 total, 0 up, 0 in 2024-04-26T21:44:33.575 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:33 smithi072 ceph-mon[28138]: mgrmap e13: x(active, since 83s) 2024-04-26T21:44:33.575 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:33 smithi072 ceph-mon[28138]: overall HEALTH_OK 2024-04-26T21:44:33.575 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:33 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:33.575 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:33 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:33.600 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:33 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T21:44:33.600 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:33 smithi164 ceph-mon[31916]: mon.a calling monitor election 2024-04-26T21:44:33.600 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:33 smithi164 ceph-mon[31916]: mon.c calling monitor election 2024-04-26T21:44:33.600 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:33 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T21:44:33.601 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:33 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T21:44:33.601 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:33 smithi164 ceph-mon[31916]: from='client.? 172.21.15.164:0/2707820133' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-26T21:44:33.601 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:33 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T21:44:33.601 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:33 smithi164 ceph-mon[31916]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:44:33.601 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:33 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T21:44:33.601 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:33 smithi164 ceph-mon[31916]: mon.b calling monitor election 2024-04-26T21:44:33.601 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:33 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T21:44:33.601 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:33 smithi164 ceph-mon[31916]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:44:33.601 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:33 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T21:44:33.601 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:33 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T21:44:33.601 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:33 smithi164 ceph-mon[31916]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-26T21:44:33.601 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:33 smithi164 ceph-mon[31916]: monmap epoch 3 2024-04-26T21:44:33.601 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:33 smithi164 ceph-mon[31916]: fsid a1780950-0415-11ef-bc93-c7b262605968 2024-04-26T21:44:33.601 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:33 smithi164 ceph-mon[31916]: last_changed 2024-04-26T21:44:28.195062+0000 2024-04-26T21:44:33.602 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:33 smithi164 ceph-mon[31916]: created 2024-04-26T21:41:57.024587+0000 2024-04-26T21:44:33.602 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:33 smithi164 ceph-mon[31916]: min_mon_release 19 (squid) 2024-04-26T21:44:33.602 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:33 smithi164 ceph-mon[31916]: election_strategy: 1 2024-04-26T21:44:33.602 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:33 smithi164 ceph-mon[31916]: 0: [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] mon.a 2024-04-26T21:44:33.602 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:33 smithi164 ceph-mon[31916]: 1: [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] mon.c 2024-04-26T21:44:33.602 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:33 smithi164 ceph-mon[31916]: 2: [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] mon.b 2024-04-26T21:44:33.602 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:33 smithi164 ceph-mon[31916]: fsmap 2024-04-26T21:44:33.602 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:33 smithi164 ceph-mon[31916]: osdmap e4: 0 total, 0 up, 0 in 2024-04-26T21:44:33.602 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:33 smithi164 ceph-mon[31916]: mgrmap e13: x(active, since 83s) 2024-04-26T21:44:33.602 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:33 smithi164 ceph-mon[31916]: overall HEALTH_OK 2024-04-26T21:44:33.603 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:33 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:33.603 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:33 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:33.997 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-04-26T21:44:33.997 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a1780950-0415-11ef-bc93-c7b262605968 -- ceph config generate-minimal-conf 2024-04-26T21:44:34.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:34 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:34.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:34 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:34.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:34 smithi072 ceph-mon[28138]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:44:34.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:34 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:44:34.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:34 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T21:44:34.418 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/a1780950-0415-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:44:34.564 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:34 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:34.564 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:34 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:34.564 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:34 smithi076 ceph-mon[31823]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:44:34.564 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:34 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:44:34.565 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:34 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T21:44:34.690 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:34 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:34.690 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:34 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:34.690 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:34 smithi164 ceph-mon[31916]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:44:34.690 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:34 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:44:34.691 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:34 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T21:44:35.572 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 26 21:44:35 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-mgr-x[28363]: 2024-04-26T21:44:35.195+0000 7fb17a600700 -1 mgr.server handle_report got status from non-daemon mon.b 2024-04-26T21:44:35.940 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:35 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:35.941 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:35 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:35.941 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:35 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:35.941 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:35 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:35.941 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:35 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:35.941 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:35 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:44:35.941 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:35 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:44:35.997 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:35 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:35.997 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:35 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:35.997 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:35 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:35.997 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:35 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:35.997 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:35 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:35.997 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:35 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:44:35.997 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:35 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:44:36.064 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:35 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:36.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:35 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:36.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:35 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:36.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:35 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:36.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:35 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:36.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:35 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:44:36.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:35 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:44:36.505 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:44:36.503+0000 7fb2fbc00700 1 -- 172.21.15.72:0/2694185892 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb2f4075850 msgr2=0x7fb2f4075c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:36.505 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:44:36.503+0000 7fb2fbc00700 1 --2- 172.21.15.72:0/2694185892 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb2f4075850 0x7fb2f4075c30 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fb2e8009910 tx=0x7fb2e8009c20 comp rx=0 tx=0).stop 2024-04-26T21:44:36.505 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:44:36.503+0000 7fb2fbc00700 1 -- 172.21.15.72:0/2694185892 shutdown_connections 2024-04-26T21:44:36.505 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:44:36.503+0000 7fb2fbc00700 1 --2- 172.21.15.72:0/2694185892 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb2f4075850 0x7fb2f4075c30 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:36.505 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:44:36.503+0000 7fb2fbc00700 1 -- 172.21.15.72:0/2694185892 >> 172.21.15.72:0/2694185892 conn(0x7fb2f40fe960 msgr2=0x7fb2f4100d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:36.505 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:44:36.503+0000 7fb2fbc00700 1 -- 172.21.15.72:0/2694185892 shutdown_connections 2024-04-26T21:44:36.505 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:44:36.503+0000 7fb2fbc00700 1 -- 172.21.15.72:0/2694185892 wait complete. 2024-04-26T21:44:36.506 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:44:36.504+0000 7fb2fbc00700 1 Processor -- start 2024-04-26T21:44:36.506 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:44:36.504+0000 7fb2fbc00700 1 -- start start 2024-04-26T21:44:36.507 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:44:36.505+0000 7fb2fbc00700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb2f4075850 0x7fb2f41a08d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:36.507 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:44:36.505+0000 7fb2fbc00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb2f41a0e10 0x7fb2f419aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:36.507 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:44:36.505+0000 7fb2fbc00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb2f419b020 0x7fb2f419b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:36.507 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:44:36.505+0000 7fb2fbc00700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fb2f410f440 con 0x7fb2f41a0e10 2024-04-26T21:44:36.507 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:44:36.505+0000 7fb2fbc00700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fb2f410f2c0 con 0x7fb2f4075850 2024-04-26T21:44:36.507 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:44:36.505+0000 7fb2fbc00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fb2f410f5c0 con 0x7fb2f419b020 2024-04-26T21:44:36.507 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:44:36.505+0000 7fb2f3e00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb2f41a0e10 0x7fb2f419aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:36.507 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:44:36.505+0000 7fb2f3e00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb2f41a0e10 0x7fb2f419aae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:33082/0 (socket says 172.21.15.72:33082) 2024-04-26T21:44:36.507 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:44:36.505+0000 7fb2f3e00700 1 -- 172.21.15.72:0/1479048744 learned_addr learned my addr 172.21.15.72:0/1479048744 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:44:36.508 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:44:36.505+0000 7fb2f9c00700 1 --2- 172.21.15.72:0/1479048744 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb2f419b020 0x7fb2f419b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:36.508 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:44:36.505+0000 7fb2f9200700 1 --2- 172.21.15.72:0/1479048744 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb2f4075850 0x7fb2f41a08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:36.508 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:44:36.505+0000 7fb2f3e00700 1 -- 172.21.15.72:0/1479048744 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb2f419b020 msgr2=0x7fb2f419b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:36.508 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:44:36.505+0000 7fb2f3e00700 1 --2- 172.21.15.72:0/1479048744 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb2f419b020 0x7fb2f419b4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:36.509 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:44:36.505+0000 7fb2f3e00700 1 -- 172.21.15.72:0/1479048744 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb2f4075850 msgr2=0x7fb2f41a08d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:36.509 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:44:36.505+0000 7fb2f3e00700 1 --2- 172.21.15.72:0/1479048744 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb2f4075850 0x7fb2f41a08d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:36.509 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:44:36.505+0000 7fb2f3e00700 1 -- 172.21.15.72:0/1479048744 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2f41a77a0 con 0x7fb2f41a0e10 2024-04-26T21:44:36.509 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:44:36.505+0000 7fb2f9c00700 1 --2- 172.21.15.72:0/1479048744 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb2f419b020 0x7fb2f419b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:44:36.509 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:44:36.506+0000 7fb2f3e00700 1 --2- 172.21.15.72:0/1479048744 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb2f41a0e10 0x7fb2f419aae0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fb2e000b780 tx=0x7fb2e000bb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:36.509 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:44:36.506+0000 7fb2f1600700 1 -- 172.21.15.72:0/1479048744 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2e0004230 con 0x7fb2f41a0e10 2024-04-26T21:44:36.509 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:44:36.506+0000 7fb2f1600700 1 -- 172.21.15.72:0/1479048744 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb2e00043d0 con 0x7fb2f41a0e10 2024-04-26T21:44:36.509 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:44:36.506+0000 7fb2fbc00700 1 -- 172.21.15.72:0/1479048744 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2f41a79d0 con 0x7fb2f41a0e10 2024-04-26T21:44:36.509 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:44:36.506+0000 7fb2f1600700 1 -- 172.21.15.72:0/1479048744 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2e0011ac0 con 0x7fb2f41a0e10 2024-04-26T21:44:36.509 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:44:36.506+0000 7fb2fbc00700 1 -- 172.21.15.72:0/1479048744 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2f41a7f60 con 0x7fb2f41a0e10 2024-04-26T21:44:36.510 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:44:36.507+0000 7fb2fbc00700 1 -- 172.21.15.72:0/1479048744 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2f4064a80 con 0x7fb2f41a0e10 2024-04-26T21:44:36.510 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:44:36.508+0000 7fb2f1600700 1 -- 172.21.15.72:0/1479048744 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fb2e00036a0 con 0x7fb2f41a0e10 2024-04-26T21:44:36.510 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:44:36.509+0000 7fb2f1600700 1 --2- 172.21.15.72:0/1479048744 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fb2dc039ae0 0x7fb2dc03bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:36.511 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:44:36.509+0000 7fb2f1600700 1 -- 172.21.15.72:0/1479048744 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fb2e004c510 con 0x7fb2f41a0e10 2024-04-26T21:44:36.511 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:44:36.509+0000 7fb2f9200700 1 --2- 172.21.15.72:0/1479048744 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fb2dc039ae0 0x7fb2dc03bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:36.511 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:44:36.510+0000 7fb2f9200700 1 --2- 172.21.15.72:0/1479048744 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fb2dc039ae0 0x7fb2dc03bfa0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fb2e8005920 tx=0x7fb2e8005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:36.517 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:44:36.516+0000 7fb2f1600700 1 -- 172.21.15.72:0/1479048744 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fb2e0003990 con 0x7fb2f41a0e10 2024-04-26T21:44:36.774 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:44:36.773+0000 7fb2fbc00700 1 -- 172.21.15.72:0/1479048744 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7fb2f41a8250 con 0x7fb2f41a0e10 2024-04-26T21:44:36.775 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:44:36.773+0000 7fb2f1600700 1 -- 172.21.15.72:0/1479048744 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+273 (secure 0 0 0) 0x7fb2e0018e30 con 0x7fb2f41a0e10 2024-04-26T21:44:36.775 INFO:teuthology.orchestra.run.smithi072.stdout:# minimal ceph.conf for a1780950-0415-11ef-bc93-c7b262605968 2024-04-26T21:44:36.775 INFO:teuthology.orchestra.run.smithi072.stdout:[global] 2024-04-26T21:44:36.775 INFO:teuthology.orchestra.run.smithi072.stdout: fsid = a1780950-0415-11ef-bc93-c7b262605968 2024-04-26T21:44:36.775 INFO:teuthology.orchestra.run.smithi072.stdout: mon_host = [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] 2024-04-26T21:44:36.777 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:44:36.776+0000 7fb2fbc00700 1 -- 172.21.15.72:0/1479048744 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fb2dc039ae0 msgr2=0x7fb2dc03bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:36.777 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:44:36.776+0000 7fb2fbc00700 1 --2- 172.21.15.72:0/1479048744 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fb2dc039ae0 0x7fb2dc03bfa0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fb2e8005920 tx=0x7fb2e8005fb0 comp rx=0 tx=0).stop 2024-04-26T21:44:36.778 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:44:36.776+0000 7fb2fbc00700 1 -- 172.21.15.72:0/1479048744 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb2f41a0e10 msgr2=0x7fb2f419aae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:36.778 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:44:36.776+0000 7fb2fbc00700 1 --2- 172.21.15.72:0/1479048744 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb2f41a0e10 0x7fb2f419aae0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fb2e000b780 tx=0x7fb2e000bb40 comp rx=0 tx=0).stop 2024-04-26T21:44:36.778 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:44:36.776+0000 7fb2fbc00700 1 -- 172.21.15.72:0/1479048744 shutdown_connections 2024-04-26T21:44:36.778 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:44:36.776+0000 7fb2fbc00700 1 --2- 172.21.15.72:0/1479048744 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fb2dc039ae0 0x7fb2dc03bfa0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:36.778 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:44:36.776+0000 7fb2fbc00700 1 --2- 172.21.15.72:0/1479048744 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb2f419b020 0x7fb2f419b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:36.778 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:44:36.776+0000 7fb2fbc00700 1 --2- 172.21.15.72:0/1479048744 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb2f4075850 0x7fb2f41a08d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:36.778 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:44:36.776+0000 7fb2fbc00700 1 --2- 172.21.15.72:0/1479048744 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb2f41a0e10 0x7fb2f419aae0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:36.778 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:44:36.776+0000 7fb2fbc00700 1 -- 172.21.15.72:0/1479048744 >> 172.21.15.72:0/1479048744 conn(0x7fb2f40fe960 msgr2=0x7fb2f41090e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:36.778 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:44:36.776+0000 7fb2fbc00700 1 -- 172.21.15.72:0/1479048744 shutdown_connections 2024-04-26T21:44:36.778 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:44:36.776+0000 7fb2fbc00700 1 -- 172.21.15.72:0/1479048744 wait complete. 2024-04-26T21:44:37.064 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:36 smithi076 ceph-mon[31823]: Updating smithi072:/etc/ceph/ceph.conf 2024-04-26T21:44:37.064 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:36 smithi076 ceph-mon[31823]: Updating smithi076:/etc/ceph/ceph.conf 2024-04-26T21:44:37.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:36 smithi076 ceph-mon[31823]: Updating smithi164:/etc/ceph/ceph.conf 2024-04-26T21:44:37.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:36 smithi076 ceph-mon[31823]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:44:37.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:36 smithi072 ceph-mon[28138]: Updating smithi072:/etc/ceph/ceph.conf 2024-04-26T21:44:37.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:36 smithi072 ceph-mon[28138]: Updating smithi076:/etc/ceph/ceph.conf 2024-04-26T21:44:37.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:36 smithi072 ceph-mon[28138]: Updating smithi164:/etc/ceph/ceph.conf 2024-04-26T21:44:37.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:36 smithi072 ceph-mon[28138]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:44:37.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:36 smithi164 ceph-mon[31916]: Updating smithi072:/etc/ceph/ceph.conf 2024-04-26T21:44:37.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:36 smithi164 ceph-mon[31916]: Updating smithi076:/etc/ceph/ceph.conf 2024-04-26T21:44:37.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:36 smithi164 ceph-mon[31916]: Updating smithi164:/etc/ceph/ceph.conf 2024-04-26T21:44:37.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:36 smithi164 ceph-mon[31916]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:44:37.533 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-04-26T21:44:37.533 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2024-04-26T21:44:37.533 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd of=/etc/ceph/ceph.conf 2024-04-26T21:44:37.614 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2024-04-26T21:44:37.614 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-26T21:44:37.720 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:37 smithi072 ceph-mon[28138]: Updating smithi072:/var/lib/ceph/a1780950-0415-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T21:44:37.721 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:37 smithi072 ceph-mon[28138]: Updating smithi076:/var/lib/ceph/a1780950-0415-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T21:44:37.721 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:37 smithi072 ceph-mon[28138]: Updating smithi164:/var/lib/ceph/a1780950-0415-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T21:44:37.721 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:37 smithi072 ceph-mon[28138]: from='client.? 172.21.15.72:0/1479048744' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:44:37.721 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:37 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:37.721 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:37 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:37.721 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:37 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:37.721 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:37 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:37.721 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:37 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:37.721 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:37 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:37.721 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:37 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:37.721 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:37 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:37.721 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:37 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:37.721 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:37 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:37.721 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:37 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:37.722 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:37 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T21:44:37.722 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:37 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-26T21:44:37.722 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:37 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:44:37.723 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-04-26T21:44:37.723 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd of=/etc/ceph/ceph.conf 2024-04-26T21:44:37.815 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-04-26T21:44:37.815 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-26T21:44:37.942 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-04-26T21:44:37.942 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd of=/etc/ceph/ceph.conf 2024-04-26T21:44:38.022 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:37 smithi164 ceph-mon[31916]: Updating smithi072:/var/lib/ceph/a1780950-0415-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T21:44:38.022 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:37 smithi164 ceph-mon[31916]: Updating smithi076:/var/lib/ceph/a1780950-0415-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T21:44:38.023 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:37 smithi164 ceph-mon[31916]: Updating smithi164:/var/lib/ceph/a1780950-0415-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T21:44:38.023 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:37 smithi164 ceph-mon[31916]: from='client.? 172.21.15.72:0/1479048744' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:44:38.023 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:37 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:38.023 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:37 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:38.023 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:37 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:38.023 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:37 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:38.023 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:37 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:38.023 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:37 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:38.023 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:37 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:38.023 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:37 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:38.023 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:37 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:38.023 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:37 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:38.023 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:37 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:38.024 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:37 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T21:44:38.024 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:37 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-26T21:44:38.024 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:37 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:44:38.035 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-04-26T21:44:38.035 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-26T21:44:38.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:37 smithi076 ceph-mon[31823]: Updating smithi072:/var/lib/ceph/a1780950-0415-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T21:44:38.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:37 smithi076 ceph-mon[31823]: Updating smithi076:/var/lib/ceph/a1780950-0415-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T21:44:38.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:37 smithi076 ceph-mon[31823]: Updating smithi164:/var/lib/ceph/a1780950-0415-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T21:44:38.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:37 smithi076 ceph-mon[31823]: from='client.? 172.21.15.72:0/1479048744' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:44:38.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:37 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:38.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:37 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:38.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:37 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:38.066 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:37 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:38.066 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:37 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:38.066 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:37 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:38.066 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:37 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:38.066 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:37 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:38.066 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:37 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:38.066 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:37 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:38.066 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:37 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:38.066 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:37 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T21:44:38.066 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:37 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-26T21:44:38.066 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:37 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:44:38.162 INFO:tasks.cephadm:Adding mgr.x on smithi072 2024-04-26T21:44:38.162 INFO:tasks.cephadm:Adding mgr.y on smithi076 2024-04-26T21:44:38.163 INFO:tasks.cephadm:Adding mgr.z on smithi164 2024-04-26T21:44:38.163 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a1780950-0415-11ef-bc93-c7b262605968 -- ceph orch apply mgr '3;smithi072=x;smithi076=y;smithi164=z' 2024-04-26T21:44:38.625 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/a1780950-0415-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T21:44:39.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:38 smithi076 ceph-mon[31823]: Reconfiguring mon.a (unknown last config time)... 2024-04-26T21:44:39.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:38 smithi076 ceph-mon[31823]: Reconfiguring daemon mon.a on smithi072 2024-04-26T21:44:39.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:38 smithi076 ceph-mon[31823]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:44:39.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:38 smithi072 ceph-mon[28138]: Reconfiguring mon.a (unknown last config time)... 2024-04-26T21:44:39.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:38 smithi072 ceph-mon[28138]: Reconfiguring daemon mon.a on smithi072 2024-04-26T21:44:39.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:38 smithi072 ceph-mon[28138]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:44:39.099 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:38 smithi164 ceph-mon[31916]: Reconfiguring mon.a (unknown last config time)... 2024-04-26T21:44:39.099 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:38 smithi164 ceph-mon[31916]: Reconfiguring daemon mon.a on smithi072 2024-04-26T21:44:39.099 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:38 smithi164 ceph-mon[31916]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:44:40.815 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:40 smithi076 ceph-mon[31823]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:44:40.816 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:40 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:40.816 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:40 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:40.816 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:40 smithi076 ceph-mon[31823]: Reconfiguring mon.b (monmap changed)... 2024-04-26T21:44:40.816 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:40 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T21:44:40.816 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:40 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-26T21:44:40.816 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:40 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:44:40.816 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:40 smithi076 ceph-mon[31823]: Reconfiguring daemon mon.b on smithi076 2024-04-26T21:44:40.884 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:40 smithi164 ceph-mon[31916]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:44:40.884 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:40 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:40.884 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:40 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:40.884 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:40 smithi164 ceph-mon[31916]: Reconfiguring mon.b (monmap changed)... 2024-04-26T21:44:40.884 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:40 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T21:44:40.884 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:40 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-26T21:44:40.884 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:40 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:44:40.884 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:40 smithi164 ceph-mon[31916]: Reconfiguring daemon mon.b on smithi076 2024-04-26T21:44:41.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:40 smithi072 ceph-mon[28138]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:44:41.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:40 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:41.073 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:40 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:41.073 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:40 smithi072 ceph-mon[28138]: Reconfiguring mon.b (monmap changed)... 2024-04-26T21:44:41.073 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:40 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T21:44:41.073 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:40 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-26T21:44:41.073 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:40 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:44:41.073 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:40 smithi072 ceph-mon[28138]: Reconfiguring daemon mon.b on smithi076 2024-04-26T21:44:41.356 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:41.355+0000 7fc11c400700 1 -- 172.21.15.164:0/1339031923 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc1140773d0 msgr2=0x7fc114075830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:41.356 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:41.355+0000 7fc11c400700 1 --2- 172.21.15.164:0/1339031923 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc1140773d0 0x7fc114075830 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fc108009910 tx=0x7fc108009c20 comp rx=0 tx=0).stop 2024-04-26T21:44:41.356 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:41.355+0000 7fc11c400700 1 -- 172.21.15.164:0/1339031923 shutdown_connections 2024-04-26T21:44:41.356 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:41.355+0000 7fc11c400700 1 --2- 172.21.15.164:0/1339031923 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc1140773d0 0x7fc114075830 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:41.356 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:41.355+0000 7fc11c400700 1 -- 172.21.15.164:0/1339031923 >> 172.21.15.164:0/1339031923 conn(0x7fc1140fe560 msgr2=0x7fc1141009a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:41.356 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:41.355+0000 7fc11c400700 1 -- 172.21.15.164:0/1339031923 shutdown_connections 2024-04-26T21:44:41.356 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:41.355+0000 7fc11c400700 1 -- 172.21.15.164:0/1339031923 wait complete. 2024-04-26T21:44:41.357 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:41.356+0000 7fc11c400700 1 Processor -- start 2024-04-26T21:44:41.357 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:41.356+0000 7fc11c400700 1 -- start start 2024-04-26T21:44:41.357 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:41.357+0000 7fc11c400700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc1140773d0 0x7fc11419a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:41.357 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:41.357+0000 7fc11c400700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc11419ad10 0x7fc11419fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:41.358 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:41.357+0000 7fc11c400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc11419b190 0x7fc1141a2310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:41.358 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:41.357+0000 7fc11c400700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fc11410afa0 con 0x7fc11419ad10 2024-04-26T21:44:41.358 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:41.357+0000 7fc11c400700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fc11410ae20 con 0x7fc1140773d0 2024-04-26T21:44:41.358 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:41.357+0000 7fc11c400700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fc11410b120 con 0x7fc11419b190 2024-04-26T21:44:41.358 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:41.357+0000 7fc11a400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc11419b190 0x7fc1141a2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:41.358 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:41.357+0000 7fc11a400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc11419b190 0x7fc1141a2310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:35876/0 (socket says 172.21.15.164:35876) 2024-04-26T21:44:41.358 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:41.357+0000 7fc11a400700 1 -- 172.21.15.164:0/1733882644 learned_addr learned my addr 172.21.15.164:0/1733882644 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-04-26T21:44:41.358 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:41.357+0000 7fc119000700 1 --2- 172.21.15.164:0/1733882644 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc11419ad10 0x7fc11419fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:41.359 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:41.358+0000 7fc11a400700 1 -- 172.21.15.164:0/1733882644 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc1140773d0 msgr2=0x7fc11419a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:41.359 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:41.358+0000 7fc11a400700 1 --2- 172.21.15.164:0/1733882644 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc1140773d0 0x7fc11419a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:41.359 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:41.358+0000 7fc11a400700 1 -- 172.21.15.164:0/1733882644 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc11419ad10 msgr2=0x7fc11419fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:41.359 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:41.358+0000 7fc11a400700 1 --2- 172.21.15.164:0/1733882644 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc11419ad10 0x7fc11419fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:41.359 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:41.358+0000 7fc11a400700 1 -- 172.21.15.164:0/1733882644 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1141a2be0 con 0x7fc11419b190 2024-04-26T21:44:41.360 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:41.358+0000 7fc11a400700 1 --2- 172.21.15.164:0/1733882644 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc11419b190 0x7fc1141a2310 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fc10c00ba50 tx=0x7fc10c00be10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:41.360 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:41.358+0000 7fc112000700 1 -- 172.21.15.164:0/1733882644 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc10c019070 con 0x7fc11419b190 2024-04-26T21:44:41.360 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:41.358+0000 7fc11c400700 1 -- 172.21.15.164:0/1733882644 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1141a2ed0 con 0x7fc11419b190 2024-04-26T21:44:41.360 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:41.358+0000 7fc112000700 1 -- 172.21.15.164:0/1733882644 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc10c014440 con 0x7fc11419b190 2024-04-26T21:44:41.360 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:41.358+0000 7fc112000700 1 -- 172.21.15.164:0/1733882644 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc10c0075b0 con 0x7fc11419b190 2024-04-26T21:44:41.360 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:41.359+0000 7fc11c400700 1 -- 172.21.15.164:0/1733882644 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1141a33b0 con 0x7fc11419b190 2024-04-26T21:44:41.360 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:41.360+0000 7fc112000700 1 -- 172.21.15.164:0/1733882644 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fc10c01c070 con 0x7fc11419b190 2024-04-26T21:44:41.361 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:41.360+0000 7fc11c400700 1 -- 172.21.15.164:0/1733882644 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc114064a80 con 0x7fc11419b190 2024-04-26T21:44:41.361 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:41.360+0000 7fc112000700 1 --2- 172.21.15.164:0/1733882644 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fc0fc039ae0 0x7fc0fc03bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:41.361 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:41.360+0000 7fc112000700 1 -- 172.21.15.164:0/1733882644 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fc10c04bfb0 con 0x7fc11419b190 2024-04-26T21:44:41.361 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:41.361+0000 7fc119a00700 1 --2- 172.21.15.164:0/1733882644 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fc0fc039ae0 0x7fc0fc03bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:41.362 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:41.362+0000 7fc119a00700 1 --2- 172.21.15.164:0/1733882644 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fc0fc039ae0 0x7fc0fc03bfa0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fc1080051f0 tx=0x7fc1080058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:41.367 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:41.367+0000 7fc112000700 1 -- 172.21.15.164:0/1733882644 <== mon.1 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fc10c01b2c0 con 0x7fc11419b190 2024-04-26T21:44:41.607 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:41.607+0000 7fc11c400700 1 -- 172.21.15.164:0/1733882644 --> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi072=x;smithi076=y;smithi164=z", "target": ["mon-mgr", ""]}) -- 0x7fc1140638f0 con 0x7fc0fc039ae0 2024-04-26T21:44:41.622 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:41.622+0000 7fc112000700 1 -- 172.21.15.164:0/1733882644 <== mgr.14152 v2:172.21.15.72:6800/588150172 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fc1140638f0 con 0x7fc0fc039ae0 2024-04-26T21:44:41.622 INFO:teuthology.orchestra.run.smithi164.stdout:Scheduled mgr update... 2024-04-26T21:44:41.625 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:41.624+0000 7fc11c400700 1 -- 172.21.15.164:0/1733882644 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fc0fc039ae0 msgr2=0x7fc0fc03bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:41.625 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:41.624+0000 7fc11c400700 1 --2- 172.21.15.164:0/1733882644 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fc0fc039ae0 0x7fc0fc03bfa0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fc1080051f0 tx=0x7fc1080058e0 comp rx=0 tx=0).stop 2024-04-26T21:44:41.625 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:41.624+0000 7fc11c400700 1 -- 172.21.15.164:0/1733882644 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc11419b190 msgr2=0x7fc1141a2310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:41.625 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:41.624+0000 7fc11c400700 1 --2- 172.21.15.164:0/1733882644 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc11419b190 0x7fc1141a2310 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fc10c00ba50 tx=0x7fc10c00be10 comp rx=0 tx=0).stop 2024-04-26T21:44:41.625 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:41.625+0000 7fc11c400700 1 -- 172.21.15.164:0/1733882644 shutdown_connections 2024-04-26T21:44:41.625 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:41.625+0000 7fc11c400700 1 --2- 172.21.15.164:0/1733882644 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fc0fc039ae0 0x7fc0fc03bfa0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:41.626 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:41.625+0000 7fc11c400700 1 --2- 172.21.15.164:0/1733882644 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc11419b190 0x7fc1141a2310 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:41.626 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:41.625+0000 7fc11c400700 1 --2- 172.21.15.164:0/1733882644 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc1140773d0 0x7fc11419a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:41.626 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:41.625+0000 7fc11c400700 1 --2- 172.21.15.164:0/1733882644 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc11419ad10 0x7fc11419fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:41.626 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:41.625+0000 7fc11c400700 1 -- 172.21.15.164:0/1733882644 >> 172.21.15.164:0/1733882644 conn(0x7fc1140fe560 msgr2=0x7fc1140ff260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:41.626 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:41.625+0000 7fc11c400700 1 -- 172.21.15.164:0/1733882644 shutdown_connections 2024-04-26T21:44:41.626 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:44:41.625+0000 7fc11c400700 1 -- 172.21.15.164:0/1733882644 wait complete. 2024-04-26T21:44:42.315 DEBUG:teuthology.orchestra.run.smithi076:mgr.y> sudo journalctl -f -n 0 -u ceph-a1780950-0415-11ef-bc93-c7b262605968@mgr.y.service 2024-04-26T21:44:42.323 DEBUG:teuthology.orchestra.run.smithi164:mgr.z> sudo journalctl -f -n 0 -u ceph-a1780950-0415-11ef-bc93-c7b262605968@mgr.z.service 2024-04-26T21:44:42.328 INFO:tasks.cephadm:Deploying OSDs... 2024-04-26T21:44:42.329 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2024-04-26T21:44:42.329 DEBUG:teuthology.orchestra.run.smithi072:> dd if=/scratch_devs of=/dev/stdout 2024-04-26T21:44:42.386 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-26T21:44:42.386 DEBUG:teuthology.orchestra.run.smithi072:> stat /dev/vg_nvme/lv_1 2024-04-26T21:44:42.439 INFO:teuthology.orchestra.run.smithi072.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-26T21:44:42.439 INFO:teuthology.orchestra.run.smithi072.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T21:44:42.439 INFO:teuthology.orchestra.run.smithi072.stdout:Device: 6h/6d Inode: 629 Links: 1 2024-04-26T21:44:42.439 INFO:teuthology.orchestra.run.smithi072.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T21:44:42.439 INFO:teuthology.orchestra.run.smithi072.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T21:44:42.440 INFO:teuthology.orchestra.run.smithi072.stdout:Access: 2024-04-26 21:43:29.973285506 +0000 2024-04-26T21:44:42.440 INFO:teuthology.orchestra.run.smithi072.stdout:Modify: 2024-04-26 21:37:09.635897047 +0000 2024-04-26T21:44:42.440 INFO:teuthology.orchestra.run.smithi072.stdout:Change: 2024-04-26 21:37:09.635897047 +0000 2024-04-26T21:44:42.440 INFO:teuthology.orchestra.run.smithi072.stdout: Birth: 2024-04-26 21:37:09.635897047 +0000 2024-04-26T21:44:42.440 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-26T21:44:42.520 INFO:teuthology.orchestra.run.smithi072.stderr:1+0 records in 2024-04-26T21:44:42.520 INFO:teuthology.orchestra.run.smithi072.stderr:1+0 records out 2024-04-26T21:44:42.521 INFO:teuthology.orchestra.run.smithi072.stderr:512 bytes copied, 0.000419344 s, 1.2 MB/s 2024-04-26T21:44:42.524 DEBUG:teuthology.orchestra.run.smithi072:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-26T21:44:42.579 DEBUG:teuthology.orchestra.run.smithi072:> stat /dev/vg_nvme/lv_2 2024-04-26T21:44:42.630 INFO:teuthology.orchestra.run.smithi072.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-26T21:44:42.630 INFO:teuthology.orchestra.run.smithi072.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T21:44:42.630 INFO:teuthology.orchestra.run.smithi072.stdout:Device: 6h/6d Inode: 650 Links: 1 2024-04-26T21:44:42.630 INFO:teuthology.orchestra.run.smithi072.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T21:44:42.631 INFO:teuthology.orchestra.run.smithi072.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T21:44:42.631 INFO:teuthology.orchestra.run.smithi072.stdout:Access: 2024-04-26 21:43:30.037284213 +0000 2024-04-26T21:44:42.631 INFO:teuthology.orchestra.run.smithi072.stdout:Modify: 2024-04-26 21:37:10.218885246 +0000 2024-04-26T21:44:42.631 INFO:teuthology.orchestra.run.smithi072.stdout:Change: 2024-04-26 21:37:10.218885246 +0000 2024-04-26T21:44:42.631 INFO:teuthology.orchestra.run.smithi072.stdout: Birth: 2024-04-26 21:37:10.218885246 +0000 2024-04-26T21:44:42.631 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-26T21:44:42.707 INFO:teuthology.orchestra.run.smithi072.stderr:1+0 records in 2024-04-26T21:44:42.708 INFO:teuthology.orchestra.run.smithi072.stderr:1+0 records out 2024-04-26T21:44:42.708 INFO:teuthology.orchestra.run.smithi072.stderr:512 bytes copied, 0.000390424 s, 1.3 MB/s 2024-04-26T21:44:42.711 DEBUG:teuthology.orchestra.run.smithi072:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-26T21:44:42.766 DEBUG:teuthology.orchestra.run.smithi072:> stat /dev/vg_nvme/lv_3 2024-04-26T21:44:42.815 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:42 smithi076 ceph-mon[31823]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi072=x;smithi076=y;smithi164=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:44:42.815 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:42 smithi076 ceph-mon[31823]: Saving service mgr spec with placement smithi072=x;smithi076=y;smithi164=z;count:3 2024-04-26T21:44:42.815 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:42 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:42.815 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:42 smithi076 ceph-mon[31823]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:44:42.815 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:42 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:42.815 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:42 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:42.815 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:42 smithi076 ceph-mon[31823]: Reconfiguring mon.c (monmap changed)... 2024-04-26T21:44:42.815 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:42 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T21:44:42.815 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:42 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-26T21:44:42.815 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:42 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:44:42.815 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:42 smithi076 ceph-mon[31823]: Reconfiguring daemon mon.c on smithi164 2024-04-26T21:44:42.817 INFO:teuthology.orchestra.run.smithi072.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-26T21:44:42.817 INFO:teuthology.orchestra.run.smithi072.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T21:44:42.817 INFO:teuthology.orchestra.run.smithi072.stdout:Device: 6h/6d Inode: 670 Links: 1 2024-04-26T21:44:42.817 INFO:teuthology.orchestra.run.smithi072.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T21:44:42.817 INFO:teuthology.orchestra.run.smithi072.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T21:44:42.817 INFO:teuthology.orchestra.run.smithi072.stdout:Access: 2024-04-26 21:43:30.096283022 +0000 2024-04-26T21:44:42.817 INFO:teuthology.orchestra.run.smithi072.stdout:Modify: 2024-04-26 21:37:10.794873587 +0000 2024-04-26T21:44:42.817 INFO:teuthology.orchestra.run.smithi072.stdout:Change: 2024-04-26 21:37:10.794873587 +0000 2024-04-26T21:44:42.817 INFO:teuthology.orchestra.run.smithi072.stdout: Birth: 2024-04-26 21:37:10.794873587 +0000 2024-04-26T21:44:42.817 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-26T21:44:42.826 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:42 smithi072 ceph-mon[28138]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi072=x;smithi076=y;smithi164=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:44:42.826 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:42 smithi072 ceph-mon[28138]: Saving service mgr spec with placement smithi072=x;smithi076=y;smithi164=z;count:3 2024-04-26T21:44:42.827 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:42 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:42.827 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:42 smithi072 ceph-mon[28138]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:44:42.827 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:42 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:42.827 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:42 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:42.827 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:42 smithi072 ceph-mon[28138]: Reconfiguring mon.c (monmap changed)... 2024-04-26T21:44:42.827 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:42 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T21:44:42.827 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:42 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-26T21:44:42.827 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:42 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:44:42.827 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:42 smithi072 ceph-mon[28138]: Reconfiguring daemon mon.c on smithi164 2024-04-26T21:44:42.899 INFO:teuthology.orchestra.run.smithi072.stderr:1+0 records in 2024-04-26T21:44:42.900 INFO:teuthology.orchestra.run.smithi072.stderr:1+0 records out 2024-04-26T21:44:42.900 INFO:teuthology.orchestra.run.smithi072.stderr:512 bytes copied, 0.000359772 s, 1.4 MB/s 2024-04-26T21:44:42.903 DEBUG:teuthology.orchestra.run.smithi072:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-26T21:44:42.941 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:42 smithi164 ceph-mon[31916]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi072=x;smithi076=y;smithi164=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:44:42.941 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:42 smithi164 ceph-mon[31916]: Saving service mgr spec with placement smithi072=x;smithi076=y;smithi164=z;count:3 2024-04-26T21:44:42.941 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:42 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:42.941 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:42 smithi164 ceph-mon[31916]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:44:42.941 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:42 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:42.941 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:42 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:42.942 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:42 smithi164 ceph-mon[31916]: Reconfiguring mon.c (monmap changed)... 2024-04-26T21:44:42.942 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:42 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T21:44:42.942 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:42 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-26T21:44:42.942 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:42 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:44:42.942 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:42 smithi164 ceph-mon[31916]: Reconfiguring daemon mon.c on smithi164 2024-04-26T21:44:42.960 DEBUG:teuthology.orchestra.run.smithi072:> stat /dev/vg_nvme/lv_4 2024-04-26T21:44:43.011 INFO:teuthology.orchestra.run.smithi072.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-26T21:44:43.011 INFO:teuthology.orchestra.run.smithi072.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T21:44:43.011 INFO:teuthology.orchestra.run.smithi072.stdout:Device: 6h/6d Inode: 686 Links: 1 2024-04-26T21:44:43.011 INFO:teuthology.orchestra.run.smithi072.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T21:44:43.011 INFO:teuthology.orchestra.run.smithi072.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T21:44:43.012 INFO:teuthology.orchestra.run.smithi072.stdout:Access: 2024-04-26 21:43:30.155281831 +0000 2024-04-26T21:44:43.012 INFO:teuthology.orchestra.run.smithi072.stdout:Modify: 2024-04-26 21:37:11.397861381 +0000 2024-04-26T21:44:43.012 INFO:teuthology.orchestra.run.smithi072.stdout:Change: 2024-04-26 21:37:11.397861381 +0000 2024-04-26T21:44:43.012 INFO:teuthology.orchestra.run.smithi072.stdout: Birth: 2024-04-26 21:37:11.397861381 +0000 2024-04-26T21:44:43.012 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-26T21:44:43.089 INFO:teuthology.orchestra.run.smithi072.stderr:1+0 records in 2024-04-26T21:44:43.090 INFO:teuthology.orchestra.run.smithi072.stderr:1+0 records out 2024-04-26T21:44:43.090 INFO:teuthology.orchestra.run.smithi072.stderr:512 bytes copied, 0.000365634 s, 1.4 MB/s 2024-04-26T21:44:43.093 DEBUG:teuthology.orchestra.run.smithi072:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-26T21:44:43.149 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-04-26T21:44:43.149 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/scratch_devs of=/dev/stdout 2024-04-26T21:44:43.207 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-26T21:44:43.207 DEBUG:teuthology.orchestra.run.smithi076:> stat /dev/vg_nvme/lv_1 2024-04-26T21:44:43.259 INFO:teuthology.orchestra.run.smithi076.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-26T21:44:43.259 INFO:teuthology.orchestra.run.smithi076.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T21:44:43.259 INFO:teuthology.orchestra.run.smithi076.stdout:Device: 6h/6d Inode: 634 Links: 1 2024-04-26T21:44:43.259 INFO:teuthology.orchestra.run.smithi076.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T21:44:43.259 INFO:teuthology.orchestra.run.smithi076.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T21:44:43.259 INFO:teuthology.orchestra.run.smithi076.stdout:Access: 2024-04-26 21:44:27.294616986 +0000 2024-04-26T21:44:43.259 INFO:teuthology.orchestra.run.smithi076.stdout:Modify: 2024-04-26 21:37:08.023519628 +0000 2024-04-26T21:44:43.259 INFO:teuthology.orchestra.run.smithi076.stdout:Change: 2024-04-26 21:37:08.023519628 +0000 2024-04-26T21:44:43.260 INFO:teuthology.orchestra.run.smithi076.stdout: Birth: 2024-04-26 21:37:08.023519628 +0000 2024-04-26T21:44:43.260 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-26T21:44:43.339 INFO:teuthology.orchestra.run.smithi076.stderr:1+0 records in 2024-04-26T21:44:43.339 INFO:teuthology.orchestra.run.smithi076.stderr:1+0 records out 2024-04-26T21:44:43.339 INFO:teuthology.orchestra.run.smithi076.stderr:512 bytes copied, 0.000470245 s, 1.1 MB/s 2024-04-26T21:44:43.343 DEBUG:teuthology.orchestra.run.smithi076:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-26T21:44:43.398 DEBUG:teuthology.orchestra.run.smithi076:> stat /dev/vg_nvme/lv_2 2024-04-26T21:44:43.448 INFO:teuthology.orchestra.run.smithi076.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-26T21:44:43.448 INFO:teuthology.orchestra.run.smithi076.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T21:44:43.448 INFO:teuthology.orchestra.run.smithi076.stdout:Device: 6h/6d Inode: 653 Links: 1 2024-04-26T21:44:43.448 INFO:teuthology.orchestra.run.smithi076.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T21:44:43.448 INFO:teuthology.orchestra.run.smithi076.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T21:44:43.448 INFO:teuthology.orchestra.run.smithi076.stdout:Access: 2024-04-26 21:44:27.294616986 +0000 2024-04-26T21:44:43.448 INFO:teuthology.orchestra.run.smithi076.stdout:Modify: 2024-04-26 21:37:08.623501015 +0000 2024-04-26T21:44:43.448 INFO:teuthology.orchestra.run.smithi076.stdout:Change: 2024-04-26 21:37:08.623501015 +0000 2024-04-26T21:44:43.448 INFO:teuthology.orchestra.run.smithi076.stdout: Birth: 2024-04-26 21:37:08.623501015 +0000 2024-04-26T21:44:43.448 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-26T21:44:43.523 INFO:teuthology.orchestra.run.smithi076.stderr:1+0 records in 2024-04-26T21:44:43.524 INFO:teuthology.orchestra.run.smithi076.stderr:1+0 records out 2024-04-26T21:44:43.524 INFO:teuthology.orchestra.run.smithi076.stderr:512 bytes copied, 0.000390505 s, 1.3 MB/s 2024-04-26T21:44:43.527 DEBUG:teuthology.orchestra.run.smithi076:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-26T21:44:43.580 DEBUG:teuthology.orchestra.run.smithi076:> stat /dev/vg_nvme/lv_3 2024-04-26T21:44:43.626 INFO:teuthology.orchestra.run.smithi076.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-26T21:44:43.626 INFO:teuthology.orchestra.run.smithi076.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T21:44:43.626 INFO:teuthology.orchestra.run.smithi076.stdout:Device: 6h/6d Inode: 670 Links: 1 2024-04-26T21:44:43.626 INFO:teuthology.orchestra.run.smithi076.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T21:44:43.626 INFO:teuthology.orchestra.run.smithi076.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T21:44:43.627 INFO:teuthology.orchestra.run.smithi076.stdout:Access: 2024-04-26 21:44:27.295616956 +0000 2024-04-26T21:44:43.627 INFO:teuthology.orchestra.run.smithi076.stdout:Modify: 2024-04-26 21:37:09.267481037 +0000 2024-04-26T21:44:43.627 INFO:teuthology.orchestra.run.smithi076.stdout:Change: 2024-04-26 21:37:09.267481037 +0000 2024-04-26T21:44:43.627 INFO:teuthology.orchestra.run.smithi076.stdout: Birth: 2024-04-26 21:37:09.267481037 +0000 2024-04-26T21:44:43.627 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-26T21:44:43.739 INFO:teuthology.orchestra.run.smithi076.stderr:1+0 records in 2024-04-26T21:44:43.739 INFO:teuthology.orchestra.run.smithi076.stderr:1+0 records out 2024-04-26T21:44:43.739 INFO:teuthology.orchestra.run.smithi076.stderr:512 bytes copied, 0.000358901 s, 1.4 MB/s 2024-04-26T21:44:43.742 DEBUG:teuthology.orchestra.run.smithi076:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-26T21:44:43.793 DEBUG:teuthology.orchestra.run.smithi076:> stat /dev/vg_nvme/lv_4 2024-04-26T21:44:43.835 INFO:teuthology.orchestra.run.smithi076.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-26T21:44:43.835 INFO:teuthology.orchestra.run.smithi076.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T21:44:43.835 INFO:teuthology.orchestra.run.smithi076.stdout:Device: 6h/6d Inode: 688 Links: 1 2024-04-26T21:44:43.836 INFO:teuthology.orchestra.run.smithi076.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T21:44:43.836 INFO:teuthology.orchestra.run.smithi076.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T21:44:43.836 INFO:teuthology.orchestra.run.smithi076.stdout:Access: 2024-04-26 21:44:27.296616927 +0000 2024-04-26T21:44:43.836 INFO:teuthology.orchestra.run.smithi076.stdout:Modify: 2024-04-26 21:37:09.863462548 +0000 2024-04-26T21:44:43.836 INFO:teuthology.orchestra.run.smithi076.stdout:Change: 2024-04-26 21:37:09.863462548 +0000 2024-04-26T21:44:43.836 INFO:teuthology.orchestra.run.smithi076.stdout: Birth: 2024-04-26 21:37:09.863462548 +0000 2024-04-26T21:44:43.836 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-26T21:44:43.952 INFO:teuthology.orchestra.run.smithi076.stderr:1+0 records in 2024-04-26T21:44:43.952 INFO:teuthology.orchestra.run.smithi076.stderr:1+0 records out 2024-04-26T21:44:43.952 INFO:teuthology.orchestra.run.smithi076.stderr:512 bytes copied, 0.00041115 s, 1.2 MB/s 2024-04-26T21:44:43.955 DEBUG:teuthology.orchestra.run.smithi076:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-26T21:44:44.007 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-04-26T21:44:44.007 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/scratch_devs of=/dev/stdout 2024-04-26T21:44:44.056 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-26T21:44:44.056 DEBUG:teuthology.orchestra.run.smithi164:> stat /dev/vg_nvme/lv_1 2024-04-26T21:44:44.146 INFO:teuthology.orchestra.run.smithi164.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-26T21:44:44.147 INFO:teuthology.orchestra.run.smithi164.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T21:44:44.147 INFO:teuthology.orchestra.run.smithi164.stdout:Device: 6h/6d Inode: 633 Links: 1 2024-04-26T21:44:44.147 INFO:teuthology.orchestra.run.smithi164.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T21:44:44.147 INFO:teuthology.orchestra.run.smithi164.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T21:44:44.147 INFO:teuthology.orchestra.run.smithi164.stdout:Access: 2024-04-26 21:44:15.840527767 +0000 2024-04-26T21:44:44.147 INFO:teuthology.orchestra.run.smithi164.stdout:Modify: 2024-04-26 21:37:08.375443736 +0000 2024-04-26T21:44:44.147 INFO:teuthology.orchestra.run.smithi164.stdout:Change: 2024-04-26 21:37:08.375443736 +0000 2024-04-26T21:44:44.147 INFO:teuthology.orchestra.run.smithi164.stdout: Birth: 2024-04-26 21:37:08.375443736 +0000 2024-04-26T21:44:44.147 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-26T21:44:44.211 INFO:teuthology.orchestra.run.smithi164.stderr:1+0 records in 2024-04-26T21:44:44.211 INFO:teuthology.orchestra.run.smithi164.stderr:1+0 records out 2024-04-26T21:44:44.211 INFO:teuthology.orchestra.run.smithi164.stderr:512 bytes copied, 0.0003269 s, 1.6 MB/s 2024-04-26T21:44:44.214 DEBUG:teuthology.orchestra.run.smithi164:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-26T21:44:44.265 DEBUG:teuthology.orchestra.run.smithi164:> stat /dev/vg_nvme/lv_2 2024-04-26T21:44:44.310 INFO:teuthology.orchestra.run.smithi164.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-26T21:44:44.310 INFO:teuthology.orchestra.run.smithi164.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T21:44:44.310 INFO:teuthology.orchestra.run.smithi164.stdout:Device: 6h/6d Inode: 651 Links: 1 2024-04-26T21:44:44.310 INFO:teuthology.orchestra.run.smithi164.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T21:44:44.310 INFO:teuthology.orchestra.run.smithi164.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T21:44:44.311 INFO:teuthology.orchestra.run.smithi164.stdout:Access: 2024-04-26 21:44:15.841527749 +0000 2024-04-26T21:44:44.311 INFO:teuthology.orchestra.run.smithi164.stdout:Modify: 2024-04-26 21:37:08.968432501 +0000 2024-04-26T21:44:44.311 INFO:teuthology.orchestra.run.smithi164.stdout:Change: 2024-04-26 21:37:08.968432501 +0000 2024-04-26T21:44:44.311 INFO:teuthology.orchestra.run.smithi164.stdout: Birth: 2024-04-26 21:37:08.968432501 +0000 2024-04-26T21:44:44.311 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-26T21:44:44.428 INFO:teuthology.orchestra.run.smithi164.stderr:1+0 records in 2024-04-26T21:44:44.428 INFO:teuthology.orchestra.run.smithi164.stderr:1+0 records out 2024-04-26T21:44:44.428 INFO:teuthology.orchestra.run.smithi164.stderr:512 bytes copied, 0.000350571 s, 1.5 MB/s 2024-04-26T21:44:44.432 DEBUG:teuthology.orchestra.run.smithi164:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-26T21:44:44.481 DEBUG:teuthology.orchestra.run.smithi164:> stat /dev/vg_nvme/lv_3 2024-04-26T21:44:44.522 INFO:teuthology.orchestra.run.smithi164.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-26T21:44:44.522 INFO:teuthology.orchestra.run.smithi164.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T21:44:44.522 INFO:teuthology.orchestra.run.smithi164.stdout:Device: 6h/6d Inode: 672 Links: 1 2024-04-26T21:44:44.522 INFO:teuthology.orchestra.run.smithi164.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T21:44:44.522 INFO:teuthology.orchestra.run.smithi164.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T21:44:44.522 INFO:teuthology.orchestra.run.smithi164.stdout:Access: 2024-04-26 21:44:15.841527749 +0000 2024-04-26T21:44:44.522 INFO:teuthology.orchestra.run.smithi164.stdout:Modify: 2024-04-26 21:37:09.554421399 +0000 2024-04-26T21:44:44.522 INFO:teuthology.orchestra.run.smithi164.stdout:Change: 2024-04-26 21:37:09.554421399 +0000 2024-04-26T21:44:44.522 INFO:teuthology.orchestra.run.smithi164.stdout: Birth: 2024-04-26 21:37:09.554421399 +0000 2024-04-26T21:44:44.523 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-26T21:44:44.626 INFO:teuthology.orchestra.run.smithi164.stderr:1+0 records in 2024-04-26T21:44:44.626 INFO:teuthology.orchestra.run.smithi164.stderr:1+0 records out 2024-04-26T21:44:44.626 INFO:teuthology.orchestra.run.smithi164.stderr:512 bytes copied, 0.000303003 s, 1.7 MB/s 2024-04-26T21:44:44.629 DEBUG:teuthology.orchestra.run.smithi164:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-26T21:44:44.673 DEBUG:teuthology.orchestra.run.smithi164:> stat /dev/vg_nvme/lv_4 2024-04-26T21:44:44.756 INFO:teuthology.orchestra.run.smithi164.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-26T21:44:44.756 INFO:teuthology.orchestra.run.smithi164.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T21:44:44.756 INFO:teuthology.orchestra.run.smithi164.stdout:Device: 6h/6d Inode: 688 Links: 1 2024-04-26T21:44:44.756 INFO:teuthology.orchestra.run.smithi164.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T21:44:44.756 INFO:teuthology.orchestra.run.smithi164.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T21:44:44.756 INFO:teuthology.orchestra.run.smithi164.stdout:Access: 2024-04-26 21:44:15.841527749 +0000 2024-04-26T21:44:44.757 INFO:teuthology.orchestra.run.smithi164.stdout:Modify: 2024-04-26 21:37:10.181409520 +0000 2024-04-26T21:44:44.757 INFO:teuthology.orchestra.run.smithi164.stdout:Change: 2024-04-26 21:37:10.181409520 +0000 2024-04-26T21:44:44.757 INFO:teuthology.orchestra.run.smithi164.stdout: Birth: 2024-04-26 21:37:10.181409520 +0000 2024-04-26T21:44:44.757 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-26T21:44:44.869 INFO:teuthology.orchestra.run.smithi164.stderr:1+0 records in 2024-04-26T21:44:44.870 INFO:teuthology.orchestra.run.smithi164.stderr:1+0 records out 2024-04-26T21:44:44.870 INFO:teuthology.orchestra.run.smithi164.stderr:512 bytes copied, 0.000358784 s, 1.4 MB/s 2024-04-26T21:44:44.871 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:44 smithi164 ceph-mon[31916]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:44:44.871 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:44 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:44.871 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:44 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:44.871 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:44 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:44:44.871 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:44 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:44:44.871 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:44 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:44:44.871 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:44 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:44.871 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:44 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-26T21:44:44.871 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:44 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-26T21:44:44.871 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:44 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-26T21:44:44.871 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:44 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:44:44.871 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:44 smithi164 ceph-mon[31916]: Deploying daemon mgr.z on smithi164 2024-04-26T21:44:44.874 DEBUG:teuthology.orchestra.run.smithi164:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-26T21:44:44.926 INFO:tasks.cephadm:Deploying osd.0 on smithi072 with /dev/vg_nvme/lv_4... 2024-04-26T21:44:44.926 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a1780950-0415-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-26T21:44:45.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:44 smithi076 ceph-mon[31823]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:44:45.066 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:44 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:45.066 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:44 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:45.066 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:44 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:44:45.066 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:44 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:44:45.066 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:44 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:44:45.066 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:44 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:45.066 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:44 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-26T21:44:45.066 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:44 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-26T21:44:45.066 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:44 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-26T21:44:45.066 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:44 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:44:45.066 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:44 smithi076 ceph-mon[31823]: Deploying daemon mgr.z on smithi164 2024-04-26T21:44:45.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:44 smithi072 ceph-mon[28138]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:44:45.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:44 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:45.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:44 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:45.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:44 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:44:45.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:44 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:44:45.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:44 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:44:45.073 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:44 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:45.073 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:44 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-26T21:44:45.073 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:44 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-26T21:44:45.073 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:44 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-26T21:44:45.073 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:44 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:44:45.073 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:44 smithi072 ceph-mon[28138]: Deploying daemon mgr.z on smithi164 2024-04-26T21:44:45.363 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/a1780950-0415-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:44:47.064 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:46 smithi076 ceph-mon[31823]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:44:47.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:46 smithi072 ceph-mon[28138]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:44:47.136 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:46 smithi164 ceph-mon[31916]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:44:47.419 INFO:journalctl@ceph.mgr.z.smithi164.stdout:Apr 26 21:44:47 smithi164 podman[33630]: 2024-04-26 21:44:47.023632187 +0000 UTC m=+0.033314928 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-26T21:44:47.670 INFO:journalctl@ceph.mgr.z.smithi164.stdout:Apr 26 21:44:47 smithi164 podman[33630]: 2024-04-26 21:44:47.418386185 +0000 UTC m=+0.428068975 container init 28ad3af3bba17086d3d40a6b79cc6e1b84de81f57839e8bc7ce787885a1e40c0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-mgr-z, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff) 2024-04-26T21:44:47.670 INFO:journalctl@ceph.mgr.z.smithi164.stdout:Apr 26 21:44:47 smithi164 podman[33630]: 2024-04-26 21:44:47.42846994 +0000 UTC m=+0.438152685 container start 28ad3af3bba17086d3d40a6b79cc6e1b84de81f57839e8bc7ce787885a1e40c0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-mgr-z, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.vendor=CentOS) 2024-04-26T21:44:47.670 INFO:journalctl@ceph.mgr.z.smithi164.stdout:Apr 26 21:44:47 smithi164 ceph-mgr[33648]: -- 172.21.15.164:0/2703630014 <== mon.1 v2:172.21.15.164:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55bec41b85a0 con 0x55bec418c800 2024-04-26T21:44:47.670 INFO:journalctl@ceph.mgr.z.smithi164.stdout:Apr 26 21:44:47 smithi164 bash[33630]: 28ad3af3bba17086d3d40a6b79cc6e1b84de81f57839e8bc7ce787885a1e40c0 2024-04-26T21:44:47.670 INFO:journalctl@ceph.mgr.z.smithi164.stdout:Apr 26 21:44:47 smithi164 systemd[1]: Started Ceph mgr.z for a1780950-0415-11ef-bc93-c7b262605968. 2024-04-26T21:44:47.670 INFO:journalctl@ceph.mgr.z.smithi164.stdout:Apr 26 21:44:47 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-mgr-z[33644]: 2024-04-26T21:44:47.667+0000 7f129518e200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-26T21:44:48.185 INFO:teuthology.orchestra.run.smithi072.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-26T21:44:48.186 INFO:teuthology.orchestra.run.smithi072.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-26T21:44:48.186 INFO:teuthology.orchestra.run.smithi072.stderr: stderr: 10+0 records in 2024-04-26T21:44:48.186 INFO:teuthology.orchestra.run.smithi072.stderr:10+0 records out 2024-04-26T21:44:48.186 INFO:teuthology.orchestra.run.smithi072.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0370069 s, 283 MB/s 2024-04-26T21:44:48.186 INFO:teuthology.orchestra.run.smithi072.stderr:--> Zapping successful for: 2024-04-26T21:44:48.190 INFO:journalctl@ceph.mgr.z.smithi164.stdout:Apr 26 21:44:47 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-mgr-z[33644]: 2024-04-26T21:44:47.918+0000 7f129518e200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-26T21:44:48.190 INFO:journalctl@ceph.mgr.z.smithi164.stdout:Apr 26 21:44:48 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-mgr-z[33644]: 2024-04-26T21:44:48.049+0000 7f129518e200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-26T21:44:48.822 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:48 smithi072 ceph-mon[28138]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:44:48.822 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:48 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:48.822 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:48 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:48.822 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:48 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:48.823 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:48 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-26T21:44:48.823 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:48 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-26T21:44:48.823 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:48 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-26T21:44:48.823 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:48 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:44:48.823 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:48 smithi072 ceph-mon[28138]: Deploying daemon mgr.y on smithi076 2024-04-26T21:44:48.922 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:48 smithi076 ceph-mon[31823]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:44:48.922 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:48 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:48.922 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:48 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:48.922 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:48 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:48.922 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:48 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-26T21:44:48.922 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:48 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-26T21:44:48.922 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:48 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-26T21:44:48.922 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:48 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:44:48.923 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:48 smithi076 ceph-mon[31823]: Deploying daemon mgr.y on smithi076 2024-04-26T21:44:48.939 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a1780950-0415-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi072:vg_nvme/lv_4 2024-04-26T21:44:48.940 INFO:journalctl@ceph.mgr.z.smithi164.stdout:Apr 26 21:44:48 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-mgr-z[33644]: 2024-04-26T21:44:48.453+0000 7f129518e200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-26T21:44:48.940 INFO:journalctl@ceph.mgr.z.smithi164.stdout:Apr 26 21:44:48 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-mgr-z[33644]: 2024-04-26T21:44:48.565+0000 7f129518e200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-26T21:44:48.941 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:48 smithi164 ceph-mon[31916]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:44:48.941 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:48 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:48.941 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:48 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:48.941 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:48 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:48.941 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:48 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-26T21:44:48.941 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:48 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-26T21:44:48.941 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:48 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-26T21:44:48.941 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:48 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:44:48.941 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:48 smithi164 ceph-mon[31916]: Deploying daemon mgr.y on smithi076 2024-04-26T21:44:49.373 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/a1780950-0415-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:44:50.440 INFO:journalctl@ceph.mgr.z.smithi164.stdout:Apr 26 21:44:50 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-mgr-z[33644]: 2024-04-26T21:44:50.140+0000 7f129518e200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-26T21:44:50.441 INFO:journalctl@ceph.mgr.z.smithi164.stdout:Apr 26 21:44:50 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-mgr-z[33644]: 2024-04-26T21:44:50.366+0000 7f129518e200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-26T21:44:50.822 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:50 smithi072 ceph-mon[28138]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:44:50.940 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:50 smithi164 ceph-mon[31916]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:44:50.940 INFO:journalctl@ceph.mgr.z.smithi164.stdout:Apr 26 21:44:50 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-mgr-z[33644]: 2024-04-26T21:44:50.580+0000 7f129518e200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-26T21:44:51.001 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:50 smithi076 ceph-mon[31823]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:44:51.315 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Apr 26 21:44:51 smithi076 podman[33185]: 2024-04-26 21:44:50.912334706 +0000 UTC m=+0.035886991 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-26T21:44:51.354 INFO:journalctl@ceph.mgr.z.smithi164.stdout:Apr 26 21:44:50 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-mgr-z[33644]: 2024-04-26T21:44:50.959+0000 7f129518e200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-26T21:44:51.354 INFO:journalctl@ceph.mgr.z.smithi164.stdout:Apr 26 21:44:51 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-mgr-z[33644]: 2024-04-26T21:44:51.080+0000 7f129518e200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-26T21:44:51.410 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:44:51.409+0000 7f3106c00700 1 -- 172.21.15.72:0/230175235 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f31001056c0 msgr2=0x7f3100105ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:51.410 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:44:51.409+0000 7f3106c00700 1 --2- 172.21.15.72:0/230175235 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f31001056c0 0x7f3100105ae0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f30f00099c0 tx=0x7f30f0009cd0 comp rx=0 tx=0).stop 2024-04-26T21:44:51.410 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:44:51.409+0000 7f3106c00700 1 -- 172.21.15.72:0/230175235 shutdown_connections 2024-04-26T21:44:51.410 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:44:51.409+0000 7f3106c00700 1 --2- 172.21.15.72:0/230175235 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f31001056c0 0x7f3100105ae0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:51.410 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:44:51.409+0000 7f3106c00700 1 --2- 172.21.15.72:0/230175235 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3100107260 0x7f310010db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:51.410 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:44:51.409+0000 7f3106c00700 1 --2- 172.21.15.72:0/230175235 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f31001068c0 0x7f3100106d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:51.410 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:44:51.409+0000 7f3106c00700 1 -- 172.21.15.72:0/230175235 >> 172.21.15.72:0/230175235 conn(0x7f3100100c80 msgr2=0x7f31001030a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:51.411 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:44:51.409+0000 7f3106c00700 1 -- 172.21.15.72:0/230175235 shutdown_connections 2024-04-26T21:44:51.411 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:44:51.409+0000 7f3106c00700 1 -- 172.21.15.72:0/230175235 wait complete. 2024-04-26T21:44:51.412 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:44:51.410+0000 7f3106c00700 1 Processor -- start 2024-04-26T21:44:51.412 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:44:51.410+0000 7f3106c00700 1 -- start start 2024-04-26T21:44:51.412 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:44:51.411+0000 7f3106c00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f31001056c0 0x7f310019b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:51.412 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:44:51.411+0000 7f3106c00700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f31001068c0 0x7f310019bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:51.412 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:44:51.411+0000 7f3106c00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3100107260 0x7f31001a2d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:51.413 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:44:51.411+0000 7f3106c00700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f3100111030 con 0x7f31001056c0 2024-04-26T21:44:51.413 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:44:51.411+0000 7f3106c00700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f3100110eb0 con 0x7f31001068c0 2024-04-26T21:44:51.413 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:44:51.411+0000 7f3106c00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f31001111b0 con 0x7f3100107260 2024-04-26T21:44:51.413 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:44:51.411+0000 7f30ffe00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f31001056c0 0x7f310019b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:51.413 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:44:51.411+0000 7f30ffe00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f31001056c0 0x7f310019b790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:46258/0 (socket says 172.21.15.72:46258) 2024-04-26T21:44:51.413 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:44:51.411+0000 7f30ffe00700 1 -- 172.21.15.72:0/2297508051 learned_addr learned my addr 172.21.15.72:0/2297508051 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:44:51.413 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:44:51.411+0000 7f30ff400700 1 --2- 172.21.15.72:0/2297508051 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f31001068c0 0x7f310019bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:51.413 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:44:51.411+0000 7f3104c00700 1 --2- 172.21.15.72:0/2297508051 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3100107260 0x7f31001a2d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:51.413 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:44:51.411+0000 7f30ffe00700 1 -- 172.21.15.72:0/2297508051 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3100107260 msgr2=0x7f31001a2d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:51.413 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:44:51.411+0000 7f30ffe00700 1 --2- 172.21.15.72:0/2297508051 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3100107260 0x7f31001a2d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:51.413 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:44:51.411+0000 7f30ffe00700 1 -- 172.21.15.72:0/2297508051 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f31001068c0 msgr2=0x7f310019bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:51.413 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:44:51.411+0000 7f30ffe00700 1 --2- 172.21.15.72:0/2297508051 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f31001068c0 0x7f310019bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:51.413 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:44:51.411+0000 7f30ffe00700 1 -- 172.21.15.72:0/2297508051 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31001a33a0 con 0x7f31001056c0 2024-04-26T21:44:51.413 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:44:51.412+0000 7f30ffe00700 1 --2- 172.21.15.72:0/2297508051 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f31001056c0 0x7f310019b790 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f30f0003e60 tx=0x7f30f0003e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:51.415 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:44:51.412+0000 7f30fcc00700 1 -- 172.21.15.72:0/2297508051 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30f0009e60 con 0x7f31001056c0 2024-04-26T21:44:51.415 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:44:51.412+0000 7f30fcc00700 1 -- 172.21.15.72:0/2297508051 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f30f000f460 con 0x7f31001056c0 2024-04-26T21:44:51.415 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:44:51.412+0000 7f3106c00700 1 -- 172.21.15.72:0/2297508051 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31001a3630 con 0x7f31001056c0 2024-04-26T21:44:51.415 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:44:51.412+0000 7f30fcc00700 1 -- 172.21.15.72:0/2297508051 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30f0017650 con 0x7f31001056c0 2024-04-26T21:44:51.415 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:44:51.412+0000 7f3106c00700 1 -- 172.21.15.72:0/2297508051 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31001a3ae0 con 0x7f31001056c0 2024-04-26T21:44:51.415 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:44:51.413+0000 7f30fcc00700 1 -- 172.21.15.72:0/2297508051 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f30f000f910 con 0x7f31001056c0 2024-04-26T21:44:51.415 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:44:51.413+0000 7f3106c00700 1 -- 172.21.15.72:0/2297508051 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f310006a110 con 0x7f31001056c0 2024-04-26T21:44:51.415 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:44:51.414+0000 7f30fcc00700 1 --2- 172.21.15.72:0/2297508051 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f30e4039890 0x7f30e403bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:51.415 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:44:51.414+0000 7f30fcc00700 1 -- 172.21.15.72:0/2297508051 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f30f004d860 con 0x7f31001056c0 2024-04-26T21:44:51.415 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:44:51.414+0000 7f30ff400700 1 --2- 172.21.15.72:0/2297508051 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f30e4039890 0x7f30e403bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:51.416 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:44:51.415+0000 7f30ff400700 1 --2- 172.21.15.72:0/2297508051 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f30e4039890 0x7f30e403bd50 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f30ec007c60 tx=0x7f30ec0073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:51.420 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:44:51.420+0000 7f30fcc00700 1 -- 172.21.15.72:0/2297508051 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f30f0013070 con 0x7f31001056c0 2024-04-26T21:44:51.579 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Apr 26 21:44:51 smithi076 podman[33185]: 2024-04-26 21:44:51.328941482 +0000 UTC m=+0.452493810 container init 94ad75b255dbab5cf932bef15ed8ed469ad7902a52046655778c28c29b7f271c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-mgr-y, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424) 2024-04-26T21:44:51.580 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Apr 26 21:44:51 smithi076 podman[33185]: 2024-04-26 21:44:51.339838783 +0000 UTC m=+0.463391111 container start 94ad75b255dbab5cf932bef15ed8ed469ad7902a52046655778c28c29b7f271c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-mgr-y, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-26T21:44:51.580 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Apr 26 21:44:51 smithi076 ceph-mgr[33206]: -- 172.21.15.76:0/3835931504 <== mon.2 v2:172.21.15.76:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a51b7ac5a0 con 0x55a51b780c00 2024-04-26T21:44:51.580 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Apr 26 21:44:51 smithi076 bash[33185]: 94ad75b255dbab5cf932bef15ed8ed469ad7902a52046655778c28c29b7f271c 2024-04-26T21:44:51.580 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Apr 26 21:44:51 smithi076 systemd[1]: Started Ceph mgr.y for a1780950-0415-11ef-bc93-c7b262605968. 2024-04-26T21:44:51.609 INFO:journalctl@ceph.mgr.z.smithi164.stdout:Apr 26 21:44:51 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-mgr-z[33644]: 2024-04-26T21:44:51.352+0000 7f129518e200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-26T21:44:51.661 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:44:51.660+0000 7f3106c00700 1 -- 172.21.15.72:0/2297508051 --> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi072:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f3100063860 con 0x7f30e4039890 2024-04-26T21:44:51.839 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Apr 26 21:44:51 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-mgr-y[33202]: 2024-04-26T21:44:51.577+0000 7f4ed71c3200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-26T21:44:51.839 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Apr 26 21:44:51 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-mgr-y[33202]: 2024-04-26T21:44:51.837+0000 7f4ed71c3200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-26T21:44:51.941 INFO:journalctl@ceph.mgr.z.smithi164.stdout:Apr 26 21:44:51 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-mgr-z[33644]: 2024-04-26T21:44:51.607+0000 7f129518e200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-26T21:44:52.314 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Apr 26 21:44:51 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-mgr-y[33202]: 2024-04-26T21:44:51.978+0000 7f4ed71c3200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-26T21:44:52.690 INFO:journalctl@ceph.mgr.z.smithi164.stdout:Apr 26 21:44:52 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-mgr-z[33644]: 2024-04-26T21:44:52.422+0000 7f129518e200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-26T21:44:52.691 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:52 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:52.691 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:52 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:52.691 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:52 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:52.691 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:52 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:52.691 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:52 smithi164 ceph-mon[31916]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:44:52.691 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:52 smithi164 ceph-mon[31916]: from='client.14226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi072:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:44:52.691 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:52 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:44:52.691 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:52 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:44:52.691 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:52 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:44:52.691 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:52 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:44:52.814 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Apr 26 21:44:52 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-mgr-y[33202]: 2024-04-26T21:44:52.394+0000 7f4ed71c3200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-26T21:44:52.814 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Apr 26 21:44:52 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-mgr-y[33202]: 2024-04-26T21:44:52.509+0000 7f4ed71c3200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-26T21:44:52.815 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:52 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:52.815 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:52 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:52.815 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:52 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:52.815 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:52 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:52.815 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:52 smithi076 ceph-mon[31823]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:44:52.815 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:52 smithi076 ceph-mon[31823]: from='client.14226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi072:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:44:52.815 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:52 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:44:52.815 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:52 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:44:52.816 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:52 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:44:52.816 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:52 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:44:52.822 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:52 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:52.822 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:52 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:52.822 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:52 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:52.823 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:52 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:52.823 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:52 smithi072 ceph-mon[28138]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:44:52.823 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:52 smithi072 ceph-mon[28138]: from='client.14226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi072:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:44:52.823 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:52 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:44:52.823 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:52 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:44:52.823 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:52 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:44:52.823 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:52 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:44:53.309 INFO:journalctl@ceph.mgr.z.smithi164.stdout:Apr 26 21:44:53 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-mgr-z[33644]: 2024-04-26T21:44:53.011+0000 7f129518e200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-26T21:44:53.309 INFO:journalctl@ceph.mgr.z.smithi164.stdout:Apr 26 21:44:53 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-mgr-z[33644]: 2024-04-26T21:44:53.142+0000 7f129518e200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-26T21:44:53.691 INFO:journalctl@ceph.mgr.z.smithi164.stdout:Apr 26 21:44:53 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-mgr-z[33644]: 2024-04-26T21:44:53.415+0000 7f129518e200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-26T21:44:54.441 INFO:journalctl@ceph.mgr.z.smithi164.stdout:Apr 26 21:44:54 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-mgr-z[33644]: 2024-04-26T21:44:54.075+0000 7f129518e200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-26T21:44:54.441 INFO:journalctl@ceph.mgr.z.smithi164.stdout:Apr 26 21:44:54 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-mgr-z[33644]: 2024-04-26T21:44:54.200+0000 7f129518e200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-26T21:44:54.564 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Apr 26 21:44:54 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-mgr-y[33202]: 2024-04-26T21:44:54.177+0000 7f4ed71c3200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-26T21:44:54.565 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Apr 26 21:44:54 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-mgr-y[33202]: 2024-04-26T21:44:54.414+0000 7f4ed71c3200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-26T21:44:54.834 INFO:journalctl@ceph.mgr.z.smithi164.stdout:Apr 26 21:44:54 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-mgr-z[33644]: 2024-04-26T21:44:54.522+0000 7f129518e200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-26T21:44:54.834 INFO:journalctl@ceph.mgr.z.smithi164.stdout:Apr 26 21:44:54 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-mgr-z[33644]: 2024-04-26T21:44:54.702+0000 7f129518e200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-26T21:44:54.834 INFO:journalctl@ceph.mgr.z.smithi164.stdout:Apr 26 21:44:54 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-mgr-z[33644]: 2024-04-26T21:44:54.832+0000 7f129518e200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-26T21:44:54.835 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:54 smithi164 ceph-mon[31916]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:44:54.835 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:54 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:54.835 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:54 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:54.835 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:54 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:54.835 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:54 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:54.835 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:54 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:55.033 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Apr 26 21:44:54 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-mgr-y[33202]: 2024-04-26T21:44:54.636+0000 7f4ed71c3200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-26T21:44:55.033 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:54 smithi076 ceph-mon[31823]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:44:55.033 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:54 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:55.033 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:54 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:55.034 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:54 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:55.034 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:54 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:55.034 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:54 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:55.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:54 smithi072 ceph-mon[28138]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:44:55.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:54 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:55.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:54 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:55.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:54 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:55.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:54 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:55.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:54 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:55.190 INFO:journalctl@ceph.mgr.z.smithi164.stdout:Apr 26 21:44:54 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-mgr-z[33644]: 2024-04-26T21:44:54.988+0000 7f129518e200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-26T21:44:55.315 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Apr 26 21:44:55 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-mgr-y[33202]: 2024-04-26T21:44:55.030+0000 7f4ed71c3200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-26T21:44:55.315 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Apr 26 21:44:55 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-mgr-y[33202]: 2024-04-26T21:44:55.159+0000 7f4ed71c3200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-26T21:44:55.558 INFO:journalctl@ceph.mgr.z.smithi164.stdout:Apr 26 21:44:55 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-mgr-z[33644]: 2024-04-26T21:44:55.374+0000 7f129518e200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-26T21:44:55.769 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Apr 26 21:44:55 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-mgr-y[33202]: 2024-04-26T21:44:55.412+0000 7f4ed71c3200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-26T21:44:55.770 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Apr 26 21:44:55 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-mgr-y[33202]: 2024-04-26T21:44:55.661+0000 7f4ed71c3200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-26T21:44:55.861 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:55 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:44:55.862 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:55 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:44:55.862 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:55 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:55.862 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:55 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-26T21:44:55.862 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:55 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-26T21:44:55.862 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:55 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:44:55.862 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:55 smithi164 ceph-mon[31916]: from='client.? 172.21.15.72:0/2476028548' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "04e06884-b355-4762-9b29-9034b7201e9a"}]: dispatch 2024-04-26T21:44:55.862 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:55 smithi164 ceph-mon[31916]: from='client.? 172.21.15.72:0/2476028548' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "04e06884-b355-4762-9b29-9034b7201e9a"}]': finished 2024-04-26T21:44:55.862 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:55 smithi164 ceph-mon[31916]: osdmap e5: 1 total, 0 up, 1 in 2024-04-26T21:44:55.862 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:55 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T21:44:56.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:55 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:44:56.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:55 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:44:56.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:55 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:56.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:55 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-26T21:44:56.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:55 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-26T21:44:56.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:55 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:44:56.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:55 smithi076 ceph-mon[31823]: from='client.? 172.21.15.72:0/2476028548' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "04e06884-b355-4762-9b29-9034b7201e9a"}]: dispatch 2024-04-26T21:44:56.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:55 smithi076 ceph-mon[31823]: from='client.? 172.21.15.72:0/2476028548' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "04e06884-b355-4762-9b29-9034b7201e9a"}]': finished 2024-04-26T21:44:56.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:55 smithi076 ceph-mon[31823]: osdmap e5: 1 total, 0 up, 1 in 2024-04-26T21:44:56.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:55 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T21:44:56.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:55 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:44:56.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:55 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:44:56.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:55 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:44:56.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:55 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-26T21:44:56.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:55 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-26T21:44:56.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:55 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:44:56.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:55 smithi072 ceph-mon[28138]: from='client.? 172.21.15.72:0/2476028548' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "04e06884-b355-4762-9b29-9034b7201e9a"}]: dispatch 2024-04-26T21:44:56.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:55 smithi072 ceph-mon[28138]: from='client.? 172.21.15.72:0/2476028548' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "04e06884-b355-4762-9b29-9034b7201e9a"}]': finished 2024-04-26T21:44:56.073 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:55 smithi072 ceph-mon[28138]: osdmap e5: 1 total, 0 up, 1 in 2024-04-26T21:44:56.073 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:55 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T21:44:56.440 INFO:journalctl@ceph.mgr.z.smithi164.stdout:Apr 26 21:44:56 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-mgr-z[33644]: 2024-04-26T21:44:56.156+0000 7f129518e200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-26T21:44:56.753 INFO:journalctl@ceph.mgr.z.smithi164.stdout:Apr 26 21:44:56 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-mgr-z[33644]: 2024-04-26T21:44:56.447+0000 7f129518e200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-26T21:44:56.814 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Apr 26 21:44:56 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-mgr-y[33202]: 2024-04-26T21:44:56.443+0000 7f4ed71c3200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-26T21:44:56.828 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:44:56.827+0000 7f30fcc00700 1 -- 172.21.15.72:0/2297508051 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mgrmap(e 14) ==== 92602+0+0 (secure 0 0 0) 0x7f30f0021cc0 con 0x7f31001056c0 2024-04-26T21:44:57.118 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Apr 26 21:44:56 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-mgr-y[33202]: 2024-04-26T21:44:56.991+0000 7f4ed71c3200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-26T21:44:57.118 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:56 smithi076 ceph-mon[31823]: Reconfiguring mgr.x (unknown last config time)... 2024-04-26T21:44:57.119 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:56 smithi076 ceph-mon[31823]: Reconfiguring daemon mgr.x on smithi072 2024-04-26T21:44:57.119 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:56 smithi076 ceph-mon[31823]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:44:57.119 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:56 smithi076 ceph-mon[31823]: from='client.? 172.21.15.72:0/3666761646' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:44:57.119 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:56 smithi076 ceph-mon[31823]: Standby manager daemon z started 2024-04-26T21:44:57.119 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:56 smithi076 ceph-mon[31823]: from='mgr.? 172.21.15.164:0/3249086179' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-26T21:44:57.119 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:56 smithi076 ceph-mon[31823]: from='mgr.? 172.21.15.164:0/3249086179' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-26T21:44:57.119 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:56 smithi076 ceph-mon[31823]: from='mgr.? 172.21.15.164:0/3249086179' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-26T21:44:57.119 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:56 smithi076 ceph-mon[31823]: from='mgr.? 172.21.15.164:0/3249086179' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-26T21:44:57.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:56 smithi164 ceph-mon[31916]: Reconfiguring mgr.x (unknown last config time)... 2024-04-26T21:44:57.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:56 smithi164 ceph-mon[31916]: Reconfiguring daemon mgr.x on smithi072 2024-04-26T21:44:57.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:56 smithi164 ceph-mon[31916]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:44:57.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:56 smithi164 ceph-mon[31916]: from='client.? 172.21.15.72:0/3666761646' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:44:57.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:56 smithi164 ceph-mon[31916]: Standby manager daemon z started 2024-04-26T21:44:57.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:56 smithi164 ceph-mon[31916]: from='mgr.? 172.21.15.164:0/3249086179' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-26T21:44:57.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:56 smithi164 ceph-mon[31916]: from='mgr.? 172.21.15.164:0/3249086179' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-26T21:44:57.192 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:56 smithi164 ceph-mon[31916]: from='mgr.? 172.21.15.164:0/3249086179' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-26T21:44:57.192 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:56 smithi164 ceph-mon[31916]: from='mgr.? 172.21.15.164:0/3249086179' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-26T21:44:57.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:56 smithi072 ceph-mon[28138]: Reconfiguring mgr.x (unknown last config time)... 2024-04-26T21:44:57.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:56 smithi072 ceph-mon[28138]: Reconfiguring daemon mgr.x on smithi072 2024-04-26T21:44:57.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:56 smithi072 ceph-mon[28138]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:44:57.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:56 smithi072 ceph-mon[28138]: from='client.? 172.21.15.72:0/3666761646' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:44:57.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:56 smithi072 ceph-mon[28138]: Standby manager daemon z started 2024-04-26T21:44:57.323 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:56 smithi072 ceph-mon[28138]: from='mgr.? 172.21.15.164:0/3249086179' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-26T21:44:57.323 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:56 smithi072 ceph-mon[28138]: from='mgr.? 172.21.15.164:0/3249086179' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-26T21:44:57.323 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:56 smithi072 ceph-mon[28138]: from='mgr.? 172.21.15.164:0/3249086179' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-26T21:44:57.323 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:56 smithi072 ceph-mon[28138]: from='mgr.? 172.21.15.164:0/3249086179' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-26T21:44:57.814 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Apr 26 21:44:57 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-mgr-y[33202]: 2024-04-26T21:44:57.117+0000 7f4ed71c3200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-26T21:44:57.815 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Apr 26 21:44:57 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-mgr-y[33202]: 2024-04-26T21:44:57.380+0000 7f4ed71c3200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-26T21:44:58.144 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:57 smithi076 ceph-mon[31823]: mgrmap e14: x(active, since 107s), standbys: z 2024-04-26T21:44:58.144 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:57 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-26T21:44:58.144 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Apr 26 21:44:58 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-mgr-y[33202]: 2024-04-26T21:44:58.020+0000 7f4ed71c3200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-26T21:44:58.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:57 smithi164 ceph-mon[31916]: mgrmap e14: x(active, since 107s), standbys: z 2024-04-26T21:44:58.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:57 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-26T21:44:58.223 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:57 smithi072 ceph-mon[28138]: mgrmap e14: x(active, since 107s), standbys: z 2024-04-26T21:44:58.224 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:57 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-26T21:44:58.461 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Apr 26 21:44:58 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-mgr-y[33202]: 2024-04-26T21:44:58.143+0000 7f4ed71c3200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-26T21:44:58.461 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Apr 26 21:44:58 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-mgr-y[33202]: 2024-04-26T21:44:58.460+0000 7f4ed71c3200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-26T21:44:58.760 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Apr 26 21:44:58 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-mgr-y[33202]: 2024-04-26T21:44:58.635+0000 7f4ed71c3200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-26T21:44:58.783 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Apr 26 21:44:58 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-mgr-y[33202]: 2024-04-26T21:44:58.759+0000 7f4ed71c3200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-26T21:44:58.950 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:44:58 smithi072 ceph-mon[28138]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:44:59.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:44:58 smithi076 ceph-mon[31823]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:44:59.065 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Apr 26 21:44:58 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-mgr-y[33202]: 2024-04-26T21:44:58.908+0000 7f4ed71c3200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-26T21:44:59.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:44:58 smithi164 ceph-mon[31916]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:44:59.564 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Apr 26 21:44:59 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-mgr-y[33202]: 2024-04-26T21:44:59.275+0000 7f4ed71c3200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-26T21:45:00.314 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Apr 26 21:45:00 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-mgr-y[33202]: 2024-04-26T21:45:00.033+0000 7f4ed71c3200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-26T21:45:00.814 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Apr 26 21:45:00 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-mgr-y[33202]: 2024-04-26T21:45:00.314+0000 7f4ed71c3200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-26T21:45:00.888 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:45:00.886+0000 7f30fcc00700 1 -- 172.21.15.72:0/2297508051 <== mon.0 v2:172.21.15.72:3300/0 8 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7f30f00163a0 con 0x7f31001056c0 2024-04-26T21:45:01.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:45:00 smithi164 ceph-mon[31916]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:45:01.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:45:00 smithi164 ceph-mon[31916]: Standby manager daemon y started 2024-04-26T21:45:01.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:45:00 smithi164 ceph-mon[31916]: from='mgr.? 172.21.15.76:0/2785025989' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-26T21:45:01.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:45:00 smithi164 ceph-mon[31916]: from='mgr.? 172.21.15.76:0/2785025989' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-26T21:45:01.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:45:00 smithi164 ceph-mon[31916]: from='mgr.? 172.21.15.76:0/2785025989' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-26T21:45:01.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:45:00 smithi164 ceph-mon[31916]: from='mgr.? 172.21.15.76:0/2785025989' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-26T21:45:01.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:45:00 smithi076 ceph-mon[31823]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:45:01.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:45:00 smithi076 ceph-mon[31823]: Standby manager daemon y started 2024-04-26T21:45:01.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:45:00 smithi076 ceph-mon[31823]: from='mgr.? 172.21.15.76:0/2785025989' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-26T21:45:01.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:45:00 smithi076 ceph-mon[31823]: from='mgr.? 172.21.15.76:0/2785025989' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-26T21:45:01.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:45:00 smithi076 ceph-mon[31823]: from='mgr.? 172.21.15.76:0/2785025989' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-26T21:45:01.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:45:00 smithi076 ceph-mon[31823]: from='mgr.? 172.21.15.76:0/2785025989' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-26T21:45:01.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:45:00 smithi072 ceph-mon[28138]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:45:01.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:45:00 smithi072 ceph-mon[28138]: Standby manager daemon y started 2024-04-26T21:45:01.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:45:00 smithi072 ceph-mon[28138]: from='mgr.? 172.21.15.76:0/2785025989' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-26T21:45:01.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:45:00 smithi072 ceph-mon[28138]: from='mgr.? 172.21.15.76:0/2785025989' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-26T21:45:01.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:45:00 smithi072 ceph-mon[28138]: from='mgr.? 172.21.15.76:0/2785025989' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-26T21:45:01.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:45:00 smithi072 ceph-mon[28138]: from='mgr.? 172.21.15.76:0/2785025989' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-26T21:45:02.140 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:45:01 smithi072 ceph-mon[28138]: mgrmap e15: x(active, since 111s), standbys: z, y 2024-04-26T21:45:02.140 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:45:01 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-26T21:45:02.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:45:01 smithi164 ceph-mon[31916]: mgrmap e15: x(active, since 111s), standbys: z, y 2024-04-26T21:45:02.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:45:01 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-26T21:45:02.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:45:01 smithi076 ceph-mon[31823]: mgrmap e15: x(active, since 111s), standbys: z, y 2024-04-26T21:45:02.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:45:01 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-26T21:45:03.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:45:02 smithi072 ceph-mon[28138]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:45:03.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:45:02 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:45:03.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:45:02 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:45:03.073 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:45:02 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:45:03.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:45:02 smithi164 ceph-mon[31916]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:45:03.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:45:02 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:45:03.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:45:02 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:45:03.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:45:02 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:45:03.216 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:45:02 smithi076 ceph-mon[31823]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:45:03.216 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:45:02 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:45:03.216 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:45:02 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:45:03.216 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:45:02 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:45:04.056 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:45:03 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:45:04.057 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:45:03 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:45:04.057 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:45:03 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:45:04.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:45:03 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:45:04.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:45:03 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:45:04.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:45:03 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:45:04.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:45:03 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:45:04.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:45:03 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:45:04.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:45:03 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:45:05.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:45:04 smithi076 ceph-mon[31823]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:45:05.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:45:04 smithi164 ceph-mon[31916]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:45:05.310 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:45:04 smithi072 ceph-mon[28138]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:45:06.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:45:05 smithi164 ceph-mon[31916]: pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:45:06.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:45:05 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-26T21:45:06.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:45:05 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:45:06.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:45:05 smithi164 ceph-mon[31916]: Deploying daemon osd.0 on smithi072 2024-04-26T21:45:06.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:45:05 smithi076 ceph-mon[31823]: pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:45:06.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:45:05 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-26T21:45:06.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:45:05 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:45:06.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:45:05 smithi076 ceph-mon[31823]: Deploying daemon osd.0 on smithi072 2024-04-26T21:45:06.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:45:05 smithi072 ceph-mon[28138]: pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:45:06.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:45:05 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-26T21:45:06.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:45:05 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:45:06.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:45:05 smithi072 ceph-mon[28138]: Deploying daemon osd.0 on smithi072 2024-04-26T21:45:08.965 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:45:08 smithi072 ceph-mon[28138]: pgmap v54: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:45:09.064 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:45:08 smithi076 ceph-mon[31823]: pgmap v54: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:45:09.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:45:08 smithi164 ceph-mon[31916]: pgmap v54: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:45:10.800 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:45:10 smithi072 ceph-mon[28138]: pgmap v55: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:45:11.064 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:45:10 smithi076 ceph-mon[31823]: pgmap v55: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:45:11.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:45:10 smithi164 ceph-mon[31916]: pgmap v55: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:45:13.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:45:12 smithi076 ceph-mon[31823]: pgmap v56: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:45:13.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:45:12 smithi072 ceph-mon[28138]: pgmap v56: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:45:13.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:45:12 smithi164 ceph-mon[31916]: pgmap v56: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:45:15.014 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:45:14 smithi072 ceph-mon[28138]: pgmap v57: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:45:15.014 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:45:14 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:45:15.014 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:45:14 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:45:15.014 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:45:14 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:45:15.064 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:45:14 smithi076 ceph-mon[31823]: pgmap v57: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:45:15.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:45:14 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:45:15.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:45:14 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:45:15.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:45:14 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:45:15.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:45:14 smithi164 ceph-mon[31916]: pgmap v57: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:45:15.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:45:14 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:45:15.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:45:14 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:45:15.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:45:14 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:45:17.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:45:16 smithi076 ceph-mon[31823]: pgmap v58: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:45:17.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:45:16 smithi072 ceph-mon[28138]: pgmap v58: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:45:17.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:45:16 smithi164 ceph-mon[31916]: pgmap v58: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:45:18.818 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:45:18.817+0000 7f30fcc00700 1 -- 172.21.15.72:0/2297508051 <== mgr.14152 v2:172.21.15.72:6800/588150172 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f3100063860 con 0x7f30e4039890 2024-04-26T21:45:18.819 INFO:teuthology.orchestra.run.smithi072.stdout:Created osd(s) 0 on host 'smithi072' 2024-04-26T21:45:18.822 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:45:18.821+0000 7f3106c00700 1 -- 172.21.15.72:0/2297508051 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f30e4039890 msgr2=0x7f30e403bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:18.822 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:45:18.821+0000 7f3106c00700 1 --2- 172.21.15.72:0/2297508051 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f30e4039890 0x7f30e403bd50 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f30ec007c60 tx=0x7f30ec0073d0 comp rx=0 tx=0).stop 2024-04-26T21:45:18.822 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:45:18.821+0000 7f3106c00700 1 -- 172.21.15.72:0/2297508051 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f31001056c0 msgr2=0x7f310019b790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:18.822 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:45:18.821+0000 7f3106c00700 1 --2- 172.21.15.72:0/2297508051 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f31001056c0 0x7f310019b790 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f30f0003e60 tx=0x7f30f0003e90 comp rx=0 tx=0).stop 2024-04-26T21:45:18.822 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:45:18.821+0000 7f3106c00700 1 -- 172.21.15.72:0/2297508051 shutdown_connections 2024-04-26T21:45:18.823 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:45:18.821+0000 7f3106c00700 1 --2- 172.21.15.72:0/2297508051 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f30e4039890 0x7f30e403bd50 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:18.823 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:45:18.821+0000 7f3106c00700 1 --2- 172.21.15.72:0/2297508051 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f31001056c0 0x7f310019b790 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:18.823 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:45:18.821+0000 7f3106c00700 1 --2- 172.21.15.72:0/2297508051 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3100107260 0x7f31001a2d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:18.823 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:45:18.821+0000 7f3106c00700 1 --2- 172.21.15.72:0/2297508051 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f31001068c0 0x7f310019bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:18.823 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:45:18.821+0000 7f3106c00700 1 -- 172.21.15.72:0/2297508051 >> 172.21.15.72:0/2297508051 conn(0x7f3100100c80 msgr2=0x7f3100109ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:18.823 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:45:18.821+0000 7f3106c00700 1 -- 172.21.15.72:0/2297508051 shutdown_connections 2024-04-26T21:45:18.823 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:45:18.822+0000 7f3106c00700 1 -- 172.21.15.72:0/2297508051 wait complete. 2024-04-26T21:45:18.841 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:45:18 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:45:18.841 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:45:18 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:45:18.841 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:45:18 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:45:18.841 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:45:18 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:45:18.841 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:45:18 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:45:18.841 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:45:18 smithi072 ceph-mon[28138]: pgmap v59: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:45:18.841 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:45:18 smithi072 ceph-mon[28138]: from='osd.0 [v2:172.21.15.72:6802/1416765179,v1:172.21.15.72:6803/1416765179]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-26T21:45:18.941 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:45:18 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:45:18.942 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:45:18 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:45:18.942 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:45:18 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:45:18.942 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:45:18 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:45:18.942 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:45:18 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:45:18.942 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:45:18 smithi164 ceph-mon[31916]: pgmap v59: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:45:18.942 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:45:18 smithi164 ceph-mon[31916]: from='osd.0 [v2:172.21.15.72:6802/1416765179,v1:172.21.15.72:6803/1416765179]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-26T21:45:19.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:45:18 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:45:19.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:45:18 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:45:19.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:45:18 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:45:19.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:45:18 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:45:19.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:45:18 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:45:19.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:45:18 smithi076 ceph-mon[31823]: pgmap v59: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:45:19.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:45:18 smithi076 ceph-mon[31823]: from='osd.0 [v2:172.21.15.72:6802/1416765179,v1:172.21.15.72:6803/1416765179]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-26T21:45:19.592 DEBUG:teuthology.orchestra.run.smithi072:osd.0> sudo journalctl -f -n 0 -u ceph-a1780950-0415-11ef-bc93-c7b262605968@osd.0.service 2024-04-26T21:45:19.596 INFO:tasks.cephadm:Deploying osd.1 on smithi076 with /dev/vg_nvme/lv_4... 2024-04-26T21:45:19.596 DEBUG:teuthology.orchestra.run.smithi076:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a1780950-0415-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-26T21:45:19.822 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:45:19 smithi072 ceph-mon[28138]: from='osd.0 [v2:172.21.15.72:6802/1416765179,v1:172.21.15.72:6803/1416765179]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-26T21:45:19.822 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:45:19 smithi072 ceph-mon[28138]: osdmap e6: 1 total, 0 up, 1 in 2024-04-26T21:45:19.822 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:45:19 smithi072 ceph-mon[28138]: from='osd.0 [v2:172.21.15.72:6802/1416765179,v1:172.21.15.72:6803/1416765179]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi072", "root=default"]}]: dispatch 2024-04-26T21:45:19.822 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:45:19 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T21:45:19.822 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:45:19 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:45:19.822 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:45:19 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:45:19.822 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:45:19 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:45:19.941 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:45:19 smithi164 ceph-mon[31916]: from='osd.0 [v2:172.21.15.72:6802/1416765179,v1:172.21.15.72:6803/1416765179]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-26T21:45:19.941 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:45:19 smithi164 ceph-mon[31916]: osdmap e6: 1 total, 0 up, 1 in 2024-04-26T21:45:19.941 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:45:19 smithi164 ceph-mon[31916]: from='osd.0 [v2:172.21.15.72:6802/1416765179,v1:172.21.15.72:6803/1416765179]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi072", "root=default"]}]: dispatch 2024-04-26T21:45:19.941 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:45:19 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T21:45:19.941 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:45:19 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:45:19.941 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:45:19 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:45:19.941 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:45:19 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:45:20.029 INFO:teuthology.orchestra.run.smithi076.stderr:Inferring config /var/lib/ceph/a1780950-0415-11ef-bc93-c7b262605968/mon.b/config 2024-04-26T21:45:20.064 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:45:19 smithi076 ceph-mon[31823]: from='osd.0 [v2:172.21.15.72:6802/1416765179,v1:172.21.15.72:6803/1416765179]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-26T21:45:20.064 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:45:19 smithi076 ceph-mon[31823]: osdmap e6: 1 total, 0 up, 1 in 2024-04-26T21:45:20.064 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:45:19 smithi076 ceph-mon[31823]: from='osd.0 [v2:172.21.15.72:6802/1416765179,v1:172.21.15.72:6803/1416765179]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi072", "root=default"]}]: dispatch 2024-04-26T21:45:20.064 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:45:19 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T21:45:20.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:45:19 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:45:20.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:45:19 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:45:20.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:45:19 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:45:20.815 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:45:20 smithi076 ceph-mon[31823]: from='osd.0 [v2:172.21.15.72:6802/1416765179,v1:172.21.15.72:6803/1416765179]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi072", "root=default"]}]': finished 2024-04-26T21:45:20.815 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:45:20 smithi076 ceph-mon[31823]: osdmap e7: 1 total, 0 up, 1 in 2024-04-26T21:45:20.815 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:45:20 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T21:45:20.815 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:45:20 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T21:45:20.815 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:45:20 smithi076 ceph-mon[31823]: pgmap v62: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:45:20.815 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:45:20 smithi076 ceph-mon[31823]: from='osd.0 [v2:172.21.15.72:6802/1416765179,v1:172.21.15.72:6803/1416765179]' entity='osd.0' 2024-04-26T21:45:20.857 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:45:20 smithi072 ceph-mon[28138]: from='osd.0 [v2:172.21.15.72:6802/1416765179,v1:172.21.15.72:6803/1416765179]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi072", "root=default"]}]': finished 2024-04-26T21:45:20.857 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:45:20 smithi072 ceph-mon[28138]: osdmap e7: 1 total, 0 up, 1 in 2024-04-26T21:45:20.857 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:45:20 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T21:45:20.857 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:45:20 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T21:45:20.857 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:45:20 smithi072 ceph-mon[28138]: pgmap v62: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:45:20.857 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:45:20 smithi072 ceph-mon[28138]: from='osd.0 [v2:172.21.15.72:6802/1416765179,v1:172.21.15.72:6803/1416765179]' entity='osd.0' 2024-04-26T21:45:20.857 INFO:journalctl@ceph.osd.0.smithi072.stdout:Apr 26 21:45:20 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-0[43226]: 2024-04-26T21:45:20.575+0000 7fa86e600700 -1 osd.0 0 waiting for initial osdmap 2024-04-26T21:45:20.858 INFO:journalctl@ceph.osd.0.smithi072.stdout:Apr 26 21:45:20 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-0[43226]: 2024-04-26T21:45:20.649+0000 7fa864400700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T21:45:20.941 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:45:20 smithi164 ceph-mon[31916]: from='osd.0 [v2:172.21.15.72:6802/1416765179,v1:172.21.15.72:6803/1416765179]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi072", "root=default"]}]': finished 2024-04-26T21:45:20.941 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:45:20 smithi164 ceph-mon[31916]: osdmap e7: 1 total, 0 up, 1 in 2024-04-26T21:45:20.941 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:45:20 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T21:45:20.941 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:45:20 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T21:45:20.941 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:45:20 smithi164 ceph-mon[31916]: pgmap v62: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:45:20.941 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:45:20 smithi164 ceph-mon[31916]: from='osd.0 [v2:172.21.15.72:6802/1416765179,v1:172.21.15.72:6803/1416765179]' entity='osd.0' 2024-04-26T21:45:21.769 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:45:21 smithi076 ceph-mon[31823]: purged_snaps scrub starts 2024-04-26T21:45:21.769 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:45:21 smithi076 ceph-mon[31823]: purged_snaps scrub ok 2024-04-26T21:45:21.769 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:45:21 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T21:45:21.769 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:45:21 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T21:45:21.822 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:45:21 smithi072 ceph-mon[28138]: purged_snaps scrub starts 2024-04-26T21:45:21.822 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:45:21 smithi072 ceph-mon[28138]: purged_snaps scrub ok 2024-04-26T21:45:21.823 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:45:21 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T21:45:21.823 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:45:21 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T21:45:21.940 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:45:21 smithi164 ceph-mon[31916]: purged_snaps scrub starts 2024-04-26T21:45:21.941 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:45:21 smithi164 ceph-mon[31916]: purged_snaps scrub ok 2024-04-26T21:45:21.941 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:45:21 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T21:45:21.941 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:45:21 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T21:45:22.670 INFO:teuthology.orchestra.run.smithi076.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-26T21:45:22.670 INFO:teuthology.orchestra.run.smithi076.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-26T21:45:22.670 INFO:teuthology.orchestra.run.smithi076.stderr: stderr: 10+0 records in 2024-04-26T21:45:22.670 INFO:teuthology.orchestra.run.smithi076.stderr:10+0 records out 2024-04-26T21:45:22.670 INFO:teuthology.orchestra.run.smithi076.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0373225 s, 281 MB/s 2024-04-26T21:45:22.670 INFO:teuthology.orchestra.run.smithi076.stderr:--> Zapping successful for: 2024-04-26T21:45:22.941 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:45:22 smithi164 ceph-mon[31916]: osd.0 [v2:172.21.15.72:6802/1416765179,v1:172.21.15.72:6803/1416765179] boot 2024-04-26T21:45:22.941 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:45:22 smithi164 ceph-mon[31916]: osdmap e8: 1 total, 1 up, 1 in 2024-04-26T21:45:22.941 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:45:22 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T21:45:22.941 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:45:22 smithi164 ceph-mon[31916]: pgmap v64: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:45:22.994 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:45:22 smithi076 ceph-mon[31823]: osd.0 [v2:172.21.15.72:6802/1416765179,v1:172.21.15.72:6803/1416765179] boot 2024-04-26T21:45:22.994 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:45:22 smithi076 ceph-mon[31823]: osdmap e8: 1 total, 1 up, 1 in 2024-04-26T21:45:22.994 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:45:22 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T21:45:22.994 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:45:22 smithi076 ceph-mon[31823]: pgmap v64: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:45:23.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:45:22 smithi072 ceph-mon[28138]: osd.0 [v2:172.21.15.72:6802/1416765179,v1:172.21.15.72:6803/1416765179] boot 2024-04-26T21:45:23.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:45:22 smithi072 ceph-mon[28138]: osdmap e8: 1 total, 1 up, 1 in 2024-04-26T21:45:23.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:45:22 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T21:45:23.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:45:22 smithi072 ceph-mon[28138]: pgmap v64: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:45:23.475 DEBUG:teuthology.orchestra.run.smithi076:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a1780950-0415-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi076:vg_nvme/lv_4 2024-04-26T21:45:23.900 INFO:teuthology.orchestra.run.smithi076.stderr:Inferring config /var/lib/ceph/a1780950-0415-11ef-bc93-c7b262605968/mon.b/config 2024-04-26T21:45:23.940 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:45:23 smithi164 ceph-mon[31916]: osdmap e9: 1 total, 1 up, 1 in 2024-04-26T21:45:23.986 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:45:23 smithi072 ceph-mon[28138]: osdmap e9: 1 total, 1 up, 1 in 2024-04-26T21:45:24.049 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:45:23 smithi076 ceph-mon[31823]: osdmap e9: 1 total, 1 up, 1 in 2024-04-26T21:45:25.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:45:24 smithi076 ceph-mon[31823]: pgmap v66: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:45:25.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:45:24 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:45:25.066 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:45:24 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:45:25.066 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:45:24 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:45:25.066 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:45:24 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:45:25.066 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:45:24 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:45:25.066 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:45:24 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:45:25.066 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:45:24 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:45:25.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:45:24 smithi072 ceph-mon[28138]: pgmap v66: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:45:25.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:45:24 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:45:25.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:45:24 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:45:25.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:45:24 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:45:25.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:45:24 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:45:25.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:45:24 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:45:25.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:45:24 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:45:25.073 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:45:24 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:45:25.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:45:24 smithi164 ceph-mon[31916]: pgmap v66: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:45:25.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:45:24 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:45:25.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:45:24 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:45:25.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:45:24 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:45:25.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:45:24 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:45:25.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:45:24 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:45:25.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:45:24 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:45:25.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:45:24 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:45:25.979 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:45:25.977+0000 7fc434a00700 1 -- 172.21.15.76:0/678865922 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc430077450 msgr2=0x7fc4300758b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:25.980 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:45:25.977+0000 7fc434a00700 1 --2- 172.21.15.76:0/678865922 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc430077450 0x7fc4300758b0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc420009990 tx=0x7fc420009ca0 comp rx=0 tx=0).stop 2024-04-26T21:45:25.980 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:45:25.977+0000 7fc434a00700 1 -- 172.21.15.76:0/678865922 shutdown_connections 2024-04-26T21:45:25.980 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:45:25.978+0000 7fc434a00700 1 --2- 172.21.15.76:0/678865922 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc43010fa70 0x7fc430111e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:25.980 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:45:25.978+0000 7fc434a00700 1 --2- 172.21.15.76:0/678865922 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc430077450 0x7fc4300758b0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:25.980 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:45:25.978+0000 7fc434a00700 1 --2- 172.21.15.76:0/678865922 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc430075e80 0x7fc430076300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:25.980 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:45:25.978+0000 7fc434a00700 1 -- 172.21.15.76:0/678865922 >> 172.21.15.76:0/678865922 conn(0x7fc4300fe9f0 msgr2=0x7fc430100e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:25.980 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:45:25.978+0000 7fc434a00700 1 -- 172.21.15.76:0/678865922 shutdown_connections 2024-04-26T21:45:25.980 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:45:25.978+0000 7fc434a00700 1 -- 172.21.15.76:0/678865922 wait complete. 2024-04-26T21:45:25.980 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:45:25 smithi076 ceph-mon[31823]: Detected new or changed devices on smithi072 2024-04-26T21:45:25.981 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:45:25 smithi076 ceph-mon[31823]: Adjusting osd_memory_target on smithi072 to 17089M 2024-04-26T21:45:25.982 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:45:25.979+0000 7fc434a00700 1 Processor -- start 2024-04-26T21:45:25.982 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:45:25.979+0000 7fc434a00700 1 -- start start 2024-04-26T21:45:25.982 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:45:25.979+0000 7fc434a00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc430075e80 0x7fc43019fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:25.982 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:45:25.979+0000 7fc434a00700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc430077450 0x7fc4301a0070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:25.982 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:45:25.979+0000 7fc434a00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc43010fa70 0x7fc4301a70f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:25.983 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:45:25.979+0000 7fc434a00700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fc430115430 con 0x7fc430075e80 2024-04-26T21:45:25.983 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:45:25.979+0000 7fc434a00700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fc4301152b0 con 0x7fc430077450 2024-04-26T21:45:25.983 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:45:25.979+0000 7fc434a00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fc4301155b0 con 0x7fc43010fa70 2024-04-26T21:45:25.983 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:45:25.980+0000 7fc42de00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc430075e80 0x7fc43019fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:25.983 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:45:25.980+0000 7fc42de00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc430075e80 0x7fc43019fb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.76:46268/0 (socket says 172.21.15.76:46268) 2024-04-26T21:45:25.983 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:45:25.980+0000 7fc42de00700 1 -- 172.21.15.76:0/304005919 learned_addr learned my addr 172.21.15.76:0/304005919 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-26T21:45:25.983 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:45:25.980+0000 7fc425400700 1 --2- 172.21.15.76:0/304005919 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc430077450 0x7fc4301a0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:25.983 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:45:25.981+0000 7fc42e800700 1 --2- 172.21.15.76:0/304005919 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc43010fa70 0x7fc4301a70f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:25.983 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:45:25.981+0000 7fc425400700 1 -- 172.21.15.76:0/304005919 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc43010fa70 msgr2=0x7fc4301a70f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:25.983 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:45:25.981+0000 7fc425400700 1 --2- 172.21.15.76:0/304005919 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc43010fa70 0x7fc4301a70f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:25.983 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:45:25.981+0000 7fc425400700 1 -- 172.21.15.76:0/304005919 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc430075e80 msgr2=0x7fc43019fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:25.983 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:45:25.981+0000 7fc425400700 1 --2- 172.21.15.76:0/304005919 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc430075e80 0x7fc43019fb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:25.983 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:45:25.981+0000 7fc425400700 1 -- 172.21.15.76:0/304005919 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4301a7790 con 0x7fc430077450 2024-04-26T21:45:25.983 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:45:25.981+0000 7fc42e800700 1 --2- 172.21.15.76:0/304005919 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc43010fa70 0x7fc4301a70f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:45:25.983 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:45:25.981+0000 7fc425400700 1 --2- 172.21.15.76:0/304005919 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc430077450 0x7fc4301a0070 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc41800d740 tx=0x7fc41800da50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:25.984 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:45:25.981+0000 7fc427400700 1 -- 172.21.15.76:0/304005919 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc418017070 con 0x7fc430077450 2024-04-26T21:45:25.984 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:45:25.981+0000 7fc427400700 1 -- 172.21.15.76:0/304005919 <== mon.2 v2:172.21.15.76:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc41800f690 con 0x7fc430077450 2024-04-26T21:45:25.984 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:45:25.981+0000 7fc434a00700 1 -- 172.21.15.76:0/304005919 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4301a7a80 con 0x7fc430077450 2024-04-26T21:45:25.984 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:45:25.981+0000 7fc427400700 1 -- 172.21.15.76:0/304005919 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc418003ae0 con 0x7fc430077450 2024-04-26T21:45:25.984 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:45:25.982+0000 7fc434a00700 1 -- 172.21.15.76:0/304005919 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4301a8010 con 0x7fc430077450 2024-04-26T21:45:25.984 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:45:25.983+0000 7fc434a00700 1 -- 172.21.15.76:0/304005919 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc43006a110 con 0x7fc430077450 2024-04-26T21:45:25.986 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:45:25.985+0000 7fc427400700 1 -- 172.21.15.76:0/304005919 <== mon.2 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7fc418003c80 con 0x7fc430077450 2024-04-26T21:45:25.989 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:45:25.987+0000 7fc427400700 1 --2- 172.21.15.76:0/304005919 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fc4140a3c30 0x7fc4140a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:25.989 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:45:25.988+0000 7fc427400700 1 -- 172.21.15.76:0/304005919 <== mon.2 v2:172.21.15.76:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1767+0+0 (secure 0 0 0) 0x7fc4180ce460 con 0x7fc430077450 2024-04-26T21:45:25.989 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:45:25.988+0000 7fc42de00700 1 --2- 172.21.15.76:0/304005919 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fc4140a3c30 0x7fc4140a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:25.990 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:45:25.989+0000 7fc42de00700 1 --2- 172.21.15.76:0/304005919 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fc4140a3c30 0x7fc4140a60f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fc42000b5c0 tx=0x7fc420005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:25.993 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:45:25.991+0000 7fc427400700 1 -- 172.21.15.76:0/304005919 <== mon.2 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fc41809c890 con 0x7fc430077450 2024-04-26T21:45:26.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:45:25 smithi072 ceph-mon[28138]: Detected new or changed devices on smithi072 2024-04-26T21:45:26.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:45:25 smithi072 ceph-mon[28138]: Adjusting osd_memory_target on smithi072 to 17089M 2024-04-26T21:45:26.093 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:45:25 smithi164 ceph-mon[31916]: Detected new or changed devices on smithi072 2024-04-26T21:45:26.094 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:45:25 smithi164 ceph-mon[31916]: Adjusting osd_memory_target on smithi072 to 17089M 2024-04-26T21:45:26.229 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:45:26.227+0000 7fc434a00700 1 -- 172.21.15.76:0/304005919 --> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi076:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fc430063860 con 0x7fc4140a3c30 2024-04-26T21:45:27.064 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:45:26 smithi076 ceph-mon[31823]: pgmap v67: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:45:27.064 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:45:26 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:45:27.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:45:26 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:45:27.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:45:26 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:45:27.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:45:26 smithi072 ceph-mon[28138]: pgmap v67: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:45:27.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:45:26 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:45:27.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:45:26 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:45:27.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:45:26 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:45:27.129 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:45:26 smithi164 ceph-mon[31916]: pgmap v67: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:45:27.129 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:45:26 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:45:27.129 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:45:26 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:45:27.129 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:45:26 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:45:27.814 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:45:27 smithi076 ceph-mon[31823]: from='client.34139 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi076:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:45:28.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:45:27 smithi072 ceph-mon[28138]: from='client.34139 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi076:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:45:28.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:45:27 smithi164 ceph-mon[31916]: from='client.34139 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi076:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:45:28.815 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:45:28 smithi076 ceph-mon[31823]: pgmap v68: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:45:28.941 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:45:28 smithi164 ceph-mon[31916]: pgmap v68: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:45:29.003 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:45:28 smithi072 ceph-mon[28138]: pgmap v68: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:45:30.815 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:45:30 smithi076 ceph-mon[31823]: pgmap v69: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:45:30.815 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:45:30 smithi076 ceph-mon[31823]: from='client.? 172.21.15.76:0/2652363352' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d75a479a-4a2e-48e9-a01e-2b1c9617416a"}]: dispatch 2024-04-26T21:45:30.815 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:45:30 smithi076 ceph-mon[31823]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d75a479a-4a2e-48e9-a01e-2b1c9617416a"}]: dispatch 2024-04-26T21:45:30.815 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:45:30 smithi076 ceph-mon[31823]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d75a479a-4a2e-48e9-a01e-2b1c9617416a"}]': finished 2024-04-26T21:45:30.815 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:45:30 smithi076 ceph-mon[31823]: osdmap e10: 2 total, 1 up, 2 in 2024-04-26T21:45:30.815 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:45:30 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T21:45:31.073 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:45:30 smithi072 ceph-mon[28138]: pgmap v69: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:45:31.073 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:45:30 smithi072 ceph-mon[28138]: from='client.? 172.21.15.76:0/2652363352' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d75a479a-4a2e-48e9-a01e-2b1c9617416a"}]: dispatch 2024-04-26T21:45:31.073 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:45:30 smithi072 ceph-mon[28138]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d75a479a-4a2e-48e9-a01e-2b1c9617416a"}]: dispatch 2024-04-26T21:45:31.073 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:45:30 smithi072 ceph-mon[28138]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d75a479a-4a2e-48e9-a01e-2b1c9617416a"}]': finished 2024-04-26T21:45:31.073 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:45:30 smithi072 ceph-mon[28138]: osdmap e10: 2 total, 1 up, 2 in 2024-04-26T21:45:31.073 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:45:30 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T21:45:31.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:45:30 smithi164 ceph-mon[31916]: pgmap v69: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:45:31.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:45:30 smithi164 ceph-mon[31916]: from='client.? 172.21.15.76:0/2652363352' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d75a479a-4a2e-48e9-a01e-2b1c9617416a"}]: dispatch 2024-04-26T21:45:31.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:45:30 smithi164 ceph-mon[31916]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d75a479a-4a2e-48e9-a01e-2b1c9617416a"}]: dispatch 2024-04-26T21:45:31.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:45:30 smithi164 ceph-mon[31916]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d75a479a-4a2e-48e9-a01e-2b1c9617416a"}]': finished 2024-04-26T21:45:31.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:45:30 smithi164 ceph-mon[31916]: osdmap e10: 2 total, 1 up, 2 in 2024-04-26T21:45:31.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:45:30 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T21:45:32.064 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:45:31 smithi076 ceph-mon[31823]: from='client.? 172.21.15.76:0/1540330945' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:45:32.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:45:31 smithi072 ceph-mon[28138]: from='client.? 172.21.15.76:0/1540330945' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:45:32.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:45:31 smithi164 ceph-mon[31916]: from='client.? 172.21.15.76:0/1540330945' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:45:33.064 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:45:32 smithi076 ceph-mon[31823]: pgmap v71: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:45:33.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:45:32 smithi072 ceph-mon[28138]: pgmap v71: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:45:33.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:45:32 smithi164 ceph-mon[31916]: pgmap v71: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:45:35.064 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:45:34 smithi076 ceph-mon[31823]: pgmap v72: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:45:35.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:45:34 smithi072 ceph-mon[28138]: pgmap v72: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:45:35.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:45:34 smithi164 ceph-mon[31916]: pgmap v72: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:45:37.042 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:45:36 smithi076 ceph-mon[31823]: pgmap v73: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:45:37.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:45:36 smithi072 ceph-mon[28138]: pgmap v73: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:45:37.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:45:36 smithi164 ceph-mon[31916]: pgmap v73: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:45:39.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:45:38 smithi076 ceph-mon[31823]: pgmap v74: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:45:39.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:45:38 smithi072 ceph-mon[28138]: pgmap v74: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:45:39.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:45:38 smithi164 ceph-mon[31916]: pgmap v74: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:45:40.046 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:45:39 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-26T21:45:40.046 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:45:39 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:45:40.064 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:45:39 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-26T21:45:40.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:45:39 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:45:40.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:45:39 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-26T21:45:40.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:45:39 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:45:41.064 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:45:40 smithi076 ceph-mon[31823]: Deploying daemon osd.1 on smithi076 2024-04-26T21:45:41.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:45:40 smithi076 ceph-mon[31823]: pgmap v75: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:45:41.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:45:40 smithi072 ceph-mon[28138]: Deploying daemon osd.1 on smithi076 2024-04-26T21:45:41.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:45:40 smithi072 ceph-mon[28138]: pgmap v75: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:45:41.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:45:40 smithi164 ceph-mon[31916]: Deploying daemon osd.1 on smithi076 2024-04-26T21:45:41.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:45:40 smithi164 ceph-mon[31916]: pgmap v75: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:45:43.064 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:45:42 smithi076 ceph-mon[31823]: pgmap v76: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:45:43.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:45:42 smithi072 ceph-mon[28138]: pgmap v76: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:45:43.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:45:42 smithi164 ceph-mon[31916]: pgmap v76: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:45:44.936 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:45:44 smithi076 ceph-mon[31823]: pgmap v77: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:45:45.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:45:44 smithi072 ceph-mon[28138]: pgmap v77: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:45:45.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:45:44 smithi164 ceph-mon[31916]: pgmap v77: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:45:46.988 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:45:46 smithi076 ceph-mon[31823]: pgmap v78: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:45:47.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:45:46 smithi072 ceph-mon[28138]: pgmap v78: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:45:47.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:45:46 smithi164 ceph-mon[31916]: pgmap v78: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:45:48.719 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:45:48 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:45:48.719 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:45:48 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:45:48.719 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:45:48 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:45:48.719 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:45:48 smithi076 ceph-mon[31823]: pgmap v79: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:45:48.822 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:45:48 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:45:48.822 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:45:48 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:45:48.822 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:45:48 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:45:48.822 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:45:48 smithi072 ceph-mon[28138]: pgmap v79: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:45:48.941 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:45:48 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:45:48.941 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:45:48 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:45:48.941 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:45:48 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:45:48.941 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:45:48 smithi164 ceph-mon[31916]: pgmap v79: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:45:50.815 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:45:50 smithi076 ceph-mon[31823]: pgmap v80: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:45:51.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:45:50 smithi072 ceph-mon[28138]: pgmap v80: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:45:51.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:45:50 smithi164 ceph-mon[31916]: pgmap v80: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:45:52.015 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:45:51 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:45:52.016 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:45:51 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:45:52.016 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:45:51 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:45:52.016 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:45:51 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:45:52.016 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:45:51 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:45:52.016 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:45:51 smithi076 ceph-mon[31823]: from='osd.1 [v2:172.21.15.76:6800/755426949,v1:172.21.15.76:6801/755426949]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-26T21:45:52.016 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:45:51 smithi076 ceph-mon[31823]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-26T21:45:52.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:45:51 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:45:52.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:45:51 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:45:52.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:45:51 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:45:52.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:45:51 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:45:52.073 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:45:51 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:45:52.073 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:45:51 smithi072 ceph-mon[28138]: from='osd.1 [v2:172.21.15.76:6800/755426949,v1:172.21.15.76:6801/755426949]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-26T21:45:52.073 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:45:51 smithi072 ceph-mon[28138]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-26T21:45:52.096 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:45:52.093+0000 7fc427400700 1 -- 172.21.15.76:0/304005919 <== mgr.14152 v2:172.21.15.72:6800/588150172 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fc430063860 con 0x7fc4140a3c30 2024-04-26T21:45:52.096 INFO:teuthology.orchestra.run.smithi076.stdout:Created osd(s) 1 on host 'smithi076' 2024-04-26T21:45:52.098 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:45:52.097+0000 7fc434a00700 1 -- 172.21.15.76:0/304005919 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fc4140a3c30 msgr2=0x7fc4140a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:52.098 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:45:52.097+0000 7fc434a00700 1 --2- 172.21.15.76:0/304005919 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fc4140a3c30 0x7fc4140a60f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fc42000b5c0 tx=0x7fc420005fd0 comp rx=0 tx=0).stop 2024-04-26T21:45:52.098 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:45:52.097+0000 7fc434a00700 1 -- 172.21.15.76:0/304005919 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc430077450 msgr2=0x7fc4301a0070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:52.098 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:45:52.097+0000 7fc434a00700 1 --2- 172.21.15.76:0/304005919 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc430077450 0x7fc4301a0070 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc41800d740 tx=0x7fc41800da50 comp rx=0 tx=0).stop 2024-04-26T21:45:52.098 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:45:52.097+0000 7fc434a00700 1 -- 172.21.15.76:0/304005919 shutdown_connections 2024-04-26T21:45:52.098 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:45:52.097+0000 7fc434a00700 1 --2- 172.21.15.76:0/304005919 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fc4140a3c30 0x7fc4140a60f0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:52.098 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:45:52.097+0000 7fc434a00700 1 --2- 172.21.15.76:0/304005919 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc430075e80 0x7fc43019fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:52.099 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:45:52.097+0000 7fc434a00700 1 --2- 172.21.15.76:0/304005919 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc43010fa70 0x7fc4301a70f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:52.099 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:45:52.097+0000 7fc434a00700 1 --2- 172.21.15.76:0/304005919 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc430077450 0x7fc4301a0070 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:52.099 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:45:52.097+0000 7fc434a00700 1 -- 172.21.15.76:0/304005919 >> 172.21.15.76:0/304005919 conn(0x7fc4300fe9f0 msgr2=0x7fc430102e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:52.099 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:45:52.097+0000 7fc434a00700 1 -- 172.21.15.76:0/304005919 shutdown_connections 2024-04-26T21:45:52.099 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:45:52.098+0000 7fc434a00700 1 -- 172.21.15.76:0/304005919 wait complete. 2024-04-26T21:45:52.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:45:51 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:45:52.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:45:51 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:45:52.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:45:51 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:45:52.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:45:51 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:45:52.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:45:51 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:45:52.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:45:51 smithi164 ceph-mon[31916]: from='osd.1 [v2:172.21.15.76:6800/755426949,v1:172.21.15.76:6801/755426949]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-26T21:45:52.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:45:51 smithi164 ceph-mon[31916]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-26T21:45:52.829 DEBUG:teuthology.orchestra.run.smithi076:osd.1> sudo journalctl -f -n 0 -u ceph-a1780950-0415-11ef-bc93-c7b262605968@osd.1.service 2024-04-26T21:45:52.833 INFO:tasks.cephadm:Deploying osd.2 on smithi164 with /dev/vg_nvme/lv_4... 2024-04-26T21:45:52.833 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a1780950-0415-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-26T21:45:53.054 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:45:52 smithi076 ceph-mon[31823]: pgmap v81: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:45:53.055 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:45:52 smithi076 ceph-mon[31823]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-26T21:45:53.055 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:45:52 smithi076 ceph-mon[31823]: osdmap e11: 2 total, 1 up, 2 in 2024-04-26T21:45:53.055 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:45:52 smithi076 ceph-mon[31823]: from='osd.1 [v2:172.21.15.76:6800/755426949,v1:172.21.15.76:6801/755426949]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi076", "root=default"]}]: dispatch 2024-04-26T21:45:53.055 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:45:52 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T21:45:53.055 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:45:52 smithi076 ceph-mon[31823]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi076", "root=default"]}]: dispatch 2024-04-26T21:45:53.055 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:45:52 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:45:53.055 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:45:52 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:45:53.055 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:45:52 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:45:53.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:45:52 smithi072 ceph-mon[28138]: pgmap v81: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:45:53.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:45:52 smithi072 ceph-mon[28138]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-26T21:45:53.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:45:52 smithi072 ceph-mon[28138]: osdmap e11: 2 total, 1 up, 2 in 2024-04-26T21:45:53.073 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:45:52 smithi072 ceph-mon[28138]: from='osd.1 [v2:172.21.15.76:6800/755426949,v1:172.21.15.76:6801/755426949]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi076", "root=default"]}]: dispatch 2024-04-26T21:45:53.073 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:45:52 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T21:45:53.073 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:45:52 smithi072 ceph-mon[28138]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi076", "root=default"]}]: dispatch 2024-04-26T21:45:53.073 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:45:52 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:45:53.073 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:45:52 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:45:53.073 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:45:52 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:45:53.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:45:52 smithi164 ceph-mon[31916]: pgmap v81: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:45:53.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:45:52 smithi164 ceph-mon[31916]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-26T21:45:53.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:45:52 smithi164 ceph-mon[31916]: osdmap e11: 2 total, 1 up, 2 in 2024-04-26T21:45:53.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:45:52 smithi164 ceph-mon[31916]: from='osd.1 [v2:172.21.15.76:6800/755426949,v1:172.21.15.76:6801/755426949]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi076", "root=default"]}]: dispatch 2024-04-26T21:45:53.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:45:52 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T21:45:53.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:45:52 smithi164 ceph-mon[31916]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi076", "root=default"]}]: dispatch 2024-04-26T21:45:53.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:45:52 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:45:53.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:45:52 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:45:53.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:45:52 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:45:53.263 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/a1780950-0415-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T21:45:54.032 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:45:53 smithi076 ceph-mon[31823]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi076", "root=default"]}]': finished 2024-04-26T21:45:54.033 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:45:53 smithi076 ceph-mon[31823]: osdmap e12: 2 total, 1 up, 2 in 2024-04-26T21:45:54.033 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:45:53 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T21:45:54.033 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:45:53 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T21:45:54.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:45:53 smithi072 ceph-mon[28138]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi076", "root=default"]}]': finished 2024-04-26T21:45:54.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:45:53 smithi072 ceph-mon[28138]: osdmap e12: 2 total, 1 up, 2 in 2024-04-26T21:45:54.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:45:53 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T21:45:54.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:45:53 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T21:45:54.175 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:45:53 smithi164 ceph-mon[31916]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi076", "root=default"]}]': finished 2024-04-26T21:45:54.175 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:45:53 smithi164 ceph-mon[31916]: osdmap e12: 2 total, 1 up, 2 in 2024-04-26T21:45:54.175 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:45:53 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T21:45:54.175 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:45:53 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T21:45:55.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:45:54 smithi076 ceph-mon[31823]: purged_snaps scrub starts 2024-04-26T21:45:55.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:45:54 smithi076 ceph-mon[31823]: purged_snaps scrub ok 2024-04-26T21:45:55.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:45:54 smithi076 ceph-mon[31823]: pgmap v84: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:45:55.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:45:54 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T21:45:55.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:45:54 smithi076 ceph-mon[31823]: osd.1 [v2:172.21.15.76:6800/755426949,v1:172.21.15.76:6801/755426949] boot 2024-04-26T21:45:55.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:45:54 smithi076 ceph-mon[31823]: osdmap e13: 2 total, 2 up, 2 in 2024-04-26T21:45:55.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:45:54 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T21:45:55.179 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:45:54 smithi164 ceph-mon[31916]: purged_snaps scrub starts 2024-04-26T21:45:55.179 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:45:54 smithi164 ceph-mon[31916]: purged_snaps scrub ok 2024-04-26T21:45:55.179 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:45:54 smithi164 ceph-mon[31916]: pgmap v84: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:45:55.179 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:45:54 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T21:45:55.180 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:45:54 smithi164 ceph-mon[31916]: osd.1 [v2:172.21.15.76:6800/755426949,v1:172.21.15.76:6801/755426949] boot 2024-04-26T21:45:55.180 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:45:54 smithi164 ceph-mon[31916]: osdmap e13: 2 total, 2 up, 2 in 2024-04-26T21:45:55.180 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:45:54 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T21:45:55.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:45:54 smithi072 ceph-mon[28138]: purged_snaps scrub starts 2024-04-26T21:45:55.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:45:54 smithi072 ceph-mon[28138]: purged_snaps scrub ok 2024-04-26T21:45:55.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:45:54 smithi072 ceph-mon[28138]: pgmap v84: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:45:55.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:45:54 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T21:45:55.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:45:54 smithi072 ceph-mon[28138]: osd.1 [v2:172.21.15.76:6800/755426949,v1:172.21.15.76:6801/755426949] boot 2024-04-26T21:45:55.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:45:54 smithi072 ceph-mon[28138]: osdmap e13: 2 total, 2 up, 2 in 2024-04-26T21:45:55.323 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:45:54 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T21:45:56.091 INFO:teuthology.orchestra.run.smithi164.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-26T21:45:56.091 INFO:teuthology.orchestra.run.smithi164.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-26T21:45:56.092 INFO:teuthology.orchestra.run.smithi164.stderr: stderr: 10+0 records in 2024-04-26T21:45:56.092 INFO:teuthology.orchestra.run.smithi164.stderr:10+0 records out 2024-04-26T21:45:56.092 INFO:teuthology.orchestra.run.smithi164.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0354506 s, 296 MB/s 2024-04-26T21:45:56.092 INFO:teuthology.orchestra.run.smithi164.stderr:--> Zapping successful for: 2024-04-26T21:45:56.137 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:45:55 smithi164 ceph-mon[31916]: osdmap e14: 2 total, 2 up, 2 in 2024-04-26T21:45:56.215 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:45:55 smithi076 ceph-mon[31823]: osdmap e14: 2 total, 2 up, 2 in 2024-04-26T21:45:56.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:45:55 smithi072 ceph-mon[28138]: osdmap e14: 2 total, 2 up, 2 in 2024-04-26T21:45:56.917 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a1780950-0415-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi164:vg_nvme/lv_4 2024-04-26T21:45:57.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:45:56 smithi164 ceph-mon[31916]: pgmap v87: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:45:57.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:45:56 smithi076 ceph-mon[31823]: pgmap v87: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:45:57.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:45:56 smithi072 ceph-mon[28138]: pgmap v87: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:45:57.362 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/a1780950-0415-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T21:45:58.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:45:58 smithi076 ceph-mon[31823]: Detected new or changed devices on smithi076 2024-04-26T21:45:58.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:45:58 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:45:58.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:45:58 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:45:58.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:45:58 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:45:58.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:45:58 smithi076 ceph-mon[31823]: Adjusting osd_memory_target on smithi076 to 17089M 2024-04-26T21:45:58.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:45:58 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:45:58.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:45:58 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:45:58.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:45:58 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:45:58.316 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:45:58 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:45:58.316 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:45:58 smithi076 ceph-mon[31823]: pgmap v88: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:45:58.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:45:58 smithi072 ceph-mon[28138]: Detected new or changed devices on smithi076 2024-04-26T21:45:58.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:45:58 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:45:58.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:45:58 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:45:58.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:45:58 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:45:58.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:45:58 smithi072 ceph-mon[28138]: Adjusting osd_memory_target on smithi076 to 17089M 2024-04-26T21:45:58.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:45:58 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:45:58.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:45:58 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:45:58.323 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:45:58 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:45:58.323 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:45:58 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:45:58.323 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:45:58 smithi072 ceph-mon[28138]: pgmap v88: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:45:58.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:45:58 smithi164 ceph-mon[31916]: Detected new or changed devices on smithi076 2024-04-26T21:45:58.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:45:58 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:45:58.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:45:58 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:45:58.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:45:58 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:45:58.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:45:58 smithi164 ceph-mon[31916]: Adjusting osd_memory_target on smithi076 to 17089M 2024-04-26T21:45:58.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:45:58 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:45:58.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:45:58 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:45:58.442 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:45:58 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:45:58.442 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:45:58 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:45:58.442 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:45:58 smithi164 ceph-mon[31916]: pgmap v88: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:45:59.380 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:45:59.378+0000 7f1ea7400700 1 -- 172.21.15.164:0/180093008 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1ea0107260 msgr2=0x7f1ea010db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:59.380 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:45:59.378+0000 7f1ea7400700 1 --2- 172.21.15.164:0/180093008 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1ea0107260 0x7f1ea010db10 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f1e9800b0a0 tx=0x7f1e9800b3b0 comp rx=0 tx=0).stop 2024-04-26T21:45:59.380 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:45:59.378+0000 7f1ea7400700 1 -- 172.21.15.164:0/180093008 shutdown_connections 2024-04-26T21:45:59.380 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:45:59.378+0000 7f1ea7400700 1 --2- 172.21.15.164:0/180093008 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1ea0107260 0x7f1ea010db10 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:59.380 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:45:59.378+0000 7f1ea7400700 1 --2- 172.21.15.164:0/180093008 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1ea01056c0 0x7f1ea0105ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:59.381 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:45:59.378+0000 7f1ea7400700 1 --2- 172.21.15.164:0/180093008 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1ea01068c0 0x7f1ea0106d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:59.381 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:45:59.378+0000 7f1ea7400700 1 -- 172.21.15.164:0/180093008 >> 172.21.15.164:0/180093008 conn(0x7f1ea0100c80 msgr2=0x7f1ea01030a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:59.381 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:45:59.378+0000 7f1ea7400700 1 -- 172.21.15.164:0/180093008 shutdown_connections 2024-04-26T21:45:59.381 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:45:59.378+0000 7f1ea7400700 1 -- 172.21.15.164:0/180093008 wait complete. 2024-04-26T21:45:59.383 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:45:59.379+0000 7f1ea7400700 1 Processor -- start 2024-04-26T21:45:59.383 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:45:59.379+0000 7f1ea7400700 1 -- start start 2024-04-26T21:45:59.383 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:45:59.380+0000 7f1ea7400700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1ea01056c0 0x7f1ea019b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:59.383 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:45:59.380+0000 7f1ea7400700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1ea01068c0 0x7f1ea019bb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:59.383 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:45:59.380+0000 7f1ea7400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1ea0107260 0x7f1ea01a2b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:59.383 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:45:59.380+0000 7f1ea7400700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f1ea0111040 con 0x7f1ea01056c0 2024-04-26T21:45:59.383 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:45:59.380+0000 7f1ea7400700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f1ea0110ec0 con 0x7f1ea01068c0 2024-04-26T21:45:59.383 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:45:59.380+0000 7f1ea7400700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f1ea01111c0 con 0x7f1ea0107260 2024-04-26T21:45:59.384 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:45:59.380+0000 7f1ea5400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1ea0107260 0x7f1ea01a2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:59.384 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:45:59.380+0000 7f1ea5400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1ea0107260 0x7f1ea01a2b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:59612/0 (socket says 172.21.15.164:59612) 2024-04-26T21:45:59.384 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:45:59.380+0000 7f1ea5400700 1 -- 172.21.15.164:0/1244053114 learned_addr learned my addr 172.21.15.164:0/1244053114 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-04-26T21:45:59.384 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:45:59.380+0000 7f1e9fe00700 1 --2- 172.21.15.164:0/1244053114 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1ea01068c0 0x7f1ea019bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:59.384 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:45:59.380+0000 7f1ea4a00700 1 --2- 172.21.15.164:0/1244053114 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1ea01056c0 0x7f1ea019b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:59.384 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:45:59.380+0000 7f1ea5400700 1 -- 172.21.15.164:0/1244053114 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1ea01068c0 msgr2=0x7f1ea019bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:59.384 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:45:59.380+0000 7f1ea5400700 1 --2- 172.21.15.164:0/1244053114 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1ea01068c0 0x7f1ea019bb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:59.384 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:45:59.380+0000 7f1ea5400700 1 -- 172.21.15.164:0/1244053114 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1ea01056c0 msgr2=0x7f1ea019b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:59.384 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:45:59.380+0000 7f1ea5400700 1 --2- 172.21.15.164:0/1244053114 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1ea01056c0 0x7f1ea019b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:59.384 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:45:59.380+0000 7f1ea5400700 1 -- 172.21.15.164:0/1244053114 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ea01a3220 con 0x7f1ea0107260 2024-04-26T21:45:59.384 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:45:59.381+0000 7f1ea5400700 1 --2- 172.21.15.164:0/1244053114 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1ea0107260 0x7f1ea01a2b80 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f1e9800b070 tx=0x7f1e98007dc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:59.384 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:45:59.381+0000 7f1e9d600700 1 -- 172.21.15.164:0/1244053114 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e98003e10 con 0x7f1ea0107260 2024-04-26T21:45:59.384 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:45:59.381+0000 7f1e9d600700 1 -- 172.21.15.164:0/1244053114 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1e98005500 con 0x7f1ea0107260 2024-04-26T21:45:59.385 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:45:59.381+0000 7f1e9d600700 1 -- 172.21.15.164:0/1244053114 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e9801bad0 con 0x7f1ea0107260 2024-04-26T21:45:59.385 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:45:59.381+0000 7f1ea7400700 1 -- 172.21.15.164:0/1244053114 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ea01a34b0 con 0x7f1ea0107260 2024-04-26T21:45:59.385 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:45:59.381+0000 7f1ea7400700 1 -- 172.21.15.164:0/1244053114 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ea01a3960 con 0x7f1ea0107260 2024-04-26T21:45:59.385 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:45:59.383+0000 7f1ea7400700 1 -- 172.21.15.164:0/1244053114 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ea006a110 con 0x7f1ea0107260 2024-04-26T21:45:59.385 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:45:59.385+0000 7f1e9d600700 1 -- 172.21.15.164:0/1244053114 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7f1e980050d0 con 0x7f1ea0107260 2024-04-26T21:45:59.387 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:45:59.386+0000 7f1e9d600700 1 --2- 172.21.15.164:0/1244053114 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f1e880a3c30 0x7f1e880a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:59.387 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:45:59.386+0000 7f1e9d600700 1 -- 172.21.15.164:0/1244053114 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2309+0+0 (secure 0 0 0) 0x7f1e9800f020 con 0x7f1ea0107260 2024-04-26T21:45:59.387 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:45:59.387+0000 7f1ea4a00700 1 --2- 172.21.15.164:0/1244053114 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f1e880a3c30 0x7f1e880a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:59.389 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:45:59.388+0000 7f1ea4a00700 1 --2- 172.21.15.164:0/1244053114 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f1e880a3c30 0x7f1e880a60f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f1e94005fd0 tx=0x7f1e94005e50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:59.390 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:45:59.389+0000 7f1e9d600700 1 -- 172.21.15.164:0/1244053114 <== mon.1 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f1e9809ea00 con 0x7f1ea0107260 2024-04-26T21:45:59.628 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:45:59.626+0000 7f1ea7400700 1 -- 172.21.15.164:0/1244053114 --> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi164:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f1ea0063860 con 0x7f1e880a3c30 2024-04-26T21:45:59.940 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:45:59 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:45:59.941 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:45:59 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:45:59.941 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:45:59 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:46:00.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:45:59 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:46:00.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:45:59 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:46:00.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:45:59 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:46:00.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:45:59 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:46:00.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:45:59 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:46:00.073 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:45:59 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:46:00.846 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:46:00 smithi164 ceph-mon[31916]: from='client.24154 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi164:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:46:00.846 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:46:00 smithi164 ceph-mon[31916]: pgmap v89: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:46:01.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:46:00 smithi076 ceph-mon[31823]: from='client.24154 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi164:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:46:01.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:46:00 smithi076 ceph-mon[31823]: pgmap v89: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:46:01.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:46:00 smithi072 ceph-mon[28138]: from='client.24154 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi164:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:46:01.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:46:00 smithi072 ceph-mon[28138]: pgmap v89: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:46:03.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:46:02 smithi076 ceph-mon[31823]: pgmap v90: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:46:03.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:46:02 smithi072 ceph-mon[28138]: pgmap v90: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:46:03.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:46:02 smithi164 ceph-mon[31916]: pgmap v90: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:46:03.975 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:46:03 smithi164 ceph-mon[31916]: from='client.? 172.21.15.164:0/374646981' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b6f8da58-4428-4e8c-a944-7ea5645d6d92"}]: dispatch 2024-04-26T21:46:03.975 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:46:03 smithi164 ceph-mon[31916]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b6f8da58-4428-4e8c-a944-7ea5645d6d92"}]: dispatch 2024-04-26T21:46:03.975 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:46:03 smithi164 ceph-mon[31916]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b6f8da58-4428-4e8c-a944-7ea5645d6d92"}]': finished 2024-04-26T21:46:03.975 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:46:03 smithi164 ceph-mon[31916]: osdmap e15: 3 total, 2 up, 3 in 2024-04-26T21:46:03.975 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:46:03 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T21:46:03.983 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:46:03 smithi076 ceph-mon[31823]: from='client.? 172.21.15.164:0/374646981' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b6f8da58-4428-4e8c-a944-7ea5645d6d92"}]: dispatch 2024-04-26T21:46:03.983 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:46:03 smithi076 ceph-mon[31823]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b6f8da58-4428-4e8c-a944-7ea5645d6d92"}]: dispatch 2024-04-26T21:46:03.983 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:46:03 smithi076 ceph-mon[31823]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b6f8da58-4428-4e8c-a944-7ea5645d6d92"}]': finished 2024-04-26T21:46:03.983 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:46:03 smithi076 ceph-mon[31823]: osdmap e15: 3 total, 2 up, 3 in 2024-04-26T21:46:03.983 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:46:03 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T21:46:04.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:46:03 smithi072 ceph-mon[28138]: from='client.? 172.21.15.164:0/374646981' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b6f8da58-4428-4e8c-a944-7ea5645d6d92"}]: dispatch 2024-04-26T21:46:04.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:46:03 smithi072 ceph-mon[28138]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b6f8da58-4428-4e8c-a944-7ea5645d6d92"}]: dispatch 2024-04-26T21:46:04.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:46:03 smithi072 ceph-mon[28138]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b6f8da58-4428-4e8c-a944-7ea5645d6d92"}]': finished 2024-04-26T21:46:04.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:46:03 smithi072 ceph-mon[28138]: osdmap e15: 3 total, 2 up, 3 in 2024-04-26T21:46:04.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:46:03 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T21:46:05.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:46:04 smithi076 ceph-mon[31823]: pgmap v92: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:46:05.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:46:04 smithi076 ceph-mon[31823]: from='client.? 172.21.15.164:0/1815615950' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:46:05.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:46:04 smithi072 ceph-mon[28138]: pgmap v92: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:46:05.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:46:04 smithi072 ceph-mon[28138]: from='client.? 172.21.15.164:0/1815615950' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:46:05.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:46:04 smithi164 ceph-mon[31916]: pgmap v92: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:46:05.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:46:04 smithi164 ceph-mon[31916]: from='client.? 172.21.15.164:0/1815615950' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:46:07.064 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:46:06 smithi076 ceph-mon[31823]: pgmap v93: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:46:07.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:46:06 smithi072 ceph-mon[28138]: pgmap v93: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:46:07.183 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:46:06 smithi164 ceph-mon[31916]: pgmap v93: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:46:08.940 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:46:08 smithi164 ceph-mon[31916]: pgmap v94: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:46:09.064 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:46:08 smithi076 ceph-mon[31823]: pgmap v94: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:46:09.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:46:08 smithi072 ceph-mon[28138]: pgmap v94: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:46:11.052 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:46:10 smithi164 ceph-mon[31916]: pgmap v95: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:46:11.064 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:46:10 smithi076 ceph-mon[31823]: pgmap v95: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:46:11.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:46:10 smithi072 ceph-mon[28138]: pgmap v95: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:46:12.940 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:46:12 smithi164 ceph-mon[31916]: pgmap v96: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:46:12.941 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:46:12 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-26T21:46:12.941 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:46:12 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:46:13.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:46:12 smithi076 ceph-mon[31823]: pgmap v96: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:46:13.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:46:12 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-26T21:46:13.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:46:12 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:46:13.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:46:12 smithi072 ceph-mon[28138]: pgmap v96: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:46:13.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:46:12 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-26T21:46:13.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:46:12 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:46:13.940 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:46:13 smithi164 ceph-mon[31916]: Deploying daemon osd.2 on smithi164 2024-04-26T21:46:14.038 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:46:13 smithi076 ceph-mon[31823]: Deploying daemon osd.2 on smithi164 2024-04-26T21:46:14.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:46:13 smithi072 ceph-mon[28138]: Deploying daemon osd.2 on smithi164 2024-04-26T21:46:14.941 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:46:14 smithi164 ceph-mon[31916]: pgmap v97: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:46:15.064 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:46:14 smithi076 ceph-mon[31823]: pgmap v97: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:46:15.071 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:46:14 smithi072 ceph-mon[28138]: pgmap v97: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:46:17.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:46:16 smithi076 ceph-mon[31823]: pgmap v98: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:46:17.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:46:16 smithi072 ceph-mon[28138]: pgmap v98: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:46:17.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:46:16 smithi164 ceph-mon[31916]: pgmap v98: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:46:19.064 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:46:18 smithi076 ceph-mon[31823]: pgmap v99: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:46:19.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:46:18 smithi072 ceph-mon[28138]: pgmap v99: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:46:19.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:46:18 smithi164 ceph-mon[31916]: pgmap v99: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:46:20.940 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:46:20 smithi164 ceph-mon[31916]: pgmap v100: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:46:21.064 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:46:20 smithi076 ceph-mon[31823]: pgmap v100: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:46:21.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:46:20 smithi072 ceph-mon[28138]: pgmap v100: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:46:22.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:46:21 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:46:22.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:46:21 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:46:22.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:46:21 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:46:22.094 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:46:21 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:46:22.094 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:46:21 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:46:22.094 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:46:21 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:46:22.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:46:21 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:46:22.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:46:21 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:46:22.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:46:21 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:46:23.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:46:22 smithi164 ceph-mon[31916]: pgmap v101: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:46:23.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:46:22 smithi076 ceph-mon[31823]: pgmap v101: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:46:23.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:46:22 smithi072 ceph-mon[28138]: pgmap v101: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:46:25.013 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:46:24 smithi164 ceph-mon[31916]: pgmap v102: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:46:25.014 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:46:24 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:46:25.014 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:46:24 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:46:25.064 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:46:24 smithi076 ceph-mon[31823]: pgmap v102: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:46:25.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:46:24 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:46:25.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:46:24 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:46:25.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:46:24 smithi072 ceph-mon[28138]: pgmap v102: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:46:25.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:46:24 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:46:25.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:46:24 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:46:25.709 INFO:teuthology.orchestra.run.smithi164.stdout:Created osd(s) 2 on host 'smithi164' 2024-04-26T21:46:25.713 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:46:25.708+0000 7f1e9d600700 1 -- 172.21.15.164:0/1244053114 <== mgr.14152 v2:172.21.15.72:6800/588150172 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f1ea0063860 con 0x7f1e880a3c30 2024-04-26T21:46:25.713 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:46:25.712+0000 7f1ea7400700 1 -- 172.21.15.164:0/1244053114 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f1e880a3c30 msgr2=0x7f1e880a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:25.713 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:46:25.712+0000 7f1ea7400700 1 --2- 172.21.15.164:0/1244053114 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f1e880a3c30 0x7f1e880a60f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f1e94005fd0 tx=0x7f1e94005e50 comp rx=0 tx=0).stop 2024-04-26T21:46:25.714 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:46:25.712+0000 7f1ea7400700 1 -- 172.21.15.164:0/1244053114 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1ea0107260 msgr2=0x7f1ea01a2b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:25.714 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:46:25.712+0000 7f1ea7400700 1 --2- 172.21.15.164:0/1244053114 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1ea0107260 0x7f1ea01a2b80 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f1e9800b070 tx=0x7f1e98007dc0 comp rx=0 tx=0).stop 2024-04-26T21:46:25.714 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:46:25.712+0000 7f1ea7400700 1 -- 172.21.15.164:0/1244053114 shutdown_connections 2024-04-26T21:46:25.714 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:46:25.712+0000 7f1ea7400700 1 --2- 172.21.15.164:0/1244053114 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f1e880a3c30 0x7f1e880a60f0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:25.714 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:46:25.712+0000 7f1ea7400700 1 --2- 172.21.15.164:0/1244053114 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1ea01056c0 0x7f1ea019b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:25.714 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:46:25.712+0000 7f1ea7400700 1 --2- 172.21.15.164:0/1244053114 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1ea0107260 0x7f1ea01a2b80 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:25.714 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:46:25.712+0000 7f1ea7400700 1 --2- 172.21.15.164:0/1244053114 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1ea01068c0 0x7f1ea019bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:25.714 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:46:25.712+0000 7f1ea7400700 1 -- 172.21.15.164:0/1244053114 >> 172.21.15.164:0/1244053114 conn(0x7f1ea0100c80 msgr2=0x7f1ea0109af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:25.714 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:46:25.713+0000 7f1ea7400700 1 -- 172.21.15.164:0/1244053114 shutdown_connections 2024-04-26T21:46:25.714 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:46:25.713+0000 7f1ea7400700 1 -- 172.21.15.164:0/1244053114 wait complete. 2024-04-26T21:46:25.941 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:46:25 smithi164 ceph-mon[31916]: from='osd.2 [v2:172.21.15.164:6800/852265973,v1:172.21.15.164:6801/852265973]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-26T21:46:25.941 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:46:25 smithi164 ceph-mon[31916]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-26T21:46:25.941 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:46:25 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:46:25.941 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:46:25 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:46:25.941 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:46:25 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:46:25.941 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:46:25 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:46:25.941 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:46:25 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:46:25.941 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:46:25 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:46:26.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:46:25 smithi076 ceph-mon[31823]: from='osd.2 [v2:172.21.15.164:6800/852265973,v1:172.21.15.164:6801/852265973]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-26T21:46:26.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:46:25 smithi076 ceph-mon[31823]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-26T21:46:26.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:46:25 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:46:26.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:46:25 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:46:26.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:46:25 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:46:26.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:46:25 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:46:26.066 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:46:25 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:46:26.066 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:46:25 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:46:26.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:46:25 smithi072 ceph-mon[28138]: from='osd.2 [v2:172.21.15.164:6800/852265973,v1:172.21.15.164:6801/852265973]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-26T21:46:26.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:46:25 smithi072 ceph-mon[28138]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-26T21:46:26.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:46:25 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:46:26.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:46:25 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:46:26.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:46:25 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:46:26.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:46:25 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:46:26.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:46:25 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:46:26.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:46:25 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:46:26.344 DEBUG:teuthology.orchestra.run.smithi164:osd.2> sudo journalctl -f -n 0 -u ceph-a1780950-0415-11ef-bc93-c7b262605968@osd.2.service 2024-04-26T21:46:26.348 INFO:tasks.cephadm:Deploying osd.3 on smithi072 with /dev/vg_nvme/lv_3... 2024-04-26T21:46:26.348 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a1780950-0415-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-26T21:46:26.784 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/a1780950-0415-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:46:26.863 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:46:26 smithi164 ceph-mon[31916]: pgmap v103: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:46:26.863 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:46:26 smithi164 ceph-mon[31916]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-26T21:46:26.863 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:46:26 smithi164 ceph-mon[31916]: from='osd.2 [v2:172.21.15.164:6800/852265973,v1:172.21.15.164:6801/852265973]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-04-26T21:46:26.863 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:46:26 smithi164 ceph-mon[31916]: osdmap e16: 3 total, 2 up, 3 in 2024-04-26T21:46:26.863 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:46:26 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T21:46:26.863 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:46:26 smithi164 ceph-mon[31916]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-04-26T21:46:27.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:46:26 smithi076 ceph-mon[31823]: pgmap v103: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:46:27.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:46:26 smithi076 ceph-mon[31823]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-26T21:46:27.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:46:26 smithi076 ceph-mon[31823]: from='osd.2 [v2:172.21.15.164:6800/852265973,v1:172.21.15.164:6801/852265973]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-04-26T21:46:27.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:46:26 smithi076 ceph-mon[31823]: osdmap e16: 3 total, 2 up, 3 in 2024-04-26T21:46:27.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:46:26 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T21:46:27.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:46:26 smithi076 ceph-mon[31823]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-04-26T21:46:27.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:46:26 smithi072 ceph-mon[28138]: pgmap v103: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:46:27.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:46:26 smithi072 ceph-mon[28138]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-26T21:46:27.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:46:26 smithi072 ceph-mon[28138]: from='osd.2 [v2:172.21.15.164:6800/852265973,v1:172.21.15.164:6801/852265973]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-04-26T21:46:27.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:46:26 smithi072 ceph-mon[28138]: osdmap e16: 3 total, 2 up, 3 in 2024-04-26T21:46:27.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:46:26 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T21:46:27.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:46:26 smithi072 ceph-mon[28138]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-04-26T21:46:27.941 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:46:27 smithi164 ceph-mon[31916]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]': finished 2024-04-26T21:46:27.941 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:46:27 smithi164 ceph-mon[31916]: osdmap e17: 3 total, 2 up, 3 in 2024-04-26T21:46:27.941 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:46:27 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T21:46:27.941 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:46:27 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T21:46:27.941 INFO:journalctl@ceph.osd.2.smithi164.stdout:Apr 26 21:46:27 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-2[36918]: 2024-04-26T21:46:27.779+0000 7feebfc00700 -1 osd.2 0 waiting for initial osdmap 2024-04-26T21:46:27.941 INFO:journalctl@ceph.osd.2.smithi164.stdout:Apr 26 21:46:27 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-2[36918]: 2024-04-26T21:46:27.815+0000 7feeb5000700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T21:46:28.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:46:27 smithi076 ceph-mon[31823]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]': finished 2024-04-26T21:46:28.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:46:27 smithi076 ceph-mon[31823]: osdmap e17: 3 total, 2 up, 3 in 2024-04-26T21:46:28.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:46:27 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T21:46:28.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:46:27 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T21:46:28.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:46:27 smithi072 ceph-mon[28138]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]': finished 2024-04-26T21:46:28.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:46:27 smithi072 ceph-mon[28138]: osdmap e17: 3 total, 2 up, 3 in 2024-04-26T21:46:28.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:46:27 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T21:46:28.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:46:27 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T21:46:29.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:46:28 smithi076 ceph-mon[31823]: purged_snaps scrub starts 2024-04-26T21:46:29.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:46:28 smithi076 ceph-mon[31823]: purged_snaps scrub ok 2024-04-26T21:46:29.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:46:28 smithi076 ceph-mon[31823]: pgmap v106: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:46:29.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:46:28 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T21:46:29.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:46:28 smithi076 ceph-mon[31823]: from='osd.2 ' entity='osd.2' 2024-04-26T21:46:29.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:46:28 smithi072 ceph-mon[28138]: purged_snaps scrub starts 2024-04-26T21:46:29.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:46:28 smithi072 ceph-mon[28138]: purged_snaps scrub ok 2024-04-26T21:46:29.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:46:28 smithi072 ceph-mon[28138]: pgmap v106: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:46:29.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:46:28 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T21:46:29.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:46:28 smithi072 ceph-mon[28138]: from='osd.2 ' entity='osd.2' 2024-04-26T21:46:29.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:46:28 smithi164 ceph-mon[31916]: purged_snaps scrub starts 2024-04-26T21:46:29.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:46:28 smithi164 ceph-mon[31916]: purged_snaps scrub ok 2024-04-26T21:46:29.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:46:28 smithi164 ceph-mon[31916]: pgmap v106: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:46:29.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:46:28 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T21:46:29.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:46:28 smithi164 ceph-mon[31916]: from='osd.2 ' entity='osd.2' 2024-04-26T21:46:29.738 INFO:teuthology.orchestra.run.smithi072.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-26T21:46:29.738 INFO:teuthology.orchestra.run.smithi072.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-26T21:46:29.739 INFO:teuthology.orchestra.run.smithi072.stderr: stderr: 10+0 records in 2024-04-26T21:46:29.739 INFO:teuthology.orchestra.run.smithi072.stderr:10+0 records out 2024-04-26T21:46:29.739 INFO:teuthology.orchestra.run.smithi072.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0398106 s, 263 MB/s 2024-04-26T21:46:29.739 INFO:teuthology.orchestra.run.smithi072.stderr:--> Zapping successful for: 2024-04-26T21:46:30.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:46:29 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T21:46:30.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:46:29 smithi076 ceph-mon[31823]: osd.2 [v2:172.21.15.164:6800/852265973,v1:172.21.15.164:6801/852265973] boot 2024-04-26T21:46:30.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:46:29 smithi076 ceph-mon[31823]: osdmap e18: 3 total, 3 up, 3 in 2024-04-26T21:46:30.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:46:29 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T21:46:30.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:46:29 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T21:46:30.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:46:29 smithi072 ceph-mon[28138]: osd.2 [v2:172.21.15.164:6800/852265973,v1:172.21.15.164:6801/852265973] boot 2024-04-26T21:46:30.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:46:29 smithi072 ceph-mon[28138]: osdmap e18: 3 total, 3 up, 3 in 2024-04-26T21:46:30.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:46:29 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T21:46:30.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:46:29 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T21:46:30.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:46:29 smithi164 ceph-mon[31916]: osd.2 [v2:172.21.15.164:6800/852265973,v1:172.21.15.164:6801/852265973] boot 2024-04-26T21:46:30.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:46:29 smithi164 ceph-mon[31916]: osdmap e18: 3 total, 3 up, 3 in 2024-04-26T21:46:30.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:46:29 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T21:46:30.664 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a1780950-0415-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi072:vg_nvme/lv_3 2024-04-26T21:46:31.064 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:46:30 smithi076 ceph-mon[31823]: pgmap v108: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:46:31.064 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:46:30 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-26T21:46:31.064 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:46:30 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-26T21:46:31.064 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:46:30 smithi076 ceph-mon[31823]: osdmap e19: 3 total, 3 up, 3 in 2024-04-26T21:46:31.064 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:46:30 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-26T21:46:31.071 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:46:30 smithi072 ceph-mon[28138]: pgmap v108: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:46:31.071 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:46:30 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-26T21:46:31.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:46:30 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-26T21:46:31.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:46:30 smithi072 ceph-mon[28138]: osdmap e19: 3 total, 3 up, 3 in 2024-04-26T21:46:31.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:46:30 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-26T21:46:31.111 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/a1780950-0415-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:46:31.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:46:30 smithi164 ceph-mon[31916]: pgmap v108: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:46:31.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:46:30 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-26T21:46:31.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:46:30 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-26T21:46:31.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:46:30 smithi164 ceph-mon[31916]: osdmap e19: 3 total, 3 up, 3 in 2024-04-26T21:46:31.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:46:30 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-26T21:46:31.425 INFO:journalctl@ceph.osd.0.smithi072.stdout:Apr 26 21:46:31 smithi072 sudo[47646]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-26T21:46:31.426 INFO:journalctl@ceph.osd.0.smithi072.stdout:Apr 26 21:46:31 smithi072 sudo[47646]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T21:46:31.426 INFO:journalctl@ceph.osd.0.smithi072.stdout:Apr 26 21:46:31 smithi072 sudo[47646]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-26T21:46:31.426 INFO:journalctl@ceph.osd.0.smithi072.stdout:Apr 26 21:46:31 smithi072 sudo[47646]: pam_unix(sudo:session): session closed for user root 2024-04-26T21:46:31.426 INFO:journalctl@ceph.osd.0.smithi072.stdout:Apr 26 21:46:31 smithi072 sudo[47661]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-26T21:46:31.426 INFO:journalctl@ceph.osd.0.smithi072.stdout:Apr 26 21:46:31 smithi072 sudo[47661]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T21:46:31.426 INFO:journalctl@ceph.osd.0.smithi072.stdout:Apr 26 21:46:31 smithi072 sudo[47661]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-26T21:46:31.426 INFO:journalctl@ceph.osd.0.smithi072.stdout:Apr 26 21:46:31 smithi072 sudo[47661]: pam_unix(sudo:session): session closed for user root 2024-04-26T21:46:31.566 INFO:journalctl@ceph.osd.1.smithi076.stdout:Apr 26 21:46:31 smithi076 sudo[40178]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-26T21:46:31.566 INFO:journalctl@ceph.osd.1.smithi076.stdout:Apr 26 21:46:31 smithi076 sudo[40178]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T21:46:31.566 INFO:journalctl@ceph.osd.1.smithi076.stdout:Apr 26 21:46:31 smithi076 sudo[40178]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-26T21:46:31.566 INFO:journalctl@ceph.osd.1.smithi076.stdout:Apr 26 21:46:31 smithi076 sudo[40178]: pam_unix(sudo:session): session closed for user root 2024-04-26T21:46:31.566 INFO:journalctl@ceph.osd.1.smithi076.stdout:Apr 26 21:46:31 smithi076 sudo[40181]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-26T21:46:31.566 INFO:journalctl@ceph.osd.1.smithi076.stdout:Apr 26 21:46:31 smithi076 sudo[40181]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T21:46:31.566 INFO:journalctl@ceph.osd.1.smithi076.stdout:Apr 26 21:46:31 smithi076 sudo[40181]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-26T21:46:31.566 INFO:journalctl@ceph.osd.1.smithi076.stdout:Apr 26 21:46:31 smithi076 sudo[40181]: pam_unix(sudo:session): session closed for user root 2024-04-26T21:46:31.709 INFO:journalctl@ceph.osd.2.smithi164.stdout:Apr 26 21:46:31 smithi164 sudo[40644]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-26T21:46:31.709 INFO:journalctl@ceph.osd.2.smithi164.stdout:Apr 26 21:46:31 smithi164 sudo[40644]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T21:46:31.709 INFO:journalctl@ceph.osd.2.smithi164.stdout:Apr 26 21:46:31 smithi164 sudo[40644]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-26T21:46:31.709 INFO:journalctl@ceph.osd.2.smithi164.stdout:Apr 26 21:46:31 smithi164 sudo[40644]: pam_unix(sudo:session): session closed for user root 2024-04-26T21:46:31.709 INFO:journalctl@ceph.osd.2.smithi164.stdout:Apr 26 21:46:31 smithi164 sudo[40647]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-26T21:46:31.709 INFO:journalctl@ceph.osd.2.smithi164.stdout:Apr 26 21:46:31 smithi164 sudo[40647]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T21:46:31.709 INFO:journalctl@ceph.osd.2.smithi164.stdout:Apr 26 21:46:31 smithi164 sudo[40647]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-26T21:46:31.709 INFO:journalctl@ceph.osd.2.smithi164.stdout:Apr 26 21:46:31 smithi164 sudo[40647]: pam_unix(sudo:session): session closed for user root 2024-04-26T21:46:31.717 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:46:31 smithi072 sudo[47682]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-26T21:46:31.717 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:46:31 smithi072 sudo[47682]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T21:46:31.718 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:46:31 smithi072 sudo[47682]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-26T21:46:31.718 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:46:31 smithi072 sudo[47682]: pam_unix(sudo:session): session closed for user root 2024-04-26T21:46:31.718 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:46:31 smithi072 sudo[47693]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-26T21:46:31.718 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:46:31 smithi072 sudo[47693]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T21:46:31.718 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:46:31 smithi072 sudo[47693]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-26T21:46:31.718 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:46:31 smithi072 sudo[47693]: pam_unix(sudo:session): session closed for user root 2024-04-26T21:46:32.031 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:46:31 smithi072 ceph-mon[28138]: Detected new or changed devices on smithi164 2024-04-26T21:46:32.031 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:46:31 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:46:32.031 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:46:31 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:46:32.031 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:46:31 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:46:32.031 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:46:31 smithi072 ceph-mon[28138]: Adjusting osd_memory_target on smithi164 to 17089M 2024-04-26T21:46:32.031 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:46:31 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-26T21:46:32.031 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:46:31 smithi072 ceph-mon[28138]: osdmap e20: 3 total, 3 up, 3 in 2024-04-26T21:46:32.032 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:46:31 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:46:32.032 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:46:31 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:46:32.032 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:46:31 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:46:32.032 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:46:31 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:46:32.032 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:46:31 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:46:32.032 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:46:31 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:46:32.032 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:46:31 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:46:32.032 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:46:31 smithi072 ceph-mon[28138]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-26T21:46:32.032 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:46:31 smithi072 ceph-mon[28138]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-26T21:46:32.032 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:46:31 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T21:46:32.032 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:46:31 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T21:46:32.032 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:46:31 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T21:46:32.032 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:46:31 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T21:46:32.032 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:46:31 smithi072 ceph-mon[28138]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-26T21:46:32.032 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:46:31 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T21:46:32.033 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:46:31 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T21:46:32.033 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:46:31 smithi072 ceph-mon[28138]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-26T21:46:32.033 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:46:31 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T21:46:32.033 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:46:31 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T21:46:32.033 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:46:31 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T21:46:32.066 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:46:31 smithi076 sudo[40184]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-26T21:46:32.066 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:46:31 smithi076 sudo[40184]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T21:46:32.067 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:46:31 smithi076 sudo[40184]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-26T21:46:32.067 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:46:31 smithi076 sudo[40184]: pam_unix(sudo:session): session closed for user root 2024-04-26T21:46:32.067 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:46:31 smithi076 sudo[40187]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-26T21:46:32.067 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:46:31 smithi076 sudo[40187]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T21:46:32.067 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:46:31 smithi076 sudo[40187]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-26T21:46:32.067 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:46:31 smithi076 sudo[40187]: pam_unix(sudo:session): session closed for user root 2024-04-26T21:46:32.067 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:46:31 smithi076 ceph-mon[31823]: Detected new or changed devices on smithi164 2024-04-26T21:46:32.067 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:46:31 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:46:32.067 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:46:31 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:46:32.067 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:46:31 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:46:32.067 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:46:31 smithi076 ceph-mon[31823]: Adjusting osd_memory_target on smithi164 to 17089M 2024-04-26T21:46:32.067 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:46:31 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-26T21:46:32.067 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:46:31 smithi076 ceph-mon[31823]: osdmap e20: 3 total, 3 up, 3 in 2024-04-26T21:46:32.067 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:46:31 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:46:32.068 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:46:31 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:46:32.068 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:46:31 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:46:32.068 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:46:31 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:46:32.068 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:46:31 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:46:32.068 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:46:31 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:46:32.068 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:46:31 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:46:32.068 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:46:31 smithi076 ceph-mon[31823]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-26T21:46:32.068 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:46:31 smithi076 ceph-mon[31823]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-26T21:46:32.068 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:46:31 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T21:46:32.068 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:46:31 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T21:46:32.068 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:46:31 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T21:46:32.068 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:46:31 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T21:46:32.068 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:46:31 smithi076 ceph-mon[31823]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-26T21:46:32.068 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:46:31 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T21:46:32.068 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:46:31 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T21:46:32.069 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:46:31 smithi076 ceph-mon[31823]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-26T21:46:32.069 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:46:31 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T21:46:32.069 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:46:31 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T21:46:32.069 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:46:31 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T21:46:32.192 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:46:31 smithi164 sudo[40650]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-26T21:46:32.192 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:46:31 smithi164 sudo[40650]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T21:46:32.192 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:46:31 smithi164 sudo[40650]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-26T21:46:32.192 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:46:31 smithi164 sudo[40650]: pam_unix(sudo:session): session closed for user root 2024-04-26T21:46:32.192 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:46:31 smithi164 sudo[40653]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-26T21:46:32.192 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:46:31 smithi164 sudo[40653]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T21:46:32.192 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:46:31 smithi164 sudo[40653]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-26T21:46:32.192 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:46:31 smithi164 sudo[40653]: pam_unix(sudo:session): session closed for user root 2024-04-26T21:46:32.192 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:46:31 smithi164 ceph-mon[31916]: Detected new or changed devices on smithi164 2024-04-26T21:46:32.193 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:46:31 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:46:32.193 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:46:31 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:46:32.193 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:46:31 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:46:32.193 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:46:31 smithi164 ceph-mon[31916]: Adjusting osd_memory_target on smithi164 to 17089M 2024-04-26T21:46:32.193 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:46:31 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-26T21:46:32.193 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:46:31 smithi164 ceph-mon[31916]: osdmap e20: 3 total, 3 up, 3 in 2024-04-26T21:46:32.193 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:46:31 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:46:32.193 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:46:31 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:46:32.193 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:46:31 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:46:32.193 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:46:31 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:46:32.193 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:46:31 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:46:32.193 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:46:31 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:46:32.193 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:46:31 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:46:32.193 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:46:31 smithi164 ceph-mon[31916]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-26T21:46:32.193 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:46:31 smithi164 ceph-mon[31916]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-26T21:46:32.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:46:31 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T21:46:32.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:46:31 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T21:46:32.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:46:31 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T21:46:32.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:46:31 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T21:46:32.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:46:31 smithi164 ceph-mon[31916]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-26T21:46:32.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:46:31 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T21:46:32.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:46:31 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T21:46:32.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:46:31 smithi164 ceph-mon[31916]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-26T21:46:32.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:46:31 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T21:46:32.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:46:31 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T21:46:32.194 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:46:31 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T21:46:32.938 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:46:32 smithi072 ceph-mon[28138]: pgmap v111: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:46:32.938 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:46:32 smithi072 ceph-mon[28138]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-26T21:46:32.938 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:46:32 smithi072 ceph-mon[28138]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-26T21:46:32.938 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:46:32 smithi072 ceph-mon[28138]: osdmap e21: 3 total, 3 up, 3 in 2024-04-26T21:46:32.939 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:46:32 smithi072 ceph-mon[28138]: mgrmap e16: x(active, since 3m), standbys: z, y 2024-04-26T21:46:33.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:46:32 smithi164 ceph-mon[31916]: pgmap v111: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:46:33.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:46:32 smithi164 ceph-mon[31916]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-26T21:46:33.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:46:32 smithi164 ceph-mon[31916]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-26T21:46:33.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:46:32 smithi164 ceph-mon[31916]: osdmap e21: 3 total, 3 up, 3 in 2024-04-26T21:46:33.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:46:32 smithi164 ceph-mon[31916]: mgrmap e16: x(active, since 3m), standbys: z, y 2024-04-26T21:46:33.211 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:46:33.210+0000 7f2134800700 1 -- 172.21.15.72:0/3934830550 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f212c107270 msgr2=0x7f212c10db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:33.211 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:46:33.210+0000 7f2134800700 1 --2- 172.21.15.72:0/3934830550 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f212c107270 0x7f212c10db20 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f212800b0d0 tx=0x7f212800b3e0 comp rx=0 tx=0).stop 2024-04-26T21:46:33.211 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:46:33.210+0000 7f2134800700 1 -- 172.21.15.72:0/3934830550 shutdown_connections 2024-04-26T21:46:33.211 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:46:33.210+0000 7f2134800700 1 --2- 172.21.15.72:0/3934830550 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f212c107270 0x7f212c10db20 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:33.212 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:46:33.210+0000 7f2134800700 1 --2- 172.21.15.72:0/3934830550 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f212c1056d0 0x7f212c105af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:33.212 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:46:33.210+0000 7f2134800700 1 --2- 172.21.15.72:0/3934830550 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f212c1068d0 0x7f212c106d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:33.212 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:46:33.210+0000 7f2134800700 1 -- 172.21.15.72:0/3934830550 >> 172.21.15.72:0/3934830550 conn(0x7f212c100c70 msgr2=0x7f212c1030b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:33.212 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:46:33.210+0000 7f2134800700 1 -- 172.21.15.72:0/3934830550 shutdown_connections 2024-04-26T21:46:33.212 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:46:33.210+0000 7f2134800700 1 -- 172.21.15.72:0/3934830550 wait complete. 2024-04-26T21:46:33.212 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:46:33.212+0000 7f2134800700 1 Processor -- start 2024-04-26T21:46:33.212 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:46:33.212+0000 7f2134800700 1 -- start start 2024-04-26T21:46:33.213 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:46:33.212+0000 7f2134800700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f212c1056d0 0x7f212c19b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:33.214 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:46:33.212+0000 7f2134800700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f212c1068d0 0x7f212c19bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:33.214 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:46:33.212+0000 7f2134800700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f212c107270 0x7f212c1a2b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:33.214 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:46:33.212+0000 7f2134800700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f212c111050 con 0x7f212c107270 2024-04-26T21:46:33.214 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:46:33.212+0000 7f2134800700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f212c110ed0 con 0x7f212c1068d0 2024-04-26T21:46:33.214 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:46:33.212+0000 7f2134800700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f212c1111d0 con 0x7f212c1056d0 2024-04-26T21:46:33.214 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:46:33.213+0000 7f2132800700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f212c107270 0x7f212c1a2b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:33.214 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:46:33.213+0000 7f2132800700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f212c107270 0x7f212c1a2b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:59702/0 (socket says 172.21.15.72:59702) 2024-04-26T21:46:33.214 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:46:33.213+0000 7f2132800700 1 -- 172.21.15.72:0/2537316292 learned_addr learned my addr 172.21.15.72:0/2537316292 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:46:33.214 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:46:33.213+0000 7f2131e00700 1 --2- 172.21.15.72:0/2537316292 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f212c1056d0 0x7f212c19b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:33.214 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:46:33.213+0000 7f2131400700 1 --2- 172.21.15.72:0/2537316292 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f212c1068d0 0x7f212c19bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:33.214 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:46:33.213+0000 7f2132800700 1 -- 172.21.15.72:0/2537316292 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f212c1056d0 msgr2=0x7f212c19b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:33.214 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:46:33.213+0000 7f2132800700 1 --2- 172.21.15.72:0/2537316292 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f212c1056d0 0x7f212c19b5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:33.215 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:46:33.213+0000 7f2132800700 1 -- 172.21.15.72:0/2537316292 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f212c1068d0 msgr2=0x7f212c19bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:33.215 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:46:33.213+0000 7f2132800700 1 --2- 172.21.15.72:0/2537316292 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f212c1068d0 0x7f212c19bae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:33.215 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:46:33.213+0000 7f2132800700 1 -- 172.21.15.72:0/2537316292 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f212c1a3200 con 0x7f212c107270 2024-04-26T21:46:33.216 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:46:33.214+0000 7f2132800700 1 --2- 172.21.15.72:0/2537316292 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f212c107270 0x7f212c1a2b60 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f2128009b80 tx=0x7f2128009c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:33.216 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:46:33.214+0000 7f2126a00700 1 -- 172.21.15.72:0/2537316292 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f212801f070 con 0x7f212c107270 2024-04-26T21:46:33.216 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:46:33.214+0000 7f2126a00700 1 -- 172.21.15.72:0/2537316292 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f21280054d0 con 0x7f212c107270 2024-04-26T21:46:33.216 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:46:33.214+0000 7f2134800700 1 -- 172.21.15.72:0/2537316292 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f212c1a3490 con 0x7f212c107270 2024-04-26T21:46:33.216 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:46:33.214+0000 7f2126a00700 1 -- 172.21.15.72:0/2537316292 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f212801b830 con 0x7f212c107270 2024-04-26T21:46:33.216 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:46:33.214+0000 7f2134800700 1 -- 172.21.15.72:0/2537316292 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f212c1a3890 con 0x7f212c107270 2024-04-26T21:46:33.216 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:46:33.215+0000 7f2134800700 1 -- 172.21.15.72:0/2537316292 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f212c06a110 con 0x7f212c107270 2024-04-26T21:46:33.217 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:46:33.217+0000 7f2126a00700 1 -- 172.21.15.72:0/2537316292 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f2128022070 con 0x7f212c107270 2024-04-26T21:46:33.219 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:46:33.218+0000 7f2126a00700 1 --2- 172.21.15.72:0/2537316292 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f21140a3c40 0x7f21140a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:33.219 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:46:33.219+0000 7f2126a00700 1 -- 172.21.15.72:0/2537316292 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 3237+0+0 (secure 0 0 0) 0x7f21280cffa0 con 0x7f212c107270 2024-04-26T21:46:33.219 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:46:33.219+0000 7f2131e00700 1 --2- 172.21.15.72:0/2537316292 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f21140a3c40 0x7f21140a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:33.220 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:46:33.219+0000 7f2131e00700 1 --2- 172.21.15.72:0/2537316292 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f21140a3c40 0x7f21140a6100 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f2120005fd0 tx=0x7f2120005e50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:33.224 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:46:33.224+0000 7f2126a00700 1 -- 172.21.15.72:0/2537316292 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f212809dde0 con 0x7f212c107270 2024-04-26T21:46:33.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:46:32 smithi076 ceph-mon[31823]: pgmap v111: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:46:33.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:46:32 smithi076 ceph-mon[31823]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-26T21:46:33.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:46:32 smithi076 ceph-mon[31823]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-26T21:46:33.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:46:32 smithi076 ceph-mon[31823]: osdmap e21: 3 total, 3 up, 3 in 2024-04-26T21:46:33.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:46:32 smithi076 ceph-mon[31823]: mgrmap e16: x(active, since 3m), standbys: z, y 2024-04-26T21:46:33.463 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:46:33.462+0000 7f2134800700 1 -- 172.21.15.72:0/2537316292 --> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi072:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f212c063860 con 0x7f21140a3c40 2024-04-26T21:46:34.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:46:33 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:46:34.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:46:33 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:46:34.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:46:33 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:46:34.216 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:46:33 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:46:34.217 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:46:33 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:46:34.217 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:46:33 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:46:34.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:46:33 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:46:34.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:46:33 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:46:34.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:46:33 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:46:35.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:46:34 smithi072 ceph-mon[28138]: from='client.14301 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi072:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:46:35.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:46:34 smithi072 ceph-mon[28138]: pgmap v113: 1 pgs: 1 creating+peering; 0 B data, 89 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:46:35.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:46:34 smithi164 ceph-mon[31916]: from='client.14301 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi072:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:46:35.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:46:34 smithi164 ceph-mon[31916]: pgmap v113: 1 pgs: 1 creating+peering; 0 B data, 89 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:46:35.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:46:34 smithi076 ceph-mon[31823]: from='client.14301 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi072:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:46:35.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:46:34 smithi076 ceph-mon[31823]: pgmap v113: 1 pgs: 1 creating+peering; 0 B data, 89 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:46:37.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:46:36 smithi164 ceph-mon[31916]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:46:37.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:46:36 smithi076 ceph-mon[31823]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:46:37.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:46:36 smithi072 ceph-mon[28138]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:46:38.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:46:37 smithi164 ceph-mon[31916]: from='client.? 172.21.15.72:0/4125150424' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "76d83409-649e-41e5-b4f4-43bbd5b00ea5"}]: dispatch 2024-04-26T21:46:38.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:46:37 smithi164 ceph-mon[31916]: from='client.? 172.21.15.72:0/4125150424' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "76d83409-649e-41e5-b4f4-43bbd5b00ea5"}]': finished 2024-04-26T21:46:38.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:46:37 smithi164 ceph-mon[31916]: osdmap e22: 4 total, 3 up, 4 in 2024-04-26T21:46:38.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:46:37 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T21:46:38.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:46:37 smithi164 ceph-mon[31916]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:46:38.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:46:37 smithi076 ceph-mon[31823]: from='client.? 172.21.15.72:0/4125150424' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "76d83409-649e-41e5-b4f4-43bbd5b00ea5"}]: dispatch 2024-04-26T21:46:38.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:46:37 smithi076 ceph-mon[31823]: from='client.? 172.21.15.72:0/4125150424' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "76d83409-649e-41e5-b4f4-43bbd5b00ea5"}]': finished 2024-04-26T21:46:38.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:46:37 smithi076 ceph-mon[31823]: osdmap e22: 4 total, 3 up, 4 in 2024-04-26T21:46:38.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:46:37 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T21:46:38.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:46:37 smithi076 ceph-mon[31823]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:46:38.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:46:37 smithi072 ceph-mon[28138]: from='client.? 172.21.15.72:0/4125150424' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "76d83409-649e-41e5-b4f4-43bbd5b00ea5"}]: dispatch 2024-04-26T21:46:38.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:46:37 smithi072 ceph-mon[28138]: from='client.? 172.21.15.72:0/4125150424' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "76d83409-649e-41e5-b4f4-43bbd5b00ea5"}]': finished 2024-04-26T21:46:38.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:46:37 smithi072 ceph-mon[28138]: osdmap e22: 4 total, 3 up, 4 in 2024-04-26T21:46:38.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:46:37 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T21:46:38.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:46:37 smithi072 ceph-mon[28138]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:46:39.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:46:38 smithi164 ceph-mon[31916]: from='client.? 172.21.15.72:0/659276468' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:46:39.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:46:38 smithi076 ceph-mon[31823]: from='client.? 172.21.15.72:0/659276468' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:46:39.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:46:38 smithi072 ceph-mon[28138]: from='client.? 172.21.15.72:0/659276468' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:46:40.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:46:39 smithi164 ceph-mon[31916]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:46:40.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:46:39 smithi076 ceph-mon[31823]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:46:40.321 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:46:39 smithi072 ceph-mon[28138]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:46:43.064 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:46:42 smithi076 ceph-mon[31823]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:46:43.071 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:46:42 smithi072 ceph-mon[28138]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:46:43.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:46:42 smithi164 ceph-mon[31916]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:46:44.989 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:46:44 smithi072 ceph-mon[28138]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:46:45.064 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:46:44 smithi076 ceph-mon[31823]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:46:45.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:46:44 smithi164 ceph-mon[31916]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:46:46.821 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:46:46 smithi072 ceph-mon[28138]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:46:46.822 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:46:46 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-26T21:46:46.822 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:46:46 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:46:47.064 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:46:46 smithi076 ceph-mon[31823]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:46:47.064 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:46:46 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-26T21:46:47.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:46:46 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:46:47.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:46:46 smithi164 ceph-mon[31916]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:46:47.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:46:46 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-26T21:46:47.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:46:46 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:46:47.960 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:46:47 smithi072 ceph-mon[28138]: Deploying daemon osd.3 on smithi072 2024-04-26T21:46:48.064 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:46:47 smithi076 ceph-mon[31823]: Deploying daemon osd.3 on smithi072 2024-04-26T21:46:48.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:46:47 smithi164 ceph-mon[31916]: Deploying daemon osd.3 on smithi072 2024-04-26T21:46:48.781 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:46:48 smithi072 ceph-mon[28138]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:46:49.064 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:46:48 smithi076 ceph-mon[31823]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:46:49.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:46:48 smithi164 ceph-mon[31916]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:46:51.064 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:46:50 smithi076 ceph-mon[31823]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:46:51.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:46:50 smithi072 ceph-mon[28138]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:46:51.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:46:50 smithi164 ceph-mon[31916]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:46:53.064 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:46:52 smithi076 ceph-mon[31823]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:46:53.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:46:52 smithi072 ceph-mon[28138]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:46:53.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:46:52 smithi164 ceph-mon[31916]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:46:54.822 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:46:54 smithi072 ceph-mon[28138]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:46:55.064 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:46:54 smithi076 ceph-mon[31823]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:46:55.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:46:54 smithi164 ceph-mon[31916]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:46:56.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:46:56 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:46:56.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:46:56 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:46:56.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:46:56 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:46:56.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:46:56 smithi072 ceph-mon[28138]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:46:56.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:46:56 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:46:56.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:46:56 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:46:56.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:46:56 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:46:56.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:46:56 smithi164 ceph-mon[31916]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:46:56.565 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:46:56 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:46:56.565 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:46:56 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:46:56.565 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:46:56 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:46:56.565 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:46:56 smithi076 ceph-mon[31823]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:46:58.988 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:46:58 smithi072 ceph-mon[28138]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:46:58.988 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:46:58 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:46:59.035 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:46:58 smithi164 ceph-mon[31916]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:46:59.035 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:46:58 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:46:59.064 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:46:58 smithi076 ceph-mon[31823]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:46:59.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:46:58 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:47:00.064 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:46:59 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:47:00.064 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:46:59 smithi076 ceph-mon[31823]: from='osd.3 [v2:172.21.15.72:6810/4027763789,v1:172.21.15.72:6811/4027763789]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-26T21:47:00.071 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:46:59 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:47:00.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:46:59 smithi072 ceph-mon[28138]: from='osd.3 [v2:172.21.15.72:6810/4027763789,v1:172.21.15.72:6811/4027763789]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-26T21:47:00.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:46:59 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:47:00.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:46:59 smithi164 ceph-mon[31916]: from='osd.3 [v2:172.21.15.72:6810/4027763789,v1:172.21.15.72:6811/4027763789]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-26T21:47:00.404 INFO:teuthology.orchestra.run.smithi072.stdout:Created osd(s) 3 on host 'smithi072' 2024-04-26T21:47:00.404 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:47:00.402+0000 7f2126a00700 1 -- 172.21.15.72:0/2537316292 <== mgr.14152 v2:172.21.15.72:6800/588150172 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f212c063860 con 0x7f21140a3c40 2024-04-26T21:47:00.410 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:47:00.409+0000 7f2134800700 1 -- 172.21.15.72:0/2537316292 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f21140a3c40 msgr2=0x7f21140a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:00.410 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:47:00.409+0000 7f2134800700 1 --2- 172.21.15.72:0/2537316292 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f21140a3c40 0x7f21140a6100 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f2120005fd0 tx=0x7f2120005e50 comp rx=0 tx=0).stop 2024-04-26T21:47:00.410 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:47:00.409+0000 7f2134800700 1 -- 172.21.15.72:0/2537316292 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f212c107270 msgr2=0x7f212c1a2b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:00.410 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:47:00.409+0000 7f2134800700 1 --2- 172.21.15.72:0/2537316292 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f212c107270 0x7f212c1a2b60 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f2128009b80 tx=0x7f2128009c60 comp rx=0 tx=0).stop 2024-04-26T21:47:00.410 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:47:00.409+0000 7f2134800700 1 -- 172.21.15.72:0/2537316292 shutdown_connections 2024-04-26T21:47:00.411 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:47:00.409+0000 7f2134800700 1 --2- 172.21.15.72:0/2537316292 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f21140a3c40 0x7f21140a6100 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:00.411 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:47:00.409+0000 7f2134800700 1 --2- 172.21.15.72:0/2537316292 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f212c107270 0x7f212c1a2b60 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:00.411 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:47:00.409+0000 7f2134800700 1 --2- 172.21.15.72:0/2537316292 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f212c1056d0 0x7f212c19b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:00.411 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:47:00.409+0000 7f2134800700 1 --2- 172.21.15.72:0/2537316292 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f212c1068d0 0x7f212c19bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:00.411 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:47:00.409+0000 7f2134800700 1 -- 172.21.15.72:0/2537316292 >> 172.21.15.72:0/2537316292 conn(0x7f212c100c70 msgr2=0x7f212c109b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:00.411 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:47:00.410+0000 7f2134800700 1 -- 172.21.15.72:0/2537316292 shutdown_connections 2024-04-26T21:47:00.411 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:47:00.410+0000 7f2134800700 1 -- 172.21.15.72:0/2537316292 wait complete. 2024-04-26T21:47:01.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:47:00 smithi076 ceph-mon[31823]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:47:01.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:47:00 smithi076 ceph-mon[31823]: from='osd.3 [v2:172.21.15.72:6810/4027763789,v1:172.21.15.72:6811/4027763789]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-26T21:47:01.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:47:00 smithi076 ceph-mon[31823]: osdmap e23: 4 total, 3 up, 4 in 2024-04-26T21:47:01.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:47:00 smithi076 ceph-mon[31823]: from='osd.3 [v2:172.21.15.72:6810/4027763789,v1:172.21.15.72:6811/4027763789]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi072", "root=default"]}]: dispatch 2024-04-26T21:47:01.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:47:00 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T21:47:01.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:47:00 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:47:01.066 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:47:00 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:47:01.066 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:47:00 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:47:01.066 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:47:00 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:47:01.066 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:47:00 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:47:01.066 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:47:00 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:47:01.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:47:00 smithi072 ceph-mon[28138]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:47:01.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:47:00 smithi072 ceph-mon[28138]: from='osd.3 [v2:172.21.15.72:6810/4027763789,v1:172.21.15.72:6811/4027763789]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-26T21:47:01.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:47:00 smithi072 ceph-mon[28138]: osdmap e23: 4 total, 3 up, 4 in 2024-04-26T21:47:01.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:47:00 smithi072 ceph-mon[28138]: from='osd.3 [v2:172.21.15.72:6810/4027763789,v1:172.21.15.72:6811/4027763789]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi072", "root=default"]}]: dispatch 2024-04-26T21:47:01.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:47:00 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T21:47:01.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:47:00 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:47:01.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:47:00 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:47:01.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:47:00 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:47:01.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:47:00 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:47:01.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:47:00 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:47:01.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:47:00 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:47:01.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:47:00 smithi164 ceph-mon[31916]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:47:01.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:47:00 smithi164 ceph-mon[31916]: from='osd.3 [v2:172.21.15.72:6810/4027763789,v1:172.21.15.72:6811/4027763789]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-26T21:47:01.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:47:00 smithi164 ceph-mon[31916]: osdmap e23: 4 total, 3 up, 4 in 2024-04-26T21:47:01.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:47:00 smithi164 ceph-mon[31916]: from='osd.3 [v2:172.21.15.72:6810/4027763789,v1:172.21.15.72:6811/4027763789]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi072", "root=default"]}]: dispatch 2024-04-26T21:47:01.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:47:00 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T21:47:01.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:47:00 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:47:01.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:47:00 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:47:01.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:47:00 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:47:01.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:47:00 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:47:01.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:47:00 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:47:01.192 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:47:00 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:47:01.223 DEBUG:teuthology.orchestra.run.smithi072:osd.3> sudo journalctl -f -n 0 -u ceph-a1780950-0415-11ef-bc93-c7b262605968@osd.3.service 2024-04-26T21:47:01.227 INFO:tasks.cephadm:Deploying osd.4 on smithi076 with /dev/vg_nvme/lv_3... 2024-04-26T21:47:01.227 DEBUG:teuthology.orchestra.run.smithi076:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a1780950-0415-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-26T21:47:01.675 INFO:teuthology.orchestra.run.smithi076.stderr:Inferring config /var/lib/ceph/a1780950-0415-11ef-bc93-c7b262605968/mon.b/config 2024-04-26T21:47:02.037 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:47:01 smithi076 ceph-mon[31823]: from='osd.3 [v2:172.21.15.72:6810/4027763789,v1:172.21.15.72:6811/4027763789]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi072", "root=default"]}]': finished 2024-04-26T21:47:02.037 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:47:01 smithi076 ceph-mon[31823]: osdmap e24: 4 total, 3 up, 4 in 2024-04-26T21:47:02.037 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:47:01 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T21:47:02.037 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:47:01 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T21:47:02.037 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:47:01 smithi076 ceph-mon[31823]: osd.3 [v2:172.21.15.72:6810/4027763789,v1:172.21.15.72:6811/4027763789] boot 2024-04-26T21:47:02.037 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:47:01 smithi076 ceph-mon[31823]: osdmap e25: 4 total, 4 up, 4 in 2024-04-26T21:47:02.038 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:47:01 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T21:47:02.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:47:01 smithi072 ceph-mon[28138]: from='osd.3 [v2:172.21.15.72:6810/4027763789,v1:172.21.15.72:6811/4027763789]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi072", "root=default"]}]': finished 2024-04-26T21:47:02.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:47:01 smithi072 ceph-mon[28138]: osdmap e24: 4 total, 3 up, 4 in 2024-04-26T21:47:02.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:47:01 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T21:47:02.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:47:01 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T21:47:02.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:47:01 smithi072 ceph-mon[28138]: osd.3 [v2:172.21.15.72:6810/4027763789,v1:172.21.15.72:6811/4027763789] boot 2024-04-26T21:47:02.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:47:01 smithi072 ceph-mon[28138]: osdmap e25: 4 total, 4 up, 4 in 2024-04-26T21:47:02.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:47:01 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T21:47:02.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:47:01 smithi164 ceph-mon[31916]: from='osd.3 [v2:172.21.15.72:6810/4027763789,v1:172.21.15.72:6811/4027763789]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi072", "root=default"]}]': finished 2024-04-26T21:47:02.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:47:01 smithi164 ceph-mon[31916]: osdmap e24: 4 total, 3 up, 4 in 2024-04-26T21:47:02.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:47:01 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T21:47:02.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:47:01 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T21:47:02.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:47:01 smithi164 ceph-mon[31916]: osd.3 [v2:172.21.15.72:6810/4027763789,v1:172.21.15.72:6811/4027763789] boot 2024-04-26T21:47:02.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:47:01 smithi164 ceph-mon[31916]: osdmap e25: 4 total, 4 up, 4 in 2024-04-26T21:47:02.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:47:01 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T21:47:02.944 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:47:02 smithi072 ceph-mon[28138]: purged_snaps scrub starts 2024-04-26T21:47:02.944 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:47:02 smithi072 ceph-mon[28138]: purged_snaps scrub ok 2024-04-26T21:47:02.944 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:47:02 smithi072 ceph-mon[28138]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:47:02.945 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:47:02 smithi072 ceph-mon[28138]: osdmap e26: 4 total, 4 up, 4 in 2024-04-26T21:47:03.050 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:47:02 smithi076 ceph-mon[31823]: purged_snaps scrub starts 2024-04-26T21:47:03.050 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:47:02 smithi076 ceph-mon[31823]: purged_snaps scrub ok 2024-04-26T21:47:03.050 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:47:02 smithi076 ceph-mon[31823]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:47:03.050 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:47:02 smithi076 ceph-mon[31823]: osdmap e26: 4 total, 4 up, 4 in 2024-04-26T21:47:03.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:47:02 smithi164 ceph-mon[31916]: purged_snaps scrub starts 2024-04-26T21:47:03.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:47:02 smithi164 ceph-mon[31916]: purged_snaps scrub ok 2024-04-26T21:47:03.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:47:02 smithi164 ceph-mon[31916]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:47:03.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:47:02 smithi164 ceph-mon[31916]: osdmap e26: 4 total, 4 up, 4 in 2024-04-26T21:47:04.583 INFO:teuthology.orchestra.run.smithi076.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-26T21:47:04.584 INFO:teuthology.orchestra.run.smithi076.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-26T21:47:04.584 INFO:teuthology.orchestra.run.smithi076.stderr: stderr: 10+0 records in 2024-04-26T21:47:04.584 INFO:teuthology.orchestra.run.smithi076.stderr:10+0 records out 2024-04-26T21:47:04.584 INFO:teuthology.orchestra.run.smithi076.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0369513 s, 284 MB/s 2024-04-26T21:47:04.584 INFO:teuthology.orchestra.run.smithi076.stderr:--> Zapping successful for: 2024-04-26T21:47:04.946 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:47:04 smithi076 ceph-mon[31823]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-04-26T21:47:04.946 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:47:04 smithi076 ceph-mon[31823]: osdmap e27: 4 total, 4 up, 4 in 2024-04-26T21:47:05.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:47:04 smithi072 ceph-mon[28138]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-04-26T21:47:05.073 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:47:04 smithi072 ceph-mon[28138]: osdmap e27: 4 total, 4 up, 4 in 2024-04-26T21:47:05.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:47:04 smithi164 ceph-mon[31916]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-04-26T21:47:05.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:47:04 smithi164 ceph-mon[31916]: osdmap e27: 4 total, 4 up, 4 in 2024-04-26T21:47:05.440 DEBUG:teuthology.orchestra.run.smithi076:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a1780950-0415-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi076:vg_nvme/lv_3 2024-04-26T21:47:05.840 INFO:teuthology.orchestra.run.smithi076.stderr:Inferring config /var/lib/ceph/a1780950-0415-11ef-bc93-c7b262605968/mon.b/config 2024-04-26T21:47:06.064 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:47:05 smithi076 ceph-mon[31823]: osdmap e28: 4 total, 4 up, 4 in 2024-04-26T21:47:06.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:47:05 smithi072 ceph-mon[28138]: osdmap e28: 4 total, 4 up, 4 in 2024-04-26T21:47:06.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:47:05 smithi164 ceph-mon[31916]: osdmap e28: 4 total, 4 up, 4 in 2024-04-26T21:47:07.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:47:06 smithi076 ceph-mon[31823]: pgmap v136: 1 pgs: 1 peering; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-26T21:47:07.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:47:06 smithi076 ceph-mon[31823]: osdmap e29: 4 total, 4 up, 4 in 2024-04-26T21:47:07.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:47:06 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:47:07.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:47:06 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:47:07.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:47:06 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:47:07.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:47:06 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:47:07.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:47:06 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:47:07.066 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:47:06 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:47:07.066 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:47:06 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:47:07.066 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:47:06 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:47:07.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:47:06 smithi072 ceph-mon[28138]: pgmap v136: 1 pgs: 1 peering; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-26T21:47:07.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:47:06 smithi072 ceph-mon[28138]: osdmap e29: 4 total, 4 up, 4 in 2024-04-26T21:47:07.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:47:06 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:47:07.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:47:06 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:47:07.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:47:06 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:47:07.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:47:06 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:47:07.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:47:06 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:47:07.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:47:06 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:47:07.073 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:47:06 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:47:07.073 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:47:06 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:47:07.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:47:06 smithi164 ceph-mon[31916]: pgmap v136: 1 pgs: 1 peering; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-26T21:47:07.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:47:06 smithi164 ceph-mon[31916]: osdmap e29: 4 total, 4 up, 4 in 2024-04-26T21:47:07.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:47:06 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:47:07.192 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:47:06 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:47:07.192 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:47:06 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:47:07.192 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:47:06 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:47:07.192 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:47:06 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:47:07.192 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:47:06 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:47:07.192 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:47:06 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:47:07.192 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:47:06 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:47:07.907 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:47:07 smithi076 ceph-mon[31823]: Detected new or changed devices on smithi072 2024-04-26T21:47:07.907 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:47:07 smithi076 ceph-mon[31823]: Adjusting osd_memory_target on smithi072 to 8544M 2024-04-26T21:47:07.988 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:47:07.986+0000 7fc387e00700 1 -- 172.21.15.76:0/432017157 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc3880799e0 msgr2=0x7fc38807a260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:07.988 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:47:07.986+0000 7fc387e00700 1 --2- 172.21.15.76:0/432017157 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc3880799e0 0x7fc38807a260 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fc38000b0a0 tx=0x7fc38000b3b0 comp rx=0 tx=0).stop 2024-04-26T21:47:07.988 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:47:07.986+0000 7fc387e00700 1 -- 172.21.15.76:0/432017157 shutdown_connections 2024-04-26T21:47:07.988 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:47:07.986+0000 7fc387e00700 1 --2- 172.21.15.76:0/432017157 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc388077dd0 0x7fc3880781f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:07.988 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:47:07.986+0000 7fc387e00700 1 --2- 172.21.15.76:0/432017157 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc3880799e0 0x7fc38807a260 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:07.989 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:47:07.986+0000 7fc387e00700 1 --2- 172.21.15.76:0/432017157 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc388079020 0x7fc3880794a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:07.989 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:47:07.986+0000 7fc387e00700 1 -- 172.21.15.76:0/432017157 >> 172.21.15.76:0/432017157 conn(0x7fc388100b40 msgr2=0x7fc388102f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:07.989 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:47:07.986+0000 7fc387e00700 1 -- 172.21.15.76:0/432017157 shutdown_connections 2024-04-26T21:47:07.989 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:47:07.986+0000 7fc387e00700 1 -- 172.21.15.76:0/432017157 wait complete. 2024-04-26T21:47:07.990 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:47:07.988+0000 7fc387e00700 1 Processor -- start 2024-04-26T21:47:07.990 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:47:07.988+0000 7fc387e00700 1 -- start start 2024-04-26T21:47:07.991 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:47:07.988+0000 7fc387e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc388077dd0 0x7fc38819f890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:07.991 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:47:07.988+0000 7fc387e00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc388079020 0x7fc38819fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:07.991 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:47:07.988+0000 7fc387e00700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc3880799e0 0x7fc3881a6e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:07.991 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:47:07.988+0000 7fc387e00700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fc388115320 con 0x7fc388079020 2024-04-26T21:47:07.991 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:47:07.988+0000 7fc387e00700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fc3881151a0 con 0x7fc3880799e0 2024-04-26T21:47:07.991 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:47:07.988+0000 7fc387e00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fc3881154a0 con 0x7fc388077dd0 2024-04-26T21:47:07.991 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:47:07.989+0000 7fc387400700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc3880799e0 0x7fc3881a6e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:07.991 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:47:07.989+0000 7fc387400700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc3880799e0 0x7fc3881a6e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:57136/0 (socket says 172.21.15.76:57136) 2024-04-26T21:47:07.991 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:47:07.989+0000 7fc387400700 1 -- 172.21.15.76:0/547713915 learned_addr learned my addr 172.21.15.76:0/547713915 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-26T21:47:07.991 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:47:07.989+0000 7fc386a00700 1 --2- 172.21.15.76:0/547713915 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc388077dd0 0x7fc38819f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:07.991 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:47:07.989+0000 7fc37fe00700 1 --2- 172.21.15.76:0/547713915 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc388079020 0x7fc38819fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:07.991 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:47:07.989+0000 7fc387400700 1 -- 172.21.15.76:0/547713915 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc388077dd0 msgr2=0x7fc38819f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:07.992 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:47:07.989+0000 7fc387400700 1 --2- 172.21.15.76:0/547713915 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc388077dd0 0x7fc38819f890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:07.992 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:47:07.989+0000 7fc387400700 1 -- 172.21.15.76:0/547713915 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc388079020 msgr2=0x7fc38819fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:07.992 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:47:07.989+0000 7fc387400700 1 --2- 172.21.15.76:0/547713915 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc388079020 0x7fc38819fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:07.992 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:47:07.989+0000 7fc387400700 1 -- 172.21.15.76:0/547713915 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3881a74f0 con 0x7fc3880799e0 2024-04-26T21:47:07.992 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:47:07.989+0000 7fc386a00700 1 --2- 172.21.15.76:0/547713915 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc388077dd0 0x7fc38819f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:47:07.992 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:47:07.989+0000 7fc37fe00700 1 --2- 172.21.15.76:0/547713915 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc388079020 0x7fc38819fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:47:07.993 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:47:07.990+0000 7fc387400700 1 --2- 172.21.15.76:0/547713915 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc3880799e0 0x7fc3881a6e50 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fc380007ed0 tx=0x7fc38000be20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:07.993 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:47:07.990+0000 7fc38ca00700 1 -- 172.21.15.76:0/547713915 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc380021070 con 0x7fc3880799e0 2024-04-26T21:47:07.993 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:47:07.990+0000 7fc38ca00700 1 -- 172.21.15.76:0/547713915 <== mon.2 v2:172.21.15.76:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc3800054d0 con 0x7fc3880799e0 2024-04-26T21:47:07.993 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:47:07.990+0000 7fc387e00700 1 -- 172.21.15.76:0/547713915 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3881a7780 con 0x7fc3880799e0 2024-04-26T21:47:07.993 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:47:07.990+0000 7fc38ca00700 1 -- 172.21.15.76:0/547713915 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc38001b8d0 con 0x7fc3880799e0 2024-04-26T21:47:07.993 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:47:07.990+0000 7fc387e00700 1 -- 172.21.15.76:0/547713915 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3881a7bd0 con 0x7fc3880799e0 2024-04-26T21:47:07.993 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:47:07.991+0000 7fc387e00700 1 -- 172.21.15.76:0/547713915 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc38806a110 con 0x7fc3880799e0 2024-04-26T21:47:07.994 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:47:07.993+0000 7fc38ca00700 1 -- 172.21.15.76:0/547713915 <== mon.2 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fc380024070 con 0x7fc3880799e0 2024-04-26T21:47:07.996 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:47:07.995+0000 7fc38ca00700 1 --2- 172.21.15.76:0/547713915 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fc36c0a3c40 0x7fc36c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:07.996 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:47:07.995+0000 7fc38ca00700 1 -- 172.21.15.76:0/547713915 <== mon.2 v2:172.21.15.76:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 3669+0+0 (secure 0 0 0) 0x7fc38000f020 con 0x7fc3880799e0 2024-04-26T21:47:07.996 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:47:07.995+0000 7fc386a00700 1 --2- 172.21.15.76:0/547713915 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fc36c0a3c40 0x7fc36c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:07.997 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:47:07.996+0000 7fc386a00700 1 --2- 172.21.15.76:0/547713915 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fc36c0a3c40 0x7fc36c0a6100 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc378005fd0 tx=0x7fc378005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:07.999 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:47:07.998+0000 7fc38ca00700 1 -- 172.21.15.76:0/547713915 <== mon.2 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fc38002b020 con 0x7fc3880799e0 2024-04-26T21:47:08.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:47:07 smithi072 ceph-mon[28138]: Detected new or changed devices on smithi072 2024-04-26T21:47:08.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:47:07 smithi072 ceph-mon[28138]: Adjusting osd_memory_target on smithi072 to 8544M 2024-04-26T21:47:08.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:47:07 smithi164 ceph-mon[31916]: Detected new or changed devices on smithi072 2024-04-26T21:47:08.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:47:07 smithi164 ceph-mon[31916]: Adjusting osd_memory_target on smithi072 to 8544M 2024-04-26T21:47:08.234 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:47:08.232+0000 7fc387e00700 1 -- 172.21.15.76:0/547713915 --> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi076:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fc388063920 con 0x7fc36c0a3c40 2024-04-26T21:47:09.064 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:47:08 smithi076 ceph-mon[31823]: pgmap v138: 1 pgs: 1 peering; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 117 KiB/s, 0 objects/s recovering 2024-04-26T21:47:09.064 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:47:08 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:47:09.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:47:08 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:47:09.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:47:08 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:47:09.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:47:08 smithi072 ceph-mon[28138]: pgmap v138: 1 pgs: 1 peering; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 117 KiB/s, 0 objects/s recovering 2024-04-26T21:47:09.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:47:08 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:47:09.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:47:08 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:47:09.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:47:08 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:47:09.149 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:47:08 smithi164 ceph-mon[31916]: pgmap v138: 1 pgs: 1 peering; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 117 KiB/s, 0 objects/s recovering 2024-04-26T21:47:09.149 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:47:08 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:47:09.149 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:47:08 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:47:09.149 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:47:08 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:47:10.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:47:09 smithi072 ceph-mon[28138]: from='client.34205 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi076:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:47:10.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:47:09 smithi164 ceph-mon[31916]: from='client.34205 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi076:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:47:10.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:47:09 smithi076 ceph-mon[31823]: from='client.34205 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi076:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:47:11.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:47:10 smithi076 ceph-mon[31823]: pgmap v139: 1 pgs: 1 peering; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-26T21:47:11.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:47:10 smithi072 ceph-mon[28138]: pgmap v139: 1 pgs: 1 peering; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-26T21:47:11.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:47:10 smithi164 ceph-mon[31916]: pgmap v139: 1 pgs: 1 peering; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-26T21:47:13.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:47:12 smithi164 ceph-mon[31916]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-04-26T21:47:13.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:47:12 smithi164 ceph-mon[31916]: from='client.? 172.21.15.76:0/2765693347' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d2cfeea7-1661-4156-bdcd-f770a8d1a878"}]: dispatch 2024-04-26T21:47:13.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:47:12 smithi164 ceph-mon[31916]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d2cfeea7-1661-4156-bdcd-f770a8d1a878"}]: dispatch 2024-04-26T21:47:13.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:47:12 smithi164 ceph-mon[31916]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d2cfeea7-1661-4156-bdcd-f770a8d1a878"}]': finished 2024-04-26T21:47:13.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:47:12 smithi164 ceph-mon[31916]: osdmap e30: 5 total, 4 up, 5 in 2024-04-26T21:47:13.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:47:12 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T21:47:13.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:47:12 smithi076 ceph-mon[31823]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-04-26T21:47:13.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:47:12 smithi076 ceph-mon[31823]: from='client.? 172.21.15.76:0/2765693347' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d2cfeea7-1661-4156-bdcd-f770a8d1a878"}]: dispatch 2024-04-26T21:47:13.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:47:12 smithi076 ceph-mon[31823]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d2cfeea7-1661-4156-bdcd-f770a8d1a878"}]: dispatch 2024-04-26T21:47:13.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:47:12 smithi076 ceph-mon[31823]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d2cfeea7-1661-4156-bdcd-f770a8d1a878"}]': finished 2024-04-26T21:47:13.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:47:12 smithi076 ceph-mon[31823]: osdmap e30: 5 total, 4 up, 5 in 2024-04-26T21:47:13.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:47:12 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T21:47:13.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:47:12 smithi072 ceph-mon[28138]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-04-26T21:47:13.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:47:12 smithi072 ceph-mon[28138]: from='client.? 172.21.15.76:0/2765693347' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d2cfeea7-1661-4156-bdcd-f770a8d1a878"}]: dispatch 2024-04-26T21:47:13.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:47:12 smithi072 ceph-mon[28138]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d2cfeea7-1661-4156-bdcd-f770a8d1a878"}]: dispatch 2024-04-26T21:47:13.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:47:12 smithi072 ceph-mon[28138]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d2cfeea7-1661-4156-bdcd-f770a8d1a878"}]': finished 2024-04-26T21:47:13.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:47:12 smithi072 ceph-mon[28138]: osdmap e30: 5 total, 4 up, 5 in 2024-04-26T21:47:13.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:47:12 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T21:47:14.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:47:13 smithi164 ceph-mon[31916]: from='client.? 172.21.15.76:0/2075086968' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:47:14.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:47:13 smithi076 ceph-mon[31823]: from='client.? 172.21.15.76:0/2075086968' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:47:14.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:47:13 smithi072 ceph-mon[28138]: from='client.? 172.21.15.76:0/2075086968' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:47:15.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:47:14 smithi164 ceph-mon[31916]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-04-26T21:47:15.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:47:14 smithi076 ceph-mon[31823]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-04-26T21:47:15.321 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:47:14 smithi072 ceph-mon[28138]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-04-26T21:47:17.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:47:16 smithi164 ceph-mon[31916]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-04-26T21:47:17.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:47:16 smithi076 ceph-mon[31823]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-04-26T21:47:17.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:47:16 smithi072 ceph-mon[28138]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-04-26T21:47:19.167 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:47:18 smithi076 ceph-mon[31823]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-04-26T21:47:19.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:47:18 smithi164 ceph-mon[31916]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-04-26T21:47:19.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:47:18 smithi072 ceph-mon[28138]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-04-26T21:47:21.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:47:20 smithi076 ceph-mon[31823]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-04-26T21:47:21.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:47:20 smithi164 ceph-mon[31916]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-04-26T21:47:21.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:47:20 smithi072 ceph-mon[28138]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-04-26T21:47:23.151 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:47:22 smithi076 ceph-mon[31823]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-04-26T21:47:23.152 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:47:22 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-26T21:47:23.152 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:47:22 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:47:23.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:47:22 smithi164 ceph-mon[31916]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-04-26T21:47:23.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:47:22 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-26T21:47:23.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:47:22 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:47:23.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:47:22 smithi072 ceph-mon[28138]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-04-26T21:47:23.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:47:22 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-26T21:47:23.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:47:22 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:47:24.064 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:47:23 smithi076 ceph-mon[31823]: Deploying daemon osd.4 on smithi076 2024-04-26T21:47:24.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:47:23 smithi164 ceph-mon[31916]: Deploying daemon osd.4 on smithi076 2024-04-26T21:47:24.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:47:23 smithi072 ceph-mon[28138]: Deploying daemon osd.4 on smithi076 2024-04-26T21:47:25.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:47:24 smithi076 ceph-mon[31823]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-04-26T21:47:25.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:47:24 smithi164 ceph-mon[31916]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-04-26T21:47:25.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:47:24 smithi072 ceph-mon[28138]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-04-26T21:47:26.959 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:47:26 smithi076 ceph-mon[31823]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-04-26T21:47:27.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:47:26 smithi164 ceph-mon[31916]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-04-26T21:47:27.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:47:26 smithi072 ceph-mon[28138]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-04-26T21:47:29.164 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:47:28 smithi072 ceph-mon[28138]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-04-26T21:47:29.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:47:28 smithi164 ceph-mon[31916]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-04-26T21:47:29.263 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:47:28 smithi076 ceph-mon[31823]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-04-26T21:47:31.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:47:30 smithi072 ceph-mon[28138]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-04-26T21:47:31.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:47:30 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:47:31.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:47:30 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:47:31.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:47:30 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:47:31.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:47:30 smithi164 ceph-mon[31916]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-04-26T21:47:31.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:47:30 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:47:31.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:47:30 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:47:31.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:47:30 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:47:31.257 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:47:30 smithi076 ceph-mon[31823]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-04-26T21:47:31.257 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:47:30 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:47:31.257 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:47:30 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:47:31.257 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:47:30 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:47:33.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:47:32 smithi164 ceph-mon[31916]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-04-26T21:47:33.255 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:47:32 smithi076 ceph-mon[31823]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-04-26T21:47:33.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:47:32 smithi072 ceph-mon[28138]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-04-26T21:47:35.025 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:47:34 smithi076 ceph-mon[31823]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-04-26T21:47:35.026 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:47:34 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:47:35.026 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:47:34 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:47:35.026 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:47:34 smithi076 ceph-mon[31823]: from='osd.4 [v2:172.21.15.76:6808/3718867827,v1:172.21.15.76:6809/3718867827]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-26T21:47:35.026 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:47:34 smithi076 ceph-mon[31823]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-26T21:47:35.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:47:34 smithi072 ceph-mon[28138]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-04-26T21:47:35.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:47:34 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:47:35.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:47:34 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:47:35.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:47:34 smithi072 ceph-mon[28138]: from='osd.4 [v2:172.21.15.76:6808/3718867827,v1:172.21.15.76:6809/3718867827]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-26T21:47:35.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:47:34 smithi072 ceph-mon[28138]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-26T21:47:35.168 INFO:teuthology.orchestra.run.smithi076.stdout:Created osd(s) 4 on host 'smithi076' 2024-04-26T21:47:35.168 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:47:35.167+0000 7fc38ca00700 1 -- 172.21.15.76:0/547713915 <== mgr.14152 v2:172.21.15.72:6800/588150172 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fc388063920 con 0x7fc36c0a3c40 2024-04-26T21:47:35.172 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:47:35.171+0000 7fc387e00700 1 -- 172.21.15.76:0/547713915 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fc36c0a3c40 msgr2=0x7fc36c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:35.172 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:47:35.171+0000 7fc387e00700 1 --2- 172.21.15.76:0/547713915 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fc36c0a3c40 0x7fc36c0a6100 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc378005fd0 tx=0x7fc378005e30 comp rx=0 tx=0).stop 2024-04-26T21:47:35.172 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:47:35.171+0000 7fc387e00700 1 -- 172.21.15.76:0/547713915 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc3880799e0 msgr2=0x7fc3881a6e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:35.172 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:47:35.171+0000 7fc387e00700 1 --2- 172.21.15.76:0/547713915 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc3880799e0 0x7fc3881a6e50 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fc380007ed0 tx=0x7fc38000be20 comp rx=0 tx=0).stop 2024-04-26T21:47:35.172 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:47:35.171+0000 7fc387e00700 1 -- 172.21.15.76:0/547713915 shutdown_connections 2024-04-26T21:47:35.172 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:47:35.171+0000 7fc387e00700 1 --2- 172.21.15.76:0/547713915 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fc36c0a3c40 0x7fc36c0a6100 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:35.172 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:47:35.171+0000 7fc387e00700 1 --2- 172.21.15.76:0/547713915 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc3880799e0 0x7fc3881a6e50 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:35.172 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:47:35.171+0000 7fc387e00700 1 --2- 172.21.15.76:0/547713915 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc388077dd0 0x7fc38819f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:35.172 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:47:35.171+0000 7fc387e00700 1 --2- 172.21.15.76:0/547713915 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc388079020 0x7fc38819fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:35.172 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:47:35.171+0000 7fc387e00700 1 -- 172.21.15.76:0/547713915 >> 172.21.15.76:0/547713915 conn(0x7fc388100b40 msgr2=0x7fc388102640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:35.173 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:47:35.172+0000 7fc387e00700 1 -- 172.21.15.76:0/547713915 shutdown_connections 2024-04-26T21:47:35.173 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:47:35.172+0000 7fc387e00700 1 -- 172.21.15.76:0/547713915 wait complete. 2024-04-26T21:47:35.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:47:34 smithi164 ceph-mon[31916]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-04-26T21:47:35.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:47:34 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:47:35.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:47:34 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:47:35.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:47:34 smithi164 ceph-mon[31916]: from='osd.4 [v2:172.21.15.76:6808/3718867827,v1:172.21.15.76:6809/3718867827]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-26T21:47:35.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:47:34 smithi164 ceph-mon[31916]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-26T21:47:35.934 DEBUG:teuthology.orchestra.run.smithi076:osd.4> sudo journalctl -f -n 0 -u ceph-a1780950-0415-11ef-bc93-c7b262605968@osd.4.service 2024-04-26T21:47:35.940 INFO:tasks.cephadm:Deploying osd.5 on smithi164 with /dev/vg_nvme/lv_3... 2024-04-26T21:47:35.940 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a1780950-0415-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-26T21:47:36.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:47:35 smithi164 ceph-mon[31916]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-26T21:47:36.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:47:35 smithi164 ceph-mon[31916]: osdmap e31: 5 total, 4 up, 5 in 2024-04-26T21:47:36.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:47:35 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T21:47:36.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:47:35 smithi164 ceph-mon[31916]: from='osd.4 [v2:172.21.15.76:6808/3718867827,v1:172.21.15.76:6809/3718867827]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi076", "root=default"]}]: dispatch 2024-04-26T21:47:36.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:47:35 smithi164 ceph-mon[31916]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi076", "root=default"]}]: dispatch 2024-04-26T21:47:36.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:47:35 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:47:36.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:47:35 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:47:36.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:47:35 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:47:36.192 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:47:35 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:47:36.192 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:47:35 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:47:36.192 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:47:35 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:47:36.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:47:35 smithi076 ceph-mon[31823]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-26T21:47:36.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:47:35 smithi076 ceph-mon[31823]: osdmap e31: 5 total, 4 up, 5 in 2024-04-26T21:47:36.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:47:35 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T21:47:36.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:47:35 smithi076 ceph-mon[31823]: from='osd.4 [v2:172.21.15.76:6808/3718867827,v1:172.21.15.76:6809/3718867827]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi076", "root=default"]}]: dispatch 2024-04-26T21:47:36.316 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:47:35 smithi076 ceph-mon[31823]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi076", "root=default"]}]: dispatch 2024-04-26T21:47:36.316 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:47:35 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:47:36.316 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:47:35 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:47:36.316 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:47:35 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:47:36.316 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:47:35 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:47:36.316 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:47:35 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:47:36.316 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:47:35 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:47:36.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:47:35 smithi072 ceph-mon[28138]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-26T21:47:36.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:47:35 smithi072 ceph-mon[28138]: osdmap e31: 5 total, 4 up, 5 in 2024-04-26T21:47:36.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:47:35 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T21:47:36.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:47:35 smithi072 ceph-mon[28138]: from='osd.4 [v2:172.21.15.76:6808/3718867827,v1:172.21.15.76:6809/3718867827]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi076", "root=default"]}]: dispatch 2024-04-26T21:47:36.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:47:35 smithi072 ceph-mon[28138]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi076", "root=default"]}]: dispatch 2024-04-26T21:47:36.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:47:35 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:47:36.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:47:35 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:47:36.323 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:47:35 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:47:36.323 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:47:35 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:47:36.323 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:47:35 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:47:36.323 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:47:35 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:47:36.373 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/a1780950-0415-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T21:47:37.015 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:47:36 smithi076 ceph-mon[31823]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-04-26T21:47:37.015 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:47:36 smithi076 ceph-mon[31823]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi076", "root=default"]}]': finished 2024-04-26T21:47:37.015 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:47:36 smithi076 ceph-mon[31823]: osdmap e32: 5 total, 4 up, 5 in 2024-04-26T21:47:37.015 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:47:36 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T21:47:37.015 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:47:36 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T21:47:37.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:47:36 smithi164 ceph-mon[31916]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-04-26T21:47:37.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:47:36 smithi164 ceph-mon[31916]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi076", "root=default"]}]': finished 2024-04-26T21:47:37.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:47:36 smithi164 ceph-mon[31916]: osdmap e32: 5 total, 4 up, 5 in 2024-04-26T21:47:37.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:47:36 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T21:47:37.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:47:36 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T21:47:37.315 INFO:journalctl@ceph.osd.4.smithi076.stdout:Apr 26 21:47:37 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-4[42901]: 2024-04-26T21:47:37.013+0000 7f805e400700 -1 osd.4 0 waiting for initial osdmap 2024-04-26T21:47:37.315 INFO:journalctl@ceph.osd.4.smithi076.stdout:Apr 26 21:47:37 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-4[42901]: 2024-04-26T21:47:37.049+0000 7f8054200700 -1 osd.4 32 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T21:47:37.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:47:36 smithi072 ceph-mon[28138]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-04-26T21:47:37.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:47:36 smithi072 ceph-mon[28138]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi076", "root=default"]}]': finished 2024-04-26T21:47:37.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:47:36 smithi072 ceph-mon[28138]: osdmap e32: 5 total, 4 up, 5 in 2024-04-26T21:47:37.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:47:36 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T21:47:37.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:47:36 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T21:47:37.943 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:47:37 smithi076 ceph-mon[31823]: purged_snaps scrub starts 2024-04-26T21:47:37.943 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:47:37 smithi076 ceph-mon[31823]: purged_snaps scrub ok 2024-04-26T21:47:37.944 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:47:37 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T21:47:37.944 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:47:37 smithi076 ceph-mon[31823]: from='osd.4 ' entity='osd.4' 2024-04-26T21:47:37.944 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:47:37 smithi076 ceph-mon[31823]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-04-26T21:47:38.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:47:37 smithi164 ceph-mon[31916]: purged_snaps scrub starts 2024-04-26T21:47:38.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:47:37 smithi164 ceph-mon[31916]: purged_snaps scrub ok 2024-04-26T21:47:38.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:47:37 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T21:47:38.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:47:37 smithi164 ceph-mon[31916]: from='osd.4 ' entity='osd.4' 2024-04-26T21:47:38.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:47:37 smithi164 ceph-mon[31916]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-04-26T21:47:38.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:47:37 smithi072 ceph-mon[28138]: purged_snaps scrub starts 2024-04-26T21:47:38.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:47:37 smithi072 ceph-mon[28138]: purged_snaps scrub ok 2024-04-26T21:47:38.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:47:37 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T21:47:38.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:47:37 smithi072 ceph-mon[28138]: from='osd.4 ' entity='osd.4' 2024-04-26T21:47:38.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:47:37 smithi072 ceph-mon[28138]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-04-26T21:47:39.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:47:38 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T21:47:39.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:47:38 smithi164 ceph-mon[31916]: osd.4 [v2:172.21.15.76:6808/3718867827,v1:172.21.15.76:6809/3718867827] boot 2024-04-26T21:47:39.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:47:38 smithi164 ceph-mon[31916]: osdmap e33: 5 total, 5 up, 5 in 2024-04-26T21:47:39.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:47:38 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T21:47:39.245 INFO:teuthology.orchestra.run.smithi164.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-26T21:47:39.246 INFO:teuthology.orchestra.run.smithi164.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-26T21:47:39.246 INFO:teuthology.orchestra.run.smithi164.stderr: stderr: 10+0 records in 2024-04-26T21:47:39.246 INFO:teuthology.orchestra.run.smithi164.stderr:10+0 records out 2024-04-26T21:47:39.246 INFO:teuthology.orchestra.run.smithi164.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0468118 s, 224 MB/s 2024-04-26T21:47:39.246 INFO:teuthology.orchestra.run.smithi164.stderr:--> Zapping successful for: 2024-04-26T21:47:39.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:47:38 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T21:47:39.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:47:38 smithi076 ceph-mon[31823]: osd.4 [v2:172.21.15.76:6808/3718867827,v1:172.21.15.76:6809/3718867827] boot 2024-04-26T21:47:39.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:47:38 smithi076 ceph-mon[31823]: osdmap e33: 5 total, 5 up, 5 in 2024-04-26T21:47:39.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:47:38 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T21:47:39.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:47:38 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T21:47:39.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:47:38 smithi072 ceph-mon[28138]: osd.4 [v2:172.21.15.76:6808/3718867827,v1:172.21.15.76:6809/3718867827] boot 2024-04-26T21:47:39.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:47:38 smithi072 ceph-mon[28138]: osdmap e33: 5 total, 5 up, 5 in 2024-04-26T21:47:39.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:47:38 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T21:47:39.983 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a1780950-0415-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi164:vg_nvme/lv_3 2024-04-26T21:47:40.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:47:40 smithi164 ceph-mon[31916]: osdmap e34: 5 total, 5 up, 5 in 2024-04-26T21:47:40.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:47:40 smithi164 ceph-mon[31916]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:47:40.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:47:40 smithi076 ceph-mon[31823]: osdmap e34: 5 total, 5 up, 5 in 2024-04-26T21:47:40.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:47:40 smithi076 ceph-mon[31823]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:47:40.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:47:40 smithi072 ceph-mon[28138]: osdmap e34: 5 total, 5 up, 5 in 2024-04-26T21:47:40.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:47:40 smithi072 ceph-mon[28138]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:47:40.396 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/a1780950-0415-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T21:47:41.691 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:47:41 smithi164 ceph-mon[31916]: Detected new or changed devices on smithi076 2024-04-26T21:47:41.691 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:47:41 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:47:41.691 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:47:41 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:47:41.691 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:47:41 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:47:41.691 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:47:41 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:47:41.691 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:47:41 smithi164 ceph-mon[31916]: Adjusting osd_memory_target on smithi076 to 8544M 2024-04-26T21:47:41.691 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:47:41 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:47:41.691 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:47:41 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:47:41.691 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:47:41 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:47:41.691 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:47:41 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:47:42.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:47:41 smithi076 ceph-mon[31823]: Detected new or changed devices on smithi076 2024-04-26T21:47:42.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:47:41 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:47:42.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:47:41 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:47:42.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:47:41 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:47:42.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:47:41 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:47:42.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:47:41 smithi076 ceph-mon[31823]: Adjusting osd_memory_target on smithi076 to 8544M 2024-04-26T21:47:42.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:47:41 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:47:42.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:47:41 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:47:42.066 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:47:41 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:47:42.066 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:47:41 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:47:42.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:47:41 smithi072 ceph-mon[28138]: Detected new or changed devices on smithi076 2024-04-26T21:47:42.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:47:41 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:47:42.073 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:47:41 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:47:42.073 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:47:41 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:47:42.073 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:47:41 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:47:42.073 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:47:41 smithi072 ceph-mon[28138]: Adjusting osd_memory_target on smithi076 to 8544M 2024-04-26T21:47:42.073 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:47:41 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:47:42.073 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:47:41 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:47:42.073 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:47:41 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:47:42.073 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:47:41 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:47:42.505 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:47:42.504+0000 7f6534a00700 1 -- 172.21.15.164:0/3141209474 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6530109ac0 msgr2=0x7f6530109ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:42.505 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:47:42.504+0000 7f6534a00700 1 --2- 172.21.15.164:0/3141209474 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6530109ac0 0x7f6530109ee0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f6520009990 tx=0x7f6520009ca0 comp rx=0 tx=0).stop 2024-04-26T21:47:42.505 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:47:42.504+0000 7f6534a00700 1 -- 172.21.15.164:0/3141209474 shutdown_connections 2024-04-26T21:47:42.505 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:47:42.504+0000 7f6534a00700 1 --2- 172.21.15.164:0/3141209474 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f653010b660 0x7f6530111f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:42.505 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:47:42.504+0000 7f6534a00700 1 --2- 172.21.15.164:0/3141209474 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6530109ac0 0x7f6530109ee0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:42.505 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:47:42.504+0000 7f6534a00700 1 --2- 172.21.15.164:0/3141209474 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f653010acc0 0x7f653010b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:42.505 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:47:42.504+0000 7f6534a00700 1 -- 172.21.15.164:0/3141209474 >> 172.21.15.164:0/3141209474 conn(0x7f6530078a90 msgr2=0x7f653007aed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:42.505 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:47:42.504+0000 7f6534a00700 1 -- 172.21.15.164:0/3141209474 shutdown_connections 2024-04-26T21:47:42.505 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:47:42.504+0000 7f6534a00700 1 -- 172.21.15.164:0/3141209474 wait complete. 2024-04-26T21:47:42.506 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:47:42.506+0000 7f6534a00700 1 Processor -- start 2024-04-26T21:47:42.506 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:47:42.506+0000 7f6534a00700 1 -- start start 2024-04-26T21:47:42.508 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:47:42.506+0000 7f6534a00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6530109ac0 0x7f653019fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:42.508 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:47:42.506+0000 7f6534a00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f653010acc0 0x7f65301a0060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:42.508 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:47:42.506+0000 7f6534a00700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f653010b660 0x7f65301a70e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:42.508 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:47:42.506+0000 7f6534a00700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f6530115450 con 0x7f653010acc0 2024-04-26T21:47:42.509 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:47:42.506+0000 7f6534a00700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f65301152d0 con 0x7f653010b660 2024-04-26T21:47:42.509 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:47:42.506+0000 7f6534a00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f65301155d0 con 0x7f6530109ac0 2024-04-26T21:47:42.509 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:47:42.507+0000 7f652de00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6530109ac0 0x7f653019fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:42.509 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:47:42.507+0000 7f652de00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6530109ac0 0x7f653019fb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:52410/0 (socket says 172.21.15.164:52410) 2024-04-26T21:47:42.509 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:47:42.507+0000 7f652de00700 1 -- 172.21.15.164:0/799298366 learned_addr learned my addr 172.21.15.164:0/799298366 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-04-26T21:47:42.509 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:47:42.507+0000 7f652d400700 1 --2- 172.21.15.164:0/799298366 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f653010acc0 0x7f65301a0060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:42.509 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:47:42.507+0000 7f652de00700 1 -- 172.21.15.164:0/799298366 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f653010b660 msgr2=0x7f65301a70e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:42.509 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:47:42.507+0000 7f652e800700 1 --2- 172.21.15.164:0/799298366 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f653010b660 0x7f65301a70e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:42.509 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:47:42.507+0000 7f652de00700 1 --2- 172.21.15.164:0/799298366 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f653010b660 0x7f65301a70e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:42.509 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:47:42.507+0000 7f652de00700 1 -- 172.21.15.164:0/799298366 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f653010acc0 msgr2=0x7f65301a0060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:42.509 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:47:42.507+0000 7f652de00700 1 --2- 172.21.15.164:0/799298366 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f653010acc0 0x7f65301a0060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:42.509 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:47:42.507+0000 7f652de00700 1 -- 172.21.15.164:0/799298366 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65301a7780 con 0x7f6530109ac0 2024-04-26T21:47:42.510 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:47:42.507+0000 7f652d400700 1 --2- 172.21.15.164:0/799298366 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f653010acc0 0x7f65301a0060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:47:42.510 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:47:42.507+0000 7f652e800700 1 --2- 172.21.15.164:0/799298366 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f653010b660 0x7f65301a70e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:47:42.510 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:47:42.508+0000 7f652de00700 1 --2- 172.21.15.164:0/799298366 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6530109ac0 0x7f653019fb20 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f6520005f50 tx=0x7f6520009ee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:42.510 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:47:42.508+0000 7f6526a00700 1 -- 172.21.15.164:0/799298366 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6520027070 con 0x7f6530109ac0 2024-04-26T21:47:42.510 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:47:42.508+0000 7f6526a00700 1 -- 172.21.15.164:0/799298366 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f65200040e0 con 0x7f6530109ac0 2024-04-26T21:47:42.510 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:47:42.508+0000 7f6526a00700 1 -- 172.21.15.164:0/799298366 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6520022e20 con 0x7f6530109ac0 2024-04-26T21:47:42.510 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:47:42.508+0000 7f6534a00700 1 -- 172.21.15.164:0/799298366 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65301a7a10 con 0x7f6530109ac0 2024-04-26T21:47:42.510 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:47:42.508+0000 7f6534a00700 1 -- 172.21.15.164:0/799298366 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65301a7ec0 con 0x7f6530109ac0 2024-04-26T21:47:42.511 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:47:42.509+0000 7f6534a00700 1 -- 172.21.15.164:0/799298366 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f653006a110 con 0x7f6530109ac0 2024-04-26T21:47:42.512 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:47:42.511+0000 7f6526a00700 1 -- 172.21.15.164:0/799298366 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f652002a070 con 0x7f6530109ac0 2024-04-26T21:47:42.513 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:47:42.513+0000 7f6526a00700 1 --2- 172.21.15.164:0/799298366 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f65140a3cc0 0x7f65140a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:42.513 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:47:42.513+0000 7f6526a00700 1 -- 172.21.15.164:0/799298366 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 4101+0+0 (secure 0 0 0) 0x7f65200d0700 con 0x7f6530109ac0 2024-04-26T21:47:42.513 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:47:42.513+0000 7f652d400700 1 --2- 172.21.15.164:0/799298366 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f65140a3cc0 0x7f65140a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:42.514 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:47:42.514+0000 7f652d400700 1 --2- 172.21.15.164:0/799298366 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f65140a3cc0 0x7f65140a6180 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f651c005fd0 tx=0x7f651c005dc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:42.516 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:47:42.516+0000 7f6526a00700 1 -- 172.21.15.164:0/799298366 <== mon.1 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f6520013070 con 0x7f6530109ac0 2024-04-26T21:47:42.753 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:47:42.753+0000 7f6534a00700 1 -- 172.21.15.164:0/799298366 --> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi164:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f6530063860 con 0x7f65140a3cc0 2024-04-26T21:47:42.940 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:47:42 smithi164 ceph-mon[31916]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:47:43.064 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:47:42 smithi076 ceph-mon[31823]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:47:43.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:47:42 smithi072 ceph-mon[28138]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:47:43.869 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:47:43 smithi164 ceph-mon[31916]: from='client.24220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi164:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:47:43.870 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:47:43 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:47:43.870 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:47:43 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:47:43.870 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:47:43 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:47:44.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:47:43 smithi076 ceph-mon[31823]: from='client.24220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi164:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:47:44.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:47:43 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:47:44.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:47:43 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:47:44.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:47:43 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:47:44.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:47:43 smithi072 ceph-mon[28138]: from='client.24220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi164:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:47:44.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:47:43 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:47:44.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:47:43 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:47:44.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:47:43 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:47:44.871 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:47:44 smithi164 ceph-mon[31916]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:47:45.064 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:47:44 smithi076 ceph-mon[31823]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:47:45.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:47:44 smithi072 ceph-mon[28138]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:47:47.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:47:46 smithi076 ceph-mon[31823]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:47:47.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:47:46 smithi076 ceph-mon[31823]: from='client.? 172.21.15.164:0/1371458364' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f3896ad6-a040-4ea3-a431-ff9b7fc05c36"}]: dispatch 2024-04-26T21:47:47.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:47:46 smithi076 ceph-mon[31823]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f3896ad6-a040-4ea3-a431-ff9b7fc05c36"}]: dispatch 2024-04-26T21:47:47.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:47:46 smithi076 ceph-mon[31823]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f3896ad6-a040-4ea3-a431-ff9b7fc05c36"}]': finished 2024-04-26T21:47:47.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:47:46 smithi076 ceph-mon[31823]: osdmap e35: 6 total, 5 up, 6 in 2024-04-26T21:47:47.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:47:46 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T21:47:47.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:47:46 smithi072 ceph-mon[28138]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:47:47.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:47:46 smithi072 ceph-mon[28138]: from='client.? 172.21.15.164:0/1371458364' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f3896ad6-a040-4ea3-a431-ff9b7fc05c36"}]: dispatch 2024-04-26T21:47:47.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:47:46 smithi072 ceph-mon[28138]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f3896ad6-a040-4ea3-a431-ff9b7fc05c36"}]: dispatch 2024-04-26T21:47:47.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:47:46 smithi072 ceph-mon[28138]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f3896ad6-a040-4ea3-a431-ff9b7fc05c36"}]': finished 2024-04-26T21:47:47.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:47:46 smithi072 ceph-mon[28138]: osdmap e35: 6 total, 5 up, 6 in 2024-04-26T21:47:47.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:47:46 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T21:47:47.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:47:46 smithi164 ceph-mon[31916]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:47:47.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:47:46 smithi164 ceph-mon[31916]: from='client.? 172.21.15.164:0/1371458364' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f3896ad6-a040-4ea3-a431-ff9b7fc05c36"}]: dispatch 2024-04-26T21:47:47.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:47:46 smithi164 ceph-mon[31916]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f3896ad6-a040-4ea3-a431-ff9b7fc05c36"}]: dispatch 2024-04-26T21:47:47.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:47:46 smithi164 ceph-mon[31916]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f3896ad6-a040-4ea3-a431-ff9b7fc05c36"}]': finished 2024-04-26T21:47:47.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:47:46 smithi164 ceph-mon[31916]: osdmap e35: 6 total, 5 up, 6 in 2024-04-26T21:47:47.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:47:46 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T21:47:48.064 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:47:47 smithi076 ceph-mon[31823]: from='client.? 172.21.15.164:0/2797054633' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:47:48.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:47:47 smithi072 ceph-mon[28138]: from='client.? 172.21.15.164:0/2797054633' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:47:48.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:47:47 smithi164 ceph-mon[31916]: from='client.? 172.21.15.164:0/2797054633' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:47:49.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:47:48 smithi072 ceph-mon[28138]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:47:49.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:47:48 smithi164 ceph-mon[31916]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:47:49.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:47:48 smithi076 ceph-mon[31823]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:47:51.073 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:47:50 smithi072 ceph-mon[28138]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:47:51.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:47:50 smithi164 ceph-mon[31916]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:47:51.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:47:50 smithi076 ceph-mon[31823]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:47:53.092 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:47:52 smithi164 ceph-mon[31916]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:47:53.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:47:52 smithi076 ceph-mon[31823]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:47:53.321 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:47:52 smithi072 ceph-mon[28138]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:47:55.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:47:54 smithi164 ceph-mon[31916]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:47:55.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:47:54 smithi076 ceph-mon[31823]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:47:55.321 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:47:54 smithi072 ceph-mon[28138]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:47:57.103 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:47:56 smithi164 ceph-mon[31916]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:47:57.103 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:47:56 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-26T21:47:57.103 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:47:56 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:47:57.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:47:56 smithi076 ceph-mon[31823]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:47:57.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:47:56 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-26T21:47:57.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:47:56 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:47:57.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:47:56 smithi072 ceph-mon[28138]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:47:57.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:47:56 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-26T21:47:57.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:47:56 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:47:58.108 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:47:57 smithi164 ceph-mon[31916]: Deploying daemon osd.5 on smithi164 2024-04-26T21:47:58.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:47:57 smithi076 ceph-mon[31823]: Deploying daemon osd.5 on smithi164 2024-04-26T21:47:58.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:47:57 smithi072 ceph-mon[28138]: Deploying daemon osd.5 on smithi164 2024-04-26T21:47:59.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:47:58 smithi164 ceph-mon[31916]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:47:59.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:47:58 smithi076 ceph-mon[31823]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:47:59.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:47:58 smithi072 ceph-mon[28138]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:48:00.940 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:48:00 smithi164 ceph-mon[31916]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:48:01.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:48:00 smithi076 ceph-mon[31823]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:48:01.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:48:00 smithi072 ceph-mon[28138]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:48:03.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:48:02 smithi164 ceph-mon[31916]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:48:03.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:48:02 smithi076 ceph-mon[31823]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:48:03.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:48:02 smithi072 ceph-mon[28138]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:48:05.135 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:48:04 smithi164 ceph-mon[31916]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:48:05.135 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:48:04 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:48:05.135 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:48:04 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:48:05.135 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:48:04 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:48:05.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:48:04 smithi076 ceph-mon[31823]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:48:05.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:48:04 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:48:05.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:48:04 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:48:05.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:48:04 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:48:05.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:48:04 smithi072 ceph-mon[28138]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:48:05.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:48:04 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:48:05.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:48:04 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:48:05.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:48:04 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:48:07.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:48:06 smithi164 ceph-mon[31916]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:48:07.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:48:06 smithi076 ceph-mon[31823]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:48:07.321 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:48:06 smithi072 ceph-mon[28138]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:48:09.015 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:48:08 smithi164 ceph-mon[31916]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:48:09.015 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:48:08 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:48:09.015 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:48:08 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:48:09.015 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:48:08 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:48:09.016 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:48:08 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:48:09.016 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:48:08 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:48:09.016 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:48:08 smithi164 ceph-mon[31916]: from='osd.5 [v2:172.21.15.164:6808/2543954772,v1:172.21.15.164:6809/2543954772]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-26T21:48:09.016 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:48:08 smithi164 ceph-mon[31916]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-26T21:48:09.016 INFO:teuthology.orchestra.run.smithi164.stdout:Created osd(s) 5 on host 'smithi164' 2024-04-26T21:48:09.016 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:48:09.013+0000 7f6526a00700 1 -- 172.21.15.164:0/799298366 <== mgr.14152 v2:172.21.15.72:6800/588150172 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f6530063860 con 0x7f65140a3cc0 2024-04-26T21:48:09.019 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:48:09.018+0000 7f6534a00700 1 -- 172.21.15.164:0/799298366 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f65140a3cc0 msgr2=0x7f65140a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:09.019 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:48:09.018+0000 7f6534a00700 1 --2- 172.21.15.164:0/799298366 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f65140a3cc0 0x7f65140a6180 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f651c005fd0 tx=0x7f651c005dc0 comp rx=0 tx=0).stop 2024-04-26T21:48:09.019 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:48:09.018+0000 7f6534a00700 1 -- 172.21.15.164:0/799298366 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6530109ac0 msgr2=0x7f653019fb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:09.019 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:48:09.018+0000 7f6534a00700 1 --2- 172.21.15.164:0/799298366 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6530109ac0 0x7f653019fb20 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f6520005f50 tx=0x7f6520009ee0 comp rx=0 tx=0).stop 2024-04-26T21:48:09.020 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:48:09.018+0000 7f6534a00700 1 -- 172.21.15.164:0/799298366 shutdown_connections 2024-04-26T21:48:09.020 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:48:09.018+0000 7f6534a00700 1 --2- 172.21.15.164:0/799298366 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f65140a3cc0 0x7f65140a6180 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:09.020 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:48:09.018+0000 7f6534a00700 1 --2- 172.21.15.164:0/799298366 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f653010b660 0x7f65301a70e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:09.020 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:48:09.018+0000 7f6534a00700 1 --2- 172.21.15.164:0/799298366 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6530109ac0 0x7f653019fb20 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:09.020 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:48:09.018+0000 7f6534a00700 1 --2- 172.21.15.164:0/799298366 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f653010acc0 0x7f65301a0060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:09.020 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:48:09.018+0000 7f6534a00700 1 -- 172.21.15.164:0/799298366 >> 172.21.15.164:0/799298366 conn(0x7f6530078a90 msgr2=0x7f653010e060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:48:09.020 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:48:09.018+0000 7f6534a00700 1 -- 172.21.15.164:0/799298366 shutdown_connections 2024-04-26T21:48:09.020 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:48:09.019+0000 7f6534a00700 1 -- 172.21.15.164:0/799298366 wait complete. 2024-04-26T21:48:09.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:48:08 smithi076 ceph-mon[31823]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:48:09.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:48:08 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:48:09.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:48:08 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:48:09.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:48:08 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:48:09.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:48:08 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:48:09.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:48:08 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:48:09.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:48:08 smithi076 ceph-mon[31823]: from='osd.5 [v2:172.21.15.164:6808/2543954772,v1:172.21.15.164:6809/2543954772]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-26T21:48:09.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:48:08 smithi076 ceph-mon[31823]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-26T21:48:09.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:48:08 smithi072 ceph-mon[28138]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:48:09.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:48:08 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:48:09.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:48:08 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:48:09.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:48:08 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:48:09.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:48:08 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:48:09.073 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:48:08 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:48:09.073 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:48:08 smithi072 ceph-mon[28138]: from='osd.5 [v2:172.21.15.164:6808/2543954772,v1:172.21.15.164:6809/2543954772]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-26T21:48:09.073 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:48:08 smithi072 ceph-mon[28138]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-26T21:48:09.795 DEBUG:teuthology.orchestra.run.smithi164:osd.5> sudo journalctl -f -n 0 -u ceph-a1780950-0415-11ef-bc93-c7b262605968@osd.5.service 2024-04-26T21:48:09.799 INFO:tasks.cephadm:Deploying osd.6 on smithi072 with /dev/vg_nvme/lv_2... 2024-04-26T21:48:09.799 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a1780950-0415-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-26T21:48:10.020 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:48:09 smithi164 ceph-mon[31916]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-26T21:48:10.020 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:48:09 smithi164 ceph-mon[31916]: osdmap e36: 6 total, 5 up, 6 in 2024-04-26T21:48:10.020 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:48:09 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T21:48:10.020 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:48:09 smithi164 ceph-mon[31916]: from='osd.5 [v2:172.21.15.164:6808/2543954772,v1:172.21.15.164:6809/2543954772]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-04-26T21:48:10.021 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:48:09 smithi164 ceph-mon[31916]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-04-26T21:48:10.021 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:48:09 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:48:10.021 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:48:09 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:48:10.021 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:48:09 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:48:10.243 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/a1780950-0415-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:48:10.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:48:09 smithi076 ceph-mon[31823]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-26T21:48:10.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:48:09 smithi076 ceph-mon[31823]: osdmap e36: 6 total, 5 up, 6 in 2024-04-26T21:48:10.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:48:09 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T21:48:10.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:48:09 smithi076 ceph-mon[31823]: from='osd.5 [v2:172.21.15.164:6808/2543954772,v1:172.21.15.164:6809/2543954772]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-04-26T21:48:10.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:48:09 smithi076 ceph-mon[31823]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-04-26T21:48:10.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:48:09 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:48:10.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:48:09 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:48:10.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:48:09 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:48:10.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:48:09 smithi072 ceph-mon[28138]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-26T21:48:10.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:48:09 smithi072 ceph-mon[28138]: osdmap e36: 6 total, 5 up, 6 in 2024-04-26T21:48:10.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:48:09 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T21:48:10.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:48:09 smithi072 ceph-mon[28138]: from='osd.5 [v2:172.21.15.164:6808/2543954772,v1:172.21.15.164:6809/2543954772]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-04-26T21:48:10.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:48:09 smithi072 ceph-mon[28138]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-04-26T21:48:10.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:48:09 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:48:10.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:48:09 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:48:10.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:48:09 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:48:11.060 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:48:10 smithi164 ceph-mon[31916]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:48:11.060 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:48:10 smithi164 ceph-mon[31916]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]': finished 2024-04-26T21:48:11.060 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:48:10 smithi164 ceph-mon[31916]: osdmap e37: 6 total, 5 up, 6 in 2024-04-26T21:48:11.061 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:48:10 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T21:48:11.061 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:48:10 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T21:48:11.061 INFO:journalctl@ceph.osd.5.smithi164.stdout:Apr 26 21:48:10 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-5[43528]: 2024-04-26T21:48:10.927+0000 7fb914600700 -1 osd.5 0 waiting for initial osdmap 2024-04-26T21:48:11.061 INFO:journalctl@ceph.osd.5.smithi164.stdout:Apr 26 21:48:10 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-5[43528]: 2024-04-26T21:48:10.969+0000 7fb90a400700 -1 osd.5 38 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T21:48:11.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:48:10 smithi072 ceph-mon[28138]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:48:11.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:48:10 smithi072 ceph-mon[28138]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]': finished 2024-04-26T21:48:11.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:48:10 smithi072 ceph-mon[28138]: osdmap e37: 6 total, 5 up, 6 in 2024-04-26T21:48:11.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:48:10 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T21:48:11.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:48:10 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T21:48:11.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:48:10 smithi076 ceph-mon[31823]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:48:11.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:48:10 smithi076 ceph-mon[31823]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]': finished 2024-04-26T21:48:11.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:48:10 smithi076 ceph-mon[31823]: osdmap e37: 6 total, 5 up, 6 in 2024-04-26T21:48:11.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:48:10 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T21:48:11.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:48:10 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T21:48:12.181 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:48:11 smithi072 ceph-mon[28138]: purged_snaps scrub starts 2024-04-26T21:48:12.182 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:48:11 smithi072 ceph-mon[28138]: purged_snaps scrub ok 2024-04-26T21:48:12.182 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:48:11 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T21:48:12.182 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:48:11 smithi072 ceph-mon[28138]: osdmap e38: 6 total, 5 up, 6 in 2024-04-26T21:48:12.182 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:48:11 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T21:48:12.182 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:48:11 smithi072 ceph-mon[28138]: from='osd.5 ' entity='osd.5' 2024-04-26T21:48:12.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:48:11 smithi164 ceph-mon[31916]: purged_snaps scrub starts 2024-04-26T21:48:12.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:48:11 smithi164 ceph-mon[31916]: purged_snaps scrub ok 2024-04-26T21:48:12.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:48:11 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T21:48:12.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:48:11 smithi164 ceph-mon[31916]: osdmap e38: 6 total, 5 up, 6 in 2024-04-26T21:48:12.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:48:11 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T21:48:12.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:48:11 smithi164 ceph-mon[31916]: from='osd.5 ' entity='osd.5' 2024-04-26T21:48:12.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:48:11 smithi076 ceph-mon[31823]: purged_snaps scrub starts 2024-04-26T21:48:12.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:48:11 smithi076 ceph-mon[31823]: purged_snaps scrub ok 2024-04-26T21:48:12.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:48:11 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T21:48:12.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:48:11 smithi076 ceph-mon[31823]: osdmap e38: 6 total, 5 up, 6 in 2024-04-26T21:48:12.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:48:11 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T21:48:12.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:48:11 smithi076 ceph-mon[31823]: from='osd.5 ' entity='osd.5' 2024-04-26T21:48:13.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:48:12 smithi164 ceph-mon[31916]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:48:13.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:48:12 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T21:48:13.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:48:12 smithi164 ceph-mon[31916]: osd.5 [v2:172.21.15.164:6808/2543954772,v1:172.21.15.164:6809/2543954772] boot 2024-04-26T21:48:13.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:48:12 smithi164 ceph-mon[31916]: osdmap e39: 6 total, 6 up, 6 in 2024-04-26T21:48:13.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:48:12 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T21:48:13.218 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:48:12 smithi072 ceph-mon[28138]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:48:13.218 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:48:12 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T21:48:13.218 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:48:12 smithi072 ceph-mon[28138]: osd.5 [v2:172.21.15.164:6808/2543954772,v1:172.21.15.164:6809/2543954772] boot 2024-04-26T21:48:13.219 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:48:12 smithi072 ceph-mon[28138]: osdmap e39: 6 total, 6 up, 6 in 2024-04-26T21:48:13.219 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:48:12 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T21:48:13.219 INFO:teuthology.orchestra.run.smithi072.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-26T21:48:13.219 INFO:teuthology.orchestra.run.smithi072.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-26T21:48:13.219 INFO:teuthology.orchestra.run.smithi072.stderr: stderr: 10+0 records in 2024-04-26T21:48:13.219 INFO:teuthology.orchestra.run.smithi072.stderr:10+0 records out 2024-04-26T21:48:13.219 INFO:teuthology.orchestra.run.smithi072.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0324456 s, 323 MB/s 2024-04-26T21:48:13.219 INFO:teuthology.orchestra.run.smithi072.stderr:--> Zapping successful for: 2024-04-26T21:48:13.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:48:12 smithi076 ceph-mon[31823]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:48:13.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:48:12 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T21:48:13.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:48:12 smithi076 ceph-mon[31823]: osd.5 [v2:172.21.15.164:6808/2543954772,v1:172.21.15.164:6809/2543954772] boot 2024-04-26T21:48:13.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:48:12 smithi076 ceph-mon[31823]: osdmap e39: 6 total, 6 up, 6 in 2024-04-26T21:48:13.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:48:12 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T21:48:14.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:48:13 smithi072 ceph-mon[28138]: osdmap e40: 6 total, 6 up, 6 in 2024-04-26T21:48:14.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:48:13 smithi164 ceph-mon[31916]: osdmap e40: 6 total, 6 up, 6 in 2024-04-26T21:48:14.233 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a1780950-0415-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi072:vg_nvme/lv_2 2024-04-26T21:48:14.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:48:13 smithi076 ceph-mon[31823]: osdmap e40: 6 total, 6 up, 6 in 2024-04-26T21:48:14.667 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/a1780950-0415-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:48:15.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:48:14 smithi072 ceph-mon[28138]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:48:15.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:48:14 smithi072 ceph-mon[28138]: osdmap e41: 6 total, 6 up, 6 in 2024-04-26T21:48:15.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:48:14 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:48:15.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:48:14 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:48:15.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:48:14 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:48:15.073 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:48:14 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:48:15.073 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:48:14 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:48:15.073 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:48:14 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:48:15.073 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:48:14 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:48:15.073 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:48:14 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:48:15.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:48:14 smithi164 ceph-mon[31916]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:48:15.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:48:14 smithi164 ceph-mon[31916]: osdmap e41: 6 total, 6 up, 6 in 2024-04-26T21:48:15.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:48:14 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:48:15.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:48:14 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:48:15.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:48:14 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:48:15.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:48:14 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:48:15.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:48:14 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:48:15.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:48:14 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:48:15.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:48:14 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:48:15.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:48:14 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:48:15.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:48:14 smithi076 ceph-mon[31823]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:48:15.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:48:14 smithi076 ceph-mon[31823]: osdmap e41: 6 total, 6 up, 6 in 2024-04-26T21:48:15.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:48:14 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:48:15.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:48:14 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:48:15.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:48:14 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:48:15.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:48:14 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:48:15.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:48:14 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:48:15.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:48:14 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:48:15.316 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:48:14 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:48:15.316 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:48:14 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:48:16.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:48:15 smithi072 ceph-mon[28138]: Detected new or changed devices on smithi164 2024-04-26T21:48:16.073 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:48:15 smithi072 ceph-mon[28138]: Adjusting osd_memory_target on smithi164 to 8544M 2024-04-26T21:48:16.073 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:48:15 smithi072 ceph-mon[28138]: osdmap e42: 6 total, 6 up, 6 in 2024-04-26T21:48:16.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:48:15 smithi164 ceph-mon[31916]: Detected new or changed devices on smithi164 2024-04-26T21:48:16.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:48:15 smithi164 ceph-mon[31916]: Adjusting osd_memory_target on smithi164 to 8544M 2024-04-26T21:48:16.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:48:15 smithi164 ceph-mon[31916]: osdmap e42: 6 total, 6 up, 6 in 2024-04-26T21:48:16.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:48:15 smithi076 ceph-mon[31823]: Detected new or changed devices on smithi164 2024-04-26T21:48:16.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:48:15 smithi076 ceph-mon[31823]: Adjusting osd_memory_target on smithi164 to 8544M 2024-04-26T21:48:16.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:48:15 smithi076 ceph-mon[31823]: osdmap e42: 6 total, 6 up, 6 in 2024-04-26T21:48:17.085 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:48:17.084+0000 7efc88800700 1 -- 172.21.15.72:0/359186531 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efc80103fb0 msgr2=0x7efc801063a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:17.086 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:48:17.084+0000 7efc88800700 1 --2- 172.21.15.72:0/359186531 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efc80103fb0 0x7efc801063a0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7efc74009990 tx=0x7efc74009ca0 comp rx=0 tx=0).stop 2024-04-26T21:48:17.086 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:48:17.085+0000 7efc88800700 1 -- 172.21.15.72:0/359186531 shutdown_connections 2024-04-26T21:48:17.086 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:48:17.085+0000 7efc88800700 1 --2- 172.21.15.72:0/359186531 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efc80103fb0 0x7efc801063a0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:17.086 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:48:17.085+0000 7efc88800700 1 --2- 172.21.15.72:0/359186531 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efc80109210 0x7efc8010b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:17.086 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:48:17.085+0000 7efc88800700 1 --2- 172.21.15.72:0/359186531 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efc801068e0 0x7efc80108cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:17.086 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:48:17.085+0000 7efc88800700 1 -- 172.21.15.72:0/359186531 >> 172.21.15.72:0/359186531 conn(0x7efc800fd920 msgr2=0x7efc800ffd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:48:17.086 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:48:17.085+0000 7efc88800700 1 -- 172.21.15.72:0/359186531 shutdown_connections 2024-04-26T21:48:17.086 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:48:17.085+0000 7efc88800700 1 -- 172.21.15.72:0/359186531 wait complete. 2024-04-26T21:48:17.088 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:48:17.086+0000 7efc88800700 1 Processor -- start 2024-04-26T21:48:17.088 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:48:17.086+0000 7efc88800700 1 -- start start 2024-04-26T21:48:17.088 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:48:17.086+0000 7efc88800700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efc80103fb0 0x7efc80103910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:17.088 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:48:17.086+0000 7efc88800700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efc801068e0 0x7efc80101f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:17.088 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:48:17.086+0000 7efc88800700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efc80109210 0x7efc801024a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:17.089 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:48:17.086+0000 7efc88800700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7efc8010ee00 con 0x7efc801068e0 2024-04-26T21:48:17.089 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:48:17.086+0000 7efc88800700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7efc8010ec80 con 0x7efc80103fb0 2024-04-26T21:48:17.089 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:48:17.086+0000 7efc88800700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7efc8010ef80 con 0x7efc80109210 2024-04-26T21:48:17.089 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:48:17.086+0000 7efc85400700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efc801068e0 0x7efc80101f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:17.089 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:48:17.087+0000 7efc85400700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efc801068e0 0x7efc80101f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:41258/0 (socket says 172.21.15.72:41258) 2024-04-26T21:48:17.089 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:48:17.087+0000 7efc85400700 1 -- 172.21.15.72:0/4092882170 learned_addr learned my addr 172.21.15.72:0/4092882170 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:48:17.089 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:48:17.087+0000 7efc86800700 1 --2- 172.21.15.72:0/4092882170 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efc80109210 0x7efc801024a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:17.089 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:48:17.087+0000 7efc85e00700 1 --2- 172.21.15.72:0/4092882170 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efc80103fb0 0x7efc80103910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:17.089 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:48:17.087+0000 7efc85400700 1 -- 172.21.15.72:0/4092882170 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efc80109210 msgr2=0x7efc801024a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:17.089 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:48:17.087+0000 7efc85400700 1 --2- 172.21.15.72:0/4092882170 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efc80109210 0x7efc801024a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:17.089 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:48:17.087+0000 7efc85400700 1 -- 172.21.15.72:0/4092882170 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efc80103fb0 msgr2=0x7efc80103910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:17.089 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:48:17.087+0000 7efc85400700 1 --2- 172.21.15.72:0/4092882170 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efc80103fb0 0x7efc80103910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:17.089 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:48:17.087+0000 7efc85400700 1 -- 172.21.15.72:0/4092882170 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc80102d50 con 0x7efc801068e0 2024-04-26T21:48:17.089 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:48:17.087+0000 7efc85400700 1 --2- 172.21.15.72:0/4092882170 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efc801068e0 0x7efc80101f60 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7efc7000d980 tx=0x7efc7000dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:48:17.090 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:48:17.088+0000 7efc7aa00700 1 -- 172.21.15.72:0/4092882170 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc70017070 con 0x7efc801068e0 2024-04-26T21:48:17.090 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:48:17.088+0000 7efc7aa00700 1 -- 172.21.15.72:0/4092882170 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efc70004830 con 0x7efc801068e0 2024-04-26T21:48:17.090 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:48:17.088+0000 7efc88800700 1 -- 172.21.15.72:0/4092882170 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc801a3700 con 0x7efc801068e0 2024-04-26T21:48:17.090 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:48:17.088+0000 7efc7aa00700 1 -- 172.21.15.72:0/4092882170 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc700053a0 con 0x7efc801068e0 2024-04-26T21:48:17.090 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:48:17.088+0000 7efc88800700 1 -- 172.21.15.72:0/4092882170 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc801a3c10 con 0x7efc801068e0 2024-04-26T21:48:17.090 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:48:17.089+0000 7efc88800700 1 -- 172.21.15.72:0/4092882170 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc8006a110 con 0x7efc801068e0 2024-04-26T21:48:17.092 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:48:17.092+0000 7efc7aa00700 1 -- 172.21.15.72:0/4092882170 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7efc700049e0 con 0x7efc801068e0 2024-04-26T21:48:17.093 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:48:17.093+0000 7efc7aa00700 1 --2- 172.21.15.72:0/4092882170 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7efc680a3cc0 0x7efc680a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:17.093 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:48:17.093+0000 7efc7aa00700 1 -- 172.21.15.72:0/4092882170 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 4533+0+0 (secure 0 0 0) 0x7efc700cf4e0 con 0x7efc801068e0 2024-04-26T21:48:17.094 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:48:17.093+0000 7efc85e00700 1 --2- 172.21.15.72:0/4092882170 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7efc680a3cc0 0x7efc680a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:17.094 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:48:17.094+0000 7efc85e00700 1 --2- 172.21.15.72:0/4092882170 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7efc680a3cc0 0x7efc680a6180 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7efc74006010 tx=0x7efc740037e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:48:17.098 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:48:17.098+0000 7efc7aa00700 1 -- 172.21.15.72:0/4092882170 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7efc70020020 con 0x7efc801068e0 2024-04-26T21:48:17.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:48:16 smithi164 ceph-mon[31916]: pgmap v185: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-26T21:48:17.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:48:16 smithi164 ceph-mon[31916]: osdmap e43: 6 total, 6 up, 6 in 2024-04-26T21:48:17.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:48:16 smithi076 ceph-mon[31823]: pgmap v185: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-26T21:48:17.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:48:16 smithi076 ceph-mon[31823]: osdmap e43: 6 total, 6 up, 6 in 2024-04-26T21:48:17.321 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:48:16 smithi072 ceph-mon[28138]: pgmap v185: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-26T21:48:17.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:48:16 smithi072 ceph-mon[28138]: osdmap e43: 6 total, 6 up, 6 in 2024-04-26T21:48:17.339 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:48:17.339+0000 7efc88800700 1 -- 172.21.15.72:0/4092882170 --> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi072:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7efc80063860 con 0x7efc680a3cc0 2024-04-26T21:48:18.286 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:48:17 smithi072 ceph-mon[28138]: from='client.14361 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi072:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:48:18.286 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:48:17 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:48:18.287 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:48:17 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:48:18.287 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:48:17 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:48:18.287 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:48:17 smithi072 ceph-mon[28138]: pgmap v187: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail; 121 KiB/s, 0 objects/s recovering 2024-04-26T21:48:18.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:48:17 smithi076 ceph-mon[31823]: from='client.14361 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi072:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:48:18.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:48:17 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:48:18.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:48:17 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:48:18.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:48:17 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:48:18.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:48:17 smithi076 ceph-mon[31823]: pgmap v187: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail; 121 KiB/s, 0 objects/s recovering 2024-04-26T21:48:18.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:48:17 smithi164 ceph-mon[31916]: from='client.14361 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi072:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:48:18.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:48:17 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:48:18.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:48:17 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:48:18.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:48:17 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:48:18.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:48:17 smithi164 ceph-mon[31916]: pgmap v187: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail; 121 KiB/s, 0 objects/s recovering 2024-04-26T21:48:21.064 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:48:20 smithi076 ceph-mon[31823]: pgmap v188: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-26T21:48:21.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:48:20 smithi072 ceph-mon[28138]: pgmap v188: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-26T21:48:21.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:48:20 smithi164 ceph-mon[31916]: pgmap v188: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-26T21:48:22.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:48:21 smithi076 ceph-mon[31823]: from='client.? 172.21.15.72:0/4075633374' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d5d56650-be7e-4ba5-ba3a-d7d473294dab"}]: dispatch 2024-04-26T21:48:22.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:48:21 smithi076 ceph-mon[31823]: from='client.? 172.21.15.72:0/4075633374' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d5d56650-be7e-4ba5-ba3a-d7d473294dab"}]': finished 2024-04-26T21:48:22.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:48:21 smithi076 ceph-mon[31823]: osdmap e44: 7 total, 6 up, 7 in 2024-04-26T21:48:22.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:48:21 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T21:48:22.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:48:21 smithi072 ceph-mon[28138]: from='client.? 172.21.15.72:0/4075633374' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d5d56650-be7e-4ba5-ba3a-d7d473294dab"}]: dispatch 2024-04-26T21:48:22.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:48:21 smithi072 ceph-mon[28138]: from='client.? 172.21.15.72:0/4075633374' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d5d56650-be7e-4ba5-ba3a-d7d473294dab"}]': finished 2024-04-26T21:48:22.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:48:21 smithi072 ceph-mon[28138]: osdmap e44: 7 total, 6 up, 7 in 2024-04-26T21:48:22.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:48:21 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T21:48:22.080 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:48:21 smithi164 ceph-mon[31916]: from='client.? 172.21.15.72:0/4075633374' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d5d56650-be7e-4ba5-ba3a-d7d473294dab"}]: dispatch 2024-04-26T21:48:22.080 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:48:21 smithi164 ceph-mon[31916]: from='client.? 172.21.15.72:0/4075633374' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d5d56650-be7e-4ba5-ba3a-d7d473294dab"}]': finished 2024-04-26T21:48:22.080 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:48:21 smithi164 ceph-mon[31916]: osdmap e44: 7 total, 6 up, 7 in 2024-04-26T21:48:22.080 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:48:21 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T21:48:23.064 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:48:22 smithi076 ceph-mon[31823]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:48:23.064 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:48:22 smithi076 ceph-mon[31823]: from='client.? 172.21.15.72:0/1941434916' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:48:23.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:48:22 smithi072 ceph-mon[28138]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:48:23.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:48:22 smithi072 ceph-mon[28138]: from='client.? 172.21.15.72:0/1941434916' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:48:23.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:48:22 smithi164 ceph-mon[31916]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:48:23.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:48:22 smithi164 ceph-mon[31916]: from='client.? 172.21.15.72:0/1941434916' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:48:25.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:48:24 smithi076 ceph-mon[31823]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:48:25.071 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:48:24 smithi072 ceph-mon[28138]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:48:25.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:48:24 smithi164 ceph-mon[31916]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:48:26.822 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:48:26 smithi072 ceph-mon[28138]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:48:27.064 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:48:26 smithi076 ceph-mon[31823]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:48:27.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:48:26 smithi164 ceph-mon[31916]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:48:28.940 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:48:28 smithi164 ceph-mon[31916]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:48:29.064 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:48:28 smithi076 ceph-mon[31823]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:48:29.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:48:28 smithi072 ceph-mon[28138]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:48:31.064 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:48:30 smithi076 ceph-mon[31823]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:48:31.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:48:30 smithi072 ceph-mon[28138]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:48:31.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:48:30 smithi164 ceph-mon[31916]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:48:32.064 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:48:31 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-26T21:48:32.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:48:31 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:48:32.071 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:48:31 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-26T21:48:32.071 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:48:31 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:48:32.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:48:31 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-26T21:48:32.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:48:31 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:48:33.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:48:32 smithi072 ceph-mon[28138]: Deploying daemon osd.6 on smithi072 2024-04-26T21:48:33.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:48:32 smithi072 ceph-mon[28138]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:48:33.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:48:32 smithi164 ceph-mon[31916]: Deploying daemon osd.6 on smithi072 2024-04-26T21:48:33.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:48:32 smithi164 ceph-mon[31916]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:48:33.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:48:32 smithi076 ceph-mon[31823]: Deploying daemon osd.6 on smithi072 2024-04-26T21:48:33.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:48:32 smithi076 ceph-mon[31823]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:48:35.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:48:34 smithi072 ceph-mon[28138]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:48:35.117 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:48:34 smithi076 ceph-mon[31823]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:48:35.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:48:34 smithi164 ceph-mon[31916]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:48:37.060 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:48:36 smithi072 ceph-mon[28138]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:48:37.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:48:36 smithi164 ceph-mon[31916]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:48:37.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:48:36 smithi076 ceph-mon[31823]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:48:39.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:48:38 smithi072 ceph-mon[28138]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:48:39.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:48:38 smithi164 ceph-mon[31916]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:48:39.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:48:38 smithi076 ceph-mon[31823]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:48:41.180 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:48:40 smithi072 ceph-mon[28138]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:48:41.180 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:48:40 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:48:41.180 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:48:40 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:48:41.180 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:48:40 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:48:41.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:48:40 smithi164 ceph-mon[31916]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:48:41.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:48:40 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:48:41.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:48:40 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:48:41.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:48:40 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:48:41.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:48:40 smithi076 ceph-mon[31823]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:48:41.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:48:40 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:48:41.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:48:40 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:48:41.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:48:40 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:48:43.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:48:42 smithi164 ceph-mon[31916]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:48:43.214 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:48:42 smithi072 ceph-mon[28138]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:48:43.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:48:42 smithi076 ceph-mon[31823]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:48:45.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:48:44 smithi076 ceph-mon[31823]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:48:45.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:48:44 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:48:45.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:48:44 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:48:45.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:48:44 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:48:45.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:48:44 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:48:45.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:48:44 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:48:45.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:48:44 smithi076 ceph-mon[31823]: from='osd.6 [v2:172.21.15.72:6818/2080659027,v1:172.21.15.72:6819/2080659027]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-26T21:48:45.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:48:44 smithi072 ceph-mon[28138]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:48:45.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:48:44 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:48:45.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:48:44 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:48:45.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:48:44 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:48:45.073 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:48:44 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:48:45.073 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:48:44 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:48:45.073 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:48:44 smithi072 ceph-mon[28138]: from='osd.6 [v2:172.21.15.72:6818/2080659027,v1:172.21.15.72:6819/2080659027]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-26T21:48:45.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:48:44 smithi164 ceph-mon[31916]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:48:45.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:48:44 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:48:45.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:48:44 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:48:45.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:48:44 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:48:45.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:48:44 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:48:45.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:48:44 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:48:45.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:48:44 smithi164 ceph-mon[31916]: from='osd.6 [v2:172.21.15.72:6818/2080659027,v1:172.21.15.72:6819/2080659027]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-26T21:48:45.257 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:48:45.256+0000 7efc7aa00700 1 -- 172.21.15.72:0/4092882170 <== mgr.14152 v2:172.21.15.72:6800/588150172 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7efc80063860 con 0x7efc680a3cc0 2024-04-26T21:48:45.258 INFO:teuthology.orchestra.run.smithi072.stdout:Created osd(s) 6 on host 'smithi072' 2024-04-26T21:48:45.261 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:48:45.259+0000 7efc88800700 1 -- 172.21.15.72:0/4092882170 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7efc680a3cc0 msgr2=0x7efc680a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:45.261 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:48:45.259+0000 7efc88800700 1 --2- 172.21.15.72:0/4092882170 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7efc680a3cc0 0x7efc680a6180 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7efc74006010 tx=0x7efc740037e0 comp rx=0 tx=0).stop 2024-04-26T21:48:45.261 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:48:45.259+0000 7efc88800700 1 -- 172.21.15.72:0/4092882170 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efc801068e0 msgr2=0x7efc80101f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:45.261 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:48:45.259+0000 7efc88800700 1 --2- 172.21.15.72:0/4092882170 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efc801068e0 0x7efc80101f60 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7efc7000d980 tx=0x7efc7000dd40 comp rx=0 tx=0).stop 2024-04-26T21:48:45.261 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:48:45.260+0000 7efc88800700 1 -- 172.21.15.72:0/4092882170 shutdown_connections 2024-04-26T21:48:45.261 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:48:45.260+0000 7efc88800700 1 --2- 172.21.15.72:0/4092882170 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7efc680a3cc0 0x7efc680a6180 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:45.262 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:48:45.260+0000 7efc88800700 1 --2- 172.21.15.72:0/4092882170 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efc80109210 0x7efc801024a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:45.262 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:48:45.260+0000 7efc88800700 1 --2- 172.21.15.72:0/4092882170 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efc80103fb0 0x7efc80103910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:45.262 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:48:45.260+0000 7efc88800700 1 --2- 172.21.15.72:0/4092882170 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efc801068e0 0x7efc80101f60 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:45.262 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:48:45.260+0000 7efc88800700 1 -- 172.21.15.72:0/4092882170 >> 172.21.15.72:0/4092882170 conn(0x7efc800fd920 msgr2=0x7efc80104970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:48:45.262 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:48:45.260+0000 7efc88800700 1 -- 172.21.15.72:0/4092882170 shutdown_connections 2024-04-26T21:48:45.262 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:48:45.260+0000 7efc88800700 1 -- 172.21.15.72:0/4092882170 wait complete. 2024-04-26T21:48:45.979 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:48:45 smithi072 ceph-mon[28138]: from='osd.6 [v2:172.21.15.72:6818/2080659027,v1:172.21.15.72:6819/2080659027]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-26T21:48:45.979 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:48:45 smithi072 ceph-mon[28138]: osdmap e45: 7 total, 6 up, 7 in 2024-04-26T21:48:45.979 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:48:45 smithi072 ceph-mon[28138]: from='osd.6 [v2:172.21.15.72:6818/2080659027,v1:172.21.15.72:6819/2080659027]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi072", "root=default"]}]: dispatch 2024-04-26T21:48:45.979 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:48:45 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T21:48:45.979 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:48:45 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:48:45.979 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:48:45 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:48:45.979 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:48:45 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:48:46.027 DEBUG:teuthology.orchestra.run.smithi072:osd.6> sudo journalctl -f -n 0 -u ceph-a1780950-0415-11ef-bc93-c7b262605968@osd.6.service 2024-04-26T21:48:46.031 INFO:tasks.cephadm:Deploying osd.7 on smithi076 with /dev/vg_nvme/lv_2... 2024-04-26T21:48:46.031 DEBUG:teuthology.orchestra.run.smithi076:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a1780950-0415-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-26T21:48:46.114 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:48:45 smithi076 ceph-mon[31823]: from='osd.6 [v2:172.21.15.72:6818/2080659027,v1:172.21.15.72:6819/2080659027]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-26T21:48:46.115 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:48:45 smithi076 ceph-mon[31823]: osdmap e45: 7 total, 6 up, 7 in 2024-04-26T21:48:46.115 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:48:45 smithi076 ceph-mon[31823]: from='osd.6 [v2:172.21.15.72:6818/2080659027,v1:172.21.15.72:6819/2080659027]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi072", "root=default"]}]: dispatch 2024-04-26T21:48:46.115 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:48:45 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T21:48:46.115 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:48:45 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:48:46.115 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:48:45 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:48:46.115 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:48:45 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:48:46.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:48:45 smithi164 ceph-mon[31916]: from='osd.6 [v2:172.21.15.72:6818/2080659027,v1:172.21.15.72:6819/2080659027]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-26T21:48:46.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:48:45 smithi164 ceph-mon[31916]: osdmap e45: 7 total, 6 up, 7 in 2024-04-26T21:48:46.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:48:45 smithi164 ceph-mon[31916]: from='osd.6 [v2:172.21.15.72:6818/2080659027,v1:172.21.15.72:6819/2080659027]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi072", "root=default"]}]: dispatch 2024-04-26T21:48:46.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:48:45 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T21:48:46.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:48:45 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:48:46.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:48:45 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:48:46.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:48:45 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:48:46.470 INFO:teuthology.orchestra.run.smithi076.stderr:Inferring config /var/lib/ceph/a1780950-0415-11ef-bc93-c7b262605968/mon.b/config 2024-04-26T21:48:46.939 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:48:46 smithi072 ceph-mon[28138]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:48:46.939 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:48:46 smithi072 ceph-mon[28138]: from='osd.6 [v2:172.21.15.72:6818/2080659027,v1:172.21.15.72:6819/2080659027]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi072", "root=default"]}]': finished 2024-04-26T21:48:46.939 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:48:46 smithi072 ceph-mon[28138]: osdmap e46: 7 total, 6 up, 7 in 2024-04-26T21:48:46.939 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:48:46 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T21:48:46.939 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:48:46 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T21:48:46.939 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:48:46 smithi072 ceph-mon[28138]: from='osd.6 [v2:172.21.15.72:6818/2080659027,v1:172.21.15.72:6819/2080659027]' entity='osd.6' 2024-04-26T21:48:46.939 INFO:journalctl@ceph.osd.6.smithi072.stdout:Apr 26 21:48:46 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-6[57091]: 2024-04-26T21:48:46.851+0000 7fb15d600700 -1 osd.6 0 waiting for initial osdmap 2024-04-26T21:48:46.939 INFO:journalctl@ceph.osd.6.smithi072.stdout:Apr 26 21:48:46 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-6[57091]: 2024-04-26T21:48:46.937+0000 7fb152a00700 -1 osd.6 46 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T21:48:47.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:48:46 smithi164 ceph-mon[31916]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:48:47.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:48:46 smithi164 ceph-mon[31916]: from='osd.6 [v2:172.21.15.72:6818/2080659027,v1:172.21.15.72:6819/2080659027]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi072", "root=default"]}]': finished 2024-04-26T21:48:47.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:48:46 smithi164 ceph-mon[31916]: osdmap e46: 7 total, 6 up, 7 in 2024-04-26T21:48:47.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:48:46 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T21:48:47.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:48:46 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T21:48:47.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:48:46 smithi164 ceph-mon[31916]: from='osd.6 [v2:172.21.15.72:6818/2080659027,v1:172.21.15.72:6819/2080659027]' entity='osd.6' 2024-04-26T21:48:47.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:48:46 smithi076 ceph-mon[31823]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:48:47.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:48:46 smithi076 ceph-mon[31823]: from='osd.6 [v2:172.21.15.72:6818/2080659027,v1:172.21.15.72:6819/2080659027]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi072", "root=default"]}]': finished 2024-04-26T21:48:47.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:48:46 smithi076 ceph-mon[31823]: osdmap e46: 7 total, 6 up, 7 in 2024-04-26T21:48:47.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:48:46 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T21:48:47.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:48:46 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T21:48:47.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:48:46 smithi076 ceph-mon[31823]: from='osd.6 [v2:172.21.15.72:6818/2080659027,v1:172.21.15.72:6819/2080659027]' entity='osd.6' 2024-04-26T21:48:48.052 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:48:47 smithi076 ceph-mon[31823]: purged_snaps scrub starts 2024-04-26T21:48:48.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:48:47 smithi076 ceph-mon[31823]: purged_snaps scrub ok 2024-04-26T21:48:48.053 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:48:47 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T21:48:48.098 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:48:47 smithi072 ceph-mon[28138]: purged_snaps scrub starts 2024-04-26T21:48:48.098 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:48:47 smithi072 ceph-mon[28138]: purged_snaps scrub ok 2024-04-26T21:48:48.098 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:48:47 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T21:48:48.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:48:47 smithi164 ceph-mon[31916]: purged_snaps scrub starts 2024-04-26T21:48:48.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:48:47 smithi164 ceph-mon[31916]: purged_snaps scrub ok 2024-04-26T21:48:48.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:48:47 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T21:48:49.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:48:48 smithi164 ceph-mon[31916]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:48:49.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:48:48 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T21:48:49.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:48:48 smithi164 ceph-mon[31916]: osd.6 [v2:172.21.15.72:6818/2080659027,v1:172.21.15.72:6819/2080659027] boot 2024-04-26T21:48:49.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:48:48 smithi164 ceph-mon[31916]: osdmap e47: 7 total, 7 up, 7 in 2024-04-26T21:48:49.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:48:48 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T21:48:49.235 INFO:teuthology.orchestra.run.smithi076.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-26T21:48:49.235 INFO:teuthology.orchestra.run.smithi076.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-26T21:48:49.235 INFO:teuthology.orchestra.run.smithi076.stderr: stderr: 10+0 records in 2024-04-26T21:48:49.235 INFO:teuthology.orchestra.run.smithi076.stderr:10+0 records out 2024-04-26T21:48:49.235 INFO:teuthology.orchestra.run.smithi076.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0354613 s, 296 MB/s 2024-04-26T21:48:49.236 INFO:teuthology.orchestra.run.smithi076.stderr:--> Zapping successful for: 2024-04-26T21:48:49.236 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:48:48 smithi076 ceph-mon[31823]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:48:49.236 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:48:48 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T21:48:49.236 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:48:48 smithi076 ceph-mon[31823]: osd.6 [v2:172.21.15.72:6818/2080659027,v1:172.21.15.72:6819/2080659027] boot 2024-04-26T21:48:49.236 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:48:48 smithi076 ceph-mon[31823]: osdmap e47: 7 total, 7 up, 7 in 2024-04-26T21:48:49.236 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:48:48 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T21:48:49.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:48:48 smithi072 ceph-mon[28138]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:48:49.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:48:48 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T21:48:49.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:48:48 smithi072 ceph-mon[28138]: osd.6 [v2:172.21.15.72:6818/2080659027,v1:172.21.15.72:6819/2080659027] boot 2024-04-26T21:48:49.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:48:48 smithi072 ceph-mon[28138]: osdmap e47: 7 total, 7 up, 7 in 2024-04-26T21:48:49.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:48:48 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T21:48:50.109 DEBUG:teuthology.orchestra.run.smithi076:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a1780950-0415-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi076:vg_nvme/lv_2 2024-04-26T21:48:50.185 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:48:49 smithi076 ceph-mon[31823]: osdmap e48: 7 total, 7 up, 7 in 2024-04-26T21:48:50.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:48:49 smithi164 ceph-mon[31916]: osdmap e48: 7 total, 7 up, 7 in 2024-04-26T21:48:50.241 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:48:49 smithi072 ceph-mon[28138]: osdmap e48: 7 total, 7 up, 7 in 2024-04-26T21:48:50.548 INFO:teuthology.orchestra.run.smithi076.stderr:Inferring config /var/lib/ceph/a1780950-0415-11ef-bc93-c7b262605968/mon.b/config 2024-04-26T21:48:50.981 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:48:50 smithi076 ceph-mon[31823]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:48:50.981 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:48:50 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:48:50.981 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:48:50 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:48:50.981 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:48:50 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:48:50.981 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:48:50 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:48:50.981 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:48:50 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:48:50.981 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:48:50 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:48:50.981 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:48:50 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:48:50.981 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:48:50 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:48:50.981 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:48:50 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:48:51.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:48:50 smithi072 ceph-mon[28138]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:48:51.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:48:50 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:48:51.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:48:50 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:48:51.073 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:48:50 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:48:51.073 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:48:50 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:48:51.073 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:48:50 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:48:51.073 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:48:50 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:48:51.073 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:48:50 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:48:51.073 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:48:50 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:48:51.073 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:48:50 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:48:51.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:48:50 smithi164 ceph-mon[31916]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:48:51.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:48:50 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:48:51.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:48:50 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:48:51.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:48:50 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:48:51.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:48:50 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:48:51.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:48:50 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:48:51.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:48:50 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:48:51.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:48:50 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:48:51.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:48:50 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:48:51.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:48:50 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:48:52.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:48:51 smithi164 ceph-mon[31916]: Detected new or changed devices on smithi072 2024-04-26T21:48:52.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:48:51 smithi164 ceph-mon[31916]: Adjusting osd_memory_target on smithi072 to 5696M 2024-04-26T21:48:52.251 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:48:51 smithi076 ceph-mon[31823]: Detected new or changed devices on smithi072 2024-04-26T21:48:52.251 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:48:51 smithi076 ceph-mon[31823]: Adjusting osd_memory_target on smithi072 to 5696M 2024-04-26T21:48:52.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:48:51 smithi072 ceph-mon[28138]: Detected new or changed devices on smithi072 2024-04-26T21:48:52.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:48:51 smithi072 ceph-mon[28138]: Adjusting osd_memory_target on smithi072 to 5696M 2024-04-26T21:48:52.732 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:48:52.729+0000 7f3550400700 1 -- 172.21.15.76:0/3815812651 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3548109ac0 msgr2=0x7f3548109ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:52.732 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:48:52.729+0000 7f3550400700 1 --2- 172.21.15.76:0/3815812651 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3548109ac0 0x7f3548109ee0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f353c0099c0 tx=0x7f353c009cd0 comp rx=0 tx=0).stop 2024-04-26T21:48:52.732 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:48:52.730+0000 7f3550400700 1 -- 172.21.15.76:0/3815812651 shutdown_connections 2024-04-26T21:48:52.732 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:48:52.730+0000 7f3550400700 1 --2- 172.21.15.76:0/3815812651 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f354810b660 0x7f3548111f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:52.732 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:48:52.730+0000 7f3550400700 1 --2- 172.21.15.76:0/3815812651 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3548109ac0 0x7f3548109ee0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:52.732 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:48:52.730+0000 7f3550400700 1 --2- 172.21.15.76:0/3815812651 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f354810acc0 0x7f354810b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:52.732 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:48:52.730+0000 7f3550400700 1 -- 172.21.15.76:0/3815812651 >> 172.21.15.76:0/3815812651 conn(0x7f3548078a90 msgr2=0x7f354807aed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:48:52.732 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:48:52.730+0000 7f3550400700 1 -- 172.21.15.76:0/3815812651 shutdown_connections 2024-04-26T21:48:52.732 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:48:52.730+0000 7f3550400700 1 -- 172.21.15.76:0/3815812651 wait complete. 2024-04-26T21:48:52.733 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:48:52.731+0000 7f3550400700 1 Processor -- start 2024-04-26T21:48:52.733 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:48:52.731+0000 7f3550400700 1 -- start start 2024-04-26T21:48:52.733 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:48:52.732+0000 7f3550400700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3548109ac0 0x7f354819fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:52.733 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:48:52.732+0000 7f3550400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f354810acc0 0x7f35481a0090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:52.733 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:48:52.732+0000 7f3550400700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f354810b660 0x7f35481a7110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:52.733 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:48:52.732+0000 7f3550400700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f3548115450 con 0x7f354810b660 2024-04-26T21:48:52.733 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:48:52.732+0000 7f3550400700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f35481152d0 con 0x7f3548109ac0 2024-04-26T21:48:52.733 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:48:52.732+0000 7f3550400700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f35481155d0 con 0x7f354810acc0 2024-04-26T21:48:52.735 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:48:52.732+0000 7f354da00700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3548109ac0 0x7f354819fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:52.735 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:48:52.732+0000 7f354da00700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3548109ac0 0x7f354819fb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:36174/0 (socket says 172.21.15.76:36174) 2024-04-26T21:48:52.735 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:48:52.732+0000 7f354da00700 1 -- 172.21.15.76:0/2803543395 learned_addr learned my addr 172.21.15.76:0/2803543395 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-26T21:48:52.735 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:48:52.733+0000 7f354d000700 1 --2- 172.21.15.76:0/2803543395 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f354810acc0 0x7f35481a0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:52.735 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:48:52.733+0000 7f354e400700 1 --2- 172.21.15.76:0/2803543395 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f354810b660 0x7f35481a7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:52.735 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:48:52.733+0000 7f354da00700 1 -- 172.21.15.76:0/2803543395 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f354810acc0 msgr2=0x7f35481a0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:52.735 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:48:52.733+0000 7f354da00700 1 --2- 172.21.15.76:0/2803543395 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f354810acc0 0x7f35481a0090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:52.735 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:48:52.733+0000 7f354da00700 1 -- 172.21.15.76:0/2803543395 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f354810b660 msgr2=0x7f35481a7110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:52.736 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:48:52.733+0000 7f354da00700 1 --2- 172.21.15.76:0/2803543395 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f354810b660 0x7f35481a7110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:52.736 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:48:52.733+0000 7f354da00700 1 -- 172.21.15.76:0/2803543395 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35481a77b0 con 0x7f3548109ac0 2024-04-26T21:48:52.736 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:48:52.733+0000 7f354da00700 1 --2- 172.21.15.76:0/2803543395 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3548109ac0 0x7f354819fb50 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f353c0098e0 tx=0x7f353c00f740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:48:52.736 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:48:52.733+0000 7f3546000700 1 -- 172.21.15.76:0/2803543395 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f353c027070 con 0x7f3548109ac0 2024-04-26T21:48:52.736 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:48:52.733+0000 7f3546000700 1 -- 172.21.15.76:0/2803543395 <== mon.2 v2:172.21.15.76:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f353c00bd10 con 0x7f3548109ac0 2024-04-26T21:48:52.736 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:48:52.733+0000 7f3550400700 1 -- 172.21.15.76:0/2803543395 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35481a7a40 con 0x7f3548109ac0 2024-04-26T21:48:52.736 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:48:52.734+0000 7f3546000700 1 -- 172.21.15.76:0/2803543395 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f353c0037f0 con 0x7f3548109ac0 2024-04-26T21:48:52.736 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:48:52.734+0000 7f3550400700 1 -- 172.21.15.76:0/2803543395 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35481a7e90 con 0x7f3548109ac0 2024-04-26T21:48:52.736 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:48:52.735+0000 7f3550400700 1 -- 172.21.15.76:0/2803543395 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f354806a110 con 0x7f3548109ac0 2024-04-26T21:48:52.738 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:48:52.737+0000 7f3546000700 1 -- 172.21.15.76:0/2803543395 <== mon.2 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f353c02a070 con 0x7f3548109ac0 2024-04-26T21:48:52.739 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:48:52.738+0000 7f3546000700 1 --2- 172.21.15.76:0/2803543395 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f35300a3c40 0x7f35300a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:52.739 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:48:52.738+0000 7f3546000700 1 -- 172.21.15.76:0/2803543395 <== mon.2 v2:172.21.15.76:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 4965+0+0 (secure 0 0 0) 0x7f353c0d0780 con 0x7f3548109ac0 2024-04-26T21:48:52.740 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:48:52.739+0000 7f354d000700 1 --2- 172.21.15.76:0/2803543395 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f35300a3c40 0x7f35300a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:52.741 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:48:52.740+0000 7f354d000700 1 --2- 172.21.15.76:0/2803543395 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f35300a3c40 0x7f35300a6100 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f3534005950 tx=0x7f35340058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:48:52.744 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:48:52.744+0000 7f3546000700 1 -- 172.21.15.76:0/2803543395 <== mon.2 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f353c05f060 con 0x7f3548109ac0 2024-04-26T21:48:52.983 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:48:52.981+0000 7f3550400700 1 -- 172.21.15.76:0/2803543395 --> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi076:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f3548063860 con 0x7f35300a3c40 2024-04-26T21:48:53.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:48:52 smithi164 ceph-mon[31916]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:48:53.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:48:52 smithi076 ceph-mon[31823]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:48:53.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:48:52 smithi072 ceph-mon[28138]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:48:54.246 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:48:53 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:48:54.246 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:48:53 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:48:54.246 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:48:53 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:48:54.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:48:53 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:48:54.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:48:53 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:48:54.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:48:53 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:48:54.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:48:53 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:48:54.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:48:53 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:48:54.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:48:53 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:48:55.064 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:48:54 smithi076 ceph-mon[31823]: from='client.34265 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi076:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:48:55.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:48:54 smithi076 ceph-mon[31823]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:48:55.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:48:54 smithi072 ceph-mon[28138]: from='client.34265 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi076:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:48:55.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:48:54 smithi072 ceph-mon[28138]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:48:55.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:48:54 smithi164 ceph-mon[31916]: from='client.34265 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi076:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:48:55.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:48:54 smithi164 ceph-mon[31916]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:48:56.064 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:48:55 smithi076 ceph-mon[31823]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:48:56.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:48:55 smithi072 ceph-mon[28138]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:48:56.342 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:48:55 smithi164 ceph-mon[31916]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:48:57.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:48:57 smithi076 ceph-mon[31823]: from='client.? 172.21.15.76:0/1666440043' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "18975b5a-9309-4a0f-9f37-3ce09c2e745d"}]: dispatch 2024-04-26T21:48:57.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:48:57 smithi076 ceph-mon[31823]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "18975b5a-9309-4a0f-9f37-3ce09c2e745d"}]: dispatch 2024-04-26T21:48:57.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:48:57 smithi076 ceph-mon[31823]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "18975b5a-9309-4a0f-9f37-3ce09c2e745d"}]': finished 2024-04-26T21:48:57.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:48:57 smithi076 ceph-mon[31823]: osdmap e49: 8 total, 7 up, 8 in 2024-04-26T21:48:57.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:48:57 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T21:48:57.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:48:57 smithi072 ceph-mon[28138]: from='client.? 172.21.15.76:0/1666440043' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "18975b5a-9309-4a0f-9f37-3ce09c2e745d"}]: dispatch 2024-04-26T21:48:57.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:48:57 smithi072 ceph-mon[28138]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "18975b5a-9309-4a0f-9f37-3ce09c2e745d"}]: dispatch 2024-04-26T21:48:57.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:48:57 smithi072 ceph-mon[28138]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "18975b5a-9309-4a0f-9f37-3ce09c2e745d"}]': finished 2024-04-26T21:48:57.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:48:57 smithi072 ceph-mon[28138]: osdmap e49: 8 total, 7 up, 8 in 2024-04-26T21:48:57.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:48:57 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T21:48:57.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:48:57 smithi164 ceph-mon[31916]: from='client.? 172.21.15.76:0/1666440043' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "18975b5a-9309-4a0f-9f37-3ce09c2e745d"}]: dispatch 2024-04-26T21:48:57.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:48:57 smithi164 ceph-mon[31916]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "18975b5a-9309-4a0f-9f37-3ce09c2e745d"}]: dispatch 2024-04-26T21:48:57.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:48:57 smithi164 ceph-mon[31916]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "18975b5a-9309-4a0f-9f37-3ce09c2e745d"}]': finished 2024-04-26T21:48:57.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:48:57 smithi164 ceph-mon[31916]: osdmap e49: 8 total, 7 up, 8 in 2024-04-26T21:48:57.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:48:57 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T21:48:58.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:48:58 smithi164 ceph-mon[31916]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:48:58.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:48:58 smithi164 ceph-mon[31916]: from='client.? 172.21.15.76:0/1943370355' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:48:58.564 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:48:58 smithi076 ceph-mon[31823]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:48:58.564 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:48:58 smithi076 ceph-mon[31823]: from='client.? 172.21.15.76:0/1943370355' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:48:58.572 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:48:58 smithi072 ceph-mon[28138]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:48:58.572 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:48:58 smithi072 ceph-mon[28138]: from='client.? 172.21.15.76:0/1943370355' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:49:01.064 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:49:00 smithi076 ceph-mon[31823]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:49:01.071 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:49:00 smithi072 ceph-mon[28138]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:49:01.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:49:00 smithi164 ceph-mon[31916]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:49:03.064 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:49:02 smithi076 ceph-mon[31823]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:49:03.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:49:02 smithi072 ceph-mon[28138]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:49:03.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:49:02 smithi164 ceph-mon[31916]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:49:05.064 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:49:04 smithi076 ceph-mon[31823]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:49:05.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:49:04 smithi072 ceph-mon[28138]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:49:05.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:49:04 smithi164 ceph-mon[31916]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:49:07.064 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:49:06 smithi076 ceph-mon[31823]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:49:07.064 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:49:06 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-26T21:49:07.064 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:49:06 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:49:07.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:49:06 smithi072 ceph-mon[28138]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:49:07.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:49:06 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-26T21:49:07.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:49:06 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:49:07.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:49:06 smithi164 ceph-mon[31916]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:49:07.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:49:06 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-26T21:49:07.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:49:06 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:49:07.910 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:49:07 smithi076 ceph-mon[31823]: Deploying daemon osd.7 on smithi076 2024-04-26T21:49:08.071 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:49:07 smithi072 ceph-mon[28138]: Deploying daemon osd.7 on smithi076 2024-04-26T21:49:08.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:49:07 smithi164 ceph-mon[31916]: Deploying daemon osd.7 on smithi076 2024-04-26T21:49:09.064 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:49:08 smithi076 ceph-mon[31823]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:49:09.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:49:08 smithi072 ceph-mon[28138]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:49:09.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:49:08 smithi164 ceph-mon[31916]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:49:11.048 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:49:10 smithi076 ceph-mon[31823]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:49:11.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:49:10 smithi072 ceph-mon[28138]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:49:11.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:49:10 smithi164 ceph-mon[31916]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:49:13.064 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:49:12 smithi076 ceph-mon[31823]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:49:13.071 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:49:12 smithi072 ceph-mon[28138]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:49:13.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:49:12 smithi164 ceph-mon[31916]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:49:15.064 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:49:14 smithi076 ceph-mon[31823]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:49:15.071 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:49:14 smithi072 ceph-mon[28138]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:49:15.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:49:14 smithi164 ceph-mon[31916]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:49:16.141 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:49:15 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:49:16.141 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:49:15 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:49:16.141 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:49:15 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:49:16.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:49:15 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:49:16.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:49:15 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:49:16.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:49:15 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:49:16.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:49:15 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:49:16.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:49:15 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:49:16.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:49:15 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:49:17.064 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:49:16 smithi076 ceph-mon[31823]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:49:17.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:49:16 smithi164 ceph-mon[31916]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:49:17.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:49:16 smithi072 ceph-mon[28138]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:49:19.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:49:18 smithi164 ceph-mon[31916]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:49:19.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:49:18 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:49:19.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:49:18 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:49:19.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:49:18 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:49:19.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:49:18 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:49:19.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:49:18 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:49:19.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:49:18 smithi164 ceph-mon[31916]: from='osd.7 [v2:172.21.15.76:6816/3144631323,v1:172.21.15.76:6817/3144631323]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-26T21:49:19.192 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:49:18 smithi164 ceph-mon[31916]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-26T21:49:19.248 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:49:18 smithi076 ceph-mon[31823]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:49:19.249 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:49:18 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:49:19.249 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:49:18 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:49:19.249 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:49:18 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:49:19.249 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:49:18 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:49:19.249 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:49:18 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:49:19.249 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:49:18 smithi076 ceph-mon[31823]: from='osd.7 [v2:172.21.15.76:6816/3144631323,v1:172.21.15.76:6817/3144631323]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-26T21:49:19.249 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:49:18 smithi076 ceph-mon[31823]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-26T21:49:19.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:49:18 smithi072 ceph-mon[28138]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:49:19.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:49:18 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:49:19.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:49:18 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:49:19.323 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:49:18 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:49:19.323 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:49:18 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:49:19.323 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:49:18 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:49:19.323 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:49:18 smithi072 ceph-mon[28138]: from='osd.7 [v2:172.21.15.76:6816/3144631323,v1:172.21.15.76:6817/3144631323]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-26T21:49:19.323 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:49:18 smithi072 ceph-mon[28138]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-26T21:49:19.695 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:49:19.693+0000 7f3546000700 1 -- 172.21.15.76:0/2803543395 <== mgr.14152 v2:172.21.15.72:6800/588150172 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f3548063860 con 0x7f35300a3c40 2024-04-26T21:49:19.695 INFO:teuthology.orchestra.run.smithi076.stdout:Created osd(s) 7 on host 'smithi076' 2024-04-26T21:49:19.698 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:49:19.696+0000 7f3550400700 1 -- 172.21.15.76:0/2803543395 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f35300a3c40 msgr2=0x7f35300a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:49:19.698 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:49:19.696+0000 7f3550400700 1 --2- 172.21.15.76:0/2803543395 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f35300a3c40 0x7f35300a6100 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f3534005950 tx=0x7f35340058e0 comp rx=0 tx=0).stop 2024-04-26T21:49:19.698 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:49:19.696+0000 7f3550400700 1 -- 172.21.15.76:0/2803543395 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3548109ac0 msgr2=0x7f354819fb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:49:19.698 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:49:19.696+0000 7f3550400700 1 --2- 172.21.15.76:0/2803543395 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3548109ac0 0x7f354819fb50 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f353c0098e0 tx=0x7f353c00f740 comp rx=0 tx=0).stop 2024-04-26T21:49:19.698 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:49:19.697+0000 7f3550400700 1 -- 172.21.15.76:0/2803543395 shutdown_connections 2024-04-26T21:49:19.698 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:49:19.697+0000 7f3550400700 1 --2- 172.21.15.76:0/2803543395 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f35300a3c40 0x7f35300a6100 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:49:19.699 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:49:19.697+0000 7f3550400700 1 --2- 172.21.15.76:0/2803543395 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f354810b660 0x7f35481a7110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:49:19.699 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:49:19.697+0000 7f3550400700 1 --2- 172.21.15.76:0/2803543395 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3548109ac0 0x7f354819fb50 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:49:19.699 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:49:19.697+0000 7f3550400700 1 --2- 172.21.15.76:0/2803543395 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f354810acc0 0x7f35481a0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:49:19.699 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:49:19.697+0000 7f3550400700 1 -- 172.21.15.76:0/2803543395 >> 172.21.15.76:0/2803543395 conn(0x7f3548078a90 msgr2=0x7f354810e090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:49:19.699 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:49:19.697+0000 7f3550400700 1 -- 172.21.15.76:0/2803543395 shutdown_connections 2024-04-26T21:49:19.699 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:49:19.697+0000 7f3550400700 1 -- 172.21.15.76:0/2803543395 wait complete. 2024-04-26T21:49:20.459 DEBUG:teuthology.orchestra.run.smithi076:osd.7> sudo journalctl -f -n 0 -u ceph-a1780950-0415-11ef-bc93-c7b262605968@osd.7.service 2024-04-26T21:49:20.464 INFO:tasks.cephadm:Deploying osd.8 on smithi164 with /dev/vg_nvme/lv_2... 2024-04-26T21:49:20.465 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a1780950-0415-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-26T21:49:20.690 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:49:20 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:49:20.690 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:49:20 smithi076 ceph-mon[31823]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-26T21:49:20.690 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:49:20 smithi076 ceph-mon[31823]: osdmap e50: 8 total, 7 up, 8 in 2024-04-26T21:49:20.690 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:49:20 smithi076 ceph-mon[31823]: from='osd.7 [v2:172.21.15.76:6816/3144631323,v1:172.21.15.76:6817/3144631323]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi076", "root=default"]}]: dispatch 2024-04-26T21:49:20.690 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:49:20 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T21:49:20.690 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:49:20 smithi076 ceph-mon[31823]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi076", "root=default"]}]: dispatch 2024-04-26T21:49:20.690 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:49:20 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:49:20.690 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:49:20 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:49:20.691 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:49:20 smithi076 ceph-mon[31823]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:49:20.907 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/a1780950-0415-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T21:49:20.940 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:49:20 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:49:20.940 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:49:20 smithi164 ceph-mon[31916]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-26T21:49:20.940 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:49:20 smithi164 ceph-mon[31916]: osdmap e50: 8 total, 7 up, 8 in 2024-04-26T21:49:20.941 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:49:20 smithi164 ceph-mon[31916]: from='osd.7 [v2:172.21.15.76:6816/3144631323,v1:172.21.15.76:6817/3144631323]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi076", "root=default"]}]: dispatch 2024-04-26T21:49:20.941 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:49:20 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T21:49:20.941 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:49:20 smithi164 ceph-mon[31916]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi076", "root=default"]}]: dispatch 2024-04-26T21:49:20.941 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:49:20 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:49:20.941 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:49:20 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:49:20.941 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:49:20 smithi164 ceph-mon[31916]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:49:21.037 INFO:journalctl@ceph.osd.7.smithi076.stdout:Apr 26 21:49:20 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-7[49518]: 2024-04-26T21:49:20.693+0000 7fcc07e00700 -1 osd.7 0 waiting for initial osdmap 2024-04-26T21:49:21.037 INFO:journalctl@ceph.osd.7.smithi076.stdout:Apr 26 21:49:20 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-7[49518]: 2024-04-26T21:49:20.728+0000 7fcc02c00700 -1 osd.7 51 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T21:49:21.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:49:20 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:49:21.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:49:20 smithi072 ceph-mon[28138]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-26T21:49:21.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:49:20 smithi072 ceph-mon[28138]: osdmap e50: 8 total, 7 up, 8 in 2024-04-26T21:49:21.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:49:20 smithi072 ceph-mon[28138]: from='osd.7 [v2:172.21.15.76:6816/3144631323,v1:172.21.15.76:6817/3144631323]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi076", "root=default"]}]: dispatch 2024-04-26T21:49:21.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:49:20 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T21:49:21.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:49:20 smithi072 ceph-mon[28138]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi076", "root=default"]}]: dispatch 2024-04-26T21:49:21.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:49:20 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:49:21.073 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:49:20 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:49:21.073 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:49:20 smithi072 ceph-mon[28138]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:49:21.815 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:49:21 smithi076 ceph-mon[31823]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi076", "root=default"]}]': finished 2024-04-26T21:49:21.815 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:49:21 smithi076 ceph-mon[31823]: osdmap e51: 8 total, 7 up, 8 in 2024-04-26T21:49:21.815 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:49:21 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T21:49:21.815 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:49:21 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T21:49:21.941 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:49:21 smithi164 ceph-mon[31916]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi076", "root=default"]}]': finished 2024-04-26T21:49:21.941 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:49:21 smithi164 ceph-mon[31916]: osdmap e51: 8 total, 7 up, 8 in 2024-04-26T21:49:21.941 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:49:21 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T21:49:21.941 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:49:21 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T21:49:22.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:49:21 smithi072 ceph-mon[28138]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi076", "root=default"]}]': finished 2024-04-26T21:49:22.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:49:21 smithi072 ceph-mon[28138]: osdmap e51: 8 total, 7 up, 8 in 2024-04-26T21:49:22.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:49:21 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T21:49:22.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:49:21 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T21:49:22.815 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:49:22 smithi076 ceph-mon[31823]: purged_snaps scrub starts 2024-04-26T21:49:22.815 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:49:22 smithi076 ceph-mon[31823]: purged_snaps scrub ok 2024-04-26T21:49:22.815 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:49:22 smithi076 ceph-mon[31823]: osd.7 [v2:172.21.15.76:6816/3144631323,v1:172.21.15.76:6817/3144631323] boot 2024-04-26T21:49:22.815 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:49:22 smithi076 ceph-mon[31823]: osdmap e52: 8 total, 8 up, 8 in 2024-04-26T21:49:22.815 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:49:22 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T21:49:22.815 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:49:22 smithi076 ceph-mon[31823]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:49:22.941 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:49:22 smithi164 ceph-mon[31916]: purged_snaps scrub starts 2024-04-26T21:49:22.941 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:49:22 smithi164 ceph-mon[31916]: purged_snaps scrub ok 2024-04-26T21:49:22.941 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:49:22 smithi164 ceph-mon[31916]: osd.7 [v2:172.21.15.76:6816/3144631323,v1:172.21.15.76:6817/3144631323] boot 2024-04-26T21:49:22.941 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:49:22 smithi164 ceph-mon[31916]: osdmap e52: 8 total, 8 up, 8 in 2024-04-26T21:49:22.941 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:49:22 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T21:49:22.941 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:49:22 smithi164 ceph-mon[31916]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:49:23.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:49:22 smithi072 ceph-mon[28138]: purged_snaps scrub starts 2024-04-26T21:49:23.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:49:22 smithi072 ceph-mon[28138]: purged_snaps scrub ok 2024-04-26T21:49:23.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:49:22 smithi072 ceph-mon[28138]: osd.7 [v2:172.21.15.76:6816/3144631323,v1:172.21.15.76:6817/3144631323] boot 2024-04-26T21:49:23.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:49:22 smithi072 ceph-mon[28138]: osdmap e52: 8 total, 8 up, 8 in 2024-04-26T21:49:23.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:49:22 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T21:49:23.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:49:22 smithi072 ceph-mon[28138]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:49:23.882 INFO:teuthology.orchestra.run.smithi164.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-26T21:49:23.882 INFO:teuthology.orchestra.run.smithi164.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-26T21:49:23.882 INFO:teuthology.orchestra.run.smithi164.stderr: stderr: 10+0 records in 2024-04-26T21:49:23.882 INFO:teuthology.orchestra.run.smithi164.stderr:10+0 records out 2024-04-26T21:49:23.882 INFO:teuthology.orchestra.run.smithi164.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0340831 s, 308 MB/s 2024-04-26T21:49:23.882 INFO:teuthology.orchestra.run.smithi164.stderr:--> Zapping successful for: 2024-04-26T21:49:24.064 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:49:23 smithi076 ceph-mon[31823]: osdmap e53: 8 total, 8 up, 8 in 2024-04-26T21:49:24.071 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:49:23 smithi072 ceph-mon[28138]: osdmap e53: 8 total, 8 up, 8 in 2024-04-26T21:49:24.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:49:23 smithi164 ceph-mon[31916]: osdmap e53: 8 total, 8 up, 8 in 2024-04-26T21:49:24.792 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a1780950-0415-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi164:vg_nvme/lv_2 2024-04-26T21:49:24.814 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:49:24 smithi076 ceph-mon[31823]: pgmap v231: 1 pgs: 1 remapped+peering; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:49:24.815 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:49:24 smithi076 ceph-mon[31823]: osdmap e54: 8 total, 8 up, 8 in 2024-04-26T21:49:24.815 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:49:24 smithi076 ceph-mon[31823]: Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY) 2024-04-26T21:49:24.862 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:49:24 smithi164 ceph-mon[31916]: pgmap v231: 1 pgs: 1 remapped+peering; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:49:24.862 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:49:24 smithi164 ceph-mon[31916]: osdmap e54: 8 total, 8 up, 8 in 2024-04-26T21:49:24.862 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:49:24 smithi164 ceph-mon[31916]: Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY) 2024-04-26T21:49:25.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:49:24 smithi072 ceph-mon[28138]: pgmap v231: 1 pgs: 1 remapped+peering; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:49:25.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:49:24 smithi072 ceph-mon[28138]: osdmap e54: 8 total, 8 up, 8 in 2024-04-26T21:49:25.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:49:24 smithi072 ceph-mon[28138]: Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY) 2024-04-26T21:49:25.221 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/a1780950-0415-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T21:49:25.879 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:49:25 smithi164 ceph-mon[31916]: osdmap e55: 8 total, 8 up, 8 in 2024-04-26T21:49:25.879 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:49:25 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:49:25.879 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:49:25 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:49:25.879 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:49:25 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:49:25.879 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:49:25 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:49:25.879 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:49:25 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:49:25.879 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:49:25 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:49:25.879 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:49:25 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:49:25.879 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:49:25 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:49:25.879 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:49:25 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:49:26.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:49:25 smithi076 ceph-mon[31823]: osdmap e55: 8 total, 8 up, 8 in 2024-04-26T21:49:26.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:49:25 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:49:26.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:49:25 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:49:26.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:49:25 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:49:26.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:49:25 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:49:26.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:49:25 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:49:26.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:49:25 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:49:26.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:49:25 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:49:26.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:49:25 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:49:26.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:49:25 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:49:26.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:49:25 smithi072 ceph-mon[28138]: osdmap e55: 8 total, 8 up, 8 in 2024-04-26T21:49:26.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:49:25 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:49:26.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:49:25 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:49:26.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:49:25 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:49:26.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:49:25 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:49:26.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:49:25 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:49:26.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:49:25 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:49:26.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:49:25 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:49:26.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:49:25 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:49:26.073 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:49:25 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:49:27.064 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:49:26 smithi076 ceph-mon[31823]: Detected new or changed devices on smithi076 2024-04-26T21:49:27.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:49:26 smithi076 ceph-mon[31823]: Adjusting osd_memory_target on smithi076 to 5696M 2024-04-26T21:49:27.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:49:26 smithi076 ceph-mon[31823]: pgmap v233: 1 pgs: 1 remapped+peering; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:49:27.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:49:26 smithi076 ceph-mon[31823]: osdmap e56: 8 total, 8 up, 8 in 2024-04-26T21:49:27.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:49:26 smithi072 ceph-mon[28138]: Detected new or changed devices on smithi076 2024-04-26T21:49:27.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:49:26 smithi072 ceph-mon[28138]: Adjusting osd_memory_target on smithi076 to 5696M 2024-04-26T21:49:27.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:49:26 smithi072 ceph-mon[28138]: pgmap v233: 1 pgs: 1 remapped+peering; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:49:27.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:49:26 smithi072 ceph-mon[28138]: osdmap e56: 8 total, 8 up, 8 in 2024-04-26T21:49:27.133 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:49:26 smithi164 ceph-mon[31916]: Detected new or changed devices on smithi076 2024-04-26T21:49:27.133 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:49:26 smithi164 ceph-mon[31916]: Adjusting osd_memory_target on smithi076 to 5696M 2024-04-26T21:49:27.133 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:49:26 smithi164 ceph-mon[31916]: pgmap v233: 1 pgs: 1 remapped+peering; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:49:27.133 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:49:26 smithi164 ceph-mon[31916]: osdmap e56: 8 total, 8 up, 8 in 2024-04-26T21:49:27.608 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:49:27.607+0000 7ff509e00700 1 -- 172.21.15.164:0/258891474 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff504107160 msgr2=0x7ff50410da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:49:27.608 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:49:27.607+0000 7ff509e00700 1 --2- 172.21.15.164:0/258891474 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff504107160 0x7ff50410da10 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7ff4fc00b0a0 tx=0x7ff4fc00b3b0 comp rx=0 tx=0).stop 2024-04-26T21:49:27.608 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:49:27.608+0000 7ff509e00700 1 -- 172.21.15.164:0/258891474 shutdown_connections 2024-04-26T21:49:27.608 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:49:27.608+0000 7ff509e00700 1 --2- 172.21.15.164:0/258891474 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff504107160 0x7ff50410da10 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:49:27.608 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:49:27.608+0000 7ff509e00700 1 --2- 172.21.15.164:0/258891474 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff5041055c0 0x7ff5041059e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:49:27.608 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:49:27.608+0000 7ff509e00700 1 --2- 172.21.15.164:0/258891474 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff5041067c0 0x7ff504106c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:49:27.608 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:49:27.608+0000 7ff509e00700 1 -- 172.21.15.164:0/258891474 >> 172.21.15.164:0/258891474 conn(0x7ff504100b60 msgr2=0x7ff504102fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:49:27.609 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:49:27.608+0000 7ff509e00700 1 -- 172.21.15.164:0/258891474 shutdown_connections 2024-04-26T21:49:27.609 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:49:27.608+0000 7ff509e00700 1 -- 172.21.15.164:0/258891474 wait complete. 2024-04-26T21:49:27.610 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:49:27.609+0000 7ff509e00700 1 Processor -- start 2024-04-26T21:49:27.610 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:49:27.609+0000 7ff509e00700 1 -- start start 2024-04-26T21:49:27.610 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:49:27.609+0000 7ff509e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff5041055c0 0x7ff50419b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:49:27.610 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:49:27.609+0000 7ff509e00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff5041067c0 0x7ff50419ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:49:27.610 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:49:27.609+0000 7ff509e00700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff504107160 0x7ff5041a2ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:49:27.610 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:49:27.610+0000 7ff509e00700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7ff504110f30 con 0x7ff5041067c0 2024-04-26T21:49:27.610 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:49:27.610+0000 7ff509e00700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7ff504110db0 con 0x7ff504107160 2024-04-26T21:49:27.610 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:49:27.610+0000 7ff509e00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ff5041110b0 con 0x7ff5041055c0 2024-04-26T21:49:27.610 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:49:27.610+0000 7ff508a00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff5041055c0 0x7ff50419b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:49:27.610 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:49:27.610+0000 7ff508a00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff5041055c0 0x7ff50419b510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:45266/0 (socket says 172.21.15.164:45266) 2024-04-26T21:49:27.610 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:49:27.610+0000 7ff508a00700 1 -- 172.21.15.164:0/1723243705 learned_addr learned my addr 172.21.15.164:0/1723243705 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-04-26T21:49:27.611 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:49:27.610+0000 7ff503e00700 1 --2- 172.21.15.164:0/1723243705 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff5041067c0 0x7ff50419ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:49:27.611 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:49:27.610+0000 7ff509400700 1 --2- 172.21.15.164:0/1723243705 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff504107160 0x7ff5041a2ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:49:27.612 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:49:27.610+0000 7ff508a00700 1 -- 172.21.15.164:0/1723243705 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff504107160 msgr2=0x7ff5041a2ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:49:27.612 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:49:27.610+0000 7ff508a00700 1 --2- 172.21.15.164:0/1723243705 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff504107160 0x7ff5041a2ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:49:27.612 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:49:27.610+0000 7ff508a00700 1 -- 172.21.15.164:0/1723243705 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff5041067c0 msgr2=0x7ff50419ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:49:27.612 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:49:27.610+0000 7ff508a00700 1 --2- 172.21.15.164:0/1723243705 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff5041067c0 0x7ff50419ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:49:27.612 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:49:27.610+0000 7ff508a00700 1 -- 172.21.15.164:0/1723243705 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5041a3170 con 0x7ff5041055c0 2024-04-26T21:49:27.612 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:49:27.610+0000 7ff509400700 1 --2- 172.21.15.164:0/1723243705 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff504107160 0x7ff5041a2ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:49:27.612 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:49:27.611+0000 7ff508a00700 1 --2- 172.21.15.164:0/1723243705 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff5041055c0 0x7ff50419b510 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7ff4f800c9a0 tx=0x7ff4f800cd60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:49:27.612 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:49:27.611+0000 7ff501600700 1 -- 172.21.15.164:0/1723243705 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4f8004220 con 0x7ff5041055c0 2024-04-26T21:49:27.612 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:49:27.611+0000 7ff501600700 1 -- 172.21.15.164:0/1723243705 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff4f8007cf0 con 0x7ff5041055c0 2024-04-26T21:49:27.612 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:49:27.611+0000 7ff509e00700 1 -- 172.21.15.164:0/1723243705 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5041a3460 con 0x7ff5041055c0 2024-04-26T21:49:27.612 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:49:27.611+0000 7ff501600700 1 -- 172.21.15.164:0/1723243705 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4f8003c50 con 0x7ff5041055c0 2024-04-26T21:49:27.612 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:49:27.611+0000 7ff509e00700 1 -- 172.21.15.164:0/1723243705 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5041a39a0 con 0x7ff5041055c0 2024-04-26T21:49:27.613 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:49:27.612+0000 7ff509e00700 1 -- 172.21.15.164:0/1723243705 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff50406a110 con 0x7ff5041055c0 2024-04-26T21:49:27.614 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:49:27.614+0000 7ff501600700 1 -- 172.21.15.164:0/1723243705 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7ff4f80043c0 con 0x7ff5041055c0 2024-04-26T21:49:27.616 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:49:27.616+0000 7ff501600700 1 --2- 172.21.15.164:0/1723243705 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7ff4ec0a3c40 0x7ff4ec0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:49:27.616 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:49:27.616+0000 7ff501600700 1 -- 172.21.15.164:0/1723243705 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5397+0+0 (secure 0 0 0) 0x7ff4f80cf630 con 0x7ff5041055c0 2024-04-26T21:49:27.616 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:49:27.616+0000 7ff503e00700 1 --2- 172.21.15.164:0/1723243705 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7ff4ec0a3c40 0x7ff4ec0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:49:27.617 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:49:27.618+0000 7ff503e00700 1 --2- 172.21.15.164:0/1723243705 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7ff4ec0a3c40 0x7ff4ec0a6100 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7ff4f0005ea0 tx=0x7ff4f0005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:49:27.621 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:49:27.621+0000 7ff501600700 1 -- 172.21.15.164:0/1723243705 <== mon.1 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7ff4f8014030 con 0x7ff5041055c0 2024-04-26T21:49:27.855 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:49:27.855+0000 7ff509e00700 1 -- 172.21.15.164:0/1723243705 --> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi164:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7ff504063860 con 0x7ff4ec0a3c40 2024-04-26T21:49:28.064 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:49:27 smithi076 ceph-mon[31823]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 1 pg peering) 2024-04-26T21:49:28.064 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:49:27 smithi076 ceph-mon[31823]: Cluster is now healthy 2024-04-26T21:49:28.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:49:27 smithi072 ceph-mon[28138]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 1 pg peering) 2024-04-26T21:49:28.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:49:27 smithi072 ceph-mon[28138]: Cluster is now healthy 2024-04-26T21:49:28.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:49:27 smithi164 ceph-mon[31916]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 1 pg peering) 2024-04-26T21:49:28.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:49:27 smithi164 ceph-mon[31916]: Cluster is now healthy 2024-04-26T21:49:29.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:49:28 smithi076 ceph-mon[31823]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:49:29.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:49:28 smithi076 ceph-mon[31823]: from='client.24280 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi164:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:49:29.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:49:28 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:49:29.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:49:28 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:49:29.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:49:28 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:49:29.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:49:28 smithi072 ceph-mon[28138]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:49:29.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:49:28 smithi072 ceph-mon[28138]: from='client.24280 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi164:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:49:29.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:49:28 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:49:29.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:49:28 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:49:29.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:49:28 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:49:29.090 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:49:28 smithi164 ceph-mon[31916]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:49:29.090 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:49:28 smithi164 ceph-mon[31916]: from='client.24280 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi164:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:49:29.090 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:49:28 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:49:29.090 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:49:28 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:49:29.090 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:49:28 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:49:31.064 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:49:30 smithi076 ceph-mon[31823]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:49:31.071 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:49:30 smithi072 ceph-mon[28138]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:49:31.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:49:30 smithi164 ceph-mon[31916]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:49:32.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:49:31 smithi164 ceph-mon[31916]: from='client.? 172.21.15.164:0/236383915' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "779e9c1e-8e97-48d4-a854-68ca13f38e0a"}]: dispatch 2024-04-26T21:49:32.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:49:31 smithi164 ceph-mon[31916]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "779e9c1e-8e97-48d4-a854-68ca13f38e0a"}]: dispatch 2024-04-26T21:49:32.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:49:31 smithi164 ceph-mon[31916]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "779e9c1e-8e97-48d4-a854-68ca13f38e0a"}]': finished 2024-04-26T21:49:32.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:49:31 smithi164 ceph-mon[31916]: osdmap e57: 9 total, 8 up, 9 in 2024-04-26T21:49:32.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:49:31 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T21:49:32.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:49:31 smithi076 ceph-mon[31823]: from='client.? 172.21.15.164:0/236383915' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "779e9c1e-8e97-48d4-a854-68ca13f38e0a"}]: dispatch 2024-04-26T21:49:32.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:49:31 smithi076 ceph-mon[31823]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "779e9c1e-8e97-48d4-a854-68ca13f38e0a"}]: dispatch 2024-04-26T21:49:32.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:49:31 smithi076 ceph-mon[31823]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "779e9c1e-8e97-48d4-a854-68ca13f38e0a"}]': finished 2024-04-26T21:49:32.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:49:31 smithi076 ceph-mon[31823]: osdmap e57: 9 total, 8 up, 9 in 2024-04-26T21:49:32.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:49:31 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T21:49:32.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:49:31 smithi072 ceph-mon[28138]: from='client.? 172.21.15.164:0/236383915' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "779e9c1e-8e97-48d4-a854-68ca13f38e0a"}]: dispatch 2024-04-26T21:49:32.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:49:31 smithi072 ceph-mon[28138]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "779e9c1e-8e97-48d4-a854-68ca13f38e0a"}]: dispatch 2024-04-26T21:49:32.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:49:31 smithi072 ceph-mon[28138]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "779e9c1e-8e97-48d4-a854-68ca13f38e0a"}]': finished 2024-04-26T21:49:32.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:49:31 smithi072 ceph-mon[28138]: osdmap e57: 9 total, 8 up, 9 in 2024-04-26T21:49:32.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:49:31 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T21:49:33.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:49:32 smithi164 ceph-mon[31916]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:49:33.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:49:32 smithi164 ceph-mon[31916]: from='client.? 172.21.15.164:0/1831063395' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:49:33.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:49:32 smithi076 ceph-mon[31823]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:49:33.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:49:32 smithi076 ceph-mon[31823]: from='client.? 172.21.15.164:0/1831063395' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:49:33.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:49:32 smithi072 ceph-mon[28138]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:49:33.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:49:32 smithi072 ceph-mon[28138]: from='client.? 172.21.15.164:0/1831063395' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:49:35.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:49:34 smithi164 ceph-mon[31916]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:49:35.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:49:34 smithi076 ceph-mon[31823]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:49:35.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:49:34 smithi072 ceph-mon[28138]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:49:37.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:49:36 smithi076 ceph-mon[31823]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:49:37.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:49:36 smithi164 ceph-mon[31916]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:49:37.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:49:36 smithi072 ceph-mon[28138]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:49:39.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:49:38 smithi164 ceph-mon[31916]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:49:39.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:49:38 smithi076 ceph-mon[31823]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:49:39.321 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:49:38 smithi072 ceph-mon[28138]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:49:41.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:49:40 smithi164 ceph-mon[31916]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:49:41.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:49:40 smithi076 ceph-mon[31823]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:49:41.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:49:40 smithi072 ceph-mon[28138]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:49:42.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:49:41 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-26T21:49:42.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:49:41 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:49:42.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:49:41 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-26T21:49:42.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:49:41 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:49:42.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:49:41 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-26T21:49:42.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:49:41 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:49:43.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:49:42 smithi164 ceph-mon[31916]: Deploying daemon osd.8 on smithi164 2024-04-26T21:49:43.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:49:42 smithi164 ceph-mon[31916]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:49:43.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:49:42 smithi076 ceph-mon[31823]: Deploying daemon osd.8 on smithi164 2024-04-26T21:49:43.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:49:42 smithi076 ceph-mon[31823]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:49:43.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:49:42 smithi072 ceph-mon[28138]: Deploying daemon osd.8 on smithi164 2024-04-26T21:49:43.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:49:42 smithi072 ceph-mon[28138]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:49:44.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:49:43 smithi164 ceph-mon[31916]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:49:44.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:49:43 smithi076 ceph-mon[31823]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:49:44.321 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:49:43 smithi072 ceph-mon[28138]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:49:47.064 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:49:46 smithi076 ceph-mon[31823]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:49:47.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:49:46 smithi072 ceph-mon[28138]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:49:47.136 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:49:46 smithi164 ceph-mon[31916]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:49:49.064 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:49:48 smithi076 ceph-mon[31823]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:49:49.071 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:49:48 smithi072 ceph-mon[28138]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:49:49.117 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:49:48 smithi164 ceph-mon[31916]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:49:51.064 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:49:50 smithi076 ceph-mon[31823]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:49:51.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:49:50 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:49:51.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:49:50 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:49:51.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:49:50 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:49:51.070 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:49:50 smithi164 ceph-mon[31916]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:49:51.071 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:49:50 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:49:51.071 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:49:50 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:49:51.071 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:49:50 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:49:51.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:49:50 smithi072 ceph-mon[28138]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:49:51.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:49:50 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:49:51.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:49:50 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:49:51.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:49:50 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:49:53.064 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:49:52 smithi076 ceph-mon[31823]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:49:53.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:49:52 smithi072 ceph-mon[28138]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:49:53.100 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:49:52 smithi164 ceph-mon[31916]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:49:54.923 INFO:teuthology.orchestra.run.smithi164.stdout:Created osd(s) 8 on host 'smithi164' 2024-04-26T21:49:54.923 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:49:54.922+0000 7ff501600700 1 -- 172.21.15.164:0/1723243705 <== mgr.14152 v2:172.21.15.72:6800/588150172 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7ff504063860 con 0x7ff4ec0a3c40 2024-04-26T21:49:54.926 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:49:54.925+0000 7ff509e00700 1 -- 172.21.15.164:0/1723243705 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7ff4ec0a3c40 msgr2=0x7ff4ec0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:49:54.926 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:49:54.925+0000 7ff509e00700 1 --2- 172.21.15.164:0/1723243705 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7ff4ec0a3c40 0x7ff4ec0a6100 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7ff4f0005ea0 tx=0x7ff4f0005e30 comp rx=0 tx=0).stop 2024-04-26T21:49:54.926 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:49:54.925+0000 7ff509e00700 1 -- 172.21.15.164:0/1723243705 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff5041055c0 msgr2=0x7ff50419b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:49:54.926 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:49:54.925+0000 7ff509e00700 1 --2- 172.21.15.164:0/1723243705 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff5041055c0 0x7ff50419b510 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7ff4f800c9a0 tx=0x7ff4f800cd60 comp rx=0 tx=0).stop 2024-04-26T21:49:54.926 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:49:54.925+0000 7ff509e00700 1 -- 172.21.15.164:0/1723243705 shutdown_connections 2024-04-26T21:49:54.926 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:49:54.925+0000 7ff509e00700 1 --2- 172.21.15.164:0/1723243705 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7ff4ec0a3c40 0x7ff4ec0a6100 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:49:54.926 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:49:54.925+0000 7ff509e00700 1 --2- 172.21.15.164:0/1723243705 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff504107160 0x7ff5041a2ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:49:54.926 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:49:54.925+0000 7ff509e00700 1 --2- 172.21.15.164:0/1723243705 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff5041055c0 0x7ff50419b510 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:49:54.926 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:49:54.925+0000 7ff509e00700 1 --2- 172.21.15.164:0/1723243705 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff5041067c0 0x7ff50419ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:49:54.927 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:49:54.925+0000 7ff509e00700 1 -- 172.21.15.164:0/1723243705 >> 172.21.15.164:0/1723243705 conn(0x7ff504100b60 msgr2=0x7ff504109ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:49:54.927 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:49:54.925+0000 7ff509e00700 1 -- 172.21.15.164:0/1723243705 shutdown_connections 2024-04-26T21:49:54.927 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:49:54.925+0000 7ff509e00700 1 -- 172.21.15.164:0/1723243705 wait complete. 2024-04-26T21:49:55.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:49:54 smithi076 ceph-mon[31823]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:49:55.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:49:54 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:49:55.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:49:54 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:49:55.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:49:54 smithi076 ceph-mon[31823]: from='osd.8 [v2:172.21.15.164:6816/627888883,v1:172.21.15.164:6817/627888883]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-26T21:49:55.066 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:49:54 smithi076 ceph-mon[31823]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-26T21:49:55.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:49:54 smithi072 ceph-mon[28138]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:49:55.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:49:54 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:49:55.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:49:54 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:49:55.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:49:54 smithi072 ceph-mon[28138]: from='osd.8 [v2:172.21.15.164:6816/627888883,v1:172.21.15.164:6817/627888883]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-26T21:49:55.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:49:54 smithi072 ceph-mon[28138]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-26T21:49:55.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:49:54 smithi164 ceph-mon[31916]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:49:55.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:49:54 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:49:55.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:49:54 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:49:55.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:49:54 smithi164 ceph-mon[31916]: from='osd.8 [v2:172.21.15.164:6816/627888883,v1:172.21.15.164:6817/627888883]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-26T21:49:55.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:49:54 smithi164 ceph-mon[31916]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-26T21:49:55.707 DEBUG:teuthology.orchestra.run.smithi164:osd.8> sudo journalctl -f -n 0 -u ceph-a1780950-0415-11ef-bc93-c7b262605968@osd.8.service 2024-04-26T21:49:55.711 INFO:tasks.cephadm:Deploying osd.9 on smithi072 with /dev/vg_nvme/lv_1... 2024-04-26T21:49:55.711 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a1780950-0415-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-26T21:49:55.940 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:49:55 smithi164 ceph-mon[31916]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-26T21:49:55.940 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:49:55 smithi164 ceph-mon[31916]: osdmap e58: 9 total, 8 up, 9 in 2024-04-26T21:49:55.940 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:49:55 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T21:49:55.940 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:49:55 smithi164 ceph-mon[31916]: from='osd.8 [v2:172.21.15.164:6816/627888883,v1:172.21.15.164:6817/627888883]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-04-26T21:49:55.940 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:49:55 smithi164 ceph-mon[31916]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-04-26T21:49:55.940 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:49:55 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:49:55.940 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:49:55 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:49:56.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:49:55 smithi072 ceph-mon[28138]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-26T21:49:56.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:49:55 smithi072 ceph-mon[28138]: osdmap e58: 9 total, 8 up, 9 in 2024-04-26T21:49:56.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:49:55 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T21:49:56.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:49:55 smithi072 ceph-mon[28138]: from='osd.8 [v2:172.21.15.164:6816/627888883,v1:172.21.15.164:6817/627888883]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-04-26T21:49:56.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:49:55 smithi072 ceph-mon[28138]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-04-26T21:49:56.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:49:55 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:49:56.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:49:55 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:49:56.165 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/a1780950-0415-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:49:56.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:49:55 smithi076 ceph-mon[31823]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-26T21:49:56.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:49:55 smithi076 ceph-mon[31823]: osdmap e58: 9 total, 8 up, 9 in 2024-04-26T21:49:56.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:49:55 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T21:49:56.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:49:55 smithi076 ceph-mon[31823]: from='osd.8 [v2:172.21.15.164:6816/627888883,v1:172.21.15.164:6817/627888883]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-04-26T21:49:56.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:49:55 smithi076 ceph-mon[31823]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-04-26T21:49:56.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:49:55 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:49:56.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:49:55 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:49:57.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:49:56 smithi164 ceph-mon[31916]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:49:57.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:49:56 smithi164 ceph-mon[31916]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]': finished 2024-04-26T21:49:57.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:49:56 smithi164 ceph-mon[31916]: osdmap e59: 9 total, 8 up, 9 in 2024-04-26T21:49:57.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:49:56 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T21:49:57.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:49:56 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T21:49:57.192 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 26 21:49:56 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-8[50342]: 2024-04-26T21:49:56.872+0000 7f1536800700 -1 osd.8 0 waiting for initial osdmap 2024-04-26T21:49:57.192 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 26 21:49:56 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-8[50342]: 2024-04-26T21:49:56.906+0000 7f152c600700 -1 osd.8 60 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T21:49:57.222 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:49:56 smithi072 ceph-mon[28138]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:49:57.222 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:49:56 smithi072 ceph-mon[28138]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]': finished 2024-04-26T21:49:57.222 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:49:56 smithi072 ceph-mon[28138]: osdmap e59: 9 total, 8 up, 9 in 2024-04-26T21:49:57.222 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:49:56 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T21:49:57.222 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:49:56 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T21:49:57.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:49:56 smithi076 ceph-mon[31823]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:49:57.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:49:56 smithi076 ceph-mon[31823]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]': finished 2024-04-26T21:49:57.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:49:56 smithi076 ceph-mon[31823]: osdmap e59: 9 total, 8 up, 9 in 2024-04-26T21:49:57.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:49:56 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T21:49:57.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:49:56 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T21:49:57.941 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:49:57 smithi164 ceph-mon[31916]: purged_snaps scrub starts 2024-04-26T21:49:57.941 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:49:57 smithi164 ceph-mon[31916]: purged_snaps scrub ok 2024-04-26T21:49:57.941 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:49:57 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T21:49:57.941 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:49:57 smithi164 ceph-mon[31916]: osdmap e60: 9 total, 8 up, 9 in 2024-04-26T21:49:57.941 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:49:57 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T21:49:57.941 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:49:57 smithi164 ceph-mon[31916]: from='osd.8 ' entity='osd.8' 2024-04-26T21:49:57.941 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:49:57 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T21:49:57.989 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:49:57 smithi072 ceph-mon[28138]: purged_snaps scrub starts 2024-04-26T21:49:57.989 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:49:57 smithi072 ceph-mon[28138]: purged_snaps scrub ok 2024-04-26T21:49:57.989 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:49:57 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T21:49:57.989 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:49:57 smithi072 ceph-mon[28138]: osdmap e60: 9 total, 8 up, 9 in 2024-04-26T21:49:57.989 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:49:57 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T21:49:57.989 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:49:57 smithi072 ceph-mon[28138]: from='osd.8 ' entity='osd.8' 2024-04-26T21:49:57.989 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:49:57 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T21:49:58.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:49:57 smithi076 ceph-mon[31823]: purged_snaps scrub starts 2024-04-26T21:49:58.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:49:57 smithi076 ceph-mon[31823]: purged_snaps scrub ok 2024-04-26T21:49:58.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:49:57 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T21:49:58.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:49:57 smithi076 ceph-mon[31823]: osdmap e60: 9 total, 8 up, 9 in 2024-04-26T21:49:58.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:49:57 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T21:49:58.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:49:57 smithi076 ceph-mon[31823]: from='osd.8 ' entity='osd.8' 2024-04-26T21:49:58.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:49:57 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T21:49:59.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:49:58 smithi164 ceph-mon[31916]: pgmap v254: 1 pgs: 1 remapped+peering; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:49:59.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:49:58 smithi164 ceph-mon[31916]: osd.8 [v2:172.21.15.164:6816/627888883,v1:172.21.15.164:6817/627888883] boot 2024-04-26T21:49:59.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:49:58 smithi164 ceph-mon[31916]: osdmap e61: 9 total, 9 up, 9 in 2024-04-26T21:49:59.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:49:58 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T21:49:59.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:49:58 smithi164 ceph-mon[31916]: osdmap e62: 9 total, 9 up, 9 in 2024-04-26T21:49:59.255 INFO:teuthology.orchestra.run.smithi072.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-26T21:49:59.255 INFO:teuthology.orchestra.run.smithi072.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-26T21:49:59.255 INFO:teuthology.orchestra.run.smithi072.stderr: stderr: 10+0 records in 2024-04-26T21:49:59.255 INFO:teuthology.orchestra.run.smithi072.stderr:10+0 records out 2024-04-26T21:49:59.255 INFO:teuthology.orchestra.run.smithi072.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0476568 s, 220 MB/s 2024-04-26T21:49:59.255 INFO:teuthology.orchestra.run.smithi072.stderr:--> Zapping successful for: 2024-04-26T21:49:59.255 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:49:58 smithi072 ceph-mon[28138]: pgmap v254: 1 pgs: 1 remapped+peering; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:49:59.255 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:49:58 smithi072 ceph-mon[28138]: osd.8 [v2:172.21.15.164:6816/627888883,v1:172.21.15.164:6817/627888883] boot 2024-04-26T21:49:59.255 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:49:58 smithi072 ceph-mon[28138]: osdmap e61: 9 total, 9 up, 9 in 2024-04-26T21:49:59.256 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:49:58 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T21:49:59.256 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:49:58 smithi072 ceph-mon[28138]: osdmap e62: 9 total, 9 up, 9 in 2024-04-26T21:49:59.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:49:58 smithi076 ceph-mon[31823]: pgmap v254: 1 pgs: 1 remapped+peering; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:49:59.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:49:58 smithi076 ceph-mon[31823]: osd.8 [v2:172.21.15.164:6816/627888883,v1:172.21.15.164:6817/627888883] boot 2024-04-26T21:49:59.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:49:58 smithi076 ceph-mon[31823]: osdmap e61: 9 total, 9 up, 9 in 2024-04-26T21:49:59.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:49:58 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T21:49:59.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:49:58 smithi076 ceph-mon[31823]: osdmap e62: 9 total, 9 up, 9 in 2024-04-26T21:50:00.113 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a1780950-0415-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi072:vg_nvme/lv_1 2024-04-26T21:50:00.539 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/a1780950-0415-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:50:00.691 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:50:00 smithi164 ceph-mon[31916]: osdmap e63: 9 total, 9 up, 9 in 2024-04-26T21:50:00.691 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:50:00 smithi164 ceph-mon[31916]: pgmap v258: 1 pgs: 1 remapped+peering; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:50:00.691 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:50:00 smithi164 ceph-mon[31916]: overall HEALTH_OK 2024-04-26T21:50:00.691 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:50:00 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:50:00.815 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:50:00 smithi076 ceph-mon[31823]: osdmap e63: 9 total, 9 up, 9 in 2024-04-26T21:50:00.815 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:50:00 smithi076 ceph-mon[31823]: pgmap v258: 1 pgs: 1 remapped+peering; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:50:00.815 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:50:00 smithi076 ceph-mon[31823]: overall HEALTH_OK 2024-04-26T21:50:00.815 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:50:00 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:50:00.822 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:50:00 smithi072 ceph-mon[28138]: osdmap e63: 9 total, 9 up, 9 in 2024-04-26T21:50:00.822 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:50:00 smithi072 ceph-mon[28138]: pgmap v258: 1 pgs: 1 remapped+peering; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:50:00.822 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:50:00 smithi072 ceph-mon[28138]: overall HEALTH_OK 2024-04-26T21:50:00.822 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:50:00 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:50:01.701 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:50:01 smithi072 ceph-mon[28138]: Detected new or changed devices on smithi164 2024-04-26T21:50:01.702 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:50:01 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:50:01.702 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:50:01 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:50:01.702 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:50:01 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:50:01.702 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:50:01 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:50:01.702 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:50:01 smithi072 ceph-mon[28138]: Adjusting osd_memory_target on smithi164 to 5696M 2024-04-26T21:50:01.702 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:50:01 smithi072 ceph-mon[28138]: osdmap e64: 9 total, 9 up, 9 in 2024-04-26T21:50:01.702 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:50:01 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:50:01.702 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:50:01 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:50:01.702 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:50:01 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:50:01.702 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:50:01 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:50:01.702 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:50:01 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:50:01.702 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:50:01 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:50:01.703 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:50:01 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:50:01.703 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:50:01 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:50:01.815 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:50:01 smithi076 ceph-mon[31823]: Detected new or changed devices on smithi164 2024-04-26T21:50:01.815 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:50:01 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:50:01.815 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:50:01 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:50:01.815 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:50:01 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:50:01.815 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:50:01 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:50:01.815 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:50:01 smithi076 ceph-mon[31823]: Adjusting osd_memory_target on smithi164 to 5696M 2024-04-26T21:50:01.816 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:50:01 smithi076 ceph-mon[31823]: osdmap e64: 9 total, 9 up, 9 in 2024-04-26T21:50:01.816 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:50:01 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:50:01.816 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:50:01 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:50:01.816 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:50:01 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:50:01.816 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:50:01 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:50:01.816 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:50:01 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:50:01.816 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:50:01 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:50:01.816 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:50:01 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:50:01.816 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:50:01 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:50:01.941 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:50:01 smithi164 ceph-mon[31916]: Detected new or changed devices on smithi164 2024-04-26T21:50:01.941 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:50:01 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:50:01.941 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:50:01 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:50:01.941 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:50:01 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:50:01.941 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:50:01 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:50:01.941 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:50:01 smithi164 ceph-mon[31916]: Adjusting osd_memory_target on smithi164 to 5696M 2024-04-26T21:50:01.941 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:50:01 smithi164 ceph-mon[31916]: osdmap e64: 9 total, 9 up, 9 in 2024-04-26T21:50:01.941 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:50:01 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:50:01.942 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:50:01 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:50:01.942 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:50:01 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:50:01.942 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:50:01 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:50:01.942 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:50:01 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:50:01.942 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:50:01 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:50:01.942 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:50:01 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:50:01.942 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:50:01 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:50:02.692 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:50:02.691+0000 7f91fbe00700 1 -- 172.21.15.72:0/2644643600 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f91fc10f9a0 msgr2=0x7f91fc111d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:02.692 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:50:02.691+0000 7f91fbe00700 1 --2- 172.21.15.72:0/2644643600 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f91fc10f9a0 0x7f91fc111d90 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f91f400b0a0 tx=0x7f91f400b3b0 comp rx=0 tx=0).stop 2024-04-26T21:50:02.692 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:50:02.691+0000 7f91fbe00700 1 -- 172.21.15.72:0/2644643600 shutdown_connections 2024-04-26T21:50:02.692 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:50:02.691+0000 7f91fbe00700 1 --2- 172.21.15.72:0/2644643600 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f91fc10f9a0 0x7f91fc111d90 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:02.692 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:50:02.691+0000 7f91fbe00700 1 --2- 172.21.15.72:0/2644643600 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f91fc077450 0x7f91fc0758b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:02.693 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:50:02.691+0000 7f91fbe00700 1 --2- 172.21.15.72:0/2644643600 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f91fc075e80 0x7f91fc076300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:02.693 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:50:02.691+0000 7f91fbe00700 1 -- 172.21.15.72:0/2644643600 >> 172.21.15.72:0/2644643600 conn(0x7f91fc0fe960 msgr2=0x7f91fc100da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:50:02.693 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:50:02.691+0000 7f91fbe00700 1 -- 172.21.15.72:0/2644643600 shutdown_connections 2024-04-26T21:50:02.693 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:50:02.691+0000 7f91fbe00700 1 -- 172.21.15.72:0/2644643600 wait complete. 2024-04-26T21:50:02.695 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:50:02.693+0000 7f91fbe00700 1 Processor -- start 2024-04-26T21:50:02.695 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:50:02.693+0000 7f91fbe00700 1 -- start start 2024-04-26T21:50:02.695 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:50:02.693+0000 7f91fbe00700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f91fc075e80 0x7f91fc19f900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:02.695 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:50:02.693+0000 7f91fbe00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f91fc077450 0x7f91fc19fe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:02.695 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:50:02.693+0000 7f91fbe00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f91fc10f9a0 0x7f91fc1a6ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:02.696 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:50:02.693+0000 7f91fbe00700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f91fc115320 con 0x7f91fc10f9a0 2024-04-26T21:50:02.696 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:50:02.693+0000 7f91fbe00700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f91fc1151a0 con 0x7f91fc075e80 2024-04-26T21:50:02.696 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:50:02.693+0000 7f91fbe00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f91fc1154a0 con 0x7f91fc077450 2024-04-26T21:50:02.696 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:50:02.693+0000 7f91fb400700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f91fc10f9a0 0x7f91fc1a6ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:02.696 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:50:02.693+0000 7f91fb400700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f91fc10f9a0 0x7f91fc1a6ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:39348/0 (socket says 172.21.15.72:39348) 2024-04-26T21:50:02.696 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:50:02.693+0000 7f91fb400700 1 -- 172.21.15.72:0/2244041209 learned_addr learned my addr 172.21.15.72:0/2244041209 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:50:02.696 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:50:02.694+0000 7f91fa000700 1 --2- 172.21.15.72:0/2244041209 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f91fc077450 0x7f91fc19fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:02.696 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:50:02.694+0000 7f91faa00700 1 --2- 172.21.15.72:0/2244041209 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f91fc075e80 0x7f91fc19f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:02.696 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:50:02.694+0000 7f91fb400700 1 -- 172.21.15.72:0/2244041209 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f91fc077450 msgr2=0x7f91fc19fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:02.696 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:50:02.694+0000 7f91fb400700 1 --2- 172.21.15.72:0/2244041209 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f91fc077450 0x7f91fc19fe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:02.696 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:50:02.694+0000 7f91fb400700 1 -- 172.21.15.72:0/2244041209 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f91fc075e80 msgr2=0x7f91fc19f900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:02.696 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:50:02.694+0000 7f91fb400700 1 --2- 172.21.15.72:0/2244041209 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f91fc075e80 0x7f91fc19f900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:02.696 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:50:02.694+0000 7f91fb400700 1 -- 172.21.15.72:0/2244041209 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91fc1a7560 con 0x7f91fc10f9a0 2024-04-26T21:50:02.696 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:50:02.694+0000 7f91faa00700 1 --2- 172.21.15.72:0/2244041209 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f91fc075e80 0x7f91fc19f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:50:02.697 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:50:02.694+0000 7f91fb400700 1 --2- 172.21.15.72:0/2244041209 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f91fc10f9a0 0x7f91fc1a6ec0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f91f400b070 tx=0x7f91f4009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:02.697 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:50:02.695+0000 7f91f3400700 1 -- 172.21.15.72:0/2244041209 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91f4021070 con 0x7f91fc10f9a0 2024-04-26T21:50:02.697 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:50:02.695+0000 7f91f3400700 1 -- 172.21.15.72:0/2244041209 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f91f4007c00 con 0x7f91fc10f9a0 2024-04-26T21:50:02.697 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:50:02.695+0000 7f91fbe00700 1 -- 172.21.15.72:0/2244041209 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91fc1a77f0 con 0x7f91fc10f9a0 2024-04-26T21:50:02.697 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:50:02.695+0000 7f91f3400700 1 -- 172.21.15.72:0/2244041209 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91f401b720 con 0x7f91fc10f9a0 2024-04-26T21:50:02.697 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:50:02.695+0000 7f91fbe00700 1 -- 172.21.15.72:0/2244041209 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91fc1a7c40 con 0x7f91fc10f9a0 2024-04-26T21:50:02.697 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:50:02.696+0000 7f91fbe00700 1 -- 172.21.15.72:0/2244041209 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91fc06a110 con 0x7f91fc10f9a0 2024-04-26T21:50:02.699 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:50:02.699+0000 7f91f3400700 1 -- 172.21.15.72:0/2244041209 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f91f4024070 con 0x7f91fc10f9a0 2024-04-26T21:50:02.700 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:50:02.700+0000 7f91f3400700 1 --2- 172.21.15.72:0/2244041209 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f91e00a3c40 0x7f91e00a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:02.700 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:50:02.700+0000 7f91f3400700 1 -- 172.21.15.72:0/2244041209 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 5829+0+0 (secure 0 0 0) 0x7f91f400f020 con 0x7f91fc10f9a0 2024-04-26T21:50:02.701 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:50:02.701+0000 7f91faa00700 1 --2- 172.21.15.72:0/2244041209 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f91e00a3c40 0x7f91e00a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:02.701 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:50:02.701+0000 7f91faa00700 1 --2- 172.21.15.72:0/2244041209 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f91e00a3c40 0x7f91e00a6100 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f91fc077830 tx=0x7f91ec00b2c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:02.705 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:50:02.705+0000 7f91f3400700 1 -- 172.21.15.72:0/2244041209 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f91f402b020 con 0x7f91fc10f9a0 2024-04-26T21:50:02.814 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:50:02 smithi076 ceph-mon[31823]: osdmap e65: 9 total, 9 up, 9 in 2024-04-26T21:50:02.815 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:50:02 smithi076 ceph-mon[31823]: pgmap v261: 1 pgs: 1 remapped+peering; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:50:02.822 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:50:02 smithi072 ceph-mon[28138]: osdmap e65: 9 total, 9 up, 9 in 2024-04-26T21:50:02.822 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:50:02 smithi072 ceph-mon[28138]: pgmap v261: 1 pgs: 1 remapped+peering; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:50:02.940 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:50:02 smithi164 ceph-mon[31916]: osdmap e65: 9 total, 9 up, 9 in 2024-04-26T21:50:02.940 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:50:02 smithi164 ceph-mon[31916]: pgmap v261: 1 pgs: 1 remapped+peering; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:50:02.947 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:50:02.947+0000 7f91fbe00700 1 -- 172.21.15.72:0/2244041209 --> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi072:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f91fc063860 con 0x7f91e00a3c40 2024-04-26T21:50:03.814 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:50:03 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:50:03.815 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:50:03 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:50:03.815 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:50:03 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:50:03.821 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:50:03 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:50:03.821 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:50:03 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:50:03.822 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:50:03 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:50:03.941 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:50:03 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:50:03.941 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:50:03 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:50:03.941 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:50:03 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:50:04.815 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:50:04 smithi076 ceph-mon[31823]: from='client.14421 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi072:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:50:04.815 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:50:04 smithi076 ceph-mon[31823]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 109 KiB/s, 0 objects/s recovering 2024-04-26T21:50:04.822 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:50:04 smithi072 ceph-mon[28138]: from='client.14421 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi072:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:50:04.822 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:50:04 smithi072 ceph-mon[28138]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 109 KiB/s, 0 objects/s recovering 2024-04-26T21:50:04.940 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:50:04 smithi164 ceph-mon[31916]: from='client.14421 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi072:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:50:04.941 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:50:04 smithi164 ceph-mon[31916]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 109 KiB/s, 0 objects/s recovering 2024-04-26T21:50:07.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:50:06 smithi076 ceph-mon[31823]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail; 92 KiB/s, 0 objects/s recovering 2024-04-26T21:50:07.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:50:06 smithi072 ceph-mon[28138]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail; 92 KiB/s, 0 objects/s recovering 2024-04-26T21:50:07.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:50:06 smithi164 ceph-mon[31916]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail; 92 KiB/s, 0 objects/s recovering 2024-04-26T21:50:08.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:50:07 smithi076 ceph-mon[31823]: from='client.? 172.21.15.72:0/997013101' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d0317850-7b2d-4850-b2cf-0b919505c1fa"}]: dispatch 2024-04-26T21:50:08.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:50:07 smithi076 ceph-mon[31823]: from='client.? 172.21.15.72:0/997013101' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d0317850-7b2d-4850-b2cf-0b919505c1fa"}]': finished 2024-04-26T21:50:08.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:50:07 smithi076 ceph-mon[31823]: osdmap e66: 10 total, 9 up, 10 in 2024-04-26T21:50:08.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:50:07 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T21:50:08.071 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:50:07 smithi072 ceph-mon[28138]: from='client.? 172.21.15.72:0/997013101' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d0317850-7b2d-4850-b2cf-0b919505c1fa"}]: dispatch 2024-04-26T21:50:08.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:50:07 smithi072 ceph-mon[28138]: from='client.? 172.21.15.72:0/997013101' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d0317850-7b2d-4850-b2cf-0b919505c1fa"}]': finished 2024-04-26T21:50:08.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:50:07 smithi072 ceph-mon[28138]: osdmap e66: 10 total, 9 up, 10 in 2024-04-26T21:50:08.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:50:07 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T21:50:08.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:50:07 smithi164 ceph-mon[31916]: from='client.? 172.21.15.72:0/997013101' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d0317850-7b2d-4850-b2cf-0b919505c1fa"}]: dispatch 2024-04-26T21:50:08.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:50:07 smithi164 ceph-mon[31916]: from='client.? 172.21.15.72:0/997013101' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d0317850-7b2d-4850-b2cf-0b919505c1fa"}]': finished 2024-04-26T21:50:08.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:50:07 smithi164 ceph-mon[31916]: osdmap e66: 10 total, 9 up, 10 in 2024-04-26T21:50:08.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:50:07 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T21:50:09.064 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:50:08 smithi076 ceph-mon[31823]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 804 GiB / 805 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-04-26T21:50:09.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:50:08 smithi076 ceph-mon[31823]: from='client.? 172.21.15.72:0/1210965854' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:50:09.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:50:08 smithi072 ceph-mon[28138]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 804 GiB / 805 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-04-26T21:50:09.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:50:08 smithi072 ceph-mon[28138]: from='client.? 172.21.15.72:0/1210965854' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:50:09.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:50:08 smithi164 ceph-mon[31916]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 804 GiB / 805 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-04-26T21:50:09.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:50:08 smithi164 ceph-mon[31916]: from='client.? 172.21.15.72:0/1210965854' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:50:11.064 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:50:10 smithi076 ceph-mon[31823]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 804 GiB / 805 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-04-26T21:50:11.071 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:50:10 smithi072 ceph-mon[28138]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 804 GiB / 805 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-04-26T21:50:11.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:50:10 smithi164 ceph-mon[31916]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 804 GiB / 805 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-04-26T21:50:13.064 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:50:12 smithi076 ceph-mon[31823]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-26T21:50:13.071 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:50:12 smithi072 ceph-mon[28138]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-26T21:50:13.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:50:12 smithi164 ceph-mon[31916]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-26T21:50:14.931 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:50:14 smithi072 ceph-mon[28138]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:50:15.064 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:50:14 smithi076 ceph-mon[31823]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:50:15.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:50:14 smithi164 ceph-mon[31916]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:50:17.064 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:50:16 smithi076 ceph-mon[31823]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:50:17.071 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:50:16 smithi072 ceph-mon[28138]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:50:17.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:50:16 smithi164 ceph-mon[31916]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:50:18.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:50:17 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-26T21:50:18.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:50:17 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:50:18.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:50:17 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-26T21:50:18.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:50:17 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:50:18.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:50:17 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-26T21:50:18.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:50:17 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:50:19.161 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:50:18 smithi072 ceph-mon[28138]: Deploying daemon osd.9 on smithi072 2024-04-26T21:50:19.161 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:50:18 smithi072 ceph-mon[28138]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:50:19.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:50:18 smithi164 ceph-mon[31916]: Deploying daemon osd.9 on smithi072 2024-04-26T21:50:19.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:50:18 smithi164 ceph-mon[31916]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:50:19.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:50:18 smithi076 ceph-mon[31823]: Deploying daemon osd.9 on smithi072 2024-04-26T21:50:19.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:50:18 smithi076 ceph-mon[31823]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:50:21.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:50:20 smithi164 ceph-mon[31916]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:50:21.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:50:20 smithi076 ceph-mon[31823]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:50:21.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:50:20 smithi072 ceph-mon[28138]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:50:23.165 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:50:22 smithi072 ceph-mon[28138]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:50:23.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:50:22 smithi164 ceph-mon[31916]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:50:23.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:50:22 smithi076 ceph-mon[31823]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:50:25.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:50:24 smithi164 ceph-mon[31916]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:50:25.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:50:24 smithi076 ceph-mon[31823]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:50:25.318 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:50:24 smithi072 ceph-mon[28138]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:50:27.071 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:50:26 smithi072 ceph-mon[28138]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:50:27.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:50:26 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:50:27.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:50:26 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:50:27.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:50:26 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:50:27.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:50:26 smithi164 ceph-mon[31916]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:50:27.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:50:26 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:50:27.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:50:26 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:50:27.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:50:26 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:50:27.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:50:26 smithi076 ceph-mon[31823]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:50:27.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:50:26 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:50:27.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:50:26 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:50:27.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:50:26 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:50:28.965 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:50:28 smithi072 ceph-mon[28138]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:50:29.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:50:28 smithi164 ceph-mon[31916]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:50:29.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:50:28 smithi076 ceph-mon[31823]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:50:30.588 INFO:teuthology.orchestra.run.smithi072.stdout:Created osd(s) 9 on host 'smithi072' 2024-04-26T21:50:30.588 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:50:30.587+0000 7f91f3400700 1 -- 172.21.15.72:0/2244041209 <== mgr.14152 v2:172.21.15.72:6800/588150172 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f91fc063860 con 0x7f91e00a3c40 2024-04-26T21:50:30.591 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:50:30.590+0000 7f91fbe00700 1 -- 172.21.15.72:0/2244041209 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f91e00a3c40 msgr2=0x7f91e00a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:30.591 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:50:30.590+0000 7f91fbe00700 1 --2- 172.21.15.72:0/2244041209 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f91e00a3c40 0x7f91e00a6100 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f91fc077830 tx=0x7f91ec00b2c0 comp rx=0 tx=0).stop 2024-04-26T21:50:30.591 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:50:30.590+0000 7f91fbe00700 1 -- 172.21.15.72:0/2244041209 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f91fc10f9a0 msgr2=0x7f91fc1a6ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:30.591 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:50:30.590+0000 7f91fbe00700 1 --2- 172.21.15.72:0/2244041209 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f91fc10f9a0 0x7f91fc1a6ec0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f91f400b070 tx=0x7f91f4009bf0 comp rx=0 tx=0).stop 2024-04-26T21:50:30.591 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:50:30.591+0000 7f91fbe00700 1 -- 172.21.15.72:0/2244041209 shutdown_connections 2024-04-26T21:50:30.591 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:50:30.591+0000 7f91fbe00700 1 --2- 172.21.15.72:0/2244041209 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f91e00a3c40 0x7f91e00a6100 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:30.592 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:50:30.591+0000 7f91fbe00700 1 --2- 172.21.15.72:0/2244041209 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f91fc10f9a0 0x7f91fc1a6ec0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:30.592 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:50:30.591+0000 7f91fbe00700 1 --2- 172.21.15.72:0/2244041209 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f91fc075e80 0x7f91fc19f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:30.592 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:50:30.591+0000 7f91fbe00700 1 --2- 172.21.15.72:0/2244041209 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f91fc077450 0x7f91fc19fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:30.592 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:50:30.591+0000 7f91fbe00700 1 -- 172.21.15.72:0/2244041209 >> 172.21.15.72:0/2244041209 conn(0x7f91fc0fe960 msgr2=0x7f91fc102d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:50:30.592 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:50:30.591+0000 7f91fbe00700 1 -- 172.21.15.72:0/2244041209 shutdown_connections 2024-04-26T21:50:30.592 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:50:30.591+0000 7f91fbe00700 1 -- 172.21.15.72:0/2244041209 wait complete. 2024-04-26T21:50:30.941 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:50:30 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:50:30.941 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:50:30 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:50:30.941 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:50:30 smithi164 ceph-mon[31916]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:50:30.941 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:50:30 smithi164 ceph-mon[31916]: from='osd.9 [v2:172.21.15.72:6826/1273118485,v1:172.21.15.72:6827/1273118485]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-26T21:50:30.941 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:50:30 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:50:30.941 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:50:30 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:50:31.006 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:50:30 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:50:31.006 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:50:30 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:50:31.007 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:50:30 smithi072 ceph-mon[28138]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:50:31.007 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:50:30 smithi072 ceph-mon[28138]: from='osd.9 [v2:172.21.15.72:6826/1273118485,v1:172.21.15.72:6827/1273118485]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-26T21:50:31.007 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:50:30 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:50:31.007 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:50:30 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:50:31.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:50:30 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:50:31.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:50:30 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:50:31.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:50:30 smithi076 ceph-mon[31823]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:50:31.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:50:30 smithi076 ceph-mon[31823]: from='osd.9 [v2:172.21.15.72:6826/1273118485,v1:172.21.15.72:6827/1273118485]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-26T21:50:31.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:50:30 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:50:31.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:50:30 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:50:31.427 DEBUG:teuthology.orchestra.run.smithi072:osd.9> sudo journalctl -f -n 0 -u ceph-a1780950-0415-11ef-bc93-c7b262605968@osd.9.service 2024-04-26T21:50:31.430 INFO:tasks.cephadm:Deploying osd.10 on smithi076 with /dev/vg_nvme/lv_1... 2024-04-26T21:50:31.431 DEBUG:teuthology.orchestra.run.smithi076:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a1780950-0415-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-26T21:50:31.642 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:50:31 smithi072 ceph-mon[28138]: from='osd.9 [v2:172.21.15.72:6826/1273118485,v1:172.21.15.72:6827/1273118485]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-26T21:50:31.642 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:50:31 smithi072 ceph-mon[28138]: osdmap e67: 10 total, 9 up, 10 in 2024-04-26T21:50:31.642 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:50:31 smithi072 ceph-mon[28138]: from='osd.9 [v2:172.21.15.72:6826/1273118485,v1:172.21.15.72:6827/1273118485]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi072", "root=default"]}]: dispatch 2024-04-26T21:50:31.642 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:50:31 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T21:50:31.814 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:50:31 smithi076 ceph-mon[31823]: from='osd.9 [v2:172.21.15.72:6826/1273118485,v1:172.21.15.72:6827/1273118485]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-26T21:50:31.814 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:50:31 smithi076 ceph-mon[31823]: osdmap e67: 10 total, 9 up, 10 in 2024-04-26T21:50:31.814 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:50:31 smithi076 ceph-mon[31823]: from='osd.9 [v2:172.21.15.72:6826/1273118485,v1:172.21.15.72:6827/1273118485]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi072", "root=default"]}]: dispatch 2024-04-26T21:50:31.814 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:50:31 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T21:50:31.869 INFO:teuthology.orchestra.run.smithi076.stderr:Inferring config /var/lib/ceph/a1780950-0415-11ef-bc93-c7b262605968/mon.b/config 2024-04-26T21:50:31.940 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:50:31 smithi164 ceph-mon[31916]: from='osd.9 [v2:172.21.15.72:6826/1273118485,v1:172.21.15.72:6827/1273118485]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-26T21:50:31.940 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:50:31 smithi164 ceph-mon[31916]: osdmap e67: 10 total, 9 up, 10 in 2024-04-26T21:50:31.941 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:50:31 smithi164 ceph-mon[31916]: from='osd.9 [v2:172.21.15.72:6826/1273118485,v1:172.21.15.72:6827/1273118485]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi072", "root=default"]}]: dispatch 2024-04-26T21:50:31.941 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:50:31 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T21:50:32.940 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:50:32 smithi164 ceph-mon[31916]: from='osd.9 [v2:172.21.15.72:6826/1273118485,v1:172.21.15.72:6827/1273118485]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi072", "root=default"]}]': finished 2024-04-26T21:50:32.941 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:50:32 smithi164 ceph-mon[31916]: osdmap e68: 10 total, 9 up, 10 in 2024-04-26T21:50:32.941 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:50:32 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T21:50:32.941 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:50:32 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T21:50:32.941 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:50:32 smithi164 ceph-mon[31916]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:50:32.941 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:50:32 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T21:50:32.955 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:50:32 smithi072 ceph-mon[28138]: from='osd.9 [v2:172.21.15.72:6826/1273118485,v1:172.21.15.72:6827/1273118485]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi072", "root=default"]}]': finished 2024-04-26T21:50:32.955 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:50:32 smithi072 ceph-mon[28138]: osdmap e68: 10 total, 9 up, 10 in 2024-04-26T21:50:32.955 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:50:32 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T21:50:32.955 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:50:32 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T21:50:32.955 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:50:32 smithi072 ceph-mon[28138]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:50:32.955 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:50:32 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T21:50:32.956 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 21:50:32 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T21:50:32.680+0000 7f785fa00700 -1 osd.9 0 waiting for initial osdmap 2024-04-26T21:50:32.956 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 21:50:32 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T21:50:32.723+0000 7f7855800700 -1 osd.9 68 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T21:50:33.046 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:50:32 smithi076 ceph-mon[31823]: from='osd.9 [v2:172.21.15.72:6826/1273118485,v1:172.21.15.72:6827/1273118485]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi072", "root=default"]}]': finished 2024-04-26T21:50:33.046 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:50:32 smithi076 ceph-mon[31823]: osdmap e68: 10 total, 9 up, 10 in 2024-04-26T21:50:33.046 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:50:32 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T21:50:33.046 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:50:32 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T21:50:33.046 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:50:32 smithi076 ceph-mon[31823]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:50:33.046 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:50:32 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T21:50:33.815 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:50:33 smithi076 ceph-mon[31823]: purged_snaps scrub starts 2024-04-26T21:50:33.815 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:50:33 smithi076 ceph-mon[31823]: purged_snaps scrub ok 2024-04-26T21:50:33.815 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:50:33 smithi076 ceph-mon[31823]: from='osd.9 [v2:172.21.15.72:6826/1273118485,v1:172.21.15.72:6827/1273118485]' entity='osd.9' 2024-04-26T21:50:33.815 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:50:33 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T21:50:34.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:50:33 smithi072 ceph-mon[28138]: purged_snaps scrub starts 2024-04-26T21:50:34.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:50:33 smithi072 ceph-mon[28138]: purged_snaps scrub ok 2024-04-26T21:50:34.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:50:33 smithi072 ceph-mon[28138]: from='osd.9 [v2:172.21.15.72:6826/1273118485,v1:172.21.15.72:6827/1273118485]' entity='osd.9' 2024-04-26T21:50:34.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:50:33 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T21:50:34.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:50:33 smithi164 ceph-mon[31916]: purged_snaps scrub starts 2024-04-26T21:50:34.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:50:33 smithi164 ceph-mon[31916]: purged_snaps scrub ok 2024-04-26T21:50:34.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:50:33 smithi164 ceph-mon[31916]: from='osd.9 [v2:172.21.15.72:6826/1273118485,v1:172.21.15.72:6827/1273118485]' entity='osd.9' 2024-04-26T21:50:34.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:50:33 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T21:50:34.771 INFO:teuthology.orchestra.run.smithi076.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-26T21:50:34.771 INFO:teuthology.orchestra.run.smithi076.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-26T21:50:34.771 INFO:teuthology.orchestra.run.smithi076.stderr: stderr: 10+0 records in 2024-04-26T21:50:34.771 INFO:teuthology.orchestra.run.smithi076.stderr:10+0 records out 2024-04-26T21:50:34.771 INFO:teuthology.orchestra.run.smithi076.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.033972 s, 309 MB/s 2024-04-26T21:50:34.772 INFO:teuthology.orchestra.run.smithi076.stderr:--> Zapping successful for: 2024-04-26T21:50:35.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:50:34 smithi076 ceph-mon[31823]: osd.9 [v2:172.21.15.72:6826/1273118485,v1:172.21.15.72:6827/1273118485] boot 2024-04-26T21:50:35.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:50:34 smithi076 ceph-mon[31823]: osdmap e69: 10 total, 10 up, 10 in 2024-04-26T21:50:35.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:50:34 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T21:50:35.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:50:34 smithi076 ceph-mon[31823]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:50:35.071 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:50:34 smithi072 ceph-mon[28138]: osd.9 [v2:172.21.15.72:6826/1273118485,v1:172.21.15.72:6827/1273118485] boot 2024-04-26T21:50:35.071 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:50:34 smithi072 ceph-mon[28138]: osdmap e69: 10 total, 10 up, 10 in 2024-04-26T21:50:35.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:50:34 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T21:50:35.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:50:34 smithi072 ceph-mon[28138]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:50:35.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:50:34 smithi164 ceph-mon[31916]: osd.9 [v2:172.21.15.72:6826/1273118485,v1:172.21.15.72:6827/1273118485] boot 2024-04-26T21:50:35.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:50:34 smithi164 ceph-mon[31916]: osdmap e69: 10 total, 10 up, 10 in 2024-04-26T21:50:35.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:50:34 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T21:50:35.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:50:34 smithi164 ceph-mon[31916]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:50:35.699 DEBUG:teuthology.orchestra.run.smithi076:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a1780950-0415-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi076:vg_nvme/lv_1 2024-04-26T21:50:35.997 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:50:35 smithi072 ceph-mon[28138]: osdmap e70: 10 total, 10 up, 10 in 2024-04-26T21:50:36.064 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:50:35 smithi076 ceph-mon[31823]: osdmap e70: 10 total, 10 up, 10 in 2024-04-26T21:50:36.104 INFO:teuthology.orchestra.run.smithi076.stderr:Inferring config /var/lib/ceph/a1780950-0415-11ef-bc93-c7b262605968/mon.b/config 2024-04-26T21:50:36.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:50:35 smithi164 ceph-mon[31916]: osdmap e70: 10 total, 10 up, 10 in 2024-04-26T21:50:36.815 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:50:36 smithi076 ceph-mon[31823]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:50:36.815 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:50:36 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:50:36.815 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:50:36 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:50:36.815 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:50:36 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:50:36.815 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:50:36 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:50:36.815 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:50:36 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:50:36.816 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:50:36 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:50:36.816 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:50:36 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:50:36.816 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:50:36 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:50:36.816 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:50:36 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:50:36.816 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:50:36 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:50:36.816 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:50:36 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:50:36.816 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:50:36 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:50:36.816 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:50:36 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:50:36.816 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:50:36 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:50:37.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:50:36 smithi072 ceph-mon[28138]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:50:37.073 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:50:36 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:50:37.073 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:50:36 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:50:37.073 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:50:36 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:50:37.073 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:50:36 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:50:37.073 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:50:36 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:50:37.073 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:50:36 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:50:37.073 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:50:36 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:50:37.073 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:50:36 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:50:37.073 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:50:36 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:50:37.073 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:50:36 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:50:37.073 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:50:36 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:50:37.074 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:50:36 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:50:37.074 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:50:36 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:50:37.074 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:50:36 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:50:37.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:50:36 smithi164 ceph-mon[31916]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:50:37.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:50:36 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:50:37.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:50:36 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:50:37.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:50:36 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:50:37.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:50:36 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:50:37.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:50:36 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:50:37.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:50:36 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:50:37.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:50:36 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:50:37.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:50:36 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:50:37.192 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:50:36 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:50:37.192 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:50:36 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:50:37.192 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:50:36 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:50:37.192 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:50:36 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:50:37.192 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:50:36 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:50:37.192 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:50:36 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:50:37.814 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:50:37 smithi076 ceph-mon[31823]: Detected new or changed devices on smithi072 2024-04-26T21:50:37.814 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:50:37 smithi076 ceph-mon[31823]: Adjusting osd_memory_target on smithi072 to 4272M 2024-04-26T21:50:38.027 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:50:37 smithi164 ceph-mon[31916]: Detected new or changed devices on smithi072 2024-04-26T21:50:38.027 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:50:37 smithi164 ceph-mon[31916]: Adjusting osd_memory_target on smithi072 to 4272M 2024-04-26T21:50:38.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:50:37 smithi072 ceph-mon[28138]: Detected new or changed devices on smithi072 2024-04-26T21:50:38.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:50:37 smithi072 ceph-mon[28138]: Adjusting osd_memory_target on smithi072 to 4272M 2024-04-26T21:50:38.323 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:50:38.322+0000 7faf99600700 1 -- 172.21.15.76:0/588733652 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faf9410b660 msgr2=0x7faf94111f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:38.323 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:50:38.322+0000 7faf99600700 1 --2- 172.21.15.76:0/588733652 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faf9410b660 0x7faf94111f10 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7faf8c00b0d0 tx=0x7faf8c00b3e0 comp rx=0 tx=0).stop 2024-04-26T21:50:38.323 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:50:38.323+0000 7faf99600700 1 -- 172.21.15.76:0/588733652 shutdown_connections 2024-04-26T21:50:38.324 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:50:38.323+0000 7faf99600700 1 --2- 172.21.15.76:0/588733652 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7faf94109ac0 0x7faf94109ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:38.324 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:50:38.323+0000 7faf99600700 1 --2- 172.21.15.76:0/588733652 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faf9410b660 0x7faf94111f10 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:38.324 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:50:38.323+0000 7faf99600700 1 --2- 172.21.15.76:0/588733652 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faf9410acc0 0x7faf9410b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:38.324 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:50:38.323+0000 7faf99600700 1 -- 172.21.15.76:0/588733652 >> 172.21.15.76:0/588733652 conn(0x7faf94078a90 msgr2=0x7faf9407aed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:50:38.324 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:50:38.323+0000 7faf99600700 1 -- 172.21.15.76:0/588733652 shutdown_connections 2024-04-26T21:50:38.324 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:50:38.323+0000 7faf99600700 1 -- 172.21.15.76:0/588733652 wait complete. 2024-04-26T21:50:38.326 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:50:38.324+0000 7faf99600700 1 Processor -- start 2024-04-26T21:50:38.326 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:50:38.324+0000 7faf99600700 1 -- start start 2024-04-26T21:50:38.326 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:50:38.324+0000 7faf99600700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faf94109ac0 0x7faf9419fad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:38.326 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:50:38.324+0000 7faf99600700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7faf9410acc0 0x7faf941a0010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:38.326 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:50:38.324+0000 7faf99600700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faf9410b660 0x7faf941a7090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:38.326 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:50:38.324+0000 7faf99600700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7faf94115460 con 0x7faf9410acc0 2024-04-26T21:50:38.326 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:50:38.324+0000 7faf99600700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7faf941152e0 con 0x7faf9410b660 2024-04-26T21:50:38.326 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:50:38.324+0000 7faf99600700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7faf941155e0 con 0x7faf94109ac0 2024-04-26T21:50:38.326 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:50:38.325+0000 7faf93400700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faf9410b660 0x7faf941a7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:38.327 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:50:38.325+0000 7faf93400700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faf9410b660 0x7faf941a7090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:46524/0 (socket says 172.21.15.76:46524) 2024-04-26T21:50:38.327 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:50:38.325+0000 7faf93400700 1 -- 172.21.15.76:0/634168943 learned_addr learned my addr 172.21.15.76:0/634168943 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-26T21:50:38.327 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:50:38.325+0000 7faf92a00700 1 --2- 172.21.15.76:0/634168943 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faf94109ac0 0x7faf9419fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:38.327 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:50:38.325+0000 7faf92000700 1 --2- 172.21.15.76:0/634168943 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7faf9410acc0 0x7faf941a0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:38.327 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:50:38.325+0000 7faf93400700 1 -- 172.21.15.76:0/634168943 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faf94109ac0 msgr2=0x7faf9419fad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:38.327 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:50:38.325+0000 7faf93400700 1 --2- 172.21.15.76:0/634168943 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faf94109ac0 0x7faf9419fad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:38.327 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:50:38.325+0000 7faf93400700 1 -- 172.21.15.76:0/634168943 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7faf9410acc0 msgr2=0x7faf941a0010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:38.327 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:50:38.325+0000 7faf93400700 1 --2- 172.21.15.76:0/634168943 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7faf9410acc0 0x7faf941a0010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:38.327 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:50:38.325+0000 7faf93400700 1 -- 172.21.15.76:0/634168943 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf941a7730 con 0x7faf9410b660 2024-04-26T21:50:38.327 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:50:38.325+0000 7faf92a00700 1 --2- 172.21.15.76:0/634168943 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faf94109ac0 0x7faf9419fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:50:38.327 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:50:38.325+0000 7faf92000700 1 --2- 172.21.15.76:0/634168943 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7faf9410acc0 0x7faf941a0010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:50:38.327 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:50:38.325+0000 7faf93400700 1 --2- 172.21.15.76:0/634168943 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faf9410b660 0x7faf941a7090 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7faf8c007980 tx=0x7faf8c007a60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:38.327 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:50:38.326+0000 7faf8b400700 1 -- 172.21.15.76:0/634168943 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf8c01f070 con 0x7faf9410b660 2024-04-26T21:50:38.327 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:50:38.326+0000 7faf8b400700 1 -- 172.21.15.76:0/634168943 <== mon.2 v2:172.21.15.76:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7faf8c005500 con 0x7faf9410b660 2024-04-26T21:50:38.328 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:50:38.326+0000 7faf8b400700 1 -- 172.21.15.76:0/634168943 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf8c01b690 con 0x7faf9410b660 2024-04-26T21:50:38.328 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:50:38.326+0000 7faf99600700 1 -- 172.21.15.76:0/634168943 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf941a79c0 con 0x7faf9410b660 2024-04-26T21:50:38.328 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:50:38.326+0000 7faf99600700 1 -- 172.21.15.76:0/634168943 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf941a7dc0 con 0x7faf9410b660 2024-04-26T21:50:38.328 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:50:38.327+0000 7faf99600700 1 -- 172.21.15.76:0/634168943 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf9406a110 con 0x7faf9410b660 2024-04-26T21:50:38.329 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:50:38.329+0000 7faf8b400700 1 -- 172.21.15.76:0/634168943 <== mon.2 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7faf8c022070 con 0x7faf9410b660 2024-04-26T21:50:38.330 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:50:38.330+0000 7faf8b400700 1 --2- 172.21.15.76:0/634168943 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7faf780a3cc0 0x7faf780a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:38.330 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:50:38.330+0000 7faf8b400700 1 -- 172.21.15.76:0/634168943 <== mon.2 v2:172.21.15.76:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 6261+0+0 (secure 0 0 0) 0x7faf8c00f020 con 0x7faf9410b660 2024-04-26T21:50:38.331 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:50:38.331+0000 7faf92a00700 1 --2- 172.21.15.76:0/634168943 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7faf780a3cc0 0x7faf780a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:38.332 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:50:38.332+0000 7faf92a00700 1 --2- 172.21.15.76:0/634168943 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7faf780a3cc0 0x7faf780a6180 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7faf9410ab20 tx=0x7faf84009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:38.335 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:50:38.335+0000 7faf8b400700 1 -- 172.21.15.76:0/634168943 <== mon.2 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7faf8c029020 con 0x7faf9410b660 2024-04-26T21:50:38.579 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:50:38.579+0000 7faf99600700 1 -- 172.21.15.76:0/634168943 --> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi076:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7faf94063860 con 0x7faf780a3cc0 2024-04-26T21:50:39.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:50:38 smithi076 ceph-mon[31823]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:50:39.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:50:38 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:50:39.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:50:38 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:50:39.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:50:38 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:50:39.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:50:38 smithi072 ceph-mon[28138]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:50:39.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:50:38 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:50:39.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:50:38 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:50:39.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:50:38 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:50:39.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:50:38 smithi164 ceph-mon[31916]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:50:39.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:50:38 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:50:39.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:50:38 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:50:39.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:50:38 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:50:40.064 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:50:39 smithi076 ceph-mon[31823]: from='client.34328 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi076:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:50:40.071 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:50:39 smithi072 ceph-mon[28138]: from='client.34328 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi076:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:50:40.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:50:39 smithi164 ceph-mon[31916]: from='client.34328 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi076:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:50:41.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:50:40 smithi076 ceph-mon[31823]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:50:41.071 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:50:40 smithi072 ceph-mon[28138]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:50:41.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:50:40 smithi164 ceph-mon[31916]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:50:43.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:50:42 smithi076 ceph-mon[31823]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:50:43.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:50:42 smithi076 ceph-mon[31823]: from='client.? 172.21.15.76:0/245434158' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e0aa38d8-6011-4d2a-b67b-5bff6d999e47"}]: dispatch 2024-04-26T21:50:43.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:50:42 smithi076 ceph-mon[31823]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e0aa38d8-6011-4d2a-b67b-5bff6d999e47"}]: dispatch 2024-04-26T21:50:43.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:50:42 smithi076 ceph-mon[31823]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e0aa38d8-6011-4d2a-b67b-5bff6d999e47"}]': finished 2024-04-26T21:50:43.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:50:42 smithi076 ceph-mon[31823]: osdmap e71: 11 total, 10 up, 11 in 2024-04-26T21:50:43.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:50:42 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T21:50:43.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:50:42 smithi072 ceph-mon[28138]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:50:43.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:50:42 smithi072 ceph-mon[28138]: from='client.? 172.21.15.76:0/245434158' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e0aa38d8-6011-4d2a-b67b-5bff6d999e47"}]: dispatch 2024-04-26T21:50:43.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:50:42 smithi072 ceph-mon[28138]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e0aa38d8-6011-4d2a-b67b-5bff6d999e47"}]: dispatch 2024-04-26T21:50:43.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:50:42 smithi072 ceph-mon[28138]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e0aa38d8-6011-4d2a-b67b-5bff6d999e47"}]': finished 2024-04-26T21:50:43.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:50:42 smithi072 ceph-mon[28138]: osdmap e71: 11 total, 10 up, 11 in 2024-04-26T21:50:43.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:50:42 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T21:50:43.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:50:42 smithi164 ceph-mon[31916]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:50:43.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:50:42 smithi164 ceph-mon[31916]: from='client.? 172.21.15.76:0/245434158' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e0aa38d8-6011-4d2a-b67b-5bff6d999e47"}]: dispatch 2024-04-26T21:50:43.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:50:42 smithi164 ceph-mon[31916]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e0aa38d8-6011-4d2a-b67b-5bff6d999e47"}]: dispatch 2024-04-26T21:50:43.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:50:42 smithi164 ceph-mon[31916]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e0aa38d8-6011-4d2a-b67b-5bff6d999e47"}]': finished 2024-04-26T21:50:43.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:50:42 smithi164 ceph-mon[31916]: osdmap e71: 11 total, 10 up, 11 in 2024-04-26T21:50:43.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:50:42 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T21:50:44.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:50:43 smithi164 ceph-mon[31916]: from='client.? 172.21.15.76:0/2446918367' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:50:44.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:50:43 smithi076 ceph-mon[31823]: from='client.? 172.21.15.76:0/2446918367' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:50:44.321 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:50:43 smithi072 ceph-mon[28138]: from='client.? 172.21.15.76:0/2446918367' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:50:45.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:50:44 smithi164 ceph-mon[31916]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:50:45.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:50:44 smithi076 ceph-mon[31823]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:50:45.321 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:50:44 smithi072 ceph-mon[28138]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:50:47.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:50:46 smithi164 ceph-mon[31916]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:50:47.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:50:46 smithi076 ceph-mon[31823]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:50:47.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:50:46 smithi072 ceph-mon[28138]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:50:49.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:50:48 smithi164 ceph-mon[31916]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:50:49.293 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:50:48 smithi076 ceph-mon[31823]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:50:49.321 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:50:48 smithi072 ceph-mon[28138]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:50:51.027 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:50:50 smithi076 ceph-mon[31823]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:50:51.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:50:50 smithi164 ceph-mon[31916]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:50:51.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:50:50 smithi072 ceph-mon[28138]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:50:52.902 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:50:52 smithi076 ceph-mon[31823]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:50:52.902 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:50:52 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-26T21:50:52.902 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:50:52 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:50:53.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:50:52 smithi164 ceph-mon[31916]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:50:53.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:50:52 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-26T21:50:53.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:50:52 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:50:53.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:50:52 smithi072 ceph-mon[28138]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:50:53.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:50:52 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-26T21:50:53.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:50:52 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:50:54.037 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:50:53 smithi076 ceph-mon[31823]: Deploying daemon osd.10 on smithi076 2024-04-26T21:50:54.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:50:53 smithi164 ceph-mon[31916]: Deploying daemon osd.10 on smithi076 2024-04-26T21:50:54.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:50:53 smithi072 ceph-mon[28138]: Deploying daemon osd.10 on smithi076 2024-04-26T21:50:55.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:50:54 smithi164 ceph-mon[31916]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:50:55.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:50:54 smithi076 ceph-mon[31823]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:50:55.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:50:54 smithi072 ceph-mon[28138]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:50:57.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:50:56 smithi164 ceph-mon[31916]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:50:57.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:50:56 smithi076 ceph-mon[31823]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:50:57.321 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:50:56 smithi072 ceph-mon[28138]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:50:59.064 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:50:58 smithi076 ceph-mon[31823]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:50:59.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:50:58 smithi164 ceph-mon[31916]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:50:59.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:50:58 smithi072 ceph-mon[28138]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:51:01.126 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:51:00 smithi076 ceph-mon[31823]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:51:01.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:51:00 smithi164 ceph-mon[31916]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:51:01.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:51:00 smithi072 ceph-mon[28138]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:51:02.564 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:51:02 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:51:02.565 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:51:02 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:51:02.565 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:51:02 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:51:02.565 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:51:02 smithi076 ceph-mon[31823]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:51:02.690 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:51:02 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:51:02.691 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:51:02 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:51:02.691 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:51:02 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:51:02.691 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:51:02 smithi072 ceph-mon[28138]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:51:02.691 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:51:02 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:51:02.691 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:51:02 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:51:02.691 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:51:02 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:51:02.691 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:51:02 smithi164 ceph-mon[31916]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:51:05.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:51:04 smithi076 ceph-mon[31823]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:51:05.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:51:04 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:51:05.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:51:04 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:51:05.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:51:04 smithi072 ceph-mon[28138]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:51:05.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:51:04 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:51:05.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:51:04 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:51:05.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:51:04 smithi164 ceph-mon[31916]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:51:05.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:51:04 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:51:05.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:51:04 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:51:06.064 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:51:05 smithi076 ceph-mon[31823]: from='osd.10 [v2:172.21.15.76:6824/2808954459,v1:172.21.15.76:6825/2808954459]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-26T21:51:06.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:51:05 smithi076 ceph-mon[31823]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-26T21:51:06.071 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:51:05 smithi072 ceph-mon[28138]: from='osd.10 [v2:172.21.15.76:6824/2808954459,v1:172.21.15.76:6825/2808954459]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-26T21:51:06.071 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:51:05 smithi072 ceph-mon[28138]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-26T21:51:06.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:51:05 smithi164 ceph-mon[31916]: from='osd.10 [v2:172.21.15.76:6824/2808954459,v1:172.21.15.76:6825/2808954459]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-26T21:51:06.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:51:05 smithi164 ceph-mon[31916]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-26T21:51:06.355 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:51:06.353+0000 7faf8b400700 1 -- 172.21.15.76:0/634168943 <== mgr.14152 v2:172.21.15.72:6800/588150172 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7faf94063860 con 0x7faf780a3cc0 2024-04-26T21:51:06.355 INFO:teuthology.orchestra.run.smithi076.stdout:Created osd(s) 10 on host 'smithi076' 2024-04-26T21:51:06.358 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:51:06.356+0000 7faf99600700 1 -- 172.21.15.76:0/634168943 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7faf780a3cc0 msgr2=0x7faf780a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:06.358 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:51:06.356+0000 7faf99600700 1 --2- 172.21.15.76:0/634168943 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7faf780a3cc0 0x7faf780a6180 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7faf9410ab20 tx=0x7faf84009260 comp rx=0 tx=0).stop 2024-04-26T21:51:06.358 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:51:06.356+0000 7faf99600700 1 -- 172.21.15.76:0/634168943 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faf9410b660 msgr2=0x7faf941a7090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:06.358 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:51:06.356+0000 7faf99600700 1 --2- 172.21.15.76:0/634168943 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faf9410b660 0x7faf941a7090 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7faf8c007980 tx=0x7faf8c007a60 comp rx=0 tx=0).stop 2024-04-26T21:51:06.358 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:51:06.356+0000 7faf99600700 1 -- 172.21.15.76:0/634168943 shutdown_connections 2024-04-26T21:51:06.358 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:51:06.356+0000 7faf99600700 1 --2- 172.21.15.76:0/634168943 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7faf780a3cc0 0x7faf780a6180 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:06.358 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:51:06.356+0000 7faf99600700 1 --2- 172.21.15.76:0/634168943 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faf9410b660 0x7faf941a7090 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:06.358 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:51:06.356+0000 7faf99600700 1 --2- 172.21.15.76:0/634168943 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faf94109ac0 0x7faf9419fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:06.358 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:51:06.356+0000 7faf99600700 1 --2- 172.21.15.76:0/634168943 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7faf9410acc0 0x7faf941a0010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:06.358 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:51:06.356+0000 7faf99600700 1 -- 172.21.15.76:0/634168943 >> 172.21.15.76:0/634168943 conn(0x7faf94078a90 msgr2=0x7faf9410e040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:51:06.358 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:51:06.356+0000 7faf99600700 1 -- 172.21.15.76:0/634168943 shutdown_connections 2024-04-26T21:51:06.358 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-26T21:51:06.356+0000 7faf99600700 1 -- 172.21.15.76:0/634168943 wait complete. 2024-04-26T21:51:07.034 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:51:06 smithi076 ceph-mon[31823]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:51:07.034 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:51:06 smithi076 ceph-mon[31823]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-26T21:51:07.035 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:51:06 smithi076 ceph-mon[31823]: from='osd.10 [v2:172.21.15.76:6824/2808954459,v1:172.21.15.76:6825/2808954459]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi076", "root=default"]}]: dispatch 2024-04-26T21:51:07.035 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:51:06 smithi076 ceph-mon[31823]: osdmap e72: 11 total, 10 up, 11 in 2024-04-26T21:51:07.035 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:51:06 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T21:51:07.035 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:51:06 smithi076 ceph-mon[31823]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi076", "root=default"]}]: dispatch 2024-04-26T21:51:07.035 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:51:06 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:51:07.035 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:51:06 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:51:07.035 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:51:06 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:51:07.035 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:51:06 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:51:07.035 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:51:06 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:51:07.035 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:51:06 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:51:07.087 DEBUG:teuthology.orchestra.run.smithi076:osd.10> sudo journalctl -f -n 0 -u ceph-a1780950-0415-11ef-bc93-c7b262605968@osd.10.service 2024-04-26T21:51:07.091 INFO:tasks.cephadm:Deploying osd.11 on smithi164 with /dev/vg_nvme/lv_1... 2024-04-26T21:51:07.092 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a1780950-0415-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-26T21:51:07.167 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:51:06 smithi164 ceph-mon[31916]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:51:07.167 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:51:06 smithi164 ceph-mon[31916]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-26T21:51:07.167 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:51:06 smithi164 ceph-mon[31916]: from='osd.10 [v2:172.21.15.76:6824/2808954459,v1:172.21.15.76:6825/2808954459]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi076", "root=default"]}]: dispatch 2024-04-26T21:51:07.167 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:51:06 smithi164 ceph-mon[31916]: osdmap e72: 11 total, 10 up, 11 in 2024-04-26T21:51:07.167 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:51:06 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T21:51:07.167 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:51:06 smithi164 ceph-mon[31916]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi076", "root=default"]}]: dispatch 2024-04-26T21:51:07.167 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:51:06 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:51:07.168 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:51:06 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:51:07.168 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:51:06 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:51:07.168 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:51:06 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:51:07.168 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:51:06 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:51:07.168 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:51:06 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:51:07.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:51:06 smithi072 ceph-mon[28138]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:51:07.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:51:06 smithi072 ceph-mon[28138]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-26T21:51:07.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:51:06 smithi072 ceph-mon[28138]: from='osd.10 [v2:172.21.15.76:6824/2808954459,v1:172.21.15.76:6825/2808954459]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi076", "root=default"]}]: dispatch 2024-04-26T21:51:07.323 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:51:06 smithi072 ceph-mon[28138]: osdmap e72: 11 total, 10 up, 11 in 2024-04-26T21:51:07.323 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:51:06 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T21:51:07.323 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:51:06 smithi072 ceph-mon[28138]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi076", "root=default"]}]: dispatch 2024-04-26T21:51:07.323 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:51:06 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:51:07.323 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:51:06 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:51:07.323 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:51:06 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:51:07.323 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:51:06 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:51:07.323 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:51:06 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:51:07.323 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:51:06 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:51:07.521 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/a1780950-0415-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T21:51:07.920 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:51:07 smithi164 ceph-mon[31916]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi076", "root=default"]}]': finished 2024-04-26T21:51:07.920 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:51:07 smithi164 ceph-mon[31916]: osdmap e73: 11 total, 10 up, 11 in 2024-04-26T21:51:07.920 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:51:07 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T21:51:07.920 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:51:07 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T21:51:08.064 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:51:07 smithi076 ceph-mon[31823]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi076", "root=default"]}]': finished 2024-04-26T21:51:08.064 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:51:07 smithi076 ceph-mon[31823]: osdmap e73: 11 total, 10 up, 11 in 2024-04-26T21:51:08.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:51:07 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T21:51:08.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:51:07 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T21:51:08.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:51:07 smithi072 ceph-mon[28138]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi076", "root=default"]}]': finished 2024-04-26T21:51:08.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:51:07 smithi072 ceph-mon[28138]: osdmap e73: 11 total, 10 up, 11 in 2024-04-26T21:51:08.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:51:07 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T21:51:08.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:51:07 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T21:51:09.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:51:08 smithi076 ceph-mon[31823]: purged_snaps scrub starts 2024-04-26T21:51:09.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:51:08 smithi076 ceph-mon[31823]: purged_snaps scrub ok 2024-04-26T21:51:09.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:51:08 smithi076 ceph-mon[31823]: pgmap v302: 1 pgs: 1 peering; 577 KiB data, 317 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:51:09.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:51:08 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T21:51:09.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:51:08 smithi076 ceph-mon[31823]: osd.10 [v2:172.21.15.76:6824/2808954459,v1:172.21.15.76:6825/2808954459] boot 2024-04-26T21:51:09.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:51:08 smithi076 ceph-mon[31823]: osdmap e74: 11 total, 11 up, 11 in 2024-04-26T21:51:09.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:51:08 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T21:51:09.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:51:08 smithi076 ceph-mon[31823]: osdmap e75: 11 total, 11 up, 11 in 2024-04-26T21:51:09.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:51:08 smithi164 ceph-mon[31916]: purged_snaps scrub starts 2024-04-26T21:51:09.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:51:08 smithi164 ceph-mon[31916]: purged_snaps scrub ok 2024-04-26T21:51:09.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:51:08 smithi164 ceph-mon[31916]: pgmap v302: 1 pgs: 1 peering; 577 KiB data, 317 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:51:09.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:51:08 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T21:51:09.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:51:08 smithi164 ceph-mon[31916]: osd.10 [v2:172.21.15.76:6824/2808954459,v1:172.21.15.76:6825/2808954459] boot 2024-04-26T21:51:09.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:51:08 smithi164 ceph-mon[31916]: osdmap e74: 11 total, 11 up, 11 in 2024-04-26T21:51:09.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:51:08 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T21:51:09.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:51:08 smithi164 ceph-mon[31916]: osdmap e75: 11 total, 11 up, 11 in 2024-04-26T21:51:09.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:51:08 smithi072 ceph-mon[28138]: purged_snaps scrub starts 2024-04-26T21:51:09.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:51:08 smithi072 ceph-mon[28138]: purged_snaps scrub ok 2024-04-26T21:51:09.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:51:08 smithi072 ceph-mon[28138]: pgmap v302: 1 pgs: 1 peering; 577 KiB data, 317 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:51:09.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:51:08 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T21:51:09.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:51:08 smithi072 ceph-mon[28138]: osd.10 [v2:172.21.15.76:6824/2808954459,v1:172.21.15.76:6825/2808954459] boot 2024-04-26T21:51:09.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:51:08 smithi072 ceph-mon[28138]: osdmap e74: 11 total, 11 up, 11 in 2024-04-26T21:51:09.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:51:08 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T21:51:09.323 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:51:08 smithi072 ceph-mon[28138]: osdmap e75: 11 total, 11 up, 11 in 2024-04-26T21:51:10.520 INFO:teuthology.orchestra.run.smithi164.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-26T21:51:10.520 INFO:teuthology.orchestra.run.smithi164.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-26T21:51:10.520 INFO:teuthology.orchestra.run.smithi164.stderr: stderr: 10+0 records in 2024-04-26T21:51:10.520 INFO:teuthology.orchestra.run.smithi164.stderr:10+0 records out 2024-04-26T21:51:10.520 INFO:teuthology.orchestra.run.smithi164.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0350088 s, 300 MB/s 2024-04-26T21:51:10.520 INFO:teuthology.orchestra.run.smithi164.stderr:--> Zapping successful for: 2024-04-26T21:51:10.748 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:51:10 smithi076 ceph-mon[31823]: osdmap e76: 11 total, 11 up, 11 in 2024-04-26T21:51:10.748 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:51:10 smithi076 ceph-mon[31823]: pgmap v306: 1 pgs: 1 peering; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:51:10.822 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:51:10 smithi072 ceph-mon[28138]: osdmap e76: 11 total, 11 up, 11 in 2024-04-26T21:51:10.822 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:51:10 smithi072 ceph-mon[28138]: pgmap v306: 1 pgs: 1 peering; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:51:10.940 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:51:10 smithi164 ceph-mon[31916]: osdmap e76: 11 total, 11 up, 11 in 2024-04-26T21:51:10.941 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:51:10 smithi164 ceph-mon[31916]: pgmap v306: 1 pgs: 1 peering; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:51:11.425 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a1780950-0415-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi164:vg_nvme/lv_1 2024-04-26T21:51:11.690 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:51:11 smithi164 ceph-mon[31916]: osdmap e77: 11 total, 11 up, 11 in 2024-04-26T21:51:11.814 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:51:11 smithi076 ceph-mon[31823]: osdmap e77: 11 total, 11 up, 11 in 2024-04-26T21:51:11.821 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:51:11 smithi072 ceph-mon[28138]: osdmap e77: 11 total, 11 up, 11 in 2024-04-26T21:51:11.867 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/a1780950-0415-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T21:51:12.691 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:51:12 smithi164 ceph-mon[31916]: osdmap e78: 11 total, 11 up, 11 in 2024-04-26T21:51:12.691 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:51:12 smithi164 ceph-mon[31916]: Detected new or changed devices on smithi076 2024-04-26T21:51:12.691 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:51:12 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:51:12.691 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:51:12 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:51:12.691 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:51:12 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:51:12.691 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:51:12 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:51:12.692 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:51:12 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:51:12.692 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:51:12 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:51:12.692 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:51:12 smithi164 ceph-mon[31916]: Adjusting osd_memory_target on smithi076 to 4272M 2024-04-26T21:51:12.692 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:51:12 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:51:12.692 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:51:12 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:51:12.692 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:51:12 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:51:12.692 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:51:12 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:51:12.693 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:51:12 smithi164 ceph-mon[31916]: pgmap v309: 1 pgs: 1 peering; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:51:12.795 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:51:12 smithi072 ceph-mon[28138]: osdmap e78: 11 total, 11 up, 11 in 2024-04-26T21:51:12.795 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:51:12 smithi072 ceph-mon[28138]: Detected new or changed devices on smithi076 2024-04-26T21:51:12.795 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:51:12 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:51:12.795 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:51:12 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:51:12.795 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:51:12 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:51:12.796 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:51:12 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:51:12.796 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:51:12 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:51:12.796 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:51:12 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:51:12.796 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:51:12 smithi072 ceph-mon[28138]: Adjusting osd_memory_target on smithi076 to 4272M 2024-04-26T21:51:12.796 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:51:12 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:51:12.796 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:51:12 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:51:12.796 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:51:12 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:51:12.796 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:51:12 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:51:12.796 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:51:12 smithi072 ceph-mon[28138]: pgmap v309: 1 pgs: 1 peering; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:51:12.815 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:51:12 smithi076 ceph-mon[31823]: osdmap e78: 11 total, 11 up, 11 in 2024-04-26T21:51:12.815 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:51:12 smithi076 ceph-mon[31823]: Detected new or changed devices on smithi076 2024-04-26T21:51:12.815 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:51:12 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:51:12.815 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:51:12 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:51:12.815 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:51:12 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:51:12.815 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:51:12 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:51:12.816 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:51:12 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:51:12.816 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:51:12 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:51:12.816 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:51:12 smithi076 ceph-mon[31823]: Adjusting osd_memory_target on smithi076 to 4272M 2024-04-26T21:51:12.816 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:51:12 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:51:12.816 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:51:12 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:51:12.816 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:51:12 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:51:12.816 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:51:12 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:51:12.816 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:51:12 smithi076 ceph-mon[31823]: pgmap v309: 1 pgs: 1 peering; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:51:14.071 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:51:14.067+0000 7fd1b5a00700 1 -- 172.21.15.164:0/2176193346 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd1b01067c0 msgr2=0x7fd1b0106c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:14.071 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:51:14.067+0000 7fd1b5a00700 1 --2- 172.21.15.164:0/2176193346 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd1b01067c0 0x7fd1b0106c20 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fd19c009990 tx=0x7fd19c009ca0 comp rx=0 tx=0).stop 2024-04-26T21:51:14.072 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:51:14.068+0000 7fd1b5a00700 1 -- 172.21.15.164:0/2176193346 shutdown_connections 2024-04-26T21:51:14.072 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:51:14.068+0000 7fd1b5a00700 1 --2- 172.21.15.164:0/2176193346 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd1b01055c0 0x7fd1b01059e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:14.072 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:51:14.068+0000 7fd1b5a00700 1 --2- 172.21.15.164:0/2176193346 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd1b0107160 0x7fd1b010da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:14.072 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:51:14.068+0000 7fd1b5a00700 1 --2- 172.21.15.164:0/2176193346 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd1b01067c0 0x7fd1b0106c20 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:14.072 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:51:14.068+0000 7fd1b5a00700 1 -- 172.21.15.164:0/2176193346 >> 172.21.15.164:0/2176193346 conn(0x7fd1b0100b40 msgr2=0x7fd1b0102fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:51:14.072 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:51:14.068+0000 7fd1b5a00700 1 -- 172.21.15.164:0/2176193346 shutdown_connections 2024-04-26T21:51:14.072 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:51:14.068+0000 7fd1b5a00700 1 -- 172.21.15.164:0/2176193346 wait complete. 2024-04-26T21:51:14.072 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:51:14.069+0000 7fd1b5a00700 1 Processor -- start 2024-04-26T21:51:14.072 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:51:14.069+0000 7fd1b5a00700 1 -- start start 2024-04-26T21:51:14.072 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:51:14.069+0000 7fd1b5a00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd1b01055c0 0x7fd1b019b590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:14.072 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:51:14.069+0000 7fd1b5a00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd1b01067c0 0x7fd1b019bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:14.072 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:51:14.069+0000 7fd1b5a00700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd1b0107160 0x7fd1b01a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:14.073 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:51:14.069+0000 7fd1b5a00700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fd1b0110f50 con 0x7fd1b01067c0 2024-04-26T21:51:14.073 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:51:14.069+0000 7fd1b5a00700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fd1b0110dd0 con 0x7fd1b0107160 2024-04-26T21:51:14.073 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:51:14.069+0000 7fd1b5a00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd1b01110d0 con 0x7fd1b01055c0 2024-04-26T21:51:14.073 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:51:14.070+0000 7fd1afe00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd1b01055c0 0x7fd1b019b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:14.073 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:51:14.070+0000 7fd1afe00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd1b01055c0 0x7fd1b019b590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:39788/0 (socket says 172.21.15.164:39788) 2024-04-26T21:51:14.073 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:51:14.070+0000 7fd1afe00700 1 -- 172.21.15.164:0/3928575435 learned_addr learned my addr 172.21.15.164:0/3928575435 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-04-26T21:51:14.073 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:51:14.070+0000 7fd1afe00700 1 -- 172.21.15.164:0/3928575435 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd1b0107160 msgr2=0x7fd1b01a2b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:14.073 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:51:14.070+0000 7fd1afe00700 1 --2- 172.21.15.164:0/3928575435 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd1b0107160 0x7fd1b01a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:14.073 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:51:14.070+0000 7fd1afe00700 1 -- 172.21.15.164:0/3928575435 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd1b01067c0 msgr2=0x7fd1b019bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:14.073 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:51:14.070+0000 7fd1af400700 1 --2- 172.21.15.164:0/3928575435 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd1b01067c0 0x7fd1b019bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:14.073 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:51:14.070+0000 7fd1afe00700 1 --2- 172.21.15.164:0/3928575435 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd1b01067c0 0x7fd1b019bad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:14.073 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:51:14.070+0000 7fd1afe00700 1 -- 172.21.15.164:0/3928575435 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1b01a31f0 con 0x7fd1b01055c0 2024-04-26T21:51:14.073 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:51:14.070+0000 7fd1afe00700 1 --2- 172.21.15.164:0/3928575435 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd1b01055c0 0x7fd1b019b590 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fd1a000c9c0 tx=0x7fd1a000cd80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:51:14.073 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:51:14.071+0000 7fd1acc00700 1 -- 172.21.15.164:0/3928575435 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1a0007ac0 con 0x7fd1b01055c0 2024-04-26T21:51:14.074 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:51:14.071+0000 7fd1acc00700 1 -- 172.21.15.164:0/3928575435 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd1a0007c60 con 0x7fd1b01055c0 2024-04-26T21:51:14.074 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:51:14.071+0000 7fd1b5a00700 1 -- 172.21.15.164:0/3928575435 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1b01a34e0 con 0x7fd1b01055c0 2024-04-26T21:51:14.074 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:51:14.071+0000 7fd1acc00700 1 -- 172.21.15.164:0/3928575435 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1a0018660 con 0x7fd1b01055c0 2024-04-26T21:51:14.074 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:51:14.071+0000 7fd1b5a00700 1 -- 172.21.15.164:0/3928575435 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1b01a3a70 con 0x7fd1b01055c0 2024-04-26T21:51:14.074 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:51:14.072+0000 7fd1b5a00700 1 -- 172.21.15.164:0/3928575435 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1b006a110 con 0x7fd1b01055c0 2024-04-26T21:51:14.074 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:51:14.074+0000 7fd1acc00700 1 -- 172.21.15.164:0/3928575435 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fd1a000f450 con 0x7fd1b01055c0 2024-04-26T21:51:14.075 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:51:14.076+0000 7fd1acc00700 1 --2- 172.21.15.164:0/3928575435 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fd1940a3cc0 0x7fd1940a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:14.076 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:51:14.076+0000 7fd1acc00700 1 -- 172.21.15.164:0/3928575435 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6694+0+0 (secure 0 0 0) 0x7fd1a00cfe20 con 0x7fd1b01055c0 2024-04-26T21:51:14.076 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:51:14.076+0000 7fd1af400700 1 --2- 172.21.15.164:0/3928575435 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fd1940a3cc0 0x7fd1940a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:14.078 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:51:14.078+0000 7fd1af400700 1 --2- 172.21.15.164:0/3928575435 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fd1940a3cc0 0x7fd1940a6180 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fd19c006010 tx=0x7fd19c0038c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:51:14.080 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:51:14.081+0000 7fd1acc00700 1 -- 172.21.15.164:0/3928575435 <== mon.1 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fd1a0014030 con 0x7fd1b01055c0 2024-04-26T21:51:14.315 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:51:14.315+0000 7fd1b5a00700 1 -- 172.21.15.164:0/3928575435 --> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi164:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fd1b0063860 con 0x7fd1940a3cc0 2024-04-26T21:51:15.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:51:14 smithi076 ceph-mon[31823]: pgmap v310: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail; 109 KiB/s, 0 objects/s recovering 2024-04-26T21:51:15.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:51:14 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:51:15.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:51:14 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:51:15.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:51:14 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:51:15.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:51:14 smithi072 ceph-mon[28138]: pgmap v310: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail; 109 KiB/s, 0 objects/s recovering 2024-04-26T21:51:15.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:51:14 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:51:15.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:51:14 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:51:15.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:51:14 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:51:15.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:51:14 smithi164 ceph-mon[31916]: pgmap v310: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail; 109 KiB/s, 0 objects/s recovering 2024-04-26T21:51:15.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:51:14 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:51:15.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:51:14 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:51:15.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:51:14 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:51:15.940 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:51:15 smithi164 ceph-mon[31916]: from='client.24340 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi164:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:51:16.064 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:51:15 smithi076 ceph-mon[31823]: from='client.24340 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi164:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:51:16.071 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:51:15 smithi072 ceph-mon[28138]: from='client.24340 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi164:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:51:16.895 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:51:16 smithi164 ceph-mon[31916]: pgmap v311: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail; 92 KiB/s, 0 objects/s recovering 2024-04-26T21:51:17.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:51:16 smithi072 ceph-mon[28138]: pgmap v311: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail; 92 KiB/s, 0 objects/s recovering 2024-04-26T21:51:17.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:51:16 smithi076 ceph-mon[31823]: pgmap v311: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail; 92 KiB/s, 0 objects/s recovering 2024-04-26T21:51:19.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:51:18 smithi076 ceph-mon[31823]: pgmap v312: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-26T21:51:19.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:51:18 smithi076 ceph-mon[31823]: from='client.? 172.21.15.164:0/1805230799' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "981b3d4d-916e-4937-9916-7cc21a891e43"}]: dispatch 2024-04-26T21:51:19.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:51:18 smithi076 ceph-mon[31823]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "981b3d4d-916e-4937-9916-7cc21a891e43"}]: dispatch 2024-04-26T21:51:19.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:51:18 smithi076 ceph-mon[31823]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "981b3d4d-916e-4937-9916-7cc21a891e43"}]': finished 2024-04-26T21:51:19.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:51:18 smithi076 ceph-mon[31823]: osdmap e79: 12 total, 11 up, 12 in 2024-04-26T21:51:19.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:51:18 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T21:51:19.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:51:18 smithi072 ceph-mon[28138]: pgmap v312: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-26T21:51:19.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:51:18 smithi072 ceph-mon[28138]: from='client.? 172.21.15.164:0/1805230799' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "981b3d4d-916e-4937-9916-7cc21a891e43"}]: dispatch 2024-04-26T21:51:19.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:51:18 smithi072 ceph-mon[28138]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "981b3d4d-916e-4937-9916-7cc21a891e43"}]: dispatch 2024-04-26T21:51:19.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:51:18 smithi072 ceph-mon[28138]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "981b3d4d-916e-4937-9916-7cc21a891e43"}]': finished 2024-04-26T21:51:19.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:51:18 smithi072 ceph-mon[28138]: osdmap e79: 12 total, 11 up, 12 in 2024-04-26T21:51:19.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:51:18 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T21:51:19.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:51:18 smithi164 ceph-mon[31916]: pgmap v312: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-26T21:51:19.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:51:18 smithi164 ceph-mon[31916]: from='client.? 172.21.15.164:0/1805230799' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "981b3d4d-916e-4937-9916-7cc21a891e43"}]: dispatch 2024-04-26T21:51:19.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:51:18 smithi164 ceph-mon[31916]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "981b3d4d-916e-4937-9916-7cc21a891e43"}]: dispatch 2024-04-26T21:51:19.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:51:18 smithi164 ceph-mon[31916]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "981b3d4d-916e-4937-9916-7cc21a891e43"}]': finished 2024-04-26T21:51:19.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:51:18 smithi164 ceph-mon[31916]: osdmap e79: 12 total, 11 up, 12 in 2024-04-26T21:51:19.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:51:18 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T21:51:20.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:51:19 smithi164 ceph-mon[31916]: from='client.? 172.21.15.164:0/3375053001' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:51:20.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:51:19 smithi076 ceph-mon[31823]: from='client.? 172.21.15.164:0/3375053001' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:51:20.321 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:51:19 smithi072 ceph-mon[28138]: from='client.? 172.21.15.164:0/3375053001' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:51:21.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:51:20 smithi164 ceph-mon[31916]: pgmap v314: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-04-26T21:51:21.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:51:20 smithi076 ceph-mon[31823]: pgmap v314: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-04-26T21:51:21.321 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:51:20 smithi072 ceph-mon[28138]: pgmap v314: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-04-26T21:51:23.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:51:22 smithi164 ceph-mon[31916]: pgmap v315: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-26T21:51:23.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:51:22 smithi076 ceph-mon[31823]: pgmap v315: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-26T21:51:23.321 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:51:22 smithi072 ceph-mon[28138]: pgmap v315: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-26T21:51:25.090 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:51:24 smithi164 ceph-mon[31916]: pgmap v316: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:51:25.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:51:24 smithi076 ceph-mon[31823]: pgmap v316: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:51:25.321 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:51:24 smithi072 ceph-mon[28138]: pgmap v316: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:51:26.995 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:51:26 smithi164 ceph-mon[31916]: pgmap v317: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:51:27.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:51:26 smithi076 ceph-mon[31823]: pgmap v317: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:51:27.321 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:51:26 smithi072 ceph-mon[28138]: pgmap v317: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:51:29.165 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:51:28 smithi164 ceph-mon[31916]: pgmap v318: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:51:29.166 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:51:28 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-26T21:51:29.166 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:51:28 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:51:29.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:51:28 smithi076 ceph-mon[31823]: pgmap v318: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:51:29.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:51:28 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-26T21:51:29.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:51:28 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:51:29.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:51:28 smithi072 ceph-mon[28138]: pgmap v318: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:51:29.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:51:28 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-26T21:51:29.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:51:28 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:51:30.176 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:51:29 smithi164 ceph-mon[31916]: Deploying daemon osd.11 on smithi164 2024-04-26T21:51:30.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:51:29 smithi076 ceph-mon[31823]: Deploying daemon osd.11 on smithi164 2024-04-26T21:51:30.321 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:51:29 smithi072 ceph-mon[28138]: Deploying daemon osd.11 on smithi164 2024-04-26T21:51:31.028 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:51:30 smithi164 ceph-mon[31916]: pgmap v319: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:51:31.223 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:51:30 smithi072 ceph-mon[28138]: pgmap v319: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:51:31.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:51:30 smithi076 ceph-mon[31823]: pgmap v319: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:51:32.940 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:51:32 smithi164 ceph-mon[31916]: pgmap v320: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:51:33.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:51:32 smithi076 ceph-mon[31823]: pgmap v320: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:51:33.321 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:51:32 smithi072 ceph-mon[28138]: pgmap v320: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:51:35.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:51:34 smithi164 ceph-mon[31916]: pgmap v321: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:51:35.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:51:34 smithi076 ceph-mon[31823]: pgmap v321: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:51:35.321 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:51:34 smithi072 ceph-mon[28138]: pgmap v321: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:51:37.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:51:36 smithi164 ceph-mon[31916]: pgmap v322: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:51:37.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:51:36 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:51:37.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:51:36 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:51:37.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:51:36 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:51:37.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:51:36 smithi076 ceph-mon[31823]: pgmap v322: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:51:37.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:51:36 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:51:37.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:51:36 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:51:37.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:51:36 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:51:37.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:51:36 smithi072 ceph-mon[28138]: pgmap v322: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:51:37.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:51:36 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:51:37.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:51:36 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:51:37.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:51:36 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:51:39.064 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:51:38 smithi076 ceph-mon[31823]: pgmap v323: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:51:39.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:51:38 smithi164 ceph-mon[31916]: pgmap v323: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:51:39.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:51:38 smithi072 ceph-mon[28138]: pgmap v323: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:51:41.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:51:40 smithi164 ceph-mon[31916]: pgmap v324: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:51:41.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:51:40 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:51:41.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:51:40 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:51:41.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:51:40 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:51:41.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:51:40 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:51:41.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:51:40 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:51:41.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:51:40 smithi164 ceph-mon[31916]: from='osd.11 [v2:172.21.15.164:6824/2354951784,v1:172.21.15.164:6825/2354951784]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-26T21:51:41.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:51:40 smithi164 ceph-mon[31916]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-26T21:51:41.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:51:40 smithi076 ceph-mon[31823]: pgmap v324: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:51:41.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:51:40 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:51:41.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:51:40 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:51:41.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:51:40 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:51:41.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:51:40 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:51:41.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:51:40 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:51:41.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:51:40 smithi076 ceph-mon[31823]: from='osd.11 [v2:172.21.15.164:6824/2354951784,v1:172.21.15.164:6825/2354951784]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-26T21:51:41.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:51:40 smithi076 ceph-mon[31823]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-26T21:51:41.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:51:40 smithi072 ceph-mon[28138]: pgmap v324: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:51:41.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:51:40 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:51:41.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:51:40 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:51:41.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:51:40 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:51:41.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:51:40 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:51:41.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:51:40 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:51:41.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:51:40 smithi072 ceph-mon[28138]: from='osd.11 [v2:172.21.15.164:6824/2354951784,v1:172.21.15.164:6825/2354951784]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-26T21:51:41.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:51:40 smithi072 ceph-mon[28138]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-26T21:51:41.477 INFO:teuthology.orchestra.run.smithi164.stdout:Created osd(s) 11 on host 'smithi164' 2024-04-26T21:51:41.477 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:51:41.476+0000 7fd1acc00700 1 -- 172.21.15.164:0/3928575435 <== mgr.14152 v2:172.21.15.72:6800/588150172 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fd1b0063860 con 0x7fd1940a3cc0 2024-04-26T21:51:41.480 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:51:41.479+0000 7fd1b5a00700 1 -- 172.21.15.164:0/3928575435 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fd1940a3cc0 msgr2=0x7fd1940a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:41.480 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:51:41.479+0000 7fd1b5a00700 1 --2- 172.21.15.164:0/3928575435 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fd1940a3cc0 0x7fd1940a6180 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fd19c006010 tx=0x7fd19c0038c0 comp rx=0 tx=0).stop 2024-04-26T21:51:41.480 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:51:41.479+0000 7fd1b5a00700 1 -- 172.21.15.164:0/3928575435 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd1b01055c0 msgr2=0x7fd1b019b590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:41.480 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:51:41.479+0000 7fd1b5a00700 1 --2- 172.21.15.164:0/3928575435 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd1b01055c0 0x7fd1b019b590 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fd1a000c9c0 tx=0x7fd1a000cd80 comp rx=0 tx=0).stop 2024-04-26T21:51:41.480 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:51:41.479+0000 7fd1b5a00700 1 -- 172.21.15.164:0/3928575435 shutdown_connections 2024-04-26T21:51:41.480 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:51:41.479+0000 7fd1b5a00700 1 --2- 172.21.15.164:0/3928575435 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fd1940a3cc0 0x7fd1940a6180 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:41.480 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:51:41.479+0000 7fd1b5a00700 1 --2- 172.21.15.164:0/3928575435 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd1b0107160 0x7fd1b01a2b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:41.480 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:51:41.479+0000 7fd1b5a00700 1 --2- 172.21.15.164:0/3928575435 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd1b01055c0 0x7fd1b019b590 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:41.480 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:51:41.479+0000 7fd1b5a00700 1 --2- 172.21.15.164:0/3928575435 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd1b01067c0 0x7fd1b019bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:41.481 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:51:41.479+0000 7fd1b5a00700 1 -- 172.21.15.164:0/3928575435 >> 172.21.15.164:0/3928575435 conn(0x7fd1b0100b40 msgr2=0x7fd1b0109ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:51:41.481 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:51:41.479+0000 7fd1b5a00700 1 -- 172.21.15.164:0/3928575435 shutdown_connections 2024-04-26T21:51:41.481 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:51:41.479+0000 7fd1b5a00700 1 -- 172.21.15.164:0/3928575435 wait complete. 2024-04-26T21:51:42.147 DEBUG:teuthology.orchestra.run.smithi164:osd.11> sudo journalctl -f -n 0 -u ceph-a1780950-0415-11ef-bc93-c7b262605968@osd.11.service 2024-04-26T21:51:42.151 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-04-26T21:51:42.151 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a1780950-0415-11ef-bc93-c7b262605968 -- ceph osd stat -f json 2024-04-26T21:51:42.572 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:51:42 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:51:42.572 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:51:42 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:51:42.572 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:51:42 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:51:42.572 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:51:42 smithi072 ceph-mon[28138]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-26T21:51:42.572 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:51:42 smithi072 ceph-mon[28138]: osdmap e80: 12 total, 11 up, 12 in 2024-04-26T21:51:42.572 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:51:42 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T21:51:42.572 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:51:42 smithi072 ceph-mon[28138]: from='osd.11 [v2:172.21.15.164:6824/2354951784,v1:172.21.15.164:6825/2354951784]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-04-26T21:51:42.572 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:51:42 smithi072 ceph-mon[28138]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-04-26T21:51:42.572 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:51:42 smithi072 ceph-mon[28138]: pgmap v326: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:51:42.589 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/a1780950-0415-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:51:42.691 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:51:42 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:51:42.691 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:51:42 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:51:42.691 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:51:42 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:51:42.691 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:51:42 smithi164 ceph-mon[31916]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-26T21:51:42.691 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:51:42 smithi164 ceph-mon[31916]: osdmap e80: 12 total, 11 up, 12 in 2024-04-26T21:51:42.691 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:51:42 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T21:51:42.691 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:51:42 smithi164 ceph-mon[31916]: from='osd.11 [v2:172.21.15.164:6824/2354951784,v1:172.21.15.164:6825/2354951784]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-04-26T21:51:42.691 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:51:42 smithi164 ceph-mon[31916]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-04-26T21:51:42.691 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:51:42 smithi164 ceph-mon[31916]: pgmap v326: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:51:42.692 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 21:51:42 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T21:51:42.505+0000 7f9800000700 -1 osd.11 0 waiting for initial osdmap 2024-04-26T21:51:42.692 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 21:51:42 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T21:51:42.542+0000 7f97fae00700 -1 osd.11 81 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T21:51:42.815 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:51:42 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:51:42.815 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:51:42 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:51:42.815 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:51:42 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:51:42.815 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:51:42 smithi076 ceph-mon[31823]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-26T21:51:42.815 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:51:42 smithi076 ceph-mon[31823]: osdmap e80: 12 total, 11 up, 12 in 2024-04-26T21:51:42.815 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:51:42 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T21:51:42.815 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:51:42 smithi076 ceph-mon[31823]: from='osd.11 [v2:172.21.15.164:6824/2354951784,v1:172.21.15.164:6825/2354951784]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-04-26T21:51:42.815 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:51:42 smithi076 ceph-mon[31823]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-04-26T21:51:42.816 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:51:42 smithi076 ceph-mon[31823]: pgmap v326: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:51:43.690 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:51:43 smithi164 ceph-mon[31916]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]': finished 2024-04-26T21:51:43.691 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:51:43 smithi164 ceph-mon[31916]: osdmap e81: 12 total, 11 up, 12 in 2024-04-26T21:51:43.691 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:51:43 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T21:51:43.691 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:51:43 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T21:51:43.789 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:51:43 smithi072 ceph-mon[28138]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]': finished 2024-04-26T21:51:43.790 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:51:43 smithi072 ceph-mon[28138]: osdmap e81: 12 total, 11 up, 12 in 2024-04-26T21:51:43.790 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:51:43 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T21:51:43.790 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:51:43 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T21:51:43.815 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:51:43 smithi076 ceph-mon[31823]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]': finished 2024-04-26T21:51:43.815 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:51:43 smithi076 ceph-mon[31823]: osdmap e81: 12 total, 11 up, 12 in 2024-04-26T21:51:43.815 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:51:43 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T21:51:43.815 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:51:43 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T21:51:44.657 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:51:44 smithi072 ceph-mon[28138]: purged_snaps scrub starts 2024-04-26T21:51:44.657 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:51:44 smithi072 ceph-mon[28138]: purged_snaps scrub ok 2024-04-26T21:51:44.657 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:51:44 smithi072 ceph-mon[28138]: osd.11 [v2:172.21.15.164:6824/2354951784,v1:172.21.15.164:6825/2354951784] boot 2024-04-26T21:51:44.657 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:51:44 smithi072 ceph-mon[28138]: osdmap e82: 12 total, 12 up, 12 in 2024-04-26T21:51:44.657 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:51:44 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T21:51:44.657 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:51:44 smithi072 ceph-mon[28138]: pgmap v329: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:51:44.815 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:51:44 smithi076 ceph-mon[31823]: purged_snaps scrub starts 2024-04-26T21:51:44.815 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:51:44 smithi076 ceph-mon[31823]: purged_snaps scrub ok 2024-04-26T21:51:44.815 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:51:44 smithi076 ceph-mon[31823]: osd.11 [v2:172.21.15.164:6824/2354951784,v1:172.21.15.164:6825/2354951784] boot 2024-04-26T21:51:44.815 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:51:44 smithi076 ceph-mon[31823]: osdmap e82: 12 total, 12 up, 12 in 2024-04-26T21:51:44.815 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:51:44 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T21:51:44.815 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:51:44 smithi076 ceph-mon[31823]: pgmap v329: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:51:44.941 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:51:44 smithi164 ceph-mon[31916]: purged_snaps scrub starts 2024-04-26T21:51:44.941 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:51:44 smithi164 ceph-mon[31916]: purged_snaps scrub ok 2024-04-26T21:51:44.941 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:51:44 smithi164 ceph-mon[31916]: osd.11 [v2:172.21.15.164:6824/2354951784,v1:172.21.15.164:6825/2354951784] boot 2024-04-26T21:51:44.941 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:51:44 smithi164 ceph-mon[31916]: osdmap e82: 12 total, 12 up, 12 in 2024-04-26T21:51:44.941 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:51:44 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T21:51:44.941 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:51:44 smithi164 ceph-mon[31916]: pgmap v329: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:51:44.945 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:44.944+0000 7ffb1da00700 1 -- 172.21.15.72:0/3225718405 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ffb181055c0 msgr2=0x7ffb181059e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:44.945 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:44.944+0000 7ffb1da00700 1 --2- 172.21.15.72:0/3225718405 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ffb181055c0 0x7ffb181059e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7ffb080099c0 tx=0x7ffb08009cd0 comp rx=0 tx=0).stop 2024-04-26T21:51:44.945 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:44.944+0000 7ffb1da00700 1 -- 172.21.15.72:0/3225718405 shutdown_connections 2024-04-26T21:51:44.945 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:44.944+0000 7ffb1da00700 1 --2- 172.21.15.72:0/3225718405 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ffb181055c0 0x7ffb181059e0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:44.945 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:44.944+0000 7ffb1da00700 1 --2- 172.21.15.72:0/3225718405 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffb18107160 0x7ffb1810da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:44.945 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:44.944+0000 7ffb1da00700 1 --2- 172.21.15.72:0/3225718405 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffb181067c0 0x7ffb18106c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:44.945 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:44.944+0000 7ffb1da00700 1 -- 172.21.15.72:0/3225718405 >> 172.21.15.72:0/3225718405 conn(0x7ffb18100b60 msgr2=0x7ffb18102fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:51:44.945 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:44.944+0000 7ffb1da00700 1 -- 172.21.15.72:0/3225718405 shutdown_connections 2024-04-26T21:51:44.945 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:44.944+0000 7ffb1da00700 1 -- 172.21.15.72:0/3225718405 wait complete. 2024-04-26T21:51:44.947 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:44.945+0000 7ffb1da00700 1 Processor -- start 2024-04-26T21:51:44.947 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:44.945+0000 7ffb1da00700 1 -- start start 2024-04-26T21:51:44.947 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:44.946+0000 7ffb1da00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ffb181055c0 0x7ffb1819b590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:44.947 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:44.946+0000 7ffb1da00700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffb181067c0 0x7ffb1819bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:44.948 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:44.946+0000 7ffb1da00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffb18107160 0x7ffb181a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:44.948 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:44.946+0000 7ffb1da00700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7ffb18110ee0 con 0x7ffb181055c0 2024-04-26T21:51:44.948 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:44.946+0000 7ffb1da00700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7ffb18110d60 con 0x7ffb181067c0 2024-04-26T21:51:44.948 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:44.946+0000 7ffb1da00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ffb18111060 con 0x7ffb18107160 2024-04-26T21:51:44.948 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:44.946+0000 7ffb17e00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ffb181055c0 0x7ffb1819b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:44.948 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:44.946+0000 7ffb17e00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ffb181055c0 0x7ffb1819b590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:44444/0 (socket says 172.21.15.72:44444) 2024-04-26T21:51:44.948 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:44.946+0000 7ffb17e00700 1 -- 172.21.15.72:0/107195974 learned_addr learned my addr 172.21.15.72:0/107195974 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:51:44.948 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:44.946+0000 7ffb17e00700 1 -- 172.21.15.72:0/107195974 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffb18107160 msgr2=0x7ffb181a2b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:44.948 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:44.946+0000 7ffb17400700 1 --2- 172.21.15.72:0/107195974 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffb181067c0 0x7ffb1819bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:44.948 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:44.946+0000 7ffb1d000700 1 --2- 172.21.15.72:0/107195974 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffb18107160 0x7ffb181a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:44.948 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:44.946+0000 7ffb17e00700 1 --2- 172.21.15.72:0/107195974 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffb18107160 0x7ffb181a2b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:44.948 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:44.946+0000 7ffb17e00700 1 -- 172.21.15.72:0/107195974 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffb181067c0 msgr2=0x7ffb1819bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:44.948 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:44.946+0000 7ffb17e00700 1 --2- 172.21.15.72:0/107195974 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffb181067c0 0x7ffb1819bad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:44.949 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:44.946+0000 7ffb17e00700 1 -- 172.21.15.72:0/107195974 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb181a31f0 con 0x7ffb181055c0 2024-04-26T21:51:44.949 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:44.946+0000 7ffb1d000700 1 --2- 172.21.15.72:0/107195974 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffb18107160 0x7ffb181a2b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:51:44.949 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:44.947+0000 7ffb17e00700 1 --2- 172.21.15.72:0/107195974 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ffb181055c0 0x7ffb1819b590 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7ffb0800b580 tx=0x7ffb0800f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:51:44.949 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:44.947+0000 7ffb14c00700 1 -- 172.21.15.72:0/107195974 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb08027070 con 0x7ffb181055c0 2024-04-26T21:51:44.949 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:44.947+0000 7ffb1da00700 1 -- 172.21.15.72:0/107195974 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb181a3480 con 0x7ffb181055c0 2024-04-26T21:51:44.949 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:44.947+0000 7ffb14c00700 1 -- 172.21.15.72:0/107195974 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ffb0800fba0 con 0x7ffb181055c0 2024-04-26T21:51:44.949 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:44.947+0000 7ffb14c00700 1 -- 172.21.15.72:0/107195974 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb080037f0 con 0x7ffb181055c0 2024-04-26T21:51:44.949 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:44.947+0000 7ffb1da00700 1 -- 172.21.15.72:0/107195974 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb181a39b0 con 0x7ffb181055c0 2024-04-26T21:51:44.949 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:44.948+0000 7ffb1da00700 1 -- 172.21.15.72:0/107195974 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb1806a110 con 0x7ffb181055c0 2024-04-26T21:51:44.952 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:44.952+0000 7ffb14c00700 1 -- 172.21.15.72:0/107195974 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7ffb0802a070 con 0x7ffb181055c0 2024-04-26T21:51:44.953 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:44.953+0000 7ffb14c00700 1 --2- 172.21.15.72:0/107195974 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7ffafc0a3c40 0x7ffafc0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:44.953 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:44.953+0000 7ffb14c00700 1 -- 172.21.15.72:0/107195974 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7ffb080d1460 con 0x7ffb181055c0 2024-04-26T21:51:44.954 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:44.953+0000 7ffb17400700 1 --2- 172.21.15.72:0/107195974 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7ffafc0a3c40 0x7ffafc0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:44.954 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:44.954+0000 7ffb17400700 1 --2- 172.21.15.72:0/107195974 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7ffafc0a3c40 0x7ffafc0a6100 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7ffb0400ba60 tx=0x7ffb04005d50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:51:44.956 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:44.956+0000 7ffb14c00700 1 -- 172.21.15.72:0/107195974 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7ffb08031070 con 0x7ffb181055c0 2024-04-26T21:51:45.183 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:45.183+0000 7ffb1da00700 1 -- 172.21.15.72:0/107195974 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7ffb1802e490 con 0x7ffb181055c0 2024-04-26T21:51:45.184 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:45.184+0000 7ffb14c00700 1 -- 172.21.15.72:0/107195974 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v83) ==== 74+0+133 (secure 0 0 0) 0x7ffb08013070 con 0x7ffb181055c0 2024-04-26T21:51:45.184 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T21:51:45.187 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:45.186+0000 7ffb1da00700 1 -- 172.21.15.72:0/107195974 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7ffafc0a3c40 msgr2=0x7ffafc0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:45.187 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:45.186+0000 7ffb1da00700 1 --2- 172.21.15.72:0/107195974 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7ffafc0a3c40 0x7ffafc0a6100 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7ffb0400ba60 tx=0x7ffb04005d50 comp rx=0 tx=0).stop 2024-04-26T21:51:45.187 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:45.186+0000 7ffb1da00700 1 -- 172.21.15.72:0/107195974 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ffb181055c0 msgr2=0x7ffb1819b590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:45.187 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:45.186+0000 7ffb1da00700 1 --2- 172.21.15.72:0/107195974 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ffb181055c0 0x7ffb1819b590 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7ffb0800b580 tx=0x7ffb0800f740 comp rx=0 tx=0).stop 2024-04-26T21:51:45.187 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:45.186+0000 7ffb1da00700 1 -- 172.21.15.72:0/107195974 shutdown_connections 2024-04-26T21:51:45.187 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:45.186+0000 7ffb1da00700 1 --2- 172.21.15.72:0/107195974 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7ffafc0a3c40 0x7ffafc0a6100 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:45.188 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:45.186+0000 7ffb1da00700 1 --2- 172.21.15.72:0/107195974 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ffb181055c0 0x7ffb1819b590 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:45.188 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:45.186+0000 7ffb1da00700 1 --2- 172.21.15.72:0/107195974 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffb18107160 0x7ffb181a2b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:45.188 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:45.186+0000 7ffb1da00700 1 --2- 172.21.15.72:0/107195974 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffb181067c0 0x7ffb1819bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:45.188 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:45.186+0000 7ffb1da00700 1 -- 172.21.15.72:0/107195974 >> 172.21.15.72:0/107195974 conn(0x7ffb18100b60 msgr2=0x7ffb18109b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:51:45.188 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:45.187+0000 7ffb1da00700 1 -- 172.21.15.72:0/107195974 shutdown_connections 2024-04-26T21:51:45.188 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:45.187+0000 7ffb1da00700 1 -- 172.21.15.72:0/107195974 wait complete. 2024-04-26T21:51:45.814 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:51:45 smithi076 ceph-mon[31823]: osdmap e83: 12 total, 12 up, 12 in 2024-04-26T21:51:45.815 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:51:45 smithi076 ceph-mon[31823]: from='client.? 172.21.15.72:0/107195974' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-26T21:51:45.822 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:51:45 smithi072 ceph-mon[28138]: osdmap e83: 12 total, 12 up, 12 in 2024-04-26T21:51:45.822 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:51:45 smithi072 ceph-mon[28138]: from='client.? 172.21.15.72:0/107195974' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-26T21:51:45.890 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:51:45 smithi164 ceph-mon[31916]: osdmap e83: 12 total, 12 up, 12 in 2024-04-26T21:51:45.890 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:51:45 smithi164 ceph-mon[31916]: from='client.? 172.21.15.72:0/107195974' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-26T21:51:46.086 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":83,"num_osds":12,"num_up_osds":12,"osd_up_since":1714168303,"num_in_osds":12,"osd_in_since":1714168278,"num_remapped_pgs":0} 2024-04-26T21:51:46.087 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid a1780950-0415-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-26T21:51:46.512 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/a1780950-0415-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:51:46.649 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:51:46 smithi164 ceph-mon[31916]: pgmap v331: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:51:46.814 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:51:46 smithi076 ceph-mon[31823]: pgmap v331: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:51:46.821 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:51:46 smithi072 ceph-mon[28138]: pgmap v331: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:51:47.822 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:51:47 smithi072 ceph-mon[28138]: Detected new or changed devices on smithi164 2024-04-26T21:51:47.822 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:51:47 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:51:47.822 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:51:47 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:51:47.823 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:51:47 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:51:47.823 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:51:47 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:51:47.823 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:51:47 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:51:47.823 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:51:47 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:51:47.823 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:51:47 smithi072 ceph-mon[28138]: Adjusting osd_memory_target on smithi164 to 4272M 2024-04-26T21:51:47.823 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:51:47 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:51:47.823 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:51:47 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:51:47.823 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:51:47 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:51:47.823 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:51:47 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:51:48.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:51:47 smithi076 ceph-mon[31823]: Detected new or changed devices on smithi164 2024-04-26T21:51:48.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:51:47 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:51:48.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:51:47 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:51:48.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:51:47 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:51:48.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:51:47 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:51:48.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:51:47 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:51:48.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:51:47 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:51:48.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:51:47 smithi076 ceph-mon[31823]: Adjusting osd_memory_target on smithi164 to 4272M 2024-04-26T21:51:48.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:51:47 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:51:48.066 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:51:47 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:51:48.066 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:51:47 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:51:48.066 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:51:47 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:51:48.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:51:47 smithi164 ceph-mon[31916]: Detected new or changed devices on smithi164 2024-04-26T21:51:48.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:51:47 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:51:48.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:51:47 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:51:48.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:51:47 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:51:48.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:51:47 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:51:48.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:51:47 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:51:48.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:51:47 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:51:48.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:51:47 smithi164 ceph-mon[31916]: Adjusting osd_memory_target on smithi164 to 4272M 2024-04-26T21:51:48.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:51:47 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:51:48.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:51:47 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:51:48.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:51:47 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:51:48.192 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:51:47 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:51:48.666 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:48.665+0000 7f18e7000700 1 -- 172.21.15.72:0/3284231497 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f18e010fa50 msgr2=0x7f18e010fe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:48.667 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:48.665+0000 7f18e7000700 1 --2- 172.21.15.72:0/3284231497 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f18e010fa50 0x7f18e010fe30 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f18d0009990 tx=0x7f18d0009ca0 comp rx=0 tx=0).stop 2024-04-26T21:51:48.667 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:48.666+0000 7f18e7000700 1 -- 172.21.15.72:0/3284231497 shutdown_connections 2024-04-26T21:51:48.667 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:48.666+0000 7f18e7000700 1 --2- 172.21.15.72:0/3284231497 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f18e010fa50 0x7f18e010fe30 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:48.667 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:48.666+0000 7f18e7000700 1 --2- 172.21.15.72:0/3284231497 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f18e010a410 0x7f18e0116920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:48.667 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:48.666+0000 7f18e7000700 1 --2- 172.21.15.72:0/3284231497 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f18e0109a50 0x7f18e0109ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:48.667 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:48.666+0000 7f18e7000700 1 -- 172.21.15.72:0/3284231497 >> 172.21.15.72:0/3284231497 conn(0x7f18e0076bf0 msgr2=0x7f18e0108240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:51:48.667 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:48.666+0000 7f18e7000700 1 -- 172.21.15.72:0/3284231497 shutdown_connections 2024-04-26T21:51:48.667 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:48.666+0000 7f18e7000700 1 -- 172.21.15.72:0/3284231497 wait complete. 2024-04-26T21:51:48.669 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:48.667+0000 7f18e7000700 1 Processor -- start 2024-04-26T21:51:48.669 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:48.667+0000 7f18e7000700 1 -- start start 2024-04-26T21:51:48.669 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:48.667+0000 7f18e7000700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f18e0109a50 0x7f18e019fff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:48.669 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:48.667+0000 7f18e7000700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f18e010a410 0x7f18e01a0530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:48.669 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:48.667+0000 7f18e7000700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f18e010fa50 0x7f18e01a48c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:48.670 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:48.667+0000 7f18e7000700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f18e0119b40 con 0x7f18e0109a50 2024-04-26T21:51:48.670 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:48.667+0000 7f18e7000700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f18e01199c0 con 0x7f18e010a410 2024-04-26T21:51:48.670 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:48.667+0000 7f18e7000700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f18e0119cc0 con 0x7f18e010fa50 2024-04-26T21:51:48.670 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:48.668+0000 7f18dfe00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f18e0109a50 0x7f18e019fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:48.670 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:48.668+0000 7f18dfe00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f18e0109a50 0x7f18e019fff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:44466/0 (socket says 172.21.15.72:44466) 2024-04-26T21:51:48.670 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:48.668+0000 7f18dfe00700 1 -- 172.21.15.72:0/1975962782 learned_addr learned my addr 172.21.15.72:0/1975962782 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:51:48.670 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:48.668+0000 7f18df400700 1 --2- 172.21.15.72:0/1975962782 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f18e010a410 0x7f18e01a0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:48.670 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:48.668+0000 7f18e5000700 1 --2- 172.21.15.72:0/1975962782 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f18e010fa50 0x7f18e01a48c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:48.670 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:48.668+0000 7f18dfe00700 1 -- 172.21.15.72:0/1975962782 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f18e010fa50 msgr2=0x7f18e01a48c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:48.670 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:48.668+0000 7f18dfe00700 1 --2- 172.21.15.72:0/1975962782 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f18e010fa50 0x7f18e01a48c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:48.670 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:48.668+0000 7f18dfe00700 1 -- 172.21.15.72:0/1975962782 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f18e010a410 msgr2=0x7f18e01a0530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:48.670 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:48.668+0000 7f18dfe00700 1 --2- 172.21.15.72:0/1975962782 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f18e010a410 0x7f18e01a0530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:48.670 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:48.668+0000 7f18dfe00700 1 -- 172.21.15.72:0/1975962782 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18e01a5000 con 0x7f18e0109a50 2024-04-26T21:51:48.671 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:48.668+0000 7f18e5000700 1 --2- 172.21.15.72:0/1975962782 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f18e010fa50 0x7f18e01a48c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:51:48.671 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:48.668+0000 7f18dfe00700 1 --2- 172.21.15.72:0/1975962782 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f18e0109a50 0x7f18e019fff0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f18d0000c00 tx=0x7f18d0005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:51:48.671 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:48.669+0000 7f18dcc00700 1 -- 172.21.15.72:0/1975962782 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18d0027070 con 0x7f18e0109a50 2024-04-26T21:51:48.671 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:48.669+0000 7f18dcc00700 1 -- 172.21.15.72:0/1975962782 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f18d000fb80 con 0x7f18e0109a50 2024-04-26T21:51:48.671 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:48.669+0000 7f18dcc00700 1 -- 172.21.15.72:0/1975962782 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18d0022c30 con 0x7f18e0109a50 2024-04-26T21:51:48.671 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:48.669+0000 7f18e7000700 1 -- 172.21.15.72:0/1975962782 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18e01a5290 con 0x7f18e0109a50 2024-04-26T21:51:48.671 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:48.669+0000 7f18e7000700 1 -- 172.21.15.72:0/1975962782 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18e01acb70 con 0x7f18e0109a50 2024-04-26T21:51:48.671 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:48.670+0000 7f18e7000700 1 -- 172.21.15.72:0/1975962782 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18e006a110 con 0x7f18e0109a50 2024-04-26T21:51:48.672 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:48.672+0000 7f18dcc00700 1 -- 172.21.15.72:0/1975962782 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f18d002a070 con 0x7f18e0109a50 2024-04-26T21:51:48.673 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:48.673+0000 7f18dcc00700 1 --2- 172.21.15.72:0/1975962782 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f18c40a3c40 0x7f18c40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:48.674 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:48.673+0000 7f18dcc00700 1 -- 172.21.15.72:0/1975962782 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7f18d00d0e70 con 0x7f18e0109a50 2024-04-26T21:51:48.674 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:48.674+0000 7f18df400700 1 --2- 172.21.15.72:0/1975962782 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f18c40a3c40 0x7f18c40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:48.674 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:48.674+0000 7f18df400700 1 --2- 172.21.15.72:0/1975962782 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f18c40a3c40 0x7f18c40a6100 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f18cc005950 tx=0x7f18cc0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:51:48.678 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:48.678+0000 7f18dcc00700 1 -- 172.21.15.72:0/1975962782 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f18d005f060 con 0x7f18e0109a50 2024-04-26T21:51:48.906 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:48.905+0000 7f18e7000700 1 -- 172.21.15.72:0/1975962782 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f18e002e490 con 0x7f18e0109a50 2024-04-26T21:51:48.908 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:48.907+0000 7f18dcc00700 1 -- 172.21.15.72:0/1975962782 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+18409 (secure 0 0 0) 0x7f18d0013070 con 0x7f18e0109a50 2024-04-26T21:51:48.908 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T21:51:48.908 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":83,"fsid":"a1780950-0415-11ef-bc93-c7b262605968","created":"2024-04-26T21:42:01.504173+0000","modified":"2024-04-26T21:51:44.487133+0000","last_up_change":"2024-04-26T21:51:43.482437+0000","last_in_change":"2024-04-26T21:51:18.468587+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:46:29.785427+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"04e06884-b355-4762-9b29-9034b7201e9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6802","nonce":1416765179},{"type":"v1","addr":"172.21.15.72:6803","nonce":1416765179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6804","nonce":1416765179},{"type":"v1","addr":"172.21.15.72:6805","nonce":1416765179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6808","nonce":1416765179},{"type":"v1","addr":"172.21.15.72:6809","nonce":1416765179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6806","nonce":1416765179},{"type":"v1","addr":"172.21.15.72:6807","nonce":1416765179}]},"public_addr":"172.21.15.72:6803/1416765179","cluster_addr":"172.21.15.72:6805/1416765179","heartbeat_back_addr":"172.21.15.72:6809/1416765179","heartbeat_front_addr":"172.21.15.72:6807/1416765179","state":["exists","up"]},{"osd":1,"uuid":"d75a479a-4a2e-48e9-a01e-2b1c9617416a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":755426949},{"type":"v1","addr":"172.21.15.76:6801","nonce":755426949}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":755426949},{"type":"v1","addr":"172.21.15.76:6803","nonce":755426949}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":755426949},{"type":"v1","addr":"172.21.15.76:6807","nonce":755426949}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":755426949},{"type":"v1","addr":"172.21.15.76:6805","nonce":755426949}]},"public_addr":"172.21.15.76:6801/755426949","cluster_addr":"172.21.15.76:6803/755426949","heartbeat_back_addr":"172.21.15.76:6807/755426949","heartbeat_front_addr":"172.21.15.76:6805/755426949","state":["exists","up"]},{"osd":2,"uuid":"b6f8da58-4428-4e8c-a944-7ea5645d6d92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":852265973},{"type":"v1","addr":"172.21.15.164:6801","nonce":852265973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":852265973},{"type":"v1","addr":"172.21.15.164:6803","nonce":852265973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":852265973},{"type":"v1","addr":"172.21.15.164:6807","nonce":852265973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":852265973},{"type":"v1","addr":"172.21.15.164:6805","nonce":852265973}]},"public_addr":"172.21.15.164:6801/852265973","cluster_addr":"172.21.15.164:6803/852265973","heartbeat_back_addr":"172.21.15.164:6807/852265973","heartbeat_front_addr":"172.21.15.164:6805/852265973","state":["exists","up"]},{"osd":3,"uuid":"76d83409-649e-41e5-b4f4-43bbd5b00ea5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6810","nonce":4027763789},{"type":"v1","addr":"172.21.15.72:6811","nonce":4027763789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6812","nonce":4027763789},{"type":"v1","addr":"172.21.15.72:6813","nonce":4027763789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6816","nonce":4027763789},{"type":"v1","addr":"172.21.15.72:6817","nonce":4027763789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6814","nonce":4027763789},{"type":"v1","addr":"172.21.15.72:6815","nonce":4027763789}]},"public_addr":"172.21.15.72:6811/4027763789","cluster_addr":"172.21.15.72:6813/4027763789","heartbeat_back_addr":"172.21.15.72:6817/4027763789","heartbeat_front_addr":"172.21.15.72:6815/4027763789","state":["exists","up"]},{"osd":4,"uuid":"d2cfeea7-1661-4156-bdcd-f770a8d1a878","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":3718867827},{"type":"v1","addr":"172.21.15.76:6809","nonce":3718867827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":3718867827},{"type":"v1","addr":"172.21.15.76:6811","nonce":3718867827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":3718867827},{"type":"v1","addr":"172.21.15.76:6815","nonce":3718867827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":3718867827},{"type":"v1","addr":"172.21.15.76:6813","nonce":3718867827}]},"public_addr":"172.21.15.76:6809/3718867827","cluster_addr":"172.21.15.76:6811/3718867827","heartbeat_back_addr":"172.21.15.76:6815/3718867827","heartbeat_front_addr":"172.21.15.76:6813/3718867827","state":["exists","up"]},{"osd":5,"uuid":"f3896ad6-a040-4ea3-a431-ff9b7fc05c36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":2543954772},{"type":"v1","addr":"172.21.15.164:6809","nonce":2543954772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":2543954772},{"type":"v1","addr":"172.21.15.164:6811","nonce":2543954772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":2543954772},{"type":"v1","addr":"172.21.15.164:6815","nonce":2543954772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":2543954772},{"type":"v1","addr":"172.21.15.164:6813","nonce":2543954772}]},"public_addr":"172.21.15.164:6809/2543954772","cluster_addr":"172.21.15.164:6811/2543954772","heartbeat_back_addr":"172.21.15.164:6815/2543954772","heartbeat_front_addr":"172.21.15.164:6813/2543954772","state":["exists","up"]},{"osd":6,"uuid":"d5d56650-be7e-4ba5-ba3a-d7d473294dab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6818","nonce":2080659027},{"type":"v1","addr":"172.21.15.72:6819","nonce":2080659027}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6820","nonce":2080659027},{"type":"v1","addr":"172.21.15.72:6821","nonce":2080659027}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6824","nonce":2080659027},{"type":"v1","addr":"172.21.15.72:6825","nonce":2080659027}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6822","nonce":2080659027},{"type":"v1","addr":"172.21.15.72:6823","nonce":2080659027}]},"public_addr":"172.21.15.72:6819/2080659027","cluster_addr":"172.21.15.72:6821/2080659027","heartbeat_back_addr":"172.21.15.72:6825/2080659027","heartbeat_front_addr":"172.21.15.72:6823/2080659027","state":["exists","up"]},{"osd":7,"uuid":"18975b5a-9309-4a0f-9f37-3ce09c2e745d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":3144631323},{"type":"v1","addr":"172.21.15.76:6817","nonce":3144631323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":3144631323},{"type":"v1","addr":"172.21.15.76:6819","nonce":3144631323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":3144631323},{"type":"v1","addr":"172.21.15.76:6823","nonce":3144631323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":3144631323},{"type":"v1","addr":"172.21.15.76:6821","nonce":3144631323}]},"public_addr":"172.21.15.76:6817/3144631323","cluster_addr":"172.21.15.76:6819/3144631323","heartbeat_back_addr":"172.21.15.76:6823/3144631323","heartbeat_front_addr":"172.21.15.76:6821/3144631323","state":["exists","up"]},{"osd":8,"uuid":"779e9c1e-8e97-48d4-a854-68ca13f38e0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":627888883},{"type":"v1","addr":"172.21.15.164:6817","nonce":627888883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":627888883},{"type":"v1","addr":"172.21.15.164:6819","nonce":627888883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":627888883},{"type":"v1","addr":"172.21.15.164:6823","nonce":627888883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":627888883},{"type":"v1","addr":"172.21.15.164:6821","nonce":627888883}]},"public_addr":"172.21.15.164:6817/627888883","cluster_addr":"172.21.15.164:6819/627888883","heartbeat_back_addr":"172.21.15.164:6823/627888883","heartbeat_front_addr":"172.21.15.164:6821/627888883","state":["exists","up"]},{"osd":9,"uuid":"d0317850-7b2d-4850-b2cf-0b919505c1fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6826","nonce":1273118485},{"type":"v1","addr":"172.21.15.72:6827","nonce":1273118485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6828","nonce":1273118485},{"type":"v1","addr":"172.21.15.72:6829","nonce":1273118485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6832","nonce":1273118485},{"type":"v1","addr":"172.21.15.72:6833","nonce":1273118485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6830","nonce":1273118485},{"type":"v1","addr":"172.21.15.72:6831","nonce":1273118485}]},"public_addr":"172.21.15.72:6827/1273118485","cluster_addr":"172.21.15.72:6829/1273118485","heartbeat_back_addr":"172.21.15.72:6833/1273118485","heartbeat_front_addr":"172.21.15.72:6831/1273118485","state":["exists","up"]},{"osd":10,"uuid":"e0aa38d8-6011-4d2a-b67b-5bff6d999e47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":2808954459},{"type":"v1","addr":"172.21.15.76:6825","nonce":2808954459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":2808954459},{"type":"v1","addr":"172.21.15.76:6827","nonce":2808954459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":2808954459},{"type":"v1","addr":"172.21.15.76:6831","nonce":2808954459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":2808954459},{"type":"v1","addr":"172.21.15.76:6829","nonce":2808954459}]},"public_addr":"172.21.15.76:6825/2808954459","cluster_addr":"172.21.15.76:6827/2808954459","heartbeat_back_addr":"172.21.15.76:6831/2808954459","heartbeat_front_addr":"172.21.15.76:6829/2808954459","state":["exists","up"]},{"osd":11,"uuid":"981b3d4d-916e-4937-9916-7cc21a891e43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":2354951784},{"type":"v1","addr":"172.21.15.164:6825","nonce":2354951784}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":2354951784},{"type":"v1","addr":"172.21.15.164:6827","nonce":2354951784}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":2354951784},{"type":"v1","addr":"172.21.15.164:6831","nonce":2354951784}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":2354951784},{"type":"v1","addr":"172.21.15.164:6829","nonce":2354951784}]},"public_addr":"172.21.15.164:6825/2354951784","cluster_addr":"172.21.15.164:6827/2354951784","heartbeat_back_addr":"172.21.15.164:6831/2354951784","heartbeat_front_addr":"172.21.15.164:6829/2354951784","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:18.997924+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:52.439081+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:46:25.893288+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:47:00.112807+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:47:35.116874+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:48:09.216344+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:48:45.028853+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:49:19.905397+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:49:55.030536+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:50:30.767662+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:51:06.423267+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:51:41.631211+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.72:0/416277784":"2024-04-27T21:43:09.602791+0000","172.21.15.72:6800/4010358672":"2024-04-27T21:43:09.602791+0000","172.21.15.72:6801/1012617371":"2024-04-27T21:42:34.684837+0000","172.21.15.72:6801/4010358672":"2024-04-27T21:43:09.602791+0000","172.21.15.72:0/3555448179":"2024-04-27T21:43:09.602791+0000","172.21.15.72:6800/1012617371":"2024-04-27T21:42:34.684837+0000","172.21.15.72:0/2920378761":"2024-04-27T21:42:34.684837+0000","172.21.15.72:0/1933740041":"2024-04-27T21:43:09.602791+0000","172.21.15.72:0/3225295775":"2024-04-27T21:42:34.684837+0000","172.21.15.72:0/2326016180":"2024-04-27T21:42:34.684837+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:51:48.911 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:48.911+0000 7f18e7000700 1 -- 172.21.15.72:0/1975962782 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f18c40a3c40 msgr2=0x7f18c40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:48.911 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:48.911+0000 7f18e7000700 1 --2- 172.21.15.72:0/1975962782 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f18c40a3c40 0x7f18c40a6100 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f18cc005950 tx=0x7f18cc0058e0 comp rx=0 tx=0).stop 2024-04-26T21:51:48.911 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:48.911+0000 7f18e7000700 1 -- 172.21.15.72:0/1975962782 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f18e0109a50 msgr2=0x7f18e019fff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:48.911 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:48.911+0000 7f18e7000700 1 --2- 172.21.15.72:0/1975962782 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f18e0109a50 0x7f18e019fff0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f18d0000c00 tx=0x7f18d0005ef0 comp rx=0 tx=0).stop 2024-04-26T21:51:48.912 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:48.911+0000 7f18e7000700 1 -- 172.21.15.72:0/1975962782 shutdown_connections 2024-04-26T21:51:48.912 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:48.911+0000 7f18e7000700 1 --2- 172.21.15.72:0/1975962782 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f18c40a3c40 0x7f18c40a6100 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:48.912 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:48.911+0000 7f18e7000700 1 --2- 172.21.15.72:0/1975962782 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f18e0109a50 0x7f18e019fff0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:48.912 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:48.911+0000 7f18e7000700 1 --2- 172.21.15.72:0/1975962782 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f18e010fa50 0x7f18e01a48c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:48.912 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:48.911+0000 7f18e7000700 1 --2- 172.21.15.72:0/1975962782 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f18e010a410 0x7f18e01a0530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:48.912 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:48.911+0000 7f18e7000700 1 -- 172.21.15.72:0/1975962782 >> 172.21.15.72:0/1975962782 conn(0x7f18e0076bf0 msgr2=0x7f18e0077210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:51:48.912 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:48.911+0000 7f18e7000700 1 -- 172.21.15.72:0/1975962782 shutdown_connections 2024-04-26T21:51:48.912 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:48.911+0000 7f18e7000700 1 -- 172.21.15.72:0/1975962782 wait complete. 2024-04-26T21:51:49.064 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:51:48 smithi076 ceph-mon[31823]: pgmap v332: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:51:49.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:51:48 smithi164 ceph-mon[31916]: pgmap v332: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:51:49.281 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:51:48 smithi072 ceph-mon[28138]: pgmap v332: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:51:49.822 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:46:29.785427+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:51:49.822 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid a1780950-0415-11ef-bc93-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-04-26T21:51:50.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:51:49 smithi072 ceph-mon[28138]: from='client.? 172.21.15.72:0/1975962782' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:51:50.139 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:51:49 smithi164 ceph-mon[31916]: from='client.? 172.21.15.72:0/1975962782' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:51:50.253 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/a1780950-0415-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:51:50.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:51:49 smithi076 ceph-mon[31823]: from='client.? 172.21.15.72:0/1975962782' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:51:51.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:51:50 smithi072 ceph-mon[28138]: pgmap v333: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:51:51.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:51:50 smithi164 ceph-mon[31916]: pgmap v333: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:51:51.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:51:50 smithi076 ceph-mon[31823]: pgmap v333: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:51:52.451 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:52.450+0000 7f9208600700 1 -- 172.21.15.72:0/2230116372 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9200075650 msgr2=0x7f9200075a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:52.451 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:52.450+0000 7f9208600700 1 --2- 172.21.15.72:0/2230116372 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9200075650 0x7f9200075a30 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f91f4009990 tx=0x7f91f4009ca0 comp rx=0 tx=0).stop 2024-04-26T21:51:52.452 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:52.450+0000 7f9208600700 1 -- 172.21.15.72:0/2230116372 shutdown_connections 2024-04-26T21:51:52.452 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:52.451+0000 7f9208600700 1 --2- 172.21.15.72:0/2230116372 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9200075650 0x7f9200075a30 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:52.452 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:52.451+0000 7f9208600700 1 --2- 172.21.15.72:0/2230116372 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f920010fe20 0x7f9200112210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:52.452 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:52.451+0000 7f9208600700 1 --2- 172.21.15.72:0/2230116372 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9200075f70 0x7f920010f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:52.452 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:52.451+0000 7f9208600700 1 -- 172.21.15.72:0/2230116372 >> 172.21.15.72:0/2230116372 conn(0x7f920007a580 msgr2=0x7f920007a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:51:52.452 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:52.451+0000 7f9208600700 1 -- 172.21.15.72:0/2230116372 shutdown_connections 2024-04-26T21:51:52.452 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:52.451+0000 7f9208600700 1 -- 172.21.15.72:0/2230116372 wait complete. 2024-04-26T21:51:52.454 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:52.452+0000 7f9208600700 1 Processor -- start 2024-04-26T21:51:52.454 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:52.452+0000 7f9208600700 1 -- start start 2024-04-26T21:51:52.454 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:52.453+0000 7f9208600700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9200075650 0x7f92001a1910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:52.454 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:52.453+0000 7f9208600700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9200075f70 0x7f92001a1e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:52.454 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:52.453+0000 7f9208600700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f920010fe20 0x7f920019bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:52.454 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:52.453+0000 7f9208600700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f92001156d0 con 0x7f9200075650 2024-04-26T21:51:52.454 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:52.453+0000 7f9208600700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f9200115550 con 0x7f920010fe20 2024-04-26T21:51:52.454 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:52.453+0000 7f9208600700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f9200115850 con 0x7f9200075f70 2024-04-26T21:51:52.454 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:52.453+0000 7f9205c00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9200075650 0x7f92001a1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:52.454 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:52.453+0000 7f9205c00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9200075650 0x7f92001a1910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:55852/0 (socket says 172.21.15.72:55852) 2024-04-26T21:51:52.455 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:52.453+0000 7f9205c00700 1 -- 172.21.15.72:0/518457015 learned_addr learned my addr 172.21.15.72:0/518457015 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:51:52.455 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:52.453+0000 7f9206600700 1 --2- 172.21.15.72:0/518457015 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f920010fe20 0x7f920019bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:52.455 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:52.453+0000 7f9205200700 1 --2- 172.21.15.72:0/518457015 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9200075f70 0x7f92001a1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:52.455 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:52.453+0000 7f9205c00700 1 -- 172.21.15.72:0/518457015 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9200075f70 msgr2=0x7f92001a1e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:52.455 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:52.453+0000 7f9205c00700 1 --2- 172.21.15.72:0/518457015 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9200075f70 0x7f92001a1e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:52.455 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:52.453+0000 7f9205c00700 1 -- 172.21.15.72:0/518457015 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f920010fe20 msgr2=0x7f920019bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:52.455 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:52.453+0000 7f9205c00700 1 --2- 172.21.15.72:0/518457015 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f920010fe20 0x7f920019bad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:52.455 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:52.454+0000 7f9205c00700 1 -- 172.21.15.72:0/518457015 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f920019c230 con 0x7f9200075650 2024-04-26T21:51:52.455 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:52.454+0000 7f9205200700 1 --2- 172.21.15.72:0/518457015 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9200075f70 0x7f92001a1e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:51:52.456 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:52.454+0000 7f9205c00700 1 --2- 172.21.15.72:0/518457015 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9200075650 0x7f92001a1910 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f91f4000c00 tx=0x7f91f4005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:51:52.456 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:52.454+0000 7f91fe000700 1 -- 172.21.15.72:0/518457015 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91f4027070 con 0x7f9200075650 2024-04-26T21:51:52.456 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:52.454+0000 7f91fe000700 1 -- 172.21.15.72:0/518457015 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f91f400fb80 con 0x7f9200075650 2024-04-26T21:51:52.457 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:52.454+0000 7f91fe000700 1 -- 172.21.15.72:0/518457015 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91f4022c30 con 0x7f9200075650 2024-04-26T21:51:52.457 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:52.454+0000 7f9208600700 1 -- 172.21.15.72:0/518457015 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f920019c4c0 con 0x7f9200075650 2024-04-26T21:51:52.457 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:52.454+0000 7f9208600700 1 -- 172.21.15.72:0/518457015 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92001a87e0 con 0x7f9200075650 2024-04-26T21:51:52.457 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:52.456+0000 7f9208600700 1 -- 172.21.15.72:0/518457015 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9200076d40 con 0x7f9200075650 2024-04-26T21:51:52.457 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:52.457+0000 7f91fe000700 1 -- 172.21.15.72:0/518457015 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f91f402a070 con 0x7f9200075650 2024-04-26T21:51:52.460 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:52.460+0000 7f91fe000700 1 --2- 172.21.15.72:0/518457015 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f91e80a3c40 0x7f91e80a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:52.460 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:52.460+0000 7f91fe000700 1 -- 172.21.15.72:0/518457015 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7f91f40d0f50 con 0x7f9200075650 2024-04-26T21:51:52.460 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:52.460+0000 7f9205200700 1 --2- 172.21.15.72:0/518457015 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f91e80a3c40 0x7f91e80a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:52.461 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:52.461+0000 7f9205200700 1 --2- 172.21.15.72:0/518457015 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f91e80a3c40 0x7f91e80a6100 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f91ec005fd0 tx=0x7f91ec005f00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:51:52.464 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:52.464+0000 7f91fe000700 1 -- 172.21.15.72:0/518457015 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f91f405f060 con 0x7f9200075650 2024-04-26T21:51:52.693 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:52.692+0000 7f9208600700 1 -- 172.21.15.72:0/518457015 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f920019cfc0 con 0x7f9200075650 2024-04-26T21:51:52.694 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:52.693+0000 7f91fe000700 1 -- 172.21.15.72:0/518457015 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v83) ==== 93+0+10 (secure 0 0 0) 0x7f91f4013070 con 0x7f9200075650 2024-04-26T21:51:52.694 INFO:teuthology.orchestra.run.smithi072.stdout:pg_num: 1 2024-04-26T21:51:52.697 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:52.696+0000 7f9208600700 1 -- 172.21.15.72:0/518457015 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f91e80a3c40 msgr2=0x7f91e80a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:52.697 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:52.696+0000 7f9208600700 1 --2- 172.21.15.72:0/518457015 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f91e80a3c40 0x7f91e80a6100 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f91ec005fd0 tx=0x7f91ec005f00 comp rx=0 tx=0).stop 2024-04-26T21:51:52.697 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:52.696+0000 7f9208600700 1 -- 172.21.15.72:0/518457015 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9200075650 msgr2=0x7f92001a1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:52.698 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:52.696+0000 7f9208600700 1 --2- 172.21.15.72:0/518457015 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9200075650 0x7f92001a1910 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f91f4000c00 tx=0x7f91f4005ef0 comp rx=0 tx=0).stop 2024-04-26T21:51:52.698 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:52.696+0000 7f9208600700 1 -- 172.21.15.72:0/518457015 shutdown_connections 2024-04-26T21:51:52.698 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:52.696+0000 7f9208600700 1 --2- 172.21.15.72:0/518457015 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f91e80a3c40 0x7f91e80a6100 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:52.698 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:52.696+0000 7f9208600700 1 --2- 172.21.15.72:0/518457015 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9200075650 0x7f92001a1910 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:52.698 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:52.696+0000 7f9208600700 1 --2- 172.21.15.72:0/518457015 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f920010fe20 0x7f920019bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:52.698 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:52.696+0000 7f9208600700 1 --2- 172.21.15.72:0/518457015 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9200075f70 0x7f92001a1e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:52.698 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:52.696+0000 7f9208600700 1 -- 172.21.15.72:0/518457015 >> 172.21.15.72:0/518457015 conn(0x7f920007a580 msgr2=0x7f92001019e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:51:52.699 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:52.696+0000 7f9208600700 1 -- 172.21.15.72:0/518457015 shutdown_connections 2024-04-26T21:51:52.699 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:52.696+0000 7f9208600700 1 -- 172.21.15.72:0/518457015 wait complete. 2024-04-26T21:51:53.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:51:52 smithi072 ceph-mon[28138]: pgmap v334: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:51:53.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:51:52 smithi072 ceph-mon[28138]: from='client.? 172.21.15.72:0/518457015' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T21:51:53.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:51:52 smithi164 ceph-mon[31916]: pgmap v334: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:51:53.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:51:52 smithi164 ceph-mon[31916]: from='client.? 172.21.15.72:0/518457015' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T21:51:53.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:51:52 smithi076 ceph-mon[31823]: pgmap v334: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:51:53.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:51:52 smithi076 ceph-mon[31823]: from='client.? 172.21.15.72:0/518457015' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T21:51:53.659 INFO:tasks.cephadm:Adding mds.a on smithi072 2024-04-26T21:51:53.659 INFO:tasks.cephadm:Adding mds.d on smithi072 2024-04-26T21:51:53.660 INFO:tasks.cephadm:Adding mds.g on smithi072 2024-04-26T21:51:53.660 INFO:tasks.cephadm:Adding mds.j on smithi072 2024-04-26T21:51:53.660 INFO:tasks.cephadm:Adding mds.b on smithi076 2024-04-26T21:51:53.660 INFO:tasks.cephadm:Adding mds.e on smithi076 2024-04-26T21:51:53.660 INFO:tasks.cephadm:Adding mds.h on smithi076 2024-04-26T21:51:53.660 INFO:tasks.cephadm:Adding mds.k on smithi076 2024-04-26T21:51:53.660 INFO:tasks.cephadm:Adding mds.c on smithi164 2024-04-26T21:51:53.660 INFO:tasks.cephadm:Adding mds.f on smithi164 2024-04-26T21:51:53.660 INFO:tasks.cephadm:Adding mds.i on smithi164 2024-04-26T21:51:53.660 INFO:tasks.cephadm:Adding mds.l on smithi164 2024-04-26T21:51:53.660 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a1780950-0415-11ef-bc93-c7b262605968 -- ceph orch apply mds all '12;smithi072=a;smithi072=d;smithi072=g;smithi072=j;smithi076=b;smithi076=e;smithi076=h;smithi076=k;smithi164=c;smithi164=f;smithi164=i;smithi164=l' 2024-04-26T21:51:54.097 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/a1780950-0415-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T21:51:55.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:51:54 smithi164 ceph-mon[31916]: pgmap v335: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:51:55.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:51:54 smithi076 ceph-mon[31823]: pgmap v335: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:51:55.321 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:51:54 smithi072 ceph-mon[28138]: pgmap v335: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:51:56.140 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:51:56.139+0000 7fee11a00700 1 -- 172.21.15.164:0/4236534530 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fee0c1068c0 msgr2=0x7fee0c106d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:56.140 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:51:56.139+0000 7fee11a00700 1 --2- 172.21.15.164:0/4236534530 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fee0c1068c0 0x7fee0c106d20 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fedf8009990 tx=0x7fedf8009ca0 comp rx=0 tx=0).stop 2024-04-26T21:51:56.140 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:51:56.139+0000 7fee11a00700 1 -- 172.21.15.164:0/4236534530 shutdown_connections 2024-04-26T21:51:56.141 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:51:56.139+0000 7fee11a00700 1 --2- 172.21.15.164:0/4236534530 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fee0c107260 0x7fee0c10db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:56.141 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:51:56.139+0000 7fee11a00700 1 --2- 172.21.15.164:0/4236534530 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fee0c1056c0 0x7fee0c105ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:56.141 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:51:56.139+0000 7fee11a00700 1 --2- 172.21.15.164:0/4236534530 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fee0c1068c0 0x7fee0c106d20 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:56.141 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:51:56.139+0000 7fee11a00700 1 -- 172.21.15.164:0/4236534530 >> 172.21.15.164:0/4236534530 conn(0x7fee0c100c80 msgr2=0x7fee0c1030a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:51:56.141 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:51:56.139+0000 7fee11a00700 1 -- 172.21.15.164:0/4236534530 shutdown_connections 2024-04-26T21:51:56.141 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:51:56.139+0000 7fee11a00700 1 -- 172.21.15.164:0/4236534530 wait complete. 2024-04-26T21:51:56.143 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:51:56.140+0000 7fee11a00700 1 Processor -- start 2024-04-26T21:51:56.143 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:51:56.140+0000 7fee11a00700 1 -- start start 2024-04-26T21:51:56.143 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:51:56.140+0000 7fee11a00700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fee0c1056c0 0x7fee0c19b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:56.143 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:51:56.140+0000 7fee11a00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fee0c1068c0 0x7fee0c19bca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:56.143 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:51:56.140+0000 7fee11a00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fee0c107260 0x7fee0c1a2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:56.143 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:51:56.140+0000 7fee11a00700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fee0c111030 con 0x7fee0c1068c0 2024-04-26T21:51:56.143 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:51:56.140+0000 7fee11a00700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fee0c110eb0 con 0x7fee0c1056c0 2024-04-26T21:51:56.143 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:51:56.140+0000 7fee11a00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fee0c1111b0 con 0x7fee0c107260 2024-04-26T21:51:56.144 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:51:56.141+0000 7fee0b400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fee0c107260 0x7fee0c1a2cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:56.144 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:51:56.141+0000 7fee0b400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fee0c107260 0x7fee0c1a2cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:53640/0 (socket says 172.21.15.164:53640) 2024-04-26T21:51:56.144 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:51:56.141+0000 7fee0b400700 1 -- 172.21.15.164:0/3193791052 learned_addr learned my addr 172.21.15.164:0/3193791052 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-04-26T21:51:56.144 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:51:56.141+0000 7fee0aa00700 1 --2- 172.21.15.164:0/3193791052 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fee0c1056c0 0x7fee0c19b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:56.144 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:51:56.141+0000 7fee0a000700 1 --2- 172.21.15.164:0/3193791052 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fee0c1068c0 0x7fee0c19bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:56.144 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:51:56.141+0000 7fee0b400700 1 -- 172.21.15.164:0/3193791052 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fee0c1056c0 msgr2=0x7fee0c19b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:56.144 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:51:56.141+0000 7fee0b400700 1 --2- 172.21.15.164:0/3193791052 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fee0c1056c0 0x7fee0c19b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:56.144 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:51:56.141+0000 7fee0b400700 1 -- 172.21.15.164:0/3193791052 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fee0c1068c0 msgr2=0x7fee0c19bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:56.144 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:51:56.141+0000 7fee0b400700 1 --2- 172.21.15.164:0/3193791052 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fee0c1068c0 0x7fee0c19bca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:56.144 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:51:56.141+0000 7fee0b400700 1 -- 172.21.15.164:0/3193791052 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee0c1a3370 con 0x7fee0c107260 2024-04-26T21:51:56.144 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:51:56.142+0000 7fee0b400700 1 --2- 172.21.15.164:0/3193791052 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fee0c107260 0x7fee0c1a2cd0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fee0400add0 tx=0x7fee0400c330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:51:56.144 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:51:56.142+0000 7fee03400700 1 -- 172.21.15.164:0/3193791052 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee04017020 con 0x7fee0c107260 2024-04-26T21:51:56.144 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:51:56.142+0000 7fee03400700 1 -- 172.21.15.164:0/3193791052 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fee0400cdd0 con 0x7fee0c107260 2024-04-26T21:51:56.144 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:51:56.142+0000 7fee11a00700 1 -- 172.21.15.164:0/3193791052 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee0c1a3660 con 0x7fee0c107260 2024-04-26T21:51:56.145 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:51:56.142+0000 7fee03400700 1 -- 172.21.15.164:0/3193791052 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee04005330 con 0x7fee0c107260 2024-04-26T21:51:56.145 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:51:56.142+0000 7fee11a00700 1 -- 172.21.15.164:0/3193791052 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee0c1a3ba0 con 0x7fee0c107260 2024-04-26T21:51:56.145 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:51:56.143+0000 7fee11a00700 1 -- 172.21.15.164:0/3193791052 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee0c06a110 con 0x7fee0c107260 2024-04-26T21:51:56.146 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:51:56.145+0000 7fee03400700 1 -- 172.21.15.164:0/3193791052 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fee04004ad0 con 0x7fee0c107260 2024-04-26T21:51:56.147 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:51:56.147+0000 7fee03400700 1 --2- 172.21.15.164:0/3193791052 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fedf00a3cc0 0x7fedf00a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:56.147 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:51:56.147+0000 7fee03400700 1 -- 172.21.15.164:0/3193791052 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7fee040cf760 con 0x7fee0c107260 2024-04-26T21:51:56.148 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:51:56.147+0000 7fee0aa00700 1 --2- 172.21.15.164:0/3193791052 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fedf00a3cc0 0x7fedf00a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:56.149 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:51:56.148+0000 7fee0aa00700 1 --2- 172.21.15.164:0/3193791052 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fedf00a3cc0 0x7fedf00a6180 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fedfc005fd0 tx=0x7fedfc005dd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:51:56.152 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:51:56.151+0000 7fee03400700 1 -- 172.21.15.164:0/3193791052 <== mon.1 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fee04021020 con 0x7fee0c107260 2024-04-26T21:51:56.389 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:51:56.388+0000 7fee11a00700 1 -- 172.21.15.164:0/3193791052 --> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi072=a;smithi072=d;smithi072=g;smithi072=j;smithi076=b;smithi076=e;smithi076=h;smithi076=k;smithi164=c;smithi164=f;smithi164=i;smithi164=l", "target": ["mon-mgr", ""]}) -- 0x7fee0c063b20 con 0x7fedf00a3cc0 2024-04-26T21:51:56.406 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:51:56.405+0000 7fee03400700 1 -- 172.21.15.164:0/3193791052 <== mgr.14152 v2:172.21.15.72:6800/588150172 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7fee0c063b20 con 0x7fedf00a3cc0 2024-04-26T21:51:56.407 INFO:teuthology.orchestra.run.smithi164.stdout:Scheduled mds.all update... 2024-04-26T21:51:56.409 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:51:56.408+0000 7fee11a00700 1 -- 172.21.15.164:0/3193791052 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fedf00a3cc0 msgr2=0x7fedf00a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:56.409 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:51:56.408+0000 7fee11a00700 1 --2- 172.21.15.164:0/3193791052 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fedf00a3cc0 0x7fedf00a6180 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fedfc005fd0 tx=0x7fedfc005dd0 comp rx=0 tx=0).stop 2024-04-26T21:51:56.409 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:51:56.408+0000 7fee11a00700 1 -- 172.21.15.164:0/3193791052 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fee0c107260 msgr2=0x7fee0c1a2cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:56.410 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:51:56.408+0000 7fee11a00700 1 --2- 172.21.15.164:0/3193791052 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fee0c107260 0x7fee0c1a2cd0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fee0400add0 tx=0x7fee0400c330 comp rx=0 tx=0).stop 2024-04-26T21:51:56.410 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:51:56.408+0000 7fee11a00700 1 -- 172.21.15.164:0/3193791052 shutdown_connections 2024-04-26T21:51:56.410 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:51:56.408+0000 7fee11a00700 1 --2- 172.21.15.164:0/3193791052 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fedf00a3cc0 0x7fedf00a6180 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:56.410 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:51:56.408+0000 7fee11a00700 1 --2- 172.21.15.164:0/3193791052 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fee0c107260 0x7fee0c1a2cd0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:56.410 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:51:56.408+0000 7fee11a00700 1 --2- 172.21.15.164:0/3193791052 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fee0c1056c0 0x7fee0c19b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:56.410 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:51:56.408+0000 7fee11a00700 1 --2- 172.21.15.164:0/3193791052 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fee0c1068c0 0x7fee0c19bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:56.410 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:51:56.408+0000 7fee11a00700 1 -- 172.21.15.164:0/3193791052 >> 172.21.15.164:0/3193791052 conn(0x7fee0c100c80 msgr2=0x7fee0c109ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:51:56.410 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:51:56.409+0000 7fee11a00700 1 -- 172.21.15.164:0/3193791052 shutdown_connections 2024-04-26T21:51:56.410 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T21:51:56.409+0000 7fee11a00700 1 -- 172.21.15.164:0/3193791052 wait complete. 2024-04-26T21:51:57.102 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:51:56 smithi164 ceph-mon[31916]: pgmap v336: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:51:57.102 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:51:56 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:51:57.102 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:51:56 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:51:57.102 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:51:56 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:51:57.102 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:51:56 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:51:57.102 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:51:56 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:51:57.102 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:51:56 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:51:57.102 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:51:56 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:51:57.103 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:51:56 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:51:57.255 DEBUG:teuthology.orchestra.run.smithi072:mds.a> sudo journalctl -f -n 0 -u ceph-a1780950-0415-11ef-bc93-c7b262605968@mds.a.service 2024-04-26T21:51:57.263 DEBUG:teuthology.orchestra.run.smithi072:mds.d> sudo journalctl -f -n 0 -u ceph-a1780950-0415-11ef-bc93-c7b262605968@mds.d.service 2024-04-26T21:51:57.309 DEBUG:teuthology.orchestra.run.smithi072:mds.g> sudo journalctl -f -n 0 -u ceph-a1780950-0415-11ef-bc93-c7b262605968@mds.g.service 2024-04-26T21:51:57.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:51:56 smithi076 ceph-mon[31823]: pgmap v336: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:51:57.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:51:56 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:51:57.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:51:56 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:51:57.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:51:56 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:51:57.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:51:56 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:51:57.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:51:56 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:51:57.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:51:56 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:51:57.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:51:56 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:51:57.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:51:56 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:51:57.321 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:51:56 smithi072 ceph-mon[28138]: pgmap v336: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:51:57.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:51:56 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:51:57.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:51:56 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:51:57.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:51:56 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:51:57.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:51:56 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:51:57.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:51:56 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:51:57.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:51:56 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:51:57.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:51:56 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:51:57.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:51:56 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:51:57.327 DEBUG:teuthology.orchestra.run.smithi072:mds.j> sudo journalctl -f -n 0 -u ceph-a1780950-0415-11ef-bc93-c7b262605968@mds.j.service 2024-04-26T21:51:57.373 DEBUG:teuthology.orchestra.run.smithi076:mds.b> sudo journalctl -f -n 0 -u ceph-a1780950-0415-11ef-bc93-c7b262605968@mds.b.service 2024-04-26T21:51:57.380 DEBUG:teuthology.orchestra.run.smithi076:mds.e> sudo journalctl -f -n 0 -u ceph-a1780950-0415-11ef-bc93-c7b262605968@mds.e.service 2024-04-26T21:51:57.426 DEBUG:teuthology.orchestra.run.smithi076:mds.h> sudo journalctl -f -n 0 -u ceph-a1780950-0415-11ef-bc93-c7b262605968@mds.h.service 2024-04-26T21:51:57.474 DEBUG:teuthology.orchestra.run.smithi076:mds.k> sudo journalctl -f -n 0 -u ceph-a1780950-0415-11ef-bc93-c7b262605968@mds.k.service 2024-04-26T21:51:57.519 DEBUG:teuthology.orchestra.run.smithi164:mds.c> sudo journalctl -f -n 0 -u ceph-a1780950-0415-11ef-bc93-c7b262605968@mds.c.service 2024-04-26T21:51:57.524 DEBUG:teuthology.orchestra.run.smithi164:mds.f> sudo journalctl -f -n 0 -u ceph-a1780950-0415-11ef-bc93-c7b262605968@mds.f.service 2024-04-26T21:51:57.570 DEBUG:teuthology.orchestra.run.smithi164:mds.i> sudo journalctl -f -n 0 -u ceph-a1780950-0415-11ef-bc93-c7b262605968@mds.i.service 2024-04-26T21:51:57.616 DEBUG:teuthology.orchestra.run.smithi164:mds.l> sudo journalctl -f -n 0 -u ceph-a1780950-0415-11ef-bc93-c7b262605968@mds.l.service 2024-04-26T21:51:57.663 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-04-26T21:51:57.664 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:51:57.883 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:57.882+0000 7fe40d600640 1 -- 172.21.15.72:0/3206324219 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe408104b90 msgr2=0x7fe408104fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:57.883 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:57.882+0000 7fe40d600640 1 --2- 172.21.15.72:0/3206324219 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe408104b90 0x7fe408104fb0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fe3f8009900 tx=0x7fe3f802a3f0 comp rx=0 tx=0).stop 2024-04-26T21:51:57.883 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:57.882+0000 7fe40d600640 1 -- 172.21.15.72:0/3206324219 shutdown_connections 2024-04-26T21:51:57.883 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:57.882+0000 7fe40d600640 1 --2- 172.21.15.72:0/3206324219 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe408106740 0x7fe40810cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:57.883 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:57.882+0000 7fe40d600640 1 --2- 172.21.15.72:0/3206324219 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe408105d80 0x7fe408106200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:57.884 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:57.882+0000 7fe40d600640 1 --2- 172.21.15.72:0/3206324219 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe408104b90 0x7fe408104fb0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:57.884 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:57.882+0000 7fe40d600640 1 -- 172.21.15.72:0/3206324219 >> 172.21.15.72:0/3206324219 conn(0x7fe408100120 msgr2=0x7fe408102540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:51:57.884 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:57.882+0000 7fe40d600640 1 -- 172.21.15.72:0/3206324219 shutdown_connections 2024-04-26T21:51:57.884 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:57.883+0000 7fe40d600640 1 -- 172.21.15.72:0/3206324219 wait complete. 2024-04-26T21:51:57.886 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:57.883+0000 7fe40d600640 1 Processor -- start 2024-04-26T21:51:57.886 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:57.884+0000 7fe40d600640 1 -- start start 2024-04-26T21:51:57.886 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:57.884+0000 7fe40d600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe408104b90 0x7fe40819ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:57.886 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:57.884+0000 7fe40d600640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe408105d80 0x7fe40819b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:57.886 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:57.884+0000 7fe40d600640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe408106740 0x7fe4081a2210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:57.886 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:57.884+0000 7fe40d600640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fe408110590 con 0x7fe408105d80 2024-04-26T21:51:57.886 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:57.884+0000 7fe40d600640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fe408110410 con 0x7fe408106740 2024-04-26T21:51:57.886 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:57.884+0000 7fe40d600640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe408110710 con 0x7fe408104b90 2024-04-26T21:51:57.887 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:57.884+0000 7fe406000640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe408105d80 0x7fe40819b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:57.887 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:57.884+0000 7fe406000640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe408105d80 0x7fe40819b1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:55872/0 (socket says 172.21.15.72:55872) 2024-04-26T21:51:57.887 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:57.884+0000 7fe406000640 1 -- 172.21.15.72:0/627961408 learned_addr learned my addr 172.21.15.72:0/627961408 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:51:57.887 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:57.884+0000 7fe406a00640 1 --2- 172.21.15.72:0/627961408 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe408104b90 0x7fe40819ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:57.887 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:57.885+0000 7fe407400640 1 --2- 172.21.15.72:0/627961408 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe408106740 0x7fe4081a2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:57.887 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:57.885+0000 7fe406000640 1 -- 172.21.15.72:0/627961408 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe408104b90 msgr2=0x7fe40819ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:57.887 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:57.885+0000 7fe406000640 1 --2- 172.21.15.72:0/627961408 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe408104b90 0x7fe40819ac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:57.887 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:57.885+0000 7fe406000640 1 -- 172.21.15.72:0/627961408 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe408106740 msgr2=0x7fe4081a2210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:57.887 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:57.885+0000 7fe406000640 1 --2- 172.21.15.72:0/627961408 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe408106740 0x7fe4081a2210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:57.887 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:57.885+0000 7fe406000640 1 -- 172.21.15.72:0/627961408 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4081a28b0 con 0x7fe408105d80 2024-04-26T21:51:57.887 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:57.885+0000 7fe406000640 1 --2- 172.21.15.72:0/627961408 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe408105d80 0x7fe40819b1b0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fe3f400c890 tx=0x7fe3f400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:51:57.888 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:57.887+0000 7fe3ff400640 1 -- 172.21.15.72:0/627961408 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3f4004220 con 0x7fe408105d80 2024-04-26T21:51:57.888 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:57.887+0000 7fe3ff400640 1 -- 172.21.15.72:0/627961408 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe3f4007e40 con 0x7fe408105d80 2024-04-26T21:51:57.888 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:57.887+0000 7fe40d600640 1 -- 172.21.15.72:0/627961408 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4081a2ba0 con 0x7fe408105d80 2024-04-26T21:51:57.888 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:57.887+0000 7fe3ff400640 1 -- 172.21.15.72:0/627961408 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3f4004a00 con 0x7fe408105d80 2024-04-26T21:51:57.888 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:57.887+0000 7fe40d600640 1 -- 172.21.15.72:0/627961408 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4081a3110 con 0x7fe408105d80 2024-04-26T21:51:57.889 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:57.889+0000 7fe40d600640 1 -- 172.21.15.72:0/627961408 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe40810bed0 con 0x7fe408105d80 2024-04-26T21:51:57.891 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:57.891+0000 7fe3ff400640 1 -- 172.21.15.72:0/627961408 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fe3f40079b0 con 0x7fe408105d80 2024-04-26T21:51:57.892 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:57.892+0000 7fe3ff400640 1 --2- 172.21.15.72:0/627961408 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fe3ec0a3c30 0x7fe3ec0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:57.893 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:57.892+0000 7fe3ff400640 1 -- 172.21.15.72:0/627961408 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7fe3f40cfc50 con 0x7fe408105d80 2024-04-26T21:51:57.893 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:57.893+0000 7fe406a00640 1 --2- 172.21.15.72:0/627961408 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fe3ec0a3c30 0x7fe3ec0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:57.893 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:57.893+0000 7fe406a00640 1 --2- 172.21.15.72:0/627961408 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fe3ec0a3c30 0x7fe3ec0a60f0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fe3f80057c0 tx=0x7fe3f8005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:51:57.897 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:57.897+0000 7fe3ff400640 1 -- 172.21.15.72:0/627961408 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fe3f4014030 con 0x7fe408105d80 2024-04-26T21:51:58.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:51:57 smithi072 ceph-mon[28138]: from='client.24370 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi072=a;smithi072=d;smithi072=g;smithi072=j;smithi076=b;smithi076=e;smithi076=h;smithi076=k;smithi164=c;smithi164=f;smithi164=i;smithi164=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:51:58.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:51:57 smithi072 ceph-mon[28138]: Saving service mds.all spec with placement smithi072=a;smithi072=d;smithi072=g;smithi072=j;smithi076=b;smithi076=e;smithi076=h;smithi076=k;smithi164=c;smithi164=f;smithi164=i;smithi164=l;count:12 2024-04-26T21:51:58.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:51:57 smithi072 ceph-mon[28138]: Deploying daemon mds.l on smithi164 2024-04-26T21:51:58.099 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:58.099+0000 7fe40d600640 1 -- 172.21.15.72:0/627961408 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe408063530 con 0x7fe408105d80 2024-04-26T21:51:58.103 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:58.102+0000 7fe3ff400640 1 -- 172.21.15.72:0/627961408 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+18409 (secure 0 0 0) 0x7fe3f409ca30 con 0x7fe408105d80 2024-04-26T21:51:58.103 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T21:51:58.103 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":83,"fsid":"a1780950-0415-11ef-bc93-c7b262605968","created":"2024-04-26T21:42:01.504173+0000","modified":"2024-04-26T21:51:44.487133+0000","last_up_change":"2024-04-26T21:51:43.482437+0000","last_in_change":"2024-04-26T21:51:18.468587+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:46:29.785427+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"04e06884-b355-4762-9b29-9034b7201e9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6802","nonce":1416765179},{"type":"v1","addr":"172.21.15.72:6803","nonce":1416765179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6804","nonce":1416765179},{"type":"v1","addr":"172.21.15.72:6805","nonce":1416765179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6808","nonce":1416765179},{"type":"v1","addr":"172.21.15.72:6809","nonce":1416765179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6806","nonce":1416765179},{"type":"v1","addr":"172.21.15.72:6807","nonce":1416765179}]},"public_addr":"172.21.15.72:6803/1416765179","cluster_addr":"172.21.15.72:6805/1416765179","heartbeat_back_addr":"172.21.15.72:6809/1416765179","heartbeat_front_addr":"172.21.15.72:6807/1416765179","state":["exists","up"]},{"osd":1,"uuid":"d75a479a-4a2e-48e9-a01e-2b1c9617416a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":755426949},{"type":"v1","addr":"172.21.15.76:6801","nonce":755426949}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":755426949},{"type":"v1","addr":"172.21.15.76:6803","nonce":755426949}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":755426949},{"type":"v1","addr":"172.21.15.76:6807","nonce":755426949}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":755426949},{"type":"v1","addr":"172.21.15.76:6805","nonce":755426949}]},"public_addr":"172.21.15.76:6801/755426949","cluster_addr":"172.21.15.76:6803/755426949","heartbeat_back_addr":"172.21.15.76:6807/755426949","heartbeat_front_addr":"172.21.15.76:6805/755426949","state":["exists","up"]},{"osd":2,"uuid":"b6f8da58-4428-4e8c-a944-7ea5645d6d92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":852265973},{"type":"v1","addr":"172.21.15.164:6801","nonce":852265973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":852265973},{"type":"v1","addr":"172.21.15.164:6803","nonce":852265973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":852265973},{"type":"v1","addr":"172.21.15.164:6807","nonce":852265973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":852265973},{"type":"v1","addr":"172.21.15.164:6805","nonce":852265973}]},"public_addr":"172.21.15.164:6801/852265973","cluster_addr":"172.21.15.164:6803/852265973","heartbeat_back_addr":"172.21.15.164:6807/852265973","heartbeat_front_addr":"172.21.15.164:6805/852265973","state":["exists","up"]},{"osd":3,"uuid":"76d83409-649e-41e5-b4f4-43bbd5b00ea5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6810","nonce":4027763789},{"type":"v1","addr":"172.21.15.72:6811","nonce":4027763789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6812","nonce":4027763789},{"type":"v1","addr":"172.21.15.72:6813","nonce":4027763789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6816","nonce":4027763789},{"type":"v1","addr":"172.21.15.72:6817","nonce":4027763789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6814","nonce":4027763789},{"type":"v1","addr":"172.21.15.72:6815","nonce":4027763789}]},"public_addr":"172.21.15.72:6811/4027763789","cluster_addr":"172.21.15.72:6813/4027763789","heartbeat_back_addr":"172.21.15.72:6817/4027763789","heartbeat_front_addr":"172.21.15.72:6815/4027763789","state":["exists","up"]},{"osd":4,"uuid":"d2cfeea7-1661-4156-bdcd-f770a8d1a878","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":3718867827},{"type":"v1","addr":"172.21.15.76:6809","nonce":3718867827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":3718867827},{"type":"v1","addr":"172.21.15.76:6811","nonce":3718867827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":3718867827},{"type":"v1","addr":"172.21.15.76:6815","nonce":3718867827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":3718867827},{"type":"v1","addr":"172.21.15.76:6813","nonce":3718867827}]},"public_addr":"172.21.15.76:6809/3718867827","cluster_addr":"172.21.15.76:6811/3718867827","heartbeat_back_addr":"172.21.15.76:6815/3718867827","heartbeat_front_addr":"172.21.15.76:6813/3718867827","state":["exists","up"]},{"osd":5,"uuid":"f3896ad6-a040-4ea3-a431-ff9b7fc05c36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":2543954772},{"type":"v1","addr":"172.21.15.164:6809","nonce":2543954772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":2543954772},{"type":"v1","addr":"172.21.15.164:6811","nonce":2543954772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":2543954772},{"type":"v1","addr":"172.21.15.164:6815","nonce":2543954772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":2543954772},{"type":"v1","addr":"172.21.15.164:6813","nonce":2543954772}]},"public_addr":"172.21.15.164:6809/2543954772","cluster_addr":"172.21.15.164:6811/2543954772","heartbeat_back_addr":"172.21.15.164:6815/2543954772","heartbeat_front_addr":"172.21.15.164:6813/2543954772","state":["exists","up"]},{"osd":6,"uuid":"d5d56650-be7e-4ba5-ba3a-d7d473294dab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6818","nonce":2080659027},{"type":"v1","addr":"172.21.15.72:6819","nonce":2080659027}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6820","nonce":2080659027},{"type":"v1","addr":"172.21.15.72:6821","nonce":2080659027}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6824","nonce":2080659027},{"type":"v1","addr":"172.21.15.72:6825","nonce":2080659027}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6822","nonce":2080659027},{"type":"v1","addr":"172.21.15.72:6823","nonce":2080659027}]},"public_addr":"172.21.15.72:6819/2080659027","cluster_addr":"172.21.15.72:6821/2080659027","heartbeat_back_addr":"172.21.15.72:6825/2080659027","heartbeat_front_addr":"172.21.15.72:6823/2080659027","state":["exists","up"]},{"osd":7,"uuid":"18975b5a-9309-4a0f-9f37-3ce09c2e745d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":3144631323},{"type":"v1","addr":"172.21.15.76:6817","nonce":3144631323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":3144631323},{"type":"v1","addr":"172.21.15.76:6819","nonce":3144631323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":3144631323},{"type":"v1","addr":"172.21.15.76:6823","nonce":3144631323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":3144631323},{"type":"v1","addr":"172.21.15.76:6821","nonce":3144631323}]},"public_addr":"172.21.15.76:6817/3144631323","cluster_addr":"172.21.15.76:6819/3144631323","heartbeat_back_addr":"172.21.15.76:6823/3144631323","heartbeat_front_addr":"172.21.15.76:6821/3144631323","state":["exists","up"]},{"osd":8,"uuid":"779e9c1e-8e97-48d4-a854-68ca13f38e0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":627888883},{"type":"v1","addr":"172.21.15.164:6817","nonce":627888883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":627888883},{"type":"v1","addr":"172.21.15.164:6819","nonce":627888883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":627888883},{"type":"v1","addr":"172.21.15.164:6823","nonce":627888883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":627888883},{"type":"v1","addr":"172.21.15.164:6821","nonce":627888883}]},"public_addr":"172.21.15.164:6817/627888883","cluster_addr":"172.21.15.164:6819/627888883","heartbeat_back_addr":"172.21.15.164:6823/627888883","heartbeat_front_addr":"172.21.15.164:6821/627888883","state":["exists","up"]},{"osd":9,"uuid":"d0317850-7b2d-4850-b2cf-0b919505c1fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6826","nonce":1273118485},{"type":"v1","addr":"172.21.15.72:6827","nonce":1273118485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6828","nonce":1273118485},{"type":"v1","addr":"172.21.15.72:6829","nonce":1273118485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6832","nonce":1273118485},{"type":"v1","addr":"172.21.15.72:6833","nonce":1273118485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6830","nonce":1273118485},{"type":"v1","addr":"172.21.15.72:6831","nonce":1273118485}]},"public_addr":"172.21.15.72:6827/1273118485","cluster_addr":"172.21.15.72:6829/1273118485","heartbeat_back_addr":"172.21.15.72:6833/1273118485","heartbeat_front_addr":"172.21.15.72:6831/1273118485","state":["exists","up"]},{"osd":10,"uuid":"e0aa38d8-6011-4d2a-b67b-5bff6d999e47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":2808954459},{"type":"v1","addr":"172.21.15.76:6825","nonce":2808954459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":2808954459},{"type":"v1","addr":"172.21.15.76:6827","nonce":2808954459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":2808954459},{"type":"v1","addr":"172.21.15.76:6831","nonce":2808954459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":2808954459},{"type":"v1","addr":"172.21.15.76:6829","nonce":2808954459}]},"public_addr":"172.21.15.76:6825/2808954459","cluster_addr":"172.21.15.76:6827/2808954459","heartbeat_back_addr":"172.21.15.76:6831/2808954459","heartbeat_front_addr":"172.21.15.76:6829/2808954459","state":["exists","up"]},{"osd":11,"uuid":"981b3d4d-916e-4937-9916-7cc21a891e43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":2354951784},{"type":"v1","addr":"172.21.15.164:6825","nonce":2354951784}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":2354951784},{"type":"v1","addr":"172.21.15.164:6827","nonce":2354951784}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":2354951784},{"type":"v1","addr":"172.21.15.164:6831","nonce":2354951784}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":2354951784},{"type":"v1","addr":"172.21.15.164:6829","nonce":2354951784}]},"public_addr":"172.21.15.164:6825/2354951784","cluster_addr":"172.21.15.164:6827/2354951784","heartbeat_back_addr":"172.21.15.164:6831/2354951784","heartbeat_front_addr":"172.21.15.164:6829/2354951784","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:18.997924+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:52.439081+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:46:25.893288+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:47:00.112807+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:47:35.116874+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:48:09.216344+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:48:45.028853+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:49:19.905397+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:49:55.030536+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:50:30.767662+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:51:06.423267+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:51:41.631211+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.72:0/416277784":"2024-04-27T21:43:09.602791+0000","172.21.15.72:6800/4010358672":"2024-04-27T21:43:09.602791+0000","172.21.15.72:6801/1012617371":"2024-04-27T21:42:34.684837+0000","172.21.15.72:6801/4010358672":"2024-04-27T21:43:09.602791+0000","172.21.15.72:0/3555448179":"2024-04-27T21:43:09.602791+0000","172.21.15.72:6800/1012617371":"2024-04-27T21:42:34.684837+0000","172.21.15.72:0/2920378761":"2024-04-27T21:42:34.684837+0000","172.21.15.72:0/1933740041":"2024-04-27T21:43:09.602791+0000","172.21.15.72:0/3225295775":"2024-04-27T21:42:34.684837+0000","172.21.15.72:0/2326016180":"2024-04-27T21:42:34.684837+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:51:58.105 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:58.105+0000 7fe40d600640 1 -- 172.21.15.72:0/627961408 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fe3ec0a3c30 msgr2=0x7fe3ec0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:58.106 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:58.105+0000 7fe40d600640 1 --2- 172.21.15.72:0/627961408 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fe3ec0a3c30 0x7fe3ec0a60f0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fe3f80057c0 tx=0x7fe3f8005a20 comp rx=0 tx=0).stop 2024-04-26T21:51:58.106 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:58.105+0000 7fe40d600640 1 -- 172.21.15.72:0/627961408 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe408105d80 msgr2=0x7fe40819b1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:58.106 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:58.105+0000 7fe40d600640 1 --2- 172.21.15.72:0/627961408 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe408105d80 0x7fe40819b1b0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fe3f400c890 tx=0x7fe3f400cd50 comp rx=0 tx=0).stop 2024-04-26T21:51:58.106 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:58.105+0000 7fe40d600640 1 -- 172.21.15.72:0/627961408 shutdown_connections 2024-04-26T21:51:58.106 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:58.105+0000 7fe40d600640 1 --2- 172.21.15.72:0/627961408 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe408106740 0x7fe4081a2210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:58.106 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:58.105+0000 7fe40d600640 1 --2- 172.21.15.72:0/627961408 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe408105d80 0x7fe40819b1b0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:58.106 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:58.105+0000 7fe40d600640 1 --2- 172.21.15.72:0/627961408 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fe3ec0a3c30 0x7fe3ec0a60f0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:58.106 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:58.105+0000 7fe40d600640 1 --2- 172.21.15.72:0/627961408 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe408104b90 0x7fe40819ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:58.106 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:58.105+0000 7fe40d600640 1 -- 172.21.15.72:0/627961408 >> 172.21.15.72:0/627961408 conn(0x7fe408100120 msgr2=0x7fe4081090a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:51:58.106 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:58.105+0000 7fe40d600640 1 -- 172.21.15.72:0/627961408 shutdown_connections 2024-04-26T21:51:58.106 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:58.105+0000 7fe40d600640 1 -- 172.21.15.72:0/627961408 wait complete. 2024-04-26T21:51:58.126 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:46:29.785427+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:51:58.127 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:51:58.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:51:57 smithi164 ceph-mon[31916]: from='client.24370 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi072=a;smithi072=d;smithi072=g;smithi072=j;smithi076=b;smithi076=e;smithi076=h;smithi076=k;smithi164=c;smithi164=f;smithi164=i;smithi164=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:51:58.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:51:57 smithi164 ceph-mon[31916]: Saving service mds.all spec with placement smithi072=a;smithi072=d;smithi072=g;smithi072=j;smithi076=b;smithi076=e;smithi076=h;smithi076=k;smithi164=c;smithi164=f;smithi164=i;smithi164=l;count:12 2024-04-26T21:51:58.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:51:57 smithi164 ceph-mon[31916]: Deploying daemon mds.l on smithi164 2024-04-26T21:51:58.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:51:57 smithi076 ceph-mon[31823]: from='client.24370 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi072=a;smithi072=d;smithi072=g;smithi072=j;smithi076=b;smithi076=e;smithi076=h;smithi076=k;smithi164=c;smithi164=f;smithi164=i;smithi164=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:51:58.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:51:57 smithi076 ceph-mon[31823]: Saving service mds.all spec with placement smithi072=a;smithi072=d;smithi072=g;smithi072=j;smithi076=b;smithi076=e;smithi076=h;smithi076=k;smithi164=c;smithi164=f;smithi164=i;smithi164=l;count:12 2024-04-26T21:51:58.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:51:57 smithi076 ceph-mon[31823]: Deploying daemon mds.l on smithi164 2024-04-26T21:51:58.334 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:58.334+0000 7ff332800640 1 -- 172.21.15.72:0/3384396402 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff32c105cc0 msgr2=0x7ff32c1080b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:58.334 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:58.334+0000 7ff332800640 1 --2- 172.21.15.72:0/3384396402 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff32c105cc0 0x7ff32c1080b0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7ff318009900 tx=0x7ff31802a3f0 comp rx=0 tx=0).stop 2024-04-26T21:51:58.334 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:58.334+0000 7ff332800640 1 -- 172.21.15.72:0/3384396402 shutdown_connections 2024-04-26T21:51:58.334 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:58.334+0000 7ff332800640 1 --2- 172.21.15.72:0/3384396402 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff32c1085f0 0x7ff32c10a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:58.334 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:58.334+0000 7ff332800640 1 --2- 172.21.15.72:0/3384396402 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff32c105cc0 0x7ff32c1080b0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:58.334 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:58.334+0000 7ff332800640 1 --2- 172.21.15.72:0/3384396402 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff32c103390 0x7ff32c105780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:58.335 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:58.334+0000 7ff332800640 1 -- 172.21.15.72:0/3384396402 >> 172.21.15.72:0/3384396402 conn(0x7ff32c0fcce0 msgr2=0x7ff32c0ff100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:51:58.335 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:58.334+0000 7ff332800640 1 -- 172.21.15.72:0/3384396402 shutdown_connections 2024-04-26T21:51:58.335 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:58.334+0000 7ff332800640 1 -- 172.21.15.72:0/3384396402 wait complete. 2024-04-26T21:51:58.337 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:58.335+0000 7ff332800640 1 Processor -- start 2024-04-26T21:51:58.337 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:58.335+0000 7ff332800640 1 -- start start 2024-04-26T21:51:58.337 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:58.335+0000 7ff332800640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff32c103390 0x7ff32c19ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:58.337 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:58.335+0000 7ff332800640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff32c105cc0 0x7ff32c19b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:58.337 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:58.335+0000 7ff332800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff32c1085f0 0x7ff32c1a21f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:58.337 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:58.335+0000 7ff332800640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7ff32c10e2e0 con 0x7ff32c105cc0 2024-04-26T21:51:58.337 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:58.335+0000 7ff332800640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7ff32c10e160 con 0x7ff32c103390 2024-04-26T21:51:58.337 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:58.335+0000 7ff332800640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ff32c10e460 con 0x7ff32c1085f0 2024-04-26T21:51:58.337 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:58.335+0000 7ff330a00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff32c105cc0 0x7ff32c19b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:58.337 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:58.336+0000 7ff330a00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff32c105cc0 0x7ff32c19b1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:55880/0 (socket says 172.21.15.72:55880) 2024-04-26T21:51:58.338 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:58.336+0000 7ff330a00640 1 -- 172.21.15.72:0/1521801099 learned_addr learned my addr 172.21.15.72:0/1521801099 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:51:58.338 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:58.336+0000 7ff331e00640 1 --2- 172.21.15.72:0/1521801099 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff32c1085f0 0x7ff32c1a21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:58.338 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:58.336+0000 7ff331400640 1 --2- 172.21.15.72:0/1521801099 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff32c103390 0x7ff32c19ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:58.338 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:58.336+0000 7ff330a00640 1 -- 172.21.15.72:0/1521801099 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff32c1085f0 msgr2=0x7ff32c1a21f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:58.338 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:58.336+0000 7ff330a00640 1 --2- 172.21.15.72:0/1521801099 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff32c1085f0 0x7ff32c1a21f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:58.338 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:58.336+0000 7ff330a00640 1 -- 172.21.15.72:0/1521801099 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff32c103390 msgr2=0x7ff32c19ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:58.338 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:58.336+0000 7ff330a00640 1 --2- 172.21.15.72:0/1521801099 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff32c103390 0x7ff32c19ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:58.338 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:58.336+0000 7ff330a00640 1 -- 172.21.15.72:0/1521801099 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff32c1a2890 con 0x7ff32c105cc0 2024-04-26T21:51:58.338 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:58.336+0000 7ff331400640 1 --2- 172.21.15.72:0/1521801099 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff32c103390 0x7ff32c19ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:51:58.338 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:58.336+0000 7ff330a00640 1 --2- 172.21.15.72:0/1521801099 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff32c105cc0 0x7ff32c19b1c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7ff318031460 tx=0x7ff318003d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:51:58.338 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:58.337+0000 7ff32a000640 1 -- 172.21.15.72:0/1521801099 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff31803f070 con 0x7ff32c105cc0 2024-04-26T21:51:58.338 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:58.337+0000 7ff32a000640 1 -- 172.21.15.72:0/1521801099 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff318031da0 con 0x7ff32c105cc0 2024-04-26T21:51:58.339 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:58.337+0000 7ff32a000640 1 -- 172.21.15.72:0/1521801099 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff318035030 con 0x7ff32c105cc0 2024-04-26T21:51:58.339 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:58.337+0000 7ff332800640 1 -- 172.21.15.72:0/1521801099 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff32c1a2b20 con 0x7ff32c105cc0 2024-04-26T21:51:58.339 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:58.337+0000 7ff332800640 1 -- 172.21.15.72:0/1521801099 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff32c1a2fc0 con 0x7ff32c105cc0 2024-04-26T21:51:58.339 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:58.338+0000 7ff332800640 1 -- 172.21.15.72:0/1521801099 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff32c10a0e0 con 0x7ff32c105cc0 2024-04-26T21:51:58.340 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:58.340+0000 7ff32a000640 1 -- 172.21.15.72:0/1521801099 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7ff318042070 con 0x7ff32c105cc0 2024-04-26T21:51:58.342 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:58.342+0000 7ff32a000640 1 --2- 172.21.15.72:0/1521801099 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7ff3140a3bb0 0x7ff3140a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:58.343 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:58.343+0000 7ff32a000640 1 -- 172.21.15.72:0/1521801099 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7ff3180f0a40 con 0x7ff32c105cc0 2024-04-26T21:51:58.343 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:58.343+0000 7ff331400640 1 --2- 172.21.15.72:0/1521801099 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7ff3140a3bb0 0x7ff3140a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:58.343 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:58.343+0000 7ff331400640 1 --2- 172.21.15.72:0/1521801099 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7ff3140a3bb0 0x7ff3140a6070 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7ff3200058c0 tx=0x7ff320005d50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:51:58.347 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:58.347+0000 7ff32a000640 1 -- 172.21.15.72:0/1521801099 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7ff3180bd8c0 con 0x7ff32c105cc0 2024-04-26T21:51:58.549 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:58.548+0000 7ff332800640 1 -- 172.21.15.72:0/1521801099 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff32c02dcf0 con 0x7ff32c105cc0 2024-04-26T21:51:58.549 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:58.549+0000 7ff32a000640 1 -- 172.21.15.72:0/1521801099 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v83) ==== 93+0+10 (secure 0 0 0) 0x7ff3180bd1c0 con 0x7ff32c105cc0 2024-04-26T21:51:58.550 INFO:teuthology.orchestra.run.smithi072.stdout:pg_num: 1 2024-04-26T21:51:58.553 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:58.552+0000 7ff332800640 1 -- 172.21.15.72:0/1521801099 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7ff3140a3bb0 msgr2=0x7ff3140a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:58.553 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:58.552+0000 7ff332800640 1 --2- 172.21.15.72:0/1521801099 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7ff3140a3bb0 0x7ff3140a6070 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7ff3200058c0 tx=0x7ff320005d50 comp rx=0 tx=0).stop 2024-04-26T21:51:58.553 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:58.552+0000 7ff332800640 1 -- 172.21.15.72:0/1521801099 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff32c105cc0 msgr2=0x7ff32c19b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:58.553 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:58.552+0000 7ff332800640 1 --2- 172.21.15.72:0/1521801099 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff32c105cc0 0x7ff32c19b1c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7ff318031460 tx=0x7ff318003d50 comp rx=0 tx=0).stop 2024-04-26T21:51:58.553 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:58.552+0000 7ff332800640 1 -- 172.21.15.72:0/1521801099 shutdown_connections 2024-04-26T21:51:58.553 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:58.552+0000 7ff332800640 1 --2- 172.21.15.72:0/1521801099 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7ff3140a3bb0 0x7ff3140a6070 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:58.554 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:58.552+0000 7ff332800640 1 --2- 172.21.15.72:0/1521801099 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff32c1085f0 0x7ff32c1a21f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:58.554 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:58.552+0000 7ff332800640 1 --2- 172.21.15.72:0/1521801099 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff32c105cc0 0x7ff32c19b1c0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:58.554 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:58.552+0000 7ff332800640 1 --2- 172.21.15.72:0/1521801099 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff32c103390 0x7ff32c19ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:58.554 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:58.552+0000 7ff332800640 1 -- 172.21.15.72:0/1521801099 >> 172.21.15.72:0/1521801099 conn(0x7ff32c0fcce0 msgr2=0x7ff32c102be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:51:58.554 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:58.552+0000 7ff332800640 1 -- 172.21.15.72:0/1521801099 shutdown_connections 2024-04-26T21:51:58.554 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:58.552+0000 7ff332800640 1 -- 172.21.15.72:0/1521801099 wait complete. 2024-04-26T21:51:58.573 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:51:58.776 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:58.776+0000 7f1e96200640 1 -- 172.21.15.72:0/1363276947 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1e90104960 msgr2=0x7f1e90104d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:58.777 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:58.776+0000 7f1e96200640 1 --2- 172.21.15.72:0/1363276947 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1e90104960 0x7f1e90104d80 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f1e80009ca0 tx=0x7f1e8002a8c0 comp rx=0 tx=0).stop 2024-04-26T21:51:58.777 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:58.776+0000 7f1e96200640 1 -- 172.21.15.72:0/1363276947 shutdown_connections 2024-04-26T21:51:58.777 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:58.776+0000 7f1e96200640 1 --2- 172.21.15.72:0/1363276947 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1e90106510 0x7f1e9010cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:58.777 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:58.776+0000 7f1e96200640 1 --2- 172.21.15.72:0/1363276947 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1e90105b50 0x7f1e90105fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:58.777 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:58.776+0000 7f1e96200640 1 --2- 172.21.15.72:0/1363276947 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1e90104960 0x7f1e90104d80 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:58.777 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:58.776+0000 7f1e96200640 1 -- 172.21.15.72:0/1363276947 >> 172.21.15.72:0/1363276947 conn(0x7f1e900ffeb0 msgr2=0x7f1e90102310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:51:58.777 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:58.776+0000 7f1e96200640 1 -- 172.21.15.72:0/1363276947 shutdown_connections 2024-04-26T21:51:58.777 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:58.776+0000 7f1e96200640 1 -- 172.21.15.72:0/1363276947 wait complete. 2024-04-26T21:51:58.779 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:58.777+0000 7f1e96200640 1 Processor -- start 2024-04-26T21:51:58.779 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:58.777+0000 7f1e96200640 1 -- start start 2024-04-26T21:51:58.780 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:58.777+0000 7f1e96200640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1e90104960 0x7f1e9019aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:58.780 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:58.777+0000 7f1e96200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1e90105b50 0x7f1e9019aff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:58.780 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:58.777+0000 7f1e96200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1e90106510 0x7f1e901a2060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:58.780 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:58.777+0000 7f1e96200640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f1e90110360 con 0x7f1e90104960 2024-04-26T21:51:58.780 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:58.777+0000 7f1e96200640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f1e901101e0 con 0x7f1e90106510 2024-04-26T21:51:58.780 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:58.777+0000 7f1e96200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f1e901104e0 con 0x7f1e90105b50 2024-04-26T21:51:58.780 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:58.777+0000 7f1e94e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1e90104960 0x7f1e9019aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:58.780 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:58.778+0000 7f1e94e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1e90104960 0x7f1e9019aab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:55886/0 (socket says 172.21.15.72:55886) 2024-04-26T21:51:58.780 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:58.778+0000 7f1e94e00640 1 -- 172.21.15.72:0/1750954574 learned_addr learned my addr 172.21.15.72:0/1750954574 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:51:58.780 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:58.778+0000 7f1e8fe00640 1 --2- 172.21.15.72:0/1750954574 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1e90105b50 0x7f1e9019aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:58.781 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:58.778+0000 7f1e95800640 1 --2- 172.21.15.72:0/1750954574 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1e90106510 0x7f1e901a2060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:58.781 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:58.778+0000 7f1e94e00640 1 -- 172.21.15.72:0/1750954574 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1e90105b50 msgr2=0x7f1e9019aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:58.781 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:58.778+0000 7f1e94e00640 1 --2- 172.21.15.72:0/1750954574 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1e90105b50 0x7f1e9019aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:58.781 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:58.778+0000 7f1e94e00640 1 -- 172.21.15.72:0/1750954574 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1e90106510 msgr2=0x7f1e901a2060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:58.781 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:58.778+0000 7f1e94e00640 1 --2- 172.21.15.72:0/1750954574 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1e90106510 0x7f1e901a2060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:58.781 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:58.778+0000 7f1e94e00640 1 -- 172.21.15.72:0/1750954574 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e901a26d0 con 0x7f1e90104960 2024-04-26T21:51:58.781 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:58.778+0000 7f1e94e00640 1 --2- 172.21.15.72:0/1750954574 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1e90104960 0x7f1e9019aab0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f1e8002aeb0 tx=0x7f1e80003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:51:58.781 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:58.779+0000 7f1e8d600640 1 -- 172.21.15.72:0/1750954574 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e80003e10 con 0x7f1e90104960 2024-04-26T21:51:58.781 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:58.779+0000 7f1e8d600640 1 -- 172.21.15.72:0/1750954574 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1e800044d0 con 0x7f1e90104960 2024-04-26T21:51:58.781 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:58.779+0000 7f1e96200640 1 -- 172.21.15.72:0/1750954574 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e901a2960 con 0x7f1e90104960 2024-04-26T21:51:58.781 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:58.779+0000 7f1e8d600640 1 -- 172.21.15.72:0/1750954574 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e80030d80 con 0x7f1e90104960 2024-04-26T21:51:58.781 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:58.779+0000 7f1e96200640 1 -- 172.21.15.72:0/1750954574 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e901a2f00 con 0x7f1e90104960 2024-04-26T21:51:58.782 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:58.780+0000 7f1e96200640 1 -- 172.21.15.72:0/1750954574 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e9010bd40 con 0x7f1e90104960 2024-04-26T21:51:58.782 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:58.782+0000 7f1e8d600640 1 -- 172.21.15.72:0/1750954574 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f1e800305c0 con 0x7f1e90104960 2024-04-26T21:51:58.783 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:58.783+0000 7f1e8d600640 1 --2- 172.21.15.72:0/1750954574 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f1e740a3c30 0x7f1e740a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:58.783 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:58.783+0000 7f1e8d600640 1 -- 172.21.15.72:0/1750954574 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7f1e800f15a0 con 0x7f1e90104960 2024-04-26T21:51:58.784 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:58.783+0000 7f1e8fe00640 1 --2- 172.21.15.72:0/1750954574 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f1e740a3c30 0x7f1e740a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:58.784 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:58.784+0000 7f1e8fe00640 1 --2- 172.21.15.72:0/1750954574 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f1e740a3c30 0x7f1e740a60f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f1e9019bfc0 tx=0x7f1e7c00b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:51:58.787 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:58.787+0000 7f1e8d600640 1 -- 172.21.15.72:0/1750954574 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f1e800be380 con 0x7f1e90104960 2024-04-26T21:51:59.035 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:59.035+0000 7f1e96200640 1 -- 172.21.15.72:0/1750954574 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1e9002dad0 con 0x7f1e90104960 2024-04-26T21:51:59.036 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:59.035+0000 7f1e8d600640 1 -- 172.21.15.72:0/1750954574 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+569 (secure 0 0 0) 0x7f1e800bdc80 con 0x7f1e90104960 2024-04-26T21:51:59.036 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T21:51:59.036 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-04-26T21:51:59.036 INFO:teuthology.orchestra.run.smithi072.stderr:dumped fsmap epoch 1 2024-04-26T21:51:59.041 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:59.038+0000 7f1e96200640 1 -- 172.21.15.72:0/1750954574 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f1e740a3c30 msgr2=0x7f1e740a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:59.041 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:59.038+0000 7f1e96200640 1 --2- 172.21.15.72:0/1750954574 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f1e740a3c30 0x7f1e740a60f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f1e9019bfc0 tx=0x7f1e7c00b390 comp rx=0 tx=0).stop 2024-04-26T21:51:59.042 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:59.038+0000 7f1e96200640 1 -- 172.21.15.72:0/1750954574 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1e90104960 msgr2=0x7f1e9019aab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:59.042 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:59.038+0000 7f1e96200640 1 --2- 172.21.15.72:0/1750954574 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1e90104960 0x7f1e9019aab0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f1e8002aeb0 tx=0x7f1e80003c20 comp rx=0 tx=0).stop 2024-04-26T21:51:59.042 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:59.038+0000 7f1e96200640 1 -- 172.21.15.72:0/1750954574 shutdown_connections 2024-04-26T21:51:59.042 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:59.038+0000 7f1e96200640 1 --2- 172.21.15.72:0/1750954574 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1e90106510 0x7f1e901a2060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:59.042 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:59.038+0000 7f1e96200640 1 --2- 172.21.15.72:0/1750954574 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f1e740a3c30 0x7f1e740a60f0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:59.042 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:59.038+0000 7f1e96200640 1 --2- 172.21.15.72:0/1750954574 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1e90105b50 0x7f1e9019aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:59.042 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:59.038+0000 7f1e96200640 1 --2- 172.21.15.72:0/1750954574 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1e90104960 0x7f1e9019aab0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:59.042 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:59.038+0000 7f1e96200640 1 -- 172.21.15.72:0/1750954574 >> 172.21.15.72:0/1750954574 conn(0x7f1e900ffeb0 msgr2=0x7f1e90108f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:51:59.042 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:59.039+0000 7f1e96200640 1 -- 172.21.15.72:0/1750954574 shutdown_connections 2024-04-26T21:51:59.042 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:51:59.039+0000 7f1e96200640 1 -- 172.21.15.72:0/1750954574 wait complete. 2024-04-26T21:51:59.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:51:58 smithi164 ceph-mon[31916]: pgmap v337: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:51:59.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:51:58 smithi164 ceph-mon[31916]: from='client.? 172.21.15.72:0/627961408' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:51:59.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:51:58 smithi164 ceph-mon[31916]: from='client.? 172.21.15.72:0/1521801099' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T21:51:59.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:51:58 smithi076 ceph-mon[31823]: pgmap v337: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:51:59.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:51:58 smithi076 ceph-mon[31823]: from='client.? 172.21.15.72:0/627961408' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:51:59.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:51:58 smithi076 ceph-mon[31823]: from='client.? 172.21.15.72:0/1521801099' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T21:51:59.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:51:58 smithi072 ceph-mon[28138]: pgmap v337: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:51:59.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:51:58 smithi072 ceph-mon[28138]: from='client.? 172.21.15.72:0/627961408' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:51:59.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:51:58 smithi072 ceph-mon[28138]: from='client.? 172.21.15.72:0/1521801099' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T21:51:59.900 INFO:journalctl@ceph.mds.l.smithi164.stdout:Apr 26 21:51:59 smithi164 systemd[1]: Starting Ceph mds.l for a1780950-0415-11ef-bc93-c7b262605968... 2024-04-26T21:52:00.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:51:59 smithi164 ceph-mon[31916]: from='client.? 172.21.15.72:0/1750954574' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:52:00.193 INFO:journalctl@ceph.mds.l.smithi164.stdout:Apr 26 21:51:59 smithi164 podman[61731]: 2024-04-26 21:51:59.848999852 +0000 UTC m=+0.040614458 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-26T21:52:00.193 INFO:journalctl@ceph.mds.l.smithi164.stdout:Apr 26 21:51:59 smithi164 podman[61731]: 2024-04-26 21:51:59.991570387 +0000 UTC m=+0.183185030 container create 781cee5595e7aad29cb0722224a71255015944254d7c489e40a3affbb709e289 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-mds-l, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-04-26T21:52:00.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:51:59 smithi076 ceph-mon[31823]: from='client.? 172.21.15.72:0/1750954574' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:52:00.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:51:59 smithi072 ceph-mon[28138]: from='client.? 172.21.15.72:0/1750954574' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:52:00.691 INFO:journalctl@ceph.mds.l.smithi164.stdout:Apr 26 21:52:00 smithi164 podman[61731]: 2024-04-26 21:52:00.242901639 +0000 UTC m=+0.434516289 container init 781cee5595e7aad29cb0722224a71255015944254d7c489e40a3affbb709e289 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-mds-l, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-26T21:52:00.691 INFO:journalctl@ceph.mds.l.smithi164.stdout:Apr 26 21:52:00 smithi164 podman[61731]: 2024-04-26 21:52:00.253552688 +0000 UTC m=+0.445167313 container start 781cee5595e7aad29cb0722224a71255015944254d7c489e40a3affbb709e289 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-mds-l, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-26T21:52:00.692 INFO:journalctl@ceph.mds.l.smithi164.stdout:Apr 26 21:52:00 smithi164 ceph-mds[61750]: -- 172.21.15.164:0/2468920221 <== mon.1 v2:172.21.15.164:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561b09881a40 con 0x561b09854800 2024-04-26T21:52:00.692 INFO:journalctl@ceph.mds.l.smithi164.stdout:Apr 26 21:52:00 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-mds-l[61745]: starting mds.l at 2024-04-26T21:52:00.692 INFO:journalctl@ceph.mds.l.smithi164.stdout:Apr 26 21:52:00 smithi164 bash[61731]: 781cee5595e7aad29cb0722224a71255015944254d7c489e40a3affbb709e289 2024-04-26T21:52:00.692 INFO:journalctl@ceph.mds.l.smithi164.stdout:Apr 26 21:52:00 smithi164 systemd[1]: Started Ceph mds.l for a1780950-0415-11ef-bc93-c7b262605968. 2024-04-26T21:52:01.059 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:52:01.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:00 smithi164 ceph-mon[31916]: pgmap v338: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:52:01.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:00 smithi164 ceph-mon[31916]: mds.? [v2:172.21.15.164:6832/4236668465,v1:172.21.15.164:6833/4236668465] up:boot 2024-04-26T21:52:01.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:00 smithi164 ceph-mon[31916]: fsmap 1 up:standby 2024-04-26T21:52:01.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:00 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-26T21:52:01.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:00 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:52:01.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:00 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:52:01.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:00 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:52:01.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:00 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:52:01.192 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:00 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:52:01.192 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:00 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:52:01.225 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:00 smithi072 ceph-mon[28138]: pgmap v338: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:52:01.225 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:00 smithi072 ceph-mon[28138]: mds.? [v2:172.21.15.164:6832/4236668465,v1:172.21.15.164:6833/4236668465] up:boot 2024-04-26T21:52:01.225 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:00 smithi072 ceph-mon[28138]: fsmap 1 up:standby 2024-04-26T21:52:01.225 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:00 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-26T21:52:01.226 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:00 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:52:01.226 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:00 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:52:01.226 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:00 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:52:01.226 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:00 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:52:01.226 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:00 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:52:01.226 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:00 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:52:01.266 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:01.264+0000 7f06bfe00640 1 -- 172.21.15.72:0/3729124018 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f06c0103960 msgr2=0x7f06c0103de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:01.266 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:01.264+0000 7f06bfe00640 1 --2- 172.21.15.72:0/3729124018 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f06c0103960 0x7f06c0103de0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f06ac009900 tx=0x7f06ac02a3f0 comp rx=0 tx=0).stop 2024-04-26T21:52:01.266 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:01.264+0000 7f06bfe00640 1 -- 172.21.15.72:0/3729124018 shutdown_connections 2024-04-26T21:52:01.266 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:01.264+0000 7f06bfe00640 1 --2- 172.21.15.72:0/3729124018 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f06c0104320 0x7f06c010abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:01.266 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:01.264+0000 7f06bfe00640 1 --2- 172.21.15.72:0/3729124018 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f06c0103960 0x7f06c0103de0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:01.266 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:01.264+0000 7f06bfe00640 1 --2- 172.21.15.72:0/3729124018 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f06c0102770 0x7f06c0102b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:01.267 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:01.264+0000 7f06bfe00640 1 -- 172.21.15.72:0/3729124018 >> 172.21.15.72:0/3729124018 conn(0x7f06c00fdcc0 msgr2=0x7f06c0100120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:01.267 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:01.265+0000 7f06bfe00640 1 -- 172.21.15.72:0/3729124018 shutdown_connections 2024-04-26T21:52:01.267 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:01.265+0000 7f06bfe00640 1 -- 172.21.15.72:0/3729124018 wait complete. 2024-04-26T21:52:01.267 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:01.266+0000 7f06bfe00640 1 Processor -- start 2024-04-26T21:52:01.267 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:01.266+0000 7f06bfe00640 1 -- start start 2024-04-26T21:52:01.268 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:01.267+0000 7f06bfe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f06c0102770 0x7f06c0198810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:01.269 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:01.267+0000 7f06bfe00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f06c0103960 0x7f06c0198d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:01.269 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:01.267+0000 7f06bfe00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f06c0104320 0x7f06c019fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:01.269 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:01.267+0000 7f06bfe00640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f06c010e130 con 0x7f06c0103960 2024-04-26T21:52:01.269 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:01.267+0000 7f06bfe00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f06c010dfb0 con 0x7f06c0104320 2024-04-26T21:52:01.269 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:01.267+0000 7f06bfe00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f06c010e2b0 con 0x7f06c0102770 2024-04-26T21:52:01.269 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:01.267+0000 7f06be000640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f06c0103960 0x7f06c0198d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:01.269 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:01.267+0000 7f06be000640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f06c0103960 0x7f06c0198d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:59190/0 (socket says 172.21.15.72:59190) 2024-04-26T21:52:01.269 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:01.267+0000 7f06be000640 1 -- 172.21.15.72:0/2370189739 learned_addr learned my addr 172.21.15.72:0/2370189739 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:52:01.269 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:01.267+0000 7f06bea00640 1 --2- 172.21.15.72:0/2370189739 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f06c0102770 0x7f06c0198810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:01.269 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:01.267+0000 7f06be000640 1 -- 172.21.15.72:0/2370189739 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f06c0102770 msgr2=0x7f06c0198810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:01.269 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:01.267+0000 7f06be000640 1 --2- 172.21.15.72:0/2370189739 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f06c0102770 0x7f06c0198810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:01.269 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:01.267+0000 7f06bf400640 1 --2- 172.21.15.72:0/2370189739 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f06c0104320 0x7f06c019fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:01.270 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:01.267+0000 7f06be000640 1 -- 172.21.15.72:0/2370189739 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f06c0104320 msgr2=0x7f06c019fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:01.270 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:01.267+0000 7f06be000640 1 --2- 172.21.15.72:0/2370189739 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f06c0104320 0x7f06c019fdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:01.270 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:01.267+0000 7f06be000640 1 -- 172.21.15.72:0/2370189739 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06c01a0430 con 0x7f06c0103960 2024-04-26T21:52:01.270 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:01.267+0000 7f06bf400640 1 --2- 172.21.15.72:0/2370189739 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f06c0104320 0x7f06c019fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:52:01.270 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:01.268+0000 7f06be000640 1 --2- 172.21.15.72:0/2370189739 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f06c0103960 0x7f06c0198d50 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f06ac009230 tx=0x7f06ac0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:01.271 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:01.268+0000 7f06c4a00640 1 -- 172.21.15.72:0/2370189739 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06ac03f030 con 0x7f06c0103960 2024-04-26T21:52:01.271 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:01.268+0000 7f06c4a00640 1 -- 172.21.15.72:0/2370189739 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f06ac003c90 con 0x7f06c0103960 2024-04-26T21:52:01.271 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:01.268+0000 7f06bfe00640 1 -- 172.21.15.72:0/2370189739 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06c01a06c0 con 0x7f06c0103960 2024-04-26T21:52:01.271 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:01.268+0000 7f06c4a00640 1 -- 172.21.15.72:0/2370189739 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06ac035070 con 0x7f06c0103960 2024-04-26T21:52:01.271 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:01.268+0000 7f06bfe00640 1 -- 172.21.15.72:0/2370189739 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06c01a0bb0 con 0x7f06c0103960 2024-04-26T21:52:01.271 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:01.269+0000 7f06bfe00640 1 -- 172.21.15.72:0/2370189739 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06c0109b10 con 0x7f06c0103960 2024-04-26T21:52:01.272 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:01.271+0000 7f06c4a00640 1 -- 172.21.15.72:0/2370189739 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f06ac042030 con 0x7f06c0103960 2024-04-26T21:52:01.274 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:01.273+0000 7f06c4a00640 1 --2- 172.21.15.72:0/2370189739 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f06a40a3bb0 0x7f06a40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:01.274 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:01.273+0000 7f06c4a00640 1 -- 172.21.15.72:0/2370189739 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7f06ac0f0860 con 0x7f06c0103960 2024-04-26T21:52:01.274 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:01.273+0000 7f06bea00640 1 --2- 172.21.15.72:0/2370189739 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f06a40a3bb0 0x7f06a40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:01.275 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:01.274+0000 7f06bea00640 1 --2- 172.21.15.72:0/2370189739 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f06a40a3bb0 0x7f06a40a6070 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f06b0005fd0 tx=0x7f06b0005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:01.278 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:01.277+0000 7f06c4a00640 1 -- 172.21.15.72:0/2370189739 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f06ac0bd640 con 0x7f06c0103960 2024-04-26T21:52:01.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:00 smithi076 ceph-mon[31823]: pgmap v338: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:52:01.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:00 smithi076 ceph-mon[31823]: mds.? [v2:172.21.15.164:6832/4236668465,v1:172.21.15.164:6833/4236668465] up:boot 2024-04-26T21:52:01.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:00 smithi076 ceph-mon[31823]: fsmap 1 up:standby 2024-04-26T21:52:01.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:00 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-26T21:52:01.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:00 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:52:01.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:00 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:52:01.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:00 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:52:01.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:00 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:52:01.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:00 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:52:01.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:00 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:52:01.522 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:01.521+0000 7f06bfe00640 1 -- 172.21.15.72:0/2370189739 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f06c0063530 con 0x7f06c0103960 2024-04-26T21:52:01.523 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:01.522+0000 7f06c4a00640 1 -- 172.21.15.72:0/2370189739 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1396 (secure 0 0 0) 0x7f06ac0bcf40 con 0x7f06c0103960 2024-04-26T21:52:01.523 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T21:52:01.523 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24388,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6833/4236668465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":4236668465},{"type":"v1","addr":"172.21.15.164:6833","nonce":4236668465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-26T21:52:01.524 INFO:teuthology.orchestra.run.smithi072.stderr:dumped fsmap epoch 2 2024-04-26T21:52:01.526 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:01.524+0000 7f06bfe00640 1 -- 172.21.15.72:0/2370189739 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f06a40a3bb0 msgr2=0x7f06a40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:01.526 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:01.524+0000 7f06bfe00640 1 --2- 172.21.15.72:0/2370189739 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f06a40a3bb0 0x7f06a40a6070 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f06b0005fd0 tx=0x7f06b0005850 comp rx=0 tx=0).stop 2024-04-26T21:52:01.526 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:01.525+0000 7f06bfe00640 1 -- 172.21.15.72:0/2370189739 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f06c0103960 msgr2=0x7f06c0198d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:01.526 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:01.525+0000 7f06bfe00640 1 --2- 172.21.15.72:0/2370189739 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f06c0103960 0x7f06c0198d50 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f06ac009230 tx=0x7f06ac0042f0 comp rx=0 tx=0).stop 2024-04-26T21:52:01.526 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:01.525+0000 7f06bfe00640 1 -- 172.21.15.72:0/2370189739 shutdown_connections 2024-04-26T21:52:01.527 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:01.525+0000 7f06bfe00640 1 --2- 172.21.15.72:0/2370189739 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f06c0104320 0x7f06c019fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:01.527 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:01.525+0000 7f06bfe00640 1 --2- 172.21.15.72:0/2370189739 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f06c0103960 0x7f06c0198d50 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:01.527 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:01.525+0000 7f06bfe00640 1 --2- 172.21.15.72:0/2370189739 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f06a40a3bb0 0x7f06a40a6070 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:01.527 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:01.525+0000 7f06bfe00640 1 --2- 172.21.15.72:0/2370189739 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f06c0102770 0x7f06c0198810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:01.527 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:01.525+0000 7f06bfe00640 1 -- 172.21.15.72:0/2370189739 >> 172.21.15.72:0/2370189739 conn(0x7f06c00fdcc0 msgr2=0x7f06c0106ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:01.527 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:01.525+0000 7f06bfe00640 1 -- 172.21.15.72:0/2370189739 shutdown_connections 2024-04-26T21:52:01.527 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:01.525+0000 7f06bfe00640 1 -- 172.21.15.72:0/2370189739 wait complete. 2024-04-26T21:52:02.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:01 smithi164 ceph-mon[31916]: Deploying daemon mds.k on smithi076 2024-04-26T21:52:02.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:01 smithi164 ceph-mon[31916]: from='client.? 172.21.15.72:0/2370189739' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:52:02.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:01 smithi076 ceph-mon[31823]: Deploying daemon mds.k on smithi076 2024-04-26T21:52:02.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:01 smithi076 ceph-mon[31823]: from='client.? 172.21.15.72:0/2370189739' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:52:02.321 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:01 smithi072 ceph-mon[28138]: Deploying daemon mds.k on smithi076 2024-04-26T21:52:02.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:01 smithi072 ceph-mon[28138]: from='client.? 172.21.15.72:0/2370189739' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:52:03.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:02 smithi072 ceph-mon[28138]: pgmap v339: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:52:03.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:02 smithi164 ceph-mon[31916]: pgmap v339: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:52:03.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:02 smithi076 ceph-mon[31823]: pgmap v339: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:52:03.545 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:52:03.761 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:03.759+0000 7f51a7e00640 1 -- 172.21.15.72:0/3648822745 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f51a8105ba0 msgr2=0x7f51a8106020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:03.761 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:03.759+0000 7f51a7e00640 1 --2- 172.21.15.72:0/3648822745 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f51a8105ba0 0x7f51a8106020 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f5194009900 tx=0x7f519402a410 comp rx=0 tx=0).stop 2024-04-26T21:52:03.761 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:03.759+0000 7f51a7e00640 1 -- 172.21.15.72:0/3648822745 shutdown_connections 2024-04-26T21:52:03.761 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:03.759+0000 7f51a7e00640 1 --2- 172.21.15.72:0/3648822745 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f51a8106560 0x7f51a810cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:03.761 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:03.759+0000 7f51a7e00640 1 --2- 172.21.15.72:0/3648822745 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f51a8105ba0 0x7f51a8106020 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:03.761 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:03.759+0000 7f51a7e00640 1 --2- 172.21.15.72:0/3648822745 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f51a81049b0 0x7f51a8104dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:03.761 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:03.759+0000 7f51a7e00640 1 -- 172.21.15.72:0/3648822745 >> 172.21.15.72:0/3648822745 conn(0x7f51a80fff40 msgr2=0x7f51a8102360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:03.761 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:03.759+0000 7f51a7e00640 1 -- 172.21.15.72:0/3648822745 shutdown_connections 2024-04-26T21:52:03.761 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:03.759+0000 7f51a7e00640 1 -- 172.21.15.72:0/3648822745 wait complete. 2024-04-26T21:52:03.764 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:03.760+0000 7f51a7e00640 1 Processor -- start 2024-04-26T21:52:03.764 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:03.760+0000 7f51a7e00640 1 -- start start 2024-04-26T21:52:03.764 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:03.760+0000 7f51a7e00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f51a81049b0 0x7f51a819aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:03.764 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:03.760+0000 7f51a7e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f51a8105ba0 0x7f51a819afd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:03.764 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:03.760+0000 7f51a7e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f51a8106560 0x7f51a81a2040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:03.764 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:03.760+0000 7f51a7e00640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f51a8110390 con 0x7f51a8105ba0 2024-04-26T21:52:03.764 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:03.760+0000 7f51a7e00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f51a8110210 con 0x7f51a81049b0 2024-04-26T21:52:03.764 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:03.760+0000 7f51a7e00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f51a8110510 con 0x7f51a8106560 2024-04-26T21:52:03.764 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:03.761+0000 7f51a6000640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f51a8105ba0 0x7f51a819afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:03.764 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:03.761+0000 7f51a6a00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f51a81049b0 0x7f51a819aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:03.764 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:03.761+0000 7f51a6000640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f51a8105ba0 0x7f51a819afd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:59208/0 (socket says 172.21.15.72:59208) 2024-04-26T21:52:03.764 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:03.761+0000 7f51a6000640 1 -- 172.21.15.72:0/31327814 learned_addr learned my addr 172.21.15.72:0/31327814 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:52:03.765 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:03.761+0000 7f51a7400640 1 --2- 172.21.15.72:0/31327814 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f51a8106560 0x7f51a81a2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:03.765 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:03.761+0000 7f51a6000640 1 -- 172.21.15.72:0/31327814 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f51a8106560 msgr2=0x7f51a81a2040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:03.765 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:03.761+0000 7f51a6000640 1 --2- 172.21.15.72:0/31327814 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f51a8106560 0x7f51a81a2040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:03.765 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:03.761+0000 7f51a6000640 1 -- 172.21.15.72:0/31327814 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f51a81049b0 msgr2=0x7f51a819aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:03.765 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:03.761+0000 7f51a6000640 1 --2- 172.21.15.72:0/31327814 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f51a81049b0 0x7f51a819aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:03.765 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:03.761+0000 7f51a6000640 1 -- 172.21.15.72:0/31327814 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51a81a26b0 con 0x7f51a8105ba0 2024-04-26T21:52:03.765 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:03.761+0000 7f51a6a00640 1 --2- 172.21.15.72:0/31327814 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f51a81049b0 0x7f51a819aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:52:03.765 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:03.762+0000 7f51a6000640 1 --2- 172.21.15.72:0/31327814 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f51a8105ba0 0x7f51a819afd0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f5194009a20 tx=0x7f51940040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:03.765 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:03.762+0000 7f51aca00640 1 -- 172.21.15.72:0/31327814 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f519403f030 con 0x7f51a8105ba0 2024-04-26T21:52:03.765 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:03.762+0000 7f51aca00640 1 -- 172.21.15.72:0/31327814 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5194003a80 con 0x7f51a8105ba0 2024-04-26T21:52:03.765 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:03.762+0000 7f51a7e00640 1 -- 172.21.15.72:0/31327814 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51a81a2940 con 0x7f51a8105ba0 2024-04-26T21:52:03.765 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:03.762+0000 7f51aca00640 1 -- 172.21.15.72:0/31327814 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5194035070 con 0x7f51a8105ba0 2024-04-26T21:52:03.765 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:03.762+0000 7f51a7e00640 1 -- 172.21.15.72:0/31327814 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51a81a2e80 con 0x7f51a8105ba0 2024-04-26T21:52:03.766 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:03.763+0000 7f51a7e00640 1 -- 172.21.15.72:0/31327814 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51a810bcf0 con 0x7f51a8105ba0 2024-04-26T21:52:03.766 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:03.765+0000 7f51aca00640 1 -- 172.21.15.72:0/31327814 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f5194042030 con 0x7f51a8105ba0 2024-04-26T21:52:03.768 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:03.766+0000 7f51aca00640 1 --2- 172.21.15.72:0/31327814 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f518c0a3c30 0x7f518c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:03.768 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:03.766+0000 7f51aca00640 1 -- 172.21.15.72:0/31327814 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7f51940f05c0 con 0x7f51a8105ba0 2024-04-26T21:52:03.768 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:03.767+0000 7f51a6a00640 1 --2- 172.21.15.72:0/31327814 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f518c0a3c30 0x7f518c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:03.768 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:03.767+0000 7f51a6a00640 1 --2- 172.21.15.72:0/31327814 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f518c0a3c30 0x7f518c0a60f0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f51980058c0 tx=0x7f519800a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:03.772 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:03.771+0000 7f51aca00640 1 -- 172.21.15.72:0/31327814 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f51940bd3a0 con 0x7f51a8105ba0 2024-04-26T21:52:04.014 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:04.012+0000 7f51a7e00640 1 -- 172.21.15.72:0/31327814 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f51a8063560 con 0x7f51a8105ba0 2024-04-26T21:52:04.015 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:04.013+0000 7f51aca00640 1 -- 172.21.15.72:0/31327814 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1396 (secure 0 0 0) 0x7f51940bcca0 con 0x7f51a8105ba0 2024-04-26T21:52:04.015 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T21:52:04.015 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24388,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6833/4236668465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":4236668465},{"type":"v1","addr":"172.21.15.164:6833","nonce":4236668465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-26T21:52:04.015 INFO:teuthology.orchestra.run.smithi072.stderr:dumped fsmap epoch 2 2024-04-26T21:52:04.018 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:04.017+0000 7f51a7e00640 1 -- 172.21.15.72:0/31327814 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f518c0a3c30 msgr2=0x7f518c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:04.018 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:04.017+0000 7f51a7e00640 1 --2- 172.21.15.72:0/31327814 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f518c0a3c30 0x7f518c0a60f0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f51980058c0 tx=0x7f519800a2d0 comp rx=0 tx=0).stop 2024-04-26T21:52:04.018 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:04.017+0000 7f51a7e00640 1 -- 172.21.15.72:0/31327814 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f51a8105ba0 msgr2=0x7f51a819afd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:04.018 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:04.017+0000 7f51a7e00640 1 --2- 172.21.15.72:0/31327814 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f51a8105ba0 0x7f51a819afd0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f5194009a20 tx=0x7f51940040c0 comp rx=0 tx=0).stop 2024-04-26T21:52:04.018 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:04.017+0000 7f51a7e00640 1 -- 172.21.15.72:0/31327814 shutdown_connections 2024-04-26T21:52:04.018 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:04.017+0000 7f51a7e00640 1 --2- 172.21.15.72:0/31327814 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f518c0a3c30 0x7f518c0a60f0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:04.019 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:04.017+0000 7f51a7e00640 1 --2- 172.21.15.72:0/31327814 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f51a8106560 0x7f51a81a2040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:04.019 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:04.017+0000 7f51a7e00640 1 --2- 172.21.15.72:0/31327814 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f51a8105ba0 0x7f51a819afd0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:04.019 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:04.017+0000 7f51a7e00640 1 --2- 172.21.15.72:0/31327814 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f51a81049b0 0x7f51a819aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:04.019 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:04.017+0000 7f51a7e00640 1 -- 172.21.15.72:0/31327814 >> 172.21.15.72:0/31327814 conn(0x7f51a80fff40 msgr2=0x7f51a8108ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:04.019 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:04.017+0000 7f51a7e00640 1 -- 172.21.15.72:0/31327814 shutdown_connections 2024-04-26T21:52:04.019 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:04.017+0000 7f51a7e00640 1 -- 172.21.15.72:0/31327814 wait complete. 2024-04-26T21:52:04.315 INFO:journalctl@ceph.mds.k.smithi076.stdout:Apr 26 21:52:04 smithi076 podman[60307]: 2024-04-26 21:52:04.152086021 +0000 UTC m=+0.170675694 container create c0523893e3a14c459a8b80fb3582cad49ca432e316067808b57da99afeb40f30 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-mds-k, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-04-26T21:52:04.815 INFO:journalctl@ceph.mds.k.smithi076.stdout:Apr 26 21:52:04 smithi076 podman[60307]: 2024-04-26 21:52:04.426519236 +0000 UTC m=+0.445108891 container init c0523893e3a14c459a8b80fb3582cad49ca432e316067808b57da99afeb40f30 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-mds-k, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff) 2024-04-26T21:52:04.815 INFO:journalctl@ceph.mds.k.smithi076.stdout:Apr 26 21:52:04 smithi076 podman[60307]: 2024-04-26 21:52:04.434816751 +0000 UTC m=+0.453406390 container start c0523893e3a14c459a8b80fb3582cad49ca432e316067808b57da99afeb40f30 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-mds-k, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-04-26T21:52:04.815 INFO:journalctl@ceph.mds.k.smithi076.stdout:Apr 26 21:52:04 smithi076 ceph-mds[60333]: -- 172.21.15.76:0/553338586 <== mon.2 v2:172.21.15.76:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55dda2f79a40 con 0x55dda2f4d000 2024-04-26T21:52:04.815 INFO:journalctl@ceph.mds.k.smithi076.stdout:Apr 26 21:52:04 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-mds-k[60327]: starting mds.k at 2024-04-26T21:52:04.815 INFO:journalctl@ceph.mds.k.smithi076.stdout:Apr 26 21:52:04 smithi076 bash[60307]: c0523893e3a14c459a8b80fb3582cad49ca432e316067808b57da99afeb40f30 2024-04-26T21:52:04.815 INFO:journalctl@ceph.mds.k.smithi076.stdout:Apr 26 21:52:04 smithi076 systemd[1]: Started Ceph mds.k for a1780950-0415-11ef-bc93-c7b262605968. 2024-04-26T21:52:05.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:04 smithi164 ceph-mon[31916]: pgmap v340: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:52:05.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:04 smithi164 ceph-mon[31916]: from='client.? 172.21.15.72:0/31327814' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:52:05.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:04 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:52:05.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:04 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:52:05.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:04 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:52:05.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:04 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:52:05.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:04 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:52:05.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:04 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:52:05.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:04 smithi076 ceph-mon[31823]: pgmap v340: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:52:05.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:04 smithi076 ceph-mon[31823]: from='client.? 172.21.15.72:0/31327814' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:52:05.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:04 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:52:05.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:04 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:52:05.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:04 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:52:05.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:04 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:52:05.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:04 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:52:05.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:04 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:52:05.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:04 smithi072 ceph-mon[28138]: pgmap v340: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:52:05.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:04 smithi072 ceph-mon[28138]: from='client.? 172.21.15.72:0/31327814' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:52:05.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:04 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:52:05.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:04 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:52:05.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:04 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:52:05.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:04 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:52:05.323 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:04 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:52:05.323 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:04 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:52:05.979 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:05 smithi072 ceph-mon[28138]: Deploying daemon mds.j on smithi072 2024-04-26T21:52:05.979 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:05 smithi072 ceph-mon[28138]: mds.? [v2:172.21.15.76:6832/3869483750,v1:172.21.15.76:6833/3869483750] up:boot 2024-04-26T21:52:05.979 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:05 smithi072 ceph-mon[28138]: fsmap 2 up:standby 2024-04-26T21:52:05.979 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:05 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-26T21:52:06.038 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:52:06.244 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:06.244+0000 7fe19d000640 1 -- 172.21.15.72:0/1475153984 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe198104b90 msgr2=0x7fe198104fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:06.244 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:06.244+0000 7fe19d000640 1 --2- 172.21.15.72:0/1475153984 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe198104b90 0x7fe198104fb0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fe188009900 tx=0x7fe18802a410 comp rx=0 tx=0).stop 2024-04-26T21:52:06.245 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:06.244+0000 7fe19d000640 1 -- 172.21.15.72:0/1475153984 shutdown_connections 2024-04-26T21:52:06.245 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:06.244+0000 7fe19d000640 1 --2- 172.21.15.72:0/1475153984 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe198106740 0x7fe19810cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:06.245 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:06.244+0000 7fe19d000640 1 --2- 172.21.15.72:0/1475153984 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe198105d80 0x7fe198106200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:06.245 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:06.244+0000 7fe19d000640 1 --2- 172.21.15.72:0/1475153984 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe198104b90 0x7fe198104fb0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:06.245 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:06.244+0000 7fe19d000640 1 -- 172.21.15.72:0/1475153984 >> 172.21.15.72:0/1475153984 conn(0x7fe198100120 msgr2=0x7fe198102540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:06.245 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:06.244+0000 7fe19d000640 1 -- 172.21.15.72:0/1475153984 shutdown_connections 2024-04-26T21:52:06.245 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:06.244+0000 7fe19d000640 1 -- 172.21.15.72:0/1475153984 wait complete. 2024-04-26T21:52:06.247 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:06.245+0000 7fe19d000640 1 Processor -- start 2024-04-26T21:52:06.247 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:06.245+0000 7fe19d000640 1 -- start start 2024-04-26T21:52:06.247 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:06.245+0000 7fe19d000640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe198104b90 0x7fe19819acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:06.248 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:06.245+0000 7fe19d000640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe198105d80 0x7fe19819b200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:06.248 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:06.245+0000 7fe19d000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe198106740 0x7fe1981a2260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:06.248 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:06.245+0000 7fe19d000640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fe1981105c0 con 0x7fe198105d80 2024-04-26T21:52:06.248 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:06.245+0000 7fe19d000640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fe198110440 con 0x7fe198104b90 2024-04-26T21:52:06.248 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:06.245+0000 7fe19d000640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe198110740 con 0x7fe198106740 2024-04-26T21:52:06.248 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:06.246+0000 7fe195400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe198105d80 0x7fe19819b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:06.248 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:06.246+0000 7fe195400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe198105d80 0x7fe19819b200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:59224/0 (socket says 172.21.15.72:59224) 2024-04-26T21:52:06.248 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:06.246+0000 7fe195400640 1 -- 172.21.15.72:0/3471690471 learned_addr learned my addr 172.21.15.72:0/3471690471 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:52:06.248 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:06.246+0000 7fe195e00640 1 --2- 172.21.15.72:0/3471690471 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe198104b90 0x7fe19819acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:06.248 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:06.246+0000 7fe195400640 1 -- 172.21.15.72:0/3471690471 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe198106740 msgr2=0x7fe1981a2260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:06.248 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:06.246+0000 7fe195400640 1 --2- 172.21.15.72:0/3471690471 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe198106740 0x7fe1981a2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:06.248 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:06.246+0000 7fe195400640 1 -- 172.21.15.72:0/3471690471 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe198104b90 msgr2=0x7fe19819acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:06.249 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:06.246+0000 7fe195400640 1 --2- 172.21.15.72:0/3471690471 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe198104b90 0x7fe19819acc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:06.249 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:06.246+0000 7fe195400640 1 -- 172.21.15.72:0/3471690471 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1981a2900 con 0x7fe198105d80 2024-04-26T21:52:06.249 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:06.246+0000 7fe195400640 1 --2- 172.21.15.72:0/3471690471 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe198105d80 0x7fe19819b200 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fe18400d950 tx=0x7fe18400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:06.249 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:06.247+0000 7fe18ea00640 1 -- 172.21.15.72:0/3471690471 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe184017030 con 0x7fe198105d80 2024-04-26T21:52:06.249 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:06.247+0000 7fe18ea00640 1 -- 172.21.15.72:0/3471690471 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe184004020 con 0x7fe198105d80 2024-04-26T21:52:06.249 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:06.247+0000 7fe19d000640 1 -- 172.21.15.72:0/3471690471 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1981a2bf0 con 0x7fe198105d80 2024-04-26T21:52:06.249 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:06.247+0000 7fe18ea00640 1 -- 172.21.15.72:0/3471690471 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1840055d0 con 0x7fe198105d80 2024-04-26T21:52:06.249 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:06.247+0000 7fe19d000640 1 -- 172.21.15.72:0/3471690471 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1981a3190 con 0x7fe198105d80 2024-04-26T21:52:06.250 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:06.248+0000 7fe19d000640 1 -- 172.21.15.72:0/3471690471 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe19810bf80 con 0x7fe198105d80 2024-04-26T21:52:06.250 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:06.250+0000 7fe18ea00640 1 -- 172.21.15.72:0/3471690471 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fe184009a30 con 0x7fe198105d80 2024-04-26T21:52:06.252 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:06.252+0000 7fe18ea00640 1 --2- 172.21.15.72:0/3471690471 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fe17c0a3c30 0x7fe17c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:06.252 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:06.252+0000 7fe18ea00640 1 -- 172.21.15.72:0/3471690471 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7fe1840cfaa0 con 0x7fe198105d80 2024-04-26T21:52:06.252 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:06.252+0000 7fe195e00640 1 --2- 172.21.15.72:0/3471690471 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fe17c0a3c30 0x7fe17c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:06.253 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:06.253+0000 7fe195e00640 1 --2- 172.21.15.72:0/3471690471 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fe17c0a3c30 0x7fe17c0a60f0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fe188031460 tx=0x7fe188000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:06.256 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:06.256+0000 7fe18ea00640 1 -- 172.21.15.72:0/3471690471 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fe18409c880 con 0x7fe198105d80 2024-04-26T21:52:06.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:05 smithi076 ceph-mon[31823]: Deploying daemon mds.j on smithi072 2024-04-26T21:52:06.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:05 smithi076 ceph-mon[31823]: mds.? [v2:172.21.15.76:6832/3869483750,v1:172.21.15.76:6833/3869483750] up:boot 2024-04-26T21:52:06.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:05 smithi076 ceph-mon[31823]: fsmap 2 up:standby 2024-04-26T21:52:06.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:05 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-26T21:52:06.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:05 smithi164 ceph-mon[31916]: Deploying daemon mds.j on smithi072 2024-04-26T21:52:06.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:05 smithi164 ceph-mon[31916]: mds.? [v2:172.21.15.76:6832/3869483750,v1:172.21.15.76:6833/3869483750] up:boot 2024-04-26T21:52:06.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:05 smithi164 ceph-mon[31916]: fsmap 2 up:standby 2024-04-26T21:52:06.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:05 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-26T21:52:06.506 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:06.506+0000 7fe19d000640 1 -- 172.21.15.72:0/3471690471 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe19802dad0 con 0x7fe198105d80 2024-04-26T21:52:06.506 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:06.506+0000 7fe18ea00640 1 -- 172.21.15.72:0/3471690471 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2221 (secure 0 0 0) 0x7fe18409c180 con 0x7fe198105d80 2024-04-26T21:52:06.507 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T21:52:06.507 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24388,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6833/4236668465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":4236668465},{"type":"v1","addr":"172.21.15.164:6833","nonce":4236668465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":34406,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6833/3869483750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":3869483750},{"type":"v1","addr":"172.21.15.76:6833","nonce":3869483750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-26T21:52:06.507 INFO:teuthology.orchestra.run.smithi072.stderr:dumped fsmap epoch 3 2024-04-26T21:52:06.510 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:06.509+0000 7fe19d000640 1 -- 172.21.15.72:0/3471690471 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fe17c0a3c30 msgr2=0x7fe17c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:06.510 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:06.509+0000 7fe19d000640 1 --2- 172.21.15.72:0/3471690471 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fe17c0a3c30 0x7fe17c0a60f0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fe188031460 tx=0x7fe188000bc0 comp rx=0 tx=0).stop 2024-04-26T21:52:06.510 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:06.509+0000 7fe19d000640 1 -- 172.21.15.72:0/3471690471 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe198105d80 msgr2=0x7fe19819b200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:06.510 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:06.509+0000 7fe19d000640 1 --2- 172.21.15.72:0/3471690471 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe198105d80 0x7fe19819b200 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fe18400d950 tx=0x7fe18400de10 comp rx=0 tx=0).stop 2024-04-26T21:52:06.510 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:06.510+0000 7fe19d000640 1 -- 172.21.15.72:0/3471690471 shutdown_connections 2024-04-26T21:52:06.510 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:06.510+0000 7fe19d000640 1 --2- 172.21.15.72:0/3471690471 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fe17c0a3c30 0x7fe17c0a60f0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:06.510 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:06.510+0000 7fe19d000640 1 --2- 172.21.15.72:0/3471690471 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe198106740 0x7fe1981a2260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:06.510 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:06.510+0000 7fe19d000640 1 --2- 172.21.15.72:0/3471690471 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe198105d80 0x7fe19819b200 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:06.510 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:06.510+0000 7fe19d000640 1 --2- 172.21.15.72:0/3471690471 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe198104b90 0x7fe19819acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:06.510 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:06.510+0000 7fe19d000640 1 -- 172.21.15.72:0/3471690471 >> 172.21.15.72:0/3471690471 conn(0x7fe198100120 msgr2=0x7fe198109150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:06.510 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:06.510+0000 7fe19d000640 1 -- 172.21.15.72:0/3471690471 shutdown_connections 2024-04-26T21:52:06.511 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:06.510+0000 7fe19d000640 1 -- 172.21.15.72:0/3471690471 wait complete. 2024-04-26T21:52:07.042 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:06 smithi072 ceph-mon[28138]: pgmap v341: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:52:07.042 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:06 smithi072 ceph-mon[28138]: from='client.? 172.21.15.72:0/3471690471' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:52:07.217 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:06 smithi076 ceph-mon[31823]: pgmap v341: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:52:07.217 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:06 smithi076 ceph-mon[31823]: from='client.? 172.21.15.72:0/3471690471' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:52:07.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:06 smithi164 ceph-mon[31916]: pgmap v341: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:52:07.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:06 smithi164 ceph-mon[31916]: from='client.? 172.21.15.72:0/3471690471' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:52:08.203 INFO:journalctl@ceph.mds.j.smithi072.stdout:Apr 26 21:52:07 smithi072 systemd[1]: Starting Ceph mds.j for a1780950-0415-11ef-bc93-c7b262605968... 2024-04-26T21:52:08.505 INFO:journalctl@ceph.mds.j.smithi072.stdout:Apr 26 21:52:08 smithi072 podman[69076]: 2024-04-26 21:52:08.106858338 +0000 UTC m=+0.033205068 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-26T21:52:08.505 INFO:journalctl@ceph.mds.j.smithi072.stdout:Apr 26 21:52:08 smithi072 podman[69076]: 2024-04-26 21:52:08.280212402 +0000 UTC m=+0.206559155 container create efaf9a1d4cd2e9d3bdbb79292b0f581e3d95744bdf4ed9961658e3d0f3d7d8ff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-mds-j, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T21:52:08.505 INFO:journalctl@ceph.mds.j.smithi072.stdout:Apr 26 21:52:08 smithi072 podman[69076]: 2024-04-26 21:52:08.504281734 +0000 UTC m=+0.430628490 container init efaf9a1d4cd2e9d3bdbb79292b0f581e3d95744bdf4ed9961658e3d0f3d7d8ff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-mds-j, org.label-schema.build-date=20240424, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-04-26T21:52:08.529 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:52:08.727 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:08.726+0000 7eff1e000640 1 -- 172.21.15.72:0/756394066 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7eff18108f90 msgr2=0x7eff181093b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:08.727 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:08.726+0000 7eff1e000640 1 --2- 172.21.15.72:0/756394066 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7eff18108f90 0x7eff181093b0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7eff08009900 tx=0x7eff0802a3f0 comp rx=0 tx=0).stop 2024-04-26T21:52:08.727 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:08.726+0000 7eff1e000640 1 -- 172.21.15.72:0/756394066 shutdown_connections 2024-04-26T21:52:08.727 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:08.726+0000 7eff1e000640 1 --2- 172.21.15.72:0/756394066 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7eff1810ab40 0x7eff181113d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:08.727 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:08.726+0000 7eff1e000640 1 --2- 172.21.15.72:0/756394066 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7eff1810a180 0x7eff1810a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:08.727 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:08.726+0000 7eff1e000640 1 --2- 172.21.15.72:0/756394066 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7eff18108f90 0x7eff181093b0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:08.727 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:08.726+0000 7eff1e000640 1 -- 172.21.15.72:0/756394066 >> 172.21.15.72:0/756394066 conn(0x7eff18077f40 msgr2=0x7eff1807a360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:08.728 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:08.726+0000 7eff1e000640 1 -- 172.21.15.72:0/756394066 shutdown_connections 2024-04-26T21:52:08.728 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:08.726+0000 7eff1e000640 1 -- 172.21.15.72:0/756394066 wait complete. 2024-04-26T21:52:08.729 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:08.727+0000 7eff1e000640 1 Processor -- start 2024-04-26T21:52:08.730 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:08.727+0000 7eff1e000640 1 -- start start 2024-04-26T21:52:08.730 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:08.728+0000 7eff1e000640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7eff18108f90 0x7eff1819f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:08.730 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:08.728+0000 7eff1e000640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7eff1810a180 0x7eff1819f570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:08.730 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:08.728+0000 7eff1e000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7eff1810ab40 0x7eff181a65e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:08.730 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:08.728+0000 7eff1e000640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7eff181149d0 con 0x7eff18108f90 2024-04-26T21:52:08.730 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:08.728+0000 7eff1e000640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7eff18114850 con 0x7eff1810a180 2024-04-26T21:52:08.730 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:08.728+0000 7eff1e000640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7eff18114b50 con 0x7eff1810ab40 2024-04-26T21:52:08.730 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:08.728+0000 7eff17400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7eff18108f90 0x7eff1819f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:08.730 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:08.728+0000 7eff17400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7eff18108f90 0x7eff1819f030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:59258/0 (socket says 172.21.15.72:59258) 2024-04-26T21:52:08.730 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:08.728+0000 7eff17400640 1 -- 172.21.15.72:0/2300896725 learned_addr learned my addr 172.21.15.72:0/2300896725 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:52:08.730 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:08.728+0000 7eff17400640 1 -- 172.21.15.72:0/2300896725 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7eff1810ab40 msgr2=0x7eff181a65e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:08.730 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:08.728+0000 7eff17e00640 1 --2- 172.21.15.72:0/2300896725 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7eff1810ab40 0x7eff181a65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:08.731 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:08.728+0000 7eff17400640 1 --2- 172.21.15.72:0/2300896725 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7eff1810ab40 0x7eff181a65e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:08.731 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:08.728+0000 7eff17400640 1 -- 172.21.15.72:0/2300896725 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7eff1810a180 msgr2=0x7eff1819f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:08.731 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:08.728+0000 7eff17400640 1 --2- 172.21.15.72:0/2300896725 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7eff1810a180 0x7eff1819f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:08.731 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:08.729+0000 7eff17400640 1 -- 172.21.15.72:0/2300896725 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff181a6c50 con 0x7eff18108f90 2024-04-26T21:52:08.731 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:08.729+0000 7eff17400640 1 --2- 172.21.15.72:0/2300896725 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7eff18108f90 0x7eff1819f030 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7eff08031460 tx=0x7eff08003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:08.731 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:08.729+0000 7eff14c00640 1 -- 172.21.15.72:0/2300896725 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff0803f070 con 0x7eff18108f90 2024-04-26T21:52:08.731 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:08.729+0000 7eff14c00640 1 -- 172.21.15.72:0/2300896725 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7eff08031da0 con 0x7eff18108f90 2024-04-26T21:52:08.731 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:08.729+0000 7eff1e000640 1 -- 172.21.15.72:0/2300896725 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff181a6ee0 con 0x7eff18108f90 2024-04-26T21:52:08.731 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:08.729+0000 7eff14c00640 1 -- 172.21.15.72:0/2300896725 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff08035030 con 0x7eff18108f90 2024-04-26T21:52:08.731 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:08.729+0000 7eff1e000640 1 -- 172.21.15.72:0/2300896725 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff181a7340 con 0x7eff18108f90 2024-04-26T21:52:08.732 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:08.730+0000 7eff1e000640 1 -- 172.21.15.72:0/2300896725 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff18110330 con 0x7eff18108f90 2024-04-26T21:52:08.732 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:08.732+0000 7eff14c00640 1 -- 172.21.15.72:0/2300896725 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7eff08042070 con 0x7eff18108f90 2024-04-26T21:52:08.734 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:08.734+0000 7eff14c00640 1 --2- 172.21.15.72:0/2300896725 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7efef80a3bb0 0x7efef80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:08.734 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:08.734+0000 7eff14c00640 1 -- 172.21.15.72:0/2300896725 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7eff080f0980 con 0x7eff18108f90 2024-04-26T21:52:08.734 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:08.734+0000 7eff0fe00640 1 --2- 172.21.15.72:0/2300896725 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7efef80a3bb0 0x7efef80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:08.735 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:08.735+0000 7eff0fe00640 1 --2- 172.21.15.72:0/2300896725 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7efef80a3bb0 0x7efef80a6070 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7eff181a0540 tx=0x7eff00005c70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:08.739 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:08.739+0000 7eff14c00640 1 -- 172.21.15.72:0/2300896725 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7eff080bd760 con 0x7eff18108f90 2024-04-26T21:52:08.823 INFO:journalctl@ceph.mds.j.smithi072.stdout:Apr 26 21:52:08 smithi072 podman[69076]: 2024-04-26 21:52:08.514479913 +0000 UTC m=+0.440826646 container start efaf9a1d4cd2e9d3bdbb79292b0f581e3d95744bdf4ed9961658e3d0f3d7d8ff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-mds-j, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T21:52:08.823 INFO:journalctl@ceph.mds.j.smithi072.stdout:Apr 26 21:52:08 smithi072 ceph-mds[69095]: -- 172.21.15.72:0/1213975078 <== mon.0 v2:172.21.15.72:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5627a7185a40 con 0x5627a7158800 2024-04-26T21:52:08.823 INFO:journalctl@ceph.mds.j.smithi072.stdout:Apr 26 21:52:08 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-mds-j[69091]: starting mds.j at 2024-04-26T21:52:08.823 INFO:journalctl@ceph.mds.j.smithi072.stdout:Apr 26 21:52:08 smithi072 bash[69076]: efaf9a1d4cd2e9d3bdbb79292b0f581e3d95744bdf4ed9961658e3d0f3d7d8ff 2024-04-26T21:52:08.823 INFO:journalctl@ceph.mds.j.smithi072.stdout:Apr 26 21:52:08 smithi072 systemd[1]: Started Ceph mds.j for a1780950-0415-11ef-bc93-c7b262605968. 2024-04-26T21:52:08.990 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:08.989+0000 7eff1e000640 1 -- 172.21.15.72:0/2300896725 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7eff18063600 con 0x7eff18108f90 2024-04-26T21:52:08.990 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:08.990+0000 7eff14c00640 1 -- 172.21.15.72:0/2300896725 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3046 (secure 0 0 0) 0x7eff080bd060 con 0x7eff18108f90 2024-04-26T21:52:08.991 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T21:52:08.991 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6835/1320381959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6834","nonce":1320381959},{"type":"v1","addr":"172.21.15.72:6835","nonce":1320381959}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24388,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6833/4236668465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":4236668465},{"type":"v1","addr":"172.21.15.164:6833","nonce":4236668465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":34406,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6833/3869483750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":3869483750},{"type":"v1","addr":"172.21.15.76:6833","nonce":3869483750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-26T21:52:08.991 INFO:teuthology.orchestra.run.smithi072.stderr:dumped fsmap epoch 4 2024-04-26T21:52:08.993 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:08.993+0000 7eff1e000640 1 -- 172.21.15.72:0/2300896725 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7efef80a3bb0 msgr2=0x7efef80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:08.993 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:08.993+0000 7eff1e000640 1 --2- 172.21.15.72:0/2300896725 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7efef80a3bb0 0x7efef80a6070 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7eff181a0540 tx=0x7eff00005c70 comp rx=0 tx=0).stop 2024-04-26T21:52:08.993 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:08.993+0000 7eff1e000640 1 -- 172.21.15.72:0/2300896725 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7eff18108f90 msgr2=0x7eff1819f030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:08.994 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:08.993+0000 7eff1e000640 1 --2- 172.21.15.72:0/2300896725 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7eff18108f90 0x7eff1819f030 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7eff08031460 tx=0x7eff08003c80 comp rx=0 tx=0).stop 2024-04-26T21:52:08.994 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:08.993+0000 7eff1e000640 1 -- 172.21.15.72:0/2300896725 shutdown_connections 2024-04-26T21:52:08.994 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:08.993+0000 7eff1e000640 1 --2- 172.21.15.72:0/2300896725 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7efef80a3bb0 0x7efef80a6070 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:08.994 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:08.993+0000 7eff1e000640 1 --2- 172.21.15.72:0/2300896725 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7eff1810ab40 0x7eff181a65e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:08.994 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:08.993+0000 7eff1e000640 1 --2- 172.21.15.72:0/2300896725 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7eff1810a180 0x7eff1819f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:08.994 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:08.993+0000 7eff1e000640 1 --2- 172.21.15.72:0/2300896725 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7eff18108f90 0x7eff1819f030 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:08.994 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:08.994+0000 7eff1e000640 1 -- 172.21.15.72:0/2300896725 >> 172.21.15.72:0/2300896725 conn(0x7eff18077f40 msgr2=0x7eff1810d500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:08.995 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:08.994+0000 7eff1e000640 1 -- 172.21.15.72:0/2300896725 shutdown_connections 2024-04-26T21:52:08.995 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:08.994+0000 7eff1e000640 1 -- 172.21.15.72:0/2300896725 wait complete. 2024-04-26T21:52:09.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:08 smithi076 ceph-mon[31823]: pgmap v342: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:52:09.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:08 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:52:09.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:08 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:52:09.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:08 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:52:09.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:08 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:52:09.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:08 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:52:09.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:08 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:52:09.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:08 smithi072 ceph-mon[28138]: pgmap v342: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:52:09.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:08 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:52:09.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:08 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:52:09.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:08 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:52:09.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:08 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:52:09.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:08 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:52:09.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:08 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:52:09.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:08 smithi164 ceph-mon[31916]: pgmap v342: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:52:09.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:08 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:52:09.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:08 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:52:09.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:08 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:52:09.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:08 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:52:09.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:08 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:52:09.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:08 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:52:10.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:09 smithi076 ceph-mon[31823]: Deploying daemon mds.d on smithi072 2024-04-26T21:52:10.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:09 smithi076 ceph-mon[31823]: mds.? [v2:172.21.15.72:6834/1320381959,v1:172.21.15.72:6835/1320381959] up:boot 2024-04-26T21:52:10.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:09 smithi076 ceph-mon[31823]: fsmap 3 up:standby 2024-04-26T21:52:10.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:09 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-26T21:52:10.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:09 smithi076 ceph-mon[31823]: from='client.? 172.21.15.72:0/2300896725' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:52:10.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:09 smithi072 ceph-mon[28138]: Deploying daemon mds.d on smithi072 2024-04-26T21:52:10.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:09 smithi072 ceph-mon[28138]: mds.? [v2:172.21.15.72:6834/1320381959,v1:172.21.15.72:6835/1320381959] up:boot 2024-04-26T21:52:10.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:09 smithi072 ceph-mon[28138]: fsmap 3 up:standby 2024-04-26T21:52:10.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:09 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-26T21:52:10.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:09 smithi072 ceph-mon[28138]: from='client.? 172.21.15.72:0/2300896725' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:52:10.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:09 smithi164 ceph-mon[31916]: Deploying daemon mds.d on smithi072 2024-04-26T21:52:10.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:09 smithi164 ceph-mon[31916]: mds.? [v2:172.21.15.72:6834/1320381959,v1:172.21.15.72:6835/1320381959] up:boot 2024-04-26T21:52:10.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:09 smithi164 ceph-mon[31916]: fsmap 3 up:standby 2024-04-26T21:52:10.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:09 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-26T21:52:10.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:09 smithi164 ceph-mon[31916]: from='client.? 172.21.15.72:0/2300896725' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:52:11.012 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:52:11.313 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:10 smithi072 ceph-mon[28138]: pgmap v343: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:52:11.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:10 smithi076 ceph-mon[31823]: pgmap v343: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:52:11.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:10 smithi164 ceph-mon[31916]: pgmap v343: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:52:11.452 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:11.451+0000 7fb235c00640 1 -- 172.21.15.72:0/2755744598 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb230109930 msgr2=0x7fb230109d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:11.452 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:11.451+0000 7fb235c00640 1 --2- 172.21.15.72:0/2755744598 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb230109930 0x7fb230109d50 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fb218009900 tx=0x7fb21802a410 comp rx=0 tx=0).stop 2024-04-26T21:52:11.452 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:11.451+0000 7fb235c00640 1 -- 172.21.15.72:0/2755744598 shutdown_connections 2024-04-26T21:52:11.452 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:11.451+0000 7fb235c00640 1 --2- 172.21.15.72:0/2755744598 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb2301111b0 0x7fb2301135a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:11.452 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:11.451+0000 7fb235c00640 1 --2- 172.21.15.72:0/2755744598 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb23010a290 0x7fb230110c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:11.452 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:11.451+0000 7fb235c00640 1 --2- 172.21.15.72:0/2755744598 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb230109930 0x7fb230109d50 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:11.452 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:11.451+0000 7fb235c00640 1 -- 172.21.15.72:0/2755744598 >> 172.21.15.72:0/2755744598 conn(0x7fb23006fe00 msgr2=0x7fb230072240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:11.452 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:11.452+0000 7fb235c00640 1 -- 172.21.15.72:0/2755744598 shutdown_connections 2024-04-26T21:52:11.452 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:11.452+0000 7fb235c00640 1 -- 172.21.15.72:0/2755744598 wait complete. 2024-04-26T21:52:11.453 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:11.453+0000 7fb235c00640 1 Processor -- start 2024-04-26T21:52:11.453 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:11.453+0000 7fb235c00640 1 -- start start 2024-04-26T21:52:11.453 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:11.453+0000 7fb235c00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb230109930 0x7fb23010c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:11.453 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:11.453+0000 7fb235c00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb23010a290 0x7fb23010ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:11.454 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:11.453+0000 7fb235c00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb2301111b0 0x7fb23010b140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:11.454 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:11.453+0000 7fb235c00640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fb230116c40 con 0x7fb2301111b0 2024-04-26T21:52:11.454 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:11.453+0000 7fb235c00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fb230116ac0 con 0x7fb230109930 2024-04-26T21:52:11.454 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:11.453+0000 7fb235c00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fb230116dc0 con 0x7fb23010a290 2024-04-26T21:52:11.456 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:11.454+0000 7fb22f400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb2301111b0 0x7fb23010b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:11.456 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:11.454+0000 7fb22f400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb2301111b0 0x7fb23010b140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:60150/0 (socket says 172.21.15.72:60150) 2024-04-26T21:52:11.456 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:11.454+0000 7fb22f400640 1 -- 172.21.15.72:0/3895899866 learned_addr learned my addr 172.21.15.72:0/3895899866 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:52:11.456 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:11.454+0000 7fb22ea00640 1 --2- 172.21.15.72:0/3895899866 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb230109930 0x7fb23010c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:11.456 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:11.454+0000 7fb22e000640 1 --2- 172.21.15.72:0/3895899866 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb23010a290 0x7fb23010ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:11.456 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:11.454+0000 7fb22f400640 1 -- 172.21.15.72:0/3895899866 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb23010a290 msgr2=0x7fb23010ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:11.456 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:11.454+0000 7fb22f400640 1 --2- 172.21.15.72:0/3895899866 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb23010a290 0x7fb23010ac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:11.456 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:11.454+0000 7fb22f400640 1 -- 172.21.15.72:0/3895899866 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb230109930 msgr2=0x7fb23010c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:11.456 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:11.454+0000 7fb22f400640 1 --2- 172.21.15.72:0/3895899866 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb230109930 0x7fb23010c5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:11.456 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:11.454+0000 7fb22f400640 1 -- 172.21.15.72:0/3895899866 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb23010b9d0 con 0x7fb2301111b0 2024-04-26T21:52:11.456 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:11.455+0000 7fb22f400640 1 --2- 172.21.15.72:0/3895899866 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb2301111b0 0x7fb23010b140 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fb228009c90 tx=0x7fb22800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:11.456 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:11.455+0000 7fb227400640 1 -- 172.21.15.72:0/3895899866 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb228003b70 con 0x7fb2301111b0 2024-04-26T21:52:11.456 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:11.455+0000 7fb227400640 1 -- 172.21.15.72:0/3895899866 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb228003d10 con 0x7fb2301111b0 2024-04-26T21:52:11.457 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:11.455+0000 7fb227400640 1 -- 172.21.15.72:0/3895899866 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb228010720 con 0x7fb2301111b0 2024-04-26T21:52:11.457 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:11.455+0000 7fb235c00640 1 -- 172.21.15.72:0/3895899866 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2301ab4e0 con 0x7fb2301111b0 2024-04-26T21:52:11.457 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:11.455+0000 7fb235c00640 1 -- 172.21.15.72:0/3895899866 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2301aba60 con 0x7fb2301111b0 2024-04-26T21:52:11.457 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:11.457+0000 7fb235c00640 1 -- 172.21.15.72:0/3895899866 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb230110370 con 0x7fb2301111b0 2024-04-26T21:52:11.459 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:11.459+0000 7fb227400640 1 -- 172.21.15.72:0/3895899866 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fb228004000 con 0x7fb2301111b0 2024-04-26T21:52:11.461 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:11.461+0000 7fb227400640 1 --2- 172.21.15.72:0/3895899866 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fb2140a3c30 0x7fb2140a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:11.462 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:11.462+0000 7fb227400640 1 -- 172.21.15.72:0/3895899866 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7fb2280d0530 con 0x7fb2301111b0 2024-04-26T21:52:11.462 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:11.462+0000 7fb22ea00640 1 --2- 172.21.15.72:0/3895899866 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fb2140a3c30 0x7fb2140a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:11.462 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:11.462+0000 7fb22ea00640 1 --2- 172.21.15.72:0/3895899866 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fb2140a3c30 0x7fb2140a60f0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fb2180057c0 tx=0x7fb218005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:11.466 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:11.466+0000 7fb227400640 1 -- 172.21.15.72:0/3895899866 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fb22809d310 con 0x7fb2301111b0 2024-04-26T21:52:11.710 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:11.710+0000 7fb235c00640 1 -- 172.21.15.72:0/3895899866 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb23002dad0 con 0x7fb2301111b0 2024-04-26T21:52:11.711 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:11.711+0000 7fb227400640 1 -- 172.21.15.72:0/3895899866 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3046 (secure 0 0 0) 0x7fb22809cc10 con 0x7fb2301111b0 2024-04-26T21:52:11.712 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T21:52:11.712 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6835/1320381959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6834","nonce":1320381959},{"type":"v1","addr":"172.21.15.72:6835","nonce":1320381959}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24388,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6833/4236668465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":4236668465},{"type":"v1","addr":"172.21.15.164:6833","nonce":4236668465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":34406,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6833/3869483750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":3869483750},{"type":"v1","addr":"172.21.15.76:6833","nonce":3869483750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-26T21:52:11.712 INFO:teuthology.orchestra.run.smithi072.stderr:dumped fsmap epoch 4 2024-04-26T21:52:11.714 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:11.714+0000 7fb235c00640 1 -- 172.21.15.72:0/3895899866 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fb2140a3c30 msgr2=0x7fb2140a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:11.714 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:11.714+0000 7fb235c00640 1 --2- 172.21.15.72:0/3895899866 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fb2140a3c30 0x7fb2140a60f0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fb2180057c0 tx=0x7fb218005a20 comp rx=0 tx=0).stop 2024-04-26T21:52:11.714 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:11.714+0000 7fb235c00640 1 -- 172.21.15.72:0/3895899866 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb2301111b0 msgr2=0x7fb23010b140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:11.715 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:11.714+0000 7fb235c00640 1 --2- 172.21.15.72:0/3895899866 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb2301111b0 0x7fb23010b140 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fb228009c90 tx=0x7fb22800c6a0 comp rx=0 tx=0).stop 2024-04-26T21:52:11.715 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:11.714+0000 7fb235c00640 1 -- 172.21.15.72:0/3895899866 shutdown_connections 2024-04-26T21:52:11.715 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:11.714+0000 7fb235c00640 1 --2- 172.21.15.72:0/3895899866 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb2301111b0 0x7fb23010b140 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:11.715 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:11.714+0000 7fb235c00640 1 --2- 172.21.15.72:0/3895899866 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fb2140a3c30 0x7fb2140a60f0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:11.715 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:11.714+0000 7fb235c00640 1 --2- 172.21.15.72:0/3895899866 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb23010a290 0x7fb23010ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:11.715 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:11.714+0000 7fb235c00640 1 --2- 172.21.15.72:0/3895899866 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb230109930 0x7fb23010c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:11.715 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:11.714+0000 7fb235c00640 1 -- 172.21.15.72:0/3895899866 >> 172.21.15.72:0/3895899866 conn(0x7fb23006fe00 msgr2=0x7fb230071b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:11.715 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:11.714+0000 7fb235c00640 1 -- 172.21.15.72:0/3895899866 shutdown_connections 2024-04-26T21:52:11.715 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:11.714+0000 7fb235c00640 1 -- 172.21.15.72:0/3895899866 wait complete. 2024-04-26T21:52:12.262 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:11 smithi072 ceph-mon[28138]: from='client.? 172.21.15.72:0/3895899866' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:52:12.262 INFO:journalctl@ceph.mds.d.smithi072.stdout:Apr 26 21:52:12 smithi072 podman[69465]: 2024-04-26 21:52:12.08571629 +0000 UTC m=+0.192959979 container create d54a4c15c6839e891b7fb18622148741d1f880042430b03270006bc1d86a539b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-mds-d, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff) 2024-04-26T21:52:12.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:11 smithi076 ceph-mon[31823]: from='client.? 172.21.15.72:0/3895899866' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:52:12.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:11 smithi164 ceph-mon[31916]: from='client.? 172.21.15.72:0/3895899866' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:52:12.573 INFO:journalctl@ceph.mds.d.smithi072.stdout:Apr 26 21:52:12 smithi072 podman[69465]: 2024-04-26 21:52:12.368195531 +0000 UTC m=+0.475439227 container init d54a4c15c6839e891b7fb18622148741d1f880042430b03270006bc1d86a539b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-mds-d, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-04-26T21:52:12.573 INFO:journalctl@ceph.mds.d.smithi072.stdout:Apr 26 21:52:12 smithi072 podman[69465]: 2024-04-26 21:52:12.378979534 +0000 UTC m=+0.486223233 container start d54a4c15c6839e891b7fb18622148741d1f880042430b03270006bc1d86a539b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-mds-d, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-04-26T21:52:12.573 INFO:journalctl@ceph.mds.d.smithi072.stdout:Apr 26 21:52:12 smithi072 ceph-mds[69484]: -- 172.21.15.72:0/2348197939 <== mon.0 v2:172.21.15.72:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x562267535a40 con 0x562267508c00 2024-04-26T21:52:12.573 INFO:journalctl@ceph.mds.d.smithi072.stdout:Apr 26 21:52:12 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-mds-d[69480]: starting mds.d at 2024-04-26T21:52:12.573 INFO:journalctl@ceph.mds.d.smithi072.stdout:Apr 26 21:52:12 smithi072 bash[69465]: d54a4c15c6839e891b7fb18622148741d1f880042430b03270006bc1d86a539b 2024-04-26T21:52:12.573 INFO:journalctl@ceph.mds.d.smithi072.stdout:Apr 26 21:52:12 smithi072 systemd[1]: Started Ceph mds.d for a1780950-0415-11ef-bc93-c7b262605968. 2024-04-26T21:52:13.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:12 smithi076 ceph-mon[31823]: pgmap v344: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:52:13.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:12 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:52:13.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:12 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:52:13.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:12 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:52:13.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:12 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:52:13.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:12 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:52:13.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:12 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:52:13.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:12 smithi076 ceph-mon[31823]: Deploying daemon mds.e on smithi076 2024-04-26T21:52:13.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:12 smithi072 ceph-mon[28138]: pgmap v344: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:52:13.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:12 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:52:13.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:12 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:52:13.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:12 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:52:13.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:12 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:52:13.323 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:12 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:52:13.323 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:12 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:52:13.323 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:12 smithi072 ceph-mon[28138]: Deploying daemon mds.e on smithi076 2024-04-26T21:52:13.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:12 smithi164 ceph-mon[31916]: pgmap v344: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:52:13.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:12 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:52:13.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:12 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:52:13.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:12 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:52:13.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:12 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:52:13.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:12 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:52:13.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:12 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:52:13.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:12 smithi164 ceph-mon[31916]: Deploying daemon mds.e on smithi076 2024-04-26T21:52:13.735 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:52:13.956 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:13.956+0000 7ffa99600640 1 -- 172.21.15.72:0/1703721908 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ffa94104980 msgr2=0x7ffa94104da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:13.957 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:13.956+0000 7ffa99600640 1 --2- 172.21.15.72:0/1703721908 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ffa94104980 0x7ffa94104da0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ffa84009900 tx=0x7ffa8402a410 comp rx=0 tx=0).stop 2024-04-26T21:52:13.957 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:13.956+0000 7ffa99600640 1 -- 172.21.15.72:0/1703721908 shutdown_connections 2024-04-26T21:52:13.957 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:13.956+0000 7ffa99600640 1 --2- 172.21.15.72:0/1703721908 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffa94106530 0x7ffa9410cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:13.957 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:13.956+0000 7ffa99600640 1 --2- 172.21.15.72:0/1703721908 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffa94105b70 0x7ffa94105ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:13.957 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:13.956+0000 7ffa99600640 1 --2- 172.21.15.72:0/1703721908 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ffa94104980 0x7ffa94104da0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:13.957 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:13.956+0000 7ffa99600640 1 -- 172.21.15.72:0/1703721908 >> 172.21.15.72:0/1703721908 conn(0x7ffa940fff10 msgr2=0x7ffa94102330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:13.957 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:13.956+0000 7ffa99600640 1 -- 172.21.15.72:0/1703721908 shutdown_connections 2024-04-26T21:52:13.957 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:13.956+0000 7ffa99600640 1 -- 172.21.15.72:0/1703721908 wait complete. 2024-04-26T21:52:13.961 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:13.957+0000 7ffa99600640 1 Processor -- start 2024-04-26T21:52:13.961 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:13.957+0000 7ffa99600640 1 -- start start 2024-04-26T21:52:13.961 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:13.957+0000 7ffa99600640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffa94104980 0x7ffa9419aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:13.961 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:13.957+0000 7ffa99600640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ffa94105b70 0x7ffa9419b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:13.961 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:13.958+0000 7ffa99600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffa94106530 0x7ffa941a2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:13.961 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:13.958+0000 7ffa99600640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7ffa941103a0 con 0x7ffa94105b70 2024-04-26T21:52:13.961 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:13.958+0000 7ffa99600640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7ffa94110220 con 0x7ffa94104980 2024-04-26T21:52:13.961 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:13.958+0000 7ffa99600640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ffa94110520 con 0x7ffa94106530 2024-04-26T21:52:13.962 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:13.958+0000 7ffa93400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ffa94105b70 0x7ffa9419b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:13.962 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:13.958+0000 7ffa93400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ffa94105b70 0x7ffa9419b010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:60192/0 (socket says 172.21.15.72:60192) 2024-04-26T21:52:13.962 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:13.958+0000 7ffa93400640 1 -- 172.21.15.72:0/4060884980 learned_addr learned my addr 172.21.15.72:0/4060884980 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:52:13.962 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:13.958+0000 7ffa93e00640 1 --2- 172.21.15.72:0/4060884980 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffa94104980 0x7ffa9419aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:13.962 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:13.958+0000 7ffa98c00640 1 --2- 172.21.15.72:0/4060884980 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffa94106530 0x7ffa941a2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:13.962 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:13.958+0000 7ffa93400640 1 -- 172.21.15.72:0/4060884980 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffa94106530 msgr2=0x7ffa941a2080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:13.962 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:13.958+0000 7ffa93400640 1 --2- 172.21.15.72:0/4060884980 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffa94106530 0x7ffa941a2080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:13.963 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:13.958+0000 7ffa93400640 1 -- 172.21.15.72:0/4060884980 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffa94104980 msgr2=0x7ffa9419aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:13.963 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:13.958+0000 7ffa93400640 1 --2- 172.21.15.72:0/4060884980 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffa94104980 0x7ffa9419aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:13.963 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:13.958+0000 7ffa93400640 1 -- 172.21.15.72:0/4060884980 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa941a26f0 con 0x7ffa94105b70 2024-04-26T21:52:13.963 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:13.959+0000 7ffa93400640 1 --2- 172.21.15.72:0/4060884980 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ffa94105b70 0x7ffa9419b010 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ffa8000d770 tx=0x7ffa8000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:13.963 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:13.959+0000 7ffa90c00640 1 -- 172.21.15.72:0/4060884980 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa80017070 con 0x7ffa94105b70 2024-04-26T21:52:13.963 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:13.959+0000 7ffa90c00640 1 -- 172.21.15.72:0/4060884980 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ffa800054b0 con 0x7ffa94105b70 2024-04-26T21:52:13.963 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:13.959+0000 7ffa99600640 1 -- 172.21.15.72:0/4060884980 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa941a29e0 con 0x7ffa94105b70 2024-04-26T21:52:13.963 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:13.959+0000 7ffa90c00640 1 -- 172.21.15.72:0/4060884980 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa80004c60 con 0x7ffa94105b70 2024-04-26T21:52:13.964 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:13.959+0000 7ffa99600640 1 -- 172.21.15.72:0/4060884980 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa941a2ee0 con 0x7ffa94105b70 2024-04-26T21:52:13.964 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:13.960+0000 7ffa99600640 1 -- 172.21.15.72:0/4060884980 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa9410bd10 con 0x7ffa94105b70 2024-04-26T21:52:13.964 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:13.963+0000 7ffa90c00640 1 -- 172.21.15.72:0/4060884980 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7ffa80004e00 con 0x7ffa94105b70 2024-04-26T21:52:13.965 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:13.965+0000 7ffa90c00640 1 --2- 172.21.15.72:0/4060884980 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7ffa780a3bb0 0x7ffa780a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:13.965 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:13.965+0000 7ffa90c00640 1 -- 172.21.15.72:0/4060884980 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7ffa800cfb10 con 0x7ffa94105b70 2024-04-26T21:52:13.966 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:13.965+0000 7ffa93e00640 1 --2- 172.21.15.72:0/4060884980 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7ffa780a3bb0 0x7ffa780a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:13.966 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:13.966+0000 7ffa93e00640 1 --2- 172.21.15.72:0/4060884980 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7ffa780a3bb0 0x7ffa780a6070 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7ffa840057c0 tx=0x7ffa84000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:13.970 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:13.970+0000 7ffa90c00640 1 -- 172.21.15.72:0/4060884980 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7ffa8009c8f0 con 0x7ffa94105b70 2024-04-26T21:52:14.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:14 smithi072 ceph-mon[28138]: mds.? [v2:172.21.15.72:6836/2142200512,v1:172.21.15.72:6837/2142200512] up:boot 2024-04-26T21:52:14.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:14 smithi072 ceph-mon[28138]: fsmap 4 up:standby 2024-04-26T21:52:14.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:14 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-26T21:52:14.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:14 smithi072 ceph-mon[28138]: pgmap v345: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:52:14.214 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:14.214+0000 7ffa99600640 1 -- 172.21.15.72:0/4060884980 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ffa9402dad0 con 0x7ffa94105b70 2024-04-26T21:52:14.215 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:14.215+0000 7ffa90c00640 1 -- 172.21.15.72:0/4060884980 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3871 (secure 0 0 0) 0x7ffa8009c1f0 con 0x7ffa94105b70 2024-04-26T21:52:14.215 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T21:52:14.215 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6835/1320381959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6834","nonce":1320381959},{"type":"v1","addr":"172.21.15.72:6835","nonce":1320381959}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6837/2142200512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6836","nonce":2142200512},{"type":"v1","addr":"172.21.15.72:6837","nonce":2142200512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24388,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6833/4236668465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":4236668465},{"type":"v1","addr":"172.21.15.164:6833","nonce":4236668465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":34406,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6833/3869483750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":3869483750},{"type":"v1","addr":"172.21.15.76:6833","nonce":3869483750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-26T21:52:14.215 INFO:teuthology.orchestra.run.smithi072.stderr:dumped fsmap epoch 5 2024-04-26T21:52:14.217 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:14.217+0000 7ffa99600640 1 -- 172.21.15.72:0/4060884980 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7ffa780a3bb0 msgr2=0x7ffa780a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:14.218 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:14.217+0000 7ffa99600640 1 --2- 172.21.15.72:0/4060884980 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7ffa780a3bb0 0x7ffa780a6070 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7ffa840057c0 tx=0x7ffa84000bc0 comp rx=0 tx=0).stop 2024-04-26T21:52:14.218 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:14.217+0000 7ffa99600640 1 -- 172.21.15.72:0/4060884980 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ffa94105b70 msgr2=0x7ffa9419b010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:14.218 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:14.217+0000 7ffa99600640 1 --2- 172.21.15.72:0/4060884980 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ffa94105b70 0x7ffa9419b010 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ffa8000d770 tx=0x7ffa8000dc30 comp rx=0 tx=0).stop 2024-04-26T21:52:14.218 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:14.217+0000 7ffa99600640 1 -- 172.21.15.72:0/4060884980 shutdown_connections 2024-04-26T21:52:14.218 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:14.217+0000 7ffa99600640 1 --2- 172.21.15.72:0/4060884980 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7ffa780a3bb0 0x7ffa780a6070 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:14.218 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:14.217+0000 7ffa99600640 1 --2- 172.21.15.72:0/4060884980 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffa94106530 0x7ffa941a2080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:14.218 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:14.217+0000 7ffa99600640 1 --2- 172.21.15.72:0/4060884980 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ffa94105b70 0x7ffa9419b010 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:14.218 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:14.217+0000 7ffa99600640 1 --2- 172.21.15.72:0/4060884980 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffa94104980 0x7ffa9419aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:14.218 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:14.217+0000 7ffa99600640 1 -- 172.21.15.72:0/4060884980 >> 172.21.15.72:0/4060884980 conn(0x7ffa940fff10 msgr2=0x7ffa94108ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:14.218 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:14.217+0000 7ffa99600640 1 -- 172.21.15.72:0/4060884980 shutdown_connections 2024-04-26T21:52:14.218 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:14.217+0000 7ffa99600640 1 -- 172.21.15.72:0/4060884980 wait complete. 2024-04-26T21:52:14.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:14 smithi076 ceph-mon[31823]: mds.? [v2:172.21.15.72:6836/2142200512,v1:172.21.15.72:6837/2142200512] up:boot 2024-04-26T21:52:14.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:14 smithi076 ceph-mon[31823]: fsmap 4 up:standby 2024-04-26T21:52:14.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:14 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-26T21:52:14.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:14 smithi076 ceph-mon[31823]: pgmap v345: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:52:14.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:14 smithi164 ceph-mon[31916]: mds.? [v2:172.21.15.72:6836/2142200512,v1:172.21.15.72:6837/2142200512] up:boot 2024-04-26T21:52:14.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:14 smithi164 ceph-mon[31916]: fsmap 4 up:standby 2024-04-26T21:52:14.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:14 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-26T21:52:14.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:14 smithi164 ceph-mon[31916]: pgmap v345: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:52:15.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:15 smithi076 ceph-mon[31823]: from='client.? 172.21.15.72:0/4060884980' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:52:15.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:15 smithi072 ceph-mon[28138]: from='client.? 172.21.15.72:0/4060884980' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:52:15.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:15 smithi164 ceph-mon[31916]: from='client.? 172.21.15.72:0/4060884980' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:52:16.031 INFO:journalctl@ceph.mds.e.smithi076.stdout:Apr 26 21:52:15 smithi076 systemd[1]: Starting Ceph mds.e for a1780950-0415-11ef-bc93-c7b262605968... 2024-04-26T21:52:16.239 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:52:16.292 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:16 smithi076 ceph-mon[31823]: pgmap v346: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:52:16.292 INFO:journalctl@ceph.mds.e.smithi076.stdout:Apr 26 21:52:16 smithi076 podman[60614]: 2024-04-26 21:52:16.045972168 +0000 UTC m=+0.038113120 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-26T21:52:16.293 INFO:journalctl@ceph.mds.e.smithi076.stdout:Apr 26 21:52:16 smithi076 podman[60614]: 2024-04-26 21:52:16.144832655 +0000 UTC m=+0.136973641 container create 5c97ebe3d413c16b14c3e0412f42bd87db1427dc328470a7dcd9281bce9c06f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-mds-e, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, io.buildah.version=1.33.5, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-04-26T21:52:16.315 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:16 smithi072 ceph-mon[28138]: pgmap v346: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:52:16.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:16 smithi164 ceph-mon[31916]: pgmap v346: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:52:16.450 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:16.449+0000 7ff303e00640 1 -- 172.21.15.72:0/330833980 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff304108d90 msgr2=0x7ff3041091b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:16.450 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:16.449+0000 7ff303e00640 1 --2- 172.21.15.72:0/330833980 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff304108d90 0x7ff3041091b0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7ff2f4009900 tx=0x7ff2f402a3f0 comp rx=0 tx=0).stop 2024-04-26T21:52:16.450 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:16.449+0000 7ff303e00640 1 -- 172.21.15.72:0/330833980 shutdown_connections 2024-04-26T21:52:16.450 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:16.449+0000 7ff303e00640 1 --2- 172.21.15.72:0/330833980 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff30410a940 0x7ff3041111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:16.450 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:16.449+0000 7ff303e00640 1 --2- 172.21.15.72:0/330833980 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff304109f80 0x7ff30410a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:16.450 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:16.449+0000 7ff303e00640 1 --2- 172.21.15.72:0/330833980 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff304108d90 0x7ff3041091b0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:16.450 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:16.449+0000 7ff303e00640 1 -- 172.21.15.72:0/330833980 >> 172.21.15.72:0/330833980 conn(0x7ff304077d30 msgr2=0x7ff30407a170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:16.451 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:16.450+0000 7ff303e00640 1 -- 172.21.15.72:0/330833980 shutdown_connections 2024-04-26T21:52:16.451 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:16.450+0000 7ff303e00640 1 -- 172.21.15.72:0/330833980 wait complete. 2024-04-26T21:52:16.453 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:16.451+0000 7ff303e00640 1 Processor -- start 2024-04-26T21:52:16.453 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:16.451+0000 7ff303e00640 1 -- start start 2024-04-26T21:52:16.453 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:16.451+0000 7ff303e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff304108d90 0x7ff30419ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:16.453 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:16.451+0000 7ff303e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff304109f80 0x7ff30419f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:16.453 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:16.451+0000 7ff303e00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff30410a940 0x7ff3041a6510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:16.454 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:16.451+0000 7ff303e00640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7ff304114810 con 0x7ff304108d90 2024-04-26T21:52:16.454 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:16.451+0000 7ff303e00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7ff304114690 con 0x7ff30410a940 2024-04-26T21:52:16.454 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:16.451+0000 7ff303e00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ff304114990 con 0x7ff304109f80 2024-04-26T21:52:16.454 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:16.451+0000 7ff302a00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff304108d90 0x7ff30419ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:16.454 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:16.451+0000 7ff302a00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff304108d90 0x7ff30419ef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:60212/0 (socket says 172.21.15.72:60212) 2024-04-26T21:52:16.454 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:16.451+0000 7ff302a00640 1 -- 172.21.15.72:0/3162149641 learned_addr learned my addr 172.21.15.72:0/3162149641 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:52:16.454 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:16.451+0000 7ff303400640 1 --2- 172.21.15.72:0/3162149641 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff30410a940 0x7ff3041a6510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:16.454 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:16.452+0000 7ff302a00640 1 -- 172.21.15.72:0/3162149641 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff304109f80 msgr2=0x7ff30419f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:16.454 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:16.452+0000 7ff302000640 1 --2- 172.21.15.72:0/3162149641 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff304109f80 0x7ff30419f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:16.454 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:16.452+0000 7ff302a00640 1 --2- 172.21.15.72:0/3162149641 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff304109f80 0x7ff30419f4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:16.454 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:16.452+0000 7ff302a00640 1 -- 172.21.15.72:0/3162149641 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff30410a940 msgr2=0x7ff3041a6510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:16.455 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:16.452+0000 7ff302a00640 1 --2- 172.21.15.72:0/3162149641 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff30410a940 0x7ff3041a6510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:16.455 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:16.452+0000 7ff302a00640 1 -- 172.21.15.72:0/3162149641 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3041a6b80 con 0x7ff304108d90 2024-04-26T21:52:16.455 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:16.452+0000 7ff302000640 1 --2- 172.21.15.72:0/3162149641 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff304109f80 0x7ff30419f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:52:16.455 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:16.452+0000 7ff302a00640 1 --2- 172.21.15.72:0/3162149641 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff304108d90 0x7ff30419ef60 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7ff2f4031460 tx=0x7ff2f4003d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:16.455 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:16.452+0000 7ff308a00640 1 -- 172.21.15.72:0/3162149641 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2f403f070 con 0x7ff304108d90 2024-04-26T21:52:16.455 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:16.452+0000 7ff308a00640 1 -- 172.21.15.72:0/3162149641 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff2f4031da0 con 0x7ff304108d90 2024-04-26T21:52:16.455 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:16.452+0000 7ff303e00640 1 -- 172.21.15.72:0/3162149641 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3041a6e10 con 0x7ff304108d90 2024-04-26T21:52:16.455 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:16.452+0000 7ff308a00640 1 -- 172.21.15.72:0/3162149641 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2f4035030 con 0x7ff304108d90 2024-04-26T21:52:16.455 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:16.453+0000 7ff303e00640 1 -- 172.21.15.72:0/3162149641 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3041a72a0 con 0x7ff304108d90 2024-04-26T21:52:16.456 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:16.454+0000 7ff303e00640 1 -- 172.21.15.72:0/3162149641 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3041101d0 con 0x7ff304108d90 2024-04-26T21:52:16.456 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:16.456+0000 7ff308a00640 1 -- 172.21.15.72:0/3162149641 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7ff2f4042070 con 0x7ff304108d90 2024-04-26T21:52:16.457 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:16.457+0000 7ff308a00640 1 --2- 172.21.15.72:0/3162149641 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7ff2e80a3bb0 0x7ff2e80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:16.457 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:16.457+0000 7ff308a00640 1 -- 172.21.15.72:0/3162149641 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7ff2f40f0a50 con 0x7ff304108d90 2024-04-26T21:52:16.457 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:16.457+0000 7ff302000640 1 --2- 172.21.15.72:0/3162149641 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7ff2e80a3bb0 0x7ff2e80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:16.458 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:16.458+0000 7ff302000640 1 --2- 172.21.15.72:0/3162149641 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7ff2e80a3bb0 0x7ff2e80a6070 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7ff3041a0470 tx=0x7ff2f0005d30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:16.461 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:16.461+0000 7ff308a00640 1 -- 172.21.15.72:0/3162149641 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7ff2f40bd830 con 0x7ff304108d90 2024-04-26T21:52:16.705 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:16.705+0000 7ff303e00640 1 -- 172.21.15.72:0/3162149641 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff304062d70 con 0x7ff304108d90 2024-04-26T21:52:16.706 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:16.706+0000 7ff308a00640 1 -- 172.21.15.72:0/3162149641 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3871 (secure 0 0 0) 0x7ff2f40bd130 con 0x7ff304108d90 2024-04-26T21:52:16.707 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T21:52:16.707 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6835/1320381959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6834","nonce":1320381959},{"type":"v1","addr":"172.21.15.72:6835","nonce":1320381959}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6837/2142200512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6836","nonce":2142200512},{"type":"v1","addr":"172.21.15.72:6837","nonce":2142200512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24388,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6833/4236668465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":4236668465},{"type":"v1","addr":"172.21.15.164:6833","nonce":4236668465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":34406,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6833/3869483750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":3869483750},{"type":"v1","addr":"172.21.15.76:6833","nonce":3869483750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-26T21:52:16.707 INFO:teuthology.orchestra.run.smithi072.stderr:dumped fsmap epoch 5 2024-04-26T21:52:16.709 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:16.709+0000 7ff303e00640 1 -- 172.21.15.72:0/3162149641 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7ff2e80a3bb0 msgr2=0x7ff2e80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:16.709 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:16.709+0000 7ff303e00640 1 --2- 172.21.15.72:0/3162149641 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7ff2e80a3bb0 0x7ff2e80a6070 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7ff3041a0470 tx=0x7ff2f0005d30 comp rx=0 tx=0).stop 2024-04-26T21:52:16.710 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:16.709+0000 7ff303e00640 1 -- 172.21.15.72:0/3162149641 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff304108d90 msgr2=0x7ff30419ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:16.710 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:16.709+0000 7ff303e00640 1 --2- 172.21.15.72:0/3162149641 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff304108d90 0x7ff30419ef60 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7ff2f4031460 tx=0x7ff2f4003d50 comp rx=0 tx=0).stop 2024-04-26T21:52:16.710 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:16.709+0000 7ff303e00640 1 -- 172.21.15.72:0/3162149641 shutdown_connections 2024-04-26T21:52:16.710 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:16.709+0000 7ff303e00640 1 --2- 172.21.15.72:0/3162149641 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff30410a940 0x7ff3041a6510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:16.710 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:16.709+0000 7ff303e00640 1 --2- 172.21.15.72:0/3162149641 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7ff2e80a3bb0 0x7ff2e80a6070 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:16.710 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:16.709+0000 7ff303e00640 1 --2- 172.21.15.72:0/3162149641 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff304109f80 0x7ff30419f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:16.710 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:16.709+0000 7ff303e00640 1 --2- 172.21.15.72:0/3162149641 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff304108d90 0x7ff30419ef60 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:16.710 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:16.709+0000 7ff303e00640 1 -- 172.21.15.72:0/3162149641 >> 172.21.15.72:0/3162149641 conn(0x7ff304077d30 msgr2=0x7ff30410d3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:16.710 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:16.709+0000 7ff303e00640 1 -- 172.21.15.72:0/3162149641 shutdown_connections 2024-04-26T21:52:16.710 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:16.709+0000 7ff303e00640 1 -- 172.21.15.72:0/3162149641 wait complete. 2024-04-26T21:52:16.815 INFO:journalctl@ceph.mds.e.smithi076.stdout:Apr 26 21:52:16 smithi076 podman[60614]: 2024-04-26 21:52:16.440512352 +0000 UTC m=+0.432653342 container init 5c97ebe3d413c16b14c3e0412f42bd87db1427dc328470a7dcd9281bce9c06f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-mds-e, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T21:52:16.815 INFO:journalctl@ceph.mds.e.smithi076.stdout:Apr 26 21:52:16 smithi076 podman[60614]: 2024-04-26 21:52:16.450650684 +0000 UTC m=+0.442791637 container start 5c97ebe3d413c16b14c3e0412f42bd87db1427dc328470a7dcd9281bce9c06f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-mds-e, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T21:52:16.815 INFO:journalctl@ceph.mds.e.smithi076.stdout:Apr 26 21:52:16 smithi076 ceph-mds[60633]: -- 172.21.15.76:0/2241162771 <== mon.2 v2:172.21.15.76:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d5526b9a40 con 0x55d55268cc00 2024-04-26T21:52:16.815 INFO:journalctl@ceph.mds.e.smithi076.stdout:Apr 26 21:52:16 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-mds-e[60629]: starting mds.e at 2024-04-26T21:52:16.815 INFO:journalctl@ceph.mds.e.smithi076.stdout:Apr 26 21:52:16 smithi076 bash[60614]: 5c97ebe3d413c16b14c3e0412f42bd87db1427dc328470a7dcd9281bce9c06f9 2024-04-26T21:52:16.815 INFO:journalctl@ceph.mds.e.smithi076.stdout:Apr 26 21:52:16 smithi076 systemd[1]: Started Ceph mds.e for a1780950-0415-11ef-bc93-c7b262605968. 2024-04-26T21:52:17.941 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:17 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:52:17.941 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:17 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:52:17.941 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:17 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:52:17.941 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:17 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:52:17.941 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:17 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:52:17.941 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:17 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:52:17.941 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:17 smithi164 ceph-mon[31916]: Deploying daemon mds.f on smithi164 2024-04-26T21:52:17.941 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:17 smithi164 ceph-mon[31916]: from='client.? 172.21.15.72:0/3162149641' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:52:17.941 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:17 smithi164 ceph-mon[31916]: mds.? [v2:172.21.15.76:6834/3517429435,v1:172.21.15.76:6835/3517429435] up:boot 2024-04-26T21:52:17.941 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:17 smithi164 ceph-mon[31916]: fsmap 5 up:standby 2024-04-26T21:52:17.942 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:17 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-26T21:52:18.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:17 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:52:18.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:17 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:52:18.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:17 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:52:18.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:17 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:52:18.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:17 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:52:18.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:17 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:52:18.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:17 smithi076 ceph-mon[31823]: Deploying daemon mds.f on smithi164 2024-04-26T21:52:18.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:17 smithi076 ceph-mon[31823]: from='client.? 172.21.15.72:0/3162149641' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:52:18.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:17 smithi076 ceph-mon[31823]: mds.? [v2:172.21.15.76:6834/3517429435,v1:172.21.15.76:6835/3517429435] up:boot 2024-04-26T21:52:18.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:17 smithi076 ceph-mon[31823]: fsmap 5 up:standby 2024-04-26T21:52:18.066 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:17 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-26T21:52:18.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:17 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:52:18.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:17 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:52:18.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:17 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:52:18.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:17 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:52:18.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:17 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:52:18.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:17 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:52:18.073 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:17 smithi072 ceph-mon[28138]: Deploying daemon mds.f on smithi164 2024-04-26T21:52:18.073 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:17 smithi072 ceph-mon[28138]: from='client.? 172.21.15.72:0/3162149641' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:52:18.073 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:17 smithi072 ceph-mon[28138]: mds.? [v2:172.21.15.76:6834/3517429435,v1:172.21.15.76:6835/3517429435] up:boot 2024-04-26T21:52:18.073 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:17 smithi072 ceph-mon[28138]: fsmap 5 up:standby 2024-04-26T21:52:18.073 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:17 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-26T21:52:18.728 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:52:18.878 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:18 smithi164 ceph-mon[31916]: pgmap v347: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:52:18.938 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:18.937+0000 7f5e3f200640 1 -- 172.21.15.72:0/3245101821 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5e401027b0 msgr2=0x7f5e40102bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:18.938 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:18.937+0000 7f5e3f200640 1 --2- 172.21.15.72:0/3245101821 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5e401027b0 0x7f5e40102bd0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f5e30009ca0 tx=0x7f5e3002a8c0 comp rx=0 tx=0).stop 2024-04-26T21:52:18.938 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:18.937+0000 7f5e3f200640 1 -- 172.21.15.72:0/3245101821 shutdown_connections 2024-04-26T21:52:18.938 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:18.937+0000 7f5e3f200640 1 --2- 172.21.15.72:0/3245101821 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5e40104360 0x7f5e4010abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:18.938 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:18.937+0000 7f5e3f200640 1 --2- 172.21.15.72:0/3245101821 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5e401039a0 0x7f5e40103e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:18.938 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:18.937+0000 7f5e3f200640 1 --2- 172.21.15.72:0/3245101821 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5e401027b0 0x7f5e40102bd0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:18.938 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:18.937+0000 7f5e3f200640 1 -- 172.21.15.72:0/3245101821 >> 172.21.15.72:0/3245101821 conn(0x7f5e400fdd40 msgr2=0x7f5e40100160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:18.938 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:18.937+0000 7f5e3f200640 1 -- 172.21.15.72:0/3245101821 shutdown_connections 2024-04-26T21:52:18.938 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:18.937+0000 7f5e3f200640 1 -- 172.21.15.72:0/3245101821 wait complete. 2024-04-26T21:52:18.941 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:18.938+0000 7f5e3f200640 1 Processor -- start 2024-04-26T21:52:18.941 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:18.938+0000 7f5e3f200640 1 -- start start 2024-04-26T21:52:18.941 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:18.938+0000 7f5e3f200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5e401027b0 0x7f5e4019aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:18.941 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:18.938+0000 7f5e3f200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5e401039a0 0x7f5e4019b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:18.941 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:18.938+0000 7f5e3f200640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5e40104360 0x7f5e401a2150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:18.941 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:18.938+0000 7f5e3f200640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f5e4010e2b0 con 0x7f5e40104360 2024-04-26T21:52:18.941 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:18.938+0000 7f5e3f200640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f5e4010e130 con 0x7f5e401027b0 2024-04-26T21:52:18.941 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:18.938+0000 7f5e3f200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f5e4010e430 con 0x7f5e401039a0 2024-04-26T21:52:18.941 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:18.939+0000 7f5e3e800640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5e40104360 0x7f5e401a2150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:18.941 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:18.939+0000 7f5e3e800640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5e40104360 0x7f5e401a2150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:60222/0 (socket says 172.21.15.72:60222) 2024-04-26T21:52:18.941 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:18.939+0000 7f5e3e800640 1 -- 172.21.15.72:0/3079318749 learned_addr learned my addr 172.21.15.72:0/3079318749 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:52:18.941 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:18.939+0000 7f5e35400640 1 --2- 172.21.15.72:0/3079318749 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5e401039a0 0x7f5e4019b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:18.942 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:18.939+0000 7f5e3de00640 1 --2- 172.21.15.72:0/3079318749 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5e401027b0 0x7f5e4019aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:18.942 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:18.939+0000 7f5e3e800640 1 -- 172.21.15.72:0/3079318749 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5e401039a0 msgr2=0x7f5e4019b0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:18.942 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:18.939+0000 7f5e3e800640 1 --2- 172.21.15.72:0/3079318749 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5e401039a0 0x7f5e4019b0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:18.942 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:18.939+0000 7f5e3e800640 1 -- 172.21.15.72:0/3079318749 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5e401027b0 msgr2=0x7f5e4019aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:18.942 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:18.939+0000 7f5e3e800640 1 --2- 172.21.15.72:0/3079318749 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5e401027b0 0x7f5e4019aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:18.942 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:18.939+0000 7f5e3e800640 1 -- 172.21.15.72:0/3079318749 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e401a27c0 con 0x7f5e40104360 2024-04-26T21:52:18.942 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:18.939+0000 7f5e3de00640 1 --2- 172.21.15.72:0/3079318749 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5e401027b0 0x7f5e4019aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:52:18.942 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:18.939+0000 7f5e3e800640 1 --2- 172.21.15.72:0/3079318749 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5e40104360 0x7f5e401a2150 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f5e38009fd0 tx=0x7f5e3800b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:18.942 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:18.940+0000 7f5e37400640 1 -- 172.21.15.72:0/3079318749 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e38011830 con 0x7f5e40104360 2024-04-26T21:52:18.942 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:18.940+0000 7f5e37400640 1 -- 172.21.15.72:0/3079318749 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5e38016070 con 0x7f5e40104360 2024-04-26T21:52:18.942 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:18.940+0000 7f5e3f200640 1 -- 172.21.15.72:0/3079318749 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e401a2ab0 con 0x7f5e40104360 2024-04-26T21:52:18.942 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:18.940+0000 7f5e37400640 1 -- 172.21.15.72:0/3079318749 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e38010d20 con 0x7f5e40104360 2024-04-26T21:52:18.942 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:18.940+0000 7f5e3f200640 1 -- 172.21.15.72:0/3079318749 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e401a3030 con 0x7f5e40104360 2024-04-26T21:52:18.943 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:18.941+0000 7f5e3f200640 1 -- 172.21.15.72:0/3079318749 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e401a32c0 con 0x7f5e40104360 2024-04-26T21:52:18.943 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:18.943+0000 7f5e37400640 1 -- 172.21.15.72:0/3079318749 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f5e3801b460 con 0x7f5e40104360 2024-04-26T21:52:18.945 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:18.944+0000 7f5e37400640 1 --2- 172.21.15.72:0/3079318749 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f5e240a3bb0 0x7f5e240a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:18.945 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:18.944+0000 7f5e37400640 1 -- 172.21.15.72:0/3079318749 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7f5e380cf720 con 0x7f5e40104360 2024-04-26T21:52:18.945 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:18.945+0000 7f5e3de00640 1 --2- 172.21.15.72:0/3079318749 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f5e240a3bb0 0x7f5e240a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:18.945 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:18.945+0000 7f5e3de00640 1 --2- 172.21.15.72:0/3079318749 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f5e240a3bb0 0x7f5e240a6070 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f5e30009ca0 tx=0x7f5e30000f40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:18.949 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:18.949+0000 7f5e37400640 1 -- 172.21.15.72:0/3079318749 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f5e38021250 con 0x7f5e40104360 2024-04-26T21:52:19.064 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:18 smithi076 ceph-mon[31823]: pgmap v347: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:52:19.071 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:18 smithi072 ceph-mon[28138]: pgmap v347: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:52:19.189 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:19.188+0000 7f5e3f200640 1 -- 172.21.15.72:0/3079318749 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5e401a3530 con 0x7f5e40104360 2024-04-26T21:52:19.190 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:19.189+0000 7f5e37400640 1 -- 172.21.15.72:0/3079318749 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4696 (secure 0 0 0) 0x7f5e3809c500 con 0x7f5e40104360 2024-04-26T21:52:19.190 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T21:52:19.190 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6835/1320381959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6834","nonce":1320381959},{"type":"v1","addr":"172.21.15.72:6835","nonce":1320381959}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6837/2142200512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6836","nonce":2142200512},{"type":"v1","addr":"172.21.15.72:6837","nonce":2142200512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24388,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6833/4236668465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":4236668465},{"type":"v1","addr":"172.21.15.164:6833","nonce":4236668465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":34406,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6833/3869483750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":3869483750},{"type":"v1","addr":"172.21.15.76:6833","nonce":3869483750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34442,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/3517429435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":3517429435},{"type":"v1","addr":"172.21.15.76:6835","nonce":3517429435}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-04-26T21:52:19.190 INFO:teuthology.orchestra.run.smithi072.stderr:dumped fsmap epoch 6 2024-04-26T21:52:19.193 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:19.192+0000 7f5e3f200640 1 -- 172.21.15.72:0/3079318749 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f5e240a3bb0 msgr2=0x7f5e240a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:19.193 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:19.192+0000 7f5e3f200640 1 --2- 172.21.15.72:0/3079318749 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f5e240a3bb0 0x7f5e240a6070 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f5e30009ca0 tx=0x7f5e30000f40 comp rx=0 tx=0).stop 2024-04-26T21:52:19.193 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:19.192+0000 7f5e3f200640 1 -- 172.21.15.72:0/3079318749 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5e40104360 msgr2=0x7f5e401a2150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:19.193 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:19.192+0000 7f5e3f200640 1 --2- 172.21.15.72:0/3079318749 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5e40104360 0x7f5e401a2150 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f5e38009fd0 tx=0x7f5e3800b8e0 comp rx=0 tx=0).stop 2024-04-26T21:52:19.193 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:19.192+0000 7f5e3f200640 1 -- 172.21.15.72:0/3079318749 shutdown_connections 2024-04-26T21:52:19.193 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:19.192+0000 7f5e3f200640 1 --2- 172.21.15.72:0/3079318749 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5e40104360 0x7f5e401a2150 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:19.193 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:19.192+0000 7f5e3f200640 1 --2- 172.21.15.72:0/3079318749 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f5e240a3bb0 0x7f5e240a6070 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:19.193 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:19.192+0000 7f5e3f200640 1 --2- 172.21.15.72:0/3079318749 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5e401039a0 0x7f5e4019b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:19.193 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:19.192+0000 7f5e3f200640 1 --2- 172.21.15.72:0/3079318749 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5e401027b0 0x7f5e4019aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:19.193 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:19.192+0000 7f5e3f200640 1 -- 172.21.15.72:0/3079318749 >> 172.21.15.72:0/3079318749 conn(0x7f5e400fdd40 msgr2=0x7f5e40074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:19.193 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:19.192+0000 7f5e3f200640 1 -- 172.21.15.72:0/3079318749 shutdown_connections 2024-04-26T21:52:19.193 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:19.193+0000 7f5e3f200640 1 -- 172.21.15.72:0/3079318749 wait complete. 2024-04-26T21:52:19.691 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:19 smithi164 ceph-mon[31916]: from='client.? 172.21.15.72:0/3079318749' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:52:20.064 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:19 smithi076 ceph-mon[31823]: from='client.? 172.21.15.72:0/3079318749' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:52:20.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:19 smithi072 ceph-mon[28138]: from='client.? 172.21.15.72:0/3079318749' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:52:20.095 INFO:journalctl@ceph.mds.f.smithi164.stdout:Apr 26 21:52:19 smithi164 systemd[1]: Starting Ceph mds.f for a1780950-0415-11ef-bc93-c7b262605968... 2024-04-26T21:52:20.401 INFO:journalctl@ceph.mds.f.smithi164.stdout:Apr 26 21:52:20 smithi164 podman[62034]: 2024-04-26 21:52:20.001097456 +0000 UTC m=+0.036787996 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-26T21:52:20.401 INFO:journalctl@ceph.mds.f.smithi164.stdout:Apr 26 21:52:20 smithi164 podman[62034]: 2024-04-26 21:52:20.129583158 +0000 UTC m=+0.165273735 container create 3bb4b5de753c8267a2dbb8e1da0e0900591c523a288b966f9b9e160414c7c944 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-mds-f, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-04-26T21:52:20.658 INFO:journalctl@ceph.mds.f.smithi164.stdout:Apr 26 21:52:20 smithi164 podman[62034]: 2024-04-26 21:52:20.401278264 +0000 UTC m=+0.436968819 container init 3bb4b5de753c8267a2dbb8e1da0e0900591c523a288b966f9b9e160414c7c944 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-mds-f, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-04-26T21:52:20.658 INFO:journalctl@ceph.mds.f.smithi164.stdout:Apr 26 21:52:20 smithi164 podman[62034]: 2024-04-26 21:52:20.409524517 +0000 UTC m=+0.445215056 container start 3bb4b5de753c8267a2dbb8e1da0e0900591c523a288b966f9b9e160414c7c944 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-mds-f, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-04-26T21:52:20.658 INFO:journalctl@ceph.mds.f.smithi164.stdout:Apr 26 21:52:20 smithi164 ceph-mds[62054]: -- 172.21.15.164:0/1530829786 <== mon.1 v2:172.21.15.164:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55fc8e3bba40 con 0x55fc8e38e800 2024-04-26T21:52:20.658 INFO:journalctl@ceph.mds.f.smithi164.stdout:Apr 26 21:52:20 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-mds-f[62048]: starting mds.f at 2024-04-26T21:52:20.658 INFO:journalctl@ceph.mds.f.smithi164.stdout:Apr 26 21:52:20 smithi164 bash[62034]: 3bb4b5de753c8267a2dbb8e1da0e0900591c523a288b966f9b9e160414c7c944 2024-04-26T21:52:20.658 INFO:journalctl@ceph.mds.f.smithi164.stdout:Apr 26 21:52:20 smithi164 systemd[1]: Started Ceph mds.f for a1780950-0415-11ef-bc93-c7b262605968. 2024-04-26T21:52:20.940 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:20 smithi164 ceph-mon[31916]: pgmap v348: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:52:21.064 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:20 smithi076 ceph-mon[31823]: pgmap v348: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:52:21.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:20 smithi072 ceph-mon[28138]: pgmap v348: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:52:21.213 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:52:21.423 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:21.422+0000 7f902f200640 1 -- 172.21.15.72:0/2006695657 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9028078ed0 msgr2=0x7f9028079770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:21.423 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:21.422+0000 7f902f200640 1 --2- 172.21.15.72:0/2006695657 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9028078ed0 0x7f9028079770 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f9020009be0 tx=0x7f902002a6c0 comp rx=0 tx=0).stop 2024-04-26T21:52:21.423 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:21.423+0000 7f902f200640 1 -- 172.21.15.72:0/2006695657 shutdown_connections 2024-04-26T21:52:21.423 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:21.423+0000 7f902f200640 1 --2- 172.21.15.72:0/2006695657 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9028078ed0 0x7f9028079770 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:21.423 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:21.423+0000 7f902f200640 1 --2- 172.21.15.72:0/2006695657 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9028078530 0x7f9028078990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:21.424 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:21.423+0000 7f902f200640 1 --2- 172.21.15.72:0/2006695657 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f90280772f0 0x7f9028077710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:21.424 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:21.423+0000 7f902f200640 1 -- 172.21.15.72:0/2006695657 >> 172.21.15.72:0/2006695657 conn(0x7f9028100120 msgr2=0x7f9028102540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:21.424 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:21.423+0000 7f902f200640 1 -- 172.21.15.72:0/2006695657 shutdown_connections 2024-04-26T21:52:21.424 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:21.423+0000 7f902f200640 1 -- 172.21.15.72:0/2006695657 wait complete. 2024-04-26T21:52:21.426 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:21.424+0000 7f902f200640 1 Processor -- start 2024-04-26T21:52:21.426 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:21.424+0000 7f902f200640 1 -- start start 2024-04-26T21:52:21.426 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:21.424+0000 7f902f200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f90280772f0 0x7f902819f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:21.426 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:21.424+0000 7f902f200640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9028078530 0x7f902819f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:21.427 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:21.424+0000 7f902f200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9028078ed0 0x7f90281a6620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:21.427 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:21.424+0000 7f902f200640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f9028114980 con 0x7f9028078530 2024-04-26T21:52:21.427 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:21.424+0000 7f902f200640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f9028114800 con 0x7f90280772f0 2024-04-26T21:52:21.427 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:21.424+0000 7f902f200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f9028114b00 con 0x7f9028078ed0 2024-04-26T21:52:21.427 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:21.424+0000 7f9027400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9028078530 0x7f902819f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:21.427 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:21.424+0000 7f9027400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9028078530 0x7f902819f5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:59408/0 (socket says 172.21.15.72:59408) 2024-04-26T21:52:21.427 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:21.424+0000 7f902d200640 1 --2- 172.21.15.72:0/3254252343 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9028078ed0 0x7f90281a6620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:21.427 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:21.424+0000 7f9027400640 1 -- 172.21.15.72:0/3254252343 learned_addr learned my addr 172.21.15.72:0/3254252343 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:52:21.427 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:21.425+0000 7f9027e00640 1 --2- 172.21.15.72:0/3254252343 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f90280772f0 0x7f902819f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:21.427 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:21.425+0000 7f9027400640 1 -- 172.21.15.72:0/3254252343 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9028078ed0 msgr2=0x7f90281a6620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:21.427 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:21.425+0000 7f9027400640 1 --2- 172.21.15.72:0/3254252343 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9028078ed0 0x7f90281a6620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:21.427 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:21.425+0000 7f9027400640 1 -- 172.21.15.72:0/3254252343 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f90280772f0 msgr2=0x7f902819f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:21.427 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:21.425+0000 7f9027400640 1 --2- 172.21.15.72:0/3254252343 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f90280772f0 0x7f902819f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:21.428 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:21.425+0000 7f9027400640 1 -- 172.21.15.72:0/3254252343 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90281a6c90 con 0x7f9028078530 2024-04-26T21:52:21.428 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:21.425+0000 7f902d200640 1 --2- 172.21.15.72:0/3254252343 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9028078ed0 0x7f90281a6620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:52:21.428 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:21.425+0000 7f9027e00640 1 --2- 172.21.15.72:0/3254252343 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f90280772f0 0x7f902819f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:52:21.428 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:21.425+0000 7f9027400640 1 --2- 172.21.15.72:0/3254252343 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9028078530 0x7f902819f5b0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f901400d950 tx=0x7f901400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:21.428 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:21.425+0000 7f9024c00640 1 -- 172.21.15.72:0/3254252343 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9014017070 con 0x7f9028078530 2024-04-26T21:52:21.428 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:21.425+0000 7f9024c00640 1 -- 172.21.15.72:0/3254252343 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9014009c30 con 0x7f9028078530 2024-04-26T21:52:21.428 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:21.425+0000 7f902f200640 1 -- 172.21.15.72:0/3254252343 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90281a6f80 con 0x7f9028078530 2024-04-26T21:52:21.428 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:21.425+0000 7f9024c00640 1 -- 172.21.15.72:0/3254252343 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f901400f620 con 0x7f9028078530 2024-04-26T21:52:21.428 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:21.425+0000 7f902f200640 1 -- 172.21.15.72:0/3254252343 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90281a7440 con 0x7f9028078530 2024-04-26T21:52:21.428 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:21.427+0000 7f902f200640 1 -- 172.21.15.72:0/3254252343 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9028111780 con 0x7f9028078530 2024-04-26T21:52:21.429 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:21.429+0000 7f9024c00640 1 -- 172.21.15.72:0/3254252343 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f9014010460 con 0x7f9028078530 2024-04-26T21:52:21.431 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:21.431+0000 7f9024c00640 1 --2- 172.21.15.72:0/3254252343 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f900c0a3bb0 0x7f900c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:21.431 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:21.431+0000 7f9024c00640 1 -- 172.21.15.72:0/3254252343 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7f90140cfc50 con 0x7f9028078530 2024-04-26T21:52:21.431 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:21.431+0000 7f9027e00640 1 --2- 172.21.15.72:0/3254252343 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f900c0a3bb0 0x7f900c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:21.431 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:21.432+0000 7f9027e00640 1 --2- 172.21.15.72:0/3254252343 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f900c0a3bb0 0x7f900c0a6070 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f9018005fd0 tx=0x7f9018009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:21.437 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:21.435+0000 7f9024c00640 1 -- 172.21.15.72:0/3254252343 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f901409ca30 con 0x7f9028078530 2024-04-26T21:52:21.674 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:21.674+0000 7f902f200640 1 -- 172.21.15.72:0/3254252343 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f902802dad0 con 0x7f9028078530 2024-04-26T21:52:21.680 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:21.680+0000 7f9024c00640 1 -- 172.21.15.72:0/3254252343 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5524 (secure 0 0 0) 0x7f9014009de0 con 0x7f9028078530 2024-04-26T21:52:21.681 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T21:52:21.681 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6835/1320381959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6834","nonce":1320381959},{"type":"v1","addr":"172.21.15.72:6835","nonce":1320381959}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6837/2142200512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6836","nonce":2142200512},{"type":"v1","addr":"172.21.15.72:6837","nonce":2142200512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24388,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6833/4236668465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":4236668465},{"type":"v1","addr":"172.21.15.164:6833","nonce":4236668465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24424,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/2391114169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":2391114169},{"type":"v1","addr":"172.21.15.164:6835","nonce":2391114169}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":34406,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6833/3869483750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":3869483750},{"type":"v1","addr":"172.21.15.76:6833","nonce":3869483750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34442,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/3517429435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":3517429435},{"type":"v1","addr":"172.21.15.76:6835","nonce":3517429435}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-04-26T21:52:21.681 INFO:teuthology.orchestra.run.smithi072.stderr:dumped fsmap epoch 7 2024-04-26T21:52:21.683 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:21.683+0000 7f902f200640 1 -- 172.21.15.72:0/3254252343 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f900c0a3bb0 msgr2=0x7f900c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:21.684 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:21.683+0000 7f902f200640 1 --2- 172.21.15.72:0/3254252343 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f900c0a3bb0 0x7f900c0a6070 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f9018005fd0 tx=0x7f9018009260 comp rx=0 tx=0).stop 2024-04-26T21:52:21.684 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:21.683+0000 7f902f200640 1 -- 172.21.15.72:0/3254252343 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9028078530 msgr2=0x7f902819f5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:21.684 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:21.683+0000 7f902f200640 1 --2- 172.21.15.72:0/3254252343 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9028078530 0x7f902819f5b0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f901400d950 tx=0x7f901400de10 comp rx=0 tx=0).stop 2024-04-26T21:52:21.684 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:21.683+0000 7f902f200640 1 -- 172.21.15.72:0/3254252343 shutdown_connections 2024-04-26T21:52:21.684 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:21.683+0000 7f902f200640 1 --2- 172.21.15.72:0/3254252343 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f900c0a3bb0 0x7f900c0a6070 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:21.684 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:21.683+0000 7f902f200640 1 --2- 172.21.15.72:0/3254252343 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9028078ed0 0x7f90281a6620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:21.684 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:21.683+0000 7f902f200640 1 --2- 172.21.15.72:0/3254252343 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9028078530 0x7f902819f5b0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:21.684 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:21.683+0000 7f902f200640 1 --2- 172.21.15.72:0/3254252343 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f90280772f0 0x7f902819f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:21.684 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:21.683+0000 7f902f200640 1 -- 172.21.15.72:0/3254252343 >> 172.21.15.72:0/3254252343 conn(0x7f9028100120 msgr2=0x7f90281018e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:21.684 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:21.683+0000 7f902f200640 1 -- 172.21.15.72:0/3254252343 shutdown_connections 2024-04-26T21:52:21.684 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:21.683+0000 7f902f200640 1 -- 172.21.15.72:0/3254252343 wait complete. 2024-04-26T21:52:21.940 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:21 smithi164 ceph-mon[31916]: mds.? [v2:172.21.15.164:6834/2391114169,v1:172.21.15.164:6835/2391114169] up:boot 2024-04-26T21:52:21.941 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:21 smithi164 ceph-mon[31916]: fsmap 6 up:standby 2024-04-26T21:52:21.941 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:21 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-26T21:52:21.941 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:21 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:52:21.941 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:21 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:52:21.941 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:21 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:52:21.941 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:21 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:52:21.941 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:21 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:52:21.941 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:21 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:52:21.941 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:21 smithi164 ceph-mon[31916]: Deploying daemon mds.c on smithi164 2024-04-26T21:52:22.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:21 smithi076 ceph-mon[31823]: mds.? [v2:172.21.15.164:6834/2391114169,v1:172.21.15.164:6835/2391114169] up:boot 2024-04-26T21:52:22.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:21 smithi076 ceph-mon[31823]: fsmap 6 up:standby 2024-04-26T21:52:22.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:21 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-26T21:52:22.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:21 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:52:22.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:21 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:52:22.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:21 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:52:22.066 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:21 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:52:22.066 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:21 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:52:22.066 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:21 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:52:22.066 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:21 smithi076 ceph-mon[31823]: Deploying daemon mds.c on smithi164 2024-04-26T21:52:22.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:21 smithi072 ceph-mon[28138]: mds.? [v2:172.21.15.164:6834/2391114169,v1:172.21.15.164:6835/2391114169] up:boot 2024-04-26T21:52:22.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:21 smithi072 ceph-mon[28138]: fsmap 6 up:standby 2024-04-26T21:52:22.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:21 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-26T21:52:22.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:21 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:52:22.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:21 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:52:22.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:21 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:52:22.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:21 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:52:22.073 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:21 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:52:22.073 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:21 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:52:22.073 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:21 smithi072 ceph-mon[28138]: Deploying daemon mds.c on smithi164 2024-04-26T21:52:22.941 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:22 smithi164 ceph-mon[31916]: from='client.? 172.21.15.72:0/3254252343' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:52:22.941 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:22 smithi164 ceph-mon[31916]: pgmap v349: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:52:23.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:22 smithi076 ceph-mon[31823]: from='client.? 172.21.15.72:0/3254252343' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:52:23.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:22 smithi076 ceph-mon[31823]: pgmap v349: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:52:23.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:22 smithi072 ceph-mon[28138]: from='client.? 172.21.15.72:0/3254252343' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:52:23.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:22 smithi072 ceph-mon[28138]: pgmap v349: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:52:23.703 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:52:23.914 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:23.914+0000 7f4c1ec00640 1 -- 172.21.15.72:0/3608552346 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4c18109f80 msgr2=0x7f4c1810a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:23.914 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:23.914+0000 7f4c1ec00640 1 --2- 172.21.15.72:0/3608552346 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4c18109f80 0x7f4c1810a400 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f4c04009860 tx=0x7f4c0402a340 comp rx=0 tx=0).stop 2024-04-26T21:52:23.914 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:23.914+0000 7f4c1ec00640 1 -- 172.21.15.72:0/3608552346 shutdown_connections 2024-04-26T21:52:23.915 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:23.914+0000 7f4c1ec00640 1 --2- 172.21.15.72:0/3608552346 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4c1810a940 0x7f4c181111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:23.915 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:23.914+0000 7f4c1ec00640 1 --2- 172.21.15.72:0/3608552346 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4c18109f80 0x7f4c1810a400 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:23.915 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:23.914+0000 7f4c1ec00640 1 --2- 172.21.15.72:0/3608552346 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4c18108d90 0x7f4c181091b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:23.915 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:23.914+0000 7f4c1ec00640 1 -- 172.21.15.72:0/3608552346 >> 172.21.15.72:0/3608552346 conn(0x7f4c18077d30 msgr2=0x7f4c1807a170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:23.915 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:23.914+0000 7f4c1ec00640 1 -- 172.21.15.72:0/3608552346 shutdown_connections 2024-04-26T21:52:23.915 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:23.914+0000 7f4c1ec00640 1 -- 172.21.15.72:0/3608552346 wait complete. 2024-04-26T21:52:23.918 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:23.915+0000 7f4c1ec00640 1 Processor -- start 2024-04-26T21:52:23.918 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:23.915+0000 7f4c1ec00640 1 -- start start 2024-04-26T21:52:23.918 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:23.915+0000 7f4c1ec00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4c18108d90 0x7f4c1819ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:23.918 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:23.916+0000 7f4c1ec00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4c18109f80 0x7f4c1819f460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:23.918 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:23.916+0000 7f4c1ec00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4c1810a940 0x7f4c181a6480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:23.918 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:23.916+0000 7f4c1ec00640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f4c18114770 con 0x7f4c18109f80 2024-04-26T21:52:23.918 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:23.916+0000 7f4c1ec00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f4c181145f0 con 0x7f4c18108d90 2024-04-26T21:52:23.918 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:23.916+0000 7f4c1ec00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f4c181148f0 con 0x7f4c1810a940 2024-04-26T21:52:23.918 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:23.916+0000 7f4c1ce00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4c18109f80 0x7f4c1819f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:23.919 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:23.916+0000 7f4c1ce00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4c18109f80 0x7f4c1819f460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:59424/0 (socket says 172.21.15.72:59424) 2024-04-26T21:52:23.919 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:23.916+0000 7f4c1ce00640 1 -- 172.21.15.72:0/2077234878 learned_addr learned my addr 172.21.15.72:0/2077234878 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:52:23.919 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:23.916+0000 7f4c1e200640 1 --2- 172.21.15.72:0/2077234878 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4c1810a940 0x7f4c181a6480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:23.919 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:23.916+0000 7f4c1d800640 1 --2- 172.21.15.72:0/2077234878 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4c18108d90 0x7f4c1819ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:23.919 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:23.916+0000 7f4c1ce00640 1 -- 172.21.15.72:0/2077234878 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4c1810a940 msgr2=0x7f4c181a6480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:23.919 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:23.916+0000 7f4c1ce00640 1 --2- 172.21.15.72:0/2077234878 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4c1810a940 0x7f4c181a6480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:23.919 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:23.916+0000 7f4c1ce00640 1 -- 172.21.15.72:0/2077234878 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4c18108d90 msgr2=0x7f4c1819ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:23.919 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:23.916+0000 7f4c1ce00640 1 --2- 172.21.15.72:0/2077234878 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4c18108d90 0x7f4c1819ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:23.919 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:23.916+0000 7f4c1ce00640 1 -- 172.21.15.72:0/2077234878 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c181a6af0 con 0x7f4c18109f80 2024-04-26T21:52:23.919 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:23.917+0000 7f4c1ce00640 1 --2- 172.21.15.72:0/2077234878 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4c18109f80 0x7f4c1819f460 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f4c04005cd0 tx=0x7f4c040306e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:23.919 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:23.917+0000 7f4c16000640 1 -- 172.21.15.72:0/2077234878 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c04042070 con 0x7f4c18109f80 2024-04-26T21:52:23.919 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:23.917+0000 7f4c16000640 1 -- 172.21.15.72:0/2077234878 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4c04032df0 con 0x7f4c18109f80 2024-04-26T21:52:23.920 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:23.917+0000 7f4c1ec00640 1 -- 172.21.15.72:0/2077234878 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c181a6d80 con 0x7f4c18109f80 2024-04-26T21:52:23.920 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:23.917+0000 7f4c16000640 1 -- 172.21.15.72:0/2077234878 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c04037040 con 0x7f4c18109f80 2024-04-26T21:52:23.920 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:23.917+0000 7f4c1ec00640 1 -- 172.21.15.72:0/2077234878 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c181a7210 con 0x7f4c18109f80 2024-04-26T21:52:23.920 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:23.918+0000 7f4c1ec00640 1 -- 172.21.15.72:0/2077234878 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c18110170 con 0x7f4c18109f80 2024-04-26T21:52:23.921 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:23.920+0000 7f4c16000640 1 -- 172.21.15.72:0/2077234878 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f4c04045070 con 0x7f4c18109f80 2024-04-26T21:52:23.922 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:23.922+0000 7f4c16000640 1 --2- 172.21.15.72:0/2077234878 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f4c000a3bb0 0x7f4c000a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:23.923 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:23.922+0000 7f4c16000640 1 -- 172.21.15.72:0/2077234878 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7f4c040ef930 con 0x7f4c18109f80 2024-04-26T21:52:23.923 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:23.922+0000 7f4c1d800640 1 --2- 172.21.15.72:0/2077234878 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f4c000a3bb0 0x7f4c000a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:23.923 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:23.923+0000 7f4c1d800640 1 --2- 172.21.15.72:0/2077234878 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f4c000a3bb0 0x7f4c000a6070 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f4c0c0096c0 tx=0x7f4c0c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:23.927 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:23.927+0000 7f4c16000640 1 -- 172.21.15.72:0/2077234878 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f4c040bc710 con 0x7f4c18109f80 2024-04-26T21:52:24.170 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:24.170+0000 7f4c1ec00640 1 -- 172.21.15.72:0/2077234878 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4c1802dad0 con 0x7f4c18109f80 2024-04-26T21:52:24.171 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:24.171+0000 7f4c16000640 1 -- 172.21.15.72:0/2077234878 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5524 (secure 0 0 0) 0x7f4c040bc010 con 0x7f4c18109f80 2024-04-26T21:52:24.172 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T21:52:24.172 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6835/1320381959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6834","nonce":1320381959},{"type":"v1","addr":"172.21.15.72:6835","nonce":1320381959}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6837/2142200512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6836","nonce":2142200512},{"type":"v1","addr":"172.21.15.72:6837","nonce":2142200512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24388,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6833/4236668465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":4236668465},{"type":"v1","addr":"172.21.15.164:6833","nonce":4236668465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24424,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/2391114169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":2391114169},{"type":"v1","addr":"172.21.15.164:6835","nonce":2391114169}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":34406,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6833/3869483750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":3869483750},{"type":"v1","addr":"172.21.15.76:6833","nonce":3869483750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34442,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/3517429435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":3517429435},{"type":"v1","addr":"172.21.15.76:6835","nonce":3517429435}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-04-26T21:52:24.172 INFO:teuthology.orchestra.run.smithi072.stderr:dumped fsmap epoch 7 2024-04-26T21:52:24.175 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:24.174+0000 7f4c1ec00640 1 -- 172.21.15.72:0/2077234878 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f4c000a3bb0 msgr2=0x7f4c000a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:24.175 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:24.174+0000 7f4c1ec00640 1 --2- 172.21.15.72:0/2077234878 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f4c000a3bb0 0x7f4c000a6070 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f4c0c0096c0 tx=0x7f4c0c009260 comp rx=0 tx=0).stop 2024-04-26T21:52:24.175 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:24.174+0000 7f4c1ec00640 1 -- 172.21.15.72:0/2077234878 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4c18109f80 msgr2=0x7f4c1819f460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:24.175 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:24.174+0000 7f4c1ec00640 1 --2- 172.21.15.72:0/2077234878 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4c18109f80 0x7f4c1819f460 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f4c04005cd0 tx=0x7f4c040306e0 comp rx=0 tx=0).stop 2024-04-26T21:52:24.175 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:24.174+0000 7f4c1ec00640 1 -- 172.21.15.72:0/2077234878 shutdown_connections 2024-04-26T21:52:24.175 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:24.174+0000 7f4c1ec00640 1 --2- 172.21.15.72:0/2077234878 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f4c000a3bb0 0x7f4c000a6070 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:24.175 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:24.174+0000 7f4c1ec00640 1 --2- 172.21.15.72:0/2077234878 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4c1810a940 0x7f4c181a6480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:24.175 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:24.174+0000 7f4c1ec00640 1 --2- 172.21.15.72:0/2077234878 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4c18109f80 0x7f4c1819f460 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:24.175 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:24.174+0000 7f4c1ec00640 1 --2- 172.21.15.72:0/2077234878 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4c18108d90 0x7f4c1819ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:24.175 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:24.174+0000 7f4c1ec00640 1 -- 172.21.15.72:0/2077234878 >> 172.21.15.72:0/2077234878 conn(0x7f4c18077d30 msgr2=0x7f4c1810d340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:24.175 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:24.175+0000 7f4c1ec00640 1 -- 172.21.15.72:0/2077234878 shutdown_connections 2024-04-26T21:52:24.175 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:24.175+0000 7f4c1ec00640 1 -- 172.21.15.72:0/2077234878 wait complete. 2024-04-26T21:52:24.189 INFO:journalctl@ceph.mds.c.smithi164.stdout:Apr 26 21:52:23 smithi164 systemd[1]: Starting Ceph mds.c for a1780950-0415-11ef-bc93-c7b262605968... 2024-04-26T21:52:24.441 INFO:journalctl@ceph.mds.c.smithi164.stdout:Apr 26 21:52:24 smithi164 podman[62348]: 2024-04-26 21:52:24.093196436 +0000 UTC m=+0.032477029 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-26T21:52:24.441 INFO:journalctl@ceph.mds.c.smithi164.stdout:Apr 26 21:52:24 smithi164 podman[62348]: 2024-04-26 21:52:24.199538365 +0000 UTC m=+0.138818995 container create a2fd1d91709b00c4056662cb6db264689d23b313584591ce5719c5cbbb63db3b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-mds-c, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux ) 2024-04-26T21:52:24.832 INFO:journalctl@ceph.mds.c.smithi164.stdout:Apr 26 21:52:24 smithi164 podman[62348]: 2024-04-26 21:52:24.478960361 +0000 UTC m=+0.418241021 container init a2fd1d91709b00c4056662cb6db264689d23b313584591ce5719c5cbbb63db3b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-mds-c, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-04-26T21:52:24.833 INFO:journalctl@ceph.mds.c.smithi164.stdout:Apr 26 21:52:24 smithi164 podman[62348]: 2024-04-26 21:52:24.489489071 +0000 UTC m=+0.428769671 container start a2fd1d91709b00c4056662cb6db264689d23b313584591ce5719c5cbbb63db3b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-mds-c, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-04-26T21:52:24.833 INFO:journalctl@ceph.mds.c.smithi164.stdout:Apr 26 21:52:24 smithi164 ceph-mds[62367]: -- 172.21.15.164:0/2331403446 <== mon.1 v2:172.21.15.164:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x564c77df3a40 con 0x564c77dc6c00 2024-04-26T21:52:24.833 INFO:journalctl@ceph.mds.c.smithi164.stdout:Apr 26 21:52:24 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-mds-c[62363]: starting mds.c at 2024-04-26T21:52:24.833 INFO:journalctl@ceph.mds.c.smithi164.stdout:Apr 26 21:52:24 smithi164 bash[62348]: a2fd1d91709b00c4056662cb6db264689d23b313584591ce5719c5cbbb63db3b 2024-04-26T21:52:24.833 INFO:journalctl@ceph.mds.c.smithi164.stdout:Apr 26 21:52:24 smithi164 systemd[1]: Started Ceph mds.c for a1780950-0415-11ef-bc93-c7b262605968. 2024-04-26T21:52:25.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:24 smithi164 ceph-mon[31916]: pgmap v350: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:52:25.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:24 smithi164 ceph-mon[31916]: from='client.? 172.21.15.72:0/2077234878' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:52:25.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:24 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:52:25.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:24 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:52:25.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:24 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:52:25.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:24 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:52:25.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:24 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:52:25.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:24 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:52:25.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:24 smithi076 ceph-mon[31823]: pgmap v350: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:52:25.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:24 smithi076 ceph-mon[31823]: from='client.? 172.21.15.72:0/2077234878' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:52:25.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:24 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:52:25.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:24 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:52:25.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:24 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:52:25.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:24 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:52:25.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:24 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:52:25.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:24 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:52:25.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:24 smithi072 ceph-mon[28138]: pgmap v350: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:52:25.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:24 smithi072 ceph-mon[28138]: from='client.? 172.21.15.72:0/2077234878' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:52:25.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:24 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:52:25.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:24 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:52:25.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:24 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:52:25.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:24 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:52:25.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:24 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:52:25.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:24 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:52:26.020 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:25 smithi076 ceph-mon[31823]: Deploying daemon mds.h on smithi076 2024-04-26T21:52:26.020 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:25 smithi076 ceph-mon[31823]: mds.? [v2:172.21.15.164:6836/3585829992,v1:172.21.15.164:6837/3585829992] up:boot 2024-04-26T21:52:26.020 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:25 smithi076 ceph-mon[31823]: fsmap 7 up:standby 2024-04-26T21:52:26.020 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:25 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-26T21:52:26.093 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:25 smithi164 ceph-mon[31916]: Deploying daemon mds.h on smithi076 2024-04-26T21:52:26.093 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:25 smithi164 ceph-mon[31916]: mds.? [v2:172.21.15.164:6836/3585829992,v1:172.21.15.164:6837/3585829992] up:boot 2024-04-26T21:52:26.093 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:25 smithi164 ceph-mon[31916]: fsmap 7 up:standby 2024-04-26T21:52:26.093 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:25 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-26T21:52:26.194 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:52:26.272 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:25 smithi072 ceph-mon[28138]: Deploying daemon mds.h on smithi076 2024-04-26T21:52:26.272 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:25 smithi072 ceph-mon[28138]: mds.? [v2:172.21.15.164:6836/3585829992,v1:172.21.15.164:6837/3585829992] up:boot 2024-04-26T21:52:26.272 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:25 smithi072 ceph-mon[28138]: fsmap 7 up:standby 2024-04-26T21:52:26.272 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:25 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-26T21:52:26.410 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:26.409+0000 7f6507200640 1 -- 172.21.15.72:0/764130620 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6508106540 msgr2=0x7f650810cdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:26.410 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:26.409+0000 7f6507200640 1 --2- 172.21.15.72:0/764130620 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6508106540 0x7f650810cdd0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f6500009900 tx=0x7f650002a3f0 comp rx=0 tx=0).stop 2024-04-26T21:52:26.410 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:26.409+0000 7f6507200640 1 -- 172.21.15.72:0/764130620 shutdown_connections 2024-04-26T21:52:26.410 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:26.409+0000 7f6507200640 1 --2- 172.21.15.72:0/764130620 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6508106540 0x7f650810cdd0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:26.410 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:26.409+0000 7f6507200640 1 --2- 172.21.15.72:0/764130620 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6508105b80 0x7f6508106000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:26.410 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:26.409+0000 7f6507200640 1 --2- 172.21.15.72:0/764130620 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6508104990 0x7f6508104db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:26.410 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:26.409+0000 7f6507200640 1 -- 172.21.15.72:0/764130620 >> 172.21.15.72:0/764130620 conn(0x7f65080fff00 msgr2=0x7f6508102340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:26.411 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:26.409+0000 7f6507200640 1 -- 172.21.15.72:0/764130620 shutdown_connections 2024-04-26T21:52:26.411 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:26.409+0000 7f6507200640 1 -- 172.21.15.72:0/764130620 wait complete. 2024-04-26T21:52:26.413 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:26.410+0000 7f6507200640 1 Processor -- start 2024-04-26T21:52:26.413 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:26.411+0000 7f6507200640 1 -- start start 2024-04-26T21:52:26.413 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:26.411+0000 7f6507200640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6508104990 0x7f650819aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:26.413 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:26.411+0000 7f6507200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6508105b80 0x7f650819b000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:26.413 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:26.411+0000 7f6507200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6508106540 0x7f65081a2020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:26.413 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:26.411+0000 7f6507200640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f65081103c0 con 0x7f6508104990 2024-04-26T21:52:26.413 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:26.411+0000 7f6507200640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f6508110240 con 0x7f6508106540 2024-04-26T21:52:26.413 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:26.411+0000 7f6507200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f6508110540 con 0x7f6508105b80 2024-04-26T21:52:26.413 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:26.411+0000 7f6505e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6508104990 0x7f650819aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:26.413 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:26.411+0000 7f6505e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6508104990 0x7f650819aac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:59436/0 (socket says 172.21.15.72:59436) 2024-04-26T21:52:26.413 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:26.411+0000 7f6505e00640 1 -- 172.21.15.72:0/3748896553 learned_addr learned my addr 172.21.15.72:0/3748896553 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:52:26.414 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:26.411+0000 7f6505e00640 1 -- 172.21.15.72:0/3748896553 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6508105b80 msgr2=0x7f650819b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:26.414 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:26.411+0000 7f64fd400640 1 --2- 172.21.15.72:0/3748896553 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6508105b80 0x7f650819b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:26.414 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:26.412+0000 7f6505e00640 1 --2- 172.21.15.72:0/3748896553 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6508105b80 0x7f650819b000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:26.414 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:26.412+0000 7f6506800640 1 --2- 172.21.15.72:0/3748896553 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6508106540 0x7f65081a2020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:26.414 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:26.412+0000 7f6505e00640 1 -- 172.21.15.72:0/3748896553 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6508106540 msgr2=0x7f65081a2020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:26.414 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:26.412+0000 7f6505e00640 1 --2- 172.21.15.72:0/3748896553 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6508106540 0x7f65081a2020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:26.414 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:26.412+0000 7f6505e00640 1 -- 172.21.15.72:0/3748896553 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65081a2690 con 0x7f6508104990 2024-04-26T21:52:26.414 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:26.412+0000 7f64fd400640 1 --2- 172.21.15.72:0/3748896553 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6508105b80 0x7f650819b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:52:26.414 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:26.412+0000 7f6505e00640 1 --2- 172.21.15.72:0/3748896553 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6508104990 0x7f650819aac0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f64f800ea80 tx=0x7f64f800ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:26.414 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:26.412+0000 7f64ff400640 1 -- 172.21.15.72:0/3748896553 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64f800ce40 con 0x7f6508104990 2024-04-26T21:52:26.414 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:26.412+0000 7f64ff400640 1 -- 172.21.15.72:0/3748896553 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f64f8011900 con 0x7f6508104990 2024-04-26T21:52:26.414 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:26.412+0000 7f6507200640 1 -- 172.21.15.72:0/3748896553 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65081a2980 con 0x7f6508104990 2024-04-26T21:52:26.414 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:26.412+0000 7f64ff400640 1 -- 172.21.15.72:0/3748896553 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64f801a9e0 con 0x7f6508104990 2024-04-26T21:52:26.415 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:26.412+0000 7f6507200640 1 -- 172.21.15.72:0/3748896553 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65081a2f20 con 0x7f6508104990 2024-04-26T21:52:26.415 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:26.413+0000 7f6507200640 1 -- 172.21.15.72:0/3748896553 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f650810bcd0 con 0x7f6508104990 2024-04-26T21:52:26.415 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:26.415+0000 7f64ff400640 1 -- 172.21.15.72:0/3748896553 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f64f801ab80 con 0x7f6508104990 2024-04-26T21:52:26.418 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:26.418+0000 7f64ff400640 1 --2- 172.21.15.72:0/3748896553 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f64ec0a3c30 0x7f64ec0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:26.418 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:26.418+0000 7f64fd400640 1 --2- 172.21.15.72:0/3748896553 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f64ec0a3c30 0x7f64ec0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:26.419 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:26.418+0000 7f64ff400640 1 -- 172.21.15.72:0/3748896553 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7f64f80d05b0 con 0x7f6508104990 2024-04-26T21:52:26.419 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:26.419+0000 7f64fd400640 1 --2- 172.21.15.72:0/3748896553 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f64ec0a3c30 0x7f64ec0a60f0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f650819bf80 tx=0x7f64f0005d30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:26.424 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:26.424+0000 7f64ff400640 1 -- 172.21.15.72:0/3748896553 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f64f8016030 con 0x7f6508104990 2024-04-26T21:52:26.668 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:26.668+0000 7f6507200640 1 -- 172.21.15.72:0/3748896553 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f650802dad0 con 0x7f6508104990 2024-04-26T21:52:26.669 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:26.669+0000 7f64ff400640 1 -- 172.21.15.72:0/3748896553 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6352 (secure 0 0 0) 0x7f64f809d390 con 0x7f6508104990 2024-04-26T21:52:26.669 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T21:52:26.669 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6835/1320381959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6834","nonce":1320381959},{"type":"v1","addr":"172.21.15.72:6835","nonce":1320381959}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6837/2142200512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6836","nonce":2142200512},{"type":"v1","addr":"172.21.15.72:6837","nonce":2142200512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24388,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6833/4236668465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":4236668465},{"type":"v1","addr":"172.21.15.164:6833","nonce":4236668465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24424,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/2391114169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":2391114169},{"type":"v1","addr":"172.21.15.164:6835","nonce":2391114169}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24442,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6837/3585829992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":3585829992},{"type":"v1","addr":"172.21.15.164:6837","nonce":3585829992}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":34406,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6833/3869483750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":3869483750},{"type":"v1","addr":"172.21.15.76:6833","nonce":3869483750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34442,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/3517429435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":3517429435},{"type":"v1","addr":"172.21.15.76:6835","nonce":3517429435}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-04-26T21:52:26.670 INFO:teuthology.orchestra.run.smithi072.stderr:dumped fsmap epoch 8 2024-04-26T21:52:26.673 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:26.672+0000 7f6507200640 1 -- 172.21.15.72:0/3748896553 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f64ec0a3c30 msgr2=0x7f64ec0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:26.673 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:26.672+0000 7f6507200640 1 --2- 172.21.15.72:0/3748896553 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f64ec0a3c30 0x7f64ec0a60f0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f650819bf80 tx=0x7f64f0005d30 comp rx=0 tx=0).stop 2024-04-26T21:52:26.673 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:26.672+0000 7f6507200640 1 -- 172.21.15.72:0/3748896553 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6508104990 msgr2=0x7f650819aac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:26.673 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:26.672+0000 7f6507200640 1 --2- 172.21.15.72:0/3748896553 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6508104990 0x7f650819aac0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f64f800ea80 tx=0x7f64f800ef40 comp rx=0 tx=0).stop 2024-04-26T21:52:26.673 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:26.672+0000 7f6507200640 1 -- 172.21.15.72:0/3748896553 shutdown_connections 2024-04-26T21:52:26.673 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:26.672+0000 7f6507200640 1 --2- 172.21.15.72:0/3748896553 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6508106540 0x7f65081a2020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:26.673 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:26.672+0000 7f6507200640 1 --2- 172.21.15.72:0/3748896553 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f64ec0a3c30 0x7f64ec0a60f0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:26.673 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:26.672+0000 7f6507200640 1 --2- 172.21.15.72:0/3748896553 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6508105b80 0x7f650819b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:26.673 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:26.672+0000 7f6507200640 1 --2- 172.21.15.72:0/3748896553 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6508104990 0x7f650819aac0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:26.673 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:26.672+0000 7f6507200640 1 -- 172.21.15.72:0/3748896553 >> 172.21.15.72:0/3748896553 conn(0x7f65080fff00 msgr2=0x7f6508108ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:26.673 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:26.673+0000 7f6507200640 1 -- 172.21.15.72:0/3748896553 shutdown_connections 2024-04-26T21:52:26.673 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:26.673+0000 7f6507200640 1 -- 172.21.15.72:0/3748896553 wait complete. 2024-04-26T21:52:27.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:26 smithi072 ceph-mon[28138]: pgmap v351: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:52:27.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:26 smithi072 ceph-mon[28138]: from='client.? 172.21.15.72:0/3748896553' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:52:27.114 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:26 smithi164 ceph-mon[31916]: pgmap v351: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:52:27.114 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:26 smithi164 ceph-mon[31916]: from='client.? 172.21.15.72:0/3748896553' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:52:27.120 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:26 smithi076 ceph-mon[31823]: pgmap v351: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:52:27.120 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:26 smithi076 ceph-mon[31823]: from='client.? 172.21.15.72:0/3748896553' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:52:28.151 INFO:journalctl@ceph.mds.h.smithi076.stdout:Apr 26 21:52:27 smithi076 systemd[1]: Starting Ceph mds.h for a1780950-0415-11ef-bc93-c7b262605968... 2024-04-26T21:52:28.407 INFO:journalctl@ceph.mds.h.smithi076.stdout:Apr 26 21:52:28 smithi076 podman[60919]: 2024-04-26 21:52:28.150825165 +0000 UTC m=+0.111669585 container create 13e97c00bd326abdb7bfff9390f0d2c68c2d6ee147a64418377b978067ab513c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-mds-h, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff) 2024-04-26T21:52:28.407 INFO:journalctl@ceph.mds.h.smithi076.stdout:Apr 26 21:52:28 smithi076 podman[60919]: 2024-04-26 21:52:28.071587576 +0000 UTC m=+0.032431958 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-26T21:52:28.407 INFO:journalctl@ceph.mds.h.smithi076.stdout:Apr 26 21:52:28 smithi076 podman[60919]: 2024-04-26 21:52:28.404942108 +0000 UTC m=+0.365786535 container init 13e97c00bd326abdb7bfff9390f0d2c68c2d6ee147a64418377b978067ab513c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-mds-h, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-04-26T21:52:28.692 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:52:28.815 INFO:journalctl@ceph.mds.h.smithi076.stdout:Apr 26 21:52:28 smithi076 podman[60919]: 2024-04-26 21:52:28.414656749 +0000 UTC m=+0.375501142 container start 13e97c00bd326abdb7bfff9390f0d2c68c2d6ee147a64418377b978067ab513c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-mds-h, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=) 2024-04-26T21:52:28.815 INFO:journalctl@ceph.mds.h.smithi076.stdout:Apr 26 21:52:28 smithi076 ceph-mds[60938]: -- 172.21.15.76:0/1879700131 <== mon.2 v2:172.21.15.76:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5602de0dda40 con 0x5602de0b0c00 2024-04-26T21:52:28.815 INFO:journalctl@ceph.mds.h.smithi076.stdout:Apr 26 21:52:28 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-mds-h[60934]: starting mds.h at 2024-04-26T21:52:28.815 INFO:journalctl@ceph.mds.h.smithi076.stdout:Apr 26 21:52:28 smithi076 bash[60919]: 13e97c00bd326abdb7bfff9390f0d2c68c2d6ee147a64418377b978067ab513c 2024-04-26T21:52:28.815 INFO:journalctl@ceph.mds.h.smithi076.stdout:Apr 26 21:52:28 smithi076 systemd[1]: Started Ceph mds.h for a1780950-0415-11ef-bc93-c7b262605968. 2024-04-26T21:52:28.901 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:28.900+0000 7f87a6e00640 1 -- 172.21.15.72:0/4157839491 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f87a0106530 msgr2=0x7f87a010cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:28.901 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:28.900+0000 7f87a6e00640 1 --2- 172.21.15.72:0/4157839491 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f87a0106530 0x7f87a010cdc0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f8798009ca0 tx=0x7f879802a8c0 comp rx=0 tx=0).stop 2024-04-26T21:52:28.901 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:28.900+0000 7f87a6e00640 1 -- 172.21.15.72:0/4157839491 shutdown_connections 2024-04-26T21:52:28.901 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:28.900+0000 7f87a6e00640 1 --2- 172.21.15.72:0/4157839491 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f87a0106530 0x7f87a010cdc0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:28.901 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:28.900+0000 7f87a6e00640 1 --2- 172.21.15.72:0/4157839491 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f87a0105b70 0x7f87a0105ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:28.902 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:28.900+0000 7f87a6e00640 1 --2- 172.21.15.72:0/4157839491 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f87a0104980 0x7f87a0104da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:28.902 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:28.900+0000 7f87a6e00640 1 -- 172.21.15.72:0/4157839491 >> 172.21.15.72:0/4157839491 conn(0x7f87a00fff10 msgr2=0x7f87a0102330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:28.902 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:28.901+0000 7f87a6e00640 1 -- 172.21.15.72:0/4157839491 shutdown_connections 2024-04-26T21:52:28.902 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:28.901+0000 7f87a6e00640 1 -- 172.21.15.72:0/4157839491 wait complete. 2024-04-26T21:52:28.904 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:28.901+0000 7f87a6e00640 1 Processor -- start 2024-04-26T21:52:28.904 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:28.902+0000 7f87a6e00640 1 -- start start 2024-04-26T21:52:28.904 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:28.902+0000 7f87a6e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f87a0104980 0x7f87a019aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:28.904 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:28.902+0000 7f87a6e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f87a0105b70 0x7f87a019afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:28.904 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:28.902+0000 7f87a6e00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f87a0106530 0x7f87a01a2010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:28.904 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:28.902+0000 7f87a6e00640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f87a01103b0 con 0x7f87a0104980 2024-04-26T21:52:28.904 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:28.902+0000 7f87a6e00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f87a0110230 con 0x7f87a0106530 2024-04-26T21:52:28.905 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:28.902+0000 7f87a6e00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f87a0110530 con 0x7f87a0105b70 2024-04-26T21:52:28.905 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:28.902+0000 7f87a5a00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f87a0104980 0x7f87a019aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:28.905 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:28.902+0000 7f87a5a00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f87a0104980 0x7f87a019aa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:59454/0 (socket says 172.21.15.72:59454) 2024-04-26T21:52:28.905 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:28.902+0000 7f87a5a00640 1 -- 172.21.15.72:0/3930073289 learned_addr learned my addr 172.21.15.72:0/3930073289 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:52:28.905 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:28.902+0000 7f87a5000640 1 --2- 172.21.15.72:0/3930073289 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f87a0105b70 0x7f87a019afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:28.905 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:28.903+0000 7f87a5a00640 1 -- 172.21.15.72:0/3930073289 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f87a0105b70 msgr2=0x7f87a019afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:28.905 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:28.903+0000 7f87a5a00640 1 --2- 172.21.15.72:0/3930073289 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f87a0105b70 0x7f87a019afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:28.905 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:28.903+0000 7f87a5a00640 1 -- 172.21.15.72:0/3930073289 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f87a0106530 msgr2=0x7f87a01a2010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:28.905 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:28.903+0000 7f87a5a00640 1 --2- 172.21.15.72:0/3930073289 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f87a0106530 0x7f87a01a2010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:28.905 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:28.903+0000 7f87a5a00640 1 -- 172.21.15.72:0/3930073289 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87a01a2680 con 0x7f87a0104980 2024-04-26T21:52:28.905 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:28.903+0000 7f87a5a00640 1 --2- 172.21.15.72:0/3930073289 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f87a0104980 0x7f87a019aa60 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f879400e9c0 tx=0x7f879400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:28.905 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:28.903+0000 7f879e000640 1 -- 172.21.15.72:0/3930073289 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f879400cd80 con 0x7f87a0104980 2024-04-26T21:52:28.905 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:28.903+0000 7f879e000640 1 -- 172.21.15.72:0/3930073289 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f87940035f0 con 0x7f87a0104980 2024-04-26T21:52:28.905 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:28.903+0000 7f879e000640 1 -- 172.21.15.72:0/3930073289 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87940044c0 con 0x7f87a0104980 2024-04-26T21:52:28.905 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:28.903+0000 7f87a6e00640 1 -- 172.21.15.72:0/3930073289 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87a01a2970 con 0x7f87a0104980 2024-04-26T21:52:28.906 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:28.903+0000 7f87a6e00640 1 -- 172.21.15.72:0/3930073289 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87a01a2ec0 con 0x7f87a0104980 2024-04-26T21:52:28.906 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:28.904+0000 7f87a6e00640 1 -- 172.21.15.72:0/3930073289 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87a010bca0 con 0x7f87a0104980 2024-04-26T21:52:28.906 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:28.906+0000 7f879e000640 1 -- 172.21.15.72:0/3930073289 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f8794004000 con 0x7f87a0104980 2024-04-26T21:52:28.908 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:28.908+0000 7f879e000640 1 --2- 172.21.15.72:0/3930073289 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f87880a3c30 0x7f87880a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:28.908 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:28.908+0000 7f879e000640 1 -- 172.21.15.72:0/3930073289 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7f87940d0380 con 0x7f87a0104980 2024-04-26T21:52:28.908 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:28.908+0000 7f87a5000640 1 --2- 172.21.15.72:0/3930073289 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f87880a3c30 0x7f87880a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:28.909 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:28.909+0000 7f87a5000640 1 --2- 172.21.15.72:0/3930073289 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f87880a3c30 0x7f87880a60f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f87a019bf70 tx=0x7f878c00b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:28.912 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:28.912+0000 7f879e000640 1 -- 172.21.15.72:0/3930073289 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f879409d160 con 0x7f87a0104980 2024-04-26T21:52:29.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:28 smithi072 ceph-mon[28138]: pgmap v352: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:52:29.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:28 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:52:29.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:28 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:52:29.073 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:28 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:52:29.073 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:28 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:52:29.073 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:28 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:52:29.073 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:28 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:52:29.156 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:29.156+0000 7f87a6e00640 1 -- 172.21.15.72:0/3930073289 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f87a002dad0 con 0x7f87a0104980 2024-04-26T21:52:29.157 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:29.157+0000 7f879e000640 1 -- 172.21.15.72:0/3930073289 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7177 (secure 0 0 0) 0x7f879409ca60 con 0x7f87a0104980 2024-04-26T21:52:29.158 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T21:52:29.158 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6835/1320381959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6834","nonce":1320381959},{"type":"v1","addr":"172.21.15.72:6835","nonce":1320381959}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6837/2142200512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6836","nonce":2142200512},{"type":"v1","addr":"172.21.15.72:6837","nonce":2142200512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24388,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6833/4236668465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":4236668465},{"type":"v1","addr":"172.21.15.164:6833","nonce":4236668465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24424,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/2391114169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":2391114169},{"type":"v1","addr":"172.21.15.164:6835","nonce":2391114169}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24442,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6837/3585829992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":3585829992},{"type":"v1","addr":"172.21.15.164:6837","nonce":3585829992}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":34406,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6833/3869483750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":3869483750},{"type":"v1","addr":"172.21.15.76:6833","nonce":3869483750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34442,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/3517429435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":3517429435},{"type":"v1","addr":"172.21.15.76:6835","nonce":3517429435}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":34472,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6837/1150678991","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":1150678991},{"type":"v1","addr":"172.21.15.76:6837","nonce":1150678991}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-04-26T21:52:29.158 INFO:teuthology.orchestra.run.smithi072.stderr:dumped fsmap epoch 9 2024-04-26T21:52:29.160 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:29.160+0000 7f87a6e00640 1 -- 172.21.15.72:0/3930073289 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f87880a3c30 msgr2=0x7f87880a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:29.160 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:29.160+0000 7f87a6e00640 1 --2- 172.21.15.72:0/3930073289 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f87880a3c30 0x7f87880a60f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f87a019bf70 tx=0x7f878c00b390 comp rx=0 tx=0).stop 2024-04-26T21:52:29.160 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:29.160+0000 7f87a6e00640 1 -- 172.21.15.72:0/3930073289 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f87a0104980 msgr2=0x7f87a019aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:29.161 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:29.160+0000 7f87a6e00640 1 --2- 172.21.15.72:0/3930073289 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f87a0104980 0x7f87a019aa60 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f879400e9c0 tx=0x7f879400ee80 comp rx=0 tx=0).stop 2024-04-26T21:52:29.161 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:29.160+0000 7f87a6e00640 1 -- 172.21.15.72:0/3930073289 shutdown_connections 2024-04-26T21:52:29.161 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:29.160+0000 7f87a6e00640 1 --2- 172.21.15.72:0/3930073289 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f87a0106530 0x7f87a01a2010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:29.161 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:29.160+0000 7f87a6e00640 1 --2- 172.21.15.72:0/3930073289 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f87880a3c30 0x7f87880a60f0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:29.161 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:29.160+0000 7f87a6e00640 1 --2- 172.21.15.72:0/3930073289 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f87a0105b70 0x7f87a019afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:29.161 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:29.160+0000 7f87a6e00640 1 --2- 172.21.15.72:0/3930073289 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f87a0104980 0x7f87a019aa60 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:29.161 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:29.160+0000 7f87a6e00640 1 -- 172.21.15.72:0/3930073289 >> 172.21.15.72:0/3930073289 conn(0x7f87a00fff10 msgr2=0x7f87a0108e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:29.161 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:29.160+0000 7f87a6e00640 1 -- 172.21.15.72:0/3930073289 shutdown_connections 2024-04-26T21:52:29.161 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:29.160+0000 7f87a6e00640 1 -- 172.21.15.72:0/3930073289 wait complete. 2024-04-26T21:52:29.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:28 smithi164 ceph-mon[31916]: pgmap v352: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:52:29.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:28 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:52:29.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:28 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:52:29.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:28 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:52:29.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:28 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:52:29.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:28 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:52:29.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:28 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:52:29.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:28 smithi076 ceph-mon[31823]: pgmap v352: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:52:29.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:28 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:52:29.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:28 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:52:29.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:28 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:52:29.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:28 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:52:29.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:28 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:52:29.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:28 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:52:29.940 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:29 smithi164 ceph-mon[31916]: Deploying daemon mds.i on smithi164 2024-04-26T21:52:29.941 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:29 smithi164 ceph-mon[31916]: mds.? [v2:172.21.15.76:6836/1150678991,v1:172.21.15.76:6837/1150678991] up:boot 2024-04-26T21:52:29.941 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:29 smithi164 ceph-mon[31916]: fsmap 8 up:standby 2024-04-26T21:52:29.941 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:29 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-26T21:52:29.941 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:29 smithi164 ceph-mon[31916]: from='client.? 172.21.15.72:0/3930073289' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:52:30.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:29 smithi076 ceph-mon[31823]: Deploying daemon mds.i on smithi164 2024-04-26T21:52:30.361 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:29 smithi076 ceph-mon[31823]: mds.? [v2:172.21.15.76:6836/1150678991,v1:172.21.15.76:6837/1150678991] up:boot 2024-04-26T21:52:30.361 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:29 smithi076 ceph-mon[31823]: fsmap 8 up:standby 2024-04-26T21:52:30.361 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:29 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-26T21:52:30.361 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:29 smithi076 ceph-mon[31823]: from='client.? 172.21.15.72:0/3930073289' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:52:30.362 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:29 smithi072 ceph-mon[28138]: Deploying daemon mds.i on smithi164 2024-04-26T21:52:30.362 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:29 smithi072 ceph-mon[28138]: mds.? [v2:172.21.15.76:6836/1150678991,v1:172.21.15.76:6837/1150678991] up:boot 2024-04-26T21:52:30.362 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:29 smithi072 ceph-mon[28138]: fsmap 8 up:standby 2024-04-26T21:52:30.362 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:29 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-26T21:52:30.362 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:29 smithi072 ceph-mon[28138]: from='client.? 172.21.15.72:0/3930073289' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:52:31.181 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:52:31.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:30 smithi164 ceph-mon[31916]: pgmap v353: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:52:31.223 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:30 smithi072 ceph-mon[28138]: pgmap v353: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:52:31.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:30 smithi076 ceph-mon[31823]: pgmap v353: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:52:31.393 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:31.392+0000 7f50f9800640 1 -- 172.21.15.72:0/2105412743 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f50f41011c0 msgr2=0x7f50f41015e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:31.393 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:31.392+0000 7f50f9800640 1 --2- 172.21.15.72:0/2105412743 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f50f41011c0 0x7f50f41015e0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f50e4009900 tx=0x7f50e402a410 comp rx=0 tx=0).stop 2024-04-26T21:52:31.393 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:31.392+0000 7f50f9800640 1 -- 172.21.15.72:0/2105412743 shutdown_connections 2024-04-26T21:52:31.393 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:31.392+0000 7f50f9800640 1 --2- 172.21.15.72:0/2105412743 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f50f41024c0 0x7f50f4102940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:31.393 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:31.393+0000 7f50f9800640 1 --2- 172.21.15.72:0/2105412743 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f50f4101b20 0x7f50f4101f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:31.393 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:31.393+0000 7f50f9800640 1 --2- 172.21.15.72:0/2105412743 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f50f41011c0 0x7f50f41015e0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:31.393 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:31.393+0000 7f50f9800640 1 -- 172.21.15.72:0/2105412743 >> 172.21.15.72:0/2105412743 conn(0x7f50f40fcdc0 msgr2=0x7f50f40ff200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:31.394 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:31.393+0000 7f50f9800640 1 -- 172.21.15.72:0/2105412743 shutdown_connections 2024-04-26T21:52:31.394 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:31.393+0000 7f50f9800640 1 -- 172.21.15.72:0/2105412743 wait complete. 2024-04-26T21:52:31.396 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:31.394+0000 7f50f9800640 1 Processor -- start 2024-04-26T21:52:31.396 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:31.394+0000 7f50f9800640 1 -- start start 2024-04-26T21:52:31.396 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:31.394+0000 7f50f9800640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f50f41011c0 0x7f50f419ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:31.396 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:31.394+0000 7f50f9800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f50f4101b20 0x7f50f419b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:31.396 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:31.394+0000 7f50f9800640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f50f41024c0 0x7f50f41a22f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:31.397 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:31.394+0000 7f50f9800640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f50f41105c0 con 0x7f50f41024c0 2024-04-26T21:52:31.397 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:31.394+0000 7f50f9800640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f50f4110440 con 0x7f50f41011c0 2024-04-26T21:52:31.397 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:31.394+0000 7f50f9800640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f50f4110740 con 0x7f50f4101b20 2024-04-26T21:52:31.397 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:31.394+0000 7f50f3400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f50f41024c0 0x7f50f41a22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:31.397 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:31.395+0000 7f50f3400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f50f41024c0 0x7f50f41a22f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:34952/0 (socket says 172.21.15.72:34952) 2024-04-26T21:52:31.397 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:31.395+0000 7f50f3400640 1 -- 172.21.15.72:0/296806299 learned_addr learned my addr 172.21.15.72:0/296806299 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:52:31.397 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:31.395+0000 7f50f2000640 1 --2- 172.21.15.72:0/296806299 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f50f4101b20 0x7f50f419b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:31.397 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:31.395+0000 7f50f2a00640 1 --2- 172.21.15.72:0/296806299 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f50f41011c0 0x7f50f419ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:31.397 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:31.395+0000 7f50f3400640 1 -- 172.21.15.72:0/296806299 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f50f4101b20 msgr2=0x7f50f419b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:31.397 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:31.395+0000 7f50f3400640 1 --2- 172.21.15.72:0/296806299 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f50f4101b20 0x7f50f419b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:31.397 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:31.395+0000 7f50f3400640 1 -- 172.21.15.72:0/296806299 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f50f41011c0 msgr2=0x7f50f419ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:31.397 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:31.395+0000 7f50f3400640 1 --2- 172.21.15.72:0/296806299 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f50f41011c0 0x7f50f419ad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:31.397 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:31.395+0000 7f50f3400640 1 -- 172.21.15.72:0/296806299 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50f41a2960 con 0x7f50f41024c0 2024-04-26T21:52:31.398 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:31.395+0000 7f50f2a00640 1 --2- 172.21.15.72:0/296806299 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f50f41011c0 0x7f50f419ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:52:31.398 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:31.395+0000 7f50f3400640 1 --2- 172.21.15.72:0/296806299 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f50f41024c0 0x7f50f41a22f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f50ec00cad0 tx=0x7f50ec00ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:31.398 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:31.396+0000 7f50eb400640 1 -- 172.21.15.72:0/296806299 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50ec00ad60 con 0x7f50f41024c0 2024-04-26T21:52:31.398 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:31.396+0000 7f50eb400640 1 -- 172.21.15.72:0/296806299 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f50ec005020 con 0x7f50f41024c0 2024-04-26T21:52:31.398 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:31.396+0000 7f50eb400640 1 -- 172.21.15.72:0/296806299 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50ec00f6b0 con 0x7f50f41024c0 2024-04-26T21:52:31.398 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:31.396+0000 7f50f9800640 1 -- 172.21.15.72:0/296806299 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50f41a2c50 con 0x7f50f41024c0 2024-04-26T21:52:31.398 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:31.396+0000 7f50f9800640 1 -- 172.21.15.72:0/296806299 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50f41a31a0 con 0x7f50f41024c0 2024-04-26T21:52:31.398 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:31.397+0000 7f50f9800640 1 -- 172.21.15.72:0/296806299 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50f410c1e0 con 0x7f50f41024c0 2024-04-26T21:52:31.399 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:31.399+0000 7f50eb400640 1 -- 172.21.15.72:0/296806299 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f50ec0051d0 con 0x7f50f41024c0 2024-04-26T21:52:31.400 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:31.400+0000 7f50eb400640 1 --2- 172.21.15.72:0/296806299 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f50d80a3c30 0x7f50d80a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:31.401 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:31.401+0000 7f50eb400640 1 -- 172.21.15.72:0/296806299 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7f50ec0cffe0 con 0x7f50f41024c0 2024-04-26T21:52:31.401 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:31.401+0000 7f50f2a00640 1 --2- 172.21.15.72:0/296806299 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f50d80a3c30 0x7f50d80a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:31.402 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:31.402+0000 7f50f2a00640 1 --2- 172.21.15.72:0/296806299 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f50d80a3c30 0x7f50d80a60f0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f50e4006010 tx=0x7f50e4031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:31.405 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:31.405+0000 7f50eb400640 1 -- 172.21.15.72:0/296806299 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f50ec014030 con 0x7f50f41024c0 2024-04-26T21:52:31.648 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:31.648+0000 7f50f9800640 1 -- 172.21.15.72:0/296806299 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f50f402dad0 con 0x7f50f41024c0 2024-04-26T21:52:31.650 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:31.649+0000 7f50eb400640 1 -- 172.21.15.72:0/296806299 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7177 (secure 0 0 0) 0x7f50ec09ce20 con 0x7f50f41024c0 2024-04-26T21:52:31.650 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T21:52:31.650 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6835/1320381959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6834","nonce":1320381959},{"type":"v1","addr":"172.21.15.72:6835","nonce":1320381959}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6837/2142200512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6836","nonce":2142200512},{"type":"v1","addr":"172.21.15.72:6837","nonce":2142200512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24388,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6833/4236668465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":4236668465},{"type":"v1","addr":"172.21.15.164:6833","nonce":4236668465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24424,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/2391114169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":2391114169},{"type":"v1","addr":"172.21.15.164:6835","nonce":2391114169}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24442,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6837/3585829992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":3585829992},{"type":"v1","addr":"172.21.15.164:6837","nonce":3585829992}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":34406,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6833/3869483750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":3869483750},{"type":"v1","addr":"172.21.15.76:6833","nonce":3869483750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34442,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/3517429435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":3517429435},{"type":"v1","addr":"172.21.15.76:6835","nonce":3517429435}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":34472,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6837/1150678991","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":1150678991},{"type":"v1","addr":"172.21.15.76:6837","nonce":1150678991}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-04-26T21:52:31.650 INFO:teuthology.orchestra.run.smithi072.stderr:dumped fsmap epoch 9 2024-04-26T21:52:31.653 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:31.652+0000 7f50f9800640 1 -- 172.21.15.72:0/296806299 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f50d80a3c30 msgr2=0x7f50d80a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:31.653 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:31.652+0000 7f50f9800640 1 --2- 172.21.15.72:0/296806299 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f50d80a3c30 0x7f50d80a60f0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f50e4006010 tx=0x7f50e4031400 comp rx=0 tx=0).stop 2024-04-26T21:52:31.653 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:31.652+0000 7f50f9800640 1 -- 172.21.15.72:0/296806299 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f50f41024c0 msgr2=0x7f50f41a22f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:31.653 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:31.652+0000 7f50f9800640 1 --2- 172.21.15.72:0/296806299 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f50f41024c0 0x7f50f41a22f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f50ec00cad0 tx=0x7f50ec00ced0 comp rx=0 tx=0).stop 2024-04-26T21:52:31.653 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:31.652+0000 7f50f9800640 1 -- 172.21.15.72:0/296806299 shutdown_connections 2024-04-26T21:52:31.653 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:31.652+0000 7f50f9800640 1 --2- 172.21.15.72:0/296806299 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f50f41024c0 0x7f50f41a22f0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:31.653 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:31.652+0000 7f50f9800640 1 --2- 172.21.15.72:0/296806299 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f50d80a3c30 0x7f50d80a60f0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:31.653 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:31.652+0000 7f50f9800640 1 --2- 172.21.15.72:0/296806299 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f50f4101b20 0x7f50f419b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:31.653 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:31.652+0000 7f50f9800640 1 --2- 172.21.15.72:0/296806299 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f50f41011c0 0x7f50f419ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:31.653 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:31.652+0000 7f50f9800640 1 -- 172.21.15.72:0/296806299 >> 172.21.15.72:0/296806299 conn(0x7f50f40fcdc0 msgr2=0x7f50f40fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:31.653 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:31.653+0000 7f50f9800640 1 -- 172.21.15.72:0/296806299 shutdown_connections 2024-04-26T21:52:31.653 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:31.653+0000 7f50f9800640 1 -- 172.21.15.72:0/296806299 wait complete. 2024-04-26T21:52:32.165 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:31 smithi164 ceph-mon[31916]: from='client.? 172.21.15.72:0/296806299' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:52:32.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:31 smithi076 ceph-mon[31823]: from='client.? 172.21.15.72:0/296806299' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:52:32.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:31 smithi072 ceph-mon[28138]: from='client.? 172.21.15.72:0/296806299' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:52:32.432 INFO:journalctl@ceph.mds.i.smithi164.stdout:Apr 26 21:52:32 smithi164 podman[62653]: 2024-04-26 21:52:32.177178291 +0000 UTC m=+0.141605843 container create 9b322e4f6c6d0b46dcc2efe226853e8a0fbed9c0b8a43754e95374190ce124eb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-mds-i, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-04-26T21:52:32.432 INFO:journalctl@ceph.mds.i.smithi164.stdout:Apr 26 21:52:32 smithi164 podman[62653]: 2024-04-26 21:52:32.431038879 +0000 UTC m=+0.395466474 container init 9b322e4f6c6d0b46dcc2efe226853e8a0fbed9c0b8a43754e95374190ce124eb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-mds-i, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-04-26T21:52:32.691 INFO:journalctl@ceph.mds.i.smithi164.stdout:Apr 26 21:52:32 smithi164 podman[62653]: 2024-04-26 21:52:32.441534335 +0000 UTC m=+0.405961868 container start 9b322e4f6c6d0b46dcc2efe226853e8a0fbed9c0b8a43754e95374190ce124eb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-mds-i, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=) 2024-04-26T21:52:32.691 INFO:journalctl@ceph.mds.i.smithi164.stdout:Apr 26 21:52:32 smithi164 ceph-mds[62672]: -- 172.21.15.164:0/2016351296 <== mon.1 v2:172.21.15.164:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x564cd5f6ba40 con 0x564cd5f3ec00 2024-04-26T21:52:32.692 INFO:journalctl@ceph.mds.i.smithi164.stdout:Apr 26 21:52:32 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-mds-i[62668]: starting mds.i at 2024-04-26T21:52:32.692 INFO:journalctl@ceph.mds.i.smithi164.stdout:Apr 26 21:52:32 smithi164 bash[62653]: 9b322e4f6c6d0b46dcc2efe226853e8a0fbed9c0b8a43754e95374190ce124eb 2024-04-26T21:52:32.692 INFO:journalctl@ceph.mds.i.smithi164.stdout:Apr 26 21:52:32 smithi164 systemd[1]: Started Ceph mds.i for a1780950-0415-11ef-bc93-c7b262605968. 2024-04-26T21:52:33.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:32 smithi164 ceph-mon[31916]: pgmap v354: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:52:33.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:32 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:52:33.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:32 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:52:33.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:32 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:52:33.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:32 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:52:33.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:32 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:52:33.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:32 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:52:33.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:32 smithi076 ceph-mon[31823]: pgmap v354: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:52:33.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:32 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:52:33.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:32 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:52:33.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:32 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:52:33.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:32 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:52:33.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:32 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:52:33.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:32 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:52:33.321 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:32 smithi072 ceph-mon[28138]: pgmap v354: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:52:33.321 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:32 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:52:33.321 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:32 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:52:33.321 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:32 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:52:33.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:32 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:52:33.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:32 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:52:33.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:32 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:52:33.673 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:52:33.881 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:33.881+0000 7fa4f1000640 1 -- 172.21.15.72:0/12937008 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa4ec10a800 msgr2=0x7fa4ec10cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:33.881 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:33.881+0000 7fa4f1000640 1 --2- 172.21.15.72:0/12937008 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa4ec10a800 0x7fa4ec10cca0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fa4e4009ca0 tx=0x7fa4e402a950 comp rx=0 tx=0).stop 2024-04-26T21:52:33.882 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:33.881+0000 7fa4f1000640 1 -- 172.21.15.72:0/12937008 shutdown_connections 2024-04-26T21:52:33.882 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:33.881+0000 7fa4f1000640 1 --2- 172.21.15.72:0/12937008 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa4ec10a800 0x7fa4ec10cca0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:33.882 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:33.881+0000 7fa4f1000640 1 --2- 172.21.15.72:0/12937008 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa4ec075330 0x7fa4ec075790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:33.882 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:33.881+0000 7fa4f1000640 1 --2- 172.21.15.72:0/12937008 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa4ec076910 0x7fa4ec074d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:33.882 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:33.881+0000 7fa4f1000640 1 -- 172.21.15.72:0/12937008 >> 172.21.15.72:0/12937008 conn(0x7fa4ec0fdee0 msgr2=0x7fa4ec100320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:33.882 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:33.881+0000 7fa4f1000640 1 -- 172.21.15.72:0/12937008 shutdown_connections 2024-04-26T21:52:33.882 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:33.881+0000 7fa4f1000640 1 -- 172.21.15.72:0/12937008 wait complete. 2024-04-26T21:52:33.883 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:33.883+0000 7fa4f1000640 1 Processor -- start 2024-04-26T21:52:33.883 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:33.883+0000 7fa4f1000640 1 -- start start 2024-04-26T21:52:33.885 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:33.884+0000 7fa4f1000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa4ec075330 0x7fa4ec19abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:33.885 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:33.884+0000 7fa4f1000640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa4ec076910 0x7fa4ec19b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:33.885 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:33.884+0000 7fa4f1000640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa4ec10a800 0x7fa4ec1a2160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:33.885 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:33.884+0000 7fa4f1000640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fa4ec110580 con 0x7fa4ec10a800 2024-04-26T21:52:33.886 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:33.884+0000 7fa4f1000640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fa4ec110400 con 0x7fa4ec076910 2024-04-26T21:52:33.886 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:33.884+0000 7fa4f1000640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa4ec110700 con 0x7fa4ec075330 2024-04-26T21:52:33.886 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:33.884+0000 7fa4ea800640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa4ec10a800 0x7fa4ec1a2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:33.886 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:33.884+0000 7fa4ea800640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa4ec10a800 0x7fa4ec1a2160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:34960/0 (socket says 172.21.15.72:34960) 2024-04-26T21:52:33.886 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:33.884+0000 7fa4ea800640 1 -- 172.21.15.72:0/3284101917 learned_addr learned my addr 172.21.15.72:0/3284101917 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:52:33.886 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:33.884+0000 7fa4e9400640 1 --2- 172.21.15.72:0/3284101917 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa4ec076910 0x7fa4ec19b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:33.886 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:33.884+0000 7fa4e9e00640 1 --2- 172.21.15.72:0/3284101917 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa4ec075330 0x7fa4ec19abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:33.886 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:33.884+0000 7fa4ea800640 1 -- 172.21.15.72:0/3284101917 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa4ec075330 msgr2=0x7fa4ec19abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:33.886 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:33.884+0000 7fa4ea800640 1 --2- 172.21.15.72:0/3284101917 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa4ec075330 0x7fa4ec19abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:33.886 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:33.884+0000 7fa4ea800640 1 -- 172.21.15.72:0/3284101917 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa4ec076910 msgr2=0x7fa4ec19b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:33.886 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:33.884+0000 7fa4ea800640 1 --2- 172.21.15.72:0/3284101917 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa4ec076910 0x7fa4ec19b0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:33.886 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:33.884+0000 7fa4ea800640 1 -- 172.21.15.72:0/3284101917 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4ec1a27d0 con 0x7fa4ec10a800 2024-04-26T21:52:33.886 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:33.885+0000 7fa4ea800640 1 --2- 172.21.15.72:0/3284101917 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa4ec10a800 0x7fa4ec1a2160 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fa4e402f8c0 tx=0x7fa4e4003f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:33.886 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:33.885+0000 7fa4e2a00640 1 -- 172.21.15.72:0/3284101917 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4e4034040 con 0x7fa4ec10a800 2024-04-26T21:52:33.887 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:33.885+0000 7fa4e2a00640 1 -- 172.21.15.72:0/3284101917 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa4e4030880 con 0x7fa4ec10a800 2024-04-26T21:52:33.887 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:33.885+0000 7fa4f1000640 1 -- 172.21.15.72:0/3284101917 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4ec1a2a60 con 0x7fa4ec10a800 2024-04-26T21:52:33.887 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:33.885+0000 7fa4e2a00640 1 -- 172.21.15.72:0/3284101917 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4e4039450 con 0x7fa4ec10a800 2024-04-26T21:52:33.887 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:33.885+0000 7fa4f1000640 1 -- 172.21.15.72:0/3284101917 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4ec1a2f50 con 0x7fa4ec10a800 2024-04-26T21:52:33.887 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:33.886+0000 7fa4f1000640 1 -- 172.21.15.72:0/3284101917 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4ec1095c0 con 0x7fa4ec10a800 2024-04-26T21:52:33.888 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:33.888+0000 7fa4e2a00640 1 -- 172.21.15.72:0/3284101917 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fa4e4037020 con 0x7fa4ec10a800 2024-04-26T21:52:33.890 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:33.890+0000 7fa4e2a00640 1 --2- 172.21.15.72:0/3284101917 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fa4d00a3bb0 0x7fa4d00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:33.890 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:33.890+0000 7fa4e2a00640 1 -- 172.21.15.72:0/3284101917 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7fa4e40f1310 con 0x7fa4ec10a800 2024-04-26T21:52:33.890 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:33.890+0000 7fa4e9e00640 1 --2- 172.21.15.72:0/3284101917 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fa4d00a3bb0 0x7fa4d00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:33.890 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:33.890+0000 7fa4e9e00640 1 --2- 172.21.15.72:0/3284101917 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fa4d00a3bb0 0x7fa4d00a6070 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fa4dc0058c0 tx=0x7fa4dc00a4c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:33.894 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:33.894+0000 7fa4e2a00640 1 -- 172.21.15.72:0/3284101917 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fa4e40be0f0 con 0x7fa4ec10a800 2024-04-26T21:52:34.121 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:34.121+0000 7fa4f1000640 1 -- 172.21.15.72:0/3284101917 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa4ec02dad0 con 0x7fa4ec10a800 2024-04-26T21:52:34.122 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:34.122+0000 7fa4e2a00640 1 -- 172.21.15.72:0/3284101917 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8007 (secure 0 0 0) 0x7fa4e40bd9f0 con 0x7fa4ec10a800 2024-04-26T21:52:34.123 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T21:52:34.123 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6835/1320381959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6834","nonce":1320381959},{"type":"v1","addr":"172.21.15.72:6835","nonce":1320381959}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6837/2142200512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6836","nonce":2142200512},{"type":"v1","addr":"172.21.15.72:6837","nonce":2142200512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24388,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6833/4236668465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":4236668465},{"type":"v1","addr":"172.21.15.164:6833","nonce":4236668465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24424,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/2391114169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":2391114169},{"type":"v1","addr":"172.21.15.164:6835","nonce":2391114169}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24442,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6837/3585829992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":3585829992},{"type":"v1","addr":"172.21.15.164:6837","nonce":3585829992}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24463,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6839/1636226820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":1636226820},{"type":"v1","addr":"172.21.15.164:6839","nonce":1636226820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":34406,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6833/3869483750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":3869483750},{"type":"v1","addr":"172.21.15.76:6833","nonce":3869483750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34442,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/3517429435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":3517429435},{"type":"v1","addr":"172.21.15.76:6835","nonce":3517429435}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":34472,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6837/1150678991","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":1150678991},{"type":"v1","addr":"172.21.15.76:6837","nonce":1150678991}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-04-26T21:52:34.123 INFO:teuthology.orchestra.run.smithi072.stderr:dumped fsmap epoch 10 2024-04-26T21:52:34.125 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:34.124+0000 7fa4f1000640 1 -- 172.21.15.72:0/3284101917 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fa4d00a3bb0 msgr2=0x7fa4d00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:34.125 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:34.124+0000 7fa4f1000640 1 --2- 172.21.15.72:0/3284101917 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fa4d00a3bb0 0x7fa4d00a6070 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fa4dc0058c0 tx=0x7fa4dc00a4c0 comp rx=0 tx=0).stop 2024-04-26T21:52:34.125 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:34.124+0000 7fa4f1000640 1 -- 172.21.15.72:0/3284101917 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa4ec10a800 msgr2=0x7fa4ec1a2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:34.125 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:34.124+0000 7fa4f1000640 1 --2- 172.21.15.72:0/3284101917 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa4ec10a800 0x7fa4ec1a2160 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fa4e402f8c0 tx=0x7fa4e4003f60 comp rx=0 tx=0).stop 2024-04-26T21:52:34.125 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:34.125+0000 7fa4f1000640 1 -- 172.21.15.72:0/3284101917 shutdown_connections 2024-04-26T21:52:34.126 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:34.125+0000 7fa4f1000640 1 --2- 172.21.15.72:0/3284101917 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa4ec10a800 0x7fa4ec1a2160 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:34.126 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:34.125+0000 7fa4f1000640 1 --2- 172.21.15.72:0/3284101917 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa4ec076910 0x7fa4ec19b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:34.126 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:34.125+0000 7fa4f1000640 1 --2- 172.21.15.72:0/3284101917 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fa4d00a3bb0 0x7fa4d00a6070 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:34.126 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:34.125+0000 7fa4f1000640 1 --2- 172.21.15.72:0/3284101917 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa4ec075330 0x7fa4ec19abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:34.126 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:34.125+0000 7fa4f1000640 1 -- 172.21.15.72:0/3284101917 >> 172.21.15.72:0/3284101917 conn(0x7fa4ec0fdee0 msgr2=0x7fa4ec106790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:34.126 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:34.125+0000 7fa4f1000640 1 -- 172.21.15.72:0/3284101917 shutdown_connections 2024-04-26T21:52:34.126 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:34.125+0000 7fa4f1000640 1 -- 172.21.15.72:0/3284101917 wait complete. 2024-04-26T21:52:34.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:33 smithi164 ceph-mon[31916]: Deploying daemon mds.g on smithi072 2024-04-26T21:52:34.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:33 smithi164 ceph-mon[31916]: mds.? [v2:172.21.15.164:6838/1636226820,v1:172.21.15.164:6839/1636226820] up:boot 2024-04-26T21:52:34.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:33 smithi164 ceph-mon[31916]: fsmap 9 up:standby 2024-04-26T21:52:34.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:33 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-26T21:52:34.199 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:33 smithi072 ceph-mon[28138]: Deploying daemon mds.g on smithi072 2024-04-26T21:52:34.199 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:33 smithi072 ceph-mon[28138]: mds.? [v2:172.21.15.164:6838/1636226820,v1:172.21.15.164:6839/1636226820] up:boot 2024-04-26T21:52:34.199 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:33 smithi072 ceph-mon[28138]: fsmap 9 up:standby 2024-04-26T21:52:34.199 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:33 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-26T21:52:34.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:33 smithi076 ceph-mon[31823]: Deploying daemon mds.g on smithi072 2024-04-26T21:52:34.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:33 smithi076 ceph-mon[31823]: mds.? [v2:172.21.15.164:6838/1636226820,v1:172.21.15.164:6839/1636226820] up:boot 2024-04-26T21:52:34.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:33 smithi076 ceph-mon[31823]: fsmap 9 up:standby 2024-04-26T21:52:34.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:33 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-26T21:52:35.032 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:34 smithi072 ceph-mon[28138]: pgmap v355: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:52:35.032 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:34 smithi072 ceph-mon[28138]: from='client.? 172.21.15.72:0/3284101917' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:52:35.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:34 smithi164 ceph-mon[31916]: pgmap v355: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:52:35.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:34 smithi164 ceph-mon[31916]: from='client.? 172.21.15.72:0/3284101917' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:52:35.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:34 smithi076 ceph-mon[31823]: pgmap v355: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:52:35.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:34 smithi076 ceph-mon[31823]: from='client.? 172.21.15.72:0/3284101917' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:52:36.073 INFO:journalctl@ceph.mds.g.smithi072.stdout:Apr 26 21:52:35 smithi072 systemd[1]: Starting Ceph mds.g for a1780950-0415-11ef-bc93-c7b262605968... 2024-04-26T21:52:36.143 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:52:36.330 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:36.330+0000 7f319b000640 1 -- 172.21.15.72:0/790305329 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3194105b20 msgr2=0x7f3194107f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:36.331 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:36.330+0000 7f319b000640 1 --2- 172.21.15.72:0/790305329 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3194105b20 0x7f3194107f10 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f3180009900 tx=0x7f318002a410 comp rx=0 tx=0).stop 2024-04-26T21:52:36.331 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:36.330+0000 7f319b000640 1 -- 172.21.15.72:0/790305329 shutdown_connections 2024-04-26T21:52:36.331 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:36.330+0000 7f319b000640 1 --2- 172.21.15.72:0/790305329 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3194108450 0x7f319410a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:36.331 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:36.330+0000 7f319b000640 1 --2- 172.21.15.72:0/790305329 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3194105b20 0x7f3194107f10 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:36.331 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:36.330+0000 7f319b000640 1 --2- 172.21.15.72:0/790305329 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f31941031f0 0x7f31941055e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:36.331 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:36.330+0000 7f319b000640 1 -- 172.21.15.72:0/790305329 >> 172.21.15.72:0/790305329 conn(0x7f31940fcbc0 msgr2=0x7f31940ff020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:36.331 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:36.330+0000 7f319b000640 1 -- 172.21.15.72:0/790305329 shutdown_connections 2024-04-26T21:52:36.331 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:36.330+0000 7f319b000640 1 -- 172.21.15.72:0/790305329 wait complete. 2024-04-26T21:52:36.333 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:36.331+0000 7f319b000640 1 Processor -- start 2024-04-26T21:52:36.333 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:36.331+0000 7f319b000640 1 -- start start 2024-04-26T21:52:36.333 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:36.331+0000 7f319b000640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f31941031f0 0x7f319419ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:36.333 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:36.331+0000 7f319b000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3194105b20 0x7f319419b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:36.334 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:36.331+0000 7f319b000640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3194108450 0x7f31941a20f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:36.334 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:36.331+0000 7f319b000640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f319410e1f0 con 0x7f31941031f0 2024-04-26T21:52:36.334 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:36.331+0000 7f319b000640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f319410e070 con 0x7f3194108450 2024-04-26T21:52:36.334 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:36.331+0000 7f319b000640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f319410e370 con 0x7f3194105b20 2024-04-26T21:52:36.334 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:36.332+0000 7f3199c00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f31941031f0 0x7f319419ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:36.334 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:36.332+0000 7f3199c00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f31941031f0 0x7f319419ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:34992/0 (socket says 172.21.15.72:34992) 2024-04-26T21:52:36.334 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:36.332+0000 7f3199c00640 1 -- 172.21.15.72:0/3188327689 learned_addr learned my addr 172.21.15.72:0/3188327689 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:52:36.334 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:36.332+0000 7f319a600640 1 --2- 172.21.15.72:0/3188327689 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3194108450 0x7f31941a20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:36.334 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:36.332+0000 7f3199200640 1 --2- 172.21.15.72:0/3188327689 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3194105b20 0x7f319419b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:36.334 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:36.332+0000 7f3199c00640 1 -- 172.21.15.72:0/3188327689 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3194105b20 msgr2=0x7f319419b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:36.334 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:36.332+0000 7f3199c00640 1 --2- 172.21.15.72:0/3188327689 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3194105b20 0x7f319419b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:36.334 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:36.332+0000 7f3199c00640 1 -- 172.21.15.72:0/3188327689 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3194108450 msgr2=0x7f31941a20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:36.334 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:36.332+0000 7f3199c00640 1 --2- 172.21.15.72:0/3188327689 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3194108450 0x7f31941a20f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:36.334 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:36.332+0000 7f3199c00640 1 -- 172.21.15.72:0/3188327689 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31941a2760 con 0x7f31941031f0 2024-04-26T21:52:36.335 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:36.332+0000 7f3199c00640 1 --2- 172.21.15.72:0/3188327689 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f31941031f0 0x7f319419ab40 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f318800c890 tx=0x7f318800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:36.335 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:36.333+0000 7f3192000640 1 -- 172.21.15.72:0/3188327689 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3188004220 con 0x7f31941031f0 2024-04-26T21:52:36.335 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:36.333+0000 7f3192000640 1 -- 172.21.15.72:0/3188327689 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3188007e40 con 0x7f31941031f0 2024-04-26T21:52:36.335 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:36.333+0000 7f319b000640 1 -- 172.21.15.72:0/3188327689 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31941a2a50 con 0x7f31941031f0 2024-04-26T21:52:36.335 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:36.333+0000 7f3192000640 1 -- 172.21.15.72:0/3188327689 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3188004a00 con 0x7f31941031f0 2024-04-26T21:52:36.335 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:36.333+0000 7f319b000640 1 -- 172.21.15.72:0/3188327689 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31941a2fc0 con 0x7f31941031f0 2024-04-26T21:52:36.335 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:36.334+0000 7f319b000640 1 -- 172.21.15.72:0/3188327689 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3194109f40 con 0x7f31941031f0 2024-04-26T21:52:36.336 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:36.336+0000 7f3192000640 1 -- 172.21.15.72:0/3188327689 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f31880079b0 con 0x7f31941031f0 2024-04-26T21:52:36.338 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:36.338+0000 7f3192000640 1 --2- 172.21.15.72:0/3188327689 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f317c0a3c30 0x7f317c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:36.339 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:36.339+0000 7f3192000640 1 -- 172.21.15.72:0/3188327689 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7f31880cfd00 con 0x7f31941031f0 2024-04-26T21:52:36.339 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:36.339+0000 7f3199200640 1 --2- 172.21.15.72:0/3188327689 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f317c0a3c30 0x7f317c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:36.339 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:36.339+0000 7f3199200640 1 --2- 172.21.15.72:0/3188327689 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f317c0a3c30 0x7f317c0a60f0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f31800057c0 tx=0x7f3180000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:36.343 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:36.343+0000 7f3192000640 1 -- 172.21.15.72:0/3188327689 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f3188014030 con 0x7f31941031f0 2024-04-26T21:52:36.480 INFO:journalctl@ceph.mds.g.smithi072.stdout:Apr 26 21:52:36 smithi072 podman[70149]: 2024-04-26 21:52:36.10403044 +0000 UTC m=+0.038313483 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-26T21:52:36.480 INFO:journalctl@ceph.mds.g.smithi072.stdout:Apr 26 21:52:36 smithi072 podman[70149]: 2024-04-26 21:52:36.20350235 +0000 UTC m=+0.137785459 container create e193c142ef16960efaad05391a0c095bb14cfe94819f4a28dd03555c6535a50f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-mds-g, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-04-26T21:52:36.480 INFO:journalctl@ceph.mds.g.smithi072.stdout:Apr 26 21:52:36 smithi072 podman[70149]: 2024-04-26 21:52:36.479147432 +0000 UTC m=+0.413430569 container init e193c142ef16960efaad05391a0c095bb14cfe94819f4a28dd03555c6535a50f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-mds-g, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-04-26T21:52:36.593 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:36.592+0000 7f319b000640 1 -- 172.21.15.72:0/3188327689 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f319402dad0 con 0x7f31941031f0 2024-04-26T21:52:36.594 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:36.594+0000 7f3192000640 1 -- 172.21.15.72:0/3188327689 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8007 (secure 0 0 0) 0x7f318809cae0 con 0x7f31941031f0 2024-04-26T21:52:36.594 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T21:52:36.594 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6835/1320381959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6834","nonce":1320381959},{"type":"v1","addr":"172.21.15.72:6835","nonce":1320381959}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6837/2142200512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6836","nonce":2142200512},{"type":"v1","addr":"172.21.15.72:6837","nonce":2142200512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24388,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6833/4236668465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":4236668465},{"type":"v1","addr":"172.21.15.164:6833","nonce":4236668465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24424,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/2391114169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":2391114169},{"type":"v1","addr":"172.21.15.164:6835","nonce":2391114169}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24442,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6837/3585829992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":3585829992},{"type":"v1","addr":"172.21.15.164:6837","nonce":3585829992}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24463,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6839/1636226820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":1636226820},{"type":"v1","addr":"172.21.15.164:6839","nonce":1636226820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":34406,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6833/3869483750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":3869483750},{"type":"v1","addr":"172.21.15.76:6833","nonce":3869483750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34442,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/3517429435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":3517429435},{"type":"v1","addr":"172.21.15.76:6835","nonce":3517429435}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":34472,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6837/1150678991","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":1150678991},{"type":"v1","addr":"172.21.15.76:6837","nonce":1150678991}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-04-26T21:52:36.595 INFO:teuthology.orchestra.run.smithi072.stderr:dumped fsmap epoch 10 2024-04-26T21:52:36.596 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:36.596+0000 7f319b000640 1 -- 172.21.15.72:0/3188327689 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f317c0a3c30 msgr2=0x7f317c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:36.597 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:36.596+0000 7f319b000640 1 --2- 172.21.15.72:0/3188327689 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f317c0a3c30 0x7f317c0a60f0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f31800057c0 tx=0x7f3180000bc0 comp rx=0 tx=0).stop 2024-04-26T21:52:36.597 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:36.596+0000 7f319b000640 1 -- 172.21.15.72:0/3188327689 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f31941031f0 msgr2=0x7f319419ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:36.597 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:36.596+0000 7f319b000640 1 --2- 172.21.15.72:0/3188327689 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f31941031f0 0x7f319419ab40 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f318800c890 tx=0x7f318800cd50 comp rx=0 tx=0).stop 2024-04-26T21:52:36.597 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:36.597+0000 7f319b000640 1 -- 172.21.15.72:0/3188327689 shutdown_connections 2024-04-26T21:52:36.597 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:36.597+0000 7f319b000640 1 --2- 172.21.15.72:0/3188327689 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3194108450 0x7f31941a20f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:36.597 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:36.597+0000 7f319b000640 1 --2- 172.21.15.72:0/3188327689 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f317c0a3c30 0x7f317c0a60f0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:36.598 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:36.597+0000 7f319b000640 1 --2- 172.21.15.72:0/3188327689 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3194105b20 0x7f319419b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:36.598 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:36.597+0000 7f319b000640 1 --2- 172.21.15.72:0/3188327689 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f31941031f0 0x7f319419ab40 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:36.598 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:36.597+0000 7f319b000640 1 -- 172.21.15.72:0/3188327689 >> 172.21.15.72:0/3188327689 conn(0x7f31940fcbc0 msgr2=0x7f3194102a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:36.598 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:36.597+0000 7f319b000640 1 -- 172.21.15.72:0/3188327689 shutdown_connections 2024-04-26T21:52:36.598 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:36.597+0000 7f319b000640 1 -- 172.21.15.72:0/3188327689 wait complete. 2024-04-26T21:52:36.822 INFO:journalctl@ceph.mds.g.smithi072.stdout:Apr 26 21:52:36 smithi072 podman[70149]: 2024-04-26 21:52:36.488165803 +0000 UTC m=+0.422448846 container start e193c142ef16960efaad05391a0c095bb14cfe94819f4a28dd03555c6535a50f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-mds-g, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-04-26T21:52:36.822 INFO:journalctl@ceph.mds.g.smithi072.stdout:Apr 26 21:52:36 smithi072 ceph-mds[70207]: -- 172.21.15.72:0/3681391393 <== mon.0 v2:172.21.15.72:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x565170c37a40 con 0x565170c0a800 2024-04-26T21:52:36.823 INFO:journalctl@ceph.mds.g.smithi072.stdout:Apr 26 21:52:36 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-mds-g[70185]: starting mds.g at 2024-04-26T21:52:36.823 INFO:journalctl@ceph.mds.g.smithi072.stdout:Apr 26 21:52:36 smithi072 bash[70149]: e193c142ef16960efaad05391a0c095bb14cfe94819f4a28dd03555c6535a50f 2024-04-26T21:52:36.823 INFO:journalctl@ceph.mds.g.smithi072.stdout:Apr 26 21:52:36 smithi072 systemd[1]: Started Ceph mds.g for a1780950-0415-11ef-bc93-c7b262605968. 2024-04-26T21:52:37.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:36 smithi164 ceph-mon[31916]: pgmap v356: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:52:37.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:36 smithi164 ceph-mon[31916]: from='client.? 172.21.15.72:0/3188327689' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:52:37.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:36 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:52:37.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:36 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:52:37.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:36 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:52:37.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:36 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:52:37.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:36 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:52:37.192 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:36 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:52:37.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:36 smithi076 ceph-mon[31823]: pgmap v356: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:52:37.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:36 smithi076 ceph-mon[31823]: from='client.? 172.21.15.72:0/3188327689' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:52:37.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:36 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:52:37.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:36 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:52:37.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:36 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:52:37.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:36 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:52:37.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:36 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:52:37.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:36 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:52:37.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:36 smithi072 ceph-mon[28138]: pgmap v356: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:52:37.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:36 smithi072 ceph-mon[28138]: from='client.? 172.21.15.72:0/3188327689' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:52:37.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:36 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:52:37.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:36 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:52:37.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:36 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:52:37.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:36 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:52:37.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:36 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:52:37.323 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:36 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:52:38.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:37 smithi076 ceph-mon[31823]: Deploying daemon mds.a on smithi072 2024-04-26T21:52:38.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:37 smithi076 ceph-mon[31823]: mds.? [v2:172.21.15.72:6838/1312898281,v1:172.21.15.72:6839/1312898281] up:boot 2024-04-26T21:52:38.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:37 smithi076 ceph-mon[31823]: fsmap 10 up:standby 2024-04-26T21:52:38.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:37 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-26T21:52:38.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:37 smithi072 ceph-mon[28138]: Deploying daemon mds.a on smithi072 2024-04-26T21:52:38.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:37 smithi072 ceph-mon[28138]: mds.? [v2:172.21.15.72:6838/1312898281,v1:172.21.15.72:6839/1312898281] up:boot 2024-04-26T21:52:38.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:37 smithi072 ceph-mon[28138]: fsmap 10 up:standby 2024-04-26T21:52:38.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:37 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-26T21:52:38.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:37 smithi164 ceph-mon[31916]: Deploying daemon mds.a on smithi072 2024-04-26T21:52:38.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:37 smithi164 ceph-mon[31916]: mds.? [v2:172.21.15.72:6838/1312898281,v1:172.21.15.72:6839/1312898281] up:boot 2024-04-26T21:52:38.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:37 smithi164 ceph-mon[31916]: fsmap 10 up:standby 2024-04-26T21:52:38.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:37 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-26T21:52:38.617 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:52:38.821 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:38.820+0000 7f486f200640 1 -- 172.21.15.72:0/3657030098 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4868104b70 msgr2=0x7f4868104f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:38.821 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:38.820+0000 7f486f200640 1 --2- 172.21.15.72:0/3657030098 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4868104b70 0x7f4868104f90 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f4858009900 tx=0x7f485802a410 comp rx=0 tx=0).stop 2024-04-26T21:52:38.821 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:38.820+0000 7f486f200640 1 -- 172.21.15.72:0/3657030098 shutdown_connections 2024-04-26T21:52:38.821 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:38.820+0000 7f486f200640 1 --2- 172.21.15.72:0/3657030098 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4868106720 0x7f486810cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:38.821 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:38.820+0000 7f486f200640 1 --2- 172.21.15.72:0/3657030098 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4868105d60 0x7f48681061e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:38.821 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:38.820+0000 7f486f200640 1 --2- 172.21.15.72:0/3657030098 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4868104b70 0x7f4868104f90 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:38.821 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:38.820+0000 7f486f200640 1 -- 172.21.15.72:0/3657030098 >> 172.21.15.72:0/3657030098 conn(0x7f4868100120 msgr2=0x7f4868102540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:38.821 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:38.820+0000 7f486f200640 1 -- 172.21.15.72:0/3657030098 shutdown_connections 2024-04-26T21:52:38.821 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:38.820+0000 7f486f200640 1 -- 172.21.15.72:0/3657030098 wait complete. 2024-04-26T21:52:38.824 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:38.821+0000 7f486f200640 1 Processor -- start 2024-04-26T21:52:38.824 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:38.821+0000 7f486f200640 1 -- start start 2024-04-26T21:52:38.824 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:38.822+0000 7f486f200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4868104b70 0x7f486819ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:38.824 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:38.822+0000 7f486f200640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4868105d60 0x7f486819b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:38.824 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:38.822+0000 7f486f200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4868106720 0x7f48681a2270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:38.824 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:38.822+0000 7f486f200640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f4868110550 con 0x7f4868105d60 2024-04-26T21:52:38.824 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:38.822+0000 7f486f200640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f48681103d0 con 0x7f4868106720 2024-04-26T21:52:38.824 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:38.822+0000 7f486f200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f48681106d0 con 0x7f4868104b70 2024-04-26T21:52:38.824 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:38.822+0000 7f4867400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4868105d60 0x7f486819b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:38.824 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:38.822+0000 7f4867400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4868105d60 0x7f486819b240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:35038/0 (socket says 172.21.15.72:35038) 2024-04-26T21:52:38.824 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:38.822+0000 7f4867400640 1 -- 172.21.15.72:0/2500885141 learned_addr learned my addr 172.21.15.72:0/2500885141 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:52:38.824 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:38.822+0000 7f4867e00640 1 --2- 172.21.15.72:0/2500885141 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4868104b70 0x7f486819ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:38.825 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:38.822+0000 7f486d200640 1 --2- 172.21.15.72:0/2500885141 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4868106720 0x7f48681a2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:38.825 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:38.823+0000 7f4867400640 1 -- 172.21.15.72:0/2500885141 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4868104b70 msgr2=0x7f486819ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:38.825 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:38.823+0000 7f4867400640 1 --2- 172.21.15.72:0/2500885141 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4868104b70 0x7f486819ad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:38.825 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:38.823+0000 7f4867400640 1 -- 172.21.15.72:0/2500885141 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4868106720 msgr2=0x7f48681a2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:38.825 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:38.823+0000 7f4867400640 1 --2- 172.21.15.72:0/2500885141 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4868106720 0x7f48681a2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:38.825 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:38.823+0000 7f4867400640 1 -- 172.21.15.72:0/2500885141 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48681a2910 con 0x7f4868105d60 2024-04-26T21:52:38.825 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:38.823+0000 7f4867400640 1 --2- 172.21.15.72:0/2500885141 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4868105d60 0x7f486819b240 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f485400d950 tx=0x7f485400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:38.825 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:38.823+0000 7f4864c00640 1 -- 172.21.15.72:0/2500885141 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4854017070 con 0x7f4868105d60 2024-04-26T21:52:38.825 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:38.823+0000 7f486f200640 1 -- 172.21.15.72:0/2500885141 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48681a2c00 con 0x7f4868105d60 2024-04-26T21:52:38.825 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:38.823+0000 7f4864c00640 1 -- 172.21.15.72:0/2500885141 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4854009c30 con 0x7f4868105d60 2024-04-26T21:52:38.825 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:38.823+0000 7f4864c00640 1 -- 172.21.15.72:0/2500885141 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4854020060 con 0x7f4868105d60 2024-04-26T21:52:38.825 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:38.823+0000 7f486f200640 1 -- 172.21.15.72:0/2500885141 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48681a3150 con 0x7f4868105d60 2024-04-26T21:52:38.826 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:38.825+0000 7f486f200640 1 -- 172.21.15.72:0/2500885141 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f486810bf50 con 0x7f4868105d60 2024-04-26T21:52:38.828 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:38.827+0000 7f4864c00640 1 -- 172.21.15.72:0/2500885141 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f4854010460 con 0x7f4868105d60 2024-04-26T21:52:38.829 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:38.829+0000 7f4864c00640 1 --2- 172.21.15.72:0/2500885141 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f484c0a3bb0 0x7f484c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:38.829 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:38.829+0000 7f4864c00640 1 -- 172.21.15.72:0/2500885141 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7f48540cfe80 con 0x7f4868105d60 2024-04-26T21:52:38.829 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:38.829+0000 7f4867e00640 1 --2- 172.21.15.72:0/2500885141 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f484c0a3bb0 0x7f484c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:38.830 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:38.830+0000 7f4867e00640 1 --2- 172.21.15.72:0/2500885141 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f484c0a3bb0 0x7f484c0a6070 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f48580057c0 tx=0x7f48580095f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:38.834 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:38.834+0000 7f4864c00640 1 -- 172.21.15.72:0/2500885141 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f485409cc60 con 0x7f4868105d60 2024-04-26T21:52:39.064 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:38 smithi076 ceph-mon[31823]: pgmap v357: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:52:39.087 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:39.086+0000 7f486f200640 1 -- 172.21.15.72:0/2500885141 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f48680635d0 con 0x7f4868105d60 2024-04-26T21:52:39.088 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:39.088+0000 7f4864c00640 1 -- 172.21.15.72:0/2500885141 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8833 (secure 0 0 0) 0x7f4854009de0 con 0x7f4868105d60 2024-04-26T21:52:39.088 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T21:52:39.088 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6835/1320381959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6834","nonce":1320381959},{"type":"v1","addr":"172.21.15.72:6835","nonce":1320381959}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6837/2142200512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6836","nonce":2142200512},{"type":"v1","addr":"172.21.15.72:6837","nonce":2142200512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14652,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6839/1312898281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6838","nonce":1312898281},{"type":"v1","addr":"172.21.15.72:6839","nonce":1312898281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24388,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6833/4236668465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":4236668465},{"type":"v1","addr":"172.21.15.164:6833","nonce":4236668465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24424,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/2391114169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":2391114169},{"type":"v1","addr":"172.21.15.164:6835","nonce":2391114169}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24442,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6837/3585829992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":3585829992},{"type":"v1","addr":"172.21.15.164:6837","nonce":3585829992}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24463,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6839/1636226820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":1636226820},{"type":"v1","addr":"172.21.15.164:6839","nonce":1636226820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":34406,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6833/3869483750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":3869483750},{"type":"v1","addr":"172.21.15.76:6833","nonce":3869483750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34442,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/3517429435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":3517429435},{"type":"v1","addr":"172.21.15.76:6835","nonce":3517429435}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":34472,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6837/1150678991","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":1150678991},{"type":"v1","addr":"172.21.15.76:6837","nonce":1150678991}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-04-26T21:52:39.089 INFO:teuthology.orchestra.run.smithi072.stderr:dumped fsmap epoch 11 2024-04-26T21:52:39.091 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:39.090+0000 7f486f200640 1 -- 172.21.15.72:0/2500885141 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f484c0a3bb0 msgr2=0x7f484c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:39.091 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:39.090+0000 7f486f200640 1 --2- 172.21.15.72:0/2500885141 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f484c0a3bb0 0x7f484c0a6070 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f48580057c0 tx=0x7f48580095f0 comp rx=0 tx=0).stop 2024-04-26T21:52:39.091 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:39.090+0000 7f486f200640 1 -- 172.21.15.72:0/2500885141 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4868105d60 msgr2=0x7f486819b240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:39.092 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:39.090+0000 7f486f200640 1 --2- 172.21.15.72:0/2500885141 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4868105d60 0x7f486819b240 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f485400d950 tx=0x7f485400de10 comp rx=0 tx=0).stop 2024-04-26T21:52:39.092 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:39.091+0000 7f486f200640 1 -- 172.21.15.72:0/2500885141 shutdown_connections 2024-04-26T21:52:39.092 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:39.091+0000 7f486f200640 1 --2- 172.21.15.72:0/2500885141 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4868106720 0x7f48681a2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:39.092 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:39.091+0000 7f486f200640 1 --2- 172.21.15.72:0/2500885141 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4868105d60 0x7f486819b240 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:39.092 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:39.091+0000 7f486f200640 1 --2- 172.21.15.72:0/2500885141 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f484c0a3bb0 0x7f484c0a6070 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:39.092 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:39.091+0000 7f486f200640 1 --2- 172.21.15.72:0/2500885141 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4868104b70 0x7f486819ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:39.092 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:39.091+0000 7f486f200640 1 -- 172.21.15.72:0/2500885141 >> 172.21.15.72:0/2500885141 conn(0x7f4868100120 msgr2=0x7f4868109120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:39.092 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:39.091+0000 7f486f200640 1 -- 172.21.15.72:0/2500885141 shutdown_connections 2024-04-26T21:52:39.092 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:39.091+0000 7f486f200640 1 -- 172.21.15.72:0/2500885141 wait complete. 2024-04-26T21:52:39.321 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:38 smithi072 ceph-mon[28138]: pgmap v357: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:52:39.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:38 smithi164 ceph-mon[31916]: pgmap v357: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:52:40.305 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:39 smithi072 ceph-mon[28138]: from='client.? 172.21.15.72:0/2500885141' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:52:40.305 INFO:journalctl@ceph.mds.a.smithi072.stdout:Apr 26 21:52:39 smithi072 systemd[1]: Starting Ceph mds.a for a1780950-0415-11ef-bc93-c7b262605968... 2024-04-26T21:52:40.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:39 smithi076 ceph-mon[31823]: from='client.? 172.21.15.72:0/2500885141' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:52:40.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:39 smithi164 ceph-mon[31916]: from='client.? 172.21.15.72:0/2500885141' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:52:40.573 INFO:journalctl@ceph.mds.a.smithi072.stdout:Apr 26 21:52:40 smithi072 podman[70534]: 2024-04-26 21:52:40.209637802 +0000 UTC m=+0.033187772 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-26T21:52:40.573 INFO:journalctl@ceph.mds.a.smithi072.stdout:Apr 26 21:52:40 smithi072 podman[70534]: 2024-04-26 21:52:40.340988616 +0000 UTC m=+0.164538623 container create fb63d88d4bc44337f9e673b040c52b6bd3b30eeeb0b347ddd09d985b0945f214 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-mds-a, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240424) 2024-04-26T21:52:40.966 INFO:journalctl@ceph.mds.a.smithi072.stdout:Apr 26 21:52:40 smithi072 podman[70534]: 2024-04-26 21:52:40.607650282 +0000 UTC m=+0.431200366 container init fb63d88d4bc44337f9e673b040c52b6bd3b30eeeb0b347ddd09d985b0945f214 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-mds-a, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2) 2024-04-26T21:52:40.966 INFO:journalctl@ceph.mds.a.smithi072.stdout:Apr 26 21:52:40 smithi072 podman[70534]: 2024-04-26 21:52:40.617673957 +0000 UTC m=+0.441223970 container start fb63d88d4bc44337f9e673b040c52b6bd3b30eeeb0b347ddd09d985b0945f214 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-mds-a, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T21:52:40.966 INFO:journalctl@ceph.mds.a.smithi072.stdout:Apr 26 21:52:40 smithi072 ceph-mds[70553]: -- 172.21.15.72:0/2960016027 <== mon.0 v2:172.21.15.72:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5639da6d9a40 con 0x5639da6ad000 2024-04-26T21:52:40.966 INFO:journalctl@ceph.mds.a.smithi072.stdout:Apr 26 21:52:40 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-mds-a[70549]: starting mds.a at 2024-04-26T21:52:40.966 INFO:journalctl@ceph.mds.a.smithi072.stdout:Apr 26 21:52:40 smithi072 bash[70534]: fb63d88d4bc44337f9e673b040c52b6bd3b30eeeb0b347ddd09d985b0945f214 2024-04-26T21:52:40.966 INFO:journalctl@ceph.mds.a.smithi072.stdout:Apr 26 21:52:40 smithi072 systemd[1]: Started Ceph mds.a for a1780950-0415-11ef-bc93-c7b262605968. 2024-04-26T21:52:41.112 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:52:41.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:40 smithi076 ceph-mon[31823]: pgmap v358: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:52:41.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:40 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:52:41.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:40 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:52:41.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:40 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:52:41.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:40 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:52:41.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:40 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:52:41.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:40 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:52:41.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:40 smithi072 ceph-mon[28138]: pgmap v358: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:52:41.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:40 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:52:41.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:40 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:52:41.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:40 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:52:41.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:40 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:52:41.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:40 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:52:41.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:40 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:52:41.329 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:41.328+0000 7f0425200640 1 -- 172.21.15.72:0/2010556825 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0420075390 msgr2=0x7f0420075810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:41.329 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:41.328+0000 7f0425200640 1 --2- 172.21.15.72:0/2010556825 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0420075390 0x7f0420075810 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f040c009930 tx=0x7f040c02a420 comp rx=0 tx=0).stop 2024-04-26T21:52:41.329 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:41.329+0000 7f0425200640 1 -- 172.21.15.72:0/2010556825 shutdown_connections 2024-04-26T21:52:41.329 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:41.329+0000 7f0425200640 1 --2- 172.21.15.72:0/2010556825 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f042010ed10 0x7f0420111100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:41.329 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:41.329+0000 7f0425200640 1 --2- 172.21.15.72:0/2010556825 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0420075390 0x7f0420075810 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:41.329 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:41.329+0000 7f0425200640 1 --2- 172.21.15.72:0/2010556825 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0420076970 0x7f0420074dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:41.329 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:41.329+0000 7f0425200640 1 -- 172.21.15.72:0/2010556825 >> 172.21.15.72:0/2010556825 conn(0x7f04200fdcb0 msgr2=0x7f04201000f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:41.330 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:41.329+0000 7f0425200640 1 -- 172.21.15.72:0/2010556825 shutdown_connections 2024-04-26T21:52:41.330 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:41.329+0000 7f0425200640 1 -- 172.21.15.72:0/2010556825 wait complete. 2024-04-26T21:52:41.332 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:41.330+0000 7f0425200640 1 Processor -- start 2024-04-26T21:52:41.332 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:41.330+0000 7f0425200640 1 -- start start 2024-04-26T21:52:41.332 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:41.330+0000 7f0425200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0420075390 0x7f042019ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:41.332 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:41.330+0000 7f0425200640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0420076970 0x7f042019f460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:41.332 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:41.330+0000 7f0425200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f042010ed10 0x7f04201a64d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:41.332 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:41.330+0000 7f0425200640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f04201147c0 con 0x7f0420076970 2024-04-26T21:52:41.332 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:41.330+0000 7f0425200640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f0420114640 con 0x7f0420075390 2024-04-26T21:52:41.332 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:41.330+0000 7f0425200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f0420114940 con 0x7f042010ed10 2024-04-26T21:52:41.332 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:41.330+0000 7f041ea00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0420076970 0x7f042019f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:41.332 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:41.330+0000 7f041ea00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0420076970 0x7f042019f460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:45158/0 (socket says 172.21.15.72:45158) 2024-04-26T21:52:41.332 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:41.330+0000 7f041ea00640 1 -- 172.21.15.72:0/454124511 learned_addr learned my addr 172.21.15.72:0/454124511 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:52:41.332 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:41.330+0000 7f041f400640 1 --2- 172.21.15.72:0/454124511 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0420075390 0x7f042019ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:41.333 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:41.331+0000 7f041fe00640 1 --2- 172.21.15.72:0/454124511 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f042010ed10 0x7f04201a64d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:41.333 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:41.331+0000 7f041ea00640 1 -- 172.21.15.72:0/454124511 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f042010ed10 msgr2=0x7f04201a64d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:41.333 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:41.331+0000 7f041ea00640 1 --2- 172.21.15.72:0/454124511 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f042010ed10 0x7f04201a64d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:41.333 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:41.331+0000 7f041ea00640 1 -- 172.21.15.72:0/454124511 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0420075390 msgr2=0x7f042019ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:41.333 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:41.331+0000 7f041ea00640 1 --2- 172.21.15.72:0/454124511 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0420075390 0x7f042019ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:41.333 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:41.331+0000 7f041ea00640 1 -- 172.21.15.72:0/454124511 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04201a6b40 con 0x7f0420076970 2024-04-26T21:52:41.333 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:41.331+0000 7f041ea00640 1 --2- 172.21.15.72:0/454124511 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0420076970 0x7f042019f460 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f040c031460 tx=0x7f040c003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:41.333 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:41.331+0000 7f0417e00640 1 -- 172.21.15.72:0/454124511 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f040c03f070 con 0x7f0420076970 2024-04-26T21:52:41.333 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:41.331+0000 7f0417e00640 1 -- 172.21.15.72:0/454124511 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f040c031da0 con 0x7f0420076970 2024-04-26T21:52:41.333 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:41.331+0000 7f0425200640 1 -- 172.21.15.72:0/454124511 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04201a6dd0 con 0x7f0420076970 2024-04-26T21:52:41.333 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:41.331+0000 7f0417e00640 1 -- 172.21.15.72:0/454124511 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f040c035030 con 0x7f0420076970 2024-04-26T21:52:41.333 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:41.332+0000 7f0425200640 1 -- 172.21.15.72:0/454124511 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04201a72e0 con 0x7f0420076970 2024-04-26T21:52:41.334 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:41.333+0000 7f0425200640 1 -- 172.21.15.72:0/454124511 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0420104ee0 con 0x7f0420076970 2024-04-26T21:52:41.334 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:41.335+0000 7f0417e00640 1 -- 172.21.15.72:0/454124511 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f040c042070 con 0x7f0420076970 2024-04-26T21:52:41.337 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:41.337+0000 7f0417e00640 1 --2- 172.21.15.72:0/454124511 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f04040a3bb0 0x7f04040a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:41.337 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:41.337+0000 7f0417e00640 1 -- 172.21.15.72:0/454124511 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7f040c0f0950 con 0x7f0420076970 2024-04-26T21:52:41.337 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:41.337+0000 7f041f400640 1 --2- 172.21.15.72:0/454124511 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f04040a3bb0 0x7f04040a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:41.338 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:41.338+0000 7f041f400640 1 --2- 172.21.15.72:0/454124511 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f04040a3bb0 0x7f04040a6070 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f0410009890 tx=0x7f0410008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:41.341 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:41.341+0000 7f0417e00640 1 -- 172.21.15.72:0/454124511 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f040c0bd730 con 0x7f0420076970 2024-04-26T21:52:41.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:40 smithi164 ceph-mon[31916]: pgmap v358: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:52:41.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:40 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:52:41.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:40 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:52:41.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:40 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:52:41.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:40 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:52:41.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:40 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:52:41.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:40 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:52:41.583 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:41.583+0000 7f0425200640 1 -- 172.21.15.72:0/454124511 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0420063530 con 0x7f0420076970 2024-04-26T21:52:41.584 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:41.584+0000 7f0417e00640 1 -- 172.21.15.72:0/454124511 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9659 (secure 0 0 0) 0x7f040c0bd030 con 0x7f0420076970 2024-04-26T21:52:41.585 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T21:52:41.585 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6835/1320381959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6834","nonce":1320381959},{"type":"v1","addr":"172.21.15.72:6835","nonce":1320381959}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6837/2142200512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6836","nonce":2142200512},{"type":"v1","addr":"172.21.15.72:6837","nonce":2142200512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14652,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6839/1312898281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6838","nonce":1312898281},{"type":"v1","addr":"172.21.15.72:6839","nonce":1312898281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14664,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6841/3257692339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6840","nonce":3257692339},{"type":"v1","addr":"172.21.15.72:6841","nonce":3257692339}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24388,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6833/4236668465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":4236668465},{"type":"v1","addr":"172.21.15.164:6833","nonce":4236668465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24424,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/2391114169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":2391114169},{"type":"v1","addr":"172.21.15.164:6835","nonce":2391114169}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24442,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6837/3585829992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":3585829992},{"type":"v1","addr":"172.21.15.164:6837","nonce":3585829992}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24463,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6839/1636226820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":1636226820},{"type":"v1","addr":"172.21.15.164:6839","nonce":1636226820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":34406,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6833/3869483750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":3869483750},{"type":"v1","addr":"172.21.15.76:6833","nonce":3869483750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34442,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/3517429435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":3517429435},{"type":"v1","addr":"172.21.15.76:6835","nonce":3517429435}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":34472,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6837/1150678991","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":1150678991},{"type":"v1","addr":"172.21.15.76:6837","nonce":1150678991}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-04-26T21:52:41.585 INFO:teuthology.orchestra.run.smithi072.stderr:dumped fsmap epoch 12 2024-04-26T21:52:41.588 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:41.587+0000 7f0425200640 1 -- 172.21.15.72:0/454124511 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f04040a3bb0 msgr2=0x7f04040a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:41.588 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:41.587+0000 7f0425200640 1 --2- 172.21.15.72:0/454124511 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f04040a3bb0 0x7f04040a6070 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f0410009890 tx=0x7f0410008040 comp rx=0 tx=0).stop 2024-04-26T21:52:41.588 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:41.587+0000 7f0425200640 1 -- 172.21.15.72:0/454124511 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0420076970 msgr2=0x7f042019f460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:41.588 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:41.587+0000 7f0425200640 1 --2- 172.21.15.72:0/454124511 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0420076970 0x7f042019f460 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f040c031460 tx=0x7f040c003c80 comp rx=0 tx=0).stop 2024-04-26T21:52:41.588 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:41.587+0000 7f0425200640 1 -- 172.21.15.72:0/454124511 shutdown_connections 2024-04-26T21:52:41.588 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:41.587+0000 7f0425200640 1 --2- 172.21.15.72:0/454124511 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f04040a3bb0 0x7f04040a6070 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:41.589 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:41.587+0000 7f0425200640 1 --2- 172.21.15.72:0/454124511 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f042010ed10 0x7f04201a64d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:41.589 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:41.587+0000 7f0425200640 1 --2- 172.21.15.72:0/454124511 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0420076970 0x7f042019f460 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:41.589 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:41.587+0000 7f0425200640 1 --2- 172.21.15.72:0/454124511 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0420075390 0x7f042019ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:41.589 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:41.587+0000 7f0425200640 1 -- 172.21.15.72:0/454124511 >> 172.21.15.72:0/454124511 conn(0x7f04200fdcb0 msgr2=0x7f04201020b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:41.589 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:41.588+0000 7f0425200640 1 -- 172.21.15.72:0/454124511 shutdown_connections 2024-04-26T21:52:41.589 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:41.588+0000 7f0425200640 1 -- 172.21.15.72:0/454124511 wait complete. 2024-04-26T21:52:42.127 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:41 smithi076 ceph-mon[31823]: Deploying daemon mds.b on smithi076 2024-04-26T21:52:42.127 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:41 smithi076 ceph-mon[31823]: mds.? [v2:172.21.15.72:6840/3257692339,v1:172.21.15.72:6841/3257692339] up:boot 2024-04-26T21:52:42.127 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:41 smithi076 ceph-mon[31823]: fsmap 11 up:standby 2024-04-26T21:52:42.127 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:41 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-26T21:52:42.128 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:41 smithi076 ceph-mon[31823]: from='client.? 172.21.15.72:0/454124511' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:52:42.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:41 smithi072 ceph-mon[28138]: Deploying daemon mds.b on smithi076 2024-04-26T21:52:42.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:41 smithi072 ceph-mon[28138]: mds.? [v2:172.21.15.72:6840/3257692339,v1:172.21.15.72:6841/3257692339] up:boot 2024-04-26T21:52:42.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:41 smithi072 ceph-mon[28138]: fsmap 11 up:standby 2024-04-26T21:52:42.323 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:41 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-26T21:52:42.323 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:41 smithi072 ceph-mon[28138]: from='client.? 172.21.15.72:0/454124511' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:52:42.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:41 smithi164 ceph-mon[31916]: Deploying daemon mds.b on smithi076 2024-04-26T21:52:42.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:41 smithi164 ceph-mon[31916]: mds.? [v2:172.21.15.72:6840/3257692339,v1:172.21.15.72:6841/3257692339] up:boot 2024-04-26T21:52:42.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:41 smithi164 ceph-mon[31916]: fsmap 11 up:standby 2024-04-26T21:52:42.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:41 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-26T21:52:42.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:41 smithi164 ceph-mon[31916]: from='client.? 172.21.15.72:0/454124511' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:52:43.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:42 smithi076 ceph-mon[31823]: pgmap v359: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:52:43.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:42 smithi072 ceph-mon[28138]: pgmap v359: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:52:43.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:42 smithi164 ceph-mon[31916]: pgmap v359: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:52:43.609 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:52:43.814 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:43.813+0000 7f22fea00640 1 -- 172.21.15.72:0/4101416441 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f22f810a940 msgr2=0x7f22f81111d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:43.814 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:43.813+0000 7f22fea00640 1 --2- 172.21.15.72:0/4101416441 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f22f810a940 0x7f22f81111d0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f22f0009ca0 tx=0x7f22f002a8b0 comp rx=0 tx=0).stop 2024-04-26T21:52:43.814 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:43.813+0000 7f22fea00640 1 -- 172.21.15.72:0/4101416441 shutdown_connections 2024-04-26T21:52:43.814 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:43.813+0000 7f22fea00640 1 --2- 172.21.15.72:0/4101416441 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f22f810a940 0x7f22f81111d0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:43.815 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:43.813+0000 7f22fea00640 1 --2- 172.21.15.72:0/4101416441 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f22f8109f80 0x7f22f810a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:43.815 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:43.813+0000 7f22fea00640 1 --2- 172.21.15.72:0/4101416441 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f22f8108d90 0x7f22f81091b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:43.815 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:43.813+0000 7f22fea00640 1 -- 172.21.15.72:0/4101416441 >> 172.21.15.72:0/4101416441 conn(0x7f22f8077d30 msgr2=0x7f22f807a170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:43.815 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:43.813+0000 7f22fea00640 1 -- 172.21.15.72:0/4101416441 shutdown_connections 2024-04-26T21:52:43.815 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:43.814+0000 7f22fea00640 1 -- 172.21.15.72:0/4101416441 wait complete. 2024-04-26T21:52:43.817 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:43.814+0000 7f22fea00640 1 Processor -- start 2024-04-26T21:52:43.817 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:43.815+0000 7f22fea00640 1 -- start start 2024-04-26T21:52:43.817 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:43.815+0000 7f22fea00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f22f8108d90 0x7f22f819edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:43.817 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:43.815+0000 7f22fea00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f22f8109f80 0x7f22f819f300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:43.817 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:43.815+0000 7f22fea00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f22f810a940 0x7f22f81a6370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:43.817 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:43.815+0000 7f22fea00640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f22f81147a0 con 0x7f22f8108d90 2024-04-26T21:52:43.817 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:43.815+0000 7f22fea00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f22f8114620 con 0x7f22f810a940 2024-04-26T21:52:43.817 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:43.815+0000 7f22fea00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f22f8114920 con 0x7f22f8109f80 2024-04-26T21:52:43.818 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:43.815+0000 7f22fd600640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f22f8108d90 0x7f22f819edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:43.818 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:43.815+0000 7f22fd600640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f22f8108d90 0x7f22f819edc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:45176/0 (socket says 172.21.15.72:45176) 2024-04-26T21:52:43.818 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:43.815+0000 7f22fd600640 1 -- 172.21.15.72:0/2198446996 learned_addr learned my addr 172.21.15.72:0/2198446996 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:52:43.818 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:43.815+0000 7f22fe000640 1 --2- 172.21.15.72:0/2198446996 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f22f810a940 0x7f22f81a6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:43.818 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:43.816+0000 7f22fd600640 1 -- 172.21.15.72:0/2198446996 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f22f8109f80 msgr2=0x7f22f819f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:43.818 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:43.816+0000 7f22fd600640 1 --2- 172.21.15.72:0/2198446996 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f22f8109f80 0x7f22f819f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:43.818 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:43.816+0000 7f22fd600640 1 -- 172.21.15.72:0/2198446996 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f22f810a940 msgr2=0x7f22f81a6370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:43.818 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:43.816+0000 7f22fd600640 1 --2- 172.21.15.72:0/2198446996 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f22f810a940 0x7f22f81a6370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:43.818 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:43.816+0000 7f22fd600640 1 -- 172.21.15.72:0/2198446996 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22f81a69e0 con 0x7f22f8108d90 2024-04-26T21:52:43.818 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:43.816+0000 7f22fd600640 1 --2- 172.21.15.72:0/2198446996 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f22f8108d90 0x7f22f819edc0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f22ec00d950 tx=0x7f22ec00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:43.818 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:43.816+0000 7f22f6000640 1 -- 172.21.15.72:0/2198446996 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22ec017020 con 0x7f22f8108d90 2024-04-26T21:52:43.818 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:43.816+0000 7f22f6000640 1 -- 172.21.15.72:0/2198446996 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f22ec009c30 con 0x7f22f8108d90 2024-04-26T21:52:43.818 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:43.816+0000 7f22fea00640 1 -- 172.21.15.72:0/2198446996 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22f81a6cd0 con 0x7f22f8108d90 2024-04-26T21:52:43.819 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:43.816+0000 7f22f6000640 1 -- 172.21.15.72:0/2198446996 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22ec00f620 con 0x7f22f8108d90 2024-04-26T21:52:43.819 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:43.816+0000 7f22fea00640 1 -- 172.21.15.72:0/2198446996 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22f81a7210 con 0x7f22f8108d90 2024-04-26T21:52:43.819 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:43.817+0000 7f22fea00640 1 -- 172.21.15.72:0/2198446996 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22f8110120 con 0x7f22f8108d90 2024-04-26T21:52:43.819 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:43.819+0000 7f22f6000640 1 -- 172.21.15.72:0/2198446996 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f22ec01a020 con 0x7f22f8108d90 2024-04-26T21:52:43.821 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:43.821+0000 7f22f6000640 1 --2- 172.21.15.72:0/2198446996 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f22e00a3c30 0x7f22e00a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:43.821 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:43.821+0000 7f22f6000640 1 -- 172.21.15.72:0/2198446996 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7f22ec0d02d0 con 0x7f22f8108d90 2024-04-26T21:52:43.821 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:43.821+0000 7f22fcc00640 1 --2- 172.21.15.72:0/2198446996 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f22e00a3c30 0x7f22e00a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:43.822 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:43.822+0000 7f22fcc00640 1 --2- 172.21.15.72:0/2198446996 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f22e00a3c30 0x7f22e00a60f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f22f81a02d0 tx=0x7f22e4005c00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:43.823 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:43.824+0000 7f22f6000640 1 -- 172.21.15.72:0/2198446996 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f22ec021020 con 0x7f22f8108d90 2024-04-26T21:52:44.066 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:44.066+0000 7f22fea00640 1 -- 172.21.15.72:0/2198446996 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f22f802dad0 con 0x7f22f8108d90 2024-04-26T21:52:44.068 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:44.067+0000 7f22f6000640 1 -- 172.21.15.72:0/2198446996 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9659 (secure 0 0 0) 0x7f22ec009de0 con 0x7f22f8108d90 2024-04-26T21:52:44.068 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T21:52:44.068 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6835/1320381959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6834","nonce":1320381959},{"type":"v1","addr":"172.21.15.72:6835","nonce":1320381959}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6837/2142200512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6836","nonce":2142200512},{"type":"v1","addr":"172.21.15.72:6837","nonce":2142200512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14652,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6839/1312898281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6838","nonce":1312898281},{"type":"v1","addr":"172.21.15.72:6839","nonce":1312898281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14664,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6841/3257692339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6840","nonce":3257692339},{"type":"v1","addr":"172.21.15.72:6841","nonce":3257692339}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24388,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6833/4236668465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":4236668465},{"type":"v1","addr":"172.21.15.164:6833","nonce":4236668465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24424,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/2391114169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":2391114169},{"type":"v1","addr":"172.21.15.164:6835","nonce":2391114169}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24442,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6837/3585829992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":3585829992},{"type":"v1","addr":"172.21.15.164:6837","nonce":3585829992}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24463,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6839/1636226820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":1636226820},{"type":"v1","addr":"172.21.15.164:6839","nonce":1636226820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":34406,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6833/3869483750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":3869483750},{"type":"v1","addr":"172.21.15.76:6833","nonce":3869483750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34442,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/3517429435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":3517429435},{"type":"v1","addr":"172.21.15.76:6835","nonce":3517429435}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":34472,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6837/1150678991","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":1150678991},{"type":"v1","addr":"172.21.15.76:6837","nonce":1150678991}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-04-26T21:52:44.068 INFO:teuthology.orchestra.run.smithi072.stderr:dumped fsmap epoch 12 2024-04-26T21:52:44.071 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:44.070+0000 7f22fea00640 1 -- 172.21.15.72:0/2198446996 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f22e00a3c30 msgr2=0x7f22e00a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:44.071 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:44.070+0000 7f22fea00640 1 --2- 172.21.15.72:0/2198446996 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f22e00a3c30 0x7f22e00a60f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f22f81a02d0 tx=0x7f22e4005c00 comp rx=0 tx=0).stop 2024-04-26T21:52:44.071 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:44.070+0000 7f22fea00640 1 -- 172.21.15.72:0/2198446996 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f22f8108d90 msgr2=0x7f22f819edc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:44.071 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:44.070+0000 7f22fea00640 1 --2- 172.21.15.72:0/2198446996 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f22f8108d90 0x7f22f819edc0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f22ec00d950 tx=0x7f22ec00de10 comp rx=0 tx=0).stop 2024-04-26T21:52:44.071 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:44.070+0000 7f22fea00640 1 -- 172.21.15.72:0/2198446996 shutdown_connections 2024-04-26T21:52:44.071 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:44.070+0000 7f22fea00640 1 --2- 172.21.15.72:0/2198446996 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f22f810a940 0x7f22f81a6370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:44.071 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:44.070+0000 7f22fea00640 1 --2- 172.21.15.72:0/2198446996 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f22e00a3c30 0x7f22e00a60f0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:44.071 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:44.070+0000 7f22fea00640 1 --2- 172.21.15.72:0/2198446996 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f22f8109f80 0x7f22f819f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:44.071 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:44.070+0000 7f22fea00640 1 --2- 172.21.15.72:0/2198446996 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f22f8108d90 0x7f22f819edc0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:44.071 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:44.070+0000 7f22fea00640 1 -- 172.21.15.72:0/2198446996 >> 172.21.15.72:0/2198446996 conn(0x7f22f8077d30 msgr2=0x7f22f810d2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:44.071 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:44.070+0000 7f22fea00640 1 -- 172.21.15.72:0/2198446996 shutdown_connections 2024-04-26T21:52:44.071 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:44.070+0000 7f22fea00640 1 -- 172.21.15.72:0/2198446996 wait complete. 2024-04-26T21:52:44.445 INFO:journalctl@ceph.mds.b.smithi076.stdout:Apr 26 21:52:44 smithi076 systemd[1]: Starting Ceph mds.b for a1780950-0415-11ef-bc93-c7b262605968... 2024-04-26T21:52:44.763 INFO:journalctl@ceph.mds.b.smithi076.stdout:Apr 26 21:52:44 smithi076 podman[61238]: 2024-04-26 21:52:44.3494446 +0000 UTC m=+0.035028822 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-26T21:52:44.764 INFO:journalctl@ceph.mds.b.smithi076.stdout:Apr 26 21:52:44 smithi076 podman[61238]: 2024-04-26 21:52:44.474021193 +0000 UTC m=+0.159605452 container create 3e314c4fe51de52f0d1edcd98ca3f8fb2cce235400c8ba2daebce9d756a10eb7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-mds-b, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T21:52:44.764 INFO:journalctl@ceph.mds.b.smithi076.stdout:Apr 26 21:52:44 smithi076 podman[61238]: 2024-04-26 21:52:44.76209633 +0000 UTC m=+0.447680596 container init 3e314c4fe51de52f0d1edcd98ca3f8fb2cce235400c8ba2daebce9d756a10eb7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-mds-b, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424) 2024-04-26T21:52:45.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:44 smithi076 ceph-mon[31823]: pgmap v360: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:52:45.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:44 smithi076 ceph-mon[31823]: from='client.? 172.21.15.72:0/2198446996' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:52:45.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:44 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:52:45.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:44 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:52:45.065 INFO:journalctl@ceph.mds.b.smithi076.stdout:Apr 26 21:52:44 smithi076 podman[61238]: 2024-04-26 21:52:44.771878554 +0000 UTC m=+0.457462810 container start 3e314c4fe51de52f0d1edcd98ca3f8fb2cce235400c8ba2daebce9d756a10eb7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-mds-b, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2) 2024-04-26T21:52:45.065 INFO:journalctl@ceph.mds.b.smithi076.stdout:Apr 26 21:52:44 smithi076 ceph-mds[61257]: -- 172.21.15.76:0/3428006731 <== mon.2 v2:172.21.15.76:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563fda0afa40 con 0x563fda083000 2024-04-26T21:52:45.066 INFO:journalctl@ceph.mds.b.smithi076.stdout:Apr 26 21:52:44 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-mds-b[61253]: starting mds.b at 2024-04-26T21:52:45.066 INFO:journalctl@ceph.mds.b.smithi076.stdout:Apr 26 21:52:44 smithi076 bash[61238]: 3e314c4fe51de52f0d1edcd98ca3f8fb2cce235400c8ba2daebce9d756a10eb7 2024-04-26T21:52:45.066 INFO:journalctl@ceph.mds.b.smithi076.stdout:Apr 26 21:52:44 smithi076 systemd[1]: Started Ceph mds.b for a1780950-0415-11ef-bc93-c7b262605968. 2024-04-26T21:52:45.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:44 smithi072 ceph-mon[28138]: pgmap v360: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:52:45.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:44 smithi072 ceph-mon[28138]: from='client.? 172.21.15.72:0/2198446996' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:52:45.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:44 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:52:45.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:44 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:52:45.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:44 smithi164 ceph-mon[31916]: pgmap v360: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:52:45.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:44 smithi164 ceph-mon[31916]: from='client.? 172.21.15.72:0/2198446996' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:52:45.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:44 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:52:45.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:44 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:52:46.089 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:52:46.256 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:45 smithi072 ceph-mon[28138]: mds.? [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] up:boot 2024-04-26T21:52:46.256 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:46 smithi072 ceph-mon[28138]: fsmap 12 up:standby 2024-04-26T21:52:46.256 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:46 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-26T21:52:46.256 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:46 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:52:46.257 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:46 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:52:46.257 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:46 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:52:46.257 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:46 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:52:46.257 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:46 smithi072 ceph-mon[28138]: pgmap v361: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:52:46.282 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:46.282+0000 7f93f0200640 1 -- 172.21.15.72:0/138664518 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f93e8104520 msgr2=0x7f93e810adb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:46.283 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:46.282+0000 7f93f0200640 1 --2- 172.21.15.72:0/138664518 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f93e8104520 0x7f93e810adb0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f93e0009ca0 tx=0x7f93e002a8c0 comp rx=0 tx=0).stop 2024-04-26T21:52:46.283 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:46.282+0000 7f93f0200640 1 -- 172.21.15.72:0/138664518 shutdown_connections 2024-04-26T21:52:46.283 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:46.282+0000 7f93f0200640 1 --2- 172.21.15.72:0/138664518 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f93e8104520 0x7f93e810adb0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:46.283 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:46.282+0000 7f93f0200640 1 --2- 172.21.15.72:0/138664518 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f93e8103b60 0x7f93e8103fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:46.283 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:46.282+0000 7f93f0200640 1 --2- 172.21.15.72:0/138664518 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f93e8102970 0x7f93e8102d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:46.283 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:46.282+0000 7f93f0200640 1 -- 172.21.15.72:0/138664518 >> 172.21.15.72:0/138664518 conn(0x7f93e80fdee0 msgr2=0x7f93e8100320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:46.283 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:46.282+0000 7f93f0200640 1 -- 172.21.15.72:0/138664518 shutdown_connections 2024-04-26T21:52:46.283 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:46.282+0000 7f93f0200640 1 -- 172.21.15.72:0/138664518 wait complete. 2024-04-26T21:52:46.285 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:46.283+0000 7f93f0200640 1 Processor -- start 2024-04-26T21:52:46.285 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:46.283+0000 7f93f0200640 1 -- start start 2024-04-26T21:52:46.285 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:46.283+0000 7f93f0200640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f93e8102970 0x7f93e819ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:46.285 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:46.283+0000 7f93f0200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f93e8103b60 0x7f93e819b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:46.285 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:46.283+0000 7f93f0200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f93e8104520 0x7f93e81a21b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:46.286 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:46.283+0000 7f93f0200640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f93e810e380 con 0x7f93e8102970 2024-04-26T21:52:46.286 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:46.283+0000 7f93f0200640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f93e810e200 con 0x7f93e8103b60 2024-04-26T21:52:46.286 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:46.283+0000 7f93f0200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f93e810e500 con 0x7f93e8104520 2024-04-26T21:52:46.286 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:46.283+0000 7f93ed800640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f93e8102970 0x7f93e819ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:46.286 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:46.284+0000 7f93ed800640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f93e8102970 0x7f93e819ac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:45192/0 (socket says 172.21.15.72:45192) 2024-04-26T21:52:46.286 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:46.284+0000 7f93ed800640 1 -- 172.21.15.72:0/2147400083 learned_addr learned my addr 172.21.15.72:0/2147400083 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:52:46.286 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:46.284+0000 7f93ece00640 1 --2- 172.21.15.72:0/2147400083 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f93e8103b60 0x7f93e819b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:46.286 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:46.284+0000 7f93ee200640 1 --2- 172.21.15.72:0/2147400083 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f93e8104520 0x7f93e81a21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:46.286 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:46.284+0000 7f93ed800640 1 -- 172.21.15.72:0/2147400083 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f93e8104520 msgr2=0x7f93e81a21b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:46.286 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:46.284+0000 7f93ed800640 1 --2- 172.21.15.72:0/2147400083 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f93e8104520 0x7f93e81a21b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:46.286 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:46.284+0000 7f93ed800640 1 -- 172.21.15.72:0/2147400083 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f93e8103b60 msgr2=0x7f93e819b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:46.286 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:46.284+0000 7f93ed800640 1 --2- 172.21.15.72:0/2147400083 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f93e8103b60 0x7f93e819b180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:46.286 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:46.284+0000 7f93ed800640 1 -- 172.21.15.72:0/2147400083 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93e81a2850 con 0x7f93e8102970 2024-04-26T21:52:46.287 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:46.284+0000 7f93ed800640 1 --2- 172.21.15.72:0/2147400083 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f93e8102970 0x7f93e819ac40 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f93dc00e9c0 tx=0x7f93dc00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:46.287 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:46.285+0000 7f93e6000640 1 -- 172.21.15.72:0/2147400083 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93dc00cd80 con 0x7f93e8102970 2024-04-26T21:52:46.287 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:46.285+0000 7f93e6000640 1 -- 172.21.15.72:0/2147400083 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f93dc0044b0 con 0x7f93e8102970 2024-04-26T21:52:46.287 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:46.285+0000 7f93f0200640 1 -- 172.21.15.72:0/2147400083 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93e81a2b40 con 0x7f93e8102970 2024-04-26T21:52:46.287 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:46.285+0000 7f93e6000640 1 -- 172.21.15.72:0/2147400083 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93dc010430 con 0x7f93e8102970 2024-04-26T21:52:46.287 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:46.285+0000 7f93f0200640 1 -- 172.21.15.72:0/2147400083 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93e81a3090 con 0x7f93e8102970 2024-04-26T21:52:46.287 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:46.286+0000 7f93f0200640 1 -- 172.21.15.72:0/2147400083 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93e81a3260 con 0x7f93e8102970 2024-04-26T21:52:46.289 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:46.289+0000 7f93e6000640 1 -- 172.21.15.72:0/2147400083 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f93dc003f00 con 0x7f93e8102970 2024-04-26T21:52:46.291 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:46.290+0000 7f93e6000640 1 --2- 172.21.15.72:0/2147400083 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f93d00a3bb0 0x7f93d00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:46.291 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:46.291+0000 7f93e6000640 1 -- 172.21.15.72:0/2147400083 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7f93dc0d03a0 con 0x7f93e8102970 2024-04-26T21:52:46.291 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:46.291+0000 7f93ece00640 1 --2- 172.21.15.72:0/2147400083 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f93d00a3bb0 0x7f93d00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:46.291 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:46.291+0000 7f93ece00640 1 --2- 172.21.15.72:0/2147400083 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f93d00a3bb0 0x7f93d00a6070 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f93d4005d00 tx=0x7f93d400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:46.295 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:46.295+0000 7f93e6000640 1 -- 172.21.15.72:0/2147400083 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f93dc09d220 con 0x7f93e8102970 2024-04-26T21:52:46.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:46 smithi076 ceph-mon[31823]: mds.? [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] up:boot 2024-04-26T21:52:46.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:46 smithi076 ceph-mon[31823]: fsmap 12 up:standby 2024-04-26T21:52:46.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:46 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-26T21:52:46.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:46 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:52:46.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:46 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:52:46.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:46 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:52:46.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:46 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:52:46.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:46 smithi076 ceph-mon[31823]: pgmap v361: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:52:46.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:46 smithi164 ceph-mon[31916]: mds.? [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] up:boot 2024-04-26T21:52:46.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:46 smithi164 ceph-mon[31916]: fsmap 12 up:standby 2024-04-26T21:52:46.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:46 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-26T21:52:46.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:46 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:52:46.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:46 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:52:46.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:46 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:52:46.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:46 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:52:46.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:46 smithi164 ceph-mon[31916]: pgmap v361: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:52:46.581 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:46.581+0000 7f93f0200640 1 -- 172.21.15.72:0/2147400083 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f93e81a3430 con 0x7f93e8102970 2024-04-26T21:52:46.582 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:46.582+0000 7f93e6000640 1 -- 172.21.15.72:0/2147400083 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10482 (secure 0 0 0) 0x7f93dc09cb20 con 0x7f93e8102970 2024-04-26T21:52:46.582 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T21:52:46.583 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6835/1320381959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6834","nonce":1320381959},{"type":"v1","addr":"172.21.15.72:6835","nonce":1320381959}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6837/2142200512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6836","nonce":2142200512},{"type":"v1","addr":"172.21.15.72:6837","nonce":2142200512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14652,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6839/1312898281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6838","nonce":1312898281},{"type":"v1","addr":"172.21.15.72:6839","nonce":1312898281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14664,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6841/3257692339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6840","nonce":3257692339},{"type":"v1","addr":"172.21.15.72:6841","nonce":3257692339}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24388,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6833/4236668465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":4236668465},{"type":"v1","addr":"172.21.15.164:6833","nonce":4236668465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24424,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/2391114169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":2391114169},{"type":"v1","addr":"172.21.15.164:6835","nonce":2391114169}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24442,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6837/3585829992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":3585829992},{"type":"v1","addr":"172.21.15.164:6837","nonce":3585829992}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24463,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6839/1636226820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":1636226820},{"type":"v1","addr":"172.21.15.164:6839","nonce":1636226820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":34406,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6833/3869483750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":3869483750},{"type":"v1","addr":"172.21.15.76:6833","nonce":3869483750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34442,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/3517429435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":3517429435},{"type":"v1","addr":"172.21.15.76:6835","nonce":3517429435}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":34472,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6837/1150678991","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":1150678991},{"type":"v1","addr":"172.21.15.76:6837","nonce":1150678991}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":34517,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6839/327145636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6838","nonce":327145636},{"type":"v1","addr":"172.21.15.76:6839","nonce":327145636}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-04-26T21:52:46.583 INFO:teuthology.orchestra.run.smithi072.stderr:dumped fsmap epoch 13 2024-04-26T21:52:46.585 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:46.585+0000 7f93f0200640 1 -- 172.21.15.72:0/2147400083 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f93d00a3bb0 msgr2=0x7f93d00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:46.585 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:46.585+0000 7f93f0200640 1 --2- 172.21.15.72:0/2147400083 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f93d00a3bb0 0x7f93d00a6070 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f93d4005d00 tx=0x7f93d400b410 comp rx=0 tx=0).stop 2024-04-26T21:52:46.585 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:46.585+0000 7f93f0200640 1 -- 172.21.15.72:0/2147400083 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f93e8102970 msgr2=0x7f93e819ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:46.585 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:46.585+0000 7f93f0200640 1 --2- 172.21.15.72:0/2147400083 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f93e8102970 0x7f93e819ac40 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f93dc00e9c0 tx=0x7f93dc00ee80 comp rx=0 tx=0).stop 2024-04-26T21:52:46.585 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:46.585+0000 7f93f0200640 1 -- 172.21.15.72:0/2147400083 shutdown_connections 2024-04-26T21:52:46.585 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:46.585+0000 7f93f0200640 1 --2- 172.21.15.72:0/2147400083 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f93d00a3bb0 0x7f93d00a6070 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:46.585 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:46.585+0000 7f93f0200640 1 --2- 172.21.15.72:0/2147400083 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f93e8104520 0x7f93e81a21b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:46.586 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:46.585+0000 7f93f0200640 1 --2- 172.21.15.72:0/2147400083 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f93e8103b60 0x7f93e819b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:46.586 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:46.585+0000 7f93f0200640 1 --2- 172.21.15.72:0/2147400083 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f93e8102970 0x7f93e819ac40 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:46.586 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:46.585+0000 7f93f0200640 1 -- 172.21.15.72:0/2147400083 >> 172.21.15.72:0/2147400083 conn(0x7f93e80fdee0 msgr2=0x7f93e8074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:46.586 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:46.585+0000 7f93f0200640 1 -- 172.21.15.72:0/2147400083 shutdown_connections 2024-04-26T21:52:46.586 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:46.585+0000 7f93f0200640 1 -- 172.21.15.72:0/2147400083 wait complete. 2024-04-26T21:52:46.605 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:52:46.789 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:46.788+0000 7f16de800640 1 -- 172.21.15.72:0/3730365079 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f16d810a180 msgr2=0x7f16d810a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:46.789 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:46.788+0000 7f16de800640 1 --2- 172.21.15.72:0/3730365079 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f16d810a180 0x7f16d810a600 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f16c8009900 tx=0x7f16c802a3f0 comp rx=0 tx=0).stop 2024-04-26T21:52:46.789 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:46.788+0000 7f16de800640 1 -- 172.21.15.72:0/3730365079 shutdown_connections 2024-04-26T21:52:46.789 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:46.788+0000 7f16de800640 1 --2- 172.21.15.72:0/3730365079 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f16d810ab40 0x7f16d81113d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:46.789 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:46.788+0000 7f16de800640 1 --2- 172.21.15.72:0/3730365079 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f16d810a180 0x7f16d810a600 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:46.789 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:46.788+0000 7f16de800640 1 --2- 172.21.15.72:0/3730365079 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f16d8108f90 0x7f16d81093b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:46.789 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:46.788+0000 7f16de800640 1 -- 172.21.15.72:0/3730365079 >> 172.21.15.72:0/3730365079 conn(0x7f16d8077f40 msgr2=0x7f16d807a360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:46.789 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:46.788+0000 7f16de800640 1 -- 172.21.15.72:0/3730365079 shutdown_connections 2024-04-26T21:52:46.789 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:46.788+0000 7f16de800640 1 -- 172.21.15.72:0/3730365079 wait complete. 2024-04-26T21:52:46.792 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:46.789+0000 7f16de800640 1 Processor -- start 2024-04-26T21:52:46.792 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:46.789+0000 7f16de800640 1 -- start start 2024-04-26T21:52:46.792 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:46.789+0000 7f16de800640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f16d8108f90 0x7f16d819f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:46.792 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:46.789+0000 7f16de800640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f16d810a180 0x7f16d819f560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:46.792 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:46.789+0000 7f16de800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f16d810ab40 0x7f16d81a65d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:46.792 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:46.789+0000 7f16de800640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f16d81149d0 con 0x7f16d810a180 2024-04-26T21:52:46.792 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:46.789+0000 7f16de800640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f16d8114850 con 0x7f16d8108f90 2024-04-26T21:52:46.792 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:46.789+0000 7f16de800640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f16d8114b50 con 0x7f16d810ab40 2024-04-26T21:52:46.793 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:46.790+0000 7f16d6a00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f16d810a180 0x7f16d819f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:46.793 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:46.790+0000 7f16d6a00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f16d810a180 0x7f16d819f560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:45218/0 (socket says 172.21.15.72:45218) 2024-04-26T21:52:46.793 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:46.790+0000 7f16d6a00640 1 -- 172.21.15.72:0/3813516442 learned_addr learned my addr 172.21.15.72:0/3813516442 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:52:46.793 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:46.790+0000 7f16d7400640 1 --2- 172.21.15.72:0/3813516442 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f16d8108f90 0x7f16d819f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:46.793 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:46.790+0000 7f16d7e00640 1 --2- 172.21.15.72:0/3813516442 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f16d810ab40 0x7f16d81a65d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:46.793 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:46.790+0000 7f16d6a00640 1 -- 172.21.15.72:0/3813516442 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f16d810ab40 msgr2=0x7f16d81a65d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:46.793 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:46.790+0000 7f16d6a00640 1 --2- 172.21.15.72:0/3813516442 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f16d810ab40 0x7f16d81a65d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:46.793 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:46.790+0000 7f16d6a00640 1 -- 172.21.15.72:0/3813516442 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f16d8108f90 msgr2=0x7f16d819f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:46.793 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:46.790+0000 7f16d6a00640 1 --2- 172.21.15.72:0/3813516442 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f16d8108f90 0x7f16d819f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:46.793 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:46.790+0000 7f16d6a00640 1 -- 172.21.15.72:0/3813516442 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16d81a6c40 con 0x7f16d810a180 2024-04-26T21:52:46.793 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:46.790+0000 7f16d7400640 1 --2- 172.21.15.72:0/3813516442 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f16d8108f90 0x7f16d819f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:52:46.793 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:46.790+0000 7f16d7e00640 1 --2- 172.21.15.72:0/3813516442 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f16d810ab40 0x7f16d81a65d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:52:46.793 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:46.790+0000 7f16d6a00640 1 --2- 172.21.15.72:0/3813516442 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f16d810a180 0x7f16d819f560 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f16c8031460 tx=0x7f16c8003d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:46.794 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:46.791+0000 7f16cfe00640 1 -- 172.21.15.72:0/3813516442 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16c803f070 con 0x7f16d810a180 2024-04-26T21:52:46.794 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:46.791+0000 7f16cfe00640 1 -- 172.21.15.72:0/3813516442 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f16c8031da0 con 0x7f16d810a180 2024-04-26T21:52:46.794 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:46.791+0000 7f16de800640 1 -- 172.21.15.72:0/3813516442 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16d81a6ed0 con 0x7f16d810a180 2024-04-26T21:52:46.794 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:46.791+0000 7f16cfe00640 1 -- 172.21.15.72:0/3813516442 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16c8035030 con 0x7f16d810a180 2024-04-26T21:52:46.794 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:46.791+0000 7f16de800640 1 -- 172.21.15.72:0/3813516442 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16d81a7340 con 0x7f16d810a180 2024-04-26T21:52:46.794 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:46.792+0000 7f16de800640 1 -- 172.21.15.72:0/3813516442 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16d8110300 con 0x7f16d810a180 2024-04-26T21:52:46.794 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:46.794+0000 7f16cfe00640 1 -- 172.21.15.72:0/3813516442 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f16c8042070 con 0x7f16d810a180 2024-04-26T21:52:46.796 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:46.796+0000 7f16cfe00640 1 --2- 172.21.15.72:0/3813516442 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f16bc0a3c30 0x7f16bc0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:46.796 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:46.796+0000 7f16cfe00640 1 -- 172.21.15.72:0/3813516442 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7f16c80f09b0 con 0x7f16d810a180 2024-04-26T21:52:46.796 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:46.796+0000 7f16d7400640 1 --2- 172.21.15.72:0/3813516442 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f16bc0a3c30 0x7f16bc0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:46.797 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:46.797+0000 7f16d7400640 1 --2- 172.21.15.72:0/3813516442 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f16bc0a3c30 0x7f16bc0a60f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f16d8109fe0 tx=0x7f16c0005db0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:46.801 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:46.801+0000 7f16cfe00640 1 -- 172.21.15.72:0/3813516442 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f16c80bd790 con 0x7f16d810a180 2024-04-26T21:52:47.007 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:47.007+0000 7f16de800640 1 -- 172.21.15.72:0/3813516442 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f16d80635d0 con 0x7f16d810a180 2024-04-26T21:52:47.008 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:47.008+0000 7f16cfe00640 1 -- 172.21.15.72:0/3813516442 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+18409 (secure 0 0 0) 0x7f16c80bd090 con 0x7f16d810a180 2024-04-26T21:52:47.009 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T21:52:47.009 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":83,"fsid":"a1780950-0415-11ef-bc93-c7b262605968","created":"2024-04-26T21:42:01.504173+0000","modified":"2024-04-26T21:51:44.487133+0000","last_up_change":"2024-04-26T21:51:43.482437+0000","last_in_change":"2024-04-26T21:51:18.468587+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:46:29.785427+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"04e06884-b355-4762-9b29-9034b7201e9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6802","nonce":1416765179},{"type":"v1","addr":"172.21.15.72:6803","nonce":1416765179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6804","nonce":1416765179},{"type":"v1","addr":"172.21.15.72:6805","nonce":1416765179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6808","nonce":1416765179},{"type":"v1","addr":"172.21.15.72:6809","nonce":1416765179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6806","nonce":1416765179},{"type":"v1","addr":"172.21.15.72:6807","nonce":1416765179}]},"public_addr":"172.21.15.72:6803/1416765179","cluster_addr":"172.21.15.72:6805/1416765179","heartbeat_back_addr":"172.21.15.72:6809/1416765179","heartbeat_front_addr":"172.21.15.72:6807/1416765179","state":["exists","up"]},{"osd":1,"uuid":"d75a479a-4a2e-48e9-a01e-2b1c9617416a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":755426949},{"type":"v1","addr":"172.21.15.76:6801","nonce":755426949}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":755426949},{"type":"v1","addr":"172.21.15.76:6803","nonce":755426949}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":755426949},{"type":"v1","addr":"172.21.15.76:6807","nonce":755426949}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":755426949},{"type":"v1","addr":"172.21.15.76:6805","nonce":755426949}]},"public_addr":"172.21.15.76:6801/755426949","cluster_addr":"172.21.15.76:6803/755426949","heartbeat_back_addr":"172.21.15.76:6807/755426949","heartbeat_front_addr":"172.21.15.76:6805/755426949","state":["exists","up"]},{"osd":2,"uuid":"b6f8da58-4428-4e8c-a944-7ea5645d6d92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":852265973},{"type":"v1","addr":"172.21.15.164:6801","nonce":852265973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":852265973},{"type":"v1","addr":"172.21.15.164:6803","nonce":852265973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":852265973},{"type":"v1","addr":"172.21.15.164:6807","nonce":852265973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":852265973},{"type":"v1","addr":"172.21.15.164:6805","nonce":852265973}]},"public_addr":"172.21.15.164:6801/852265973","cluster_addr":"172.21.15.164:6803/852265973","heartbeat_back_addr":"172.21.15.164:6807/852265973","heartbeat_front_addr":"172.21.15.164:6805/852265973","state":["exists","up"]},{"osd":3,"uuid":"76d83409-649e-41e5-b4f4-43bbd5b00ea5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6810","nonce":4027763789},{"type":"v1","addr":"172.21.15.72:6811","nonce":4027763789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6812","nonce":4027763789},{"type":"v1","addr":"172.21.15.72:6813","nonce":4027763789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6816","nonce":4027763789},{"type":"v1","addr":"172.21.15.72:6817","nonce":4027763789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6814","nonce":4027763789},{"type":"v1","addr":"172.21.15.72:6815","nonce":4027763789}]},"public_addr":"172.21.15.72:6811/4027763789","cluster_addr":"172.21.15.72:6813/4027763789","heartbeat_back_addr":"172.21.15.72:6817/4027763789","heartbeat_front_addr":"172.21.15.72:6815/4027763789","state":["exists","up"]},{"osd":4,"uuid":"d2cfeea7-1661-4156-bdcd-f770a8d1a878","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":3718867827},{"type":"v1","addr":"172.21.15.76:6809","nonce":3718867827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":3718867827},{"type":"v1","addr":"172.21.15.76:6811","nonce":3718867827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":3718867827},{"type":"v1","addr":"172.21.15.76:6815","nonce":3718867827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":3718867827},{"type":"v1","addr":"172.21.15.76:6813","nonce":3718867827}]},"public_addr":"172.21.15.76:6809/3718867827","cluster_addr":"172.21.15.76:6811/3718867827","heartbeat_back_addr":"172.21.15.76:6815/3718867827","heartbeat_front_addr":"172.21.15.76:6813/3718867827","state":["exists","up"]},{"osd":5,"uuid":"f3896ad6-a040-4ea3-a431-ff9b7fc05c36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":2543954772},{"type":"v1","addr":"172.21.15.164:6809","nonce":2543954772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":2543954772},{"type":"v1","addr":"172.21.15.164:6811","nonce":2543954772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":2543954772},{"type":"v1","addr":"172.21.15.164:6815","nonce":2543954772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":2543954772},{"type":"v1","addr":"172.21.15.164:6813","nonce":2543954772}]},"public_addr":"172.21.15.164:6809/2543954772","cluster_addr":"172.21.15.164:6811/2543954772","heartbeat_back_addr":"172.21.15.164:6815/2543954772","heartbeat_front_addr":"172.21.15.164:6813/2543954772","state":["exists","up"]},{"osd":6,"uuid":"d5d56650-be7e-4ba5-ba3a-d7d473294dab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6818","nonce":2080659027},{"type":"v1","addr":"172.21.15.72:6819","nonce":2080659027}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6820","nonce":2080659027},{"type":"v1","addr":"172.21.15.72:6821","nonce":2080659027}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6824","nonce":2080659027},{"type":"v1","addr":"172.21.15.72:6825","nonce":2080659027}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6822","nonce":2080659027},{"type":"v1","addr":"172.21.15.72:6823","nonce":2080659027}]},"public_addr":"172.21.15.72:6819/2080659027","cluster_addr":"172.21.15.72:6821/2080659027","heartbeat_back_addr":"172.21.15.72:6825/2080659027","heartbeat_front_addr":"172.21.15.72:6823/2080659027","state":["exists","up"]},{"osd":7,"uuid":"18975b5a-9309-4a0f-9f37-3ce09c2e745d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":3144631323},{"type":"v1","addr":"172.21.15.76:6817","nonce":3144631323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":3144631323},{"type":"v1","addr":"172.21.15.76:6819","nonce":3144631323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":3144631323},{"type":"v1","addr":"172.21.15.76:6823","nonce":3144631323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":3144631323},{"type":"v1","addr":"172.21.15.76:6821","nonce":3144631323}]},"public_addr":"172.21.15.76:6817/3144631323","cluster_addr":"172.21.15.76:6819/3144631323","heartbeat_back_addr":"172.21.15.76:6823/3144631323","heartbeat_front_addr":"172.21.15.76:6821/3144631323","state":["exists","up"]},{"osd":8,"uuid":"779e9c1e-8e97-48d4-a854-68ca13f38e0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":627888883},{"type":"v1","addr":"172.21.15.164:6817","nonce":627888883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":627888883},{"type":"v1","addr":"172.21.15.164:6819","nonce":627888883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":627888883},{"type":"v1","addr":"172.21.15.164:6823","nonce":627888883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":627888883},{"type":"v1","addr":"172.21.15.164:6821","nonce":627888883}]},"public_addr":"172.21.15.164:6817/627888883","cluster_addr":"172.21.15.164:6819/627888883","heartbeat_back_addr":"172.21.15.164:6823/627888883","heartbeat_front_addr":"172.21.15.164:6821/627888883","state":["exists","up"]},{"osd":9,"uuid":"d0317850-7b2d-4850-b2cf-0b919505c1fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6826","nonce":1273118485},{"type":"v1","addr":"172.21.15.72:6827","nonce":1273118485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6828","nonce":1273118485},{"type":"v1","addr":"172.21.15.72:6829","nonce":1273118485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6832","nonce":1273118485},{"type":"v1","addr":"172.21.15.72:6833","nonce":1273118485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6830","nonce":1273118485},{"type":"v1","addr":"172.21.15.72:6831","nonce":1273118485}]},"public_addr":"172.21.15.72:6827/1273118485","cluster_addr":"172.21.15.72:6829/1273118485","heartbeat_back_addr":"172.21.15.72:6833/1273118485","heartbeat_front_addr":"172.21.15.72:6831/1273118485","state":["exists","up"]},{"osd":10,"uuid":"e0aa38d8-6011-4d2a-b67b-5bff6d999e47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":2808954459},{"type":"v1","addr":"172.21.15.76:6825","nonce":2808954459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":2808954459},{"type":"v1","addr":"172.21.15.76:6827","nonce":2808954459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":2808954459},{"type":"v1","addr":"172.21.15.76:6831","nonce":2808954459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":2808954459},{"type":"v1","addr":"172.21.15.76:6829","nonce":2808954459}]},"public_addr":"172.21.15.76:6825/2808954459","cluster_addr":"172.21.15.76:6827/2808954459","heartbeat_back_addr":"172.21.15.76:6831/2808954459","heartbeat_front_addr":"172.21.15.76:6829/2808954459","state":["exists","up"]},{"osd":11,"uuid":"981b3d4d-916e-4937-9916-7cc21a891e43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":2354951784},{"type":"v1","addr":"172.21.15.164:6825","nonce":2354951784}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":2354951784},{"type":"v1","addr":"172.21.15.164:6827","nonce":2354951784}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":2354951784},{"type":"v1","addr":"172.21.15.164:6831","nonce":2354951784}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":2354951784},{"type":"v1","addr":"172.21.15.164:6829","nonce":2354951784}]},"public_addr":"172.21.15.164:6825/2354951784","cluster_addr":"172.21.15.164:6827/2354951784","heartbeat_back_addr":"172.21.15.164:6831/2354951784","heartbeat_front_addr":"172.21.15.164:6829/2354951784","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:18.997924+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:52.439081+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:46:25.893288+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:47:00.112807+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:47:35.116874+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:48:09.216344+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:48:45.028853+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:49:19.905397+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:49:55.030536+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:50:30.767662+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:51:06.423267+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:51:41.631211+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.72:0/416277784":"2024-04-27T21:43:09.602791+0000","172.21.15.72:6800/4010358672":"2024-04-27T21:43:09.602791+0000","172.21.15.72:6801/1012617371":"2024-04-27T21:42:34.684837+0000","172.21.15.72:6801/4010358672":"2024-04-27T21:43:09.602791+0000","172.21.15.72:0/3555448179":"2024-04-27T21:43:09.602791+0000","172.21.15.72:6800/1012617371":"2024-04-27T21:42:34.684837+0000","172.21.15.72:0/2920378761":"2024-04-27T21:42:34.684837+0000","172.21.15.72:0/1933740041":"2024-04-27T21:43:09.602791+0000","172.21.15.72:0/3225295775":"2024-04-27T21:42:34.684837+0000","172.21.15.72:0/2326016180":"2024-04-27T21:42:34.684837+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:52:47.011 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:47.011+0000 7f16de800640 1 -- 172.21.15.72:0/3813516442 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f16bc0a3c30 msgr2=0x7f16bc0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:47.012 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:47.011+0000 7f16de800640 1 --2- 172.21.15.72:0/3813516442 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f16bc0a3c30 0x7f16bc0a60f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f16d8109fe0 tx=0x7f16c0005db0 comp rx=0 tx=0).stop 2024-04-26T21:52:47.012 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:47.011+0000 7f16de800640 1 -- 172.21.15.72:0/3813516442 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f16d810a180 msgr2=0x7f16d819f560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:47.012 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:47.011+0000 7f16de800640 1 --2- 172.21.15.72:0/3813516442 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f16d810a180 0x7f16d819f560 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f16c8031460 tx=0x7f16c8003d50 comp rx=0 tx=0).stop 2024-04-26T21:52:47.012 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:47.011+0000 7f16de800640 1 -- 172.21.15.72:0/3813516442 shutdown_connections 2024-04-26T21:52:47.012 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:47.011+0000 7f16de800640 1 --2- 172.21.15.72:0/3813516442 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f16bc0a3c30 0x7f16bc0a60f0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:47.012 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:47.011+0000 7f16de800640 1 --2- 172.21.15.72:0/3813516442 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f16d810ab40 0x7f16d81a65d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:47.012 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:47.011+0000 7f16de800640 1 --2- 172.21.15.72:0/3813516442 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f16d810a180 0x7f16d819f560 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:47.012 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:47.011+0000 7f16de800640 1 --2- 172.21.15.72:0/3813516442 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f16d8108f90 0x7f16d819f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:47.012 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:47.011+0000 7f16de800640 1 -- 172.21.15.72:0/3813516442 >> 172.21.15.72:0/3813516442 conn(0x7f16d8077f40 msgr2=0x7f16d810d4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:47.012 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:47.011+0000 7f16de800640 1 -- 172.21.15.72:0/3813516442 shutdown_connections 2024-04-26T21:52:47.012 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:47.011+0000 7f16de800640 1 -- 172.21.15.72:0/3813516442 wait complete. 2024-04-26T21:52:47.029 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:46:29.785427+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:52:47.030 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:52:47.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:47 smithi164 ceph-mon[31916]: from='client.? 172.21.15.72:0/2147400083' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:52:47.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:47 smithi164 ceph-mon[31916]: from='client.? 172.21.15.72:0/3813516442' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:52:47.207 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:47.207+0000 7f35a7200640 1 -- 172.21.15.72:0/4257902465 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f35a8103380 msgr2=0x7f35a8105770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:47.208 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:47.207+0000 7f35a7200640 1 --2- 172.21.15.72:0/4257902465 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f35a8103380 0x7f35a8105770 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f3598009900 tx=0x7f359802a3f0 comp rx=0 tx=0).stop 2024-04-26T21:52:47.208 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:47.207+0000 7f35a7200640 1 -- 172.21.15.72:0/4257902465 shutdown_connections 2024-04-26T21:52:47.208 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:47.207+0000 7f35a7200640 1 --2- 172.21.15.72:0/4257902465 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f35a81085e0 0x7f35a810a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:47.208 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:47.207+0000 7f35a7200640 1 --2- 172.21.15.72:0/4257902465 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f35a8105cb0 0x7f35a81080a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:47.208 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:47.207+0000 7f35a7200640 1 --2- 172.21.15.72:0/4257902465 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f35a8103380 0x7f35a8105770 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:47.208 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:47.207+0000 7f35a7200640 1 -- 172.21.15.72:0/4257902465 >> 172.21.15.72:0/4257902465 conn(0x7f35a80fccd0 msgr2=0x7f35a80ff0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:47.208 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:47.207+0000 7f35a7200640 1 -- 172.21.15.72:0/4257902465 shutdown_connections 2024-04-26T21:52:47.208 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:47.207+0000 7f35a7200640 1 -- 172.21.15.72:0/4257902465 wait complete. 2024-04-26T21:52:47.210 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:47.208+0000 7f35a7200640 1 Processor -- start 2024-04-26T21:52:47.210 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:47.208+0000 7f35a7200640 1 -- start start 2024-04-26T21:52:47.210 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:47.209+0000 7f35a7200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f35a8103380 0x7f35a819ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:47.213 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:47.209+0000 7f35a7200640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f35a8105cb0 0x7f35a819b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:47.213 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:47.209+0000 7f35a7200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f35a81085e0 0x7f35a81a2280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:47.213 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:47.209+0000 7f35a7200640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f35a810e370 con 0x7f35a8105cb0 2024-04-26T21:52:47.213 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:47.209+0000 7f35a7200640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f35a810e1f0 con 0x7f35a8103380 2024-04-26T21:52:47.213 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:47.209+0000 7f35a7200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f35a810e4f0 con 0x7f35a81085e0 2024-04-26T21:52:47.213 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:47.209+0000 7f35a5400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f35a8105cb0 0x7f35a819b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:47.213 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:47.209+0000 7f35a5400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f35a8105cb0 0x7f35a819b250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:45242/0 (socket says 172.21.15.72:45242) 2024-04-26T21:52:47.213 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:47.209+0000 7f35a5400640 1 -- 172.21.15.72:0/1352254668 learned_addr learned my addr 172.21.15.72:0/1352254668 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:52:47.213 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:47.209+0000 7f35a5e00640 1 --2- 172.21.15.72:0/1352254668 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f35a8103380 0x7f35a819ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:47.213 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:47.209+0000 7f35a5400640 1 -- 172.21.15.72:0/1352254668 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f35a81085e0 msgr2=0x7f35a81a2280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:47.214 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:47.209+0000 7f35a5400640 1 --2- 172.21.15.72:0/1352254668 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f35a81085e0 0x7f35a81a2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:47.214 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:47.209+0000 7f35a5400640 1 -- 172.21.15.72:0/1352254668 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f35a8103380 msgr2=0x7f35a819ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:47.214 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:47.209+0000 7f35a5400640 1 --2- 172.21.15.72:0/1352254668 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f35a8103380 0x7f35a819ad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:47.214 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:47.209+0000 7f35a5400640 1 -- 172.21.15.72:0/1352254668 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35a81a2920 con 0x7f35a8105cb0 2024-04-26T21:52:47.214 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:47.210+0000 7f35a5400640 1 --2- 172.21.15.72:0/1352254668 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f35a8105cb0 0x7f35a819b250 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f359400b450 tx=0x7f359400b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:47.214 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:47.210+0000 7f359ea00640 1 -- 172.21.15.72:0/1352254668 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3594015020 con 0x7f35a8105cb0 2024-04-26T21:52:47.214 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:47.210+0000 7f359ea00640 1 -- 172.21.15.72:0/1352254668 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3594011d20 con 0x7f35a8105cb0 2024-04-26T21:52:47.214 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:47.210+0000 7f35a7200640 1 -- 172.21.15.72:0/1352254668 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35a81a2c10 con 0x7f35a8105cb0 2024-04-26T21:52:47.214 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:47.210+0000 7f359ea00640 1 -- 172.21.15.72:0/1352254668 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3594010e10 con 0x7f35a8105cb0 2024-04-26T21:52:47.214 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:47.210+0000 7f35a7200640 1 -- 172.21.15.72:0/1352254668 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35a81a3230 con 0x7f35a8105cb0 2024-04-26T21:52:47.215 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:47.211+0000 7f35a7200640 1 -- 172.21.15.72:0/1352254668 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35a810a0d0 con 0x7f35a8105cb0 2024-04-26T21:52:47.215 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:47.214+0000 7f359ea00640 1 -- 172.21.15.72:0/1352254668 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f3594018020 con 0x7f35a8105cb0 2024-04-26T21:52:47.215 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:47.215+0000 7f359ea00640 1 --2- 172.21.15.72:0/1352254668 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f358c0a3c30 0x7f358c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:47.215 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:47.215+0000 7f359ea00640 1 -- 172.21.15.72:0/1352254668 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7f35940d0200 con 0x7f35a8105cb0 2024-04-26T21:52:47.215 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:47.215+0000 7f35a5e00640 1 --2- 172.21.15.72:0/1352254668 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f358c0a3c30 0x7f358c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:47.216 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:47.216+0000 7f35a5e00640 1 --2- 172.21.15.72:0/1352254668 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f358c0a3c30 0x7f358c0a60f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f359802a910 tx=0x7f3598000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:47.217 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:47.217+0000 7f359ea00640 1 -- 172.21.15.72:0/1352254668 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f359401f020 con 0x7f35a8105cb0 2024-04-26T21:52:47.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:47 smithi076 ceph-mon[31823]: from='client.? 172.21.15.72:0/2147400083' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:52:47.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:47 smithi076 ceph-mon[31823]: from='client.? 172.21.15.72:0/3813516442' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:52:47.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:47 smithi072 ceph-mon[28138]: from='client.? 172.21.15.72:0/2147400083' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:52:47.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:47 smithi072 ceph-mon[28138]: from='client.? 172.21.15.72:0/3813516442' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:52:47.420 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:47.419+0000 7f35a7200640 1 -- 172.21.15.72:0/1352254668 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f35a802dd30 con 0x7f35a8105cb0 2024-04-26T21:52:47.420 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:47.420+0000 7f359ea00640 1 -- 172.21.15.72:0/1352254668 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v83) ==== 93+0+10 (secure 0 0 0) 0x7f359400d040 con 0x7f35a8105cb0 2024-04-26T21:52:47.420 INFO:teuthology.orchestra.run.smithi072.stdout:pg_num: 1 2024-04-26T21:52:47.423 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:47.422+0000 7f35a7200640 1 -- 172.21.15.72:0/1352254668 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f358c0a3c30 msgr2=0x7f358c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:47.423 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:47.422+0000 7f35a7200640 1 --2- 172.21.15.72:0/1352254668 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f358c0a3c30 0x7f358c0a60f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f359802a910 tx=0x7f3598000bc0 comp rx=0 tx=0).stop 2024-04-26T21:52:47.423 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:47.422+0000 7f35a7200640 1 -- 172.21.15.72:0/1352254668 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f35a8105cb0 msgr2=0x7f35a819b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:47.423 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:47.422+0000 7f35a7200640 1 --2- 172.21.15.72:0/1352254668 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f35a8105cb0 0x7f35a819b250 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f359400b450 tx=0x7f359400b910 comp rx=0 tx=0).stop 2024-04-26T21:52:47.423 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:47.422+0000 7f35a7200640 1 -- 172.21.15.72:0/1352254668 shutdown_connections 2024-04-26T21:52:47.423 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:47.422+0000 7f35a7200640 1 --2- 172.21.15.72:0/1352254668 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f358c0a3c30 0x7f358c0a60f0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:47.423 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:47.422+0000 7f35a7200640 1 --2- 172.21.15.72:0/1352254668 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f35a81085e0 0x7f35a81a2280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:47.423 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:47.422+0000 7f35a7200640 1 --2- 172.21.15.72:0/1352254668 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f35a8105cb0 0x7f35a819b250 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:47.423 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:47.422+0000 7f35a7200640 1 --2- 172.21.15.72:0/1352254668 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f35a8103380 0x7f35a819ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:47.423 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:47.422+0000 7f35a7200640 1 -- 172.21.15.72:0/1352254668 >> 172.21.15.72:0/1352254668 conn(0x7f35a80fccd0 msgr2=0x7f35a81029e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:47.423 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:47.422+0000 7f35a7200640 1 -- 172.21.15.72:0/1352254668 shutdown_connections 2024-04-26T21:52:47.424 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:47.423+0000 7f35a7200640 1 -- 172.21.15.72:0/1352254668 wait complete. 2024-04-26T21:52:47.440 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-26T21:52:47.619 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:47.619+0000 7f7726c00640 1 -- 172.21.15.72:0/1702829154 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7720078ed0 msgr2=0x7f7720079770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:47.619 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:47.619+0000 7f7726c00640 1 --2- 172.21.15.72:0/1702829154 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7720078ed0 0x7f7720079770 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f7718009ca0 tx=0x7f771802a8c0 comp rx=0 tx=0).stop 2024-04-26T21:52:47.620 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:47.619+0000 7f7726c00640 1 -- 172.21.15.72:0/1702829154 shutdown_connections 2024-04-26T21:52:47.620 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:47.619+0000 7f7726c00640 1 --2- 172.21.15.72:0/1702829154 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7720078ed0 0x7f7720079770 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:47.620 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:47.619+0000 7f7726c00640 1 --2- 172.21.15.72:0/1702829154 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7720078530 0x7f7720078990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:47.620 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:47.619+0000 7f7726c00640 1 --2- 172.21.15.72:0/1702829154 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f77200772f0 0x7f7720077710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:47.620 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:47.619+0000 7f7726c00640 1 -- 172.21.15.72:0/1702829154 >> 172.21.15.72:0/1702829154 conn(0x7f7720100120 msgr2=0x7f7720102540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:47.620 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:47.619+0000 7f7726c00640 1 -- 172.21.15.72:0/1702829154 shutdown_connections 2024-04-26T21:52:47.620 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:47.619+0000 7f7726c00640 1 -- 172.21.15.72:0/1702829154 wait complete. 2024-04-26T21:52:47.621 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:47.620+0000 7f7726c00640 1 Processor -- start 2024-04-26T21:52:47.621 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:47.620+0000 7f7726c00640 1 -- start start 2024-04-26T21:52:47.621 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:47.620+0000 7f7726c00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f77200772f0 0x7f772019f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:47.622 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:47.620+0000 7f7726c00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7720078530 0x7f772019f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:47.622 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:47.620+0000 7f7726c00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7720078ed0 0x7f77201a6600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:47.622 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:47.620+0000 7f7726c00640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f77201149c0 con 0x7f7720078530 2024-04-26T21:52:47.623 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:47.620+0000 7f7726c00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f7720114840 con 0x7f77200772f0 2024-04-26T21:52:47.623 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:47.620+0000 7f7726c00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f7720114b40 con 0x7f7720078ed0 2024-04-26T21:52:47.623 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:47.621+0000 7f771f400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7720078530 0x7f772019f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:47.623 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:47.621+0000 7f771f400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7720078530 0x7f772019f590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:45262/0 (socket says 172.21.15.72:45262) 2024-04-26T21:52:47.623 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:47.621+0000 7f771f400640 1 -- 172.21.15.72:0/2185352204 learned_addr learned my addr 172.21.15.72:0/2185352204 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:52:47.623 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:47.621+0000 7f771fe00640 1 --2- 172.21.15.72:0/2185352204 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f77200772f0 0x7f772019f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:47.623 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:47.621+0000 7f7724c00640 1 --2- 172.21.15.72:0/2185352204 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7720078ed0 0x7f77201a6600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:47.623 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:47.621+0000 7f771f400640 1 -- 172.21.15.72:0/2185352204 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7720078ed0 msgr2=0x7f77201a6600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:47.623 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:47.621+0000 7f771f400640 1 --2- 172.21.15.72:0/2185352204 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7720078ed0 0x7f77201a6600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:47.623 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:47.621+0000 7f771f400640 1 -- 172.21.15.72:0/2185352204 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f77200772f0 msgr2=0x7f772019f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:47.623 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:47.621+0000 7f771f400640 1 --2- 172.21.15.72:0/2185352204 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f77200772f0 0x7f772019f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:47.623 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:47.621+0000 7f771f400640 1 -- 172.21.15.72:0/2185352204 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77201a6c70 con 0x7f7720078530 2024-04-26T21:52:47.623 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:47.621+0000 7f771f400640 1 --2- 172.21.15.72:0/2185352204 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7720078530 0x7f772019f590 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f770c00d950 tx=0x7f770c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:47.623 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:47.622+0000 7f771cc00640 1 -- 172.21.15.72:0/2185352204 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f770c017030 con 0x7f7720078530 2024-04-26T21:52:47.623 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:47.622+0000 7f771cc00640 1 -- 172.21.15.72:0/2185352204 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f770c004020 con 0x7f7720078530 2024-04-26T21:52:47.624 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:47.622+0000 7f771cc00640 1 -- 172.21.15.72:0/2185352204 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f770c0055d0 con 0x7f7720078530 2024-04-26T21:52:47.624 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:47.622+0000 7f7726c00640 1 -- 172.21.15.72:0/2185352204 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77201a6f60 con 0x7f7720078530 2024-04-26T21:52:47.624 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:47.622+0000 7f7726c00640 1 -- 172.21.15.72:0/2185352204 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77201a73d0 con 0x7f7720078530 2024-04-26T21:52:47.624 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:47.624+0000 7f7726c00640 1 -- 172.21.15.72:0/2185352204 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7720111700 con 0x7f7720078530 2024-04-26T21:52:47.625 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:47.625+0000 7f771cc00640 1 -- 172.21.15.72:0/2185352204 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f770c009a30 con 0x7f7720078530 2024-04-26T21:52:47.627 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:47.626+0000 7f771cc00640 1 --2- 172.21.15.72:0/2185352204 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f77040a3bb0 0x7f77040a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:47.627 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:47.627+0000 7f771cc00640 1 -- 172.21.15.72:0/2185352204 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7f770c0cf8e0 con 0x7f7720078530 2024-04-26T21:52:47.627 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:47.627+0000 7f771fe00640 1 --2- 172.21.15.72:0/2185352204 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f77040a3bb0 0x7f77040a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:47.627 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:47.627+0000 7f771fe00640 1 --2- 172.21.15.72:0/2185352204 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f77040a3bb0 0x7f77040a6070 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f7710005d00 tx=0x7f771000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:47.633 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:47.633+0000 7f771cc00640 1 -- 172.21.15.72:0/2185352204 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f770c09c6c0 con 0x7f7720078530 2024-04-26T21:52:47.860 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:47.860+0000 7f7726c00640 1 -- 172.21.15.72:0/2185352204 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f7720063560 con 0x7f7720078530 2024-04-26T21:52:47.861 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:47.861+0000 7f771cc00640 1 -- 172.21.15.72:0/2185352204 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v83) ==== 84+0+65 (secure 0 0 0) 0x7f770c09bfc0 con 0x7f7720078530 2024-04-26T21:52:47.861 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T21:52:47.861 INFO:teuthology.orchestra.run.smithi072.stdout:[ 2024-04-26T21:52:47.861 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-26T21:52:47.861 INFO:teuthology.orchestra.run.smithi072.stdout: "poolnum": 1, 2024-04-26T21:52:47.861 INFO:teuthology.orchestra.run.smithi072.stdout: "poolname": ".mgr" 2024-04-26T21:52:47.861 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:52:47.861 INFO:teuthology.orchestra.run.smithi072.stdout:] 2024-04-26T21:52:47.863 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:47.863+0000 7f7726c00640 1 -- 172.21.15.72:0/2185352204 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f77040a3bb0 msgr2=0x7f77040a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:47.863 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:47.863+0000 7f7726c00640 1 --2- 172.21.15.72:0/2185352204 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f77040a3bb0 0x7f77040a6070 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f7710005d00 tx=0x7f771000b410 comp rx=0 tx=0).stop 2024-04-26T21:52:47.863 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:47.863+0000 7f7726c00640 1 -- 172.21.15.72:0/2185352204 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7720078530 msgr2=0x7f772019f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:47.864 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:47.863+0000 7f7726c00640 1 --2- 172.21.15.72:0/2185352204 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7720078530 0x7f772019f590 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f770c00d950 tx=0x7f770c00de10 comp rx=0 tx=0).stop 2024-04-26T21:52:47.864 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:47.863+0000 7f7726c00640 1 -- 172.21.15.72:0/2185352204 shutdown_connections 2024-04-26T21:52:47.864 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:47.863+0000 7f7726c00640 1 --2- 172.21.15.72:0/2185352204 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f77040a3bb0 0x7f77040a6070 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:47.864 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:47.863+0000 7f7726c00640 1 --2- 172.21.15.72:0/2185352204 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7720078ed0 0x7f77201a6600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:47.864 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:47.863+0000 7f7726c00640 1 --2- 172.21.15.72:0/2185352204 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7720078530 0x7f772019f590 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:47.864 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:47.863+0000 7f7726c00640 1 --2- 172.21.15.72:0/2185352204 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f77200772f0 0x7f772019f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:47.864 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:47.863+0000 7f7726c00640 1 -- 172.21.15.72:0/2185352204 >> 172.21.15.72:0/2185352204 conn(0x7f7720100120 msgr2=0x7f7720101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:47.864 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:47.863+0000 7f7726c00640 1 -- 172.21.15.72:0/2185352204 shutdown_connections 2024-04-26T21:52:47.864 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:47.863+0000 7f7726c00640 1 -- 172.21.15.72:0/2185352204 wait complete. 2024-04-26T21:52:47.883 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-26T21:52:47.883 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-26T21:52:48.076 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:48.075+0000 7faf6a400640 1 -- 172.21.15.72:0/3486050929 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7faf64103a50 msgr2=0x7faf64103ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:48.076 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:48.075+0000 7faf6a400640 1 --2- 172.21.15.72:0/3486050929 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7faf64103a50 0x7faf64103ed0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7faf4c009900 tx=0x7faf4c02a410 comp rx=0 tx=0).stop 2024-04-26T21:52:48.076 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:48.075+0000 7faf6a400640 1 -- 172.21.15.72:0/3486050929 shutdown_connections 2024-04-26T21:52:48.076 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:48.075+0000 7faf6a400640 1 --2- 172.21.15.72:0/3486050929 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faf64104410 0x7faf6410aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:48.076 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:48.075+0000 7faf6a400640 1 --2- 172.21.15.72:0/3486050929 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7faf64103a50 0x7faf64103ed0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:48.076 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:48.075+0000 7faf6a400640 1 --2- 172.21.15.72:0/3486050929 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faf64102860 0x7faf64102c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:48.076 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:48.075+0000 7faf6a400640 1 -- 172.21.15.72:0/3486050929 >> 172.21.15.72:0/3486050929 conn(0x7faf640fddd0 msgr2=0x7faf64100210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:48.076 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:48.075+0000 7faf6a400640 1 -- 172.21.15.72:0/3486050929 shutdown_connections 2024-04-26T21:52:48.076 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:48.075+0000 7faf6a400640 1 -- 172.21.15.72:0/3486050929 wait complete. 2024-04-26T21:52:48.079 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:48.076+0000 7faf6a400640 1 Processor -- start 2024-04-26T21:52:48.079 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:48.076+0000 7faf6a400640 1 -- start start 2024-04-26T21:52:48.079 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:48.077+0000 7faf6a400640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faf64102860 0x7faf6419abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:48.079 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:48.077+0000 7faf6a400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faf64103a50 0x7faf6419b110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:48.079 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:48.077+0000 7faf6a400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7faf64104410 0x7faf641a2180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:48.079 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:48.077+0000 7faf6a400640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7faf6410e270 con 0x7faf64104410 2024-04-26T21:52:48.079 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:48.077+0000 7faf6a400640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7faf6410e0f0 con 0x7faf64102860 2024-04-26T21:52:48.079 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:48.077+0000 7faf6a400640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7faf6410e3f0 con 0x7faf64103a50 2024-04-26T21:52:48.079 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:48.077+0000 7faf69a00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7faf64104410 0x7faf641a2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:48.079 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:48.077+0000 7faf5be00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faf64103a50 0x7faf6419b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:48.079 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:48.077+0000 7faf5be00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faf64103a50 0x7faf6419b110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.72:32864/0 (socket says 172.21.15.72:32864) 2024-04-26T21:52:48.079 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:48.077+0000 7faf5be00640 1 -- 172.21.15.72:0/2900170921 learned_addr learned my addr 172.21.15.72:0/2900170921 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:52:48.079 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:48.077+0000 7faf69a00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7faf64104410 0x7faf641a2180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:45278/0 (socket says 172.21.15.72:45278) 2024-04-26T21:52:48.079 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:48.077+0000 7faf69000640 1 --2- 172.21.15.72:0/2900170921 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faf64102860 0x7faf6419abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:48.080 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:48.078+0000 7faf5be00640 1 -- 172.21.15.72:0/2900170921 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faf64102860 msgr2=0x7faf6419abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:48.080 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:48.078+0000 7faf5be00640 1 --2- 172.21.15.72:0/2900170921 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faf64102860 0x7faf6419abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:48.080 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:48.078+0000 7faf5be00640 1 -- 172.21.15.72:0/2900170921 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7faf64104410 msgr2=0x7faf641a2180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:48.080 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:48.078+0000 7faf5be00640 1 --2- 172.21.15.72:0/2900170921 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7faf64104410 0x7faf641a2180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:48.080 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:48.078+0000 7faf5be00640 1 -- 172.21.15.72:0/2900170921 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf641a27f0 con 0x7faf64103a50 2024-04-26T21:52:48.080 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:48.078+0000 7faf69a00640 1 --2- 172.21.15.72:0/2900170921 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7faf64104410 0x7faf641a2180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:52:48.080 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:48.078+0000 7faf5be00640 1 --2- 172.21.15.72:0/2900170921 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faf64103a50 0x7faf6419b110 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7faf4c031460 tx=0x7faf4c003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:48.080 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:48.078+0000 7faf62000640 1 -- 172.21.15.72:0/2900170921 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf4c03f070 con 0x7faf64103a50 2024-04-26T21:52:48.080 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:48.078+0000 7faf6a400640 1 -- 172.21.15.72:0/2900170921 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf641a2a80 con 0x7faf64103a50 2024-04-26T21:52:48.080 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:48.078+0000 7faf62000640 1 -- 172.21.15.72:0/2900170921 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7faf4c031ba0 con 0x7faf64103a50 2024-04-26T21:52:48.080 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:48.078+0000 7faf62000640 1 -- 172.21.15.72:0/2900170921 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf4c035030 con 0x7faf64103a50 2024-04-26T21:52:48.080 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:48.079+0000 7faf6a400640 1 -- 172.21.15.72:0/2900170921 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf641a2f70 con 0x7faf64103a50 2024-04-26T21:52:48.081 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:48.080+0000 7faf6a400640 1 -- 172.21.15.72:0/2900170921 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf641a3260 con 0x7faf64103a50 2024-04-26T21:52:48.082 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:48.082+0000 7faf62000640 1 -- 172.21.15.72:0/2900170921 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7faf4c042070 con 0x7faf64103a50 2024-04-26T21:52:48.083 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:48.083+0000 7faf62000640 1 --2- 172.21.15.72:0/2900170921 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7faf480a3bb0 0x7faf480a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:48.084 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:48.084+0000 7faf62000640 1 -- 172.21.15.72:0/2900170921 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7faf4c0f0ae0 con 0x7faf64103a50 2024-04-26T21:52:48.084 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:48.084+0000 7faf69000640 1 --2- 172.21.15.72:0/2900170921 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7faf480a3bb0 0x7faf480a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:48.084 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:48.084+0000 7faf69000640 1 --2- 172.21.15.72:0/2900170921 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7faf480a3bb0 0x7faf480a6070 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7faf641038b0 tx=0x7faf54005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:48.087 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:48.087+0000 7faf62000640 1 -- 172.21.15.72:0/2900170921 <== mon.1 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7faf4c0bd270 con 0x7faf64103a50 2024-04-26T21:52:48.308 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:48.308+0000 7faf6a400640 1 -- 172.21.15.72:0/2900170921 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7faf641a3430 con 0x7faf64103a50 2024-04-26T21:52:48.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:48 smithi076 ceph-mon[31823]: from='client.? 172.21.15.72:0/1352254668' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T21:52:48.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:48 smithi076 ceph-mon[31823]: pgmap v362: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:52:48.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:48 smithi076 ceph-mon[31823]: from='client.? 172.21.15.72:0/2185352204' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-26T21:52:48.321 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:48 smithi072 ceph-mon[28138]: from='client.? 172.21.15.72:0/1352254668' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T21:52:48.321 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:48 smithi072 ceph-mon[28138]: pgmap v362: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:52:48.321 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:48 smithi072 ceph-mon[28138]: from='client.? 172.21.15.72:0/2185352204' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-26T21:52:48.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:48 smithi164 ceph-mon[31916]: from='client.? 172.21.15.72:0/1352254668' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T21:52:48.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:48 smithi164 ceph-mon[31916]: pgmap v362: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:52:48.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:48 smithi164 ceph-mon[31916]: from='client.? 172.21.15.72:0/2185352204' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-26T21:52:49.067 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:49.067+0000 7faf62000640 1 -- 172.21.15.72:0/2900170921 <== mon.1 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v84) ==== 138+0+0 (secure 0 0 0) 0x7faf4c0c1570 con 0x7faf64103a50 2024-04-26T21:52:49.068 INFO:teuthology.orchestra.run.smithi072.stderr:pool 'cephfs_metadata' created 2024-04-26T21:52:49.071 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:49.071+0000 7faf6a400640 1 -- 172.21.15.72:0/2900170921 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7faf480a3bb0 msgr2=0x7faf480a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:49.071 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:49.071+0000 7faf6a400640 1 --2- 172.21.15.72:0/2900170921 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7faf480a3bb0 0x7faf480a6070 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7faf641038b0 tx=0x7faf54005f90 comp rx=0 tx=0).stop 2024-04-26T21:52:49.071 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:49.071+0000 7faf6a400640 1 -- 172.21.15.72:0/2900170921 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faf64103a50 msgr2=0x7faf6419b110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:49.072 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:49.071+0000 7faf6a400640 1 --2- 172.21.15.72:0/2900170921 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faf64103a50 0x7faf6419b110 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7faf4c031460 tx=0x7faf4c003c10 comp rx=0 tx=0).stop 2024-04-26T21:52:49.072 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:49.072+0000 7faf6a400640 1 -- 172.21.15.72:0/2900170921 shutdown_connections 2024-04-26T21:52:49.072 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:49.072+0000 7faf6a400640 1 --2- 172.21.15.72:0/2900170921 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7faf64104410 0x7faf641a2180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:49.073 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:49.072+0000 7faf6a400640 1 --2- 172.21.15.72:0/2900170921 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7faf480a3bb0 0x7faf480a6070 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:49.073 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:49.072+0000 7faf6a400640 1 --2- 172.21.15.72:0/2900170921 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faf64103a50 0x7faf6419b110 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:49.073 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:49.072+0000 7faf6a400640 1 --2- 172.21.15.72:0/2900170921 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faf64102860 0x7faf6419abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:49.073 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:49.072+0000 7faf6a400640 1 -- 172.21.15.72:0/2900170921 >> 172.21.15.72:0/2900170921 conn(0x7faf640fddd0 msgr2=0x7faf641091f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:49.074 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:49.074+0000 7faf6a400640 1 -- 172.21.15.72:0/2900170921 shutdown_connections 2024-04-26T21:52:49.076 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:49.076+0000 7faf6a400640 1 -- 172.21.15.72:0/2900170921 wait complete. 2024-04-26T21:52:49.097 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-26T21:52:49.288 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:49.287+0000 7f3d5aa00640 1 -- 172.21.15.72:0/36557955 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3d5410ee30 msgr2=0x7f3d54111220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:49.288 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:49.287+0000 7f3d5aa00640 1 --2- 172.21.15.72:0/36557955 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3d5410ee30 0x7f3d54111220 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f3d4c009cd0 tx=0x7f3d4c02a8e0 comp rx=0 tx=0).stop 2024-04-26T21:52:49.288 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:49.287+0000 7f3d5aa00640 1 -- 172.21.15.72:0/36557955 shutdown_connections 2024-04-26T21:52:49.288 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:49.287+0000 7f3d5aa00640 1 --2- 172.21.15.72:0/36557955 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3d5410ee30 0x7f3d54111220 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:49.289 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:49.287+0000 7f3d5aa00640 1 --2- 172.21.15.72:0/36557955 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3d54075390 0x7f3d54075810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:49.289 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:49.287+0000 7f3d5aa00640 1 --2- 172.21.15.72:0/36557955 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3d54076970 0x7f3d54074dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:49.289 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:49.287+0000 7f3d5aa00640 1 -- 172.21.15.72:0/36557955 >> 172.21.15.72:0/36557955 conn(0x7f3d540fddd0 msgr2=0x7f3d54100210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:49.289 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:49.288+0000 7f3d5aa00640 1 -- 172.21.15.72:0/36557955 shutdown_connections 2024-04-26T21:52:49.289 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:49.288+0000 7f3d5aa00640 1 -- 172.21.15.72:0/36557955 wait complete. 2024-04-26T21:52:49.291 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:49.288+0000 7f3d5aa00640 1 Processor -- start 2024-04-26T21:52:49.291 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:49.289+0000 7f3d5aa00640 1 -- start start 2024-04-26T21:52:49.291 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:49.289+0000 7f3d5aa00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3d54075390 0x7f3d5419eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:49.291 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:49.289+0000 7f3d5aa00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3d54076970 0x7f3d5419f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:49.291 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:49.289+0000 7f3d5aa00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3d5410ee30 0x7f3d541a6480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:49.291 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:49.289+0000 7f3d5aa00640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f3d54114860 con 0x7f3d54076970 2024-04-26T21:52:49.291 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:49.289+0000 7f3d5aa00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f3d541146e0 con 0x7f3d5410ee30 2024-04-26T21:52:49.291 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:49.289+0000 7f3d5aa00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f3d541149e0 con 0x7f3d54075390 2024-04-26T21:52:49.291 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:49.289+0000 7f3d58c00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3d54076970 0x7f3d5419f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:49.292 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:49.289+0000 7f3d58c00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3d54076970 0x7f3d5419f410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:45298/0 (socket says 172.21.15.72:45298) 2024-04-26T21:52:49.292 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:49.289+0000 7f3d58c00640 1 -- 172.21.15.72:0/1167581152 learned_addr learned my addr 172.21.15.72:0/1167581152 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:52:49.292 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:49.290+0000 7f3d59600640 1 --2- 172.21.15.72:0/1167581152 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3d54075390 0x7f3d5419eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:49.292 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:49.290+0000 7f3d5a000640 1 --2- 172.21.15.72:0/1167581152 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3d5410ee30 0x7f3d541a6480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:49.292 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:49.290+0000 7f3d58c00640 1 -- 172.21.15.72:0/1167581152 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3d54075390 msgr2=0x7f3d5419eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:49.292 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:49.290+0000 7f3d58c00640 1 --2- 172.21.15.72:0/1167581152 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3d54075390 0x7f3d5419eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:49.292 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:49.290+0000 7f3d58c00640 1 -- 172.21.15.72:0/1167581152 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3d5410ee30 msgr2=0x7f3d541a6480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:49.292 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:49.290+0000 7f3d58c00640 1 --2- 172.21.15.72:0/1167581152 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3d5410ee30 0x7f3d541a6480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:49.292 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:49.290+0000 7f3d58c00640 1 -- 172.21.15.72:0/1167581152 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d541a6af0 con 0x7f3d54076970 2024-04-26T21:52:49.292 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:49.290+0000 7f3d58c00640 1 --2- 172.21.15.72:0/1167581152 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3d54076970 0x7f3d5419f410 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f3d4000c920 tx=0x7f3d4000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:49.292 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:49.290+0000 7f3d52000640 1 -- 172.21.15.72:0/1167581152 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d40007c70 con 0x7f3d54076970 2024-04-26T21:52:49.292 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:49.290+0000 7f3d52000640 1 -- 172.21.15.72:0/1167581152 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3d40007e10 con 0x7f3d54076970 2024-04-26T21:52:49.292 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:49.290+0000 7f3d52000640 1 -- 172.21.15.72:0/1167581152 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d40018710 con 0x7f3d54076970 2024-04-26T21:52:49.292 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:49.290+0000 7f3d5aa00640 1 -- 172.21.15.72:0/1167581152 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d541a6de0 con 0x7f3d54076970 2024-04-26T21:52:49.293 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:49.291+0000 7f3d5aa00640 1 -- 172.21.15.72:0/1167581152 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d541a72a0 con 0x7f3d54076970 2024-04-26T21:52:49.293 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:49.292+0000 7f3d5aa00640 1 -- 172.21.15.72:0/1167581152 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d54105000 con 0x7f3d54076970 2024-04-26T21:52:49.296 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:49.296+0000 7f3d52000640 1 -- 172.21.15.72:0/1167581152 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f3d4000f450 con 0x7f3d54076970 2024-04-26T21:52:49.297 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:49.297+0000 7f3d52000640 1 --2- 172.21.15.72:0/1167581152 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f3d3c0a3bb0 0x7f3d3c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:49.297 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:49.297+0000 7f3d52000640 1 -- 172.21.15.72:0/1167581152 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7497+0+0 (secure 0 0 0) 0x7f3d400d0640 con 0x7f3d54076970 2024-04-26T21:52:49.298 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:49.297+0000 7f3d59600640 1 --2- 172.21.15.72:0/1167581152 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f3d3c0a3bb0 0x7f3d3c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:49.298 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:49.298+0000 7f3d59600640 1 --2- 172.21.15.72:0/1167581152 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f3d3c0a3bb0 0x7f3d3c0a6070 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f3d48005fd0 tx=0x7f3d48005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:49.301 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:49.301+0000 7f3d52000640 1 -- 172.21.15.72:0/1167581152 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f3d40014030 con 0x7f3d54076970 2024-04-26T21:52:49.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:49 smithi076 ceph-mon[31823]: from='client.? 172.21.15.72:0/2900170921' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-26T21:52:49.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:49 smithi076 ceph-mon[31823]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-26T21:52:49.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:49 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:52:49.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:49 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:52:49.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:49 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:52:49.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:49 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:52:49.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:49 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:52:49.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:49 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:52:49.321 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:49 smithi072 ceph-mon[28138]: from='client.? 172.21.15.72:0/2900170921' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-26T21:52:49.321 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:49 smithi072 ceph-mon[28138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-26T21:52:49.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:49 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:52:49.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:49 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:52:49.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:49 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:52:49.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:49 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:52:49.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:49 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:52:49.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:49 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:52:49.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:49 smithi164 ceph-mon[31916]: from='client.? 172.21.15.72:0/2900170921' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-26T21:52:49.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:49 smithi164 ceph-mon[31916]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-26T21:52:49.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:49 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:52:49.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:49 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:52:49.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:49 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:52:49.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:49 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:52:49.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:49 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:52:49.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:49 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:52:49.507 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:49.507+0000 7f3d5aa00640 1 -- 172.21.15.72:0/1167581152 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f3d5402dad0 con 0x7f3d54076970 2024-04-26T21:52:50.070 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:50.069+0000 7f3d52000640 1 -- 172.21.15.72:0/1167581152 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v85) ==== 170+0+0 (secure 0 0 0) 0x7f3d4000de20 con 0x7f3d54076970 2024-04-26T21:52:50.070 INFO:teuthology.orchestra.run.smithi072.stderr:pool 'cephfs_data' created 2024-04-26T21:52:50.074 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:50.074+0000 7f3d5aa00640 1 -- 172.21.15.72:0/1167581152 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f3d3c0a3bb0 msgr2=0x7f3d3c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:50.074 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:50.074+0000 7f3d5aa00640 1 --2- 172.21.15.72:0/1167581152 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f3d3c0a3bb0 0x7f3d3c0a6070 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f3d48005fd0 tx=0x7f3d48005850 comp rx=0 tx=0).stop 2024-04-26T21:52:50.074 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:50.074+0000 7f3d5aa00640 1 -- 172.21.15.72:0/1167581152 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3d54076970 msgr2=0x7f3d5419f410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:50.074 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:50.074+0000 7f3d5aa00640 1 --2- 172.21.15.72:0/1167581152 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3d54076970 0x7f3d5419f410 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f3d4000c920 tx=0x7f3d4000cde0 comp rx=0 tx=0).stop 2024-04-26T21:52:50.074 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:50.074+0000 7f3d5aa00640 1 -- 172.21.15.72:0/1167581152 shutdown_connections 2024-04-26T21:52:50.074 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:50.074+0000 7f3d5aa00640 1 --2- 172.21.15.72:0/1167581152 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3d5410ee30 0x7f3d541a6480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:50.074 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:50.074+0000 7f3d5aa00640 1 --2- 172.21.15.72:0/1167581152 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3d54076970 0x7f3d5419f410 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:50.075 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:50.074+0000 7f3d5aa00640 1 --2- 172.21.15.72:0/1167581152 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f3d3c0a3bb0 0x7f3d3c0a6070 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:50.075 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:50.074+0000 7f3d5aa00640 1 --2- 172.21.15.72:0/1167581152 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3d54075390 0x7f3d5419eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:50.075 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:50.074+0000 7f3d5aa00640 1 -- 172.21.15.72:0/1167581152 >> 172.21.15.72:0/1167581152 conn(0x7f3d540fddd0 msgr2=0x7f3d541021d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:50.075 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:50.075+0000 7f3d5aa00640 1 -- 172.21.15.72:0/1167581152 shutdown_connections 2024-04-26T21:52:50.075 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:50.075+0000 7f3d5aa00640 1 -- 172.21.15.72:0/1167581152 wait complete. 2024-04-26T21:52:50.113 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-26T21:52:50.294 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:50.292+0000 7f36cec00640 1 -- 172.21.15.72:0/1667878701 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f36c8103410 msgr2=0x7f36c8105800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:50.294 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:50.292+0000 7f36cec00640 1 --2- 172.21.15.72:0/1667878701 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f36c8103410 0x7f36c8105800 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f36b8009900 tx=0x7f36b802a410 comp rx=0 tx=0).stop 2024-04-26T21:52:50.294 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:50.292+0000 7f36cec00640 1 -- 172.21.15.72:0/1667878701 shutdown_connections 2024-04-26T21:52:50.294 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:50.292+0000 7f36cec00640 1 --2- 172.21.15.72:0/1667878701 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f36c8108670 0x7f36c810aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:50.294 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:50.292+0000 7f36cec00640 1 --2- 172.21.15.72:0/1667878701 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f36c8105d40 0x7f36c8108130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:50.294 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:50.292+0000 7f36cec00640 1 --2- 172.21.15.72:0/1667878701 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f36c8103410 0x7f36c8105800 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:50.294 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:50.292+0000 7f36cec00640 1 -- 172.21.15.72:0/1667878701 >> 172.21.15.72:0/1667878701 conn(0x7f36c80fce00 msgr2=0x7f36c80ff240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:50.294 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:50.292+0000 7f36cec00640 1 -- 172.21.15.72:0/1667878701 shutdown_connections 2024-04-26T21:52:50.294 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:50.292+0000 7f36cec00640 1 -- 172.21.15.72:0/1667878701 wait complete. 2024-04-26T21:52:50.296 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:50.293+0000 7f36cec00640 1 Processor -- start 2024-04-26T21:52:50.297 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:50.293+0000 7f36cec00640 1 -- start start 2024-04-26T21:52:50.297 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:50.294+0000 7f36cec00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f36c8103410 0x7f36c819ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:50.297 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:50.294+0000 7f36cec00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f36c8105d40 0x7f36c819b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:50.297 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:50.294+0000 7f36cec00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f36c8108670 0x7f36c81a22f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:50.297 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:50.294+0000 7f36cec00640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f36c810e3e0 con 0x7f36c8105d40 2024-04-26T21:52:50.297 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:50.294+0000 7f36cec00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f36c810e260 con 0x7f36c8103410 2024-04-26T21:52:50.297 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:50.294+0000 7f36cec00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f36c810e560 con 0x7f36c8108670 2024-04-26T21:52:50.297 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:50.294+0000 7f36c7400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f36c8105d40 0x7f36c819b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:50.297 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:50.294+0000 7f36c7400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f36c8105d40 0x7f36c819b280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:45314/0 (socket says 172.21.15.72:45314) 2024-04-26T21:52:50.297 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:50.294+0000 7f36c7400640 1 -- 172.21.15.72:0/2258182875 learned_addr learned my addr 172.21.15.72:0/2258182875 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:52:50.297 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:50.294+0000 7f36c7e00640 1 --2- 172.21.15.72:0/2258182875 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f36c8103410 0x7f36c819ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:50.297 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:50.294+0000 7f36ccc00640 1 --2- 172.21.15.72:0/2258182875 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f36c8108670 0x7f36c81a22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:50.298 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:50.295+0000 7f36c7400640 1 -- 172.21.15.72:0/2258182875 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f36c8108670 msgr2=0x7f36c81a22f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:50.298 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:50.295+0000 7f36c7400640 1 --2- 172.21.15.72:0/2258182875 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f36c8108670 0x7f36c81a22f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:50.298 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:50.295+0000 7f36c7400640 1 -- 172.21.15.72:0/2258182875 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f36c8103410 msgr2=0x7f36c819ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:50.298 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:50.295+0000 7f36c7400640 1 --2- 172.21.15.72:0/2258182875 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f36c8103410 0x7f36c819ad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:50.298 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:50.295+0000 7f36c7400640 1 -- 172.21.15.72:0/2258182875 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36c81a2960 con 0x7f36c8105d40 2024-04-26T21:52:50.298 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:50.295+0000 7f36c7400640 1 --2- 172.21.15.72:0/2258182875 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f36c8105d40 0x7f36c819b280 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f36b400d770 tx=0x7f36b400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:50.298 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:50.295+0000 7f36c4c00640 1 -- 172.21.15.72:0/2258182875 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36b4017070 con 0x7f36c8105d40 2024-04-26T21:52:50.298 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:50.295+0000 7f36c4c00640 1 -- 172.21.15.72:0/2258182875 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f36b4009c30 con 0x7f36c8105d40 2024-04-26T21:52:50.298 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:50.295+0000 7f36cec00640 1 -- 172.21.15.72:0/2258182875 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36c81a2c50 con 0x7f36c8105d40 2024-04-26T21:52:50.298 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:50.295+0000 7f36c4c00640 1 -- 172.21.15.72:0/2258182875 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36b400f620 con 0x7f36c8105d40 2024-04-26T21:52:50.299 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:50.295+0000 7f36cec00640 1 -- 172.21.15.72:0/2258182875 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36c81a31a0 con 0x7f36c8105d40 2024-04-26T21:52:50.299 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:50.297+0000 7f36cec00640 1 -- 172.21.15.72:0/2258182875 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36c810a1c0 con 0x7f36c8105d40 2024-04-26T21:52:50.300 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:50.299+0000 7f36c4c00640 1 -- 172.21.15.72:0/2258182875 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f36b401a070 con 0x7f36c8105d40 2024-04-26T21:52:50.302 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:50.301+0000 7f36c4c00640 1 --2- 172.21.15.72:0/2258182875 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f36ac0a3c30 0x7f36ac0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:50.303 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:50.302+0000 7f36c4c00640 1 -- 172.21.15.72:0/2258182875 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7879+0+0 (secure 0 0 0) 0x7f36b40cfbe0 con 0x7f36c8105d40 2024-04-26T21:52:50.303 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:50.302+0000 7f36c7e00640 1 --2- 172.21.15.72:0/2258182875 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f36ac0a3c30 0x7f36ac0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:50.303 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:50.302+0000 7f36c7e00640 1 --2- 172.21.15.72:0/2258182875 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f36ac0a3c30 0x7f36ac0a60f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f36b8005ab0 tx=0x7f36b8005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:50.306 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:50.305+0000 7f36c4c00640 1 -- 172.21.15.72:0/2258182875 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f36b4009de0 con 0x7f36c8105d40 2024-04-26T21:52:50.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:50 smithi164 ceph-mon[31916]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-26T21:52:50.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:50 smithi164 ceph-mon[31916]: osdmap e84: 12 total, 12 up, 12 in 2024-04-26T21:52:50.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:50 smithi164 ceph-mon[31916]: from='client.? 172.21.15.72:0/1167581152' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-26T21:52:50.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:50 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:52:50.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:50 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:52:50.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:50 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:52:50.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:50 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:52:50.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:50 smithi164 ceph-mon[31916]: pgmap v364: 33 pgs: 3 creating+peering, 1 active+clean, 29 unknown; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:52:50.565 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:50 smithi076 ceph-mon[31823]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-26T21:52:50.565 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:50 smithi076 ceph-mon[31823]: osdmap e84: 12 total, 12 up, 12 in 2024-04-26T21:52:50.565 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:50 smithi076 ceph-mon[31823]: from='client.? 172.21.15.72:0/1167581152' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-26T21:52:50.565 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:50 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:52:50.565 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:50 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:52:50.565 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:50 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:52:50.565 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:50 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:52:50.565 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:50 smithi076 ceph-mon[31823]: pgmap v364: 33 pgs: 3 creating+peering, 1 active+clean, 29 unknown; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:52:50.566 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:50.565+0000 7f36cec00640 1 -- 172.21.15.72:0/2258182875 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f36c802dad0 con 0x7f36c8105d40 2024-04-26T21:52:50.571 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:50 smithi072 ceph-mon[28138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-26T21:52:50.571 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:50 smithi072 ceph-mon[28138]: osdmap e84: 12 total, 12 up, 12 in 2024-04-26T21:52:50.571 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:50 smithi072 ceph-mon[28138]: from='client.? 172.21.15.72:0/1167581152' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-26T21:52:50.572 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:50 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:52:50.572 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:50 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:52:50.572 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:50 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:52:50.572 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:50 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:52:50.572 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:50 smithi072 ceph-mon[28138]: pgmap v364: 33 pgs: 3 creating+peering, 1 active+clean, 29 unknown; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:52:51.097 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:51.096+0000 7f36c4c00640 1 -- 172.21.15.72:0/2258182875 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7f36b409c1a0 con 0x7f36c8105d40 2024-04-26T21:52:51.098 INFO:teuthology.orchestra.run.smithi072.stderr:new fs with metadata pool 2 and data pool 3 2024-04-26T21:52:51.100 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:51.098+0000 7f36cec00640 1 -- 172.21.15.72:0/2258182875 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f36ac0a3c30 msgr2=0x7f36ac0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:51.100 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:51.098+0000 7f36cec00640 1 --2- 172.21.15.72:0/2258182875 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f36ac0a3c30 0x7f36ac0a60f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f36b8005ab0 tx=0x7f36b8005a20 comp rx=0 tx=0).stop 2024-04-26T21:52:51.100 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:51.098+0000 7f36cec00640 1 -- 172.21.15.72:0/2258182875 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f36c8105d40 msgr2=0x7f36c819b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:51.100 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:51.098+0000 7f36cec00640 1 --2- 172.21.15.72:0/2258182875 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f36c8105d40 0x7f36c819b280 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f36b400d770 tx=0x7f36b400dc30 comp rx=0 tx=0).stop 2024-04-26T21:52:51.100 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:51.099+0000 7f36cec00640 1 -- 172.21.15.72:0/2258182875 shutdown_connections 2024-04-26T21:52:51.100 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:51.099+0000 7f36cec00640 1 --2- 172.21.15.72:0/2258182875 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f36ac0a3c30 0x7f36ac0a60f0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:51.100 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:51.099+0000 7f36cec00640 1 --2- 172.21.15.72:0/2258182875 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f36c8108670 0x7f36c81a22f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:51.100 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:51.099+0000 7f36cec00640 1 --2- 172.21.15.72:0/2258182875 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f36c8105d40 0x7f36c819b280 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:51.100 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:51.099+0000 7f36cec00640 1 --2- 172.21.15.72:0/2258182875 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f36c8103410 0x7f36c819ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:51.101 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:51.099+0000 7f36cec00640 1 -- 172.21.15.72:0/2258182875 >> 172.21.15.72:0/2258182875 conn(0x7f36c80fce00 msgr2=0x7f36c8102b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:51.101 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:51.099+0000 7f36cec00640 1 -- 172.21.15.72:0/2258182875 shutdown_connections 2024-04-26T21:52:51.101 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:51.099+0000 7f36cec00640 1 -- 172.21.15.72:0/2258182875 wait complete. 2024-04-26T21:52:51.122 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:52:51.289 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:51.287+0000 7f784a800640 1 -- 172.21.15.72:0/2104479536 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7844106720 msgr2=0x7f784410cfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:51.289 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:51.287+0000 7f784a800640 1 --2- 172.21.15.72:0/2104479536 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7844106720 0x7f784410cfb0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f783c009ca0 tx=0x7f783c02a8c0 comp rx=0 tx=0).stop 2024-04-26T21:52:51.289 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:51.287+0000 7f784a800640 1 -- 172.21.15.72:0/2104479536 shutdown_connections 2024-04-26T21:52:51.289 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:51.287+0000 7f784a800640 1 --2- 172.21.15.72:0/2104479536 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7844106720 0x7f784410cfb0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:51.289 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:51.287+0000 7f784a800640 1 --2- 172.21.15.72:0/2104479536 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7844105d60 0x7f78441061e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:51.289 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:51.287+0000 7f784a800640 1 --2- 172.21.15.72:0/2104479536 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7844104b70 0x7f7844104f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:51.289 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:51.287+0000 7f784a800640 1 -- 172.21.15.72:0/2104479536 >> 172.21.15.72:0/2104479536 conn(0x7f7844100120 msgr2=0x7f7844102540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:51.289 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:51.287+0000 7f784a800640 1 -- 172.21.15.72:0/2104479536 shutdown_connections 2024-04-26T21:52:51.289 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:51.287+0000 7f784a800640 1 -- 172.21.15.72:0/2104479536 wait complete. 2024-04-26T21:52:51.291 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:51.288+0000 7f784a800640 1 Processor -- start 2024-04-26T21:52:51.292 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:51.288+0000 7f784a800640 1 -- start start 2024-04-26T21:52:51.292 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:51.289+0000 7f784a800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7844104b70 0x7f784419ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:51.292 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:51.289+0000 7f784a800640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7844105d60 0x7f784419b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:51.292 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:51.289+0000 7f784a800640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7844106720 0x7f78441a2310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:51.292 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:51.289+0000 7f784a800640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f7844110690 con 0x7f7844105d60 2024-04-26T21:52:51.292 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:51.289+0000 7f784a800640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f7844110510 con 0x7f7844106720 2024-04-26T21:52:51.292 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:51.289+0000 7f784a800640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f7844110810 con 0x7f7844104b70 2024-04-26T21:52:51.292 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:51.289+0000 7f7842a00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7844105d60 0x7f784419b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:51.292 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:51.289+0000 7f7842a00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7844105d60 0x7f784419b2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:35000/0 (socket says 172.21.15.72:35000) 2024-04-26T21:52:51.292 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:51.289+0000 7f7842a00640 1 -- 172.21.15.72:0/2735163000 learned_addr learned my addr 172.21.15.72:0/2735163000 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:52:51.292 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:51.289+0000 7f7843400640 1 --2- 172.21.15.72:0/2735163000 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7844104b70 0x7f784419ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:51.292 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:51.289+0000 7f7843e00640 1 --2- 172.21.15.72:0/2735163000 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7844106720 0x7f78441a2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:51.292 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:51.289+0000 7f7842a00640 1 -- 172.21.15.72:0/2735163000 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7844104b70 msgr2=0x7f784419ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:51.292 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:51.289+0000 7f7842a00640 1 --2- 172.21.15.72:0/2735163000 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7844104b70 0x7f784419ad60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:51.293 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:51.289+0000 7f7842a00640 1 -- 172.21.15.72:0/2735163000 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7844106720 msgr2=0x7f78441a2310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:51.293 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:51.289+0000 7f7842a00640 1 --2- 172.21.15.72:0/2735163000 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7844106720 0x7f78441a2310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:51.293 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:51.289+0000 7f7842a00640 1 -- 172.21.15.72:0/2735163000 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78441a2980 con 0x7f7844105d60 2024-04-26T21:52:51.293 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:51.290+0000 7f7842a00640 1 --2- 172.21.15.72:0/2735163000 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7844105d60 0x7f784419b2a0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f783000b4a0 tx=0x7f783000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:51.293 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:51.290+0000 7f783be00640 1 -- 172.21.15.72:0/2735163000 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7830014070 con 0x7f7844105d60 2024-04-26T21:52:51.293 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:51.290+0000 7f783be00640 1 -- 172.21.15.72:0/2735163000 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7830010aa0 con 0x7f7844105d60 2024-04-26T21:52:51.293 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:51.290+0000 7f784a800640 1 -- 172.21.15.72:0/2735163000 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78441a2c70 con 0x7f7844105d60 2024-04-26T21:52:51.293 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:51.290+0000 7f783be00640 1 -- 172.21.15.72:0/2735163000 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7830017b70 con 0x7f7844105d60 2024-04-26T21:52:51.293 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:51.290+0000 7f784a800640 1 -- 172.21.15.72:0/2735163000 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78441a3240 con 0x7f7844105d60 2024-04-26T21:52:51.293 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:51.291+0000 7f784a800640 1 -- 172.21.15.72:0/2735163000 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f784410bf60 con 0x7f7844105d60 2024-04-26T21:52:51.294 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:51.294+0000 7f783be00640 1 -- 172.21.15.72:0/2735163000 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f7830017070 con 0x7f7844105d60 2024-04-26T21:52:51.297 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:51.296+0000 7f783be00640 1 --2- 172.21.15.72:0/2735163000 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f78280a3c30 0x7f78280a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:51.297 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:51.296+0000 7f783be00640 1 -- 172.21.15.72:0/2735163000 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f78300cfa70 con 0x7f7844105d60 2024-04-26T21:52:51.297 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:51.296+0000 7f7843400640 1 --2- 172.21.15.72:0/2735163000 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f78280a3c30 0x7f78280a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:51.297 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:51.296+0000 7f7843400640 1 --2- 172.21.15.72:0/2735163000 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f78280a3c30 0x7f78280a60f0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f7844105bc0 tx=0x7f7834008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:51.301 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:51.300+0000 7f783be00640 1 -- 172.21.15.72:0/2735163000 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f783009c500 con 0x7f7844105d60 2024-04-26T21:52:51.357 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:51 smithi072 ceph-mon[28138]: from='client.? 172.21.15.72:0/1167581152' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-26T21:52:51.357 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:51 smithi072 ceph-mon[28138]: osdmap e85: 12 total, 12 up, 12 in 2024-04-26T21:52:51.357 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:51 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:52:51.357 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:51 smithi072 ceph-mon[28138]: from='client.? 172.21.15.72:0/2258182875' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-26T21:52:51.357 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:51 smithi072 ceph-mon[28138]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-26T21:52:51.357 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:51 smithi072 ceph-mon[28138]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-26T21:52:51.357 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:51 smithi072 ceph-mon[28138]: osdmap e86: 12 total, 12 up, 12 in 2024-04-26T21:52:51.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:51 smithi164 ceph-mon[31916]: from='client.? 172.21.15.72:0/1167581152' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-26T21:52:51.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:51 smithi164 ceph-mon[31916]: osdmap e85: 12 total, 12 up, 12 in 2024-04-26T21:52:51.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:51 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:52:51.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:51 smithi164 ceph-mon[31916]: from='client.? 172.21.15.72:0/2258182875' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-26T21:52:51.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:51 smithi164 ceph-mon[31916]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-26T21:52:51.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:51 smithi164 ceph-mon[31916]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-26T21:52:51.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:51 smithi164 ceph-mon[31916]: osdmap e86: 12 total, 12 up, 12 in 2024-04-26T21:52:51.453 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:51 smithi076 ceph-mon[31823]: from='client.? 172.21.15.72:0/1167581152' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-26T21:52:51.453 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:51 smithi076 ceph-mon[31823]: osdmap e85: 12 total, 12 up, 12 in 2024-04-26T21:52:51.453 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:51 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:52:51.453 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:51 smithi076 ceph-mon[31823]: from='client.? 172.21.15.72:0/2258182875' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-26T21:52:51.453 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:51 smithi076 ceph-mon[31823]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-26T21:52:51.453 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:51 smithi076 ceph-mon[31823]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-26T21:52:51.453 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:51 smithi076 ceph-mon[31823]: osdmap e86: 12 total, 12 up, 12 in 2024-04-26T21:52:51.505 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:51.504+0000 7f784a800640 1 -- 172.21.15.72:0/2735163000 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7844063530 con 0x7f7844105d60 2024-04-26T21:52:51.508 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:51.506+0000 7f783be00640 1 -- 172.21.15.72:0/2735163000 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+22185 (secure 0 0 0) 0x7f783009be00 con 0x7f7844105d60 2024-04-26T21:52:51.508 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T21:52:51.508 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":86,"fsid":"a1780950-0415-11ef-bc93-c7b262605968","created":"2024-04-26T21:42:01.504173+0000","modified":"2024-04-26T21:52:50.567988+0000","last_up_change":"2024-04-26T21:51:43.482437+0000","last_in_change":"2024-04-26T21:51:18.468587+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:46:29.785427+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:52:48.320391+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T21:52:49.517493+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"04e06884-b355-4762-9b29-9034b7201e9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6802","nonce":1416765179},{"type":"v1","addr":"172.21.15.72:6803","nonce":1416765179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6804","nonce":1416765179},{"type":"v1","addr":"172.21.15.72:6805","nonce":1416765179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6808","nonce":1416765179},{"type":"v1","addr":"172.21.15.72:6809","nonce":1416765179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6806","nonce":1416765179},{"type":"v1","addr":"172.21.15.72:6807","nonce":1416765179}]},"public_addr":"172.21.15.72:6803/1416765179","cluster_addr":"172.21.15.72:6805/1416765179","heartbeat_back_addr":"172.21.15.72:6809/1416765179","heartbeat_front_addr":"172.21.15.72:6807/1416765179","state":["exists","up"]},{"osd":1,"uuid":"d75a479a-4a2e-48e9-a01e-2b1c9617416a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":755426949},{"type":"v1","addr":"172.21.15.76:6801","nonce":755426949}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":755426949},{"type":"v1","addr":"172.21.15.76:6803","nonce":755426949}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":755426949},{"type":"v1","addr":"172.21.15.76:6807","nonce":755426949}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":755426949},{"type":"v1","addr":"172.21.15.76:6805","nonce":755426949}]},"public_addr":"172.21.15.76:6801/755426949","cluster_addr":"172.21.15.76:6803/755426949","heartbeat_back_addr":"172.21.15.76:6807/755426949","heartbeat_front_addr":"172.21.15.76:6805/755426949","state":["exists","up"]},{"osd":2,"uuid":"b6f8da58-4428-4e8c-a944-7ea5645d6d92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":852265973},{"type":"v1","addr":"172.21.15.164:6801","nonce":852265973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":852265973},{"type":"v1","addr":"172.21.15.164:6803","nonce":852265973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":852265973},{"type":"v1","addr":"172.21.15.164:6807","nonce":852265973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":852265973},{"type":"v1","addr":"172.21.15.164:6805","nonce":852265973}]},"public_addr":"172.21.15.164:6801/852265973","cluster_addr":"172.21.15.164:6803/852265973","heartbeat_back_addr":"172.21.15.164:6807/852265973","heartbeat_front_addr":"172.21.15.164:6805/852265973","state":["exists","up"]},{"osd":3,"uuid":"76d83409-649e-41e5-b4f4-43bbd5b00ea5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6810","nonce":4027763789},{"type":"v1","addr":"172.21.15.72:6811","nonce":4027763789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6812","nonce":4027763789},{"type":"v1","addr":"172.21.15.72:6813","nonce":4027763789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6816","nonce":4027763789},{"type":"v1","addr":"172.21.15.72:6817","nonce":4027763789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6814","nonce":4027763789},{"type":"v1","addr":"172.21.15.72:6815","nonce":4027763789}]},"public_addr":"172.21.15.72:6811/4027763789","cluster_addr":"172.21.15.72:6813/4027763789","heartbeat_back_addr":"172.21.15.72:6817/4027763789","heartbeat_front_addr":"172.21.15.72:6815/4027763789","state":["exists","up"]},{"osd":4,"uuid":"d2cfeea7-1661-4156-bdcd-f770a8d1a878","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":3718867827},{"type":"v1","addr":"172.21.15.76:6809","nonce":3718867827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":3718867827},{"type":"v1","addr":"172.21.15.76:6811","nonce":3718867827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":3718867827},{"type":"v1","addr":"172.21.15.76:6815","nonce":3718867827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":3718867827},{"type":"v1","addr":"172.21.15.76:6813","nonce":3718867827}]},"public_addr":"172.21.15.76:6809/3718867827","cluster_addr":"172.21.15.76:6811/3718867827","heartbeat_back_addr":"172.21.15.76:6815/3718867827","heartbeat_front_addr":"172.21.15.76:6813/3718867827","state":["exists","up"]},{"osd":5,"uuid":"f3896ad6-a040-4ea3-a431-ff9b7fc05c36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":2543954772},{"type":"v1","addr":"172.21.15.164:6809","nonce":2543954772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":2543954772},{"type":"v1","addr":"172.21.15.164:6811","nonce":2543954772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":2543954772},{"type":"v1","addr":"172.21.15.164:6815","nonce":2543954772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":2543954772},{"type":"v1","addr":"172.21.15.164:6813","nonce":2543954772}]},"public_addr":"172.21.15.164:6809/2543954772","cluster_addr":"172.21.15.164:6811/2543954772","heartbeat_back_addr":"172.21.15.164:6815/2543954772","heartbeat_front_addr":"172.21.15.164:6813/2543954772","state":["exists","up"]},{"osd":6,"uuid":"d5d56650-be7e-4ba5-ba3a-d7d473294dab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6818","nonce":2080659027},{"type":"v1","addr":"172.21.15.72:6819","nonce":2080659027}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6820","nonce":2080659027},{"type":"v1","addr":"172.21.15.72:6821","nonce":2080659027}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6824","nonce":2080659027},{"type":"v1","addr":"172.21.15.72:6825","nonce":2080659027}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6822","nonce":2080659027},{"type":"v1","addr":"172.21.15.72:6823","nonce":2080659027}]},"public_addr":"172.21.15.72:6819/2080659027","cluster_addr":"172.21.15.72:6821/2080659027","heartbeat_back_addr":"172.21.15.72:6825/2080659027","heartbeat_front_addr":"172.21.15.72:6823/2080659027","state":["exists","up"]},{"osd":7,"uuid":"18975b5a-9309-4a0f-9f37-3ce09c2e745d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":3144631323},{"type":"v1","addr":"172.21.15.76:6817","nonce":3144631323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":3144631323},{"type":"v1","addr":"172.21.15.76:6819","nonce":3144631323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":3144631323},{"type":"v1","addr":"172.21.15.76:6823","nonce":3144631323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":3144631323},{"type":"v1","addr":"172.21.15.76:6821","nonce":3144631323}]},"public_addr":"172.21.15.76:6817/3144631323","cluster_addr":"172.21.15.76:6819/3144631323","heartbeat_back_addr":"172.21.15.76:6823/3144631323","heartbeat_front_addr":"172.21.15.76:6821/3144631323","state":["exists","up"]},{"osd":8,"uuid":"779e9c1e-8e97-48d4-a854-68ca13f38e0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":627888883},{"type":"v1","addr":"172.21.15.164:6817","nonce":627888883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":627888883},{"type":"v1","addr":"172.21.15.164:6819","nonce":627888883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":627888883},{"type":"v1","addr":"172.21.15.164:6823","nonce":627888883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":627888883},{"type":"v1","addr":"172.21.15.164:6821","nonce":627888883}]},"public_addr":"172.21.15.164:6817/627888883","cluster_addr":"172.21.15.164:6819/627888883","heartbeat_back_addr":"172.21.15.164:6823/627888883","heartbeat_front_addr":"172.21.15.164:6821/627888883","state":["exists","up"]},{"osd":9,"uuid":"d0317850-7b2d-4850-b2cf-0b919505c1fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6826","nonce":1273118485},{"type":"v1","addr":"172.21.15.72:6827","nonce":1273118485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6828","nonce":1273118485},{"type":"v1","addr":"172.21.15.72:6829","nonce":1273118485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6832","nonce":1273118485},{"type":"v1","addr":"172.21.15.72:6833","nonce":1273118485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6830","nonce":1273118485},{"type":"v1","addr":"172.21.15.72:6831","nonce":1273118485}]},"public_addr":"172.21.15.72:6827/1273118485","cluster_addr":"172.21.15.72:6829/1273118485","heartbeat_back_addr":"172.21.15.72:6833/1273118485","heartbeat_front_addr":"172.21.15.72:6831/1273118485","state":["exists","up"]},{"osd":10,"uuid":"e0aa38d8-6011-4d2a-b67b-5bff6d999e47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":2808954459},{"type":"v1","addr":"172.21.15.76:6825","nonce":2808954459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":2808954459},{"type":"v1","addr":"172.21.15.76:6827","nonce":2808954459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":2808954459},{"type":"v1","addr":"172.21.15.76:6831","nonce":2808954459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":2808954459},{"type":"v1","addr":"172.21.15.76:6829","nonce":2808954459}]},"public_addr":"172.21.15.76:6825/2808954459","cluster_addr":"172.21.15.76:6827/2808954459","heartbeat_back_addr":"172.21.15.76:6831/2808954459","heartbeat_front_addr":"172.21.15.76:6829/2808954459","state":["exists","up"]},{"osd":11,"uuid":"981b3d4d-916e-4937-9916-7cc21a891e43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":2354951784},{"type":"v1","addr":"172.21.15.164:6825","nonce":2354951784}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":2354951784},{"type":"v1","addr":"172.21.15.164:6827","nonce":2354951784}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":2354951784},{"type":"v1","addr":"172.21.15.164:6831","nonce":2354951784}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":2354951784},{"type":"v1","addr":"172.21.15.164:6829","nonce":2354951784}]},"public_addr":"172.21.15.164:6825/2354951784","cluster_addr":"172.21.15.164:6827/2354951784","heartbeat_back_addr":"172.21.15.164:6831/2354951784","heartbeat_front_addr":"172.21.15.164:6829/2354951784","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:18.997924+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:52.439081+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:46:25.893288+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:47:00.112807+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:47:35.116874+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:48:09.216344+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:48:45.028853+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:49:19.905397+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:49:55.030536+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:50:30.767662+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:51:06.423267+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:51:41.631211+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.72:0/416277784":"2024-04-27T21:43:09.602791+0000","172.21.15.72:6800/4010358672":"2024-04-27T21:43:09.602791+0000","172.21.15.72:6801/1012617371":"2024-04-27T21:42:34.684837+0000","172.21.15.72:6801/4010358672":"2024-04-27T21:43:09.602791+0000","172.21.15.72:0/3555448179":"2024-04-27T21:43:09.602791+0000","172.21.15.72:6800/1012617371":"2024-04-27T21:42:34.684837+0000","172.21.15.72:0/2920378761":"2024-04-27T21:42:34.684837+0000","172.21.15.72:0/1933740041":"2024-04-27T21:43:09.602791+0000","172.21.15.72:0/3225295775":"2024-04-27T21:42:34.684837+0000","172.21.15.72:0/2326016180":"2024-04-27T21:42:34.684837+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:52:51.510 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:51.509+0000 7f784a800640 1 -- 172.21.15.72:0/2735163000 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f78280a3c30 msgr2=0x7f78280a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:51.510 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:51.509+0000 7f784a800640 1 --2- 172.21.15.72:0/2735163000 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f78280a3c30 0x7f78280a60f0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f7844105bc0 tx=0x7f7834008040 comp rx=0 tx=0).stop 2024-04-26T21:52:51.510 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:51.509+0000 7f784a800640 1 -- 172.21.15.72:0/2735163000 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7844105d60 msgr2=0x7f784419b2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:51.510 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:51.509+0000 7f784a800640 1 --2- 172.21.15.72:0/2735163000 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7844105d60 0x7f784419b2a0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f783000b4a0 tx=0x7f783000b960 comp rx=0 tx=0).stop 2024-04-26T21:52:51.510 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:51.509+0000 7f784a800640 1 -- 172.21.15.72:0/2735163000 shutdown_connections 2024-04-26T21:52:51.510 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:51.509+0000 7f784a800640 1 --2- 172.21.15.72:0/2735163000 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7844106720 0x7f78441a2310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:51.510 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:51.509+0000 7f784a800640 1 --2- 172.21.15.72:0/2735163000 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7844105d60 0x7f784419b2a0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:51.511 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:51.509+0000 7f784a800640 1 --2- 172.21.15.72:0/2735163000 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f78280a3c30 0x7f78280a60f0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:51.511 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:51.509+0000 7f784a800640 1 --2- 172.21.15.72:0/2735163000 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7844104b70 0x7f784419ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:51.511 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:51.509+0000 7f784a800640 1 -- 172.21.15.72:0/2735163000 >> 172.21.15.72:0/2735163000 conn(0x7f7844100120 msgr2=0x7f7844109130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:51.511 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:51.509+0000 7f784a800640 1 -- 172.21.15.72:0/2735163000 shutdown_connections 2024-04-26T21:52:51.511 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:51.509+0000 7f784a800640 1 -- 172.21.15.72:0/2735163000 wait complete. 2024-04-26T21:52:51.528 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:52:51.705 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:51.704+0000 7f5daec00640 1 -- 172.21.15.72:0/3405862672 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5da8105b80 msgr2=0x7f5da8106000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:51.705 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:51.704+0000 7f5daec00640 1 --2- 172.21.15.72:0/3405862672 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5da8105b80 0x7f5da8106000 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f5d94009900 tx=0x7f5d9402a3f0 comp rx=0 tx=0).stop 2024-04-26T21:52:51.706 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:51.704+0000 7f5daec00640 1 -- 172.21.15.72:0/3405862672 shutdown_connections 2024-04-26T21:52:51.706 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:51.704+0000 7f5daec00640 1 --2- 172.21.15.72:0/3405862672 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5da8106540 0x7f5da810cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:51.706 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:51.704+0000 7f5daec00640 1 --2- 172.21.15.72:0/3405862672 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5da8105b80 0x7f5da8106000 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:51.706 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:51.704+0000 7f5daec00640 1 --2- 172.21.15.72:0/3405862672 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5da8104990 0x7f5da8104db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:51.706 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:51.704+0000 7f5daec00640 1 -- 172.21.15.72:0/3405862672 >> 172.21.15.72:0/3405862672 conn(0x7f5da80fff00 msgr2=0x7f5da8102340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:51.706 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:51.704+0000 7f5daec00640 1 -- 172.21.15.72:0/3405862672 shutdown_connections 2024-04-26T21:52:51.706 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:51.704+0000 7f5daec00640 1 -- 172.21.15.72:0/3405862672 wait complete. 2024-04-26T21:52:51.708 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:51.705+0000 7f5daec00640 1 Processor -- start 2024-04-26T21:52:51.708 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:51.705+0000 7f5daec00640 1 -- start start 2024-04-26T21:52:51.708 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:51.705+0000 7f5daec00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5da8104990 0x7f5da819aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:51.709 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:51.705+0000 7f5daec00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5da8105b80 0x7f5da819afe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:51.709 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:51.705+0000 7f5daec00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5da8106540 0x7f5da81a2050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:51.709 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:51.705+0000 7f5daec00640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f5da81103a0 con 0x7f5da8106540 2024-04-26T21:52:51.709 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:51.705+0000 7f5daec00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f5da8110220 con 0x7f5da8105b80 2024-04-26T21:52:51.709 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:51.705+0000 7f5daec00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f5da8110520 con 0x7f5da8104990 2024-04-26T21:52:51.709 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:51.705+0000 7f5dae200640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5da8106540 0x7f5da81a2050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:51.709 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:51.705+0000 7f5dae200640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5da8106540 0x7f5da81a2050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:35026/0 (socket says 172.21.15.72:35026) 2024-04-26T21:52:51.709 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:51.706+0000 7f5dae200640 1 -- 172.21.15.72:0/632470321 learned_addr learned my addr 172.21.15.72:0/632470321 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:52:51.709 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:51.706+0000 7f5dad800640 1 --2- 172.21.15.72:0/632470321 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5da8104990 0x7f5da819aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:51.709 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:51.706+0000 7f5dace00640 1 --2- 172.21.15.72:0/632470321 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5da8105b80 0x7f5da819afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:51.709 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:51.706+0000 7f5dae200640 1 -- 172.21.15.72:0/632470321 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5da8104990 msgr2=0x7f5da819aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:51.709 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:51.706+0000 7f5dae200640 1 --2- 172.21.15.72:0/632470321 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5da8104990 0x7f5da819aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:51.709 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:51.706+0000 7f5dae200640 1 -- 172.21.15.72:0/632470321 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5da8105b80 msgr2=0x7f5da819afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:51.709 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:51.706+0000 7f5dae200640 1 --2- 172.21.15.72:0/632470321 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5da8105b80 0x7f5da819afe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:51.709 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:51.706+0000 7f5dae200640 1 -- 172.21.15.72:0/632470321 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5da81a26c0 con 0x7f5da8106540 2024-04-26T21:52:51.710 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:51.706+0000 7f5dace00640 1 --2- 172.21.15.72:0/632470321 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5da8105b80 0x7f5da819afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:52:51.710 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:51.706+0000 7f5dae200640 1 --2- 172.21.15.72:0/632470321 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5da8106540 0x7f5da81a2050 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f5da000eaa0 tx=0x7f5da000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:51.710 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:51.706+0000 7f5da6000640 1 -- 172.21.15.72:0/632470321 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5da000cdb0 con 0x7f5da8106540 2024-04-26T21:52:51.710 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:51.706+0000 7f5da6000640 1 -- 172.21.15.72:0/632470321 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5da0005020 con 0x7f5da8106540 2024-04-26T21:52:51.710 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:51.706+0000 7f5daec00640 1 -- 172.21.15.72:0/632470321 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5da81a29b0 con 0x7f5da8106540 2024-04-26T21:52:51.710 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:51.706+0000 7f5da6000640 1 -- 172.21.15.72:0/632470321 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5da00116d0 con 0x7f5da8106540 2024-04-26T21:52:51.710 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:51.707+0000 7f5daec00640 1 -- 172.21.15.72:0/632470321 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5da81a2eb0 con 0x7f5da8106540 2024-04-26T21:52:51.710 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:51.708+0000 7f5daec00640 1 -- 172.21.15.72:0/632470321 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5da810bd50 con 0x7f5da8106540 2024-04-26T21:52:51.711 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:51.710+0000 7f5da6000640 1 -- 172.21.15.72:0/632470321 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f5da00051d0 con 0x7f5da8106540 2024-04-26T21:52:51.712 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:51.711+0000 7f5da6000640 1 --2- 172.21.15.72:0/632470321 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f5d900a3bb0 0x7f5d900a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:51.712 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:51.711+0000 7f5da6000640 1 -- 172.21.15.72:0/632470321 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f5da00d0450 con 0x7f5da8106540 2024-04-26T21:52:51.712 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:51.711+0000 7f5dad800640 1 --2- 172.21.15.72:0/632470321 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f5d900a3bb0 0x7f5d900a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:51.713 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:51.712+0000 7f5dad800640 1 --2- 172.21.15.72:0/632470321 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f5d900a3bb0 0x7f5d900a6070 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f5d9c005fd0 tx=0x7f5d9c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:51.716 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:51.715+0000 7f5da6000640 1 -- 172.21.15.72:0/632470321 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f5da0016030 con 0x7f5da8106540 2024-04-26T21:52:51.912 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:51.911+0000 7f5daec00640 1 -- 172.21.15.72:0/632470321 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5da8063530 con 0x7f5da8106540 2024-04-26T21:52:51.916 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:51.915+0000 7f5da6000640 1 -- 172.21.15.72:0/632470321 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+22172 (secure 0 0 0) 0x7f5da009cf80 con 0x7f5da8106540 2024-04-26T21:52:51.916 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T21:52:51.917 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":87,"fsid":"a1780950-0415-11ef-bc93-c7b262605968","created":"2024-04-26T21:42:01.504173+0000","modified":"2024-04-26T21:52:51.746452+0000","last_up_change":"2024-04-26T21:51:43.482437+0000","last_in_change":"2024-04-26T21:51:18.468587+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:46:29.785427+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:52:48.320391+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T21:52:49.517493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"04e06884-b355-4762-9b29-9034b7201e9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6802","nonce":1416765179},{"type":"v1","addr":"172.21.15.72:6803","nonce":1416765179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6804","nonce":1416765179},{"type":"v1","addr":"172.21.15.72:6805","nonce":1416765179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6808","nonce":1416765179},{"type":"v1","addr":"172.21.15.72:6809","nonce":1416765179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6806","nonce":1416765179},{"type":"v1","addr":"172.21.15.72:6807","nonce":1416765179}]},"public_addr":"172.21.15.72:6803/1416765179","cluster_addr":"172.21.15.72:6805/1416765179","heartbeat_back_addr":"172.21.15.72:6809/1416765179","heartbeat_front_addr":"172.21.15.72:6807/1416765179","state":["exists","up"]},{"osd":1,"uuid":"d75a479a-4a2e-48e9-a01e-2b1c9617416a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":755426949},{"type":"v1","addr":"172.21.15.76:6801","nonce":755426949}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":755426949},{"type":"v1","addr":"172.21.15.76:6803","nonce":755426949}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":755426949},{"type":"v1","addr":"172.21.15.76:6807","nonce":755426949}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":755426949},{"type":"v1","addr":"172.21.15.76:6805","nonce":755426949}]},"public_addr":"172.21.15.76:6801/755426949","cluster_addr":"172.21.15.76:6803/755426949","heartbeat_back_addr":"172.21.15.76:6807/755426949","heartbeat_front_addr":"172.21.15.76:6805/755426949","state":["exists","up"]},{"osd":2,"uuid":"b6f8da58-4428-4e8c-a944-7ea5645d6d92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":852265973},{"type":"v1","addr":"172.21.15.164:6801","nonce":852265973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":852265973},{"type":"v1","addr":"172.21.15.164:6803","nonce":852265973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":852265973},{"type":"v1","addr":"172.21.15.164:6807","nonce":852265973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":852265973},{"type":"v1","addr":"172.21.15.164:6805","nonce":852265973}]},"public_addr":"172.21.15.164:6801/852265973","cluster_addr":"172.21.15.164:6803/852265973","heartbeat_back_addr":"172.21.15.164:6807/852265973","heartbeat_front_addr":"172.21.15.164:6805/852265973","state":["exists","up"]},{"osd":3,"uuid":"76d83409-649e-41e5-b4f4-43bbd5b00ea5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6810","nonce":4027763789},{"type":"v1","addr":"172.21.15.72:6811","nonce":4027763789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6812","nonce":4027763789},{"type":"v1","addr":"172.21.15.72:6813","nonce":4027763789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6816","nonce":4027763789},{"type":"v1","addr":"172.21.15.72:6817","nonce":4027763789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6814","nonce":4027763789},{"type":"v1","addr":"172.21.15.72:6815","nonce":4027763789}]},"public_addr":"172.21.15.72:6811/4027763789","cluster_addr":"172.21.15.72:6813/4027763789","heartbeat_back_addr":"172.21.15.72:6817/4027763789","heartbeat_front_addr":"172.21.15.72:6815/4027763789","state":["exists","up"]},{"osd":4,"uuid":"d2cfeea7-1661-4156-bdcd-f770a8d1a878","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":3718867827},{"type":"v1","addr":"172.21.15.76:6809","nonce":3718867827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":3718867827},{"type":"v1","addr":"172.21.15.76:6811","nonce":3718867827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":3718867827},{"type":"v1","addr":"172.21.15.76:6815","nonce":3718867827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":3718867827},{"type":"v1","addr":"172.21.15.76:6813","nonce":3718867827}]},"public_addr":"172.21.15.76:6809/3718867827","cluster_addr":"172.21.15.76:6811/3718867827","heartbeat_back_addr":"172.21.15.76:6815/3718867827","heartbeat_front_addr":"172.21.15.76:6813/3718867827","state":["exists","up"]},{"osd":5,"uuid":"f3896ad6-a040-4ea3-a431-ff9b7fc05c36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":2543954772},{"type":"v1","addr":"172.21.15.164:6809","nonce":2543954772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":2543954772},{"type":"v1","addr":"172.21.15.164:6811","nonce":2543954772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":2543954772},{"type":"v1","addr":"172.21.15.164:6815","nonce":2543954772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":2543954772},{"type":"v1","addr":"172.21.15.164:6813","nonce":2543954772}]},"public_addr":"172.21.15.164:6809/2543954772","cluster_addr":"172.21.15.164:6811/2543954772","heartbeat_back_addr":"172.21.15.164:6815/2543954772","heartbeat_front_addr":"172.21.15.164:6813/2543954772","state":["exists","up"]},{"osd":6,"uuid":"d5d56650-be7e-4ba5-ba3a-d7d473294dab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6818","nonce":2080659027},{"type":"v1","addr":"172.21.15.72:6819","nonce":2080659027}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6820","nonce":2080659027},{"type":"v1","addr":"172.21.15.72:6821","nonce":2080659027}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6824","nonce":2080659027},{"type":"v1","addr":"172.21.15.72:6825","nonce":2080659027}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6822","nonce":2080659027},{"type":"v1","addr":"172.21.15.72:6823","nonce":2080659027}]},"public_addr":"172.21.15.72:6819/2080659027","cluster_addr":"172.21.15.72:6821/2080659027","heartbeat_back_addr":"172.21.15.72:6825/2080659027","heartbeat_front_addr":"172.21.15.72:6823/2080659027","state":["exists","up"]},{"osd":7,"uuid":"18975b5a-9309-4a0f-9f37-3ce09c2e745d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":3144631323},{"type":"v1","addr":"172.21.15.76:6817","nonce":3144631323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":3144631323},{"type":"v1","addr":"172.21.15.76:6819","nonce":3144631323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":3144631323},{"type":"v1","addr":"172.21.15.76:6823","nonce":3144631323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":3144631323},{"type":"v1","addr":"172.21.15.76:6821","nonce":3144631323}]},"public_addr":"172.21.15.76:6817/3144631323","cluster_addr":"172.21.15.76:6819/3144631323","heartbeat_back_addr":"172.21.15.76:6823/3144631323","heartbeat_front_addr":"172.21.15.76:6821/3144631323","state":["exists","up"]},{"osd":8,"uuid":"779e9c1e-8e97-48d4-a854-68ca13f38e0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":627888883},{"type":"v1","addr":"172.21.15.164:6817","nonce":627888883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":627888883},{"type":"v1","addr":"172.21.15.164:6819","nonce":627888883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":627888883},{"type":"v1","addr":"172.21.15.164:6823","nonce":627888883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":627888883},{"type":"v1","addr":"172.21.15.164:6821","nonce":627888883}]},"public_addr":"172.21.15.164:6817/627888883","cluster_addr":"172.21.15.164:6819/627888883","heartbeat_back_addr":"172.21.15.164:6823/627888883","heartbeat_front_addr":"172.21.15.164:6821/627888883","state":["exists","up"]},{"osd":9,"uuid":"d0317850-7b2d-4850-b2cf-0b919505c1fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6826","nonce":1273118485},{"type":"v1","addr":"172.21.15.72:6827","nonce":1273118485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6828","nonce":1273118485},{"type":"v1","addr":"172.21.15.72:6829","nonce":1273118485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6832","nonce":1273118485},{"type":"v1","addr":"172.21.15.72:6833","nonce":1273118485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6830","nonce":1273118485},{"type":"v1","addr":"172.21.15.72:6831","nonce":1273118485}]},"public_addr":"172.21.15.72:6827/1273118485","cluster_addr":"172.21.15.72:6829/1273118485","heartbeat_back_addr":"172.21.15.72:6833/1273118485","heartbeat_front_addr":"172.21.15.72:6831/1273118485","state":["exists","up"]},{"osd":10,"uuid":"e0aa38d8-6011-4d2a-b67b-5bff6d999e47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":2808954459},{"type":"v1","addr":"172.21.15.76:6825","nonce":2808954459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":2808954459},{"type":"v1","addr":"172.21.15.76:6827","nonce":2808954459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":2808954459},{"type":"v1","addr":"172.21.15.76:6831","nonce":2808954459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":2808954459},{"type":"v1","addr":"172.21.15.76:6829","nonce":2808954459}]},"public_addr":"172.21.15.76:6825/2808954459","cluster_addr":"172.21.15.76:6827/2808954459","heartbeat_back_addr":"172.21.15.76:6831/2808954459","heartbeat_front_addr":"172.21.15.76:6829/2808954459","state":["exists","up"]},{"osd":11,"uuid":"981b3d4d-916e-4937-9916-7cc21a891e43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":2354951784},{"type":"v1","addr":"172.21.15.164:6825","nonce":2354951784}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":2354951784},{"type":"v1","addr":"172.21.15.164:6827","nonce":2354951784}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":2354951784},{"type":"v1","addr":"172.21.15.164:6831","nonce":2354951784}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":2354951784},{"type":"v1","addr":"172.21.15.164:6829","nonce":2354951784}]},"public_addr":"172.21.15.164:6825/2354951784","cluster_addr":"172.21.15.164:6827/2354951784","heartbeat_back_addr":"172.21.15.164:6831/2354951784","heartbeat_front_addr":"172.21.15.164:6829/2354951784","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:18.997924+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:52.439081+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:46:25.893288+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:47:00.112807+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:47:35.116874+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:48:09.216344+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:48:45.028853+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:49:19.905397+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:49:55.030536+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:50:30.767662+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:51:06.423267+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:51:41.631211+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.72:0/416277784":"2024-04-27T21:43:09.602791+0000","172.21.15.72:6800/4010358672":"2024-04-27T21:43:09.602791+0000","172.21.15.72:6801/1012617371":"2024-04-27T21:42:34.684837+0000","172.21.15.72:6801/4010358672":"2024-04-27T21:43:09.602791+0000","172.21.15.72:0/3555448179":"2024-04-27T21:43:09.602791+0000","172.21.15.72:6800/1012617371":"2024-04-27T21:42:34.684837+0000","172.21.15.72:0/2920378761":"2024-04-27T21:42:34.684837+0000","172.21.15.72:0/1933740041":"2024-04-27T21:43:09.602791+0000","172.21.15.72:0/3225295775":"2024-04-27T21:42:34.684837+0000","172.21.15.72:0/2326016180":"2024-04-27T21:42:34.684837+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:52:51.918 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:51.917+0000 7f5daec00640 1 -- 172.21.15.72:0/632470321 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f5d900a3bb0 msgr2=0x7f5d900a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:51.918 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:51.917+0000 7f5daec00640 1 --2- 172.21.15.72:0/632470321 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f5d900a3bb0 0x7f5d900a6070 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f5d9c005fd0 tx=0x7f5d9c00b410 comp rx=0 tx=0).stop 2024-04-26T21:52:51.919 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:51.917+0000 7f5daec00640 1 -- 172.21.15.72:0/632470321 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5da8106540 msgr2=0x7f5da81a2050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:51.919 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:51.917+0000 7f5daec00640 1 --2- 172.21.15.72:0/632470321 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5da8106540 0x7f5da81a2050 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f5da000eaa0 tx=0x7f5da000ef60 comp rx=0 tx=0).stop 2024-04-26T21:52:51.919 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:51.918+0000 7f5daec00640 1 -- 172.21.15.72:0/632470321 shutdown_connections 2024-04-26T21:52:51.919 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:51.918+0000 7f5daec00640 1 --2- 172.21.15.72:0/632470321 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5da8106540 0x7f5da81a2050 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:51.919 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:51.918+0000 7f5daec00640 1 --2- 172.21.15.72:0/632470321 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5da8105b80 0x7f5da819afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:51.919 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:51.918+0000 7f5daec00640 1 --2- 172.21.15.72:0/632470321 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f5d900a3bb0 0x7f5d900a6070 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:51.919 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:51.918+0000 7f5daec00640 1 --2- 172.21.15.72:0/632470321 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5da8104990 0x7f5da819aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:51.920 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:51.918+0000 7f5daec00640 1 -- 172.21.15.72:0/632470321 >> 172.21.15.72:0/632470321 conn(0x7f5da80fff00 msgr2=0x7f5da8108f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:51.920 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:51.918+0000 7f5daec00640 1 -- 172.21.15.72:0/632470321 shutdown_connections 2024-04-26T21:52:51.920 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:51.918+0000 7f5daec00640 1 -- 172.21.15.72:0/632470321 wait complete. 2024-04-26T21:52:51.938 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-26T21:52:52.141 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:52.139+0000 7f67d7200640 1 -- 172.21.15.72:0/1351732203 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f67d8102840 msgr2=0x7f67d8102c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:52.141 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:52.139+0000 7f67d7200640 1 --2- 172.21.15.72:0/1351732203 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f67d8102840 0x7f67d8102c60 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f67c8009900 tx=0x7f67c802a3f0 comp rx=0 tx=0).stop 2024-04-26T21:52:52.141 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:52.139+0000 7f67d7200640 1 -- 172.21.15.72:0/1351732203 shutdown_connections 2024-04-26T21:52:52.141 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:52.139+0000 7f67d7200640 1 --2- 172.21.15.72:0/1351732203 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f67d81043f0 0x7f67d810ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:52.141 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:52.139+0000 7f67d7200640 1 --2- 172.21.15.72:0/1351732203 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f67d8103a30 0x7f67d8103eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:52.141 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:52.139+0000 7f67d7200640 1 --2- 172.21.15.72:0/1351732203 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f67d8102840 0x7f67d8102c60 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:52.141 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:52.139+0000 7f67d7200640 1 -- 172.21.15.72:0/1351732203 >> 172.21.15.72:0/1351732203 conn(0x7f67d80fddd0 msgr2=0x7f67d81001f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:52.141 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:52.139+0000 7f67d7200640 1 -- 172.21.15.72:0/1351732203 shutdown_connections 2024-04-26T21:52:52.141 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:52.139+0000 7f67d7200640 1 -- 172.21.15.72:0/1351732203 wait complete. 2024-04-26T21:52:52.143 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:52.140+0000 7f67d7200640 1 Processor -- start 2024-04-26T21:52:52.143 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:52.140+0000 7f67d7200640 1 -- start start 2024-04-26T21:52:52.143 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:52.141+0000 7f67d7200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f67d8102840 0x7f67d819abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:52.143 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:52.141+0000 7f67d7200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f67d8103a30 0x7f67d819b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:52.143 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:52.141+0000 7f67d7200640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f67d81043f0 0x7f67d81a2160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:52.143 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:52.141+0000 7f67d7200640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f67d810e270 con 0x7f67d81043f0 2024-04-26T21:52:52.143 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:52.141+0000 7f67d7200640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f67d810e0f0 con 0x7f67d8102840 2024-04-26T21:52:52.144 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:52.141+0000 7f67d7200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f67d810e3f0 con 0x7f67d8103a30 2024-04-26T21:52:52.144 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:52.141+0000 7f67d5e00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f67d8102840 0x7f67d819abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:52.144 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:52.141+0000 7f67d5e00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f67d8102840 0x7f67d819abb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.72:50382/0 (socket says 172.21.15.72:50382) 2024-04-26T21:52:52.144 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:52.141+0000 7f67d5e00640 1 -- 172.21.15.72:0/3130083084 learned_addr learned my addr 172.21.15.72:0/3130083084 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:52:52.144 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:52.141+0000 7f67d6800640 1 --2- 172.21.15.72:0/3130083084 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f67d81043f0 0x7f67d81a2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:52.144 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:52.141+0000 7f67d5400640 1 --2- 172.21.15.72:0/3130083084 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f67d8103a30 0x7f67d819b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:52.144 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:52.142+0000 7f67d5e00640 1 -- 172.21.15.72:0/3130083084 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f67d8103a30 msgr2=0x7f67d819b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:52.144 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:52.142+0000 7f67d5e00640 1 --2- 172.21.15.72:0/3130083084 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f67d8103a30 0x7f67d819b0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:52.144 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:52.142+0000 7f67d5e00640 1 -- 172.21.15.72:0/3130083084 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f67d81043f0 msgr2=0x7f67d81a2160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:52.144 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:52.142+0000 7f67d5e00640 1 --2- 172.21.15.72:0/3130083084 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f67d81043f0 0x7f67d81a2160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:52.144 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:52.142+0000 7f67d5e00640 1 -- 172.21.15.72:0/3130083084 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67d81a27d0 con 0x7f67d8102840 2024-04-26T21:52:52.144 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:52.142+0000 7f67d5e00640 1 --2- 172.21.15.72:0/3130083084 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f67d8102840 0x7f67d819abb0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f67c8031460 tx=0x7f67c8003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:52.145 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:52.142+0000 7f67cea00640 1 -- 172.21.15.72:0/3130083084 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67c803f070 con 0x7f67d8102840 2024-04-26T21:52:52.145 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:52.142+0000 7f67d7200640 1 -- 172.21.15.72:0/3130083084 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67d81a2a60 con 0x7f67d8102840 2024-04-26T21:52:52.145 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:52.143+0000 7f67cea00640 1 -- 172.21.15.72:0/3130083084 <== mon.2 v2:172.21.15.76:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f67c8031ba0 con 0x7f67d8102840 2024-04-26T21:52:52.145 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:52.143+0000 7f67cea00640 1 -- 172.21.15.72:0/3130083084 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67c8035030 con 0x7f67d8102840 2024-04-26T21:52:52.145 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:52.143+0000 7f67d7200640 1 -- 172.21.15.72:0/3130083084 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67d81a2fa0 con 0x7f67d8102840 2024-04-26T21:52:52.145 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:52.144+0000 7f67d7200640 1 -- 172.21.15.72:0/3130083084 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67d81a3290 con 0x7f67d8102840 2024-04-26T21:52:52.147 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:52.146+0000 7f67cea00640 1 -- 172.21.15.72:0/3130083084 <== mon.2 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f67c8042070 con 0x7f67d8102840 2024-04-26T21:52:52.148 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:52.147+0000 7f67cea00640 1 --2- 172.21.15.72:0/3130083084 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f67bc0a3c30 0x7f67bc0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:52.149 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:52.148+0000 7f67cea00640 1 -- 172.21.15.72:0/3130083084 <== mon.2 v2:172.21.15.76:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7f67c80f1330 con 0x7f67d8102840 2024-04-26T21:52:52.149 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:52.148+0000 7f67d5400640 1 --2- 172.21.15.72:0/3130083084 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f67bc0a3c30 0x7f67bc0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:52.149 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:52.148+0000 7f67d5400640 1 --2- 172.21.15.72:0/3130083084 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f67bc0a3c30 0x7f67bc0a60f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f67d819c0c0 tx=0x7f67c4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:52.152 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:52.151+0000 7f67cea00640 1 -- 172.21.15.72:0/3130083084 <== mon.2 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f67c80bd770 con 0x7f67d8102840 2024-04-26T21:52:52.321 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:52 smithi072 ceph-mon[28138]: from='client.? 172.21.15.72:0/2258182875' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-26T21:52:52.321 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:52 smithi072 ceph-mon[28138]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-26T21:52:52.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:52 smithi072 ceph-mon[28138]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-26T21:52:52.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:52 smithi072 ceph-mon[28138]: from='client.? 172.21.15.72:0/2735163000' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:52:52.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:52 smithi072 ceph-mon[28138]: osdmap e87: 12 total, 12 up, 12 in 2024-04-26T21:52:52.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:52 smithi072 ceph-mon[28138]: pgmap v368: 97 pgs: 7 creating+peering, 1 active+clean, 89 unknown; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:52:52.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:52 smithi072 ceph-mon[28138]: from='client.? 172.21.15.72:0/632470321' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:52:52.424 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:52.423+0000 7f67d7200640 1 -- 172.21.15.72:0/3130083084 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f67d81a3460 con 0x7f67d8102840 2024-04-26T21:52:52.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:52 smithi164 ceph-mon[31916]: from='client.? 172.21.15.72:0/2258182875' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-26T21:52:52.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:52 smithi164 ceph-mon[31916]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-26T21:52:52.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:52 smithi164 ceph-mon[31916]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-26T21:52:52.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:52 smithi164 ceph-mon[31916]: from='client.? 172.21.15.72:0/2735163000' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:52:52.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:52 smithi164 ceph-mon[31916]: osdmap e87: 12 total, 12 up, 12 in 2024-04-26T21:52:52.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:52 smithi164 ceph-mon[31916]: pgmap v368: 97 pgs: 7 creating+peering, 1 active+clean, 89 unknown; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:52:52.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:52 smithi164 ceph-mon[31916]: from='client.? 172.21.15.72:0/632470321' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:52:52.564 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:52 smithi076 ceph-mon[31823]: from='client.? 172.21.15.72:0/2258182875' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-26T21:52:52.564 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:52 smithi076 ceph-mon[31823]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-26T21:52:52.565 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:52 smithi076 ceph-mon[31823]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-26T21:52:52.565 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:52 smithi076 ceph-mon[31823]: from='client.? 172.21.15.72:0/2735163000' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:52:52.565 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:52 smithi076 ceph-mon[31823]: osdmap e87: 12 total, 12 up, 12 in 2024-04-26T21:52:52.565 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:52 smithi076 ceph-mon[31823]: pgmap v368: 97 pgs: 7 creating+peering, 1 active+clean, 89 unknown; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:52:52.565 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:52 smithi076 ceph-mon[31823]: from='client.? 172.21.15.72:0/632470321' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:52:53.133 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:53.132+0000 7f67cea00640 1 -- 172.21.15.72:0/3130083084 <== mon.2 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) ==== 118+0+0 (secure 0 0 0) 0x7f67c80c1a70 con 0x7f67d8102840 2024-04-26T21:52:53.136 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:53.135+0000 7f67d7200640 1 -- 172.21.15.72:0/3130083084 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f67bc0a3c30 msgr2=0x7f67bc0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:53.136 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:53.135+0000 7f67d7200640 1 --2- 172.21.15.72:0/3130083084 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f67bc0a3c30 0x7f67bc0a60f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f67d819c0c0 tx=0x7f67c4005f90 comp rx=0 tx=0).stop 2024-04-26T21:52:53.136 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:53.135+0000 7f67d7200640 1 -- 172.21.15.72:0/3130083084 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f67d8102840 msgr2=0x7f67d819abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:53.137 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:53.135+0000 7f67d7200640 1 --2- 172.21.15.72:0/3130083084 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f67d8102840 0x7f67d819abb0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f67c8031460 tx=0x7f67c8003c10 comp rx=0 tx=0).stop 2024-04-26T21:52:53.137 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:53.135+0000 7f67d7200640 1 -- 172.21.15.72:0/3130083084 shutdown_connections 2024-04-26T21:52:53.137 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:53.135+0000 7f67d7200640 1 --2- 172.21.15.72:0/3130083084 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f67d81043f0 0x7f67d81a2160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:53.137 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:53.135+0000 7f67d7200640 1 --2- 172.21.15.72:0/3130083084 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f67bc0a3c30 0x7f67bc0a60f0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:53.137 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:53.135+0000 7f67d7200640 1 --2- 172.21.15.72:0/3130083084 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f67d8103a30 0x7f67d819b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:53.137 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:53.135+0000 7f67d7200640 1 --2- 172.21.15.72:0/3130083084 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f67d8102840 0x7f67d819abb0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:53.137 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:53.135+0000 7f67d7200640 1 -- 172.21.15.72:0/3130083084 >> 172.21.15.72:0/3130083084 conn(0x7f67d80fddd0 msgr2=0x7f67d81091f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:53.137 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:53.135+0000 7f67d7200640 1 -- 172.21.15.72:0/3130083084 shutdown_connections 2024-04-26T21:52:53.137 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:53.135+0000 7f67d7200640 1 -- 172.21.15.72:0/3130083084 wait complete. 2024-04-26T21:52:53.158 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2}} 2024-04-26T21:52:53.158 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 5 2024-04-26T21:52:53.342 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:53.340+0000 7f7e8be00640 1 -- 172.21.15.72:0/455792349 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7e8c106620 msgr2=0x7f7e8c10ceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:53.342 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:53.340+0000 7f7e8be00640 1 --2- 172.21.15.72:0/455792349 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7e8c106620 0x7f7e8c10ceb0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f7e84009900 tx=0x7f7e8402a3f0 comp rx=0 tx=0).stop 2024-04-26T21:52:53.342 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:53.340+0000 7f7e8be00640 1 -- 172.21.15.72:0/455792349 shutdown_connections 2024-04-26T21:52:53.342 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:53.340+0000 7f7e8be00640 1 --2- 172.21.15.72:0/455792349 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7e8c106620 0x7f7e8c10ceb0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:53.342 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:53.340+0000 7f7e8be00640 1 --2- 172.21.15.72:0/455792349 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7e8c105c60 0x7f7e8c1060e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:53.342 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:53.340+0000 7f7e8be00640 1 --2- 172.21.15.72:0/455792349 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7e8c104a70 0x7f7e8c104e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:53.342 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:53.340+0000 7f7e8be00640 1 -- 172.21.15.72:0/455792349 >> 172.21.15.72:0/455792349 conn(0x7f7e8c0fffe0 msgr2=0x7f7e8c102420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:53.342 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:53.340+0000 7f7e8be00640 1 -- 172.21.15.72:0/455792349 shutdown_connections 2024-04-26T21:52:53.342 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:53.340+0000 7f7e8be00640 1 -- 172.21.15.72:0/455792349 wait complete. 2024-04-26T21:52:53.344 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:53.341+0000 7f7e8be00640 1 Processor -- start 2024-04-26T21:52:53.345 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:53.341+0000 7f7e8be00640 1 -- start start 2024-04-26T21:52:53.345 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:53.341+0000 7f7e8be00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7e8c104a70 0x7f7e8c19ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:53.345 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:53.341+0000 7f7e8be00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7e8c105c60 0x7f7e8c19b060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:53.345 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:53.341+0000 7f7e8be00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7e8c106620 0x7f7e8c1a20d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:53.345 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:53.341+0000 7f7e8be00640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f7e8c110510 con 0x7f7e8c105c60 2024-04-26T21:52:53.345 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:53.341+0000 7f7e8be00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f7e8c110390 con 0x7f7e8c104a70 2024-04-26T21:52:53.345 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:53.341+0000 7f7e8be00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f7e8c110690 con 0x7f7e8c106620 2024-04-26T21:52:53.345 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:53.342+0000 7f7e82000640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7e8c105c60 0x7f7e8c19b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:53.345 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:53.342+0000 7f7e82000640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7e8c105c60 0x7f7e8c19b060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:35066/0 (socket says 172.21.15.72:35066) 2024-04-26T21:52:53.345 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:53.342+0000 7f7e82000640 1 -- 172.21.15.72:0/304301699 learned_addr learned my addr 172.21.15.72:0/304301699 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:52:53.345 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:53.342+0000 7f7e8aa00640 1 --2- 172.21.15.72:0/304301699 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7e8c104a70 0x7f7e8c19ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:53.345 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:53.342+0000 7f7e8b400640 1 --2- 172.21.15.72:0/304301699 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7e8c106620 0x7f7e8c1a20d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:53.345 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:53.342+0000 7f7e82000640 1 -- 172.21.15.72:0/304301699 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7e8c106620 msgr2=0x7f7e8c1a20d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:53.345 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:53.342+0000 7f7e82000640 1 --2- 172.21.15.72:0/304301699 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7e8c106620 0x7f7e8c1a20d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:53.346 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:53.342+0000 7f7e82000640 1 -- 172.21.15.72:0/304301699 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7e8c104a70 msgr2=0x7f7e8c19ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:53.346 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:53.342+0000 7f7e82000640 1 --2- 172.21.15.72:0/304301699 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7e8c104a70 0x7f7e8c19ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:53.346 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:53.342+0000 7f7e82000640 1 -- 172.21.15.72:0/304301699 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e8c1a2740 con 0x7f7e8c105c60 2024-04-26T21:52:53.346 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:53.342+0000 7f7e82000640 1 --2- 172.21.15.72:0/304301699 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7e8c105c60 0x7f7e8c19b060 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f7e7400c920 tx=0x7f7e7400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:53.346 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:53.343+0000 7f7e83e00640 1 -- 172.21.15.72:0/304301699 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e74007c70 con 0x7f7e8c105c60 2024-04-26T21:52:53.346 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:53.343+0000 7f7e83e00640 1 -- 172.21.15.72:0/304301699 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7e74007e10 con 0x7f7e8c105c60 2024-04-26T21:52:53.346 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:53.343+0000 7f7e8be00640 1 -- 172.21.15.72:0/304301699 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e8c1a2a30 con 0x7f7e8c105c60 2024-04-26T21:52:53.346 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:53.343+0000 7f7e83e00640 1 -- 172.21.15.72:0/304301699 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e74018710 con 0x7f7e8c105c60 2024-04-26T21:52:53.346 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:53.343+0000 7f7e8be00640 1 -- 172.21.15.72:0/304301699 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e8c1a2fd0 con 0x7f7e8c105c60 2024-04-26T21:52:53.347 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:53.344+0000 7f7e8be00640 1 -- 172.21.15.72:0/304301699 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e8c10bde0 con 0x7f7e8c105c60 2024-04-26T21:52:53.347 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:53.346+0000 7f7e83e00640 1 -- 172.21.15.72:0/304301699 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f7e7400f450 con 0x7f7e8c105c60 2024-04-26T21:52:53.348 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:53.347+0000 7f7e83e00640 1 --2- 172.21.15.72:0/304301699 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f7e700a3c30 0x7f7e700a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:53.349 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:53.347+0000 7f7e83e00640 1 -- 172.21.15.72:0/304301699 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7f7e740d0520 con 0x7f7e8c105c60 2024-04-26T21:52:53.349 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:53.348+0000 7f7e8aa00640 1 --2- 172.21.15.72:0/304301699 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f7e700a3c30 0x7f7e700a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:53.349 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:53.348+0000 7f7e8aa00640 1 --2- 172.21.15.72:0/304301699 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f7e700a3c30 0x7f7e700a60f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f7e8c105ac0 tx=0x7f7e7c0094f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:53.353 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:53.352+0000 7f7e83e00640 1 -- 172.21.15.72:0/304301699 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f7e74014030 con 0x7f7e8c105c60 2024-04-26T21:52:53.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:53 smithi164 ceph-mon[31916]: mds.? [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] up:active 2024-04-26T21:52:53.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:53 smithi164 ceph-mon[31916]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-26T21:52:53.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:53 smithi164 ceph-mon[31916]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-26T21:52:53.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:53 smithi164 ceph-mon[31916]: from='client.? 172.21.15.72:0/3130083084' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-26T21:52:53.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:53 smithi164 ceph-mon[31916]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-26T21:52:53.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:53 smithi164 ceph-mon[31916]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-26T21:52:53.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:53 smithi164 ceph-mon[31916]: Cluster is now healthy 2024-04-26T21:52:53.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:53 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:52:53.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:53 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:52:53.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:53 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:52:53.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:53 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:52:53.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:53 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:52:53.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:53 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:52:53.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:53 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:52:53.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:53 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:52:53.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:53 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:52:53.565 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:53 smithi076 ceph-mon[31823]: mds.? [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] up:active 2024-04-26T21:52:53.565 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:53 smithi076 ceph-mon[31823]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-26T21:52:53.565 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:53 smithi076 ceph-mon[31823]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-26T21:52:53.565 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:53 smithi076 ceph-mon[31823]: from='client.? 172.21.15.72:0/3130083084' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-26T21:52:53.565 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:53 smithi076 ceph-mon[31823]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-26T21:52:53.565 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:53 smithi076 ceph-mon[31823]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-26T21:52:53.565 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:53 smithi076 ceph-mon[31823]: Cluster is now healthy 2024-04-26T21:52:53.565 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:53 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:52:53.566 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:53 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:52:53.566 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:53 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:52:53.566 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:53 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:52:53.566 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:53 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:52:53.566 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:53 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:52:53.566 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:53 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:52:53.566 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:53 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:52:53.566 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:53 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:52:53.572 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:53 smithi072 ceph-mon[28138]: mds.? [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] up:active 2024-04-26T21:52:53.572 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:53 smithi072 ceph-mon[28138]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-26T21:52:53.572 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:53 smithi072 ceph-mon[28138]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-26T21:52:53.572 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:53 smithi072 ceph-mon[28138]: from='client.? 172.21.15.72:0/3130083084' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-26T21:52:53.572 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:53 smithi072 ceph-mon[28138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-26T21:52:53.572 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:53 smithi072 ceph-mon[28138]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-26T21:52:53.573 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:53 smithi072 ceph-mon[28138]: Cluster is now healthy 2024-04-26T21:52:53.573 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:53 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:52:53.573 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:53 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:52:53.573 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:53 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:52:53.573 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:53 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:52:53.573 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:53 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:52:53.573 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:53 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:52:53.573 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:53 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:52:53.573 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:53 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:52:53.573 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:53 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:52:53.589 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:53.588+0000 7f7e8be00640 1 -- 172.21.15.72:0/304301699 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) -- 0x7f7e8c063530 con 0x7f7e8c105c60 2024-04-26T21:52:54.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:54 smithi164 ceph-mon[31916]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-26T21:52:54.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:54 smithi164 ceph-mon[31916]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-26T21:52:54.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:54 smithi164 ceph-mon[31916]: from='client.? 172.21.15.72:0/304301699' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-04-26T21:52:54.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:54 smithi164 ceph-mon[31916]: pgmap v369: 97 pgs: 47 active+clean, 7 creating+peering, 43 unknown; 577 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 435 B/s wr, 1 op/s 2024-04-26T21:52:54.514 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:54.512+0000 7f7e83e00640 1 -- 172.21.15.72:0/304301699 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v18) ==== 105+0+0 (secure 0 0 0) 0x7f7e7409cfb0 con 0x7f7e8c105c60 2024-04-26T21:52:54.518 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:54.516+0000 7f7e8be00640 1 -- 172.21.15.72:0/304301699 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f7e700a3c30 msgr2=0x7f7e700a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:54.518 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:54.516+0000 7f7e8be00640 1 --2- 172.21.15.72:0/304301699 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f7e700a3c30 0x7f7e700a60f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f7e8c105ac0 tx=0x7f7e7c0094f0 comp rx=0 tx=0).stop 2024-04-26T21:52:54.518 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:54.516+0000 7f7e8be00640 1 -- 172.21.15.72:0/304301699 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7e8c105c60 msgr2=0x7f7e8c19b060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:54.518 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:54.516+0000 7f7e8be00640 1 --2- 172.21.15.72:0/304301699 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7e8c105c60 0x7f7e8c19b060 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f7e7400c920 tx=0x7f7e7400cde0 comp rx=0 tx=0).stop 2024-04-26T21:52:54.518 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:54.516+0000 7f7e8be00640 1 -- 172.21.15.72:0/304301699 shutdown_connections 2024-04-26T21:52:54.518 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:54.516+0000 7f7e8be00640 1 --2- 172.21.15.72:0/304301699 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f7e700a3c30 0x7f7e700a60f0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:54.518 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:54.516+0000 7f7e8be00640 1 --2- 172.21.15.72:0/304301699 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7e8c106620 0x7f7e8c1a20d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:54.518 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:54.516+0000 7f7e8be00640 1 --2- 172.21.15.72:0/304301699 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7e8c105c60 0x7f7e8c19b060 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:54.518 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:54.516+0000 7f7e8be00640 1 --2- 172.21.15.72:0/304301699 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7e8c104a70 0x7f7e8c19ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:54.518 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:54.516+0000 7f7e8be00640 1 -- 172.21.15.72:0/304301699 >> 172.21.15.72:0/304301699 conn(0x7f7e8c0fffe0 msgr2=0x7f7e8c108fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:54.518 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:54.516+0000 7f7e8be00640 1 -- 172.21.15.72:0/304301699 shutdown_connections 2024-04-26T21:52:54.519 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:54.516+0000 7f7e8be00640 1 -- 172.21.15.72:0/304301699 wait complete. 2024-04-26T21:52:54.536 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:54 smithi072 ceph-mon[28138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-26T21:52:54.536 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:54 smithi072 ceph-mon[28138]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-26T21:52:54.536 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:54 smithi072 ceph-mon[28138]: from='client.? 172.21.15.72:0/304301699' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-04-26T21:52:54.536 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:54 smithi072 ceph-mon[28138]: pgmap v369: 97 pgs: 47 active+clean, 7 creating+peering, 43 unknown; 577 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 435 B/s wr, 1 op/s 2024-04-26T21:52:54.539 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2024-04-26T21:52:54.564 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:54 smithi076 ceph-mon[31823]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-26T21:52:54.564 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:54 smithi076 ceph-mon[31823]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-26T21:52:54.564 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:54 smithi076 ceph-mon[31823]: from='client.? 172.21.15.72:0/304301699' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-04-26T21:52:54.565 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:54 smithi076 ceph-mon[31823]: pgmap v369: 97 pgs: 47 active+clean, 7 creating+peering, 43 unknown; 577 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 435 B/s wr, 1 op/s 2024-04-26T21:52:54.736 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:54.731+0000 7f2ab3e00640 1 -- 172.21.15.72:0/2139528723 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2ab4102630 msgr2=0x7f2ab4108ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:54.737 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:54.731+0000 7f2ab3e00640 1 --2- 172.21.15.72:0/2139528723 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2ab4102630 0x7f2ab4108ec0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f2aac009670 tx=0x7f2aac02a580 comp rx=0 tx=0).stop 2024-04-26T21:52:54.737 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:54.731+0000 7f2ab3e00640 1 -- 172.21.15.72:0/2139528723 shutdown_connections 2024-04-26T21:52:54.737 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:54.731+0000 7f2ab3e00640 1 --2- 172.21.15.72:0/2139528723 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2ab4102630 0x7f2ab4108ec0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:54.737 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:54.731+0000 7f2ab3e00640 1 --2- 172.21.15.72:0/2139528723 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2ab4101c70 0x7f2ab41020f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:54.737 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:54.731+0000 7f2ab3e00640 1 --2- 172.21.15.72:0/2139528723 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2ab4100a80 0x7f2ab4100ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:54.738 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:54.731+0000 7f2ab3e00640 1 -- 172.21.15.72:0/2139528723 >> 172.21.15.72:0/2139528723 conn(0x7f2ab40fbff0 msgr2=0x7f2ab40fe430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:54.738 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:54.731+0000 7f2ab3e00640 1 -- 172.21.15.72:0/2139528723 shutdown_connections 2024-04-26T21:52:54.738 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:54.731+0000 7f2ab3e00640 1 -- 172.21.15.72:0/2139528723 wait complete. 2024-04-26T21:52:54.738 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:54.732+0000 7f2ab3e00640 1 Processor -- start 2024-04-26T21:52:54.738 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:54.732+0000 7f2ab3e00640 1 -- start start 2024-04-26T21:52:54.738 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:54.733+0000 7f2ab3e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2ab4100a80 0x7f2ab419abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:54.739 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:54.733+0000 7f2ab3e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2ab4101c70 0x7f2ab419b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:54.739 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:54.733+0000 7f2ab3e00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2ab4102630 0x7f2ab41a2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:54.739 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:54.733+0000 7f2ab3e00640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f2ab410c030 con 0x7f2ab4101c70 2024-04-26T21:52:54.739 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:54.733+0000 7f2ab3e00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f2ab410beb0 con 0x7f2ab4102630 2024-04-26T21:52:54.739 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:54.733+0000 7f2ab3e00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f2ab410c1b0 con 0x7f2ab4100a80 2024-04-26T21:52:54.739 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:54.733+0000 7f2aaa000640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2ab4101c70 0x7f2ab419b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:54.740 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:54.733+0000 7f2aaa000640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2ab4101c70 0x7f2ab419b100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:35080/0 (socket says 172.21.15.72:35080) 2024-04-26T21:52:54.740 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:54.733+0000 7f2aaa000640 1 -- 172.21.15.72:0/1094075255 learned_addr learned my addr 172.21.15.72:0/1094075255 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:52:54.740 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:54.733+0000 7f2ab3400640 1 --2- 172.21.15.72:0/1094075255 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2ab4102630 0x7f2ab41a2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:54.740 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:54.733+0000 7f2aaa000640 1 -- 172.21.15.72:0/1094075255 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2ab4100a80 msgr2=0x7f2ab419abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:54.740 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:54.733+0000 7f2aaa000640 1 --2- 172.21.15.72:0/1094075255 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2ab4100a80 0x7f2ab419abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:54.740 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:54.733+0000 7f2aaa000640 1 -- 172.21.15.72:0/1094075255 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2ab4102630 msgr2=0x7f2ab41a2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:54.741 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:54.733+0000 7f2aaa000640 1 --2- 172.21.15.72:0/1094075255 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2ab4102630 0x7f2ab41a2170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:54.741 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:54.733+0000 7f2aaa000640 1 -- 172.21.15.72:0/1094075255 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ab41a27e0 con 0x7f2ab4101c70 2024-04-26T21:52:54.741 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:54.734+0000 7f2aaa000640 1 --2- 172.21.15.72:0/1094075255 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2ab4101c70 0x7f2ab419b100 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f2a9c00c970 tx=0x7f2a9c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:54.741 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:54.734+0000 7f2aabe00640 1 -- 172.21.15.72:0/1094075255 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a9c007bb0 con 0x7f2ab4101c70 2024-04-26T21:52:54.741 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:54.734+0000 7f2aabe00640 1 -- 172.21.15.72:0/1094075255 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2a9c007d50 con 0x7f2ab4101c70 2024-04-26T21:52:54.741 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:54.734+0000 7f2aabe00640 1 -- 172.21.15.72:0/1094075255 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a9c004e40 con 0x7f2ab4101c70 2024-04-26T21:52:54.742 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:54.734+0000 7f2ab3e00640 1 -- 172.21.15.72:0/1094075255 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ab41a2ad0 con 0x7f2ab4101c70 2024-04-26T21:52:54.742 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:54.734+0000 7f2ab3e00640 1 -- 172.21.15.72:0/1094075255 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ab41a2fd0 con 0x7f2ab4101c70 2024-04-26T21:52:54.742 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:54.735+0000 7f2ab3e00640 1 -- 172.21.15.72:0/1094075255 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ab4107e70 con 0x7f2ab4101c70 2024-04-26T21:52:54.743 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:54.737+0000 7f2aabe00640 1 -- 172.21.15.72:0/1094075255 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f2a9c005020 con 0x7f2ab4101c70 2024-04-26T21:52:54.744 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:54.738+0000 7f2aabe00640 1 --2- 172.21.15.72:0/1094075255 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f2a980a3bb0 0x7f2a980a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:54.744 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:54.739+0000 7f2aabe00640 1 -- 172.21.15.72:0/1094075255 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7f2a9c0d0180 con 0x7f2ab4101c70 2024-04-26T21:52:54.744 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:54.739+0000 7f2ab2a00640 1 --2- 172.21.15.72:0/1094075255 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f2a980a3bb0 0x7f2a980a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:54.744 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:54.739+0000 7f2ab2a00640 1 --2- 172.21.15.72:0/1094075255 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f2a980a3bb0 0x7f2a980a6070 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f2aa4005fd0 tx=0x7f2aa400b510 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:54.745 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:54.743+0000 7f2aabe00640 1 -- 172.21.15.72:0/1094075255 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f2a9c014030 con 0x7f2ab4101c70 2024-04-26T21:52:54.988 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:54.988+0000 7f2ab3e00640 1 -- 172.21.15.72:0/1094075255 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7f2ab402dad0 con 0x7f2ab4101c70 2024-04-26T21:52:55.314 INFO:journalctl@ceph.mds.b.smithi076.stdout:Apr 26 21:52:55 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-mds-b[61253]: 2024-04-26T21:52:55.016+0000 7f6585400700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-26T21:52:55.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:55 smithi076 ceph-mon[31823]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-26T21:52:55.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:55 smithi076 ceph-mon[31823]: from='client.? 172.21.15.72:0/304301699' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-04-26T21:52:55.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:55 smithi076 ceph-mon[31823]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-26T21:52:55.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:55 smithi076 ceph-mon[31823]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-26T21:52:55.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:55 smithi076 ceph-mon[31823]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-04-26T21:52:55.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:55 smithi076 ceph-mon[31823]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-04-26T21:52:55.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:55 smithi076 ceph-mon[31823]: from='client.? 172.21.15.72:0/1094075255' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-26T21:52:55.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:55 smithi164 ceph-mon[31916]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-26T21:52:55.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:55 smithi164 ceph-mon[31916]: from='client.? 172.21.15.72:0/304301699' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-04-26T21:52:55.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:55 smithi164 ceph-mon[31916]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-26T21:52:55.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:55 smithi164 ceph-mon[31916]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-26T21:52:55.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:55 smithi164 ceph-mon[31916]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-04-26T21:52:55.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:55 smithi164 ceph-mon[31916]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-04-26T21:52:55.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:55 smithi164 ceph-mon[31916]: from='client.? 172.21.15.72:0/1094075255' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-26T21:52:55.527 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:55.526+0000 7f2aabe00640 1 -- 172.21.15.72:0/1094075255 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v20) ==== 121+0+0 (secure 0 0 0) 0x7f2a9c09cc10 con 0x7f2ab4101c70 2024-04-26T21:52:55.531 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:55.530+0000 7f2ab3e00640 1 -- 172.21.15.72:0/1094075255 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f2a980a3bb0 msgr2=0x7f2a980a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:55.531 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:55.530+0000 7f2ab3e00640 1 --2- 172.21.15.72:0/1094075255 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f2a980a3bb0 0x7f2a980a6070 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f2aa4005fd0 tx=0x7f2aa400b510 comp rx=0 tx=0).stop 2024-04-26T21:52:55.531 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:55.530+0000 7f2ab3e00640 1 -- 172.21.15.72:0/1094075255 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2ab4101c70 msgr2=0x7f2ab419b100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:55.531 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:55.530+0000 7f2ab3e00640 1 --2- 172.21.15.72:0/1094075255 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2ab4101c70 0x7f2ab419b100 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f2a9c00c970 tx=0x7f2a9c00ce30 comp rx=0 tx=0).stop 2024-04-26T21:52:55.531 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:55.530+0000 7f2ab3e00640 1 -- 172.21.15.72:0/1094075255 shutdown_connections 2024-04-26T21:52:55.531 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:55.530+0000 7f2ab3e00640 1 --2- 172.21.15.72:0/1094075255 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2ab4102630 0x7f2ab41a2170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:55.531 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:55.530+0000 7f2ab3e00640 1 --2- 172.21.15.72:0/1094075255 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2ab4101c70 0x7f2ab419b100 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:55.531 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:55.530+0000 7f2ab3e00640 1 --2- 172.21.15.72:0/1094075255 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f2a980a3bb0 0x7f2a980a6070 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:55.531 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:55.530+0000 7f2ab3e00640 1 --2- 172.21.15.72:0/1094075255 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2ab4100a80 0x7f2ab419abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:55.532 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:55.530+0000 7f2ab3e00640 1 -- 172.21.15.72:0/1094075255 >> 172.21.15.72:0/1094075255 conn(0x7f2ab40fbff0 msgr2=0x7f2ab4105040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:55.532 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:55.531+0000 7f2ab3e00640 1 -- 172.21.15.72:0/1094075255 shutdown_connections 2024-04-26T21:52:55.532 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:55.531+0000 7f2ab3e00640 1 -- 172.21.15.72:0/1094075255 wait complete. 2024-04-26T21:52:55.559 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:55 smithi072 ceph-mon[28138]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-26T21:52:55.559 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:55 smithi072 ceph-mon[28138]: from='client.? 172.21.15.72:0/304301699' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-04-26T21:52:55.559 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:55 smithi072 ceph-mon[28138]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-26T21:52:55.559 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:55 smithi072 ceph-mon[28138]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-26T21:52:55.559 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:55 smithi072 ceph-mon[28138]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-04-26T21:52:55.559 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:55 smithi072 ceph-mon[28138]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-04-26T21:52:55.559 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:55 smithi072 ceph-mon[28138]: from='client.? 172.21.15.72:0/1094075255' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-26T21:52:55.563 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2024-04-26T21:52:55.757 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:55.757+0000 7fecdd200640 1 -- 172.21.15.72:0/3313144843 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fecd8105c60 msgr2=0x7fecd81060e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:55.758 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:55.757+0000 7fecdd200640 1 --2- 172.21.15.72:0/3313144843 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fecd8105c60 0x7fecd81060e0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fecc0009900 tx=0x7fecc002a3f0 comp rx=0 tx=0).stop 2024-04-26T21:52:55.758 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:55.757+0000 7fecdd200640 1 -- 172.21.15.72:0/3313144843 shutdown_connections 2024-04-26T21:52:55.758 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:55.757+0000 7fecdd200640 1 --2- 172.21.15.72:0/3313144843 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fecd8106620 0x7fecd810ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:55.758 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:55.757+0000 7fecdd200640 1 --2- 172.21.15.72:0/3313144843 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fecd8105c60 0x7fecd81060e0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:55.758 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:55.757+0000 7fecdd200640 1 --2- 172.21.15.72:0/3313144843 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fecd8104a70 0x7fecd8104e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:55.758 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:55.757+0000 7fecdd200640 1 -- 172.21.15.72:0/3313144843 >> 172.21.15.72:0/3313144843 conn(0x7fecd80fffe0 msgr2=0x7fecd8102420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:55.758 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:55.757+0000 7fecdd200640 1 -- 172.21.15.72:0/3313144843 shutdown_connections 2024-04-26T21:52:55.758 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:55.757+0000 7fecdd200640 1 -- 172.21.15.72:0/3313144843 wait complete. 2024-04-26T21:52:55.760 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:55.758+0000 7fecdd200640 1 Processor -- start 2024-04-26T21:52:55.760 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:55.758+0000 7fecdd200640 1 -- start start 2024-04-26T21:52:55.760 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:55.758+0000 7fecdd200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fecd8104a70 0x7fecd819ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:55.760 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:55.758+0000 7fecdd200640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fecd8105c60 0x7fecd819b050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:55.761 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:55.758+0000 7fecdd200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fecd8106620 0x7fecd81a20c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:55.761 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:55.758+0000 7fecdd200640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fecd8110430 con 0x7fecd8105c60 2024-04-26T21:52:55.761 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:55.758+0000 7fecdd200640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fecd81102b0 con 0x7fecd8106620 2024-04-26T21:52:55.761 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:55.758+0000 7fecdd200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fecd81105b0 con 0x7fecd8104a70 2024-04-26T21:52:55.761 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:55.759+0000 7feccfe00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fecd8105c60 0x7fecd819b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:55.761 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:55.759+0000 7feccfe00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fecd8105c60 0x7fecd819b050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:35094/0 (socket says 172.21.15.72:35094) 2024-04-26T21:52:55.761 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:55.759+0000 7feccfe00640 1 -- 172.21.15.72:0/38170579 learned_addr learned my addr 172.21.15.72:0/38170579 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:52:55.761 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:55.759+0000 7fecd7e00640 1 --2- 172.21.15.72:0/38170579 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fecd8106620 0x7fecd81a20c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:55.761 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:55.759+0000 7fecd7400640 1 --2- 172.21.15.72:0/38170579 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fecd8104a70 0x7fecd819ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:55.761 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:55.759+0000 7feccfe00640 1 -- 172.21.15.72:0/38170579 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fecd8104a70 msgr2=0x7fecd819ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:55.761 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:55.759+0000 7feccfe00640 1 --2- 172.21.15.72:0/38170579 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fecd8104a70 0x7fecd819ab10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:55.761 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:55.759+0000 7feccfe00640 1 -- 172.21.15.72:0/38170579 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fecd8106620 msgr2=0x7fecd81a20c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:55.761 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:55.759+0000 7feccfe00640 1 --2- 172.21.15.72:0/38170579 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fecd8106620 0x7fecd81a20c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:55.761 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:55.759+0000 7feccfe00640 1 -- 172.21.15.72:0/38170579 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecd81a2730 con 0x7fecd8105c60 2024-04-26T21:52:55.762 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:55.760+0000 7feccfe00640 1 --2- 172.21.15.72:0/38170579 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fecd8105c60 0x7fecd819b050 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fecc0009230 tx=0x7fecc00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:55.762 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:55.760+0000 7fecd4c00640 1 -- 172.21.15.72:0/38170579 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecc003f030 con 0x7fecd8105c60 2024-04-26T21:52:55.762 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:55.760+0000 7fecdd200640 1 -- 172.21.15.72:0/38170579 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fecd81a29c0 con 0x7fecd8105c60 2024-04-26T21:52:55.762 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:55.760+0000 7fecd4c00640 1 -- 172.21.15.72:0/38170579 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fecc0003c90 con 0x7fecd8105c60 2024-04-26T21:52:55.762 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:55.760+0000 7fecd4c00640 1 -- 172.21.15.72:0/38170579 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecc0035070 con 0x7fecd8105c60 2024-04-26T21:52:55.762 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:55.760+0000 7fecdd200640 1 -- 172.21.15.72:0/38170579 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fecd81a2eb0 con 0x7fecd8105c60 2024-04-26T21:52:55.762 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:55.761+0000 7fecdd200640 1 -- 172.21.15.72:0/38170579 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fecd810be10 con 0x7fecd8105c60 2024-04-26T21:52:55.763 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:55.763+0000 7fecd4c00640 1 -- 172.21.15.72:0/38170579 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fecc0042030 con 0x7fecd8105c60 2024-04-26T21:52:55.765 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:55.764+0000 7fecd4c00640 1 --2- 172.21.15.72:0/38170579 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fecbc0a3bb0 0x7fecbc0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:55.765 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:55.765+0000 7fecd4c00640 1 -- 172.21.15.72:0/38170579 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7fecc00f0b20 con 0x7fecd8105c60 2024-04-26T21:52:55.765 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:55.765+0000 7fecd7400640 1 --2- 172.21.15.72:0/38170579 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fecbc0a3bb0 0x7fecbc0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:55.765 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:55.766+0000 7fecd7400640 1 --2- 172.21.15.72:0/38170579 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fecbc0a3bb0 0x7fecbc0a6070 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fecc8005fd0 tx=0x7fecc8005ce0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:55.769 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:55.769+0000 7fecd4c00640 1 -- 172.21.15.72:0/38170579 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fecc00bd710 con 0x7fecd8105c60 2024-04-26T21:52:56.013 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:56.012+0000 7fecdd200640 1 -- 172.21.15.72:0/38170579 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7fecd802dad0 con 0x7fecd8105c60 2024-04-26T21:52:56.549 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:56.549+0000 7fecd4c00640 1 -- 172.21.15.72:0/38170579 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v22) ==== 115+0+0 (secure 0 0 0) 0x7fecc00bd010 con 0x7fecd8105c60 2024-04-26T21:52:56.553 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:56.552+0000 7fecdd200640 1 -- 172.21.15.72:0/38170579 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fecbc0a3bb0 msgr2=0x7fecbc0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:56.553 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:56.552+0000 7fecdd200640 1 --2- 172.21.15.72:0/38170579 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fecbc0a3bb0 0x7fecbc0a6070 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fecc8005fd0 tx=0x7fecc8005ce0 comp rx=0 tx=0).stop 2024-04-26T21:52:56.553 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:56.552+0000 7fecdd200640 1 -- 172.21.15.72:0/38170579 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fecd8105c60 msgr2=0x7fecd819b050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:56.553 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:56.552+0000 7fecdd200640 1 --2- 172.21.15.72:0/38170579 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fecd8105c60 0x7fecd819b050 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fecc0009230 tx=0x7fecc00042f0 comp rx=0 tx=0).stop 2024-04-26T21:52:56.553 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:56.552+0000 7fecdd200640 1 -- 172.21.15.72:0/38170579 shutdown_connections 2024-04-26T21:52:56.554 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:56.553+0000 7fecdd200640 1 --2- 172.21.15.72:0/38170579 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fecd8106620 0x7fecd81a20c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:56.554 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:56.553+0000 7fecdd200640 1 --2- 172.21.15.72:0/38170579 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fecd8105c60 0x7fecd819b050 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:56.554 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:56.553+0000 7fecdd200640 1 --2- 172.21.15.72:0/38170579 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fecbc0a3bb0 0x7fecbc0a6070 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:56.554 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:56.553+0000 7fecdd200640 1 --2- 172.21.15.72:0/38170579 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fecd8104a70 0x7fecd819ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:56.554 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:56.553+0000 7fecdd200640 1 -- 172.21.15.72:0/38170579 >> 172.21.15.72:0/38170579 conn(0x7fecd80fffe0 msgr2=0x7fecd8108fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:56.554 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:56.553+0000 7fecdd200640 1 -- 172.21.15.72:0/38170579 shutdown_connections 2024-04-26T21:52:56.554 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:56.553+0000 7fecdd200640 1 -- 172.21.15.72:0/38170579 wait complete. 2024-04-26T21:52:56.579 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2} subvols for filesystem 'cephfs' 2024-04-26T21:52:56.579 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs qa 2024-04-26T21:52:56.767 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:56.766+0000 7f831f600640 1 -- 172.21.15.72:0/489933613 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8318102980 msgr2=0x7f8318102da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:56.767 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:56.766+0000 7f831f600640 1 --2- 172.21.15.72:0/489933613 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8318102980 0x7f8318102da0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f8308009ca0 tx=0x7f830802a8c0 comp rx=0 tx=0).stop 2024-04-26T21:52:56.767 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:56.766+0000 7f831f600640 1 -- 172.21.15.72:0/489933613 shutdown_connections 2024-04-26T21:52:56.767 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:56.766+0000 7f831f600640 1 --2- 172.21.15.72:0/489933613 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8318104530 0x7f831810adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:56.767 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:56.766+0000 7f831f600640 1 --2- 172.21.15.72:0/489933613 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8318103b70 0x7f8318103ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:56.767 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:56.766+0000 7f831f600640 1 --2- 172.21.15.72:0/489933613 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8318102980 0x7f8318102da0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:56.767 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:56.766+0000 7f831f600640 1 -- 172.21.15.72:0/489933613 >> 172.21.15.72:0/489933613 conn(0x7f83180fdf10 msgr2=0x7f8318100330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:56.767 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:56.766+0000 7f831f600640 1 -- 172.21.15.72:0/489933613 shutdown_connections 2024-04-26T21:52:56.768 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:56.766+0000 7f831f600640 1 -- 172.21.15.72:0/489933613 wait complete. 2024-04-26T21:52:56.770 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:56.767+0000 7f831f600640 1 Processor -- start 2024-04-26T21:52:56.770 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:56.768+0000 7f831f600640 1 -- start start 2024-04-26T21:52:56.770 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:56.768+0000 7f831f600640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8318102980 0x7f831819ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:56.770 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:56.768+0000 7f831f600640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8318103b70 0x7f831819b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:56.770 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:56.768+0000 7f831f600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8318104530 0x7f83181a22a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:56.770 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:56.768+0000 7f831f600640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f831810e3b0 con 0x7f8318102980 2024-04-26T21:52:56.771 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:56.768+0000 7f831f600640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f831810e230 con 0x7f8318103b70 2024-04-26T21:52:56.771 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:56.768+0000 7f831f600640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f831810e530 con 0x7f8318104530 2024-04-26T21:52:56.771 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:56.768+0000 7f831cc00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8318102980 0x7f831819ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:56.771 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:56.768+0000 7f831cc00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8318102980 0x7f831819ad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:35118/0 (socket says 172.21.15.72:35118) 2024-04-26T21:52:56.771 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:56.768+0000 7f831cc00640 1 -- 172.21.15.72:0/4137271583 learned_addr learned my addr 172.21.15.72:0/4137271583 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:52:56.771 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:56.768+0000 7f8317e00640 1 --2- 172.21.15.72:0/4137271583 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8318103b70 0x7f831819b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:56.771 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:56.769+0000 7f831d600640 1 --2- 172.21.15.72:0/4137271583 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8318104530 0x7f83181a22a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:56.771 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:56.769+0000 7f831cc00640 1 -- 172.21.15.72:0/4137271583 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8318104530 msgr2=0x7f83181a22a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:56.771 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:56.769+0000 7f831cc00640 1 --2- 172.21.15.72:0/4137271583 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8318104530 0x7f83181a22a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:56.771 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:56.769+0000 7f831cc00640 1 -- 172.21.15.72:0/4137271583 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8318103b70 msgr2=0x7f831819b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:56.771 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:56.769+0000 7f831cc00640 1 --2- 172.21.15.72:0/4137271583 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8318103b70 0x7f831819b270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:56.771 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:56.769+0000 7f831cc00640 1 -- 172.21.15.72:0/4137271583 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83181a2940 con 0x7f8318102980 2024-04-26T21:52:56.771 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:56.769+0000 7f831d600640 1 --2- 172.21.15.72:0/4137271583 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8318104530 0x7f83181a22a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:52:56.771 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:56.769+0000 7f831cc00640 1 --2- 172.21.15.72:0/4137271583 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8318102980 0x7f831819ad30 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f830802aeb0 tx=0x7f8308003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:56.772 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:56.769+0000 7f8315600640 1 -- 172.21.15.72:0/4137271583 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8308003d60 con 0x7f8318102980 2024-04-26T21:52:56.772 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:56.769+0000 7f8315600640 1 -- 172.21.15.72:0/4137271583 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f83080044d0 con 0x7f8318102980 2024-04-26T21:52:56.772 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:56.769+0000 7f831f600640 1 -- 172.21.15.72:0/4137271583 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83181a2bd0 con 0x7f8318102980 2024-04-26T21:52:56.772 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:56.769+0000 7f8315600640 1 -- 172.21.15.72:0/4137271583 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8308030c80 con 0x7f8318102980 2024-04-26T21:52:56.772 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:56.770+0000 7f831f600640 1 -- 172.21.15.72:0/4137271583 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83181a3170 con 0x7f8318102980 2024-04-26T21:52:56.772 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:56.771+0000 7f831f600640 1 -- 172.21.15.72:0/4137271583 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83181a3340 con 0x7f8318102980 2024-04-26T21:52:56.773 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:56.773+0000 7f8315600640 1 -- 172.21.15.72:0/4137271583 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f8308004040 con 0x7f8318102980 2024-04-26T21:52:56.774 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:56.774+0000 7f8315600640 1 --2- 172.21.15.72:0/4137271583 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f82fc0a3c30 0x7f82fc0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:56.774 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:56.774+0000 7f8315600640 1 -- 172.21.15.72:0/4137271583 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7f83080f1810 con 0x7f8318102980 2024-04-26T21:52:56.774 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:56.774+0000 7f8317e00640 1 --2- 172.21.15.72:0/4137271583 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f82fc0a3c30 0x7f82fc0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:56.775 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:56.775+0000 7f8317e00640 1 --2- 172.21.15.72:0/4137271583 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f82fc0a3c30 0x7f82fc0a60f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f8300009c50 tx=0x7f8300005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:56.778 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:56.778+0000 7f8315600640 1 -- 172.21.15.72:0/4137271583 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f83080be2a0 con 0x7f8318102980 2024-04-26T21:52:56.815 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:56 smithi076 ceph-mon[31823]: from='client.? 172.21.15.72:0/1094075255' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-26T21:52:56.815 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:56 smithi076 ceph-mon[31823]: mds.? [v2:172.21.15.76:6836/1150678991,v1:172.21.15.76:6837/1150678991] up:active 2024-04-26T21:52:56.815 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:56 smithi076 ceph-mon[31823]: daemon mds.e assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-26T21:52:56.815 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:56 smithi076 ceph-mon[31823]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-04-26T21:52:56.815 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:56 smithi076 ceph-mon[31823]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:creating} 9 up:standby 2024-04-26T21:52:56.816 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:56 smithi076 ceph-mon[31823]: daemon mds.e is now active in filesystem cephfs as rank 2 2024-04-26T21:52:56.816 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:56 smithi076 ceph-mon[31823]: pgmap v370: 97 pgs: 97 active+clean; 579 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 8 op/s 2024-04-26T21:52:56.816 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:56 smithi076 ceph-mon[31823]: from='client.? 172.21.15.72:0/38170579' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-26T21:52:56.822 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:56 smithi072 ceph-mon[28138]: from='client.? 172.21.15.72:0/1094075255' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-26T21:52:56.822 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:56 smithi072 ceph-mon[28138]: mds.? [v2:172.21.15.76:6836/1150678991,v1:172.21.15.76:6837/1150678991] up:active 2024-04-26T21:52:56.822 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:56 smithi072 ceph-mon[28138]: daemon mds.e assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-26T21:52:56.822 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:56 smithi072 ceph-mon[28138]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-04-26T21:52:56.822 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:56 smithi072 ceph-mon[28138]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:creating} 9 up:standby 2024-04-26T21:52:56.822 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:56 smithi072 ceph-mon[28138]: daemon mds.e is now active in filesystem cephfs as rank 2 2024-04-26T21:52:56.822 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:56 smithi072 ceph-mon[28138]: pgmap v370: 97 pgs: 97 active+clean; 579 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 8 op/s 2024-04-26T21:52:56.822 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:56 smithi072 ceph-mon[28138]: from='client.? 172.21.15.72:0/38170579' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-26T21:52:56.941 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:56 smithi164 ceph-mon[31916]: from='client.? 172.21.15.72:0/1094075255' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-26T21:52:56.941 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:56 smithi164 ceph-mon[31916]: mds.? [v2:172.21.15.76:6836/1150678991,v1:172.21.15.76:6837/1150678991] up:active 2024-04-26T21:52:56.941 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:56 smithi164 ceph-mon[31916]: daemon mds.e assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-26T21:52:56.941 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:56 smithi164 ceph-mon[31916]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-04-26T21:52:56.941 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:56 smithi164 ceph-mon[31916]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:creating} 9 up:standby 2024-04-26T21:52:56.941 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:56 smithi164 ceph-mon[31916]: daemon mds.e is now active in filesystem cephfs as rank 2 2024-04-26T21:52:56.942 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:56 smithi164 ceph-mon[31916]: pgmap v370: 97 pgs: 97 active+clean; 579 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 8 op/s 2024-04-26T21:52:56.942 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:56 smithi164 ceph-mon[31916]: from='client.? 172.21.15.72:0/38170579' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-26T21:52:57.005 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:57.005+0000 7f831f600640 1 -- 172.21.15.72:0/4137271583 --> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f83181a3510 con 0x7f82fc0a3c30 2024-04-26T21:52:57.322 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 26 21:52:57 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-mgr-x[28363]: 2024-04-26T21:52:57.018+0000 7fb10a800700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T21:52:57.323 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 26 21:52:57 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-mgr-x[28363]: 2024-04-26T21:52:57.018+0000 7fb10a800700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T21:52:57.323 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 26 21:52:57 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-mgr-x[28363]: 2024-04-26T21:52:57.018+0000 7fb10a800700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T21:52:57.323 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 26 21:52:57 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-mgr-x[28363]: 2024-04-26T21:52:57.018+0000 7fb10a800700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T21:52:57.323 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 26 21:52:57 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-mgr-x[28363]: 2024-04-26T21:52:57.018+0000 7fb10a800700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T21:52:57.627 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:57 smithi164 ceph-mon[31916]: from='client.? 172.21.15.72:0/38170579' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-26T21:52:57.627 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:57 smithi164 ceph-mon[31916]: mds.? [v2:172.21.15.76:6834/3517429435,v1:172.21.15.76:6835/3517429435] up:active 2024-04-26T21:52:57.627 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:57 smithi164 ceph-mon[31916]: daemon mds.k assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-04-26T21:52:57.627 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:57 smithi164 ceph-mon[31916]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 9 up:standby 2024-04-26T21:52:57.628 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:57 smithi164 ceph-mon[31916]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=e=up:active,3=k=up:creating} 8 up:standby 2024-04-26T21:52:57.628 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:57 smithi164 ceph-mon[31916]: daemon mds.k is now active in filesystem cephfs as rank 3 2024-04-26T21:52:57.815 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:57 smithi076 ceph-mon[31823]: from='client.? 172.21.15.72:0/38170579' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-26T21:52:57.815 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:57 smithi076 ceph-mon[31823]: mds.? [v2:172.21.15.76:6834/3517429435,v1:172.21.15.76:6835/3517429435] up:active 2024-04-26T21:52:57.815 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:57 smithi076 ceph-mon[31823]: daemon mds.k assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-04-26T21:52:57.815 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:57 smithi076 ceph-mon[31823]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 9 up:standby 2024-04-26T21:52:57.815 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:57 smithi076 ceph-mon[31823]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=e=up:active,3=k=up:creating} 8 up:standby 2024-04-26T21:52:57.815 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:57 smithi076 ceph-mon[31823]: daemon mds.k is now active in filesystem cephfs as rank 3 2024-04-26T21:52:57.822 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:57 smithi072 ceph-mon[28138]: from='client.? 172.21.15.72:0/38170579' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-26T21:52:57.822 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:57 smithi072 ceph-mon[28138]: mds.? [v2:172.21.15.76:6834/3517429435,v1:172.21.15.76:6835/3517429435] up:active 2024-04-26T21:52:57.822 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:57 smithi072 ceph-mon[28138]: daemon mds.k assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-04-26T21:52:57.822 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:57 smithi072 ceph-mon[28138]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 9 up:standby 2024-04-26T21:52:57.822 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:57 smithi072 ceph-mon[28138]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=e=up:active,3=k=up:creating} 8 up:standby 2024-04-26T21:52:57.822 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:57 smithi072 ceph-mon[28138]: daemon mds.k is now active in filesystem cephfs as rank 3 2024-04-26T21:52:58.548 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:58.548+0000 7f8315600640 1 -- 172.21.15.72:0/4137271583 <== mgr.14152 v2:172.21.15.72:6800/588150172 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f83181a3510 con 0x7f82fc0a3c30 2024-04-26T21:52:58.552 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:58.551+0000 7f831f600640 1 -- 172.21.15.72:0/4137271583 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f82fc0a3c30 msgr2=0x7f82fc0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:58.552 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:58.551+0000 7f831f600640 1 --2- 172.21.15.72:0/4137271583 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f82fc0a3c30 0x7f82fc0a60f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f8300009c50 tx=0x7f8300005f90 comp rx=0 tx=0).stop 2024-04-26T21:52:58.552 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:58.551+0000 7f831f600640 1 -- 172.21.15.72:0/4137271583 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8318102980 msgr2=0x7f831819ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:58.552 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:58.551+0000 7f831f600640 1 --2- 172.21.15.72:0/4137271583 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8318102980 0x7f831819ad30 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f830802aeb0 tx=0x7f8308003970 comp rx=0 tx=0).stop 2024-04-26T21:52:58.552 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:58.551+0000 7f831f600640 1 -- 172.21.15.72:0/4137271583 shutdown_connections 2024-04-26T21:52:58.552 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:58.551+0000 7f831f600640 1 --2- 172.21.15.72:0/4137271583 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f82fc0a3c30 0x7f82fc0a60f0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:58.552 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:58.551+0000 7f831f600640 1 --2- 172.21.15.72:0/4137271583 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8318104530 0x7f83181a22a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:58.552 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:58.551+0000 7f831f600640 1 --2- 172.21.15.72:0/4137271583 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8318103b70 0x7f831819b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:58.552 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:58.551+0000 7f831f600640 1 --2- 172.21.15.72:0/4137271583 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8318102980 0x7f831819ad30 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:58.552 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:58.551+0000 7f831f600640 1 -- 172.21.15.72:0/4137271583 >> 172.21.15.72:0/4137271583 conn(0x7f83180fdf10 msgr2=0x7f83181093e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:58.552 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:58.552+0000 7f831f600640 1 -- 172.21.15.72:0/4137271583 shutdown_connections 2024-04-26T21:52:58.552 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:58.552+0000 7f831f600640 1 -- 172.21.15.72:0/4137271583 wait complete. 2024-04-26T21:52:58.572 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 --group_name qa 2024-04-26T21:52:58.764 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:58.763+0000 7fdcf7a00640 1 -- 172.21.15.72:0/4256941255 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdcf010ab40 msgr2=0x7fdcf01113d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:58.764 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:58.763+0000 7fdcf7a00640 1 --2- 172.21.15.72:0/4256941255 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdcf010ab40 0x7fdcf01113d0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fdce8009ca0 tx=0x7fdce802a8c0 comp rx=0 tx=0).stop 2024-04-26T21:52:58.764 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:58.763+0000 7fdcf7a00640 1 -- 172.21.15.72:0/4256941255 shutdown_connections 2024-04-26T21:52:58.764 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:58.763+0000 7fdcf7a00640 1 --2- 172.21.15.72:0/4256941255 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdcf010ab40 0x7fdcf01113d0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:58.764 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:58.763+0000 7fdcf7a00640 1 --2- 172.21.15.72:0/4256941255 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdcf010a180 0x7fdcf010a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:58.764 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:58.763+0000 7fdcf7a00640 1 --2- 172.21.15.72:0/4256941255 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdcf0108f90 0x7fdcf01093b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:58.764 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:58.763+0000 7fdcf7a00640 1 -- 172.21.15.72:0/4256941255 >> 172.21.15.72:0/4256941255 conn(0x7fdcf0077f30 msgr2=0x7fdcf007a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:58.764 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:58.763+0000 7fdcf7a00640 1 -- 172.21.15.72:0/4256941255 shutdown_connections 2024-04-26T21:52:58.764 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:58.764+0000 7fdcf7a00640 1 -- 172.21.15.72:0/4256941255 wait complete. 2024-04-26T21:52:58.767 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:58.764+0000 7fdcf7a00640 1 Processor -- start 2024-04-26T21:52:58.767 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:58.765+0000 7fdcf7a00640 1 -- start start 2024-04-26T21:52:58.767 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:58.765+0000 7fdcf7a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdcf0108f90 0x7fdcf019f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:58.767 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:58.765+0000 7fdcf7a00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdcf010a180 0x7fdcf019f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:58.767 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:58.765+0000 7fdcf7a00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdcf010ab40 0x7fdcf01a6650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:58.767 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:58.765+0000 7fdcf7a00640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fdcf0114a30 con 0x7fdcf010ab40 2024-04-26T21:52:58.767 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:58.765+0000 7fdcf7a00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fdcf01148b0 con 0x7fdcf010a180 2024-04-26T21:52:58.767 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:58.765+0000 7fdcf7a00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fdcf0114bb0 con 0x7fdcf0108f90 2024-04-26T21:52:58.768 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:58.765+0000 7fdcf5a00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdcf010ab40 0x7fdcf01a6650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:58.768 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:58.765+0000 7fdcf5a00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdcf010ab40 0x7fdcf01a6650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:35134/0 (socket says 172.21.15.72:35134) 2024-04-26T21:52:58.768 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:58.765+0000 7fdcf5a00640 1 -- 172.21.15.72:0/2126233652 learned_addr learned my addr 172.21.15.72:0/2126233652 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:52:58.768 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:58.766+0000 7fdcf5000640 1 --2- 172.21.15.72:0/2126233652 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdcf0108f90 0x7fdcf019f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:58.768 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:58.766+0000 7fdcefe00640 1 --2- 172.21.15.72:0/2126233652 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdcf010a180 0x7fdcf019f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:58.768 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:58.766+0000 7fdcf5a00640 1 -- 172.21.15.72:0/2126233652 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdcf0108f90 msgr2=0x7fdcf019f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:58.768 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:58.766+0000 7fdcf5a00640 1 --2- 172.21.15.72:0/2126233652 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdcf0108f90 0x7fdcf019f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:58.768 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:58.766+0000 7fdcf5a00640 1 -- 172.21.15.72:0/2126233652 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdcf010a180 msgr2=0x7fdcf019f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:58.768 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:58.766+0000 7fdcf5a00640 1 --2- 172.21.15.72:0/2126233652 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdcf010a180 0x7fdcf019f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:58.768 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:58.766+0000 7fdcf5a00640 1 -- 172.21.15.72:0/2126233652 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcf01a6cc0 con 0x7fdcf010ab40 2024-04-26T21:52:58.768 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:58.766+0000 7fdcf5a00640 1 --2- 172.21.15.72:0/2126233652 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdcf010ab40 0x7fdcf01a6650 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fdce8031040 tx=0x7fdce8003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:58.768 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:58.766+0000 7fdced600640 1 -- 172.21.15.72:0/2126233652 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdce8004240 con 0x7fdcf010ab40 2024-04-26T21:52:58.768 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:58.766+0000 7fdced600640 1 -- 172.21.15.72:0/2126233652 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdce8003e40 con 0x7fdcf010ab40 2024-04-26T21:52:58.769 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:58.766+0000 7fdced600640 1 -- 172.21.15.72:0/2126233652 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdce803b410 con 0x7fdcf010ab40 2024-04-26T21:52:58.769 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:58.766+0000 7fdcf7a00640 1 -- 172.21.15.72:0/2126233652 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdcf01a6f50 con 0x7fdcf010ab40 2024-04-26T21:52:58.769 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:58.767+0000 7fdcf7a00640 1 -- 172.21.15.72:0/2126233652 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdcf01a7410 con 0x7fdcf010ab40 2024-04-26T21:52:58.769 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:58.768+0000 7fdcf7a00640 1 -- 172.21.15.72:0/2126233652 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdcf0110300 con 0x7fdcf010ab40 2024-04-26T21:52:58.770 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:58.770+0000 7fdced600640 1 -- 172.21.15.72:0/2126233652 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fdce80305c0 con 0x7fdcf010ab40 2024-04-26T21:52:58.772 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:58.771+0000 7fdced600640 1 --2- 172.21.15.72:0/2126233652 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fdcd80a3bb0 0x7fdcd80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:58.772 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:58.771+0000 7fdced600640 1 -- 172.21.15.72:0/2126233652 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7fdce80f1dc0 con 0x7fdcf010ab40 2024-04-26T21:52:58.772 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:58.772+0000 7fdcf5000640 1 --2- 172.21.15.72:0/2126233652 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fdcd80a3bb0 0x7fdcd80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:58.772 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:58.772+0000 7fdcf5000640 1 --2- 172.21.15.72:0/2126233652 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fdcd80a3bb0 0x7fdcd80a6070 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fdcf0109fe0 tx=0x7fdce4008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:58.774 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:58.774+0000 7fdced600640 1 -- 172.21.15.72:0/2126233652 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fdce80be8e0 con 0x7fdcf010ab40 2024-04-26T21:52:58.822 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:58 smithi072 ceph-mon[28138]: from='client.14766 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:52:58.822 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:58 smithi072 ceph-mon[28138]: mds.? [v2:172.21.15.76:6832/3869483750,v1:172.21.15.76:6833/3869483750] up:active 2024-04-26T21:52:58.822 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:58 smithi072 ceph-mon[28138]: daemon mds.i assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-04-26T21:52:58.822 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:58 smithi072 ceph-mon[28138]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-26T21:52:58.822 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:58 smithi072 ceph-mon[28138]: Cluster is now healthy 2024-04-26T21:52:58.822 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:58 smithi072 ceph-mon[28138]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=e=up:active,3=k=up:active} 8 up:standby 2024-04-26T21:52:58.823 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:58 smithi072 ceph-mon[28138]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-04-26T21:52:58.823 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:58 smithi072 ceph-mon[28138]: daemon mds.i is now active in filesystem cephfs as rank 4 2024-04-26T21:52:58.823 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:58 smithi072 ceph-mon[28138]: pgmap v371: 97 pgs: 97 active+clean; 580 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s wr, 9 op/s 2024-04-26T21:52:58.823 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:58 smithi072 ceph-mon[28138]: mds.? [v2:172.21.15.164:6838/1636226820,v1:172.21.15.164:6839/1636226820] up:active 2024-04-26T21:52:58.823 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:52:58 smithi072 ceph-mon[28138]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-26T21:52:58.941 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:58 smithi164 ceph-mon[31916]: from='client.14766 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:52:58.941 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:58 smithi164 ceph-mon[31916]: mds.? [v2:172.21.15.76:6832/3869483750,v1:172.21.15.76:6833/3869483750] up:active 2024-04-26T21:52:58.941 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:58 smithi164 ceph-mon[31916]: daemon mds.i assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-04-26T21:52:58.941 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:58 smithi164 ceph-mon[31916]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-26T21:52:58.941 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:58 smithi164 ceph-mon[31916]: Cluster is now healthy 2024-04-26T21:52:58.941 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:58 smithi164 ceph-mon[31916]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=e=up:active,3=k=up:active} 8 up:standby 2024-04-26T21:52:58.941 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:58 smithi164 ceph-mon[31916]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-04-26T21:52:58.941 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:58 smithi164 ceph-mon[31916]: daemon mds.i is now active in filesystem cephfs as rank 4 2024-04-26T21:52:58.941 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:58 smithi164 ceph-mon[31916]: pgmap v371: 97 pgs: 97 active+clean; 580 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s wr, 9 op/s 2024-04-26T21:52:58.942 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:58 smithi164 ceph-mon[31916]: mds.? [v2:172.21.15.164:6838/1636226820,v1:172.21.15.164:6839/1636226820] up:active 2024-04-26T21:52:58.942 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:52:58 smithi164 ceph-mon[31916]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-26T21:52:59.013 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:59.012+0000 7fdcf7a00640 1 -- 172.21.15.72:0/2126233652 --> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fdcf0062d70 con 0x7fdcd80a3bb0 2024-04-26T21:52:59.065 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:58 smithi076 ceph-mon[31823]: from='client.14766 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:52:59.066 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:58 smithi076 ceph-mon[31823]: mds.? [v2:172.21.15.76:6832/3869483750,v1:172.21.15.76:6833/3869483750] up:active 2024-04-26T21:52:59.066 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:58 smithi076 ceph-mon[31823]: daemon mds.i assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-04-26T21:52:59.066 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:58 smithi076 ceph-mon[31823]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-26T21:52:59.066 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:58 smithi076 ceph-mon[31823]: Cluster is now healthy 2024-04-26T21:52:59.066 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:58 smithi076 ceph-mon[31823]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=e=up:active,3=k=up:active} 8 up:standby 2024-04-26T21:52:59.066 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:58 smithi076 ceph-mon[31823]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-04-26T21:52:59.066 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:58 smithi076 ceph-mon[31823]: daemon mds.i is now active in filesystem cephfs as rank 4 2024-04-26T21:52:59.066 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:58 smithi076 ceph-mon[31823]: pgmap v371: 97 pgs: 97 active+clean; 580 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s wr, 9 op/s 2024-04-26T21:52:59.066 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:58 smithi076 ceph-mon[31823]: mds.? [v2:172.21.15.164:6838/1636226820,v1:172.21.15.164:6839/1636226820] up:active 2024-04-26T21:52:59.066 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:52:58 smithi076 ceph-mon[31823]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-26T21:52:59.081 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:59.081+0000 7fdced600640 1 -- 172.21.15.72:0/2126233652 <== mgr.14152 v2:172.21.15.72:6800/588150172 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fdcf0062d70 con 0x7fdcd80a3bb0 2024-04-26T21:52:59.084 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:59.084+0000 7fdcf7a00640 1 -- 172.21.15.72:0/2126233652 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fdcd80a3bb0 msgr2=0x7fdcd80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:59.084 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:59.084+0000 7fdcf7a00640 1 --2- 172.21.15.72:0/2126233652 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fdcd80a3bb0 0x7fdcd80a6070 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fdcf0109fe0 tx=0x7fdce4008040 comp rx=0 tx=0).stop 2024-04-26T21:52:59.085 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:59.084+0000 7fdcf7a00640 1 -- 172.21.15.72:0/2126233652 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdcf010ab40 msgr2=0x7fdcf01a6650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:59.085 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:59.084+0000 7fdcf7a00640 1 --2- 172.21.15.72:0/2126233652 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdcf010ab40 0x7fdcf01a6650 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fdce8031040 tx=0x7fdce8003ce0 comp rx=0 tx=0).stop 2024-04-26T21:52:59.085 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:59.084+0000 7fdcf7a00640 1 -- 172.21.15.72:0/2126233652 shutdown_connections 2024-04-26T21:52:59.085 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:59.084+0000 7fdcf7a00640 1 --2- 172.21.15.72:0/2126233652 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdcf010ab40 0x7fdcf01a6650 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:59.085 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:59.084+0000 7fdcf7a00640 1 --2- 172.21.15.72:0/2126233652 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdcf010a180 0x7fdcf019f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:59.085 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:59.084+0000 7fdcf7a00640 1 --2- 172.21.15.72:0/2126233652 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fdcd80a3bb0 0x7fdcd80a6070 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:59.085 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:59.084+0000 7fdcf7a00640 1 --2- 172.21.15.72:0/2126233652 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdcf0108f90 0x7fdcf019f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:59.085 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:59.084+0000 7fdcf7a00640 1 -- 172.21.15.72:0/2126233652 >> 172.21.15.72:0/2126233652 conn(0x7fdcf0077f30 msgr2=0x7fdcf010d4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:59.085 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:59.084+0000 7fdcf7a00640 1 -- 172.21.15.72:0/2126233652 shutdown_connections 2024-04-26T21:52:59.085 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:59.084+0000 7fdcf7a00640 1 -- 172.21.15.72:0/2126233652 wait complete. 2024-04-26T21:52:59.105 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-04-26T21:52:59.302 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:59.301+0000 7f014e200640 1 -- 172.21.15.72:0/2661518191 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0148106740 msgr2=0x7f014810cfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:59.302 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:59.301+0000 7f014e200640 1 --2- 172.21.15.72:0/2661518191 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0148106740 0x7f014810cfd0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f0140009ca0 tx=0x7f014002a8b0 comp rx=0 tx=0).stop 2024-04-26T21:52:59.302 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:59.302+0000 7f014e200640 1 -- 172.21.15.72:0/2661518191 shutdown_connections 2024-04-26T21:52:59.303 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:59.302+0000 7f014e200640 1 --2- 172.21.15.72:0/2661518191 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0148106740 0x7f014810cfd0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:59.303 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:59.302+0000 7f014e200640 1 --2- 172.21.15.72:0/2661518191 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0148105d80 0x7f0148106200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:59.303 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:59.302+0000 7f014e200640 1 --2- 172.21.15.72:0/2661518191 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0148104b90 0x7f0148104fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:59.303 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:59.302+0000 7f014e200640 1 -- 172.21.15.72:0/2661518191 >> 172.21.15.72:0/2661518191 conn(0x7f0148100120 msgr2=0x7f0148102540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:59.303 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:59.302+0000 7f014e200640 1 -- 172.21.15.72:0/2661518191 shutdown_connections 2024-04-26T21:52:59.303 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:59.302+0000 7f014e200640 1 -- 172.21.15.72:0/2661518191 wait complete. 2024-04-26T21:52:59.304 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:59.303+0000 7f014e200640 1 Processor -- start 2024-04-26T21:52:59.304 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:59.303+0000 7f014e200640 1 -- start start 2024-04-26T21:52:59.304 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:59.303+0000 7f014e200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0148104b90 0x7f014819ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:59.304 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:59.303+0000 7f014e200640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0148105d80 0x7f014819b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:59.304 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:59.303+0000 7f014e200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0148106740 0x7f01481a21a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:59.305 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:59.303+0000 7f014e200640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f01481105c0 con 0x7f0148105d80 2024-04-26T21:52:59.305 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:59.303+0000 7f014e200640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f0148110440 con 0x7f0148104b90 2024-04-26T21:52:59.305 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:59.303+0000 7f014e200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f0148110740 con 0x7f0148106740 2024-04-26T21:52:59.305 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:59.303+0000 7f013fe00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0148105d80 0x7f014819b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:59.305 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:59.304+0000 7f013fe00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0148105d80 0x7f014819b170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:35146/0 (socket says 172.21.15.72:35146) 2024-04-26T21:52:59.305 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:59.304+0000 7f013fe00640 1 -- 172.21.15.72:0/3709319175 learned_addr learned my addr 172.21.15.72:0/3709319175 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:52:59.305 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:59.304+0000 7f0147e00640 1 --2- 172.21.15.72:0/3709319175 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0148106740 0x7f01481a21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:59.305 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:59.304+0000 7f0147400640 1 --2- 172.21.15.72:0/3709319175 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0148104b90 0x7f014819ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:59.305 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:59.304+0000 7f013fe00640 1 -- 172.21.15.72:0/3709319175 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0148106740 msgr2=0x7f01481a21a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:59.305 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:59.304+0000 7f013fe00640 1 --2- 172.21.15.72:0/3709319175 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0148106740 0x7f01481a21a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:59.305 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:59.304+0000 7f013fe00640 1 -- 172.21.15.72:0/3709319175 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0148104b90 msgr2=0x7f014819ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:59.305 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:59.304+0000 7f013fe00640 1 --2- 172.21.15.72:0/3709319175 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0148104b90 0x7f014819ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:59.305 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:59.304+0000 7f013fe00640 1 -- 172.21.15.72:0/3709319175 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01481a2840 con 0x7f0148105d80 2024-04-26T21:52:59.306 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:59.305+0000 7f013fe00640 1 --2- 172.21.15.72:0/3709319175 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0148105d80 0x7f014819b170 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f013000c920 tx=0x7f013000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:59.306 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:59.305+0000 7f0144c00640 1 -- 172.21.15.72:0/3709319175 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0130007c70 con 0x7f0148105d80 2024-04-26T21:52:59.306 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:59.305+0000 7f0144c00640 1 -- 172.21.15.72:0/3709319175 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0130007e10 con 0x7f0148105d80 2024-04-26T21:52:59.306 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:59.305+0000 7f014e200640 1 -- 172.21.15.72:0/3709319175 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01481a2b30 con 0x7f0148105d80 2024-04-26T21:52:59.307 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:59.305+0000 7f0144c00640 1 -- 172.21.15.72:0/3709319175 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0130018710 con 0x7f0148105d80 2024-04-26T21:52:59.307 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:59.305+0000 7f014e200640 1 -- 172.21.15.72:0/3709319175 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01481a3080 con 0x7f0148105d80 2024-04-26T21:52:59.307 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:59.306+0000 7f014e200640 1 -- 172.21.15.72:0/3709319175 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f014810beb0 con 0x7f0148105d80 2024-04-26T21:52:59.308 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:59.308+0000 7f0144c00640 1 -- 172.21.15.72:0/3709319175 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f013000f450 con 0x7f0148105d80 2024-04-26T21:52:59.310 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:59.310+0000 7f0144c00640 1 --2- 172.21.15.72:0/3709319175 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f012c0a3bb0 0x7f012c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:59.310 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:59.310+0000 7f0144c00640 1 -- 172.21.15.72:0/3709319175 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7f01300d04f0 con 0x7f0148105d80 2024-04-26T21:52:59.310 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:59.310+0000 7f0147400640 1 --2- 172.21.15.72:0/3709319175 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f012c0a3bb0 0x7f012c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:59.311 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:59.311+0000 7f0147400640 1 --2- 172.21.15.72:0/3709319175 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f012c0a3bb0 0x7f012c0a6070 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f01380058c0 tx=0x7f013800a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:59.317 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:59.314+0000 7f0144c00640 1 -- 172.21.15.72:0/3709319175 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f0130014030 con 0x7f0148105d80 2024-04-26T21:52:59.548 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:59.548+0000 7f014e200640 1 -- 172.21.15.72:0/3709319175 --> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7f0148062d70 con 0x7f012c0a3bb0 2024-04-26T21:52:59.560 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:59.560+0000 7f0144c00640 1 -- 172.21.15.72:0/3709319175 <== mgr.14152 v2:172.21.15.72:6800/588150172 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f0148062d70 con 0x7f012c0a3bb0 2024-04-26T21:52:59.561 INFO:teuthology.orchestra.run.smithi072.stdout:/volumes/qa/sv_0/2b6b095b-1e64-4476-8fef-65001d22a864 2024-04-26T21:52:59.564 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:59.563+0000 7f014e200640 1 -- 172.21.15.72:0/3709319175 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f012c0a3bb0 msgr2=0x7f012c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:59.564 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:59.563+0000 7f014e200640 1 --2- 172.21.15.72:0/3709319175 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f012c0a3bb0 0x7f012c0a6070 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f01380058c0 tx=0x7f013800a2d0 comp rx=0 tx=0).stop 2024-04-26T21:52:59.564 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:59.563+0000 7f014e200640 1 -- 172.21.15.72:0/3709319175 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0148105d80 msgr2=0x7f014819b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:59.564 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:59.563+0000 7f014e200640 1 --2- 172.21.15.72:0/3709319175 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0148105d80 0x7f014819b170 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f013000c920 tx=0x7f013000cde0 comp rx=0 tx=0).stop 2024-04-26T21:52:59.564 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:59.563+0000 7f014e200640 1 -- 172.21.15.72:0/3709319175 shutdown_connections 2024-04-26T21:52:59.564 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:59.563+0000 7f014e200640 1 --2- 172.21.15.72:0/3709319175 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f012c0a3bb0 0x7f012c0a6070 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:59.564 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:59.563+0000 7f014e200640 1 --2- 172.21.15.72:0/3709319175 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0148106740 0x7f01481a21a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:59.565 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:59.563+0000 7f014e200640 1 --2- 172.21.15.72:0/3709319175 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0148105d80 0x7f014819b170 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:59.565 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:59.563+0000 7f014e200640 1 --2- 172.21.15.72:0/3709319175 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0148104b90 0x7f014819ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:59.565 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:59.564+0000 7f014e200640 1 -- 172.21.15.72:0/3709319175 >> 172.21.15.72:0/3709319175 conn(0x7f0148100120 msgr2=0x7f0148109080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:59.565 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:59.564+0000 7f014e200640 1 -- 172.21.15.72:0/3709319175 shutdown_connections 2024-04-26T21:52:59.565 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:59.564+0000 7f014e200640 1 -- 172.21.15.72:0/3709319175 wait complete. 2024-04-26T21:52:59.582 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 --group_name qa 2024-04-26T21:52:59.782 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:59.781+0000 7ffa06e00640 1 -- 172.21.15.72:0/4208182511 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ffa00075310 msgr2=0x7ffa00075770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:59.782 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:59.781+0000 7ffa06e00640 1 --2- 172.21.15.72:0/4208182511 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ffa00075310 0x7ffa00075770 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7ff9f0009900 tx=0x7ff9f002a480 comp rx=0 tx=0).stop 2024-04-26T21:52:59.782 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:59.781+0000 7ffa06e00640 1 -- 172.21.15.72:0/4208182511 shutdown_connections 2024-04-26T21:52:59.782 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:59.781+0000 7ffa06e00640 1 --2- 172.21.15.72:0/4208182511 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffa0010a6f0 0x7ffa0010cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:59.782 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:59.781+0000 7ffa06e00640 1 --2- 172.21.15.72:0/4208182511 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ffa00075310 0x7ffa00075770 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:59.782 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:59.781+0000 7ffa06e00640 1 --2- 172.21.15.72:0/4208182511 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffa000768f0 0x7ffa00074d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:59.782 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:59.781+0000 7ffa06e00640 1 -- 172.21.15.72:0/4208182511 >> 172.21.15.72:0/4208182511 conn(0x7ffa000fddd0 msgr2=0x7ffa001001f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:59.783 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:59.782+0000 7ffa06e00640 1 -- 172.21.15.72:0/4208182511 shutdown_connections 2024-04-26T21:52:59.783 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:59.782+0000 7ffa06e00640 1 -- 172.21.15.72:0/4208182511 wait complete. 2024-04-26T21:52:59.785 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:59.782+0000 7ffa06e00640 1 Processor -- start 2024-04-26T21:52:59.785 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:59.783+0000 7ffa06e00640 1 -- start start 2024-04-26T21:52:59.785 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:59.783+0000 7ffa06e00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffa00075310 0x7ffa0019abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:59.785 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:59.783+0000 7ffa06e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffa000768f0 0x7ffa0019b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:59.785 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:59.783+0000 7ffa06e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ffa0010a6f0 0x7ffa001a2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:59.785 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:59.783+0000 7ffa06e00640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7ffa00110400 con 0x7ffa0010a6f0 2024-04-26T21:52:59.785 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:59.783+0000 7ffa06e00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7ffa00110280 con 0x7ffa00075310 2024-04-26T21:52:59.785 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:59.783+0000 7ffa06e00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ffa00110580 con 0x7ffa000768f0 2024-04-26T21:52:59.785 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:59.783+0000 7ffa06400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ffa0010a6f0 0x7ffa001a2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:59.785 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:59.783+0000 7ffa06400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ffa0010a6f0 0x7ffa001a2170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:35176/0 (socket says 172.21.15.72:35176) 2024-04-26T21:52:59.785 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:59.783+0000 7ffa06400640 1 -- 172.21.15.72:0/2222251257 learned_addr learned my addr 172.21.15.72:0/2222251257 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:52:59.785 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:59.784+0000 7ffa05a00640 1 --2- 172.21.15.72:0/2222251257 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffa00075310 0x7ffa0019abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:59.786 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:59.784+0000 7ffa05000640 1 --2- 172.21.15.72:0/2222251257 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffa000768f0 0x7ffa0019b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:59.786 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:59.784+0000 7ffa06400640 1 -- 172.21.15.72:0/2222251257 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffa000768f0 msgr2=0x7ffa0019b100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:59.786 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:59.784+0000 7ffa06400640 1 --2- 172.21.15.72:0/2222251257 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffa000768f0 0x7ffa0019b100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:59.786 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:59.784+0000 7ffa06400640 1 -- 172.21.15.72:0/2222251257 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffa00075310 msgr2=0x7ffa0019abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:59.786 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:59.784+0000 7ffa06400640 1 --2- 172.21.15.72:0/2222251257 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffa00075310 0x7ffa0019abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:59.786 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:59.784+0000 7ffa06400640 1 -- 172.21.15.72:0/2222251257 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa001a27e0 con 0x7ffa0010a6f0 2024-04-26T21:52:59.786 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:59.784+0000 7ffa05a00640 1 --2- 172.21.15.72:0/2222251257 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffa00075310 0x7ffa0019abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:52:59.786 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:59.784+0000 7ffa06400640 1 --2- 172.21.15.72:0/2222251257 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ffa0010a6f0 0x7ffa001a2170 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7ff9f8009fd0 tx=0x7ff9f800b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:59.786 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:59.784+0000 7ff9fe000640 1 -- 172.21.15.72:0/2222251257 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9f8017070 con 0x7ffa0010a6f0 2024-04-26T21:52:59.786 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:59.784+0000 7ffa06e00640 1 -- 172.21.15.72:0/2222251257 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa001a2ad0 con 0x7ffa0010a6f0 2024-04-26T21:52:59.786 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:59.784+0000 7ff9fe000640 1 -- 172.21.15.72:0/2222251257 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff9f8003e00 con 0x7ffa0010a6f0 2024-04-26T21:52:59.786 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:59.784+0000 7ff9fe000640 1 -- 172.21.15.72:0/2222251257 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9f8010460 con 0x7ffa0010a6f0 2024-04-26T21:52:59.786 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:59.784+0000 7ffa06e00640 1 -- 172.21.15.72:0/2222251257 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa001a2fd0 con 0x7ffa0010a6f0 2024-04-26T21:52:59.787 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:59.785+0000 7ffa06e00640 1 -- 172.21.15.72:0/2222251257 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa001094b0 con 0x7ffa0010a6f0 2024-04-26T21:52:59.787 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:59.787+0000 7ff9fe000640 1 -- 172.21.15.72:0/2222251257 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7ff9f8003970 con 0x7ffa0010a6f0 2024-04-26T21:52:59.789 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:59.789+0000 7ff9fe000640 1 --2- 172.21.15.72:0/2222251257 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7ff9e80a3bb0 0x7ff9e80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:59.789 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:59.789+0000 7ff9fe000640 1 -- 172.21.15.72:0/2222251257 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7ff9f80d0580 con 0x7ffa0010a6f0 2024-04-26T21:52:59.789 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:59.789+0000 7ffa05a00640 1 --2- 172.21.15.72:0/2222251257 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7ff9e80a3bb0 0x7ff9e80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:59.789 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:59.790+0000 7ffa05a00640 1 --2- 172.21.15.72:0/2222251257 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7ff9e80a3bb0 0x7ff9e80a6070 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ff9f4009b40 tx=0x7ff9f4005e50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:59.793 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:52:59.793+0000 7ff9fe000640 1 -- 172.21.15.72:0/2222251257 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7ff9f809d010 con 0x7ffa0010a6f0 2024-04-26T21:53:00.018 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:00.018+0000 7ffa06e00640 1 -- 172.21.15.72:0/2222251257 --> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7ffa00062e00 con 0x7ff9e80a3bb0 2024-04-26T21:53:00.083 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:00.083+0000 7ff9fe000640 1 -- 172.21.15.72:0/2222251257 <== mgr.14152 v2:172.21.15.72:6800/588150172 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7ffa00062e00 con 0x7ff9e80a3bb0 2024-04-26T21:53:00.087 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:00.086+0000 7ffa06e00640 1 -- 172.21.15.72:0/2222251257 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7ff9e80a3bb0 msgr2=0x7ff9e80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:00.087 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:00.086+0000 7ffa06e00640 1 --2- 172.21.15.72:0/2222251257 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7ff9e80a3bb0 0x7ff9e80a6070 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ff9f4009b40 tx=0x7ff9f4005e50 comp rx=0 tx=0).stop 2024-04-26T21:53:00.088 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:00.086+0000 7ffa06e00640 1 -- 172.21.15.72:0/2222251257 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ffa0010a6f0 msgr2=0x7ffa001a2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:00.088 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:00.086+0000 7ffa06e00640 1 --2- 172.21.15.72:0/2222251257 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ffa0010a6f0 0x7ffa001a2170 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7ff9f8009fd0 tx=0x7ff9f800b260 comp rx=0 tx=0).stop 2024-04-26T21:53:00.088 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:00.087+0000 7ffa06e00640 1 -- 172.21.15.72:0/2222251257 shutdown_connections 2024-04-26T21:53:00.088 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:00.087+0000 7ffa06e00640 1 --2- 172.21.15.72:0/2222251257 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ffa0010a6f0 0x7ffa001a2170 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:00.088 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:00.087+0000 7ffa06e00640 1 --2- 172.21.15.72:0/2222251257 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7ff9e80a3bb0 0x7ff9e80a6070 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:00.088 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:00.087+0000 7ffa06e00640 1 --2- 172.21.15.72:0/2222251257 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffa000768f0 0x7ffa0019b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:00.088 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:00.087+0000 7ffa06e00640 1 --2- 172.21.15.72:0/2222251257 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffa00075310 0x7ffa0019abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:00.088 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:00.087+0000 7ffa06e00640 1 -- 172.21.15.72:0/2222251257 >> 172.21.15.72:0/2222251257 conn(0x7ffa000fddd0 msgr2=0x7ffa00106680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:53:00.088 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:00.087+0000 7ffa06e00640 1 -- 172.21.15.72:0/2222251257 shutdown_connections 2024-04-26T21:53:00.088 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:00.087+0000 7ffa06e00640 1 -- 172.21.15.72:0/2222251257 wait complete. 2024-04-26T21:53:00.108 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-04-26T21:53:00.315 INFO:journalctl@ceph.mds.b.smithi076.stdout:Apr 26 21:53:00 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-mds-b[61253]: 2024-04-26T21:53:00.017+0000 7f6585400700 -1 mds.pinger is_rank_lagging: rank=3 was never sent ping request. 2024-04-26T21:53:00.315 INFO:journalctl@ceph.mds.b.smithi076.stdout:Apr 26 21:53:00 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-mds-b[61253]: 2024-04-26T21:53:00.017+0000 7f6585400700 -1 mds.pinger is_rank_lagging: rank=4 was never sent ping request. 2024-04-26T21:53:00.317 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:00.316+0000 7f6952600640 1 -- 172.21.15.72:0/3346545595 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f694c102840 msgr2=0x7f694c102c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:00.317 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:00.316+0000 7f6952600640 1 --2- 172.21.15.72:0/3346545595 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f694c102840 0x7f694c102c60 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f693c009ca0 tx=0x7f693c02a8c0 comp rx=0 tx=0).stop 2024-04-26T21:53:00.317 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:00.316+0000 7f6952600640 1 -- 172.21.15.72:0/3346545595 shutdown_connections 2024-04-26T21:53:00.317 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:00.316+0000 7f6952600640 1 --2- 172.21.15.72:0/3346545595 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f694c1043f0 0x7f694c10ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:00.317 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:00.316+0000 7f6952600640 1 --2- 172.21.15.72:0/3346545595 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f694c103a30 0x7f694c103eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:00.317 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:00.316+0000 7f6952600640 1 --2- 172.21.15.72:0/3346545595 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f694c102840 0x7f694c102c60 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:00.317 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:00.316+0000 7f6952600640 1 -- 172.21.15.72:0/3346545595 >> 172.21.15.72:0/3346545595 conn(0x7f694c0fddd0 msgr2=0x7f694c1001f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:53:00.317 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:00.316+0000 7f6952600640 1 -- 172.21.15.72:0/3346545595 shutdown_connections 2024-04-26T21:53:00.317 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:00.316+0000 7f6952600640 1 -- 172.21.15.72:0/3346545595 wait complete. 2024-04-26T21:53:00.319 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:00.317+0000 7f6952600640 1 Processor -- start 2024-04-26T21:53:00.319 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:00.317+0000 7f6952600640 1 -- start start 2024-04-26T21:53:00.319 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:00.318+0000 7f6952600640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f694c102840 0x7f694c19acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:00.320 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:00.318+0000 7f6952600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f694c103a30 0x7f694c19b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:00.320 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:00.318+0000 7f6952600640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f694c1043f0 0x7f694c1a2250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:00.320 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:00.318+0000 7f6952600640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f694c10e340 con 0x7f694c1043f0 2024-04-26T21:53:00.320 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:00.318+0000 7f6952600640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f694c10e1c0 con 0x7f694c102840 2024-04-26T21:53:00.320 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:00.318+0000 7f6952600640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f694c10e4c0 con 0x7f694c103a30 2024-04-26T21:53:00.320 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:00.318+0000 7f6951c00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f694c1043f0 0x7f694c1a2250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:00.320 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:00.318+0000 7f6951c00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f694c1043f0 0x7f694c1a2250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:35190/0 (socket says 172.21.15.72:35190) 2024-04-26T21:53:00.320 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:00.318+0000 7f6951c00640 1 -- 172.21.15.72:0/2756355850 learned_addr learned my addr 172.21.15.72:0/2756355850 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:53:00.320 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:00.318+0000 7f6951200640 1 --2- 172.21.15.72:0/2756355850 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f694c102840 0x7f694c19acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:00.320 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:00.318+0000 7f6943e00640 1 --2- 172.21.15.72:0/2756355850 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f694c103a30 0x7f694c19b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:00.320 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:00.318+0000 7f6951c00640 1 -- 172.21.15.72:0/2756355850 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f694c103a30 msgr2=0x7f694c19b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:00.320 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:00.318+0000 7f6951c00640 1 --2- 172.21.15.72:0/2756355850 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f694c103a30 0x7f694c19b1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:00.320 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:00.318+0000 7f6951c00640 1 -- 172.21.15.72:0/2756355850 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f694c102840 msgr2=0x7f694c19acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:00.321 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:00.318+0000 7f6951c00640 1 --2- 172.21.15.72:0/2756355850 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f694c102840 0x7f694c19acb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:00.321 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:00.318+0000 7f6951c00640 1 -- 172.21.15.72:0/2756355850 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f694c1a28f0 con 0x7f694c1043f0 2024-04-26T21:53:00.321 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:00.319+0000 7f6951c00640 1 --2- 172.21.15.72:0/2756355850 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f694c1043f0 0x7f694c1a2250 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f6944009fd0 tx=0x7f694400b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:53:00.321 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:00.319+0000 7f694a000640 1 -- 172.21.15.72:0/2756355850 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6944011830 con 0x7f694c1043f0 2024-04-26T21:53:00.321 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:00.319+0000 7f694a000640 1 -- 172.21.15.72:0/2756355850 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6944016070 con 0x7f694c1043f0 2024-04-26T21:53:00.321 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:00.319+0000 7f694a000640 1 -- 172.21.15.72:0/2756355850 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6944010d20 con 0x7f694c1043f0 2024-04-26T21:53:00.321 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:00.319+0000 7f6952600640 1 -- 172.21.15.72:0/2756355850 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f694c1a2be0 con 0x7f694c1043f0 2024-04-26T21:53:00.321 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:00.319+0000 7f6952600640 1 -- 172.21.15.72:0/2756355850 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f694c1a31b0 con 0x7f694c1043f0 2024-04-26T21:53:00.321 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:00.320+0000 7f6952600640 1 -- 172.21.15.72:0/2756355850 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f694c1a33e0 con 0x7f694c1043f0 2024-04-26T21:53:00.322 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:00.322+0000 7f694a000640 1 -- 172.21.15.72:0/2756355850 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f694401b460 con 0x7f694c1043f0 2024-04-26T21:53:00.324 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:00.324+0000 7f694a000640 1 --2- 172.21.15.72:0/2756355850 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f69300a3c30 0x7f69300a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:00.324 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:00.324+0000 7f694a000640 1 -- 172.21.15.72:0/2756355850 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7f69440cf9f0 con 0x7f694c1043f0 2024-04-26T21:53:00.324 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:00.324+0000 7f6951200640 1 --2- 172.21.15.72:0/2756355850 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f69300a3c30 0x7f69300a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:00.325 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:00.325+0000 7f6951200640 1 --2- 172.21.15.72:0/2756355850 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f69300a3c30 0x7f69300a60f0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f693c006010 tx=0x7f693c000f40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:53:00.328 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:00.328+0000 7f694a000640 1 -- 172.21.15.72:0/2756355850 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f694409c480 con 0x7f694c1043f0 2024-04-26T21:53:00.562 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:00.562+0000 7f6952600640 1 -- 172.21.15.72:0/2756355850 --> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7f694c10a9c0 con 0x7f69300a3c30 2024-04-26T21:53:00.573 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:00.573+0000 7f694a000640 1 -- 172.21.15.72:0/2756355850 <== mgr.14152 v2:172.21.15.72:6800/588150172 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f694c10a9c0 con 0x7f69300a3c30 2024-04-26T21:53:00.573 INFO:teuthology.orchestra.run.smithi072.stdout:/volumes/qa/sv_1/bfdb55b2-261f-4f9d-87b1-7e41f6187243 2024-04-26T21:53:00.576 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:00.576+0000 7f6952600640 1 -- 172.21.15.72:0/2756355850 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f69300a3c30 msgr2=0x7f69300a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:00.576 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:00.576+0000 7f6952600640 1 --2- 172.21.15.72:0/2756355850 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f69300a3c30 0x7f69300a60f0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f693c006010 tx=0x7f693c000f40 comp rx=0 tx=0).stop 2024-04-26T21:53:00.576 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:00.576+0000 7f6952600640 1 -- 172.21.15.72:0/2756355850 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f694c1043f0 msgr2=0x7f694c1a2250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:00.576 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:00.576+0000 7f6952600640 1 --2- 172.21.15.72:0/2756355850 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f694c1043f0 0x7f694c1a2250 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f6944009fd0 tx=0x7f694400b8e0 comp rx=0 tx=0).stop 2024-04-26T21:53:00.577 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:00.576+0000 7f6952600640 1 -- 172.21.15.72:0/2756355850 shutdown_connections 2024-04-26T21:53:00.577 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:00.576+0000 7f6952600640 1 --2- 172.21.15.72:0/2756355850 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f694c1043f0 0x7f694c1a2250 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:00.577 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:00.576+0000 7f6952600640 1 --2- 172.21.15.72:0/2756355850 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f69300a3c30 0x7f69300a60f0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:00.577 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:00.576+0000 7f6952600640 1 --2- 172.21.15.72:0/2756355850 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f694c103a30 0x7f694c19b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:00.577 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:00.576+0000 7f6952600640 1 --2- 172.21.15.72:0/2756355850 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f694c102840 0x7f694c19acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:00.577 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:00.576+0000 7f6952600640 1 -- 172.21.15.72:0/2756355850 >> 172.21.15.72:0/2756355850 conn(0x7f694c0fddd0 msgr2=0x7f694c1092a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:53:00.577 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:00.577+0000 7f6952600640 1 -- 172.21.15.72:0/2756355850 shutdown_connections 2024-04-26T21:53:00.577 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:00.577+0000 7f6952600640 1 -- 172.21.15.72:0/2756355850 wait complete. 2024-04-26T21:53:00.596 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:53:00.801 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:00.800+0000 7f7c21600640 1 -- 172.21.15.72:0/4000664128 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7c1c105b20 msgr2=0x7f7c1c107f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:00.801 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:00.800+0000 7f7c21600640 1 --2- 172.21.15.72:0/4000664128 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7c1c105b20 0x7f7c1c107f10 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f7c04009900 tx=0x7f7c0402a3f0 comp rx=0 tx=0).stop 2024-04-26T21:53:00.801 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:00.800+0000 7f7c21600640 1 -- 172.21.15.72:0/4000664128 shutdown_connections 2024-04-26T21:53:00.801 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:00.800+0000 7f7c21600640 1 --2- 172.21.15.72:0/4000664128 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7c1c108450 0x7f7c1c10a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:00.801 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:00.800+0000 7f7c21600640 1 --2- 172.21.15.72:0/4000664128 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7c1c105b20 0x7f7c1c107f10 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:00.801 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:00.800+0000 7f7c21600640 1 --2- 172.21.15.72:0/4000664128 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7c1c1031f0 0x7f7c1c1055e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:00.801 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:00.800+0000 7f7c21600640 1 -- 172.21.15.72:0/4000664128 >> 172.21.15.72:0/4000664128 conn(0x7f7c1c0fcbc0 msgr2=0x7f7c1c0ff020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:53:00.802 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:00.800+0000 7f7c21600640 1 -- 172.21.15.72:0/4000664128 shutdown_connections 2024-04-26T21:53:00.802 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:00.800+0000 7f7c21600640 1 -- 172.21.15.72:0/4000664128 wait complete. 2024-04-26T21:53:00.802 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:00.801+0000 7f7c21600640 1 Processor -- start 2024-04-26T21:53:00.802 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:00.801+0000 7f7c21600640 1 -- start start 2024-04-26T21:53:00.804 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:00.801+0000 7f7c21600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7c1c1031f0 0x7f7c1c19aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:00.804 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:00.801+0000 7f7c21600640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7c1c105b20 0x7f7c1c19afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:00.804 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:00.801+0000 7f7c21600640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7c1c108450 0x7f7c1c1a2010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:00.804 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:00.801+0000 7f7c21600640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f7c1c10e150 con 0x7f7c1c105b20 2024-04-26T21:53:00.804 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:00.801+0000 7f7c21600640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f7c1c10dfd0 con 0x7f7c1c108450 2024-04-26T21:53:00.805 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:00.801+0000 7f7c21600640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f7c1c10e2d0 con 0x7f7c1c1031f0 2024-04-26T21:53:00.805 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:00.802+0000 7f7c13400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7c1c105b20 0x7f7c1c19afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:00.805 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:00.802+0000 7f7c13400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7c1c105b20 0x7f7c1c19afa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:55960/0 (socket says 172.21.15.72:55960) 2024-04-26T21:53:00.805 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:00.802+0000 7f7c13400640 1 -- 172.21.15.72:0/4123900921 learned_addr learned my addr 172.21.15.72:0/4123900921 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:53:00.805 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:00.802+0000 7f7c20c00640 1 --2- 172.21.15.72:0/4123900921 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7c1c108450 0x7f7c1c1a2010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:00.805 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:00.802+0000 7f7c13400640 1 -- 172.21.15.72:0/4123900921 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7c1c1031f0 msgr2=0x7f7c1c19aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:00.806 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:00.802+0000 7f7c1be00640 1 --2- 172.21.15.72:0/4123900921 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7c1c1031f0 0x7f7c1c19aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:00.806 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:00.802+0000 7f7c13400640 1 --2- 172.21.15.72:0/4123900921 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7c1c1031f0 0x7f7c1c19aa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:00.806 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:00.802+0000 7f7c13400640 1 -- 172.21.15.72:0/4123900921 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7c1c108450 msgr2=0x7f7c1c1a2010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:00.806 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:00.802+0000 7f7c13400640 1 --2- 172.21.15.72:0/4123900921 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7c1c108450 0x7f7c1c1a2010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:00.806 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:00.802+0000 7f7c13400640 1 -- 172.21.15.72:0/4123900921 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c1c1a2680 con 0x7f7c1c105b20 2024-04-26T21:53:00.806 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:00.802+0000 7f7c20c00640 1 --2- 172.21.15.72:0/4123900921 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7c1c108450 0x7f7c1c1a2010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:53:00.807 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:00.802+0000 7f7c13400640 1 --2- 172.21.15.72:0/4123900921 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7c1c105b20 0x7f7c1c19afa0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f7c04005ab0 tx=0x7f7c040042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:53:00.807 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:00.803+0000 7f7c19600640 1 -- 172.21.15.72:0/4123900921 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c0403f030 con 0x7f7c1c105b20 2024-04-26T21:53:00.807 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:00.803+0000 7f7c19600640 1 -- 172.21.15.72:0/4123900921 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7c04003c90 con 0x7f7c1c105b20 2024-04-26T21:53:00.807 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:00.803+0000 7f7c19600640 1 -- 172.21.15.72:0/4123900921 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c04035070 con 0x7f7c1c105b20 2024-04-26T21:53:00.807 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:00.803+0000 7f7c21600640 1 -- 172.21.15.72:0/4123900921 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c1c1a2910 con 0x7f7c1c105b20 2024-04-26T21:53:00.807 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:00.803+0000 7f7c21600640 1 -- 172.21.15.72:0/4123900921 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c1c1a2db0 con 0x7f7c1c105b20 2024-04-26T21:53:00.807 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:00.804+0000 7f7c21600640 1 -- 172.21.15.72:0/4123900921 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c1c109fa0 con 0x7f7c1c105b20 2024-04-26T21:53:00.808 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:00.806+0000 7f7c19600640 1 -- 172.21.15.72:0/4123900921 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f7c04042030 con 0x7f7c1c105b20 2024-04-26T21:53:00.808 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:00.808+0000 7f7c19600640 1 --2- 172.21.15.72:0/4123900921 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f7c000a3c30 0x7f7c000a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:00.808 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:00.808+0000 7f7c19600640 1 -- 172.21.15.72:0/4123900921 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7f7c040f0d20 con 0x7f7c1c105b20 2024-04-26T21:53:00.809 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:00.808+0000 7f7c1be00640 1 --2- 172.21.15.72:0/4123900921 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f7c000a3c30 0x7f7c000a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:00.809 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:00.809+0000 7f7c1be00640 1 --2- 172.21.15.72:0/4123900921 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f7c000a3c30 0x7f7c000a60f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f7c0c008d20 tx=0x7f7c0c0082b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:53:00.812 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:00.812+0000 7f7c19600640 1 -- 172.21.15.72:0/4123900921 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f7c040bd7b0 con 0x7f7c1c105b20 2024-04-26T21:53:00.872 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:00.871+0000 7f7c19600640 1 -- 172.21.15.72:0/4123900921 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f7c040b9720 con 0x7f7c1c105b20 2024-04-26T21:53:01.064 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:01.064+0000 7f7c21600640 1 -- 172.21.15.72:0/4123900921 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7c1c02dbf0 con 0x7f7c1c105b20 2024-04-26T21:53:01.066 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:01.066+0000 7f7c19600640 1 -- 172.21.15.72:0/4123900921 <== mon.0 v2:172.21.15.72:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11931 (secure 0 0 0) 0x7f7c040e5c90 con 0x7f7c1c105b20 2024-04-26T21:53:01.066 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T21:53:01.067 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6835/1320381959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6834","nonce":1320381959},{"type":"v1","addr":"172.21.15.72:6835","nonce":1320381959}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6837/2142200512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6836","nonce":2142200512},{"type":"v1","addr":"172.21.15.72:6837","nonce":2142200512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T21:52:50.567857+0000","modified":"2024-04-26T21:52:58.508126+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34517,"mds_1":34472,"mds_2":34442,"mds_3":34406,"mds_4":24463},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14652":{"gid":14652,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.72:6839/1312898281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6838","nonce":1312898281},{"type":"v1","addr":"172.21.15.72:6839","nonce":1312898281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14664":{"gid":14664,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.72:6841/3257692339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6840","nonce":3257692339},{"type":"v1","addr":"172.21.15.72:6841","nonce":3257692339}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24388":{"gid":24388,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6833/4236668465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":4236668465},{"type":"v1","addr":"172.21.15.164:6833","nonce":4236668465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6835/2391114169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":2391114169},{"type":"v1","addr":"172.21.15.164:6835","nonce":2391114169}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6837/3585829992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":3585829992},{"type":"v1","addr":"172.21.15.164:6837","nonce":3585829992}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.164:6839/1636226820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":1636226820},{"type":"v1","addr":"172.21.15.164:6839","nonce":1636226820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34406":{"gid":34406,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":15,"addr":"172.21.15.76:6833/3869483750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":3869483750},{"type":"v1","addr":"172.21.15.76:6833","nonce":3869483750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34442":{"gid":34442,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.76:6835/3517429435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":3517429435},{"type":"v1","addr":"172.21.15.76:6835","nonce":3517429435}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34472":{"gid":34472,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.76:6837/1150678991","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":1150678991},{"type":"v1","addr":"172.21.15.76:6837","nonce":1150678991}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34517":{"gid":34517,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.76:6839/327145636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6838","nonce":327145636},{"type":"v1","addr":"172.21.15.76:6839","nonce":327145636}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34517,"qdb_cluster":[34406,24463,34442,34517,34472]},"id":1}]} 2024-04-26T21:53:01.067 INFO:teuthology.orchestra.run.smithi072.stderr:dumped fsmap epoch 26 2024-04-26T21:53:01.069 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:01.068+0000 7f7c21600640 1 -- 172.21.15.72:0/4123900921 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f7c000a3c30 msgr2=0x7f7c000a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:01.069 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:01.068+0000 7f7c21600640 1 --2- 172.21.15.72:0/4123900921 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f7c000a3c30 0x7f7c000a60f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f7c0c008d20 tx=0x7f7c0c0082b0 comp rx=0 tx=0).stop 2024-04-26T21:53:01.069 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:01.068+0000 7f7c21600640 1 -- 172.21.15.72:0/4123900921 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7c1c105b20 msgr2=0x7f7c1c19afa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:01.069 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:01.068+0000 7f7c21600640 1 --2- 172.21.15.72:0/4123900921 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7c1c105b20 0x7f7c1c19afa0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f7c04005ab0 tx=0x7f7c040042f0 comp rx=0 tx=0).stop 2024-04-26T21:53:01.069 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:01.069+0000 7f7c21600640 1 -- 172.21.15.72:0/4123900921 shutdown_connections 2024-04-26T21:53:01.069 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:01.069+0000 7f7c21600640 1 --2- 172.21.15.72:0/4123900921 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7c1c108450 0x7f7c1c1a2010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:01.069 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:01.069+0000 7f7c21600640 1 --2- 172.21.15.72:0/4123900921 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7c1c105b20 0x7f7c1c19afa0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:01.070 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:01.069+0000 7f7c21600640 1 --2- 172.21.15.72:0/4123900921 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f7c000a3c30 0x7f7c000a60f0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:01.070 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:01.069+0000 7f7c21600640 1 --2- 172.21.15.72:0/4123900921 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7c1c1031f0 0x7f7c1c19aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:01.070 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:01.069+0000 7f7c21600640 1 -- 172.21.15.72:0/4123900921 >> 172.21.15.72:0/4123900921 conn(0x7f7c1c0fcbc0 msgr2=0x7f7c1c102960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:53:01.070 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:01.069+0000 7f7c21600640 1 -- 172.21.15.72:0/4123900921 shutdown_connections 2024-04-26T21:53:01.070 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:01.069+0000 7f7c21600640 1 -- 172.21.15.72:0/4123900921 wait complete. 2024-04-26T21:53:01.071 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:53:00 smithi072 ceph-mon[28138]: from='client.14778 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:53:01.071 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:53:00 smithi072 ceph-mon[28138]: from='client.14784 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:53:01.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:53:00 smithi072 ceph-mon[28138]: pgmap v372: 97 pgs: 97 active+clean; 590 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 7.5 KiB/s wr, 24 op/s 2024-04-26T21:53:01.088 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:53:01.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:53:00 smithi164 ceph-mon[31916]: from='client.14778 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:53:01.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:53:00 smithi164 ceph-mon[31916]: from='client.14784 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:53:01.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:53:00 smithi164 ceph-mon[31916]: pgmap v372: 97 pgs: 97 active+clean; 590 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 7.5 KiB/s wr, 24 op/s 2024-04-26T21:53:01.271 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:01.270+0000 7ff17be00640 1 -- 172.21.15.72:0/3027906538 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff17c102860 msgr2=0x7ff17c102ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:01.271 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:01.270+0000 7ff17be00640 1 --2- 172.21.15.72:0/3027906538 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff17c102860 0x7ff17c102ce0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7ff164009900 tx=0x7ff16402a3f0 comp rx=0 tx=0).stop 2024-04-26T21:53:01.271 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:01.270+0000 7ff17be00640 1 -- 172.21.15.72:0/3027906538 shutdown_connections 2024-04-26T21:53:01.271 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:01.270+0000 7ff17be00640 1 --2- 172.21.15.72:0/3027906538 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff17c103220 0x7ff17c109ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:01.271 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:01.270+0000 7ff17be00640 1 --2- 172.21.15.72:0/3027906538 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff17c102860 0x7ff17c102ce0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:01.271 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:01.270+0000 7ff17be00640 1 --2- 172.21.15.72:0/3027906538 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff17c101670 0x7ff17c101a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:01.271 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:01.270+0000 7ff17be00640 1 -- 172.21.15.72:0/3027906538 >> 172.21.15.72:0/3027906538 conn(0x7ff17c0fcbc0 msgr2=0x7ff17c0ff020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:53:01.271 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:01.270+0000 7ff17be00640 1 -- 172.21.15.72:0/3027906538 shutdown_connections 2024-04-26T21:53:01.271 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:01.270+0000 7ff17be00640 1 -- 172.21.15.72:0/3027906538 wait complete. 2024-04-26T21:53:01.274 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:01.271+0000 7ff17be00640 1 Processor -- start 2024-04-26T21:53:01.274 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:01.271+0000 7ff17be00640 1 -- start start 2024-04-26T21:53:01.275 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:01.271+0000 7ff17be00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff17c101670 0x7ff17c19aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:01.275 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:01.271+0000 7ff17be00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff17c102860 0x7ff17c19afb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:01.275 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:01.272+0000 7ff17be00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff17c103220 0x7ff17c1a2020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:01.275 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:01.272+0000 7ff17be00640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7ff17c10e1d0 con 0x7ff17c102860 2024-04-26T21:53:01.275 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:01.272+0000 7ff17be00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7ff17c10e050 con 0x7ff17c101670 2024-04-26T21:53:01.276 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:01.272+0000 7ff17be00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ff17c10e350 con 0x7ff17c103220 2024-04-26T21:53:01.276 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:01.272+0000 7ff173e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff17c102860 0x7ff17c19afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:01.276 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:01.272+0000 7ff173e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff17c102860 0x7ff17c19afb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:55986/0 (socket says 172.21.15.72:55986) 2024-04-26T21:53:01.276 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:01.272+0000 7ff173e00640 1 -- 172.21.15.72:0/1424070977 learned_addr learned my addr 172.21.15.72:0/1424070977 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:53:01.276 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:01.272+0000 7ff17aa00640 1 --2- 172.21.15.72:0/1424070977 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff17c101670 0x7ff17c19aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:01.276 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:01.272+0000 7ff17b400640 1 --2- 172.21.15.72:0/1424070977 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff17c103220 0x7ff17c1a2020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:01.276 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:01.272+0000 7ff173e00640 1 -- 172.21.15.72:0/1424070977 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff17c103220 msgr2=0x7ff17c1a2020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:01.277 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:01.272+0000 7ff173e00640 1 --2- 172.21.15.72:0/1424070977 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff17c103220 0x7ff17c1a2020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:01.277 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:01.272+0000 7ff173e00640 1 -- 172.21.15.72:0/1424070977 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff17c101670 msgr2=0x7ff17c19aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:01.277 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:01.272+0000 7ff173e00640 1 --2- 172.21.15.72:0/1424070977 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff17c101670 0x7ff17c19aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:01.277 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:01.272+0000 7ff173e00640 1 -- 172.21.15.72:0/1424070977 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff17c1a2690 con 0x7ff17c102860 2024-04-26T21:53:01.277 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:01.272+0000 7ff17aa00640 1 --2- 172.21.15.72:0/1424070977 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff17c101670 0x7ff17c19aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:53:01.277 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:01.273+0000 7ff173e00640 1 --2- 172.21.15.72:0/1424070977 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff17c102860 0x7ff17c19afb0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7ff164031460 tx=0x7ff164003d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:53:01.277 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:01.273+0000 7ff173400640 1 -- 172.21.15.72:0/1424070977 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff16403f070 con 0x7ff17c102860 2024-04-26T21:53:01.277 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:01.273+0000 7ff173400640 1 -- 172.21.15.72:0/1424070977 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff164031da0 con 0x7ff17c102860 2024-04-26T21:53:01.277 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:01.273+0000 7ff173400640 1 -- 172.21.15.72:0/1424070977 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff164035030 con 0x7ff17c102860 2024-04-26T21:53:01.277 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:01.273+0000 7ff17be00640 1 -- 172.21.15.72:0/1424070977 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff17c1a2920 con 0x7ff17c102860 2024-04-26T21:53:01.277 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:01.273+0000 7ff17be00640 1 -- 172.21.15.72:0/1424070977 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff17c1a2dc0 con 0x7ff17c102860 2024-04-26T21:53:01.277 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:01.274+0000 7ff17be00640 1 -- 172.21.15.72:0/1424070977 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff17c1a2fc0 con 0x7ff17c102860 2024-04-26T21:53:01.278 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:01.276+0000 7ff173400640 1 -- 172.21.15.72:0/1424070977 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7ff164042070 con 0x7ff17c102860 2024-04-26T21:53:01.278 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:01.278+0000 7ff173400640 1 --2- 172.21.15.72:0/1424070977 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7ff1600a3cc0 0x7ff1600a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:01.278 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:01.278+0000 7ff173400640 1 -- 172.21.15.72:0/1424070977 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7ff1640f1530 con 0x7ff17c102860 2024-04-26T21:53:01.279 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:01.278+0000 7ff17aa00640 1 --2- 172.21.15.72:0/1424070977 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7ff1600a3cc0 0x7ff1600a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:01.279 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:01.278+0000 7ff17aa00640 1 --2- 172.21.15.72:0/1424070977 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7ff1600a3cc0 0x7ff1600a6180 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7ff16c0058c0 tx=0x7ff16c005d50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:53:01.285 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:01.285+0000 7ff173400640 1 -- 172.21.15.72:0/1424070977 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7ff1640bde60 con 0x7ff17c102860 2024-04-26T21:53:01.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:53:00 smithi076 ceph-mon[31823]: from='client.14778 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:53:01.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:53:00 smithi076 ceph-mon[31823]: from='client.14784 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:53:01.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:53:00 smithi076 ceph-mon[31823]: pgmap v372: 97 pgs: 97 active+clean; 590 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 7.5 KiB/s wr, 24 op/s 2024-04-26T21:53:01.488 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:01.488+0000 7ff17be00640 1 -- 172.21.15.72:0/1424070977 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff17c1a3300 con 0x7ff17c102860 2024-04-26T21:53:01.492 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:01.491+0000 7ff173400640 1 -- 172.21.15.72:0/1424070977 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+22172 (secure 0 0 0) 0x7ff1640bd760 con 0x7ff17c102860 2024-04-26T21:53:01.493 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T21:53:01.493 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":87,"fsid":"a1780950-0415-11ef-bc93-c7b262605968","created":"2024-04-26T21:42:01.504173+0000","modified":"2024-04-26T21:52:51.746452+0000","last_up_change":"2024-04-26T21:51:43.482437+0000","last_in_change":"2024-04-26T21:51:18.468587+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:46:29.785427+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:52:48.320391+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T21:52:49.517493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"04e06884-b355-4762-9b29-9034b7201e9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6802","nonce":1416765179},{"type":"v1","addr":"172.21.15.72:6803","nonce":1416765179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6804","nonce":1416765179},{"type":"v1","addr":"172.21.15.72:6805","nonce":1416765179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6808","nonce":1416765179},{"type":"v1","addr":"172.21.15.72:6809","nonce":1416765179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6806","nonce":1416765179},{"type":"v1","addr":"172.21.15.72:6807","nonce":1416765179}]},"public_addr":"172.21.15.72:6803/1416765179","cluster_addr":"172.21.15.72:6805/1416765179","heartbeat_back_addr":"172.21.15.72:6809/1416765179","heartbeat_front_addr":"172.21.15.72:6807/1416765179","state":["exists","up"]},{"osd":1,"uuid":"d75a479a-4a2e-48e9-a01e-2b1c9617416a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":755426949},{"type":"v1","addr":"172.21.15.76:6801","nonce":755426949}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":755426949},{"type":"v1","addr":"172.21.15.76:6803","nonce":755426949}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":755426949},{"type":"v1","addr":"172.21.15.76:6807","nonce":755426949}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":755426949},{"type":"v1","addr":"172.21.15.76:6805","nonce":755426949}]},"public_addr":"172.21.15.76:6801/755426949","cluster_addr":"172.21.15.76:6803/755426949","heartbeat_back_addr":"172.21.15.76:6807/755426949","heartbeat_front_addr":"172.21.15.76:6805/755426949","state":["exists","up"]},{"osd":2,"uuid":"b6f8da58-4428-4e8c-a944-7ea5645d6d92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":852265973},{"type":"v1","addr":"172.21.15.164:6801","nonce":852265973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":852265973},{"type":"v1","addr":"172.21.15.164:6803","nonce":852265973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":852265973},{"type":"v1","addr":"172.21.15.164:6807","nonce":852265973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":852265973},{"type":"v1","addr":"172.21.15.164:6805","nonce":852265973}]},"public_addr":"172.21.15.164:6801/852265973","cluster_addr":"172.21.15.164:6803/852265973","heartbeat_back_addr":"172.21.15.164:6807/852265973","heartbeat_front_addr":"172.21.15.164:6805/852265973","state":["exists","up"]},{"osd":3,"uuid":"76d83409-649e-41e5-b4f4-43bbd5b00ea5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6810","nonce":4027763789},{"type":"v1","addr":"172.21.15.72:6811","nonce":4027763789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6812","nonce":4027763789},{"type":"v1","addr":"172.21.15.72:6813","nonce":4027763789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6816","nonce":4027763789},{"type":"v1","addr":"172.21.15.72:6817","nonce":4027763789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6814","nonce":4027763789},{"type":"v1","addr":"172.21.15.72:6815","nonce":4027763789}]},"public_addr":"172.21.15.72:6811/4027763789","cluster_addr":"172.21.15.72:6813/4027763789","heartbeat_back_addr":"172.21.15.72:6817/4027763789","heartbeat_front_addr":"172.21.15.72:6815/4027763789","state":["exists","up"]},{"osd":4,"uuid":"d2cfeea7-1661-4156-bdcd-f770a8d1a878","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":3718867827},{"type":"v1","addr":"172.21.15.76:6809","nonce":3718867827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":3718867827},{"type":"v1","addr":"172.21.15.76:6811","nonce":3718867827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":3718867827},{"type":"v1","addr":"172.21.15.76:6815","nonce":3718867827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":3718867827},{"type":"v1","addr":"172.21.15.76:6813","nonce":3718867827}]},"public_addr":"172.21.15.76:6809/3718867827","cluster_addr":"172.21.15.76:6811/3718867827","heartbeat_back_addr":"172.21.15.76:6815/3718867827","heartbeat_front_addr":"172.21.15.76:6813/3718867827","state":["exists","up"]},{"osd":5,"uuid":"f3896ad6-a040-4ea3-a431-ff9b7fc05c36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":2543954772},{"type":"v1","addr":"172.21.15.164:6809","nonce":2543954772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":2543954772},{"type":"v1","addr":"172.21.15.164:6811","nonce":2543954772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":2543954772},{"type":"v1","addr":"172.21.15.164:6815","nonce":2543954772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":2543954772},{"type":"v1","addr":"172.21.15.164:6813","nonce":2543954772}]},"public_addr":"172.21.15.164:6809/2543954772","cluster_addr":"172.21.15.164:6811/2543954772","heartbeat_back_addr":"172.21.15.164:6815/2543954772","heartbeat_front_addr":"172.21.15.164:6813/2543954772","state":["exists","up"]},{"osd":6,"uuid":"d5d56650-be7e-4ba5-ba3a-d7d473294dab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6818","nonce":2080659027},{"type":"v1","addr":"172.21.15.72:6819","nonce":2080659027}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6820","nonce":2080659027},{"type":"v1","addr":"172.21.15.72:6821","nonce":2080659027}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6824","nonce":2080659027},{"type":"v1","addr":"172.21.15.72:6825","nonce":2080659027}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6822","nonce":2080659027},{"type":"v1","addr":"172.21.15.72:6823","nonce":2080659027}]},"public_addr":"172.21.15.72:6819/2080659027","cluster_addr":"172.21.15.72:6821/2080659027","heartbeat_back_addr":"172.21.15.72:6825/2080659027","heartbeat_front_addr":"172.21.15.72:6823/2080659027","state":["exists","up"]},{"osd":7,"uuid":"18975b5a-9309-4a0f-9f37-3ce09c2e745d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":3144631323},{"type":"v1","addr":"172.21.15.76:6817","nonce":3144631323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":3144631323},{"type":"v1","addr":"172.21.15.76:6819","nonce":3144631323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":3144631323},{"type":"v1","addr":"172.21.15.76:6823","nonce":3144631323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":3144631323},{"type":"v1","addr":"172.21.15.76:6821","nonce":3144631323}]},"public_addr":"172.21.15.76:6817/3144631323","cluster_addr":"172.21.15.76:6819/3144631323","heartbeat_back_addr":"172.21.15.76:6823/3144631323","heartbeat_front_addr":"172.21.15.76:6821/3144631323","state":["exists","up"]},{"osd":8,"uuid":"779e9c1e-8e97-48d4-a854-68ca13f38e0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":627888883},{"type":"v1","addr":"172.21.15.164:6817","nonce":627888883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":627888883},{"type":"v1","addr":"172.21.15.164:6819","nonce":627888883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":627888883},{"type":"v1","addr":"172.21.15.164:6823","nonce":627888883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":627888883},{"type":"v1","addr":"172.21.15.164:6821","nonce":627888883}]},"public_addr":"172.21.15.164:6817/627888883","cluster_addr":"172.21.15.164:6819/627888883","heartbeat_back_addr":"172.21.15.164:6823/627888883","heartbeat_front_addr":"172.21.15.164:6821/627888883","state":["exists","up"]},{"osd":9,"uuid":"d0317850-7b2d-4850-b2cf-0b919505c1fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6826","nonce":1273118485},{"type":"v1","addr":"172.21.15.72:6827","nonce":1273118485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6828","nonce":1273118485},{"type":"v1","addr":"172.21.15.72:6829","nonce":1273118485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6832","nonce":1273118485},{"type":"v1","addr":"172.21.15.72:6833","nonce":1273118485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6830","nonce":1273118485},{"type":"v1","addr":"172.21.15.72:6831","nonce":1273118485}]},"public_addr":"172.21.15.72:6827/1273118485","cluster_addr":"172.21.15.72:6829/1273118485","heartbeat_back_addr":"172.21.15.72:6833/1273118485","heartbeat_front_addr":"172.21.15.72:6831/1273118485","state":["exists","up"]},{"osd":10,"uuid":"e0aa38d8-6011-4d2a-b67b-5bff6d999e47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":2808954459},{"type":"v1","addr":"172.21.15.76:6825","nonce":2808954459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":2808954459},{"type":"v1","addr":"172.21.15.76:6827","nonce":2808954459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":2808954459},{"type":"v1","addr":"172.21.15.76:6831","nonce":2808954459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":2808954459},{"type":"v1","addr":"172.21.15.76:6829","nonce":2808954459}]},"public_addr":"172.21.15.76:6825/2808954459","cluster_addr":"172.21.15.76:6827/2808954459","heartbeat_back_addr":"172.21.15.76:6831/2808954459","heartbeat_front_addr":"172.21.15.76:6829/2808954459","state":["exists","up"]},{"osd":11,"uuid":"981b3d4d-916e-4937-9916-7cc21a891e43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":2354951784},{"type":"v1","addr":"172.21.15.164:6825","nonce":2354951784}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":2354951784},{"type":"v1","addr":"172.21.15.164:6827","nonce":2354951784}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":2354951784},{"type":"v1","addr":"172.21.15.164:6831","nonce":2354951784}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":2354951784},{"type":"v1","addr":"172.21.15.164:6829","nonce":2354951784}]},"public_addr":"172.21.15.164:6825/2354951784","cluster_addr":"172.21.15.164:6827/2354951784","heartbeat_back_addr":"172.21.15.164:6831/2354951784","heartbeat_front_addr":"172.21.15.164:6829/2354951784","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:18.997924+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:52.439081+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:46:25.893288+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:47:00.112807+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:47:35.116874+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:48:09.216344+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:48:45.028853+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:49:19.905397+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:49:55.030536+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:50:30.767662+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:51:06.423267+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:51:41.631211+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.72:0/416277784":"2024-04-27T21:43:09.602791+0000","172.21.15.72:6800/4010358672":"2024-04-27T21:43:09.602791+0000","172.21.15.72:6801/1012617371":"2024-04-27T21:42:34.684837+0000","172.21.15.72:6801/4010358672":"2024-04-27T21:43:09.602791+0000","172.21.15.72:0/3555448179":"2024-04-27T21:43:09.602791+0000","172.21.15.72:6800/1012617371":"2024-04-27T21:42:34.684837+0000","172.21.15.72:0/2920378761":"2024-04-27T21:42:34.684837+0000","172.21.15.72:0/1933740041":"2024-04-27T21:43:09.602791+0000","172.21.15.72:0/3225295775":"2024-04-27T21:42:34.684837+0000","172.21.15.72:0/2326016180":"2024-04-27T21:42:34.684837+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:53:01.494 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:01.494+0000 7ff17be00640 1 -- 172.21.15.72:0/1424070977 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7ff1600a3cc0 msgr2=0x7ff1600a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:01.494 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:01.494+0000 7ff17be00640 1 --2- 172.21.15.72:0/1424070977 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7ff1600a3cc0 0x7ff1600a6180 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7ff16c0058c0 tx=0x7ff16c005d50 comp rx=0 tx=0).stop 2024-04-26T21:53:01.495 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:01.494+0000 7ff17be00640 1 -- 172.21.15.72:0/1424070977 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff17c102860 msgr2=0x7ff17c19afb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:01.495 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:01.494+0000 7ff17be00640 1 --2- 172.21.15.72:0/1424070977 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff17c102860 0x7ff17c19afb0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7ff164031460 tx=0x7ff164003d50 comp rx=0 tx=0).stop 2024-04-26T21:53:01.495 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:01.494+0000 7ff17be00640 1 -- 172.21.15.72:0/1424070977 shutdown_connections 2024-04-26T21:53:01.495 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:01.494+0000 7ff17be00640 1 --2- 172.21.15.72:0/1424070977 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7ff1600a3cc0 0x7ff1600a6180 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:01.495 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:01.494+0000 7ff17be00640 1 --2- 172.21.15.72:0/1424070977 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff17c103220 0x7ff17c1a2020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:01.495 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:01.494+0000 7ff17be00640 1 --2- 172.21.15.72:0/1424070977 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff17c102860 0x7ff17c19afb0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:01.495 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:01.494+0000 7ff17be00640 1 --2- 172.21.15.72:0/1424070977 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff17c101670 0x7ff17c19aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:01.495 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:01.494+0000 7ff17be00640 1 -- 172.21.15.72:0/1424070977 >> 172.21.15.72:0/1424070977 conn(0x7ff17c0fcbc0 msgr2=0x7ff17c108000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:53:01.495 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:01.494+0000 7ff17be00640 1 -- 172.21.15.72:0/1424070977 shutdown_connections 2024-04-26T21:53:01.495 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:01.494+0000 7ff17be00640 1 -- 172.21.15.72:0/1424070977 wait complete. 2024-04-26T21:53:01.514 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-26T21:53:01.514 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T21:53:01.689 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:01.688+0000 7fe5b9e00640 1 -- 172.21.15.72:0/4139138119 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe5b410a820 msgr2=0x7fe5b410ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:01.689 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:01.688+0000 7fe5b9e00640 1 --2- 172.21.15.72:0/4139138119 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe5b410a820 0x7fe5b410ccb0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fe5ac009ca0 tx=0x7fe5ac02a950 comp rx=0 tx=0).stop 2024-04-26T21:53:01.689 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:01.688+0000 7fe5b9e00640 1 -- 172.21.15.72:0/4139138119 shutdown_connections 2024-04-26T21:53:01.689 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:01.688+0000 7fe5b9e00640 1 --2- 172.21.15.72:0/4139138119 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe5b410a820 0x7fe5b410ccb0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:01.689 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:01.688+0000 7fe5b9e00640 1 --2- 172.21.15.72:0/4139138119 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe5b4075310 0x7fe5b4075770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:01.689 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:01.688+0000 7fe5b9e00640 1 --2- 172.21.15.72:0/4139138119 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe5b40768f0 0x7fe5b4074d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:01.689 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:01.688+0000 7fe5b9e00640 1 -- 172.21.15.72:0/4139138119 >> 172.21.15.72:0/4139138119 conn(0x7fe5b40fdee0 msgr2=0x7fe5b4100320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:53:01.689 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:01.688+0000 7fe5b9e00640 1 -- 172.21.15.72:0/4139138119 shutdown_connections 2024-04-26T21:53:01.689 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:01.688+0000 7fe5b9e00640 1 -- 172.21.15.72:0/4139138119 wait complete. 2024-04-26T21:53:01.691 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:01.689+0000 7fe5b9e00640 1 Processor -- start 2024-04-26T21:53:01.691 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:01.689+0000 7fe5b9e00640 1 -- start start 2024-04-26T21:53:01.691 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:01.689+0000 7fe5b9e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe5b4075310 0x7fe5b419ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:01.691 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:01.689+0000 7fe5b9e00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe5b40768f0 0x7fe5b419b160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:01.691 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:01.689+0000 7fe5b9e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe5b410a820 0x7fe5b41a2190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:01.691 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:01.689+0000 7fe5b9e00640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fe5b4110590 con 0x7fe5b4075310 2024-04-26T21:53:01.692 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:01.689+0000 7fe5b9e00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fe5b4110410 con 0x7fe5b40768f0 2024-04-26T21:53:01.692 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:01.689+0000 7fe5b9e00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe5b4110710 con 0x7fe5b410a820 2024-04-26T21:53:01.692 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:01.690+0000 7fe5b2a00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe5b4075310 0x7fe5b419ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:01.692 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:01.690+0000 7fe5b2a00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe5b4075310 0x7fe5b419ac20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:55996/0 (socket says 172.21.15.72:55996) 2024-04-26T21:53:01.692 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:01.690+0000 7fe5b2a00640 1 -- 172.21.15.72:0/4282764222 learned_addr learned my addr 172.21.15.72:0/4282764222 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:53:01.692 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:01.690+0000 7fe5b2000640 1 --2- 172.21.15.72:0/4282764222 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe5b40768f0 0x7fe5b419b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:01.692 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:01.690+0000 7fe5b3400640 1 --2- 172.21.15.72:0/4282764222 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe5b410a820 0x7fe5b41a2190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:01.692 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:01.690+0000 7fe5b2a00640 1 -- 172.21.15.72:0/4282764222 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe5b410a820 msgr2=0x7fe5b41a2190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:01.692 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:01.690+0000 7fe5b2a00640 1 --2- 172.21.15.72:0/4282764222 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe5b410a820 0x7fe5b41a2190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:01.692 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:01.690+0000 7fe5b2a00640 1 -- 172.21.15.72:0/4282764222 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe5b40768f0 msgr2=0x7fe5b419b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:01.692 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:01.690+0000 7fe5b2a00640 1 --2- 172.21.15.72:0/4282764222 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe5b40768f0 0x7fe5b419b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:01.692 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:01.690+0000 7fe5b2a00640 1 -- 172.21.15.72:0/4282764222 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5b41a2830 con 0x7fe5b4075310 2024-04-26T21:53:01.693 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:01.690+0000 7fe5b2000640 1 --2- 172.21.15.72:0/4282764222 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe5b40768f0 0x7fe5b419b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:53:01.693 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:01.690+0000 7fe5b2a00640 1 --2- 172.21.15.72:0/4282764222 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe5b4075310 0x7fe5b419ac20 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fe5a400c890 tx=0x7fe5a400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:53:01.693 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:01.691+0000 7fe5b8a00640 1 -- 172.21.15.72:0/4282764222 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5a4004220 con 0x7fe5b4075310 2024-04-26T21:53:01.693 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:01.691+0000 7fe5b8a00640 1 -- 172.21.15.72:0/4282764222 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe5a4007e40 con 0x7fe5b4075310 2024-04-26T21:53:01.693 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:01.691+0000 7fe5b9e00640 1 -- 172.21.15.72:0/4282764222 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5b41a2b20 con 0x7fe5b4075310 2024-04-26T21:53:01.693 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:01.691+0000 7fe5b8a00640 1 -- 172.21.15.72:0/4282764222 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5a4004a00 con 0x7fe5b4075310 2024-04-26T21:53:01.693 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:01.691+0000 7fe5b9e00640 1 -- 172.21.15.72:0/4282764222 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5b41a3070 con 0x7fe5b4075310 2024-04-26T21:53:01.693 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:01.692+0000 7fe5b9e00640 1 -- 172.21.15.72:0/4282764222 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5b41095e0 con 0x7fe5b4075310 2024-04-26T21:53:01.694 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:01.694+0000 7fe5b8a00640 1 -- 172.21.15.72:0/4282764222 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fe5a40079b0 con 0x7fe5b4075310 2024-04-26T21:53:01.695 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:01.695+0000 7fe5b8a00640 1 --2- 172.21.15.72:0/4282764222 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fe5980a3c40 0x7fe5980a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:01.695 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:01.695+0000 7fe5b8a00640 1 -- 172.21.15.72:0/4282764222 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7fe5a40d02f0 con 0x7fe5b4075310 2024-04-26T21:53:01.696 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:01.695+0000 7fe5b2000640 1 --2- 172.21.15.72:0/4282764222 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fe5980a3c40 0x7fe5980a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:01.696 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:01.696+0000 7fe5b2000640 1 --2- 172.21.15.72:0/4282764222 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fe5980a3c40 0x7fe5980a6100 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fe5a000baa0 tx=0x7fe5a0005d90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:53:01.700 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:01.700+0000 7fe5b8a00640 1 -- 172.21.15.72:0/4282764222 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fe5a4014030 con 0x7fe5b4075310 2024-04-26T21:53:01.902 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:01.902+0000 7fe5b9e00640 1 -- 172.21.15.72:0/4282764222 --> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe5b4062d70 con 0x7fe5980a3c40 2024-04-26T21:53:01.929 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:01.929+0000 7fe5b8a00640 1 -- 172.21.15.72:0/4282764222 <== mgr.14152 v2:172.21.15.72:6800/588150172 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258836 (secure 0 0 0) 0x7fe5b4062d70 con 0x7fe5980a3c40 2024-04-26T21:53:01.933 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T21:53:01.934 INFO:teuthology.orchestra.run.smithi072.stderr:dumped all 2024-04-26T21:53:01.936 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:01.934+0000 7fe5b9e00640 1 -- 172.21.15.72:0/4282764222 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fe5980a3c40 msgr2=0x7fe5980a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:01.936 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:01.934+0000 7fe5b9e00640 1 --2- 172.21.15.72:0/4282764222 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fe5980a3c40 0x7fe5980a6100 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fe5a000baa0 tx=0x7fe5a0005d90 comp rx=0 tx=0).stop 2024-04-26T21:53:01.936 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:01.934+0000 7fe5b9e00640 1 -- 172.21.15.72:0/4282764222 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe5b4075310 msgr2=0x7fe5b419ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:01.936 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:01.934+0000 7fe5b9e00640 1 --2- 172.21.15.72:0/4282764222 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe5b4075310 0x7fe5b419ac20 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fe5a400c890 tx=0x7fe5a400cd50 comp rx=0 tx=0).stop 2024-04-26T21:53:01.936 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:01.934+0000 7fe5b9e00640 1 -- 172.21.15.72:0/4282764222 shutdown_connections 2024-04-26T21:53:01.936 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:01.934+0000 7fe5b9e00640 1 --2- 172.21.15.72:0/4282764222 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fe5980a3c40 0x7fe5980a6100 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:01.936 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:01.934+0000 7fe5b9e00640 1 --2- 172.21.15.72:0/4282764222 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe5b410a820 0x7fe5b41a2190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:01.937 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:01.934+0000 7fe5b9e00640 1 --2- 172.21.15.72:0/4282764222 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe5b40768f0 0x7fe5b419b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:01.937 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:01.934+0000 7fe5b9e00640 1 --2- 172.21.15.72:0/4282764222 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe5b4075310 0x7fe5b419ac20 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:01.937 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:01.934+0000 7fe5b9e00640 1 -- 172.21.15.72:0/4282764222 >> 172.21.15.72:0/4282764222 conn(0x7fe5b40fdee0 msgr2=0x7fe5b41067b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:53:01.937 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:01.934+0000 7fe5b9e00640 1 -- 172.21.15.72:0/4282764222 shutdown_connections 2024-04-26T21:53:01.937 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:01.934+0000 7fe5b9e00640 1 -- 172.21.15.72:0/4282764222 wait complete. 2024-04-26T21:53:01.955 INFO:teuthology.orchestra.run.smithi072.stdout:{"pg_ready":true,"pg_map":{"version":373,"stamp":"2024-04-26T21:53:01.775330+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":605393,"num_objects":99,"num_object_clones":0,"num_object_copies":297,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":0,"num_read":160,"num_read_kb":140,"num_write":389,"num_write_kb":3010,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":65,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":319,"ondisk_log_size":319,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":391388,"kb_used_data":13116,"kb_used_omap":18,"kb_used_meta":378157,"kb_avail":1124550436,"statfs":{"total":1151940427776,"available":1151539646464,"internally_reserved":0,"allocated":13430784,"data_stored":7219087,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":387233082},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":20,"apply_latency_ms":20,"commit_latency_ns":20000000,"apply_latency_ns":20000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":15025,"num_objects":97,"num_object_clones":0,"num_object_copies":291,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":97,"num_whiteouts":0,"num_read":34,"num_read_kb":31,"num_write":208,"num_write_kb":72,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":65,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.023886"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:52.483427+0000","last_change":"2024-04-26T21:52:50.787765+0000","last_active":"2024-04-26T21:52:52.483427+0000","last_peered":"2024-04-26T21:52:52.483427+0000","last_clean":"2024-04-26T21:52:52.483427+0000","last_became_active":"2024-04-26T21:52:50.786969+0000","last_became_peered":"2024-04-26T21:52:50.786969+0000","last_unstale":"2024-04-26T21:52:52.483427+0000","last_undegraded":"2024-04-26T21:52:52.483427+0000","last_fullsized":"2024-04-26T21:52:52.483427+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:31:50.659210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:51.755499+0000","last_change":"2024-04-26T21:52:50.779735+0000","last_active":"2024-04-26T21:52:51.755499+0000","last_peered":"2024-04-26T21:52:51.755499+0000","last_clean":"2024-04-26T21:52:51.755499+0000","last_became_active":"2024-04-26T21:52:50.779358+0000","last_became_peered":"2024-04-26T21:52:50.779358+0000","last_unstale":"2024-04-26T21:52:51.755499+0000","last_undegraded":"2024-04-26T21:52:51.755499+0000","last_fullsized":"2024-04-26T21:52:51.755499+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:58:45.631365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:51.754519+0000","last_change":"2024-04-26T21:52:50.788220+0000","last_active":"2024-04-26T21:52:51.754519+0000","last_peered":"2024-04-26T21:52:51.754519+0000","last_clean":"2024-04-26T21:52:51.754519+0000","last_became_active":"2024-04-26T21:52:50.787282+0000","last_became_peered":"2024-04-26T21:52:50.787282+0000","last_unstale":"2024-04-26T21:52:51.754519+0000","last_undegraded":"2024-04-26T21:52:51.754519+0000","last_fullsized":"2024-04-26T21:52:51.754519+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:12:28.577893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:51.756281+0000","last_change":"2024-04-26T21:52:50.795367+0000","last_active":"2024-04-26T21:52:51.756281+0000","last_peered":"2024-04-26T21:52:51.756281+0000","last_clean":"2024-04-26T21:52:51.756281+0000","last_became_active":"2024-04-26T21:52:50.793602+0000","last_became_peered":"2024-04-26T21:52:50.793602+0000","last_unstale":"2024-04-26T21:52:51.756281+0000","last_undegraded":"2024-04-26T21:52:51.756281+0000","last_fullsized":"2024-04-26T21:52:51.756281+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:37:23.356792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:51.756020+0000","last_change":"2024-04-26T21:52:50.787333+0000","last_active":"2024-04-26T21:52:51.756020+0000","last_peered":"2024-04-26T21:52:51.756020+0000","last_clean":"2024-04-26T21:52:51.756020+0000","last_became_active":"2024-04-26T21:52:50.785319+0000","last_became_peered":"2024-04-26T21:52:50.785319+0000","last_unstale":"2024-04-26T21:52:51.756020+0000","last_undegraded":"2024-04-26T21:52:51.756020+0000","last_fullsized":"2024-04-26T21:52:51.756020+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:06:29.154825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:52.481166+0000","last_change":"2024-04-26T21:52:50.780829+0000","last_active":"2024-04-26T21:52:52.481166+0000","last_peered":"2024-04-26T21:52:52.481166+0000","last_clean":"2024-04-26T21:52:52.481166+0000","last_became_active":"2024-04-26T21:52:50.780439+0000","last_became_peered":"2024-04-26T21:52:50.780439+0000","last_unstale":"2024-04-26T21:52:52.481166+0000","last_undegraded":"2024-04-26T21:52:52.481166+0000","last_fullsized":"2024-04-26T21:52:52.481166+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:20:14.533853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:51.873603+0000","last_change":"2024-04-26T21:52:50.787679+0000","last_active":"2024-04-26T21:52:51.873603+0000","last_peered":"2024-04-26T21:52:51.873603+0000","last_clean":"2024-04-26T21:52:51.873603+0000","last_became_active":"2024-04-26T21:52:50.787277+0000","last_became_peered":"2024-04-26T21:52:50.787277+0000","last_unstale":"2024-04-26T21:52:51.873603+0000","last_undegraded":"2024-04-26T21:52:51.873603+0000","last_fullsized":"2024-04-26T21:52:51.873603+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:29:02.131503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:52.115260+0000","last_change":"2024-04-26T21:52:50.776468+0000","last_active":"2024-04-26T21:52:52.115260+0000","last_peered":"2024-04-26T21:52:52.115260+0000","last_clean":"2024-04-26T21:52:52.115260+0000","last_became_active":"2024-04-26T21:52:50.774710+0000","last_became_peered":"2024-04-26T21:52:50.774710+0000","last_unstale":"2024-04-26T21:52:52.115260+0000","last_undegraded":"2024-04-26T21:52:52.115260+0000","last_fullsized":"2024-04-26T21:52:52.115260+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:55:21.540496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:51.779245+0000","last_change":"2024-04-26T21:52:50.779701+0000","last_active":"2024-04-26T21:52:51.779245+0000","last_peered":"2024-04-26T21:52:51.779245+0000","last_clean":"2024-04-26T21:52:51.779245+0000","last_became_active":"2024-04-26T21:52:50.779331+0000","last_became_peered":"2024-04-26T21:52:50.779331+0000","last_unstale":"2024-04-26T21:52:51.779245+0000","last_undegraded":"2024-04-26T21:52:51.779245+0000","last_fullsized":"2024-04-26T21:52:51.779245+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:38:15.569806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:52.481498+0000","last_change":"2024-04-26T21:52:50.790081+0000","last_active":"2024-04-26T21:52:52.481498+0000","last_peered":"2024-04-26T21:52:52.481498+0000","last_clean":"2024-04-26T21:52:52.481498+0000","last_became_active":"2024-04-26T21:52:50.786575+0000","last_became_peered":"2024-04-26T21:52:50.786575+0000","last_unstale":"2024-04-26T21:52:52.481498+0000","last_undegraded":"2024-04-26T21:52:52.481498+0000","last_fullsized":"2024-04-26T21:52:52.481498+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:56:47.467128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:51.874528+0000","last_change":"2024-04-26T21:52:50.791983+0000","last_active":"2024-04-26T21:52:51.874528+0000","last_peered":"2024-04-26T21:52:51.874528+0000","last_clean":"2024-04-26T21:52:51.874528+0000","last_became_active":"2024-04-26T21:52:50.788312+0000","last_became_peered":"2024-04-26T21:52:50.788312+0000","last_unstale":"2024-04-26T21:52:51.874528+0000","last_undegraded":"2024-04-26T21:52:51.874528+0000","last_fullsized":"2024-04-26T21:52:51.874528+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:47:47.410267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:51.755892+0000","last_change":"2024-04-26T21:52:50.794237+0000","last_active":"2024-04-26T21:52:51.755892+0000","last_peered":"2024-04-26T21:52:51.755892+0000","last_clean":"2024-04-26T21:52:51.755892+0000","last_became_active":"2024-04-26T21:52:50.792260+0000","last_became_peered":"2024-04-26T21:52:50.792260+0000","last_unstale":"2024-04-26T21:52:51.755892+0000","last_undegraded":"2024-04-26T21:52:51.755892+0000","last_fullsized":"2024-04-26T21:52:51.755892+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:36:05.467667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:52.480858+0000","last_change":"2024-04-26T21:52:50.777687+0000","last_active":"2024-04-26T21:52:52.480858+0000","last_peered":"2024-04-26T21:52:52.480858+0000","last_clean":"2024-04-26T21:52:52.480858+0000","last_became_active":"2024-04-26T21:52:50.777200+0000","last_became_peered":"2024-04-26T21:52:50.777200+0000","last_unstale":"2024-04-26T21:52:52.480858+0000","last_undegraded":"2024-04-26T21:52:52.480858+0000","last_fullsized":"2024-04-26T21:52:52.480858+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:40:45.167144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:51.755105+0000","last_change":"2024-04-26T21:52:50.787008+0000","last_active":"2024-04-26T21:52:51.755105+0000","last_peered":"2024-04-26T21:52:51.755105+0000","last_clean":"2024-04-26T21:52:51.755105+0000","last_became_active":"2024-04-26T21:52:50.785656+0000","last_became_peered":"2024-04-26T21:52:50.785656+0000","last_unstale":"2024-04-26T21:52:51.755105+0000","last_undegraded":"2024-04-26T21:52:51.755105+0000","last_fullsized":"2024-04-26T21:52:51.755105+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:20:12.441281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:51.754196+0000","last_change":"2024-04-26T21:52:50.796537+0000","last_active":"2024-04-26T21:52:51.754196+0000","last_peered":"2024-04-26T21:52:51.754196+0000","last_clean":"2024-04-26T21:52:51.754196+0000","last_became_active":"2024-04-26T21:52:50.796229+0000","last_became_peered":"2024-04-26T21:52:50.796229+0000","last_unstale":"2024-04-26T21:52:51.754196+0000","last_undegraded":"2024-04-26T21:52:51.754196+0000","last_fullsized":"2024-04-26T21:52:51.754196+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:38:46.404444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:51.777837+0000","last_change":"2024-04-26T21:52:50.789211+0000","last_active":"2024-04-26T21:52:51.777837+0000","last_peered":"2024-04-26T21:52:51.777837+0000","last_clean":"2024-04-26T21:52:51.777837+0000","last_became_active":"2024-04-26T21:52:50.788889+0000","last_became_peered":"2024-04-26T21:52:50.788889+0000","last_unstale":"2024-04-26T21:52:51.777837+0000","last_undegraded":"2024-04-26T21:52:51.777837+0000","last_fullsized":"2024-04-26T21:52:51.777837+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:25:48.497840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:52.483655+0000","last_change":"2024-04-26T21:52:50.788035+0000","last_active":"2024-04-26T21:52:52.483655+0000","last_peered":"2024-04-26T21:52:52.483655+0000","last_clean":"2024-04-26T21:52:52.483655+0000","last_became_active":"2024-04-26T21:52:50.787453+0000","last_became_peered":"2024-04-26T21:52:50.787453+0000","last_unstale":"2024-04-26T21:52:52.483655+0000","last_undegraded":"2024-04-26T21:52:52.483655+0000","last_fullsized":"2024-04-26T21:52:52.483655+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:40:29.718537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:51.871413+0000","last_change":"2024-04-26T21:52:50.778228+0000","last_active":"2024-04-26T21:52:51.871413+0000","last_peered":"2024-04-26T21:52:51.871413+0000","last_clean":"2024-04-26T21:52:51.871413+0000","last_became_active":"2024-04-26T21:52:50.777934+0000","last_became_peered":"2024-04-26T21:52:50.777934+0000","last_unstale":"2024-04-26T21:52:51.871413+0000","last_undegraded":"2024-04-26T21:52:51.871413+0000","last_fullsized":"2024-04-26T21:52:51.871413+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:33:34.598325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:52.484945+0000","last_change":"2024-04-26T21:52:50.790795+0000","last_active":"2024-04-26T21:52:52.484945+0000","last_peered":"2024-04-26T21:52:52.484945+0000","last_clean":"2024-04-26T21:52:52.484945+0000","last_became_active":"2024-04-26T21:52:50.788254+0000","last_became_peered":"2024-04-26T21:52:50.788254+0000","last_unstale":"2024-04-26T21:52:52.484945+0000","last_undegraded":"2024-04-26T21:52:52.484945+0000","last_fullsized":"2024-04-26T21:52:52.484945+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:20:28.235137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"87'13","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:58.529618+0000","last_change":"2024-04-26T21:52:50.105513+0000","last_active":"2024-04-26T21:52:58.529618+0000","last_peered":"2024-04-26T21:52:58.529618+0000","last_clean":"2024-04-26T21:52:58.529618+0000","last_became_active":"2024-04-26T21:52:50.105131+0000","last_became_peered":"2024-04-26T21:52:50.105131+0000","last_unstale":"2024-04-26T21:52:58.529618+0000","last_undegraded":"2024-04-26T21:52:58.529618+0000","last_fullsized":"2024-04-26T21:52:58.529618+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:36:58.085803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:52.116654+0000","last_change":"2024-04-26T21:52:50.795025+0000","last_active":"2024-04-26T21:52:52.116654+0000","last_peered":"2024-04-26T21:52:52.116654+0000","last_clean":"2024-04-26T21:52:52.116654+0000","last_became_active":"2024-04-26T21:52:50.793608+0000","last_became_peered":"2024-04-26T21:52:50.793608+0000","last_unstale":"2024-04-26T21:52:52.116654+0000","last_undegraded":"2024-04-26T21:52:52.116654+0000","last_fullsized":"2024-04-26T21:52:52.116654+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:49:47.764848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"87'2","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:52.116883+0000","last_change":"2024-04-26T21:52:50.109197+0000","last_active":"2024-04-26T21:52:52.116883+0000","last_peered":"2024-04-26T21:52:52.116883+0000","last_clean":"2024-04-26T21:52:52.116883+0000","last_became_active":"2024-04-26T21:52:50.108835+0000","last_became_peered":"2024-04-26T21:52:50.108835+0000","last_unstale":"2024-04-26T21:52:52.116883+0000","last_undegraded":"2024-04-26T21:52:52.116883+0000","last_fullsized":"2024-04-26T21:52:52.116883+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:45:09.777747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:51.755270+0000","last_change":"2024-04-26T21:52:50.783094+0000","last_active":"2024-04-26T21:52:51.755270+0000","last_peered":"2024-04-26T21:52:51.755270+0000","last_clean":"2024-04-26T21:52:51.755270+0000","last_became_active":"2024-04-26T21:52:50.782468+0000","last_became_peered":"2024-04-26T21:52:50.782468+0000","last_unstale":"2024-04-26T21:52:51.755270+0000","last_undegraded":"2024-04-26T21:52:51.755270+0000","last_fullsized":"2024-04-26T21:52:51.755270+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:53:12.977054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"87'12","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:58.522410+0000","last_change":"2024-04-26T21:52:50.104824+0000","last_active":"2024-04-26T21:52:58.522410+0000","last_peered":"2024-04-26T21:52:58.522410+0000","last_clean":"2024-04-26T21:52:58.522410+0000","last_became_active":"2024-04-26T21:52:50.104514+0000","last_became_peered":"2024-04-26T21:52:50.104514+0000","last_unstale":"2024-04-26T21:52:58.522410+0000","last_undegraded":"2024-04-26T21:52:58.522410+0000","last_fullsized":"2024-04-26T21:52:58.522410+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:45:31.541940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:51.778523+0000","last_change":"2024-04-26T21:52:50.779695+0000","last_active":"2024-04-26T21:52:51.778523+0000","last_peered":"2024-04-26T21:52:51.778523+0000","last_clean":"2024-04-26T21:52:51.778523+0000","last_became_active":"2024-04-26T21:52:50.779234+0000","last_became_peered":"2024-04-26T21:52:50.779234+0000","last_unstale":"2024-04-26T21:52:51.778523+0000","last_undegraded":"2024-04-26T21:52:51.778523+0000","last_fullsized":"2024-04-26T21:52:51.778523+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:33:07.316543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"87'5","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:57.595629+0000","last_change":"2024-04-26T21:52:50.107403+0000","last_active":"2024-04-26T21:52:57.595629+0000","last_peered":"2024-04-26T21:52:57.595629+0000","last_clean":"2024-04-26T21:52:57.595629+0000","last_became_active":"2024-04-26T21:52:50.107126+0000","last_became_peered":"2024-04-26T21:52:50.107126+0000","last_unstale":"2024-04-26T21:52:57.595629+0000","last_undegraded":"2024-04-26T21:52:57.595629+0000","last_fullsized":"2024-04-26T21:52:57.595629+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:52:34.985448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:52.486606+0000","last_change":"2024-04-26T21:52:50.783674+0000","last_active":"2024-04-26T21:52:52.486606+0000","last_peered":"2024-04-26T21:52:52.486606+0000","last_clean":"2024-04-26T21:52:52.486606+0000","last_became_active":"2024-04-26T21:52:50.782283+0000","last_became_peered":"2024-04-26T21:52:50.782283+0000","last_unstale":"2024-04-26T21:52:52.486606+0000","last_undegraded":"2024-04-26T21:52:52.486606+0000","last_fullsized":"2024-04-26T21:52:52.486606+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:41:03.144601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"87'5","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:54.556735+0000","last_change":"2024-04-26T21:52:50.105204+0000","last_active":"2024-04-26T21:52:54.556735+0000","last_peered":"2024-04-26T21:52:54.556735+0000","last_clean":"2024-04-26T21:52:54.556735+0000","last_became_active":"2024-04-26T21:52:50.103980+0000","last_became_peered":"2024-04-26T21:52:50.103980+0000","last_unstale":"2024-04-26T21:52:54.556735+0000","last_undegraded":"2024-04-26T21:52:54.556735+0000","last_fullsized":"2024-04-26T21:52:54.556735+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:28:20.741673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:51.756064+0000","last_change":"2024-04-26T21:52:50.779541+0000","last_active":"2024-04-26T21:52:51.756064+0000","last_peered":"2024-04-26T21:52:51.756064+0000","last_clean":"2024-04-26T21:52:51.756064+0000","last_became_active":"2024-04-26T21:52:50.777800+0000","last_became_peered":"2024-04-26T21:52:50.777800+0000","last_unstale":"2024-04-26T21:52:51.756064+0000","last_undegraded":"2024-04-26T21:52:51.756064+0000","last_fullsized":"2024-04-26T21:52:51.756064+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:40:52.680799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"87'7","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:56.575049+0000","last_change":"2024-04-26T21:52:50.108552+0000","last_active":"2024-04-26T21:52:56.575049+0000","last_peered":"2024-04-26T21:52:56.575049+0000","last_clean":"2024-04-26T21:52:56.575049+0000","last_became_active":"2024-04-26T21:52:50.108071+0000","last_became_peered":"2024-04-26T21:52:50.108071+0000","last_unstale":"2024-04-26T21:52:56.575049+0000","last_undegraded":"2024-04-26T21:52:56.575049+0000","last_fullsized":"2024-04-26T21:52:56.575049+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:05:42.990950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:52.486390+0000","last_change":"2024-04-26T21:52:50.783788+0000","last_active":"2024-04-26T21:52:52.486390+0000","last_peered":"2024-04-26T21:52:52.486390+0000","last_clean":"2024-04-26T21:52:52.486390+0000","last_became_active":"2024-04-26T21:52:50.776645+0000","last_became_peered":"2024-04-26T21:52:50.776645+0000","last_unstale":"2024-04-26T21:52:52.486390+0000","last_undegraded":"2024-04-26T21:52:52.486390+0000","last_fullsized":"2024-04-26T21:52:52.486390+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:29:12.058885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"87'3","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:55.560252+0000","last_change":"2024-04-26T21:52:50.107564+0000","last_active":"2024-04-26T21:52:55.560252+0000","last_peered":"2024-04-26T21:52:55.560252+0000","last_clean":"2024-04-26T21:52:55.560252+0000","last_became_active":"2024-04-26T21:52:50.106653+0000","last_became_peered":"2024-04-26T21:52:50.106653+0000","last_unstale":"2024-04-26T21:52:55.560252+0000","last_undegraded":"2024-04-26T21:52:55.560252+0000","last_fullsized":"2024-04-26T21:52:55.560252+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:05:36.594729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:52.115266+0000","last_change":"2024-04-26T21:52:50.780279+0000","last_active":"2024-04-26T21:52:52.115266+0000","last_peered":"2024-04-26T21:52:52.115266+0000","last_clean":"2024-04-26T21:52:52.115266+0000","last_became_active":"2024-04-26T21:52:50.779950+0000","last_became_peered":"2024-04-26T21:52:50.779950+0000","last_unstale":"2024-04-26T21:52:52.115266+0000","last_undegraded":"2024-04-26T21:52:52.115266+0000","last_fullsized":"2024-04-26T21:52:52.115266+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:11:36.282319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"87'3","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:56.576436+0000","last_change":"2024-04-26T21:52:50.106269+0000","last_active":"2024-04-26T21:52:56.576436+0000","last_peered":"2024-04-26T21:52:56.576436+0000","last_clean":"2024-04-26T21:52:56.576436+0000","last_became_active":"2024-04-26T21:52:50.105948+0000","last_became_peered":"2024-04-26T21:52:50.105948+0000","last_unstale":"2024-04-26T21:52:56.576436+0000","last_undegraded":"2024-04-26T21:52:56.576436+0000","last_fullsized":"2024-04-26T21:52:56.576436+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:39:31.024362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:51.870542+0000","last_change":"2024-04-26T21:52:50.778938+0000","last_active":"2024-04-26T21:52:51.870542+0000","last_peered":"2024-04-26T21:52:51.870542+0000","last_clean":"2024-04-26T21:52:51.870542+0000","last_became_active":"2024-04-26T21:52:50.778590+0000","last_became_peered":"2024-04-26T21:52:50.778590+0000","last_unstale":"2024-04-26T21:52:51.870542+0000","last_undegraded":"2024-04-26T21:52:51.870542+0000","last_fullsized":"2024-04-26T21:52:51.870542+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:03:10.325793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"87'4","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:58.524883+0000","last_change":"2024-04-26T21:52:50.107243+0000","last_active":"2024-04-26T21:52:58.524883+0000","last_peered":"2024-04-26T21:52:58.524883+0000","last_clean":"2024-04-26T21:52:58.524883+0000","last_became_active":"2024-04-26T21:52:50.106729+0000","last_became_peered":"2024-04-26T21:52:50.106729+0000","last_unstale":"2024-04-26T21:52:58.524883+0000","last_undegraded":"2024-04-26T21:52:58.524883+0000","last_fullsized":"2024-04-26T21:52:58.524883+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:28:11.197997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:51.779017+0000","last_change":"2024-04-26T21:52:50.776906+0000","last_active":"2024-04-26T21:52:51.779017+0000","last_peered":"2024-04-26T21:52:51.779017+0000","last_clean":"2024-04-26T21:52:51.779017+0000","last_became_active":"2024-04-26T21:52:50.776567+0000","last_became_peered":"2024-04-26T21:52:50.776567+0000","last_unstale":"2024-04-26T21:52:51.779017+0000","last_undegraded":"2024-04-26T21:52:51.779017+0000","last_fullsized":"2024-04-26T21:52:51.779017+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:51:39.512572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"87'4","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:57.596522+0000","last_change":"2024-04-26T21:52:50.109792+0000","last_active":"2024-04-26T21:52:57.596522+0000","last_peered":"2024-04-26T21:52:57.596522+0000","last_clean":"2024-04-26T21:52:57.596522+0000","last_became_active":"2024-04-26T21:52:50.106662+0000","last_became_peered":"2024-04-26T21:52:50.106662+0000","last_unstale":"2024-04-26T21:52:57.596522+0000","last_undegraded":"2024-04-26T21:52:57.596522+0000","last_fullsized":"2024-04-26T21:52:57.596522+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:41:27.390780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:51.754737+0000","last_change":"2024-04-26T21:52:50.782798+0000","last_active":"2024-04-26T21:52:51.754737+0000","last_peered":"2024-04-26T21:52:51.754737+0000","last_clean":"2024-04-26T21:52:51.754737+0000","last_became_active":"2024-04-26T21:52:50.781975+0000","last_became_peered":"2024-04-26T21:52:50.781975+0000","last_unstale":"2024-04-26T21:52:51.754737+0000","last_undegraded":"2024-04-26T21:52:51.754737+0000","last_fullsized":"2024-04-26T21:52:51.754737+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:13:07.261095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"87'5","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:54.549751+0000","last_change":"2024-04-26T21:52:50.099296+0000","last_active":"2024-04-26T21:52:54.549751+0000","last_peered":"2024-04-26T21:52:54.549751+0000","last_clean":"2024-04-26T21:52:54.549751+0000","last_became_active":"2024-04-26T21:52:50.098940+0000","last_became_peered":"2024-04-26T21:52:50.098940+0000","last_unstale":"2024-04-26T21:52:54.549751+0000","last_undegraded":"2024-04-26T21:52:54.549751+0000","last_fullsized":"2024-04-26T21:52:54.549751+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:49:15.765474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:51.778734+0000","last_change":"2024-04-26T21:52:50.783341+0000","last_active":"2024-04-26T21:52:51.778734+0000","last_peered":"2024-04-26T21:52:51.778734+0000","last_clean":"2024-04-26T21:52:51.778734+0000","last_became_active":"2024-04-26T21:52:50.783025+0000","last_became_peered":"2024-04-26T21:52:50.783025+0000","last_unstale":"2024-04-26T21:52:51.778734+0000","last_undegraded":"2024-04-26T21:52:51.778734+0000","last_fullsized":"2024-04-26T21:52:51.778734+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:54:06.070550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"87'13","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:58.533229+0000","last_change":"2024-04-26T21:52:50.106617+0000","last_active":"2024-04-26T21:52:58.533229+0000","last_peered":"2024-04-26T21:52:58.533229+0000","last_clean":"2024-04-26T21:52:58.533229+0000","last_became_active":"2024-04-26T21:52:50.106161+0000","last_became_peered":"2024-04-26T21:52:50.106161+0000","last_unstale":"2024-04-26T21:52:58.533229+0000","last_undegraded":"2024-04-26T21:52:58.533229+0000","last_fullsized":"2024-04-26T21:52:58.533229+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:52:07.977749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8212,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":15,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:51.757241+0000","last_change":"2024-04-26T21:52:50.794567+0000","last_active":"2024-04-26T21:52:51.757241+0000","last_peered":"2024-04-26T21:52:51.757241+0000","last_clean":"2024-04-26T21:52:51.757241+0000","last_became_active":"2024-04-26T21:52:50.794245+0000","last_became_peered":"2024-04-26T21:52:50.794245+0000","last_unstale":"2024-04-26T21:52:51.757241+0000","last_undegraded":"2024-04-26T21:52:51.757241+0000","last_fullsized":"2024-04-26T21:52:51.757241+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:58:11.501104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:58.522406+0000","last_change":"2024-04-26T21:52:50.102156+0000","last_active":"2024-04-26T21:52:58.522406+0000","last_peered":"2024-04-26T21:52:58.522406+0000","last_clean":"2024-04-26T21:52:58.522406+0000","last_became_active":"2024-04-26T21:52:50.101721+0000","last_became_peered":"2024-04-26T21:52:50.101721+0000","last_unstale":"2024-04-26T21:52:58.522406+0000","last_undegraded":"2024-04-26T21:52:58.522406+0000","last_fullsized":"2024-04-26T21:52:58.522406+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:48:42.081874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:52.480661+0000","last_change":"2024-04-26T21:52:50.789199+0000","last_active":"2024-04-26T21:52:52.480661+0000","last_peered":"2024-04-26T21:52:52.480661+0000","last_clean":"2024-04-26T21:52:52.480661+0000","last_became_active":"2024-04-26T21:52:50.788868+0000","last_became_peered":"2024-04-26T21:52:50.788868+0000","last_unstale":"2024-04-26T21:52:52.480661+0000","last_undegraded":"2024-04-26T21:52:52.480661+0000","last_fullsized":"2024-04-26T21:52:52.480661+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:29:55.755564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"87'5","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:56.579192+0000","last_change":"2024-04-26T21:52:50.104920+0000","last_active":"2024-04-26T21:52:56.579192+0000","last_peered":"2024-04-26T21:52:56.579192+0000","last_clean":"2024-04-26T21:52:56.579192+0000","last_became_active":"2024-04-26T21:52:50.104579+0000","last_became_peered":"2024-04-26T21:52:50.104579+0000","last_unstale":"2024-04-26T21:52:56.579192+0000","last_undegraded":"2024-04-26T21:52:56.579192+0000","last_fullsized":"2024-04-26T21:52:56.579192+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:09:35.760918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:51.874565+0000","last_change":"2024-04-26T21:52:50.787016+0000","last_active":"2024-04-26T21:52:51.874565+0000","last_peered":"2024-04-26T21:52:51.874565+0000","last_clean":"2024-04-26T21:52:51.874565+0000","last_became_active":"2024-04-26T21:52:50.778439+0000","last_became_peered":"2024-04-26T21:52:50.778439+0000","last_unstale":"2024-04-26T21:52:51.874565+0000","last_undegraded":"2024-04-26T21:52:51.874565+0000","last_fullsized":"2024-04-26T21:52:51.874565+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:13:35.735480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"87'5","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:58.526816+0000","last_change":"2024-04-26T21:52:50.110175+0000","last_active":"2024-04-26T21:52:58.526816+0000","last_peered":"2024-04-26T21:52:58.526816+0000","last_clean":"2024-04-26T21:52:58.526816+0000","last_became_active":"2024-04-26T21:52:50.109779+0000","last_became_peered":"2024-04-26T21:52:50.109779+0000","last_unstale":"2024-04-26T21:52:58.526816+0000","last_undegraded":"2024-04-26T21:52:58.526816+0000","last_fullsized":"2024-04-26T21:52:58.526816+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:06:46.756327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:51.873835+0000","last_change":"2024-04-26T21:52:50.784199+0000","last_active":"2024-04-26T21:52:51.873835+0000","last_peered":"2024-04-26T21:52:51.873835+0000","last_clean":"2024-04-26T21:52:51.873835+0000","last_became_active":"2024-04-26T21:52:50.783820+0000","last_became_peered":"2024-04-26T21:52:50.783820+0000","last_unstale":"2024-04-26T21:52:51.873835+0000","last_undegraded":"2024-04-26T21:52:51.873835+0000","last_fullsized":"2024-04-26T21:52:51.873835+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:45:24.776722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"87'17","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:57.598650+0000","last_change":"2024-04-26T21:52:50.095685+0000","last_active":"2024-04-26T21:52:57.598650+0000","last_peered":"2024-04-26T21:52:57.598650+0000","last_clean":"2024-04-26T21:52:57.598650+0000","last_became_active":"2024-04-26T21:52:50.095298+0000","last_became_peered":"2024-04-26T21:52:50.095298+0000","last_unstale":"2024-04-26T21:52:57.598650+0000","last_undegraded":"2024-04-26T21:52:57.598650+0000","last_fullsized":"2024-04-26T21:52:57.598650+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:47:26.061531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:52.485286+0000","last_change":"2024-04-26T21:52:50.786607+0000","last_active":"2024-04-26T21:52:52.485286+0000","last_peered":"2024-04-26T21:52:52.485286+0000","last_clean":"2024-04-26T21:52:52.485286+0000","last_became_active":"2024-04-26T21:52:50.786261+0000","last_became_peered":"2024-04-26T21:52:50.786261+0000","last_unstale":"2024-04-26T21:52:52.485286+0000","last_undegraded":"2024-04-26T21:52:52.485286+0000","last_fullsized":"2024-04-26T21:52:52.485286+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:08:45.902260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"87'2","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:57.550958+0000","last_change":"2024-04-26T21:52:50.095782+0000","last_active":"2024-04-26T21:52:57.550958+0000","last_peered":"2024-04-26T21:52:57.550958+0000","last_clean":"2024-04-26T21:52:57.550958+0000","last_became_active":"2024-04-26T21:52:50.093983+0000","last_became_peered":"2024-04-26T21:52:50.093983+0000","last_unstale":"2024-04-26T21:52:57.550958+0000","last_undegraded":"2024-04-26T21:52:57.550958+0000","last_fullsized":"2024-04-26T21:52:57.550958+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:18:40.103099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:52.482451+0000","last_change":"2024-04-26T21:52:50.791969+0000","last_active":"2024-04-26T21:52:52.482451+0000","last_peered":"2024-04-26T21:52:52.482451+0000","last_clean":"2024-04-26T21:52:52.482451+0000","last_became_active":"2024-04-26T21:52:50.790441+0000","last_became_peered":"2024-04-26T21:52:50.790441+0000","last_unstale":"2024-04-26T21:52:52.482451+0000","last_undegraded":"2024-04-26T21:52:52.482451+0000","last_fullsized":"2024-04-26T21:52:52.482451+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:11:04.964439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"87'1","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:51.870987+0000","last_change":"2024-04-26T21:52:50.101807+0000","last_active":"2024-04-26T21:52:51.870987+0000","last_peered":"2024-04-26T21:52:51.870987+0000","last_clean":"2024-04-26T21:52:51.870987+0000","last_became_active":"2024-04-26T21:52:50.101329+0000","last_became_peered":"2024-04-26T21:52:50.101329+0000","last_unstale":"2024-04-26T21:52:51.870987+0000","last_undegraded":"2024-04-26T21:52:51.870987+0000","last_fullsized":"2024-04-26T21:52:51.870987+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:54:47.788172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:52.481361+0000","last_change":"2024-04-26T21:52:50.780872+0000","last_active":"2024-04-26T21:52:52.481361+0000","last_peered":"2024-04-26T21:52:52.481361+0000","last_clean":"2024-04-26T21:52:52.481361+0000","last_became_active":"2024-04-26T21:52:50.778757+0000","last_became_peered":"2024-04-26T21:52:50.778757+0000","last_unstale":"2024-04-26T21:52:52.481361+0000","last_undegraded":"2024-04-26T21:52:52.481361+0000","last_fullsized":"2024-04-26T21:52:52.481361+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:07:21.595091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"87'5","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:57.600350+0000","last_change":"2024-04-26T21:52:50.106886+0000","last_active":"2024-04-26T21:52:57.600350+0000","last_peered":"2024-04-26T21:52:57.600350+0000","last_clean":"2024-04-26T21:52:57.600350+0000","last_became_active":"2024-04-26T21:52:50.106150+0000","last_became_peered":"2024-04-26T21:52:50.106150+0000","last_unstale":"2024-04-26T21:52:57.600350+0000","last_undegraded":"2024-04-26T21:52:57.600350+0000","last_fullsized":"2024-04-26T21:52:57.600350+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:20:54.920416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:52.481461+0000","last_change":"2024-04-26T21:52:50.792037+0000","last_active":"2024-04-26T21:52:52.481461+0000","last_peered":"2024-04-26T21:52:52.481461+0000","last_clean":"2024-04-26T21:52:52.481461+0000","last_became_active":"2024-04-26T21:52:50.791684+0000","last_became_peered":"2024-04-26T21:52:50.791684+0000","last_unstale":"2024-04-26T21:52:52.481461+0000","last_undegraded":"2024-04-26T21:52:52.481461+0000","last_fullsized":"2024-04-26T21:52:52.481461+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:11:00.489287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:52.485049+0000","last_change":"2024-04-26T21:52:50.785257+0000","last_active":"2024-04-26T21:52:52.485049+0000","last_peered":"2024-04-26T21:52:52.485049+0000","last_clean":"2024-04-26T21:52:52.485049+0000","last_became_active":"2024-04-26T21:52:50.784906+0000","last_became_peered":"2024-04-26T21:52:50.784906+0000","last_unstale":"2024-04-26T21:52:52.485049+0000","last_undegraded":"2024-04-26T21:52:52.485049+0000","last_fullsized":"2024-04-26T21:52:52.485049+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:04:50.921663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"87'5","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:58.522577+0000","last_change":"2024-04-26T21:52:50.102425+0000","last_active":"2024-04-26T21:52:58.522577+0000","last_peered":"2024-04-26T21:52:58.522577+0000","last_clean":"2024-04-26T21:52:58.522577+0000","last_became_active":"2024-04-26T21:52:50.101703+0000","last_became_peered":"2024-04-26T21:52:50.101703+0000","last_unstale":"2024-04-26T21:52:58.522577+0000","last_undegraded":"2024-04-26T21:52:58.522577+0000","last_fullsized":"2024-04-26T21:52:58.522577+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:13:39.910519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:52.116006+0000","last_change":"2024-04-26T21:52:50.775036+0000","last_active":"2024-04-26T21:52:52.116006+0000","last_peered":"2024-04-26T21:52:52.116006+0000","last_clean":"2024-04-26T21:52:52.116006+0000","last_became_active":"2024-04-26T21:52:50.773347+0000","last_became_peered":"2024-04-26T21:52:50.773347+0000","last_unstale":"2024-04-26T21:52:52.116006+0000","last_undegraded":"2024-04-26T21:52:52.116006+0000","last_fullsized":"2024-04-26T21:52:52.116006+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:59:55.303014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"87'6","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:58.525913+0000","last_change":"2024-04-26T21:52:50.109437+0000","last_active":"2024-04-26T21:52:58.525913+0000","last_peered":"2024-04-26T21:52:58.525913+0000","last_clean":"2024-04-26T21:52:58.525913+0000","last_became_active":"2024-04-26T21:52:50.108605+0000","last_became_peered":"2024-04-26T21:52:50.108605+0000","last_unstale":"2024-04-26T21:52:58.525913+0000","last_undegraded":"2024-04-26T21:52:58.525913+0000","last_fullsized":"2024-04-26T21:52:58.525913+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:31:04.208334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":341,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:51.873223+0000","last_change":"2024-04-26T21:51:11.498577+0000","last_active":"2024-04-26T21:52:51.873223+0000","last_peered":"2024-04-26T21:52:51.873223+0000","last_clean":"2024-04-26T21:52:51.873223+0000","last_became_active":"2024-04-26T21:51:11.497422+0000","last_became_peered":"2024-04-26T21:51:11.497422+0000","last_unstale":"2024-04-26T21:52:51.873223+0000","last_undegraded":"2024-04-26T21:52:51.873223+0000","last_fullsized":"2024-04-26T21:52:51.873223+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:46:29.789400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:46:29.789400+0000","last_clean_scrub_stamp":"2024-04-26T21:46:29.789400+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:54:13.450359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:52.480552+0000","last_change":"2024-04-26T21:52:50.794270+0000","last_active":"2024-04-26T21:52:52.480552+0000","last_peered":"2024-04-26T21:52:52.480552+0000","last_clean":"2024-04-26T21:52:52.480552+0000","last_became_active":"2024-04-26T21:52:50.793966+0000","last_became_peered":"2024-04-26T21:52:50.793966+0000","last_unstale":"2024-04-26T21:52:52.480552+0000","last_undegraded":"2024-04-26T21:52:52.480552+0000","last_fullsized":"2024-04-26T21:52:52.480552+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:01:37.668466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"87'4","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:55.561049+0000","last_change":"2024-04-26T21:52:50.102700+0000","last_active":"2024-04-26T21:52:55.561049+0000","last_peered":"2024-04-26T21:52:55.561049+0000","last_clean":"2024-04-26T21:52:55.561049+0000","last_became_active":"2024-04-26T21:52:50.101267+0000","last_became_peered":"2024-04-26T21:52:50.101267+0000","last_unstale":"2024-04-26T21:52:55.561049+0000","last_undegraded":"2024-04-26T21:52:55.561049+0000","last_fullsized":"2024-04-26T21:52:55.561049+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:41:21.727411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"87'4","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:57.598687+0000","last_change":"2024-04-26T21:52:50.102191+0000","last_active":"2024-04-26T21:52:57.598687+0000","last_peered":"2024-04-26T21:52:57.598687+0000","last_clean":"2024-04-26T21:52:57.598687+0000","last_became_active":"2024-04-26T21:52:50.101307+0000","last_became_peered":"2024-04-26T21:52:50.101307+0000","last_unstale":"2024-04-26T21:52:57.598687+0000","last_undegraded":"2024-04-26T21:52:57.598687+0000","last_fullsized":"2024-04-26T21:52:57.598687+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:59:12.465291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:52.481172+0000","last_change":"2024-04-26T21:52:50.792020+0000","last_active":"2024-04-26T21:52:52.481172+0000","last_peered":"2024-04-26T21:52:52.481172+0000","last_clean":"2024-04-26T21:52:52.481172+0000","last_became_active":"2024-04-26T21:52:50.791677+0000","last_became_peered":"2024-04-26T21:52:50.791677+0000","last_unstale":"2024-04-26T21:52:52.481172+0000","last_undegraded":"2024-04-26T21:52:52.481172+0000","last_fullsized":"2024-04-26T21:52:52.481172+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:11:42.218294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"87'8","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:56.581638+0000","last_change":"2024-04-26T21:52:50.110060+0000","last_active":"2024-04-26T21:52:56.581638+0000","last_peered":"2024-04-26T21:52:56.581638+0000","last_clean":"2024-04-26T21:52:56.581638+0000","last_became_active":"2024-04-26T21:52:50.109092+0000","last_became_peered":"2024-04-26T21:52:50.109092+0000","last_unstale":"2024-04-26T21:52:56.581638+0000","last_undegraded":"2024-04-26T21:52:56.581638+0000","last_fullsized":"2024-04-26T21:52:56.581638+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:09:35.789887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:52.117243+0000","last_change":"2024-04-26T21:52:50.779775+0000","last_active":"2024-04-26T21:52:52.117243+0000","last_peered":"2024-04-26T21:52:52.117243+0000","last_clean":"2024-04-26T21:52:52.117243+0000","last_became_active":"2024-04-26T21:52:50.778857+0000","last_became_peered":"2024-04-26T21:52:50.778857+0000","last_unstale":"2024-04-26T21:52:52.117243+0000","last_undegraded":"2024-04-26T21:52:52.117243+0000","last_fullsized":"2024-04-26T21:52:52.117243+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:59:50.946578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"87'7","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:57.600430+0000","last_change":"2024-04-26T21:52:50.113027+0000","last_active":"2024-04-26T21:52:57.600430+0000","last_peered":"2024-04-26T21:52:57.600430+0000","last_clean":"2024-04-26T21:52:57.600430+0000","last_became_active":"2024-04-26T21:52:50.112637+0000","last_became_peered":"2024-04-26T21:52:50.112637+0000","last_unstale":"2024-04-26T21:52:57.600430+0000","last_undegraded":"2024-04-26T21:52:57.600430+0000","last_fullsized":"2024-04-26T21:52:57.600430+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:09:47.798184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:51.755142+0000","last_change":"2024-04-26T21:52:50.785119+0000","last_active":"2024-04-26T21:52:51.755142+0000","last_peered":"2024-04-26T21:52:51.755142+0000","last_clean":"2024-04-26T21:52:51.755142+0000","last_became_active":"2024-04-26T21:52:50.781005+0000","last_became_peered":"2024-04-26T21:52:50.781005+0000","last_unstale":"2024-04-26T21:52:51.755142+0000","last_undegraded":"2024-04-26T21:52:51.755142+0000","last_fullsized":"2024-04-26T21:52:51.755142+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:16:32.986810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"87'2","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:54.554877+0000","last_change":"2024-04-26T21:52:50.112263+0000","last_active":"2024-04-26T21:52:54.554877+0000","last_peered":"2024-04-26T21:52:54.554877+0000","last_clean":"2024-04-26T21:52:54.554877+0000","last_became_active":"2024-04-26T21:52:50.110649+0000","last_became_peered":"2024-04-26T21:52:50.110649+0000","last_unstale":"2024-04-26T21:52:54.554877+0000","last_undegraded":"2024-04-26T21:52:54.554877+0000","last_fullsized":"2024-04-26T21:52:54.554877+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:14:23.245000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:51.755989+0000","last_change":"2024-04-26T21:52:50.790674+0000","last_active":"2024-04-26T21:52:51.755989+0000","last_peered":"2024-04-26T21:52:51.755989+0000","last_clean":"2024-04-26T21:52:51.755989+0000","last_became_active":"2024-04-26T21:52:50.790286+0000","last_became_peered":"2024-04-26T21:52:50.790286+0000","last_unstale":"2024-04-26T21:52:51.755989+0000","last_undegraded":"2024-04-26T21:52:51.755989+0000","last_fullsized":"2024-04-26T21:52:51.755989+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:22:32.246321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"87'5","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:56.576887+0000","last_change":"2024-04-26T21:52:50.112158+0000","last_active":"2024-04-26T21:52:56.576887+0000","last_peered":"2024-04-26T21:52:56.576887+0000","last_clean":"2024-04-26T21:52:56.576887+0000","last_became_active":"2024-04-26T21:52:50.111453+0000","last_became_peered":"2024-04-26T21:52:50.111453+0000","last_unstale":"2024-04-26T21:52:56.576887+0000","last_undegraded":"2024-04-26T21:52:56.576887+0000","last_fullsized":"2024-04-26T21:52:56.576887+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:16:58.626985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:52.481913+0000","last_change":"2024-04-26T21:52:50.798241+0000","last_active":"2024-04-26T21:52:52.481913+0000","last_peered":"2024-04-26T21:52:52.481913+0000","last_clean":"2024-04-26T21:52:52.481913+0000","last_became_active":"2024-04-26T21:52:50.797878+0000","last_became_peered":"2024-04-26T21:52:50.797878+0000","last_unstale":"2024-04-26T21:52:52.481913+0000","last_undegraded":"2024-04-26T21:52:52.481913+0000","last_fullsized":"2024-04-26T21:52:52.481913+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:42:49.363849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"87'3","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:56.581807+0000","last_change":"2024-04-26T21:52:50.103848+0000","last_active":"2024-04-26T21:52:56.581807+0000","last_peered":"2024-04-26T21:52:56.581807+0000","last_clean":"2024-04-26T21:52:56.581807+0000","last_became_active":"2024-04-26T21:52:50.102606+0000","last_became_peered":"2024-04-26T21:52:50.102606+0000","last_unstale":"2024-04-26T21:52:56.581807+0000","last_undegraded":"2024-04-26T21:52:56.581807+0000","last_fullsized":"2024-04-26T21:52:56.581807+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:35:37.023110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:52.116330+0000","last_change":"2024-04-26T21:52:50.776333+0000","last_active":"2024-04-26T21:52:52.116330+0000","last_peered":"2024-04-26T21:52:52.116330+0000","last_clean":"2024-04-26T21:52:52.116330+0000","last_became_active":"2024-04-26T21:52:50.776001+0000","last_became_peered":"2024-04-26T21:52:50.776001+0000","last_unstale":"2024-04-26T21:52:52.116330+0000","last_undegraded":"2024-04-26T21:52:52.116330+0000","last_fullsized":"2024-04-26T21:52:52.116330+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:38:26.578928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"87'4","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:54.556970+0000","last_change":"2024-04-26T21:52:50.106008+0000","last_active":"2024-04-26T21:52:54.556970+0000","last_peered":"2024-04-26T21:52:54.556970+0000","last_clean":"2024-04-26T21:52:54.556970+0000","last_became_active":"2024-04-26T21:52:50.105565+0000","last_became_peered":"2024-04-26T21:52:50.105565+0000","last_unstale":"2024-04-26T21:52:54.556970+0000","last_undegraded":"2024-04-26T21:52:54.556970+0000","last_fullsized":"2024-04-26T21:52:54.556970+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:15:17.355461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:51.873809+0000","last_change":"2024-04-26T21:52:50.776051+0000","last_active":"2024-04-26T21:52:51.873809+0000","last_peered":"2024-04-26T21:52:51.873809+0000","last_clean":"2024-04-26T21:52:51.873809+0000","last_became_active":"2024-04-26T21:52:50.775716+0000","last_became_peered":"2024-04-26T21:52:50.775716+0000","last_unstale":"2024-04-26T21:52:51.873809+0000","last_undegraded":"2024-04-26T21:52:51.873809+0000","last_fullsized":"2024-04-26T21:52:51.873809+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:34:21.110773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"87'6","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:58.531241+0000","last_change":"2024-04-26T21:52:50.111280+0000","last_active":"2024-04-26T21:52:58.531241+0000","last_peered":"2024-04-26T21:52:58.531241+0000","last_clean":"2024-04-26T21:52:58.531241+0000","last_became_active":"2024-04-26T21:52:50.110972+0000","last_became_peered":"2024-04-26T21:52:50.110972+0000","last_unstale":"2024-04-26T21:52:58.531241+0000","last_undegraded":"2024-04-26T21:52:58.531241+0000","last_fullsized":"2024-04-26T21:52:58.531241+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:41:48.203134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1561,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:51.874639+0000","last_change":"2024-04-26T21:52:50.777488+0000","last_active":"2024-04-26T21:52:51.874639+0000","last_peered":"2024-04-26T21:52:51.874639+0000","last_clean":"2024-04-26T21:52:51.874639+0000","last_became_active":"2024-04-26T21:52:50.777162+0000","last_became_peered":"2024-04-26T21:52:50.777162+0000","last_unstale":"2024-04-26T21:52:51.874639+0000","last_undegraded":"2024-04-26T21:52:51.874639+0000","last_fullsized":"2024-04-26T21:52:51.874639+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:07:22.683109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"87'3","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:56.577754+0000","last_change":"2024-04-26T21:52:50.104508+0000","last_active":"2024-04-26T21:52:56.577754+0000","last_peered":"2024-04-26T21:52:56.577754+0000","last_clean":"2024-04-26T21:52:56.577754+0000","last_became_active":"2024-04-26T21:52:50.103924+0000","last_became_peered":"2024-04-26T21:52:50.103924+0000","last_unstale":"2024-04-26T21:52:56.577754+0000","last_undegraded":"2024-04-26T21:52:56.577754+0000","last_fullsized":"2024-04-26T21:52:56.577754+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:39:25.861339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:51.874293+0000","last_change":"2024-04-26T21:52:50.782915+0000","last_active":"2024-04-26T21:52:51.874293+0000","last_peered":"2024-04-26T21:52:51.874293+0000","last_clean":"2024-04-26T21:52:51.874293+0000","last_became_active":"2024-04-26T21:52:50.782253+0000","last_became_peered":"2024-04-26T21:52:50.782253+0000","last_unstale":"2024-04-26T21:52:51.874293+0000","last_undegraded":"2024-04-26T21:52:51.874293+0000","last_fullsized":"2024-04-26T21:52:51.874293+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:12:51.624162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"87'5","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:57.596130+0000","last_change":"2024-04-26T21:52:50.100008+0000","last_active":"2024-04-26T21:52:57.596130+0000","last_peered":"2024-04-26T21:52:57.596130+0000","last_clean":"2024-04-26T21:52:57.596130+0000","last_became_active":"2024-04-26T21:52:50.099537+0000","last_became_peered":"2024-04-26T21:52:50.099537+0000","last_unstale":"2024-04-26T21:52:57.596130+0000","last_undegraded":"2024-04-26T21:52:57.596130+0000","last_fullsized":"2024-04-26T21:52:57.596130+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:12:11.901178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:51.874353+0000","last_change":"2024-04-26T21:52:50.787777+0000","last_active":"2024-04-26T21:52:51.874353+0000","last_peered":"2024-04-26T21:52:51.874353+0000","last_clean":"2024-04-26T21:52:51.874353+0000","last_became_active":"2024-04-26T21:52:50.782403+0000","last_became_peered":"2024-04-26T21:52:50.782403+0000","last_unstale":"2024-04-26T21:52:51.874353+0000","last_undegraded":"2024-04-26T21:52:51.874353+0000","last_fullsized":"2024-04-26T21:52:51.874353+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:28:58.425802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:52.482233+0000","last_change":"2024-04-26T21:52:50.800010+0000","last_active":"2024-04-26T21:52:52.482233+0000","last_peered":"2024-04-26T21:52:52.482233+0000","last_clean":"2024-04-26T21:52:52.482233+0000","last_became_active":"2024-04-26T21:52:50.799308+0000","last_became_peered":"2024-04-26T21:52:50.799308+0000","last_unstale":"2024-04-26T21:52:52.482233+0000","last_undegraded":"2024-04-26T21:52:52.482233+0000","last_fullsized":"2024-04-26T21:52:52.482233+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:54:25.772312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:52.484126+0000","last_change":"2024-04-26T21:52:50.798281+0000","last_active":"2024-04-26T21:52:52.484126+0000","last_peered":"2024-04-26T21:52:52.484126+0000","last_clean":"2024-04-26T21:52:52.484126+0000","last_became_active":"2024-04-26T21:52:50.795879+0000","last_became_peered":"2024-04-26T21:52:50.795879+0000","last_unstale":"2024-04-26T21:52:52.484126+0000","last_undegraded":"2024-04-26T21:52:52.484126+0000","last_fullsized":"2024-04-26T21:52:52.484126+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:36:00.769991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:52.484457+0000","last_change":"2024-04-26T21:52:50.791058+0000","last_active":"2024-04-26T21:52:52.484457+0000","last_peered":"2024-04-26T21:52:52.484457+0000","last_clean":"2024-04-26T21:52:52.484457+0000","last_became_active":"2024-04-26T21:52:50.790721+0000","last_became_peered":"2024-04-26T21:52:50.790721+0000","last_unstale":"2024-04-26T21:52:52.484457+0000","last_undegraded":"2024-04-26T21:52:52.484457+0000","last_fullsized":"2024-04-26T21:52:52.484457+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:56:03.239622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:51.754785+0000","last_change":"2024-04-26T21:52:50.788193+0000","last_active":"2024-04-26T21:52:51.754785+0000","last_peered":"2024-04-26T21:52:51.754785+0000","last_clean":"2024-04-26T21:52:51.754785+0000","last_became_active":"2024-04-26T21:52:50.786749+0000","last_became_peered":"2024-04-26T21:52:50.786749+0000","last_unstale":"2024-04-26T21:52:51.754785+0000","last_undegraded":"2024-04-26T21:52:51.754785+0000","last_fullsized":"2024-04-26T21:52:51.754785+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:22:11.277217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:52.481032+0000","last_change":"2024-04-26T21:52:50.782508+0000","last_active":"2024-04-26T21:52:52.481032+0000","last_peered":"2024-04-26T21:52:52.481032+0000","last_clean":"2024-04-26T21:52:52.481032+0000","last_became_active":"2024-04-26T21:52:50.781505+0000","last_became_peered":"2024-04-26T21:52:50.781505+0000","last_unstale":"2024-04-26T21:52:52.481032+0000","last_undegraded":"2024-04-26T21:52:52.481032+0000","last_fullsized":"2024-04-26T21:52:52.481032+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:22:57.867546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:52.482378+0000","last_change":"2024-04-26T21:52:50.789205+0000","last_active":"2024-04-26T21:52:52.482378+0000","last_peered":"2024-04-26T21:52:52.482378+0000","last_clean":"2024-04-26T21:52:52.482378+0000","last_became_active":"2024-04-26T21:52:50.788898+0000","last_became_peered":"2024-04-26T21:52:50.788898+0000","last_unstale":"2024-04-26T21:52:52.482378+0000","last_undegraded":"2024-04-26T21:52:52.482378+0000","last_fullsized":"2024-04-26T21:52:52.482378+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:18:51.401968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:52.485015+0000","last_change":"2024-04-26T21:52:50.793451+0000","last_active":"2024-04-26T21:52:52.485015+0000","last_peered":"2024-04-26T21:52:52.485015+0000","last_clean":"2024-04-26T21:52:52.485015+0000","last_became_active":"2024-04-26T21:52:50.791696+0000","last_became_peered":"2024-04-26T21:52:50.791696+0000","last_unstale":"2024-04-26T21:52:52.485015+0000","last_undegraded":"2024-04-26T21:52:52.485015+0000","last_fullsized":"2024-04-26T21:52:52.485015+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:51:43.543500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:52.117016+0000","last_change":"2024-04-26T21:52:50.786077+0000","last_active":"2024-04-26T21:52:52.117016+0000","last_peered":"2024-04-26T21:52:52.117016+0000","last_clean":"2024-04-26T21:52:52.117016+0000","last_became_active":"2024-04-26T21:52:50.785757+0000","last_became_peered":"2024-04-26T21:52:50.785757+0000","last_unstale":"2024-04-26T21:52:52.117016+0000","last_undegraded":"2024-04-26T21:52:52.117016+0000","last_fullsized":"2024-04-26T21:52:52.117016+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:53:18.645179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:52.484791+0000","last_change":"2024-04-26T21:52:50.782752+0000","last_active":"2024-04-26T21:52:52.484791+0000","last_peered":"2024-04-26T21:52:52.484791+0000","last_clean":"2024-04-26T21:52:52.484791+0000","last_became_active":"2024-04-26T21:52:50.777773+0000","last_became_peered":"2024-04-26T21:52:50.777773+0000","last_unstale":"2024-04-26T21:52:52.484791+0000","last_undegraded":"2024-04-26T21:52:52.484791+0000","last_fullsized":"2024-04-26T21:52:52.484791+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:54:39.656756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:52.115676+0000","last_change":"2024-04-26T21:52:50.794580+0000","last_active":"2024-04-26T21:52:52.115676+0000","last_peered":"2024-04-26T21:52:52.115676+0000","last_clean":"2024-04-26T21:52:52.115676+0000","last_became_active":"2024-04-26T21:52:50.792398+0000","last_became_peered":"2024-04-26T21:52:50.792398+0000","last_unstale":"2024-04-26T21:52:52.115676+0000","last_undegraded":"2024-04-26T21:52:52.115676+0000","last_fullsized":"2024-04-26T21:52:52.115676+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:21:38.913644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:51.778289+0000","last_change":"2024-04-26T21:52:50.790087+0000","last_active":"2024-04-26T21:52:51.778289+0000","last_peered":"2024-04-26T21:52:51.778289+0000","last_clean":"2024-04-26T21:52:51.778289+0000","last_became_active":"2024-04-26T21:52:50.788685+0000","last_became_peered":"2024-04-26T21:52:50.788685+0000","last_unstale":"2024-04-26T21:52:51.778289+0000","last_undegraded":"2024-04-26T21:52:51.778289+0000","last_fullsized":"2024-04-26T21:52:51.778289+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:41:25.094794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:52.480828+0000","last_change":"2024-04-26T21:52:50.792166+0000","last_active":"2024-04-26T21:52:52.480828+0000","last_peered":"2024-04-26T21:52:52.480828+0000","last_clean":"2024-04-26T21:52:52.480828+0000","last_became_active":"2024-04-26T21:52:50.790709+0000","last_became_peered":"2024-04-26T21:52:50.790709+0000","last_unstale":"2024-04-26T21:52:52.480828+0000","last_undegraded":"2024-04-26T21:52:52.480828+0000","last_fullsized":"2024-04-26T21:52:52.480828+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:05:48.200573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:52.481689+0000","last_change":"2024-04-26T21:52:50.791161+0000","last_active":"2024-04-26T21:52:52.481689+0000","last_peered":"2024-04-26T21:52:52.481689+0000","last_clean":"2024-04-26T21:52:52.481689+0000","last_became_active":"2024-04-26T21:52:50.790793+0000","last_became_peered":"2024-04-26T21:52:50.790793+0000","last_unstale":"2024-04-26T21:52:52.481689+0000","last_undegraded":"2024-04-26T21:52:52.481689+0000","last_fullsized":"2024-04-26T21:52:52.481689+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:13:53.831932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":15025,"num_objects":97,"num_object_clones":0,"num_object_copies":291,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":97,"num_whiteouts":0,"num_read":34,"num_read_kb":31,"num_write":208,"num_write_kb":72,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":65,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":434176,"data_stored":75034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":182,"ondisk_log_size":182,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":82,"seq":352187318289,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":950272,"data_stored":450388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":10,"up_from":74,"seq":317827579928,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28568,"kb_used_data":1552,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716584,"statfs":{"total":95995035648,"available":95965782016,"internally_reserved":0,"allocated":1589248,"data_stored":1042912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":69,"seq":296352743456,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36288,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708864,"statfs":{"total":95995035648,"available":95957876736,"internally_reserved":0,"allocated":974848,"data_stored":449927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":8,"up_from":61,"seq":261993005096,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36284,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708868,"statfs":{"total":95995035648,"available":95957880832,"internally_reserved":0,"allocated":970752,"data_stored":450305,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299436,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28536,"kb_used_data":1520,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716616,"statfs":{"total":95995035648,"available":95965814784,"internally_reserved":0,"allocated":1556480,"data_stored":1041637,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462965,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32108,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713044,"statfs":{"total":95995035648,"available":95962157056,"internally_reserved":0,"allocated":954368,"data_stored":449527,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574935,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":978944,"data_stored":471452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738462,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36284,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708868,"statfs":{"total":95995035648,"available":95957880832,"internally_reserved":0,"allocated":970752,"data_stored":450871,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411409,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36312,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708840,"statfs":{"total":95995035648,"available":95957852160,"internally_reserved":0,"allocated":999424,"data_stored":472063,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182472,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":954368,"data_stored":449089,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920836,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36272,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708880,"statfs":{"total":95995035648,"available":95957893120,"internally_reserved":0,"allocated":958464,"data_stored":449505,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724602,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36872,"kb_used_data":1536,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708280,"statfs":{"total":95995035648,"available":95957278720,"internally_reserved":0,"allocated":1572864,"data_stored":1041411,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":27262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":23258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":9948,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T21:53:01.959 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T21:53:02.058 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:53:01 smithi072 ceph-mon[28138]: from='client.14790 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:53:02.058 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:53:01 smithi072 ceph-mon[28138]: from='client.14796 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:53:02.058 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:53:01 smithi072 ceph-mon[28138]: mgrmap e17: x(active, since 9m), standbys: z, y 2024-04-26T21:53:02.058 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:53:01 smithi072 ceph-mon[28138]: from='client.? 172.21.15.72:0/4123900921' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:53:02.058 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:53:01 smithi072 ceph-mon[28138]: from='client.? 172.21.15.72:0/1424070977' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:53:02.184 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:02.183+0000 7f6367200640 1 -- 172.21.15.72:0/784306342 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6368103960 msgr2=0x7f6368103de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:02.184 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:02.183+0000 7f6367200640 1 --2- 172.21.15.72:0/784306342 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6368103960 0x7f6368103de0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f6354009900 tx=0x7f635402a410 comp rx=0 tx=0).stop 2024-04-26T21:53:02.184 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:02.183+0000 7f6367200640 1 -- 172.21.15.72:0/784306342 shutdown_connections 2024-04-26T21:53:02.184 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:02.183+0000 7f6367200640 1 --2- 172.21.15.72:0/784306342 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6368104320 0x7f636810abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:02.184 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:02.183+0000 7f6367200640 1 --2- 172.21.15.72:0/784306342 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6368103960 0x7f6368103de0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:02.184 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:02.183+0000 7f6367200640 1 --2- 172.21.15.72:0/784306342 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6368102770 0x7f6368102b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:02.185 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:02.183+0000 7f6367200640 1 -- 172.21.15.72:0/784306342 >> 172.21.15.72:0/784306342 conn(0x7f63680fdcc0 msgr2=0x7f6368100120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:53:02.185 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:02.183+0000 7f6367200640 1 -- 172.21.15.72:0/784306342 shutdown_connections 2024-04-26T21:53:02.185 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:02.183+0000 7f6367200640 1 -- 172.21.15.72:0/784306342 wait complete. 2024-04-26T21:53:02.187 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:02.184+0000 7f6367200640 1 Processor -- start 2024-04-26T21:53:02.187 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:02.184+0000 7f6367200640 1 -- start start 2024-04-26T21:53:02.187 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:02.184+0000 7f6367200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6368102770 0x7f6368198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:02.188 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:02.185+0000 7f6367200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6368103960 0x7f6368198e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:02.188 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:02.185+0000 7f6367200640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6368104320 0x7f636819fee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:02.188 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:02.185+0000 7f6367200640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f636810e210 con 0x7f6368104320 2024-04-26T21:53:02.188 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:02.185+0000 7f6367200640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f636810e090 con 0x7f6368103960 2024-04-26T21:53:02.188 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:02.185+0000 7f6367200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f636810e390 con 0x7f6368102770 2024-04-26T21:53:02.188 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:02.185+0000 7f6366800640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6368104320 0x7f636819fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:02.188 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:02.185+0000 7f6366800640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6368104320 0x7f636819fee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:56022/0 (socket says 172.21.15.72:56022) 2024-04-26T21:53:02.188 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:02.185+0000 7f6366800640 1 -- 172.21.15.72:0/822085370 learned_addr learned my addr 172.21.15.72:0/822085370 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:53:02.188 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:02.185+0000 7f6365e00640 1 --2- 172.21.15.72:0/822085370 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6368102770 0x7f6368198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:02.188 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:02.185+0000 7f6365400640 1 --2- 172.21.15.72:0/822085370 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6368103960 0x7f6368198e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:02.188 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:02.185+0000 7f6366800640 1 -- 172.21.15.72:0/822085370 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6368102770 msgr2=0x7f6368198930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:02.188 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:02.185+0000 7f6366800640 1 --2- 172.21.15.72:0/822085370 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6368102770 0x7f6368198930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:02.188 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:02.185+0000 7f6366800640 1 -- 172.21.15.72:0/822085370 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6368103960 msgr2=0x7f6368198e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:02.189 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:02.185+0000 7f6366800640 1 --2- 172.21.15.72:0/822085370 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6368103960 0x7f6368198e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:02.189 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:02.185+0000 7f6366800640 1 -- 172.21.15.72:0/822085370 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63681a0550 con 0x7f6368104320 2024-04-26T21:53:02.189 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:02.186+0000 7f6366800640 1 --2- 172.21.15.72:0/822085370 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6368104320 0x7f636819fee0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f636000cad0 tx=0x7f636000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:53:02.189 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:02.186+0000 7f635ea00640 1 -- 172.21.15.72:0/822085370 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f636000ae20 con 0x7f6368104320 2024-04-26T21:53:02.189 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:02.186+0000 7f635ea00640 1 -- 172.21.15.72:0/822085370 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6360005020 con 0x7f6368104320 2024-04-26T21:53:02.189 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:02.186+0000 7f635ea00640 1 -- 172.21.15.72:0/822085370 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f636000f6b0 con 0x7f6368104320 2024-04-26T21:53:02.189 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:02.186+0000 7f6367200640 1 -- 172.21.15.72:0/822085370 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63681a0840 con 0x7f6368104320 2024-04-26T21:53:02.189 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:02.186+0000 7f6367200640 1 -- 172.21.15.72:0/822085370 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63681a0de0 con 0x7f6368104320 2024-04-26T21:53:02.189 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:02.187+0000 7f6367200640 1 -- 172.21.15.72:0/822085370 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6368109b60 con 0x7f6368104320 2024-04-26T21:53:02.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:53:01 smithi164 ceph-mon[31916]: from='client.14790 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:53:02.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:53:01 smithi164 ceph-mon[31916]: from='client.14796 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:53:02.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:53:01 smithi164 ceph-mon[31916]: mgrmap e17: x(active, since 9m), standbys: z, y 2024-04-26T21:53:02.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:53:01 smithi164 ceph-mon[31916]: from='client.? 172.21.15.72:0/4123900921' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:53:02.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:53:01 smithi164 ceph-mon[31916]: from='client.? 172.21.15.72:0/1424070977' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:53:02.191 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:02.191+0000 7f635ea00640 1 -- 172.21.15.72:0/822085370 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f63600051d0 con 0x7f6368104320 2024-04-26T21:53:02.192 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:02.192+0000 7f635ea00640 1 --2- 172.21.15.72:0/822085370 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f634c0a3cc0 0x7f634c0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:02.192 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:02.192+0000 7f635ea00640 1 -- 172.21.15.72:0/822085370 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7f63600d0830 con 0x7f6368104320 2024-04-26T21:53:02.192 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:02.192+0000 7f6365e00640 1 --2- 172.21.15.72:0/822085370 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f634c0a3cc0 0x7f634c0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:02.193 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:02.193+0000 7f6365e00640 1 --2- 172.21.15.72:0/822085370 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f634c0a3cc0 0x7f634c0a6180 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f63681037c0 tx=0x7f6358008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:53:02.196 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:02.196+0000 7f635ea00640 1 -- 172.21.15.72:0/822085370 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f6360014030 con 0x7f6368104320 2024-04-26T21:53:02.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:53:01 smithi076 ceph-mon[31823]: from='client.14790 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:53:02.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:53:01 smithi076 ceph-mon[31823]: from='client.14796 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:53:02.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:53:01 smithi076 ceph-mon[31823]: mgrmap e17: x(active, since 9m), standbys: z, y 2024-04-26T21:53:02.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:53:01 smithi076 ceph-mon[31823]: from='client.? 172.21.15.72:0/4123900921' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:53:02.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:53:01 smithi076 ceph-mon[31823]: from='client.? 172.21.15.72:0/1424070977' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:53:02.393 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:02.392+0000 7f6367200640 1 -- 172.21.15.72:0/822085370 --> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6368062d70 con 0x7f634c0a3cc0 2024-04-26T21:53:02.405 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:02.405+0000 7f635ea00640 1 -- 172.21.15.72:0/822085370 <== mgr.14152 v2:172.21.15.72:6800/588150172 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258836 (secure 0 0 0) 0x7f6368062d70 con 0x7f634c0a3cc0 2024-04-26T21:53:02.408 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T21:53:02.409 INFO:teuthology.orchestra.run.smithi072.stderr:dumped all 2024-04-26T21:53:02.411 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:02.409+0000 7f6367200640 1 -- 172.21.15.72:0/822085370 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f634c0a3cc0 msgr2=0x7f634c0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:02.411 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:02.409+0000 7f6367200640 1 --2- 172.21.15.72:0/822085370 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f634c0a3cc0 0x7f634c0a6180 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f63681037c0 tx=0x7f6358008040 comp rx=0 tx=0).stop 2024-04-26T21:53:02.411 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:02.409+0000 7f6367200640 1 -- 172.21.15.72:0/822085370 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6368104320 msgr2=0x7f636819fee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:02.411 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:02.409+0000 7f6367200640 1 --2- 172.21.15.72:0/822085370 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6368104320 0x7f636819fee0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f636000cad0 tx=0x7f636000cf90 comp rx=0 tx=0).stop 2024-04-26T21:53:02.411 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:02.410+0000 7f6367200640 1 -- 172.21.15.72:0/822085370 shutdown_connections 2024-04-26T21:53:02.412 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:02.410+0000 7f6367200640 1 --2- 172.21.15.72:0/822085370 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6368104320 0x7f636819fee0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:02.412 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:02.410+0000 7f6367200640 1 --2- 172.21.15.72:0/822085370 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6368103960 0x7f6368198e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:02.412 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:02.410+0000 7f6367200640 1 --2- 172.21.15.72:0/822085370 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f634c0a3cc0 0x7f634c0a6180 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:02.412 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:02.410+0000 7f6367200640 1 --2- 172.21.15.72:0/822085370 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6368102770 0x7f6368198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:02.412 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:02.410+0000 7f6367200640 1 -- 172.21.15.72:0/822085370 >> 172.21.15.72:0/822085370 conn(0x7f63680fdcc0 msgr2=0x7f6368106d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:53:02.413 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:02.410+0000 7f6367200640 1 -- 172.21.15.72:0/822085370 shutdown_connections 2024-04-26T21:53:02.413 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:02.410+0000 7f6367200640 1 -- 172.21.15.72:0/822085370 wait complete. 2024-04-26T21:53:02.428 INFO:teuthology.orchestra.run.smithi072.stdout:{"pg_ready":true,"pg_map":{"version":373,"stamp":"2024-04-26T21:53:01.775330+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":605393,"num_objects":99,"num_object_clones":0,"num_object_copies":297,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":0,"num_read":160,"num_read_kb":140,"num_write":389,"num_write_kb":3010,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":65,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":319,"ondisk_log_size":319,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":391388,"kb_used_data":13116,"kb_used_omap":18,"kb_used_meta":378157,"kb_avail":1124550436,"statfs":{"total":1151940427776,"available":1151539646464,"internally_reserved":0,"allocated":13430784,"data_stored":7219087,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":387233082},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":20,"apply_latency_ms":20,"commit_latency_ns":20000000,"apply_latency_ns":20000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":15025,"num_objects":97,"num_object_clones":0,"num_object_copies":291,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":97,"num_whiteouts":0,"num_read":34,"num_read_kb":31,"num_write":208,"num_write_kb":72,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":65,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.023886"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:52.483427+0000","last_change":"2024-04-26T21:52:50.787765+0000","last_active":"2024-04-26T21:52:52.483427+0000","last_peered":"2024-04-26T21:52:52.483427+0000","last_clean":"2024-04-26T21:52:52.483427+0000","last_became_active":"2024-04-26T21:52:50.786969+0000","last_became_peered":"2024-04-26T21:52:50.786969+0000","last_unstale":"2024-04-26T21:52:52.483427+0000","last_undegraded":"2024-04-26T21:52:52.483427+0000","last_fullsized":"2024-04-26T21:52:52.483427+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:31:50.659210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:51.755499+0000","last_change":"2024-04-26T21:52:50.779735+0000","last_active":"2024-04-26T21:52:51.755499+0000","last_peered":"2024-04-26T21:52:51.755499+0000","last_clean":"2024-04-26T21:52:51.755499+0000","last_became_active":"2024-04-26T21:52:50.779358+0000","last_became_peered":"2024-04-26T21:52:50.779358+0000","last_unstale":"2024-04-26T21:52:51.755499+0000","last_undegraded":"2024-04-26T21:52:51.755499+0000","last_fullsized":"2024-04-26T21:52:51.755499+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:58:45.631365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:51.754519+0000","last_change":"2024-04-26T21:52:50.788220+0000","last_active":"2024-04-26T21:52:51.754519+0000","last_peered":"2024-04-26T21:52:51.754519+0000","last_clean":"2024-04-26T21:52:51.754519+0000","last_became_active":"2024-04-26T21:52:50.787282+0000","last_became_peered":"2024-04-26T21:52:50.787282+0000","last_unstale":"2024-04-26T21:52:51.754519+0000","last_undegraded":"2024-04-26T21:52:51.754519+0000","last_fullsized":"2024-04-26T21:52:51.754519+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:12:28.577893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:51.756281+0000","last_change":"2024-04-26T21:52:50.795367+0000","last_active":"2024-04-26T21:52:51.756281+0000","last_peered":"2024-04-26T21:52:51.756281+0000","last_clean":"2024-04-26T21:52:51.756281+0000","last_became_active":"2024-04-26T21:52:50.793602+0000","last_became_peered":"2024-04-26T21:52:50.793602+0000","last_unstale":"2024-04-26T21:52:51.756281+0000","last_undegraded":"2024-04-26T21:52:51.756281+0000","last_fullsized":"2024-04-26T21:52:51.756281+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:37:23.356792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:51.756020+0000","last_change":"2024-04-26T21:52:50.787333+0000","last_active":"2024-04-26T21:52:51.756020+0000","last_peered":"2024-04-26T21:52:51.756020+0000","last_clean":"2024-04-26T21:52:51.756020+0000","last_became_active":"2024-04-26T21:52:50.785319+0000","last_became_peered":"2024-04-26T21:52:50.785319+0000","last_unstale":"2024-04-26T21:52:51.756020+0000","last_undegraded":"2024-04-26T21:52:51.756020+0000","last_fullsized":"2024-04-26T21:52:51.756020+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:06:29.154825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:52.481166+0000","last_change":"2024-04-26T21:52:50.780829+0000","last_active":"2024-04-26T21:52:52.481166+0000","last_peered":"2024-04-26T21:52:52.481166+0000","last_clean":"2024-04-26T21:52:52.481166+0000","last_became_active":"2024-04-26T21:52:50.780439+0000","last_became_peered":"2024-04-26T21:52:50.780439+0000","last_unstale":"2024-04-26T21:52:52.481166+0000","last_undegraded":"2024-04-26T21:52:52.481166+0000","last_fullsized":"2024-04-26T21:52:52.481166+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:20:14.533853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:51.873603+0000","last_change":"2024-04-26T21:52:50.787679+0000","last_active":"2024-04-26T21:52:51.873603+0000","last_peered":"2024-04-26T21:52:51.873603+0000","last_clean":"2024-04-26T21:52:51.873603+0000","last_became_active":"2024-04-26T21:52:50.787277+0000","last_became_peered":"2024-04-26T21:52:50.787277+0000","last_unstale":"2024-04-26T21:52:51.873603+0000","last_undegraded":"2024-04-26T21:52:51.873603+0000","last_fullsized":"2024-04-26T21:52:51.873603+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:29:02.131503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:52.115260+0000","last_change":"2024-04-26T21:52:50.776468+0000","last_active":"2024-04-26T21:52:52.115260+0000","last_peered":"2024-04-26T21:52:52.115260+0000","last_clean":"2024-04-26T21:52:52.115260+0000","last_became_active":"2024-04-26T21:52:50.774710+0000","last_became_peered":"2024-04-26T21:52:50.774710+0000","last_unstale":"2024-04-26T21:52:52.115260+0000","last_undegraded":"2024-04-26T21:52:52.115260+0000","last_fullsized":"2024-04-26T21:52:52.115260+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:55:21.540496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:51.779245+0000","last_change":"2024-04-26T21:52:50.779701+0000","last_active":"2024-04-26T21:52:51.779245+0000","last_peered":"2024-04-26T21:52:51.779245+0000","last_clean":"2024-04-26T21:52:51.779245+0000","last_became_active":"2024-04-26T21:52:50.779331+0000","last_became_peered":"2024-04-26T21:52:50.779331+0000","last_unstale":"2024-04-26T21:52:51.779245+0000","last_undegraded":"2024-04-26T21:52:51.779245+0000","last_fullsized":"2024-04-26T21:52:51.779245+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:38:15.569806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:52.481498+0000","last_change":"2024-04-26T21:52:50.790081+0000","last_active":"2024-04-26T21:52:52.481498+0000","last_peered":"2024-04-26T21:52:52.481498+0000","last_clean":"2024-04-26T21:52:52.481498+0000","last_became_active":"2024-04-26T21:52:50.786575+0000","last_became_peered":"2024-04-26T21:52:50.786575+0000","last_unstale":"2024-04-26T21:52:52.481498+0000","last_undegraded":"2024-04-26T21:52:52.481498+0000","last_fullsized":"2024-04-26T21:52:52.481498+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:56:47.467128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:51.874528+0000","last_change":"2024-04-26T21:52:50.791983+0000","last_active":"2024-04-26T21:52:51.874528+0000","last_peered":"2024-04-26T21:52:51.874528+0000","last_clean":"2024-04-26T21:52:51.874528+0000","last_became_active":"2024-04-26T21:52:50.788312+0000","last_became_peered":"2024-04-26T21:52:50.788312+0000","last_unstale":"2024-04-26T21:52:51.874528+0000","last_undegraded":"2024-04-26T21:52:51.874528+0000","last_fullsized":"2024-04-26T21:52:51.874528+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:47:47.410267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:51.755892+0000","last_change":"2024-04-26T21:52:50.794237+0000","last_active":"2024-04-26T21:52:51.755892+0000","last_peered":"2024-04-26T21:52:51.755892+0000","last_clean":"2024-04-26T21:52:51.755892+0000","last_became_active":"2024-04-26T21:52:50.792260+0000","last_became_peered":"2024-04-26T21:52:50.792260+0000","last_unstale":"2024-04-26T21:52:51.755892+0000","last_undegraded":"2024-04-26T21:52:51.755892+0000","last_fullsized":"2024-04-26T21:52:51.755892+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:36:05.467667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:52.480858+0000","last_change":"2024-04-26T21:52:50.777687+0000","last_active":"2024-04-26T21:52:52.480858+0000","last_peered":"2024-04-26T21:52:52.480858+0000","last_clean":"2024-04-26T21:52:52.480858+0000","last_became_active":"2024-04-26T21:52:50.777200+0000","last_became_peered":"2024-04-26T21:52:50.777200+0000","last_unstale":"2024-04-26T21:52:52.480858+0000","last_undegraded":"2024-04-26T21:52:52.480858+0000","last_fullsized":"2024-04-26T21:52:52.480858+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:40:45.167144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:51.755105+0000","last_change":"2024-04-26T21:52:50.787008+0000","last_active":"2024-04-26T21:52:51.755105+0000","last_peered":"2024-04-26T21:52:51.755105+0000","last_clean":"2024-04-26T21:52:51.755105+0000","last_became_active":"2024-04-26T21:52:50.785656+0000","last_became_peered":"2024-04-26T21:52:50.785656+0000","last_unstale":"2024-04-26T21:52:51.755105+0000","last_undegraded":"2024-04-26T21:52:51.755105+0000","last_fullsized":"2024-04-26T21:52:51.755105+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:20:12.441281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:51.754196+0000","last_change":"2024-04-26T21:52:50.796537+0000","last_active":"2024-04-26T21:52:51.754196+0000","last_peered":"2024-04-26T21:52:51.754196+0000","last_clean":"2024-04-26T21:52:51.754196+0000","last_became_active":"2024-04-26T21:52:50.796229+0000","last_became_peered":"2024-04-26T21:52:50.796229+0000","last_unstale":"2024-04-26T21:52:51.754196+0000","last_undegraded":"2024-04-26T21:52:51.754196+0000","last_fullsized":"2024-04-26T21:52:51.754196+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:38:46.404444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:51.777837+0000","last_change":"2024-04-26T21:52:50.789211+0000","last_active":"2024-04-26T21:52:51.777837+0000","last_peered":"2024-04-26T21:52:51.777837+0000","last_clean":"2024-04-26T21:52:51.777837+0000","last_became_active":"2024-04-26T21:52:50.788889+0000","last_became_peered":"2024-04-26T21:52:50.788889+0000","last_unstale":"2024-04-26T21:52:51.777837+0000","last_undegraded":"2024-04-26T21:52:51.777837+0000","last_fullsized":"2024-04-26T21:52:51.777837+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:25:48.497840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:52.483655+0000","last_change":"2024-04-26T21:52:50.788035+0000","last_active":"2024-04-26T21:52:52.483655+0000","last_peered":"2024-04-26T21:52:52.483655+0000","last_clean":"2024-04-26T21:52:52.483655+0000","last_became_active":"2024-04-26T21:52:50.787453+0000","last_became_peered":"2024-04-26T21:52:50.787453+0000","last_unstale":"2024-04-26T21:52:52.483655+0000","last_undegraded":"2024-04-26T21:52:52.483655+0000","last_fullsized":"2024-04-26T21:52:52.483655+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:40:29.718537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:51.871413+0000","last_change":"2024-04-26T21:52:50.778228+0000","last_active":"2024-04-26T21:52:51.871413+0000","last_peered":"2024-04-26T21:52:51.871413+0000","last_clean":"2024-04-26T21:52:51.871413+0000","last_became_active":"2024-04-26T21:52:50.777934+0000","last_became_peered":"2024-04-26T21:52:50.777934+0000","last_unstale":"2024-04-26T21:52:51.871413+0000","last_undegraded":"2024-04-26T21:52:51.871413+0000","last_fullsized":"2024-04-26T21:52:51.871413+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:33:34.598325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:52.484945+0000","last_change":"2024-04-26T21:52:50.790795+0000","last_active":"2024-04-26T21:52:52.484945+0000","last_peered":"2024-04-26T21:52:52.484945+0000","last_clean":"2024-04-26T21:52:52.484945+0000","last_became_active":"2024-04-26T21:52:50.788254+0000","last_became_peered":"2024-04-26T21:52:50.788254+0000","last_unstale":"2024-04-26T21:52:52.484945+0000","last_undegraded":"2024-04-26T21:52:52.484945+0000","last_fullsized":"2024-04-26T21:52:52.484945+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:20:28.235137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"87'13","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:58.529618+0000","last_change":"2024-04-26T21:52:50.105513+0000","last_active":"2024-04-26T21:52:58.529618+0000","last_peered":"2024-04-26T21:52:58.529618+0000","last_clean":"2024-04-26T21:52:58.529618+0000","last_became_active":"2024-04-26T21:52:50.105131+0000","last_became_peered":"2024-04-26T21:52:50.105131+0000","last_unstale":"2024-04-26T21:52:58.529618+0000","last_undegraded":"2024-04-26T21:52:58.529618+0000","last_fullsized":"2024-04-26T21:52:58.529618+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:36:58.085803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:52.116654+0000","last_change":"2024-04-26T21:52:50.795025+0000","last_active":"2024-04-26T21:52:52.116654+0000","last_peered":"2024-04-26T21:52:52.116654+0000","last_clean":"2024-04-26T21:52:52.116654+0000","last_became_active":"2024-04-26T21:52:50.793608+0000","last_became_peered":"2024-04-26T21:52:50.793608+0000","last_unstale":"2024-04-26T21:52:52.116654+0000","last_undegraded":"2024-04-26T21:52:52.116654+0000","last_fullsized":"2024-04-26T21:52:52.116654+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:49:47.764848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"87'2","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:52.116883+0000","last_change":"2024-04-26T21:52:50.109197+0000","last_active":"2024-04-26T21:52:52.116883+0000","last_peered":"2024-04-26T21:52:52.116883+0000","last_clean":"2024-04-26T21:52:52.116883+0000","last_became_active":"2024-04-26T21:52:50.108835+0000","last_became_peered":"2024-04-26T21:52:50.108835+0000","last_unstale":"2024-04-26T21:52:52.116883+0000","last_undegraded":"2024-04-26T21:52:52.116883+0000","last_fullsized":"2024-04-26T21:52:52.116883+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:45:09.777747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:51.755270+0000","last_change":"2024-04-26T21:52:50.783094+0000","last_active":"2024-04-26T21:52:51.755270+0000","last_peered":"2024-04-26T21:52:51.755270+0000","last_clean":"2024-04-26T21:52:51.755270+0000","last_became_active":"2024-04-26T21:52:50.782468+0000","last_became_peered":"2024-04-26T21:52:50.782468+0000","last_unstale":"2024-04-26T21:52:51.755270+0000","last_undegraded":"2024-04-26T21:52:51.755270+0000","last_fullsized":"2024-04-26T21:52:51.755270+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:53:12.977054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"87'12","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:58.522410+0000","last_change":"2024-04-26T21:52:50.104824+0000","last_active":"2024-04-26T21:52:58.522410+0000","last_peered":"2024-04-26T21:52:58.522410+0000","last_clean":"2024-04-26T21:52:58.522410+0000","last_became_active":"2024-04-26T21:52:50.104514+0000","last_became_peered":"2024-04-26T21:52:50.104514+0000","last_unstale":"2024-04-26T21:52:58.522410+0000","last_undegraded":"2024-04-26T21:52:58.522410+0000","last_fullsized":"2024-04-26T21:52:58.522410+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:45:31.541940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:51.778523+0000","last_change":"2024-04-26T21:52:50.779695+0000","last_active":"2024-04-26T21:52:51.778523+0000","last_peered":"2024-04-26T21:52:51.778523+0000","last_clean":"2024-04-26T21:52:51.778523+0000","last_became_active":"2024-04-26T21:52:50.779234+0000","last_became_peered":"2024-04-26T21:52:50.779234+0000","last_unstale":"2024-04-26T21:52:51.778523+0000","last_undegraded":"2024-04-26T21:52:51.778523+0000","last_fullsized":"2024-04-26T21:52:51.778523+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:33:07.316543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"87'5","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:57.595629+0000","last_change":"2024-04-26T21:52:50.107403+0000","last_active":"2024-04-26T21:52:57.595629+0000","last_peered":"2024-04-26T21:52:57.595629+0000","last_clean":"2024-04-26T21:52:57.595629+0000","last_became_active":"2024-04-26T21:52:50.107126+0000","last_became_peered":"2024-04-26T21:52:50.107126+0000","last_unstale":"2024-04-26T21:52:57.595629+0000","last_undegraded":"2024-04-26T21:52:57.595629+0000","last_fullsized":"2024-04-26T21:52:57.595629+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:52:34.985448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:52.486606+0000","last_change":"2024-04-26T21:52:50.783674+0000","last_active":"2024-04-26T21:52:52.486606+0000","last_peered":"2024-04-26T21:52:52.486606+0000","last_clean":"2024-04-26T21:52:52.486606+0000","last_became_active":"2024-04-26T21:52:50.782283+0000","last_became_peered":"2024-04-26T21:52:50.782283+0000","last_unstale":"2024-04-26T21:52:52.486606+0000","last_undegraded":"2024-04-26T21:52:52.486606+0000","last_fullsized":"2024-04-26T21:52:52.486606+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:41:03.144601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"87'5","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:54.556735+0000","last_change":"2024-04-26T21:52:50.105204+0000","last_active":"2024-04-26T21:52:54.556735+0000","last_peered":"2024-04-26T21:52:54.556735+0000","last_clean":"2024-04-26T21:52:54.556735+0000","last_became_active":"2024-04-26T21:52:50.103980+0000","last_became_peered":"2024-04-26T21:52:50.103980+0000","last_unstale":"2024-04-26T21:52:54.556735+0000","last_undegraded":"2024-04-26T21:52:54.556735+0000","last_fullsized":"2024-04-26T21:52:54.556735+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:28:20.741673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:51.756064+0000","last_change":"2024-04-26T21:52:50.779541+0000","last_active":"2024-04-26T21:52:51.756064+0000","last_peered":"2024-04-26T21:52:51.756064+0000","last_clean":"2024-04-26T21:52:51.756064+0000","last_became_active":"2024-04-26T21:52:50.777800+0000","last_became_peered":"2024-04-26T21:52:50.777800+0000","last_unstale":"2024-04-26T21:52:51.756064+0000","last_undegraded":"2024-04-26T21:52:51.756064+0000","last_fullsized":"2024-04-26T21:52:51.756064+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:40:52.680799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"87'7","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:56.575049+0000","last_change":"2024-04-26T21:52:50.108552+0000","last_active":"2024-04-26T21:52:56.575049+0000","last_peered":"2024-04-26T21:52:56.575049+0000","last_clean":"2024-04-26T21:52:56.575049+0000","last_became_active":"2024-04-26T21:52:50.108071+0000","last_became_peered":"2024-04-26T21:52:50.108071+0000","last_unstale":"2024-04-26T21:52:56.575049+0000","last_undegraded":"2024-04-26T21:52:56.575049+0000","last_fullsized":"2024-04-26T21:52:56.575049+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:05:42.990950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:52.486390+0000","last_change":"2024-04-26T21:52:50.783788+0000","last_active":"2024-04-26T21:52:52.486390+0000","last_peered":"2024-04-26T21:52:52.486390+0000","last_clean":"2024-04-26T21:52:52.486390+0000","last_became_active":"2024-04-26T21:52:50.776645+0000","last_became_peered":"2024-04-26T21:52:50.776645+0000","last_unstale":"2024-04-26T21:52:52.486390+0000","last_undegraded":"2024-04-26T21:52:52.486390+0000","last_fullsized":"2024-04-26T21:52:52.486390+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:29:12.058885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"87'3","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:55.560252+0000","last_change":"2024-04-26T21:52:50.107564+0000","last_active":"2024-04-26T21:52:55.560252+0000","last_peered":"2024-04-26T21:52:55.560252+0000","last_clean":"2024-04-26T21:52:55.560252+0000","last_became_active":"2024-04-26T21:52:50.106653+0000","last_became_peered":"2024-04-26T21:52:50.106653+0000","last_unstale":"2024-04-26T21:52:55.560252+0000","last_undegraded":"2024-04-26T21:52:55.560252+0000","last_fullsized":"2024-04-26T21:52:55.560252+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:05:36.594729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:52.115266+0000","last_change":"2024-04-26T21:52:50.780279+0000","last_active":"2024-04-26T21:52:52.115266+0000","last_peered":"2024-04-26T21:52:52.115266+0000","last_clean":"2024-04-26T21:52:52.115266+0000","last_became_active":"2024-04-26T21:52:50.779950+0000","last_became_peered":"2024-04-26T21:52:50.779950+0000","last_unstale":"2024-04-26T21:52:52.115266+0000","last_undegraded":"2024-04-26T21:52:52.115266+0000","last_fullsized":"2024-04-26T21:52:52.115266+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:11:36.282319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"87'3","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:56.576436+0000","last_change":"2024-04-26T21:52:50.106269+0000","last_active":"2024-04-26T21:52:56.576436+0000","last_peered":"2024-04-26T21:52:56.576436+0000","last_clean":"2024-04-26T21:52:56.576436+0000","last_became_active":"2024-04-26T21:52:50.105948+0000","last_became_peered":"2024-04-26T21:52:50.105948+0000","last_unstale":"2024-04-26T21:52:56.576436+0000","last_undegraded":"2024-04-26T21:52:56.576436+0000","last_fullsized":"2024-04-26T21:52:56.576436+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:39:31.024362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:51.870542+0000","last_change":"2024-04-26T21:52:50.778938+0000","last_active":"2024-04-26T21:52:51.870542+0000","last_peered":"2024-04-26T21:52:51.870542+0000","last_clean":"2024-04-26T21:52:51.870542+0000","last_became_active":"2024-04-26T21:52:50.778590+0000","last_became_peered":"2024-04-26T21:52:50.778590+0000","last_unstale":"2024-04-26T21:52:51.870542+0000","last_undegraded":"2024-04-26T21:52:51.870542+0000","last_fullsized":"2024-04-26T21:52:51.870542+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:03:10.325793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"87'4","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:58.524883+0000","last_change":"2024-04-26T21:52:50.107243+0000","last_active":"2024-04-26T21:52:58.524883+0000","last_peered":"2024-04-26T21:52:58.524883+0000","last_clean":"2024-04-26T21:52:58.524883+0000","last_became_active":"2024-04-26T21:52:50.106729+0000","last_became_peered":"2024-04-26T21:52:50.106729+0000","last_unstale":"2024-04-26T21:52:58.524883+0000","last_undegraded":"2024-04-26T21:52:58.524883+0000","last_fullsized":"2024-04-26T21:52:58.524883+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:28:11.197997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:51.779017+0000","last_change":"2024-04-26T21:52:50.776906+0000","last_active":"2024-04-26T21:52:51.779017+0000","last_peered":"2024-04-26T21:52:51.779017+0000","last_clean":"2024-04-26T21:52:51.779017+0000","last_became_active":"2024-04-26T21:52:50.776567+0000","last_became_peered":"2024-04-26T21:52:50.776567+0000","last_unstale":"2024-04-26T21:52:51.779017+0000","last_undegraded":"2024-04-26T21:52:51.779017+0000","last_fullsized":"2024-04-26T21:52:51.779017+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:51:39.512572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"87'4","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:57.596522+0000","last_change":"2024-04-26T21:52:50.109792+0000","last_active":"2024-04-26T21:52:57.596522+0000","last_peered":"2024-04-26T21:52:57.596522+0000","last_clean":"2024-04-26T21:52:57.596522+0000","last_became_active":"2024-04-26T21:52:50.106662+0000","last_became_peered":"2024-04-26T21:52:50.106662+0000","last_unstale":"2024-04-26T21:52:57.596522+0000","last_undegraded":"2024-04-26T21:52:57.596522+0000","last_fullsized":"2024-04-26T21:52:57.596522+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:41:27.390780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:51.754737+0000","last_change":"2024-04-26T21:52:50.782798+0000","last_active":"2024-04-26T21:52:51.754737+0000","last_peered":"2024-04-26T21:52:51.754737+0000","last_clean":"2024-04-26T21:52:51.754737+0000","last_became_active":"2024-04-26T21:52:50.781975+0000","last_became_peered":"2024-04-26T21:52:50.781975+0000","last_unstale":"2024-04-26T21:52:51.754737+0000","last_undegraded":"2024-04-26T21:52:51.754737+0000","last_fullsized":"2024-04-26T21:52:51.754737+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:13:07.261095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"87'5","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:54.549751+0000","last_change":"2024-04-26T21:52:50.099296+0000","last_active":"2024-04-26T21:52:54.549751+0000","last_peered":"2024-04-26T21:52:54.549751+0000","last_clean":"2024-04-26T21:52:54.549751+0000","last_became_active":"2024-04-26T21:52:50.098940+0000","last_became_peered":"2024-04-26T21:52:50.098940+0000","last_unstale":"2024-04-26T21:52:54.549751+0000","last_undegraded":"2024-04-26T21:52:54.549751+0000","last_fullsized":"2024-04-26T21:52:54.549751+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:49:15.765474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:51.778734+0000","last_change":"2024-04-26T21:52:50.783341+0000","last_active":"2024-04-26T21:52:51.778734+0000","last_peered":"2024-04-26T21:52:51.778734+0000","last_clean":"2024-04-26T21:52:51.778734+0000","last_became_active":"2024-04-26T21:52:50.783025+0000","last_became_peered":"2024-04-26T21:52:50.783025+0000","last_unstale":"2024-04-26T21:52:51.778734+0000","last_undegraded":"2024-04-26T21:52:51.778734+0000","last_fullsized":"2024-04-26T21:52:51.778734+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:54:06.070550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"87'13","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:58.533229+0000","last_change":"2024-04-26T21:52:50.106617+0000","last_active":"2024-04-26T21:52:58.533229+0000","last_peered":"2024-04-26T21:52:58.533229+0000","last_clean":"2024-04-26T21:52:58.533229+0000","last_became_active":"2024-04-26T21:52:50.106161+0000","last_became_peered":"2024-04-26T21:52:50.106161+0000","last_unstale":"2024-04-26T21:52:58.533229+0000","last_undegraded":"2024-04-26T21:52:58.533229+0000","last_fullsized":"2024-04-26T21:52:58.533229+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:52:07.977749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8212,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":15,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:51.757241+0000","last_change":"2024-04-26T21:52:50.794567+0000","last_active":"2024-04-26T21:52:51.757241+0000","last_peered":"2024-04-26T21:52:51.757241+0000","last_clean":"2024-04-26T21:52:51.757241+0000","last_became_active":"2024-04-26T21:52:50.794245+0000","last_became_peered":"2024-04-26T21:52:50.794245+0000","last_unstale":"2024-04-26T21:52:51.757241+0000","last_undegraded":"2024-04-26T21:52:51.757241+0000","last_fullsized":"2024-04-26T21:52:51.757241+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:58:11.501104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:58.522406+0000","last_change":"2024-04-26T21:52:50.102156+0000","last_active":"2024-04-26T21:52:58.522406+0000","last_peered":"2024-04-26T21:52:58.522406+0000","last_clean":"2024-04-26T21:52:58.522406+0000","last_became_active":"2024-04-26T21:52:50.101721+0000","last_became_peered":"2024-04-26T21:52:50.101721+0000","last_unstale":"2024-04-26T21:52:58.522406+0000","last_undegraded":"2024-04-26T21:52:58.522406+0000","last_fullsized":"2024-04-26T21:52:58.522406+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:48:42.081874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:52.480661+0000","last_change":"2024-04-26T21:52:50.789199+0000","last_active":"2024-04-26T21:52:52.480661+0000","last_peered":"2024-04-26T21:52:52.480661+0000","last_clean":"2024-04-26T21:52:52.480661+0000","last_became_active":"2024-04-26T21:52:50.788868+0000","last_became_peered":"2024-04-26T21:52:50.788868+0000","last_unstale":"2024-04-26T21:52:52.480661+0000","last_undegraded":"2024-04-26T21:52:52.480661+0000","last_fullsized":"2024-04-26T21:52:52.480661+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:29:55.755564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"87'5","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:56.579192+0000","last_change":"2024-04-26T21:52:50.104920+0000","last_active":"2024-04-26T21:52:56.579192+0000","last_peered":"2024-04-26T21:52:56.579192+0000","last_clean":"2024-04-26T21:52:56.579192+0000","last_became_active":"2024-04-26T21:52:50.104579+0000","last_became_peered":"2024-04-26T21:52:50.104579+0000","last_unstale":"2024-04-26T21:52:56.579192+0000","last_undegraded":"2024-04-26T21:52:56.579192+0000","last_fullsized":"2024-04-26T21:52:56.579192+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:09:35.760918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:51.874565+0000","last_change":"2024-04-26T21:52:50.787016+0000","last_active":"2024-04-26T21:52:51.874565+0000","last_peered":"2024-04-26T21:52:51.874565+0000","last_clean":"2024-04-26T21:52:51.874565+0000","last_became_active":"2024-04-26T21:52:50.778439+0000","last_became_peered":"2024-04-26T21:52:50.778439+0000","last_unstale":"2024-04-26T21:52:51.874565+0000","last_undegraded":"2024-04-26T21:52:51.874565+0000","last_fullsized":"2024-04-26T21:52:51.874565+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:13:35.735480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"87'5","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:58.526816+0000","last_change":"2024-04-26T21:52:50.110175+0000","last_active":"2024-04-26T21:52:58.526816+0000","last_peered":"2024-04-26T21:52:58.526816+0000","last_clean":"2024-04-26T21:52:58.526816+0000","last_became_active":"2024-04-26T21:52:50.109779+0000","last_became_peered":"2024-04-26T21:52:50.109779+0000","last_unstale":"2024-04-26T21:52:58.526816+0000","last_undegraded":"2024-04-26T21:52:58.526816+0000","last_fullsized":"2024-04-26T21:52:58.526816+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:06:46.756327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:51.873835+0000","last_change":"2024-04-26T21:52:50.784199+0000","last_active":"2024-04-26T21:52:51.873835+0000","last_peered":"2024-04-26T21:52:51.873835+0000","last_clean":"2024-04-26T21:52:51.873835+0000","last_became_active":"2024-04-26T21:52:50.783820+0000","last_became_peered":"2024-04-26T21:52:50.783820+0000","last_unstale":"2024-04-26T21:52:51.873835+0000","last_undegraded":"2024-04-26T21:52:51.873835+0000","last_fullsized":"2024-04-26T21:52:51.873835+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:45:24.776722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"87'17","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:57.598650+0000","last_change":"2024-04-26T21:52:50.095685+0000","last_active":"2024-04-26T21:52:57.598650+0000","last_peered":"2024-04-26T21:52:57.598650+0000","last_clean":"2024-04-26T21:52:57.598650+0000","last_became_active":"2024-04-26T21:52:50.095298+0000","last_became_peered":"2024-04-26T21:52:50.095298+0000","last_unstale":"2024-04-26T21:52:57.598650+0000","last_undegraded":"2024-04-26T21:52:57.598650+0000","last_fullsized":"2024-04-26T21:52:57.598650+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:47:26.061531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:52.485286+0000","last_change":"2024-04-26T21:52:50.786607+0000","last_active":"2024-04-26T21:52:52.485286+0000","last_peered":"2024-04-26T21:52:52.485286+0000","last_clean":"2024-04-26T21:52:52.485286+0000","last_became_active":"2024-04-26T21:52:50.786261+0000","last_became_peered":"2024-04-26T21:52:50.786261+0000","last_unstale":"2024-04-26T21:52:52.485286+0000","last_undegraded":"2024-04-26T21:52:52.485286+0000","last_fullsized":"2024-04-26T21:52:52.485286+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:08:45.902260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"87'2","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:57.550958+0000","last_change":"2024-04-26T21:52:50.095782+0000","last_active":"2024-04-26T21:52:57.550958+0000","last_peered":"2024-04-26T21:52:57.550958+0000","last_clean":"2024-04-26T21:52:57.550958+0000","last_became_active":"2024-04-26T21:52:50.093983+0000","last_became_peered":"2024-04-26T21:52:50.093983+0000","last_unstale":"2024-04-26T21:52:57.550958+0000","last_undegraded":"2024-04-26T21:52:57.550958+0000","last_fullsized":"2024-04-26T21:52:57.550958+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:18:40.103099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:52.482451+0000","last_change":"2024-04-26T21:52:50.791969+0000","last_active":"2024-04-26T21:52:52.482451+0000","last_peered":"2024-04-26T21:52:52.482451+0000","last_clean":"2024-04-26T21:52:52.482451+0000","last_became_active":"2024-04-26T21:52:50.790441+0000","last_became_peered":"2024-04-26T21:52:50.790441+0000","last_unstale":"2024-04-26T21:52:52.482451+0000","last_undegraded":"2024-04-26T21:52:52.482451+0000","last_fullsized":"2024-04-26T21:52:52.482451+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:11:04.964439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"87'1","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:51.870987+0000","last_change":"2024-04-26T21:52:50.101807+0000","last_active":"2024-04-26T21:52:51.870987+0000","last_peered":"2024-04-26T21:52:51.870987+0000","last_clean":"2024-04-26T21:52:51.870987+0000","last_became_active":"2024-04-26T21:52:50.101329+0000","last_became_peered":"2024-04-26T21:52:50.101329+0000","last_unstale":"2024-04-26T21:52:51.870987+0000","last_undegraded":"2024-04-26T21:52:51.870987+0000","last_fullsized":"2024-04-26T21:52:51.870987+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:54:47.788172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:52.481361+0000","last_change":"2024-04-26T21:52:50.780872+0000","last_active":"2024-04-26T21:52:52.481361+0000","last_peered":"2024-04-26T21:52:52.481361+0000","last_clean":"2024-04-26T21:52:52.481361+0000","last_became_active":"2024-04-26T21:52:50.778757+0000","last_became_peered":"2024-04-26T21:52:50.778757+0000","last_unstale":"2024-04-26T21:52:52.481361+0000","last_undegraded":"2024-04-26T21:52:52.481361+0000","last_fullsized":"2024-04-26T21:52:52.481361+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:07:21.595091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"87'5","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:57.600350+0000","last_change":"2024-04-26T21:52:50.106886+0000","last_active":"2024-04-26T21:52:57.600350+0000","last_peered":"2024-04-26T21:52:57.600350+0000","last_clean":"2024-04-26T21:52:57.600350+0000","last_became_active":"2024-04-26T21:52:50.106150+0000","last_became_peered":"2024-04-26T21:52:50.106150+0000","last_unstale":"2024-04-26T21:52:57.600350+0000","last_undegraded":"2024-04-26T21:52:57.600350+0000","last_fullsized":"2024-04-26T21:52:57.600350+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:20:54.920416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:52.481461+0000","last_change":"2024-04-26T21:52:50.792037+0000","last_active":"2024-04-26T21:52:52.481461+0000","last_peered":"2024-04-26T21:52:52.481461+0000","last_clean":"2024-04-26T21:52:52.481461+0000","last_became_active":"2024-04-26T21:52:50.791684+0000","last_became_peered":"2024-04-26T21:52:50.791684+0000","last_unstale":"2024-04-26T21:52:52.481461+0000","last_undegraded":"2024-04-26T21:52:52.481461+0000","last_fullsized":"2024-04-26T21:52:52.481461+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:11:00.489287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:52.485049+0000","last_change":"2024-04-26T21:52:50.785257+0000","last_active":"2024-04-26T21:52:52.485049+0000","last_peered":"2024-04-26T21:52:52.485049+0000","last_clean":"2024-04-26T21:52:52.485049+0000","last_became_active":"2024-04-26T21:52:50.784906+0000","last_became_peered":"2024-04-26T21:52:50.784906+0000","last_unstale":"2024-04-26T21:52:52.485049+0000","last_undegraded":"2024-04-26T21:52:52.485049+0000","last_fullsized":"2024-04-26T21:52:52.485049+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:04:50.921663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"87'5","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:58.522577+0000","last_change":"2024-04-26T21:52:50.102425+0000","last_active":"2024-04-26T21:52:58.522577+0000","last_peered":"2024-04-26T21:52:58.522577+0000","last_clean":"2024-04-26T21:52:58.522577+0000","last_became_active":"2024-04-26T21:52:50.101703+0000","last_became_peered":"2024-04-26T21:52:50.101703+0000","last_unstale":"2024-04-26T21:52:58.522577+0000","last_undegraded":"2024-04-26T21:52:58.522577+0000","last_fullsized":"2024-04-26T21:52:58.522577+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:13:39.910519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:52.116006+0000","last_change":"2024-04-26T21:52:50.775036+0000","last_active":"2024-04-26T21:52:52.116006+0000","last_peered":"2024-04-26T21:52:52.116006+0000","last_clean":"2024-04-26T21:52:52.116006+0000","last_became_active":"2024-04-26T21:52:50.773347+0000","last_became_peered":"2024-04-26T21:52:50.773347+0000","last_unstale":"2024-04-26T21:52:52.116006+0000","last_undegraded":"2024-04-26T21:52:52.116006+0000","last_fullsized":"2024-04-26T21:52:52.116006+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:59:55.303014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"87'6","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:58.525913+0000","last_change":"2024-04-26T21:52:50.109437+0000","last_active":"2024-04-26T21:52:58.525913+0000","last_peered":"2024-04-26T21:52:58.525913+0000","last_clean":"2024-04-26T21:52:58.525913+0000","last_became_active":"2024-04-26T21:52:50.108605+0000","last_became_peered":"2024-04-26T21:52:50.108605+0000","last_unstale":"2024-04-26T21:52:58.525913+0000","last_undegraded":"2024-04-26T21:52:58.525913+0000","last_fullsized":"2024-04-26T21:52:58.525913+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:31:04.208334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":341,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:51.873223+0000","last_change":"2024-04-26T21:51:11.498577+0000","last_active":"2024-04-26T21:52:51.873223+0000","last_peered":"2024-04-26T21:52:51.873223+0000","last_clean":"2024-04-26T21:52:51.873223+0000","last_became_active":"2024-04-26T21:51:11.497422+0000","last_became_peered":"2024-04-26T21:51:11.497422+0000","last_unstale":"2024-04-26T21:52:51.873223+0000","last_undegraded":"2024-04-26T21:52:51.873223+0000","last_fullsized":"2024-04-26T21:52:51.873223+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:46:29.789400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:46:29.789400+0000","last_clean_scrub_stamp":"2024-04-26T21:46:29.789400+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:54:13.450359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:52.480552+0000","last_change":"2024-04-26T21:52:50.794270+0000","last_active":"2024-04-26T21:52:52.480552+0000","last_peered":"2024-04-26T21:52:52.480552+0000","last_clean":"2024-04-26T21:52:52.480552+0000","last_became_active":"2024-04-26T21:52:50.793966+0000","last_became_peered":"2024-04-26T21:52:50.793966+0000","last_unstale":"2024-04-26T21:52:52.480552+0000","last_undegraded":"2024-04-26T21:52:52.480552+0000","last_fullsized":"2024-04-26T21:52:52.480552+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:01:37.668466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"87'4","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:55.561049+0000","last_change":"2024-04-26T21:52:50.102700+0000","last_active":"2024-04-26T21:52:55.561049+0000","last_peered":"2024-04-26T21:52:55.561049+0000","last_clean":"2024-04-26T21:52:55.561049+0000","last_became_active":"2024-04-26T21:52:50.101267+0000","last_became_peered":"2024-04-26T21:52:50.101267+0000","last_unstale":"2024-04-26T21:52:55.561049+0000","last_undegraded":"2024-04-26T21:52:55.561049+0000","last_fullsized":"2024-04-26T21:52:55.561049+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:41:21.727411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"87'4","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:57.598687+0000","last_change":"2024-04-26T21:52:50.102191+0000","last_active":"2024-04-26T21:52:57.598687+0000","last_peered":"2024-04-26T21:52:57.598687+0000","last_clean":"2024-04-26T21:52:57.598687+0000","last_became_active":"2024-04-26T21:52:50.101307+0000","last_became_peered":"2024-04-26T21:52:50.101307+0000","last_unstale":"2024-04-26T21:52:57.598687+0000","last_undegraded":"2024-04-26T21:52:57.598687+0000","last_fullsized":"2024-04-26T21:52:57.598687+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:59:12.465291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:52.481172+0000","last_change":"2024-04-26T21:52:50.792020+0000","last_active":"2024-04-26T21:52:52.481172+0000","last_peered":"2024-04-26T21:52:52.481172+0000","last_clean":"2024-04-26T21:52:52.481172+0000","last_became_active":"2024-04-26T21:52:50.791677+0000","last_became_peered":"2024-04-26T21:52:50.791677+0000","last_unstale":"2024-04-26T21:52:52.481172+0000","last_undegraded":"2024-04-26T21:52:52.481172+0000","last_fullsized":"2024-04-26T21:52:52.481172+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:11:42.218294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"87'8","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:56.581638+0000","last_change":"2024-04-26T21:52:50.110060+0000","last_active":"2024-04-26T21:52:56.581638+0000","last_peered":"2024-04-26T21:52:56.581638+0000","last_clean":"2024-04-26T21:52:56.581638+0000","last_became_active":"2024-04-26T21:52:50.109092+0000","last_became_peered":"2024-04-26T21:52:50.109092+0000","last_unstale":"2024-04-26T21:52:56.581638+0000","last_undegraded":"2024-04-26T21:52:56.581638+0000","last_fullsized":"2024-04-26T21:52:56.581638+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:09:35.789887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:52.117243+0000","last_change":"2024-04-26T21:52:50.779775+0000","last_active":"2024-04-26T21:52:52.117243+0000","last_peered":"2024-04-26T21:52:52.117243+0000","last_clean":"2024-04-26T21:52:52.117243+0000","last_became_active":"2024-04-26T21:52:50.778857+0000","last_became_peered":"2024-04-26T21:52:50.778857+0000","last_unstale":"2024-04-26T21:52:52.117243+0000","last_undegraded":"2024-04-26T21:52:52.117243+0000","last_fullsized":"2024-04-26T21:52:52.117243+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:59:50.946578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"87'7","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:57.600430+0000","last_change":"2024-04-26T21:52:50.113027+0000","last_active":"2024-04-26T21:52:57.600430+0000","last_peered":"2024-04-26T21:52:57.600430+0000","last_clean":"2024-04-26T21:52:57.600430+0000","last_became_active":"2024-04-26T21:52:50.112637+0000","last_became_peered":"2024-04-26T21:52:50.112637+0000","last_unstale":"2024-04-26T21:52:57.600430+0000","last_undegraded":"2024-04-26T21:52:57.600430+0000","last_fullsized":"2024-04-26T21:52:57.600430+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:09:47.798184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:51.755142+0000","last_change":"2024-04-26T21:52:50.785119+0000","last_active":"2024-04-26T21:52:51.755142+0000","last_peered":"2024-04-26T21:52:51.755142+0000","last_clean":"2024-04-26T21:52:51.755142+0000","last_became_active":"2024-04-26T21:52:50.781005+0000","last_became_peered":"2024-04-26T21:52:50.781005+0000","last_unstale":"2024-04-26T21:52:51.755142+0000","last_undegraded":"2024-04-26T21:52:51.755142+0000","last_fullsized":"2024-04-26T21:52:51.755142+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:16:32.986810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"87'2","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:54.554877+0000","last_change":"2024-04-26T21:52:50.112263+0000","last_active":"2024-04-26T21:52:54.554877+0000","last_peered":"2024-04-26T21:52:54.554877+0000","last_clean":"2024-04-26T21:52:54.554877+0000","last_became_active":"2024-04-26T21:52:50.110649+0000","last_became_peered":"2024-04-26T21:52:50.110649+0000","last_unstale":"2024-04-26T21:52:54.554877+0000","last_undegraded":"2024-04-26T21:52:54.554877+0000","last_fullsized":"2024-04-26T21:52:54.554877+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:14:23.245000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:51.755989+0000","last_change":"2024-04-26T21:52:50.790674+0000","last_active":"2024-04-26T21:52:51.755989+0000","last_peered":"2024-04-26T21:52:51.755989+0000","last_clean":"2024-04-26T21:52:51.755989+0000","last_became_active":"2024-04-26T21:52:50.790286+0000","last_became_peered":"2024-04-26T21:52:50.790286+0000","last_unstale":"2024-04-26T21:52:51.755989+0000","last_undegraded":"2024-04-26T21:52:51.755989+0000","last_fullsized":"2024-04-26T21:52:51.755989+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:22:32.246321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"87'5","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:56.576887+0000","last_change":"2024-04-26T21:52:50.112158+0000","last_active":"2024-04-26T21:52:56.576887+0000","last_peered":"2024-04-26T21:52:56.576887+0000","last_clean":"2024-04-26T21:52:56.576887+0000","last_became_active":"2024-04-26T21:52:50.111453+0000","last_became_peered":"2024-04-26T21:52:50.111453+0000","last_unstale":"2024-04-26T21:52:56.576887+0000","last_undegraded":"2024-04-26T21:52:56.576887+0000","last_fullsized":"2024-04-26T21:52:56.576887+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:16:58.626985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:52.481913+0000","last_change":"2024-04-26T21:52:50.798241+0000","last_active":"2024-04-26T21:52:52.481913+0000","last_peered":"2024-04-26T21:52:52.481913+0000","last_clean":"2024-04-26T21:52:52.481913+0000","last_became_active":"2024-04-26T21:52:50.797878+0000","last_became_peered":"2024-04-26T21:52:50.797878+0000","last_unstale":"2024-04-26T21:52:52.481913+0000","last_undegraded":"2024-04-26T21:52:52.481913+0000","last_fullsized":"2024-04-26T21:52:52.481913+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:42:49.363849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"87'3","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:56.581807+0000","last_change":"2024-04-26T21:52:50.103848+0000","last_active":"2024-04-26T21:52:56.581807+0000","last_peered":"2024-04-26T21:52:56.581807+0000","last_clean":"2024-04-26T21:52:56.581807+0000","last_became_active":"2024-04-26T21:52:50.102606+0000","last_became_peered":"2024-04-26T21:52:50.102606+0000","last_unstale":"2024-04-26T21:52:56.581807+0000","last_undegraded":"2024-04-26T21:52:56.581807+0000","last_fullsized":"2024-04-26T21:52:56.581807+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:35:37.023110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:52.116330+0000","last_change":"2024-04-26T21:52:50.776333+0000","last_active":"2024-04-26T21:52:52.116330+0000","last_peered":"2024-04-26T21:52:52.116330+0000","last_clean":"2024-04-26T21:52:52.116330+0000","last_became_active":"2024-04-26T21:52:50.776001+0000","last_became_peered":"2024-04-26T21:52:50.776001+0000","last_unstale":"2024-04-26T21:52:52.116330+0000","last_undegraded":"2024-04-26T21:52:52.116330+0000","last_fullsized":"2024-04-26T21:52:52.116330+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:38:26.578928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"87'4","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:54.556970+0000","last_change":"2024-04-26T21:52:50.106008+0000","last_active":"2024-04-26T21:52:54.556970+0000","last_peered":"2024-04-26T21:52:54.556970+0000","last_clean":"2024-04-26T21:52:54.556970+0000","last_became_active":"2024-04-26T21:52:50.105565+0000","last_became_peered":"2024-04-26T21:52:50.105565+0000","last_unstale":"2024-04-26T21:52:54.556970+0000","last_undegraded":"2024-04-26T21:52:54.556970+0000","last_fullsized":"2024-04-26T21:52:54.556970+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:15:17.355461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:51.873809+0000","last_change":"2024-04-26T21:52:50.776051+0000","last_active":"2024-04-26T21:52:51.873809+0000","last_peered":"2024-04-26T21:52:51.873809+0000","last_clean":"2024-04-26T21:52:51.873809+0000","last_became_active":"2024-04-26T21:52:50.775716+0000","last_became_peered":"2024-04-26T21:52:50.775716+0000","last_unstale":"2024-04-26T21:52:51.873809+0000","last_undegraded":"2024-04-26T21:52:51.873809+0000","last_fullsized":"2024-04-26T21:52:51.873809+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:34:21.110773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"87'6","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:58.531241+0000","last_change":"2024-04-26T21:52:50.111280+0000","last_active":"2024-04-26T21:52:58.531241+0000","last_peered":"2024-04-26T21:52:58.531241+0000","last_clean":"2024-04-26T21:52:58.531241+0000","last_became_active":"2024-04-26T21:52:50.110972+0000","last_became_peered":"2024-04-26T21:52:50.110972+0000","last_unstale":"2024-04-26T21:52:58.531241+0000","last_undegraded":"2024-04-26T21:52:58.531241+0000","last_fullsized":"2024-04-26T21:52:58.531241+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:41:48.203134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1561,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:51.874639+0000","last_change":"2024-04-26T21:52:50.777488+0000","last_active":"2024-04-26T21:52:51.874639+0000","last_peered":"2024-04-26T21:52:51.874639+0000","last_clean":"2024-04-26T21:52:51.874639+0000","last_became_active":"2024-04-26T21:52:50.777162+0000","last_became_peered":"2024-04-26T21:52:50.777162+0000","last_unstale":"2024-04-26T21:52:51.874639+0000","last_undegraded":"2024-04-26T21:52:51.874639+0000","last_fullsized":"2024-04-26T21:52:51.874639+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:07:22.683109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"87'3","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:56.577754+0000","last_change":"2024-04-26T21:52:50.104508+0000","last_active":"2024-04-26T21:52:56.577754+0000","last_peered":"2024-04-26T21:52:56.577754+0000","last_clean":"2024-04-26T21:52:56.577754+0000","last_became_active":"2024-04-26T21:52:50.103924+0000","last_became_peered":"2024-04-26T21:52:50.103924+0000","last_unstale":"2024-04-26T21:52:56.577754+0000","last_undegraded":"2024-04-26T21:52:56.577754+0000","last_fullsized":"2024-04-26T21:52:56.577754+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:39:25.861339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:51.874293+0000","last_change":"2024-04-26T21:52:50.782915+0000","last_active":"2024-04-26T21:52:51.874293+0000","last_peered":"2024-04-26T21:52:51.874293+0000","last_clean":"2024-04-26T21:52:51.874293+0000","last_became_active":"2024-04-26T21:52:50.782253+0000","last_became_peered":"2024-04-26T21:52:50.782253+0000","last_unstale":"2024-04-26T21:52:51.874293+0000","last_undegraded":"2024-04-26T21:52:51.874293+0000","last_fullsized":"2024-04-26T21:52:51.874293+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:12:51.624162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"87'5","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:57.596130+0000","last_change":"2024-04-26T21:52:50.100008+0000","last_active":"2024-04-26T21:52:57.596130+0000","last_peered":"2024-04-26T21:52:57.596130+0000","last_clean":"2024-04-26T21:52:57.596130+0000","last_became_active":"2024-04-26T21:52:50.099537+0000","last_became_peered":"2024-04-26T21:52:50.099537+0000","last_unstale":"2024-04-26T21:52:57.596130+0000","last_undegraded":"2024-04-26T21:52:57.596130+0000","last_fullsized":"2024-04-26T21:52:57.596130+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:12:11.901178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:51.874353+0000","last_change":"2024-04-26T21:52:50.787777+0000","last_active":"2024-04-26T21:52:51.874353+0000","last_peered":"2024-04-26T21:52:51.874353+0000","last_clean":"2024-04-26T21:52:51.874353+0000","last_became_active":"2024-04-26T21:52:50.782403+0000","last_became_peered":"2024-04-26T21:52:50.782403+0000","last_unstale":"2024-04-26T21:52:51.874353+0000","last_undegraded":"2024-04-26T21:52:51.874353+0000","last_fullsized":"2024-04-26T21:52:51.874353+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:28:58.425802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:52.482233+0000","last_change":"2024-04-26T21:52:50.800010+0000","last_active":"2024-04-26T21:52:52.482233+0000","last_peered":"2024-04-26T21:52:52.482233+0000","last_clean":"2024-04-26T21:52:52.482233+0000","last_became_active":"2024-04-26T21:52:50.799308+0000","last_became_peered":"2024-04-26T21:52:50.799308+0000","last_unstale":"2024-04-26T21:52:52.482233+0000","last_undegraded":"2024-04-26T21:52:52.482233+0000","last_fullsized":"2024-04-26T21:52:52.482233+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:54:25.772312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:52.484126+0000","last_change":"2024-04-26T21:52:50.798281+0000","last_active":"2024-04-26T21:52:52.484126+0000","last_peered":"2024-04-26T21:52:52.484126+0000","last_clean":"2024-04-26T21:52:52.484126+0000","last_became_active":"2024-04-26T21:52:50.795879+0000","last_became_peered":"2024-04-26T21:52:50.795879+0000","last_unstale":"2024-04-26T21:52:52.484126+0000","last_undegraded":"2024-04-26T21:52:52.484126+0000","last_fullsized":"2024-04-26T21:52:52.484126+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:36:00.769991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:52.484457+0000","last_change":"2024-04-26T21:52:50.791058+0000","last_active":"2024-04-26T21:52:52.484457+0000","last_peered":"2024-04-26T21:52:52.484457+0000","last_clean":"2024-04-26T21:52:52.484457+0000","last_became_active":"2024-04-26T21:52:50.790721+0000","last_became_peered":"2024-04-26T21:52:50.790721+0000","last_unstale":"2024-04-26T21:52:52.484457+0000","last_undegraded":"2024-04-26T21:52:52.484457+0000","last_fullsized":"2024-04-26T21:52:52.484457+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:56:03.239622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:51.754785+0000","last_change":"2024-04-26T21:52:50.788193+0000","last_active":"2024-04-26T21:52:51.754785+0000","last_peered":"2024-04-26T21:52:51.754785+0000","last_clean":"2024-04-26T21:52:51.754785+0000","last_became_active":"2024-04-26T21:52:50.786749+0000","last_became_peered":"2024-04-26T21:52:50.786749+0000","last_unstale":"2024-04-26T21:52:51.754785+0000","last_undegraded":"2024-04-26T21:52:51.754785+0000","last_fullsized":"2024-04-26T21:52:51.754785+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:22:11.277217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:52.481032+0000","last_change":"2024-04-26T21:52:50.782508+0000","last_active":"2024-04-26T21:52:52.481032+0000","last_peered":"2024-04-26T21:52:52.481032+0000","last_clean":"2024-04-26T21:52:52.481032+0000","last_became_active":"2024-04-26T21:52:50.781505+0000","last_became_peered":"2024-04-26T21:52:50.781505+0000","last_unstale":"2024-04-26T21:52:52.481032+0000","last_undegraded":"2024-04-26T21:52:52.481032+0000","last_fullsized":"2024-04-26T21:52:52.481032+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:22:57.867546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:52.482378+0000","last_change":"2024-04-26T21:52:50.789205+0000","last_active":"2024-04-26T21:52:52.482378+0000","last_peered":"2024-04-26T21:52:52.482378+0000","last_clean":"2024-04-26T21:52:52.482378+0000","last_became_active":"2024-04-26T21:52:50.788898+0000","last_became_peered":"2024-04-26T21:52:50.788898+0000","last_unstale":"2024-04-26T21:52:52.482378+0000","last_undegraded":"2024-04-26T21:52:52.482378+0000","last_fullsized":"2024-04-26T21:52:52.482378+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:18:51.401968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:52.485015+0000","last_change":"2024-04-26T21:52:50.793451+0000","last_active":"2024-04-26T21:52:52.485015+0000","last_peered":"2024-04-26T21:52:52.485015+0000","last_clean":"2024-04-26T21:52:52.485015+0000","last_became_active":"2024-04-26T21:52:50.791696+0000","last_became_peered":"2024-04-26T21:52:50.791696+0000","last_unstale":"2024-04-26T21:52:52.485015+0000","last_undegraded":"2024-04-26T21:52:52.485015+0000","last_fullsized":"2024-04-26T21:52:52.485015+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:51:43.543500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:52.117016+0000","last_change":"2024-04-26T21:52:50.786077+0000","last_active":"2024-04-26T21:52:52.117016+0000","last_peered":"2024-04-26T21:52:52.117016+0000","last_clean":"2024-04-26T21:52:52.117016+0000","last_became_active":"2024-04-26T21:52:50.785757+0000","last_became_peered":"2024-04-26T21:52:50.785757+0000","last_unstale":"2024-04-26T21:52:52.117016+0000","last_undegraded":"2024-04-26T21:52:52.117016+0000","last_fullsized":"2024-04-26T21:52:52.117016+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:53:18.645179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:52.484791+0000","last_change":"2024-04-26T21:52:50.782752+0000","last_active":"2024-04-26T21:52:52.484791+0000","last_peered":"2024-04-26T21:52:52.484791+0000","last_clean":"2024-04-26T21:52:52.484791+0000","last_became_active":"2024-04-26T21:52:50.777773+0000","last_became_peered":"2024-04-26T21:52:50.777773+0000","last_unstale":"2024-04-26T21:52:52.484791+0000","last_undegraded":"2024-04-26T21:52:52.484791+0000","last_fullsized":"2024-04-26T21:52:52.484791+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:54:39.656756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:52.115676+0000","last_change":"2024-04-26T21:52:50.794580+0000","last_active":"2024-04-26T21:52:52.115676+0000","last_peered":"2024-04-26T21:52:52.115676+0000","last_clean":"2024-04-26T21:52:52.115676+0000","last_became_active":"2024-04-26T21:52:50.792398+0000","last_became_peered":"2024-04-26T21:52:50.792398+0000","last_unstale":"2024-04-26T21:52:52.115676+0000","last_undegraded":"2024-04-26T21:52:52.115676+0000","last_fullsized":"2024-04-26T21:52:52.115676+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:21:38.913644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:51.778289+0000","last_change":"2024-04-26T21:52:50.790087+0000","last_active":"2024-04-26T21:52:51.778289+0000","last_peered":"2024-04-26T21:52:51.778289+0000","last_clean":"2024-04-26T21:52:51.778289+0000","last_became_active":"2024-04-26T21:52:50.788685+0000","last_became_peered":"2024-04-26T21:52:50.788685+0000","last_unstale":"2024-04-26T21:52:51.778289+0000","last_undegraded":"2024-04-26T21:52:51.778289+0000","last_fullsized":"2024-04-26T21:52:51.778289+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:41:25.094794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:52.480828+0000","last_change":"2024-04-26T21:52:50.792166+0000","last_active":"2024-04-26T21:52:52.480828+0000","last_peered":"2024-04-26T21:52:52.480828+0000","last_clean":"2024-04-26T21:52:52.480828+0000","last_became_active":"2024-04-26T21:52:50.790709+0000","last_became_peered":"2024-04-26T21:52:50.790709+0000","last_unstale":"2024-04-26T21:52:52.480828+0000","last_undegraded":"2024-04-26T21:52:52.480828+0000","last_fullsized":"2024-04-26T21:52:52.480828+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:05:48.200573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:52.481689+0000","last_change":"2024-04-26T21:52:50.791161+0000","last_active":"2024-04-26T21:52:52.481689+0000","last_peered":"2024-04-26T21:52:52.481689+0000","last_clean":"2024-04-26T21:52:52.481689+0000","last_became_active":"2024-04-26T21:52:50.790793+0000","last_became_peered":"2024-04-26T21:52:50.790793+0000","last_unstale":"2024-04-26T21:52:52.481689+0000","last_undegraded":"2024-04-26T21:52:52.481689+0000","last_fullsized":"2024-04-26T21:52:52.481689+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:13:53.831932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":15025,"num_objects":97,"num_object_clones":0,"num_object_copies":291,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":97,"num_whiteouts":0,"num_read":34,"num_read_kb":31,"num_write":208,"num_write_kb":72,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":65,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":434176,"data_stored":75034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":182,"ondisk_log_size":182,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":82,"seq":352187318289,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":950272,"data_stored":450388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":10,"up_from":74,"seq":317827579928,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28568,"kb_used_data":1552,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716584,"statfs":{"total":95995035648,"available":95965782016,"internally_reserved":0,"allocated":1589248,"data_stored":1042912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":69,"seq":296352743456,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36288,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708864,"statfs":{"total":95995035648,"available":95957876736,"internally_reserved":0,"allocated":974848,"data_stored":449927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":8,"up_from":61,"seq":261993005096,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36284,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708868,"statfs":{"total":95995035648,"available":95957880832,"internally_reserved":0,"allocated":970752,"data_stored":450305,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299436,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28536,"kb_used_data":1520,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716616,"statfs":{"total":95995035648,"available":95965814784,"internally_reserved":0,"allocated":1556480,"data_stored":1041637,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462965,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32108,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713044,"statfs":{"total":95995035648,"available":95962157056,"internally_reserved":0,"allocated":954368,"data_stored":449527,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574935,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":978944,"data_stored":471452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738462,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36284,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708868,"statfs":{"total":95995035648,"available":95957880832,"internally_reserved":0,"allocated":970752,"data_stored":450871,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411409,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36312,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708840,"statfs":{"total":95995035648,"available":95957852160,"internally_reserved":0,"allocated":999424,"data_stored":472063,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182472,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":954368,"data_stored":449089,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920836,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36272,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708880,"statfs":{"total":95995035648,"available":95957893120,"internally_reserved":0,"allocated":958464,"data_stored":449505,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724602,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36872,"kb_used_data":1536,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708280,"statfs":{"total":95995035648,"available":95957278720,"internally_reserved":0,"allocated":1572864,"data_stored":1041411,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":27262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":23258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":9948,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T21:53:02.433 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-26T21:53:02.434 INFO:tasks.cephadm:Setting up client nodes... 2024-04-26T21:53:02.434 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a1780950-0415-11ef-bc93-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-04-26T21:53:02.888 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/a1780950-0415-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:53:03.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:53:02 smithi164 ceph-mon[31916]: pgmap v373: 97 pgs: 97 active+clean; 591 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 7.2 KiB/s wr, 24 op/s 2024-04-26T21:53:03.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:53:02 smithi164 ceph-mon[31916]: from='client.14814 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T21:53:03.224 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:53:02 smithi072 ceph-mon[28138]: pgmap v373: 97 pgs: 97 active+clean; 591 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 7.2 KiB/s wr, 24 op/s 2024-04-26T21:53:03.224 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:53:02 smithi072 ceph-mon[28138]: from='client.14814 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T21:53:03.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:53:02 smithi076 ceph-mon[31823]: pgmap v373: 97 pgs: 97 active+clean; 591 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 7.2 KiB/s wr, 24 op/s 2024-04-26T21:53:03.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:53:02 smithi076 ceph-mon[31823]: from='client.14814 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T21:53:04.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:53:03 smithi164 ceph-mon[31916]: from='client.14820 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T21:53:04.290 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:53:03 smithi072 ceph-mon[28138]: from='client.14820 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T21:53:04.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:53:03 smithi076 ceph-mon[31823]: from='client.14820 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T21:53:05.183 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:53:04 smithi072 ceph-mon[28138]: pgmap v374: 97 pgs: 97 active+clean; 591 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6.1 KiB/s wr, 21 op/s 2024-04-26T21:53:05.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:53:04 smithi164 ceph-mon[31916]: pgmap v374: 97 pgs: 97 active+clean; 591 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6.1 KiB/s wr, 21 op/s 2024-04-26T21:53:05.219 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:05.218+0000 7fd2dbe00700 1 -- 172.21.15.72:0/961720793 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd2dc106700 msgr2=0x7fd2dc106b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:05.220 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:05.218+0000 7fd2dbe00700 1 --2- 172.21.15.72:0/961720793 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd2dc106700 0x7fd2dc106b60 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fd2c80099c0 tx=0x7fd2c8009cd0 comp rx=0 tx=0).stop 2024-04-26T21:53:05.220 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:05.218+0000 7fd2dbe00700 1 -- 172.21.15.72:0/961720793 shutdown_connections 2024-04-26T21:53:05.220 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:05.218+0000 7fd2dbe00700 1 --2- 172.21.15.72:0/961720793 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd2dc1070a0 0x7fd2dc10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:05.220 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:05.218+0000 7fd2dbe00700 1 --2- 172.21.15.72:0/961720793 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd2dc105500 0x7fd2dc105920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:05.220 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:05.218+0000 7fd2dbe00700 1 --2- 172.21.15.72:0/961720793 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd2dc106700 0x7fd2dc106b60 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:05.220 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:05.218+0000 7fd2dbe00700 1 -- 172.21.15.72:0/961720793 >> 172.21.15.72:0/961720793 conn(0x7fd2dc100aa0 msgr2=0x7fd2dc102ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:53:05.220 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:05.219+0000 7fd2dbe00700 1 -- 172.21.15.72:0/961720793 shutdown_connections 2024-04-26T21:53:05.220 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:05.219+0000 7fd2dbe00700 1 -- 172.21.15.72:0/961720793 wait complete. 2024-04-26T21:53:05.223 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:05.220+0000 7fd2dbe00700 1 Processor -- start 2024-04-26T21:53:05.223 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:05.220+0000 7fd2dbe00700 1 -- start start 2024-04-26T21:53:05.223 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:05.220+0000 7fd2dbe00700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd2dc105500 0x7fd2dc19f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:05.223 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:05.220+0000 7fd2dbe00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd2dc106700 0x7fd2dc19fde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:05.223 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:05.220+0000 7fd2dbe00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd2dc1070a0 0x7fd2dc1a6e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:05.223 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:05.220+0000 7fd2dbe00700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fd2dc110ea0 con 0x7fd2dc106700 2024-04-26T21:53:05.223 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:05.220+0000 7fd2dbe00700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fd2dc110d20 con 0x7fd2dc105500 2024-04-26T21:53:05.223 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:05.220+0000 7fd2dbe00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd2dc111020 con 0x7fd2dc1070a0 2024-04-26T21:53:05.223 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:05.221+0000 7fd2da000700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd2dc106700 0x7fd2dc19fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:05.223 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:05.221+0000 7fd2da000700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd2dc106700 0x7fd2dc19fde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:56034/0 (socket says 172.21.15.72:56034) 2024-04-26T21:53:05.224 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:05.221+0000 7fd2da000700 1 -- 172.21.15.72:0/311851592 learned_addr learned my addr 172.21.15.72:0/311851592 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:53:05.224 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:05.221+0000 7fd2daa00700 1 --2- 172.21.15.72:0/311851592 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd2dc105500 0x7fd2dc19f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:05.224 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:05.221+0000 7fd2db400700 1 --2- 172.21.15.72:0/311851592 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd2dc1070a0 0x7fd2dc1a6e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:05.224 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:05.221+0000 7fd2da000700 1 -- 172.21.15.72:0/311851592 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd2dc1070a0 msgr2=0x7fd2dc1a6e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:05.224 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:05.221+0000 7fd2da000700 1 --2- 172.21.15.72:0/311851592 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd2dc1070a0 0x7fd2dc1a6e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:05.224 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:05.221+0000 7fd2da000700 1 -- 172.21.15.72:0/311851592 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd2dc105500 msgr2=0x7fd2dc19f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:05.224 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:05.221+0000 7fd2da000700 1 --2- 172.21.15.72:0/311851592 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd2dc105500 0x7fd2dc19f8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:05.224 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:05.221+0000 7fd2da000700 1 -- 172.21.15.72:0/311851592 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2dc1a7500 con 0x7fd2dc106700 2024-04-26T21:53:05.224 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:05.221+0000 7fd2db400700 1 --2- 172.21.15.72:0/311851592 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd2dc1070a0 0x7fd2dc1a6e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:53:05.224 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:05.222+0000 7fd2da000700 1 --2- 172.21.15.72:0/311851592 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd2dc106700 0x7fd2dc19fde0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fd2c800b580 tx=0x7fd2c800f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:53:05.224 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:05.222+0000 7fd2d3400700 1 -- 172.21.15.72:0/311851592 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2c8027070 con 0x7fd2dc106700 2024-04-26T21:53:05.224 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:05.222+0000 7fd2d3400700 1 -- 172.21.15.72:0/311851592 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd2c800bd10 con 0x7fd2dc106700 2024-04-26T21:53:05.224 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:05.222+0000 7fd2d3400700 1 -- 172.21.15.72:0/311851592 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2c8022e00 con 0x7fd2dc106700 2024-04-26T21:53:05.224 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:05.222+0000 7fd2dbe00700 1 -- 172.21.15.72:0/311851592 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2dc1a7790 con 0x7fd2dc106700 2024-04-26T21:53:05.224 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:05.222+0000 7fd2dbe00700 1 -- 172.21.15.72:0/311851592 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2dc1a7b90 con 0x7fd2dc106700 2024-04-26T21:53:05.225 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:05.224+0000 7fd2dbe00700 1 -- 172.21.15.72:0/311851592 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2dc064a80 con 0x7fd2dc106700 2024-04-26T21:53:05.226 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:05.226+0000 7fd2d3400700 1 -- 172.21.15.72:0/311851592 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fd2c802a070 con 0x7fd2dc106700 2024-04-26T21:53:05.227 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:05.227+0000 7fd2d3400700 1 --2- 172.21.15.72:0/311851592 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fd2c00a3e00 0x7fd2c00a62c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:05.227 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:05.227+0000 7fd2d3400700 1 -- 172.21.15.72:0/311851592 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7fd2c80d11c0 con 0x7fd2dc106700 2024-04-26T21:53:05.227 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:05.227+0000 7fd2daa00700 1 --2- 172.21.15.72:0/311851592 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fd2c00a3e00 0x7fd2c00a62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:05.228 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:05.228+0000 7fd2daa00700 1 --2- 172.21.15.72:0/311851592 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fd2c00a3e00 0x7fd2c00a62c0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fd2dc106560 tx=0x7fd2cc00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:53:05.232 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:05.232+0000 7fd2d3400700 1 -- 172.21.15.72:0/311851592 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fd2c8013070 con 0x7fd2dc106700 2024-04-26T21:53:05.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:53:04 smithi076 ceph-mon[31823]: pgmap v374: 97 pgs: 97 active+clean; 591 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6.1 KiB/s wr, 21 op/s 2024-04-26T21:53:05.559 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:05.558+0000 7fd2dbe00700 1 -- 172.21.15.72:0/311851592 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7fd2dc1a8160 con 0x7fd2dc106700 2024-04-26T21:53:05.564 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:05.563+0000 7fd2d3400700 1 -- 172.21.15.72:0/311851592 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v33) ==== 170+0+59 (secure 0 0 0) 0x7fd2dc1a8160 con 0x7fd2dc106700 2024-04-26T21:53:05.564 INFO:teuthology.orchestra.run.smithi072.stdout:[client.0] 2024-04-26T21:53:05.564 INFO:teuthology.orchestra.run.smithi072.stdout: key = AQBBIixmf35mIRAAOCij0mpzJhdAkh1cF6ukBw== 2024-04-26T21:53:05.567 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:05.566+0000 7fd2dbe00700 1 -- 172.21.15.72:0/311851592 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fd2c00a3e00 msgr2=0x7fd2c00a62c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:05.567 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:05.566+0000 7fd2dbe00700 1 --2- 172.21.15.72:0/311851592 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fd2c00a3e00 0x7fd2c00a62c0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fd2dc106560 tx=0x7fd2cc00b410 comp rx=0 tx=0).stop 2024-04-26T21:53:05.567 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:05.566+0000 7fd2dbe00700 1 -- 172.21.15.72:0/311851592 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd2dc106700 msgr2=0x7fd2dc19fde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:05.567 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:05.566+0000 7fd2dbe00700 1 --2- 172.21.15.72:0/311851592 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd2dc106700 0x7fd2dc19fde0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fd2c800b580 tx=0x7fd2c800f690 comp rx=0 tx=0).stop 2024-04-26T21:53:05.567 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:05.566+0000 7fd2dbe00700 1 -- 172.21.15.72:0/311851592 shutdown_connections 2024-04-26T21:53:05.567 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:05.566+0000 7fd2dbe00700 1 --2- 172.21.15.72:0/311851592 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fd2c00a3e00 0x7fd2c00a62c0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:05.568 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:05.566+0000 7fd2dbe00700 1 --2- 172.21.15.72:0/311851592 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd2dc1070a0 0x7fd2dc1a6e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:05.568 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:05.566+0000 7fd2dbe00700 1 --2- 172.21.15.72:0/311851592 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd2dc105500 0x7fd2dc19f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:05.568 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:05.566+0000 7fd2dbe00700 1 --2- 172.21.15.72:0/311851592 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd2dc106700 0x7fd2dc19fde0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:05.568 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:05.566+0000 7fd2dbe00700 1 -- 172.21.15.72:0/311851592 >> 172.21.15.72:0/311851592 conn(0x7fd2dc100aa0 msgr2=0x7fd2dc079870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:53:05.568 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:05.567+0000 7fd2dbe00700 1 -- 172.21.15.72:0/311851592 shutdown_connections 2024-04-26T21:53:05.568 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:05.567+0000 7fd2dbe00700 1 -- 172.21.15.72:0/311851592 wait complete. 2024-04-26T21:53:06.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:53:05 smithi164 ceph-mon[31916]: from='client.? 172.21.15.72:0/311851592' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-26T21:53:06.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:53:05 smithi164 ceph-mon[31916]: from='client.? 172.21.15.72:0/311851592' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-26T21:53:06.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:53:05 smithi076 ceph-mon[31823]: from='client.? 172.21.15.72:0/311851592' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-26T21:53:06.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:53:05 smithi076 ceph-mon[31823]: from='client.? 172.21.15.72:0/311851592' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-26T21:53:06.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:53:05 smithi072 ceph-mon[28138]: from='client.? 172.21.15.72:0/311851592' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-26T21:53:06.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:53:05 smithi072 ceph-mon[28138]: from='client.? 172.21.15.72:0/311851592' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-26T21:53:06.388 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2024-04-26T21:53:06.388 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-04-26T21:53:06.388 DEBUG:teuthology.orchestra.run.smithi072:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-26T21:53:06.516 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-26T21:53:06.517 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-04-26T21:53:06.517 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid a1780950-0415-11ef-bc93-c7b262605968 -- ceph mgr dump --format=json 2024-04-26T21:53:07.009 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/a1780950-0415-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:53:07.071 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:53:06 smithi072 ceph-mon[28138]: pgmap v375: 97 pgs: 97 active+clean; 621 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 8.7 KiB/s wr, 26 op/s 2024-04-26T21:53:07.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:53:06 smithi164 ceph-mon[31916]: pgmap v375: 97 pgs: 97 active+clean; 621 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 8.7 KiB/s wr, 26 op/s 2024-04-26T21:53:07.217 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:53:06 smithi076 ceph-mon[31823]: pgmap v375: 97 pgs: 97 active+clean; 621 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 8.7 KiB/s wr, 26 op/s 2024-04-26T21:53:09.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:53:08 smithi072 ceph-mon[28138]: pgmap v376: 97 pgs: 97 active+clean; 621 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 7.7 KiB/s wr, 22 op/s 2024-04-26T21:53:09.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:53:08 smithi164 ceph-mon[31916]: pgmap v376: 97 pgs: 97 active+clean; 621 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 7.7 KiB/s wr, 22 op/s 2024-04-26T21:53:09.248 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:09.247+0000 7f5807e00700 1 -- 172.21.15.72:0/667125674 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5800075f70 msgr2=0x7f580010f8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:09.248 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:09.247+0000 7f5807e00700 1 --2- 172.21.15.72:0/667125674 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5800075f70 0x7f580010f8e0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f57f0009990 tx=0x7f57f0009ca0 comp rx=0 tx=0).stop 2024-04-26T21:53:09.248 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:09.247+0000 7f5807e00700 1 -- 172.21.15.72:0/667125674 shutdown_connections 2024-04-26T21:53:09.248 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:09.247+0000 7f5807e00700 1 --2- 172.21.15.72:0/667125674 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f580010fe20 0x7f5800112210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:09.248 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:09.247+0000 7f5807e00700 1 --2- 172.21.15.72:0/667125674 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5800075650 0x7f5800075a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:09.248 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:09.247+0000 7f5807e00700 1 --2- 172.21.15.72:0/667125674 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5800075f70 0x7f580010f8e0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:09.248 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:09.247+0000 7f5807e00700 1 -- 172.21.15.72:0/667125674 >> 172.21.15.72:0/667125674 conn(0x7f580007a580 msgr2=0x7f580007a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:53:09.248 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:09.247+0000 7f5807e00700 1 -- 172.21.15.72:0/667125674 shutdown_connections 2024-04-26T21:53:09.248 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:09.247+0000 7f5807e00700 1 -- 172.21.15.72:0/667125674 wait complete. 2024-04-26T21:53:09.249 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:09.248+0000 7f5807e00700 1 Processor -- start 2024-04-26T21:53:09.249 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:09.248+0000 7f5807e00700 1 -- start start 2024-04-26T21:53:09.250 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:09.249+0000 7f5807e00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5800075650 0x7f58001a18c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:09.250 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:09.249+0000 7f5807e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5800075f70 0x7f58001a1e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:09.250 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:09.249+0000 7f5807e00700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f580010fe20 0x7f580019ba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:09.250 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:09.249+0000 7f5807e00700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f5800115720 con 0x7f5800075650 2024-04-26T21:53:09.250 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:09.249+0000 7f5807e00700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f58001155a0 con 0x7f580010fe20 2024-04-26T21:53:09.250 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:09.249+0000 7f5807e00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f58001158a0 con 0x7f5800075f70 2024-04-26T21:53:09.250 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:09.249+0000 7f5805400700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5800075650 0x7f58001a18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:09.250 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:09.249+0000 7f5805400700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5800075650 0x7f58001a18c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:56054/0 (socket says 172.21.15.72:56054) 2024-04-26T21:53:09.250 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:09.249+0000 7f5805400700 1 -- 172.21.15.72:0/3346018856 learned_addr learned my addr 172.21.15.72:0/3346018856 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:53:09.251 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:09.250+0000 7f5805400700 1 -- 172.21.15.72:0/3346018856 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5800075f70 msgr2=0x7f58001a1e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:09.251 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:09.250+0000 7f5805400700 1 --2- 172.21.15.72:0/3346018856 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5800075f70 0x7f58001a1e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:09.251 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:09.250+0000 7f5805400700 1 -- 172.21.15.72:0/3346018856 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f580010fe20 msgr2=0x7f580019ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:09.251 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:09.250+0000 7f5805400700 1 --2- 172.21.15.72:0/3346018856 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f580010fe20 0x7f580019ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:09.251 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:09.250+0000 7f5805400700 1 -- 172.21.15.72:0/3346018856 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f580019c1e0 con 0x7f5800075650 2024-04-26T21:53:09.252 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:09.250+0000 7f5805400700 1 --2- 172.21.15.72:0/3346018856 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5800075650 0x7f58001a18c0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f57f400c9b0 tx=0x7f57f400cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:53:09.252 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:09.250+0000 7f57fe000700 1 -- 172.21.15.72:0/3346018856 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57f4007b00 con 0x7f5800075650 2024-04-26T21:53:09.252 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:09.250+0000 7f57fe000700 1 -- 172.21.15.72:0/3346018856 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f57f4007ca0 con 0x7f5800075650 2024-04-26T21:53:09.252 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:09.250+0000 7f57fe000700 1 -- 172.21.15.72:0/3346018856 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57f4018660 con 0x7f5800075650 2024-04-26T21:53:09.252 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:09.250+0000 7f5807e00700 1 -- 172.21.15.72:0/3346018856 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f580019c4d0 con 0x7f5800075650 2024-04-26T21:53:09.252 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:09.250+0000 7f5807e00700 1 -- 172.21.15.72:0/3346018856 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58001a8790 con 0x7f5800075650 2024-04-26T21:53:09.252 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:09.252+0000 7f5807e00700 1 -- 172.21.15.72:0/3346018856 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f580006a110 con 0x7f5800075650 2024-04-26T21:53:09.253 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:09.253+0000 7f57fe000700 1 -- 172.21.15.72:0/3346018856 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f57f400f450 con 0x7f5800075650 2024-04-26T21:53:09.255 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:09.255+0000 7f57fe000700 1 --2- 172.21.15.72:0/3346018856 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f57e80a3d50 0x7f57e80a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:09.255 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:09.255+0000 7f57fe000700 1 -- 172.21.15.72:0/3346018856 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7f57f40d0510 con 0x7f5800075650 2024-04-26T21:53:09.255 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:09.255+0000 7f5804a00700 1 --2- 172.21.15.72:0/3346018856 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f57e80a3d50 0x7f57e80a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:09.256 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:09.256+0000 7f5804a00700 1 --2- 172.21.15.72:0/3346018856 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f57e80a3d50 0x7f57e80a6210 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f57f0003760 tx=0x7f57f0005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:53:09.259 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:09.259+0000 7f57fe000700 1 -- 172.21.15.72:0/3346018856 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f57f4014030 con 0x7f5800075650 2024-04-26T21:53:09.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:53:08 smithi076 ceph-mon[31823]: pgmap v376: 97 pgs: 97 active+clean; 621 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 7.7 KiB/s wr, 22 op/s 2024-04-26T21:53:09.545 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:09.544+0000 7f5807e00700 1 -- 172.21.15.72:0/3346018856 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f580002e490 con 0x7f5800075650 2024-04-26T21:53:09.556 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:09.556+0000 7f57fe000700 1 -- 172.21.15.72:0/3346018856 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+264993 (secure 0 0 0) 0x7f57f409ce40 con 0x7f5800075650 2024-04-26T21:53:09.559 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T21:53:09.563 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:09.562+0000 7f5807e00700 1 -- 172.21.15.72:0/3346018856 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f57e80a3d50 msgr2=0x7f57e80a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:09.563 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:09.562+0000 7f5807e00700 1 --2- 172.21.15.72:0/3346018856 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f57e80a3d50 0x7f57e80a6210 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f57f0003760 tx=0x7f57f0005fb0 comp rx=0 tx=0).stop 2024-04-26T21:53:09.563 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:09.562+0000 7f5807e00700 1 -- 172.21.15.72:0/3346018856 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5800075650 msgr2=0x7f58001a18c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:09.563 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:09.562+0000 7f5807e00700 1 --2- 172.21.15.72:0/3346018856 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5800075650 0x7f58001a18c0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f57f400c9b0 tx=0x7f57f400cd70 comp rx=0 tx=0).stop 2024-04-26T21:53:09.563 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:09.562+0000 7f5807e00700 1 -- 172.21.15.72:0/3346018856 shutdown_connections 2024-04-26T21:53:09.564 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:09.562+0000 7f5807e00700 1 --2- 172.21.15.72:0/3346018856 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f57e80a3d50 0x7f57e80a6210 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:09.564 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:09.562+0000 7f5807e00700 1 --2- 172.21.15.72:0/3346018856 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5800075650 0x7f58001a18c0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:09.564 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:09.562+0000 7f5807e00700 1 --2- 172.21.15.72:0/3346018856 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f580010fe20 0x7f580019ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:09.564 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:09.562+0000 7f5807e00700 1 --2- 172.21.15.72:0/3346018856 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5800075f70 0x7f58001a1e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:09.564 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:09.562+0000 7f5807e00700 1 -- 172.21.15.72:0/3346018856 >> 172.21.15.72:0/3346018856 conn(0x7f580007a580 msgr2=0x7f5800100690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:53:09.564 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:09.563+0000 7f5807e00700 1 -- 172.21.15.72:0/3346018856 shutdown_connections 2024-04-26T21:53:09.564 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:09.563+0000 7f5807e00700 1 -- 172.21.15.72:0/3346018856 wait complete. 2024-04-26T21:53:10.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:53:09 smithi164 ceph-mon[31916]: from='client.? 172.21.15.72:0/3346018856' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-26T21:53:10.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:53:09 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]: dispatch 2024-04-26T21:53:10.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:53:09 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 1]}]: dispatch 2024-04-26T21:53:10.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:53:09 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]: dispatch 2024-04-26T21:53:10.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:53:09 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 7]}]: dispatch 2024-04-26T21:53:10.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:53:09 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 6]}]: dispatch 2024-04-26T21:53:10.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:53:09 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 8]}]: dispatch 2024-04-26T21:53:10.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:53:09 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 6]}]: dispatch 2024-04-26T21:53:10.192 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:53:09 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2a", "id": [3, 8]}]: dispatch 2024-04-26T21:53:10.192 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:53:09 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]: dispatch 2024-04-26T21:53:10.192 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:53:09 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]: dispatch 2024-04-26T21:53:10.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:53:09 smithi076 ceph-mon[31823]: from='client.? 172.21.15.72:0/3346018856' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-26T21:53:10.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:53:09 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]: dispatch 2024-04-26T21:53:10.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:53:09 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 1]}]: dispatch 2024-04-26T21:53:10.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:53:09 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]: dispatch 2024-04-26T21:53:10.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:53:09 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 7]}]: dispatch 2024-04-26T21:53:10.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:53:09 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 6]}]: dispatch 2024-04-26T21:53:10.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:53:09 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 8]}]: dispatch 2024-04-26T21:53:10.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:53:09 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 6]}]: dispatch 2024-04-26T21:53:10.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:53:09 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2a", "id": [3, 8]}]: dispatch 2024-04-26T21:53:10.316 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:53:09 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]: dispatch 2024-04-26T21:53:10.316 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:53:09 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]: dispatch 2024-04-26T21:53:10.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:53:09 smithi072 ceph-mon[28138]: from='client.? 172.21.15.72:0/3346018856' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-26T21:53:10.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:53:09 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]: dispatch 2024-04-26T21:53:10.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:53:09 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 1]}]: dispatch 2024-04-26T21:53:10.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:53:09 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]: dispatch 2024-04-26T21:53:10.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:53:09 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 7]}]: dispatch 2024-04-26T21:53:10.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:53:09 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 6]}]: dispatch 2024-04-26T21:53:10.323 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:53:09 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 8]}]: dispatch 2024-04-26T21:53:10.323 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:53:09 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 6]}]: dispatch 2024-04-26T21:53:10.323 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:53:09 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2a", "id": [3, 8]}]: dispatch 2024-04-26T21:53:10.323 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:53:09 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]: dispatch 2024-04-26T21:53:10.323 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:53:09 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]: dispatch 2024-04-26T21:53:10.517 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":17,"flags":0,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6800","nonce":588150172},{"type":"v1","addr":"172.21.15.72:6801","nonce":588150172}]},"active_addr":"172.21.15.72:6801/588150172","active_change":"2024-04-26T21:43:09.603138+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":34109,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.72:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.72:0","nonce":834906530}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.72:0","nonce":355605634}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.72:0","nonce":2799657309}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.72:0","nonce":4234165608}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.72:0","nonce":2253420320}]}]} 2024-04-26T21:53:10.522 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-04-26T21:53:10.522 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-04-26T21:53:10.523 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid a1780950-0415-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-26T21:53:11.019 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/a1780950-0415-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:53:11.181 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:53:10 smithi072 ceph-mon[28138]: pgmap v377: 97 pgs: 97 active+clean; 632 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.7 KiB/s wr, 25 op/s 2024-04-26T21:53:11.182 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:53:10 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]': finished 2024-04-26T21:53:11.182 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:53:10 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 1]}]': finished 2024-04-26T21:53:11.182 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:53:10 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2024-04-26T21:53:11.182 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:53:10 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 7]}]': finished 2024-04-26T21:53:11.182 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:53:10 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 6]}]': finished 2024-04-26T21:53:11.182 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:53:10 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 8]}]': finished 2024-04-26T21:53:11.183 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:53:10 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 6]}]': finished 2024-04-26T21:53:11.183 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:53:10 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2a", "id": [3, 8]}]': finished 2024-04-26T21:53:11.183 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:53:10 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]': finished 2024-04-26T21:53:11.183 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:53:10 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]': finished 2024-04-26T21:53:11.183 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:53:10 smithi072 ceph-mon[28138]: osdmap e88: 12 total, 12 up, 12 in 2024-04-26T21:53:11.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:53:10 smithi164 ceph-mon[31916]: pgmap v377: 97 pgs: 97 active+clean; 632 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.7 KiB/s wr, 25 op/s 2024-04-26T21:53:11.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:53:10 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]': finished 2024-04-26T21:53:11.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:53:10 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 1]}]': finished 2024-04-26T21:53:11.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:53:10 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2024-04-26T21:53:11.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:53:10 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 7]}]': finished 2024-04-26T21:53:11.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:53:10 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 6]}]': finished 2024-04-26T21:53:11.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:53:10 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 8]}]': finished 2024-04-26T21:53:11.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:53:10 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 6]}]': finished 2024-04-26T21:53:11.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:53:10 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2a", "id": [3, 8]}]': finished 2024-04-26T21:53:11.192 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:53:10 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]': finished 2024-04-26T21:53:11.192 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:53:10 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]': finished 2024-04-26T21:53:11.192 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:53:10 smithi164 ceph-mon[31916]: osdmap e88: 12 total, 12 up, 12 in 2024-04-26T21:53:11.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:53:10 smithi076 ceph-mon[31823]: pgmap v377: 97 pgs: 97 active+clean; 632 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.7 KiB/s wr, 25 op/s 2024-04-26T21:53:11.375 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:53:10 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]': finished 2024-04-26T21:53:11.375 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:53:10 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 1]}]': finished 2024-04-26T21:53:11.375 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:53:10 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2024-04-26T21:53:11.375 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:53:10 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 7]}]': finished 2024-04-26T21:53:11.375 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:53:10 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 6]}]': finished 2024-04-26T21:53:11.375 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:53:10 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 8]}]': finished 2024-04-26T21:53:11.375 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:53:10 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 6]}]': finished 2024-04-26T21:53:11.375 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:53:10 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2a", "id": [3, 8]}]': finished 2024-04-26T21:53:11.375 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:53:10 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]': finished 2024-04-26T21:53:11.375 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:53:10 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]': finished 2024-04-26T21:53:11.376 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:53:10 smithi076 ceph-mon[31823]: osdmap e88: 12 total, 12 up, 12 in 2024-04-26T21:53:12.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:53:11 smithi164 ceph-mon[31916]: osdmap e89: 12 total, 12 up, 12 in 2024-04-26T21:53:12.192 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:53:11 smithi072 ceph-mon[28138]: osdmap e89: 12 total, 12 up, 12 in 2024-04-26T21:53:12.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:53:11 smithi076 ceph-mon[31823]: osdmap e89: 12 total, 12 up, 12 in 2024-04-26T21:53:12.976 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:53:12 smithi072 ceph-mon[28138]: pgmap v380: 97 pgs: 97 active+clean; 632 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.0 KiB/s wr, 14 op/s 2024-04-26T21:53:13.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:53:12 smithi164 ceph-mon[31916]: pgmap v380: 97 pgs: 97 active+clean; 632 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.0 KiB/s wr, 14 op/s 2024-04-26T21:53:13.252 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:13.251+0000 7f01f8200700 1 -- 172.21.15.72:0/3796541400 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f01f010b640 msgr2=0x7f01f010ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:13.252 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:13.251+0000 7f01f8200700 1 --2- 172.21.15.72:0/3796541400 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f01f010b640 0x7f01f010ba20 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f01e4009990 tx=0x7f01e4009ca0 comp rx=0 tx=0).stop 2024-04-26T21:53:13.252 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:13.251+0000 7f01f8200700 1 -- 172.21.15.72:0/3796541400 shutdown_connections 2024-04-26T21:53:13.252 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:13.251+0000 7f01f8200700 1 --2- 172.21.15.72:0/3796541400 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f01f010b640 0x7f01f010ba20 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:13.252 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:13.251+0000 7f01f8200700 1 --2- 172.21.15.72:0/3796541400 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f01f0106000 0x7f01f0112510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:13.252 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:13.251+0000 7f01f8200700 1 --2- 172.21.15.72:0/3796541400 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f01f0105640 0x7f01f0105ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:13.252 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:13.251+0000 7f01f8200700 1 -- 172.21.15.72:0/3796541400 >> 172.21.15.72:0/3796541400 conn(0x7f01f01010f0 msgr2=0x7f01f0103510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:53:13.252 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:13.251+0000 7f01f8200700 1 -- 172.21.15.72:0/3796541400 shutdown_connections 2024-04-26T21:53:13.252 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:13.251+0000 7f01f8200700 1 -- 172.21.15.72:0/3796541400 wait complete. 2024-04-26T21:53:13.254 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:13.252+0000 7f01f8200700 1 Processor -- start 2024-04-26T21:53:13.254 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:13.253+0000 7f01f8200700 1 -- start start 2024-04-26T21:53:13.254 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:13.253+0000 7f01f8200700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f01f0105640 0x7f01f01a1860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:13.254 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:13.253+0000 7f01f8200700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f01f0106000 0x7f01f01a1da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:13.254 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:13.253+0000 7f01f8200700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f01f010b640 0x7f01f019ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:13.254 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:13.253+0000 7f01f8200700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f01f01156f0 con 0x7f01f0105640 2024-04-26T21:53:13.254 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:13.253+0000 7f01f8200700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f01f0115570 con 0x7f01f0106000 2024-04-26T21:53:13.254 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:13.253+0000 7f01f8200700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f01f0115870 con 0x7f01f010b640 2024-04-26T21:53:13.254 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:13.253+0000 7f01f5800700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f01f0105640 0x7f01f01a1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:13.254 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:13.253+0000 7f01f5800700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f01f0105640 0x7f01f01a1860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:56562/0 (socket says 172.21.15.72:56562) 2024-04-26T21:53:13.254 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:13.253+0000 7f01f5800700 1 -- 172.21.15.72:0/1565386616 learned_addr learned my addr 172.21.15.72:0/1565386616 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:53:13.254 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:13.254+0000 7f01f4e00700 1 --2- 172.21.15.72:0/1565386616 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f01f0106000 0x7f01f01a1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:13.255 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:13.254+0000 7f01f5800700 1 -- 172.21.15.72:0/1565386616 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f01f010b640 msgr2=0x7f01f019ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-26T21:53:13.255 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:13.254+0000 7f01f5800700 1 --2- 172.21.15.72:0/1565386616 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f01f010b640 0x7f01f019ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:13.256 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:13.254+0000 7f01f5800700 1 -- 172.21.15.72:0/1565386616 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f01f0106000 msgr2=0x7f01f01a1da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:13.256 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:13.254+0000 7f01f5800700 1 --2- 172.21.15.72:0/1565386616 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f01f0106000 0x7f01f01a1da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:13.256 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:13.254+0000 7f01f5800700 1 -- 172.21.15.72:0/1565386616 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01f019c1d0 con 0x7f01f0105640 2024-04-26T21:53:13.256 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:13.254+0000 7f01f5800700 1 --2- 172.21.15.72:0/1565386616 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f01f0105640 0x7f01f01a1860 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f01e4000c00 tx=0x7f01e4009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:53:13.256 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:13.254+0000 7f01ee000700 1 -- 172.21.15.72:0/1565386616 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01e4027070 con 0x7f01f0105640 2024-04-26T21:53:13.256 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:13.254+0000 7f01ee000700 1 -- 172.21.15.72:0/1565386616 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f01e400fc70 con 0x7f01f0105640 2024-04-26T21:53:13.256 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:13.254+0000 7f01f8200700 1 -- 172.21.15.72:0/1565386616 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01f019c460 con 0x7f01f0105640 2024-04-26T21:53:13.256 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:13.255+0000 7f01ee000700 1 -- 172.21.15.72:0/1565386616 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01e4022d40 con 0x7f01f0105640 2024-04-26T21:53:13.256 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:13.255+0000 7f01f8200700 1 -- 172.21.15.72:0/1565386616 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01f01a8730 con 0x7f01f0105640 2024-04-26T21:53:13.256 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:13.256+0000 7f01f8200700 1 -- 172.21.15.72:0/1565386616 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01f006a110 con 0x7f01f0105640 2024-04-26T21:53:13.258 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:13.258+0000 7f01ee000700 1 -- 172.21.15.72:0/1565386616 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f01e402a070 con 0x7f01f0105640 2024-04-26T21:53:13.259 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:13.259+0000 7f01ee000700 1 --2- 172.21.15.72:0/1565386616 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f01d80a3d50 0x7f01d80a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:13.260 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:13.259+0000 7f01ee000700 1 -- 172.21.15.72:0/1565386616 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8269+0+0 (secure 0 0 0) 0x7f01e40d1a20 con 0x7f01f0105640 2024-04-26T21:53:13.260 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:13.260+0000 7f01f4e00700 1 --2- 172.21.15.72:0/1565386616 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f01d80a3d50 0x7f01d80a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:13.260 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:13.260+0000 7f01f4e00700 1 --2- 172.21.15.72:0/1565386616 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f01d80a3d50 0x7f01d80a6210 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f01dc005fd0 tx=0x7f01dc005f60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:53:13.262 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:13.262+0000 7f01ee000700 1 -- 172.21.15.72:0/1565386616 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f01e4031070 con 0x7f01f0105640 2024-04-26T21:53:13.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:53:12 smithi076 ceph-mon[31823]: pgmap v380: 97 pgs: 97 active+clean; 632 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.0 KiB/s wr, 14 op/s 2024-04-26T21:53:13.489 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:13.489+0000 7f01f8200700 1 -- 172.21.15.72:0/1565386616 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f01f019cf50 con 0x7f01f0105640 2024-04-26T21:53:13.493 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:13.493+0000 7f01ee000700 1 -- 172.21.15.72:0/1565386616 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+22642 (secure 0 0 0) 0x7f01e4013070 con 0x7f01f0105640 2024-04-26T21:53:13.494 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T21:53:13.494 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":89,"fsid":"a1780950-0415-11ef-bc93-c7b262605968","created":"2024-04-26T21:42:01.504173+0000","modified":"2024-04-26T21:53:10.900013+0000","last_up_change":"2024-04-26T21:51:43.482437+0000","last_in_change":"2024-04-26T21:51:18.468587+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:46:29.785427+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:52:48.320391+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T21:52:49.517493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"04e06884-b355-4762-9b29-9034b7201e9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6802","nonce":1416765179},{"type":"v1","addr":"172.21.15.72:6803","nonce":1416765179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6804","nonce":1416765179},{"type":"v1","addr":"172.21.15.72:6805","nonce":1416765179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6808","nonce":1416765179},{"type":"v1","addr":"172.21.15.72:6809","nonce":1416765179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6806","nonce":1416765179},{"type":"v1","addr":"172.21.15.72:6807","nonce":1416765179}]},"public_addr":"172.21.15.72:6803/1416765179","cluster_addr":"172.21.15.72:6805/1416765179","heartbeat_back_addr":"172.21.15.72:6809/1416765179","heartbeat_front_addr":"172.21.15.72:6807/1416765179","state":["exists","up"]},{"osd":1,"uuid":"d75a479a-4a2e-48e9-a01e-2b1c9617416a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":755426949},{"type":"v1","addr":"172.21.15.76:6801","nonce":755426949}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":755426949},{"type":"v1","addr":"172.21.15.76:6803","nonce":755426949}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":755426949},{"type":"v1","addr":"172.21.15.76:6807","nonce":755426949}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":755426949},{"type":"v1","addr":"172.21.15.76:6805","nonce":755426949}]},"public_addr":"172.21.15.76:6801/755426949","cluster_addr":"172.21.15.76:6803/755426949","heartbeat_back_addr":"172.21.15.76:6807/755426949","heartbeat_front_addr":"172.21.15.76:6805/755426949","state":["exists","up"]},{"osd":2,"uuid":"b6f8da58-4428-4e8c-a944-7ea5645d6d92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":852265973},{"type":"v1","addr":"172.21.15.164:6801","nonce":852265973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":852265973},{"type":"v1","addr":"172.21.15.164:6803","nonce":852265973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":852265973},{"type":"v1","addr":"172.21.15.164:6807","nonce":852265973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":852265973},{"type":"v1","addr":"172.21.15.164:6805","nonce":852265973}]},"public_addr":"172.21.15.164:6801/852265973","cluster_addr":"172.21.15.164:6803/852265973","heartbeat_back_addr":"172.21.15.164:6807/852265973","heartbeat_front_addr":"172.21.15.164:6805/852265973","state":["exists","up"]},{"osd":3,"uuid":"76d83409-649e-41e5-b4f4-43bbd5b00ea5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6810","nonce":4027763789},{"type":"v1","addr":"172.21.15.72:6811","nonce":4027763789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6812","nonce":4027763789},{"type":"v1","addr":"172.21.15.72:6813","nonce":4027763789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6816","nonce":4027763789},{"type":"v1","addr":"172.21.15.72:6817","nonce":4027763789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6814","nonce":4027763789},{"type":"v1","addr":"172.21.15.72:6815","nonce":4027763789}]},"public_addr":"172.21.15.72:6811/4027763789","cluster_addr":"172.21.15.72:6813/4027763789","heartbeat_back_addr":"172.21.15.72:6817/4027763789","heartbeat_front_addr":"172.21.15.72:6815/4027763789","state":["exists","up"]},{"osd":4,"uuid":"d2cfeea7-1661-4156-bdcd-f770a8d1a878","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":3718867827},{"type":"v1","addr":"172.21.15.76:6809","nonce":3718867827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":3718867827},{"type":"v1","addr":"172.21.15.76:6811","nonce":3718867827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":3718867827},{"type":"v1","addr":"172.21.15.76:6815","nonce":3718867827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":3718867827},{"type":"v1","addr":"172.21.15.76:6813","nonce":3718867827}]},"public_addr":"172.21.15.76:6809/3718867827","cluster_addr":"172.21.15.76:6811/3718867827","heartbeat_back_addr":"172.21.15.76:6815/3718867827","heartbeat_front_addr":"172.21.15.76:6813/3718867827","state":["exists","up"]},{"osd":5,"uuid":"f3896ad6-a040-4ea3-a431-ff9b7fc05c36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":2543954772},{"type":"v1","addr":"172.21.15.164:6809","nonce":2543954772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":2543954772},{"type":"v1","addr":"172.21.15.164:6811","nonce":2543954772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":2543954772},{"type":"v1","addr":"172.21.15.164:6815","nonce":2543954772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":2543954772},{"type":"v1","addr":"172.21.15.164:6813","nonce":2543954772}]},"public_addr":"172.21.15.164:6809/2543954772","cluster_addr":"172.21.15.164:6811/2543954772","heartbeat_back_addr":"172.21.15.164:6815/2543954772","heartbeat_front_addr":"172.21.15.164:6813/2543954772","state":["exists","up"]},{"osd":6,"uuid":"d5d56650-be7e-4ba5-ba3a-d7d473294dab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6818","nonce":2080659027},{"type":"v1","addr":"172.21.15.72:6819","nonce":2080659027}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6820","nonce":2080659027},{"type":"v1","addr":"172.21.15.72:6821","nonce":2080659027}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6824","nonce":2080659027},{"type":"v1","addr":"172.21.15.72:6825","nonce":2080659027}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6822","nonce":2080659027},{"type":"v1","addr":"172.21.15.72:6823","nonce":2080659027}]},"public_addr":"172.21.15.72:6819/2080659027","cluster_addr":"172.21.15.72:6821/2080659027","heartbeat_back_addr":"172.21.15.72:6825/2080659027","heartbeat_front_addr":"172.21.15.72:6823/2080659027","state":["exists","up"]},{"osd":7,"uuid":"18975b5a-9309-4a0f-9f37-3ce09c2e745d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":3144631323},{"type":"v1","addr":"172.21.15.76:6817","nonce":3144631323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":3144631323},{"type":"v1","addr":"172.21.15.76:6819","nonce":3144631323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":3144631323},{"type":"v1","addr":"172.21.15.76:6823","nonce":3144631323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":3144631323},{"type":"v1","addr":"172.21.15.76:6821","nonce":3144631323}]},"public_addr":"172.21.15.76:6817/3144631323","cluster_addr":"172.21.15.76:6819/3144631323","heartbeat_back_addr":"172.21.15.76:6823/3144631323","heartbeat_front_addr":"172.21.15.76:6821/3144631323","state":["exists","up"]},{"osd":8,"uuid":"779e9c1e-8e97-48d4-a854-68ca13f38e0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":627888883},{"type":"v1","addr":"172.21.15.164:6817","nonce":627888883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":627888883},{"type":"v1","addr":"172.21.15.164:6819","nonce":627888883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":627888883},{"type":"v1","addr":"172.21.15.164:6823","nonce":627888883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":627888883},{"type":"v1","addr":"172.21.15.164:6821","nonce":627888883}]},"public_addr":"172.21.15.164:6817/627888883","cluster_addr":"172.21.15.164:6819/627888883","heartbeat_back_addr":"172.21.15.164:6823/627888883","heartbeat_front_addr":"172.21.15.164:6821/627888883","state":["exists","up"]},{"osd":9,"uuid":"d0317850-7b2d-4850-b2cf-0b919505c1fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6826","nonce":1273118485},{"type":"v1","addr":"172.21.15.72:6827","nonce":1273118485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6828","nonce":1273118485},{"type":"v1","addr":"172.21.15.72:6829","nonce":1273118485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6832","nonce":1273118485},{"type":"v1","addr":"172.21.15.72:6833","nonce":1273118485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6830","nonce":1273118485},{"type":"v1","addr":"172.21.15.72:6831","nonce":1273118485}]},"public_addr":"172.21.15.72:6827/1273118485","cluster_addr":"172.21.15.72:6829/1273118485","heartbeat_back_addr":"172.21.15.72:6833/1273118485","heartbeat_front_addr":"172.21.15.72:6831/1273118485","state":["exists","up"]},{"osd":10,"uuid":"e0aa38d8-6011-4d2a-b67b-5bff6d999e47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":2808954459},{"type":"v1","addr":"172.21.15.76:6825","nonce":2808954459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":2808954459},{"type":"v1","addr":"172.21.15.76:6827","nonce":2808954459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":2808954459},{"type":"v1","addr":"172.21.15.76:6831","nonce":2808954459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":2808954459},{"type":"v1","addr":"172.21.15.76:6829","nonce":2808954459}]},"public_addr":"172.21.15.76:6825/2808954459","cluster_addr":"172.21.15.76:6827/2808954459","heartbeat_back_addr":"172.21.15.76:6831/2808954459","heartbeat_front_addr":"172.21.15.76:6829/2808954459","state":["exists","up"]},{"osd":11,"uuid":"981b3d4d-916e-4937-9916-7cc21a891e43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":2354951784},{"type":"v1","addr":"172.21.15.164:6825","nonce":2354951784}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":2354951784},{"type":"v1","addr":"172.21.15.164:6827","nonce":2354951784}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":2354951784},{"type":"v1","addr":"172.21.15.164:6831","nonce":2354951784}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":2354951784},{"type":"v1","addr":"172.21.15.164:6829","nonce":2354951784}]},"public_addr":"172.21.15.164:6825/2354951784","cluster_addr":"172.21.15.164:6827/2354951784","heartbeat_back_addr":"172.21.15.164:6831/2354951784","heartbeat_front_addr":"172.21.15.164:6829/2354951784","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:18.997924+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:52.439081+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:46:25.893288+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:47:00.112807+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:47:35.116874+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:48:09.216344+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:48:45.028853+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:49:19.905397+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:49:55.030536+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:50:30.767662+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:51:06.423267+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:51:41.631211+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.1a","mappings":[{"from":5,"to":7}]},{"pgid":"3.1b","mappings":[{"from":4,"to":6}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.28","mappings":[{"from":5,"to":6}]},{"pgid":"3.2a","mappings":[{"from":3,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.72:0/416277784":"2024-04-27T21:43:09.602791+0000","172.21.15.72:6800/4010358672":"2024-04-27T21:43:09.602791+0000","172.21.15.72:6801/1012617371":"2024-04-27T21:42:34.684837+0000","172.21.15.72:6801/4010358672":"2024-04-27T21:43:09.602791+0000","172.21.15.72:0/3555448179":"2024-04-27T21:43:09.602791+0000","172.21.15.72:6800/1012617371":"2024-04-27T21:42:34.684837+0000","172.21.15.72:0/2920378761":"2024-04-27T21:42:34.684837+0000","172.21.15.72:0/1933740041":"2024-04-27T21:43:09.602791+0000","172.21.15.72:0/3225295775":"2024-04-27T21:42:34.684837+0000","172.21.15.72:0/2326016180":"2024-04-27T21:42:34.684837+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:53:13.496 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:13.496+0000 7f01f8200700 1 -- 172.21.15.72:0/1565386616 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f01d80a3d50 msgr2=0x7f01d80a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:13.496 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:13.496+0000 7f01f8200700 1 --2- 172.21.15.72:0/1565386616 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f01d80a3d50 0x7f01d80a6210 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f01dc005fd0 tx=0x7f01dc005f60 comp rx=0 tx=0).stop 2024-04-26T21:53:13.497 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:13.496+0000 7f01f8200700 1 -- 172.21.15.72:0/1565386616 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f01f0105640 msgr2=0x7f01f01a1860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:13.497 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:13.496+0000 7f01f8200700 1 --2- 172.21.15.72:0/1565386616 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f01f0105640 0x7f01f01a1860 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f01e4000c00 tx=0x7f01e4009f90 comp rx=0 tx=0).stop 2024-04-26T21:53:13.497 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:13.496+0000 7f01f8200700 1 -- 172.21.15.72:0/1565386616 shutdown_connections 2024-04-26T21:53:13.497 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:13.496+0000 7f01f8200700 1 --2- 172.21.15.72:0/1565386616 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f01d80a3d50 0x7f01d80a6210 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:13.497 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:13.496+0000 7f01f8200700 1 --2- 172.21.15.72:0/1565386616 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f01f0105640 0x7f01f01a1860 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:13.497 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:13.496+0000 7f01f8200700 1 --2- 172.21.15.72:0/1565386616 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f01f010b640 0x7f01f019ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:13.497 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:13.496+0000 7f01f8200700 1 --2- 172.21.15.72:0/1565386616 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f01f0106000 0x7f01f01a1da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:13.497 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:13.496+0000 7f01f8200700 1 -- 172.21.15.72:0/1565386616 >> 172.21.15.72:0/1565386616 conn(0x7f01f01010f0 msgr2=0x7f01f0101980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:53:13.497 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:13.496+0000 7f01f8200700 1 -- 172.21.15.72:0/1565386616 shutdown_connections 2024-04-26T21:53:13.497 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:13.496+0000 7f01f8200700 1 -- 172.21.15.72:0/1565386616 wait complete. 2024-04-26T21:53:14.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:53:13 smithi076 ceph-mon[31823]: from='client.? 172.21.15.72:0/1565386616' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:53:14.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:53:13 smithi072 ceph-mon[28138]: from='client.? 172.21.15.72:0/1565386616' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:53:14.385 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-04-26T21:53:14.385 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid a1780950-0415-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-26T21:53:14.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:53:13 smithi164 ceph-mon[31916]: from='client.? 172.21.15.72:0/1565386616' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:53:14.827 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/a1780950-0415-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:53:15.207 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:53:14 smithi072 ceph-mon[28138]: pgmap v381: 97 pgs: 97 active+clean; 632 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.9 KiB/s wr, 8 op/s 2024-04-26T21:53:15.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:53:14 smithi076 ceph-mon[31823]: pgmap v381: 97 pgs: 97 active+clean; 632 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.9 KiB/s wr, 8 op/s 2024-04-26T21:53:15.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:53:14 smithi164 ceph-mon[31916]: pgmap v381: 97 pgs: 97 active+clean; 632 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.9 KiB/s wr, 8 op/s 2024-04-26T21:53:17.208 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:17.206+0000 7f82f9a00700 1 -- 172.21.15.72:0/2841325577 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f82f41094c0 msgr2=0x7f82f41098a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:17.208 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:17.206+0000 7f82f9a00700 1 --2- 172.21.15.72:0/2841325577 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f82f41094c0 0x7f82f41098a0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f82e4009990 tx=0x7f82e4009ca0 comp rx=0 tx=0).stop 2024-04-26T21:53:17.208 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:17.207+0000 7f82f9a00700 1 -- 172.21.15.72:0/2841325577 shutdown_connections 2024-04-26T21:53:17.208 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:17.207+0000 7f82f9a00700 1 --2- 172.21.15.72:0/2841325577 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f82f41094c0 0x7f82f41098a0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:17.208 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:17.207+0000 7f82f9a00700 1 --2- 172.21.15.72:0/2841325577 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f82f4103e30 0x7f82f4110290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:17.208 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:17.207+0000 7f82f9a00700 1 --2- 172.21.15.72:0/2841325577 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f82f4103470 0x7f82f41038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:17.208 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:17.207+0000 7f82f9a00700 1 -- 172.21.15.72:0/2841325577 >> 172.21.15.72:0/2841325577 conn(0x7f82f407a580 msgr2=0x7f82f407a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:53:17.208 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:17.207+0000 7f82f9a00700 1 -- 172.21.15.72:0/2841325577 shutdown_connections 2024-04-26T21:53:17.209 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:17.207+0000 7f82f9a00700 1 -- 172.21.15.72:0/2841325577 wait complete. 2024-04-26T21:53:17.209 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:53:16 smithi072 ceph-mon[28138]: pgmap v382: 97 pgs: 97 active+clean; 632 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 1.9 KiB/s wr, 13 op/s; 0 B/s, 0 objects/s recovering 2024-04-26T21:53:17.211 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:17.208+0000 7f82f9a00700 1 Processor -- start 2024-04-26T21:53:17.211 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:17.208+0000 7f82f9a00700 1 -- start start 2024-04-26T21:53:17.211 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:17.208+0000 7f82f9a00700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f82f4103470 0x7f82f4075720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:17.211 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:17.208+0000 7f82f9a00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f82f4103e30 0x7f82f4075c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:17.211 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:17.208+0000 7f82f9a00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f82f41094c0 0x7f82f40761a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:17.211 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:17.208+0000 7f82f9a00700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f82f41134f0 con 0x7f82f41094c0 2024-04-26T21:53:17.211 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:17.208+0000 7f82f9a00700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f82f4113370 con 0x7f82f4103470 2024-04-26T21:53:17.211 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:17.208+0000 7f82f9a00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f82f4113670 con 0x7f82f4103e30 2024-04-26T21:53:17.211 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:17.209+0000 7f82f3400700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f82f41094c0 0x7f82f40761a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:17.211 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:17.209+0000 7f82f3400700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f82f41094c0 0x7f82f40761a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:56570/0 (socket says 172.21.15.72:56570) 2024-04-26T21:53:17.212 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:17.209+0000 7f82f3400700 1 -- 172.21.15.72:0/2153517706 learned_addr learned my addr 172.21.15.72:0/2153517706 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:53:17.212 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:17.209+0000 7f82f2a00700 1 --2- 172.21.15.72:0/2153517706 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f82f4103470 0x7f82f4075720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:17.212 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:17.209+0000 7f82ea000700 1 --2- 172.21.15.72:0/2153517706 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f82f4103e30 0x7f82f4075c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:17.212 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:17.209+0000 7f82f3400700 1 -- 172.21.15.72:0/2153517706 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f82f4103e30 msgr2=0x7f82f4075c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:17.212 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:17.209+0000 7f82f3400700 1 --2- 172.21.15.72:0/2153517706 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f82f4103e30 0x7f82f4075c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:17.212 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:17.209+0000 7f82f3400700 1 -- 172.21.15.72:0/2153517706 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f82f4103470 msgr2=0x7f82f4075720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:17.212 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:17.209+0000 7f82f3400700 1 --2- 172.21.15.72:0/2153517706 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f82f4103470 0x7f82f4075720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:17.212 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:17.209+0000 7f82f3400700 1 -- 172.21.15.72:0/2153517706 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82f41a7f50 con 0x7f82f41094c0 2024-04-26T21:53:17.212 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:17.210+0000 7f82f3400700 1 --2- 172.21.15.72:0/2153517706 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f82f41094c0 0x7f82f40761a0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f82ec007d70 tx=0x7f82ec00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:53:17.212 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:17.210+0000 7f82ebe00700 1 -- 172.21.15.72:0/2153517706 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82ec016020 con 0x7f82f41094c0 2024-04-26T21:53:17.212 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:17.210+0000 7f82ebe00700 1 -- 172.21.15.72:0/2153517706 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f82ec00ad40 con 0x7f82f41094c0 2024-04-26T21:53:17.212 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:17.210+0000 7f82f9a00700 1 -- 172.21.15.72:0/2153517706 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82f41a8240 con 0x7f82f41094c0 2024-04-26T21:53:17.212 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:17.210+0000 7f82ebe00700 1 -- 172.21.15.72:0/2153517706 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82ec0115e0 con 0x7f82f41094c0 2024-04-26T21:53:17.212 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:17.210+0000 7f82f9a00700 1 -- 172.21.15.72:0/2153517706 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82f41a8780 con 0x7f82f41094c0 2024-04-26T21:53:17.213 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:17.211+0000 7f82f9a00700 1 -- 172.21.15.72:0/2153517706 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82f410b930 con 0x7f82f41094c0 2024-04-26T21:53:17.213 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:17.213+0000 7f82ebe00700 1 -- 172.21.15.72:0/2153517706 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f82ec019020 con 0x7f82f41094c0 2024-04-26T21:53:17.215 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:17.215+0000 7f82ebe00700 1 --2- 172.21.15.72:0/2153517706 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f82d80a3d50 0x7f82d80a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:17.215 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:17.215+0000 7f82ebe00700 1 -- 172.21.15.72:0/2153517706 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8269+0+0 (secure 0 0 0) 0x7f82ec0d03a0 con 0x7f82f41094c0 2024-04-26T21:53:17.215 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:17.215+0000 7f82f2a00700 1 --2- 172.21.15.72:0/2153517706 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f82d80a3d50 0x7f82d80a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:17.216 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:17.216+0000 7f82f2a00700 1 --2- 172.21.15.72:0/2153517706 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f82d80a3d50 0x7f82d80a6210 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f82e4005b00 tx=0x7f82e4005a90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:53:17.220 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:17.220+0000 7f82ebe00700 1 -- 172.21.15.72:0/2153517706 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f82ec020020 con 0x7f82f41094c0 2024-04-26T21:53:17.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:53:16 smithi076 ceph-mon[31823]: pgmap v382: 97 pgs: 97 active+clean; 632 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 1.9 KiB/s wr, 13 op/s; 0 B/s, 0 objects/s recovering 2024-04-26T21:53:17.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:53:16 smithi164 ceph-mon[31916]: pgmap v382: 97 pgs: 97 active+clean; 632 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 1.9 KiB/s wr, 13 op/s; 0 B/s, 0 objects/s recovering 2024-04-26T21:53:17.446 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:17.445+0000 7f82f9a00700 1 -- 172.21.15.72:0/2153517706 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f82f410b310 con 0x7f82f41094c0 2024-04-26T21:53:17.449 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:17.449+0000 7f82ebe00700 1 -- 172.21.15.72:0/2153517706 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+22642 (secure 0 0 0) 0x7f82ec09ccb0 con 0x7f82f41094c0 2024-04-26T21:53:17.450 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T21:53:17.450 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":89,"fsid":"a1780950-0415-11ef-bc93-c7b262605968","created":"2024-04-26T21:42:01.504173+0000","modified":"2024-04-26T21:53:10.900013+0000","last_up_change":"2024-04-26T21:51:43.482437+0000","last_in_change":"2024-04-26T21:51:18.468587+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:46:29.785427+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:52:48.320391+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T21:52:49.517493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"04e06884-b355-4762-9b29-9034b7201e9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6802","nonce":1416765179},{"type":"v1","addr":"172.21.15.72:6803","nonce":1416765179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6804","nonce":1416765179},{"type":"v1","addr":"172.21.15.72:6805","nonce":1416765179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6808","nonce":1416765179},{"type":"v1","addr":"172.21.15.72:6809","nonce":1416765179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6806","nonce":1416765179},{"type":"v1","addr":"172.21.15.72:6807","nonce":1416765179}]},"public_addr":"172.21.15.72:6803/1416765179","cluster_addr":"172.21.15.72:6805/1416765179","heartbeat_back_addr":"172.21.15.72:6809/1416765179","heartbeat_front_addr":"172.21.15.72:6807/1416765179","state":["exists","up"]},{"osd":1,"uuid":"d75a479a-4a2e-48e9-a01e-2b1c9617416a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":755426949},{"type":"v1","addr":"172.21.15.76:6801","nonce":755426949}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":755426949},{"type":"v1","addr":"172.21.15.76:6803","nonce":755426949}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":755426949},{"type":"v1","addr":"172.21.15.76:6807","nonce":755426949}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":755426949},{"type":"v1","addr":"172.21.15.76:6805","nonce":755426949}]},"public_addr":"172.21.15.76:6801/755426949","cluster_addr":"172.21.15.76:6803/755426949","heartbeat_back_addr":"172.21.15.76:6807/755426949","heartbeat_front_addr":"172.21.15.76:6805/755426949","state":["exists","up"]},{"osd":2,"uuid":"b6f8da58-4428-4e8c-a944-7ea5645d6d92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":852265973},{"type":"v1","addr":"172.21.15.164:6801","nonce":852265973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":852265973},{"type":"v1","addr":"172.21.15.164:6803","nonce":852265973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":852265973},{"type":"v1","addr":"172.21.15.164:6807","nonce":852265973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":852265973},{"type":"v1","addr":"172.21.15.164:6805","nonce":852265973}]},"public_addr":"172.21.15.164:6801/852265973","cluster_addr":"172.21.15.164:6803/852265973","heartbeat_back_addr":"172.21.15.164:6807/852265973","heartbeat_front_addr":"172.21.15.164:6805/852265973","state":["exists","up"]},{"osd":3,"uuid":"76d83409-649e-41e5-b4f4-43bbd5b00ea5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6810","nonce":4027763789},{"type":"v1","addr":"172.21.15.72:6811","nonce":4027763789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6812","nonce":4027763789},{"type":"v1","addr":"172.21.15.72:6813","nonce":4027763789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6816","nonce":4027763789},{"type":"v1","addr":"172.21.15.72:6817","nonce":4027763789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6814","nonce":4027763789},{"type":"v1","addr":"172.21.15.72:6815","nonce":4027763789}]},"public_addr":"172.21.15.72:6811/4027763789","cluster_addr":"172.21.15.72:6813/4027763789","heartbeat_back_addr":"172.21.15.72:6817/4027763789","heartbeat_front_addr":"172.21.15.72:6815/4027763789","state":["exists","up"]},{"osd":4,"uuid":"d2cfeea7-1661-4156-bdcd-f770a8d1a878","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":3718867827},{"type":"v1","addr":"172.21.15.76:6809","nonce":3718867827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":3718867827},{"type":"v1","addr":"172.21.15.76:6811","nonce":3718867827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":3718867827},{"type":"v1","addr":"172.21.15.76:6815","nonce":3718867827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":3718867827},{"type":"v1","addr":"172.21.15.76:6813","nonce":3718867827}]},"public_addr":"172.21.15.76:6809/3718867827","cluster_addr":"172.21.15.76:6811/3718867827","heartbeat_back_addr":"172.21.15.76:6815/3718867827","heartbeat_front_addr":"172.21.15.76:6813/3718867827","state":["exists","up"]},{"osd":5,"uuid":"f3896ad6-a040-4ea3-a431-ff9b7fc05c36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":2543954772},{"type":"v1","addr":"172.21.15.164:6809","nonce":2543954772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":2543954772},{"type":"v1","addr":"172.21.15.164:6811","nonce":2543954772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":2543954772},{"type":"v1","addr":"172.21.15.164:6815","nonce":2543954772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":2543954772},{"type":"v1","addr":"172.21.15.164:6813","nonce":2543954772}]},"public_addr":"172.21.15.164:6809/2543954772","cluster_addr":"172.21.15.164:6811/2543954772","heartbeat_back_addr":"172.21.15.164:6815/2543954772","heartbeat_front_addr":"172.21.15.164:6813/2543954772","state":["exists","up"]},{"osd":6,"uuid":"d5d56650-be7e-4ba5-ba3a-d7d473294dab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6818","nonce":2080659027},{"type":"v1","addr":"172.21.15.72:6819","nonce":2080659027}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6820","nonce":2080659027},{"type":"v1","addr":"172.21.15.72:6821","nonce":2080659027}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6824","nonce":2080659027},{"type":"v1","addr":"172.21.15.72:6825","nonce":2080659027}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6822","nonce":2080659027},{"type":"v1","addr":"172.21.15.72:6823","nonce":2080659027}]},"public_addr":"172.21.15.72:6819/2080659027","cluster_addr":"172.21.15.72:6821/2080659027","heartbeat_back_addr":"172.21.15.72:6825/2080659027","heartbeat_front_addr":"172.21.15.72:6823/2080659027","state":["exists","up"]},{"osd":7,"uuid":"18975b5a-9309-4a0f-9f37-3ce09c2e745d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":3144631323},{"type":"v1","addr":"172.21.15.76:6817","nonce":3144631323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":3144631323},{"type":"v1","addr":"172.21.15.76:6819","nonce":3144631323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":3144631323},{"type":"v1","addr":"172.21.15.76:6823","nonce":3144631323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":3144631323},{"type":"v1","addr":"172.21.15.76:6821","nonce":3144631323}]},"public_addr":"172.21.15.76:6817/3144631323","cluster_addr":"172.21.15.76:6819/3144631323","heartbeat_back_addr":"172.21.15.76:6823/3144631323","heartbeat_front_addr":"172.21.15.76:6821/3144631323","state":["exists","up"]},{"osd":8,"uuid":"779e9c1e-8e97-48d4-a854-68ca13f38e0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":627888883},{"type":"v1","addr":"172.21.15.164:6817","nonce":627888883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":627888883},{"type":"v1","addr":"172.21.15.164:6819","nonce":627888883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":627888883},{"type":"v1","addr":"172.21.15.164:6823","nonce":627888883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":627888883},{"type":"v1","addr":"172.21.15.164:6821","nonce":627888883}]},"public_addr":"172.21.15.164:6817/627888883","cluster_addr":"172.21.15.164:6819/627888883","heartbeat_back_addr":"172.21.15.164:6823/627888883","heartbeat_front_addr":"172.21.15.164:6821/627888883","state":["exists","up"]},{"osd":9,"uuid":"d0317850-7b2d-4850-b2cf-0b919505c1fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6826","nonce":1273118485},{"type":"v1","addr":"172.21.15.72:6827","nonce":1273118485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6828","nonce":1273118485},{"type":"v1","addr":"172.21.15.72:6829","nonce":1273118485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6832","nonce":1273118485},{"type":"v1","addr":"172.21.15.72:6833","nonce":1273118485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6830","nonce":1273118485},{"type":"v1","addr":"172.21.15.72:6831","nonce":1273118485}]},"public_addr":"172.21.15.72:6827/1273118485","cluster_addr":"172.21.15.72:6829/1273118485","heartbeat_back_addr":"172.21.15.72:6833/1273118485","heartbeat_front_addr":"172.21.15.72:6831/1273118485","state":["exists","up"]},{"osd":10,"uuid":"e0aa38d8-6011-4d2a-b67b-5bff6d999e47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":2808954459},{"type":"v1","addr":"172.21.15.76:6825","nonce":2808954459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":2808954459},{"type":"v1","addr":"172.21.15.76:6827","nonce":2808954459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":2808954459},{"type":"v1","addr":"172.21.15.76:6831","nonce":2808954459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":2808954459},{"type":"v1","addr":"172.21.15.76:6829","nonce":2808954459}]},"public_addr":"172.21.15.76:6825/2808954459","cluster_addr":"172.21.15.76:6827/2808954459","heartbeat_back_addr":"172.21.15.76:6831/2808954459","heartbeat_front_addr":"172.21.15.76:6829/2808954459","state":["exists","up"]},{"osd":11,"uuid":"981b3d4d-916e-4937-9916-7cc21a891e43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":2354951784},{"type":"v1","addr":"172.21.15.164:6825","nonce":2354951784}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":2354951784},{"type":"v1","addr":"172.21.15.164:6827","nonce":2354951784}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":2354951784},{"type":"v1","addr":"172.21.15.164:6831","nonce":2354951784}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":2354951784},{"type":"v1","addr":"172.21.15.164:6829","nonce":2354951784}]},"public_addr":"172.21.15.164:6825/2354951784","cluster_addr":"172.21.15.164:6827/2354951784","heartbeat_back_addr":"172.21.15.164:6831/2354951784","heartbeat_front_addr":"172.21.15.164:6829/2354951784","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:18.997924+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:52.439081+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:46:25.893288+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:47:00.112807+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:47:35.116874+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:48:09.216344+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:48:45.028853+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:49:19.905397+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:49:55.030536+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:50:30.767662+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:51:06.423267+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:51:41.631211+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.1a","mappings":[{"from":5,"to":7}]},{"pgid":"3.1b","mappings":[{"from":4,"to":6}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.28","mappings":[{"from":5,"to":6}]},{"pgid":"3.2a","mappings":[{"from":3,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.72:0/416277784":"2024-04-27T21:43:09.602791+0000","172.21.15.72:6800/4010358672":"2024-04-27T21:43:09.602791+0000","172.21.15.72:6801/1012617371":"2024-04-27T21:42:34.684837+0000","172.21.15.72:6801/4010358672":"2024-04-27T21:43:09.602791+0000","172.21.15.72:0/3555448179":"2024-04-27T21:43:09.602791+0000","172.21.15.72:6800/1012617371":"2024-04-27T21:42:34.684837+0000","172.21.15.72:0/2920378761":"2024-04-27T21:42:34.684837+0000","172.21.15.72:0/1933740041":"2024-04-27T21:43:09.602791+0000","172.21.15.72:0/3225295775":"2024-04-27T21:42:34.684837+0000","172.21.15.72:0/2326016180":"2024-04-27T21:42:34.684837+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:53:17.454 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:17.453+0000 7f82f9a00700 1 -- 172.21.15.72:0/2153517706 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f82d80a3d50 msgr2=0x7f82d80a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:17.454 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:17.453+0000 7f82f9a00700 1 --2- 172.21.15.72:0/2153517706 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f82d80a3d50 0x7f82d80a6210 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f82e4005b00 tx=0x7f82e4005a90 comp rx=0 tx=0).stop 2024-04-26T21:53:17.455 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:17.453+0000 7f82f9a00700 1 -- 172.21.15.72:0/2153517706 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f82f41094c0 msgr2=0x7f82f40761a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:17.455 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:17.453+0000 7f82f9a00700 1 --2- 172.21.15.72:0/2153517706 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f82f41094c0 0x7f82f40761a0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f82ec007d70 tx=0x7f82ec00a340 comp rx=0 tx=0).stop 2024-04-26T21:53:17.455 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:17.453+0000 7f82f9a00700 1 -- 172.21.15.72:0/2153517706 shutdown_connections 2024-04-26T21:53:17.455 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:17.453+0000 7f82f9a00700 1 --2- 172.21.15.72:0/2153517706 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f82d80a3d50 0x7f82d80a6210 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:17.455 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:17.453+0000 7f82f9a00700 1 --2- 172.21.15.72:0/2153517706 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f82f41094c0 0x7f82f40761a0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:17.455 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:17.453+0000 7f82f9a00700 1 --2- 172.21.15.72:0/2153517706 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f82f4103470 0x7f82f4075720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:17.455 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:17.453+0000 7f82f9a00700 1 --2- 172.21.15.72:0/2153517706 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f82f4103e30 0x7f82f4075c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:17.455 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:17.453+0000 7f82f9a00700 1 -- 172.21.15.72:0/2153517706 >> 172.21.15.72:0/2153517706 conn(0x7f82f407a580 msgr2=0x7f82f41022b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:53:17.455 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:17.454+0000 7f82f9a00700 1 -- 172.21.15.72:0/2153517706 shutdown_connections 2024-04-26T21:53:17.455 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:17.454+0000 7f82f9a00700 1 -- 172.21.15.72:0/2153517706 wait complete. 2024-04-26T21:53:18.300 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:53:17 smithi076 ceph-mon[31823]: from='client.? 172.21.15.72:0/2153517706' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:53:18.321 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:53:17 smithi072 ceph-mon[28138]: from='client.? 172.21.15.72:0/2153517706' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:53:18.439 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid a1780950-0415-11ef-bc93-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-04-26T21:53:18.440 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid a1780950-0415-11ef-bc93-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-04-26T21:53:18.440 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid a1780950-0415-11ef-bc93-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-04-26T21:53:18.440 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid a1780950-0415-11ef-bc93-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-04-26T21:53:18.441 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid a1780950-0415-11ef-bc93-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-04-26T21:53:18.441 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid a1780950-0415-11ef-bc93-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-04-26T21:53:18.442 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid a1780950-0415-11ef-bc93-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-04-26T21:53:18.442 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid a1780950-0415-11ef-bc93-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-04-26T21:53:18.443 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid a1780950-0415-11ef-bc93-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-04-26T21:53:18.443 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid a1780950-0415-11ef-bc93-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-04-26T21:53:18.443 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid a1780950-0415-11ef-bc93-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-04-26T21:53:18.444 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid a1780950-0415-11ef-bc93-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-04-26T21:53:18.446 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:53:17 smithi164 ceph-mon[31916]: from='client.? 172.21.15.72:0/2153517706' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:53:18.969 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:53:18 smithi072 ceph-mon[28138]: pgmap v383: 97 pgs: 97 active+clean; 632 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 7 op/s; 0 B/s, 0 objects/s recovering 2024-04-26T21:53:19.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:53:18 smithi076 ceph-mon[31823]: pgmap v383: 97 pgs: 97 active+clean; 632 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 7 op/s; 0 B/s, 0 objects/s recovering 2024-04-26T21:53:19.436 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/a1780950-0415-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:53:19.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:53:18 smithi164 ceph-mon[31916]: pgmap v383: 97 pgs: 97 active+clean; 632 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 7 op/s; 0 B/s, 0 objects/s recovering 2024-04-26T21:53:19.605 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/a1780950-0415-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:53:19.625 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/a1780950-0415-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:53:19.630 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/a1780950-0415-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:53:19.630 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/a1780950-0415-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:53:19.635 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/a1780950-0415-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:53:19.639 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/a1780950-0415-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:53:19.639 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/a1780950-0415-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:53:19.645 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/a1780950-0415-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:53:19.648 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/a1780950-0415-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:53:19.651 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/a1780950-0415-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:53:19.657 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/a1780950-0415-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:53:21.062 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:53:20 smithi072 ceph-mon[28138]: pgmap v384: 97 pgs: 97 active+clean; 633 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 310 B/s wr, 9 op/s; 0 B/s, 0 objects/s recovering 2024-04-26T21:53:21.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:53:20 smithi076 ceph-mon[31823]: pgmap v384: 97 pgs: 97 active+clean; 633 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 310 B/s wr, 9 op/s; 0 B/s, 0 objects/s recovering 2024-04-26T21:53:21.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:53:20 smithi164 ceph-mon[31916]: pgmap v384: 97 pgs: 97 active+clean; 633 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 310 B/s wr, 9 op/s; 0 B/s, 0 objects/s recovering 2024-04-26T21:53:23.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:53:22 smithi072 ceph-mon[28138]: pgmap v385: 97 pgs: 97 active+clean; 633 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 282 B/s wr, 9 op/s; 0 B/s, 0 objects/s recovering 2024-04-26T21:53:23.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:53:22 smithi076 ceph-mon[31823]: pgmap v385: 97 pgs: 97 active+clean; 633 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 282 B/s wr, 9 op/s; 0 B/s, 0 objects/s recovering 2024-04-26T21:53:23.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:53:22 smithi164 ceph-mon[31916]: pgmap v385: 97 pgs: 97 active+clean; 633 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 282 B/s wr, 9 op/s; 0 B/s, 0 objects/s recovering 2024-04-26T21:53:25.280 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:53:24 smithi072 ceph-mon[28138]: pgmap v386: 97 pgs: 97 active+clean; 633 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 255 B/s wr, 9 op/s; 0 B/s, 0 objects/s recovering 2024-04-26T21:53:25.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:53:24 smithi076 ceph-mon[31823]: pgmap v386: 97 pgs: 97 active+clean; 633 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 255 B/s wr, 9 op/s; 0 B/s, 0 objects/s recovering 2024-04-26T21:53:25.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:53:24 smithi164 ceph-mon[31916]: pgmap v386: 97 pgs: 97 active+clean; 633 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 255 B/s wr, 9 op/s; 0 B/s, 0 objects/s recovering 2024-04-26T21:53:27.245 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:53:26 smithi164 ceph-mon[31916]: pgmap v387: 97 pgs: 97 active+clean; 633 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 255 B/s wr, 11 op/s; 0 B/s, 0 objects/s recovering 2024-04-26T21:53:27.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:53:26 smithi076 ceph-mon[31823]: pgmap v387: 97 pgs: 97 active+clean; 633 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 255 B/s wr, 11 op/s; 0 B/s, 0 objects/s recovering 2024-04-26T21:53:27.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:53:26 smithi072 ceph-mon[28138]: pgmap v387: 97 pgs: 97 active+clean; 633 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 255 B/s wr, 11 op/s; 0 B/s, 0 objects/s recovering 2024-04-26T21:53:28.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:53:28 smithi072 ceph-mon[28138]: pgmap v388: 97 pgs: 97 active+clean; 633 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-26T21:53:28.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:53:28 smithi076 ceph-mon[31823]: pgmap v388: 97 pgs: 97 active+clean; 633 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-26T21:53:28.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:53:28 smithi164 ceph-mon[31916]: pgmap v388: 97 pgs: 97 active+clean; 633 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-26T21:53:31.130 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:53:30 smithi072 ceph-mon[28138]: pgmap v389: 97 pgs: 97 active+clean; 633 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 255 B/s wr, 11 op/s 2024-04-26T21:53:31.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:53:30 smithi164 ceph-mon[31916]: pgmap v389: 97 pgs: 97 active+clean; 633 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 255 B/s wr, 11 op/s 2024-04-26T21:53:31.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:53:30 smithi076 ceph-mon[31823]: pgmap v389: 97 pgs: 97 active+clean; 633 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 255 B/s wr, 11 op/s 2024-04-26T21:53:31.967 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:31.928+0000 7fc6d5600700 1 -- 172.21.15.72:0/3907657799 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc6d01094c0 msgr2=0x7fc6d01098a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:31.968 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:31.928+0000 7fc6d5600700 1 --2- 172.21.15.72:0/3907657799 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc6d01094c0 0x7fc6d01098a0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fc6c0009990 tx=0x7fc6c0009ca0 comp rx=0 tx=0).stop 2024-04-26T21:53:31.968 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:31.928+0000 7fc6d5600700 1 -- 172.21.15.72:0/3907657799 shutdown_connections 2024-04-26T21:53:31.968 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:31.928+0000 7fc6d5600700 1 --2- 172.21.15.72:0/3907657799 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc6d01094c0 0x7fc6d01098a0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:31.968 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:31.928+0000 7fc6d5600700 1 --2- 172.21.15.72:0/3907657799 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc6d0103e30 0x7fc6d0110290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:31.968 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:31.928+0000 7fc6d5600700 1 --2- 172.21.15.72:0/3907657799 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc6d0103470 0x7fc6d01038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:31.968 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:31.928+0000 7fc6d5600700 1 -- 172.21.15.72:0/3907657799 >> 172.21.15.72:0/3907657799 conn(0x7fc6d007a580 msgr2=0x7fc6d007a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:53:31.968 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:31.928+0000 7fc6d5600700 1 -- 172.21.15.72:0/3907657799 shutdown_connections 2024-04-26T21:53:31.968 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:31.928+0000 7fc6d5600700 1 -- 172.21.15.72:0/3907657799 wait complete. 2024-04-26T21:53:31.968 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:31.930+0000 7fc6d5600700 1 Processor -- start 2024-04-26T21:53:31.969 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:31.930+0000 7fc6d5600700 1 -- start start 2024-04-26T21:53:31.969 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:31.930+0000 7fc6d5600700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc6d0103470 0x7fc6d01a18f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:31.969 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:31.930+0000 7fc6d5600700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc6d0103e30 0x7fc6d01a1e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:31.969 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:31.930+0000 7fc6d5600700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc6d01094c0 0x7fc6d019bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:31.969 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:31.930+0000 7fc6d5600700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fc6d01134f0 con 0x7fc6d01094c0 2024-04-26T21:53:31.969 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:31.930+0000 7fc6d5600700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fc6d0113370 con 0x7fc6d0103e30 2024-04-26T21:53:31.969 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:31.930+0000 7fc6d5600700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fc6d0113670 con 0x7fc6d0103470 2024-04-26T21:53:31.969 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:31.931+0000 7fc6cf400700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc6d01094c0 0x7fc6d019bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:31.969 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:31.931+0000 7fc6cf400700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc6d01094c0 0x7fc6d019bab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:56330/0 (socket says 172.21.15.72:56330) 2024-04-26T21:53:31.969 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:31.931+0000 7fc6cf400700 1 -- 172.21.15.72:0/2617585215 learned_addr learned my addr 172.21.15.72:0/2617585215 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:53:31.969 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:31.931+0000 7fc6cea00700 1 --2- 172.21.15.72:0/2617585215 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc6d0103470 0x7fc6d01a18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:31.969 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:31.931+0000 7fc6ce000700 1 --2- 172.21.15.72:0/2617585215 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc6d0103e30 0x7fc6d01a1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:31.969 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:31.932+0000 7fc6cf400700 1 -- 172.21.15.72:0/2617585215 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc6d0103470 msgr2=0x7fc6d01a18f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:31.970 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:31.932+0000 7fc6cf400700 1 --2- 172.21.15.72:0/2617585215 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc6d0103470 0x7fc6d01a18f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:31.970 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:31.932+0000 7fc6cf400700 1 -- 172.21.15.72:0/2617585215 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc6d0103e30 msgr2=0x7fc6d01a1e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:31.970 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:31.932+0000 7fc6cf400700 1 --2- 172.21.15.72:0/2617585215 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc6d0103e30 0x7fc6d01a1e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:31.970 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:31.932+0000 7fc6cf400700 1 -- 172.21.15.72:0/2617585215 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6d019c210 con 0x7fc6d01094c0 2024-04-26T21:53:31.970 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:31.932+0000 7fc6cf400700 1 --2- 172.21.15.72:0/2617585215 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc6d01094c0 0x7fc6d019bab0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fc6c8007ee0 tx=0x7fc6c800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:53:31.970 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:31.932+0000 7fc6c7400700 1 -- 172.21.15.72:0/2617585215 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6c8004bb0 con 0x7fc6d01094c0 2024-04-26T21:53:31.970 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:31.933+0000 7fc6d5600700 1 -- 172.21.15.72:0/2617585215 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6d019c500 con 0x7fc6d01094c0 2024-04-26T21:53:31.970 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:31.933+0000 7fc6c7400700 1 -- 172.21.15.72:0/2617585215 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc6c8004d50 con 0x7fc6d01094c0 2024-04-26T21:53:31.971 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:31.933+0000 7fc6c7400700 1 -- 172.21.15.72:0/2617585215 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6c80058b0 con 0x7fc6d01094c0 2024-04-26T21:53:31.971 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:31.933+0000 7fc6d5600700 1 -- 172.21.15.72:0/2617585215 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6d01a87c0 con 0x7fc6d01094c0 2024-04-26T21:53:31.971 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:31.934+0000 7fc6d5600700 1 -- 172.21.15.72:0/2617585215 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc6d0063b70 con 0x7fc6d01094c0 2024-04-26T21:53:31.971 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:31.937+0000 7fc6c7400700 1 -- 172.21.15.72:0/2617585215 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fc6c801a020 con 0x7fc6d01094c0 2024-04-26T21:53:31.971 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:31.939+0000 7fc6c7400700 1 --2- 172.21.15.72:0/2617585215 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fc6b40a3cd0 0x7fc6b40a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:31.971 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:31.939+0000 7fc6c7400700 1 -- 172.21.15.72:0/2617585215 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8269+0+0 (secure 0 0 0) 0x7fc6c80d02d0 con 0x7fc6d01094c0 2024-04-26T21:53:31.971 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:31.939+0000 7fc6cea00700 1 --2- 172.21.15.72:0/2617585215 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fc6b40a3cd0 0x7fc6b40a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:31.971 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:31.939+0000 7fc6c7400700 1 --2- 172.21.15.72:0/2617585215 >> [v2:172.21.15.72:6826/1273118485,v1:172.21.15.72:6827/1273118485] conn(0x7fc6b40ad910 0x7fc6b40afdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:31.971 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:31.939+0000 7fc6c7400700 1 -- 172.21.15.72:0/2617585215 --> [v2:172.21.15.72:6826/1273118485,v1:172.21.15.72:6827/1273118485] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc6c8005a50 con 0x7fc6b40ad910 2024-04-26T21:53:31.971 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:31.940+0000 7fc6c7400700 1 -- 172.21.15.72:0/2617585215 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7fc6c80d0680 con 0x7fc6d01094c0 2024-04-26T21:53:31.971 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:31.940+0000 7fc6ce000700 1 --2- 172.21.15.72:0/2617585215 >> [v2:172.21.15.72:6826/1273118485,v1:172.21.15.72:6827/1273118485] conn(0x7fc6b40ad910 0x7fc6b40afdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:31.971 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:31.940+0000 7fc6cea00700 1 --2- 172.21.15.72:0/2617585215 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fc6b40a3cd0 0x7fc6b40a6190 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fc6c0003760 tx=0x7fc6c001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:53:31.972 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:31.941+0000 7fc6ce000700 1 --2- 172.21.15.72:0/2617585215 >> [v2:172.21.15.72:6826/1273118485,v1:172.21.15.72:6827/1273118485] conn(0x7fc6b40ad910 0x7fc6b40afdb0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:53:31.972 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:31.942+0000 7fc6c7400700 1 -- 172.21.15.72:0/2617585215 <== osd.9 v2:172.21.15.72:6826/1273118485 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc6c8005a50 con 0x7fc6b40ad910 2024-04-26T21:53:31.972 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:31.965+0000 7fc6d5600700 1 -- 172.21.15.72:0/2617585215 --> [v2:172.21.15.72:6826/1273118485,v1:172.21.15.72:6827/1273118485] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fc6d0063fa0 con 0x7fc6b40ad910 2024-04-26T21:53:31.972 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:31.965+0000 7fc6c7400700 1 -- 172.21.15.72:0/2617585215 <== osd.9 v2:172.21.15.72:6826/1273118485 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fc6d0063fa0 con 0x7fc6b40ad910 2024-04-26T21:53:31.972 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:31.966+0000 7fc6d5600700 1 -- 172.21.15.72:0/2617585215 >> [v2:172.21.15.72:6826/1273118485,v1:172.21.15.72:6827/1273118485] conn(0x7fc6b40ad910 msgr2=0x7fc6b40afdb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:31.972 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:31.966+0000 7fc6d5600700 1 --2- 172.21.15.72:0/2617585215 >> [v2:172.21.15.72:6826/1273118485,v1:172.21.15.72:6827/1273118485] conn(0x7fc6b40ad910 0x7fc6b40afdb0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:31.972 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:31.966+0000 7fc6d5600700 1 -- 172.21.15.72:0/2617585215 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fc6b40a3cd0 msgr2=0x7fc6b40a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:31.972 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:31.966+0000 7fc6d5600700 1 --2- 172.21.15.72:0/2617585215 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fc6b40a3cd0 0x7fc6b40a6190 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fc6c0003760 tx=0x7fc6c001a040 comp rx=0 tx=0).stop 2024-04-26T21:53:31.972 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:31.966+0000 7fc6d5600700 1 -- 172.21.15.72:0/2617585215 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc6d01094c0 msgr2=0x7fc6d019bab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:31.972 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:31.966+0000 7fc6d5600700 1 --2- 172.21.15.72:0/2617585215 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc6d01094c0 0x7fc6d019bab0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fc6c8007ee0 tx=0x7fc6c800d3b0 comp rx=0 tx=0).stop 2024-04-26T21:53:31.972 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:31.966+0000 7fc6cf400700 1 -- 172.21.15.72:0/2617585215 reap_dead start 2024-04-26T21:53:31.972 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:31.966+0000 7fc6d5600700 1 -- 172.21.15.72:0/2617585215 shutdown_connections 2024-04-26T21:53:31.972 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:31.966+0000 7fc6d5600700 1 -- 172.21.15.72:0/2617585215 >> 172.21.15.72:0/2617585215 conn(0x7fc6d007a580 msgr2=0x7fc6d010e9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:53:31.972 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:31.966+0000 7fc6d5600700 1 -- 172.21.15.72:0/2617585215 shutdown_connections 2024-04-26T21:53:31.973 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:31.966+0000 7fc6d5600700 1 -- 172.21.15.72:0/2617585215 wait complete. 2024-04-26T21:53:32.179 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:32.049+0000 7fb078600700 1 -- 172.21.15.72:0/1714034327 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb070075f70 msgr2=0x7fb0701140b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:32.179 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:32.049+0000 7fb078600700 1 --2- 172.21.15.72:0/1714034327 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb070075f70 0x7fb0701140b0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fb05c009990 tx=0x7fb05c009ca0 comp rx=0 tx=0).stop 2024-04-26T21:53:32.179 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:32.049+0000 7fb078600700 1 -- 172.21.15.72:0/1714034327 shutdown_connections 2024-04-26T21:53:32.179 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:32.049+0000 7fb078600700 1 --2- 172.21.15.72:0/1714034327 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb0701145f0 0x7fb0701169e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:32.179 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:32.049+0000 7fb078600700 1 --2- 172.21.15.72:0/1714034327 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb070075650 0x7fb070075a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:32.179 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:32.049+0000 7fb078600700 1 --2- 172.21.15.72:0/1714034327 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb070075f70 0x7fb0701140b0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:32.179 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:32.049+0000 7fb078600700 1 -- 172.21.15.72:0/1714034327 >> 172.21.15.72:0/1714034327 conn(0x7fb07007a5d0 msgr2=0x7fb07007a9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:53:32.180 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:32.049+0000 7fb078600700 1 -- 172.21.15.72:0/1714034327 shutdown_connections 2024-04-26T21:53:32.180 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:32.049+0000 7fb078600700 1 -- 172.21.15.72:0/1714034327 wait complete. 2024-04-26T21:53:32.180 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:32.050+0000 7fb078600700 1 Processor -- start 2024-04-26T21:53:32.180 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:32.051+0000 7fb078600700 1 -- start start 2024-04-26T21:53:32.180 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:32.051+0000 7fb078600700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb070075650 0x7fb07019ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:32.180 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:32.051+0000 7fb078600700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb070075f70 0x7fb0701a0500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:32.180 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:32.051+0000 7fb078600700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb0701145f0 0x7fb0701a4890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:32.180 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:32.051+0000 7fb078600700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fb070119b30 con 0x7fb070075f70 2024-04-26T21:53:32.180 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:32.051+0000 7fb078600700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fb0701199b0 con 0x7fb070075650 2024-04-26T21:53:32.180 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:32.051+0000 7fb078600700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fb070119cb0 con 0x7fb0701145f0 2024-04-26T21:53:32.180 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:32.051+0000 7fb075200700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb070075f70 0x7fb0701a0500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:32.180 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:32.051+0000 7fb075200700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb070075f70 0x7fb0701a0500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:56354/0 (socket says 172.21.15.72:56354) 2024-04-26T21:53:32.181 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:32.051+0000 7fb075200700 1 -- 172.21.15.72:0/703746356 learned_addr learned my addr 172.21.15.72:0/703746356 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:53:32.181 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:32.052+0000 7fb076600700 1 --2- 172.21.15.72:0/703746356 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb0701145f0 0x7fb0701a4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:32.181 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:32.052+0000 7fb075c00700 1 --2- 172.21.15.72:0/703746356 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb070075650 0x7fb07019ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:32.181 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:32.052+0000 7fb075200700 1 -- 172.21.15.72:0/703746356 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb0701145f0 msgr2=0x7fb0701a4890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:32.181 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:32.052+0000 7fb075200700 1 --2- 172.21.15.72:0/703746356 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb0701145f0 0x7fb0701a4890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:32.181 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:32.052+0000 7fb075200700 1 -- 172.21.15.72:0/703746356 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb070075650 msgr2=0x7fb07019ffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:32.181 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:32.052+0000 7fb075200700 1 --2- 172.21.15.72:0/703746356 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb070075650 0x7fb07019ffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:32.181 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:32.052+0000 7fb075200700 1 -- 172.21.15.72:0/703746356 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0701a4fd0 con 0x7fb070075f70 2024-04-26T21:53:32.181 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:32.052+0000 7fb075200700 1 --2- 172.21.15.72:0/703746356 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb070075f70 0x7fb0701a0500 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fb05c000c00 tx=0x7fb05c005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:53:32.181 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:32.053+0000 7fb06e000700 1 -- 172.21.15.72:0/703746356 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb05c027070 con 0x7fb070075f70 2024-04-26T21:53:32.181 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:32.053+0000 7fb06e000700 1 -- 172.21.15.72:0/703746356 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb05c00fb80 con 0x7fb070075f70 2024-04-26T21:53:32.181 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:32.053+0000 7fb06e000700 1 -- 172.21.15.72:0/703746356 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb05c022c30 con 0x7fb070075f70 2024-04-26T21:53:32.181 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:32.053+0000 7fb078600700 1 -- 172.21.15.72:0/703746356 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0701a5260 con 0x7fb070075f70 2024-04-26T21:53:32.181 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:32.053+0000 7fb078600700 1 -- 172.21.15.72:0/703746356 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0701acb40 con 0x7fb070075f70 2024-04-26T21:53:32.182 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:32.054+0000 7fb078600700 1 -- 172.21.15.72:0/703746356 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb070110af0 con 0x7fb070075f70 2024-04-26T21:53:32.182 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:32.056+0000 7fb06e000700 1 -- 172.21.15.72:0/703746356 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fb05c02a070 con 0x7fb070075f70 2024-04-26T21:53:32.182 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:32.057+0000 7fb06e000700 1 --2- 172.21.15.72:0/703746356 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fb0580a3cd0 0x7fb0580a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:32.182 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:32.058+0000 7fb06e000700 1 -- 172.21.15.72:0/703746356 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8269+0+0 (secure 0 0 0) 0x7fb05c0d1990 con 0x7fb070075f70 2024-04-26T21:53:32.182 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:32.058+0000 7fb075c00700 1 --2- 172.21.15.72:0/703746356 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fb0580a3cd0 0x7fb0580a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:32.182 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:32.058+0000 7fb06e000700 1 --2- 172.21.15.72:0/703746356 >> [v2:172.21.15.164:6816/627888883,v1:172.21.15.164:6817/627888883] conn(0x7fb0580ad910 0x7fb0580afdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:32.182 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:32.058+0000 7fb06e000700 1 -- 172.21.15.72:0/703746356 --> [v2:172.21.15.164:6816/627888883,v1:172.21.15.164:6817/627888883] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb05c017e50 con 0x7fb0580ad910 2024-04-26T21:53:32.182 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:32.058+0000 7fb06e000700 1 -- 172.21.15.72:0/703746356 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7fb05c0d1d40 con 0x7fb070075f70 2024-04-26T21:53:32.182 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:32.059+0000 7fb076600700 1 --2- 172.21.15.72:0/703746356 >> [v2:172.21.15.164:6816/627888883,v1:172.21.15.164:6817/627888883] conn(0x7fb0580ad910 0x7fb0580afdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:32.182 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:32.059+0000 7fb075c00700 1 --2- 172.21.15.72:0/703746356 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fb0580a3cd0 0x7fb0580a6190 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fb064005950 tx=0x7fb0640058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:53:32.182 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:32.060+0000 7fb076600700 1 --2- 172.21.15.72:0/703746356 >> [v2:172.21.15.164:6816/627888883,v1:172.21.15.164:6817/627888883] conn(0x7fb0580ad910 0x7fb0580afdb0 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:53:32.182 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:32.062+0000 7fb06e000700 1 -- 172.21.15.72:0/703746356 <== osd.8 v2:172.21.15.164:6816/627888883 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb05c017e50 con 0x7fb0580ad910 2024-04-26T21:53:32.182 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:32.081+0000 7fb078600700 1 -- 172.21.15.72:0/703746356 --> [v2:172.21.15.164:6816/627888883,v1:172.21.15.164:6817/627888883] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fb070076d70 con 0x7fb0580ad910 2024-04-26T21:53:32.182 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:32.082+0000 7fb06e000700 1 -- 172.21.15.72:0/703746356 <== osd.8 v2:172.21.15.164:6816/627888883 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fb070076d70 con 0x7fb0580ad910 2024-04-26T21:53:32.183 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:32.082+0000 7fb078600700 1 -- 172.21.15.72:0/703746356 >> [v2:172.21.15.164:6816/627888883,v1:172.21.15.164:6817/627888883] conn(0x7fb0580ad910 msgr2=0x7fb0580afdb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:32.183 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:32.082+0000 7fb078600700 1 --2- 172.21.15.72:0/703746356 >> [v2:172.21.15.164:6816/627888883,v1:172.21.15.164:6817/627888883] conn(0x7fb0580ad910 0x7fb0580afdb0 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:32.183 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:32.082+0000 7fb078600700 1 -- 172.21.15.72:0/703746356 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fb0580a3cd0 msgr2=0x7fb0580a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:32.183 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:32.082+0000 7fb078600700 1 --2- 172.21.15.72:0/703746356 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fb0580a3cd0 0x7fb0580a6190 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fb064005950 tx=0x7fb0640058e0 comp rx=0 tx=0).stop 2024-04-26T21:53:32.183 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:32.082+0000 7fb078600700 1 -- 172.21.15.72:0/703746356 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb070075f70 msgr2=0x7fb0701a0500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:32.183 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:32.082+0000 7fb078600700 1 --2- 172.21.15.72:0/703746356 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb070075f70 0x7fb0701a0500 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fb05c000c00 tx=0x7fb05c005ef0 comp rx=0 tx=0).stop 2024-04-26T21:53:32.184 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:32.082+0000 7fb076600700 1 -- 172.21.15.72:0/703746356 reap_dead start 2024-04-26T21:53:32.184 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:32.082+0000 7fb078600700 1 -- 172.21.15.72:0/703746356 shutdown_connections 2024-04-26T21:53:32.184 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:32.082+0000 7fb078600700 1 -- 172.21.15.72:0/703746356 >> 172.21.15.72:0/703746356 conn(0x7fb07007a5d0 msgr2=0x7fb070110090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:53:32.184 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:32.082+0000 7fb078600700 1 -- 172.21.15.72:0/703746356 shutdown_connections 2024-04-26T21:53:32.184 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:32.083+0000 7fb078600700 1 -- 172.21.15.72:0/703746356 wait complete. 2024-04-26T21:53:33.187 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:53:32 smithi072 ceph-mon[28138]: pgmap v390: 97 pgs: 97 active+clean; 633 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:53:33.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:53:32 smithi164 ceph-mon[31916]: pgmap v390: 97 pgs: 97 active+clean; 633 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:53:33.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:53:32 smithi076 ceph-mon[31823]: pgmap v390: 97 pgs: 97 active+clean; 633 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:53:34.864 INFO:teuthology.orchestra.run.smithi072.stdout:261993005103 2024-04-26T21:53:34.865 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid a1780950-0415-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-04-26T21:53:35.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:53:34 smithi072 ceph-mon[28138]: pgmap v391: 97 pgs: 97 active+clean; 633 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 9 op/s 2024-04-26T21:53:35.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:53:34 smithi164 ceph-mon[31916]: pgmap v391: 97 pgs: 97 active+clean; 633 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 9 op/s 2024-04-26T21:53:35.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:53:34 smithi076 ceph-mon[31823]: pgmap v391: 97 pgs: 97 active+clean; 633 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 9 op/s 2024-04-26T21:53:37.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:53:36 smithi072 ceph-mon[28138]: pgmap v392: 97 pgs: 97 active+clean; 633 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-26T21:53:37.117 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:53:36 smithi076 ceph-mon[31823]: pgmap v392: 97 pgs: 97 active+clean; 633 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-26T21:53:37.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:53:36 smithi164 ceph-mon[31916]: pgmap v392: 97 pgs: 97 active+clean; 633 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-26T21:53:37.244 INFO:teuthology.orchestra.run.smithi072.stdout:296352743463 2024-04-26T21:53:37.244 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid a1780950-0415-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-04-26T21:53:38.075 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:38.074+0000 7fd1dda00700 1 -- 172.21.15.72:0/599958647 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd1d8103e30 msgr2=0x7fd1d8110290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:38.075 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:38.074+0000 7fd1dda00700 1 --2- 172.21.15.72:0/599958647 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd1d8103e30 0x7fd1d8110290 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fd1d000b0a0 tx=0x7fd1d000b3b0 comp rx=0 tx=0).stop 2024-04-26T21:53:38.075 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:38.074+0000 7fd1dda00700 1 -- 172.21.15.72:0/599958647 shutdown_connections 2024-04-26T21:53:38.075 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:38.074+0000 7fd1dda00700 1 --2- 172.21.15.72:0/599958647 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd1d8103e30 0x7fd1d8110290 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:38.075 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:38.074+0000 7fd1dda00700 1 --2- 172.21.15.72:0/599958647 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd1d81094c0 0x7fd1d81098a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:38.076 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:38.074+0000 7fd1dda00700 1 --2- 172.21.15.72:0/599958647 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd1d8103470 0x7fd1d81038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:38.076 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:38.074+0000 7fd1dda00700 1 -- 172.21.15.72:0/599958647 >> 172.21.15.72:0/599958647 conn(0x7fd1d807a580 msgr2=0x7fd1d807a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:53:38.076 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:38.074+0000 7fd1dda00700 1 -- 172.21.15.72:0/599958647 shutdown_connections 2024-04-26T21:53:38.076 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:38.074+0000 7fd1dda00700 1 -- 172.21.15.72:0/599958647 wait complete. 2024-04-26T21:53:38.077 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:38.076+0000 7fd1dda00700 1 Processor -- start 2024-04-26T21:53:38.077 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:38.076+0000 7fd1dda00700 1 -- start start 2024-04-26T21:53:38.077 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:38.076+0000 7fd1dda00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd1d8103470 0x7fd1d8075720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:38.077 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:38.076+0000 7fd1dda00700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd1d8103e30 0x7fd1d8075c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:38.077 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:38.076+0000 7fd1dda00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd1d81094c0 0x7fd1d80761a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:38.077 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:38.076+0000 7fd1dda00700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fd1d81134e0 con 0x7fd1d8103470 2024-04-26T21:53:38.078 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:38.076+0000 7fd1dda00700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fd1d8113360 con 0x7fd1d8103e30 2024-04-26T21:53:38.078 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:38.076+0000 7fd1dda00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd1d8113660 con 0x7fd1d81094c0 2024-04-26T21:53:38.078 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:38.076+0000 7fd1d6a00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd1d8103470 0x7fd1d8075720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:38.078 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:38.076+0000 7fd1d6a00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd1d8103470 0x7fd1d8075720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:56396/0 (socket says 172.21.15.72:56396) 2024-04-26T21:53:38.078 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:38.076+0000 7fd1d6a00700 1 -- 172.21.15.72:0/556027679 learned_addr learned my addr 172.21.15.72:0/556027679 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:53:38.078 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:38.077+0000 7fd1cfe00700 1 --2- 172.21.15.72:0/556027679 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd1d8103e30 0x7fd1d8075c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:38.078 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:38.077+0000 7fd1d7400700 1 --2- 172.21.15.72:0/556027679 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd1d81094c0 0x7fd1d80761a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:38.078 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:38.077+0000 7fd1d6a00700 1 -- 172.21.15.72:0/556027679 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd1d81094c0 msgr2=0x7fd1d80761a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:38.078 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:38.077+0000 7fd1d6a00700 1 --2- 172.21.15.72:0/556027679 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd1d81094c0 0x7fd1d80761a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:38.078 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:38.077+0000 7fd1d6a00700 1 -- 172.21.15.72:0/556027679 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd1d8103e30 msgr2=0x7fd1d8075c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:38.078 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:38.077+0000 7fd1d6a00700 1 --2- 172.21.15.72:0/556027679 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd1d8103e30 0x7fd1d8075c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:38.078 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:38.077+0000 7fd1d6a00700 1 -- 172.21.15.72:0/556027679 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1d81a7ee0 con 0x7fd1d8103470 2024-04-26T21:53:38.078 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:38.077+0000 7fd1d6a00700 1 --2- 172.21.15.72:0/556027679 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd1d8103470 0x7fd1d8075720 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fd1c800d980 tx=0x7fd1c800dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:53:38.079 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:38.077+0000 7fd1cf400700 1 -- 172.21.15.72:0/556027679 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1c8017070 con 0x7fd1d8103470 2024-04-26T21:53:38.079 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:38.078+0000 7fd1cf400700 1 -- 172.21.15.72:0/556027679 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd1c8004830 con 0x7fd1d8103470 2024-04-26T21:53:38.079 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:38.078+0000 7fd1dda00700 1 -- 172.21.15.72:0/556027679 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1d81a8170 con 0x7fd1d8103470 2024-04-26T21:53:38.079 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:38.078+0000 7fd1cf400700 1 -- 172.21.15.72:0/556027679 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1c80053a0 con 0x7fd1d8103470 2024-04-26T21:53:38.079 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:38.078+0000 7fd1dda00700 1 -- 172.21.15.72:0/556027679 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1d81a8660 con 0x7fd1d8103470 2024-04-26T21:53:38.080 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:38.078+0000 7fd1dda00700 1 -- 172.21.15.72:0/556027679 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd1d8063b30 con 0x7fd1d8103470 2024-04-26T21:53:38.081 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:38.081+0000 7fd1cf400700 1 -- 172.21.15.72:0/556027679 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fd1c80049e0 con 0x7fd1d8103470 2024-04-26T21:53:38.082 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:38.082+0000 7fd1cf400700 1 --2- 172.21.15.72:0/556027679 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fd1bc0a3cd0 0x7fd1bc0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:38.083 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:38.082+0000 7fd1cf400700 1 -- 172.21.15.72:0/556027679 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8269+0+0 (secure 0 0 0) 0x7fd1c80d0ab0 con 0x7fd1d8103470 2024-04-26T21:53:38.083 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:38.083+0000 7fd1cfe00700 1 --2- 172.21.15.72:0/556027679 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fd1bc0a3cd0 0x7fd1bc0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:38.083 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:38.083+0000 7fd1cf400700 1 --2- 172.21.15.72:0/556027679 >> [v2:172.21.15.76:6808/3718867827,v1:172.21.15.76:6809/3718867827] conn(0x7fd1bc0ad910 0x7fd1bc0afdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:38.083 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:38.083+0000 7fd1cf400700 1 -- 172.21.15.72:0/556027679 --> [v2:172.21.15.76:6808/3718867827,v1:172.21.15.76:6809/3718867827] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd1c8009e20 con 0x7fd1bc0ad910 2024-04-26T21:53:38.083 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:38.083+0000 7fd1cf400700 1 -- 172.21.15.72:0/556027679 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7fd1c80d0e60 con 0x7fd1d8103470 2024-04-26T21:53:38.084 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:38.083+0000 7fd1cfe00700 1 --2- 172.21.15.72:0/556027679 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fd1bc0a3cd0 0x7fd1bc0a6190 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fd1c0005ea0 tx=0x7fd1c0005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:53:38.084 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:38.084+0000 7fd1d7400700 1 --2- 172.21.15.72:0/556027679 >> [v2:172.21.15.76:6808/3718867827,v1:172.21.15.76:6809/3718867827] conn(0x7fd1bc0ad910 0x7fd1bc0afdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:38.084 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:38.085+0000 7fd1d7400700 1 --2- 172.21.15.72:0/556027679 >> [v2:172.21.15.76:6808/3718867827,v1:172.21.15.76:6809/3718867827] conn(0x7fd1bc0ad910 0x7fd1bc0afdb0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:53:38.087 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:38.087+0000 7fd1cf400700 1 -- 172.21.15.72:0/556027679 <== osd.4 v2:172.21.15.76:6808/3718867827 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd1c8009e20 con 0x7fd1bc0ad910 2024-04-26T21:53:38.106 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:38.106+0000 7fd1dda00700 1 -- 172.21.15.72:0/556027679 --> [v2:172.21.15.76:6808/3718867827,v1:172.21.15.76:6809/3718867827] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fd1d8063fb0 con 0x7fd1bc0ad910 2024-04-26T21:53:38.108 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:38.107+0000 7fd1cf400700 1 -- 172.21.15.72:0/556027679 <== osd.4 v2:172.21.15.76:6808/3718867827 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fd1d8063fb0 con 0x7fd1bc0ad910 2024-04-26T21:53:38.108 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:38.108+0000 7fd1dda00700 1 -- 172.21.15.72:0/556027679 >> [v2:172.21.15.76:6808/3718867827,v1:172.21.15.76:6809/3718867827] conn(0x7fd1bc0ad910 msgr2=0x7fd1bc0afdb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:38.108 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:38.108+0000 7fd1dda00700 1 --2- 172.21.15.72:0/556027679 >> [v2:172.21.15.76:6808/3718867827,v1:172.21.15.76:6809/3718867827] conn(0x7fd1bc0ad910 0x7fd1bc0afdb0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:38.109 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:38.108+0000 7fd1dda00700 1 -- 172.21.15.72:0/556027679 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fd1bc0a3cd0 msgr2=0x7fd1bc0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:38.109 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:38.108+0000 7fd1dda00700 1 --2- 172.21.15.72:0/556027679 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fd1bc0a3cd0 0x7fd1bc0a6190 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fd1c0005ea0 tx=0x7fd1c0005e30 comp rx=0 tx=0).stop 2024-04-26T21:53:38.109 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:38.108+0000 7fd1dda00700 1 -- 172.21.15.72:0/556027679 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd1d8103470 msgr2=0x7fd1d8075720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:38.109 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:38.108+0000 7fd1dda00700 1 --2- 172.21.15.72:0/556027679 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd1d8103470 0x7fd1d8075720 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fd1c800d980 tx=0x7fd1c800dd40 comp rx=0 tx=0).stop 2024-04-26T21:53:38.109 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:38.108+0000 7fd1d7400700 1 -- 172.21.15.72:0/556027679 reap_dead start 2024-04-26T21:53:38.109 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:38.108+0000 7fd1dda00700 1 -- 172.21.15.72:0/556027679 shutdown_connections 2024-04-26T21:53:38.109 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:38.108+0000 7fd1dda00700 1 -- 172.21.15.72:0/556027679 >> 172.21.15.72:0/556027679 conn(0x7fd1d807a580 msgr2=0x7fd1d8102170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:53:38.109 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:38.109+0000 7fd1dda00700 1 -- 172.21.15.72:0/556027679 shutdown_connections 2024-04-26T21:53:38.109 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:38.109+0000 7fd1dda00700 1 -- 172.21.15.72:0/556027679 wait complete. 2024-04-26T21:53:38.540 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:36.376+0000 7f6ddd200700 1 -- 172.21.15.72:0/2006586702 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6dd81035b0 msgr2=0x7f6dd8103990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:38.540 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:36.376+0000 7f6ddd200700 1 --2- 172.21.15.72:0/2006586702 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6dd81035b0 0x7f6dd8103990 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f6dc80099c0 tx=0x7f6dc8009cd0 comp rx=0 tx=0).stop 2024-04-26T21:53:38.540 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:36.376+0000 7f6ddd200700 1 -- 172.21.15.72:0/2006586702 shutdown_connections 2024-04-26T21:53:38.540 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:36.377+0000 7f6ddd200700 1 --2- 172.21.15.72:0/2006586702 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6dd81035b0 0x7f6dd8103990 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:38.540 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:36.377+0000 7f6ddd200700 1 --2- 172.21.15.72:0/2006586702 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6dd8108690 0x7f6dd8110260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:38.540 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:36.377+0000 7f6ddd200700 1 --2- 172.21.15.72:0/2006586702 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6dd8103f60 0x7f6dd8107fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:38.540 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:36.377+0000 7f6ddd200700 1 -- 172.21.15.72:0/2006586702 >> 172.21.15.72:0/2006586702 conn(0x7f6dd80feea0 msgr2=0x7f6dd81012c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:53:38.540 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:36.377+0000 7f6ddd200700 1 -- 172.21.15.72:0/2006586702 shutdown_connections 2024-04-26T21:53:38.540 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:36.377+0000 7f6ddd200700 1 -- 172.21.15.72:0/2006586702 wait complete. 2024-04-26T21:53:38.540 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:36.378+0000 7f6ddd200700 1 Processor -- start 2024-04-26T21:53:38.541 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:36.378+0000 7f6ddd200700 1 -- start start 2024-04-26T21:53:38.541 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:36.378+0000 7f6ddd200700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6dd81035b0 0x7f6dd81a1860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:38.541 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:36.378+0000 7f6ddd200700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6dd8103f60 0x7f6dd81a1da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:38.541 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:36.378+0000 7f6ddd200700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6dd8108690 0x7f6dd819ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:38.541 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:36.378+0000 7f6ddd200700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f6dd8113440 con 0x7f6dd8108690 2024-04-26T21:53:38.541 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:36.378+0000 7f6ddd200700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f6dd81132c0 con 0x7f6dd8103f60 2024-04-26T21:53:38.541 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:36.378+0000 7f6ddd200700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f6dd81135c0 con 0x7f6dd81035b0 2024-04-26T21:53:38.541 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:36.379+0000 7f6dd7e00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6dd8108690 0x7f6dd819ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:38.541 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:36.379+0000 7f6dd7e00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6dd8108690 0x7f6dd819ba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:56378/0 (socket says 172.21.15.72:56378) 2024-04-26T21:53:38.541 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:36.379+0000 7f6dd7e00700 1 -- 172.21.15.72:0/306873904 learned_addr learned my addr 172.21.15.72:0/306873904 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:53:38.541 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:36.379+0000 7f6dd7400700 1 --2- 172.21.15.72:0/306873904 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6dd81035b0 0x7f6dd81a1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:38.541 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:36.379+0000 7f6dd6a00700 1 --2- 172.21.15.72:0/306873904 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6dd8103f60 0x7f6dd81a1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:38.541 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:36.379+0000 7f6dd7e00700 1 -- 172.21.15.72:0/306873904 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6dd81035b0 msgr2=0x7f6dd81a1860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:38.542 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:36.380+0000 7f6dd7e00700 1 --2- 172.21.15.72:0/306873904 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6dd81035b0 0x7f6dd81a1860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:38.542 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:36.380+0000 7f6dd7e00700 1 -- 172.21.15.72:0/306873904 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6dd8103f60 msgr2=0x7f6dd81a1da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:38.542 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:36.380+0000 7f6dd7e00700 1 --2- 172.21.15.72:0/306873904 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6dd8103f60 0x7f6dd81a1da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:38.542 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:36.380+0000 7f6dd7e00700 1 -- 172.21.15.72:0/306873904 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6dd819c180 con 0x7f6dd8108690 2024-04-26T21:53:38.542 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:36.380+0000 7f6dd7e00700 1 --2- 172.21.15.72:0/306873904 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6dd8108690 0x7f6dd819ba20 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f6dd0007ee0 tx=0x7f6dd000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:53:38.542 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:36.380+0000 7f6dcfe00700 1 -- 172.21.15.72:0/306873904 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dd000dbd0 con 0x7f6dd8108690 2024-04-26T21:53:38.542 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:36.380+0000 7f6ddd200700 1 -- 172.21.15.72:0/306873904 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6dd819c470 con 0x7f6dd8108690 2024-04-26T21:53:38.542 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:36.380+0000 7f6dcfe00700 1 -- 172.21.15.72:0/306873904 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6dd000dd70 con 0x7f6dd8108690 2024-04-26T21:53:38.542 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:36.380+0000 7f6dcfe00700 1 -- 172.21.15.72:0/306873904 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dd00058d0 con 0x7f6dd8108690 2024-04-26T21:53:38.542 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:36.380+0000 7f6ddd200700 1 -- 172.21.15.72:0/306873904 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6dd81a8730 con 0x7f6dd8108690 2024-04-26T21:53:38.542 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:36.381+0000 7f6ddd200700 1 -- 172.21.15.72:0/306873904 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6dd80768f0 con 0x7f6dd8108690 2024-04-26T21:53:38.542 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:36.383+0000 7f6dcfe00700 1 -- 172.21.15.72:0/306873904 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f6dd001a020 con 0x7f6dd8108690 2024-04-26T21:53:38.542 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:36.385+0000 7f6dcfe00700 1 --2- 172.21.15.72:0/306873904 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f6dbc0a3cd0 0x7f6dbc0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:38.542 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:36.385+0000 7f6dcfe00700 1 -- 172.21.15.72:0/306873904 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8269+0+0 (secure 0 0 0) 0x7f6dd00d0440 con 0x7f6dd8108690 2024-04-26T21:53:38.543 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:36.385+0000 7f6dd7400700 1 --2- 172.21.15.72:0/306873904 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f6dbc0a3cd0 0x7f6dbc0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:38.543 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:36.385+0000 7f6dcfe00700 1 --2- 172.21.15.72:0/306873904 >> [v2:172.21.15.164:6824/2354951784,v1:172.21.15.164:6825/2354951784] conn(0x7f6dbc0ad910 0x7f6dbc0afdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:38.543 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:36.385+0000 7f6dcfe00700 1 -- 172.21.15.72:0/306873904 --> [v2:172.21.15.164:6824/2354951784,v1:172.21.15.164:6825/2354951784] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6dbc0b0480 con 0x7f6dbc0ad910 2024-04-26T21:53:38.543 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:36.385+0000 7f6dcfe00700 1 -- 172.21.15.72:0/306873904 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7f6dd00d07f0 con 0x7f6dd8108690 2024-04-26T21:53:38.543 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:36.386+0000 7f6dd7400700 1 --2- 172.21.15.72:0/306873904 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f6dbc0a3cd0 0x7f6dbc0a6190 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f6dc8009990 tx=0x7f6dc8005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:53:38.543 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:36.386+0000 7f6dd6a00700 1 --2- 172.21.15.72:0/306873904 >> [v2:172.21.15.164:6824/2354951784,v1:172.21.15.164:6825/2354951784] conn(0x7f6dbc0ad910 0x7f6dbc0afdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:38.543 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:36.387+0000 7f6dd6a00700 1 --2- 172.21.15.72:0/306873904 >> [v2:172.21.15.164:6824/2354951784,v1:172.21.15.164:6825/2354951784] conn(0x7f6dbc0ad910 0x7f6dbc0afdb0 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:53:38.543 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:36.389+0000 7f6dcfe00700 1 -- 172.21.15.72:0/306873904 <== osd.11 v2:172.21.15.164:6824/2354951784 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6dbc0b0480 con 0x7f6dbc0ad910 2024-04-26T21:53:38.543 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:36.408+0000 7f6ddd200700 1 -- 172.21.15.72:0/306873904 --> [v2:172.21.15.164:6824/2354951784,v1:172.21.15.164:6825/2354951784] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f6dd8104a90 con 0x7f6dbc0ad910 2024-04-26T21:53:38.543 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:36.409+0000 7f6dcfe00700 1 -- 172.21.15.72:0/306873904 <== osd.11 v2:172.21.15.164:6824/2354951784 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f6dd8104a90 con 0x7f6dbc0ad910 2024-04-26T21:53:38.544 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:36.409+0000 7f6ddd200700 1 -- 172.21.15.72:0/306873904 >> [v2:172.21.15.164:6824/2354951784,v1:172.21.15.164:6825/2354951784] conn(0x7f6dbc0ad910 msgr2=0x7f6dbc0afdb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:38.544 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:36.409+0000 7f6ddd200700 1 --2- 172.21.15.72:0/306873904 >> [v2:172.21.15.164:6824/2354951784,v1:172.21.15.164:6825/2354951784] conn(0x7f6dbc0ad910 0x7f6dbc0afdb0 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:38.544 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:36.410+0000 7f6ddd200700 1 -- 172.21.15.72:0/306873904 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f6dbc0a3cd0 msgr2=0x7f6dbc0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:38.544 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:36.410+0000 7f6ddd200700 1 --2- 172.21.15.72:0/306873904 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f6dbc0a3cd0 0x7f6dbc0a6190 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f6dc8009990 tx=0x7f6dc8005fb0 comp rx=0 tx=0).stop 2024-04-26T21:53:38.544 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:36.410+0000 7f6ddd200700 1 -- 172.21.15.72:0/306873904 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6dd8108690 msgr2=0x7f6dd819ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:38.544 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:36.410+0000 7f6ddd200700 1 --2- 172.21.15.72:0/306873904 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6dd8108690 0x7f6dd819ba20 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f6dd0007ee0 tx=0x7f6dd000d3b0 comp rx=0 tx=0).stop 2024-04-26T21:53:38.544 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:36.410+0000 7f6dd7e00700 1 -- 172.21.15.72:0/306873904 reap_dead start 2024-04-26T21:53:38.544 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:36.410+0000 7f6ddd200700 1 -- 172.21.15.72:0/306873904 shutdown_connections 2024-04-26T21:53:38.544 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:36.410+0000 7f6ddd200700 1 -- 172.21.15.72:0/306873904 >> 172.21.15.72:0/306873904 conn(0x7f6dd80feea0 msgr2=0x7f6dd81011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:53:38.544 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:36.410+0000 7f6ddd200700 1 -- 172.21.15.72:0/306873904 shutdown_connections 2024-04-26T21:53:38.544 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:36.410+0000 7f6ddd200700 1 -- 172.21.15.72:0/306873904 wait complete. 2024-04-26T21:53:39.186 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:53:38 smithi072 ceph-mon[28138]: pgmap v393: 97 pgs: 97 active+clean; 633 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:53:39.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:53:38 smithi164 ceph-mon[31916]: pgmap v393: 97 pgs: 97 active+clean; 633 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:53:39.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:53:38 smithi076 ceph-mon[31823]: pgmap v393: 97 pgs: 97 active+clean; 633 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:53:39.570 INFO:teuthology.orchestra.run.smithi072.stdout:352187318297 2024-04-26T21:53:39.571 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid a1780950-0415-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-04-26T21:53:39.979 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:39.978+0000 7fbf79200700 1 -- 172.21.15.72:0/1857815717 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbf74103e30 msgr2=0x7fbf74110290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:39.979 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:39.978+0000 7fbf79200700 1 --2- 172.21.15.72:0/1857815717 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbf74103e30 0x7fbf74110290 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fbf6c00b0a0 tx=0x7fbf6c00b3b0 comp rx=0 tx=0).stop 2024-04-26T21:53:39.979 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:39.978+0000 7fbf79200700 1 -- 172.21.15.72:0/1857815717 shutdown_connections 2024-04-26T21:53:39.980 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:39.978+0000 7fbf79200700 1 --2- 172.21.15.72:0/1857815717 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbf74103e30 0x7fbf74110290 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:39.980 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:39.978+0000 7fbf79200700 1 --2- 172.21.15.72:0/1857815717 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbf741094c0 0x7fbf741098a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:39.980 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:39.978+0000 7fbf79200700 1 --2- 172.21.15.72:0/1857815717 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbf74103470 0x7fbf741038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:39.980 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:39.978+0000 7fbf79200700 1 -- 172.21.15.72:0/1857815717 >> 172.21.15.72:0/1857815717 conn(0x7fbf7407a580 msgr2=0x7fbf7407a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:53:39.980 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:39.979+0000 7fbf79200700 1 -- 172.21.15.72:0/1857815717 shutdown_connections 2024-04-26T21:53:39.980 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:39.979+0000 7fbf79200700 1 -- 172.21.15.72:0/1857815717 wait complete. 2024-04-26T21:53:39.981 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:39.980+0000 7fbf79200700 1 Processor -- start 2024-04-26T21:53:39.981 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:39.980+0000 7fbf79200700 1 -- start start 2024-04-26T21:53:39.981 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:39.980+0000 7fbf79200700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbf74103470 0x7fbf741a18a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:39.981 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:39.980+0000 7fbf79200700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbf74103e30 0x7fbf741a1de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:39.981 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:39.980+0000 7fbf79200700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbf741094c0 0x7fbf7419bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:39.981 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:39.980+0000 7fbf79200700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fbf741134f0 con 0x7fbf74103e30 2024-04-26T21:53:39.982 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:39.980+0000 7fbf79200700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fbf74113370 con 0x7fbf741094c0 2024-04-26T21:53:39.982 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:39.980+0000 7fbf79200700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fbf74113670 con 0x7fbf74103470 2024-04-26T21:53:39.982 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:39.981+0000 7fbf71400700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbf74103e30 0x7fbf741a1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:39.983 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:39.981+0000 7fbf71400700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbf74103e30 0x7fbf741a1de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:56434/0 (socket says 172.21.15.72:56434) 2024-04-26T21:53:39.983 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:39.981+0000 7fbf71400700 1 -- 172.21.15.72:0/286784458 learned_addr learned my addr 172.21.15.72:0/286784458 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:53:39.984 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:39.981+0000 7fbf72800700 1 --2- 172.21.15.72:0/286784458 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbf741094c0 0x7fbf7419bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:39.984 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:39.981+0000 7fbf71e00700 1 --2- 172.21.15.72:0/286784458 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbf74103470 0x7fbf741a18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:39.984 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:39.982+0000 7fbf71400700 1 -- 172.21.15.72:0/286784458 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbf74103470 msgr2=0x7fbf741a18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:39.984 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:39.982+0000 7fbf71400700 1 --2- 172.21.15.72:0/286784458 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbf74103470 0x7fbf741a18a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:39.984 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:39.982+0000 7fbf71400700 1 -- 172.21.15.72:0/286784458 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbf741094c0 msgr2=0x7fbf7419bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:39.984 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:39.982+0000 7fbf71400700 1 --2- 172.21.15.72:0/286784458 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbf741094c0 0x7fbf7419bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:39.984 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:39.982+0000 7fbf71400700 1 -- 172.21.15.72:0/286784458 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf7419c210 con 0x7fbf74103e30 2024-04-26T21:53:39.984 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:39.982+0000 7fbf71e00700 1 --2- 172.21.15.72:0/286784458 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbf74103470 0x7fbf741a18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:53:39.984 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:39.982+0000 7fbf71400700 1 --2- 172.21.15.72:0/286784458 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbf74103e30 0x7fbf741a1de0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fbf6000d980 tx=0x7fbf6000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:53:39.984 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:39.982+0000 7fbf6aa00700 1 -- 172.21.15.72:0/286784458 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf60017070 con 0x7fbf74103e30 2024-04-26T21:53:39.985 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:39.982+0000 7fbf79200700 1 -- 172.21.15.72:0/286784458 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf7419c4a0 con 0x7fbf74103e30 2024-04-26T21:53:39.985 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:39.983+0000 7fbf79200700 1 -- 172.21.15.72:0/286784458 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf741a8770 con 0x7fbf74103e30 2024-04-26T21:53:39.985 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:39.983+0000 7fbf6aa00700 1 -- 172.21.15.72:0/286784458 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbf60005020 con 0x7fbf74103e30 2024-04-26T21:53:39.985 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:39.983+0000 7fbf6aa00700 1 -- 172.21.15.72:0/286784458 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf60009ab0 con 0x7fbf74103e30 2024-04-26T21:53:39.985 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:39.984+0000 7fbf79200700 1 -- 172.21.15.72:0/286784458 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fbf54001030 con 0x7fbf74103e30 2024-04-26T21:53:39.987 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:39.987+0000 7fbf6aa00700 1 -- 172.21.15.72:0/286784458 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fbf60009c50 con 0x7fbf74103e30 2024-04-26T21:53:39.988 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:39.988+0000 7fbf6aa00700 1 --2- 172.21.15.72:0/286784458 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fbf580a3d50 0x7fbf580a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:39.989 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:39.988+0000 7fbf6aa00700 1 -- 172.21.15.72:0/286784458 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8269+0+0 (secure 0 0 0) 0x7fbf600d0860 con 0x7fbf74103e30 2024-04-26T21:53:39.989 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:39.988+0000 7fbf71e00700 1 --2- 172.21.15.72:0/286784458 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fbf580a3d50 0x7fbf580a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:39.989 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:39.989+0000 7fbf6aa00700 1 --2- 172.21.15.72:0/286784458 >> [v2:172.21.15.72:6818/2080659027,v1:172.21.15.72:6819/2080659027] conn(0x7fbf580ad990 0x7fbf580afe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:39.989 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:39.989+0000 7fbf6aa00700 1 -- 172.21.15.72:0/286784458 --> [v2:172.21.15.72:6818/2080659027,v1:172.21.15.72:6819/2080659027] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fbf60013e50 con 0x7fbf580ad990 2024-04-26T21:53:39.989 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:39.989+0000 7fbf6aa00700 1 -- 172.21.15.72:0/286784458 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7fbf600d0c10 con 0x7fbf74103e30 2024-04-26T21:53:39.990 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:39.989+0000 7fbf72800700 1 --2- 172.21.15.72:0/286784458 >> [v2:172.21.15.72:6818/2080659027,v1:172.21.15.72:6819/2080659027] conn(0x7fbf580ad990 0x7fbf580afe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:39.990 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:39.989+0000 7fbf71e00700 1 --2- 172.21.15.72:0/286784458 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fbf580a3d50 0x7fbf580a6210 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fbf64005e10 tx=0x7fbf64005d60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:53:39.990 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:39.990+0000 7fbf72800700 1 --2- 172.21.15.72:0/286784458 >> [v2:172.21.15.72:6818/2080659027,v1:172.21.15.72:6819/2080659027] conn(0x7fbf580ad990 0x7fbf580afe30 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:53:39.991 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:39.991+0000 7fbf6aa00700 1 -- 172.21.15.72:0/286784458 <== osd.6 v2:172.21.15.72:6818/2080659027 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fbf60013e50 con 0x7fbf580ad990 2024-04-26T21:53:40.009 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:40.008+0000 7fbf79200700 1 -- 172.21.15.72:0/286784458 --> [v2:172.21.15.72:6818/2080659027,v1:172.21.15.72:6819/2080659027] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fbf54002e60 con 0x7fbf580ad990 2024-04-26T21:53:40.009 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:40.009+0000 7fbf6aa00700 1 -- 172.21.15.72:0/286784458 <== osd.6 v2:172.21.15.72:6818/2080659027 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fbf54002e60 con 0x7fbf580ad990 2024-04-26T21:53:40.010 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:40.009+0000 7fbf79200700 1 -- 172.21.15.72:0/286784458 >> [v2:172.21.15.72:6818/2080659027,v1:172.21.15.72:6819/2080659027] conn(0x7fbf580ad990 msgr2=0x7fbf580afe30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:40.010 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:40.009+0000 7fbf79200700 1 --2- 172.21.15.72:0/286784458 >> [v2:172.21.15.72:6818/2080659027,v1:172.21.15.72:6819/2080659027] conn(0x7fbf580ad990 0x7fbf580afe30 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:40.010 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:40.010+0000 7fbf79200700 1 -- 172.21.15.72:0/286784458 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fbf580a3d50 msgr2=0x7fbf580a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:40.010 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:40.010+0000 7fbf79200700 1 --2- 172.21.15.72:0/286784458 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fbf580a3d50 0x7fbf580a6210 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fbf64005e10 tx=0x7fbf64005d60 comp rx=0 tx=0).stop 2024-04-26T21:53:40.010 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:40.010+0000 7fbf79200700 1 -- 172.21.15.72:0/286784458 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbf74103e30 msgr2=0x7fbf741a1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:40.010 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:40.010+0000 7fbf79200700 1 --2- 172.21.15.72:0/286784458 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbf74103e30 0x7fbf741a1de0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fbf6000d980 tx=0x7fbf6000dc90 comp rx=0 tx=0).stop 2024-04-26T21:53:40.011 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:40.010+0000 7fbf72800700 1 -- 172.21.15.72:0/286784458 reap_dead start 2024-04-26T21:53:40.011 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:40.010+0000 7fbf79200700 1 -- 172.21.15.72:0/286784458 shutdown_connections 2024-04-26T21:53:40.011 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:40.010+0000 7fbf79200700 1 -- 172.21.15.72:0/286784458 >> 172.21.15.72:0/286784458 conn(0x7fbf7407a580 msgr2=0x7fbf74101fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:53:40.011 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:40.010+0000 7fbf79200700 1 -- 172.21.15.72:0/286784458 shutdown_connections 2024-04-26T21:53:40.011 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:40.010+0000 7fbf79200700 1 -- 172.21.15.72:0/286784458 wait complete. 2024-04-26T21:53:41.136 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:53:40 smithi072 ceph-mon[28138]: pgmap v394: 97 pgs: 97 active+clean; 633 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-26T21:53:41.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:53:40 smithi164 ceph-mon[31916]: pgmap v394: 97 pgs: 97 active+clean; 633 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-26T21:53:41.217 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:41.215+0000 7f5848000700 1 -- 172.21.15.72:0/1347017419 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f584010fa50 msgr2=0x7f584010fe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:41.217 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:41.215+0000 7f5848000700 1 --2- 172.21.15.72:0/1347017419 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f584010fa50 0x7f584010fe30 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f5834009990 tx=0x7f5834009ca0 comp rx=0 tx=0).stop 2024-04-26T21:53:41.218 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:41.215+0000 7f5848000700 1 -- 172.21.15.72:0/1347017419 shutdown_connections 2024-04-26T21:53:41.218 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:41.215+0000 7f5848000700 1 --2- 172.21.15.72:0/1347017419 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f584010fa50 0x7f584010fe30 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:41.218 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:41.215+0000 7f5848000700 1 --2- 172.21.15.72:0/1347017419 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f584010a410 0x7f5840116920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:41.218 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:41.215+0000 7f5848000700 1 --2- 172.21.15.72:0/1347017419 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5840109a50 0x7f5840109ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:41.218 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:41.215+0000 7f5848000700 1 -- 172.21.15.72:0/1347017419 >> 172.21.15.72:0/1347017419 conn(0x7f5840076bf0 msgr2=0x7f5840108240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:53:41.218 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:41.215+0000 7f5848000700 1 -- 172.21.15.72:0/1347017419 shutdown_connections 2024-04-26T21:53:41.218 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:41.215+0000 7f5848000700 1 -- 172.21.15.72:0/1347017419 wait complete. 2024-04-26T21:53:41.220 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:41.217+0000 7f5848000700 1 Processor -- start 2024-04-26T21:53:41.220 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:41.217+0000 7f5848000700 1 -- start start 2024-04-26T21:53:41.220 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:41.217+0000 7f5848000700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5840109a50 0x7f584019fff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:41.220 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:41.217+0000 7f5848000700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f584010a410 0x7f58401a0530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:41.221 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:41.217+0000 7f5848000700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f584010fa50 0x7f58401a48c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:41.221 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:41.217+0000 7f5848000700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f5840119b00 con 0x7f584010a410 2024-04-26T21:53:41.221 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:41.217+0000 7f5848000700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f5840119980 con 0x7f584010fa50 2024-04-26T21:53:41.221 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:41.217+0000 7f5848000700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f5840119c80 con 0x7f5840109a50 2024-04-26T21:53:41.221 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:41.218+0000 7f5844c00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f584010a410 0x7f58401a0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:41.221 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:41.218+0000 7f5844c00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f584010a410 0x7f58401a0530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:57726/0 (socket says 172.21.15.72:57726) 2024-04-26T21:53:41.221 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:41.218+0000 7f5844c00700 1 -- 172.21.15.72:0/708162482 learned_addr learned my addr 172.21.15.72:0/708162482 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:53:41.221 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:41.218+0000 7f5845600700 1 --2- 172.21.15.72:0/708162482 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5840109a50 0x7f584019fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:41.221 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:41.218+0000 7f5846000700 1 --2- 172.21.15.72:0/708162482 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f584010fa50 0x7f58401a48c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:41.221 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:41.218+0000 7f5844c00700 1 -- 172.21.15.72:0/708162482 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5840109a50 msgr2=0x7f584019fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:41.221 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:41.218+0000 7f5844c00700 1 --2- 172.21.15.72:0/708162482 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5840109a50 0x7f584019fff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:41.221 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:41.218+0000 7f5844c00700 1 -- 172.21.15.72:0/708162482 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f584010fa50 msgr2=0x7f58401a48c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:41.222 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:41.218+0000 7f5844c00700 1 --2- 172.21.15.72:0/708162482 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f584010fa50 0x7f58401a48c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:41.222 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:41.218+0000 7f5844c00700 1 -- 172.21.15.72:0/708162482 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58401a5000 con 0x7f584010a410 2024-04-26T21:53:41.222 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:41.218+0000 7f5846000700 1 --2- 172.21.15.72:0/708162482 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f584010fa50 0x7f58401a48c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:53:41.222 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:41.218+0000 7f5845600700 1 --2- 172.21.15.72:0/708162482 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5840109a50 0x7f584019fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:53:41.222 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:41.218+0000 7f5844c00700 1 --2- 172.21.15.72:0/708162482 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f584010a410 0x7f58401a0530 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f582c00d980 tx=0x7f582c00dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:53:41.222 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:41.219+0000 7f583e000700 1 -- 172.21.15.72:0/708162482 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f582c017070 con 0x7f584010a410 2024-04-26T21:53:41.222 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:41.219+0000 7f583e000700 1 -- 172.21.15.72:0/708162482 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f582c004830 con 0x7f584010a410 2024-04-26T21:53:41.222 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:41.219+0000 7f5848000700 1 -- 172.21.15.72:0/708162482 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58401a52f0 con 0x7f584010a410 2024-04-26T21:53:41.222 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:41.219+0000 7f583e000700 1 -- 172.21.15.72:0/708162482 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f582c0053a0 con 0x7f584010a410 2024-04-26T21:53:41.222 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:41.219+0000 7f5848000700 1 -- 172.21.15.72:0/708162482 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58401acb70 con 0x7f584010a410 2024-04-26T21:53:41.223 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:41.220+0000 7f5848000700 1 -- 172.21.15.72:0/708162482 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5840063bb0 con 0x7f584010a410 2024-04-26T21:53:41.223 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:41.222+0000 7f583e000700 1 -- 172.21.15.72:0/708162482 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f582c0049e0 con 0x7f584010a410 2024-04-26T21:53:41.225 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:41.223+0000 7f583e000700 1 --2- 172.21.15.72:0/708162482 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f58280a3cd0 0x7f58280a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:41.225 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:41.224+0000 7f583e000700 1 -- 172.21.15.72:0/708162482 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8269+0+0 (secure 0 0 0) 0x7f582c0d0b20 con 0x7f584010a410 2024-04-26T21:53:41.225 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:41.224+0000 7f5845600700 1 --2- 172.21.15.72:0/708162482 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f58280a3cd0 0x7f58280a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:41.225 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:41.224+0000 7f583e000700 1 --2- 172.21.15.72:0/708162482 >> [v2:172.21.15.76:6824/2808954459,v1:172.21.15.76:6825/2808954459] conn(0x7f58280ad910 0x7f58280afdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:41.225 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:41.224+0000 7f583e000700 1 -- 172.21.15.72:0/708162482 --> [v2:172.21.15.76:6824/2808954459,v1:172.21.15.76:6825/2808954459] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f582c009e20 con 0x7f58280ad910 2024-04-26T21:53:41.225 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:41.224+0000 7f583e000700 1 -- 172.21.15.72:0/708162482 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7f582c0d0ed0 con 0x7f584010a410 2024-04-26T21:53:41.226 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:41.224+0000 7f5845600700 1 --2- 172.21.15.72:0/708162482 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f58280a3cd0 0x7f58280a6190 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f583400b5c0 tx=0x7f583401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:53:41.226 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:41.225+0000 7f5846000700 1 --2- 172.21.15.72:0/708162482 >> [v2:172.21.15.76:6824/2808954459,v1:172.21.15.76:6825/2808954459] conn(0x7f58280ad910 0x7f58280afdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:41.227 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:41.226+0000 7f5846000700 1 --2- 172.21.15.72:0/708162482 >> [v2:172.21.15.76:6824/2808954459,v1:172.21.15.76:6825/2808954459] conn(0x7f58280ad910 0x7f58280afdb0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:53:41.229 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:41.228+0000 7f583e000700 1 -- 172.21.15.72:0/708162482 <== osd.10 v2:172.21.15.76:6824/2808954459 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f582c009e20 con 0x7f58280ad910 2024-04-26T21:53:41.248 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:41.246+0000 7f5848000700 1 -- 172.21.15.72:0/708162482 --> [v2:172.21.15.76:6824/2808954459,v1:172.21.15.76:6825/2808954459] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f5840064030 con 0x7f58280ad910 2024-04-26T21:53:41.249 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:41.247+0000 7f583e000700 1 -- 172.21.15.72:0/708162482 <== osd.10 v2:172.21.15.76:6824/2808954459 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f5840064030 con 0x7f58280ad910 2024-04-26T21:53:41.249 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:41.248+0000 7f5848000700 1 -- 172.21.15.72:0/708162482 >> [v2:172.21.15.76:6824/2808954459,v1:172.21.15.76:6825/2808954459] conn(0x7f58280ad910 msgr2=0x7f58280afdb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:41.249 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:41.248+0000 7f5848000700 1 --2- 172.21.15.72:0/708162482 >> [v2:172.21.15.76:6824/2808954459,v1:172.21.15.76:6825/2808954459] conn(0x7f58280ad910 0x7f58280afdb0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:41.250 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:41.248+0000 7f5848000700 1 -- 172.21.15.72:0/708162482 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f58280a3cd0 msgr2=0x7f58280a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:41.250 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:41.248+0000 7f5848000700 1 --2- 172.21.15.72:0/708162482 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f58280a3cd0 0x7f58280a6190 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f583400b5c0 tx=0x7f583401a040 comp rx=0 tx=0).stop 2024-04-26T21:53:41.250 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:41.248+0000 7f5848000700 1 -- 172.21.15.72:0/708162482 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f584010a410 msgr2=0x7f58401a0530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:41.250 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:41.248+0000 7f5848000700 1 --2- 172.21.15.72:0/708162482 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f584010a410 0x7f58401a0530 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f582c00d980 tx=0x7f582c00dd40 comp rx=0 tx=0).stop 2024-04-26T21:53:41.250 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:41.248+0000 7f5846000700 1 -- 172.21.15.72:0/708162482 reap_dead start 2024-04-26T21:53:41.251 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:41.249+0000 7f5848000700 1 -- 172.21.15.72:0/708162482 shutdown_connections 2024-04-26T21:53:41.251 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:41.249+0000 7f5848000700 1 -- 172.21.15.72:0/708162482 >> 172.21.15.72:0/708162482 conn(0x7f5840076bf0 msgr2=0x7f58400772b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:53:41.251 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:41.249+0000 7f5848000700 1 -- 172.21.15.72:0/708162482 shutdown_connections 2024-04-26T21:53:41.251 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:41.249+0000 7f5848000700 1 -- 172.21.15.72:0/708162482 wait complete. 2024-04-26T21:53:41.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:53:40 smithi076 ceph-mon[31823]: pgmap v394: 97 pgs: 97 active+clean; 633 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-26T21:53:41.386 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:41.384+0000 7f526c600700 1 -- 172.21.15.72:0/2046633188 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5264078270 msgr2=0x7f5264113d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:41.386 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:41.384+0000 7f526c600700 1 --2- 172.21.15.72:0/2046633188 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5264078270 0x7f5264113d70 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f5250009990 tx=0x7f5250009ca0 comp rx=0 tx=0).stop 2024-04-26T21:53:41.386 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:41.385+0000 7f526c600700 1 -- 172.21.15.72:0/2046633188 shutdown_connections 2024-04-26T21:53:41.387 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:41.385+0000 7f526c600700 1 --2- 172.21.15.72:0/2046633188 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f52641142b0 0x7f52641166a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:41.387 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:41.385+0000 7f526c600700 1 --2- 172.21.15.72:0/2046633188 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5264077950 0x7f5264077d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:41.387 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:41.385+0000 7f526c600700 1 --2- 172.21.15.72:0/2046633188 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5264078270 0x7f5264113d70 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:41.387 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:41.385+0000 7f526c600700 1 -- 172.21.15.72:0/2046633188 >> 172.21.15.72:0/2046633188 conn(0x7f5264101100 msgr2=0x7f5264103520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:53:41.387 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:41.385+0000 7f526c600700 1 -- 172.21.15.72:0/2046633188 shutdown_connections 2024-04-26T21:53:41.387 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:41.385+0000 7f526c600700 1 -- 172.21.15.72:0/2046633188 wait complete. 2024-04-26T21:53:41.387 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:41.386+0000 7f526c600700 1 Processor -- start 2024-04-26T21:53:41.387 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:41.386+0000 7f526c600700 1 -- start start 2024-04-26T21:53:41.388 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:41.387+0000 7f526c600700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5264077950 0x7f52641a0040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:41.388 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:41.387+0000 7f526c600700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5264078270 0x7f52641a0580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:41.388 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:41.387+0000 7f526c600700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f52641142b0 0x7f52641a4910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:41.388 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:41.387+0000 7f526c600700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f5264119af0 con 0x7f5264078270 2024-04-26T21:53:41.388 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:41.387+0000 7f526c600700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f5264119970 con 0x7f52641142b0 2024-04-26T21:53:41.388 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:41.387+0000 7f526c600700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f5264119c70 con 0x7f5264077950 2024-04-26T21:53:41.389 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:41.387+0000 7f5269200700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5264078270 0x7f52641a0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:41.389 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:41.387+0000 7f5269200700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5264078270 0x7f52641a0580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:57740/0 (socket says 172.21.15.72:57740) 2024-04-26T21:53:41.389 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:41.387+0000 7f5269200700 1 -- 172.21.15.72:0/4128703460 learned_addr learned my addr 172.21.15.72:0/4128703460 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:53:41.390 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:41.388+0000 7f5269c00700 1 --2- 172.21.15.72:0/4128703460 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5264077950 0x7f52641a0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:41.390 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:41.388+0000 7f526a600700 1 --2- 172.21.15.72:0/4128703460 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f52641142b0 0x7f52641a4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:41.390 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:41.388+0000 7f5269200700 1 -- 172.21.15.72:0/4128703460 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5264077950 msgr2=0x7f52641a0040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:41.390 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:41.388+0000 7f5269200700 1 --2- 172.21.15.72:0/4128703460 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5264077950 0x7f52641a0040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:41.390 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:41.388+0000 7f5269200700 1 -- 172.21.15.72:0/4128703460 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f52641142b0 msgr2=0x7f52641a4910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:41.390 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:41.388+0000 7f5269200700 1 --2- 172.21.15.72:0/4128703460 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f52641142b0 0x7f52641a4910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:41.390 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:41.388+0000 7f5269200700 1 -- 172.21.15.72:0/4128703460 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52641a5050 con 0x7f5264078270 2024-04-26T21:53:41.390 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:41.388+0000 7f5269200700 1 --2- 172.21.15.72:0/4128703460 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5264078270 0x7f52641a0580 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f5250000c00 tx=0x7f5250009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:53:41.390 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:41.388+0000 7f5262000700 1 -- 172.21.15.72:0/4128703460 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5250027070 con 0x7f5264078270 2024-04-26T21:53:41.390 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:41.389+0000 7f5262000700 1 -- 172.21.15.72:0/4128703460 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f525000fc70 con 0x7f5264078270 2024-04-26T21:53:41.390 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:41.389+0000 7f526c600700 1 -- 172.21.15.72:0/4128703460 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52641a52e0 con 0x7f5264078270 2024-04-26T21:53:41.391 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:41.389+0000 7f5262000700 1 -- 172.21.15.72:0/4128703460 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5250022d40 con 0x7f5264078270 2024-04-26T21:53:41.391 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:41.389+0000 7f526c600700 1 -- 172.21.15.72:0/4128703460 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52641acbc0 con 0x7f5264078270 2024-04-26T21:53:41.391 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:41.390+0000 7f526c600700 1 -- 172.21.15.72:0/4128703460 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5264063bb0 con 0x7f5264078270 2024-04-26T21:53:41.393 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:41.392+0000 7f5262000700 1 -- 172.21.15.72:0/4128703460 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f525002a070 con 0x7f5264078270 2024-04-26T21:53:41.394 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:41.393+0000 7f5262000700 1 --2- 172.21.15.72:0/4128703460 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f524c0a3cd0 0x7f524c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:41.394 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:41.393+0000 7f5262000700 1 -- 172.21.15.72:0/4128703460 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8269+0+0 (secure 0 0 0) 0x7f52500d1d60 con 0x7f5264078270 2024-04-26T21:53:41.395 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:41.393+0000 7f5269c00700 1 --2- 172.21.15.72:0/4128703460 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f524c0a3cd0 0x7f524c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:41.395 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:41.394+0000 7f5262000700 1 --2- 172.21.15.72:0/4128703460 >> [v2:172.21.15.72:6810/4027763789,v1:172.21.15.72:6811/4027763789] conn(0x7f524c0ad910 0x7f524c0afdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:41.395 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:41.394+0000 7f5262000700 1 -- 172.21.15.72:0/4128703460 --> [v2:172.21.15.72:6810/4027763789,v1:172.21.15.72:6811/4027763789] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f52500095d0 con 0x7f524c0ad910 2024-04-26T21:53:41.395 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:41.394+0000 7f5262000700 1 -- 172.21.15.72:0/4128703460 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7f52500d21b0 con 0x7f5264078270 2024-04-26T21:53:41.395 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:41.394+0000 7f526a600700 1 --2- 172.21.15.72:0/4128703460 >> [v2:172.21.15.72:6810/4027763789,v1:172.21.15.72:6811/4027763789] conn(0x7f524c0ad910 0x7f524c0afdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:41.396 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:41.394+0000 7f5269c00700 1 --2- 172.21.15.72:0/4128703460 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f524c0a3cd0 0x7f524c0a6190 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f5258005950 tx=0x7f52580058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:53:41.396 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:41.395+0000 7f526a600700 1 --2- 172.21.15.72:0/4128703460 >> [v2:172.21.15.72:6810/4027763789,v1:172.21.15.72:6811/4027763789] conn(0x7f524c0ad910 0x7f524c0afdb0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:53:41.397 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:41.396+0000 7f5262000700 1 -- 172.21.15.72:0/4128703460 <== osd.3 v2:172.21.15.72:6810/4027763789 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f52500095d0 con 0x7f524c0ad910 2024-04-26T21:53:41.415 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:41.414+0000 7f526c600700 1 -- 172.21.15.72:0/4128703460 --> [v2:172.21.15.72:6810/4027763789,v1:172.21.15.72:6811/4027763789] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f5264064030 con 0x7f524c0ad910 2024-04-26T21:53:41.415 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:41.414+0000 7f5262000700 1 -- 172.21.15.72:0/4128703460 <== osd.3 v2:172.21.15.72:6810/4027763789 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f5264064030 con 0x7f524c0ad910 2024-04-26T21:53:41.416 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:41.414+0000 7f526c600700 1 -- 172.21.15.72:0/4128703460 >> [v2:172.21.15.72:6810/4027763789,v1:172.21.15.72:6811/4027763789] conn(0x7f524c0ad910 msgr2=0x7f524c0afdb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:41.416 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:41.414+0000 7f526c600700 1 --2- 172.21.15.72:0/4128703460 >> [v2:172.21.15.72:6810/4027763789,v1:172.21.15.72:6811/4027763789] conn(0x7f524c0ad910 0x7f524c0afdb0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:41.416 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:41.415+0000 7f526c600700 1 -- 172.21.15.72:0/4128703460 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f524c0a3cd0 msgr2=0x7f524c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:41.416 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:41.415+0000 7f526c600700 1 --2- 172.21.15.72:0/4128703460 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f524c0a3cd0 0x7f524c0a6190 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f5258005950 tx=0x7f52580058e0 comp rx=0 tx=0).stop 2024-04-26T21:53:41.416 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:41.415+0000 7f526c600700 1 -- 172.21.15.72:0/4128703460 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5264078270 msgr2=0x7f52641a0580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:41.416 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:41.415+0000 7f526c600700 1 --2- 172.21.15.72:0/4128703460 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5264078270 0x7f52641a0580 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f5250000c00 tx=0x7f5250009f90 comp rx=0 tx=0).stop 2024-04-26T21:53:41.416 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:41.415+0000 7f526a600700 1 -- 172.21.15.72:0/4128703460 reap_dead start 2024-04-26T21:53:41.417 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:41.415+0000 7f526c600700 1 -- 172.21.15.72:0/4128703460 shutdown_connections 2024-04-26T21:53:41.417 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:41.415+0000 7f526c600700 1 -- 172.21.15.72:0/4128703460 >> 172.21.15.72:0/4128703460 conn(0x7f5264101100 msgr2=0x7f52641034f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:53:41.417 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:41.415+0000 7f526c600700 1 -- 172.21.15.72:0/4128703460 shutdown_connections 2024-04-26T21:53:41.417 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:41.415+0000 7f526c600700 1 -- 172.21.15.72:0/4128703460 wait complete. 2024-04-26T21:53:42.658 INFO:teuthology.orchestra.run.smithi072.stdout:317827579937 2024-04-26T21:53:42.658 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid a1780950-0415-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-04-26T21:53:42.739 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:39.830+0000 7f7f11600700 1 -- 172.21.15.72:0/4231803161 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7f0c06b130 msgr2=0x7f7f0c06b590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:42.740 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:39.830+0000 7f7f11600700 1 --2- 172.21.15.72:0/4231803161 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7f0c06b130 0x7f7f0c06b590 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f7ef8009990 tx=0x7f7ef8009ca0 comp rx=0 tx=0).stop 2024-04-26T21:53:42.740 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:39.830+0000 7f7f11600700 1 -- 172.21.15.72:0/4231803161 shutdown_connections 2024-04-26T21:53:42.740 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:39.830+0000 7f7f11600700 1 --2- 172.21.15.72:0/4231803161 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7f0c06bad0 0x7f7f0c1102e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:42.740 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:39.830+0000 7f7f11600700 1 --2- 172.21.15.72:0/4231803161 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7f0c1094a0 0x7f7f0c109880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:42.740 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:39.830+0000 7f7f11600700 1 --2- 172.21.15.72:0/4231803161 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7f0c06b130 0x7f7f0c06b590 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:42.740 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:39.830+0000 7f7f11600700 1 -- 172.21.15.72:0/4231803161 >> 172.21.15.72:0/4231803161 conn(0x7f7f0c0792d0 msgr2=0x7f7f0c0796e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:53:42.740 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:39.830+0000 7f7f11600700 1 -- 172.21.15.72:0/4231803161 shutdown_connections 2024-04-26T21:53:42.740 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:39.830+0000 7f7f11600700 1 -- 172.21.15.72:0/4231803161 wait complete. 2024-04-26T21:53:42.740 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:39.831+0000 7f7f11600700 1 Processor -- start 2024-04-26T21:53:42.740 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:39.832+0000 7f7f11600700 1 -- start start 2024-04-26T21:53:42.740 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:39.832+0000 7f7f11600700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7f0c06b130 0x7f7f0c1a18c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:42.740 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:39.832+0000 7f7f11600700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7f0c06bad0 0x7f7f0c1a1e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:42.740 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:39.832+0000 7f7f11600700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7f0c1094a0 0x7f7f0c19ba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:42.740 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:39.832+0000 7f7f11600700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f7f0c113480 con 0x7f7f0c1094a0 2024-04-26T21:53:42.741 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:39.832+0000 7f7f11600700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f7f0c113300 con 0x7f7f0c06b130 2024-04-26T21:53:42.741 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:39.832+0000 7f7f11600700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f7f0c113600 con 0x7f7f0c06bad0 2024-04-26T21:53:42.741 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:39.833+0000 7f7f0b400700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7f0c1094a0 0x7f7f0c19ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:42.741 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:39.833+0000 7f7f0b400700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7f0c1094a0 0x7f7f0c19ba80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:56422/0 (socket says 172.21.15.72:56422) 2024-04-26T21:53:42.741 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:39.833+0000 7f7f0b400700 1 -- 172.21.15.72:0/4007421820 learned_addr learned my addr 172.21.15.72:0/4007421820 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:53:42.741 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:39.833+0000 7f7f0aa00700 1 --2- 172.21.15.72:0/4007421820 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7f0c06b130 0x7f7f0c1a18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:42.741 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:39.833+0000 7f7f0a000700 1 --2- 172.21.15.72:0/4007421820 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7f0c06bad0 0x7f7f0c1a1e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:42.741 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:39.833+0000 7f7f0b400700 1 -- 172.21.15.72:0/4007421820 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7f0c06bad0 msgr2=0x7f7f0c1a1e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:42.741 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:39.833+0000 7f7f0b400700 1 --2- 172.21.15.72:0/4007421820 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7f0c06bad0 0x7f7f0c1a1e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:42.741 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:39.833+0000 7f7f0b400700 1 -- 172.21.15.72:0/4007421820 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7f0c06b130 msgr2=0x7f7f0c1a18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:42.741 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:39.833+0000 7f7f0b400700 1 --2- 172.21.15.72:0/4007421820 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7f0c06b130 0x7f7f0c1a18c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:42.741 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:39.833+0000 7f7f0b400700 1 -- 172.21.15.72:0/4007421820 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f0c19c1e0 con 0x7f7f0c1094a0 2024-04-26T21:53:42.741 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:39.834+0000 7f7f0b400700 1 --2- 172.21.15.72:0/4007421820 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7f0c1094a0 0x7f7f0c19ba80 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f7f0400ada0 tx=0x7f7f0400c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:53:42.741 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:39.834+0000 7f7f03400700 1 -- 172.21.15.72:0/4007421820 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f04017020 con 0x7f7f0c1094a0 2024-04-26T21:53:42.741 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:39.834+0000 7f7f03400700 1 -- 172.21.15.72:0/4007421820 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7f0400cd60 con 0x7f7f0c1094a0 2024-04-26T21:53:42.742 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:39.834+0000 7f7f03400700 1 -- 172.21.15.72:0/4007421820 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f04005280 con 0x7f7f0c1094a0 2024-04-26T21:53:42.742 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:39.834+0000 7f7f11600700 1 -- 172.21.15.72:0/4007421820 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f0c19c4d0 con 0x7f7f0c1094a0 2024-04-26T21:53:42.742 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:39.834+0000 7f7f11600700 1 -- 172.21.15.72:0/4007421820 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f0c1a8790 con 0x7f7f0c1094a0 2024-04-26T21:53:42.742 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:39.835+0000 7f7f11600700 1 -- 172.21.15.72:0/4007421820 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7f0c063bb0 con 0x7f7f0c1094a0 2024-04-26T21:53:42.742 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:39.839+0000 7f7f03400700 1 -- 172.21.15.72:0/4007421820 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f7f0400c8d0 con 0x7f7f0c1094a0 2024-04-26T21:53:42.742 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:39.842+0000 7f7f03400700 1 --2- 172.21.15.72:0/4007421820 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f7ef00a3cd0 0x7f7ef00a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:42.742 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:39.842+0000 7f7f03400700 1 -- 172.21.15.72:0/4007421820 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8269+0+0 (secure 0 0 0) 0x7f7f040d0730 con 0x7f7f0c1094a0 2024-04-26T21:53:42.742 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:39.842+0000 7f7f0aa00700 1 --2- 172.21.15.72:0/4007421820 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f7ef00a3cd0 0x7f7ef00a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:42.742 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:39.842+0000 7f7f03400700 1 --2- 172.21.15.72:0/4007421820 >> [v2:172.21.15.164:6808/2543954772,v1:172.21.15.164:6809/2543954772] conn(0x7f7ef00ad910 0x7f7ef00afdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:42.742 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:39.842+0000 7f7f03400700 1 -- 172.21.15.72:0/4007421820 --> [v2:172.21.15.164:6808/2543954772,v1:172.21.15.164:6809/2543954772] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7f04019e50 con 0x7f7ef00ad910 2024-04-26T21:53:42.742 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:39.842+0000 7f7f03400700 1 -- 172.21.15.72:0/4007421820 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7f7f040d0ae0 con 0x7f7f0c1094a0 2024-04-26T21:53:42.742 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:39.843+0000 7f7f0aa00700 1 --2- 172.21.15.72:0/4007421820 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f7ef00a3cd0 0x7f7ef00a6190 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f7efc005950 tx=0x7f7efc00a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:53:42.742 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:39.843+0000 7f7f0a000700 1 --2- 172.21.15.72:0/4007421820 >> [v2:172.21.15.164:6808/2543954772,v1:172.21.15.164:6809/2543954772] conn(0x7f7ef00ad910 0x7f7ef00afdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:42.743 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:39.844+0000 7f7f0a000700 1 --2- 172.21.15.72:0/4007421820 >> [v2:172.21.15.164:6808/2543954772,v1:172.21.15.164:6809/2543954772] conn(0x7f7ef00ad910 0x7f7ef00afdb0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:53:42.743 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:39.846+0000 7f7f03400700 1 -- 172.21.15.72:0/4007421820 <== osd.5 v2:172.21.15.164:6808/2543954772 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7f04019e50 con 0x7f7ef00ad910 2024-04-26T21:53:42.743 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:39.864+0000 7f7f11600700 1 -- 172.21.15.72:0/4007421820 --> [v2:172.21.15.164:6808/2543954772,v1:172.21.15.164:6809/2543954772] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f7f0c063fb0 con 0x7f7ef00ad910 2024-04-26T21:53:42.743 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:39.865+0000 7f7f03400700 1 -- 172.21.15.72:0/4007421820 <== osd.5 v2:172.21.15.164:6808/2543954772 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f7f0c063fb0 con 0x7f7ef00ad910 2024-04-26T21:53:42.743 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:39.866+0000 7f7f11600700 1 -- 172.21.15.72:0/4007421820 >> [v2:172.21.15.164:6808/2543954772,v1:172.21.15.164:6809/2543954772] conn(0x7f7ef00ad910 msgr2=0x7f7ef00afdb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:42.743 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:39.866+0000 7f7f11600700 1 --2- 172.21.15.72:0/4007421820 >> [v2:172.21.15.164:6808/2543954772,v1:172.21.15.164:6809/2543954772] conn(0x7f7ef00ad910 0x7f7ef00afdb0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:42.743 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:39.866+0000 7f7f11600700 1 -- 172.21.15.72:0/4007421820 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f7ef00a3cd0 msgr2=0x7f7ef00a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:42.743 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:39.866+0000 7f7f11600700 1 --2- 172.21.15.72:0/4007421820 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f7ef00a3cd0 0x7f7ef00a6190 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f7efc005950 tx=0x7f7efc00a300 comp rx=0 tx=0).stop 2024-04-26T21:53:42.744 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:39.866+0000 7f7f11600700 1 -- 172.21.15.72:0/4007421820 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7f0c1094a0 msgr2=0x7f7f0c19ba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:42.744 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:39.866+0000 7f7f11600700 1 --2- 172.21.15.72:0/4007421820 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7f0c1094a0 0x7f7f0c19ba80 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f7f0400ada0 tx=0x7f7f0400c330 comp rx=0 tx=0).stop 2024-04-26T21:53:42.744 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:39.866+0000 7f7f0b400700 1 -- 172.21.15.72:0/4007421820 reap_dead start 2024-04-26T21:53:42.744 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:39.866+0000 7f7f11600700 1 -- 172.21.15.72:0/4007421820 shutdown_connections 2024-04-26T21:53:42.744 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:39.866+0000 7f7f11600700 1 -- 172.21.15.72:0/4007421820 >> 172.21.15.72:0/4007421820 conn(0x7f7f0c0792d0 msgr2=0x7f7f0c10e980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:53:42.744 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:39.866+0000 7f7f11600700 1 -- 172.21.15.72:0/4007421820 shutdown_connections 2024-04-26T21:53:42.744 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:39.866+0000 7f7f11600700 1 -- 172.21.15.72:0/4007421820 wait complete. 2024-04-26T21:53:43.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:53:42 smithi164 ceph-mon[31916]: pgmap v395: 97 pgs: 97 active+clean; 633 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-04-26T21:53:43.223 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:53:42 smithi072 ceph-mon[28138]: pgmap v395: 97 pgs: 97 active+clean; 633 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-04-26T21:53:43.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:53:42 smithi076 ceph-mon[31823]: pgmap v395: 97 pgs: 97 active+clean; 633 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-04-26T21:53:43.416 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:42.346+0000 7f5515600700 1 -- 172.21.15.72:0/756295946 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f551010fec0 msgr2=0x7f55101122b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:43.416 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:42.346+0000 7f5515600700 1 --2- 172.21.15.72:0/756295946 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f551010fec0 0x7f55101122b0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f5508009990 tx=0x7f5508009ca0 comp rx=0 tx=0).stop 2024-04-26T21:53:43.416 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:42.346+0000 7f5515600700 1 -- 172.21.15.72:0/756295946 shutdown_connections 2024-04-26T21:53:43.417 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:42.346+0000 7f5515600700 1 --2- 172.21.15.72:0/756295946 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f551010fec0 0x7f55101122b0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:43.417 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:42.346+0000 7f5515600700 1 --2- 172.21.15.72:0/756295946 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f551006b9c0 0x7f551006bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:43.417 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:42.346+0000 7f5515600700 1 --2- 172.21.15.72:0/756295946 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f551006c2e0 0x7f551010f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:43.417 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:42.346+0000 7f5515600700 1 -- 172.21.15.72:0/756295946 >> 172.21.15.72:0/756295946 conn(0x7f5510079290 msgr2=0x7f55100796a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:53:43.417 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:42.346+0000 7f5515600700 1 -- 172.21.15.72:0/756295946 shutdown_connections 2024-04-26T21:53:43.417 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:42.346+0000 7f5515600700 1 -- 172.21.15.72:0/756295946 wait complete. 2024-04-26T21:53:43.417 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:42.347+0000 7f5515600700 1 Processor -- start 2024-04-26T21:53:43.417 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:42.347+0000 7f5515600700 1 -- start start 2024-04-26T21:53:43.417 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:42.348+0000 7f5515600700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f551006b9c0 0x7f55101a1880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:43.417 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:42.348+0000 7f5515600700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f551006c2e0 0x7f55101a1dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:43.417 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:42.348+0000 7f5515600700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f551010fec0 0x7f551019ba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:43.417 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:42.348+0000 7f5515600700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f55101156d0 con 0x7f551010fec0 2024-04-26T21:53:43.417 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:42.348+0000 7f5515600700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f5510115550 con 0x7f551006c2e0 2024-04-26T21:53:43.417 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:42.348+0000 7f5515600700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f5510115850 con 0x7f551006b9c0 2024-04-26T21:53:43.417 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:42.348+0000 7f550f400700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f551010fec0 0x7f551019ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:43.418 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:42.348+0000 7f550f400700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f551010fec0 0x7f551019ba90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:57766/0 (socket says 172.21.15.72:57766) 2024-04-26T21:53:43.418 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:42.348+0000 7f550f400700 1 -- 172.21.15.72:0/3478579248 learned_addr learned my addr 172.21.15.72:0/3478579248 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:53:43.418 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:42.349+0000 7f5506000700 1 --2- 172.21.15.72:0/3478579248 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f551006c2e0 0x7f55101a1dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:43.418 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:42.348+0000 7f550ea00700 1 --2- 172.21.15.72:0/3478579248 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f551006b9c0 0x7f55101a1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:43.418 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:42.349+0000 7f550f400700 1 -- 172.21.15.72:0/3478579248 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f551006b9c0 msgr2=0x7f55101a1880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:43.418 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:42.349+0000 7f550f400700 1 --2- 172.21.15.72:0/3478579248 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f551006b9c0 0x7f55101a1880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:43.418 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:42.349+0000 7f550f400700 1 -- 172.21.15.72:0/3478579248 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f551006c2e0 msgr2=0x7f55101a1dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:43.418 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:42.349+0000 7f550f400700 1 --2- 172.21.15.72:0/3478579248 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f551006c2e0 0x7f55101a1dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:43.418 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:42.349+0000 7f550f400700 1 -- 172.21.15.72:0/3478579248 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f551019c1f0 con 0x7f551010fec0 2024-04-26T21:53:43.418 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:42.349+0000 7f550f400700 1 --2- 172.21.15.72:0/3478579248 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f551010fec0 0x7f551019ba90 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f55080057f0 tx=0x7f5508009ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:53:43.418 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:42.349+0000 7f5507e00700 1 -- 172.21.15.72:0/3478579248 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5508027070 con 0x7f551010fec0 2024-04-26T21:53:43.419 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:42.349+0000 7f5507e00700 1 -- 172.21.15.72:0/3478579248 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f550800fc00 con 0x7f551010fec0 2024-04-26T21:53:43.419 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:42.349+0000 7f5507e00700 1 -- 172.21.15.72:0/3478579248 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5508022d60 con 0x7f551010fec0 2024-04-26T21:53:43.419 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:42.349+0000 7f5515600700 1 -- 172.21.15.72:0/3478579248 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f551019c480 con 0x7f551010fec0 2024-04-26T21:53:43.419 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:42.350+0000 7f5515600700 1 -- 172.21.15.72:0/3478579248 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55101a8750 con 0x7f551010fec0 2024-04-26T21:53:43.419 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:42.350+0000 7f5515600700 1 -- 172.21.15.72:0/3478579248 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5510063b30 con 0x7f551010fec0 2024-04-26T21:53:43.419 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:42.353+0000 7f5507e00700 1 -- 172.21.15.72:0/3478579248 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f550802a070 con 0x7f551010fec0 2024-04-26T21:53:43.419 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:42.354+0000 7f5507e00700 1 --2- 172.21.15.72:0/3478579248 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f54f40a3dd0 0x7f54f40a6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:43.419 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:42.354+0000 7f5507e00700 1 -- 172.21.15.72:0/3478579248 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8269+0+0 (secure 0 0 0) 0x7f55080d18b0 con 0x7f551010fec0 2024-04-26T21:53:43.419 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:42.354+0000 7f550ea00700 1 --2- 172.21.15.72:0/3478579248 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f54f40a3dd0 0x7f54f40a6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:43.419 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:42.355+0000 7f5507e00700 1 --2- 172.21.15.72:0/3478579248 >> [v2:172.21.15.164:6800/852265973,v1:172.21.15.164:6801/852265973] conn(0x7f54f40ada10 0x7f54f40afeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:43.420 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:42.355+0000 7f5507e00700 1 -- 172.21.15.72:0/3478579248 --> [v2:172.21.15.164:6800/852265973,v1:172.21.15.164:6801/852265973] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5508009030 con 0x7f54f40ada10 2024-04-26T21:53:43.420 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:42.355+0000 7f5507e00700 1 -- 172.21.15.72:0/3478579248 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7f55080d1ca0 con 0x7f551010fec0 2024-04-26T21:53:43.420 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:42.355+0000 7f550ea00700 1 --2- 172.21.15.72:0/3478579248 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f54f40a3dd0 0x7f54f40a6290 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f55000060b0 tx=0x7f5500007790 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:53:43.420 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:42.355+0000 7f5506000700 1 --2- 172.21.15.72:0/3478579248 >> [v2:172.21.15.164:6800/852265973,v1:172.21.15.164:6801/852265973] conn(0x7f54f40ada10 0x7f54f40afeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:43.420 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:42.356+0000 7f5506000700 1 --2- 172.21.15.72:0/3478579248 >> [v2:172.21.15.164:6800/852265973,v1:172.21.15.164:6801/852265973] conn(0x7f54f40ada10 0x7f54f40afeb0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:53:43.420 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:42.359+0000 7f5507e00700 1 -- 172.21.15.72:0/3478579248 <== osd.2 v2:172.21.15.164:6800/852265973 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5508009030 con 0x7f54f40ada10 2024-04-26T21:53:43.420 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:42.377+0000 7f5515600700 1 -- 172.21.15.72:0/3478579248 --> [v2:172.21.15.164:6800/852265973,v1:172.21.15.164:6801/852265973] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f5510063f30 con 0x7f54f40ada10 2024-04-26T21:53:43.420 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:42.378+0000 7f5507e00700 1 -- 172.21.15.72:0/3478579248 <== osd.2 v2:172.21.15.164:6800/852265973 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f5510063f30 con 0x7f54f40ada10 2024-04-26T21:53:43.420 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:42.378+0000 7f5515600700 1 -- 172.21.15.72:0/3478579248 >> [v2:172.21.15.164:6800/852265973,v1:172.21.15.164:6801/852265973] conn(0x7f54f40ada10 msgr2=0x7f54f40afeb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:43.420 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:42.378+0000 7f5515600700 1 --2- 172.21.15.72:0/3478579248 >> [v2:172.21.15.164:6800/852265973,v1:172.21.15.164:6801/852265973] conn(0x7f54f40ada10 0x7f54f40afeb0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:43.420 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:42.378+0000 7f5515600700 1 -- 172.21.15.72:0/3478579248 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f54f40a3dd0 msgr2=0x7f54f40a6290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:43.420 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:42.378+0000 7f5515600700 1 --2- 172.21.15.72:0/3478579248 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f54f40a3dd0 0x7f54f40a6290 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f55000060b0 tx=0x7f5500007790 comp rx=0 tx=0).stop 2024-04-26T21:53:43.420 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:42.378+0000 7f5515600700 1 -- 172.21.15.72:0/3478579248 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f551010fec0 msgr2=0x7f551019ba90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:43.421 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:42.378+0000 7f5515600700 1 --2- 172.21.15.72:0/3478579248 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f551010fec0 0x7f551019ba90 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f55080057f0 tx=0x7f5508009ee0 comp rx=0 tx=0).stop 2024-04-26T21:53:43.421 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:42.378+0000 7f550f400700 1 -- 172.21.15.72:0/3478579248 reap_dead start 2024-04-26T21:53:43.421 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:42.379+0000 7f5515600700 1 -- 172.21.15.72:0/3478579248 shutdown_connections 2024-04-26T21:53:43.421 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:42.379+0000 7f5515600700 1 -- 172.21.15.72:0/3478579248 >> 172.21.15.72:0/3478579248 conn(0x7f5510079290 msgr2=0x7f55101113b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:53:43.421 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:42.379+0000 7f5515600700 1 -- 172.21.15.72:0/3478579248 shutdown_connections 2024-04-26T21:53:43.421 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:42.379+0000 7f5515600700 1 -- 172.21.15.72:0/3478579248 wait complete. 2024-04-26T21:53:44.172 INFO:teuthology.orchestra.run.smithi072.stdout:167503724611 2024-04-26T21:53:44.172 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid a1780950-0415-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-04-26T21:53:44.792 INFO:teuthology.orchestra.run.smithi072.stdout:77309411418 2024-04-26T21:53:44.793 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid a1780950-0415-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-04-26T21:53:44.993 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:53:44 smithi072 ceph-mon[28138]: pgmap v396: 97 pgs: 97 active+clean; 633 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 10 op/s 2024-04-26T21:53:45.062 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:45.061+0000 7f044e200700 1 -- 172.21.15.72:0/255735325 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0448102cd0 msgr2=0x7f044810f1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:45.062 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:45.061+0000 7f044e200700 1 --2- 172.21.15.72:0/255735325 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0448102cd0 0x7f044810f1c0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f044000b0a0 tx=0x7f044000b3b0 comp rx=0 tx=0).stop 2024-04-26T21:53:45.062 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:45.061+0000 7f044e200700 1 -- 172.21.15.72:0/255735325 shutdown_connections 2024-04-26T21:53:45.062 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:45.061+0000 7f044e200700 1 --2- 172.21.15.72:0/255735325 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0448102cd0 0x7f044810f1c0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:45.062 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:45.061+0000 7f044e200700 1 --2- 172.21.15.72:0/255735325 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0448108380 0x7f0448108760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:45.062 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:45.061+0000 7f044e200700 1 --2- 172.21.15.72:0/255735325 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0448102310 0x7f0448102790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:45.063 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:45.061+0000 7f044e200700 1 -- 172.21.15.72:0/255735325 >> 172.21.15.72:0/255735325 conn(0x7f0448079290 msgr2=0x7f04480796a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:53:45.063 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:45.062+0000 7f044e200700 1 -- 172.21.15.72:0/255735325 shutdown_connections 2024-04-26T21:53:45.063 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:45.062+0000 7f044e200700 1 -- 172.21.15.72:0/255735325 wait complete. 2024-04-26T21:53:45.064 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:45.063+0000 7f044e200700 1 Processor -- start 2024-04-26T21:53:45.064 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:45.063+0000 7f044e200700 1 -- start start 2024-04-26T21:53:45.064 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:45.063+0000 7f044e200700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0448102310 0x7f044819f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:45.064 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:45.063+0000 7f044e200700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0448102cd0 0x7f044819fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:45.064 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:45.063+0000 7f044e200700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0448108380 0x7f0448199840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:45.064 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:45.063+0000 7f044e200700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f0448112370 con 0x7f0448102cd0 2024-04-26T21:53:45.064 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:45.063+0000 7f044e200700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f04481121f0 con 0x7f0448108380 2024-04-26T21:53:45.064 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:45.063+0000 7f044e200700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f04481124f0 con 0x7f0448102310 2024-04-26T21:53:45.065 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:45.063+0000 7f0446a00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0448102cd0 0x7f044819fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:45.065 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:45.063+0000 7f0446a00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0448102cd0 0x7f044819fc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:57784/0 (socket says 172.21.15.72:57784) 2024-04-26T21:53:45.065 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:45.064+0000 7f0446a00700 1 -- 172.21.15.72:0/139161298 learned_addr learned my addr 172.21.15.72:0/139161298 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:53:45.065 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:45.064+0000 7f0447400700 1 --2- 172.21.15.72:0/139161298 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0448102310 0x7f044819f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:45.065 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:45.064+0000 7f0447e00700 1 --2- 172.21.15.72:0/139161298 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0448108380 0x7f0448199840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:45.065 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:45.064+0000 7f0446a00700 1 -- 172.21.15.72:0/139161298 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0448102310 msgr2=0x7f044819f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:45.066 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:45.064+0000 7f0446a00700 1 --2- 172.21.15.72:0/139161298 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0448102310 0x7f044819f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:45.066 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:45.064+0000 7f0446a00700 1 -- 172.21.15.72:0/139161298 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0448108380 msgr2=0x7f0448199840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:45.066 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:45.064+0000 7f0446a00700 1 --2- 172.21.15.72:0/139161298 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0448108380 0x7f0448199840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:45.066 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:45.064+0000 7f0446a00700 1 -- 172.21.15.72:0/139161298 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f044819a020 con 0x7f0448102cd0 2024-04-26T21:53:45.066 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:45.064+0000 7f0447e00700 1 --2- 172.21.15.72:0/139161298 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0448108380 0x7f0448199840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:53:45.066 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:45.064+0000 7f0446a00700 1 --2- 172.21.15.72:0/139161298 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0448102cd0 0x7f044819fc00 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f043400d950 tx=0x7f043400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:53:45.066 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:45.065+0000 7f043fe00700 1 -- 172.21.15.72:0/139161298 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0434016070 con 0x7f0448102cd0 2024-04-26T21:53:45.066 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:45.065+0000 7f043fe00700 1 -- 172.21.15.72:0/139161298 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0434009dd0 con 0x7f0448102cd0 2024-04-26T21:53:45.066 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:45.065+0000 7f043fe00700 1 -- 172.21.15.72:0/139161298 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f043400f3c0 con 0x7f0448102cd0 2024-04-26T21:53:45.066 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:45.065+0000 7f044e200700 1 -- 172.21.15.72:0/139161298 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f044819a310 con 0x7f0448102cd0 2024-04-26T21:53:45.066 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:45.065+0000 7f044e200700 1 -- 172.21.15.72:0/139161298 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04481a6520 con 0x7f0448102cd0 2024-04-26T21:53:45.067 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:45.066+0000 7f044e200700 1 -- 172.21.15.72:0/139161298 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f044806bc60 con 0x7f0448102cd0 2024-04-26T21:53:45.069 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:45.069+0000 7f043fe00700 1 -- 172.21.15.72:0/139161298 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f0434019070 con 0x7f0448102cd0 2024-04-26T21:53:45.071 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:45.070+0000 7f043fe00700 1 --2- 172.21.15.72:0/139161298 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f042c0a3cd0 0x7f042c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:45.071 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:45.071+0000 7f043fe00700 1 -- 172.21.15.72:0/139161298 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8269+0+0 (secure 0 0 0) 0x7f04340d0790 con 0x7f0448102cd0 2024-04-26T21:53:45.071 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:45.071+0000 7f0447400700 1 --2- 172.21.15.72:0/139161298 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f042c0a3cd0 0x7f042c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:45.071 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:45.071+0000 7f043fe00700 1 --2- 172.21.15.72:0/139161298 >> [v2:172.21.15.76:6800/755426949,v1:172.21.15.76:6801/755426949] conn(0x7f042c0ad910 0x7f042c0afdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:45.071 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:45.071+0000 7f043fe00700 1 -- 172.21.15.72:0/139161298 --> [v2:172.21.15.76:6800/755426949,v1:172.21.15.76:6801/755426949] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0434013ec0 con 0x7f042c0ad910 2024-04-26T21:53:45.071 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:45.071+0000 7f043fe00700 1 -- 172.21.15.72:0/139161298 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7f04340d0b40 con 0x7f0448102cd0 2024-04-26T21:53:45.072 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:45.072+0000 7f0447400700 1 --2- 172.21.15.72:0/139161298 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f042c0a3cd0 0x7f042c0a6190 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f04380045b0 tx=0x7f043800b3c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:53:45.072 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:45.072+0000 7f0447e00700 1 --2- 172.21.15.72:0/139161298 >> [v2:172.21.15.76:6800/755426949,v1:172.21.15.76:6801/755426949] conn(0x7f042c0ad910 0x7f042c0afdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:45.073 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:45.073+0000 7f0447e00700 1 --2- 172.21.15.72:0/139161298 >> [v2:172.21.15.76:6800/755426949,v1:172.21.15.76:6801/755426949] conn(0x7f042c0ad910 0x7f042c0afdb0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:53:45.075 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:45.075+0000 7f043fe00700 1 -- 172.21.15.72:0/139161298 <== osd.1 v2:172.21.15.76:6800/755426949 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0434013ec0 con 0x7f042c0ad910 2024-04-26T21:53:45.108 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:45.108+0000 7f044e200700 1 -- 172.21.15.72:0/139161298 --> [v2:172.21.15.76:6800/755426949,v1:172.21.15.76:6801/755426949] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0448103a30 con 0x7f042c0ad910 2024-04-26T21:53:45.109 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:45.109+0000 7f043fe00700 1 -- 172.21.15.72:0/139161298 <== osd.1 v2:172.21.15.76:6800/755426949 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f0448103a30 con 0x7f042c0ad910 2024-04-26T21:53:45.109 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:45.109+0000 7f044e200700 1 -- 172.21.15.72:0/139161298 >> [v2:172.21.15.76:6800/755426949,v1:172.21.15.76:6801/755426949] conn(0x7f042c0ad910 msgr2=0x7f042c0afdb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:45.109 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:45.109+0000 7f044e200700 1 --2- 172.21.15.72:0/139161298 >> [v2:172.21.15.76:6800/755426949,v1:172.21.15.76:6801/755426949] conn(0x7f042c0ad910 0x7f042c0afdb0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:45.110 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:45.110+0000 7f044e200700 1 -- 172.21.15.72:0/139161298 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f042c0a3cd0 msgr2=0x7f042c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:45.110 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:45.110+0000 7f044e200700 1 --2- 172.21.15.72:0/139161298 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f042c0a3cd0 0x7f042c0a6190 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f04380045b0 tx=0x7f043800b3c0 comp rx=0 tx=0).stop 2024-04-26T21:53:45.110 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:45.110+0000 7f044e200700 1 -- 172.21.15.72:0/139161298 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0448102cd0 msgr2=0x7f044819fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:45.110 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:45.110+0000 7f044e200700 1 --2- 172.21.15.72:0/139161298 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0448102cd0 0x7f044819fc00 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f043400d950 tx=0x7f043400dc60 comp rx=0 tx=0).stop 2024-04-26T21:53:45.110 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:45.110+0000 7f0447e00700 1 -- 172.21.15.72:0/139161298 reap_dead start 2024-04-26T21:53:45.110 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:45.110+0000 7f044e200700 1 -- 172.21.15.72:0/139161298 shutdown_connections 2024-04-26T21:53:45.111 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:45.110+0000 7f044e200700 1 -- 172.21.15.72:0/139161298 >> 172.21.15.72:0/139161298 conn(0x7f0448079290 msgr2=0x7f04481065d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:53:45.111 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:45.110+0000 7f044e200700 1 -- 172.21.15.72:0/139161298 shutdown_connections 2024-04-26T21:53:45.111 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:45.110+0000 7f044e200700 1 -- 172.21.15.72:0/139161298 wait complete. 2024-04-26T21:53:45.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:53:44 smithi164 ceph-mon[31916]: pgmap v396: 97 pgs: 97 active+clean; 633 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 10 op/s 2024-04-26T21:53:45.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:53:44 smithi076 ceph-mon[31823]: pgmap v396: 97 pgs: 97 active+clean; 633 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 10 op/s 2024-04-26T21:53:46.388 INFO:teuthology.orchestra.run.smithi072.stdout:201863462974 2024-04-26T21:53:46.389 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid a1780950-0415-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-04-26T21:53:47.103 INFO:teuthology.orchestra.run.smithi072.stdout:107374182481 2024-04-26T21:53:47.103 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid a1780950-0415-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-04-26T21:53:47.161 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:53:46 smithi072 ceph-mon[28138]: pgmap v397: 97 pgs: 97 active+clean; 633 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 11 op/s 2024-04-26T21:53:47.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:53:46 smithi164 ceph-mon[31916]: pgmap v397: 97 pgs: 97 active+clean; 633 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 11 op/s 2024-04-26T21:53:47.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:53:46 smithi076 ceph-mon[31823]: pgmap v397: 97 pgs: 97 active+clean; 633 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 11 op/s 2024-04-26T21:53:47.859 INFO:teuthology.orchestra.run.smithi072.stdout:141733920844 2024-04-26T21:53:47.860 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid a1780950-0415-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-04-26T21:53:48.067 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:48.066+0000 7f181ca00700 1 -- 172.21.15.72:0/1155710479 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f18181035b0 msgr2=0x7f1818103990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:48.067 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:48.066+0000 7f181ca00700 1 --2- 172.21.15.72:0/1155710479 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f18181035b0 0x7f1818103990 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f18080099c0 tx=0x7f1808009cd0 comp rx=0 tx=0).stop 2024-04-26T21:53:48.067 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:48.067+0000 7f181ca00700 1 -- 172.21.15.72:0/1155710479 shutdown_connections 2024-04-26T21:53:48.067 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:48.067+0000 7f181ca00700 1 --2- 172.21.15.72:0/1155710479 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f18181035b0 0x7f1818103990 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:48.068 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:48.067+0000 7f181ca00700 1 --2- 172.21.15.72:0/1155710479 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1818108690 0x7f1818110260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:48.068 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:48.067+0000 7f181ca00700 1 --2- 172.21.15.72:0/1155710479 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1818103f60 0x7f1818107fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:48.068 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:48.067+0000 7f181ca00700 1 -- 172.21.15.72:0/1155710479 >> 172.21.15.72:0/1155710479 conn(0x7f18180feea0 msgr2=0x7f18181012c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:53:48.068 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:48.067+0000 7f181ca00700 1 -- 172.21.15.72:0/1155710479 shutdown_connections 2024-04-26T21:53:48.068 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:48.067+0000 7f181ca00700 1 -- 172.21.15.72:0/1155710479 wait complete. 2024-04-26T21:53:48.069 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:48.068+0000 7f181ca00700 1 Processor -- start 2024-04-26T21:53:48.069 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:48.068+0000 7f181ca00700 1 -- start start 2024-04-26T21:53:48.069 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:48.068+0000 7f181ca00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f18181035b0 0x7f1818075720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:48.069 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:48.068+0000 7f181ca00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1818103f60 0x7f1818075c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:48.069 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:48.068+0000 7f181ca00700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1818108690 0x7f18180761a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:48.070 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:48.068+0000 7f181ca00700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f1818113440 con 0x7f1818103f60 2024-04-26T21:53:48.070 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:48.068+0000 7f181ca00700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f18181132c0 con 0x7f1818108690 2024-04-26T21:53:48.070 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:48.068+0000 7f181ca00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f18181135c0 con 0x7f18181035b0 2024-04-26T21:53:48.070 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:48.069+0000 7f180fe00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1818103f60 0x7f1818075c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:48.070 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:48.069+0000 7f180fe00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1818103f60 0x7f1818075c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:57808/0 (socket says 172.21.15.72:57808) 2024-04-26T21:53:48.070 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:48.069+0000 7f180fe00700 1 -- 172.21.15.72:0/2824669543 learned_addr learned my addr 172.21.15.72:0/2824669543 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:53:48.070 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:48.069+0000 7f1816800700 1 --2- 172.21.15.72:0/2824669543 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f18181035b0 0x7f1818075720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:48.070 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:48.069+0000 7f1817200700 1 --2- 172.21.15.72:0/2824669543 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1818108690 0x7f18180761a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:48.071 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:48.069+0000 7f180fe00700 1 -- 172.21.15.72:0/2824669543 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f18181035b0 msgr2=0x7f1818075720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:48.071 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:48.069+0000 7f180fe00700 1 --2- 172.21.15.72:0/2824669543 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f18181035b0 0x7f1818075720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:48.072 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:48.069+0000 7f180fe00700 1 -- 172.21.15.72:0/2824669543 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1818108690 msgr2=0x7f18180761a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:48.072 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:48.069+0000 7f180fe00700 1 --2- 172.21.15.72:0/2824669543 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1818108690 0x7f18180761a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:48.072 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:48.069+0000 7f180fe00700 1 -- 172.21.15.72:0/2824669543 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18181a7e00 con 0x7f1818103f60 2024-04-26T21:53:48.072 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:48.070+0000 7f180fe00700 1 --2- 172.21.15.72:0/2824669543 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1818103f60 0x7f1818075c60 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f180000c970 tx=0x7f180000cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:53:48.072 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:48.070+0000 7f180f400700 1 -- 172.21.15.72:0/2824669543 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1800007b00 con 0x7f1818103f60 2024-04-26T21:53:48.072 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:48.070+0000 7f180f400700 1 -- 172.21.15.72:0/2824669543 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1800004cc0 con 0x7f1818103f60 2024-04-26T21:53:48.072 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:48.070+0000 7f180f400700 1 -- 172.21.15.72:0/2824669543 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1800005580 con 0x7f1818103f60 2024-04-26T21:53:48.072 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:48.070+0000 7f181ca00700 1 -- 172.21.15.72:0/2824669543 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18181a80f0 con 0x7f1818103f60 2024-04-26T21:53:48.072 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:48.070+0000 7f181ca00700 1 -- 172.21.15.72:0/2824669543 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18181a85e0 con 0x7f1818103f60 2024-04-26T21:53:48.072 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:48.071+0000 7f181ca00700 1 -- 172.21.15.72:0/2824669543 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1818063b30 con 0x7f1818103f60 2024-04-26T21:53:48.075 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:48.075+0000 7f180f400700 1 -- 172.21.15.72:0/2824669543 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f1800007ca0 con 0x7f1818103f60 2024-04-26T21:53:48.077 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:48.077+0000 7f180f400700 1 --2- 172.21.15.72:0/2824669543 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f17fc0a3cd0 0x7f17fc0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:48.077 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:48.077+0000 7f180f400700 1 -- 172.21.15.72:0/2824669543 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8269+0+0 (secure 0 0 0) 0x7f18000d0670 con 0x7f1818103f60 2024-04-26T21:53:48.078 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:48.077+0000 7f1816800700 1 --2- 172.21.15.72:0/2824669543 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f17fc0a3cd0 0x7f17fc0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:48.078 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:48.078+0000 7f180f400700 1 --2- 172.21.15.72:0/2824669543 >> [v2:172.21.15.72:6802/1416765179,v1:172.21.15.72:6803/1416765179] conn(0x7f17fc0ad910 0x7f17fc0afdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:48.078 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:48.078+0000 7f180f400700 1 -- 172.21.15.72:0/2824669543 --> [v2:172.21.15.72:6802/1416765179,v1:172.21.15.72:6803/1416765179] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1800012ec0 con 0x7f17fc0ad910 2024-04-26T21:53:48.078 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:48.078+0000 7f180f400700 1 -- 172.21.15.72:0/2824669543 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7f18000d0a20 con 0x7f1818103f60 2024-04-26T21:53:48.078 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:48.078+0000 7f1817200700 1 --2- 172.21.15.72:0/2824669543 >> [v2:172.21.15.72:6802/1416765179,v1:172.21.15.72:6803/1416765179] conn(0x7f17fc0ad910 0x7f17fc0afdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:48.079 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:48.078+0000 7f1816800700 1 --2- 172.21.15.72:0/2824669543 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f17fc0a3cd0 0x7f17fc0a6190 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f1808000c00 tx=0x7f180801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:53:48.079 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:48.079+0000 7f1817200700 1 --2- 172.21.15.72:0/2824669543 >> [v2:172.21.15.72:6802/1416765179,v1:172.21.15.72:6803/1416765179] conn(0x7f17fc0ad910 0x7f17fc0afdb0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:53:48.080 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:48.080+0000 7f180f400700 1 -- 172.21.15.72:0/2824669543 <== osd.0 v2:172.21.15.72:6802/1416765179 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1800012ec0 con 0x7f17fc0ad910 2024-04-26T21:53:48.098 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:48.098+0000 7f181ca00700 1 -- 172.21.15.72:0/2824669543 --> [v2:172.21.15.72:6802/1416765179,v1:172.21.15.72:6803/1416765179] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f1818063fb0 con 0x7f17fc0ad910 2024-04-26T21:53:48.098 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:48.098+0000 7f180f400700 1 -- 172.21.15.72:0/2824669543 <== osd.0 v2:172.21.15.72:6802/1416765179 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f1818063fb0 con 0x7f17fc0ad910 2024-04-26T21:53:48.099 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:48.099+0000 7f181ca00700 1 -- 172.21.15.72:0/2824669543 >> [v2:172.21.15.72:6802/1416765179,v1:172.21.15.72:6803/1416765179] conn(0x7f17fc0ad910 msgr2=0x7f17fc0afdb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:48.099 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:48.099+0000 7f181ca00700 1 --2- 172.21.15.72:0/2824669543 >> [v2:172.21.15.72:6802/1416765179,v1:172.21.15.72:6803/1416765179] conn(0x7f17fc0ad910 0x7f17fc0afdb0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:48.100 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:48.099+0000 7f181ca00700 1 -- 172.21.15.72:0/2824669543 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f17fc0a3cd0 msgr2=0x7f17fc0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:48.100 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:48.100+0000 7f181ca00700 1 --2- 172.21.15.72:0/2824669543 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f17fc0a3cd0 0x7f17fc0a6190 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f1808000c00 tx=0x7f180801a040 comp rx=0 tx=0).stop 2024-04-26T21:53:48.100 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:48.100+0000 7f181ca00700 1 -- 172.21.15.72:0/2824669543 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1818103f60 msgr2=0x7f1818075c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:48.100 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:48.100+0000 7f181ca00700 1 --2- 172.21.15.72:0/2824669543 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1818103f60 0x7f1818075c60 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f180000c970 tx=0x7f180000cd30 comp rx=0 tx=0).stop 2024-04-26T21:53:48.101 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:48.100+0000 7f1817200700 1 -- 172.21.15.72:0/2824669543 reap_dead start 2024-04-26T21:53:48.101 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:48.100+0000 7f181ca00700 1 -- 172.21.15.72:0/2824669543 shutdown_connections 2024-04-26T21:53:48.101 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:48.100+0000 7f181ca00700 1 -- 172.21.15.72:0/2824669543 >> 172.21.15.72:0/2824669543 conn(0x7f18180feea0 msgr2=0x7f18181011c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:53:48.101 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:48.100+0000 7f181ca00700 1 -- 172.21.15.72:0/2824669543 shutdown_connections 2024-04-26T21:53:48.101 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:48.100+0000 7f181ca00700 1 -- 172.21.15.72:0/2824669543 wait complete. 2024-04-26T21:53:49.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:53:48 smithi164 ceph-mon[31916]: pgmap v398: 97 pgs: 97 active+clean; 633 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-26T21:53:49.211 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:53:48 smithi072 ceph-mon[28138]: pgmap v398: 97 pgs: 97 active+clean; 633 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-26T21:53:49.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:53:48 smithi076 ceph-mon[31823]: pgmap v398: 97 pgs: 97 active+clean; 633 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-26T21:53:49.537 INFO:teuthology.orchestra.run.smithi072.stdout:34359738473 2024-04-26T21:53:49.538 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid a1780950-0415-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-04-26T21:53:49.666 INFO:teuthology.orchestra.run.smithi072.stdout:55834574945 2024-04-26T21:53:49.667 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid a1780950-0415-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-04-26T21:53:49.986 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:49.954+0000 7f91d5200700 1 -- 172.21.15.72:0/1613326270 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f91d010fb90 msgr2=0x7f91d0112000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:49.987 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:49.954+0000 7f91d5200700 1 --2- 172.21.15.72:0/1613326270 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f91d010fb90 0x7f91d0112000 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f91c800b0a0 tx=0x7f91c800b3b0 comp rx=0 tx=0).stop 2024-04-26T21:53:49.987 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:49.954+0000 7f91d5200700 1 -- 172.21.15.72:0/1613326270 shutdown_connections 2024-04-26T21:53:49.987 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:49.954+0000 7f91d5200700 1 --2- 172.21.15.72:0/1613326270 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f91d010fb90 0x7f91d0112000 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:49.987 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:49.954+0000 7f91d5200700 1 --2- 172.21.15.72:0/1613326270 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f91d0075850 0x7f91d0075c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:49.987 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:49.954+0000 7f91d5200700 1 --2- 172.21.15.72:0/1613326270 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f91d0076170 0x7f91d010f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:49.987 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:49.954+0000 7f91d5200700 1 -- 172.21.15.72:0/1613326270 >> 172.21.15.72:0/1613326270 conn(0x7f91d00feea0 msgr2=0x7f91d01012c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:53:49.987 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:49.955+0000 7f91d5200700 1 -- 172.21.15.72:0/1613326270 shutdown_connections 2024-04-26T21:53:49.987 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:49.955+0000 7f91d5200700 1 -- 172.21.15.72:0/1613326270 wait complete. 2024-04-26T21:53:49.987 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:49.956+0000 7f91d5200700 1 Processor -- start 2024-04-26T21:53:49.987 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:49.956+0000 7f91d5200700 1 -- start start 2024-04-26T21:53:49.987 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:49.956+0000 7f91d5200700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f91d0075850 0x7f91d01a16c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:49.987 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:49.956+0000 7f91d5200700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f91d0076170 0x7f91d01a1c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:49.988 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:49.956+0000 7f91d5200700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f91d010fb90 0x7f91d019b840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:49.988 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:49.956+0000 7f91d5200700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f91d0115640 con 0x7f91d010fb90 2024-04-26T21:53:49.988 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:49.956+0000 7f91d5200700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f91d01154c0 con 0x7f91d0076170 2024-04-26T21:53:49.988 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:49.956+0000 7f91d5200700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f91d01157c0 con 0x7f91d0075850 2024-04-26T21:53:49.988 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:49.957+0000 7f91cfe00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f91d010fb90 0x7f91d019b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:49.988 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:49.957+0000 7f91cfe00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f91d010fb90 0x7f91d019b840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:57830/0 (socket says 172.21.15.72:57830) 2024-04-26T21:53:49.988 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:49.957+0000 7f91cfe00700 1 -- 172.21.15.72:0/1106875870 learned_addr learned my addr 172.21.15.72:0/1106875870 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:53:49.988 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:49.957+0000 7f91cf400700 1 --2- 172.21.15.72:0/1106875870 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f91d0075850 0x7f91d01a16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:49.988 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:49.957+0000 7f91cea00700 1 --2- 172.21.15.72:0/1106875870 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f91d0076170 0x7f91d01a1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:49.988 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:49.957+0000 7f91cfe00700 1 -- 172.21.15.72:0/1106875870 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f91d0075850 msgr2=0x7f91d01a16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:49.988 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:49.957+0000 7f91cfe00700 1 --2- 172.21.15.72:0/1106875870 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f91d0075850 0x7f91d01a16c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:49.988 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:49.957+0000 7f91cfe00700 1 -- 172.21.15.72:0/1106875870 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f91d0076170 msgr2=0x7f91d01a1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:49.988 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:49.957+0000 7f91cfe00700 1 --2- 172.21.15.72:0/1106875870 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f91d0076170 0x7f91d01a1c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:49.989 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:49.957+0000 7f91cfe00700 1 -- 172.21.15.72:0/1106875870 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91d019c020 con 0x7f91d010fb90 2024-04-26T21:53:49.989 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:49.958+0000 7f91cfe00700 1 --2- 172.21.15.72:0/1106875870 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f91d010fb90 0x7f91d019b840 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f91c8004dc0 tx=0x7f91c8004ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:53:49.989 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:49.958+0000 7f91c7e00700 1 -- 172.21.15.72:0/1106875870 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91c8021020 con 0x7f91d010fb90 2024-04-26T21:53:49.989 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:49.958+0000 7f91d5200700 1 -- 172.21.15.72:0/1106875870 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91d019c2b0 con 0x7f91d010fb90 2024-04-26T21:53:49.989 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:49.958+0000 7f91d5200700 1 -- 172.21.15.72:0/1106875870 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91d01a8520 con 0x7f91d010fb90 2024-04-26T21:53:49.989 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:49.958+0000 7f91c7e00700 1 -- 172.21.15.72:0/1106875870 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f91c8007cf0 con 0x7f91d010fb90 2024-04-26T21:53:49.989 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:49.958+0000 7f91c7e00700 1 -- 172.21.15.72:0/1106875870 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91c8005170 con 0x7f91d010fb90 2024-04-26T21:53:49.989 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:49.959+0000 7f91d5200700 1 -- 172.21.15.72:0/1106875870 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f91d0100e30 con 0x7f91d010fb90 2024-04-26T21:53:49.989 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:49.961+0000 7f91c7e00700 1 -- 172.21.15.72:0/1106875870 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f91c8024020 con 0x7f91d010fb90 2024-04-26T21:53:49.989 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:49.963+0000 7f91c7e00700 1 --2- 172.21.15.72:0/1106875870 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f91b40a3d50 0x7f91b40a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:49.989 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:49.963+0000 7f91c7e00700 1 -- 172.21.15.72:0/1106875870 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8269+0+0 (secure 0 0 0) 0x7f91c80d14b0 con 0x7f91d010fb90 2024-04-26T21:53:49.989 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:49.963+0000 7f91cf400700 1 --2- 172.21.15.72:0/1106875870 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f91b40a3d50 0x7f91b40a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:49.989 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:49.963+0000 7f91c7e00700 1 --2- 172.21.15.72:0/1106875870 >> [v2:172.21.15.76:6816/3144631323,v1:172.21.15.76:6817/3144631323] conn(0x7f91b40ad990 0x7f91b40afe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:49.989 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:49.963+0000 7f91c7e00700 1 -- 172.21.15.72:0/1106875870 --> [v2:172.21.15.76:6816/3144631323,v1:172.21.15.76:6817/3144631323] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f91b40b0500 con 0x7f91b40ad990 2024-04-26T21:53:49.990 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:49.963+0000 7f91c7e00700 1 -- 172.21.15.72:0/1106875870 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7f91c80d8050 con 0x7f91d010fb90 2024-04-26T21:53:49.990 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:49.964+0000 7f91cf400700 1 --2- 172.21.15.72:0/1106875870 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f91b40a3d50 0x7f91b40a6210 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f91c0005ea0 tx=0x7f91c0005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:53:49.990 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:49.964+0000 7f91cea00700 1 --2- 172.21.15.72:0/1106875870 >> [v2:172.21.15.76:6816/3144631323,v1:172.21.15.76:6817/3144631323] conn(0x7f91b40ad990 0x7f91b40afe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:49.990 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:49.965+0000 7f91cea00700 1 --2- 172.21.15.72:0/1106875870 >> [v2:172.21.15.76:6816/3144631323,v1:172.21.15.76:6817/3144631323] conn(0x7f91b40ad990 0x7f91b40afe30 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:53:49.990 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:49.967+0000 7f91c7e00700 1 -- 172.21.15.72:0/1106875870 <== osd.7 v2:172.21.15.76:6816/3144631323 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f91b40b0500 con 0x7f91b40ad990 2024-04-26T21:53:49.990 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:49.985+0000 7f91d5200700 1 -- 172.21.15.72:0/1106875870 --> [v2:172.21.15.76:6816/3144631323,v1:172.21.15.76:6817/3144631323] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f91d0076f00 con 0x7f91b40ad990 2024-04-26T21:53:49.990 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:49.986+0000 7f91c7e00700 1 -- 172.21.15.72:0/1106875870 <== osd.7 v2:172.21.15.76:6816/3144631323 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f91d0076f00 con 0x7f91b40ad990 2024-04-26T21:53:49.991 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:49.986+0000 7f91d5200700 1 -- 172.21.15.72:0/1106875870 >> [v2:172.21.15.76:6816/3144631323,v1:172.21.15.76:6817/3144631323] conn(0x7f91b40ad990 msgr2=0x7f91b40afe30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:49.991 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:49.986+0000 7f91d5200700 1 --2- 172.21.15.72:0/1106875870 >> [v2:172.21.15.76:6816/3144631323,v1:172.21.15.76:6817/3144631323] conn(0x7f91b40ad990 0x7f91b40afe30 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:49.991 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:49.987+0000 7f91d5200700 1 -- 172.21.15.72:0/1106875870 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f91b40a3d50 msgr2=0x7f91b40a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:49.991 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:49.987+0000 7f91d5200700 1 --2- 172.21.15.72:0/1106875870 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f91b40a3d50 0x7f91b40a6210 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f91c0005ea0 tx=0x7f91c0005e30 comp rx=0 tx=0).stop 2024-04-26T21:53:49.991 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:49.987+0000 7f91d5200700 1 -- 172.21.15.72:0/1106875870 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f91d010fb90 msgr2=0x7f91d019b840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:49.991 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:49.987+0000 7f91d5200700 1 --2- 172.21.15.72:0/1106875870 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f91d010fb90 0x7f91d019b840 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f91c8004dc0 tx=0x7f91c8004ea0 comp rx=0 tx=0).stop 2024-04-26T21:53:49.991 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:49.987+0000 7f91cfe00700 1 -- 172.21.15.72:0/1106875870 reap_dead start 2024-04-26T21:53:49.991 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:49.987+0000 7f91d5200700 1 -- 172.21.15.72:0/1106875870 shutdown_connections 2024-04-26T21:53:49.992 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:49.987+0000 7f91d5200700 1 -- 172.21.15.72:0/1106875870 >> 172.21.15.72:0/1106875870 conn(0x7f91d00feea0 msgr2=0x7f91d0111df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:53:49.992 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:49.987+0000 7f91d5200700 1 -- 172.21.15.72:0/1106875870 shutdown_connections 2024-04-26T21:53:49.992 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:49.987+0000 7f91d5200700 1 -- 172.21.15.72:0/1106875870 wait complete. 2024-04-26T21:53:50.209 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/a1780950-0415-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:53:50.210 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/a1780950-0415-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:53:50.220 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/a1780950-0415-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:53:50.220 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/a1780950-0415-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:53:50.848 INFO:teuthology.orchestra.run.smithi072.stdout:223338299447 2024-04-26T21:53:50.848 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid a1780950-0415-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-04-26T21:53:51.069 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:53:50 smithi072 ceph-mon[28138]: pgmap v399: 97 pgs: 97 active+clean; 633 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-26T21:53:51.088 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/a1780950-0415-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:53:51.091 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/a1780950-0415-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:53:51.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:53:50 smithi164 ceph-mon[31916]: pgmap v399: 97 pgs: 97 active+clean; 633 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-26T21:53:51.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:53:50 smithi076 ceph-mon[31823]: pgmap v399: 97 pgs: 97 active+clean; 633 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-26T21:53:53.055 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:53:52 smithi072 ceph-mon[28138]: pgmap v400: 97 pgs: 97 active+clean; 633 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-26T21:53:53.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:53:52 smithi164 ceph-mon[31916]: pgmap v400: 97 pgs: 97 active+clean; 633 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-26T21:53:53.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:53:52 smithi076 ceph-mon[31823]: pgmap v400: 97 pgs: 97 active+clean; 633 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-26T21:53:53.328 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/a1780950-0415-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:53:53.329 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/a1780950-0415-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:53:54.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:53:53 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:53:54.253 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:53:53 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:53:54.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:53:53 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:53:55.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:53:54 smithi072 ceph-mon[28138]: pgmap v401: 97 pgs: 97 active+clean; 633 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 8 op/s 2024-04-26T21:53:55.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:53:54 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:53:55.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:53:54 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:53:55.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:53:54 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:53:55.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:53:54 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:53:55.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:53:54 smithi164 ceph-mon[31916]: pgmap v401: 97 pgs: 97 active+clean; 633 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 8 op/s 2024-04-26T21:53:55.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:53:54 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:53:55.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:53:54 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:53:55.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:53:54 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:53:55.192 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:53:54 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:53:55.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:53:54 smithi076 ceph-mon[31823]: pgmap v401: 97 pgs: 97 active+clean; 633 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 8 op/s 2024-04-26T21:53:55.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:53:54 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:53:55.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:53:54 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:53:55.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:53:54 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:53:55.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:53:54 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:53:57.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:53:56 smithi164 ceph-mon[31916]: pgmap v402: 97 pgs: 97 active+clean; 633 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 10 op/s 2024-04-26T21:53:57.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:53:56 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:53:57.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:53:56 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:53:57.248 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:53:56 smithi072 ceph-mon[28138]: pgmap v402: 97 pgs: 97 active+clean; 633 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 10 op/s 2024-04-26T21:53:57.248 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:53:56 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:53:57.248 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:53:56 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:53:57.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:53:56 smithi076 ceph-mon[31823]: pgmap v402: 97 pgs: 97 active+clean; 633 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 10 op/s 2024-04-26T21:53:57.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:53:56 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:53:57.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:53:56 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:53:58.101 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/a1780950-0415-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:53:58.101 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/a1780950-0415-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:53:58.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:53:58 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:53:58.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:53:58 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:53:58.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:53:58 smithi072 ceph-mon[28138]: pgmap v403: 97 pgs: 97 active+clean; 633 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:53:58.565 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:53:58 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:53:58.565 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:53:58 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:53:58.565 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:53:58 smithi076 ceph-mon[31823]: pgmap v403: 97 pgs: 97 active+clean; 633 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:53:58.691 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:53:58 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:53:58.691 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:53:58 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:53:58.691 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:53:58 smithi164 ceph-mon[31916]: pgmap v403: 97 pgs: 97 active+clean; 633 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:53:59.490 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:58.703+0000 7ff267200700 1 -- 172.21.15.72:0/419837945 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff260109a50 msgr2=0x7ff260109ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:59.490 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:58.703+0000 7ff267200700 1 --2- 172.21.15.72:0/419837945 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff260109a50 0x7ff260109ed0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7ff24c009990 tx=0x7ff24c009ca0 comp rx=0 tx=0).stop 2024-04-26T21:53:59.490 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:58.703+0000 7ff267200700 1 -- 172.21.15.72:0/419837945 shutdown_connections 2024-04-26T21:53:59.490 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:58.703+0000 7ff267200700 1 --2- 172.21.15.72:0/419837945 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff26010a410 0x7ff260116920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:59.490 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:58.703+0000 7ff267200700 1 --2- 172.21.15.72:0/419837945 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff26010fa50 0x7ff26010fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:59.490 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:58.703+0000 7ff267200700 1 --2- 172.21.15.72:0/419837945 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff260109a50 0x7ff260109ed0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:59.490 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:58.703+0000 7ff267200700 1 -- 172.21.15.72:0/419837945 >> 172.21.15.72:0/419837945 conn(0x7ff260076bf0 msgr2=0x7ff260108240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:53:59.490 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:58.703+0000 7ff267200700 1 -- 172.21.15.72:0/419837945 shutdown_connections 2024-04-26T21:53:59.490 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:58.703+0000 7ff267200700 1 -- 172.21.15.72:0/419837945 wait complete. 2024-04-26T21:53:59.490 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:58.704+0000 7ff267200700 1 Processor -- start 2024-04-26T21:53:59.491 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:58.705+0000 7ff267200700 1 -- start start 2024-04-26T21:53:59.491 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:58.705+0000 7ff267200700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff260109a50 0x7ff26019fff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:59.491 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:58.705+0000 7ff267200700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff26010a410 0x7ff2601a0530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:59.491 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:58.705+0000 7ff267200700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff26010fa50 0x7ff2601a4870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:59.491 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:58.705+0000 7ff267200700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7ff260119b40 con 0x7ff260109a50 2024-04-26T21:53:59.491 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:58.705+0000 7ff267200700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7ff2601199c0 con 0x7ff26010a410 2024-04-26T21:53:59.491 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:58.705+0000 7ff267200700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ff260119cc0 con 0x7ff26010fa50 2024-04-26T21:53:59.491 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:58.705+0000 7ff25fe00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff260109a50 0x7ff26019fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:59.491 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:58.705+0000 7ff25fe00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff260109a50 0x7ff26019fff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:34060/0 (socket says 172.21.15.72:34060) 2024-04-26T21:53:59.491 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:58.705+0000 7ff25fe00700 1 -- 172.21.15.72:0/264035551 learned_addr learned my addr 172.21.15.72:0/264035551 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:53:59.491 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:58.706+0000 7ff25f400700 1 --2- 172.21.15.72:0/264035551 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff26010a410 0x7ff2601a0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:59.491 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:58.706+0000 7ff25fe00700 1 -- 172.21.15.72:0/264035551 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff26010fa50 msgr2=0x7ff2601a4870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:59.492 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:58.706+0000 7ff265200700 1 --2- 172.21.15.72:0/264035551 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff26010fa50 0x7ff2601a4870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:59.492 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:58.706+0000 7ff25fe00700 1 --2- 172.21.15.72:0/264035551 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff26010fa50 0x7ff2601a4870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:59.492 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:58.706+0000 7ff25fe00700 1 -- 172.21.15.72:0/264035551 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff26010a410 msgr2=0x7ff2601a0530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:59.492 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:58.706+0000 7ff25fe00700 1 --2- 172.21.15.72:0/264035551 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff26010a410 0x7ff2601a0530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:59.492 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:58.706+0000 7ff25fe00700 1 -- 172.21.15.72:0/264035551 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2601a4fb0 con 0x7ff260109a50 2024-04-26T21:53:59.492 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:58.706+0000 7ff25f400700 1 --2- 172.21.15.72:0/264035551 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff26010a410 0x7ff2601a0530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:53:59.492 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:58.706+0000 7ff25fe00700 1 --2- 172.21.15.72:0/264035551 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff260109a50 0x7ff26019fff0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7ff25000c970 tx=0x7ff25000cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:53:59.492 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:58.706+0000 7ff25cc00700 1 -- 172.21.15.72:0/264035551 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff250007aa0 con 0x7ff260109a50 2024-04-26T21:53:59.492 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:58.706+0000 7ff25cc00700 1 -- 172.21.15.72:0/264035551 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff250004cc0 con 0x7ff260109a50 2024-04-26T21:53:59.492 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:58.706+0000 7ff25cc00700 1 -- 172.21.15.72:0/264035551 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff250005500 con 0x7ff260109a50 2024-04-26T21:53:59.492 INFO:teuthology.orchestra.run.smithi072.stdout:107374182485 2024-04-26T21:53:59.493 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:58.706+0000 7ff267200700 1 -- 172.21.15.72:0/264035551 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2601a52a0 con 0x7ff260109a50 2024-04-26T21:53:59.493 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:58.707+0000 7ff267200700 1 -- 172.21.15.72:0/264035551 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2601acb20 con 0x7ff260109a50 2024-04-26T21:53:59.493 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:58.708+0000 7ff267200700 1 -- 172.21.15.72:0/264035551 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff260111fe0 con 0x7ff260109a50 2024-04-26T21:53:59.493 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:58.710+0000 7ff25cc00700 1 -- 172.21.15.72:0/264035551 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7ff250007c40 con 0x7ff260109a50 2024-04-26T21:53:59.493 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:58.711+0000 7ff25cc00700 1 --2- 172.21.15.72:0/264035551 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7ff2440a3d50 0x7ff2440a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:59.493 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:58.711+0000 7ff25cc00700 1 -- 172.21.15.72:0/264035551 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8269+0+0 (secure 0 0 0) 0x7ff2500d0330 con 0x7ff260109a50 2024-04-26T21:53:59.493 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:58.711+0000 7ff25f400700 1 --2- 172.21.15.72:0/264035551 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7ff2440a3d50 0x7ff2440a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:59.493 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:58.712+0000 7ff25f400700 1 --2- 172.21.15.72:0/264035551 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7ff2440a3d50 0x7ff2440a6210 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7ff24c00b5c0 tx=0x7ff24c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:53:59.494 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:58.716+0000 7ff25cc00700 1 -- 172.21.15.72:0/264035551 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7ff25009cbc0 con 0x7ff260109a50 2024-04-26T21:53:59.494 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:58.951+0000 7ff267200700 1 -- 172.21.15.72:0/264035551 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7ff26006a110 con 0x7ff260109a50 2024-04-26T21:53:59.494 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:58.952+0000 7ff25cc00700 1 -- 172.21.15.72:0/264035551 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7ff250005700 con 0x7ff260109a50 2024-04-26T21:53:59.494 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:58.954+0000 7ff267200700 1 -- 172.21.15.72:0/264035551 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7ff2440a3d50 msgr2=0x7ff2440a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:59.494 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:58.954+0000 7ff267200700 1 --2- 172.21.15.72:0/264035551 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7ff2440a3d50 0x7ff2440a6210 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7ff24c00b5c0 tx=0x7ff24c005fb0 comp rx=0 tx=0).stop 2024-04-26T21:53:59.494 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:58.954+0000 7ff267200700 1 -- 172.21.15.72:0/264035551 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff260109a50 msgr2=0x7ff26019fff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:59.494 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:58.954+0000 7ff267200700 1 --2- 172.21.15.72:0/264035551 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff260109a50 0x7ff26019fff0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7ff25000c970 tx=0x7ff25000cd30 comp rx=0 tx=0).stop 2024-04-26T21:53:59.494 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:58.955+0000 7ff267200700 1 -- 172.21.15.72:0/264035551 shutdown_connections 2024-04-26T21:53:59.494 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:58.955+0000 7ff267200700 1 --2- 172.21.15.72:0/264035551 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7ff2440a3d50 0x7ff2440a6210 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:59.494 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:58.955+0000 7ff267200700 1 --2- 172.21.15.72:0/264035551 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff260109a50 0x7ff26019fff0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:59.494 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:58.955+0000 7ff267200700 1 --2- 172.21.15.72:0/264035551 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff26010fa50 0x7ff2601a4870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:59.494 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:58.955+0000 7ff267200700 1 --2- 172.21.15.72:0/264035551 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff26010a410 0x7ff2601a0530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:59.494 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:58.955+0000 7ff267200700 1 -- 172.21.15.72:0/264035551 >> 172.21.15.72:0/264035551 conn(0x7ff260076bf0 msgr2=0x7ff260077250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:53:59.495 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:58.955+0000 7ff267200700 1 -- 172.21.15.72:0/264035551 shutdown_connections 2024-04-26T21:53:59.495 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:53:58.955+0000 7ff267200700 1 -- 172.21.15.72:0/264035551 wait complete. 2024-04-26T21:53:59.495 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:53:59 smithi072 ceph-mon[28138]: from='client.? 172.21.15.72:0/264035551' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-26T21:53:59.564 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:53:59 smithi076 ceph-mon[31823]: from='client.? 172.21.15.72:0/264035551' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-26T21:53:59.690 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:53:59 smithi164 ceph-mon[31916]: from='client.? 172.21.15.72:0/264035551' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-26T21:54:00.301 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:00.233+0000 7f286be00700 1 -- 172.21.15.72:0/277124099 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f286406bad0 msgr2=0x7f28641102e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:00.301 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:00.233+0000 7f286be00700 1 --2- 172.21.15.72:0/277124099 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f286406bad0 0x7f28641102e0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f285c00b0a0 tx=0x7f285c00b3b0 comp rx=0 tx=0).stop 2024-04-26T21:54:00.301 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:00.233+0000 7f286be00700 1 -- 172.21.15.72:0/277124099 shutdown_connections 2024-04-26T21:54:00.301 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:00.233+0000 7f286be00700 1 --2- 172.21.15.72:0/277124099 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f286406bad0 0x7f28641102e0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:00.301 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:00.233+0000 7f286be00700 1 --2- 172.21.15.72:0/277124099 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f28641094a0 0x7f2864109880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:00.301 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:00.233+0000 7f286be00700 1 --2- 172.21.15.72:0/277124099 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f286406b130 0x7f286406b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:00.301 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:00.233+0000 7f286be00700 1 -- 172.21.15.72:0/277124099 >> 172.21.15.72:0/277124099 conn(0x7f28640792d0 msgr2=0x7f28640796e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:54:00.301 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:00.233+0000 7f286be00700 1 -- 172.21.15.72:0/277124099 shutdown_connections 2024-04-26T21:54:00.302 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:00.233+0000 7f286be00700 1 -- 172.21.15.72:0/277124099 wait complete. 2024-04-26T21:54:00.302 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:00.234+0000 7f286be00700 1 Processor -- start 2024-04-26T21:54:00.302 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:00.235+0000 7f286be00700 1 -- start start 2024-04-26T21:54:00.302 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:00.235+0000 7f286be00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f286406b130 0x7f28641a1880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:00.302 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:00.235+0000 7f286be00700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f286406bad0 0x7f28641a1dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:00.302 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:00.235+0000 7f286be00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f28641094a0 0x7f286419ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:00.302 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:00.235+0000 7f286be00700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f28641134c0 con 0x7f286406b130 2024-04-26T21:54:00.302 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:00.235+0000 7f286be00700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f2864113340 con 0x7f286406bad0 2024-04-26T21:54:00.302 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:00.235+0000 7f286be00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f2864113640 con 0x7f28641094a0 2024-04-26T21:54:00.302 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:00.235+0000 7f2869400700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f286406b130 0x7f28641a1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:00.302 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:00.235+0000 7f2869400700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f286406b130 0x7f28641a1880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:34082/0 (socket says 172.21.15.72:34082) 2024-04-26T21:54:00.302 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:00.235+0000 7f2869400700 1 -- 172.21.15.72:0/3295160716 learned_addr learned my addr 172.21.15.72:0/3295160716 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:54:00.302 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:00.236+0000 7f2868a00700 1 --2- 172.21.15.72:0/3295160716 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f286406bad0 0x7f28641a1dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:00.303 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:00.236+0000 7f2869400700 1 -- 172.21.15.72:0/3295160716 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f28641094a0 msgr2=0x7f286419ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:00.303 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:00.236+0000 7f2869e00700 1 --2- 172.21.15.72:0/3295160716 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f28641094a0 0x7f286419ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:00.303 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:00.236+0000 7f2869400700 1 --2- 172.21.15.72:0/3295160716 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f28641094a0 0x7f286419ba40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:00.303 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:00.236+0000 7f2869400700 1 -- 172.21.15.72:0/3295160716 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f286406bad0 msgr2=0x7f28641a1dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:00.303 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:00.236+0000 7f2869400700 1 --2- 172.21.15.72:0/3295160716 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f286406bad0 0x7f28641a1dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:00.303 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:00.236+0000 7f2869400700 1 -- 172.21.15.72:0/3295160716 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f286419c1a0 con 0x7f286406b130 2024-04-26T21:54:00.303 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:00.236+0000 7f2869400700 1 --2- 172.21.15.72:0/3295160716 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f286406b130 0x7f28641a1880 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f285800d980 tx=0x7f285800dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:54:00.303 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:00.236+0000 7f2862000700 1 -- 172.21.15.72:0/3295160716 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2858017070 con 0x7f286406b130 2024-04-26T21:54:00.303 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:00.236+0000 7f2862000700 1 -- 172.21.15.72:0/3295160716 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2858004830 con 0x7f286406b130 2024-04-26T21:54:00.303 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:00.236+0000 7f286be00700 1 -- 172.21.15.72:0/3295160716 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f286419c430 con 0x7f286406b130 2024-04-26T21:54:00.303 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:00.236+0000 7f2862000700 1 -- 172.21.15.72:0/3295160716 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28580053a0 con 0x7f286406b130 2024-04-26T21:54:00.303 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:00.236+0000 7f286be00700 1 -- 172.21.15.72:0/3295160716 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28641a8750 con 0x7f286406b130 2024-04-26T21:54:00.303 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:00.238+0000 7f286be00700 1 -- 172.21.15.72:0/3295160716 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f286406a110 con 0x7f286406b130 2024-04-26T21:54:00.303 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:00.239+0000 7f2862000700 1 -- 172.21.15.72:0/3295160716 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f28580049e0 con 0x7f286406b130 2024-04-26T21:54:00.304 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:00.241+0000 7f2862000700 1 --2- 172.21.15.72:0/3295160716 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f284c0a3cd0 0x7f284c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:00.304 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:00.241+0000 7f2862000700 1 -- 172.21.15.72:0/3295160716 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8269+0+0 (secure 0 0 0) 0x7f28580d0510 con 0x7f286406b130 2024-04-26T21:54:00.304 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:00.241+0000 7f2868a00700 1 --2- 172.21.15.72:0/3295160716 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f284c0a3cd0 0x7f284c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:00.304 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:00.242+0000 7f2868a00700 1 --2- 172.21.15.72:0/3295160716 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f284c0a3cd0 0x7f284c0a6190 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f2850005950 tx=0x7f28500058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:54:00.304 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:00.245+0000 7f2862000700 1 -- 172.21.15.72:0/3295160716 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f2858020020 con 0x7f286406b130 2024-04-26T21:54:00.410 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:54:00 smithi072 ceph-mon[28138]: pgmap v404: 97 pgs: 97 active+clean; 633 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-26T21:54:00.495 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:00.494+0000 7f286be00700 1 -- 172.21.15.72:0/3295160716 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7f28641a8920 con 0x7f286406b130 2024-04-26T21:54:00.495 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:00.495+0000 7f2862000700 1 -- 172.21.15.72:0/3295160716 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f285809cd10 con 0x7f286406b130 2024-04-26T21:54:00.495 INFO:teuthology.orchestra.run.smithi072.stdout:261993005109 2024-04-26T21:54:00.497 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:00.497+0000 7f286be00700 1 -- 172.21.15.72:0/3295160716 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f284c0a3cd0 msgr2=0x7f284c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:00.498 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:00.497+0000 7f286be00700 1 --2- 172.21.15.72:0/3295160716 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f284c0a3cd0 0x7f284c0a6190 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f2850005950 tx=0x7f28500058e0 comp rx=0 tx=0).stop 2024-04-26T21:54:00.498 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:00.497+0000 7f286be00700 1 -- 172.21.15.72:0/3295160716 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f286406b130 msgr2=0x7f28641a1880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:00.498 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:00.497+0000 7f286be00700 1 --2- 172.21.15.72:0/3295160716 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f286406b130 0x7f28641a1880 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f285800d980 tx=0x7f285800dd40 comp rx=0 tx=0).stop 2024-04-26T21:54:00.498 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:00.498+0000 7f286be00700 1 -- 172.21.15.72:0/3295160716 shutdown_connections 2024-04-26T21:54:00.499 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:00.498+0000 7f286be00700 1 --2- 172.21.15.72:0/3295160716 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f284c0a3cd0 0x7f284c0a6190 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:00.499 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:00.498+0000 7f286be00700 1 --2- 172.21.15.72:0/3295160716 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f286406b130 0x7f28641a1880 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:00.499 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:00.498+0000 7f286be00700 1 --2- 172.21.15.72:0/3295160716 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f28641094a0 0x7f286419ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:00.499 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:00.498+0000 7f286be00700 1 --2- 172.21.15.72:0/3295160716 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f286406bad0 0x7f28641a1dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:00.499 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:00.498+0000 7f286be00700 1 -- 172.21.15.72:0/3295160716 >> 172.21.15.72:0/3295160716 conn(0x7f28640792d0 msgr2=0x7f2864100190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:54:00.499 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:00.498+0000 7f286be00700 1 -- 172.21.15.72:0/3295160716 shutdown_connections 2024-04-26T21:54:00.499 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:00.498+0000 7f286be00700 1 -- 172.21.15.72:0/3295160716 wait complete. 2024-04-26T21:54:00.565 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:54:00 smithi076 ceph-mon[31823]: pgmap v404: 97 pgs: 97 active+clean; 633 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-26T21:54:00.690 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:54:00 smithi164 ceph-mon[31916]: pgmap v404: 97 pgs: 97 active+clean; 633 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-26T21:54:01.565 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:54:01 smithi076 ceph-mon[31823]: from='client.? 172.21.15.72:0/3295160716' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-26T21:54:01.565 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:54:01 smithi076 ceph-mon[31823]: from='client.? 172.21.15.72:0/3800275929' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-26T21:54:01.565 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:54:01 smithi076 ceph-mon[31823]: from='client.? 172.21.15.72:0/473627857' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-26T21:54:01.572 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:54:01 smithi072 ceph-mon[28138]: from='client.? 172.21.15.72:0/3295160716' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-26T21:54:01.572 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:54:01 smithi072 ceph-mon[28138]: from='client.? 172.21.15.72:0/3800275929' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-26T21:54:01.572 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:54:01 smithi072 ceph-mon[28138]: from='client.? 172.21.15.72:0/473627857' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-26T21:54:01.579 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:00.945+0000 7f5ae6200700 1 -- 172.21.15.72:0/2244348159 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5ae010fe20 msgr2=0x7f5ae0112210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:01.580 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:00.945+0000 7f5ae6200700 1 --2- 172.21.15.72:0/2244348159 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5ae010fe20 0x7f5ae0112210 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f5ad800b0a0 tx=0x7f5ad800b3b0 comp rx=0 tx=0).stop 2024-04-26T21:54:01.580 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:00.945+0000 7f5ae6200700 1 -- 172.21.15.72:0/2244348159 shutdown_connections 2024-04-26T21:54:01.580 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:00.945+0000 7f5ae6200700 1 --2- 172.21.15.72:0/2244348159 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5ae010fe20 0x7f5ae0112210 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:01.580 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:00.945+0000 7f5ae6200700 1 --2- 172.21.15.72:0/2244348159 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5ae0075650 0x7f5ae0075a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:01.580 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:00.945+0000 7f5ae6200700 1 --2- 172.21.15.72:0/2244348159 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5ae0075f70 0x7f5ae010f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:01.580 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:00.945+0000 7f5ae6200700 1 -- 172.21.15.72:0/2244348159 >> 172.21.15.72:0/2244348159 conn(0x7f5ae007a580 msgr2=0x7f5ae007a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:54:01.580 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:00.945+0000 7f5ae6200700 1 -- 172.21.15.72:0/2244348159 shutdown_connections 2024-04-26T21:54:01.580 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:00.945+0000 7f5ae6200700 1 -- 172.21.15.72:0/2244348159 wait complete. 2024-04-26T21:54:01.580 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:00.946+0000 7f5ae6200700 1 Processor -- start 2024-04-26T21:54:01.580 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:00.947+0000 7f5ae6200700 1 -- start start 2024-04-26T21:54:01.580 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:00.947+0000 7f5ae6200700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5ae0075650 0x7f5ae01a17a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:01.580 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:00.947+0000 7f5ae6200700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5ae0075f70 0x7f5ae01a1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:01.580 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:00.947+0000 7f5ae6200700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5ae010fe20 0x7f5ae019b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:01.580 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:00.947+0000 7f5ae6200700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f5ae01156d0 con 0x7f5ae010fe20 2024-04-26T21:54:01.581 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:00.947+0000 7f5ae6200700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f5ae0115550 con 0x7f5ae0075650 2024-04-26T21:54:01.581 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:00.947+0000 7f5ae6200700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f5ae0115850 con 0x7f5ae0075f70 2024-04-26T21:54:01.581 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:00.948+0000 7f5adfe00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5ae010fe20 0x7f5ae019b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:01.581 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:00.948+0000 7f5adfe00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5ae010fe20 0x7f5ae019b9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:52062/0 (socket says 172.21.15.72:52062) 2024-04-26T21:54:01.581 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:00.948+0000 7f5adfe00700 1 -- 172.21.15.72:0/473627857 learned_addr learned my addr 172.21.15.72:0/473627857 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:54:01.581 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:00.948+0000 7f5adea00700 1 --2- 172.21.15.72:0/473627857 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5ae0075f70 0x7f5ae01a1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:01.581 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:00.948+0000 7f5adfe00700 1 -- 172.21.15.72:0/473627857 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5ae0075f70 msgr2=0x7f5ae01a1ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:01.581 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:00.948+0000 7f5adfe00700 1 --2- 172.21.15.72:0/473627857 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5ae0075f70 0x7f5ae01a1ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:01.581 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:00.948+0000 7f5adfe00700 1 -- 172.21.15.72:0/473627857 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5ae0075650 msgr2=0x7f5ae01a17a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:01.581 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:00.948+0000 7f5adfe00700 1 --2- 172.21.15.72:0/473627857 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5ae0075650 0x7f5ae01a17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:01.581 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:00.948+0000 7f5adfe00700 1 -- 172.21.15.72:0/473627857 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ae019c110 con 0x7f5ae010fe20 2024-04-26T21:54:01.581 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:00.949+0000 7f5adfe00700 1 --2- 172.21.15.72:0/473627857 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5ae010fe20 0x7f5ae019b9b0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f5ad800b070 tx=0x7f5ad8004dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:54:01.581 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:00.949+0000 7f5ad7e00700 1 -- 172.21.15.72:0/473627857 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ad8021070 con 0x7f5ae010fe20 2024-04-26T21:54:01.581 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:00.949+0000 7f5ad7e00700 1 -- 172.21.15.72:0/473627857 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5ad8007c00 con 0x7f5ae010fe20 2024-04-26T21:54:01.582 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:00.949+0000 7f5ad7e00700 1 -- 172.21.15.72:0/473627857 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ad8004780 con 0x7f5ae010fe20 2024-04-26T21:54:01.582 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:00.949+0000 7f5ae6200700 1 -- 172.21.15.72:0/473627857 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ae019c3a0 con 0x7f5ae010fe20 2024-04-26T21:54:01.582 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:00.949+0000 7f5ae6200700 1 -- 172.21.15.72:0/473627857 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ae01a8670 con 0x7f5ae010fe20 2024-04-26T21:54:01.582 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:00.951+0000 7f5ae6200700 1 -- 172.21.15.72:0/473627857 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ae0076d40 con 0x7f5ae010fe20 2024-04-26T21:54:01.582 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:00.954+0000 7f5ad7e00700 1 -- 172.21.15.72:0/473627857 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f5ad8003a00 con 0x7f5ae010fe20 2024-04-26T21:54:01.582 INFO:teuthology.orchestra.run.smithi072.stdout:201863462978 2024-04-26T21:54:01.582 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:00.955+0000 7f5ad7e00700 1 --2- 172.21.15.72:0/473627857 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f5ac40a3d50 0x7f5ac40a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:01.582 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:00.955+0000 7f5ad7e00700 1 -- 172.21.15.72:0/473627857 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8269+0+0 (secure 0 0 0) 0x7f5ad800f020 con 0x7f5ae010fe20 2024-04-26T21:54:01.583 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:00.955+0000 7f5adf400700 1 --2- 172.21.15.72:0/473627857 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f5ac40a3d50 0x7f5ac40a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:01.583 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:00.956+0000 7f5adf400700 1 --2- 172.21.15.72:0/473627857 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f5ac40a3d50 0x7f5ac40a6210 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f5ae0103200 tx=0x7f5ad0005c80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:54:01.583 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:00.960+0000 7f5ad7e00700 1 -- 172.21.15.72:0/473627857 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f5ad802a020 con 0x7f5ae010fe20 2024-04-26T21:54:01.583 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:01.189+0000 7f5ae6200700 1 -- 172.21.15.72:0/473627857 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f5ae002e490 con 0x7f5ae010fe20 2024-04-26T21:54:01.583 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:01.189+0000 7f5ad7e00700 1 -- 172.21.15.72:0/473627857 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f5ad809e490 con 0x7f5ae010fe20 2024-04-26T21:54:01.583 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:01.192+0000 7f5ae6200700 1 -- 172.21.15.72:0/473627857 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f5ac40a3d50 msgr2=0x7f5ac40a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:01.583 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:01.192+0000 7f5ae6200700 1 --2- 172.21.15.72:0/473627857 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f5ac40a3d50 0x7f5ac40a6210 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f5ae0103200 tx=0x7f5ad0005c80 comp rx=0 tx=0).stop 2024-04-26T21:54:01.583 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:01.192+0000 7f5ae6200700 1 -- 172.21.15.72:0/473627857 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5ae010fe20 msgr2=0x7f5ae019b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:01.583 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:01.192+0000 7f5ae6200700 1 --2- 172.21.15.72:0/473627857 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5ae010fe20 0x7f5ae019b9b0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f5ad800b070 tx=0x7f5ad8004dc0 comp rx=0 tx=0).stop 2024-04-26T21:54:01.583 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:01.193+0000 7f5ae6200700 1 -- 172.21.15.72:0/473627857 shutdown_connections 2024-04-26T21:54:01.583 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:01.193+0000 7f5ae6200700 1 --2- 172.21.15.72:0/473627857 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f5ac40a3d50 0x7f5ac40a6210 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:01.583 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:01.193+0000 7f5ae6200700 1 --2- 172.21.15.72:0/473627857 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5ae010fe20 0x7f5ae019b9b0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:01.583 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:01.193+0000 7f5ae6200700 1 --2- 172.21.15.72:0/473627857 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5ae0075650 0x7f5ae01a17a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:01.583 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:01.193+0000 7f5ae6200700 1 --2- 172.21.15.72:0/473627857 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5ae0075f70 0x7f5ae01a1ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:01.583 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:01.193+0000 7f5ae6200700 1 -- 172.21.15.72:0/473627857 >> 172.21.15.72:0/473627857 conn(0x7f5ae007a580 msgr2=0x7f5ae0100630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:54:01.584 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:01.193+0000 7f5ae6200700 1 -- 172.21.15.72:0/473627857 shutdown_connections 2024-04-26T21:54:01.584 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:01.193+0000 7f5ae6200700 1 -- 172.21.15.72:0/473627857 wait complete. 2024-04-26T21:54:01.690 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:54:01 smithi164 ceph-mon[31916]: from='client.? 172.21.15.72:0/3295160716' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-26T21:54:01.691 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:54:01 smithi164 ceph-mon[31916]: from='client.? 172.21.15.72:0/3800275929' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-26T21:54:01.691 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:54:01 smithi164 ceph-mon[31916]: from='client.? 172.21.15.72:0/473627857' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-26T21:54:01.911 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182481 got 107374182485 for osd.3 2024-04-26T21:54:01.911 DEBUG:teuthology.parallel:result is None 2024-04-26T21:54:02.422 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:54:02 smithi072 ceph-mon[28138]: pgmap v405: 97 pgs: 97 active+clean; 633 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:54:02.690 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:54:02 smithi164 ceph-mon[31916]: pgmap v405: 97 pgs: 97 active+clean; 633 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:54:02.814 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:54:02 smithi076 ceph-mon[31823]: pgmap v405: 97 pgs: 97 active+clean; 633 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:54:04.465 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:03.785+0000 7f2d6d400700 1 -- 172.21.15.72:0/2747575344 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2d68108690 msgr2=0x7f2d68110260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:04.465 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:03.785+0000 7f2d6d400700 1 --2- 172.21.15.72:0/2747575344 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2d68108690 0x7f2d68110260 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f2d6000b0d0 tx=0x7f2d6000b3e0 comp rx=0 tx=0).stop 2024-04-26T21:54:04.465 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:03.785+0000 7f2d6d400700 1 -- 172.21.15.72:0/2747575344 shutdown_connections 2024-04-26T21:54:04.466 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:03.785+0000 7f2d6d400700 1 --2- 172.21.15.72:0/2747575344 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2d68108690 0x7f2d68110260 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:04.466 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:03.785+0000 7f2d6d400700 1 --2- 172.21.15.72:0/2747575344 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2d681035b0 0x7f2d68103990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:04.466 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:03.785+0000 7f2d6d400700 1 --2- 172.21.15.72:0/2747575344 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2d68103f60 0x7f2d68107fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:04.466 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:03.785+0000 7f2d6d400700 1 -- 172.21.15.72:0/2747575344 >> 172.21.15.72:0/2747575344 conn(0x7f2d680feea0 msgr2=0x7f2d681012c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:54:04.466 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:03.785+0000 7f2d6d400700 1 -- 172.21.15.72:0/2747575344 shutdown_connections 2024-04-26T21:54:04.467 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:03.785+0000 7f2d6d400700 1 -- 172.21.15.72:0/2747575344 wait complete. 2024-04-26T21:54:04.467 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:03.786+0000 7f2d6d400700 1 Processor -- start 2024-04-26T21:54:04.467 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:03.787+0000 7f2d6d400700 1 -- start start 2024-04-26T21:54:04.467 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:03.787+0000 7f2d6d400700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2d681035b0 0x7f2d68077310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:04.467 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:03.787+0000 7f2d6d400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2d68103f60 0x7f2d68075720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:04.467 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:03.787+0000 7f2d6d400700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2d68108690 0x7f2d68075d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:04.467 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:03.787+0000 7f2d6d400700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f2d681133e0 con 0x7f2d681035b0 2024-04-26T21:54:04.468 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:03.787+0000 7f2d6d400700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f2d68113260 con 0x7f2d68108690 2024-04-26T21:54:04.468 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:03.787+0000 7f2d6d400700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f2d68113560 con 0x7f2d68103f60 2024-04-26T21:54:04.468 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:03.787+0000 7f2d67e00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2d681035b0 0x7f2d68077310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:04.468 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:03.787+0000 7f2d67e00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2d681035b0 0x7f2d68077310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:52088/0 (socket says 172.21.15.72:52088) 2024-04-26T21:54:04.468 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:03.787+0000 7f2d67e00700 1 -- 172.21.15.72:0/1030308429 learned_addr learned my addr 172.21.15.72:0/1030308429 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:54:04.468 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:03.788+0000 7f2d67400700 1 --2- 172.21.15.72:0/1030308429 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2d68103f60 0x7f2d68075720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:04.469 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:03.788+0000 7f2d6ca00700 1 --2- 172.21.15.72:0/1030308429 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2d68108690 0x7f2d68075d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:04.469 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:03.788+0000 7f2d67e00700 1 -- 172.21.15.72:0/1030308429 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2d68103f60 msgr2=0x7f2d68075720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:04.469 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:03.788+0000 7f2d67e00700 1 --2- 172.21.15.72:0/1030308429 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2d68103f60 0x7f2d68075720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:04.469 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:03.788+0000 7f2d67e00700 1 -- 172.21.15.72:0/1030308429 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2d68108690 msgr2=0x7f2d68075d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:04.469 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:03.788+0000 7f2d67e00700 1 --2- 172.21.15.72:0/1030308429 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2d68108690 0x7f2d68075d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:04.469 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:03.788+0000 7f2d67e00700 1 -- 172.21.15.72:0/1030308429 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d681ac220 con 0x7f2d681035b0 2024-04-26T21:54:04.470 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:03.788+0000 7f2d67400700 1 --2- 172.21.15.72:0/1030308429 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2d68103f60 0x7f2d68075720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:54:04.470 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:03.788+0000 7f2d6ca00700 1 --2- 172.21.15.72:0/1030308429 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2d68108690 0x7f2d68075d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:54:04.470 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:03.788+0000 7f2d67e00700 1 --2- 172.21.15.72:0/1030308429 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2d681035b0 0x7f2d68077310 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f2d5800d950 tx=0x7f2d5800dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:54:04.470 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:03.788+0000 7f2d64c00700 1 -- 172.21.15.72:0/1030308429 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d58016070 con 0x7f2d681035b0 2024-04-26T21:54:04.470 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:03.788+0000 7f2d64c00700 1 -- 172.21.15.72:0/1030308429 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2d58010460 con 0x7f2d681035b0 2024-04-26T21:54:04.470 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:03.788+0000 7f2d6d400700 1 -- 172.21.15.72:0/1030308429 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d681ac4b0 con 0x7f2d681035b0 2024-04-26T21:54:04.470 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:03.788+0000 7f2d64c00700 1 -- 172.21.15.72:0/1030308429 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d5800f620 con 0x7f2d681035b0 2024-04-26T21:54:04.471 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:03.789+0000 7f2d6d400700 1 -- 172.21.15.72:0/1030308429 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d681ac9c0 con 0x7f2d681035b0 2024-04-26T21:54:04.471 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:03.790+0000 7f2d6d400700 1 -- 172.21.15.72:0/1030308429 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d68104a90 con 0x7f2d681035b0 2024-04-26T21:54:04.471 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:03.792+0000 7f2d64c00700 1 -- 172.21.15.72:0/1030308429 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f2d58019070 con 0x7f2d681035b0 2024-04-26T21:54:04.471 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:03.793+0000 7f2d64c00700 1 --2- 172.21.15.72:0/1030308429 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f2d4c0a3cd0 0x7f2d4c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:04.471 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:03.793+0000 7f2d64c00700 1 -- 172.21.15.72:0/1030308429 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8269+0+0 (secure 0 0 0) 0x7f2d580d0210 con 0x7f2d681035b0 2024-04-26T21:54:04.471 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:03.793+0000 7f2d67400700 1 --2- 172.21.15.72:0/1030308429 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f2d4c0a3cd0 0x7f2d4c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:04.472 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:03.794+0000 7f2d67400700 1 --2- 172.21.15.72:0/1030308429 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f2d4c0a3cd0 0x7f2d4c0a6190 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f2d54005fd0 tx=0x7f2d54005dc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:54:04.472 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:03.796+0000 7f2d64c00700 1 -- 172.21.15.72:0/1030308429 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f2d5809ca10 con 0x7f2d681035b0 2024-04-26T21:54:04.472 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:04.028+0000 7f2d6d400700 1 -- 172.21.15.72:0/1030308429 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7f2d6802e490 con 0x7f2d681035b0 2024-04-26T21:54:04.472 INFO:teuthology.orchestra.run.smithi072.stdout:352187318302 2024-04-26T21:54:04.474 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:04.029+0000 7f2d64c00700 1 -- 172.21.15.72:0/1030308429 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f2d5809c310 con 0x7f2d681035b0 2024-04-26T21:54:04.474 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:04.032+0000 7f2d6d400700 1 -- 172.21.15.72:0/1030308429 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f2d4c0a3cd0 msgr2=0x7f2d4c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:04.474 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:04.032+0000 7f2d6d400700 1 --2- 172.21.15.72:0/1030308429 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f2d4c0a3cd0 0x7f2d4c0a6190 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f2d54005fd0 tx=0x7f2d54005dc0 comp rx=0 tx=0).stop 2024-04-26T21:54:04.474 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:04.032+0000 7f2d6d400700 1 -- 172.21.15.72:0/1030308429 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2d681035b0 msgr2=0x7f2d68077310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:04.474 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:04.032+0000 7f2d6d400700 1 --2- 172.21.15.72:0/1030308429 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2d681035b0 0x7f2d68077310 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f2d5800d950 tx=0x7f2d5800dd10 comp rx=0 tx=0).stop 2024-04-26T21:54:04.475 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:04.032+0000 7f2d6d400700 1 -- 172.21.15.72:0/1030308429 shutdown_connections 2024-04-26T21:54:04.475 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:04.032+0000 7f2d6d400700 1 --2- 172.21.15.72:0/1030308429 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f2d4c0a3cd0 0x7f2d4c0a6190 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:04.475 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:04.032+0000 7f2d6d400700 1 --2- 172.21.15.72:0/1030308429 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2d681035b0 0x7f2d68077310 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:04.475 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:04.032+0000 7f2d6d400700 1 --2- 172.21.15.72:0/1030308429 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2d68108690 0x7f2d68075d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:04.475 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:04.032+0000 7f2d6d400700 1 --2- 172.21.15.72:0/1030308429 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2d68103f60 0x7f2d68075720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:04.475 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:04.032+0000 7f2d6d400700 1 -- 172.21.15.72:0/1030308429 >> 172.21.15.72:0/1030308429 conn(0x7f2d680feea0 msgr2=0x7f2d68101160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:54:04.475 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:04.032+0000 7f2d6d400700 1 -- 172.21.15.72:0/1030308429 shutdown_connections 2024-04-26T21:54:04.476 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:04.032+0000 7f2d6d400700 1 -- 172.21.15.72:0/1030308429 wait complete. 2024-04-26T21:54:04.814 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:54:04 smithi076 ceph-mon[31823]: pgmap v406: 97 pgs: 97 active+clean; 633 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:54:04.815 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:54:04 smithi076 ceph-mon[31823]: from='client.? 172.21.15.72:0/1030308429' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-26T21:54:04.822 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:54:04 smithi072 ceph-mon[28138]: pgmap v406: 97 pgs: 97 active+clean; 633 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:54:04.822 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:54:04 smithi072 ceph-mon[28138]: from='client.? 172.21.15.72:0/1030308429' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-26T21:54:04.940 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:54:04 smithi164 ceph-mon[31916]: pgmap v406: 97 pgs: 97 active+clean; 633 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:54:04.941 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:54:04 smithi164 ceph-mon[31916]: from='client.? 172.21.15.72:0/1030308429' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-26T21:54:05.889 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:05.790+0000 7f3d27e00700 1 -- 172.21.15.72:0/3400329604 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3d28108750 msgr2=0x7f3d28110290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:05.889 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:05.790+0000 7f3d27e00700 1 --2- 172.21.15.72:0/3400329604 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3d28108750 0x7f3d28110290 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f3d2000b0a0 tx=0x7f3d2000b3b0 comp rx=0 tx=0).stop 2024-04-26T21:54:05.889 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:05.790+0000 7f3d27e00700 1 -- 172.21.15.72:0/3400329604 shutdown_connections 2024-04-26T21:54:05.889 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:05.790+0000 7f3d27e00700 1 --2- 172.21.15.72:0/3400329604 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3d28108750 0x7f3d28110290 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:05.889 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:05.790+0000 7f3d27e00700 1 --2- 172.21.15.72:0/3400329604 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3d2806bbf0 0x7f3d2806bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:05.889 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:05.790+0000 7f3d27e00700 1 --2- 172.21.15.72:0/3400329604 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3d2806c5a0 0x7f3d28108020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:05.889 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:05.790+0000 7f3d27e00700 1 -- 172.21.15.72:0/3400329604 >> 172.21.15.72:0/3400329604 conn(0x7f3d28079160 msgr2=0x7f3d28079570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:54:05.890 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:05.790+0000 7f3d27e00700 1 -- 172.21.15.72:0/3400329604 shutdown_connections 2024-04-26T21:54:05.890 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:05.790+0000 7f3d27e00700 1 -- 172.21.15.72:0/3400329604 wait complete. 2024-04-26T21:54:05.890 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:05.792+0000 7f3d27e00700 1 Processor -- start 2024-04-26T21:54:05.890 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:05.792+0000 7f3d27e00700 1 -- start start 2024-04-26T21:54:05.890 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:05.792+0000 7f3d27e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3d2806bbf0 0x7f3d281025b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:05.890 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:05.792+0000 7f3d27e00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3d2806c5a0 0x7f3d28102af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:05.890 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:05.792+0000 7f3d27e00700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3d28108750 0x7f3d28103030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:05.890 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:05.792+0000 7f3d27e00700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f3d28113410 con 0x7f3d2806c5a0 2024-04-26T21:54:05.890 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:05.792+0000 7f3d27e00700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f3d28113290 con 0x7f3d28108750 2024-04-26T21:54:05.890 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:05.792+0000 7f3d27e00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f3d28113590 con 0x7f3d2806bbf0 2024-04-26T21:54:05.890 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:05.792+0000 7f3d1fe00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3d2806c5a0 0x7f3d28102af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:05.891 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:05.792+0000 7f3d1fe00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3d2806c5a0 0x7f3d28102af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:52108/0 (socket says 172.21.15.72:52108) 2024-04-26T21:54:05.891 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:05.793+0000 7f3d1fe00700 1 -- 172.21.15.72:0/4101789527 learned_addr learned my addr 172.21.15.72:0/4101789527 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:54:05.891 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:05.793+0000 7f3d26a00700 1 --2- 172.21.15.72:0/4101789527 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3d2806bbf0 0x7f3d281025b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:05.891 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:05.793+0000 7f3d27400700 1 --2- 172.21.15.72:0/4101789527 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3d28108750 0x7f3d28103030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:05.891 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:05.793+0000 7f3d1fe00700 1 -- 172.21.15.72:0/4101789527 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3d2806bbf0 msgr2=0x7f3d281025b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:05.891 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:05.793+0000 7f3d1fe00700 1 --2- 172.21.15.72:0/4101789527 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3d2806bbf0 0x7f3d281025b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:05.891 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:05.793+0000 7f3d1fe00700 1 -- 172.21.15.72:0/4101789527 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3d28108750 msgr2=0x7f3d28103030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:05.891 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:05.793+0000 7f3d1fe00700 1 --2- 172.21.15.72:0/4101789527 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3d28108750 0x7f3d28103030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:05.891 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:05.793+0000 7f3d1fe00700 1 -- 172.21.15.72:0/4101789527 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d281a7d20 con 0x7f3d2806c5a0 2024-04-26T21:54:05.891 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:05.793+0000 7f3d1fe00700 1 --2- 172.21.15.72:0/4101789527 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3d2806c5a0 0x7f3d28102af0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f3d1000c9b0 tx=0x7f3d1000cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:54:05.891 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:05.794+0000 7f3d1f400700 1 -- 172.21.15.72:0/4101789527 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d10007aa0 con 0x7f3d2806c5a0 2024-04-26T21:54:05.891 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:05.794+0000 7f3d1f400700 1 -- 172.21.15.72:0/4101789527 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3d10007c40 con 0x7f3d2806c5a0 2024-04-26T21:54:05.892 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:05.794+0000 7f3d1f400700 1 -- 172.21.15.72:0/4101789527 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d10018660 con 0x7f3d2806c5a0 2024-04-26T21:54:05.892 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:05.794+0000 7f3d27e00700 1 -- 172.21.15.72:0/4101789527 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d281a7fb0 con 0x7f3d2806c5a0 2024-04-26T21:54:05.892 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:05.794+0000 7f3d27e00700 1 -- 172.21.15.72:0/4101789527 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d281a84b0 con 0x7f3d2806c5a0 2024-04-26T21:54:05.892 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:05.795+0000 7f3d27e00700 1 -- 172.21.15.72:0/4101789527 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d2806a110 con 0x7f3d2806c5a0 2024-04-26T21:54:05.892 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:05.797+0000 7f3d1f400700 1 -- 172.21.15.72:0/4101789527 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f3d1000f450 con 0x7f3d2806c5a0 2024-04-26T21:54:05.892 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:05.798+0000 7f3d1f400700 1 --2- 172.21.15.72:0/4101789527 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f3d0c0a3cd0 0x7f3d0c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:05.892 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:05.798+0000 7f3d1f400700 1 -- 172.21.15.72:0/4101789527 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8269+0+0 (secure 0 0 0) 0x7f3d100d0530 con 0x7f3d2806c5a0 2024-04-26T21:54:05.892 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:05.799+0000 7f3d26a00700 1 --2- 172.21.15.72:0/4101789527 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f3d0c0a3cd0 0x7f3d0c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:05.892 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:05.799+0000 7f3d26a00700 1 --2- 172.21.15.72:0/4101789527 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f3d0c0a3cd0 0x7f3d0c0a6190 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f3d18005950 tx=0x7f3d18005eb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:54:05.892 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:05.803+0000 7f3d1f400700 1 -- 172.21.15.72:0/4101789527 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f3d10014030 con 0x7f3d2806c5a0 2024-04-26T21:54:06.034 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:06.034+0000 7f3d27e00700 1 -- 172.21.15.72:0/4101789527 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f3d2802e650 con 0x7f3d2806c5a0 2024-04-26T21:54:06.035 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:06.035+0000 7f3d1f400700 1 -- 172.21.15.72:0/4101789527 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f3d1009cd30 con 0x7f3d2806c5a0 2024-04-26T21:54:06.035 INFO:teuthology.orchestra.run.smithi072.stdout:77309411423 2024-04-26T21:54:06.038 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:06.038+0000 7f3d27e00700 1 -- 172.21.15.72:0/4101789527 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f3d0c0a3cd0 msgr2=0x7f3d0c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:06.038 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:06.038+0000 7f3d27e00700 1 --2- 172.21.15.72:0/4101789527 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f3d0c0a3cd0 0x7f3d0c0a6190 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f3d18005950 tx=0x7f3d18005eb0 comp rx=0 tx=0).stop 2024-04-26T21:54:06.038 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:06.038+0000 7f3d27e00700 1 -- 172.21.15.72:0/4101789527 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3d2806c5a0 msgr2=0x7f3d28102af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:06.038 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:06.038+0000 7f3d27e00700 1 --2- 172.21.15.72:0/4101789527 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3d2806c5a0 0x7f3d28102af0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f3d1000c9b0 tx=0x7f3d1000cd70 comp rx=0 tx=0).stop 2024-04-26T21:54:06.039 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:06.038+0000 7f3d27e00700 1 -- 172.21.15.72:0/4101789527 shutdown_connections 2024-04-26T21:54:06.039 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:06.038+0000 7f3d27e00700 1 --2- 172.21.15.72:0/4101789527 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f3d0c0a3cd0 0x7f3d0c0a6190 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:06.039 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:06.038+0000 7f3d27e00700 1 --2- 172.21.15.72:0/4101789527 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3d28108750 0x7f3d28103030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:06.039 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:06.038+0000 7f3d27e00700 1 --2- 172.21.15.72:0/4101789527 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3d2806bbf0 0x7f3d281025b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:06.039 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:06.038+0000 7f3d27e00700 1 --2- 172.21.15.72:0/4101789527 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3d2806c5a0 0x7f3d28102af0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:06.039 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:06.038+0000 7f3d27e00700 1 -- 172.21.15.72:0/4101789527 >> 172.21.15.72:0/4101789527 conn(0x7f3d28079160 msgr2=0x7f3d28100530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:54:06.039 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:06.038+0000 7f3d27e00700 1 -- 172.21.15.72:0/4101789527 shutdown_connections 2024-04-26T21:54:06.039 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:06.038+0000 7f3d27e00700 1 -- 172.21.15.72:0/4101789527 wait complete. 2024-04-26T21:54:06.405 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:00.324+0000 7ff6cb000700 1 -- 172.21.15.72:0/1646516201 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff6c4105630 msgr2=0x7ff6c4105ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:06.405 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:00.324+0000 7ff6cb000700 1 --2- 172.21.15.72:0/1646516201 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff6c4105630 0x7ff6c4105ab0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7ff6b0009990 tx=0x7ff6b0009ca0 comp rx=0 tx=0).stop 2024-04-26T21:54:06.405 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:00.324+0000 7ff6cb000700 1 -- 172.21.15.72:0/1646516201 shutdown_connections 2024-04-26T21:54:06.405 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:00.324+0000 7ff6cb000700 1 --2- 172.21.15.72:0/1646516201 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff6c4105ff0 0x7ff6c4112500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:06.405 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:00.324+0000 7ff6cb000700 1 --2- 172.21.15.72:0/1646516201 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff6c410b630 0x7ff6c410ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:06.405 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:00.324+0000 7ff6cb000700 1 --2- 172.21.15.72:0/1646516201 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff6c4105630 0x7ff6c4105ab0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:06.406 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:00.324+0000 7ff6cb000700 1 -- 172.21.15.72:0/1646516201 >> 172.21.15.72:0/1646516201 conn(0x7ff6c4101120 msgr2=0x7ff6c4103540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:54:06.406 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:00.325+0000 7ff6cb000700 1 -- 172.21.15.72:0/1646516201 shutdown_connections 2024-04-26T21:54:06.406 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:00.325+0000 7ff6cb000700 1 -- 172.21.15.72:0/1646516201 wait complete. 2024-04-26T21:54:06.406 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:00.326+0000 7ff6cb000700 1 Processor -- start 2024-04-26T21:54:06.406 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:00.326+0000 7ff6cb000700 1 -- start start 2024-04-26T21:54:06.406 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:00.326+0000 7ff6cb000700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff6c4105630 0x7ff6c41a1890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:06.406 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:00.327+0000 7ff6cb000700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff6c4105ff0 0x7ff6c41a1dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:06.406 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:00.327+0000 7ff6cb000700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff6c410b630 0x7ff6c419ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:06.406 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:00.327+0000 7ff6cb000700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7ff6c4115710 con 0x7ff6c4105630 2024-04-26T21:54:06.406 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:00.327+0000 7ff6cb000700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7ff6c4115590 con 0x7ff6c410b630 2024-04-26T21:54:06.406 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:00.327+0000 7ff6cb000700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ff6c4115890 con 0x7ff6c4105ff0 2024-04-26T21:54:06.406 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:00.327+0000 7ff6c3e00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff6c4105630 0x7ff6c41a1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:06.406 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:00.327+0000 7ff6c3e00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff6c4105630 0x7ff6c41a1890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:34088/0 (socket says 172.21.15.72:34088) 2024-04-26T21:54:06.406 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:00.327+0000 7ff6c3e00700 1 -- 172.21.15.72:0/3800275929 learned_addr learned my addr 172.21.15.72:0/3800275929 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:54:06.407 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:00.327+0000 7ff6c9000700 1 --2- 172.21.15.72:0/3800275929 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff6c410b630 0x7ff6c419ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:06.407 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:00.327+0000 7ff6c3400700 1 --2- 172.21.15.72:0/3800275929 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff6c4105ff0 0x7ff6c41a1dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:06.407 INFO:teuthology.orchestra.run.smithi072.stdout:317827579941 2024-04-26T21:54:06.407 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:00.327+0000 7ff6c3e00700 1 -- 172.21.15.72:0/3800275929 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff6c4105ff0 msgr2=0x7ff6c41a1dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:06.407 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:00.327+0000 7ff6c3e00700 1 --2- 172.21.15.72:0/3800275929 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff6c4105ff0 0x7ff6c41a1dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:06.407 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:00.327+0000 7ff6c3e00700 1 -- 172.21.15.72:0/3800275929 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff6c410b630 msgr2=0x7ff6c419ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:06.407 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:00.327+0000 7ff6c3e00700 1 --2- 172.21.15.72:0/3800275929 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff6c410b630 0x7ff6c419ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:06.407 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:00.327+0000 7ff6c3e00700 1 -- 172.21.15.72:0/3800275929 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6c419c1b0 con 0x7ff6c4105630 2024-04-26T21:54:06.408 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:00.327+0000 7ff6c3400700 1 --2- 172.21.15.72:0/3800275929 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff6c4105ff0 0x7ff6c41a1dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:54:06.408 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:00.328+0000 7ff6c3e00700 1 --2- 172.21.15.72:0/3800275929 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff6c4105630 0x7ff6c41a1890 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7ff6b400c9b0 tx=0x7ff6b400cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:54:06.408 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:00.328+0000 7ff6c0c00700 1 -- 172.21.15.72:0/3800275929 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6b4007b00 con 0x7ff6c4105630 2024-04-26T21:54:06.408 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:00.328+0000 7ff6c0c00700 1 -- 172.21.15.72:0/3800275929 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff6b4007ca0 con 0x7ff6c4105630 2024-04-26T21:54:06.408 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:00.328+0000 7ff6cb000700 1 -- 172.21.15.72:0/3800275929 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6c419c4a0 con 0x7ff6c4105630 2024-04-26T21:54:06.408 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:00.328+0000 7ff6c0c00700 1 -- 172.21.15.72:0/3800275929 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6b4018660 con 0x7ff6c4105630 2024-04-26T21:54:06.408 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:00.328+0000 7ff6cb000700 1 -- 172.21.15.72:0/3800275929 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6c41a8760 con 0x7ff6c4105630 2024-04-26T21:54:06.408 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:00.329+0000 7ff6cb000700 1 -- 172.21.15.72:0/3800275929 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6c406a110 con 0x7ff6c4105630 2024-04-26T21:54:06.408 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:00.332+0000 7ff6c0c00700 1 -- 172.21.15.72:0/3800275929 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7ff6b400f450 con 0x7ff6c4105630 2024-04-26T21:54:06.408 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:00.333+0000 7ff6c0c00700 1 --2- 172.21.15.72:0/3800275929 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7ff6a80a3cd0 0x7ff6a80a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:06.408 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:00.333+0000 7ff6c0c00700 1 -- 172.21.15.72:0/3800275929 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8269+0+0 (secure 0 0 0) 0x7ff6b40d0900 con 0x7ff6c4105630 2024-04-26T21:54:06.408 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:00.333+0000 7ff6c3400700 1 --2- 172.21.15.72:0/3800275929 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7ff6a80a3cd0 0x7ff6a80a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:06.408 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:00.334+0000 7ff6c3400700 1 --2- 172.21.15.72:0/3800275929 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7ff6a80a3cd0 0x7ff6a80a6190 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7ff6b000b5c0 tx=0x7ff6b0005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:54:06.409 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:00.335+0000 7ff6c0c00700 1 -- 172.21.15.72:0/3800275929 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7ff6b4014030 con 0x7ff6c4105630 2024-04-26T21:54:06.409 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:00.583+0000 7ff6cb000700 1 -- 172.21.15.72:0/3800275929 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7ff6c419cf80 con 0x7ff6c4105630 2024-04-26T21:54:06.409 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:00.584+0000 7ff6c0c00700 1 -- 172.21.15.72:0/3800275929 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7ff6b409d190 con 0x7ff6c4105630 2024-04-26T21:54:06.409 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:00.587+0000 7ff6cb000700 1 -- 172.21.15.72:0/3800275929 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7ff6a80a3cd0 msgr2=0x7ff6a80a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:06.409 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:00.587+0000 7ff6cb000700 1 --2- 172.21.15.72:0/3800275929 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7ff6a80a3cd0 0x7ff6a80a6190 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7ff6b000b5c0 tx=0x7ff6b0005fb0 comp rx=0 tx=0).stop 2024-04-26T21:54:06.409 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:00.587+0000 7ff6cb000700 1 -- 172.21.15.72:0/3800275929 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff6c4105630 msgr2=0x7ff6c41a1890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:06.409 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:00.587+0000 7ff6cb000700 1 --2- 172.21.15.72:0/3800275929 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff6c4105630 0x7ff6c41a1890 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7ff6b400c9b0 tx=0x7ff6b400cd70 comp rx=0 tx=0).stop 2024-04-26T21:54:06.409 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:00.587+0000 7ff6cb000700 1 -- 172.21.15.72:0/3800275929 shutdown_connections 2024-04-26T21:54:06.409 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:00.587+0000 7ff6cb000700 1 --2- 172.21.15.72:0/3800275929 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7ff6a80a3cd0 0x7ff6a80a6190 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:06.409 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:00.587+0000 7ff6cb000700 1 --2- 172.21.15.72:0/3800275929 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff6c4105630 0x7ff6c41a1890 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:06.409 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:00.587+0000 7ff6cb000700 1 --2- 172.21.15.72:0/3800275929 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff6c410b630 0x7ff6c419ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:06.409 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:00.587+0000 7ff6cb000700 1 --2- 172.21.15.72:0/3800275929 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff6c4105ff0 0x7ff6c41a1dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:06.409 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:00.587+0000 7ff6cb000700 1 -- 172.21.15.72:0/3800275929 >> 172.21.15.72:0/3800275929 conn(0x7ff6c4101120 msgr2=0x7ff6c41017c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:54:06.409 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:00.587+0000 7ff6cb000700 1 -- 172.21.15.72:0/3800275929 shutdown_connections 2024-04-26T21:54:06.410 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:00.587+0000 7ff6cb000700 1 -- 172.21.15.72:0/3800275929 wait complete. 2024-04-26T21:54:07.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:54:06 smithi072 ceph-mon[28138]: pgmap v407: 97 pgs: 97 active+clean; 633 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-26T21:54:07.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:54:06 smithi072 ceph-mon[28138]: from='client.? 172.21.15.72:0/4101789527' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-26T21:54:07.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:54:06 smithi164 ceph-mon[31916]: pgmap v407: 97 pgs: 97 active+clean; 633 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-26T21:54:07.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:54:06 smithi164 ceph-mon[31916]: from='client.? 172.21.15.72:0/4101789527' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-26T21:54:07.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:54:06 smithi076 ceph-mon[31823]: pgmap v407: 97 pgs: 97 active+clean; 633 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-26T21:54:07.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:54:06 smithi076 ceph-mon[31823]: from='client.? 172.21.15.72:0/4101789527' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-26T21:54:07.613 INFO:tasks.cephadm.ceph_manager.ceph:need seq 261993005103 got 261993005109 for osd.8 2024-04-26T21:54:07.613 DEBUG:teuthology.parallel:result is None 2024-04-26T21:54:07.732 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:07.430+0000 7fcf20a00700 1 -- 172.21.15.72:0/575904665 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcf1c06bad0 msgr2=0x7fcf1c1102e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:07.732 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:07.430+0000 7fcf20a00700 1 --2- 172.21.15.72:0/575904665 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcf1c06bad0 0x7fcf1c1102e0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fcf1400b0a0 tx=0x7fcf1400b3b0 comp rx=0 tx=0).stop 2024-04-26T21:54:07.732 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:07.430+0000 7fcf20a00700 1 -- 172.21.15.72:0/575904665 shutdown_connections 2024-04-26T21:54:07.732 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:07.430+0000 7fcf20a00700 1 --2- 172.21.15.72:0/575904665 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcf1c06bad0 0x7fcf1c1102e0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:07.732 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:07.430+0000 7fcf20a00700 1 --2- 172.21.15.72:0/575904665 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcf1c1094a0 0x7fcf1c109880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:07.732 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:07.430+0000 7fcf20a00700 1 --2- 172.21.15.72:0/575904665 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcf1c06b130 0x7fcf1c06b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:07.732 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:07.430+0000 7fcf20a00700 1 -- 172.21.15.72:0/575904665 >> 172.21.15.72:0/575904665 conn(0x7fcf1c0792d0 msgr2=0x7fcf1c0796e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:54:07.732 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:07.430+0000 7fcf20a00700 1 -- 172.21.15.72:0/575904665 shutdown_connections 2024-04-26T21:54:07.732 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:07.430+0000 7fcf20a00700 1 -- 172.21.15.72:0/575904665 wait complete. 2024-04-26T21:54:07.732 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:07.431+0000 7fcf20a00700 1 Processor -- start 2024-04-26T21:54:07.733 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:07.432+0000 7fcf20a00700 1 -- start start 2024-04-26T21:54:07.733 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:07.432+0000 7fcf20a00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcf1c06b130 0x7fcf1c1a17d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:07.733 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:07.432+0000 7fcf20a00700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcf1c06bad0 0x7fcf1c1a1d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:07.733 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:07.432+0000 7fcf20a00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcf1c1094a0 0x7fcf1c19b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:07.733 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:07.432+0000 7fcf20a00700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fcf1c1134d0 con 0x7fcf1c06b130 2024-04-26T21:54:07.733 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:07.432+0000 7fcf20a00700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fcf1c113350 con 0x7fcf1c06bad0 2024-04-26T21:54:07.733 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:07.432+0000 7fcf20a00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fcf1c113650 con 0x7fcf1c1094a0 2024-04-26T21:54:07.733 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:07.432+0000 7fcf19e00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcf1c06b130 0x7fcf1c1a17d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:07.733 INFO:teuthology.orchestra.run.smithi072.stdout:167503724616 2024-04-26T21:54:07.734 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:07.432+0000 7fcf19e00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcf1c06b130 0x7fcf1c1a17d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:52132/0 (socket says 172.21.15.72:52132) 2024-04-26T21:54:07.734 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:07.432+0000 7fcf19e00700 1 -- 172.21.15.72:0/3091811741 learned_addr learned my addr 172.21.15.72:0/3091811741 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:54:07.734 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:07.433+0000 7fcf19400700 1 --2- 172.21.15.72:0/3091811741 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcf1c06bad0 0x7fcf1c1a1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:07.734 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:07.433+0000 7fcf1a800700 1 --2- 172.21.15.72:0/3091811741 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcf1c1094a0 0x7fcf1c19b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:07.734 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:07.433+0000 7fcf19e00700 1 -- 172.21.15.72:0/3091811741 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcf1c1094a0 msgr2=0x7fcf1c19b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:07.734 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:07.433+0000 7fcf19e00700 1 --2- 172.21.15.72:0/3091811741 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcf1c1094a0 0x7fcf1c19b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:07.734 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:07.433+0000 7fcf19e00700 1 -- 172.21.15.72:0/3091811741 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcf1c06bad0 msgr2=0x7fcf1c1a1d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:07.735 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:07.433+0000 7fcf19e00700 1 --2- 172.21.15.72:0/3091811741 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcf1c06bad0 0x7fcf1c1a1d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:07.735 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:07.433+0000 7fcf19e00700 1 -- 172.21.15.72:0/3091811741 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf1c19c0f0 con 0x7fcf1c06b130 2024-04-26T21:54:07.735 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:07.433+0000 7fcf19e00700 1 --2- 172.21.15.72:0/3091811741 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcf1c06b130 0x7fcf1c1a17d0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fcf0c00c9b0 tx=0x7fcf0c00cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:54:07.735 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:07.433+0000 7fcf12a00700 1 -- 172.21.15.72:0/3091811741 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf0c007b00 con 0x7fcf1c06b130 2024-04-26T21:54:07.735 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:07.433+0000 7fcf20a00700 1 -- 172.21.15.72:0/3091811741 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf1c19c3e0 con 0x7fcf1c06b130 2024-04-26T21:54:07.735 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:07.433+0000 7fcf12a00700 1 -- 172.21.15.72:0/3091811741 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcf0c007ca0 con 0x7fcf1c06b130 2024-04-26T21:54:07.735 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:07.434+0000 7fcf12a00700 1 -- 172.21.15.72:0/3091811741 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf0c018660 con 0x7fcf1c06b130 2024-04-26T21:54:07.735 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:07.434+0000 7fcf20a00700 1 -- 172.21.15.72:0/3091811741 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf1c1a86a0 con 0x7fcf1c06b130 2024-04-26T21:54:07.735 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:07.436+0000 7fcf20a00700 1 -- 172.21.15.72:0/3091811741 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf1c06a110 con 0x7fcf1c06b130 2024-04-26T21:54:07.735 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:07.437+0000 7fcf12a00700 1 -- 172.21.15.72:0/3091811741 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fcf0c00f450 con 0x7fcf1c06b130 2024-04-26T21:54:07.735 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:07.438+0000 7fcf12a00700 1 --2- 172.21.15.72:0/3091811741 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fcf000a3cd0 0x7fcf000a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:07.736 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:07.438+0000 7fcf12a00700 1 -- 172.21.15.72:0/3091811741 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8269+0+0 (secure 0 0 0) 0x7fcf0c0d0660 con 0x7fcf1c06b130 2024-04-26T21:54:07.736 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:07.439+0000 7fcf19400700 1 --2- 172.21.15.72:0/3091811741 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fcf000a3cd0 0x7fcf000a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:07.736 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:07.439+0000 7fcf19400700 1 --2- 172.21.15.72:0/3091811741 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fcf000a3cd0 0x7fcf000a6190 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fcf08005950 tx=0x7fcf080058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:54:07.736 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:07.442+0000 7fcf12a00700 1 -- 172.21.15.72:0/3091811741 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fcf0c014030 con 0x7fcf1c06b130 2024-04-26T21:54:07.736 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:07.674+0000 7fcf20a00700 1 -- 172.21.15.72:0/3091811741 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7fcf1c1a8930 con 0x7fcf1c06b130 2024-04-26T21:54:07.736 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:07.675+0000 7fcf12a00700 1 -- 172.21.15.72:0/3091811741 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fcf0c09ce60 con 0x7fcf1c06b130 2024-04-26T21:54:07.736 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:07.678+0000 7fcf20a00700 1 -- 172.21.15.72:0/3091811741 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fcf000a3cd0 msgr2=0x7fcf000a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:07.736 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:07.678+0000 7fcf20a00700 1 --2- 172.21.15.72:0/3091811741 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fcf000a3cd0 0x7fcf000a6190 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fcf08005950 tx=0x7fcf080058e0 comp rx=0 tx=0).stop 2024-04-26T21:54:07.736 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:07.678+0000 7fcf20a00700 1 -- 172.21.15.72:0/3091811741 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcf1c06b130 msgr2=0x7fcf1c1a17d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:07.736 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:07.678+0000 7fcf20a00700 1 --2- 172.21.15.72:0/3091811741 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcf1c06b130 0x7fcf1c1a17d0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fcf0c00c9b0 tx=0x7fcf0c00cd70 comp rx=0 tx=0).stop 2024-04-26T21:54:07.736 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:07.678+0000 7fcf20a00700 1 -- 172.21.15.72:0/3091811741 shutdown_connections 2024-04-26T21:54:07.736 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:07.678+0000 7fcf20a00700 1 --2- 172.21.15.72:0/3091811741 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fcf000a3cd0 0x7fcf000a6190 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:07.736 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:07.678+0000 7fcf20a00700 1 --2- 172.21.15.72:0/3091811741 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcf1c06b130 0x7fcf1c1a17d0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:07.737 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:07.678+0000 7fcf20a00700 1 --2- 172.21.15.72:0/3091811741 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcf1c1094a0 0x7fcf1c19b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:07.737 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:07.678+0000 7fcf20a00700 1 --2- 172.21.15.72:0/3091811741 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcf1c06bad0 0x7fcf1c1a1d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:07.737 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:07.678+0000 7fcf20a00700 1 -- 172.21.15.72:0/3091811741 >> 172.21.15.72:0/3091811741 conn(0x7fcf1c0792d0 msgr2=0x7fcf1c100530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:54:07.737 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:07.678+0000 7fcf20a00700 1 -- 172.21.15.72:0/3091811741 shutdown_connections 2024-04-26T21:54:07.737 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:07.678+0000 7fcf20a00700 1 -- 172.21.15.72:0/3091811741 wait complete. 2024-04-26T21:54:08.022 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:54:07 smithi072 ceph-mon[28138]: from='client.? 172.21.15.72:0/3091811741' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-26T21:54:08.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:54:07 smithi164 ceph-mon[31916]: from='client.? 172.21.15.72:0/3091811741' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-26T21:54:08.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:54:07 smithi076 ceph-mon[31823]: from='client.? 172.21.15.72:0/3091811741' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-26T21:54:08.973 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:54:08 smithi072 ceph-mon[28138]: pgmap v408: 97 pgs: 97 active+clean; 633 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:54:09.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:54:08 smithi164 ceph-mon[31916]: pgmap v408: 97 pgs: 97 active+clean; 633 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:54:09.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:54:08 smithi076 ceph-mon[31823]: pgmap v408: 97 pgs: 97 active+clean; 633 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:54:09.502 INFO:tasks.cephadm.ceph_manager.ceph:need seq 352187318297 got 352187318302 for osd.11 2024-04-26T21:54:09.502 DEBUG:teuthology.parallel:result is None 2024-04-26T21:54:09.764 INFO:tasks.cephadm.ceph_manager.ceph:need seq 201863462974 got 201863462978 for osd.6 2024-04-26T21:54:09.764 DEBUG:teuthology.parallel:result is None 2024-04-26T21:54:10.759 INFO:tasks.cephadm.ceph_manager.ceph:need seq 317827579937 got 317827579941 for osd.10 2024-04-26T21:54:10.759 DEBUG:teuthology.parallel:result is None 2024-04-26T21:54:10.985 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:10.491+0000 7efe17200700 1 -- 172.21.15.72:0/1021212999 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efe18105f10 msgr2=0x7efe181123d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:10.985 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:10.491+0000 7efe17200700 1 --2- 172.21.15.72:0/1021212999 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efe18105f10 0x7efe181123d0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7efe1000b0d0 tx=0x7efe1000b3e0 comp rx=0 tx=0).stop 2024-04-26T21:54:10.985 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:10.492+0000 7efe17200700 1 -- 172.21.15.72:0/1021212999 shutdown_connections 2024-04-26T21:54:10.985 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:10.492+0000 7efe17200700 1 --2- 172.21.15.72:0/1021212999 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efe18105f10 0x7efe181123d0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:10.985 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:10.492+0000 7efe17200700 1 --2- 172.21.15.72:0/1021212999 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efe1810b600 0x7efe1810b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:10.986 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:10.492+0000 7efe17200700 1 --2- 172.21.15.72:0/1021212999 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efe18105550 0x7efe181059d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:10.986 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:10.492+0000 7efe17200700 1 -- 172.21.15.72:0/1021212999 >> 172.21.15.72:0/1021212999 conn(0x7efe18101000 msgr2=0x7efe18103420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:54:10.986 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:10.492+0000 7efe17200700 1 -- 172.21.15.72:0/1021212999 shutdown_connections 2024-04-26T21:54:10.986 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:10.492+0000 7efe17200700 1 -- 172.21.15.72:0/1021212999 wait complete. 2024-04-26T21:54:10.986 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:10.493+0000 7efe17200700 1 Processor -- start 2024-04-26T21:54:10.986 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:10.493+0000 7efe17200700 1 -- start start 2024-04-26T21:54:10.986 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:10.493+0000 7efe17200700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efe18105550 0x7efe181a16b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:10.986 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:10.493+0000 7efe17200700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efe18105f10 0x7efe181a1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:10.986 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:10.494+0000 7efe17200700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efe1810b600 0x7efe1819b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:10.986 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:10.494+0000 7efe17200700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7efe18115660 con 0x7efe1810b600 2024-04-26T21:54:10.986 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:10.494+0000 7efe17200700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7efe181154e0 con 0x7efe18105550 2024-04-26T21:54:10.986 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:10.494+0000 7efe17200700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7efe181157e0 con 0x7efe18105f10 2024-04-26T21:54:10.986 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:10.494+0000 7efe16800700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efe1810b600 0x7efe1819b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:10.987 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:10.494+0000 7efe16800700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efe1810b600 0x7efe1819b8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:52156/0 (socket says 172.21.15.72:52156) 2024-04-26T21:54:10.987 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:10.494+0000 7efe16800700 1 -- 172.21.15.72:0/3495975605 learned_addr learned my addr 172.21.15.72:0/3495975605 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:54:10.987 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:10.494+0000 7efe15e00700 1 --2- 172.21.15.72:0/3495975605 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efe18105550 0x7efe181a16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:10.987 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:10.494+0000 7efe15400700 1 --2- 172.21.15.72:0/3495975605 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efe18105f10 0x7efe181a1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:10.987 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:10.494+0000 7efe16800700 1 -- 172.21.15.72:0/3495975605 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efe18105f10 msgr2=0x7efe181a1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:10.987 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:10.494+0000 7efe16800700 1 --2- 172.21.15.72:0/3495975605 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efe18105f10 0x7efe181a1bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:10.987 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:10.494+0000 7efe16800700 1 -- 172.21.15.72:0/3495975605 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efe18105550 msgr2=0x7efe181a16b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:10.987 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:10.494+0000 7efe16800700 1 --2- 172.21.15.72:0/3495975605 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efe18105550 0x7efe181a16b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:10.987 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:10.494+0000 7efe16800700 1 -- 172.21.15.72:0/3495975605 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe1819c020 con 0x7efe1810b600 2024-04-26T21:54:10.987 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:10.494+0000 7efe15e00700 1 --2- 172.21.15.72:0/3495975605 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efe18105550 0x7efe181a16b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:54:10.987 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:10.495+0000 7efe16800700 1 --2- 172.21.15.72:0/3495975605 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efe1810b600 0x7efe1819b8c0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7efe10007d10 tx=0x7efe10007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:54:10.987 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:10.495+0000 7efe0ea00700 1 -- 172.21.15.72:0/3495975605 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe1001f070 con 0x7efe1810b600 2024-04-26T21:54:10.987 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:10.495+0000 7efe0ea00700 1 -- 172.21.15.72:0/3495975605 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efe10003e10 con 0x7efe1810b600 2024-04-26T21:54:10.987 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:10.495+0000 7efe17200700 1 -- 172.21.15.72:0/3495975605 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe1819c2b0 con 0x7efe1810b600 2024-04-26T21:54:10.988 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:10.495+0000 7efe0ea00700 1 -- 172.21.15.72:0/3495975605 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe1001b820 con 0x7efe1810b600 2024-04-26T21:54:10.988 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:10.495+0000 7efe17200700 1 -- 172.21.15.72:0/3495975605 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe181a8580 con 0x7efe1810b600 2024-04-26T21:54:10.988 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:10.496+0000 7efe17200700 1 -- 172.21.15.72:0/3495975605 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe1806a110 con 0x7efe1810b600 2024-04-26T21:54:10.988 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:10.499+0000 7efe0ea00700 1 -- 172.21.15.72:0/3495975605 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7efe10022070 con 0x7efe1810b600 2024-04-26T21:54:10.988 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:10.500+0000 7efe0ea00700 1 --2- 172.21.15.72:0/3495975605 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7efdfc0a3d50 0x7efdfc0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:10.988 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:10.500+0000 7efe0ea00700 1 -- 172.21.15.72:0/3495975605 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8269+0+0 (secure 0 0 0) 0x7efe1000f020 con 0x7efe1810b600 2024-04-26T21:54:10.988 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:10.500+0000 7efe15e00700 1 --2- 172.21.15.72:0/3495975605 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7efdfc0a3d50 0x7efdfc0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:10.988 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:10.501+0000 7efe15e00700 1 --2- 172.21.15.72:0/3495975605 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7efdfc0a3d50 0x7efdfc0a6210 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7efe08005950 tx=0x7efe080058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:54:10.988 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:10.502+0000 7efe0ea00700 1 -- 172.21.15.72:0/3495975605 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7efe10029020 con 0x7efe1810b600 2024-04-26T21:54:10.988 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:10.740+0000 7efe17200700 1 -- 172.21.15.72:0/3495975605 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7efe1802e490 con 0x7efe1810b600 2024-04-26T21:54:10.988 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:10.741+0000 7efe0ea00700 1 -- 172.21.15.72:0/3495975605 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7efe1009e720 con 0x7efe1810b600 2024-04-26T21:54:10.989 INFO:teuthology.orchestra.run.smithi072.stdout:296352743471 2024-04-26T21:54:10.989 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:10.744+0000 7efe17200700 1 -- 172.21.15.72:0/3495975605 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7efdfc0a3d50 msgr2=0x7efdfc0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:10.989 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:10.744+0000 7efe17200700 1 --2- 172.21.15.72:0/3495975605 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7efdfc0a3d50 0x7efdfc0a6210 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7efe08005950 tx=0x7efe080058e0 comp rx=0 tx=0).stop 2024-04-26T21:54:10.989 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:10.744+0000 7efe17200700 1 -- 172.21.15.72:0/3495975605 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efe1810b600 msgr2=0x7efe1819b8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:10.990 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:10.744+0000 7efe17200700 1 --2- 172.21.15.72:0/3495975605 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efe1810b600 0x7efe1819b8c0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7efe10007d10 tx=0x7efe10007df0 comp rx=0 tx=0).stop 2024-04-26T21:54:10.990 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:10.744+0000 7efe17200700 1 -- 172.21.15.72:0/3495975605 shutdown_connections 2024-04-26T21:54:10.990 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:10.744+0000 7efe17200700 1 --2- 172.21.15.72:0/3495975605 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7efdfc0a3d50 0x7efdfc0a6210 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:10.990 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:10.744+0000 7efe17200700 1 --2- 172.21.15.72:0/3495975605 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efe1810b600 0x7efe1819b8c0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:10.990 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:10.744+0000 7efe17200700 1 --2- 172.21.15.72:0/3495975605 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efe18105550 0x7efe181a16b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:10.990 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:10.744+0000 7efe17200700 1 --2- 172.21.15.72:0/3495975605 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efe18105f10 0x7efe181a1bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:10.990 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:10.744+0000 7efe17200700 1 -- 172.21.15.72:0/3495975605 >> 172.21.15.72:0/3495975605 conn(0x7efe18101000 msgr2=0x7efe18102830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:54:10.990 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:10.745+0000 7efe17200700 1 -- 172.21.15.72:0/3495975605 shutdown_connections 2024-04-26T21:54:10.990 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:10.745+0000 7efe17200700 1 -- 172.21.15.72:0/3495975605 wait complete. 2024-04-26T21:54:11.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:54:10 smithi072 ceph-mon[28138]: pgmap v409: 97 pgs: 97 active+clean; 633 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-26T21:54:11.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:54:10 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]: dispatch 2024-04-26T21:54:11.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:54:10 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-04-26T21:54:11.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:54:10 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]: dispatch 2024-04-26T21:54:11.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:54:10 smithi072 ceph-mon[28138]: from='client.? 172.21.15.72:0/3495975605' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-26T21:54:11.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:54:10 smithi164 ceph-mon[31916]: pgmap v409: 97 pgs: 97 active+clean; 633 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-26T21:54:11.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:54:10 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]: dispatch 2024-04-26T21:54:11.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:54:10 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-04-26T21:54:11.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:54:10 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]: dispatch 2024-04-26T21:54:11.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:54:10 smithi164 ceph-mon[31916]: from='client.? 172.21.15.72:0/3495975605' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-26T21:54:11.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:54:10 smithi076 ceph-mon[31823]: pgmap v409: 97 pgs: 97 active+clean; 633 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-26T21:54:11.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:54:10 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]: dispatch 2024-04-26T21:54:11.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:54:10 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-04-26T21:54:11.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:54:10 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]: dispatch 2024-04-26T21:54:11.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:54:10 smithi076 ceph-mon[31823]: from='client.? 172.21.15.72:0/3495975605' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-26T21:54:11.595 INFO:tasks.cephadm.ceph_manager.ceph:need seq 167503724611 got 167503724616 for osd.5 2024-04-26T21:54:11.595 DEBUG:teuthology.parallel:result is None 2024-04-26T21:54:11.854 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/a1780950-0415-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:54:12.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:54:11 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]': finished 2024-04-26T21:54:12.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:54:11 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-04-26T21:54:12.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:54:11 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]': finished 2024-04-26T21:54:12.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:54:11 smithi164 ceph-mon[31916]: osdmap e90: 12 total, 12 up, 12 in 2024-04-26T21:54:12.294 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:54:11 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]': finished 2024-04-26T21:54:12.295 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:54:11 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-04-26T21:54:12.295 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:54:11 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]': finished 2024-04-26T21:54:12.295 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:54:11 smithi072 ceph-mon[28138]: osdmap e90: 12 total, 12 up, 12 in 2024-04-26T21:54:12.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:54:11 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]': finished 2024-04-26T21:54:12.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:54:11 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-04-26T21:54:12.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:54:11 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]': finished 2024-04-26T21:54:12.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:54:11 smithi076 ceph-mon[31823]: osdmap e90: 12 total, 12 up, 12 in 2024-04-26T21:54:12.364 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411418 got 77309411423 for osd.2 2024-04-26T21:54:12.364 DEBUG:teuthology.parallel:result is None 2024-04-26T21:54:13.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:54:12 smithi164 ceph-mon[31916]: pgmap v411: 97 pgs: 97 active+clean; 633 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-26T21:54:13.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:54:12 smithi164 ceph-mon[31916]: osdmap e91: 12 total, 12 up, 12 in 2024-04-26T21:54:13.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:54:12 smithi076 ceph-mon[31823]: pgmap v411: 97 pgs: 97 active+clean; 633 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-26T21:54:13.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:54:12 smithi076 ceph-mon[31823]: osdmap e91: 12 total, 12 up, 12 in 2024-04-26T21:54:13.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:54:12 smithi072 ceph-mon[28138]: pgmap v411: 97 pgs: 97 active+clean; 633 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-26T21:54:13.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:54:12 smithi072 ceph-mon[28138]: osdmap e91: 12 total, 12 up, 12 in 2024-04-26T21:54:13.605 INFO:tasks.cephadm.ceph_manager.ceph:need seq 296352743463 got 296352743471 for osd.9 2024-04-26T21:54:13.605 DEBUG:teuthology.parallel:result is None 2024-04-26T21:54:14.565 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/a1780950-0415-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:54:15.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:54:14 smithi072 ceph-mon[28138]: pgmap v413: 97 pgs: 97 active+clean; 633 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 7 op/s; 77 B/s, 0 objects/s recovering 2024-04-26T21:54:15.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:54:14 smithi164 ceph-mon[31916]: pgmap v413: 97 pgs: 97 active+clean; 633 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 7 op/s; 77 B/s, 0 objects/s recovering 2024-04-26T21:54:15.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:54:14 smithi076 ceph-mon[31823]: pgmap v413: 97 pgs: 97 active+clean; 633 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 7 op/s; 77 B/s, 0 objects/s recovering 2024-04-26T21:54:15.526 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:15.525+0000 7f5952e00700 1 -- 172.21.15.72:0/3407741498 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f594c06bad0 msgr2=0x7f594c1102e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:15.526 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:15.525+0000 7f5952e00700 1 --2- 172.21.15.72:0/3407741498 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f594c06bad0 0x7f594c1102e0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f593c009990 tx=0x7f593c009ca0 comp rx=0 tx=0).stop 2024-04-26T21:54:15.526 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:15.525+0000 7f5952e00700 1 -- 172.21.15.72:0/3407741498 shutdown_connections 2024-04-26T21:54:15.526 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:15.525+0000 7f5952e00700 1 --2- 172.21.15.72:0/3407741498 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f594c06bad0 0x7f594c1102e0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:15.526 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:15.525+0000 7f5952e00700 1 --2- 172.21.15.72:0/3407741498 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f594c1094a0 0x7f594c109880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:15.526 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:15.525+0000 7f5952e00700 1 --2- 172.21.15.72:0/3407741498 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f594c06b130 0x7f594c06b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:15.526 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:15.525+0000 7f5952e00700 1 -- 172.21.15.72:0/3407741498 >> 172.21.15.72:0/3407741498 conn(0x7f594c0792d0 msgr2=0x7f594c0796e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:54:15.526 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:15.525+0000 7f5952e00700 1 -- 172.21.15.72:0/3407741498 shutdown_connections 2024-04-26T21:54:15.526 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:15.525+0000 7f5952e00700 1 -- 172.21.15.72:0/3407741498 wait complete. 2024-04-26T21:54:15.527 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:15.526+0000 7f5952e00700 1 Processor -- start 2024-04-26T21:54:15.527 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:15.527+0000 7f5952e00700 1 -- start start 2024-04-26T21:54:15.527 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:15.527+0000 7f5952e00700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f594c06b130 0x7f594c1a1790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:15.528 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:15.527+0000 7f5952e00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f594c06bad0 0x7f594c1a1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:15.528 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:15.527+0000 7f5952e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f594c1094a0 0x7f594c19b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:15.528 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:15.527+0000 7f5952e00700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f594c113450 con 0x7f594c06bad0 2024-04-26T21:54:15.528 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:15.527+0000 7f5952e00700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f594c1132d0 con 0x7f594c06b130 2024-04-26T21:54:15.528 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:15.527+0000 7f5952e00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f594c1135d0 con 0x7f594c1094a0 2024-04-26T21:54:15.530 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:15.528+0000 7f594b400700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f594c06bad0 0x7f594c1a1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:15.530 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:15.528+0000 7f594b400700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f594c06bad0 0x7f594c1a1cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:55176/0 (socket says 172.21.15.72:55176) 2024-04-26T21:54:15.530 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:15.528+0000 7f594b400700 1 -- 172.21.15.72:0/3192043987 learned_addr learned my addr 172.21.15.72:0/3192043987 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:54:15.530 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:15.528+0000 7f594be00700 1 --2- 172.21.15.72:0/3192043987 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f594c06b130 0x7f594c1a1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:15.530 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:15.528+0000 7f5950e00700 1 --2- 172.21.15.72:0/3192043987 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f594c1094a0 0x7f594c19b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:15.530 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:15.528+0000 7f594b400700 1 -- 172.21.15.72:0/3192043987 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f594c1094a0 msgr2=0x7f594c19b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:15.530 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:15.528+0000 7f594b400700 1 --2- 172.21.15.72:0/3192043987 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f594c1094a0 0x7f594c19b950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:15.530 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:15.528+0000 7f594b400700 1 -- 172.21.15.72:0/3192043987 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f594c06b130 msgr2=0x7f594c1a1790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:15.530 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:15.528+0000 7f594b400700 1 --2- 172.21.15.72:0/3192043987 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f594c06b130 0x7f594c1a1790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:15.530 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:15.528+0000 7f594b400700 1 -- 172.21.15.72:0/3192043987 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f594c19c0b0 con 0x7f594c06bad0 2024-04-26T21:54:15.530 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:15.528+0000 7f5950e00700 1 --2- 172.21.15.72:0/3192043987 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f594c1094a0 0x7f594c19b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:54:15.530 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:15.528+0000 7f594b400700 1 --2- 172.21.15.72:0/3192043987 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f594c06bad0 0x7f594c1a1cd0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f5934007e20 tx=0x7f593400a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:54:15.531 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:15.529+0000 7f5948c00700 1 -- 172.21.15.72:0/3192043987 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5934016020 con 0x7f594c06bad0 2024-04-26T21:54:15.531 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:15.529+0000 7f5952e00700 1 -- 172.21.15.72:0/3192043987 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f594c19c3a0 con 0x7f594c06bad0 2024-04-26T21:54:15.531 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:15.529+0000 7f5948c00700 1 -- 172.21.15.72:0/3192043987 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f593400ae00 con 0x7f594c06bad0 2024-04-26T21:54:15.531 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:15.529+0000 7f5948c00700 1 -- 172.21.15.72:0/3192043987 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5934011670 con 0x7f594c06bad0 2024-04-26T21:54:15.531 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:15.529+0000 7f5952e00700 1 -- 172.21.15.72:0/3192043987 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f594c1a8660 con 0x7f594c06bad0 2024-04-26T21:54:15.531 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:15.530+0000 7f5952e00700 1 -- 172.21.15.72:0/3192043987 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f594c06a110 con 0x7f594c06bad0 2024-04-26T21:54:15.532 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:15.532+0000 7f5948c00700 1 -- 172.21.15.72:0/3192043987 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f5934019020 con 0x7f594c06bad0 2024-04-26T21:54:15.534 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:15.534+0000 7f5948c00700 1 --2- 172.21.15.72:0/3192043987 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f59300a3cd0 0x7f59300a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:15.534 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:15.534+0000 7f5948c00700 1 -- 172.21.15.72:0/3192043987 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7f59340d0350 con 0x7f594c06bad0 2024-04-26T21:54:15.534 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:15.534+0000 7f594be00700 1 --2- 172.21.15.72:0/3192043987 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f59300a3cd0 0x7f59300a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:15.535 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:15.535+0000 7f594be00700 1 --2- 172.21.15.72:0/3192043987 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f59300a3cd0 0x7f59300a6190 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f594c19ca90 tx=0x7f594400b5b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:54:15.539 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:15.539+0000 7f5948c00700 1 -- 172.21.15.72:0/3192043987 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f593409cc00 con 0x7f594c06bad0 2024-04-26T21:54:15.774 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:15.773+0000 7f5952e00700 1 -- 172.21.15.72:0/3192043987 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f594c1a88f0 con 0x7f594c06bad0 2024-04-26T21:54:15.774 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:15.774+0000 7f5948c00700 1 -- 172.21.15.72:0/3192043987 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f593409c500 con 0x7f594c06bad0 2024-04-26T21:54:15.775 INFO:teuthology.orchestra.run.smithi072.stdout:141733920851 2024-04-26T21:54:15.777 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:15.777+0000 7f5952e00700 1 -- 172.21.15.72:0/3192043987 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f59300a3cd0 msgr2=0x7f59300a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:15.777 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:15.777+0000 7f5952e00700 1 --2- 172.21.15.72:0/3192043987 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f59300a3cd0 0x7f59300a6190 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f594c19ca90 tx=0x7f594400b5b0 comp rx=0 tx=0).stop 2024-04-26T21:54:15.777 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:15.777+0000 7f5952e00700 1 -- 172.21.15.72:0/3192043987 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f594c06bad0 msgr2=0x7f594c1a1cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:15.777 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:15.777+0000 7f5952e00700 1 --2- 172.21.15.72:0/3192043987 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f594c06bad0 0x7f594c1a1cd0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f5934007e20 tx=0x7f593400a340 comp rx=0 tx=0).stop 2024-04-26T21:54:15.779 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:15.777+0000 7f5952e00700 1 -- 172.21.15.72:0/3192043987 shutdown_connections 2024-04-26T21:54:15.779 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:15.777+0000 7f5952e00700 1 --2- 172.21.15.72:0/3192043987 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f59300a3cd0 0x7f59300a6190 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:15.779 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:15.777+0000 7f5952e00700 1 --2- 172.21.15.72:0/3192043987 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f594c1094a0 0x7f594c19b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:15.779 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:15.777+0000 7f5952e00700 1 --2- 172.21.15.72:0/3192043987 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f594c06b130 0x7f594c1a1790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:15.779 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:15.777+0000 7f5952e00700 1 --2- 172.21.15.72:0/3192043987 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f594c06bad0 0x7f594c1a1cd0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:15.779 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:15.777+0000 7f5952e00700 1 -- 172.21.15.72:0/3192043987 >> 172.21.15.72:0/3192043987 conn(0x7f594c0792d0 msgr2=0x7f594c100470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:54:15.780 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:15.777+0000 7f5952e00700 1 -- 172.21.15.72:0/3192043987 shutdown_connections 2024-04-26T21:54:15.780 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:15.778+0000 7f5952e00700 1 -- 172.21.15.72:0/3192043987 wait complete. 2024-04-26T21:54:16.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:54:15 smithi072 ceph-mon[28138]: from='client.? 172.21.15.72:0/3192043987' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-26T21:54:16.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:54:15 smithi164 ceph-mon[31916]: from='client.? 172.21.15.72:0/3192043987' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-26T21:54:16.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:54:15 smithi076 ceph-mon[31823]: from='client.? 172.21.15.72:0/3192043987' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-26T21:54:17.049 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:54:16 smithi072 ceph-mon[28138]: pgmap v414: 97 pgs: 97 active+clean; 633 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 127 B/s wr, 11 op/s; 77 B/s, 0 keys/s, 0 objects/s recovering 2024-04-26T21:54:17.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:54:16 smithi164 ceph-mon[31916]: pgmap v414: 97 pgs: 97 active+clean; 633 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 127 B/s wr, 11 op/s; 77 B/s, 0 keys/s, 0 objects/s recovering 2024-04-26T21:54:17.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:54:16 smithi076 ceph-mon[31823]: pgmap v414: 97 pgs: 97 active+clean; 633 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 127 B/s wr, 11 op/s; 77 B/s, 0 keys/s, 0 objects/s recovering 2024-04-26T21:54:18.431 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:18.430+0000 7f5cb5000700 1 -- 172.21.15.72:0/267965664 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5cb006bad0 msgr2=0x7f5cb01102e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:18.431 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:18.430+0000 7f5cb5000700 1 --2- 172.21.15.72:0/267965664 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5cb006bad0 0x7f5cb01102e0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f5ca800b0a0 tx=0x7f5ca800b3b0 comp rx=0 tx=0).stop 2024-04-26T21:54:18.431 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:18.430+0000 7f5cb5000700 1 -- 172.21.15.72:0/267965664 shutdown_connections 2024-04-26T21:54:18.431 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:18.430+0000 7f5cb5000700 1 --2- 172.21.15.72:0/267965664 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5cb006bad0 0x7f5cb01102e0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:18.432 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:18.430+0000 7f5cb5000700 1 --2- 172.21.15.72:0/267965664 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5cb01094a0 0x7f5cb0109880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:18.432 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:18.430+0000 7f5cb5000700 1 --2- 172.21.15.72:0/267965664 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5cb006b130 0x7f5cb006b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:18.432 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:18.430+0000 7f5cb5000700 1 -- 172.21.15.72:0/267965664 >> 172.21.15.72:0/267965664 conn(0x7f5cb00792d0 msgr2=0x7f5cb00796e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:54:18.432 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:18.430+0000 7f5cb5000700 1 -- 172.21.15.72:0/267965664 shutdown_connections 2024-04-26T21:54:18.432 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:18.430+0000 7f5cb5000700 1 -- 172.21.15.72:0/267965664 wait complete. 2024-04-26T21:54:18.434 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:18.432+0000 7f5cb5000700 1 Processor -- start 2024-04-26T21:54:18.435 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:18.432+0000 7f5cb5000700 1 -- start start 2024-04-26T21:54:18.435 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:18.432+0000 7f5cb5000700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5cb006b130 0x7f5cb01023f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:18.435 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:18.432+0000 7f5cb5000700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5cb006bad0 0x7f5cb0102930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:18.435 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:18.432+0000 7f5cb5000700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5cb01094a0 0x7f5cb0102e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:18.435 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:18.432+0000 7f5cb5000700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f5cb01134c0 con 0x7f5cb01094a0 2024-04-26T21:54:18.435 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:18.432+0000 7f5cb5000700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f5cb0113340 con 0x7f5cb006bad0 2024-04-26T21:54:18.435 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:18.432+0000 7f5cb5000700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f5cb0113640 con 0x7f5cb006b130 2024-04-26T21:54:18.435 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:18.433+0000 7f5cae800700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5cb01094a0 0x7f5cb0102e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:18.435 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:18.433+0000 7f5cae800700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5cb01094a0 0x7f5cb0102e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:55220/0 (socket says 172.21.15.72:55220) 2024-04-26T21:54:18.436 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:18.433+0000 7f5cae800700 1 -- 172.21.15.72:0/3461674891 learned_addr learned my addr 172.21.15.72:0/3461674891 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:54:18.436 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:18.433+0000 7f5cade00700 1 --2- 172.21.15.72:0/3461674891 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5cb006b130 0x7f5cb01023f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:18.436 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:18.433+0000 7f5cad400700 1 --2- 172.21.15.72:0/3461674891 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5cb006bad0 0x7f5cb0102930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:18.436 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:18.433+0000 7f5cae800700 1 -- 172.21.15.72:0/3461674891 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5cb006b130 msgr2=0x7f5cb01023f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:18.436 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:18.433+0000 7f5cae800700 1 --2- 172.21.15.72:0/3461674891 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5cb006b130 0x7f5cb01023f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:18.436 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:18.433+0000 7f5cae800700 1 -- 172.21.15.72:0/3461674891 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5cb006bad0 msgr2=0x7f5cb0102930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:18.436 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:18.433+0000 7f5cae800700 1 --2- 172.21.15.72:0/3461674891 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5cb006bad0 0x7f5cb0102930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:18.436 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:18.433+0000 7f5cae800700 1 -- 172.21.15.72:0/3461674891 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cb01a7fa0 con 0x7f5cb01094a0 2024-04-26T21:54:18.436 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:18.433+0000 7f5cad400700 1 --2- 172.21.15.72:0/3461674891 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5cb006bad0 0x7f5cb0102930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:54:18.436 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:18.434+0000 7f5cae800700 1 --2- 172.21.15.72:0/3461674891 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5cb01094a0 0x7f5cb0102e70 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f5ca800b070 tx=0x7f5ca8004e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:54:18.436 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:18.434+0000 7f5ca6a00700 1 -- 172.21.15.72:0/3461674891 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ca8021070 con 0x7f5cb01094a0 2024-04-26T21:54:18.436 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:18.434+0000 7f5ca6a00700 1 -- 172.21.15.72:0/3461674891 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5ca8007c00 con 0x7f5cb01094a0 2024-04-26T21:54:18.436 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:18.434+0000 7f5ca6a00700 1 -- 172.21.15.72:0/3461674891 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ca8004780 con 0x7f5cb01094a0 2024-04-26T21:54:18.436 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:18.434+0000 7f5cb5000700 1 -- 172.21.15.72:0/3461674891 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5cb01a8230 con 0x7f5cb01094a0 2024-04-26T21:54:18.437 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:18.434+0000 7f5cb5000700 1 -- 172.21.15.72:0/3461674891 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5cb01a8630 con 0x7f5cb01094a0 2024-04-26T21:54:18.437 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:18.436+0000 7f5cb5000700 1 -- 172.21.15.72:0/3461674891 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5cb006a110 con 0x7f5cb01094a0 2024-04-26T21:54:18.439 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:18.439+0000 7f5ca6a00700 1 -- 172.21.15.72:0/3461674891 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f5ca8003a00 con 0x7f5cb01094a0 2024-04-26T21:54:18.441 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:18.440+0000 7f5ca6a00700 1 --2- 172.21.15.72:0/3461674891 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f5c940a3d50 0x7f5c940a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:18.441 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:18.441+0000 7f5ca6a00700 1 -- 172.21.15.72:0/3461674891 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7f5ca800f020 con 0x7f5cb01094a0 2024-04-26T21:54:18.441 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:18.441+0000 7f5cade00700 1 --2- 172.21.15.72:0/3461674891 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f5c940a3d50 0x7f5c940a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:18.442 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:18.442+0000 7f5cade00700 1 --2- 172.21.15.72:0/3461674891 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f5c940a3d50 0x7f5c940a6210 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f5ca000bbb0 tx=0x7f5ca000b590 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:54:18.443 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:18.443+0000 7f5ca6a00700 1 -- 172.21.15.72:0/3461674891 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f5ca802a020 con 0x7f5cb01094a0 2024-04-26T21:54:18.684 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:18.684+0000 7f5cb5000700 1 -- 172.21.15.72:0/3461674891 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f5cb002e490 con 0x7f5cb01094a0 2024-04-26T21:54:18.685 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:18.685+0000 7f5ca6a00700 1 -- 172.21.15.72:0/3461674891 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f5ca809e8c0 con 0x7f5cb01094a0 2024-04-26T21:54:18.685 INFO:teuthology.orchestra.run.smithi072.stdout:223338299453 2024-04-26T21:54:18.688 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:18.687+0000 7f5cb5000700 1 -- 172.21.15.72:0/3461674891 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f5c940a3d50 msgr2=0x7f5c940a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:18.688 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:18.687+0000 7f5cb5000700 1 --2- 172.21.15.72:0/3461674891 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f5c940a3d50 0x7f5c940a6210 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f5ca000bbb0 tx=0x7f5ca000b590 comp rx=0 tx=0).stop 2024-04-26T21:54:18.688 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:18.687+0000 7f5cb5000700 1 -- 172.21.15.72:0/3461674891 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5cb01094a0 msgr2=0x7f5cb0102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:18.688 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:18.687+0000 7f5cb5000700 1 --2- 172.21.15.72:0/3461674891 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5cb01094a0 0x7f5cb0102e70 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f5ca800b070 tx=0x7f5ca8004e00 comp rx=0 tx=0).stop 2024-04-26T21:54:18.688 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:18.688+0000 7f5cb5000700 1 -- 172.21.15.72:0/3461674891 shutdown_connections 2024-04-26T21:54:18.688 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:18.688+0000 7f5cb5000700 1 --2- 172.21.15.72:0/3461674891 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f5c940a3d50 0x7f5c940a6210 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:18.688 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:18.688+0000 7f5cb5000700 1 --2- 172.21.15.72:0/3461674891 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5cb01094a0 0x7f5cb0102e70 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:18.688 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:18.688+0000 7f5cb5000700 1 --2- 172.21.15.72:0/3461674891 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5cb006b130 0x7f5cb01023f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:18.689 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:18.688+0000 7f5cb5000700 1 --2- 172.21.15.72:0/3461674891 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5cb006bad0 0x7f5cb0102930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:18.689 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:18.688+0000 7f5cb5000700 1 -- 172.21.15.72:0/3461674891 >> 172.21.15.72:0/3461674891 conn(0x7f5cb00792d0 msgr2=0x7f5cb01004d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:54:18.689 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:18.688+0000 7f5cb5000700 1 -- 172.21.15.72:0/3461674891 shutdown_connections 2024-04-26T21:54:18.689 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:18.688+0000 7f5cb5000700 1 -- 172.21.15.72:0/3461674891 wait complete. 2024-04-26T21:54:18.807 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:18.339+0000 7fcaf7800700 1 -- 172.21.15.72:0/468484396 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcaf0078270 msgr2=0x7fcaf0113d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:18.807 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:18.339+0000 7fcaf7800700 1 --2- 172.21.15.72:0/468484396 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcaf0078270 0x7fcaf0113d90 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fcadc009990 tx=0x7fcadc009ca0 comp rx=0 tx=0).stop 2024-04-26T21:54:18.807 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:18.340+0000 7fcaf7800700 1 -- 172.21.15.72:0/468484396 shutdown_connections 2024-04-26T21:54:18.807 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:18.340+0000 7fcaf7800700 1 --2- 172.21.15.72:0/468484396 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcaf01142d0 0x7fcaf01166c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:18.807 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:18.340+0000 7fcaf7800700 1 --2- 172.21.15.72:0/468484396 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcaf0077950 0x7fcaf0077d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:18.807 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:18.340+0000 7fcaf7800700 1 --2- 172.21.15.72:0/468484396 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcaf0078270 0x7fcaf0113d90 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:18.807 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:18.340+0000 7fcaf7800700 1 -- 172.21.15.72:0/468484396 >> 172.21.15.72:0/468484396 conn(0x7fcaf0101120 msgr2=0x7fcaf0103540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:54:18.807 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:18.340+0000 7fcaf7800700 1 -- 172.21.15.72:0/468484396 shutdown_connections 2024-04-26T21:54:18.807 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:18.340+0000 7fcaf7800700 1 -- 172.21.15.72:0/468484396 wait complete. 2024-04-26T21:54:18.808 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:18.341+0000 7fcaf7800700 1 Processor -- start 2024-04-26T21:54:18.808 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:18.341+0000 7fcaf7800700 1 -- start start 2024-04-26T21:54:18.808 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:18.341+0000 7fcaf7800700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcaf0077950 0x7fcaf01a00f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:18.808 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:18.341+0000 7fcaf7800700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcaf0078270 0x7fcaf01a0630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:18.808 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:18.342+0000 7fcaf7800700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcaf01142d0 0x7fcaf01a4970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:18.808 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:18.342+0000 7fcaf7800700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fcaf0119b60 con 0x7fcaf0077950 2024-04-26T21:54:18.808 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:18.342+0000 7fcaf7800700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fcaf01199e0 con 0x7fcaf0078270 2024-04-26T21:54:18.808 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:18.342+0000 7fcaf7800700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fcaf0119ce0 con 0x7fcaf01142d0 2024-04-26T21:54:18.808 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:18.342+0000 7fcaf4e00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcaf0077950 0x7fcaf01a00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:18.808 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:18.342+0000 7fcaf4e00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcaf0077950 0x7fcaf01a00f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:55198/0 (socket says 172.21.15.72:55198) 2024-04-26T21:54:18.808 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:18.342+0000 7fcaf4e00700 1 -- 172.21.15.72:0/2351074897 learned_addr learned my addr 172.21.15.72:0/2351074897 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:54:18.808 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:18.342+0000 7fcaefe00700 1 --2- 172.21.15.72:0/2351074897 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcaf0078270 0x7fcaf01a0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:18.808 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:18.342+0000 7fcaf4e00700 1 -- 172.21.15.72:0/2351074897 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcaf01142d0 msgr2=0x7fcaf01a4970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:18.808 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:18.342+0000 7fcaf4e00700 1 --2- 172.21.15.72:0/2351074897 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcaf01142d0 0x7fcaf01a4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:18.808 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:18.342+0000 7fcaf4e00700 1 -- 172.21.15.72:0/2351074897 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcaf0078270 msgr2=0x7fcaf01a0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:18.809 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:18.342+0000 7fcaf4e00700 1 --2- 172.21.15.72:0/2351074897 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcaf0078270 0x7fcaf01a0630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:18.809 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:18.342+0000 7fcaf4e00700 1 -- 172.21.15.72:0/2351074897 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcaf01a50b0 con 0x7fcaf0077950 2024-04-26T21:54:18.809 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:18.342+0000 7fcaf4e00700 1 --2- 172.21.15.72:0/2351074897 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcaf0077950 0x7fcaf01a00f0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fcae400d950 tx=0x7fcae400dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:54:18.809 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:18.343+0000 7fcaed600700 1 -- 172.21.15.72:0/2351074897 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcae4016070 con 0x7fcaf0077950 2024-04-26T21:54:18.809 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:18.343+0000 7fcaed600700 1 -- 172.21.15.72:0/2351074897 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcae4010460 con 0x7fcaf0077950 2024-04-26T21:54:18.809 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:18.343+0000 7fcaed600700 1 -- 172.21.15.72:0/2351074897 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcae400f620 con 0x7fcaf0077950 2024-04-26T21:54:18.809 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:18.343+0000 7fcaf7800700 1 -- 172.21.15.72:0/2351074897 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcaf01a53a0 con 0x7fcaf0077950 2024-04-26T21:54:18.809 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:18.343+0000 7fcaf7800700 1 -- 172.21.15.72:0/2351074897 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcaf01acc20 con 0x7fcaf0077950 2024-04-26T21:54:18.809 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:18.344+0000 7fcaf7800700 1 -- 172.21.15.72:0/2351074897 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcaf006a110 con 0x7fcaf0077950 2024-04-26T21:54:18.809 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:18.346+0000 7fcaed600700 1 -- 172.21.15.72:0/2351074897 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fcae4019070 con 0x7fcaf0077950 2024-04-26T21:54:18.809 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:18.348+0000 7fcaed600700 1 --2- 172.21.15.72:0/2351074897 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fcad80a3cd0 0x7fcad80a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:18.809 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:18.348+0000 7fcaed600700 1 -- 172.21.15.72:0/2351074897 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7fcae40d0260 con 0x7fcaf0077950 2024-04-26T21:54:18.809 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:18.348+0000 7fcaefe00700 1 --2- 172.21.15.72:0/2351074897 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fcad80a3cd0 0x7fcad80a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:18.809 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:18.349+0000 7fcaefe00700 1 --2- 172.21.15.72:0/2351074897 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fcad80a3cd0 0x7fcad80a6190 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fcadc00b5c0 tx=0x7fcadc01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:54:18.810 INFO:teuthology.orchestra.run.smithi072.stdout:55834574951 2024-04-26T21:54:18.810 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:18.352+0000 7fcaed600700 1 -- 172.21.15.72:0/2351074897 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fcae409ca00 con 0x7fcaf0077950 2024-04-26T21:54:18.810 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:18.585+0000 7fcaf7800700 1 -- 172.21.15.72:0/2351074897 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7fcaf002e650 con 0x7fcaf0077950 2024-04-26T21:54:18.810 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:18.585+0000 7fcaed600700 1 -- 172.21.15.72:0/2351074897 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fcae409c300 con 0x7fcaf0077950 2024-04-26T21:54:18.810 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:18.588+0000 7fcaf7800700 1 -- 172.21.15.72:0/2351074897 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fcad80a3cd0 msgr2=0x7fcad80a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:18.810 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:18.588+0000 7fcaf7800700 1 --2- 172.21.15.72:0/2351074897 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fcad80a3cd0 0x7fcad80a6190 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fcadc00b5c0 tx=0x7fcadc01a040 comp rx=0 tx=0).stop 2024-04-26T21:54:18.810 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:18.588+0000 7fcaf7800700 1 -- 172.21.15.72:0/2351074897 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcaf0077950 msgr2=0x7fcaf01a00f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:18.811 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:18.588+0000 7fcaf7800700 1 --2- 172.21.15.72:0/2351074897 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcaf0077950 0x7fcaf01a00f0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fcae400d950 tx=0x7fcae400dd10 comp rx=0 tx=0).stop 2024-04-26T21:54:18.811 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:18.589+0000 7fcaf7800700 1 -- 172.21.15.72:0/2351074897 shutdown_connections 2024-04-26T21:54:18.811 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:18.589+0000 7fcaf7800700 1 --2- 172.21.15.72:0/2351074897 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fcad80a3cd0 0x7fcad80a6190 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:18.811 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:18.589+0000 7fcaf7800700 1 --2- 172.21.15.72:0/2351074897 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcaf0077950 0x7fcaf01a00f0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:18.811 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:18.589+0000 7fcaf7800700 1 --2- 172.21.15.72:0/2351074897 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcaf01142d0 0x7fcaf01a4970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:18.811 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:18.589+0000 7fcaf7800700 1 --2- 172.21.15.72:0/2351074897 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcaf0078270 0x7fcaf01a0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:18.811 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:18.589+0000 7fcaf7800700 1 -- 172.21.15.72:0/2351074897 >> 172.21.15.72:0/2351074897 conn(0x7fcaf0101120 msgr2=0x7fcaf0103510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:54:18.811 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:18.589+0000 7fcaf7800700 1 -- 172.21.15.72:0/2351074897 shutdown_connections 2024-04-26T21:54:18.811 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:18.589+0000 7fcaf7800700 1 -- 172.21.15.72:0/2351074897 wait complete. 2024-04-26T21:54:19.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:54:18 smithi164 ceph-mon[31916]: pgmap v415: 97 pgs: 97 active+clean; 633 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 127 B/s wr, 7 op/s; 77 B/s, 0 keys/s, 0 objects/s recovering 2024-04-26T21:54:19.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:54:18 smithi164 ceph-mon[31916]: from='client.? 172.21.15.72:0/2351074897' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-26T21:54:19.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:54:18 smithi164 ceph-mon[31916]: from='client.? 172.21.15.72:0/3461674891' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-26T21:54:19.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:54:18 smithi076 ceph-mon[31823]: pgmap v415: 97 pgs: 97 active+clean; 633 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 127 B/s wr, 7 op/s; 77 B/s, 0 keys/s, 0 objects/s recovering 2024-04-26T21:54:19.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:54:18 smithi076 ceph-mon[31823]: from='client.? 172.21.15.72:0/2351074897' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-26T21:54:19.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:54:18 smithi076 ceph-mon[31823]: from='client.? 172.21.15.72:0/3461674891' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-26T21:54:19.321 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:54:18 smithi072 ceph-mon[28138]: pgmap v415: 97 pgs: 97 active+clean; 633 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 127 B/s wr, 7 op/s; 77 B/s, 0 keys/s, 0 objects/s recovering 2024-04-26T21:54:19.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:54:18 smithi072 ceph-mon[28138]: from='client.? 172.21.15.72:0/2351074897' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-26T21:54:19.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:54:18 smithi072 ceph-mon[28138]: from='client.? 172.21.15.72:0/3461674891' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-26T21:54:19.711 INFO:tasks.cephadm.ceph_manager.ceph:need seq 141733920844 got 141733920851 for osd.4 2024-04-26T21:54:19.711 DEBUG:teuthology.parallel:result is None 2024-04-26T21:54:20.096 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:19.966+0000 7f5ace800700 1 -- 172.21.15.72:0/363078541 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5ac806baf0 msgr2=0x7f5ac806bf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:20.096 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:19.966+0000 7f5ace800700 1 --2- 172.21.15.72:0/363078541 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5ac806baf0 0x7f5ac806bf50 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f5ab4009990 tx=0x7f5ab4009ca0 comp rx=0 tx=0).stop 2024-04-26T21:54:20.096 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:19.966+0000 7f5ace800700 1 -- 172.21.15.72:0/363078541 shutdown_connections 2024-04-26T21:54:20.096 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:19.966+0000 7f5ace800700 1 --2- 172.21.15.72:0/363078541 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5ac806c490 0x7f5ac8110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:20.097 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:19.966+0000 7f5ace800700 1 --2- 172.21.15.72:0/363078541 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5ac81094c0 0x7f5ac81098a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:20.097 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:19.966+0000 7f5ace800700 1 --2- 172.21.15.72:0/363078541 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5ac806baf0 0x7f5ac806bf50 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:20.097 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:19.966+0000 7f5ace800700 1 -- 172.21.15.72:0/363078541 >> 172.21.15.72:0/363078541 conn(0x7f5ac80792d0 msgr2=0x7f5ac80796e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:54:20.097 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:19.966+0000 7f5ace800700 1 -- 172.21.15.72:0/363078541 shutdown_connections 2024-04-26T21:54:20.097 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:19.966+0000 7f5ace800700 1 -- 172.21.15.72:0/363078541 wait complete. 2024-04-26T21:54:20.097 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:19.968+0000 7f5ace800700 1 Processor -- start 2024-04-26T21:54:20.097 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:19.968+0000 7f5ace800700 1 -- start start 2024-04-26T21:54:20.097 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:19.968+0000 7f5ace800700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5ac806baf0 0x7f5ac819f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:20.097 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:19.968+0000 7f5ace800700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5ac806c490 0x7f5ac819fb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:20.097 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:19.968+0000 7f5ace800700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5ac81094c0 0x7f5ac81997d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:20.097 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:19.968+0000 7f5ace800700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f5ac8113490 con 0x7f5ac806c490 2024-04-26T21:54:20.097 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:19.968+0000 7f5ace800700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f5ac8113310 con 0x7f5ac806baf0 2024-04-26T21:54:20.097 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:19.968+0000 7f5ace800700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f5ac8113610 con 0x7f5ac81094c0 2024-04-26T21:54:20.098 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:19.968+0000 7f5ac6a00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5ac806c490 0x7f5ac819fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:20.098 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:19.969+0000 7f5ac6a00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5ac806c490 0x7f5ac819fb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:55240/0 (socket says 172.21.15.72:55240) 2024-04-26T21:54:20.098 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:19.969+0000 7f5ac6a00700 1 -- 172.21.15.72:0/855459483 learned_addr learned my addr 172.21.15.72:0/855459483 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:54:20.098 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:19.969+0000 7f5ac7400700 1 --2- 172.21.15.72:0/855459483 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5ac806baf0 0x7f5ac819f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:20.098 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:19.969+0000 7f5ac7e00700 1 --2- 172.21.15.72:0/855459483 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5ac81094c0 0x7f5ac81997d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:20.098 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:19.969+0000 7f5ac6a00700 1 -- 172.21.15.72:0/855459483 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5ac81094c0 msgr2=0x7f5ac81997d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:20.098 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:19.969+0000 7f5ac6a00700 1 --2- 172.21.15.72:0/855459483 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5ac81094c0 0x7f5ac81997d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:20.098 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:19.969+0000 7f5ac6a00700 1 -- 172.21.15.72:0/855459483 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5ac806baf0 msgr2=0x7f5ac819f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:20.098 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:19.969+0000 7f5ac6a00700 1 --2- 172.21.15.72:0/855459483 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5ac806baf0 0x7f5ac819f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:20.098 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:19.969+0000 7f5ac6a00700 1 -- 172.21.15.72:0/855459483 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ac8199fb0 con 0x7f5ac806c490 2024-04-26T21:54:20.098 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:19.969+0000 7f5ac7400700 1 --2- 172.21.15.72:0/855459483 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5ac806baf0 0x7f5ac819f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:54:20.098 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:19.969+0000 7f5ac7e00700 1 --2- 172.21.15.72:0/855459483 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5ac81094c0 0x7f5ac81997d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:54:20.098 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:19.970+0000 7f5ac6a00700 1 --2- 172.21.15.72:0/855459483 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5ac806c490 0x7f5ac819fb90 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f5ab40057c0 tx=0x7f5ab4009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:54:20.099 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:19.970+0000 7f5abfe00700 1 -- 172.21.15.72:0/855459483 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ab4027070 con 0x7f5ac806c490 2024-04-26T21:54:20.099 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:19.970+0000 7f5abfe00700 1 -- 172.21.15.72:0/855459483 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5ab400fda0 con 0x7f5ac806c490 2024-04-26T21:54:20.099 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:19.970+0000 7f5abfe00700 1 -- 172.21.15.72:0/855459483 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ab4022ca0 con 0x7f5ac806c490 2024-04-26T21:54:20.099 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:19.970+0000 7f5ace800700 1 -- 172.21.15.72:0/855459483 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ac819a240 con 0x7f5ac806c490 2024-04-26T21:54:20.099 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:19.970+0000 7f5ace800700 1 -- 172.21.15.72:0/855459483 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ac81a64b0 con 0x7f5ac806c490 2024-04-26T21:54:20.099 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:19.971+0000 7f5ace800700 1 -- 172.21.15.72:0/855459483 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ac806b3a0 con 0x7f5ac806c490 2024-04-26T21:54:20.099 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:19.973+0000 7f5abfe00700 1 -- 172.21.15.72:0/855459483 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f5ab402a070 con 0x7f5ac806c490 2024-04-26T21:54:20.099 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:19.974+0000 7f5abfe00700 1 --2- 172.21.15.72:0/855459483 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f5aac0a3d50 0x7f5aac0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:20.099 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:19.975+0000 7f5abfe00700 1 -- 172.21.15.72:0/855459483 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7f5ab40d1690 con 0x7f5ac806c490 2024-04-26T21:54:20.099 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:19.975+0000 7f5ac7400700 1 --2- 172.21.15.72:0/855459483 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f5aac0a3d50 0x7f5aac0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:20.099 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:19.975+0000 7f5ac7400700 1 --2- 172.21.15.72:0/855459483 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f5aac0a3d50 0x7f5aac0a6210 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f5ab8005fd0 tx=0x7f5ab800a5e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:54:20.099 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:19.977+0000 7f5abfe00700 1 -- 172.21.15.72:0/855459483 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f5ab4031080 con 0x7f5ac806c490 2024-04-26T21:54:20.204 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:20.203+0000 7f5ace800700 1 -- 172.21.15.72:0/855459483 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f5ac8104b70 con 0x7f5ac806c490 2024-04-26T21:54:20.204 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:20.204+0000 7f5abfe00700 1 -- 172.21.15.72:0/855459483 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f5ab4013070 con 0x7f5ac806c490 2024-04-26T21:54:20.205 INFO:teuthology.orchestra.run.smithi072.stdout:34359738479 2024-04-26T21:54:20.208 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:20.208+0000 7f5ace800700 1 -- 172.21.15.72:0/855459483 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f5aac0a3d50 msgr2=0x7f5aac0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:20.208 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:20.208+0000 7f5ace800700 1 --2- 172.21.15.72:0/855459483 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f5aac0a3d50 0x7f5aac0a6210 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f5ab8005fd0 tx=0x7f5ab800a5e0 comp rx=0 tx=0).stop 2024-04-26T21:54:20.209 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:20.208+0000 7f5ace800700 1 -- 172.21.15.72:0/855459483 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5ac806c490 msgr2=0x7f5ac819fb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:20.209 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:20.208+0000 7f5ace800700 1 --2- 172.21.15.72:0/855459483 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5ac806c490 0x7f5ac819fb90 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f5ab40057c0 tx=0x7f5ab4009f90 comp rx=0 tx=0).stop 2024-04-26T21:54:20.209 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:20.208+0000 7f5ace800700 1 -- 172.21.15.72:0/855459483 shutdown_connections 2024-04-26T21:54:20.209 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:20.208+0000 7f5ace800700 1 --2- 172.21.15.72:0/855459483 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f5aac0a3d50 0x7f5aac0a6210 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:20.209 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:20.208+0000 7f5ace800700 1 --2- 172.21.15.72:0/855459483 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5ac81094c0 0x7f5ac81997d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:20.209 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:20.208+0000 7f5ace800700 1 --2- 172.21.15.72:0/855459483 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5ac806baf0 0x7f5ac819f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:20.209 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:20.208+0000 7f5ace800700 1 --2- 172.21.15.72:0/855459483 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5ac806c490 0x7f5ac819fb90 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:20.209 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:20.208+0000 7f5ace800700 1 -- 172.21.15.72:0/855459483 >> 172.21.15.72:0/855459483 conn(0x7f5ac80792d0 msgr2=0x7f5ac81007f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:54:20.209 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:20.208+0000 7f5ace800700 1 -- 172.21.15.72:0/855459483 shutdown_connections 2024-04-26T21:54:20.209 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:20.208+0000 7f5ace800700 1 -- 172.21.15.72:0/855459483 wait complete. 2024-04-26T21:54:20.515 INFO:tasks.cephadm.ceph_manager.ceph:need seq 223338299447 got 223338299453 for osd.7 2024-04-26T21:54:20.515 DEBUG:teuthology.parallel:result is None 2024-04-26T21:54:21.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:54:20 smithi072 ceph-mon[28138]: pgmap v416: 97 pgs: 97 active+clean; 633 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 114 B/s wr, 10 op/s; 69 B/s, 0 keys/s, 0 objects/s recovering 2024-04-26T21:54:21.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:54:20 smithi072 ceph-mon[28138]: from='client.? 172.21.15.72:0/855459483' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-26T21:54:21.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:54:20 smithi164 ceph-mon[31916]: pgmap v416: 97 pgs: 97 active+clean; 633 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 114 B/s wr, 10 op/s; 69 B/s, 0 keys/s, 0 objects/s recovering 2024-04-26T21:54:21.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:54:20 smithi164 ceph-mon[31916]: from='client.? 172.21.15.72:0/855459483' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-26T21:54:21.223 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738473 got 34359738479 for osd.0 2024-04-26T21:54:21.224 DEBUG:teuthology.parallel:result is None 2024-04-26T21:54:21.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:54:20 smithi076 ceph-mon[31823]: pgmap v416: 97 pgs: 97 active+clean; 633 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 114 B/s wr, 10 op/s; 69 B/s, 0 keys/s, 0 objects/s recovering 2024-04-26T21:54:21.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:54:20 smithi076 ceph-mon[31823]: from='client.? 172.21.15.72:0/855459483' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-26T21:54:21.844 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574945 got 55834574951 for osd.1 2024-04-26T21:54:21.845 DEBUG:teuthology.parallel:result is None 2024-04-26T21:54:21.845 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-04-26T21:54:21.845 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid a1780950-0415-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-26T21:54:22.284 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/a1780950-0415-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:54:23.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:54:22 smithi072 ceph-mon[28138]: pgmap v417: 97 pgs: 97 active+clean; 633 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 102 B/s wr, 9 op/s; 62 B/s, 0 keys/s, 0 objects/s recovering 2024-04-26T21:54:23.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:54:22 smithi076 ceph-mon[31823]: pgmap v417: 97 pgs: 97 active+clean; 633 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 102 B/s wr, 9 op/s; 62 B/s, 0 keys/s, 0 objects/s recovering 2024-04-26T21:54:23.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:54:22 smithi164 ceph-mon[31916]: pgmap v417: 97 pgs: 97 active+clean; 633 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 102 B/s wr, 9 op/s; 62 B/s, 0 keys/s, 0 objects/s recovering 2024-04-26T21:54:24.483 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:24.483+0000 7f1c4bc00700 1 -- 172.21.15.72:0/2049662848 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1c44103e30 msgr2=0x7f1c44110290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:24.484 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:24.483+0000 7f1c4bc00700 1 --2- 172.21.15.72:0/2049662848 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1c44103e30 0x7f1c44110290 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f1c3c00b0a0 tx=0x7f1c3c00b3b0 comp rx=0 tx=0).stop 2024-04-26T21:54:24.484 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:24.483+0000 7f1c4bc00700 1 -- 172.21.15.72:0/2049662848 shutdown_connections 2024-04-26T21:54:24.484 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:24.483+0000 7f1c4bc00700 1 --2- 172.21.15.72:0/2049662848 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1c44103e30 0x7f1c44110290 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:24.484 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:24.483+0000 7f1c4bc00700 1 --2- 172.21.15.72:0/2049662848 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1c441094c0 0x7f1c441098a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:24.484 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:24.483+0000 7f1c4bc00700 1 --2- 172.21.15.72:0/2049662848 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1c44103470 0x7f1c441038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:24.484 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:24.483+0000 7f1c4bc00700 1 -- 172.21.15.72:0/2049662848 >> 172.21.15.72:0/2049662848 conn(0x7f1c4407a580 msgr2=0x7f1c4407a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:54:24.484 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:24.483+0000 7f1c4bc00700 1 -- 172.21.15.72:0/2049662848 shutdown_connections 2024-04-26T21:54:24.484 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:24.483+0000 7f1c4bc00700 1 -- 172.21.15.72:0/2049662848 wait complete. 2024-04-26T21:54:24.487 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:24.484+0000 7f1c4bc00700 1 Processor -- start 2024-04-26T21:54:24.487 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:24.484+0000 7f1c4bc00700 1 -- start start 2024-04-26T21:54:24.487 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:24.485+0000 7f1c4bc00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1c44103470 0x7f1c441a1850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:24.487 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:24.485+0000 7f1c4bc00700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1c44103e30 0x7f1c441a1d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:24.487 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:24.485+0000 7f1c4bc00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1c441094c0 0x7f1c4419ba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:24.487 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:24.485+0000 7f1c4bc00700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f1c44113540 con 0x7f1c44103470 2024-04-26T21:54:24.487 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:24.485+0000 7f1c4bc00700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f1c441133c0 con 0x7f1c44103e30 2024-04-26T21:54:24.487 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:24.485+0000 7f1c4bc00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f1c441136c0 con 0x7f1c441094c0 2024-04-26T21:54:24.488 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:24.485+0000 7f1c49200700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1c44103470 0x7f1c441a1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:24.488 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:24.485+0000 7f1c49200700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1c44103470 0x7f1c441a1850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:32836/0 (socket says 172.21.15.72:32836) 2024-04-26T21:54:24.488 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:24.485+0000 7f1c49200700 1 -- 172.21.15.72:0/831964053 learned_addr learned my addr 172.21.15.72:0/831964053 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:54:24.488 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:24.485+0000 7f1c43e00700 1 --2- 172.21.15.72:0/831964053 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1c44103e30 0x7f1c441a1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:24.488 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:24.485+0000 7f1c49c00700 1 --2- 172.21.15.72:0/831964053 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1c441094c0 0x7f1c4419ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:24.488 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:24.485+0000 7f1c49200700 1 -- 172.21.15.72:0/831964053 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1c441094c0 msgr2=0x7f1c4419ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:24.488 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:24.485+0000 7f1c49200700 1 --2- 172.21.15.72:0/831964053 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1c441094c0 0x7f1c4419ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:24.488 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:24.485+0000 7f1c49200700 1 -- 172.21.15.72:0/831964053 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1c44103e30 msgr2=0x7f1c441a1d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:24.488 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:24.485+0000 7f1c49200700 1 --2- 172.21.15.72:0/831964053 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1c44103e30 0x7f1c441a1d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:24.488 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:24.485+0000 7f1c49200700 1 -- 172.21.15.72:0/831964053 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c4419c170 con 0x7f1c44103470 2024-04-26T21:54:24.488 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:24.485+0000 7f1c49c00700 1 --2- 172.21.15.72:0/831964053 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1c441094c0 0x7f1c4419ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:54:24.489 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:24.486+0000 7f1c49200700 1 --2- 172.21.15.72:0/831964053 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1c44103470 0x7f1c441a1850 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f1c3800c9b0 tx=0x7f1c3800cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:54:24.489 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:24.486+0000 7f1c41600700 1 -- 172.21.15.72:0/831964053 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c38007b00 con 0x7f1c44103470 2024-04-26T21:54:24.489 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:24.486+0000 7f1c41600700 1 -- 172.21.15.72:0/831964053 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1c38007ca0 con 0x7f1c44103470 2024-04-26T21:54:24.489 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:24.486+0000 7f1c4bc00700 1 -- 172.21.15.72:0/831964053 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c4419c400 con 0x7f1c44103470 2024-04-26T21:54:24.489 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:24.486+0000 7f1c41600700 1 -- 172.21.15.72:0/831964053 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c38018660 con 0x7f1c44103470 2024-04-26T21:54:24.489 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:24.486+0000 7f1c4bc00700 1 -- 172.21.15.72:0/831964053 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c441a8720 con 0x7f1c44103470 2024-04-26T21:54:24.490 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:24.488+0000 7f1c4bc00700 1 -- 172.21.15.72:0/831964053 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c4406a110 con 0x7f1c44103470 2024-04-26T21:54:24.490 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:24.490+0000 7f1c41600700 1 -- 172.21.15.72:0/831964053 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f1c3800f450 con 0x7f1c44103470 2024-04-26T21:54:24.492 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:24.492+0000 7f1c41600700 1 --2- 172.21.15.72:0/831964053 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f1c2c0a3d50 0x7f1c2c0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:24.492 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:24.492+0000 7f1c41600700 1 -- 172.21.15.72:0/831964053 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7f1c380d06a0 con 0x7f1c44103470 2024-04-26T21:54:24.492 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:24.492+0000 7f1c43e00700 1 --2- 172.21.15.72:0/831964053 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f1c2c0a3d50 0x7f1c2c0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:24.493 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:24.493+0000 7f1c43e00700 1 --2- 172.21.15.72:0/831964053 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f1c2c0a3d50 0x7f1c2c0a6210 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f1c30006fd0 tx=0x7f1c30008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:54:24.496 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:24.496+0000 7f1c41600700 1 -- 172.21.15.72:0/831964053 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f1c38014030 con 0x7f1c44103470 2024-04-26T21:54:24.723 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:24.723+0000 7f1c4bc00700 1 -- 172.21.15.72:0/831964053 --> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f1c44102610 con 0x7f1c2c0a3d50 2024-04-26T21:54:24.740 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:24.740+0000 7f1c41600700 1 -- 172.21.15.72:0/831964053 <== mgr.14152 v2:172.21.15.72:6800/588150172 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258775 (secure 0 0 0) 0x7f1c44102610 con 0x7f1c2c0a3d50 2024-04-26T21:54:24.745 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T21:54:24.748 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:24.745+0000 7f1c4bc00700 1 -- 172.21.15.72:0/831964053 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f1c2c0a3d50 msgr2=0x7f1c2c0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:24.748 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:24.745+0000 7f1c4bc00700 1 --2- 172.21.15.72:0/831964053 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f1c2c0a3d50 0x7f1c2c0a6210 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f1c30006fd0 tx=0x7f1c30008040 comp rx=0 tx=0).stop 2024-04-26T21:54:24.748 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:24.745+0000 7f1c4bc00700 1 -- 172.21.15.72:0/831964053 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1c44103470 msgr2=0x7f1c441a1850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:24.748 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:24.745+0000 7f1c4bc00700 1 --2- 172.21.15.72:0/831964053 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1c44103470 0x7f1c441a1850 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f1c3800c9b0 tx=0x7f1c3800cd70 comp rx=0 tx=0).stop 2024-04-26T21:54:24.748 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:24.745+0000 7f1c4bc00700 1 -- 172.21.15.72:0/831964053 shutdown_connections 2024-04-26T21:54:24.748 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:24.745+0000 7f1c4bc00700 1 --2- 172.21.15.72:0/831964053 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f1c2c0a3d50 0x7f1c2c0a6210 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:24.748 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:24.745+0000 7f1c4bc00700 1 --2- 172.21.15.72:0/831964053 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1c44103470 0x7f1c441a1850 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:24.748 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:24.745+0000 7f1c4bc00700 1 --2- 172.21.15.72:0/831964053 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1c441094c0 0x7f1c4419ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:24.748 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:24.745+0000 7f1c4bc00700 1 --2- 172.21.15.72:0/831964053 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1c44103e30 0x7f1c441a1d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:24.748 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:24.745+0000 7f1c4bc00700 1 -- 172.21.15.72:0/831964053 >> 172.21.15.72:0/831964053 conn(0x7f1c4407a580 msgr2=0x7f1c44101f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:54:24.748 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:24.745+0000 7f1c4bc00700 1 -- 172.21.15.72:0/831964053 shutdown_connections 2024-04-26T21:54:24.748 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:24.745+0000 7f1c4bc00700 1 -- 172.21.15.72:0/831964053 wait complete. 2024-04-26T21:54:24.748 INFO:teuthology.orchestra.run.smithi072.stderr:dumped all 2024-04-26T21:54:25.071 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:54:24 smithi072 ceph-mon[28138]: pgmap v418: 97 pgs: 97 active+clean; 633 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 86 B/s wr, 8 op/s; 52 B/s, 0 keys/s, 0 objects/s recovering 2024-04-26T21:54:25.071 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:54:24 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:54:25.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:54:24 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:54:25.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:54:24 smithi076 ceph-mon[31823]: pgmap v418: 97 pgs: 97 active+clean; 633 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 86 B/s wr, 8 op/s; 52 B/s, 0 keys/s, 0 objects/s recovering 2024-04-26T21:54:25.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:54:24 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:54:25.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:54:24 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:54:25.442 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:54:24 smithi164 ceph-mon[31916]: pgmap v418: 97 pgs: 97 active+clean; 633 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 86 B/s wr, 8 op/s; 52 B/s, 0 keys/s, 0 objects/s recovering 2024-04-26T21:54:25.442 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:54:24 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:54:25.442 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:54:24 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:54:25.532 INFO:teuthology.orchestra.run.smithi072.stdout:{"pg_ready":true,"pg_map":{"version":418,"stamp":"2024-04-26T21:54:23.809410+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":648366,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":975,"num_read_kb":595,"num_write":420,"num_write_kb":3063,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":1181358,"num_keys_recovered":7,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":343,"ondisk_log_size":343,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":396556,"kb_used_data":14052,"kb_used_omap":18,"kb_used_meta":382317,"kb_avail":1124545268,"statfs":{"total":1151940427776,"available":1151534354432,"internally_reserved":0,"allocated":14389248,"data_stored":7744938,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":391492922},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":104,"num_read_kb":48,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":622,"num_keys_recovered":7,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.906376"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.921345+0000","last_change":"2024-04-26T21:52:50.787765+0000","last_active":"2024-04-26T21:54:11.921345+0000","last_peered":"2024-04-26T21:54:11.921345+0000","last_clean":"2024-04-26T21:54:11.921345+0000","last_became_active":"2024-04-26T21:52:50.786969+0000","last_became_peered":"2024-04-26T21:52:50.786969+0000","last_unstale":"2024-04-26T21:54:11.921345+0000","last_undegraded":"2024-04-26T21:54:11.921345+0000","last_fullsized":"2024-04-26T21:54:11.921345+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:31:50.659210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:12.034215+0000","last_change":"2024-04-26T21:52:50.779735+0000","last_active":"2024-04-26T21:54:12.034215+0000","last_peered":"2024-04-26T21:54:12.034215+0000","last_clean":"2024-04-26T21:54:12.034215+0000","last_became_active":"2024-04-26T21:52:50.779358+0000","last_became_peered":"2024-04-26T21:52:50.779358+0000","last_unstale":"2024-04-26T21:54:12.034215+0000","last_undegraded":"2024-04-26T21:54:12.034215+0000","last_fullsized":"2024-04-26T21:54:12.034215+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:58:45.631365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:12.033834+0000","last_change":"2024-04-26T21:52:50.788220+0000","last_active":"2024-04-26T21:54:12.033834+0000","last_peered":"2024-04-26T21:54:12.033834+0000","last_clean":"2024-04-26T21:54:12.033834+0000","last_became_active":"2024-04-26T21:52:50.787282+0000","last_became_peered":"2024-04-26T21:52:50.787282+0000","last_unstale":"2024-04-26T21:54:12.033834+0000","last_undegraded":"2024-04-26T21:54:12.033834+0000","last_fullsized":"2024-04-26T21:54:12.033834+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:12:28.577893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.934391+0000","last_change":"2024-04-26T21:52:50.795367+0000","last_active":"2024-04-26T21:54:11.934391+0000","last_peered":"2024-04-26T21:54:11.934391+0000","last_clean":"2024-04-26T21:54:11.934391+0000","last_became_active":"2024-04-26T21:52:50.793602+0000","last_became_peered":"2024-04-26T21:52:50.793602+0000","last_unstale":"2024-04-26T21:54:11.934391+0000","last_undegraded":"2024-04-26T21:54:11.934391+0000","last_fullsized":"2024-04-26T21:54:11.934391+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:37:23.356792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.912047+0000","last_change":"2024-04-26T21:52:50.787333+0000","last_active":"2024-04-26T21:54:11.912047+0000","last_peered":"2024-04-26T21:54:11.912047+0000","last_clean":"2024-04-26T21:54:11.912047+0000","last_became_active":"2024-04-26T21:52:50.785319+0000","last_became_peered":"2024-04-26T21:52:50.785319+0000","last_unstale":"2024-04-26T21:54:11.912047+0000","last_undegraded":"2024-04-26T21:54:11.912047+0000","last_fullsized":"2024-04-26T21:54:11.912047+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:06:29.154825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:12.396920+0000","last_change":"2024-04-26T21:52:50.780829+0000","last_active":"2024-04-26T21:54:12.396920+0000","last_peered":"2024-04-26T21:54:12.396920+0000","last_clean":"2024-04-26T21:54:12.396920+0000","last_became_active":"2024-04-26T21:52:50.780439+0000","last_became_peered":"2024-04-26T21:52:50.780439+0000","last_unstale":"2024-04-26T21:54:12.396920+0000","last_undegraded":"2024-04-26T21:54:12.396920+0000","last_fullsized":"2024-04-26T21:54:12.396920+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:20:14.533853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.922082+0000","last_change":"2024-04-26T21:52:50.787679+0000","last_active":"2024-04-26T21:54:11.922082+0000","last_peered":"2024-04-26T21:54:11.922082+0000","last_clean":"2024-04-26T21:54:11.922082+0000","last_became_active":"2024-04-26T21:52:50.787277+0000","last_became_peered":"2024-04-26T21:52:50.787277+0000","last_unstale":"2024-04-26T21:54:11.922082+0000","last_undegraded":"2024-04-26T21:54:11.922082+0000","last_fullsized":"2024-04-26T21:54:11.922082+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:29:02.131503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":15,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.905091+0000","last_change":"2024-04-26T21:53:10.927384+0000","last_active":"2024-04-26T21:54:11.905091+0000","last_peered":"2024-04-26T21:54:11.905091+0000","last_clean":"2024-04-26T21:54:11.905091+0000","last_became_active":"2024-04-26T21:53:10.926996+0000","last_became_peered":"2024-04-26T21:53:10.926996+0000","last_unstale":"2024-04-26T21:54:11.905091+0000","last_undegraded":"2024-04-26T21:54:11.905091+0000","last_fullsized":"2024-04-26T21:54:11.905091+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:25:10.174532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.909424+0000","last_change":"2024-04-26T21:52:50.779701+0000","last_active":"2024-04-26T21:54:11.909424+0000","last_peered":"2024-04-26T21:54:11.909424+0000","last_clean":"2024-04-26T21:54:11.909424+0000","last_became_active":"2024-04-26T21:52:50.779331+0000","last_became_peered":"2024-04-26T21:52:50.779331+0000","last_unstale":"2024-04-26T21:54:11.909424+0000","last_undegraded":"2024-04-26T21:54:11.909424+0000","last_fullsized":"2024-04-26T21:54:11.909424+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:38:15.569806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:12.397805+0000","last_change":"2024-04-26T21:52:50.790081+0000","last_active":"2024-04-26T21:54:12.397805+0000","last_peered":"2024-04-26T21:54:12.397805+0000","last_clean":"2024-04-26T21:54:12.397805+0000","last_became_active":"2024-04-26T21:52:50.786575+0000","last_became_peered":"2024-04-26T21:52:50.786575+0000","last_unstale":"2024-04-26T21:54:12.397805+0000","last_undegraded":"2024-04-26T21:54:12.397805+0000","last_fullsized":"2024-04-26T21:54:12.397805+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:56:47.467128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.922863+0000","last_change":"2024-04-26T21:52:50.791983+0000","last_active":"2024-04-26T21:54:11.922863+0000","last_peered":"2024-04-26T21:54:11.922863+0000","last_clean":"2024-04-26T21:54:11.922863+0000","last_became_active":"2024-04-26T21:52:50.788312+0000","last_became_peered":"2024-04-26T21:52:50.788312+0000","last_unstale":"2024-04-26T21:54:11.922863+0000","last_undegraded":"2024-04-26T21:54:11.922863+0000","last_fullsized":"2024-04-26T21:54:11.922863+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:47:47.410267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.934110+0000","last_change":"2024-04-26T21:52:50.794237+0000","last_active":"2024-04-26T21:54:11.934110+0000","last_peered":"2024-04-26T21:54:11.934110+0000","last_clean":"2024-04-26T21:54:11.934110+0000","last_became_active":"2024-04-26T21:52:50.792260+0000","last_became_peered":"2024-04-26T21:52:50.792260+0000","last_unstale":"2024-04-26T21:54:11.934110+0000","last_undegraded":"2024-04-26T21:54:11.934110+0000","last_fullsized":"2024-04-26T21:54:11.934110+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:36:05.467667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:12.396013+0000","last_change":"2024-04-26T21:52:50.777687+0000","last_active":"2024-04-26T21:54:12.396013+0000","last_peered":"2024-04-26T21:54:12.396013+0000","last_clean":"2024-04-26T21:54:12.396013+0000","last_became_active":"2024-04-26T21:52:50.777200+0000","last_became_peered":"2024-04-26T21:52:50.777200+0000","last_unstale":"2024-04-26T21:54:12.396013+0000","last_undegraded":"2024-04-26T21:54:12.396013+0000","last_fullsized":"2024-04-26T21:54:12.396013+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:40:45.167144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:12.033275+0000","last_change":"2024-04-26T21:52:50.787008+0000","last_active":"2024-04-26T21:54:12.033275+0000","last_peered":"2024-04-26T21:54:12.033275+0000","last_clean":"2024-04-26T21:54:12.033275+0000","last_became_active":"2024-04-26T21:52:50.785656+0000","last_became_peered":"2024-04-26T21:52:50.785656+0000","last_unstale":"2024-04-26T21:54:12.033275+0000","last_undegraded":"2024-04-26T21:54:12.033275+0000","last_fullsized":"2024-04-26T21:54:12.033275+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:20:12.441281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:12.033239+0000","last_change":"2024-04-26T21:52:50.796537+0000","last_active":"2024-04-26T21:54:12.033239+0000","last_peered":"2024-04-26T21:54:12.033239+0000","last_clean":"2024-04-26T21:54:12.033239+0000","last_became_active":"2024-04-26T21:52:50.796229+0000","last_became_peered":"2024-04-26T21:52:50.796229+0000","last_unstale":"2024-04-26T21:54:12.033239+0000","last_undegraded":"2024-04-26T21:54:12.033239+0000","last_fullsized":"2024-04-26T21:54:12.033239+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:38:46.404444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.908267+0000","last_change":"2024-04-26T21:52:50.789211+0000","last_active":"2024-04-26T21:54:11.908267+0000","last_peered":"2024-04-26T21:54:11.908267+0000","last_clean":"2024-04-26T21:54:11.908267+0000","last_became_active":"2024-04-26T21:52:50.788889+0000","last_became_peered":"2024-04-26T21:52:50.788889+0000","last_unstale":"2024-04-26T21:54:11.908267+0000","last_undegraded":"2024-04-26T21:54:11.908267+0000","last_fullsized":"2024-04-26T21:54:11.908267+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:25:48.497840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.921414+0000","last_change":"2024-04-26T21:52:50.788035+0000","last_active":"2024-04-26T21:54:11.921414+0000","last_peered":"2024-04-26T21:54:11.921414+0000","last_clean":"2024-04-26T21:54:11.921414+0000","last_became_active":"2024-04-26T21:52:50.787453+0000","last_became_peered":"2024-04-26T21:52:50.787453+0000","last_unstale":"2024-04-26T21:54:11.921414+0000","last_undegraded":"2024-04-26T21:54:11.921414+0000","last_fullsized":"2024-04-26T21:54:11.921414+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:40:29.718537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.905521+0000","last_change":"2024-04-26T21:52:50.778228+0000","last_active":"2024-04-26T21:54:11.905521+0000","last_peered":"2024-04-26T21:54:11.905521+0000","last_clean":"2024-04-26T21:54:11.905521+0000","last_became_active":"2024-04-26T21:52:50.777934+0000","last_became_peered":"2024-04-26T21:52:50.777934+0000","last_unstale":"2024-04-26T21:54:11.905521+0000","last_undegraded":"2024-04-26T21:54:11.905521+0000","last_fullsized":"2024-04-26T21:54:11.905521+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:33:34.598325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.909087+0000","last_change":"2024-04-26T21:52:50.790795+0000","last_active":"2024-04-26T21:54:11.909087+0000","last_peered":"2024-04-26T21:54:11.909087+0000","last_clean":"2024-04-26T21:54:11.909087+0000","last_became_active":"2024-04-26T21:52:50.788254+0000","last_became_peered":"2024-04-26T21:52:50.788254+0000","last_unstale":"2024-04-26T21:54:11.909087+0000","last_undegraded":"2024-04-26T21:54:11.909087+0000","last_fullsized":"2024-04-26T21:54:11.909087+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:20:28.235137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"87'13","reported_seq":118,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:17.756475+0000","last_change":"2024-04-26T21:52:50.105513+0000","last_active":"2024-04-26T21:54:17.756475+0000","last_peered":"2024-04-26T21:54:17.756475+0000","last_clean":"2024-04-26T21:54:17.756475+0000","last_became_active":"2024-04-26T21:52:50.105131+0000","last_became_peered":"2024-04-26T21:52:50.105131+0000","last_unstale":"2024-04-26T21:54:17.756475+0000","last_undegraded":"2024-04-26T21:54:17.756475+0000","last_fullsized":"2024-04-26T21:54:17.756475+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:36:58.085803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":82,"num_read_kb":82,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.908266+0000","last_change":"2024-04-26T21:52:50.795025+0000","last_active":"2024-04-26T21:54:11.908266+0000","last_peered":"2024-04-26T21:54:11.908266+0000","last_clean":"2024-04-26T21:54:11.908266+0000","last_became_active":"2024-04-26T21:52:50.793608+0000","last_became_peered":"2024-04-26T21:52:50.793608+0000","last_unstale":"2024-04-26T21:54:11.908266+0000","last_undegraded":"2024-04-26T21:54:11.908266+0000","last_fullsized":"2024-04-26T21:54:11.908266+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:49:47.764848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"87'2","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.908318+0000","last_change":"2024-04-26T21:52:50.109197+0000","last_active":"2024-04-26T21:54:11.908318+0000","last_peered":"2024-04-26T21:54:11.908318+0000","last_clean":"2024-04-26T21:54:11.908318+0000","last_became_active":"2024-04-26T21:52:50.108835+0000","last_became_peered":"2024-04-26T21:52:50.108835+0000","last_unstale":"2024-04-26T21:54:11.908318+0000","last_undegraded":"2024-04-26T21:54:11.908318+0000","last_fullsized":"2024-04-26T21:54:11.908318+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:45:09.777747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:12.034355+0000","last_change":"2024-04-26T21:52:50.783094+0000","last_active":"2024-04-26T21:54:12.034355+0000","last_peered":"2024-04-26T21:54:12.034355+0000","last_clean":"2024-04-26T21:54:12.034355+0000","last_became_active":"2024-04-26T21:52:50.782468+0000","last_became_peered":"2024-04-26T21:52:50.782468+0000","last_unstale":"2024-04-26T21:54:12.034355+0000","last_undegraded":"2024-04-26T21:54:12.034355+0000","last_fullsized":"2024-04-26T21:54:12.034355+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:53:12.977054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"87'12","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.913322+0000","last_change":"2024-04-26T21:52:50.104824+0000","last_active":"2024-04-26T21:54:11.913322+0000","last_peered":"2024-04-26T21:54:11.913322+0000","last_clean":"2024-04-26T21:54:11.913322+0000","last_became_active":"2024-04-26T21:52:50.104514+0000","last_became_peered":"2024-04-26T21:52:50.104514+0000","last_unstale":"2024-04-26T21:54:11.913322+0000","last_undegraded":"2024-04-26T21:54:11.913322+0000","last_fullsized":"2024-04-26T21:54:11.913322+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:45:31.541940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.909084+0000","last_change":"2024-04-26T21:52:50.779695+0000","last_active":"2024-04-26T21:54:11.909084+0000","last_peered":"2024-04-26T21:54:11.909084+0000","last_clean":"2024-04-26T21:54:11.909084+0000","last_became_active":"2024-04-26T21:52:50.779234+0000","last_became_peered":"2024-04-26T21:52:50.779234+0000","last_unstale":"2024-04-26T21:54:11.909084+0000","last_undegraded":"2024-04-26T21:54:11.909084+0000","last_fullsized":"2024-04-26T21:54:11.909084+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:33:07.316543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"87'5","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.912832+0000","last_change":"2024-04-26T21:52:50.107403+0000","last_active":"2024-04-26T21:54:11.912832+0000","last_peered":"2024-04-26T21:54:11.912832+0000","last_clean":"2024-04-26T21:54:11.912832+0000","last_became_active":"2024-04-26T21:52:50.107126+0000","last_became_peered":"2024-04-26T21:52:50.107126+0000","last_unstale":"2024-04-26T21:54:11.912832+0000","last_undegraded":"2024-04-26T21:54:11.912832+0000","last_fullsized":"2024-04-26T21:54:11.912832+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:52:34.985448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.916814+0000","last_change":"2024-04-26T21:52:50.783674+0000","last_active":"2024-04-26T21:54:11.916814+0000","last_peered":"2024-04-26T21:54:11.916814+0000","last_clean":"2024-04-26T21:54:11.916814+0000","last_became_active":"2024-04-26T21:52:50.782283+0000","last_became_peered":"2024-04-26T21:52:50.782283+0000","last_unstale":"2024-04-26T21:54:11.916814+0000","last_undegraded":"2024-04-26T21:54:11.916814+0000","last_fullsized":"2024-04-26T21:54:11.916814+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:41:03.144601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"87'5","reported_seq":126,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:20.968685+0000","last_change":"2024-04-26T21:54:11.949461+0000","last_active":"2024-04-26T21:54:20.968685+0000","last_peered":"2024-04-26T21:54:20.968685+0000","last_clean":"2024-04-26T21:54:20.968685+0000","last_became_active":"2024-04-26T21:54:11.936983+0000","last_became_peered":"2024-04-26T21:54:11.936983+0000","last_unstale":"2024-04-26T21:54:20.968685+0000","last_undegraded":"2024-04-26T21:54:20.968685+0000","last_fullsized":"2024-04-26T21:54:20.968685+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:36:40.757999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":85,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":622,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.912191+0000","last_change":"2024-04-26T21:53:11.575919+0000","last_active":"2024-04-26T21:54:11.912191+0000","last_peered":"2024-04-26T21:54:11.912191+0000","last_clean":"2024-04-26T21:54:11.912191+0000","last_became_active":"2024-04-26T21:53:11.574898+0000","last_became_peered":"2024-04-26T21:53:11.574898+0000","last_unstale":"2024-04-26T21:54:11.912191+0000","last_undegraded":"2024-04-26T21:54:11.912191+0000","last_fullsized":"2024-04-26T21:54:11.912191+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:44:16.323430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,7],"acting":[8,9,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"87'7","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:12.396707+0000","last_change":"2024-04-26T21:53:10.938377+0000","last_active":"2024-04-26T21:54:12.396707+0000","last_peered":"2024-04-26T21:54:12.396707+0000","last_clean":"2024-04-26T21:54:12.396707+0000","last_became_active":"2024-04-26T21:53:10.921998+0000","last_became_peered":"2024-04-26T21:53:10.921998+0000","last_unstale":"2024-04-26T21:54:12.396707+0000","last_undegraded":"2024-04-26T21:54:12.396707+0000","last_fullsized":"2024-04-26T21:54:12.396707+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:51:20.389344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,6],"acting":[10,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.918891+0000","last_change":"2024-04-26T21:53:10.931444+0000","last_active":"2024-04-26T21:54:11.918891+0000","last_peered":"2024-04-26T21:54:11.918891+0000","last_clean":"2024-04-26T21:54:11.918891+0000","last_became_active":"2024-04-26T21:53:10.930761+0000","last_became_peered":"2024-04-26T21:53:10.930761+0000","last_unstale":"2024-04-26T21:54:11.918891+0000","last_undegraded":"2024-04-26T21:54:11.918891+0000","last_fullsized":"2024-04-26T21:54:11.918891+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:56:17.803811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,3],"acting":[5,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"87'3","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:12.395172+0000","last_change":"2024-04-26T21:52:50.107564+0000","last_active":"2024-04-26T21:54:12.395172+0000","last_peered":"2024-04-26T21:54:12.395172+0000","last_clean":"2024-04-26T21:54:12.395172+0000","last_became_active":"2024-04-26T21:52:50.106653+0000","last_became_peered":"2024-04-26T21:52:50.106653+0000","last_unstale":"2024-04-26T21:54:12.395172+0000","last_undegraded":"2024-04-26T21:54:12.395172+0000","last_fullsized":"2024-04-26T21:54:12.395172+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:05:36.594729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.907464+0000","last_change":"2024-04-26T21:52:50.780279+0000","last_active":"2024-04-26T21:54:11.907464+0000","last_peered":"2024-04-26T21:54:11.907464+0000","last_clean":"2024-04-26T21:54:11.907464+0000","last_became_active":"2024-04-26T21:52:50.779950+0000","last_became_peered":"2024-04-26T21:52:50.779950+0000","last_unstale":"2024-04-26T21:54:11.907464+0000","last_undegraded":"2024-04-26T21:54:11.907464+0000","last_fullsized":"2024-04-26T21:54:11.907464+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:11:36.282319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"87'3","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.917399+0000","last_change":"2024-04-26T21:52:50.106269+0000","last_active":"2024-04-26T21:54:11.917399+0000","last_peered":"2024-04-26T21:54:11.917399+0000","last_clean":"2024-04-26T21:54:11.917399+0000","last_became_active":"2024-04-26T21:52:50.105948+0000","last_became_peered":"2024-04-26T21:52:50.105948+0000","last_unstale":"2024-04-26T21:54:11.917399+0000","last_undegraded":"2024-04-26T21:54:11.917399+0000","last_fullsized":"2024-04-26T21:54:11.917399+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:39:31.024362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.905248+0000","last_change":"2024-04-26T21:52:50.778938+0000","last_active":"2024-04-26T21:54:11.905248+0000","last_peered":"2024-04-26T21:54:11.905248+0000","last_clean":"2024-04-26T21:54:11.905248+0000","last_became_active":"2024-04-26T21:52:50.778590+0000","last_became_peered":"2024-04-26T21:52:50.778590+0000","last_unstale":"2024-04-26T21:54:11.905248+0000","last_undegraded":"2024-04-26T21:54:11.905248+0000","last_fullsized":"2024-04-26T21:54:11.905248+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:03:10.325793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"87'4","reported_seq":188,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:17.959449+0000","last_change":"2024-04-26T21:52:50.107243+0000","last_active":"2024-04-26T21:54:17.959449+0000","last_peered":"2024-04-26T21:54:17.959449+0000","last_clean":"2024-04-26T21:54:17.959449+0000","last_became_active":"2024-04-26T21:52:50.106729+0000","last_became_peered":"2024-04-26T21:52:50.106729+0000","last_unstale":"2024-04-26T21:54:17.959449+0000","last_undegraded":"2024-04-26T21:54:17.959449+0000","last_fullsized":"2024-04-26T21:54:17.959449+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:28:11.197997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":161,"num_read_kb":161,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.909218+0000","last_change":"2024-04-26T21:52:50.776906+0000","last_active":"2024-04-26T21:54:11.909218+0000","last_peered":"2024-04-26T21:54:11.909218+0000","last_clean":"2024-04-26T21:54:11.909218+0000","last_became_active":"2024-04-26T21:52:50.776567+0000","last_became_peered":"2024-04-26T21:52:50.776567+0000","last_unstale":"2024-04-26T21:54:11.909218+0000","last_undegraded":"2024-04-26T21:54:11.909218+0000","last_fullsized":"2024-04-26T21:54:11.909218+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:51:39.512572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"87'4","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.923209+0000","last_change":"2024-04-26T21:52:50.109792+0000","last_active":"2024-04-26T21:54:11.923209+0000","last_peered":"2024-04-26T21:54:11.923209+0000","last_clean":"2024-04-26T21:54:11.923209+0000","last_became_active":"2024-04-26T21:52:50.106662+0000","last_became_peered":"2024-04-26T21:52:50.106662+0000","last_unstale":"2024-04-26T21:54:11.923209+0000","last_undegraded":"2024-04-26T21:54:11.923209+0000","last_fullsized":"2024-04-26T21:54:11.923209+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:41:27.390780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:12.034290+0000","last_change":"2024-04-26T21:52:50.782798+0000","last_active":"2024-04-26T21:54:12.034290+0000","last_peered":"2024-04-26T21:54:12.034290+0000","last_clean":"2024-04-26T21:54:12.034290+0000","last_became_active":"2024-04-26T21:52:50.781975+0000","last_became_peered":"2024-04-26T21:52:50.781975+0000","last_unstale":"2024-04-26T21:54:12.034290+0000","last_undegraded":"2024-04-26T21:54:12.034290+0000","last_fullsized":"2024-04-26T21:54:12.034290+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:13:07.261095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"87'5","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.917071+0000","last_change":"2024-04-26T21:52:50.099296+0000","last_active":"2024-04-26T21:54:11.917071+0000","last_peered":"2024-04-26T21:54:11.917071+0000","last_clean":"2024-04-26T21:54:11.917071+0000","last_became_active":"2024-04-26T21:52:50.098940+0000","last_became_peered":"2024-04-26T21:52:50.098940+0000","last_unstale":"2024-04-26T21:54:11.917071+0000","last_undegraded":"2024-04-26T21:54:11.917071+0000","last_fullsized":"2024-04-26T21:54:11.917071+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:49:15.765474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.909449+0000","last_change":"2024-04-26T21:52:50.783341+0000","last_active":"2024-04-26T21:54:11.909449+0000","last_peered":"2024-04-26T21:54:11.909449+0000","last_clean":"2024-04-26T21:54:11.909449+0000","last_became_active":"2024-04-26T21:52:50.783025+0000","last_became_peered":"2024-04-26T21:52:50.783025+0000","last_unstale":"2024-04-26T21:54:11.909449+0000","last_undegraded":"2024-04-26T21:54:11.909449+0000","last_fullsized":"2024-04-26T21:54:11.909449+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:54:06.070550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"89'29","reported_seq":218,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:17.957203+0000","last_change":"2024-04-26T21:52:50.106617+0000","last_active":"2024-04-26T21:54:17.957203+0000","last_peered":"2024-04-26T21:54:17.957203+0000","last_clean":"2024-04-26T21:54:17.957203+0000","last_became_active":"2024-04-26T21:52:50.106161+0000","last_became_peered":"2024-04-26T21:52:50.106161+0000","last_unstale":"2024-04-26T21:54:17.957203+0000","last_undegraded":"2024-04-26T21:54:17.957203+0000","last_fullsized":"2024-04-26T21:54:17.957203+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:52:07.977749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":50951,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":166,"num_read_kb":133,"num_write":31,"num_write_kb":59,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.912822+0000","last_change":"2024-04-26T21:52:50.794567+0000","last_active":"2024-04-26T21:54:11.912822+0000","last_peered":"2024-04-26T21:54:11.912822+0000","last_clean":"2024-04-26T21:54:11.912822+0000","last_became_active":"2024-04-26T21:52:50.794245+0000","last_became_peered":"2024-04-26T21:52:50.794245+0000","last_unstale":"2024-04-26T21:54:11.912822+0000","last_undegraded":"2024-04-26T21:54:11.912822+0000","last_fullsized":"2024-04-26T21:54:11.912822+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:58:11.501104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"87'9","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.910158+0000","last_change":"2024-04-26T21:52:50.102156+0000","last_active":"2024-04-26T21:54:11.910158+0000","last_peered":"2024-04-26T21:54:11.910158+0000","last_clean":"2024-04-26T21:54:11.910158+0000","last_became_active":"2024-04-26T21:52:50.101721+0000","last_became_peered":"2024-04-26T21:52:50.101721+0000","last_unstale":"2024-04-26T21:54:11.910158+0000","last_undegraded":"2024-04-26T21:54:11.910158+0000","last_fullsized":"2024-04-26T21:54:11.910158+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:48:42.081874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:12.395687+0000","last_change":"2024-04-26T21:52:50.789199+0000","last_active":"2024-04-26T21:54:12.395687+0000","last_peered":"2024-04-26T21:54:12.395687+0000","last_clean":"2024-04-26T21:54:12.395687+0000","last_became_active":"2024-04-26T21:52:50.788868+0000","last_became_peered":"2024-04-26T21:52:50.788868+0000","last_unstale":"2024-04-26T21:54:12.395687+0000","last_undegraded":"2024-04-26T21:54:12.395687+0000","last_fullsized":"2024-04-26T21:54:12.395687+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:29:55.755564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"87'5","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.918157+0000","last_change":"2024-04-26T21:52:50.104920+0000","last_active":"2024-04-26T21:54:11.918157+0000","last_peered":"2024-04-26T21:54:11.918157+0000","last_clean":"2024-04-26T21:54:11.918157+0000","last_became_active":"2024-04-26T21:52:50.104579+0000","last_became_peered":"2024-04-26T21:52:50.104579+0000","last_unstale":"2024-04-26T21:54:11.918157+0000","last_undegraded":"2024-04-26T21:54:11.918157+0000","last_fullsized":"2024-04-26T21:54:11.918157+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:09:35.760918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.923266+0000","last_change":"2024-04-26T21:52:50.787016+0000","last_active":"2024-04-26T21:54:11.923266+0000","last_peered":"2024-04-26T21:54:11.923266+0000","last_clean":"2024-04-26T21:54:11.923266+0000","last_became_active":"2024-04-26T21:52:50.778439+0000","last_became_peered":"2024-04-26T21:52:50.778439+0000","last_unstale":"2024-04-26T21:54:11.923266+0000","last_undegraded":"2024-04-26T21:54:11.923266+0000","last_fullsized":"2024-04-26T21:54:11.923266+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:13:35.735480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"87'5","reported_seq":111,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:17.769328+0000","last_change":"2024-04-26T21:52:50.110175+0000","last_active":"2024-04-26T21:54:17.769328+0000","last_peered":"2024-04-26T21:54:17.769328+0000","last_clean":"2024-04-26T21:54:17.769328+0000","last_became_active":"2024-04-26T21:52:50.109779+0000","last_became_peered":"2024-04-26T21:52:50.109779+0000","last_unstale":"2024-04-26T21:54:17.769328+0000","last_undegraded":"2024-04-26T21:54:17.769328+0000","last_fullsized":"2024-04-26T21:54:17.769328+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:06:46.756327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":84,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.922431+0000","last_change":"2024-04-26T21:52:50.784199+0000","last_active":"2024-04-26T21:54:11.922431+0000","last_peered":"2024-04-26T21:54:11.922431+0000","last_clean":"2024-04-26T21:54:11.922431+0000","last_became_active":"2024-04-26T21:52:50.783820+0000","last_became_peered":"2024-04-26T21:52:50.783820+0000","last_unstale":"2024-04-26T21:54:11.922431+0000","last_undegraded":"2024-04-26T21:54:11.922431+0000","last_fullsized":"2024-04-26T21:54:11.922431+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:45:24.776722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"87'17","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.912583+0000","last_change":"2024-04-26T21:52:50.095685+0000","last_active":"2024-04-26T21:54:11.912583+0000","last_peered":"2024-04-26T21:54:11.912583+0000","last_clean":"2024-04-26T21:54:11.912583+0000","last_became_active":"2024-04-26T21:52:50.095298+0000","last_became_peered":"2024-04-26T21:52:50.095298+0000","last_unstale":"2024-04-26T21:54:11.912583+0000","last_undegraded":"2024-04-26T21:54:11.912583+0000","last_fullsized":"2024-04-26T21:54:11.912583+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:47:26.061531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.917725+0000","last_change":"2024-04-26T21:52:50.786607+0000","last_active":"2024-04-26T21:54:11.917725+0000","last_peered":"2024-04-26T21:54:11.917725+0000","last_clean":"2024-04-26T21:54:11.917725+0000","last_became_active":"2024-04-26T21:52:50.786261+0000","last_became_peered":"2024-04-26T21:52:50.786261+0000","last_unstale":"2024-04-26T21:54:11.917725+0000","last_undegraded":"2024-04-26T21:54:11.917725+0000","last_fullsized":"2024-04-26T21:54:11.917725+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:08:45.902260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"87'2","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.913774+0000","last_change":"2024-04-26T21:52:50.095782+0000","last_active":"2024-04-26T21:54:11.913774+0000","last_peered":"2024-04-26T21:54:11.913774+0000","last_clean":"2024-04-26T21:54:11.913774+0000","last_became_active":"2024-04-26T21:52:50.093983+0000","last_became_peered":"2024-04-26T21:52:50.093983+0000","last_unstale":"2024-04-26T21:54:11.913774+0000","last_undegraded":"2024-04-26T21:54:11.913774+0000","last_fullsized":"2024-04-26T21:54:11.913774+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:18:40.103099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.920696+0000","last_change":"2024-04-26T21:52:50.791969+0000","last_active":"2024-04-26T21:54:11.920696+0000","last_peered":"2024-04-26T21:54:11.920696+0000","last_clean":"2024-04-26T21:54:11.920696+0000","last_became_active":"2024-04-26T21:52:50.790441+0000","last_became_peered":"2024-04-26T21:52:50.790441+0000","last_unstale":"2024-04-26T21:54:11.920696+0000","last_undegraded":"2024-04-26T21:54:11.920696+0000","last_fullsized":"2024-04-26T21:54:11.920696+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:11:04.964439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"87'1","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.905358+0000","last_change":"2024-04-26T21:52:50.101807+0000","last_active":"2024-04-26T21:54:11.905358+0000","last_peered":"2024-04-26T21:54:11.905358+0000","last_clean":"2024-04-26T21:54:11.905358+0000","last_became_active":"2024-04-26T21:52:50.101329+0000","last_became_peered":"2024-04-26T21:52:50.101329+0000","last_unstale":"2024-04-26T21:54:11.905358+0000","last_undegraded":"2024-04-26T21:54:11.905358+0000","last_fullsized":"2024-04-26T21:54:11.905358+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:54:47.788172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"87'1","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:12.397539+0000","last_change":"2024-04-26T21:52:50.780872+0000","last_active":"2024-04-26T21:54:12.397539+0000","last_peered":"2024-04-26T21:54:12.397539+0000","last_clean":"2024-04-26T21:54:12.397539+0000","last_became_active":"2024-04-26T21:52:50.778757+0000","last_became_peered":"2024-04-26T21:52:50.778757+0000","last_unstale":"2024-04-26T21:54:12.397539+0000","last_undegraded":"2024-04-26T21:54:12.397539+0000","last_fullsized":"2024-04-26T21:54:12.397539+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:07:21.595091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"87'5","reported_seq":113,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:20.765421+0000","last_change":"2024-04-26T21:52:50.106886+0000","last_active":"2024-04-26T21:54:20.765421+0000","last_peered":"2024-04-26T21:54:20.765421+0000","last_clean":"2024-04-26T21:54:20.765421+0000","last_became_active":"2024-04-26T21:52:50.106150+0000","last_became_peered":"2024-04-26T21:52:50.106150+0000","last_unstale":"2024-04-26T21:54:20.765421+0000","last_undegraded":"2024-04-26T21:54:20.765421+0000","last_fullsized":"2024-04-26T21:54:20.765421+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:20:54.920416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":85,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:12.397142+0000","last_change":"2024-04-26T21:53:11.069186+0000","last_active":"2024-04-26T21:54:12.397142+0000","last_peered":"2024-04-26T21:54:12.397142+0000","last_clean":"2024-04-26T21:54:12.397142+0000","last_became_active":"2024-04-26T21:53:11.068418+0000","last_became_peered":"2024-04-26T21:53:11.068418+0000","last_unstale":"2024-04-26T21:54:12.397142+0000","last_undegraded":"2024-04-26T21:54:12.397142+0000","last_fullsized":"2024-04-26T21:54:12.397142+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:57:59.663173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.917667+0000","last_change":"2024-04-26T21:52:50.785257+0000","last_active":"2024-04-26T21:54:11.917667+0000","last_peered":"2024-04-26T21:54:11.917667+0000","last_clean":"2024-04-26T21:54:11.917667+0000","last_became_active":"2024-04-26T21:52:50.784906+0000","last_became_peered":"2024-04-26T21:52:50.784906+0000","last_unstale":"2024-04-26T21:54:11.917667+0000","last_undegraded":"2024-04-26T21:54:11.917667+0000","last_fullsized":"2024-04-26T21:54:11.917667+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:04:50.921663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"87'5","reported_seq":110,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:17.768042+0000","last_change":"2024-04-26T21:52:50.102425+0000","last_active":"2024-04-26T21:54:17.768042+0000","last_peered":"2024-04-26T21:54:17.768042+0000","last_clean":"2024-04-26T21:54:17.768042+0000","last_became_active":"2024-04-26T21:52:50.101703+0000","last_became_peered":"2024-04-26T21:52:50.101703+0000","last_unstale":"2024-04-26T21:54:17.768042+0000","last_undegraded":"2024-04-26T21:54:17.768042+0000","last_fullsized":"2024-04-26T21:54:17.768042+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:13:39.910519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":82,"num_read_kb":82,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.907705+0000","last_change":"2024-04-26T21:52:50.775036+0000","last_active":"2024-04-26T21:54:11.907705+0000","last_peered":"2024-04-26T21:54:11.907705+0000","last_clean":"2024-04-26T21:54:11.907705+0000","last_became_active":"2024-04-26T21:52:50.773347+0000","last_became_peered":"2024-04-26T21:52:50.773347+0000","last_unstale":"2024-04-26T21:54:11.907705+0000","last_undegraded":"2024-04-26T21:54:11.907705+0000","last_fullsized":"2024-04-26T21:54:11.907705+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:59:55.303014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"87'6","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.934145+0000","last_change":"2024-04-26T21:52:50.109437+0000","last_active":"2024-04-26T21:54:11.934145+0000","last_peered":"2024-04-26T21:54:11.934145+0000","last_clean":"2024-04-26T21:54:11.934145+0000","last_became_active":"2024-04-26T21:52:50.108605+0000","last_became_peered":"2024-04-26T21:52:50.108605+0000","last_unstale":"2024-04-26T21:54:11.934145+0000","last_undegraded":"2024-04-26T21:54:11.934145+0000","last_fullsized":"2024-04-26T21:54:11.934145+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:31:04.208334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":350,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.922023+0000","last_change":"2024-04-26T21:51:11.498577+0000","last_active":"2024-04-26T21:54:11.922023+0000","last_peered":"2024-04-26T21:54:11.922023+0000","last_clean":"2024-04-26T21:54:11.922023+0000","last_became_active":"2024-04-26T21:51:11.497422+0000","last_became_peered":"2024-04-26T21:51:11.497422+0000","last_unstale":"2024-04-26T21:54:11.922023+0000","last_undegraded":"2024-04-26T21:54:11.922023+0000","last_fullsized":"2024-04-26T21:54:11.922023+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:46:29.789400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:46:29.789400+0000","last_clean_scrub_stamp":"2024-04-26T21:46:29.789400+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:54:13.450359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:12.394977+0000","last_change":"2024-04-26T21:52:50.794270+0000","last_active":"2024-04-26T21:54:12.394977+0000","last_peered":"2024-04-26T21:54:12.394977+0000","last_clean":"2024-04-26T21:54:12.394977+0000","last_became_active":"2024-04-26T21:52:50.793966+0000","last_became_peered":"2024-04-26T21:52:50.793966+0000","last_unstale":"2024-04-26T21:54:12.394977+0000","last_undegraded":"2024-04-26T21:54:12.394977+0000","last_fullsized":"2024-04-26T21:54:12.394977+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:01:37.668466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"87'4","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.921875+0000","last_change":"2024-04-26T21:52:50.102700+0000","last_active":"2024-04-26T21:54:11.921875+0000","last_peered":"2024-04-26T21:54:11.921875+0000","last_clean":"2024-04-26T21:54:11.921875+0000","last_became_active":"2024-04-26T21:52:50.101267+0000","last_became_peered":"2024-04-26T21:52:50.101267+0000","last_unstale":"2024-04-26T21:54:11.921875+0000","last_undegraded":"2024-04-26T21:54:11.921875+0000","last_fullsized":"2024-04-26T21:54:11.921875+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:41:21.727411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"87'4","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.913859+0000","last_change":"2024-04-26T21:52:50.102191+0000","last_active":"2024-04-26T21:54:11.913859+0000","last_peered":"2024-04-26T21:54:11.913859+0000","last_clean":"2024-04-26T21:54:11.913859+0000","last_became_active":"2024-04-26T21:52:50.101307+0000","last_became_peered":"2024-04-26T21:52:50.101307+0000","last_unstale":"2024-04-26T21:54:11.913859+0000","last_undegraded":"2024-04-26T21:54:11.913859+0000","last_fullsized":"2024-04-26T21:54:11.913859+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:59:12.465291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:12.395409+0000","last_change":"2024-04-26T21:52:50.792020+0000","last_active":"2024-04-26T21:54:12.395409+0000","last_peered":"2024-04-26T21:54:12.395409+0000","last_clean":"2024-04-26T21:54:12.395409+0000","last_became_active":"2024-04-26T21:52:50.791677+0000","last_became_peered":"2024-04-26T21:52:50.791677+0000","last_unstale":"2024-04-26T21:54:12.395409+0000","last_undegraded":"2024-04-26T21:54:12.395409+0000","last_fullsized":"2024-04-26T21:54:12.395409+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:11:42.218294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"89'12","reported_seq":51,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.936807+0000","last_change":"2024-04-26T21:54:11.936807+0000","last_active":"2024-04-26T21:54:11.936807+0000","last_peered":"2024-04-26T21:54:11.936807+0000","last_clean":"2024-04-26T21:54:11.936807+0000","last_became_active":"2024-04-26T21:54:11.923482+0000","last_became_peered":"2024-04-26T21:54:11.923482+0000","last_unstale":"2024-04-26T21:54:11.936807+0000","last_undegraded":"2024-04-26T21:54:11.936807+0000","last_fullsized":"2024-04-26T21:54:11.936807+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:15:46.945453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.908047+0000","last_change":"2024-04-26T21:52:50.779775+0000","last_active":"2024-04-26T21:54:11.908047+0000","last_peered":"2024-04-26T21:54:11.908047+0000","last_clean":"2024-04-26T21:54:11.908047+0000","last_became_active":"2024-04-26T21:52:50.778857+0000","last_became_peered":"2024-04-26T21:52:50.778857+0000","last_unstale":"2024-04-26T21:54:11.908047+0000","last_undegraded":"2024-04-26T21:54:11.908047+0000","last_fullsized":"2024-04-26T21:54:11.908047+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:59:50.946578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"87'7","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.934291+0000","last_change":"2024-04-26T21:52:50.113027+0000","last_active":"2024-04-26T21:54:11.934291+0000","last_peered":"2024-04-26T21:54:11.934291+0000","last_clean":"2024-04-26T21:54:11.934291+0000","last_became_active":"2024-04-26T21:52:50.112637+0000","last_became_peered":"2024-04-26T21:52:50.112637+0000","last_unstale":"2024-04-26T21:54:11.934291+0000","last_undegraded":"2024-04-26T21:54:11.934291+0000","last_fullsized":"2024-04-26T21:54:11.934291+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:09:47.798184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:12.034057+0000","last_change":"2024-04-26T21:52:50.785119+0000","last_active":"2024-04-26T21:54:12.034057+0000","last_peered":"2024-04-26T21:54:12.034057+0000","last_clean":"2024-04-26T21:54:12.034057+0000","last_became_active":"2024-04-26T21:52:50.781005+0000","last_became_peered":"2024-04-26T21:52:50.781005+0000","last_unstale":"2024-04-26T21:54:12.034057+0000","last_undegraded":"2024-04-26T21:54:12.034057+0000","last_fullsized":"2024-04-26T21:54:12.034057+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:16:32.986810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"87'2","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.908291+0000","last_change":"2024-04-26T21:52:50.112263+0000","last_active":"2024-04-26T21:54:11.908291+0000","last_peered":"2024-04-26T21:54:11.908291+0000","last_clean":"2024-04-26T21:54:11.908291+0000","last_became_active":"2024-04-26T21:52:50.110649+0000","last_became_peered":"2024-04-26T21:52:50.110649+0000","last_unstale":"2024-04-26T21:54:11.908291+0000","last_undegraded":"2024-04-26T21:54:11.908291+0000","last_fullsized":"2024-04-26T21:54:11.908291+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:14:23.245000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.912952+0000","last_change":"2024-04-26T21:52:50.790674+0000","last_active":"2024-04-26T21:54:11.912952+0000","last_peered":"2024-04-26T21:54:11.912952+0000","last_clean":"2024-04-26T21:54:11.912952+0000","last_became_active":"2024-04-26T21:52:50.790286+0000","last_became_peered":"2024-04-26T21:52:50.790286+0000","last_unstale":"2024-04-26T21:54:11.912952+0000","last_undegraded":"2024-04-26T21:54:11.912952+0000","last_fullsized":"2024-04-26T21:54:11.912952+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:22:32.246321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"87'7","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.907528+0000","last_change":"2024-04-26T21:52:50.112158+0000","last_active":"2024-04-26T21:54:11.907528+0000","last_peered":"2024-04-26T21:54:11.907528+0000","last_clean":"2024-04-26T21:54:11.907528+0000","last_became_active":"2024-04-26T21:52:50.111453+0000","last_became_peered":"2024-04-26T21:52:50.111453+0000","last_unstale":"2024-04-26T21:54:11.907528+0000","last_undegraded":"2024-04-26T21:54:11.907528+0000","last_fullsized":"2024-04-26T21:54:11.907528+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:16:58.626985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.915460+0000","last_change":"2024-04-26T21:52:50.798241+0000","last_active":"2024-04-26T21:54:11.915460+0000","last_peered":"2024-04-26T21:54:11.915460+0000","last_clean":"2024-04-26T21:54:11.915460+0000","last_became_active":"2024-04-26T21:52:50.797878+0000","last_became_peered":"2024-04-26T21:52:50.797878+0000","last_unstale":"2024-04-26T21:54:11.915460+0000","last_undegraded":"2024-04-26T21:54:11.915460+0000","last_fullsized":"2024-04-26T21:54:11.915460+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:42:49.363849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"87'3","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.948118+0000","last_change":"2024-04-26T21:54:11.946852+0000","last_active":"2024-04-26T21:54:11.948118+0000","last_peered":"2024-04-26T21:54:11.948118+0000","last_clean":"2024-04-26T21:54:11.948118+0000","last_became_active":"2024-04-26T21:54:11.923399+0000","last_became_peered":"2024-04-26T21:54:11.923399+0000","last_unstale":"2024-04-26T21:54:11.948118+0000","last_undegraded":"2024-04-26T21:54:11.948118+0000","last_fullsized":"2024-04-26T21:54:11.948118+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:05:34.688374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.908182+0000","last_change":"2024-04-26T21:52:50.776333+0000","last_active":"2024-04-26T21:54:11.908182+0000","last_peered":"2024-04-26T21:54:11.908182+0000","last_clean":"2024-04-26T21:54:11.908182+0000","last_became_active":"2024-04-26T21:52:50.776001+0000","last_became_peered":"2024-04-26T21:52:50.776001+0000","last_unstale":"2024-04-26T21:54:11.908182+0000","last_undegraded":"2024-04-26T21:54:11.908182+0000","last_fullsized":"2024-04-26T21:54:11.908182+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:38:26.578928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"87'4","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.913278+0000","last_change":"2024-04-26T21:52:50.106008+0000","last_active":"2024-04-26T21:54:11.913278+0000","last_peered":"2024-04-26T21:54:11.913278+0000","last_clean":"2024-04-26T21:54:11.913278+0000","last_became_active":"2024-04-26T21:52:50.105565+0000","last_became_peered":"2024-04-26T21:52:50.105565+0000","last_unstale":"2024-04-26T21:54:11.913278+0000","last_undegraded":"2024-04-26T21:54:11.913278+0000","last_fullsized":"2024-04-26T21:54:11.913278+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:15:17.355461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.923264+0000","last_change":"2024-04-26T21:52:50.776051+0000","last_active":"2024-04-26T21:54:11.923264+0000","last_peered":"2024-04-26T21:54:11.923264+0000","last_clean":"2024-04-26T21:54:11.923264+0000","last_became_active":"2024-04-26T21:52:50.775716+0000","last_became_peered":"2024-04-26T21:52:50.775716+0000","last_unstale":"2024-04-26T21:54:11.923264+0000","last_undegraded":"2024-04-26T21:54:11.923264+0000","last_fullsized":"2024-04-26T21:54:11.923264+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:34:21.110773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"87'6","reported_seq":113,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:18.763789+0000","last_change":"2024-04-26T21:52:50.111280+0000","last_active":"2024-04-26T21:54:18.763789+0000","last_peered":"2024-04-26T21:54:18.763789+0000","last_clean":"2024-04-26T21:54:18.763789+0000","last_became_active":"2024-04-26T21:52:50.110972+0000","last_became_peered":"2024-04-26T21:52:50.110972+0000","last_unstale":"2024-04-26T21:54:18.763789+0000","last_undegraded":"2024-04-26T21:54:18.763789+0000","last_fullsized":"2024-04-26T21:54:18.763789+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:41:48.203134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1561,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":84,"num_read_kb":2,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.923193+0000","last_change":"2024-04-26T21:52:50.777488+0000","last_active":"2024-04-26T21:54:11.923193+0000","last_peered":"2024-04-26T21:54:11.923193+0000","last_clean":"2024-04-26T21:54:11.923193+0000","last_became_active":"2024-04-26T21:52:50.777162+0000","last_became_peered":"2024-04-26T21:52:50.777162+0000","last_unstale":"2024-04-26T21:54:11.923193+0000","last_undegraded":"2024-04-26T21:54:11.923193+0000","last_fullsized":"2024-04-26T21:54:11.923193+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:07:22.683109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"87'3","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.916471+0000","last_change":"2024-04-26T21:52:50.104508+0000","last_active":"2024-04-26T21:54:11.916471+0000","last_peered":"2024-04-26T21:54:11.916471+0000","last_clean":"2024-04-26T21:54:11.916471+0000","last_became_active":"2024-04-26T21:52:50.103924+0000","last_became_peered":"2024-04-26T21:52:50.103924+0000","last_unstale":"2024-04-26T21:54:11.916471+0000","last_undegraded":"2024-04-26T21:54:11.916471+0000","last_fullsized":"2024-04-26T21:54:11.916471+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:39:25.861339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"87'1","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.922484+0000","last_change":"2024-04-26T21:52:50.782915+0000","last_active":"2024-04-26T21:54:11.922484+0000","last_peered":"2024-04-26T21:54:11.922484+0000","last_clean":"2024-04-26T21:54:11.922484+0000","last_became_active":"2024-04-26T21:52:50.782253+0000","last_became_peered":"2024-04-26T21:52:50.782253+0000","last_unstale":"2024-04-26T21:54:11.922484+0000","last_undegraded":"2024-04-26T21:54:11.922484+0000","last_fullsized":"2024-04-26T21:54:11.922484+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:12:51.624162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"87'5","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:12.395276+0000","last_change":"2024-04-26T21:53:11.590613+0000","last_active":"2024-04-26T21:54:12.395276+0000","last_peered":"2024-04-26T21:54:12.395276+0000","last_clean":"2024-04-26T21:54:12.395276+0000","last_became_active":"2024-04-26T21:53:11.578727+0000","last_became_peered":"2024-04-26T21:53:11.578727+0000","last_unstale":"2024-04-26T21:54:12.395276+0000","last_undegraded":"2024-04-26T21:54:12.395276+0000","last_fullsized":"2024-04-26T21:54:12.395276+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:44:38.867231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.922794+0000","last_change":"2024-04-26T21:52:50.787777+0000","last_active":"2024-04-26T21:54:11.922794+0000","last_peered":"2024-04-26T21:54:11.922794+0000","last_clean":"2024-04-26T21:54:11.922794+0000","last_became_active":"2024-04-26T21:52:50.782403+0000","last_became_peered":"2024-04-26T21:52:50.782403+0000","last_unstale":"2024-04-26T21:54:11.922794+0000","last_undegraded":"2024-04-26T21:54:11.922794+0000","last_fullsized":"2024-04-26T21:54:11.922794+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:28:58.425802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.915785+0000","last_change":"2024-04-26T21:52:50.800010+0000","last_active":"2024-04-26T21:54:11.915785+0000","last_peered":"2024-04-26T21:54:11.915785+0000","last_clean":"2024-04-26T21:54:11.915785+0000","last_became_active":"2024-04-26T21:52:50.799308+0000","last_became_peered":"2024-04-26T21:52:50.799308+0000","last_unstale":"2024-04-26T21:54:11.915785+0000","last_undegraded":"2024-04-26T21:54:11.915785+0000","last_fullsized":"2024-04-26T21:54:11.915785+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:54:25.772312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.908880+0000","last_change":"2024-04-26T21:52:50.798281+0000","last_active":"2024-04-26T21:54:11.908880+0000","last_peered":"2024-04-26T21:54:11.908880+0000","last_clean":"2024-04-26T21:54:11.908880+0000","last_became_active":"2024-04-26T21:52:50.795879+0000","last_became_peered":"2024-04-26T21:52:50.795879+0000","last_unstale":"2024-04-26T21:54:11.908880+0000","last_undegraded":"2024-04-26T21:54:11.908880+0000","last_fullsized":"2024-04-26T21:54:11.908880+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:36:00.769991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.908572+0000","last_change":"2024-04-26T21:52:50.791058+0000","last_active":"2024-04-26T21:54:11.908572+0000","last_peered":"2024-04-26T21:54:11.908572+0000","last_clean":"2024-04-26T21:54:11.908572+0000","last_became_active":"2024-04-26T21:52:50.790721+0000","last_became_peered":"2024-04-26T21:52:50.790721+0000","last_unstale":"2024-04-26T21:54:11.908572+0000","last_undegraded":"2024-04-26T21:54:11.908572+0000","last_fullsized":"2024-04-26T21:54:11.908572+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:56:03.239622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:12.033600+0000","last_change":"2024-04-26T21:52:50.788193+0000","last_active":"2024-04-26T21:54:12.033600+0000","last_peered":"2024-04-26T21:54:12.033600+0000","last_clean":"2024-04-26T21:54:12.033600+0000","last_became_active":"2024-04-26T21:52:50.786749+0000","last_became_peered":"2024-04-26T21:52:50.786749+0000","last_unstale":"2024-04-26T21:54:12.033600+0000","last_undegraded":"2024-04-26T21:54:12.033600+0000","last_fullsized":"2024-04-26T21:54:12.033600+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:22:11.277217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:12.395541+0000","last_change":"2024-04-26T21:52:50.782508+0000","last_active":"2024-04-26T21:54:12.395541+0000","last_peered":"2024-04-26T21:54:12.395541+0000","last_clean":"2024-04-26T21:54:12.395541+0000","last_became_active":"2024-04-26T21:52:50.781505+0000","last_became_peered":"2024-04-26T21:52:50.781505+0000","last_unstale":"2024-04-26T21:54:12.395541+0000","last_undegraded":"2024-04-26T21:54:12.395541+0000","last_fullsized":"2024-04-26T21:54:12.395541+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:22:57.867546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.916462+0000","last_change":"2024-04-26T21:52:50.789205+0000","last_active":"2024-04-26T21:54:11.916462+0000","last_peered":"2024-04-26T21:54:11.916462+0000","last_clean":"2024-04-26T21:54:11.916462+0000","last_became_active":"2024-04-26T21:52:50.788898+0000","last_became_peered":"2024-04-26T21:52:50.788898+0000","last_unstale":"2024-04-26T21:54:11.916462+0000","last_undegraded":"2024-04-26T21:54:11.916462+0000","last_fullsized":"2024-04-26T21:54:11.916462+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:18:51.401968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.909237+0000","last_change":"2024-04-26T21:52:50.793451+0000","last_active":"2024-04-26T21:54:11.909237+0000","last_peered":"2024-04-26T21:54:11.909237+0000","last_clean":"2024-04-26T21:54:11.909237+0000","last_became_active":"2024-04-26T21:52:50.791696+0000","last_became_peered":"2024-04-26T21:52:50.791696+0000","last_unstale":"2024-04-26T21:54:11.909237+0000","last_undegraded":"2024-04-26T21:54:11.909237+0000","last_fullsized":"2024-04-26T21:54:11.909237+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:51:43.543500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.907944+0000","last_change":"2024-04-26T21:53:10.919691+0000","last_active":"2024-04-26T21:54:11.907944+0000","last_peered":"2024-04-26T21:54:11.907944+0000","last_clean":"2024-04-26T21:54:11.907944+0000","last_became_active":"2024-04-26T21:53:10.919262+0000","last_became_peered":"2024-04-26T21:53:10.919262+0000","last_unstale":"2024-04-26T21:54:11.907944+0000","last_undegraded":"2024-04-26T21:54:11.907944+0000","last_fullsized":"2024-04-26T21:54:11.907944+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:15:35.719661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,8,2],"acting":[3,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":15,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:12.395038+0000","last_change":"2024-04-26T21:53:10.914854+0000","last_active":"2024-04-26T21:54:12.395038+0000","last_peered":"2024-04-26T21:54:12.395038+0000","last_clean":"2024-04-26T21:54:12.395038+0000","last_became_active":"2024-04-26T21:53:10.914458+0000","last_became_peered":"2024-04-26T21:53:10.914458+0000","last_unstale":"2024-04-26T21:54:12.395038+0000","last_undegraded":"2024-04-26T21:54:12.395038+0000","last_fullsized":"2024-04-26T21:54:12.395038+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:26:41.538053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,3],"acting":[6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.907469+0000","last_change":"2024-04-26T21:52:50.794580+0000","last_active":"2024-04-26T21:54:11.907469+0000","last_peered":"2024-04-26T21:54:11.907469+0000","last_clean":"2024-04-26T21:54:11.907469+0000","last_became_active":"2024-04-26T21:52:50.792398+0000","last_became_peered":"2024-04-26T21:52:50.792398+0000","last_unstale":"2024-04-26T21:54:11.907469+0000","last_undegraded":"2024-04-26T21:54:11.907469+0000","last_fullsized":"2024-04-26T21:54:11.907469+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:21:38.913644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.909449+0000","last_change":"2024-04-26T21:53:11.580576+0000","last_active":"2024-04-26T21:54:11.909449+0000","last_peered":"2024-04-26T21:54:11.909449+0000","last_clean":"2024-04-26T21:54:11.909449+0000","last_became_active":"2024-04-26T21:53:11.578632+0000","last_became_peered":"2024-04-26T21:53:11.578632+0000","last_unstale":"2024-04-26T21:54:11.909449+0000","last_undegraded":"2024-04-26T21:54:11.909449+0000","last_fullsized":"2024-04-26T21:54:11.909449+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:30:24.412417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,8],"acting":[1,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:12.395881+0000","last_change":"2024-04-26T21:52:50.792166+0000","last_active":"2024-04-26T21:54:12.395881+0000","last_peered":"2024-04-26T21:54:12.395881+0000","last_clean":"2024-04-26T21:54:12.395881+0000","last_became_active":"2024-04-26T21:52:50.790709+0000","last_became_peered":"2024-04-26T21:52:50.790709+0000","last_unstale":"2024-04-26T21:54:12.395881+0000","last_undegraded":"2024-04-26T21:54:12.395881+0000","last_fullsized":"2024-04-26T21:54:12.395881+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:05:48.200573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.915301+0000","last_change":"2024-04-26T21:53:10.930526+0000","last_active":"2024-04-26T21:54:11.915301+0000","last_peered":"2024-04-26T21:54:11.915301+0000","last_clean":"2024-04-26T21:54:11.915301+0000","last_became_active":"2024-04-26T21:53:10.928850+0000","last_became_peered":"2024-04-26T21:53:10.928850+0000","last_unstale":"2024-04-26T21:54:11.915301+0000","last_undegraded":"2024-04-26T21:54:11.915301+0000","last_fullsized":"2024-04-26T21:54:11.915301+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:54:03.420855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":57764,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":849,"num_read_kb":486,"num_write":237,"num_write_kb":123,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":622,"num_keys_recovered":7,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":540672,"data_stored":173292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":204,"ondisk_log_size":204,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":82,"seq":352187318307,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1015808,"data_stored":484903,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":74,"seq":317827579945,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28688,"kb_used_data":1668,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716464,"statfs":{"total":95995035648,"available":95965659136,"internally_reserved":0,"allocated":1708032,"data_stored":1126492,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":69,"seq":296352743473,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36356,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708796,"statfs":{"total":95995035648,"available":95957807104,"internally_reserved":0,"allocated":1040384,"data_stored":484442,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":61,"seq":261993005113,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36352,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708800,"statfs":{"total":95995035648,"available":95957811200,"internally_reserved":0,"allocated":1036288,"data_stored":484820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299454,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28620,"kb_used_data":1596,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716532,"statfs":{"total":95995035648,"available":95965728768,"internally_reserved":0,"allocated":1634304,"data_stored":1076947,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462982,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36340,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708812,"statfs":{"total":95995035648,"available":95957823488,"internally_reserved":0,"allocated":1024000,"data_stored":484159,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574952,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28076,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717076,"statfs":{"total":95995035648,"available":95966285824,"internally_reserved":0,"allocated":1077248,"data_stored":536023,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738480,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36364,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708788,"statfs":{"total":95995035648,"available":95957798912,"internally_reserved":0,"allocated":1044480,"data_stored":485889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411426,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36408,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708744,"statfs":{"total":95995035648,"available":95957753856,"internally_reserved":0,"allocated":1089536,"data_stored":536012,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182490,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1040384,"data_stored":483984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920853,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36352,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708800,"statfs":{"total":95995035648,"available":95957811200,"internally_reserved":0,"allocated":1032192,"data_stored":484642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724620,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36948,"kb_used_data":1608,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708204,"statfs":{"total":95995035648,"available":95957200896,"internally_reserved":0,"allocated":1646592,"data_stored":1076625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":52703,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":52692,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1322,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":52687,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T21:54:25.537 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid a1780950-0415-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-26T21:54:25.971 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/a1780950-0415-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:54:26.289 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:54:25 smithi072 ceph-mon[28138]: from='client.14994 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T21:54:26.290 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:54:25 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:54:26.290 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:54:25 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:54:26.290 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:54:25 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:54:26.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:54:25 smithi076 ceph-mon[31823]: from='client.14994 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T21:54:26.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:54:25 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:54:26.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:54:25 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:54:26.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:54:25 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:54:26.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:54:25 smithi164 ceph-mon[31916]: from='client.14994 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T21:54:26.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:54:25 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:54:26.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:54:25 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:54:26.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:54:25 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:54:27.314 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:54:26 smithi072 ceph-mon[28138]: pgmap v419: 97 pgs: 97 active+clean; 633 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 85 B/s wr, 11 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-04-26T21:54:27.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:54:26 smithi076 ceph-mon[31823]: pgmap v419: 97 pgs: 97 active+clean; 633 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 85 B/s wr, 11 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-04-26T21:54:27.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:54:26 smithi164 ceph-mon[31916]: pgmap v419: 97 pgs: 97 active+clean; 633 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 85 B/s wr, 11 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-04-26T21:54:28.197 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:28.196+0000 7feca8c00700 1 -- 172.21.15.72:0/2206481568 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7feca4104580 msgr2=0x7feca4104960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:28.198 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:28.196+0000 7feca8c00700 1 --2- 172.21.15.72:0/2206481568 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7feca4104580 0x7feca4104960 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fec94009990 tx=0x7fec94009ca0 comp rx=0 tx=0).stop 2024-04-26T21:54:28.198 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:28.196+0000 7feca8c00700 1 -- 172.21.15.72:0/2206481568 shutdown_connections 2024-04-26T21:54:28.198 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:28.196+0000 7feca8c00700 1 --2- 172.21.15.72:0/2206481568 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7feca4104580 0x7feca4104960 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:28.198 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:28.196+0000 7feca8c00700 1 --2- 172.21.15.72:0/2206481568 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feca410d9b0 0x7feca410fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:28.198 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:28.196+0000 7feca8c00700 1 --2- 172.21.15.72:0/2206481568 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7feca4104ea0 0x7feca410d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:28.198 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:28.196+0000 7feca8c00700 1 -- 172.21.15.72:0/2206481568 >> 172.21.15.72:0/2206481568 conn(0x7feca40fddd0 msgr2=0x7feca41001f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:54:28.198 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:28.196+0000 7feca8c00700 1 -- 172.21.15.72:0/2206481568 shutdown_connections 2024-04-26T21:54:28.198 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:28.196+0000 7feca8c00700 1 -- 172.21.15.72:0/2206481568 wait complete. 2024-04-26T21:54:28.198 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:28.197+0000 7feca8c00700 1 Processor -- start 2024-04-26T21:54:28.198 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:28.198+0000 7feca8c00700 1 -- start start 2024-04-26T21:54:28.200 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:28.198+0000 7feca8c00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7feca4104ea0 0x7feca419bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:28.200 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:28.198+0000 7feca8c00700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7feca410d9b0 0x7feca419c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:28.200 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:28.198+0000 7feca8c00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feca41a06c0 0x7feca41a0b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:28.200 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:28.198+0000 7feca8c00700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7feca41134c0 con 0x7feca4104ea0 2024-04-26T21:54:28.200 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:28.198+0000 7feca8c00700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7feca4113340 con 0x7feca410d9b0 2024-04-26T21:54:28.200 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:28.198+0000 7feca8c00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7feca4113640 con 0x7feca41a06c0 2024-04-26T21:54:28.200 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:28.198+0000 7feca3400700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7feca4104ea0 0x7feca419bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:28.201 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:28.198+0000 7feca3400700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7feca4104ea0 0x7feca419bdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:32852/0 (socket says 172.21.15.72:32852) 2024-04-26T21:54:28.201 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:28.198+0000 7feca3400700 1 -- 172.21.15.72:0/1229241511 learned_addr learned my addr 172.21.15.72:0/1229241511 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:54:28.201 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:28.199+0000 7feca3e00700 1 --2- 172.21.15.72:0/1229241511 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feca41a06c0 0x7feca41a0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:28.201 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:28.199+0000 7feca2a00700 1 --2- 172.21.15.72:0/1229241511 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7feca410d9b0 0x7feca419c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:28.201 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:28.199+0000 7feca3400700 1 -- 172.21.15.72:0/1229241511 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feca41a06c0 msgr2=0x7feca41a0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:28.201 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:28.199+0000 7feca3400700 1 --2- 172.21.15.72:0/1229241511 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feca41a06c0 0x7feca41a0b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:28.201 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:28.199+0000 7feca3400700 1 -- 172.21.15.72:0/1229241511 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7feca410d9b0 msgr2=0x7feca419c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:28.201 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:28.199+0000 7feca3400700 1 --2- 172.21.15.72:0/1229241511 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7feca410d9b0 0x7feca419c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:28.201 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:28.199+0000 7feca3400700 1 -- 172.21.15.72:0/1229241511 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feca41a11e0 con 0x7feca4104ea0 2024-04-26T21:54:28.201 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:28.199+0000 7feca2a00700 1 --2- 172.21.15.72:0/1229241511 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7feca410d9b0 0x7feca419c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:54:28.201 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:28.199+0000 7feca3400700 1 --2- 172.21.15.72:0/1229241511 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7feca4104ea0 0x7feca419bdf0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fec9400b5c0 tx=0x7fec940047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:54:28.201 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:28.199+0000 7fec9be00700 1 -- 172.21.15.72:0/1229241511 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec94004830 con 0x7feca4104ea0 2024-04-26T21:54:28.201 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:28.200+0000 7fec9be00700 1 -- 172.21.15.72:0/1229241511 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fec94005020 con 0x7feca4104ea0 2024-04-26T21:54:28.201 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:28.200+0000 7fec9be00700 1 -- 172.21.15.72:0/1229241511 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec9400f950 con 0x7feca4104ea0 2024-04-26T21:54:28.202 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:28.200+0000 7feca8c00700 1 -- 172.21.15.72:0/1229241511 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feca41a1470 con 0x7feca4104ea0 2024-04-26T21:54:28.202 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:28.200+0000 7feca8c00700 1 -- 172.21.15.72:0/1229241511 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feca41a8b40 con 0x7feca4104ea0 2024-04-26T21:54:28.202 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:28.201+0000 7feca8c00700 1 -- 172.21.15.72:0/1229241511 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feca406a110 con 0x7feca4104ea0 2024-04-26T21:54:28.203 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:28.203+0000 7fec9be00700 1 -- 172.21.15.72:0/1229241511 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fec940049d0 con 0x7feca4104ea0 2024-04-26T21:54:28.205 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:28.205+0000 7fec9be00700 1 --2- 172.21.15.72:0/1229241511 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fec880a3d50 0x7fec880a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:28.206 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:28.205+0000 7fec9be00700 1 -- 172.21.15.72:0/1229241511 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7fec940d1080 con 0x7feca4104ea0 2024-04-26T21:54:28.206 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:28.206+0000 7feca2a00700 1 --2- 172.21.15.72:0/1229241511 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fec880a3d50 0x7fec880a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:28.206 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:28.206+0000 7feca2a00700 1 --2- 172.21.15.72:0/1229241511 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fec880a3d50 0x7fec880a6210 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7feca41002a0 tx=0x7fec90005ec0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:54:28.210 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:28.210+0000 7fec9be00700 1 -- 172.21.15.72:0/1229241511 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fec94013070 con 0x7feca4104ea0 2024-04-26T21:54:28.435 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:28.434+0000 7feca8c00700 1 -- 172.21.15.72:0/1229241511 --> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7feca41039b0 con 0x7fec880a3d50 2024-04-26T21:54:28.443 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:28.443+0000 7fec9be00700 1 -- 172.21.15.72:0/1229241511 <== mgr.14152 v2:172.21.15.72:6800/588150172 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258774 (secure 0 0 0) 0x7feca41039b0 con 0x7fec880a3d50 2024-04-26T21:54:28.448 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T21:54:28.450 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:28.447+0000 7feca8c00700 1 -- 172.21.15.72:0/1229241511 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fec880a3d50 msgr2=0x7fec880a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:28.450 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:28.447+0000 7feca8c00700 1 --2- 172.21.15.72:0/1229241511 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fec880a3d50 0x7fec880a6210 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7feca41002a0 tx=0x7fec90005ec0 comp rx=0 tx=0).stop 2024-04-26T21:54:28.450 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:28.447+0000 7feca8c00700 1 -- 172.21.15.72:0/1229241511 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7feca4104ea0 msgr2=0x7feca419bdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:28.450 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:28.447+0000 7feca8c00700 1 --2- 172.21.15.72:0/1229241511 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7feca4104ea0 0x7feca419bdf0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fec9400b5c0 tx=0x7fec940047b0 comp rx=0 tx=0).stop 2024-04-26T21:54:28.450 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:28.448+0000 7feca8c00700 1 -- 172.21.15.72:0/1229241511 shutdown_connections 2024-04-26T21:54:28.450 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:28.448+0000 7feca8c00700 1 --2- 172.21.15.72:0/1229241511 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fec880a3d50 0x7fec880a6210 secure :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7feca41002a0 tx=0x7fec90005ec0 comp rx=0 tx=0).stop 2024-04-26T21:54:28.450 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:28.448+0000 7feca8c00700 1 --2- 172.21.15.72:0/1229241511 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7feca4104ea0 0x7feca419bdf0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:28.450 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:28.448+0000 7feca8c00700 1 --2- 172.21.15.72:0/1229241511 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feca41a06c0 0x7feca41a0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:28.450 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:28.448+0000 7feca8c00700 1 --2- 172.21.15.72:0/1229241511 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7feca410d9b0 0x7feca419c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:28.450 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:28.448+0000 7feca8c00700 1 -- 172.21.15.72:0/1229241511 >> 172.21.15.72:0/1229241511 conn(0x7feca40fddd0 msgr2=0x7feca410ed50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:54:28.451 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:28.448+0000 7feca8c00700 1 -- 172.21.15.72:0/1229241511 shutdown_connections 2024-04-26T21:54:28.451 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:28.448+0000 7feca8c00700 1 -- 172.21.15.72:0/1229241511 wait complete. 2024-04-26T21:54:28.451 INFO:teuthology.orchestra.run.smithi072.stderr:dumped all 2024-04-26T21:54:29.243 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:54:28 smithi164 ceph-mon[31916]: pgmap v420: 97 pgs: 97 active+clean; 633 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:54:29.266 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:54:28 smithi072 ceph-mon[28138]: pgmap v420: 97 pgs: 97 active+clean; 633 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:54:29.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:54:28 smithi076 ceph-mon[31823]: pgmap v420: 97 pgs: 97 active+clean; 633 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:54:29.337 INFO:teuthology.orchestra.run.smithi072.stdout:{"pg_ready":true,"pg_map":{"version":420,"stamp":"2024-04-26T21:54:27.810467+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":648366,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1020,"num_read_kb":620,"num_write":420,"num_write_kb":3063,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":1181358,"num_keys_recovered":7,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":343,"ondisk_log_size":343,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":396556,"kb_used_data":14052,"kb_used_omap":18,"kb_used_meta":382317,"kb_avail":1124545268,"statfs":{"total":1151940427776,"available":1151534354432,"internally_reserved":0,"allocated":14389248,"data_stored":7744938,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":391492922},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":104,"num_read_kb":50,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004489"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.921345+0000","last_change":"2024-04-26T21:52:50.787765+0000","last_active":"2024-04-26T21:54:11.921345+0000","last_peered":"2024-04-26T21:54:11.921345+0000","last_clean":"2024-04-26T21:54:11.921345+0000","last_became_active":"2024-04-26T21:52:50.786969+0000","last_became_peered":"2024-04-26T21:52:50.786969+0000","last_unstale":"2024-04-26T21:54:11.921345+0000","last_undegraded":"2024-04-26T21:54:11.921345+0000","last_fullsized":"2024-04-26T21:54:11.921345+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:31:50.659210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:12.034215+0000","last_change":"2024-04-26T21:52:50.779735+0000","last_active":"2024-04-26T21:54:12.034215+0000","last_peered":"2024-04-26T21:54:12.034215+0000","last_clean":"2024-04-26T21:54:12.034215+0000","last_became_active":"2024-04-26T21:52:50.779358+0000","last_became_peered":"2024-04-26T21:52:50.779358+0000","last_unstale":"2024-04-26T21:54:12.034215+0000","last_undegraded":"2024-04-26T21:54:12.034215+0000","last_fullsized":"2024-04-26T21:54:12.034215+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:58:45.631365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:12.033834+0000","last_change":"2024-04-26T21:52:50.788220+0000","last_active":"2024-04-26T21:54:12.033834+0000","last_peered":"2024-04-26T21:54:12.033834+0000","last_clean":"2024-04-26T21:54:12.033834+0000","last_became_active":"2024-04-26T21:52:50.787282+0000","last_became_peered":"2024-04-26T21:52:50.787282+0000","last_unstale":"2024-04-26T21:54:12.033834+0000","last_undegraded":"2024-04-26T21:54:12.033834+0000","last_fullsized":"2024-04-26T21:54:12.033834+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:12:28.577893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.934391+0000","last_change":"2024-04-26T21:52:50.795367+0000","last_active":"2024-04-26T21:54:11.934391+0000","last_peered":"2024-04-26T21:54:11.934391+0000","last_clean":"2024-04-26T21:54:11.934391+0000","last_became_active":"2024-04-26T21:52:50.793602+0000","last_became_peered":"2024-04-26T21:52:50.793602+0000","last_unstale":"2024-04-26T21:54:11.934391+0000","last_undegraded":"2024-04-26T21:54:11.934391+0000","last_fullsized":"2024-04-26T21:54:11.934391+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:37:23.356792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.912047+0000","last_change":"2024-04-26T21:52:50.787333+0000","last_active":"2024-04-26T21:54:11.912047+0000","last_peered":"2024-04-26T21:54:11.912047+0000","last_clean":"2024-04-26T21:54:11.912047+0000","last_became_active":"2024-04-26T21:52:50.785319+0000","last_became_peered":"2024-04-26T21:52:50.785319+0000","last_unstale":"2024-04-26T21:54:11.912047+0000","last_undegraded":"2024-04-26T21:54:11.912047+0000","last_fullsized":"2024-04-26T21:54:11.912047+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:06:29.154825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:12.396920+0000","last_change":"2024-04-26T21:52:50.780829+0000","last_active":"2024-04-26T21:54:12.396920+0000","last_peered":"2024-04-26T21:54:12.396920+0000","last_clean":"2024-04-26T21:54:12.396920+0000","last_became_active":"2024-04-26T21:52:50.780439+0000","last_became_peered":"2024-04-26T21:52:50.780439+0000","last_unstale":"2024-04-26T21:54:12.396920+0000","last_undegraded":"2024-04-26T21:54:12.396920+0000","last_fullsized":"2024-04-26T21:54:12.396920+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:20:14.533853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.922082+0000","last_change":"2024-04-26T21:52:50.787679+0000","last_active":"2024-04-26T21:54:11.922082+0000","last_peered":"2024-04-26T21:54:11.922082+0000","last_clean":"2024-04-26T21:54:11.922082+0000","last_became_active":"2024-04-26T21:52:50.787277+0000","last_became_peered":"2024-04-26T21:52:50.787277+0000","last_unstale":"2024-04-26T21:54:11.922082+0000","last_undegraded":"2024-04-26T21:54:11.922082+0000","last_fullsized":"2024-04-26T21:54:11.922082+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:29:02.131503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":15,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.905091+0000","last_change":"2024-04-26T21:53:10.927384+0000","last_active":"2024-04-26T21:54:11.905091+0000","last_peered":"2024-04-26T21:54:11.905091+0000","last_clean":"2024-04-26T21:54:11.905091+0000","last_became_active":"2024-04-26T21:53:10.926996+0000","last_became_peered":"2024-04-26T21:53:10.926996+0000","last_unstale":"2024-04-26T21:54:11.905091+0000","last_undegraded":"2024-04-26T21:54:11.905091+0000","last_fullsized":"2024-04-26T21:54:11.905091+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:25:10.174532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.909424+0000","last_change":"2024-04-26T21:52:50.779701+0000","last_active":"2024-04-26T21:54:11.909424+0000","last_peered":"2024-04-26T21:54:11.909424+0000","last_clean":"2024-04-26T21:54:11.909424+0000","last_became_active":"2024-04-26T21:52:50.779331+0000","last_became_peered":"2024-04-26T21:52:50.779331+0000","last_unstale":"2024-04-26T21:54:11.909424+0000","last_undegraded":"2024-04-26T21:54:11.909424+0000","last_fullsized":"2024-04-26T21:54:11.909424+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:38:15.569806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:12.397805+0000","last_change":"2024-04-26T21:52:50.790081+0000","last_active":"2024-04-26T21:54:12.397805+0000","last_peered":"2024-04-26T21:54:12.397805+0000","last_clean":"2024-04-26T21:54:12.397805+0000","last_became_active":"2024-04-26T21:52:50.786575+0000","last_became_peered":"2024-04-26T21:52:50.786575+0000","last_unstale":"2024-04-26T21:54:12.397805+0000","last_undegraded":"2024-04-26T21:54:12.397805+0000","last_fullsized":"2024-04-26T21:54:12.397805+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:56:47.467128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.922863+0000","last_change":"2024-04-26T21:52:50.791983+0000","last_active":"2024-04-26T21:54:11.922863+0000","last_peered":"2024-04-26T21:54:11.922863+0000","last_clean":"2024-04-26T21:54:11.922863+0000","last_became_active":"2024-04-26T21:52:50.788312+0000","last_became_peered":"2024-04-26T21:52:50.788312+0000","last_unstale":"2024-04-26T21:54:11.922863+0000","last_undegraded":"2024-04-26T21:54:11.922863+0000","last_fullsized":"2024-04-26T21:54:11.922863+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:47:47.410267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.934110+0000","last_change":"2024-04-26T21:52:50.794237+0000","last_active":"2024-04-26T21:54:11.934110+0000","last_peered":"2024-04-26T21:54:11.934110+0000","last_clean":"2024-04-26T21:54:11.934110+0000","last_became_active":"2024-04-26T21:52:50.792260+0000","last_became_peered":"2024-04-26T21:52:50.792260+0000","last_unstale":"2024-04-26T21:54:11.934110+0000","last_undegraded":"2024-04-26T21:54:11.934110+0000","last_fullsized":"2024-04-26T21:54:11.934110+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:36:05.467667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:12.396013+0000","last_change":"2024-04-26T21:52:50.777687+0000","last_active":"2024-04-26T21:54:12.396013+0000","last_peered":"2024-04-26T21:54:12.396013+0000","last_clean":"2024-04-26T21:54:12.396013+0000","last_became_active":"2024-04-26T21:52:50.777200+0000","last_became_peered":"2024-04-26T21:52:50.777200+0000","last_unstale":"2024-04-26T21:54:12.396013+0000","last_undegraded":"2024-04-26T21:54:12.396013+0000","last_fullsized":"2024-04-26T21:54:12.396013+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:40:45.167144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:12.033275+0000","last_change":"2024-04-26T21:52:50.787008+0000","last_active":"2024-04-26T21:54:12.033275+0000","last_peered":"2024-04-26T21:54:12.033275+0000","last_clean":"2024-04-26T21:54:12.033275+0000","last_became_active":"2024-04-26T21:52:50.785656+0000","last_became_peered":"2024-04-26T21:52:50.785656+0000","last_unstale":"2024-04-26T21:54:12.033275+0000","last_undegraded":"2024-04-26T21:54:12.033275+0000","last_fullsized":"2024-04-26T21:54:12.033275+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:20:12.441281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:12.033239+0000","last_change":"2024-04-26T21:52:50.796537+0000","last_active":"2024-04-26T21:54:12.033239+0000","last_peered":"2024-04-26T21:54:12.033239+0000","last_clean":"2024-04-26T21:54:12.033239+0000","last_became_active":"2024-04-26T21:52:50.796229+0000","last_became_peered":"2024-04-26T21:52:50.796229+0000","last_unstale":"2024-04-26T21:54:12.033239+0000","last_undegraded":"2024-04-26T21:54:12.033239+0000","last_fullsized":"2024-04-26T21:54:12.033239+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:38:46.404444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.908267+0000","last_change":"2024-04-26T21:52:50.789211+0000","last_active":"2024-04-26T21:54:11.908267+0000","last_peered":"2024-04-26T21:54:11.908267+0000","last_clean":"2024-04-26T21:54:11.908267+0000","last_became_active":"2024-04-26T21:52:50.788889+0000","last_became_peered":"2024-04-26T21:52:50.788889+0000","last_unstale":"2024-04-26T21:54:11.908267+0000","last_undegraded":"2024-04-26T21:54:11.908267+0000","last_fullsized":"2024-04-26T21:54:11.908267+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:25:48.497840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.921414+0000","last_change":"2024-04-26T21:52:50.788035+0000","last_active":"2024-04-26T21:54:11.921414+0000","last_peered":"2024-04-26T21:54:11.921414+0000","last_clean":"2024-04-26T21:54:11.921414+0000","last_became_active":"2024-04-26T21:52:50.787453+0000","last_became_peered":"2024-04-26T21:52:50.787453+0000","last_unstale":"2024-04-26T21:54:11.921414+0000","last_undegraded":"2024-04-26T21:54:11.921414+0000","last_fullsized":"2024-04-26T21:54:11.921414+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:40:29.718537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.905521+0000","last_change":"2024-04-26T21:52:50.778228+0000","last_active":"2024-04-26T21:54:11.905521+0000","last_peered":"2024-04-26T21:54:11.905521+0000","last_clean":"2024-04-26T21:54:11.905521+0000","last_became_active":"2024-04-26T21:52:50.777934+0000","last_became_peered":"2024-04-26T21:52:50.777934+0000","last_unstale":"2024-04-26T21:54:11.905521+0000","last_undegraded":"2024-04-26T21:54:11.905521+0000","last_fullsized":"2024-04-26T21:54:11.905521+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:33:34.598325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.909087+0000","last_change":"2024-04-26T21:52:50.790795+0000","last_active":"2024-04-26T21:54:11.909087+0000","last_peered":"2024-04-26T21:54:11.909087+0000","last_clean":"2024-04-26T21:54:11.909087+0000","last_became_active":"2024-04-26T21:52:50.788254+0000","last_became_peered":"2024-04-26T21:52:50.788254+0000","last_unstale":"2024-04-26T21:54:11.909087+0000","last_undegraded":"2024-04-26T21:54:11.909087+0000","last_fullsized":"2024-04-26T21:54:11.909087+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:20:28.235137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"87'13","reported_seq":123,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:22.769616+0000","last_change":"2024-04-26T21:52:50.105513+0000","last_active":"2024-04-26T21:54:22.769616+0000","last_peered":"2024-04-26T21:54:22.769616+0000","last_clean":"2024-04-26T21:54:22.769616+0000","last_became_active":"2024-04-26T21:52:50.105131+0000","last_became_peered":"2024-04-26T21:52:50.105131+0000","last_unstale":"2024-04-26T21:54:22.769616+0000","last_undegraded":"2024-04-26T21:54:22.769616+0000","last_fullsized":"2024-04-26T21:54:22.769616+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:36:58.085803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":87,"num_read_kb":87,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.908266+0000","last_change":"2024-04-26T21:52:50.795025+0000","last_active":"2024-04-26T21:54:11.908266+0000","last_peered":"2024-04-26T21:54:11.908266+0000","last_clean":"2024-04-26T21:54:11.908266+0000","last_became_active":"2024-04-26T21:52:50.793608+0000","last_became_peered":"2024-04-26T21:52:50.793608+0000","last_unstale":"2024-04-26T21:54:11.908266+0000","last_undegraded":"2024-04-26T21:54:11.908266+0000","last_fullsized":"2024-04-26T21:54:11.908266+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:49:47.764848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"87'2","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.908318+0000","last_change":"2024-04-26T21:52:50.109197+0000","last_active":"2024-04-26T21:54:11.908318+0000","last_peered":"2024-04-26T21:54:11.908318+0000","last_clean":"2024-04-26T21:54:11.908318+0000","last_became_active":"2024-04-26T21:52:50.108835+0000","last_became_peered":"2024-04-26T21:52:50.108835+0000","last_unstale":"2024-04-26T21:54:11.908318+0000","last_undegraded":"2024-04-26T21:54:11.908318+0000","last_fullsized":"2024-04-26T21:54:11.908318+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:45:09.777747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:12.034355+0000","last_change":"2024-04-26T21:52:50.783094+0000","last_active":"2024-04-26T21:54:12.034355+0000","last_peered":"2024-04-26T21:54:12.034355+0000","last_clean":"2024-04-26T21:54:12.034355+0000","last_became_active":"2024-04-26T21:52:50.782468+0000","last_became_peered":"2024-04-26T21:52:50.782468+0000","last_unstale":"2024-04-26T21:54:12.034355+0000","last_undegraded":"2024-04-26T21:54:12.034355+0000","last_fullsized":"2024-04-26T21:54:12.034355+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:53:12.977054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"87'12","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.913322+0000","last_change":"2024-04-26T21:52:50.104824+0000","last_active":"2024-04-26T21:54:11.913322+0000","last_peered":"2024-04-26T21:54:11.913322+0000","last_clean":"2024-04-26T21:54:11.913322+0000","last_became_active":"2024-04-26T21:52:50.104514+0000","last_became_peered":"2024-04-26T21:52:50.104514+0000","last_unstale":"2024-04-26T21:54:11.913322+0000","last_undegraded":"2024-04-26T21:54:11.913322+0000","last_fullsized":"2024-04-26T21:54:11.913322+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:45:31.541940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.909084+0000","last_change":"2024-04-26T21:52:50.779695+0000","last_active":"2024-04-26T21:54:11.909084+0000","last_peered":"2024-04-26T21:54:11.909084+0000","last_clean":"2024-04-26T21:54:11.909084+0000","last_became_active":"2024-04-26T21:52:50.779234+0000","last_became_peered":"2024-04-26T21:52:50.779234+0000","last_unstale":"2024-04-26T21:54:11.909084+0000","last_undegraded":"2024-04-26T21:54:11.909084+0000","last_fullsized":"2024-04-26T21:54:11.909084+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:33:07.316543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"87'5","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.912832+0000","last_change":"2024-04-26T21:52:50.107403+0000","last_active":"2024-04-26T21:54:11.912832+0000","last_peered":"2024-04-26T21:54:11.912832+0000","last_clean":"2024-04-26T21:54:11.912832+0000","last_became_active":"2024-04-26T21:52:50.107126+0000","last_became_peered":"2024-04-26T21:52:50.107126+0000","last_unstale":"2024-04-26T21:54:11.912832+0000","last_undegraded":"2024-04-26T21:54:11.912832+0000","last_fullsized":"2024-04-26T21:54:11.912832+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:52:34.985448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.916814+0000","last_change":"2024-04-26T21:52:50.783674+0000","last_active":"2024-04-26T21:54:11.916814+0000","last_peered":"2024-04-26T21:54:11.916814+0000","last_clean":"2024-04-26T21:54:11.916814+0000","last_became_active":"2024-04-26T21:52:50.782283+0000","last_became_peered":"2024-04-26T21:52:50.782283+0000","last_unstale":"2024-04-26T21:54:11.916814+0000","last_undegraded":"2024-04-26T21:54:11.916814+0000","last_fullsized":"2024-04-26T21:54:11.916814+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:41:03.144601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"87'5","reported_seq":126,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:20.968685+0000","last_change":"2024-04-26T21:54:11.949461+0000","last_active":"2024-04-26T21:54:20.968685+0000","last_peered":"2024-04-26T21:54:20.968685+0000","last_clean":"2024-04-26T21:54:20.968685+0000","last_became_active":"2024-04-26T21:54:11.936983+0000","last_became_peered":"2024-04-26T21:54:11.936983+0000","last_unstale":"2024-04-26T21:54:20.968685+0000","last_undegraded":"2024-04-26T21:54:20.968685+0000","last_fullsized":"2024-04-26T21:54:20.968685+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:36:40.757999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":85,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":622,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.912191+0000","last_change":"2024-04-26T21:53:11.575919+0000","last_active":"2024-04-26T21:54:11.912191+0000","last_peered":"2024-04-26T21:54:11.912191+0000","last_clean":"2024-04-26T21:54:11.912191+0000","last_became_active":"2024-04-26T21:53:11.574898+0000","last_became_peered":"2024-04-26T21:53:11.574898+0000","last_unstale":"2024-04-26T21:54:11.912191+0000","last_undegraded":"2024-04-26T21:54:11.912191+0000","last_fullsized":"2024-04-26T21:54:11.912191+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:44:16.323430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,7],"acting":[8,9,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"87'7","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:12.396707+0000","last_change":"2024-04-26T21:53:10.938377+0000","last_active":"2024-04-26T21:54:12.396707+0000","last_peered":"2024-04-26T21:54:12.396707+0000","last_clean":"2024-04-26T21:54:12.396707+0000","last_became_active":"2024-04-26T21:53:10.921998+0000","last_became_peered":"2024-04-26T21:53:10.921998+0000","last_unstale":"2024-04-26T21:54:12.396707+0000","last_undegraded":"2024-04-26T21:54:12.396707+0000","last_fullsized":"2024-04-26T21:54:12.396707+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:51:20.389344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,6],"acting":[10,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.918891+0000","last_change":"2024-04-26T21:53:10.931444+0000","last_active":"2024-04-26T21:54:11.918891+0000","last_peered":"2024-04-26T21:54:11.918891+0000","last_clean":"2024-04-26T21:54:11.918891+0000","last_became_active":"2024-04-26T21:53:10.930761+0000","last_became_peered":"2024-04-26T21:53:10.930761+0000","last_unstale":"2024-04-26T21:54:11.918891+0000","last_undegraded":"2024-04-26T21:54:11.918891+0000","last_fullsized":"2024-04-26T21:54:11.918891+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:56:17.803811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,3],"acting":[5,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"87'3","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:12.395172+0000","last_change":"2024-04-26T21:52:50.107564+0000","last_active":"2024-04-26T21:54:12.395172+0000","last_peered":"2024-04-26T21:54:12.395172+0000","last_clean":"2024-04-26T21:54:12.395172+0000","last_became_active":"2024-04-26T21:52:50.106653+0000","last_became_peered":"2024-04-26T21:52:50.106653+0000","last_unstale":"2024-04-26T21:54:12.395172+0000","last_undegraded":"2024-04-26T21:54:12.395172+0000","last_fullsized":"2024-04-26T21:54:12.395172+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:05:36.594729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.907464+0000","last_change":"2024-04-26T21:52:50.780279+0000","last_active":"2024-04-26T21:54:11.907464+0000","last_peered":"2024-04-26T21:54:11.907464+0000","last_clean":"2024-04-26T21:54:11.907464+0000","last_became_active":"2024-04-26T21:52:50.779950+0000","last_became_peered":"2024-04-26T21:52:50.779950+0000","last_unstale":"2024-04-26T21:54:11.907464+0000","last_undegraded":"2024-04-26T21:54:11.907464+0000","last_fullsized":"2024-04-26T21:54:11.907464+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:11:36.282319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"87'3","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.917399+0000","last_change":"2024-04-26T21:52:50.106269+0000","last_active":"2024-04-26T21:54:11.917399+0000","last_peered":"2024-04-26T21:54:11.917399+0000","last_clean":"2024-04-26T21:54:11.917399+0000","last_became_active":"2024-04-26T21:52:50.105948+0000","last_became_peered":"2024-04-26T21:52:50.105948+0000","last_unstale":"2024-04-26T21:54:11.917399+0000","last_undegraded":"2024-04-26T21:54:11.917399+0000","last_fullsized":"2024-04-26T21:54:11.917399+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:39:31.024362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.905248+0000","last_change":"2024-04-26T21:52:50.778938+0000","last_active":"2024-04-26T21:54:11.905248+0000","last_peered":"2024-04-26T21:54:11.905248+0000","last_clean":"2024-04-26T21:54:11.905248+0000","last_became_active":"2024-04-26T21:52:50.778590+0000","last_became_peered":"2024-04-26T21:52:50.778590+0000","last_unstale":"2024-04-26T21:54:11.905248+0000","last_undegraded":"2024-04-26T21:54:11.905248+0000","last_fullsized":"2024-04-26T21:54:11.905248+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:03:10.325793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"87'4","reported_seq":198,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:22.973131+0000","last_change":"2024-04-26T21:52:50.107243+0000","last_active":"2024-04-26T21:54:22.973131+0000","last_peered":"2024-04-26T21:54:22.973131+0000","last_clean":"2024-04-26T21:54:22.973131+0000","last_became_active":"2024-04-26T21:52:50.106729+0000","last_became_peered":"2024-04-26T21:52:50.106729+0000","last_unstale":"2024-04-26T21:54:22.973131+0000","last_undegraded":"2024-04-26T21:54:22.973131+0000","last_fullsized":"2024-04-26T21:54:22.973131+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:28:11.197997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":171,"num_read_kb":171,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.909218+0000","last_change":"2024-04-26T21:52:50.776906+0000","last_active":"2024-04-26T21:54:11.909218+0000","last_peered":"2024-04-26T21:54:11.909218+0000","last_clean":"2024-04-26T21:54:11.909218+0000","last_became_active":"2024-04-26T21:52:50.776567+0000","last_became_peered":"2024-04-26T21:52:50.776567+0000","last_unstale":"2024-04-26T21:54:11.909218+0000","last_undegraded":"2024-04-26T21:54:11.909218+0000","last_fullsized":"2024-04-26T21:54:11.909218+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:51:39.512572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"87'4","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.923209+0000","last_change":"2024-04-26T21:52:50.109792+0000","last_active":"2024-04-26T21:54:11.923209+0000","last_peered":"2024-04-26T21:54:11.923209+0000","last_clean":"2024-04-26T21:54:11.923209+0000","last_became_active":"2024-04-26T21:52:50.106662+0000","last_became_peered":"2024-04-26T21:52:50.106662+0000","last_unstale":"2024-04-26T21:54:11.923209+0000","last_undegraded":"2024-04-26T21:54:11.923209+0000","last_fullsized":"2024-04-26T21:54:11.923209+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:41:27.390780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:12.034290+0000","last_change":"2024-04-26T21:52:50.782798+0000","last_active":"2024-04-26T21:54:12.034290+0000","last_peered":"2024-04-26T21:54:12.034290+0000","last_clean":"2024-04-26T21:54:12.034290+0000","last_became_active":"2024-04-26T21:52:50.781975+0000","last_became_peered":"2024-04-26T21:52:50.781975+0000","last_unstale":"2024-04-26T21:54:12.034290+0000","last_undegraded":"2024-04-26T21:54:12.034290+0000","last_fullsized":"2024-04-26T21:54:12.034290+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:13:07.261095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"87'5","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.917071+0000","last_change":"2024-04-26T21:52:50.099296+0000","last_active":"2024-04-26T21:54:11.917071+0000","last_peered":"2024-04-26T21:54:11.917071+0000","last_clean":"2024-04-26T21:54:11.917071+0000","last_became_active":"2024-04-26T21:52:50.098940+0000","last_became_peered":"2024-04-26T21:52:50.098940+0000","last_unstale":"2024-04-26T21:54:11.917071+0000","last_undegraded":"2024-04-26T21:54:11.917071+0000","last_fullsized":"2024-04-26T21:54:11.917071+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:49:15.765474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.909449+0000","last_change":"2024-04-26T21:52:50.783341+0000","last_active":"2024-04-26T21:54:11.909449+0000","last_peered":"2024-04-26T21:54:11.909449+0000","last_clean":"2024-04-26T21:54:11.909449+0000","last_became_active":"2024-04-26T21:52:50.783025+0000","last_became_peered":"2024-04-26T21:52:50.783025+0000","last_unstale":"2024-04-26T21:54:11.909449+0000","last_undegraded":"2024-04-26T21:54:11.909449+0000","last_fullsized":"2024-04-26T21:54:11.909449+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:54:06.070550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"89'29","reported_seq":228,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:22.972325+0000","last_change":"2024-04-26T21:52:50.106617+0000","last_active":"2024-04-26T21:54:22.972325+0000","last_peered":"2024-04-26T21:54:22.972325+0000","last_clean":"2024-04-26T21:54:22.972325+0000","last_became_active":"2024-04-26T21:52:50.106161+0000","last_became_peered":"2024-04-26T21:52:50.106161+0000","last_unstale":"2024-04-26T21:54:22.972325+0000","last_undegraded":"2024-04-26T21:54:22.972325+0000","last_fullsized":"2024-04-26T21:54:22.972325+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:52:07.977749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":50951,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":176,"num_read_kb":138,"num_write":31,"num_write_kb":59,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.912822+0000","last_change":"2024-04-26T21:52:50.794567+0000","last_active":"2024-04-26T21:54:11.912822+0000","last_peered":"2024-04-26T21:54:11.912822+0000","last_clean":"2024-04-26T21:54:11.912822+0000","last_became_active":"2024-04-26T21:52:50.794245+0000","last_became_peered":"2024-04-26T21:52:50.794245+0000","last_unstale":"2024-04-26T21:54:11.912822+0000","last_undegraded":"2024-04-26T21:54:11.912822+0000","last_fullsized":"2024-04-26T21:54:11.912822+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:58:11.501104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"87'9","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.910158+0000","last_change":"2024-04-26T21:52:50.102156+0000","last_active":"2024-04-26T21:54:11.910158+0000","last_peered":"2024-04-26T21:54:11.910158+0000","last_clean":"2024-04-26T21:54:11.910158+0000","last_became_active":"2024-04-26T21:52:50.101721+0000","last_became_peered":"2024-04-26T21:52:50.101721+0000","last_unstale":"2024-04-26T21:54:11.910158+0000","last_undegraded":"2024-04-26T21:54:11.910158+0000","last_fullsized":"2024-04-26T21:54:11.910158+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:48:42.081874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:12.395687+0000","last_change":"2024-04-26T21:52:50.789199+0000","last_active":"2024-04-26T21:54:12.395687+0000","last_peered":"2024-04-26T21:54:12.395687+0000","last_clean":"2024-04-26T21:54:12.395687+0000","last_became_active":"2024-04-26T21:52:50.788868+0000","last_became_peered":"2024-04-26T21:52:50.788868+0000","last_unstale":"2024-04-26T21:54:12.395687+0000","last_undegraded":"2024-04-26T21:54:12.395687+0000","last_fullsized":"2024-04-26T21:54:12.395687+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:29:55.755564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"87'5","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.918157+0000","last_change":"2024-04-26T21:52:50.104920+0000","last_active":"2024-04-26T21:54:11.918157+0000","last_peered":"2024-04-26T21:54:11.918157+0000","last_clean":"2024-04-26T21:54:11.918157+0000","last_became_active":"2024-04-26T21:52:50.104579+0000","last_became_peered":"2024-04-26T21:52:50.104579+0000","last_unstale":"2024-04-26T21:54:11.918157+0000","last_undegraded":"2024-04-26T21:54:11.918157+0000","last_fullsized":"2024-04-26T21:54:11.918157+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:09:35.760918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.923266+0000","last_change":"2024-04-26T21:52:50.787016+0000","last_active":"2024-04-26T21:54:11.923266+0000","last_peered":"2024-04-26T21:54:11.923266+0000","last_clean":"2024-04-26T21:54:11.923266+0000","last_became_active":"2024-04-26T21:52:50.778439+0000","last_became_peered":"2024-04-26T21:52:50.778439+0000","last_unstale":"2024-04-26T21:54:11.923266+0000","last_undegraded":"2024-04-26T21:54:11.923266+0000","last_fullsized":"2024-04-26T21:54:11.923266+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:13:35.735480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"87'5","reported_seq":116,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:22.783740+0000","last_change":"2024-04-26T21:52:50.110175+0000","last_active":"2024-04-26T21:54:22.783740+0000","last_peered":"2024-04-26T21:54:22.783740+0000","last_clean":"2024-04-26T21:54:22.783740+0000","last_became_active":"2024-04-26T21:52:50.109779+0000","last_became_peered":"2024-04-26T21:52:50.109779+0000","last_unstale":"2024-04-26T21:54:22.783740+0000","last_undegraded":"2024-04-26T21:54:22.783740+0000","last_fullsized":"2024-04-26T21:54:22.783740+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:06:46.756327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":89,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.922431+0000","last_change":"2024-04-26T21:52:50.784199+0000","last_active":"2024-04-26T21:54:11.922431+0000","last_peered":"2024-04-26T21:54:11.922431+0000","last_clean":"2024-04-26T21:54:11.922431+0000","last_became_active":"2024-04-26T21:52:50.783820+0000","last_became_peered":"2024-04-26T21:52:50.783820+0000","last_unstale":"2024-04-26T21:54:11.922431+0000","last_undegraded":"2024-04-26T21:54:11.922431+0000","last_fullsized":"2024-04-26T21:54:11.922431+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:45:24.776722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"87'17","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.912583+0000","last_change":"2024-04-26T21:52:50.095685+0000","last_active":"2024-04-26T21:54:11.912583+0000","last_peered":"2024-04-26T21:54:11.912583+0000","last_clean":"2024-04-26T21:54:11.912583+0000","last_became_active":"2024-04-26T21:52:50.095298+0000","last_became_peered":"2024-04-26T21:52:50.095298+0000","last_unstale":"2024-04-26T21:54:11.912583+0000","last_undegraded":"2024-04-26T21:54:11.912583+0000","last_fullsized":"2024-04-26T21:54:11.912583+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:47:26.061531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.917725+0000","last_change":"2024-04-26T21:52:50.786607+0000","last_active":"2024-04-26T21:54:11.917725+0000","last_peered":"2024-04-26T21:54:11.917725+0000","last_clean":"2024-04-26T21:54:11.917725+0000","last_became_active":"2024-04-26T21:52:50.786261+0000","last_became_peered":"2024-04-26T21:52:50.786261+0000","last_unstale":"2024-04-26T21:54:11.917725+0000","last_undegraded":"2024-04-26T21:54:11.917725+0000","last_fullsized":"2024-04-26T21:54:11.917725+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:08:45.902260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"87'2","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.913774+0000","last_change":"2024-04-26T21:52:50.095782+0000","last_active":"2024-04-26T21:54:11.913774+0000","last_peered":"2024-04-26T21:54:11.913774+0000","last_clean":"2024-04-26T21:54:11.913774+0000","last_became_active":"2024-04-26T21:52:50.093983+0000","last_became_peered":"2024-04-26T21:52:50.093983+0000","last_unstale":"2024-04-26T21:54:11.913774+0000","last_undegraded":"2024-04-26T21:54:11.913774+0000","last_fullsized":"2024-04-26T21:54:11.913774+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:18:40.103099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.920696+0000","last_change":"2024-04-26T21:52:50.791969+0000","last_active":"2024-04-26T21:54:11.920696+0000","last_peered":"2024-04-26T21:54:11.920696+0000","last_clean":"2024-04-26T21:54:11.920696+0000","last_became_active":"2024-04-26T21:52:50.790441+0000","last_became_peered":"2024-04-26T21:52:50.790441+0000","last_unstale":"2024-04-26T21:54:11.920696+0000","last_undegraded":"2024-04-26T21:54:11.920696+0000","last_fullsized":"2024-04-26T21:54:11.920696+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:11:04.964439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"87'1","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.905358+0000","last_change":"2024-04-26T21:52:50.101807+0000","last_active":"2024-04-26T21:54:11.905358+0000","last_peered":"2024-04-26T21:54:11.905358+0000","last_clean":"2024-04-26T21:54:11.905358+0000","last_became_active":"2024-04-26T21:52:50.101329+0000","last_became_peered":"2024-04-26T21:52:50.101329+0000","last_unstale":"2024-04-26T21:54:11.905358+0000","last_undegraded":"2024-04-26T21:54:11.905358+0000","last_fullsized":"2024-04-26T21:54:11.905358+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:54:47.788172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"87'1","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:12.397539+0000","last_change":"2024-04-26T21:52:50.780872+0000","last_active":"2024-04-26T21:54:12.397539+0000","last_peered":"2024-04-26T21:54:12.397539+0000","last_clean":"2024-04-26T21:54:12.397539+0000","last_became_active":"2024-04-26T21:52:50.778757+0000","last_became_peered":"2024-04-26T21:52:50.778757+0000","last_unstale":"2024-04-26T21:54:12.397539+0000","last_undegraded":"2024-04-26T21:54:12.397539+0000","last_fullsized":"2024-04-26T21:54:12.397539+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:07:21.595091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"87'5","reported_seq":118,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:25.777565+0000","last_change":"2024-04-26T21:52:50.106886+0000","last_active":"2024-04-26T21:54:25.777565+0000","last_peered":"2024-04-26T21:54:25.777565+0000","last_clean":"2024-04-26T21:54:25.777565+0000","last_became_active":"2024-04-26T21:52:50.106150+0000","last_became_peered":"2024-04-26T21:52:50.106150+0000","last_unstale":"2024-04-26T21:54:25.777565+0000","last_undegraded":"2024-04-26T21:54:25.777565+0000","last_fullsized":"2024-04-26T21:54:25.777565+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:20:54.920416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":90,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:12.397142+0000","last_change":"2024-04-26T21:53:11.069186+0000","last_active":"2024-04-26T21:54:12.397142+0000","last_peered":"2024-04-26T21:54:12.397142+0000","last_clean":"2024-04-26T21:54:12.397142+0000","last_became_active":"2024-04-26T21:53:11.068418+0000","last_became_peered":"2024-04-26T21:53:11.068418+0000","last_unstale":"2024-04-26T21:54:12.397142+0000","last_undegraded":"2024-04-26T21:54:12.397142+0000","last_fullsized":"2024-04-26T21:54:12.397142+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:57:59.663173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.917667+0000","last_change":"2024-04-26T21:52:50.785257+0000","last_active":"2024-04-26T21:54:11.917667+0000","last_peered":"2024-04-26T21:54:11.917667+0000","last_clean":"2024-04-26T21:54:11.917667+0000","last_became_active":"2024-04-26T21:52:50.784906+0000","last_became_peered":"2024-04-26T21:52:50.784906+0000","last_unstale":"2024-04-26T21:54:11.917667+0000","last_undegraded":"2024-04-26T21:54:11.917667+0000","last_fullsized":"2024-04-26T21:54:11.917667+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:04:50.921663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"87'5","reported_seq":115,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:22.782555+0000","last_change":"2024-04-26T21:52:50.102425+0000","last_active":"2024-04-26T21:54:22.782555+0000","last_peered":"2024-04-26T21:54:22.782555+0000","last_clean":"2024-04-26T21:54:22.782555+0000","last_became_active":"2024-04-26T21:52:50.101703+0000","last_became_peered":"2024-04-26T21:52:50.101703+0000","last_unstale":"2024-04-26T21:54:22.782555+0000","last_undegraded":"2024-04-26T21:54:22.782555+0000","last_fullsized":"2024-04-26T21:54:22.782555+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:13:39.910519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":87,"num_read_kb":87,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.907705+0000","last_change":"2024-04-26T21:52:50.775036+0000","last_active":"2024-04-26T21:54:11.907705+0000","last_peered":"2024-04-26T21:54:11.907705+0000","last_clean":"2024-04-26T21:54:11.907705+0000","last_became_active":"2024-04-26T21:52:50.773347+0000","last_became_peered":"2024-04-26T21:52:50.773347+0000","last_unstale":"2024-04-26T21:54:11.907705+0000","last_undegraded":"2024-04-26T21:54:11.907705+0000","last_fullsized":"2024-04-26T21:54:11.907705+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:59:55.303014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"87'6","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.934145+0000","last_change":"2024-04-26T21:52:50.109437+0000","last_active":"2024-04-26T21:54:11.934145+0000","last_peered":"2024-04-26T21:54:11.934145+0000","last_clean":"2024-04-26T21:54:11.934145+0000","last_became_active":"2024-04-26T21:52:50.108605+0000","last_became_peered":"2024-04-26T21:52:50.108605+0000","last_unstale":"2024-04-26T21:54:11.934145+0000","last_undegraded":"2024-04-26T21:54:11.934145+0000","last_fullsized":"2024-04-26T21:54:11.934145+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:31:04.208334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":350,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.922023+0000","last_change":"2024-04-26T21:51:11.498577+0000","last_active":"2024-04-26T21:54:11.922023+0000","last_peered":"2024-04-26T21:54:11.922023+0000","last_clean":"2024-04-26T21:54:11.922023+0000","last_became_active":"2024-04-26T21:51:11.497422+0000","last_became_peered":"2024-04-26T21:51:11.497422+0000","last_unstale":"2024-04-26T21:54:11.922023+0000","last_undegraded":"2024-04-26T21:54:11.922023+0000","last_fullsized":"2024-04-26T21:54:11.922023+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:46:29.789400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:46:29.789400+0000","last_clean_scrub_stamp":"2024-04-26T21:46:29.789400+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:54:13.450359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:12.394977+0000","last_change":"2024-04-26T21:52:50.794270+0000","last_active":"2024-04-26T21:54:12.394977+0000","last_peered":"2024-04-26T21:54:12.394977+0000","last_clean":"2024-04-26T21:54:12.394977+0000","last_became_active":"2024-04-26T21:52:50.793966+0000","last_became_peered":"2024-04-26T21:52:50.793966+0000","last_unstale":"2024-04-26T21:54:12.394977+0000","last_undegraded":"2024-04-26T21:54:12.394977+0000","last_fullsized":"2024-04-26T21:54:12.394977+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:01:37.668466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"87'4","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.921875+0000","last_change":"2024-04-26T21:52:50.102700+0000","last_active":"2024-04-26T21:54:11.921875+0000","last_peered":"2024-04-26T21:54:11.921875+0000","last_clean":"2024-04-26T21:54:11.921875+0000","last_became_active":"2024-04-26T21:52:50.101267+0000","last_became_peered":"2024-04-26T21:52:50.101267+0000","last_unstale":"2024-04-26T21:54:11.921875+0000","last_undegraded":"2024-04-26T21:54:11.921875+0000","last_fullsized":"2024-04-26T21:54:11.921875+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:41:21.727411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"87'4","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.913859+0000","last_change":"2024-04-26T21:52:50.102191+0000","last_active":"2024-04-26T21:54:11.913859+0000","last_peered":"2024-04-26T21:54:11.913859+0000","last_clean":"2024-04-26T21:54:11.913859+0000","last_became_active":"2024-04-26T21:52:50.101307+0000","last_became_peered":"2024-04-26T21:52:50.101307+0000","last_unstale":"2024-04-26T21:54:11.913859+0000","last_undegraded":"2024-04-26T21:54:11.913859+0000","last_fullsized":"2024-04-26T21:54:11.913859+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:59:12.465291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:12.395409+0000","last_change":"2024-04-26T21:52:50.792020+0000","last_active":"2024-04-26T21:54:12.395409+0000","last_peered":"2024-04-26T21:54:12.395409+0000","last_clean":"2024-04-26T21:54:12.395409+0000","last_became_active":"2024-04-26T21:52:50.791677+0000","last_became_peered":"2024-04-26T21:52:50.791677+0000","last_unstale":"2024-04-26T21:54:12.395409+0000","last_undegraded":"2024-04-26T21:54:12.395409+0000","last_fullsized":"2024-04-26T21:54:12.395409+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:11:42.218294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"89'12","reported_seq":51,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.936807+0000","last_change":"2024-04-26T21:54:11.936807+0000","last_active":"2024-04-26T21:54:11.936807+0000","last_peered":"2024-04-26T21:54:11.936807+0000","last_clean":"2024-04-26T21:54:11.936807+0000","last_became_active":"2024-04-26T21:54:11.923482+0000","last_became_peered":"2024-04-26T21:54:11.923482+0000","last_unstale":"2024-04-26T21:54:11.936807+0000","last_undegraded":"2024-04-26T21:54:11.936807+0000","last_fullsized":"2024-04-26T21:54:11.936807+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:15:46.945453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.908047+0000","last_change":"2024-04-26T21:52:50.779775+0000","last_active":"2024-04-26T21:54:11.908047+0000","last_peered":"2024-04-26T21:54:11.908047+0000","last_clean":"2024-04-26T21:54:11.908047+0000","last_became_active":"2024-04-26T21:52:50.778857+0000","last_became_peered":"2024-04-26T21:52:50.778857+0000","last_unstale":"2024-04-26T21:54:11.908047+0000","last_undegraded":"2024-04-26T21:54:11.908047+0000","last_fullsized":"2024-04-26T21:54:11.908047+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:59:50.946578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"87'7","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.934291+0000","last_change":"2024-04-26T21:52:50.113027+0000","last_active":"2024-04-26T21:54:11.934291+0000","last_peered":"2024-04-26T21:54:11.934291+0000","last_clean":"2024-04-26T21:54:11.934291+0000","last_became_active":"2024-04-26T21:52:50.112637+0000","last_became_peered":"2024-04-26T21:52:50.112637+0000","last_unstale":"2024-04-26T21:54:11.934291+0000","last_undegraded":"2024-04-26T21:54:11.934291+0000","last_fullsized":"2024-04-26T21:54:11.934291+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:09:47.798184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:12.034057+0000","last_change":"2024-04-26T21:52:50.785119+0000","last_active":"2024-04-26T21:54:12.034057+0000","last_peered":"2024-04-26T21:54:12.034057+0000","last_clean":"2024-04-26T21:54:12.034057+0000","last_became_active":"2024-04-26T21:52:50.781005+0000","last_became_peered":"2024-04-26T21:52:50.781005+0000","last_unstale":"2024-04-26T21:54:12.034057+0000","last_undegraded":"2024-04-26T21:54:12.034057+0000","last_fullsized":"2024-04-26T21:54:12.034057+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:16:32.986810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"87'2","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.908291+0000","last_change":"2024-04-26T21:52:50.112263+0000","last_active":"2024-04-26T21:54:11.908291+0000","last_peered":"2024-04-26T21:54:11.908291+0000","last_clean":"2024-04-26T21:54:11.908291+0000","last_became_active":"2024-04-26T21:52:50.110649+0000","last_became_peered":"2024-04-26T21:52:50.110649+0000","last_unstale":"2024-04-26T21:54:11.908291+0000","last_undegraded":"2024-04-26T21:54:11.908291+0000","last_fullsized":"2024-04-26T21:54:11.908291+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:14:23.245000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.912952+0000","last_change":"2024-04-26T21:52:50.790674+0000","last_active":"2024-04-26T21:54:11.912952+0000","last_peered":"2024-04-26T21:54:11.912952+0000","last_clean":"2024-04-26T21:54:11.912952+0000","last_became_active":"2024-04-26T21:52:50.790286+0000","last_became_peered":"2024-04-26T21:52:50.790286+0000","last_unstale":"2024-04-26T21:54:11.912952+0000","last_undegraded":"2024-04-26T21:54:11.912952+0000","last_fullsized":"2024-04-26T21:54:11.912952+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:22:32.246321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"87'7","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.907528+0000","last_change":"2024-04-26T21:52:50.112158+0000","last_active":"2024-04-26T21:54:11.907528+0000","last_peered":"2024-04-26T21:54:11.907528+0000","last_clean":"2024-04-26T21:54:11.907528+0000","last_became_active":"2024-04-26T21:52:50.111453+0000","last_became_peered":"2024-04-26T21:52:50.111453+0000","last_unstale":"2024-04-26T21:54:11.907528+0000","last_undegraded":"2024-04-26T21:54:11.907528+0000","last_fullsized":"2024-04-26T21:54:11.907528+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:16:58.626985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.915460+0000","last_change":"2024-04-26T21:52:50.798241+0000","last_active":"2024-04-26T21:54:11.915460+0000","last_peered":"2024-04-26T21:54:11.915460+0000","last_clean":"2024-04-26T21:54:11.915460+0000","last_became_active":"2024-04-26T21:52:50.797878+0000","last_became_peered":"2024-04-26T21:52:50.797878+0000","last_unstale":"2024-04-26T21:54:11.915460+0000","last_undegraded":"2024-04-26T21:54:11.915460+0000","last_fullsized":"2024-04-26T21:54:11.915460+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:42:49.363849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"87'3","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.948118+0000","last_change":"2024-04-26T21:54:11.946852+0000","last_active":"2024-04-26T21:54:11.948118+0000","last_peered":"2024-04-26T21:54:11.948118+0000","last_clean":"2024-04-26T21:54:11.948118+0000","last_became_active":"2024-04-26T21:54:11.923399+0000","last_became_peered":"2024-04-26T21:54:11.923399+0000","last_unstale":"2024-04-26T21:54:11.948118+0000","last_undegraded":"2024-04-26T21:54:11.948118+0000","last_fullsized":"2024-04-26T21:54:11.948118+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:05:34.688374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.908182+0000","last_change":"2024-04-26T21:52:50.776333+0000","last_active":"2024-04-26T21:54:11.908182+0000","last_peered":"2024-04-26T21:54:11.908182+0000","last_clean":"2024-04-26T21:54:11.908182+0000","last_became_active":"2024-04-26T21:52:50.776001+0000","last_became_peered":"2024-04-26T21:52:50.776001+0000","last_unstale":"2024-04-26T21:54:11.908182+0000","last_undegraded":"2024-04-26T21:54:11.908182+0000","last_fullsized":"2024-04-26T21:54:11.908182+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:38:26.578928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"87'4","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.913278+0000","last_change":"2024-04-26T21:52:50.106008+0000","last_active":"2024-04-26T21:54:11.913278+0000","last_peered":"2024-04-26T21:54:11.913278+0000","last_clean":"2024-04-26T21:54:11.913278+0000","last_became_active":"2024-04-26T21:52:50.105565+0000","last_became_peered":"2024-04-26T21:52:50.105565+0000","last_unstale":"2024-04-26T21:54:11.913278+0000","last_undegraded":"2024-04-26T21:54:11.913278+0000","last_fullsized":"2024-04-26T21:54:11.913278+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:15:17.355461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.923264+0000","last_change":"2024-04-26T21:52:50.776051+0000","last_active":"2024-04-26T21:54:11.923264+0000","last_peered":"2024-04-26T21:54:11.923264+0000","last_clean":"2024-04-26T21:54:11.923264+0000","last_became_active":"2024-04-26T21:52:50.775716+0000","last_became_peered":"2024-04-26T21:52:50.775716+0000","last_unstale":"2024-04-26T21:54:11.923264+0000","last_undegraded":"2024-04-26T21:54:11.923264+0000","last_fullsized":"2024-04-26T21:54:11.923264+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:34:21.110773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"87'6","reported_seq":118,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:23.778125+0000","last_change":"2024-04-26T21:52:50.111280+0000","last_active":"2024-04-26T21:54:23.778125+0000","last_peered":"2024-04-26T21:54:23.778125+0000","last_clean":"2024-04-26T21:54:23.778125+0000","last_became_active":"2024-04-26T21:52:50.110972+0000","last_became_peered":"2024-04-26T21:52:50.110972+0000","last_unstale":"2024-04-26T21:54:23.778125+0000","last_undegraded":"2024-04-26T21:54:23.778125+0000","last_fullsized":"2024-04-26T21:54:23.778125+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:41:48.203134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1561,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":89,"num_read_kb":2,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.923193+0000","last_change":"2024-04-26T21:52:50.777488+0000","last_active":"2024-04-26T21:54:11.923193+0000","last_peered":"2024-04-26T21:54:11.923193+0000","last_clean":"2024-04-26T21:54:11.923193+0000","last_became_active":"2024-04-26T21:52:50.777162+0000","last_became_peered":"2024-04-26T21:52:50.777162+0000","last_unstale":"2024-04-26T21:54:11.923193+0000","last_undegraded":"2024-04-26T21:54:11.923193+0000","last_fullsized":"2024-04-26T21:54:11.923193+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:07:22.683109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"87'3","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.916471+0000","last_change":"2024-04-26T21:52:50.104508+0000","last_active":"2024-04-26T21:54:11.916471+0000","last_peered":"2024-04-26T21:54:11.916471+0000","last_clean":"2024-04-26T21:54:11.916471+0000","last_became_active":"2024-04-26T21:52:50.103924+0000","last_became_peered":"2024-04-26T21:52:50.103924+0000","last_unstale":"2024-04-26T21:54:11.916471+0000","last_undegraded":"2024-04-26T21:54:11.916471+0000","last_fullsized":"2024-04-26T21:54:11.916471+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:39:25.861339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"87'1","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.922484+0000","last_change":"2024-04-26T21:52:50.782915+0000","last_active":"2024-04-26T21:54:11.922484+0000","last_peered":"2024-04-26T21:54:11.922484+0000","last_clean":"2024-04-26T21:54:11.922484+0000","last_became_active":"2024-04-26T21:52:50.782253+0000","last_became_peered":"2024-04-26T21:52:50.782253+0000","last_unstale":"2024-04-26T21:54:11.922484+0000","last_undegraded":"2024-04-26T21:54:11.922484+0000","last_fullsized":"2024-04-26T21:54:11.922484+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:12:51.624162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"87'5","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:12.395276+0000","last_change":"2024-04-26T21:53:11.590613+0000","last_active":"2024-04-26T21:54:12.395276+0000","last_peered":"2024-04-26T21:54:12.395276+0000","last_clean":"2024-04-26T21:54:12.395276+0000","last_became_active":"2024-04-26T21:53:11.578727+0000","last_became_peered":"2024-04-26T21:53:11.578727+0000","last_unstale":"2024-04-26T21:54:12.395276+0000","last_undegraded":"2024-04-26T21:54:12.395276+0000","last_fullsized":"2024-04-26T21:54:12.395276+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:44:38.867231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.922794+0000","last_change":"2024-04-26T21:52:50.787777+0000","last_active":"2024-04-26T21:54:11.922794+0000","last_peered":"2024-04-26T21:54:11.922794+0000","last_clean":"2024-04-26T21:54:11.922794+0000","last_became_active":"2024-04-26T21:52:50.782403+0000","last_became_peered":"2024-04-26T21:52:50.782403+0000","last_unstale":"2024-04-26T21:54:11.922794+0000","last_undegraded":"2024-04-26T21:54:11.922794+0000","last_fullsized":"2024-04-26T21:54:11.922794+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:28:58.425802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.915785+0000","last_change":"2024-04-26T21:52:50.800010+0000","last_active":"2024-04-26T21:54:11.915785+0000","last_peered":"2024-04-26T21:54:11.915785+0000","last_clean":"2024-04-26T21:54:11.915785+0000","last_became_active":"2024-04-26T21:52:50.799308+0000","last_became_peered":"2024-04-26T21:52:50.799308+0000","last_unstale":"2024-04-26T21:54:11.915785+0000","last_undegraded":"2024-04-26T21:54:11.915785+0000","last_fullsized":"2024-04-26T21:54:11.915785+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:54:25.772312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.908880+0000","last_change":"2024-04-26T21:52:50.798281+0000","last_active":"2024-04-26T21:54:11.908880+0000","last_peered":"2024-04-26T21:54:11.908880+0000","last_clean":"2024-04-26T21:54:11.908880+0000","last_became_active":"2024-04-26T21:52:50.795879+0000","last_became_peered":"2024-04-26T21:52:50.795879+0000","last_unstale":"2024-04-26T21:54:11.908880+0000","last_undegraded":"2024-04-26T21:54:11.908880+0000","last_fullsized":"2024-04-26T21:54:11.908880+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:36:00.769991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.908572+0000","last_change":"2024-04-26T21:52:50.791058+0000","last_active":"2024-04-26T21:54:11.908572+0000","last_peered":"2024-04-26T21:54:11.908572+0000","last_clean":"2024-04-26T21:54:11.908572+0000","last_became_active":"2024-04-26T21:52:50.790721+0000","last_became_peered":"2024-04-26T21:52:50.790721+0000","last_unstale":"2024-04-26T21:54:11.908572+0000","last_undegraded":"2024-04-26T21:54:11.908572+0000","last_fullsized":"2024-04-26T21:54:11.908572+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:56:03.239622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:12.033600+0000","last_change":"2024-04-26T21:52:50.788193+0000","last_active":"2024-04-26T21:54:12.033600+0000","last_peered":"2024-04-26T21:54:12.033600+0000","last_clean":"2024-04-26T21:54:12.033600+0000","last_became_active":"2024-04-26T21:52:50.786749+0000","last_became_peered":"2024-04-26T21:52:50.786749+0000","last_unstale":"2024-04-26T21:54:12.033600+0000","last_undegraded":"2024-04-26T21:54:12.033600+0000","last_fullsized":"2024-04-26T21:54:12.033600+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:22:11.277217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:12.395541+0000","last_change":"2024-04-26T21:52:50.782508+0000","last_active":"2024-04-26T21:54:12.395541+0000","last_peered":"2024-04-26T21:54:12.395541+0000","last_clean":"2024-04-26T21:54:12.395541+0000","last_became_active":"2024-04-26T21:52:50.781505+0000","last_became_peered":"2024-04-26T21:52:50.781505+0000","last_unstale":"2024-04-26T21:54:12.395541+0000","last_undegraded":"2024-04-26T21:54:12.395541+0000","last_fullsized":"2024-04-26T21:54:12.395541+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:22:57.867546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.916462+0000","last_change":"2024-04-26T21:52:50.789205+0000","last_active":"2024-04-26T21:54:11.916462+0000","last_peered":"2024-04-26T21:54:11.916462+0000","last_clean":"2024-04-26T21:54:11.916462+0000","last_became_active":"2024-04-26T21:52:50.788898+0000","last_became_peered":"2024-04-26T21:52:50.788898+0000","last_unstale":"2024-04-26T21:54:11.916462+0000","last_undegraded":"2024-04-26T21:54:11.916462+0000","last_fullsized":"2024-04-26T21:54:11.916462+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:18:51.401968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.909237+0000","last_change":"2024-04-26T21:52:50.793451+0000","last_active":"2024-04-26T21:54:11.909237+0000","last_peered":"2024-04-26T21:54:11.909237+0000","last_clean":"2024-04-26T21:54:11.909237+0000","last_became_active":"2024-04-26T21:52:50.791696+0000","last_became_peered":"2024-04-26T21:52:50.791696+0000","last_unstale":"2024-04-26T21:54:11.909237+0000","last_undegraded":"2024-04-26T21:54:11.909237+0000","last_fullsized":"2024-04-26T21:54:11.909237+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:51:43.543500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.907944+0000","last_change":"2024-04-26T21:53:10.919691+0000","last_active":"2024-04-26T21:54:11.907944+0000","last_peered":"2024-04-26T21:54:11.907944+0000","last_clean":"2024-04-26T21:54:11.907944+0000","last_became_active":"2024-04-26T21:53:10.919262+0000","last_became_peered":"2024-04-26T21:53:10.919262+0000","last_unstale":"2024-04-26T21:54:11.907944+0000","last_undegraded":"2024-04-26T21:54:11.907944+0000","last_fullsized":"2024-04-26T21:54:11.907944+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:15:35.719661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,8,2],"acting":[3,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":15,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:12.395038+0000","last_change":"2024-04-26T21:53:10.914854+0000","last_active":"2024-04-26T21:54:12.395038+0000","last_peered":"2024-04-26T21:54:12.395038+0000","last_clean":"2024-04-26T21:54:12.395038+0000","last_became_active":"2024-04-26T21:53:10.914458+0000","last_became_peered":"2024-04-26T21:53:10.914458+0000","last_unstale":"2024-04-26T21:54:12.395038+0000","last_undegraded":"2024-04-26T21:54:12.395038+0000","last_fullsized":"2024-04-26T21:54:12.395038+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:26:41.538053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,3],"acting":[6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.907469+0000","last_change":"2024-04-26T21:52:50.794580+0000","last_active":"2024-04-26T21:54:11.907469+0000","last_peered":"2024-04-26T21:54:11.907469+0000","last_clean":"2024-04-26T21:54:11.907469+0000","last_became_active":"2024-04-26T21:52:50.792398+0000","last_became_peered":"2024-04-26T21:52:50.792398+0000","last_unstale":"2024-04-26T21:54:11.907469+0000","last_undegraded":"2024-04-26T21:54:11.907469+0000","last_fullsized":"2024-04-26T21:54:11.907469+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:21:38.913644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.909449+0000","last_change":"2024-04-26T21:53:11.580576+0000","last_active":"2024-04-26T21:54:11.909449+0000","last_peered":"2024-04-26T21:54:11.909449+0000","last_clean":"2024-04-26T21:54:11.909449+0000","last_became_active":"2024-04-26T21:53:11.578632+0000","last_became_peered":"2024-04-26T21:53:11.578632+0000","last_unstale":"2024-04-26T21:54:11.909449+0000","last_undegraded":"2024-04-26T21:54:11.909449+0000","last_fullsized":"2024-04-26T21:54:11.909449+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:30:24.412417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,8],"acting":[1,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:12.395881+0000","last_change":"2024-04-26T21:52:50.792166+0000","last_active":"2024-04-26T21:54:12.395881+0000","last_peered":"2024-04-26T21:54:12.395881+0000","last_clean":"2024-04-26T21:54:12.395881+0000","last_became_active":"2024-04-26T21:52:50.790709+0000","last_became_peered":"2024-04-26T21:52:50.790709+0000","last_unstale":"2024-04-26T21:54:12.395881+0000","last_undegraded":"2024-04-26T21:54:12.395881+0000","last_fullsized":"2024-04-26T21:54:12.395881+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:05:48.200573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.915301+0000","last_change":"2024-04-26T21:53:10.930526+0000","last_active":"2024-04-26T21:54:11.915301+0000","last_peered":"2024-04-26T21:54:11.915301+0000","last_clean":"2024-04-26T21:54:11.915301+0000","last_became_active":"2024-04-26T21:53:10.928850+0000","last_became_peered":"2024-04-26T21:53:10.928850+0000","last_unstale":"2024-04-26T21:54:11.915301+0000","last_undegraded":"2024-04-26T21:54:11.915301+0000","last_fullsized":"2024-04-26T21:54:11.915301+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:54:03.420855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":57764,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":894,"num_read_kb":511,"num_write":237,"num_write_kb":123,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":622,"num_keys_recovered":7,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":540672,"data_stored":173292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":204,"ondisk_log_size":204,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":82,"seq":352187318307,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1015808,"data_stored":484903,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":74,"seq":317827579946,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28688,"kb_used_data":1668,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716464,"statfs":{"total":95995035648,"available":95965659136,"internally_reserved":0,"allocated":1708032,"data_stored":1126492,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":69,"seq":296352743474,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36356,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708796,"statfs":{"total":95995035648,"available":95957807104,"internally_reserved":0,"allocated":1040384,"data_stored":484442,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":61,"seq":261993005114,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36352,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708800,"statfs":{"total":95995035648,"available":95957811200,"internally_reserved":0,"allocated":1036288,"data_stored":484820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299455,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28620,"kb_used_data":1596,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716532,"statfs":{"total":95995035648,"available":95965728768,"internally_reserved":0,"allocated":1634304,"data_stored":1076947,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462983,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36340,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708812,"statfs":{"total":95995035648,"available":95957823488,"internally_reserved":0,"allocated":1024000,"data_stored":484159,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574953,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28076,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717076,"statfs":{"total":95995035648,"available":95966285824,"internally_reserved":0,"allocated":1077248,"data_stored":536023,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738481,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36364,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708788,"statfs":{"total":95995035648,"available":95957798912,"internally_reserved":0,"allocated":1044480,"data_stored":485889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411427,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36408,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708744,"statfs":{"total":95995035648,"available":95957753856,"internally_reserved":0,"allocated":1089536,"data_stored":536012,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182491,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1040384,"data_stored":483984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920854,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36352,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708800,"statfs":{"total":95995035648,"available":95957811200,"internally_reserved":0,"allocated":1032192,"data_stored":484642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724620,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36948,"kb_used_data":1608,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708204,"statfs":{"total":95995035648,"available":95957200896,"internally_reserved":0,"allocated":1646592,"data_stored":1076625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":52703,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":52692,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1322,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":52687,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T21:54:29.341 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-04-26T21:54:29.341 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-26T21:54:29.341 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-04-26T21:54:29.341 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid a1780950-0415-11ef-bc93-c7b262605968 -- ceph health --format=json 2024-04-26T21:54:29.814 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/a1780950-0415-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:54:30.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:54:29 smithi076 ceph-mon[31823]: from='client.15000 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T21:54:30.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:54:29 smithi072 ceph-mon[28138]: from='client.15000 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T21:54:30.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:54:29 smithi164 ceph-mon[31916]: from='client.15000 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T21:54:31.298 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:54:30 smithi072 ceph-mon[28138]: pgmap v421: 97 pgs: 97 active+clean; 633 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-26T21:54:31.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:54:30 smithi076 ceph-mon[31823]: pgmap v421: 97 pgs: 97 active+clean; 633 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-26T21:54:31.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:54:30 smithi164 ceph-mon[31916]: pgmap v421: 97 pgs: 97 active+clean; 633 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-26T21:54:32.165 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:32.163+0000 7fbb80a00700 1 -- 172.21.15.72:0/201556422 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbb7c106000 msgr2=0x7fbb7c112510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:32.165 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:32.163+0000 7fbb80a00700 1 --2- 172.21.15.72:0/201556422 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbb7c106000 0x7fbb7c112510 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fbb7400b0a0 tx=0x7fbb7400b3b0 comp rx=0 tx=0).stop 2024-04-26T21:54:32.165 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:32.163+0000 7fbb80a00700 1 -- 172.21.15.72:0/201556422 shutdown_connections 2024-04-26T21:54:32.165 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:32.163+0000 7fbb80a00700 1 --2- 172.21.15.72:0/201556422 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbb7c106000 0x7fbb7c112510 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:32.165 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:32.163+0000 7fbb80a00700 1 --2- 172.21.15.72:0/201556422 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbb7c10b640 0x7fbb7c10ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:32.165 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:32.163+0000 7fbb80a00700 1 --2- 172.21.15.72:0/201556422 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbb7c105640 0x7fbb7c105ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:32.165 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:32.163+0000 7fbb80a00700 1 -- 172.21.15.72:0/201556422 >> 172.21.15.72:0/201556422 conn(0x7fbb7c101130 msgr2=0x7fbb7c103550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:54:32.165 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:32.163+0000 7fbb80a00700 1 -- 172.21.15.72:0/201556422 shutdown_connections 2024-04-26T21:54:32.165 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:32.164+0000 7fbb80a00700 1 -- 172.21.15.72:0/201556422 wait complete. 2024-04-26T21:54:32.166 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:32.165+0000 7fbb80a00700 1 Processor -- start 2024-04-26T21:54:32.166 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:32.165+0000 7fbb80a00700 1 -- start start 2024-04-26T21:54:32.167 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:32.165+0000 7fbb80a00700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbb7c105640 0x7fbb7c077930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:32.167 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:32.165+0000 7fbb80a00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbb7c106000 0x7fbb7c077e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:32.167 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:32.165+0000 7fbb80a00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbb7c10b640 0x7fbb7c0783b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:32.168 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:32.165+0000 7fbb80a00700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fbb7c115730 con 0x7fbb7c106000 2024-04-26T21:54:32.168 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:32.165+0000 7fbb80a00700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fbb7c1155b0 con 0x7fbb7c105640 2024-04-26T21:54:32.168 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:32.165+0000 7fbb80a00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fbb7c1158b0 con 0x7fbb7c10b640 2024-04-26T21:54:32.168 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:32.166+0000 7fbb79400700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbb7c106000 0x7fbb7c077e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:32.168 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:32.166+0000 7fbb79400700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbb7c106000 0x7fbb7c077e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:38898/0 (socket says 172.21.15.72:38898) 2024-04-26T21:54:32.168 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:32.166+0000 7fbb79400700 1 -- 172.21.15.72:0/3499828968 learned_addr learned my addr 172.21.15.72:0/3499828968 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:54:32.168 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:32.166+0000 7fbb79e00700 1 --2- 172.21.15.72:0/3499828968 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbb7c105640 0x7fbb7c077930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:32.168 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:32.166+0000 7fbb7a800700 1 --2- 172.21.15.72:0/3499828968 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbb7c10b640 0x7fbb7c0783b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:32.168 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:32.166+0000 7fbb79400700 1 -- 172.21.15.72:0/3499828968 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbb7c10b640 msgr2=0x7fbb7c0783b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:32.168 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:32.166+0000 7fbb79400700 1 --2- 172.21.15.72:0/3499828968 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbb7c10b640 0x7fbb7c0783b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:32.168 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:32.166+0000 7fbb79400700 1 -- 172.21.15.72:0/3499828968 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbb7c105640 msgr2=0x7fbb7c077930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:32.168 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:32.166+0000 7fbb79400700 1 --2- 172.21.15.72:0/3499828968 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbb7c105640 0x7fbb7c077930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:32.168 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:32.166+0000 7fbb79400700 1 -- 172.21.15.72:0/3499828968 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb7c1ac2b0 con 0x7fbb7c106000 2024-04-26T21:54:32.169 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:32.166+0000 7fbb79400700 1 --2- 172.21.15.72:0/3499828968 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbb7c106000 0x7fbb7c077e70 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fbb6800e9c0 tx=0x7fbb6800ed80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:54:32.169 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:32.167+0000 7fbb72a00700 1 -- 172.21.15.72:0/3499828968 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb6800cbd0 con 0x7fbb7c106000 2024-04-26T21:54:32.169 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:32.167+0000 7fbb72a00700 1 -- 172.21.15.72:0/3499828968 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbb6800cd70 con 0x7fbb7c106000 2024-04-26T21:54:32.169 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:32.167+0000 7fbb80a00700 1 -- 172.21.15.72:0/3499828968 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb7c1ac5a0 con 0x7fbb7c106000 2024-04-26T21:54:32.170 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:32.167+0000 7fbb72a00700 1 -- 172.21.15.72:0/3499828968 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb68005680 con 0x7fbb7c106000 2024-04-26T21:54:32.170 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:32.167+0000 7fbb80a00700 1 -- 172.21.15.72:0/3499828968 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb7c1aca90 con 0x7fbb7c106000 2024-04-26T21:54:32.170 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:32.168+0000 7fbb80a00700 1 -- 172.21.15.72:0/3499828968 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb7c106d80 con 0x7fbb7c106000 2024-04-26T21:54:32.171 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:32.171+0000 7fbb72a00700 1 -- 172.21.15.72:0/3499828968 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fbb68004750 con 0x7fbb7c106000 2024-04-26T21:54:32.174 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:32.173+0000 7fbb72a00700 1 --2- 172.21.15.72:0/3499828968 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fbb600a3cd0 0x7fbb600a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:32.174 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:32.173+0000 7fbb72a00700 1 -- 172.21.15.72:0/3499828968 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7fbb680d0a10 con 0x7fbb7c106000 2024-04-26T21:54:32.174 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:32.173+0000 7fbb79e00700 1 --2- 172.21.15.72:0/3499828968 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fbb600a3cd0 0x7fbb600a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:32.175 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:32.174+0000 7fbb79e00700 1 --2- 172.21.15.72:0/3499828968 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fbb600a3cd0 0x7fbb600a6190 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fbb6c009bd0 tx=0x7fbb6c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:54:32.178 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:32.178+0000 7fbb72a00700 1 -- 172.21.15.72:0/3499828968 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fbb6809d1b0 con 0x7fbb7c106000 2024-04-26T21:54:32.455 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:32.454+0000 7fbb80a00700 1 -- 172.21.15.72:0/3499828968 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fbb7c02e490 con 0x7fbb7c106000 2024-04-26T21:54:32.456 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:32.454+0000 7fbb72a00700 1 -- 172.21.15.72:0/3499828968 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7fbb6809cab0 con 0x7fbb7c106000 2024-04-26T21:54:32.456 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T21:54:32.456 INFO:teuthology.orchestra.run.smithi072.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-26T21:54:32.458 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:32.457+0000 7fbb80a00700 1 -- 172.21.15.72:0/3499828968 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fbb600a3cd0 msgr2=0x7fbb600a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:32.459 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:32.457+0000 7fbb80a00700 1 --2- 172.21.15.72:0/3499828968 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fbb600a3cd0 0x7fbb600a6190 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fbb6c009bd0 tx=0x7fbb6c009260 comp rx=0 tx=0).stop 2024-04-26T21:54:32.459 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:32.457+0000 7fbb80a00700 1 -- 172.21.15.72:0/3499828968 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbb7c106000 msgr2=0x7fbb7c077e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:32.459 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:32.457+0000 7fbb80a00700 1 --2- 172.21.15.72:0/3499828968 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbb7c106000 0x7fbb7c077e70 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fbb6800e9c0 tx=0x7fbb6800ed80 comp rx=0 tx=0).stop 2024-04-26T21:54:32.459 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:32.458+0000 7fbb80a00700 1 -- 172.21.15.72:0/3499828968 shutdown_connections 2024-04-26T21:54:32.460 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:32.458+0000 7fbb80a00700 1 --2- 172.21.15.72:0/3499828968 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fbb600a3cd0 0x7fbb600a6190 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:32.460 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:32.458+0000 7fbb80a00700 1 --2- 172.21.15.72:0/3499828968 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbb7c10b640 0x7fbb7c0783b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:32.460 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:32.458+0000 7fbb80a00700 1 --2- 172.21.15.72:0/3499828968 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbb7c105640 0x7fbb7c077930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:32.460 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:32.458+0000 7fbb80a00700 1 --2- 172.21.15.72:0/3499828968 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbb7c106000 0x7fbb7c077e70 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:32.460 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:32.458+0000 7fbb80a00700 1 -- 172.21.15.72:0/3499828968 >> 172.21.15.72:0/3499828968 conn(0x7fbb7c101130 msgr2=0x7fbb7c101980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:54:32.460 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:32.458+0000 7fbb80a00700 1 -- 172.21.15.72:0/3499828968 shutdown_connections 2024-04-26T21:54:32.460 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:32.458+0000 7fbb80a00700 1 -- 172.21.15.72:0/3499828968 wait complete. 2024-04-26T21:54:33.162 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:54:32 smithi072 ceph-mon[28138]: pgmap v422: 97 pgs: 97 active+clean; 633 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:54:33.162 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:54:32 smithi072 ceph-mon[28138]: from='client.? 172.21.15.72:0/3499828968' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-26T21:54:33.299 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-04-26T21:54:33.300 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:54:33.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:54:32 smithi076 ceph-mon[31823]: pgmap v422: 97 pgs: 97 active+clean; 633 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:54:33.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:54:32 smithi076 ceph-mon[31823]: from='client.? 172.21.15.72:0/3499828968' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-26T21:54:33.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:54:32 smithi164 ceph-mon[31916]: pgmap v422: 97 pgs: 97 active+clean; 633 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:54:33.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:54:32 smithi164 ceph-mon[31916]: from='client.? 172.21.15.72:0/3499828968' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-26T21:54:33.497 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:33.496+0000 7f410ae00640 1 -- 172.21.15.72:0/3690202576 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f410410a600 msgr2=0x7f410410caa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:33.497 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:33.496+0000 7f410ae00640 1 --2- 172.21.15.72:0/3690202576 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f410410a600 0x7f410410caa0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f40fc009900 tx=0x7f40fc02a480 comp rx=0 tx=0).stop 2024-04-26T21:54:33.497 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:33.496+0000 7f410ae00640 1 -- 172.21.15.72:0/3690202576 shutdown_connections 2024-04-26T21:54:33.497 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:33.496+0000 7f410ae00640 1 --2- 172.21.15.72:0/3690202576 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f410410a600 0x7f410410caa0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:33.498 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:33.496+0000 7f410ae00640 1 --2- 172.21.15.72:0/3690202576 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4104075330 0x7f4104075790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:33.498 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:33.496+0000 7f410ae00640 1 --2- 172.21.15.72:0/3690202576 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4104076910 0x7f4104074d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:33.498 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:33.496+0000 7f410ae00640 1 -- 172.21.15.72:0/3690202576 >> 172.21.15.72:0/3690202576 conn(0x7f41040fdcc0 msgr2=0x7f4104100120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:54:33.498 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:33.496+0000 7f410ae00640 1 -- 172.21.15.72:0/3690202576 shutdown_connections 2024-04-26T21:54:33.498 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:33.496+0000 7f410ae00640 1 -- 172.21.15.72:0/3690202576 wait complete. 2024-04-26T21:54:33.500 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:33.497+0000 7f410ae00640 1 Processor -- start 2024-04-26T21:54:33.500 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:33.497+0000 7f410ae00640 1 -- start start 2024-04-26T21:54:33.500 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:33.497+0000 7f410ae00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4104075330 0x7f410419ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:33.500 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:33.497+0000 7f410ae00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4104076910 0x7f410419b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:33.500 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:33.497+0000 7f410ae00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f410410a600 0x7f41041a2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:33.500 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:33.497+0000 7f410ae00640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f4104110470 con 0x7f4104075330 2024-04-26T21:54:33.501 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:33.497+0000 7f410ae00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f41041102f0 con 0x7f410410a600 2024-04-26T21:54:33.501 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:33.497+0000 7f410ae00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f41041105f0 con 0x7f4104076910 2024-04-26T21:54:33.501 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:33.497+0000 7f4109a00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4104075330 0x7f410419ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:33.501 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:33.497+0000 7f4109a00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4104075330 0x7f410419ab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:38926/0 (socket says 172.21.15.72:38926) 2024-04-26T21:54:33.501 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:33.497+0000 7f4109a00640 1 -- 172.21.15.72:0/197059648 learned_addr learned my addr 172.21.15.72:0/197059648 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:54:33.501 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:33.498+0000 7f4109000640 1 --2- 172.21.15.72:0/197059648 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4104076910 0x7f410419b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:33.501 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:33.498+0000 7f4109a00640 1 -- 172.21.15.72:0/197059648 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4104076910 msgr2=0x7f410419b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:33.501 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:33.498+0000 7f410a400640 1 --2- 172.21.15.72:0/197059648 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f410410a600 0x7f41041a2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:33.501 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:33.498+0000 7f4109a00640 1 --2- 172.21.15.72:0/197059648 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4104076910 0x7f410419b0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:33.501 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:33.498+0000 7f4109a00640 1 -- 172.21.15.72:0/197059648 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f410410a600 msgr2=0x7f41041a2120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:33.501 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:33.498+0000 7f4109a00640 1 --2- 172.21.15.72:0/197059648 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f410410a600 0x7f41041a2120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:33.501 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:33.498+0000 7f4109a00640 1 -- 172.21.15.72:0/197059648 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41041a2790 con 0x7f4104075330 2024-04-26T21:54:33.501 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:33.498+0000 7f410a400640 1 --2- 172.21.15.72:0/197059648 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f410410a600 0x7f41041a2120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:54:33.501 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:33.498+0000 7f4109a00640 1 --2- 172.21.15.72:0/197059648 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4104075330 0x7f410419ab70 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f40f800ba50 tx=0x7f40f800bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:54:33.501 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:33.498+0000 7f4102000640 1 -- 172.21.15.72:0/197059648 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40f8019070 con 0x7f4104075330 2024-04-26T21:54:33.502 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:33.498+0000 7f4102000640 1 -- 172.21.15.72:0/197059648 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f40f800cbe0 con 0x7f4104075330 2024-04-26T21:54:33.502 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:33.498+0000 7f410ae00640 1 -- 172.21.15.72:0/197059648 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41041a2a80 con 0x7f4104075330 2024-04-26T21:54:33.502 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:33.498+0000 7f4102000640 1 -- 172.21.15.72:0/197059648 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40f80075d0 con 0x7f4104075330 2024-04-26T21:54:33.502 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:33.498+0000 7f410ae00640 1 -- 172.21.15.72:0/197059648 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41041a3020 con 0x7f4104075330 2024-04-26T21:54:33.502 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:33.500+0000 7f410ae00640 1 -- 172.21.15.72:0/197059648 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41041093c0 con 0x7f4104075330 2024-04-26T21:54:33.502 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:33.501+0000 7f4102000640 1 -- 172.21.15.72:0/197059648 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f40f801c070 con 0x7f4104075330 2024-04-26T21:54:33.505 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:33.503+0000 7f4102000640 1 --2- 172.21.15.72:0/197059648 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f40ec0a3cc0 0x7f40ec0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:33.505 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:33.504+0000 7f4102000640 1 -- 172.21.15.72:0/197059648 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7f40f80cfc90 con 0x7f4104075330 2024-04-26T21:54:33.505 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:33.504+0000 7f4109000640 1 --2- 172.21.15.72:0/197059648 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f40ec0a3cc0 0x7f40ec0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:33.505 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:33.504+0000 7f4109000640 1 --2- 172.21.15.72:0/197059648 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f40ec0a3cc0 0x7f40ec0a6180 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f410419c080 tx=0x7f40f0008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:54:33.509 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:33.508+0000 7f4102000640 1 -- 172.21.15.72:0/197059648 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f40f809c540 con 0x7f4104075330 2024-04-26T21:54:33.725 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:33.724+0000 7f410ae00640 1 -- 172.21.15.72:0/197059648 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f410402dca0 con 0x7f4104075330 2024-04-26T21:54:33.727 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:33.726+0000 7f4102000640 1 -- 172.21.15.72:0/197059648 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+22783 (secure 0 0 0) 0x7f40f809be40 con 0x7f4104075330 2024-04-26T21:54:33.728 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T21:54:33.728 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":91,"fsid":"a1780950-0415-11ef-bc93-c7b262605968","created":"2024-04-26T21:42:01.504173+0000","modified":"2024-04-26T21:54:11.892872+0000","last_up_change":"2024-04-26T21:51:43.482437+0000","last_in_change":"2024-04-26T21:51:18.468587+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:46:29.785427+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:52:48.320391+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T21:52:49.517493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"04e06884-b355-4762-9b29-9034b7201e9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6802","nonce":1416765179},{"type":"v1","addr":"172.21.15.72:6803","nonce":1416765179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6804","nonce":1416765179},{"type":"v1","addr":"172.21.15.72:6805","nonce":1416765179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6808","nonce":1416765179},{"type":"v1","addr":"172.21.15.72:6809","nonce":1416765179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6806","nonce":1416765179},{"type":"v1","addr":"172.21.15.72:6807","nonce":1416765179}]},"public_addr":"172.21.15.72:6803/1416765179","cluster_addr":"172.21.15.72:6805/1416765179","heartbeat_back_addr":"172.21.15.72:6809/1416765179","heartbeat_front_addr":"172.21.15.72:6807/1416765179","state":["exists","up"]},{"osd":1,"uuid":"d75a479a-4a2e-48e9-a01e-2b1c9617416a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":755426949},{"type":"v1","addr":"172.21.15.76:6801","nonce":755426949}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":755426949},{"type":"v1","addr":"172.21.15.76:6803","nonce":755426949}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":755426949},{"type":"v1","addr":"172.21.15.76:6807","nonce":755426949}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":755426949},{"type":"v1","addr":"172.21.15.76:6805","nonce":755426949}]},"public_addr":"172.21.15.76:6801/755426949","cluster_addr":"172.21.15.76:6803/755426949","heartbeat_back_addr":"172.21.15.76:6807/755426949","heartbeat_front_addr":"172.21.15.76:6805/755426949","state":["exists","up"]},{"osd":2,"uuid":"b6f8da58-4428-4e8c-a944-7ea5645d6d92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":852265973},{"type":"v1","addr":"172.21.15.164:6801","nonce":852265973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":852265973},{"type":"v1","addr":"172.21.15.164:6803","nonce":852265973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":852265973},{"type":"v1","addr":"172.21.15.164:6807","nonce":852265973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":852265973},{"type":"v1","addr":"172.21.15.164:6805","nonce":852265973}]},"public_addr":"172.21.15.164:6801/852265973","cluster_addr":"172.21.15.164:6803/852265973","heartbeat_back_addr":"172.21.15.164:6807/852265973","heartbeat_front_addr":"172.21.15.164:6805/852265973","state":["exists","up"]},{"osd":3,"uuid":"76d83409-649e-41e5-b4f4-43bbd5b00ea5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6810","nonce":4027763789},{"type":"v1","addr":"172.21.15.72:6811","nonce":4027763789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6812","nonce":4027763789},{"type":"v1","addr":"172.21.15.72:6813","nonce":4027763789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6816","nonce":4027763789},{"type":"v1","addr":"172.21.15.72:6817","nonce":4027763789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6814","nonce":4027763789},{"type":"v1","addr":"172.21.15.72:6815","nonce":4027763789}]},"public_addr":"172.21.15.72:6811/4027763789","cluster_addr":"172.21.15.72:6813/4027763789","heartbeat_back_addr":"172.21.15.72:6817/4027763789","heartbeat_front_addr":"172.21.15.72:6815/4027763789","state":["exists","up"]},{"osd":4,"uuid":"d2cfeea7-1661-4156-bdcd-f770a8d1a878","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":3718867827},{"type":"v1","addr":"172.21.15.76:6809","nonce":3718867827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":3718867827},{"type":"v1","addr":"172.21.15.76:6811","nonce":3718867827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":3718867827},{"type":"v1","addr":"172.21.15.76:6815","nonce":3718867827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":3718867827},{"type":"v1","addr":"172.21.15.76:6813","nonce":3718867827}]},"public_addr":"172.21.15.76:6809/3718867827","cluster_addr":"172.21.15.76:6811/3718867827","heartbeat_back_addr":"172.21.15.76:6815/3718867827","heartbeat_front_addr":"172.21.15.76:6813/3718867827","state":["exists","up"]},{"osd":5,"uuid":"f3896ad6-a040-4ea3-a431-ff9b7fc05c36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":2543954772},{"type":"v1","addr":"172.21.15.164:6809","nonce":2543954772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":2543954772},{"type":"v1","addr":"172.21.15.164:6811","nonce":2543954772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":2543954772},{"type":"v1","addr":"172.21.15.164:6815","nonce":2543954772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":2543954772},{"type":"v1","addr":"172.21.15.164:6813","nonce":2543954772}]},"public_addr":"172.21.15.164:6809/2543954772","cluster_addr":"172.21.15.164:6811/2543954772","heartbeat_back_addr":"172.21.15.164:6815/2543954772","heartbeat_front_addr":"172.21.15.164:6813/2543954772","state":["exists","up"]},{"osd":6,"uuid":"d5d56650-be7e-4ba5-ba3a-d7d473294dab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6818","nonce":2080659027},{"type":"v1","addr":"172.21.15.72:6819","nonce":2080659027}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6820","nonce":2080659027},{"type":"v1","addr":"172.21.15.72:6821","nonce":2080659027}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6824","nonce":2080659027},{"type":"v1","addr":"172.21.15.72:6825","nonce":2080659027}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6822","nonce":2080659027},{"type":"v1","addr":"172.21.15.72:6823","nonce":2080659027}]},"public_addr":"172.21.15.72:6819/2080659027","cluster_addr":"172.21.15.72:6821/2080659027","heartbeat_back_addr":"172.21.15.72:6825/2080659027","heartbeat_front_addr":"172.21.15.72:6823/2080659027","state":["exists","up"]},{"osd":7,"uuid":"18975b5a-9309-4a0f-9f37-3ce09c2e745d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":3144631323},{"type":"v1","addr":"172.21.15.76:6817","nonce":3144631323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":3144631323},{"type":"v1","addr":"172.21.15.76:6819","nonce":3144631323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":3144631323},{"type":"v1","addr":"172.21.15.76:6823","nonce":3144631323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":3144631323},{"type":"v1","addr":"172.21.15.76:6821","nonce":3144631323}]},"public_addr":"172.21.15.76:6817/3144631323","cluster_addr":"172.21.15.76:6819/3144631323","heartbeat_back_addr":"172.21.15.76:6823/3144631323","heartbeat_front_addr":"172.21.15.76:6821/3144631323","state":["exists","up"]},{"osd":8,"uuid":"779e9c1e-8e97-48d4-a854-68ca13f38e0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":627888883},{"type":"v1","addr":"172.21.15.164:6817","nonce":627888883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":627888883},{"type":"v1","addr":"172.21.15.164:6819","nonce":627888883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":627888883},{"type":"v1","addr":"172.21.15.164:6823","nonce":627888883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":627888883},{"type":"v1","addr":"172.21.15.164:6821","nonce":627888883}]},"public_addr":"172.21.15.164:6817/627888883","cluster_addr":"172.21.15.164:6819/627888883","heartbeat_back_addr":"172.21.15.164:6823/627888883","heartbeat_front_addr":"172.21.15.164:6821/627888883","state":["exists","up"]},{"osd":9,"uuid":"d0317850-7b2d-4850-b2cf-0b919505c1fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6826","nonce":1273118485},{"type":"v1","addr":"172.21.15.72:6827","nonce":1273118485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6828","nonce":1273118485},{"type":"v1","addr":"172.21.15.72:6829","nonce":1273118485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6832","nonce":1273118485},{"type":"v1","addr":"172.21.15.72:6833","nonce":1273118485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6830","nonce":1273118485},{"type":"v1","addr":"172.21.15.72:6831","nonce":1273118485}]},"public_addr":"172.21.15.72:6827/1273118485","cluster_addr":"172.21.15.72:6829/1273118485","heartbeat_back_addr":"172.21.15.72:6833/1273118485","heartbeat_front_addr":"172.21.15.72:6831/1273118485","state":["exists","up"]},{"osd":10,"uuid":"e0aa38d8-6011-4d2a-b67b-5bff6d999e47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":2808954459},{"type":"v1","addr":"172.21.15.76:6825","nonce":2808954459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":2808954459},{"type":"v1","addr":"172.21.15.76:6827","nonce":2808954459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":2808954459},{"type":"v1","addr":"172.21.15.76:6831","nonce":2808954459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":2808954459},{"type":"v1","addr":"172.21.15.76:6829","nonce":2808954459}]},"public_addr":"172.21.15.76:6825/2808954459","cluster_addr":"172.21.15.76:6827/2808954459","heartbeat_back_addr":"172.21.15.76:6831/2808954459","heartbeat_front_addr":"172.21.15.76:6829/2808954459","state":["exists","up"]},{"osd":11,"uuid":"981b3d4d-916e-4937-9916-7cc21a891e43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":2354951784},{"type":"v1","addr":"172.21.15.164:6825","nonce":2354951784}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":2354951784},{"type":"v1","addr":"172.21.15.164:6827","nonce":2354951784}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":2354951784},{"type":"v1","addr":"172.21.15.164:6831","nonce":2354951784}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":2354951784},{"type":"v1","addr":"172.21.15.164:6829","nonce":2354951784}]},"public_addr":"172.21.15.164:6825/2354951784","cluster_addr":"172.21.15.164:6827/2354951784","heartbeat_back_addr":"172.21.15.164:6831/2354951784","heartbeat_front_addr":"172.21.15.164:6829/2354951784","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:18.997924+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:52.439081+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:46:25.893288+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:47:00.112807+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:47:35.116874+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:48:09.216344+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:48:45.028853+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:49:19.905397+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:49:55.030536+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:50:30.767662+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:51:06.423267+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:51:41.631211+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.1a","mappings":[{"from":5,"to":7}]},{"pgid":"3.1b","mappings":[{"from":4,"to":6}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.28","mappings":[{"from":5,"to":6}]},{"pgid":"3.2a","mappings":[{"from":3,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.72:0/416277784":"2024-04-27T21:43:09.602791+0000","172.21.15.72:6800/4010358672":"2024-04-27T21:43:09.602791+0000","172.21.15.72:6801/1012617371":"2024-04-27T21:42:34.684837+0000","172.21.15.72:6801/4010358672":"2024-04-27T21:43:09.602791+0000","172.21.15.72:0/3555448179":"2024-04-27T21:43:09.602791+0000","172.21.15.72:6800/1012617371":"2024-04-27T21:42:34.684837+0000","172.21.15.72:0/2920378761":"2024-04-27T21:42:34.684837+0000","172.21.15.72:0/1933740041":"2024-04-27T21:43:09.602791+0000","172.21.15.72:0/3225295775":"2024-04-27T21:42:34.684837+0000","172.21.15.72:0/2326016180":"2024-04-27T21:42:34.684837+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:54:33.731 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:33.729+0000 7f410ae00640 1 -- 172.21.15.72:0/197059648 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f40ec0a3cc0 msgr2=0x7f40ec0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:33.731 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:33.729+0000 7f410ae00640 1 --2- 172.21.15.72:0/197059648 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f40ec0a3cc0 0x7f40ec0a6180 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f410419c080 tx=0x7f40f0008040 comp rx=0 tx=0).stop 2024-04-26T21:54:33.731 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:33.729+0000 7f410ae00640 1 -- 172.21.15.72:0/197059648 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4104075330 msgr2=0x7f410419ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:33.731 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:33.729+0000 7f410ae00640 1 --2- 172.21.15.72:0/197059648 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4104075330 0x7f410419ab70 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f40f800ba50 tx=0x7f40f800bf10 comp rx=0 tx=0).stop 2024-04-26T21:54:33.731 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:33.729+0000 7f410ae00640 1 -- 172.21.15.72:0/197059648 shutdown_connections 2024-04-26T21:54:33.731 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:33.729+0000 7f410ae00640 1 --2- 172.21.15.72:0/197059648 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f410410a600 0x7f41041a2120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:33.731 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:33.729+0000 7f410ae00640 1 --2- 172.21.15.72:0/197059648 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f40ec0a3cc0 0x7f40ec0a6180 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:33.731 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:33.729+0000 7f410ae00640 1 --2- 172.21.15.72:0/197059648 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4104076910 0x7f410419b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:33.731 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:33.729+0000 7f410ae00640 1 --2- 172.21.15.72:0/197059648 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4104075330 0x7f410419ab70 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:33.731 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:33.729+0000 7f410ae00640 1 -- 172.21.15.72:0/197059648 >> 172.21.15.72:0/197059648 conn(0x7f41040fdcc0 msgr2=0x7f4104106590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:54:33.731 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:33.729+0000 7f410ae00640 1 -- 172.21.15.72:0/197059648 shutdown_connections 2024-04-26T21:54:33.731 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:33.730+0000 7f410ae00640 1 -- 172.21.15.72:0/197059648 wait complete. 2024-04-26T21:54:33.748 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:46:29.785427+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T21:52:48.320391+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T21:52:49.517493+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:54:33.749 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:54:33.939 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:33.938+0000 7f4fb7e00640 1 -- 172.21.15.72:0/3080266191 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4fb8104470 msgr2=0x7f4fb810ad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:33.939 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:33.938+0000 7f4fb7e00640 1 --2- 172.21.15.72:0/3080266191 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4fb8104470 0x7f4fb810ad00 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f4fb0009ca0 tx=0x7f4fb002a8c0 comp rx=0 tx=0).stop 2024-04-26T21:54:33.939 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:33.938+0000 7f4fb7e00640 1 -- 172.21.15.72:0/3080266191 shutdown_connections 2024-04-26T21:54:33.939 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:33.938+0000 7f4fb7e00640 1 --2- 172.21.15.72:0/3080266191 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4fb8104470 0x7f4fb810ad00 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:33.940 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:33.938+0000 7f4fb7e00640 1 --2- 172.21.15.72:0/3080266191 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4fb8103ab0 0x7f4fb8103f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:33.940 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:33.938+0000 7f4fb7e00640 1 --2- 172.21.15.72:0/3080266191 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4fb81028c0 0x7f4fb8102ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:33.940 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:33.938+0000 7f4fb7e00640 1 -- 172.21.15.72:0/3080266191 >> 172.21.15.72:0/3080266191 conn(0x7f4fb80fde30 msgr2=0x7f4fb8100270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:54:33.940 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:33.938+0000 7f4fb7e00640 1 -- 172.21.15.72:0/3080266191 shutdown_connections 2024-04-26T21:54:33.940 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:33.938+0000 7f4fb7e00640 1 -- 172.21.15.72:0/3080266191 wait complete. 2024-04-26T21:54:33.942 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:33.939+0000 7f4fb7e00640 1 Processor -- start 2024-04-26T21:54:33.943 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:33.939+0000 7f4fb7e00640 1 -- start start 2024-04-26T21:54:33.943 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:33.939+0000 7f4fb7e00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4fb81028c0 0x7f4fb819ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:33.943 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:33.939+0000 7f4fb7e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4fb8103ab0 0x7f4fb819b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:33.943 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:33.939+0000 7f4fb7e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4fb8104470 0x7f4fb81a20f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:33.943 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:33.939+0000 7f4fb7e00640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f4fb810e280 con 0x7f4fb8104470 2024-04-26T21:54:33.943 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:33.939+0000 7f4fb7e00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f4fb810e100 con 0x7f4fb81028c0 2024-04-26T21:54:33.943 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:33.939+0000 7f4fb7e00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f4fb810e400 con 0x7f4fb8103ab0 2024-04-26T21:54:33.943 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:33.939+0000 7f4fb7400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4fb8104470 0x7f4fb81a20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:33.943 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:33.939+0000 7f4fb7400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4fb8104470 0x7f4fb81a20f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:38942/0 (socket says 172.21.15.72:38942) 2024-04-26T21:54:33.943 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:33.940+0000 7f4fb7400640 1 -- 172.21.15.72:0/779413999 learned_addr learned my addr 172.21.15.72:0/779413999 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:54:33.943 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:33.940+0000 7f4fb6a00640 1 --2- 172.21.15.72:0/779413999 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4fb81028c0 0x7f4fb819ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:33.943 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:33.940+0000 7f4fb6000640 1 --2- 172.21.15.72:0/779413999 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4fb8103ab0 0x7f4fb819b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:33.943 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:33.940+0000 7f4fb7400640 1 -- 172.21.15.72:0/779413999 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4fb8103ab0 msgr2=0x7f4fb819b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:33.943 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:33.940+0000 7f4fb7400640 1 --2- 172.21.15.72:0/779413999 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4fb8103ab0 0x7f4fb819b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:33.944 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:33.940+0000 7f4fb7400640 1 -- 172.21.15.72:0/779413999 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4fb81028c0 msgr2=0x7f4fb819ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:33.944 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:33.940+0000 7f4fb7400640 1 --2- 172.21.15.72:0/779413999 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4fb81028c0 0x7f4fb819ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:33.944 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:33.940+0000 7f4fb7400640 1 -- 172.21.15.72:0/779413999 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fb81a2760 con 0x7f4fb8104470 2024-04-26T21:54:33.944 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:33.940+0000 7f4fb6a00640 1 --2- 172.21.15.72:0/779413999 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4fb81028c0 0x7f4fb819ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:54:33.944 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:33.940+0000 7f4fb7400640 1 --2- 172.21.15.72:0/779413999 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4fb8104470 0x7f4fb81a20f0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f4fb002aeb0 tx=0x7f4fb0003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:54:33.944 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:33.940+0000 7f4faf400640 1 -- 172.21.15.72:0/779413999 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fb0032ed0 con 0x7f4fb8104470 2024-04-26T21:54:33.944 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:33.941+0000 7f4faf400640 1 -- 172.21.15.72:0/779413999 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4fb0003e10 con 0x7f4fb8104470 2024-04-26T21:54:33.944 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:33.941+0000 7f4fb7e00640 1 -- 172.21.15.72:0/779413999 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fb81a29f0 con 0x7f4fb8104470 2024-04-26T21:54:33.944 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:33.941+0000 7f4faf400640 1 -- 172.21.15.72:0/779413999 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fb0030cd0 con 0x7f4fb8104470 2024-04-26T21:54:33.944 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:33.941+0000 7f4fb7e00640 1 -- 172.21.15.72:0/779413999 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fb81a2f90 con 0x7f4fb8104470 2024-04-26T21:54:33.945 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:33.942+0000 7f4fb7e00640 1 -- 172.21.15.72:0/779413999 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4fb81a31c0 con 0x7f4fb8104470 2024-04-26T21:54:33.945 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:33.944+0000 7f4faf400640 1 -- 172.21.15.72:0/779413999 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f4fb0004240 con 0x7f4fb8104470 2024-04-26T21:54:33.946 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:33.945+0000 7f4faf400640 1 --2- 172.21.15.72:0/779413999 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f4f9c0a3cc0 0x7f4f9c0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:33.946 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:33.945+0000 7f4faf400640 1 -- 172.21.15.72:0/779413999 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7f4fb00f1c10 con 0x7f4fb8104470 2024-04-26T21:54:33.946 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:33.945+0000 7f4fb6a00640 1 --2- 172.21.15.72:0/779413999 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f4f9c0a3cc0 0x7f4f9c0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:33.947 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:33.946+0000 7f4fb6a00640 1 --2- 172.21.15.72:0/779413999 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f4f9c0a3cc0 0x7f4f9c0a6180 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f4fa80058c0 tx=0x7f4fa8005d50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:54:33.954 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:33.954+0000 7f4faf400640 1 -- 172.21.15.72:0/779413999 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f4fb00be500 con 0x7f4fb8104470 2024-04-26T21:54:34.071 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:54:33 smithi072 ceph-mon[28138]: from='client.? 172.21.15.72:0/197059648' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:54:34.162 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:34.160+0000 7f4fb7e00640 1 -- 172.21.15.72:0/779413999 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f4fb81a3550 con 0x7f4fb8104470 2024-04-26T21:54:34.163 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:34.162+0000 7f4faf400640 1 -- 172.21.15.72:0/779413999 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) ==== 93+0+10 (secure 0 0 0) 0x7f4fb00bde00 con 0x7f4fb8104470 2024-04-26T21:54:34.163 INFO:teuthology.orchestra.run.smithi072.stdout:pg_num: 1 2024-04-26T21:54:34.165 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:34.164+0000 7f4fb7e00640 1 -- 172.21.15.72:0/779413999 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f4f9c0a3cc0 msgr2=0x7f4f9c0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:34.165 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:34.164+0000 7f4fb7e00640 1 --2- 172.21.15.72:0/779413999 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f4f9c0a3cc0 0x7f4f9c0a6180 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f4fa80058c0 tx=0x7f4fa8005d50 comp rx=0 tx=0).stop 2024-04-26T21:54:34.165 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:34.164+0000 7f4fb7e00640 1 -- 172.21.15.72:0/779413999 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4fb8104470 msgr2=0x7f4fb81a20f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:34.165 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:34.164+0000 7f4fb7e00640 1 --2- 172.21.15.72:0/779413999 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4fb8104470 0x7f4fb81a20f0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f4fb002aeb0 tx=0x7f4fb0003c20 comp rx=0 tx=0).stop 2024-04-26T21:54:34.165 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:34.164+0000 7f4fb7e00640 1 -- 172.21.15.72:0/779413999 shutdown_connections 2024-04-26T21:54:34.165 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:34.164+0000 7f4fb7e00640 1 --2- 172.21.15.72:0/779413999 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4fb8104470 0x7f4fb81a20f0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:34.166 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:34.164+0000 7f4fb7e00640 1 --2- 172.21.15.72:0/779413999 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f4f9c0a3cc0 0x7f4f9c0a6180 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:34.166 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:34.164+0000 7f4fb7e00640 1 --2- 172.21.15.72:0/779413999 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4fb8103ab0 0x7f4fb819b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:34.166 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:34.164+0000 7f4fb7e00640 1 --2- 172.21.15.72:0/779413999 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4fb81028c0 0x7f4fb819ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:34.166 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:34.164+0000 7f4fb7e00640 1 -- 172.21.15.72:0/779413999 >> 172.21.15.72:0/779413999 conn(0x7f4fb80fde30 msgr2=0x7f4fb8074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:54:34.166 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:34.164+0000 7f4fb7e00640 1 -- 172.21.15.72:0/779413999 shutdown_connections 2024-04-26T21:54:34.166 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:34.164+0000 7f4fb7e00640 1 -- 172.21.15.72:0/779413999 wait complete. 2024-04-26T21:54:34.182 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T21:54:34.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:54:33 smithi076 ceph-mon[31823]: from='client.? 172.21.15.72:0/197059648' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:54:34.367 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:34.365+0000 7f7b29000640 1 -- 172.21.15.72:0/17116571 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7b24103a30 msgr2=0x7f7b24103eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:34.367 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:34.365+0000 7f7b29000640 1 --2- 172.21.15.72:0/17116571 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7b24103a30 0x7f7b24103eb0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f7b14009900 tx=0x7f7b1402a3f0 comp rx=0 tx=0).stop 2024-04-26T21:54:34.367 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:34.365+0000 7f7b29000640 1 -- 172.21.15.72:0/17116571 shutdown_connections 2024-04-26T21:54:34.367 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:34.365+0000 7f7b29000640 1 --2- 172.21.15.72:0/17116571 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7b241043f0 0x7f7b2410ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:34.367 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:34.365+0000 7f7b29000640 1 --2- 172.21.15.72:0/17116571 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7b24103a30 0x7f7b24103eb0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:34.367 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:34.365+0000 7f7b29000640 1 --2- 172.21.15.72:0/17116571 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7b24102840 0x7f7b24102c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:34.367 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:34.365+0000 7f7b29000640 1 -- 172.21.15.72:0/17116571 >> 172.21.15.72:0/17116571 conn(0x7f7b240fddd0 msgr2=0x7f7b241001f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:54:34.367 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:34.365+0000 7f7b29000640 1 -- 172.21.15.72:0/17116571 shutdown_connections 2024-04-26T21:54:34.367 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:34.365+0000 7f7b29000640 1 -- 172.21.15.72:0/17116571 wait complete. 2024-04-26T21:54:34.370 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:34.366+0000 7f7b29000640 1 Processor -- start 2024-04-26T21:54:34.370 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:34.367+0000 7f7b29000640 1 -- start start 2024-04-26T21:54:34.370 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:34.367+0000 7f7b29000640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7b24102840 0x7f7b241989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:34.370 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:34.367+0000 7f7b29000640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7b24103a30 0x7f7b24198ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:34.370 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:34.367+0000 7f7b29000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7b241043f0 0x7f7b2419ff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:34.370 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:34.367+0000 7f7b29000640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f7b2410e280 con 0x7f7b24103a30 2024-04-26T21:54:34.370 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:34.367+0000 7f7b29000640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f7b2410e100 con 0x7f7b24102840 2024-04-26T21:54:34.370 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:34.367+0000 7f7b29000640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f7b2410e400 con 0x7f7b241043f0 2024-04-26T21:54:34.370 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:34.367+0000 7f7b1be00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7b24103a30 0x7f7b24198ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:34.370 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:34.367+0000 7f7b1be00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7b24103a30 0x7f7b24198ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:38950/0 (socket says 172.21.15.72:38950) 2024-04-26T21:54:34.371 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:34.367+0000 7f7b1be00640 1 -- 172.21.15.72:0/2252787296 learned_addr learned my addr 172.21.15.72:0/2252787296 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:54:34.371 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:34.367+0000 7f7b23400640 1 --2- 172.21.15.72:0/2252787296 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7b24102840 0x7f7b241989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:34.371 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:34.367+0000 7f7b23e00640 1 --2- 172.21.15.72:0/2252787296 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7b241043f0 0x7f7b2419ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:34.371 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:34.368+0000 7f7b1be00640 1 -- 172.21.15.72:0/2252787296 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7b241043f0 msgr2=0x7f7b2419ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:34.371 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:34.368+0000 7f7b1be00640 1 --2- 172.21.15.72:0/2252787296 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7b241043f0 0x7f7b2419ff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:34.371 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:34.368+0000 7f7b1be00640 1 -- 172.21.15.72:0/2252787296 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7b24102840 msgr2=0x7f7b241989a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:34.371 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:34.368+0000 7f7b1be00640 1 --2- 172.21.15.72:0/2252787296 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7b24102840 0x7f7b241989a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:34.371 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:34.368+0000 7f7b1be00640 1 -- 172.21.15.72:0/2252787296 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b241a05c0 con 0x7f7b24103a30 2024-04-26T21:54:34.371 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:34.368+0000 7f7b23e00640 1 --2- 172.21.15.72:0/2252787296 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7b241043f0 0x7f7b2419ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:54:34.371 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:34.368+0000 7f7b1be00640 1 --2- 172.21.15.72:0/2252787296 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7b24103a30 0x7f7b24198ee0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f7b14009230 tx=0x7f7b140042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:54:34.371 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:34.368+0000 7f7b20c00640 1 -- 172.21.15.72:0/2252787296 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b1403f030 con 0x7f7b24103a30 2024-04-26T21:54:34.371 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:34.368+0000 7f7b20c00640 1 -- 172.21.15.72:0/2252787296 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7b14003c90 con 0x7f7b24103a30 2024-04-26T21:54:34.371 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:34.368+0000 7f7b29000640 1 -- 172.21.15.72:0/2252787296 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b241a0850 con 0x7f7b24103a30 2024-04-26T21:54:34.371 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:34.368+0000 7f7b20c00640 1 -- 172.21.15.72:0/2252787296 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b14035070 con 0x7f7b24103a30 2024-04-26T21:54:34.372 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:34.368+0000 7f7b29000640 1 -- 172.21.15.72:0/2252787296 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b241a0dc0 con 0x7f7b24103a30 2024-04-26T21:54:34.372 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:34.369+0000 7f7b29000640 1 -- 172.21.15.72:0/2252787296 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b24109c80 con 0x7f7b24103a30 2024-04-26T21:54:34.372 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:34.371+0000 7f7b20c00640 1 -- 172.21.15.72:0/2252787296 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f7b14042030 con 0x7f7b24103a30 2024-04-26T21:54:34.373 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:34.372+0000 7f7b20c00640 1 --2- 172.21.15.72:0/2252787296 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f7b080a3c40 0x7f7b080a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:34.374 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:34.373+0000 7f7b20c00640 1 -- 172.21.15.72:0/2252787296 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7f7b140f0ea0 con 0x7f7b24103a30 2024-04-26T21:54:34.374 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:34.373+0000 7f7b23400640 1 --2- 172.21.15.72:0/2252787296 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f7b080a3c40 0x7f7b080a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:34.374 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:34.373+0000 7f7b23400640 1 --2- 172.21.15.72:0/2252787296 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f7b080a3c40 0x7f7b080a6100 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f7b0c0058c0 tx=0x7f7b0c005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:54:34.377 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:34.376+0000 7f7b20c00640 1 -- 172.21.15.72:0/2252787296 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f7b140bd640 con 0x7f7b24103a30 2024-04-26T21:54:34.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:54:33 smithi164 ceph-mon[31916]: from='client.? 172.21.15.72:0/197059648' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:54:34.564 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:34.563+0000 7f7b29000640 1 -- 172.21.15.72:0/2252787296 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f7b2402dcf0 con 0x7f7b24103a30 2024-04-26T21:54:34.564 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:34.563+0000 7f7b20c00640 1 -- 172.21.15.72:0/2252787296 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v91) ==== 104+0+11 (secure 0 0 0) 0x7f7b140bcf40 con 0x7f7b24103a30 2024-04-26T21:54:34.565 INFO:teuthology.orchestra.run.smithi072.stdout:pg_num: 32 2024-04-26T21:54:34.567 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:34.566+0000 7f7b29000640 1 -- 172.21.15.72:0/2252787296 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f7b080a3c40 msgr2=0x7f7b080a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:34.567 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:34.566+0000 7f7b29000640 1 --2- 172.21.15.72:0/2252787296 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f7b080a3c40 0x7f7b080a6100 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f7b0c0058c0 tx=0x7f7b0c005850 comp rx=0 tx=0).stop 2024-04-26T21:54:34.567 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:34.566+0000 7f7b29000640 1 -- 172.21.15.72:0/2252787296 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7b24103a30 msgr2=0x7f7b24198ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:34.567 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:34.566+0000 7f7b29000640 1 --2- 172.21.15.72:0/2252787296 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7b24103a30 0x7f7b24198ee0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f7b14009230 tx=0x7f7b140042f0 comp rx=0 tx=0).stop 2024-04-26T21:54:34.568 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:34.566+0000 7f7b29000640 1 -- 172.21.15.72:0/2252787296 shutdown_connections 2024-04-26T21:54:34.568 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:34.566+0000 7f7b29000640 1 --2- 172.21.15.72:0/2252787296 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f7b080a3c40 0x7f7b080a6100 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:34.568 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:34.566+0000 7f7b29000640 1 --2- 172.21.15.72:0/2252787296 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7b241043f0 0x7f7b2419ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:34.568 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:34.566+0000 7f7b29000640 1 --2- 172.21.15.72:0/2252787296 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7b24103a30 0x7f7b24198ee0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:34.568 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:34.566+0000 7f7b29000640 1 --2- 172.21.15.72:0/2252787296 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7b24102840 0x7f7b241989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:34.568 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:34.566+0000 7f7b29000640 1 -- 172.21.15.72:0/2252787296 >> 172.21.15.72:0/2252787296 conn(0x7f7b240fddd0 msgr2=0x7f7b24106e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:54:34.568 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:34.566+0000 7f7b29000640 1 -- 172.21.15.72:0/2252787296 shutdown_connections 2024-04-26T21:54:34.568 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:34.566+0000 7f7b29000640 1 -- 172.21.15.72:0/2252787296 wait complete. 2024-04-26T21:54:34.585 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T21:54:34.785 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:34.783+0000 7f46d7e00640 1 -- 172.21.15.72:0/3465091848 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f46d8102840 msgr2=0x7f46d8102cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:34.785 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:34.783+0000 7f46d7e00640 1 --2- 172.21.15.72:0/3465091848 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f46d8102840 0x7f46d8102cc0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f46c4009900 tx=0x7f46c402a410 comp rx=0 tx=0).stop 2024-04-26T21:54:34.785 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:34.783+0000 7f46d7e00640 1 -- 172.21.15.72:0/3465091848 shutdown_connections 2024-04-26T21:54:34.785 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:34.783+0000 7f46d7e00640 1 --2- 172.21.15.72:0/3465091848 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f46d8103200 0x7f46d8109a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:34.785 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:34.783+0000 7f46d7e00640 1 --2- 172.21.15.72:0/3465091848 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f46d8102840 0x7f46d8102cc0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:34.785 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:34.783+0000 7f46d7e00640 1 --2- 172.21.15.72:0/3465091848 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f46d8101650 0x7f46d8101a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:34.785 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:34.783+0000 7f46d7e00640 1 -- 172.21.15.72:0/3465091848 >> 172.21.15.72:0/3465091848 conn(0x7f46d80fcc00 msgr2=0x7f46d80ff020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:54:34.785 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:34.783+0000 7f46d7e00640 1 -- 172.21.15.72:0/3465091848 shutdown_connections 2024-04-26T21:54:34.786 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:34.783+0000 7f46d7e00640 1 -- 172.21.15.72:0/3465091848 wait complete. 2024-04-26T21:54:34.788 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:34.786+0000 7f46d7e00640 1 Processor -- start 2024-04-26T21:54:34.788 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:34.786+0000 7f46d7e00640 1 -- start start 2024-04-26T21:54:34.788 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:34.786+0000 7f46d7e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f46d8101650 0x7f46d81988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:34.788 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:34.786+0000 7f46d7e00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f46d8102840 0x7f46d8198df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:34.788 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:34.786+0000 7f46d7e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f46d8103200 0x7f46d819fe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:34.788 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:34.786+0000 7f46d7e00640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f46d810d060 con 0x7f46d8103200 2024-04-26T21:54:34.788 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:34.786+0000 7f46d7e00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f46d810cee0 con 0x7f46d8102840 2024-04-26T21:54:34.788 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:34.786+0000 7f46d7e00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f46d810d1e0 con 0x7f46d8101650 2024-04-26T21:54:34.788 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:34.786+0000 7f46d7400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f46d8103200 0x7f46d819fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:34.788 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:34.786+0000 7f46d7400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f46d8103200 0x7f46d819fe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:38976/0 (socket says 172.21.15.72:38976) 2024-04-26T21:54:34.789 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:34.786+0000 7f46d7400640 1 -- 172.21.15.72:0/4196582163 learned_addr learned my addr 172.21.15.72:0/4196582163 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:54:34.789 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:34.786+0000 7f46d6a00640 1 --2- 172.21.15.72:0/4196582163 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f46d8101650 0x7f46d81988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:34.789 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:34.786+0000 7f46d7400640 1 -- 172.21.15.72:0/4196582163 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f46d8101650 msgr2=0x7f46d81988b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:34.789 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:34.786+0000 7f46d7400640 1 --2- 172.21.15.72:0/4196582163 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f46d8101650 0x7f46d81988b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:34.789 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:34.786+0000 7f46d7400640 1 -- 172.21.15.72:0/4196582163 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f46d8102840 msgr2=0x7f46d8198df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:34.789 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:34.786+0000 7f46d7400640 1 --2- 172.21.15.72:0/4196582163 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f46d8102840 0x7f46d8198df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:34.789 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:34.786+0000 7f46d7400640 1 -- 172.21.15.72:0/4196582163 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46d81a04d0 con 0x7f46d8103200 2024-04-26T21:54:34.789 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:34.786+0000 7f46d7400640 1 --2- 172.21.15.72:0/4196582163 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f46d8103200 0x7f46d819fe60 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f46c800d750 tx=0x7f46c800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:54:34.789 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:34.786+0000 7f46cf400640 1 -- 172.21.15.72:0/4196582163 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46c8017070 con 0x7f46d8103200 2024-04-26T21:54:34.789 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:34.786+0000 7f46cf400640 1 -- 172.21.15.72:0/4196582163 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f46c80054b0 con 0x7f46d8103200 2024-04-26T21:54:34.789 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:34.786+0000 7f46d7e00640 1 -- 172.21.15.72:0/4196582163 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46d81a07c0 con 0x7f46d8103200 2024-04-26T21:54:34.789 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:34.786+0000 7f46cf400640 1 -- 172.21.15.72:0/4196582163 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46c8004b90 con 0x7f46d8103200 2024-04-26T21:54:34.790 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:34.787+0000 7f46d7e00640 1 -- 172.21.15.72:0/4196582163 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46d81a0d10 con 0x7f46d8103200 2024-04-26T21:54:34.790 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:34.788+0000 7f46d7e00640 1 -- 172.21.15.72:0/4196582163 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46d81a0f70 con 0x7f46d8103200 2024-04-26T21:54:34.791 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:34.790+0000 7f46cf400640 1 -- 172.21.15.72:0/4196582163 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f46c8004d30 con 0x7f46d8103200 2024-04-26T21:54:34.793 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:34.792+0000 7f46cf400640 1 --2- 172.21.15.72:0/4196582163 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f46bc0a3cc0 0x7f46bc0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:34.793 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:34.792+0000 7f46cf400640 1 -- 172.21.15.72:0/4196582163 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7f46c80d00c0 con 0x7f46d8103200 2024-04-26T21:54:34.793 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:34.792+0000 7f46d6a00640 1 --2- 172.21.15.72:0/4196582163 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f46bc0a3cc0 0x7f46bc0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:34.793 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:34.793+0000 7f46d6a00640 1 --2- 172.21.15.72:0/4196582163 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f46bc0a3cc0 0x7f46bc0a6180 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f46d81026a0 tx=0x7f46d0009870 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:54:34.797 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:34.796+0000 7f46cf400640 1 -- 172.21.15.72:0/4196582163 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f46c809c8c0 con 0x7f46d8103200 2024-04-26T21:54:34.985 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:34.984+0000 7f46d7e00640 1 -- 172.21.15.72:0/4196582163 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f46d81a1260 con 0x7f46d8103200 2024-04-26T21:54:34.986 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:34.985+0000 7f46cf400640 1 -- 172.21.15.72:0/4196582163 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v91) ==== 100+0+11 (secure 0 0 0) 0x7f46c809c1c0 con 0x7f46d8103200 2024-04-26T21:54:34.986 INFO:teuthology.orchestra.run.smithi072.stdout:pg_num: 64 2024-04-26T21:54:34.989 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:34.988+0000 7f46d7e00640 1 -- 172.21.15.72:0/4196582163 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f46bc0a3cc0 msgr2=0x7f46bc0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:34.989 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:34.988+0000 7f46d7e00640 1 --2- 172.21.15.72:0/4196582163 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f46bc0a3cc0 0x7f46bc0a6180 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f46d81026a0 tx=0x7f46d0009870 comp rx=0 tx=0).stop 2024-04-26T21:54:34.989 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:34.988+0000 7f46d7e00640 1 -- 172.21.15.72:0/4196582163 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f46d8103200 msgr2=0x7f46d819fe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:34.990 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:34.988+0000 7f46d7e00640 1 --2- 172.21.15.72:0/4196582163 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f46d8103200 0x7f46d819fe60 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f46c800d750 tx=0x7f46c800dc10 comp rx=0 tx=0).stop 2024-04-26T21:54:34.990 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:34.988+0000 7f46d7e00640 1 -- 172.21.15.72:0/4196582163 shutdown_connections 2024-04-26T21:54:34.990 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:34.988+0000 7f46d7e00640 1 --2- 172.21.15.72:0/4196582163 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f46d8103200 0x7f46d819fe60 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:34.990 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:34.988+0000 7f46d7e00640 1 --2- 172.21.15.72:0/4196582163 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f46d8102840 0x7f46d8198df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:34.990 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:34.988+0000 7f46d7e00640 1 --2- 172.21.15.72:0/4196582163 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f46bc0a3cc0 0x7f46bc0a6180 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:34.991 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:34.988+0000 7f46d7e00640 1 --2- 172.21.15.72:0/4196582163 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f46d8101650 0x7f46d81988b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:34.991 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:34.988+0000 7f46d7e00640 1 -- 172.21.15.72:0/4196582163 >> 172.21.15.72:0/4196582163 conn(0x7f46d80fcc00 msgr2=0x7f46d80fe710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:54:34.991 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:34.989+0000 7f46d7e00640 1 -- 172.21.15.72:0/4196582163 shutdown_connections 2024-04-26T21:54:34.991 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:34.989+0000 7f46d7e00640 1 -- 172.21.15.72:0/4196582163 wait complete. 2024-04-26T21:54:35.007 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:54:35.206 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:35.203+0000 7fb527200640 1 -- 172.21.15.72:0/3196333389 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb528104300 msgr2=0x7fb52810ab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:35.207 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:35.203+0000 7fb527200640 1 --2- 172.21.15.72:0/3196333389 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb528104300 0x7fb52810ab90 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fb520009ca0 tx=0x7fb52002a8c0 comp rx=0 tx=0).stop 2024-04-26T21:54:35.207 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:35.203+0000 7fb527200640 1 -- 172.21.15.72:0/3196333389 shutdown_connections 2024-04-26T21:54:35.207 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:35.203+0000 7fb527200640 1 --2- 172.21.15.72:0/3196333389 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb528104300 0x7fb52810ab90 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:35.207 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:35.203+0000 7fb527200640 1 --2- 172.21.15.72:0/3196333389 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb528103940 0x7fb528103dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:35.207 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:35.203+0000 7fb527200640 1 --2- 172.21.15.72:0/3196333389 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb528102750 0x7fb528102b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:35.207 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:35.203+0000 7fb527200640 1 -- 172.21.15.72:0/3196333389 >> 172.21.15.72:0/3196333389 conn(0x7fb5280fdcc0 msgr2=0x7fb528100100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:54:35.207 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:35.203+0000 7fb527200640 1 -- 172.21.15.72:0/3196333389 shutdown_connections 2024-04-26T21:54:35.207 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:35.203+0000 7fb527200640 1 -- 172.21.15.72:0/3196333389 wait complete. 2024-04-26T21:54:35.208 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:35.204+0000 7fb527200640 1 Processor -- start 2024-04-26T21:54:35.208 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:35.204+0000 7fb527200640 1 -- start start 2024-04-26T21:54:35.208 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:35.204+0000 7fb527200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb528102750 0x7fb52819aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:35.208 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:35.204+0000 7fb527200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb528103940 0x7fb52819af80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:35.208 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:35.204+0000 7fb527200640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb528104300 0x7fb5281a1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:35.208 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:35.204+0000 7fb527200640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fb52810e180 con 0x7fb528104300 2024-04-26T21:54:35.208 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:35.204+0000 7fb527200640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fb52810e000 con 0x7fb528102750 2024-04-26T21:54:35.208 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:35.204+0000 7fb527200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fb52810e300 con 0x7fb528103940 2024-04-26T21:54:35.209 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:35.204+0000 7fb526800640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb528104300 0x7fb5281a1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:35.209 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:35.205+0000 7fb526800640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb528104300 0x7fb5281a1ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:39006/0 (socket says 172.21.15.72:39006) 2024-04-26T21:54:35.209 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:35.205+0000 7fb526800640 1 -- 172.21.15.72:0/2562473914 learned_addr learned my addr 172.21.15.72:0/2562473914 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:54:35.209 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:35.205+0000 7fb525400640 1 --2- 172.21.15.72:0/2562473914 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb528103940 0x7fb52819af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:35.209 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:35.205+0000 7fb525e00640 1 --2- 172.21.15.72:0/2562473914 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb528102750 0x7fb52819aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:35.209 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:35.205+0000 7fb526800640 1 -- 172.21.15.72:0/2562473914 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb528103940 msgr2=0x7fb52819af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:35.209 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:35.205+0000 7fb526800640 1 --2- 172.21.15.72:0/2562473914 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb528103940 0x7fb52819af80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:35.209 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:35.205+0000 7fb526800640 1 -- 172.21.15.72:0/2562473914 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb528102750 msgr2=0x7fb52819aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:35.209 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:35.205+0000 7fb526800640 1 --2- 172.21.15.72:0/2562473914 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb528102750 0x7fb52819aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:35.209 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:35.205+0000 7fb526800640 1 -- 172.21.15.72:0/2562473914 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5281a2660 con 0x7fb528104300 2024-04-26T21:54:35.209 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:35.205+0000 7fb525400640 1 --2- 172.21.15.72:0/2562473914 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb528103940 0x7fb52819af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:54:35.210 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:35.205+0000 7fb526800640 1 --2- 172.21.15.72:0/2562473914 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb528104300 0x7fb5281a1ff0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fb520031040 tx=0x7fb520003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:54:35.210 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:35.205+0000 7fb51ea00640 1 -- 172.21.15.72:0/2562473914 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb520004040 con 0x7fb528104300 2024-04-26T21:54:35.210 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:35.206+0000 7fb51ea00640 1 -- 172.21.15.72:0/2562473914 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb520004670 con 0x7fb528104300 2024-04-26T21:54:35.210 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:35.206+0000 7fb51ea00640 1 -- 172.21.15.72:0/2562473914 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb520030d60 con 0x7fb528104300 2024-04-26T21:54:35.210 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:35.206+0000 7fb527200640 1 -- 172.21.15.72:0/2562473914 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5281a28f0 con 0x7fb528104300 2024-04-26T21:54:35.210 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:35.206+0000 7fb527200640 1 -- 172.21.15.72:0/2562473914 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5281a2e90 con 0x7fb528104300 2024-04-26T21:54:35.210 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:35.207+0000 7fb527200640 1 -- 172.21.15.72:0/2562473914 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5281a30c0 con 0x7fb528104300 2024-04-26T21:54:35.210 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:35.209+0000 7fb51ea00640 1 -- 172.21.15.72:0/2562473914 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fb5200305c0 con 0x7fb528104300 2024-04-26T21:54:35.211 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:35.211+0000 7fb51ea00640 1 --2- 172.21.15.72:0/2562473914 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fb50c0a3cc0 0x7fb50c0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:35.211 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:35.211+0000 7fb51ea00640 1 -- 172.21.15.72:0/2562473914 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7fb5200f1ac0 con 0x7fb528104300 2024-04-26T21:54:35.211 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:35.211+0000 7fb525e00640 1 --2- 172.21.15.72:0/2562473914 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fb50c0a3cc0 0x7fb50c0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:35.212 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:35.212+0000 7fb525e00640 1 --2- 172.21.15.72:0/2562473914 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fb50c0a3cc0 0x7fb50c0a6180 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fb518005fd0 tx=0x7fb518005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:54:35.215 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:35.215+0000 7fb51ea00640 1 -- 172.21.15.72:0/2562473914 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fb5200be370 con 0x7fb528104300 2024-04-26T21:54:35.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:54:34 smithi076 ceph-mon[31823]: pgmap v423: 97 pgs: 97 active+clean; 633 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-26T21:54:35.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:54:34 smithi076 ceph-mon[31823]: from='client.? 172.21.15.72:0/779413999' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T21:54:35.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:54:34 smithi076 ceph-mon[31823]: from='client.? 172.21.15.72:0/2252787296' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T21:54:35.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:54:34 smithi076 ceph-mon[31823]: from='client.? 172.21.15.72:0/4196582163' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T21:54:35.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:54:34 smithi072 ceph-mon[28138]: pgmap v423: 97 pgs: 97 active+clean; 633 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-26T21:54:35.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:54:34 smithi072 ceph-mon[28138]: from='client.? 172.21.15.72:0/779413999' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T21:54:35.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:54:34 smithi072 ceph-mon[28138]: from='client.? 172.21.15.72:0/2252787296' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T21:54:35.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:54:34 smithi072 ceph-mon[28138]: from='client.? 172.21.15.72:0/4196582163' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T21:54:35.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:54:34 smithi164 ceph-mon[31916]: pgmap v423: 97 pgs: 97 active+clean; 633 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-26T21:54:35.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:54:34 smithi164 ceph-mon[31916]: from='client.? 172.21.15.72:0/779413999' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T21:54:35.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:54:34 smithi164 ceph-mon[31916]: from='client.? 172.21.15.72:0/2252787296' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T21:54:35.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:54:34 smithi164 ceph-mon[31916]: from='client.? 172.21.15.72:0/4196582163' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T21:54:35.462 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:35.461+0000 7fb527200640 1 -- 172.21.15.72:0/2562473914 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb5281a3450 con 0x7fb528104300 2024-04-26T21:54:35.463 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:35.463+0000 7fb51ea00640 1 -- 172.21.15.72:0/2562473914 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11931 (secure 0 0 0) 0x7fb5200bdc70 con 0x7fb528104300 2024-04-26T21:54:35.463 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T21:54:35.464 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6835/1320381959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6834","nonce":1320381959},{"type":"v1","addr":"172.21.15.72:6835","nonce":1320381959}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6837/2142200512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6836","nonce":2142200512},{"type":"v1","addr":"172.21.15.72:6837","nonce":2142200512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T21:52:50.567857+0000","modified":"2024-04-26T21:52:58.508126+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34517,"mds_1":34472,"mds_2":34442,"mds_3":34406,"mds_4":24463},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14652":{"gid":14652,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.72:6839/1312898281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6838","nonce":1312898281},{"type":"v1","addr":"172.21.15.72:6839","nonce":1312898281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14664":{"gid":14664,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.72:6841/3257692339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6840","nonce":3257692339},{"type":"v1","addr":"172.21.15.72:6841","nonce":3257692339}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24388":{"gid":24388,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6833/4236668465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":4236668465},{"type":"v1","addr":"172.21.15.164:6833","nonce":4236668465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6835/2391114169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":2391114169},{"type":"v1","addr":"172.21.15.164:6835","nonce":2391114169}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6837/3585829992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":3585829992},{"type":"v1","addr":"172.21.15.164:6837","nonce":3585829992}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.164:6839/1636226820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":1636226820},{"type":"v1","addr":"172.21.15.164:6839","nonce":1636226820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34406":{"gid":34406,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":15,"addr":"172.21.15.76:6833/3869483750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":3869483750},{"type":"v1","addr":"172.21.15.76:6833","nonce":3869483750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34442":{"gid":34442,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.76:6835/3517429435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":3517429435},{"type":"v1","addr":"172.21.15.76:6835","nonce":3517429435}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34472":{"gid":34472,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.76:6837/1150678991","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":1150678991},{"type":"v1","addr":"172.21.15.76:6837","nonce":1150678991}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34517":{"gid":34517,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.76:6839/327145636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6838","nonce":327145636},{"type":"v1","addr":"172.21.15.76:6839","nonce":327145636}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34517,"qdb_cluster":[34406,24463,34442,34517,34472]},"id":1}]} 2024-04-26T21:54:35.464 INFO:teuthology.orchestra.run.smithi072.stderr:dumped fsmap epoch 26 2024-04-26T21:54:35.467 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:35.465+0000 7fb527200640 1 -- 172.21.15.72:0/2562473914 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fb50c0a3cc0 msgr2=0x7fb50c0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:35.467 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:35.465+0000 7fb527200640 1 --2- 172.21.15.72:0/2562473914 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fb50c0a3cc0 0x7fb50c0a6180 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fb518005fd0 tx=0x7fb518005850 comp rx=0 tx=0).stop 2024-04-26T21:54:35.467 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:35.466+0000 7fb527200640 1 -- 172.21.15.72:0/2562473914 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb528104300 msgr2=0x7fb5281a1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:35.467 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:35.466+0000 7fb527200640 1 --2- 172.21.15.72:0/2562473914 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb528104300 0x7fb5281a1ff0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fb520031040 tx=0x7fb520003ba0 comp rx=0 tx=0).stop 2024-04-26T21:54:35.467 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:35.466+0000 7fb527200640 1 -- 172.21.15.72:0/2562473914 shutdown_connections 2024-04-26T21:54:35.467 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:35.466+0000 7fb527200640 1 --2- 172.21.15.72:0/2562473914 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb528104300 0x7fb5281a1ff0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:35.468 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:35.466+0000 7fb527200640 1 --2- 172.21.15.72:0/2562473914 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fb50c0a3cc0 0x7fb50c0a6180 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:35.468 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:35.466+0000 7fb527200640 1 --2- 172.21.15.72:0/2562473914 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb528103940 0x7fb52819af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:35.468 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:35.466+0000 7fb527200640 1 --2- 172.21.15.72:0/2562473914 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb528102750 0x7fb52819aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:35.468 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:35.466+0000 7fb527200640 1 -- 172.21.15.72:0/2562473914 >> 172.21.15.72:0/2562473914 conn(0x7fb5280fdcc0 msgr2=0x7fb528074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:54:35.468 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:35.466+0000 7fb527200640 1 -- 172.21.15.72:0/2562473914 shutdown_connections 2024-04-26T21:54:35.468 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:35.466+0000 7fb527200640 1 -- 172.21.15.72:0/2562473914 wait complete. 2024-04-26T21:54:35.484 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:54:35.691 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:35.687+0000 7f7282000640 1 -- 172.21.15.72:0/616113856 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f727c10ef50 msgr2=0x7f727c111340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:35.691 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:35.687+0000 7f7282000640 1 --2- 172.21.15.72:0/616113856 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f727c10ef50 0x7f727c111340 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f7274009cd0 tx=0x7f727402a8f0 comp rx=0 tx=0).stop 2024-04-26T21:54:35.692 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:35.688+0000 7f7282000640 1 -- 172.21.15.72:0/616113856 shutdown_connections 2024-04-26T21:54:35.692 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:35.688+0000 7f7282000640 1 --2- 172.21.15.72:0/616113856 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f727c10ef50 0x7f727c111340 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:35.692 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:35.688+0000 7f7282000640 1 --2- 172.21.15.72:0/616113856 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f727c075390 0x7f727c075810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:35.692 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:35.688+0000 7f7282000640 1 --2- 172.21.15.72:0/616113856 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f727c076970 0x7f727c074dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:35.692 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:35.688+0000 7f7282000640 1 -- 172.21.15.72:0/616113856 >> 172.21.15.72:0/616113856 conn(0x7f727c0fdf10 msgr2=0x7f727c100330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:54:35.692 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:35.688+0000 7f7282000640 1 -- 172.21.15.72:0/616113856 shutdown_connections 2024-04-26T21:54:35.692 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:35.688+0000 7f7282000640 1 -- 172.21.15.72:0/616113856 wait complete. 2024-04-26T21:54:35.692 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:35.689+0000 7f7282000640 1 Processor -- start 2024-04-26T21:54:35.692 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:35.689+0000 7f7282000640 1 -- start start 2024-04-26T21:54:35.692 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:35.689+0000 7f7282000640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f727c075390 0x7f727c19f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:35.692 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:35.689+0000 7f7282000640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f727c076970 0x7f727c19f570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:35.692 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:35.689+0000 7f7282000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f727c10ef50 0x7f727c1a65e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:35.692 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:35.689+0000 7f7282000640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f727c114980 con 0x7f727c076970 2024-04-26T21:54:35.692 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:35.689+0000 7f7282000640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f727c114800 con 0x7f727c075390 2024-04-26T21:54:35.693 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:35.689+0000 7f7282000640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f727c114b00 con 0x7f727c10ef50 2024-04-26T21:54:35.693 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:35.690+0000 7f727aa00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f727c076970 0x7f727c19f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:35.693 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:35.690+0000 7f727aa00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f727c076970 0x7f727c19f570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:39032/0 (socket says 172.21.15.72:39032) 2024-04-26T21:54:35.693 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:35.690+0000 7f727aa00640 1 -- 172.21.15.72:0/2395842473 learned_addr learned my addr 172.21.15.72:0/2395842473 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:54:35.693 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:35.690+0000 7f727be00640 1 --2- 172.21.15.72:0/2395842473 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f727c10ef50 0x7f727c1a65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:35.693 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:35.690+0000 7f727b400640 1 --2- 172.21.15.72:0/2395842473 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f727c075390 0x7f727c19f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:35.693 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:35.690+0000 7f727aa00640 1 -- 172.21.15.72:0/2395842473 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f727c10ef50 msgr2=0x7f727c1a65e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:35.693 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:35.690+0000 7f727aa00640 1 --2- 172.21.15.72:0/2395842473 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f727c10ef50 0x7f727c1a65e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:35.693 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:35.690+0000 7f727aa00640 1 -- 172.21.15.72:0/2395842473 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f727c075390 msgr2=0x7f727c19f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:35.693 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:35.690+0000 7f727aa00640 1 --2- 172.21.15.72:0/2395842473 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f727c075390 0x7f727c19f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:35.693 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:35.690+0000 7f727aa00640 1 -- 172.21.15.72:0/2395842473 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f727c1a6c50 con 0x7f727c076970 2024-04-26T21:54:35.693 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:35.690+0000 7f727aa00640 1 --2- 172.21.15.72:0/2395842473 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f727c076970 0x7f727c19f570 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f726800c920 tx=0x7f726800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:54:35.693 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:35.691+0000 7f7273e00640 1 -- 172.21.15.72:0/2395842473 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7268007c70 con 0x7f727c076970 2024-04-26T21:54:35.694 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:35.691+0000 7f7273e00640 1 -- 172.21.15.72:0/2395842473 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7268007e10 con 0x7f727c076970 2024-04-26T21:54:35.694 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:35.691+0000 7f7282000640 1 -- 172.21.15.72:0/2395842473 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f727c1a6f40 con 0x7f727c076970 2024-04-26T21:54:35.694 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:35.691+0000 7f7273e00640 1 -- 172.21.15.72:0/2395842473 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7268018710 con 0x7f727c076970 2024-04-26T21:54:35.694 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:35.691+0000 7f7282000640 1 -- 172.21.15.72:0/2395842473 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f727c1a7400 con 0x7f727c076970 2024-04-26T21:54:35.694 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:35.692+0000 7f7282000640 1 -- 172.21.15.72:0/2395842473 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f727c105120 con 0x7f727c076970 2024-04-26T21:54:35.695 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:35.695+0000 7f7273e00640 1 -- 172.21.15.72:0/2395842473 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f726800f450 con 0x7f727c076970 2024-04-26T21:54:35.696 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:35.696+0000 7f7273e00640 1 --2- 172.21.15.72:0/2395842473 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f72600a3c40 0x7f72600a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:35.696 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:35.696+0000 7f7273e00640 1 -- 172.21.15.72:0/2395842473 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7f72680d08a0 con 0x7f727c076970 2024-04-26T21:54:35.696 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:35.696+0000 7f727b400640 1 --2- 172.21.15.72:0/2395842473 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f72600a3c40 0x7f72600a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:35.697 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:35.697+0000 7f727b400640 1 --2- 172.21.15.72:0/2395842473 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f72600a3c40 0x7f72600a6100 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f726c005fd0 tx=0x7f726c005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:54:35.700 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:35.700+0000 7f7273e00640 1 -- 172.21.15.72:0/2395842473 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f7268014030 con 0x7f727c076970 2024-04-26T21:54:35.907 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:35.906+0000 7f7282000640 1 -- 172.21.15.72:0/2395842473 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f727c02dd20 con 0x7f727c076970 2024-04-26T21:54:35.910 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:35.909+0000 7f7273e00640 1 -- 172.21.15.72:0/2395842473 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+22783 (secure 0 0 0) 0x7f726800de20 con 0x7f727c076970 2024-04-26T21:54:35.910 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T21:54:35.910 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":91,"fsid":"a1780950-0415-11ef-bc93-c7b262605968","created":"2024-04-26T21:42:01.504173+0000","modified":"2024-04-26T21:54:11.892872+0000","last_up_change":"2024-04-26T21:51:43.482437+0000","last_in_change":"2024-04-26T21:51:18.468587+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:46:29.785427+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:52:48.320391+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T21:52:49.517493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"04e06884-b355-4762-9b29-9034b7201e9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6802","nonce":1416765179},{"type":"v1","addr":"172.21.15.72:6803","nonce":1416765179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6804","nonce":1416765179},{"type":"v1","addr":"172.21.15.72:6805","nonce":1416765179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6808","nonce":1416765179},{"type":"v1","addr":"172.21.15.72:6809","nonce":1416765179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6806","nonce":1416765179},{"type":"v1","addr":"172.21.15.72:6807","nonce":1416765179}]},"public_addr":"172.21.15.72:6803/1416765179","cluster_addr":"172.21.15.72:6805/1416765179","heartbeat_back_addr":"172.21.15.72:6809/1416765179","heartbeat_front_addr":"172.21.15.72:6807/1416765179","state":["exists","up"]},{"osd":1,"uuid":"d75a479a-4a2e-48e9-a01e-2b1c9617416a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":755426949},{"type":"v1","addr":"172.21.15.76:6801","nonce":755426949}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":755426949},{"type":"v1","addr":"172.21.15.76:6803","nonce":755426949}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":755426949},{"type":"v1","addr":"172.21.15.76:6807","nonce":755426949}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":755426949},{"type":"v1","addr":"172.21.15.76:6805","nonce":755426949}]},"public_addr":"172.21.15.76:6801/755426949","cluster_addr":"172.21.15.76:6803/755426949","heartbeat_back_addr":"172.21.15.76:6807/755426949","heartbeat_front_addr":"172.21.15.76:6805/755426949","state":["exists","up"]},{"osd":2,"uuid":"b6f8da58-4428-4e8c-a944-7ea5645d6d92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":852265973},{"type":"v1","addr":"172.21.15.164:6801","nonce":852265973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":852265973},{"type":"v1","addr":"172.21.15.164:6803","nonce":852265973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":852265973},{"type":"v1","addr":"172.21.15.164:6807","nonce":852265973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":852265973},{"type":"v1","addr":"172.21.15.164:6805","nonce":852265973}]},"public_addr":"172.21.15.164:6801/852265973","cluster_addr":"172.21.15.164:6803/852265973","heartbeat_back_addr":"172.21.15.164:6807/852265973","heartbeat_front_addr":"172.21.15.164:6805/852265973","state":["exists","up"]},{"osd":3,"uuid":"76d83409-649e-41e5-b4f4-43bbd5b00ea5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6810","nonce":4027763789},{"type":"v1","addr":"172.21.15.72:6811","nonce":4027763789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6812","nonce":4027763789},{"type":"v1","addr":"172.21.15.72:6813","nonce":4027763789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6816","nonce":4027763789},{"type":"v1","addr":"172.21.15.72:6817","nonce":4027763789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6814","nonce":4027763789},{"type":"v1","addr":"172.21.15.72:6815","nonce":4027763789}]},"public_addr":"172.21.15.72:6811/4027763789","cluster_addr":"172.21.15.72:6813/4027763789","heartbeat_back_addr":"172.21.15.72:6817/4027763789","heartbeat_front_addr":"172.21.15.72:6815/4027763789","state":["exists","up"]},{"osd":4,"uuid":"d2cfeea7-1661-4156-bdcd-f770a8d1a878","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":3718867827},{"type":"v1","addr":"172.21.15.76:6809","nonce":3718867827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":3718867827},{"type":"v1","addr":"172.21.15.76:6811","nonce":3718867827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":3718867827},{"type":"v1","addr":"172.21.15.76:6815","nonce":3718867827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":3718867827},{"type":"v1","addr":"172.21.15.76:6813","nonce":3718867827}]},"public_addr":"172.21.15.76:6809/3718867827","cluster_addr":"172.21.15.76:6811/3718867827","heartbeat_back_addr":"172.21.15.76:6815/3718867827","heartbeat_front_addr":"172.21.15.76:6813/3718867827","state":["exists","up"]},{"osd":5,"uuid":"f3896ad6-a040-4ea3-a431-ff9b7fc05c36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":2543954772},{"type":"v1","addr":"172.21.15.164:6809","nonce":2543954772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":2543954772},{"type":"v1","addr":"172.21.15.164:6811","nonce":2543954772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":2543954772},{"type":"v1","addr":"172.21.15.164:6815","nonce":2543954772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":2543954772},{"type":"v1","addr":"172.21.15.164:6813","nonce":2543954772}]},"public_addr":"172.21.15.164:6809/2543954772","cluster_addr":"172.21.15.164:6811/2543954772","heartbeat_back_addr":"172.21.15.164:6815/2543954772","heartbeat_front_addr":"172.21.15.164:6813/2543954772","state":["exists","up"]},{"osd":6,"uuid":"d5d56650-be7e-4ba5-ba3a-d7d473294dab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6818","nonce":2080659027},{"type":"v1","addr":"172.21.15.72:6819","nonce":2080659027}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6820","nonce":2080659027},{"type":"v1","addr":"172.21.15.72:6821","nonce":2080659027}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6824","nonce":2080659027},{"type":"v1","addr":"172.21.15.72:6825","nonce":2080659027}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6822","nonce":2080659027},{"type":"v1","addr":"172.21.15.72:6823","nonce":2080659027}]},"public_addr":"172.21.15.72:6819/2080659027","cluster_addr":"172.21.15.72:6821/2080659027","heartbeat_back_addr":"172.21.15.72:6825/2080659027","heartbeat_front_addr":"172.21.15.72:6823/2080659027","state":["exists","up"]},{"osd":7,"uuid":"18975b5a-9309-4a0f-9f37-3ce09c2e745d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":3144631323},{"type":"v1","addr":"172.21.15.76:6817","nonce":3144631323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":3144631323},{"type":"v1","addr":"172.21.15.76:6819","nonce":3144631323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":3144631323},{"type":"v1","addr":"172.21.15.76:6823","nonce":3144631323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":3144631323},{"type":"v1","addr":"172.21.15.76:6821","nonce":3144631323}]},"public_addr":"172.21.15.76:6817/3144631323","cluster_addr":"172.21.15.76:6819/3144631323","heartbeat_back_addr":"172.21.15.76:6823/3144631323","heartbeat_front_addr":"172.21.15.76:6821/3144631323","state":["exists","up"]},{"osd":8,"uuid":"779e9c1e-8e97-48d4-a854-68ca13f38e0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":627888883},{"type":"v1","addr":"172.21.15.164:6817","nonce":627888883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":627888883},{"type":"v1","addr":"172.21.15.164:6819","nonce":627888883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":627888883},{"type":"v1","addr":"172.21.15.164:6823","nonce":627888883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":627888883},{"type":"v1","addr":"172.21.15.164:6821","nonce":627888883}]},"public_addr":"172.21.15.164:6817/627888883","cluster_addr":"172.21.15.164:6819/627888883","heartbeat_back_addr":"172.21.15.164:6823/627888883","heartbeat_front_addr":"172.21.15.164:6821/627888883","state":["exists","up"]},{"osd":9,"uuid":"d0317850-7b2d-4850-b2cf-0b919505c1fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6826","nonce":1273118485},{"type":"v1","addr":"172.21.15.72:6827","nonce":1273118485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6828","nonce":1273118485},{"type":"v1","addr":"172.21.15.72:6829","nonce":1273118485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6832","nonce":1273118485},{"type":"v1","addr":"172.21.15.72:6833","nonce":1273118485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6830","nonce":1273118485},{"type":"v1","addr":"172.21.15.72:6831","nonce":1273118485}]},"public_addr":"172.21.15.72:6827/1273118485","cluster_addr":"172.21.15.72:6829/1273118485","heartbeat_back_addr":"172.21.15.72:6833/1273118485","heartbeat_front_addr":"172.21.15.72:6831/1273118485","state":["exists","up"]},{"osd":10,"uuid":"e0aa38d8-6011-4d2a-b67b-5bff6d999e47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":2808954459},{"type":"v1","addr":"172.21.15.76:6825","nonce":2808954459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":2808954459},{"type":"v1","addr":"172.21.15.76:6827","nonce":2808954459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":2808954459},{"type":"v1","addr":"172.21.15.76:6831","nonce":2808954459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":2808954459},{"type":"v1","addr":"172.21.15.76:6829","nonce":2808954459}]},"public_addr":"172.21.15.76:6825/2808954459","cluster_addr":"172.21.15.76:6827/2808954459","heartbeat_back_addr":"172.21.15.76:6831/2808954459","heartbeat_front_addr":"172.21.15.76:6829/2808954459","state":["exists","up"]},{"osd":11,"uuid":"981b3d4d-916e-4937-9916-7cc21a891e43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":2354951784},{"type":"v1","addr":"172.21.15.164:6825","nonce":2354951784}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":2354951784},{"type":"v1","addr":"172.21.15.164:6827","nonce":2354951784}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":2354951784},{"type":"v1","addr":"172.21.15.164:6831","nonce":2354951784}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":2354951784},{"type":"v1","addr":"172.21.15.164:6829","nonce":2354951784}]},"public_addr":"172.21.15.164:6825/2354951784","cluster_addr":"172.21.15.164:6827/2354951784","heartbeat_back_addr":"172.21.15.164:6831/2354951784","heartbeat_front_addr":"172.21.15.164:6829/2354951784","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:18.997924+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:52.439081+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:46:25.893288+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:47:00.112807+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:47:35.116874+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:48:09.216344+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:48:45.028853+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:49:19.905397+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:49:55.030536+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:50:30.767662+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:51:06.423267+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:51:41.631211+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.1a","mappings":[{"from":5,"to":7}]},{"pgid":"3.1b","mappings":[{"from":4,"to":6}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.28","mappings":[{"from":5,"to":6}]},{"pgid":"3.2a","mappings":[{"from":3,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.72:0/416277784":"2024-04-27T21:43:09.602791+0000","172.21.15.72:6800/4010358672":"2024-04-27T21:43:09.602791+0000","172.21.15.72:6801/1012617371":"2024-04-27T21:42:34.684837+0000","172.21.15.72:6801/4010358672":"2024-04-27T21:43:09.602791+0000","172.21.15.72:0/3555448179":"2024-04-27T21:43:09.602791+0000","172.21.15.72:6800/1012617371":"2024-04-27T21:42:34.684837+0000","172.21.15.72:0/2920378761":"2024-04-27T21:42:34.684837+0000","172.21.15.72:0/1933740041":"2024-04-27T21:43:09.602791+0000","172.21.15.72:0/3225295775":"2024-04-27T21:42:34.684837+0000","172.21.15.72:0/2326016180":"2024-04-27T21:42:34.684837+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:54:35.913 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:35.912+0000 7f7282000640 1 -- 172.21.15.72:0/2395842473 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f72600a3c40 msgr2=0x7f72600a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:35.913 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:35.912+0000 7f7282000640 1 --2- 172.21.15.72:0/2395842473 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f72600a3c40 0x7f72600a6100 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f726c005fd0 tx=0x7f726c005850 comp rx=0 tx=0).stop 2024-04-26T21:54:35.913 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:35.912+0000 7f7282000640 1 -- 172.21.15.72:0/2395842473 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f727c076970 msgr2=0x7f727c19f570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:35.913 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:35.912+0000 7f7282000640 1 --2- 172.21.15.72:0/2395842473 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f727c076970 0x7f727c19f570 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f726800c920 tx=0x7f726800cde0 comp rx=0 tx=0).stop 2024-04-26T21:54:35.913 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:35.913+0000 7f7282000640 1 -- 172.21.15.72:0/2395842473 shutdown_connections 2024-04-26T21:54:35.913 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:35.913+0000 7f7282000640 1 --2- 172.21.15.72:0/2395842473 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f72600a3c40 0x7f72600a6100 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:35.913 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:35.913+0000 7f7282000640 1 --2- 172.21.15.72:0/2395842473 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f727c10ef50 0x7f727c1a65e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:35.913 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:35.913+0000 7f7282000640 1 --2- 172.21.15.72:0/2395842473 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f727c076970 0x7f727c19f570 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:35.913 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:35.913+0000 7f7282000640 1 --2- 172.21.15.72:0/2395842473 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f727c075390 0x7f727c19f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:35.914 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:35.913+0000 7f7282000640 1 -- 172.21.15.72:0/2395842473 >> 172.21.15.72:0/2395842473 conn(0x7f727c0fdf10 msgr2=0x7f727c1022f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:54:35.914 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:35.913+0000 7f7282000640 1 -- 172.21.15.72:0/2395842473 shutdown_connections 2024-04-26T21:54:35.914 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:35.913+0000 7f7282000640 1 -- 172.21.15.72:0/2395842473 wait complete. 2024-04-26T21:54:35.932 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:46:29.785427+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T21:52:48.320391+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T21:52:49.517493+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:54:35.933 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:54:36.122 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:36.121+0000 7f145f000640 1 -- 172.21.15.72:0/2175746349 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1458109fc0 msgr2=0x7f145810a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:36.122 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:36.121+0000 7f145f000640 1 --2- 172.21.15.72:0/2175746349 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1458109fc0 0x7f145810a440 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f144c009900 tx=0x7f144c02a410 comp rx=0 tx=0).stop 2024-04-26T21:54:36.122 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:36.121+0000 7f145f000640 1 -- 172.21.15.72:0/2175746349 shutdown_connections 2024-04-26T21:54:36.122 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:36.121+0000 7f145f000640 1 --2- 172.21.15.72:0/2175746349 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f145810a980 0x7f14581112a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:36.122 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:36.121+0000 7f145f000640 1 --2- 172.21.15.72:0/2175746349 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1458109fc0 0x7f145810a440 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:36.122 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:36.121+0000 7f145f000640 1 --2- 172.21.15.72:0/2175746349 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1458108e70 0x7f1458109290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:36.122 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:36.121+0000 7f145f000640 1 -- 172.21.15.72:0/2175746349 >> 172.21.15.72:0/2175746349 conn(0x7f1458077e00 msgr2=0x7f145807a240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:54:36.123 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:36.122+0000 7f145f000640 1 -- 172.21.15.72:0/2175746349 shutdown_connections 2024-04-26T21:54:36.123 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:36.122+0000 7f145f000640 1 -- 172.21.15.72:0/2175746349 wait complete. 2024-04-26T21:54:36.124 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:36.122+0000 7f145f000640 1 Processor -- start 2024-04-26T21:54:36.125 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:36.123+0000 7f145f000640 1 -- start start 2024-04-26T21:54:36.125 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:36.123+0000 7f145f000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1458108e70 0x7f145819eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:36.125 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:36.123+0000 7f145f000640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1458109fc0 0x7f145819f430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:36.125 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:36.123+0000 7f145f000640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f145810a980 0x7f14581a64a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:36.125 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:36.123+0000 7f145f000640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f1458114850 con 0x7f145810a980 2024-04-26T21:54:36.125 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:36.123+0000 7f145f000640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f14581146d0 con 0x7f1458109fc0 2024-04-26T21:54:36.125 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:36.123+0000 7f145f000640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f14581149d0 con 0x7f1458108e70 2024-04-26T21:54:36.125 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:36.123+0000 7f145e600640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f145810a980 0x7f14581a64a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:36.125 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:36.123+0000 7f145e600640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f145810a980 0x7f14581a64a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:39050/0 (socket says 172.21.15.72:39050) 2024-04-26T21:54:36.125 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:36.123+0000 7f145e600640 1 -- 172.21.15.72:0/2712497660 learned_addr learned my addr 172.21.15.72:0/2712497660 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:54:36.125 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:36.123+0000 7f145d200640 1 --2- 172.21.15.72:0/2712497660 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1458109fc0 0x7f145819f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:36.125 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:36.123+0000 7f145dc00640 1 --2- 172.21.15.72:0/2712497660 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1458108e70 0x7f145819eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:36.126 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:36.123+0000 7f145e600640 1 -- 172.21.15.72:0/2712497660 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1458108e70 msgr2=0x7f145819eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:36.126 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:36.123+0000 7f145e600640 1 --2- 172.21.15.72:0/2712497660 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1458108e70 0x7f145819eef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:36.126 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:36.123+0000 7f145e600640 1 -- 172.21.15.72:0/2712497660 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1458109fc0 msgr2=0x7f145819f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:36.126 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:36.123+0000 7f145e600640 1 --2- 172.21.15.72:0/2712497660 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1458109fc0 0x7f145819f430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:36.126 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:36.123+0000 7f145e600640 1 -- 172.21.15.72:0/2712497660 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14581a6b10 con 0x7f145810a980 2024-04-26T21:54:36.126 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:36.124+0000 7f145e600640 1 --2- 172.21.15.72:0/2712497660 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f145810a980 0x7f14581a64a0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f145000eaa0 tx=0x7f145000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:54:36.126 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:36.124+0000 7f1456000640 1 -- 172.21.15.72:0/2712497660 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f145000cdb0 con 0x7f145810a980 2024-04-26T21:54:36.126 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:36.124+0000 7f1456000640 1 -- 172.21.15.72:0/2712497660 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1450005020 con 0x7f145810a980 2024-04-26T21:54:36.126 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:36.124+0000 7f1456000640 1 -- 172.21.15.72:0/2712497660 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14500116d0 con 0x7f145810a980 2024-04-26T21:54:36.126 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:36.124+0000 7f145f000640 1 -- 172.21.15.72:0/2712497660 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14581a6e00 con 0x7f145810a980 2024-04-26T21:54:36.126 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:36.124+0000 7f145f000640 1 -- 172.21.15.72:0/2712497660 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14581a72c0 con 0x7f145810a980 2024-04-26T21:54:36.127 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:36.125+0000 7f145f000640 1 -- 172.21.15.72:0/2712497660 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1458110200 con 0x7f145810a980 2024-04-26T21:54:36.127 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:36.127+0000 7f1456000640 1 -- 172.21.15.72:0/2712497660 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f14500051d0 con 0x7f145810a980 2024-04-26T21:54:36.128 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:36.128+0000 7f1456000640 1 --2- 172.21.15.72:0/2712497660 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f14400a3cc0 0x7f14400a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:36.128 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:36.128+0000 7f1456000640 1 -- 172.21.15.72:0/2712497660 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7f14500d0880 con 0x7f145810a980 2024-04-26T21:54:36.129 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:36.129+0000 7f145dc00640 1 --2- 172.21.15.72:0/2712497660 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f14400a3cc0 0x7f14400a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:36.129 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:36.129+0000 7f145dc00640 1 --2- 172.21.15.72:0/2712497660 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f14400a3cc0 0x7f14400a6180 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f144800ba40 tx=0x7f1448005cc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:54:36.132 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:36.133+0000 7f1456000640 1 -- 172.21.15.72:0/2712497660 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f1450016030 con 0x7f145810a980 2024-04-26T21:54:36.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:54:35 smithi076 ceph-mon[31823]: from='client.? 172.21.15.72:0/2562473914' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:54:36.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:54:35 smithi076 ceph-mon[31823]: from='client.? 172.21.15.72:0/2395842473' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:54:36.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:54:35 smithi072 ceph-mon[28138]: from='client.? 172.21.15.72:0/2562473914' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:54:36.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:54:35 smithi072 ceph-mon[28138]: from='client.? 172.21.15.72:0/2395842473' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:54:36.330 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:36.329+0000 7f145f000640 1 -- 172.21.15.72:0/2712497660 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f145802dd20 con 0x7f145810a980 2024-04-26T21:54:36.330 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:36.330+0000 7f1456000640 1 -- 172.21.15.72:0/2712497660 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) ==== 93+0+10 (secure 0 0 0) 0x7f145009d020 con 0x7f145810a980 2024-04-26T21:54:36.331 INFO:teuthology.orchestra.run.smithi072.stdout:pg_num: 1 2024-04-26T21:54:36.333 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:36.333+0000 7f145f000640 1 -- 172.21.15.72:0/2712497660 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f14400a3cc0 msgr2=0x7f14400a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:36.333 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:36.333+0000 7f145f000640 1 --2- 172.21.15.72:0/2712497660 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f14400a3cc0 0x7f14400a6180 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f144800ba40 tx=0x7f1448005cc0 comp rx=0 tx=0).stop 2024-04-26T21:54:36.333 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:36.333+0000 7f145f000640 1 -- 172.21.15.72:0/2712497660 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f145810a980 msgr2=0x7f14581a64a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:36.333 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:36.333+0000 7f145f000640 1 --2- 172.21.15.72:0/2712497660 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f145810a980 0x7f14581a64a0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f145000eaa0 tx=0x7f145000ef60 comp rx=0 tx=0).stop 2024-04-26T21:54:36.334 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:36.333+0000 7f145f000640 1 -- 172.21.15.72:0/2712497660 shutdown_connections 2024-04-26T21:54:36.334 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:36.333+0000 7f145f000640 1 --2- 172.21.15.72:0/2712497660 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f145810a980 0x7f14581a64a0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:36.334 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:36.333+0000 7f145f000640 1 --2- 172.21.15.72:0/2712497660 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1458109fc0 0x7f145819f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:36.334 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:36.333+0000 7f145f000640 1 --2- 172.21.15.72:0/2712497660 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f14400a3cc0 0x7f14400a6180 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:36.334 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:36.333+0000 7f145f000640 1 --2- 172.21.15.72:0/2712497660 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1458108e70 0x7f145819eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:36.334 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:36.333+0000 7f145f000640 1 -- 172.21.15.72:0/2712497660 >> 172.21.15.72:0/2712497660 conn(0x7f1458077e00 msgr2=0x7f145810d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:54:36.334 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:36.333+0000 7f145f000640 1 -- 172.21.15.72:0/2712497660 shutdown_connections 2024-04-26T21:54:36.334 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:36.333+0000 7f145f000640 1 -- 172.21.15.72:0/2712497660 wait complete. 2024-04-26T21:54:36.353 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T21:54:36.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:54:35 smithi164 ceph-mon[31916]: from='client.? 172.21.15.72:0/2562473914' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:54:36.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:54:35 smithi164 ceph-mon[31916]: from='client.? 172.21.15.72:0/2395842473' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:54:36.561 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:36.561+0000 7f2bef400640 1 -- 172.21.15.72:0/2911770248 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2be8103b60 msgr2=0x7f2be8103fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:36.561 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:36.561+0000 7f2bef400640 1 --2- 172.21.15.72:0/2911770248 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2be8103b60 0x7f2be8103fe0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f2bd0009900 tx=0x7f2bd002a410 comp rx=0 tx=0).stop 2024-04-26T21:54:36.561 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:36.561+0000 7f2bef400640 1 -- 172.21.15.72:0/2911770248 shutdown_connections 2024-04-26T21:54:36.561 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:36.561+0000 7f2bef400640 1 --2- 172.21.15.72:0/2911770248 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2be8104520 0x7f2be810adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:36.561 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:36.561+0000 7f2bef400640 1 --2- 172.21.15.72:0/2911770248 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2be8103b60 0x7f2be8103fe0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:36.561 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:36.561+0000 7f2bef400640 1 --2- 172.21.15.72:0/2911770248 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2be8102970 0x7f2be8102d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:36.561 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:36.561+0000 7f2bef400640 1 -- 172.21.15.72:0/2911770248 >> 172.21.15.72:0/2911770248 conn(0x7f2be80fdee0 msgr2=0x7f2be8100320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:54:36.562 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:36.561+0000 7f2bef400640 1 -- 172.21.15.72:0/2911770248 shutdown_connections 2024-04-26T21:54:36.562 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:36.561+0000 7f2bef400640 1 -- 172.21.15.72:0/2911770248 wait complete. 2024-04-26T21:54:36.563 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:36.562+0000 7f2bef400640 1 Processor -- start 2024-04-26T21:54:36.564 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:36.562+0000 7f2bef400640 1 -- start start 2024-04-26T21:54:36.564 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:36.562+0000 7f2bef400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2be8102970 0x7f2be819ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:36.564 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:36.562+0000 7f2bef400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2be8103b60 0x7f2be819b290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:36.564 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:36.562+0000 7f2bef400640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2be8104520 0x7f2be81a2300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:36.564 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:36.562+0000 7f2bef400640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f2be810e350 con 0x7f2be8102970 2024-04-26T21:54:36.564 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:36.562+0000 7f2bef400640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f2be810e1d0 con 0x7f2be8104520 2024-04-26T21:54:36.564 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:36.562+0000 7f2bef400640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f2be810e4d0 con 0x7f2be8103b60 2024-04-26T21:54:36.564 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:36.562+0000 7f2beca00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2be8102970 0x7f2be819ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:36.564 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:36.562+0000 7f2beca00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2be8102970 0x7f2be819ad50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:39082/0 (socket says 172.21.15.72:39082) 2024-04-26T21:54:36.564 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:36.562+0000 7f2beca00640 1 -- 172.21.15.72:0/3907297749 learned_addr learned my addr 172.21.15.72:0/3907297749 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:54:36.564 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:36.563+0000 7f2beca00640 1 -- 172.21.15.72:0/3907297749 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2be8103b60 msgr2=0x7f2be819b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:36.564 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:36.563+0000 7f2beca00640 1 --2- 172.21.15.72:0/3907297749 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2be8103b60 0x7f2be819b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:36.564 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:36.563+0000 7f2beca00640 1 -- 172.21.15.72:0/3907297749 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2be8104520 msgr2=0x7f2be81a2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:36.564 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:36.563+0000 7f2beca00640 1 --2- 172.21.15.72:0/3907297749 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2be8104520 0x7f2be81a2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:36.564 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:36.563+0000 7f2beca00640 1 -- 172.21.15.72:0/3907297749 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2be81a2970 con 0x7f2be8102970 2024-04-26T21:54:36.565 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:36.563+0000 7f2beca00640 1 --2- 172.21.15.72:0/3907297749 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2be8102970 0x7f2be819ad50 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f2bd800c970 tx=0x7f2bd800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:54:36.565 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:36.563+0000 7f2be5600640 1 -- 172.21.15.72:0/3907297749 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bd8004220 con 0x7f2be8102970 2024-04-26T21:54:36.565 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:36.563+0000 7f2be5600640 1 -- 172.21.15.72:0/3907297749 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2bd8007e40 con 0x7f2be8102970 2024-04-26T21:54:36.565 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:36.563+0000 7f2be5600640 1 -- 172.21.15.72:0/3907297749 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bd80047c0 con 0x7f2be8102970 2024-04-26T21:54:36.565 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:36.563+0000 7f2bef400640 1 -- 172.21.15.72:0/3907297749 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2be81a2c60 con 0x7f2be8102970 2024-04-26T21:54:36.565 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:36.563+0000 7f2bef400640 1 -- 172.21.15.72:0/3907297749 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2be81a3160 con 0x7f2be8102970 2024-04-26T21:54:36.565 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:36.565+0000 7f2bef400640 1 -- 172.21.15.72:0/3907297749 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2be81a3390 con 0x7f2be8102970 2024-04-26T21:54:36.566 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:36.566+0000 7f2be5600640 1 -- 172.21.15.72:0/3907297749 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f2bd80079b0 con 0x7f2be8102970 2024-04-26T21:54:36.568 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:36.568+0000 7f2be5600640 1 --2- 172.21.15.72:0/3907297749 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f2bc80a3c40 0x7f2bc80a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:36.568 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:36.568+0000 7f2be5600640 1 -- 172.21.15.72:0/3907297749 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7f2bd80d0400 con 0x7f2be8102970 2024-04-26T21:54:36.568 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:36.568+0000 7f2be7e00640 1 --2- 172.21.15.72:0/3907297749 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f2bc80a3c40 0x7f2bc80a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:36.569 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:36.569+0000 7f2be7e00640 1 --2- 172.21.15.72:0/3907297749 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f2bc80a3c40 0x7f2bc80a6100 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f2bd002a930 tx=0x7f2bd0000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:54:36.571 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:36.572+0000 7f2be5600640 1 -- 172.21.15.72:0/3907297749 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f2bd8014030 con 0x7f2be8102970 2024-04-26T21:54:36.761 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:36.761+0000 7f2bef400640 1 -- 172.21.15.72:0/3907297749 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f2be81a3880 con 0x7f2be8102970 2024-04-26T21:54:36.762 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:36.762+0000 7f2be5600640 1 -- 172.21.15.72:0/3907297749 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v91) ==== 104+0+11 (secure 0 0 0) 0x7f2bd809cc00 con 0x7f2be8102970 2024-04-26T21:54:36.762 INFO:teuthology.orchestra.run.smithi072.stdout:pg_num: 32 2024-04-26T21:54:36.765 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:36.764+0000 7f2bef400640 1 -- 172.21.15.72:0/3907297749 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f2bc80a3c40 msgr2=0x7f2bc80a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:36.765 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:36.764+0000 7f2bef400640 1 --2- 172.21.15.72:0/3907297749 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f2bc80a3c40 0x7f2bc80a6100 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f2bd002a930 tx=0x7f2bd0000bc0 comp rx=0 tx=0).stop 2024-04-26T21:54:36.765 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:36.764+0000 7f2bef400640 1 -- 172.21.15.72:0/3907297749 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2be8102970 msgr2=0x7f2be819ad50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:36.765 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:36.764+0000 7f2bef400640 1 --2- 172.21.15.72:0/3907297749 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2be8102970 0x7f2be819ad50 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f2bd800c970 tx=0x7f2bd800ce30 comp rx=0 tx=0).stop 2024-04-26T21:54:36.765 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:36.764+0000 7f2bef400640 1 -- 172.21.15.72:0/3907297749 shutdown_connections 2024-04-26T21:54:36.765 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:36.764+0000 7f2bef400640 1 --2- 172.21.15.72:0/3907297749 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2be8104520 0x7f2be81a2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:36.765 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:36.765+0000 7f2bef400640 1 --2- 172.21.15.72:0/3907297749 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f2bc80a3c40 0x7f2bc80a6100 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:36.765 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:36.765+0000 7f2bef400640 1 --2- 172.21.15.72:0/3907297749 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2be8103b60 0x7f2be819b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:36.765 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:36.765+0000 7f2bef400640 1 --2- 172.21.15.72:0/3907297749 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2be8102970 0x7f2be819ad50 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:36.765 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:36.765+0000 7f2bef400640 1 -- 172.21.15.72:0/3907297749 >> 172.21.15.72:0/3907297749 conn(0x7f2be80fdee0 msgr2=0x7f2be8109350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:54:36.766 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:36.765+0000 7f2bef400640 1 -- 172.21.15.72:0/3907297749 shutdown_connections 2024-04-26T21:54:36.766 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:36.765+0000 7f2bef400640 1 -- 172.21.15.72:0/3907297749 wait complete. 2024-04-26T21:54:36.781 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T21:54:36.983 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:36.983+0000 7f0150e00640 1 -- 172.21.15.72:0/1643255517 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f014c104990 msgr2=0x7f014c104db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:36.984 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:36.983+0000 7f0150e00640 1 --2- 172.21.15.72:0/1643255517 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f014c104990 0x7f014c104db0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f013c009900 tx=0x7f013c02a410 comp rx=0 tx=0).stop 2024-04-26T21:54:36.984 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:36.983+0000 7f0150e00640 1 -- 172.21.15.72:0/1643255517 shutdown_connections 2024-04-26T21:54:36.984 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:36.983+0000 7f0150e00640 1 --2- 172.21.15.72:0/1643255517 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f014c106540 0x7f014c10cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:36.984 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:36.983+0000 7f0150e00640 1 --2- 172.21.15.72:0/1643255517 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f014c105b80 0x7f014c106000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:36.984 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:36.983+0000 7f0150e00640 1 --2- 172.21.15.72:0/1643255517 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f014c104990 0x7f014c104db0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:36.984 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:36.983+0000 7f0150e00640 1 -- 172.21.15.72:0/1643255517 >> 172.21.15.72:0/1643255517 conn(0x7f014c0fff00 msgr2=0x7f014c102340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:54:36.984 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:36.983+0000 7f0150e00640 1 -- 172.21.15.72:0/1643255517 shutdown_connections 2024-04-26T21:54:36.984 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:36.983+0000 7f0150e00640 1 -- 172.21.15.72:0/1643255517 wait complete. 2024-04-26T21:54:36.986 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:36.984+0000 7f0150e00640 1 Processor -- start 2024-04-26T21:54:36.986 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:36.984+0000 7f0150e00640 1 -- start start 2024-04-26T21:54:36.986 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:36.984+0000 7f0150e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f014c104990 0x7f014c19ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:36.987 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:36.984+0000 7f0150e00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f014c105b80 0x7f014c19b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:36.987 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:36.984+0000 7f0150e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f014c106540 0x7f014c1a20e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:36.987 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:36.984+0000 7f0150e00640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f014c110370 con 0x7f014c104990 2024-04-26T21:54:36.987 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:36.984+0000 7f0150e00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f014c1101f0 con 0x7f014c105b80 2024-04-26T21:54:36.987 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:36.984+0000 7f0150e00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f014c1104f0 con 0x7f014c106540 2024-04-26T21:54:36.987 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:36.985+0000 7f014b400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f014c104990 0x7f014c19ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:36.987 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:36.985+0000 7f014b400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f014c104990 0x7f014c19ab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:39104/0 (socket says 172.21.15.72:39104) 2024-04-26T21:54:36.987 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:36.985+0000 7f014b400640 1 -- 172.21.15.72:0/1274583716 learned_addr learned my addr 172.21.15.72:0/1274583716 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:54:36.987 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:36.985+0000 7f0143e00640 1 --2- 172.21.15.72:0/1274583716 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f014c105b80 0x7f014c19b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:36.987 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:36.985+0000 7f014b400640 1 -- 172.21.15.72:0/1274583716 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f014c106540 msgr2=0x7f014c1a20e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:36.987 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:36.985+0000 7f014be00640 1 --2- 172.21.15.72:0/1274583716 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f014c106540 0x7f014c1a20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:36.987 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:36.985+0000 7f014b400640 1 --2- 172.21.15.72:0/1274583716 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f014c106540 0x7f014c1a20e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:36.987 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:36.985+0000 7f014b400640 1 -- 172.21.15.72:0/1274583716 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f014c105b80 msgr2=0x7f014c19b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:36.987 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:36.985+0000 7f014b400640 1 --2- 172.21.15.72:0/1274583716 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f014c105b80 0x7f014c19b070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:36.988 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:36.985+0000 7f014b400640 1 -- 172.21.15.72:0/1274583716 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f014c1a2750 con 0x7f014c104990 2024-04-26T21:54:36.988 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:36.985+0000 7f0143e00640 1 --2- 172.21.15.72:0/1274583716 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f014c105b80 0x7f014c19b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:54:36.988 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:36.985+0000 7f014b400640 1 --2- 172.21.15.72:0/1274583716 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f014c104990 0x7f014c19ab30 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f013c005ab0 tx=0x7f013c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:54:36.988 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:36.986+0000 7f0148c00640 1 -- 172.21.15.72:0/1274583716 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f013c03f030 con 0x7f014c104990 2024-04-26T21:54:36.988 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:36.986+0000 7f0148c00640 1 -- 172.21.15.72:0/1274583716 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f013c003c90 con 0x7f014c104990 2024-04-26T21:54:36.988 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:36.986+0000 7f0148c00640 1 -- 172.21.15.72:0/1274583716 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f013c035070 con 0x7f014c104990 2024-04-26T21:54:36.988 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:36.986+0000 7f0150e00640 1 -- 172.21.15.72:0/1274583716 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f014c1a29e0 con 0x7f014c104990 2024-04-26T21:54:36.988 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:36.986+0000 7f0150e00640 1 -- 172.21.15.72:0/1274583716 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f014c1a2e80 con 0x7f014c104990 2024-04-26T21:54:36.989 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:36.987+0000 7f0150e00640 1 -- 172.21.15.72:0/1274583716 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f014c10bda0 con 0x7f014c104990 2024-04-26T21:54:36.989 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:36.989+0000 7f0148c00640 1 -- 172.21.15.72:0/1274583716 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f013c042030 con 0x7f014c104990 2024-04-26T21:54:36.990 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:36.990+0000 7f0148c00640 1 --2- 172.21.15.72:0/1274583716 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f01300a3cc0 0x7f01300a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:36.990 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:36.990+0000 7f0148c00640 1 -- 172.21.15.72:0/1274583716 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7f013c0f0ff0 con 0x7f014c104990 2024-04-26T21:54:36.990 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:36.990+0000 7f0143e00640 1 --2- 172.21.15.72:0/1274583716 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f01300a3cc0 0x7f01300a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:36.991 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:36.991+0000 7f0143e00640 1 --2- 172.21.15.72:0/1274583716 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f01300a3cc0 0x7f01300a6180 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f014c19c040 tx=0x7f0134005e50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:54:36.994 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:36.994+0000 7f0148c00640 1 -- 172.21.15.72:0/1274583716 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f013c0bd790 con 0x7f014c104990 2024-04-26T21:54:37.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:54:36 smithi072 ceph-mon[28138]: pgmap v424: 97 pgs: 97 active+clean; 633 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-26T21:54:37.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:54:36 smithi072 ceph-mon[28138]: from='client.? 172.21.15.72:0/2712497660' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T21:54:37.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:54:36 smithi072 ceph-mon[28138]: from='client.? 172.21.15.72:0/3907297749' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T21:54:37.197 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:37.197+0000 7f0150e00640 1 -- 172.21.15.72:0/1274583716 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f014c063630 con 0x7f014c104990 2024-04-26T21:54:37.198 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:37.198+0000 7f0148c00640 1 -- 172.21.15.72:0/1274583716 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v91) ==== 100+0+11 (secure 0 0 0) 0x7f013c0bd090 con 0x7f014c104990 2024-04-26T21:54:37.198 INFO:teuthology.orchestra.run.smithi072.stdout:pg_num: 64 2024-04-26T21:54:37.201 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:37.201+0000 7f0150e00640 1 -- 172.21.15.72:0/1274583716 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f01300a3cc0 msgr2=0x7f01300a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:37.201 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:37.201+0000 7f0150e00640 1 --2- 172.21.15.72:0/1274583716 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f01300a3cc0 0x7f01300a6180 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f014c19c040 tx=0x7f0134005e50 comp rx=0 tx=0).stop 2024-04-26T21:54:37.201 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:37.201+0000 7f0150e00640 1 -- 172.21.15.72:0/1274583716 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f014c104990 msgr2=0x7f014c19ab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:37.201 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:37.201+0000 7f0150e00640 1 --2- 172.21.15.72:0/1274583716 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f014c104990 0x7f014c19ab30 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f013c005ab0 tx=0x7f013c0042f0 comp rx=0 tx=0).stop 2024-04-26T21:54:37.202 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:37.201+0000 7f0150e00640 1 -- 172.21.15.72:0/1274583716 shutdown_connections 2024-04-26T21:54:37.202 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:37.201+0000 7f0150e00640 1 --2- 172.21.15.72:0/1274583716 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f01300a3cc0 0x7f01300a6180 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:37.202 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:37.201+0000 7f0150e00640 1 --2- 172.21.15.72:0/1274583716 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f014c106540 0x7f014c1a20e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:37.202 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:37.201+0000 7f0150e00640 1 --2- 172.21.15.72:0/1274583716 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f014c105b80 0x7f014c19b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:37.202 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:37.201+0000 7f0150e00640 1 --2- 172.21.15.72:0/1274583716 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f014c104990 0x7f014c19ab30 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:37.202 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:37.201+0000 7f0150e00640 1 -- 172.21.15.72:0/1274583716 >> 172.21.15.72:0/1274583716 conn(0x7f014c0fff00 msgr2=0x7f014c108f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:54:37.202 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:37.201+0000 7f0150e00640 1 -- 172.21.15.72:0/1274583716 shutdown_connections 2024-04-26T21:54:37.202 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:37.201+0000 7f0150e00640 1 -- 172.21.15.72:0/1274583716 wait complete. 2024-04-26T21:54:37.221 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:54:37.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:54:36 smithi076 ceph-mon[31823]: pgmap v424: 97 pgs: 97 active+clean; 633 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-26T21:54:37.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:54:37 smithi076 ceph-mon[31823]: from='client.? 172.21.15.72:0/2712497660' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T21:54:37.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:54:37 smithi076 ceph-mon[31823]: from='client.? 172.21.15.72:0/3907297749' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T21:54:37.418 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:37.417+0000 7f5e0da00640 1 -- 172.21.15.72:0/2464785866 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5e081031d0 msgr2=0x7f5e081055c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:37.418 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:37.417+0000 7f5e0da00640 1 --2- 172.21.15.72:0/2464785866 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5e081031d0 0x7f5e081055c0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f5df8009900 tx=0x7f5df802a410 comp rx=0 tx=0).stop 2024-04-26T21:54:37.418 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:37.418+0000 7f5e0da00640 1 -- 172.21.15.72:0/2464785866 shutdown_connections 2024-04-26T21:54:37.418 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:37.418+0000 7f5e0da00640 1 --2- 172.21.15.72:0/2464785866 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5e08108430 0x7f5e0810a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:37.418 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:37.418+0000 7f5e0da00640 1 --2- 172.21.15.72:0/2464785866 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5e08105b00 0x7f5e08107ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:37.418 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:37.418+0000 7f5e0da00640 1 --2- 172.21.15.72:0/2464785866 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5e081031d0 0x7f5e081055c0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:37.419 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:37.418+0000 7f5e0da00640 1 -- 172.21.15.72:0/2464785866 >> 172.21.15.72:0/2464785866 conn(0x7f5e080fcbe0 msgr2=0x7f5e080ff000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:54:37.419 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:37.418+0000 7f5e0da00640 1 -- 172.21.15.72:0/2464785866 shutdown_connections 2024-04-26T21:54:37.419 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:37.418+0000 7f5e0da00640 1 -- 172.21.15.72:0/2464785866 wait complete. 2024-04-26T21:54:37.420 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:37.419+0000 7f5e0da00640 1 Processor -- start 2024-04-26T21:54:37.420 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:37.419+0000 7f5e0da00640 1 -- start start 2024-04-26T21:54:37.420 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:37.419+0000 7f5e0da00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5e081031d0 0x7f5e0819aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:37.420 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:37.419+0000 7f5e0da00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5e08105b00 0x7f5e0819b030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:37.420 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:37.420+0000 7f5e0da00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5e08108430 0x7f5e081a20a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:37.420 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:37.420+0000 7f5e0da00640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f5e0810e130 con 0x7f5e081031d0 2024-04-26T21:54:37.420 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:37.420+0000 7f5e0da00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f5e0810dfb0 con 0x7f5e08105b00 2024-04-26T21:54:37.420 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:37.420+0000 7f5e0da00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f5e0810e2b0 con 0x7f5e08108430 2024-04-26T21:54:37.420 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:37.420+0000 7f5e07e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5e081031d0 0x7f5e0819aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:37.420 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:37.420+0000 7f5e07e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5e081031d0 0x7f5e0819aaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:39122/0 (socket says 172.21.15.72:39122) 2024-04-26T21:54:37.420 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:37.420+0000 7f5e07e00640 1 -- 172.21.15.72:0/234151850 learned_addr learned my addr 172.21.15.72:0/234151850 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:54:37.422 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:37.420+0000 7f5e07400640 1 --2- 172.21.15.72:0/234151850 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5e08105b00 0x7f5e0819b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:37.422 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:37.420+0000 7f5e0d000640 1 --2- 172.21.15.72:0/234151850 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5e08108430 0x7f5e081a20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:37.422 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:37.420+0000 7f5e07e00640 1 -- 172.21.15.72:0/234151850 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5e08108430 msgr2=0x7f5e081a20a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:37.422 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:37.420+0000 7f5e07e00640 1 --2- 172.21.15.72:0/234151850 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5e08108430 0x7f5e081a20a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:37.422 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:37.420+0000 7f5e07e00640 1 -- 172.21.15.72:0/234151850 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5e08105b00 msgr2=0x7f5e0819b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:37.422 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:37.420+0000 7f5e07e00640 1 --2- 172.21.15.72:0/234151850 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5e08105b00 0x7f5e0819b030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:37.422 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:37.420+0000 7f5e07e00640 1 -- 172.21.15.72:0/234151850 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e081a2710 con 0x7f5e081031d0 2024-04-26T21:54:37.422 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:37.421+0000 7f5e07e00640 1 --2- 172.21.15.72:0/234151850 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5e081031d0 0x7f5e0819aaf0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f5df8009a20 tx=0x7f5df80040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:54:37.422 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:37.421+0000 7f5e04c00640 1 -- 172.21.15.72:0/234151850 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5df803f030 con 0x7f5e081031d0 2024-04-26T21:54:37.422 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:37.421+0000 7f5e04c00640 1 -- 172.21.15.72:0/234151850 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5df8003a80 con 0x7f5e081031d0 2024-04-26T21:54:37.422 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:37.421+0000 7f5e0da00640 1 -- 172.21.15.72:0/234151850 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e081a29a0 con 0x7f5e081031d0 2024-04-26T21:54:37.423 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:37.421+0000 7f5e04c00640 1 -- 172.21.15.72:0/234151850 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5df8035070 con 0x7f5e081031d0 2024-04-26T21:54:37.423 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:37.421+0000 7f5e0da00640 1 -- 172.21.15.72:0/234151850 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e081a2ee0 con 0x7f5e081031d0 2024-04-26T21:54:37.423 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:37.422+0000 7f5e0da00640 1 -- 172.21.15.72:0/234151850 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e08109f80 con 0x7f5e081031d0 2024-04-26T21:54:37.424 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:37.424+0000 7f5e04c00640 1 -- 172.21.15.72:0/234151850 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f5df8042030 con 0x7f5e081031d0 2024-04-26T21:54:37.426 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:37.426+0000 7f5e04c00640 1 --2- 172.21.15.72:0/234151850 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f5dec0a3cc0 0x7f5dec0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:37.426 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:37.426+0000 7f5e04c00640 1 -- 172.21.15.72:0/234151850 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7f5df80f0d10 con 0x7f5e081031d0 2024-04-26T21:54:37.426 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:37.426+0000 7f5e07400640 1 --2- 172.21.15.72:0/234151850 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f5dec0a3cc0 0x7f5dec0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:37.426 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:37.427+0000 7f5e07400640 1 --2- 172.21.15.72:0/234151850 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f5dec0a3cc0 0x7f5dec0a6180 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f5e0819c000 tx=0x7f5df4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:54:37.430 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:37.430+0000 7f5e04c00640 1 -- 172.21.15.72:0/234151850 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f5df80f6020 con 0x7f5e081031d0 2024-04-26T21:54:37.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:54:37 smithi164 ceph-mon[31916]: pgmap v424: 97 pgs: 97 active+clean; 633 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-26T21:54:37.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:54:37 smithi164 ceph-mon[31916]: from='client.? 172.21.15.72:0/2712497660' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T21:54:37.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:54:37 smithi164 ceph-mon[31916]: from='client.? 172.21.15.72:0/3907297749' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T21:54:37.671 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:37.670+0000 7f5e0da00640 1 -- 172.21.15.72:0/234151850 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5e0802dad0 con 0x7f5e081031d0 2024-04-26T21:54:37.672 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:37.672+0000 7f5e04c00640 1 -- 172.21.15.72:0/234151850 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11931 (secure 0 0 0) 0x7f5df80bd4b0 con 0x7f5e081031d0 2024-04-26T21:54:37.672 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T21:54:37.673 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6835/1320381959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6834","nonce":1320381959},{"type":"v1","addr":"172.21.15.72:6835","nonce":1320381959}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6837/2142200512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6836","nonce":2142200512},{"type":"v1","addr":"172.21.15.72:6837","nonce":2142200512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T21:52:50.567857+0000","modified":"2024-04-26T21:52:58.508126+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34517,"mds_1":34472,"mds_2":34442,"mds_3":34406,"mds_4":24463},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14652":{"gid":14652,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.72:6839/1312898281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6838","nonce":1312898281},{"type":"v1","addr":"172.21.15.72:6839","nonce":1312898281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14664":{"gid":14664,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.72:6841/3257692339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6840","nonce":3257692339},{"type":"v1","addr":"172.21.15.72:6841","nonce":3257692339}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24388":{"gid":24388,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6833/4236668465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":4236668465},{"type":"v1","addr":"172.21.15.164:6833","nonce":4236668465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6835/2391114169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":2391114169},{"type":"v1","addr":"172.21.15.164:6835","nonce":2391114169}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6837/3585829992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":3585829992},{"type":"v1","addr":"172.21.15.164:6837","nonce":3585829992}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.164:6839/1636226820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":1636226820},{"type":"v1","addr":"172.21.15.164:6839","nonce":1636226820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34406":{"gid":34406,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":15,"addr":"172.21.15.76:6833/3869483750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":3869483750},{"type":"v1","addr":"172.21.15.76:6833","nonce":3869483750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34442":{"gid":34442,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.76:6835/3517429435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":3517429435},{"type":"v1","addr":"172.21.15.76:6835","nonce":3517429435}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34472":{"gid":34472,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.76:6837/1150678991","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":1150678991},{"type":"v1","addr":"172.21.15.76:6837","nonce":1150678991}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34517":{"gid":34517,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.76:6839/327145636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6838","nonce":327145636},{"type":"v1","addr":"172.21.15.76:6839","nonce":327145636}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34517,"qdb_cluster":[34406,24463,34442,34517,34472]},"id":1}]} 2024-04-26T21:54:37.673 INFO:teuthology.orchestra.run.smithi072.stderr:dumped fsmap epoch 26 2024-04-26T21:54:37.675 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:37.674+0000 7f5e0da00640 1 -- 172.21.15.72:0/234151850 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f5dec0a3cc0 msgr2=0x7f5dec0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:37.675 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:37.674+0000 7f5e0da00640 1 --2- 172.21.15.72:0/234151850 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f5dec0a3cc0 0x7f5dec0a6180 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f5e0819c000 tx=0x7f5df4005f90 comp rx=0 tx=0).stop 2024-04-26T21:54:37.675 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:37.674+0000 7f5e0da00640 1 -- 172.21.15.72:0/234151850 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5e081031d0 msgr2=0x7f5e0819aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:37.675 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:37.674+0000 7f5e0da00640 1 --2- 172.21.15.72:0/234151850 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5e081031d0 0x7f5e0819aaf0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f5df8009a20 tx=0x7f5df80040c0 comp rx=0 tx=0).stop 2024-04-26T21:54:37.675 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:37.674+0000 7f5e0da00640 1 -- 172.21.15.72:0/234151850 shutdown_connections 2024-04-26T21:54:37.675 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:37.674+0000 7f5e0da00640 1 --2- 172.21.15.72:0/234151850 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f5dec0a3cc0 0x7f5dec0a6180 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:37.675 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:37.674+0000 7f5e0da00640 1 --2- 172.21.15.72:0/234151850 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5e08108430 0x7f5e081a20a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:37.675 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:37.674+0000 7f5e0da00640 1 --2- 172.21.15.72:0/234151850 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5e08105b00 0x7f5e0819b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:37.675 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:37.675+0000 7f5e0da00640 1 --2- 172.21.15.72:0/234151850 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5e081031d0 0x7f5e0819aaf0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:37.675 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:37.675+0000 7f5e0da00640 1 -- 172.21.15.72:0/234151850 >> 172.21.15.72:0/234151850 conn(0x7f5e080fcbe0 msgr2=0x7f5e081029c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:54:37.675 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:37.675+0000 7f5e0da00640 1 -- 172.21.15.72:0/234151850 shutdown_connections 2024-04-26T21:54:37.675 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:37.675+0000 7f5e0da00640 1 -- 172.21.15.72:0/234151850 wait complete. 2024-04-26T21:54:37.693 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:54:37.885 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:37.884+0000 7fda57000640 1 -- 172.21.15.72:0/2879522153 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fda50108450 msgr2=0x7fda5010a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:37.885 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:37.884+0000 7fda57000640 1 --2- 172.21.15.72:0/2879522153 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fda50108450 0x7fda5010a8a0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fda40009900 tx=0x7fda4002a410 comp rx=0 tx=0).stop 2024-04-26T21:54:37.885 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:37.884+0000 7fda57000640 1 -- 172.21.15.72:0/2879522153 shutdown_connections 2024-04-26T21:54:37.885 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:37.884+0000 7fda57000640 1 --2- 172.21.15.72:0/2879522153 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fda50108450 0x7fda5010a8a0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:37.885 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:37.884+0000 7fda57000640 1 --2- 172.21.15.72:0/2879522153 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fda50105b20 0x7fda50107f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:37.885 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:37.884+0000 7fda57000640 1 --2- 172.21.15.72:0/2879522153 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fda501031f0 0x7fda501055e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:37.885 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:37.884+0000 7fda57000640 1 -- 172.21.15.72:0/2879522153 >> 172.21.15.72:0/2879522153 conn(0x7fda500fcbc0 msgr2=0x7fda500ff020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:54:37.885 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:37.884+0000 7fda57000640 1 -- 172.21.15.72:0/2879522153 shutdown_connections 2024-04-26T21:54:37.885 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:37.885+0000 7fda57000640 1 -- 172.21.15.72:0/2879522153 wait complete. 2024-04-26T21:54:37.887 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:37.885+0000 7fda57000640 1 Processor -- start 2024-04-26T21:54:37.888 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:37.886+0000 7fda57000640 1 -- start start 2024-04-26T21:54:37.888 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:37.886+0000 7fda57000640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fda501031f0 0x7fda5019aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:37.888 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:37.886+0000 7fda57000640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fda50105b20 0x7fda5019af70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:37.888 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:37.886+0000 7fda57000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fda50108450 0x7fda501a1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:37.888 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:37.886+0000 7fda57000640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fda5010e1a0 con 0x7fda50105b20 2024-04-26T21:54:37.888 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:37.886+0000 7fda57000640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fda5010e020 con 0x7fda501031f0 2024-04-26T21:54:37.888 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:37.886+0000 7fda57000640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fda5010e320 con 0x7fda50108450 2024-04-26T21:54:37.888 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:37.886+0000 7fda55200640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fda50105b20 0x7fda5019af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:37.888 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:37.886+0000 7fda55200640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fda50105b20 0x7fda5019af70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:39140/0 (socket says 172.21.15.72:39140) 2024-04-26T21:54:37.888 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:37.886+0000 7fda55200640 1 -- 172.21.15.72:0/4023616044 learned_addr learned my addr 172.21.15.72:0/4023616044 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:54:37.888 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:37.886+0000 7fda55c00640 1 --2- 172.21.15.72:0/4023616044 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fda501031f0 0x7fda5019aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:37.888 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:37.887+0000 7fda56600640 1 --2- 172.21.15.72:0/4023616044 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fda50108450 0x7fda501a1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:37.888 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:37.887+0000 7fda55200640 1 -- 172.21.15.72:0/4023616044 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fda50108450 msgr2=0x7fda501a1fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:37.888 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:37.887+0000 7fda55200640 1 --2- 172.21.15.72:0/4023616044 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fda50108450 0x7fda501a1fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:37.889 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:37.887+0000 7fda55200640 1 -- 172.21.15.72:0/4023616044 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fda501031f0 msgr2=0x7fda5019aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:37.889 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:37.887+0000 7fda55200640 1 --2- 172.21.15.72:0/4023616044 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fda501031f0 0x7fda5019aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:37.889 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:37.887+0000 7fda55200640 1 -- 172.21.15.72:0/4023616044 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda501a2650 con 0x7fda50105b20 2024-04-26T21:54:37.889 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:37.887+0000 7fda56600640 1 --2- 172.21.15.72:0/4023616044 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fda50108450 0x7fda501a1fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:54:37.889 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:37.887+0000 7fda55200640 1 --2- 172.21.15.72:0/4023616044 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fda50105b20 0x7fda5019af70 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fda3c00c970 tx=0x7fda3c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:54:37.889 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:37.887+0000 7fda4e000640 1 -- 172.21.15.72:0/4023616044 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda3c007bb0 con 0x7fda50105b20 2024-04-26T21:54:37.889 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:37.887+0000 7fda4e000640 1 -- 172.21.15.72:0/4023616044 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fda3c007d50 con 0x7fda50105b20 2024-04-26T21:54:37.889 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:37.887+0000 7fda4e000640 1 -- 172.21.15.72:0/4023616044 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda3c004e10 con 0x7fda50105b20 2024-04-26T21:54:37.889 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:37.887+0000 7fda57000640 1 -- 172.21.15.72:0/4023616044 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda501a2940 con 0x7fda50105b20 2024-04-26T21:54:37.889 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:37.887+0000 7fda57000640 1 -- 172.21.15.72:0/4023616044 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda501a2e90 con 0x7fda50105b20 2024-04-26T21:54:37.889 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:37.889+0000 7fda57000640 1 -- 172.21.15.72:0/4023616044 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda50109f40 con 0x7fda50105b20 2024-04-26T21:54:37.891 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:37.891+0000 7fda4e000640 1 -- 172.21.15.72:0/4023616044 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fda3c005000 con 0x7fda50105b20 2024-04-26T21:54:37.893 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:37.893+0000 7fda4e000640 1 --2- 172.21.15.72:0/4023616044 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fda340a3c40 0x7fda340a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:37.893 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:37.893+0000 7fda4e000640 1 -- 172.21.15.72:0/4023616044 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7fda3c0d07a0 con 0x7fda50105b20 2024-04-26T21:54:37.893 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:37.893+0000 7fda55c00640 1 --2- 172.21.15.72:0/4023616044 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fda340a3c40 0x7fda340a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:37.893 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:37.894+0000 7fda55c00640 1 --2- 172.21.15.72:0/4023616044 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fda340a3c40 0x7fda340a6100 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fda48005fd0 tx=0x7fda48009870 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:54:37.896 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:37.897+0000 7fda4e000640 1 -- 172.21.15.72:0/4023616044 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fda3c014030 con 0x7fda50105b20 2024-04-26T21:54:38.098 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.098+0000 7fda57000640 1 -- 172.21.15.72:0/4023616044 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fda5002dd40 con 0x7fda50105b20 2024-04-26T21:54:38.102 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.102+0000 7fda4e000640 1 -- 172.21.15.72:0/4023616044 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+22783 (secure 0 0 0) 0x7fda3c09cf40 con 0x7fda50105b20 2024-04-26T21:54:38.103 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T21:54:38.103 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":91,"fsid":"a1780950-0415-11ef-bc93-c7b262605968","created":"2024-04-26T21:42:01.504173+0000","modified":"2024-04-26T21:54:11.892872+0000","last_up_change":"2024-04-26T21:51:43.482437+0000","last_in_change":"2024-04-26T21:51:18.468587+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:46:29.785427+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:52:48.320391+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T21:52:49.517493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"04e06884-b355-4762-9b29-9034b7201e9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6802","nonce":1416765179},{"type":"v1","addr":"172.21.15.72:6803","nonce":1416765179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6804","nonce":1416765179},{"type":"v1","addr":"172.21.15.72:6805","nonce":1416765179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6808","nonce":1416765179},{"type":"v1","addr":"172.21.15.72:6809","nonce":1416765179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6806","nonce":1416765179},{"type":"v1","addr":"172.21.15.72:6807","nonce":1416765179}]},"public_addr":"172.21.15.72:6803/1416765179","cluster_addr":"172.21.15.72:6805/1416765179","heartbeat_back_addr":"172.21.15.72:6809/1416765179","heartbeat_front_addr":"172.21.15.72:6807/1416765179","state":["exists","up"]},{"osd":1,"uuid":"d75a479a-4a2e-48e9-a01e-2b1c9617416a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":755426949},{"type":"v1","addr":"172.21.15.76:6801","nonce":755426949}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":755426949},{"type":"v1","addr":"172.21.15.76:6803","nonce":755426949}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":755426949},{"type":"v1","addr":"172.21.15.76:6807","nonce":755426949}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":755426949},{"type":"v1","addr":"172.21.15.76:6805","nonce":755426949}]},"public_addr":"172.21.15.76:6801/755426949","cluster_addr":"172.21.15.76:6803/755426949","heartbeat_back_addr":"172.21.15.76:6807/755426949","heartbeat_front_addr":"172.21.15.76:6805/755426949","state":["exists","up"]},{"osd":2,"uuid":"b6f8da58-4428-4e8c-a944-7ea5645d6d92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":852265973},{"type":"v1","addr":"172.21.15.164:6801","nonce":852265973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":852265973},{"type":"v1","addr":"172.21.15.164:6803","nonce":852265973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":852265973},{"type":"v1","addr":"172.21.15.164:6807","nonce":852265973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":852265973},{"type":"v1","addr":"172.21.15.164:6805","nonce":852265973}]},"public_addr":"172.21.15.164:6801/852265973","cluster_addr":"172.21.15.164:6803/852265973","heartbeat_back_addr":"172.21.15.164:6807/852265973","heartbeat_front_addr":"172.21.15.164:6805/852265973","state":["exists","up"]},{"osd":3,"uuid":"76d83409-649e-41e5-b4f4-43bbd5b00ea5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6810","nonce":4027763789},{"type":"v1","addr":"172.21.15.72:6811","nonce":4027763789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6812","nonce":4027763789},{"type":"v1","addr":"172.21.15.72:6813","nonce":4027763789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6816","nonce":4027763789},{"type":"v1","addr":"172.21.15.72:6817","nonce":4027763789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6814","nonce":4027763789},{"type":"v1","addr":"172.21.15.72:6815","nonce":4027763789}]},"public_addr":"172.21.15.72:6811/4027763789","cluster_addr":"172.21.15.72:6813/4027763789","heartbeat_back_addr":"172.21.15.72:6817/4027763789","heartbeat_front_addr":"172.21.15.72:6815/4027763789","state":["exists","up"]},{"osd":4,"uuid":"d2cfeea7-1661-4156-bdcd-f770a8d1a878","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":3718867827},{"type":"v1","addr":"172.21.15.76:6809","nonce":3718867827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":3718867827},{"type":"v1","addr":"172.21.15.76:6811","nonce":3718867827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":3718867827},{"type":"v1","addr":"172.21.15.76:6815","nonce":3718867827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":3718867827},{"type":"v1","addr":"172.21.15.76:6813","nonce":3718867827}]},"public_addr":"172.21.15.76:6809/3718867827","cluster_addr":"172.21.15.76:6811/3718867827","heartbeat_back_addr":"172.21.15.76:6815/3718867827","heartbeat_front_addr":"172.21.15.76:6813/3718867827","state":["exists","up"]},{"osd":5,"uuid":"f3896ad6-a040-4ea3-a431-ff9b7fc05c36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":2543954772},{"type":"v1","addr":"172.21.15.164:6809","nonce":2543954772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":2543954772},{"type":"v1","addr":"172.21.15.164:6811","nonce":2543954772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":2543954772},{"type":"v1","addr":"172.21.15.164:6815","nonce":2543954772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":2543954772},{"type":"v1","addr":"172.21.15.164:6813","nonce":2543954772}]},"public_addr":"172.21.15.164:6809/2543954772","cluster_addr":"172.21.15.164:6811/2543954772","heartbeat_back_addr":"172.21.15.164:6815/2543954772","heartbeat_front_addr":"172.21.15.164:6813/2543954772","state":["exists","up"]},{"osd":6,"uuid":"d5d56650-be7e-4ba5-ba3a-d7d473294dab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6818","nonce":2080659027},{"type":"v1","addr":"172.21.15.72:6819","nonce":2080659027}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6820","nonce":2080659027},{"type":"v1","addr":"172.21.15.72:6821","nonce":2080659027}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6824","nonce":2080659027},{"type":"v1","addr":"172.21.15.72:6825","nonce":2080659027}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6822","nonce":2080659027},{"type":"v1","addr":"172.21.15.72:6823","nonce":2080659027}]},"public_addr":"172.21.15.72:6819/2080659027","cluster_addr":"172.21.15.72:6821/2080659027","heartbeat_back_addr":"172.21.15.72:6825/2080659027","heartbeat_front_addr":"172.21.15.72:6823/2080659027","state":["exists","up"]},{"osd":7,"uuid":"18975b5a-9309-4a0f-9f37-3ce09c2e745d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":3144631323},{"type":"v1","addr":"172.21.15.76:6817","nonce":3144631323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":3144631323},{"type":"v1","addr":"172.21.15.76:6819","nonce":3144631323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":3144631323},{"type":"v1","addr":"172.21.15.76:6823","nonce":3144631323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":3144631323},{"type":"v1","addr":"172.21.15.76:6821","nonce":3144631323}]},"public_addr":"172.21.15.76:6817/3144631323","cluster_addr":"172.21.15.76:6819/3144631323","heartbeat_back_addr":"172.21.15.76:6823/3144631323","heartbeat_front_addr":"172.21.15.76:6821/3144631323","state":["exists","up"]},{"osd":8,"uuid":"779e9c1e-8e97-48d4-a854-68ca13f38e0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":627888883},{"type":"v1","addr":"172.21.15.164:6817","nonce":627888883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":627888883},{"type":"v1","addr":"172.21.15.164:6819","nonce":627888883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":627888883},{"type":"v1","addr":"172.21.15.164:6823","nonce":627888883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":627888883},{"type":"v1","addr":"172.21.15.164:6821","nonce":627888883}]},"public_addr":"172.21.15.164:6817/627888883","cluster_addr":"172.21.15.164:6819/627888883","heartbeat_back_addr":"172.21.15.164:6823/627888883","heartbeat_front_addr":"172.21.15.164:6821/627888883","state":["exists","up"]},{"osd":9,"uuid":"d0317850-7b2d-4850-b2cf-0b919505c1fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6826","nonce":1273118485},{"type":"v1","addr":"172.21.15.72:6827","nonce":1273118485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6828","nonce":1273118485},{"type":"v1","addr":"172.21.15.72:6829","nonce":1273118485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6832","nonce":1273118485},{"type":"v1","addr":"172.21.15.72:6833","nonce":1273118485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6830","nonce":1273118485},{"type":"v1","addr":"172.21.15.72:6831","nonce":1273118485}]},"public_addr":"172.21.15.72:6827/1273118485","cluster_addr":"172.21.15.72:6829/1273118485","heartbeat_back_addr":"172.21.15.72:6833/1273118485","heartbeat_front_addr":"172.21.15.72:6831/1273118485","state":["exists","up"]},{"osd":10,"uuid":"e0aa38d8-6011-4d2a-b67b-5bff6d999e47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":2808954459},{"type":"v1","addr":"172.21.15.76:6825","nonce":2808954459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":2808954459},{"type":"v1","addr":"172.21.15.76:6827","nonce":2808954459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":2808954459},{"type":"v1","addr":"172.21.15.76:6831","nonce":2808954459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":2808954459},{"type":"v1","addr":"172.21.15.76:6829","nonce":2808954459}]},"public_addr":"172.21.15.76:6825/2808954459","cluster_addr":"172.21.15.76:6827/2808954459","heartbeat_back_addr":"172.21.15.76:6831/2808954459","heartbeat_front_addr":"172.21.15.76:6829/2808954459","state":["exists","up"]},{"osd":11,"uuid":"981b3d4d-916e-4937-9916-7cc21a891e43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":2354951784},{"type":"v1","addr":"172.21.15.164:6825","nonce":2354951784}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":2354951784},{"type":"v1","addr":"172.21.15.164:6827","nonce":2354951784}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":2354951784},{"type":"v1","addr":"172.21.15.164:6831","nonce":2354951784}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":2354951784},{"type":"v1","addr":"172.21.15.164:6829","nonce":2354951784}]},"public_addr":"172.21.15.164:6825/2354951784","cluster_addr":"172.21.15.164:6827/2354951784","heartbeat_back_addr":"172.21.15.164:6831/2354951784","heartbeat_front_addr":"172.21.15.164:6829/2354951784","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:18.997924+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:52.439081+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:46:25.893288+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:47:00.112807+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:47:35.116874+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:48:09.216344+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:48:45.028853+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:49:19.905397+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:49:55.030536+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:50:30.767662+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:51:06.423267+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:51:41.631211+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.1a","mappings":[{"from":5,"to":7}]},{"pgid":"3.1b","mappings":[{"from":4,"to":6}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.28","mappings":[{"from":5,"to":6}]},{"pgid":"3.2a","mappings":[{"from":3,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.72:0/416277784":"2024-04-27T21:43:09.602791+0000","172.21.15.72:6800/4010358672":"2024-04-27T21:43:09.602791+0000","172.21.15.72:6801/1012617371":"2024-04-27T21:42:34.684837+0000","172.21.15.72:6801/4010358672":"2024-04-27T21:43:09.602791+0000","172.21.15.72:0/3555448179":"2024-04-27T21:43:09.602791+0000","172.21.15.72:6800/1012617371":"2024-04-27T21:42:34.684837+0000","172.21.15.72:0/2920378761":"2024-04-27T21:42:34.684837+0000","172.21.15.72:0/1933740041":"2024-04-27T21:43:09.602791+0000","172.21.15.72:0/3225295775":"2024-04-27T21:42:34.684837+0000","172.21.15.72:0/2326016180":"2024-04-27T21:42:34.684837+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:54:38.105 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.105+0000 7fda57000640 1 -- 172.21.15.72:0/4023616044 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fda340a3c40 msgr2=0x7fda340a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:38.105 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.105+0000 7fda57000640 1 --2- 172.21.15.72:0/4023616044 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fda340a3c40 0x7fda340a6100 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fda48005fd0 tx=0x7fda48009870 comp rx=0 tx=0).stop 2024-04-26T21:54:38.105 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.105+0000 7fda57000640 1 -- 172.21.15.72:0/4023616044 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fda50105b20 msgr2=0x7fda5019af70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:38.105 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.105+0000 7fda57000640 1 --2- 172.21.15.72:0/4023616044 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fda50105b20 0x7fda5019af70 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fda3c00c970 tx=0x7fda3c00ce30 comp rx=0 tx=0).stop 2024-04-26T21:54:38.105 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.105+0000 7fda57000640 1 -- 172.21.15.72:0/4023616044 shutdown_connections 2024-04-26T21:54:38.105 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.105+0000 7fda57000640 1 --2- 172.21.15.72:0/4023616044 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fda340a3c40 0x7fda340a6100 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:38.105 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.105+0000 7fda57000640 1 --2- 172.21.15.72:0/4023616044 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fda50108450 0x7fda501a1fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:38.105 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.105+0000 7fda57000640 1 --2- 172.21.15.72:0/4023616044 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fda50105b20 0x7fda5019af70 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:38.105 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.105+0000 7fda57000640 1 --2- 172.21.15.72:0/4023616044 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fda501031f0 0x7fda5019aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:38.105 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.105+0000 7fda57000640 1 -- 172.21.15.72:0/4023616044 >> 172.21.15.72:0/4023616044 conn(0x7fda500fcbc0 msgr2=0x7fda501028e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:54:38.106 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.105+0000 7fda57000640 1 -- 172.21.15.72:0/4023616044 shutdown_connections 2024-04-26T21:54:38.106 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.105+0000 7fda57000640 1 -- 172.21.15.72:0/4023616044 wait complete. 2024-04-26T21:54:38.124 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:54:38.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:54:38 smithi076 ceph-mon[31823]: from='client.? 172.21.15.72:0/1274583716' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T21:54:38.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:54:38 smithi076 ceph-mon[31823]: from='client.? 172.21.15.72:0/234151850' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:54:38.321 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:54:38 smithi072 ceph-mon[28138]: from='client.? 172.21.15.72:0/1274583716' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T21:54:38.321 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:54:38 smithi072 ceph-mon[28138]: from='client.? 172.21.15.72:0/234151850' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:54:38.328 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.328+0000 7fc7c2000640 1 -- 172.21.15.72:0/1571128407 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc7bc078530 msgr2=0x7fc7bc078990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:38.328 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.328+0000 7fc7c2000640 1 --2- 172.21.15.72:0/1571128407 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc7bc078530 0x7fc7bc078990 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fc7a4009900 tx=0x7fc7a402a410 comp rx=0 tx=0).stop 2024-04-26T21:54:38.328 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.328+0000 7fc7c2000640 1 -- 172.21.15.72:0/1571128407 shutdown_connections 2024-04-26T21:54:38.328 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.328+0000 7fc7c2000640 1 --2- 172.21.15.72:0/1571128407 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc7bc078ed0 0x7fc7bc079770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:38.328 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.328+0000 7fc7c2000640 1 --2- 172.21.15.72:0/1571128407 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc7bc078530 0x7fc7bc078990 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:38.328 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.328+0000 7fc7c2000640 1 --2- 172.21.15.72:0/1571128407 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc7bc0772f0 0x7fc7bc077710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:38.329 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.328+0000 7fc7c2000640 1 -- 172.21.15.72:0/1571128407 >> 172.21.15.72:0/1571128407 conn(0x7fc7bc0ffeb0 msgr2=0x7fc7bc102310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:54:38.329 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.328+0000 7fc7c2000640 1 -- 172.21.15.72:0/1571128407 shutdown_connections 2024-04-26T21:54:38.329 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.328+0000 7fc7c2000640 1 -- 172.21.15.72:0/1571128407 wait complete. 2024-04-26T21:54:38.331 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.329+0000 7fc7c2000640 1 Processor -- start 2024-04-26T21:54:38.331 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.329+0000 7fc7c2000640 1 -- start start 2024-04-26T21:54:38.331 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.329+0000 7fc7c2000640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc7bc0772f0 0x7fc7bc19eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:38.331 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.329+0000 7fc7c2000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc7bc078530 0x7fc7bc19f430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:38.331 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.329+0000 7fc7c2000640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc7bc078ed0 0x7fc7bc1a6450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:38.331 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.329+0000 7fc7c2000640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fc7bc114750 con 0x7fc7bc0772f0 2024-04-26T21:54:38.331 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.329+0000 7fc7c2000640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fc7bc1145d0 con 0x7fc7bc078ed0 2024-04-26T21:54:38.331 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.330+0000 7fc7c2000640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fc7bc1148d0 con 0x7fc7bc078530 2024-04-26T21:54:38.331 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.330+0000 7fc7c0c00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc7bc0772f0 0x7fc7bc19eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:38.332 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.330+0000 7fc7c0c00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc7bc0772f0 0x7fc7bc19eef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:39160/0 (socket says 172.21.15.72:39160) 2024-04-26T21:54:38.332 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.330+0000 7fc7c0c00640 1 -- 172.21.15.72:0/2906267032 learned_addr learned my addr 172.21.15.72:0/2906267032 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:54:38.332 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.330+0000 7fc7bbe00640 1 --2- 172.21.15.72:0/2906267032 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc7bc078530 0x7fc7bc19f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:38.332 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.330+0000 7fc7c1600640 1 --2- 172.21.15.72:0/2906267032 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc7bc078ed0 0x7fc7bc1a6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:38.332 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.330+0000 7fc7c0c00640 1 -- 172.21.15.72:0/2906267032 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc7bc078530 msgr2=0x7fc7bc19f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:38.332 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.330+0000 7fc7c0c00640 1 --2- 172.21.15.72:0/2906267032 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc7bc078530 0x7fc7bc19f430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:38.332 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.330+0000 7fc7c0c00640 1 -- 172.21.15.72:0/2906267032 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc7bc078ed0 msgr2=0x7fc7bc1a6450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:38.332 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.330+0000 7fc7c0c00640 1 --2- 172.21.15.72:0/2906267032 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc7bc078ed0 0x7fc7bc1a6450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:38.332 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.330+0000 7fc7c0c00640 1 -- 172.21.15.72:0/2906267032 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7bc1a6ac0 con 0x7fc7bc0772f0 2024-04-26T21:54:38.332 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.330+0000 7fc7c1600640 1 --2- 172.21.15.72:0/2906267032 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc7bc078ed0 0x7fc7bc1a6450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:54:38.332 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.330+0000 7fc7bbe00640 1 --2- 172.21.15.72:0/2906267032 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc7bc078530 0x7fc7bc19f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:54:38.332 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.331+0000 7fc7c0c00640 1 --2- 172.21.15.72:0/2906267032 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc7bc0772f0 0x7fc7bc19eef0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fc7b400eab0 tx=0x7fc7b400ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:54:38.332 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.331+0000 7fc7b9600640 1 -- 172.21.15.72:0/2906267032 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7b400ce40 con 0x7fc7bc0772f0 2024-04-26T21:54:38.332 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.331+0000 7fc7b9600640 1 -- 172.21.15.72:0/2906267032 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc7b4011900 con 0x7fc7bc0772f0 2024-04-26T21:54:38.332 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.331+0000 7fc7b9600640 1 -- 172.21.15.72:0/2906267032 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7b4010990 con 0x7fc7bc0772f0 2024-04-26T21:54:38.333 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.331+0000 7fc7c2000640 1 -- 172.21.15.72:0/2906267032 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7bc1a6db0 con 0x7fc7bc0772f0 2024-04-26T21:54:38.333 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.331+0000 7fc7c2000640 1 -- 172.21.15.72:0/2906267032 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7bc1a7270 con 0x7fc7bc0772f0 2024-04-26T21:54:38.333 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.332+0000 7fc7c2000640 1 -- 172.21.15.72:0/2906267032 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7bc1112d0 con 0x7fc7bc0772f0 2024-04-26T21:54:38.334 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.334+0000 7fc7b9600640 1 -- 172.21.15.72:0/2906267032 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fc7b4011470 con 0x7fc7bc0772f0 2024-04-26T21:54:38.335 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.335+0000 7fc7b9600640 1 --2- 172.21.15.72:0/2906267032 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fc7a00a3cc0 0x7fc7a00a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:38.336 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.336+0000 7fc7b9600640 1 -- 172.21.15.72:0/2906267032 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7fc7b40d0cc0 con 0x7fc7bc0772f0 2024-04-26T21:54:38.336 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.336+0000 7fc7bbe00640 1 --2- 172.21.15.72:0/2906267032 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fc7a00a3cc0 0x7fc7a00a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:38.336 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.336+0000 7fc7bbe00640 1 --2- 172.21.15.72:0/2906267032 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fc7a00a3cc0 0x7fc7a00a6180 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fc7a402a930 tx=0x7fc7a4000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:54:38.339 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.340+0000 7fc7b9600640 1 -- 172.21.15.72:0/2906267032 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fc7b40d7080 con 0x7fc7bc0772f0 2024-04-26T21:54:38.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:54:38 smithi164 ceph-mon[31916]: from='client.? 172.21.15.72:0/1274583716' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T21:54:38.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:54:38 smithi164 ceph-mon[31916]: from='client.? 172.21.15.72:0/234151850' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:54:38.583 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.582+0000 7fc7c2000640 1 -- 172.21.15.72:0/2906267032 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc7bc02dad0 con 0x7fc7bc0772f0 2024-04-26T21:54:38.584 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.584+0000 7fc7b9600640 1 -- 172.21.15.72:0/2906267032 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11931 (secure 0 0 0) 0x7fc7b4016030 con 0x7fc7bc0772f0 2024-04-26T21:54:38.584 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T21:54:38.585 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6835/1320381959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6834","nonce":1320381959},{"type":"v1","addr":"172.21.15.72:6835","nonce":1320381959}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6837/2142200512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6836","nonce":2142200512},{"type":"v1","addr":"172.21.15.72:6837","nonce":2142200512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T21:52:50.567857+0000","modified":"2024-04-26T21:52:58.508126+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34517,"mds_1":34472,"mds_2":34442,"mds_3":34406,"mds_4":24463},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14652":{"gid":14652,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.72:6839/1312898281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6838","nonce":1312898281},{"type":"v1","addr":"172.21.15.72:6839","nonce":1312898281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14664":{"gid":14664,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.72:6841/3257692339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6840","nonce":3257692339},{"type":"v1","addr":"172.21.15.72:6841","nonce":3257692339}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24388":{"gid":24388,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6833/4236668465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":4236668465},{"type":"v1","addr":"172.21.15.164:6833","nonce":4236668465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6835/2391114169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":2391114169},{"type":"v1","addr":"172.21.15.164:6835","nonce":2391114169}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6837/3585829992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":3585829992},{"type":"v1","addr":"172.21.15.164:6837","nonce":3585829992}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.164:6839/1636226820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":1636226820},{"type":"v1","addr":"172.21.15.164:6839","nonce":1636226820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34406":{"gid":34406,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":15,"addr":"172.21.15.76:6833/3869483750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":3869483750},{"type":"v1","addr":"172.21.15.76:6833","nonce":3869483750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34442":{"gid":34442,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.76:6835/3517429435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":3517429435},{"type":"v1","addr":"172.21.15.76:6835","nonce":3517429435}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34472":{"gid":34472,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.76:6837/1150678991","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":1150678991},{"type":"v1","addr":"172.21.15.76:6837","nonce":1150678991}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34517":{"gid":34517,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.76:6839/327145636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6838","nonce":327145636},{"type":"v1","addr":"172.21.15.76:6839","nonce":327145636}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34517,"qdb_cluster":[34406,24463,34442,34517,34472]},"id":1}]} 2024-04-26T21:54:38.585 INFO:teuthology.orchestra.run.smithi072.stderr:dumped fsmap epoch 26 2024-04-26T21:54:38.587 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.587+0000 7fc7c2000640 1 -- 172.21.15.72:0/2906267032 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fc7a00a3cc0 msgr2=0x7fc7a00a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:38.587 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.587+0000 7fc7c2000640 1 --2- 172.21.15.72:0/2906267032 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fc7a00a3cc0 0x7fc7a00a6180 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fc7a402a930 tx=0x7fc7a4000bc0 comp rx=0 tx=0).stop 2024-04-26T21:54:38.587 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.587+0000 7fc7c2000640 1 -- 172.21.15.72:0/2906267032 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc7bc0772f0 msgr2=0x7fc7bc19eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:38.587 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.587+0000 7fc7c2000640 1 --2- 172.21.15.72:0/2906267032 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc7bc0772f0 0x7fc7bc19eef0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fc7b400eab0 tx=0x7fc7b400ef70 comp rx=0 tx=0).stop 2024-04-26T21:54:38.587 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.587+0000 7fc7c2000640 1 -- 172.21.15.72:0/2906267032 shutdown_connections 2024-04-26T21:54:38.587 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.587+0000 7fc7c2000640 1 --2- 172.21.15.72:0/2906267032 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc7bc078ed0 0x7fc7bc1a6450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:38.587 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.587+0000 7fc7c2000640 1 --2- 172.21.15.72:0/2906267032 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fc7a00a3cc0 0x7fc7a00a6180 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:38.588 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.587+0000 7fc7c2000640 1 --2- 172.21.15.72:0/2906267032 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc7bc078530 0x7fc7bc19f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:38.588 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.587+0000 7fc7c2000640 1 --2- 172.21.15.72:0/2906267032 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc7bc0772f0 0x7fc7bc19eef0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:38.588 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.587+0000 7fc7c2000640 1 -- 172.21.15.72:0/2906267032 >> 172.21.15.72:0/2906267032 conn(0x7fc7bc0ffeb0 msgr2=0x7fc7bc1019b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:54:38.588 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.587+0000 7fc7c2000640 1 -- 172.21.15.72:0/2906267032 shutdown_connections 2024-04-26T21:54:38.588 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.587+0000 7fc7c2000640 1 -- 172.21.15.72:0/2906267032 wait complete. 2024-04-26T21:54:38.607 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 26, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-26T21:52:50.567857+0000', 'modified': '2024-04-26T21:52:58.508126+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 34517, 'mds_1': 34472, 'mds_2': 34442, 'mds_3': 34406, 'mds_4': 24463}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14652': {'gid': 14652, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.72:6839/1312898281', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.72:6838', 'nonce': 1312898281}, {'type': 'v1', 'addr': '172.21.15.72:6839', 'nonce': 1312898281}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14664': {'gid': 14664, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.72:6841/3257692339', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.72:6840', 'nonce': 3257692339}, {'type': 'v1', 'addr': '172.21.15.72:6841', 'nonce': 3257692339}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24388': {'gid': 24388, 'name': 'l', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.164:6833/4236668465', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6832', 'nonce': 4236668465}, {'type': 'v1', 'addr': '172.21.15.164:6833', 'nonce': 4236668465}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24424': {'gid': 24424, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.164:6835/2391114169', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6834', 'nonce': 2391114169}, {'type': 'v1', 'addr': '172.21.15.164:6835', 'nonce': 2391114169}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24442': {'gid': 24442, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.164:6837/3585829992', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6836', 'nonce': 3585829992}, {'type': 'v1', 'addr': '172.21.15.164:6837', 'nonce': 3585829992}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24463': {'gid': 24463, 'name': 'i', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.164:6839/1636226820', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6838', 'nonce': 1636226820}, {'type': 'v1', 'addr': '172.21.15.164:6839', 'nonce': 1636226820}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_34406': {'gid': 34406, 'name': 'k', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 15, 'addr': '172.21.15.76:6833/3869483750', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6832', 'nonce': 3869483750}, {'type': 'v1', 'addr': '172.21.15.76:6833', 'nonce': 3869483750}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_34442': {'gid': 34442, 'name': 'e', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.76:6835/3517429435', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6834', 'nonce': 3517429435}, {'type': 'v1', 'addr': '172.21.15.76:6835', 'nonce': 3517429435}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_34472': {'gid': 34472, 'name': 'h', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.76:6837/1150678991', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6836', 'nonce': 1150678991}, {'type': 'v1', 'addr': '172.21.15.76:6837', 'nonce': 1150678991}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_34517': {'gid': 34517, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.76:6839/327145636', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6838', 'nonce': 327145636}, {'type': 'v1', 'addr': '172.21.15.76:6839', 'nonce': 327145636}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 34517, 'qdb_cluster': [34406, 24463, 34442, 34517, 34472]} 2024-04-26T21:54:38.607 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-04-26T21:54:38.608 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-04-26T21:54:38.795 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.795+0000 7fa145a00640 1 -- 172.21.15.72:0/3758302425 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa140068de0 msgr2=0x7fa140101b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:38.795 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.795+0000 7fa145a00640 1 --2- 172.21.15.72:0/3758302425 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa140068de0 0x7fa140101b90 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fa130009900 tx=0x7fa13002a3f0 comp rx=0 tx=0).stop 2024-04-26T21:54:38.795 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.795+0000 7fa145a00640 1 -- 172.21.15.72:0/3758302425 shutdown_connections 2024-04-26T21:54:38.796 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.795+0000 7fa145a00640 1 --2- 172.21.15.72:0/3758302425 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa140106dd0 0x7fa1401091c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:38.796 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.795+0000 7fa145a00640 1 --2- 172.21.15.72:0/3758302425 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa1401020d0 0x7fa140102550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:38.796 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.795+0000 7fa145a00640 1 --2- 172.21.15.72:0/3758302425 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa140068de0 0x7fa140101b90 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:38.796 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.795+0000 7fa145a00640 1 -- 172.21.15.72:0/3758302425 >> 172.21.15.72:0/3758302425 conn(0x7fa1400fbf10 msgr2=0x7fa1400fe350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:54:38.796 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.795+0000 7fa145a00640 1 -- 172.21.15.72:0/3758302425 shutdown_connections 2024-04-26T21:54:38.796 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.795+0000 7fa145a00640 1 -- 172.21.15.72:0/3758302425 wait complete. 2024-04-26T21:54:38.798 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.796+0000 7fa145a00640 1 Processor -- start 2024-04-26T21:54:38.798 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.796+0000 7fa145a00640 1 -- start start 2024-04-26T21:54:38.798 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.796+0000 7fa145a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa140068de0 0x7fa14010c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:38.798 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.796+0000 7fa145a00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa1401020d0 0x7fa14010ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:38.798 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.796+0000 7fa145a00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa140106dd0 0x7fa14010b090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:38.799 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.796+0000 7fa145a00640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fa14010e1e0 con 0x7fa1401020d0 2024-04-26T21:54:38.799 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.796+0000 7fa145a00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fa14010e060 con 0x7fa140106dd0 2024-04-26T21:54:38.799 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.796+0000 7fa145a00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa14010e360 con 0x7fa140068de0 2024-04-26T21:54:38.799 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.797+0000 7fa137400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa1401020d0 0x7fa14010ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:38.799 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.797+0000 7fa137400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa1401020d0 0x7fa14010ab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:39190/0 (socket says 172.21.15.72:39190) 2024-04-26T21:54:38.799 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.797+0000 7fa137400640 1 -- 172.21.15.72:0/3711720613 learned_addr learned my addr 172.21.15.72:0/3711720613 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:54:38.799 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.797+0000 7fa13fe00640 1 --2- 172.21.15.72:0/3711720613 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa140068de0 0x7fa14010c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:38.799 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.797+0000 7fa145000640 1 --2- 172.21.15.72:0/3711720613 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa140106dd0 0x7fa14010b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:38.799 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.797+0000 7fa137400640 1 -- 172.21.15.72:0/3711720613 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa140068de0 msgr2=0x7fa14010c510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:38.799 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.797+0000 7fa137400640 1 --2- 172.21.15.72:0/3711720613 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa140068de0 0x7fa14010c510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:38.799 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.797+0000 7fa137400640 1 -- 172.21.15.72:0/3711720613 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa140106dd0 msgr2=0x7fa14010b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:38.799 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.797+0000 7fa137400640 1 --2- 172.21.15.72:0/3711720613 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa140106dd0 0x7fa14010b090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:38.799 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.797+0000 7fa137400640 1 -- 172.21.15.72:0/3711720613 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa14010b920 con 0x7fa1401020d0 2024-04-26T21:54:38.799 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.797+0000 7fa137400640 1 --2- 172.21.15.72:0/3711720613 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa1401020d0 0x7fa14010ab50 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fa12800c970 tx=0x7fa12800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:54:38.799 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.797+0000 7fa13d600640 1 -- 172.21.15.72:0/3711720613 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa128007c10 con 0x7fa1401020d0 2024-04-26T21:54:38.800 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.798+0000 7fa13d600640 1 -- 172.21.15.72:0/3711720613 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa128007db0 con 0x7fa1401020d0 2024-04-26T21:54:38.800 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.798+0000 7fa145a00640 1 -- 172.21.15.72:0/3711720613 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1401a2960 con 0x7fa1401020d0 2024-04-26T21:54:38.800 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.798+0000 7fa13d600640 1 -- 172.21.15.72:0/3711720613 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa12800f7d0 con 0x7fa1401020d0 2024-04-26T21:54:38.800 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.798+0000 7fa145a00640 1 -- 172.21.15.72:0/3711720613 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1401a2f00 con 0x7fa1401020d0 2024-04-26T21:54:38.801 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.801+0000 7fa149e35740 1 Processor -- start 2024-04-26T21:54:38.801 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.801+0000 7fa13d600640 1 -- 172.21.15.72:0/3711720613 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fa128018480 con 0x7fa1401020d0 2024-04-26T21:54:38.801 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.801+0000 7fa149e35740 1 -- start start 2024-04-26T21:54:38.801 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.801+0000 7fa149e35740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5565bed702c0 0x5565bed706e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:38.801 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.801+0000 7fa149e35740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5565bed70cb0 0x5565bed757b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:38.802 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.801+0000 7fa149e35740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5565bed75e20 0x5565bed782b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:38.802 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.801+0000 7fa149e35740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x5565becd4600 con 0x5565bed70cb0 2024-04-26T21:54:38.802 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.801+0000 7fa149e35740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5565bec71100 con 0x5565bed702c0 2024-04-26T21:54:38.802 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.801+0000 7fa149e35740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5565becd05c0 con 0x5565bed75e20 2024-04-26T21:54:38.802 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.801+0000 7fa13fe00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5565bed70cb0 0x5565bed757b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:38.802 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.801+0000 7fa13fe00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5565bed70cb0 0x5565bed757b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:39196/0 (socket says 172.21.15.72:39196) 2024-04-26T21:54:38.802 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.801+0000 7fa13fe00640 1 -- 172.21.15.72:0/2089882709 learned_addr learned my addr 172.21.15.72:0/2089882709 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:54:38.807 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.802+0000 7fa13fe00640 1 -- 172.21.15.72:0/2089882709 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5565bed75e20 msgr2=0x5565bed782b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:54:38.807 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.802+0000 7fa137400640 1 --2- 172.21.15.72:0/2089882709 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5565bed75e20 0x5565bed782b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:38.807 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.802+0000 7fa145000640 1 --2- 172.21.15.72:0/2089882709 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5565bed702c0 0x5565bed706e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:38.807 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.802+0000 7fa13fe00640 1 --2- 172.21.15.72:0/2089882709 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5565bed75e20 0x5565bed782b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:38.807 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.802+0000 7fa13fe00640 1 -- 172.21.15.72:0/2089882709 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5565bed702c0 msgr2=0x5565bed706e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:54:38.807 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.802+0000 7fa13fe00640 1 --2- 172.21.15.72:0/2089882709 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5565bed702c0 0x5565bed706e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:38.808 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.802+0000 7fa13fe00640 1 -- 172.21.15.72:0/2089882709 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5565be9e62d0 con 0x5565bed70cb0 2024-04-26T21:54:38.808 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.802+0000 7fa13fe00640 1 --2- 172.21.15.72:0/2089882709 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5565bed70cb0 0x5565bed757b0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fa130005ab0 tx=0x7fa130004200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e006e40ec8b50f9f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:54:38.808 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.802+0000 7fa135600640 1 -- 172.21.15.72:0/2089882709 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa130035030 con 0x5565bed70cb0 2024-04-26T21:54:38.808 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.802+0000 7fa135600640 1 -- 172.21.15.72:0/2089882709 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa130003a80 con 0x5565bed70cb0 2024-04-26T21:54:38.808 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.802+0000 7fa13d600640 1 --2- 172.21.15.72:0/3711720613 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fa1240a3cc0 0x7fa1240a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:38.808 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.802+0000 7fa135600640 1 -- 172.21.15.72:0/2089882709 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa130031ad0 con 0x5565bed70cb0 2024-04-26T21:54:38.808 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.802+0000 7fa13d600640 1 -- 172.21.15.72:0/3711720613 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7fa1280d0a40 con 0x7fa1401020d0 2024-04-26T21:54:38.808 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.803+0000 7fa145000640 1 --2- 172.21.15.72:0/3711720613 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fa1240a3cc0 0x7fa1240a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:38.808 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.803+0000 7fa149e35740 1 -- 172.21.15.72:0/2089882709 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5565bed70cb0 msgr2=0x5565bed757b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:38.808 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.803+0000 7fa149e35740 1 --2- 172.21.15.72:0/2089882709 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5565bed70cb0 0x5565bed757b0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fa130005ab0 tx=0x7fa130004200 comp rx=0 tx=0).stop 2024-04-26T21:54:38.808 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.803+0000 7fa149e35740 1 -- 172.21.15.72:0/2089882709 shutdown_connections 2024-04-26T21:54:38.808 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.803+0000 7fa149e35740 1 --2- 172.21.15.72:0/2089882709 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5565bed75e20 0x5565bed782b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:38.809 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.803+0000 7fa149e35740 1 --2- 172.21.15.72:0/2089882709 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5565bed70cb0 0x5565bed757b0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:38.809 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.803+0000 7fa149e35740 1 --2- 172.21.15.72:0/2089882709 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5565bed702c0 0x5565bed706e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:38.809 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.803+0000 7fa149e35740 1 -- 172.21.15.72:0/2089882709 >> 172.21.15.72:0/2089882709 conn(0x5565bed04d10 msgr2=0x5565bed6bd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:54:38.809 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.803+0000 7fa149e35740 1 -- 172.21.15.72:0/2089882709 shutdown_connections 2024-04-26T21:54:38.809 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.803+0000 7fa145000640 1 --2- 172.21.15.72:0/3711720613 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fa1240a3cc0 0x7fa1240a6180 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x5565bed71510 tx=0x7fa138009710 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:54:38.809 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.803+0000 7fa149e35740 1 -- 172.21.15.72:0/2089882709 wait complete. 2024-04-26T21:54:38.809 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.804+0000 7fa149e35740 1 -- start start 2024-04-26T21:54:38.809 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.804+0000 7fa149e35740 1 Processor -- start 2024-04-26T21:54:38.809 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.804+0000 7fa149e35740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5565bed702c0 0x5565bed7a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:38.809 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.804+0000 7fa149e35740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5565bed70cb0 0x5565bed7abf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:38.809 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.804+0000 7fa149e35740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5565bed75e20 0x5565bed81c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:38.809 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.804+0000 7fa149e35740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x5565bed78d70 con 0x5565bed702c0 2024-04-26T21:54:38.809 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.804+0000 7fa149e35740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5565bec5bf70 con 0x5565bed75e20 2024-04-26T21:54:38.810 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.804+0000 7fa149e35740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5565bed78ef0 con 0x5565bed70cb0 2024-04-26T21:54:38.810 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.804+0000 7fa149e35740 10 client.0 _ll_register_callbacks cb 0x5565becca480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:54:38.810 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.805+0000 7fa13fe00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5565bed702c0 0x5565bed7a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:38.810 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.805+0000 7fa13fe00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5565bed702c0 0x5565bed7a6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:39210/0 (socket says 172.21.15.72:39210) 2024-04-26T21:54:38.810 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.805+0000 7fa13fe00640 1 -- 172.21.15.72:0/2994744577 learned_addr learned my addr 172.21.15.72:0/2994744577 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:54:38.810 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.805+0000 7fa137400640 1 --2- 172.21.15.72:0/2994744577 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5565bed70cb0 0x5565bed7abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:38.810 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.805+0000 7fa13fe00640 1 -- 172.21.15.72:0/2994744577 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5565bed70cb0 msgr2=0x5565bed7abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:54:38.810 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.805+0000 7fa13fe00640 1 --2- 172.21.15.72:0/2994744577 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5565bed70cb0 0x5565bed7abf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:38.810 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.805+0000 7fa13fe00640 1 -- 172.21.15.72:0/2994744577 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5565bed75e20 msgr2=0x5565bed81c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:54:38.810 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.805+0000 7fa145000640 1 --2- 172.21.15.72:0/2994744577 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5565bed75e20 0x5565bed81c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:38.810 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.805+0000 7fa13fe00640 1 --2- 172.21.15.72:0/2994744577 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5565bed75e20 0x5565bed81c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:38.810 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.805+0000 7fa13fe00640 1 -- 172.21.15.72:0/2994744577 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5565beab06b0 con 0x5565bed702c0 2024-04-26T21:54:38.810 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.805+0000 7fa13fe00640 1 --2- 172.21.15.72:0/2994744577 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5565bed702c0 0x5565bed7a6b0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fa13002a9e0 tx=0x7fa130003e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ad47ee4fe5d40f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:54:38.810 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.805+0000 7fa12e000640 10 client.0 ms_handle_connect on v2:172.21.15.72:3300/0 2024-04-26T21:54:38.811 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.806+0000 7fa12e000640 1 -- 172.21.15.72:0/2994744577 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa130035030 con 0x5565bed702c0 2024-04-26T21:54:38.811 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.806+0000 7fa12e000640 1 -- 172.21.15.72:0/2994744577 <== mon.0 v2:172.21.15.72:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7fa1300318b0 con 0x5565bed702c0 2024-04-26T21:54:38.811 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.806+0000 7fa149e35740 1 -- 172.21.15.72:0/2994744577 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa130003a80 con 0x5565bed702c0 2024-04-26T21:54:38.811 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.806+0000 7fa12e000640 1 -- 172.21.15.72:0/2994744577 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fa130053030 con 0x5565bed702c0 2024-04-26T21:54:38.811 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.806+0000 7fa149e35740 10 client.15090 fetch_fsmap learned FSMap version 26 2024-04-26T21:54:38.811 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.806+0000 7fa149e35740 1 -- 172.21.15.72:0/2994744577 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5565bed84c10 con 0x5565bed702c0 2024-04-26T21:54:38.811 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.806+0000 7fa12e000640 1 -- 172.21.15.72:0/2994744577 <== mon.0 v2:172.21.15.72:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa130003a80 con 0x5565bed702c0 2024-04-26T21:54:38.811 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.806+0000 7fa12e000640 1 -- 172.21.15.72:0/2994744577 <== mon.0 v2:172.21.15.72:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fa13003c9b0 con 0x5565bed702c0 2024-04-26T21:54:38.811 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.806+0000 7fa149e35740 10 client.15090 fetch_fsmap finished waiting for FSMap version 26 2024-04-26T21:54:38.811 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.806+0000 7fa149e35740 10 client.15090 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-26T21:54:38.811 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.806+0000 7fa149e35740 20 client.15090 populate_metadata read hostname 'smithi072' 2024-04-26T21:54:38.811 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.806+0000 7fa149e35740 1 --2- 172.21.15.72:0/2994744577 >> [v2:172.21.15.164:6838/1636226820,v1:172.21.15.164:6839/1636226820] conn(0x5565bed86ec0 0x5565bed88330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:38.811 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.807+0000 7fa149e35740 4 client.15090 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:54:38.811 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.807+0000 7fa149e35740 1 -- 172.21.15.72:0/2994744577 --> [v2:172.21.15.164:6838/1636226820,v1:172.21.15.164:6839/1636226820] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5565bece2210 con 0x5565bed86ec0 2024-04-26T21:54:38.812 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.807+0000 7fa137400640 1 --2- 172.21.15.72:0/2994744577 >> [v2:172.21.15.164:6838/1636226820,v1:172.21.15.164:6839/1636226820] conn(0x5565bed86ec0 0x5565bed88330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:38.813 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.808+0000 7fa137400640 1 --2- 172.21.15.72:0/2994744577 >> [v2:172.21.15.164:6838/1636226820,v1:172.21.15.164:6839/1636226820] conn(0x5565bed86ec0 0x5565bed88330 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f049bf11b21af82b server_cookie=7e5766f6bef5f23d in_seq=0 out_seq=0 2024-04-26T21:54:38.813 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.808+0000 7fa12e000640 10 client.15090 ms_handle_connect on v2:172.21.15.164:6838/1636226820 2024-04-26T21:54:38.813 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.810+0000 7fa12e000640 1 -- 172.21.15.72:0/2994744577 <== mds.4 v2:172.21.15.164:6838/1636226820 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5565bece2210 con 0x5565bed86ec0 2024-04-26T21:54:38.813 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.810+0000 7fa12e000640 10 client.15090 handle_command_reply: tid=0 2024-04-26T21:54:38.813 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.810+0000 7fa149e35740 1 client.15090 shutdown 2024-04-26T21:54:38.813 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.811+0000 7fa149e35740 1 -- 172.21.15.72:0/2994744577 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5565bed702c0 msgr2=0x5565bed7a6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:38.813 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.811+0000 7fa149e35740 1 --2- 172.21.15.72:0/2994744577 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5565bed702c0 0x5565bed7a6b0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fa13002a9e0 tx=0x7fa130003e60 comp rx=0 tx=0).stop 2024-04-26T21:54:38.813 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.811+0000 7fa149e35740 1 -- 172.21.15.72:0/2994744577 shutdown_connections 2024-04-26T21:54:38.813 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.811+0000 7fa149e35740 1 --2- 172.21.15.72:0/2994744577 >> [v2:172.21.15.164:6838/1636226820,v1:172.21.15.164:6839/1636226820] conn(0x5565bed86ec0 0x5565bed88330 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:38.813 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.811+0000 7fa149e35740 1 --2- 172.21.15.72:0/2994744577 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5565bed75e20 0x5565bed81c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:38.813 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.811+0000 7fa149e35740 1 --2- 172.21.15.72:0/2994744577 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5565bed70cb0 0x5565bed7abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:38.814 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.811+0000 7fa149e35740 1 --2- 172.21.15.72:0/2994744577 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5565bed702c0 0x5565bed7a6b0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:38.814 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.811+0000 7fa12e000640 0 client.15090 ms_handle_reset on v2:172.21.15.164:6838/1636226820 2024-04-26T21:54:38.814 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.811+0000 7fa149e35740 1 -- 172.21.15.72:0/2994744577 >> 172.21.15.72:0/2994744577 conn(0x5565bed04d10 msgr2=0x5565bed6af60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:54:38.814 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.811+0000 7fa149e35740 1 -- 172.21.15.72:0/2994744577 shutdown_connections 2024-04-26T21:54:38.814 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.811+0000 7fa149e35740 1 -- 172.21.15.72:0/2994744577 wait complete. 2024-04-26T21:54:38.814 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.811+0000 7fa149e35740 20 client.15090 trim_cache size 0 max 16384 2024-04-26T21:54:38.831 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.831+0000 7fa149e35740 1 Processor -- start 2024-04-26T21:54:38.831 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.831+0000 7fa149e35740 1 -- start start 2024-04-26T21:54:38.831 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.831+0000 7fa149e35740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5565bed702c0 0x5565bed6ade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:38.831 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.831+0000 7fa149e35740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5565bed70cb0 0x5565bed81bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:38.831 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.831+0000 7fa149e35740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5565bed75e20 0x5565bed79980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:38.831 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.831+0000 7fa149e35740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x5565bed86670 con 0x5565bed75e20 2024-04-26T21:54:38.831 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.831+0000 7fa149e35740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5565bed86d40 con 0x5565bed70cb0 2024-04-26T21:54:38.832 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.831+0000 7fa149e35740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5565bed87410 con 0x5565bed702c0 2024-04-26T21:54:38.833 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.832+0000 7fa145000640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5565bed75e20 0x5565bed79980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:38.833 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.832+0000 7fa145000640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5565bed75e20 0x5565bed79980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:39220/0 (socket says 172.21.15.72:39220) 2024-04-26T21:54:38.833 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.832+0000 7fa145000640 1 -- 172.21.15.72:0/2042409242 learned_addr learned my addr 172.21.15.72:0/2042409242 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:54:38.833 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.832+0000 7fa13fe00640 1 --2- 172.21.15.72:0/2042409242 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5565bed702c0 0x5565bed6ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:38.833 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.832+0000 7fa137400640 1 --2- 172.21.15.72:0/2042409242 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5565bed70cb0 0x5565bed81bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:38.833 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.832+0000 7fa145000640 1 -- 172.21.15.72:0/2042409242 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5565bed702c0 msgr2=0x5565bed6ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:54:38.833 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.832+0000 7fa145000640 1 --2- 172.21.15.72:0/2042409242 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5565bed702c0 0x5565bed6ade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:38.833 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.832+0000 7fa145000640 1 -- 172.21.15.72:0/2042409242 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5565bed70cb0 msgr2=0x5565bed81bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:54:38.833 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.832+0000 7fa145000640 1 --2- 172.21.15.72:0/2042409242 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5565bed70cb0 0x5565bed81bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:38.834 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.832+0000 7fa145000640 1 -- 172.21.15.72:0/2042409242 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5565bed67460 con 0x5565bed75e20 2024-04-26T21:54:38.834 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.832+0000 7fa145000640 1 --2- 172.21.15.72:0/2042409242 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5565bed75e20 0x5565bed79980 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fa138011600 tx=0x7fa138013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7381c01748cf4c05 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:54:38.834 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.833+0000 7fa134c00640 1 -- 172.21.15.72:0/2042409242 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa13801d6f0 con 0x5565bed75e20 2024-04-26T21:54:38.834 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.833+0000 7fa134c00640 1 -- 172.21.15.72:0/2042409242 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa13801dd20 con 0x5565bed75e20 2024-04-26T21:54:38.834 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.833+0000 7fa134c00640 1 -- 172.21.15.72:0/2042409242 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa13801edf0 con 0x5565bed75e20 2024-04-26T21:54:38.837 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.833+0000 7fa149e35740 1 -- 172.21.15.72:0/2042409242 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5565bed75e20 msgr2=0x5565bed79980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:38.837 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.833+0000 7fa149e35740 1 --2- 172.21.15.72:0/2042409242 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5565bed75e20 0x5565bed79980 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fa138011600 tx=0x7fa138013000 comp rx=0 tx=0).stop 2024-04-26T21:54:38.837 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.833+0000 7fa149e35740 1 -- 172.21.15.72:0/2042409242 shutdown_connections 2024-04-26T21:54:38.837 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.833+0000 7fa149e35740 1 --2- 172.21.15.72:0/2042409242 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5565bed75e20 0x5565bed79980 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:38.837 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.833+0000 7fa149e35740 1 --2- 172.21.15.72:0/2042409242 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5565bed70cb0 0x5565bed81bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:38.838 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.833+0000 7fa149e35740 1 --2- 172.21.15.72:0/2042409242 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5565bed702c0 0x5565bed6ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:38.838 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.833+0000 7fa149e35740 1 -- 172.21.15.72:0/2042409242 >> 172.21.15.72:0/2042409242 conn(0x5565bed04d10 msgr2=0x5565bed6a6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:54:38.838 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.834+0000 7fa149e35740 1 -- 172.21.15.72:0/2042409242 shutdown_connections 2024-04-26T21:54:38.838 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.834+0000 7fa149e35740 1 -- 172.21.15.72:0/2042409242 wait complete. 2024-04-26T21:54:38.838 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.834+0000 7fa149e35740 1 -- start start 2024-04-26T21:54:38.838 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.834+0000 7fa149e35740 1 Processor -- start 2024-04-26T21:54:38.838 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.834+0000 7fa149e35740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5565bed702c0 0x5565bed7de20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:38.838 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.834+0000 7fa149e35740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5565bed70cb0 0x5565bed7e360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:38.838 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.834+0000 7fa149e35740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5565bed75e20 0x5565bed810a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:38.838 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.835+0000 7fa149e35740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x5565bed881b0 con 0x5565bed702c0 2024-04-26T21:54:38.838 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.835+0000 7fa149e35740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5565bed87ae0 con 0x5565bed70cb0 2024-04-26T21:54:38.838 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.835+0000 7fa149e35740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5565bed88880 con 0x5565bed75e20 2024-04-26T21:54:38.839 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.835+0000 7fa149e35740 10 client.0 _ll_register_callbacks cb 0x7fa1300040d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:54:38.839 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.835+0000 7fa13fe00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5565bed702c0 0x5565bed7de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:38.839 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.835+0000 7fa13fe00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5565bed702c0 0x5565bed7de20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:39228/0 (socket says 172.21.15.72:39228) 2024-04-26T21:54:38.839 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.835+0000 7fa13fe00640 1 -- 172.21.15.72:0/190498175 learned_addr learned my addr 172.21.15.72:0/190498175 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:54:38.839 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.835+0000 7fa13fe00640 1 -- 172.21.15.72:0/190498175 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5565bed75e20 msgr2=0x5565bed810a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:54:38.839 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.835+0000 7fa13fe00640 1 --2- 172.21.15.72:0/190498175 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5565bed75e20 0x5565bed810a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:38.839 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.835+0000 7fa137400640 1 --2- 172.21.15.72:0/190498175 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5565bed70cb0 0x5565bed7e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:38.839 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.835+0000 7fa13fe00640 1 -- 172.21.15.72:0/190498175 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5565bed70cb0 msgr2=0x5565bed7e360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:54:38.839 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.835+0000 7fa13fe00640 1 --2- 172.21.15.72:0/190498175 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5565bed70cb0 0x5565bed7e360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:38.839 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.835+0000 7fa13fe00640 1 -- 172.21.15.72:0/190498175 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5565bed816e0 con 0x5565bed702c0 2024-04-26T21:54:38.839 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.835+0000 7fa13fe00640 1 --2- 172.21.15.72:0/190498175 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5565bed702c0 0x5565bed7de20 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fa1300057c0 tx=0x7fa130003b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f39854b05d629886 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:54:38.839 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.835+0000 7fa12e000640 10 client.0 ms_handle_connect on v2:172.21.15.72:3300/0 2024-04-26T21:54:38.839 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.836+0000 7fa12e000640 1 -- 172.21.15.72:0/190498175 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa130035040 con 0x5565bed702c0 2024-04-26T21:54:38.839 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.836+0000 7fa12e000640 1 -- 172.21.15.72:0/190498175 <== mon.0 v2:172.21.15.72:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7fa130054070 con 0x5565bed702c0 2024-04-26T21:54:38.839 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.836+0000 7fa149e35740 1 -- 172.21.15.72:0/190498175 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa13801dd20 con 0x5565bed702c0 2024-04-26T21:54:38.840 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.836+0000 7fa12e000640 1 -- 172.21.15.72:0/190498175 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fa130004180 con 0x5565bed702c0 2024-04-26T21:54:38.840 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.836+0000 7fa149e35740 10 client.15096 fetch_fsmap learned FSMap version 26 2024-04-26T21:54:38.840 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.836+0000 7fa149e35740 1 -- 172.21.15.72:0/190498175 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5565bedf5f00 con 0x5565bed702c0 2024-04-26T21:54:38.840 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.836+0000 7fa12e000640 1 -- 172.21.15.72:0/190498175 <== mon.0 v2:172.21.15.72:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa13801dd20 con 0x5565bed702c0 2024-04-26T21:54:38.840 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.836+0000 7fa12e000640 1 -- 172.21.15.72:0/190498175 <== mon.0 v2:172.21.15.72:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fa13003c9e0 con 0x5565bed702c0 2024-04-26T21:54:38.840 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.836+0000 7fa149e35740 10 client.15096 fetch_fsmap finished waiting for FSMap version 26 2024-04-26T21:54:38.840 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.836+0000 7fa149e35740 10 client.15096 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-26T21:54:38.840 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.836+0000 7fa149e35740 20 client.15096 populate_metadata read hostname 'smithi072' 2024-04-26T21:54:38.840 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.836+0000 7fa149e35740 1 --2- 172.21.15.72:0/190498175 >> [v2:172.21.15.164:6838/1636226820,v1:172.21.15.164:6839/1636226820] conn(0x5565bedfaa90 0x5565bedf97a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:38.840 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.836+0000 7fa149e35740 4 client.15096 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T21:54:38.840 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.836+0000 7fa149e35740 1 -- 172.21.15.72:0/190498175 --> [v2:172.21.15.164:6838/1636226820,v1:172.21.15.164:6839/1636226820] -- command(tid 0: {"prefix": "status"}) -- 0x5565becdc080 con 0x5565bedfaa90 2024-04-26T21:54:38.840 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.837+0000 7fa137400640 1 --2- 172.21.15.72:0/190498175 >> [v2:172.21.15.164:6838/1636226820,v1:172.21.15.164:6839/1636226820] conn(0x5565bedfaa90 0x5565bedf97a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:38.841 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.838+0000 7fa137400640 1 --2- 172.21.15.72:0/190498175 >> [v2:172.21.15.164:6838/1636226820,v1:172.21.15.164:6839/1636226820] conn(0x5565bedfaa90 0x5565bedf97a0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ac77fce6c11bcc3a server_cookie=a20fa679dab8ff0b in_seq=0 out_seq=0 2024-04-26T21:54:38.842 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.838+0000 7fa12e000640 10 client.15096 ms_handle_connect on v2:172.21.15.164:6838/1636226820 2024-04-26T21:54:38.842 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.838+0000 7fa12e000640 1 -- 172.21.15.72:0/190498175 <== mds.4 v2:172.21.15.164:6838/1636226820 1 ==== command_reply(tid 0: 0 ) ==== 8+0+321 (crc 0 0 0) 0x5565becdc080 con 0x5565bedfaa90 2024-04-26T21:54:38.842 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.839+0000 7fa12e000640 10 client.15096 handle_command_reply: tid=0 2024-04-26T21:54:38.842 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.839+0000 7fa149e35740 1 client.15096 shutdown 2024-04-26T21:54:38.842 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.839+0000 7fa149e35740 1 -- 172.21.15.72:0/190498175 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5565bed702c0 msgr2=0x5565bed7de20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:38.842 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.839+0000 7fa149e35740 1 --2- 172.21.15.72:0/190498175 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5565bed702c0 0x5565bed7de20 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fa1300057c0 tx=0x7fa130003b10 comp rx=0 tx=0).stop 2024-04-26T21:54:38.842 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.839+0000 7fa149e35740 1 -- 172.21.15.72:0/190498175 shutdown_connections 2024-04-26T21:54:38.842 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.839+0000 7fa149e35740 1 --2- 172.21.15.72:0/190498175 >> [v2:172.21.15.164:6838/1636226820,v1:172.21.15.164:6839/1636226820] conn(0x5565bedfaa90 0x5565bedf97a0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:38.842 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.839+0000 7fa149e35740 1 --2- 172.21.15.72:0/190498175 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5565bed75e20 0x5565bed810a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:38.842 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.839+0000 7fa149e35740 1 --2- 172.21.15.72:0/190498175 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5565bed70cb0 0x5565bed7e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:38.842 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.839+0000 7fa149e35740 1 --2- 172.21.15.72:0/190498175 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5565bed702c0 0x5565bed7de20 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:38.842 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.839+0000 7fa12e000640 0 client.15096 ms_handle_reset on v2:172.21.15.164:6838/1636226820 2024-04-26T21:54:38.842 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.839+0000 7fa149e35740 1 -- 172.21.15.72:0/190498175 >> 172.21.15.72:0/190498175 conn(0x5565bed04d10 msgr2=0x5565bed6f5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:54:38.842 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.839+0000 7fa149e35740 1 -- 172.21.15.72:0/190498175 shutdown_connections 2024-04-26T21:54:38.843 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.839+0000 7fa149e35740 1 -- 172.21.15.72:0/190498175 wait complete. 2024-04-26T21:54:38.843 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.839+0000 7fa149e35740 20 client.15096 trim_cache size 0 max 16384 2024-04-26T21:54:38.843 INFO:teuthology.orchestra.run.smithi072.stdout:{ 2024-04-26T21:54:38.843 INFO:teuthology.orchestra.run.smithi072.stdout: "cluster_fsid": "a1780950-0415-11ef-bc93-c7b262605968", 2024-04-26T21:54:38.843 INFO:teuthology.orchestra.run.smithi072.stdout: "whoami": 4, 2024-04-26T21:54:38.843 INFO:teuthology.orchestra.run.smithi072.stdout: "id": 24463, 2024-04-26T21:54:38.843 INFO:teuthology.orchestra.run.smithi072.stdout: "want_state": "up:active", 2024-04-26T21:54:38.843 INFO:teuthology.orchestra.run.smithi072.stdout: "state": "up:active", 2024-04-26T21:54:38.843 INFO:teuthology.orchestra.run.smithi072.stdout: "fs_name": "cephfs", 2024-04-26T21:54:38.843 INFO:teuthology.orchestra.run.smithi072.stdout: "rank_uptime": 101.260640268, 2024-04-26T21:54:38.843 INFO:teuthology.orchestra.run.smithi072.stdout: "mdsmap_epoch": 26, 2024-04-26T21:54:38.844 INFO:teuthology.orchestra.run.smithi072.stdout: "osdmap_epoch": 91, 2024-04-26T21:54:38.844 INFO:teuthology.orchestra.run.smithi072.stdout: "osdmap_epoch_barrier": 87, 2024-04-26T21:54:38.844 INFO:teuthology.orchestra.run.smithi072.stdout: "uptime": 126.34450474 2024-04-26T21:54:38.844 INFO:teuthology.orchestra.run.smithi072.stdout:} 2024-04-26T21:54:38.844 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.840+0000 7fa145a00640 1 -- 172.21.15.72:0/3711720613 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fa1240a3cc0 msgr2=0x7fa1240a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:38.845 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.840+0000 7fa145a00640 1 --2- 172.21.15.72:0/3711720613 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fa1240a3cc0 0x7fa1240a6180 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x5565bed71510 tx=0x7fa138009710 comp rx=0 tx=0).stop 2024-04-26T21:54:38.845 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.840+0000 7fa145a00640 1 -- 172.21.15.72:0/3711720613 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa1401020d0 msgr2=0x7fa14010ab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:38.845 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.840+0000 7fa145a00640 1 --2- 172.21.15.72:0/3711720613 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa1401020d0 0x7fa14010ab50 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fa12800c970 tx=0x7fa12800ce30 comp rx=0 tx=0).stop 2024-04-26T21:54:38.845 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.840+0000 7fa145a00640 1 -- 172.21.15.72:0/3711720613 shutdown_connections 2024-04-26T21:54:38.845 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.840+0000 7fa145a00640 1 --2- 172.21.15.72:0/3711720613 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa140106dd0 0x7fa14010b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:38.845 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.840+0000 7fa145a00640 1 --2- 172.21.15.72:0/3711720613 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa1401020d0 0x7fa14010ab50 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:38.845 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.840+0000 7fa145a00640 1 --2- 172.21.15.72:0/3711720613 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fa1240a3cc0 0x7fa1240a6180 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:38.845 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.840+0000 7fa145a00640 1 --2- 172.21.15.72:0/3711720613 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa140068de0 0x7fa14010c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:38.845 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.840+0000 7fa145a00640 1 -- 172.21.15.72:0/3711720613 >> 172.21.15.72:0/3711720613 conn(0x7fa1400fbf10 msgr2=0x7fa140102db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:54:38.845 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.840+0000 7fa145a00640 1 -- 172.21.15.72:0/3711720613 shutdown_connections 2024-04-26T21:54:38.845 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:38.840+0000 7fa145a00640 1 -- 172.21.15.72:0/3711720613 wait complete. 2024-04-26T21:54:38.857 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.k status 2024-04-26T21:54:39.048 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.047+0000 7f7447e00640 1 -- 172.21.15.72:0/423727613 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7448100e10 msgr2=0x7f7448101230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:39.048 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.047+0000 7f7447e00640 1 --2- 172.21.15.72:0/423727613 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7448100e10 0x7f7448101230 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f7438009900 tx=0x7f743802a410 comp rx=0 tx=0).stop 2024-04-26T21:54:39.048 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.048+0000 7f7447e00640 1 -- 172.21.15.72:0/423727613 shutdown_connections 2024-04-26T21:54:39.048 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.048+0000 7f7447e00640 1 --2- 172.21.15.72:0/423727613 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7448102920 0x7f7448109240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:39.048 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.048+0000 7f7447e00640 1 --2- 172.21.15.72:0/423727613 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7448101f60 0x7f74481023e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:39.048 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.048+0000 7f7447e00640 1 --2- 172.21.15.72:0/423727613 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7448100e10 0x7f7448101230 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:39.049 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.048+0000 7f7447e00640 1 -- 172.21.15.72:0/423727613 >> 172.21.15.72:0/423727613 conn(0x7f74480fc380 msgr2=0x7f74480fe7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:54:39.049 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.048+0000 7f7447e00640 1 -- 172.21.15.72:0/423727613 shutdown_connections 2024-04-26T21:54:39.049 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.048+0000 7f7447e00640 1 -- 172.21.15.72:0/423727613 wait complete. 2024-04-26T21:54:39.051 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.049+0000 7f7447e00640 1 Processor -- start 2024-04-26T21:54:39.051 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.049+0000 7f7447e00640 1 -- start start 2024-04-26T21:54:39.051 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.049+0000 7f7447e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7448100e10 0x7f744819aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:39.051 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.049+0000 7f7447e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7448101f60 0x7f744819b030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:39.051 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.049+0000 7f7447e00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7448102920 0x7f74481a20a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:39.051 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.049+0000 7f7447e00640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f744810e1d0 con 0x7f7448101f60 2024-04-26T21:54:39.051 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.049+0000 7f7447e00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f744810e050 con 0x7f7448102920 2024-04-26T21:54:39.052 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.049+0000 7f7447e00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f744810e350 con 0x7f7448100e10 2024-04-26T21:54:39.052 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.049+0000 7f743e000640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7448101f60 0x7f744819b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:39.052 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.049+0000 7f743e000640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7448101f60 0x7f744819b030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:39254/0 (socket says 172.21.15.72:39254) 2024-04-26T21:54:39.052 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.049+0000 7f743e000640 1 -- 172.21.15.72:0/3791963794 learned_addr learned my addr 172.21.15.72:0/3791963794 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:54:39.052 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.050+0000 7f7446a00640 1 --2- 172.21.15.72:0/3791963794 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7448100e10 0x7f744819aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:39.052 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.050+0000 7f743e000640 1 -- 172.21.15.72:0/3791963794 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7448100e10 msgr2=0x7f744819aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:39.052 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.050+0000 7f743e000640 1 --2- 172.21.15.72:0/3791963794 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7448100e10 0x7f744819aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:39.052 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.050+0000 7f7447400640 1 --2- 172.21.15.72:0/3791963794 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7448102920 0x7f74481a20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:39.052 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.050+0000 7f743e000640 1 -- 172.21.15.72:0/3791963794 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7448102920 msgr2=0x7f74481a20a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:39.052 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.050+0000 7f743e000640 1 --2- 172.21.15.72:0/3791963794 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7448102920 0x7f74481a20a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:39.052 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.050+0000 7f743e000640 1 -- 172.21.15.72:0/3791963794 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74481a2710 con 0x7f7448101f60 2024-04-26T21:54:39.052 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.050+0000 7f7446a00640 1 --2- 172.21.15.72:0/3791963794 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7448100e10 0x7f744819aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:54:39.052 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.050+0000 7f743e000640 1 --2- 172.21.15.72:0/3791963794 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7448101f60 0x7f744819b030 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f743000c970 tx=0x7f743000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:54:39.052 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.050+0000 7f743fe00640 1 -- 172.21.15.72:0/3791963794 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7430004220 con 0x7f7448101f60 2024-04-26T21:54:39.053 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.050+0000 7f743fe00640 1 -- 172.21.15.72:0/3791963794 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7430007e40 con 0x7f7448101f60 2024-04-26T21:54:39.053 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.050+0000 7f743fe00640 1 -- 172.21.15.72:0/3791963794 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f743000f450 con 0x7f7448101f60 2024-04-26T21:54:39.053 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.050+0000 7f7447e00640 1 -- 172.21.15.72:0/3791963794 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74481a2a00 con 0x7f7448101f60 2024-04-26T21:54:39.053 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.051+0000 7f7447e00640 1 -- 172.21.15.72:0/3791963794 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74481a2fa0 con 0x7f7448101f60 2024-04-26T21:54:39.053 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.053+0000 7f74507ec740 1 Processor -- start 2024-04-26T21:54:39.054 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.053+0000 7f743fe00640 1 -- 172.21.15.72:0/3791963794 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f7430003610 con 0x7f7448101f60 2024-04-26T21:54:39.054 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.054+0000 7f74507ec740 1 -- start start 2024-04-26T21:54:39.055 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.054+0000 7f74507ec740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55904d4282c0 0x55904d4286e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:39.055 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.054+0000 7f74507ec740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55904d428cb0 0x55904d42d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:39.055 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.054+0000 7f74507ec740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55904d42de20 0x55904d4302b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:39.055 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.054+0000 7f74507ec740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55904d38c600 con 0x55904d4282c0 2024-04-26T21:54:39.055 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.054+0000 7f74507ec740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55904d329100 con 0x55904d42de20 2024-04-26T21:54:39.055 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.054+0000 7f74507ec740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55904d3885c0 con 0x55904d428cb0 2024-04-26T21:54:39.055 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.054+0000 7f7447400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55904d4282c0 0x55904d4286e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:39.055 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.054+0000 7f7447400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55904d4282c0 0x55904d4286e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:39268/0 (socket says 172.21.15.72:39268) 2024-04-26T21:54:39.056 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.054+0000 7f7447400640 1 -- 172.21.15.72:0/614823456 learned_addr learned my addr 172.21.15.72:0/614823456 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:54:39.056 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.054+0000 7f7447400640 1 -- 172.21.15.72:0/614823456 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55904d428cb0 msgr2=0x55904d42d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:54:39.056 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.054+0000 7f7447400640 1 --2- 172.21.15.72:0/614823456 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55904d428cb0 0x55904d42d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:39.056 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.055+0000 7f7447400640 1 -- 172.21.15.72:0/614823456 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55904d42de20 msgr2=0x55904d4302b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:54:39.056 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.055+0000 7f7447400640 1 --2- 172.21.15.72:0/614823456 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55904d42de20 0x55904d4302b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:39.056 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.055+0000 7f7447400640 1 -- 172.21.15.72:0/614823456 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55904d09e2d0 con 0x55904d4282c0 2024-04-26T21:54:39.056 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.055+0000 7f7447400640 1 --2- 172.21.15.72:0/614823456 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55904d4282c0 0x55904d4286e0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f7440008d00 tx=0x7f744000d710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=161ab1ee0cc4911f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:54:39.056 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.055+0000 7f743fe00640 1 --2- 172.21.15.72:0/3791963794 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f742c0a3cc0 0x7f742c0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:39.061 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.055+0000 7f7437e00640 1 -- 172.21.15.72:0/614823456 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7440018680 con 0x55904d4282c0 2024-04-26T21:54:39.061 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.055+0000 7f7437e00640 1 -- 172.21.15.72:0/614823456 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7440018cb0 con 0x55904d4282c0 2024-04-26T21:54:39.061 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.055+0000 7f7437e00640 1 -- 172.21.15.72:0/614823456 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f744000fcd0 con 0x55904d4282c0 2024-04-26T21:54:39.061 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.055+0000 7f743fe00640 1 -- 172.21.15.72:0/3791963794 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7f74300d0320 con 0x7f7448101f60 2024-04-26T21:54:39.061 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.055+0000 7f7446a00640 1 --2- 172.21.15.72:0/3791963794 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f742c0a3cc0 0x7f742c0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:39.061 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.056+0000 7f74507ec740 1 -- 172.21.15.72:0/614823456 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55904d4282c0 msgr2=0x55904d4286e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:39.061 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.056+0000 7f74507ec740 1 --2- 172.21.15.72:0/614823456 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55904d4282c0 0x55904d4286e0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f7440008d00 tx=0x7f744000d710 comp rx=0 tx=0).stop 2024-04-26T21:54:39.061 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.056+0000 7f7446a00640 1 --2- 172.21.15.72:0/3791963794 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f742c0a3cc0 0x7f742c0a6180 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f743802a930 tx=0x7f7438000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:54:39.061 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.056+0000 7f74507ec740 1 -- 172.21.15.72:0/614823456 shutdown_connections 2024-04-26T21:54:39.061 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.056+0000 7f74507ec740 1 --2- 172.21.15.72:0/614823456 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55904d42de20 0x55904d4302b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:39.061 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.056+0000 7f74507ec740 1 --2- 172.21.15.72:0/614823456 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55904d428cb0 0x55904d42d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:39.061 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.056+0000 7f74507ec740 1 --2- 172.21.15.72:0/614823456 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55904d4282c0 0x55904d4286e0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:39.061 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.056+0000 7f74507ec740 1 -- 172.21.15.72:0/614823456 >> 172.21.15.72:0/614823456 conn(0x55904d3bcd10 msgr2=0x55904d423d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:54:39.062 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.056+0000 7f74507ec740 1 -- 172.21.15.72:0/614823456 shutdown_connections 2024-04-26T21:54:39.062 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.056+0000 7f74507ec740 1 -- 172.21.15.72:0/614823456 wait complete. 2024-04-26T21:54:39.062 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.056+0000 7f74507ec740 1 -- start start 2024-04-26T21:54:39.062 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.057+0000 7f74507ec740 1 Processor -- start 2024-04-26T21:54:39.062 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.057+0000 7f74507ec740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55904d4282c0 0x55904d4327e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:39.062 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.057+0000 7f74507ec740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55904d428cb0 0x55904d432d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:39.062 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.057+0000 7f74507ec740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55904d42de20 0x55904d439d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:39.062 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.057+0000 7f74507ec740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55904d430dd0 con 0x55904d42de20 2024-04-26T21:54:39.062 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.057+0000 7f74507ec740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55904d313f70 con 0x55904d4282c0 2024-04-26T21:54:39.062 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.057+0000 7f74507ec740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55904d430f50 con 0x55904d428cb0 2024-04-26T21:54:39.062 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.057+0000 7f74507ec740 10 client.0 _ll_register_callbacks cb 0x55904d382480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:54:39.062 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.057+0000 7f7447400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55904d42de20 0x55904d439d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:39.062 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.057+0000 7f7447400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55904d42de20 0x55904d439d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:39280/0 (socket says 172.21.15.72:39280) 2024-04-26T21:54:39.062 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.057+0000 7f7447400640 1 -- 172.21.15.72:0/2627120455 learned_addr learned my addr 172.21.15.72:0/2627120455 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:54:39.063 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.058+0000 7f7446a00640 1 --2- 172.21.15.72:0/2627120455 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55904d4282c0 0x55904d4327e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:39.063 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.058+0000 7f743e000640 1 --2- 172.21.15.72:0/2627120455 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55904d428cb0 0x55904d432d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:39.063 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.058+0000 7f7447400640 1 -- 172.21.15.72:0/2627120455 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55904d428cb0 msgr2=0x55904d432d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:54:39.063 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.058+0000 7f7447400640 1 --2- 172.21.15.72:0/2627120455 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55904d428cb0 0x55904d432d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:39.063 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.058+0000 7f7447400640 1 -- 172.21.15.72:0/2627120455 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55904d4282c0 msgr2=0x55904d4327e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:54:39.063 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.058+0000 7f7447400640 1 --2- 172.21.15.72:0/2627120455 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55904d4282c0 0x55904d4327e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:39.063 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.058+0000 7f7447400640 1 -- 172.21.15.72:0/2627120455 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55904d1686b0 con 0x55904d42de20 2024-04-26T21:54:39.063 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.058+0000 7f743e000640 1 --2- 172.21.15.72:0/2627120455 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55904d428cb0 0x55904d432d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:54:39.063 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.058+0000 7f7446a00640 1 --2- 172.21.15.72:0/2627120455 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55904d4282c0 0x55904d4327e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:54:39.063 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.058+0000 7f7447400640 1 --2- 172.21.15.72:0/2627120455 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55904d42de20 0x55904d439d60 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f744000fd60 tx=0x7f744000fd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ce2147f29dbf72 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:54:39.063 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.058+0000 7f7434c00640 10 client.0 ms_handle_connect on v2:172.21.15.72:3300/0 2024-04-26T21:54:39.063 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.058+0000 7f7434c00640 1 -- 172.21.15.72:0/2627120455 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f744001f070 con 0x55904d42de20 2024-04-26T21:54:39.063 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.058+0000 7f74507ec740 1 -- 172.21.15.72:0/2627120455 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7440018cb0 con 0x55904d42de20 2024-04-26T21:54:39.063 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.058+0000 7f7434c00640 1 -- 172.21.15.72:0/2627120455 <== mon.0 v2:172.21.15.72:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7f7440018890 con 0x55904d42de20 2024-04-26T21:54:39.064 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.059+0000 7f7434c00640 1 -- 172.21.15.72:0/2627120455 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f7440018a90 con 0x55904d42de20 2024-04-26T21:54:39.064 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.059+0000 7f74507ec740 10 client.15108 fetch_fsmap learned FSMap version 26 2024-04-26T21:54:39.064 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.059+0000 7f74507ec740 1 -- 172.21.15.72:0/2627120455 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55904d43cdc0 con 0x55904d42de20 2024-04-26T21:54:39.064 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.059+0000 7f7434c00640 1 -- 172.21.15.72:0/2627120455 <== mon.0 v2:172.21.15.72:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7440018cb0 con 0x55904d42de20 2024-04-26T21:54:39.064 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.059+0000 7f7434c00640 1 -- 172.21.15.72:0/2627120455 <== mon.0 v2:172.21.15.72:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f744002d410 con 0x55904d42de20 2024-04-26T21:54:39.064 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.059+0000 7f74507ec740 10 client.15108 fetch_fsmap finished waiting for FSMap version 26 2024-04-26T21:54:39.064 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.059+0000 7f74507ec740 10 client.15108 resolve_mds: resolved name 'k' to daemon mds.k 2024-04-26T21:54:39.064 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.059+0000 7f74507ec740 20 client.15108 populate_metadata read hostname 'smithi072' 2024-04-26T21:54:39.064 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.059+0000 7f74507ec740 1 --2- 172.21.15.72:0/2627120455 >> [v2:172.21.15.76:6832/3869483750,v1:172.21.15.76:6833/3869483750] conn(0x55904d43f010 0x55904d440480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:39.064 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.059+0000 7f74507ec740 4 client.15108 mds_command: new command op to 34406 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:54:39.064 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.059+0000 7f74507ec740 1 -- 172.21.15.72:0/2627120455 --> [v2:172.21.15.76:6832/3869483750,v1:172.21.15.76:6833/3869483750] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55904d39a210 con 0x55904d43f010 2024-04-26T21:54:39.064 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.060+0000 7f7446a00640 1 --2- 172.21.15.72:0/2627120455 >> [v2:172.21.15.76:6832/3869483750,v1:172.21.15.76:6833/3869483750] conn(0x55904d43f010 0x55904d440480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:39.064 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.061+0000 7f7446a00640 1 --2- 172.21.15.72:0/2627120455 >> [v2:172.21.15.76:6832/3869483750,v1:172.21.15.76:6833/3869483750] conn(0x55904d43f010 0x55904d440480 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=25d16599958594f6 server_cookie=90cc96b30c4388d0 in_seq=0 out_seq=0 2024-04-26T21:54:39.064 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.061+0000 7f7434c00640 10 client.15108 ms_handle_connect on v2:172.21.15.76:6832/3869483750 2024-04-26T21:54:39.065 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.063+0000 7f7434c00640 1 -- 172.21.15.72:0/2627120455 <== mds.3 v2:172.21.15.76:6832/3869483750 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55904d39a210 con 0x55904d43f010 2024-04-26T21:54:39.066 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.063+0000 7f7434c00640 10 client.15108 handle_command_reply: tid=0 2024-04-26T21:54:39.066 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.063+0000 7f74507ec740 1 client.15108 shutdown 2024-04-26T21:54:39.066 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.063+0000 7f74507ec740 1 -- 172.21.15.72:0/2627120455 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55904d42de20 msgr2=0x55904d439d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:39.066 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.063+0000 7f74507ec740 1 --2- 172.21.15.72:0/2627120455 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55904d42de20 0x55904d439d60 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f744000fd60 tx=0x7f744000fd90 comp rx=0 tx=0).stop 2024-04-26T21:54:39.066 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.063+0000 7f74507ec740 1 -- 172.21.15.72:0/2627120455 shutdown_connections 2024-04-26T21:54:39.066 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.063+0000 7f74507ec740 1 --2- 172.21.15.72:0/2627120455 >> [v2:172.21.15.76:6832/3869483750,v1:172.21.15.76:6833/3869483750] conn(0x55904d43f010 0x55904d440480 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:39.066 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.063+0000 7f74507ec740 1 --2- 172.21.15.72:0/2627120455 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55904d42de20 0x55904d439d60 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:39.066 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.063+0000 7f74507ec740 1 --2- 172.21.15.72:0/2627120455 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55904d428cb0 0x55904d432d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:39.066 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.063+0000 7f74507ec740 1 --2- 172.21.15.72:0/2627120455 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55904d4282c0 0x55904d4327e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:39.066 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.063+0000 7f7434c00640 0 client.15108 ms_handle_reset on v2:172.21.15.76:6832/3869483750 2024-04-26T21:54:39.066 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.063+0000 7f74507ec740 1 -- 172.21.15.72:0/2627120455 >> 172.21.15.72:0/2627120455 conn(0x55904d3bcd10 msgr2=0x55904d422f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:54:39.066 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.064+0000 7f74507ec740 1 -- 172.21.15.72:0/2627120455 shutdown_connections 2024-04-26T21:54:39.066 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.064+0000 7f74507ec740 1 -- 172.21.15.72:0/2627120455 wait complete. 2024-04-26T21:54:39.067 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.064+0000 7f74507ec740 20 client.15108 trim_cache size 0 max 16384 2024-04-26T21:54:39.083 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.082+0000 7f74507ec740 1 Processor -- start 2024-04-26T21:54:39.083 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.083+0000 7f74507ec740 1 -- start start 2024-04-26T21:54:39.083 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.083+0000 7f74507ec740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55904d4282c0 0x55904d41edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:39.083 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.083+0000 7f74507ec740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55904d428cb0 0x55904d439d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:39.083 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.083+0000 7f74507ec740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55904d42de20 0x55904d4327e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:39.083 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.083+0000 7f74507ec740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55904d43e7c0 con 0x55904d4282c0 2024-04-26T21:54:39.083 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.083+0000 7f74507ec740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55904d43ee90 con 0x55904d42de20 2024-04-26T21:54:39.083 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.083+0000 7f74507ec740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55904d43f560 con 0x55904d428cb0 2024-04-26T21:54:39.084 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.083+0000 7f7446a00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55904d4282c0 0x55904d41edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:39.084 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.083+0000 7f7446a00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55904d4282c0 0x55904d41edf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:39296/0 (socket says 172.21.15.72:39296) 2024-04-26T21:54:39.084 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.083+0000 7f7446a00640 1 -- 172.21.15.72:0/1194430494 learned_addr learned my addr 172.21.15.72:0/1194430494 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:54:39.085 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.083+0000 7f7447400640 1 --2- 172.21.15.72:0/1194430494 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55904d42de20 0x55904d4327e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:39.085 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.083+0000 7f743e000640 1 --2- 172.21.15.72:0/1194430494 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55904d428cb0 0x55904d439d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:39.085 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.084+0000 7f7446a00640 1 -- 172.21.15.72:0/1194430494 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55904d428cb0 msgr2=0x55904d439d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:54:39.085 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.084+0000 7f7446a00640 1 --2- 172.21.15.72:0/1194430494 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55904d428cb0 0x55904d439d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:39.085 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.084+0000 7f7446a00640 1 -- 172.21.15.72:0/1194430494 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55904d42de20 msgr2=0x55904d4327e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:54:39.085 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.084+0000 7f7446a00640 1 --2- 172.21.15.72:0/1194430494 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55904d42de20 0x55904d4327e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:39.085 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.084+0000 7f7446a00640 1 -- 172.21.15.72:0/1194430494 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55904d429540 con 0x55904d4282c0 2024-04-26T21:54:39.085 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.084+0000 7f7446a00640 1 --2- 172.21.15.72:0/1194430494 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55904d4282c0 0x55904d41edf0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f7438009900 tx=0x7f743803c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c636199bfac196e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:54:39.085 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.084+0000 7f7437400640 1 -- 172.21.15.72:0/1194430494 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7438055070 con 0x55904d4282c0 2024-04-26T21:54:39.085 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.084+0000 7f7437400640 1 -- 172.21.15.72:0/1194430494 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7438042070 con 0x55904d4282c0 2024-04-26T21:54:39.085 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.085+0000 7f7437400640 1 -- 172.21.15.72:0/1194430494 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7438003850 con 0x55904d4282c0 2024-04-26T21:54:39.089 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.085+0000 7f74507ec740 1 -- 172.21.15.72:0/1194430494 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55904d4282c0 msgr2=0x55904d41edf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:39.089 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.085+0000 7f74507ec740 1 --2- 172.21.15.72:0/1194430494 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55904d4282c0 0x55904d41edf0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f7438009900 tx=0x7f743803c000 comp rx=0 tx=0).stop 2024-04-26T21:54:39.089 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.085+0000 7f74507ec740 1 -- 172.21.15.72:0/1194430494 shutdown_connections 2024-04-26T21:54:39.089 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.085+0000 7f74507ec740 1 --2- 172.21.15.72:0/1194430494 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55904d42de20 0x55904d4327e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:39.089 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.085+0000 7f74507ec740 1 --2- 172.21.15.72:0/1194430494 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55904d428cb0 0x55904d439d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:39.089 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.085+0000 7f74507ec740 1 --2- 172.21.15.72:0/1194430494 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55904d4282c0 0x55904d41edf0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:39.089 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.085+0000 7f74507ec740 1 -- 172.21.15.72:0/1194430494 >> 172.21.15.72:0/1194430494 conn(0x55904d3bcd10 msgr2=0x55904d431380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:54:39.089 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.085+0000 7f74507ec740 1 -- 172.21.15.72:0/1194430494 shutdown_connections 2024-04-26T21:54:39.089 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.085+0000 7f74507ec740 1 -- 172.21.15.72:0/1194430494 wait complete. 2024-04-26T21:54:39.090 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.086+0000 7f74507ec740 1 -- start start 2024-04-26T21:54:39.090 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.086+0000 7f74507ec740 1 Processor -- start 2024-04-26T21:54:39.090 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.086+0000 7f74507ec740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55904d4282c0 0x55904d4358c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:39.090 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.086+0000 7f74507ec740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55904d428cb0 0x55904d435e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:39.090 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.086+0000 7f74507ec740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55904d42de20 0x55904d438dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:39.090 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.086+0000 7f74507ec740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55904d440300 con 0x55904d42de20 2024-04-26T21:54:39.090 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.086+0000 7f74507ec740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55904d43fc30 con 0x55904d428cb0 2024-04-26T21:54:39.090 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.086+0000 7f74507ec740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55904d4409d0 con 0x55904d4282c0 2024-04-26T21:54:39.090 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.086+0000 7f74507ec740 10 client.0 _ll_register_callbacks cb 0x7f7440008b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:54:39.090 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.086+0000 7f7447400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55904d42de20 0x55904d438dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:39.090 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.086+0000 7f7447400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55904d42de20 0x55904d438dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:39298/0 (socket says 172.21.15.72:39298) 2024-04-26T21:54:39.090 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.086+0000 7f7447400640 1 -- 172.21.15.72:0/1460220585 learned_addr learned my addr 172.21.15.72:0/1460220585 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:54:39.090 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.087+0000 7f7446a00640 1 --2- 172.21.15.72:0/1460220585 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55904d4282c0 0x55904d4358c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:39.090 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.087+0000 7f743e000640 1 --2- 172.21.15.72:0/1460220585 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55904d428cb0 0x55904d435e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:39.091 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.087+0000 7f7447400640 1 -- 172.21.15.72:0/1460220585 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55904d4282c0 msgr2=0x55904d4358c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:54:39.091 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.087+0000 7f7447400640 1 --2- 172.21.15.72:0/1460220585 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55904d4282c0 0x55904d4358c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:39.091 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.087+0000 7f7447400640 1 -- 172.21.15.72:0/1460220585 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55904d428cb0 msgr2=0x55904d435e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:54:39.091 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.087+0000 7f7447400640 1 --2- 172.21.15.72:0/1460220585 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55904d428cb0 0x55904d435e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:39.091 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.087+0000 7f7447400640 1 -- 172.21.15.72:0/1460220585 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55904d4393c0 con 0x55904d42de20 2024-04-26T21:54:39.091 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.087+0000 7f743e000640 1 --2- 172.21.15.72:0/1460220585 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55904d428cb0 0x55904d435e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:54:39.091 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.087+0000 7f7447400640 1 --2- 172.21.15.72:0/1460220585 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55904d42de20 0x55904d438dd0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f7440002230 tx=0x7f744000f9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=990f72f1a64b7880 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:54:39.091 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.087+0000 7f7434c00640 10 client.0 ms_handle_connect on v2:172.21.15.72:3300/0 2024-04-26T21:54:39.091 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.087+0000 7f7434c00640 1 -- 172.21.15.72:0/1460220585 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f744001f040 con 0x55904d42de20 2024-04-26T21:54:39.091 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.087+0000 7f74507ec740 1 -- 172.21.15.72:0/1460220585 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7438042070 con 0x55904d42de20 2024-04-26T21:54:39.091 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.087+0000 7f7434c00640 1 -- 172.21.15.72:0/1460220585 <== mon.0 v2:172.21.15.72:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7f7440018690 con 0x55904d42de20 2024-04-26T21:54:39.091 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.088+0000 7f7434c00640 1 -- 172.21.15.72:0/1460220585 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f74400188d0 con 0x55904d42de20 2024-04-26T21:54:39.091 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.088+0000 7f74507ec740 10 client.15114 fetch_fsmap learned FSMap version 26 2024-04-26T21:54:39.091 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.088+0000 7f74507ec740 1 -- 172.21.15.72:0/1460220585 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55904d4a5420 con 0x55904d42de20 2024-04-26T21:54:39.092 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.088+0000 7f7434c00640 1 -- 172.21.15.72:0/1460220585 <== mon.0 v2:172.21.15.72:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7438042070 con 0x55904d42de20 2024-04-26T21:54:39.092 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.088+0000 7f7434c00640 1 -- 172.21.15.72:0/1460220585 <== mon.0 v2:172.21.15.72:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f74400050d0 con 0x55904d42de20 2024-04-26T21:54:39.092 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.088+0000 7f74507ec740 10 client.15114 fetch_fsmap finished waiting for FSMap version 26 2024-04-26T21:54:39.092 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.088+0000 7f74507ec740 10 client.15114 resolve_mds: resolved name 'k' to daemon mds.k 2024-04-26T21:54:39.092 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.088+0000 7f74507ec740 20 client.15114 populate_metadata read hostname 'smithi072' 2024-04-26T21:54:39.092 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.088+0000 7f74507ec740 1 --2- 172.21.15.72:0/1460220585 >> [v2:172.21.15.76:6832/3869483750,v1:172.21.15.76:6833/3869483750] conn(0x55904d4aa080 0x55904d4a8d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:39.092 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.088+0000 7f74507ec740 4 client.15114 mds_command: new command op to 34406 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T21:54:39.092 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.088+0000 7f74507ec740 1 -- 172.21.15.72:0/1460220585 --> [v2:172.21.15.76:6832/3869483750,v1:172.21.15.76:6833/3869483750] -- command(tid 0: {"prefix": "status"}) -- 0x55904d394080 con 0x55904d4aa080 2024-04-26T21:54:39.092 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.089+0000 7f7446a00640 1 --2- 172.21.15.72:0/1460220585 >> [v2:172.21.15.76:6832/3869483750,v1:172.21.15.76:6833/3869483750] conn(0x55904d4aa080 0x55904d4a8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:39.093 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.090+0000 7f7446a00640 1 --2- 172.21.15.72:0/1460220585 >> [v2:172.21.15.76:6832/3869483750,v1:172.21.15.76:6833/3869483750] conn(0x55904d4aa080 0x55904d4a8d90 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=e4efe5acefe3a7fa server_cookie=8c8d17bf53f23792 in_seq=0 out_seq=0 2024-04-26T21:54:39.093 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.090+0000 7f7434c00640 10 client.15114 ms_handle_connect on v2:172.21.15.76:6832/3869483750 2024-04-26T21:54:39.093 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.090+0000 7f7434c00640 1 -- 172.21.15.72:0/1460220585 <== mds.3 v2:172.21.15.76:6832/3869483750 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55904d394080 con 0x55904d4aa080 2024-04-26T21:54:39.094 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.090+0000 7f7434c00640 10 client.15114 handle_command_reply: tid=0 2024-04-26T21:54:39.094 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.090+0000 7f74507ec740 1 client.15114 shutdown 2024-04-26T21:54:39.094 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.091+0000 7f74507ec740 1 -- 172.21.15.72:0/1460220585 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55904d42de20 msgr2=0x55904d438dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:39.094 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.091+0000 7f74507ec740 1 --2- 172.21.15.72:0/1460220585 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55904d42de20 0x55904d438dd0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f7440002230 tx=0x7f744000f9f0 comp rx=0 tx=0).stop 2024-04-26T21:54:39.094 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.091+0000 7f74507ec740 1 -- 172.21.15.72:0/1460220585 shutdown_connections 2024-04-26T21:54:39.094 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.091+0000 7f74507ec740 1 --2- 172.21.15.72:0/1460220585 >> [v2:172.21.15.76:6832/3869483750,v1:172.21.15.76:6833/3869483750] conn(0x55904d4aa080 0x55904d4a8d90 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:39.094 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.091+0000 7f74507ec740 1 --2- 172.21.15.72:0/1460220585 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55904d42de20 0x55904d438dd0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:39.094 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.091+0000 7f74507ec740 1 --2- 172.21.15.72:0/1460220585 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55904d428cb0 0x55904d435e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:39.094 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.091+0000 7f74507ec740 1 --2- 172.21.15.72:0/1460220585 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55904d4282c0 0x55904d4358c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:39.094 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.091+0000 7f7434c00640 0 client.15114 ms_handle_reset on v2:172.21.15.76:6832/3869483750 2024-04-26T21:54:39.094 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.091+0000 7f74507ec740 1 -- 172.21.15.72:0/1460220585 >> 172.21.15.72:0/1460220585 conn(0x55904d3bcd10 msgr2=0x55904d41edf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:54:39.094 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.091+0000 7f74507ec740 1 -- 172.21.15.72:0/1460220585 shutdown_connections 2024-04-26T21:54:39.094 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.091+0000 7f74507ec740 1 -- 172.21.15.72:0/1460220585 wait complete. 2024-04-26T21:54:39.094 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.091+0000 7f74507ec740 20 client.15114 trim_cache size 0 max 16384 2024-04-26T21:54:39.095 INFO:teuthology.orchestra.run.smithi072.stdout:{ 2024-04-26T21:54:39.095 INFO:teuthology.orchestra.run.smithi072.stdout: "cluster_fsid": "a1780950-0415-11ef-bc93-c7b262605968", 2024-04-26T21:54:39.095 INFO:teuthology.orchestra.run.smithi072.stdout: "whoami": 3, 2024-04-26T21:54:39.095 INFO:teuthology.orchestra.run.smithi072.stdout: "id": 34406, 2024-04-26T21:54:39.095 INFO:teuthology.orchestra.run.smithi072.stdout: "want_state": "up:active", 2024-04-26T21:54:39.095 INFO:teuthology.orchestra.run.smithi072.stdout: "state": "up:active", 2024-04-26T21:54:39.095 INFO:teuthology.orchestra.run.smithi072.stdout: "fs_name": "cephfs", 2024-04-26T21:54:39.095 INFO:teuthology.orchestra.run.smithi072.stdout: "rank_uptime": 102.530289739, 2024-04-26T21:54:39.095 INFO:teuthology.orchestra.run.smithi072.stdout: "mdsmap_epoch": 26, 2024-04-26T21:54:39.095 INFO:teuthology.orchestra.run.smithi072.stdout: "osdmap_epoch": 91, 2024-04-26T21:54:39.096 INFO:teuthology.orchestra.run.smithi072.stdout: "osdmap_epoch_barrier": 87, 2024-04-26T21:54:39.096 INFO:teuthology.orchestra.run.smithi072.stdout: "uptime": 154.60931111900001 2024-04-26T21:54:39.096 INFO:teuthology.orchestra.run.smithi072.stdout:} 2024-04-26T21:54:39.096 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.091+0000 7f7447e00640 1 -- 172.21.15.72:0/3791963794 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f742c0a3cc0 msgr2=0x7f742c0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:39.096 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.091+0000 7f7447e00640 1 --2- 172.21.15.72:0/3791963794 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f742c0a3cc0 0x7f742c0a6180 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f743802a930 tx=0x7f7438000bc0 comp rx=0 tx=0).stop 2024-04-26T21:54:39.096 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.091+0000 7f7447e00640 1 -- 172.21.15.72:0/3791963794 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7448101f60 msgr2=0x7f744819b030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:39.096 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.091+0000 7f7447e00640 1 --2- 172.21.15.72:0/3791963794 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7448101f60 0x7f744819b030 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f743000c970 tx=0x7f743000ce30 comp rx=0 tx=0).stop 2024-04-26T21:54:39.097 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.092+0000 7f7447e00640 1 -- 172.21.15.72:0/3791963794 shutdown_connections 2024-04-26T21:54:39.097 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.092+0000 7f7447e00640 1 --2- 172.21.15.72:0/3791963794 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7448102920 0x7f74481a20a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:39.097 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.092+0000 7f7447e00640 1 --2- 172.21.15.72:0/3791963794 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7448101f60 0x7f744819b030 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:39.097 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.092+0000 7f7447e00640 1 --2- 172.21.15.72:0/3791963794 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f742c0a3cc0 0x7f742c0a6180 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:39.097 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.092+0000 7f7447e00640 1 --2- 172.21.15.72:0/3791963794 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7448100e10 0x7f744819aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:39.097 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.092+0000 7f7447e00640 1 -- 172.21.15.72:0/3791963794 >> 172.21.15.72:0/3791963794 conn(0x7f74480fc380 msgr2=0x7f7448107810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:54:39.097 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.092+0000 7f7447e00640 1 -- 172.21.15.72:0/3791963794 shutdown_connections 2024-04-26T21:54:39.097 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.092+0000 7f7447e00640 1 -- 172.21.15.72:0/3791963794 wait complete. 2024-04-26T21:54:39.106 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:54:39 smithi072 ceph-mon[28138]: pgmap v425: 97 pgs: 97 active+clean; 633 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:54:39.106 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:54:39 smithi072 ceph-mon[28138]: from='client.? 172.21.15.72:0/4023616044' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:54:39.106 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:54:39 smithi072 ceph-mon[28138]: from='client.? 172.21.15.72:0/2906267032' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:54:39.108 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.e status 2024-04-26T21:54:39.292 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.292+0000 7fd0f5400640 1 -- 172.21.15.72:0/2084308573 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd0f0105b80 msgr2=0x7fd0f0106000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:39.293 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.292+0000 7fd0f5400640 1 --2- 172.21.15.72:0/2084308573 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd0f0105b80 0x7fd0f0106000 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fd0dc009900 tx=0x7fd0dc02a410 comp rx=0 tx=0).stop 2024-04-26T21:54:39.293 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.292+0000 7fd0f5400640 1 -- 172.21.15.72:0/2084308573 shutdown_connections 2024-04-26T21:54:39.293 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.292+0000 7fd0f5400640 1 --2- 172.21.15.72:0/2084308573 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd0f0106540 0x7fd0f010cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:39.293 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.292+0000 7fd0f5400640 1 --2- 172.21.15.72:0/2084308573 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd0f0105b80 0x7fd0f0106000 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:39.293 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.292+0000 7fd0f5400640 1 --2- 172.21.15.72:0/2084308573 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd0f0104990 0x7fd0f0104db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:39.293 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.292+0000 7fd0f5400640 1 -- 172.21.15.72:0/2084308573 >> 172.21.15.72:0/2084308573 conn(0x7fd0f00fff00 msgr2=0x7fd0f0102340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:54:39.293 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.292+0000 7fd0f5400640 1 -- 172.21.15.72:0/2084308573 shutdown_connections 2024-04-26T21:54:39.293 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.292+0000 7fd0f5400640 1 -- 172.21.15.72:0/2084308573 wait complete. 2024-04-26T21:54:39.295 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.293+0000 7fd0f5400640 1 Processor -- start 2024-04-26T21:54:39.296 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.293+0000 7fd0f5400640 1 -- start start 2024-04-26T21:54:39.296 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.293+0000 7fd0f5400640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd0f0104990 0x7fd0f019aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:39.296 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.293+0000 7fd0f5400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd0f0105b80 0x7fd0f019afc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:39.296 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.293+0000 7fd0f5400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd0f0106540 0x7fd0f01a2030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:39.296 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.293+0000 7fd0f5400640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fd0f01103c0 con 0x7fd0f0106540 2024-04-26T21:54:39.296 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.294+0000 7fd0f5400640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fd0f0110240 con 0x7fd0f0104990 2024-04-26T21:54:39.296 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.294+0000 7fd0f5400640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd0f0110540 con 0x7fd0f0105b80 2024-04-26T21:54:39.296 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.294+0000 7fd0f4a00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd0f0106540 0x7fd0f01a2030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:39.296 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.294+0000 7fd0f4a00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd0f0106540 0x7fd0f01a2030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:39318/0 (socket says 172.21.15.72:39318) 2024-04-26T21:54:39.296 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.294+0000 7fd0f4a00640 1 -- 172.21.15.72:0/1167965658 learned_addr learned my addr 172.21.15.72:0/1167965658 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:54:39.296 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.294+0000 7fd0efe00640 1 --2- 172.21.15.72:0/1167965658 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd0f0104990 0x7fd0f019aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:39.296 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.294+0000 7fd0ef400640 1 --2- 172.21.15.72:0/1167965658 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd0f0105b80 0x7fd0f019afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:39.297 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.294+0000 7fd0f4a00640 1 -- 172.21.15.72:0/1167965658 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd0f0105b80 msgr2=0x7fd0f019afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:39.297 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.294+0000 7fd0f4a00640 1 --2- 172.21.15.72:0/1167965658 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd0f0105b80 0x7fd0f019afc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:39.297 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.294+0000 7fd0f4a00640 1 -- 172.21.15.72:0/1167965658 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd0f0104990 msgr2=0x7fd0f019aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:39.297 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.294+0000 7fd0f4a00640 1 --2- 172.21.15.72:0/1167965658 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd0f0104990 0x7fd0f019aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:39.297 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.294+0000 7fd0f4a00640 1 -- 172.21.15.72:0/1167965658 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0f01a26a0 con 0x7fd0f0106540 2024-04-26T21:54:39.297 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.295+0000 7fd0f4a00640 1 --2- 172.21.15.72:0/1167965658 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd0f0106540 0x7fd0f01a2030 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fd0e800eaa0 tx=0x7fd0e800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:54:39.297 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.295+0000 7fd0ecc00640 1 -- 172.21.15.72:0/1167965658 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0e800cdb0 con 0x7fd0f0106540 2024-04-26T21:54:39.297 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.295+0000 7fd0ecc00640 1 -- 172.21.15.72:0/1167965658 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd0e8005020 con 0x7fd0f0106540 2024-04-26T21:54:39.297 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.295+0000 7fd0f5400640 1 -- 172.21.15.72:0/1167965658 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0f01a2990 con 0x7fd0f0106540 2024-04-26T21:54:39.297 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.295+0000 7fd0ecc00640 1 -- 172.21.15.72:0/1167965658 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0e80116d0 con 0x7fd0f0106540 2024-04-26T21:54:39.297 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.295+0000 7fd0f5400640 1 -- 172.21.15.72:0/1167965658 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0f01a2ee0 con 0x7fd0f0106540 2024-04-26T21:54:39.299 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.299+0000 7fd0f9993740 1 Processor -- start 2024-04-26T21:54:39.300 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.299+0000 7fd0ecc00640 1 -- 172.21.15.72:0/1167965658 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fd0e8004a40 con 0x7fd0f0106540 2024-04-26T21:54:39.300 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.299+0000 7fd0f9993740 1 -- start start 2024-04-26T21:54:39.300 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.300+0000 7fd0f9993740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a74a52a2c0 0x55a74a52a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:39.300 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.300+0000 7fd0f9993740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a74a52acb0 0x55a74a52f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:39.300 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.300+0000 7fd0f9993740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a74a52fe20 0x55a74a5322b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:39.300 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.300+0000 7fd0f9993740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55a74a48e600 con 0x55a74a52acb0 2024-04-26T21:54:39.300 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.300+0000 7fd0f9993740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55a74a42b100 con 0x55a74a52fe20 2024-04-26T21:54:39.300 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.300+0000 7fd0f9993740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55a74a48a5c0 con 0x55a74a52a2c0 2024-04-26T21:54:39.301 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.300+0000 7fd0efe00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a74a52acb0 0x55a74a52f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:39.301 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.300+0000 7fd0efe00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a74a52acb0 0x55a74a52f7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:39332/0 (socket says 172.21.15.72:39332) 2024-04-26T21:54:39.301 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.300+0000 7fd0efe00640 1 -- 172.21.15.72:0/2526681118 learned_addr learned my addr 172.21.15.72:0/2526681118 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:54:39.306 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.300+0000 7fd0efe00640 1 -- 172.21.15.72:0/2526681118 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a74a52a2c0 msgr2=0x55a74a52a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:54:39.306 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.300+0000 7fd0efe00640 1 --2- 172.21.15.72:0/2526681118 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a74a52a2c0 0x55a74a52a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:39.306 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.300+0000 7fd0efe00640 1 -- 172.21.15.72:0/2526681118 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a74a52fe20 msgr2=0x55a74a5322b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:54:39.306 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.300+0000 7fd0efe00640 1 --2- 172.21.15.72:0/2526681118 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a74a52fe20 0x55a74a5322b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:39.306 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.300+0000 7fd0efe00640 1 -- 172.21.15.72:0/2526681118 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a74a1a02d0 con 0x55a74a52acb0 2024-04-26T21:54:39.306 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.301+0000 7fd0efe00640 1 --2- 172.21.15.72:0/2526681118 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a74a52acb0 0x55a74a52f7b0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fd0e000bc90 tx=0x7fd0e000f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85c91ce715eee48c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:54:39.306 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.301+0000 7fd0e5200640 1 -- 172.21.15.72:0/2526681118 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0e0015070 con 0x55a74a52acb0 2024-04-26T21:54:39.306 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.301+0000 7fd0e5200640 1 -- 172.21.15.72:0/2526681118 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd0e00035f0 con 0x55a74a52acb0 2024-04-26T21:54:39.307 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.301+0000 7fd0e5200640 1 -- 172.21.15.72:0/2526681118 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0e0011640 con 0x55a74a52acb0 2024-04-26T21:54:39.307 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.301+0000 7fd0ecc00640 1 --2- 172.21.15.72:0/1167965658 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fd0d40a3c40 0x7fd0d40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:39.307 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.301+0000 7fd0ecc00640 1 -- 172.21.15.72:0/1167965658 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7fd0e80d0890 con 0x7fd0f0106540 2024-04-26T21:54:39.307 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.301+0000 7fd0ef400640 1 --2- 172.21.15.72:0/1167965658 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fd0d40a3c40 0x7fd0d40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:39.307 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.302+0000 7fd0f9993740 1 -- 172.21.15.72:0/2526681118 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a74a52acb0 msgr2=0x55a74a52f7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:39.307 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.302+0000 7fd0f9993740 1 --2- 172.21.15.72:0/2526681118 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a74a52acb0 0x55a74a52f7b0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fd0e000bc90 tx=0x7fd0e000f730 comp rx=0 tx=0).stop 2024-04-26T21:54:39.307 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.302+0000 7fd0f9993740 1 -- 172.21.15.72:0/2526681118 shutdown_connections 2024-04-26T21:54:39.307 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.302+0000 7fd0f9993740 1 --2- 172.21.15.72:0/2526681118 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a74a52fe20 0x55a74a5322b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:39.307 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.302+0000 7fd0f9993740 1 --2- 172.21.15.72:0/2526681118 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a74a52acb0 0x55a74a52f7b0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:39.307 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.302+0000 7fd0f9993740 1 --2- 172.21.15.72:0/2526681118 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a74a52a2c0 0x55a74a52a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:39.307 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.302+0000 7fd0f9993740 1 -- 172.21.15.72:0/2526681118 >> 172.21.15.72:0/2526681118 conn(0x55a74a4bed10 msgr2=0x55a74a525d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:54:39.307 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.302+0000 7fd0f9993740 1 -- 172.21.15.72:0/2526681118 shutdown_connections 2024-04-26T21:54:39.307 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.302+0000 7fd0f9993740 1 -- 172.21.15.72:0/2526681118 wait complete. 2024-04-26T21:54:39.308 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.302+0000 7fd0ef400640 1 --2- 172.21.15.72:0/1167965658 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fd0d40a3c40 0x7fd0d40a6100 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x55a74a52b510 tx=0x7fd0dc031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:54:39.308 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.302+0000 7fd0f9993740 1 -- start start 2024-04-26T21:54:39.308 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.303+0000 7fd0f9993740 1 Processor -- start 2024-04-26T21:54:39.308 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.303+0000 7fd0f9993740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a74a52a2c0 0x55a74a534780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:39.308 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.303+0000 7fd0f9993740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a74a52acb0 0x55a74a534cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:39.308 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.303+0000 7fd0f9993740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a74a52fe20 0x55a74a53bd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:39.308 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.303+0000 7fd0f9993740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55a74a532dd0 con 0x55a74a52fe20 2024-04-26T21:54:39.308 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.303+0000 7fd0f9993740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55a74a415f70 con 0x55a74a52acb0 2024-04-26T21:54:39.308 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.303+0000 7fd0f9993740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55a74a532f50 con 0x55a74a52a2c0 2024-04-26T21:54:39.308 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.303+0000 7fd0f9993740 10 client.0 _ll_register_callbacks cb 0x55a74a484480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:54:39.308 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.303+0000 7fd0f4a00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a74a52fe20 0x55a74a53bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:39.308 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.303+0000 7fd0f4a00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a74a52fe20 0x55a74a53bd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:39334/0 (socket says 172.21.15.72:39334) 2024-04-26T21:54:39.308 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.303+0000 7fd0f4a00640 1 -- 172.21.15.72:0/2064289088 learned_addr learned my addr 172.21.15.72:0/2064289088 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:54:39.308 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.304+0000 7fd0efe00640 1 --2- 172.21.15.72:0/2064289088 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a74a52a2c0 0x55a74a534780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:39.309 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.304+0000 7fd0ef400640 1 --2- 172.21.15.72:0/2064289088 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a74a52acb0 0x55a74a534cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:39.309 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.304+0000 7fd0f4a00640 1 -- 172.21.15.72:0/2064289088 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a74a52a2c0 msgr2=0x55a74a534780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:54:39.309 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.304+0000 7fd0f4a00640 1 --2- 172.21.15.72:0/2064289088 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a74a52a2c0 0x55a74a534780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:39.309 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.304+0000 7fd0f4a00640 1 -- 172.21.15.72:0/2064289088 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a74a52acb0 msgr2=0x55a74a534cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:54:39.309 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.304+0000 7fd0f4a00640 1 --2- 172.21.15.72:0/2064289088 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a74a52acb0 0x55a74a534cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:39.309 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.304+0000 7fd0f4a00640 1 -- 172.21.15.72:0/2064289088 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a74a26a6b0 con 0x55a74a52fe20 2024-04-26T21:54:39.309 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.304+0000 7fd0ef400640 1 --2- 172.21.15.72:0/2064289088 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a74a52acb0 0x55a74a534cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:54:39.309 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.304+0000 7fd0f4a00640 1 --2- 172.21.15.72:0/2064289088 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a74a52fe20 0x55a74a53bd00 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fd0e80a5070 tx=0x7fd0e809d170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adafec9d3af17bbb server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:54:39.309 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.304+0000 7fd0d9600640 10 client.0 ms_handle_connect on v2:172.21.15.72:3300/0 2024-04-26T21:54:39.309 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.304+0000 7fd0d9600640 1 -- 172.21.15.72:0/2064289088 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0e801f040 con 0x55a74a52fe20 2024-04-26T21:54:39.309 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.304+0000 7fd0d9600640 1 -- 172.21.15.72:0/2064289088 <== mon.0 v2:172.21.15.72:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7fd0e80ce3b0 con 0x55a74a52fe20 2024-04-26T21:54:39.309 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.304+0000 7fd0f9993740 1 -- 172.21.15.72:0/2064289088 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd0e00035f0 con 0x55a74a52fe20 2024-04-26T21:54:39.309 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.305+0000 7fd0d9600640 1 -- 172.21.15.72:0/2064289088 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fd0e80b9070 con 0x55a74a52fe20 2024-04-26T21:54:39.309 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.305+0000 7fd0f9993740 10 client.15126 fetch_fsmap learned FSMap version 26 2024-04-26T21:54:39.310 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.305+0000 7fd0f9993740 1 -- 172.21.15.72:0/2064289088 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55a74a53edb0 con 0x55a74a52fe20 2024-04-26T21:54:39.310 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.305+0000 7fd0d9600640 1 -- 172.21.15.72:0/2064289088 <== mon.0 v2:172.21.15.72:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd0e00035f0 con 0x55a74a52fe20 2024-04-26T21:54:39.310 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.305+0000 7fd0d9600640 1 -- 172.21.15.72:0/2064289088 <== mon.0 v2:172.21.15.72:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fd0e800f040 con 0x55a74a52fe20 2024-04-26T21:54:39.310 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.305+0000 7fd0f9993740 10 client.15126 fetch_fsmap finished waiting for FSMap version 26 2024-04-26T21:54:39.310 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.305+0000 7fd0f9993740 10 client.15126 resolve_mds: resolved name 'e' to daemon mds.e 2024-04-26T21:54:39.310 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.305+0000 7fd0f9993740 20 client.15126 populate_metadata read hostname 'smithi072' 2024-04-26T21:54:39.310 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.305+0000 7fd0f9993740 1 --2- 172.21.15.72:0/2064289088 >> [v2:172.21.15.76:6834/3517429435,v1:172.21.15.76:6835/3517429435] conn(0x55a74a541460 0x55a74a5428d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:39.310 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.305+0000 7fd0f9993740 4 client.15126 mds_command: new command op to 34442 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:54:39.310 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.305+0000 7fd0f9993740 1 -- 172.21.15.72:0/2064289088 --> [v2:172.21.15.76:6834/3517429435,v1:172.21.15.76:6835/3517429435] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a74a49c210 con 0x55a74a541460 2024-04-26T21:54:39.310 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.306+0000 7fd0efe00640 1 --2- 172.21.15.72:0/2064289088 >> [v2:172.21.15.76:6834/3517429435,v1:172.21.15.76:6835/3517429435] conn(0x55a74a541460 0x55a74a5428d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:39.311 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.307+0000 7fd0efe00640 1 --2- 172.21.15.72:0/2064289088 >> [v2:172.21.15.76:6834/3517429435,v1:172.21.15.76:6835/3517429435] conn(0x55a74a541460 0x55a74a5428d0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c79f60a14287aeca server_cookie=4a354b64b8fd8b44 in_seq=0 out_seq=0 2024-04-26T21:54:39.312 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.307+0000 7fd0d9600640 10 client.15126 ms_handle_connect on v2:172.21.15.76:6834/3517429435 2024-04-26T21:54:39.312 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.309+0000 7fd0d9600640 1 -- 172.21.15.72:0/2064289088 <== mds.2 v2:172.21.15.76:6834/3517429435 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a74a49c210 con 0x55a74a541460 2024-04-26T21:54:39.312 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.309+0000 7fd0d9600640 10 client.15126 handle_command_reply: tid=0 2024-04-26T21:54:39.312 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.309+0000 7fd0f9993740 1 client.15126 shutdown 2024-04-26T21:54:39.312 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.309+0000 7fd0f9993740 1 -- 172.21.15.72:0/2064289088 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a74a52fe20 msgr2=0x55a74a53bd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:39.312 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.309+0000 7fd0f9993740 1 --2- 172.21.15.72:0/2064289088 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a74a52fe20 0x55a74a53bd00 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fd0e80a5070 tx=0x7fd0e809d170 comp rx=0 tx=0).stop 2024-04-26T21:54:39.312 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.309+0000 7fd0f9993740 1 -- 172.21.15.72:0/2064289088 shutdown_connections 2024-04-26T21:54:39.312 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.309+0000 7fd0f9993740 1 --2- 172.21.15.72:0/2064289088 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a74a52fe20 0x55a74a53bd00 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:39.312 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.309+0000 7fd0f9993740 1 --2- 172.21.15.72:0/2064289088 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a74a52acb0 0x55a74a534cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:39.312 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.309+0000 7fd0f9993740 1 --2- 172.21.15.72:0/2064289088 >> [v2:172.21.15.76:6834/3517429435,v1:172.21.15.76:6835/3517429435] conn(0x55a74a541460 0x55a74a5428d0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:39.312 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.309+0000 7fd0f9993740 1 --2- 172.21.15.72:0/2064289088 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a74a52a2c0 0x55a74a534780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:39.312 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.309+0000 7fd0f9993740 1 -- 172.21.15.72:0/2064289088 >> 172.21.15.72:0/2064289088 conn(0x55a74a4bed10 msgr2=0x55a74a524f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:54:39.312 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.309+0000 7fd0d9600640 0 client.15126 ms_handle_reset on v2:172.21.15.76:6834/3517429435 2024-04-26T21:54:39.313 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.309+0000 7fd0f9993740 1 -- 172.21.15.72:0/2064289088 shutdown_connections 2024-04-26T21:54:39.313 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.309+0000 7fd0f9993740 1 -- 172.21.15.72:0/2064289088 wait complete. 2024-04-26T21:54:39.313 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.309+0000 7fd0f9993740 20 client.15126 trim_cache size 0 max 16384 2024-04-26T21:54:39.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:54:39 smithi076 ceph-mon[31823]: pgmap v425: 97 pgs: 97 active+clean; 633 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:54:39.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:54:39 smithi076 ceph-mon[31823]: from='client.? 172.21.15.72:0/4023616044' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:54:39.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:54:39 smithi076 ceph-mon[31823]: from='client.? 172.21.15.72:0/2906267032' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:54:39.328 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.328+0000 7fd0f9993740 1 Processor -- start 2024-04-26T21:54:39.328 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.328+0000 7fd0f9993740 1 -- start start 2024-04-26T21:54:39.328 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.328+0000 7fd0f9993740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a74a52a2c0 0x55a74a533350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:39.328 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.328+0000 7fd0f9993740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a74a52acb0 0x55a74a53bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:39.328 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.328+0000 7fd0f9993740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a74a52fe20 0x55a74a533a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:39.328 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.328+0000 7fd0f9993740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55a74a540c10 con 0x55a74a52a2c0 2024-04-26T21:54:39.329 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.328+0000 7fd0f9993740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55a74a5412e0 con 0x55a74a52acb0 2024-04-26T21:54:39.329 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.328+0000 7fd0f9993740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55a74a5419b0 con 0x55a74a52fe20 2024-04-26T21:54:39.331 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.329+0000 7fd0efe00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a74a52a2c0 0x55a74a533350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:39.331 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.329+0000 7fd0efe00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a74a52a2c0 0x55a74a533350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:39346/0 (socket says 172.21.15.72:39346) 2024-04-26T21:54:39.331 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.329+0000 7fd0efe00640 1 -- 172.21.15.72:0/2523540962 learned_addr learned my addr 172.21.15.72:0/2523540962 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:54:39.331 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.329+0000 7fd0f4a00640 1 --2- 172.21.15.72:0/2523540962 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a74a52fe20 0x55a74a533a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:39.331 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.329+0000 7fd0ef400640 1 --2- 172.21.15.72:0/2523540962 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a74a52acb0 0x55a74a53bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:39.331 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.329+0000 7fd0efe00640 1 -- 172.21.15.72:0/2523540962 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a74a52fe20 msgr2=0x55a74a533a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:54:39.331 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.329+0000 7fd0efe00640 1 --2- 172.21.15.72:0/2523540962 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a74a52fe20 0x55a74a533a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:39.331 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.329+0000 7fd0efe00640 1 -- 172.21.15.72:0/2523540962 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a74a52acb0 msgr2=0x55a74a53bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:54:39.331 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.329+0000 7fd0efe00640 1 --2- 172.21.15.72:0/2523540962 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a74a52acb0 0x55a74a53bcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:39.331 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.329+0000 7fd0efe00640 1 -- 172.21.15.72:0/2523540962 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a74a52b540 con 0x55a74a52a2c0 2024-04-26T21:54:39.331 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.329+0000 7fd0ef400640 1 --2- 172.21.15.72:0/2523540962 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a74a52acb0 0x55a74a53bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:54:39.331 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.329+0000 7fd0efe00640 1 --2- 172.21.15.72:0/2523540962 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a74a52a2c0 0x55a74a533350 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fd0e000bc60 tx=0x7fd0e000b580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=446a6b7e3bc5d932 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:54:39.332 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.329+0000 7fd0dbe00640 1 -- 172.21.15.72:0/2523540962 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0e0015050 con 0x55a74a52a2c0 2024-04-26T21:54:39.332 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.330+0000 7fd0dbe00640 1 -- 172.21.15.72:0/2523540962 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd0e00035f0 con 0x55a74a52a2c0 2024-04-26T21:54:39.332 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.330+0000 7fd0dbe00640 1 -- 172.21.15.72:0/2523540962 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0e0011440 con 0x55a74a52a2c0 2024-04-26T21:54:39.332 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.330+0000 7fd0f9993740 1 -- 172.21.15.72:0/2523540962 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a74a52a2c0 msgr2=0x55a74a533350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:39.332 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.330+0000 7fd0f9993740 1 --2- 172.21.15.72:0/2523540962 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a74a52a2c0 0x55a74a533350 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fd0e000bc60 tx=0x7fd0e000b580 comp rx=0 tx=0).stop 2024-04-26T21:54:39.332 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.330+0000 7fd0f9993740 1 -- 172.21.15.72:0/2523540962 shutdown_connections 2024-04-26T21:54:39.332 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.330+0000 7fd0f9993740 1 --2- 172.21.15.72:0/2523540962 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a74a52fe20 0x55a74a533a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:39.332 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.330+0000 7fd0f9993740 1 --2- 172.21.15.72:0/2523540962 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a74a52acb0 0x55a74a53bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:39.332 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.330+0000 7fd0f9993740 1 --2- 172.21.15.72:0/2523540962 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a74a52a2c0 0x55a74a533350 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:39.332 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.330+0000 7fd0f9993740 1 -- 172.21.15.72:0/2523540962 >> 172.21.15.72:0/2523540962 conn(0x55a74a4bed10 msgr2=0x55a74a52c110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:54:39.332 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.330+0000 7fd0f9993740 1 -- 172.21.15.72:0/2523540962 shutdown_connections 2024-04-26T21:54:39.332 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.330+0000 7fd0f9993740 1 -- 172.21.15.72:0/2523540962 wait complete. 2024-04-26T21:54:39.332 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.331+0000 7fd0f9993740 1 -- start start 2024-04-26T21:54:39.332 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.331+0000 7fd0f9993740 1 Processor -- start 2024-04-26T21:54:39.335 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.331+0000 7fd0f9993740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a74a52a2c0 0x55a74a537590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:39.335 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.331+0000 7fd0f9993740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a74a52acb0 0x55a74a537ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:39.335 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.331+0000 7fd0f9993740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a74a52fe20 0x55a74a53aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:39.335 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.331+0000 7fd0f9993740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55a74a542750 con 0x55a74a52fe20 2024-04-26T21:54:39.335 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.331+0000 7fd0f9993740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55a74a542080 con 0x55a74a52acb0 2024-04-26T21:54:39.335 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.331+0000 7fd0f9993740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55a74a542e20 con 0x55a74a52a2c0 2024-04-26T21:54:39.336 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.331+0000 7fd0f9993740 10 client.0 _ll_register_callbacks cb 0x7fd0e8005a90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:54:39.336 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.331+0000 7fd0f4a00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a74a52fe20 0x55a74a53aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:39.336 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.332+0000 7fd0f4a00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a74a52fe20 0x55a74a53aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:39350/0 (socket says 172.21.15.72:39350) 2024-04-26T21:54:39.336 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.332+0000 7fd0f4a00640 1 -- 172.21.15.72:0/3001233251 learned_addr learned my addr 172.21.15.72:0/3001233251 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:54:39.336 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.332+0000 7fd0efe00640 1 --2- 172.21.15.72:0/3001233251 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a74a52a2c0 0x55a74a537590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:39.336 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.332+0000 7fd0f4a00640 1 -- 172.21.15.72:0/3001233251 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a74a52a2c0 msgr2=0x55a74a537590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:54:39.336 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.332+0000 7fd0f4a00640 1 --2- 172.21.15.72:0/3001233251 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a74a52a2c0 0x55a74a537590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:39.336 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.332+0000 7fd0f4a00640 1 -- 172.21.15.72:0/3001233251 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a74a52acb0 msgr2=0x55a74a537ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:54:39.336 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.332+0000 7fd0ef400640 1 --2- 172.21.15.72:0/3001233251 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a74a52acb0 0x55a74a537ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:39.336 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.332+0000 7fd0f4a00640 1 --2- 172.21.15.72:0/3001233251 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a74a52acb0 0x55a74a537ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:39.336 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.332+0000 7fd0f4a00640 1 -- 172.21.15.72:0/3001233251 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a74a53b070 con 0x55a74a52fe20 2024-04-26T21:54:39.337 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.332+0000 7fd0f4a00640 1 --2- 172.21.15.72:0/3001233251 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a74a52fe20 0x55a74a53aa80 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fd0e8099300 tx=0x7fd0e809c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ebfc0a4e3047caa server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:54:39.337 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.332+0000 7fd0d9600640 10 client.0 ms_handle_connect on v2:172.21.15.72:3300/0 2024-04-26T21:54:39.337 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.332+0000 7fd0d9600640 1 -- 172.21.15.72:0/3001233251 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0e801f070 con 0x55a74a52fe20 2024-04-26T21:54:39.337 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.332+0000 7fd0d9600640 1 -- 172.21.15.72:0/3001233251 <== mon.0 v2:172.21.15.72:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7fd0e8004d30 con 0x55a74a52fe20 2024-04-26T21:54:39.337 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.332+0000 7fd0f9993740 1 -- 172.21.15.72:0/3001233251 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd0e00035f0 con 0x55a74a52fe20 2024-04-26T21:54:39.337 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.333+0000 7fd0d9600640 1 -- 172.21.15.72:0/3001233251 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fd0e8080070 con 0x55a74a52fe20 2024-04-26T21:54:39.337 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.333+0000 7fd0f9993740 10 client.15132 fetch_fsmap learned FSMap version 26 2024-04-26T21:54:39.337 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.333+0000 7fd0f9993740 1 -- 172.21.15.72:0/3001233251 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55a74a5a74d0 con 0x55a74a52fe20 2024-04-26T21:54:39.337 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.333+0000 7fd0d9600640 1 -- 172.21.15.72:0/3001233251 <== mon.0 v2:172.21.15.72:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd0e00035f0 con 0x55a74a52fe20 2024-04-26T21:54:39.337 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.333+0000 7fd0d9600640 1 -- 172.21.15.72:0/3001233251 <== mon.0 v2:172.21.15.72:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fd0e8098680 con 0x55a74a52fe20 2024-04-26T21:54:39.337 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.333+0000 7fd0f9993740 10 client.15132 fetch_fsmap finished waiting for FSMap version 26 2024-04-26T21:54:39.337 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.333+0000 7fd0f9993740 10 client.15132 resolve_mds: resolved name 'e' to daemon mds.e 2024-04-26T21:54:39.337 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.333+0000 7fd0f9993740 20 client.15132 populate_metadata read hostname 'smithi072' 2024-04-26T21:54:39.338 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.333+0000 7fd0f9993740 1 --2- 172.21.15.72:0/3001233251 >> [v2:172.21.15.76:6834/3517429435,v1:172.21.15.76:6835/3517429435] conn(0x55a74a5ac130 0x55a74a5aae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:39.338 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.333+0000 7fd0f9993740 4 client.15132 mds_command: new command op to 34442 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T21:54:39.338 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.333+0000 7fd0f9993740 1 -- 172.21.15.72:0/3001233251 --> [v2:172.21.15.76:6834/3517429435,v1:172.21.15.76:6835/3517429435] -- command(tid 0: {"prefix": "status"}) -- 0x55a74a496080 con 0x55a74a5ac130 2024-04-26T21:54:39.338 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.334+0000 7fd0efe00640 1 --2- 172.21.15.72:0/3001233251 >> [v2:172.21.15.76:6834/3517429435,v1:172.21.15.76:6835/3517429435] conn(0x55a74a5ac130 0x55a74a5aae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:39.338 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.335+0000 7fd0d9600640 10 client.15132 ms_handle_connect on v2:172.21.15.76:6834/3517429435 2024-04-26T21:54:39.338 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.335+0000 7fd0efe00640 1 --2- 172.21.15.72:0/3001233251 >> [v2:172.21.15.76:6834/3517429435,v1:172.21.15.76:6835/3517429435] conn(0x55a74a5ac130 0x55a74a5aae40 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=caa11ca2843ccecf server_cookie=c42dbcd09cda9f7d in_seq=0 out_seq=0 2024-04-26T21:54:39.339 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.335+0000 7fd0d9600640 1 -- 172.21.15.72:0/3001233251 <== mds.2 v2:172.21.15.76:6834/3517429435 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55a74a496080 con 0x55a74a5ac130 2024-04-26T21:54:39.339 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.335+0000 7fd0d9600640 10 client.15132 handle_command_reply: tid=0 2024-04-26T21:54:39.339 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.335+0000 7fd0f9993740 1 client.15132 shutdown 2024-04-26T21:54:39.339 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.336+0000 7fd0f9993740 1 -- 172.21.15.72:0/3001233251 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a74a52fe20 msgr2=0x55a74a53aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:39.339 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.336+0000 7fd0f9993740 1 --2- 172.21.15.72:0/3001233251 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a74a52fe20 0x55a74a53aa80 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fd0e8099300 tx=0x7fd0e809c4e0 comp rx=0 tx=0).stop 2024-04-26T21:54:39.340 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.336+0000 7fd0f9993740 1 -- 172.21.15.72:0/3001233251 shutdown_connections 2024-04-26T21:54:39.340 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.336+0000 7fd0f9993740 1 --2- 172.21.15.72:0/3001233251 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a74a52fe20 0x55a74a53aa80 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:39.340 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.336+0000 7fd0f9993740 1 --2- 172.21.15.72:0/3001233251 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a74a52acb0 0x55a74a537ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:39.340 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.336+0000 7fd0f9993740 1 --2- 172.21.15.72:0/3001233251 >> [v2:172.21.15.76:6834/3517429435,v1:172.21.15.76:6835/3517429435] conn(0x55a74a5ac130 0x55a74a5aae40 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:39.340 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.336+0000 7fd0f9993740 1 --2- 172.21.15.72:0/3001233251 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a74a52a2c0 0x55a74a537590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:39.340 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.336+0000 7fd0f9993740 1 -- 172.21.15.72:0/3001233251 >> 172.21.15.72:0/3001233251 conn(0x55a74a4bed10 msgr2=0x55a74a533a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:54:39.340 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.336+0000 7fd0d9600640 0 client.15132 ms_handle_reset on v2:172.21.15.76:6834/3517429435 2024-04-26T21:54:39.340 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.336+0000 7fd0f9993740 1 -- 172.21.15.72:0/3001233251 shutdown_connections 2024-04-26T21:54:39.340 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.336+0000 7fd0f9993740 1 -- 172.21.15.72:0/3001233251 wait complete. 2024-04-26T21:54:39.340 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.336+0000 7fd0f9993740 20 client.15132 trim_cache size 0 max 16384 2024-04-26T21:54:39.340 INFO:teuthology.orchestra.run.smithi072.stdout:{ 2024-04-26T21:54:39.340 INFO:teuthology.orchestra.run.smithi072.stdout: "cluster_fsid": "a1780950-0415-11ef-bc93-c7b262605968", 2024-04-26T21:54:39.341 INFO:teuthology.orchestra.run.smithi072.stdout: "whoami": 2, 2024-04-26T21:54:39.341 INFO:teuthology.orchestra.run.smithi072.stdout: "id": 34442, 2024-04-26T21:54:39.341 INFO:teuthology.orchestra.run.smithi072.stdout: "want_state": "up:active", 2024-04-26T21:54:39.341 INFO:teuthology.orchestra.run.smithi072.stdout: "state": "up:active", 2024-04-26T21:54:39.341 INFO:teuthology.orchestra.run.smithi072.stdout: "fs_name": "cephfs", 2024-04-26T21:54:39.341 INFO:teuthology.orchestra.run.smithi072.stdout: "rank_uptime": 103.79634716299999, 2024-04-26T21:54:39.341 INFO:teuthology.orchestra.run.smithi072.stdout: "mdsmap_epoch": 26, 2024-04-26T21:54:39.341 INFO:teuthology.orchestra.run.smithi072.stdout: "osdmap_epoch": 91, 2024-04-26T21:54:39.341 INFO:teuthology.orchestra.run.smithi072.stdout: "osdmap_epoch_barrier": 87, 2024-04-26T21:54:39.341 INFO:teuthology.orchestra.run.smithi072.stdout: "uptime": 142.832756077 2024-04-26T21:54:39.341 INFO:teuthology.orchestra.run.smithi072.stdout:} 2024-04-26T21:54:39.342 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.336+0000 7fd0f5400640 1 -- 172.21.15.72:0/1167965658 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fd0d40a3c40 msgr2=0x7fd0d40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:39.342 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.336+0000 7fd0f5400640 1 --2- 172.21.15.72:0/1167965658 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fd0d40a3c40 0x7fd0d40a6100 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x55a74a52b510 tx=0x7fd0dc031400 comp rx=0 tx=0).stop 2024-04-26T21:54:39.342 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.336+0000 7fd0f5400640 1 -- 172.21.15.72:0/1167965658 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd0f0106540 msgr2=0x7fd0f01a2030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:39.342 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.336+0000 7fd0f5400640 1 --2- 172.21.15.72:0/1167965658 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd0f0106540 0x7fd0f01a2030 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fd0e800eaa0 tx=0x7fd0e800ef60 comp rx=0 tx=0).stop 2024-04-26T21:54:39.342 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.337+0000 7fd0f5400640 1 -- 172.21.15.72:0/1167965658 shutdown_connections 2024-04-26T21:54:39.342 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.337+0000 7fd0f5400640 1 --2- 172.21.15.72:0/1167965658 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd0f0106540 0x7fd0f01a2030 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:39.342 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.337+0000 7fd0f5400640 1 --2- 172.21.15.72:0/1167965658 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fd0d40a3c40 0x7fd0d40a6100 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:39.343 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.337+0000 7fd0f5400640 1 --2- 172.21.15.72:0/1167965658 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd0f0105b80 0x7fd0f019afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:39.343 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.337+0000 7fd0f5400640 1 --2- 172.21.15.72:0/1167965658 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd0f0104990 0x7fd0f019aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:39.343 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.337+0000 7fd0f5400640 1 -- 172.21.15.72:0/1167965658 >> 172.21.15.72:0/1167965658 conn(0x7fd0f00fff00 msgr2=0x7fd0f0108f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:54:39.343 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.337+0000 7fd0f5400640 1 -- 172.21.15.72:0/1167965658 shutdown_connections 2024-04-26T21:54:39.343 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.337+0000 7fd0f5400640 1 -- 172.21.15.72:0/1167965658 wait complete. 2024-04-26T21:54:39.353 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2024-04-26T21:54:39.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:54:39 smithi164 ceph-mon[31916]: pgmap v425: 97 pgs: 97 active+clean; 633 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:54:39.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:54:39 smithi164 ceph-mon[31916]: from='client.? 172.21.15.72:0/4023616044' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:54:39.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:54:39 smithi164 ceph-mon[31916]: from='client.? 172.21.15.72:0/2906267032' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:54:39.542 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.541+0000 7f499be00640 1 -- 172.21.15.72:0/2964004808 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f49940768f0 msgr2=0x7f4994074d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:39.542 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.541+0000 7f499be00640 1 --2- 172.21.15.72:0/2964004808 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f49940768f0 0x7f4994074d40 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f4988009900 tx=0x7f498802a480 comp rx=0 tx=0).stop 2024-04-26T21:54:39.542 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.541+0000 7f499be00640 1 -- 172.21.15.72:0/2964004808 shutdown_connections 2024-04-26T21:54:39.542 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.541+0000 7f499be00640 1 --2- 172.21.15.72:0/2964004808 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f499410a820 0x7f499410ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:39.542 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.541+0000 7f499be00640 1 --2- 172.21.15.72:0/2964004808 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4994075310 0x7f4994075770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:39.542 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.541+0000 7f499be00640 1 --2- 172.21.15.72:0/2964004808 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f49940768f0 0x7f4994074d40 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:39.542 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.541+0000 7f499be00640 1 -- 172.21.15.72:0/2964004808 >> 172.21.15.72:0/2964004808 conn(0x7f49940fdee0 msgr2=0x7f4994100320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:54:39.542 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.541+0000 7f499be00640 1 -- 172.21.15.72:0/2964004808 shutdown_connections 2024-04-26T21:54:39.543 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.541+0000 7f499be00640 1 -- 172.21.15.72:0/2964004808 wait complete. 2024-04-26T21:54:39.543 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.542+0000 7f499be00640 1 Processor -- start 2024-04-26T21:54:39.545 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.542+0000 7f499be00640 1 -- start start 2024-04-26T21:54:39.545 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.542+0000 7f499be00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4994075310 0x7f499419abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:39.545 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.543+0000 7f499be00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f49940768f0 0x7f499419b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:39.545 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.543+0000 7f499be00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f499410a820 0x7f49941a21a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:39.545 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.543+0000 7f499be00640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f4994110550 con 0x7f499410a820 2024-04-26T21:54:39.545 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.543+0000 7f499be00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f49941103d0 con 0x7f4994075310 2024-04-26T21:54:39.545 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.543+0000 7f499be00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f49941106d0 con 0x7f49940768f0 2024-04-26T21:54:39.545 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.543+0000 7f4999e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f499410a820 0x7f49941a21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:39.545 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.543+0000 7f4999e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f499410a820 0x7f49941a21a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:39380/0 (socket says 172.21.15.72:39380) 2024-04-26T21:54:39.545 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.543+0000 7f4999e00640 1 -- 172.21.15.72:0/1866600812 learned_addr learned my addr 172.21.15.72:0/1866600812 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:54:39.545 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.543+0000 7f4998a00640 1 --2- 172.21.15.72:0/1866600812 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f49940768f0 0x7f499419b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:39.545 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.543+0000 7f4999400640 1 --2- 172.21.15.72:0/1866600812 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4994075310 0x7f499419abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:39.546 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.543+0000 7f4999e00640 1 -- 172.21.15.72:0/1866600812 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f49940768f0 msgr2=0x7f499419b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:39.546 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.543+0000 7f4999e00640 1 --2- 172.21.15.72:0/1866600812 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f49940768f0 0x7f499419b130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:39.546 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.543+0000 7f4999e00640 1 -- 172.21.15.72:0/1866600812 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4994075310 msgr2=0x7f499419abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:39.546 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.543+0000 7f4999e00640 1 --2- 172.21.15.72:0/1866600812 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4994075310 0x7f499419abf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:39.546 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.543+0000 7f4999e00640 1 -- 172.21.15.72:0/1866600812 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49941a2810 con 0x7f499410a820 2024-04-26T21:54:39.546 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.543+0000 7f4999e00640 1 --2- 172.21.15.72:0/1866600812 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f499410a820 0x7f49941a21a0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f498c00eaa0 tx=0x7f498c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:54:39.546 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.544+0000 7f4992000640 1 -- 172.21.15.72:0/1866600812 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f498c00cdb0 con 0x7f499410a820 2024-04-26T21:54:39.546 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.544+0000 7f4992000640 1 -- 172.21.15.72:0/1866600812 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f498c005020 con 0x7f499410a820 2024-04-26T21:54:39.546 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.544+0000 7f4992000640 1 -- 172.21.15.72:0/1866600812 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f498c0116d0 con 0x7f499410a820 2024-04-26T21:54:39.546 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.544+0000 7f499be00640 1 -- 172.21.15.72:0/1866600812 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49941a2b00 con 0x7f499410a820 2024-04-26T21:54:39.546 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.544+0000 7f499be00640 1 -- 172.21.15.72:0/1866600812 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49941a3000 con 0x7f499410a820 2024-04-26T21:54:39.549 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.547+0000 7f4992000640 1 -- 172.21.15.72:0/1866600812 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f498c0051d0 con 0x7f499410a820 2024-04-26T21:54:39.549 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.547+0000 7f499eba6740 1 Processor -- start 2024-04-26T21:54:39.549 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.547+0000 7f499eba6740 1 -- start start 2024-04-26T21:54:39.549 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.547+0000 7f499eba6740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557e9d9bc0f0 0x557e9d9bc510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:39.549 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.547+0000 7f499eba6740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557e9d9bcae0 0x557e9d9c15e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:39.549 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.547+0000 7f499eba6740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557e9d9c1c50 0x557e9d9c40e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:39.549 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.547+0000 7f499eba6740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x557e9d920600 con 0x557e9d9bcae0 2024-04-26T21:54:39.550 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.547+0000 7f499eba6740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x557e9d8bd100 con 0x557e9d9c1c50 2024-04-26T21:54:39.550 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.547+0000 7f499eba6740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x557e9d91c5c0 con 0x557e9d9bc0f0 2024-04-26T21:54:39.550 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.548+0000 7f4999400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557e9d9bcae0 0x557e9d9c15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:39.550 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.548+0000 7f4999400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557e9d9bcae0 0x557e9d9c15e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:39388/0 (socket says 172.21.15.72:39388) 2024-04-26T21:54:39.550 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.548+0000 7f4999400640 1 -- 172.21.15.72:0/1784925411 learned_addr learned my addr 172.21.15.72:0/1784925411 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:54:39.550 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.548+0000 7f4998a00640 1 --2- 172.21.15.72:0/1784925411 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557e9d9bc0f0 0x557e9d9bc510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:39.550 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.548+0000 7f4998a00640 1 --2- 172.21.15.72:0/1784925411 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557e9d9c1c50 0x557e9d9c40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:39.550 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.548+0000 7f4999400640 1 -- 172.21.15.72:0/1784925411 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557e9d9bc0f0 msgr2=0x557e9d9bc510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:54:39.550 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.548+0000 7f4999400640 1 --2- 172.21.15.72:0/1784925411 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557e9d9bc0f0 0x557e9d9bc510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:39.550 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.548+0000 7f4999400640 1 -- 172.21.15.72:0/1784925411 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557e9d9c1c50 msgr2=0x557e9d9c40e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:54:39.550 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.548+0000 7f4999400640 1 --2- 172.21.15.72:0/1784925411 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557e9d9c1c50 0x557e9d9c40e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:39.550 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.548+0000 7f4999400640 1 -- 172.21.15.72:0/1784925411 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557e9d6322d0 con 0x557e9d9bcae0 2024-04-26T21:54:39.550 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.548+0000 7f4992000640 1 --2- 172.21.15.72:0/1866600812 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f497c0a3cc0 0x7f497c0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:39.551 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.549+0000 7f4999400640 1 --2- 172.21.15.72:0/1784925411 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557e9d9bcae0 0x557e9d9c15e0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f4988006010 tx=0x7f4988031580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c359541d2de08040 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:54:39.551 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.549+0000 7f4992000640 1 -- 172.21.15.72:0/1866600812 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7f498c0d0840 con 0x7f499410a820 2024-04-26T21:54:39.551 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.549+0000 7f4998a00640 1 --2- 172.21.15.72:0/1866600812 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f497c0a3cc0 0x7f497c0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:39.551 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.549+0000 7f4982600640 1 -- 172.21.15.72:0/1784925411 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4988035030 con 0x557e9d9bcae0 2024-04-26T21:54:39.551 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.549+0000 7f4982600640 1 -- 172.21.15.72:0/1784925411 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4988004030 con 0x557e9d9bcae0 2024-04-26T21:54:39.551 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.549+0000 7f4982600640 1 -- 172.21.15.72:0/1784925411 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4988003b10 con 0x557e9d9bcae0 2024-04-26T21:54:39.555 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.550+0000 7f499eba6740 1 -- 172.21.15.72:0/1784925411 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557e9d9bcae0 msgr2=0x557e9d9c15e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:39.555 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.550+0000 7f499eba6740 1 --2- 172.21.15.72:0/1784925411 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557e9d9bcae0 0x557e9d9c15e0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f4988006010 tx=0x7f4988031580 comp rx=0 tx=0).stop 2024-04-26T21:54:39.555 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.550+0000 7f4998a00640 1 --2- 172.21.15.72:0/1866600812 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f497c0a3cc0 0x7f497c0a6180 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x557e9d9bd340 tx=0x7f49840046c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:54:39.555 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.550+0000 7f499eba6740 1 -- 172.21.15.72:0/1784925411 shutdown_connections 2024-04-26T21:54:39.555 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.550+0000 7f499eba6740 1 --2- 172.21.15.72:0/1784925411 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557e9d9c1c50 0x557e9d9c40e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:39.555 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.550+0000 7f499eba6740 1 --2- 172.21.15.72:0/1784925411 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557e9d9bcae0 0x557e9d9c15e0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:39.555 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.550+0000 7f499eba6740 1 --2- 172.21.15.72:0/1784925411 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557e9d9bc0f0 0x557e9d9bc510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:39.555 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.550+0000 7f499eba6740 1 -- 172.21.15.72:0/1784925411 >> 172.21.15.72:0/1784925411 conn(0x557e9d950d10 msgr2=0x557e9d9b7b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:54:39.555 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.550+0000 7f499eba6740 1 -- 172.21.15.72:0/1784925411 shutdown_connections 2024-04-26T21:54:39.556 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.550+0000 7f499eba6740 1 -- 172.21.15.72:0/1784925411 wait complete. 2024-04-26T21:54:39.556 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.550+0000 7f499eba6740 1 -- start start 2024-04-26T21:54:39.556 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.550+0000 7f499eba6740 1 Processor -- start 2024-04-26T21:54:39.556 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.551+0000 7f499eba6740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557e9d9bc0f0 0x557e9d9c6540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:39.556 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.551+0000 7f499eba6740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557e9d9bcae0 0x557e9d9c6a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:39.556 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.551+0000 7f499eba6740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557e9d9c1c50 0x557e9d9cdac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:39.556 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.551+0000 7f499eba6740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x557e9d9c4ba0 con 0x557e9d9c1c50 2024-04-26T21:54:39.556 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.551+0000 7f499eba6740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x557e9d8a7f70 con 0x557e9d9bcae0 2024-04-26T21:54:39.556 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.551+0000 7f499eba6740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x557e9d9c4d20 con 0x557e9d9bc0f0 2024-04-26T21:54:39.556 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.551+0000 7f499eba6740 10 client.0 _ll_register_callbacks cb 0x557e9d916480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:54:39.556 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.551+0000 7f4999e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557e9d9c1c50 0x557e9d9cdac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:39.556 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.551+0000 7f4999e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557e9d9c1c50 0x557e9d9cdac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:39398/0 (socket says 172.21.15.72:39398) 2024-04-26T21:54:39.556 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.551+0000 7f4999e00640 1 -- 172.21.15.72:0/3686593524 learned_addr learned my addr 172.21.15.72:0/3686593524 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:54:39.556 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.551+0000 7f4999400640 1 --2- 172.21.15.72:0/3686593524 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557e9d9bc0f0 0x557e9d9c6540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:39.557 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.552+0000 7f4999e00640 1 -- 172.21.15.72:0/3686593524 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557e9d9bc0f0 msgr2=0x557e9d9c6540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:54:39.557 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.552+0000 7f4999e00640 1 --2- 172.21.15.72:0/3686593524 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557e9d9bc0f0 0x557e9d9c6540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:39.557 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.552+0000 7f4999e00640 1 -- 172.21.15.72:0/3686593524 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557e9d9bcae0 msgr2=0x557e9d9c6a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:54:39.557 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.552+0000 7f4999e00640 1 --2- 172.21.15.72:0/3686593524 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557e9d9bcae0 0x557e9d9c6a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:39.557 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.552+0000 7f4999e00640 1 -- 172.21.15.72:0/3686593524 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557e9d6fc6b0 con 0x557e9d9c1c50 2024-04-26T21:54:39.557 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.552+0000 7f4999400640 1 --2- 172.21.15.72:0/3686593524 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557e9d9bc0f0 0x557e9d9c6540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:54:39.557 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.552+0000 7f4999e00640 1 --2- 172.21.15.72:0/3686593524 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557e9d9c1c50 0x557e9d9cdac0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f498c0a5020 tx=0x7f498c09c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d09f158fb6081076 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:54:39.557 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.552+0000 7f4976a00640 10 client.0 ms_handle_connect on v2:172.21.15.72:3300/0 2024-04-26T21:54:39.557 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.552+0000 7f4976a00640 1 -- 172.21.15.72:0/3686593524 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f498c01f040 con 0x557e9d9c1c50 2024-04-26T21:54:39.557 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.552+0000 7f4976a00640 1 -- 172.21.15.72:0/3686593524 <== mon.0 v2:172.21.15.72:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7f498c004a40 con 0x557e9d9c1c50 2024-04-26T21:54:39.557 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.552+0000 7f499eba6740 1 -- 172.21.15.72:0/3686593524 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4988004030 con 0x557e9d9c1c50 2024-04-26T21:54:39.557 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.552+0000 7f4976a00640 1 -- 172.21.15.72:0/3686593524 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f498c004c40 con 0x557e9d9c1c50 2024-04-26T21:54:39.557 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.552+0000 7f499eba6740 10 client.15144 fetch_fsmap learned FSMap version 26 2024-04-26T21:54:39.557 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.552+0000 7f499eba6740 1 -- 172.21.15.72:0/3686593524 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x557e9d9d0b70 con 0x557e9d9c1c50 2024-04-26T21:54:39.558 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.552+0000 7f4976a00640 1 -- 172.21.15.72:0/3686593524 <== mon.0 v2:172.21.15.72:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4988004030 con 0x557e9d9c1c50 2024-04-26T21:54:39.558 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.553+0000 7f4976a00640 1 -- 172.21.15.72:0/3686593524 <== mon.0 v2:172.21.15.72:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f498c00f040 con 0x557e9d9c1c50 2024-04-26T21:54:39.558 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.553+0000 7f499eba6740 10 client.15144 fetch_fsmap finished waiting for FSMap version 26 2024-04-26T21:54:39.558 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.553+0000 7f499eba6740 10 client.15144 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-26T21:54:39.558 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.553+0000 7f499eba6740 20 client.15144 populate_metadata read hostname 'smithi072' 2024-04-26T21:54:39.558 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.553+0000 7f499eba6740 1 --2- 172.21.15.72:0/3686593524 >> [v2:172.21.15.76:6836/1150678991,v1:172.21.15.76:6837/1150678991] conn(0x557e9d9d3220 0x557e9d9d4690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:39.558 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.553+0000 7f499eba6740 4 client.15144 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:54:39.558 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.553+0000 7f499eba6740 1 -- 172.21.15.72:0/3686593524 --> [v2:172.21.15.76:6836/1150678991,v1:172.21.15.76:6837/1150678991] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557e9d92e210 con 0x557e9d9d3220 2024-04-26T21:54:39.558 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.554+0000 7f4999400640 1 --2- 172.21.15.72:0/3686593524 >> [v2:172.21.15.76:6836/1150678991,v1:172.21.15.76:6837/1150678991] conn(0x557e9d9d3220 0x557e9d9d4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:39.558 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.554+0000 7f4999400640 1 --2- 172.21.15.72:0/3686593524 >> [v2:172.21.15.76:6836/1150678991,v1:172.21.15.76:6837/1150678991] conn(0x557e9d9d3220 0x557e9d9d4690 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d32879673fd67781 server_cookie=d592e3ece71b9512 in_seq=0 out_seq=0 2024-04-26T21:54:39.558 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.554+0000 7f4976a00640 10 client.15144 ms_handle_connect on v2:172.21.15.76:6836/1150678991 2024-04-26T21:54:39.559 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.556+0000 7f4976a00640 1 -- 172.21.15.72:0/3686593524 <== mds.1 v2:172.21.15.76:6836/1150678991 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557e9d92e210 con 0x557e9d9d3220 2024-04-26T21:54:39.559 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.556+0000 7f4976a00640 10 client.15144 handle_command_reply: tid=0 2024-04-26T21:54:39.559 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.557+0000 7f499eba6740 1 client.15144 shutdown 2024-04-26T21:54:39.560 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.557+0000 7f499eba6740 1 -- 172.21.15.72:0/3686593524 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557e9d9c1c50 msgr2=0x557e9d9cdac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:39.560 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.557+0000 7f499eba6740 1 --2- 172.21.15.72:0/3686593524 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557e9d9c1c50 0x557e9d9cdac0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f498c0a5020 tx=0x7f498c09c9a0 comp rx=0 tx=0).stop 2024-04-26T21:54:39.560 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.557+0000 7f499eba6740 1 -- 172.21.15.72:0/3686593524 shutdown_connections 2024-04-26T21:54:39.560 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.557+0000 7f499eba6740 1 --2- 172.21.15.72:0/3686593524 >> [v2:172.21.15.76:6836/1150678991,v1:172.21.15.76:6837/1150678991] conn(0x557e9d9d3220 0x557e9d9d4690 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:39.560 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.557+0000 7f499eba6740 1 --2- 172.21.15.72:0/3686593524 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557e9d9c1c50 0x557e9d9cdac0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:39.560 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.557+0000 7f499eba6740 1 --2- 172.21.15.72:0/3686593524 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557e9d9bcae0 0x557e9d9c6a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:39.560 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.557+0000 7f499eba6740 1 --2- 172.21.15.72:0/3686593524 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557e9d9bc0f0 0x557e9d9c6540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:39.560 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.557+0000 7f4976a00640 0 client.15144 ms_handle_reset on v2:172.21.15.76:6836/1150678991 2024-04-26T21:54:39.560 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.557+0000 7f499eba6740 1 -- 172.21.15.72:0/3686593524 >> 172.21.15.72:0/3686593524 conn(0x557e9d950d10 msgr2=0x557e9d9b6eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:54:39.560 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.557+0000 7f499eba6740 1 -- 172.21.15.72:0/3686593524 shutdown_connections 2024-04-26T21:54:39.560 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.557+0000 7f499eba6740 1 -- 172.21.15.72:0/3686593524 wait complete. 2024-04-26T21:54:39.560 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.557+0000 7f499eba6740 20 client.15144 trim_cache size 0 max 16384 2024-04-26T21:54:39.577 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.577+0000 7f499eba6740 1 Processor -- start 2024-04-26T21:54:39.577 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.577+0000 7f499eba6740 1 -- start start 2024-04-26T21:54:39.577 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.577+0000 7f499eba6740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557e9d9bc0f0 0x557e9d9c15a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:39.577 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.577+0000 7f499eba6740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557e9d9bcae0 0x557e9d9cda80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:39.578 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.577+0000 7f499eba6740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557e9d9c1c50 0x557e9d9bdf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:39.578 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.577+0000 7f499eba6740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x557e9d9d29d0 con 0x557e9d9c1c50 2024-04-26T21:54:39.578 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.577+0000 7f499eba6740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x557e9d9d30a0 con 0x557e9d9bc0f0 2024-04-26T21:54:39.578 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.577+0000 7f499eba6740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x557e9d9d3770 con 0x557e9d9bcae0 2024-04-26T21:54:39.580 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.578+0000 7f4999e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557e9d9c1c50 0x557e9d9bdf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:39.580 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.578+0000 7f4999e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557e9d9c1c50 0x557e9d9bdf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:39410/0 (socket says 172.21.15.72:39410) 2024-04-26T21:54:39.580 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.578+0000 7f4999e00640 1 -- 172.21.15.72:0/4254026957 learned_addr learned my addr 172.21.15.72:0/4254026957 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:54:39.580 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.578+0000 7f4998a00640 1 --2- 172.21.15.72:0/4254026957 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557e9d9bcae0 0x557e9d9cda80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:39.580 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.578+0000 7f4999400640 1 --2- 172.21.15.72:0/4254026957 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557e9d9bc0f0 0x557e9d9c15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:39.580 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.578+0000 7f4999e00640 1 -- 172.21.15.72:0/4254026957 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557e9d9bcae0 msgr2=0x557e9d9cda80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:54:39.580 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.578+0000 7f4999e00640 1 --2- 172.21.15.72:0/4254026957 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557e9d9bcae0 0x557e9d9cda80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:39.580 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.578+0000 7f4999e00640 1 -- 172.21.15.72:0/4254026957 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557e9d9bc0f0 msgr2=0x557e9d9c15a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:54:39.580 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.578+0000 7f4999e00640 1 --2- 172.21.15.72:0/4254026957 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557e9d9bc0f0 0x557e9d9c15a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:39.581 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.578+0000 7f4999e00640 1 -- 172.21.15.72:0/4254026957 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557e9d9bd370 con 0x557e9d9c1c50 2024-04-26T21:54:39.581 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.578+0000 7f4999400640 1 --2- 172.21.15.72:0/4254026957 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557e9d9bc0f0 0x557e9d9c15a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:54:39.581 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.578+0000 7f4999e00640 1 --2- 172.21.15.72:0/4254026957 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557e9d9c1c50 0x557e9d9bdf80 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f498c09c870 tx=0x7f498c000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1d249bfe28a09a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:54:39.581 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.578+0000 7f4981c00640 1 -- 172.21.15.72:0/4254026957 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f498c07f020 con 0x557e9d9c1c50 2024-04-26T21:54:39.581 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.579+0000 7f4981c00640 1 -- 172.21.15.72:0/4254026957 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f498c0ce360 con 0x557e9d9c1c50 2024-04-26T21:54:39.581 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.579+0000 7f4981c00640 1 -- 172.21.15.72:0/4254026957 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f498c01f030 con 0x557e9d9c1c50 2024-04-26T21:54:39.581 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.579+0000 7f499eba6740 1 -- 172.21.15.72:0/4254026957 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557e9d9c1c50 msgr2=0x557e9d9bdf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:39.581 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.579+0000 7f499eba6740 1 --2- 172.21.15.72:0/4254026957 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557e9d9c1c50 0x557e9d9bdf80 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f498c09c870 tx=0x7f498c000f40 comp rx=0 tx=0).stop 2024-04-26T21:54:39.581 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.579+0000 7f499eba6740 1 -- 172.21.15.72:0/4254026957 shutdown_connections 2024-04-26T21:54:39.581 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.579+0000 7f499eba6740 1 --2- 172.21.15.72:0/4254026957 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557e9d9c1c50 0x557e9d9bdf80 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:39.581 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.579+0000 7f499eba6740 1 --2- 172.21.15.72:0/4254026957 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557e9d9bcae0 0x557e9d9cda80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:39.581 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.579+0000 7f499eba6740 1 --2- 172.21.15.72:0/4254026957 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557e9d9bc0f0 0x557e9d9c15a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:39.581 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.579+0000 7f499eba6740 1 -- 172.21.15.72:0/4254026957 >> 172.21.15.72:0/4254026957 conn(0x557e9d950d10 msgr2=0x557e9d9c6540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:54:39.581 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.579+0000 7f499eba6740 1 -- 172.21.15.72:0/4254026957 shutdown_connections 2024-04-26T21:54:39.581 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.579+0000 7f499eba6740 1 -- 172.21.15.72:0/4254026957 wait complete. 2024-04-26T21:54:39.582 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.580+0000 7f499eba6740 1 -- start start 2024-04-26T21:54:39.582 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.580+0000 7f499eba6740 1 Processor -- start 2024-04-26T21:54:39.585 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.580+0000 7f499eba6740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557e9d9bc0f0 0x557e9d9cd5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:39.585 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.580+0000 7f499eba6740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557e9d9bcae0 0x557e9d9ca160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:39.585 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.580+0000 7f499eba6740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557e9d9c1c50 0x557e9d9ca6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:39.585 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.580+0000 7f499eba6740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x557e9d9d4510 con 0x557e9d9c1c50 2024-04-26T21:54:39.586 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.580+0000 7f499eba6740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x557e9d9d3e40 con 0x557e9d9bcae0 2024-04-26T21:54:39.586 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.580+0000 7f499eba6740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x557e9d9d4be0 con 0x557e9d9bc0f0 2024-04-26T21:54:39.586 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.580+0000 7f499eba6740 10 client.0 _ll_register_callbacks cb 0x7f498c098b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:54:39.586 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.581+0000 7f4999e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557e9d9c1c50 0x557e9d9ca6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:39.586 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.581+0000 7f4999e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557e9d9c1c50 0x557e9d9ca6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:39424/0 (socket says 172.21.15.72:39424) 2024-04-26T21:54:39.586 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.581+0000 7f4999e00640 1 -- 172.21.15.72:0/3493349958 learned_addr learned my addr 172.21.15.72:0/3493349958 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:54:39.586 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.581+0000 7f4999400640 1 --2- 172.21.15.72:0/3493349958 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557e9d9bc0f0 0x557e9d9cd5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:39.586 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.581+0000 7f4998a00640 1 --2- 172.21.15.72:0/3493349958 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557e9d9bcae0 0x557e9d9ca160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:39.586 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.581+0000 7f4999e00640 1 -- 172.21.15.72:0/3493349958 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557e9d9bc0f0 msgr2=0x557e9d9cd5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:54:39.586 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.581+0000 7f4999e00640 1 --2- 172.21.15.72:0/3493349958 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557e9d9bc0f0 0x557e9d9cd5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:39.586 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.581+0000 7f4999e00640 1 -- 172.21.15.72:0/3493349958 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557e9d9bcae0 msgr2=0x557e9d9ca160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:54:39.586 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.581+0000 7f4999e00640 1 --2- 172.21.15.72:0/3493349958 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557e9d9bcae0 0x557e9d9ca160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:39.586 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.581+0000 7f4999e00640 1 -- 172.21.15.72:0/3493349958 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557e9d9cdc10 con 0x557e9d9c1c50 2024-04-26T21:54:39.587 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.581+0000 7f4998a00640 1 --2- 172.21.15.72:0/3493349958 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557e9d9bcae0 0x557e9d9ca160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:54:39.587 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.581+0000 7f4999e00640 1 --2- 172.21.15.72:0/3493349958 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557e9d9c1c50 0x557e9d9ca6a0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f498c005bc0 tx=0x7f498c098ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f72b7bda32f66c55 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:54:39.587 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.581+0000 7f4976a00640 10 client.0 ms_handle_connect on v2:172.21.15.72:3300/0 2024-04-26T21:54:39.587 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.582+0000 7f4976a00640 1 -- 172.21.15.72:0/3493349958 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f498c07f070 con 0x557e9d9c1c50 2024-04-26T21:54:39.587 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.582+0000 7f4976a00640 1 -- 172.21.15.72:0/3493349958 <== mon.0 v2:172.21.15.72:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7f498c0bbcb0 con 0x557e9d9c1c50 2024-04-26T21:54:39.587 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.582+0000 7f499eba6740 1 -- 172.21.15.72:0/3493349958 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f498c0ce360 con 0x557e9d9c1c50 2024-04-26T21:54:39.587 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.582+0000 7f4976a00640 1 -- 172.21.15.72:0/3493349958 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f498c01f070 con 0x557e9d9c1c50 2024-04-26T21:54:39.587 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.582+0000 7f499eba6740 10 client.15150 fetch_fsmap learned FSMap version 26 2024-04-26T21:54:39.587 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.582+0000 7f499eba6740 1 -- 172.21.15.72:0/3493349958 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x557e9da3a040 con 0x557e9d9c1c50 2024-04-26T21:54:39.587 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.582+0000 7f4976a00640 1 -- 172.21.15.72:0/3493349958 <== mon.0 v2:172.21.15.72:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f498c0ce360 con 0x557e9d9c1c50 2024-04-26T21:54:39.587 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.582+0000 7f4976a00640 1 -- 172.21.15.72:0/3493349958 <== mon.0 v2:172.21.15.72:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f498c098040 con 0x557e9d9c1c50 2024-04-26T21:54:39.587 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.582+0000 7f499eba6740 10 client.15150 fetch_fsmap finished waiting for FSMap version 26 2024-04-26T21:54:39.587 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.582+0000 7f499eba6740 10 client.15150 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-26T21:54:39.587 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.582+0000 7f499eba6740 20 client.15150 populate_metadata read hostname 'smithi072' 2024-04-26T21:54:39.587 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.582+0000 7f499eba6740 1 --2- 172.21.15.72:0/3493349958 >> [v2:172.21.15.76:6836/1150678991,v1:172.21.15.76:6837/1150678991] conn(0x557e9da3e590 0x557e9da3d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:39.588 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.582+0000 7f499eba6740 4 client.15150 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T21:54:39.588 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.582+0000 7f499eba6740 1 -- 172.21.15.72:0/3493349958 --> [v2:172.21.15.76:6836/1150678991,v1:172.21.15.76:6837/1150678991] -- command(tid 0: {"prefix": "status"}) -- 0x557e9d928080 con 0x557e9da3e590 2024-04-26T21:54:39.588 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.583+0000 7f4999400640 1 --2- 172.21.15.72:0/3493349958 >> [v2:172.21.15.76:6836/1150678991,v1:172.21.15.76:6837/1150678991] conn(0x557e9da3e590 0x557e9da3d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:39.588 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.584+0000 7f4999400640 1 --2- 172.21.15.72:0/3493349958 >> [v2:172.21.15.76:6836/1150678991,v1:172.21.15.76:6837/1150678991] conn(0x557e9da3e590 0x557e9da3d2a0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4b00af0a87c64c11 server_cookie=8e08737d618cf803 in_seq=0 out_seq=0 2024-04-26T21:54:39.588 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.584+0000 7f4976a00640 10 client.15150 ms_handle_connect on v2:172.21.15.76:6836/1150678991 2024-04-26T21:54:39.588 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.585+0000 7f4976a00640 1 -- 172.21.15.72:0/3493349958 <== mds.1 v2:172.21.15.76:6836/1150678991 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x557e9d928080 con 0x557e9da3e590 2024-04-26T21:54:39.588 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.585+0000 7f4976a00640 10 client.15150 handle_command_reply: tid=0 2024-04-26T21:54:39.588 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.585+0000 7f499eba6740 1 client.15150 shutdown 2024-04-26T21:54:39.588 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.585+0000 7f499eba6740 1 -- 172.21.15.72:0/3493349958 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557e9d9c1c50 msgr2=0x557e9d9ca6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:39.588 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.585+0000 7f499eba6740 1 --2- 172.21.15.72:0/3493349958 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557e9d9c1c50 0x557e9d9ca6a0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f498c005bc0 tx=0x7f498c098ae0 comp rx=0 tx=0).stop 2024-04-26T21:54:39.588 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.585+0000 7f499eba6740 1 -- 172.21.15.72:0/3493349958 shutdown_connections 2024-04-26T21:54:39.588 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.585+0000 7f499eba6740 1 --2- 172.21.15.72:0/3493349958 >> [v2:172.21.15.76:6836/1150678991,v1:172.21.15.76:6837/1150678991] conn(0x557e9da3e590 0x557e9da3d2a0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:39.588 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.585+0000 7f499eba6740 1 --2- 172.21.15.72:0/3493349958 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557e9d9c1c50 0x557e9d9ca6a0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:39.588 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.585+0000 7f499eba6740 1 --2- 172.21.15.72:0/3493349958 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557e9d9bcae0 0x557e9d9ca160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:39.589 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.585+0000 7f499eba6740 1 --2- 172.21.15.72:0/3493349958 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557e9d9bc0f0 0x557e9d9cd5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:39.589 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.585+0000 7f4976a00640 0 client.15150 ms_handle_reset on v2:172.21.15.76:6836/1150678991 2024-04-26T21:54:39.589 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.585+0000 7f499eba6740 1 -- 172.21.15.72:0/3493349958 >> 172.21.15.72:0/3493349958 conn(0x557e9d950d10 msgr2=0x557e9d9bdf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:54:39.589 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.585+0000 7f499eba6740 1 -- 172.21.15.72:0/3493349958 shutdown_connections 2024-04-26T21:54:39.589 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.585+0000 7f499eba6740 1 -- 172.21.15.72:0/3493349958 wait complete. 2024-04-26T21:54:39.589 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.585+0000 7f499eba6740 20 client.15150 trim_cache size 0 max 16384 2024-04-26T21:54:39.589 INFO:teuthology.orchestra.run.smithi072.stdout:{ 2024-04-26T21:54:39.589 INFO:teuthology.orchestra.run.smithi072.stdout: "cluster_fsid": "a1780950-0415-11ef-bc93-c7b262605968", 2024-04-26T21:54:39.589 INFO:teuthology.orchestra.run.smithi072.stdout: "whoami": 1, 2024-04-26T21:54:39.589 INFO:teuthology.orchestra.run.smithi072.stdout: "id": 34472, 2024-04-26T21:54:39.590 INFO:teuthology.orchestra.run.smithi072.stdout: "want_state": "up:active", 2024-04-26T21:54:39.590 INFO:teuthology.orchestra.run.smithi072.stdout: "state": "up:active", 2024-04-26T21:54:39.590 INFO:teuthology.orchestra.run.smithi072.stdout: "fs_name": "cephfs", 2024-04-26T21:54:39.590 INFO:teuthology.orchestra.run.smithi072.stdout: "rank_uptime": 105.057563268, 2024-04-26T21:54:39.590 INFO:teuthology.orchestra.run.smithi072.stdout: "mdsmap_epoch": 26, 2024-04-26T21:54:39.590 INFO:teuthology.orchestra.run.smithi072.stdout: "osdmap_epoch": 91, 2024-04-26T21:54:39.590 INFO:teuthology.orchestra.run.smithi072.stdout: "osdmap_epoch_barrier": 87, 2024-04-26T21:54:39.590 INFO:teuthology.orchestra.run.smithi072.stdout: "uptime": 131.11615604799999 2024-04-26T21:54:39.590 INFO:teuthology.orchestra.run.smithi072.stdout:} 2024-04-26T21:54:39.591 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.586+0000 7f499be00640 1 -- 172.21.15.72:0/1866600812 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f497c0a3cc0 msgr2=0x7f497c0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:39.591 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.586+0000 7f499be00640 1 --2- 172.21.15.72:0/1866600812 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f497c0a3cc0 0x7f497c0a6180 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x557e9d9bd340 tx=0x7f49840046c0 comp rx=0 tx=0).stop 2024-04-26T21:54:39.591 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.586+0000 7f499be00640 1 -- 172.21.15.72:0/1866600812 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f499410a820 msgr2=0x7f49941a21a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:39.591 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.586+0000 7f499be00640 1 --2- 172.21.15.72:0/1866600812 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f499410a820 0x7f49941a21a0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f498c00eaa0 tx=0x7f498c00ef60 comp rx=0 tx=0).stop 2024-04-26T21:54:39.591 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.586+0000 7f499be00640 1 -- 172.21.15.72:0/1866600812 shutdown_connections 2024-04-26T21:54:39.591 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.586+0000 7f499be00640 1 --2- 172.21.15.72:0/1866600812 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f499410a820 0x7f49941a21a0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:39.591 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.586+0000 7f499be00640 1 --2- 172.21.15.72:0/1866600812 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f497c0a3cc0 0x7f497c0a6180 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:39.591 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.586+0000 7f499be00640 1 --2- 172.21.15.72:0/1866600812 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f49940768f0 0x7f499419b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:39.592 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.586+0000 7f499be00640 1 --2- 172.21.15.72:0/1866600812 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4994075310 0x7f499419abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:39.592 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.586+0000 7f499be00640 1 -- 172.21.15.72:0/1866600812 >> 172.21.15.72:0/1866600812 conn(0x7f49940fdee0 msgr2=0x7f49941067b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:54:39.592 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.586+0000 7f499be00640 1 -- 172.21.15.72:0/1866600812 shutdown_connections 2024-04-26T21:54:39.592 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.586+0000 7f499be00640 1 -- 172.21.15.72:0/1866600812 wait complete. 2024-04-26T21:54:39.605 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-26T21:54:39.806 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.805+0000 7fb89be00640 1 -- 172.21.15.72:0/1681668841 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb89c108d90 msgr2=0x7fb89c1091b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:39.806 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.805+0000 7fb89be00640 1 --2- 172.21.15.72:0/1681668841 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb89c108d90 0x7fb89c1091b0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fb88c009900 tx=0x7fb88c02a3f0 comp rx=0 tx=0).stop 2024-04-26T21:54:39.806 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.805+0000 7fb89be00640 1 -- 172.21.15.72:0/1681668841 shutdown_connections 2024-04-26T21:54:39.806 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.805+0000 7fb89be00640 1 --2- 172.21.15.72:0/1681668841 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb89c10a940 0x7fb89c1111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:39.806 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.805+0000 7fb89be00640 1 --2- 172.21.15.72:0/1681668841 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb89c109f80 0x7fb89c10a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:39.806 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.805+0000 7fb89be00640 1 --2- 172.21.15.72:0/1681668841 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb89c108d90 0x7fb89c1091b0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:39.806 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.805+0000 7fb89be00640 1 -- 172.21.15.72:0/1681668841 >> 172.21.15.72:0/1681668841 conn(0x7fb89c077d30 msgr2=0x7fb89c07a170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:54:39.807 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.805+0000 7fb89be00640 1 -- 172.21.15.72:0/1681668841 shutdown_connections 2024-04-26T21:54:39.807 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.806+0000 7fb89be00640 1 -- 172.21.15.72:0/1681668841 wait complete. 2024-04-26T21:54:39.809 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.806+0000 7fb89be00640 1 Processor -- start 2024-04-26T21:54:39.809 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.807+0000 7fb89be00640 1 -- start start 2024-04-26T21:54:39.809 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.807+0000 7fb89be00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb89c108d90 0x7fb89c19ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:39.809 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.807+0000 7fb89be00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb89c109f80 0x7fb89c19f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:39.809 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.807+0000 7fb89be00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb89c10a940 0x7fb89c1a63f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:39.809 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.807+0000 7fb89be00640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fb89c1147c0 con 0x7fb89c109f80 2024-04-26T21:54:39.809 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.807+0000 7fb89be00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fb89c114640 con 0x7fb89c10a940 2024-04-26T21:54:39.809 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.807+0000 7fb89be00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fb89c114940 con 0x7fb89c108d90 2024-04-26T21:54:39.809 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.807+0000 7fb89aa00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb89c108d90 0x7fb89c19ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:39.809 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.808+0000 7fb89aa00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb89c108d90 0x7fb89c19ee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.72:55286/0 (socket says 172.21.15.72:55286) 2024-04-26T21:54:39.810 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.808+0000 7fb89aa00640 1 -- 172.21.15.72:0/300466996 learned_addr learned my addr 172.21.15.72:0/300466996 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:54:39.810 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.808+0000 7fb89a000640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb89c109f80 0x7fb89c19f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:39.810 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.808+0000 7fb89b400640 1 --2- 172.21.15.72:0/300466996 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb89c10a940 0x7fb89c1a63f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:39.810 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.808+0000 7fb89a000640 1 -- 172.21.15.72:0/300466996 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb89c108d90 msgr2=0x7fb89c19ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:39.810 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.808+0000 7fb89a000640 1 --2- 172.21.15.72:0/300466996 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb89c108d90 0x7fb89c19ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:39.810 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.808+0000 7fb89a000640 1 -- 172.21.15.72:0/300466996 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb89c10a940 msgr2=0x7fb89c1a63f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:39.810 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.808+0000 7fb89a000640 1 --2- 172.21.15.72:0/300466996 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb89c10a940 0x7fb89c1a63f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:39.810 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.808+0000 7fb89a000640 1 -- 172.21.15.72:0/300466996 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb89c1a6a60 con 0x7fb89c109f80 2024-04-26T21:54:39.810 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.808+0000 7fb89b400640 1 --2- 172.21.15.72:0/300466996 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb89c10a940 0x7fb89c1a63f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:54:39.810 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.808+0000 7fb89a000640 1 --2- 172.21.15.72:0/300466996 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb89c109f80 0x7fb89c19f380 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fb88800c970 tx=0x7fb88800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:54:39.810 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.809+0000 7fb893400640 1 -- 172.21.15.72:0/300466996 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb888007bb0 con 0x7fb89c109f80 2024-04-26T21:54:39.810 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.809+0000 7fb893400640 1 -- 172.21.15.72:0/300466996 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb888007d50 con 0x7fb89c109f80 2024-04-26T21:54:39.810 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.809+0000 7fb89be00640 1 -- 172.21.15.72:0/300466996 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb89c1a6d50 con 0x7fb89c109f80 2024-04-26T21:54:39.811 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.809+0000 7fb893400640 1 -- 172.21.15.72:0/300466996 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8880186f0 con 0x7fb89c109f80 2024-04-26T21:54:39.811 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.809+0000 7fb89be00640 1 -- 172.21.15.72:0/300466996 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb89c1a7260 con 0x7fb89c109f80 2024-04-26T21:54:39.815 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.812+0000 7fb8a439d740 1 Processor -- start 2024-04-26T21:54:39.815 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.812+0000 7fb8a439d740 1 -- start start 2024-04-26T21:54:39.815 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.813+0000 7fb8a439d740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55607b1ca2c0 0x55607b1ca6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:39.815 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.813+0000 7fb8a439d740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55607b1cacb0 0x55607b1cf7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:39.815 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.813+0000 7fb8a439d740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55607b1cfe20 0x55607b1d22b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:39.815 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.813+0000 7fb8a439d740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55607b12e600 con 0x55607b1cfe20 2024-04-26T21:54:39.816 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.813+0000 7fb8a439d740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55607b0cb100 con 0x55607b1cacb0 2024-04-26T21:54:39.816 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.813+0000 7fb8a439d740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55607b12a5c0 con 0x55607b1ca2c0 2024-04-26T21:54:39.816 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.813+0000 7fb893400640 1 -- 172.21.15.72:0/300466996 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fb88800f450 con 0x7fb89c109f80 2024-04-26T21:54:39.816 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.813+0000 7fb89aa00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55607b1cacb0 0x55607b1cf7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:39.816 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.813+0000 7fb89b400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55607b1ca2c0 0x55607b1ca6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:39.816 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.813+0000 7fb89a000640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55607b1cfe20 0x55607b1d22b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:39.816 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.813+0000 7fb89b400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55607b1ca2c0 0x55607b1ca6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.72:55294/0 (socket says 172.21.15.72:55294) 2024-04-26T21:54:39.816 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.813+0000 7fb89b400640 1 -- 172.21.15.72:0/3702132083 learned_addr learned my addr 172.21.15.72:0/3702132083 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:54:39.816 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.814+0000 7fb89a000640 1 -- 172.21.15.72:0/3702132083 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55607b1ca2c0 msgr2=0x55607b1ca6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:54:39.816 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.814+0000 7fb89a000640 1 --2- 172.21.15.72:0/3702132083 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55607b1ca2c0 0x55607b1ca6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:39.816 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.814+0000 7fb89a000640 1 -- 172.21.15.72:0/3702132083 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55607b1cacb0 msgr2=0x55607b1cf7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:54:39.816 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.814+0000 7fb89a000640 1 --2- 172.21.15.72:0/3702132083 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55607b1cacb0 0x55607b1cf7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:39.816 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.814+0000 7fb89a000640 1 -- 172.21.15.72:0/3702132083 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55607ae402d0 con 0x55607b1cfe20 2024-04-26T21:54:39.816 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.814+0000 7fb89a000640 1 --2- 172.21.15.72:0/3702132083 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55607b1cfe20 0x55607b1d22b0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fb8880cdf30 tx=0x7fb8880d6cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72ee279d971eca29 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:54:39.816 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.815+0000 7fb887400640 1 -- 172.21.15.72:0/3702132083 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8880df070 con 0x55607b1cfe20 2024-04-26T21:54:39.816 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.815+0000 7fb887400640 1 -- 172.21.15.72:0/3702132083 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb8880d8d00 con 0x55607b1cfe20 2024-04-26T21:54:39.817 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.815+0000 7fb887400640 1 -- 172.21.15.72:0/3702132083 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8880d27f0 con 0x55607b1cfe20 2024-04-26T21:54:39.817 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.815+0000 7fb893400640 1 --2- 172.21.15.72:0/300466996 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fb8800a3cc0 0x7fb8800a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:39.817 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.815+0000 7fb893400640 1 -- 172.21.15.72:0/300466996 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7fb8880d0600 con 0x7fb89c109f80 2024-04-26T21:54:39.817 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.815+0000 7fb89b400640 1 --2- 172.21.15.72:0/300466996 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fb8800a3cc0 0x7fb8800a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:39.820 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.816+0000 7fb8a439d740 1 -- 172.21.15.72:0/3702132083 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55607b1cfe20 msgr2=0x55607b1d22b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:39.820 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.816+0000 7fb8a439d740 1 --2- 172.21.15.72:0/3702132083 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55607b1cfe20 0x55607b1d22b0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fb8880cdf30 tx=0x7fb8880d6cd0 comp rx=0 tx=0).stop 2024-04-26T21:54:39.820 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.816+0000 7fb89b400640 1 --2- 172.21.15.72:0/300466996 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fb8800a3cc0 0x7fb8800a6180 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x55607b1cb510 tx=0x7fb89400a830 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:54:39.821 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.816+0000 7fb8a439d740 1 -- 172.21.15.72:0/3702132083 shutdown_connections 2024-04-26T21:54:39.821 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.816+0000 7fb8a439d740 1 --2- 172.21.15.72:0/3702132083 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55607b1cfe20 0x55607b1d22b0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:39.821 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.816+0000 7fb8a439d740 1 --2- 172.21.15.72:0/3702132083 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55607b1cacb0 0x55607b1cf7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:39.821 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.816+0000 7fb8a439d740 1 --2- 172.21.15.72:0/3702132083 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55607b1ca2c0 0x55607b1ca6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:39.821 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.816+0000 7fb8a439d740 1 -- 172.21.15.72:0/3702132083 >> 172.21.15.72:0/3702132083 conn(0x55607b15ed10 msgr2=0x55607b1c5d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:54:39.821 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.816+0000 7fb8a439d740 1 -- 172.21.15.72:0/3702132083 shutdown_connections 2024-04-26T21:54:39.821 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.816+0000 7fb8a439d740 1 -- 172.21.15.72:0/3702132083 wait complete. 2024-04-26T21:54:39.821 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.816+0000 7fb8a439d740 1 -- start start 2024-04-26T21:54:39.821 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.817+0000 7fb8a439d740 1 Processor -- start 2024-04-26T21:54:39.821 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.817+0000 7fb8a439d740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55607b1ca2c0 0x55607b1d4690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:39.821 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.817+0000 7fb8a439d740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55607b1cacb0 0x55607b1d4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:39.821 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.817+0000 7fb8a439d740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55607b1cfe20 0x55607b1dbc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:39.821 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.817+0000 7fb8a439d740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55607b1d2dd0 con 0x55607b1ca2c0 2024-04-26T21:54:39.821 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.817+0000 7fb8a439d740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55607b0b5f70 con 0x55607b1cacb0 2024-04-26T21:54:39.821 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.817+0000 7fb8a439d740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55607b1d2f50 con 0x55607b1cfe20 2024-04-26T21:54:39.822 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.817+0000 7fb8a439d740 10 client.0 _ll_register_callbacks cb 0x55607b124480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:54:39.822 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.817+0000 7fb89aa00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55607b1ca2c0 0x55607b1d4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:39.822 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.817+0000 7fb89aa00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55607b1ca2c0 0x55607b1d4690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:39460/0 (socket says 172.21.15.72:39460) 2024-04-26T21:54:39.822 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.817+0000 7fb89aa00640 1 -- 172.21.15.72:0/2875355875 learned_addr learned my addr 172.21.15.72:0/2875355875 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:54:39.822 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.818+0000 7fb89a000640 1 --2- 172.21.15.72:0/2875355875 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55607b1cacb0 0x55607b1d4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:39.822 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.818+0000 7fb89b400640 1 --2- 172.21.15.72:0/2875355875 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55607b1cfe20 0x55607b1dbc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:39.822 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.818+0000 7fb89aa00640 1 -- 172.21.15.72:0/2875355875 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55607b1cfe20 msgr2=0x55607b1dbc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:54:39.822 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.818+0000 7fb89aa00640 1 --2- 172.21.15.72:0/2875355875 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55607b1cfe20 0x55607b1dbc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:39.822 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.818+0000 7fb89aa00640 1 -- 172.21.15.72:0/2875355875 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55607b1cacb0 msgr2=0x55607b1d4bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:54:39.822 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.818+0000 7fb89aa00640 1 --2- 172.21.15.72:0/2875355875 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55607b1cacb0 0x55607b1d4bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:39.822 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.818+0000 7fb89aa00640 1 -- 172.21.15.72:0/2875355875 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55607af0a6b0 con 0x55607b1ca2c0 2024-04-26T21:54:39.822 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.818+0000 7fb89aa00640 1 --2- 172.21.15.72:0/2875355875 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55607b1ca2c0 0x55607b1d4690 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fb88c0057c0 tx=0x7fb88c003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e45506bb03095f67 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:54:39.822 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.818+0000 7fb87be00640 10 client.0 ms_handle_connect on v2:172.21.15.72:3300/0 2024-04-26T21:54:39.822 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.818+0000 7fb87be00640 1 -- 172.21.15.72:0/2875355875 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb88c035040 con 0x55607b1ca2c0 2024-04-26T21:54:39.823 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.818+0000 7fb87be00640 1 -- 172.21.15.72:0/2875355875 <== mon.0 v2:172.21.15.72:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7fb88c031500 con 0x55607b1ca2c0 2024-04-26T21:54:39.823 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.818+0000 7fb8a439d740 1 -- 172.21.15.72:0/2875355875 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb8880d8d00 con 0x55607b1ca2c0 2024-04-26T21:54:39.823 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.819+0000 7fb87be00640 1 -- 172.21.15.72:0/2875355875 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb88c04b030 con 0x55607b1ca2c0 2024-04-26T21:54:39.823 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.819+0000 7fb8a439d740 10 client.15162 fetch_fsmap learned FSMap version 26 2024-04-26T21:54:39.823 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.819+0000 7fb8a439d740 1 -- 172.21.15.72:0/2875355875 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55607b1decb0 con 0x55607b1ca2c0 2024-04-26T21:54:39.823 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.819+0000 7fb87be00640 1 -- 172.21.15.72:0/2875355875 <== mon.0 v2:172.21.15.72:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb8880d8d00 con 0x55607b1ca2c0 2024-04-26T21:54:39.823 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.819+0000 7fb87be00640 1 -- 172.21.15.72:0/2875355875 <== mon.0 v2:172.21.15.72:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fb88c03c670 con 0x55607b1ca2c0 2024-04-26T21:54:39.823 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.819+0000 7fb8a439d740 10 client.15162 fetch_fsmap finished waiting for FSMap version 26 2024-04-26T21:54:39.823 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.819+0000 7fb8a439d740 10 client.15162 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-26T21:54:39.823 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.820+0000 7fb8a439d740 20 client.15162 populate_metadata read hostname 'smithi072' 2024-04-26T21:54:39.823 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.820+0000 7fb8a439d740 1 --2- 172.21.15.72:0/2875355875 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x55607b1e1410 0x55607b1e2880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:39.823 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.820+0000 7fb8a439d740 4 client.15162 mds_command: new command op to 34517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:54:39.823 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.820+0000 7fb8a439d740 1 -- 172.21.15.72:0/2875355875 --> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55607b13c210 con 0x55607b1e1410 2024-04-26T21:54:39.823 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.820+0000 7fb89a000640 1 --2- 172.21.15.72:0/2875355875 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x55607b1e1410 0x55607b1e2880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:39.825 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.821+0000 7fb87be00640 10 client.15162 ms_handle_connect on v2:172.21.15.76:6838/327145636 2024-04-26T21:54:39.825 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.821+0000 7fb89a000640 1 --2- 172.21.15.72:0/2875355875 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x55607b1e1410 0x55607b1e2880 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=58b6a00598ae73af server_cookie=a26bef4a2ea8bd2f in_seq=0 out_seq=0 2024-04-26T21:54:39.825 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.823+0000 7fb87be00640 1 -- 172.21.15.72:0/2875355875 <== mds.0 v2:172.21.15.76:6838/327145636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55607b13c210 con 0x55607b1e1410 2024-04-26T21:54:39.825 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.823+0000 7fb87be00640 10 client.15162 handle_command_reply: tid=0 2024-04-26T21:54:39.825 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.824+0000 7fb8a439d740 1 client.15162 shutdown 2024-04-26T21:54:39.825 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.824+0000 7fb8a439d740 1 -- 172.21.15.72:0/2875355875 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55607b1ca2c0 msgr2=0x55607b1d4690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:39.825 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.824+0000 7fb8a439d740 1 --2- 172.21.15.72:0/2875355875 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55607b1ca2c0 0x55607b1d4690 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fb88c0057c0 tx=0x7fb88c003820 comp rx=0 tx=0).stop 2024-04-26T21:54:39.825 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.824+0000 7fb8a439d740 1 -- 172.21.15.72:0/2875355875 shutdown_connections 2024-04-26T21:54:39.825 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.824+0000 7fb8a439d740 1 --2- 172.21.15.72:0/2875355875 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x55607b1e1410 0x55607b1e2880 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:39.825 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.824+0000 7fb8a439d740 1 --2- 172.21.15.72:0/2875355875 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55607b1cfe20 0x55607b1dbc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:39.825 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.824+0000 7fb8a439d740 1 --2- 172.21.15.72:0/2875355875 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55607b1cacb0 0x55607b1d4bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:39.825 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.824+0000 7fb8a439d740 1 --2- 172.21.15.72:0/2875355875 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55607b1ca2c0 0x55607b1d4690 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:39.826 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.824+0000 7fb8a439d740 1 -- 172.21.15.72:0/2875355875 >> 172.21.15.72:0/2875355875 conn(0x55607b15ed10 msgr2=0x55607b1c4f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:54:39.826 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.824+0000 7fb87be00640 0 client.15162 ms_handle_reset on v2:172.21.15.76:6838/327145636 2024-04-26T21:54:39.826 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.824+0000 7fb8a439d740 1 -- 172.21.15.72:0/2875355875 shutdown_connections 2024-04-26T21:54:39.826 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.824+0000 7fb8a439d740 1 -- 172.21.15.72:0/2875355875 wait complete. 2024-04-26T21:54:39.826 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.824+0000 7fb8a439d740 20 client.15162 trim_cache size 0 max 16384 2024-04-26T21:54:39.845 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.844+0000 7fb8a439d740 1 Processor -- start 2024-04-26T21:54:39.845 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.844+0000 7fb8a439d740 1 -- start start 2024-04-26T21:54:39.845 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.844+0000 7fb8a439d740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55607b1ca2c0 0x55607b1c4de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:39.845 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.844+0000 7fb8a439d740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55607b1cacb0 0x55607b1dbbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:39.845 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.844+0000 7fb8a439d740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55607b1cfe20 0x55607b1d3cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:39.845 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.844+0000 7fb8a439d740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55607b1e0bc0 con 0x55607b1cacb0 2024-04-26T21:54:39.845 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.844+0000 7fb8a439d740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55607b1e1290 con 0x55607b1ca2c0 2024-04-26T21:54:39.845 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.844+0000 7fb8a439d740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55607b1e1960 con 0x55607b1cfe20 2024-04-26T21:54:39.845 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.844+0000 7fb89a000640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55607b1cacb0 0x55607b1dbbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:39.845 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.844+0000 7fb89a000640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55607b1cacb0 0x55607b1dbbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:39476/0 (socket says 172.21.15.72:39476) 2024-04-26T21:54:39.845 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.844+0000 7fb89a000640 1 -- 172.21.15.72:0/3910835633 learned_addr learned my addr 172.21.15.72:0/3910835633 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:54:39.845 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.845+0000 7fb89a000640 1 -- 172.21.15.72:0/3910835633 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55607b1cfe20 msgr2=0x55607b1d3cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:54:39.846 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.845+0000 7fb89a000640 1 --2- 172.21.15.72:0/3910835633 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55607b1cfe20 0x55607b1d3cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:39.846 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.845+0000 7fb89a000640 1 -- 172.21.15.72:0/3910835633 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55607b1ca2c0 msgr2=0x55607b1c4de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:54:39.846 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.845+0000 7fb89a000640 1 --2- 172.21.15.72:0/3910835633 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55607b1ca2c0 0x55607b1c4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:39.846 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.845+0000 7fb89a000640 1 -- 172.21.15.72:0/3910835633 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55607b1cf360 con 0x55607b1cacb0 2024-04-26T21:54:39.846 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.845+0000 7fb89a000640 1 --2- 172.21.15.72:0/3910835633 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55607b1cacb0 0x55607b1dbbd0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fb88800c940 tx=0x7fb888000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f88d1c078e25c1c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:54:39.849 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.845+0000 7fb886a00640 1 -- 172.21.15.72:0/3910835633 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8880df040 con 0x55607b1cacb0 2024-04-26T21:54:39.849 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.845+0000 7fb886a00640 1 -- 172.21.15.72:0/3910835633 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb8880d6830 con 0x55607b1cacb0 2024-04-26T21:54:39.849 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.845+0000 7fb886a00640 1 -- 172.21.15.72:0/3910835633 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb88809c060 con 0x55607b1cacb0 2024-04-26T21:54:39.849 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.846+0000 7fb8a439d740 1 -- 172.21.15.72:0/3910835633 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55607b1cacb0 msgr2=0x55607b1dbbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:39.849 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.846+0000 7fb8a439d740 1 --2- 172.21.15.72:0/3910835633 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55607b1cacb0 0x55607b1dbbd0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fb88800c940 tx=0x7fb888000bc0 comp rx=0 tx=0).stop 2024-04-26T21:54:39.850 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.846+0000 7fb8a439d740 1 -- 172.21.15.72:0/3910835633 shutdown_connections 2024-04-26T21:54:39.850 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.846+0000 7fb8a439d740 1 --2- 172.21.15.72:0/3910835633 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55607b1cfe20 0x55607b1d3cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:39.850 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.846+0000 7fb8a439d740 1 --2- 172.21.15.72:0/3910835633 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55607b1cacb0 0x55607b1dbbd0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:39.850 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.846+0000 7fb8a439d740 1 --2- 172.21.15.72:0/3910835633 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55607b1ca2c0 0x55607b1c4de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:39.850 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.846+0000 7fb8a439d740 1 -- 172.21.15.72:0/3910835633 >> 172.21.15.72:0/3910835633 conn(0x55607b15ed10 msgr2=0x55607b1c46d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:54:39.850 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.846+0000 7fb8a439d740 1 -- 172.21.15.72:0/3910835633 shutdown_connections 2024-04-26T21:54:39.850 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.846+0000 7fb8a439d740 1 -- 172.21.15.72:0/3910835633 wait complete. 2024-04-26T21:54:39.850 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.846+0000 7fb8a439d740 1 -- start start 2024-04-26T21:54:39.850 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.847+0000 7fb8a439d740 1 Processor -- start 2024-04-26T21:54:39.850 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.847+0000 7fb8a439d740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55607b1ca2c0 0x55607b1d7e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:39.850 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.847+0000 7fb8a439d740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55607b1cacb0 0x55607b1d8340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:39.850 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.847+0000 7fb8a439d740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55607b1cfe20 0x55607b1dae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:39.850 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.847+0000 7fb8a439d740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55607b1e2700 con 0x55607b1cfe20 2024-04-26T21:54:39.851 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.847+0000 7fb8a439d740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55607b1e2030 con 0x55607b1cacb0 2024-04-26T21:54:39.851 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.847+0000 7fb8a439d740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55607b1e2dd0 con 0x55607b1ca2c0 2024-04-26T21:54:39.851 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.847+0000 7fb8a439d740 10 client.0 _ll_register_callbacks cb 0x7fb88c004350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:54:39.851 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.847+0000 7fb89b400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55607b1cfe20 0x55607b1dae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:39.851 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.847+0000 7fb89b400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55607b1cfe20 0x55607b1dae90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:39478/0 (socket says 172.21.15.72:39478) 2024-04-26T21:54:39.851 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.847+0000 7fb89b400640 1 -- 172.21.15.72:0/4089548036 learned_addr learned my addr 172.21.15.72:0/4089548036 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:54:39.851 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.848+0000 7fb89aa00640 1 --2- 172.21.15.72:0/4089548036 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55607b1ca2c0 0x55607b1d7e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:39.851 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.848+0000 7fb89a000640 1 --2- 172.21.15.72:0/4089548036 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55607b1cacb0 0x55607b1d8340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:39.851 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.848+0000 7fb89b400640 1 -- 172.21.15.72:0/4089548036 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55607b1ca2c0 msgr2=0x55607b1d7e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:54:39.851 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.848+0000 7fb89b400640 1 --2- 172.21.15.72:0/4089548036 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55607b1ca2c0 0x55607b1d7e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:39.851 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.848+0000 7fb89b400640 1 -- 172.21.15.72:0/4089548036 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55607b1cacb0 msgr2=0x55607b1d8340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:54:39.851 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.848+0000 7fb89b400640 1 --2- 172.21.15.72:0/4089548036 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55607b1cacb0 0x55607b1d8340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:39.851 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.848+0000 7fb89b400640 1 -- 172.21.15.72:0/4089548036 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55607b1db480 con 0x55607b1cfe20 2024-04-26T21:54:39.851 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.848+0000 7fb89b400640 1 --2- 172.21.15.72:0/4089548036 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55607b1cfe20 0x55607b1dae90 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fb89400fee0 tx=0x7fb894004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28a4bb040b51c825 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:54:39.852 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.848+0000 7fb87be00640 10 client.0 ms_handle_connect on v2:172.21.15.72:3300/0 2024-04-26T21:54:39.852 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.848+0000 7fb87be00640 1 -- 172.21.15.72:0/4089548036 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb89401edf0 con 0x55607b1cfe20 2024-04-26T21:54:39.852 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.848+0000 7fb87be00640 1 -- 172.21.15.72:0/4089548036 <== mon.0 v2:172.21.15.72:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7fb89401c480 con 0x55607b1cfe20 2024-04-26T21:54:39.852 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.848+0000 7fb8a439d740 1 -- 172.21.15.72:0/4089548036 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb8880d6830 con 0x55607b1cfe20 2024-04-26T21:54:39.852 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.848+0000 7fb87be00640 1 -- 172.21.15.72:0/4089548036 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb89401c680 con 0x55607b1cfe20 2024-04-26T21:54:39.852 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.848+0000 7fb8a439d740 10 client.15168 fetch_fsmap learned FSMap version 26 2024-04-26T21:54:39.852 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.848+0000 7fb8a439d740 1 -- 172.21.15.72:0/4089548036 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55607b247b40 con 0x55607b1cfe20 2024-04-26T21:54:39.852 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.849+0000 7fb87be00640 1 -- 172.21.15.72:0/4089548036 <== mon.0 v2:172.21.15.72:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb8880d6830 con 0x55607b1cfe20 2024-04-26T21:54:39.852 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.849+0000 7fb87be00640 1 -- 172.21.15.72:0/4089548036 <== mon.0 v2:172.21.15.72:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fb894025070 con 0x55607b1cfe20 2024-04-26T21:54:39.852 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.849+0000 7fb8a439d740 10 client.15168 fetch_fsmap finished waiting for FSMap version 26 2024-04-26T21:54:39.852 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.849+0000 7fb8a439d740 10 client.15168 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-26T21:54:39.852 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.849+0000 7fb8a439d740 20 client.15168 populate_metadata read hostname 'smithi072' 2024-04-26T21:54:39.852 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.849+0000 7fb8a439d740 1 --2- 172.21.15.72:0/4089548036 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x55607b24c780 0x55607b24b490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:39.852 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.849+0000 7fb8a439d740 4 client.15168 mds_command: new command op to 34517 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T21:54:39.852 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.849+0000 7fb8a439d740 1 -- 172.21.15.72:0/4089548036 --> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] -- command(tid 0: {"prefix": "status"}) -- 0x55607b136080 con 0x55607b24c780 2024-04-26T21:54:39.854 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.850+0000 7fb89aa00640 1 --2- 172.21.15.72:0/4089548036 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x55607b24c780 0x55607b24b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:39.854 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.850+0000 7fb89aa00640 1 --2- 172.21.15.72:0/4089548036 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x55607b24c780 0x55607b24b490 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1cbefa228934549c server_cookie=bdc24638441a05ab in_seq=0 out_seq=0 2024-04-26T21:54:39.854 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.850+0000 7fb87be00640 10 client.15168 ms_handle_connect on v2:172.21.15.76:6838/327145636 2024-04-26T21:54:39.854 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.851+0000 7fb87be00640 1 -- 172.21.15.72:0/4089548036 <== mds.0 v2:172.21.15.76:6838/327145636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55607b136080 con 0x55607b24c780 2024-04-26T21:54:39.854 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.851+0000 7fb87be00640 10 client.15168 handle_command_reply: tid=0 2024-04-26T21:54:39.854 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.851+0000 7fb8a439d740 1 client.15168 shutdown 2024-04-26T21:54:39.854 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.851+0000 7fb8a439d740 1 -- 172.21.15.72:0/4089548036 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55607b1cfe20 msgr2=0x55607b1dae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:39.855 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.851+0000 7fb8a439d740 1 --2- 172.21.15.72:0/4089548036 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55607b1cfe20 0x55607b1dae90 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fb89400fee0 tx=0x7fb894004ab0 comp rx=0 tx=0).stop 2024-04-26T21:54:39.855 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.851+0000 7fb8a439d740 1 -- 172.21.15.72:0/4089548036 shutdown_connections 2024-04-26T21:54:39.855 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.851+0000 7fb8a439d740 1 --2- 172.21.15.72:0/4089548036 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x55607b24c780 0x55607b24b490 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:39.855 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.851+0000 7fb8a439d740 1 --2- 172.21.15.72:0/4089548036 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55607b1cfe20 0x55607b1dae90 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:39.855 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.851+0000 7fb8a439d740 1 --2- 172.21.15.72:0/4089548036 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55607b1cacb0 0x55607b1d8340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:39.855 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.851+0000 7fb8a439d740 1 --2- 172.21.15.72:0/4089548036 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55607b1ca2c0 0x55607b1d7e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:39.855 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.851+0000 7fb8a439d740 1 -- 172.21.15.72:0/4089548036 >> 172.21.15.72:0/4089548036 conn(0x55607b15ed10 msgr2=0x55607b1c4f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:54:39.855 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.851+0000 7fb87be00640 0 client.15168 ms_handle_reset on v2:172.21.15.76:6838/327145636 2024-04-26T21:54:39.855 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.851+0000 7fb8a439d740 1 -- 172.21.15.72:0/4089548036 shutdown_connections 2024-04-26T21:54:39.855 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.851+0000 7fb8a439d740 1 -- 172.21.15.72:0/4089548036 wait complete. 2024-04-26T21:54:39.855 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.852+0000 7fb8a439d740 20 client.15168 trim_cache size 0 max 16384 2024-04-26T21:54:39.856 INFO:teuthology.orchestra.run.smithi072.stdout:{ 2024-04-26T21:54:39.856 INFO:teuthology.orchestra.run.smithi072.stdout: "cluster_fsid": "a1780950-0415-11ef-bc93-c7b262605968", 2024-04-26T21:54:39.856 INFO:teuthology.orchestra.run.smithi072.stdout: "whoami": 0, 2024-04-26T21:54:39.856 INFO:teuthology.orchestra.run.smithi072.stdout: "id": 34517, 2024-04-26T21:54:39.856 INFO:teuthology.orchestra.run.smithi072.stdout: "want_state": "up:active", 2024-04-26T21:54:39.856 INFO:teuthology.orchestra.run.smithi072.stdout: "state": "up:active", 2024-04-26T21:54:39.856 INFO:teuthology.orchestra.run.smithi072.stdout: "fs_name": "cephfs", 2024-04-26T21:54:39.856 INFO:teuthology.orchestra.run.smithi072.stdout: "rank_uptime": 108.74997907700001, 2024-04-26T21:54:39.856 INFO:teuthology.orchestra.run.smithi072.stdout: "mdsmap_epoch": 26, 2024-04-26T21:54:39.856 INFO:teuthology.orchestra.run.smithi072.stdout: "osdmap_epoch": 91, 2024-04-26T21:54:39.856 INFO:teuthology.orchestra.run.smithi072.stdout: "osdmap_epoch_barrier": 87, 2024-04-26T21:54:39.856 INFO:teuthology.orchestra.run.smithi072.stdout: "uptime": 115.026548335 2024-04-26T21:54:39.856 INFO:teuthology.orchestra.run.smithi072.stdout:} 2024-04-26T21:54:39.857 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.852+0000 7fb89be00640 1 -- 172.21.15.72:0/300466996 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fb8800a3cc0 msgr2=0x7fb8800a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:39.857 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.852+0000 7fb89be00640 1 --2- 172.21.15.72:0/300466996 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fb8800a3cc0 0x7fb8800a6180 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x55607b1cb510 tx=0x7fb89400a830 comp rx=0 tx=0).stop 2024-04-26T21:54:39.857 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.852+0000 7fb89be00640 1 -- 172.21.15.72:0/300466996 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb89c109f80 msgr2=0x7fb89c19f380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:39.857 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.852+0000 7fb89be00640 1 --2- 172.21.15.72:0/300466996 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb89c109f80 0x7fb89c19f380 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fb88800c970 tx=0x7fb88800ce30 comp rx=0 tx=0).stop 2024-04-26T21:54:39.857 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.852+0000 7fb89be00640 1 -- 172.21.15.72:0/300466996 shutdown_connections 2024-04-26T21:54:39.858 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.852+0000 7fb89be00640 1 --2- 172.21.15.72:0/300466996 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb89c10a940 0x7fb89c1a63f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:39.858 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.852+0000 7fb89be00640 1 --2- 172.21.15.72:0/300466996 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb89c109f80 0x7fb89c19f380 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:39.858 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.852+0000 7fb89be00640 1 --2- 172.21.15.72:0/300466996 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fb8800a3cc0 0x7fb8800a6180 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:39.858 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.852+0000 7fb89be00640 1 --2- 172.21.15.72:0/300466996 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb89c108d90 0x7fb89c19ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:39.858 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.852+0000 7fb89be00640 1 -- 172.21.15.72:0/300466996 >> 172.21.15.72:0/300466996 conn(0x7fb89c077d30 msgr2=0x7fb89c10d280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:54:39.858 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.852+0000 7fb89be00640 1 -- 172.21.15.72:0/300466996 shutdown_connections 2024-04-26T21:54:39.858 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:39.852+0000 7fb89be00640 1 -- 172.21.15.72:0/300466996 wait complete. 2024-04-26T21:54:39.869 INFO:tasks.cephadm:Setup complete, yielding 2024-04-26T21:54:39.870 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-26T21:54:39.880 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi072.front.sepia.ceph.com 2024-04-26T21:54:39.880 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a1780950-0415-11ef-bc93-c7b262605968 -- bash -c 'ceph orch status' 2024-04-26T21:54:40.307 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/a1780950-0415-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:54:40.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:54:40 smithi076 ceph-mon[31823]: pgmap v426: 97 pgs: 97 active+clean; 633 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-26T21:54:40.321 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:54:40 smithi072 ceph-mon[28138]: pgmap v426: 97 pgs: 97 active+clean; 633 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-26T21:54:40.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:54:40 smithi164 ceph-mon[31916]: pgmap v426: 97 pgs: 97 active+clean; 633 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-26T21:54:42.510 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:42.508+0000 7fcb47e00700 1 -- 172.21.15.72:0/1547943040 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcb40101d00 msgr2=0x7fcb40104130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:42.510 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:42.508+0000 7fcb47e00700 1 --2- 172.21.15.72:0/1547943040 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcb40101d00 0x7fcb40104130 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fcb34009990 tx=0x7fcb34009ca0 comp rx=0 tx=0).stop 2024-04-26T21:54:42.510 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:42.508+0000 7fcb47e00700 1 -- 172.21.15.72:0/1547943040 shutdown_connections 2024-04-26T21:54:42.510 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:42.508+0000 7fcb47e00700 1 --2- 172.21.15.72:0/1547943040 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcb40101d00 0x7fcb40104130 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:42.510 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:42.508+0000 7fcb47e00700 1 --2- 172.21.15.72:0/1547943040 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcb40107040 0x7fcb401074c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:42.510 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:42.508+0000 7fcb47e00700 1 --2- 172.21.15.72:0/1547943040 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcb40104670 0x7fcb40106b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:42.510 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:42.508+0000 7fcb47e00700 1 -- 172.21.15.72:0/1547943040 >> 172.21.15.72:0/1547943040 conn(0x7fcb400fd920 msgr2=0x7fcb400ffd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:54:42.510 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:42.508+0000 7fcb47e00700 1 -- 172.21.15.72:0/1547943040 shutdown_connections 2024-04-26T21:54:42.510 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:42.508+0000 7fcb47e00700 1 -- 172.21.15.72:0/1547943040 wait complete. 2024-04-26T21:54:42.510 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:42.509+0000 7fcb47e00700 1 Processor -- start 2024-04-26T21:54:42.511 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:42.510+0000 7fcb47e00700 1 -- start start 2024-04-26T21:54:42.512 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:42.510+0000 7fcb47e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcb40101d00 0x7fcb4019fab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:42.512 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:42.510+0000 7fcb47e00700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcb40104670 0x7fcb4019fff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:42.512 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:42.510+0000 7fcb47e00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcb40107040 0x7fcb401a7020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:42.512 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:42.510+0000 7fcb47e00700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fcb40115340 con 0x7fcb40107040 2024-04-26T21:54:42.512 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:42.510+0000 7fcb47e00700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fcb401151c0 con 0x7fcb40104670 2024-04-26T21:54:42.512 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:42.510+0000 7fcb47e00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fcb401154c0 con 0x7fcb40101d00 2024-04-26T21:54:42.512 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:42.510+0000 7fcb45e00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcb40107040 0x7fcb401a7020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:42.513 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:42.510+0000 7fcb45e00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcb40107040 0x7fcb401a7020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:49772/0 (socket says 172.21.15.72:49772) 2024-04-26T21:54:42.513 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:42.510+0000 7fcb45e00700 1 -- 172.21.15.72:0/455144644 learned_addr learned my addr 172.21.15.72:0/455144644 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:54:42.513 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:42.511+0000 7fcb45400700 1 --2- 172.21.15.72:0/455144644 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcb40101d00 0x7fcb4019fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:42.513 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:42.511+0000 7fcb44a00700 1 --2- 172.21.15.72:0/455144644 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcb40104670 0x7fcb4019fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:42.513 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:42.511+0000 7fcb45e00700 1 -- 172.21.15.72:0/455144644 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcb40101d00 msgr2=0x7fcb4019fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:42.513 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:42.511+0000 7fcb45e00700 1 --2- 172.21.15.72:0/455144644 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcb40101d00 0x7fcb4019fab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:42.513 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:42.511+0000 7fcb45e00700 1 -- 172.21.15.72:0/455144644 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcb40104670 msgr2=0x7fcb4019fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:42.513 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:42.511+0000 7fcb45e00700 1 --2- 172.21.15.72:0/455144644 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcb40104670 0x7fcb4019fff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:42.513 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:42.511+0000 7fcb45e00700 1 -- 172.21.15.72:0/455144644 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb401a76c0 con 0x7fcb40107040 2024-04-26T21:54:42.513 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:42.511+0000 7fcb45e00700 1 --2- 172.21.15.72:0/455144644 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcb40107040 0x7fcb401a7020 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fcb38007da0 tx=0x7fcb3800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:54:42.513 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:42.511+0000 7fcb3e000700 1 -- 172.21.15.72:0/455144644 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb38016020 con 0x7fcb40107040 2024-04-26T21:54:42.513 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:42.511+0000 7fcb3e000700 1 -- 172.21.15.72:0/455144644 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcb3800ad00 con 0x7fcb40107040 2024-04-26T21:54:42.513 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:42.511+0000 7fcb47e00700 1 -- 172.21.15.72:0/455144644 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb401a79b0 con 0x7fcb40107040 2024-04-26T21:54:42.513 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:42.511+0000 7fcb3e000700 1 -- 172.21.15.72:0/455144644 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb380115e0 con 0x7fcb40107040 2024-04-26T21:54:42.513 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:42.512+0000 7fcb47e00700 1 -- 172.21.15.72:0/455144644 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb401a7ea0 con 0x7fcb40107040 2024-04-26T21:54:42.514 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:42.513+0000 7fcb47e00700 1 -- 172.21.15.72:0/455144644 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb4006a110 con 0x7fcb40107040 2024-04-26T21:54:42.515 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:42.515+0000 7fcb3e000700 1 -- 172.21.15.72:0/455144644 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fcb38019020 con 0x7fcb40107040 2024-04-26T21:54:42.516 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:42.516+0000 7fcb3e000700 1 --2- 172.21.15.72:0/455144644 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fcb280a3cd0 0x7fcb280a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:42.516 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:42.516+0000 7fcb3e000700 1 -- 172.21.15.72:0/455144644 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7fcb380d05a0 con 0x7fcb40107040 2024-04-26T21:54:42.517 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:42.516+0000 7fcb45400700 1 --2- 172.21.15.72:0/455144644 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fcb280a3cd0 0x7fcb280a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:42.517 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:42.517+0000 7fcb45400700 1 --2- 172.21.15.72:0/455144644 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fcb280a3cd0 0x7fcb280a6190 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fcb3400b5c0 tx=0x7fcb34005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:54:42.521 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:42.521+0000 7fcb3e000700 1 -- 172.21.15.72:0/455144644 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fcb38020020 con 0x7fcb40107040 2024-04-26T21:54:42.769 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:42.769+0000 7fcb47e00700 1 -- 172.21.15.72:0/455144644 --> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7fcb40063860 con 0x7fcb280a3cd0 2024-04-26T21:54:42.772 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:42.772+0000 7fcb3e000700 1 -- 172.21.15.72:0/455144644 <== mgr.14152 v2:172.21.15.72:6800/588150172 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7fcb40063860 con 0x7fcb280a3cd0 2024-04-26T21:54:42.772 INFO:teuthology.orchestra.run.smithi072.stdout:Backend: cephadm 2024-04-26T21:54:42.773 INFO:teuthology.orchestra.run.smithi072.stdout:Available: Yes 2024-04-26T21:54:42.773 INFO:teuthology.orchestra.run.smithi072.stdout:Paused: No 2024-04-26T21:54:42.775 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:42.774+0000 7fcb47e00700 1 -- 172.21.15.72:0/455144644 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fcb280a3cd0 msgr2=0x7fcb280a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:42.775 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:42.774+0000 7fcb47e00700 1 --2- 172.21.15.72:0/455144644 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fcb280a3cd0 0x7fcb280a6190 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fcb3400b5c0 tx=0x7fcb34005fb0 comp rx=0 tx=0).stop 2024-04-26T21:54:42.775 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:42.775+0000 7fcb47e00700 1 -- 172.21.15.72:0/455144644 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcb40107040 msgr2=0x7fcb401a7020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:42.775 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:42.775+0000 7fcb47e00700 1 --2- 172.21.15.72:0/455144644 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcb40107040 0x7fcb401a7020 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fcb38007da0 tx=0x7fcb3800a340 comp rx=0 tx=0).stop 2024-04-26T21:54:42.776 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:42.775+0000 7fcb47e00700 1 -- 172.21.15.72:0/455144644 shutdown_connections 2024-04-26T21:54:42.776 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:42.775+0000 7fcb47e00700 1 --2- 172.21.15.72:0/455144644 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fcb280a3cd0 0x7fcb280a6190 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:42.776 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:42.775+0000 7fcb47e00700 1 --2- 172.21.15.72:0/455144644 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcb40107040 0x7fcb401a7020 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:42.776 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:42.775+0000 7fcb47e00700 1 --2- 172.21.15.72:0/455144644 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcb40101d00 0x7fcb4019fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:42.776 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:42.775+0000 7fcb47e00700 1 --2- 172.21.15.72:0/455144644 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcb40104670 0x7fcb4019fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:42.776 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:42.775+0000 7fcb47e00700 1 -- 172.21.15.72:0/455144644 >> 172.21.15.72:0/455144644 conn(0x7fcb400fd920 msgr2=0x7fcb40106500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:54:42.776 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:42.775+0000 7fcb47e00700 1 -- 172.21.15.72:0/455144644 shutdown_connections 2024-04-26T21:54:42.776 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:42.775+0000 7fcb47e00700 1 -- 172.21.15.72:0/455144644 wait complete. 2024-04-26T21:54:43.071 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:54:42 smithi072 ceph-mon[28138]: pgmap v427: 97 pgs: 97 active+clean; 633 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:54:43.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:54:42 smithi164 ceph-mon[31916]: pgmap v427: 97 pgs: 97 active+clean; 633 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:54:43.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:54:42 smithi076 ceph-mon[31823]: pgmap v427: 97 pgs: 97 active+clean; 633 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:54:43.537 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a1780950-0415-11ef-bc93-c7b262605968 -- bash -c 'ceph orch ps' 2024-04-26T21:54:43.987 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/a1780950-0415-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:54:44.132 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:54:43 smithi072 ceph-mon[28138]: from='client.15174 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:54:44.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:54:43 smithi164 ceph-mon[31916]: from='client.15174 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:54:44.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:54:43 smithi076 ceph-mon[31823]: from='client.15174 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:54:45.170 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:54:44 smithi072 ceph-mon[28138]: pgmap v428: 97 pgs: 97 active+clean; 633 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-26T21:54:45.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:54:44 smithi164 ceph-mon[31916]: pgmap v428: 97 pgs: 97 active+clean; 633 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-26T21:54:45.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:54:44 smithi076 ceph-mon[31823]: pgmap v428: 97 pgs: 97 active+clean; 633 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-26T21:54:46.173 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:46.172+0000 7fc0bec00700 1 -- 172.21.15.72:0/2360899909 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc0b810b660 msgr2=0x7fc0b8111f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:46.173 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:46.172+0000 7fc0bec00700 1 --2- 172.21.15.72:0/2360899909 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc0b810b660 0x7fc0b8111f10 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fc0b000b0d0 tx=0x7fc0b000b3e0 comp rx=0 tx=0).stop 2024-04-26T21:54:46.173 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:46.172+0000 7fc0bec00700 1 -- 172.21.15.72:0/2360899909 shutdown_connections 2024-04-26T21:54:46.173 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:46.172+0000 7fc0bec00700 1 --2- 172.21.15.72:0/2360899909 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc0b810b660 0x7fc0b8111f10 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:46.173 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:46.172+0000 7fc0bec00700 1 --2- 172.21.15.72:0/2360899909 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc0b8109ac0 0x7fc0b8109ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:46.173 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:46.172+0000 7fc0bec00700 1 --2- 172.21.15.72:0/2360899909 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc0b810acc0 0x7fc0b810b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:46.173 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:46.172+0000 7fc0bec00700 1 -- 172.21.15.72:0/2360899909 >> 172.21.15.72:0/2360899909 conn(0x7fc0b8078a90 msgr2=0x7fc0b807aed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:54:46.173 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:46.172+0000 7fc0bec00700 1 -- 172.21.15.72:0/2360899909 shutdown_connections 2024-04-26T21:54:46.173 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:46.172+0000 7fc0bec00700 1 -- 172.21.15.72:0/2360899909 wait complete. 2024-04-26T21:54:46.175 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:46.173+0000 7fc0bec00700 1 Processor -- start 2024-04-26T21:54:46.175 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:46.173+0000 7fc0bec00700 1 -- start start 2024-04-26T21:54:46.175 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:46.174+0000 7fc0bec00700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc0b8109ac0 0x7fc0b819f930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:46.175 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:46.174+0000 7fc0bec00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc0b810acc0 0x7fc0b819fe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:46.175 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:46.174+0000 7fc0bec00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc0b810b660 0x7fc0b81a6ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:46.176 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:46.174+0000 7fc0bec00700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fc0b8115440 con 0x7fc0b810acc0 2024-04-26T21:54:46.176 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:46.174+0000 7fc0bec00700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fc0b81152c0 con 0x7fc0b8109ac0 2024-04-26T21:54:46.176 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:46.174+0000 7fc0bec00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fc0b81155c0 con 0x7fc0b810b660 2024-04-26T21:54:46.176 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:46.174+0000 7fc0b7400700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc0b810acc0 0x7fc0b819fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:46.176 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:46.174+0000 7fc0b7400700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc0b810acc0 0x7fc0b819fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:49786/0 (socket says 172.21.15.72:49786) 2024-04-26T21:54:46.176 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:46.174+0000 7fc0b7400700 1 -- 172.21.15.72:0/1834201014 learned_addr learned my addr 172.21.15.72:0/1834201014 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:54:46.176 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:46.174+0000 7fc0bcc00700 1 --2- 172.21.15.72:0/1834201014 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc0b810b660 0x7fc0b81a6ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:46.176 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:46.174+0000 7fc0b7e00700 1 --2- 172.21.15.72:0/1834201014 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc0b8109ac0 0x7fc0b819f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:46.176 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:46.174+0000 7fc0b7400700 1 -- 172.21.15.72:0/1834201014 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc0b810b660 msgr2=0x7fc0b81a6ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:46.176 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:46.174+0000 7fc0b7400700 1 --2- 172.21.15.72:0/1834201014 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc0b810b660 0x7fc0b81a6ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:46.176 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:46.174+0000 7fc0b7400700 1 -- 172.21.15.72:0/1834201014 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc0b8109ac0 msgr2=0x7fc0b819f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:46.176 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:46.174+0000 7fc0b7400700 1 --2- 172.21.15.72:0/1834201014 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc0b8109ac0 0x7fc0b819f930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:46.176 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:46.175+0000 7fc0b7400700 1 -- 172.21.15.72:0/1834201014 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0b81a7590 con 0x7fc0b810acc0 2024-04-26T21:54:46.177 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:46.175+0000 7fc0b7400700 1 --2- 172.21.15.72:0/1834201014 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc0b810acc0 0x7fc0b819fe70 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fc0a400c9e0 tx=0x7fc0a400cda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:54:46.177 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:46.175+0000 7fc0b4c00700 1 -- 172.21.15.72:0/1834201014 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0a4007ac0 con 0x7fc0b810acc0 2024-04-26T21:54:46.177 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:46.175+0000 7fc0b4c00700 1 -- 172.21.15.72:0/1834201014 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc0a4007c60 con 0x7fc0b810acc0 2024-04-26T21:54:46.177 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:46.175+0000 7fc0bec00700 1 -- 172.21.15.72:0/1834201014 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0b81a7880 con 0x7fc0b810acc0 2024-04-26T21:54:46.177 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:46.175+0000 7fc0b4c00700 1 -- 172.21.15.72:0/1834201014 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0a4018660 con 0x7fc0b810acc0 2024-04-26T21:54:46.177 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:46.175+0000 7fc0bec00700 1 -- 172.21.15.72:0/1834201014 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0b81a7dc0 con 0x7fc0b810acc0 2024-04-26T21:54:46.177 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:46.176+0000 7fc0bec00700 1 -- 172.21.15.72:0/1834201014 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0b806a110 con 0x7fc0b810acc0 2024-04-26T21:54:46.178 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:46.178+0000 7fc0b4c00700 1 -- 172.21.15.72:0/1834201014 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fc0a400f450 con 0x7fc0b810acc0 2024-04-26T21:54:46.180 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:46.180+0000 7fc0b4c00700 1 --2- 172.21.15.72:0/1834201014 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fc09c0a3cd0 0x7fc09c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:46.180 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:46.180+0000 7fc0b4c00700 1 -- 172.21.15.72:0/1834201014 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7fc0a40d0670 con 0x7fc0b810acc0 2024-04-26T21:54:46.180 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:46.180+0000 7fc0b7e00700 1 --2- 172.21.15.72:0/1834201014 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fc09c0a3cd0 0x7fc09c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:46.180 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:46.181+0000 7fc0b7e00700 1 --2- 172.21.15.72:0/1834201014 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fc09c0a3cd0 0x7fc09c0a6190 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fc0a8005fd0 tx=0x7fc0a8005e50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:54:46.184 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:46.184+0000 7fc0b4c00700 1 -- 172.21.15.72:0/1834201014 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fc0a4014030 con 0x7fc0b810acc0 2024-04-26T21:54:46.425 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:46.424+0000 7fc0bec00700 1 -- 172.21.15.72:0/1834201014 --> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7fc0b8063860 con 0x7fc09c0a3cd0 2024-04-26T21:54:46.465 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:46.465+0000 7fc0b4c00700 1 -- 172.21.15.72:0/1834201014 <== mgr.14152 v2:172.21.15.72:6800/588150172 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4123 (secure 0 0 0) 0x7fc0b8063860 con 0x7fc09c0a3cd0 2024-04-26T21:54:46.466 INFO:teuthology.orchestra.run.smithi072.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-26T21:54:46.466 INFO:teuthology.orchestra.run.smithi072.stdout:mds.a smithi072 running (2m) 22s ago 2m 16.1M - 19.0.0-3306-gbc612ffb 55dc54019ad8 fb63d88d4bc4 2024-04-26T21:54:46.467 INFO:teuthology.orchestra.run.smithi072.stdout:mds.b smithi076 running (2m) 49s ago 2m 24.5M - 19.0.0-3306-gbc612ffb 55dc54019ad8 3e314c4fe51d 2024-04-26T21:54:46.467 INFO:teuthology.orchestra.run.smithi072.stdout:mds.c smithi164 running (2m) 50s ago 2m 16.7M - 19.0.0-3306-gbc612ffb 55dc54019ad8 a2fd1d91709b 2024-04-26T21:54:46.467 INFO:teuthology.orchestra.run.smithi072.stdout:mds.d smithi072 running (2m) 22s ago 2m 12.5M - 19.0.0-3306-gbc612ffb 55dc54019ad8 d54a4c15c683 2024-04-26T21:54:46.467 INFO:teuthology.orchestra.run.smithi072.stdout:mds.e smithi076 running (2m) 49s ago 2m 16.3M - 19.0.0-3306-gbc612ffb 55dc54019ad8 5c97ebe3d413 2024-04-26T21:54:46.467 INFO:teuthology.orchestra.run.smithi072.stdout:mds.f smithi164 running (2m) 50s ago 2m 15.4M - 19.0.0-3306-gbc612ffb 55dc54019ad8 3bb4b5de753c 2024-04-26T21:54:46.467 INFO:teuthology.orchestra.run.smithi072.stdout:mds.g smithi072 running (2m) 22s ago 2m 16.2M - 19.0.0-3306-gbc612ffb 55dc54019ad8 e193c142ef16 2024-04-26T21:54:46.468 INFO:teuthology.orchestra.run.smithi072.stdout:mds.h smithi076 running (2m) 49s ago 2m 16.3M - 19.0.0-3306-gbc612ffb 55dc54019ad8 13e97c00bd32 2024-04-26T21:54:46.468 INFO:teuthology.orchestra.run.smithi072.stdout:mds.i smithi164 running (2m) 50s ago 2m 15.8M - 19.0.0-3306-gbc612ffb 55dc54019ad8 9b322e4f6c6d 2024-04-26T21:54:46.468 INFO:teuthology.orchestra.run.smithi072.stdout:mds.j smithi072 running (2m) 22s ago 2m 12.6M - 19.0.0-3306-gbc612ffb 55dc54019ad8 efaf9a1d4cd2 2024-04-26T21:54:46.468 INFO:teuthology.orchestra.run.smithi072.stdout:mds.k smithi076 running (2m) 49s ago 2m 16.3M - 19.0.0-3306-gbc612ffb 55dc54019ad8 c0523893e3a1 2024-04-26T21:54:46.468 INFO:teuthology.orchestra.run.smithi072.stdout:mds.l smithi164 running (2m) 50s ago 2m 15.5M - 19.0.0-3306-gbc612ffb 55dc54019ad8 781cee5595e7 2024-04-26T21:54:46.468 INFO:teuthology.orchestra.run.smithi072.stdout:mgr.x smithi072 *:9283,8765 running (12m) 22s ago 12m 515M - 19.0.0-3306-gbc612ffb 55dc54019ad8 dd3246b2bcf1 2024-04-26T21:54:46.469 INFO:teuthology.orchestra.run.smithi072.stdout:mgr.y smithi076 *:8443,8765 running (9m) 49s ago 9m 452M - 19.0.0-3306-gbc612ffb 55dc54019ad8 94ad75b255db 2024-04-26T21:54:46.469 INFO:teuthology.orchestra.run.smithi072.stdout:mgr.z smithi164 *:8443,8765 running (9m) 50s ago 9m 452M - 19.0.0-3306-gbc612ffb 55dc54019ad8 28ad3af3bba1 2024-04-26T21:54:46.469 INFO:teuthology.orchestra.run.smithi072.stdout:mon.a smithi072 running (12m) 22s ago 12m 50.9M 2048M 19.0.0-3306-gbc612ffb 55dc54019ad8 bf7f10eb1c09 2024-04-26T21:54:46.469 INFO:teuthology.orchestra.run.smithi072.stdout:mon.b smithi076 running (10m) 49s ago 10m 39.8M 2048M 19.0.0-3306-gbc612ffb 55dc54019ad8 3c5a685d42be 2024-04-26T21:54:46.469 INFO:teuthology.orchestra.run.smithi072.stdout:mon.c smithi164 running (10m) 50s ago 10m 38.9M 2048M 19.0.0-3306-gbc612ffb 55dc54019ad8 acf9ab094680 2024-04-26T21:54:46.469 INFO:teuthology.orchestra.run.smithi072.stdout:osd.0 smithi072 running (9m) 22s ago 9m 55.8M 4272M 19.0.0-3306-gbc612ffb 55dc54019ad8 4cb4f728f75d 2024-04-26T21:54:46.469 INFO:teuthology.orchestra.run.smithi072.stdout:osd.1 smithi076 running (8m) 49s ago 8m 48.8M 4272M 19.0.0-3306-gbc612ffb 55dc54019ad8 fbed269c3f57 2024-04-26T21:54:46.469 INFO:teuthology.orchestra.run.smithi072.stdout:osd.2 smithi164 running (8m) 50s ago 8m 58.1M 4272M 19.0.0-3306-gbc612ffb 55dc54019ad8 60a3c1cb6581 2024-04-26T21:54:46.470 INFO:teuthology.orchestra.run.smithi072.stdout:osd.3 smithi072 running (7m) 22s ago 7m 46.2M 4272M 19.0.0-3306-gbc612ffb 55dc54019ad8 b0c2882cf052 2024-04-26T21:54:46.470 INFO:teuthology.orchestra.run.smithi072.stdout:osd.4 smithi076 running (7m) 49s ago 7m 51.9M 4272M 19.0.0-3306-gbc612ffb 55dc54019ad8 690a9f400b41 2024-04-26T21:54:46.470 INFO:teuthology.orchestra.run.smithi072.stdout:osd.5 smithi164 running (6m) 50s ago 6m 52.9M 4272M 19.0.0-3306-gbc612ffb 55dc54019ad8 e91b532bd968 2024-04-26T21:54:46.470 INFO:teuthology.orchestra.run.smithi072.stdout:osd.6 smithi072 running (6m) 22s ago 6m 52.0M 4272M 19.0.0-3306-gbc612ffb 55dc54019ad8 617ccd1509f1 2024-04-26T21:54:46.470 INFO:teuthology.orchestra.run.smithi072.stdout:osd.7 smithi076 running (5m) 49s ago 5m 45.7M 4272M 19.0.0-3306-gbc612ffb 55dc54019ad8 ba938167c1fd 2024-04-26T21:54:46.470 INFO:teuthology.orchestra.run.smithi072.stdout:osd.8 smithi164 running (4m) 50s ago 4m 53.4M 4272M 19.0.0-3306-gbc612ffb 55dc54019ad8 54bf3f3060a0 2024-04-26T21:54:46.470 INFO:teuthology.orchestra.run.smithi072.stdout:osd.9 smithi072 running (4m) 22s ago 4m 51.0M 4272M 19.0.0-3306-gbc612ffb 55dc54019ad8 9741ab8d063b 2024-04-26T21:54:46.470 INFO:teuthology.orchestra.run.smithi072.stdout:osd.10 smithi076 running (3m) 49s ago 3m 47.5M 4272M 19.0.0-3306-gbc612ffb 55dc54019ad8 4fce955adb6c 2024-04-26T21:54:46.470 INFO:teuthology.orchestra.run.smithi072.stdout:osd.11 smithi164 running (3m) 50s ago 3m 42.2M 4272M 19.0.0-3306-gbc612ffb 55dc54019ad8 ff2513c70c06 2024-04-26T21:54:46.471 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:46.469+0000 7fc0bec00700 1 -- 172.21.15.72:0/1834201014 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fc09c0a3cd0 msgr2=0x7fc09c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:46.471 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:46.470+0000 7fc0bec00700 1 --2- 172.21.15.72:0/1834201014 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fc09c0a3cd0 0x7fc09c0a6190 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fc0a8005fd0 tx=0x7fc0a8005e50 comp rx=0 tx=0).stop 2024-04-26T21:54:46.471 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:46.470+0000 7fc0bec00700 1 -- 172.21.15.72:0/1834201014 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc0b810acc0 msgr2=0x7fc0b819fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:46.471 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:46.470+0000 7fc0bec00700 1 --2- 172.21.15.72:0/1834201014 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc0b810acc0 0x7fc0b819fe70 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fc0a400c9e0 tx=0x7fc0a400cda0 comp rx=0 tx=0).stop 2024-04-26T21:54:46.471 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:46.470+0000 7fc0bec00700 1 -- 172.21.15.72:0/1834201014 shutdown_connections 2024-04-26T21:54:46.471 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:46.470+0000 7fc0bec00700 1 --2- 172.21.15.72:0/1834201014 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fc09c0a3cd0 0x7fc09c0a6190 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:46.471 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:46.470+0000 7fc0bec00700 1 --2- 172.21.15.72:0/1834201014 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc0b810b660 0x7fc0b81a6ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:46.472 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:46.470+0000 7fc0bec00700 1 --2- 172.21.15.72:0/1834201014 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc0b8109ac0 0x7fc0b819f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:46.472 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:46.470+0000 7fc0bec00700 1 --2- 172.21.15.72:0/1834201014 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc0b810acc0 0x7fc0b819fe70 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:46.472 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:46.470+0000 7fc0bec00700 1 -- 172.21.15.72:0/1834201014 >> 172.21.15.72:0/1834201014 conn(0x7fc0b8078a90 msgr2=0x7fc0b810def0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:54:46.472 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:46.470+0000 7fc0bec00700 1 -- 172.21.15.72:0/1834201014 shutdown_connections 2024-04-26T21:54:46.472 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:46.470+0000 7fc0bec00700 1 -- 172.21.15.72:0/1834201014 wait complete. 2024-04-26T21:54:47.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:54:46 smithi164 ceph-mon[31916]: pgmap v429: 97 pgs: 97 active+clean; 633 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-26T21:54:47.214 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:54:46 smithi072 ceph-mon[28138]: pgmap v429: 97 pgs: 97 active+clean; 633 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-26T21:54:47.266 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a1780950-0415-11ef-bc93-c7b262605968 -- bash -c 'ceph orch ls' 2024-04-26T21:54:47.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:54:46 smithi076 ceph-mon[31823]: pgmap v429: 97 pgs: 97 active+clean; 633 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-26T21:54:47.705 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/a1780950-0415-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:54:48.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:54:47 smithi164 ceph-mon[31916]: from='client.15180 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:54:48.213 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:54:47 smithi072 ceph-mon[28138]: from='client.15180 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:54:48.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:54:47 smithi076 ceph-mon[31823]: from='client.15180 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:54:49.071 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:54:48 smithi072 ceph-mon[28138]: pgmap v430: 97 pgs: 97 active+clean; 633 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:54:49.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:54:48 smithi164 ceph-mon[31916]: pgmap v430: 97 pgs: 97 active+clean; 633 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:54:49.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:54:48 smithi076 ceph-mon[31823]: pgmap v430: 97 pgs: 97 active+clean; 633 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:54:49.985 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:49.984+0000 7f3d4d200700 1 -- 172.21.15.72:0/1791557291 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3d48103080 msgr2=0x7f3d48103500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:49.985 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:49.984+0000 7f3d4d200700 1 --2- 172.21.15.72:0/1791557291 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3d48103080 0x7f3d48103500 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f3d4000b0a0 tx=0x7f3d4000b3b0 comp rx=0 tx=0).stop 2024-04-26T21:54:49.986 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:49.984+0000 7f3d4d200700 1 -- 172.21.15.72:0/1791557291 shutdown_connections 2024-04-26T21:54:49.986 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:49.984+0000 7f3d4d200700 1 --2- 172.21.15.72:0/1791557291 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3d48103080 0x7f3d48103500 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:49.986 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:49.984+0000 7f3d4d200700 1 --2- 172.21.15.72:0/1791557291 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3d48101d60 0x7f3d48102180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:49.986 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:49.984+0000 7f3d4d200700 1 --2- 172.21.15.72:0/1791557291 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3d481026c0 0x7f3d48102b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:49.986 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:49.984+0000 7f3d4d200700 1 -- 172.21.15.72:0/1791557291 >> 172.21.15.72:0/1791557291 conn(0x7f3d480fd920 msgr2=0x7f3d480ffd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:54:49.986 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:49.984+0000 7f3d4d200700 1 -- 172.21.15.72:0/1791557291 shutdown_connections 2024-04-26T21:54:49.986 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:49.984+0000 7f3d4d200700 1 -- 172.21.15.72:0/1791557291 wait complete. 2024-04-26T21:54:49.987 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:49.986+0000 7f3d4d200700 1 Processor -- start 2024-04-26T21:54:49.987 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:49.986+0000 7f3d4d200700 1 -- start start 2024-04-26T21:54:49.987 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:49.987+0000 7f3d4d200700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3d48101d60 0x7f3d4819b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:49.987 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:49.987+0000 7f3d4d200700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3d481026c0 0x7f3d4819bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:49.987 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:49.987+0000 7f3d4d200700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3d48103080 0x7f3d481a2c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:49.987 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:49.987+0000 7f3d4d200700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f3d48111000 con 0x7f3d48101d60 2024-04-26T21:54:49.988 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:49.987+0000 7f3d4d200700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f3d48110e80 con 0x7f3d481026c0 2024-04-26T21:54:49.988 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:49.987+0000 7f3d4d200700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f3d48111180 con 0x7f3d48103080 2024-04-26T21:54:49.988 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:49.987+0000 7f3d45e00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3d48101d60 0x7f3d4819b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:49.988 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:49.987+0000 7f3d45e00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3d48101d60 0x7f3d4819b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:49802/0 (socket says 172.21.15.72:49802) 2024-04-26T21:54:49.988 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:49.987+0000 7f3d45e00700 1 -- 172.21.15.72:0/2714478099 learned_addr learned my addr 172.21.15.72:0/2714478099 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:54:49.988 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:49.987+0000 7f3d45400700 1 --2- 172.21.15.72:0/2714478099 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3d481026c0 0x7f3d4819bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:49.989 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:49.987+0000 7f3d46800700 1 --2- 172.21.15.72:0/2714478099 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3d48103080 0x7f3d481a2c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:49.989 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:49.987+0000 7f3d45e00700 1 -- 172.21.15.72:0/2714478099 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3d48103080 msgr2=0x7f3d481a2c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:49.989 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:49.987+0000 7f3d45e00700 1 --2- 172.21.15.72:0/2714478099 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3d48103080 0x7f3d481a2c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:49.989 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:49.987+0000 7f3d45e00700 1 -- 172.21.15.72:0/2714478099 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3d481026c0 msgr2=0x7f3d4819bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:49.989 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:49.987+0000 7f3d45e00700 1 --2- 172.21.15.72:0/2714478099 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3d481026c0 0x7f3d4819bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:49.989 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:49.987+0000 7f3d45e00700 1 -- 172.21.15.72:0/2714478099 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d481a3300 con 0x7f3d48101d60 2024-04-26T21:54:49.989 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:49.988+0000 7f3d45e00700 1 --2- 172.21.15.72:0/2714478099 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3d48101d60 0x7f3d4819b6a0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f3d3800d9b0 tx=0x7f3d3800dd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:54:49.989 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:49.988+0000 7f3d3ea00700 1 -- 172.21.15.72:0/2714478099 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d38017070 con 0x7f3d48101d60 2024-04-26T21:54:49.989 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:49.988+0000 7f3d3ea00700 1 -- 172.21.15.72:0/2714478099 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3d38004830 con 0x7f3d48101d60 2024-04-26T21:54:49.990 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:49.988+0000 7f3d4d200700 1 -- 172.21.15.72:0/2714478099 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d481a35f0 con 0x7f3d48101d60 2024-04-26T21:54:49.990 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:49.988+0000 7f3d3ea00700 1 -- 172.21.15.72:0/2714478099 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d380053a0 con 0x7f3d48101d60 2024-04-26T21:54:49.990 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:49.988+0000 7f3d4d200700 1 -- 172.21.15.72:0/2714478099 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d481a3b30 con 0x7f3d48101d60 2024-04-26T21:54:49.990 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:49.989+0000 7f3d4d200700 1 -- 172.21.15.72:0/2714478099 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d4806a110 con 0x7f3d48101d60 2024-04-26T21:54:49.991 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:49.991+0000 7f3d3ea00700 1 -- 172.21.15.72:0/2714478099 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f3d380049e0 con 0x7f3d48101d60 2024-04-26T21:54:49.993 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:49.993+0000 7f3d3ea00700 1 --2- 172.21.15.72:0/2714478099 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f3d2c0a3d50 0x7f3d2c0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:49.993 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:49.993+0000 7f3d3ea00700 1 -- 172.21.15.72:0/2714478099 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7f3d380d0560 con 0x7f3d48101d60 2024-04-26T21:54:49.993 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:49.993+0000 7f3d45400700 1 --2- 172.21.15.72:0/2714478099 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f3d2c0a3d50 0x7f3d2c0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:49.993 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:49.993+0000 7f3d45400700 1 --2- 172.21.15.72:0/2714478099 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f3d2c0a3d50 0x7f3d2c0a6210 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f3d34005ea0 tx=0x7f3d34005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:54:49.997 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:49.998+0000 7f3d3ea00700 1 -- 172.21.15.72:0/2714478099 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f3d38020020 con 0x7f3d48101d60 2024-04-26T21:54:50.237 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:50.237+0000 7f3d4d200700 1 -- 172.21.15.72:0/2714478099 --> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7f3d48063860 con 0x7f3d2c0a3d50 2024-04-26T21:54:50.245 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:50.245+0000 7f3d3ea00700 1 -- 172.21.15.72:0/2714478099 <== mgr.14152 v2:172.21.15.72:6800/588150172 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7f3d48063860 con 0x7f3d2c0a3d50 2024-04-26T21:54:50.245 INFO:teuthology.orchestra.run.smithi072.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-04-26T21:54:50.245 INFO:teuthology.orchestra.run.smithi072.stdout:mds.all 12/12 53s ago 2m smithi072=a;smithi072=d;smithi072=g;smithi072=j;smithi076=b;smithi076=e;smithi076=h;smithi076=k;smithi164=c;smithi164=f;smithi164=i;smithi164=l;count:12 2024-04-26T21:54:50.245 INFO:teuthology.orchestra.run.smithi072.stdout:mgr 3/3 53s ago 10m smithi072=x;smithi076=y;smithi164=z;count:3 2024-04-26T21:54:50.246 INFO:teuthology.orchestra.run.smithi072.stdout:mon 3/3 53s ago 10m smithi072:172.21.15.72=a;smithi076:172.21.15.76=b;smithi164:172.21.15.164=c;count:3 2024-04-26T21:54:50.246 INFO:teuthology.orchestra.run.smithi072.stdout:osd 12 53s ago - 2024-04-26T21:54:50.249 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:50.248+0000 7f3d4d200700 1 -- 172.21.15.72:0/2714478099 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f3d2c0a3d50 msgr2=0x7f3d2c0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:50.249 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:50.248+0000 7f3d4d200700 1 --2- 172.21.15.72:0/2714478099 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f3d2c0a3d50 0x7f3d2c0a6210 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f3d34005ea0 tx=0x7f3d34005e30 comp rx=0 tx=0).stop 2024-04-26T21:54:50.249 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:50.248+0000 7f3d4d200700 1 -- 172.21.15.72:0/2714478099 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3d48101d60 msgr2=0x7f3d4819b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:50.249 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:50.248+0000 7f3d4d200700 1 --2- 172.21.15.72:0/2714478099 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3d48101d60 0x7f3d4819b6a0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f3d3800d9b0 tx=0x7f3d3800dd70 comp rx=0 tx=0).stop 2024-04-26T21:54:50.249 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:50.248+0000 7f3d4d200700 1 -- 172.21.15.72:0/2714478099 shutdown_connections 2024-04-26T21:54:50.249 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:50.249+0000 7f3d4d200700 1 --2- 172.21.15.72:0/2714478099 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f3d2c0a3d50 0x7f3d2c0a6210 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:50.249 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:50.249+0000 7f3d4d200700 1 --2- 172.21.15.72:0/2714478099 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3d48101d60 0x7f3d4819b6a0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:50.249 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:50.249+0000 7f3d4d200700 1 --2- 172.21.15.72:0/2714478099 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3d48103080 0x7f3d481a2c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:50.249 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:50.249+0000 7f3d4d200700 1 --2- 172.21.15.72:0/2714478099 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3d481026c0 0x7f3d4819bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:50.249 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:50.249+0000 7f3d4d200700 1 -- 172.21.15.72:0/2714478099 >> 172.21.15.72:0/2714478099 conn(0x7f3d480fd920 msgr2=0x7f3d48100c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:54:50.249 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:50.249+0000 7f3d4d200700 1 -- 172.21.15.72:0/2714478099 shutdown_connections 2024-04-26T21:54:50.250 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:50.249+0000 7f3d4d200700 1 -- 172.21.15.72:0/2714478099 wait complete. 2024-04-26T21:54:51.120 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a1780950-0415-11ef-bc93-c7b262605968 -- bash -c 'ceph orch host ls' 2024-04-26T21:54:51.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:54:50 smithi164 ceph-mon[31916]: pgmap v431: 97 pgs: 97 active+clean; 633 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-26T21:54:51.197 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:54:50 smithi072 ceph-mon[28138]: pgmap v431: 97 pgs: 97 active+clean; 633 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-26T21:54:51.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:54:50 smithi076 ceph-mon[31823]: pgmap v431: 97 pgs: 97 active+clean; 633 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-26T21:54:51.568 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/a1780950-0415-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:54:51.975 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:54:51 smithi072 ceph-mon[28138]: from='client.15186 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:54:52.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:54:51 smithi164 ceph-mon[31916]: from='client.15186 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:54:52.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:54:51 smithi076 ceph-mon[31823]: from='client.15186 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:54:53.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:54:52 smithi072 ceph-mon[28138]: pgmap v432: 97 pgs: 97 active+clean; 633 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:54:53.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:54:52 smithi164 ceph-mon[31916]: pgmap v432: 97 pgs: 97 active+clean; 633 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:54:53.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:54:52 smithi076 ceph-mon[31823]: pgmap v432: 97 pgs: 97 active+clean; 633 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:54:53.947 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:53.945+0000 7fa39ac00700 1 -- 172.21.15.72:0/1192406584 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa394075e80 msgr2=0x7fa394076300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:53.947 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:53.945+0000 7fa39ac00700 1 --2- 172.21.15.72:0/1192406584 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa394075e80 0x7fa394076300 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fa37c009990 tx=0x7fa37c009ca0 comp rx=0 tx=0).stop 2024-04-26T21:54:53.947 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:53.945+0000 7fa39ac00700 1 -- 172.21.15.72:0/1192406584 shutdown_connections 2024-04-26T21:54:53.947 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:53.946+0000 7fa39ac00700 1 --2- 172.21.15.72:0/1192406584 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa39410fa70 0x7fa394111e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:53.947 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:53.946+0000 7fa39ac00700 1 --2- 172.21.15.72:0/1192406584 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa394077450 0x7fa3940758b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:53.947 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:53.946+0000 7fa39ac00700 1 --2- 172.21.15.72:0/1192406584 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa394075e80 0x7fa394076300 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:53.947 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:53.946+0000 7fa39ac00700 1 -- 172.21.15.72:0/1192406584 >> 172.21.15.72:0/1192406584 conn(0x7fa3940fe9f0 msgr2=0x7fa394100e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:54:53.947 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:53.946+0000 7fa39ac00700 1 -- 172.21.15.72:0/1192406584 shutdown_connections 2024-04-26T21:54:53.947 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:53.946+0000 7fa39ac00700 1 -- 172.21.15.72:0/1192406584 wait complete. 2024-04-26T21:54:53.947 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:53.947+0000 7fa39ac00700 1 Processor -- start 2024-04-26T21:54:53.948 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:53.947+0000 7fa39ac00700 1 -- start start 2024-04-26T21:54:53.949 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:53.947+0000 7fa39ac00700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa394075e80 0x7fa39419fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:53.949 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:53.947+0000 7fa39ac00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa394077450 0x7fa3941a0130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:53.949 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:53.947+0000 7fa39ac00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa39410fa70 0x7fa3941a7160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:53.949 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:53.947+0000 7fa39ac00700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fa394115440 con 0x7fa394077450 2024-04-26T21:54:53.949 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:53.947+0000 7fa39ac00700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fa3941152c0 con 0x7fa394075e80 2024-04-26T21:54:53.949 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:53.947+0000 7fa39ac00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa3941155c0 con 0x7fa39410fa70 2024-04-26T21:54:53.950 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:53.947+0000 7fa38b400700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa394077450 0x7fa3941a0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:53.950 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:53.948+0000 7fa38b400700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa394077450 0x7fa3941a0130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:44304/0 (socket says 172.21.15.72:44304) 2024-04-26T21:54:53.950 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:53.948+0000 7fa38b400700 1 -- 172.21.15.72:0/67559823 learned_addr learned my addr 172.21.15.72:0/67559823 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:54:53.950 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:53.948+0000 7fa393e00700 1 --2- 172.21.15.72:0/67559823 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa394075e80 0x7fa39419fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:53.950 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:53.948+0000 7fa398c00700 1 --2- 172.21.15.72:0/67559823 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa39410fa70 0x7fa3941a7160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:53.950 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:53.948+0000 7fa38b400700 1 -- 172.21.15.72:0/67559823 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa39410fa70 msgr2=0x7fa3941a7160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:53.950 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:53.948+0000 7fa38b400700 1 --2- 172.21.15.72:0/67559823 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa39410fa70 0x7fa3941a7160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:53.950 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:53.948+0000 7fa38b400700 1 -- 172.21.15.72:0/67559823 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa394075e80 msgr2=0x7fa39419fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:53.950 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:53.948+0000 7fa38b400700 1 --2- 172.21.15.72:0/67559823 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa394075e80 0x7fa39419fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:53.950 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:53.948+0000 7fa38b400700 1 -- 172.21.15.72:0/67559823 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3941a7800 con 0x7fa394077450 2024-04-26T21:54:53.950 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:53.948+0000 7fa38b400700 1 --2- 172.21.15.72:0/67559823 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa394077450 0x7fa3941a0130 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fa37c000c00 tx=0x7fa37c005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:54:53.950 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:53.949+0000 7fa391600700 1 -- 172.21.15.72:0/67559823 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa37c027070 con 0x7fa394077450 2024-04-26T21:54:53.950 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:53.949+0000 7fa391600700 1 -- 172.21.15.72:0/67559823 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa37c004060 con 0x7fa394077450 2024-04-26T21:54:53.951 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:53.949+0000 7fa39ac00700 1 -- 172.21.15.72:0/67559823 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3941a7a90 con 0x7fa394077450 2024-04-26T21:54:53.951 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:53.949+0000 7fa391600700 1 -- 172.21.15.72:0/67559823 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa37c022c50 con 0x7fa394077450 2024-04-26T21:54:53.951 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:53.949+0000 7fa39ac00700 1 -- 172.21.15.72:0/67559823 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3941a7f40 con 0x7fa394077450 2024-04-26T21:54:53.951 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:53.950+0000 7fa39ac00700 1 -- 172.21.15.72:0/67559823 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa39406a110 con 0x7fa394077450 2024-04-26T21:54:53.952 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:53.952+0000 7fa391600700 1 -- 172.21.15.72:0/67559823 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fa37c02a070 con 0x7fa394077450 2024-04-26T21:54:53.954 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:53.954+0000 7fa391600700 1 --2- 172.21.15.72:0/67559823 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fa3780a3d50 0x7fa3780a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:53.954 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:53.954+0000 7fa391600700 1 -- 172.21.15.72:0/67559823 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7fa37c0d1fe0 con 0x7fa394077450 2024-04-26T21:54:53.954 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:53.954+0000 7fa393e00700 1 --2- 172.21.15.72:0/67559823 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fa3780a3d50 0x7fa3780a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:53.955 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:53.955+0000 7fa393e00700 1 --2- 172.21.15.72:0/67559823 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fa3780a3d50 0x7fa3780a6210 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fa394077830 tx=0x7fa384009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:54:53.956 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:53.957+0000 7fa391600700 1 -- 172.21.15.72:0/67559823 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fa37c013070 con 0x7fa394077450 2024-04-26T21:54:54.188 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:54.188+0000 7fa39ac00700 1 -- 172.21.15.72:0/67559823 --> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7fa394063860 con 0x7fa3780a3d50 2024-04-26T21:54:54.192 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:54.191+0000 7fa391600700 1 -- 172.21.15.72:0/67559823 <== mgr.14152 v2:172.21.15.72:6800/588150172 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7fa394063860 con 0x7fa3780a3d50 2024-04-26T21:54:54.192 INFO:teuthology.orchestra.run.smithi072.stdout:HOST ADDR LABELS STATUS 2024-04-26T21:54:54.192 INFO:teuthology.orchestra.run.smithi072.stdout:smithi072 172.21.15.72 2024-04-26T21:54:54.192 INFO:teuthology.orchestra.run.smithi072.stdout:smithi076 172.21.15.76 2024-04-26T21:54:54.192 INFO:teuthology.orchestra.run.smithi072.stdout:smithi164 172.21.15.164 2024-04-26T21:54:54.192 INFO:teuthology.orchestra.run.smithi072.stdout:3 hosts in cluster 2024-04-26T21:54:54.195 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:54.194+0000 7fa39ac00700 1 -- 172.21.15.72:0/67559823 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fa3780a3d50 msgr2=0x7fa3780a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:54.195 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:54.194+0000 7fa39ac00700 1 --2- 172.21.15.72:0/67559823 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fa3780a3d50 0x7fa3780a6210 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fa394077830 tx=0x7fa384009040 comp rx=0 tx=0).stop 2024-04-26T21:54:54.195 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:54.194+0000 7fa39ac00700 1 -- 172.21.15.72:0/67559823 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa394077450 msgr2=0x7fa3941a0130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:54.195 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:54.194+0000 7fa39ac00700 1 --2- 172.21.15.72:0/67559823 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa394077450 0x7fa3941a0130 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fa37c000c00 tx=0x7fa37c005e80 comp rx=0 tx=0).stop 2024-04-26T21:54:54.195 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:54.194+0000 7fa39ac00700 1 -- 172.21.15.72:0/67559823 shutdown_connections 2024-04-26T21:54:54.195 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:54.194+0000 7fa39ac00700 1 --2- 172.21.15.72:0/67559823 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fa3780a3d50 0x7fa3780a6210 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:54.195 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:54.194+0000 7fa39ac00700 1 --2- 172.21.15.72:0/67559823 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa39410fa70 0x7fa3941a7160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:54.195 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:54.194+0000 7fa39ac00700 1 --2- 172.21.15.72:0/67559823 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa394075e80 0x7fa39419fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:54.195 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:54.194+0000 7fa39ac00700 1 --2- 172.21.15.72:0/67559823 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa394077450 0x7fa3941a0130 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:54.196 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:54.194+0000 7fa39ac00700 1 -- 172.21.15.72:0/67559823 >> 172.21.15.72:0/67559823 conn(0x7fa3940fe9f0 msgr2=0x7fa394102e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:54:54.196 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:54.195+0000 7fa39ac00700 1 -- 172.21.15.72:0/67559823 shutdown_connections 2024-04-26T21:54:54.196 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:54.195+0000 7fa39ac00700 1 -- 172.21.15.72:0/67559823 wait complete. 2024-04-26T21:54:55.013 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a1780950-0415-11ef-bc93-c7b262605968 -- bash -c 'ceph orch device ls' 2024-04-26T21:54:55.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:54:54 smithi164 ceph-mon[31916]: pgmap v433: 97 pgs: 97 active+clean; 633 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-26T21:54:55.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:54:54 smithi076 ceph-mon[31823]: pgmap v433: 97 pgs: 97 active+clean; 633 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-26T21:54:55.321 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:54:54 smithi072 ceph-mon[28138]: pgmap v433: 97 pgs: 97 active+clean; 633 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-26T21:54:55.462 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/a1780950-0415-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:54:56.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:54:55 smithi076 ceph-mon[31823]: from='client.15192 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:54:56.321 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:54:55 smithi072 ceph-mon[28138]: from='client.15192 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:54:56.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:54:55 smithi164 ceph-mon[31916]: from='client.15192 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:54:57.188 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:54:56 smithi072 ceph-mon[28138]: pgmap v434: 97 pgs: 97 active+clean; 633 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-26T21:54:57.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:54:56 smithi076 ceph-mon[31823]: pgmap v434: 97 pgs: 97 active+clean; 633 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-26T21:54:57.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:54:56 smithi164 ceph-mon[31916]: pgmap v434: 97 pgs: 97 active+clean; 633 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-26T21:54:57.754 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:57.753+0000 7f3a22600700 1 -- 172.21.15.72:0/2863970963 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3a1c075e80 msgr2=0x7f3a1c076300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:57.754 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:57.753+0000 7f3a22600700 1 --2- 172.21.15.72:0/2863970963 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3a1c075e80 0x7f3a1c076300 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f3a04009990 tx=0x7f3a04009ca0 comp rx=0 tx=0).stop 2024-04-26T21:54:57.754 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:57.753+0000 7f3a22600700 1 -- 172.21.15.72:0/2863970963 shutdown_connections 2024-04-26T21:54:57.754 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:57.753+0000 7f3a22600700 1 --2- 172.21.15.72:0/2863970963 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3a1c10b360 0x7f3a1c10d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:57.754 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:57.753+0000 7f3a22600700 1 --2- 172.21.15.72:0/2863970963 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3a1c077450 0x7f3a1c0758b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:57.754 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:57.753+0000 7f3a22600700 1 --2- 172.21.15.72:0/2863970963 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3a1c075e80 0x7f3a1c076300 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:57.755 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:57.753+0000 7f3a22600700 1 -- 172.21.15.72:0/2863970963 >> 172.21.15.72:0/2863970963 conn(0x7f3a1c0fe9f0 msgr2=0x7f3a1c100e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:54:57.755 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:57.754+0000 7f3a22600700 1 -- 172.21.15.72:0/2863970963 shutdown_connections 2024-04-26T21:54:57.755 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:57.754+0000 7f3a22600700 1 -- 172.21.15.72:0/2863970963 wait complete. 2024-04-26T21:54:57.757 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:57.755+0000 7f3a22600700 1 Processor -- start 2024-04-26T21:54:57.757 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:57.755+0000 7f3a22600700 1 -- start start 2024-04-26T21:54:57.757 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:57.755+0000 7f3a22600700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3a1c075e80 0x7f3a1c19b800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:57.757 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:57.755+0000 7f3a22600700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3a1c077450 0x7f3a1c19bd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:57.757 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:57.755+0000 7f3a22600700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3a1c10b360 0x7f3a1c1a2d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:57.757 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:57.755+0000 7f3a22600700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f3a1c111020 con 0x7f3a1c10b360 2024-04-26T21:54:57.757 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:57.755+0000 7f3a22600700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f3a1c110ea0 con 0x7f3a1c077450 2024-04-26T21:54:57.757 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:57.755+0000 7f3a22600700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f3a1c1111a0 con 0x7f3a1c075e80 2024-04-26T21:54:57.757 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:57.755+0000 7f3a1be00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3a1c10b360 0x7f3a1c1a2d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:57.757 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:57.756+0000 7f3a1be00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3a1c10b360 0x7f3a1c1a2d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:44322/0 (socket says 172.21.15.72:44322) 2024-04-26T21:54:57.757 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:57.756+0000 7f3a1be00700 1 -- 172.21.15.72:0/1977305777 learned_addr learned my addr 172.21.15.72:0/1977305777 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:54:57.757 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:57.756+0000 7f3a1b400700 1 --2- 172.21.15.72:0/1977305777 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3a1c075e80 0x7f3a1c19b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:57.758 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:57.756+0000 7f3a12a00700 1 --2- 172.21.15.72:0/1977305777 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3a1c077450 0x7f3a1c19bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:57.758 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:57.756+0000 7f3a1be00700 1 -- 172.21.15.72:0/1977305777 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3a1c075e80 msgr2=0x7f3a1c19b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:57.758 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:57.756+0000 7f3a1be00700 1 --2- 172.21.15.72:0/1977305777 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3a1c075e80 0x7f3a1c19b800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:57.758 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:57.756+0000 7f3a1be00700 1 -- 172.21.15.72:0/1977305777 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3a1c077450 msgr2=0x7f3a1c19bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:57.758 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:57.756+0000 7f3a1be00700 1 --2- 172.21.15.72:0/1977305777 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3a1c077450 0x7f3a1c19bd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:57.758 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:57.756+0000 7f3a1be00700 1 -- 172.21.15.72:0/1977305777 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a1c1a3410 con 0x7f3a1c10b360 2024-04-26T21:54:57.758 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:57.756+0000 7f3a1b400700 1 --2- 172.21.15.72:0/1977305777 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3a1c075e80 0x7f3a1c19b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:54:57.758 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:57.756+0000 7f3a1be00700 1 --2- 172.21.15.72:0/1977305777 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3a1c10b360 0x7f3a1c1a2d70 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f3a1400add0 tx=0x7f3a1400c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:54:57.758 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:57.757+0000 7f3a18c00700 1 -- 172.21.15.72:0/1977305777 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a14017020 con 0x7f3a1c10b360 2024-04-26T21:54:57.758 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:57.757+0000 7f3a18c00700 1 -- 172.21.15.72:0/1977305777 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3a1400cdd0 con 0x7f3a1c10b360 2024-04-26T21:54:57.759 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:57.757+0000 7f3a22600700 1 -- 172.21.15.72:0/1977305777 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a1c1a3700 con 0x7f3a1c10b360 2024-04-26T21:54:57.759 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:57.757+0000 7f3a18c00700 1 -- 172.21.15.72:0/1977305777 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a14005330 con 0x7f3a1c10b360 2024-04-26T21:54:57.759 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:57.757+0000 7f3a22600700 1 -- 172.21.15.72:0/1977305777 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a1c1a3c40 con 0x7f3a1c10b360 2024-04-26T21:54:57.759 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:57.758+0000 7f3a22600700 1 -- 172.21.15.72:0/1977305777 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a1c06a110 con 0x7f3a1c10b360 2024-04-26T21:54:57.760 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:57.760+0000 7f3a18c00700 1 -- 172.21.15.72:0/1977305777 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f3a14004ad0 con 0x7f3a1c10b360 2024-04-26T21:54:57.761 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:57.761+0000 7f3a18c00700 1 --2- 172.21.15.72:0/1977305777 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f3a000a3d50 0x7f3a000a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:57.762 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:57.762+0000 7f3a18c00700 1 -- 172.21.15.72:0/1977305777 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7f3a140cfdd0 con 0x7f3a1c10b360 2024-04-26T21:54:57.762 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:57.762+0000 7f3a1b400700 1 --2- 172.21.15.72:0/1977305777 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f3a000a3d50 0x7f3a000a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:57.762 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:57.762+0000 7f3a1b400700 1 --2- 172.21.15.72:0/1977305777 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f3a000a3d50 0x7f3a000a6210 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f3a0c005950 tx=0x7f3a0c005f50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:54:57.766 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:57.766+0000 7f3a18c00700 1 -- 172.21.15.72:0/1977305777 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f3a14021020 con 0x7f3a1c10b360 2024-04-26T21:54:58.003 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:58.003+0000 7f3a22600700 1 -- 172.21.15.72:0/1977305777 --> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7f3a1c063860 con 0x7f3a000a3d50 2024-04-26T21:54:58.007 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:58.007+0000 7f3a18c00700 1 -- 172.21.15.72:0/1977305777 <== mgr.14152 v2:172.21.15.72:6800/588150172 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7f3a1c063860 con 0x7f3a000a3d50 2024-04-26T21:54:58.008 INFO:teuthology.orchestra.run.smithi072.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-04-26T21:54:58.008 INFO:teuthology.orchestra.run.smithi072.stdout:smithi072 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400V7400BGN 372G No 4m ago Has a FileSystem, LVM detected 2024-04-26T21:54:58.008 INFO:teuthology.orchestra.run.smithi072.stdout:smithi076 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400U4400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-04-26T21:54:58.008 INFO:teuthology.orchestra.run.smithi072.stdout:smithi164 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT623300H0400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-04-26T21:54:58.011 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:58.010+0000 7f3a22600700 1 -- 172.21.15.72:0/1977305777 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f3a000a3d50 msgr2=0x7f3a000a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:58.011 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:58.010+0000 7f3a22600700 1 --2- 172.21.15.72:0/1977305777 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f3a000a3d50 0x7f3a000a6210 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f3a0c005950 tx=0x7f3a0c005f50 comp rx=0 tx=0).stop 2024-04-26T21:54:58.011 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:58.010+0000 7f3a22600700 1 -- 172.21.15.72:0/1977305777 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3a1c10b360 msgr2=0x7f3a1c1a2d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:58.011 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:58.010+0000 7f3a22600700 1 --2- 172.21.15.72:0/1977305777 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3a1c10b360 0x7f3a1c1a2d70 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f3a1400add0 tx=0x7f3a1400c330 comp rx=0 tx=0).stop 2024-04-26T21:54:58.011 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:58.010+0000 7f3a22600700 1 -- 172.21.15.72:0/1977305777 shutdown_connections 2024-04-26T21:54:58.011 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:58.010+0000 7f3a22600700 1 --2- 172.21.15.72:0/1977305777 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f3a000a3d50 0x7f3a000a6210 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:58.011 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:58.010+0000 7f3a22600700 1 --2- 172.21.15.72:0/1977305777 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3a1c10b360 0x7f3a1c1a2d70 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:58.011 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:58.010+0000 7f3a22600700 1 --2- 172.21.15.72:0/1977305777 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3a1c075e80 0x7f3a1c19b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:58.011 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:58.010+0000 7f3a22600700 1 --2- 172.21.15.72:0/1977305777 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3a1c077450 0x7f3a1c19bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:58.011 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:58.010+0000 7f3a22600700 1 -- 172.21.15.72:0/1977305777 >> 172.21.15.72:0/1977305777 conn(0x7f3a1c0fe9f0 msgr2=0x7f3a1c107260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:54:58.011 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:58.010+0000 7f3a22600700 1 -- 172.21.15.72:0/1977305777 shutdown_connections 2024-04-26T21:54:58.011 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:54:58.010+0000 7f3a22600700 1 -- 172.21.15.72:0/1977305777 wait complete. 2024-04-26T21:54:58.838 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-26T21:54:58.847 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi072.front.sepia.ceph.com 2024-04-26T21:54:58.847 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a1780950-0415-11ef-bc93-c7b262605968 -- bash -c 'ceph fs dump' 2024-04-26T21:54:59.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:54:58 smithi072 ceph-mon[28138]: pgmap v435: 97 pgs: 97 active+clean; 633 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:54:59.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:54:58 smithi072 ceph-mon[28138]: from='client.15198 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:54:59.243 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:54:58 smithi164 ceph-mon[31916]: pgmap v435: 97 pgs: 97 active+clean; 633 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:54:59.244 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:54:58 smithi164 ceph-mon[31916]: from='client.15198 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:54:59.299 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/a1780950-0415-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:54:59.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:54:58 smithi076 ceph-mon[31823]: pgmap v435: 97 pgs: 97 active+clean; 633 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:54:59.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:54:58 smithi076 ceph-mon[31823]: from='client.15198 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:55:01.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:55:00 smithi072 ceph-mon[28138]: pgmap v436: 97 pgs: 97 active+clean; 633 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-26T21:55:01.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:55:00 smithi076 ceph-mon[31823]: pgmap v436: 97 pgs: 97 active+clean; 633 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-26T21:55:01.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:55:00 smithi164 ceph-mon[31916]: pgmap v436: 97 pgs: 97 active+clean; 633 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-26T21:55:01.684 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:01.683+0000 7ff287600700 1 -- 172.21.15.72:0/1492830866 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff2801068c0 msgr2=0x7ff280108cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:01.684 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:01.683+0000 7ff287600700 1 --2- 172.21.15.72:0/1492830866 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff2801068c0 0x7ff280108cb0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7ff26c009990 tx=0x7ff26c009ca0 comp rx=0 tx=0).stop 2024-04-26T21:55:01.684 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:01.683+0000 7ff287600700 1 -- 172.21.15.72:0/1492830866 shutdown_connections 2024-04-26T21:55:01.684 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:01.683+0000 7ff287600700 1 --2- 172.21.15.72:0/1492830866 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff2801091f0 0x7ff28010b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:01.684 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:01.683+0000 7ff287600700 1 --2- 172.21.15.72:0/1492830866 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff280103f90 0x7ff280106380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:01.685 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:01.683+0000 7ff287600700 1 --2- 172.21.15.72:0/1492830866 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff2801068c0 0x7ff280108cb0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:01.685 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:01.683+0000 7ff287600700 1 -- 172.21.15.72:0/1492830866 >> 172.21.15.72:0/1492830866 conn(0x7ff2800fd920 msgr2=0x7ff2800ffd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:55:01.685 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:01.683+0000 7ff287600700 1 -- 172.21.15.72:0/1492830866 shutdown_connections 2024-04-26T21:55:01.685 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:01.683+0000 7ff287600700 1 -- 172.21.15.72:0/1492830866 wait complete. 2024-04-26T21:55:01.685 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:01.685+0000 7ff287600700 1 Processor -- start 2024-04-26T21:55:01.685 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:01.685+0000 7ff287600700 1 -- start start 2024-04-26T21:55:01.687 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:01.686+0000 7ff287600700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff280103f90 0x7ff28019b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:01.687 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:01.686+0000 7ff287600700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff2801068c0 0x7ff28019bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:01.687 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:01.686+0000 7ff287600700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff2801091f0 0x7ff2801a2dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:01.687 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:01.686+0000 7ff287600700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7ff28010ee30 con 0x7ff280103f90 2024-04-26T21:55:01.687 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:01.686+0000 7ff287600700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7ff28010ecb0 con 0x7ff2801068c0 2024-04-26T21:55:01.688 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:01.686+0000 7ff287600700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ff28010efb0 con 0x7ff2801091f0 2024-04-26T21:55:01.688 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:01.686+0000 7ff284c00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff280103f90 0x7ff28019b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:01.688 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:01.686+0000 7ff284c00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff280103f90 0x7ff28019b810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:46198/0 (socket says 172.21.15.72:46198) 2024-04-26T21:55:01.688 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:01.686+0000 7ff284c00700 1 -- 172.21.15.72:0/375727895 learned_addr learned my addr 172.21.15.72:0/375727895 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:55:01.688 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:01.686+0000 7ff27fe00700 1 --2- 172.21.15.72:0/375727895 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff2801068c0 0x7ff28019bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:01.688 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:01.687+0000 7ff285600700 1 --2- 172.21.15.72:0/375727895 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff2801091f0 0x7ff2801a2dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:01.688 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:01.687+0000 7ff284c00700 1 -- 172.21.15.72:0/375727895 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff2801091f0 msgr2=0x7ff2801a2dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:01.688 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:01.687+0000 7ff284c00700 1 --2- 172.21.15.72:0/375727895 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff2801091f0 0x7ff2801a2dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:01.688 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:01.687+0000 7ff284c00700 1 -- 172.21.15.72:0/375727895 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff2801068c0 msgr2=0x7ff28019bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:01.688 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:01.687+0000 7ff284c00700 1 --2- 172.21.15.72:0/375727895 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff2801068c0 0x7ff28019bd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:01.688 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:01.687+0000 7ff284c00700 1 -- 172.21.15.72:0/375727895 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2801a3470 con 0x7ff280103f90 2024-04-26T21:55:01.688 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:01.687+0000 7ff27fe00700 1 --2- 172.21.15.72:0/375727895 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff2801068c0 0x7ff28019bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:55:01.689 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:01.687+0000 7ff284c00700 1 --2- 172.21.15.72:0/375727895 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff280103f90 0x7ff28019b810 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7ff27400b560 tx=0x7ff27400b920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:55:01.689 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:01.688+0000 7ff27d600700 1 -- 172.21.15.72:0/375727895 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff274015020 con 0x7ff280103f90 2024-04-26T21:55:01.690 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:01.688+0000 7ff27d600700 1 -- 172.21.15.72:0/375727895 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff274011cc0 con 0x7ff280103f90 2024-04-26T21:55:01.690 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:01.688+0000 7ff287600700 1 -- 172.21.15.72:0/375727895 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2801a3760 con 0x7ff280103f90 2024-04-26T21:55:01.690 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:01.688+0000 7ff27d600700 1 -- 172.21.15.72:0/375727895 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff274010d30 con 0x7ff280103f90 2024-04-26T21:55:01.690 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:01.688+0000 7ff287600700 1 -- 172.21.15.72:0/375727895 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2801a3d20 con 0x7ff280103f90 2024-04-26T21:55:01.690 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:01.689+0000 7ff287600700 1 -- 172.21.15.72:0/375727895 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff28006a110 con 0x7ff280103f90 2024-04-26T21:55:01.692 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:01.692+0000 7ff27d600700 1 -- 172.21.15.72:0/375727895 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7ff274018020 con 0x7ff280103f90 2024-04-26T21:55:01.694 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:01.694+0000 7ff27d600700 1 --2- 172.21.15.72:0/375727895 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7ff2680a3cd0 0x7ff2680a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:01.695 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:01.695+0000 7ff27d600700 1 -- 172.21.15.72:0/375727895 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7ff2740d0320 con 0x7ff280103f90 2024-04-26T21:55:01.695 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:01.695+0000 7ff27fe00700 1 --2- 172.21.15.72:0/375727895 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7ff2680a3cd0 0x7ff2680a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:01.696 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:01.696+0000 7ff27fe00700 1 --2- 172.21.15.72:0/375727895 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7ff2680a3cd0 0x7ff2680a6190 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7ff26c00b5c0 tx=0x7ff26c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:55:01.697 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:01.697+0000 7ff27d600700 1 -- 172.21.15.72:0/375727895 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7ff27401f020 con 0x7ff280103f90 2024-04-26T21:55:01.973 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:01.973+0000 7ff287600700 1 -- 172.21.15.72:0/375727895 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7ff28002e4c0 con 0x7ff280103f90 2024-04-26T21:55:01.974 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:01.974+0000 7ff27d600700 1 -- 172.21.15.72:0/375727895 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 26 v26) ==== 76+0+3090 (secure 0 0 0) 0x7ff27409cbd0 con 0x7ff280103f90 2024-04-26T21:55:01.976 INFO:teuthology.orchestra.run.smithi072.stdout:e26 2024-04-26T21:55:01.976 INFO:teuthology.orchestra.run.smithi072.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-26T21:55:01.976 INFO:teuthology.orchestra.run.smithi072.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-26T21:55:01.976 INFO:teuthology.orchestra.run.smithi072.stdout:legacy client fscid: 1 2024-04-26T21:55:01.976 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T21:55:01.976 INFO:teuthology.orchestra.run.smithi072.stdout:Filesystem 'cephfs' (1) 2024-04-26T21:55:01.976 INFO:teuthology.orchestra.run.smithi072.stdout:fs_name cephfs 2024-04-26T21:55:01.976 INFO:teuthology.orchestra.run.smithi072.stdout:epoch 26 2024-04-26T21:55:01.976 INFO:teuthology.orchestra.run.smithi072.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-04-26T21:55:01.976 INFO:teuthology.orchestra.run.smithi072.stdout:created 2024-04-26T21:52:50.567857+0000 2024-04-26T21:55:01.977 INFO:teuthology.orchestra.run.smithi072.stdout:modified 2024-04-26T21:52:58.508126+0000 2024-04-26T21:55:01.977 INFO:teuthology.orchestra.run.smithi072.stdout:tableserver 0 2024-04-26T21:55:01.977 INFO:teuthology.orchestra.run.smithi072.stdout:root 0 2024-04-26T21:55:01.977 INFO:teuthology.orchestra.run.smithi072.stdout:session_timeout 300 2024-04-26T21:55:01.977 INFO:teuthology.orchestra.run.smithi072.stdout:session_autoclose 300 2024-04-26T21:55:01.977 INFO:teuthology.orchestra.run.smithi072.stdout:max_file_size 1099511627776 2024-04-26T21:55:01.977 INFO:teuthology.orchestra.run.smithi072.stdout:max_xattr_size 65536 2024-04-26T21:55:01.977 INFO:teuthology.orchestra.run.smithi072.stdout:required_client_features {} 2024-04-26T21:55:01.977 INFO:teuthology.orchestra.run.smithi072.stdout:last_failure 0 2024-04-26T21:55:01.977 INFO:teuthology.orchestra.run.smithi072.stdout:last_failure_osd_epoch 0 2024-04-26T21:55:01.977 INFO:teuthology.orchestra.run.smithi072.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-26T21:55:01.977 INFO:teuthology.orchestra.run.smithi072.stdout:max_mds 5 2024-04-26T21:55:01.977 INFO:teuthology.orchestra.run.smithi072.stdout:in 0,1,2,3,4 2024-04-26T21:55:01.977 INFO:teuthology.orchestra.run.smithi072.stdout:up {0=34517,1=34472,2=34442,3=34406,4=24463} 2024-04-26T21:55:01.977 INFO:teuthology.orchestra.run.smithi072.stdout:failed 2024-04-26T21:55:01.978 INFO:teuthology.orchestra.run.smithi072.stdout:damaged 2024-04-26T21:55:01.978 INFO:teuthology.orchestra.run.smithi072.stdout:stopped 2024-04-26T21:55:01.978 INFO:teuthology.orchestra.run.smithi072.stdout:data_pools [3] 2024-04-26T21:55:01.978 INFO:teuthology.orchestra.run.smithi072.stdout:metadata_pool 2 2024-04-26T21:55:01.978 INFO:teuthology.orchestra.run.smithi072.stdout:inline_data disabled 2024-04-26T21:55:01.978 INFO:teuthology.orchestra.run.smithi072.stdout:balancer 2024-04-26T21:55:01.978 INFO:teuthology.orchestra.run.smithi072.stdout:bal_rank_mask -1 2024-04-26T21:55:01.978 INFO:teuthology.orchestra.run.smithi072.stdout:standby_count_wanted 0 2024-04-26T21:55:01.978 INFO:teuthology.orchestra.run.smithi072.stdout:qdb_cluster leader: 34517 members: 34406,24463,34442,34517,34472 2024-04-26T21:55:01.978 INFO:teuthology.orchestra.run.smithi072.stdout:[mds.b{0:34517} state up:active seq 3 addr [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T21:55:01.978 INFO:teuthology.orchestra.run.smithi072.stdout:[mds.c{0:24442} state up:standby-replay seq 1 addr [v2:172.21.15.164:6836/3585829992,v1:172.21.15.164:6837/3585829992] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T21:55:01.978 INFO:teuthology.orchestra.run.smithi072.stdout:[mds.h{1:34472} state up:active seq 8 addr [v2:172.21.15.76:6836/1150678991,v1:172.21.15.76:6837/1150678991] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T21:55:01.978 INFO:teuthology.orchestra.run.smithi072.stdout:[mds.f{1:24424} state up:standby-replay seq 1 addr [v2:172.21.15.164:6834/2391114169,v1:172.21.15.164:6835/2391114169] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T21:55:01.978 INFO:teuthology.orchestra.run.smithi072.stdout:[mds.e{2:34442} state up:active seq 11 addr [v2:172.21.15.76:6834/3517429435,v1:172.21.15.76:6835/3517429435] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T21:55:01.979 INFO:teuthology.orchestra.run.smithi072.stdout:[mds.l{2:24388} state up:standby-replay seq 1 addr [v2:172.21.15.164:6832/4236668465,v1:172.21.15.164:6833/4236668465] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T21:55:01.979 INFO:teuthology.orchestra.run.smithi072.stdout:[mds.k{3:34406} state up:active seq 15 addr [v2:172.21.15.76:6832/3869483750,v1:172.21.15.76:6833/3869483750] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T21:55:01.979 INFO:teuthology.orchestra.run.smithi072.stdout:[mds.a{3:14664} state up:standby-replay seq 1 addr [v2:172.21.15.72:6840/3257692339,v1:172.21.15.72:6841/3257692339] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T21:55:01.979 INFO:teuthology.orchestra.run.smithi072.stdout:[mds.i{4:24463} state up:active seq 8 addr [v2:172.21.15.164:6838/1636226820,v1:172.21.15.164:6839/1636226820] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T21:55:01.979 INFO:teuthology.orchestra.run.smithi072.stdout:[mds.g{4:14652} state up:standby-replay seq 1 addr [v2:172.21.15.72:6838/1312898281,v1:172.21.15.72:6839/1312898281] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T21:55:01.979 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T21:55:01.979 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T21:55:01.979 INFO:teuthology.orchestra.run.smithi072.stdout:Standby daemons: 2024-04-26T21:55:01.979 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T21:55:01.979 INFO:teuthology.orchestra.run.smithi072.stdout:[mds.j{-1:14544} state up:standby seq 1 addr [v2:172.21.15.72:6834/1320381959,v1:172.21.15.72:6835/1320381959] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T21:55:01.979 INFO:teuthology.orchestra.run.smithi072.stdout:[mds.d{-1:14559} state up:standby seq 1 addr [v2:172.21.15.72:6836/2142200512,v1:172.21.15.72:6837/2142200512] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T21:55:01.980 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:01.979+0000 7ff287600700 1 -- 172.21.15.72:0/375727895 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7ff2680a3cd0 msgr2=0x7ff2680a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:01.980 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:01.979+0000 7ff287600700 1 --2- 172.21.15.72:0/375727895 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7ff2680a3cd0 0x7ff2680a6190 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7ff26c00b5c0 tx=0x7ff26c005fb0 comp rx=0 tx=0).stop 2024-04-26T21:55:01.980 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:01.979+0000 7ff287600700 1 -- 172.21.15.72:0/375727895 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff280103f90 msgr2=0x7ff28019b810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:01.980 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:01.979+0000 7ff287600700 1 --2- 172.21.15.72:0/375727895 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff280103f90 0x7ff28019b810 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7ff27400b560 tx=0x7ff27400b920 comp rx=0 tx=0).stop 2024-04-26T21:55:01.980 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:01.979+0000 7ff287600700 1 -- 172.21.15.72:0/375727895 shutdown_connections 2024-04-26T21:55:01.980 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:01.979+0000 7ff287600700 1 --2- 172.21.15.72:0/375727895 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7ff2680a3cd0 0x7ff2680a6190 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:01.980 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:01.979+0000 7ff287600700 1 --2- 172.21.15.72:0/375727895 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff280103f90 0x7ff28019b810 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:01.980 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:01.979+0000 7ff287600700 1 --2- 172.21.15.72:0/375727895 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff2801091f0 0x7ff2801a2dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:01.980 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:01.979+0000 7ff287600700 1 --2- 172.21.15.72:0/375727895 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff2801068c0 0x7ff28019bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:01.980 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:01.979+0000 7ff287600700 1 -- 172.21.15.72:0/375727895 >> 172.21.15.72:0/375727895 conn(0x7ff2800fd920 msgr2=0x7ff2801037d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:55:01.981 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:01.980+0000 7ff287600700 1 -- 172.21.15.72:0/375727895 shutdown_connections 2024-04-26T21:55:01.981 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:01.980+0000 7ff287600700 1 -- 172.21.15.72:0/375727895 wait complete. 2024-04-26T21:55:01.981 INFO:teuthology.orchestra.run.smithi072.stderr:dumped fsmap epoch 26 2024-04-26T21:55:02.853 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a1780950-0415-11ef-bc93-c7b262605968 -- bash -c 'ceph osd dump' 2024-04-26T21:55:03.071 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:55:02 smithi072 ceph-mon[28138]: pgmap v437: 97 pgs: 97 active+clean; 633 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-26T21:55:03.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:55:02 smithi072 ceph-mon[28138]: from='client.? 172.21.15.72:0/375727895' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-26T21:55:03.288 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/a1780950-0415-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:55:03.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:55:02 smithi076 ceph-mon[31823]: pgmap v437: 97 pgs: 97 active+clean; 633 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-26T21:55:03.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:55:02 smithi076 ceph-mon[31823]: from='client.? 172.21.15.72:0/375727895' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-26T21:55:03.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:55:02 smithi164 ceph-mon[31916]: pgmap v437: 97 pgs: 97 active+clean; 633 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-26T21:55:03.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:55:02 smithi164 ceph-mon[31916]: from='client.? 172.21.15.72:0/375727895' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-26T21:55:05.235 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:55:04 smithi072 ceph-mon[28138]: pgmap v438: 97 pgs: 97 active+clean; 633 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 9 op/s 2024-04-26T21:55:05.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:55:04 smithi076 ceph-mon[31823]: pgmap v438: 97 pgs: 97 active+clean; 633 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 9 op/s 2024-04-26T21:55:05.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:55:04 smithi164 ceph-mon[31916]: pgmap v438: 97 pgs: 97 active+clean; 633 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 9 op/s 2024-04-26T21:55:05.511 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:05.510+0000 7f0d41e00700 1 -- 172.21.15.72:0/3174031316 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0d3c10fa70 msgr2=0x7f0d3c111e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:05.511 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:05.510+0000 7f0d41e00700 1 --2- 172.21.15.72:0/3174031316 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0d3c10fa70 0x7f0d3c111e60 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f0d34009990 tx=0x7f0d34009ca0 comp rx=0 tx=0).stop 2024-04-26T21:55:05.511 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:05.510+0000 7f0d41e00700 1 -- 172.21.15.72:0/3174031316 shutdown_connections 2024-04-26T21:55:05.511 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:05.510+0000 7f0d41e00700 1 --2- 172.21.15.72:0/3174031316 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0d3c10fa70 0x7f0d3c111e60 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:05.511 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:05.510+0000 7f0d41e00700 1 --2- 172.21.15.72:0/3174031316 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0d3c077450 0x7f0d3c0758b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:05.512 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:05.510+0000 7f0d41e00700 1 --2- 172.21.15.72:0/3174031316 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0d3c075e80 0x7f0d3c076300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:05.512 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:05.510+0000 7f0d41e00700 1 -- 172.21.15.72:0/3174031316 >> 172.21.15.72:0/3174031316 conn(0x7f0d3c0fe9f0 msgr2=0x7f0d3c100e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:55:05.512 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:05.511+0000 7f0d41e00700 1 -- 172.21.15.72:0/3174031316 shutdown_connections 2024-04-26T21:55:05.512 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:05.511+0000 7f0d41e00700 1 -- 172.21.15.72:0/3174031316 wait complete. 2024-04-26T21:55:05.513 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:05.512+0000 7f0d41e00700 1 Processor -- start 2024-04-26T21:55:05.513 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:05.512+0000 7f0d41e00700 1 -- start start 2024-04-26T21:55:05.513 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:05.512+0000 7f0d41e00700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0d3c075e80 0x7f0d3c19fa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:05.514 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:05.512+0000 7f0d41e00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0d3c077450 0x7f0d3c19ff70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:05.514 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:05.512+0000 7f0d41e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0d3c10fa70 0x7f0d3c1a6ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:05.514 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:05.512+0000 7f0d41e00700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f0d3c1153f0 con 0x7f0d3c077450 2024-04-26T21:55:05.514 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:05.512+0000 7f0d41e00700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f0d3c115270 con 0x7f0d3c075e80 2024-04-26T21:55:05.514 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:05.512+0000 7f0d41e00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f0d3c115570 con 0x7f0d3c10fa70 2024-04-26T21:55:05.514 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:05.512+0000 7f0d3a000700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0d3c077450 0x7f0d3c19ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:05.514 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:05.513+0000 7f0d3a000700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0d3c077450 0x7f0d3c19ff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:46206/0 (socket says 172.21.15.72:46206) 2024-04-26T21:55:05.514 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:05.513+0000 7f0d3a000700 1 -- 172.21.15.72:0/297346494 learned_addr learned my addr 172.21.15.72:0/297346494 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:55:05.514 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:05.513+0000 7f0d3b400700 1 --2- 172.21.15.72:0/297346494 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0d3c10fa70 0x7f0d3c1a6ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:05.514 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:05.513+0000 7f0d3aa00700 1 --2- 172.21.15.72:0/297346494 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0d3c075e80 0x7f0d3c19fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:05.514 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:05.513+0000 7f0d3a000700 1 -- 172.21.15.72:0/297346494 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0d3c10fa70 msgr2=0x7f0d3c1a6ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:05.514 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:05.513+0000 7f0d3a000700 1 --2- 172.21.15.72:0/297346494 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0d3c10fa70 0x7f0d3c1a6ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:05.514 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:05.513+0000 7f0d3a000700 1 -- 172.21.15.72:0/297346494 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0d3c075e80 msgr2=0x7f0d3c19fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:05.515 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:05.513+0000 7f0d3a000700 1 --2- 172.21.15.72:0/297346494 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0d3c075e80 0x7f0d3c19fa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:05.515 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:05.513+0000 7f0d3a000700 1 -- 172.21.15.72:0/297346494 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d3c1a7690 con 0x7f0d3c077450 2024-04-26T21:55:05.515 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:05.513+0000 7f0d3b400700 1 --2- 172.21.15.72:0/297346494 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0d3c10fa70 0x7f0d3c1a6ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:55:05.515 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:05.513+0000 7f0d3a000700 1 --2- 172.21.15.72:0/297346494 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0d3c077450 0x7f0d3c19ff70 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f0d2800d9b0 tx=0x7f0d2800dd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:55:05.515 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:05.514+0000 7f0d40a00700 1 -- 172.21.15.72:0/297346494 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d28017070 con 0x7f0d3c077450 2024-04-26T21:55:05.515 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:05.514+0000 7f0d40a00700 1 -- 172.21.15.72:0/297346494 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0d28010460 con 0x7f0d3c077450 2024-04-26T21:55:05.515 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:05.514+0000 7f0d41e00700 1 -- 172.21.15.72:0/297346494 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d3c1a7980 con 0x7f0d3c077450 2024-04-26T21:55:05.516 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:05.514+0000 7f0d40a00700 1 -- 172.21.15.72:0/297346494 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d2800f620 con 0x7f0d3c077450 2024-04-26T21:55:05.516 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:05.514+0000 7f0d41e00700 1 -- 172.21.15.72:0/297346494 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d3c1a7e70 con 0x7f0d3c077450 2024-04-26T21:55:05.516 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:05.515+0000 7f0d41e00700 1 -- 172.21.15.72:0/297346494 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d3c06a110 con 0x7f0d3c077450 2024-04-26T21:55:05.518 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:05.518+0000 7f0d40a00700 1 -- 172.21.15.72:0/297346494 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f0d28010610 con 0x7f0d3c077450 2024-04-26T21:55:05.519 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:05.519+0000 7f0d40a00700 1 --2- 172.21.15.72:0/297346494 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f0d200a3cd0 0x7f0d200a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:05.519 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:05.519+0000 7f0d40a00700 1 -- 172.21.15.72:0/297346494 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7f0d280d01e0 con 0x7f0d3c077450 2024-04-26T21:55:05.520 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:05.520+0000 7f0d3aa00700 1 --2- 172.21.15.72:0/297346494 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f0d200a3cd0 0x7f0d200a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:05.520 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:05.520+0000 7f0d3aa00700 1 --2- 172.21.15.72:0/297346494 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f0d200a3cd0 0x7f0d200a6190 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f0d3c077830 tx=0x7f0d2c009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:55:05.524 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:05.524+0000 7f0d40a00700 1 -- 172.21.15.72:0/297346494 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f0d2809c980 con 0x7f0d3c077450 2024-04-26T21:55:05.751 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:05.750+0000 7f0d41e00700 1 -- 172.21.15.72:0/297346494 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7f0d3c064030 con 0x7f0d3c077450 2024-04-26T21:55:05.753 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:05.753+0000 7f0d40a00700 1 -- 172.21.15.72:0/297346494 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v91) ==== 56+0+5397 (secure 0 0 0) 0x7f0d2809c280 con 0x7f0d3c077450 2024-04-26T21:55:05.754 INFO:teuthology.orchestra.run.smithi072.stdout:epoch 91 2024-04-26T21:55:05.754 INFO:teuthology.orchestra.run.smithi072.stdout:fsid a1780950-0415-11ef-bc93-c7b262605968 2024-04-26T21:55:05.754 INFO:teuthology.orchestra.run.smithi072.stdout:created 2024-04-26T21:42:01.504173+0000 2024-04-26T21:55:05.754 INFO:teuthology.orchestra.run.smithi072.stdout:modified 2024-04-26T21:54:11.892872+0000 2024-04-26T21:55:05.754 INFO:teuthology.orchestra.run.smithi072.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-04-26T21:55:05.754 INFO:teuthology.orchestra.run.smithi072.stdout:crush_version 26 2024-04-26T21:55:05.754 INFO:teuthology.orchestra.run.smithi072.stdout:full_ratio 0.95 2024-04-26T21:55:05.755 INFO:teuthology.orchestra.run.smithi072.stdout:backfillfull_ratio 0.9 2024-04-26T21:55:05.755 INFO:teuthology.orchestra.run.smithi072.stdout:nearfull_ratio 0.85 2024-04-26T21:55:05.755 INFO:teuthology.orchestra.run.smithi072.stdout:require_min_compat_client luminous 2024-04-26T21:55:05.755 INFO:teuthology.orchestra.run.smithi072.stdout:min_compat_client luminous 2024-04-26T21:55:05.755 INFO:teuthology.orchestra.run.smithi072.stdout:require_osd_release squid 2024-04-26T21:55:05.755 INFO:teuthology.orchestra.run.smithi072.stdout:stretch_mode_enabled false 2024-04-26T21:55:05.755 INFO:teuthology.orchestra.run.smithi072.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-04-26T21:55:05.755 INFO:teuthology.orchestra.run.smithi072.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 86 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 2.25 2024-04-26T21:55:05.755 INFO:teuthology.orchestra.run.smithi072.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 87 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2024-04-26T21:55:05.755 INFO:teuthology.orchestra.run.smithi072.stdout:max_osd 12 2024-04-26T21:55:05.755 INFO:teuthology.orchestra.run.smithi072.stdout:osd.0 up in weight 1 up_from 8 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.72:6802/1416765179,v1:172.21.15.72:6803/1416765179] [v2:172.21.15.72:6804/1416765179,v1:172.21.15.72:6805/1416765179] exists,up 04e06884-b355-4762-9b29-9034b7201e9a 2024-04-26T21:55:05.755 INFO:teuthology.orchestra.run.smithi072.stdout:osd.1 up in weight 1 up_from 13 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.76:6800/755426949,v1:172.21.15.76:6801/755426949] [v2:172.21.15.76:6802/755426949,v1:172.21.15.76:6803/755426949] exists,up d75a479a-4a2e-48e9-a01e-2b1c9617416a 2024-04-26T21:55:05.755 INFO:teuthology.orchestra.run.smithi072.stdout:osd.2 up in weight 1 up_from 18 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.164:6800/852265973,v1:172.21.15.164:6801/852265973] [v2:172.21.15.164:6802/852265973,v1:172.21.15.164:6803/852265973] exists,up b6f8da58-4428-4e8c-a944-7ea5645d6d92 2024-04-26T21:55:05.755 INFO:teuthology.orchestra.run.smithi072.stdout:osd.3 up in weight 1 up_from 25 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.72:6810/4027763789,v1:172.21.15.72:6811/4027763789] [v2:172.21.15.72:6812/4027763789,v1:172.21.15.72:6813/4027763789] exists,up 76d83409-649e-41e5-b4f4-43bbd5b00ea5 2024-04-26T21:55:05.755 INFO:teuthology.orchestra.run.smithi072.stdout:osd.4 up in weight 1 up_from 33 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.76:6808/3718867827,v1:172.21.15.76:6809/3718867827] [v2:172.21.15.76:6810/3718867827,v1:172.21.15.76:6811/3718867827] exists,up d2cfeea7-1661-4156-bdcd-f770a8d1a878 2024-04-26T21:55:05.756 INFO:teuthology.orchestra.run.smithi072.stdout:osd.5 up in weight 1 up_from 39 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.164:6808/2543954772,v1:172.21.15.164:6809/2543954772] [v2:172.21.15.164:6810/2543954772,v1:172.21.15.164:6811/2543954772] exists,up f3896ad6-a040-4ea3-a431-ff9b7fc05c36 2024-04-26T21:55:05.756 INFO:teuthology.orchestra.run.smithi072.stdout:osd.6 up in weight 1 up_from 47 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.72:6818/2080659027,v1:172.21.15.72:6819/2080659027] [v2:172.21.15.72:6820/2080659027,v1:172.21.15.72:6821/2080659027] exists,up d5d56650-be7e-4ba5-ba3a-d7d473294dab 2024-04-26T21:55:05.756 INFO:teuthology.orchestra.run.smithi072.stdout:osd.7 up in weight 1 up_from 52 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.76:6816/3144631323,v1:172.21.15.76:6817/3144631323] [v2:172.21.15.76:6818/3144631323,v1:172.21.15.76:6819/3144631323] exists,up 18975b5a-9309-4a0f-9f37-3ce09c2e745d 2024-04-26T21:55:05.756 INFO:teuthology.orchestra.run.smithi072.stdout:osd.8 up in weight 1 up_from 61 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.164:6816/627888883,v1:172.21.15.164:6817/627888883] [v2:172.21.15.164:6818/627888883,v1:172.21.15.164:6819/627888883] exists,up 779e9c1e-8e97-48d4-a854-68ca13f38e0a 2024-04-26T21:55:05.756 INFO:teuthology.orchestra.run.smithi072.stdout:osd.9 up in weight 1 up_from 69 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.72:6826/1273118485,v1:172.21.15.72:6827/1273118485] [v2:172.21.15.72:6828/1273118485,v1:172.21.15.72:6829/1273118485] exists,up d0317850-7b2d-4850-b2cf-0b919505c1fa 2024-04-26T21:55:05.756 INFO:teuthology.orchestra.run.smithi072.stdout:osd.10 up in weight 1 up_from 74 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.76:6824/2808954459,v1:172.21.15.76:6825/2808954459] [v2:172.21.15.76:6826/2808954459,v1:172.21.15.76:6827/2808954459] exists,up e0aa38d8-6011-4d2a-b67b-5bff6d999e47 2024-04-26T21:55:05.756 INFO:teuthology.orchestra.run.smithi072.stdout:osd.11 up in weight 1 up_from 82 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.164:6824/2354951784,v1:172.21.15.164:6825/2354951784] [v2:172.21.15.164:6826/2354951784,v1:172.21.15.164:6827/2354951784] exists,up 981b3d4d-916e-4937-9916-7cc21a891e43 2024-04-26T21:55:05.756 INFO:teuthology.orchestra.run.smithi072.stdout:pg_upmap_items 2.6 [8,11] 2024-04-26T21:55:05.756 INFO:teuthology.orchestra.run.smithi072.stdout:pg_upmap_items 2.a [2,11] 2024-04-26T21:55:05.756 INFO:teuthology.orchestra.run.smithi072.stdout:pg_upmap_items 2.e [2,4] 2024-04-26T21:55:05.756 INFO:teuthology.orchestra.run.smithi072.stdout:pg_upmap_items 2.1a [8,1] 2024-04-26T21:55:05.756 INFO:teuthology.orchestra.run.smithi072.stdout:pg_upmap_items 2.1b [2,4] 2024-04-26T21:55:05.756 INFO:teuthology.orchestra.run.smithi072.stdout:pg_upmap_items 3.3 [4,9] 2024-04-26T21:55:05.756 INFO:teuthology.orchestra.run.smithi072.stdout:pg_upmap_items 3.1a [5,7] 2024-04-26T21:55:05.756 INFO:teuthology.orchestra.run.smithi072.stdout:pg_upmap_items 3.1b [4,6] 2024-04-26T21:55:05.757 INFO:teuthology.orchestra.run.smithi072.stdout:pg_upmap_items 3.27 [4,8] 2024-04-26T21:55:05.757 INFO:teuthology.orchestra.run.smithi072.stdout:pg_upmap_items 3.28 [5,6] 2024-04-26T21:55:05.757 INFO:teuthology.orchestra.run.smithi072.stdout:pg_upmap_items 3.2a [3,8] 2024-04-26T21:55:05.757 INFO:teuthology.orchestra.run.smithi072.stdout:pg_upmap_items 3.2c [4,9] 2024-04-26T21:55:05.757 INFO:teuthology.orchestra.run.smithi072.stdout:pg_upmap_items 3.38 [3,9] 2024-04-26T21:55:05.757 INFO:teuthology.orchestra.run.smithi072.stdout:blocklist 172.21.15.72:0/416277784 expires 2024-04-27T21:43:09.602791+0000 2024-04-26T21:55:05.757 INFO:teuthology.orchestra.run.smithi072.stdout:blocklist 172.21.15.72:6800/4010358672 expires 2024-04-27T21:43:09.602791+0000 2024-04-26T21:55:05.757 INFO:teuthology.orchestra.run.smithi072.stdout:blocklist 172.21.15.72:6801/1012617371 expires 2024-04-27T21:42:34.684837+0000 2024-04-26T21:55:05.757 INFO:teuthology.orchestra.run.smithi072.stdout:blocklist 172.21.15.72:6801/4010358672 expires 2024-04-27T21:43:09.602791+0000 2024-04-26T21:55:05.757 INFO:teuthology.orchestra.run.smithi072.stdout:blocklist 172.21.15.72:0/3555448179 expires 2024-04-27T21:43:09.602791+0000 2024-04-26T21:55:05.757 INFO:teuthology.orchestra.run.smithi072.stdout:blocklist 172.21.15.72:6800/1012617371 expires 2024-04-27T21:42:34.684837+0000 2024-04-26T21:55:05.757 INFO:teuthology.orchestra.run.smithi072.stdout:blocklist 172.21.15.72:0/2920378761 expires 2024-04-27T21:42:34.684837+0000 2024-04-26T21:55:05.757 INFO:teuthology.orchestra.run.smithi072.stdout:blocklist 172.21.15.72:0/1933740041 expires 2024-04-27T21:43:09.602791+0000 2024-04-26T21:55:05.757 INFO:teuthology.orchestra.run.smithi072.stdout:blocklist 172.21.15.72:0/3225295775 expires 2024-04-27T21:42:34.684837+0000 2024-04-26T21:55:05.757 INFO:teuthology.orchestra.run.smithi072.stdout:blocklist 172.21.15.72:0/2326016180 expires 2024-04-27T21:42:34.684837+0000 2024-04-26T21:55:05.758 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:05.756+0000 7f0d41e00700 1 -- 172.21.15.72:0/297346494 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f0d200a3cd0 msgr2=0x7f0d200a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:05.758 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:05.756+0000 7f0d41e00700 1 --2- 172.21.15.72:0/297346494 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f0d200a3cd0 0x7f0d200a6190 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f0d3c077830 tx=0x7f0d2c009040 comp rx=0 tx=0).stop 2024-04-26T21:55:05.758 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:05.756+0000 7f0d41e00700 1 -- 172.21.15.72:0/297346494 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0d3c077450 msgr2=0x7f0d3c19ff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:05.758 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:05.756+0000 7f0d41e00700 1 --2- 172.21.15.72:0/297346494 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0d3c077450 0x7f0d3c19ff70 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f0d2800d9b0 tx=0x7f0d2800dd70 comp rx=0 tx=0).stop 2024-04-26T21:55:05.758 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:05.756+0000 7f0d41e00700 1 -- 172.21.15.72:0/297346494 shutdown_connections 2024-04-26T21:55:05.758 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:05.756+0000 7f0d41e00700 1 --2- 172.21.15.72:0/297346494 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f0d200a3cd0 0x7f0d200a6190 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:05.758 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:05.756+0000 7f0d41e00700 1 --2- 172.21.15.72:0/297346494 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0d3c10fa70 0x7f0d3c1a6ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:05.759 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:05.756+0000 7f0d41e00700 1 --2- 172.21.15.72:0/297346494 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0d3c075e80 0x7f0d3c19fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:05.759 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:05.756+0000 7f0d41e00700 1 --2- 172.21.15.72:0/297346494 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0d3c077450 0x7f0d3c19ff70 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:05.759 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:05.756+0000 7f0d41e00700 1 -- 172.21.15.72:0/297346494 >> 172.21.15.72:0/297346494 conn(0x7f0d3c0fe9f0 msgr2=0x7f0d3c102e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:55:05.759 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:05.757+0000 7f0d41e00700 1 -- 172.21.15.72:0/297346494 shutdown_connections 2024-04-26T21:55:05.759 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:05.757+0000 7f0d41e00700 1 -- 172.21.15.72:0/297346494 wait complete. 2024-04-26T21:55:06.071 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:55:05 smithi072 ceph-mon[28138]: from='client.? 172.21.15.72:0/297346494' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-26T21:55:06.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:55:05 smithi076 ceph-mon[31823]: from='client.? 172.21.15.72:0/297346494' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-26T21:55:06.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:55:05 smithi164 ceph-mon[31916]: from='client.? 172.21.15.72:0/297346494' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-26T21:55:06.640 INFO:teuthology.run_tasks:Running task fs.ready... 2024-04-26T21:55:06.652 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:55:06.871 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:06.870+0000 7fe10b200640 1 -- 172.21.15.72:0/479652314 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe10c101670 msgr2=0x7fe10c101a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:06.871 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:06.870+0000 7fe10b200640 1 --2- 172.21.15.72:0/479652314 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe10c101670 0x7fe10c101a90 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fe0fc009900 tx=0x7fe0fc02a410 comp rx=0 tx=0).stop 2024-04-26T21:55:06.871 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:06.870+0000 7fe10b200640 1 -- 172.21.15.72:0/479652314 shutdown_connections 2024-04-26T21:55:06.871 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:06.870+0000 7fe10b200640 1 --2- 172.21.15.72:0/479652314 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe10c103220 0x7fe10c109ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:06.871 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:06.870+0000 7fe10b200640 1 --2- 172.21.15.72:0/479652314 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe10c102860 0x7fe10c102ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:06.871 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:06.870+0000 7fe10b200640 1 --2- 172.21.15.72:0/479652314 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe10c101670 0x7fe10c101a90 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:06.871 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:06.870+0000 7fe10b200640 1 -- 172.21.15.72:0/479652314 >> 172.21.15.72:0/479652314 conn(0x7fe10c0fcbc0 msgr2=0x7fe10c0ff020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:55:06.871 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:06.870+0000 7fe10b200640 1 -- 172.21.15.72:0/479652314 shutdown_connections 2024-04-26T21:55:06.871 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:06.871+0000 7fe10b200640 1 -- 172.21.15.72:0/479652314 wait complete. 2024-04-26T21:55:06.874 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:06.872+0000 7fe10b200640 1 Processor -- start 2024-04-26T21:55:06.874 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:06.872+0000 7fe10b200640 1 -- start start 2024-04-26T21:55:06.874 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:06.872+0000 7fe10b200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe10c101670 0x7fe10c1988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:06.874 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:06.872+0000 7fe10b200640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe10c102860 0x7fe10c198e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:06.874 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:06.872+0000 7fe10b200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe10c103220 0x7fe10c19fea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:06.874 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:06.872+0000 7fe10b200640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fe10c10d0a0 con 0x7fe10c102860 2024-04-26T21:55:06.874 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:06.872+0000 7fe10b200640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fe10c10cf20 con 0x7fe10c101670 2024-04-26T21:55:06.874 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:06.872+0000 7fe10b200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe10c10d220 con 0x7fe10c103220 2024-04-26T21:55:06.874 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:06.872+0000 7fe103e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe10c102860 0x7fe10c198e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:06.874 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:06.872+0000 7fe103e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe10c102860 0x7fe10c198e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:46222/0 (socket says 172.21.15.72:46222) 2024-04-26T21:55:06.874 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:06.872+0000 7fe103e00640 1 -- 172.21.15.72:0/3137721325 learned_addr learned my addr 172.21.15.72:0/3137721325 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:55:06.875 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:06.873+0000 7fe10a800640 1 --2- 172.21.15.72:0/3137721325 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe10c103220 0x7fe10c19fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:06.875 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:06.873+0000 7fe109e00640 1 --2- 172.21.15.72:0/3137721325 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe10c101670 0x7fe10c1988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:06.875 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:06.873+0000 7fe103e00640 1 -- 172.21.15.72:0/3137721325 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe10c103220 msgr2=0x7fe10c19fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:06.875 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:06.873+0000 7fe103e00640 1 --2- 172.21.15.72:0/3137721325 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe10c103220 0x7fe10c19fea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:06.875 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:06.873+0000 7fe103e00640 1 -- 172.21.15.72:0/3137721325 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe10c101670 msgr2=0x7fe10c1988f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:06.875 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:06.873+0000 7fe103e00640 1 --2- 172.21.15.72:0/3137721325 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe10c101670 0x7fe10c1988f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:06.875 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:06.873+0000 7fe103e00640 1 -- 172.21.15.72:0/3137721325 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe10c1a0510 con 0x7fe10c102860 2024-04-26T21:55:06.875 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:06.873+0000 7fe10a800640 1 --2- 172.21.15.72:0/3137721325 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe10c103220 0x7fe10c19fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:55:06.875 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:06.873+0000 7fe109e00640 1 --2- 172.21.15.72:0/3137721325 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe10c101670 0x7fe10c1988f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:55:06.875 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:06.873+0000 7fe103e00640 1 --2- 172.21.15.72:0/3137721325 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe10c102860 0x7fe10c198e30 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fe0f400d950 tx=0x7fe0f400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:55:06.875 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:06.873+0000 7fe102a00640 1 -- 172.21.15.72:0/3137721325 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0f4017070 con 0x7fe10c102860 2024-04-26T21:55:06.875 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:06.873+0000 7fe102a00640 1 -- 172.21.15.72:0/3137721325 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe0f4009c30 con 0x7fe10c102860 2024-04-26T21:55:06.875 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:06.873+0000 7fe10b200640 1 -- 172.21.15.72:0/3137721325 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe10c1a0800 con 0x7fe10c102860 2024-04-26T21:55:06.875 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:06.873+0000 7fe102a00640 1 -- 172.21.15.72:0/3137721325 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0f400f620 con 0x7fe10c102860 2024-04-26T21:55:06.876 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:06.874+0000 7fe10b200640 1 -- 172.21.15.72:0/3137721325 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe10c1a0d50 con 0x7fe10c102860 2024-04-26T21:55:06.876 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:06.875+0000 7fe10b200640 1 -- 172.21.15.72:0/3137721325 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe10c1a0fb0 con 0x7fe10c102860 2024-04-26T21:55:06.877 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:06.877+0000 7fe102a00640 1 -- 172.21.15.72:0/3137721325 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fe0f4010460 con 0x7fe10c102860 2024-04-26T21:55:06.878 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:06.878+0000 7fe102a00640 1 --2- 172.21.15.72:0/3137721325 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fe0f00a3c40 0x7fe0f00a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:06.878 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:06.878+0000 7fe102a00640 1 -- 172.21.15.72:0/3137721325 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7fe0f40d0330 con 0x7fe10c102860 2024-04-26T21:55:06.878 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:06.878+0000 7fe109e00640 1 --2- 172.21.15.72:0/3137721325 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fe0f00a3c40 0x7fe0f00a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:06.879 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:06.879+0000 7fe109e00640 1 --2- 172.21.15.72:0/3137721325 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fe0f00a3c40 0x7fe0f00a6100 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fe0fc02a930 tx=0x7fe0fc000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:55:06.883 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:06.883+0000 7fe102a00640 1 -- 172.21.15.72:0/3137721325 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fe0f409cad0 con 0x7fe10c102860 2024-04-26T21:55:07.088 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:07.087+0000 7fe10b200640 1 -- 172.21.15.72:0/3137721325 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe10c1a12a0 con 0x7fe10c102860 2024-04-26T21:55:07.091 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:07.090+0000 7fe102a00640 1 -- 172.21.15.72:0/3137721325 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+22783 (secure 0 0 0) 0x7fe0f4009de0 con 0x7fe10c102860 2024-04-26T21:55:07.091 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T21:55:07.091 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":91,"fsid":"a1780950-0415-11ef-bc93-c7b262605968","created":"2024-04-26T21:42:01.504173+0000","modified":"2024-04-26T21:54:11.892872+0000","last_up_change":"2024-04-26T21:51:43.482437+0000","last_in_change":"2024-04-26T21:51:18.468587+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:46:29.785427+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:52:48.320391+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T21:52:49.517493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"04e06884-b355-4762-9b29-9034b7201e9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6802","nonce":1416765179},{"type":"v1","addr":"172.21.15.72:6803","nonce":1416765179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6804","nonce":1416765179},{"type":"v1","addr":"172.21.15.72:6805","nonce":1416765179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6808","nonce":1416765179},{"type":"v1","addr":"172.21.15.72:6809","nonce":1416765179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6806","nonce":1416765179},{"type":"v1","addr":"172.21.15.72:6807","nonce":1416765179}]},"public_addr":"172.21.15.72:6803/1416765179","cluster_addr":"172.21.15.72:6805/1416765179","heartbeat_back_addr":"172.21.15.72:6809/1416765179","heartbeat_front_addr":"172.21.15.72:6807/1416765179","state":["exists","up"]},{"osd":1,"uuid":"d75a479a-4a2e-48e9-a01e-2b1c9617416a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":755426949},{"type":"v1","addr":"172.21.15.76:6801","nonce":755426949}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":755426949},{"type":"v1","addr":"172.21.15.76:6803","nonce":755426949}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":755426949},{"type":"v1","addr":"172.21.15.76:6807","nonce":755426949}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":755426949},{"type":"v1","addr":"172.21.15.76:6805","nonce":755426949}]},"public_addr":"172.21.15.76:6801/755426949","cluster_addr":"172.21.15.76:6803/755426949","heartbeat_back_addr":"172.21.15.76:6807/755426949","heartbeat_front_addr":"172.21.15.76:6805/755426949","state":["exists","up"]},{"osd":2,"uuid":"b6f8da58-4428-4e8c-a944-7ea5645d6d92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":852265973},{"type":"v1","addr":"172.21.15.164:6801","nonce":852265973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":852265973},{"type":"v1","addr":"172.21.15.164:6803","nonce":852265973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":852265973},{"type":"v1","addr":"172.21.15.164:6807","nonce":852265973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":852265973},{"type":"v1","addr":"172.21.15.164:6805","nonce":852265973}]},"public_addr":"172.21.15.164:6801/852265973","cluster_addr":"172.21.15.164:6803/852265973","heartbeat_back_addr":"172.21.15.164:6807/852265973","heartbeat_front_addr":"172.21.15.164:6805/852265973","state":["exists","up"]},{"osd":3,"uuid":"76d83409-649e-41e5-b4f4-43bbd5b00ea5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6810","nonce":4027763789},{"type":"v1","addr":"172.21.15.72:6811","nonce":4027763789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6812","nonce":4027763789},{"type":"v1","addr":"172.21.15.72:6813","nonce":4027763789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6816","nonce":4027763789},{"type":"v1","addr":"172.21.15.72:6817","nonce":4027763789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6814","nonce":4027763789},{"type":"v1","addr":"172.21.15.72:6815","nonce":4027763789}]},"public_addr":"172.21.15.72:6811/4027763789","cluster_addr":"172.21.15.72:6813/4027763789","heartbeat_back_addr":"172.21.15.72:6817/4027763789","heartbeat_front_addr":"172.21.15.72:6815/4027763789","state":["exists","up"]},{"osd":4,"uuid":"d2cfeea7-1661-4156-bdcd-f770a8d1a878","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":3718867827},{"type":"v1","addr":"172.21.15.76:6809","nonce":3718867827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":3718867827},{"type":"v1","addr":"172.21.15.76:6811","nonce":3718867827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":3718867827},{"type":"v1","addr":"172.21.15.76:6815","nonce":3718867827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":3718867827},{"type":"v1","addr":"172.21.15.76:6813","nonce":3718867827}]},"public_addr":"172.21.15.76:6809/3718867827","cluster_addr":"172.21.15.76:6811/3718867827","heartbeat_back_addr":"172.21.15.76:6815/3718867827","heartbeat_front_addr":"172.21.15.76:6813/3718867827","state":["exists","up"]},{"osd":5,"uuid":"f3896ad6-a040-4ea3-a431-ff9b7fc05c36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":2543954772},{"type":"v1","addr":"172.21.15.164:6809","nonce":2543954772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":2543954772},{"type":"v1","addr":"172.21.15.164:6811","nonce":2543954772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":2543954772},{"type":"v1","addr":"172.21.15.164:6815","nonce":2543954772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":2543954772},{"type":"v1","addr":"172.21.15.164:6813","nonce":2543954772}]},"public_addr":"172.21.15.164:6809/2543954772","cluster_addr":"172.21.15.164:6811/2543954772","heartbeat_back_addr":"172.21.15.164:6815/2543954772","heartbeat_front_addr":"172.21.15.164:6813/2543954772","state":["exists","up"]},{"osd":6,"uuid":"d5d56650-be7e-4ba5-ba3a-d7d473294dab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6818","nonce":2080659027},{"type":"v1","addr":"172.21.15.72:6819","nonce":2080659027}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6820","nonce":2080659027},{"type":"v1","addr":"172.21.15.72:6821","nonce":2080659027}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6824","nonce":2080659027},{"type":"v1","addr":"172.21.15.72:6825","nonce":2080659027}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6822","nonce":2080659027},{"type":"v1","addr":"172.21.15.72:6823","nonce":2080659027}]},"public_addr":"172.21.15.72:6819/2080659027","cluster_addr":"172.21.15.72:6821/2080659027","heartbeat_back_addr":"172.21.15.72:6825/2080659027","heartbeat_front_addr":"172.21.15.72:6823/2080659027","state":["exists","up"]},{"osd":7,"uuid":"18975b5a-9309-4a0f-9f37-3ce09c2e745d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":3144631323},{"type":"v1","addr":"172.21.15.76:6817","nonce":3144631323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":3144631323},{"type":"v1","addr":"172.21.15.76:6819","nonce":3144631323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":3144631323},{"type":"v1","addr":"172.21.15.76:6823","nonce":3144631323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":3144631323},{"type":"v1","addr":"172.21.15.76:6821","nonce":3144631323}]},"public_addr":"172.21.15.76:6817/3144631323","cluster_addr":"172.21.15.76:6819/3144631323","heartbeat_back_addr":"172.21.15.76:6823/3144631323","heartbeat_front_addr":"172.21.15.76:6821/3144631323","state":["exists","up"]},{"osd":8,"uuid":"779e9c1e-8e97-48d4-a854-68ca13f38e0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":627888883},{"type":"v1","addr":"172.21.15.164:6817","nonce":627888883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":627888883},{"type":"v1","addr":"172.21.15.164:6819","nonce":627888883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":627888883},{"type":"v1","addr":"172.21.15.164:6823","nonce":627888883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":627888883},{"type":"v1","addr":"172.21.15.164:6821","nonce":627888883}]},"public_addr":"172.21.15.164:6817/627888883","cluster_addr":"172.21.15.164:6819/627888883","heartbeat_back_addr":"172.21.15.164:6823/627888883","heartbeat_front_addr":"172.21.15.164:6821/627888883","state":["exists","up"]},{"osd":9,"uuid":"d0317850-7b2d-4850-b2cf-0b919505c1fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6826","nonce":1273118485},{"type":"v1","addr":"172.21.15.72:6827","nonce":1273118485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6828","nonce":1273118485},{"type":"v1","addr":"172.21.15.72:6829","nonce":1273118485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6832","nonce":1273118485},{"type":"v1","addr":"172.21.15.72:6833","nonce":1273118485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6830","nonce":1273118485},{"type":"v1","addr":"172.21.15.72:6831","nonce":1273118485}]},"public_addr":"172.21.15.72:6827/1273118485","cluster_addr":"172.21.15.72:6829/1273118485","heartbeat_back_addr":"172.21.15.72:6833/1273118485","heartbeat_front_addr":"172.21.15.72:6831/1273118485","state":["exists","up"]},{"osd":10,"uuid":"e0aa38d8-6011-4d2a-b67b-5bff6d999e47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":2808954459},{"type":"v1","addr":"172.21.15.76:6825","nonce":2808954459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":2808954459},{"type":"v1","addr":"172.21.15.76:6827","nonce":2808954459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":2808954459},{"type":"v1","addr":"172.21.15.76:6831","nonce":2808954459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":2808954459},{"type":"v1","addr":"172.21.15.76:6829","nonce":2808954459}]},"public_addr":"172.21.15.76:6825/2808954459","cluster_addr":"172.21.15.76:6827/2808954459","heartbeat_back_addr":"172.21.15.76:6831/2808954459","heartbeat_front_addr":"172.21.15.76:6829/2808954459","state":["exists","up"]},{"osd":11,"uuid":"981b3d4d-916e-4937-9916-7cc21a891e43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":2354951784},{"type":"v1","addr":"172.21.15.164:6825","nonce":2354951784}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":2354951784},{"type":"v1","addr":"172.21.15.164:6827","nonce":2354951784}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":2354951784},{"type":"v1","addr":"172.21.15.164:6831","nonce":2354951784}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":2354951784},{"type":"v1","addr":"172.21.15.164:6829","nonce":2354951784}]},"public_addr":"172.21.15.164:6825/2354951784","cluster_addr":"172.21.15.164:6827/2354951784","heartbeat_back_addr":"172.21.15.164:6831/2354951784","heartbeat_front_addr":"172.21.15.164:6829/2354951784","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:18.997924+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:52.439081+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:46:25.893288+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:47:00.112807+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:47:35.116874+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:48:09.216344+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:48:45.028853+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:49:19.905397+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:49:55.030536+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:50:30.767662+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:51:06.423267+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:51:41.631211+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.1a","mappings":[{"from":5,"to":7}]},{"pgid":"3.1b","mappings":[{"from":4,"to":6}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.28","mappings":[{"from":5,"to":6}]},{"pgid":"3.2a","mappings":[{"from":3,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.72:0/416277784":"2024-04-27T21:43:09.602791+0000","172.21.15.72:6800/4010358672":"2024-04-27T21:43:09.602791+0000","172.21.15.72:6801/1012617371":"2024-04-27T21:42:34.684837+0000","172.21.15.72:6801/4010358672":"2024-04-27T21:43:09.602791+0000","172.21.15.72:0/3555448179":"2024-04-27T21:43:09.602791+0000","172.21.15.72:6800/1012617371":"2024-04-27T21:42:34.684837+0000","172.21.15.72:0/2920378761":"2024-04-27T21:42:34.684837+0000","172.21.15.72:0/1933740041":"2024-04-27T21:43:09.602791+0000","172.21.15.72:0/3225295775":"2024-04-27T21:42:34.684837+0000","172.21.15.72:0/2326016180":"2024-04-27T21:42:34.684837+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:55:07.094 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:07.093+0000 7fe10b200640 1 -- 172.21.15.72:0/3137721325 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fe0f00a3c40 msgr2=0x7fe0f00a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:07.094 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:07.093+0000 7fe10b200640 1 --2- 172.21.15.72:0/3137721325 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fe0f00a3c40 0x7fe0f00a6100 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fe0fc02a930 tx=0x7fe0fc000bc0 comp rx=0 tx=0).stop 2024-04-26T21:55:07.094 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:07.093+0000 7fe10b200640 1 -- 172.21.15.72:0/3137721325 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe10c102860 msgr2=0x7fe10c198e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:07.094 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:07.093+0000 7fe10b200640 1 --2- 172.21.15.72:0/3137721325 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe10c102860 0x7fe10c198e30 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fe0f400d950 tx=0x7fe0f400de10 comp rx=0 tx=0).stop 2024-04-26T21:55:07.094 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:07.093+0000 7fe10b200640 1 -- 172.21.15.72:0/3137721325 shutdown_connections 2024-04-26T21:55:07.094 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:07.093+0000 7fe10b200640 1 --2- 172.21.15.72:0/3137721325 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fe0f00a3c40 0x7fe0f00a6100 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:07.094 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:07.093+0000 7fe10b200640 1 --2- 172.21.15.72:0/3137721325 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe10c103220 0x7fe10c19fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:07.094 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:07.093+0000 7fe10b200640 1 --2- 172.21.15.72:0/3137721325 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe10c102860 0x7fe10c198e30 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:07.094 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:07.093+0000 7fe10b200640 1 --2- 172.21.15.72:0/3137721325 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe10c101670 0x7fe10c1988f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:07.094 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:07.093+0000 7fe10b200640 1 -- 172.21.15.72:0/3137721325 >> 172.21.15.72:0/3137721325 conn(0x7fe10c0fcbc0 msgr2=0x7fe10c069b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:55:07.095 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:07.093+0000 7fe10b200640 1 -- 172.21.15.72:0/3137721325 shutdown_connections 2024-04-26T21:55:07.095 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:07.093+0000 7fe10b200640 1 -- 172.21.15.72:0/3137721325 wait complete. 2024-04-26T21:55:07.112 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:46:29.785427+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T21:52:48.320391+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T21:52:49.517493+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:55:07.112 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:55:07.314 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:07.313+0000 7f4d86a00640 1 -- 172.21.15.72:0/3254397811 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4d80100590 msgr2=0x7f4d80100990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:07.314 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:07.313+0000 7f4d86a00640 1 --2- 172.21.15.72:0/3254397811 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4d80100590 0x7f4d80100990 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f4d70009900 tx=0x7f4d7002a410 comp rx=0 tx=0).stop 2024-04-26T21:55:07.314 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:07.314+0000 7f4d86a00640 1 -- 172.21.15.72:0/3254397811 shutdown_connections 2024-04-26T21:55:07.314 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:07.314+0000 7f4d86a00640 1 --2- 172.21.15.72:0/3254397811 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4d801086a0 0x7f4d8010aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:07.314 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:07.314+0000 7f4d86a00640 1 --2- 172.21.15.72:0/3254397811 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4d80100f60 0x7f4d80108160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:07.315 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:07.314+0000 7f4d86a00640 1 --2- 172.21.15.72:0/3254397811 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4d80100590 0x7f4d80100990 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:07.315 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:07.314+0000 7f4d86a00640 1 -- 172.21.15.72:0/3254397811 >> 172.21.15.72:0/3254397811 conn(0x7f4d800fc130 msgr2=0x7f4d800fe550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:55:07.315 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:07.314+0000 7f4d86a00640 1 -- 172.21.15.72:0/3254397811 shutdown_connections 2024-04-26T21:55:07.315 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:07.314+0000 7f4d86a00640 1 -- 172.21.15.72:0/3254397811 wait complete. 2024-04-26T21:55:07.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:55:06 smithi076 ceph-mon[31823]: pgmap v439: 97 pgs: 97 active+clean; 633 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-26T21:55:07.317 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:07.315+0000 7f4d86a00640 1 Processor -- start 2024-04-26T21:55:07.317 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:07.315+0000 7f4d86a00640 1 -- start start 2024-04-26T21:55:07.317 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:07.315+0000 7f4d86a00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4d80100590 0x7f4d8019ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:07.317 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:07.315+0000 7f4d86a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4d80100f60 0x7f4d8019b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:07.317 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:07.315+0000 7f4d86a00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4d801086a0 0x7f4d801a22a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:07.318 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:07.315+0000 7f4d86a00640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f4d8010e3b0 con 0x7f4d801086a0 2024-04-26T21:55:07.318 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:07.315+0000 7f4d86a00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f4d8010e230 con 0x7f4d80100590 2024-04-26T21:55:07.318 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:07.315+0000 7f4d86a00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f4d8010e530 con 0x7f4d80100f60 2024-04-26T21:55:07.318 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:07.315+0000 7f4d84a00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4d801086a0 0x7f4d801a22a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:07.318 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:07.316+0000 7f4d84a00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4d801086a0 0x7f4d801a22a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:46236/0 (socket says 172.21.15.72:46236) 2024-04-26T21:55:07.318 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:07.316+0000 7f4d84a00640 1 -- 172.21.15.72:0/4035139909 learned_addr learned my addr 172.21.15.72:0/4035139909 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:55:07.318 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:07.316+0000 7f4d7fe00640 1 --2- 172.21.15.72:0/4035139909 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4d80100590 0x7f4d8019ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:07.318 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:07.316+0000 7f4d77e00640 1 --2- 172.21.15.72:0/4035139909 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4d80100f60 0x7f4d8019b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:07.318 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:07.316+0000 7f4d84a00640 1 -- 172.21.15.72:0/4035139909 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4d80100f60 msgr2=0x7f4d8019b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:07.318 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:07.316+0000 7f4d84a00640 1 --2- 172.21.15.72:0/4035139909 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4d80100f60 0x7f4d8019b2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:07.318 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:07.316+0000 7f4d84a00640 1 -- 172.21.15.72:0/4035139909 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4d80100590 msgr2=0x7f4d8019ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:07.318 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:07.316+0000 7f4d84a00640 1 --2- 172.21.15.72:0/4035139909 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4d80100590 0x7f4d8019ad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:07.318 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:07.316+0000 7f4d84a00640 1 -- 172.21.15.72:0/4035139909 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d801a2940 con 0x7f4d801086a0 2024-04-26T21:55:07.319 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:07.316+0000 7f4d84a00640 1 --2- 172.21.15.72:0/4035139909 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4d801086a0 0x7f4d801a22a0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f4d7800e9d0 tx=0x7f4d7800ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:55:07.319 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:07.316+0000 7f4d7d600640 1 -- 172.21.15.72:0/4035139909 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d7800cd80 con 0x7f4d801086a0 2024-04-26T21:55:07.319 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:07.317+0000 7f4d7d600640 1 -- 172.21.15.72:0/4035139909 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4d78005020 con 0x7f4d801086a0 2024-04-26T21:55:07.319 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:07.317+0000 7f4d7d600640 1 -- 172.21.15.72:0/4035139909 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d780116d0 con 0x7f4d801086a0 2024-04-26T21:55:07.319 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:07.317+0000 7f4d86a00640 1 -- 172.21.15.72:0/4035139909 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d801a2c30 con 0x7f4d801086a0 2024-04-26T21:55:07.319 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:07.317+0000 7f4d86a00640 1 -- 172.21.15.72:0/4035139909 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d801a31d0 con 0x7f4d801086a0 2024-04-26T21:55:07.319 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:07.318+0000 7f4d86a00640 1 -- 172.21.15.72:0/4035139909 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d801a3400 con 0x7f4d801086a0 2024-04-26T21:55:07.320 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:07.320+0000 7f4d7d600640 1 -- 172.21.15.72:0/4035139909 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f4d78004a40 con 0x7f4d801086a0 2024-04-26T21:55:07.321 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:55:06 smithi072 ceph-mon[28138]: pgmap v439: 97 pgs: 97 active+clean; 633 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-26T21:55:07.322 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:07.322+0000 7f4d7d600640 1 --2- 172.21.15.72:0/4035139909 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f4d640a3cc0 0x7f4d640a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:07.323 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:07.323+0000 7f4d7d600640 1 -- 172.21.15.72:0/4035139909 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7f4d780d0c10 con 0x7f4d801086a0 2024-04-26T21:55:07.323 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:07.323+0000 7f4d7fe00640 1 --2- 172.21.15.72:0/4035139909 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f4d640a3cc0 0x7f4d640a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:07.323 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:07.323+0000 7f4d7fe00640 1 --2- 172.21.15.72:0/4035139909 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f4d640a3cc0 0x7f4d640a6180 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f4d8019c200 tx=0x7f4d70031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:55:07.327 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:07.327+0000 7f4d7d600640 1 -- 172.21.15.72:0/4035139909 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f4d78016030 con 0x7f4d801086a0 2024-04-26T21:55:07.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:55:06 smithi164 ceph-mon[31916]: pgmap v439: 97 pgs: 97 active+clean; 633 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-26T21:55:07.531 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:07.531+0000 7f4d86a00640 1 -- 172.21.15.72:0/4035139909 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f4d801a35d0 con 0x7f4d801086a0 2024-04-26T21:55:07.532 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:07.532+0000 7f4d7d600640 1 -- 172.21.15.72:0/4035139909 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) ==== 93+0+10 (secure 0 0 0) 0x7f4d78087020 con 0x7f4d801086a0 2024-04-26T21:55:07.533 INFO:teuthology.orchestra.run.smithi072.stdout:pg_num: 1 2024-04-26T21:55:07.536 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:07.535+0000 7f4d86a00640 1 -- 172.21.15.72:0/4035139909 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f4d640a3cc0 msgr2=0x7f4d640a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:07.536 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:07.535+0000 7f4d86a00640 1 --2- 172.21.15.72:0/4035139909 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f4d640a3cc0 0x7f4d640a6180 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f4d8019c200 tx=0x7f4d70031400 comp rx=0 tx=0).stop 2024-04-26T21:55:07.536 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:07.535+0000 7f4d86a00640 1 -- 172.21.15.72:0/4035139909 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4d801086a0 msgr2=0x7f4d801a22a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:07.536 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:07.535+0000 7f4d86a00640 1 --2- 172.21.15.72:0/4035139909 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4d801086a0 0x7f4d801a22a0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f4d7800e9d0 tx=0x7f4d7800ee90 comp rx=0 tx=0).stop 2024-04-26T21:55:07.536 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:07.535+0000 7f4d86a00640 1 -- 172.21.15.72:0/4035139909 shutdown_connections 2024-04-26T21:55:07.536 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:07.535+0000 7f4d86a00640 1 --2- 172.21.15.72:0/4035139909 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4d801086a0 0x7f4d801a22a0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:07.536 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:07.535+0000 7f4d86a00640 1 --2- 172.21.15.72:0/4035139909 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f4d640a3cc0 0x7f4d640a6180 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:07.536 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:07.535+0000 7f4d86a00640 1 --2- 172.21.15.72:0/4035139909 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4d80100f60 0x7f4d8019b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:07.536 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:07.535+0000 7f4d86a00640 1 --2- 172.21.15.72:0/4035139909 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4d80100590 0x7f4d8019ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:07.536 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:07.535+0000 7f4d86a00640 1 -- 172.21.15.72:0/4035139909 >> 172.21.15.72:0/4035139909 conn(0x7f4d800fc130 msgr2=0x7f4d800fe520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:55:07.537 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:07.536+0000 7f4d86a00640 1 -- 172.21.15.72:0/4035139909 shutdown_connections 2024-04-26T21:55:07.537 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:07.536+0000 7f4d86a00640 1 -- 172.21.15.72:0/4035139909 wait complete. 2024-04-26T21:55:07.557 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T21:55:07.755 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:07.755+0000 7f7d3fe00640 1 -- 172.21.15.72:0/903950074 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7d40106620 msgr2=0x7f7d4010ceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:07.755 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:07.755+0000 7f7d3fe00640 1 --2- 172.21.15.72:0/903950074 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7d40106620 0x7f7d4010ceb0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f7d38009ca0 tx=0x7f7d3802a8c0 comp rx=0 tx=0).stop 2024-04-26T21:55:07.755 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:07.755+0000 7f7d3fe00640 1 -- 172.21.15.72:0/903950074 shutdown_connections 2024-04-26T21:55:07.755 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:07.755+0000 7f7d3fe00640 1 --2- 172.21.15.72:0/903950074 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7d40106620 0x7f7d4010ceb0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:07.756 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:07.755+0000 7f7d3fe00640 1 --2- 172.21.15.72:0/903950074 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7d40105c60 0x7f7d401060e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:07.756 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:07.755+0000 7f7d3fe00640 1 --2- 172.21.15.72:0/903950074 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7d40104a70 0x7f7d40104e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:07.756 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:07.755+0000 7f7d3fe00640 1 -- 172.21.15.72:0/903950074 >> 172.21.15.72:0/903950074 conn(0x7f7d400fffe0 msgr2=0x7f7d40102420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:55:07.756 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:07.755+0000 7f7d3fe00640 1 -- 172.21.15.72:0/903950074 shutdown_connections 2024-04-26T21:55:07.756 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:07.755+0000 7f7d3fe00640 1 -- 172.21.15.72:0/903950074 wait complete. 2024-04-26T21:55:07.758 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:07.756+0000 7f7d3fe00640 1 Processor -- start 2024-04-26T21:55:07.758 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:07.756+0000 7f7d3fe00640 1 -- start start 2024-04-26T21:55:07.758 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:07.757+0000 7f7d3fe00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7d40104a70 0x7f7d4019ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:07.758 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:07.757+0000 7f7d3fe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7d40105c60 0x7f7d4019b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:07.758 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:07.757+0000 7f7d3fe00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7d40106620 0x7f7d401a2130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:07.759 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:07.757+0000 7f7d3fe00640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f7d401104a0 con 0x7f7d40104a70 2024-04-26T21:55:07.759 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:07.757+0000 7f7d3fe00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f7d40110320 con 0x7f7d40106620 2024-04-26T21:55:07.759 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:07.757+0000 7f7d3fe00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f7d40110620 con 0x7f7d40105c60 2024-04-26T21:55:07.759 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:07.757+0000 7f7d3ea00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7d40104a70 0x7f7d4019ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:07.759 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:07.757+0000 7f7d3ea00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7d40104a70 0x7f7d4019ab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:46246/0 (socket says 172.21.15.72:46246) 2024-04-26T21:55:07.759 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:07.757+0000 7f7d3ea00640 1 -- 172.21.15.72:0/2865859058 learned_addr learned my addr 172.21.15.72:0/2865859058 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:55:07.759 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:07.757+0000 7f7d3f400640 1 --2- 172.21.15.72:0/2865859058 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7d40106620 0x7f7d401a2130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:07.759 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:07.757+0000 7f7d3ea00640 1 -- 172.21.15.72:0/2865859058 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7d40105c60 msgr2=0x7f7d4019b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:07.759 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:07.757+0000 7f7d3ea00640 1 --2- 172.21.15.72:0/2865859058 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7d40105c60 0x7f7d4019b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:07.759 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:07.757+0000 7f7d3ea00640 1 -- 172.21.15.72:0/2865859058 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7d40106620 msgr2=0x7f7d401a2130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:07.759 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:07.757+0000 7f7d3ea00640 1 --2- 172.21.15.72:0/2865859058 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7d40106620 0x7f7d401a2130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:07.759 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:07.757+0000 7f7d3ea00640 1 -- 172.21.15.72:0/2865859058 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d401a27a0 con 0x7f7d40104a70 2024-04-26T21:55:07.759 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:07.758+0000 7f7d3ea00640 1 --2- 172.21.15.72:0/2865859058 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7d40104a70 0x7f7d4019ab80 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f7d3000d950 tx=0x7f7d3000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:55:07.759 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:07.758+0000 7f7d37400640 1 -- 172.21.15.72:0/2865859058 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d30017070 con 0x7f7d40104a70 2024-04-26T21:55:07.759 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:07.758+0000 7f7d37400640 1 -- 172.21.15.72:0/2865859058 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7d30009c30 con 0x7f7d40104a70 2024-04-26T21:55:07.760 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:07.758+0000 7f7d3fe00640 1 -- 172.21.15.72:0/2865859058 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d401a2a90 con 0x7f7d40104a70 2024-04-26T21:55:07.760 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:07.758+0000 7f7d37400640 1 -- 172.21.15.72:0/2865859058 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d3000f620 con 0x7f7d40104a70 2024-04-26T21:55:07.760 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:07.758+0000 7f7d3fe00640 1 -- 172.21.15.72:0/2865859058 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d401a2fe0 con 0x7f7d40104a70 2024-04-26T21:55:07.760 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:07.759+0000 7f7d3fe00640 1 -- 172.21.15.72:0/2865859058 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d4010be10 con 0x7f7d40104a70 2024-04-26T21:55:07.762 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:07.762+0000 7f7d37400640 1 -- 172.21.15.72:0/2865859058 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f7d3001a070 con 0x7f7d40104a70 2024-04-26T21:55:07.763 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:07.763+0000 7f7d37400640 1 --2- 172.21.15.72:0/2865859058 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f7d240a3cc0 0x7f7d240a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:07.763 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:07.763+0000 7f7d37400640 1 -- 172.21.15.72:0/2865859058 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7f7d300d00f0 con 0x7f7d40104a70 2024-04-26T21:55:07.763 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:07.763+0000 7f7d3e000640 1 --2- 172.21.15.72:0/2865859058 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f7d240a3cc0 0x7f7d240a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:07.764 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:07.764+0000 7f7d3e000640 1 --2- 172.21.15.72:0/2865859058 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f7d240a3cc0 0x7f7d240a6180 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f7d2c005fd0 tx=0x7f7d2c005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:55:07.768 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:07.768+0000 7f7d37400640 1 -- 172.21.15.72:0/2865859058 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f7d30009de0 con 0x7f7d40104a70 2024-04-26T21:55:07.970 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:07.970+0000 7f7d3fe00640 1 -- 172.21.15.72:0/2865859058 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f7d4002dd80 con 0x7f7d40104a70 2024-04-26T21:55:07.971 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:07.971+0000 7f7d37400640 1 -- 172.21.15.72:0/2865859058 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v91) ==== 104+0+11 (secure 0 0 0) 0x7f7d3009c890 con 0x7f7d40104a70 2024-04-26T21:55:07.971 INFO:teuthology.orchestra.run.smithi072.stdout:pg_num: 32 2024-04-26T21:55:07.974 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:07.973+0000 7f7d3fe00640 1 -- 172.21.15.72:0/2865859058 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f7d240a3cc0 msgr2=0x7f7d240a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:07.974 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:07.973+0000 7f7d3fe00640 1 --2- 172.21.15.72:0/2865859058 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f7d240a3cc0 0x7f7d240a6180 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f7d2c005fd0 tx=0x7f7d2c005850 comp rx=0 tx=0).stop 2024-04-26T21:55:07.974 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:07.973+0000 7f7d3fe00640 1 -- 172.21.15.72:0/2865859058 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7d40104a70 msgr2=0x7f7d4019ab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:07.974 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:07.973+0000 7f7d3fe00640 1 --2- 172.21.15.72:0/2865859058 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7d40104a70 0x7f7d4019ab80 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f7d3000d950 tx=0x7f7d3000de10 comp rx=0 tx=0).stop 2024-04-26T21:55:07.974 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:07.974+0000 7f7d3fe00640 1 -- 172.21.15.72:0/2865859058 shutdown_connections 2024-04-26T21:55:07.974 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:07.974+0000 7f7d3fe00640 1 --2- 172.21.15.72:0/2865859058 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7d40106620 0x7f7d401a2130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:07.974 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:07.974+0000 7f7d3fe00640 1 --2- 172.21.15.72:0/2865859058 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f7d240a3cc0 0x7f7d240a6180 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:07.974 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:07.974+0000 7f7d3fe00640 1 --2- 172.21.15.72:0/2865859058 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7d40105c60 0x7f7d4019b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:07.974 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:07.974+0000 7f7d3fe00640 1 --2- 172.21.15.72:0/2865859058 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7d40104a70 0x7f7d4019ab80 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:07.974 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:07.974+0000 7f7d3fe00640 1 -- 172.21.15.72:0/2865859058 >> 172.21.15.72:0/2865859058 conn(0x7f7d400fffe0 msgr2=0x7f7d40108fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:55:07.974 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:07.974+0000 7f7d3fe00640 1 -- 172.21.15.72:0/2865859058 shutdown_connections 2024-04-26T21:55:07.975 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:07.974+0000 7f7d3fe00640 1 -- 172.21.15.72:0/2865859058 wait complete. 2024-04-26T21:55:07.993 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T21:55:08.201 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:08.200+0000 7fe629800640 1 -- 172.21.15.72:0/1650396621 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe624108d90 msgr2=0x7fe6241091b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:08.201 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:08.200+0000 7fe629800640 1 --2- 172.21.15.72:0/1650396621 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe624108d90 0x7fe6241091b0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fe614009900 tx=0x7fe61402a410 comp rx=0 tx=0).stop 2024-04-26T21:55:08.201 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:08.200+0000 7fe629800640 1 -- 172.21.15.72:0/1650396621 shutdown_connections 2024-04-26T21:55:08.201 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:08.200+0000 7fe629800640 1 --2- 172.21.15.72:0/1650396621 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe62410a940 0x7fe6241111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:08.201 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:08.200+0000 7fe629800640 1 --2- 172.21.15.72:0/1650396621 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe624109f80 0x7fe62410a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:08.201 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:08.200+0000 7fe629800640 1 --2- 172.21.15.72:0/1650396621 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe624108d90 0x7fe6241091b0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:08.201 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:08.200+0000 7fe629800640 1 -- 172.21.15.72:0/1650396621 >> 172.21.15.72:0/1650396621 conn(0x7fe624077d30 msgr2=0x7fe62407a170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:55:08.201 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:08.200+0000 7fe629800640 1 -- 172.21.15.72:0/1650396621 shutdown_connections 2024-04-26T21:55:08.201 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:08.200+0000 7fe629800640 1 -- 172.21.15.72:0/1650396621 wait complete. 2024-04-26T21:55:08.203 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:08.201+0000 7fe629800640 1 Processor -- start 2024-04-26T21:55:08.203 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:08.201+0000 7fe629800640 1 -- start start 2024-04-26T21:55:08.203 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:08.202+0000 7fe629800640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe624108d90 0x7fe62419ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:08.204 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:08.202+0000 7fe629800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe624109f80 0x7fe62419f490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:08.204 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:08.202+0000 7fe629800640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe62410a940 0x7fe6241a64b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:08.204 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:08.202+0000 7fe629800640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fe6241147c0 con 0x7fe624108d90 2024-04-26T21:55:08.204 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:08.202+0000 7fe629800640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fe624114640 con 0x7fe62410a940 2024-04-26T21:55:08.204 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:08.202+0000 7fe629800640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe624114940 con 0x7fe624109f80 2024-04-26T21:55:08.204 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:08.202+0000 7fe623e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe624108d90 0x7fe62419ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:08.204 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:08.202+0000 7fe623e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe624108d90 0x7fe62419ef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:46274/0 (socket says 172.21.15.72:46274) 2024-04-26T21:55:08.204 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:08.202+0000 7fe623e00640 1 -- 172.21.15.72:0/2681435266 learned_addr learned my addr 172.21.15.72:0/2681435266 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:55:08.204 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:08.202+0000 7fe628e00640 1 --2- 172.21.15.72:0/2681435266 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe62410a940 0x7fe6241a64b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:08.204 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:08.202+0000 7fe623e00640 1 -- 172.21.15.72:0/2681435266 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe624109f80 msgr2=0x7fe62419f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:08.204 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:08.202+0000 7fe623e00640 1 --2- 172.21.15.72:0/2681435266 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe624109f80 0x7fe62419f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:08.204 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:08.202+0000 7fe623e00640 1 -- 172.21.15.72:0/2681435266 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe62410a940 msgr2=0x7fe6241a64b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:08.204 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:08.202+0000 7fe623e00640 1 --2- 172.21.15.72:0/2681435266 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe62410a940 0x7fe6241a64b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:08.204 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:08.202+0000 7fe623e00640 1 -- 172.21.15.72:0/2681435266 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6241a6b20 con 0x7fe624108d90 2024-04-26T21:55:08.204 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:08.203+0000 7fe623e00640 1 --2- 172.21.15.72:0/2681435266 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe624108d90 0x7fe62419ef50 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fe614031460 tx=0x7fe614003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:55:08.205 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:08.203+0000 7fe620c00640 1 -- 172.21.15.72:0/2681435266 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe61403f070 con 0x7fe624108d90 2024-04-26T21:55:08.205 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:08.203+0000 7fe620c00640 1 -- 172.21.15.72:0/2681435266 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe614031da0 con 0x7fe624108d90 2024-04-26T21:55:08.205 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:08.203+0000 7fe620c00640 1 -- 172.21.15.72:0/2681435266 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe614035030 con 0x7fe624108d90 2024-04-26T21:55:08.205 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:08.203+0000 7fe629800640 1 -- 172.21.15.72:0/2681435266 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6241a6db0 con 0x7fe624108d90 2024-04-26T21:55:08.205 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:08.203+0000 7fe629800640 1 -- 172.21.15.72:0/2681435266 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6241a7260 con 0x7fe624108d90 2024-04-26T21:55:08.205 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:08.204+0000 7fe629800640 1 -- 172.21.15.72:0/2681435266 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe624110220 con 0x7fe624108d90 2024-04-26T21:55:08.206 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:08.206+0000 7fe620c00640 1 -- 172.21.15.72:0/2681435266 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fe614042070 con 0x7fe624108d90 2024-04-26T21:55:08.208 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:08.208+0000 7fe620c00640 1 --2- 172.21.15.72:0/2681435266 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fe6080a3cc0 0x7fe6080a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:08.209 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:08.209+0000 7fe620c00640 1 -- 172.21.15.72:0/2681435266 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7fe6140f0f20 con 0x7fe624108d90 2024-04-26T21:55:08.209 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:08.209+0000 7fe623400640 1 --2- 172.21.15.72:0/2681435266 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fe6080a3cc0 0x7fe6080a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:08.209 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:08.209+0000 7fe623400640 1 --2- 172.21.15.72:0/2681435266 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fe6080a3cc0 0x7fe6080a6180 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fe610005fd0 tx=0x7fe610005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:55:08.213 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:08.213+0000 7fe620c00640 1 -- 172.21.15.72:0/2681435266 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fe6140bd6c0 con 0x7fe624108d90 2024-04-26T21:55:08.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:55:07 smithi076 ceph-mon[31823]: from='client.? 172.21.15.72:0/3137721325' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:55:08.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:55:07 smithi076 ceph-mon[31823]: from='client.? 172.21.15.72:0/4035139909' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T21:55:08.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:55:07 smithi076 ceph-mon[31823]: from='client.? 172.21.15.72:0/2865859058' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T21:55:08.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:55:07 smithi072 ceph-mon[28138]: from='client.? 172.21.15.72:0/3137721325' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:55:08.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:55:07 smithi072 ceph-mon[28138]: from='client.? 172.21.15.72:0/4035139909' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T21:55:08.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:55:07 smithi072 ceph-mon[28138]: from='client.? 172.21.15.72:0/2865859058' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T21:55:08.416 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:08.415+0000 7fe629800640 1 -- 172.21.15.72:0/2681435266 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fe62402dd60 con 0x7fe624108d90 2024-04-26T21:55:08.416 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:08.416+0000 7fe620c00640 1 -- 172.21.15.72:0/2681435266 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v91) ==== 100+0+11 (secure 0 0 0) 0x7fe6140bcfc0 con 0x7fe624108d90 2024-04-26T21:55:08.417 INFO:teuthology.orchestra.run.smithi072.stdout:pg_num: 64 2024-04-26T21:55:08.419 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:08.419+0000 7fe629800640 1 -- 172.21.15.72:0/2681435266 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fe6080a3cc0 msgr2=0x7fe6080a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:08.419 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:08.419+0000 7fe629800640 1 --2- 172.21.15.72:0/2681435266 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fe6080a3cc0 0x7fe6080a6180 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fe610005fd0 tx=0x7fe610005850 comp rx=0 tx=0).stop 2024-04-26T21:55:08.419 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:08.419+0000 7fe629800640 1 -- 172.21.15.72:0/2681435266 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe624108d90 msgr2=0x7fe62419ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:08.419 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:08.419+0000 7fe629800640 1 --2- 172.21.15.72:0/2681435266 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe624108d90 0x7fe62419ef50 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fe614031460 tx=0x7fe614003c80 comp rx=0 tx=0).stop 2024-04-26T21:55:08.420 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:08.419+0000 7fe629800640 1 -- 172.21.15.72:0/2681435266 shutdown_connections 2024-04-26T21:55:08.420 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:08.419+0000 7fe629800640 1 --2- 172.21.15.72:0/2681435266 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe62410a940 0x7fe6241a64b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:08.420 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:08.419+0000 7fe629800640 1 --2- 172.21.15.72:0/2681435266 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fe6080a3cc0 0x7fe6080a6180 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:08.420 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:08.419+0000 7fe629800640 1 --2- 172.21.15.72:0/2681435266 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe624109f80 0x7fe62419f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:08.420 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:08.419+0000 7fe629800640 1 --2- 172.21.15.72:0/2681435266 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe624108d90 0x7fe62419ef50 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:08.420 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:08.419+0000 7fe629800640 1 -- 172.21.15.72:0/2681435266 >> 172.21.15.72:0/2681435266 conn(0x7fe624077d30 msgr2=0x7fe62410d3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:55:08.420 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:08.419+0000 7fe629800640 1 -- 172.21.15.72:0/2681435266 shutdown_connections 2024-04-26T21:55:08.420 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:08.420+0000 7fe629800640 1 -- 172.21.15.72:0/2681435266 wait complete. 2024-04-26T21:55:08.439 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:55:08.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:55:07 smithi164 ceph-mon[31916]: from='client.? 172.21.15.72:0/3137721325' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:55:08.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:55:07 smithi164 ceph-mon[31916]: from='client.? 172.21.15.72:0/4035139909' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T21:55:08.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:55:07 smithi164 ceph-mon[31916]: from='client.? 172.21.15.72:0/2865859058' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T21:55:08.644 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:08.644+0000 7f5bb5200640 1 -- 172.21.15.72:0/4208512308 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5bb0075330 msgr2=0x7f5bb0075790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:08.644 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:08.644+0000 7f5bb5200640 1 --2- 172.21.15.72:0/4208512308 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5bb0075330 0x7f5bb0075790 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f5b9c009900 tx=0x7f5b9c02a480 comp rx=0 tx=0).stop 2024-04-26T21:55:08.644 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:08.644+0000 7f5bb5200640 1 -- 172.21.15.72:0/4208512308 shutdown_connections 2024-04-26T21:55:08.645 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:08.644+0000 7f5bb5200640 1 --2- 172.21.15.72:0/4208512308 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5bb010a640 0x7f5bb010cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:08.645 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:08.644+0000 7f5bb5200640 1 --2- 172.21.15.72:0/4208512308 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5bb0075330 0x7f5bb0075790 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:08.645 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:08.644+0000 7f5bb5200640 1 --2- 172.21.15.72:0/4208512308 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5bb0076910 0x7f5bb0074d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:08.645 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:08.644+0000 7f5bb5200640 1 -- 172.21.15.72:0/4208512308 >> 172.21.15.72:0/4208512308 conn(0x7f5bb00fdd40 msgr2=0x7f5bb0100160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:55:08.645 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:08.644+0000 7f5bb5200640 1 -- 172.21.15.72:0/4208512308 shutdown_connections 2024-04-26T21:55:08.645 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:08.644+0000 7f5bb5200640 1 -- 172.21.15.72:0/4208512308 wait complete. 2024-04-26T21:55:08.647 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:08.645+0000 7f5bb5200640 1 Processor -- start 2024-04-26T21:55:08.647 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:08.645+0000 7f5bb5200640 1 -- start start 2024-04-26T21:55:08.648 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:08.645+0000 7f5bb5200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5bb0075330 0x7f5bb019aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:08.648 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:08.645+0000 7f5bb5200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5bb0076910 0x7f5bb019afe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:08.648 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:08.645+0000 7f5bb5200640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5bb010a640 0x7f5bb01a2050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:08.648 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:08.645+0000 7f5bb5200640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f5bb0110390 con 0x7f5bb010a640 2024-04-26T21:55:08.648 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:08.645+0000 7f5bb5200640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f5bb0110210 con 0x7f5bb0076910 2024-04-26T21:55:08.648 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:08.645+0000 7f5bb5200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f5bb0110510 con 0x7f5bb0075330 2024-04-26T21:55:08.648 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:08.646+0000 7f5bafe00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5bb010a640 0x7f5bb01a2050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:08.648 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:08.646+0000 7f5bafe00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5bb010a640 0x7f5bb01a2050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:46300/0 (socket says 172.21.15.72:46300) 2024-04-26T21:55:08.648 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:08.646+0000 7f5bafe00640 1 -- 172.21.15.72:0/2617571208 learned_addr learned my addr 172.21.15.72:0/2617571208 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:55:08.648 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:08.646+0000 7f5baf400640 1 --2- 172.21.15.72:0/2617571208 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5bb0075330 0x7f5bb019aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:08.648 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:08.646+0000 7f5baea00640 1 --2- 172.21.15.72:0/2617571208 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5bb0076910 0x7f5bb019afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:08.648 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:08.646+0000 7f5bafe00640 1 -- 172.21.15.72:0/2617571208 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5bb0075330 msgr2=0x7f5bb019aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:08.648 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:08.646+0000 7f5bafe00640 1 --2- 172.21.15.72:0/2617571208 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5bb0075330 0x7f5bb019aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:08.648 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:08.646+0000 7f5bafe00640 1 -- 172.21.15.72:0/2617571208 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5bb0076910 msgr2=0x7f5bb019afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:08.649 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:08.646+0000 7f5bafe00640 1 --2- 172.21.15.72:0/2617571208 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5bb0076910 0x7f5bb019afe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:08.649 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:08.646+0000 7f5bafe00640 1 -- 172.21.15.72:0/2617571208 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bb01a26c0 con 0x7f5bb010a640 2024-04-26T21:55:08.649 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:08.646+0000 7f5bafe00640 1 --2- 172.21.15.72:0/2617571208 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5bb010a640 0x7f5bb01a2050 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f5ba800eaa0 tx=0x7f5ba800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:55:08.649 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:08.647+0000 7f5ba7e00640 1 -- 172.21.15.72:0/2617571208 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ba800cdb0 con 0x7f5bb010a640 2024-04-26T21:55:08.649 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:08.647+0000 7f5ba7e00640 1 -- 172.21.15.72:0/2617571208 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5ba8005020 con 0x7f5bb010a640 2024-04-26T21:55:08.649 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:08.647+0000 7f5bb5200640 1 -- 172.21.15.72:0/2617571208 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5bb01a29b0 con 0x7f5bb010a640 2024-04-26T21:55:08.649 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:08.647+0000 7f5ba7e00640 1 -- 172.21.15.72:0/2617571208 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ba80116d0 con 0x7f5bb010a640 2024-04-26T21:55:08.649 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:08.647+0000 7f5bb5200640 1 -- 172.21.15.72:0/2617571208 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5bb01a2ed0 con 0x7f5bb010a640 2024-04-26T21:55:08.649 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:08.648+0000 7f5bb5200640 1 -- 172.21.15.72:0/2617571208 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5bb0109400 con 0x7f5bb010a640 2024-04-26T21:55:08.650 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:08.650+0000 7f5ba7e00640 1 -- 172.21.15.72:0/2617571208 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f5ba80051d0 con 0x7f5bb010a640 2024-04-26T21:55:08.652 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:08.652+0000 7f5ba7e00640 1 --2- 172.21.15.72:0/2617571208 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f5b940a3cc0 0x7f5b940a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:08.653 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:08.653+0000 7f5ba7e00640 1 -- 172.21.15.72:0/2617571208 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7f5ba80d0820 con 0x7f5bb010a640 2024-04-26T21:55:08.653 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:08.653+0000 7f5baf400640 1 --2- 172.21.15.72:0/2617571208 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f5b940a3cc0 0x7f5b940a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:08.653 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:08.653+0000 7f5baf400640 1 --2- 172.21.15.72:0/2617571208 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f5b940a3cc0 0x7f5b940a6180 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f5ba0005fd0 tx=0x7f5ba0005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:55:08.657 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:08.657+0000 7f5ba7e00640 1 -- 172.21.15.72:0/2617571208 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f5ba8016030 con 0x7f5bb010a640 2024-04-26T21:55:08.902 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:08.902+0000 7f5bb5200640 1 -- 172.21.15.72:0/2617571208 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5bb002dad0 con 0x7f5bb010a640 2024-04-26T21:55:08.904 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:08.903+0000 7f5ba7e00640 1 -- 172.21.15.72:0/2617571208 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11931 (secure 0 0 0) 0x7f5ba809d020 con 0x7f5bb010a640 2024-04-26T21:55:08.904 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T21:55:08.904 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6835/1320381959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6834","nonce":1320381959},{"type":"v1","addr":"172.21.15.72:6835","nonce":1320381959}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6837/2142200512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6836","nonce":2142200512},{"type":"v1","addr":"172.21.15.72:6837","nonce":2142200512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T21:52:50.567857+0000","modified":"2024-04-26T21:52:58.508126+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34517,"mds_1":34472,"mds_2":34442,"mds_3":34406,"mds_4":24463},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14652":{"gid":14652,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.72:6839/1312898281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6838","nonce":1312898281},{"type":"v1","addr":"172.21.15.72:6839","nonce":1312898281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14664":{"gid":14664,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.72:6841/3257692339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6840","nonce":3257692339},{"type":"v1","addr":"172.21.15.72:6841","nonce":3257692339}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24388":{"gid":24388,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6833/4236668465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":4236668465},{"type":"v1","addr":"172.21.15.164:6833","nonce":4236668465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6835/2391114169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":2391114169},{"type":"v1","addr":"172.21.15.164:6835","nonce":2391114169}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6837/3585829992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":3585829992},{"type":"v1","addr":"172.21.15.164:6837","nonce":3585829992}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.164:6839/1636226820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":1636226820},{"type":"v1","addr":"172.21.15.164:6839","nonce":1636226820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34406":{"gid":34406,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":15,"addr":"172.21.15.76:6833/3869483750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":3869483750},{"type":"v1","addr":"172.21.15.76:6833","nonce":3869483750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34442":{"gid":34442,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.76:6835/3517429435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":3517429435},{"type":"v1","addr":"172.21.15.76:6835","nonce":3517429435}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34472":{"gid":34472,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.76:6837/1150678991","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":1150678991},{"type":"v1","addr":"172.21.15.76:6837","nonce":1150678991}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34517":{"gid":34517,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.76:6839/327145636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6838","nonce":327145636},{"type":"v1","addr":"172.21.15.76:6839","nonce":327145636}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34517,"qdb_cluster":[34406,24463,34442,34517,34472]},"id":1}]} 2024-04-26T21:55:08.904 INFO:teuthology.orchestra.run.smithi072.stderr:dumped fsmap epoch 26 2024-04-26T21:55:08.906 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:08.906+0000 7f5bb5200640 1 -- 172.21.15.72:0/2617571208 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f5b940a3cc0 msgr2=0x7f5b940a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:08.907 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:08.906+0000 7f5bb5200640 1 --2- 172.21.15.72:0/2617571208 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f5b940a3cc0 0x7f5b940a6180 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f5ba0005fd0 tx=0x7f5ba0005850 comp rx=0 tx=0).stop 2024-04-26T21:55:08.907 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:08.906+0000 7f5bb5200640 1 -- 172.21.15.72:0/2617571208 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5bb010a640 msgr2=0x7f5bb01a2050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:08.907 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:08.906+0000 7f5bb5200640 1 --2- 172.21.15.72:0/2617571208 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5bb010a640 0x7f5bb01a2050 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f5ba800eaa0 tx=0x7f5ba800ef60 comp rx=0 tx=0).stop 2024-04-26T21:55:08.907 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:08.906+0000 7f5bb5200640 1 -- 172.21.15.72:0/2617571208 shutdown_connections 2024-04-26T21:55:08.907 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:08.906+0000 7f5bb5200640 1 --2- 172.21.15.72:0/2617571208 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5bb010a640 0x7f5bb01a2050 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:08.907 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:08.906+0000 7f5bb5200640 1 --2- 172.21.15.72:0/2617571208 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5bb0076910 0x7f5bb019afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:08.907 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:08.906+0000 7f5bb5200640 1 --2- 172.21.15.72:0/2617571208 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f5b940a3cc0 0x7f5b940a6180 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:08.907 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:08.906+0000 7f5bb5200640 1 --2- 172.21.15.72:0/2617571208 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5bb0075330 0x7f5bb019aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:08.907 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:08.906+0000 7f5bb5200640 1 -- 172.21.15.72:0/2617571208 >> 172.21.15.72:0/2617571208 conn(0x7f5bb00fdd40 msgr2=0x7f5bb01065d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:55:08.907 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:08.907+0000 7f5bb5200640 1 -- 172.21.15.72:0/2617571208 shutdown_connections 2024-04-26T21:55:08.907 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:08.907+0000 7f5bb5200640 1 -- 172.21.15.72:0/2617571208 wait complete. 2024-04-26T21:55:08.925 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:55:09.129 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:09.128+0000 7fc5e3200640 1 -- 172.21.15.72:0/2978560469 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc5e4104320 msgr2=0x7fc5e410abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:09.129 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:09.128+0000 7fc5e3200640 1 --2- 172.21.15.72:0/2978560469 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc5e4104320 0x7fc5e410abb0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fc5dc009ca0 tx=0x7fc5dc02a8c0 comp rx=0 tx=0).stop 2024-04-26T21:55:09.129 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:09.128+0000 7fc5e3200640 1 -- 172.21.15.72:0/2978560469 shutdown_connections 2024-04-26T21:55:09.129 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:09.128+0000 7fc5e3200640 1 --2- 172.21.15.72:0/2978560469 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc5e4104320 0x7fc5e410abb0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:09.129 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:09.128+0000 7fc5e3200640 1 --2- 172.21.15.72:0/2978560469 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc5e4103960 0x7fc5e4103de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:09.129 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:09.128+0000 7fc5e3200640 1 --2- 172.21.15.72:0/2978560469 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc5e4102770 0x7fc5e4102b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:09.129 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:09.128+0000 7fc5e3200640 1 -- 172.21.15.72:0/2978560469 >> 172.21.15.72:0/2978560469 conn(0x7fc5e40fdce0 msgr2=0x7fc5e4100120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:55:09.129 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:09.128+0000 7fc5e3200640 1 -- 172.21.15.72:0/2978560469 shutdown_connections 2024-04-26T21:55:09.129 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:09.129+0000 7fc5e3200640 1 -- 172.21.15.72:0/2978560469 wait complete. 2024-04-26T21:55:09.131 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:09.129+0000 7fc5e3200640 1 Processor -- start 2024-04-26T21:55:09.132 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:09.130+0000 7fc5e3200640 1 -- start start 2024-04-26T21:55:09.132 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:09.130+0000 7fc5e3200640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc5e4102770 0x7fc5e419aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:09.132 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:09.130+0000 7fc5e3200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc5e4103960 0x7fc5e419afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:09.132 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:09.130+0000 7fc5e3200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc5e4104320 0x7fc5e41a2010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:09.132 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:09.130+0000 7fc5e3200640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fc5e410e1a0 con 0x7fc5e4102770 2024-04-26T21:55:09.132 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:09.130+0000 7fc5e3200640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fc5e410e020 con 0x7fc5e4103960 2024-04-26T21:55:09.132 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:09.130+0000 7fc5e3200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fc5e410e320 con 0x7fc5e4104320 2024-04-26T21:55:09.132 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:09.130+0000 7fc5e1e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc5e4102770 0x7fc5e419aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:09.132 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:09.130+0000 7fc5e1e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc5e4102770 0x7fc5e419aa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:46324/0 (socket says 172.21.15.72:46324) 2024-04-26T21:55:09.132 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:09.130+0000 7fc5e1e00640 1 -- 172.21.15.72:0/3889719323 learned_addr learned my addr 172.21.15.72:0/3889719323 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:55:09.132 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:09.131+0000 7fc5d9400640 1 --2- 172.21.15.72:0/3889719323 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc5e4103960 0x7fc5e419afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:09.132 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:09.131+0000 7fc5e1e00640 1 -- 172.21.15.72:0/3889719323 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc5e4104320 msgr2=0x7fc5e41a2010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:09.133 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:09.131+0000 7fc5e2800640 1 --2- 172.21.15.72:0/3889719323 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc5e4104320 0x7fc5e41a2010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:09.133 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:09.131+0000 7fc5e1e00640 1 --2- 172.21.15.72:0/3889719323 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc5e4104320 0x7fc5e41a2010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:09.133 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:09.131+0000 7fc5e1e00640 1 -- 172.21.15.72:0/3889719323 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc5e4103960 msgr2=0x7fc5e419afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:09.133 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:09.131+0000 7fc5e1e00640 1 --2- 172.21.15.72:0/3889719323 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc5e4103960 0x7fc5e419afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:09.133 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:09.131+0000 7fc5e1e00640 1 -- 172.21.15.72:0/3889719323 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5e41a2680 con 0x7fc5e4102770 2024-04-26T21:55:09.133 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:09.131+0000 7fc5e2800640 1 --2- 172.21.15.72:0/3889719323 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc5e4104320 0x7fc5e41a2010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:55:09.133 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:09.131+0000 7fc5d9400640 1 --2- 172.21.15.72:0/3889719323 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc5e4103960 0x7fc5e419afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:55:09.133 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:09.131+0000 7fc5e1e00640 1 --2- 172.21.15.72:0/3889719323 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc5e4102770 0x7fc5e419aa60 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fc5d400d770 tx=0x7fc5d400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:55:09.133 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:09.131+0000 7fc5db400640 1 -- 172.21.15.72:0/3889719323 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5d4017070 con 0x7fc5e4102770 2024-04-26T21:55:09.133 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:09.131+0000 7fc5db400640 1 -- 172.21.15.72:0/3889719323 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc5d4009c30 con 0x7fc5e4102770 2024-04-26T21:55:09.133 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:09.131+0000 7fc5e3200640 1 -- 172.21.15.72:0/3889719323 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5e41a2970 con 0x7fc5e4102770 2024-04-26T21:55:09.133 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:09.131+0000 7fc5db400640 1 -- 172.21.15.72:0/3889719323 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5d4003c50 con 0x7fc5e4102770 2024-04-26T21:55:09.133 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:09.132+0000 7fc5e3200640 1 -- 172.21.15.72:0/3889719323 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5e41a2ec0 con 0x7fc5e4102770 2024-04-26T21:55:09.134 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:09.133+0000 7fc5e3200640 1 -- 172.21.15.72:0/3889719323 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5e41a30f0 con 0x7fc5e4102770 2024-04-26T21:55:09.134 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:09.135+0000 7fc5db400640 1 -- 172.21.15.72:0/3889719323 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fc5d4003df0 con 0x7fc5e4102770 2024-04-26T21:55:09.136 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:09.136+0000 7fc5db400640 1 --2- 172.21.15.72:0/3889719323 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fc5c80a3c40 0x7fc5c80a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:09.136 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:09.136+0000 7fc5db400640 1 -- 172.21.15.72:0/3889719323 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7fc5d40d00e0 con 0x7fc5e4102770 2024-04-26T21:55:09.136 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:09.136+0000 7fc5d9400640 1 --2- 172.21.15.72:0/3889719323 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fc5c80a3c40 0x7fc5c80a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:09.137 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:09.137+0000 7fc5d9400640 1 --2- 172.21.15.72:0/3889719323 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fc5c80a3c40 0x7fc5c80a6100 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fc5e419bf70 tx=0x7fc5cc005e50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:55:09.140 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:09.140+0000 7fc5db400640 1 -- 172.21.15.72:0/3889719323 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fc5d409c8e0 con 0x7fc5e4102770 2024-04-26T21:55:09.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:55:09 smithi076 ceph-mon[31823]: pgmap v440: 97 pgs: 97 active+clean; 633 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-26T21:55:09.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:55:09 smithi076 ceph-mon[31823]: from='client.? 172.21.15.72:0/2681435266' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T21:55:09.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:55:09 smithi076 ceph-mon[31823]: from='client.? 172.21.15.72:0/2617571208' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:55:09.321 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:55:09 smithi072 ceph-mon[28138]: pgmap v440: 97 pgs: 97 active+clean; 633 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-26T21:55:09.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:55:09 smithi072 ceph-mon[28138]: from='client.? 172.21.15.72:0/2681435266' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T21:55:09.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:55:09 smithi072 ceph-mon[28138]: from='client.? 172.21.15.72:0/2617571208' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:55:09.344 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:09.344+0000 7fc5e3200640 1 -- 172.21.15.72:0/3889719323 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc5e41a3360 con 0x7fc5e4102770 2024-04-26T21:55:09.347 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:09.347+0000 7fc5db400640 1 -- 172.21.15.72:0/3889719323 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+22783 (secure 0 0 0) 0x7fc5d4009de0 con 0x7fc5e4102770 2024-04-26T21:55:09.347 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T21:55:09.347 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":91,"fsid":"a1780950-0415-11ef-bc93-c7b262605968","created":"2024-04-26T21:42:01.504173+0000","modified":"2024-04-26T21:54:11.892872+0000","last_up_change":"2024-04-26T21:51:43.482437+0000","last_in_change":"2024-04-26T21:51:18.468587+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:46:29.785427+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:52:48.320391+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T21:52:49.517493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"04e06884-b355-4762-9b29-9034b7201e9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6802","nonce":1416765179},{"type":"v1","addr":"172.21.15.72:6803","nonce":1416765179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6804","nonce":1416765179},{"type":"v1","addr":"172.21.15.72:6805","nonce":1416765179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6808","nonce":1416765179},{"type":"v1","addr":"172.21.15.72:6809","nonce":1416765179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6806","nonce":1416765179},{"type":"v1","addr":"172.21.15.72:6807","nonce":1416765179}]},"public_addr":"172.21.15.72:6803/1416765179","cluster_addr":"172.21.15.72:6805/1416765179","heartbeat_back_addr":"172.21.15.72:6809/1416765179","heartbeat_front_addr":"172.21.15.72:6807/1416765179","state":["exists","up"]},{"osd":1,"uuid":"d75a479a-4a2e-48e9-a01e-2b1c9617416a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":755426949},{"type":"v1","addr":"172.21.15.76:6801","nonce":755426949}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":755426949},{"type":"v1","addr":"172.21.15.76:6803","nonce":755426949}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":755426949},{"type":"v1","addr":"172.21.15.76:6807","nonce":755426949}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":755426949},{"type":"v1","addr":"172.21.15.76:6805","nonce":755426949}]},"public_addr":"172.21.15.76:6801/755426949","cluster_addr":"172.21.15.76:6803/755426949","heartbeat_back_addr":"172.21.15.76:6807/755426949","heartbeat_front_addr":"172.21.15.76:6805/755426949","state":["exists","up"]},{"osd":2,"uuid":"b6f8da58-4428-4e8c-a944-7ea5645d6d92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":852265973},{"type":"v1","addr":"172.21.15.164:6801","nonce":852265973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":852265973},{"type":"v1","addr":"172.21.15.164:6803","nonce":852265973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":852265973},{"type":"v1","addr":"172.21.15.164:6807","nonce":852265973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":852265973},{"type":"v1","addr":"172.21.15.164:6805","nonce":852265973}]},"public_addr":"172.21.15.164:6801/852265973","cluster_addr":"172.21.15.164:6803/852265973","heartbeat_back_addr":"172.21.15.164:6807/852265973","heartbeat_front_addr":"172.21.15.164:6805/852265973","state":["exists","up"]},{"osd":3,"uuid":"76d83409-649e-41e5-b4f4-43bbd5b00ea5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6810","nonce":4027763789},{"type":"v1","addr":"172.21.15.72:6811","nonce":4027763789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6812","nonce":4027763789},{"type":"v1","addr":"172.21.15.72:6813","nonce":4027763789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6816","nonce":4027763789},{"type":"v1","addr":"172.21.15.72:6817","nonce":4027763789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6814","nonce":4027763789},{"type":"v1","addr":"172.21.15.72:6815","nonce":4027763789}]},"public_addr":"172.21.15.72:6811/4027763789","cluster_addr":"172.21.15.72:6813/4027763789","heartbeat_back_addr":"172.21.15.72:6817/4027763789","heartbeat_front_addr":"172.21.15.72:6815/4027763789","state":["exists","up"]},{"osd":4,"uuid":"d2cfeea7-1661-4156-bdcd-f770a8d1a878","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":3718867827},{"type":"v1","addr":"172.21.15.76:6809","nonce":3718867827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":3718867827},{"type":"v1","addr":"172.21.15.76:6811","nonce":3718867827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":3718867827},{"type":"v1","addr":"172.21.15.76:6815","nonce":3718867827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":3718867827},{"type":"v1","addr":"172.21.15.76:6813","nonce":3718867827}]},"public_addr":"172.21.15.76:6809/3718867827","cluster_addr":"172.21.15.76:6811/3718867827","heartbeat_back_addr":"172.21.15.76:6815/3718867827","heartbeat_front_addr":"172.21.15.76:6813/3718867827","state":["exists","up"]},{"osd":5,"uuid":"f3896ad6-a040-4ea3-a431-ff9b7fc05c36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":2543954772},{"type":"v1","addr":"172.21.15.164:6809","nonce":2543954772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":2543954772},{"type":"v1","addr":"172.21.15.164:6811","nonce":2543954772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":2543954772},{"type":"v1","addr":"172.21.15.164:6815","nonce":2543954772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":2543954772},{"type":"v1","addr":"172.21.15.164:6813","nonce":2543954772}]},"public_addr":"172.21.15.164:6809/2543954772","cluster_addr":"172.21.15.164:6811/2543954772","heartbeat_back_addr":"172.21.15.164:6815/2543954772","heartbeat_front_addr":"172.21.15.164:6813/2543954772","state":["exists","up"]},{"osd":6,"uuid":"d5d56650-be7e-4ba5-ba3a-d7d473294dab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6818","nonce":2080659027},{"type":"v1","addr":"172.21.15.72:6819","nonce":2080659027}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6820","nonce":2080659027},{"type":"v1","addr":"172.21.15.72:6821","nonce":2080659027}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6824","nonce":2080659027},{"type":"v1","addr":"172.21.15.72:6825","nonce":2080659027}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6822","nonce":2080659027},{"type":"v1","addr":"172.21.15.72:6823","nonce":2080659027}]},"public_addr":"172.21.15.72:6819/2080659027","cluster_addr":"172.21.15.72:6821/2080659027","heartbeat_back_addr":"172.21.15.72:6825/2080659027","heartbeat_front_addr":"172.21.15.72:6823/2080659027","state":["exists","up"]},{"osd":7,"uuid":"18975b5a-9309-4a0f-9f37-3ce09c2e745d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":3144631323},{"type":"v1","addr":"172.21.15.76:6817","nonce":3144631323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":3144631323},{"type":"v1","addr":"172.21.15.76:6819","nonce":3144631323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":3144631323},{"type":"v1","addr":"172.21.15.76:6823","nonce":3144631323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":3144631323},{"type":"v1","addr":"172.21.15.76:6821","nonce":3144631323}]},"public_addr":"172.21.15.76:6817/3144631323","cluster_addr":"172.21.15.76:6819/3144631323","heartbeat_back_addr":"172.21.15.76:6823/3144631323","heartbeat_front_addr":"172.21.15.76:6821/3144631323","state":["exists","up"]},{"osd":8,"uuid":"779e9c1e-8e97-48d4-a854-68ca13f38e0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":627888883},{"type":"v1","addr":"172.21.15.164:6817","nonce":627888883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":627888883},{"type":"v1","addr":"172.21.15.164:6819","nonce":627888883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":627888883},{"type":"v1","addr":"172.21.15.164:6823","nonce":627888883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":627888883},{"type":"v1","addr":"172.21.15.164:6821","nonce":627888883}]},"public_addr":"172.21.15.164:6817/627888883","cluster_addr":"172.21.15.164:6819/627888883","heartbeat_back_addr":"172.21.15.164:6823/627888883","heartbeat_front_addr":"172.21.15.164:6821/627888883","state":["exists","up"]},{"osd":9,"uuid":"d0317850-7b2d-4850-b2cf-0b919505c1fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6826","nonce":1273118485},{"type":"v1","addr":"172.21.15.72:6827","nonce":1273118485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6828","nonce":1273118485},{"type":"v1","addr":"172.21.15.72:6829","nonce":1273118485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6832","nonce":1273118485},{"type":"v1","addr":"172.21.15.72:6833","nonce":1273118485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6830","nonce":1273118485},{"type":"v1","addr":"172.21.15.72:6831","nonce":1273118485}]},"public_addr":"172.21.15.72:6827/1273118485","cluster_addr":"172.21.15.72:6829/1273118485","heartbeat_back_addr":"172.21.15.72:6833/1273118485","heartbeat_front_addr":"172.21.15.72:6831/1273118485","state":["exists","up"]},{"osd":10,"uuid":"e0aa38d8-6011-4d2a-b67b-5bff6d999e47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":2808954459},{"type":"v1","addr":"172.21.15.76:6825","nonce":2808954459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":2808954459},{"type":"v1","addr":"172.21.15.76:6827","nonce":2808954459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":2808954459},{"type":"v1","addr":"172.21.15.76:6831","nonce":2808954459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":2808954459},{"type":"v1","addr":"172.21.15.76:6829","nonce":2808954459}]},"public_addr":"172.21.15.76:6825/2808954459","cluster_addr":"172.21.15.76:6827/2808954459","heartbeat_back_addr":"172.21.15.76:6831/2808954459","heartbeat_front_addr":"172.21.15.76:6829/2808954459","state":["exists","up"]},{"osd":11,"uuid":"981b3d4d-916e-4937-9916-7cc21a891e43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":2354951784},{"type":"v1","addr":"172.21.15.164:6825","nonce":2354951784}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":2354951784},{"type":"v1","addr":"172.21.15.164:6827","nonce":2354951784}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":2354951784},{"type":"v1","addr":"172.21.15.164:6831","nonce":2354951784}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":2354951784},{"type":"v1","addr":"172.21.15.164:6829","nonce":2354951784}]},"public_addr":"172.21.15.164:6825/2354951784","cluster_addr":"172.21.15.164:6827/2354951784","heartbeat_back_addr":"172.21.15.164:6831/2354951784","heartbeat_front_addr":"172.21.15.164:6829/2354951784","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:18.997924+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:52.439081+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:46:25.893288+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:47:00.112807+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:47:35.116874+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:48:09.216344+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:48:45.028853+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:49:19.905397+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:49:55.030536+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:50:30.767662+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:51:06.423267+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:51:41.631211+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.1a","mappings":[{"from":5,"to":7}]},{"pgid":"3.1b","mappings":[{"from":4,"to":6}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.28","mappings":[{"from":5,"to":6}]},{"pgid":"3.2a","mappings":[{"from":3,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.72:0/416277784":"2024-04-27T21:43:09.602791+0000","172.21.15.72:6800/4010358672":"2024-04-27T21:43:09.602791+0000","172.21.15.72:6801/1012617371":"2024-04-27T21:42:34.684837+0000","172.21.15.72:6801/4010358672":"2024-04-27T21:43:09.602791+0000","172.21.15.72:0/3555448179":"2024-04-27T21:43:09.602791+0000","172.21.15.72:6800/1012617371":"2024-04-27T21:42:34.684837+0000","172.21.15.72:0/2920378761":"2024-04-27T21:42:34.684837+0000","172.21.15.72:0/1933740041":"2024-04-27T21:43:09.602791+0000","172.21.15.72:0/3225295775":"2024-04-27T21:42:34.684837+0000","172.21.15.72:0/2326016180":"2024-04-27T21:42:34.684837+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:55:09.349 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:09.349+0000 7fc5e3200640 1 -- 172.21.15.72:0/3889719323 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fc5c80a3c40 msgr2=0x7fc5c80a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:09.349 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:09.349+0000 7fc5e3200640 1 --2- 172.21.15.72:0/3889719323 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fc5c80a3c40 0x7fc5c80a6100 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fc5e419bf70 tx=0x7fc5cc005e50 comp rx=0 tx=0).stop 2024-04-26T21:55:09.349 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:09.350+0000 7fc5e3200640 1 -- 172.21.15.72:0/3889719323 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc5e4102770 msgr2=0x7fc5e419aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:09.350 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:09.350+0000 7fc5e3200640 1 --2- 172.21.15.72:0/3889719323 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc5e4102770 0x7fc5e419aa60 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fc5d400d770 tx=0x7fc5d400dc30 comp rx=0 tx=0).stop 2024-04-26T21:55:09.350 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:09.350+0000 7fc5e3200640 1 -- 172.21.15.72:0/3889719323 shutdown_connections 2024-04-26T21:55:09.350 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:09.350+0000 7fc5e3200640 1 --2- 172.21.15.72:0/3889719323 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fc5c80a3c40 0x7fc5c80a6100 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:09.350 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:09.350+0000 7fc5e3200640 1 --2- 172.21.15.72:0/3889719323 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc5e4104320 0x7fc5e41a2010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:09.351 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:09.350+0000 7fc5e3200640 1 --2- 172.21.15.72:0/3889719323 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc5e4103960 0x7fc5e419afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:09.351 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:09.350+0000 7fc5e3200640 1 --2- 172.21.15.72:0/3889719323 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc5e4102770 0x7fc5e419aa60 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:09.351 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:09.350+0000 7fc5e3200640 1 -- 172.21.15.72:0/3889719323 >> 172.21.15.72:0/3889719323 conn(0x7fc5e40fdce0 msgr2=0x7fc5e4074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:55:09.351 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:09.350+0000 7fc5e3200640 1 -- 172.21.15.72:0/3889719323 shutdown_connections 2024-04-26T21:55:09.351 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:09.350+0000 7fc5e3200640 1 -- 172.21.15.72:0/3889719323 wait complete. 2024-04-26T21:55:09.369 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:46:29.785427+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T21:52:48.320391+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T21:52:49.517493+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:55:09.370 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:55:09.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:55:09 smithi164 ceph-mon[31916]: pgmap v440: 97 pgs: 97 active+clean; 633 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-26T21:55:09.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:55:09 smithi164 ceph-mon[31916]: from='client.? 172.21.15.72:0/2681435266' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T21:55:09.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:55:09 smithi164 ceph-mon[31916]: from='client.? 172.21.15.72:0/2617571208' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:55:09.574 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:09.574+0000 7f7a5fe00640 1 -- 172.21.15.72:0/2849383887 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7a60100b10 msgr2=0x7f7a60100f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:09.574 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:09.574+0000 7f7a5fe00640 1 --2- 172.21.15.72:0/2849383887 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7a60100b10 0x7f7a60100f30 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f7a50009900 tx=0x7f7a5002a3f0 comp rx=0 tx=0).stop 2024-04-26T21:55:09.575 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:09.574+0000 7f7a5fe00640 1 -- 172.21.15.72:0/2849383887 shutdown_connections 2024-04-26T21:55:09.575 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:09.574+0000 7f7a5fe00640 1 --2- 172.21.15.72:0/2849383887 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7a60102230 0x7f7a60102ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:09.575 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:09.574+0000 7f7a5fe00640 1 --2- 172.21.15.72:0/2849383887 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7a60101870 0x7f7a60101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:09.575 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:09.574+0000 7f7a5fe00640 1 --2- 172.21.15.72:0/2849383887 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7a60100b10 0x7f7a60100f30 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:09.575 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:09.574+0000 7f7a5fe00640 1 -- 172.21.15.72:0/2849383887 >> 172.21.15.72:0/2849383887 conn(0x7f7a600fc460 msgr2=0x7f7a600fe880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:55:09.575 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:09.574+0000 7f7a5fe00640 1 -- 172.21.15.72:0/2849383887 shutdown_connections 2024-04-26T21:55:09.575 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:09.574+0000 7f7a5fe00640 1 -- 172.21.15.72:0/2849383887 wait complete. 2024-04-26T21:55:09.577 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:09.575+0000 7f7a5fe00640 1 Processor -- start 2024-04-26T21:55:09.577 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:09.575+0000 7f7a5fe00640 1 -- start start 2024-04-26T21:55:09.577 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:09.575+0000 7f7a5fe00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7a60100b10 0x7f7a6019ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:09.577 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:09.575+0000 7f7a5fe00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7a60101870 0x7f7a6019b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:09.577 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:09.575+0000 7f7a5fe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7a60102230 0x7f7a601a20f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:09.578 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:09.575+0000 7f7a5fe00640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f7a60110470 con 0x7f7a60101870 2024-04-26T21:55:09.578 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:09.575+0000 7f7a5fe00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f7a601102f0 con 0x7f7a60100b10 2024-04-26T21:55:09.578 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:09.575+0000 7f7a5fe00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f7a601105f0 con 0x7f7a60102230 2024-04-26T21:55:09.578 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:09.576+0000 7f7a57e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7a60101870 0x7f7a6019b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:09.578 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:09.576+0000 7f7a57e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7a60101870 0x7f7a6019b080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:46332/0 (socket says 172.21.15.72:46332) 2024-04-26T21:55:09.578 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:09.576+0000 7f7a57e00640 1 -- 172.21.15.72:0/159703843 learned_addr learned my addr 172.21.15.72:0/159703843 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:55:09.578 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:09.576+0000 7f7a5ea00640 1 --2- 172.21.15.72:0/159703843 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7a60100b10 0x7f7a6019ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:09.578 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:09.576+0000 7f7a5f400640 1 --2- 172.21.15.72:0/159703843 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7a60102230 0x7f7a601a20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:09.578 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:09.576+0000 7f7a57e00640 1 -- 172.21.15.72:0/159703843 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7a60102230 msgr2=0x7f7a601a20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:09.578 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:09.576+0000 7f7a57e00640 1 --2- 172.21.15.72:0/159703843 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7a60102230 0x7f7a601a20f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:09.578 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:09.576+0000 7f7a57e00640 1 -- 172.21.15.72:0/159703843 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7a60100b10 msgr2=0x7f7a6019ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:09.578 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:09.576+0000 7f7a57e00640 1 --2- 172.21.15.72:0/159703843 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7a60100b10 0x7f7a6019ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:09.578 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:09.576+0000 7f7a57e00640 1 -- 172.21.15.72:0/159703843 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a601a2760 con 0x7f7a60101870 2024-04-26T21:55:09.579 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:09.576+0000 7f7a5f400640 1 --2- 172.21.15.72:0/159703843 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7a60102230 0x7f7a601a20f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:55:09.579 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:09.576+0000 7f7a57e00640 1 --2- 172.21.15.72:0/159703843 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7a60101870 0x7f7a6019b080 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f7a4800c970 tx=0x7f7a4800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:55:09.579 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:09.577+0000 7f7a64a00640 1 -- 172.21.15.72:0/159703843 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a48007c10 con 0x7f7a60101870 2024-04-26T21:55:09.579 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:09.577+0000 7f7a64a00640 1 -- 172.21.15.72:0/159703843 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7a48007db0 con 0x7f7a60101870 2024-04-26T21:55:09.579 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:09.577+0000 7f7a5fe00640 1 -- 172.21.15.72:0/159703843 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a601a2a50 con 0x7f7a60101870 2024-04-26T21:55:09.579 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:09.577+0000 7f7a64a00640 1 -- 172.21.15.72:0/159703843 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a4800f7d0 con 0x7f7a60101870 2024-04-26T21:55:09.579 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:09.577+0000 7f7a5fe00640 1 -- 172.21.15.72:0/159703843 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a601a2fa0 con 0x7f7a60101870 2024-04-26T21:55:09.579 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:09.578+0000 7f7a5fe00640 1 -- 172.21.15.72:0/159703843 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a6010c620 con 0x7f7a60101870 2024-04-26T21:55:09.580 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:09.580+0000 7f7a64a00640 1 -- 172.21.15.72:0/159703843 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f7a48018480 con 0x7f7a60101870 2024-04-26T21:55:09.582 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:09.581+0000 7f7a64a00640 1 --2- 172.21.15.72:0/159703843 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f7a440a3c40 0x7f7a440a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:09.582 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:09.582+0000 7f7a64a00640 1 -- 172.21.15.72:0/159703843 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7f7a480d09f0 con 0x7f7a60101870 2024-04-26T21:55:09.582 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:09.582+0000 7f7a5ea00640 1 --2- 172.21.15.72:0/159703843 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f7a440a3c40 0x7f7a440a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:09.582 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:09.582+0000 7f7a5ea00640 1 --2- 172.21.15.72:0/159703843 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f7a440a3c40 0x7f7a440a6100 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f7a50006010 tx=0x7f7a50031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:55:09.586 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:09.586+0000 7f7a64a00640 1 -- 172.21.15.72:0/159703843 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f7a4809d280 con 0x7f7a60101870 2024-04-26T21:55:09.787 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:09.787+0000 7f7a5fe00640 1 -- 172.21.15.72:0/159703843 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f7a6002dd20 con 0x7f7a60101870 2024-04-26T21:55:09.788 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:09.788+0000 7f7a64a00640 1 -- 172.21.15.72:0/159703843 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) ==== 93+0+10 (secure 0 0 0) 0x7f7a4809cb80 con 0x7f7a60101870 2024-04-26T21:55:09.788 INFO:teuthology.orchestra.run.smithi072.stdout:pg_num: 1 2024-04-26T21:55:09.790 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:09.790+0000 7f7a5fe00640 1 -- 172.21.15.72:0/159703843 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f7a440a3c40 msgr2=0x7f7a440a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:09.791 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:09.790+0000 7f7a5fe00640 1 --2- 172.21.15.72:0/159703843 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f7a440a3c40 0x7f7a440a6100 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f7a50006010 tx=0x7f7a50031400 comp rx=0 tx=0).stop 2024-04-26T21:55:09.791 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:09.790+0000 7f7a5fe00640 1 -- 172.21.15.72:0/159703843 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7a60101870 msgr2=0x7f7a6019b080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:09.791 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:09.790+0000 7f7a5fe00640 1 --2- 172.21.15.72:0/159703843 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7a60101870 0x7f7a6019b080 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f7a4800c970 tx=0x7f7a4800ce30 comp rx=0 tx=0).stop 2024-04-26T21:55:09.791 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:09.790+0000 7f7a5fe00640 1 -- 172.21.15.72:0/159703843 shutdown_connections 2024-04-26T21:55:09.791 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:09.790+0000 7f7a5fe00640 1 --2- 172.21.15.72:0/159703843 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f7a440a3c40 0x7f7a440a6100 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:09.791 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:09.790+0000 7f7a5fe00640 1 --2- 172.21.15.72:0/159703843 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7a60102230 0x7f7a601a20f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:09.791 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:09.790+0000 7f7a5fe00640 1 --2- 172.21.15.72:0/159703843 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7a60101870 0x7f7a6019b080 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:09.791 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:09.790+0000 7f7a5fe00640 1 --2- 172.21.15.72:0/159703843 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7a60100b10 0x7f7a6019ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:09.791 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:09.790+0000 7f7a5fe00640 1 -- 172.21.15.72:0/159703843 >> 172.21.15.72:0/159703843 conn(0x7f7a600fc460 msgr2=0x7f7a600fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:55:09.791 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:09.791+0000 7f7a5fe00640 1 -- 172.21.15.72:0/159703843 shutdown_connections 2024-04-26T21:55:09.791 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:09.791+0000 7f7a5fe00640 1 -- 172.21.15.72:0/159703843 wait complete. 2024-04-26T21:55:09.810 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T21:55:10.015 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:10.015+0000 7f6c5fe00640 1 -- 172.21.15.72:0/3008722748 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6c60104a70 msgr2=0x7f6c60104e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:10.015 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:10.015+0000 7f6c5fe00640 1 --2- 172.21.15.72:0/3008722748 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6c60104a70 0x7f6c60104e90 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f6c50009ca0 tx=0x7f6c5002a8c0 comp rx=0 tx=0).stop 2024-04-26T21:55:10.016 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:10.015+0000 7f6c5fe00640 1 -- 172.21.15.72:0/3008722748 shutdown_connections 2024-04-26T21:55:10.016 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:10.015+0000 7f6c5fe00640 1 --2- 172.21.15.72:0/3008722748 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6c60106620 0x7f6c6010ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:10.016 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:10.015+0000 7f6c5fe00640 1 --2- 172.21.15.72:0/3008722748 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6c60105c60 0x7f6c601060e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:10.016 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:10.015+0000 7f6c5fe00640 1 --2- 172.21.15.72:0/3008722748 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6c60104a70 0x7f6c60104e90 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:10.016 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:10.015+0000 7f6c5fe00640 1 -- 172.21.15.72:0/3008722748 >> 172.21.15.72:0/3008722748 conn(0x7f6c600fffe0 msgr2=0x7f6c60102420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:55:10.016 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:10.015+0000 7f6c5fe00640 1 -- 172.21.15.72:0/3008722748 shutdown_connections 2024-04-26T21:55:10.016 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:10.015+0000 7f6c5fe00640 1 -- 172.21.15.72:0/3008722748 wait complete. 2024-04-26T21:55:10.018 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:10.016+0000 7f6c5fe00640 1 Processor -- start 2024-04-26T21:55:10.018 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:10.016+0000 7f6c5fe00640 1 -- start start 2024-04-26T21:55:10.018 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:10.016+0000 7f6c5fe00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6c60104a70 0x7f6c6019abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:10.018 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:10.016+0000 7f6c5fe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6c60105c60 0x7f6c6019b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:10.018 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:10.017+0000 7f6c5fe00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6c60106620 0x7f6c601a2160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:10.018 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:10.017+0000 7f6c5fe00640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f6c601104a0 con 0x7f6c60106620 2024-04-26T21:55:10.018 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:10.017+0000 7f6c5fe00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f6c60110320 con 0x7f6c60104a70 2024-04-26T21:55:10.018 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:10.017+0000 7f6c5fe00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f6c60110620 con 0x7f6c60105c60 2024-04-26T21:55:10.018 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:10.017+0000 7f6c5f400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6c60106620 0x7f6c601a2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:10.019 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:10.017+0000 7f6c5f400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6c60106620 0x7f6c601a2160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:46338/0 (socket says 172.21.15.72:46338) 2024-04-26T21:55:10.019 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:10.017+0000 7f6c5f400640 1 -- 172.21.15.72:0/2431743625 learned_addr learned my addr 172.21.15.72:0/2431743625 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:55:10.019 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:10.017+0000 7f6c5ea00640 1 --2- 172.21.15.72:0/2431743625 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6c60104a70 0x7f6c6019abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:10.019 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:10.017+0000 7f6c5f400640 1 -- 172.21.15.72:0/2431743625 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6c60105c60 msgr2=0x7f6c6019b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:10.019 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:10.017+0000 7f6c5f400640 1 --2- 172.21.15.72:0/2431743625 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6c60105c60 0x7f6c6019b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:10.019 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:10.017+0000 7f6c5f400640 1 -- 172.21.15.72:0/2431743625 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6c60104a70 msgr2=0x7f6c6019abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:10.019 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:10.017+0000 7f6c5f400640 1 --2- 172.21.15.72:0/2431743625 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6c60104a70 0x7f6c6019abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:10.019 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:10.017+0000 7f6c5f400640 1 -- 172.21.15.72:0/2431743625 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c601a27d0 con 0x7f6c60106620 2024-04-26T21:55:10.019 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:10.018+0000 7f6c5f400640 1 --2- 172.21.15.72:0/2431743625 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6c60106620 0x7f6c601a2160 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f6c5800d750 tx=0x7f6c5800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:55:10.019 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:10.018+0000 7f6c64a00640 1 -- 172.21.15.72:0/2431743625 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c58017070 con 0x7f6c60106620 2024-04-26T21:55:10.019 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:10.018+0000 7f6c64a00640 1 -- 172.21.15.72:0/2431743625 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6c580054b0 con 0x7f6c60106620 2024-04-26T21:55:10.019 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:10.018+0000 7f6c5fe00640 1 -- 172.21.15.72:0/2431743625 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c601a2ac0 con 0x7f6c60106620 2024-04-26T21:55:10.019 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:10.018+0000 7f6c64a00640 1 -- 172.21.15.72:0/2431743625 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c58004b90 con 0x7f6c60106620 2024-04-26T21:55:10.019 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:10.018+0000 7f6c5fe00640 1 -- 172.21.15.72:0/2431743625 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c601a3010 con 0x7f6c60106620 2024-04-26T21:55:10.020 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:10.019+0000 7f6c5fe00640 1 -- 172.21.15.72:0/2431743625 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c6010be80 con 0x7f6c60106620 2024-04-26T21:55:10.021 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:10.021+0000 7f6c64a00640 1 -- 172.21.15.72:0/2431743625 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f6c58004d30 con 0x7f6c60106620 2024-04-26T21:55:10.022 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:10.022+0000 7f6c64a00640 1 --2- 172.21.15.72:0/2431743625 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f6c440a3c40 0x7f6c440a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:10.022 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:10.022+0000 7f6c64a00640 1 -- 172.21.15.72:0/2431743625 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7f6c580d00a0 con 0x7f6c60106620 2024-04-26T21:55:10.022 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:10.023+0000 7f6c5ea00640 1 --2- 172.21.15.72:0/2431743625 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f6c440a3c40 0x7f6c440a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:10.023 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:10.023+0000 7f6c5ea00640 1 --2- 172.21.15.72:0/2431743625 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f6c440a3c40 0x7f6c440a6100 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f6c50000f80 tx=0x7f6c500319a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:55:10.026 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:10.026+0000 7f6c64a00640 1 -- 172.21.15.72:0/2431743625 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f6c5809c840 con 0x7f6c60106620 2024-04-26T21:55:10.071 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:55:10 smithi072 ceph-mon[28138]: from='client.? 172.21.15.72:0/3889719323' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:55:10.071 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:55:10 smithi072 ceph-mon[28138]: from='client.? 172.21.15.72:0/159703843' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T21:55:10.229 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:10.229+0000 7f6c5fe00640 1 -- 172.21.15.72:0/2431743625 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f6c6002dcd0 con 0x7f6c60106620 2024-04-26T21:55:10.229 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:10.230+0000 7f6c64a00640 1 -- 172.21.15.72:0/2431743625 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v91) ==== 104+0+11 (secure 0 0 0) 0x7f6c5809c140 con 0x7f6c60106620 2024-04-26T21:55:10.230 INFO:teuthology.orchestra.run.smithi072.stdout:pg_num: 32 2024-04-26T21:55:10.233 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:10.232+0000 7f6c5fe00640 1 -- 172.21.15.72:0/2431743625 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f6c440a3c40 msgr2=0x7f6c440a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:10.233 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:10.232+0000 7f6c5fe00640 1 --2- 172.21.15.72:0/2431743625 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f6c440a3c40 0x7f6c440a6100 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f6c50000f80 tx=0x7f6c500319a0 comp rx=0 tx=0).stop 2024-04-26T21:55:10.233 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:10.232+0000 7f6c5fe00640 1 -- 172.21.15.72:0/2431743625 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6c60106620 msgr2=0x7f6c601a2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:10.233 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:10.232+0000 7f6c5fe00640 1 --2- 172.21.15.72:0/2431743625 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6c60106620 0x7f6c601a2160 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f6c5800d750 tx=0x7f6c5800dc10 comp rx=0 tx=0).stop 2024-04-26T21:55:10.233 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:10.233+0000 7f6c5fe00640 1 -- 172.21.15.72:0/2431743625 shutdown_connections 2024-04-26T21:55:10.233 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:10.233+0000 7f6c5fe00640 1 --2- 172.21.15.72:0/2431743625 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6c60106620 0x7f6c601a2160 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:10.233 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:10.233+0000 7f6c5fe00640 1 --2- 172.21.15.72:0/2431743625 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f6c440a3c40 0x7f6c440a6100 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:10.233 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:10.233+0000 7f6c5fe00640 1 --2- 172.21.15.72:0/2431743625 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6c60105c60 0x7f6c6019b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:10.233 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:10.233+0000 7f6c5fe00640 1 --2- 172.21.15.72:0/2431743625 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6c60104a70 0x7f6c6019abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:10.233 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:10.233+0000 7f6c5fe00640 1 -- 172.21.15.72:0/2431743625 >> 172.21.15.72:0/2431743625 conn(0x7f6c600fffe0 msgr2=0x7f6c60109050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:55:10.233 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:10.233+0000 7f6c5fe00640 1 -- 172.21.15.72:0/2431743625 shutdown_connections 2024-04-26T21:55:10.233 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:10.233+0000 7f6c5fe00640 1 -- 172.21.15.72:0/2431743625 wait complete. 2024-04-26T21:55:10.252 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T21:55:10.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:55:10 smithi076 ceph-mon[31823]: from='client.? 172.21.15.72:0/3889719323' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:55:10.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:55:10 smithi076 ceph-mon[31823]: from='client.? 172.21.15.72:0/159703843' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T21:55:10.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:55:10 smithi164 ceph-mon[31916]: from='client.? 172.21.15.72:0/3889719323' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:55:10.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:55:10 smithi164 ceph-mon[31916]: from='client.? 172.21.15.72:0/159703843' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T21:55:10.456 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:10.455+0000 7f410d800640 1 -- 172.21.15.72:0/3553373556 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4108106540 msgr2=0x7f410810cdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:10.456 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:10.455+0000 7f410d800640 1 --2- 172.21.15.72:0/3553373556 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4108106540 0x7f410810cdd0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f4100009ca0 tx=0x7f410002a8c0 comp rx=0 tx=0).stop 2024-04-26T21:55:10.456 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:10.455+0000 7f410d800640 1 -- 172.21.15.72:0/3553373556 shutdown_connections 2024-04-26T21:55:10.456 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:10.455+0000 7f410d800640 1 --2- 172.21.15.72:0/3553373556 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4108106540 0x7f410810cdd0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:10.456 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:10.455+0000 7f410d800640 1 --2- 172.21.15.72:0/3553373556 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4108105b80 0x7f4108106000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:10.456 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:10.455+0000 7f410d800640 1 --2- 172.21.15.72:0/3553373556 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4108104990 0x7f4108104db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:10.456 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:10.455+0000 7f410d800640 1 -- 172.21.15.72:0/3553373556 >> 172.21.15.72:0/3553373556 conn(0x7f41080fff00 msgr2=0x7f4108102340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:55:10.456 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:10.455+0000 7f410d800640 1 -- 172.21.15.72:0/3553373556 shutdown_connections 2024-04-26T21:55:10.456 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:10.455+0000 7f410d800640 1 -- 172.21.15.72:0/3553373556 wait complete. 2024-04-26T21:55:10.458 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:10.456+0000 7f410d800640 1 Processor -- start 2024-04-26T21:55:10.458 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:10.456+0000 7f410d800640 1 -- start start 2024-04-26T21:55:10.458 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:10.457+0000 7f410d800640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4108104990 0x7f410819a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:10.458 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:10.457+0000 7f410d800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4108105b80 0x7f410819af10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:10.459 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:10.457+0000 7f410d800640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4108106540 0x7f41081a1f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:10.459 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:10.457+0000 7f410d800640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f41081103c0 con 0x7f4108104990 2024-04-26T21:55:10.459 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:10.457+0000 7f410d800640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f4108110240 con 0x7f4108106540 2024-04-26T21:55:10.459 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:10.457+0000 7f410d800640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f4108110540 con 0x7f4108105b80 2024-04-26T21:55:10.459 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:10.457+0000 7f4107e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4108104990 0x7f410819a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:10.459 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:10.457+0000 7f4107e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4108104990 0x7f410819a9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:46366/0 (socket says 172.21.15.72:46366) 2024-04-26T21:55:10.459 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:10.457+0000 7f4107e00640 1 -- 172.21.15.72:0/2993323731 learned_addr learned my addr 172.21.15.72:0/2993323731 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:55:10.459 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:10.457+0000 7f410ce00640 1 --2- 172.21.15.72:0/2993323731 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4108106540 0x7f41081a1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:10.459 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:10.457+0000 7f4107e00640 1 -- 172.21.15.72:0/2993323731 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4108105b80 msgr2=0x7f410819af10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:10.459 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:10.457+0000 7f4107e00640 1 --2- 172.21.15.72:0/2993323731 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4108105b80 0x7f410819af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:10.459 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:10.457+0000 7f4107e00640 1 -- 172.21.15.72:0/2993323731 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4108106540 msgr2=0x7f41081a1f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:10.459 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:10.457+0000 7f4107e00640 1 --2- 172.21.15.72:0/2993323731 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4108106540 0x7f41081a1f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:10.459 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:10.457+0000 7f4107e00640 1 -- 172.21.15.72:0/2993323731 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41081a25f0 con 0x7f4108104990 2024-04-26T21:55:10.459 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:10.458+0000 7f4107e00640 1 --2- 172.21.15.72:0/2993323731 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4108104990 0x7f410819a9d0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f40f800c970 tx=0x7f40f800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:55:10.459 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:10.458+0000 7f4105600640 1 -- 172.21.15.72:0/2993323731 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40f8007c10 con 0x7f4108104990 2024-04-26T21:55:10.460 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:10.458+0000 7f4105600640 1 -- 172.21.15.72:0/2993323731 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f40f8007db0 con 0x7f4108104990 2024-04-26T21:55:10.460 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:10.458+0000 7f4105600640 1 -- 172.21.15.72:0/2993323731 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40f800f7d0 con 0x7f4108104990 2024-04-26T21:55:10.460 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:10.458+0000 7f410d800640 1 -- 172.21.15.72:0/2993323731 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41081a28e0 con 0x7f4108104990 2024-04-26T21:55:10.460 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:10.458+0000 7f410d800640 1 -- 172.21.15.72:0/2993323731 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41081a2e30 con 0x7f4108104990 2024-04-26T21:55:10.460 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:10.459+0000 7f410d800640 1 -- 172.21.15.72:0/2993323731 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f410810bcb0 con 0x7f4108104990 2024-04-26T21:55:10.461 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:10.461+0000 7f4105600640 1 -- 172.21.15.72:0/2993323731 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f40f8018480 con 0x7f4108104990 2024-04-26T21:55:10.463 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:10.463+0000 7f4105600640 1 --2- 172.21.15.72:0/2993323731 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f40ec0a3c40 0x7f40ec0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:10.463 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:10.463+0000 7f4105600640 1 -- 172.21.15.72:0/2993323731 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7f40f80d09f0 con 0x7f4108104990 2024-04-26T21:55:10.463 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:10.463+0000 7f40ffe00640 1 --2- 172.21.15.72:0/2993323731 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f40ec0a3c40 0x7f40ec0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:10.463 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:10.463+0000 7f40ffe00640 1 --2- 172.21.15.72:0/2993323731 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f40ec0a3c40 0x7f40ec0a6100 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f40f0005fd0 tx=0x7f40f0005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:55:10.467 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:10.467+0000 7f4105600640 1 -- 172.21.15.72:0/2993323731 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f40f809d280 con 0x7f4108104990 2024-04-26T21:55:10.668 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:10.668+0000 7f410d800640 1 -- 172.21.15.72:0/2993323731 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f410802dd40 con 0x7f4108104990 2024-04-26T21:55:10.669 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:10.668+0000 7f4105600640 1 -- 172.21.15.72:0/2993323731 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v91) ==== 100+0+11 (secure 0 0 0) 0x7f40f809cb80 con 0x7f4108104990 2024-04-26T21:55:10.669 INFO:teuthology.orchestra.run.smithi072.stdout:pg_num: 64 2024-04-26T21:55:10.671 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:10.671+0000 7f410d800640 1 -- 172.21.15.72:0/2993323731 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f40ec0a3c40 msgr2=0x7f40ec0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:10.671 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:10.671+0000 7f410d800640 1 --2- 172.21.15.72:0/2993323731 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f40ec0a3c40 0x7f40ec0a6100 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f40f0005fd0 tx=0x7f40f0005850 comp rx=0 tx=0).stop 2024-04-26T21:55:10.671 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:10.671+0000 7f410d800640 1 -- 172.21.15.72:0/2993323731 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4108104990 msgr2=0x7f410819a9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:10.671 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:10.671+0000 7f410d800640 1 --2- 172.21.15.72:0/2993323731 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4108104990 0x7f410819a9d0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f40f800c970 tx=0x7f40f800ce30 comp rx=0 tx=0).stop 2024-04-26T21:55:10.671 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:10.671+0000 7f410d800640 1 -- 172.21.15.72:0/2993323731 shutdown_connections 2024-04-26T21:55:10.672 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:10.671+0000 7f410d800640 1 --2- 172.21.15.72:0/2993323731 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4108106540 0x7f41081a1f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:10.672 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:10.671+0000 7f410d800640 1 --2- 172.21.15.72:0/2993323731 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f40ec0a3c40 0x7f40ec0a6100 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:10.672 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:10.671+0000 7f410d800640 1 --2- 172.21.15.72:0/2993323731 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4108105b80 0x7f410819af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:10.672 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:10.671+0000 7f410d800640 1 --2- 172.21.15.72:0/2993323731 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4108104990 0x7f410819a9d0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:10.672 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:10.671+0000 7f410d800640 1 -- 172.21.15.72:0/2993323731 >> 172.21.15.72:0/2993323731 conn(0x7f41080fff00 msgr2=0x7f4108108e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:55:10.672 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:10.671+0000 7f410d800640 1 -- 172.21.15.72:0/2993323731 shutdown_connections 2024-04-26T21:55:10.672 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:10.671+0000 7f410d800640 1 -- 172.21.15.72:0/2993323731 wait complete. 2024-04-26T21:55:10.691 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:55:10.895 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:10.894+0000 7f7f22e00640 1 -- 172.21.15.72:0/2629011404 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7f1c105b80 msgr2=0x7f7f1c106000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:10.895 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:10.894+0000 7f7f22e00640 1 --2- 172.21.15.72:0/2629011404 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7f1c105b80 0x7f7f1c106000 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f7f08009900 tx=0x7f7f0802a3f0 comp rx=0 tx=0).stop 2024-04-26T21:55:10.895 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:10.894+0000 7f7f22e00640 1 -- 172.21.15.72:0/2629011404 shutdown_connections 2024-04-26T21:55:10.895 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:10.894+0000 7f7f22e00640 1 --2- 172.21.15.72:0/2629011404 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7f1c106540 0x7f7f1c10cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:10.895 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:10.894+0000 7f7f22e00640 1 --2- 172.21.15.72:0/2629011404 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7f1c105b80 0x7f7f1c106000 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:10.895 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:10.894+0000 7f7f22e00640 1 --2- 172.21.15.72:0/2629011404 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7f1c104990 0x7f7f1c104db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:10.895 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:10.894+0000 7f7f22e00640 1 -- 172.21.15.72:0/2629011404 >> 172.21.15.72:0/2629011404 conn(0x7f7f1c0fff00 msgr2=0x7f7f1c102340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:55:10.895 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:10.894+0000 7f7f22e00640 1 -- 172.21.15.72:0/2629011404 shutdown_connections 2024-04-26T21:55:10.895 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:10.895+0000 7f7f22e00640 1 -- 172.21.15.72:0/2629011404 wait complete. 2024-04-26T21:55:10.897 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:10.895+0000 7f7f22e00640 1 Processor -- start 2024-04-26T21:55:10.898 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:10.896+0000 7f7f22e00640 1 -- start start 2024-04-26T21:55:10.898 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:10.896+0000 7f7f22e00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7f1c104990 0x7f7f1c19aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:10.898 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:10.896+0000 7f7f22e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7f1c105b80 0x7f7f1c19af90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:10.898 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:10.896+0000 7f7f22e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7f1c106540 0x7f7f1c1a2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:10.898 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:10.896+0000 7f7f22e00640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f7f1c1103a0 con 0x7f7f1c106540 2024-04-26T21:55:10.898 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:10.896+0000 7f7f22e00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f7f1c110220 con 0x7f7f1c104990 2024-04-26T21:55:10.898 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:10.896+0000 7f7f22e00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f7f1c110520 con 0x7f7f1c105b80 2024-04-26T21:55:10.898 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:10.896+0000 7f7f22400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7f1c106540 0x7f7f1c1a2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:10.898 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:10.896+0000 7f7f22400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7f1c106540 0x7f7f1c1a2000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:34074/0 (socket says 172.21.15.72:34074) 2024-04-26T21:55:10.898 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:10.896+0000 7f7f22400640 1 -- 172.21.15.72:0/3957081723 learned_addr learned my addr 172.21.15.72:0/3957081723 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:55:10.898 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:10.897+0000 7f7f21a00640 1 --2- 172.21.15.72:0/3957081723 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7f1c104990 0x7f7f1c19aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:10.898 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:10.897+0000 7f7f22400640 1 -- 172.21.15.72:0/3957081723 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7f1c105b80 msgr2=0x7f7f1c19af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:10.898 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:10.897+0000 7f7f22400640 1 --2- 172.21.15.72:0/3957081723 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7f1c105b80 0x7f7f1c19af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:10.898 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:10.897+0000 7f7f22400640 1 -- 172.21.15.72:0/3957081723 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7f1c104990 msgr2=0x7f7f1c19aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:10.899 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:10.897+0000 7f7f22400640 1 --2- 172.21.15.72:0/3957081723 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7f1c104990 0x7f7f1c19aa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:10.899 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:10.897+0000 7f7f22400640 1 -- 172.21.15.72:0/3957081723 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f1c1a2670 con 0x7f7f1c106540 2024-04-26T21:55:10.899 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:10.897+0000 7f7f22400640 1 --2- 172.21.15.72:0/3957081723 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7f1c106540 0x7f7f1c1a2000 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f7f1400cce0 tx=0x7f7f1400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:55:10.899 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:10.897+0000 7f7f1a000640 1 -- 172.21.15.72:0/3957081723 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f1400f450 con 0x7f7f1c106540 2024-04-26T21:55:10.899 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:10.897+0000 7f7f1a000640 1 -- 172.21.15.72:0/3957081723 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7f1400fa80 con 0x7f7f1c106540 2024-04-26T21:55:10.899 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:10.897+0000 7f7f22e00640 1 -- 172.21.15.72:0/3957081723 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f1c1a2960 con 0x7f7f1c106540 2024-04-26T21:55:10.899 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:10.897+0000 7f7f1a000640 1 -- 172.21.15.72:0/3957081723 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f14018ad0 con 0x7f7f1c106540 2024-04-26T21:55:10.899 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:10.897+0000 7f7f22e00640 1 -- 172.21.15.72:0/3957081723 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f1c1a2e60 con 0x7f7f1c106540 2024-04-26T21:55:10.899 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:10.898+0000 7f7f22e00640 1 -- 172.21.15.72:0/3957081723 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f1c10bdd0 con 0x7f7f1c106540 2024-04-26T21:55:10.900 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:10.900+0000 7f7f1a000640 1 -- 172.21.15.72:0/3957081723 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f7f1400f5f0 con 0x7f7f1c106540 2024-04-26T21:55:10.902 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:10.902+0000 7f7f1a000640 1 --2- 172.21.15.72:0/3957081723 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f7f040a3c40 0x7f7f040a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:10.902 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:10.902+0000 7f7f1a000640 1 -- 172.21.15.72:0/3957081723 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7f7f140d0ee0 con 0x7f7f1c106540 2024-04-26T21:55:10.902 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:10.902+0000 7f7f21a00640 1 --2- 172.21.15.72:0/3957081723 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f7f040a3c40 0x7f7f040a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:10.903 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:10.903+0000 7f7f21a00640 1 --2- 172.21.15.72:0/3957081723 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f7f040a3c40 0x7f7f040a6100 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f7f10005fd0 tx=0x7f7f10005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:55:10.906 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:10.906+0000 7f7f1a000640 1 -- 172.21.15.72:0/3957081723 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f7f14014030 con 0x7f7f1c106540 2024-04-26T21:55:11.152 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.152+0000 7f7f22e00640 1 -- 172.21.15.72:0/3957081723 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7f1c02dd20 con 0x7f7f1c106540 2024-04-26T21:55:11.153 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.153+0000 7f7f1a000640 1 -- 172.21.15.72:0/3957081723 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11931 (secure 0 0 0) 0x7f7f1409d680 con 0x7f7f1c106540 2024-04-26T21:55:11.153 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T21:55:11.154 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6835/1320381959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6834","nonce":1320381959},{"type":"v1","addr":"172.21.15.72:6835","nonce":1320381959}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6837/2142200512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6836","nonce":2142200512},{"type":"v1","addr":"172.21.15.72:6837","nonce":2142200512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T21:52:50.567857+0000","modified":"2024-04-26T21:52:58.508126+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34517,"mds_1":34472,"mds_2":34442,"mds_3":34406,"mds_4":24463},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14652":{"gid":14652,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.72:6839/1312898281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6838","nonce":1312898281},{"type":"v1","addr":"172.21.15.72:6839","nonce":1312898281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14664":{"gid":14664,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.72:6841/3257692339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6840","nonce":3257692339},{"type":"v1","addr":"172.21.15.72:6841","nonce":3257692339}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24388":{"gid":24388,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6833/4236668465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":4236668465},{"type":"v1","addr":"172.21.15.164:6833","nonce":4236668465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6835/2391114169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":2391114169},{"type":"v1","addr":"172.21.15.164:6835","nonce":2391114169}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6837/3585829992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":3585829992},{"type":"v1","addr":"172.21.15.164:6837","nonce":3585829992}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.164:6839/1636226820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":1636226820},{"type":"v1","addr":"172.21.15.164:6839","nonce":1636226820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34406":{"gid":34406,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":15,"addr":"172.21.15.76:6833/3869483750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":3869483750},{"type":"v1","addr":"172.21.15.76:6833","nonce":3869483750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34442":{"gid":34442,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.76:6835/3517429435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":3517429435},{"type":"v1","addr":"172.21.15.76:6835","nonce":3517429435}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34472":{"gid":34472,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.76:6837/1150678991","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":1150678991},{"type":"v1","addr":"172.21.15.76:6837","nonce":1150678991}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34517":{"gid":34517,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.76:6839/327145636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6838","nonce":327145636},{"type":"v1","addr":"172.21.15.76:6839","nonce":327145636}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34517,"qdb_cluster":[34406,24463,34442,34517,34472]},"id":1}]} 2024-04-26T21:55:11.154 INFO:teuthology.orchestra.run.smithi072.stderr:dumped fsmap epoch 26 2024-04-26T21:55:11.156 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.156+0000 7f7f22e00640 1 -- 172.21.15.72:0/3957081723 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f7f040a3c40 msgr2=0x7f7f040a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:11.156 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.156+0000 7f7f22e00640 1 --2- 172.21.15.72:0/3957081723 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f7f040a3c40 0x7f7f040a6100 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f7f10005fd0 tx=0x7f7f10005850 comp rx=0 tx=0).stop 2024-04-26T21:55:11.157 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.156+0000 7f7f22e00640 1 -- 172.21.15.72:0/3957081723 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7f1c106540 msgr2=0x7f7f1c1a2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:11.157 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.156+0000 7f7f22e00640 1 --2- 172.21.15.72:0/3957081723 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7f1c106540 0x7f7f1c1a2000 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f7f1400cce0 tx=0x7f7f1400a620 comp rx=0 tx=0).stop 2024-04-26T21:55:11.157 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.156+0000 7f7f22e00640 1 -- 172.21.15.72:0/3957081723 shutdown_connections 2024-04-26T21:55:11.157 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.156+0000 7f7f22e00640 1 --2- 172.21.15.72:0/3957081723 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7f1c106540 0x7f7f1c1a2000 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:11.157 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.156+0000 7f7f22e00640 1 --2- 172.21.15.72:0/3957081723 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f7f040a3c40 0x7f7f040a6100 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:11.157 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.156+0000 7f7f22e00640 1 --2- 172.21.15.72:0/3957081723 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7f1c105b80 0x7f7f1c19af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:11.157 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.156+0000 7f7f22e00640 1 --2- 172.21.15.72:0/3957081723 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7f1c104990 0x7f7f1c19aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:11.157 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.156+0000 7f7f22e00640 1 -- 172.21.15.72:0/3957081723 >> 172.21.15.72:0/3957081723 conn(0x7f7f1c0fff00 msgr2=0x7f7f1c108fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:55:11.157 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.156+0000 7f7f22e00640 1 -- 172.21.15.72:0/3957081723 shutdown_connections 2024-04-26T21:55:11.157 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.156+0000 7f7f22e00640 1 -- 172.21.15.72:0/3957081723 wait complete. 2024-04-26T21:55:11.175 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:55:11.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:55:11 smithi076 ceph-mon[31823]: pgmap v441: 97 pgs: 97 active+clean; 633 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-26T21:55:11.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:55:11 smithi076 ceph-mon[31823]: from='client.? 172.21.15.72:0/2431743625' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T21:55:11.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:55:11 smithi076 ceph-mon[31823]: from='client.? 172.21.15.72:0/2993323731' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T21:55:11.321 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:55:11 smithi072 ceph-mon[28138]: pgmap v441: 97 pgs: 97 active+clean; 633 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-26T21:55:11.321 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:55:11 smithi072 ceph-mon[28138]: from='client.? 172.21.15.72:0/2431743625' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T21:55:11.321 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:55:11 smithi072 ceph-mon[28138]: from='client.? 172.21.15.72:0/2993323731' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T21:55:11.376 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.375+0000 7f6ee7200640 1 -- 172.21.15.72:0/1609391394 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6ee01042f0 msgr2=0x7f6ee010ab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:11.376 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.375+0000 7f6ee7200640 1 --2- 172.21.15.72:0/1609391394 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6ee01042f0 0x7f6ee010ab80 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f6edc009ca0 tx=0x7f6edc02a8c0 comp rx=0 tx=0).stop 2024-04-26T21:55:11.376 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.375+0000 7f6ee7200640 1 -- 172.21.15.72:0/1609391394 shutdown_connections 2024-04-26T21:55:11.376 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.375+0000 7f6ee7200640 1 --2- 172.21.15.72:0/1609391394 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6ee01042f0 0x7f6ee010ab80 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:11.376 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.375+0000 7f6ee7200640 1 --2- 172.21.15.72:0/1609391394 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6ee0103930 0x7f6ee0103db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:11.376 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.375+0000 7f6ee7200640 1 --2- 172.21.15.72:0/1609391394 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6ee0102740 0x7f6ee0102b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:11.376 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.375+0000 7f6ee7200640 1 -- 172.21.15.72:0/1609391394 >> 172.21.15.72:0/1609391394 conn(0x7f6ee00fdcb0 msgr2=0x7f6ee01000f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:55:11.376 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.376+0000 7f6ee7200640 1 -- 172.21.15.72:0/1609391394 shutdown_connections 2024-04-26T21:55:11.376 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.376+0000 7f6ee7200640 1 -- 172.21.15.72:0/1609391394 wait complete. 2024-04-26T21:55:11.379 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.377+0000 7f6ee7200640 1 Processor -- start 2024-04-26T21:55:11.379 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.377+0000 7f6ee7200640 1 -- start start 2024-04-26T21:55:11.379 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.377+0000 7f6ee7200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6ee0102740 0x7f6ee0198880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:11.379 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.377+0000 7f6ee7200640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6ee0103930 0x7f6ee0198dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:11.379 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.377+0000 7f6ee7200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6ee01042f0 0x7f6ee019fe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:11.379 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.377+0000 7f6ee7200640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f6ee010e170 con 0x7f6ee0103930 2024-04-26T21:55:11.379 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.377+0000 7f6ee7200640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f6ee010dff0 con 0x7f6ee01042f0 2024-04-26T21:55:11.379 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.377+0000 7f6ee7200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f6ee010e2f0 con 0x7f6ee0102740 2024-04-26T21:55:11.379 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.377+0000 7f6ee5400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6ee0103930 0x7f6ee0198dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:11.380 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.377+0000 7f6ee5400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6ee0103930 0x7f6ee0198dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:34084/0 (socket says 172.21.15.72:34084) 2024-04-26T21:55:11.380 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.377+0000 7f6ee5400640 1 -- 172.21.15.72:0/3368602131 learned_addr learned my addr 172.21.15.72:0/3368602131 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:55:11.380 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.378+0000 7f6ee5e00640 1 --2- 172.21.15.72:0/3368602131 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6ee0102740 0x7f6ee0198880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:11.380 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.378+0000 7f6ee6800640 1 --2- 172.21.15.72:0/3368602131 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6ee01042f0 0x7f6ee019fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:11.380 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.378+0000 7f6ee5400640 1 -- 172.21.15.72:0/3368602131 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6ee0102740 msgr2=0x7f6ee0198880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:11.380 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.378+0000 7f6ee5400640 1 --2- 172.21.15.72:0/3368602131 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6ee0102740 0x7f6ee0198880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:11.380 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.378+0000 7f6ee5400640 1 -- 172.21.15.72:0/3368602131 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6ee01042f0 msgr2=0x7f6ee019fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:11.380 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.378+0000 7f6ee5400640 1 --2- 172.21.15.72:0/3368602131 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6ee01042f0 0x7f6ee019fe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:11.380 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.378+0000 7f6ee5400640 1 -- 172.21.15.72:0/3368602131 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ee01a04a0 con 0x7f6ee0103930 2024-04-26T21:55:11.380 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.378+0000 7f6ee6800640 1 --2- 172.21.15.72:0/3368602131 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6ee01042f0 0x7f6ee019fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:55:11.380 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.378+0000 7f6ee5400640 1 --2- 172.21.15.72:0/3368602131 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6ee0103930 0x7f6ee0198dc0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f6ed400d770 tx=0x7f6ed400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:55:11.380 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.378+0000 7f6edaa00640 1 -- 172.21.15.72:0/3368602131 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ed4017070 con 0x7f6ee0103930 2024-04-26T21:55:11.380 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.378+0000 7f6edaa00640 1 -- 172.21.15.72:0/3368602131 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6ed40054b0 con 0x7f6ee0103930 2024-04-26T21:55:11.380 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.378+0000 7f6edaa00640 1 -- 172.21.15.72:0/3368602131 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ed4004c60 con 0x7f6ee0103930 2024-04-26T21:55:11.381 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.378+0000 7f6ee7200640 1 -- 172.21.15.72:0/3368602131 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ee01a0790 con 0x7f6ee0103930 2024-04-26T21:55:11.381 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.379+0000 7f6ee7200640 1 -- 172.21.15.72:0/3368602131 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ee01a0c90 con 0x7f6ee0103930 2024-04-26T21:55:11.381 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.380+0000 7f6ee7200640 1 -- 172.21.15.72:0/3368602131 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ee0109ae0 con 0x7f6ee0103930 2024-04-26T21:55:11.382 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.382+0000 7f6edaa00640 1 -- 172.21.15.72:0/3368602131 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f6ed4004e00 con 0x7f6ee0103930 2024-04-26T21:55:11.383 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.383+0000 7f6edaa00640 1 --2- 172.21.15.72:0/3368602131 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f6ec80a3c40 0x7f6ec80a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:11.383 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.383+0000 7f6edaa00640 1 -- 172.21.15.72:0/3368602131 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7f6ed40d0180 con 0x7f6ee0103930 2024-04-26T21:55:11.383 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.383+0000 7f6ee5e00640 1 --2- 172.21.15.72:0/3368602131 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f6ec80a3c40 0x7f6ec80a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:11.384 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.384+0000 7f6ee5e00640 1 --2- 172.21.15.72:0/3368602131 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f6ec80a3c40 0x7f6ec80a6100 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f6ecc005fd0 tx=0x7f6ecc005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:55:11.387 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.387+0000 7f6edaa00640 1 -- 172.21.15.72:0/3368602131 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f6ed409c920 con 0x7f6ee0103930 2024-04-26T21:55:11.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:55:11 smithi164 ceph-mon[31916]: pgmap v441: 97 pgs: 97 active+clean; 633 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-26T21:55:11.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:55:11 smithi164 ceph-mon[31916]: from='client.? 172.21.15.72:0/2431743625' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T21:55:11.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:55:11 smithi164 ceph-mon[31916]: from='client.? 172.21.15.72:0/2993323731' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T21:55:11.587 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.587+0000 7f6ee7200640 1 -- 172.21.15.72:0/3368602131 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6ee002dd40 con 0x7f6ee0103930 2024-04-26T21:55:11.591 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.591+0000 7f6edaa00640 1 -- 172.21.15.72:0/3368602131 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+22783 (secure 0 0 0) 0x7f6ed409c220 con 0x7f6ee0103930 2024-04-26T21:55:11.592 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T21:55:11.592 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":91,"fsid":"a1780950-0415-11ef-bc93-c7b262605968","created":"2024-04-26T21:42:01.504173+0000","modified":"2024-04-26T21:54:11.892872+0000","last_up_change":"2024-04-26T21:51:43.482437+0000","last_in_change":"2024-04-26T21:51:18.468587+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:46:29.785427+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:52:48.320391+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T21:52:49.517493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"04e06884-b355-4762-9b29-9034b7201e9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6802","nonce":1416765179},{"type":"v1","addr":"172.21.15.72:6803","nonce":1416765179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6804","nonce":1416765179},{"type":"v1","addr":"172.21.15.72:6805","nonce":1416765179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6808","nonce":1416765179},{"type":"v1","addr":"172.21.15.72:6809","nonce":1416765179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6806","nonce":1416765179},{"type":"v1","addr":"172.21.15.72:6807","nonce":1416765179}]},"public_addr":"172.21.15.72:6803/1416765179","cluster_addr":"172.21.15.72:6805/1416765179","heartbeat_back_addr":"172.21.15.72:6809/1416765179","heartbeat_front_addr":"172.21.15.72:6807/1416765179","state":["exists","up"]},{"osd":1,"uuid":"d75a479a-4a2e-48e9-a01e-2b1c9617416a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":755426949},{"type":"v1","addr":"172.21.15.76:6801","nonce":755426949}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":755426949},{"type":"v1","addr":"172.21.15.76:6803","nonce":755426949}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":755426949},{"type":"v1","addr":"172.21.15.76:6807","nonce":755426949}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":755426949},{"type":"v1","addr":"172.21.15.76:6805","nonce":755426949}]},"public_addr":"172.21.15.76:6801/755426949","cluster_addr":"172.21.15.76:6803/755426949","heartbeat_back_addr":"172.21.15.76:6807/755426949","heartbeat_front_addr":"172.21.15.76:6805/755426949","state":["exists","up"]},{"osd":2,"uuid":"b6f8da58-4428-4e8c-a944-7ea5645d6d92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":852265973},{"type":"v1","addr":"172.21.15.164:6801","nonce":852265973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":852265973},{"type":"v1","addr":"172.21.15.164:6803","nonce":852265973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":852265973},{"type":"v1","addr":"172.21.15.164:6807","nonce":852265973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":852265973},{"type":"v1","addr":"172.21.15.164:6805","nonce":852265973}]},"public_addr":"172.21.15.164:6801/852265973","cluster_addr":"172.21.15.164:6803/852265973","heartbeat_back_addr":"172.21.15.164:6807/852265973","heartbeat_front_addr":"172.21.15.164:6805/852265973","state":["exists","up"]},{"osd":3,"uuid":"76d83409-649e-41e5-b4f4-43bbd5b00ea5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6810","nonce":4027763789},{"type":"v1","addr":"172.21.15.72:6811","nonce":4027763789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6812","nonce":4027763789},{"type":"v1","addr":"172.21.15.72:6813","nonce":4027763789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6816","nonce":4027763789},{"type":"v1","addr":"172.21.15.72:6817","nonce":4027763789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6814","nonce":4027763789},{"type":"v1","addr":"172.21.15.72:6815","nonce":4027763789}]},"public_addr":"172.21.15.72:6811/4027763789","cluster_addr":"172.21.15.72:6813/4027763789","heartbeat_back_addr":"172.21.15.72:6817/4027763789","heartbeat_front_addr":"172.21.15.72:6815/4027763789","state":["exists","up"]},{"osd":4,"uuid":"d2cfeea7-1661-4156-bdcd-f770a8d1a878","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":3718867827},{"type":"v1","addr":"172.21.15.76:6809","nonce":3718867827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":3718867827},{"type":"v1","addr":"172.21.15.76:6811","nonce":3718867827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":3718867827},{"type":"v1","addr":"172.21.15.76:6815","nonce":3718867827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":3718867827},{"type":"v1","addr":"172.21.15.76:6813","nonce":3718867827}]},"public_addr":"172.21.15.76:6809/3718867827","cluster_addr":"172.21.15.76:6811/3718867827","heartbeat_back_addr":"172.21.15.76:6815/3718867827","heartbeat_front_addr":"172.21.15.76:6813/3718867827","state":["exists","up"]},{"osd":5,"uuid":"f3896ad6-a040-4ea3-a431-ff9b7fc05c36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":2543954772},{"type":"v1","addr":"172.21.15.164:6809","nonce":2543954772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":2543954772},{"type":"v1","addr":"172.21.15.164:6811","nonce":2543954772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":2543954772},{"type":"v1","addr":"172.21.15.164:6815","nonce":2543954772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":2543954772},{"type":"v1","addr":"172.21.15.164:6813","nonce":2543954772}]},"public_addr":"172.21.15.164:6809/2543954772","cluster_addr":"172.21.15.164:6811/2543954772","heartbeat_back_addr":"172.21.15.164:6815/2543954772","heartbeat_front_addr":"172.21.15.164:6813/2543954772","state":["exists","up"]},{"osd":6,"uuid":"d5d56650-be7e-4ba5-ba3a-d7d473294dab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6818","nonce":2080659027},{"type":"v1","addr":"172.21.15.72:6819","nonce":2080659027}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6820","nonce":2080659027},{"type":"v1","addr":"172.21.15.72:6821","nonce":2080659027}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6824","nonce":2080659027},{"type":"v1","addr":"172.21.15.72:6825","nonce":2080659027}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6822","nonce":2080659027},{"type":"v1","addr":"172.21.15.72:6823","nonce":2080659027}]},"public_addr":"172.21.15.72:6819/2080659027","cluster_addr":"172.21.15.72:6821/2080659027","heartbeat_back_addr":"172.21.15.72:6825/2080659027","heartbeat_front_addr":"172.21.15.72:6823/2080659027","state":["exists","up"]},{"osd":7,"uuid":"18975b5a-9309-4a0f-9f37-3ce09c2e745d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":3144631323},{"type":"v1","addr":"172.21.15.76:6817","nonce":3144631323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":3144631323},{"type":"v1","addr":"172.21.15.76:6819","nonce":3144631323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":3144631323},{"type":"v1","addr":"172.21.15.76:6823","nonce":3144631323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":3144631323},{"type":"v1","addr":"172.21.15.76:6821","nonce":3144631323}]},"public_addr":"172.21.15.76:6817/3144631323","cluster_addr":"172.21.15.76:6819/3144631323","heartbeat_back_addr":"172.21.15.76:6823/3144631323","heartbeat_front_addr":"172.21.15.76:6821/3144631323","state":["exists","up"]},{"osd":8,"uuid":"779e9c1e-8e97-48d4-a854-68ca13f38e0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":627888883},{"type":"v1","addr":"172.21.15.164:6817","nonce":627888883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":627888883},{"type":"v1","addr":"172.21.15.164:6819","nonce":627888883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":627888883},{"type":"v1","addr":"172.21.15.164:6823","nonce":627888883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":627888883},{"type":"v1","addr":"172.21.15.164:6821","nonce":627888883}]},"public_addr":"172.21.15.164:6817/627888883","cluster_addr":"172.21.15.164:6819/627888883","heartbeat_back_addr":"172.21.15.164:6823/627888883","heartbeat_front_addr":"172.21.15.164:6821/627888883","state":["exists","up"]},{"osd":9,"uuid":"d0317850-7b2d-4850-b2cf-0b919505c1fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6826","nonce":1273118485},{"type":"v1","addr":"172.21.15.72:6827","nonce":1273118485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6828","nonce":1273118485},{"type":"v1","addr":"172.21.15.72:6829","nonce":1273118485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6832","nonce":1273118485},{"type":"v1","addr":"172.21.15.72:6833","nonce":1273118485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6830","nonce":1273118485},{"type":"v1","addr":"172.21.15.72:6831","nonce":1273118485}]},"public_addr":"172.21.15.72:6827/1273118485","cluster_addr":"172.21.15.72:6829/1273118485","heartbeat_back_addr":"172.21.15.72:6833/1273118485","heartbeat_front_addr":"172.21.15.72:6831/1273118485","state":["exists","up"]},{"osd":10,"uuid":"e0aa38d8-6011-4d2a-b67b-5bff6d999e47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":2808954459},{"type":"v1","addr":"172.21.15.76:6825","nonce":2808954459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":2808954459},{"type":"v1","addr":"172.21.15.76:6827","nonce":2808954459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":2808954459},{"type":"v1","addr":"172.21.15.76:6831","nonce":2808954459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":2808954459},{"type":"v1","addr":"172.21.15.76:6829","nonce":2808954459}]},"public_addr":"172.21.15.76:6825/2808954459","cluster_addr":"172.21.15.76:6827/2808954459","heartbeat_back_addr":"172.21.15.76:6831/2808954459","heartbeat_front_addr":"172.21.15.76:6829/2808954459","state":["exists","up"]},{"osd":11,"uuid":"981b3d4d-916e-4937-9916-7cc21a891e43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":2354951784},{"type":"v1","addr":"172.21.15.164:6825","nonce":2354951784}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":2354951784},{"type":"v1","addr":"172.21.15.164:6827","nonce":2354951784}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":2354951784},{"type":"v1","addr":"172.21.15.164:6831","nonce":2354951784}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":2354951784},{"type":"v1","addr":"172.21.15.164:6829","nonce":2354951784}]},"public_addr":"172.21.15.164:6825/2354951784","cluster_addr":"172.21.15.164:6827/2354951784","heartbeat_back_addr":"172.21.15.164:6831/2354951784","heartbeat_front_addr":"172.21.15.164:6829/2354951784","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:18.997924+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:52.439081+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:46:25.893288+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:47:00.112807+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:47:35.116874+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:48:09.216344+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:48:45.028853+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:49:19.905397+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:49:55.030536+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:50:30.767662+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:51:06.423267+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:51:41.631211+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.1a","mappings":[{"from":5,"to":7}]},{"pgid":"3.1b","mappings":[{"from":4,"to":6}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.28","mappings":[{"from":5,"to":6}]},{"pgid":"3.2a","mappings":[{"from":3,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.72:0/416277784":"2024-04-27T21:43:09.602791+0000","172.21.15.72:6800/4010358672":"2024-04-27T21:43:09.602791+0000","172.21.15.72:6801/1012617371":"2024-04-27T21:42:34.684837+0000","172.21.15.72:6801/4010358672":"2024-04-27T21:43:09.602791+0000","172.21.15.72:0/3555448179":"2024-04-27T21:43:09.602791+0000","172.21.15.72:6800/1012617371":"2024-04-27T21:42:34.684837+0000","172.21.15.72:0/2920378761":"2024-04-27T21:42:34.684837+0000","172.21.15.72:0/1933740041":"2024-04-27T21:43:09.602791+0000","172.21.15.72:0/3225295775":"2024-04-27T21:42:34.684837+0000","172.21.15.72:0/2326016180":"2024-04-27T21:42:34.684837+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:55:11.594 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.593+0000 7f6ee7200640 1 -- 172.21.15.72:0/3368602131 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f6ec80a3c40 msgr2=0x7f6ec80a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:11.594 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.594+0000 7f6ee7200640 1 --2- 172.21.15.72:0/3368602131 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f6ec80a3c40 0x7f6ec80a6100 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f6ecc005fd0 tx=0x7f6ecc005850 comp rx=0 tx=0).stop 2024-04-26T21:55:11.594 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.594+0000 7f6ee7200640 1 -- 172.21.15.72:0/3368602131 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6ee0103930 msgr2=0x7f6ee0198dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:11.594 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.594+0000 7f6ee7200640 1 --2- 172.21.15.72:0/3368602131 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6ee0103930 0x7f6ee0198dc0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f6ed400d770 tx=0x7f6ed400dc30 comp rx=0 tx=0).stop 2024-04-26T21:55:11.594 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.594+0000 7f6ee7200640 1 -- 172.21.15.72:0/3368602131 shutdown_connections 2024-04-26T21:55:11.594 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.594+0000 7f6ee7200640 1 --2- 172.21.15.72:0/3368602131 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6ee01042f0 0x7f6ee019fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:11.594 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.594+0000 7f6ee7200640 1 --2- 172.21.15.72:0/3368602131 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6ee0103930 0x7f6ee0198dc0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:11.594 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.594+0000 7f6ee7200640 1 --2- 172.21.15.72:0/3368602131 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f6ec80a3c40 0x7f6ec80a6100 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:11.594 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.594+0000 7f6ee7200640 1 --2- 172.21.15.72:0/3368602131 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6ee0102740 0x7f6ee0198880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:11.595 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.594+0000 7f6ee7200640 1 -- 172.21.15.72:0/3368602131 >> 172.21.15.72:0/3368602131 conn(0x7f6ee00fdcb0 msgr2=0x7f6ee0106cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:55:11.595 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.594+0000 7f6ee7200640 1 -- 172.21.15.72:0/3368602131 shutdown_connections 2024-04-26T21:55:11.595 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.594+0000 7f6ee7200640 1 -- 172.21.15.72:0/3368602131 wait complete. 2024-04-26T21:55:11.614 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 26, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-26T21:52:50.567857+0000', 'modified': '2024-04-26T21:52:58.508126+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 34517, 'mds_1': 34472, 'mds_2': 34442, 'mds_3': 34406, 'mds_4': 24463}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14652': {'gid': 14652, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.72:6839/1312898281', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.72:6838', 'nonce': 1312898281}, {'type': 'v1', 'addr': '172.21.15.72:6839', 'nonce': 1312898281}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14664': {'gid': 14664, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.72:6841/3257692339', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.72:6840', 'nonce': 3257692339}, {'type': 'v1', 'addr': '172.21.15.72:6841', 'nonce': 3257692339}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24388': {'gid': 24388, 'name': 'l', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.164:6833/4236668465', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6832', 'nonce': 4236668465}, {'type': 'v1', 'addr': '172.21.15.164:6833', 'nonce': 4236668465}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24424': {'gid': 24424, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.164:6835/2391114169', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6834', 'nonce': 2391114169}, {'type': 'v1', 'addr': '172.21.15.164:6835', 'nonce': 2391114169}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24442': {'gid': 24442, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.164:6837/3585829992', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6836', 'nonce': 3585829992}, {'type': 'v1', 'addr': '172.21.15.164:6837', 'nonce': 3585829992}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24463': {'gid': 24463, 'name': 'i', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.164:6839/1636226820', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6838', 'nonce': 1636226820}, {'type': 'v1', 'addr': '172.21.15.164:6839', 'nonce': 1636226820}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_34406': {'gid': 34406, 'name': 'k', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 15, 'addr': '172.21.15.76:6833/3869483750', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6832', 'nonce': 3869483750}, {'type': 'v1', 'addr': '172.21.15.76:6833', 'nonce': 3869483750}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_34442': {'gid': 34442, 'name': 'e', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.76:6835/3517429435', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6834', 'nonce': 3517429435}, {'type': 'v1', 'addr': '172.21.15.76:6835', 'nonce': 3517429435}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_34472': {'gid': 34472, 'name': 'h', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.76:6837/1150678991', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6836', 'nonce': 1150678991}, {'type': 'v1', 'addr': '172.21.15.76:6837', 'nonce': 1150678991}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_34517': {'gid': 34517, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.76:6839/327145636', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6838', 'nonce': 327145636}, {'type': 'v1', 'addr': '172.21.15.76:6839', 'nonce': 327145636}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 34517, 'qdb_cluster': [34406, 24463, 34442, 34517, 34472]} 2024-04-26T21:55:11.614 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-04-26T21:55:11.614 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-04-26T21:55:11.816 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.815+0000 7f95bca00640 1 -- 172.21.15.72:0/3613655763 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f95b8104980 msgr2=0x7f95b8104da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:11.817 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.815+0000 7f95bca00640 1 --2- 172.21.15.72:0/3613655763 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f95b8104980 0x7f95b8104da0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f95a8009900 tx=0x7f95a802a410 comp rx=0 tx=0).stop 2024-04-26T21:55:11.817 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.815+0000 7f95bca00640 1 -- 172.21.15.72:0/3613655763 shutdown_connections 2024-04-26T21:55:11.817 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.815+0000 7f95bca00640 1 --2- 172.21.15.72:0/3613655763 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f95b8106530 0x7f95b810cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:11.817 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.815+0000 7f95bca00640 1 --2- 172.21.15.72:0/3613655763 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f95b8105b70 0x7f95b8105ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:11.817 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.815+0000 7f95bca00640 1 --2- 172.21.15.72:0/3613655763 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f95b8104980 0x7f95b8104da0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:11.817 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.815+0000 7f95bca00640 1 -- 172.21.15.72:0/3613655763 >> 172.21.15.72:0/3613655763 conn(0x7f95b80fff10 msgr2=0x7f95b8102330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:55:11.817 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.816+0000 7f95bca00640 1 -- 172.21.15.72:0/3613655763 shutdown_connections 2024-04-26T21:55:11.817 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.816+0000 7f95bca00640 1 -- 172.21.15.72:0/3613655763 wait complete. 2024-04-26T21:55:11.817 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.816+0000 7f95bca00640 1 Processor -- start 2024-04-26T21:55:11.817 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.817+0000 7f95bca00640 1 -- start start 2024-04-26T21:55:11.819 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.817+0000 7f95bca00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f95b8104980 0x7f95b819aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:11.819 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.817+0000 7f95bca00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f95b8105b70 0x7f95b819b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:11.819 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.817+0000 7f95bca00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f95b8106530 0x7f95b81a2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:11.819 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.817+0000 7f95bca00640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f95b81103b0 con 0x7f95b8105b70 2024-04-26T21:55:11.819 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.817+0000 7f95bca00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f95b8110230 con 0x7f95b8104980 2024-04-26T21:55:11.820 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.817+0000 7f95bca00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f95b8110530 con 0x7f95b8106530 2024-04-26T21:55:11.820 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.817+0000 7f95afe00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f95b8105b70 0x7f95b819b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:11.820 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.817+0000 7f95afe00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f95b8105b70 0x7f95b819b020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:34098/0 (socket says 172.21.15.72:34098) 2024-04-26T21:55:11.820 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.817+0000 7f95afe00640 1 -- 172.21.15.72:0/3419530532 learned_addr learned my addr 172.21.15.72:0/3419530532 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:55:11.820 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.817+0000 7f95b7400640 1 --2- 172.21.15.72:0/3419530532 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f95b8104980 0x7f95b819aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:11.820 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.817+0000 7f95b7e00640 1 --2- 172.21.15.72:0/3419530532 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f95b8106530 0x7f95b81a2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:11.820 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.817+0000 7f95afe00640 1 -- 172.21.15.72:0/3419530532 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f95b8106530 msgr2=0x7f95b81a2090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:11.820 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.818+0000 7f95afe00640 1 --2- 172.21.15.72:0/3419530532 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f95b8106530 0x7f95b81a2090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:11.820 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.818+0000 7f95afe00640 1 -- 172.21.15.72:0/3419530532 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f95b8104980 msgr2=0x7f95b819aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:11.820 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.818+0000 7f95afe00640 1 --2- 172.21.15.72:0/3419530532 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f95b8104980 0x7f95b819aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:11.820 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.818+0000 7f95afe00640 1 -- 172.21.15.72:0/3419530532 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95b81a2700 con 0x7f95b8105b70 2024-04-26T21:55:11.820 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.818+0000 7f95afe00640 1 --2- 172.21.15.72:0/3419530532 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f95b8105b70 0x7f95b819b020 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f95a000d950 tx=0x7f95a000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:55:11.820 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.818+0000 7f95b4c00640 1 -- 172.21.15.72:0/3419530532 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95a0017070 con 0x7f95b8105b70 2024-04-26T21:55:11.820 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.818+0000 7f95b4c00640 1 -- 172.21.15.72:0/3419530532 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f95a0009c30 con 0x7f95b8105b70 2024-04-26T21:55:11.821 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.818+0000 7f95bca00640 1 -- 172.21.15.72:0/3419530532 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95b81a29f0 con 0x7f95b8105b70 2024-04-26T21:55:11.821 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.818+0000 7f95b4c00640 1 -- 172.21.15.72:0/3419530532 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95a000f620 con 0x7f95b8105b70 2024-04-26T21:55:11.821 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.818+0000 7f95bca00640 1 -- 172.21.15.72:0/3419530532 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95b81a2f40 con 0x7f95b8105b70 2024-04-26T21:55:11.823 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.822+0000 7f95c0fbb740 1 Processor -- start 2024-04-26T21:55:11.823 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.822+0000 7f95c0fbb740 1 -- start start 2024-04-26T21:55:11.823 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.823+0000 7f95c0fbb740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ae11b092c0 0x55ae11b096e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:11.823 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.823+0000 7f95c0fbb740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ae11b09cb0 0x55ae11b0e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:11.823 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.823+0000 7f95c0fbb740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ae11b0ee20 0x55ae11b112b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:11.823 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.823+0000 7f95c0fbb740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55ae11a6d600 con 0x55ae11b092c0 2024-04-26T21:55:11.823 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.823+0000 7f95c0fbb740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55ae11a0a100 con 0x55ae11b09cb0 2024-04-26T21:55:11.823 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.823+0000 7f95c0fbb740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55ae11a695c0 con 0x55ae11b0ee20 2024-04-26T21:55:11.825 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.823+0000 7f95b7e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ae11b092c0 0x55ae11b096e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:11.826 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.823+0000 7f95b4c00640 1 -- 172.21.15.72:0/3419530532 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f95a0010460 con 0x7f95b8105b70 2024-04-26T21:55:11.826 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.823+0000 7f95b7e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ae11b092c0 0x55ae11b096e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:34112/0 (socket says 172.21.15.72:34112) 2024-04-26T21:55:11.826 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.823+0000 7f95b7e00640 1 -- 172.21.15.72:0/949805904 learned_addr learned my addr 172.21.15.72:0/949805904 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:55:11.826 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.824+0000 7f95b7400640 1 --2- 172.21.15.72:0/949805904 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ae11b09cb0 0x55ae11b0e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:11.826 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.824+0000 7f95b7e00640 1 -- 172.21.15.72:0/949805904 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ae11b0ee20 msgr2=0x55ae11b112b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:55:11.826 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.824+0000 7f95b7e00640 1 --2- 172.21.15.72:0/949805904 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ae11b0ee20 0x55ae11b112b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:11.826 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.824+0000 7f95b7e00640 1 -- 172.21.15.72:0/949805904 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ae11b09cb0 msgr2=0x55ae11b0e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:55:11.826 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.824+0000 7f95b7e00640 1 --2- 172.21.15.72:0/949805904 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ae11b09cb0 0x55ae11b0e7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:11.826 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.824+0000 7f95b7e00640 1 -- 172.21.15.72:0/949805904 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ae1177f2d0 con 0x55ae11b092c0 2024-04-26T21:55:11.826 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.824+0000 7f95b7400640 1 --2- 172.21.15.72:0/949805904 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ae11b09cb0 0x55ae11b0e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:55:11.826 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.824+0000 7f95b7e00640 1 --2- 172.21.15.72:0/949805904 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ae11b092c0 0x55ae11b096e0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f95b0009df0 tx=0x7f95b000f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2068c303fa13ce0d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:55:11.826 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.824+0000 7f95a7e00640 1 -- 172.21.15.72:0/949805904 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95b0003970 con 0x55ae11b092c0 2024-04-26T21:55:11.826 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.824+0000 7f95a7e00640 1 -- 172.21.15.72:0/949805904 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f95b0004030 con 0x55ae11b092c0 2024-04-26T21:55:11.826 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.824+0000 7f95a7e00640 1 -- 172.21.15.72:0/949805904 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95b0011660 con 0x55ae11b092c0 2024-04-26T21:55:11.827 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.825+0000 7f95b4c00640 1 --2- 172.21.15.72:0/3419530532 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f959c0a3c40 0x7f959c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:11.827 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.825+0000 7f95c0fbb740 1 -- 172.21.15.72:0/949805904 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ae11b092c0 msgr2=0x55ae11b096e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:11.827 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.825+0000 7f95c0fbb740 1 --2- 172.21.15.72:0/949805904 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ae11b092c0 0x55ae11b096e0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f95b0009df0 tx=0x7f95b000f730 comp rx=0 tx=0).stop 2024-04-26T21:55:11.827 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.825+0000 7f95b4c00640 1 -- 172.21.15.72:0/3419530532 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7f95a00d02d0 con 0x7f95b8105b70 2024-04-26T21:55:11.827 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.825+0000 7f95b7400640 1 --2- 172.21.15.72:0/3419530532 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f959c0a3c40 0x7f959c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:11.827 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.825+0000 7f95c0fbb740 1 -- 172.21.15.72:0/949805904 shutdown_connections 2024-04-26T21:55:11.827 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.825+0000 7f95c0fbb740 1 --2- 172.21.15.72:0/949805904 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ae11b0ee20 0x55ae11b112b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:11.827 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.825+0000 7f95c0fbb740 1 --2- 172.21.15.72:0/949805904 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ae11b09cb0 0x55ae11b0e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:11.827 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.825+0000 7f95c0fbb740 1 --2- 172.21.15.72:0/949805904 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ae11b092c0 0x55ae11b096e0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:11.827 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.825+0000 7f95c0fbb740 1 -- 172.21.15.72:0/949805904 >> 172.21.15.72:0/949805904 conn(0x55ae11a9dd10 msgr2=0x55ae11b04d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:55:11.827 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.825+0000 7f95c0fbb740 1 -- 172.21.15.72:0/949805904 shutdown_connections 2024-04-26T21:55:11.827 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.826+0000 7f95c0fbb740 1 -- 172.21.15.72:0/949805904 wait complete. 2024-04-26T21:55:11.830 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.826+0000 7f95c0fbb740 1 -- start start 2024-04-26T21:55:11.830 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.826+0000 7f95b7400640 1 --2- 172.21.15.72:0/3419530532 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f959c0a3c40 0x7f959c0a6100 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f95a80057c0 tx=0x7f95a8005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:55:11.831 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.826+0000 7f95c0fbb740 1 Processor -- start 2024-04-26T21:55:11.831 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.827+0000 7f95c0fbb740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ae11b092c0 0x55ae11b136b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:11.831 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.827+0000 7f95c0fbb740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ae11b09cb0 0x55ae11b13bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:11.831 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.827+0000 7f95c0fbb740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ae11b0ee20 0x55ae11b1ac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:11.831 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.827+0000 7f95c0fbb740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55ae11b11dd0 con 0x55ae11b0ee20 2024-04-26T21:55:11.831 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.827+0000 7f95c0fbb740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55ae119f4f70 con 0x55ae11b092c0 2024-04-26T21:55:11.831 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.827+0000 7f95c0fbb740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55ae11b11f50 con 0x55ae11b09cb0 2024-04-26T21:55:11.831 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.827+0000 7f95c0fbb740 10 client.0 _ll_register_callbacks cb 0x55ae11a63480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:55:11.831 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.827+0000 7f95b7e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ae11b0ee20 0x55ae11b1ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:11.831 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.827+0000 7f95b7e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ae11b0ee20 0x55ae11b1ac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:34116/0 (socket says 172.21.15.72:34116) 2024-04-26T21:55:11.831 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.827+0000 7f95b7e00640 1 -- 172.21.15.72:0/762881937 learned_addr learned my addr 172.21.15.72:0/762881937 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:55:11.831 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.827+0000 7f95b7400640 1 --2- 172.21.15.72:0/762881937 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ae11b092c0 0x55ae11b136b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:11.831 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.827+0000 7f95afe00640 1 --2- 172.21.15.72:0/762881937 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ae11b09cb0 0x55ae11b13bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:11.831 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.827+0000 7f95b7e00640 1 -- 172.21.15.72:0/762881937 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ae11b09cb0 msgr2=0x55ae11b13bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:55:11.832 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.828+0000 7f95b7e00640 1 --2- 172.21.15.72:0/762881937 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ae11b09cb0 0x55ae11b13bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:11.832 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.828+0000 7f95b7e00640 1 -- 172.21.15.72:0/762881937 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ae11b092c0 msgr2=0x55ae11b136b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:55:11.832 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.828+0000 7f95b7e00640 1 --2- 172.21.15.72:0/762881937 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ae11b092c0 0x55ae11b136b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:11.832 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.828+0000 7f95b7e00640 1 -- 172.21.15.72:0/762881937 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ae118496b0 con 0x55ae11b0ee20 2024-04-26T21:55:11.832 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.828+0000 7f95b7400640 1 --2- 172.21.15.72:0/762881937 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ae11b092c0 0x55ae11b136b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:55:11.832 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.828+0000 7f95b7e00640 1 --2- 172.21.15.72:0/762881937 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ae11b0ee20 0x55ae11b1ac30 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f95b0009dc0 tx=0x7f95b0004570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df1dd77443e45a87 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:55:11.832 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.828+0000 7f95a4c00640 10 client.0 ms_handle_connect on v2:172.21.15.72:3300/0 2024-04-26T21:55:11.832 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.828+0000 7f95a4c00640 1 -- 172.21.15.72:0/762881937 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95b0029060 con 0x55ae11b0ee20 2024-04-26T21:55:11.832 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.828+0000 7f95a4c00640 1 -- 172.21.15.72:0/762881937 <== mon.0 v2:172.21.15.72:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7f95b0004c90 con 0x55ae11b0ee20 2024-04-26T21:55:11.832 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.828+0000 7f95c0fbb740 1 -- 172.21.15.72:0/762881937 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f95b0004030 con 0x55ae11b0ee20 2024-04-26T21:55:11.832 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.828+0000 7f95a4c00640 1 -- 172.21.15.72:0/762881937 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f95b0003970 con 0x55ae11b0ee20 2024-04-26T21:55:11.832 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.828+0000 7f95c0fbb740 10 client.15288 fetch_fsmap learned FSMap version 26 2024-04-26T21:55:11.832 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.828+0000 7f95c0fbb740 1 -- 172.21.15.72:0/762881937 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ae11b1db80 con 0x55ae11b0ee20 2024-04-26T21:55:11.832 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.829+0000 7f95a4c00640 1 -- 172.21.15.72:0/762881937 <== mon.0 v2:172.21.15.72:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f95b0004030 con 0x55ae11b0ee20 2024-04-26T21:55:11.832 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.829+0000 7f95a4c00640 1 -- 172.21.15.72:0/762881937 <== mon.0 v2:172.21.15.72:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f95b0005690 con 0x55ae11b0ee20 2024-04-26T21:55:11.832 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.829+0000 7f95c0fbb740 10 client.15288 fetch_fsmap finished waiting for FSMap version 26 2024-04-26T21:55:11.833 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.829+0000 7f95c0fbb740 10 client.15288 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-26T21:55:11.833 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.829+0000 7f95c0fbb740 20 client.15288 populate_metadata read hostname 'smithi072' 2024-04-26T21:55:11.833 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.829+0000 7f95c0fbb740 1 --2- 172.21.15.72:0/762881937 >> [v2:172.21.15.164:6838/1636226820,v1:172.21.15.164:6839/1636226820] conn(0x55ae11b1fe70 0x55ae11b212e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:11.833 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.829+0000 7f95c0fbb740 4 client.15288 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:55:11.833 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.829+0000 7f95c0fbb740 1 -- 172.21.15.72:0/762881937 --> [v2:172.21.15.164:6838/1636226820,v1:172.21.15.164:6839/1636226820] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ae11a7b210 con 0x55ae11b1fe70 2024-04-26T21:55:11.833 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.830+0000 7f95b7400640 1 --2- 172.21.15.72:0/762881937 >> [v2:172.21.15.164:6838/1636226820,v1:172.21.15.164:6839/1636226820] conn(0x55ae11b1fe70 0x55ae11b212e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:11.834 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.831+0000 7f95b7400640 1 --2- 172.21.15.72:0/762881937 >> [v2:172.21.15.164:6838/1636226820,v1:172.21.15.164:6839/1636226820] conn(0x55ae11b1fe70 0x55ae11b212e0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=fad10ad378fb2f0c server_cookie=6530502cf90025c in_seq=0 out_seq=0 2024-04-26T21:55:11.834 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.831+0000 7f95a4c00640 10 client.15288 ms_handle_connect on v2:172.21.15.164:6838/1636226820 2024-04-26T21:55:11.834 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.833+0000 7f95a4c00640 1 -- 172.21.15.72:0/762881937 <== mds.4 v2:172.21.15.164:6838/1636226820 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ae11a7b210 con 0x55ae11b1fe70 2024-04-26T21:55:11.834 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.833+0000 7f95a4c00640 10 client.15288 handle_command_reply: tid=0 2024-04-26T21:55:11.834 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.833+0000 7f95c0fbb740 1 client.15288 shutdown 2024-04-26T21:55:11.835 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.833+0000 7f95c0fbb740 1 -- 172.21.15.72:0/762881937 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ae11b0ee20 msgr2=0x55ae11b1ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:11.835 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.833+0000 7f95c0fbb740 1 --2- 172.21.15.72:0/762881937 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ae11b0ee20 0x55ae11b1ac30 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f95b0009dc0 tx=0x7f95b0004570 comp rx=0 tx=0).stop 2024-04-26T21:55:11.835 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.833+0000 7f95c0fbb740 1 -- 172.21.15.72:0/762881937 shutdown_connections 2024-04-26T21:55:11.835 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.833+0000 7f95c0fbb740 1 --2- 172.21.15.72:0/762881937 >> [v2:172.21.15.164:6838/1636226820,v1:172.21.15.164:6839/1636226820] conn(0x55ae11b1fe70 0x55ae11b212e0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:11.835 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.833+0000 7f95c0fbb740 1 --2- 172.21.15.72:0/762881937 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ae11b0ee20 0x55ae11b1ac30 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:11.835 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.833+0000 7f95c0fbb740 1 --2- 172.21.15.72:0/762881937 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ae11b09cb0 0x55ae11b13bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:11.835 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.833+0000 7f95a4c00640 0 client.15288 ms_handle_reset on v2:172.21.15.164:6838/1636226820 2024-04-26T21:55:11.835 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.833+0000 7f95c0fbb740 1 --2- 172.21.15.72:0/762881937 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ae11b092c0 0x55ae11b136b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:11.835 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.833+0000 7f95c0fbb740 1 -- 172.21.15.72:0/762881937 >> 172.21.15.72:0/762881937 conn(0x55ae11a9dd10 msgr2=0x55ae11b03f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:55:11.835 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.833+0000 7f95c0fbb740 1 -- 172.21.15.72:0/762881937 shutdown_connections 2024-04-26T21:55:11.835 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.834+0000 7f95c0fbb740 1 -- 172.21.15.72:0/762881937 wait complete. 2024-04-26T21:55:11.835 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.834+0000 7f95c0fbb740 20 client.15288 trim_cache size 0 max 16384 2024-04-26T21:55:11.854 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.853+0000 7f95c0fbb740 1 Processor -- start 2024-04-26T21:55:11.854 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.853+0000 7f95c0fbb740 1 -- start start 2024-04-26T21:55:11.854 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.854+0000 7f95c0fbb740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ae11b092c0 0x55ae11b0b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:11.854 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.854+0000 7f95c0fbb740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ae11b09cb0 0x55ae11b1abf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:11.854 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.854+0000 7f95c0fbb740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ae11b0ee20 0x55ae11b0b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:11.854 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.854+0000 7f95c0fbb740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55ae11b1f620 con 0x55ae11b092c0 2024-04-26T21:55:11.854 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.854+0000 7f95c0fbb740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55ae11b1fcf0 con 0x55ae11b0ee20 2024-04-26T21:55:11.854 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.854+0000 7f95c0fbb740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55ae11b203c0 con 0x55ae11b09cb0 2024-04-26T21:55:11.854 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.854+0000 7f95b7400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ae11b092c0 0x55ae11b0b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:11.854 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.854+0000 7f95b7400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ae11b092c0 0x55ae11b0b080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:34122/0 (socket says 172.21.15.72:34122) 2024-04-26T21:55:11.855 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.854+0000 7f95b7400640 1 -- 172.21.15.72:0/2202522429 learned_addr learned my addr 172.21.15.72:0/2202522429 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:55:11.858 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.854+0000 7f95afe00640 1 --2- 172.21.15.72:0/2202522429 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ae11b09cb0 0x55ae11b1abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:11.858 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.854+0000 7f95b7400640 1 -- 172.21.15.72:0/2202522429 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ae11b09cb0 msgr2=0x55ae11b1abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:55:11.858 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.854+0000 7f95b7400640 1 --2- 172.21.15.72:0/2202522429 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ae11b09cb0 0x55ae11b1abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:11.858 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.854+0000 7f95b7400640 1 -- 172.21.15.72:0/2202522429 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ae11b0ee20 msgr2=0x55ae11b0b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:55:11.859 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.854+0000 7f95b7400640 1 --2- 172.21.15.72:0/2202522429 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ae11b0ee20 0x55ae11b0b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:11.859 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.854+0000 7f95b7400640 1 -- 172.21.15.72:0/2202522429 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ae11b14550 con 0x55ae11b092c0 2024-04-26T21:55:11.859 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.854+0000 7f95afe00640 1 --2- 172.21.15.72:0/2202522429 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ae11b09cb0 0x55ae11b1abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:55:11.859 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.855+0000 7f95b7400640 1 --2- 172.21.15.72:0/2202522429 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ae11b092c0 0x55ae11b0b080 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f95a8009340 tx=0x7f95a803d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b17f639a7ecd9d6b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:55:11.859 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.855+0000 7f95a7400640 1 -- 172.21.15.72:0/2202522429 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95a8050c10 con 0x55ae11b092c0 2024-04-26T21:55:11.859 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.855+0000 7f95a7400640 1 -- 172.21.15.72:0/2202522429 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f95a8056070 con 0x55ae11b092c0 2024-04-26T21:55:11.859 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.855+0000 7f95a7400640 1 -- 172.21.15.72:0/2202522429 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95a8003c10 con 0x55ae11b092c0 2024-04-26T21:55:11.859 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.856+0000 7f95c0fbb740 1 -- 172.21.15.72:0/2202522429 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ae11b092c0 msgr2=0x55ae11b0b080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:11.859 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.856+0000 7f95c0fbb740 1 --2- 172.21.15.72:0/2202522429 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ae11b092c0 0x55ae11b0b080 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f95a8009340 tx=0x7f95a803d000 comp rx=0 tx=0).stop 2024-04-26T21:55:11.859 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.856+0000 7f95c0fbb740 1 -- 172.21.15.72:0/2202522429 shutdown_connections 2024-04-26T21:55:11.859 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.856+0000 7f95c0fbb740 1 --2- 172.21.15.72:0/2202522429 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ae11b0ee20 0x55ae11b0b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:11.859 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.856+0000 7f95c0fbb740 1 --2- 172.21.15.72:0/2202522429 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ae11b09cb0 0x55ae11b1abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:11.859 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.856+0000 7f95c0fbb740 1 --2- 172.21.15.72:0/2202522429 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ae11b092c0 0x55ae11b0b080 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:11.859 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.856+0000 7f95c0fbb740 1 -- 172.21.15.72:0/2202522429 >> 172.21.15.72:0/2202522429 conn(0x55ae11a9dd10 msgr2=0x55ae11b047c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:55:11.860 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.856+0000 7f95c0fbb740 1 -- 172.21.15.72:0/2202522429 shutdown_connections 2024-04-26T21:55:11.860 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.856+0000 7f95c0fbb740 1 -- 172.21.15.72:0/2202522429 wait complete. 2024-04-26T21:55:11.860 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.856+0000 7f95c0fbb740 1 -- start start 2024-04-26T21:55:11.860 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.856+0000 7f95c0fbb740 1 Processor -- start 2024-04-26T21:55:11.860 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.857+0000 7f95c0fbb740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ae11b092c0 0x55ae11b16a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:11.860 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.857+0000 7f95c0fbb740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ae11b09cb0 0x55ae11b16fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:11.860 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.857+0000 7f95c0fbb740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ae11b0ee20 0x55ae11b19f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:11.860 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.857+0000 7f95c0fbb740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55ae11b21160 con 0x55ae11b092c0 2024-04-26T21:55:11.860 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.857+0000 7f95c0fbb740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55ae11b20a90 con 0x55ae11b0ee20 2024-04-26T21:55:11.860 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.857+0000 7f95c0fbb740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55ae11b21830 con 0x55ae11b09cb0 2024-04-26T21:55:11.860 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.857+0000 7f95c0fbb740 10 client.0 _ll_register_callbacks cb 0x7f95b0009c40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:55:11.860 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.857+0000 7f95b7400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ae11b092c0 0x55ae11b16a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:11.860 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.857+0000 7f95b7400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ae11b092c0 0x55ae11b16a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:34128/0 (socket says 172.21.15.72:34128) 2024-04-26T21:55:11.860 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.857+0000 7f95b7400640 1 -- 172.21.15.72:0/2351755629 learned_addr learned my addr 172.21.15.72:0/2351755629 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:55:11.860 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.857+0000 7f95b7400640 1 -- 172.21.15.72:0/2351755629 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ae11b09cb0 msgr2=0x55ae11b16fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:55:11.861 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.857+0000 7f95b7400640 1 --2- 172.21.15.72:0/2351755629 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ae11b09cb0 0x55ae11b16fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:11.861 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.857+0000 7f95b7400640 1 -- 172.21.15.72:0/2351755629 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ae11b0ee20 msgr2=0x55ae11b19f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:55:11.861 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.857+0000 7f95b7400640 1 --2- 172.21.15.72:0/2351755629 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ae11b0ee20 0x55ae11b19f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:11.861 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.857+0000 7f95b7400640 1 -- 172.21.15.72:0/2351755629 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ae11b0eb30 con 0x55ae11b092c0 2024-04-26T21:55:11.861 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.858+0000 7f95b7400640 1 --2- 172.21.15.72:0/2351755629 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ae11b092c0 0x55ae11b16a80 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f95a8053040 tx=0x7f95a8051f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab2d1c70909a86cb server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:55:11.861 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.858+0000 7f95a4c00640 10 client.0 ms_handle_connect on v2:172.21.15.72:3300/0 2024-04-26T21:55:11.861 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.858+0000 7f95a4c00640 1 -- 172.21.15.72:0/2351755629 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95a805e020 con 0x55ae11b092c0 2024-04-26T21:55:11.861 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.858+0000 7f95a4c00640 1 -- 172.21.15.72:0/2351755629 <== mon.0 v2:172.21.15.72:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7f95a8003de0 con 0x55ae11b092c0 2024-04-26T21:55:11.861 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.858+0000 7f95c0fbb740 1 -- 172.21.15.72:0/2351755629 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f95a8056070 con 0x55ae11b092c0 2024-04-26T21:55:11.861 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.858+0000 7f95a4c00640 1 -- 172.21.15.72:0/2351755629 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f95a8069050 con 0x55ae11b092c0 2024-04-26T21:55:11.861 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.858+0000 7f95c0fbb740 10 client.15294 fetch_fsmap learned FSMap version 26 2024-04-26T21:55:11.861 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.858+0000 7f95c0fbb740 1 -- 172.21.15.72:0/2351755629 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ae11b8ea40 con 0x55ae11b092c0 2024-04-26T21:55:11.863 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.859+0000 7f95a4c00640 1 -- 172.21.15.72:0/2351755629 <== mon.0 v2:172.21.15.72:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f95a8056070 con 0x55ae11b092c0 2024-04-26T21:55:11.863 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.859+0000 7f95a4c00640 1 -- 172.21.15.72:0/2351755629 <== mon.0 v2:172.21.15.72:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f95a8031aa0 con 0x55ae11b092c0 2024-04-26T21:55:11.864 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.859+0000 7f95c0fbb740 10 client.15294 fetch_fsmap finished waiting for FSMap version 26 2024-04-26T21:55:11.864 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.859+0000 7f95c0fbb740 10 client.15294 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-26T21:55:11.864 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.859+0000 7f95c0fbb740 20 client.15294 populate_metadata read hostname 'smithi072' 2024-04-26T21:55:11.864 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.859+0000 7f95c0fbb740 1 --2- 172.21.15.72:0/2351755629 >> [v2:172.21.15.164:6838/1636226820,v1:172.21.15.164:6839/1636226820] conn(0x55ae11b93280 0x55ae11b91f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:11.864 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.859+0000 7f95c0fbb740 4 client.15294 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T21:55:11.864 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.859+0000 7f95c0fbb740 1 -- 172.21.15.72:0/2351755629 --> [v2:172.21.15.164:6838/1636226820,v1:172.21.15.164:6839/1636226820] -- command(tid 0: {"prefix": "status"}) -- 0x55ae11a75080 con 0x55ae11b93280 2024-04-26T21:55:11.864 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.860+0000 7f95afe00640 1 --2- 172.21.15.72:0/2351755629 >> [v2:172.21.15.164:6838/1636226820,v1:172.21.15.164:6839/1636226820] conn(0x55ae11b93280 0x55ae11b91f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:11.864 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.860+0000 7f95afe00640 1 --2- 172.21.15.72:0/2351755629 >> [v2:172.21.15.164:6838/1636226820,v1:172.21.15.164:6839/1636226820] conn(0x55ae11b93280 0x55ae11b91f90 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=fb90eaf360225911 server_cookie=9da99a83dd1aa36c in_seq=0 out_seq=0 2024-04-26T21:55:11.864 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.860+0000 7f95a4c00640 10 client.15294 ms_handle_connect on v2:172.21.15.164:6838/1636226820 2024-04-26T21:55:11.864 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.861+0000 7f95a4c00640 1 -- 172.21.15.72:0/2351755629 <== mds.4 v2:172.21.15.164:6838/1636226820 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55ae11a75080 con 0x55ae11b93280 2024-04-26T21:55:11.864 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.861+0000 7f95a4c00640 10 client.15294 handle_command_reply: tid=0 2024-04-26T21:55:11.864 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.861+0000 7f95c0fbb740 1 client.15294 shutdown 2024-04-26T21:55:11.865 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.862+0000 7f95c0fbb740 1 -- 172.21.15.72:0/2351755629 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ae11b092c0 msgr2=0x55ae11b16a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:11.865 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.862+0000 7f95c0fbb740 1 --2- 172.21.15.72:0/2351755629 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ae11b092c0 0x55ae11b16a80 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f95a8053040 tx=0x7f95a8051f90 comp rx=0 tx=0).stop 2024-04-26T21:55:11.865 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.862+0000 7f95c0fbb740 1 -- 172.21.15.72:0/2351755629 shutdown_connections 2024-04-26T21:55:11.865 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.862+0000 7f95c0fbb740 1 --2- 172.21.15.72:0/2351755629 >> [v2:172.21.15.164:6838/1636226820,v1:172.21.15.164:6839/1636226820] conn(0x55ae11b93280 0x55ae11b91f90 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:11.865 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.862+0000 7f95c0fbb740 1 --2- 172.21.15.72:0/2351755629 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ae11b0ee20 0x55ae11b19f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:11.865 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.862+0000 7f95c0fbb740 1 --2- 172.21.15.72:0/2351755629 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ae11b09cb0 0x55ae11b16fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:11.865 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.862+0000 7f95c0fbb740 1 --2- 172.21.15.72:0/2351755629 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ae11b092c0 0x55ae11b16a80 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:11.865 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.862+0000 7f95a4c00640 0 client.15294 ms_handle_reset on v2:172.21.15.164:6838/1636226820 2024-04-26T21:55:11.865 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.862+0000 7f95c0fbb740 1 -- 172.21.15.72:0/2351755629 >> 172.21.15.72:0/2351755629 conn(0x55ae11a9dd10 msgr2=0x55ae11b04700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:55:11.865 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.862+0000 7f95c0fbb740 1 -- 172.21.15.72:0/2351755629 shutdown_connections 2024-04-26T21:55:11.865 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.862+0000 7f95c0fbb740 1 -- 172.21.15.72:0/2351755629 wait complete. 2024-04-26T21:55:11.865 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.862+0000 7f95c0fbb740 20 client.15294 trim_cache size 0 max 16384 2024-04-26T21:55:11.865 INFO:teuthology.orchestra.run.smithi072.stdout:{ 2024-04-26T21:55:11.866 INFO:teuthology.orchestra.run.smithi072.stdout: "cluster_fsid": "a1780950-0415-11ef-bc93-c7b262605968", 2024-04-26T21:55:11.866 INFO:teuthology.orchestra.run.smithi072.stdout: "whoami": 4, 2024-04-26T21:55:11.866 INFO:teuthology.orchestra.run.smithi072.stdout: "id": 24463, 2024-04-26T21:55:11.866 INFO:teuthology.orchestra.run.smithi072.stdout: "want_state": "up:active", 2024-04-26T21:55:11.866 INFO:teuthology.orchestra.run.smithi072.stdout: "state": "up:active", 2024-04-26T21:55:11.866 INFO:teuthology.orchestra.run.smithi072.stdout: "fs_name": "cephfs", 2024-04-26T21:55:11.866 INFO:teuthology.orchestra.run.smithi072.stdout: "rank_uptime": 134.28321045300001, 2024-04-26T21:55:11.866 INFO:teuthology.orchestra.run.smithi072.stdout: "mdsmap_epoch": 26, 2024-04-26T21:55:11.866 INFO:teuthology.orchestra.run.smithi072.stdout: "osdmap_epoch": 91, 2024-04-26T21:55:11.866 INFO:teuthology.orchestra.run.smithi072.stdout: "osdmap_epoch_barrier": 87, 2024-04-26T21:55:11.866 INFO:teuthology.orchestra.run.smithi072.stdout: "uptime": 159.36707598800001 2024-04-26T21:55:11.866 INFO:teuthology.orchestra.run.smithi072.stdout:} 2024-04-26T21:55:11.867 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.862+0000 7f95bca00640 1 -- 172.21.15.72:0/3419530532 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f959c0a3c40 msgr2=0x7f959c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:11.867 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.862+0000 7f95bca00640 1 --2- 172.21.15.72:0/3419530532 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f959c0a3c40 0x7f959c0a6100 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f95a80057c0 tx=0x7f95a8005a20 comp rx=0 tx=0).stop 2024-04-26T21:55:11.867 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.862+0000 7f95bca00640 1 -- 172.21.15.72:0/3419530532 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f95b8105b70 msgr2=0x7f95b819b020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:11.867 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.862+0000 7f95bca00640 1 --2- 172.21.15.72:0/3419530532 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f95b8105b70 0x7f95b819b020 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f95a000d950 tx=0x7f95a000de10 comp rx=0 tx=0).stop 2024-04-26T21:55:11.867 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.863+0000 7f95bca00640 1 -- 172.21.15.72:0/3419530532 shutdown_connections 2024-04-26T21:55:11.867 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.863+0000 7f95bca00640 1 --2- 172.21.15.72:0/3419530532 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f959c0a3c40 0x7f959c0a6100 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:11.868 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.863+0000 7f95bca00640 1 --2- 172.21.15.72:0/3419530532 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f95b8106530 0x7f95b81a2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:11.868 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.863+0000 7f95bca00640 1 --2- 172.21.15.72:0/3419530532 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f95b8105b70 0x7f95b819b020 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:11.868 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.863+0000 7f95bca00640 1 --2- 172.21.15.72:0/3419530532 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f95b8104980 0x7f95b819aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:11.868 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.863+0000 7f95bca00640 1 -- 172.21.15.72:0/3419530532 >> 172.21.15.72:0/3419530532 conn(0x7f95b80fff10 msgr2=0x7f95b8108fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:55:11.868 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.863+0000 7f95bca00640 1 -- 172.21.15.72:0/3419530532 shutdown_connections 2024-04-26T21:55:11.868 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:11.863+0000 7f95bca00640 1 -- 172.21.15.72:0/3419530532 wait complete. 2024-04-26T21:55:11.881 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.k status 2024-04-26T21:55:12.083 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.083+0000 7f89bea00640 1 -- 172.21.15.72:0/902145612 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f89b810ab60 msgr2=0x7f89b81113f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:12.083 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.083+0000 7f89bea00640 1 --2- 172.21.15.72:0/902145612 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f89b810ab60 0x7f89b81113f0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f89b0009ca0 tx=0x7f89b002a8c0 comp rx=0 tx=0).stop 2024-04-26T21:55:12.083 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.083+0000 7f89bea00640 1 -- 172.21.15.72:0/902145612 shutdown_connections 2024-04-26T21:55:12.083 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.083+0000 7f89bea00640 1 --2- 172.21.15.72:0/902145612 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f89b810ab60 0x7f89b81113f0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:12.083 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.083+0000 7f89bea00640 1 --2- 172.21.15.72:0/902145612 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f89b810a1a0 0x7f89b810a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:12.083 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.083+0000 7f89bea00640 1 --2- 172.21.15.72:0/902145612 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f89b8108fb0 0x7f89b81093d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:12.084 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.083+0000 7f89bea00640 1 -- 172.21.15.72:0/902145612 >> 172.21.15.72:0/902145612 conn(0x7f89b8077f40 msgr2=0x7f89b807a380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:55:12.084 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.083+0000 7f89bea00640 1 -- 172.21.15.72:0/902145612 shutdown_connections 2024-04-26T21:55:12.084 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.083+0000 7f89bea00640 1 -- 172.21.15.72:0/902145612 wait complete. 2024-04-26T21:55:12.085 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.084+0000 7f89bea00640 1 Processor -- start 2024-04-26T21:55:12.085 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.084+0000 7f89bea00640 1 -- start start 2024-04-26T21:55:12.086 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.084+0000 7f89bea00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f89b8108fb0 0x7f89b819f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:12.086 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.084+0000 7f89bea00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f89b810a1a0 0x7f89b819f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:12.086 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.084+0000 7f89bea00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f89b810ab60 0x7f89b81a6640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:12.086 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.084+0000 7f89bea00640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f89b8114a50 con 0x7f89b810ab60 2024-04-26T21:55:12.086 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.084+0000 7f89bea00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f89b81148d0 con 0x7f89b8108fb0 2024-04-26T21:55:12.086 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.084+0000 7f89bea00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f89b8114bd0 con 0x7f89b810a1a0 2024-04-26T21:55:12.086 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.085+0000 7f89bca00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f89b810ab60 0x7f89b81a6640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:12.086 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.085+0000 7f89bca00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f89b810ab60 0x7f89b81a6640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:34158/0 (socket says 172.21.15.72:34158) 2024-04-26T21:55:12.086 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.085+0000 7f89bca00640 1 -- 172.21.15.72:0/94637082 learned_addr learned my addr 172.21.15.72:0/94637082 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:55:12.086 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.085+0000 7f89b7e00640 1 --2- 172.21.15.72:0/94637082 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f89b8108fb0 0x7f89b819f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:12.086 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.085+0000 7f89afe00640 1 --2- 172.21.15.72:0/94637082 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f89b810a1a0 0x7f89b819f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:12.086 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.085+0000 7f89bca00640 1 -- 172.21.15.72:0/94637082 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f89b810a1a0 msgr2=0x7f89b819f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:12.087 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.085+0000 7f89bca00640 1 --2- 172.21.15.72:0/94637082 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f89b810a1a0 0x7f89b819f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:12.087 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.085+0000 7f89bca00640 1 -- 172.21.15.72:0/94637082 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f89b8108fb0 msgr2=0x7f89b819f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:12.087 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.085+0000 7f89bca00640 1 --2- 172.21.15.72:0/94637082 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f89b8108fb0 0x7f89b819f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:12.087 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.085+0000 7f89bca00640 1 -- 172.21.15.72:0/94637082 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89b81a6cb0 con 0x7f89b810ab60 2024-04-26T21:55:12.087 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.085+0000 7f89b7e00640 1 --2- 172.21.15.72:0/94637082 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f89b8108fb0 0x7f89b819f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:55:12.087 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.086+0000 7f89bca00640 1 --2- 172.21.15.72:0/94637082 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f89b810ab60 0x7f89b81a6640 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f89b0009dc0 tx=0x7f89b0003c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:55:12.088 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.086+0000 7f89b5600640 1 -- 172.21.15.72:0/94637082 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89b0004240 con 0x7f89b810ab60 2024-04-26T21:55:12.088 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.086+0000 7f89b5600640 1 -- 172.21.15.72:0/94637082 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f89b00305c0 con 0x7f89b810ab60 2024-04-26T21:55:12.088 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.086+0000 7f89bea00640 1 -- 172.21.15.72:0/94637082 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89b81a6f40 con 0x7f89b810ab60 2024-04-26T21:55:12.088 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.086+0000 7f89b5600640 1 -- 172.21.15.72:0/94637082 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89b003c670 con 0x7f89b810ab60 2024-04-26T21:55:12.088 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.086+0000 7f89bea00640 1 -- 172.21.15.72:0/94637082 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89b81a7400 con 0x7f89b810ab60 2024-04-26T21:55:12.090 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.089+0000 7f89b5600640 1 -- 172.21.15.72:0/94637082 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f89b0030a70 con 0x7f89b810ab60 2024-04-26T21:55:12.090 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.089+0000 7f89c19e9740 1 Processor -- start 2024-04-26T21:55:12.090 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.089+0000 7f89c19e9740 1 -- start start 2024-04-26T21:55:12.090 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.090+0000 7f89c19e9740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5578189e10f0 0x5578189e1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:12.090 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.090+0000 7f89c19e9740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5578189e1ae0 0x5578189e65e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:12.090 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.090+0000 7f89c19e9740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5578189e6c50 0x5578189e90e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:12.090 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.090+0000 7f89c19e9740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x557818945600 con 0x5578189e6c50 2024-04-26T21:55:12.090 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.090+0000 7f89c19e9740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5578188e2100 con 0x5578189e1ae0 2024-04-26T21:55:12.090 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.090+0000 7f89c19e9740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5578189415c0 con 0x5578189e10f0 2024-04-26T21:55:12.092 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.090+0000 7f89afe00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5578189e6c50 0x5578189e90e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:12.092 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.090+0000 7f89afe00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5578189e6c50 0x5578189e90e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:34170/0 (socket says 172.21.15.72:34170) 2024-04-26T21:55:12.092 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.090+0000 7f89afe00640 1 -- 172.21.15.72:0/3065385791 learned_addr learned my addr 172.21.15.72:0/3065385791 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:55:12.093 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.090+0000 7f89afe00640 1 --2- 172.21.15.72:0/3065385791 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5578189e10f0 0x5578189e1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:12.093 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.090+0000 7f89b7e00640 1 --2- 172.21.15.72:0/3065385791 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5578189e1ae0 0x5578189e65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:12.093 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.091+0000 7f89afe00640 1 -- 172.21.15.72:0/3065385791 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5578189e10f0 msgr2=0x5578189e1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:55:12.093 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.091+0000 7f89afe00640 1 --2- 172.21.15.72:0/3065385791 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5578189e10f0 0x5578189e1510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:12.093 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.091+0000 7f89afe00640 1 -- 172.21.15.72:0/3065385791 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5578189e1ae0 msgr2=0x5578189e65e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:55:12.093 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.091+0000 7f89afe00640 1 --2- 172.21.15.72:0/3065385791 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5578189e1ae0 0x5578189e65e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:12.093 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.091+0000 7f89afe00640 1 -- 172.21.15.72:0/3065385791 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5578186572d0 con 0x5578189e6c50 2024-04-26T21:55:12.093 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.091+0000 7f89afe00640 1 --2- 172.21.15.72:0/3065385791 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5578189e6c50 0x5578189e90e0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f89a000bf70 tx=0x7f89a000b640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2926ad76b02a1b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:55:12.093 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.091+0000 7f89b5600640 1 --2- 172.21.15.72:0/94637082 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f899c0a3c40 0x7f899c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:12.093 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.091+0000 7f89ad200640 1 -- 172.21.15.72:0/3065385791 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89a0018030 con 0x5578189e6c50 2024-04-26T21:55:12.093 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.091+0000 7f89ad200640 1 -- 172.21.15.72:0/3065385791 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f89a0010bb0 con 0x5578189e6c50 2024-04-26T21:55:12.093 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.091+0000 7f89ad200640 1 -- 172.21.15.72:0/3065385791 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89a000ea50 con 0x5578189e6c50 2024-04-26T21:55:12.093 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.091+0000 7f89b5600640 1 -- 172.21.15.72:0/94637082 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7f89b00f2050 con 0x7f89b810ab60 2024-04-26T21:55:12.093 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.091+0000 7f89b7e00640 1 --2- 172.21.15.72:0/94637082 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f899c0a3c40 0x7f899c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:12.094 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.092+0000 7f89c19e9740 1 -- 172.21.15.72:0/3065385791 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5578189e6c50 msgr2=0x5578189e90e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:12.094 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.092+0000 7f89c19e9740 1 --2- 172.21.15.72:0/3065385791 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5578189e6c50 0x5578189e90e0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f89a000bf70 tx=0x7f89a000b640 comp rx=0 tx=0).stop 2024-04-26T21:55:12.094 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.092+0000 7f89c19e9740 1 -- 172.21.15.72:0/3065385791 shutdown_connections 2024-04-26T21:55:12.094 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.092+0000 7f89c19e9740 1 --2- 172.21.15.72:0/3065385791 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5578189e6c50 0x5578189e90e0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:12.094 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.092+0000 7f89c19e9740 1 --2- 172.21.15.72:0/3065385791 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5578189e1ae0 0x5578189e65e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:12.094 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.092+0000 7f89c19e9740 1 --2- 172.21.15.72:0/3065385791 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5578189e10f0 0x5578189e1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:12.094 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.092+0000 7f89c19e9740 1 -- 172.21.15.72:0/3065385791 >> 172.21.15.72:0/3065385791 conn(0x557818975d10 msgr2=0x5578189dcb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:55:12.094 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.092+0000 7f89b7e00640 1 --2- 172.21.15.72:0/94637082 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f899c0a3c40 0x7f899c0a6100 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x5578189e1ee0 tx=0x7f89a8008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:55:12.094 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.092+0000 7f89c19e9740 1 -- 172.21.15.72:0/3065385791 shutdown_connections 2024-04-26T21:55:12.094 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.092+0000 7f89c19e9740 1 -- 172.21.15.72:0/3065385791 wait complete. 2024-04-26T21:55:12.094 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.093+0000 7f89c19e9740 1 -- start start 2024-04-26T21:55:12.097 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.093+0000 7f89c19e9740 1 Processor -- start 2024-04-26T21:55:12.097 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.093+0000 7f89c19e9740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5578189e10f0 0x5578189eb570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:12.098 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.093+0000 7f89c19e9740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5578189e1ae0 0x5578189ebab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:12.098 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.093+0000 7f89c19e9740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5578189e6c50 0x5578189f2af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:12.098 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.093+0000 7f89c19e9740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x5578189e9c00 con 0x5578189e6c50 2024-04-26T21:55:12.098 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.093+0000 7f89c19e9740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5578188ccf70 con 0x5578189e1ae0 2024-04-26T21:55:12.098 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.093+0000 7f89c19e9740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5578189e9d80 con 0x5578189e10f0 2024-04-26T21:55:12.098 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.093+0000 7f89c19e9740 10 client.0 _ll_register_callbacks cb 0x55781893b480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:55:12.098 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.094+0000 7f89bca00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5578189e6c50 0x5578189f2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:12.098 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.094+0000 7f89bca00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5578189e6c50 0x5578189f2af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:34180/0 (socket says 172.21.15.72:34180) 2024-04-26T21:55:12.098 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.094+0000 7f89bca00640 1 -- 172.21.15.72:0/1598192717 learned_addr learned my addr 172.21.15.72:0/1598192717 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:55:12.098 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.094+0000 7f89b7e00640 1 --2- 172.21.15.72:0/1598192717 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5578189e10f0 0x5578189eb570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:12.098 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.094+0000 7f89bca00640 1 -- 172.21.15.72:0/1598192717 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5578189e10f0 msgr2=0x5578189eb570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:55:12.098 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.094+0000 7f89bca00640 1 --2- 172.21.15.72:0/1598192717 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5578189e10f0 0x5578189eb570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:12.099 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.094+0000 7f89bca00640 1 -- 172.21.15.72:0/1598192717 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5578189e1ae0 msgr2=0x5578189ebab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:55:12.099 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.094+0000 7f89bca00640 1 --2- 172.21.15.72:0/1598192717 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5578189e1ae0 0x5578189ebab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:12.099 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.094+0000 7f89bca00640 1 -- 172.21.15.72:0/1598192717 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5578187216b0 con 0x5578189e6c50 2024-04-26T21:55:12.099 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.094+0000 7f89b7e00640 1 --2- 172.21.15.72:0/1598192717 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5578189e10f0 0x5578189eb570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:55:12.099 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.094+0000 7f89bca00640 1 --2- 172.21.15.72:0/1598192717 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5578189e6c50 0x5578189f2af0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f89b00ba6a0 tx=0x7f89b00bea00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=955c59576e6bfe8b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:55:12.099 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.094+0000 7f89a5600640 10 client.0 ms_handle_connect on v2:172.21.15.72:3300/0 2024-04-26T21:55:12.099 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.094+0000 7f89a5600640 1 -- 172.21.15.72:0/1598192717 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89b00c6eb0 con 0x5578189e6c50 2024-04-26T21:55:12.099 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.094+0000 7f89a5600640 1 -- 172.21.15.72:0/1598192717 <== mon.0 v2:172.21.15.72:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7f89b0030770 con 0x5578189e6c50 2024-04-26T21:55:12.099 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.094+0000 7f89c19e9740 1 -- 172.21.15.72:0/1598192717 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f89a0010bb0 con 0x5578189e6c50 2024-04-26T21:55:12.099 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.095+0000 7f89a5600640 1 -- 172.21.15.72:0/1598192717 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f89b00dc030 con 0x5578189e6c50 2024-04-26T21:55:12.099 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.095+0000 7f89c19e9740 10 client.15306 fetch_fsmap learned FSMap version 26 2024-04-26T21:55:12.099 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.095+0000 7f89c19e9740 1 -- 172.21.15.72:0/1598192717 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5578189e9c00 con 0x5578189e6c50 2024-04-26T21:55:12.099 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.095+0000 7f89a5600640 1 -- 172.21.15.72:0/1598192717 <== mon.0 v2:172.21.15.72:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f89a0010bb0 con 0x5578189e6c50 2024-04-26T21:55:12.099 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.095+0000 7f89a5600640 1 -- 172.21.15.72:0/1598192717 <== mon.0 v2:172.21.15.72:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f89b00d2ce0 con 0x5578189e6c50 2024-04-26T21:55:12.099 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.095+0000 7f89c19e9740 10 client.15306 fetch_fsmap finished waiting for FSMap version 26 2024-04-26T21:55:12.100 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.095+0000 7f89c19e9740 10 client.15306 resolve_mds: resolved name 'k' to daemon mds.k 2024-04-26T21:55:12.100 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.095+0000 7f89c19e9740 20 client.15306 populate_metadata read hostname 'smithi072' 2024-04-26T21:55:12.100 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.095+0000 7f89c19e9740 1 --2- 172.21.15.72:0/1598192717 >> [v2:172.21.15.76:6832/3869483750,v1:172.21.15.76:6833/3869483750] conn(0x5578189f82a0 0x5578189f9710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:12.100 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.095+0000 7f89c19e9740 4 client.15306 mds_command: new command op to 34406 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:55:12.100 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.095+0000 7f89c19e9740 1 -- 172.21.15.72:0/1598192717 --> [v2:172.21.15.76:6832/3869483750,v1:172.21.15.76:6833/3869483750] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557818953210 con 0x5578189f82a0 2024-04-26T21:55:12.100 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.096+0000 7f89b7e00640 1 --2- 172.21.15.72:0/1598192717 >> [v2:172.21.15.76:6832/3869483750,v1:172.21.15.76:6833/3869483750] conn(0x5578189f82a0 0x5578189f9710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:12.100 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.097+0000 7f89b7e00640 1 --2- 172.21.15.72:0/1598192717 >> [v2:172.21.15.76:6832/3869483750,v1:172.21.15.76:6833/3869483750] conn(0x5578189f82a0 0x5578189f9710 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=9485fef280342d7d server_cookie=f628a99a768ee4cb in_seq=0 out_seq=0 2024-04-26T21:55:12.100 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.097+0000 7f89a5600640 10 client.15306 ms_handle_connect on v2:172.21.15.76:6832/3869483750 2024-04-26T21:55:12.101 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.099+0000 7f89a5600640 1 -- 172.21.15.72:0/1598192717 <== mds.3 v2:172.21.15.76:6832/3869483750 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557818953210 con 0x5578189f82a0 2024-04-26T21:55:12.101 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.099+0000 7f89a5600640 10 client.15306 handle_command_reply: tid=0 2024-04-26T21:55:12.101 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.099+0000 7f89c19e9740 1 client.15306 shutdown 2024-04-26T21:55:12.101 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.099+0000 7f89c19e9740 1 -- 172.21.15.72:0/1598192717 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5578189e6c50 msgr2=0x5578189f2af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:12.102 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.099+0000 7f89c19e9740 1 --2- 172.21.15.72:0/1598192717 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5578189e6c50 0x5578189f2af0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f89b00ba6a0 tx=0x7f89b00bea00 comp rx=0 tx=0).stop 2024-04-26T21:55:12.102 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.099+0000 7f89c19e9740 1 -- 172.21.15.72:0/1598192717 shutdown_connections 2024-04-26T21:55:12.102 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.099+0000 7f89c19e9740 1 --2- 172.21.15.72:0/1598192717 >> [v2:172.21.15.76:6832/3869483750,v1:172.21.15.76:6833/3869483750] conn(0x5578189f82a0 0x5578189f9710 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:12.102 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.099+0000 7f89c19e9740 1 --2- 172.21.15.72:0/1598192717 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5578189e6c50 0x5578189f2af0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:12.102 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.099+0000 7f89c19e9740 1 --2- 172.21.15.72:0/1598192717 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5578189e1ae0 0x5578189ebab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:12.102 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.099+0000 7f89c19e9740 1 --2- 172.21.15.72:0/1598192717 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5578189e10f0 0x5578189eb570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:12.102 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.099+0000 7f89a5600640 0 client.15306 ms_handle_reset on v2:172.21.15.76:6832/3869483750 2024-04-26T21:55:12.102 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.099+0000 7f89c19e9740 1 -- 172.21.15.72:0/1598192717 >> 172.21.15.72:0/1598192717 conn(0x557818975d10 msgr2=0x5578189dbeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:55:12.102 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.099+0000 7f89c19e9740 1 -- 172.21.15.72:0/1598192717 shutdown_connections 2024-04-26T21:55:12.102 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.099+0000 7f89c19e9740 1 -- 172.21.15.72:0/1598192717 wait complete. 2024-04-26T21:55:12.102 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.100+0000 7f89c19e9740 20 client.15306 trim_cache size 0 max 16384 2024-04-26T21:55:12.120 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.119+0000 7f89c19e9740 1 Processor -- start 2024-04-26T21:55:12.120 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.119+0000 7f89c19e9740 1 -- start start 2024-04-26T21:55:12.120 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.120+0000 7f89c19e9740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5578189e10f0 0x5578189e6570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:12.120 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.120+0000 7f89c19e9740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5578189e1ae0 0x5578189f2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:12.120 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.120+0000 7f89c19e9740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5578189e6c50 0x5578189f7360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:12.120 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.120+0000 7f89c19e9740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x5578189f7a50 con 0x5578189e6c50 2024-04-26T21:55:12.120 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.120+0000 7f89c19e9740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5578189f8120 con 0x5578189e1ae0 2024-04-26T21:55:12.120 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.120+0000 7f89c19e9740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5578189f87f0 con 0x5578189e10f0 2024-04-26T21:55:12.120 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.120+0000 7f89bca00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5578189e6c50 0x5578189f7360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:12.120 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.120+0000 7f89bca00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5578189e6c50 0x5578189f7360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:34182/0 (socket says 172.21.15.72:34182) 2024-04-26T21:55:12.121 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.120+0000 7f89bca00640 1 -- 172.21.15.72:0/745885321 learned_addr learned my addr 172.21.15.72:0/745885321 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:55:12.125 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.120+0000 7f89b7e00640 1 --2- 172.21.15.72:0/745885321 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5578189e10f0 0x5578189e6570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:12.125 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.120+0000 7f89bca00640 1 -- 172.21.15.72:0/745885321 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5578189e10f0 msgr2=0x5578189e6570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:55:12.125 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.120+0000 7f89bca00640 1 --2- 172.21.15.72:0/745885321 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5578189e10f0 0x5578189e6570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:12.125 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.120+0000 7f89afe00640 1 --2- 172.21.15.72:0/745885321 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5578189e1ae0 0x5578189f2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:12.125 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.120+0000 7f89bca00640 1 -- 172.21.15.72:0/745885321 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5578189e1ae0 msgr2=0x5578189f2ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:55:12.125 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.120+0000 7f89bca00640 1 --2- 172.21.15.72:0/745885321 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5578189e1ae0 0x5578189f2ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:12.125 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.120+0000 7f89bca00640 1 -- 172.21.15.72:0/745885321 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5578189e0a40 con 0x5578189e6c50 2024-04-26T21:55:12.125 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.121+0000 7f89bca00640 1 --2- 172.21.15.72:0/745885321 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5578189e6c50 0x5578189f7360 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f89b00f22f0 tx=0x7f89b0005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6912a2dce8f27e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:55:12.125 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.121+0000 7f89a7e00640 1 -- 172.21.15.72:0/745885321 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89b00c6eb0 con 0x5578189e6c50 2024-04-26T21:55:12.125 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.121+0000 7f89a7e00640 1 -- 172.21.15.72:0/745885321 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f89b0036020 con 0x5578189e6c50 2024-04-26T21:55:12.125 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.121+0000 7f89a7e00640 1 -- 172.21.15.72:0/745885321 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89b00ba8b0 con 0x5578189e6c50 2024-04-26T21:55:12.125 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.121+0000 7f89c19e9740 1 -- 172.21.15.72:0/745885321 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5578189e6c50 msgr2=0x5578189f7360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:12.126 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.121+0000 7f89c19e9740 1 --2- 172.21.15.72:0/745885321 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5578189e6c50 0x5578189f7360 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f89b00f22f0 tx=0x7f89b0005ef0 comp rx=0 tx=0).stop 2024-04-26T21:55:12.126 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.122+0000 7f89c19e9740 1 -- 172.21.15.72:0/745885321 shutdown_connections 2024-04-26T21:55:12.126 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.122+0000 7f89c19e9740 1 --2- 172.21.15.72:0/745885321 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5578189e6c50 0x5578189f7360 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:12.126 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.122+0000 7f89c19e9740 1 --2- 172.21.15.72:0/745885321 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5578189e1ae0 0x5578189f2ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:12.126 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.122+0000 7f89c19e9740 1 --2- 172.21.15.72:0/745885321 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5578189e10f0 0x5578189e6570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:12.126 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.122+0000 7f89c19e9740 1 -- 172.21.15.72:0/745885321 >> 172.21.15.72:0/745885321 conn(0x557818975d10 msgr2=0x5578189f6c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:55:12.126 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.122+0000 7f89c19e9740 1 -- 172.21.15.72:0/745885321 shutdown_connections 2024-04-26T21:55:12.126 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.122+0000 7f89c19e9740 1 -- 172.21.15.72:0/745885321 wait complete. 2024-04-26T21:55:12.126 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.122+0000 7f89c19e9740 1 -- start start 2024-04-26T21:55:12.126 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.122+0000 7f89c19e9740 1 Processor -- start 2024-04-26T21:55:12.126 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.123+0000 7f89c19e9740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5578189e10f0 0x5578189f2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:12.126 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.123+0000 7f89c19e9740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5578189e1ae0 0x5578189f2940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:12.126 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.123+0000 7f89c19e9740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5578189e6c50 0x5578189eeea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:12.126 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.123+0000 7f89c19e9740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x5578189f9590 con 0x5578189e6c50 2024-04-26T21:55:12.127 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.123+0000 7f89c19e9740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5578189f8ec0 con 0x5578189e1ae0 2024-04-26T21:55:12.127 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.123+0000 7f89c19e9740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5578189f9c60 con 0x5578189e10f0 2024-04-26T21:55:12.127 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.123+0000 7f89c19e9740 10 client.0 _ll_register_callbacks cb 0x7f89b00095c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:55:12.127 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.123+0000 7f89bca00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5578189e6c50 0x5578189eeea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:12.127 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.123+0000 7f89bca00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5578189e6c50 0x5578189eeea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:34188/0 (socket says 172.21.15.72:34188) 2024-04-26T21:55:12.127 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.123+0000 7f89bca00640 1 -- 172.21.15.72:0/1013358014 learned_addr learned my addr 172.21.15.72:0/1013358014 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:55:12.127 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.123+0000 7f89b7e00640 1 --2- 172.21.15.72:0/1013358014 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5578189e10f0 0x5578189f2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:12.127 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.123+0000 7f89afe00640 1 --2- 172.21.15.72:0/1013358014 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5578189e1ae0 0x5578189f2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:12.127 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.123+0000 7f89bca00640 1 -- 172.21.15.72:0/1013358014 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5578189e10f0 msgr2=0x5578189f2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:55:12.127 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.123+0000 7f89bca00640 1 --2- 172.21.15.72:0/1013358014 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5578189e10f0 0x5578189f2400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:12.127 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.123+0000 7f89bca00640 1 -- 172.21.15.72:0/1013358014 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5578189e1ae0 msgr2=0x5578189f2940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:55:12.127 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.123+0000 7f89bca00640 1 --2- 172.21.15.72:0/1013358014 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5578189e1ae0 0x5578189f2940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:12.127 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.123+0000 7f89bca00640 1 -- 172.21.15.72:0/1013358014 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5578189ef630 con 0x5578189e6c50 2024-04-26T21:55:12.127 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.123+0000 7f89afe00640 1 --2- 172.21.15.72:0/1013358014 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5578189e1ae0 0x5578189f2940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:55:12.128 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.124+0000 7f89bca00640 1 --2- 172.21.15.72:0/1013358014 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5578189e6c50 0x5578189eeea0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f89b00bdd10 tx=0x7f89b00bdeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1cc22655a9ea568 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:55:12.128 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.124+0000 7f89a5600640 10 client.0 ms_handle_connect on v2:172.21.15.72:3300/0 2024-04-26T21:55:12.128 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.124+0000 7f89a5600640 1 -- 172.21.15.72:0/1013358014 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89b00be4a0 con 0x5578189e6c50 2024-04-26T21:55:12.128 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.124+0000 7f89a5600640 1 -- 172.21.15.72:0/1013358014 <== mon.0 v2:172.21.15.72:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7f89b00a7530 con 0x5578189e6c50 2024-04-26T21:55:12.128 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.124+0000 7f89c19e9740 1 -- 172.21.15.72:0/1013358014 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f89b0036020 con 0x5578189e6c50 2024-04-26T21:55:12.128 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.124+0000 7f89a5600640 1 -- 172.21.15.72:0/1013358014 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f89b00be640 con 0x5578189e6c50 2024-04-26T21:55:12.128 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.124+0000 7f89c19e9740 10 client.15312 fetch_fsmap learned FSMap version 26 2024-04-26T21:55:12.128 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.124+0000 7f89c19e9740 1 -- 172.21.15.72:0/1013358014 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5578189ef630 con 0x5578189e6c50 2024-04-26T21:55:12.128 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.124+0000 7f89a5600640 1 -- 172.21.15.72:0/1013358014 <== mon.0 v2:172.21.15.72:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f89b0036020 con 0x5578189e6c50 2024-04-26T21:55:12.128 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.124+0000 7f89a5600640 1 -- 172.21.15.72:0/1013358014 <== mon.0 v2:172.21.15.72:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f89b00a3070 con 0x5578189e6c50 2024-04-26T21:55:12.128 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.125+0000 7f89c19e9740 10 client.15312 fetch_fsmap finished waiting for FSMap version 26 2024-04-26T21:55:12.128 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.125+0000 7f89c19e9740 10 client.15312 resolve_mds: resolved name 'k' to daemon mds.k 2024-04-26T21:55:12.128 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.125+0000 7f89c19e9740 20 client.15312 populate_metadata read hostname 'smithi072' 2024-04-26T21:55:12.128 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.125+0000 7f89c19e9740 1 --2- 172.21.15.72:0/1013358014 >> [v2:172.21.15.76:6832/3869483750,v1:172.21.15.76:6833/3869483750] conn(0x557818a63500 0x557818a62210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:12.128 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.125+0000 7f89c19e9740 4 client.15312 mds_command: new command op to 34406 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T21:55:12.129 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.125+0000 7f89c19e9740 1 -- 172.21.15.72:0/1013358014 --> [v2:172.21.15.76:6832/3869483750,v1:172.21.15.76:6833/3869483750] -- command(tid 0: {"prefix": "status"}) -- 0x55781894d080 con 0x557818a63500 2024-04-26T21:55:12.130 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.125+0000 7f89b7e00640 1 --2- 172.21.15.72:0/1013358014 >> [v2:172.21.15.76:6832/3869483750,v1:172.21.15.76:6833/3869483750] conn(0x557818a63500 0x557818a62210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:12.130 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.126+0000 7f89b7e00640 1 --2- 172.21.15.72:0/1013358014 >> [v2:172.21.15.76:6832/3869483750,v1:172.21.15.76:6833/3869483750] conn(0x557818a63500 0x557818a62210 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=1ef605e0d8203773 server_cookie=b5cf025e7c0fd27f in_seq=0 out_seq=0 2024-04-26T21:55:12.130 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.126+0000 7f89a5600640 10 client.15312 ms_handle_connect on v2:172.21.15.76:6832/3869483750 2024-04-26T21:55:12.130 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.127+0000 7f89a5600640 1 -- 172.21.15.72:0/1013358014 <== mds.3 v2:172.21.15.76:6832/3869483750 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55781894d080 con 0x557818a63500 2024-04-26T21:55:12.130 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.127+0000 7f89a5600640 10 client.15312 handle_command_reply: tid=0 2024-04-26T21:55:12.130 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.127+0000 7f89c19e9740 1 client.15312 shutdown 2024-04-26T21:55:12.130 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.127+0000 7f89c19e9740 1 -- 172.21.15.72:0/1013358014 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5578189e6c50 msgr2=0x5578189eeea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:12.131 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.127+0000 7f89c19e9740 1 --2- 172.21.15.72:0/1013358014 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5578189e6c50 0x5578189eeea0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f89b00bdd10 tx=0x7f89b00bdeb0 comp rx=0 tx=0).stop 2024-04-26T21:55:12.131 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.127+0000 7f89c19e9740 1 -- 172.21.15.72:0/1013358014 shutdown_connections 2024-04-26T21:55:12.131 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.127+0000 7f89c19e9740 1 --2- 172.21.15.72:0/1013358014 >> [v2:172.21.15.76:6832/3869483750,v1:172.21.15.76:6833/3869483750] conn(0x557818a63500 0x557818a62210 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:12.131 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.127+0000 7f89c19e9740 1 --2- 172.21.15.72:0/1013358014 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5578189e6c50 0x5578189eeea0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:12.131 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.127+0000 7f89c19e9740 1 --2- 172.21.15.72:0/1013358014 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5578189e1ae0 0x5578189f2940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:12.131 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.127+0000 7f89c19e9740 1 --2- 172.21.15.72:0/1013358014 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5578189e10f0 0x5578189f2400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:12.131 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.127+0000 7f89a5600640 0 client.15312 ms_handle_reset on v2:172.21.15.76:6832/3869483750 2024-04-26T21:55:12.131 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.127+0000 7f89c19e9740 1 -- 172.21.15.72:0/1013358014 >> 172.21.15.72:0/1013358014 conn(0x557818975d10 msgr2=0x5578189dfa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:55:12.131 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.128+0000 7f89c19e9740 1 -- 172.21.15.72:0/1013358014 shutdown_connections 2024-04-26T21:55:12.131 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.128+0000 7f89c19e9740 1 -- 172.21.15.72:0/1013358014 wait complete. 2024-04-26T21:55:12.131 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.128+0000 7f89c19e9740 20 client.15312 trim_cache size 0 max 16384 2024-04-26T21:55:12.131 INFO:teuthology.orchestra.run.smithi072.stdout:{ 2024-04-26T21:55:12.132 INFO:teuthology.orchestra.run.smithi072.stdout: "cluster_fsid": "a1780950-0415-11ef-bc93-c7b262605968", 2024-04-26T21:55:12.132 INFO:teuthology.orchestra.run.smithi072.stdout: "whoami": 3, 2024-04-26T21:55:12.132 INFO:teuthology.orchestra.run.smithi072.stdout: "id": 34406, 2024-04-26T21:55:12.132 INFO:teuthology.orchestra.run.smithi072.stdout: "want_state": "up:active", 2024-04-26T21:55:12.132 INFO:teuthology.orchestra.run.smithi072.stdout: "state": "up:active", 2024-04-26T21:55:12.132 INFO:teuthology.orchestra.run.smithi072.stdout: "fs_name": "cephfs", 2024-04-26T21:55:12.132 INFO:teuthology.orchestra.run.smithi072.stdout: "rank_uptime": 135.56684295299999, 2024-04-26T21:55:12.132 INFO:teuthology.orchestra.run.smithi072.stdout: "mdsmap_epoch": 26, 2024-04-26T21:55:12.132 INFO:teuthology.orchestra.run.smithi072.stdout: "osdmap_epoch": 91, 2024-04-26T21:55:12.132 INFO:teuthology.orchestra.run.smithi072.stdout: "osdmap_epoch_barrier": 87, 2024-04-26T21:55:12.132 INFO:teuthology.orchestra.run.smithi072.stdout: "uptime": 187.64585458600001 2024-04-26T21:55:12.132 INFO:teuthology.orchestra.run.smithi072.stdout:} 2024-04-26T21:55:12.133 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.128+0000 7f89bea00640 1 -- 172.21.15.72:0/94637082 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f899c0a3c40 msgr2=0x7f899c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:12.133 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.128+0000 7f89bea00640 1 --2- 172.21.15.72:0/94637082 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f899c0a3c40 0x7f899c0a6100 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x5578189e1ee0 tx=0x7f89a8008040 comp rx=0 tx=0).stop 2024-04-26T21:55:12.133 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.128+0000 7f89bea00640 1 -- 172.21.15.72:0/94637082 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f89b810ab60 msgr2=0x7f89b81a6640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:12.133 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.128+0000 7f89bea00640 1 --2- 172.21.15.72:0/94637082 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f89b810ab60 0x7f89b81a6640 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f89b0009dc0 tx=0x7f89b0003c90 comp rx=0 tx=0).stop 2024-04-26T21:55:12.133 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.128+0000 7f89bea00640 1 -- 172.21.15.72:0/94637082 shutdown_connections 2024-04-26T21:55:12.133 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.128+0000 7f89bea00640 1 --2- 172.21.15.72:0/94637082 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f89b810ab60 0x7f89b81a6640 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:12.133 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.128+0000 7f89bea00640 1 --2- 172.21.15.72:0/94637082 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f899c0a3c40 0x7f899c0a6100 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:12.133 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.128+0000 7f89bea00640 1 --2- 172.21.15.72:0/94637082 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f89b810a1a0 0x7f89b819f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:12.134 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.128+0000 7f89bea00640 1 --2- 172.21.15.72:0/94637082 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f89b8108fb0 0x7f89b819f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:12.134 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.128+0000 7f89bea00640 1 -- 172.21.15.72:0/94637082 >> 172.21.15.72:0/94637082 conn(0x7f89b8077f40 msgr2=0x7f89b810d4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:55:12.134 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.129+0000 7f89bea00640 1 -- 172.21.15.72:0/94637082 shutdown_connections 2024-04-26T21:55:12.134 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.129+0000 7f89bea00640 1 -- 172.21.15.72:0/94637082 wait complete. 2024-04-26T21:55:12.144 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:55:12 smithi072 ceph-mon[28138]: from='client.? 172.21.15.72:0/3957081723' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:55:12.144 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:55:12 smithi072 ceph-mon[28138]: from='client.? 172.21.15.72:0/3368602131' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:55:12.144 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:55:12 smithi072 ceph-mon[28138]: pgmap v442: 97 pgs: 97 active+clean; 633 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:55:12.147 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.e status 2024-04-26T21:55:12.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:55:12 smithi076 ceph-mon[31823]: from='client.? 172.21.15.72:0/3957081723' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:55:12.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:55:12 smithi076 ceph-mon[31823]: from='client.? 172.21.15.72:0/3368602131' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:55:12.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:55:12 smithi076 ceph-mon[31823]: pgmap v442: 97 pgs: 97 active+clean; 633 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:55:12.340 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.339+0000 7fafd9800640 1 -- 172.21.15.72:0/708434272 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fafd41019b0 msgr2=0x7fafd4101e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:12.340 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.339+0000 7fafd9800640 1 --2- 172.21.15.72:0/708434272 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fafd41019b0 0x7fafd4101e10 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fafbc009900 tx=0x7fafbc02a480 comp rx=0 tx=0).stop 2024-04-26T21:55:12.340 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.340+0000 7fafd9800640 1 -- 172.21.15.72:0/708434272 shutdown_connections 2024-04-26T21:55:12.340 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.340+0000 7fafd9800640 1 --2- 172.21.15.72:0/708434272 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fafd4102350 0x7fafd41027d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:12.340 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.340+0000 7fafd9800640 1 --2- 172.21.15.72:0/708434272 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fafd41019b0 0x7fafd4101e10 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:12.340 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.340+0000 7fafd9800640 1 --2- 172.21.15.72:0/708434272 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fafd4101050 0x7fafd4101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:12.341 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.340+0000 7fafd9800640 1 -- 172.21.15.72:0/708434272 >> 172.21.15.72:0/708434272 conn(0x7fafd40fcbb0 msgr2=0x7fafd40ff010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:55:12.341 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.340+0000 7fafd9800640 1 -- 172.21.15.72:0/708434272 shutdown_connections 2024-04-26T21:55:12.341 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.340+0000 7fafd9800640 1 -- 172.21.15.72:0/708434272 wait complete. 2024-04-26T21:55:12.342 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.341+0000 7fafd9800640 1 Processor -- start 2024-04-26T21:55:12.342 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.341+0000 7fafd9800640 1 -- start start 2024-04-26T21:55:12.342 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.341+0000 7fafd9800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fafd4101050 0x7fafd419ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:12.342 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.341+0000 7fafd9800640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fafd41019b0 0x7fafd419b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:12.343 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.342+0000 7fafd9800640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fafd4102350 0x7fafd41a2110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:12.343 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.342+0000 7fafd9800640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fafd4110390 con 0x7fafd41019b0 2024-04-26T21:55:12.343 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.342+0000 7fafd9800640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fafd4110210 con 0x7fafd4102350 2024-04-26T21:55:12.343 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.342+0000 7fafd9800640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fafd4110510 con 0x7fafd4101050 2024-04-26T21:55:12.343 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.342+0000 7fafcbe00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fafd41019b0 0x7fafd419b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:12.343 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.342+0000 7fafcbe00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fafd41019b0 0x7fafd419b0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:34204/0 (socket says 172.21.15.72:34204) 2024-04-26T21:55:12.343 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.342+0000 7fafcbe00640 1 -- 172.21.15.72:0/2407547472 learned_addr learned my addr 172.21.15.72:0/2407547472 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:55:12.343 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.342+0000 7fafd3e00640 1 --2- 172.21.15.72:0/2407547472 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fafd4101050 0x7fafd419ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:12.343 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.342+0000 7fafd8e00640 1 --2- 172.21.15.72:0/2407547472 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fafd4102350 0x7fafd41a2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:12.343 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.342+0000 7fafcbe00640 1 -- 172.21.15.72:0/2407547472 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fafd4101050 msgr2=0x7fafd419ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:12.343 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.342+0000 7fafcbe00640 1 --2- 172.21.15.72:0/2407547472 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fafd4101050 0x7fafd419ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:12.343 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.342+0000 7fafcbe00640 1 -- 172.21.15.72:0/2407547472 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fafd4102350 msgr2=0x7fafd41a2110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:12.343 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.342+0000 7fafcbe00640 1 --2- 172.21.15.72:0/2407547472 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fafd4102350 0x7fafd41a2110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:12.343 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.342+0000 7fafcbe00640 1 -- 172.21.15.72:0/2407547472 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafd41a2780 con 0x7fafd41019b0 2024-04-26T21:55:12.343 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.342+0000 7fafd3e00640 1 --2- 172.21.15.72:0/2407547472 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fafd4101050 0x7fafd419ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:55:12.344 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.343+0000 7fafcbe00640 1 --2- 172.21.15.72:0/2407547472 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fafd41019b0 0x7fafd419b0a0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fafbc009230 tx=0x7fafbc0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:55:12.344 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.343+0000 7fafd1600640 1 -- 172.21.15.72:0/2407547472 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafbc03f030 con 0x7fafd41019b0 2024-04-26T21:55:12.344 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.343+0000 7fafd1600640 1 -- 172.21.15.72:0/2407547472 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fafbc003c90 con 0x7fafd41019b0 2024-04-26T21:55:12.344 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.343+0000 7fafd1600640 1 -- 172.21.15.72:0/2407547472 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafbc035070 con 0x7fafd41019b0 2024-04-26T21:55:12.345 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.343+0000 7fafd9800640 1 -- 172.21.15.72:0/2407547472 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafd41a2a10 con 0x7fafd41019b0 2024-04-26T21:55:12.345 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.343+0000 7fafd9800640 1 -- 172.21.15.72:0/2407547472 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafd41a2f80 con 0x7fafd41019b0 2024-04-26T21:55:12.346 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.347+0000 7fafddbfc740 1 Processor -- start 2024-04-26T21:55:12.346 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.347+0000 7fafddbfc740 1 -- start start 2024-04-26T21:55:12.348 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.347+0000 7fafddbfc740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562917c342c0 0x562917c346e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:12.348 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.347+0000 7fafddbfc740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562917c34cb0 0x562917c397b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:12.348 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.347+0000 7fafddbfc740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x562917c39e20 0x562917c3c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:12.348 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.347+0000 7fafddbfc740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x562917b98600 con 0x562917c39e20 2024-04-26T21:55:12.348 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.347+0000 7fafddbfc740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x562917b35100 con 0x562917c34cb0 2024-04-26T21:55:12.348 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.347+0000 7fafddbfc740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x562917b945c0 con 0x562917c342c0 2024-04-26T21:55:12.348 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.348+0000 7fafd8e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562917c342c0 0x562917c346e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:12.348 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.348+0000 7fafd3e00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562917c34cb0 0x562917c397b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:12.348 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.348+0000 7fafd8e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562917c342c0 0x562917c346e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.72:43500/0 (socket says 172.21.15.72:43500) 2024-04-26T21:55:12.348 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.348+0000 7fafd8e00640 1 -- 172.21.15.72:0/2534512108 learned_addr learned my addr 172.21.15.72:0/2534512108 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:55:12.348 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.348+0000 7fafd1600640 1 -- 172.21.15.72:0/2407547472 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fafbc042030 con 0x7fafd41019b0 2024-04-26T21:55:12.349 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.348+0000 7fafd3e00640 1 -- 172.21.15.72:0/2534512108 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562917c342c0 msgr2=0x562917c346e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:55:12.349 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.348+0000 7fafd3e00640 1 --2- 172.21.15.72:0/2534512108 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562917c342c0 0x562917c346e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:12.349 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.348+0000 7fafd3e00640 1 -- 172.21.15.72:0/2534512108 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x562917c39e20 msgr2=0x562917c3c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:55:12.349 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.348+0000 7fafcbe00640 1 --2- 172.21.15.72:0/2534512108 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x562917c39e20 0x562917c3c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:12.350 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.349+0000 7fafd3e00640 1 --2- 172.21.15.72:0/2534512108 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x562917c39e20 0x562917c3c2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:12.350 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.349+0000 7fafd3e00640 1 -- 172.21.15.72:0/2534512108 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5629178aa2d0 con 0x562917c34cb0 2024-04-26T21:55:12.350 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.349+0000 7fafcbe00640 1 --2- 172.21.15.72:0/2534512108 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x562917c39e20 0x562917c3c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:55:12.350 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.349+0000 7fafd3e00640 1 --2- 172.21.15.72:0/2534512108 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562917c34cb0 0x562917c397b0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fafc4007760 tx=0x7fafc4007c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b981ccbb0b4baa87 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:55:12.350 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.350+0000 7fafd1600640 1 --2- 172.21.15.72:0/2407547472 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fafb80a3c40 0x7fafb80a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:12.350 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.350+0000 7fafc9600640 1 -- 172.21.15.72:0/2534512108 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafc400d950 con 0x562917c34cb0 2024-04-26T21:55:12.351 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.350+0000 7fafc9600640 1 -- 172.21.15.72:0/2534512108 <== mon.2 v2:172.21.15.76:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fafc4018070 con 0x562917c34cb0 2024-04-26T21:55:12.351 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.350+0000 7fafc9600640 1 -- 172.21.15.72:0/2534512108 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafc4013ca0 con 0x562917c34cb0 2024-04-26T21:55:12.351 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.350+0000 7fafd1600640 1 -- 172.21.15.72:0/2407547472 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7fafbc0f0e40 con 0x7fafd41019b0 2024-04-26T21:55:12.351 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.350+0000 7fafd8e00640 1 --2- 172.21.15.72:0/2407547472 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fafb80a3c40 0x7fafb80a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:12.351 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.351+0000 7fafddbfc740 1 -- 172.21.15.72:0/2534512108 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562917c34cb0 msgr2=0x562917c397b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:12.352 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.351+0000 7fafddbfc740 1 --2- 172.21.15.72:0/2534512108 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562917c34cb0 0x562917c397b0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fafc4007760 tx=0x7fafc4007c20 comp rx=0 tx=0).stop 2024-04-26T21:55:12.352 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.351+0000 7fafd8e00640 1 --2- 172.21.15.72:0/2407547472 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fafb80a3c40 0x7fafb80a6100 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x562917c35510 tx=0x7fafcc0086b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:55:12.352 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.351+0000 7fafddbfc740 1 -- 172.21.15.72:0/2534512108 shutdown_connections 2024-04-26T21:55:12.352 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.351+0000 7fafddbfc740 1 --2- 172.21.15.72:0/2534512108 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x562917c39e20 0x562917c3c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:12.352 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.351+0000 7fafddbfc740 1 --2- 172.21.15.72:0/2534512108 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562917c34cb0 0x562917c397b0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:12.352 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.351+0000 7fafddbfc740 1 --2- 172.21.15.72:0/2534512108 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562917c342c0 0x562917c346e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:12.352 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.351+0000 7fafddbfc740 1 -- 172.21.15.72:0/2534512108 >> 172.21.15.72:0/2534512108 conn(0x562917bc8d10 msgr2=0x562917c2fd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:55:12.352 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.351+0000 7fafddbfc740 1 -- 172.21.15.72:0/2534512108 shutdown_connections 2024-04-26T21:55:12.352 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.351+0000 7fafddbfc740 1 -- 172.21.15.72:0/2534512108 wait complete. 2024-04-26T21:55:12.352 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.351+0000 7fafddbfc740 1 -- start start 2024-04-26T21:55:12.355 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.352+0000 7fafddbfc740 1 Processor -- start 2024-04-26T21:55:12.355 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.352+0000 7fafddbfc740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562917c342c0 0x562917c3e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:12.355 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.352+0000 7fafddbfc740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x562917c34cb0 0x562917c3ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:12.355 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.352+0000 7fafddbfc740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562917c39e20 0x562917c45d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:12.355 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.352+0000 7fafddbfc740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x562917c3cdd0 con 0x562917c34cb0 2024-04-26T21:55:12.355 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.352+0000 7fafddbfc740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x562917b1ff70 con 0x562917c39e20 2024-04-26T21:55:12.356 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.352+0000 7fafddbfc740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x562917c3cf50 con 0x562917c342c0 2024-04-26T21:55:12.356 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.352+0000 7fafddbfc740 10 client.0 _ll_register_callbacks cb 0x562917b8e480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:55:12.356 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.352+0000 7fafcbe00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x562917c34cb0 0x562917c3ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:12.356 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.353+0000 7fafcbe00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x562917c34cb0 0x562917c3ecf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:34220/0 (socket says 172.21.15.72:34220) 2024-04-26T21:55:12.356 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.353+0000 7fafcbe00640 1 -- 172.21.15.72:0/3138208504 learned_addr learned my addr 172.21.15.72:0/3138208504 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:55:12.356 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.353+0000 7fafd3e00640 1 --2- 172.21.15.72:0/3138208504 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562917c342c0 0x562917c3e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:12.356 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.353+0000 7fafd8e00640 1 --2- 172.21.15.72:0/3138208504 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562917c39e20 0x562917c45d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:12.356 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.353+0000 7fafcbe00640 1 -- 172.21.15.72:0/3138208504 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562917c342c0 msgr2=0x562917c3e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:55:12.356 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.353+0000 7fafcbe00640 1 --2- 172.21.15.72:0/3138208504 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562917c342c0 0x562917c3e7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:12.356 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.353+0000 7fafcbe00640 1 -- 172.21.15.72:0/3138208504 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562917c39e20 msgr2=0x562917c45d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:55:12.356 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.353+0000 7fafcbe00640 1 --2- 172.21.15.72:0/3138208504 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562917c39e20 0x562917c45d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:12.356 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.353+0000 7fafcbe00640 1 -- 172.21.15.72:0/3138208504 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5629179746b0 con 0x562917c34cb0 2024-04-26T21:55:12.357 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.353+0000 7fafcbe00640 1 --2- 172.21.15.72:0/3138208504 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x562917c34cb0 0x562917c3ecf0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fafbc0c5680 tx=0x7fafbc0bc8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cc38ae9809d5bce server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:55:12.357 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.353+0000 7fafc2000640 10 client.0 ms_handle_connect on v2:172.21.15.72:3300/0 2024-04-26T21:55:12.357 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.354+0000 7fafc2000640 1 -- 172.21.15.72:0/3138208504 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafbc0d6060 con 0x562917c34cb0 2024-04-26T21:55:12.357 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.354+0000 7fafc2000640 1 -- 172.21.15.72:0/3138208504 <== mon.0 v2:172.21.15.72:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7fafbc0b95b0 con 0x562917c34cb0 2024-04-26T21:55:12.357 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.354+0000 7fafddbfc740 1 -- 172.21.15.72:0/3138208504 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fafc4018070 con 0x562917c34cb0 2024-04-26T21:55:12.357 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.354+0000 7fafc2000640 1 -- 172.21.15.72:0/3138208504 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fafbc049040 con 0x562917c34cb0 2024-04-26T21:55:12.357 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.354+0000 7fafddbfc740 10 client.15321 fetch_fsmap learned FSMap version 26 2024-04-26T21:55:12.357 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.354+0000 7fafddbfc740 1 -- 172.21.15.72:0/3138208504 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x562917c48c80 con 0x562917c34cb0 2024-04-26T21:55:12.357 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.354+0000 7fafc2000640 1 -- 172.21.15.72:0/3138208504 <== mon.0 v2:172.21.15.72:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fafc4018070 con 0x562917c34cb0 2024-04-26T21:55:12.357 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.354+0000 7fafc2000640 1 -- 172.21.15.72:0/3138208504 <== mon.0 v2:172.21.15.72:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fafbc0c5720 con 0x562917c34cb0 2024-04-26T21:55:12.357 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.355+0000 7fafddbfc740 10 client.15321 fetch_fsmap finished waiting for FSMap version 26 2024-04-26T21:55:12.357 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.355+0000 7fafddbfc740 10 client.15321 resolve_mds: resolved name 'e' to daemon mds.e 2024-04-26T21:55:12.357 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.355+0000 7fafddbfc740 20 client.15321 populate_metadata read hostname 'smithi072' 2024-04-26T21:55:12.357 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.355+0000 7fafddbfc740 1 --2- 172.21.15.72:0/3138208504 >> [v2:172.21.15.76:6834/3517429435,v1:172.21.15.76:6835/3517429435] conn(0x562917c4b300 0x562917c4c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:12.357 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.355+0000 7fafddbfc740 4 client.15321 mds_command: new command op to 34442 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:55:12.358 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.355+0000 7fafddbfc740 1 -- 172.21.15.72:0/3138208504 --> [v2:172.21.15.76:6834/3517429435,v1:172.21.15.76:6835/3517429435] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562917ba6210 con 0x562917c4b300 2024-04-26T21:55:12.358 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.355+0000 7fafd3e00640 1 --2- 172.21.15.72:0/3138208504 >> [v2:172.21.15.76:6834/3517429435,v1:172.21.15.76:6835/3517429435] conn(0x562917c4b300 0x562917c4c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:12.359 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.356+0000 7fafd3e00640 1 --2- 172.21.15.72:0/3138208504 >> [v2:172.21.15.76:6834/3517429435,v1:172.21.15.76:6835/3517429435] conn(0x562917c4b300 0x562917c4c770 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8caa8463eb0bccbd server_cookie=667ed2e586961838 in_seq=0 out_seq=0 2024-04-26T21:55:12.359 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.356+0000 7fafc2000640 10 client.15321 ms_handle_connect on v2:172.21.15.76:6834/3517429435 2024-04-26T21:55:12.359 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.358+0000 7fafc2000640 1 -- 172.21.15.72:0/3138208504 <== mds.2 v2:172.21.15.76:6834/3517429435 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562917ba6210 con 0x562917c4b300 2024-04-26T21:55:12.360 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.358+0000 7fafc2000640 10 client.15321 handle_command_reply: tid=0 2024-04-26T21:55:12.360 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.358+0000 7fafddbfc740 1 client.15321 shutdown 2024-04-26T21:55:12.360 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.359+0000 7fafddbfc740 1 -- 172.21.15.72:0/3138208504 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x562917c34cb0 msgr2=0x562917c3ecf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:12.360 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.359+0000 7fafddbfc740 1 --2- 172.21.15.72:0/3138208504 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x562917c34cb0 0x562917c3ecf0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fafbc0c5680 tx=0x7fafbc0bc8a0 comp rx=0 tx=0).stop 2024-04-26T21:55:12.360 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.359+0000 7fafddbfc740 1 -- 172.21.15.72:0/3138208504 shutdown_connections 2024-04-26T21:55:12.360 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.359+0000 7fafddbfc740 1 --2- 172.21.15.72:0/3138208504 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562917c39e20 0x562917c45d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:12.360 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.359+0000 7fafddbfc740 1 --2- 172.21.15.72:0/3138208504 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x562917c34cb0 0x562917c3ecf0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:12.360 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.359+0000 7fafddbfc740 1 --2- 172.21.15.72:0/3138208504 >> [v2:172.21.15.76:6834/3517429435,v1:172.21.15.76:6835/3517429435] conn(0x562917c4b300 0x562917c4c770 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:12.360 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.359+0000 7fafddbfc740 1 --2- 172.21.15.72:0/3138208504 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562917c342c0 0x562917c3e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:12.360 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.359+0000 7fafc2000640 0 client.15321 ms_handle_reset on v2:172.21.15.76:6834/3517429435 2024-04-26T21:55:12.360 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.359+0000 7fafddbfc740 1 -- 172.21.15.72:0/3138208504 >> 172.21.15.72:0/3138208504 conn(0x562917bc8d10 msgr2=0x562917c2ef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:55:12.360 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.359+0000 7fafddbfc740 1 -- 172.21.15.72:0/3138208504 shutdown_connections 2024-04-26T21:55:12.360 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.359+0000 7fafddbfc740 1 -- 172.21.15.72:0/3138208504 wait complete. 2024-04-26T21:55:12.360 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.359+0000 7fafddbfc740 20 client.15321 trim_cache size 0 max 16384 2024-04-26T21:55:12.396 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.395+0000 7fafddbfc740 1 Processor -- start 2024-04-26T21:55:12.396 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.395+0000 7fafddbfc740 1 -- start start 2024-04-26T21:55:12.396 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.395+0000 7fafddbfc740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562917c342c0 0x562917c53a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:12.396 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.395+0000 7fafddbfc740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562917c34cb0 0x562917c45cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:12.396 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.395+0000 7fafddbfc740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x562917c39e20 0x562917c53f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:12.396 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.395+0000 7fafddbfc740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x562917c4aab0 con 0x562917c39e20 2024-04-26T21:55:12.396 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.395+0000 7fafddbfc740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x562917c4b180 con 0x562917c34cb0 2024-04-26T21:55:12.396 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.395+0000 7fafddbfc740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x562917c4b850 con 0x562917c342c0 2024-04-26T21:55:12.396 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.395+0000 7fafd8e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x562917c39e20 0x562917c53f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:12.396 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.396+0000 7fafd8e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x562917c39e20 0x562917c53f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:34224/0 (socket says 172.21.15.72:34224) 2024-04-26T21:55:12.397 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.396+0000 7fafd8e00640 1 -- 172.21.15.72:0/3153482407 learned_addr learned my addr 172.21.15.72:0/3153482407 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:55:12.397 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.396+0000 7fafcbe00640 1 --2- 172.21.15.72:0/3153482407 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562917c34cb0 0x562917c45cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:12.397 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.396+0000 7fafd3e00640 1 --2- 172.21.15.72:0/3153482407 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562917c342c0 0x562917c53a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:12.398 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.396+0000 7fafd8e00640 1 -- 172.21.15.72:0/3153482407 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562917c342c0 msgr2=0x562917c53a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:55:12.398 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.396+0000 7fafd8e00640 1 --2- 172.21.15.72:0/3153482407 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562917c342c0 0x562917c53a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:12.398 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.396+0000 7fafd8e00640 1 -- 172.21.15.72:0/3153482407 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562917c34cb0 msgr2=0x562917c45cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:55:12.398 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.396+0000 7fafd8e00640 1 --2- 172.21.15.72:0/3153482407 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562917c34cb0 0x562917c45cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:12.398 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.396+0000 7fafd8e00640 1 -- 172.21.15.72:0/3153482407 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562917c2b680 con 0x562917c39e20 2024-04-26T21:55:12.398 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.396+0000 7fafcbe00640 1 --2- 172.21.15.72:0/3153482407 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562917c34cb0 0x562917c45cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:55:12.398 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.397+0000 7fafd8e00640 1 --2- 172.21.15.72:0/3153482407 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x562917c39e20 0x562917c53f80 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fafcc01e550 tx=0x7fafcc014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=479a6d41b88d697a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:55:12.398 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.397+0000 7fafc8c00640 1 -- 172.21.15.72:0/3153482407 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafcc01c840 con 0x562917c39e20 2024-04-26T21:55:12.398 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.397+0000 7fafc8c00640 1 -- 172.21.15.72:0/3153482407 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fafcc01ce70 con 0x562917c39e20 2024-04-26T21:55:12.398 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.397+0000 7fafc8c00640 1 -- 172.21.15.72:0/3153482407 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafcc026e60 con 0x562917c39e20 2024-04-26T21:55:12.400 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.398+0000 7fafddbfc740 1 -- 172.21.15.72:0/3153482407 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x562917c39e20 msgr2=0x562917c53f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:12.400 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.398+0000 7fafddbfc740 1 --2- 172.21.15.72:0/3153482407 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x562917c39e20 0x562917c53f80 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fafcc01e550 tx=0x7fafcc014000 comp rx=0 tx=0).stop 2024-04-26T21:55:12.400 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.398+0000 7fafddbfc740 1 -- 172.21.15.72:0/3153482407 shutdown_connections 2024-04-26T21:55:12.400 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.398+0000 7fafddbfc740 1 --2- 172.21.15.72:0/3153482407 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x562917c39e20 0x562917c53f80 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:12.400 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.398+0000 7fafddbfc740 1 --2- 172.21.15.72:0/3153482407 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562917c34cb0 0x562917c45cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:12.401 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.398+0000 7fafddbfc740 1 --2- 172.21.15.72:0/3153482407 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562917c342c0 0x562917c53a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:12.401 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.398+0000 7fafddbfc740 1 -- 172.21.15.72:0/3153482407 >> 172.21.15.72:0/3153482407 conn(0x562917bc8d10 msgr2=0x562917c3e790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:55:12.401 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.398+0000 7fafddbfc740 1 -- 172.21.15.72:0/3153482407 shutdown_connections 2024-04-26T21:55:12.401 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.398+0000 7fafddbfc740 1 -- 172.21.15.72:0/3153482407 wait complete. 2024-04-26T21:55:12.401 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.399+0000 7fafddbfc740 1 -- start start 2024-04-26T21:55:12.401 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.399+0000 7fafddbfc740 1 Processor -- start 2024-04-26T21:55:12.401 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.399+0000 7fafddbfc740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562917c342c0 0x562917c41fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:12.401 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.399+0000 7fafddbfc740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562917c34cb0 0x562917c42500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:12.401 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.399+0000 7fafddbfc740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x562917c39e20 0x562917c454d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:12.401 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.399+0000 7fafddbfc740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x562917c4c5f0 con 0x562917c39e20 2024-04-26T21:55:12.401 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.399+0000 7fafddbfc740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x562917c4bf20 con 0x562917c342c0 2024-04-26T21:55:12.401 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.399+0000 7fafddbfc740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x562917c4ccc0 con 0x562917c34cb0 2024-04-26T21:55:12.401 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.399+0000 7fafddbfc740 10 client.0 _ll_register_callbacks cb 0x7fafbc009260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:55:12.401 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.399+0000 7fafd8e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x562917c39e20 0x562917c454d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:12.401 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.400+0000 7fafd8e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x562917c39e20 0x562917c454d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:34238/0 (socket says 172.21.15.72:34238) 2024-04-26T21:55:12.402 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.400+0000 7fafd8e00640 1 -- 172.21.15.72:0/2480735780 learned_addr learned my addr 172.21.15.72:0/2480735780 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:55:12.402 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.400+0000 7fafd3e00640 1 --2- 172.21.15.72:0/2480735780 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562917c342c0 0x562917c41fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:12.402 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.400+0000 7fafcbe00640 1 --2- 172.21.15.72:0/2480735780 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562917c34cb0 0x562917c42500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:12.402 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.400+0000 7fafd8e00640 1 -- 172.21.15.72:0/2480735780 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562917c34cb0 msgr2=0x562917c42500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:55:12.402 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.400+0000 7fafd8e00640 1 --2- 172.21.15.72:0/2480735780 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562917c34cb0 0x562917c42500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:12.402 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.400+0000 7fafd8e00640 1 -- 172.21.15.72:0/2480735780 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562917c342c0 msgr2=0x562917c41fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:55:12.402 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.400+0000 7fafd8e00640 1 --2- 172.21.15.72:0/2480735780 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562917c342c0 0x562917c41fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:12.402 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.400+0000 7fafd8e00640 1 -- 172.21.15.72:0/2480735780 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562917c45ac0 con 0x562917c39e20 2024-04-26T21:55:12.402 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.400+0000 7fafd8e00640 1 --2- 172.21.15.72:0/2480735780 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x562917c39e20 0x562917c454d0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fafcc01e520 tx=0x7fafcc026620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=540d29be882be961 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:55:12.402 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.400+0000 7fafc2000640 10 client.0 ms_handle_connect on v2:172.21.15.72:3300/0 2024-04-26T21:55:12.404 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.401+0000 7fafc2000640 1 -- 172.21.15.72:0/2480735780 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafcc035020 con 0x562917c39e20 2024-04-26T21:55:12.404 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.401+0000 7fafddbfc740 1 -- 172.21.15.72:0/2480735780 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fafcc01ce70 con 0x562917c39e20 2024-04-26T21:55:12.404 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.401+0000 7fafc2000640 1 -- 172.21.15.72:0/2480735780 <== mon.0 v2:172.21.15.72:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7fafcc004c50 con 0x562917c39e20 2024-04-26T21:55:12.405 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.401+0000 7fafc2000640 1 -- 172.21.15.72:0/2480735780 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fafcc040050 con 0x562917c39e20 2024-04-26T21:55:12.405 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.401+0000 7fafddbfc740 10 client.15327 fetch_fsmap learned FSMap version 26 2024-04-26T21:55:12.405 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.401+0000 7fafddbfc740 1 -- 172.21.15.72:0/2480735780 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x562917cba5e0 con 0x562917c39e20 2024-04-26T21:55:12.405 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.401+0000 7fafc2000640 1 -- 172.21.15.72:0/2480735780 <== mon.0 v2:172.21.15.72:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fafcc01ce70 con 0x562917c39e20 2024-04-26T21:55:12.405 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.401+0000 7fafc2000640 1 -- 172.21.15.72:0/2480735780 <== mon.0 v2:172.21.15.72:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fafcc025030 con 0x562917c39e20 2024-04-26T21:55:12.405 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.401+0000 7fafddbfc740 10 client.15327 fetch_fsmap finished waiting for FSMap version 26 2024-04-26T21:55:12.405 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.401+0000 7fafddbfc740 10 client.15327 resolve_mds: resolved name 'e' to daemon mds.e 2024-04-26T21:55:12.405 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.401+0000 7fafddbfc740 20 client.15327 populate_metadata read hostname 'smithi072' 2024-04-26T21:55:12.405 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.401+0000 7fafddbfc740 1 --2- 172.21.15.72:0/2480735780 >> [v2:172.21.15.76:6834/3517429435,v1:172.21.15.76:6835/3517429435] conn(0x562917cbef40 0x562917cbdc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:12.405 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.401+0000 7fafddbfc740 4 client.15327 mds_command: new command op to 34442 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T21:55:12.405 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.401+0000 7fafddbfc740 1 -- 172.21.15.72:0/2480735780 --> [v2:172.21.15.76:6834/3517429435,v1:172.21.15.76:6835/3517429435] -- command(tid 0: {"prefix": "status"}) -- 0x562917ba0080 con 0x562917cbef40 2024-04-26T21:55:12.405 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.402+0000 7fafd3e00640 1 --2- 172.21.15.72:0/2480735780 >> [v2:172.21.15.76:6834/3517429435,v1:172.21.15.76:6835/3517429435] conn(0x562917cbef40 0x562917cbdc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:12.405 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.403+0000 7fafd3e00640 1 --2- 172.21.15.72:0/2480735780 >> [v2:172.21.15.76:6834/3517429435,v1:172.21.15.76:6835/3517429435] conn(0x562917cbef40 0x562917cbdc50 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2dc602c58ea88fd7 server_cookie=687ed88d1d7c7330 in_seq=0 out_seq=0 2024-04-26T21:55:12.405 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.403+0000 7fafc2000640 10 client.15327 ms_handle_connect on v2:172.21.15.76:6834/3517429435 2024-04-26T21:55:12.406 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.404+0000 7fafc2000640 1 -- 172.21.15.72:0/2480735780 <== mds.2 v2:172.21.15.76:6834/3517429435 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x562917ba0080 con 0x562917cbef40 2024-04-26T21:55:12.406 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.404+0000 7fafc2000640 10 client.15327 handle_command_reply: tid=0 2024-04-26T21:55:12.406 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.404+0000 7fafddbfc740 1 client.15327 shutdown 2024-04-26T21:55:12.406 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.404+0000 7fafddbfc740 1 -- 172.21.15.72:0/2480735780 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x562917c39e20 msgr2=0x562917c454d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:12.406 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.404+0000 7fafddbfc740 1 --2- 172.21.15.72:0/2480735780 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x562917c39e20 0x562917c454d0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fafcc01e520 tx=0x7fafcc026620 comp rx=0 tx=0).stop 2024-04-26T21:55:12.406 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.404+0000 7fafddbfc740 1 -- 172.21.15.72:0/2480735780 shutdown_connections 2024-04-26T21:55:12.406 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.404+0000 7fafddbfc740 1 --2- 172.21.15.72:0/2480735780 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x562917c39e20 0x562917c454d0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:12.406 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.404+0000 7fafddbfc740 1 --2- 172.21.15.72:0/2480735780 >> [v2:172.21.15.76:6834/3517429435,v1:172.21.15.76:6835/3517429435] conn(0x562917cbef40 0x562917cbdc50 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:12.406 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.404+0000 7fafddbfc740 1 --2- 172.21.15.72:0/2480735780 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562917c34cb0 0x562917c42500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:12.406 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.404+0000 7fafddbfc740 1 --2- 172.21.15.72:0/2480735780 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562917c342c0 0x562917c41fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:12.406 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.404+0000 7fafddbfc740 1 -- 172.21.15.72:0/2480735780 >> 172.21.15.72:0/2480735780 conn(0x562917bc8d10 msgr2=0x562917c32cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:55:12.406 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.404+0000 7fafc2000640 0 client.15327 ms_handle_reset on v2:172.21.15.76:6834/3517429435 2024-04-26T21:55:12.406 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.404+0000 7fafddbfc740 1 -- 172.21.15.72:0/2480735780 shutdown_connections 2024-04-26T21:55:12.406 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.404+0000 7fafddbfc740 1 -- 172.21.15.72:0/2480735780 wait complete. 2024-04-26T21:55:12.406 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.405+0000 7fafddbfc740 20 client.15327 trim_cache size 0 max 16384 2024-04-26T21:55:12.407 INFO:teuthology.orchestra.run.smithi072.stdout:{ 2024-04-26T21:55:12.407 INFO:teuthology.orchestra.run.smithi072.stdout: "cluster_fsid": "a1780950-0415-11ef-bc93-c7b262605968", 2024-04-26T21:55:12.407 INFO:teuthology.orchestra.run.smithi072.stdout: "whoami": 2, 2024-04-26T21:55:12.407 INFO:teuthology.orchestra.run.smithi072.stdout: "id": 34442, 2024-04-26T21:55:12.407 INFO:teuthology.orchestra.run.smithi072.stdout: "want_state": "up:active", 2024-04-26T21:55:12.407 INFO:teuthology.orchestra.run.smithi072.stdout: "state": "up:active", 2024-04-26T21:55:12.407 INFO:teuthology.orchestra.run.smithi072.stdout: "fs_name": "cephfs", 2024-04-26T21:55:12.407 INFO:teuthology.orchestra.run.smithi072.stdout: "rank_uptime": 136.86476733999999, 2024-04-26T21:55:12.408 INFO:teuthology.orchestra.run.smithi072.stdout: "mdsmap_epoch": 26, 2024-04-26T21:55:12.408 INFO:teuthology.orchestra.run.smithi072.stdout: "osdmap_epoch": 91, 2024-04-26T21:55:12.408 INFO:teuthology.orchestra.run.smithi072.stdout: "osdmap_epoch_barrier": 87, 2024-04-26T21:55:12.408 INFO:teuthology.orchestra.run.smithi072.stdout: "uptime": 175.90117619099999 2024-04-26T21:55:12.408 INFO:teuthology.orchestra.run.smithi072.stdout:} 2024-04-26T21:55:12.408 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.405+0000 7fafd9800640 1 -- 172.21.15.72:0/2407547472 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fafb80a3c40 msgr2=0x7fafb80a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:12.409 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.405+0000 7fafd9800640 1 --2- 172.21.15.72:0/2407547472 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fafb80a3c40 0x7fafb80a6100 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x562917c35510 tx=0x7fafcc0086b0 comp rx=0 tx=0).stop 2024-04-26T21:55:12.409 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.405+0000 7fafd9800640 1 -- 172.21.15.72:0/2407547472 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fafd41019b0 msgr2=0x7fafd419b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:12.409 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.405+0000 7fafd9800640 1 --2- 172.21.15.72:0/2407547472 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fafd41019b0 0x7fafd419b0a0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fafbc009230 tx=0x7fafbc0042f0 comp rx=0 tx=0).stop 2024-04-26T21:55:12.409 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.406+0000 7fafd9800640 1 -- 172.21.15.72:0/2407547472 shutdown_connections 2024-04-26T21:55:12.409 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.406+0000 7fafd9800640 1 --2- 172.21.15.72:0/2407547472 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fafd4102350 0x7fafd41a2110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:12.409 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.406+0000 7fafd9800640 1 --2- 172.21.15.72:0/2407547472 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fafd41019b0 0x7fafd419b0a0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:12.409 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.406+0000 7fafd9800640 1 --2- 172.21.15.72:0/2407547472 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fafb80a3c40 0x7fafb80a6100 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:12.409 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.406+0000 7fafd9800640 1 --2- 172.21.15.72:0/2407547472 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fafd4101050 0x7fafd419ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:12.409 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.406+0000 7fafd9800640 1 -- 172.21.15.72:0/2407547472 >> 172.21.15.72:0/2407547472 conn(0x7fafd40fcbb0 msgr2=0x7fafd40fff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:55:12.409 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.406+0000 7fafd9800640 1 -- 172.21.15.72:0/2407547472 shutdown_connections 2024-04-26T21:55:12.409 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.406+0000 7fafd9800640 1 -- 172.21.15.72:0/2407547472 wait complete. 2024-04-26T21:55:12.424 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2024-04-26T21:55:12.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:55:12 smithi164 ceph-mon[31916]: from='client.? 172.21.15.72:0/3957081723' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:55:12.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:55:12 smithi164 ceph-mon[31916]: from='client.? 172.21.15.72:0/3368602131' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:55:12.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:55:12 smithi164 ceph-mon[31916]: pgmap v442: 97 pgs: 97 active+clean; 633 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:55:12.621 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.621+0000 7f825be00640 1 -- 172.21.15.72:0/2206276062 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f825c103960 msgr2=0x7f825c103de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:12.622 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.621+0000 7f825be00640 1 --2- 172.21.15.72:0/2206276062 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f825c103960 0x7f825c103de0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f8244009900 tx=0x7f824402a410 comp rx=0 tx=0).stop 2024-04-26T21:55:12.622 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.621+0000 7f825be00640 1 -- 172.21.15.72:0/2206276062 shutdown_connections 2024-04-26T21:55:12.622 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.621+0000 7f825be00640 1 --2- 172.21.15.72:0/2206276062 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f825c104320 0x7f825c10abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:12.622 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.621+0000 7f825be00640 1 --2- 172.21.15.72:0/2206276062 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f825c103960 0x7f825c103de0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:12.622 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.621+0000 7f825be00640 1 --2- 172.21.15.72:0/2206276062 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f825c102770 0x7f825c102b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:12.622 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.621+0000 7f825be00640 1 -- 172.21.15.72:0/2206276062 >> 172.21.15.72:0/2206276062 conn(0x7f825c0fdce0 msgr2=0x7f825c100120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:55:12.622 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.621+0000 7f825be00640 1 -- 172.21.15.72:0/2206276062 shutdown_connections 2024-04-26T21:55:12.622 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.621+0000 7f825be00640 1 -- 172.21.15.72:0/2206276062 wait complete. 2024-04-26T21:55:12.624 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.622+0000 7f825be00640 1 Processor -- start 2024-04-26T21:55:12.624 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.622+0000 7f825be00640 1 -- start start 2024-04-26T21:55:12.624 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.623+0000 7f825be00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f825c102770 0x7f825c19aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:12.624 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.623+0000 7f825be00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f825c103960 0x7f825c19b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:12.625 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.623+0000 7f825be00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f825c104320 0x7f825c1a2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:12.625 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.623+0000 7f825be00640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f825c10e1a0 con 0x7f825c104320 2024-04-26T21:55:12.625 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.623+0000 7f825be00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f825c10e020 con 0x7f825c102770 2024-04-26T21:55:12.625 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.623+0000 7f825be00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f825c10e320 con 0x7f825c103960 2024-04-26T21:55:12.625 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.623+0000 7f825b400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f825c104320 0x7f825c1a2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:12.625 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.623+0000 7f825b400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f825c104320 0x7f825c1a2090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:34262/0 (socket says 172.21.15.72:34262) 2024-04-26T21:55:12.625 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.623+0000 7f825b400640 1 -- 172.21.15.72:0/215230690 learned_addr learned my addr 172.21.15.72:0/215230690 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:55:12.625 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.623+0000 7f825aa00640 1 --2- 172.21.15.72:0/215230690 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f825c102770 0x7f825c19aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:12.625 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.623+0000 7f8252000640 1 --2- 172.21.15.72:0/215230690 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f825c103960 0x7f825c19b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:12.625 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.623+0000 7f825b400640 1 -- 172.21.15.72:0/215230690 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f825c103960 msgr2=0x7f825c19b020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:12.625 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.623+0000 7f825b400640 1 --2- 172.21.15.72:0/215230690 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f825c103960 0x7f825c19b020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:12.625 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.623+0000 7f825b400640 1 -- 172.21.15.72:0/215230690 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f825c102770 msgr2=0x7f825c19aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:12.625 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.623+0000 7f825b400640 1 --2- 172.21.15.72:0/215230690 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f825c102770 0x7f825c19aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:12.625 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.623+0000 7f825b400640 1 -- 172.21.15.72:0/215230690 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f825c1a2700 con 0x7f825c104320 2024-04-26T21:55:12.626 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.623+0000 7f8252000640 1 --2- 172.21.15.72:0/215230690 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f825c103960 0x7f825c19b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:55:12.626 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.624+0000 7f825b400640 1 --2- 172.21.15.72:0/215230690 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f825c104320 0x7f825c1a2090 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f825400eaa0 tx=0x7f825400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:55:12.626 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.624+0000 7f8253e00640 1 -- 172.21.15.72:0/215230690 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f825400cdb0 con 0x7f825c104320 2024-04-26T21:55:12.626 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.624+0000 7f8253e00640 1 -- 172.21.15.72:0/215230690 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8254005020 con 0x7f825c104320 2024-04-26T21:55:12.626 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.624+0000 7f8253e00640 1 -- 172.21.15.72:0/215230690 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82540116d0 con 0x7f825c104320 2024-04-26T21:55:12.626 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.624+0000 7f825be00640 1 -- 172.21.15.72:0/215230690 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f825c1a29f0 con 0x7f825c104320 2024-04-26T21:55:12.626 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.624+0000 7f825be00640 1 -- 172.21.15.72:0/215230690 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f825c1a2ef0 con 0x7f825c104320 2024-04-26T21:55:12.629 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.628+0000 7f8253e00640 1 -- 172.21.15.72:0/215230690 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f82540051d0 con 0x7f825c104320 2024-04-26T21:55:12.630 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.628+0000 7f82649f5740 1 Processor -- start 2024-04-26T21:55:12.630 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.628+0000 7f82649f5740 1 -- start start 2024-04-26T21:55:12.630 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.628+0000 7f82649f5740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5637d573e2c0 0x5637d573e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:12.630 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.628+0000 7f82649f5740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5637d573ecb0 0x5637d57437b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:12.630 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.628+0000 7f82649f5740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5637d5743e20 0x5637d57462b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:12.630 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.628+0000 7f82649f5740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x5637d56a2600 con 0x5637d573ecb0 2024-04-26T21:55:12.630 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.628+0000 7f82649f5740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5637d563f100 con 0x5637d573e2c0 2024-04-26T21:55:12.630 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.628+0000 7f82649f5740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5637d569e5c0 con 0x5637d5743e20 2024-04-26T21:55:12.630 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.628+0000 7f825aa00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5637d573ecb0 0x5637d57437b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:12.630 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.628+0000 7f825aa00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5637d573ecb0 0x5637d57437b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:34272/0 (socket says 172.21.15.72:34272) 2024-04-26T21:55:12.630 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.628+0000 7f825aa00640 1 -- 172.21.15.72:0/110743378 learned_addr learned my addr 172.21.15.72:0/110743378 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:55:12.630 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.629+0000 7f8252000640 1 --2- 172.21.15.72:0/110743378 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5637d573e2c0 0x5637d573e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:12.630 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.629+0000 7f8252000640 1 --2- 172.21.15.72:0/110743378 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5637d5743e20 0x5637d57462b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:12.631 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.629+0000 7f825aa00640 1 -- 172.21.15.72:0/110743378 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5637d5743e20 msgr2=0x5637d57462b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:55:12.631 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.629+0000 7f825aa00640 1 --2- 172.21.15.72:0/110743378 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5637d5743e20 0x5637d57462b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:12.631 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.629+0000 7f825aa00640 1 -- 172.21.15.72:0/110743378 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5637d573e2c0 msgr2=0x5637d573e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:55:12.631 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.629+0000 7f825aa00640 1 --2- 172.21.15.72:0/110743378 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5637d573e2c0 0x5637d573e6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:12.631 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.629+0000 7f825aa00640 1 -- 172.21.15.72:0/110743378 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5637d53b42d0 con 0x5637d573ecb0 2024-04-26T21:55:12.631 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.629+0000 7f8252000640 1 --2- 172.21.15.72:0/110743378 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5637d5743e20 0x5637d57462b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:55:12.631 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.629+0000 7f8253e00640 1 --2- 172.21.15.72:0/215230690 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f82400a3c40 0x7f82400a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:12.631 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.629+0000 7f825aa00640 1 --2- 172.21.15.72:0/110743378 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5637d573ecb0 0x5637d57437b0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f824c00bc80 tx=0x7f824c00f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e83d1a8c66435278 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:55:12.631 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.629+0000 7f8253e00640 1 -- 172.21.15.72:0/215230690 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7f82540d08e0 con 0x7f825c104320 2024-04-26T21:55:12.631 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.629+0000 7f8252000640 1 --2- 172.21.15.72:0/215230690 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f82400a3c40 0x7f82400a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:12.631 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.630+0000 7f824be00640 1 -- 172.21.15.72:0/110743378 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f824c015070 con 0x5637d573ecb0 2024-04-26T21:55:12.631 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.630+0000 7f824be00640 1 -- 172.21.15.72:0/110743378 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f824c01a8f0 con 0x5637d573ecb0 2024-04-26T21:55:12.631 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.630+0000 7f824be00640 1 -- 172.21.15.72:0/110743378 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f824c011800 con 0x5637d573ecb0 2024-04-26T21:55:12.635 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.630+0000 7f82649f5740 1 -- 172.21.15.72:0/110743378 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5637d573ecb0 msgr2=0x5637d57437b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:12.636 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.630+0000 7f82649f5740 1 --2- 172.21.15.72:0/110743378 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5637d573ecb0 0x5637d57437b0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f824c00bc80 tx=0x7f824c00f730 comp rx=0 tx=0).stop 2024-04-26T21:55:12.636 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.630+0000 7f8252000640 1 --2- 172.21.15.72:0/215230690 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f82400a3c40 0x7f82400a6100 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f82440096c0 tx=0x7f8244000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:55:12.636 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.630+0000 7f82649f5740 1 -- 172.21.15.72:0/110743378 shutdown_connections 2024-04-26T21:55:12.636 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.630+0000 7f82649f5740 1 --2- 172.21.15.72:0/110743378 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5637d5743e20 0x5637d57462b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:12.636 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.630+0000 7f82649f5740 1 --2- 172.21.15.72:0/110743378 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5637d573ecb0 0x5637d57437b0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:12.636 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.630+0000 7f82649f5740 1 --2- 172.21.15.72:0/110743378 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5637d573e2c0 0x5637d573e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:12.636 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.631+0000 7f82649f5740 1 -- 172.21.15.72:0/110743378 >> 172.21.15.72:0/110743378 conn(0x5637d56d2d10 msgr2=0x5637d5739d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:55:12.636 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.631+0000 7f82649f5740 1 -- 172.21.15.72:0/110743378 shutdown_connections 2024-04-26T21:55:12.636 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.631+0000 7f82649f5740 1 -- 172.21.15.72:0/110743378 wait complete. 2024-04-26T21:55:12.636 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.631+0000 7f82649f5740 1 -- start start 2024-04-26T21:55:12.636 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.631+0000 7f82649f5740 1 Processor -- start 2024-04-26T21:55:12.636 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.632+0000 7f82649f5740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5637d573e2c0 0x5637d5748790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:12.636 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.632+0000 7f82649f5740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5637d573ecb0 0x5637d5748cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:12.637 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.632+0000 7f82649f5740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5637d5743e20 0x5637d574fca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:12.637 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.632+0000 7f82649f5740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x5637d5746dd0 con 0x5637d5743e20 2024-04-26T21:55:12.637 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.632+0000 7f82649f5740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5637d5629f70 con 0x5637d573e2c0 2024-04-26T21:55:12.637 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.632+0000 7f82649f5740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5637d5746f50 con 0x5637d573ecb0 2024-04-26T21:55:12.637 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.632+0000 7f82649f5740 10 client.0 _ll_register_callbacks cb 0x5637d5698480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:55:12.637 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.632+0000 7f825b400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5637d5743e20 0x5637d574fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:12.637 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.632+0000 7f825b400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5637d5743e20 0x5637d574fca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:34276/0 (socket says 172.21.15.72:34276) 2024-04-26T21:55:12.637 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.632+0000 7f825b400640 1 -- 172.21.15.72:0/2750978998 learned_addr learned my addr 172.21.15.72:0/2750978998 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:55:12.637 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.632+0000 7f825aa00640 1 --2- 172.21.15.72:0/2750978998 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5637d573e2c0 0x5637d5748790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:12.637 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.632+0000 7f8252000640 1 --2- 172.21.15.72:0/2750978998 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5637d573ecb0 0x5637d5748cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:12.637 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.632+0000 7f825b400640 1 -- 172.21.15.72:0/2750978998 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5637d573ecb0 msgr2=0x5637d5748cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:55:12.637 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.632+0000 7f825b400640 1 --2- 172.21.15.72:0/2750978998 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5637d573ecb0 0x5637d5748cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:12.637 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.632+0000 7f825b400640 1 -- 172.21.15.72:0/2750978998 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5637d573e2c0 msgr2=0x5637d5748790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:55:12.638 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.632+0000 7f825b400640 1 --2- 172.21.15.72:0/2750978998 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5637d573e2c0 0x5637d5748790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:12.638 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.632+0000 7f825b400640 1 -- 172.21.15.72:0/2750978998 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5637d547e6b0 con 0x5637d5743e20 2024-04-26T21:55:12.638 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.633+0000 7f825aa00640 1 --2- 172.21.15.72:0/2750978998 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5637d573e2c0 0x5637d5748790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:55:12.638 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.633+0000 7f8252000640 1 --2- 172.21.15.72:0/2750978998 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5637d573ecb0 0x5637d5748cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:55:12.638 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.633+0000 7f825b400640 1 --2- 172.21.15.72:0/2750978998 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5637d5743e20 0x5637d574fca0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f8254009780 tx=0x7f825409ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6711138887f030ec server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:55:12.638 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.633+0000 7f8248c00640 10 client.0 ms_handle_connect on v2:172.21.15.72:3300/0 2024-04-26T21:55:12.638 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.633+0000 7f8248c00640 1 -- 172.21.15.72:0/2750978998 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f825401f040 con 0x5637d5743e20 2024-04-26T21:55:12.638 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.633+0000 7f8248c00640 1 -- 172.21.15.72:0/2750978998 <== mon.0 v2:172.21.15.72:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7f82540870c0 con 0x5637d5743e20 2024-04-26T21:55:12.638 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.633+0000 7f82649f5740 1 -- 172.21.15.72:0/2750978998 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f824c01a8f0 con 0x5637d5743e20 2024-04-26T21:55:12.638 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.633+0000 7f8248c00640 1 -- 172.21.15.72:0/2750978998 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f82540872c0 con 0x5637d5743e20 2024-04-26T21:55:12.638 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.633+0000 7f82649f5740 10 client.15339 fetch_fsmap learned FSMap version 26 2024-04-26T21:55:12.638 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.633+0000 7f82649f5740 1 -- 172.21.15.72:0/2750978998 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5637d5752cc0 con 0x5637d5743e20 2024-04-26T21:55:12.638 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.633+0000 7f8248c00640 1 -- 172.21.15.72:0/2750978998 <== mon.0 v2:172.21.15.72:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f824c01a8f0 con 0x5637d5743e20 2024-04-26T21:55:12.638 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.634+0000 7f8248c00640 1 -- 172.21.15.72:0/2750978998 <== mon.0 v2:172.21.15.72:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f825400f040 con 0x5637d5743e20 2024-04-26T21:55:12.639 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.634+0000 7f82649f5740 10 client.15339 fetch_fsmap finished waiting for FSMap version 26 2024-04-26T21:55:12.639 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.634+0000 7f82649f5740 10 client.15339 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-26T21:55:12.639 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.634+0000 7f82649f5740 20 client.15339 populate_metadata read hostname 'smithi072' 2024-04-26T21:55:12.639 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.634+0000 7f82649f5740 1 --2- 172.21.15.72:0/2750978998 >> [v2:172.21.15.76:6836/1150678991,v1:172.21.15.76:6837/1150678991] conn(0x5637d5755400 0x5637d5756870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:12.639 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.634+0000 7f82649f5740 4 client.15339 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:55:12.639 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.634+0000 7f82649f5740 1 -- 172.21.15.72:0/2750978998 --> [v2:172.21.15.76:6836/1150678991,v1:172.21.15.76:6837/1150678991] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5637d56b0210 con 0x5637d5755400 2024-04-26T21:55:12.639 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.635+0000 7f825aa00640 1 --2- 172.21.15.72:0/2750978998 >> [v2:172.21.15.76:6836/1150678991,v1:172.21.15.76:6837/1150678991] conn(0x5637d5755400 0x5637d5756870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:12.639 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.635+0000 7f825aa00640 1 --2- 172.21.15.72:0/2750978998 >> [v2:172.21.15.76:6836/1150678991,v1:172.21.15.76:6837/1150678991] conn(0x5637d5755400 0x5637d5756870 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d77eb2a2200bf65c server_cookie=6f4dbad87316867f in_seq=0 out_seq=0 2024-04-26T21:55:12.639 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.635+0000 7f8248c00640 10 client.15339 ms_handle_connect on v2:172.21.15.76:6836/1150678991 2024-04-26T21:55:12.640 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.637+0000 7f8248c00640 1 -- 172.21.15.72:0/2750978998 <== mds.1 v2:172.21.15.76:6836/1150678991 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5637d56b0210 con 0x5637d5755400 2024-04-26T21:55:12.640 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.637+0000 7f8248c00640 10 client.15339 handle_command_reply: tid=0 2024-04-26T21:55:12.640 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.637+0000 7f82649f5740 1 client.15339 shutdown 2024-04-26T21:55:12.640 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.638+0000 7f82649f5740 1 -- 172.21.15.72:0/2750978998 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5637d5743e20 msgr2=0x5637d574fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:12.640 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.638+0000 7f82649f5740 1 --2- 172.21.15.72:0/2750978998 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5637d5743e20 0x5637d574fca0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f8254009780 tx=0x7f825409ca40 comp rx=0 tx=0).stop 2024-04-26T21:55:12.641 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.638+0000 7f82649f5740 1 -- 172.21.15.72:0/2750978998 shutdown_connections 2024-04-26T21:55:12.641 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.638+0000 7f82649f5740 1 --2- 172.21.15.72:0/2750978998 >> [v2:172.21.15.76:6836/1150678991,v1:172.21.15.76:6837/1150678991] conn(0x5637d5755400 0x5637d5756870 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:12.641 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.638+0000 7f82649f5740 1 --2- 172.21.15.72:0/2750978998 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5637d5743e20 0x5637d574fca0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:12.641 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.638+0000 7f82649f5740 1 --2- 172.21.15.72:0/2750978998 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5637d573ecb0 0x5637d5748cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:12.641 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.638+0000 7f82649f5740 1 --2- 172.21.15.72:0/2750978998 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5637d573e2c0 0x5637d5748790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:12.641 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.638+0000 7f8248c00640 0 client.15339 ms_handle_reset on v2:172.21.15.76:6836/1150678991 2024-04-26T21:55:12.641 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.638+0000 7f82649f5740 1 -- 172.21.15.72:0/2750978998 >> 172.21.15.72:0/2750978998 conn(0x5637d56d2d10 msgr2=0x5637d5738f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:55:12.641 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.638+0000 7f82649f5740 1 -- 172.21.15.72:0/2750978998 shutdown_connections 2024-04-26T21:55:12.641 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.638+0000 7f82649f5740 1 -- 172.21.15.72:0/2750978998 wait complete. 2024-04-26T21:55:12.641 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.638+0000 7f82649f5740 20 client.15339 trim_cache size 0 max 16384 2024-04-26T21:55:12.659 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.658+0000 7f82649f5740 1 Processor -- start 2024-04-26T21:55:12.659 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.658+0000 7f82649f5740 1 -- start start 2024-04-26T21:55:12.659 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.658+0000 7f82649f5740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5637d573e2c0 0x5637d5747380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:12.659 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.658+0000 7f82649f5740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5637d573ecb0 0x5637d574fc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:12.659 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.658+0000 7f82649f5740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5637d5743e20 0x5637d5740150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:12.659 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.658+0000 7f82649f5740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x5637d5754bb0 con 0x5637d573e2c0 2024-04-26T21:55:12.659 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.658+0000 7f82649f5740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5637d5755280 con 0x5637d573ecb0 2024-04-26T21:55:12.659 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.658+0000 7f82649f5740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5637d5755950 con 0x5637d5743e20 2024-04-26T21:55:12.659 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.658+0000 7f825aa00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5637d573e2c0 0x5637d5747380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:12.660 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.658+0000 7f825aa00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5637d573e2c0 0x5637d5747380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:34282/0 (socket says 172.21.15.72:34282) 2024-04-26T21:55:12.660 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.658+0000 7f825aa00640 1 -- 172.21.15.72:0/1340779333 learned_addr learned my addr 172.21.15.72:0/1340779333 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:55:12.660 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.659+0000 7f825aa00640 1 -- 172.21.15.72:0/1340779333 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5637d5743e20 msgr2=0x5637d5740150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:55:12.660 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.659+0000 7f825aa00640 1 --2- 172.21.15.72:0/1340779333 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5637d5743e20 0x5637d5740150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:12.660 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.659+0000 7f8252000640 1 --2- 172.21.15.72:0/1340779333 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5637d573ecb0 0x5637d574fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:12.660 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.659+0000 7f825aa00640 1 -- 172.21.15.72:0/1340779333 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5637d573ecb0 msgr2=0x5637d574fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:55:12.660 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.659+0000 7f825aa00640 1 --2- 172.21.15.72:0/1340779333 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5637d573ecb0 0x5637d574fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:12.660 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.659+0000 7f825aa00640 1 -- 172.21.15.72:0/1340779333 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5637d573f540 con 0x5637d573e2c0 2024-04-26T21:55:12.660 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.659+0000 7f8252000640 1 --2- 172.21.15.72:0/1340779333 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5637d573ecb0 0x5637d574fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:55:12.660 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.659+0000 7f825aa00640 1 --2- 172.21.15.72:0/1340779333 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5637d573e2c0 0x5637d5747380 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f824c00fc30 tx=0x7f824c005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3faada37d50e80cb server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:55:12.664 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.659+0000 7f824b400640 1 -- 172.21.15.72:0/1340779333 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f824c015040 con 0x5637d573e2c0 2024-04-26T21:55:12.664 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.659+0000 7f824b400640 1 -- 172.21.15.72:0/1340779333 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f824c0054c0 con 0x5637d573e2c0 2024-04-26T21:55:12.664 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.659+0000 7f824b400640 1 -- 172.21.15.72:0/1340779333 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f824c004b50 con 0x5637d573e2c0 2024-04-26T21:55:12.664 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.660+0000 7f82649f5740 1 -- 172.21.15.72:0/1340779333 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5637d573e2c0 msgr2=0x5637d5747380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:12.664 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.660+0000 7f82649f5740 1 --2- 172.21.15.72:0/1340779333 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5637d573e2c0 0x5637d5747380 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f824c00fc30 tx=0x7f824c005850 comp rx=0 tx=0).stop 2024-04-26T21:55:12.664 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.660+0000 7f82649f5740 1 -- 172.21.15.72:0/1340779333 shutdown_connections 2024-04-26T21:55:12.664 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.660+0000 7f82649f5740 1 --2- 172.21.15.72:0/1340779333 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5637d5743e20 0x5637d5740150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:12.664 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.660+0000 7f82649f5740 1 --2- 172.21.15.72:0/1340779333 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5637d573ecb0 0x5637d574fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:12.664 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.660+0000 7f82649f5740 1 --2- 172.21.15.72:0/1340779333 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5637d573e2c0 0x5637d5747380 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:12.664 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.660+0000 7f82649f5740 1 -- 172.21.15.72:0/1340779333 >> 172.21.15.72:0/1340779333 conn(0x5637d56d2d10 msgr2=0x5637d5748790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:55:12.664 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.660+0000 7f82649f5740 1 -- 172.21.15.72:0/1340779333 shutdown_connections 2024-04-26T21:55:12.664 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.660+0000 7f82649f5740 1 -- 172.21.15.72:0/1340779333 wait complete. 2024-04-26T21:55:12.664 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.661+0000 7f82649f5740 1 -- start start 2024-04-26T21:55:12.665 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.661+0000 7f82649f5740 1 Processor -- start 2024-04-26T21:55:12.665 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.661+0000 7f82649f5740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5637d573e2c0 0x5637d574be90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:12.665 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.661+0000 7f82649f5740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5637d573ecb0 0x5637d574c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:12.665 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.661+0000 7f82649f5740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5637d5743e20 0x5637d574eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:12.665 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.661+0000 7f82649f5740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x5637d57566f0 con 0x5637d573e2c0 2024-04-26T21:55:12.665 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.661+0000 7f82649f5740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5637d5756020 con 0x5637d5743e20 2024-04-26T21:55:12.665 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.661+0000 7f82649f5740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5637d5756dc0 con 0x5637d573ecb0 2024-04-26T21:55:12.665 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.661+0000 7f82649f5740 10 client.0 _ll_register_callbacks cb 0x7f8254098c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:55:12.665 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.661+0000 7f825aa00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5637d573e2c0 0x5637d574be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:12.665 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.662+0000 7f825aa00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5637d573e2c0 0x5637d574be90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:34290/0 (socket says 172.21.15.72:34290) 2024-04-26T21:55:12.665 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.662+0000 7f825aa00640 1 -- 172.21.15.72:0/3371177640 learned_addr learned my addr 172.21.15.72:0/3371177640 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:55:12.665 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.662+0000 7f825aa00640 1 -- 172.21.15.72:0/3371177640 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5637d573ecb0 msgr2=0x5637d574c3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:55:12.665 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.662+0000 7f825aa00640 1 --2- 172.21.15.72:0/3371177640 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5637d573ecb0 0x5637d574c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:12.665 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.662+0000 7f825aa00640 1 -- 172.21.15.72:0/3371177640 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5637d5743e20 msgr2=0x5637d574eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:55:12.666 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.662+0000 7f825aa00640 1 --2- 172.21.15.72:0/3371177640 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5637d5743e20 0x5637d574eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:12.666 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.662+0000 7f825aa00640 1 -- 172.21.15.72:0/3371177640 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5637d574f4a0 con 0x5637d573e2c0 2024-04-26T21:55:12.666 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.662+0000 7f825aa00640 1 --2- 172.21.15.72:0/3371177640 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5637d573e2c0 0x5637d574be90 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f824c00ba40 tx=0x7f824c005670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2295cf182847cbb6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:55:12.666 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.662+0000 7f8248c00640 10 client.0 ms_handle_connect on v2:172.21.15.72:3300/0 2024-04-26T21:55:12.666 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.662+0000 7f8248c00640 1 -- 172.21.15.72:0/3371177640 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f824c034020 con 0x5637d573e2c0 2024-04-26T21:55:12.666 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.662+0000 7f8248c00640 1 -- 172.21.15.72:0/3371177640 <== mon.0 v2:172.21.15.72:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7f824c0262c0 con 0x5637d573e2c0 2024-04-26T21:55:12.666 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.662+0000 7f82649f5740 1 -- 172.21.15.72:0/3371177640 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f824c0054c0 con 0x5637d573e2c0 2024-04-26T21:55:12.666 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.663+0000 7f8248c00640 1 -- 172.21.15.72:0/3371177640 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f824c03f050 con 0x5637d573e2c0 2024-04-26T21:55:12.666 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.663+0000 7f82649f5740 10 client.15345 fetch_fsmap learned FSMap version 26 2024-04-26T21:55:12.666 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.663+0000 7f82649f5740 1 -- 172.21.15.72:0/3371177640 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5637d57bb9b0 con 0x5637d573e2c0 2024-04-26T21:55:12.666 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.663+0000 7f8248c00640 1 -- 172.21.15.72:0/3371177640 <== mon.0 v2:172.21.15.72:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f824c0054c0 con 0x5637d573e2c0 2024-04-26T21:55:12.666 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.663+0000 7f8248c00640 1 -- 172.21.15.72:0/3371177640 <== mon.0 v2:172.21.15.72:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f824c030900 con 0x5637d573e2c0 2024-04-26T21:55:12.666 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.663+0000 7f82649f5740 10 client.15345 fetch_fsmap finished waiting for FSMap version 26 2024-04-26T21:55:12.666 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.663+0000 7f82649f5740 10 client.15345 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-26T21:55:12.666 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.663+0000 7f82649f5740 20 client.15345 populate_metadata read hostname 'smithi072' 2024-04-26T21:55:12.667 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.663+0000 7f82649f5740 1 --2- 172.21.15.72:0/3371177640 >> [v2:172.21.15.76:6836/1150678991,v1:172.21.15.76:6837/1150678991] conn(0x5637d57c0150 0x5637d57bee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:12.667 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.663+0000 7f82649f5740 4 client.15345 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T21:55:12.667 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.663+0000 7f82649f5740 1 -- 172.21.15.72:0/3371177640 --> [v2:172.21.15.76:6836/1150678991,v1:172.21.15.76:6837/1150678991] -- command(tid 0: {"prefix": "status"}) -- 0x5637d56aa080 con 0x5637d57c0150 2024-04-26T21:55:12.668 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.664+0000 7f8252000640 1 --2- 172.21.15.72:0/3371177640 >> [v2:172.21.15.76:6836/1150678991,v1:172.21.15.76:6837/1150678991] conn(0x5637d57c0150 0x5637d57bee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:12.668 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.665+0000 7f8252000640 1 --2- 172.21.15.72:0/3371177640 >> [v2:172.21.15.76:6836/1150678991,v1:172.21.15.76:6837/1150678991] conn(0x5637d57c0150 0x5637d57bee60 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a30b56d66eb9504a server_cookie=59158b334676f95d in_seq=0 out_seq=0 2024-04-26T21:55:12.668 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.665+0000 7f8248c00640 10 client.15345 ms_handle_connect on v2:172.21.15.76:6836/1150678991 2024-04-26T21:55:12.668 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.666+0000 7f8248c00640 1 -- 172.21.15.72:0/3371177640 <== mds.1 v2:172.21.15.76:6836/1150678991 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5637d56aa080 con 0x5637d57c0150 2024-04-26T21:55:12.668 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.666+0000 7f8248c00640 10 client.15345 handle_command_reply: tid=0 2024-04-26T21:55:12.668 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.666+0000 7f82649f5740 1 client.15345 shutdown 2024-04-26T21:55:12.668 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.666+0000 7f82649f5740 1 -- 172.21.15.72:0/3371177640 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5637d573e2c0 msgr2=0x5637d574be90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:12.669 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.666+0000 7f82649f5740 1 --2- 172.21.15.72:0/3371177640 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5637d573e2c0 0x5637d574be90 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f824c00ba40 tx=0x7f824c005670 comp rx=0 tx=0).stop 2024-04-26T21:55:12.669 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.666+0000 7f82649f5740 1 -- 172.21.15.72:0/3371177640 shutdown_connections 2024-04-26T21:55:12.669 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.666+0000 7f82649f5740 1 --2- 172.21.15.72:0/3371177640 >> [v2:172.21.15.76:6836/1150678991,v1:172.21.15.76:6837/1150678991] conn(0x5637d57c0150 0x5637d57bee60 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:12.669 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.666+0000 7f82649f5740 1 --2- 172.21.15.72:0/3371177640 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5637d5743e20 0x5637d574eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:12.669 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.666+0000 7f82649f5740 1 --2- 172.21.15.72:0/3371177640 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5637d573ecb0 0x5637d574c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:12.669 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.666+0000 7f8248c00640 0 client.15345 ms_handle_reset on v2:172.21.15.76:6836/1150678991 2024-04-26T21:55:12.669 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.666+0000 7f82649f5740 1 --2- 172.21.15.72:0/3371177640 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5637d573e2c0 0x5637d574be90 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:12.669 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.666+0000 7f82649f5740 1 -- 172.21.15.72:0/3371177640 >> 172.21.15.72:0/3371177640 conn(0x5637d56d2d10 msgr2=0x5637d5748790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:55:12.669 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.666+0000 7f82649f5740 1 -- 172.21.15.72:0/3371177640 shutdown_connections 2024-04-26T21:55:12.669 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.666+0000 7f82649f5740 1 -- 172.21.15.72:0/3371177640 wait complete. 2024-04-26T21:55:12.669 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.667+0000 7f82649f5740 20 client.15345 trim_cache size 0 max 16384 2024-04-26T21:55:12.670 INFO:teuthology.orchestra.run.smithi072.stdout:{ 2024-04-26T21:55:12.670 INFO:teuthology.orchestra.run.smithi072.stdout: "cluster_fsid": "a1780950-0415-11ef-bc93-c7b262605968", 2024-04-26T21:55:12.670 INFO:teuthology.orchestra.run.smithi072.stdout: "whoami": 1, 2024-04-26T21:55:12.670 INFO:teuthology.orchestra.run.smithi072.stdout: "id": 34472, 2024-04-26T21:55:12.670 INFO:teuthology.orchestra.run.smithi072.stdout: "want_state": "up:active", 2024-04-26T21:55:12.670 INFO:teuthology.orchestra.run.smithi072.stdout: "state": "up:active", 2024-04-26T21:55:12.670 INFO:teuthology.orchestra.run.smithi072.stdout: "fs_name": "cephfs", 2024-04-26T21:55:12.670 INFO:teuthology.orchestra.run.smithi072.stdout: "rank_uptime": 138.13891455000001, 2024-04-26T21:55:12.670 INFO:teuthology.orchestra.run.smithi072.stdout: "mdsmap_epoch": 26, 2024-04-26T21:55:12.670 INFO:teuthology.orchestra.run.smithi072.stdout: "osdmap_epoch": 91, 2024-04-26T21:55:12.670 INFO:teuthology.orchestra.run.smithi072.stdout: "osdmap_epoch_barrier": 87, 2024-04-26T21:55:12.670 INFO:teuthology.orchestra.run.smithi072.stdout: "uptime": 164.197507707 2024-04-26T21:55:12.670 INFO:teuthology.orchestra.run.smithi072.stdout:} 2024-04-26T21:55:12.671 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.667+0000 7f825be00640 1 -- 172.21.15.72:0/215230690 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f82400a3c40 msgr2=0x7f82400a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:12.671 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.667+0000 7f825be00640 1 --2- 172.21.15.72:0/215230690 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f82400a3c40 0x7f82400a6100 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f82440096c0 tx=0x7f8244000bc0 comp rx=0 tx=0).stop 2024-04-26T21:55:12.671 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.667+0000 7f825be00640 1 -- 172.21.15.72:0/215230690 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f825c104320 msgr2=0x7f825c1a2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:12.671 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.667+0000 7f825be00640 1 --2- 172.21.15.72:0/215230690 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f825c104320 0x7f825c1a2090 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f825400eaa0 tx=0x7f825400ef60 comp rx=0 tx=0).stop 2024-04-26T21:55:12.671 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.667+0000 7f825be00640 1 -- 172.21.15.72:0/215230690 shutdown_connections 2024-04-26T21:55:12.671 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.667+0000 7f825be00640 1 --2- 172.21.15.72:0/215230690 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f825c104320 0x7f825c1a2090 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:12.672 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.667+0000 7f825be00640 1 --2- 172.21.15.72:0/215230690 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f82400a3c40 0x7f82400a6100 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:12.672 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.667+0000 7f825be00640 1 --2- 172.21.15.72:0/215230690 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f825c103960 0x7f825c19b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:12.672 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.667+0000 7f825be00640 1 --2- 172.21.15.72:0/215230690 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f825c102770 0x7f825c19aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:12.672 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.667+0000 7f825be00640 1 -- 172.21.15.72:0/215230690 >> 172.21.15.72:0/215230690 conn(0x7f825c0fdce0 msgr2=0x7f825c074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:55:12.672 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.667+0000 7f825be00640 1 -- 172.21.15.72:0/215230690 shutdown_connections 2024-04-26T21:55:12.672 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.667+0000 7f825be00640 1 -- 172.21.15.72:0/215230690 wait complete. 2024-04-26T21:55:12.686 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-26T21:55:12.889 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.889+0000 7fe9e5000640 1 -- 172.21.15.72:0/3443280906 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe9e0104310 msgr2=0x7fe9e010aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:12.889 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.889+0000 7fe9e5000640 1 --2- 172.21.15.72:0/3443280906 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe9e0104310 0x7fe9e010aba0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fe9d0009900 tx=0x7fe9d002a410 comp rx=0 tx=0).stop 2024-04-26T21:55:12.889 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.889+0000 7fe9e5000640 1 -- 172.21.15.72:0/3443280906 shutdown_connections 2024-04-26T21:55:12.889 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.889+0000 7fe9e5000640 1 --2- 172.21.15.72:0/3443280906 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe9e0104310 0x7fe9e010aba0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:12.890 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.889+0000 7fe9e5000640 1 --2- 172.21.15.72:0/3443280906 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe9e0103950 0x7fe9e0103dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:12.890 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.889+0000 7fe9e5000640 1 --2- 172.21.15.72:0/3443280906 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe9e0102760 0x7fe9e0102b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:12.890 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.889+0000 7fe9e5000640 1 -- 172.21.15.72:0/3443280906 >> 172.21.15.72:0/3443280906 conn(0x7fe9e00fdcb0 msgr2=0x7fe9e0100110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:55:12.890 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.889+0000 7fe9e5000640 1 -- 172.21.15.72:0/3443280906 shutdown_connections 2024-04-26T21:55:12.890 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.889+0000 7fe9e5000640 1 -- 172.21.15.72:0/3443280906 wait complete. 2024-04-26T21:55:12.892 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.890+0000 7fe9e5000640 1 Processor -- start 2024-04-26T21:55:12.892 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.890+0000 7fe9e5000640 1 -- start start 2024-04-26T21:55:12.892 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.890+0000 7fe9e5000640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe9e0102760 0x7fe9e019aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:12.892 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.890+0000 7fe9e5000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe9e0103950 0x7fe9e019b030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:12.892 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.890+0000 7fe9e5000640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe9e0104310 0x7fe9e01a2050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:12.892 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.890+0000 7fe9e5000640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fe9e010e120 con 0x7fe9e0104310 2024-04-26T21:55:12.892 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.890+0000 7fe9e5000640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fe9e010dfa0 con 0x7fe9e0102760 2024-04-26T21:55:12.892 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.890+0000 7fe9e5000640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe9e010e2a0 con 0x7fe9e0103950 2024-04-26T21:55:12.892 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.891+0000 7fe9dfe00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe9e0104310 0x7fe9e01a2050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:12.892 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.891+0000 7fe9dfe00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe9e0104310 0x7fe9e01a2050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:34314/0 (socket says 172.21.15.72:34314) 2024-04-26T21:55:12.892 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.891+0000 7fe9dfe00640 1 -- 172.21.15.72:0/2259273244 learned_addr learned my addr 172.21.15.72:0/2259273244 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:55:12.893 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.891+0000 7fe9df400640 1 --2- 172.21.15.72:0/2259273244 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe9e0102760 0x7fe9e019aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:12.893 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.891+0000 7fe9dfe00640 1 -- 172.21.15.72:0/2259273244 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe9e0103950 msgr2=0x7fe9e019b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:12.893 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.891+0000 7fe9dfe00640 1 --2- 172.21.15.72:0/2259273244 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe9e0103950 0x7fe9e019b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:12.893 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.891+0000 7fe9dfe00640 1 -- 172.21.15.72:0/2259273244 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe9e0102760 msgr2=0x7fe9e019aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:12.893 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.891+0000 7fe9dfe00640 1 --2- 172.21.15.72:0/2259273244 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe9e0102760 0x7fe9e019aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:12.893 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.891+0000 7fe9dfe00640 1 -- 172.21.15.72:0/2259273244 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9e01a26c0 con 0x7fe9e0104310 2024-04-26T21:55:12.893 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.891+0000 7fe9dfe00640 1 --2- 172.21.15.72:0/2259273244 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe9e0104310 0x7fe9e01a2050 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fe9d0004770 tx=0x7fe9d00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:55:12.893 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.892+0000 7fe9d7e00640 1 -- 172.21.15.72:0/2259273244 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9d003f030 con 0x7fe9e0104310 2024-04-26T21:55:12.893 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.892+0000 7fe9d7e00640 1 -- 172.21.15.72:0/2259273244 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe9d0003c90 con 0x7fe9e0104310 2024-04-26T21:55:12.893 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.892+0000 7fe9e5000640 1 -- 172.21.15.72:0/2259273244 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9e01a2950 con 0x7fe9e0104310 2024-04-26T21:55:12.893 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.892+0000 7fe9d7e00640 1 -- 172.21.15.72:0/2259273244 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9d0035070 con 0x7fe9e0104310 2024-04-26T21:55:12.893 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.892+0000 7fe9e5000640 1 -- 172.21.15.72:0/2259273244 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9e01a2e90 con 0x7fe9e0104310 2024-04-26T21:55:12.897 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.895+0000 7fe9e9594740 1 Processor -- start 2024-04-26T21:55:12.897 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.895+0000 7fe9d7e00640 1 -- 172.21.15.72:0/2259273244 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fe9d0042030 con 0x7fe9e0104310 2024-04-26T21:55:12.897 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.895+0000 7fe9e9594740 1 -- start start 2024-04-26T21:55:12.897 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.896+0000 7fe9e9594740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557a96eab2c0 0x557a96eab6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:12.897 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.896+0000 7fe9e9594740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557a96eabcb0 0x557a96eb07b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:12.897 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.896+0000 7fe9e9594740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557a96eb0e20 0x557a96eb32b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:12.897 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.896+0000 7fe9e9594740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x557a96e0f600 con 0x557a96eb0e20 2024-04-26T21:55:12.897 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.896+0000 7fe9e9594740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x557a96dac100 con 0x557a96eabcb0 2024-04-26T21:55:12.897 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.896+0000 7fe9e9594740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x557a96e0b5c0 con 0x557a96eab2c0 2024-04-26T21:55:12.897 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.896+0000 7fe9dea00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557a96eb0e20 0x557a96eb32b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:12.898 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.896+0000 7fe9dea00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557a96eb0e20 0x557a96eb32b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:34318/0 (socket says 172.21.15.72:34318) 2024-04-26T21:55:12.898 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.896+0000 7fe9dea00640 1 -- 172.21.15.72:0/1850374541 learned_addr learned my addr 172.21.15.72:0/1850374541 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:55:12.898 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.896+0000 7fe9df400640 1 --2- 172.21.15.72:0/1850374541 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557a96eabcb0 0x557a96eb07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:12.898 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.896+0000 7fe9dea00640 1 --2- 172.21.15.72:0/1850374541 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557a96eab2c0 0x557a96eab6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:12.898 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.896+0000 7fe9dea00640 1 -- 172.21.15.72:0/1850374541 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557a96eab2c0 msgr2=0x557a96eab6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:55:12.898 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.896+0000 7fe9dea00640 1 --2- 172.21.15.72:0/1850374541 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557a96eab2c0 0x557a96eab6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:12.898 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.896+0000 7fe9dea00640 1 -- 172.21.15.72:0/1850374541 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557a96eabcb0 msgr2=0x557a96eb07b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:55:12.899 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.896+0000 7fe9dea00640 1 --2- 172.21.15.72:0/1850374541 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557a96eabcb0 0x557a96eb07b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:12.899 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.896+0000 7fe9dea00640 1 -- 172.21.15.72:0/1850374541 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a96b212d0 con 0x557a96eb0e20 2024-04-26T21:55:12.899 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.897+0000 7fe9dea00640 1 --2- 172.21.15.72:0/1850374541 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557a96eb0e20 0x557a96eb32b0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fe9c8005d00 tx=0x7fe9c800d520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=226288d3dc806e34 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:55:12.899 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.897+0000 7fe9d7e00640 1 --2- 172.21.15.72:0/2259273244 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fe9c40a3cc0 0x7fe9c40a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:12.903 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.897+0000 7fe9d4c00640 1 -- 172.21.15.72:0/1850374541 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9c8012450 con 0x557a96eb0e20 2024-04-26T21:55:12.903 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.897+0000 7fe9d7e00640 1 -- 172.21.15.72:0/2259273244 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7fe9d00f0d60 con 0x7fe9e0104310 2024-04-26T21:55:12.903 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.897+0000 7fe9d4c00640 1 -- 172.21.15.72:0/1850374541 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe9c8012a80 con 0x557a96eb0e20 2024-04-26T21:55:12.904 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.897+0000 7fe9d4c00640 1 -- 172.21.15.72:0/1850374541 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9c80119a0 con 0x557a96eb0e20 2024-04-26T21:55:12.904 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.897+0000 7fe9df400640 1 --2- 172.21.15.72:0/2259273244 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fe9c40a3cc0 0x7fe9c40a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:12.904 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.898+0000 7fe9e9594740 1 -- 172.21.15.72:0/1850374541 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557a96eb0e20 msgr2=0x557a96eb32b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:12.904 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.898+0000 7fe9df400640 1 --2- 172.21.15.72:0/2259273244 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fe9c40a3cc0 0x7fe9c40a6180 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x557a96eac0b0 tx=0x7fe9d8009580 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:55:12.904 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.898+0000 7fe9e9594740 1 --2- 172.21.15.72:0/1850374541 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557a96eb0e20 0x557a96eb32b0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fe9c8005d00 tx=0x7fe9c800d520 comp rx=0 tx=0).stop 2024-04-26T21:55:12.904 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.898+0000 7fe9e9594740 1 -- 172.21.15.72:0/1850374541 shutdown_connections 2024-04-26T21:55:12.904 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.898+0000 7fe9e9594740 1 --2- 172.21.15.72:0/1850374541 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557a96eb0e20 0x557a96eb32b0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:12.904 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.898+0000 7fe9e9594740 1 --2- 172.21.15.72:0/1850374541 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557a96eabcb0 0x557a96eb07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:12.904 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.898+0000 7fe9e9594740 1 --2- 172.21.15.72:0/1850374541 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557a96eab2c0 0x557a96eab6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:12.905 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.898+0000 7fe9e9594740 1 -- 172.21.15.72:0/1850374541 >> 172.21.15.72:0/1850374541 conn(0x557a96e3fd10 msgr2=0x557a96ea6d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:55:12.905 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.898+0000 7fe9e9594740 1 -- 172.21.15.72:0/1850374541 shutdown_connections 2024-04-26T21:55:12.905 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.898+0000 7fe9e9594740 1 -- 172.21.15.72:0/1850374541 wait complete. 2024-04-26T21:55:12.905 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.899+0000 7fe9e9594740 1 -- start start 2024-04-26T21:55:12.905 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.899+0000 7fe9e9594740 1 Processor -- start 2024-04-26T21:55:12.905 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.899+0000 7fe9e9594740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557a96eab2c0 0x557a96eb57a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:12.905 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.899+0000 7fe9e9594740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557a96eabcb0 0x557a96eb5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:12.905 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.899+0000 7fe9e9594740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557a96eb0e20 0x557a96ebcd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:12.906 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.899+0000 7fe9e9594740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x557a96eb3dd0 con 0x557a96eabcb0 2024-04-26T21:55:12.906 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.899+0000 7fe9e9594740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x557a96d96f70 con 0x557a96eb0e20 2024-04-26T21:55:12.906 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.899+0000 7fe9e9594740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x557a96eb3f50 con 0x557a96eab2c0 2024-04-26T21:55:12.906 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.899+0000 7fe9e9594740 10 client.0 _ll_register_callbacks cb 0x557a96e05480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:55:12.906 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.900+0000 7fe9dea00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557a96eabcb0 0x557a96eb5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:12.906 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.900+0000 7fe9dea00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557a96eabcb0 0x557a96eb5ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:34330/0 (socket says 172.21.15.72:34330) 2024-04-26T21:55:12.906 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.900+0000 7fe9dea00640 1 -- 172.21.15.72:0/1410918219 learned_addr learned my addr 172.21.15.72:0/1410918219 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:55:12.906 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.900+0000 7fe9df400640 1 --2- 172.21.15.72:0/1410918219 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557a96eab2c0 0x557a96eb57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:12.906 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.900+0000 7fe9dea00640 1 -- 172.21.15.72:0/1410918219 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557a96eab2c0 msgr2=0x557a96eb57a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:55:12.906 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.900+0000 7fe9dea00640 1 --2- 172.21.15.72:0/1410918219 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557a96eab2c0 0x557a96eb57a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:12.906 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.900+0000 7fe9dea00640 1 -- 172.21.15.72:0/1410918219 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557a96eb0e20 msgr2=0x557a96ebcd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:55:12.906 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.900+0000 7fe9dfe00640 1 --2- 172.21.15.72:0/1410918219 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557a96eb0e20 0x557a96ebcd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:12.906 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.900+0000 7fe9dea00640 1 --2- 172.21.15.72:0/1410918219 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557a96eb0e20 0x557a96ebcd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:12.907 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.900+0000 7fe9dea00640 1 -- 172.21.15.72:0/1410918219 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a96beb6b0 con 0x557a96eabcb0 2024-04-26T21:55:12.907 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.900+0000 7fe9df400640 1 --2- 172.21.15.72:0/1410918219 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557a96eab2c0 0x557a96eb57a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:55:12.907 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.900+0000 7fe9dea00640 1 --2- 172.21.15.72:0/1410918219 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557a96eabcb0 0x557a96eb5ce0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fe9c800dd90 tx=0x7fe9c80114b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=800240522792dc8d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:55:12.907 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.900+0000 7fe9cd600640 10 client.0 ms_handle_connect on v2:172.21.15.72:3300/0 2024-04-26T21:55:12.907 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.901+0000 7fe9cd600640 1 -- 172.21.15.72:0/1410918219 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9c8023070 con 0x557a96eabcb0 2024-04-26T21:55:12.907 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.901+0000 7fe9cd600640 1 -- 172.21.15.72:0/1410918219 <== mon.0 v2:172.21.15.72:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7fe9c8012860 con 0x557a96eabcb0 2024-04-26T21:55:12.907 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.901+0000 7fe9e9594740 1 -- 172.21.15.72:0/1410918219 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe9c8012a80 con 0x557a96eabcb0 2024-04-26T21:55:12.907 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.901+0000 7fe9cd600640 1 -- 172.21.15.72:0/1410918219 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe9c8012de0 con 0x557a96eabcb0 2024-04-26T21:55:12.907 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.901+0000 7fe9e9594740 10 client.15357 fetch_fsmap learned FSMap version 26 2024-04-26T21:55:12.907 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.901+0000 7fe9e9594740 1 -- 172.21.15.72:0/1410918219 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x557a96ebfd00 con 0x557a96eabcb0 2024-04-26T21:55:12.907 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.901+0000 7fe9cd600640 1 -- 172.21.15.72:0/1410918219 <== mon.0 v2:172.21.15.72:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe9c8012a80 con 0x557a96eabcb0 2024-04-26T21:55:12.907 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.901+0000 7fe9cd600640 1 -- 172.21.15.72:0/1410918219 <== mon.0 v2:172.21.15.72:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fe9c802f070 con 0x557a96eabcb0 2024-04-26T21:55:12.907 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.902+0000 7fe9e9594740 10 client.15357 fetch_fsmap finished waiting for FSMap version 26 2024-04-26T21:55:12.907 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.902+0000 7fe9e9594740 10 client.15357 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-26T21:55:12.908 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.902+0000 7fe9e9594740 20 client.15357 populate_metadata read hostname 'smithi072' 2024-04-26T21:55:12.908 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.902+0000 7fe9e9594740 1 --2- 172.21.15.72:0/1410918219 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x557a96ec1f20 0x557a96ec3390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:12.908 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.902+0000 7fe9e9594740 4 client.15357 mds_command: new command op to 34517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:55:12.908 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.902+0000 7fe9e9594740 1 -- 172.21.15.72:0/1410918219 --> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557a96e1d210 con 0x557a96ec1f20 2024-04-26T21:55:12.908 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.902+0000 7fe9df400640 1 --2- 172.21.15.72:0/1410918219 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x557a96ec1f20 0x557a96ec3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:12.908 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.903+0000 7fe9df400640 1 --2- 172.21.15.72:0/1410918219 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x557a96ec1f20 0x557a96ec3390 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f4c36cae411e8325 server_cookie=e58a53ee22f2ca62 in_seq=0 out_seq=0 2024-04-26T21:55:12.908 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.903+0000 7fe9cd600640 10 client.15357 ms_handle_connect on v2:172.21.15.76:6838/327145636 2024-04-26T21:55:12.909 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.905+0000 7fe9cd600640 1 -- 172.21.15.72:0/1410918219 <== mds.0 v2:172.21.15.76:6838/327145636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557a96e1d210 con 0x557a96ec1f20 2024-04-26T21:55:12.909 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.905+0000 7fe9cd600640 10 client.15357 handle_command_reply: tid=0 2024-04-26T21:55:12.909 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.905+0000 7fe9e9594740 1 client.15357 shutdown 2024-04-26T21:55:12.909 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.905+0000 7fe9e9594740 1 -- 172.21.15.72:0/1410918219 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557a96eabcb0 msgr2=0x557a96eb5ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:12.909 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.905+0000 7fe9e9594740 1 --2- 172.21.15.72:0/1410918219 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557a96eabcb0 0x557a96eb5ce0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fe9c800dd90 tx=0x7fe9c80114b0 comp rx=0 tx=0).stop 2024-04-26T21:55:12.909 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.905+0000 7fe9e9594740 1 -- 172.21.15.72:0/1410918219 shutdown_connections 2024-04-26T21:55:12.910 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.905+0000 7fe9e9594740 1 --2- 172.21.15.72:0/1410918219 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x557a96ec1f20 0x557a96ec3390 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:12.910 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.905+0000 7fe9e9594740 1 --2- 172.21.15.72:0/1410918219 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557a96eb0e20 0x557a96ebcd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:12.910 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.905+0000 7fe9cd600640 0 client.15357 ms_handle_reset on v2:172.21.15.76:6838/327145636 2024-04-26T21:55:12.910 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.905+0000 7fe9e9594740 1 --2- 172.21.15.72:0/1410918219 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557a96eabcb0 0x557a96eb5ce0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:12.910 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.905+0000 7fe9e9594740 1 --2- 172.21.15.72:0/1410918219 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557a96eab2c0 0x557a96eb57a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:12.910 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.905+0000 7fe9e9594740 1 -- 172.21.15.72:0/1410918219 >> 172.21.15.72:0/1410918219 conn(0x557a96e3fd10 msgr2=0x557a96ea5f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:55:12.910 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.906+0000 7fe9e9594740 1 -- 172.21.15.72:0/1410918219 shutdown_connections 2024-04-26T21:55:12.910 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.906+0000 7fe9e9594740 1 -- 172.21.15.72:0/1410918219 wait complete. 2024-04-26T21:55:12.910 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.906+0000 7fe9e9594740 20 client.15357 trim_cache size 0 max 16384 2024-04-26T21:55:12.926 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.926+0000 7fe9e9594740 1 Processor -- start 2024-04-26T21:55:12.926 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.926+0000 7fe9e9594740 1 -- start start 2024-04-26T21:55:12.926 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.926+0000 7fe9e9594740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557a96eab2c0 0x557a96eb53b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:12.926 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.926+0000 7fe9e9594740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557a96eabcb0 0x557a96ebcce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:12.926 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.926+0000 7fe9e9594740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557a96eb0e20 0x557a96eb58f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:12.926 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.926+0000 7fe9e9594740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x557a96ec16d0 con 0x557a96eb0e20 2024-04-26T21:55:12.926 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.926+0000 7fe9e9594740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x557a96ec1da0 con 0x557a96eab2c0 2024-04-26T21:55:12.926 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.926+0000 7fe9e9594740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x557a96ec2470 con 0x557a96eabcb0 2024-04-26T21:55:12.928 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.926+0000 7fe9dfe00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557a96eb0e20 0x557a96eb58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:12.928 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.926+0000 7fe9dfe00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557a96eb0e20 0x557a96eb58f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:34334/0 (socket says 172.21.15.72:34334) 2024-04-26T21:55:12.928 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.926+0000 7fe9dfe00640 1 -- 172.21.15.72:0/266841214 learned_addr learned my addr 172.21.15.72:0/266841214 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:55:12.928 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.927+0000 7fe9dfe00640 1 -- 172.21.15.72:0/266841214 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557a96eabcb0 msgr2=0x557a96ebcce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:55:12.928 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.927+0000 7fe9dfe00640 1 --2- 172.21.15.72:0/266841214 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557a96eabcb0 0x557a96ebcce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:12.928 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.927+0000 7fe9dfe00640 1 -- 172.21.15.72:0/266841214 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557a96eab2c0 msgr2=0x557a96eb53b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:55:12.928 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.927+0000 7fe9dfe00640 1 --2- 172.21.15.72:0/266841214 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557a96eab2c0 0x557a96eb53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:12.928 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.927+0000 7fe9dfe00640 1 -- 172.21.15.72:0/266841214 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a96ea79f0 con 0x557a96eb0e20 2024-04-26T21:55:12.929 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.927+0000 7fe9dfe00640 1 --2- 172.21.15.72:0/266841214 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557a96eb0e20 0x557a96eb58f0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fe9d00c5540 tx=0x7fe9d00bc890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac13fa1ff6c76af3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:55:12.929 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.927+0000 7fe9cfe00640 1 -- 172.21.15.72:0/266841214 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9d00d6070 con 0x557a96eb0e20 2024-04-26T21:55:12.929 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.927+0000 7fe9cfe00640 1 -- 172.21.15.72:0/266841214 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe9d00bce00 con 0x557a96eb0e20 2024-04-26T21:55:12.929 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.927+0000 7fe9cfe00640 1 -- 172.21.15.72:0/266841214 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9d0049070 con 0x557a96eb0e20 2024-04-26T21:55:12.929 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.928+0000 7fe9e9594740 1 -- 172.21.15.72:0/266841214 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557a96eb0e20 msgr2=0x557a96eb58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:12.929 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.928+0000 7fe9e9594740 1 --2- 172.21.15.72:0/266841214 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557a96eb0e20 0x557a96eb58f0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fe9d00c5540 tx=0x7fe9d00bc890 comp rx=0 tx=0).stop 2024-04-26T21:55:12.929 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.928+0000 7fe9e9594740 1 -- 172.21.15.72:0/266841214 shutdown_connections 2024-04-26T21:55:12.929 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.928+0000 7fe9e9594740 1 --2- 172.21.15.72:0/266841214 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557a96eb0e20 0x557a96eb58f0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:12.929 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.928+0000 7fe9e9594740 1 --2- 172.21.15.72:0/266841214 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557a96eabcb0 0x557a96ebcce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:12.929 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.928+0000 7fe9e9594740 1 --2- 172.21.15.72:0/266841214 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557a96eab2c0 0x557a96eb53b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:12.929 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.928+0000 7fe9e9594740 1 -- 172.21.15.72:0/266841214 >> 172.21.15.72:0/266841214 conn(0x557a96e3fd10 msgr2=0x557a96ea1df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:55:12.929 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.928+0000 7fe9e9594740 1 -- 172.21.15.72:0/266841214 shutdown_connections 2024-04-26T21:55:12.929 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.928+0000 7fe9e9594740 1 -- 172.21.15.72:0/266841214 wait complete. 2024-04-26T21:55:12.932 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.928+0000 7fe9e9594740 1 -- start start 2024-04-26T21:55:12.932 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.929+0000 7fe9e9594740 1 Processor -- start 2024-04-26T21:55:12.932 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.929+0000 7fe9e9594740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557a96eab2c0 0x557a96eb8ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:12.932 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.929+0000 7fe9e9594740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557a96eabcb0 0x557a96eb9220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:12.932 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.929+0000 7fe9e9594740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557a96eb0e20 0x557a96ebc1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:12.933 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.929+0000 7fe9e9594740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x557a96ec3210 con 0x557a96eb0e20 2024-04-26T21:55:12.933 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.929+0000 7fe9e9594740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x557a96ec2b40 con 0x557a96eab2c0 2024-04-26T21:55:12.933 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.929+0000 7fe9e9594740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x557a96ec38e0 con 0x557a96eabcb0 2024-04-26T21:55:12.933 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.929+0000 7fe9e9594740 10 client.0 _ll_register_callbacks cb 0x7fe9c800df00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:55:12.933 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.929+0000 7fe9dfe00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557a96eb0e20 0x557a96ebc1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:12.933 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.929+0000 7fe9dfe00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557a96eb0e20 0x557a96ebc1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:34338/0 (socket says 172.21.15.72:34338) 2024-04-26T21:55:12.933 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.929+0000 7fe9dfe00640 1 -- 172.21.15.72:0/1982837344 learned_addr learned my addr 172.21.15.72:0/1982837344 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:55:12.933 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.930+0000 7fe9df400640 1 --2- 172.21.15.72:0/1982837344 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557a96eab2c0 0x557a96eb8ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:12.933 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.930+0000 7fe9dea00640 1 --2- 172.21.15.72:0/1982837344 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557a96eabcb0 0x557a96eb9220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:12.933 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.930+0000 7fe9dfe00640 1 -- 172.21.15.72:0/1982837344 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557a96eabcb0 msgr2=0x557a96eb9220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:55:12.933 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.930+0000 7fe9dfe00640 1 --2- 172.21.15.72:0/1982837344 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557a96eabcb0 0x557a96eb9220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:12.933 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.930+0000 7fe9dfe00640 1 -- 172.21.15.72:0/1982837344 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557a96eab2c0 msgr2=0x557a96eb8ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:55:12.933 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.930+0000 7fe9dfe00640 1 --2- 172.21.15.72:0/1982837344 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557a96eab2c0 0x557a96eb8ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:12.933 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.930+0000 7fe9dfe00640 1 -- 172.21.15.72:0/1982837344 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a96ebc7c0 con 0x557a96eb0e20 2024-04-26T21:55:12.934 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.930+0000 7fe9dfe00640 1 --2- 172.21.15.72:0/1982837344 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557a96eb0e20 0x557a96ebc1d0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fe9d00dcfd0 tx=0x7fe9d002a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edd1debc23a85c6c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:55:12.934 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.930+0000 7fe9cd600640 10 client.0 ms_handle_connect on v2:172.21.15.72:3300/0 2024-04-26T21:55:12.934 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.930+0000 7fe9cd600640 1 -- 172.21.15.72:0/1982837344 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9d00da020 con 0x557a96eb0e20 2024-04-26T21:55:12.934 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.930+0000 7fe9cd600640 1 -- 172.21.15.72:0/1982837344 <== mon.0 v2:172.21.15.72:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7fe9d00b9470 con 0x557a96eb0e20 2024-04-26T21:55:12.934 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.930+0000 7fe9e9594740 1 -- 172.21.15.72:0/1982837344 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe9d00bce00 con 0x557a96eb0e20 2024-04-26T21:55:12.934 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.930+0000 7fe9cd600640 1 -- 172.21.15.72:0/1982837344 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe9d0049050 con 0x557a96eb0e20 2024-04-26T21:55:12.934 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.931+0000 7fe9e9594740 10 client.15363 fetch_fsmap learned FSMap version 26 2024-04-26T21:55:12.934 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.931+0000 7fe9e9594740 1 -- 172.21.15.72:0/1982837344 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x557a96f28730 con 0x557a96eb0e20 2024-04-26T21:55:12.934 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.931+0000 7fe9cd600640 1 -- 172.21.15.72:0/1982837344 <== mon.0 v2:172.21.15.72:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe9d00bce00 con 0x557a96eb0e20 2024-04-26T21:55:12.934 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.931+0000 7fe9cd600640 1 -- 172.21.15.72:0/1982837344 <== mon.0 v2:172.21.15.72:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fe9d00bd0f0 con 0x557a96eb0e20 2024-04-26T21:55:12.934 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.931+0000 7fe9e9594740 10 client.15363 fetch_fsmap finished waiting for FSMap version 26 2024-04-26T21:55:12.934 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.931+0000 7fe9e9594740 10 client.15363 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-26T21:55:12.934 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.931+0000 7fe9e9594740 20 client.15363 populate_metadata read hostname 'smithi072' 2024-04-26T21:55:12.934 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.931+0000 7fe9e9594740 1 --2- 172.21.15.72:0/1982837344 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x557a96f2d060 0x557a96f2bd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:12.935 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.931+0000 7fe9e9594740 4 client.15363 mds_command: new command op to 34517 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T21:55:12.935 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.931+0000 7fe9e9594740 1 -- 172.21.15.72:0/1982837344 --> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] -- command(tid 0: {"prefix": "status"}) -- 0x557a96e17080 con 0x557a96f2d060 2024-04-26T21:55:12.935 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.932+0000 7fe9df400640 1 --2- 172.21.15.72:0/1982837344 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x557a96f2d060 0x557a96f2bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:12.936 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.933+0000 7fe9cd600640 10 client.15363 ms_handle_connect on v2:172.21.15.76:6838/327145636 2024-04-26T21:55:12.936 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.933+0000 7fe9df400640 1 --2- 172.21.15.72:0/1982837344 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x557a96f2d060 0x557a96f2bd70 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f44ddffff8268cbb server_cookie=c32a64141b7a8276 in_seq=0 out_seq=0 2024-04-26T21:55:12.936 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.933+0000 7fe9cd600640 1 -- 172.21.15.72:0/1982837344 <== mds.0 v2:172.21.15.76:6838/327145636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x557a96e17080 con 0x557a96f2d060 2024-04-26T21:55:12.936 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.933+0000 7fe9cd600640 10 client.15363 handle_command_reply: tid=0 2024-04-26T21:55:12.936 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.934+0000 7fe9e9594740 1 client.15363 shutdown 2024-04-26T21:55:12.936 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.934+0000 7fe9e9594740 1 -- 172.21.15.72:0/1982837344 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557a96eb0e20 msgr2=0x557a96ebc1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:12.936 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.934+0000 7fe9e9594740 1 --2- 172.21.15.72:0/1982837344 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557a96eb0e20 0x557a96ebc1d0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fe9d00dcfd0 tx=0x7fe9d002a410 comp rx=0 tx=0).stop 2024-04-26T21:55:12.936 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.934+0000 7fe9e9594740 1 -- 172.21.15.72:0/1982837344 shutdown_connections 2024-04-26T21:55:12.936 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.934+0000 7fe9e9594740 1 --2- 172.21.15.72:0/1982837344 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x557a96f2d060 0x557a96f2bd70 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:12.936 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.934+0000 7fe9e9594740 1 --2- 172.21.15.72:0/1982837344 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557a96eb0e20 0x557a96ebc1d0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:12.937 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.934+0000 7fe9e9594740 1 --2- 172.21.15.72:0/1982837344 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557a96eabcb0 0x557a96eb9220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:12.937 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.934+0000 7fe9e9594740 1 --2- 172.21.15.72:0/1982837344 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557a96eab2c0 0x557a96eb8ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:12.937 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.934+0000 7fe9cd600640 0 client.15363 ms_handle_reset on v2:172.21.15.76:6838/327145636 2024-04-26T21:55:12.937 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.934+0000 7fe9e9594740 1 -- 172.21.15.72:0/1982837344 >> 172.21.15.72:0/1982837344 conn(0x557a96e3fd10 msgr2=0x557a96eb5390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:55:12.937 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.934+0000 7fe9e9594740 1 -- 172.21.15.72:0/1982837344 shutdown_connections 2024-04-26T21:55:12.937 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.934+0000 7fe9e9594740 1 -- 172.21.15.72:0/1982837344 wait complete. 2024-04-26T21:55:12.937 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.934+0000 7fe9e9594740 20 client.15363 trim_cache size 0 max 16384 2024-04-26T21:55:12.937 INFO:teuthology.orchestra.run.smithi072.stdout:{ 2024-04-26T21:55:12.937 INFO:teuthology.orchestra.run.smithi072.stdout: "cluster_fsid": "a1780950-0415-11ef-bc93-c7b262605968", 2024-04-26T21:55:12.937 INFO:teuthology.orchestra.run.smithi072.stdout: "whoami": 0, 2024-04-26T21:55:12.937 INFO:teuthology.orchestra.run.smithi072.stdout: "id": 34517, 2024-04-26T21:55:12.938 INFO:teuthology.orchestra.run.smithi072.stdout: "want_state": "up:active", 2024-04-26T21:55:12.938 INFO:teuthology.orchestra.run.smithi072.stdout: "state": "up:active", 2024-04-26T21:55:12.938 INFO:teuthology.orchestra.run.smithi072.stdout: "fs_name": "cephfs", 2024-04-26T21:55:12.938 INFO:teuthology.orchestra.run.smithi072.stdout: "rank_uptime": 141.83229076399999, 2024-04-26T21:55:12.938 INFO:teuthology.orchestra.run.smithi072.stdout: "mdsmap_epoch": 26, 2024-04-26T21:55:12.938 INFO:teuthology.orchestra.run.smithi072.stdout: "osdmap_epoch": 91, 2024-04-26T21:55:12.938 INFO:teuthology.orchestra.run.smithi072.stdout: "osdmap_epoch_barrier": 87, 2024-04-26T21:55:12.938 INFO:teuthology.orchestra.run.smithi072.stdout: "uptime": 148.10890338199999 2024-04-26T21:55:12.938 INFO:teuthology.orchestra.run.smithi072.stdout:} 2024-04-26T21:55:12.939 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.935+0000 7fe9e5000640 1 -- 172.21.15.72:0/2259273244 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fe9c40a3cc0 msgr2=0x7fe9c40a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:12.939 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.935+0000 7fe9e5000640 1 --2- 172.21.15.72:0/2259273244 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fe9c40a3cc0 0x7fe9c40a6180 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x557a96eac0b0 tx=0x7fe9d8009580 comp rx=0 tx=0).stop 2024-04-26T21:55:12.939 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.935+0000 7fe9e5000640 1 -- 172.21.15.72:0/2259273244 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe9e0104310 msgr2=0x7fe9e01a2050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:12.939 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.935+0000 7fe9e5000640 1 --2- 172.21.15.72:0/2259273244 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe9e0104310 0x7fe9e01a2050 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fe9d0004770 tx=0x7fe9d00042f0 comp rx=0 tx=0).stop 2024-04-26T21:55:12.939 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.935+0000 7fe9e5000640 1 -- 172.21.15.72:0/2259273244 shutdown_connections 2024-04-26T21:55:12.939 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.935+0000 7fe9e5000640 1 --2- 172.21.15.72:0/2259273244 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe9e0104310 0x7fe9e01a2050 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:12.939 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.935+0000 7fe9e5000640 1 --2- 172.21.15.72:0/2259273244 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fe9c40a3cc0 0x7fe9c40a6180 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:12.939 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.935+0000 7fe9e5000640 1 --2- 172.21.15.72:0/2259273244 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe9e0103950 0x7fe9e019b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:12.939 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.935+0000 7fe9e5000640 1 --2- 172.21.15.72:0/2259273244 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe9e0102760 0x7fe9e019aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:12.940 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.935+0000 7fe9e5000640 1 -- 172.21.15.72:0/2259273244 >> 172.21.15.72:0/2259273244 conn(0x7fe9e00fdcb0 msgr2=0x7fe9e0074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:55:12.940 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.935+0000 7fe9e5000640 1 -- 172.21.15.72:0/2259273244 shutdown_connections 2024-04-26T21:55:12.940 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:12.935+0000 7fe9e5000640 1 -- 172.21.15.72:0/2259273244 wait complete. 2024-04-26T21:55:12.961 INFO:teuthology.run_tasks:Running task sequential... 2024-04-26T21:55:12.971 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-04-26T21:55:12.971 INFO:teuthology.task.sequential:In sequential, running task print... 2024-04-26T21:55:12.971 INFO:teuthology.task.print:Enabling mgr modules 2024-04-26T21:55:12.971 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-04-26T21:55:13.023 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-04-26T21:55:13.023 INFO:tasks.ceph_fuse:config is {'client.0': {}} 2024-04-26T21:55:13.023 INFO:tasks.ceph_fuse:client.0 config is {'mount_subvol_num': 0} 2024-04-26T21:55:13.023 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-26T21:55:13.023 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-26T21:55:13.023 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:55:13.023 DEBUG:teuthology.orchestra.run.smithi072:> ip netns list 2024-04-26T21:55:13.076 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:55:13.076 DEBUG:teuthology.orchestra.run.smithi072:> sudo ip link delete ceph-brx 2024-04-26T21:55:13.163 INFO:teuthology.orchestra.run.smithi072.stderr:Cannot find device "ceph-brx" 2024-04-26T21:55:13.166 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T21:55:13.167 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-04-26T21:55:13.167 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid a1780950-0415-11ef-bc93-c7b262605968 -- ceph fs ls 2024-04-26T21:55:13.599 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/a1780950-0415-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:55:15.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:55:14 smithi072 ceph-mon[28138]: pgmap v443: 97 pgs: 97 active+clean; 633 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-26T21:55:15.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:55:14 smithi164 ceph-mon[31916]: pgmap v443: 97 pgs: 97 active+clean; 633 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-26T21:55:15.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:55:14 smithi076 ceph-mon[31823]: pgmap v443: 97 pgs: 97 active+clean; 633 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-26T21:55:16.016 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:16.015+0000 7f28f4c00700 1 -- 172.21.15.72:0/163699732 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f28f0103e30 msgr2=0x7f28f0110290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:16.016 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:16.015+0000 7f28f4c00700 1 --2- 172.21.15.72:0/163699732 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f28f0103e30 0x7f28f0110290 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f28e800b0a0 tx=0x7f28e800b3b0 comp rx=0 tx=0).stop 2024-04-26T21:55:16.016 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:16.015+0000 7f28f4c00700 1 -- 172.21.15.72:0/163699732 shutdown_connections 2024-04-26T21:55:16.016 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:16.015+0000 7f28f4c00700 1 --2- 172.21.15.72:0/163699732 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f28f0103e30 0x7f28f0110290 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:16.016 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:16.015+0000 7f28f4c00700 1 --2- 172.21.15.72:0/163699732 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f28f01094c0 0x7f28f01098a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:16.016 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:16.015+0000 7f28f4c00700 1 --2- 172.21.15.72:0/163699732 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f28f0103470 0x7f28f01038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:16.016 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:16.015+0000 7f28f4c00700 1 -- 172.21.15.72:0/163699732 >> 172.21.15.72:0/163699732 conn(0x7f28f007a580 msgr2=0x7f28f007a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:55:16.016 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:16.015+0000 7f28f4c00700 1 -- 172.21.15.72:0/163699732 shutdown_connections 2024-04-26T21:55:16.016 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:16.015+0000 7f28f4c00700 1 -- 172.21.15.72:0/163699732 wait complete. 2024-04-26T21:55:16.017 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:16.016+0000 7f28f4c00700 1 Processor -- start 2024-04-26T21:55:16.017 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:16.016+0000 7f28f4c00700 1 -- start start 2024-04-26T21:55:16.018 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:16.017+0000 7f28f4c00700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f28f0103470 0x7f28f019f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:16.018 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:16.017+0000 7f28f4c00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f28f0103e30 0x7f28f019fb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:16.018 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:16.017+0000 7f28f4c00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f28f01094c0 0x7f28f01997d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:16.018 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:16.017+0000 7f28f4c00700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f28f0113470 con 0x7f28f0103e30 2024-04-26T21:55:16.018 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:16.017+0000 7f28f4c00700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f28f01132f0 con 0x7f28f0103470 2024-04-26T21:55:16.018 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:16.017+0000 7f28f4c00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f28f01135f0 con 0x7f28f01094c0 2024-04-26T21:55:16.018 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:16.017+0000 7f28ed400700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f28f0103e30 0x7f28f019fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:16.018 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:16.017+0000 7f28ed400700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f28f0103e30 0x7f28f019fb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:34356/0 (socket says 172.21.15.72:34356) 2024-04-26T21:55:16.018 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:16.017+0000 7f28ed400700 1 -- 172.21.15.72:0/1406773818 learned_addr learned my addr 172.21.15.72:0/1406773818 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:55:16.019 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:16.017+0000 7f28ede00700 1 --2- 172.21.15.72:0/1406773818 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f28f0103470 0x7f28f019f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:16.019 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:16.018+0000 7f28ee800700 1 --2- 172.21.15.72:0/1406773818 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f28f01094c0 0x7f28f01997d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:16.019 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:16.018+0000 7f28ed400700 1 -- 172.21.15.72:0/1406773818 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f28f01094c0 msgr2=0x7f28f01997d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:16.019 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:16.018+0000 7f28ed400700 1 --2- 172.21.15.72:0/1406773818 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f28f01094c0 0x7f28f01997d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:16.019 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:16.018+0000 7f28ed400700 1 -- 172.21.15.72:0/1406773818 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f28f0103470 msgr2=0x7f28f019f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:16.019 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:16.018+0000 7f28ed400700 1 --2- 172.21.15.72:0/1406773818 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f28f0103470 0x7f28f019f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:16.019 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:16.018+0000 7f28ed400700 1 -- 172.21.15.72:0/1406773818 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28f0199fb0 con 0x7f28f0103e30 2024-04-26T21:55:16.019 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:16.018+0000 7f28ee800700 1 --2- 172.21.15.72:0/1406773818 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f28f01094c0 0x7f28f01997d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:55:16.019 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:16.018+0000 7f28ed400700 1 --2- 172.21.15.72:0/1406773818 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f28f0103e30 0x7f28f019fb90 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f28dc00d950 tx=0x7f28dc00dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:55:16.020 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:16.018+0000 7f28e6a00700 1 -- 172.21.15.72:0/1406773818 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28dc016070 con 0x7f28f0103e30 2024-04-26T21:55:16.020 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:16.018+0000 7f28e6a00700 1 -- 172.21.15.72:0/1406773818 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f28dc010460 con 0x7f28f0103e30 2024-04-26T21:55:16.020 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:16.018+0000 7f28f4c00700 1 -- 172.21.15.72:0/1406773818 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28f019a240 con 0x7f28f0103e30 2024-04-26T21:55:16.020 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:16.018+0000 7f28e6a00700 1 -- 172.21.15.72:0/1406773818 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28dc00f620 con 0x7f28f0103e30 2024-04-26T21:55:16.020 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:16.019+0000 7f28f4c00700 1 -- 172.21.15.72:0/1406773818 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28f01a64b0 con 0x7f28f0103e30 2024-04-26T21:55:16.020 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:16.020+0000 7f28f4c00700 1 -- 172.21.15.72:0/1406773818 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28f0104b60 con 0x7f28f0103e30 2024-04-26T21:55:16.022 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:16.022+0000 7f28e6a00700 1 -- 172.21.15.72:0/1406773818 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f28dc019070 con 0x7f28f0103e30 2024-04-26T21:55:16.024 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:16.024+0000 7f28e6a00700 1 --2- 172.21.15.72:0/1406773818 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f28d40a3cd0 0x7f28d40a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:16.025 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:16.025+0000 7f28e6a00700 1 -- 172.21.15.72:0/1406773818 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7f28dc0d0260 con 0x7f28f0103e30 2024-04-26T21:55:16.025 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:16.025+0000 7f28ede00700 1 --2- 172.21.15.72:0/1406773818 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f28d40a3cd0 0x7f28d40a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:16.025 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:16.026+0000 7f28ede00700 1 --2- 172.21.15.72:0/1406773818 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f28d40a3cd0 0x7f28d40a6190 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f28e0005950 tx=0x7f28e00058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:55:16.029 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:16.029+0000 7f28e6a00700 1 -- 172.21.15.72:0/1406773818 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f28dc09ca00 con 0x7f28f0103e30 2024-04-26T21:55:16.304 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:16.303+0000 7f28f4c00700 1 -- 172.21.15.72:0/1406773818 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f28f002e490 con 0x7f28f0103e30 2024-04-26T21:55:16.304 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:16.304+0000 7f28e6a00700 1 -- 172.21.15.72:0/1406773818 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v26) ==== 53+0+73 (secure 0 0 0) 0x7f28dc09c300 con 0x7f28f0103e30 2024-04-26T21:55:16.304 INFO:teuthology.orchestra.run.smithi072.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-26T21:55:16.308 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:16.308+0000 7f28f4c00700 1 -- 172.21.15.72:0/1406773818 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f28d40a3cd0 msgr2=0x7f28d40a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:16.308 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:16.308+0000 7f28f4c00700 1 --2- 172.21.15.72:0/1406773818 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f28d40a3cd0 0x7f28d40a6190 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f28e0005950 tx=0x7f28e00058e0 comp rx=0 tx=0).stop 2024-04-26T21:55:16.308 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:16.308+0000 7f28f4c00700 1 -- 172.21.15.72:0/1406773818 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f28f0103e30 msgr2=0x7f28f019fb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:16.308 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:16.308+0000 7f28f4c00700 1 --2- 172.21.15.72:0/1406773818 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f28f0103e30 0x7f28f019fb90 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f28dc00d950 tx=0x7f28dc00dd10 comp rx=0 tx=0).stop 2024-04-26T21:55:16.308 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:16.308+0000 7f28f4c00700 1 -- 172.21.15.72:0/1406773818 shutdown_connections 2024-04-26T21:55:16.308 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:16.308+0000 7f28f4c00700 1 --2- 172.21.15.72:0/1406773818 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f28d40a3cd0 0x7f28d40a6190 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:16.308 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:16.308+0000 7f28f4c00700 1 --2- 172.21.15.72:0/1406773818 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f28f01094c0 0x7f28f01997d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:16.309 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:16.308+0000 7f28f4c00700 1 --2- 172.21.15.72:0/1406773818 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f28f0103470 0x7f28f019f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:16.309 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:16.308+0000 7f28f4c00700 1 --2- 172.21.15.72:0/1406773818 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f28f0103e30 0x7f28f019fb90 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:16.309 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:16.308+0000 7f28f4c00700 1 -- 172.21.15.72:0/1406773818 >> 172.21.15.72:0/1406773818 conn(0x7f28f007a580 msgr2=0x7f28f0102360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:55:16.309 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:16.308+0000 7f28f4c00700 1 -- 172.21.15.72:0/1406773818 shutdown_connections 2024-04-26T21:55:16.309 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:16.309+0000 7f28f4c00700 1 -- 172.21.15.72:0/1406773818 wait complete. 2024-04-26T21:55:17.018 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:55:16 smithi072 ceph-mon[28138]: pgmap v444: 97 pgs: 97 active+clean; 633 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-26T21:55:17.018 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:55:16 smithi072 ceph-mon[28138]: from='client.? 172.21.15.72:0/1406773818' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-26T21:55:17.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:55:16 smithi164 ceph-mon[31916]: pgmap v444: 97 pgs: 97 active+clean; 633 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-26T21:55:17.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:55:16 smithi164 ceph-mon[31916]: from='client.? 172.21.15.72:0/1406773818' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-26T21:55:17.281 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-26T21:55:17.282 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-26T21:55:17.282 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi072.front.sepia.ceph.com 2024-04-26T21:55:17.282 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-26T21:55:17.282 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-26T21:55:17.282 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-26T21:55:17.282 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-26T21:55:17.282 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-26T21:55:17.282 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-26T21:55:17.285 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:55:17.285 DEBUG:teuthology.orchestra.run.smithi072:> ip addr 2024-04-26T21:55:17.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:55:16 smithi076 ceph-mon[31823]: pgmap v444: 97 pgs: 97 active+clean; 633 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-26T21:55:17.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:55:16 smithi076 ceph-mon[31823]: from='client.? 172.21.15.72:0/1406773818' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-26T21:55:17.336 INFO:teuthology.orchestra.run.smithi072.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T21:55:17.336 INFO:teuthology.orchestra.run.smithi072.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T21:55:17.336 INFO:teuthology.orchestra.run.smithi072.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T21:55:17.336 INFO:teuthology.orchestra.run.smithi072.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:55:17.336 INFO:teuthology.orchestra.run.smithi072.stdout: inet6 ::1/128 scope host 2024-04-26T21:55:17.336 INFO:teuthology.orchestra.run.smithi072.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:55:17.336 INFO:teuthology.orchestra.run.smithi072.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:55:17.336 INFO:teuthology.orchestra.run.smithi072.stdout: link/ether 0c:c4:7a:d8:da:f4 brd ff:ff:ff:ff:ff:ff 2024-04-26T21:55:17.336 INFO:teuthology.orchestra.run.smithi072.stdout: altname enp7s0f0 2024-04-26T21:55:17.336 INFO:teuthology.orchestra.run.smithi072.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:55:17.336 INFO:teuthology.orchestra.run.smithi072.stdout: link/ether 0c:c4:7a:d8:da:f5 brd ff:ff:ff:ff:ff:ff 2024-04-26T21:55:17.337 INFO:teuthology.orchestra.run.smithi072.stdout: altname enp7s0f1 2024-04-26T21:55:17.337 INFO:teuthology.orchestra.run.smithi072.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:55:17.337 INFO:teuthology.orchestra.run.smithi072.stdout: link/ether 0c:c4:7a:8f:b7:64 brd ff:ff:ff:ff:ff:ff 2024-04-26T21:55:17.337 INFO:teuthology.orchestra.run.smithi072.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-26T21:55:17.337 INFO:teuthology.orchestra.run.smithi072.stdout: link/ether 0c:c4:7a:8f:b7:65 brd ff:ff:ff:ff:ff:ff 2024-04-26T21:55:17.337 INFO:teuthology.orchestra.run.smithi072.stdout: inet 172.21.15.72/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-26T21:55:17.337 INFO:teuthology.orchestra.run.smithi072.stdout: valid_lft 41789sec preferred_lft 41789sec 2024-04-26T21:55:17.337 INFO:teuthology.orchestra.run.smithi072.stdout: inet6 fe80::ec4:7aff:fe8f:b765/64 scope link 2024-04-26T21:55:17.337 INFO:teuthology.orchestra.run.smithi072.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:55:17.337 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-26T21:55:17.338 DEBUG:teuthology.orchestra.run.smithi072:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T21:55:17.338 DEBUG:teuthology.orchestra.run.smithi072:> set -e 2024-04-26T21:55:17.338 DEBUG:teuthology.orchestra.run.smithi072:> sudo ip link add name ceph-brx type bridge 2024-04-26T21:55:17.338 DEBUG:teuthology.orchestra.run.smithi072:> sudo ip addr flush dev ceph-brx 2024-04-26T21:55:17.338 DEBUG:teuthology.orchestra.run.smithi072:> sudo ip link set ceph-brx up 2024-04-26T21:55:17.338 DEBUG:teuthology.orchestra.run.smithi072:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-26T21:55:17.338 DEBUG:teuthology.orchestra.run.smithi072:> ') 2024-04-26T21:55:17.425 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:17 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T21:55:17.949 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:55:17.961 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:55:17.962 DEBUG:teuthology.orchestra.run.smithi072:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-26T21:55:18.019 INFO:teuthology.orchestra.run.smithi072.stdout:1 2024-04-26T21:55:18.023 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:55:18.023 DEBUG:teuthology.orchestra.run.smithi072:> route 2024-04-26T21:55:18.077 INFO:teuthology.orchestra.run.smithi072.stdout:Kernel IP routing table 2024-04-26T21:55:18.077 INFO:teuthology.orchestra.run.smithi072.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-26T21:55:18.077 INFO:teuthology.orchestra.run.smithi072.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-26T21:55:18.077 INFO:teuthology.orchestra.run.smithi072.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-26T21:55:18.078 INFO:teuthology.orchestra.run.smithi072.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-26T21:55:18.079 DEBUG:teuthology.orchestra.run.smithi072:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T21:55:18.079 DEBUG:teuthology.orchestra.run.smithi072:> set -e 2024-04-26T21:55:18.079 DEBUG:teuthology.orchestra.run.smithi072:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-26T21:55:18.079 DEBUG:teuthology.orchestra.run.smithi072:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-26T21:55:18.079 DEBUG:teuthology.orchestra.run.smithi072:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-26T21:55:18.079 DEBUG:teuthology.orchestra.run.smithi072:> ') 2024-04-26T21:55:18.175 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:18 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T21:55:19.121 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:55:19.132 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:55:19.132 DEBUG:teuthology.orchestra.run.smithi072:> ip netns list 2024-04-26T21:55:19.188 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:55:19.188 DEBUG:teuthology.orchestra.run.smithi072:> ip netns list-id 2024-04-26T21:55:19.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:55:18 smithi164 ceph-mon[31916]: pgmap v445: 97 pgs: 97 active+clean; 633 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-26T21:55:19.239 DEBUG:teuthology.orchestra.run.smithi072:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T21:55:19.239 DEBUG:teuthology.orchestra.run.smithi072:> set -e 2024-04-26T21:55:19.239 DEBUG:teuthology.orchestra.run.smithi072:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T21:55:19.239 DEBUG:teuthology.orchestra.run.smithi072:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-26T21:55:19.239 DEBUG:teuthology.orchestra.run.smithi072:> ') 2024-04-26T21:55:19.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:55:18 smithi076 ceph-mon[31823]: pgmap v445: 97 pgs: 97 active+clean; 633 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-26T21:55:19.321 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:55:18 smithi072 ceph-mon[28138]: pgmap v445: 97 pgs: 97 active+clean; 633 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-26T21:55:19.338 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:19 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T21:55:19.407 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:55:19.417 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-26T21:55:19.417 DEBUG:teuthology.orchestra.run.smithi072:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T21:55:19.417 DEBUG:teuthology.orchestra.run.smithi072:> set -e 2024-04-26T21:55:19.417 DEBUG:teuthology.orchestra.run.smithi072:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-26T21:55:19.417 DEBUG:teuthology.orchestra.run.smithi072:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-26T21:55:19.417 DEBUG:teuthology.orchestra.run.smithi072:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-26T21:55:19.417 DEBUG:teuthology.orchestra.run.smithi072:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-26T21:55:19.417 DEBUG:teuthology.orchestra.run.smithi072:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-26T21:55:19.417 DEBUG:teuthology.orchestra.run.smithi072:> ') 2024-04-26T21:55:19.519 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:19 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T21:55:19.793 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:55:19.803 DEBUG:teuthology.orchestra.run.smithi072:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T21:55:19.803 DEBUG:teuthology.orchestra.run.smithi072:> set -e 2024-04-26T21:55:19.803 DEBUG:teuthology.orchestra.run.smithi072:> sudo ip link set brx.0 up 2024-04-26T21:55:19.803 DEBUG:teuthology.orchestra.run.smithi072:> sudo ip link set dev brx.0 master ceph-brx 2024-04-26T21:55:19.803 DEBUG:teuthology.orchestra.run.smithi072:> ') 2024-04-26T21:55:19.895 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:19 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T21:55:19.997 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:55:20.006 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {'mount_subvol_num': 0} 2024-04-26T21:55:20.007 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T21:55:20.007 DEBUG:teuthology.orchestra.run.smithi072:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-26T21:55:20.047 INFO:teuthology.orchestra.run.smithi072.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-26T21:55:20.047 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T21:55:20.047 DEBUG:teuthology.orchestra.run.smithi072:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-26T21:55:20.142 DEBUG:teuthology.orchestra.run.smithi072:> sudo modprobe fuse 2024-04-26T21:55:20.231 DEBUG:teuthology.orchestra.run.smithi072:> cat /proc/self/mounts | awk '{print $2}' 2024-04-26T21:55:20.285 INFO:teuthology.orchestra.run.smithi072.stdout:/proc 2024-04-26T21:55:20.286 INFO:teuthology.orchestra.run.smithi072.stdout:/sys 2024-04-26T21:55:20.286 INFO:teuthology.orchestra.run.smithi072.stdout:/dev 2024-04-26T21:55:20.286 INFO:teuthology.orchestra.run.smithi072.stdout:/sys/kernel/security 2024-04-26T21:55:20.286 INFO:teuthology.orchestra.run.smithi072.stdout:/dev/shm 2024-04-26T21:55:20.286 INFO:teuthology.orchestra.run.smithi072.stdout:/dev/pts 2024-04-26T21:55:20.286 INFO:teuthology.orchestra.run.smithi072.stdout:/run 2024-04-26T21:55:20.286 INFO:teuthology.orchestra.run.smithi072.stdout:/sys/fs/cgroup 2024-04-26T21:55:20.286 INFO:teuthology.orchestra.run.smithi072.stdout:/sys/fs/pstore 2024-04-26T21:55:20.286 INFO:teuthology.orchestra.run.smithi072.stdout:/sys/fs/bpf 2024-04-26T21:55:20.286 INFO:teuthology.orchestra.run.smithi072.stdout:/ 2024-04-26T21:55:20.286 INFO:teuthology.orchestra.run.smithi072.stdout:/sys/fs/selinux 2024-04-26T21:55:20.287 INFO:teuthology.orchestra.run.smithi072.stdout:/proc/sys/fs/binfmt_misc 2024-04-26T21:55:20.287 INFO:teuthology.orchestra.run.smithi072.stdout:/dev/mqueue 2024-04-26T21:55:20.287 INFO:teuthology.orchestra.run.smithi072.stdout:/dev/hugepages 2024-04-26T21:55:20.287 INFO:teuthology.orchestra.run.smithi072.stdout:/sys/kernel/debug 2024-04-26T21:55:20.287 INFO:teuthology.orchestra.run.smithi072.stdout:/sys/kernel/tracing 2024-04-26T21:55:20.287 INFO:teuthology.orchestra.run.smithi072.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-26T21:55:20.287 INFO:teuthology.orchestra.run.smithi072.stdout:/sys/fs/fuse/connections 2024-04-26T21:55:20.287 INFO:teuthology.orchestra.run.smithi072.stdout:/sys/kernel/config 2024-04-26T21:55:20.287 INFO:teuthology.orchestra.run.smithi072.stdout:/run/credentials/systemd-sysctl.service 2024-04-26T21:55:20.287 INFO:teuthology.orchestra.run.smithi072.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-26T21:55:20.287 INFO:teuthology.orchestra.run.smithi072.stdout:/var/lib/nfs/rpc_pipefs 2024-04-26T21:55:20.287 INFO:teuthology.orchestra.run.smithi072.stdout:/run/user/1000 2024-04-26T21:55:20.287 INFO:teuthology.orchestra.run.smithi072.stdout:/var/lib/ceph 2024-04-26T21:55:20.287 INFO:teuthology.orchestra.run.smithi072.stdout:/var/lib/containers/storage/overlay 2024-04-26T21:55:20.288 INFO:teuthology.orchestra.run.smithi072.stdout:/var/lib/containers/storage/overlay/fc10633a6d40068493c7dfddd017320787b69b18da25fde4000fafcf27d247e0/merged 2024-04-26T21:55:20.288 INFO:teuthology.orchestra.run.smithi072.stdout:/var/lib/containers/storage/overlay/2ec2fde413c8408d1bc1f16441821759667c84aa089da73c28bd2bd433c0a260/merged 2024-04-26T21:55:20.288 INFO:teuthology.orchestra.run.smithi072.stdout:/run/user/0 2024-04-26T21:55:20.288 INFO:teuthology.orchestra.run.smithi072.stdout:/proc/sys/fs/binfmt_misc 2024-04-26T21:55:20.288 INFO:teuthology.orchestra.run.smithi072.stdout:/var/lib/containers/storage/overlay/30c53f8127e8dc9f9ac2105b666f786acc8b70a23c7de14ef9db58f7906cc494/merged 2024-04-26T21:55:20.288 INFO:teuthology.orchestra.run.smithi072.stdout:/var/lib/containers/storage/overlay/1995c4009fccdcd981ce20c39a62beaf018ffb29f1f07cbf8a63d752d7d49a45/merged 2024-04-26T21:55:20.288 INFO:teuthology.orchestra.run.smithi072.stdout:/var/lib/containers/storage/overlay/508b2729647d93be927b6dbbdefe8ed0f6195d21fd27badf6591b6bacb50411c/merged 2024-04-26T21:55:20.288 INFO:teuthology.orchestra.run.smithi072.stdout:/var/lib/containers/storage/overlay/16de90379b1bac42cc1e6c1e40aae28eb42126ee4f66d77f3a0cd69828a69dc4/merged 2024-04-26T21:55:20.288 INFO:teuthology.orchestra.run.smithi072.stdout:/var/lib/containers/storage/overlay/970b64e22f7265dc4f12c18b19afa4d7aa53184fd4a59546adb4ae8d4e975587/merged 2024-04-26T21:55:20.288 INFO:teuthology.orchestra.run.smithi072.stdout:/var/lib/containers/storage/overlay/4204b3eb66b7de0f9298c94fc5d0fcd9ae3237b1e1221f068db9278a0b120594/merged 2024-04-26T21:55:20.288 INFO:teuthology.orchestra.run.smithi072.stdout:/var/lib/containers/storage/overlay/aba746a21fc88dc5f32e1172cd29925901bb88caf596beede12f956fe6bfe4d6/merged 2024-04-26T21:55:20.288 INFO:teuthology.orchestra.run.smithi072.stdout:/var/lib/containers/storage/overlay/02f3eceff14af239489105a4d3c82a344f42133958d4e5859d3a36424d74ac76/merged 2024-04-26T21:55:20.288 INFO:teuthology.orchestra.run.smithi072.stdout:/run/netns 2024-04-26T21:55:20.288 INFO:teuthology.orchestra.run.smithi072.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T21:55:20.289 INFO:teuthology.orchestra.run.smithi072.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T21:55:20.289 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:55:20.290 DEBUG:teuthology.orchestra.run.smithi072:> ls /sys/fs/fuse/connections 2024-04-26T21:55:20.337 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-26T21:55:20.338 DEBUG:teuthology.orchestra.run.smithi072:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0 --client_mountpoint=/volumes/qa/sv_0/2b6b095b-1e64-4476-8fef-65001d22a864 --client_fs=cephfs) 2024-04-26T21:55:20.346 DEBUG:teuthology.orchestra.run.smithi072:> sudo modprobe fuse 2024-04-26T21:55:20.421 DEBUG:teuthology.orchestra.run.smithi072:> cat /proc/self/mounts | awk '{print $2}' 2024-04-26T21:55:20.461 INFO:teuthology.orchestra.run.smithi072.stdout:/proc 2024-04-26T21:55:20.461 INFO:teuthology.orchestra.run.smithi072.stdout:/sys 2024-04-26T21:55:20.461 INFO:teuthology.orchestra.run.smithi072.stdout:/dev 2024-04-26T21:55:20.461 INFO:teuthology.orchestra.run.smithi072.stdout:/sys/kernel/security 2024-04-26T21:55:20.461 INFO:teuthology.orchestra.run.smithi072.stdout:/dev/shm 2024-04-26T21:55:20.461 INFO:teuthology.orchestra.run.smithi072.stdout:/dev/pts 2024-04-26T21:55:20.461 INFO:teuthology.orchestra.run.smithi072.stdout:/run 2024-04-26T21:55:20.462 INFO:teuthology.orchestra.run.smithi072.stdout:/sys/fs/cgroup 2024-04-26T21:55:20.462 INFO:teuthology.orchestra.run.smithi072.stdout:/sys/fs/pstore 2024-04-26T21:55:20.462 INFO:teuthology.orchestra.run.smithi072.stdout:/sys/fs/bpf 2024-04-26T21:55:20.462 INFO:teuthology.orchestra.run.smithi072.stdout:/ 2024-04-26T21:55:20.462 INFO:teuthology.orchestra.run.smithi072.stdout:/sys/fs/selinux 2024-04-26T21:55:20.462 INFO:teuthology.orchestra.run.smithi072.stdout:/proc/sys/fs/binfmt_misc 2024-04-26T21:55:20.462 INFO:teuthology.orchestra.run.smithi072.stdout:/dev/mqueue 2024-04-26T21:55:20.462 INFO:teuthology.orchestra.run.smithi072.stdout:/dev/hugepages 2024-04-26T21:55:20.462 INFO:teuthology.orchestra.run.smithi072.stdout:/sys/kernel/debug 2024-04-26T21:55:20.462 INFO:teuthology.orchestra.run.smithi072.stdout:/sys/kernel/tracing 2024-04-26T21:55:20.462 INFO:teuthology.orchestra.run.smithi072.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-26T21:55:20.462 INFO:teuthology.orchestra.run.smithi072.stdout:/sys/fs/fuse/connections 2024-04-26T21:55:20.462 INFO:teuthology.orchestra.run.smithi072.stdout:/sys/kernel/config 2024-04-26T21:55:20.462 INFO:teuthology.orchestra.run.smithi072.stdout:/run/credentials/systemd-sysctl.service 2024-04-26T21:55:20.463 INFO:teuthology.orchestra.run.smithi072.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-26T21:55:20.463 INFO:teuthology.orchestra.run.smithi072.stdout:/var/lib/nfs/rpc_pipefs 2024-04-26T21:55:20.463 INFO:teuthology.orchestra.run.smithi072.stdout:/run/user/1000 2024-04-26T21:55:20.463 INFO:teuthology.orchestra.run.smithi072.stdout:/var/lib/ceph 2024-04-26T21:55:20.463 INFO:teuthology.orchestra.run.smithi072.stdout:/var/lib/containers/storage/overlay 2024-04-26T21:55:20.463 INFO:teuthology.orchestra.run.smithi072.stdout:/var/lib/containers/storage/overlay/fc10633a6d40068493c7dfddd017320787b69b18da25fde4000fafcf27d247e0/merged 2024-04-26T21:55:20.463 INFO:teuthology.orchestra.run.smithi072.stdout:/var/lib/containers/storage/overlay/2ec2fde413c8408d1bc1f16441821759667c84aa089da73c28bd2bd433c0a260/merged 2024-04-26T21:55:20.463 INFO:teuthology.orchestra.run.smithi072.stdout:/run/user/0 2024-04-26T21:55:20.463 INFO:teuthology.orchestra.run.smithi072.stdout:/proc/sys/fs/binfmt_misc 2024-04-26T21:55:20.463 INFO:teuthology.orchestra.run.smithi072.stdout:/var/lib/containers/storage/overlay/30c53f8127e8dc9f9ac2105b666f786acc8b70a23c7de14ef9db58f7906cc494/merged 2024-04-26T21:55:20.463 INFO:teuthology.orchestra.run.smithi072.stdout:/var/lib/containers/storage/overlay/1995c4009fccdcd981ce20c39a62beaf018ffb29f1f07cbf8a63d752d7d49a45/merged 2024-04-26T21:55:20.463 INFO:teuthology.orchestra.run.smithi072.stdout:/var/lib/containers/storage/overlay/508b2729647d93be927b6dbbdefe8ed0f6195d21fd27badf6591b6bacb50411c/merged 2024-04-26T21:55:20.463 INFO:teuthology.orchestra.run.smithi072.stdout:/var/lib/containers/storage/overlay/16de90379b1bac42cc1e6c1e40aae28eb42126ee4f66d77f3a0cd69828a69dc4/merged 2024-04-26T21:55:20.464 INFO:teuthology.orchestra.run.smithi072.stdout:/var/lib/containers/storage/overlay/970b64e22f7265dc4f12c18b19afa4d7aa53184fd4a59546adb4ae8d4e975587/merged 2024-04-26T21:55:20.464 INFO:teuthology.orchestra.run.smithi072.stdout:/var/lib/containers/storage/overlay/4204b3eb66b7de0f9298c94fc5d0fcd9ae3237b1e1221f068db9278a0b120594/merged 2024-04-26T21:55:20.464 INFO:teuthology.orchestra.run.smithi072.stdout:/var/lib/containers/storage/overlay/aba746a21fc88dc5f32e1172cd29925901bb88caf596beede12f956fe6bfe4d6/merged 2024-04-26T21:55:20.464 INFO:teuthology.orchestra.run.smithi072.stdout:/var/lib/containers/storage/overlay/02f3eceff14af239489105a4d3c82a344f42133958d4e5859d3a36424d74ac76/merged 2024-04-26T21:55:20.464 INFO:teuthology.orchestra.run.smithi072.stdout:/run/netns 2024-04-26T21:55:20.464 INFO:teuthology.orchestra.run.smithi072.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T21:55:20.464 INFO:teuthology.orchestra.run.smithi072.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T21:55:20.465 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:55:20.465 DEBUG:teuthology.orchestra.run.smithi072:> ls /sys/fs/fuse/connections 2024-04-26T21:55:20.540 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi072.stderr:2024-04-26T21:55:20.539+0000 7fd28ab38480 -1 init, newargv = 0x55c64c645680 newargc=15 2024-04-26T21:55:20.540 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi072.stderr:ceph-fuse[81138]: starting ceph client 2024-04-26T21:55:20.607 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi072.stderr:ceph-fuse[81138]: starting fuse 2024-04-26T21:55:21.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:55:20 smithi164 ceph-mon[31916]: pgmap v446: 97 pgs: 97 active+clean; 633 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-26T21:55:21.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:55:20 smithi076 ceph-mon[31823]: pgmap v446: 97 pgs: 97 active+clean; 633 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-26T21:55:21.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:55:20 smithi072 ceph-mon[28138]: pgmap v446: 97 pgs: 97 active+clean; 633 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-26T21:55:21.648 DEBUG:teuthology.orchestra.run.smithi072:> sudo modprobe fuse 2024-04-26T21:55:21.743 DEBUG:teuthology.orchestra.run.smithi072:> cat /proc/self/mounts | awk '{print $2}' 2024-04-26T21:55:21.798 INFO:teuthology.orchestra.run.smithi072.stdout:/proc 2024-04-26T21:55:21.799 INFO:teuthology.orchestra.run.smithi072.stdout:/sys 2024-04-26T21:55:21.799 INFO:teuthology.orchestra.run.smithi072.stdout:/dev 2024-04-26T21:55:21.799 INFO:teuthology.orchestra.run.smithi072.stdout:/sys/kernel/security 2024-04-26T21:55:21.799 INFO:teuthology.orchestra.run.smithi072.stdout:/dev/shm 2024-04-26T21:55:21.799 INFO:teuthology.orchestra.run.smithi072.stdout:/dev/pts 2024-04-26T21:55:21.799 INFO:teuthology.orchestra.run.smithi072.stdout:/run 2024-04-26T21:55:21.799 INFO:teuthology.orchestra.run.smithi072.stdout:/sys/fs/cgroup 2024-04-26T21:55:21.799 INFO:teuthology.orchestra.run.smithi072.stdout:/sys/fs/pstore 2024-04-26T21:55:21.799 INFO:teuthology.orchestra.run.smithi072.stdout:/sys/fs/bpf 2024-04-26T21:55:21.800 INFO:teuthology.orchestra.run.smithi072.stdout:/ 2024-04-26T21:55:21.800 INFO:teuthology.orchestra.run.smithi072.stdout:/sys/fs/selinux 2024-04-26T21:55:21.800 INFO:teuthology.orchestra.run.smithi072.stdout:/proc/sys/fs/binfmt_misc 2024-04-26T21:55:21.800 INFO:teuthology.orchestra.run.smithi072.stdout:/dev/mqueue 2024-04-26T21:55:21.800 INFO:teuthology.orchestra.run.smithi072.stdout:/dev/hugepages 2024-04-26T21:55:21.800 INFO:teuthology.orchestra.run.smithi072.stdout:/sys/kernel/debug 2024-04-26T21:55:21.800 INFO:teuthology.orchestra.run.smithi072.stdout:/sys/kernel/tracing 2024-04-26T21:55:21.800 INFO:teuthology.orchestra.run.smithi072.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-26T21:55:21.800 INFO:teuthology.orchestra.run.smithi072.stdout:/sys/fs/fuse/connections 2024-04-26T21:55:21.800 INFO:teuthology.orchestra.run.smithi072.stdout:/sys/kernel/config 2024-04-26T21:55:21.801 INFO:teuthology.orchestra.run.smithi072.stdout:/run/credentials/systemd-sysctl.service 2024-04-26T21:55:21.801 INFO:teuthology.orchestra.run.smithi072.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-26T21:55:21.801 INFO:teuthology.orchestra.run.smithi072.stdout:/var/lib/nfs/rpc_pipefs 2024-04-26T21:55:21.801 INFO:teuthology.orchestra.run.smithi072.stdout:/run/user/1000 2024-04-26T21:55:21.801 INFO:teuthology.orchestra.run.smithi072.stdout:/var/lib/ceph 2024-04-26T21:55:21.801 INFO:teuthology.orchestra.run.smithi072.stdout:/var/lib/containers/storage/overlay 2024-04-26T21:55:21.801 INFO:teuthology.orchestra.run.smithi072.stdout:/var/lib/containers/storage/overlay/fc10633a6d40068493c7dfddd017320787b69b18da25fde4000fafcf27d247e0/merged 2024-04-26T21:55:21.801 INFO:teuthology.orchestra.run.smithi072.stdout:/var/lib/containers/storage/overlay/2ec2fde413c8408d1bc1f16441821759667c84aa089da73c28bd2bd433c0a260/merged 2024-04-26T21:55:21.801 INFO:teuthology.orchestra.run.smithi072.stdout:/run/user/0 2024-04-26T21:55:21.801 INFO:teuthology.orchestra.run.smithi072.stdout:/proc/sys/fs/binfmt_misc 2024-04-26T21:55:21.802 INFO:teuthology.orchestra.run.smithi072.stdout:/var/lib/containers/storage/overlay/30c53f8127e8dc9f9ac2105b666f786acc8b70a23c7de14ef9db58f7906cc494/merged 2024-04-26T21:55:21.802 INFO:teuthology.orchestra.run.smithi072.stdout:/var/lib/containers/storage/overlay/1995c4009fccdcd981ce20c39a62beaf018ffb29f1f07cbf8a63d752d7d49a45/merged 2024-04-26T21:55:21.802 INFO:teuthology.orchestra.run.smithi072.stdout:/var/lib/containers/storage/overlay/508b2729647d93be927b6dbbdefe8ed0f6195d21fd27badf6591b6bacb50411c/merged 2024-04-26T21:55:21.802 INFO:teuthology.orchestra.run.smithi072.stdout:/var/lib/containers/storage/overlay/16de90379b1bac42cc1e6c1e40aae28eb42126ee4f66d77f3a0cd69828a69dc4/merged 2024-04-26T21:55:21.802 INFO:teuthology.orchestra.run.smithi072.stdout:/var/lib/containers/storage/overlay/970b64e22f7265dc4f12c18b19afa4d7aa53184fd4a59546adb4ae8d4e975587/merged 2024-04-26T21:55:21.802 INFO:teuthology.orchestra.run.smithi072.stdout:/var/lib/containers/storage/overlay/4204b3eb66b7de0f9298c94fc5d0fcd9ae3237b1e1221f068db9278a0b120594/merged 2024-04-26T21:55:21.802 INFO:teuthology.orchestra.run.smithi072.stdout:/var/lib/containers/storage/overlay/aba746a21fc88dc5f32e1172cd29925901bb88caf596beede12f956fe6bfe4d6/merged 2024-04-26T21:55:21.802 INFO:teuthology.orchestra.run.smithi072.stdout:/var/lib/containers/storage/overlay/02f3eceff14af239489105a4d3c82a344f42133958d4e5859d3a36424d74ac76/merged 2024-04-26T21:55:21.802 INFO:teuthology.orchestra.run.smithi072.stdout:/run/netns 2024-04-26T21:55:21.802 INFO:teuthology.orchestra.run.smithi072.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T21:55:21.802 INFO:teuthology.orchestra.run.smithi072.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T21:55:21.802 INFO:teuthology.orchestra.run.smithi072.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-26T21:55:21.803 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:55:21.803 DEBUG:teuthology.orchestra.run.smithi072:> ls /sys/fs/fuse/connections 2024-04-26T21:55:21.855 INFO:teuthology.orchestra.run.smithi072.stdout:114 2024-04-26T21:55:21.856 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [114] 2024-04-26T21:55:21.856 DEBUG:teuthology.orchestra.run.smithi072:> sudo stdin-killer -- python3 -c ' 2024-04-26T21:55:21.856 DEBUG:teuthology.orchestra.run.smithi072:> import glob 2024-04-26T21:55:21.856 DEBUG:teuthology.orchestra.run.smithi072:> import re 2024-04-26T21:55:21.856 DEBUG:teuthology.orchestra.run.smithi072:> import os 2024-04-26T21:55:21.856 DEBUG:teuthology.orchestra.run.smithi072:> import subprocess 2024-04-26T21:55:21.856 DEBUG:teuthology.orchestra.run.smithi072:> 2024-04-26T21:55:21.856 DEBUG:teuthology.orchestra.run.smithi072:> def _find_admin_socket(client_name): 2024-04-26T21:55:21.856 DEBUG:teuthology.orchestra.run.smithi072:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-26T21:55:21.856 DEBUG:teuthology.orchestra.run.smithi072:> files = glob.glob(asok_path) 2024-04-26T21:55:21.856 DEBUG:teuthology.orchestra.run.smithi072:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-26T21:55:21.857 DEBUG:teuthology.orchestra.run.smithi072:> 2024-04-26T21:55:21.857 DEBUG:teuthology.orchestra.run.smithi072:> # Given a non-glob path, it better be there 2024-04-26T21:55:21.857 DEBUG:teuthology.orchestra.run.smithi072:> if "*" not in asok_path: 2024-04-26T21:55:21.857 DEBUG:teuthology.orchestra.run.smithi072:> assert(len(files) == 1) 2024-04-26T21:55:21.857 DEBUG:teuthology.orchestra.run.smithi072:> return files[0] 2024-04-26T21:55:21.857 DEBUG:teuthology.orchestra.run.smithi072:> 2024-04-26T21:55:21.857 DEBUG:teuthology.orchestra.run.smithi072:> for f in files: 2024-04-26T21:55:21.857 DEBUG:teuthology.orchestra.run.smithi072:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-26T21:55:21.857 DEBUG:teuthology.orchestra.run.smithi072:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-26T21:55:21.857 DEBUG:teuthology.orchestra.run.smithi072:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-26T21:55:21.857 DEBUG:teuthology.orchestra.run.smithi072:> contents = proc_f.read() 2024-04-26T21:55:21.857 DEBUG:teuthology.orchestra.run.smithi072:> if mountpoint in contents: 2024-04-26T21:55:21.857 DEBUG:teuthology.orchestra.run.smithi072:> return f 2024-04-26T21:55:21.857 DEBUG:teuthology.orchestra.run.smithi072:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-26T21:55:21.857 DEBUG:teuthology.orchestra.run.smithi072:> 2024-04-26T21:55:21.857 DEBUG:teuthology.orchestra.run.smithi072:> print(_find_admin_socket("client.0")) 2024-04-26T21:55:21.857 DEBUG:teuthology.orchestra.run.smithi072:> ' 2024-04-26T21:55:22.025 INFO:teuthology.orchestra.run.smithi072.stdout:/var/run/ceph/ceph-client.0.81138.asok 2024-04-26T21:55:22.029 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:55:22.042 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.81138.asok 2024-04-26T21:55:22.043 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:55:22.043 DEBUG:teuthology.orchestra.run.smithi072:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.81138.asok status 2024-04-26T21:55:22.264 INFO:teuthology.orchestra.run.smithi072.stdout:{ 2024-04-26T21:55:22.265 INFO:teuthology.orchestra.run.smithi072.stdout: "metadata": { 2024-04-26T21:55:22.265 INFO:teuthology.orchestra.run.smithi072.stdout: "ceph_sha1": "bc612ffb9c1d5bc27536bc917a705262a36d4387", 2024-04-26T21:55:22.265 INFO:teuthology.orchestra.run.smithi072.stdout: "ceph_version": "ceph version 19.0.0-3306-gbc612ffb (bc612ffb9c1d5bc27536bc917a705262a36d4387) squid (dev)", 2024-04-26T21:55:22.265 INFO:teuthology.orchestra.run.smithi072.stdout: "entity_id": "0", 2024-04-26T21:55:22.265 INFO:teuthology.orchestra.run.smithi072.stdout: "hostname": "smithi072", 2024-04-26T21:55:22.265 INFO:teuthology.orchestra.run.smithi072.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-26T21:55:22.265 INFO:teuthology.orchestra.run.smithi072.stdout: "pid": "81138", 2024-04-26T21:55:22.265 INFO:teuthology.orchestra.run.smithi072.stdout: "root": "/volumes/qa/sv_0/2b6b095b-1e64-4476-8fef-65001d22a864" 2024-04-26T21:55:22.265 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:55:22.265 INFO:teuthology.orchestra.run.smithi072.stdout: "dentry_count": 0, 2024-04-26T21:55:22.265 INFO:teuthology.orchestra.run.smithi072.stdout: "dentry_pinned_count": 0, 2024-04-26T21:55:22.265 INFO:teuthology.orchestra.run.smithi072.stdout: "id": 15375, 2024-04-26T21:55:22.265 INFO:teuthology.orchestra.run.smithi072.stdout: "inst": { 2024-04-26T21:55:22.265 INFO:teuthology.orchestra.run.smithi072.stdout: "name": { 2024-04-26T21:55:22.266 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "client", 2024-04-26T21:55:22.266 INFO:teuthology.orchestra.run.smithi072.stdout: "num": 15375 2024-04-26T21:55:22.266 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:55:22.266 INFO:teuthology.orchestra.run.smithi072.stdout: "addr": { 2024-04-26T21:55:22.266 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "v1", 2024-04-26T21:55:22.266 INFO:teuthology.orchestra.run.smithi072.stdout: "addr": "192.168.0.1:0", 2024-04-26T21:55:22.266 INFO:teuthology.orchestra.run.smithi072.stdout: "nonce": 939650736 2024-04-26T21:55:22.266 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:55:22.266 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:55:22.266 INFO:teuthology.orchestra.run.smithi072.stdout: "addr": { 2024-04-26T21:55:22.266 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "v1", 2024-04-26T21:55:22.266 INFO:teuthology.orchestra.run.smithi072.stdout: "addr": "192.168.0.1:0", 2024-04-26T21:55:22.266 INFO:teuthology.orchestra.run.smithi072.stdout: "nonce": 939650736 2024-04-26T21:55:22.266 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:55:22.267 INFO:teuthology.orchestra.run.smithi072.stdout: "inst_str": "client.15375 192.168.0.1:0/939650736", 2024-04-26T21:55:22.267 INFO:teuthology.orchestra.run.smithi072.stdout: "addr_str": "192.168.0.1:0/939650736", 2024-04-26T21:55:22.267 INFO:teuthology.orchestra.run.smithi072.stdout: "inode_count": 5, 2024-04-26T21:55:22.267 INFO:teuthology.orchestra.run.smithi072.stdout: "mds_epoch": 26, 2024-04-26T21:55:22.267 INFO:teuthology.orchestra.run.smithi072.stdout: "osd_epoch": 91, 2024-04-26T21:55:22.267 INFO:teuthology.orchestra.run.smithi072.stdout: "osd_epoch_barrier": 0, 2024-04-26T21:55:22.267 INFO:teuthology.orchestra.run.smithi072.stdout: "blocklisted": false, 2024-04-26T21:55:22.267 INFO:teuthology.orchestra.run.smithi072.stdout: "fs_name": "cephfs" 2024-04-26T21:55:22.267 INFO:teuthology.orchestra.run.smithi072.stdout:} 2024-04-26T21:55:22.278 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:55:22.279 DEBUG:teuthology.orchestra.run.smithi072:> stat --file-system '--printf=%T 2024-04-26T21:55:22.279 DEBUG:teuthology.orchestra.run.smithi072:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-26T21:55:22.330 INFO:teuthology.orchestra.run.smithi072.stdout:fuseblk 2024-04-26T21:55:22.330 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-26T21:55:22.330 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:55:22.330 DEBUG:teuthology.orchestra.run.smithi072:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-26T21:55:22.414 INFO:teuthology.run_tasks:Running task exec... 2024-04-26T21:55:22.425 INFO:teuthology.task.exec:Executing custom commands... 2024-04-26T21:55:22.425 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi072.front.sepia.ceph.com 2024-04-26T21:55:22.426 DEBUG:teuthology.orchestra.run.smithi072:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate true' 2024-04-26T21:55:22.630 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:22.623+0000 7f50fdc00640 1 -- 172.21.15.72:0/2221025785 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f50f8105da0 msgr2=0x7f50f8106220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:22.630 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:22.623+0000 7f50fdc00640 1 --2- 172.21.15.72:0/2221025785 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f50f8105da0 0x7f50f8106220 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f50e0009900 tx=0x7f50e002a3f0 comp rx=0 tx=0).stop 2024-04-26T21:55:22.630 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:22.623+0000 7f50fdc00640 1 -- 172.21.15.72:0/2221025785 shutdown_connections 2024-04-26T21:55:22.631 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:22.623+0000 7f50fdc00640 1 --2- 172.21.15.72:0/2221025785 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f50f8106760 0x7f50f810cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:22.631 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:22.623+0000 7f50fdc00640 1 --2- 172.21.15.72:0/2221025785 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f50f8105da0 0x7f50f8106220 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:22.632 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:22.623+0000 7f50fdc00640 1 --2- 172.21.15.72:0/2221025785 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f50f8104bb0 0x7f50f8104fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:22.632 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:22.623+0000 7f50fdc00640 1 -- 172.21.15.72:0/2221025785 >> 172.21.15.72:0/2221025785 conn(0x7f50f8100120 msgr2=0x7f50f8102560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:55:22.632 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:22.623+0000 7f50fdc00640 1 -- 172.21.15.72:0/2221025785 shutdown_connections 2024-04-26T21:55:22.632 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:22.623+0000 7f50fdc00640 1 -- 172.21.15.72:0/2221025785 wait complete. 2024-04-26T21:55:22.632 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:22.624+0000 7f50fdc00640 1 Processor -- start 2024-04-26T21:55:22.632 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:22.624+0000 7f50fdc00640 1 -- start start 2024-04-26T21:55:22.633 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:22.625+0000 7f50fdc00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f50f8104bb0 0x7f50f819ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:22.633 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:22.625+0000 7f50fdc00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f50f8105da0 0x7f50f819b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:22.633 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:22.625+0000 7f50fdc00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f50f8106760 0x7f50f81a21b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:22.633 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:22.625+0000 7f50fdc00640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f50f8110640 con 0x7f50f8105da0 2024-04-26T21:55:22.633 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:22.625+0000 7f50fdc00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f50f81104c0 con 0x7f50f8104bb0 2024-04-26T21:55:22.633 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:22.625+0000 7f50fdc00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f50f81107c0 con 0x7f50f8106760 2024-04-26T21:55:22.633 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:22.625+0000 7f50ee000640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f50f8105da0 0x7f50f819b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:22.634 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:22.625+0000 7f50ee000640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f50f8105da0 0x7f50f819b180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:44022/0 (socket says 172.21.15.72:44022) 2024-04-26T21:55:22.634 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:22.625+0000 7f50ee000640 1 -- 172.21.15.72:0/1265220790 learned_addr learned my addr 172.21.15.72:0/1265220790 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:55:22.634 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:22.625+0000 7f50f7400640 1 --2- 172.21.15.72:0/1265220790 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f50f8106760 0x7f50f81a21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:22.634 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:22.625+0000 7f50f6a00640 1 --2- 172.21.15.72:0/1265220790 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f50f8104bb0 0x7f50f819ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:22.634 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:22.626+0000 7f50ee000640 1 -- 172.21.15.72:0/1265220790 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f50f8106760 msgr2=0x7f50f81a21b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:22.634 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:22.626+0000 7f50ee000640 1 --2- 172.21.15.72:0/1265220790 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f50f8106760 0x7f50f81a21b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:22.635 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:22.626+0000 7f50ee000640 1 -- 172.21.15.72:0/1265220790 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f50f8104bb0 msgr2=0x7f50f819ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:22.635 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:22.626+0000 7f50ee000640 1 --2- 172.21.15.72:0/1265220790 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f50f8104bb0 0x7f50f819ac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:22.635 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:22.626+0000 7f50ee000640 1 -- 172.21.15.72:0/1265220790 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50f81a2850 con 0x7f50f8105da0 2024-04-26T21:55:22.635 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:22.626+0000 7f50ee000640 1 --2- 172.21.15.72:0/1265220790 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f50f8105da0 0x7f50f819b180 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f50e0031460 tx=0x7f50e0003d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:55:22.635 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:22.626+0000 7f50efe00640 1 -- 172.21.15.72:0/1265220790 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50e003f070 con 0x7f50f8105da0 2024-04-26T21:55:22.635 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:22.626+0000 7f50efe00640 1 -- 172.21.15.72:0/1265220790 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f50e0031da0 con 0x7f50f8105da0 2024-04-26T21:55:22.636 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:22.626+0000 7f50fdc00640 1 -- 172.21.15.72:0/1265220790 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50f81a2ae0 con 0x7f50f8105da0 2024-04-26T21:55:22.636 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:22.626+0000 7f50efe00640 1 -- 172.21.15.72:0/1265220790 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50e0035030 con 0x7f50f8105da0 2024-04-26T21:55:22.636 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:22.626+0000 7f50fdc00640 1 -- 172.21.15.72:0/1265220790 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50f81a3020 con 0x7f50f8105da0 2024-04-26T21:55:22.636 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:22.627+0000 7f50fdc00640 1 -- 172.21.15.72:0/1265220790 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50f810be70 con 0x7f50f8105da0 2024-04-26T21:55:22.638 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:22.629+0000 7f50efe00640 1 -- 172.21.15.72:0/1265220790 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f50e0042070 con 0x7f50f8105da0 2024-04-26T21:55:22.638 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:22.631+0000 7f50efe00640 1 --2- 172.21.15.72:0/1265220790 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f50dc0a3d70 0x7f50dc0a6230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:22.638 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:22.631+0000 7f50efe00640 1 -- 172.21.15.72:0/1265220790 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7f50e00f10f0 con 0x7f50f8105da0 2024-04-26T21:55:22.639 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:22.631+0000 7f50f6a00640 1 --2- 172.21.15.72:0/1265220790 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f50dc0a3d70 0x7f50dc0a6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:22.639 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:22.632+0000 7f50f6a00640 1 --2- 172.21.15.72:0/1265220790 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f50dc0a3d70 0x7f50dc0a6230 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f50e8005fd0 tx=0x7f50e8005d00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:55:22.639 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:22.636+0000 7f50efe00640 1 -- 172.21.15.72:0/1265220790 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f50e00bd890 con 0x7f50f8105da0 2024-04-26T21:55:22.892 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:22.891+0000 7f50fdc00640 1 -- 172.21.15.72:0/1265220790 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} v 0) -- 0x7f50f802dad0 con 0x7f50f8105da0 2024-04-26T21:55:22.908 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:22.907+0000 7f50efe00640 1 -- 172.21.15.72:0/1265220790 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]=0 v27) ==== 117+0+0 (secure 0 0 0) 0x7f50e00bd190 con 0x7f50f8105da0 2024-04-26T21:55:22.912 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:22.910+0000 7f50fdc00640 1 -- 172.21.15.72:0/1265220790 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f50dc0a3d70 msgr2=0x7f50dc0a6230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:22.912 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:22.910+0000 7f50fdc00640 1 --2- 172.21.15.72:0/1265220790 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f50dc0a3d70 0x7f50dc0a6230 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f50e8005fd0 tx=0x7f50e8005d00 comp rx=0 tx=0).stop 2024-04-26T21:55:22.912 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:22.911+0000 7f50fdc00640 1 -- 172.21.15.72:0/1265220790 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f50f8105da0 msgr2=0x7f50f819b180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:22.912 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:22.911+0000 7f50fdc00640 1 --2- 172.21.15.72:0/1265220790 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f50f8105da0 0x7f50f819b180 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f50e0031460 tx=0x7f50e0003d50 comp rx=0 tx=0).stop 2024-04-26T21:55:22.912 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:22.911+0000 7f50fdc00640 1 -- 172.21.15.72:0/1265220790 shutdown_connections 2024-04-26T21:55:22.912 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:22.911+0000 7f50fdc00640 1 --2- 172.21.15.72:0/1265220790 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f50dc0a3d70 0x7f50dc0a6230 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:22.912 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:22.911+0000 7f50fdc00640 1 --2- 172.21.15.72:0/1265220790 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f50f8106760 0x7f50f81a21b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:22.912 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:22.911+0000 7f50fdc00640 1 --2- 172.21.15.72:0/1265220790 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f50f8105da0 0x7f50f819b180 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:22.912 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:22.911+0000 7f50fdc00640 1 --2- 172.21.15.72:0/1265220790 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f50f8104bb0 0x7f50f819ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:22.913 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:22.911+0000 7f50fdc00640 1 -- 172.21.15.72:0/1265220790 >> 172.21.15.72:0/1265220790 conn(0x7f50f8100120 msgr2=0x7f50f8109040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:55:22.913 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:22.911+0000 7f50fdc00640 1 -- 172.21.15.72:0/1265220790 shutdown_connections 2024-04-26T21:55:22.913 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:22.911+0000 7f50fdc00640 1 -- 172.21.15.72:0/1265220790 wait complete. 2024-04-26T21:55:22.930 INFO:teuthology.run_tasks:Running task check-counter... 2024-04-26T21:55:22.940 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-04-26T21:55:22.950 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:55:23.142 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:23.141+0000 7f284b200640 1 -- 172.21.15.72:0/4287140155 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f284c108450 msgr2=0x7f284c10a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:23.143 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:23.141+0000 7f284b200640 1 --2- 172.21.15.72:0/4287140155 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f284c108450 0x7f284c10a8a0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f2844009ca0 tx=0x7f284402a8c0 comp rx=0 tx=0).stop 2024-04-26T21:55:23.143 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:23.141+0000 7f284b200640 1 -- 172.21.15.72:0/4287140155 shutdown_connections 2024-04-26T21:55:23.143 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:23.141+0000 7f284b200640 1 --2- 172.21.15.72:0/4287140155 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f284c108450 0x7f284c10a8a0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:23.143 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:23.141+0000 7f284b200640 1 --2- 172.21.15.72:0/4287140155 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f284c105b20 0x7f284c107f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:23.143 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:23.141+0000 7f284b200640 1 --2- 172.21.15.72:0/4287140155 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f284c1031f0 0x7f284c1055e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:23.143 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:23.141+0000 7f284b200640 1 -- 172.21.15.72:0/4287140155 >> 172.21.15.72:0/4287140155 conn(0x7f284c0fcbc0 msgr2=0x7f284c0ff020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:55:23.143 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:23.141+0000 7f284b200640 1 -- 172.21.15.72:0/4287140155 shutdown_connections 2024-04-26T21:55:23.143 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:23.141+0000 7f284b200640 1 -- 172.21.15.72:0/4287140155 wait complete. 2024-04-26T21:55:23.145 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:23.142+0000 7f284b200640 1 Processor -- start 2024-04-26T21:55:23.145 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:23.142+0000 7f284b200640 1 -- start start 2024-04-26T21:55:23.145 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:23.142+0000 7f284b200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f284c1031f0 0x7f284c102b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:23.145 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:23.142+0000 7f284b200640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f284c105b20 0x7f284c1011a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:23.146 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:23.142+0000 7f284b200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f284c108450 0x7f284c1016e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:23.146 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:23.142+0000 7f284b200640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f284c10e1a0 con 0x7f284c105b20 2024-04-26T21:55:23.146 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:23.142+0000 7f284b200640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f284c10e020 con 0x7f284c108450 2024-04-26T21:55:23.146 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:23.142+0000 7f284b200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f284c10e320 con 0x7f284c1031f0 2024-04-26T21:55:23.146 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:23.143+0000 7f2843e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f284c105b20 0x7f284c1011a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:23.146 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:23.143+0000 7f2843e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f284c105b20 0x7f284c1011a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:44040/0 (socket says 172.21.15.72:44040) 2024-04-26T21:55:23.146 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:23.143+0000 7f2843e00640 1 -- 172.21.15.72:0/3966882745 learned_addr learned my addr 172.21.15.72:0/3966882745 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:55:23.146 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:23.143+0000 7f2849e00640 1 --2- 172.21.15.72:0/3966882745 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f284c1031f0 0x7f284c102b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:23.146 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:23.143+0000 7f284a800640 1 --2- 172.21.15.72:0/3966882745 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f284c108450 0x7f284c1016e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:23.146 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:23.143+0000 7f2843e00640 1 -- 172.21.15.72:0/3966882745 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f284c1031f0 msgr2=0x7f284c102b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:23.146 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:23.143+0000 7f2843e00640 1 --2- 172.21.15.72:0/3966882745 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f284c1031f0 0x7f284c102b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:23.146 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:23.143+0000 7f2843e00640 1 -- 172.21.15.72:0/3966882745 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f284c108450 msgr2=0x7f284c1016e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:23.146 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:23.143+0000 7f2843e00640 1 --2- 172.21.15.72:0/3966882745 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f284c108450 0x7f284c1016e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:23.147 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:23.143+0000 7f2843e00640 1 -- 172.21.15.72:0/3966882745 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f284c101f70 con 0x7f284c105b20 2024-04-26T21:55:23.147 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:23.144+0000 7f2843e00640 1 --2- 172.21.15.72:0/3966882745 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f284c105b20 0x7f284c1011a0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f283400c970 tx=0x7f283400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:55:23.147 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:23.144+0000 7f2842a00640 1 -- 172.21.15.72:0/3966882745 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2834014070 con 0x7f284c105b20 2024-04-26T21:55:23.147 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:23.144+0000 7f2842a00640 1 -- 172.21.15.72:0/3966882745 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2834004440 con 0x7f284c105b20 2024-04-26T21:55:23.147 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:23.144+0000 7f284b200640 1 -- 172.21.15.72:0/3966882745 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f284c1a28e0 con 0x7f284c105b20 2024-04-26T21:55:23.147 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:23.144+0000 7f2842a00640 1 -- 172.21.15.72:0/3966882745 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2834003d40 con 0x7f284c105b20 2024-04-26T21:55:23.147 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:23.144+0000 7f284b200640 1 -- 172.21.15.72:0/3966882745 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f284c1a2e80 con 0x7f284c105b20 2024-04-26T21:55:23.147 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:23.145+0000 7f284b200640 1 -- 172.21.15.72:0/3966882745 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f284c0fe6b0 con 0x7f284c105b20 2024-04-26T21:55:23.148 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:23.147+0000 7f2842a00640 1 -- 172.21.15.72:0/3966882745 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f2834017070 con 0x7f284c105b20 2024-04-26T21:55:23.149 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:23.148+0000 7f2842a00640 1 --2- 172.21.15.72:0/3966882745 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f28300a3cc0 0x7f28300a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:23.149 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:23.149+0000 7f2842a00640 1 -- 172.21.15.72:0/3966882745 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7f28340d0540 con 0x7f284c105b20 2024-04-26T21:55:23.150 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:23.149+0000 7f2849e00640 1 --2- 172.21.15.72:0/3966882745 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f28300a3cc0 0x7f28300a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:23.150 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:23.149+0000 7f2849e00640 1 --2- 172.21.15.72:0/3966882745 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f28300a3cc0 0x7f28300a6180 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f283c005fd0 tx=0x7f283c005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:55:23.153 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:23.153+0000 7f2842a00640 1 -- 172.21.15.72:0/3966882745 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f283401e020 con 0x7f284c105b20 2024-04-26T21:55:23.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:55:22 smithi164 ceph-mon[31916]: pgmap v447: 97 pgs: 97 active+clean; 633 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-26T21:55:23.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:55:22 smithi164 ceph-mon[31916]: from='client.? 172.21.15.72:0/1265220790' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-04-26T21:55:23.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:55:22 smithi076 ceph-mon[31823]: pgmap v447: 97 pgs: 97 active+clean; 633 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-26T21:55:23.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:55:22 smithi076 ceph-mon[31823]: from='client.? 172.21.15.72:0/1265220790' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-04-26T21:55:23.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:55:22 smithi072 ceph-mon[28138]: pgmap v447: 97 pgs: 97 active+clean; 633 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-26T21:55:23.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:55:22 smithi072 ceph-mon[28138]: from='client.? 172.21.15.72:0/1265220790' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-04-26T21:55:23.358 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:23.357+0000 7f284b200640 1 -- 172.21.15.72:0/3966882745 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f284c063530 con 0x7f284c105b20 2024-04-26T21:55:23.362 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:23.361+0000 7f2842a00640 1 -- 172.21.15.72:0/3966882745 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+22783 (secure 0 0 0) 0x7f2834098030 con 0x7f284c105b20 2024-04-26T21:55:23.362 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T21:55:23.363 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":91,"fsid":"a1780950-0415-11ef-bc93-c7b262605968","created":"2024-04-26T21:42:01.504173+0000","modified":"2024-04-26T21:54:11.892872+0000","last_up_change":"2024-04-26T21:51:43.482437+0000","last_in_change":"2024-04-26T21:51:18.468587+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:46:29.785427+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:52:48.320391+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T21:52:49.517493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"04e06884-b355-4762-9b29-9034b7201e9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6802","nonce":1416765179},{"type":"v1","addr":"172.21.15.72:6803","nonce":1416765179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6804","nonce":1416765179},{"type":"v1","addr":"172.21.15.72:6805","nonce":1416765179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6808","nonce":1416765179},{"type":"v1","addr":"172.21.15.72:6809","nonce":1416765179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6806","nonce":1416765179},{"type":"v1","addr":"172.21.15.72:6807","nonce":1416765179}]},"public_addr":"172.21.15.72:6803/1416765179","cluster_addr":"172.21.15.72:6805/1416765179","heartbeat_back_addr":"172.21.15.72:6809/1416765179","heartbeat_front_addr":"172.21.15.72:6807/1416765179","state":["exists","up"]},{"osd":1,"uuid":"d75a479a-4a2e-48e9-a01e-2b1c9617416a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":755426949},{"type":"v1","addr":"172.21.15.76:6801","nonce":755426949}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":755426949},{"type":"v1","addr":"172.21.15.76:6803","nonce":755426949}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":755426949},{"type":"v1","addr":"172.21.15.76:6807","nonce":755426949}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":755426949},{"type":"v1","addr":"172.21.15.76:6805","nonce":755426949}]},"public_addr":"172.21.15.76:6801/755426949","cluster_addr":"172.21.15.76:6803/755426949","heartbeat_back_addr":"172.21.15.76:6807/755426949","heartbeat_front_addr":"172.21.15.76:6805/755426949","state":["exists","up"]},{"osd":2,"uuid":"b6f8da58-4428-4e8c-a944-7ea5645d6d92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":852265973},{"type":"v1","addr":"172.21.15.164:6801","nonce":852265973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":852265973},{"type":"v1","addr":"172.21.15.164:6803","nonce":852265973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":852265973},{"type":"v1","addr":"172.21.15.164:6807","nonce":852265973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":852265973},{"type":"v1","addr":"172.21.15.164:6805","nonce":852265973}]},"public_addr":"172.21.15.164:6801/852265973","cluster_addr":"172.21.15.164:6803/852265973","heartbeat_back_addr":"172.21.15.164:6807/852265973","heartbeat_front_addr":"172.21.15.164:6805/852265973","state":["exists","up"]},{"osd":3,"uuid":"76d83409-649e-41e5-b4f4-43bbd5b00ea5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6810","nonce":4027763789},{"type":"v1","addr":"172.21.15.72:6811","nonce":4027763789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6812","nonce":4027763789},{"type":"v1","addr":"172.21.15.72:6813","nonce":4027763789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6816","nonce":4027763789},{"type":"v1","addr":"172.21.15.72:6817","nonce":4027763789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6814","nonce":4027763789},{"type":"v1","addr":"172.21.15.72:6815","nonce":4027763789}]},"public_addr":"172.21.15.72:6811/4027763789","cluster_addr":"172.21.15.72:6813/4027763789","heartbeat_back_addr":"172.21.15.72:6817/4027763789","heartbeat_front_addr":"172.21.15.72:6815/4027763789","state":["exists","up"]},{"osd":4,"uuid":"d2cfeea7-1661-4156-bdcd-f770a8d1a878","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":3718867827},{"type":"v1","addr":"172.21.15.76:6809","nonce":3718867827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":3718867827},{"type":"v1","addr":"172.21.15.76:6811","nonce":3718867827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":3718867827},{"type":"v1","addr":"172.21.15.76:6815","nonce":3718867827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":3718867827},{"type":"v1","addr":"172.21.15.76:6813","nonce":3718867827}]},"public_addr":"172.21.15.76:6809/3718867827","cluster_addr":"172.21.15.76:6811/3718867827","heartbeat_back_addr":"172.21.15.76:6815/3718867827","heartbeat_front_addr":"172.21.15.76:6813/3718867827","state":["exists","up"]},{"osd":5,"uuid":"f3896ad6-a040-4ea3-a431-ff9b7fc05c36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":2543954772},{"type":"v1","addr":"172.21.15.164:6809","nonce":2543954772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":2543954772},{"type":"v1","addr":"172.21.15.164:6811","nonce":2543954772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":2543954772},{"type":"v1","addr":"172.21.15.164:6815","nonce":2543954772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":2543954772},{"type":"v1","addr":"172.21.15.164:6813","nonce":2543954772}]},"public_addr":"172.21.15.164:6809/2543954772","cluster_addr":"172.21.15.164:6811/2543954772","heartbeat_back_addr":"172.21.15.164:6815/2543954772","heartbeat_front_addr":"172.21.15.164:6813/2543954772","state":["exists","up"]},{"osd":6,"uuid":"d5d56650-be7e-4ba5-ba3a-d7d473294dab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6818","nonce":2080659027},{"type":"v1","addr":"172.21.15.72:6819","nonce":2080659027}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6820","nonce":2080659027},{"type":"v1","addr":"172.21.15.72:6821","nonce":2080659027}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6824","nonce":2080659027},{"type":"v1","addr":"172.21.15.72:6825","nonce":2080659027}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6822","nonce":2080659027},{"type":"v1","addr":"172.21.15.72:6823","nonce":2080659027}]},"public_addr":"172.21.15.72:6819/2080659027","cluster_addr":"172.21.15.72:6821/2080659027","heartbeat_back_addr":"172.21.15.72:6825/2080659027","heartbeat_front_addr":"172.21.15.72:6823/2080659027","state":["exists","up"]},{"osd":7,"uuid":"18975b5a-9309-4a0f-9f37-3ce09c2e745d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":3144631323},{"type":"v1","addr":"172.21.15.76:6817","nonce":3144631323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":3144631323},{"type":"v1","addr":"172.21.15.76:6819","nonce":3144631323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":3144631323},{"type":"v1","addr":"172.21.15.76:6823","nonce":3144631323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":3144631323},{"type":"v1","addr":"172.21.15.76:6821","nonce":3144631323}]},"public_addr":"172.21.15.76:6817/3144631323","cluster_addr":"172.21.15.76:6819/3144631323","heartbeat_back_addr":"172.21.15.76:6823/3144631323","heartbeat_front_addr":"172.21.15.76:6821/3144631323","state":["exists","up"]},{"osd":8,"uuid":"779e9c1e-8e97-48d4-a854-68ca13f38e0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":627888883},{"type":"v1","addr":"172.21.15.164:6817","nonce":627888883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":627888883},{"type":"v1","addr":"172.21.15.164:6819","nonce":627888883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":627888883},{"type":"v1","addr":"172.21.15.164:6823","nonce":627888883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":627888883},{"type":"v1","addr":"172.21.15.164:6821","nonce":627888883}]},"public_addr":"172.21.15.164:6817/627888883","cluster_addr":"172.21.15.164:6819/627888883","heartbeat_back_addr":"172.21.15.164:6823/627888883","heartbeat_front_addr":"172.21.15.164:6821/627888883","state":["exists","up"]},{"osd":9,"uuid":"d0317850-7b2d-4850-b2cf-0b919505c1fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6826","nonce":1273118485},{"type":"v1","addr":"172.21.15.72:6827","nonce":1273118485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6828","nonce":1273118485},{"type":"v1","addr":"172.21.15.72:6829","nonce":1273118485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6832","nonce":1273118485},{"type":"v1","addr":"172.21.15.72:6833","nonce":1273118485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6830","nonce":1273118485},{"type":"v1","addr":"172.21.15.72:6831","nonce":1273118485}]},"public_addr":"172.21.15.72:6827/1273118485","cluster_addr":"172.21.15.72:6829/1273118485","heartbeat_back_addr":"172.21.15.72:6833/1273118485","heartbeat_front_addr":"172.21.15.72:6831/1273118485","state":["exists","up"]},{"osd":10,"uuid":"e0aa38d8-6011-4d2a-b67b-5bff6d999e47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":2808954459},{"type":"v1","addr":"172.21.15.76:6825","nonce":2808954459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":2808954459},{"type":"v1","addr":"172.21.15.76:6827","nonce":2808954459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":2808954459},{"type":"v1","addr":"172.21.15.76:6831","nonce":2808954459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":2808954459},{"type":"v1","addr":"172.21.15.76:6829","nonce":2808954459}]},"public_addr":"172.21.15.76:6825/2808954459","cluster_addr":"172.21.15.76:6827/2808954459","heartbeat_back_addr":"172.21.15.76:6831/2808954459","heartbeat_front_addr":"172.21.15.76:6829/2808954459","state":["exists","up"]},{"osd":11,"uuid":"981b3d4d-916e-4937-9916-7cc21a891e43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":2354951784},{"type":"v1","addr":"172.21.15.164:6825","nonce":2354951784}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":2354951784},{"type":"v1","addr":"172.21.15.164:6827","nonce":2354951784}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":2354951784},{"type":"v1","addr":"172.21.15.164:6831","nonce":2354951784}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":2354951784},{"type":"v1","addr":"172.21.15.164:6829","nonce":2354951784}]},"public_addr":"172.21.15.164:6825/2354951784","cluster_addr":"172.21.15.164:6827/2354951784","heartbeat_back_addr":"172.21.15.164:6831/2354951784","heartbeat_front_addr":"172.21.15.164:6829/2354951784","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:18.997924+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:52.439081+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:46:25.893288+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:47:00.112807+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:47:35.116874+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:48:09.216344+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:48:45.028853+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:49:19.905397+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:49:55.030536+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:50:30.767662+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:51:06.423267+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:51:41.631211+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.1a","mappings":[{"from":5,"to":7}]},{"pgid":"3.1b","mappings":[{"from":4,"to":6}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.28","mappings":[{"from":5,"to":6}]},{"pgid":"3.2a","mappings":[{"from":3,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.72:0/416277784":"2024-04-27T21:43:09.602791+0000","172.21.15.72:6800/4010358672":"2024-04-27T21:43:09.602791+0000","172.21.15.72:6801/1012617371":"2024-04-27T21:42:34.684837+0000","172.21.15.72:6801/4010358672":"2024-04-27T21:43:09.602791+0000","172.21.15.72:0/3555448179":"2024-04-27T21:43:09.602791+0000","172.21.15.72:6800/1012617371":"2024-04-27T21:42:34.684837+0000","172.21.15.72:0/2920378761":"2024-04-27T21:42:34.684837+0000","172.21.15.72:0/1933740041":"2024-04-27T21:43:09.602791+0000","172.21.15.72:0/3225295775":"2024-04-27T21:42:34.684837+0000","172.21.15.72:0/2326016180":"2024-04-27T21:42:34.684837+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:55:23.365 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:23.363+0000 7f284b200640 1 -- 172.21.15.72:0/3966882745 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f28300a3cc0 msgr2=0x7f28300a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:23.365 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:23.363+0000 7f284b200640 1 --2- 172.21.15.72:0/3966882745 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f28300a3cc0 0x7f28300a6180 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f283c005fd0 tx=0x7f283c005850 comp rx=0 tx=0).stop 2024-04-26T21:55:23.365 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:23.363+0000 7f284b200640 1 -- 172.21.15.72:0/3966882745 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f284c105b20 msgr2=0x7f284c1011a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:23.365 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:23.363+0000 7f284b200640 1 --2- 172.21.15.72:0/3966882745 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f284c105b20 0x7f284c1011a0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f283400c970 tx=0x7f283400ce30 comp rx=0 tx=0).stop 2024-04-26T21:55:23.365 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:23.364+0000 7f284b200640 1 -- 172.21.15.72:0/3966882745 shutdown_connections 2024-04-26T21:55:23.365 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:23.364+0000 7f284b200640 1 --2- 172.21.15.72:0/3966882745 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f284c108450 0x7f284c1016e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:23.365 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:23.364+0000 7f284b200640 1 --2- 172.21.15.72:0/3966882745 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f284c105b20 0x7f284c1011a0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:23.365 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:23.364+0000 7f284b200640 1 --2- 172.21.15.72:0/3966882745 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f28300a3cc0 0x7f28300a6180 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:23.365 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:23.364+0000 7f284b200640 1 --2- 172.21.15.72:0/3966882745 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f284c1031f0 0x7f284c102b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:23.365 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:23.364+0000 7f284b200640 1 -- 172.21.15.72:0/3966882745 >> 172.21.15.72:0/3966882745 conn(0x7f284c0fcbc0 msgr2=0x7f284c103a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:55:23.366 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:23.364+0000 7f284b200640 1 -- 172.21.15.72:0/3966882745 shutdown_connections 2024-04-26T21:55:23.366 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:23.364+0000 7f284b200640 1 -- 172.21.15.72:0/3966882745 wait complete. 2024-04-26T21:55:23.384 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:46:29.785427+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T21:52:48.320391+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T21:52:49.517493+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:55:23.385 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:55:23.584 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:23.583+0000 7f5b95800640 1 -- 172.21.15.72:0/3481346025 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5b90105df0 msgr2=0x7f5b901081e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:23.585 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:23.583+0000 7f5b95800640 1 --2- 172.21.15.72:0/3481346025 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5b90105df0 0x7f5b901081e0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f5b78009900 tx=0x7f5b7802a410 comp rx=0 tx=0).stop 2024-04-26T21:55:23.585 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:23.583+0000 7f5b95800640 1 -- 172.21.15.72:0/3481346025 shutdown_connections 2024-04-26T21:55:23.585 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:23.583+0000 7f5b95800640 1 --2- 172.21.15.72:0/3481346025 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5b90108720 0x7f5b9010ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:23.585 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:23.583+0000 7f5b95800640 1 --2- 172.21.15.72:0/3481346025 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5b90105df0 0x7f5b901081e0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:23.585 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:23.583+0000 7f5b95800640 1 --2- 172.21.15.72:0/3481346025 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5b901034c0 0x7f5b901058b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:23.585 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:23.583+0000 7f5b95800640 1 -- 172.21.15.72:0/3481346025 >> 172.21.15.72:0/3481346025 conn(0x7f5b900fcdf0 msgr2=0x7f5b900ff230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:55:23.585 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:23.583+0000 7f5b95800640 1 -- 172.21.15.72:0/3481346025 shutdown_connections 2024-04-26T21:55:23.585 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:23.583+0000 7f5b95800640 1 -- 172.21.15.72:0/3481346025 wait complete. 2024-04-26T21:55:23.587 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:23.584+0000 7f5b95800640 1 Processor -- start 2024-04-26T21:55:23.588 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:23.585+0000 7f5b95800640 1 -- start start 2024-04-26T21:55:23.588 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:23.585+0000 7f5b95800640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5b901034c0 0x7f5b90102e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:23.588 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:23.585+0000 7f5b95800640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5b90105df0 0x7f5b90101470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:23.588 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:23.585+0000 7f5b95800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5b90108720 0x7f5b901019b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:23.588 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:23.585+0000 7f5b95800640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f5b9010e3b0 con 0x7f5b90105df0 2024-04-26T21:55:23.588 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:23.585+0000 7f5b95800640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f5b9010e230 con 0x7f5b901034c0 2024-04-26T21:55:23.588 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:23.585+0000 7f5b95800640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f5b9010e530 con 0x7f5b90108720 2024-04-26T21:55:23.588 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:23.585+0000 7f5b87e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5b90105df0 0x7f5b90101470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:23.588 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:23.585+0000 7f5b87e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5b90105df0 0x7f5b90101470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:44056/0 (socket says 172.21.15.72:44056) 2024-04-26T21:55:23.588 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:23.585+0000 7f5b87e00640 1 -- 172.21.15.72:0/1772230931 learned_addr learned my addr 172.21.15.72:0/1772230931 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:55:23.588 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:23.586+0000 7f5b87e00640 1 -- 172.21.15.72:0/1772230931 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5b90108720 msgr2=0x7f5b901019b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:23.588 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:23.586+0000 7f5b8ea00640 1 --2- 172.21.15.72:0/1772230931 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5b901034c0 0x7f5b90102e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:23.589 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:23.586+0000 7f5b8f400640 1 --2- 172.21.15.72:0/1772230931 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5b90108720 0x7f5b901019b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:23.589 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:23.586+0000 7f5b87e00640 1 --2- 172.21.15.72:0/1772230931 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5b90108720 0x7f5b901019b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:23.589 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:23.586+0000 7f5b87e00640 1 -- 172.21.15.72:0/1772230931 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5b901034c0 msgr2=0x7f5b90102e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:23.589 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:23.586+0000 7f5b87e00640 1 --2- 172.21.15.72:0/1772230931 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5b901034c0 0x7f5b90102e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:23.589 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:23.586+0000 7f5b87e00640 1 -- 172.21.15.72:0/1772230931 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b90102240 con 0x7f5b90105df0 2024-04-26T21:55:23.589 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:23.586+0000 7f5b8ea00640 1 --2- 172.21.15.72:0/1772230931 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5b901034c0 0x7f5b90102e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:55:23.589 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:23.586+0000 7f5b87e00640 1 --2- 172.21.15.72:0/1772230931 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5b90105df0 0x7f5b90101470 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f5b78031460 tx=0x7f5b78003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:55:23.589 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:23.586+0000 7f5b87400640 1 -- 172.21.15.72:0/1772230931 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b7803f070 con 0x7f5b90105df0 2024-04-26T21:55:23.589 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:23.586+0000 7f5b87400640 1 -- 172.21.15.72:0/1772230931 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5b78031da0 con 0x7f5b90105df0 2024-04-26T21:55:23.589 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:23.586+0000 7f5b87400640 1 -- 172.21.15.72:0/1772230931 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b78035030 con 0x7f5b90105df0 2024-04-26T21:55:23.589 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:23.586+0000 7f5b95800640 1 -- 172.21.15.72:0/1772230931 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b901a2c10 con 0x7f5b90105df0 2024-04-26T21:55:23.589 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:23.586+0000 7f5b95800640 1 -- 172.21.15.72:0/1772230931 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b901a3160 con 0x7f5b90105df0 2024-04-26T21:55:23.590 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:23.588+0000 7f5b95800640 1 -- 172.21.15.72:0/1772230931 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b900fe8e0 con 0x7f5b90105df0 2024-04-26T21:55:23.591 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:23.590+0000 7f5b87400640 1 -- 172.21.15.72:0/1772230931 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f5b78042070 con 0x7f5b90105df0 2024-04-26T21:55:23.592 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:23.591+0000 7f5b87400640 1 --2- 172.21.15.72:0/1772230931 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f5b740a3c40 0x7f5b740a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:23.592 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:23.591+0000 7f5b87400640 1 -- 172.21.15.72:0/1772230931 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7f5b780f0f50 con 0x7f5b90105df0 2024-04-26T21:55:23.592 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:23.592+0000 7f5b8ea00640 1 --2- 172.21.15.72:0/1772230931 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f5b740a3c40 0x7f5b740a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:23.593 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:23.592+0000 7f5b8ea00640 1 --2- 172.21.15.72:0/1772230931 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f5b740a3c40 0x7f5b740a6100 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f5b80009b40 tx=0x7f5b80005e50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:55:23.597 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:23.596+0000 7f5b87400640 1 -- 172.21.15.72:0/1772230931 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f5b780bd6f0 con 0x7f5b90105df0 2024-04-26T21:55:23.797 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:23.796+0000 7f5b95800640 1 -- 172.21.15.72:0/1772230931 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f5b9002dd10 con 0x7f5b90105df0 2024-04-26T21:55:23.798 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:23.797+0000 7f5b87400640 1 -- 172.21.15.72:0/1772230931 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) ==== 93+0+10 (secure 0 0 0) 0x7f5b780dd070 con 0x7f5b90105df0 2024-04-26T21:55:23.798 INFO:teuthology.orchestra.run.smithi072.stdout:pg_num: 1 2024-04-26T21:55:23.800 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:23.799+0000 7f5b95800640 1 -- 172.21.15.72:0/1772230931 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f5b740a3c40 msgr2=0x7f5b740a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:23.800 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:23.799+0000 7f5b95800640 1 --2- 172.21.15.72:0/1772230931 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f5b740a3c40 0x7f5b740a6100 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f5b80009b40 tx=0x7f5b80005e50 comp rx=0 tx=0).stop 2024-04-26T21:55:23.800 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:23.799+0000 7f5b95800640 1 -- 172.21.15.72:0/1772230931 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5b90105df0 msgr2=0x7f5b90101470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:23.800 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:23.799+0000 7f5b95800640 1 --2- 172.21.15.72:0/1772230931 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5b90105df0 0x7f5b90101470 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f5b78031460 tx=0x7f5b78003c80 comp rx=0 tx=0).stop 2024-04-26T21:55:23.801 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:23.799+0000 7f5b95800640 1 -- 172.21.15.72:0/1772230931 shutdown_connections 2024-04-26T21:55:23.801 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:23.799+0000 7f5b95800640 1 --2- 172.21.15.72:0/1772230931 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f5b740a3c40 0x7f5b740a6100 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:23.801 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:23.799+0000 7f5b95800640 1 --2- 172.21.15.72:0/1772230931 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5b90108720 0x7f5b901019b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:23.801 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:23.799+0000 7f5b95800640 1 --2- 172.21.15.72:0/1772230931 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5b90105df0 0x7f5b90101470 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:23.801 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:23.799+0000 7f5b95800640 1 --2- 172.21.15.72:0/1772230931 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5b901034c0 0x7f5b90102e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:23.801 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:23.799+0000 7f5b95800640 1 -- 172.21.15.72:0/1772230931 >> 172.21.15.72:0/1772230931 conn(0x7f5b900fcdf0 msgr2=0x7f5b90105330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:55:23.801 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:23.799+0000 7f5b95800640 1 -- 172.21.15.72:0/1772230931 shutdown_connections 2024-04-26T21:55:23.801 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:23.800+0000 7f5b95800640 1 -- 172.21.15.72:0/1772230931 wait complete. 2024-04-26T21:55:23.820 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T21:55:24.036 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:24.033+0000 7faff6e00640 1 -- 172.21.15.72:0/713250263 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7faff0109fc0 msgr2=0x7faff010a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:24.036 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:24.033+0000 7faff6e00640 1 --2- 172.21.15.72:0/713250263 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7faff0109fc0 0x7faff010a440 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fafdc009900 tx=0x7fafdc02a410 comp rx=0 tx=0).stop 2024-04-26T21:55:24.036 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:24.034+0000 7faff6e00640 1 -- 172.21.15.72:0/713250263 shutdown_connections 2024-04-26T21:55:24.036 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:24.034+0000 7faff6e00640 1 --2- 172.21.15.72:0/713250263 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faff010a980 0x7faff01112a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:24.037 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:24.034+0000 7faff6e00640 1 --2- 172.21.15.72:0/713250263 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7faff0109fc0 0x7faff010a440 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:24.037 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:24.034+0000 7faff6e00640 1 --2- 172.21.15.72:0/713250263 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faff0108e70 0x7faff0109290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:24.037 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:24.034+0000 7faff6e00640 1 -- 172.21.15.72:0/713250263 >> 172.21.15.72:0/713250263 conn(0x7faff0077e00 msgr2=0x7faff007a240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:55:24.037 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:24.034+0000 7faff6e00640 1 -- 172.21.15.72:0/713250263 shutdown_connections 2024-04-26T21:55:24.037 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:24.034+0000 7faff6e00640 1 -- 172.21.15.72:0/713250263 wait complete. 2024-04-26T21:55:24.037 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:24.035+0000 7faff6e00640 1 Processor -- start 2024-04-26T21:55:24.037 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:24.035+0000 7faff6e00640 1 -- start start 2024-04-26T21:55:24.039 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:24.035+0000 7faff6e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7faff0108e70 0x7faff019eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:24.039 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:24.036+0000 7faff6e00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faff0109fc0 0x7faff019f530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:24.039 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:24.036+0000 7faff6e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faff010a980 0x7faff01a65a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:24.039 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:24.036+0000 7faff6e00640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7faff0114870 con 0x7faff0108e70 2024-04-26T21:55:24.039 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:24.036+0000 7faff6e00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7faff01146f0 con 0x7faff0109fc0 2024-04-26T21:55:24.040 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:24.036+0000 7faff6e00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7faff01149f0 con 0x7faff010a980 2024-04-26T21:55:24.040 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:24.036+0000 7faff5a00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7faff0108e70 0x7faff019eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:24.040 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:24.036+0000 7faff5a00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7faff0108e70 0x7faff019eff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:44070/0 (socket says 172.21.15.72:44070) 2024-04-26T21:55:24.040 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:24.036+0000 7faff5a00640 1 -- 172.21.15.72:0/51973114 learned_addr learned my addr 172.21.15.72:0/51973114 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:55:24.040 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:24.036+0000 7faff5000640 1 --2- 172.21.15.72:0/51973114 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faff0109fc0 0x7faff019f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:24.040 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:24.036+0000 7faff5a00640 1 -- 172.21.15.72:0/51973114 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faff010a980 msgr2=0x7faff01a65a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:24.040 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:24.036+0000 7faff6400640 1 --2- 172.21.15.72:0/51973114 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faff010a980 0x7faff01a65a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:24.040 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:24.036+0000 7faff5a00640 1 --2- 172.21.15.72:0/51973114 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faff010a980 0x7faff01a65a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:24.040 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:24.036+0000 7faff5a00640 1 -- 172.21.15.72:0/51973114 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faff0109fc0 msgr2=0x7faff019f530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:24.040 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:24.036+0000 7faff5a00640 1 --2- 172.21.15.72:0/51973114 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faff0109fc0 0x7faff019f530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:24.040 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:24.036+0000 7faff5a00640 1 -- 172.21.15.72:0/51973114 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faff01a6c10 con 0x7faff0108e70 2024-04-26T21:55:24.040 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:24.036+0000 7faff5000640 1 --2- 172.21.15.72:0/51973114 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faff0109fc0 0x7faff019f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:55:24.040 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:24.037+0000 7faff5a00640 1 --2- 172.21.15.72:0/51973114 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7faff0108e70 0x7faff019eff0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fafe400c970 tx=0x7fafe400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:55:24.040 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:24.037+0000 7fafee000640 1 -- 172.21.15.72:0/51973114 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafe4007bb0 con 0x7faff0108e70 2024-04-26T21:55:24.040 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:24.037+0000 7fafee000640 1 -- 172.21.15.72:0/51973114 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fafe4007d50 con 0x7faff0108e70 2024-04-26T21:55:24.041 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:24.037+0000 7faff6e00640 1 -- 172.21.15.72:0/51973114 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faff01a6f00 con 0x7faff0108e70 2024-04-26T21:55:24.041 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:24.037+0000 7fafee000640 1 -- 172.21.15.72:0/51973114 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafe4004e40 con 0x7faff0108e70 2024-04-26T21:55:24.041 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:24.037+0000 7faff6e00640 1 -- 172.21.15.72:0/51973114 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faff01a7360 con 0x7faff0108e70 2024-04-26T21:55:24.041 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:24.038+0000 7faff6e00640 1 -- 172.21.15.72:0/51973114 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faff0069af0 con 0x7faff0108e70 2024-04-26T21:55:24.042 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:24.040+0000 7fafee000640 1 -- 172.21.15.72:0/51973114 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fafe4005020 con 0x7faff0108e70 2024-04-26T21:55:24.043 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:24.041+0000 7fafee000640 1 --2- 172.21.15.72:0/51973114 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fafd80a3cc0 0x7fafd80a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:24.043 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:24.042+0000 7fafee000640 1 -- 172.21.15.72:0/51973114 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7fafe40d03f0 con 0x7faff0108e70 2024-04-26T21:55:24.043 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:24.042+0000 7faff5000640 1 --2- 172.21.15.72:0/51973114 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fafd80a3cc0 0x7fafd80a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:24.043 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:24.043+0000 7faff5000640 1 --2- 172.21.15.72:0/51973114 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fafd80a3cc0 0x7fafd80a6180 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fafdc000c00 tx=0x7fafdc009650 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:55:24.046 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:24.046+0000 7fafee000640 1 -- 172.21.15.72:0/51973114 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7faff0069af0 con 0x7faff0108e70 2024-04-26T21:55:24.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:55:23 smithi072 ceph-mon[28138]: from='client.? 172.21.15.72:0/1265220790' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-04-26T21:55:24.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:55:23 smithi072 ceph-mon[28138]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-26T21:55:24.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:55:23 smithi072 ceph-mon[28138]: from='client.? 172.21.15.72:0/3966882745' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:55:24.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:55:23 smithi072 ceph-mon[28138]: from='client.? 172.21.15.72:0/1772230931' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T21:55:24.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:55:23 smithi164 ceph-mon[31916]: from='client.? 172.21.15.72:0/1265220790' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-04-26T21:55:24.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:55:23 smithi164 ceph-mon[31916]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-26T21:55:24.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:55:23 smithi164 ceph-mon[31916]: from='client.? 172.21.15.72:0/3966882745' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:55:24.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:55:23 smithi164 ceph-mon[31916]: from='client.? 172.21.15.72:0/1772230931' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T21:55:24.249 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:24.247+0000 7faff6e00640 1 -- 172.21.15.72:0/51973114 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7faff002db90 con 0x7faff0108e70 2024-04-26T21:55:24.250 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:24.248+0000 7fafee000640 1 -- 172.21.15.72:0/51973114 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v91) ==== 104+0+11 (secure 0 0 0) 0x7fafe4014030 con 0x7faff0108e70 2024-04-26T21:55:24.250 INFO:teuthology.orchestra.run.smithi072.stdout:pg_num: 32 2024-04-26T21:55:24.252 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:24.251+0000 7faff6e00640 1 -- 172.21.15.72:0/51973114 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fafd80a3cc0 msgr2=0x7fafd80a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:24.252 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:24.251+0000 7faff6e00640 1 --2- 172.21.15.72:0/51973114 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fafd80a3cc0 0x7fafd80a6180 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fafdc000c00 tx=0x7fafdc009650 comp rx=0 tx=0).stop 2024-04-26T21:55:24.253 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:24.251+0000 7faff6e00640 1 -- 172.21.15.72:0/51973114 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7faff0108e70 msgr2=0x7faff019eff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:24.253 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:24.251+0000 7faff6e00640 1 --2- 172.21.15.72:0/51973114 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7faff0108e70 0x7faff019eff0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fafe400c970 tx=0x7fafe400ce30 comp rx=0 tx=0).stop 2024-04-26T21:55:24.253 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:24.251+0000 7faff6e00640 1 -- 172.21.15.72:0/51973114 shutdown_connections 2024-04-26T21:55:24.253 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:24.251+0000 7faff6e00640 1 --2- 172.21.15.72:0/51973114 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fafd80a3cc0 0x7fafd80a6180 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:24.253 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:24.251+0000 7faff6e00640 1 --2- 172.21.15.72:0/51973114 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faff010a980 0x7faff01a65a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:24.254 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:24.251+0000 7faff6e00640 1 --2- 172.21.15.72:0/51973114 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faff0109fc0 0x7faff019f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:24.254 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:24.251+0000 7faff6e00640 1 --2- 172.21.15.72:0/51973114 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7faff0108e70 0x7faff019eff0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:24.254 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:24.251+0000 7faff6e00640 1 -- 172.21.15.72:0/51973114 >> 172.21.15.72:0/51973114 conn(0x7faff0077e00 msgr2=0x7faff010d420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:55:24.254 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:24.252+0000 7faff6e00640 1 -- 172.21.15.72:0/51973114 shutdown_connections 2024-04-26T21:55:24.254 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:24.252+0000 7faff6e00640 1 -- 172.21.15.72:0/51973114 wait complete. 2024-04-26T21:55:24.271 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T21:55:24.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:55:23 smithi076 ceph-mon[31823]: from='client.? 172.21.15.72:0/1265220790' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-04-26T21:55:24.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:55:23 smithi076 ceph-mon[31823]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-26T21:55:24.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:55:23 smithi076 ceph-mon[31823]: from='client.? 172.21.15.72:0/3966882745' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:55:24.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:55:23 smithi076 ceph-mon[31823]: from='client.? 172.21.15.72:0/1772230931' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T21:55:24.463 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:24.461+0000 7f6e6b200640 1 -- 172.21.15.72:0/309457263 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6e6c108d90 msgr2=0x7f6e6c1091b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:24.463 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:24.461+0000 7f6e6b200640 1 --2- 172.21.15.72:0/309457263 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6e6c108d90 0x7f6e6c1091b0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f6e5c009900 tx=0x7f6e5c02a400 comp rx=0 tx=0).stop 2024-04-26T21:55:24.463 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:24.461+0000 7f6e6b200640 1 -- 172.21.15.72:0/309457263 shutdown_connections 2024-04-26T21:55:24.463 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:24.461+0000 7f6e6b200640 1 --2- 172.21.15.72:0/309457263 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6e6c10a940 0x7f6e6c1111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:24.463 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:24.461+0000 7f6e6b200640 1 --2- 172.21.15.72:0/309457263 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6e6c109f80 0x7f6e6c10a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:24.463 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:24.461+0000 7f6e6b200640 1 --2- 172.21.15.72:0/309457263 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6e6c108d90 0x7f6e6c1091b0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:24.463 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:24.461+0000 7f6e6b200640 1 -- 172.21.15.72:0/309457263 >> 172.21.15.72:0/309457263 conn(0x7f6e6c077d30 msgr2=0x7f6e6c07a170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:55:24.463 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:24.462+0000 7f6e6b200640 1 -- 172.21.15.72:0/309457263 shutdown_connections 2024-04-26T21:55:24.463 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:24.462+0000 7f6e6b200640 1 -- 172.21.15.72:0/309457263 wait complete. 2024-04-26T21:55:24.465 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:24.462+0000 7f6e6b200640 1 Processor -- start 2024-04-26T21:55:24.465 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:24.463+0000 7f6e6b200640 1 -- start start 2024-04-26T21:55:24.466 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:24.463+0000 7f6e6b200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6e6c108d90 0x7f6e6c19ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:24.466 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:24.463+0000 7f6e6b200640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6e6c109f80 0x7f6e6c19f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:24.466 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:24.463+0000 7f6e6b200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6e6c10a940 0x7f6e6c1a63a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:24.466 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:24.463+0000 7f6e6b200640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f6e6c114790 con 0x7f6e6c109f80 2024-04-26T21:55:24.466 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:24.463+0000 7f6e6b200640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f6e6c114610 con 0x7f6e6c108d90 2024-04-26T21:55:24.466 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:24.463+0000 7f6e6b200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f6e6c114910 con 0x7f6e6c10a940 2024-04-26T21:55:24.466 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:24.463+0000 7f6e63e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6e6c109f80 0x7f6e6c19f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:24.466 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:24.463+0000 7f6e63e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6e6c109f80 0x7f6e6c19f3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:44076/0 (socket says 172.21.15.72:44076) 2024-04-26T21:55:24.466 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:24.463+0000 7f6e63e00640 1 -- 172.21.15.72:0/2417155384 learned_addr learned my addr 172.21.15.72:0/2417155384 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:55:24.466 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:24.464+0000 7f6e69e00640 1 --2- 172.21.15.72:0/2417155384 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6e6c108d90 0x7f6e6c19ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:24.466 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:24.464+0000 7f6e6a800640 1 --2- 172.21.15.72:0/2417155384 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6e6c10a940 0x7f6e6c1a63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:24.466 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:24.464+0000 7f6e63e00640 1 -- 172.21.15.72:0/2417155384 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6e6c10a940 msgr2=0x7f6e6c1a63a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:24.466 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:24.464+0000 7f6e63e00640 1 --2- 172.21.15.72:0/2417155384 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6e6c10a940 0x7f6e6c1a63a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:24.467 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:24.464+0000 7f6e63e00640 1 -- 172.21.15.72:0/2417155384 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6e6c108d90 msgr2=0x7f6e6c19ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:24.467 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:24.464+0000 7f6e63e00640 1 --2- 172.21.15.72:0/2417155384 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6e6c108d90 0x7f6e6c19ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:24.467 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:24.464+0000 7f6e63e00640 1 -- 172.21.15.72:0/2417155384 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e6c1a6a10 con 0x7f6e6c109f80 2024-04-26T21:55:24.467 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:24.464+0000 7f6e63e00640 1 --2- 172.21.15.72:0/2417155384 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6e6c109f80 0x7f6e6c19f3d0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f6e5400c920 tx=0x7f6e5400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:55:24.467 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:24.464+0000 7f6e62a00640 1 -- 172.21.15.72:0/2417155384 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e54007c70 con 0x7f6e6c109f80 2024-04-26T21:55:24.467 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:24.464+0000 7f6e62a00640 1 -- 172.21.15.72:0/2417155384 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6e54007e10 con 0x7f6e6c109f80 2024-04-26T21:55:24.467 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:24.464+0000 7f6e62a00640 1 -- 172.21.15.72:0/2417155384 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e54018710 con 0x7f6e6c109f80 2024-04-26T21:55:24.467 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:24.464+0000 7f6e6b200640 1 -- 172.21.15.72:0/2417155384 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e6c1a6d00 con 0x7f6e6c109f80 2024-04-26T21:55:24.467 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:24.465+0000 7f6e6b200640 1 -- 172.21.15.72:0/2417155384 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e6c1a71a0 con 0x7f6e6c109f80 2024-04-26T21:55:24.467 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:24.466+0000 7f6e6b200640 1 -- 172.21.15.72:0/2417155384 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e6c1100d0 con 0x7f6e6c109f80 2024-04-26T21:55:24.469 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:24.468+0000 7f6e62a00640 1 -- 172.21.15.72:0/2417155384 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f6e5400f450 con 0x7f6e6c109f80 2024-04-26T21:55:24.470 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:24.471+0000 7f6e62a00640 1 --2- 172.21.15.72:0/2417155384 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f6e4c0a3c40 0x7f6e4c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:24.471 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:24.471+0000 7f6e62a00640 1 -- 172.21.15.72:0/2417155384 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7f6e540d08a0 con 0x7f6e6c109f80 2024-04-26T21:55:24.471 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:24.471+0000 7f6e69e00640 1 --2- 172.21.15.72:0/2417155384 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f6e4c0a3c40 0x7f6e4c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:24.472 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:24.471+0000 7f6e69e00640 1 --2- 172.21.15.72:0/2417155384 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f6e4c0a3c40 0x7f6e4c0a6100 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f6e5c0057c0 tx=0x7f6e5c000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:55:24.475 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:24.475+0000 7f6e62a00640 1 -- 172.21.15.72:0/2417155384 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f6e54014030 con 0x7f6e6c109f80 2024-04-26T21:55:24.684 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:24.682+0000 7f6e6b200640 1 -- 172.21.15.72:0/2417155384 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f6e6c02dcf0 con 0x7f6e6c109f80 2024-04-26T21:55:24.684 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:24.683+0000 7f6e62a00640 1 -- 172.21.15.72:0/2417155384 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v91) ==== 100+0+11 (secure 0 0 0) 0x7f6e5400de20 con 0x7f6e6c109f80 2024-04-26T21:55:24.685 INFO:teuthology.orchestra.run.smithi072.stdout:pg_num: 64 2024-04-26T21:55:24.687 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:24.686+0000 7f6e6b200640 1 -- 172.21.15.72:0/2417155384 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f6e4c0a3c40 msgr2=0x7f6e4c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:24.688 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:24.686+0000 7f6e6b200640 1 --2- 172.21.15.72:0/2417155384 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f6e4c0a3c40 0x7f6e4c0a6100 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f6e5c0057c0 tx=0x7f6e5c000bc0 comp rx=0 tx=0).stop 2024-04-26T21:55:24.688 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:24.686+0000 7f6e6b200640 1 -- 172.21.15.72:0/2417155384 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6e6c109f80 msgr2=0x7f6e6c19f3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:24.688 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:24.686+0000 7f6e6b200640 1 --2- 172.21.15.72:0/2417155384 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6e6c109f80 0x7f6e6c19f3d0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f6e5400c920 tx=0x7f6e5400cde0 comp rx=0 tx=0).stop 2024-04-26T21:55:24.688 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:24.686+0000 7f6e6b200640 1 -- 172.21.15.72:0/2417155384 shutdown_connections 2024-04-26T21:55:24.688 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:24.686+0000 7f6e6b200640 1 --2- 172.21.15.72:0/2417155384 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f6e4c0a3c40 0x7f6e4c0a6100 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:24.688 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:24.686+0000 7f6e6b200640 1 --2- 172.21.15.72:0/2417155384 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6e6c10a940 0x7f6e6c1a63a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:24.688 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:24.686+0000 7f6e6b200640 1 --2- 172.21.15.72:0/2417155384 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6e6c109f80 0x7f6e6c19f3d0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:24.688 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:24.686+0000 7f6e6b200640 1 --2- 172.21.15.72:0/2417155384 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6e6c108d90 0x7f6e6c19ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:24.688 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:24.686+0000 7f6e6b200640 1 -- 172.21.15.72:0/2417155384 >> 172.21.15.72:0/2417155384 conn(0x7f6e6c077d30 msgr2=0x7f6e6c10d2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:55:24.689 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:24.687+0000 7f6e6b200640 1 -- 172.21.15.72:0/2417155384 shutdown_connections 2024-04-26T21:55:24.689 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:24.687+0000 7f6e6b200640 1 -- 172.21.15.72:0/2417155384 wait complete. 2024-04-26T21:55:24.707 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:55:24.908 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:24.907+0000 7f5aa5c00640 1 -- 172.21.15.72:0/1739788710 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5aa0106740 msgr2=0x7f5aa010cfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:24.908 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:24.907+0000 7f5aa5c00640 1 --2- 172.21.15.72:0/1739788710 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5aa0106740 0x7f5aa010cfd0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f5a98009ca0 tx=0x7f5a9802a8c0 comp rx=0 tx=0).stop 2024-04-26T21:55:24.908 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:24.908+0000 7f5aa5c00640 1 -- 172.21.15.72:0/1739788710 shutdown_connections 2024-04-26T21:55:24.908 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:24.908+0000 7f5aa5c00640 1 --2- 172.21.15.72:0/1739788710 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5aa0106740 0x7f5aa010cfd0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:24.908 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:24.908+0000 7f5aa5c00640 1 --2- 172.21.15.72:0/1739788710 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5aa0105d80 0x7f5aa0106200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:24.909 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:24.908+0000 7f5aa5c00640 1 --2- 172.21.15.72:0/1739788710 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5aa0104b90 0x7f5aa0104fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:24.909 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:24.908+0000 7f5aa5c00640 1 -- 172.21.15.72:0/1739788710 >> 172.21.15.72:0/1739788710 conn(0x7f5aa0100120 msgr2=0x7f5aa0102540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:55:24.909 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:24.908+0000 7f5aa5c00640 1 -- 172.21.15.72:0/1739788710 shutdown_connections 2024-04-26T21:55:24.909 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:24.908+0000 7f5aa5c00640 1 -- 172.21.15.72:0/1739788710 wait complete. 2024-04-26T21:55:24.911 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:24.909+0000 7f5aa5c00640 1 Processor -- start 2024-04-26T21:55:24.912 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:24.909+0000 7f5aa5c00640 1 -- start start 2024-04-26T21:55:24.912 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:24.909+0000 7f5aa5c00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5aa0104b90 0x7f5aa019ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:24.912 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:24.909+0000 7f5aa5c00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5aa0105d80 0x7f5aa019b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:24.912 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:24.909+0000 7f5aa5c00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5aa0106740 0x7f5aa01a2210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:24.912 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:24.909+0000 7f5aa5c00640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f5aa01105c0 con 0x7f5aa0105d80 2024-04-26T21:55:24.912 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:24.909+0000 7f5aa5c00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f5aa0110440 con 0x7f5aa0104b90 2024-04-26T21:55:24.912 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:24.909+0000 7f5aa5c00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f5aa0110740 con 0x7f5aa0106740 2024-04-26T21:55:24.912 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:24.909+0000 7f5a96000640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5aa0105d80 0x7f5aa019b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:24.912 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:24.909+0000 7f5a96000640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5aa0105d80 0x7f5aa019b1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:44088/0 (socket says 172.21.15.72:44088) 2024-04-26T21:55:24.912 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:24.909+0000 7f5a96000640 1 -- 172.21.15.72:0/1034215189 learned_addr learned my addr 172.21.15.72:0/1034215189 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:55:24.912 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:24.910+0000 7f5a9f400640 1 --2- 172.21.15.72:0/1034215189 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5aa0106740 0x7f5aa01a2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:24.912 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:24.910+0000 7f5a9ea00640 1 --2- 172.21.15.72:0/1034215189 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5aa0104b90 0x7f5aa019ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:24.913 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:24.910+0000 7f5a96000640 1 -- 172.21.15.72:0/1034215189 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5aa0106740 msgr2=0x7f5aa01a2210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:24.913 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:24.910+0000 7f5a96000640 1 --2- 172.21.15.72:0/1034215189 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5aa0106740 0x7f5aa01a2210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:24.913 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:24.910+0000 7f5a96000640 1 -- 172.21.15.72:0/1034215189 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5aa0104b90 msgr2=0x7f5aa019ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:24.913 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:24.910+0000 7f5a96000640 1 --2- 172.21.15.72:0/1034215189 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5aa0104b90 0x7f5aa019ac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:24.913 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:24.910+0000 7f5a96000640 1 -- 172.21.15.72:0/1034215189 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5aa01a28b0 con 0x7f5aa0105d80 2024-04-26T21:55:24.913 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:24.910+0000 7f5a96000640 1 --2- 172.21.15.72:0/1034215189 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5aa0105d80 0x7f5aa019b1b0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f5a8800d950 tx=0x7f5a8800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:55:24.913 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:24.910+0000 7f5a97e00640 1 -- 172.21.15.72:0/1034215189 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a88017070 con 0x7f5aa0105d80 2024-04-26T21:55:24.913 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:24.911+0000 7f5aa5c00640 1 -- 172.21.15.72:0/1034215189 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5aa01a2ba0 con 0x7f5aa0105d80 2024-04-26T21:55:24.913 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:24.911+0000 7f5a97e00640 1 -- 172.21.15.72:0/1034215189 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5a88009c30 con 0x7f5aa0105d80 2024-04-26T21:55:24.913 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:24.911+0000 7f5a97e00640 1 -- 172.21.15.72:0/1034215189 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a8800f620 con 0x7f5aa0105d80 2024-04-26T21:55:24.913 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:24.911+0000 7f5aa5c00640 1 -- 172.21.15.72:0/1034215189 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5aa01a30f0 con 0x7f5aa0105d80 2024-04-26T21:55:24.913 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:24.912+0000 7f5aa5c00640 1 -- 172.21.15.72:0/1034215189 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5aa010beb0 con 0x7f5aa0105d80 2024-04-26T21:55:24.914 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:24.914+0000 7f5a97e00640 1 -- 172.21.15.72:0/1034215189 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f5a88010460 con 0x7f5aa0105d80 2024-04-26T21:55:24.915 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:24.915+0000 7f5a97e00640 1 --2- 172.21.15.72:0/1034215189 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f5a840a3c40 0x7f5a840a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:24.915 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:24.915+0000 7f5a97e00640 1 -- 172.21.15.72:0/1034215189 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7f5a880d0330 con 0x7f5aa0105d80 2024-04-26T21:55:24.916 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:24.916+0000 7f5a9ea00640 1 --2- 172.21.15.72:0/1034215189 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f5a840a3c40 0x7f5a840a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:24.916 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:24.916+0000 7f5a9ea00640 1 --2- 172.21.15.72:0/1034215189 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f5a840a3c40 0x7f5a840a6100 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f5a90005fd0 tx=0x7f5a90005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:55:24.920 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:24.920+0000 7f5a97e00640 1 -- 172.21.15.72:0/1034215189 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f5a8809cad0 con 0x7f5aa0105d80 2024-04-26T21:55:25.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:55:24 smithi072 ceph-mon[28138]: pgmap v448: 97 pgs: 97 active+clean; 634 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 341 B/s wr, 9 op/s 2024-04-26T21:55:25.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:55:24 smithi072 ceph-mon[28138]: from='client.? 172.21.15.72:0/51973114' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T21:55:25.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:55:24 smithi072 ceph-mon[28138]: from='client.? 172.21.15.72:0/2417155384' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T21:55:25.127 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:25.127+0000 7f5aa5c00640 1 -- 172.21.15.72:0/1034215189 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5aa00635d0 con 0x7f5aa0105d80 2024-04-26T21:55:25.129 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:25.129+0000 7f5a97e00640 1 -- 172.21.15.72:0/1034215189 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+22783 (secure 0 0 0) 0x7f5a88009de0 con 0x7f5aa0105d80 2024-04-26T21:55:25.130 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T21:55:25.130 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":91,"fsid":"a1780950-0415-11ef-bc93-c7b262605968","created":"2024-04-26T21:42:01.504173+0000","modified":"2024-04-26T21:54:11.892872+0000","last_up_change":"2024-04-26T21:51:43.482437+0000","last_in_change":"2024-04-26T21:51:18.468587+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:46:29.785427+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:52:48.320391+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T21:52:49.517493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"04e06884-b355-4762-9b29-9034b7201e9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6802","nonce":1416765179},{"type":"v1","addr":"172.21.15.72:6803","nonce":1416765179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6804","nonce":1416765179},{"type":"v1","addr":"172.21.15.72:6805","nonce":1416765179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6808","nonce":1416765179},{"type":"v1","addr":"172.21.15.72:6809","nonce":1416765179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6806","nonce":1416765179},{"type":"v1","addr":"172.21.15.72:6807","nonce":1416765179}]},"public_addr":"172.21.15.72:6803/1416765179","cluster_addr":"172.21.15.72:6805/1416765179","heartbeat_back_addr":"172.21.15.72:6809/1416765179","heartbeat_front_addr":"172.21.15.72:6807/1416765179","state":["exists","up"]},{"osd":1,"uuid":"d75a479a-4a2e-48e9-a01e-2b1c9617416a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":755426949},{"type":"v1","addr":"172.21.15.76:6801","nonce":755426949}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":755426949},{"type":"v1","addr":"172.21.15.76:6803","nonce":755426949}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":755426949},{"type":"v1","addr":"172.21.15.76:6807","nonce":755426949}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":755426949},{"type":"v1","addr":"172.21.15.76:6805","nonce":755426949}]},"public_addr":"172.21.15.76:6801/755426949","cluster_addr":"172.21.15.76:6803/755426949","heartbeat_back_addr":"172.21.15.76:6807/755426949","heartbeat_front_addr":"172.21.15.76:6805/755426949","state":["exists","up"]},{"osd":2,"uuid":"b6f8da58-4428-4e8c-a944-7ea5645d6d92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":852265973},{"type":"v1","addr":"172.21.15.164:6801","nonce":852265973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":852265973},{"type":"v1","addr":"172.21.15.164:6803","nonce":852265973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":852265973},{"type":"v1","addr":"172.21.15.164:6807","nonce":852265973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":852265973},{"type":"v1","addr":"172.21.15.164:6805","nonce":852265973}]},"public_addr":"172.21.15.164:6801/852265973","cluster_addr":"172.21.15.164:6803/852265973","heartbeat_back_addr":"172.21.15.164:6807/852265973","heartbeat_front_addr":"172.21.15.164:6805/852265973","state":["exists","up"]},{"osd":3,"uuid":"76d83409-649e-41e5-b4f4-43bbd5b00ea5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6810","nonce":4027763789},{"type":"v1","addr":"172.21.15.72:6811","nonce":4027763789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6812","nonce":4027763789},{"type":"v1","addr":"172.21.15.72:6813","nonce":4027763789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6816","nonce":4027763789},{"type":"v1","addr":"172.21.15.72:6817","nonce":4027763789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6814","nonce":4027763789},{"type":"v1","addr":"172.21.15.72:6815","nonce":4027763789}]},"public_addr":"172.21.15.72:6811/4027763789","cluster_addr":"172.21.15.72:6813/4027763789","heartbeat_back_addr":"172.21.15.72:6817/4027763789","heartbeat_front_addr":"172.21.15.72:6815/4027763789","state":["exists","up"]},{"osd":4,"uuid":"d2cfeea7-1661-4156-bdcd-f770a8d1a878","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":3718867827},{"type":"v1","addr":"172.21.15.76:6809","nonce":3718867827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":3718867827},{"type":"v1","addr":"172.21.15.76:6811","nonce":3718867827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":3718867827},{"type":"v1","addr":"172.21.15.76:6815","nonce":3718867827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":3718867827},{"type":"v1","addr":"172.21.15.76:6813","nonce":3718867827}]},"public_addr":"172.21.15.76:6809/3718867827","cluster_addr":"172.21.15.76:6811/3718867827","heartbeat_back_addr":"172.21.15.76:6815/3718867827","heartbeat_front_addr":"172.21.15.76:6813/3718867827","state":["exists","up"]},{"osd":5,"uuid":"f3896ad6-a040-4ea3-a431-ff9b7fc05c36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":2543954772},{"type":"v1","addr":"172.21.15.164:6809","nonce":2543954772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":2543954772},{"type":"v1","addr":"172.21.15.164:6811","nonce":2543954772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":2543954772},{"type":"v1","addr":"172.21.15.164:6815","nonce":2543954772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":2543954772},{"type":"v1","addr":"172.21.15.164:6813","nonce":2543954772}]},"public_addr":"172.21.15.164:6809/2543954772","cluster_addr":"172.21.15.164:6811/2543954772","heartbeat_back_addr":"172.21.15.164:6815/2543954772","heartbeat_front_addr":"172.21.15.164:6813/2543954772","state":["exists","up"]},{"osd":6,"uuid":"d5d56650-be7e-4ba5-ba3a-d7d473294dab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6818","nonce":2080659027},{"type":"v1","addr":"172.21.15.72:6819","nonce":2080659027}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6820","nonce":2080659027},{"type":"v1","addr":"172.21.15.72:6821","nonce":2080659027}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6824","nonce":2080659027},{"type":"v1","addr":"172.21.15.72:6825","nonce":2080659027}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6822","nonce":2080659027},{"type":"v1","addr":"172.21.15.72:6823","nonce":2080659027}]},"public_addr":"172.21.15.72:6819/2080659027","cluster_addr":"172.21.15.72:6821/2080659027","heartbeat_back_addr":"172.21.15.72:6825/2080659027","heartbeat_front_addr":"172.21.15.72:6823/2080659027","state":["exists","up"]},{"osd":7,"uuid":"18975b5a-9309-4a0f-9f37-3ce09c2e745d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":3144631323},{"type":"v1","addr":"172.21.15.76:6817","nonce":3144631323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":3144631323},{"type":"v1","addr":"172.21.15.76:6819","nonce":3144631323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":3144631323},{"type":"v1","addr":"172.21.15.76:6823","nonce":3144631323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":3144631323},{"type":"v1","addr":"172.21.15.76:6821","nonce":3144631323}]},"public_addr":"172.21.15.76:6817/3144631323","cluster_addr":"172.21.15.76:6819/3144631323","heartbeat_back_addr":"172.21.15.76:6823/3144631323","heartbeat_front_addr":"172.21.15.76:6821/3144631323","state":["exists","up"]},{"osd":8,"uuid":"779e9c1e-8e97-48d4-a854-68ca13f38e0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":627888883},{"type":"v1","addr":"172.21.15.164:6817","nonce":627888883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":627888883},{"type":"v1","addr":"172.21.15.164:6819","nonce":627888883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":627888883},{"type":"v1","addr":"172.21.15.164:6823","nonce":627888883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":627888883},{"type":"v1","addr":"172.21.15.164:6821","nonce":627888883}]},"public_addr":"172.21.15.164:6817/627888883","cluster_addr":"172.21.15.164:6819/627888883","heartbeat_back_addr":"172.21.15.164:6823/627888883","heartbeat_front_addr":"172.21.15.164:6821/627888883","state":["exists","up"]},{"osd":9,"uuid":"d0317850-7b2d-4850-b2cf-0b919505c1fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6826","nonce":1273118485},{"type":"v1","addr":"172.21.15.72:6827","nonce":1273118485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6828","nonce":1273118485},{"type":"v1","addr":"172.21.15.72:6829","nonce":1273118485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6832","nonce":1273118485},{"type":"v1","addr":"172.21.15.72:6833","nonce":1273118485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6830","nonce":1273118485},{"type":"v1","addr":"172.21.15.72:6831","nonce":1273118485}]},"public_addr":"172.21.15.72:6827/1273118485","cluster_addr":"172.21.15.72:6829/1273118485","heartbeat_back_addr":"172.21.15.72:6833/1273118485","heartbeat_front_addr":"172.21.15.72:6831/1273118485","state":["exists","up"]},{"osd":10,"uuid":"e0aa38d8-6011-4d2a-b67b-5bff6d999e47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":2808954459},{"type":"v1","addr":"172.21.15.76:6825","nonce":2808954459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":2808954459},{"type":"v1","addr":"172.21.15.76:6827","nonce":2808954459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":2808954459},{"type":"v1","addr":"172.21.15.76:6831","nonce":2808954459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":2808954459},{"type":"v1","addr":"172.21.15.76:6829","nonce":2808954459}]},"public_addr":"172.21.15.76:6825/2808954459","cluster_addr":"172.21.15.76:6827/2808954459","heartbeat_back_addr":"172.21.15.76:6831/2808954459","heartbeat_front_addr":"172.21.15.76:6829/2808954459","state":["exists","up"]},{"osd":11,"uuid":"981b3d4d-916e-4937-9916-7cc21a891e43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":2354951784},{"type":"v1","addr":"172.21.15.164:6825","nonce":2354951784}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":2354951784},{"type":"v1","addr":"172.21.15.164:6827","nonce":2354951784}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":2354951784},{"type":"v1","addr":"172.21.15.164:6831","nonce":2354951784}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":2354951784},{"type":"v1","addr":"172.21.15.164:6829","nonce":2354951784}]},"public_addr":"172.21.15.164:6825/2354951784","cluster_addr":"172.21.15.164:6827/2354951784","heartbeat_back_addr":"172.21.15.164:6831/2354951784","heartbeat_front_addr":"172.21.15.164:6829/2354951784","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:18.997924+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:52.439081+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:46:25.893288+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:47:00.112807+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:47:35.116874+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:48:09.216344+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:48:45.028853+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:49:19.905397+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:49:55.030536+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:50:30.767662+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:51:06.423267+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:51:41.631211+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.1a","mappings":[{"from":5,"to":7}]},{"pgid":"3.1b","mappings":[{"from":4,"to":6}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.28","mappings":[{"from":5,"to":6}]},{"pgid":"3.2a","mappings":[{"from":3,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.72:0/416277784":"2024-04-27T21:43:09.602791+0000","172.21.15.72:6800/4010358672":"2024-04-27T21:43:09.602791+0000","172.21.15.72:6801/1012617371":"2024-04-27T21:42:34.684837+0000","172.21.15.72:6801/4010358672":"2024-04-27T21:43:09.602791+0000","172.21.15.72:0/3555448179":"2024-04-27T21:43:09.602791+0000","172.21.15.72:6800/1012617371":"2024-04-27T21:42:34.684837+0000","172.21.15.72:0/2920378761":"2024-04-27T21:42:34.684837+0000","172.21.15.72:0/1933740041":"2024-04-27T21:43:09.602791+0000","172.21.15.72:0/3225295775":"2024-04-27T21:42:34.684837+0000","172.21.15.72:0/2326016180":"2024-04-27T21:42:34.684837+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:55:25.132 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:25.132+0000 7f5aa5c00640 1 -- 172.21.15.72:0/1034215189 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f5a840a3c40 msgr2=0x7f5a840a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:25.132 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:25.132+0000 7f5aa5c00640 1 --2- 172.21.15.72:0/1034215189 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f5a840a3c40 0x7f5a840a6100 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f5a90005fd0 tx=0x7f5a90005850 comp rx=0 tx=0).stop 2024-04-26T21:55:25.132 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:25.132+0000 7f5aa5c00640 1 -- 172.21.15.72:0/1034215189 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5aa0105d80 msgr2=0x7f5aa019b1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:25.132 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:25.132+0000 7f5aa5c00640 1 --2- 172.21.15.72:0/1034215189 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5aa0105d80 0x7f5aa019b1b0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f5a8800d950 tx=0x7f5a8800de10 comp rx=0 tx=0).stop 2024-04-26T21:55:25.132 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:25.132+0000 7f5aa5c00640 1 -- 172.21.15.72:0/1034215189 shutdown_connections 2024-04-26T21:55:25.133 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:25.132+0000 7f5aa5c00640 1 --2- 172.21.15.72:0/1034215189 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f5a840a3c40 0x7f5a840a6100 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:25.133 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:25.132+0000 7f5aa5c00640 1 --2- 172.21.15.72:0/1034215189 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5aa0106740 0x7f5aa01a2210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:25.133 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:25.132+0000 7f5aa5c00640 1 --2- 172.21.15.72:0/1034215189 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5aa0105d80 0x7f5aa019b1b0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:25.133 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:25.132+0000 7f5aa5c00640 1 --2- 172.21.15.72:0/1034215189 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5aa0104b90 0x7f5aa019ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:25.133 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:25.132+0000 7f5aa5c00640 1 -- 172.21.15.72:0/1034215189 >> 172.21.15.72:0/1034215189 conn(0x7f5aa0100120 msgr2=0x7f5aa0109080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:55:25.133 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:25.132+0000 7f5aa5c00640 1 -- 172.21.15.72:0/1034215189 shutdown_connections 2024-04-26T21:55:25.133 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:25.132+0000 7f5aa5c00640 1 -- 172.21.15.72:0/1034215189 wait complete. 2024-04-26T21:55:25.151 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:46:29.785427+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T21:52:48.320391+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T21:52:49.517493+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:55:25.152 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:55:25.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:55:24 smithi164 ceph-mon[31916]: pgmap v448: 97 pgs: 97 active+clean; 634 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 341 B/s wr, 9 op/s 2024-04-26T21:55:25.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:55:24 smithi164 ceph-mon[31916]: from='client.? 172.21.15.72:0/51973114' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T21:55:25.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:55:24 smithi164 ceph-mon[31916]: from='client.? 172.21.15.72:0/2417155384' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T21:55:25.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:55:24 smithi076 ceph-mon[31823]: pgmap v448: 97 pgs: 97 active+clean; 634 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 341 B/s wr, 9 op/s 2024-04-26T21:55:25.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:55:24 smithi076 ceph-mon[31823]: from='client.? 172.21.15.72:0/51973114' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T21:55:25.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:55:24 smithi076 ceph-mon[31823]: from='client.? 172.21.15.72:0/2417155384' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T21:55:25.361 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:25.361+0000 7f8a13e00640 1 -- 172.21.15.72:0/1655431297 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8a14078530 msgr2=0x7f8a14078990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:25.361 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:25.361+0000 7f8a13e00640 1 --2- 172.21.15.72:0/1655431297 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8a14078530 0x7f8a14078990 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f89fc009900 tx=0x7f89fc02a3f0 comp rx=0 tx=0).stop 2024-04-26T21:55:25.362 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:25.361+0000 7f8a13e00640 1 -- 172.21.15.72:0/1655431297 shutdown_connections 2024-04-26T21:55:25.362 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:25.361+0000 7f8a13e00640 1 --2- 172.21.15.72:0/1655431297 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8a14078ed0 0x7f8a14079770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:25.362 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:25.361+0000 7f8a13e00640 1 --2- 172.21.15.72:0/1655431297 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8a14078530 0x7f8a14078990 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:25.362 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:25.361+0000 7f8a13e00640 1 --2- 172.21.15.72:0/1655431297 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8a140772f0 0x7f8a14077710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:25.362 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:25.361+0000 7f8a13e00640 1 -- 172.21.15.72:0/1655431297 >> 172.21.15.72:0/1655431297 conn(0x7f8a140fffe0 msgr2=0x7f8a14102400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:55:25.362 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:25.361+0000 7f8a13e00640 1 -- 172.21.15.72:0/1655431297 shutdown_connections 2024-04-26T21:55:25.362 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:25.361+0000 7f8a13e00640 1 -- 172.21.15.72:0/1655431297 wait complete. 2024-04-26T21:55:25.364 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:25.362+0000 7f8a13e00640 1 Processor -- start 2024-04-26T21:55:25.364 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:25.362+0000 7f8a13e00640 1 -- start start 2024-04-26T21:55:25.364 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:25.362+0000 7f8a13e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8a140772f0 0x7f8a1419efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:25.364 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:25.362+0000 7f8a13e00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8a14078530 0x7f8a1419f510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:25.364 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:25.363+0000 7f8a13e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8a14078ed0 0x7f8a141a6580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:25.364 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:25.363+0000 7f8a13e00640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f8a14114880 con 0x7f8a14078ed0 2024-04-26T21:55:25.365 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:25.363+0000 7f8a13e00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f8a14114700 con 0x7f8a14078530 2024-04-26T21:55:25.365 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:25.363+0000 7f8a13e00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f8a14114a00 con 0x7f8a140772f0 2024-04-26T21:55:25.365 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:25.363+0000 7f8a13400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8a14078ed0 0x7f8a141a6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:25.365 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:25.363+0000 7f8a13400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8a14078ed0 0x7f8a141a6580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:44116/0 (socket says 172.21.15.72:44116) 2024-04-26T21:55:25.365 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:25.363+0000 7f8a13400640 1 -- 172.21.15.72:0/2116073535 learned_addr learned my addr 172.21.15.72:0/2116073535 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:55:25.365 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:25.363+0000 7f8a0be00640 1 --2- 172.21.15.72:0/2116073535 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8a14078530 0x7f8a1419f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:25.365 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:25.363+0000 7f8a12a00640 1 --2- 172.21.15.72:0/2116073535 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8a140772f0 0x7f8a1419efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:25.365 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:25.363+0000 7f8a13400640 1 -- 172.21.15.72:0/2116073535 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8a140772f0 msgr2=0x7f8a1419efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:25.365 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:25.363+0000 7f8a13400640 1 --2- 172.21.15.72:0/2116073535 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8a140772f0 0x7f8a1419efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:25.365 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:25.363+0000 7f8a13400640 1 -- 172.21.15.72:0/2116073535 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8a14078530 msgr2=0x7f8a1419f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:25.365 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:25.363+0000 7f8a13400640 1 --2- 172.21.15.72:0/2116073535 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8a14078530 0x7f8a1419f510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:25.365 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:25.363+0000 7f8a13400640 1 -- 172.21.15.72:0/2116073535 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a141a6bf0 con 0x7f8a14078ed0 2024-04-26T21:55:25.365 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:25.363+0000 7f8a0be00640 1 --2- 172.21.15.72:0/2116073535 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8a14078530 0x7f8a1419f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:55:25.366 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:25.364+0000 7f8a13400640 1 --2- 172.21.15.72:0/2116073535 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8a14078ed0 0x7f8a141a6580 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f8a0c00eaa0 tx=0x7f8a0c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:55:25.366 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:25.364+0000 7f8a0b400640 1 -- 172.21.15.72:0/2116073535 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a0c00cdb0 con 0x7f8a14078ed0 2024-04-26T21:55:25.366 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:25.364+0000 7f8a0b400640 1 -- 172.21.15.72:0/2116073535 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8a0c005020 con 0x7f8a14078ed0 2024-04-26T21:55:25.366 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:25.364+0000 7f8a13e00640 1 -- 172.21.15.72:0/2116073535 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a141a6ee0 con 0x7f8a14078ed0 2024-04-26T21:55:25.366 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:25.364+0000 7f8a0b400640 1 -- 172.21.15.72:0/2116073535 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a0c0116d0 con 0x7f8a14078ed0 2024-04-26T21:55:25.366 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:25.364+0000 7f8a13e00640 1 -- 172.21.15.72:0/2116073535 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a141a73a0 con 0x7f8a14078ed0 2024-04-26T21:55:25.366 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:25.365+0000 7f8a13e00640 1 -- 172.21.15.72:0/2116073535 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a141113c0 con 0x7f8a14078ed0 2024-04-26T21:55:25.367 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:25.367+0000 7f8a0b400640 1 -- 172.21.15.72:0/2116073535 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f8a0c0051d0 con 0x7f8a14078ed0 2024-04-26T21:55:25.368 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:25.368+0000 7f8a0b400640 1 --2- 172.21.15.72:0/2116073535 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f89f80a3c40 0x7f89f80a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:25.369 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:25.369+0000 7f8a0b400640 1 -- 172.21.15.72:0/2116073535 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7f8a0c0d0810 con 0x7f8a14078ed0 2024-04-26T21:55:25.369 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:25.369+0000 7f8a12a00640 1 --2- 172.21.15.72:0/2116073535 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f89f80a3c40 0x7f89f80a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:25.369 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:25.369+0000 7f8a12a00640 1 --2- 172.21.15.72:0/2116073535 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f89f80a3c40 0x7f89f80a6100 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f8a04005d00 tx=0x7f8a04005c50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:55:25.372 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:25.373+0000 7f8a0b400640 1 -- 172.21.15.72:0/2116073535 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f8a0c016030 con 0x7f8a14078ed0 2024-04-26T21:55:25.574 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:25.574+0000 7f8a13e00640 1 -- 172.21.15.72:0/2116073535 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f8a1402dd20 con 0x7f8a14078ed0 2024-04-26T21:55:25.575 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:25.575+0000 7f8a0b400640 1 -- 172.21.15.72:0/2116073535 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) ==== 93+0+10 (secure 0 0 0) 0x7f8a0c09d010 con 0x7f8a14078ed0 2024-04-26T21:55:25.575 INFO:teuthology.orchestra.run.smithi072.stdout:pg_num: 1 2024-04-26T21:55:25.578 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:25.577+0000 7f8a13e00640 1 -- 172.21.15.72:0/2116073535 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f89f80a3c40 msgr2=0x7f89f80a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:25.578 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:25.577+0000 7f8a13e00640 1 --2- 172.21.15.72:0/2116073535 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f89f80a3c40 0x7f89f80a6100 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f8a04005d00 tx=0x7f8a04005c50 comp rx=0 tx=0).stop 2024-04-26T21:55:25.578 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:25.577+0000 7f8a13e00640 1 -- 172.21.15.72:0/2116073535 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8a14078ed0 msgr2=0x7f8a141a6580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:25.578 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:25.577+0000 7f8a13e00640 1 --2- 172.21.15.72:0/2116073535 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8a14078ed0 0x7f8a141a6580 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f8a0c00eaa0 tx=0x7f8a0c00ef60 comp rx=0 tx=0).stop 2024-04-26T21:55:25.578 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:25.578+0000 7f8a13e00640 1 -- 172.21.15.72:0/2116073535 shutdown_connections 2024-04-26T21:55:25.578 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:25.578+0000 7f8a13e00640 1 --2- 172.21.15.72:0/2116073535 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8a14078ed0 0x7f8a141a6580 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:25.578 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:25.578+0000 7f8a13e00640 1 --2- 172.21.15.72:0/2116073535 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8a14078530 0x7f8a1419f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:25.579 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:25.578+0000 7f8a13e00640 1 --2- 172.21.15.72:0/2116073535 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f89f80a3c40 0x7f89f80a6100 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:25.579 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:25.578+0000 7f8a13e00640 1 --2- 172.21.15.72:0/2116073535 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8a140772f0 0x7f8a1419efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:25.579 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:25.578+0000 7f8a13e00640 1 -- 172.21.15.72:0/2116073535 >> 172.21.15.72:0/2116073535 conn(0x7f8a140fffe0 msgr2=0x7f8a14101ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:55:25.579 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:25.578+0000 7f8a13e00640 1 -- 172.21.15.72:0/2116073535 shutdown_connections 2024-04-26T21:55:25.579 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:25.578+0000 7f8a13e00640 1 -- 172.21.15.72:0/2116073535 wait complete. 2024-04-26T21:55:25.597 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T21:55:25.795 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:25.795+0000 7fbf76400640 1 -- 172.21.15.72:0/2399115466 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbf70109fc0 msgr2=0x7fbf7010a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:25.795 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:25.795+0000 7fbf76400640 1 --2- 172.21.15.72:0/2399115466 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbf70109fc0 0x7fbf7010a440 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fbf58009900 tx=0x7fbf5802a410 comp rx=0 tx=0).stop 2024-04-26T21:55:25.795 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:25.795+0000 7fbf76400640 1 -- 172.21.15.72:0/2399115466 shutdown_connections 2024-04-26T21:55:25.795 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:25.795+0000 7fbf76400640 1 --2- 172.21.15.72:0/2399115466 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbf7010a980 0x7fbf701112a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:25.796 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:25.795+0000 7fbf76400640 1 --2- 172.21.15.72:0/2399115466 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbf70109fc0 0x7fbf7010a440 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:25.796 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:25.795+0000 7fbf76400640 1 --2- 172.21.15.72:0/2399115466 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbf70108e70 0x7fbf70109290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:25.796 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:25.795+0000 7fbf76400640 1 -- 172.21.15.72:0/2399115466 >> 172.21.15.72:0/2399115466 conn(0x7fbf70077e00 msgr2=0x7fbf7007a240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:55:25.796 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:25.795+0000 7fbf76400640 1 -- 172.21.15.72:0/2399115466 shutdown_connections 2024-04-26T21:55:25.796 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:25.795+0000 7fbf76400640 1 -- 172.21.15.72:0/2399115466 wait complete. 2024-04-26T21:55:25.798 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:25.796+0000 7fbf76400640 1 Processor -- start 2024-04-26T21:55:25.798 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:25.796+0000 7fbf76400640 1 -- start start 2024-04-26T21:55:25.798 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:25.796+0000 7fbf76400640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbf70108e70 0x7fbf7019ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:25.798 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:25.796+0000 7fbf76400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbf70109fc0 0x7fbf7019f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:25.798 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:25.796+0000 7fbf76400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbf7010a980 0x7fbf701a6530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:25.798 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:25.796+0000 7fbf76400640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fbf70114850 con 0x7fbf70109fc0 2024-04-26T21:55:25.798 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:25.796+0000 7fbf76400640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fbf701146d0 con 0x7fbf70108e70 2024-04-26T21:55:25.798 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:25.796+0000 7fbf76400640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fbf701149d0 con 0x7fbf7010a980 2024-04-26T21:55:25.798 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:25.797+0000 7fbf67e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbf70109fc0 0x7fbf7019f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:25.798 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:25.797+0000 7fbf67e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbf70109fc0 0x7fbf7019f4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:44124/0 (socket says 172.21.15.72:44124) 2024-04-26T21:55:25.799 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:25.797+0000 7fbf67e00640 1 -- 172.21.15.72:0/341034674 learned_addr learned my addr 172.21.15.72:0/341034674 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:55:25.799 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:25.797+0000 7fbf75a00640 1 --2- 172.21.15.72:0/341034674 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbf7010a980 0x7fbf701a6530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:25.799 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:25.797+0000 7fbf75000640 1 --2- 172.21.15.72:0/341034674 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbf70108e70 0x7fbf7019ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:25.799 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:25.797+0000 7fbf67e00640 1 -- 172.21.15.72:0/341034674 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbf7010a980 msgr2=0x7fbf701a6530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:25.799 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:25.797+0000 7fbf67e00640 1 --2- 172.21.15.72:0/341034674 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbf7010a980 0x7fbf701a6530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:25.799 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:25.797+0000 7fbf67e00640 1 -- 172.21.15.72:0/341034674 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbf70108e70 msgr2=0x7fbf7019ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:25.799 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:25.797+0000 7fbf67e00640 1 --2- 172.21.15.72:0/341034674 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbf70108e70 0x7fbf7019ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:25.799 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:25.797+0000 7fbf67e00640 1 -- 172.21.15.72:0/341034674 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf701a6ba0 con 0x7fbf70109fc0 2024-04-26T21:55:25.799 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:25.797+0000 7fbf67e00640 1 --2- 172.21.15.72:0/341034674 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbf70109fc0 0x7fbf7019f4c0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fbf58005ab0 tx=0x7fbf580042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:55:25.799 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:25.798+0000 7fbf6e000640 1 -- 172.21.15.72:0/341034674 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf5803f030 con 0x7fbf70109fc0 2024-04-26T21:55:25.799 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:25.798+0000 7fbf6e000640 1 -- 172.21.15.72:0/341034674 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbf58003c90 con 0x7fbf70109fc0 2024-04-26T21:55:25.799 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:25.798+0000 7fbf6e000640 1 -- 172.21.15.72:0/341034674 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf58035070 con 0x7fbf70109fc0 2024-04-26T21:55:25.799 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:25.798+0000 7fbf76400640 1 -- 172.21.15.72:0/341034674 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf701a6e30 con 0x7fbf70109fc0 2024-04-26T21:55:25.799 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:25.798+0000 7fbf76400640 1 -- 172.21.15.72:0/341034674 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf701a7230 con 0x7fbf70109fc0 2024-04-26T21:55:25.800 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:25.799+0000 7fbf76400640 1 -- 172.21.15.72:0/341034674 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf70110250 con 0x7fbf70109fc0 2024-04-26T21:55:25.801 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:25.801+0000 7fbf6e000640 1 -- 172.21.15.72:0/341034674 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fbf58042030 con 0x7fbf70109fc0 2024-04-26T21:55:25.802 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:25.802+0000 7fbf6e000640 1 --2- 172.21.15.72:0/341034674 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fbf540a3c40 0x7fbf540a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:25.802 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:25.802+0000 7fbf6e000640 1 -- 172.21.15.72:0/341034674 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7fbf580f10d0 con 0x7fbf70109fc0 2024-04-26T21:55:25.803 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:25.803+0000 7fbf75000640 1 --2- 172.21.15.72:0/341034674 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fbf540a3c40 0x7fbf540a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:25.803 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:25.803+0000 7fbf75000640 1 --2- 172.21.15.72:0/341034674 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fbf540a3c40 0x7fbf540a6100 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fbf60005fd0 tx=0x7fbf60005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:55:25.806 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:25.807+0000 7fbf6e000640 1 -- 172.21.15.72:0/341034674 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fbf580bd870 con 0x7fbf70109fc0 2024-04-26T21:55:26.022 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:26.022+0000 7fbf76400640 1 -- 172.21.15.72:0/341034674 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fbf7002dd20 con 0x7fbf70109fc0 2024-04-26T21:55:26.023 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:26.023+0000 7fbf6e000640 1 -- 172.21.15.72:0/341034674 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v91) ==== 104+0+11 (secure 0 0 0) 0x7fbf580bd170 con 0x7fbf70109fc0 2024-04-26T21:55:26.023 INFO:teuthology.orchestra.run.smithi072.stdout:pg_num: 32 2024-04-26T21:55:26.025 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:26.025+0000 7fbf76400640 1 -- 172.21.15.72:0/341034674 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fbf540a3c40 msgr2=0x7fbf540a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:26.026 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:26.025+0000 7fbf76400640 1 --2- 172.21.15.72:0/341034674 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fbf540a3c40 0x7fbf540a6100 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fbf60005fd0 tx=0x7fbf60005850 comp rx=0 tx=0).stop 2024-04-26T21:55:26.026 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:26.025+0000 7fbf76400640 1 -- 172.21.15.72:0/341034674 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbf70109fc0 msgr2=0x7fbf7019f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:26.026 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:26.025+0000 7fbf76400640 1 --2- 172.21.15.72:0/341034674 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbf70109fc0 0x7fbf7019f4c0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fbf58005ab0 tx=0x7fbf580042f0 comp rx=0 tx=0).stop 2024-04-26T21:55:26.026 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:26.025+0000 7fbf76400640 1 -- 172.21.15.72:0/341034674 shutdown_connections 2024-04-26T21:55:26.026 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:26.025+0000 7fbf76400640 1 --2- 172.21.15.72:0/341034674 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fbf540a3c40 0x7fbf540a6100 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:26.026 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:26.025+0000 7fbf76400640 1 --2- 172.21.15.72:0/341034674 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbf7010a980 0x7fbf701a6530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:26.026 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:26.025+0000 7fbf76400640 1 --2- 172.21.15.72:0/341034674 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbf70109fc0 0x7fbf7019f4c0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:26.026 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:26.025+0000 7fbf76400640 1 --2- 172.21.15.72:0/341034674 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbf70108e70 0x7fbf7019ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:26.026 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:26.025+0000 7fbf76400640 1 -- 172.21.15.72:0/341034674 >> 172.21.15.72:0/341034674 conn(0x7fbf70077e00 msgr2=0x7fbf7010d420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:55:26.026 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:26.026+0000 7fbf76400640 1 -- 172.21.15.72:0/341034674 shutdown_connections 2024-04-26T21:55:26.026 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:26.026+0000 7fbf76400640 1 -- 172.21.15.72:0/341034674 wait complete. 2024-04-26T21:55:26.044 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T21:55:26.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:55:25 smithi164 ceph-mon[31916]: from='client.? 172.21.15.72:0/1034215189' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:55:26.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:55:25 smithi164 ceph-mon[31916]: from='client.? 172.21.15.72:0/2116073535' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T21:55:26.239 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:26.239+0000 7f8db8600640 1 -- 172.21.15.72:0/3670753302 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8db0108650 msgr2=0x7f8db010aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:26.239 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:26.239+0000 7f8db8600640 1 --2- 172.21.15.72:0/3670753302 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8db0108650 0x7f8db010aaa0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f8da8009ca0 tx=0x7f8da802a8c0 comp rx=0 tx=0).stop 2024-04-26T21:55:26.239 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:26.239+0000 7f8db8600640 1 -- 172.21.15.72:0/3670753302 shutdown_connections 2024-04-26T21:55:26.239 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:26.239+0000 7f8db8600640 1 --2- 172.21.15.72:0/3670753302 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8db0108650 0x7f8db010aaa0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:26.239 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:26.239+0000 7f8db8600640 1 --2- 172.21.15.72:0/3670753302 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8db0105d20 0x7f8db0108110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:26.239 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:26.239+0000 7f8db8600640 1 --2- 172.21.15.72:0/3670753302 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8db01033f0 0x7f8db01057e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:26.239 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:26.239+0000 7f8db8600640 1 -- 172.21.15.72:0/3670753302 >> 172.21.15.72:0/3670753302 conn(0x7f8db00fcde0 msgr2=0x7f8db00ff220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:55:26.240 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:26.239+0000 7f8db8600640 1 -- 172.21.15.72:0/3670753302 shutdown_connections 2024-04-26T21:55:26.240 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:26.239+0000 7f8db8600640 1 -- 172.21.15.72:0/3670753302 wait complete. 2024-04-26T21:55:26.242 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:26.240+0000 7f8db8600640 1 Processor -- start 2024-04-26T21:55:26.242 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:26.240+0000 7f8db8600640 1 -- start start 2024-04-26T21:55:26.242 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:26.240+0000 7f8db8600640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8db01033f0 0x7f8db019ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:26.242 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:26.240+0000 7f8db8600640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8db0105d20 0x7f8db019b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:26.242 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:26.240+0000 7f8db8600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8db0108650 0x7f8db01a21e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:26.242 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:26.240+0000 7f8db8600640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f8db010e3c0 con 0x7f8db01033f0 2024-04-26T21:55:26.242 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:26.240+0000 7f8db8600640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f8db010e240 con 0x7f8db0105d20 2024-04-26T21:55:26.242 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:26.240+0000 7f8db8600640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f8db010e540 con 0x7f8db0108650 2024-04-26T21:55:26.242 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:26.240+0000 7f8db5c00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8db01033f0 0x7f8db019ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:26.242 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:26.241+0000 7f8db5c00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8db01033f0 0x7f8db019ac70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:44138/0 (socket says 172.21.15.72:44138) 2024-04-26T21:55:26.243 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:26.241+0000 7f8db5c00640 1 -- 172.21.15.72:0/2554165815 learned_addr learned my addr 172.21.15.72:0/2554165815 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:55:26.243 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:26.241+0000 7f8db6600640 1 --2- 172.21.15.72:0/2554165815 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8db0108650 0x7f8db01a21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:26.243 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:26.241+0000 7f8db5200640 1 --2- 172.21.15.72:0/2554165815 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8db0105d20 0x7f8db019b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:26.243 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:26.241+0000 7f8db5c00640 1 -- 172.21.15.72:0/2554165815 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8db0108650 msgr2=0x7f8db01a21e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:26.243 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:26.241+0000 7f8db5c00640 1 --2- 172.21.15.72:0/2554165815 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8db0108650 0x7f8db01a21e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:26.243 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:26.241+0000 7f8db5c00640 1 -- 172.21.15.72:0/2554165815 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8db0105d20 msgr2=0x7f8db019b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:26.243 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:26.241+0000 7f8db5c00640 1 --2- 172.21.15.72:0/2554165815 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8db0105d20 0x7f8db019b1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:26.243 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:26.241+0000 7f8db5c00640 1 -- 172.21.15.72:0/2554165815 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8db01a2880 con 0x7f8db01033f0 2024-04-26T21:55:26.243 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:26.241+0000 7f8db5c00640 1 --2- 172.21.15.72:0/2554165815 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8db01033f0 0x7f8db019ac70 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f8da000e9f0 tx=0x7f8da000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:55:26.243 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:26.242+0000 7f8dae000640 1 -- 172.21.15.72:0/2554165815 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8da000cd80 con 0x7f8db01033f0 2024-04-26T21:55:26.243 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:26.242+0000 7f8db8600640 1 -- 172.21.15.72:0/2554165815 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8db01a2b70 con 0x7f8db01033f0 2024-04-26T21:55:26.243 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:26.242+0000 7f8dae000640 1 -- 172.21.15.72:0/2554165815 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8da00054b0 con 0x7f8db01033f0 2024-04-26T21:55:26.243 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:26.242+0000 7f8dae000640 1 -- 172.21.15.72:0/2554165815 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8da0010430 con 0x7f8db01033f0 2024-04-26T21:55:26.243 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:26.242+0000 7f8db8600640 1 -- 172.21.15.72:0/2554165815 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8db01a3070 con 0x7f8db01033f0 2024-04-26T21:55:26.244 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:26.243+0000 7f8db8600640 1 -- 172.21.15.72:0/2554165815 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8db010a140 con 0x7f8db01033f0 2024-04-26T21:55:26.245 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:26.245+0000 7f8dae000640 1 -- 172.21.15.72:0/2554165815 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f8da00046c0 con 0x7f8db01033f0 2024-04-26T21:55:26.246 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:26.246+0000 7f8dae000640 1 --2- 172.21.15.72:0/2554165815 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f8d940a3c40 0x7f8d940a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:26.247 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:26.247+0000 7f8dae000640 1 -- 172.21.15.72:0/2554165815 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7f8da00d0b50 con 0x7f8db01033f0 2024-04-26T21:55:26.247 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:26.247+0000 7f8db5200640 1 --2- 172.21.15.72:0/2554165815 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f8d940a3c40 0x7f8d940a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:26.247 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:26.248+0000 7f8db5200640 1 --2- 172.21.15.72:0/2554165815 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f8d940a3c40 0x7f8d940a6100 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f8d9c005fd0 tx=0x7f8d9c005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:55:26.251 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:26.251+0000 7f8dae000640 1 -- 172.21.15.72:0/2554165815 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f8da009d350 con 0x7f8db01033f0 2024-04-26T21:55:26.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:55:25 smithi076 ceph-mon[31823]: from='client.? 172.21.15.72:0/1034215189' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:55:26.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:55:25 smithi076 ceph-mon[31823]: from='client.? 172.21.15.72:0/2116073535' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T21:55:26.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:55:25 smithi072 ceph-mon[28138]: from='client.? 172.21.15.72:0/1034215189' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:55:26.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:55:25 smithi072 ceph-mon[28138]: from='client.? 172.21.15.72:0/2116073535' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T21:55:26.452 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:26.452+0000 7f8db8600640 1 -- 172.21.15.72:0/2554165815 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f8db002dd30 con 0x7f8db01033f0 2024-04-26T21:55:26.453 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:26.453+0000 7f8dae000640 1 -- 172.21.15.72:0/2554165815 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v91) ==== 100+0+11 (secure 0 0 0) 0x7f8da009cc50 con 0x7f8db01033f0 2024-04-26T21:55:26.453 INFO:teuthology.orchestra.run.smithi072.stdout:pg_num: 64 2024-04-26T21:55:26.456 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:26.455+0000 7f8db8600640 1 -- 172.21.15.72:0/2554165815 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f8d940a3c40 msgr2=0x7f8d940a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:26.456 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:26.455+0000 7f8db8600640 1 --2- 172.21.15.72:0/2554165815 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f8d940a3c40 0x7f8d940a6100 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f8d9c005fd0 tx=0x7f8d9c005850 comp rx=0 tx=0).stop 2024-04-26T21:55:26.456 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:26.455+0000 7f8db8600640 1 -- 172.21.15.72:0/2554165815 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8db01033f0 msgr2=0x7f8db019ac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:26.456 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:26.455+0000 7f8db8600640 1 --2- 172.21.15.72:0/2554165815 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8db01033f0 0x7f8db019ac70 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f8da000e9f0 tx=0x7f8da000edf0 comp rx=0 tx=0).stop 2024-04-26T21:55:26.456 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:26.456+0000 7f8db8600640 1 -- 172.21.15.72:0/2554165815 shutdown_connections 2024-04-26T21:55:26.456 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:26.456+0000 7f8db8600640 1 --2- 172.21.15.72:0/2554165815 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f8d940a3c40 0x7f8d940a6100 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:26.456 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:26.456+0000 7f8db8600640 1 --2- 172.21.15.72:0/2554165815 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8db0108650 0x7f8db01a21e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:26.456 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:26.456+0000 7f8db8600640 1 --2- 172.21.15.72:0/2554165815 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8db0105d20 0x7f8db019b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:26.456 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:26.456+0000 7f8db8600640 1 --2- 172.21.15.72:0/2554165815 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8db01033f0 0x7f8db019ac70 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:26.456 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:26.456+0000 7f8db8600640 1 -- 172.21.15.72:0/2554165815 >> 172.21.15.72:0/2554165815 conn(0x7f8db00fcde0 msgr2=0x7f8db0102aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:55:26.457 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:26.456+0000 7f8db8600640 1 -- 172.21.15.72:0/2554165815 shutdown_connections 2024-04-26T21:55:26.457 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:26.456+0000 7f8db8600640 1 -- 172.21.15.72:0/2554165815 wait complete. 2024-04-26T21:55:26.475 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-04-26T21:55:26.475 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:55:26.659 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:26.659+0000 7f8185000640 1 -- 172.21.15.72:0/4188889915 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8180106740 msgr2=0x7f818010cfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:26.659 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:26.659+0000 7f8185000640 1 --2- 172.21.15.72:0/4188889915 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8180106740 0x7f818010cfd0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f8178009be0 tx=0x7f817802a6c0 comp rx=0 tx=0).stop 2024-04-26T21:55:26.659 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:26.659+0000 7f8185000640 1 -- 172.21.15.72:0/4188889915 shutdown_connections 2024-04-26T21:55:26.659 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:26.659+0000 7f8185000640 1 --2- 172.21.15.72:0/4188889915 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8180106740 0x7f818010cfd0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:26.659 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:26.659+0000 7f8185000640 1 --2- 172.21.15.72:0/4188889915 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8180105d80 0x7f8180106200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:26.659 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:26.659+0000 7f8185000640 1 --2- 172.21.15.72:0/4188889915 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8180104b90 0x7f8180104fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:26.660 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:26.659+0000 7f8185000640 1 -- 172.21.15.72:0/4188889915 >> 172.21.15.72:0/4188889915 conn(0x7f8180100120 msgr2=0x7f8180102540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:55:26.660 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:26.659+0000 7f8185000640 1 -- 172.21.15.72:0/4188889915 shutdown_connections 2024-04-26T21:55:26.660 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:26.659+0000 7f8185000640 1 -- 172.21.15.72:0/4188889915 wait complete. 2024-04-26T21:55:26.661 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:26.660+0000 7f8185000640 1 Processor -- start 2024-04-26T21:55:26.661 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:26.660+0000 7f8185000640 1 -- start start 2024-04-26T21:55:26.661 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:26.661+0000 7f8185000640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8180104b90 0x7f818019ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:26.661 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:26.661+0000 7f8185000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8180105d80 0x7f818019b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:26.661 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:26.661+0000 7f8185000640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8180106740 0x7f81801a2230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:26.661 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:26.661+0000 7f8185000640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f8180110570 con 0x7f8180106740 2024-04-26T21:55:26.661 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:26.661+0000 7f8185000640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f81801103f0 con 0x7f8180104b90 2024-04-26T21:55:26.662 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:26.661+0000 7f8185000640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f81801106f0 con 0x7f8180105d80 2024-04-26T21:55:26.662 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:26.661+0000 7f817e800640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8180106740 0x7f81801a2230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:26.662 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:26.661+0000 7f817e800640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8180106740 0x7f81801a2230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:44150/0 (socket says 172.21.15.72:44150) 2024-04-26T21:55:26.662 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:26.661+0000 7f817e800640 1 -- 172.21.15.72:0/1184212258 learned_addr learned my addr 172.21.15.72:0/1184212258 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:55:26.662 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:26.661+0000 7f817de00640 1 --2- 172.21.15.72:0/1184212258 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8180104b90 0x7f818019ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:26.662 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:26.661+0000 7f817d400640 1 --2- 172.21.15.72:0/1184212258 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8180105d80 0x7f818019b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:26.663 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:26.662+0000 7f817d400640 1 -- 172.21.15.72:0/1184212258 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8180104b90 msgr2=0x7f818019ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:26.663 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:26.662+0000 7f817d400640 1 --2- 172.21.15.72:0/1184212258 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8180104b90 0x7f818019ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:26.663 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:26.662+0000 7f817d400640 1 -- 172.21.15.72:0/1184212258 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8180106740 msgr2=0x7f81801a2230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:26.663 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:26.662+0000 7f817d400640 1 --2- 172.21.15.72:0/1184212258 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8180106740 0x7f81801a2230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:26.663 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:26.662+0000 7f817d400640 1 -- 172.21.15.72:0/1184212258 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81801a28d0 con 0x7f8180105d80 2024-04-26T21:55:26.663 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:26.662+0000 7f817d400640 1 --2- 172.21.15.72:0/1184212258 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8180105d80 0x7f818019b1d0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f816c00e9f0 tx=0x7f816c00edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:55:26.663 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:26.662+0000 7f8176a00640 1 -- 172.21.15.72:0/1184212258 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f816c00ccc0 con 0x7f8180105d80 2024-04-26T21:55:26.663 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:26.662+0000 7f8185000640 1 -- 172.21.15.72:0/1184212258 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81801a2bc0 con 0x7f8180105d80 2024-04-26T21:55:26.664 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:26.662+0000 7f8176a00640 1 -- 172.21.15.72:0/1184212258 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f816c0044b0 con 0x7f8180105d80 2024-04-26T21:55:26.664 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:26.663+0000 7f8176a00640 1 -- 172.21.15.72:0/1184212258 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f816c01c070 con 0x7f8180105d80 2024-04-26T21:55:26.664 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:26.663+0000 7f8185000640 1 -- 172.21.15.72:0/1184212258 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81801a3110 con 0x7f8180105d80 2024-04-26T21:55:26.664 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:26.664+0000 7f8185000640 1 -- 172.21.15.72:0/1184212258 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f818010bfd0 con 0x7f8180105d80 2024-04-26T21:55:26.666 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:26.666+0000 7f8176a00640 1 -- 172.21.15.72:0/1184212258 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f816c0046c0 con 0x7f8180105d80 2024-04-26T21:55:26.667 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:26.667+0000 7f8176a00640 1 --2- 172.21.15.72:0/1184212258 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f81640a3c40 0x7f81640a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:26.667 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:26.668+0000 7f8176a00640 1 -- 172.21.15.72:0/1184212258 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7f816c0d0930 con 0x7f8180105d80 2024-04-26T21:55:26.668 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:26.668+0000 7f817de00640 1 --2- 172.21.15.72:0/1184212258 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f81640a3c40 0x7f81640a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:26.668 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:26.669+0000 7f817de00640 1 --2- 172.21.15.72:0/1184212258 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f81640a3c40 0x7f81640a6100 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f818019bb60 tx=0x7f8170009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:55:26.670 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:26.670+0000 7f8176a00640 1 -- 172.21.15.72:0/1184212258 <== mon.1 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f816c09d0d0 con 0x7f8180105d80 2024-04-26T21:55:26.956 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:26.956+0000 7f8185000640 1 -- 172.21.15.72:0/1184212258 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f818002dad0 con 0x7f8180105d80 2024-04-26T21:55:26.957 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:26.957+0000 7f8176a00640 1 -- 172.21.15.72:0/1184212258 <== mon.1 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11931 (secure 0 0 0) 0x7f816c09c9d0 con 0x7f8180105d80 2024-04-26T21:55:26.958 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T21:55:26.958 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6835/1320381959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6834","nonce":1320381959},{"type":"v1","addr":"172.21.15.72:6835","nonce":1320381959}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6837/2142200512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6836","nonce":2142200512},{"type":"v1","addr":"172.21.15.72:6837","nonce":2142200512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T21:52:50.567857+0000","modified":"2024-04-26T21:55:22.894383+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34517,"mds_1":34472,"mds_2":34442,"mds_3":34406,"mds_4":24463},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14652":{"gid":14652,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.72:6839/1312898281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6838","nonce":1312898281},{"type":"v1","addr":"172.21.15.72:6839","nonce":1312898281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14664":{"gid":14664,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.72:6841/3257692339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6840","nonce":3257692339},{"type":"v1","addr":"172.21.15.72:6841","nonce":3257692339}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24388":{"gid":24388,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6833/4236668465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":4236668465},{"type":"v1","addr":"172.21.15.164:6833","nonce":4236668465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6835/2391114169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":2391114169},{"type":"v1","addr":"172.21.15.164:6835","nonce":2391114169}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6837/3585829992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":3585829992},{"type":"v1","addr":"172.21.15.164:6837","nonce":3585829992}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.164:6839/1636226820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":1636226820},{"type":"v1","addr":"172.21.15.164:6839","nonce":1636226820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34406":{"gid":34406,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":15,"addr":"172.21.15.76:6833/3869483750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":3869483750},{"type":"v1","addr":"172.21.15.76:6833","nonce":3869483750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34442":{"gid":34442,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.76:6835/3517429435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":3517429435},{"type":"v1","addr":"172.21.15.76:6835","nonce":3517429435}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34472":{"gid":34472,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.76:6837/1150678991","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":1150678991},{"type":"v1","addr":"172.21.15.76:6837","nonce":1150678991}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34517":{"gid":34517,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.76:6839/327145636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6838","nonce":327145636},{"type":"v1","addr":"172.21.15.76:6839","nonce":327145636}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34517,"qdb_cluster":[34472,34517,34442,34406,24463]},"id":1}]} 2024-04-26T21:55:26.958 INFO:teuthology.orchestra.run.smithi072.stderr:dumped fsmap epoch 27 2024-04-26T21:55:26.960 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:26.960+0000 7f8185000640 1 -- 172.21.15.72:0/1184212258 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f81640a3c40 msgr2=0x7f81640a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:26.960 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:26.960+0000 7f8185000640 1 --2- 172.21.15.72:0/1184212258 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f81640a3c40 0x7f81640a6100 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f818019bb60 tx=0x7f8170009260 comp rx=0 tx=0).stop 2024-04-26T21:55:26.960 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:26.960+0000 7f8185000640 1 -- 172.21.15.72:0/1184212258 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8180105d80 msgr2=0x7f818019b1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:26.960 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:26.960+0000 7f8185000640 1 --2- 172.21.15.72:0/1184212258 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8180105d80 0x7f818019b1d0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f816c00e9f0 tx=0x7f816c00edf0 comp rx=0 tx=0).stop 2024-04-26T21:55:26.961 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:26.960+0000 7f8185000640 1 -- 172.21.15.72:0/1184212258 shutdown_connections 2024-04-26T21:55:26.961 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:26.960+0000 7f8185000640 1 --2- 172.21.15.72:0/1184212258 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8180106740 0x7f81801a2230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:26.961 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:26.960+0000 7f8185000640 1 --2- 172.21.15.72:0/1184212258 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f81640a3c40 0x7f81640a6100 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:26.961 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:26.960+0000 7f8185000640 1 --2- 172.21.15.72:0/1184212258 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8180105d80 0x7f818019b1d0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:26.961 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:26.960+0000 7f8185000640 1 --2- 172.21.15.72:0/1184212258 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8180104b90 0x7f818019ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:26.961 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:26.960+0000 7f8185000640 1 -- 172.21.15.72:0/1184212258 >> 172.21.15.72:0/1184212258 conn(0x7f8180100120 msgr2=0x7f81801091a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:55:26.961 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:26.960+0000 7f8185000640 1 -- 172.21.15.72:0/1184212258 shutdown_connections 2024-04-26T21:55:26.961 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:26.960+0000 7f8185000640 1 -- 172.21.15.72:0/1184212258 wait complete. 2024-04-26T21:55:26.978 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-04-26T21:55:26.978 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-04-26T21:55:26.978 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T21:55:27.151 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:27.150+0000 7fdfc1400640 1 -- 172.21.15.72:0/1181960309 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdfbc108f90 msgr2=0x7fdfbc1093b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:27.151 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:27.150+0000 7fdfc1400640 1 --2- 172.21.15.72:0/1181960309 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdfbc108f90 0x7fdfbc1093b0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fdfac009900 tx=0x7fdfac02a3f0 comp rx=0 tx=0).stop 2024-04-26T21:55:27.151 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:27.151+0000 7fdfc1400640 1 -- 172.21.15.72:0/1181960309 shutdown_connections 2024-04-26T21:55:27.151 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:27.151+0000 7fdfc1400640 1 --2- 172.21.15.72:0/1181960309 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdfbc10ab40 0x7fdfbc1113d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:27.151 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:27.151+0000 7fdfc1400640 1 --2- 172.21.15.72:0/1181960309 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdfbc10a180 0x7fdfbc10a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:27.151 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:27.151+0000 7fdfc1400640 1 --2- 172.21.15.72:0/1181960309 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdfbc108f90 0x7fdfbc1093b0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:27.151 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:27.151+0000 7fdfc1400640 1 -- 172.21.15.72:0/1181960309 >> 172.21.15.72:0/1181960309 conn(0x7fdfbc077f40 msgr2=0x7fdfbc07a360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:55:27.152 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:27.151+0000 7fdfc1400640 1 -- 172.21.15.72:0/1181960309 shutdown_connections 2024-04-26T21:55:27.152 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:27.151+0000 7fdfc1400640 1 -- 172.21.15.72:0/1181960309 wait complete. 2024-04-26T21:55:27.154 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:27.152+0000 7fdfc1400640 1 Processor -- start 2024-04-26T21:55:27.154 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:27.152+0000 7fdfc1400640 1 -- start start 2024-04-26T21:55:27.154 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:27.152+0000 7fdfc1400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdfbc108f90 0x7fdfbc19f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:27.154 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:27.152+0000 7fdfc1400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdfbc10a180 0x7fdfbc19f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:27.154 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:27.152+0000 7fdfc1400640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdfbc10ab40 0x7fdfbc1a6720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:27.154 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:27.152+0000 7fdfc1400640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fdfbc114a10 con 0x7fdfbc108f90 2024-04-26T21:55:27.154 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:27.152+0000 7fdfc1400640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fdfbc114890 con 0x7fdfbc10ab40 2024-04-26T21:55:27.154 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:27.152+0000 7fdfc1400640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fdfbc114b90 con 0x7fdfbc10a180 2024-04-26T21:55:27.154 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:27.152+0000 7fdfba800640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdfbc108f90 0x7fdfbc19f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:27.154 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:27.152+0000 7fdfba800640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdfbc108f90 0x7fdfbc19f170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:44172/0 (socket says 172.21.15.72:44172) 2024-04-26T21:55:27.154 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:27.152+0000 7fdfba800640 1 -- 172.21.15.72:0/1272331215 learned_addr learned my addr 172.21.15.72:0/1272331215 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:55:27.155 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:27.153+0000 7fdfbb200640 1 --2- 172.21.15.72:0/1272331215 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdfbc10ab40 0x7fdfbc1a6720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:27.155 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:27.153+0000 7fdfba800640 1 -- 172.21.15.72:0/1272331215 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdfbc10a180 msgr2=0x7fdfbc19f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:27.155 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:27.153+0000 7fdfba800640 1 --2- 172.21.15.72:0/1272331215 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdfbc10a180 0x7fdfbc19f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:27.155 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:27.153+0000 7fdfba800640 1 -- 172.21.15.72:0/1272331215 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdfbc10ab40 msgr2=0x7fdfbc1a6720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:27.155 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:27.153+0000 7fdfba800640 1 --2- 172.21.15.72:0/1272331215 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdfbc10ab40 0x7fdfbc1a6720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:27.155 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:27.153+0000 7fdfba800640 1 -- 172.21.15.72:0/1272331215 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfbc1a6d90 con 0x7fdfbc108f90 2024-04-26T21:55:27.155 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:27.153+0000 7fdfbb200640 1 --2- 172.21.15.72:0/1272331215 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdfbc10ab40 0x7fdfbc1a6720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:55:27.155 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:27.153+0000 7fdfba800640 1 --2- 172.21.15.72:0/1272331215 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdfbc108f90 0x7fdfbc19f170 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fdfac031460 tx=0x7fdfac003d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:55:27.155 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:27.153+0000 7fdfb3400640 1 -- 172.21.15.72:0/1272331215 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfac03f070 con 0x7fdfbc108f90 2024-04-26T21:55:27.155 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:27.153+0000 7fdfb3400640 1 -- 172.21.15.72:0/1272331215 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdfac031da0 con 0x7fdfbc108f90 2024-04-26T21:55:27.156 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:27.153+0000 7fdfb3400640 1 -- 172.21.15.72:0/1272331215 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfac035030 con 0x7fdfbc108f90 2024-04-26T21:55:27.156 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:27.153+0000 7fdfc1400640 1 -- 172.21.15.72:0/1272331215 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfbc1a7020 con 0x7fdfbc108f90 2024-04-26T21:55:27.156 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:27.153+0000 7fdfc1400640 1 -- 172.21.15.72:0/1272331215 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfbc1a7540 con 0x7fdfbc108f90 2024-04-26T21:55:27.156 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:27.155+0000 7fdfc1400640 1 -- 172.21.15.72:0/1272331215 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdfbc1103a0 con 0x7fdfbc108f90 2024-04-26T21:55:27.156 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:27.156+0000 7fdfb3400640 1 -- 172.21.15.72:0/1272331215 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fdfac042070 con 0x7fdfbc108f90 2024-04-26T21:55:27.159 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:27.159+0000 7fdfb3400640 1 --2- 172.21.15.72:0/1272331215 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fdfa00a3cc0 0x7fdfa00a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:27.159 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:27.159+0000 7fdfb3400640 1 -- 172.21.15.72:0/1272331215 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7fdfac0f0fc0 con 0x7fdfbc108f90 2024-04-26T21:55:27.159 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:27.159+0000 7fdfb9e00640 1 --2- 172.21.15.72:0/1272331215 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fdfa00a3cc0 0x7fdfa00a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:27.160 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:27.160+0000 7fdfb9e00640 1 --2- 172.21.15.72:0/1272331215 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fdfa00a3cc0 0x7fdfa00a6180 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc1a0680 tx=0x7fdfa8006c90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:55:27.163 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:27.163+0000 7fdfb3400640 1 -- 172.21.15.72:0/1272331215 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fdfac0bd7c0 con 0x7fdfbc108f90 2024-04-26T21:55:27.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:55:26 smithi164 ceph-mon[31916]: pgmap v449: 97 pgs: 97 active+clean; 636 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 1.2 KiB/s wr, 13 op/s 2024-04-26T21:55:27.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:55:26 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:55:27.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:55:26 smithi164 ceph-mon[31916]: from='client.? 172.21.15.72:0/341034674' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T21:55:27.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:55:26 smithi164 ceph-mon[31916]: from='client.? 172.21.15.72:0/2554165815' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T21:55:27.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:55:26 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:55:27.191 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:55:26 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:55:27.226 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:55:26 smithi076 ceph-mon[31823]: pgmap v449: 97 pgs: 97 active+clean; 636 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 1.2 KiB/s wr, 13 op/s 2024-04-26T21:55:27.226 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:55:26 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:55:27.227 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:55:26 smithi076 ceph-mon[31823]: from='client.? 172.21.15.72:0/341034674' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T21:55:27.227 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:55:26 smithi076 ceph-mon[31823]: from='client.? 172.21.15.72:0/2554165815' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T21:55:27.227 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:55:26 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:55:27.227 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:55:26 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:55:27.287 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:55:26 smithi072 ceph-mon[28138]: pgmap v449: 97 pgs: 97 active+clean; 636 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 1.2 KiB/s wr, 13 op/s 2024-04-26T21:55:27.287 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:55:26 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:55:27.287 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:55:26 smithi072 ceph-mon[28138]: from='client.? 172.21.15.72:0/341034674' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T21:55:27.287 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:55:26 smithi072 ceph-mon[28138]: from='client.? 172.21.15.72:0/2554165815' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T21:55:27.287 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:55:26 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:55:27.288 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:55:26 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:55:27.358 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:27.357+0000 7fdfc1400640 1 -- 172.21.15.72:0/1272331215 --> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fdfbc062d70 con 0x7fdfa00a3cc0 2024-04-26T21:55:27.371 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:27.371+0000 7fdfb3400640 1 -- 172.21.15.72:0/1272331215 <== mgr.14152 v2:172.21.15.72:6800/588150172 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258916 (secure 0 0 0) 0x7fdfbc062d70 con 0x7fdfa00a3cc0 2024-04-26T21:55:27.375 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T21:55:27.376 INFO:teuthology.orchestra.run.smithi072.stderr:dumped all 2024-04-26T21:55:27.378 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:27.376+0000 7fdfc1400640 1 -- 172.21.15.72:0/1272331215 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fdfa00a3cc0 msgr2=0x7fdfa00a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:27.378 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:27.376+0000 7fdfc1400640 1 --2- 172.21.15.72:0/1272331215 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fdfa00a3cc0 0x7fdfa00a6180 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc1a0680 tx=0x7fdfa8006c90 comp rx=0 tx=0).stop 2024-04-26T21:55:27.378 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:27.376+0000 7fdfc1400640 1 -- 172.21.15.72:0/1272331215 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdfbc108f90 msgr2=0x7fdfbc19f170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:27.378 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:27.376+0000 7fdfc1400640 1 --2- 172.21.15.72:0/1272331215 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdfbc108f90 0x7fdfbc19f170 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fdfac031460 tx=0x7fdfac003d50 comp rx=0 tx=0).stop 2024-04-26T21:55:27.378 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:27.376+0000 7fdfc1400640 1 -- 172.21.15.72:0/1272331215 shutdown_connections 2024-04-26T21:55:27.378 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:27.376+0000 7fdfc1400640 1 --2- 172.21.15.72:0/1272331215 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdfbc10ab40 0x7fdfbc1a6720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:27.378 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:27.376+0000 7fdfc1400640 1 --2- 172.21.15.72:0/1272331215 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fdfa00a3cc0 0x7fdfa00a6180 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:27.378 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:27.376+0000 7fdfc1400640 1 --2- 172.21.15.72:0/1272331215 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdfbc10a180 0x7fdfbc19f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:27.378 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:27.376+0000 7fdfc1400640 1 --2- 172.21.15.72:0/1272331215 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdfbc108f90 0x7fdfbc19f170 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:27.378 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:27.376+0000 7fdfc1400640 1 -- 172.21.15.72:0/1272331215 >> 172.21.15.72:0/1272331215 conn(0x7fdfbc077f40 msgr2=0x7fdfbc10d570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:55:27.378 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:27.376+0000 7fdfc1400640 1 -- 172.21.15.72:0/1272331215 shutdown_connections 2024-04-26T21:55:27.378 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:27.376+0000 7fdfc1400640 1 -- 172.21.15.72:0/1272331215 wait complete. 2024-04-26T21:55:27.396 INFO:teuthology.orchestra.run.smithi072.stdout:{"pg_ready":true,"pg_map":{"version":449,"stamp":"2024-04-26T21:55:25.832056+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":650831,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1617,"num_read_kb":923,"num_write":438,"num_write_kb":3077,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":1181358,"num_keys_recovered":7,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":357,"ondisk_log_size":357,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":396484,"kb_used_data":14052,"kb_used_omap":18,"kb_used_meta":382317,"kb_avail":1124545340,"statfs":{"total":1151940427776,"available":1151534428160,"internally_reserved":0,"allocated":14389248,"data_stored":7752333,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":391492922},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2465,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":144,"num_read_kb":79,"num_write":18,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004264"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.921489+0000","last_change":"2024-04-26T21:52:50.787765+0000","last_active":"2024-04-26T21:54:11.921489+0000","last_peered":"2024-04-26T21:54:11.921489+0000","last_clean":"2024-04-26T21:54:11.921489+0000","last_became_active":"2024-04-26T21:52:50.786969+0000","last_became_peered":"2024-04-26T21:52:50.786969+0000","last_unstale":"2024-04-26T21:54:11.921489+0000","last_undegraded":"2024-04-26T21:54:11.921489+0000","last_fullsized":"2024-04-26T21:54:11.921489+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:31:50.659210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:12.034292+0000","last_change":"2024-04-26T21:52:50.779735+0000","last_active":"2024-04-26T21:54:12.034292+0000","last_peered":"2024-04-26T21:54:12.034292+0000","last_clean":"2024-04-26T21:54:12.034292+0000","last_became_active":"2024-04-26T21:52:50.779358+0000","last_became_peered":"2024-04-26T21:52:50.779358+0000","last_unstale":"2024-04-26T21:54:12.034292+0000","last_undegraded":"2024-04-26T21:54:12.034292+0000","last_fullsized":"2024-04-26T21:54:12.034292+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:58:45.631365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:12.033978+0000","last_change":"2024-04-26T21:52:50.788220+0000","last_active":"2024-04-26T21:54:12.033978+0000","last_peered":"2024-04-26T21:54:12.033978+0000","last_clean":"2024-04-26T21:54:12.033978+0000","last_became_active":"2024-04-26T21:52:50.787282+0000","last_became_peered":"2024-04-26T21:52:50.787282+0000","last_unstale":"2024-04-26T21:54:12.033978+0000","last_undegraded":"2024-04-26T21:54:12.033978+0000","last_fullsized":"2024-04-26T21:54:12.033978+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:12:28.577893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.934516+0000","last_change":"2024-04-26T21:52:50.795367+0000","last_active":"2024-04-26T21:54:11.934516+0000","last_peered":"2024-04-26T21:54:11.934516+0000","last_clean":"2024-04-26T21:54:11.934516+0000","last_became_active":"2024-04-26T21:52:50.793602+0000","last_became_peered":"2024-04-26T21:52:50.793602+0000","last_unstale":"2024-04-26T21:54:11.934516+0000","last_undegraded":"2024-04-26T21:54:11.934516+0000","last_fullsized":"2024-04-26T21:54:11.934516+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:37:23.356792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.912197+0000","last_change":"2024-04-26T21:52:50.787333+0000","last_active":"2024-04-26T21:54:11.912197+0000","last_peered":"2024-04-26T21:54:11.912197+0000","last_clean":"2024-04-26T21:54:11.912197+0000","last_became_active":"2024-04-26T21:52:50.785319+0000","last_became_peered":"2024-04-26T21:52:50.785319+0000","last_unstale":"2024-04-26T21:54:11.912197+0000","last_undegraded":"2024-04-26T21:54:11.912197+0000","last_fullsized":"2024-04-26T21:54:11.912197+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:06:29.154825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:12.397124+0000","last_change":"2024-04-26T21:52:50.780829+0000","last_active":"2024-04-26T21:54:12.397124+0000","last_peered":"2024-04-26T21:54:12.397124+0000","last_clean":"2024-04-26T21:54:12.397124+0000","last_became_active":"2024-04-26T21:52:50.780439+0000","last_became_peered":"2024-04-26T21:52:50.780439+0000","last_unstale":"2024-04-26T21:54:12.397124+0000","last_undegraded":"2024-04-26T21:54:12.397124+0000","last_fullsized":"2024-04-26T21:54:12.397124+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:20:14.533853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.922216+0000","last_change":"2024-04-26T21:52:50.787679+0000","last_active":"2024-04-26T21:54:11.922216+0000","last_peered":"2024-04-26T21:54:11.922216+0000","last_clean":"2024-04-26T21:54:11.922216+0000","last_became_active":"2024-04-26T21:52:50.787277+0000","last_became_peered":"2024-04-26T21:52:50.787277+0000","last_unstale":"2024-04-26T21:54:11.922216+0000","last_undegraded":"2024-04-26T21:54:11.922216+0000","last_fullsized":"2024-04-26T21:54:11.922216+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:29:02.131503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":15,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.905091+0000","last_change":"2024-04-26T21:53:10.927384+0000","last_active":"2024-04-26T21:54:11.905091+0000","last_peered":"2024-04-26T21:54:11.905091+0000","last_clean":"2024-04-26T21:54:11.905091+0000","last_became_active":"2024-04-26T21:53:10.926996+0000","last_became_peered":"2024-04-26T21:53:10.926996+0000","last_unstale":"2024-04-26T21:54:11.905091+0000","last_undegraded":"2024-04-26T21:54:11.905091+0000","last_fullsized":"2024-04-26T21:54:11.905091+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:25:10.174532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.909555+0000","last_change":"2024-04-26T21:52:50.779701+0000","last_active":"2024-04-26T21:54:11.909555+0000","last_peered":"2024-04-26T21:54:11.909555+0000","last_clean":"2024-04-26T21:54:11.909555+0000","last_became_active":"2024-04-26T21:52:50.779331+0000","last_became_peered":"2024-04-26T21:52:50.779331+0000","last_unstale":"2024-04-26T21:54:11.909555+0000","last_undegraded":"2024-04-26T21:54:11.909555+0000","last_fullsized":"2024-04-26T21:54:11.909555+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:38:15.569806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:12.397905+0000","last_change":"2024-04-26T21:52:50.790081+0000","last_active":"2024-04-26T21:54:12.397905+0000","last_peered":"2024-04-26T21:54:12.397905+0000","last_clean":"2024-04-26T21:54:12.397905+0000","last_became_active":"2024-04-26T21:52:50.786575+0000","last_became_peered":"2024-04-26T21:52:50.786575+0000","last_unstale":"2024-04-26T21:54:12.397905+0000","last_undegraded":"2024-04-26T21:54:12.397905+0000","last_fullsized":"2024-04-26T21:54:12.397905+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:56:47.467128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.923025+0000","last_change":"2024-04-26T21:52:50.791983+0000","last_active":"2024-04-26T21:54:11.923025+0000","last_peered":"2024-04-26T21:54:11.923025+0000","last_clean":"2024-04-26T21:54:11.923025+0000","last_became_active":"2024-04-26T21:52:50.788312+0000","last_became_peered":"2024-04-26T21:52:50.788312+0000","last_unstale":"2024-04-26T21:54:11.923025+0000","last_undegraded":"2024-04-26T21:54:11.923025+0000","last_fullsized":"2024-04-26T21:54:11.923025+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:47:47.410267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.934246+0000","last_change":"2024-04-26T21:52:50.794237+0000","last_active":"2024-04-26T21:54:11.934246+0000","last_peered":"2024-04-26T21:54:11.934246+0000","last_clean":"2024-04-26T21:54:11.934246+0000","last_became_active":"2024-04-26T21:52:50.792260+0000","last_became_peered":"2024-04-26T21:52:50.792260+0000","last_unstale":"2024-04-26T21:54:11.934246+0000","last_undegraded":"2024-04-26T21:54:11.934246+0000","last_fullsized":"2024-04-26T21:54:11.934246+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:36:05.467667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:12.396084+0000","last_change":"2024-04-26T21:52:50.777687+0000","last_active":"2024-04-26T21:54:12.396084+0000","last_peered":"2024-04-26T21:54:12.396084+0000","last_clean":"2024-04-26T21:54:12.396084+0000","last_became_active":"2024-04-26T21:52:50.777200+0000","last_became_peered":"2024-04-26T21:52:50.777200+0000","last_unstale":"2024-04-26T21:54:12.396084+0000","last_undegraded":"2024-04-26T21:54:12.396084+0000","last_fullsized":"2024-04-26T21:54:12.396084+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:40:45.167144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:12.033391+0000","last_change":"2024-04-26T21:52:50.787008+0000","last_active":"2024-04-26T21:54:12.033391+0000","last_peered":"2024-04-26T21:54:12.033391+0000","last_clean":"2024-04-26T21:54:12.033391+0000","last_became_active":"2024-04-26T21:52:50.785656+0000","last_became_peered":"2024-04-26T21:52:50.785656+0000","last_unstale":"2024-04-26T21:54:12.033391+0000","last_undegraded":"2024-04-26T21:54:12.033391+0000","last_fullsized":"2024-04-26T21:54:12.033391+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:20:12.441281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:12.033386+0000","last_change":"2024-04-26T21:52:50.796537+0000","last_active":"2024-04-26T21:54:12.033386+0000","last_peered":"2024-04-26T21:54:12.033386+0000","last_clean":"2024-04-26T21:54:12.033386+0000","last_became_active":"2024-04-26T21:52:50.796229+0000","last_became_peered":"2024-04-26T21:52:50.796229+0000","last_unstale":"2024-04-26T21:54:12.033386+0000","last_undegraded":"2024-04-26T21:54:12.033386+0000","last_fullsized":"2024-04-26T21:54:12.033386+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:38:46.404444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.908416+0000","last_change":"2024-04-26T21:52:50.789211+0000","last_active":"2024-04-26T21:54:11.908416+0000","last_peered":"2024-04-26T21:54:11.908416+0000","last_clean":"2024-04-26T21:54:11.908416+0000","last_became_active":"2024-04-26T21:52:50.788889+0000","last_became_peered":"2024-04-26T21:52:50.788889+0000","last_unstale":"2024-04-26T21:54:11.908416+0000","last_undegraded":"2024-04-26T21:54:11.908416+0000","last_fullsized":"2024-04-26T21:54:11.908416+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:25:48.497840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.921583+0000","last_change":"2024-04-26T21:52:50.788035+0000","last_active":"2024-04-26T21:54:11.921583+0000","last_peered":"2024-04-26T21:54:11.921583+0000","last_clean":"2024-04-26T21:54:11.921583+0000","last_became_active":"2024-04-26T21:52:50.787453+0000","last_became_peered":"2024-04-26T21:52:50.787453+0000","last_unstale":"2024-04-26T21:54:11.921583+0000","last_undegraded":"2024-04-26T21:54:11.921583+0000","last_fullsized":"2024-04-26T21:54:11.921583+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:40:29.718537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.905571+0000","last_change":"2024-04-26T21:52:50.778228+0000","last_active":"2024-04-26T21:54:11.905571+0000","last_peered":"2024-04-26T21:54:11.905571+0000","last_clean":"2024-04-26T21:54:11.905571+0000","last_became_active":"2024-04-26T21:52:50.777934+0000","last_became_peered":"2024-04-26T21:52:50.777934+0000","last_unstale":"2024-04-26T21:54:11.905571+0000","last_undegraded":"2024-04-26T21:54:11.905571+0000","last_fullsized":"2024-04-26T21:54:11.905571+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:33:34.598325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.909217+0000","last_change":"2024-04-26T21:52:50.790795+0000","last_active":"2024-04-26T21:54:11.909217+0000","last_peered":"2024-04-26T21:54:11.909217+0000","last_clean":"2024-04-26T21:54:11.909217+0000","last_became_active":"2024-04-26T21:52:50.788254+0000","last_became_peered":"2024-04-26T21:52:50.788254+0000","last_unstale":"2024-04-26T21:54:11.909217+0000","last_undegraded":"2024-04-26T21:54:11.909217+0000","last_fullsized":"2024-04-26T21:54:11.909217+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:20:28.235137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"91'14","reported_seq":185,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:55:22.927455+0000","last_change":"2024-04-26T21:52:50.105513+0000","last_active":"2024-04-26T21:55:22.927455+0000","last_peered":"2024-04-26T21:55:22.927455+0000","last_clean":"2024-04-26T21:55:22.927455+0000","last_became_active":"2024-04-26T21:52:50.105131+0000","last_became_peered":"2024-04-26T21:52:50.105131+0000","last_unstale":"2024-04-26T21:55:22.927455+0000","last_undegraded":"2024-04-26T21:55:22.927455+0000","last_fullsized":"2024-04-26T21:55:22.927455+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:36:58.085803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":147,"num_read_kb":147,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.908290+0000","last_change":"2024-04-26T21:52:50.795025+0000","last_active":"2024-04-26T21:54:11.908290+0000","last_peered":"2024-04-26T21:54:11.908290+0000","last_clean":"2024-04-26T21:54:11.908290+0000","last_became_active":"2024-04-26T21:52:50.793608+0000","last_became_peered":"2024-04-26T21:52:50.793608+0000","last_unstale":"2024-04-26T21:54:11.908290+0000","last_undegraded":"2024-04-26T21:54:11.908290+0000","last_fullsized":"2024-04-26T21:54:11.908290+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:49:47.764848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"87'2","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.908343+0000","last_change":"2024-04-26T21:52:50.109197+0000","last_active":"2024-04-26T21:54:11.908343+0000","last_peered":"2024-04-26T21:54:11.908343+0000","last_clean":"2024-04-26T21:54:11.908343+0000","last_became_active":"2024-04-26T21:52:50.108835+0000","last_became_peered":"2024-04-26T21:52:50.108835+0000","last_unstale":"2024-04-26T21:54:11.908343+0000","last_undegraded":"2024-04-26T21:54:11.908343+0000","last_fullsized":"2024-04-26T21:54:11.908343+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:45:09.777747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:12.034419+0000","last_change":"2024-04-26T21:52:50.783094+0000","last_active":"2024-04-26T21:54:12.034419+0000","last_peered":"2024-04-26T21:54:12.034419+0000","last_clean":"2024-04-26T21:54:12.034419+0000","last_became_active":"2024-04-26T21:52:50.782468+0000","last_became_peered":"2024-04-26T21:52:50.782468+0000","last_unstale":"2024-04-26T21:54:12.034419+0000","last_undegraded":"2024-04-26T21:54:12.034419+0000","last_fullsized":"2024-04-26T21:54:12.034419+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:53:12.977054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"87'12","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.913469+0000","last_change":"2024-04-26T21:52:50.104824+0000","last_active":"2024-04-26T21:54:11.913469+0000","last_peered":"2024-04-26T21:54:11.913469+0000","last_clean":"2024-04-26T21:54:11.913469+0000","last_became_active":"2024-04-26T21:52:50.104514+0000","last_became_peered":"2024-04-26T21:52:50.104514+0000","last_unstale":"2024-04-26T21:54:11.913469+0000","last_undegraded":"2024-04-26T21:54:11.913469+0000","last_fullsized":"2024-04-26T21:54:11.913469+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:45:31.541940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.909219+0000","last_change":"2024-04-26T21:52:50.779695+0000","last_active":"2024-04-26T21:54:11.909219+0000","last_peered":"2024-04-26T21:54:11.909219+0000","last_clean":"2024-04-26T21:54:11.909219+0000","last_became_active":"2024-04-26T21:52:50.779234+0000","last_became_peered":"2024-04-26T21:52:50.779234+0000","last_unstale":"2024-04-26T21:54:11.909219+0000","last_undegraded":"2024-04-26T21:54:11.909219+0000","last_fullsized":"2024-04-26T21:54:11.909219+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:33:07.316543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"87'5","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.912981+0000","last_change":"2024-04-26T21:52:50.107403+0000","last_active":"2024-04-26T21:54:11.912981+0000","last_peered":"2024-04-26T21:54:11.912981+0000","last_clean":"2024-04-26T21:54:11.912981+0000","last_became_active":"2024-04-26T21:52:50.107126+0000","last_became_peered":"2024-04-26T21:52:50.107126+0000","last_unstale":"2024-04-26T21:54:11.912981+0000","last_undegraded":"2024-04-26T21:54:11.912981+0000","last_fullsized":"2024-04-26T21:54:11.912981+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:52:34.985448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.916932+0000","last_change":"2024-04-26T21:52:50.783674+0000","last_active":"2024-04-26T21:54:11.916932+0000","last_peered":"2024-04-26T21:54:11.916932+0000","last_clean":"2024-04-26T21:54:11.916932+0000","last_became_active":"2024-04-26T21:52:50.782283+0000","last_became_peered":"2024-04-26T21:52:50.782283+0000","last_unstale":"2024-04-26T21:54:11.916932+0000","last_undegraded":"2024-04-26T21:54:11.916932+0000","last_fullsized":"2024-04-26T21:54:11.916932+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:41:03.144601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"91'6","reported_seq":189,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:55:22.154100+0000","last_change":"2024-04-26T21:54:11.949461+0000","last_active":"2024-04-26T21:55:22.154100+0000","last_peered":"2024-04-26T21:55:22.154100+0000","last_clean":"2024-04-26T21:55:22.154100+0000","last_became_active":"2024-04-26T21:54:11.936983+0000","last_became_peered":"2024-04-26T21:54:11.936983+0000","last_unstale":"2024-04-26T21:55:22.154100+0000","last_undegraded":"2024-04-26T21:55:22.154100+0000","last_fullsized":"2024-04-26T21:55:22.154100+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:36:40.757999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1115,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":146,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":622,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.912376+0000","last_change":"2024-04-26T21:53:11.575919+0000","last_active":"2024-04-26T21:54:11.912376+0000","last_peered":"2024-04-26T21:54:11.912376+0000","last_clean":"2024-04-26T21:54:11.912376+0000","last_became_active":"2024-04-26T21:53:11.574898+0000","last_became_peered":"2024-04-26T21:53:11.574898+0000","last_unstale":"2024-04-26T21:54:11.912376+0000","last_undegraded":"2024-04-26T21:54:11.912376+0000","last_fullsized":"2024-04-26T21:54:11.912376+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:44:16.323430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,7],"acting":[8,9,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"87'7","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:12.396927+0000","last_change":"2024-04-26T21:53:10.938377+0000","last_active":"2024-04-26T21:54:12.396927+0000","last_peered":"2024-04-26T21:54:12.396927+0000","last_clean":"2024-04-26T21:54:12.396927+0000","last_became_active":"2024-04-26T21:53:10.921998+0000","last_became_peered":"2024-04-26T21:53:10.921998+0000","last_unstale":"2024-04-26T21:54:12.396927+0000","last_undegraded":"2024-04-26T21:54:12.396927+0000","last_fullsized":"2024-04-26T21:54:12.396927+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:51:20.389344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,6],"acting":[10,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.919025+0000","last_change":"2024-04-26T21:53:10.931444+0000","last_active":"2024-04-26T21:54:11.919025+0000","last_peered":"2024-04-26T21:54:11.919025+0000","last_clean":"2024-04-26T21:54:11.919025+0000","last_became_active":"2024-04-26T21:53:10.930761+0000","last_became_peered":"2024-04-26T21:53:10.930761+0000","last_unstale":"2024-04-26T21:54:11.919025+0000","last_undegraded":"2024-04-26T21:54:11.919025+0000","last_fullsized":"2024-04-26T21:54:11.919025+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:56:17.803811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,3],"acting":[5,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"87'3","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:12.395254+0000","last_change":"2024-04-26T21:52:50.107564+0000","last_active":"2024-04-26T21:54:12.395254+0000","last_peered":"2024-04-26T21:54:12.395254+0000","last_clean":"2024-04-26T21:54:12.395254+0000","last_became_active":"2024-04-26T21:52:50.106653+0000","last_became_peered":"2024-04-26T21:52:50.106653+0000","last_unstale":"2024-04-26T21:54:12.395254+0000","last_undegraded":"2024-04-26T21:54:12.395254+0000","last_fullsized":"2024-04-26T21:54:12.395254+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:05:36.594729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.907525+0000","last_change":"2024-04-26T21:52:50.780279+0000","last_active":"2024-04-26T21:54:11.907525+0000","last_peered":"2024-04-26T21:54:11.907525+0000","last_clean":"2024-04-26T21:54:11.907525+0000","last_became_active":"2024-04-26T21:52:50.779950+0000","last_became_peered":"2024-04-26T21:52:50.779950+0000","last_unstale":"2024-04-26T21:54:11.907525+0000","last_undegraded":"2024-04-26T21:54:11.907525+0000","last_fullsized":"2024-04-26T21:54:11.907525+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:11:36.282319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"87'3","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.917578+0000","last_change":"2024-04-26T21:52:50.106269+0000","last_active":"2024-04-26T21:54:11.917578+0000","last_peered":"2024-04-26T21:54:11.917578+0000","last_clean":"2024-04-26T21:54:11.917578+0000","last_became_active":"2024-04-26T21:52:50.105948+0000","last_became_peered":"2024-04-26T21:52:50.105948+0000","last_unstale":"2024-04-26T21:54:11.917578+0000","last_undegraded":"2024-04-26T21:54:11.917578+0000","last_fullsized":"2024-04-26T21:54:11.917578+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:39:31.024362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.905324+0000","last_change":"2024-04-26T21:52:50.778938+0000","last_active":"2024-04-26T21:54:11.905324+0000","last_peered":"2024-04-26T21:54:11.905324+0000","last_clean":"2024-04-26T21:54:11.905324+0000","last_became_active":"2024-04-26T21:52:50.778590+0000","last_became_peered":"2024-04-26T21:52:50.778590+0000","last_unstale":"2024-04-26T21:54:11.905324+0000","last_undegraded":"2024-04-26T21:54:11.905324+0000","last_fullsized":"2024-04-26T21:54:11.905324+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:03:10.325793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"91'6","reported_seq":320,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:55:22.948842+0000","last_change":"2024-04-26T21:52:50.107243+0000","last_active":"2024-04-26T21:55:22.948842+0000","last_peered":"2024-04-26T21:55:22.948842+0000","last_clean":"2024-04-26T21:55:22.948842+0000","last_became_active":"2024-04-26T21:52:50.106729+0000","last_became_peered":"2024-04-26T21:52:50.106729+0000","last_unstale":"2024-04-26T21:55:22.948842+0000","last_undegraded":"2024-04-26T21:55:22.948842+0000","last_fullsized":"2024-04-26T21:55:22.948842+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:28:11.197997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":290,"num_read_kb":290,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.909345+0000","last_change":"2024-04-26T21:52:50.776906+0000","last_active":"2024-04-26T21:54:11.909345+0000","last_peered":"2024-04-26T21:54:11.909345+0000","last_clean":"2024-04-26T21:54:11.909345+0000","last_became_active":"2024-04-26T21:52:50.776567+0000","last_became_peered":"2024-04-26T21:52:50.776567+0000","last_unstale":"2024-04-26T21:54:11.909345+0000","last_undegraded":"2024-04-26T21:54:11.909345+0000","last_fullsized":"2024-04-26T21:54:11.909345+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:51:39.512572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"87'4","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.923341+0000","last_change":"2024-04-26T21:52:50.109792+0000","last_active":"2024-04-26T21:54:11.923341+0000","last_peered":"2024-04-26T21:54:11.923341+0000","last_clean":"2024-04-26T21:54:11.923341+0000","last_became_active":"2024-04-26T21:52:50.106662+0000","last_became_peered":"2024-04-26T21:52:50.106662+0000","last_unstale":"2024-04-26T21:54:11.923341+0000","last_undegraded":"2024-04-26T21:54:11.923341+0000","last_fullsized":"2024-04-26T21:54:11.923341+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:41:27.390780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:12.034410+0000","last_change":"2024-04-26T21:52:50.782798+0000","last_active":"2024-04-26T21:54:12.034410+0000","last_peered":"2024-04-26T21:54:12.034410+0000","last_clean":"2024-04-26T21:54:12.034410+0000","last_became_active":"2024-04-26T21:52:50.781975+0000","last_became_peered":"2024-04-26T21:52:50.781975+0000","last_unstale":"2024-04-26T21:54:12.034410+0000","last_undegraded":"2024-04-26T21:54:12.034410+0000","last_fullsized":"2024-04-26T21:54:12.034410+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:13:07.261095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"87'5","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.917206+0000","last_change":"2024-04-26T21:52:50.099296+0000","last_active":"2024-04-26T21:54:11.917206+0000","last_peered":"2024-04-26T21:54:11.917206+0000","last_clean":"2024-04-26T21:54:11.917206+0000","last_became_active":"2024-04-26T21:52:50.098940+0000","last_became_peered":"2024-04-26T21:52:50.098940+0000","last_unstale":"2024-04-26T21:54:11.917206+0000","last_undegraded":"2024-04-26T21:54:11.917206+0000","last_fullsized":"2024-04-26T21:54:11.917206+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:49:15.765474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.909628+0000","last_change":"2024-04-26T21:52:50.783341+0000","last_active":"2024-04-26T21:54:11.909628+0000","last_peered":"2024-04-26T21:54:11.909628+0000","last_clean":"2024-04-26T21:54:11.909628+0000","last_became_active":"2024-04-26T21:52:50.783025+0000","last_became_peered":"2024-04-26T21:52:50.783025+0000","last_unstale":"2024-04-26T21:54:11.909628+0000","last_undegraded":"2024-04-26T21:54:11.909628+0000","last_fullsized":"2024-04-26T21:54:11.909628+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:54:06.070550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"91'31","reported_seq":352,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:55:23.155989+0000","last_change":"2024-04-26T21:52:50.106617+0000","last_active":"2024-04-26T21:55:23.155989+0000","last_peered":"2024-04-26T21:55:23.155989+0000","last_clean":"2024-04-26T21:55:23.155989+0000","last_became_active":"2024-04-26T21:52:50.106161+0000","last_became_peered":"2024-04-26T21:52:50.106161+0000","last_unstale":"2024-04-26T21:55:23.155989+0000","last_undegraded":"2024-04-26T21:55:23.155989+0000","last_fullsized":"2024-04-26T21:55:23.155989+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:52:07.977749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":51444,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":297,"num_read_kb":199,"num_write":33,"num_write_kb":61,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.912969+0000","last_change":"2024-04-26T21:52:50.794567+0000","last_active":"2024-04-26T21:54:11.912969+0000","last_peered":"2024-04-26T21:54:11.912969+0000","last_clean":"2024-04-26T21:54:11.912969+0000","last_became_active":"2024-04-26T21:52:50.794245+0000","last_became_peered":"2024-04-26T21:52:50.794245+0000","last_unstale":"2024-04-26T21:54:11.912969+0000","last_undegraded":"2024-04-26T21:54:11.912969+0000","last_fullsized":"2024-04-26T21:54:11.912969+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:58:11.501104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"87'9","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.910302+0000","last_change":"2024-04-26T21:52:50.102156+0000","last_active":"2024-04-26T21:54:11.910302+0000","last_peered":"2024-04-26T21:54:11.910302+0000","last_clean":"2024-04-26T21:54:11.910302+0000","last_became_active":"2024-04-26T21:52:50.101721+0000","last_became_peered":"2024-04-26T21:52:50.101721+0000","last_unstale":"2024-04-26T21:54:11.910302+0000","last_undegraded":"2024-04-26T21:54:11.910302+0000","last_fullsized":"2024-04-26T21:54:11.910302+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:48:42.081874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:12.395808+0000","last_change":"2024-04-26T21:52:50.789199+0000","last_active":"2024-04-26T21:54:12.395808+0000","last_peered":"2024-04-26T21:54:12.395808+0000","last_clean":"2024-04-26T21:54:12.395808+0000","last_became_active":"2024-04-26T21:52:50.788868+0000","last_became_peered":"2024-04-26T21:52:50.788868+0000","last_unstale":"2024-04-26T21:54:12.395808+0000","last_undegraded":"2024-04-26T21:54:12.395808+0000","last_fullsized":"2024-04-26T21:54:12.395808+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:29:55.755564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"87'5","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.918826+0000","last_change":"2024-04-26T21:52:50.104920+0000","last_active":"2024-04-26T21:54:11.918826+0000","last_peered":"2024-04-26T21:54:11.918826+0000","last_clean":"2024-04-26T21:54:11.918826+0000","last_became_active":"2024-04-26T21:52:50.104579+0000","last_became_peered":"2024-04-26T21:52:50.104579+0000","last_unstale":"2024-04-26T21:54:11.918826+0000","last_undegraded":"2024-04-26T21:54:11.918826+0000","last_fullsized":"2024-04-26T21:54:11.918826+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:09:35.760918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.923423+0000","last_change":"2024-04-26T21:52:50.787016+0000","last_active":"2024-04-26T21:54:11.923423+0000","last_peered":"2024-04-26T21:54:11.923423+0000","last_clean":"2024-04-26T21:54:11.923423+0000","last_became_active":"2024-04-26T21:52:50.778439+0000","last_became_peered":"2024-04-26T21:52:50.778439+0000","last_unstale":"2024-04-26T21:54:11.923423+0000","last_undegraded":"2024-04-26T21:54:11.923423+0000","last_fullsized":"2024-04-26T21:54:11.923423+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:13:35.735480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"91'6","reported_seq":179,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:55:22.964012+0000","last_change":"2024-04-26T21:52:50.110175+0000","last_active":"2024-04-26T21:55:22.964012+0000","last_peered":"2024-04-26T21:55:22.964012+0000","last_clean":"2024-04-26T21:55:22.964012+0000","last_became_active":"2024-04-26T21:52:50.109779+0000","last_became_peered":"2024-04-26T21:52:50.109779+0000","last_unstale":"2024-04-26T21:55:22.964012+0000","last_undegraded":"2024-04-26T21:55:22.964012+0000","last_fullsized":"2024-04-26T21:55:22.964012+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:06:46.756327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1081,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":150,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.922565+0000","last_change":"2024-04-26T21:52:50.784199+0000","last_active":"2024-04-26T21:54:11.922565+0000","last_peered":"2024-04-26T21:54:11.922565+0000","last_clean":"2024-04-26T21:54:11.922565+0000","last_became_active":"2024-04-26T21:52:50.783820+0000","last_became_peered":"2024-04-26T21:52:50.783820+0000","last_unstale":"2024-04-26T21:54:11.922565+0000","last_undegraded":"2024-04-26T21:54:11.922565+0000","last_fullsized":"2024-04-26T21:54:11.922565+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:45:24.776722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"87'17","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.912743+0000","last_change":"2024-04-26T21:52:50.095685+0000","last_active":"2024-04-26T21:54:11.912743+0000","last_peered":"2024-04-26T21:54:11.912743+0000","last_clean":"2024-04-26T21:54:11.912743+0000","last_became_active":"2024-04-26T21:52:50.095298+0000","last_became_peered":"2024-04-26T21:52:50.095298+0000","last_unstale":"2024-04-26T21:54:11.912743+0000","last_undegraded":"2024-04-26T21:54:11.912743+0000","last_fullsized":"2024-04-26T21:54:11.912743+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:47:26.061531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.917867+0000","last_change":"2024-04-26T21:52:50.786607+0000","last_active":"2024-04-26T21:54:11.917867+0000","last_peered":"2024-04-26T21:54:11.917867+0000","last_clean":"2024-04-26T21:54:11.917867+0000","last_became_active":"2024-04-26T21:52:50.786261+0000","last_became_peered":"2024-04-26T21:52:50.786261+0000","last_unstale":"2024-04-26T21:54:11.917867+0000","last_undegraded":"2024-04-26T21:54:11.917867+0000","last_fullsized":"2024-04-26T21:54:11.917867+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:08:45.902260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"91'3","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:55:08.888181+0000","last_change":"2024-04-26T21:52:50.095782+0000","last_active":"2024-04-26T21:55:08.888181+0000","last_peered":"2024-04-26T21:55:08.888181+0000","last_clean":"2024-04-26T21:55:08.888181+0000","last_became_active":"2024-04-26T21:52:50.093983+0000","last_became_peered":"2024-04-26T21:52:50.093983+0000","last_unstale":"2024-04-26T21:55:08.888181+0000","last_undegraded":"2024-04-26T21:55:08.888181+0000","last_fullsized":"2024-04-26T21:55:08.888181+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:18:40.103099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.921290+0000","last_change":"2024-04-26T21:52:50.791969+0000","last_active":"2024-04-26T21:54:11.921290+0000","last_peered":"2024-04-26T21:54:11.921290+0000","last_clean":"2024-04-26T21:54:11.921290+0000","last_became_active":"2024-04-26T21:52:50.790441+0000","last_became_peered":"2024-04-26T21:52:50.790441+0000","last_unstale":"2024-04-26T21:54:11.921290+0000","last_undegraded":"2024-04-26T21:54:11.921290+0000","last_fullsized":"2024-04-26T21:54:11.921290+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:11:04.964439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"87'1","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.905428+0000","last_change":"2024-04-26T21:52:50.101807+0000","last_active":"2024-04-26T21:54:11.905428+0000","last_peered":"2024-04-26T21:54:11.905428+0000","last_clean":"2024-04-26T21:54:11.905428+0000","last_became_active":"2024-04-26T21:52:50.101329+0000","last_became_peered":"2024-04-26T21:52:50.101329+0000","last_unstale":"2024-04-26T21:54:11.905428+0000","last_undegraded":"2024-04-26T21:54:11.905428+0000","last_fullsized":"2024-04-26T21:54:11.905428+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:54:47.788172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"87'1","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:12.397681+0000","last_change":"2024-04-26T21:52:50.780872+0000","last_active":"2024-04-26T21:54:12.397681+0000","last_peered":"2024-04-26T21:54:12.397681+0000","last_clean":"2024-04-26T21:54:12.397681+0000","last_became_active":"2024-04-26T21:52:50.778757+0000","last_became_peered":"2024-04-26T21:52:50.778757+0000","last_unstale":"2024-04-26T21:54:12.397681+0000","last_undegraded":"2024-04-26T21:54:12.397681+0000","last_fullsized":"2024-04-26T21:54:12.397681+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:07:21.595091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"91'6","reported_seq":176,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:55:20.923377+0000","last_change":"2024-04-26T21:52:50.106886+0000","last_active":"2024-04-26T21:55:20.923377+0000","last_peered":"2024-04-26T21:55:20.923377+0000","last_clean":"2024-04-26T21:55:20.923377+0000","last_became_active":"2024-04-26T21:52:50.106150+0000","last_became_peered":"2024-04-26T21:52:50.106150+0000","last_unstale":"2024-04-26T21:55:20.923377+0000","last_undegraded":"2024-04-26T21:55:20.923377+0000","last_fullsized":"2024-04-26T21:55:20.923377+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:20:54.920416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1081,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":146,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:12.397270+0000","last_change":"2024-04-26T21:53:11.069186+0000","last_active":"2024-04-26T21:54:12.397270+0000","last_peered":"2024-04-26T21:54:12.397270+0000","last_clean":"2024-04-26T21:54:12.397270+0000","last_became_active":"2024-04-26T21:53:11.068418+0000","last_became_peered":"2024-04-26T21:53:11.068418+0000","last_unstale":"2024-04-26T21:54:12.397270+0000","last_undegraded":"2024-04-26T21:54:12.397270+0000","last_fullsized":"2024-04-26T21:54:12.397270+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:57:59.663173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.917799+0000","last_change":"2024-04-26T21:52:50.785257+0000","last_active":"2024-04-26T21:54:11.917799+0000","last_peered":"2024-04-26T21:54:11.917799+0000","last_clean":"2024-04-26T21:54:11.917799+0000","last_became_active":"2024-04-26T21:52:50.784906+0000","last_became_peered":"2024-04-26T21:52:50.784906+0000","last_unstale":"2024-04-26T21:54:11.917799+0000","last_undegraded":"2024-04-26T21:54:11.917799+0000","last_fullsized":"2024-04-26T21:54:11.917799+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:04:50.921663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"91'6","reported_seq":176,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:55:21.959779+0000","last_change":"2024-04-26T21:52:50.102425+0000","last_active":"2024-04-26T21:55:21.959779+0000","last_peered":"2024-04-26T21:55:21.959779+0000","last_clean":"2024-04-26T21:55:21.959779+0000","last_became_active":"2024-04-26T21:52:50.101703+0000","last_became_peered":"2024-04-26T21:52:50.101703+0000","last_unstale":"2024-04-26T21:55:21.959779+0000","last_undegraded":"2024-04-26T21:55:21.959779+0000","last_fullsized":"2024-04-26T21:55:21.959779+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:13:39.910519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":146,"num_read_kb":146,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.907737+0000","last_change":"2024-04-26T21:52:50.775036+0000","last_active":"2024-04-26T21:54:11.907737+0000","last_peered":"2024-04-26T21:54:11.907737+0000","last_clean":"2024-04-26T21:54:11.907737+0000","last_became_active":"2024-04-26T21:52:50.773347+0000","last_became_peered":"2024-04-26T21:52:50.773347+0000","last_unstale":"2024-04-26T21:54:11.907737+0000","last_undegraded":"2024-04-26T21:54:11.907737+0000","last_fullsized":"2024-04-26T21:54:11.907737+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:59:55.303014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"87'6","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.934310+0000","last_change":"2024-04-26T21:52:50.109437+0000","last_active":"2024-04-26T21:54:11.934310+0000","last_peered":"2024-04-26T21:54:11.934310+0000","last_clean":"2024-04-26T21:54:11.934310+0000","last_became_active":"2024-04-26T21:52:50.108605+0000","last_became_peered":"2024-04-26T21:52:50.108605+0000","last_unstale":"2024-04-26T21:54:11.934310+0000","last_undegraded":"2024-04-26T21:54:11.934310+0000","last_fullsized":"2024-04-26T21:54:11.934310+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:31:04.208334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":350,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.922023+0000","last_change":"2024-04-26T21:51:11.498577+0000","last_active":"2024-04-26T21:54:11.922023+0000","last_peered":"2024-04-26T21:54:11.922023+0000","last_clean":"2024-04-26T21:54:11.922023+0000","last_became_active":"2024-04-26T21:51:11.497422+0000","last_became_peered":"2024-04-26T21:51:11.497422+0000","last_unstale":"2024-04-26T21:54:11.922023+0000","last_undegraded":"2024-04-26T21:54:11.922023+0000","last_fullsized":"2024-04-26T21:54:11.922023+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:46:29.789400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:46:29.789400+0000","last_clean_scrub_stamp":"2024-04-26T21:46:29.789400+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:54:13.450359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:12.395065+0000","last_change":"2024-04-26T21:52:50.794270+0000","last_active":"2024-04-26T21:54:12.395065+0000","last_peered":"2024-04-26T21:54:12.395065+0000","last_clean":"2024-04-26T21:54:12.395065+0000","last_became_active":"2024-04-26T21:52:50.793966+0000","last_became_peered":"2024-04-26T21:52:50.793966+0000","last_unstale":"2024-04-26T21:54:12.395065+0000","last_undegraded":"2024-04-26T21:54:12.395065+0000","last_fullsized":"2024-04-26T21:54:12.395065+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:01:37.668466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"87'4","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.922067+0000","last_change":"2024-04-26T21:52:50.102700+0000","last_active":"2024-04-26T21:54:11.922067+0000","last_peered":"2024-04-26T21:54:11.922067+0000","last_clean":"2024-04-26T21:54:11.922067+0000","last_became_active":"2024-04-26T21:52:50.101267+0000","last_became_peered":"2024-04-26T21:52:50.101267+0000","last_unstale":"2024-04-26T21:54:11.922067+0000","last_undegraded":"2024-04-26T21:54:11.922067+0000","last_fullsized":"2024-04-26T21:54:11.922067+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:41:21.727411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"87'4","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.914022+0000","last_change":"2024-04-26T21:52:50.102191+0000","last_active":"2024-04-26T21:54:11.914022+0000","last_peered":"2024-04-26T21:54:11.914022+0000","last_clean":"2024-04-26T21:54:11.914022+0000","last_became_active":"2024-04-26T21:52:50.101307+0000","last_became_peered":"2024-04-26T21:52:50.101307+0000","last_unstale":"2024-04-26T21:54:11.914022+0000","last_undegraded":"2024-04-26T21:54:11.914022+0000","last_fullsized":"2024-04-26T21:54:11.914022+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:59:12.465291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:12.395488+0000","last_change":"2024-04-26T21:52:50.792020+0000","last_active":"2024-04-26T21:54:12.395488+0000","last_peered":"2024-04-26T21:54:12.395488+0000","last_clean":"2024-04-26T21:54:12.395488+0000","last_became_active":"2024-04-26T21:52:50.791677+0000","last_became_peered":"2024-04-26T21:52:50.791677+0000","last_unstale":"2024-04-26T21:54:12.395488+0000","last_undegraded":"2024-04-26T21:54:12.395488+0000","last_fullsized":"2024-04-26T21:54:12.395488+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:11:42.218294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"91'13","reported_seq":53,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:59.673130+0000","last_change":"2024-04-26T21:54:11.936807+0000","last_active":"2024-04-26T21:54:59.673130+0000","last_peered":"2024-04-26T21:54:59.673130+0000","last_clean":"2024-04-26T21:54:59.673130+0000","last_became_active":"2024-04-26T21:54:11.923482+0000","last_became_peered":"2024-04-26T21:54:11.923482+0000","last_unstale":"2024-04-26T21:54:59.673130+0000","last_undegraded":"2024-04-26T21:54:59.673130+0000","last_fullsized":"2024-04-26T21:54:59.673130+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:15:46.945453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.908090+0000","last_change":"2024-04-26T21:52:50.779775+0000","last_active":"2024-04-26T21:54:11.908090+0000","last_peered":"2024-04-26T21:54:11.908090+0000","last_clean":"2024-04-26T21:54:11.908090+0000","last_became_active":"2024-04-26T21:52:50.778857+0000","last_became_peered":"2024-04-26T21:52:50.778857+0000","last_unstale":"2024-04-26T21:54:11.908090+0000","last_undegraded":"2024-04-26T21:54:11.908090+0000","last_fullsized":"2024-04-26T21:54:11.908090+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:59:50.946578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"87'7","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.934470+0000","last_change":"2024-04-26T21:52:50.113027+0000","last_active":"2024-04-26T21:54:11.934470+0000","last_peered":"2024-04-26T21:54:11.934470+0000","last_clean":"2024-04-26T21:54:11.934470+0000","last_became_active":"2024-04-26T21:52:50.112637+0000","last_became_peered":"2024-04-26T21:52:50.112637+0000","last_unstale":"2024-04-26T21:54:11.934470+0000","last_undegraded":"2024-04-26T21:54:11.934470+0000","last_fullsized":"2024-04-26T21:54:11.934470+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:09:47.798184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:12.034220+0000","last_change":"2024-04-26T21:52:50.785119+0000","last_active":"2024-04-26T21:54:12.034220+0000","last_peered":"2024-04-26T21:54:12.034220+0000","last_clean":"2024-04-26T21:54:12.034220+0000","last_became_active":"2024-04-26T21:52:50.781005+0000","last_became_peered":"2024-04-26T21:52:50.781005+0000","last_unstale":"2024-04-26T21:54:12.034220+0000","last_undegraded":"2024-04-26T21:54:12.034220+0000","last_fullsized":"2024-04-26T21:54:12.034220+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:16:32.986810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"87'2","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.908468+0000","last_change":"2024-04-26T21:52:50.112263+0000","last_active":"2024-04-26T21:54:11.908468+0000","last_peered":"2024-04-26T21:54:11.908468+0000","last_clean":"2024-04-26T21:54:11.908468+0000","last_became_active":"2024-04-26T21:52:50.110649+0000","last_became_peered":"2024-04-26T21:52:50.110649+0000","last_unstale":"2024-04-26T21:54:11.908468+0000","last_undegraded":"2024-04-26T21:54:11.908468+0000","last_fullsized":"2024-04-26T21:54:11.908468+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:14:23.245000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.913095+0000","last_change":"2024-04-26T21:52:50.790674+0000","last_active":"2024-04-26T21:54:11.913095+0000","last_peered":"2024-04-26T21:54:11.913095+0000","last_clean":"2024-04-26T21:54:11.913095+0000","last_became_active":"2024-04-26T21:52:50.790286+0000","last_became_peered":"2024-04-26T21:52:50.790286+0000","last_unstale":"2024-04-26T21:54:11.913095+0000","last_undegraded":"2024-04-26T21:54:11.913095+0000","last_fullsized":"2024-04-26T21:54:11.913095+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:22:32.246321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"87'7","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.907591+0000","last_change":"2024-04-26T21:52:50.112158+0000","last_active":"2024-04-26T21:54:11.907591+0000","last_peered":"2024-04-26T21:54:11.907591+0000","last_clean":"2024-04-26T21:54:11.907591+0000","last_became_active":"2024-04-26T21:52:50.111453+0000","last_became_peered":"2024-04-26T21:52:50.111453+0000","last_unstale":"2024-04-26T21:54:11.907591+0000","last_undegraded":"2024-04-26T21:54:11.907591+0000","last_fullsized":"2024-04-26T21:54:11.907591+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:16:58.626985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.915638+0000","last_change":"2024-04-26T21:52:50.798241+0000","last_active":"2024-04-26T21:54:11.915638+0000","last_peered":"2024-04-26T21:54:11.915638+0000","last_clean":"2024-04-26T21:54:11.915638+0000","last_became_active":"2024-04-26T21:52:50.797878+0000","last_became_peered":"2024-04-26T21:52:50.797878+0000","last_unstale":"2024-04-26T21:54:11.915638+0000","last_undegraded":"2024-04-26T21:54:11.915638+0000","last_fullsized":"2024-04-26T21:54:11.915638+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:42:49.363849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"91'5","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:55:20.595077+0000","last_change":"2024-04-26T21:54:11.946852+0000","last_active":"2024-04-26T21:55:20.595077+0000","last_peered":"2024-04-26T21:55:20.595077+0000","last_clean":"2024-04-26T21:55:20.595077+0000","last_became_active":"2024-04-26T21:54:11.923399+0000","last_became_peered":"2024-04-26T21:54:11.923399+0000","last_unstale":"2024-04-26T21:55:20.595077+0000","last_undegraded":"2024-04-26T21:55:20.595077+0000","last_fullsized":"2024-04-26T21:55:20.595077+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:05:34.688374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.908206+0000","last_change":"2024-04-26T21:52:50.776333+0000","last_active":"2024-04-26T21:54:11.908206+0000","last_peered":"2024-04-26T21:54:11.908206+0000","last_clean":"2024-04-26T21:54:11.908206+0000","last_became_active":"2024-04-26T21:52:50.776001+0000","last_became_peered":"2024-04-26T21:52:50.776001+0000","last_unstale":"2024-04-26T21:54:11.908206+0000","last_undegraded":"2024-04-26T21:54:11.908206+0000","last_fullsized":"2024-04-26T21:54:11.908206+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:38:26.578928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"87'4","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.913422+0000","last_change":"2024-04-26T21:52:50.106008+0000","last_active":"2024-04-26T21:54:11.913422+0000","last_peered":"2024-04-26T21:54:11.913422+0000","last_clean":"2024-04-26T21:54:11.913422+0000","last_became_active":"2024-04-26T21:52:50.105565+0000","last_became_peered":"2024-04-26T21:52:50.105565+0000","last_unstale":"2024-04-26T21:54:11.913422+0000","last_undegraded":"2024-04-26T21:54:11.913422+0000","last_fullsized":"2024-04-26T21:54:11.913422+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:15:17.355461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.923394+0000","last_change":"2024-04-26T21:52:50.776051+0000","last_active":"2024-04-26T21:54:11.923394+0000","last_peered":"2024-04-26T21:54:11.923394+0000","last_clean":"2024-04-26T21:54:11.923394+0000","last_became_active":"2024-04-26T21:52:50.775716+0000","last_became_peered":"2024-04-26T21:52:50.775716+0000","last_unstale":"2024-04-26T21:54:11.923394+0000","last_undegraded":"2024-04-26T21:54:11.923394+0000","last_fullsized":"2024-04-26T21:54:11.923394+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:34:21.110773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"91'7","reported_seq":180,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:55:22.950112+0000","last_change":"2024-04-26T21:52:50.111280+0000","last_active":"2024-04-26T21:55:22.950112+0000","last_peered":"2024-04-26T21:55:22.950112+0000","last_clean":"2024-04-26T21:55:22.950112+0000","last_became_active":"2024-04-26T21:52:50.110972+0000","last_became_peered":"2024-04-26T21:52:50.110972+0000","last_unstale":"2024-04-26T21:55:22.950112+0000","last_undegraded":"2024-04-26T21:55:22.950112+0000","last_fullsized":"2024-04-26T21:55:22.950112+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:41:48.203134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2054,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":149,"num_read_kb":3,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.923325+0000","last_change":"2024-04-26T21:52:50.777488+0000","last_active":"2024-04-26T21:54:11.923325+0000","last_peered":"2024-04-26T21:54:11.923325+0000","last_clean":"2024-04-26T21:54:11.923325+0000","last_became_active":"2024-04-26T21:52:50.777162+0000","last_became_peered":"2024-04-26T21:52:50.777162+0000","last_unstale":"2024-04-26T21:54:11.923325+0000","last_undegraded":"2024-04-26T21:54:11.923325+0000","last_fullsized":"2024-04-26T21:54:11.923325+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:07:22.683109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"87'3","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.916669+0000","last_change":"2024-04-26T21:52:50.104508+0000","last_active":"2024-04-26T21:54:11.916669+0000","last_peered":"2024-04-26T21:54:11.916669+0000","last_clean":"2024-04-26T21:54:11.916669+0000","last_became_active":"2024-04-26T21:52:50.103924+0000","last_became_peered":"2024-04-26T21:52:50.103924+0000","last_unstale":"2024-04-26T21:54:11.916669+0000","last_undegraded":"2024-04-26T21:54:11.916669+0000","last_fullsized":"2024-04-26T21:54:11.916669+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:39:25.861339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"87'1","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.922627+0000","last_change":"2024-04-26T21:52:50.782915+0000","last_active":"2024-04-26T21:54:11.922627+0000","last_peered":"2024-04-26T21:54:11.922627+0000","last_clean":"2024-04-26T21:54:11.922627+0000","last_became_active":"2024-04-26T21:52:50.782253+0000","last_became_peered":"2024-04-26T21:52:50.782253+0000","last_unstale":"2024-04-26T21:54:11.922627+0000","last_undegraded":"2024-04-26T21:54:11.922627+0000","last_fullsized":"2024-04-26T21:54:11.922627+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:12:51.624162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"87'5","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:12.395358+0000","last_change":"2024-04-26T21:53:11.590613+0000","last_active":"2024-04-26T21:54:12.395358+0000","last_peered":"2024-04-26T21:54:12.395358+0000","last_clean":"2024-04-26T21:54:12.395358+0000","last_became_active":"2024-04-26T21:53:11.578727+0000","last_became_peered":"2024-04-26T21:53:11.578727+0000","last_unstale":"2024-04-26T21:54:12.395358+0000","last_undegraded":"2024-04-26T21:54:12.395358+0000","last_fullsized":"2024-04-26T21:54:12.395358+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:44:38.867231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.922922+0000","last_change":"2024-04-26T21:52:50.787777+0000","last_active":"2024-04-26T21:54:11.922922+0000","last_peered":"2024-04-26T21:54:11.922922+0000","last_clean":"2024-04-26T21:54:11.922922+0000","last_became_active":"2024-04-26T21:52:50.782403+0000","last_became_peered":"2024-04-26T21:52:50.782403+0000","last_unstale":"2024-04-26T21:54:11.922922+0000","last_undegraded":"2024-04-26T21:54:11.922922+0000","last_fullsized":"2024-04-26T21:54:11.922922+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:28:58.425802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.915961+0000","last_change":"2024-04-26T21:52:50.800010+0000","last_active":"2024-04-26T21:54:11.915961+0000","last_peered":"2024-04-26T21:54:11.915961+0000","last_clean":"2024-04-26T21:54:11.915961+0000","last_became_active":"2024-04-26T21:52:50.799308+0000","last_became_peered":"2024-04-26T21:52:50.799308+0000","last_unstale":"2024-04-26T21:54:11.915961+0000","last_undegraded":"2024-04-26T21:54:11.915961+0000","last_fullsized":"2024-04-26T21:54:11.915961+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:54:25.772312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.909036+0000","last_change":"2024-04-26T21:52:50.798281+0000","last_active":"2024-04-26T21:54:11.909036+0000","last_peered":"2024-04-26T21:54:11.909036+0000","last_clean":"2024-04-26T21:54:11.909036+0000","last_became_active":"2024-04-26T21:52:50.795879+0000","last_became_peered":"2024-04-26T21:52:50.795879+0000","last_unstale":"2024-04-26T21:54:11.909036+0000","last_undegraded":"2024-04-26T21:54:11.909036+0000","last_fullsized":"2024-04-26T21:54:11.909036+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:36:00.769991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.908710+0000","last_change":"2024-04-26T21:52:50.791058+0000","last_active":"2024-04-26T21:54:11.908710+0000","last_peered":"2024-04-26T21:54:11.908710+0000","last_clean":"2024-04-26T21:54:11.908710+0000","last_became_active":"2024-04-26T21:52:50.790721+0000","last_became_peered":"2024-04-26T21:52:50.790721+0000","last_unstale":"2024-04-26T21:54:11.908710+0000","last_undegraded":"2024-04-26T21:54:11.908710+0000","last_fullsized":"2024-04-26T21:54:11.908710+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:56:03.239622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:12.033664+0000","last_change":"2024-04-26T21:52:50.788193+0000","last_active":"2024-04-26T21:54:12.033664+0000","last_peered":"2024-04-26T21:54:12.033664+0000","last_clean":"2024-04-26T21:54:12.033664+0000","last_became_active":"2024-04-26T21:52:50.786749+0000","last_became_peered":"2024-04-26T21:52:50.786749+0000","last_unstale":"2024-04-26T21:54:12.033664+0000","last_undegraded":"2024-04-26T21:54:12.033664+0000","last_fullsized":"2024-04-26T21:54:12.033664+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:22:11.277217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:12.395673+0000","last_change":"2024-04-26T21:52:50.782508+0000","last_active":"2024-04-26T21:54:12.395673+0000","last_peered":"2024-04-26T21:54:12.395673+0000","last_clean":"2024-04-26T21:54:12.395673+0000","last_became_active":"2024-04-26T21:52:50.781505+0000","last_became_peered":"2024-04-26T21:52:50.781505+0000","last_unstale":"2024-04-26T21:54:12.395673+0000","last_undegraded":"2024-04-26T21:54:12.395673+0000","last_fullsized":"2024-04-26T21:54:12.395673+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:22:57.867546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.916618+0000","last_change":"2024-04-26T21:52:50.789205+0000","last_active":"2024-04-26T21:54:11.916618+0000","last_peered":"2024-04-26T21:54:11.916618+0000","last_clean":"2024-04-26T21:54:11.916618+0000","last_became_active":"2024-04-26T21:52:50.788898+0000","last_became_peered":"2024-04-26T21:52:50.788898+0000","last_unstale":"2024-04-26T21:54:11.916618+0000","last_undegraded":"2024-04-26T21:54:11.916618+0000","last_fullsized":"2024-04-26T21:54:11.916618+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:18:51.401968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.909356+0000","last_change":"2024-04-26T21:52:50.793451+0000","last_active":"2024-04-26T21:54:11.909356+0000","last_peered":"2024-04-26T21:54:11.909356+0000","last_clean":"2024-04-26T21:54:11.909356+0000","last_became_active":"2024-04-26T21:52:50.791696+0000","last_became_peered":"2024-04-26T21:52:50.791696+0000","last_unstale":"2024-04-26T21:54:11.909356+0000","last_undegraded":"2024-04-26T21:54:11.909356+0000","last_fullsized":"2024-04-26T21:54:11.909356+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:51:43.543500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.907992+0000","last_change":"2024-04-26T21:53:10.919691+0000","last_active":"2024-04-26T21:54:11.907992+0000","last_peered":"2024-04-26T21:54:11.907992+0000","last_clean":"2024-04-26T21:54:11.907992+0000","last_became_active":"2024-04-26T21:53:10.919262+0000","last_became_peered":"2024-04-26T21:53:10.919262+0000","last_unstale":"2024-04-26T21:54:11.907992+0000","last_undegraded":"2024-04-26T21:54:11.907992+0000","last_fullsized":"2024-04-26T21:54:11.907992+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:15:35.719661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,8,2],"acting":[3,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:12.395203+0000","last_change":"2024-04-26T21:53:10.914854+0000","last_active":"2024-04-26T21:54:12.395203+0000","last_peered":"2024-04-26T21:54:12.395203+0000","last_clean":"2024-04-26T21:54:12.395203+0000","last_became_active":"2024-04-26T21:53:10.914458+0000","last_became_peered":"2024-04-26T21:53:10.914458+0000","last_unstale":"2024-04-26T21:54:12.395203+0000","last_undegraded":"2024-04-26T21:54:12.395203+0000","last_fullsized":"2024-04-26T21:54:12.395203+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:26:41.538053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,3],"acting":[6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.907533+0000","last_change":"2024-04-26T21:52:50.794580+0000","last_active":"2024-04-26T21:54:11.907533+0000","last_peered":"2024-04-26T21:54:11.907533+0000","last_clean":"2024-04-26T21:54:11.907533+0000","last_became_active":"2024-04-26T21:52:50.792398+0000","last_became_peered":"2024-04-26T21:52:50.792398+0000","last_unstale":"2024-04-26T21:54:11.907533+0000","last_undegraded":"2024-04-26T21:54:11.907533+0000","last_fullsized":"2024-04-26T21:54:11.907533+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:21:38.913644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.909622+0000","last_change":"2024-04-26T21:53:11.580576+0000","last_active":"2024-04-26T21:54:11.909622+0000","last_peered":"2024-04-26T21:54:11.909622+0000","last_clean":"2024-04-26T21:54:11.909622+0000","last_became_active":"2024-04-26T21:53:11.578632+0000","last_became_peered":"2024-04-26T21:53:11.578632+0000","last_unstale":"2024-04-26T21:54:11.909622+0000","last_undegraded":"2024-04-26T21:54:11.909622+0000","last_fullsized":"2024-04-26T21:54:11.909622+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:30:24.412417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,8],"acting":[1,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:12.396035+0000","last_change":"2024-04-26T21:52:50.792166+0000","last_active":"2024-04-26T21:54:12.396035+0000","last_peered":"2024-04-26T21:54:12.396035+0000","last_clean":"2024-04-26T21:54:12.396035+0000","last_became_active":"2024-04-26T21:52:50.790709+0000","last_became_peered":"2024-04-26T21:52:50.790709+0000","last_unstale":"2024-04-26T21:54:12.396035+0000","last_undegraded":"2024-04-26T21:54:12.396035+0000","last_fullsized":"2024-04-26T21:54:12.396035+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:05:48.200573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.915443+0000","last_change":"2024-04-26T21:53:10.930526+0000","last_active":"2024-04-26T21:54:11.915443+0000","last_peered":"2024-04-26T21:54:11.915443+0000","last_clean":"2024-04-26T21:54:11.915443+0000","last_became_active":"2024-04-26T21:53:10.928850+0000","last_became_peered":"2024-04-26T21:53:10.928850+0000","last_unstale":"2024-04-26T21:54:11.915443+0000","last_undegraded":"2024-04-26T21:54:11.915443+0000","last_fullsized":"2024-04-26T21:54:11.915443+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:54:03.420855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":60229,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1491,"num_read_kb":814,"num_write":255,"num_write_kb":137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":622,"num_keys_recovered":7,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":548864,"data_stored":189315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":218,"ondisk_log_size":218,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":82,"seq":352187318319,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1015808,"data_stored":485396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":74,"seq":317827579958,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28684,"kb_used_data":1668,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716468,"statfs":{"total":95995035648,"available":95965663232,"internally_reserved":0,"allocated":1708032,"data_stored":1127478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":69,"seq":296352743486,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36352,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708800,"statfs":{"total":95995035648,"available":95957811200,"internally_reserved":0,"allocated":1040384,"data_stored":484442,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":61,"seq":261993005126,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36348,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708804,"statfs":{"total":95995035648,"available":95957815296,"internally_reserved":0,"allocated":1036288,"data_stored":484820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299466,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28612,"kb_used_data":1596,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716540,"statfs":{"total":95995035648,"available":95965736960,"internally_reserved":0,"allocated":1634304,"data_stored":1078426,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462995,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36336,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708816,"statfs":{"total":95995035648,"available":95957827584,"internally_reserved":0,"allocated":1024000,"data_stored":484652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574965,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28068,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717084,"statfs":{"total":95995035648,"available":95966294016,"internally_reserved":0,"allocated":1077248,"data_stored":537009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738492,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36356,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708796,"statfs":{"total":95995035648,"available":95957807104,"internally_reserved":0,"allocated":1044480,"data_stored":486382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411439,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36400,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708752,"statfs":{"total":95995035648,"available":95957762048,"internally_reserved":0,"allocated":1089536,"data_stored":536998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182502,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1040384,"data_stored":483984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920866,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36344,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708808,"statfs":{"total":95995035648,"available":95957819392,"internally_reserved":0,"allocated":1032192,"data_stored":485628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724632,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36944,"kb_used_data":1608,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708208,"statfs":{"total":95995035648,"available":95957204992,"internally_reserved":0,"allocated":1646592,"data_stored":1077118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":58003,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":57992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2308,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1215,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4621,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":53673,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":2076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T21:55:27.400 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T21:55:27.613 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:27.612+0000 7f9f9e200640 1 -- 172.21.15.72:0/566725474 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9f98108d90 msgr2=0x7f9f981091b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:27.613 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:27.612+0000 7f9f9e200640 1 --2- 172.21.15.72:0/566725474 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9f98108d90 0x7f9f981091b0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f9f8c009900 tx=0x7f9f8c02a410 comp rx=0 tx=0).stop 2024-04-26T21:55:27.613 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:27.612+0000 7f9f9e200640 1 -- 172.21.15.72:0/566725474 shutdown_connections 2024-04-26T21:55:27.613 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:27.612+0000 7f9f9e200640 1 --2- 172.21.15.72:0/566725474 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9f9810a940 0x7f9f981111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:27.613 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:27.612+0000 7f9f9e200640 1 --2- 172.21.15.72:0/566725474 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9f98109f80 0x7f9f9810a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:27.613 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:27.612+0000 7f9f9e200640 1 --2- 172.21.15.72:0/566725474 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9f98108d90 0x7f9f981091b0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:27.613 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:27.612+0000 7f9f9e200640 1 -- 172.21.15.72:0/566725474 >> 172.21.15.72:0/566725474 conn(0x7f9f98077d30 msgr2=0x7f9f9807a170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:55:27.613 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:27.612+0000 7f9f9e200640 1 -- 172.21.15.72:0/566725474 shutdown_connections 2024-04-26T21:55:27.613 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:27.612+0000 7f9f9e200640 1 -- 172.21.15.72:0/566725474 wait complete. 2024-04-26T21:55:27.615 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:27.613+0000 7f9f9e200640 1 Processor -- start 2024-04-26T21:55:27.616 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:27.613+0000 7f9f9e200640 1 -- start start 2024-04-26T21:55:27.616 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:27.614+0000 7f9f9e200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9f98108d90 0x7f9f9819ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:27.616 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:27.614+0000 7f9f9e200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9f98109f80 0x7f9f9819f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:27.616 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:27.614+0000 7f9f9e200640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9f9810a940 0x7f9f981a6430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:27.616 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:27.614+0000 7f9f9e200640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f9f98114790 con 0x7f9f9810a940 2024-04-26T21:55:27.616 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:27.614+0000 7f9f9e200640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f9f98114610 con 0x7f9f98109f80 2024-04-26T21:55:27.616 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:27.614+0000 7f9f9e200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f9f98114910 con 0x7f9f98108d90 2024-04-26T21:55:27.616 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:27.614+0000 7f9f9d800640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9f9810a940 0x7f9f981a6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:27.616 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:27.614+0000 7f9f9d800640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9f9810a940 0x7f9f981a6430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:44190/0 (socket says 172.21.15.72:44190) 2024-04-26T21:55:27.616 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:27.614+0000 7f9f9d800640 1 -- 172.21.15.72:0/2289742388 learned_addr learned my addr 172.21.15.72:0/2289742388 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:55:27.616 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:27.614+0000 7f9f9ce00640 1 --2- 172.21.15.72:0/2289742388 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9f98108d90 0x7f9f9819ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:27.616 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:27.614+0000 7f9f97e00640 1 --2- 172.21.15.72:0/2289742388 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9f98109f80 0x7f9f9819f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:27.616 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:27.614+0000 7f9f9d800640 1 -- 172.21.15.72:0/2289742388 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9f98108d90 msgr2=0x7f9f9819ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:27.617 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:27.614+0000 7f9f9d800640 1 --2- 172.21.15.72:0/2289742388 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9f98108d90 0x7f9f9819ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:27.617 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:27.614+0000 7f9f9d800640 1 -- 172.21.15.72:0/2289742388 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9f98109f80 msgr2=0x7f9f9819f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:27.617 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:27.614+0000 7f9f9d800640 1 --2- 172.21.15.72:0/2289742388 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9f98109f80 0x7f9f9819f3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:27.617 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:27.614+0000 7f9f9d800640 1 -- 172.21.15.72:0/2289742388 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f981a6aa0 con 0x7f9f9810a940 2024-04-26T21:55:27.617 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:27.614+0000 7f9f97e00640 1 --2- 172.21.15.72:0/2289742388 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9f98109f80 0x7f9f9819f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:55:27.617 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:27.615+0000 7f9f9d800640 1 --2- 172.21.15.72:0/2289742388 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9f9810a940 0x7f9f981a6430 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f9f9000cce0 tx=0x7f9f9000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:55:27.617 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:27.615+0000 7f9f95600640 1 -- 172.21.15.72:0/2289742388 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f9000f450 con 0x7f9f9810a940 2024-04-26T21:55:27.617 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:27.615+0000 7f9f95600640 1 -- 172.21.15.72:0/2289742388 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9f9000fa80 con 0x7f9f9810a940 2024-04-26T21:55:27.617 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:27.615+0000 7f9f9e200640 1 -- 172.21.15.72:0/2289742388 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f981a6d90 con 0x7f9f9810a940 2024-04-26T21:55:27.617 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:27.615+0000 7f9f95600640 1 -- 172.21.15.72:0/2289742388 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f90018a70 con 0x7f9f9810a940 2024-04-26T21:55:27.617 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:27.615+0000 7f9f9e200640 1 -- 172.21.15.72:0/2289742388 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f981a7230 con 0x7f9f9810a940 2024-04-26T21:55:27.618 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:27.616+0000 7f9f9e200640 1 -- 172.21.15.72:0/2289742388 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f98110180 con 0x7f9f9810a940 2024-04-26T21:55:27.618 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:27.618+0000 7f9f95600640 1 -- 172.21.15.72:0/2289742388 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f9f90018c10 con 0x7f9f9810a940 2024-04-26T21:55:27.619 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:27.619+0000 7f9f95600640 1 --2- 172.21.15.72:0/2289742388 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f9f800a3c40 0x7f9f800a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:27.619 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:27.620+0000 7f9f95600640 1 -- 172.21.15.72:0/2289742388 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7f9f900d0ad0 con 0x7f9f9810a940 2024-04-26T21:55:27.620 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:27.620+0000 7f9f9ce00640 1 --2- 172.21.15.72:0/2289742388 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f9f800a3c40 0x7f9f800a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:27.620 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:27.620+0000 7f9f9ce00640 1 --2- 172.21.15.72:0/2289742388 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f9f800a3c40 0x7f9f800a6100 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f9f8c0057c0 tx=0x7f9f8c000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:55:27.623 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:27.623+0000 7f9f95600640 1 -- 172.21.15.72:0/2289742388 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f9f90014030 con 0x7f9f9810a940 2024-04-26T21:55:27.828 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:27.828+0000 7f9f9e200640 1 -- 172.21.15.72:0/2289742388 --> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9f98062d70 con 0x7f9f800a3c40 2024-04-26T21:55:27.837 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:27.837+0000 7f9f95600640 1 -- 172.21.15.72:0/2289742388 <== mgr.14152 v2:172.21.15.72:6800/588150172 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258916 (secure 0 0 0) 0x7f9f98062d70 con 0x7f9f800a3c40 2024-04-26T21:55:27.840 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T21:55:27.841 INFO:teuthology.orchestra.run.smithi072.stderr:dumped all 2024-04-26T21:55:27.843 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:27.841+0000 7f9f9e200640 1 -- 172.21.15.72:0/2289742388 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f9f800a3c40 msgr2=0x7f9f800a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:27.843 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:27.841+0000 7f9f9e200640 1 --2- 172.21.15.72:0/2289742388 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f9f800a3c40 0x7f9f800a6100 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f9f8c0057c0 tx=0x7f9f8c000bc0 comp rx=0 tx=0).stop 2024-04-26T21:55:27.843 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:27.841+0000 7f9f9e200640 1 -- 172.21.15.72:0/2289742388 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9f9810a940 msgr2=0x7f9f981a6430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:27.843 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:27.841+0000 7f9f9e200640 1 --2- 172.21.15.72:0/2289742388 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9f9810a940 0x7f9f981a6430 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f9f9000cce0 tx=0x7f9f9000a620 comp rx=0 tx=0).stop 2024-04-26T21:55:27.843 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:27.841+0000 7f9f9e200640 1 -- 172.21.15.72:0/2289742388 shutdown_connections 2024-04-26T21:55:27.843 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:27.841+0000 7f9f9e200640 1 --2- 172.21.15.72:0/2289742388 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9f9810a940 0x7f9f981a6430 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:27.843 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:27.841+0000 7f9f9e200640 1 --2- 172.21.15.72:0/2289742388 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9f98109f80 0x7f9f9819f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:27.843 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:27.841+0000 7f9f9e200640 1 --2- 172.21.15.72:0/2289742388 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f9f800a3c40 0x7f9f800a6100 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:27.843 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:27.841+0000 7f9f9e200640 1 --2- 172.21.15.72:0/2289742388 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9f98108d90 0x7f9f9819ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:27.843 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:27.841+0000 7f9f9e200640 1 -- 172.21.15.72:0/2289742388 >> 172.21.15.72:0/2289742388 conn(0x7f9f98077d30 msgr2=0x7f9f9810d350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:55:27.843 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:27.841+0000 7f9f9e200640 1 -- 172.21.15.72:0/2289742388 shutdown_connections 2024-04-26T21:55:27.843 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:27.841+0000 7f9f9e200640 1 -- 172.21.15.72:0/2289742388 wait complete. 2024-04-26T21:55:27.857 INFO:teuthology.orchestra.run.smithi072.stdout:{"pg_ready":true,"pg_map":{"version":449,"stamp":"2024-04-26T21:55:25.832056+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":650831,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1617,"num_read_kb":923,"num_write":438,"num_write_kb":3077,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":1181358,"num_keys_recovered":7,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":357,"ondisk_log_size":357,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":396484,"kb_used_data":14052,"kb_used_omap":18,"kb_used_meta":382317,"kb_avail":1124545340,"statfs":{"total":1151940427776,"available":1151534428160,"internally_reserved":0,"allocated":14389248,"data_stored":7752333,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":391492922},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2465,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":144,"num_read_kb":79,"num_write":18,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004264"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.921489+0000","last_change":"2024-04-26T21:52:50.787765+0000","last_active":"2024-04-26T21:54:11.921489+0000","last_peered":"2024-04-26T21:54:11.921489+0000","last_clean":"2024-04-26T21:54:11.921489+0000","last_became_active":"2024-04-26T21:52:50.786969+0000","last_became_peered":"2024-04-26T21:52:50.786969+0000","last_unstale":"2024-04-26T21:54:11.921489+0000","last_undegraded":"2024-04-26T21:54:11.921489+0000","last_fullsized":"2024-04-26T21:54:11.921489+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:31:50.659210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:12.034292+0000","last_change":"2024-04-26T21:52:50.779735+0000","last_active":"2024-04-26T21:54:12.034292+0000","last_peered":"2024-04-26T21:54:12.034292+0000","last_clean":"2024-04-26T21:54:12.034292+0000","last_became_active":"2024-04-26T21:52:50.779358+0000","last_became_peered":"2024-04-26T21:52:50.779358+0000","last_unstale":"2024-04-26T21:54:12.034292+0000","last_undegraded":"2024-04-26T21:54:12.034292+0000","last_fullsized":"2024-04-26T21:54:12.034292+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:58:45.631365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:12.033978+0000","last_change":"2024-04-26T21:52:50.788220+0000","last_active":"2024-04-26T21:54:12.033978+0000","last_peered":"2024-04-26T21:54:12.033978+0000","last_clean":"2024-04-26T21:54:12.033978+0000","last_became_active":"2024-04-26T21:52:50.787282+0000","last_became_peered":"2024-04-26T21:52:50.787282+0000","last_unstale":"2024-04-26T21:54:12.033978+0000","last_undegraded":"2024-04-26T21:54:12.033978+0000","last_fullsized":"2024-04-26T21:54:12.033978+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:12:28.577893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.934516+0000","last_change":"2024-04-26T21:52:50.795367+0000","last_active":"2024-04-26T21:54:11.934516+0000","last_peered":"2024-04-26T21:54:11.934516+0000","last_clean":"2024-04-26T21:54:11.934516+0000","last_became_active":"2024-04-26T21:52:50.793602+0000","last_became_peered":"2024-04-26T21:52:50.793602+0000","last_unstale":"2024-04-26T21:54:11.934516+0000","last_undegraded":"2024-04-26T21:54:11.934516+0000","last_fullsized":"2024-04-26T21:54:11.934516+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:37:23.356792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.912197+0000","last_change":"2024-04-26T21:52:50.787333+0000","last_active":"2024-04-26T21:54:11.912197+0000","last_peered":"2024-04-26T21:54:11.912197+0000","last_clean":"2024-04-26T21:54:11.912197+0000","last_became_active":"2024-04-26T21:52:50.785319+0000","last_became_peered":"2024-04-26T21:52:50.785319+0000","last_unstale":"2024-04-26T21:54:11.912197+0000","last_undegraded":"2024-04-26T21:54:11.912197+0000","last_fullsized":"2024-04-26T21:54:11.912197+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:06:29.154825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:12.397124+0000","last_change":"2024-04-26T21:52:50.780829+0000","last_active":"2024-04-26T21:54:12.397124+0000","last_peered":"2024-04-26T21:54:12.397124+0000","last_clean":"2024-04-26T21:54:12.397124+0000","last_became_active":"2024-04-26T21:52:50.780439+0000","last_became_peered":"2024-04-26T21:52:50.780439+0000","last_unstale":"2024-04-26T21:54:12.397124+0000","last_undegraded":"2024-04-26T21:54:12.397124+0000","last_fullsized":"2024-04-26T21:54:12.397124+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:20:14.533853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.922216+0000","last_change":"2024-04-26T21:52:50.787679+0000","last_active":"2024-04-26T21:54:11.922216+0000","last_peered":"2024-04-26T21:54:11.922216+0000","last_clean":"2024-04-26T21:54:11.922216+0000","last_became_active":"2024-04-26T21:52:50.787277+0000","last_became_peered":"2024-04-26T21:52:50.787277+0000","last_unstale":"2024-04-26T21:54:11.922216+0000","last_undegraded":"2024-04-26T21:54:11.922216+0000","last_fullsized":"2024-04-26T21:54:11.922216+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:29:02.131503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":15,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.905091+0000","last_change":"2024-04-26T21:53:10.927384+0000","last_active":"2024-04-26T21:54:11.905091+0000","last_peered":"2024-04-26T21:54:11.905091+0000","last_clean":"2024-04-26T21:54:11.905091+0000","last_became_active":"2024-04-26T21:53:10.926996+0000","last_became_peered":"2024-04-26T21:53:10.926996+0000","last_unstale":"2024-04-26T21:54:11.905091+0000","last_undegraded":"2024-04-26T21:54:11.905091+0000","last_fullsized":"2024-04-26T21:54:11.905091+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:25:10.174532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.909555+0000","last_change":"2024-04-26T21:52:50.779701+0000","last_active":"2024-04-26T21:54:11.909555+0000","last_peered":"2024-04-26T21:54:11.909555+0000","last_clean":"2024-04-26T21:54:11.909555+0000","last_became_active":"2024-04-26T21:52:50.779331+0000","last_became_peered":"2024-04-26T21:52:50.779331+0000","last_unstale":"2024-04-26T21:54:11.909555+0000","last_undegraded":"2024-04-26T21:54:11.909555+0000","last_fullsized":"2024-04-26T21:54:11.909555+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:38:15.569806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:12.397905+0000","last_change":"2024-04-26T21:52:50.790081+0000","last_active":"2024-04-26T21:54:12.397905+0000","last_peered":"2024-04-26T21:54:12.397905+0000","last_clean":"2024-04-26T21:54:12.397905+0000","last_became_active":"2024-04-26T21:52:50.786575+0000","last_became_peered":"2024-04-26T21:52:50.786575+0000","last_unstale":"2024-04-26T21:54:12.397905+0000","last_undegraded":"2024-04-26T21:54:12.397905+0000","last_fullsized":"2024-04-26T21:54:12.397905+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:56:47.467128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.923025+0000","last_change":"2024-04-26T21:52:50.791983+0000","last_active":"2024-04-26T21:54:11.923025+0000","last_peered":"2024-04-26T21:54:11.923025+0000","last_clean":"2024-04-26T21:54:11.923025+0000","last_became_active":"2024-04-26T21:52:50.788312+0000","last_became_peered":"2024-04-26T21:52:50.788312+0000","last_unstale":"2024-04-26T21:54:11.923025+0000","last_undegraded":"2024-04-26T21:54:11.923025+0000","last_fullsized":"2024-04-26T21:54:11.923025+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:47:47.410267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.934246+0000","last_change":"2024-04-26T21:52:50.794237+0000","last_active":"2024-04-26T21:54:11.934246+0000","last_peered":"2024-04-26T21:54:11.934246+0000","last_clean":"2024-04-26T21:54:11.934246+0000","last_became_active":"2024-04-26T21:52:50.792260+0000","last_became_peered":"2024-04-26T21:52:50.792260+0000","last_unstale":"2024-04-26T21:54:11.934246+0000","last_undegraded":"2024-04-26T21:54:11.934246+0000","last_fullsized":"2024-04-26T21:54:11.934246+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:36:05.467667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:12.396084+0000","last_change":"2024-04-26T21:52:50.777687+0000","last_active":"2024-04-26T21:54:12.396084+0000","last_peered":"2024-04-26T21:54:12.396084+0000","last_clean":"2024-04-26T21:54:12.396084+0000","last_became_active":"2024-04-26T21:52:50.777200+0000","last_became_peered":"2024-04-26T21:52:50.777200+0000","last_unstale":"2024-04-26T21:54:12.396084+0000","last_undegraded":"2024-04-26T21:54:12.396084+0000","last_fullsized":"2024-04-26T21:54:12.396084+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:40:45.167144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:12.033391+0000","last_change":"2024-04-26T21:52:50.787008+0000","last_active":"2024-04-26T21:54:12.033391+0000","last_peered":"2024-04-26T21:54:12.033391+0000","last_clean":"2024-04-26T21:54:12.033391+0000","last_became_active":"2024-04-26T21:52:50.785656+0000","last_became_peered":"2024-04-26T21:52:50.785656+0000","last_unstale":"2024-04-26T21:54:12.033391+0000","last_undegraded":"2024-04-26T21:54:12.033391+0000","last_fullsized":"2024-04-26T21:54:12.033391+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:20:12.441281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:12.033386+0000","last_change":"2024-04-26T21:52:50.796537+0000","last_active":"2024-04-26T21:54:12.033386+0000","last_peered":"2024-04-26T21:54:12.033386+0000","last_clean":"2024-04-26T21:54:12.033386+0000","last_became_active":"2024-04-26T21:52:50.796229+0000","last_became_peered":"2024-04-26T21:52:50.796229+0000","last_unstale":"2024-04-26T21:54:12.033386+0000","last_undegraded":"2024-04-26T21:54:12.033386+0000","last_fullsized":"2024-04-26T21:54:12.033386+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:38:46.404444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.908416+0000","last_change":"2024-04-26T21:52:50.789211+0000","last_active":"2024-04-26T21:54:11.908416+0000","last_peered":"2024-04-26T21:54:11.908416+0000","last_clean":"2024-04-26T21:54:11.908416+0000","last_became_active":"2024-04-26T21:52:50.788889+0000","last_became_peered":"2024-04-26T21:52:50.788889+0000","last_unstale":"2024-04-26T21:54:11.908416+0000","last_undegraded":"2024-04-26T21:54:11.908416+0000","last_fullsized":"2024-04-26T21:54:11.908416+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:25:48.497840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.921583+0000","last_change":"2024-04-26T21:52:50.788035+0000","last_active":"2024-04-26T21:54:11.921583+0000","last_peered":"2024-04-26T21:54:11.921583+0000","last_clean":"2024-04-26T21:54:11.921583+0000","last_became_active":"2024-04-26T21:52:50.787453+0000","last_became_peered":"2024-04-26T21:52:50.787453+0000","last_unstale":"2024-04-26T21:54:11.921583+0000","last_undegraded":"2024-04-26T21:54:11.921583+0000","last_fullsized":"2024-04-26T21:54:11.921583+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:40:29.718537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.905571+0000","last_change":"2024-04-26T21:52:50.778228+0000","last_active":"2024-04-26T21:54:11.905571+0000","last_peered":"2024-04-26T21:54:11.905571+0000","last_clean":"2024-04-26T21:54:11.905571+0000","last_became_active":"2024-04-26T21:52:50.777934+0000","last_became_peered":"2024-04-26T21:52:50.777934+0000","last_unstale":"2024-04-26T21:54:11.905571+0000","last_undegraded":"2024-04-26T21:54:11.905571+0000","last_fullsized":"2024-04-26T21:54:11.905571+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:33:34.598325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.909217+0000","last_change":"2024-04-26T21:52:50.790795+0000","last_active":"2024-04-26T21:54:11.909217+0000","last_peered":"2024-04-26T21:54:11.909217+0000","last_clean":"2024-04-26T21:54:11.909217+0000","last_became_active":"2024-04-26T21:52:50.788254+0000","last_became_peered":"2024-04-26T21:52:50.788254+0000","last_unstale":"2024-04-26T21:54:11.909217+0000","last_undegraded":"2024-04-26T21:54:11.909217+0000","last_fullsized":"2024-04-26T21:54:11.909217+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:20:28.235137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"91'14","reported_seq":185,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:55:22.927455+0000","last_change":"2024-04-26T21:52:50.105513+0000","last_active":"2024-04-26T21:55:22.927455+0000","last_peered":"2024-04-26T21:55:22.927455+0000","last_clean":"2024-04-26T21:55:22.927455+0000","last_became_active":"2024-04-26T21:52:50.105131+0000","last_became_peered":"2024-04-26T21:52:50.105131+0000","last_unstale":"2024-04-26T21:55:22.927455+0000","last_undegraded":"2024-04-26T21:55:22.927455+0000","last_fullsized":"2024-04-26T21:55:22.927455+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:36:58.085803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":147,"num_read_kb":147,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.908290+0000","last_change":"2024-04-26T21:52:50.795025+0000","last_active":"2024-04-26T21:54:11.908290+0000","last_peered":"2024-04-26T21:54:11.908290+0000","last_clean":"2024-04-26T21:54:11.908290+0000","last_became_active":"2024-04-26T21:52:50.793608+0000","last_became_peered":"2024-04-26T21:52:50.793608+0000","last_unstale":"2024-04-26T21:54:11.908290+0000","last_undegraded":"2024-04-26T21:54:11.908290+0000","last_fullsized":"2024-04-26T21:54:11.908290+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:49:47.764848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"87'2","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.908343+0000","last_change":"2024-04-26T21:52:50.109197+0000","last_active":"2024-04-26T21:54:11.908343+0000","last_peered":"2024-04-26T21:54:11.908343+0000","last_clean":"2024-04-26T21:54:11.908343+0000","last_became_active":"2024-04-26T21:52:50.108835+0000","last_became_peered":"2024-04-26T21:52:50.108835+0000","last_unstale":"2024-04-26T21:54:11.908343+0000","last_undegraded":"2024-04-26T21:54:11.908343+0000","last_fullsized":"2024-04-26T21:54:11.908343+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:45:09.777747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:12.034419+0000","last_change":"2024-04-26T21:52:50.783094+0000","last_active":"2024-04-26T21:54:12.034419+0000","last_peered":"2024-04-26T21:54:12.034419+0000","last_clean":"2024-04-26T21:54:12.034419+0000","last_became_active":"2024-04-26T21:52:50.782468+0000","last_became_peered":"2024-04-26T21:52:50.782468+0000","last_unstale":"2024-04-26T21:54:12.034419+0000","last_undegraded":"2024-04-26T21:54:12.034419+0000","last_fullsized":"2024-04-26T21:54:12.034419+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:53:12.977054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"87'12","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.913469+0000","last_change":"2024-04-26T21:52:50.104824+0000","last_active":"2024-04-26T21:54:11.913469+0000","last_peered":"2024-04-26T21:54:11.913469+0000","last_clean":"2024-04-26T21:54:11.913469+0000","last_became_active":"2024-04-26T21:52:50.104514+0000","last_became_peered":"2024-04-26T21:52:50.104514+0000","last_unstale":"2024-04-26T21:54:11.913469+0000","last_undegraded":"2024-04-26T21:54:11.913469+0000","last_fullsized":"2024-04-26T21:54:11.913469+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:45:31.541940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.909219+0000","last_change":"2024-04-26T21:52:50.779695+0000","last_active":"2024-04-26T21:54:11.909219+0000","last_peered":"2024-04-26T21:54:11.909219+0000","last_clean":"2024-04-26T21:54:11.909219+0000","last_became_active":"2024-04-26T21:52:50.779234+0000","last_became_peered":"2024-04-26T21:52:50.779234+0000","last_unstale":"2024-04-26T21:54:11.909219+0000","last_undegraded":"2024-04-26T21:54:11.909219+0000","last_fullsized":"2024-04-26T21:54:11.909219+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:33:07.316543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"87'5","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.912981+0000","last_change":"2024-04-26T21:52:50.107403+0000","last_active":"2024-04-26T21:54:11.912981+0000","last_peered":"2024-04-26T21:54:11.912981+0000","last_clean":"2024-04-26T21:54:11.912981+0000","last_became_active":"2024-04-26T21:52:50.107126+0000","last_became_peered":"2024-04-26T21:52:50.107126+0000","last_unstale":"2024-04-26T21:54:11.912981+0000","last_undegraded":"2024-04-26T21:54:11.912981+0000","last_fullsized":"2024-04-26T21:54:11.912981+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:52:34.985448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.916932+0000","last_change":"2024-04-26T21:52:50.783674+0000","last_active":"2024-04-26T21:54:11.916932+0000","last_peered":"2024-04-26T21:54:11.916932+0000","last_clean":"2024-04-26T21:54:11.916932+0000","last_became_active":"2024-04-26T21:52:50.782283+0000","last_became_peered":"2024-04-26T21:52:50.782283+0000","last_unstale":"2024-04-26T21:54:11.916932+0000","last_undegraded":"2024-04-26T21:54:11.916932+0000","last_fullsized":"2024-04-26T21:54:11.916932+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:41:03.144601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"91'6","reported_seq":189,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:55:22.154100+0000","last_change":"2024-04-26T21:54:11.949461+0000","last_active":"2024-04-26T21:55:22.154100+0000","last_peered":"2024-04-26T21:55:22.154100+0000","last_clean":"2024-04-26T21:55:22.154100+0000","last_became_active":"2024-04-26T21:54:11.936983+0000","last_became_peered":"2024-04-26T21:54:11.936983+0000","last_unstale":"2024-04-26T21:55:22.154100+0000","last_undegraded":"2024-04-26T21:55:22.154100+0000","last_fullsized":"2024-04-26T21:55:22.154100+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:36:40.757999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1115,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":146,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":622,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.912376+0000","last_change":"2024-04-26T21:53:11.575919+0000","last_active":"2024-04-26T21:54:11.912376+0000","last_peered":"2024-04-26T21:54:11.912376+0000","last_clean":"2024-04-26T21:54:11.912376+0000","last_became_active":"2024-04-26T21:53:11.574898+0000","last_became_peered":"2024-04-26T21:53:11.574898+0000","last_unstale":"2024-04-26T21:54:11.912376+0000","last_undegraded":"2024-04-26T21:54:11.912376+0000","last_fullsized":"2024-04-26T21:54:11.912376+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:44:16.323430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,7],"acting":[8,9,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"87'7","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:12.396927+0000","last_change":"2024-04-26T21:53:10.938377+0000","last_active":"2024-04-26T21:54:12.396927+0000","last_peered":"2024-04-26T21:54:12.396927+0000","last_clean":"2024-04-26T21:54:12.396927+0000","last_became_active":"2024-04-26T21:53:10.921998+0000","last_became_peered":"2024-04-26T21:53:10.921998+0000","last_unstale":"2024-04-26T21:54:12.396927+0000","last_undegraded":"2024-04-26T21:54:12.396927+0000","last_fullsized":"2024-04-26T21:54:12.396927+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:51:20.389344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,6],"acting":[10,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.919025+0000","last_change":"2024-04-26T21:53:10.931444+0000","last_active":"2024-04-26T21:54:11.919025+0000","last_peered":"2024-04-26T21:54:11.919025+0000","last_clean":"2024-04-26T21:54:11.919025+0000","last_became_active":"2024-04-26T21:53:10.930761+0000","last_became_peered":"2024-04-26T21:53:10.930761+0000","last_unstale":"2024-04-26T21:54:11.919025+0000","last_undegraded":"2024-04-26T21:54:11.919025+0000","last_fullsized":"2024-04-26T21:54:11.919025+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:56:17.803811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,3],"acting":[5,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"87'3","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:12.395254+0000","last_change":"2024-04-26T21:52:50.107564+0000","last_active":"2024-04-26T21:54:12.395254+0000","last_peered":"2024-04-26T21:54:12.395254+0000","last_clean":"2024-04-26T21:54:12.395254+0000","last_became_active":"2024-04-26T21:52:50.106653+0000","last_became_peered":"2024-04-26T21:52:50.106653+0000","last_unstale":"2024-04-26T21:54:12.395254+0000","last_undegraded":"2024-04-26T21:54:12.395254+0000","last_fullsized":"2024-04-26T21:54:12.395254+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:05:36.594729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.907525+0000","last_change":"2024-04-26T21:52:50.780279+0000","last_active":"2024-04-26T21:54:11.907525+0000","last_peered":"2024-04-26T21:54:11.907525+0000","last_clean":"2024-04-26T21:54:11.907525+0000","last_became_active":"2024-04-26T21:52:50.779950+0000","last_became_peered":"2024-04-26T21:52:50.779950+0000","last_unstale":"2024-04-26T21:54:11.907525+0000","last_undegraded":"2024-04-26T21:54:11.907525+0000","last_fullsized":"2024-04-26T21:54:11.907525+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:11:36.282319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"87'3","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.917578+0000","last_change":"2024-04-26T21:52:50.106269+0000","last_active":"2024-04-26T21:54:11.917578+0000","last_peered":"2024-04-26T21:54:11.917578+0000","last_clean":"2024-04-26T21:54:11.917578+0000","last_became_active":"2024-04-26T21:52:50.105948+0000","last_became_peered":"2024-04-26T21:52:50.105948+0000","last_unstale":"2024-04-26T21:54:11.917578+0000","last_undegraded":"2024-04-26T21:54:11.917578+0000","last_fullsized":"2024-04-26T21:54:11.917578+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:39:31.024362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.905324+0000","last_change":"2024-04-26T21:52:50.778938+0000","last_active":"2024-04-26T21:54:11.905324+0000","last_peered":"2024-04-26T21:54:11.905324+0000","last_clean":"2024-04-26T21:54:11.905324+0000","last_became_active":"2024-04-26T21:52:50.778590+0000","last_became_peered":"2024-04-26T21:52:50.778590+0000","last_unstale":"2024-04-26T21:54:11.905324+0000","last_undegraded":"2024-04-26T21:54:11.905324+0000","last_fullsized":"2024-04-26T21:54:11.905324+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:03:10.325793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"91'6","reported_seq":320,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:55:22.948842+0000","last_change":"2024-04-26T21:52:50.107243+0000","last_active":"2024-04-26T21:55:22.948842+0000","last_peered":"2024-04-26T21:55:22.948842+0000","last_clean":"2024-04-26T21:55:22.948842+0000","last_became_active":"2024-04-26T21:52:50.106729+0000","last_became_peered":"2024-04-26T21:52:50.106729+0000","last_unstale":"2024-04-26T21:55:22.948842+0000","last_undegraded":"2024-04-26T21:55:22.948842+0000","last_fullsized":"2024-04-26T21:55:22.948842+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:28:11.197997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":290,"num_read_kb":290,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.909345+0000","last_change":"2024-04-26T21:52:50.776906+0000","last_active":"2024-04-26T21:54:11.909345+0000","last_peered":"2024-04-26T21:54:11.909345+0000","last_clean":"2024-04-26T21:54:11.909345+0000","last_became_active":"2024-04-26T21:52:50.776567+0000","last_became_peered":"2024-04-26T21:52:50.776567+0000","last_unstale":"2024-04-26T21:54:11.909345+0000","last_undegraded":"2024-04-26T21:54:11.909345+0000","last_fullsized":"2024-04-26T21:54:11.909345+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:51:39.512572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"87'4","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.923341+0000","last_change":"2024-04-26T21:52:50.109792+0000","last_active":"2024-04-26T21:54:11.923341+0000","last_peered":"2024-04-26T21:54:11.923341+0000","last_clean":"2024-04-26T21:54:11.923341+0000","last_became_active":"2024-04-26T21:52:50.106662+0000","last_became_peered":"2024-04-26T21:52:50.106662+0000","last_unstale":"2024-04-26T21:54:11.923341+0000","last_undegraded":"2024-04-26T21:54:11.923341+0000","last_fullsized":"2024-04-26T21:54:11.923341+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:41:27.390780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:12.034410+0000","last_change":"2024-04-26T21:52:50.782798+0000","last_active":"2024-04-26T21:54:12.034410+0000","last_peered":"2024-04-26T21:54:12.034410+0000","last_clean":"2024-04-26T21:54:12.034410+0000","last_became_active":"2024-04-26T21:52:50.781975+0000","last_became_peered":"2024-04-26T21:52:50.781975+0000","last_unstale":"2024-04-26T21:54:12.034410+0000","last_undegraded":"2024-04-26T21:54:12.034410+0000","last_fullsized":"2024-04-26T21:54:12.034410+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:13:07.261095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"87'5","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.917206+0000","last_change":"2024-04-26T21:52:50.099296+0000","last_active":"2024-04-26T21:54:11.917206+0000","last_peered":"2024-04-26T21:54:11.917206+0000","last_clean":"2024-04-26T21:54:11.917206+0000","last_became_active":"2024-04-26T21:52:50.098940+0000","last_became_peered":"2024-04-26T21:52:50.098940+0000","last_unstale":"2024-04-26T21:54:11.917206+0000","last_undegraded":"2024-04-26T21:54:11.917206+0000","last_fullsized":"2024-04-26T21:54:11.917206+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:49:15.765474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.909628+0000","last_change":"2024-04-26T21:52:50.783341+0000","last_active":"2024-04-26T21:54:11.909628+0000","last_peered":"2024-04-26T21:54:11.909628+0000","last_clean":"2024-04-26T21:54:11.909628+0000","last_became_active":"2024-04-26T21:52:50.783025+0000","last_became_peered":"2024-04-26T21:52:50.783025+0000","last_unstale":"2024-04-26T21:54:11.909628+0000","last_undegraded":"2024-04-26T21:54:11.909628+0000","last_fullsized":"2024-04-26T21:54:11.909628+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:54:06.070550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"91'31","reported_seq":352,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:55:23.155989+0000","last_change":"2024-04-26T21:52:50.106617+0000","last_active":"2024-04-26T21:55:23.155989+0000","last_peered":"2024-04-26T21:55:23.155989+0000","last_clean":"2024-04-26T21:55:23.155989+0000","last_became_active":"2024-04-26T21:52:50.106161+0000","last_became_peered":"2024-04-26T21:52:50.106161+0000","last_unstale":"2024-04-26T21:55:23.155989+0000","last_undegraded":"2024-04-26T21:55:23.155989+0000","last_fullsized":"2024-04-26T21:55:23.155989+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:52:07.977749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":51444,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":297,"num_read_kb":199,"num_write":33,"num_write_kb":61,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.912969+0000","last_change":"2024-04-26T21:52:50.794567+0000","last_active":"2024-04-26T21:54:11.912969+0000","last_peered":"2024-04-26T21:54:11.912969+0000","last_clean":"2024-04-26T21:54:11.912969+0000","last_became_active":"2024-04-26T21:52:50.794245+0000","last_became_peered":"2024-04-26T21:52:50.794245+0000","last_unstale":"2024-04-26T21:54:11.912969+0000","last_undegraded":"2024-04-26T21:54:11.912969+0000","last_fullsized":"2024-04-26T21:54:11.912969+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:58:11.501104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"87'9","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.910302+0000","last_change":"2024-04-26T21:52:50.102156+0000","last_active":"2024-04-26T21:54:11.910302+0000","last_peered":"2024-04-26T21:54:11.910302+0000","last_clean":"2024-04-26T21:54:11.910302+0000","last_became_active":"2024-04-26T21:52:50.101721+0000","last_became_peered":"2024-04-26T21:52:50.101721+0000","last_unstale":"2024-04-26T21:54:11.910302+0000","last_undegraded":"2024-04-26T21:54:11.910302+0000","last_fullsized":"2024-04-26T21:54:11.910302+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:48:42.081874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:12.395808+0000","last_change":"2024-04-26T21:52:50.789199+0000","last_active":"2024-04-26T21:54:12.395808+0000","last_peered":"2024-04-26T21:54:12.395808+0000","last_clean":"2024-04-26T21:54:12.395808+0000","last_became_active":"2024-04-26T21:52:50.788868+0000","last_became_peered":"2024-04-26T21:52:50.788868+0000","last_unstale":"2024-04-26T21:54:12.395808+0000","last_undegraded":"2024-04-26T21:54:12.395808+0000","last_fullsized":"2024-04-26T21:54:12.395808+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:29:55.755564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"87'5","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.918826+0000","last_change":"2024-04-26T21:52:50.104920+0000","last_active":"2024-04-26T21:54:11.918826+0000","last_peered":"2024-04-26T21:54:11.918826+0000","last_clean":"2024-04-26T21:54:11.918826+0000","last_became_active":"2024-04-26T21:52:50.104579+0000","last_became_peered":"2024-04-26T21:52:50.104579+0000","last_unstale":"2024-04-26T21:54:11.918826+0000","last_undegraded":"2024-04-26T21:54:11.918826+0000","last_fullsized":"2024-04-26T21:54:11.918826+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:09:35.760918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.923423+0000","last_change":"2024-04-26T21:52:50.787016+0000","last_active":"2024-04-26T21:54:11.923423+0000","last_peered":"2024-04-26T21:54:11.923423+0000","last_clean":"2024-04-26T21:54:11.923423+0000","last_became_active":"2024-04-26T21:52:50.778439+0000","last_became_peered":"2024-04-26T21:52:50.778439+0000","last_unstale":"2024-04-26T21:54:11.923423+0000","last_undegraded":"2024-04-26T21:54:11.923423+0000","last_fullsized":"2024-04-26T21:54:11.923423+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:13:35.735480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"91'6","reported_seq":179,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:55:22.964012+0000","last_change":"2024-04-26T21:52:50.110175+0000","last_active":"2024-04-26T21:55:22.964012+0000","last_peered":"2024-04-26T21:55:22.964012+0000","last_clean":"2024-04-26T21:55:22.964012+0000","last_became_active":"2024-04-26T21:52:50.109779+0000","last_became_peered":"2024-04-26T21:52:50.109779+0000","last_unstale":"2024-04-26T21:55:22.964012+0000","last_undegraded":"2024-04-26T21:55:22.964012+0000","last_fullsized":"2024-04-26T21:55:22.964012+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:06:46.756327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1081,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":150,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.922565+0000","last_change":"2024-04-26T21:52:50.784199+0000","last_active":"2024-04-26T21:54:11.922565+0000","last_peered":"2024-04-26T21:54:11.922565+0000","last_clean":"2024-04-26T21:54:11.922565+0000","last_became_active":"2024-04-26T21:52:50.783820+0000","last_became_peered":"2024-04-26T21:52:50.783820+0000","last_unstale":"2024-04-26T21:54:11.922565+0000","last_undegraded":"2024-04-26T21:54:11.922565+0000","last_fullsized":"2024-04-26T21:54:11.922565+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:45:24.776722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"87'17","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.912743+0000","last_change":"2024-04-26T21:52:50.095685+0000","last_active":"2024-04-26T21:54:11.912743+0000","last_peered":"2024-04-26T21:54:11.912743+0000","last_clean":"2024-04-26T21:54:11.912743+0000","last_became_active":"2024-04-26T21:52:50.095298+0000","last_became_peered":"2024-04-26T21:52:50.095298+0000","last_unstale":"2024-04-26T21:54:11.912743+0000","last_undegraded":"2024-04-26T21:54:11.912743+0000","last_fullsized":"2024-04-26T21:54:11.912743+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:47:26.061531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.917867+0000","last_change":"2024-04-26T21:52:50.786607+0000","last_active":"2024-04-26T21:54:11.917867+0000","last_peered":"2024-04-26T21:54:11.917867+0000","last_clean":"2024-04-26T21:54:11.917867+0000","last_became_active":"2024-04-26T21:52:50.786261+0000","last_became_peered":"2024-04-26T21:52:50.786261+0000","last_unstale":"2024-04-26T21:54:11.917867+0000","last_undegraded":"2024-04-26T21:54:11.917867+0000","last_fullsized":"2024-04-26T21:54:11.917867+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:08:45.902260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"91'3","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:55:08.888181+0000","last_change":"2024-04-26T21:52:50.095782+0000","last_active":"2024-04-26T21:55:08.888181+0000","last_peered":"2024-04-26T21:55:08.888181+0000","last_clean":"2024-04-26T21:55:08.888181+0000","last_became_active":"2024-04-26T21:52:50.093983+0000","last_became_peered":"2024-04-26T21:52:50.093983+0000","last_unstale":"2024-04-26T21:55:08.888181+0000","last_undegraded":"2024-04-26T21:55:08.888181+0000","last_fullsized":"2024-04-26T21:55:08.888181+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:18:40.103099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.921290+0000","last_change":"2024-04-26T21:52:50.791969+0000","last_active":"2024-04-26T21:54:11.921290+0000","last_peered":"2024-04-26T21:54:11.921290+0000","last_clean":"2024-04-26T21:54:11.921290+0000","last_became_active":"2024-04-26T21:52:50.790441+0000","last_became_peered":"2024-04-26T21:52:50.790441+0000","last_unstale":"2024-04-26T21:54:11.921290+0000","last_undegraded":"2024-04-26T21:54:11.921290+0000","last_fullsized":"2024-04-26T21:54:11.921290+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:11:04.964439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"87'1","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.905428+0000","last_change":"2024-04-26T21:52:50.101807+0000","last_active":"2024-04-26T21:54:11.905428+0000","last_peered":"2024-04-26T21:54:11.905428+0000","last_clean":"2024-04-26T21:54:11.905428+0000","last_became_active":"2024-04-26T21:52:50.101329+0000","last_became_peered":"2024-04-26T21:52:50.101329+0000","last_unstale":"2024-04-26T21:54:11.905428+0000","last_undegraded":"2024-04-26T21:54:11.905428+0000","last_fullsized":"2024-04-26T21:54:11.905428+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:54:47.788172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"87'1","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:12.397681+0000","last_change":"2024-04-26T21:52:50.780872+0000","last_active":"2024-04-26T21:54:12.397681+0000","last_peered":"2024-04-26T21:54:12.397681+0000","last_clean":"2024-04-26T21:54:12.397681+0000","last_became_active":"2024-04-26T21:52:50.778757+0000","last_became_peered":"2024-04-26T21:52:50.778757+0000","last_unstale":"2024-04-26T21:54:12.397681+0000","last_undegraded":"2024-04-26T21:54:12.397681+0000","last_fullsized":"2024-04-26T21:54:12.397681+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:07:21.595091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"91'6","reported_seq":176,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:55:20.923377+0000","last_change":"2024-04-26T21:52:50.106886+0000","last_active":"2024-04-26T21:55:20.923377+0000","last_peered":"2024-04-26T21:55:20.923377+0000","last_clean":"2024-04-26T21:55:20.923377+0000","last_became_active":"2024-04-26T21:52:50.106150+0000","last_became_peered":"2024-04-26T21:52:50.106150+0000","last_unstale":"2024-04-26T21:55:20.923377+0000","last_undegraded":"2024-04-26T21:55:20.923377+0000","last_fullsized":"2024-04-26T21:55:20.923377+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:20:54.920416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1081,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":146,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:12.397270+0000","last_change":"2024-04-26T21:53:11.069186+0000","last_active":"2024-04-26T21:54:12.397270+0000","last_peered":"2024-04-26T21:54:12.397270+0000","last_clean":"2024-04-26T21:54:12.397270+0000","last_became_active":"2024-04-26T21:53:11.068418+0000","last_became_peered":"2024-04-26T21:53:11.068418+0000","last_unstale":"2024-04-26T21:54:12.397270+0000","last_undegraded":"2024-04-26T21:54:12.397270+0000","last_fullsized":"2024-04-26T21:54:12.397270+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:57:59.663173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.917799+0000","last_change":"2024-04-26T21:52:50.785257+0000","last_active":"2024-04-26T21:54:11.917799+0000","last_peered":"2024-04-26T21:54:11.917799+0000","last_clean":"2024-04-26T21:54:11.917799+0000","last_became_active":"2024-04-26T21:52:50.784906+0000","last_became_peered":"2024-04-26T21:52:50.784906+0000","last_unstale":"2024-04-26T21:54:11.917799+0000","last_undegraded":"2024-04-26T21:54:11.917799+0000","last_fullsized":"2024-04-26T21:54:11.917799+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:04:50.921663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"91'6","reported_seq":176,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:55:21.959779+0000","last_change":"2024-04-26T21:52:50.102425+0000","last_active":"2024-04-26T21:55:21.959779+0000","last_peered":"2024-04-26T21:55:21.959779+0000","last_clean":"2024-04-26T21:55:21.959779+0000","last_became_active":"2024-04-26T21:52:50.101703+0000","last_became_peered":"2024-04-26T21:52:50.101703+0000","last_unstale":"2024-04-26T21:55:21.959779+0000","last_undegraded":"2024-04-26T21:55:21.959779+0000","last_fullsized":"2024-04-26T21:55:21.959779+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:13:39.910519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":146,"num_read_kb":146,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.907737+0000","last_change":"2024-04-26T21:52:50.775036+0000","last_active":"2024-04-26T21:54:11.907737+0000","last_peered":"2024-04-26T21:54:11.907737+0000","last_clean":"2024-04-26T21:54:11.907737+0000","last_became_active":"2024-04-26T21:52:50.773347+0000","last_became_peered":"2024-04-26T21:52:50.773347+0000","last_unstale":"2024-04-26T21:54:11.907737+0000","last_undegraded":"2024-04-26T21:54:11.907737+0000","last_fullsized":"2024-04-26T21:54:11.907737+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:59:55.303014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"87'6","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.934310+0000","last_change":"2024-04-26T21:52:50.109437+0000","last_active":"2024-04-26T21:54:11.934310+0000","last_peered":"2024-04-26T21:54:11.934310+0000","last_clean":"2024-04-26T21:54:11.934310+0000","last_became_active":"2024-04-26T21:52:50.108605+0000","last_became_peered":"2024-04-26T21:52:50.108605+0000","last_unstale":"2024-04-26T21:54:11.934310+0000","last_undegraded":"2024-04-26T21:54:11.934310+0000","last_fullsized":"2024-04-26T21:54:11.934310+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:31:04.208334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":350,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.922023+0000","last_change":"2024-04-26T21:51:11.498577+0000","last_active":"2024-04-26T21:54:11.922023+0000","last_peered":"2024-04-26T21:54:11.922023+0000","last_clean":"2024-04-26T21:54:11.922023+0000","last_became_active":"2024-04-26T21:51:11.497422+0000","last_became_peered":"2024-04-26T21:51:11.497422+0000","last_unstale":"2024-04-26T21:54:11.922023+0000","last_undegraded":"2024-04-26T21:54:11.922023+0000","last_fullsized":"2024-04-26T21:54:11.922023+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:46:29.789400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:46:29.789400+0000","last_clean_scrub_stamp":"2024-04-26T21:46:29.789400+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:54:13.450359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:12.395065+0000","last_change":"2024-04-26T21:52:50.794270+0000","last_active":"2024-04-26T21:54:12.395065+0000","last_peered":"2024-04-26T21:54:12.395065+0000","last_clean":"2024-04-26T21:54:12.395065+0000","last_became_active":"2024-04-26T21:52:50.793966+0000","last_became_peered":"2024-04-26T21:52:50.793966+0000","last_unstale":"2024-04-26T21:54:12.395065+0000","last_undegraded":"2024-04-26T21:54:12.395065+0000","last_fullsized":"2024-04-26T21:54:12.395065+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:01:37.668466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"87'4","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.922067+0000","last_change":"2024-04-26T21:52:50.102700+0000","last_active":"2024-04-26T21:54:11.922067+0000","last_peered":"2024-04-26T21:54:11.922067+0000","last_clean":"2024-04-26T21:54:11.922067+0000","last_became_active":"2024-04-26T21:52:50.101267+0000","last_became_peered":"2024-04-26T21:52:50.101267+0000","last_unstale":"2024-04-26T21:54:11.922067+0000","last_undegraded":"2024-04-26T21:54:11.922067+0000","last_fullsized":"2024-04-26T21:54:11.922067+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:41:21.727411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"87'4","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.914022+0000","last_change":"2024-04-26T21:52:50.102191+0000","last_active":"2024-04-26T21:54:11.914022+0000","last_peered":"2024-04-26T21:54:11.914022+0000","last_clean":"2024-04-26T21:54:11.914022+0000","last_became_active":"2024-04-26T21:52:50.101307+0000","last_became_peered":"2024-04-26T21:52:50.101307+0000","last_unstale":"2024-04-26T21:54:11.914022+0000","last_undegraded":"2024-04-26T21:54:11.914022+0000","last_fullsized":"2024-04-26T21:54:11.914022+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:59:12.465291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:12.395488+0000","last_change":"2024-04-26T21:52:50.792020+0000","last_active":"2024-04-26T21:54:12.395488+0000","last_peered":"2024-04-26T21:54:12.395488+0000","last_clean":"2024-04-26T21:54:12.395488+0000","last_became_active":"2024-04-26T21:52:50.791677+0000","last_became_peered":"2024-04-26T21:52:50.791677+0000","last_unstale":"2024-04-26T21:54:12.395488+0000","last_undegraded":"2024-04-26T21:54:12.395488+0000","last_fullsized":"2024-04-26T21:54:12.395488+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:11:42.218294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"91'13","reported_seq":53,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:59.673130+0000","last_change":"2024-04-26T21:54:11.936807+0000","last_active":"2024-04-26T21:54:59.673130+0000","last_peered":"2024-04-26T21:54:59.673130+0000","last_clean":"2024-04-26T21:54:59.673130+0000","last_became_active":"2024-04-26T21:54:11.923482+0000","last_became_peered":"2024-04-26T21:54:11.923482+0000","last_unstale":"2024-04-26T21:54:59.673130+0000","last_undegraded":"2024-04-26T21:54:59.673130+0000","last_fullsized":"2024-04-26T21:54:59.673130+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:15:46.945453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.908090+0000","last_change":"2024-04-26T21:52:50.779775+0000","last_active":"2024-04-26T21:54:11.908090+0000","last_peered":"2024-04-26T21:54:11.908090+0000","last_clean":"2024-04-26T21:54:11.908090+0000","last_became_active":"2024-04-26T21:52:50.778857+0000","last_became_peered":"2024-04-26T21:52:50.778857+0000","last_unstale":"2024-04-26T21:54:11.908090+0000","last_undegraded":"2024-04-26T21:54:11.908090+0000","last_fullsized":"2024-04-26T21:54:11.908090+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:59:50.946578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"87'7","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.934470+0000","last_change":"2024-04-26T21:52:50.113027+0000","last_active":"2024-04-26T21:54:11.934470+0000","last_peered":"2024-04-26T21:54:11.934470+0000","last_clean":"2024-04-26T21:54:11.934470+0000","last_became_active":"2024-04-26T21:52:50.112637+0000","last_became_peered":"2024-04-26T21:52:50.112637+0000","last_unstale":"2024-04-26T21:54:11.934470+0000","last_undegraded":"2024-04-26T21:54:11.934470+0000","last_fullsized":"2024-04-26T21:54:11.934470+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:09:47.798184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:12.034220+0000","last_change":"2024-04-26T21:52:50.785119+0000","last_active":"2024-04-26T21:54:12.034220+0000","last_peered":"2024-04-26T21:54:12.034220+0000","last_clean":"2024-04-26T21:54:12.034220+0000","last_became_active":"2024-04-26T21:52:50.781005+0000","last_became_peered":"2024-04-26T21:52:50.781005+0000","last_unstale":"2024-04-26T21:54:12.034220+0000","last_undegraded":"2024-04-26T21:54:12.034220+0000","last_fullsized":"2024-04-26T21:54:12.034220+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:16:32.986810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"87'2","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.908468+0000","last_change":"2024-04-26T21:52:50.112263+0000","last_active":"2024-04-26T21:54:11.908468+0000","last_peered":"2024-04-26T21:54:11.908468+0000","last_clean":"2024-04-26T21:54:11.908468+0000","last_became_active":"2024-04-26T21:52:50.110649+0000","last_became_peered":"2024-04-26T21:52:50.110649+0000","last_unstale":"2024-04-26T21:54:11.908468+0000","last_undegraded":"2024-04-26T21:54:11.908468+0000","last_fullsized":"2024-04-26T21:54:11.908468+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:14:23.245000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.913095+0000","last_change":"2024-04-26T21:52:50.790674+0000","last_active":"2024-04-26T21:54:11.913095+0000","last_peered":"2024-04-26T21:54:11.913095+0000","last_clean":"2024-04-26T21:54:11.913095+0000","last_became_active":"2024-04-26T21:52:50.790286+0000","last_became_peered":"2024-04-26T21:52:50.790286+0000","last_unstale":"2024-04-26T21:54:11.913095+0000","last_undegraded":"2024-04-26T21:54:11.913095+0000","last_fullsized":"2024-04-26T21:54:11.913095+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:22:32.246321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"87'7","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.907591+0000","last_change":"2024-04-26T21:52:50.112158+0000","last_active":"2024-04-26T21:54:11.907591+0000","last_peered":"2024-04-26T21:54:11.907591+0000","last_clean":"2024-04-26T21:54:11.907591+0000","last_became_active":"2024-04-26T21:52:50.111453+0000","last_became_peered":"2024-04-26T21:52:50.111453+0000","last_unstale":"2024-04-26T21:54:11.907591+0000","last_undegraded":"2024-04-26T21:54:11.907591+0000","last_fullsized":"2024-04-26T21:54:11.907591+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:16:58.626985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.915638+0000","last_change":"2024-04-26T21:52:50.798241+0000","last_active":"2024-04-26T21:54:11.915638+0000","last_peered":"2024-04-26T21:54:11.915638+0000","last_clean":"2024-04-26T21:54:11.915638+0000","last_became_active":"2024-04-26T21:52:50.797878+0000","last_became_peered":"2024-04-26T21:52:50.797878+0000","last_unstale":"2024-04-26T21:54:11.915638+0000","last_undegraded":"2024-04-26T21:54:11.915638+0000","last_fullsized":"2024-04-26T21:54:11.915638+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:42:49.363849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"91'5","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:55:20.595077+0000","last_change":"2024-04-26T21:54:11.946852+0000","last_active":"2024-04-26T21:55:20.595077+0000","last_peered":"2024-04-26T21:55:20.595077+0000","last_clean":"2024-04-26T21:55:20.595077+0000","last_became_active":"2024-04-26T21:54:11.923399+0000","last_became_peered":"2024-04-26T21:54:11.923399+0000","last_unstale":"2024-04-26T21:55:20.595077+0000","last_undegraded":"2024-04-26T21:55:20.595077+0000","last_fullsized":"2024-04-26T21:55:20.595077+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:05:34.688374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.908206+0000","last_change":"2024-04-26T21:52:50.776333+0000","last_active":"2024-04-26T21:54:11.908206+0000","last_peered":"2024-04-26T21:54:11.908206+0000","last_clean":"2024-04-26T21:54:11.908206+0000","last_became_active":"2024-04-26T21:52:50.776001+0000","last_became_peered":"2024-04-26T21:52:50.776001+0000","last_unstale":"2024-04-26T21:54:11.908206+0000","last_undegraded":"2024-04-26T21:54:11.908206+0000","last_fullsized":"2024-04-26T21:54:11.908206+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:38:26.578928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"87'4","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.913422+0000","last_change":"2024-04-26T21:52:50.106008+0000","last_active":"2024-04-26T21:54:11.913422+0000","last_peered":"2024-04-26T21:54:11.913422+0000","last_clean":"2024-04-26T21:54:11.913422+0000","last_became_active":"2024-04-26T21:52:50.105565+0000","last_became_peered":"2024-04-26T21:52:50.105565+0000","last_unstale":"2024-04-26T21:54:11.913422+0000","last_undegraded":"2024-04-26T21:54:11.913422+0000","last_fullsized":"2024-04-26T21:54:11.913422+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:15:17.355461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.923394+0000","last_change":"2024-04-26T21:52:50.776051+0000","last_active":"2024-04-26T21:54:11.923394+0000","last_peered":"2024-04-26T21:54:11.923394+0000","last_clean":"2024-04-26T21:54:11.923394+0000","last_became_active":"2024-04-26T21:52:50.775716+0000","last_became_peered":"2024-04-26T21:52:50.775716+0000","last_unstale":"2024-04-26T21:54:11.923394+0000","last_undegraded":"2024-04-26T21:54:11.923394+0000","last_fullsized":"2024-04-26T21:54:11.923394+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:34:21.110773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"91'7","reported_seq":180,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:55:22.950112+0000","last_change":"2024-04-26T21:52:50.111280+0000","last_active":"2024-04-26T21:55:22.950112+0000","last_peered":"2024-04-26T21:55:22.950112+0000","last_clean":"2024-04-26T21:55:22.950112+0000","last_became_active":"2024-04-26T21:52:50.110972+0000","last_became_peered":"2024-04-26T21:52:50.110972+0000","last_unstale":"2024-04-26T21:55:22.950112+0000","last_undegraded":"2024-04-26T21:55:22.950112+0000","last_fullsized":"2024-04-26T21:55:22.950112+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:41:48.203134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2054,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":149,"num_read_kb":3,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.923325+0000","last_change":"2024-04-26T21:52:50.777488+0000","last_active":"2024-04-26T21:54:11.923325+0000","last_peered":"2024-04-26T21:54:11.923325+0000","last_clean":"2024-04-26T21:54:11.923325+0000","last_became_active":"2024-04-26T21:52:50.777162+0000","last_became_peered":"2024-04-26T21:52:50.777162+0000","last_unstale":"2024-04-26T21:54:11.923325+0000","last_undegraded":"2024-04-26T21:54:11.923325+0000","last_fullsized":"2024-04-26T21:54:11.923325+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:07:22.683109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"87'3","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.916669+0000","last_change":"2024-04-26T21:52:50.104508+0000","last_active":"2024-04-26T21:54:11.916669+0000","last_peered":"2024-04-26T21:54:11.916669+0000","last_clean":"2024-04-26T21:54:11.916669+0000","last_became_active":"2024-04-26T21:52:50.103924+0000","last_became_peered":"2024-04-26T21:52:50.103924+0000","last_unstale":"2024-04-26T21:54:11.916669+0000","last_undegraded":"2024-04-26T21:54:11.916669+0000","last_fullsized":"2024-04-26T21:54:11.916669+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:39:25.861339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"87'1","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.922627+0000","last_change":"2024-04-26T21:52:50.782915+0000","last_active":"2024-04-26T21:54:11.922627+0000","last_peered":"2024-04-26T21:54:11.922627+0000","last_clean":"2024-04-26T21:54:11.922627+0000","last_became_active":"2024-04-26T21:52:50.782253+0000","last_became_peered":"2024-04-26T21:52:50.782253+0000","last_unstale":"2024-04-26T21:54:11.922627+0000","last_undegraded":"2024-04-26T21:54:11.922627+0000","last_fullsized":"2024-04-26T21:54:11.922627+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:12:51.624162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"87'5","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:12.395358+0000","last_change":"2024-04-26T21:53:11.590613+0000","last_active":"2024-04-26T21:54:12.395358+0000","last_peered":"2024-04-26T21:54:12.395358+0000","last_clean":"2024-04-26T21:54:12.395358+0000","last_became_active":"2024-04-26T21:53:11.578727+0000","last_became_peered":"2024-04-26T21:53:11.578727+0000","last_unstale":"2024-04-26T21:54:12.395358+0000","last_undegraded":"2024-04-26T21:54:12.395358+0000","last_fullsized":"2024-04-26T21:54:12.395358+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:44:38.867231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.922922+0000","last_change":"2024-04-26T21:52:50.787777+0000","last_active":"2024-04-26T21:54:11.922922+0000","last_peered":"2024-04-26T21:54:11.922922+0000","last_clean":"2024-04-26T21:54:11.922922+0000","last_became_active":"2024-04-26T21:52:50.782403+0000","last_became_peered":"2024-04-26T21:52:50.782403+0000","last_unstale":"2024-04-26T21:54:11.922922+0000","last_undegraded":"2024-04-26T21:54:11.922922+0000","last_fullsized":"2024-04-26T21:54:11.922922+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:28:58.425802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.915961+0000","last_change":"2024-04-26T21:52:50.800010+0000","last_active":"2024-04-26T21:54:11.915961+0000","last_peered":"2024-04-26T21:54:11.915961+0000","last_clean":"2024-04-26T21:54:11.915961+0000","last_became_active":"2024-04-26T21:52:50.799308+0000","last_became_peered":"2024-04-26T21:52:50.799308+0000","last_unstale":"2024-04-26T21:54:11.915961+0000","last_undegraded":"2024-04-26T21:54:11.915961+0000","last_fullsized":"2024-04-26T21:54:11.915961+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:54:25.772312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.909036+0000","last_change":"2024-04-26T21:52:50.798281+0000","last_active":"2024-04-26T21:54:11.909036+0000","last_peered":"2024-04-26T21:54:11.909036+0000","last_clean":"2024-04-26T21:54:11.909036+0000","last_became_active":"2024-04-26T21:52:50.795879+0000","last_became_peered":"2024-04-26T21:52:50.795879+0000","last_unstale":"2024-04-26T21:54:11.909036+0000","last_undegraded":"2024-04-26T21:54:11.909036+0000","last_fullsized":"2024-04-26T21:54:11.909036+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:36:00.769991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.908710+0000","last_change":"2024-04-26T21:52:50.791058+0000","last_active":"2024-04-26T21:54:11.908710+0000","last_peered":"2024-04-26T21:54:11.908710+0000","last_clean":"2024-04-26T21:54:11.908710+0000","last_became_active":"2024-04-26T21:52:50.790721+0000","last_became_peered":"2024-04-26T21:52:50.790721+0000","last_unstale":"2024-04-26T21:54:11.908710+0000","last_undegraded":"2024-04-26T21:54:11.908710+0000","last_fullsized":"2024-04-26T21:54:11.908710+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:56:03.239622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:12.033664+0000","last_change":"2024-04-26T21:52:50.788193+0000","last_active":"2024-04-26T21:54:12.033664+0000","last_peered":"2024-04-26T21:54:12.033664+0000","last_clean":"2024-04-26T21:54:12.033664+0000","last_became_active":"2024-04-26T21:52:50.786749+0000","last_became_peered":"2024-04-26T21:52:50.786749+0000","last_unstale":"2024-04-26T21:54:12.033664+0000","last_undegraded":"2024-04-26T21:54:12.033664+0000","last_fullsized":"2024-04-26T21:54:12.033664+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:22:11.277217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:12.395673+0000","last_change":"2024-04-26T21:52:50.782508+0000","last_active":"2024-04-26T21:54:12.395673+0000","last_peered":"2024-04-26T21:54:12.395673+0000","last_clean":"2024-04-26T21:54:12.395673+0000","last_became_active":"2024-04-26T21:52:50.781505+0000","last_became_peered":"2024-04-26T21:52:50.781505+0000","last_unstale":"2024-04-26T21:54:12.395673+0000","last_undegraded":"2024-04-26T21:54:12.395673+0000","last_fullsized":"2024-04-26T21:54:12.395673+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:22:57.867546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.916618+0000","last_change":"2024-04-26T21:52:50.789205+0000","last_active":"2024-04-26T21:54:11.916618+0000","last_peered":"2024-04-26T21:54:11.916618+0000","last_clean":"2024-04-26T21:54:11.916618+0000","last_became_active":"2024-04-26T21:52:50.788898+0000","last_became_peered":"2024-04-26T21:52:50.788898+0000","last_unstale":"2024-04-26T21:54:11.916618+0000","last_undegraded":"2024-04-26T21:54:11.916618+0000","last_fullsized":"2024-04-26T21:54:11.916618+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:18:51.401968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.909356+0000","last_change":"2024-04-26T21:52:50.793451+0000","last_active":"2024-04-26T21:54:11.909356+0000","last_peered":"2024-04-26T21:54:11.909356+0000","last_clean":"2024-04-26T21:54:11.909356+0000","last_became_active":"2024-04-26T21:52:50.791696+0000","last_became_peered":"2024-04-26T21:52:50.791696+0000","last_unstale":"2024-04-26T21:54:11.909356+0000","last_undegraded":"2024-04-26T21:54:11.909356+0000","last_fullsized":"2024-04-26T21:54:11.909356+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:51:43.543500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.907992+0000","last_change":"2024-04-26T21:53:10.919691+0000","last_active":"2024-04-26T21:54:11.907992+0000","last_peered":"2024-04-26T21:54:11.907992+0000","last_clean":"2024-04-26T21:54:11.907992+0000","last_became_active":"2024-04-26T21:53:10.919262+0000","last_became_peered":"2024-04-26T21:53:10.919262+0000","last_unstale":"2024-04-26T21:54:11.907992+0000","last_undegraded":"2024-04-26T21:54:11.907992+0000","last_fullsized":"2024-04-26T21:54:11.907992+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:15:35.719661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,8,2],"acting":[3,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:12.395203+0000","last_change":"2024-04-26T21:53:10.914854+0000","last_active":"2024-04-26T21:54:12.395203+0000","last_peered":"2024-04-26T21:54:12.395203+0000","last_clean":"2024-04-26T21:54:12.395203+0000","last_became_active":"2024-04-26T21:53:10.914458+0000","last_became_peered":"2024-04-26T21:53:10.914458+0000","last_unstale":"2024-04-26T21:54:12.395203+0000","last_undegraded":"2024-04-26T21:54:12.395203+0000","last_fullsized":"2024-04-26T21:54:12.395203+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:26:41.538053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,3],"acting":[6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.907533+0000","last_change":"2024-04-26T21:52:50.794580+0000","last_active":"2024-04-26T21:54:11.907533+0000","last_peered":"2024-04-26T21:54:11.907533+0000","last_clean":"2024-04-26T21:54:11.907533+0000","last_became_active":"2024-04-26T21:52:50.792398+0000","last_became_peered":"2024-04-26T21:52:50.792398+0000","last_unstale":"2024-04-26T21:54:11.907533+0000","last_undegraded":"2024-04-26T21:54:11.907533+0000","last_fullsized":"2024-04-26T21:54:11.907533+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:21:38.913644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.909622+0000","last_change":"2024-04-26T21:53:11.580576+0000","last_active":"2024-04-26T21:54:11.909622+0000","last_peered":"2024-04-26T21:54:11.909622+0000","last_clean":"2024-04-26T21:54:11.909622+0000","last_became_active":"2024-04-26T21:53:11.578632+0000","last_became_peered":"2024-04-26T21:53:11.578632+0000","last_unstale":"2024-04-26T21:54:11.909622+0000","last_undegraded":"2024-04-26T21:54:11.909622+0000","last_fullsized":"2024-04-26T21:54:11.909622+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:30:24.412417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,8],"acting":[1,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:12.396035+0000","last_change":"2024-04-26T21:52:50.792166+0000","last_active":"2024-04-26T21:54:12.396035+0000","last_peered":"2024-04-26T21:54:12.396035+0000","last_clean":"2024-04-26T21:54:12.396035+0000","last_became_active":"2024-04-26T21:52:50.790709+0000","last_became_peered":"2024-04-26T21:52:50.790709+0000","last_unstale":"2024-04-26T21:54:12.396035+0000","last_undegraded":"2024-04-26T21:54:12.396035+0000","last_fullsized":"2024-04-26T21:54:12.396035+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:05:48.200573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.915443+0000","last_change":"2024-04-26T21:53:10.930526+0000","last_active":"2024-04-26T21:54:11.915443+0000","last_peered":"2024-04-26T21:54:11.915443+0000","last_clean":"2024-04-26T21:54:11.915443+0000","last_became_active":"2024-04-26T21:53:10.928850+0000","last_became_peered":"2024-04-26T21:53:10.928850+0000","last_unstale":"2024-04-26T21:54:11.915443+0000","last_undegraded":"2024-04-26T21:54:11.915443+0000","last_fullsized":"2024-04-26T21:54:11.915443+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:54:03.420855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":60229,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1491,"num_read_kb":814,"num_write":255,"num_write_kb":137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":622,"num_keys_recovered":7,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":548864,"data_stored":189315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":218,"ondisk_log_size":218,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":82,"seq":352187318319,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1015808,"data_stored":485396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":74,"seq":317827579958,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28684,"kb_used_data":1668,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716468,"statfs":{"total":95995035648,"available":95965663232,"internally_reserved":0,"allocated":1708032,"data_stored":1127478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":69,"seq":296352743486,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36352,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708800,"statfs":{"total":95995035648,"available":95957811200,"internally_reserved":0,"allocated":1040384,"data_stored":484442,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":61,"seq":261993005126,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36348,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708804,"statfs":{"total":95995035648,"available":95957815296,"internally_reserved":0,"allocated":1036288,"data_stored":484820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299466,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28612,"kb_used_data":1596,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716540,"statfs":{"total":95995035648,"available":95965736960,"internally_reserved":0,"allocated":1634304,"data_stored":1078426,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462995,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36336,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708816,"statfs":{"total":95995035648,"available":95957827584,"internally_reserved":0,"allocated":1024000,"data_stored":484652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574965,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28068,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717084,"statfs":{"total":95995035648,"available":95966294016,"internally_reserved":0,"allocated":1077248,"data_stored":537009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738492,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36356,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708796,"statfs":{"total":95995035648,"available":95957807104,"internally_reserved":0,"allocated":1044480,"data_stored":486382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411439,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36400,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708752,"statfs":{"total":95995035648,"available":95957762048,"internally_reserved":0,"allocated":1089536,"data_stored":536998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182502,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1040384,"data_stored":483984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920866,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36344,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708808,"statfs":{"total":95995035648,"available":95957819392,"internally_reserved":0,"allocated":1032192,"data_stored":485628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724632,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36944,"kb_used_data":1608,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708208,"statfs":{"total":95995035648,"available":95957204992,"internally_reserved":0,"allocated":1646592,"data_stored":1077118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":58003,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":57992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2308,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1215,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4621,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":53673,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":2076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T21:55:27.861 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-04-26T21:55:27.862 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T21:55:28.078 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:28.077+0000 7f1ab4800640 1 -- 172.21.15.72:0/1883195178 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1aac1033e0 msgr2=0x7f1aac1057d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:28.078 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:28.077+0000 7f1ab4800640 1 --2- 172.21.15.72:0/1883195178 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1aac1033e0 0x7f1aac1057d0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f1aa0009900 tx=0x7f1aa002a3f0 comp rx=0 tx=0).stop 2024-04-26T21:55:28.078 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:28.077+0000 7f1ab4800640 1 -- 172.21.15.72:0/1883195178 shutdown_connections 2024-04-26T21:55:28.078 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:28.078+0000 7f1ab4800640 1 --2- 172.21.15.72:0/1883195178 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1aac108640 0x7f1aac10aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:28.078 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:28.078+0000 7f1ab4800640 1 --2- 172.21.15.72:0/1883195178 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1aac105d10 0x7f1aac108100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:28.078 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:28.078+0000 7f1ab4800640 1 --2- 172.21.15.72:0/1883195178 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1aac1033e0 0x7f1aac1057d0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:28.079 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:28.078+0000 7f1ab4800640 1 -- 172.21.15.72:0/1883195178 >> 172.21.15.72:0/1883195178 conn(0x7f1aac0fcdf0 msgr2=0x7f1aac0ff210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:55:28.079 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:28.078+0000 7f1ab4800640 1 -- 172.21.15.72:0/1883195178 shutdown_connections 2024-04-26T21:55:28.079 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:28.078+0000 7f1ab4800640 1 -- 172.21.15.72:0/1883195178 wait complete. 2024-04-26T21:55:28.081 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:28.079+0000 7f1ab4800640 1 Processor -- start 2024-04-26T21:55:28.081 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:28.079+0000 7f1ab4800640 1 -- start start 2024-04-26T21:55:28.081 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:28.079+0000 7f1ab4800640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1aac1033e0 0x7f1aac19ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:28.081 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:28.079+0000 7f1ab4800640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1aac105d10 0x7f1aac19b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:28.081 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:28.079+0000 7f1ab4800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1aac108640 0x7f1aac1a2290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:28.081 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:28.079+0000 7f1ab4800640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f1aac10e400 con 0x7f1aac105d10 2024-04-26T21:55:28.081 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:28.079+0000 7f1ab4800640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f1aac10e280 con 0x7f1aac1033e0 2024-04-26T21:55:28.082 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:28.079+0000 7f1ab4800640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f1aac10e580 con 0x7f1aac108640 2024-04-26T21:55:28.082 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:28.079+0000 7f1ab1400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1aac105d10 0x7f1aac19b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:28.082 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:28.079+0000 7f1ab1400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1aac105d10 0x7f1aac19b260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:44198/0 (socket says 172.21.15.72:44198) 2024-04-26T21:55:28.082 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:28.079+0000 7f1ab1400640 1 -- 172.21.15.72:0/1541060695 learned_addr learned my addr 172.21.15.72:0/1541060695 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:55:28.082 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:28.079+0000 7f1ab1e00640 1 --2- 172.21.15.72:0/1541060695 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1aac1033e0 0x7f1aac19ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:28.082 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:28.080+0000 7f1ab2800640 1 --2- 172.21.15.72:0/1541060695 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1aac108640 0x7f1aac1a2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:28.082 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:28.080+0000 7f1ab1400640 1 -- 172.21.15.72:0/1541060695 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1aac108640 msgr2=0x7f1aac1a2290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:28.082 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:28.080+0000 7f1ab1400640 1 --2- 172.21.15.72:0/1541060695 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1aac108640 0x7f1aac1a2290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:28.082 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:28.080+0000 7f1ab1400640 1 -- 172.21.15.72:0/1541060695 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1aac1033e0 msgr2=0x7f1aac19ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:28.082 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:28.080+0000 7f1ab1400640 1 --2- 172.21.15.72:0/1541060695 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1aac1033e0 0x7f1aac19ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:28.082 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:28.080+0000 7f1ab1400640 1 -- 172.21.15.72:0/1541060695 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1aac1a2930 con 0x7f1aac105d10 2024-04-26T21:55:28.082 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:28.080+0000 7f1ab2800640 1 --2- 172.21.15.72:0/1541060695 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1aac108640 0x7f1aac1a2290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:55:28.083 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:28.080+0000 7f1ab1400640 1 --2- 172.21.15.72:0/1541060695 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1aac105d10 0x7f1aac19b260 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f1a9c00e9c0 tx=0x7f1a9c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:55:28.083 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:28.080+0000 7f1aa6a00640 1 -- 172.21.15.72:0/1541060695 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a9c00cd80 con 0x7f1aac105d10 2024-04-26T21:55:28.083 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:28.080+0000 7f1aa6a00640 1 -- 172.21.15.72:0/1541060695 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1a9c0054b0 con 0x7f1aac105d10 2024-04-26T21:55:28.083 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:28.080+0000 7f1aa6a00640 1 -- 172.21.15.72:0/1541060695 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a9c010430 con 0x7f1aac105d10 2024-04-26T21:55:28.083 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:28.080+0000 7f1ab4800640 1 -- 172.21.15.72:0/1541060695 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1aac1a2c20 con 0x7f1aac105d10 2024-04-26T21:55:28.083 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:28.081+0000 7f1ab4800640 1 -- 172.21.15.72:0/1541060695 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1aac1a3190 con 0x7f1aac105d10 2024-04-26T21:55:28.083 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:28.082+0000 7f1ab4800640 1 -- 172.21.15.72:0/1541060695 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1aac10a130 con 0x7f1aac105d10 2024-04-26T21:55:28.083 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:28.083+0000 7f1aa6a00640 1 -- 172.21.15.72:0/1541060695 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f1a9c004f20 con 0x7f1aac105d10 2024-04-26T21:55:28.085 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:28.085+0000 7f1aa6a00640 1 --2- 172.21.15.72:0/1541060695 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f1a940a3cc0 0x7f1a940a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:28.085 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:28.085+0000 7f1aa6a00640 1 -- 172.21.15.72:0/1541060695 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7f1a9c0d0c60 con 0x7f1aac105d10 2024-04-26T21:55:28.085 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:28.085+0000 7f1ab1e00640 1 --2- 172.21.15.72:0/1541060695 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f1a940a3cc0 0x7f1a940a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:28.086 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:28.086+0000 7f1ab1e00640 1 --2- 172.21.15.72:0/1541060695 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f1a940a3cc0 0x7f1a940a6180 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f1aa0005ab0 tx=0x7f1aa0005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:55:28.089 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:28.089+0000 7f1aa6a00640 1 -- 172.21.15.72:0/1541060695 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f1a9c09d400 con 0x7f1aac105d10 2024-04-26T21:55:28.227 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:55:27 smithi164 ceph-mon[31916]: from='client.? 172.21.15.72:0/1184212258' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:55:28.294 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:28.294+0000 7f1ab4800640 1 -- 172.21.15.72:0/1541060695 --> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f1aac062d70 con 0x7f1a940a3cc0 2024-04-26T21:55:28.302 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:28.302+0000 7f1aa6a00640 1 -- 172.21.15.72:0/1541060695 <== mgr.14152 v2:172.21.15.72:6800/588150172 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258900 (secure 0 0 0) 0x7f1aac062d70 con 0x7f1a940a3cc0 2024-04-26T21:55:28.306 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T21:55:28.307 INFO:teuthology.orchestra.run.smithi072.stderr:dumped all 2024-04-26T21:55:28.309 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:28.308+0000 7f1ab4800640 1 -- 172.21.15.72:0/1541060695 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f1a940a3cc0 msgr2=0x7f1a940a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:28.309 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:28.308+0000 7f1ab4800640 1 --2- 172.21.15.72:0/1541060695 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f1a940a3cc0 0x7f1a940a6180 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f1aa0005ab0 tx=0x7f1aa0005a20 comp rx=0 tx=0).stop 2024-04-26T21:55:28.309 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:28.308+0000 7f1ab4800640 1 -- 172.21.15.72:0/1541060695 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1aac105d10 msgr2=0x7f1aac19b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:28.309 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:28.308+0000 7f1ab4800640 1 --2- 172.21.15.72:0/1541060695 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1aac105d10 0x7f1aac19b260 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f1a9c00e9c0 tx=0x7f1a9c00ee80 comp rx=0 tx=0).stop 2024-04-26T21:55:28.309 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:28.308+0000 7f1ab4800640 1 -- 172.21.15.72:0/1541060695 shutdown_connections 2024-04-26T21:55:28.309 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:28.308+0000 7f1ab4800640 1 --2- 172.21.15.72:0/1541060695 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f1a940a3cc0 0x7f1a940a6180 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:28.309 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:28.308+0000 7f1ab4800640 1 --2- 172.21.15.72:0/1541060695 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1aac108640 0x7f1aac1a2290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:28.309 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:28.308+0000 7f1ab4800640 1 --2- 172.21.15.72:0/1541060695 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1aac105d10 0x7f1aac19b260 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:28.309 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:28.308+0000 7f1ab4800640 1 --2- 172.21.15.72:0/1541060695 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1aac1033e0 0x7f1aac19ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:28.309 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:28.308+0000 7f1ab4800640 1 -- 172.21.15.72:0/1541060695 >> 172.21.15.72:0/1541060695 conn(0x7f1aac0fcdf0 msgr2=0x7f1aac102b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:55:28.310 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:28.308+0000 7f1ab4800640 1 -- 172.21.15.72:0/1541060695 shutdown_connections 2024-04-26T21:55:28.310 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:28.309+0000 7f1ab4800640 1 -- 172.21.15.72:0/1541060695 wait complete. 2024-04-26T21:55:28.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:55:27 smithi076 ceph-mon[31823]: from='client.? 172.21.15.72:0/1184212258' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:55:28.321 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:55:27 smithi072 ceph-mon[28138]: from='client.? 172.21.15.72:0/1184212258' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:55:28.326 INFO:teuthology.orchestra.run.smithi072.stdout:{"pg_ready":true,"pg_map":{"version":450,"stamp":"2024-04-26T21:55:27.837547+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":650831,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1622,"num_read_kb":923,"num_write":438,"num_write_kb":3077,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":1181358,"num_keys_recovered":7,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":357,"ondisk_log_size":357,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":396484,"kb_used_data":14052,"kb_used_omap":18,"kb_used_meta":382317,"kb_avail":1124545340,"statfs":{"total":1151940427776,"available":1151534428160,"internally_reserved":0,"allocated":14389248,"data_stored":7752333,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":391492922},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2465,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":110,"num_read_kb":55,"num_write":18,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.008817"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.921489+0000","last_change":"2024-04-26T21:52:50.787765+0000","last_active":"2024-04-26T21:54:11.921489+0000","last_peered":"2024-04-26T21:54:11.921489+0000","last_clean":"2024-04-26T21:54:11.921489+0000","last_became_active":"2024-04-26T21:52:50.786969+0000","last_became_peered":"2024-04-26T21:52:50.786969+0000","last_unstale":"2024-04-26T21:54:11.921489+0000","last_undegraded":"2024-04-26T21:54:11.921489+0000","last_fullsized":"2024-04-26T21:54:11.921489+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:31:50.659210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:12.034292+0000","last_change":"2024-04-26T21:52:50.779735+0000","last_active":"2024-04-26T21:54:12.034292+0000","last_peered":"2024-04-26T21:54:12.034292+0000","last_clean":"2024-04-26T21:54:12.034292+0000","last_became_active":"2024-04-26T21:52:50.779358+0000","last_became_peered":"2024-04-26T21:52:50.779358+0000","last_unstale":"2024-04-26T21:54:12.034292+0000","last_undegraded":"2024-04-26T21:54:12.034292+0000","last_fullsized":"2024-04-26T21:54:12.034292+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:58:45.631365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:12.033978+0000","last_change":"2024-04-26T21:52:50.788220+0000","last_active":"2024-04-26T21:54:12.033978+0000","last_peered":"2024-04-26T21:54:12.033978+0000","last_clean":"2024-04-26T21:54:12.033978+0000","last_became_active":"2024-04-26T21:52:50.787282+0000","last_became_peered":"2024-04-26T21:52:50.787282+0000","last_unstale":"2024-04-26T21:54:12.033978+0000","last_undegraded":"2024-04-26T21:54:12.033978+0000","last_fullsized":"2024-04-26T21:54:12.033978+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:12:28.577893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.934516+0000","last_change":"2024-04-26T21:52:50.795367+0000","last_active":"2024-04-26T21:54:11.934516+0000","last_peered":"2024-04-26T21:54:11.934516+0000","last_clean":"2024-04-26T21:54:11.934516+0000","last_became_active":"2024-04-26T21:52:50.793602+0000","last_became_peered":"2024-04-26T21:52:50.793602+0000","last_unstale":"2024-04-26T21:54:11.934516+0000","last_undegraded":"2024-04-26T21:54:11.934516+0000","last_fullsized":"2024-04-26T21:54:11.934516+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:37:23.356792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.912197+0000","last_change":"2024-04-26T21:52:50.787333+0000","last_active":"2024-04-26T21:54:11.912197+0000","last_peered":"2024-04-26T21:54:11.912197+0000","last_clean":"2024-04-26T21:54:11.912197+0000","last_became_active":"2024-04-26T21:52:50.785319+0000","last_became_peered":"2024-04-26T21:52:50.785319+0000","last_unstale":"2024-04-26T21:54:11.912197+0000","last_undegraded":"2024-04-26T21:54:11.912197+0000","last_fullsized":"2024-04-26T21:54:11.912197+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:06:29.154825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:12.397124+0000","last_change":"2024-04-26T21:52:50.780829+0000","last_active":"2024-04-26T21:54:12.397124+0000","last_peered":"2024-04-26T21:54:12.397124+0000","last_clean":"2024-04-26T21:54:12.397124+0000","last_became_active":"2024-04-26T21:52:50.780439+0000","last_became_peered":"2024-04-26T21:52:50.780439+0000","last_unstale":"2024-04-26T21:54:12.397124+0000","last_undegraded":"2024-04-26T21:54:12.397124+0000","last_fullsized":"2024-04-26T21:54:12.397124+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:20:14.533853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.922216+0000","last_change":"2024-04-26T21:52:50.787679+0000","last_active":"2024-04-26T21:54:11.922216+0000","last_peered":"2024-04-26T21:54:11.922216+0000","last_clean":"2024-04-26T21:54:11.922216+0000","last_became_active":"2024-04-26T21:52:50.787277+0000","last_became_peered":"2024-04-26T21:52:50.787277+0000","last_unstale":"2024-04-26T21:54:11.922216+0000","last_undegraded":"2024-04-26T21:54:11.922216+0000","last_fullsized":"2024-04-26T21:54:11.922216+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:29:02.131503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":15,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.905091+0000","last_change":"2024-04-26T21:53:10.927384+0000","last_active":"2024-04-26T21:54:11.905091+0000","last_peered":"2024-04-26T21:54:11.905091+0000","last_clean":"2024-04-26T21:54:11.905091+0000","last_became_active":"2024-04-26T21:53:10.926996+0000","last_became_peered":"2024-04-26T21:53:10.926996+0000","last_unstale":"2024-04-26T21:54:11.905091+0000","last_undegraded":"2024-04-26T21:54:11.905091+0000","last_fullsized":"2024-04-26T21:54:11.905091+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:25:10.174532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.909555+0000","last_change":"2024-04-26T21:52:50.779701+0000","last_active":"2024-04-26T21:54:11.909555+0000","last_peered":"2024-04-26T21:54:11.909555+0000","last_clean":"2024-04-26T21:54:11.909555+0000","last_became_active":"2024-04-26T21:52:50.779331+0000","last_became_peered":"2024-04-26T21:52:50.779331+0000","last_unstale":"2024-04-26T21:54:11.909555+0000","last_undegraded":"2024-04-26T21:54:11.909555+0000","last_fullsized":"2024-04-26T21:54:11.909555+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:38:15.569806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:12.397905+0000","last_change":"2024-04-26T21:52:50.790081+0000","last_active":"2024-04-26T21:54:12.397905+0000","last_peered":"2024-04-26T21:54:12.397905+0000","last_clean":"2024-04-26T21:54:12.397905+0000","last_became_active":"2024-04-26T21:52:50.786575+0000","last_became_peered":"2024-04-26T21:52:50.786575+0000","last_unstale":"2024-04-26T21:54:12.397905+0000","last_undegraded":"2024-04-26T21:54:12.397905+0000","last_fullsized":"2024-04-26T21:54:12.397905+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:56:47.467128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.923025+0000","last_change":"2024-04-26T21:52:50.791983+0000","last_active":"2024-04-26T21:54:11.923025+0000","last_peered":"2024-04-26T21:54:11.923025+0000","last_clean":"2024-04-26T21:54:11.923025+0000","last_became_active":"2024-04-26T21:52:50.788312+0000","last_became_peered":"2024-04-26T21:52:50.788312+0000","last_unstale":"2024-04-26T21:54:11.923025+0000","last_undegraded":"2024-04-26T21:54:11.923025+0000","last_fullsized":"2024-04-26T21:54:11.923025+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:47:47.410267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.934246+0000","last_change":"2024-04-26T21:52:50.794237+0000","last_active":"2024-04-26T21:54:11.934246+0000","last_peered":"2024-04-26T21:54:11.934246+0000","last_clean":"2024-04-26T21:54:11.934246+0000","last_became_active":"2024-04-26T21:52:50.792260+0000","last_became_peered":"2024-04-26T21:52:50.792260+0000","last_unstale":"2024-04-26T21:54:11.934246+0000","last_undegraded":"2024-04-26T21:54:11.934246+0000","last_fullsized":"2024-04-26T21:54:11.934246+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:36:05.467667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:12.396084+0000","last_change":"2024-04-26T21:52:50.777687+0000","last_active":"2024-04-26T21:54:12.396084+0000","last_peered":"2024-04-26T21:54:12.396084+0000","last_clean":"2024-04-26T21:54:12.396084+0000","last_became_active":"2024-04-26T21:52:50.777200+0000","last_became_peered":"2024-04-26T21:52:50.777200+0000","last_unstale":"2024-04-26T21:54:12.396084+0000","last_undegraded":"2024-04-26T21:54:12.396084+0000","last_fullsized":"2024-04-26T21:54:12.396084+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:40:45.167144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:12.033391+0000","last_change":"2024-04-26T21:52:50.787008+0000","last_active":"2024-04-26T21:54:12.033391+0000","last_peered":"2024-04-26T21:54:12.033391+0000","last_clean":"2024-04-26T21:54:12.033391+0000","last_became_active":"2024-04-26T21:52:50.785656+0000","last_became_peered":"2024-04-26T21:52:50.785656+0000","last_unstale":"2024-04-26T21:54:12.033391+0000","last_undegraded":"2024-04-26T21:54:12.033391+0000","last_fullsized":"2024-04-26T21:54:12.033391+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:20:12.441281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:12.033386+0000","last_change":"2024-04-26T21:52:50.796537+0000","last_active":"2024-04-26T21:54:12.033386+0000","last_peered":"2024-04-26T21:54:12.033386+0000","last_clean":"2024-04-26T21:54:12.033386+0000","last_became_active":"2024-04-26T21:52:50.796229+0000","last_became_peered":"2024-04-26T21:52:50.796229+0000","last_unstale":"2024-04-26T21:54:12.033386+0000","last_undegraded":"2024-04-26T21:54:12.033386+0000","last_fullsized":"2024-04-26T21:54:12.033386+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:38:46.404444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.908416+0000","last_change":"2024-04-26T21:52:50.789211+0000","last_active":"2024-04-26T21:54:11.908416+0000","last_peered":"2024-04-26T21:54:11.908416+0000","last_clean":"2024-04-26T21:54:11.908416+0000","last_became_active":"2024-04-26T21:52:50.788889+0000","last_became_peered":"2024-04-26T21:52:50.788889+0000","last_unstale":"2024-04-26T21:54:11.908416+0000","last_undegraded":"2024-04-26T21:54:11.908416+0000","last_fullsized":"2024-04-26T21:54:11.908416+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:25:48.497840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.921583+0000","last_change":"2024-04-26T21:52:50.788035+0000","last_active":"2024-04-26T21:54:11.921583+0000","last_peered":"2024-04-26T21:54:11.921583+0000","last_clean":"2024-04-26T21:54:11.921583+0000","last_became_active":"2024-04-26T21:52:50.787453+0000","last_became_peered":"2024-04-26T21:52:50.787453+0000","last_unstale":"2024-04-26T21:54:11.921583+0000","last_undegraded":"2024-04-26T21:54:11.921583+0000","last_fullsized":"2024-04-26T21:54:11.921583+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:40:29.718537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.905571+0000","last_change":"2024-04-26T21:52:50.778228+0000","last_active":"2024-04-26T21:54:11.905571+0000","last_peered":"2024-04-26T21:54:11.905571+0000","last_clean":"2024-04-26T21:54:11.905571+0000","last_became_active":"2024-04-26T21:52:50.777934+0000","last_became_peered":"2024-04-26T21:52:50.777934+0000","last_unstale":"2024-04-26T21:54:11.905571+0000","last_undegraded":"2024-04-26T21:54:11.905571+0000","last_fullsized":"2024-04-26T21:54:11.905571+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:33:34.598325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.909217+0000","last_change":"2024-04-26T21:52:50.790795+0000","last_active":"2024-04-26T21:54:11.909217+0000","last_peered":"2024-04-26T21:54:11.909217+0000","last_clean":"2024-04-26T21:54:11.909217+0000","last_became_active":"2024-04-26T21:52:50.788254+0000","last_became_peered":"2024-04-26T21:52:50.788254+0000","last_unstale":"2024-04-26T21:54:11.909217+0000","last_undegraded":"2024-04-26T21:54:11.909217+0000","last_fullsized":"2024-04-26T21:54:11.909217+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:20:28.235137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"91'14","reported_seq":185,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:55:22.927455+0000","last_change":"2024-04-26T21:52:50.105513+0000","last_active":"2024-04-26T21:55:22.927455+0000","last_peered":"2024-04-26T21:55:22.927455+0000","last_clean":"2024-04-26T21:55:22.927455+0000","last_became_active":"2024-04-26T21:52:50.105131+0000","last_became_peered":"2024-04-26T21:52:50.105131+0000","last_unstale":"2024-04-26T21:55:22.927455+0000","last_undegraded":"2024-04-26T21:55:22.927455+0000","last_fullsized":"2024-04-26T21:55:22.927455+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:36:58.085803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":147,"num_read_kb":147,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.908290+0000","last_change":"2024-04-26T21:52:50.795025+0000","last_active":"2024-04-26T21:54:11.908290+0000","last_peered":"2024-04-26T21:54:11.908290+0000","last_clean":"2024-04-26T21:54:11.908290+0000","last_became_active":"2024-04-26T21:52:50.793608+0000","last_became_peered":"2024-04-26T21:52:50.793608+0000","last_unstale":"2024-04-26T21:54:11.908290+0000","last_undegraded":"2024-04-26T21:54:11.908290+0000","last_fullsized":"2024-04-26T21:54:11.908290+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:49:47.764848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"87'2","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.908343+0000","last_change":"2024-04-26T21:52:50.109197+0000","last_active":"2024-04-26T21:54:11.908343+0000","last_peered":"2024-04-26T21:54:11.908343+0000","last_clean":"2024-04-26T21:54:11.908343+0000","last_became_active":"2024-04-26T21:52:50.108835+0000","last_became_peered":"2024-04-26T21:52:50.108835+0000","last_unstale":"2024-04-26T21:54:11.908343+0000","last_undegraded":"2024-04-26T21:54:11.908343+0000","last_fullsized":"2024-04-26T21:54:11.908343+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:45:09.777747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:12.034419+0000","last_change":"2024-04-26T21:52:50.783094+0000","last_active":"2024-04-26T21:54:12.034419+0000","last_peered":"2024-04-26T21:54:12.034419+0000","last_clean":"2024-04-26T21:54:12.034419+0000","last_became_active":"2024-04-26T21:52:50.782468+0000","last_became_peered":"2024-04-26T21:52:50.782468+0000","last_unstale":"2024-04-26T21:54:12.034419+0000","last_undegraded":"2024-04-26T21:54:12.034419+0000","last_fullsized":"2024-04-26T21:54:12.034419+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:53:12.977054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"87'12","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.913469+0000","last_change":"2024-04-26T21:52:50.104824+0000","last_active":"2024-04-26T21:54:11.913469+0000","last_peered":"2024-04-26T21:54:11.913469+0000","last_clean":"2024-04-26T21:54:11.913469+0000","last_became_active":"2024-04-26T21:52:50.104514+0000","last_became_peered":"2024-04-26T21:52:50.104514+0000","last_unstale":"2024-04-26T21:54:11.913469+0000","last_undegraded":"2024-04-26T21:54:11.913469+0000","last_fullsized":"2024-04-26T21:54:11.913469+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:45:31.541940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.909219+0000","last_change":"2024-04-26T21:52:50.779695+0000","last_active":"2024-04-26T21:54:11.909219+0000","last_peered":"2024-04-26T21:54:11.909219+0000","last_clean":"2024-04-26T21:54:11.909219+0000","last_became_active":"2024-04-26T21:52:50.779234+0000","last_became_peered":"2024-04-26T21:52:50.779234+0000","last_unstale":"2024-04-26T21:54:11.909219+0000","last_undegraded":"2024-04-26T21:54:11.909219+0000","last_fullsized":"2024-04-26T21:54:11.909219+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:33:07.316543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"87'5","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.912981+0000","last_change":"2024-04-26T21:52:50.107403+0000","last_active":"2024-04-26T21:54:11.912981+0000","last_peered":"2024-04-26T21:54:11.912981+0000","last_clean":"2024-04-26T21:54:11.912981+0000","last_became_active":"2024-04-26T21:52:50.107126+0000","last_became_peered":"2024-04-26T21:52:50.107126+0000","last_unstale":"2024-04-26T21:54:11.912981+0000","last_undegraded":"2024-04-26T21:54:11.912981+0000","last_fullsized":"2024-04-26T21:54:11.912981+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:52:34.985448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.916932+0000","last_change":"2024-04-26T21:52:50.783674+0000","last_active":"2024-04-26T21:54:11.916932+0000","last_peered":"2024-04-26T21:54:11.916932+0000","last_clean":"2024-04-26T21:54:11.916932+0000","last_became_active":"2024-04-26T21:52:50.782283+0000","last_became_peered":"2024-04-26T21:52:50.782283+0000","last_unstale":"2024-04-26T21:54:11.916932+0000","last_undegraded":"2024-04-26T21:54:11.916932+0000","last_fullsized":"2024-04-26T21:54:11.916932+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:41:03.144601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"91'6","reported_seq":189,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:55:22.154100+0000","last_change":"2024-04-26T21:54:11.949461+0000","last_active":"2024-04-26T21:55:22.154100+0000","last_peered":"2024-04-26T21:55:22.154100+0000","last_clean":"2024-04-26T21:55:22.154100+0000","last_became_active":"2024-04-26T21:54:11.936983+0000","last_became_peered":"2024-04-26T21:54:11.936983+0000","last_unstale":"2024-04-26T21:55:22.154100+0000","last_undegraded":"2024-04-26T21:55:22.154100+0000","last_fullsized":"2024-04-26T21:55:22.154100+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:36:40.757999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1115,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":146,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":622,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.912376+0000","last_change":"2024-04-26T21:53:11.575919+0000","last_active":"2024-04-26T21:54:11.912376+0000","last_peered":"2024-04-26T21:54:11.912376+0000","last_clean":"2024-04-26T21:54:11.912376+0000","last_became_active":"2024-04-26T21:53:11.574898+0000","last_became_peered":"2024-04-26T21:53:11.574898+0000","last_unstale":"2024-04-26T21:54:11.912376+0000","last_undegraded":"2024-04-26T21:54:11.912376+0000","last_fullsized":"2024-04-26T21:54:11.912376+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:44:16.323430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,7],"acting":[8,9,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"87'7","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:12.396927+0000","last_change":"2024-04-26T21:53:10.938377+0000","last_active":"2024-04-26T21:54:12.396927+0000","last_peered":"2024-04-26T21:54:12.396927+0000","last_clean":"2024-04-26T21:54:12.396927+0000","last_became_active":"2024-04-26T21:53:10.921998+0000","last_became_peered":"2024-04-26T21:53:10.921998+0000","last_unstale":"2024-04-26T21:54:12.396927+0000","last_undegraded":"2024-04-26T21:54:12.396927+0000","last_fullsized":"2024-04-26T21:54:12.396927+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:51:20.389344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,6],"acting":[10,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.919025+0000","last_change":"2024-04-26T21:53:10.931444+0000","last_active":"2024-04-26T21:54:11.919025+0000","last_peered":"2024-04-26T21:54:11.919025+0000","last_clean":"2024-04-26T21:54:11.919025+0000","last_became_active":"2024-04-26T21:53:10.930761+0000","last_became_peered":"2024-04-26T21:53:10.930761+0000","last_unstale":"2024-04-26T21:54:11.919025+0000","last_undegraded":"2024-04-26T21:54:11.919025+0000","last_fullsized":"2024-04-26T21:54:11.919025+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:56:17.803811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,3],"acting":[5,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"87'3","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:12.395254+0000","last_change":"2024-04-26T21:52:50.107564+0000","last_active":"2024-04-26T21:54:12.395254+0000","last_peered":"2024-04-26T21:54:12.395254+0000","last_clean":"2024-04-26T21:54:12.395254+0000","last_became_active":"2024-04-26T21:52:50.106653+0000","last_became_peered":"2024-04-26T21:52:50.106653+0000","last_unstale":"2024-04-26T21:54:12.395254+0000","last_undegraded":"2024-04-26T21:54:12.395254+0000","last_fullsized":"2024-04-26T21:54:12.395254+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:05:36.594729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.907525+0000","last_change":"2024-04-26T21:52:50.780279+0000","last_active":"2024-04-26T21:54:11.907525+0000","last_peered":"2024-04-26T21:54:11.907525+0000","last_clean":"2024-04-26T21:54:11.907525+0000","last_became_active":"2024-04-26T21:52:50.779950+0000","last_became_peered":"2024-04-26T21:52:50.779950+0000","last_unstale":"2024-04-26T21:54:11.907525+0000","last_undegraded":"2024-04-26T21:54:11.907525+0000","last_fullsized":"2024-04-26T21:54:11.907525+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:11:36.282319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"87'3","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.917578+0000","last_change":"2024-04-26T21:52:50.106269+0000","last_active":"2024-04-26T21:54:11.917578+0000","last_peered":"2024-04-26T21:54:11.917578+0000","last_clean":"2024-04-26T21:54:11.917578+0000","last_became_active":"2024-04-26T21:52:50.105948+0000","last_became_peered":"2024-04-26T21:52:50.105948+0000","last_unstale":"2024-04-26T21:54:11.917578+0000","last_undegraded":"2024-04-26T21:54:11.917578+0000","last_fullsized":"2024-04-26T21:54:11.917578+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:39:31.024362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.905324+0000","last_change":"2024-04-26T21:52:50.778938+0000","last_active":"2024-04-26T21:54:11.905324+0000","last_peered":"2024-04-26T21:54:11.905324+0000","last_clean":"2024-04-26T21:54:11.905324+0000","last_became_active":"2024-04-26T21:52:50.778590+0000","last_became_peered":"2024-04-26T21:52:50.778590+0000","last_unstale":"2024-04-26T21:54:11.905324+0000","last_undegraded":"2024-04-26T21:54:11.905324+0000","last_fullsized":"2024-04-26T21:54:11.905324+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:03:10.325793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"91'6","reported_seq":320,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:55:22.948842+0000","last_change":"2024-04-26T21:52:50.107243+0000","last_active":"2024-04-26T21:55:22.948842+0000","last_peered":"2024-04-26T21:55:22.948842+0000","last_clean":"2024-04-26T21:55:22.948842+0000","last_became_active":"2024-04-26T21:52:50.106729+0000","last_became_peered":"2024-04-26T21:52:50.106729+0000","last_unstale":"2024-04-26T21:55:22.948842+0000","last_undegraded":"2024-04-26T21:55:22.948842+0000","last_fullsized":"2024-04-26T21:55:22.948842+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:28:11.197997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":290,"num_read_kb":290,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.909345+0000","last_change":"2024-04-26T21:52:50.776906+0000","last_active":"2024-04-26T21:54:11.909345+0000","last_peered":"2024-04-26T21:54:11.909345+0000","last_clean":"2024-04-26T21:54:11.909345+0000","last_became_active":"2024-04-26T21:52:50.776567+0000","last_became_peered":"2024-04-26T21:52:50.776567+0000","last_unstale":"2024-04-26T21:54:11.909345+0000","last_undegraded":"2024-04-26T21:54:11.909345+0000","last_fullsized":"2024-04-26T21:54:11.909345+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:51:39.512572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"87'4","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.923341+0000","last_change":"2024-04-26T21:52:50.109792+0000","last_active":"2024-04-26T21:54:11.923341+0000","last_peered":"2024-04-26T21:54:11.923341+0000","last_clean":"2024-04-26T21:54:11.923341+0000","last_became_active":"2024-04-26T21:52:50.106662+0000","last_became_peered":"2024-04-26T21:52:50.106662+0000","last_unstale":"2024-04-26T21:54:11.923341+0000","last_undegraded":"2024-04-26T21:54:11.923341+0000","last_fullsized":"2024-04-26T21:54:11.923341+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:41:27.390780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:12.034410+0000","last_change":"2024-04-26T21:52:50.782798+0000","last_active":"2024-04-26T21:54:12.034410+0000","last_peered":"2024-04-26T21:54:12.034410+0000","last_clean":"2024-04-26T21:54:12.034410+0000","last_became_active":"2024-04-26T21:52:50.781975+0000","last_became_peered":"2024-04-26T21:52:50.781975+0000","last_unstale":"2024-04-26T21:54:12.034410+0000","last_undegraded":"2024-04-26T21:54:12.034410+0000","last_fullsized":"2024-04-26T21:54:12.034410+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:13:07.261095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"87'5","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.917206+0000","last_change":"2024-04-26T21:52:50.099296+0000","last_active":"2024-04-26T21:54:11.917206+0000","last_peered":"2024-04-26T21:54:11.917206+0000","last_clean":"2024-04-26T21:54:11.917206+0000","last_became_active":"2024-04-26T21:52:50.098940+0000","last_became_peered":"2024-04-26T21:52:50.098940+0000","last_unstale":"2024-04-26T21:54:11.917206+0000","last_undegraded":"2024-04-26T21:54:11.917206+0000","last_fullsized":"2024-04-26T21:54:11.917206+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:49:15.765474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.909628+0000","last_change":"2024-04-26T21:52:50.783341+0000","last_active":"2024-04-26T21:54:11.909628+0000","last_peered":"2024-04-26T21:54:11.909628+0000","last_clean":"2024-04-26T21:54:11.909628+0000","last_became_active":"2024-04-26T21:52:50.783025+0000","last_became_peered":"2024-04-26T21:52:50.783025+0000","last_unstale":"2024-04-26T21:54:11.909628+0000","last_undegraded":"2024-04-26T21:54:11.909628+0000","last_fullsized":"2024-04-26T21:54:11.909628+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:54:06.070550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"91'31","reported_seq":352,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:55:23.155989+0000","last_change":"2024-04-26T21:52:50.106617+0000","last_active":"2024-04-26T21:55:23.155989+0000","last_peered":"2024-04-26T21:55:23.155989+0000","last_clean":"2024-04-26T21:55:23.155989+0000","last_became_active":"2024-04-26T21:52:50.106161+0000","last_became_peered":"2024-04-26T21:52:50.106161+0000","last_unstale":"2024-04-26T21:55:23.155989+0000","last_undegraded":"2024-04-26T21:55:23.155989+0000","last_fullsized":"2024-04-26T21:55:23.155989+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:52:07.977749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":51444,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":297,"num_read_kb":199,"num_write":33,"num_write_kb":61,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.912969+0000","last_change":"2024-04-26T21:52:50.794567+0000","last_active":"2024-04-26T21:54:11.912969+0000","last_peered":"2024-04-26T21:54:11.912969+0000","last_clean":"2024-04-26T21:54:11.912969+0000","last_became_active":"2024-04-26T21:52:50.794245+0000","last_became_peered":"2024-04-26T21:52:50.794245+0000","last_unstale":"2024-04-26T21:54:11.912969+0000","last_undegraded":"2024-04-26T21:54:11.912969+0000","last_fullsized":"2024-04-26T21:54:11.912969+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:58:11.501104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"87'9","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.910302+0000","last_change":"2024-04-26T21:52:50.102156+0000","last_active":"2024-04-26T21:54:11.910302+0000","last_peered":"2024-04-26T21:54:11.910302+0000","last_clean":"2024-04-26T21:54:11.910302+0000","last_became_active":"2024-04-26T21:52:50.101721+0000","last_became_peered":"2024-04-26T21:52:50.101721+0000","last_unstale":"2024-04-26T21:54:11.910302+0000","last_undegraded":"2024-04-26T21:54:11.910302+0000","last_fullsized":"2024-04-26T21:54:11.910302+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:48:42.081874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:12.395808+0000","last_change":"2024-04-26T21:52:50.789199+0000","last_active":"2024-04-26T21:54:12.395808+0000","last_peered":"2024-04-26T21:54:12.395808+0000","last_clean":"2024-04-26T21:54:12.395808+0000","last_became_active":"2024-04-26T21:52:50.788868+0000","last_became_peered":"2024-04-26T21:52:50.788868+0000","last_unstale":"2024-04-26T21:54:12.395808+0000","last_undegraded":"2024-04-26T21:54:12.395808+0000","last_fullsized":"2024-04-26T21:54:12.395808+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:29:55.755564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"87'5","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.918826+0000","last_change":"2024-04-26T21:52:50.104920+0000","last_active":"2024-04-26T21:54:11.918826+0000","last_peered":"2024-04-26T21:54:11.918826+0000","last_clean":"2024-04-26T21:54:11.918826+0000","last_became_active":"2024-04-26T21:52:50.104579+0000","last_became_peered":"2024-04-26T21:52:50.104579+0000","last_unstale":"2024-04-26T21:54:11.918826+0000","last_undegraded":"2024-04-26T21:54:11.918826+0000","last_fullsized":"2024-04-26T21:54:11.918826+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:09:35.760918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.923423+0000","last_change":"2024-04-26T21:52:50.787016+0000","last_active":"2024-04-26T21:54:11.923423+0000","last_peered":"2024-04-26T21:54:11.923423+0000","last_clean":"2024-04-26T21:54:11.923423+0000","last_became_active":"2024-04-26T21:52:50.778439+0000","last_became_peered":"2024-04-26T21:52:50.778439+0000","last_unstale":"2024-04-26T21:54:11.923423+0000","last_undegraded":"2024-04-26T21:54:11.923423+0000","last_fullsized":"2024-04-26T21:54:11.923423+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:13:35.735480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"91'6","reported_seq":179,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:55:22.964012+0000","last_change":"2024-04-26T21:52:50.110175+0000","last_active":"2024-04-26T21:55:22.964012+0000","last_peered":"2024-04-26T21:55:22.964012+0000","last_clean":"2024-04-26T21:55:22.964012+0000","last_became_active":"2024-04-26T21:52:50.109779+0000","last_became_peered":"2024-04-26T21:52:50.109779+0000","last_unstale":"2024-04-26T21:55:22.964012+0000","last_undegraded":"2024-04-26T21:55:22.964012+0000","last_fullsized":"2024-04-26T21:55:22.964012+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:06:46.756327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1081,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":150,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.922565+0000","last_change":"2024-04-26T21:52:50.784199+0000","last_active":"2024-04-26T21:54:11.922565+0000","last_peered":"2024-04-26T21:54:11.922565+0000","last_clean":"2024-04-26T21:54:11.922565+0000","last_became_active":"2024-04-26T21:52:50.783820+0000","last_became_peered":"2024-04-26T21:52:50.783820+0000","last_unstale":"2024-04-26T21:54:11.922565+0000","last_undegraded":"2024-04-26T21:54:11.922565+0000","last_fullsized":"2024-04-26T21:54:11.922565+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:45:24.776722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"87'17","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.912743+0000","last_change":"2024-04-26T21:52:50.095685+0000","last_active":"2024-04-26T21:54:11.912743+0000","last_peered":"2024-04-26T21:54:11.912743+0000","last_clean":"2024-04-26T21:54:11.912743+0000","last_became_active":"2024-04-26T21:52:50.095298+0000","last_became_peered":"2024-04-26T21:52:50.095298+0000","last_unstale":"2024-04-26T21:54:11.912743+0000","last_undegraded":"2024-04-26T21:54:11.912743+0000","last_fullsized":"2024-04-26T21:54:11.912743+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:47:26.061531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.917867+0000","last_change":"2024-04-26T21:52:50.786607+0000","last_active":"2024-04-26T21:54:11.917867+0000","last_peered":"2024-04-26T21:54:11.917867+0000","last_clean":"2024-04-26T21:54:11.917867+0000","last_became_active":"2024-04-26T21:52:50.786261+0000","last_became_peered":"2024-04-26T21:52:50.786261+0000","last_unstale":"2024-04-26T21:54:11.917867+0000","last_undegraded":"2024-04-26T21:54:11.917867+0000","last_fullsized":"2024-04-26T21:54:11.917867+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:08:45.902260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"91'3","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:55:08.888181+0000","last_change":"2024-04-26T21:52:50.095782+0000","last_active":"2024-04-26T21:55:08.888181+0000","last_peered":"2024-04-26T21:55:08.888181+0000","last_clean":"2024-04-26T21:55:08.888181+0000","last_became_active":"2024-04-26T21:52:50.093983+0000","last_became_peered":"2024-04-26T21:52:50.093983+0000","last_unstale":"2024-04-26T21:55:08.888181+0000","last_undegraded":"2024-04-26T21:55:08.888181+0000","last_fullsized":"2024-04-26T21:55:08.888181+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:18:40.103099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.921290+0000","last_change":"2024-04-26T21:52:50.791969+0000","last_active":"2024-04-26T21:54:11.921290+0000","last_peered":"2024-04-26T21:54:11.921290+0000","last_clean":"2024-04-26T21:54:11.921290+0000","last_became_active":"2024-04-26T21:52:50.790441+0000","last_became_peered":"2024-04-26T21:52:50.790441+0000","last_unstale":"2024-04-26T21:54:11.921290+0000","last_undegraded":"2024-04-26T21:54:11.921290+0000","last_fullsized":"2024-04-26T21:54:11.921290+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:11:04.964439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"87'1","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.905428+0000","last_change":"2024-04-26T21:52:50.101807+0000","last_active":"2024-04-26T21:54:11.905428+0000","last_peered":"2024-04-26T21:54:11.905428+0000","last_clean":"2024-04-26T21:54:11.905428+0000","last_became_active":"2024-04-26T21:52:50.101329+0000","last_became_peered":"2024-04-26T21:52:50.101329+0000","last_unstale":"2024-04-26T21:54:11.905428+0000","last_undegraded":"2024-04-26T21:54:11.905428+0000","last_fullsized":"2024-04-26T21:54:11.905428+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:54:47.788172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"87'1","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:12.397681+0000","last_change":"2024-04-26T21:52:50.780872+0000","last_active":"2024-04-26T21:54:12.397681+0000","last_peered":"2024-04-26T21:54:12.397681+0000","last_clean":"2024-04-26T21:54:12.397681+0000","last_became_active":"2024-04-26T21:52:50.778757+0000","last_became_peered":"2024-04-26T21:52:50.778757+0000","last_unstale":"2024-04-26T21:54:12.397681+0000","last_undegraded":"2024-04-26T21:54:12.397681+0000","last_fullsized":"2024-04-26T21:54:12.397681+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:07:21.595091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"91'6","reported_seq":181,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:55:25.936092+0000","last_change":"2024-04-26T21:52:50.106886+0000","last_active":"2024-04-26T21:55:25.936092+0000","last_peered":"2024-04-26T21:55:25.936092+0000","last_clean":"2024-04-26T21:55:25.936092+0000","last_became_active":"2024-04-26T21:52:50.106150+0000","last_became_peered":"2024-04-26T21:52:50.106150+0000","last_unstale":"2024-04-26T21:55:25.936092+0000","last_undegraded":"2024-04-26T21:55:25.936092+0000","last_fullsized":"2024-04-26T21:55:25.936092+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:20:54.920416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1081,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":151,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:12.397270+0000","last_change":"2024-04-26T21:53:11.069186+0000","last_active":"2024-04-26T21:54:12.397270+0000","last_peered":"2024-04-26T21:54:12.397270+0000","last_clean":"2024-04-26T21:54:12.397270+0000","last_became_active":"2024-04-26T21:53:11.068418+0000","last_became_peered":"2024-04-26T21:53:11.068418+0000","last_unstale":"2024-04-26T21:54:12.397270+0000","last_undegraded":"2024-04-26T21:54:12.397270+0000","last_fullsized":"2024-04-26T21:54:12.397270+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:57:59.663173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.917799+0000","last_change":"2024-04-26T21:52:50.785257+0000","last_active":"2024-04-26T21:54:11.917799+0000","last_peered":"2024-04-26T21:54:11.917799+0000","last_clean":"2024-04-26T21:54:11.917799+0000","last_became_active":"2024-04-26T21:52:50.784906+0000","last_became_peered":"2024-04-26T21:52:50.784906+0000","last_unstale":"2024-04-26T21:54:11.917799+0000","last_undegraded":"2024-04-26T21:54:11.917799+0000","last_fullsized":"2024-04-26T21:54:11.917799+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:04:50.921663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"91'6","reported_seq":176,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:55:21.959779+0000","last_change":"2024-04-26T21:52:50.102425+0000","last_active":"2024-04-26T21:55:21.959779+0000","last_peered":"2024-04-26T21:55:21.959779+0000","last_clean":"2024-04-26T21:55:21.959779+0000","last_became_active":"2024-04-26T21:52:50.101703+0000","last_became_peered":"2024-04-26T21:52:50.101703+0000","last_unstale":"2024-04-26T21:55:21.959779+0000","last_undegraded":"2024-04-26T21:55:21.959779+0000","last_fullsized":"2024-04-26T21:55:21.959779+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:13:39.910519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":146,"num_read_kb":146,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.907737+0000","last_change":"2024-04-26T21:52:50.775036+0000","last_active":"2024-04-26T21:54:11.907737+0000","last_peered":"2024-04-26T21:54:11.907737+0000","last_clean":"2024-04-26T21:54:11.907737+0000","last_became_active":"2024-04-26T21:52:50.773347+0000","last_became_peered":"2024-04-26T21:52:50.773347+0000","last_unstale":"2024-04-26T21:54:11.907737+0000","last_undegraded":"2024-04-26T21:54:11.907737+0000","last_fullsized":"2024-04-26T21:54:11.907737+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:59:55.303014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"87'6","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.934310+0000","last_change":"2024-04-26T21:52:50.109437+0000","last_active":"2024-04-26T21:54:11.934310+0000","last_peered":"2024-04-26T21:54:11.934310+0000","last_clean":"2024-04-26T21:54:11.934310+0000","last_became_active":"2024-04-26T21:52:50.108605+0000","last_became_peered":"2024-04-26T21:52:50.108605+0000","last_unstale":"2024-04-26T21:54:11.934310+0000","last_undegraded":"2024-04-26T21:54:11.934310+0000","last_fullsized":"2024-04-26T21:54:11.934310+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:31:04.208334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":351,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.922178+0000","last_change":"2024-04-26T21:51:11.498577+0000","last_active":"2024-04-26T21:54:11.922178+0000","last_peered":"2024-04-26T21:54:11.922178+0000","last_clean":"2024-04-26T21:54:11.922178+0000","last_became_active":"2024-04-26T21:51:11.497422+0000","last_became_peered":"2024-04-26T21:51:11.497422+0000","last_unstale":"2024-04-26T21:54:11.922178+0000","last_undegraded":"2024-04-26T21:54:11.922178+0000","last_fullsized":"2024-04-26T21:54:11.922178+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:46:29.789400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:46:29.789400+0000","last_clean_scrub_stamp":"2024-04-26T21:46:29.789400+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:54:13.450359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:12.395065+0000","last_change":"2024-04-26T21:52:50.794270+0000","last_active":"2024-04-26T21:54:12.395065+0000","last_peered":"2024-04-26T21:54:12.395065+0000","last_clean":"2024-04-26T21:54:12.395065+0000","last_became_active":"2024-04-26T21:52:50.793966+0000","last_became_peered":"2024-04-26T21:52:50.793966+0000","last_unstale":"2024-04-26T21:54:12.395065+0000","last_undegraded":"2024-04-26T21:54:12.395065+0000","last_fullsized":"2024-04-26T21:54:12.395065+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:01:37.668466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"87'4","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.922067+0000","last_change":"2024-04-26T21:52:50.102700+0000","last_active":"2024-04-26T21:54:11.922067+0000","last_peered":"2024-04-26T21:54:11.922067+0000","last_clean":"2024-04-26T21:54:11.922067+0000","last_became_active":"2024-04-26T21:52:50.101267+0000","last_became_peered":"2024-04-26T21:52:50.101267+0000","last_unstale":"2024-04-26T21:54:11.922067+0000","last_undegraded":"2024-04-26T21:54:11.922067+0000","last_fullsized":"2024-04-26T21:54:11.922067+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:41:21.727411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"87'4","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.914022+0000","last_change":"2024-04-26T21:52:50.102191+0000","last_active":"2024-04-26T21:54:11.914022+0000","last_peered":"2024-04-26T21:54:11.914022+0000","last_clean":"2024-04-26T21:54:11.914022+0000","last_became_active":"2024-04-26T21:52:50.101307+0000","last_became_peered":"2024-04-26T21:52:50.101307+0000","last_unstale":"2024-04-26T21:54:11.914022+0000","last_undegraded":"2024-04-26T21:54:11.914022+0000","last_fullsized":"2024-04-26T21:54:11.914022+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:59:12.465291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:12.395488+0000","last_change":"2024-04-26T21:52:50.792020+0000","last_active":"2024-04-26T21:54:12.395488+0000","last_peered":"2024-04-26T21:54:12.395488+0000","last_clean":"2024-04-26T21:54:12.395488+0000","last_became_active":"2024-04-26T21:52:50.791677+0000","last_became_peered":"2024-04-26T21:52:50.791677+0000","last_unstale":"2024-04-26T21:54:12.395488+0000","last_undegraded":"2024-04-26T21:54:12.395488+0000","last_fullsized":"2024-04-26T21:54:12.395488+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:11:42.218294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"91'13","reported_seq":53,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:59.673130+0000","last_change":"2024-04-26T21:54:11.936807+0000","last_active":"2024-04-26T21:54:59.673130+0000","last_peered":"2024-04-26T21:54:59.673130+0000","last_clean":"2024-04-26T21:54:59.673130+0000","last_became_active":"2024-04-26T21:54:11.923482+0000","last_became_peered":"2024-04-26T21:54:11.923482+0000","last_unstale":"2024-04-26T21:54:59.673130+0000","last_undegraded":"2024-04-26T21:54:59.673130+0000","last_fullsized":"2024-04-26T21:54:59.673130+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:15:46.945453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.908090+0000","last_change":"2024-04-26T21:52:50.779775+0000","last_active":"2024-04-26T21:54:11.908090+0000","last_peered":"2024-04-26T21:54:11.908090+0000","last_clean":"2024-04-26T21:54:11.908090+0000","last_became_active":"2024-04-26T21:52:50.778857+0000","last_became_peered":"2024-04-26T21:52:50.778857+0000","last_unstale":"2024-04-26T21:54:11.908090+0000","last_undegraded":"2024-04-26T21:54:11.908090+0000","last_fullsized":"2024-04-26T21:54:11.908090+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:59:50.946578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"87'7","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.934470+0000","last_change":"2024-04-26T21:52:50.113027+0000","last_active":"2024-04-26T21:54:11.934470+0000","last_peered":"2024-04-26T21:54:11.934470+0000","last_clean":"2024-04-26T21:54:11.934470+0000","last_became_active":"2024-04-26T21:52:50.112637+0000","last_became_peered":"2024-04-26T21:52:50.112637+0000","last_unstale":"2024-04-26T21:54:11.934470+0000","last_undegraded":"2024-04-26T21:54:11.934470+0000","last_fullsized":"2024-04-26T21:54:11.934470+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:09:47.798184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:12.034220+0000","last_change":"2024-04-26T21:52:50.785119+0000","last_active":"2024-04-26T21:54:12.034220+0000","last_peered":"2024-04-26T21:54:12.034220+0000","last_clean":"2024-04-26T21:54:12.034220+0000","last_became_active":"2024-04-26T21:52:50.781005+0000","last_became_peered":"2024-04-26T21:52:50.781005+0000","last_unstale":"2024-04-26T21:54:12.034220+0000","last_undegraded":"2024-04-26T21:54:12.034220+0000","last_fullsized":"2024-04-26T21:54:12.034220+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:16:32.986810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"87'2","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.908468+0000","last_change":"2024-04-26T21:52:50.112263+0000","last_active":"2024-04-26T21:54:11.908468+0000","last_peered":"2024-04-26T21:54:11.908468+0000","last_clean":"2024-04-26T21:54:11.908468+0000","last_became_active":"2024-04-26T21:52:50.110649+0000","last_became_peered":"2024-04-26T21:52:50.110649+0000","last_unstale":"2024-04-26T21:54:11.908468+0000","last_undegraded":"2024-04-26T21:54:11.908468+0000","last_fullsized":"2024-04-26T21:54:11.908468+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:14:23.245000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.913095+0000","last_change":"2024-04-26T21:52:50.790674+0000","last_active":"2024-04-26T21:54:11.913095+0000","last_peered":"2024-04-26T21:54:11.913095+0000","last_clean":"2024-04-26T21:54:11.913095+0000","last_became_active":"2024-04-26T21:52:50.790286+0000","last_became_peered":"2024-04-26T21:52:50.790286+0000","last_unstale":"2024-04-26T21:54:11.913095+0000","last_undegraded":"2024-04-26T21:54:11.913095+0000","last_fullsized":"2024-04-26T21:54:11.913095+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:22:32.246321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"87'7","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.907591+0000","last_change":"2024-04-26T21:52:50.112158+0000","last_active":"2024-04-26T21:54:11.907591+0000","last_peered":"2024-04-26T21:54:11.907591+0000","last_clean":"2024-04-26T21:54:11.907591+0000","last_became_active":"2024-04-26T21:52:50.111453+0000","last_became_peered":"2024-04-26T21:52:50.111453+0000","last_unstale":"2024-04-26T21:54:11.907591+0000","last_undegraded":"2024-04-26T21:54:11.907591+0000","last_fullsized":"2024-04-26T21:54:11.907591+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:16:58.626985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.915638+0000","last_change":"2024-04-26T21:52:50.798241+0000","last_active":"2024-04-26T21:54:11.915638+0000","last_peered":"2024-04-26T21:54:11.915638+0000","last_clean":"2024-04-26T21:54:11.915638+0000","last_became_active":"2024-04-26T21:52:50.797878+0000","last_became_peered":"2024-04-26T21:52:50.797878+0000","last_unstale":"2024-04-26T21:54:11.915638+0000","last_undegraded":"2024-04-26T21:54:11.915638+0000","last_fullsized":"2024-04-26T21:54:11.915638+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:42:49.363849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"91'5","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:55:20.595077+0000","last_change":"2024-04-26T21:54:11.946852+0000","last_active":"2024-04-26T21:55:20.595077+0000","last_peered":"2024-04-26T21:55:20.595077+0000","last_clean":"2024-04-26T21:55:20.595077+0000","last_became_active":"2024-04-26T21:54:11.923399+0000","last_became_peered":"2024-04-26T21:54:11.923399+0000","last_unstale":"2024-04-26T21:55:20.595077+0000","last_undegraded":"2024-04-26T21:55:20.595077+0000","last_fullsized":"2024-04-26T21:55:20.595077+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:05:34.688374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.908206+0000","last_change":"2024-04-26T21:52:50.776333+0000","last_active":"2024-04-26T21:54:11.908206+0000","last_peered":"2024-04-26T21:54:11.908206+0000","last_clean":"2024-04-26T21:54:11.908206+0000","last_became_active":"2024-04-26T21:52:50.776001+0000","last_became_peered":"2024-04-26T21:52:50.776001+0000","last_unstale":"2024-04-26T21:54:11.908206+0000","last_undegraded":"2024-04-26T21:54:11.908206+0000","last_fullsized":"2024-04-26T21:54:11.908206+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:38:26.578928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"87'4","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.913422+0000","last_change":"2024-04-26T21:52:50.106008+0000","last_active":"2024-04-26T21:54:11.913422+0000","last_peered":"2024-04-26T21:54:11.913422+0000","last_clean":"2024-04-26T21:54:11.913422+0000","last_became_active":"2024-04-26T21:52:50.105565+0000","last_became_peered":"2024-04-26T21:52:50.105565+0000","last_unstale":"2024-04-26T21:54:11.913422+0000","last_undegraded":"2024-04-26T21:54:11.913422+0000","last_fullsized":"2024-04-26T21:54:11.913422+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:15:17.355461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.923394+0000","last_change":"2024-04-26T21:52:50.776051+0000","last_active":"2024-04-26T21:54:11.923394+0000","last_peered":"2024-04-26T21:54:11.923394+0000","last_clean":"2024-04-26T21:54:11.923394+0000","last_became_active":"2024-04-26T21:52:50.775716+0000","last_became_peered":"2024-04-26T21:52:50.775716+0000","last_unstale":"2024-04-26T21:54:11.923394+0000","last_undegraded":"2024-04-26T21:54:11.923394+0000","last_fullsized":"2024-04-26T21:54:11.923394+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:34:21.110773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"91'7","reported_seq":180,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:55:22.950112+0000","last_change":"2024-04-26T21:52:50.111280+0000","last_active":"2024-04-26T21:55:22.950112+0000","last_peered":"2024-04-26T21:55:22.950112+0000","last_clean":"2024-04-26T21:55:22.950112+0000","last_became_active":"2024-04-26T21:52:50.110972+0000","last_became_peered":"2024-04-26T21:52:50.110972+0000","last_unstale":"2024-04-26T21:55:22.950112+0000","last_undegraded":"2024-04-26T21:55:22.950112+0000","last_fullsized":"2024-04-26T21:55:22.950112+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:41:48.203134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2054,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":149,"num_read_kb":3,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.923325+0000","last_change":"2024-04-26T21:52:50.777488+0000","last_active":"2024-04-26T21:54:11.923325+0000","last_peered":"2024-04-26T21:54:11.923325+0000","last_clean":"2024-04-26T21:54:11.923325+0000","last_became_active":"2024-04-26T21:52:50.777162+0000","last_became_peered":"2024-04-26T21:52:50.777162+0000","last_unstale":"2024-04-26T21:54:11.923325+0000","last_undegraded":"2024-04-26T21:54:11.923325+0000","last_fullsized":"2024-04-26T21:54:11.923325+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:07:22.683109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"87'3","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.916669+0000","last_change":"2024-04-26T21:52:50.104508+0000","last_active":"2024-04-26T21:54:11.916669+0000","last_peered":"2024-04-26T21:54:11.916669+0000","last_clean":"2024-04-26T21:54:11.916669+0000","last_became_active":"2024-04-26T21:52:50.103924+0000","last_became_peered":"2024-04-26T21:52:50.103924+0000","last_unstale":"2024-04-26T21:54:11.916669+0000","last_undegraded":"2024-04-26T21:54:11.916669+0000","last_fullsized":"2024-04-26T21:54:11.916669+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:39:25.861339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"87'1","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.922627+0000","last_change":"2024-04-26T21:52:50.782915+0000","last_active":"2024-04-26T21:54:11.922627+0000","last_peered":"2024-04-26T21:54:11.922627+0000","last_clean":"2024-04-26T21:54:11.922627+0000","last_became_active":"2024-04-26T21:52:50.782253+0000","last_became_peered":"2024-04-26T21:52:50.782253+0000","last_unstale":"2024-04-26T21:54:11.922627+0000","last_undegraded":"2024-04-26T21:54:11.922627+0000","last_fullsized":"2024-04-26T21:54:11.922627+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:12:51.624162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"87'5","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:12.395358+0000","last_change":"2024-04-26T21:53:11.590613+0000","last_active":"2024-04-26T21:54:12.395358+0000","last_peered":"2024-04-26T21:54:12.395358+0000","last_clean":"2024-04-26T21:54:12.395358+0000","last_became_active":"2024-04-26T21:53:11.578727+0000","last_became_peered":"2024-04-26T21:53:11.578727+0000","last_unstale":"2024-04-26T21:54:12.395358+0000","last_undegraded":"2024-04-26T21:54:12.395358+0000","last_fullsized":"2024-04-26T21:54:12.395358+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:49.050072+0000","last_clean_scrub_stamp":"2024-04-26T21:52:49.050072+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:44:38.867231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.922922+0000","last_change":"2024-04-26T21:52:50.787777+0000","last_active":"2024-04-26T21:54:11.922922+0000","last_peered":"2024-04-26T21:54:11.922922+0000","last_clean":"2024-04-26T21:54:11.922922+0000","last_became_active":"2024-04-26T21:52:50.782403+0000","last_became_peered":"2024-04-26T21:52:50.782403+0000","last_unstale":"2024-04-26T21:54:11.922922+0000","last_undegraded":"2024-04-26T21:54:11.922922+0000","last_fullsized":"2024-04-26T21:54:11.922922+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:28:58.425802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.915961+0000","last_change":"2024-04-26T21:52:50.800010+0000","last_active":"2024-04-26T21:54:11.915961+0000","last_peered":"2024-04-26T21:54:11.915961+0000","last_clean":"2024-04-26T21:54:11.915961+0000","last_became_active":"2024-04-26T21:52:50.799308+0000","last_became_peered":"2024-04-26T21:52:50.799308+0000","last_unstale":"2024-04-26T21:54:11.915961+0000","last_undegraded":"2024-04-26T21:54:11.915961+0000","last_fullsized":"2024-04-26T21:54:11.915961+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:54:25.772312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.909036+0000","last_change":"2024-04-26T21:52:50.798281+0000","last_active":"2024-04-26T21:54:11.909036+0000","last_peered":"2024-04-26T21:54:11.909036+0000","last_clean":"2024-04-26T21:54:11.909036+0000","last_became_active":"2024-04-26T21:52:50.795879+0000","last_became_peered":"2024-04-26T21:52:50.795879+0000","last_unstale":"2024-04-26T21:54:11.909036+0000","last_undegraded":"2024-04-26T21:54:11.909036+0000","last_fullsized":"2024-04-26T21:54:11.909036+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:36:00.769991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.908710+0000","last_change":"2024-04-26T21:52:50.791058+0000","last_active":"2024-04-26T21:54:11.908710+0000","last_peered":"2024-04-26T21:54:11.908710+0000","last_clean":"2024-04-26T21:54:11.908710+0000","last_became_active":"2024-04-26T21:52:50.790721+0000","last_became_peered":"2024-04-26T21:52:50.790721+0000","last_unstale":"2024-04-26T21:54:11.908710+0000","last_undegraded":"2024-04-26T21:54:11.908710+0000","last_fullsized":"2024-04-26T21:54:11.908710+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:56:03.239622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:12.033664+0000","last_change":"2024-04-26T21:52:50.788193+0000","last_active":"2024-04-26T21:54:12.033664+0000","last_peered":"2024-04-26T21:54:12.033664+0000","last_clean":"2024-04-26T21:54:12.033664+0000","last_became_active":"2024-04-26T21:52:50.786749+0000","last_became_peered":"2024-04-26T21:52:50.786749+0000","last_unstale":"2024-04-26T21:54:12.033664+0000","last_undegraded":"2024-04-26T21:54:12.033664+0000","last_fullsized":"2024-04-26T21:54:12.033664+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:22:11.277217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:12.395673+0000","last_change":"2024-04-26T21:52:50.782508+0000","last_active":"2024-04-26T21:54:12.395673+0000","last_peered":"2024-04-26T21:54:12.395673+0000","last_clean":"2024-04-26T21:54:12.395673+0000","last_became_active":"2024-04-26T21:52:50.781505+0000","last_became_peered":"2024-04-26T21:52:50.781505+0000","last_unstale":"2024-04-26T21:54:12.395673+0000","last_undegraded":"2024-04-26T21:54:12.395673+0000","last_fullsized":"2024-04-26T21:54:12.395673+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:22:57.867546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.916618+0000","last_change":"2024-04-26T21:52:50.789205+0000","last_active":"2024-04-26T21:54:11.916618+0000","last_peered":"2024-04-26T21:54:11.916618+0000","last_clean":"2024-04-26T21:54:11.916618+0000","last_became_active":"2024-04-26T21:52:50.788898+0000","last_became_peered":"2024-04-26T21:52:50.788898+0000","last_unstale":"2024-04-26T21:54:11.916618+0000","last_undegraded":"2024-04-26T21:54:11.916618+0000","last_fullsized":"2024-04-26T21:54:11.916618+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:18:51.401968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.909356+0000","last_change":"2024-04-26T21:52:50.793451+0000","last_active":"2024-04-26T21:54:11.909356+0000","last_peered":"2024-04-26T21:54:11.909356+0000","last_clean":"2024-04-26T21:54:11.909356+0000","last_became_active":"2024-04-26T21:52:50.791696+0000","last_became_peered":"2024-04-26T21:52:50.791696+0000","last_unstale":"2024-04-26T21:54:11.909356+0000","last_undegraded":"2024-04-26T21:54:11.909356+0000","last_fullsized":"2024-04-26T21:54:11.909356+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:51:43.543500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.907992+0000","last_change":"2024-04-26T21:53:10.919691+0000","last_active":"2024-04-26T21:54:11.907992+0000","last_peered":"2024-04-26T21:54:11.907992+0000","last_clean":"2024-04-26T21:54:11.907992+0000","last_became_active":"2024-04-26T21:53:10.919262+0000","last_became_peered":"2024-04-26T21:53:10.919262+0000","last_unstale":"2024-04-26T21:54:11.907992+0000","last_undegraded":"2024-04-26T21:54:11.907992+0000","last_fullsized":"2024-04-26T21:54:11.907992+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:15:35.719661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,8,2],"acting":[3,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:12.395203+0000","last_change":"2024-04-26T21:53:10.914854+0000","last_active":"2024-04-26T21:54:12.395203+0000","last_peered":"2024-04-26T21:54:12.395203+0000","last_clean":"2024-04-26T21:54:12.395203+0000","last_became_active":"2024-04-26T21:53:10.914458+0000","last_became_peered":"2024-04-26T21:53:10.914458+0000","last_unstale":"2024-04-26T21:54:12.395203+0000","last_undegraded":"2024-04-26T21:54:12.395203+0000","last_fullsized":"2024-04-26T21:54:12.395203+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:26:41.538053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,3],"acting":[6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.907533+0000","last_change":"2024-04-26T21:52:50.794580+0000","last_active":"2024-04-26T21:54:11.907533+0000","last_peered":"2024-04-26T21:54:11.907533+0000","last_clean":"2024-04-26T21:54:11.907533+0000","last_became_active":"2024-04-26T21:52:50.792398+0000","last_became_peered":"2024-04-26T21:52:50.792398+0000","last_unstale":"2024-04-26T21:54:11.907533+0000","last_undegraded":"2024-04-26T21:54:11.907533+0000","last_fullsized":"2024-04-26T21:54:11.907533+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:21:38.913644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.909622+0000","last_change":"2024-04-26T21:53:11.580576+0000","last_active":"2024-04-26T21:54:11.909622+0000","last_peered":"2024-04-26T21:54:11.909622+0000","last_clean":"2024-04-26T21:54:11.909622+0000","last_became_active":"2024-04-26T21:53:11.578632+0000","last_became_peered":"2024-04-26T21:53:11.578632+0000","last_unstale":"2024-04-26T21:54:11.909622+0000","last_undegraded":"2024-04-26T21:54:11.909622+0000","last_fullsized":"2024-04-26T21:54:11.909622+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:30:24.412417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,8],"acting":[1,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:12.396035+0000","last_change":"2024-04-26T21:52:50.792166+0000","last_active":"2024-04-26T21:54:12.396035+0000","last_peered":"2024-04-26T21:54:12.396035+0000","last_clean":"2024-04-26T21:54:12.396035+0000","last_became_active":"2024-04-26T21:52:50.790709+0000","last_became_peered":"2024-04-26T21:52:50.790709+0000","last_unstale":"2024-04-26T21:54:12.396035+0000","last_undegraded":"2024-04-26T21:54:12.396035+0000","last_fullsized":"2024-04-26T21:54:12.396035+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:05:48.200573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T21:54:11.915443+0000","last_change":"2024-04-26T21:53:10.930526+0000","last_active":"2024-04-26T21:54:11.915443+0000","last_peered":"2024-04-26T21:54:11.915443+0000","last_clean":"2024-04-26T21:54:11.915443+0000","last_became_active":"2024-04-26T21:53:10.928850+0000","last_became_peered":"2024-04-26T21:53:10.928850+0000","last_unstale":"2024-04-26T21:54:11.915443+0000","last_undegraded":"2024-04-26T21:54:11.915443+0000","last_fullsized":"2024-04-26T21:54:11.915443+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:52:50.063705+0000","last_clean_scrub_stamp":"2024-04-26T21:52:50.063705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:54:03.420855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":60229,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1496,"num_read_kb":814,"num_write":255,"num_write_kb":137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":622,"num_keys_recovered":7,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":548864,"data_stored":189315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":218,"ondisk_log_size":218,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":82,"seq":352187318319,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1015808,"data_stored":485396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":74,"seq":317827579958,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28684,"kb_used_data":1668,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716468,"statfs":{"total":95995035648,"available":95965663232,"internally_reserved":0,"allocated":1708032,"data_stored":1127478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":69,"seq":296352743486,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36352,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708800,"statfs":{"total":95995035648,"available":95957811200,"internally_reserved":0,"allocated":1040384,"data_stored":484442,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":61,"seq":261993005126,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36348,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708804,"statfs":{"total":95995035648,"available":95957815296,"internally_reserved":0,"allocated":1036288,"data_stored":484820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299467,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28612,"kb_used_data":1596,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716540,"statfs":{"total":95995035648,"available":95965736960,"internally_reserved":0,"allocated":1634304,"data_stored":1078426,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462995,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36336,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708816,"statfs":{"total":95995035648,"available":95957827584,"internally_reserved":0,"allocated":1024000,"data_stored":484652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574965,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28068,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717084,"statfs":{"total":95995035648,"available":95966294016,"internally_reserved":0,"allocated":1077248,"data_stored":537009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738493,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36356,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708796,"statfs":{"total":95995035648,"available":95957807104,"internally_reserved":0,"allocated":1044480,"data_stored":486382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411439,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36400,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708752,"statfs":{"total":95995035648,"available":95957762048,"internally_reserved":0,"allocated":1089536,"data_stored":536998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182503,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1040384,"data_stored":483984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920866,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36344,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708808,"statfs":{"total":95995035648,"available":95957819392,"internally_reserved":0,"allocated":1032192,"data_stored":485628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724632,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36944,"kb_used_data":1608,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708208,"statfs":{"total":95995035648,"available":95957204992,"internally_reserved":0,"allocated":1646592,"data_stored":1077118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":58003,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":57992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2308,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1215,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4621,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":53673,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":2076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T21:55:28.330 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:55:28.544 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:28.544+0000 7f6091c00640 1 -- 172.21.15.72:0/3388674760 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f608c104b90 msgr2=0x7f608c104fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:28.545 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:28.544+0000 7f6091c00640 1 --2- 172.21.15.72:0/3388674760 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f608c104b90 0x7f608c104fb0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f607c009900 tx=0x7f607c02a410 comp rx=0 tx=0).stop 2024-04-26T21:55:28.545 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:28.544+0000 7f6091c00640 1 -- 172.21.15.72:0/3388674760 shutdown_connections 2024-04-26T21:55:28.545 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:28.544+0000 7f6091c00640 1 --2- 172.21.15.72:0/3388674760 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f608c106740 0x7f608c10cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:28.545 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:28.544+0000 7f6091c00640 1 --2- 172.21.15.72:0/3388674760 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f608c105d80 0x7f608c106200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:28.545 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:28.544+0000 7f6091c00640 1 --2- 172.21.15.72:0/3388674760 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f608c104b90 0x7f608c104fb0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:28.545 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:28.544+0000 7f6091c00640 1 -- 172.21.15.72:0/3388674760 >> 172.21.15.72:0/3388674760 conn(0x7f608c100120 msgr2=0x7f608c102540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:55:28.545 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:28.544+0000 7f6091c00640 1 -- 172.21.15.72:0/3388674760 shutdown_connections 2024-04-26T21:55:28.545 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:28.544+0000 7f6091c00640 1 -- 172.21.15.72:0/3388674760 wait complete. 2024-04-26T21:55:28.547 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:28.545+0000 7f6091c00640 1 Processor -- start 2024-04-26T21:55:28.547 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:28.545+0000 7f6091c00640 1 -- start start 2024-04-26T21:55:28.547 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:28.546+0000 7f6091c00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f608c104b90 0x7f608c19ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:28.547 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:28.546+0000 7f6091c00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f608c105d80 0x7f608c19b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:28.547 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:28.546+0000 7f6091c00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f608c106740 0x7f608c1a2310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:28.548 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:28.546+0000 7f6091c00640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f608c1105e0 con 0x7f608c106740 2024-04-26T21:55:28.548 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:28.546+0000 7f6091c00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f608c110460 con 0x7f608c105d80 2024-04-26T21:55:28.548 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:28.546+0000 7f6091c00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f608c110760 con 0x7f608c104b90 2024-04-26T21:55:28.548 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:28.546+0000 7f608b400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f608c106740 0x7f608c1a2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:28.548 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:28.546+0000 7f608b400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f608c106740 0x7f608c1a2310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:44222/0 (socket says 172.21.15.72:44222) 2024-04-26T21:55:28.548 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:28.546+0000 7f608b400640 1 -- 172.21.15.72:0/1964527136 learned_addr learned my addr 172.21.15.72:0/1964527136 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:55:28.548 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:28.546+0000 7f608a000640 1 --2- 172.21.15.72:0/1964527136 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f608c105d80 0x7f608c19b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:28.548 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:28.546+0000 7f608aa00640 1 --2- 172.21.15.72:0/1964527136 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f608c104b90 0x7f608c19ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:28.548 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:28.546+0000 7f608b400640 1 -- 172.21.15.72:0/1964527136 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f608c104b90 msgr2=0x7f608c19ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:28.548 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:28.546+0000 7f608b400640 1 --2- 172.21.15.72:0/1964527136 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f608c104b90 0x7f608c19ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:28.548 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:28.546+0000 7f608b400640 1 -- 172.21.15.72:0/1964527136 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f608c105d80 msgr2=0x7f608c19b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:28.548 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:28.546+0000 7f608b400640 1 --2- 172.21.15.72:0/1964527136 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f608c105d80 0x7f608c19b2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:28.548 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:28.547+0000 7f608b400640 1 -- 172.21.15.72:0/1964527136 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f608c1a2980 con 0x7f608c106740 2024-04-26T21:55:28.548 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:28.547+0000 7f608a000640 1 --2- 172.21.15.72:0/1964527136 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f608c105d80 0x7f608c19b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:55:28.549 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:28.547+0000 7f608b400640 1 --2- 172.21.15.72:0/1964527136 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f608c106740 0x7f608c1a2310 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f608400eaa0 tx=0x7f608400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:55:28.549 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:28.547+0000 7f6083400640 1 -- 172.21.15.72:0/1964527136 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f608400cdb0 con 0x7f608c106740 2024-04-26T21:55:28.549 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:28.547+0000 7f6083400640 1 -- 172.21.15.72:0/1964527136 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6084005020 con 0x7f608c106740 2024-04-26T21:55:28.549 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:28.547+0000 7f6091c00640 1 -- 172.21.15.72:0/1964527136 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f608c1a2c70 con 0x7f608c106740 2024-04-26T21:55:28.549 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:28.547+0000 7f6083400640 1 -- 172.21.15.72:0/1964527136 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60840116d0 con 0x7f608c106740 2024-04-26T21:55:28.549 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:28.547+0000 7f6091c00640 1 -- 172.21.15.72:0/1964527136 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f608c1a31c0 con 0x7f608c106740 2024-04-26T21:55:28.549 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:28.549+0000 7f6091c00640 1 -- 172.21.15.72:0/1964527136 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f608c10bf70 con 0x7f608c106740 2024-04-26T21:55:28.550 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:28.550+0000 7f6083400640 1 -- 172.21.15.72:0/1964527136 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f60840051d0 con 0x7f608c106740 2024-04-26T21:55:28.552 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:28.552+0000 7f6083400640 1 --2- 172.21.15.72:0/1964527136 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f60700a3cc0 0x7f60700a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:28.552 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:28.552+0000 7f6083400640 1 -- 172.21.15.72:0/1964527136 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7f60840d0880 con 0x7f608c106740 2024-04-26T21:55:28.552 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:28.552+0000 7f608aa00640 1 --2- 172.21.15.72:0/1964527136 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f60700a3cc0 0x7f60700a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:28.553 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:28.553+0000 7f608aa00640 1 --2- 172.21.15.72:0/1964527136 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f60700a3cc0 0x7f60700a6180 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f607c006010 tx=0x7f607c031620 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:55:28.556 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:28.556+0000 7f6083400640 1 -- 172.21.15.72:0/1964527136 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f6084016030 con 0x7f608c106740 2024-04-26T21:55:28.782 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:28.782+0000 7f6091c00640 1 -- 172.21.15.72:0/1964527136 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f608c02dd20 con 0x7f608c106740 2024-04-26T21:55:28.784 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:28.784+0000 7f6083400640 1 -- 172.21.15.72:0/1964527136 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+22783 (secure 0 0 0) 0x7f608409d020 con 0x7f608c106740 2024-04-26T21:55:28.784 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T21:55:28.785 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":91,"fsid":"a1780950-0415-11ef-bc93-c7b262605968","created":"2024-04-26T21:42:01.504173+0000","modified":"2024-04-26T21:54:11.892872+0000","last_up_change":"2024-04-26T21:51:43.482437+0000","last_in_change":"2024-04-26T21:51:18.468587+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:46:29.785427+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:52:48.320391+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T21:52:49.517493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"04e06884-b355-4762-9b29-9034b7201e9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6802","nonce":1416765179},{"type":"v1","addr":"172.21.15.72:6803","nonce":1416765179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6804","nonce":1416765179},{"type":"v1","addr":"172.21.15.72:6805","nonce":1416765179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6808","nonce":1416765179},{"type":"v1","addr":"172.21.15.72:6809","nonce":1416765179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6806","nonce":1416765179},{"type":"v1","addr":"172.21.15.72:6807","nonce":1416765179}]},"public_addr":"172.21.15.72:6803/1416765179","cluster_addr":"172.21.15.72:6805/1416765179","heartbeat_back_addr":"172.21.15.72:6809/1416765179","heartbeat_front_addr":"172.21.15.72:6807/1416765179","state":["exists","up"]},{"osd":1,"uuid":"d75a479a-4a2e-48e9-a01e-2b1c9617416a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":755426949},{"type":"v1","addr":"172.21.15.76:6801","nonce":755426949}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":755426949},{"type":"v1","addr":"172.21.15.76:6803","nonce":755426949}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":755426949},{"type":"v1","addr":"172.21.15.76:6807","nonce":755426949}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":755426949},{"type":"v1","addr":"172.21.15.76:6805","nonce":755426949}]},"public_addr":"172.21.15.76:6801/755426949","cluster_addr":"172.21.15.76:6803/755426949","heartbeat_back_addr":"172.21.15.76:6807/755426949","heartbeat_front_addr":"172.21.15.76:6805/755426949","state":["exists","up"]},{"osd":2,"uuid":"b6f8da58-4428-4e8c-a944-7ea5645d6d92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":852265973},{"type":"v1","addr":"172.21.15.164:6801","nonce":852265973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":852265973},{"type":"v1","addr":"172.21.15.164:6803","nonce":852265973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":852265973},{"type":"v1","addr":"172.21.15.164:6807","nonce":852265973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":852265973},{"type":"v1","addr":"172.21.15.164:6805","nonce":852265973}]},"public_addr":"172.21.15.164:6801/852265973","cluster_addr":"172.21.15.164:6803/852265973","heartbeat_back_addr":"172.21.15.164:6807/852265973","heartbeat_front_addr":"172.21.15.164:6805/852265973","state":["exists","up"]},{"osd":3,"uuid":"76d83409-649e-41e5-b4f4-43bbd5b00ea5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6810","nonce":4027763789},{"type":"v1","addr":"172.21.15.72:6811","nonce":4027763789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6812","nonce":4027763789},{"type":"v1","addr":"172.21.15.72:6813","nonce":4027763789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6816","nonce":4027763789},{"type":"v1","addr":"172.21.15.72:6817","nonce":4027763789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6814","nonce":4027763789},{"type":"v1","addr":"172.21.15.72:6815","nonce":4027763789}]},"public_addr":"172.21.15.72:6811/4027763789","cluster_addr":"172.21.15.72:6813/4027763789","heartbeat_back_addr":"172.21.15.72:6817/4027763789","heartbeat_front_addr":"172.21.15.72:6815/4027763789","state":["exists","up"]},{"osd":4,"uuid":"d2cfeea7-1661-4156-bdcd-f770a8d1a878","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":3718867827},{"type":"v1","addr":"172.21.15.76:6809","nonce":3718867827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":3718867827},{"type":"v1","addr":"172.21.15.76:6811","nonce":3718867827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":3718867827},{"type":"v1","addr":"172.21.15.76:6815","nonce":3718867827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":3718867827},{"type":"v1","addr":"172.21.15.76:6813","nonce":3718867827}]},"public_addr":"172.21.15.76:6809/3718867827","cluster_addr":"172.21.15.76:6811/3718867827","heartbeat_back_addr":"172.21.15.76:6815/3718867827","heartbeat_front_addr":"172.21.15.76:6813/3718867827","state":["exists","up"]},{"osd":5,"uuid":"f3896ad6-a040-4ea3-a431-ff9b7fc05c36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":2543954772},{"type":"v1","addr":"172.21.15.164:6809","nonce":2543954772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":2543954772},{"type":"v1","addr":"172.21.15.164:6811","nonce":2543954772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":2543954772},{"type":"v1","addr":"172.21.15.164:6815","nonce":2543954772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":2543954772},{"type":"v1","addr":"172.21.15.164:6813","nonce":2543954772}]},"public_addr":"172.21.15.164:6809/2543954772","cluster_addr":"172.21.15.164:6811/2543954772","heartbeat_back_addr":"172.21.15.164:6815/2543954772","heartbeat_front_addr":"172.21.15.164:6813/2543954772","state":["exists","up"]},{"osd":6,"uuid":"d5d56650-be7e-4ba5-ba3a-d7d473294dab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6818","nonce":2080659027},{"type":"v1","addr":"172.21.15.72:6819","nonce":2080659027}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6820","nonce":2080659027},{"type":"v1","addr":"172.21.15.72:6821","nonce":2080659027}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6824","nonce":2080659027},{"type":"v1","addr":"172.21.15.72:6825","nonce":2080659027}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6822","nonce":2080659027},{"type":"v1","addr":"172.21.15.72:6823","nonce":2080659027}]},"public_addr":"172.21.15.72:6819/2080659027","cluster_addr":"172.21.15.72:6821/2080659027","heartbeat_back_addr":"172.21.15.72:6825/2080659027","heartbeat_front_addr":"172.21.15.72:6823/2080659027","state":["exists","up"]},{"osd":7,"uuid":"18975b5a-9309-4a0f-9f37-3ce09c2e745d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":3144631323},{"type":"v1","addr":"172.21.15.76:6817","nonce":3144631323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":3144631323},{"type":"v1","addr":"172.21.15.76:6819","nonce":3144631323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":3144631323},{"type":"v1","addr":"172.21.15.76:6823","nonce":3144631323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":3144631323},{"type":"v1","addr":"172.21.15.76:6821","nonce":3144631323}]},"public_addr":"172.21.15.76:6817/3144631323","cluster_addr":"172.21.15.76:6819/3144631323","heartbeat_back_addr":"172.21.15.76:6823/3144631323","heartbeat_front_addr":"172.21.15.76:6821/3144631323","state":["exists","up"]},{"osd":8,"uuid":"779e9c1e-8e97-48d4-a854-68ca13f38e0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":627888883},{"type":"v1","addr":"172.21.15.164:6817","nonce":627888883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":627888883},{"type":"v1","addr":"172.21.15.164:6819","nonce":627888883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":627888883},{"type":"v1","addr":"172.21.15.164:6823","nonce":627888883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":627888883},{"type":"v1","addr":"172.21.15.164:6821","nonce":627888883}]},"public_addr":"172.21.15.164:6817/627888883","cluster_addr":"172.21.15.164:6819/627888883","heartbeat_back_addr":"172.21.15.164:6823/627888883","heartbeat_front_addr":"172.21.15.164:6821/627888883","state":["exists","up"]},{"osd":9,"uuid":"d0317850-7b2d-4850-b2cf-0b919505c1fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6826","nonce":1273118485},{"type":"v1","addr":"172.21.15.72:6827","nonce":1273118485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6828","nonce":1273118485},{"type":"v1","addr":"172.21.15.72:6829","nonce":1273118485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6832","nonce":1273118485},{"type":"v1","addr":"172.21.15.72:6833","nonce":1273118485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6830","nonce":1273118485},{"type":"v1","addr":"172.21.15.72:6831","nonce":1273118485}]},"public_addr":"172.21.15.72:6827/1273118485","cluster_addr":"172.21.15.72:6829/1273118485","heartbeat_back_addr":"172.21.15.72:6833/1273118485","heartbeat_front_addr":"172.21.15.72:6831/1273118485","state":["exists","up"]},{"osd":10,"uuid":"e0aa38d8-6011-4d2a-b67b-5bff6d999e47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":2808954459},{"type":"v1","addr":"172.21.15.76:6825","nonce":2808954459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":2808954459},{"type":"v1","addr":"172.21.15.76:6827","nonce":2808954459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":2808954459},{"type":"v1","addr":"172.21.15.76:6831","nonce":2808954459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":2808954459},{"type":"v1","addr":"172.21.15.76:6829","nonce":2808954459}]},"public_addr":"172.21.15.76:6825/2808954459","cluster_addr":"172.21.15.76:6827/2808954459","heartbeat_back_addr":"172.21.15.76:6831/2808954459","heartbeat_front_addr":"172.21.15.76:6829/2808954459","state":["exists","up"]},{"osd":11,"uuid":"981b3d4d-916e-4937-9916-7cc21a891e43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":2354951784},{"type":"v1","addr":"172.21.15.164:6825","nonce":2354951784}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":2354951784},{"type":"v1","addr":"172.21.15.164:6827","nonce":2354951784}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":2354951784},{"type":"v1","addr":"172.21.15.164:6831","nonce":2354951784}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":2354951784},{"type":"v1","addr":"172.21.15.164:6829","nonce":2354951784}]},"public_addr":"172.21.15.164:6825/2354951784","cluster_addr":"172.21.15.164:6827/2354951784","heartbeat_back_addr":"172.21.15.164:6831/2354951784","heartbeat_front_addr":"172.21.15.164:6829/2354951784","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:18.997924+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:52.439081+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:46:25.893288+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:47:00.112807+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:47:35.116874+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:48:09.216344+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:48:45.028853+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:49:19.905397+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:49:55.030536+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:50:30.767662+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:51:06.423267+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:51:41.631211+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.1a","mappings":[{"from":5,"to":7}]},{"pgid":"3.1b","mappings":[{"from":4,"to":6}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.28","mappings":[{"from":5,"to":6}]},{"pgid":"3.2a","mappings":[{"from":3,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.72:0/416277784":"2024-04-27T21:43:09.602791+0000","172.21.15.72:6800/4010358672":"2024-04-27T21:43:09.602791+0000","172.21.15.72:6801/1012617371":"2024-04-27T21:42:34.684837+0000","172.21.15.72:6801/4010358672":"2024-04-27T21:43:09.602791+0000","172.21.15.72:0/3555448179":"2024-04-27T21:43:09.602791+0000","172.21.15.72:6800/1012617371":"2024-04-27T21:42:34.684837+0000","172.21.15.72:0/2920378761":"2024-04-27T21:42:34.684837+0000","172.21.15.72:0/1933740041":"2024-04-27T21:43:09.602791+0000","172.21.15.72:0/3225295775":"2024-04-27T21:42:34.684837+0000","172.21.15.72:0/2326016180":"2024-04-27T21:42:34.684837+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:55:28.786 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:28.786+0000 7f6091c00640 1 -- 172.21.15.72:0/1964527136 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f60700a3cc0 msgr2=0x7f60700a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:28.786 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:28.786+0000 7f6091c00640 1 --2- 172.21.15.72:0/1964527136 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f60700a3cc0 0x7f60700a6180 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f607c006010 tx=0x7f607c031620 comp rx=0 tx=0).stop 2024-04-26T21:55:28.786 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:28.786+0000 7f6091c00640 1 -- 172.21.15.72:0/1964527136 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f608c106740 msgr2=0x7f608c1a2310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:28.787 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:28.786+0000 7f6091c00640 1 --2- 172.21.15.72:0/1964527136 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f608c106740 0x7f608c1a2310 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f608400eaa0 tx=0x7f608400ef60 comp rx=0 tx=0).stop 2024-04-26T21:55:28.787 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:28.786+0000 7f6091c00640 1 -- 172.21.15.72:0/1964527136 shutdown_connections 2024-04-26T21:55:28.787 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:28.786+0000 7f6091c00640 1 --2- 172.21.15.72:0/1964527136 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f608c106740 0x7f608c1a2310 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:28.787 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:28.786+0000 7f6091c00640 1 --2- 172.21.15.72:0/1964527136 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f608c105d80 0x7f608c19b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:28.787 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:28.786+0000 7f6091c00640 1 --2- 172.21.15.72:0/1964527136 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f60700a3cc0 0x7f60700a6180 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:28.787 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:28.786+0000 7f6091c00640 1 --2- 172.21.15.72:0/1964527136 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f608c104b90 0x7f608c19ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:28.787 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:28.786+0000 7f6091c00640 1 -- 172.21.15.72:0/1964527136 >> 172.21.15.72:0/1964527136 conn(0x7f608c100120 msgr2=0x7f608c109140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:55:28.787 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:28.786+0000 7f6091c00640 1 -- 172.21.15.72:0/1964527136 shutdown_connections 2024-04-26T21:55:28.787 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:28.786+0000 7f6091c00640 1 -- 172.21.15.72:0/1964527136 wait complete. 2024-04-26T21:55:28.804 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:46:29.785427+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T21:52:48.320391+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T21:52:49.517493+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:55:28.805 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:55:28.983 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:28.983+0000 7f0f8a400640 1 -- 172.21.15.72:0/1319886829 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0f84102980 msgr2=0x7f0f84102da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:28.984 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:28.983+0000 7f0f8a400640 1 --2- 172.21.15.72:0/1319886829 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0f84102980 0x7f0f84102da0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f0f74009900 tx=0x7f0f7402a410 comp rx=0 tx=0).stop 2024-04-26T21:55:28.984 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:28.983+0000 7f0f8a400640 1 -- 172.21.15.72:0/1319886829 shutdown_connections 2024-04-26T21:55:28.984 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:28.983+0000 7f0f8a400640 1 --2- 172.21.15.72:0/1319886829 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0f84104530 0x7f0f8410adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:28.984 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:28.983+0000 7f0f8a400640 1 --2- 172.21.15.72:0/1319886829 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0f84103b70 0x7f0f84103ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:28.984 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:28.983+0000 7f0f8a400640 1 --2- 172.21.15.72:0/1319886829 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0f84102980 0x7f0f84102da0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:28.984 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:28.983+0000 7f0f8a400640 1 -- 172.21.15.72:0/1319886829 >> 172.21.15.72:0/1319886829 conn(0x7f0f840fdf10 msgr2=0x7f0f84100330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:55:28.984 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:28.983+0000 7f0f8a400640 1 -- 172.21.15.72:0/1319886829 shutdown_connections 2024-04-26T21:55:28.984 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:28.983+0000 7f0f8a400640 1 -- 172.21.15.72:0/1319886829 wait complete. 2024-04-26T21:55:28.986 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:28.984+0000 7f0f8a400640 1 Processor -- start 2024-04-26T21:55:28.986 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:28.984+0000 7f0f8a400640 1 -- start start 2024-04-26T21:55:28.987 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:28.985+0000 7f0f8a400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0f84102980 0x7f0f8419ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:28.987 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:28.985+0000 7f0f8a400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0f84103b70 0x7f0f8419b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:28.987 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:28.985+0000 7f0f8a400640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0f84104530 0x7f0f841a2310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:28.987 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:28.985+0000 7f0f8a400640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f0f8410e3a0 con 0x7f0f84102980 2024-04-26T21:55:28.987 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:28.985+0000 7f0f8a400640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f0f8410e220 con 0x7f0f84104530 2024-04-26T21:55:28.987 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:28.985+0000 7f0f8a400640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f0f8410e520 con 0x7f0f84103b70 2024-04-26T21:55:28.987 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:28.985+0000 7f0f83400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0f84102980 0x7f0f8419ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:28.987 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:28.985+0000 7f0f83400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0f84102980 0x7f0f8419ad60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:44248/0 (socket says 172.21.15.72:44248) 2024-04-26T21:55:28.987 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:28.985+0000 7f0f83400640 1 -- 172.21.15.72:0/903467341 learned_addr learned my addr 172.21.15.72:0/903467341 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:55:28.987 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:28.985+0000 7f0f82a00640 1 --2- 172.21.15.72:0/903467341 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0f84103b70 0x7f0f8419b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:28.987 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:28.985+0000 7f0f83e00640 1 --2- 172.21.15.72:0/903467341 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0f84104530 0x7f0f841a2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:28.987 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:28.985+0000 7f0f83400640 1 -- 172.21.15.72:0/903467341 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0f84103b70 msgr2=0x7f0f8419b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:28.988 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:28.986+0000 7f0f83400640 1 --2- 172.21.15.72:0/903467341 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0f84103b70 0x7f0f8419b2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:28.988 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:28.986+0000 7f0f83400640 1 -- 172.21.15.72:0/903467341 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0f84104530 msgr2=0x7f0f841a2310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:28.988 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:28.986+0000 7f0f83400640 1 --2- 172.21.15.72:0/903467341 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0f84104530 0x7f0f841a2310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:28.988 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:28.986+0000 7f0f83400640 1 -- 172.21.15.72:0/903467341 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f841a2980 con 0x7f0f84102980 2024-04-26T21:55:28.988 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:28.986+0000 7f0f83400640 1 --2- 172.21.15.72:0/903467341 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0f84102980 0x7f0f8419ad60 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f0f74031460 tx=0x7f0f74003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:55:28.988 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:28.986+0000 7f0f7be00640 1 -- 172.21.15.72:0/903467341 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f7403f070 con 0x7f0f84102980 2024-04-26T21:55:28.988 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:28.986+0000 7f0f7be00640 1 -- 172.21.15.72:0/903467341 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0f74031ba0 con 0x7f0f84102980 2024-04-26T21:55:28.988 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:28.986+0000 7f0f7be00640 1 -- 172.21.15.72:0/903467341 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f74035030 con 0x7f0f84102980 2024-04-26T21:55:28.988 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:28.986+0000 7f0f8a400640 1 -- 172.21.15.72:0/903467341 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f841a2c10 con 0x7f0f84102980 2024-04-26T21:55:28.988 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:28.986+0000 7f0f8a400640 1 -- 172.21.15.72:0/903467341 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f841a3150 con 0x7f0f84102980 2024-04-26T21:55:28.989 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:28.987+0000 7f0f8a400640 1 -- 172.21.15.72:0/903467341 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f841a33e0 con 0x7f0f84102980 2024-04-26T21:55:28.989 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:28.990+0000 7f0f7be00640 1 -- 172.21.15.72:0/903467341 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f0f74042070 con 0x7f0f84102980 2024-04-26T21:55:28.992 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:28.992+0000 7f0f7be00640 1 --2- 172.21.15.72:0/903467341 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f0f680a3cc0 0x7f0f680a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:28.992 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:28.992+0000 7f0f7be00640 1 -- 172.21.15.72:0/903467341 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7f0f740f1140 con 0x7f0f84102980 2024-04-26T21:55:28.992 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:28.992+0000 7f0f82a00640 1 --2- 172.21.15.72:0/903467341 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f0f680a3cc0 0x7f0f680a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:28.993 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:28.993+0000 7f0f82a00640 1 --2- 172.21.15.72:0/903467341 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f0f680a3cc0 0x7f0f680a6180 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f0f8419c270 tx=0x7f0f70005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:55:28.996 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:28.996+0000 7f0f7be00640 1 -- 172.21.15.72:0/903467341 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f0f740bd940 con 0x7f0f84102980 2024-04-26T21:55:29.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:55:28 smithi072 ceph-mon[28138]: from='client.15441 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T21:55:29.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:55:28 smithi072 ceph-mon[28138]: from='client.15447 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T21:55:29.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:55:28 smithi072 ceph-mon[28138]: pgmap v450: 97 pgs: 97 active+clean; 636 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-04-26T21:55:29.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:55:28 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:55:29.073 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:55:28 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:55:29.073 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:55:28 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:55:29.073 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:55:28 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:55:29.073 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:55:28 smithi072 ceph-mon[28138]: from='client.? 172.21.15.72:0/1964527136' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:55:29.199 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:29.199+0000 7f0f8a400640 1 -- 172.21.15.72:0/903467341 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f0f841a36d0 con 0x7f0f84102980 2024-04-26T21:55:29.200 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:29.200+0000 7f0f7be00640 1 -- 172.21.15.72:0/903467341 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) ==== 93+0+10 (secure 0 0 0) 0x7f0f740bd240 con 0x7f0f84102980 2024-04-26T21:55:29.200 INFO:teuthology.orchestra.run.smithi072.stdout:pg_num: 1 2024-04-26T21:55:29.202 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:29.202+0000 7f0f8a400640 1 -- 172.21.15.72:0/903467341 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f0f680a3cc0 msgr2=0x7f0f680a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:29.203 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:29.202+0000 7f0f8a400640 1 --2- 172.21.15.72:0/903467341 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f0f680a3cc0 0x7f0f680a6180 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f0f8419c270 tx=0x7f0f70005f90 comp rx=0 tx=0).stop 2024-04-26T21:55:29.203 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:29.202+0000 7f0f8a400640 1 -- 172.21.15.72:0/903467341 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0f84102980 msgr2=0x7f0f8419ad60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:29.203 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:29.202+0000 7f0f8a400640 1 --2- 172.21.15.72:0/903467341 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0f84102980 0x7f0f8419ad60 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f0f74031460 tx=0x7f0f74003c10 comp rx=0 tx=0).stop 2024-04-26T21:55:29.203 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:29.202+0000 7f0f8a400640 1 -- 172.21.15.72:0/903467341 shutdown_connections 2024-04-26T21:55:29.203 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:29.202+0000 7f0f8a400640 1 --2- 172.21.15.72:0/903467341 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0f84104530 0x7f0f841a2310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:29.203 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:29.202+0000 7f0f8a400640 1 --2- 172.21.15.72:0/903467341 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f0f680a3cc0 0x7f0f680a6180 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:29.203 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:29.202+0000 7f0f8a400640 1 --2- 172.21.15.72:0/903467341 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0f84103b70 0x7f0f8419b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:29.203 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:29.202+0000 7f0f8a400640 1 --2- 172.21.15.72:0/903467341 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0f84102980 0x7f0f8419ad60 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:29.203 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:29.202+0000 7f0f8a400640 1 -- 172.21.15.72:0/903467341 >> 172.21.15.72:0/903467341 conn(0x7f0f840fdf10 msgr2=0x7f0f84109400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:55:29.203 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:29.203+0000 7f0f8a400640 1 -- 172.21.15.72:0/903467341 shutdown_connections 2024-04-26T21:55:29.203 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:29.203+0000 7f0f8a400640 1 -- 172.21.15.72:0/903467341 wait complete. 2024-04-26T21:55:29.221 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T21:55:29.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:55:28 smithi076 ceph-mon[31823]: from='client.15441 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T21:55:29.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:55:28 smithi076 ceph-mon[31823]: from='client.15447 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T21:55:29.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:55:28 smithi076 ceph-mon[31823]: pgmap v450: 97 pgs: 97 active+clean; 636 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-04-26T21:55:29.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:55:28 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:55:29.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:55:28 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:55:29.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:55:28 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:55:29.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:55:28 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:55:29.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:55:28 smithi076 ceph-mon[31823]: from='client.? 172.21.15.72:0/1964527136' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:55:29.420 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:29.420+0000 7fbba6c00640 1 -- 172.21.15.72:0/3736348347 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbba0108580 msgr2=0x7fbba010a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:29.421 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:29.420+0000 7fbba6c00640 1 --2- 172.21.15.72:0/3736348347 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbba0108580 0x7fbba010a970 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fbb98009be0 tx=0x7fbb9802a6c0 comp rx=0 tx=0).stop 2024-04-26T21:55:29.421 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:29.420+0000 7fbba6c00640 1 -- 172.21.15.72:0/3736348347 shutdown_connections 2024-04-26T21:55:29.421 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:29.420+0000 7fbba6c00640 1 --2- 172.21.15.72:0/3736348347 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbba0108580 0x7fbba010a970 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:29.421 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:29.420+0000 7fbba6c00640 1 --2- 172.21.15.72:0/3736348347 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbba0105c50 0x7fbba0108040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:29.421 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:29.420+0000 7fbba6c00640 1 --2- 172.21.15.72:0/3736348347 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbba0103320 0x7fbba0105710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:29.421 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:29.420+0000 7fbba6c00640 1 -- 172.21.15.72:0/3736348347 >> 172.21.15.72:0/3736348347 conn(0x7fbba00fccd0 msgr2=0x7fbba00ff0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:55:29.421 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:29.420+0000 7fbba6c00640 1 -- 172.21.15.72:0/3736348347 shutdown_connections 2024-04-26T21:55:29.421 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:29.420+0000 7fbba6c00640 1 -- 172.21.15.72:0/3736348347 wait complete. 2024-04-26T21:55:29.421 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:29.421+0000 7fbba6c00640 1 Processor -- start 2024-04-26T21:55:29.422 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:29.422+0000 7fbba6c00640 1 -- start start 2024-04-26T21:55:29.423 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:29.422+0000 7fbba6c00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbba0103320 0x7fbba0102c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:29.423 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:29.422+0000 7fbba6c00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbba0105c50 0x7fbba01012d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:29.423 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:29.422+0000 7fbba6c00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbba0108580 0x7fbba0101810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:29.423 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:29.422+0000 7fbba6c00640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fbba010e220 con 0x7fbba0103320 2024-04-26T21:55:29.423 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:29.422+0000 7fbba6c00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fbba010e0a0 con 0x7fbba0108580 2024-04-26T21:55:29.423 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:29.422+0000 7fbba6c00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fbba010e3a0 con 0x7fbba0105c50 2024-04-26T21:55:29.423 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:29.422+0000 7fbba5800640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbba0103320 0x7fbba0102c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:29.424 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:29.422+0000 7fbba5800640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbba0103320 0x7fbba0102c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:44272/0 (socket says 172.21.15.72:44272) 2024-04-26T21:55:29.424 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:29.422+0000 7fbba5800640 1 -- 172.21.15.72:0/3222408620 learned_addr learned my addr 172.21.15.72:0/3222408620 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:55:29.424 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:29.423+0000 7fbba4e00640 1 --2- 172.21.15.72:0/3222408620 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbba0105c50 0x7fbba01012d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:29.424 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:29.423+0000 7fbba6200640 1 --2- 172.21.15.72:0/3222408620 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbba0108580 0x7fbba0101810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:29.424 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:29.423+0000 7fbba5800640 1 -- 172.21.15.72:0/3222408620 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbba0105c50 msgr2=0x7fbba01012d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:29.424 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:29.423+0000 7fbba5800640 1 --2- 172.21.15.72:0/3222408620 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbba0105c50 0x7fbba01012d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:29.424 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:29.423+0000 7fbba5800640 1 -- 172.21.15.72:0/3222408620 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbba0108580 msgr2=0x7fbba0101810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:29.424 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:29.423+0000 7fbba5800640 1 --2- 172.21.15.72:0/3222408620 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbba0108580 0x7fbba0101810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:29.424 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:29.423+0000 7fbba5800640 1 -- 172.21.15.72:0/3222408620 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbba01020a0 con 0x7fbba0103320 2024-04-26T21:55:29.424 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:29.423+0000 7fbba5800640 1 --2- 172.21.15.72:0/3222408620 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbba0103320 0x7fbba0102c90 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fbb9400e9f0 tx=0x7fbb9400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:55:29.425 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:29.424+0000 7fbb9e000640 1 -- 172.21.15.72:0/3222408620 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb9400cd80 con 0x7fbba0103320 2024-04-26T21:55:29.425 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:29.424+0000 7fbba6c00640 1 -- 172.21.15.72:0/3222408620 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbba01a2a40 con 0x7fbba0103320 2024-04-26T21:55:29.425 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:29.424+0000 7fbb9e000640 1 -- 172.21.15.72:0/3222408620 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbb940108c0 con 0x7fbba0103320 2024-04-26T21:55:29.425 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:29.424+0000 7fbb9e000640 1 -- 172.21.15.72:0/3222408620 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb940187c0 con 0x7fbba0103320 2024-04-26T21:55:29.425 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:29.424+0000 7fbba6c00640 1 -- 172.21.15.72:0/3222408620 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbba01a2f90 con 0x7fbba0103320 2024-04-26T21:55:29.425 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:29.425+0000 7fbba6c00640 1 -- 172.21.15.72:0/3222408620 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbba0069670 con 0x7fbba0103320 2024-04-26T21:55:29.427 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:29.427+0000 7fbb9e000640 1 -- 172.21.15.72:0/3222408620 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fbb94010430 con 0x7fbba0103320 2024-04-26T21:55:29.428 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:29.428+0000 7fbb9e000640 1 --2- 172.21.15.72:0/3222408620 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fbb880a3c40 0x7fbb880a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:29.428 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:29.428+0000 7fbb9e000640 1 -- 172.21.15.72:0/3222408620 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7fbb940d09e0 con 0x7fbba0103320 2024-04-26T21:55:29.428 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:29.428+0000 7fbba4e00640 1 --2- 172.21.15.72:0/3222408620 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fbb880a3c40 0x7fbb880a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:29.429 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:29.429+0000 7fbba4e00640 1 --2- 172.21.15.72:0/3222408620 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fbb880a3c40 0x7fbb880a6100 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fbba0102a00 tx=0x7fbb8c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:55:29.432 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:29.432+0000 7fbb9e000640 1 -- 172.21.15.72:0/3222408620 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fbb9409d1e0 con 0x7fbba0103320 2024-04-26T21:55:29.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:55:28 smithi164 ceph-mon[31916]: from='client.15441 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T21:55:29.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:55:28 smithi164 ceph-mon[31916]: from='client.15447 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T21:55:29.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:55:28 smithi164 ceph-mon[31916]: pgmap v450: 97 pgs: 97 active+clean; 636 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-04-26T21:55:29.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:55:28 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:55:29.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:55:28 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:55:29.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:55:28 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:55:29.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:55:28 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:55:29.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:55:28 smithi164 ceph-mon[31916]: from='client.? 172.21.15.72:0/1964527136' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:55:29.638 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:29.638+0000 7fbba6c00640 1 -- 172.21.15.72:0/3222408620 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fbba0063530 con 0x7fbba0103320 2024-04-26T21:55:29.639 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:29.639+0000 7fbb9e000640 1 -- 172.21.15.72:0/3222408620 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v91) ==== 104+0+11 (secure 0 0 0) 0x7fbb9409cae0 con 0x7fbba0103320 2024-04-26T21:55:29.639 INFO:teuthology.orchestra.run.smithi072.stdout:pg_num: 32 2024-04-26T21:55:29.642 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:29.641+0000 7fbba6c00640 1 -- 172.21.15.72:0/3222408620 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fbb880a3c40 msgr2=0x7fbb880a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:29.642 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:29.641+0000 7fbba6c00640 1 --2- 172.21.15.72:0/3222408620 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fbb880a3c40 0x7fbb880a6100 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fbba0102a00 tx=0x7fbb8c009260 comp rx=0 tx=0).stop 2024-04-26T21:55:29.642 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:29.641+0000 7fbba6c00640 1 -- 172.21.15.72:0/3222408620 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbba0103320 msgr2=0x7fbba0102c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:29.642 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:29.641+0000 7fbba6c00640 1 --2- 172.21.15.72:0/3222408620 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbba0103320 0x7fbba0102c90 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fbb9400e9f0 tx=0x7fbb9400eeb0 comp rx=0 tx=0).stop 2024-04-26T21:55:29.642 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:29.642+0000 7fbba6c00640 1 -- 172.21.15.72:0/3222408620 shutdown_connections 2024-04-26T21:55:29.642 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:29.642+0000 7fbba6c00640 1 --2- 172.21.15.72:0/3222408620 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbba0108580 0x7fbba0101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:29.642 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:29.642+0000 7fbba6c00640 1 --2- 172.21.15.72:0/3222408620 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fbb880a3c40 0x7fbb880a6100 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:29.642 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:29.642+0000 7fbba6c00640 1 --2- 172.21.15.72:0/3222408620 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbba0105c50 0x7fbba01012d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:29.642 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:29.642+0000 7fbba6c00640 1 --2- 172.21.15.72:0/3222408620 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbba0103320 0x7fbba0102c90 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:29.643 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:29.642+0000 7fbba6c00640 1 -- 172.21.15.72:0/3222408620 >> 172.21.15.72:0/3222408620 conn(0x7fbba00fccd0 msgr2=0x7fbba0103c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:55:29.643 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:29.642+0000 7fbba6c00640 1 -- 172.21.15.72:0/3222408620 shutdown_connections 2024-04-26T21:55:29.643 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:29.642+0000 7fbba6c00640 1 -- 172.21.15.72:0/3222408620 wait complete. 2024-04-26T21:55:29.660 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T21:55:29.855 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:29.854+0000 7f297f200640 1 -- 172.21.15.72:0/3183108110 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2978105b00 msgr2=0x7f2978107ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:29.855 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:29.854+0000 7f297f200640 1 --2- 172.21.15.72:0/3183108110 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2978105b00 0x7f2978107ef0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f2968009900 tx=0x7f296802a3f0 comp rx=0 tx=0).stop 2024-04-26T21:55:29.855 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:29.854+0000 7f297f200640 1 -- 172.21.15.72:0/3183108110 shutdown_connections 2024-04-26T21:55:29.855 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:29.854+0000 7f297f200640 1 --2- 172.21.15.72:0/3183108110 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2978108430 0x7f297810a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:29.855 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:29.854+0000 7f297f200640 1 --2- 172.21.15.72:0/3183108110 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2978105b00 0x7f2978107ef0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:29.855 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:29.854+0000 7f297f200640 1 --2- 172.21.15.72:0/3183108110 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f29781031d0 0x7f29781055c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:29.855 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:29.854+0000 7f297f200640 1 -- 172.21.15.72:0/3183108110 >> 172.21.15.72:0/3183108110 conn(0x7f29780fcc00 msgr2=0x7f29780ff020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:55:29.855 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:29.854+0000 7f297f200640 1 -- 172.21.15.72:0/3183108110 shutdown_connections 2024-04-26T21:55:29.855 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:29.855+0000 7f297f200640 1 -- 172.21.15.72:0/3183108110 wait complete. 2024-04-26T21:55:29.857 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:29.855+0000 7f297f200640 1 Processor -- start 2024-04-26T21:55:29.857 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:29.855+0000 7f297f200640 1 -- start start 2024-04-26T21:55:29.857 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:29.856+0000 7f297f200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f29781031d0 0x7f2978198790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:29.857 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:29.856+0000 7f297f200640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2978105b00 0x7f2978198cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:29.858 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:29.856+0000 7f297f200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2978108430 0x7f297819fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:29.858 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:29.856+0000 7f297f200640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f297810e170 con 0x7f2978105b00 2024-04-26T21:55:29.858 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:29.856+0000 7f297f200640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f297810dff0 con 0x7f29781031d0 2024-04-26T21:55:29.858 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:29.856+0000 7f297f200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f297810e2f0 con 0x7f2978108430 2024-04-26T21:55:29.858 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:29.856+0000 7f297d400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2978105b00 0x7f2978198cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:29.858 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:29.856+0000 7f297d400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2978105b00 0x7f2978198cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:44286/0 (socket says 172.21.15.72:44286) 2024-04-26T21:55:29.858 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:29.856+0000 7f297d400640 1 -- 172.21.15.72:0/1512029150 learned_addr learned my addr 172.21.15.72:0/1512029150 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:55:29.858 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:29.856+0000 7f297de00640 1 --2- 172.21.15.72:0/1512029150 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f29781031d0 0x7f2978198790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:29.858 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:29.856+0000 7f297e800640 1 --2- 172.21.15.72:0/1512029150 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2978108430 0x7f297819fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:29.858 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:29.857+0000 7f297d400640 1 -- 172.21.15.72:0/1512029150 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2978108430 msgr2=0x7f297819fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:29.858 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:29.857+0000 7f297d400640 1 --2- 172.21.15.72:0/1512029150 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2978108430 0x7f297819fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:29.858 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:29.857+0000 7f297d400640 1 -- 172.21.15.72:0/1512029150 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f29781031d0 msgr2=0x7f2978198790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:29.858 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:29.857+0000 7f297d400640 1 --2- 172.21.15.72:0/1512029150 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f29781031d0 0x7f2978198790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:29.859 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:29.857+0000 7f297d400640 1 -- 172.21.15.72:0/1512029150 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29781a03b0 con 0x7f2978105b00 2024-04-26T21:55:29.859 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:29.857+0000 7f297d400640 1 --2- 172.21.15.72:0/1512029150 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2978105b00 0x7f2978198cd0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f2968031460 tx=0x7f2968003d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:55:29.859 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:29.857+0000 7f2972a00640 1 -- 172.21.15.72:0/1512029150 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f296803f070 con 0x7f2978105b00 2024-04-26T21:55:29.859 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:29.857+0000 7f2972a00640 1 -- 172.21.15.72:0/1512029150 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2968031da0 con 0x7f2978105b00 2024-04-26T21:55:29.859 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:29.857+0000 7f297f200640 1 -- 172.21.15.72:0/1512029150 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29781a0640 con 0x7f2978105b00 2024-04-26T21:55:29.859 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:29.857+0000 7f2972a00640 1 -- 172.21.15.72:0/1512029150 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2968035030 con 0x7f2978105b00 2024-04-26T21:55:29.859 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:29.857+0000 7f297f200640 1 -- 172.21.15.72:0/1512029150 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29781a0b30 con 0x7f2978105b00 2024-04-26T21:55:29.859 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:29.859+0000 7f297f200640 1 -- 172.21.15.72:0/1512029150 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29780fe6f0 con 0x7f2978105b00 2024-04-26T21:55:29.861 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:29.861+0000 7f2972a00640 1 -- 172.21.15.72:0/1512029150 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f2968042070 con 0x7f2978105b00 2024-04-26T21:55:29.862 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:29.862+0000 7f2972a00640 1 --2- 172.21.15.72:0/1512029150 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f29600a3c40 0x7f29600a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:29.862 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:29.862+0000 7f2972a00640 1 -- 172.21.15.72:0/1512029150 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7f29680f10c0 con 0x7f2978105b00 2024-04-26T21:55:29.862 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:29.862+0000 7f297de00640 1 --2- 172.21.15.72:0/1512029150 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f29600a3c40 0x7f29600a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:29.863 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:29.863+0000 7f297de00640 1 --2- 172.21.15.72:0/1512029150 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f29600a3c40 0x7f29600a6100 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f296c005e10 tx=0x7f296c005da0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:55:29.869 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:29.869+0000 7f2972a00640 1 -- 172.21.15.72:0/1512029150 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f29680bd860 con 0x7f2978105b00 2024-04-26T21:55:30.074 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:30.073+0000 7f297f200640 1 -- 172.21.15.72:0/1512029150 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f297802dd20 con 0x7f2978105b00 2024-04-26T21:55:30.074 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:30.075+0000 7f2972a00640 1 -- 172.21.15.72:0/1512029150 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v91) ==== 100+0+11 (secure 0 0 0) 0x7f29680bd160 con 0x7f2978105b00 2024-04-26T21:55:30.075 INFO:teuthology.orchestra.run.smithi072.stdout:pg_num: 64 2024-04-26T21:55:30.077 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:30.077+0000 7f297f200640 1 -- 172.21.15.72:0/1512029150 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f29600a3c40 msgr2=0x7f29600a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:30.077 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:30.077+0000 7f297f200640 1 --2- 172.21.15.72:0/1512029150 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f29600a3c40 0x7f29600a6100 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f296c005e10 tx=0x7f296c005da0 comp rx=0 tx=0).stop 2024-04-26T21:55:30.077 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:30.077+0000 7f297f200640 1 -- 172.21.15.72:0/1512029150 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2978105b00 msgr2=0x7f2978198cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:30.077 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:30.077+0000 7f297f200640 1 --2- 172.21.15.72:0/1512029150 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2978105b00 0x7f2978198cd0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f2968031460 tx=0x7f2968003d50 comp rx=0 tx=0).stop 2024-04-26T21:55:30.077 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:30.077+0000 7f297f200640 1 -- 172.21.15.72:0/1512029150 shutdown_connections 2024-04-26T21:55:30.077 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:30.077+0000 7f297f200640 1 --2- 172.21.15.72:0/1512029150 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f29600a3c40 0x7f29600a6100 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:30.077 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:30.077+0000 7f297f200640 1 --2- 172.21.15.72:0/1512029150 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2978108430 0x7f297819fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:30.078 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:30.077+0000 7f297f200640 1 --2- 172.21.15.72:0/1512029150 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2978105b00 0x7f2978198cd0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:30.078 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:30.077+0000 7f297f200640 1 --2- 172.21.15.72:0/1512029150 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f29781031d0 0x7f2978198790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:30.078 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:30.077+0000 7f297f200640 1 -- 172.21.15.72:0/1512029150 >> 172.21.15.72:0/1512029150 conn(0x7f29780fcc00 msgr2=0x7f2978103b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:55:30.078 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:30.077+0000 7f297f200640 1 -- 172.21.15.72:0/1512029150 shutdown_connections 2024-04-26T21:55:30.078 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:30.077+0000 7f297f200640 1 -- 172.21.15.72:0/1512029150 wait complete. 2024-04-26T21:55:30.095 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:55:30.292 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:30.291+0000 7fe9cf200640 1 -- 172.21.15.72:0/3931675291 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe9d0104300 msgr2=0x7fe9d010ab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:30.292 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:30.291+0000 7fe9cf200640 1 --2- 172.21.15.72:0/3931675291 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe9d0104300 0x7fe9d010ab90 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fe9c8009ca0 tx=0x7fe9c802a8c0 comp rx=0 tx=0).stop 2024-04-26T21:55:30.292 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:30.292+0000 7fe9cf200640 1 -- 172.21.15.72:0/3931675291 shutdown_connections 2024-04-26T21:55:30.292 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:30.292+0000 7fe9cf200640 1 --2- 172.21.15.72:0/3931675291 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe9d0104300 0x7fe9d010ab90 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:30.292 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:30.292+0000 7fe9cf200640 1 --2- 172.21.15.72:0/3931675291 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe9d0103940 0x7fe9d0103dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:30.292 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:30.292+0000 7fe9cf200640 1 --2- 172.21.15.72:0/3931675291 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe9d0102750 0x7fe9d0102b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:30.292 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:30.292+0000 7fe9cf200640 1 -- 172.21.15.72:0/3931675291 >> 172.21.15.72:0/3931675291 conn(0x7fe9d00fdcc0 msgr2=0x7fe9d0100100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:55:30.292 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:30.292+0000 7fe9cf200640 1 -- 172.21.15.72:0/3931675291 shutdown_connections 2024-04-26T21:55:30.293 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:30.292+0000 7fe9cf200640 1 -- 172.21.15.72:0/3931675291 wait complete. 2024-04-26T21:55:30.295 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:30.293+0000 7fe9cf200640 1 Processor -- start 2024-04-26T21:55:30.296 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:30.293+0000 7fe9cf200640 1 -- start start 2024-04-26T21:55:30.296 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:30.293+0000 7fe9cf200640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe9d0102750 0x7fe9d01987a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:30.296 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:30.293+0000 7fe9cf200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe9d0103940 0x7fe9d0198ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:30.296 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:30.293+0000 7fe9cf200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe9d0104300 0x7fe9d019fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:30.296 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:30.293+0000 7fe9cf200640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fe9d010e180 con 0x7fe9d0102750 2024-04-26T21:55:30.296 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:30.293+0000 7fe9cf200640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fe9d010e000 con 0x7fe9d0103940 2024-04-26T21:55:30.296 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:30.293+0000 7fe9cf200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe9d010e300 con 0x7fe9d0104300 2024-04-26T21:55:30.296 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:30.293+0000 7fe9cde00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe9d0102750 0x7fe9d01987a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:30.296 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:30.293+0000 7fe9cde00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe9d0102750 0x7fe9d01987a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:44310/0 (socket says 172.21.15.72:44310) 2024-04-26T21:55:30.296 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:30.293+0000 7fe9cde00640 1 -- 172.21.15.72:0/2099086773 learned_addr learned my addr 172.21.15.72:0/2099086773 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:55:30.296 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:30.293+0000 7fe9ce800640 1 --2- 172.21.15.72:0/2099086773 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe9d0104300 0x7fe9d019fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:30.296 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:30.294+0000 7fe9c7e00640 1 --2- 172.21.15.72:0/2099086773 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe9d0103940 0x7fe9d0198ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:30.296 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:30.294+0000 7fe9cde00640 1 -- 172.21.15.72:0/2099086773 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe9d0104300 msgr2=0x7fe9d019fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:30.296 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:30.294+0000 7fe9cde00640 1 --2- 172.21.15.72:0/2099086773 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe9d0104300 0x7fe9d019fd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:30.296 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:30.294+0000 7fe9cde00640 1 -- 172.21.15.72:0/2099086773 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe9d0103940 msgr2=0x7fe9d0198ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:30.297 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:30.294+0000 7fe9cde00640 1 --2- 172.21.15.72:0/2099086773 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe9d0103940 0x7fe9d0198ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:30.297 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:30.294+0000 7fe9cde00640 1 -- 172.21.15.72:0/2099086773 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9d01a03c0 con 0x7fe9d0102750 2024-04-26T21:55:30.297 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:30.294+0000 7fe9ce800640 1 --2- 172.21.15.72:0/2099086773 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe9d0104300 0x7fe9d019fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:55:30.297 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:30.294+0000 7fe9cde00640 1 --2- 172.21.15.72:0/2099086773 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe9d0102750 0x7fe9d01987a0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fe9c000d740 tx=0x7fe9c000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:55:30.297 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:30.294+0000 7fe9c6a00640 1 -- 172.21.15.72:0/2099086773 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9c0017070 con 0x7fe9d0102750 2024-04-26T21:55:30.297 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:30.294+0000 7fe9c6a00640 1 -- 172.21.15.72:0/2099086773 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe9c000de50 con 0x7fe9d0102750 2024-04-26T21:55:30.297 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:30.294+0000 7fe9c6a00640 1 -- 172.21.15.72:0/2099086773 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9c0004b90 con 0x7fe9d0102750 2024-04-26T21:55:30.297 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:30.294+0000 7fe9cf200640 1 -- 172.21.15.72:0/2099086773 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9d01a06b0 con 0x7fe9d0102750 2024-04-26T21:55:30.297 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:30.295+0000 7fe9cf200640 1 -- 172.21.15.72:0/2099086773 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9d01a0bb0 con 0x7fe9d0102750 2024-04-26T21:55:30.297 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:30.296+0000 7fe9cf200640 1 -- 172.21.15.72:0/2099086773 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9d0109a70 con 0x7fe9d0102750 2024-04-26T21:55:30.298 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:30.298+0000 7fe9c6a00640 1 -- 172.21.15.72:0/2099086773 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fe9c0004d30 con 0x7fe9d0102750 2024-04-26T21:55:30.299 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:30.299+0000 7fe9c6a00640 1 --2- 172.21.15.72:0/2099086773 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fe9b40a3cc0 0x7fe9b40a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:30.299 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:30.299+0000 7fe9c6a00640 1 -- 172.21.15.72:0/2099086773 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7fe9c00d0060 con 0x7fe9d0102750 2024-04-26T21:55:30.299 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:30.299+0000 7fe9c7e00640 1 --2- 172.21.15.72:0/2099086773 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fe9b40a3cc0 0x7fe9b40a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:30.300 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:30.300+0000 7fe9c7e00640 1 --2- 172.21.15.72:0/2099086773 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fe9b40a3cc0 0x7fe9b40a6180 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fe9b8005e10 tx=0x7fe9b8005da0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:55:30.303 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:30.303+0000 7fe9c6a00640 1 -- 172.21.15.72:0/2099086773 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fe9c009c860 con 0x7fe9d0102750 2024-04-26T21:55:30.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:55:29 smithi076 ceph-mon[31823]: from='client.15453 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T21:55:30.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:55:29 smithi076 ceph-mon[31823]: from='client.? 172.21.15.72:0/903467341' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T21:55:30.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:55:29 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:55:30.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:55:29 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:55:30.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:55:29 smithi076 ceph-mon[31823]: from='client.? 172.21.15.72:0/3222408620' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T21:55:30.321 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:55:29 smithi072 ceph-mon[28138]: from='client.15453 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T21:55:30.321 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:55:29 smithi072 ceph-mon[28138]: from='client.? 172.21.15.72:0/903467341' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T21:55:30.321 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:55:29 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:55:30.321 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:55:29 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:55:30.321 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:55:29 smithi072 ceph-mon[28138]: from='client.? 172.21.15.72:0/3222408620' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T21:55:30.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:55:29 smithi164 ceph-mon[31916]: from='client.15453 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T21:55:30.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:55:29 smithi164 ceph-mon[31916]: from='client.? 172.21.15.72:0/903467341' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T21:55:30.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:55:29 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:55:30.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:55:29 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:55:30.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:55:29 smithi164 ceph-mon[31916]: from='client.? 172.21.15.72:0/3222408620' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T21:55:30.545 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:30.544+0000 7fe9cf200640 1 -- 172.21.15.72:0/2099086773 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe9d0063530 con 0x7fe9d0102750 2024-04-26T21:55:30.546 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:30.546+0000 7fe9c6a00640 1 -- 172.21.15.72:0/2099086773 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11931 (secure 0 0 0) 0x7fe9c009c160 con 0x7fe9d0102750 2024-04-26T21:55:30.546 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T21:55:30.546 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6835/1320381959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6834","nonce":1320381959},{"type":"v1","addr":"172.21.15.72:6835","nonce":1320381959}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6837/2142200512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6836","nonce":2142200512},{"type":"v1","addr":"172.21.15.72:6837","nonce":2142200512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T21:52:50.567857+0000","modified":"2024-04-26T21:55:22.894383+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34517,"mds_1":34472,"mds_2":34442,"mds_3":34406,"mds_4":24463},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14652":{"gid":14652,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.72:6839/1312898281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6838","nonce":1312898281},{"type":"v1","addr":"172.21.15.72:6839","nonce":1312898281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14664":{"gid":14664,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.72:6841/3257692339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6840","nonce":3257692339},{"type":"v1","addr":"172.21.15.72:6841","nonce":3257692339}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24388":{"gid":24388,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6833/4236668465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":4236668465},{"type":"v1","addr":"172.21.15.164:6833","nonce":4236668465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6835/2391114169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":2391114169},{"type":"v1","addr":"172.21.15.164:6835","nonce":2391114169}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6837/3585829992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":3585829992},{"type":"v1","addr":"172.21.15.164:6837","nonce":3585829992}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.164:6839/1636226820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":1636226820},{"type":"v1","addr":"172.21.15.164:6839","nonce":1636226820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34406":{"gid":34406,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":15,"addr":"172.21.15.76:6833/3869483750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":3869483750},{"type":"v1","addr":"172.21.15.76:6833","nonce":3869483750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34442":{"gid":34442,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.76:6835/3517429435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":3517429435},{"type":"v1","addr":"172.21.15.76:6835","nonce":3517429435}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34472":{"gid":34472,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.76:6837/1150678991","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":1150678991},{"type":"v1","addr":"172.21.15.76:6837","nonce":1150678991}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34517":{"gid":34517,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.76:6839/327145636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6838","nonce":327145636},{"type":"v1","addr":"172.21.15.76:6839","nonce":327145636}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34517,"qdb_cluster":[34472,34517,34442,34406,24463]},"id":1}]} 2024-04-26T21:55:30.546 INFO:teuthology.orchestra.run.smithi072.stderr:dumped fsmap epoch 27 2024-04-26T21:55:30.549 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:30.549+0000 7fe9cf200640 1 -- 172.21.15.72:0/2099086773 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fe9b40a3cc0 msgr2=0x7fe9b40a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:30.549 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:30.549+0000 7fe9cf200640 1 --2- 172.21.15.72:0/2099086773 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fe9b40a3cc0 0x7fe9b40a6180 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fe9b8005e10 tx=0x7fe9b8005da0 comp rx=0 tx=0).stop 2024-04-26T21:55:30.549 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:30.549+0000 7fe9cf200640 1 -- 172.21.15.72:0/2099086773 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe9d0102750 msgr2=0x7fe9d01987a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:30.549 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:30.549+0000 7fe9cf200640 1 --2- 172.21.15.72:0/2099086773 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe9d0102750 0x7fe9d01987a0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fe9c000d740 tx=0x7fe9c000dc00 comp rx=0 tx=0).stop 2024-04-26T21:55:30.549 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:30.549+0000 7fe9cf200640 1 -- 172.21.15.72:0/2099086773 shutdown_connections 2024-04-26T21:55:30.549 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:30.549+0000 7fe9cf200640 1 --2- 172.21.15.72:0/2099086773 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fe9b40a3cc0 0x7fe9b40a6180 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:30.549 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:30.549+0000 7fe9cf200640 1 --2- 172.21.15.72:0/2099086773 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe9d0104300 0x7fe9d019fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:30.549 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:30.549+0000 7fe9cf200640 1 --2- 172.21.15.72:0/2099086773 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe9d0103940 0x7fe9d0198ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:30.549 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:30.549+0000 7fe9cf200640 1 --2- 172.21.15.72:0/2099086773 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe9d0102750 0x7fe9d01987a0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:30.549 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:30.549+0000 7fe9cf200640 1 -- 172.21.15.72:0/2099086773 >> 172.21.15.72:0/2099086773 conn(0x7fe9d00fdcc0 msgr2=0x7fe9d0106c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:55:30.550 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:30.549+0000 7fe9cf200640 1 -- 172.21.15.72:0/2099086773 shutdown_connections 2024-04-26T21:55:30.550 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:30.549+0000 7fe9cf200640 1 -- 172.21.15.72:0/2099086773 wait complete. 2024-04-26T21:55:30.568 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:55:30.762 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:30.762+0000 7f3212a00640 1 -- 172.21.15.72:0/447425037 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f320c108640 msgr2=0x7f320c10aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:30.763 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:30.762+0000 7f3212a00640 1 --2- 172.21.15.72:0/447425037 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f320c108640 0x7f320c10aa90 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f3204009ca0 tx=0x7f320402a8c0 comp rx=0 tx=0).stop 2024-04-26T21:55:30.763 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:30.762+0000 7f3212a00640 1 -- 172.21.15.72:0/447425037 shutdown_connections 2024-04-26T21:55:30.763 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:30.762+0000 7f3212a00640 1 --2- 172.21.15.72:0/447425037 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f320c108640 0x7f320c10aa90 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:30.763 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:30.762+0000 7f3212a00640 1 --2- 172.21.15.72:0/447425037 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f320c105d10 0x7f320c108100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:30.763 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:30.762+0000 7f3212a00640 1 --2- 172.21.15.72:0/447425037 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f320c1033e0 0x7f320c1057d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:30.763 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:30.762+0000 7f3212a00640 1 -- 172.21.15.72:0/447425037 >> 172.21.15.72:0/447425037 conn(0x7f320c0fcdf0 msgr2=0x7f320c0ff210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:55:30.763 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:30.762+0000 7f3212a00640 1 -- 172.21.15.72:0/447425037 shutdown_connections 2024-04-26T21:55:30.763 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:30.762+0000 7f3212a00640 1 -- 172.21.15.72:0/447425037 wait complete. 2024-04-26T21:55:30.765 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:30.763+0000 7f3212a00640 1 Processor -- start 2024-04-26T21:55:30.765 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:30.763+0000 7f3212a00640 1 -- start start 2024-04-26T21:55:30.765 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:30.764+0000 7f3212a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f320c1033e0 0x7f320c19ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:30.765 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:30.764+0000 7f3212a00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f320c105d10 0x7f320c19b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:30.765 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:30.764+0000 7f3212a00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f320c108640 0x7f320c1a21f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:30.765 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:30.764+0000 7f3212a00640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f320c10e3b0 con 0x7f320c105d10 2024-04-26T21:55:30.765 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:30.764+0000 7f3212a00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f320c10e230 con 0x7f320c108640 2024-04-26T21:55:30.765 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:30.764+0000 7f3212a00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f320c10e530 con 0x7f320c1033e0 2024-04-26T21:55:30.765 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:30.764+0000 7f320b400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f320c105d10 0x7f320c19b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:30.766 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:30.764+0000 7f320b400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f320c105d10 0x7f320c19b1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:44328/0 (socket says 172.21.15.72:44328) 2024-04-26T21:55:30.766 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:30.764+0000 7f320b400640 1 -- 172.21.15.72:0/3665492020 learned_addr learned my addr 172.21.15.72:0/3665492020 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:55:30.766 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:30.764+0000 7f3210a00640 1 --2- 172.21.15.72:0/3665492020 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f320c108640 0x7f320c1a21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:30.766 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:30.764+0000 7f320be00640 1 --2- 172.21.15.72:0/3665492020 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f320c1033e0 0x7f320c19ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:30.766 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:30.764+0000 7f320b400640 1 -- 172.21.15.72:0/3665492020 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f320c1033e0 msgr2=0x7f320c19ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:30.766 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:30.764+0000 7f320b400640 1 --2- 172.21.15.72:0/3665492020 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f320c1033e0 0x7f320c19ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:30.766 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:30.764+0000 7f320b400640 1 -- 172.21.15.72:0/3665492020 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f320c108640 msgr2=0x7f320c1a21f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:30.766 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:30.764+0000 7f320b400640 1 --2- 172.21.15.72:0/3665492020 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f320c108640 0x7f320c1a21f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:30.766 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:30.764+0000 7f320b400640 1 -- 172.21.15.72:0/3665492020 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f320c1a2890 con 0x7f320c105d10 2024-04-26T21:55:30.766 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:30.765+0000 7f320b400640 1 --2- 172.21.15.72:0/3665492020 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f320c105d10 0x7f320c19b1c0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f31f800d740 tx=0x7f31f800dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:55:30.766 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:30.765+0000 7f3208c00640 1 -- 172.21.15.72:0/3665492020 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31f8017070 con 0x7f320c105d10 2024-04-26T21:55:30.766 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:30.765+0000 7f3208c00640 1 -- 172.21.15.72:0/3665492020 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f31f800de50 con 0x7f320c105d10 2024-04-26T21:55:30.766 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:30.765+0000 7f3208c00640 1 -- 172.21.15.72:0/3665492020 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31f8004c60 con 0x7f320c105d10 2024-04-26T21:55:30.766 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:30.765+0000 7f3212a00640 1 -- 172.21.15.72:0/3665492020 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f320c1a2b80 con 0x7f320c105d10 2024-04-26T21:55:30.767 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:30.765+0000 7f3212a00640 1 -- 172.21.15.72:0/3665492020 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f320c1a30d0 con 0x7f320c105d10 2024-04-26T21:55:30.767 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:30.766+0000 7f3212a00640 1 -- 172.21.15.72:0/3665492020 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f320c10a130 con 0x7f320c105d10 2024-04-26T21:55:30.769 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:30.769+0000 7f3208c00640 1 -- 172.21.15.72:0/3665492020 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f31f8004e00 con 0x7f320c105d10 2024-04-26T21:55:30.770 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:30.770+0000 7f3208c00640 1 --2- 172.21.15.72:0/3665492020 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f31f00a3cc0 0x7f31f00a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:30.770 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:30.770+0000 7f3208c00640 1 -- 172.21.15.72:0/3665492020 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7f31f80d0070 con 0x7f320c105d10 2024-04-26T21:55:30.770 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:30.770+0000 7f320be00640 1 --2- 172.21.15.72:0/3665492020 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f31f00a3cc0 0x7f31f00a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:30.771 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:30.771+0000 7f320be00640 1 --2- 172.21.15.72:0/3665492020 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f31f00a3cc0 0x7f31f00a6180 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f31fc005fd0 tx=0x7f31fc005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:55:30.774 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:30.774+0000 7f3208c00640 1 -- 172.21.15.72:0/3665492020 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f31f809c870 con 0x7f320c105d10 2024-04-26T21:55:30.976 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:30.976+0000 7f3212a00640 1 -- 172.21.15.72:0/3665492020 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f320c02dd20 con 0x7f320c105d10 2024-04-26T21:55:30.980 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:30.980+0000 7f3208c00640 1 -- 172.21.15.72:0/3665492020 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+22783 (secure 0 0 0) 0x7f31f809c170 con 0x7f320c105d10 2024-04-26T21:55:30.981 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T21:55:30.981 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":91,"fsid":"a1780950-0415-11ef-bc93-c7b262605968","created":"2024-04-26T21:42:01.504173+0000","modified":"2024-04-26T21:54:11.892872+0000","last_up_change":"2024-04-26T21:51:43.482437+0000","last_in_change":"2024-04-26T21:51:18.468587+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:46:29.785427+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:52:48.320391+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T21:52:49.517493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"04e06884-b355-4762-9b29-9034b7201e9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6802","nonce":1416765179},{"type":"v1","addr":"172.21.15.72:6803","nonce":1416765179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6804","nonce":1416765179},{"type":"v1","addr":"172.21.15.72:6805","nonce":1416765179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6808","nonce":1416765179},{"type":"v1","addr":"172.21.15.72:6809","nonce":1416765179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6806","nonce":1416765179},{"type":"v1","addr":"172.21.15.72:6807","nonce":1416765179}]},"public_addr":"172.21.15.72:6803/1416765179","cluster_addr":"172.21.15.72:6805/1416765179","heartbeat_back_addr":"172.21.15.72:6809/1416765179","heartbeat_front_addr":"172.21.15.72:6807/1416765179","state":["exists","up"]},{"osd":1,"uuid":"d75a479a-4a2e-48e9-a01e-2b1c9617416a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":755426949},{"type":"v1","addr":"172.21.15.76:6801","nonce":755426949}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":755426949},{"type":"v1","addr":"172.21.15.76:6803","nonce":755426949}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":755426949},{"type":"v1","addr":"172.21.15.76:6807","nonce":755426949}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":755426949},{"type":"v1","addr":"172.21.15.76:6805","nonce":755426949}]},"public_addr":"172.21.15.76:6801/755426949","cluster_addr":"172.21.15.76:6803/755426949","heartbeat_back_addr":"172.21.15.76:6807/755426949","heartbeat_front_addr":"172.21.15.76:6805/755426949","state":["exists","up"]},{"osd":2,"uuid":"b6f8da58-4428-4e8c-a944-7ea5645d6d92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":852265973},{"type":"v1","addr":"172.21.15.164:6801","nonce":852265973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":852265973},{"type":"v1","addr":"172.21.15.164:6803","nonce":852265973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":852265973},{"type":"v1","addr":"172.21.15.164:6807","nonce":852265973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":852265973},{"type":"v1","addr":"172.21.15.164:6805","nonce":852265973}]},"public_addr":"172.21.15.164:6801/852265973","cluster_addr":"172.21.15.164:6803/852265973","heartbeat_back_addr":"172.21.15.164:6807/852265973","heartbeat_front_addr":"172.21.15.164:6805/852265973","state":["exists","up"]},{"osd":3,"uuid":"76d83409-649e-41e5-b4f4-43bbd5b00ea5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6810","nonce":4027763789},{"type":"v1","addr":"172.21.15.72:6811","nonce":4027763789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6812","nonce":4027763789},{"type":"v1","addr":"172.21.15.72:6813","nonce":4027763789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6816","nonce":4027763789},{"type":"v1","addr":"172.21.15.72:6817","nonce":4027763789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6814","nonce":4027763789},{"type":"v1","addr":"172.21.15.72:6815","nonce":4027763789}]},"public_addr":"172.21.15.72:6811/4027763789","cluster_addr":"172.21.15.72:6813/4027763789","heartbeat_back_addr":"172.21.15.72:6817/4027763789","heartbeat_front_addr":"172.21.15.72:6815/4027763789","state":["exists","up"]},{"osd":4,"uuid":"d2cfeea7-1661-4156-bdcd-f770a8d1a878","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":3718867827},{"type":"v1","addr":"172.21.15.76:6809","nonce":3718867827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":3718867827},{"type":"v1","addr":"172.21.15.76:6811","nonce":3718867827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":3718867827},{"type":"v1","addr":"172.21.15.76:6815","nonce":3718867827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":3718867827},{"type":"v1","addr":"172.21.15.76:6813","nonce":3718867827}]},"public_addr":"172.21.15.76:6809/3718867827","cluster_addr":"172.21.15.76:6811/3718867827","heartbeat_back_addr":"172.21.15.76:6815/3718867827","heartbeat_front_addr":"172.21.15.76:6813/3718867827","state":["exists","up"]},{"osd":5,"uuid":"f3896ad6-a040-4ea3-a431-ff9b7fc05c36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":2543954772},{"type":"v1","addr":"172.21.15.164:6809","nonce":2543954772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":2543954772},{"type":"v1","addr":"172.21.15.164:6811","nonce":2543954772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":2543954772},{"type":"v1","addr":"172.21.15.164:6815","nonce":2543954772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":2543954772},{"type":"v1","addr":"172.21.15.164:6813","nonce":2543954772}]},"public_addr":"172.21.15.164:6809/2543954772","cluster_addr":"172.21.15.164:6811/2543954772","heartbeat_back_addr":"172.21.15.164:6815/2543954772","heartbeat_front_addr":"172.21.15.164:6813/2543954772","state":["exists","up"]},{"osd":6,"uuid":"d5d56650-be7e-4ba5-ba3a-d7d473294dab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6818","nonce":2080659027},{"type":"v1","addr":"172.21.15.72:6819","nonce":2080659027}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6820","nonce":2080659027},{"type":"v1","addr":"172.21.15.72:6821","nonce":2080659027}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6824","nonce":2080659027},{"type":"v1","addr":"172.21.15.72:6825","nonce":2080659027}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6822","nonce":2080659027},{"type":"v1","addr":"172.21.15.72:6823","nonce":2080659027}]},"public_addr":"172.21.15.72:6819/2080659027","cluster_addr":"172.21.15.72:6821/2080659027","heartbeat_back_addr":"172.21.15.72:6825/2080659027","heartbeat_front_addr":"172.21.15.72:6823/2080659027","state":["exists","up"]},{"osd":7,"uuid":"18975b5a-9309-4a0f-9f37-3ce09c2e745d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":3144631323},{"type":"v1","addr":"172.21.15.76:6817","nonce":3144631323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":3144631323},{"type":"v1","addr":"172.21.15.76:6819","nonce":3144631323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":3144631323},{"type":"v1","addr":"172.21.15.76:6823","nonce":3144631323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":3144631323},{"type":"v1","addr":"172.21.15.76:6821","nonce":3144631323}]},"public_addr":"172.21.15.76:6817/3144631323","cluster_addr":"172.21.15.76:6819/3144631323","heartbeat_back_addr":"172.21.15.76:6823/3144631323","heartbeat_front_addr":"172.21.15.76:6821/3144631323","state":["exists","up"]},{"osd":8,"uuid":"779e9c1e-8e97-48d4-a854-68ca13f38e0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":627888883},{"type":"v1","addr":"172.21.15.164:6817","nonce":627888883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":627888883},{"type":"v1","addr":"172.21.15.164:6819","nonce":627888883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":627888883},{"type":"v1","addr":"172.21.15.164:6823","nonce":627888883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":627888883},{"type":"v1","addr":"172.21.15.164:6821","nonce":627888883}]},"public_addr":"172.21.15.164:6817/627888883","cluster_addr":"172.21.15.164:6819/627888883","heartbeat_back_addr":"172.21.15.164:6823/627888883","heartbeat_front_addr":"172.21.15.164:6821/627888883","state":["exists","up"]},{"osd":9,"uuid":"d0317850-7b2d-4850-b2cf-0b919505c1fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6826","nonce":1273118485},{"type":"v1","addr":"172.21.15.72:6827","nonce":1273118485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6828","nonce":1273118485},{"type":"v1","addr":"172.21.15.72:6829","nonce":1273118485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6832","nonce":1273118485},{"type":"v1","addr":"172.21.15.72:6833","nonce":1273118485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6830","nonce":1273118485},{"type":"v1","addr":"172.21.15.72:6831","nonce":1273118485}]},"public_addr":"172.21.15.72:6827/1273118485","cluster_addr":"172.21.15.72:6829/1273118485","heartbeat_back_addr":"172.21.15.72:6833/1273118485","heartbeat_front_addr":"172.21.15.72:6831/1273118485","state":["exists","up"]},{"osd":10,"uuid":"e0aa38d8-6011-4d2a-b67b-5bff6d999e47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":2808954459},{"type":"v1","addr":"172.21.15.76:6825","nonce":2808954459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":2808954459},{"type":"v1","addr":"172.21.15.76:6827","nonce":2808954459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":2808954459},{"type":"v1","addr":"172.21.15.76:6831","nonce":2808954459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":2808954459},{"type":"v1","addr":"172.21.15.76:6829","nonce":2808954459}]},"public_addr":"172.21.15.76:6825/2808954459","cluster_addr":"172.21.15.76:6827/2808954459","heartbeat_back_addr":"172.21.15.76:6831/2808954459","heartbeat_front_addr":"172.21.15.76:6829/2808954459","state":["exists","up"]},{"osd":11,"uuid":"981b3d4d-916e-4937-9916-7cc21a891e43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":2354951784},{"type":"v1","addr":"172.21.15.164:6825","nonce":2354951784}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":2354951784},{"type":"v1","addr":"172.21.15.164:6827","nonce":2354951784}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":2354951784},{"type":"v1","addr":"172.21.15.164:6831","nonce":2354951784}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":2354951784},{"type":"v1","addr":"172.21.15.164:6829","nonce":2354951784}]},"public_addr":"172.21.15.164:6825/2354951784","cluster_addr":"172.21.15.164:6827/2354951784","heartbeat_back_addr":"172.21.15.164:6831/2354951784","heartbeat_front_addr":"172.21.15.164:6829/2354951784","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:18.997924+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:52.439081+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:46:25.893288+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:47:00.112807+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:47:35.116874+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:48:09.216344+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:48:45.028853+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:49:19.905397+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:49:55.030536+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:50:30.767662+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:51:06.423267+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:51:41.631211+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.1a","mappings":[{"from":5,"to":7}]},{"pgid":"3.1b","mappings":[{"from":4,"to":6}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.28","mappings":[{"from":5,"to":6}]},{"pgid":"3.2a","mappings":[{"from":3,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.72:0/416277784":"2024-04-27T21:43:09.602791+0000","172.21.15.72:6800/4010358672":"2024-04-27T21:43:09.602791+0000","172.21.15.72:6801/1012617371":"2024-04-27T21:42:34.684837+0000","172.21.15.72:6801/4010358672":"2024-04-27T21:43:09.602791+0000","172.21.15.72:0/3555448179":"2024-04-27T21:43:09.602791+0000","172.21.15.72:6800/1012617371":"2024-04-27T21:42:34.684837+0000","172.21.15.72:0/2920378761":"2024-04-27T21:42:34.684837+0000","172.21.15.72:0/1933740041":"2024-04-27T21:43:09.602791+0000","172.21.15.72:0/3225295775":"2024-04-27T21:42:34.684837+0000","172.21.15.72:0/2326016180":"2024-04-27T21:42:34.684837+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:55:30.983 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:30.983+0000 7f3212a00640 1 -- 172.21.15.72:0/3665492020 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f31f00a3cc0 msgr2=0x7f31f00a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:30.983 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:30.983+0000 7f3212a00640 1 --2- 172.21.15.72:0/3665492020 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f31f00a3cc0 0x7f31f00a6180 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f31fc005fd0 tx=0x7f31fc005850 comp rx=0 tx=0).stop 2024-04-26T21:55:30.983 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:30.983+0000 7f3212a00640 1 -- 172.21.15.72:0/3665492020 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f320c105d10 msgr2=0x7f320c19b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:30.983 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:30.983+0000 7f3212a00640 1 --2- 172.21.15.72:0/3665492020 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f320c105d10 0x7f320c19b1c0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f31f800d740 tx=0x7f31f800dc00 comp rx=0 tx=0).stop 2024-04-26T21:55:30.983 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:30.983+0000 7f3212a00640 1 -- 172.21.15.72:0/3665492020 shutdown_connections 2024-04-26T21:55:30.983 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:30.983+0000 7f3212a00640 1 --2- 172.21.15.72:0/3665492020 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f320c108640 0x7f320c1a21f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:30.983 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:30.983+0000 7f3212a00640 1 --2- 172.21.15.72:0/3665492020 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f320c105d10 0x7f320c19b1c0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:30.984 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:30.983+0000 7f3212a00640 1 --2- 172.21.15.72:0/3665492020 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f31f00a3cc0 0x7f31f00a6180 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:30.984 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:30.983+0000 7f3212a00640 1 --2- 172.21.15.72:0/3665492020 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f320c1033e0 0x7f320c19ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:30.984 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:30.983+0000 7f3212a00640 1 -- 172.21.15.72:0/3665492020 >> 172.21.15.72:0/3665492020 conn(0x7f320c0fcdf0 msgr2=0x7f320c102aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:55:30.984 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:30.983+0000 7f3212a00640 1 -- 172.21.15.72:0/3665492020 shutdown_connections 2024-04-26T21:55:30.984 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:30.983+0000 7f3212a00640 1 -- 172.21.15.72:0/3665492020 wait complete. 2024-04-26T21:55:31.002 DEBUG:tasks.fwd_scrub:Yielding 2024-04-26T21:55:31.002 INFO:teuthology.run_tasks:Running task workunit... 2024-04-26T21:55:31.014 INFO:tasks.workunit:Pulling workunits from ref bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T21:55:31.015 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-04-26T21:55:31.015 INFO:tasks.workunit:timeout=3h 2024-04-26T21:55:31.015 INFO:tasks.workunit:cleanup=True 2024-04-26T21:55:31.015 DEBUG:teuthology.orchestra.run.smithi072:> stat -- /home/ubuntu/cephtest/mnt.0 2024-04-26T21:55:31.015 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-04-26T21:55:31.015 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-26T21:55:31.016 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:55:31.016 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-26T21:55:31.058 INFO:teuthology.orchestra.run.smithi072.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-04-26T21:55:31.059 INFO:teuthology.orchestra.run.smithi072.stdout: Size: 0 Blocks: 1 IO Block: 4194304 directory 2024-04-26T21:55:31.059 INFO:teuthology.orchestra.run.smithi072.stdout:Device: 72h/114d Inode: 1 Links: 2 2024-04-26T21:55:31.059 INFO:teuthology.orchestra.run.smithi072.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T21:55:31.059 INFO:teuthology.orchestra.run.smithi072.stdout:Context: system_u:object_r:fusefs_t:s0 2024-04-26T21:55:31.059 INFO:teuthology.orchestra.run.smithi072.stdout:Access: 2024-04-26 21:52:59.025691004 +0000 2024-04-26T21:55:31.059 INFO:teuthology.orchestra.run.smithi072.stdout:Modify: 2024-04-26 21:52:59.025691004 +0000 2024-04-26T21:55:31.059 INFO:teuthology.orchestra.run.smithi072.stdout:Change: 2024-04-26 21:55:22.406051547 +0000 2024-04-26T21:55:31.059 INFO:teuthology.orchestra.run.smithi072.stdout: Birth: - 2024-04-26T21:55:31.059 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-04-26T21:55:31.059 DEBUG:teuthology.orchestra.run.smithi072:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-04-26T21:55:31.190 DEBUG:teuthology.orchestra.run.smithi072:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T21:55:31.204 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.204+0000 7f8da6a00640 1 -- 172.21.15.72:0/4224725188 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8da0103a50 msgr2=0x7f8da0103ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:31.204 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.204+0000 7f8da6a00640 1 --2- 172.21.15.72:0/4224725188 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8da0103a50 0x7f8da0103ed0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f8d8c009900 tx=0x7f8d8c02a410 comp rx=0 tx=0).stop 2024-04-26T21:55:31.205 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.204+0000 7f8da6a00640 1 -- 172.21.15.72:0/4224725188 shutdown_connections 2024-04-26T21:55:31.205 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.204+0000 7f8da6a00640 1 --2- 172.21.15.72:0/4224725188 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8da0104410 0x7f8da010aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:31.205 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.204+0000 7f8da6a00640 1 --2- 172.21.15.72:0/4224725188 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8da0103a50 0x7f8da0103ed0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:31.205 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.204+0000 7f8da6a00640 1 --2- 172.21.15.72:0/4224725188 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8da0102860 0x7f8da0102c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:31.205 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.204+0000 7f8da6a00640 1 -- 172.21.15.72:0/4224725188 >> 172.21.15.72:0/4224725188 conn(0x7f8da00fddd0 msgr2=0x7f8da0100210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:55:31.205 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.204+0000 7f8da6a00640 1 -- 172.21.15.72:0/4224725188 shutdown_connections 2024-04-26T21:55:31.205 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.204+0000 7f8da6a00640 1 -- 172.21.15.72:0/4224725188 wait complete. 2024-04-26T21:55:31.208 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.205+0000 7f8da6a00640 1 Processor -- start 2024-04-26T21:55:31.208 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.205+0000 7f8da6a00640 1 -- start start 2024-04-26T21:55:31.208 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.205+0000 7f8da6a00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8da0102860 0x7f8da00766e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:31.208 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.205+0000 7f8da6a00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8da0103a50 0x7f8da0074d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:31.208 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.205+0000 7f8da6a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8da0104410 0x7f8da0075260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:31.208 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.205+0000 7f8da6a00640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f8da010e290 con 0x7f8da0102860 2024-04-26T21:55:31.208 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.205+0000 7f8da6a00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f8da010e110 con 0x7f8da0103a50 2024-04-26T21:55:31.208 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.205+0000 7f8da6a00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f8da010e410 con 0x7f8da0104410 2024-04-26T21:55:31.208 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.206+0000 7f8da5600640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8da0102860 0x7f8da00766e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:31.208 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.206+0000 7f8da5600640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8da0102860 0x7f8da00766e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:55942/0 (socket says 172.21.15.72:55942) 2024-04-26T21:55:31.208 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.206+0000 7f8da5600640 1 -- 172.21.15.72:0/3907066789 learned_addr learned my addr 172.21.15.72:0/3907066789 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:55:31.209 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.206+0000 7f8da5600640 1 -- 172.21.15.72:0/3907066789 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8da0104410 msgr2=0x7f8da0075260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:31.209 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.206+0000 7f8da5600640 1 --2- 172.21.15.72:0/3907066789 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8da0104410 0x7f8da0075260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:31.209 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.206+0000 7f8da4c00640 1 --2- 172.21.15.72:0/3907066789 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8da0103a50 0x7f8da0074d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:31.209 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.206+0000 7f8da5600640 1 -- 172.21.15.72:0/3907066789 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8da0103a50 msgr2=0x7f8da0074d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:31.209 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.206+0000 7f8da5600640 1 --2- 172.21.15.72:0/3907066789 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8da0103a50 0x7f8da0074d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:31.209 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.206+0000 7f8da5600640 1 -- 172.21.15.72:0/3907066789 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8da0075af0 con 0x7f8da0102860 2024-04-26T21:55:31.209 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.206+0000 7f8da5600640 1 --2- 172.21.15.72:0/3907066789 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8da0102860 0x7f8da00766e0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f8d9400eaa0 tx=0x7f8d9400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:55:31.209 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.207+0000 7f8d9e000640 1 -- 172.21.15.72:0/3907066789 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d94016070 con 0x7f8da0102860 2024-04-26T21:55:31.209 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.207+0000 7f8d9e000640 1 -- 172.21.15.72:0/3907066789 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8d9400ce10 con 0x7f8da0102860 2024-04-26T21:55:31.209 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.207+0000 7f8da6a00640 1 -- 172.21.15.72:0/3907066789 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8da01a2ab0 con 0x7f8da0102860 2024-04-26T21:55:31.209 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.207+0000 7f8d9e000640 1 -- 172.21.15.72:0/3907066789 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d94011470 con 0x7f8da0102860 2024-04-26T21:55:31.209 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.207+0000 7f8da6a00640 1 -- 172.21.15.72:0/3907066789 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8da01a2fb0 con 0x7f8da0102860 2024-04-26T21:55:31.210 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.210+0000 7f8daab22740 1 Processor -- start 2024-04-26T21:55:31.211 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.210+0000 7f8d9e000640 1 -- 172.21.15.72:0/3907066789 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f8d94019070 con 0x7f8da0102860 2024-04-26T21:55:31.211 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.210+0000 7f8daab22740 1 -- start start 2024-04-26T21:55:31.211 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.210+0000 7f8daab22740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ab751c7a70 0x55ab751c7e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:31.211 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.210+0000 7f8daab22740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ab751c8460 0x55ab751ccf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:31.211 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.210+0000 7f8daab22740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ab751cd5d0 0x55ab751cfa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:31.211 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.210+0000 7f8daab22740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55ab7512c980 con 0x55ab751cd5d0 2024-04-26T21:55:31.211 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.210+0000 7f8daab22740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55ab750c9220 con 0x55ab751c8460 2024-04-26T21:55:31.211 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.210+0000 7f8daab22740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55ab75128940 con 0x55ab751c7a70 2024-04-26T21:55:31.213 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.211+0000 7f8da4c00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ab751cd5d0 0x55ab751cfa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:31.213 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.211+0000 7f8da4c00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ab751cd5d0 0x55ab751cfa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:55958/0 (socket says 172.21.15.72:55958) 2024-04-26T21:55:31.213 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.211+0000 7f8da4c00640 1 -- 172.21.15.72:0/2767705501 learned_addr learned my addr 172.21.15.72:0/2767705501 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:55:31.214 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.211+0000 7f8da5600640 1 --2- 172.21.15.72:0/2767705501 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ab751c8460 0x55ab751ccf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:31.214 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.211+0000 7f8da6000640 1 --2- 172.21.15.72:0/2767705501 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ab751c7a70 0x55ab751c7e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:31.214 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.211+0000 7f8da4c00640 1 -- 172.21.15.72:0/2767705501 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ab751c7a70 msgr2=0x55ab751c7e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:55:31.214 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.211+0000 7f8da4c00640 1 --2- 172.21.15.72:0/2767705501 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ab751c7a70 0x55ab751c7e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:31.214 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.211+0000 7f8da4c00640 1 -- 172.21.15.72:0/2767705501 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ab751c8460 msgr2=0x55ab751ccf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:55:31.214 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.211+0000 7f8da4c00640 1 --2- 172.21.15.72:0/2767705501 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ab751c8460 0x55ab751ccf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:31.214 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.211+0000 7f8da4c00640 1 -- 172.21.15.72:0/2767705501 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab74f067e0 con 0x55ab751cd5d0 2024-04-26T21:55:31.214 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.212+0000 7f8da4c00640 1 --2- 172.21.15.72:0/2767705501 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ab751cd5d0 0x55ab751cfa60 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f8d8c0040f0 tx=0x7f8d8c004120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94b70a7d27496338 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:55:31.214 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.212+0000 7f8d92600640 1 -- 172.21.15.72:0/2767705501 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d8c040070 con 0x55ab751cd5d0 2024-04-26T21:55:31.214 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.212+0000 7f8d9e000640 1 --2- 172.21.15.72:0/3907066789 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f8d880a3c40 0x7f8d880a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:31.214 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.212+0000 7f8d92600640 1 -- 172.21.15.72:0/2767705501 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8d8c003a80 con 0x55ab751cd5d0 2024-04-26T21:55:31.214 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.212+0000 7f8d92600640 1 -- 172.21.15.72:0/2767705501 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d8c035030 con 0x55ab751cd5d0 2024-04-26T21:55:31.215 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.212+0000 7f8d9e000640 1 -- 172.21.15.72:0/3907066789 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7f8d940d0860 con 0x7f8da0102860 2024-04-26T21:55:31.215 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.212+0000 7f8da6000640 1 --2- 172.21.15.72:0/3907066789 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f8d880a3c40 0x7f8d880a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:31.215 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.213+0000 7f8daab22740 1 -- 172.21.15.72:0/2767705501 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ab751cd5d0 msgr2=0x55ab751cfa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:31.215 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.213+0000 7f8daab22740 1 --2- 172.21.15.72:0/2767705501 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ab751cd5d0 0x55ab751cfa60 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f8d8c0040f0 tx=0x7f8d8c004120 comp rx=0 tx=0).stop 2024-04-26T21:55:31.215 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.213+0000 7f8daab22740 1 -- 172.21.15.72:0/2767705501 shutdown_connections 2024-04-26T21:55:31.215 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.213+0000 7f8daab22740 1 --2- 172.21.15.72:0/2767705501 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ab751cd5d0 0x55ab751cfa60 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:31.215 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.213+0000 7f8daab22740 1 --2- 172.21.15.72:0/2767705501 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ab751c8460 0x55ab751ccf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:31.215 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.213+0000 7f8daab22740 1 --2- 172.21.15.72:0/2767705501 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ab751c7a70 0x55ab751c7e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:31.215 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.213+0000 7f8daab22740 1 -- 172.21.15.72:0/2767705501 >> 172.21.15.72:0/2767705501 conn(0x55ab751c30e0 msgr2=0x55ab751c34e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:55:31.215 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.213+0000 7f8da6000640 1 --2- 172.21.15.72:0/3907066789 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f8d880a3c40 0x7f8d880a6100 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x55ab751c8cc0 tx=0x7f8d98005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:55:31.215 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.213+0000 7f8daab22740 1 -- 172.21.15.72:0/2767705501 shutdown_connections 2024-04-26T21:55:31.215 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.213+0000 7f8daab22740 1 -- 172.21.15.72:0/2767705501 wait complete. 2024-04-26T21:55:31.215 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.213+0000 7f8daab22740 1 -- start start 2024-04-26T21:55:31.215 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.214+0000 7f8daab22740 1 Processor -- start 2024-04-26T21:55:31.218 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.214+0000 7f8daab22740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ab751c7a70 0x55ab751d1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:31.218 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.214+0000 7f8daab22740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ab751c8460 0x55ab751d2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:31.218 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.214+0000 7f8daab22740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ab751cd5d0 0x55ab751d9440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:31.218 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.214+0000 7f8daab22740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55ab751d0520 con 0x55ab751c8460 2024-04-26T21:55:31.218 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.214+0000 7f8daab22740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55ab750b4170 con 0x55ab751cd5d0 2024-04-26T21:55:31.218 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.214+0000 7f8daab22740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55ab751d06a0 con 0x55ab751c7a70 2024-04-26T21:55:31.219 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.214+0000 7f8daab22740 10 client.0 _ll_register_callbacks cb 0x55ab75123770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:55:31.219 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.214+0000 7f8da4c00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ab751c8460 0x55ab751d2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:31.219 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.214+0000 7f8da4c00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ab751c8460 0x55ab751d2400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:55966/0 (socket says 172.21.15.72:55966) 2024-04-26T21:55:31.219 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.214+0000 7f8da4c00640 1 -- 172.21.15.72:0/1673379991 learned_addr learned my addr 172.21.15.72:0/1673379991 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:55:31.219 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.215+0000 7f8da5600640 1 --2- 172.21.15.72:0/1673379991 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ab751c7a70 0x55ab751d1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:31.219 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.215+0000 7f8da4c00640 1 -- 172.21.15.72:0/1673379991 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ab751c7a70 msgr2=0x55ab751d1ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:55:31.219 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.215+0000 7f8da4c00640 1 --2- 172.21.15.72:0/1673379991 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ab751c7a70 0x55ab751d1ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:31.219 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.215+0000 7f8da4c00640 1 -- 172.21.15.72:0/1673379991 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ab751cd5d0 msgr2=0x55ab751d9440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:55:31.219 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.215+0000 7f8da4c00640 1 --2- 172.21.15.72:0/1673379991 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ab751cd5d0 0x55ab751d9440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:31.219 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.215+0000 7f8da4c00640 1 -- 172.21.15.72:0/1673379991 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab74f66f70 con 0x55ab751c8460 2024-04-26T21:55:31.219 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.215+0000 7f8da4c00640 1 --2- 172.21.15.72:0/1673379991 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ab751c8460 0x55ab751d2400 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f8d8c0092a0 tx=0x7f8d8c003e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b84a2711378d041 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:55:31.219 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.215+0000 7f8d86a00640 10 client.0 ms_handle_connect on v2:172.21.15.72:3300/0 2024-04-26T21:55:31.219 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.215+0000 7f8d86a00640 1 -- 172.21.15.72:0/1673379991 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d8c04d070 con 0x55ab751c8460 2024-04-26T21:55:31.220 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.215+0000 7f8d86a00640 1 -- 172.21.15.72:0/1673379991 <== mon.0 v2:172.21.15.72:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7f8d8c031850 con 0x55ab751c8460 2024-04-26T21:55:31.220 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.215+0000 7f8daab22740 1 -- 172.21.15.72:0/1673379991 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8d8c003a80 con 0x55ab751c8460 2024-04-26T21:55:31.220 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.216+0000 7f8d86a00640 1 -- 172.21.15.72:0/1673379991 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f8d8c040070 con 0x55ab751c8460 2024-04-26T21:55:31.220 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.216+0000 7f8daab22740 10 client.15501 fetch_fsmap learned FSMap version 27 2024-04-26T21:55:31.220 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.216+0000 7f8daab22740 1 -- 172.21.15.72:0/1673379991 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ab74dfe170 con 0x55ab751c8460 2024-04-26T21:55:31.220 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.216+0000 7f8d86a00640 1 -- 172.21.15.72:0/1673379991 <== mon.0 v2:172.21.15.72:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8d8c003a80 con 0x55ab751c8460 2024-04-26T21:55:31.220 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.216+0000 7f8d86a00640 1 -- 172.21.15.72:0/1673379991 <== mon.0 v2:172.21.15.72:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f8d8c050030 con 0x55ab751c8460 2024-04-26T21:55:31.220 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.216+0000 7f8daab22740 10 client.15501 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T21:55:31.220 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.216+0000 7f8daab22740 10 client.15501 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:55:31.220 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.216+0000 7f8daab22740 20 client.15501 populate_metadata read hostname 'smithi072' 2024-04-26T21:55:31.220 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.216+0000 7f8daab22740 1 --2- 172.21.15.72:0/1673379991 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x55ab751dd5a0 0x55ab751df990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:31.220 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.216+0000 7f8daab22740 4 client.15501 mds_command: new command op to 34517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:55:31.220 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.216+0000 7f8daab22740 1 -- 172.21.15.72:0/1673379991 --> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ab7513a590 con 0x55ab751dd5a0 2024-04-26T21:55:31.220 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.217+0000 7f8da5600640 1 --2- 172.21.15.72:0/1673379991 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x55ab751dd5a0 0x55ab751df990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:31.220 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.218+0000 7f8da5600640 1 --2- 172.21.15.72:0/1673379991 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x55ab751dd5a0 0x55ab751df990 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=51e286151618dba9 server_cookie=ca7e12310bc4bc0 in_seq=0 out_seq=0 2024-04-26T21:55:31.221 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.218+0000 7f8d86a00640 10 client.15501 ms_handle_connect on v2:172.21.15.76:6838/327145636 2024-04-26T21:55:31.222 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.220+0000 7f8d86a00640 1 -- 172.21.15.72:0/1673379991 <== mds.0 v2:172.21.15.76:6838/327145636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ab7513a590 con 0x55ab751dd5a0 2024-04-26T21:55:31.222 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.220+0000 7f8d86a00640 10 client.15501 handle_command_reply: tid=0 2024-04-26T21:55:31.222 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.220+0000 7f8daab22740 1 client.15501 shutdown 2024-04-26T21:55:31.222 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.220+0000 7f8daab22740 1 -- 172.21.15.72:0/1673379991 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ab751c8460 msgr2=0x55ab751d2400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:31.222 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.220+0000 7f8daab22740 1 --2- 172.21.15.72:0/1673379991 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ab751c8460 0x55ab751d2400 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f8d8c0092a0 tx=0x7f8d8c003e60 comp rx=0 tx=0).stop 2024-04-26T21:55:31.222 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.220+0000 7f8daab22740 1 -- 172.21.15.72:0/1673379991 shutdown_connections 2024-04-26T21:55:31.222 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.220+0000 7f8daab22740 1 --2- 172.21.15.72:0/1673379991 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x55ab751dd5a0 0x55ab751df990 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:31.222 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.220+0000 7f8daab22740 1 --2- 172.21.15.72:0/1673379991 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ab751cd5d0 0x55ab751d9440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:31.222 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.220+0000 7f8daab22740 1 --2- 172.21.15.72:0/1673379991 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ab751c8460 0x55ab751d2400 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:31.222 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.220+0000 7f8d86a00640 0 client.15501 ms_handle_reset on v2:172.21.15.76:6838/327145636 2024-04-26T21:55:31.222 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.220+0000 7f8daab22740 1 --2- 172.21.15.72:0/1673379991 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ab751c7a70 0x55ab751d1ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:31.222 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.220+0000 7f8daab22740 1 -- 172.21.15.72:0/1673379991 >> 172.21.15.72:0/1673379991 conn(0x55ab751c30e0 msgr2=0x55ab751c22e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:55:31.222 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.220+0000 7f8daab22740 1 -- 172.21.15.72:0/1673379991 shutdown_connections 2024-04-26T21:55:31.222 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.220+0000 7f8daab22740 1 -- 172.21.15.72:0/1673379991 wait complete. 2024-04-26T21:55:31.222 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.220+0000 7f8daab22740 20 client.15501 trim_cache size 0 max 16384 2024-04-26T21:55:31.235 INFO:tasks.workunit.client.0.smithi072.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-04-26T21:55:31.239 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.239+0000 7f8daab22740 1 Processor -- start 2024-04-26T21:55:31.240 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.239+0000 7f8daab22740 1 -- start start 2024-04-26T21:55:31.240 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.239+0000 7f8daab22740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ab751c7a70 0x55ab751c2160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:31.240 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.239+0000 7f8daab22740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ab751c8460 0x55ab751c26a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:31.240 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.239+0000 7f8daab22740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ab751cd5d0 0x55ab751d9400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:31.240 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.239+0000 7f8daab22740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f8da01abdc0 con 0x55ab751c8460 2024-04-26T21:55:31.240 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.239+0000 7f8daab22740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f8da01ad900 con 0x55ab751c7a70 2024-04-26T21:55:31.240 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.239+0000 7f8daab22740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f8da01ac490 con 0x55ab751cd5d0 2024-04-26T21:55:31.240 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.240+0000 7f8da4c00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ab751c8460 0x55ab751c26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:31.240 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.240+0000 7f8da4c00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ab751c8460 0x55ab751c26a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:55976/0 (socket says 172.21.15.72:55976) 2024-04-26T21:55:31.240 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.240+0000 7f8da4c00640 1 -- 172.21.15.72:0/3743206085 learned_addr learned my addr 172.21.15.72:0/3743206085 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:55:31.244 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.240+0000 7f8da5600640 1 --2- 172.21.15.72:0/3743206085 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ab751c7a70 0x55ab751c2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:31.244 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.240+0000 7f8da6000640 1 --2- 172.21.15.72:0/3743206085 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ab751cd5d0 0x55ab751d9400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:31.244 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.240+0000 7f8da4c00640 1 -- 172.21.15.72:0/3743206085 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ab751cd5d0 msgr2=0x55ab751d9400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:55:31.244 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.240+0000 7f8da4c00640 1 --2- 172.21.15.72:0/3743206085 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ab751cd5d0 0x55ab751d9400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:31.244 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.240+0000 7f8da4c00640 1 -- 172.21.15.72:0/3743206085 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ab751c7a70 msgr2=0x55ab751c2160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:55:31.244 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.240+0000 7f8da4c00640 1 --2- 172.21.15.72:0/3743206085 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ab751c7a70 0x55ab751c2160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:31.244 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.240+0000 7f8da4c00640 1 -- 172.21.15.72:0/3743206085 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab751c3e50 con 0x55ab751c8460 2024-04-26T21:55:31.244 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.240+0000 7f8da5600640 1 --2- 172.21.15.72:0/3743206085 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ab751c7a70 0x55ab751c2160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:55:31.244 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.240+0000 7f8da4c00640 1 --2- 172.21.15.72:0/3743206085 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ab751c8460 0x55ab751c26a0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f8d8c02a450 tx=0x7f8d8c0045c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb3241fc344eca3b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:55:31.244 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.241+0000 7f8d91c00640 1 -- 172.21.15.72:0/3743206085 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d8c04d070 con 0x55ab751c8460 2024-04-26T21:55:31.244 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.241+0000 7f8d91c00640 1 -- 172.21.15.72:0/3743206085 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8d8c003a00 con 0x55ab751c8460 2024-04-26T21:55:31.245 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.241+0000 7f8d91c00640 1 -- 172.21.15.72:0/3743206085 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d8c035040 con 0x55ab751c8460 2024-04-26T21:55:31.245 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.241+0000 7f8daab22740 1 -- 172.21.15.72:0/3743206085 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ab751c8460 msgr2=0x55ab751c26a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:31.245 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.241+0000 7f8daab22740 1 --2- 172.21.15.72:0/3743206085 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ab751c8460 0x55ab751c26a0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f8d8c02a450 tx=0x7f8d8c0045c0 comp rx=0 tx=0).stop 2024-04-26T21:55:31.245 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.242+0000 7f8daab22740 1 -- 172.21.15.72:0/3743206085 shutdown_connections 2024-04-26T21:55:31.245 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.242+0000 7f8daab22740 1 --2- 172.21.15.72:0/3743206085 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ab751cd5d0 0x55ab751d9400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:31.245 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.242+0000 7f8daab22740 1 --2- 172.21.15.72:0/3743206085 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ab751c8460 0x55ab751c26a0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:31.245 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.242+0000 7f8daab22740 1 --2- 172.21.15.72:0/3743206085 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ab751c7a70 0x55ab751c2160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:31.245 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.242+0000 7f8daab22740 1 -- 172.21.15.72:0/3743206085 >> 172.21.15.72:0/3743206085 conn(0x55ab751c30e0 msgr2=0x55ab751c1a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:55:31.245 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.242+0000 7f8daab22740 1 -- 172.21.15.72:0/3743206085 shutdown_connections 2024-04-26T21:55:31.245 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.242+0000 7f8daab22740 1 -- 172.21.15.72:0/3743206085 wait complete. 2024-04-26T21:55:31.245 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.242+0000 7f8daab22740 1 -- start start 2024-04-26T21:55:31.245 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.242+0000 7f8daab22740 1 Processor -- start 2024-04-26T21:55:31.245 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.243+0000 7f8daab22740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ab751c7a70 0x55ab751d9160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:31.245 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.243+0000 7f8daab22740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ab751c8460 0x55ab751d7770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:31.246 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.243+0000 7f8daab22740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ab751cd5d0 0x55ab751d7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:31.246 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.243+0000 7f8daab22740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f8da01ad230 con 0x55ab751cd5d0 2024-04-26T21:55:31.246 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.243+0000 7f8daab22740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f8da01acb60 con 0x55ab751c7a70 2024-04-26T21:55:31.246 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.243+0000 7f8daab22740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f8da01a9da0 con 0x55ab751c8460 2024-04-26T21:55:31.246 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.243+0000 7f8daab22740 10 client.0 _ll_register_callbacks cb 0x7f8d8c02ab10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:55:31.246 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.243+0000 7f8da6000640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ab751cd5d0 0x55ab751d7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:31.246 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.243+0000 7f8da6000640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ab751cd5d0 0x55ab751d7cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:55978/0 (socket says 172.21.15.72:55978) 2024-04-26T21:55:31.246 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.243+0000 7f8da6000640 1 -- 172.21.15.72:0/3885117427 learned_addr learned my addr 172.21.15.72:0/3885117427 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:55:31.246 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.243+0000 7f8da5600640 1 --2- 172.21.15.72:0/3885117427 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ab751c7a70 0x55ab751d9160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:31.246 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.243+0000 7f8da4c00640 1 --2- 172.21.15.72:0/3885117427 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ab751c8460 0x55ab751d7770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:31.246 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.243+0000 7f8da6000640 1 -- 172.21.15.72:0/3885117427 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ab751c8460 msgr2=0x55ab751d7770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:55:31.246 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.243+0000 7f8da6000640 1 --2- 172.21.15.72:0/3885117427 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ab751c8460 0x55ab751d7770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:31.247 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.243+0000 7f8da6000640 1 -- 172.21.15.72:0/3885117427 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ab751c7a70 msgr2=0x55ab751d9160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:55:31.247 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.243+0000 7f8da6000640 1 --2- 172.21.15.72:0/3885117427 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ab751c7a70 0x55ab751d9160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:31.247 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.243+0000 7f8da6000640 1 -- 172.21.15.72:0/3885117427 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab751d82f0 con 0x55ab751cd5d0 2024-04-26T21:55:31.247 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.243+0000 7f8da4c00640 1 --2- 172.21.15.72:0/3885117427 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ab751c8460 0x55ab751d7770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:55:31.247 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.244+0000 7f8da6000640 1 --2- 172.21.15.72:0/3885117427 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ab751cd5d0 0x55ab751d7cb0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f8d98010990 tx=0x7f8d98010e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e56b76a34ec97194 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:55:31.247 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.244+0000 7f8d86a00640 10 client.0 ms_handle_connect on v2:172.21.15.72:3300/0 2024-04-26T21:55:31.248 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.244+0000 7f8d86a00640 1 -- 172.21.15.72:0/3885117427 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d9800fc50 con 0x55ab751cd5d0 2024-04-26T21:55:31.248 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.244+0000 7f8d86a00640 1 -- 172.21.15.72:0/3885117427 <== mon.0 v2:172.21.15.72:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7f8d9800fdf0 con 0x55ab751cd5d0 2024-04-26T21:55:31.248 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.244+0000 7f8daab22740 1 -- 172.21.15.72:0/3885117427 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8d8c003a00 con 0x55ab751cd5d0 2024-04-26T21:55:31.249 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.244+0000 7f8d86a00640 1 -- 172.21.15.72:0/3885117427 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f8d9802a020 con 0x55ab751cd5d0 2024-04-26T21:55:31.249 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.244+0000 7f8daab22740 10 client.15507 fetch_fsmap learned FSMap version 27 2024-04-26T21:55:31.249 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.244+0000 7f8daab22740 1 -- 172.21.15.72:0/3885117427 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ab75244930 con 0x55ab751cd5d0 2024-04-26T21:55:31.249 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.245+0000 7f8d86a00640 1 -- 172.21.15.72:0/3885117427 <== mon.0 v2:172.21.15.72:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8d8c003a00 con 0x55ab751cd5d0 2024-04-26T21:55:31.249 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.245+0000 7f8d86a00640 1 -- 172.21.15.72:0/3885117427 <== mon.0 v2:172.21.15.72:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f8d9801d540 con 0x55ab751cd5d0 2024-04-26T21:55:31.249 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.245+0000 7f8daab22740 10 client.15507 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T21:55:31.249 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.245+0000 7f8daab22740 10 client.15507 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:55:31.249 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.245+0000 7f8daab22740 20 client.15507 populate_metadata read hostname 'smithi072' 2024-04-26T21:55:31.249 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.245+0000 7f8daab22740 1 --2- 172.21.15.72:0/3885117427 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x55ab752452f0 0x55ab75247710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:31.249 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.245+0000 7f8daab22740 4 client.15507 mds_command: new command op to 34517 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-26T21:55:31.249 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.245+0000 7f8daab22740 1 -- 172.21.15.72:0/3885117427 --> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55ab75134400 con 0x55ab752452f0 2024-04-26T21:55:31.249 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.246+0000 7f8da5600640 1 --2- 172.21.15.72:0/3885117427 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x55ab752452f0 0x55ab75247710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:31.249 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.246+0000 7f8da5600640 1 --2- 172.21.15.72:0/3885117427 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x55ab752452f0 0x55ab75247710 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a9cf11e68c06db80 server_cookie=8f824308a3f718b4 in_seq=0 out_seq=0 2024-04-26T21:55:31.249 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.246+0000 7f8d86a00640 10 client.15507 ms_handle_connect on v2:172.21.15.76:6838/327145636 2024-04-26T21:55:31.251 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.249+0000 7f8d86a00640 1 -- 172.21.15.72:0/3885117427 <== mds.0 v2:172.21.15.76:6838/327145636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55ab75134400 con 0x55ab752452f0 2024-04-26T21:55:31.251 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.249+0000 7f8d86a00640 10 client.15507 handle_command_reply: tid=0 2024-04-26T21:55:31.251 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.249+0000 7f8daab22740 1 client.15507 shutdown 2024-04-26T21:55:31.251 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.249+0000 7f8daab22740 1 -- 172.21.15.72:0/3885117427 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ab751cd5d0 msgr2=0x55ab751d7cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:31.251 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.249+0000 7f8daab22740 1 --2- 172.21.15.72:0/3885117427 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ab751cd5d0 0x55ab751d7cb0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f8d98010990 tx=0x7f8d98010e50 comp rx=0 tx=0).stop 2024-04-26T21:55:31.251 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.250+0000 7f8daab22740 1 -- 172.21.15.72:0/3885117427 shutdown_connections 2024-04-26T21:55:31.251 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.250+0000 7f8daab22740 1 --2- 172.21.15.72:0/3885117427 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x55ab752452f0 0x55ab75247710 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:31.251 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.250+0000 7f8daab22740 1 --2- 172.21.15.72:0/3885117427 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ab751cd5d0 0x55ab751d7cb0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:31.251 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.250+0000 7f8daab22740 1 --2- 172.21.15.72:0/3885117427 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ab751c8460 0x55ab751d7770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:31.251 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.250+0000 7f8daab22740 1 --2- 172.21.15.72:0/3885117427 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ab751c7a70 0x55ab751d9160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:31.251 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.250+0000 7f8daab22740 1 -- 172.21.15.72:0/3885117427 >> 172.21.15.72:0/3885117427 conn(0x55ab751c30e0 msgr2=0x55ab751c22e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:55:31.252 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.250+0000 7f8d86a00640 0 client.15507 ms_handle_reset on v2:172.21.15.76:6838/327145636 2024-04-26T21:55:31.252 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.250+0000 7f8daab22740 1 -- 172.21.15.72:0/3885117427 shutdown_connections 2024-04-26T21:55:31.252 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.250+0000 7f8daab22740 1 -- 172.21.15.72:0/3885117427 wait complete. 2024-04-26T21:55:31.252 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.250+0000 7f8daab22740 20 client.15507 trim_cache size 0 max 16384 2024-04-26T21:55:31.252 INFO:teuthology.orchestra.run.smithi072.stdout:{ 2024-04-26T21:55:31.252 INFO:teuthology.orchestra.run.smithi072.stdout: "return_code": 0, 2024-04-26T21:55:31.252 INFO:teuthology.orchestra.run.smithi072.stdout: "scrub_tag": "6040610d-5544-4f71-8702-a670463bcb18", 2024-04-26T21:55:31.252 INFO:teuthology.orchestra.run.smithi072.stdout: "mode": "asynchronous" 2024-04-26T21:55:31.252 INFO:teuthology.orchestra.run.smithi072.stdout:} 2024-04-26T21:55:31.253 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.251+0000 7f8da6a00640 1 -- 172.21.15.72:0/3907066789 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f8d880a3c40 msgr2=0x7f8d880a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:31.253 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.251+0000 7f8da6a00640 1 --2- 172.21.15.72:0/3907066789 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f8d880a3c40 0x7f8d880a6100 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x55ab751c8cc0 tx=0x7f8d98005e30 comp rx=0 tx=0).stop 2024-04-26T21:55:31.253 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.251+0000 7f8da6a00640 1 -- 172.21.15.72:0/3907066789 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8da0102860 msgr2=0x7f8da00766e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:31.253 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.251+0000 7f8da6a00640 1 --2- 172.21.15.72:0/3907066789 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8da0102860 0x7f8da00766e0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f8d9400eaa0 tx=0x7f8d9400ef60 comp rx=0 tx=0).stop 2024-04-26T21:55:31.253 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.251+0000 7f8da6a00640 1 -- 172.21.15.72:0/3907066789 shutdown_connections 2024-04-26T21:55:31.253 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.251+0000 7f8da6a00640 1 --2- 172.21.15.72:0/3907066789 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f8d880a3c40 0x7f8d880a6100 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:31.254 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.251+0000 7f8da6a00640 1 --2- 172.21.15.72:0/3907066789 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8da0104410 0x7f8da0075260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:31.254 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.251+0000 7f8da6a00640 1 --2- 172.21.15.72:0/3907066789 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8da0103a50 0x7f8da0074d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:31.254 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.251+0000 7f8da6a00640 1 --2- 172.21.15.72:0/3907066789 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8da0102860 0x7f8da00766e0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:31.254 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.251+0000 7f8da6a00640 1 -- 172.21.15.72:0/3907066789 >> 172.21.15.72:0/3907066789 conn(0x7f8da00fddd0 msgr2=0x7f8da0106e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:55:31.254 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.251+0000 7f8da6a00640 1 -- 172.21.15.72:0/3907066789 shutdown_connections 2024-04-26T21:55:31.254 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.251+0000 7f8da6a00640 1 -- 172.21.15.72:0/3907066789 wait complete. 2024-04-26T21:55:31.269 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:55:30 smithi072 ceph-mon[28138]: pgmap v451: 97 pgs: 97 active+clean; 640 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 1.6 KiB/s wr, 13 op/s 2024-04-26T21:55:31.269 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:55:30 smithi072 ceph-mon[28138]: from='client.? 172.21.15.72:0/1512029150' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T21:55:31.270 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:55:30 smithi072 ceph-mon[28138]: from='client.? 172.21.15.72:0/2099086773' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:55:31.270 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:55:30 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:55:31.270 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:55:30 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:55:31.270 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:55:30 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:55:31.273 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-26T21:55:31.273 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-26T21:55:31.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:55:30 smithi076 ceph-mon[31823]: pgmap v451: 97 pgs: 97 active+clean; 640 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 1.6 KiB/s wr, 13 op/s 2024-04-26T21:55:31.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:55:30 smithi076 ceph-mon[31823]: from='client.? 172.21.15.72:0/1512029150' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T21:55:31.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:55:30 smithi076 ceph-mon[31823]: from='client.? 172.21.15.72:0/2099086773' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:55:31.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:55:30 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:55:31.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:55:30 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:55:31.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:55:30 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:55:31.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:55:30 smithi164 ceph-mon[31916]: pgmap v451: 97 pgs: 97 active+clean; 640 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 1.6 KiB/s wr, 13 op/s 2024-04-26T21:55:31.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:55:30 smithi164 ceph-mon[31916]: from='client.? 172.21.15.72:0/1512029150' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T21:55:31.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:55:30 smithi164 ceph-mon[31916]: from='client.? 172.21.15.72:0/2099086773' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:55:31.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:55:30 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:55:31.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:55:30 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:55:31.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:55:30 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:55:31.504 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.504+0000 7f80b7e00640 1 -- 172.21.15.72:0/3116888252 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f80b8104410 msgr2=0x7f80b810aca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:31.504 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.504+0000 7f80b7e00640 1 --2- 172.21.15.72:0/3116888252 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f80b8104410 0x7f80b810aca0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f80b0009ca0 tx=0x7f80b002a8c0 comp rx=0 tx=0).stop 2024-04-26T21:55:31.504 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.504+0000 7f80b7e00640 1 -- 172.21.15.72:0/3116888252 shutdown_connections 2024-04-26T21:55:31.505 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.504+0000 7f80b7e00640 1 --2- 172.21.15.72:0/3116888252 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f80b8104410 0x7f80b810aca0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:31.505 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.504+0000 7f80b7e00640 1 --2- 172.21.15.72:0/3116888252 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f80b8103a50 0x7f80b8103ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:31.505 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.504+0000 7f80b7e00640 1 --2- 172.21.15.72:0/3116888252 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f80b8102860 0x7f80b8102c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:31.505 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.504+0000 7f80b7e00640 1 -- 172.21.15.72:0/3116888252 >> 172.21.15.72:0/3116888252 conn(0x7f80b80fddd0 msgr2=0x7f80b8100210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:55:31.505 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.504+0000 7f80b7e00640 1 -- 172.21.15.72:0/3116888252 shutdown_connections 2024-04-26T21:55:31.505 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.504+0000 7f80b7e00640 1 -- 172.21.15.72:0/3116888252 wait complete. 2024-04-26T21:55:31.507 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.505+0000 7f80b7e00640 1 Processor -- start 2024-04-26T21:55:31.508 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.505+0000 7f80b7e00640 1 -- start start 2024-04-26T21:55:31.508 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.505+0000 7f80b7e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f80b8102860 0x7f80b819ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:31.508 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.505+0000 7f80b7e00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f80b8103a50 0x7f80b819b060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:31.508 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.505+0000 7f80b7e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f80b8104410 0x7f80b81a20d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:31.508 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.505+0000 7f80b7e00640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f80b810e270 con 0x7f80b8102860 2024-04-26T21:55:31.508 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.505+0000 7f80b7e00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f80b810e0f0 con 0x7f80b8103a50 2024-04-26T21:55:31.508 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.505+0000 7f80b7e00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f80b810e3f0 con 0x7f80b8104410 2024-04-26T21:55:31.508 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.506+0000 7f80b6a00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f80b8102860 0x7f80b819ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:31.508 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.506+0000 7f80b6a00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f80b8102860 0x7f80b819ab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:56000/0 (socket says 172.21.15.72:56000) 2024-04-26T21:55:31.508 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.506+0000 7f80b6a00640 1 -- 172.21.15.72:0/3178373545 learned_addr learned my addr 172.21.15.72:0/3178373545 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:55:31.508 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.506+0000 7f80b7400640 1 --2- 172.21.15.72:0/3178373545 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f80b8104410 0x7f80b81a20d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:31.508 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.506+0000 7f80b6a00640 1 -- 172.21.15.72:0/3178373545 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f80b8104410 msgr2=0x7f80b81a20d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:31.508 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.506+0000 7f80b6000640 1 --2- 172.21.15.72:0/3178373545 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f80b8103a50 0x7f80b819b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:31.509 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.506+0000 7f80b6a00640 1 --2- 172.21.15.72:0/3178373545 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f80b8104410 0x7f80b81a20d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:31.509 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.506+0000 7f80b6a00640 1 -- 172.21.15.72:0/3178373545 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f80b8103a50 msgr2=0x7f80b819b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:31.509 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.506+0000 7f80b6a00640 1 --2- 172.21.15.72:0/3178373545 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f80b8103a50 0x7f80b819b060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:31.509 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.506+0000 7f80b6a00640 1 -- 172.21.15.72:0/3178373545 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80b81a2740 con 0x7f80b8102860 2024-04-26T21:55:31.509 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.506+0000 7f80b6a00640 1 --2- 172.21.15.72:0/3178373545 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f80b8102860 0x7f80b819ab20 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f80a800d770 tx=0x7f80a800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:55:31.509 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.507+0000 7f80af400640 1 -- 172.21.15.72:0/3178373545 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80a8017070 con 0x7f80b8102860 2024-04-26T21:55:31.509 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.507+0000 7f80af400640 1 -- 172.21.15.72:0/3178373545 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f80a8009c30 con 0x7f80b8102860 2024-04-26T21:55:31.509 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.507+0000 7f80af400640 1 -- 172.21.15.72:0/3178373545 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80a8003c50 con 0x7f80b8102860 2024-04-26T21:55:31.509 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.507+0000 7f80b7e00640 1 -- 172.21.15.72:0/3178373545 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80b81a2a30 con 0x7f80b8102860 2024-04-26T21:55:31.509 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.507+0000 7f80b7e00640 1 -- 172.21.15.72:0/3178373545 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80b81a2f30 con 0x7f80b8102860 2024-04-26T21:55:31.512 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.510+0000 7f80c03ce740 1 Processor -- start 2024-04-26T21:55:31.512 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.510+0000 7f80af400640 1 -- 172.21.15.72:0/3178373545 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f80a8003df0 con 0x7f80b8102860 2024-04-26T21:55:31.512 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.511+0000 7f80c03ce740 1 -- start start 2024-04-26T21:55:31.512 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.511+0000 7f80c03ce740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ccceb86960 0x55ccceb86d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:31.512 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.511+0000 7f80c03ce740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ccceb87350 0x55ccceb8be50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:31.512 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.511+0000 7f80c03ce740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ccceb8c4c0 0x55ccceb8e950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:31.512 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.511+0000 7f80c03ce740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55cccea97030 con 0x55ccceb8c4c0 2024-04-26T21:55:31.512 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.511+0000 7f80c03ce740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55ccceab27f0 con 0x55ccceb87350 2024-04-26T21:55:31.513 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.511+0000 7f80c03ce740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55cccea6d7e0 con 0x55ccceb86960 2024-04-26T21:55:31.513 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.511+0000 7f80b6000640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ccceb8c4c0 0x55ccceb8e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:31.513 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.511+0000 7f80b6000640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ccceb8c4c0 0x55ccceb8e950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:56004/0 (socket says 172.21.15.72:56004) 2024-04-26T21:55:31.513 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.511+0000 7f80b6000640 1 -- 172.21.15.72:0/2732170679 learned_addr learned my addr 172.21.15.72:0/2732170679 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:55:31.513 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.512+0000 7f80b7400640 1 --2- 172.21.15.72:0/2732170679 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ccceb86960 0x55ccceb86d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:31.513 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.512+0000 7f80b6a00640 1 --2- 172.21.15.72:0/2732170679 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ccceb87350 0x55ccceb8be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:31.513 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.512+0000 7f80b6000640 1 -- 172.21.15.72:0/2732170679 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ccceb86960 msgr2=0x55ccceb86d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:55:31.513 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.512+0000 7f80b6000640 1 --2- 172.21.15.72:0/2732170679 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ccceb86960 0x55ccceb86d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:31.513 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.512+0000 7f80b6000640 1 -- 172.21.15.72:0/2732170679 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ccceb87350 msgr2=0x55ccceb8be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:55:31.513 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.512+0000 7f80b6000640 1 --2- 172.21.15.72:0/2732170679 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ccceb87350 0x55ccceb8be50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:31.513 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.512+0000 7f80b6000640 1 -- 172.21.15.72:0/2732170679 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cccea79050 con 0x55ccceb8c4c0 2024-04-26T21:55:31.513 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.512+0000 7f80b6000640 1 --2- 172.21.15.72:0/2732170679 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ccceb8c4c0 0x55ccceb8e950 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f80a4005bc0 tx=0x7f80a400e2f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e84510d27b9654bc server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:55:31.517 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.512+0000 7f80a3400640 1 -- 172.21.15.72:0/2732170679 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80a4016070 con 0x55ccceb8c4c0 2024-04-26T21:55:31.517 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.512+0000 7f80a3400640 1 -- 172.21.15.72:0/2732170679 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f80a400ed20 con 0x55ccceb8c4c0 2024-04-26T21:55:31.517 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.512+0000 7f80a3400640 1 -- 172.21.15.72:0/2732170679 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80a4004d60 con 0x55ccceb8c4c0 2024-04-26T21:55:31.517 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.513+0000 7f80af400640 1 --2- 172.21.15.72:0/3178373545 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f809c0a3c40 0x7f809c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:31.517 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.513+0000 7f80af400640 1 -- 172.21.15.72:0/3178373545 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7f80a80d0040 con 0x7f80b8102860 2024-04-26T21:55:31.517 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.513+0000 7f80b7400640 1 --2- 172.21.15.72:0/3178373545 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f809c0a3c40 0x7f809c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:31.517 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.513+0000 7f80c03ce740 1 -- 172.21.15.72:0/2732170679 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ccceb8c4c0 msgr2=0x55ccceb8e950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:31.518 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.513+0000 7f80c03ce740 1 --2- 172.21.15.72:0/2732170679 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ccceb8c4c0 0x55ccceb8e950 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f80a4005bc0 tx=0x7f80a400e2f0 comp rx=0 tx=0).stop 2024-04-26T21:55:31.518 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.513+0000 7f80c03ce740 1 -- 172.21.15.72:0/2732170679 shutdown_connections 2024-04-26T21:55:31.518 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.513+0000 7f80c03ce740 1 --2- 172.21.15.72:0/2732170679 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ccceb8c4c0 0x55ccceb8e950 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:31.518 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.513+0000 7f80c03ce740 1 --2- 172.21.15.72:0/2732170679 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ccceb87350 0x55ccceb8be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:31.518 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.513+0000 7f80c03ce740 1 --2- 172.21.15.72:0/2732170679 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ccceb86960 0x55ccceb86d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:31.518 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.513+0000 7f80c03ce740 1 -- 172.21.15.72:0/2732170679 >> 172.21.15.72:0/2732170679 conn(0x55ccceb81fd0 msgr2=0x55ccceb823d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:55:31.518 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.513+0000 7f80c03ce740 1 -- 172.21.15.72:0/2732170679 shutdown_connections 2024-04-26T21:55:31.518 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.514+0000 7f80c03ce740 1 -- 172.21.15.72:0/2732170679 wait complete. 2024-04-26T21:55:31.518 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.514+0000 7f80b7400640 1 --2- 172.21.15.72:0/3178373545 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f809c0a3c40 0x7f809c0a6100 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f80b0006010 tx=0x7f80b0031980 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:55:31.518 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.514+0000 7f80c03ce740 1 -- start start 2024-04-26T21:55:31.518 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.514+0000 7f80c03ce740 1 Processor -- start 2024-04-26T21:55:31.518 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.515+0000 7f80c03ce740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ccceb86960 0x55ccceb907a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:31.518 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.515+0000 7f80c03ce740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ccceb87350 0x55ccceb90ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:31.519 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.515+0000 7f80c03ce740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ccceb8c4c0 0x55ccceb97d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:31.519 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.515+0000 7f80c03ce740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55ccceaef200 con 0x55ccceb86960 2024-04-26T21:55:31.519 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.515+0000 7f80c03ce740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55ccceae9ee0 con 0x55ccceb87350 2024-04-26T21:55:31.519 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.515+0000 7f80c03ce740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55ccceaf6e70 con 0x55ccceb8c4c0 2024-04-26T21:55:31.519 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.515+0000 7f80c03ce740 10 client.0 _ll_register_callbacks cb 0x55ccceae0270 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:55:31.519 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.515+0000 7f80b6a00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ccceb86960 0x55ccceb907a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:31.519 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.515+0000 7f80b6a00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ccceb86960 0x55ccceb907a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:56010/0 (socket says 172.21.15.72:56010) 2024-04-26T21:55:31.519 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.515+0000 7f80b6a00640 1 -- 172.21.15.72:0/2557779456 learned_addr learned my addr 172.21.15.72:0/2557779456 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:55:31.519 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.515+0000 7f80b6a00640 1 -- 172.21.15.72:0/2557779456 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ccceb8c4c0 msgr2=0x55ccceb97d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:55:31.519 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.515+0000 7f80b6000640 1 --2- 172.21.15.72:0/2557779456 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ccceb87350 0x55ccceb90ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:31.519 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.515+0000 7f80b6a00640 1 --2- 172.21.15.72:0/2557779456 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ccceb8c4c0 0x55ccceb97d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:31.519 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.515+0000 7f80b6a00640 1 -- 172.21.15.72:0/2557779456 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ccceb87350 msgr2=0x55ccceb90ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:55:31.519 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.515+0000 7f80b6a00640 1 --2- 172.21.15.72:0/2557779456 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ccceb87350 0x55ccceb90ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:31.519 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.515+0000 7f80b6a00640 1 -- 172.21.15.72:0/2557779456 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ccce8c67d0 con 0x55ccceb86960 2024-04-26T21:55:31.519 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.516+0000 7f80b6a00640 1 --2- 172.21.15.72:0/2557779456 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ccceb86960 0x55ccceb907a0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f80a809bee0 tx=0x7f80a809bd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf8e732ee421bdfa server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:55:31.520 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.516+0000 7f8097e00640 10 client.0 ms_handle_connect on v2:172.21.15.72:3300/0 2024-04-26T21:55:31.520 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.516+0000 7f8097e00640 1 -- 172.21.15.72:0/2557779456 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80a8020070 con 0x55ccceb86960 2024-04-26T21:55:31.520 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.516+0000 7f8097e00640 1 -- 172.21.15.72:0/2557779456 <== mon.0 v2:172.21.15.72:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7f80a80a49c0 con 0x55ccceb86960 2024-04-26T21:55:31.520 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.516+0000 7f80c03ce740 1 -- 172.21.15.72:0/2557779456 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f80a400ed20 con 0x55ccceb86960 2024-04-26T21:55:31.520 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.516+0000 7f8097e00640 1 -- 172.21.15.72:0/2557779456 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f80a8009de0 con 0x55ccceb86960 2024-04-26T21:55:31.520 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.516+0000 7f80c03ce740 10 client.15519 fetch_fsmap learned FSMap version 27 2024-04-26T21:55:31.520 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.516+0000 7f80c03ce740 1 -- 172.21.15.72:0/2557779456 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ccce7fc3f0 con 0x55ccceb86960 2024-04-26T21:55:31.520 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.516+0000 7f8097e00640 1 -- 172.21.15.72:0/2557779456 <== mon.0 v2:172.21.15.72:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f80a400ed20 con 0x55ccceb86960 2024-04-26T21:55:31.520 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.516+0000 7f8097e00640 1 -- 172.21.15.72:0/2557779456 <== mon.0 v2:172.21.15.72:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f80a80a0100 con 0x55ccceb86960 2024-04-26T21:55:31.520 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.517+0000 7f80c03ce740 10 client.15519 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T21:55:31.520 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.517+0000 7f80c03ce740 10 client.15519 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:55:31.520 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.517+0000 7f80c03ce740 20 client.15519 populate_metadata read hostname 'smithi072' 2024-04-26T21:55:31.520 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.517+0000 7f80c03ce740 1 --2- 172.21.15.72:0/2557779456 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x55ccceb9c280 0x55ccceb9e740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:31.520 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.517+0000 7f80c03ce740 4 client.15519 mds_command: new command op to 34517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:55:31.521 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.517+0000 7f80c03ce740 1 -- 172.21.15.72:0/2557779456 --> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ccceaf87e0 con 0x55ccceb9c280 2024-04-26T21:55:31.521 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.517+0000 7f80b6000640 1 --2- 172.21.15.72:0/2557779456 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x55ccceb9c280 0x55ccceb9e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:31.522 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.518+0000 7f80b6000640 1 --2- 172.21.15.72:0/2557779456 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x55ccceb9c280 0x55ccceb9e740 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c5801fa7a76d20 server_cookie=fd49151d9cf4998 in_seq=0 out_seq=0 2024-04-26T21:55:31.522 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.518+0000 7f8097e00640 10 client.15519 ms_handle_connect on v2:172.21.15.76:6838/327145636 2024-04-26T21:55:31.522 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.520+0000 7f8097e00640 1 -- 172.21.15.72:0/2557779456 <== mds.0 v2:172.21.15.76:6838/327145636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ccceaf87e0 con 0x55ccceb9c280 2024-04-26T21:55:31.522 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.520+0000 7f8097e00640 10 client.15519 handle_command_reply: tid=0 2024-04-26T21:55:31.522 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.520+0000 7f80c03ce740 1 client.15519 shutdown 2024-04-26T21:55:31.522 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.520+0000 7f80c03ce740 1 -- 172.21.15.72:0/2557779456 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ccceb86960 msgr2=0x55ccceb907a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:31.522 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.520+0000 7f80c03ce740 1 --2- 172.21.15.72:0/2557779456 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ccceb86960 0x55ccceb907a0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f80a809bee0 tx=0x7f80a809bd20 comp rx=0 tx=0).stop 2024-04-26T21:55:31.522 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.520+0000 7f80c03ce740 1 -- 172.21.15.72:0/2557779456 shutdown_connections 2024-04-26T21:55:31.522 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.520+0000 7f80c03ce740 1 --2- 172.21.15.72:0/2557779456 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x55ccceb9c280 0x55ccceb9e740 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:31.522 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.520+0000 7f80c03ce740 1 --2- 172.21.15.72:0/2557779456 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ccceb8c4c0 0x55ccceb97d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:31.523 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.520+0000 7f80c03ce740 1 --2- 172.21.15.72:0/2557779456 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ccceb87350 0x55ccceb90ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:31.523 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.520+0000 7f80c03ce740 1 --2- 172.21.15.72:0/2557779456 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ccceb86960 0x55ccceb907a0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:31.523 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.520+0000 7f80c03ce740 1 -- 172.21.15.72:0/2557779456 >> 172.21.15.72:0/2557779456 conn(0x55ccceb81fd0 msgr2=0x55ccceb811f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:55:31.523 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.520+0000 7f8097e00640 0 client.15519 ms_handle_reset on v2:172.21.15.76:6838/327145636 2024-04-26T21:55:31.523 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.520+0000 7f80c03ce740 1 -- 172.21.15.72:0/2557779456 shutdown_connections 2024-04-26T21:55:31.523 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.520+0000 7f80c03ce740 1 -- 172.21.15.72:0/2557779456 wait complete. 2024-04-26T21:55:31.523 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.521+0000 7f80c03ce740 20 client.15519 trim_cache size 0 max 16384 2024-04-26T21:55:31.541 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.539+0000 7f80c03ce740 1 Processor -- start 2024-04-26T21:55:31.541 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.539+0000 7f80c03ce740 1 -- start start 2024-04-26T21:55:31.541 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.540+0000 7f80c03ce740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ccceb86960 0x55ccceb81070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:31.541 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.540+0000 7f80c03ce740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ccceb87350 0x55ccceb815b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:31.541 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.540+0000 7f80c03ce740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ccceb8c4c0 0x55ccceb97ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:31.541 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.540+0000 7f80c03ce740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f80b81ada20 con 0x55ccceb86960 2024-04-26T21:55:31.541 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.540+0000 7f80c03ce740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f80b81ac5b0 con 0x55ccceb87350 2024-04-26T21:55:31.541 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.540+0000 7f80c03ce740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f80b81acc80 con 0x55ccceb8c4c0 2024-04-26T21:55:31.541 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.540+0000 7f80b6a00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ccceb86960 0x55ccceb81070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:31.542 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.540+0000 7f80b6a00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ccceb86960 0x55ccceb81070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:56012/0 (socket says 172.21.15.72:56012) 2024-04-26T21:55:31.542 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.540+0000 7f80b6a00640 1 -- 172.21.15.72:0/855495085 learned_addr learned my addr 172.21.15.72:0/855495085 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:55:31.542 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.540+0000 7f80b6a00640 1 -- 172.21.15.72:0/855495085 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ccceb8c4c0 msgr2=0x55ccceb97ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:55:31.542 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.540+0000 7f80b6a00640 1 --2- 172.21.15.72:0/855495085 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ccceb8c4c0 0x55ccceb97ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:31.542 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.540+0000 7f80b6a00640 1 -- 172.21.15.72:0/855495085 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ccceb87350 msgr2=0x55ccceb815b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:55:31.542 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.540+0000 7f80b6000640 1 --2- 172.21.15.72:0/855495085 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ccceb87350 0x55ccceb815b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:31.542 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.540+0000 7f80b6a00640 1 --2- 172.21.15.72:0/855495085 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ccceb87350 0x55ccceb815b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:31.542 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.540+0000 7f80b6a00640 1 -- 172.21.15.72:0/855495085 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ccce7bc0f0 con 0x55ccceb86960 2024-04-26T21:55:31.542 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.541+0000 7f80b6a00640 1 --2- 172.21.15.72:0/855495085 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ccceb86960 0x55ccceb81070 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f80a80d6040 tx=0x7f80a809be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60e8f43b3853be0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:55:31.542 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.541+0000 7f80a2a00640 1 -- 172.21.15.72:0/855495085 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80a8020070 con 0x55ccceb86960 2024-04-26T21:55:31.542 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.541+0000 7f80a2a00640 1 -- 172.21.15.72:0/855495085 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f80a8015070 con 0x55ccceb86960 2024-04-26T21:55:31.542 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.541+0000 7f80a2a00640 1 -- 172.21.15.72:0/855495085 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80a8098850 con 0x55ccceb86960 2024-04-26T21:55:31.545 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.542+0000 7f80c03ce740 1 -- 172.21.15.72:0/855495085 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ccceb86960 msgr2=0x55ccceb81070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:31.545 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.542+0000 7f80c03ce740 1 --2- 172.21.15.72:0/855495085 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ccceb86960 0x55ccceb81070 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f80a80d6040 tx=0x7f80a809be50 comp rx=0 tx=0).stop 2024-04-26T21:55:31.545 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.542+0000 7f80c03ce740 1 -- 172.21.15.72:0/855495085 shutdown_connections 2024-04-26T21:55:31.546 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.542+0000 7f80c03ce740 1 --2- 172.21.15.72:0/855495085 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ccceb8c4c0 0x55ccceb97ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:31.546 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.542+0000 7f80c03ce740 1 --2- 172.21.15.72:0/855495085 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ccceb87350 0x55ccceb815b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:31.546 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.542+0000 7f80c03ce740 1 --2- 172.21.15.72:0/855495085 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ccceb86960 0x55ccceb81070 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:31.546 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.542+0000 7f80c03ce740 1 -- 172.21.15.72:0/855495085 >> 172.21.15.72:0/855495085 conn(0x55ccceb81fd0 msgr2=0x55ccceb80960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:55:31.546 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.542+0000 7f80c03ce740 1 -- 172.21.15.72:0/855495085 shutdown_connections 2024-04-26T21:55:31.546 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.542+0000 7f80c03ce740 1 -- 172.21.15.72:0/855495085 wait complete. 2024-04-26T21:55:31.546 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.543+0000 7f80c03ce740 1 -- start start 2024-04-26T21:55:31.546 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.543+0000 7f80c03ce740 1 Processor -- start 2024-04-26T21:55:31.546 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.543+0000 7f80c03ce740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ccceb86960 0x55ccceb951c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:31.546 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.543+0000 7f80c03ce740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ccceb87350 0x55ccceb95700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:31.546 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.543+0000 7f80c03ce740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ccceb8c4c0 0x55ccceb96710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:31.546 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.543+0000 7f80c03ce740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f80b81a9ec0 con 0x55ccceb86960 2024-04-26T21:55:31.546 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.543+0000 7f80c03ce740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f80b81ad350 con 0x55ccceb8c4c0 2024-04-26T21:55:31.547 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.543+0000 7f80c03ce740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f80b81aa530 con 0x55ccceb87350 2024-04-26T21:55:31.547 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.543+0000 7f80c03ce740 10 client.0 _ll_register_callbacks cb 0x7f80a80046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:55:31.547 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.543+0000 7f80b6a00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ccceb86960 0x55ccceb951c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:31.547 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.544+0000 7f80b6a00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ccceb86960 0x55ccceb951c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:56022/0 (socket says 172.21.15.72:56022) 2024-04-26T21:55:31.547 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.544+0000 7f80b6a00640 1 -- 172.21.15.72:0/3349356649 learned_addr learned my addr 172.21.15.72:0/3349356649 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:55:31.547 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.544+0000 7f80b6a00640 1 -- 172.21.15.72:0/3349356649 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ccceb87350 msgr2=0x55ccceb95700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:55:31.547 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.544+0000 7f80b6a00640 1 --2- 172.21.15.72:0/3349356649 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ccceb87350 0x55ccceb95700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:31.547 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.544+0000 7f80b7400640 1 --2- 172.21.15.72:0/3349356649 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ccceb8c4c0 0x55ccceb96710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:31.547 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.544+0000 7f80b6a00640 1 -- 172.21.15.72:0/3349356649 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ccceb8c4c0 msgr2=0x55ccceb96710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:55:31.547 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.544+0000 7f80b6a00640 1 --2- 172.21.15.72:0/3349356649 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ccceb8c4c0 0x55ccceb96710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:31.547 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.544+0000 7f80b6a00640 1 -- 172.21.15.72:0/3349356649 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ccceab2a30 con 0x55ccceb86960 2024-04-26T21:55:31.547 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.544+0000 7f80b6a00640 1 --2- 172.21.15.72:0/3349356649 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ccceb86960 0x55ccceb951c0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f80a809a490 tx=0x7f80a80a4bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=496c283c91dbe8e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:55:31.547 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.544+0000 7f8097e00640 10 client.0 ms_handle_connect on v2:172.21.15.72:3300/0 2024-04-26T21:55:31.547 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.544+0000 7f8097e00640 1 -- 172.21.15.72:0/3349356649 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80a8020070 con 0x55ccceb86960 2024-04-26T21:55:31.548 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.544+0000 7f8097e00640 1 -- 172.21.15.72:0/3349356649 <== mon.0 v2:172.21.15.72:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7f80a809b830 con 0x55ccceb86960 2024-04-26T21:55:31.548 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.544+0000 7f80c03ce740 1 -- 172.21.15.72:0/3349356649 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f80a8015070 con 0x55ccceb86960 2024-04-26T21:55:31.548 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.545+0000 7f8097e00640 1 -- 172.21.15.72:0/3349356649 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f80a809c1e0 con 0x55ccceb86960 2024-04-26T21:55:31.548 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.545+0000 7f80c03ce740 10 client.15525 fetch_fsmap learned FSMap version 27 2024-04-26T21:55:31.548 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.545+0000 7f80c03ce740 1 -- 172.21.15.72:0/3349356649 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ccceb97e40 con 0x55ccceb86960 2024-04-26T21:55:31.548 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.545+0000 7f8097e00640 1 -- 172.21.15.72:0/3349356649 <== mon.0 v2:172.21.15.72:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f80a8015070 con 0x55ccceb86960 2024-04-26T21:55:31.548 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.545+0000 7f8097e00640 1 -- 172.21.15.72:0/3349356649 <== mon.0 v2:172.21.15.72:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f80a80040e0 con 0x55ccceb86960 2024-04-26T21:55:31.548 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.545+0000 7f80c03ce740 10 client.15525 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T21:55:31.548 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.545+0000 7f80c03ce740 10 client.15525 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:55:31.548 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.545+0000 7f80c03ce740 20 client.15525 populate_metadata read hostname 'smithi072' 2024-04-26T21:55:31.548 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.545+0000 7f80c03ce740 1 --2- 172.21.15.72:0/3349356649 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x55cccec02d10 0x55cccec05130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:31.548 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.545+0000 7f80c03ce740 4 client.15525 mds_command: new command op to 34517 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-26T21:55:31.548 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.545+0000 7f80c03ce740 1 -- 172.21.15.72:0/3349356649 --> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ccceaf2650 con 0x55cccec02d10 2024-04-26T21:55:31.550 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.546+0000 7f80b6000640 1 --2- 172.21.15.72:0/3349356649 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x55cccec02d10 0x55cccec05130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:31.550 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.547+0000 7f80b6000640 1 --2- 172.21.15.72:0/3349356649 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x55cccec02d10 0x55cccec05130 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fe38a4a578a41f3e server_cookie=10d58aa223ae1df in_seq=0 out_seq=0 2024-04-26T21:55:31.550 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.547+0000 7f8097e00640 10 client.15525 ms_handle_connect on v2:172.21.15.76:6838/327145636 2024-04-26T21:55:31.550 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.547+0000 7f8097e00640 1 -- 172.21.15.72:0/3349356649 <== mds.0 v2:172.21.15.76:6838/327145636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55ccceaf2650 con 0x55cccec02d10 2024-04-26T21:55:31.550 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.547+0000 7f8097e00640 10 client.15525 handle_command_reply: tid=0 2024-04-26T21:55:31.550 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.547+0000 7f80c03ce740 1 client.15525 shutdown 2024-04-26T21:55:31.550 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.547+0000 7f80c03ce740 1 -- 172.21.15.72:0/3349356649 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ccceb86960 msgr2=0x55ccceb951c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:31.550 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.547+0000 7f80c03ce740 1 --2- 172.21.15.72:0/3349356649 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ccceb86960 0x55ccceb951c0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f80a809a490 tx=0x7f80a80a4bf0 comp rx=0 tx=0).stop 2024-04-26T21:55:31.550 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.547+0000 7f80c03ce740 1 -- 172.21.15.72:0/3349356649 shutdown_connections 2024-04-26T21:55:31.550 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.547+0000 7f80c03ce740 1 --2- 172.21.15.72:0/3349356649 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x55cccec02d10 0x55cccec05130 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:31.551 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.547+0000 7f80c03ce740 1 --2- 172.21.15.72:0/3349356649 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ccceb8c4c0 0x55ccceb96710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:31.551 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.547+0000 7f80c03ce740 1 --2- 172.21.15.72:0/3349356649 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ccceb87350 0x55ccceb95700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:31.551 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.547+0000 7f8097e00640 0 client.15525 ms_handle_reset on v2:172.21.15.76:6838/327145636 2024-04-26T21:55:31.551 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.547+0000 7f80c03ce740 1 --2- 172.21.15.72:0/3349356649 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ccceb86960 0x55ccceb951c0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:31.551 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.547+0000 7f80c03ce740 1 -- 172.21.15.72:0/3349356649 >> 172.21.15.72:0/3349356649 conn(0x55ccceb81fd0 msgr2=0x55ccceb811f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:55:31.551 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.548+0000 7f80c03ce740 1 -- 172.21.15.72:0/3349356649 shutdown_connections 2024-04-26T21:55:31.551 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.548+0000 7f80c03ce740 1 -- 172.21.15.72:0/3349356649 wait complete. 2024-04-26T21:55:31.551 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.548+0000 7f80c03ce740 20 client.15525 trim_cache size 0 max 16384 2024-04-26T21:55:31.551 INFO:teuthology.orchestra.run.smithi072.stdout:{ 2024-04-26T21:55:31.551 INFO:teuthology.orchestra.run.smithi072.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-26T21:55:31.551 INFO:teuthology.orchestra.run.smithi072.stdout: "scrubs": { 2024-04-26T21:55:31.551 INFO:teuthology.orchestra.run.smithi072.stdout: "6040610d-5544-4f71-8702-a670463bcb18": { 2024-04-26T21:55:31.552 INFO:teuthology.orchestra.run.smithi072.stdout: "path": "/", 2024-04-26T21:55:31.552 INFO:teuthology.orchestra.run.smithi072.stdout: "tag": "6040610d-5544-4f71-8702-a670463bcb18", 2024-04-26T21:55:31.552 INFO:teuthology.orchestra.run.smithi072.stdout: "options": "recursive,force" 2024-04-26T21:55:31.552 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:55:31.552 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:55:31.552 INFO:teuthology.orchestra.run.smithi072.stdout:} 2024-04-26T21:55:31.552 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.548+0000 7f80b7e00640 1 -- 172.21.15.72:0/3178373545 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f809c0a3c40 msgr2=0x7f809c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:31.552 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.548+0000 7f80b7e00640 1 --2- 172.21.15.72:0/3178373545 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f809c0a3c40 0x7f809c0a6100 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f80b0006010 tx=0x7f80b0031980 comp rx=0 tx=0).stop 2024-04-26T21:55:31.553 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.548+0000 7f80b7e00640 1 -- 172.21.15.72:0/3178373545 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f80b8102860 msgr2=0x7f80b819ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:31.553 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.548+0000 7f80b7e00640 1 --2- 172.21.15.72:0/3178373545 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f80b8102860 0x7f80b819ab20 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f80a800d770 tx=0x7f80a800dc30 comp rx=0 tx=0).stop 2024-04-26T21:55:31.553 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.548+0000 7f80b7e00640 1 -- 172.21.15.72:0/3178373545 shutdown_connections 2024-04-26T21:55:31.553 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.549+0000 7f80b7e00640 1 --2- 172.21.15.72:0/3178373545 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f809c0a3c40 0x7f809c0a6100 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:31.553 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.549+0000 7f80b7e00640 1 --2- 172.21.15.72:0/3178373545 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f80b8104410 0x7f80b81a20d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:31.553 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.549+0000 7f80b7e00640 1 --2- 172.21.15.72:0/3178373545 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f80b8103a50 0x7f80b819b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:31.553 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.549+0000 7f80b7e00640 1 --2- 172.21.15.72:0/3178373545 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f80b8102860 0x7f80b819ab20 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:31.553 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.549+0000 7f80b7e00640 1 -- 172.21.15.72:0/3178373545 >> 172.21.15.72:0/3178373545 conn(0x7f80b80fddd0 msgr2=0x7f80b8074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:55:31.553 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.549+0000 7f80b7e00640 1 -- 172.21.15.72:0/3178373545 shutdown_connections 2024-04-26T21:55:31.553 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:55:31.549+0000 7f80b7e00640 1 -- 172.21.15.72:0/3178373545 wait complete. 2024-04-26T21:55:31.565 INFO:tasks.cephfs.filesystem:scrub status for tag:6040610d-5544-4f71-8702-a670463bcb18 - {'path': '/', 'tag': '6040610d-5544-4f71-8702-a670463bcb18', 'options': 'recursive,force'} 2024-04-26T21:55:32.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:55:31 smithi076 ceph-mon[31823]: from='client.? 172.21.15.72:0/3665492020' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:55:32.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:55:31 smithi076 ceph-mon[31823]: scrub queued for path: / 2024-04-26T21:55:32.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:55:31 smithi076 ceph-mon[31823]: scrub summary: idle+waiting paths [/] 2024-04-26T21:55:32.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:55:31 smithi076 ceph-mon[31823]: scrub summary: active paths [/] 2024-04-26T21:55:32.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:55:31 smithi076 ceph-mon[31823]: scrub summary: idle+waiting paths [/] 2024-04-26T21:55:32.321 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:55:31 smithi072 ceph-mon[28138]: from='client.? 172.21.15.72:0/3665492020' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:55:32.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:55:31 smithi072 ceph-mon[28138]: scrub queued for path: / 2024-04-26T21:55:32.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:55:31 smithi072 ceph-mon[28138]: scrub summary: idle+waiting paths [/] 2024-04-26T21:55:32.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:55:31 smithi072 ceph-mon[28138]: scrub summary: active paths [/] 2024-04-26T21:55:32.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:55:31 smithi072 ceph-mon[28138]: scrub summary: idle+waiting paths [/] 2024-04-26T21:55:32.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:55:31 smithi164 ceph-mon[31916]: from='client.? 172.21.15.72:0/3665492020' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:55:32.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:55:31 smithi164 ceph-mon[31916]: scrub queued for path: / 2024-04-26T21:55:32.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:55:31 smithi164 ceph-mon[31916]: scrub summary: idle+waiting paths [/] 2024-04-26T21:55:32.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:55:31 smithi164 ceph-mon[31916]: scrub summary: active paths [/] 2024-04-26T21:55:32.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:55:31 smithi164 ceph-mon[31916]: scrub summary: idle+waiting paths [/] 2024-04-26T21:55:33.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:55:32 smithi076 ceph-mon[31823]: pgmap v452: 97 pgs: 97 active+clean; 640 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.6 KiB/s wr, 10 op/s 2024-04-26T21:55:33.321 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:55:32 smithi072 ceph-mon[28138]: pgmap v452: 97 pgs: 97 active+clean; 640 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.6 KiB/s wr, 10 op/s 2024-04-26T21:55:33.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:55:32 smithi164 ceph-mon[31916]: pgmap v452: 97 pgs: 97 active+clean; 640 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.6 KiB/s wr, 10 op/s 2024-04-26T21:55:35.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:55:34 smithi076 ceph-mon[31823]: pgmap v453: 97 pgs: 97 active+clean; 640 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2024-04-26T21:55:35.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:55:34 smithi076 ceph-mon[31823]: scrub summary: idle 2024-04-26T21:55:35.321 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:55:34 smithi072 ceph-mon[28138]: pgmap v453: 97 pgs: 97 active+clean; 640 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2024-04-26T21:55:35.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:55:34 smithi072 ceph-mon[28138]: scrub summary: idle 2024-04-26T21:55:35.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:55:34 smithi164 ceph-mon[31916]: pgmap v453: 97 pgs: 97 active+clean; 640 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2024-04-26T21:55:35.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:55:34 smithi164 ceph-mon[31916]: scrub summary: idle 2024-04-26T21:55:37.317 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:55:36 smithi076 ceph-mon[31823]: pgmap v454: 97 pgs: 97 active+clean; 645 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 1.8 KiB/s wr, 13 op/s 2024-04-26T21:55:37.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:55:36 smithi072 ceph-mon[28138]: pgmap v454: 97 pgs: 97 active+clean; 645 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 1.8 KiB/s wr, 13 op/s 2024-04-26T21:55:37.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:55:36 smithi164 ceph-mon[31916]: pgmap v454: 97 pgs: 97 active+clean; 645 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 1.8 KiB/s wr, 13 op/s 2024-04-26T21:55:39.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:55:38 smithi076 ceph-mon[31823]: pgmap v455: 97 pgs: 97 active+clean; 645 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1023 B/s wr, 9 op/s 2024-04-26T21:55:39.321 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:55:38 smithi072 ceph-mon[28138]: pgmap v455: 97 pgs: 97 active+clean; 645 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1023 B/s wr, 9 op/s 2024-04-26T21:55:39.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:55:38 smithi164 ceph-mon[31916]: pgmap v455: 97 pgs: 97 active+clean; 645 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1023 B/s wr, 9 op/s 2024-04-26T21:55:41.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:55:40 smithi076 ceph-mon[31823]: pgmap v456: 97 pgs: 97 active+clean; 645 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 1.1 KiB/s wr, 12 op/s 2024-04-26T21:55:41.321 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:55:40 smithi072 ceph-mon[28138]: pgmap v456: 97 pgs: 97 active+clean; 645 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 1.1 KiB/s wr, 12 op/s 2024-04-26T21:55:41.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:55:40 smithi164 ceph-mon[31916]: pgmap v456: 97 pgs: 97 active+clean; 645 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 1.1 KiB/s wr, 12 op/s 2024-04-26T21:55:43.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:55:43 smithi076 ceph-mon[31823]: pgmap v457: 97 pgs: 97 active+clean; 645 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 682 B/s wr, 9 op/s 2024-04-26T21:55:43.321 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:55:43 smithi072 ceph-mon[28138]: pgmap v457: 97 pgs: 97 active+clean; 645 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 682 B/s wr, 9 op/s 2024-04-26T21:55:43.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:55:43 smithi164 ceph-mon[31916]: pgmap v457: 97 pgs: 97 active+clean; 645 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 682 B/s wr, 9 op/s 2024-04-26T21:55:45.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:55:45 smithi076 ceph-mon[31823]: pgmap v458: 97 pgs: 97 active+clean; 645 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 682 B/s wr, 9 op/s 2024-04-26T21:55:45.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:55:45 smithi072 ceph-mon[28138]: pgmap v458: 97 pgs: 97 active+clean; 645 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 682 B/s wr, 9 op/s 2024-04-26T21:55:45.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:55:45 smithi164 ceph-mon[31916]: pgmap v458: 97 pgs: 97 active+clean; 645 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 682 B/s wr, 9 op/s 2024-04-26T21:55:47.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:55:47 smithi076 ceph-mon[31823]: pgmap v459: 97 pgs: 97 active+clean; 650 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-04-26T21:55:47.321 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:55:47 smithi072 ceph-mon[28138]: pgmap v459: 97 pgs: 97 active+clean; 650 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-04-26T21:55:47.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:55:47 smithi164 ceph-mon[31916]: pgmap v459: 97 pgs: 97 active+clean; 650 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-04-26T21:55:48.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:55:48 smithi164 ceph-mon[31916]: pgmap v460: 97 pgs: 97 active+clean; 650 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 9 op/s 2024-04-26T21:55:48.564 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:55:48 smithi076 ceph-mon[31823]: pgmap v460: 97 pgs: 97 active+clean; 650 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 9 op/s 2024-04-26T21:55:48.571 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:55:48 smithi072 ceph-mon[28138]: pgmap v460: 97 pgs: 97 active+clean; 650 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 9 op/s 2024-04-26T21:55:51.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:55:50 smithi164 ceph-mon[31916]: pgmap v461: 97 pgs: 97 active+clean; 650 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 682 B/s wr, 12 op/s 2024-04-26T21:55:51.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:55:50 smithi076 ceph-mon[31823]: pgmap v461: 97 pgs: 97 active+clean; 650 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 682 B/s wr, 12 op/s 2024-04-26T21:55:51.321 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:55:50 smithi072 ceph-mon[28138]: pgmap v461: 97 pgs: 97 active+clean; 650 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 682 B/s wr, 12 op/s 2024-04-26T21:55:53.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:55:52 smithi164 ceph-mon[31916]: pgmap v462: 97 pgs: 97 active+clean; 650 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 597 B/s wr, 8 op/s 2024-04-26T21:55:53.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:55:52 smithi076 ceph-mon[31823]: pgmap v462: 97 pgs: 97 active+clean; 650 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 597 B/s wr, 8 op/s 2024-04-26T21:55:53.321 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:55:52 smithi072 ceph-mon[28138]: pgmap v462: 97 pgs: 97 active+clean; 650 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 597 B/s wr, 8 op/s 2024-04-26T21:55:55.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:55:54 smithi164 ceph-mon[31916]: pgmap v463: 97 pgs: 97 active+clean; 650 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 597 B/s wr, 9 op/s 2024-04-26T21:55:55.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:55:54 smithi076 ceph-mon[31823]: pgmap v463: 97 pgs: 97 active+clean; 650 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 597 B/s wr, 9 op/s 2024-04-26T21:55:55.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:55:54 smithi072 ceph-mon[28138]: pgmap v463: 97 pgs: 97 active+clean; 650 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 597 B/s wr, 9 op/s 2024-04-26T21:55:57.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:55:56 smithi164 ceph-mon[31916]: pgmap v464: 97 pgs: 97 active+clean; 650 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 597 B/s wr, 11 op/s 2024-04-26T21:55:57.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:55:56 smithi076 ceph-mon[31823]: pgmap v464: 97 pgs: 97 active+clean; 650 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 597 B/s wr, 11 op/s 2024-04-26T21:55:57.321 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:55:56 smithi072 ceph-mon[28138]: pgmap v464: 97 pgs: 97 active+clean; 650 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 597 B/s wr, 11 op/s 2024-04-26T21:55:59.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:55:58 smithi164 ceph-mon[31916]: pgmap v465: 97 pgs: 97 active+clean; 650 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:55:59.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:55:58 smithi076 ceph-mon[31823]: pgmap v465: 97 pgs: 97 active+clean; 650 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:55:59.321 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:55:58 smithi072 ceph-mon[28138]: pgmap v465: 97 pgs: 97 active+clean; 650 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:56:01.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:56:00 smithi164 ceph-mon[31916]: pgmap v466: 97 pgs: 97 active+clean; 650 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-26T21:56:01.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:56:00 smithi076 ceph-mon[31823]: pgmap v466: 97 pgs: 97 active+clean; 650 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-26T21:56:01.321 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:56:00 smithi072 ceph-mon[28138]: pgmap v466: 97 pgs: 97 active+clean; 650 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-26T21:56:01.566 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-26T21:56:01.566 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-26T21:56:01.784 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.784+0000 7f0a4c800640 1 -- 172.21.15.72:0/4068623777 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0a441045d0 msgr2=0x7f0a4410ae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:01.784 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.784+0000 7f0a4c800640 1 --2- 172.21.15.72:0/4068623777 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0a441045d0 0x7f0a4410ae60 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f0a40009cd0 tx=0x7f0a4002a8e0 comp rx=0 tx=0).stop 2024-04-26T21:56:01.784 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.784+0000 7f0a4c800640 1 -- 172.21.15.72:0/4068623777 shutdown_connections 2024-04-26T21:56:01.785 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.784+0000 7f0a4c800640 1 --2- 172.21.15.72:0/4068623777 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0a441045d0 0x7f0a4410ae60 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:01.785 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.784+0000 7f0a4c800640 1 --2- 172.21.15.72:0/4068623777 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0a44103c10 0x7f0a44104090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:01.785 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.784+0000 7f0a4c800640 1 --2- 172.21.15.72:0/4068623777 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0a44100c40 0x7f0a44101060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:01.785 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.784+0000 7f0a4c800640 1 -- 172.21.15.72:0/4068623777 >> 172.21.15.72:0/4068623777 conn(0x7f0a440fc570 msgr2=0x7f0a440fe9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:56:01.785 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.784+0000 7f0a4c800640 1 -- 172.21.15.72:0/4068623777 shutdown_connections 2024-04-26T21:56:01.785 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.784+0000 7f0a4c800640 1 -- 172.21.15.72:0/4068623777 wait complete. 2024-04-26T21:56:01.787 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.785+0000 7f0a4c800640 1 Processor -- start 2024-04-26T21:56:01.787 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.785+0000 7f0a4c800640 1 -- start start 2024-04-26T21:56:01.787 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.785+0000 7f0a4c800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0a44100c40 0x7f0a4419ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:01.788 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.785+0000 7f0a4c800640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0a44103c10 0x7f0a4419b290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:01.788 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.785+0000 7f0a4c800640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0a441045d0 0x7f0a441a22c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:01.788 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.786+0000 7f0a4c800640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f0a4410e3d0 con 0x7f0a441045d0 2024-04-26T21:56:01.788 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.786+0000 7f0a4c800640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f0a4410e250 con 0x7f0a44103c10 2024-04-26T21:56:01.788 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.786+0000 7f0a4c800640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f0a4410e550 con 0x7f0a44100c40 2024-04-26T21:56:01.788 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.786+0000 7f0a4a800640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0a441045d0 0x7f0a441a22c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:01.788 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.786+0000 7f0a4a800640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0a441045d0 0x7f0a441a22c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:37332/0 (socket says 172.21.15.72:37332) 2024-04-26T21:56:01.788 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.786+0000 7f0a4a800640 1 -- 172.21.15.72:0/1237762732 learned_addr learned my addr 172.21.15.72:0/1237762732 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:56:01.788 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.786+0000 7f0a49e00640 1 --2- 172.21.15.72:0/1237762732 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0a44100c40 0x7f0a4419ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:01.788 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.786+0000 7f0a49400640 1 --2- 172.21.15.72:0/1237762732 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0a44103c10 0x7f0a4419b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:01.788 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.786+0000 7f0a4a800640 1 -- 172.21.15.72:0/1237762732 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0a44100c40 msgr2=0x7f0a4419ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:01.788 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.786+0000 7f0a4a800640 1 --2- 172.21.15.72:0/1237762732 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0a44100c40 0x7f0a4419ad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:01.788 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.786+0000 7f0a4a800640 1 -- 172.21.15.72:0/1237762732 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0a44103c10 msgr2=0x7f0a4419b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:01.788 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.786+0000 7f0a4a800640 1 --2- 172.21.15.72:0/1237762732 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0a44103c10 0x7f0a4419b290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:01.789 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.786+0000 7f0a4a800640 1 -- 172.21.15.72:0/1237762732 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a441a2960 con 0x7f0a441045d0 2024-04-26T21:56:01.789 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.787+0000 7f0a4a800640 1 --2- 172.21.15.72:0/1237762732 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0a441045d0 0x7f0a441a22c0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f0a40003ba0 tx=0x7f0a40003bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:56:01.789 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.787+0000 7f0a3ea00640 1 -- 172.21.15.72:0/1237762732 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a40004240 con 0x7f0a441045d0 2024-04-26T21:56:01.789 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.787+0000 7f0a3ea00640 1 -- 172.21.15.72:0/1237762732 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0a400305a0 con 0x7f0a441045d0 2024-04-26T21:56:01.789 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.787+0000 7f0a3ea00640 1 -- 172.21.15.72:0/1237762732 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a400466e0 con 0x7f0a441045d0 2024-04-26T21:56:01.789 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.787+0000 7f0a4c800640 1 -- 172.21.15.72:0/1237762732 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a441a2bf0 con 0x7f0a441045d0 2024-04-26T21:56:01.789 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.787+0000 7f0a4c800640 1 -- 172.21.15.72:0/1237762732 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a441a3140 con 0x7f0a441045d0 2024-04-26T21:56:01.792 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.790+0000 7f0a4f65e740 1 Processor -- start 2024-04-26T21:56:01.793 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.790+0000 7f0a3ea00640 1 -- 172.21.15.72:0/1237762732 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f0a40030a50 con 0x7f0a441045d0 2024-04-26T21:56:01.793 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.791+0000 7f0a4f65e740 1 -- start start 2024-04-26T21:56:01.793 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.791+0000 7f0a4f65e740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f8c0dcb790 0x55f8c0dcbbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:01.793 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.791+0000 7f0a4f65e740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f8c0dcc180 0x55f8c0dd0c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:01.793 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.791+0000 7f0a4f65e740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f8c0dd12f0 0x55f8c0dd3780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:01.793 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.791+0000 7f0a4f65e740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55f8c0cdc030 con 0x55f8c0dd12f0 2024-04-26T21:56:01.793 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.791+0000 7f0a4f65e740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55f8c0cf77f0 con 0x55f8c0dcc180 2024-04-26T21:56:01.793 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.791+0000 7f0a4f65e740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55f8c0cb27e0 con 0x55f8c0dcb790 2024-04-26T21:56:01.793 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.791+0000 7f0a49400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f8c0dd12f0 0x55f8c0dd3780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:01.793 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.791+0000 7f0a49400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f8c0dd12f0 0x55f8c0dd3780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:37340/0 (socket says 172.21.15.72:37340) 2024-04-26T21:56:01.793 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.791+0000 7f0a49400640 1 -- 172.21.15.72:0/1685770313 learned_addr learned my addr 172.21.15.72:0/1685770313 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:56:01.793 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.792+0000 7f0a49e00640 1 --2- 172.21.15.72:0/1685770313 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f8c0dcc180 0x55f8c0dd0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:01.793 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.792+0000 7f0a49400640 1 --2- 172.21.15.72:0/1685770313 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f8c0dcb790 0x55f8c0dcbbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:01.794 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.792+0000 7f0a49400640 1 -- 172.21.15.72:0/1685770313 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f8c0dcb790 msgr2=0x55f8c0dcbbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:56:01.794 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.792+0000 7f0a49400640 1 --2- 172.21.15.72:0/1685770313 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f8c0dcb790 0x55f8c0dcbbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:01.794 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.792+0000 7f0a49400640 1 -- 172.21.15.72:0/1685770313 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f8c0dcc180 msgr2=0x55f8c0dd0c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:56:01.794 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.792+0000 7f0a49400640 1 --2- 172.21.15.72:0/1685770313 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f8c0dcc180 0x55f8c0dd0c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:01.794 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.792+0000 7f0a49400640 1 -- 172.21.15.72:0/1685770313 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f8c0cbe050 con 0x55f8c0dd12f0 2024-04-26T21:56:01.794 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.792+0000 7f0a3ea00640 1 --2- 172.21.15.72:0/1237762732 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f0a2c0a3c40 0x7f0a2c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:01.794 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.792+0000 7f0a49400640 1 --2- 172.21.15.72:0/1685770313 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f8c0dd12f0 0x55f8c0dd3780 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f0a34013580 tx=0x7f0a3400c6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20641a9b968de7f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:56:01.794 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.792+0000 7f0a3ea00640 1 -- 172.21.15.72:0/1237762732 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7f0a4002e030 con 0x7f0a441045d0 2024-04-26T21:56:01.794 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.792+0000 7f0a49e00640 1 --2- 172.21.15.72:0/1237762732 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f0a2c0a3c40 0x7f0a2c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:01.794 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.792+0000 7f0a33000640 1 -- 172.21.15.72:0/1685770313 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a34019030 con 0x55f8c0dd12f0 2024-04-26T21:56:01.794 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.793+0000 7f0a33000640 1 -- 172.21.15.72:0/1685770313 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0a340054b0 con 0x55f8c0dd12f0 2024-04-26T21:56:01.794 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.793+0000 7f0a33000640 1 -- 172.21.15.72:0/1685770313 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a34011520 con 0x55f8c0dd12f0 2024-04-26T21:56:01.799 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.793+0000 7f0a4f65e740 1 -- 172.21.15.72:0/1685770313 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f8c0dd12f0 msgr2=0x55f8c0dd3780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:01.799 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.793+0000 7f0a49e00640 1 --2- 172.21.15.72:0/1237762732 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f0a2c0a3c40 0x7f0a2c0a6100 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f0a38005fd0 tx=0x7f0a38005c50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:56:01.799 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.793+0000 7f0a4f65e740 1 --2- 172.21.15.72:0/1685770313 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f8c0dd12f0 0x55f8c0dd3780 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f0a34013580 tx=0x7f0a3400c6d0 comp rx=0 tx=0).stop 2024-04-26T21:56:01.799 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.793+0000 7f0a4f65e740 1 -- 172.21.15.72:0/1685770313 shutdown_connections 2024-04-26T21:56:01.799 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.793+0000 7f0a4f65e740 1 --2- 172.21.15.72:0/1685770313 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f8c0dd12f0 0x55f8c0dd3780 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:01.799 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.793+0000 7f0a4f65e740 1 --2- 172.21.15.72:0/1685770313 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f8c0dcc180 0x55f8c0dd0c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:01.799 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.793+0000 7f0a4f65e740 1 --2- 172.21.15.72:0/1685770313 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f8c0dcb790 0x55f8c0dcbbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:01.799 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.793+0000 7f0a4f65e740 1 -- 172.21.15.72:0/1685770313 >> 172.21.15.72:0/1685770313 conn(0x55f8c0dc6db0 msgr2=0x55f8c0dc71b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:56:01.800 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.793+0000 7f0a4f65e740 1 -- 172.21.15.72:0/1685770313 shutdown_connections 2024-04-26T21:56:01.800 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.794+0000 7f0a4f65e740 1 -- 172.21.15.72:0/1685770313 wait complete. 2024-04-26T21:56:01.800 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.794+0000 7f0a4f65e740 1 -- start start 2024-04-26T21:56:01.800 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.794+0000 7f0a4f65e740 1 Processor -- start 2024-04-26T21:56:01.800 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.795+0000 7f0a4f65e740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f8c0dcb790 0x55f8c0dd5680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:01.800 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.795+0000 7f0a4f65e740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f8c0dcc180 0x55f8c0dd5bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:01.800 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.795+0000 7f0a4f65e740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f8c0dd12f0 0x55f8c0ddcc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:01.800 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.795+0000 7f0a4f65e740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55f8c0d34200 con 0x55f8c0dcc180 2024-04-26T21:56:01.800 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.795+0000 7f0a4f65e740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55f8c0d2eee0 con 0x55f8c0dd12f0 2024-04-26T21:56:01.800 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.795+0000 7f0a4f65e740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55f8c0d3be70 con 0x55f8c0dcb790 2024-04-26T21:56:01.800 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.795+0000 7f0a4f65e740 10 client.0 _ll_register_callbacks cb 0x55f8c0d25270 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:56:01.800 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.795+0000 7f0a49400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f8c0dcc180 0x55f8c0dd5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:01.800 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.795+0000 7f0a49400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f8c0dcc180 0x55f8c0dd5bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:37342/0 (socket says 172.21.15.72:37342) 2024-04-26T21:56:01.801 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.795+0000 7f0a49400640 1 -- 172.21.15.72:0/2402156867 learned_addr learned my addr 172.21.15.72:0/2402156867 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:56:01.801 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.795+0000 7f0a49e00640 1 --2- 172.21.15.72:0/2402156867 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f8c0dcb790 0x55f8c0dd5680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:01.801 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.795+0000 7f0a4a800640 1 --2- 172.21.15.72:0/2402156867 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f8c0dd12f0 0x55f8c0ddcc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:01.801 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.795+0000 7f0a49400640 1 -- 172.21.15.72:0/2402156867 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f8c0dcb790 msgr2=0x55f8c0dd5680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:56:01.801 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.795+0000 7f0a49400640 1 --2- 172.21.15.72:0/2402156867 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f8c0dcb790 0x55f8c0dd5680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:01.801 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.795+0000 7f0a49400640 1 -- 172.21.15.72:0/2402156867 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f8c0dd12f0 msgr2=0x55f8c0ddcc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:56:01.801 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.795+0000 7f0a49400640 1 --2- 172.21.15.72:0/2402156867 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f8c0dd12f0 0x55f8c0ddcc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:01.801 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.795+0000 7f0a49400640 1 -- 172.21.15.72:0/2402156867 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f8c0b0b7d0 con 0x55f8c0dcc180 2024-04-26T21:56:01.801 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.796+0000 7f0a49400640 1 --2- 172.21.15.72:0/2402156867 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f8c0dcc180 0x55f8c0dd5bc0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f0a34004b10 tx=0x7f0a3400ce50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f3d9f4410517f98 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:56:01.801 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.796+0000 7f0a27400640 10 client.0 ms_handle_connect on v2:172.21.15.72:3300/0 2024-04-26T21:56:01.801 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.796+0000 7f0a27400640 1 -- 172.21.15.72:0/2402156867 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a34019040 con 0x55f8c0dcc180 2024-04-26T21:56:01.801 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.796+0000 7f0a4f65e740 1 -- 172.21.15.72:0/2402156867 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0a340054b0 con 0x55f8c0dcc180 2024-04-26T21:56:01.801 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.796+0000 7f0a27400640 1 -- 172.21.15.72:0/2402156867 <== mon.0 v2:172.21.15.72:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7f0a34011520 con 0x55f8c0dcc180 2024-04-26T21:56:01.802 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.796+0000 7f0a27400640 1 -- 172.21.15.72:0/2402156867 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0a3402c030 con 0x55f8c0dcc180 2024-04-26T21:56:01.802 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.796+0000 7f0a4f65e740 10 client.15537 fetch_fsmap learned FSMap version 27 2024-04-26T21:56:01.802 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.796+0000 7f0a4f65e740 1 -- 172.21.15.72:0/2402156867 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f8c0a413f0 con 0x55f8c0dcc180 2024-04-26T21:56:01.802 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.797+0000 7f0a27400640 1 -- 172.21.15.72:0/2402156867 <== mon.0 v2:172.21.15.72:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0a340054b0 con 0x55f8c0dcc180 2024-04-26T21:56:01.802 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.797+0000 7f0a27400640 1 -- 172.21.15.72:0/2402156867 <== mon.0 v2:172.21.15.72:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f0a3401e660 con 0x55f8c0dcc180 2024-04-26T21:56:01.802 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.797+0000 7f0a4f65e740 10 client.15537 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T21:56:01.802 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.797+0000 7f0a4f65e740 10 client.15537 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:56:01.802 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.797+0000 7f0a4f65e740 20 client.15537 populate_metadata read hostname 'smithi072' 2024-04-26T21:56:01.802 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.797+0000 7f0a4f65e740 1 --2- 172.21.15.72:0/2402156867 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x55f8c0de0ee0 0x55f8c0de32f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:01.802 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.797+0000 7f0a4f65e740 4 client.15537 mds_command: new command op to 34517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:56:01.802 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.797+0000 7f0a4f65e740 1 -- 172.21.15.72:0/2402156867 --> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f8c0d3d7e0 con 0x55f8c0de0ee0 2024-04-26T21:56:01.802 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.798+0000 7f0a49e00640 1 --2- 172.21.15.72:0/2402156867 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x55f8c0de0ee0 0x55f8c0de32f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:01.802 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.799+0000 7f0a49e00640 1 --2- 172.21.15.72:0/2402156867 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x55f8c0de0ee0 0x55f8c0de32f0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f7a8fa658efe2203 server_cookie=2e8fbb8582823826 in_seq=0 out_seq=0 2024-04-26T21:56:01.803 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.799+0000 7f0a27400640 10 client.15537 ms_handle_connect on v2:172.21.15.76:6838/327145636 2024-04-26T21:56:01.804 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.800+0000 7f0a27400640 1 -- 172.21.15.72:0/2402156867 <== mds.0 v2:172.21.15.76:6838/327145636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f8c0d3d7e0 con 0x55f8c0de0ee0 2024-04-26T21:56:01.804 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.800+0000 7f0a27400640 10 client.15537 handle_command_reply: tid=0 2024-04-26T21:56:01.804 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.800+0000 7f0a4f65e740 1 client.15537 shutdown 2024-04-26T21:56:01.804 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.801+0000 7f0a4f65e740 1 -- 172.21.15.72:0/2402156867 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f8c0dcc180 msgr2=0x55f8c0dd5bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:01.804 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.801+0000 7f0a4f65e740 1 --2- 172.21.15.72:0/2402156867 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f8c0dcc180 0x55f8c0dd5bc0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f0a34004b10 tx=0x7f0a3400ce50 comp rx=0 tx=0).stop 2024-04-26T21:56:01.804 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.801+0000 7f0a4f65e740 1 -- 172.21.15.72:0/2402156867 shutdown_connections 2024-04-26T21:56:01.804 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.801+0000 7f0a4f65e740 1 --2- 172.21.15.72:0/2402156867 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x55f8c0de0ee0 0x55f8c0de32f0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:01.804 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.801+0000 7f0a4f65e740 1 --2- 172.21.15.72:0/2402156867 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f8c0dd12f0 0x55f8c0ddcc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:01.804 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.801+0000 7f0a4f65e740 1 --2- 172.21.15.72:0/2402156867 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f8c0dcc180 0x55f8c0dd5bc0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:01.804 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.801+0000 7f0a4f65e740 1 --2- 172.21.15.72:0/2402156867 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f8c0dcb790 0x55f8c0dd5680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:01.804 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.801+0000 7f0a27400640 0 client.15537 ms_handle_reset on v2:172.21.15.76:6838/327145636 2024-04-26T21:56:01.804 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.801+0000 7f0a4f65e740 1 -- 172.21.15.72:0/2402156867 >> 172.21.15.72:0/2402156867 conn(0x55f8c0dc6db0 msgr2=0x55f8c0dc6040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:56:01.805 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.801+0000 7f0a4f65e740 1 -- 172.21.15.72:0/2402156867 shutdown_connections 2024-04-26T21:56:01.805 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.801+0000 7f0a4f65e740 1 -- 172.21.15.72:0/2402156867 wait complete. 2024-04-26T21:56:01.805 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.801+0000 7f0a4f65e740 20 client.15537 trim_cache size 0 max 16384 2024-04-26T21:56:01.820 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.820+0000 7f0a4f65e740 1 Processor -- start 2024-04-26T21:56:01.820 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.820+0000 7f0a4f65e740 1 -- start start 2024-04-26T21:56:01.820 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.820+0000 7f0a4f65e740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f8c0dcb790 0x55f8c0dd5680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:01.820 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.820+0000 7f0a4f65e740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f8c0dcc180 0x55f8c0ddcbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:01.820 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.820+0000 7f0a4f65e740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f8c0dd12f0 0x55f8c0dcd330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:01.820 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.820+0000 7f0a4f65e740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f0a441adba0 con 0x55f8c0dd12f0 2024-04-26T21:56:01.821 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.820+0000 7f0a4f65e740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f0a441ac730 con 0x55f8c0dcc180 2024-04-26T21:56:01.821 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.820+0000 7f0a4f65e740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f0a441ace00 con 0x55f8c0dcb790 2024-04-26T21:56:01.822 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.820+0000 7f0a4a800640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f8c0dd12f0 0x55f8c0dcd330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:01.822 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.821+0000 7f0a4a800640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f8c0dd12f0 0x55f8c0dcd330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:37350/0 (socket says 172.21.15.72:37350) 2024-04-26T21:56:01.822 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.821+0000 7f0a4a800640 1 -- 172.21.15.72:0/2450038406 learned_addr learned my addr 172.21.15.72:0/2450038406 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:56:01.822 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.821+0000 7f0a4a800640 1 -- 172.21.15.72:0/2450038406 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f8c0dcb790 msgr2=0x55f8c0dd5680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:56:01.823 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.821+0000 7f0a4a800640 1 --2- 172.21.15.72:0/2450038406 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f8c0dcb790 0x55f8c0dd5680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:01.823 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.821+0000 7f0a4a800640 1 -- 172.21.15.72:0/2450038406 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f8c0dcc180 msgr2=0x55f8c0ddcbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:56:01.823 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.821+0000 7f0a4a800640 1 --2- 172.21.15.72:0/2450038406 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f8c0dcc180 0x55f8c0ddcbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:01.823 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.821+0000 7f0a4a800640 1 -- 172.21.15.72:0/2450038406 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f8c0a010f0 con 0x55f8c0dd12f0 2024-04-26T21:56:01.823 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.821+0000 7f0a4a800640 1 --2- 172.21.15.72:0/2450038406 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f8c0dd12f0 0x55f8c0dcd330 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f0a400bd860 tx=0x7f0a400bd6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdaf33709bfad37d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:56:01.823 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.821+0000 7f0a32600640 1 -- 172.21.15.72:0/2450038406 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a400be260 con 0x55f8c0dd12f0 2024-04-26T21:56:01.823 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.821+0000 7f0a32600640 1 -- 172.21.15.72:0/2450038406 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0a40036020 con 0x55f8c0dd12f0 2024-04-26T21:56:01.823 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.821+0000 7f0a32600640 1 -- 172.21.15.72:0/2450038406 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a400de0c0 con 0x55f8c0dd12f0 2024-04-26T21:56:01.823 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.822+0000 7f0a4f65e740 1 -- 172.21.15.72:0/2450038406 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f8c0dd12f0 msgr2=0x55f8c0dcd330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:01.823 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.822+0000 7f0a4f65e740 1 --2- 172.21.15.72:0/2450038406 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f8c0dd12f0 0x55f8c0dcd330 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f0a400bd860 tx=0x7f0a400bd6a0 comp rx=0 tx=0).stop 2024-04-26T21:56:01.823 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.822+0000 7f0a4f65e740 1 -- 172.21.15.72:0/2450038406 shutdown_connections 2024-04-26T21:56:01.823 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.822+0000 7f0a4f65e740 1 --2- 172.21.15.72:0/2450038406 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f8c0dd12f0 0x55f8c0dcd330 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:01.823 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.822+0000 7f0a4f65e740 1 --2- 172.21.15.72:0/2450038406 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f8c0dcc180 0x55f8c0ddcbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:01.823 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.822+0000 7f0a4f65e740 1 --2- 172.21.15.72:0/2450038406 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f8c0dcb790 0x55f8c0dd5680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:01.824 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.822+0000 7f0a4f65e740 1 -- 172.21.15.72:0/2450038406 >> 172.21.15.72:0/2450038406 conn(0x55f8c0dc6db0 msgr2=0x55f8c0dccc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:56:01.824 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.822+0000 7f0a4f65e740 1 -- 172.21.15.72:0/2450038406 shutdown_connections 2024-04-26T21:56:01.824 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.822+0000 7f0a4f65e740 1 -- 172.21.15.72:0/2450038406 wait complete. 2024-04-26T21:56:01.824 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.823+0000 7f0a4f65e740 1 -- start start 2024-04-26T21:56:01.828 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.823+0000 7f0a4f65e740 1 Processor -- start 2024-04-26T21:56:01.828 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.823+0000 7f0a4f65e740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f8c0dcb790 0x55f8c0dda530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:01.828 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.823+0000 7f0a4f65e740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f8c0dcc180 0x55f8c0ddaa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:01.828 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.823+0000 7f0a4f65e740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f8c0dd12f0 0x55f8c0ddba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:01.828 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.823+0000 7f0a4f65e740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f0a441aa040 con 0x55f8c0dcb790 2024-04-26T21:56:01.828 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.823+0000 7f0a4f65e740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f0a441ad4d0 con 0x55f8c0dd12f0 2024-04-26T21:56:01.828 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.823+0000 7f0a4f65e740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f0a441aa6b0 con 0x55f8c0dcc180 2024-04-26T21:56:01.828 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.823+0000 7f0a4f65e740 10 client.0 _ll_register_callbacks cb 0x7f0a34013460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:56:01.828 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.823+0000 7f0a49e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f8c0dcb790 0x55f8c0dda530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:01.828 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.824+0000 7f0a49e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f8c0dcb790 0x55f8c0dda530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:37356/0 (socket says 172.21.15.72:37356) 2024-04-26T21:56:01.828 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.824+0000 7f0a49e00640 1 -- 172.21.15.72:0/3483655920 learned_addr learned my addr 172.21.15.72:0/3483655920 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:56:01.829 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.824+0000 7f0a49e00640 1 -- 172.21.15.72:0/3483655920 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f8c0dcc180 msgr2=0x55f8c0ddaa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:56:01.829 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.824+0000 7f0a49400640 1 --2- 172.21.15.72:0/3483655920 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f8c0dcc180 0x55f8c0ddaa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:01.829 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.824+0000 7f0a4a800640 1 --2- 172.21.15.72:0/3483655920 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f8c0dd12f0 0x55f8c0ddba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:01.829 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.824+0000 7f0a49e00640 1 --2- 172.21.15.72:0/3483655920 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f8c0dcc180 0x55f8c0ddaa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:01.829 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.824+0000 7f0a49e00640 1 -- 172.21.15.72:0/3483655920 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f8c0dd12f0 msgr2=0x55f8c0ddba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:56:01.829 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.824+0000 7f0a49e00640 1 --2- 172.21.15.72:0/3483655920 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f8c0dd12f0 0x55f8c0ddba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:01.829 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.824+0000 7f0a49e00640 1 -- 172.21.15.72:0/3483655920 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f8c0cf7a30 con 0x55f8c0dcb790 2024-04-26T21:56:01.829 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.824+0000 7f0a49400640 1 --2- 172.21.15.72:0/3483655920 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f8c0dcc180 0x55f8c0ddaa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:56:01.829 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.824+0000 7f0a4a800640 1 --2- 172.21.15.72:0/3483655920 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f8c0dd12f0 0x55f8c0ddba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:56:01.829 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.824+0000 7f0a49e00640 1 --2- 172.21.15.72:0/3483655920 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f8c0dcb790 0x55f8c0dda530 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f0a4419c220 tx=0x7f0a38004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9fefb931e74b3a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:56:01.829 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.824+0000 7f0a27400640 10 client.0 ms_handle_connect on v2:172.21.15.72:3300/0 2024-04-26T21:56:01.829 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.825+0000 7f0a27400640 1 -- 172.21.15.72:0/3483655920 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a3802c070 con 0x55f8c0dcb790 2024-04-26T21:56:01.829 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.825+0000 7f0a27400640 1 -- 172.21.15.72:0/3483655920 <== mon.0 v2:172.21.15.72:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7f0a38028330 con 0x55f8c0dcb790 2024-04-26T21:56:01.829 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.825+0000 7f0a4f65e740 1 -- 172.21.15.72:0/3483655920 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0a40036020 con 0x55f8c0dcb790 2024-04-26T21:56:01.830 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.825+0000 7f0a27400640 1 -- 172.21.15.72:0/3483655920 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0a38037050 con 0x55f8c0dcb790 2024-04-26T21:56:01.830 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.825+0000 7f0a4f65e740 10 client.15543 fetch_fsmap learned FSMap version 27 2024-04-26T21:56:01.830 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.825+0000 7f0a4f65e740 1 -- 172.21.15.72:0/3483655920 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f8c0e47500 con 0x55f8c0dcb790 2024-04-26T21:56:01.830 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.825+0000 7f0a27400640 1 -- 172.21.15.72:0/3483655920 <== mon.0 v2:172.21.15.72:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0a40036020 con 0x55f8c0dcb790 2024-04-26T21:56:01.830 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.825+0000 7f0a27400640 1 -- 172.21.15.72:0/3483655920 <== mon.0 v2:172.21.15.72:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f0a3801f9d0 con 0x55f8c0dcb790 2024-04-26T21:56:01.830 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.825+0000 7f0a4f65e740 10 client.15543 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T21:56:01.830 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.825+0000 7f0a4f65e740 10 client.15543 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:56:01.830 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.825+0000 7f0a4f65e740 20 client.15543 populate_metadata read hostname 'smithi072' 2024-04-26T21:56:01.830 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.825+0000 7f0a4f65e740 1 --2- 172.21.15.72:0/3483655920 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x55f8c0e47f10 0x55f8c0e4a330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:01.830 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.825+0000 7f0a4f65e740 4 client.15543 mds_command: new command op to 34517 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-26T21:56:01.830 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.825+0000 7f0a4f65e740 1 -- 172.21.15.72:0/3483655920 --> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f8c0d37650 con 0x55f8c0e47f10 2024-04-26T21:56:01.830 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.826+0000 7f0a49400640 1 --2- 172.21.15.72:0/3483655920 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x55f8c0e47f10 0x55f8c0e4a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:01.830 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.827+0000 7f0a49400640 1 --2- 172.21.15.72:0/3483655920 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x55f8c0e47f10 0x55f8c0e4a330 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1362820c03061101 server_cookie=6a28700261035d5c in_seq=0 out_seq=0 2024-04-26T21:56:01.830 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.827+0000 7f0a27400640 10 client.15543 ms_handle_connect on v2:172.21.15.76:6838/327145636 2024-04-26T21:56:01.831 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.827+0000 7f0a27400640 1 -- 172.21.15.72:0/3483655920 <== mds.0 v2:172.21.15.76:6838/327145636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55f8c0d37650 con 0x55f8c0e47f10 2024-04-26T21:56:01.831 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.827+0000 7f0a27400640 10 client.15543 handle_command_reply: tid=0 2024-04-26T21:56:01.831 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.827+0000 7f0a4f65e740 1 client.15543 shutdown 2024-04-26T21:56:01.831 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.827+0000 7f0a4f65e740 1 -- 172.21.15.72:0/3483655920 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f8c0dcb790 msgr2=0x55f8c0dda530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:01.831 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.827+0000 7f0a4f65e740 1 --2- 172.21.15.72:0/3483655920 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f8c0dcb790 0x55f8c0dda530 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f0a4419c220 tx=0x7f0a38004730 comp rx=0 tx=0).stop 2024-04-26T21:56:01.831 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.828+0000 7f0a4f65e740 1 -- 172.21.15.72:0/3483655920 shutdown_connections 2024-04-26T21:56:01.831 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.828+0000 7f0a4f65e740 1 --2- 172.21.15.72:0/3483655920 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x55f8c0e47f10 0x55f8c0e4a330 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:01.831 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.828+0000 7f0a4f65e740 1 --2- 172.21.15.72:0/3483655920 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f8c0dd12f0 0x55f8c0ddba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:01.831 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.828+0000 7f0a4f65e740 1 --2- 172.21.15.72:0/3483655920 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f8c0dcc180 0x55f8c0ddaa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:01.831 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.828+0000 7f0a27400640 0 client.15543 ms_handle_reset on v2:172.21.15.76:6838/327145636 2024-04-26T21:56:01.831 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.828+0000 7f0a4f65e740 1 --2- 172.21.15.72:0/3483655920 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f8c0dcb790 0x55f8c0dda530 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:01.831 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.828+0000 7f0a4f65e740 1 -- 172.21.15.72:0/3483655920 >> 172.21.15.72:0/3483655920 conn(0x55f8c0dc6db0 msgr2=0x55f8c0dcd4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:56:01.831 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.828+0000 7f0a4f65e740 1 -- 172.21.15.72:0/3483655920 shutdown_connections 2024-04-26T21:56:01.831 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.828+0000 7f0a4f65e740 1 -- 172.21.15.72:0/3483655920 wait complete. 2024-04-26T21:56:01.832 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.828+0000 7f0a4f65e740 20 client.15543 trim_cache size 0 max 16384 2024-04-26T21:56:01.832 INFO:teuthology.orchestra.run.smithi072.stdout:{ 2024-04-26T21:56:01.832 INFO:teuthology.orchestra.run.smithi072.stdout: "status": "no active scrubs running", 2024-04-26T21:56:01.832 INFO:teuthology.orchestra.run.smithi072.stdout: "scrubs": {} 2024-04-26T21:56:01.832 INFO:teuthology.orchestra.run.smithi072.stdout:} 2024-04-26T21:56:01.833 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.828+0000 7f0a4c800640 1 -- 172.21.15.72:0/1237762732 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f0a2c0a3c40 msgr2=0x7f0a2c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:01.833 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.828+0000 7f0a4c800640 1 --2- 172.21.15.72:0/1237762732 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f0a2c0a3c40 0x7f0a2c0a6100 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f0a38005fd0 tx=0x7f0a38005c50 comp rx=0 tx=0).stop 2024-04-26T21:56:01.833 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.828+0000 7f0a4c800640 1 -- 172.21.15.72:0/1237762732 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0a441045d0 msgr2=0x7f0a441a22c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:01.833 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.828+0000 7f0a4c800640 1 --2- 172.21.15.72:0/1237762732 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0a441045d0 0x7f0a441a22c0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f0a40003ba0 tx=0x7f0a40003bd0 comp rx=0 tx=0).stop 2024-04-26T21:56:01.833 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.829+0000 7f0a4c800640 1 -- 172.21.15.72:0/1237762732 shutdown_connections 2024-04-26T21:56:01.833 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.829+0000 7f0a4c800640 1 --2- 172.21.15.72:0/1237762732 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0a441045d0 0x7f0a441a22c0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:01.833 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.829+0000 7f0a4c800640 1 --2- 172.21.15.72:0/1237762732 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0a44103c10 0x7f0a4419b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:01.833 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.829+0000 7f0a4c800640 1 --2- 172.21.15.72:0/1237762732 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f0a2c0a3c40 0x7f0a2c0a6100 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:01.833 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.829+0000 7f0a4c800640 1 --2- 172.21.15.72:0/1237762732 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0a44100c40 0x7f0a4419ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:01.833 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.829+0000 7f0a4c800640 1 -- 172.21.15.72:0/1237762732 >> 172.21.15.72:0/1237762732 conn(0x7f0a440fc570 msgr2=0x7f0a44101a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:56:01.833 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.829+0000 7f0a4c800640 1 -- 172.21.15.72:0/1237762732 shutdown_connections 2024-04-26T21:56:01.834 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:01.829+0000 7f0a4c800640 1 -- 172.21.15.72:0/1237762732 wait complete. 2024-04-26T21:56:01.848 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-26T21:56:01.848 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:56:02.053 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.052+0000 7fab79600640 1 -- 172.21.15.72:0/3492707115 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fab741027b0 msgr2=0x7fab74102bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:02.053 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.052+0000 7fab79600640 1 --2- 172.21.15.72:0/3492707115 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fab741027b0 0x7fab74102bd0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fab64009900 tx=0x7fab6402a410 comp rx=0 tx=0).stop 2024-04-26T21:56:02.053 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.052+0000 7fab79600640 1 -- 172.21.15.72:0/3492707115 shutdown_connections 2024-04-26T21:56:02.053 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.052+0000 7fab79600640 1 --2- 172.21.15.72:0/3492707115 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fab74104360 0x7fab7410abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:02.053 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.052+0000 7fab79600640 1 --2- 172.21.15.72:0/3492707115 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fab741039a0 0x7fab74103e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:02.053 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.052+0000 7fab79600640 1 --2- 172.21.15.72:0/3492707115 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fab741027b0 0x7fab74102bd0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:02.053 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.052+0000 7fab79600640 1 -- 172.21.15.72:0/3492707115 >> 172.21.15.72:0/3492707115 conn(0x7fab740fdd40 msgr2=0x7fab74100160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:56:02.053 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.052+0000 7fab79600640 1 -- 172.21.15.72:0/3492707115 shutdown_connections 2024-04-26T21:56:02.053 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.052+0000 7fab79600640 1 -- 172.21.15.72:0/3492707115 wait complete. 2024-04-26T21:56:02.056 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.054+0000 7fab79600640 1 Processor -- start 2024-04-26T21:56:02.056 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.054+0000 7fab79600640 1 -- start start 2024-04-26T21:56:02.056 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.054+0000 7fab79600640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fab741027b0 0x7fab7419ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:02.056 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.054+0000 7fab79600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fab741039a0 0x7fab7419b060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:02.056 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.054+0000 7fab79600640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fab74104360 0x7fab741a20d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:02.056 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.054+0000 7fab79600640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fab7410e1d0 con 0x7fab74104360 2024-04-26T21:56:02.056 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.054+0000 7fab79600640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fab7410e050 con 0x7fab741027b0 2024-04-26T21:56:02.056 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.054+0000 7fab79600640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fab7410e350 con 0x7fab741039a0 2024-04-26T21:56:02.056 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.054+0000 7fab78c00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fab74104360 0x7fab741a20d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:02.056 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.055+0000 7fab78c00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fab74104360 0x7fab741a20d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:37372/0 (socket says 172.21.15.72:37372) 2024-04-26T21:56:02.056 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.055+0000 7fab78c00640 1 -- 172.21.15.72:0/824602508 learned_addr learned my addr 172.21.15.72:0/824602508 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:56:02.056 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.055+0000 7fab73e00640 1 --2- 172.21.15.72:0/824602508 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fab741027b0 0x7fab7419ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:02.057 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.055+0000 7fab6be00640 1 --2- 172.21.15.72:0/824602508 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fab741039a0 0x7fab7419b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:02.057 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.055+0000 7fab78c00640 1 -- 172.21.15.72:0/824602508 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fab741039a0 msgr2=0x7fab7419b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:02.057 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.055+0000 7fab78c00640 1 --2- 172.21.15.72:0/824602508 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fab741039a0 0x7fab7419b060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:02.057 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.055+0000 7fab78c00640 1 -- 172.21.15.72:0/824602508 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fab741027b0 msgr2=0x7fab7419ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:02.057 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.055+0000 7fab78c00640 1 --2- 172.21.15.72:0/824602508 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fab741027b0 0x7fab7419ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:02.057 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.055+0000 7fab78c00640 1 -- 172.21.15.72:0/824602508 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab741a2740 con 0x7fab74104360 2024-04-26T21:56:02.057 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.055+0000 7fab78c00640 1 --2- 172.21.15.72:0/824602508 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fab74104360 0x7fab741a20d0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fab6c00eaa0 tx=0x7fab6c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:56:02.057 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.055+0000 7fab71600640 1 -- 172.21.15.72:0/824602508 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab6c00cdb0 con 0x7fab74104360 2024-04-26T21:56:02.057 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.056+0000 7fab71600640 1 -- 172.21.15.72:0/824602508 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fab6c005020 con 0x7fab74104360 2024-04-26T21:56:02.057 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.056+0000 7fab79600640 1 -- 172.21.15.72:0/824602508 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab741a2a30 con 0x7fab74104360 2024-04-26T21:56:02.057 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.056+0000 7fab71600640 1 -- 172.21.15.72:0/824602508 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab6c0116d0 con 0x7fab74104360 2024-04-26T21:56:02.057 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.056+0000 7fab79600640 1 -- 172.21.15.72:0/824602508 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab741a2f30 con 0x7fab74104360 2024-04-26T21:56:02.058 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.057+0000 7fab79600640 1 -- 172.21.15.72:0/824602508 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab741a3160 con 0x7fab74104360 2024-04-26T21:56:02.059 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.059+0000 7fab71600640 1 -- 172.21.15.72:0/824602508 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fab6c0051d0 con 0x7fab74104360 2024-04-26T21:56:02.061 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.061+0000 7fab71600640 1 --2- 172.21.15.72:0/824602508 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fab580a3c40 0x7fab580a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:02.061 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.062+0000 7fab71600640 1 -- 172.21.15.72:0/824602508 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7fab6c0d0920 con 0x7fab74104360 2024-04-26T21:56:02.062 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.062+0000 7fab73e00640 1 --2- 172.21.15.72:0/824602508 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fab580a3c40 0x7fab580a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:02.062 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.062+0000 7fab73e00640 1 --2- 172.21.15.72:0/824602508 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fab580a3c40 0x7fab580a6100 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fab640057c0 tx=0x7fab64000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:56:02.066 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.066+0000 7fab71600640 1 -- 172.21.15.72:0/824602508 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fab6c016030 con 0x7fab74104360 2024-04-26T21:56:02.317 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.317+0000 7fab79600640 1 -- 172.21.15.72:0/824602508 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fab741a33d0 con 0x7fab74104360 2024-04-26T21:56:02.318 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.318+0000 7fab71600640 1 -- 172.21.15.72:0/824602508 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11931 (secure 0 0 0) 0x7fab6c09d0c0 con 0x7fab74104360 2024-04-26T21:56:02.319 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T21:56:02.319 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6835/1320381959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6834","nonce":1320381959},{"type":"v1","addr":"172.21.15.72:6835","nonce":1320381959}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6837/2142200512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6836","nonce":2142200512},{"type":"v1","addr":"172.21.15.72:6837","nonce":2142200512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T21:52:50.567857+0000","modified":"2024-04-26T21:55:22.894383+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34517,"mds_1":34472,"mds_2":34442,"mds_3":34406,"mds_4":24463},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14652":{"gid":14652,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.72:6839/1312898281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6838","nonce":1312898281},{"type":"v1","addr":"172.21.15.72:6839","nonce":1312898281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14664":{"gid":14664,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.72:6841/3257692339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6840","nonce":3257692339},{"type":"v1","addr":"172.21.15.72:6841","nonce":3257692339}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24388":{"gid":24388,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6833/4236668465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":4236668465},{"type":"v1","addr":"172.21.15.164:6833","nonce":4236668465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6835/2391114169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":2391114169},{"type":"v1","addr":"172.21.15.164:6835","nonce":2391114169}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6837/3585829992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":3585829992},{"type":"v1","addr":"172.21.15.164:6837","nonce":3585829992}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.164:6839/1636226820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":1636226820},{"type":"v1","addr":"172.21.15.164:6839","nonce":1636226820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34406":{"gid":34406,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":15,"addr":"172.21.15.76:6833/3869483750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":3869483750},{"type":"v1","addr":"172.21.15.76:6833","nonce":3869483750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34442":{"gid":34442,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.76:6835/3517429435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":3517429435},{"type":"v1","addr":"172.21.15.76:6835","nonce":3517429435}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34472":{"gid":34472,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.76:6837/1150678991","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":1150678991},{"type":"v1","addr":"172.21.15.76:6837","nonce":1150678991}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34517":{"gid":34517,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.76:6839/327145636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6838","nonce":327145636},{"type":"v1","addr":"172.21.15.76:6839","nonce":327145636}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34517,"qdb_cluster":[34472,34517,34442,34406,24463]},"id":1}]} 2024-04-26T21:56:02.319 INFO:teuthology.orchestra.run.smithi072.stderr:dumped fsmap epoch 27 2024-04-26T21:56:02.322 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.321+0000 7fab79600640 1 -- 172.21.15.72:0/824602508 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fab580a3c40 msgr2=0x7fab580a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:02.322 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.321+0000 7fab79600640 1 --2- 172.21.15.72:0/824602508 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fab580a3c40 0x7fab580a6100 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fab640057c0 tx=0x7fab64000bc0 comp rx=0 tx=0).stop 2024-04-26T21:56:02.322 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.321+0000 7fab79600640 1 -- 172.21.15.72:0/824602508 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fab74104360 msgr2=0x7fab741a20d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:02.322 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.321+0000 7fab79600640 1 --2- 172.21.15.72:0/824602508 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fab74104360 0x7fab741a20d0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fab6c00eaa0 tx=0x7fab6c00ef60 comp rx=0 tx=0).stop 2024-04-26T21:56:02.322 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.322+0000 7fab79600640 1 -- 172.21.15.72:0/824602508 shutdown_connections 2024-04-26T21:56:02.322 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.322+0000 7fab79600640 1 --2- 172.21.15.72:0/824602508 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fab74104360 0x7fab741a20d0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:02.322 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.322+0000 7fab79600640 1 --2- 172.21.15.72:0/824602508 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fab580a3c40 0x7fab580a6100 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:02.322 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.322+0000 7fab79600640 1 --2- 172.21.15.72:0/824602508 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fab741039a0 0x7fab7419b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:02.322 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.322+0000 7fab79600640 1 --2- 172.21.15.72:0/824602508 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fab741027b0 0x7fab7419ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:02.323 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.322+0000 7fab79600640 1 -- 172.21.15.72:0/824602508 >> 172.21.15.72:0/824602508 conn(0x7fab740fdd40 msgr2=0x7fab74074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:56:02.323 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.322+0000 7fab79600640 1 -- 172.21.15.72:0/824602508 shutdown_connections 2024-04-26T21:56:02.323 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.322+0000 7fab79600640 1 -- 172.21.15.72:0/824602508 wait complete. 2024-04-26T21:56:02.341 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-26T21:56:02.546 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.546+0000 7f5f3c200640 1 -- 172.21.15.72:0/2461022288 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5f34105d60 msgr2=0x7f5f341061e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:02.547 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.546+0000 7f5f3c200640 1 --2- 172.21.15.72:0/2461022288 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5f34105d60 0x7f5f341061e0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f5f20009900 tx=0x7f5f2002a3f0 comp rx=0 tx=0).stop 2024-04-26T21:56:02.547 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.546+0000 7f5f3c200640 1 -- 172.21.15.72:0/2461022288 shutdown_connections 2024-04-26T21:56:02.547 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.546+0000 7f5f3c200640 1 --2- 172.21.15.72:0/2461022288 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5f34106720 0x7f5f3410cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:02.547 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.546+0000 7f5f3c200640 1 --2- 172.21.15.72:0/2461022288 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5f34105d60 0x7f5f341061e0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:02.547 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.546+0000 7f5f3c200640 1 --2- 172.21.15.72:0/2461022288 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5f34104b70 0x7f5f34104f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:02.547 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.546+0000 7f5f3c200640 1 -- 172.21.15.72:0/2461022288 >> 172.21.15.72:0/2461022288 conn(0x7f5f341000e0 msgr2=0x7f5f34102520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:56:02.547 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.546+0000 7f5f3c200640 1 -- 172.21.15.72:0/2461022288 shutdown_connections 2024-04-26T21:56:02.547 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.546+0000 7f5f3c200640 1 -- 172.21.15.72:0/2461022288 wait complete. 2024-04-26T21:56:02.549 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.548+0000 7f5f3c200640 1 Processor -- start 2024-04-26T21:56:02.549 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.548+0000 7f5f3c200640 1 -- start start 2024-04-26T21:56:02.549 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.548+0000 7f5f3c200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5f34104b70 0x7f5f3419ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:02.549 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.548+0000 7f5f3c200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5f34105d60 0x7f5f3419b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:02.549 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.548+0000 7f5f3c200640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5f34106720 0x7f5f341a2220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:02.549 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.548+0000 7f5f3c200640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f5f34110580 con 0x7f5f34106720 2024-04-26T21:56:02.549 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.548+0000 7f5f3c200640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f5f34110400 con 0x7f5f34105d60 2024-04-26T21:56:02.549 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.548+0000 7f5f3c200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f5f34110700 con 0x7f5f34104b70 2024-04-26T21:56:02.550 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.548+0000 7f5f3a200640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5f34106720 0x7f5f341a2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:02.550 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.549+0000 7f5f3a200640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5f34106720 0x7f5f341a2220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:37392/0 (socket says 172.21.15.72:37392) 2024-04-26T21:56:02.550 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.549+0000 7f5f3a200640 1 -- 172.21.15.72:0/638542738 learned_addr learned my addr 172.21.15.72:0/638542738 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:56:02.550 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.549+0000 7f5f38e00640 1 --2- 172.21.15.72:0/638542738 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5f34105d60 0x7f5f3419b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:02.550 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.549+0000 7f5f39800640 1 --2- 172.21.15.72:0/638542738 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5f34104b70 0x7f5f3419ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:02.550 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.549+0000 7f5f3a200640 1 -- 172.21.15.72:0/638542738 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5f34104b70 msgr2=0x7f5f3419ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:02.550 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.549+0000 7f5f3a200640 1 --2- 172.21.15.72:0/638542738 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5f34104b70 0x7f5f3419ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:02.550 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.549+0000 7f5f3a200640 1 -- 172.21.15.72:0/638542738 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5f34105d60 msgr2=0x7f5f3419b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:02.550 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.549+0000 7f5f3a200640 1 --2- 172.21.15.72:0/638542738 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5f34105d60 0x7f5f3419b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:02.550 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.549+0000 7f5f3a200640 1 -- 172.21.15.72:0/638542738 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f341a28c0 con 0x7f5f34106720 2024-04-26T21:56:02.550 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.549+0000 7f5f38e00640 1 --2- 172.21.15.72:0/638542738 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5f34105d60 0x7f5f3419b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:56:02.551 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.549+0000 7f5f3a200640 1 --2- 172.21.15.72:0/638542738 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5f34106720 0x7f5f341a2220 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f5f2c00cce0 tx=0x7f5f2c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:56:02.551 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.550+0000 7f5f32000640 1 -- 172.21.15.72:0/638542738 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f2c00f450 con 0x7f5f34106720 2024-04-26T21:56:02.551 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.550+0000 7f5f32000640 1 -- 172.21.15.72:0/638542738 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5f2c00fa80 con 0x7f5f34106720 2024-04-26T21:56:02.551 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.550+0000 7f5f3c200640 1 -- 172.21.15.72:0/638542738 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f341a2bb0 con 0x7f5f34106720 2024-04-26T21:56:02.551 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.550+0000 7f5f32000640 1 -- 172.21.15.72:0/638542738 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f2c018ad0 con 0x7f5f34106720 2024-04-26T21:56:02.551 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.550+0000 7f5f3c200640 1 -- 172.21.15.72:0/638542738 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f341a30d0 con 0x7f5f34106720 2024-04-26T21:56:02.553 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.553+0000 7f5f3efdd740 1 Processor -- start 2024-04-26T21:56:02.553 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.553+0000 7f5f32000640 1 -- 172.21.15.72:0/638542738 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f5f2c00f5f0 con 0x7f5f34106720 2024-04-26T21:56:02.553 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.553+0000 7f5f3efdd740 1 -- start start 2024-04-26T21:56:02.555 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.554+0000 7f5f3efdd740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55651a08bea0 0x55651a08c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:02.555 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.554+0000 7f5f3efdd740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55651a08c890 0x55651a091390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:02.555 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.554+0000 7f5f3efdd740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55651a091a00 0x55651a093e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:02.555 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.554+0000 7f5f3efdd740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x556519fefc70 con 0x55651a091a00 2024-04-26T21:56:02.555 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.554+0000 7f5f3efdd740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x556519fa8980 con 0x55651a08c890 2024-04-26T21:56:02.555 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.554+0000 7f5f3efdd740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x556519febc30 con 0x55651a08bea0 2024-04-26T21:56:02.555 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.554+0000 7f5f38e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55651a091a00 0x55651a093e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:02.556 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.554+0000 7f5f38e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55651a091a00 0x55651a093e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:37402/0 (socket says 172.21.15.72:37402) 2024-04-26T21:56:02.556 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.554+0000 7f5f38e00640 1 -- 172.21.15.72:0/2118861977 learned_addr learned my addr 172.21.15.72:0/2118861977 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:56:02.556 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.554+0000 7f5f39800640 1 --2- 172.21.15.72:0/2118861977 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55651a08c890 0x55651a091390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:02.556 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.555+0000 7f5f38e00640 1 --2- 172.21.15.72:0/2118861977 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55651a08bea0 0x55651a08c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:02.556 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.555+0000 7f5f38e00640 1 -- 172.21.15.72:0/2118861977 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55651a08bea0 msgr2=0x55651a08c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:56:02.556 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.555+0000 7f5f38e00640 1 --2- 172.21.15.72:0/2118861977 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55651a08bea0 0x55651a08c2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:02.556 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.555+0000 7f5f38e00640 1 -- 172.21.15.72:0/2118861977 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55651a08c890 msgr2=0x55651a091390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:56:02.556 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.555+0000 7f5f38e00640 1 --2- 172.21.15.72:0/2118861977 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55651a08c890 0x55651a091390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:02.556 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.555+0000 7f5f38e00640 1 -- 172.21.15.72:0/2118861977 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556519f807b0 con 0x55651a091a00 2024-04-26T21:56:02.556 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.555+0000 7f5f32000640 1 --2- 172.21.15.72:0/638542738 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f5f1c0a3cc0 0x7f5f1c0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:02.556 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.555+0000 7f5f32000640 1 -- 172.21.15.72:0/638542738 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7f5f2c0d0e80 con 0x7f5f34106720 2024-04-26T21:56:02.556 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.555+0000 7f5f38e00640 1 --2- 172.21.15.72:0/2118861977 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55651a091a00 0x55651a093e90 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f5f20005ec0 tx=0x7f5f20009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c5b85afc7df7cae server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:56:02.556 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.555+0000 7f5f39800640 1 --2- 172.21.15.72:0/638542738 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f5f1c0a3cc0 0x7f5f1c0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:02.560 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.556+0000 7f5f26a00640 1 -- 172.21.15.72:0/2118861977 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f20044020 con 0x55651a091a00 2024-04-26T21:56:02.560 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.556+0000 7f5f26a00640 1 -- 172.21.15.72:0/2118861977 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5f20003e00 con 0x55651a091a00 2024-04-26T21:56:02.560 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.556+0000 7f5f26a00640 1 -- 172.21.15.72:0/2118861977 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f20035070 con 0x55651a091a00 2024-04-26T21:56:02.560 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.556+0000 7f5f3efdd740 1 -- 172.21.15.72:0/2118861977 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55651a091a00 msgr2=0x55651a093e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:02.560 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.556+0000 7f5f39800640 1 --2- 172.21.15.72:0/638542738 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f5f1c0a3cc0 0x7f5f1c0a6180 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f5f28005fd0 tx=0x7f5f28005c50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:56:02.561 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.556+0000 7f5f3efdd740 1 --2- 172.21.15.72:0/2118861977 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55651a091a00 0x55651a093e90 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f5f20005ec0 tx=0x7f5f20009580 comp rx=0 tx=0).stop 2024-04-26T21:56:02.561 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.557+0000 7f5f3efdd740 1 -- 172.21.15.72:0/2118861977 shutdown_connections 2024-04-26T21:56:02.561 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.557+0000 7f5f3efdd740 1 --2- 172.21.15.72:0/2118861977 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55651a091a00 0x55651a093e90 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:02.561 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.557+0000 7f5f3efdd740 1 --2- 172.21.15.72:0/2118861977 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55651a08c890 0x55651a091390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:02.561 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.557+0000 7f5f3efdd740 1 --2- 172.21.15.72:0/2118861977 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55651a08bea0 0x55651a08c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:02.561 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.557+0000 7f5f3efdd740 1 -- 172.21.15.72:0/2118861977 >> 172.21.15.72:0/2118861977 conn(0x55651a087510 msgr2=0x55651a087910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:56:02.561 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.557+0000 7f5f3efdd740 1 -- 172.21.15.72:0/2118861977 shutdown_connections 2024-04-26T21:56:02.561 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.557+0000 7f5f3efdd740 1 -- 172.21.15.72:0/2118861977 wait complete. 2024-04-26T21:56:02.561 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.557+0000 7f5f3efdd740 1 -- start start 2024-04-26T21:56:02.561 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.557+0000 7f5f3efdd740 1 Processor -- start 2024-04-26T21:56:02.561 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.558+0000 7f5f3efdd740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55651a08bea0 0x55651a095d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:02.561 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.558+0000 7f5f3efdd740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55651a08c890 0x55651a0962d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:02.562 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.558+0000 7f5f3efdd740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55651a091a00 0x55651a09d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:02.562 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.558+0000 7f5f3efdd740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x556519feef80 con 0x55651a08c890 2024-04-26T21:56:02.562 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.558+0000 7f5f3efdd740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x556519f9c260 con 0x55651a08bea0 2024-04-26T21:56:02.562 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.558+0000 7f5f3efdd740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x556519ff42a0 con 0x55651a091a00 2024-04-26T21:56:02.562 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.558+0000 7f5f3efdd740 10 client.0 _ll_register_callbacks cb 0x556519fe6960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:56:02.562 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.558+0000 7f5f38e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55651a08c890 0x55651a0962d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:02.562 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.558+0000 7f5f38e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55651a08c890 0x55651a0962d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:37418/0 (socket says 172.21.15.72:37418) 2024-04-26T21:56:02.562 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.558+0000 7f5f38e00640 1 -- 172.21.15.72:0/1183422664 learned_addr learned my addr 172.21.15.72:0/1183422664 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:56:02.562 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.558+0000 7f5f39800640 1 --2- 172.21.15.72:0/1183422664 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55651a08bea0 0x55651a095d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:02.562 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.558+0000 7f5f38e00640 1 -- 172.21.15.72:0/1183422664 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55651a091a00 msgr2=0x55651a09d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:56:02.562 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.558+0000 7f5f38e00640 1 --2- 172.21.15.72:0/1183422664 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55651a091a00 0x55651a09d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:02.562 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.558+0000 7f5f38e00640 1 -- 172.21.15.72:0/1183422664 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55651a08bea0 msgr2=0x55651a095d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:56:02.562 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.558+0000 7f5f38e00640 1 --2- 172.21.15.72:0/1183422664 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55651a08bea0 0x55651a095d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:02.562 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.558+0000 7f5f38e00640 1 -- 172.21.15.72:0/1183422664 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556519f7e1c0 con 0x55651a08c890 2024-04-26T21:56:02.563 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.559+0000 7f5f38e00640 1 --2- 172.21.15.72:0/1183422664 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55651a08c890 0x55651a0962d0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f5f20031810 tx=0x7f5f20031840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ac422889a5f7a4e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:56:02.563 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.559+0000 7f5f1b400640 10 client.0 ms_handle_connect on v2:172.21.15.72:3300/0 2024-04-26T21:56:02.563 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.559+0000 7f5f1b400640 1 -- 172.21.15.72:0/1183422664 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f20044030 con 0x55651a08c890 2024-04-26T21:56:02.563 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.559+0000 7f5f1b400640 1 -- 172.21.15.72:0/1183422664 <== mon.0 v2:172.21.15.72:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7f5f20003b80 con 0x55651a08c890 2024-04-26T21:56:02.563 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.559+0000 7f5f3efdd740 1 -- 172.21.15.72:0/1183422664 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f20003e00 con 0x55651a08c890 2024-04-26T21:56:02.563 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.559+0000 7f5f1b400640 1 -- 172.21.15.72:0/1183422664 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f5f20056030 con 0x55651a08c890 2024-04-26T21:56:02.563 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.559+0000 7f5f3efdd740 10 client.15561 fetch_fsmap learned FSMap version 27 2024-04-26T21:56:02.563 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.559+0000 7f5f3efdd740 1 -- 172.21.15.72:0/1183422664 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556519d013f0 con 0x55651a08c890 2024-04-26T21:56:02.563 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.560+0000 7f5f1b400640 1 -- 172.21.15.72:0/1183422664 <== mon.0 v2:172.21.15.72:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5f20003e00 con 0x55651a08c890 2024-04-26T21:56:02.563 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.560+0000 7f5f1b400640 1 -- 172.21.15.72:0/1183422664 <== mon.0 v2:172.21.15.72:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f5f2005b430 con 0x55651a08c890 2024-04-26T21:56:02.563 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.560+0000 7f5f3efdd740 10 client.15561 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T21:56:02.563 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.560+0000 7f5f3efdd740 10 client.15561 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-04-26T21:56:02.563 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.560+0000 7f5f3efdd740 20 client.15561 populate_metadata read hostname 'smithi072' 2024-04-26T21:56:02.563 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.560+0000 7f5f3efdd740 1 --2- 172.21.15.72:0/1183422664 >> [v2:172.21.15.164:6838/1636226820,v1:172.21.15.164:6839/1636226820] conn(0x55651a0a1470 0x55651a0a3860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:02.564 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.560+0000 7f5f3efdd740 4 client.15561 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:56:02.564 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.560+0000 7f5f3efdd740 1 -- 172.21.15.72:0/1183422664 --> [v2:172.21.15.164:6838/1636226820,v1:172.21.15.164:6839/1636226820] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556519ffd880 con 0x55651a0a1470 2024-04-26T21:56:02.565 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.561+0000 7f5f39800640 1 --2- 172.21.15.72:0/1183422664 >> [v2:172.21.15.164:6838/1636226820,v1:172.21.15.164:6839/1636226820] conn(0x55651a0a1470 0x55651a0a3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:02.565 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.562+0000 7f5f39800640 1 --2- 172.21.15.72:0/1183422664 >> [v2:172.21.15.164:6838/1636226820,v1:172.21.15.164:6839/1636226820] conn(0x55651a0a1470 0x55651a0a3860 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b9132ce34f862f4b server_cookie=f091655470e550d3 in_seq=0 out_seq=0 2024-04-26T21:56:02.565 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.562+0000 7f5f1b400640 10 client.15561 ms_handle_connect on v2:172.21.15.164:6838/1636226820 2024-04-26T21:56:02.565 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.564+0000 7f5f1b400640 1 -- 172.21.15.72:0/1183422664 <== mds.4 v2:172.21.15.164:6838/1636226820 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556519ffd880 con 0x55651a0a1470 2024-04-26T21:56:02.565 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.564+0000 7f5f1b400640 10 client.15561 handle_command_reply: tid=0 2024-04-26T21:56:02.565 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.564+0000 7f5f3efdd740 1 client.15561 shutdown 2024-04-26T21:56:02.565 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.564+0000 7f5f3efdd740 1 -- 172.21.15.72:0/1183422664 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55651a08c890 msgr2=0x55651a0962d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:02.565 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.564+0000 7f5f3efdd740 1 --2- 172.21.15.72:0/1183422664 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55651a08c890 0x55651a0962d0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f5f20031810 tx=0x7f5f20031840 comp rx=0 tx=0).stop 2024-04-26T21:56:02.565 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.564+0000 7f5f3efdd740 1 -- 172.21.15.72:0/1183422664 shutdown_connections 2024-04-26T21:56:02.566 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.564+0000 7f5f3efdd740 1 --2- 172.21.15.72:0/1183422664 >> [v2:172.21.15.164:6838/1636226820,v1:172.21.15.164:6839/1636226820] conn(0x55651a0a1470 0x55651a0a3860 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:02.566 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.564+0000 7f5f3efdd740 1 --2- 172.21.15.72:0/1183422664 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55651a091a00 0x55651a09d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:02.566 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.564+0000 7f5f3efdd740 1 --2- 172.21.15.72:0/1183422664 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55651a08c890 0x55651a0962d0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:02.566 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.564+0000 7f5f3efdd740 1 --2- 172.21.15.72:0/1183422664 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55651a08bea0 0x55651a095d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:02.566 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.564+0000 7f5f1b400640 0 client.15561 ms_handle_reset on v2:172.21.15.164:6838/1636226820 2024-04-26T21:56:02.566 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.564+0000 7f5f3efdd740 1 -- 172.21.15.72:0/1183422664 >> 172.21.15.72:0/1183422664 conn(0x55651a087510 msgr2=0x55651a086690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:56:02.566 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.564+0000 7f5f3efdd740 1 -- 172.21.15.72:0/1183422664 shutdown_connections 2024-04-26T21:56:02.566 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.564+0000 7f5f3efdd740 1 -- 172.21.15.72:0/1183422664 wait complete. 2024-04-26T21:56:02.566 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.564+0000 7f5f3efdd740 20 client.15561 trim_cache size 0 max 16384 2024-04-26T21:56:02.580 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.579+0000 7f5f3efdd740 1 Processor -- start 2024-04-26T21:56:02.580 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.579+0000 7f5f3efdd740 1 -- start start 2024-04-26T21:56:02.580 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.579+0000 7f5f3efdd740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55651a08bea0 0x55651a08d330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:02.580 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.579+0000 7f5f3efdd740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55651a08c890 0x55651a09d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:02.580 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.579+0000 7f5f3efdd740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55651a091a00 0x55651a08d870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:02.580 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.579+0000 7f5f3efdd740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f5f341abff0 con 0x55651a08c890 2024-04-26T21:56:02.581 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.579+0000 7f5f3efdd740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f5f341adb30 con 0x55651a091a00 2024-04-26T21:56:02.581 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.579+0000 7f5f3efdd740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f5f341ac6c0 con 0x55651a08bea0 2024-04-26T21:56:02.581 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.579+0000 7f5f38e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55651a08c890 0x55651a09d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:02.581 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.579+0000 7f5f38e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55651a08c890 0x55651a09d2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:37422/0 (socket says 172.21.15.72:37422) 2024-04-26T21:56:02.581 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.579+0000 7f5f38e00640 1 -- 172.21.15.72:0/1650494359 learned_addr learned my addr 172.21.15.72:0/1650494359 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:56:02.581 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.580+0000 7f5f39800640 1 --2- 172.21.15.72:0/1650494359 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55651a08bea0 0x55651a08d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:02.581 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.580+0000 7f5f3a200640 1 --2- 172.21.15.72:0/1650494359 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55651a091a00 0x55651a08d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:02.581 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.580+0000 7f5f38e00640 1 -- 172.21.15.72:0/1650494359 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55651a08bea0 msgr2=0x55651a08d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:56:02.581 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.580+0000 7f5f38e00640 1 --2- 172.21.15.72:0/1650494359 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55651a08bea0 0x55651a08d330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:02.581 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.580+0000 7f5f38e00640 1 -- 172.21.15.72:0/1650494359 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55651a091a00 msgr2=0x55651a08d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:56:02.581 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.580+0000 7f5f38e00640 1 --2- 172.21.15.72:0/1650494359 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55651a091a00 0x55651a08d870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:02.581 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.580+0000 7f5f38e00640 1 -- 172.21.15.72:0/1650494359 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556519dcb7d0 con 0x55651a08c890 2024-04-26T21:56:02.581 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.580+0000 7f5f39800640 1 --2- 172.21.15.72:0/1650494359 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55651a08bea0 0x55651a08d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:56:02.582 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.580+0000 7f5f38e00640 1 --2- 172.21.15.72:0/1650494359 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55651a08c890 0x55651a09d2d0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f5f2005b920 tx=0x7f5f20005c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49c65a195372287f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:56:02.582 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.580+0000 7f5f26000640 1 -- 172.21.15.72:0/1650494359 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f2005f070 con 0x55651a08c890 2024-04-26T21:56:02.582 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.580+0000 7f5f26000640 1 -- 172.21.15.72:0/1650494359 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5f2004d070 con 0x55651a08c890 2024-04-26T21:56:02.582 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.580+0000 7f5f26000640 1 -- 172.21.15.72:0/1650494359 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f20044070 con 0x55651a08c890 2024-04-26T21:56:02.585 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.581+0000 7f5f3efdd740 1 -- 172.21.15.72:0/1650494359 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55651a08c890 msgr2=0x55651a09d2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:02.585 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.581+0000 7f5f3efdd740 1 --2- 172.21.15.72:0/1650494359 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55651a08c890 0x55651a09d2d0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f5f2005b920 tx=0x7f5f20005c90 comp rx=0 tx=0).stop 2024-04-26T21:56:02.585 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.581+0000 7f5f3efdd740 1 -- 172.21.15.72:0/1650494359 shutdown_connections 2024-04-26T21:56:02.586 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.581+0000 7f5f3efdd740 1 --2- 172.21.15.72:0/1650494359 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55651a091a00 0x55651a08d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:02.586 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.581+0000 7f5f3efdd740 1 --2- 172.21.15.72:0/1650494359 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55651a08c890 0x55651a09d2d0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:02.586 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.581+0000 7f5f3efdd740 1 --2- 172.21.15.72:0/1650494359 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55651a08bea0 0x55651a08d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:02.586 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.581+0000 7f5f3efdd740 1 -- 172.21.15.72:0/1650494359 >> 172.21.15.72:0/1650494359 conn(0x55651a087510 msgr2=0x55651a086690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:56:02.586 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.581+0000 7f5f3efdd740 1 -- 172.21.15.72:0/1650494359 shutdown_connections 2024-04-26T21:56:02.586 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.581+0000 7f5f3efdd740 1 -- 172.21.15.72:0/1650494359 wait complete. 2024-04-26T21:56:02.586 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.581+0000 7f5f3efdd740 1 -- start start 2024-04-26T21:56:02.586 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.582+0000 7f5f3efdd740 1 Processor -- start 2024-04-26T21:56:02.586 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.582+0000 7f5f3efdd740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55651a08bea0 0x55651a09a800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:02.586 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.582+0000 7f5f3efdd740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55651a08c890 0x55651a09ad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:02.586 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.582+0000 7f5f3efdd740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55651a091a00 0x55651a09bce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:02.586 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.582+0000 7f5f3efdd740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f5f341ad460 con 0x55651a08c890 2024-04-26T21:56:02.587 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.582+0000 7f5f3efdd740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f5f341acd90 con 0x55651a091a00 2024-04-26T21:56:02.587 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.582+0000 7f5f3efdd740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f5f341a9fd0 con 0x55651a08bea0 2024-04-26T21:56:02.587 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.582+0000 7f5f3efdd740 10 client.0 _ll_register_callbacks cb 0x7f5f20004570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:56:02.587 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.582+0000 7f5f38e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55651a08c890 0x55651a09ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:02.587 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.582+0000 7f5f38e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55651a08c890 0x55651a09ad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:37428/0 (socket says 172.21.15.72:37428) 2024-04-26T21:56:02.587 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.582+0000 7f5f38e00640 1 -- 172.21.15.72:0/4173128922 learned_addr learned my addr 172.21.15.72:0/4173128922 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:56:02.587 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.583+0000 7f5f39800640 1 --2- 172.21.15.72:0/4173128922 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55651a08bea0 0x55651a09a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:02.587 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.583+0000 7f5f3a200640 1 --2- 172.21.15.72:0/4173128922 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55651a091a00 0x55651a09bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:02.587 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.583+0000 7f5f38e00640 1 -- 172.21.15.72:0/4173128922 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55651a08bea0 msgr2=0x55651a09a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:56:02.587 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.583+0000 7f5f38e00640 1 --2- 172.21.15.72:0/4173128922 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55651a08bea0 0x55651a09a800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:02.587 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.583+0000 7f5f38e00640 1 -- 172.21.15.72:0/4173128922 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55651a091a00 msgr2=0x55651a09bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:56:02.587 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.583+0000 7f5f38e00640 1 --2- 172.21.15.72:0/4173128922 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55651a091a00 0x55651a09bce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:02.588 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.583+0000 7f5f38e00640 1 -- 172.21.15.72:0/4173128922 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556519f962a0 con 0x55651a08c890 2024-04-26T21:56:02.588 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.583+0000 7f5f3a200640 1 --2- 172.21.15.72:0/4173128922 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55651a091a00 0x55651a09bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:56:02.588 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.583+0000 7f5f38e00640 1 --2- 172.21.15.72:0/4173128922 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55651a08c890 0x55651a09ad40 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f5f20005ec0 tx=0x7f5f20031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c7aba3028c21964 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:56:02.588 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.583+0000 7f5f1b400640 10 client.0 ms_handle_connect on v2:172.21.15.72:3300/0 2024-04-26T21:56:02.588 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.583+0000 7f5f1b400640 1 -- 172.21.15.72:0/4173128922 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f20044070 con 0x55651a08c890 2024-04-26T21:56:02.588 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.583+0000 7f5f1b400640 1 -- 172.21.15.72:0/4173128922 <== mon.0 v2:172.21.15.72:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7f5f2005bb10 con 0x55651a08c890 2024-04-26T21:56:02.588 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.583+0000 7f5f3efdd740 1 -- 172.21.15.72:0/4173128922 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f2004d070 con 0x55651a08c890 2024-04-26T21:56:02.588 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.584+0000 7f5f1b400640 1 -- 172.21.15.72:0/4173128922 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f5f2006b050 con 0x55651a08c890 2024-04-26T21:56:02.588 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.584+0000 7f5f3efdd740 10 client.15567 fetch_fsmap learned FSMap version 27 2024-04-26T21:56:02.588 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.584+0000 7f5f3efdd740 1 -- 172.21.15.72:0/4173128922 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55651a09d370 con 0x55651a08c890 2024-04-26T21:56:02.588 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.584+0000 7f5f1b400640 1 -- 172.21.15.72:0/4173128922 <== mon.0 v2:172.21.15.72:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5f2004d070 con 0x55651a08c890 2024-04-26T21:56:02.588 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.584+0000 7f5f1b400640 1 -- 172.21.15.72:0/4173128922 <== mon.0 v2:172.21.15.72:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f5f20052760 con 0x55651a08c890 2024-04-26T21:56:02.588 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.584+0000 7f5f3efdd740 10 client.15567 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T21:56:02.588 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.584+0000 7f5f3efdd740 10 client.15567 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-04-26T21:56:02.588 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.584+0000 7f5f3efdd740 20 client.15567 populate_metadata read hostname 'smithi072' 2024-04-26T21:56:02.589 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.584+0000 7f5f3efdd740 1 --2- 172.21.15.72:0/4173128922 >> [v2:172.21.15.164:6838/1636226820,v1:172.21.15.164:6839/1636226820] conn(0x55651a107de0 0x55651a10a200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:02.589 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.584+0000 7f5f3efdd740 4 client.15567 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-26T21:56:02.589 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.584+0000 7f5f3efdd740 1 -- 172.21.15.72:0/4173128922 --> [v2:172.21.15.164:6838/1636226820,v1:172.21.15.164:6839/1636226820] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556519ff76f0 con 0x55651a107de0 2024-04-26T21:56:02.589 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.585+0000 7f5f39800640 1 --2- 172.21.15.72:0/4173128922 >> [v2:172.21.15.164:6838/1636226820,v1:172.21.15.164:6839/1636226820] conn(0x55651a107de0 0x55651a10a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:02.590 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.586+0000 7f5f39800640 1 --2- 172.21.15.72:0/4173128922 >> [v2:172.21.15.164:6838/1636226820,v1:172.21.15.164:6839/1636226820] conn(0x55651a107de0 0x55651a10a200 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f8c10f38329c2121 server_cookie=8473a20475577635 in_seq=0 out_seq=0 2024-04-26T21:56:02.590 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.586+0000 7f5f1b400640 10 client.15567 ms_handle_connect on v2:172.21.15.164:6838/1636226820 2024-04-26T21:56:02.590 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.586+0000 7f5f1b400640 1 -- 172.21.15.72:0/4173128922 <== mds.4 v2:172.21.15.164:6838/1636226820 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556519ff76f0 con 0x55651a107de0 2024-04-26T21:56:02.590 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.586+0000 7f5f1b400640 10 client.15567 handle_command_reply: tid=0 2024-04-26T21:56:02.590 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.586+0000 7f5f3efdd740 1 client.15567 shutdown 2024-04-26T21:56:02.590 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.587+0000 7f5f3efdd740 1 -- 172.21.15.72:0/4173128922 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55651a08c890 msgr2=0x55651a09ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:02.591 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.587+0000 7f5f3efdd740 1 --2- 172.21.15.72:0/4173128922 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55651a08c890 0x55651a09ad40 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f5f20005ec0 tx=0x7f5f20031400 comp rx=0 tx=0).stop 2024-04-26T21:56:02.591 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.587+0000 7f5f3efdd740 1 -- 172.21.15.72:0/4173128922 shutdown_connections 2024-04-26T21:56:02.591 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.587+0000 7f5f3efdd740 1 --2- 172.21.15.72:0/4173128922 >> [v2:172.21.15.164:6838/1636226820,v1:172.21.15.164:6839/1636226820] conn(0x55651a107de0 0x55651a10a200 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:02.591 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.587+0000 7f5f3efdd740 1 --2- 172.21.15.72:0/4173128922 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55651a091a00 0x55651a09bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:02.591 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.587+0000 7f5f3efdd740 1 --2- 172.21.15.72:0/4173128922 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55651a08c890 0x55651a09ad40 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:02.591 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.587+0000 7f5f3efdd740 1 --2- 172.21.15.72:0/4173128922 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55651a08bea0 0x55651a09a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:02.591 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.587+0000 7f5f3efdd740 1 -- 172.21.15.72:0/4173128922 >> 172.21.15.72:0/4173128922 conn(0x55651a087510 msgr2=0x55651a086690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:56:02.591 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.587+0000 7f5f1b400640 0 client.15567 ms_handle_reset on v2:172.21.15.164:6838/1636226820 2024-04-26T21:56:02.591 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.587+0000 7f5f3efdd740 1 -- 172.21.15.72:0/4173128922 shutdown_connections 2024-04-26T21:56:02.591 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.587+0000 7f5f3efdd740 1 -- 172.21.15.72:0/4173128922 wait complete. 2024-04-26T21:56:02.591 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.587+0000 7f5f3efdd740 20 client.15567 trim_cache size 0 max 16384 2024-04-26T21:56:02.591 INFO:teuthology.orchestra.run.smithi072.stdout:[] 2024-04-26T21:56:02.592 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.588+0000 7f5f3c200640 1 -- 172.21.15.72:0/638542738 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f5f1c0a3cc0 msgr2=0x7f5f1c0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:02.592 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.588+0000 7f5f3c200640 1 --2- 172.21.15.72:0/638542738 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f5f1c0a3cc0 0x7f5f1c0a6180 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f5f28005fd0 tx=0x7f5f28005c50 comp rx=0 tx=0).stop 2024-04-26T21:56:02.592 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.588+0000 7f5f3c200640 1 -- 172.21.15.72:0/638542738 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5f34106720 msgr2=0x7f5f341a2220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:02.592 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.588+0000 7f5f3c200640 1 --2- 172.21.15.72:0/638542738 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5f34106720 0x7f5f341a2220 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f5f2c00cce0 tx=0x7f5f2c00a620 comp rx=0 tx=0).stop 2024-04-26T21:56:02.592 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.588+0000 7f5f3c200640 1 -- 172.21.15.72:0/638542738 shutdown_connections 2024-04-26T21:56:02.592 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.588+0000 7f5f3c200640 1 --2- 172.21.15.72:0/638542738 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5f34106720 0x7f5f341a2220 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:02.593 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.588+0000 7f5f3c200640 1 --2- 172.21.15.72:0/638542738 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5f34105d60 0x7f5f3419b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:02.593 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.588+0000 7f5f3c200640 1 --2- 172.21.15.72:0/638542738 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f5f1c0a3cc0 0x7f5f1c0a6180 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:02.593 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.588+0000 7f5f3c200640 1 --2- 172.21.15.72:0/638542738 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5f34104b70 0x7f5f3419ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:02.593 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.588+0000 7f5f3c200640 1 -- 172.21.15.72:0/638542738 >> 172.21.15.72:0/638542738 conn(0x7f5f341000e0 msgr2=0x7f5f34109080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:56:02.593 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.588+0000 7f5f3c200640 1 -- 172.21.15.72:0/638542738 shutdown_connections 2024-04-26T21:56:02.593 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.588+0000 7f5f3c200640 1 -- 172.21.15.72:0/638542738 wait complete. 2024-04-26T21:56:02.605 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-26T21:56:02.800 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.800+0000 7f1cfaa00640 1 -- 172.21.15.72:0/559586343 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1cf4102860 msgr2=0x7f1cf4102c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:02.800 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.800+0000 7f1cfaa00640 1 --2- 172.21.15.72:0/559586343 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1cf4102860 0x7f1cf4102c80 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f1ce8009900 tx=0x7f1ce802a410 comp rx=0 tx=0).stop 2024-04-26T21:56:02.800 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.800+0000 7f1cfaa00640 1 -- 172.21.15.72:0/559586343 shutdown_connections 2024-04-26T21:56:02.800 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.800+0000 7f1cfaa00640 1 --2- 172.21.15.72:0/559586343 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1cf4104410 0x7f1cf410aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:02.800 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.800+0000 7f1cfaa00640 1 --2- 172.21.15.72:0/559586343 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1cf4103a50 0x7f1cf4103ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:02.801 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.800+0000 7f1cfaa00640 1 --2- 172.21.15.72:0/559586343 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1cf4102860 0x7f1cf4102c80 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:02.801 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.800+0000 7f1cfaa00640 1 -- 172.21.15.72:0/559586343 >> 172.21.15.72:0/559586343 conn(0x7f1cf40fddd0 msgr2=0x7f1cf4100210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:56:02.801 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.800+0000 7f1cfaa00640 1 -- 172.21.15.72:0/559586343 shutdown_connections 2024-04-26T21:56:02.801 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.800+0000 7f1cfaa00640 1 -- 172.21.15.72:0/559586343 wait complete. 2024-04-26T21:56:02.803 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.801+0000 7f1cfaa00640 1 Processor -- start 2024-04-26T21:56:02.803 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.801+0000 7f1cfaa00640 1 -- start start 2024-04-26T21:56:02.803 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.802+0000 7f1cfaa00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1cf4102860 0x7f1cf4076690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:02.803 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.802+0000 7f1cfaa00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1cf4103a50 0x7f1cf4074d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:02.804 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.802+0000 7f1cfaa00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1cf4104410 0x7f1cf4075260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:02.804 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.802+0000 7f1cfaa00640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f1cf410e290 con 0x7f1cf4104410 2024-04-26T21:56:02.804 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.802+0000 7f1cfaa00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f1cf410e110 con 0x7f1cf4102860 2024-04-26T21:56:02.804 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.802+0000 7f1cfaa00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f1cf410e410 con 0x7f1cf4103a50 2024-04-26T21:56:02.804 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.802+0000 7f1cfa000640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1cf4104410 0x7f1cf4075260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:02.804 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.802+0000 7f1cfa000640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1cf4104410 0x7f1cf4075260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:37442/0 (socket says 172.21.15.72:37442) 2024-04-26T21:56:02.804 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.802+0000 7f1cfa000640 1 -- 172.21.15.72:0/3049385695 learned_addr learned my addr 172.21.15.72:0/3049385695 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:56:02.804 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.802+0000 7f1cf8c00640 1 --2- 172.21.15.72:0/3049385695 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1cf4103a50 0x7f1cf4074d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:02.804 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.802+0000 7f1cf9600640 1 --2- 172.21.15.72:0/3049385695 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1cf4102860 0x7f1cf4076690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:02.804 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.803+0000 7f1cfa000640 1 -- 172.21.15.72:0/3049385695 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1cf4103a50 msgr2=0x7f1cf4074d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:02.804 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.803+0000 7f1cfa000640 1 --2- 172.21.15.72:0/3049385695 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1cf4103a50 0x7f1cf4074d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:02.804 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.803+0000 7f1cfa000640 1 -- 172.21.15.72:0/3049385695 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1cf4102860 msgr2=0x7f1cf4076690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:02.804 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.803+0000 7f1cfa000640 1 --2- 172.21.15.72:0/3049385695 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1cf4102860 0x7f1cf4076690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:02.805 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.803+0000 7f1cfa000640 1 -- 172.21.15.72:0/3049385695 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cf4075b20 con 0x7f1cf4104410 2024-04-26T21:56:02.805 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.803+0000 7f1cf8c00640 1 --2- 172.21.15.72:0/3049385695 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1cf4103a50 0x7f1cf4074d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:56:02.805 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.803+0000 7f1cf9600640 1 --2- 172.21.15.72:0/3049385695 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1cf4102860 0x7f1cf4076690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:56:02.805 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.803+0000 7f1cfa000640 1 --2- 172.21.15.72:0/3049385695 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1cf4104410 0x7f1cf4075260 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f1cec00eaa0 tx=0x7f1cec00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:56:02.805 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.803+0000 7f1cf2000640 1 -- 172.21.15.72:0/3049385695 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cec00cdb0 con 0x7f1cf4104410 2024-04-26T21:56:02.805 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.803+0000 7f1cf2000640 1 -- 172.21.15.72:0/3049385695 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1cec005020 con 0x7f1cf4104410 2024-04-26T21:56:02.805 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.803+0000 7f1cf2000640 1 -- 172.21.15.72:0/3049385695 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cec0116d0 con 0x7f1cf4104410 2024-04-26T21:56:02.805 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.803+0000 7f1cfaa00640 1 -- 172.21.15.72:0/3049385695 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1cf41a2b50 con 0x7f1cf4104410 2024-04-26T21:56:02.805 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.803+0000 7f1cfaa00640 1 -- 172.21.15.72:0/3049385695 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cf41a3020 con 0x7f1cf4104410 2024-04-26T21:56:02.806 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.807+0000 7f1cfefd7740 1 Processor -- start 2024-04-26T21:56:02.807 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.807+0000 7f1cf2000640 1 -- 172.21.15.72:0/3049385695 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f1cec0051d0 con 0x7f1cf4104410 2024-04-26T21:56:02.807 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.807+0000 7f1cfefd7740 1 -- start start 2024-04-26T21:56:02.809 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.807+0000 7f1cfefd7740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d4ea0d6050 0x55d4ea0d6470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:02.809 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.807+0000 7f1cfefd7740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d4ea0d6a40 0x55d4ea0db540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:02.809 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.807+0000 7f1cfefd7740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d4ea0dbbb0 0x55d4ea0de040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:02.809 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.807+0000 7f1cfefd7740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55d4ea039c70 con 0x55d4ea0d6050 2024-04-26T21:56:02.809 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.807+0000 7f1cfefd7740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55d4e9ff2980 con 0x55d4ea0dbbb0 2024-04-26T21:56:02.809 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.807+0000 7f1cfefd7740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55d4ea035c30 con 0x55d4ea0d6a40 2024-04-26T21:56:02.809 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.808+0000 7f1cf8c00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d4ea0d6050 0x55d4ea0d6470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:02.809 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.808+0000 7f1cf8c00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d4ea0d6050 0x55d4ea0d6470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:37450/0 (socket says 172.21.15.72:37450) 2024-04-26T21:56:02.809 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.808+0000 7f1cf8c00640 1 -- 172.21.15.72:0/3459386548 learned_addr learned my addr 172.21.15.72:0/3459386548 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:56:02.809 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.808+0000 7f1cf9600640 1 --2- 172.21.15.72:0/3459386548 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d4ea0d6a40 0x55d4ea0db540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:02.810 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.808+0000 7f1cf8c00640 1 -- 172.21.15.72:0/3459386548 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d4ea0d6a40 msgr2=0x55d4ea0db540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:56:02.810 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.808+0000 7f1cf8c00640 1 --2- 172.21.15.72:0/3459386548 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d4ea0d6a40 0x55d4ea0db540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:02.810 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.808+0000 7f1cf8c00640 1 -- 172.21.15.72:0/3459386548 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d4ea0dbbb0 msgr2=0x55d4ea0de040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:56:02.810 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.808+0000 7f1cf8c00640 1 --2- 172.21.15.72:0/3459386548 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d4ea0dbbb0 0x55d4ea0de040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:02.810 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.808+0000 7f1cf8c00640 1 -- 172.21.15.72:0/3459386548 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d4e9fca7b0 con 0x55d4ea0d6050 2024-04-26T21:56:02.810 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.809+0000 7f1cf8c00640 1 --2- 172.21.15.72:0/3459386548 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d4ea0d6050 0x55d4ea0d6470 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f1ce0009cb0 tx=0x7f1ce00106e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b38afc1c0542630 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:56:02.810 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.809+0000 7f1cf2000640 1 --2- 172.21.15.72:0/3049385695 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f1cdc0a3cc0 0x7f1cdc0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:02.810 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.809+0000 7f1ce6600640 1 -- 172.21.15.72:0/3459386548 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ce0016070 con 0x55d4ea0d6050 2024-04-26T21:56:02.810 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.809+0000 7f1cf2000640 1 -- 172.21.15.72:0/3049385695 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7f1cec0d08a0 con 0x7f1cf4104410 2024-04-26T21:56:02.810 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.809+0000 7f1cf9600640 1 --2- 172.21.15.72:0/3049385695 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f1cdc0a3cc0 0x7f1cdc0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:02.810 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.809+0000 7f1ce6600640 1 -- 172.21.15.72:0/3459386548 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1ce00054b0 con 0x55d4ea0d6050 2024-04-26T21:56:02.811 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.809+0000 7f1ce6600640 1 -- 172.21.15.72:0/3459386548 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ce0012420 con 0x55d4ea0d6050 2024-04-26T21:56:02.815 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.810+0000 7f1cfefd7740 1 -- 172.21.15.72:0/3459386548 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d4ea0d6050 msgr2=0x55d4ea0d6470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:02.815 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.810+0000 7f1cf9600640 1 --2- 172.21.15.72:0/3049385695 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f1cdc0a3cc0 0x7f1cdc0a6180 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f1ce8009230 tx=0x7f1ce8005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:56:02.815 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.810+0000 7f1cfefd7740 1 --2- 172.21.15.72:0/3459386548 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d4ea0d6050 0x55d4ea0d6470 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f1ce0009cb0 tx=0x7f1ce00106e0 comp rx=0 tx=0).stop 2024-04-26T21:56:02.815 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.810+0000 7f1cfefd7740 1 -- 172.21.15.72:0/3459386548 shutdown_connections 2024-04-26T21:56:02.815 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.810+0000 7f1cfefd7740 1 --2- 172.21.15.72:0/3459386548 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d4ea0dbbb0 0x55d4ea0de040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:02.815 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.810+0000 7f1cfefd7740 1 --2- 172.21.15.72:0/3459386548 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d4ea0d6a40 0x55d4ea0db540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:02.815 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.810+0000 7f1cfefd7740 1 --2- 172.21.15.72:0/3459386548 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d4ea0d6050 0x55d4ea0d6470 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:02.815 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.810+0000 7f1cfefd7740 1 -- 172.21.15.72:0/3459386548 >> 172.21.15.72:0/3459386548 conn(0x55d4ea0d16c0 msgr2=0x55d4ea0d1ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:56:02.815 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.810+0000 7f1cfefd7740 1 -- 172.21.15.72:0/3459386548 shutdown_connections 2024-04-26T21:56:02.816 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.810+0000 7f1cfefd7740 1 -- 172.21.15.72:0/3459386548 wait complete. 2024-04-26T21:56:02.816 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.810+0000 7f1cfefd7740 1 -- start start 2024-04-26T21:56:02.816 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.811+0000 7f1cfefd7740 1 Processor -- start 2024-04-26T21:56:02.816 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.811+0000 7f1cfefd7740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d4ea0d6050 0x55d4ea0dff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:02.816 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.811+0000 7f1cfefd7740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d4ea0d6a40 0x55d4ea0e04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:02.816 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.811+0000 7f1cfefd7740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d4ea0dbbb0 0x55d4ea0e74f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:02.816 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.811+0000 7f1cfefd7740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55d4ea038f80 con 0x55d4ea0dbbb0 2024-04-26T21:56:02.816 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.811+0000 7f1cfefd7740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55d4e9fe6260 con 0x55d4ea0d6a40 2024-04-26T21:56:02.816 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.811+0000 7f1cfefd7740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55d4ea03e2a0 con 0x55d4ea0d6050 2024-04-26T21:56:02.816 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.811+0000 7f1cfefd7740 10 client.0 _ll_register_callbacks cb 0x55d4ea030960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:56:02.816 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.811+0000 7f1cfa000640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d4ea0dbbb0 0x55d4ea0e74f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:02.816 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.811+0000 7f1cfa000640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d4ea0dbbb0 0x55d4ea0e74f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:37464/0 (socket says 172.21.15.72:37464) 2024-04-26T21:56:02.816 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.811+0000 7f1cfa000640 1 -- 172.21.15.72:0/3575714574 learned_addr learned my addr 172.21.15.72:0/3575714574 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:56:02.816 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.812+0000 7f1cf9600640 1 --2- 172.21.15.72:0/3575714574 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d4ea0d6050 0x55d4ea0dff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:02.817 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.812+0000 7f1cf8c00640 1 --2- 172.21.15.72:0/3575714574 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d4ea0d6a40 0x55d4ea0e04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:02.817 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.812+0000 7f1cfa000640 1 -- 172.21.15.72:0/3575714574 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d4ea0d6050 msgr2=0x55d4ea0dff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:56:02.817 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.812+0000 7f1cfa000640 1 --2- 172.21.15.72:0/3575714574 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d4ea0d6050 0x55d4ea0dff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:02.817 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.812+0000 7f1cfa000640 1 -- 172.21.15.72:0/3575714574 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d4ea0d6a40 msgr2=0x55d4ea0e04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:56:02.817 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.812+0000 7f1cfa000640 1 --2- 172.21.15.72:0/3575714574 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d4ea0d6a40 0x55d4ea0e04b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:02.817 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.812+0000 7f1cfa000640 1 -- 172.21.15.72:0/3575714574 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d4e9fc81c0 con 0x55d4ea0dbbb0 2024-04-26T21:56:02.817 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.812+0000 7f1cfa000640 1 --2- 172.21.15.72:0/3575714574 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d4ea0dbbb0 0x55d4ea0e74f0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f1cec09c8d0 tx=0x7f1cec09ca80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5edcf6aa7e19f4b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:56:02.817 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.812+0000 7f1cdaa00640 10 client.0 ms_handle_connect on v2:172.21.15.72:3300/0 2024-04-26T21:56:02.817 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.812+0000 7f1cdaa00640 1 -- 172.21.15.72:0/3575714574 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cec01f040 con 0x55d4ea0dbbb0 2024-04-26T21:56:02.817 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.813+0000 7f1cdaa00640 1 -- 172.21.15.72:0/3575714574 <== mon.0 v2:172.21.15.72:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7f1cec0870c0 con 0x55d4ea0dbbb0 2024-04-26T21:56:02.817 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.813+0000 7f1cfefd7740 1 -- 172.21.15.72:0/3575714574 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ce00054b0 con 0x55d4ea0dbbb0 2024-04-26T21:56:02.817 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.813+0000 7f1cdaa00640 1 -- 172.21.15.72:0/3575714574 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1cec087300 con 0x55d4ea0dbbb0 2024-04-26T21:56:02.817 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.813+0000 7f1cfefd7740 10 client.15579 fetch_fsmap learned FSMap version 27 2024-04-26T21:56:02.817 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.813+0000 7f1cfefd7740 1 -- 172.21.15.72:0/3575714574 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d4e9d4b3f0 con 0x55d4ea0dbbb0 2024-04-26T21:56:02.817 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.813+0000 7f1cdaa00640 1 -- 172.21.15.72:0/3575714574 <== mon.0 v2:172.21.15.72:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1ce00054b0 con 0x55d4ea0dbbb0 2024-04-26T21:56:02.818 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.813+0000 7f1cdaa00640 1 -- 172.21.15.72:0/3575714574 <== mon.0 v2:172.21.15.72:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f1cec00f040 con 0x55d4ea0dbbb0 2024-04-26T21:56:02.818 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.813+0000 7f1cfefd7740 10 client.15579 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T21:56:02.818 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.813+0000 7f1cfefd7740 10 client.15579 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-26T21:56:02.818 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.813+0000 7f1cfefd7740 20 client.15579 populate_metadata read hostname 'smithi072' 2024-04-26T21:56:02.818 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.813+0000 7f1cfefd7740 1 --2- 172.21.15.72:0/3575714574 >> [v2:172.21.15.76:6832/3869483750,v1:172.21.15.76:6833/3869483750] conn(0x55d4ea0ebb60 0x55d4ea0edfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:02.818 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.813+0000 7f1cfefd7740 4 client.15579 mds_command: new command op to 34406 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:56:02.818 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.813+0000 7f1cfefd7740 1 -- 172.21.15.72:0/3575714574 --> [v2:172.21.15.76:6832/3869483750,v1:172.21.15.76:6833/3869483750] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d4ea047880 con 0x55d4ea0ebb60 2024-04-26T21:56:02.818 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.814+0000 7f1cf9600640 1 --2- 172.21.15.72:0/3575714574 >> [v2:172.21.15.76:6832/3869483750,v1:172.21.15.76:6833/3869483750] conn(0x55d4ea0ebb60 0x55d4ea0edfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:02.818 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.815+0000 7f1cf9600640 1 --2- 172.21.15.72:0/3575714574 >> [v2:172.21.15.76:6832/3869483750,v1:172.21.15.76:6833/3869483750] conn(0x55d4ea0ebb60 0x55d4ea0edfa0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=397a187517cfaf13 server_cookie=15ce8ce6ffcf898 in_seq=0 out_seq=0 2024-04-26T21:56:02.818 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.815+0000 7f1cdaa00640 10 client.15579 ms_handle_connect on v2:172.21.15.76:6832/3869483750 2024-04-26T21:56:02.820 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.817+0000 7f1cdaa00640 1 -- 172.21.15.72:0/3575714574 <== mds.3 v2:172.21.15.76:6832/3869483750 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d4ea047880 con 0x55d4ea0ebb60 2024-04-26T21:56:02.820 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.817+0000 7f1cdaa00640 10 client.15579 handle_command_reply: tid=0 2024-04-26T21:56:02.820 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.817+0000 7f1cfefd7740 1 client.15579 shutdown 2024-04-26T21:56:02.820 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.817+0000 7f1cfefd7740 1 -- 172.21.15.72:0/3575714574 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d4ea0dbbb0 msgr2=0x55d4ea0e74f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:02.820 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.817+0000 7f1cfefd7740 1 --2- 172.21.15.72:0/3575714574 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d4ea0dbbb0 0x55d4ea0e74f0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f1cec09c8d0 tx=0x7f1cec09ca80 comp rx=0 tx=0).stop 2024-04-26T21:56:02.820 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.817+0000 7f1cfefd7740 1 -- 172.21.15.72:0/3575714574 shutdown_connections 2024-04-26T21:56:02.820 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.817+0000 7f1cfefd7740 1 --2- 172.21.15.72:0/3575714574 >> [v2:172.21.15.76:6832/3869483750,v1:172.21.15.76:6833/3869483750] conn(0x55d4ea0ebb60 0x55d4ea0edfa0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:02.820 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.817+0000 7f1cfefd7740 1 --2- 172.21.15.72:0/3575714574 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d4ea0dbbb0 0x55d4ea0e74f0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:02.820 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.817+0000 7f1cfefd7740 1 --2- 172.21.15.72:0/3575714574 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d4ea0d6a40 0x55d4ea0e04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:02.820 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.817+0000 7f1cfefd7740 1 --2- 172.21.15.72:0/3575714574 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d4ea0d6050 0x55d4ea0dff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:02.820 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.817+0000 7f1cfefd7740 1 -- 172.21.15.72:0/3575714574 >> 172.21.15.72:0/3575714574 conn(0x55d4ea0d16c0 msgr2=0x55d4ea0d07b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:56:02.820 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.817+0000 7f1cdaa00640 0 client.15579 ms_handle_reset on v2:172.21.15.76:6832/3869483750 2024-04-26T21:56:02.820 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.817+0000 7f1cfefd7740 1 -- 172.21.15.72:0/3575714574 shutdown_connections 2024-04-26T21:56:02.820 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.817+0000 7f1cfefd7740 1 -- 172.21.15.72:0/3575714574 wait complete. 2024-04-26T21:56:02.821 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.817+0000 7f1cfefd7740 20 client.15579 trim_cache size 0 max 16384 2024-04-26T21:56:02.836 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.832+0000 7f1cfefd7740 1 Processor -- start 2024-04-26T21:56:02.836 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.832+0000 7f1cfefd7740 1 -- start start 2024-04-26T21:56:02.836 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.832+0000 7f1cfefd7740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d4ea0d6050 0x55d4ea0df590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:02.836 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.832+0000 7f1cfefd7740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d4ea0d6a40 0x55d4ea0e74b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:02.836 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.832+0000 7f1cfefd7740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d4ea0dbbb0 0x55d4ea0d4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:02.836 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.832+0000 7f1cfefd7740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f1cf41abf90 con 0x55d4ea0d6a40 2024-04-26T21:56:02.837 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.832+0000 7f1cfefd7740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f1cf41adad0 con 0x55d4ea0dbbb0 2024-04-26T21:56:02.837 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.832+0000 7f1cfefd7740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f1cf41ac660 con 0x55d4ea0d6050 2024-04-26T21:56:02.837 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.833+0000 7f1cf8c00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d4ea0d6a40 0x55d4ea0e74b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:02.837 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.833+0000 7f1cf8c00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d4ea0d6a40 0x55d4ea0e74b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:37468/0 (socket says 172.21.15.72:37468) 2024-04-26T21:56:02.837 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.833+0000 7f1cf8c00640 1 -- 172.21.15.72:0/924848524 learned_addr learned my addr 172.21.15.72:0/924848524 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:56:02.837 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.833+0000 7f1cf9600640 1 --2- 172.21.15.72:0/924848524 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d4ea0d6050 0x55d4ea0df590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:02.837 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.833+0000 7f1cfa000640 1 --2- 172.21.15.72:0/924848524 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d4ea0dbbb0 0x55d4ea0d4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:02.837 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.833+0000 7f1cf8c00640 1 -- 172.21.15.72:0/924848524 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d4ea0d6050 msgr2=0x55d4ea0df590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:56:02.837 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.833+0000 7f1cf8c00640 1 --2- 172.21.15.72:0/924848524 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d4ea0d6050 0x55d4ea0df590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:02.837 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.833+0000 7f1cf8c00640 1 -- 172.21.15.72:0/924848524 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d4ea0dbbb0 msgr2=0x55d4ea0d4cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:56:02.837 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.833+0000 7f1cf8c00640 1 --2- 172.21.15.72:0/924848524 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d4ea0dbbb0 0x55d4ea0d4cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:02.838 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.833+0000 7f1cf8c00640 1 -- 172.21.15.72:0/924848524 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d4e9e157d0 con 0x55d4ea0d6a40 2024-04-26T21:56:02.838 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.833+0000 7f1cfa000640 1 --2- 172.21.15.72:0/924848524 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d4ea0dbbb0 0x55d4ea0d4cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:56:02.838 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.833+0000 7f1cf8c00640 1 --2- 172.21.15.72:0/924848524 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d4ea0d6a40 0x55d4ea0e74b0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f1ce0009c80 tx=0x7f1ce0009870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db033962754f6f1d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:56:02.838 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.834+0000 7f1ce5c00640 1 -- 172.21.15.72:0/924848524 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ce0016040 con 0x55d4ea0d6a40 2024-04-26T21:56:02.838 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.834+0000 7f1ce5c00640 1 -- 172.21.15.72:0/924848524 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1ce00092a0 con 0x55d4ea0d6a40 2024-04-26T21:56:02.838 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.834+0000 7f1ce5c00640 1 -- 172.21.15.72:0/924848524 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ce0004db0 con 0x55d4ea0d6a40 2024-04-26T21:56:02.838 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.834+0000 7f1cfefd7740 1 -- 172.21.15.72:0/924848524 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d4ea0d6a40 msgr2=0x55d4ea0e74b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:02.838 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.834+0000 7f1cfefd7740 1 --2- 172.21.15.72:0/924848524 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d4ea0d6a40 0x55d4ea0e74b0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f1ce0009c80 tx=0x7f1ce0009870 comp rx=0 tx=0).stop 2024-04-26T21:56:02.838 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.834+0000 7f1cfefd7740 1 -- 172.21.15.72:0/924848524 shutdown_connections 2024-04-26T21:56:02.838 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.834+0000 7f1cfefd7740 1 --2- 172.21.15.72:0/924848524 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d4ea0dbbb0 0x55d4ea0d4cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:02.838 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.834+0000 7f1cfefd7740 1 --2- 172.21.15.72:0/924848524 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d4ea0d6a40 0x55d4ea0e74b0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:02.838 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.834+0000 7f1cfefd7740 1 --2- 172.21.15.72:0/924848524 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d4ea0d6050 0x55d4ea0df590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:02.839 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.834+0000 7f1cfefd7740 1 -- 172.21.15.72:0/924848524 >> 172.21.15.72:0/924848524 conn(0x55d4ea0d16c0 msgr2=0x55d4ea0d45a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:56:02.839 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.834+0000 7f1cfefd7740 1 -- 172.21.15.72:0/924848524 shutdown_connections 2024-04-26T21:56:02.839 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.835+0000 7f1cfefd7740 1 -- 172.21.15.72:0/924848524 wait complete. 2024-04-26T21:56:02.839 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.835+0000 7f1cfefd7740 1 -- start start 2024-04-26T21:56:02.839 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.835+0000 7f1cfefd7740 1 Processor -- start 2024-04-26T21:56:02.839 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.835+0000 7f1cfefd7740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d4ea0d6050 0x55d4ea0e43a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:02.839 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.835+0000 7f1cfefd7740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d4ea0d6a40 0x55d4ea0e48e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:02.839 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.835+0000 7f1cfefd7740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d4ea0dbbb0 0x55d4ea1500d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:02.839 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.836+0000 7f1cfefd7740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f1cf41ad400 con 0x55d4ea0d6050 2024-04-26T21:56:02.839 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.836+0000 7f1cfefd7740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f1cf41acd30 con 0x55d4ea0dbbb0 2024-04-26T21:56:02.839 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.836+0000 7f1cfefd7740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f1cf41a9f70 con 0x55d4ea0d6a40 2024-04-26T21:56:02.840 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.836+0000 7f1cfefd7740 10 client.0 _ll_register_callbacks cb 0x7f1cec098c60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:56:02.840 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.836+0000 7f1cf9600640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d4ea0d6050 0x55d4ea0e43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:02.840 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.836+0000 7f1cf9600640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d4ea0d6050 0x55d4ea0e43a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:37476/0 (socket says 172.21.15.72:37476) 2024-04-26T21:56:02.840 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.836+0000 7f1cf9600640 1 -- 172.21.15.72:0/2597358569 learned_addr learned my addr 172.21.15.72:0/2597358569 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:56:02.840 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.836+0000 7f1cf9600640 1 -- 172.21.15.72:0/2597358569 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d4ea0d6a40 msgr2=0x55d4ea0e48e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:56:02.840 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.836+0000 7f1cf9600640 1 --2- 172.21.15.72:0/2597358569 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d4ea0d6a40 0x55d4ea0e48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:02.840 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.836+0000 7f1cf9600640 1 -- 172.21.15.72:0/2597358569 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d4ea0dbbb0 msgr2=0x55d4ea1500d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:56:02.840 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.836+0000 7f1cfa000640 1 --2- 172.21.15.72:0/2597358569 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d4ea0dbbb0 0x55d4ea1500d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:02.840 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.836+0000 7f1cf9600640 1 --2- 172.21.15.72:0/2597358569 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d4ea0dbbb0 0x55d4ea1500d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:02.840 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.836+0000 7f1cf9600640 1 -- 172.21.15.72:0/2597358569 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d4e9fe02a0 con 0x55d4ea0d6050 2024-04-26T21:56:02.843 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.836+0000 7f1cf9600640 1 --2- 172.21.15.72:0/2597358569 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d4ea0d6050 0x55d4ea0e43a0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f1ce8003610 tx=0x7f1ce8009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a791617bd4eb1945 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:56:02.843 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.836+0000 7f1cdaa00640 10 client.0 ms_handle_connect on v2:172.21.15.72:3300/0 2024-04-26T21:56:02.843 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.837+0000 7f1cdaa00640 1 -- 172.21.15.72:0/2597358569 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ce8035050 con 0x55d4ea0d6050 2024-04-26T21:56:02.843 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.837+0000 7f1cdaa00640 1 -- 172.21.15.72:0/2597358569 <== mon.0 v2:172.21.15.72:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7f1ce8051450 con 0x55d4ea0d6050 2024-04-26T21:56:02.843 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.837+0000 7f1cfefd7740 1 -- 172.21.15.72:0/2597358569 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ce00092a0 con 0x55d4ea0d6050 2024-04-26T21:56:02.843 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.837+0000 7f1cdaa00640 1 -- 172.21.15.72:0/2597358569 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1ce8061050 con 0x55d4ea0d6050 2024-04-26T21:56:02.843 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.837+0000 7f1cfefd7740 10 client.15585 fetch_fsmap learned FSMap version 27 2024-04-26T21:56:02.843 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.837+0000 7f1cfefd7740 1 -- 172.21.15.72:0/2597358569 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d4ea1516d0 con 0x55d4ea0d6050 2024-04-26T21:56:02.843 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.837+0000 7f1cdaa00640 1 -- 172.21.15.72:0/2597358569 <== mon.0 v2:172.21.15.72:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1ce00092a0 con 0x55d4ea0d6050 2024-04-26T21:56:02.843 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.837+0000 7f1cdaa00640 1 -- 172.21.15.72:0/2597358569 <== mon.0 v2:172.21.15.72:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f1ce8003bf0 con 0x55d4ea0d6050 2024-04-26T21:56:02.843 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.837+0000 7f1cfefd7740 10 client.15585 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T21:56:02.844 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.838+0000 7f1cfefd7740 10 client.15585 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-26T21:56:02.844 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.838+0000 7f1cfefd7740 20 client.15585 populate_metadata read hostname 'smithi072' 2024-04-26T21:56:02.844 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.838+0000 7f1cfefd7740 1 --2- 172.21.15.72:0/2597358569 >> [v2:172.21.15.76:6832/3869483750,v1:172.21.15.76:6833/3869483750] conn(0x55d4ea152060 0x55d4ea1544a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:02.844 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.838+0000 7f1cfefd7740 4 client.15585 mds_command: new command op to 34406 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-26T21:56:02.844 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.838+0000 7f1cfefd7740 1 -- 172.21.15.72:0/2597358569 --> [v2:172.21.15.76:6832/3869483750,v1:172.21.15.76:6833/3869483750] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d4ea0416f0 con 0x55d4ea152060 2024-04-26T21:56:02.844 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.838+0000 7f1cf8c00640 1 --2- 172.21.15.72:0/2597358569 >> [v2:172.21.15.76:6832/3869483750,v1:172.21.15.76:6833/3869483750] conn(0x55d4ea152060 0x55d4ea1544a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:02.844 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.839+0000 7f1cdaa00640 10 client.15585 ms_handle_connect on v2:172.21.15.76:6832/3869483750 2024-04-26T21:56:02.844 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.839+0000 7f1cf8c00640 1 --2- 172.21.15.72:0/2597358569 >> [v2:172.21.15.76:6832/3869483750,v1:172.21.15.76:6833/3869483750] conn(0x55d4ea152060 0x55d4ea1544a0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=352a6ac692e72ab6 server_cookie=b0d1384c2c4bec87 in_seq=0 out_seq=0 2024-04-26T21:56:02.844 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.840+0000 7f1cdaa00640 1 -- 172.21.15.72:0/2597358569 <== mds.3 v2:172.21.15.76:6832/3869483750 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d4ea0416f0 con 0x55d4ea152060 2024-04-26T21:56:02.844 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.840+0000 7f1cdaa00640 10 client.15585 handle_command_reply: tid=0 2024-04-26T21:56:02.844 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.840+0000 7f1cfefd7740 1 client.15585 shutdown 2024-04-26T21:56:02.844 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.840+0000 7f1cfefd7740 1 -- 172.21.15.72:0/2597358569 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d4ea0d6050 msgr2=0x55d4ea0e43a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:02.844 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.840+0000 7f1cfefd7740 1 --2- 172.21.15.72:0/2597358569 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d4ea0d6050 0x55d4ea0e43a0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f1ce8003610 tx=0x7f1ce8009260 comp rx=0 tx=0).stop 2024-04-26T21:56:02.844 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.840+0000 7f1cfefd7740 1 -- 172.21.15.72:0/2597358569 shutdown_connections 2024-04-26T21:56:02.845 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.840+0000 7f1cfefd7740 1 --2- 172.21.15.72:0/2597358569 >> [v2:172.21.15.76:6832/3869483750,v1:172.21.15.76:6833/3869483750] conn(0x55d4ea152060 0x55d4ea1544a0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:02.845 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.840+0000 7f1cfefd7740 1 --2- 172.21.15.72:0/2597358569 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d4ea0dbbb0 0x55d4ea1500d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:02.845 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.840+0000 7f1cfefd7740 1 --2- 172.21.15.72:0/2597358569 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d4ea0d6a40 0x55d4ea0e48e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:02.845 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.840+0000 7f1cfefd7740 1 --2- 172.21.15.72:0/2597358569 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d4ea0d6050 0x55d4ea0e43a0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:02.845 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.840+0000 7f1cdaa00640 0 client.15585 ms_handle_reset on v2:172.21.15.76:6832/3869483750 2024-04-26T21:56:02.845 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.840+0000 7f1cfefd7740 1 -- 172.21.15.72:0/2597358569 >> 172.21.15.72:0/2597358569 conn(0x55d4ea0d16c0 msgr2=0x55d4ea0cff20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:56:02.845 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.840+0000 7f1cfefd7740 1 -- 172.21.15.72:0/2597358569 shutdown_connections 2024-04-26T21:56:02.845 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.840+0000 7f1cfefd7740 1 -- 172.21.15.72:0/2597358569 wait complete. 2024-04-26T21:56:02.845 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.840+0000 7f1cfefd7740 20 client.15585 trim_cache size 0 max 16384 2024-04-26T21:56:02.845 INFO:teuthology.orchestra.run.smithi072.stdout:[] 2024-04-26T21:56:02.846 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.841+0000 7f1cfaa00640 1 -- 172.21.15.72:0/3049385695 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f1cdc0a3cc0 msgr2=0x7f1cdc0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:02.846 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.841+0000 7f1cfaa00640 1 --2- 172.21.15.72:0/3049385695 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f1cdc0a3cc0 0x7f1cdc0a6180 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f1ce8009230 tx=0x7f1ce8005a20 comp rx=0 tx=0).stop 2024-04-26T21:56:02.846 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.841+0000 7f1cfaa00640 1 -- 172.21.15.72:0/3049385695 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1cf4104410 msgr2=0x7f1cf4075260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:02.846 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.841+0000 7f1cfaa00640 1 --2- 172.21.15.72:0/3049385695 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1cf4104410 0x7f1cf4075260 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f1cec00eaa0 tx=0x7f1cec00ef60 comp rx=0 tx=0).stop 2024-04-26T21:56:02.846 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.841+0000 7f1cfaa00640 1 -- 172.21.15.72:0/3049385695 shutdown_connections 2024-04-26T21:56:02.847 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.841+0000 7f1cfaa00640 1 --2- 172.21.15.72:0/3049385695 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1cf4104410 0x7f1cf4075260 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:02.847 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.841+0000 7f1cfaa00640 1 --2- 172.21.15.72:0/3049385695 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f1cdc0a3cc0 0x7f1cdc0a6180 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:02.847 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.841+0000 7f1cfaa00640 1 --2- 172.21.15.72:0/3049385695 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1cf4103a50 0x7f1cf4074d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:02.847 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.841+0000 7f1cfaa00640 1 --2- 172.21.15.72:0/3049385695 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1cf4102860 0x7f1cf4076690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:02.847 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.841+0000 7f1cfaa00640 1 -- 172.21.15.72:0/3049385695 >> 172.21.15.72:0/3049385695 conn(0x7f1cf40fddd0 msgr2=0x7f1cf4106ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:56:02.847 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.841+0000 7f1cfaa00640 1 -- 172.21.15.72:0/3049385695 shutdown_connections 2024-04-26T21:56:02.847 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:02.841+0000 7f1cfaa00640 1 -- 172.21.15.72:0/3049385695 wait complete. 2024-04-26T21:56:02.860 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-26T21:56:03.056 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.055+0000 7f5272000640 1 -- 172.21.15.72:0/2661978520 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f526c108e70 msgr2=0x7f526c109290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:03.056 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.055+0000 7f5272000640 1 --2- 172.21.15.72:0/2661978520 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f526c108e70 0x7f526c109290 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f525c009900 tx=0x7f525c02a410 comp rx=0 tx=0).stop 2024-04-26T21:56:03.056 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.055+0000 7f5272000640 1 -- 172.21.15.72:0/2661978520 shutdown_connections 2024-04-26T21:56:03.056 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.055+0000 7f5272000640 1 --2- 172.21.15.72:0/2661978520 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f526c10a980 0x7f526c1112a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:03.056 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.056+0000 7f5272000640 1 --2- 172.21.15.72:0/2661978520 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f526c109fc0 0x7f526c10a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:03.056 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.056+0000 7f5272000640 1 --2- 172.21.15.72:0/2661978520 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f526c108e70 0x7f526c109290 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:03.056 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.056+0000 7f5272000640 1 -- 172.21.15.72:0/2661978520 >> 172.21.15.72:0/2661978520 conn(0x7f526c077e00 msgr2=0x7f526c07a240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:56:03.057 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.056+0000 7f5272000640 1 -- 172.21.15.72:0/2661978520 shutdown_connections 2024-04-26T21:56:03.057 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.056+0000 7f5272000640 1 -- 172.21.15.72:0/2661978520 wait complete. 2024-04-26T21:56:03.059 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.056+0000 7f5272000640 1 Processor -- start 2024-04-26T21:56:03.059 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.057+0000 7f5272000640 1 -- start start 2024-04-26T21:56:03.059 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.057+0000 7f5272000640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f526c108e70 0x7f526c19ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:03.059 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.057+0000 7f5272000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f526c109fc0 0x7f526c19f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:03.059 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.057+0000 7f5272000640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f526c10a980 0x7f526c1a64f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:03.059 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.057+0000 7f5272000640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f526c1148a0 con 0x7f526c10a980 2024-04-26T21:56:03.059 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.057+0000 7f5272000640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f526c114720 con 0x7f526c108e70 2024-04-26T21:56:03.059 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.057+0000 7f5272000640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f526c114a20 con 0x7f526c109fc0 2024-04-26T21:56:03.059 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.057+0000 7f5271600640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f526c10a980 0x7f526c1a64f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:03.060 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.057+0000 7f5271600640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f526c10a980 0x7f526c1a64f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:37496/0 (socket says 172.21.15.72:37496) 2024-04-26T21:56:03.060 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.057+0000 7f5271600640 1 -- 172.21.15.72:0/183395915 learned_addr learned my addr 172.21.15.72:0/183395915 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:56:03.060 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.058+0000 7f526be00640 1 --2- 172.21.15.72:0/183395915 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f526c109fc0 0x7f526c19f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:03.060 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.058+0000 7f5270c00640 1 --2- 172.21.15.72:0/183395915 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f526c108e70 0x7f526c19ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:03.060 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.058+0000 7f5271600640 1 -- 172.21.15.72:0/183395915 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f526c109fc0 msgr2=0x7f526c19f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:03.060 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.058+0000 7f5271600640 1 --2- 172.21.15.72:0/183395915 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f526c109fc0 0x7f526c19f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:03.060 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.058+0000 7f5271600640 1 -- 172.21.15.72:0/183395915 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f526c108e70 msgr2=0x7f526c19ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:03.060 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.058+0000 7f5271600640 1 --2- 172.21.15.72:0/183395915 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f526c108e70 0x7f526c19ef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:03.060 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.058+0000 7f5271600640 1 -- 172.21.15.72:0/183395915 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f526c1a6b60 con 0x7f526c10a980 2024-04-26T21:56:03.060 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.058+0000 7f526be00640 1 --2- 172.21.15.72:0/183395915 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f526c109fc0 0x7f526c19f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:56:03.060 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.058+0000 7f5271600640 1 --2- 172.21.15.72:0/183395915 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f526c10a980 0x7f526c1a64f0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f526400c970 tx=0x7f526400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:56:03.060 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.058+0000 7f5269600640 1 -- 172.21.15.72:0/183395915 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5264007bb0 con 0x7f526c10a980 2024-04-26T21:56:03.060 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.058+0000 7f5269600640 1 -- 172.21.15.72:0/183395915 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5264007d50 con 0x7f526c10a980 2024-04-26T21:56:03.060 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.058+0000 7f5272000640 1 -- 172.21.15.72:0/183395915 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f526c1a6e50 con 0x7f526c10a980 2024-04-26T21:56:03.061 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.058+0000 7f5269600640 1 -- 172.21.15.72:0/183395915 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5264004d50 con 0x7f526c10a980 2024-04-26T21:56:03.061 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.058+0000 7f5272000640 1 -- 172.21.15.72:0/183395915 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f526c1a72c0 con 0x7f526c10a980 2024-04-26T21:56:03.062 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.062+0000 7f527657d740 1 Processor -- start 2024-04-26T21:56:03.063 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.063+0000 7f5269600640 1 -- 172.21.15.72:0/183395915 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f5264004f70 con 0x7f526c10a980 2024-04-26T21:56:03.063 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.063+0000 7f527657d740 1 -- start start 2024-04-26T21:56:03.065 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.063+0000 7f527657d740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e3815a0050 0x55e3815a0470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:03.065 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.063+0000 7f527657d740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e3815a0a40 0x55e3815a5540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:03.065 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.063+0000 7f527657d740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e3815a5bb0 0x55e3815a8040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:03.065 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.063+0000 7f527657d740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55e381503c70 con 0x55e3815a0050 2024-04-26T21:56:03.065 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.063+0000 7f527657d740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55e3814bc980 con 0x55e3815a5bb0 2024-04-26T21:56:03.065 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.063+0000 7f527657d740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55e3814ffc30 con 0x55e3815a0a40 2024-04-26T21:56:03.066 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.064+0000 7f526be00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e3815a0050 0x55e3815a0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:03.066 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.064+0000 7f526be00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e3815a0050 0x55e3815a0470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:37502/0 (socket says 172.21.15.72:37502) 2024-04-26T21:56:03.066 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.064+0000 7f526be00640 1 -- 172.21.15.72:0/2598338106 learned_addr learned my addr 172.21.15.72:0/2598338106 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:56:03.066 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.064+0000 7f5270c00640 1 --2- 172.21.15.72:0/2598338106 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e3815a0a40 0x55e3815a5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:03.066 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.064+0000 7f526be00640 1 -- 172.21.15.72:0/2598338106 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e3815a0a40 msgr2=0x55e3815a5540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:56:03.066 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.064+0000 7f526be00640 1 --2- 172.21.15.72:0/2598338106 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e3815a0a40 0x55e3815a5540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:03.066 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.064+0000 7f526be00640 1 -- 172.21.15.72:0/2598338106 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e3815a5bb0 msgr2=0x55e3815a8040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:56:03.066 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.064+0000 7f526be00640 1 --2- 172.21.15.72:0/2598338106 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e3815a5bb0 0x55e3815a8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:03.066 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.064+0000 7f526be00640 1 -- 172.21.15.72:0/2598338106 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e3814947b0 con 0x55e3815a0050 2024-04-26T21:56:03.066 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.064+0000 7f5269600640 1 --2- 172.21.15.72:0/183395915 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f52500a3c40 0x7f52500a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:03.066 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.065+0000 7f5269600640 1 -- 172.21.15.72:0/183395915 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7f52640d03d0 con 0x7f526c10a980 2024-04-26T21:56:03.066 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.065+0000 7f5270c00640 1 --2- 172.21.15.72:0/183395915 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f52500a3c40 0x7f52500a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:03.066 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.065+0000 7f526be00640 1 --2- 172.21.15.72:0/2598338106 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e3815a0050 0x55e3815a0470 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f5258008f50 tx=0x7f525800f7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e4a071923098130 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:56:03.067 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.065+0000 7f5261c00640 1 -- 172.21.15.72:0/2598338106 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5258003b70 con 0x55e3815a0050 2024-04-26T21:56:03.067 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.065+0000 7f5261c00640 1 -- 172.21.15.72:0/2598338106 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5258003d10 con 0x55e3815a0050 2024-04-26T21:56:03.067 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.065+0000 7f5261c00640 1 -- 172.21.15.72:0/2598338106 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5258004440 con 0x55e3815a0050 2024-04-26T21:56:03.070 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.066+0000 7f527657d740 1 -- 172.21.15.72:0/2598338106 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e3815a0050 msgr2=0x55e3815a0470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:03.070 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.066+0000 7f5270c00640 1 --2- 172.21.15.72:0/183395915 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f52500a3c40 0x7f52500a6100 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f525c009230 tx=0x7f525c031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:56:03.071 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.066+0000 7f527657d740 1 --2- 172.21.15.72:0/2598338106 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e3815a0050 0x55e3815a0470 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f5258008f50 tx=0x7f525800f7f0 comp rx=0 tx=0).stop 2024-04-26T21:56:03.071 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.066+0000 7f527657d740 1 -- 172.21.15.72:0/2598338106 shutdown_connections 2024-04-26T21:56:03.071 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.066+0000 7f527657d740 1 --2- 172.21.15.72:0/2598338106 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e3815a5bb0 0x55e3815a8040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:03.071 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.066+0000 7f527657d740 1 --2- 172.21.15.72:0/2598338106 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e3815a0a40 0x55e3815a5540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:03.071 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.066+0000 7f527657d740 1 --2- 172.21.15.72:0/2598338106 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e3815a0050 0x55e3815a0470 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:03.071 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.066+0000 7f527657d740 1 -- 172.21.15.72:0/2598338106 >> 172.21.15.72:0/2598338106 conn(0x55e38159b6c0 msgr2=0x55e38159bac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:56:03.071 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.066+0000 7f527657d740 1 -- 172.21.15.72:0/2598338106 shutdown_connections 2024-04-26T21:56:03.071 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.066+0000 7f527657d740 1 -- 172.21.15.72:0/2598338106 wait complete. 2024-04-26T21:56:03.071 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.066+0000 7f527657d740 1 -- start start 2024-04-26T21:56:03.071 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.067+0000 7f527657d740 1 Processor -- start 2024-04-26T21:56:03.071 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.067+0000 7f527657d740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e3815a0050 0x55e3815a9f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:03.071 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.067+0000 7f527657d740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e3815a0a40 0x55e3815aa490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:03.071 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.067+0000 7f527657d740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e3815a5bb0 0x55e3815b14d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:03.072 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.067+0000 7f527657d740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55e381502f80 con 0x55e3815a0050 2024-04-26T21:56:03.072 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.067+0000 7f527657d740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55e3814b0260 con 0x55e3815a0a40 2024-04-26T21:56:03.072 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.067+0000 7f527657d740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55e3815082a0 con 0x55e3815a5bb0 2024-04-26T21:56:03.072 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.067+0000 7f527657d740 10 client.0 _ll_register_callbacks cb 0x55e3814fa960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:56:03.072 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.067+0000 7f5270c00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e3815a0050 0x55e3815a9f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:03.072 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.067+0000 7f5270c00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e3815a0050 0x55e3815a9f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:37510/0 (socket says 172.21.15.72:37510) 2024-04-26T21:56:03.072 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.067+0000 7f5270c00640 1 -- 172.21.15.72:0/905556415 learned_addr learned my addr 172.21.15.72:0/905556415 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:56:03.072 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.068+0000 7f5271600640 1 --2- 172.21.15.72:0/905556415 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e3815a5bb0 0x55e3815b14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:03.072 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.068+0000 7f526be00640 1 --2- 172.21.15.72:0/905556415 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e3815a0a40 0x55e3815aa490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:03.072 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.068+0000 7f5270c00640 1 -- 172.21.15.72:0/905556415 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e3815a5bb0 msgr2=0x55e3815b14d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:56:03.072 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.068+0000 7f5270c00640 1 --2- 172.21.15.72:0/905556415 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e3815a5bb0 0x55e3815b14d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:03.072 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.068+0000 7f5270c00640 1 -- 172.21.15.72:0/905556415 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e3815a0a40 msgr2=0x55e3815aa490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:56:03.072 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.068+0000 7f5270c00640 1 --2- 172.21.15.72:0/905556415 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e3815a0a40 0x55e3815aa490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:03.072 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.068+0000 7f5270c00640 1 -- 172.21.15.72:0/905556415 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e3814921c0 con 0x55e3815a0050 2024-04-26T21:56:03.072 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.068+0000 7f5270c00640 1 --2- 172.21.15.72:0/905556415 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e3815a0050 0x55e3815a9f50 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f525c02a930 tx=0x7f525c02a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=664abd94d09f7b98 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:56:03.073 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.068+0000 7f5256000640 10 client.0 ms_handle_connect on v2:172.21.15.72:3300/0 2024-04-26T21:56:03.073 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.069+0000 7f5256000640 1 -- 172.21.15.72:0/905556415 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f525c04d020 con 0x55e3815a0050 2024-04-26T21:56:03.073 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.069+0000 7f5256000640 1 -- 172.21.15.72:0/905556415 <== mon.0 v2:172.21.15.72:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7f525c003c80 con 0x55e3815a0050 2024-04-26T21:56:03.073 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.069+0000 7f527657d740 1 -- 172.21.15.72:0/905556415 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5258003d10 con 0x55e3815a0050 2024-04-26T21:56:03.073 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.069+0000 7f5256000640 1 -- 172.21.15.72:0/905556415 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f525c058050 con 0x55e3815a0050 2024-04-26T21:56:03.073 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.069+0000 7f527657d740 10 client.15597 fetch_fsmap learned FSMap version 27 2024-04-26T21:56:03.073 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.069+0000 7f527657d740 1 -- 172.21.15.72:0/905556415 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55e3812153f0 con 0x55e3815a0050 2024-04-26T21:56:03.073 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.069+0000 7f5256000640 1 -- 172.21.15.72:0/905556415 <== mon.0 v2:172.21.15.72:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5258003d10 con 0x55e3815a0050 2024-04-26T21:56:03.073 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.069+0000 7f5256000640 1 -- 172.21.15.72:0/905556415 <== mon.0 v2:172.21.15.72:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f525c049480 con 0x55e3815a0050 2024-04-26T21:56:03.073 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.070+0000 7f527657d740 10 client.15597 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T21:56:03.073 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.070+0000 7f527657d740 10 client.15597 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-26T21:56:03.073 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.070+0000 7f527657d740 20 client.15597 populate_metadata read hostname 'smithi072' 2024-04-26T21:56:03.074 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.070+0000 7f527657d740 1 --2- 172.21.15.72:0/905556415 >> [v2:172.21.15.76:6834/3517429435,v1:172.21.15.76:6835/3517429435] conn(0x55e3815b5ae0 0x55e3815b7f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:03.074 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.070+0000 7f527657d740 4 client.15597 mds_command: new command op to 34442 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:56:03.074 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.070+0000 7f527657d740 1 -- 172.21.15.72:0/905556415 --> [v2:172.21.15.76:6834/3517429435,v1:172.21.15.76:6835/3517429435] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e381511880 con 0x55e3815b5ae0 2024-04-26T21:56:03.074 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.070+0000 7f526be00640 1 --2- 172.21.15.72:0/905556415 >> [v2:172.21.15.76:6834/3517429435,v1:172.21.15.76:6835/3517429435] conn(0x55e3815b5ae0 0x55e3815b7f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:03.075 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.071+0000 7f526be00640 1 --2- 172.21.15.72:0/905556415 >> [v2:172.21.15.76:6834/3517429435,v1:172.21.15.76:6835/3517429435] conn(0x55e3815b5ae0 0x55e3815b7f80 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a92bb7bd45ab8e59 server_cookie=feff7afff1ce65b3 in_seq=0 out_seq=0 2024-04-26T21:56:03.075 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.071+0000 7f5256000640 10 client.15597 ms_handle_connect on v2:172.21.15.76:6834/3517429435 2024-04-26T21:56:03.075 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.073+0000 7f5256000640 1 -- 172.21.15.72:0/905556415 <== mds.2 v2:172.21.15.76:6834/3517429435 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e381511880 con 0x55e3815b5ae0 2024-04-26T21:56:03.075 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.073+0000 7f5256000640 10 client.15597 handle_command_reply: tid=0 2024-04-26T21:56:03.075 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.073+0000 7f527657d740 1 client.15597 shutdown 2024-04-26T21:56:03.075 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.074+0000 7f527657d740 1 -- 172.21.15.72:0/905556415 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e3815a0050 msgr2=0x55e3815a9f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:03.076 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.074+0000 7f527657d740 1 --2- 172.21.15.72:0/905556415 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e3815a0050 0x55e3815a9f50 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f525c02a930 tx=0x7f525c02a410 comp rx=0 tx=0).stop 2024-04-26T21:56:03.076 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.074+0000 7f527657d740 1 -- 172.21.15.72:0/905556415 shutdown_connections 2024-04-26T21:56:03.076 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.074+0000 7f527657d740 1 --2- 172.21.15.72:0/905556415 >> [v2:172.21.15.76:6834/3517429435,v1:172.21.15.76:6835/3517429435] conn(0x55e3815b5ae0 0x55e3815b7f80 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:03.076 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.074+0000 7f527657d740 1 --2- 172.21.15.72:0/905556415 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e3815a5bb0 0x55e3815b14d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:03.076 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.074+0000 7f527657d740 1 --2- 172.21.15.72:0/905556415 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e3815a0a40 0x55e3815aa490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:03.076 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.074+0000 7f527657d740 1 --2- 172.21.15.72:0/905556415 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e3815a0050 0x55e3815a9f50 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:03.076 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.074+0000 7f5256000640 0 client.15597 ms_handle_reset on v2:172.21.15.76:6834/3517429435 2024-04-26T21:56:03.076 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.074+0000 7f527657d740 1 -- 172.21.15.72:0/905556415 >> 172.21.15.72:0/905556415 conn(0x55e38159b6c0 msgr2=0x55e38159a7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:56:03.076 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.074+0000 7f527657d740 1 -- 172.21.15.72:0/905556415 shutdown_connections 2024-04-26T21:56:03.076 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.074+0000 7f527657d740 1 -- 172.21.15.72:0/905556415 wait complete. 2024-04-26T21:56:03.076 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.074+0000 7f527657d740 20 client.15597 trim_cache size 0 max 16384 2024-04-26T21:56:03.091 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.089+0000 7f527657d740 1 Processor -- start 2024-04-26T21:56:03.091 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.089+0000 7f527657d740 1 -- start start 2024-04-26T21:56:03.091 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.089+0000 7f527657d740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e3815a0050 0x55e3815a14b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:03.092 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.089+0000 7f527657d740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e3815a0a40 0x55e3815b1490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:03.092 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.089+0000 7f527657d740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e3815a5bb0 0x55e3815a19f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:03.092 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.089+0000 7f527657d740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f526c1b02c0 con 0x55e3815a0050 2024-04-26T21:56:03.092 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.089+0000 7f527657d740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f526c1b1e00 con 0x55e3815a0a40 2024-04-26T21:56:03.092 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.089+0000 7f527657d740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f526c1b0990 con 0x55e3815a5bb0 2024-04-26T21:56:03.092 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.089+0000 7f5270c00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e3815a0050 0x55e3815a14b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:03.092 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.089+0000 7f5270c00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e3815a0050 0x55e3815a14b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:37520/0 (socket says 172.21.15.72:37520) 2024-04-26T21:56:03.092 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.089+0000 7f5270c00640 1 -- 172.21.15.72:0/2599837288 learned_addr learned my addr 172.21.15.72:0/2599837288 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:56:03.092 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.090+0000 7f5270c00640 1 -- 172.21.15.72:0/2599837288 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e3815a5bb0 msgr2=0x55e3815a19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:56:03.093 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.090+0000 7f5270c00640 1 --2- 172.21.15.72:0/2599837288 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e3815a5bb0 0x55e3815a19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:03.093 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.090+0000 7f5270c00640 1 -- 172.21.15.72:0/2599837288 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e3815a0a40 msgr2=0x55e3815b1490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:56:03.093 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.090+0000 7f5270c00640 1 --2- 172.21.15.72:0/2599837288 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e3815a0a40 0x55e3815b1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:03.093 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.090+0000 7f5270c00640 1 -- 172.21.15.72:0/2599837288 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e3812df7d0 con 0x55e3815a0050 2024-04-26T21:56:03.093 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.090+0000 7f5270c00640 1 --2- 172.21.15.72:0/2599837288 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e3815a0050 0x55e3815a14b0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f525c0057c0 tx=0x7f525c003730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d968e0fc8375adfc server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:56:03.093 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.090+0000 7f5261200640 1 -- 172.21.15.72:0/2599837288 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f525c060020 con 0x55e3815a0050 2024-04-26T21:56:03.093 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.090+0000 7f5261200640 1 -- 172.21.15.72:0/2599837288 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f525c049d60 con 0x55e3815a0050 2024-04-26T21:56:03.093 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.090+0000 7f5261200640 1 -- 172.21.15.72:0/2599837288 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f525c04d070 con 0x55e3815a0050 2024-04-26T21:56:03.093 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.091+0000 7f527657d740 1 -- 172.21.15.72:0/2599837288 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e3815a0050 msgr2=0x55e3815a14b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:03.093 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.091+0000 7f527657d740 1 --2- 172.21.15.72:0/2599837288 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e3815a0050 0x55e3815a14b0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f525c0057c0 tx=0x7f525c003730 comp rx=0 tx=0).stop 2024-04-26T21:56:03.093 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.091+0000 7f527657d740 1 -- 172.21.15.72:0/2599837288 shutdown_connections 2024-04-26T21:56:03.094 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.091+0000 7f527657d740 1 --2- 172.21.15.72:0/2599837288 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e3815a5bb0 0x55e3815a19f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:03.094 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.091+0000 7f527657d740 1 --2- 172.21.15.72:0/2599837288 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e3815a0a40 0x55e3815b1490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:03.094 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.091+0000 7f527657d740 1 --2- 172.21.15.72:0/2599837288 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e3815a0050 0x55e3815a14b0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:03.094 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.091+0000 7f527657d740 1 -- 172.21.15.72:0/2599837288 >> 172.21.15.72:0/2599837288 conn(0x55e38159b6c0 msgr2=0x55e38159ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:56:03.094 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.091+0000 7f527657d740 1 -- 172.21.15.72:0/2599837288 shutdown_connections 2024-04-26T21:56:03.094 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.091+0000 7f527657d740 1 -- 172.21.15.72:0/2599837288 wait complete. 2024-04-26T21:56:03.094 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.092+0000 7f527657d740 1 -- start start 2024-04-26T21:56:03.098 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.092+0000 7f527657d740 1 Processor -- start 2024-04-26T21:56:03.098 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.092+0000 7f527657d740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e3815a0050 0x55e3815aea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:03.098 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.092+0000 7f527657d740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e3815a0a40 0x55e3815aef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:03.098 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.092+0000 7f527657d740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e3815a5bb0 0x55e3815aff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:03.098 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.092+0000 7f527657d740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f526c1b1730 con 0x55e3815a0a40 2024-04-26T21:56:03.098 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.092+0000 7f527657d740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f526c1b1060 con 0x55e3815a0050 2024-04-26T21:56:03.099 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.092+0000 7f527657d740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f526c1ae2a0 con 0x55e3815a5bb0 2024-04-26T21:56:03.099 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.092+0000 7f527657d740 10 client.0 _ll_register_callbacks cb 0x7f525c040070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:56:03.099 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.093+0000 7f526be00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e3815a0a40 0x55e3815aef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:03.099 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.093+0000 7f526be00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e3815a0a40 0x55e3815aef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:37536/0 (socket says 172.21.15.72:37536) 2024-04-26T21:56:03.099 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.093+0000 7f526be00640 1 -- 172.21.15.72:0/783110276 learned_addr learned my addr 172.21.15.72:0/783110276 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:56:03.099 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.093+0000 7f5270c00640 1 --2- 172.21.15.72:0/783110276 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e3815a0050 0x55e3815aea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:03.099 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.093+0000 7f5271600640 1 --2- 172.21.15.72:0/783110276 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e3815a5bb0 0x55e3815aff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:03.099 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.093+0000 7f526be00640 1 -- 172.21.15.72:0/783110276 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e3815a5bb0 msgr2=0x55e3815aff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:56:03.099 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.093+0000 7f526be00640 1 --2- 172.21.15.72:0/783110276 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e3815a5bb0 0x55e3815aff80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:03.099 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.093+0000 7f526be00640 1 -- 172.21.15.72:0/783110276 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e3815a0050 msgr2=0x55e3815aea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:56:03.099 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.093+0000 7f526be00640 1 --2- 172.21.15.72:0/783110276 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e3815a0050 0x55e3815aea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:03.099 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.093+0000 7f526be00640 1 -- 172.21.15.72:0/783110276 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e3814aa2a0 con 0x55e3815a0a40 2024-04-26T21:56:03.099 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.093+0000 7f526be00640 1 --2- 172.21.15.72:0/783110276 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e3815a0a40 0x55e3815aef70 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f5258008f50 tx=0x7f5258003a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4328826724adb40 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:56:03.099 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.093+0000 7f5256000640 10 client.0 ms_handle_connect on v2:172.21.15.72:3300/0 2024-04-26T21:56:03.100 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.093+0000 7f5256000640 1 -- 172.21.15.72:0/783110276 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5258003d00 con 0x55e3815a0a40 2024-04-26T21:56:03.100 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.094+0000 7f5256000640 1 -- 172.21.15.72:0/783110276 <== mon.0 v2:172.21.15.72:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7f5258011680 con 0x55e3815a0a40 2024-04-26T21:56:03.100 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.094+0000 7f527657d740 1 -- 172.21.15.72:0/783110276 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f525c049d60 con 0x55e3815a0a40 2024-04-26T21:56:03.100 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.094+0000 7f5256000640 1 -- 172.21.15.72:0/783110276 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f5258011880 con 0x55e3815a0a40 2024-04-26T21:56:03.100 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.094+0000 7f527657d740 10 client.15603 fetch_fsmap learned FSMap version 27 2024-04-26T21:56:03.100 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.094+0000 7f527657d740 1 -- 172.21.15.72:0/783110276 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55e3815b1740 con 0x55e3815a0a40 2024-04-26T21:56:03.100 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.094+0000 7f5256000640 1 -- 172.21.15.72:0/783110276 <== mon.0 v2:172.21.15.72:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f525c049d60 con 0x55e3815a0a40 2024-04-26T21:56:03.100 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.094+0000 7f5256000640 1 -- 172.21.15.72:0/783110276 <== mon.0 v2:172.21.15.72:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f5258003f40 con 0x55e3815a0a40 2024-04-26T21:56:03.100 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.094+0000 7f527657d740 10 client.15603 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T21:56:03.100 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.094+0000 7f527657d740 10 client.15603 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-26T21:56:03.100 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.094+0000 7f527657d740 20 client.15603 populate_metadata read hostname 'smithi072' 2024-04-26T21:56:03.100 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.094+0000 7f527657d740 1 --2- 172.21.15.72:0/783110276 >> [v2:172.21.15.76:6834/3517429435,v1:172.21.15.76:6835/3517429435] conn(0x55e38161c670 0x55e38161ea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:03.100 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.094+0000 7f527657d740 4 client.15603 mds_command: new command op to 34442 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-26T21:56:03.101 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.094+0000 7f527657d740 1 -- 172.21.15.72:0/783110276 --> [v2:172.21.15.76:6834/3517429435,v1:172.21.15.76:6835/3517429435] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e38150b6f0 con 0x55e38161c670 2024-04-26T21:56:03.101 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.095+0000 7f5270c00640 1 --2- 172.21.15.72:0/783110276 >> [v2:172.21.15.76:6834/3517429435,v1:172.21.15.76:6835/3517429435] conn(0x55e38161c670 0x55e38161ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:03.101 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.096+0000 7f5270c00640 1 --2- 172.21.15.72:0/783110276 >> [v2:172.21.15.76:6834/3517429435,v1:172.21.15.76:6835/3517429435] conn(0x55e38161c670 0x55e38161ea90 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=cb37fe9ce57d5153 server_cookie=ed2ff8841c0c62b in_seq=0 out_seq=0 2024-04-26T21:56:03.101 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.096+0000 7f5256000640 10 client.15603 ms_handle_connect on v2:172.21.15.76:6834/3517429435 2024-04-26T21:56:03.101 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.097+0000 7f5256000640 1 -- 172.21.15.72:0/783110276 <== mds.2 v2:172.21.15.76:6834/3517429435 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e38150b6f0 con 0x55e38161c670 2024-04-26T21:56:03.101 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.097+0000 7f5256000640 10 client.15603 handle_command_reply: tid=0 2024-04-26T21:56:03.101 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.097+0000 7f527657d740 1 client.15603 shutdown 2024-04-26T21:56:03.101 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.097+0000 7f527657d740 1 -- 172.21.15.72:0/783110276 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e3815a0a40 msgr2=0x55e3815aef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:03.101 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.097+0000 7f527657d740 1 --2- 172.21.15.72:0/783110276 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e3815a0a40 0x55e3815aef70 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f5258008f50 tx=0x7f5258003a00 comp rx=0 tx=0).stop 2024-04-26T21:56:03.101 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.097+0000 7f527657d740 1 -- 172.21.15.72:0/783110276 shutdown_connections 2024-04-26T21:56:03.101 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.097+0000 7f527657d740 1 --2- 172.21.15.72:0/783110276 >> [v2:172.21.15.76:6834/3517429435,v1:172.21.15.76:6835/3517429435] conn(0x55e38161c670 0x55e38161ea90 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:03.101 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.097+0000 7f527657d740 1 --2- 172.21.15.72:0/783110276 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e3815a5bb0 0x55e3815aff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:03.101 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.097+0000 7f527657d740 1 --2- 172.21.15.72:0/783110276 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e3815a0a40 0x55e3815aef70 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:03.102 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.097+0000 7f527657d740 1 --2- 172.21.15.72:0/783110276 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e3815a0050 0x55e3815aea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:03.102 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.097+0000 7f527657d740 1 -- 172.21.15.72:0/783110276 >> 172.21.15.72:0/783110276 conn(0x55e38159b6c0 msgr2=0x55e381599f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:56:03.102 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.097+0000 7f5256000640 0 client.15603 ms_handle_reset on v2:172.21.15.76:6834/3517429435 2024-04-26T21:56:03.102 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.097+0000 7f527657d740 1 -- 172.21.15.72:0/783110276 shutdown_connections 2024-04-26T21:56:03.102 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.097+0000 7f527657d740 1 -- 172.21.15.72:0/783110276 wait complete. 2024-04-26T21:56:03.102 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.097+0000 7f527657d740 20 client.15603 trim_cache size 0 max 16384 2024-04-26T21:56:03.102 INFO:teuthology.orchestra.run.smithi072.stdout:[] 2024-04-26T21:56:03.103 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.098+0000 7f5272000640 1 -- 172.21.15.72:0/183395915 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f52500a3c40 msgr2=0x7f52500a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:03.103 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.098+0000 7f5272000640 1 --2- 172.21.15.72:0/183395915 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f52500a3c40 0x7f52500a6100 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f525c009230 tx=0x7f525c031400 comp rx=0 tx=0).stop 2024-04-26T21:56:03.103 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.098+0000 7f5272000640 1 -- 172.21.15.72:0/183395915 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f526c10a980 msgr2=0x7f526c1a64f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:03.103 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.098+0000 7f5272000640 1 --2- 172.21.15.72:0/183395915 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f526c10a980 0x7f526c1a64f0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f526400c970 tx=0x7f526400ce30 comp rx=0 tx=0).stop 2024-04-26T21:56:03.103 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.098+0000 7f5272000640 1 -- 172.21.15.72:0/183395915 shutdown_connections 2024-04-26T21:56:03.103 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.098+0000 7f5272000640 1 --2- 172.21.15.72:0/183395915 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f526c10a980 0x7f526c1a64f0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:03.103 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.098+0000 7f5272000640 1 --2- 172.21.15.72:0/183395915 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f52500a3c40 0x7f52500a6100 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:03.104 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.098+0000 7f5272000640 1 --2- 172.21.15.72:0/183395915 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f526c109fc0 0x7f526c19f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:03.104 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.098+0000 7f5272000640 1 --2- 172.21.15.72:0/183395915 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f526c108e70 0x7f526c19ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:03.104 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.098+0000 7f5272000640 1 -- 172.21.15.72:0/183395915 >> 172.21.15.72:0/183395915 conn(0x7f526c077e00 msgr2=0x7f526c10d350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:56:03.104 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.098+0000 7f5272000640 1 -- 172.21.15.72:0/183395915 shutdown_connections 2024-04-26T21:56:03.104 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.098+0000 7f5272000640 1 -- 172.21.15.72:0/183395915 wait complete. 2024-04-26T21:56:03.113 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:56:02 smithi072 ceph-mon[28138]: pgmap v467: 97 pgs: 97 active+clean; 650 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:56:03.113 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:56:02 smithi072 ceph-mon[28138]: from='client.? 172.21.15.72:0/824602508' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:56:03.116 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-26T21:56:03.309 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.308+0000 7f6bde200640 1 -- 172.21.15.72:0/533621375 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6bd8105d80 msgr2=0x7f6bd8106200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:03.309 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.308+0000 7f6bde200640 1 --2- 172.21.15.72:0/533621375 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6bd8105d80 0x7f6bd8106200 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f6bc0009900 tx=0x7f6bc002a410 comp rx=0 tx=0).stop 2024-04-26T21:56:03.309 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.309+0000 7f6bde200640 1 -- 172.21.15.72:0/533621375 shutdown_connections 2024-04-26T21:56:03.309 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.309+0000 7f6bde200640 1 --2- 172.21.15.72:0/533621375 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6bd8106740 0x7f6bd810cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:03.310 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.309+0000 7f6bde200640 1 --2- 172.21.15.72:0/533621375 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6bd8105d80 0x7f6bd8106200 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:03.310 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.309+0000 7f6bde200640 1 --2- 172.21.15.72:0/533621375 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6bd8104b90 0x7f6bd8104fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:03.310 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.309+0000 7f6bde200640 1 -- 172.21.15.72:0/533621375 >> 172.21.15.72:0/533621375 conn(0x7f6bd8100120 msgr2=0x7f6bd8102540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:56:03.310 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.309+0000 7f6bde200640 1 -- 172.21.15.72:0/533621375 shutdown_connections 2024-04-26T21:56:03.310 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.309+0000 7f6bde200640 1 -- 172.21.15.72:0/533621375 wait complete. 2024-04-26T21:56:03.312 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.310+0000 7f6bde200640 1 Processor -- start 2024-04-26T21:56:03.312 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.310+0000 7f6bde200640 1 -- start start 2024-04-26T21:56:03.312 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.310+0000 7f6bde200640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6bd8104b90 0x7f6bd819ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:03.312 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.310+0000 7f6bde200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6bd8105d80 0x7f6bd819b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:03.313 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.310+0000 7f6bde200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6bd8106740 0x7f6bd81a22a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:03.313 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.310+0000 7f6bde200640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f6bd81105c0 con 0x7f6bd8104b90 2024-04-26T21:56:03.313 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.310+0000 7f6bde200640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f6bd8110440 con 0x7f6bd8105d80 2024-04-26T21:56:03.313 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.310+0000 7f6bde200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f6bd8110740 con 0x7f6bd8106740 2024-04-26T21:56:03.313 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.311+0000 7f6bd7400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6bd8104b90 0x7f6bd819ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:03.313 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.311+0000 7f6bd7400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6bd8104b90 0x7f6bd819ad80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:37552/0 (socket says 172.21.15.72:37552) 2024-04-26T21:56:03.313 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.311+0000 7f6bd7400640 1 -- 172.21.15.72:0/687785802 learned_addr learned my addr 172.21.15.72:0/687785802 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:56:03.313 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.311+0000 7f6bcea00640 1 --2- 172.21.15.72:0/687785802 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6bd8105d80 0x7f6bd819b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:03.313 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.311+0000 7f6bd7e00640 1 --2- 172.21.15.72:0/687785802 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6bd8106740 0x7f6bd81a22a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:03.314 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.311+0000 7f6bd7400640 1 -- 172.21.15.72:0/687785802 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6bd8106740 msgr2=0x7f6bd81a22a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:03.314 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.311+0000 7f6bd7400640 1 --2- 172.21.15.72:0/687785802 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6bd8106740 0x7f6bd81a22a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:03.314 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.311+0000 7f6bd7400640 1 -- 172.21.15.72:0/687785802 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6bd8105d80 msgr2=0x7f6bd819b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:03.314 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.311+0000 7f6bd7400640 1 --2- 172.21.15.72:0/687785802 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6bd8105d80 0x7f6bd819b2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:03.314 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.311+0000 7f6bd7400640 1 -- 172.21.15.72:0/687785802 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bd81a2940 con 0x7f6bd8104b90 2024-04-26T21:56:03.314 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.311+0000 7f6bcea00640 1 --2- 172.21.15.72:0/687785802 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6bd8105d80 0x7f6bd819b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:56:03.314 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.311+0000 7f6bd7400640 1 --2- 172.21.15.72:0/687785802 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6bd8104b90 0x7f6bd819ad80 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f6bc800e9c0 tx=0x7f6bc800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:56:03.315 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.312+0000 7f6bd4c00640 1 -- 172.21.15.72:0/687785802 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bc800cd80 con 0x7f6bd8104b90 2024-04-26T21:56:03.315 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.312+0000 7f6bd4c00640 1 -- 172.21.15.72:0/687785802 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6bc80035f0 con 0x7f6bd8104b90 2024-04-26T21:56:03.315 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.312+0000 7f6bde200640 1 -- 172.21.15.72:0/687785802 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6bd81a2c30 con 0x7f6bd8104b90 2024-04-26T21:56:03.315 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.312+0000 7f6bd4c00640 1 -- 172.21.15.72:0/687785802 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bc80044c0 con 0x7f6bd8104b90 2024-04-26T21:56:03.315 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.312+0000 7f6bde200640 1 -- 172.21.15.72:0/687785802 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6bd81a3180 con 0x7f6bd8104b90 2024-04-26T21:56:03.316 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:56:02 smithi076 ceph-mon[31823]: pgmap v467: 97 pgs: 97 active+clean; 650 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:56:03.316 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:56:02 smithi076 ceph-mon[31823]: from='client.? 172.21.15.72:0/824602508' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:56:03.327 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.315+0000 7f6bd4c00640 1 -- 172.21.15.72:0/687785802 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f6bc8004000 con 0x7f6bd8104b90 2024-04-26T21:56:03.328 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.315+0000 7f6be1012740 1 Processor -- start 2024-04-26T21:56:03.328 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.315+0000 7f6be1012740 1 -- start start 2024-04-26T21:56:03.328 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.316+0000 7f6be1012740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f5ec366ea0 0x55f5ec3672c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:03.328 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.316+0000 7f6be1012740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f5ec367890 0x55f5ec36c390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:03.328 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.316+0000 7f6be1012740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f5ec36ca00 0x55f5ec36ee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:03.328 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.316+0000 7f6be1012740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55f5ec2cac70 con 0x55f5ec36ca00 2024-04-26T21:56:03.328 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.316+0000 7f6be1012740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55f5ec283980 con 0x55f5ec366ea0 2024-04-26T21:56:03.328 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.316+0000 7f6be1012740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55f5ec2c6c30 con 0x55f5ec367890 2024-04-26T21:56:03.328 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.316+0000 7f6bcea00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f5ec36ca00 0x55f5ec36ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:03.328 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.316+0000 7f6bcea00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f5ec36ca00 0x55f5ec36ee90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:37554/0 (socket says 172.21.15.72:37554) 2024-04-26T21:56:03.328 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.316+0000 7f6bcea00640 1 -- 172.21.15.72:0/2280654256 learned_addr learned my addr 172.21.15.72:0/2280654256 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:56:03.329 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.316+0000 7f6bd7e00640 1 --2- 172.21.15.72:0/2280654256 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f5ec366ea0 0x55f5ec3672c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:03.329 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.316+0000 7f6bd7400640 1 --2- 172.21.15.72:0/2280654256 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f5ec367890 0x55f5ec36c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:03.329 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.317+0000 7f6bcea00640 1 -- 172.21.15.72:0/2280654256 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f5ec367890 msgr2=0x55f5ec36c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:56:03.329 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.317+0000 7f6bcea00640 1 --2- 172.21.15.72:0/2280654256 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f5ec367890 0x55f5ec36c390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:03.329 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.317+0000 7f6bcea00640 1 -- 172.21.15.72:0/2280654256 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f5ec366ea0 msgr2=0x55f5ec3672c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:56:03.329 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.317+0000 7f6bcea00640 1 --2- 172.21.15.72:0/2280654256 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f5ec366ea0 0x55f5ec3672c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:03.329 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.317+0000 7f6bcea00640 1 -- 172.21.15.72:0/2280654256 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f5ec25b7b0 con 0x55f5ec36ca00 2024-04-26T21:56:03.329 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.317+0000 7f6bd7e00640 1 --2- 172.21.15.72:0/2280654256 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f5ec366ea0 0x55f5ec3672c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:56:03.329 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.317+0000 7f6bd7400640 1 --2- 172.21.15.72:0/2280654256 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f5ec367890 0x55f5ec36c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:56:03.329 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.317+0000 7f6bd4c00640 1 --2- 172.21.15.72:0/687785802 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f6bbc0ac4d0 0x7f6bbc0ae990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:03.329 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.317+0000 7f6bd4c00640 1 -- 172.21.15.72:0/687785802 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7f6bc80d0b50 con 0x7f6bd8104b90 2024-04-26T21:56:03.329 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.317+0000 7f6bcea00640 1 --2- 172.21.15.72:0/2280654256 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f5ec36ca00 0x55f5ec36ee90 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f6bc0006010 tx=0x7f6bc0004150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6307a77f8b29d5a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:56:03.329 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.317+0000 7f6bd7e00640 1 --2- 172.21.15.72:0/687785802 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f6bbc0ac4d0 0x7f6bbc0ae990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:03.329 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.317+0000 7f6bc7e00640 1 -- 172.21.15.72:0/2280654256 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bc0040070 con 0x55f5ec36ca00 2024-04-26T21:56:03.329 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.317+0000 7f6bc7e00640 1 -- 172.21.15.72:0/2280654256 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6bc0003a80 con 0x55f5ec36ca00 2024-04-26T21:56:03.330 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.317+0000 7f6bc7e00640 1 -- 172.21.15.72:0/2280654256 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bc0035030 con 0x55f5ec36ca00 2024-04-26T21:56:03.330 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.318+0000 7f6be1012740 1 -- 172.21.15.72:0/2280654256 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f5ec36ca00 msgr2=0x55f5ec36ee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:03.330 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.318+0000 7f6be1012740 1 --2- 172.21.15.72:0/2280654256 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f5ec36ca00 0x55f5ec36ee90 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f6bc0006010 tx=0x7f6bc0004150 comp rx=0 tx=0).stop 2024-04-26T21:56:03.330 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.318+0000 7f6bd7e00640 1 --2- 172.21.15.72:0/687785802 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f6bbc0ac4d0 0x7f6bbc0ae990 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x55f5ec3680f0 tx=0x7f6bd0009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:56:03.330 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.318+0000 7f6be1012740 1 -- 172.21.15.72:0/2280654256 shutdown_connections 2024-04-26T21:56:03.330 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.318+0000 7f6be1012740 1 --2- 172.21.15.72:0/2280654256 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f5ec36ca00 0x55f5ec36ee90 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:03.330 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.318+0000 7f6be1012740 1 --2- 172.21.15.72:0/2280654256 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f5ec367890 0x55f5ec36c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:03.330 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.318+0000 7f6be1012740 1 --2- 172.21.15.72:0/2280654256 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f5ec366ea0 0x55f5ec3672c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:03.330 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.318+0000 7f6be1012740 1 -- 172.21.15.72:0/2280654256 >> 172.21.15.72:0/2280654256 conn(0x55f5ec362510 msgr2=0x55f5ec362910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:56:03.330 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.318+0000 7f6be1012740 1 -- 172.21.15.72:0/2280654256 shutdown_connections 2024-04-26T21:56:03.330 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.318+0000 7f6be1012740 1 -- 172.21.15.72:0/2280654256 wait complete. 2024-04-26T21:56:03.330 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.319+0000 7f6be1012740 1 -- start start 2024-04-26T21:56:03.330 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.319+0000 7f6be1012740 1 Processor -- start 2024-04-26T21:56:03.331 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.319+0000 7f6be1012740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f5ec366ea0 0x55f5ec370c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:03.331 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.319+0000 7f6be1012740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f5ec367890 0x55f5ec3711b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:03.331 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.319+0000 7f6be1012740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f5ec36ca00 0x55f5ec3781f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:03.331 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.319+0000 7f6be1012740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55f5ec2c9f80 con 0x55f5ec36ca00 2024-04-26T21:56:03.331 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.319+0000 7f6be1012740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55f5ec277260 con 0x55f5ec366ea0 2024-04-26T21:56:03.331 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.319+0000 7f6be1012740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55f5ec2cf2a0 con 0x55f5ec367890 2024-04-26T21:56:03.331 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.319+0000 7f6be1012740 10 client.0 _ll_register_callbacks cb 0x55f5ec2c1960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:56:03.331 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.320+0000 7f6bd7e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f5ec36ca00 0x55f5ec3781f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:03.331 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.320+0000 7f6bd7e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f5ec36ca00 0x55f5ec3781f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:37564/0 (socket says 172.21.15.72:37564) 2024-04-26T21:56:03.331 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.320+0000 7f6bd7e00640 1 -- 172.21.15.72:0/1403461793 learned_addr learned my addr 172.21.15.72:0/1403461793 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:56:03.331 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.320+0000 7f6bcea00640 1 --2- 172.21.15.72:0/1403461793 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f5ec367890 0x55f5ec3711b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:03.331 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.320+0000 7f6bd7400640 1 --2- 172.21.15.72:0/1403461793 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f5ec366ea0 0x55f5ec370c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:03.332 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.320+0000 7f6bd7e00640 1 -- 172.21.15.72:0/1403461793 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f5ec367890 msgr2=0x55f5ec3711b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:56:03.332 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.320+0000 7f6bd7e00640 1 --2- 172.21.15.72:0/1403461793 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f5ec367890 0x55f5ec3711b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:03.332 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.320+0000 7f6bd7e00640 1 -- 172.21.15.72:0/1403461793 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f5ec366ea0 msgr2=0x55f5ec370c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:56:03.332 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.320+0000 7f6bd7e00640 1 --2- 172.21.15.72:0/1403461793 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f5ec366ea0 0x55f5ec370c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:03.332 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.320+0000 7f6bd7e00640 1 -- 172.21.15.72:0/1403461793 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f5ec2591c0 con 0x55f5ec36ca00 2024-04-26T21:56:03.332 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.320+0000 7f6bd7400640 1 --2- 172.21.15.72:0/1403461793 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f5ec366ea0 0x55f5ec370c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:56:03.332 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.321+0000 7f6bd7e00640 1 --2- 172.21.15.72:0/1403461793 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f5ec36ca00 0x55f5ec3781f0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f6bd0012780 tx=0x7f6bd0004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8976b003f86345b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:56:03.332 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.321+0000 7f6bc4c00640 10 client.0 ms_handle_connect on v2:172.21.15.72:3300/0 2024-04-26T21:56:03.332 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.321+0000 7f6bc4c00640 1 -- 172.21.15.72:0/1403461793 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bd0023070 con 0x55f5ec36ca00 2024-04-26T21:56:03.332 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.321+0000 7f6bc4c00640 1 -- 172.21.15.72:0/1403461793 <== mon.0 v2:172.21.15.72:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7f6bd0011b60 con 0x55f5ec36ca00 2024-04-26T21:56:03.332 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.321+0000 7f6be1012740 1 -- 172.21.15.72:0/1403461793 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6bc0003a80 con 0x55f5ec36ca00 2024-04-26T21:56:03.333 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.321+0000 7f6bc4c00640 1 -- 172.21.15.72:0/1403461793 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6bd0011d60 con 0x55f5ec36ca00 2024-04-26T21:56:03.333 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.321+0000 7f6be1012740 10 client.15615 fetch_fsmap learned FSMap version 27 2024-04-26T21:56:03.333 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.321+0000 7f6be1012740 1 -- 172.21.15.72:0/1403461793 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f5ebfdc3f0 con 0x55f5ec36ca00 2024-04-26T21:56:03.333 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.321+0000 7f6bc4c00640 1 -- 172.21.15.72:0/1403461793 <== mon.0 v2:172.21.15.72:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6bc0003a80 con 0x55f5ec36ca00 2024-04-26T21:56:03.333 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.322+0000 7f6bc4c00640 1 -- 172.21.15.72:0/1403461793 <== mon.0 v2:172.21.15.72:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f6bd000f040 con 0x55f5ec36ca00 2024-04-26T21:56:03.333 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.322+0000 7f6be1012740 10 client.15615 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T21:56:03.333 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.322+0000 7f6be1012740 10 client.15615 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-26T21:56:03.333 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.322+0000 7f6be1012740 20 client.15615 populate_metadata read hostname 'smithi072' 2024-04-26T21:56:03.333 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.322+0000 7f6be1012740 1 --2- 172.21.15.72:0/1403461793 >> [v2:172.21.15.76:6836/1150678991,v1:172.21.15.76:6837/1150678991] conn(0x55f5ec37c7f0 0x55f5ec37ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:03.333 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.322+0000 7f6be1012740 4 client.15615 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:56:03.333 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.322+0000 7f6be1012740 1 -- 172.21.15.72:0/1403461793 --> [v2:172.21.15.76:6836/1150678991,v1:172.21.15.76:6837/1150678991] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f5ec2d8880 con 0x55f5ec37c7f0 2024-04-26T21:56:03.334 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.323+0000 7f6bd7400640 1 --2- 172.21.15.72:0/1403461793 >> [v2:172.21.15.76:6836/1150678991,v1:172.21.15.76:6837/1150678991] conn(0x55f5ec37c7f0 0x55f5ec37ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:03.334 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.323+0000 7f6bd7400640 1 --2- 172.21.15.72:0/1403461793 >> [v2:172.21.15.76:6836/1150678991,v1:172.21.15.76:6837/1150678991] conn(0x55f5ec37c7f0 0x55f5ec37ecb0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9e1237cc03d3f2a7 server_cookie=9e7e7ace8969daf5 in_seq=0 out_seq=0 2024-04-26T21:56:03.334 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.323+0000 7f6bc4c00640 10 client.15615 ms_handle_connect on v2:172.21.15.76:6836/1150678991 2024-04-26T21:56:03.334 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.325+0000 7f6bc4c00640 1 -- 172.21.15.72:0/1403461793 <== mds.1 v2:172.21.15.76:6836/1150678991 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f5ec2d8880 con 0x55f5ec37c7f0 2024-04-26T21:56:03.334 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.325+0000 7f6bc4c00640 10 client.15615 handle_command_reply: tid=0 2024-04-26T21:56:03.334 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.325+0000 7f6be1012740 1 client.15615 shutdown 2024-04-26T21:56:03.334 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.326+0000 7f6be1012740 1 -- 172.21.15.72:0/1403461793 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f5ec36ca00 msgr2=0x55f5ec3781f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:03.334 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.326+0000 7f6be1012740 1 --2- 172.21.15.72:0/1403461793 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f5ec36ca00 0x55f5ec3781f0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f6bd0012780 tx=0x7f6bd0004ab0 comp rx=0 tx=0).stop 2024-04-26T21:56:03.334 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.326+0000 7f6be1012740 1 -- 172.21.15.72:0/1403461793 shutdown_connections 2024-04-26T21:56:03.334 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.326+0000 7f6be1012740 1 --2- 172.21.15.72:0/1403461793 >> [v2:172.21.15.76:6836/1150678991,v1:172.21.15.76:6837/1150678991] conn(0x55f5ec37c7f0 0x55f5ec37ecb0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:03.334 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.326+0000 7f6be1012740 1 --2- 172.21.15.72:0/1403461793 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f5ec36ca00 0x55f5ec3781f0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:03.334 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.326+0000 7f6be1012740 1 --2- 172.21.15.72:0/1403461793 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f5ec367890 0x55f5ec3711b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:03.334 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.326+0000 7f6be1012740 1 --2- 172.21.15.72:0/1403461793 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f5ec366ea0 0x55f5ec370c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:03.334 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.326+0000 7f6bc4c00640 0 client.15615 ms_handle_reset on v2:172.21.15.76:6836/1150678991 2024-04-26T21:56:03.335 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.326+0000 7f6be1012740 1 -- 172.21.15.72:0/1403461793 >> 172.21.15.72:0/1403461793 conn(0x55f5ec362510 msgr2=0x55f5ec361690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:56:03.335 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.326+0000 7f6be1012740 1 -- 172.21.15.72:0/1403461793 shutdown_connections 2024-04-26T21:56:03.335 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.326+0000 7f6be1012740 1 -- 172.21.15.72:0/1403461793 wait complete. 2024-04-26T21:56:03.335 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.326+0000 7f6be1012740 20 client.15615 trim_cache size 0 max 16384 2024-04-26T21:56:03.342 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.340+0000 7f6be1012740 1 Processor -- start 2024-04-26T21:56:03.342 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.341+0000 7f6be1012740 1 -- start start 2024-04-26T21:56:03.342 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.341+0000 7f6be1012740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f5ec366ea0 0x55f5ec365420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:03.342 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.341+0000 7f6be1012740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f5ec367890 0x55f5ec3781b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:03.342 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.341+0000 7f6be1012740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f5ec36ca00 0x55f5ec365960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:03.342 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.341+0000 7f6be1012740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f6bd81a7c60 con 0x55f5ec36ca00 2024-04-26T21:56:03.342 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.341+0000 7f6be1012740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f6bd81a97a0 con 0x55f5ec366ea0 2024-04-26T21:56:03.342 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.341+0000 7f6be1012740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f6bd81a8330 con 0x55f5ec367890 2024-04-26T21:56:03.342 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.341+0000 7f6bd7e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f5ec36ca00 0x55f5ec365960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:03.342 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.341+0000 7f6bd7e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f5ec36ca00 0x55f5ec365960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:37576/0 (socket says 172.21.15.72:37576) 2024-04-26T21:56:03.343 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.341+0000 7f6bd7e00640 1 -- 172.21.15.72:0/1143704069 learned_addr learned my addr 172.21.15.72:0/1143704069 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:56:03.343 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.341+0000 7f6bd7400640 1 --2- 172.21.15.72:0/1143704069 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f5ec366ea0 0x55f5ec365420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:03.343 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.341+0000 7f6bd7e00640 1 -- 172.21.15.72:0/1143704069 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f5ec367890 msgr2=0x55f5ec3781b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:56:03.343 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.341+0000 7f6bcea00640 1 --2- 172.21.15.72:0/1143704069 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f5ec367890 0x55f5ec3781b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:03.343 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.342+0000 7f6bd7e00640 1 --2- 172.21.15.72:0/1143704069 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f5ec367890 0x55f5ec3781b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:03.343 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.342+0000 7f6bd7e00640 1 -- 172.21.15.72:0/1143704069 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f5ec366ea0 msgr2=0x55f5ec365420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:56:03.343 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.342+0000 7f6bd7e00640 1 --2- 172.21.15.72:0/1143704069 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f5ec366ea0 0x55f5ec365420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:03.343 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.342+0000 7f6bd7e00640 1 -- 172.21.15.72:0/1143704069 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f5ec0a67d0 con 0x55f5ec36ca00 2024-04-26T21:56:03.343 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.342+0000 7f6bd7400640 1 --2- 172.21.15.72:0/1143704069 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f5ec366ea0 0x55f5ec365420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:56:03.343 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.342+0000 7f6bcea00640 1 --2- 172.21.15.72:0/1143704069 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f5ec367890 0x55f5ec3781b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:56:03.343 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.342+0000 7f6bd7e00640 1 --2- 172.21.15.72:0/1143704069 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f5ec36ca00 0x55f5ec365960 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f6bd0009600 tx=0x7f6bd0005a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afb2d301cbadc5fe server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:56:03.351 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.342+0000 7f6bc7400640 1 -- 172.21.15.72:0/1143704069 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bd0037020 con 0x55f5ec36ca00 2024-04-26T21:56:03.351 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.342+0000 7f6bc7400640 1 -- 172.21.15.72:0/1143704069 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6bd00111b0 con 0x55f5ec36ca00 2024-04-26T21:56:03.351 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.342+0000 7f6bc7400640 1 -- 172.21.15.72:0/1143704069 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bd0023040 con 0x55f5ec36ca00 2024-04-26T21:56:03.351 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.343+0000 7f6be1012740 1 -- 172.21.15.72:0/1143704069 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f5ec36ca00 msgr2=0x55f5ec365960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:03.351 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.343+0000 7f6be1012740 1 --2- 172.21.15.72:0/1143704069 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f5ec36ca00 0x55f5ec365960 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f6bd0009600 tx=0x7f6bd0005a10 comp rx=0 tx=0).stop 2024-04-26T21:56:03.351 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.343+0000 7f6be1012740 1 -- 172.21.15.72:0/1143704069 shutdown_connections 2024-04-26T21:56:03.351 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.343+0000 7f6be1012740 1 --2- 172.21.15.72:0/1143704069 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f5ec36ca00 0x55f5ec365960 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:03.351 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.343+0000 7f6be1012740 1 --2- 172.21.15.72:0/1143704069 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f5ec367890 0x55f5ec3781b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:03.352 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.343+0000 7f6be1012740 1 --2- 172.21.15.72:0/1143704069 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f5ec366ea0 0x55f5ec365420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:03.352 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.343+0000 7f6be1012740 1 -- 172.21.15.72:0/1143704069 >> 172.21.15.72:0/1143704069 conn(0x55f5ec362510 msgr2=0x55f5ec368bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:56:03.352 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.343+0000 7f6be1012740 1 -- 172.21.15.72:0/1143704069 shutdown_connections 2024-04-26T21:56:03.352 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.343+0000 7f6be1012740 1 -- 172.21.15.72:0/1143704069 wait complete. 2024-04-26T21:56:03.352 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.343+0000 7f6be1012740 1 -- start start 2024-04-26T21:56:03.352 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.344+0000 7f6be1012740 1 Processor -- start 2024-04-26T21:56:03.352 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.344+0000 7f6be1012740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f5ec366ea0 0x55f5ec3756f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:03.352 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.344+0000 7f6be1012740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f5ec367890 0x55f5ec375c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:03.352 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.344+0000 7f6be1012740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f5ec36ca00 0x55f5ec376c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:03.352 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.344+0000 7f6be1012740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f6bd81a90d0 con 0x55f5ec36ca00 2024-04-26T21:56:03.352 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.344+0000 7f6be1012740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f6bd81a8a00 con 0x55f5ec367890 2024-04-26T21:56:03.352 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.344+0000 7f6be1012740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f6bd81a5c40 con 0x55f5ec366ea0 2024-04-26T21:56:03.352 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.344+0000 7f6be1012740 10 client.0 _ll_register_callbacks cb 0x7f6bd0016070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:56:03.352 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.344+0000 7f6bd7e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f5ec36ca00 0x55f5ec376c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:03.353 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.344+0000 7f6bd7e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f5ec36ca00 0x55f5ec376c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:37592/0 (socket says 172.21.15.72:37592) 2024-04-26T21:56:03.353 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.344+0000 7f6bd7e00640 1 -- 172.21.15.72:0/40552193 learned_addr learned my addr 172.21.15.72:0/40552193 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:56:03.353 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.344+0000 7f6bd7400640 1 --2- 172.21.15.72:0/40552193 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f5ec366ea0 0x55f5ec3756f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:03.353 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.345+0000 7f6bd7e00640 1 -- 172.21.15.72:0/40552193 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f5ec366ea0 msgr2=0x55f5ec3756f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:56:03.353 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.345+0000 7f6bd7e00640 1 --2- 172.21.15.72:0/40552193 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f5ec366ea0 0x55f5ec3756f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:03.353 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.345+0000 7f6bd7e00640 1 -- 172.21.15.72:0/40552193 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f5ec367890 msgr2=0x55f5ec375c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:56:03.353 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.345+0000 7f6bd7e00640 1 --2- 172.21.15.72:0/40552193 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f5ec367890 0x55f5ec375c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:03.353 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.345+0000 7f6bd7e00640 1 -- 172.21.15.72:0/40552193 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f5ec2712a0 con 0x55f5ec36ca00 2024-04-26T21:56:03.353 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.345+0000 7f6bd7e00640 1 --2- 172.21.15.72:0/40552193 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f5ec36ca00 0x55f5ec376c40 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f6bd0009810 tx=0x7f6bd001fda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc047f844690c9ea server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:56:03.353 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.345+0000 7f6bc4c00640 10 client.0 ms_handle_connect on v2:172.21.15.72:3300/0 2024-04-26T21:56:03.353 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.345+0000 7f6bc4c00640 1 -- 172.21.15.72:0/40552193 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bd0037030 con 0x55f5ec36ca00 2024-04-26T21:56:03.353 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.345+0000 7f6bc4c00640 1 -- 172.21.15.72:0/40552193 <== mon.0 v2:172.21.15.72:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7f6bd003c450 con 0x55f5ec36ca00 2024-04-26T21:56:03.353 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.345+0000 7f6be1012740 1 -- 172.21.15.72:0/40552193 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6bd00111b0 con 0x55f5ec36ca00 2024-04-26T21:56:03.353 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.346+0000 7f6bc4c00640 1 -- 172.21.15.72:0/40552193 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6bd004b050 con 0x55f5ec36ca00 2024-04-26T21:56:03.354 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.346+0000 7f6be1012740 10 client.15621 fetch_fsmap learned FSMap version 27 2024-04-26T21:56:03.354 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.346+0000 7f6be1012740 1 -- 172.21.15.72:0/40552193 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f5ec3782d0 con 0x55f5ec36ca00 2024-04-26T21:56:03.354 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.346+0000 7f6bc4c00640 1 -- 172.21.15.72:0/40552193 <== mon.0 v2:172.21.15.72:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6bd00111b0 con 0x55f5ec36ca00 2024-04-26T21:56:03.354 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.346+0000 7f6bc4c00640 1 -- 172.21.15.72:0/40552193 <== mon.0 v2:172.21.15.72:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f6bd00474f0 con 0x55f5ec36ca00 2024-04-26T21:56:03.354 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.346+0000 7f6be1012740 10 client.15621 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T21:56:03.354 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.346+0000 7f6be1012740 10 client.15621 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-26T21:56:03.354 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.346+0000 7f6be1012740 20 client.15621 populate_metadata read hostname 'smithi072' 2024-04-26T21:56:03.354 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.346+0000 7f6be1012740 1 --2- 172.21.15.72:0/40552193 >> [v2:172.21.15.76:6836/1150678991,v1:172.21.15.76:6837/1150678991] conn(0x55f5ec3e3400 0x55f5ec3e5820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:03.354 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.346+0000 7f6be1012740 4 client.15621 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-26T21:56:03.354 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.346+0000 7f6be1012740 1 -- 172.21.15.72:0/40552193 --> [v2:172.21.15.76:6836/1150678991,v1:172.21.15.76:6837/1150678991] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f5ec2d26f0 con 0x55f5ec3e3400 2024-04-26T21:56:03.354 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.347+0000 7f6bd7400640 1 --2- 172.21.15.72:0/40552193 >> [v2:172.21.15.76:6836/1150678991,v1:172.21.15.76:6837/1150678991] conn(0x55f5ec3e3400 0x55f5ec3e5820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:03.354 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.348+0000 7f6bd7400640 1 --2- 172.21.15.72:0/40552193 >> [v2:172.21.15.76:6836/1150678991,v1:172.21.15.76:6837/1150678991] conn(0x55f5ec3e3400 0x55f5ec3e5820 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a7b5b63af6c99746 server_cookie=7f755675e32ae98 in_seq=0 out_seq=0 2024-04-26T21:56:03.354 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.348+0000 7f6bc4c00640 10 client.15621 ms_handle_connect on v2:172.21.15.76:6836/1150678991 2024-04-26T21:56:03.355 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.348+0000 7f6bc4c00640 1 -- 172.21.15.72:0/40552193 <== mds.1 v2:172.21.15.76:6836/1150678991 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f5ec2d26f0 con 0x55f5ec3e3400 2024-04-26T21:56:03.355 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.348+0000 7f6bc4c00640 10 client.15621 handle_command_reply: tid=0 2024-04-26T21:56:03.355 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.348+0000 7f6be1012740 1 client.15621 shutdown 2024-04-26T21:56:03.355 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.348+0000 7f6be1012740 1 -- 172.21.15.72:0/40552193 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f5ec36ca00 msgr2=0x55f5ec376c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:03.355 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.348+0000 7f6be1012740 1 --2- 172.21.15.72:0/40552193 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f5ec36ca00 0x55f5ec376c40 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f6bd0009810 tx=0x7f6bd001fda0 comp rx=0 tx=0).stop 2024-04-26T21:56:03.355 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.349+0000 7f6be1012740 1 -- 172.21.15.72:0/40552193 shutdown_connections 2024-04-26T21:56:03.355 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.349+0000 7f6be1012740 1 --2- 172.21.15.72:0/40552193 >> [v2:172.21.15.76:6836/1150678991,v1:172.21.15.76:6837/1150678991] conn(0x55f5ec3e3400 0x55f5ec3e5820 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:03.355 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.349+0000 7f6be1012740 1 --2- 172.21.15.72:0/40552193 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f5ec36ca00 0x55f5ec376c40 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:03.355 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.349+0000 7f6be1012740 1 --2- 172.21.15.72:0/40552193 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f5ec367890 0x55f5ec375c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:03.355 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.349+0000 7f6be1012740 1 --2- 172.21.15.72:0/40552193 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f5ec366ea0 0x55f5ec3756f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:03.355 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.349+0000 7f6bc4c00640 0 client.15621 ms_handle_reset on v2:172.21.15.76:6836/1150678991 2024-04-26T21:56:03.355 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.349+0000 7f6be1012740 1 -- 172.21.15.72:0/40552193 >> 172.21.15.72:0/40552193 conn(0x55f5ec362510 msgr2=0x55f5ec360e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:56:03.355 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.349+0000 7f6be1012740 1 -- 172.21.15.72:0/40552193 shutdown_connections 2024-04-26T21:56:03.355 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.349+0000 7f6be1012740 1 -- 172.21.15.72:0/40552193 wait complete. 2024-04-26T21:56:03.356 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.349+0000 7f6be1012740 20 client.15621 trim_cache size 0 max 16384 2024-04-26T21:56:03.356 INFO:teuthology.orchestra.run.smithi072.stdout:[] 2024-04-26T21:56:03.356 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.350+0000 7f6bde200640 1 -- 172.21.15.72:0/687785802 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f6bbc0ac4d0 msgr2=0x7f6bbc0ae990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:03.357 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.350+0000 7f6bde200640 1 --2- 172.21.15.72:0/687785802 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f6bbc0ac4d0 0x7f6bbc0ae990 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x55f5ec3680f0 tx=0x7f6bd0009500 comp rx=0 tx=0).stop 2024-04-26T21:56:03.357 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.350+0000 7f6bde200640 1 -- 172.21.15.72:0/687785802 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6bd8104b90 msgr2=0x7f6bd819ad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:03.357 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.350+0000 7f6bde200640 1 --2- 172.21.15.72:0/687785802 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6bd8104b90 0x7f6bd819ad80 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f6bc800e9c0 tx=0x7f6bc800ee80 comp rx=0 tx=0).stop 2024-04-26T21:56:03.357 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.350+0000 7f6bde200640 1 -- 172.21.15.72:0/687785802 shutdown_connections 2024-04-26T21:56:03.357 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.350+0000 7f6bde200640 1 --2- 172.21.15.72:0/687785802 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f6bbc0ac4d0 0x7f6bbc0ae990 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:03.357 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.350+0000 7f6bde200640 1 --2- 172.21.15.72:0/687785802 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6bd8106740 0x7f6bd81a22a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:03.357 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.350+0000 7f6bde200640 1 --2- 172.21.15.72:0/687785802 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6bd8105d80 0x7f6bd819b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:03.357 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.350+0000 7f6bde200640 1 --2- 172.21.15.72:0/687785802 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6bd8104b90 0x7f6bd819ad80 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:03.357 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.350+0000 7f6bde200640 1 -- 172.21.15.72:0/687785802 >> 172.21.15.72:0/687785802 conn(0x7f6bd8100120 msgr2=0x7f6bd81091f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:56:03.357 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.350+0000 7f6bde200640 1 -- 172.21.15.72:0/687785802 shutdown_connections 2024-04-26T21:56:03.357 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.350+0000 7f6bde200640 1 -- 172.21.15.72:0/687785802 wait complete. 2024-04-26T21:56:03.367 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-26T21:56:03.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:56:02 smithi164 ceph-mon[31916]: pgmap v467: 97 pgs: 97 active+clean; 650 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:56:03.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:56:02 smithi164 ceph-mon[31916]: from='client.? 172.21.15.72:0/824602508' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:56:03.551 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.551+0000 7f0faca00640 1 -- 172.21.15.72:0/1981578191 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0fa8105c60 msgr2=0x7f0fa81060e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:03.551 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.551+0000 7f0faca00640 1 --2- 172.21.15.72:0/1981578191 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0fa8105c60 0x7f0fa81060e0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f0f98009900 tx=0x7f0f9802a3f0 comp rx=0 tx=0).stop 2024-04-26T21:56:03.551 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.551+0000 7f0faca00640 1 -- 172.21.15.72:0/1981578191 shutdown_connections 2024-04-26T21:56:03.551 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.551+0000 7f0faca00640 1 --2- 172.21.15.72:0/1981578191 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0fa8106620 0x7f0fa810ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:03.552 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.551+0000 7f0faca00640 1 --2- 172.21.15.72:0/1981578191 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0fa8105c60 0x7f0fa81060e0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:03.552 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.551+0000 7f0faca00640 1 --2- 172.21.15.72:0/1981578191 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0fa8104a70 0x7f0fa8104e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:03.552 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.551+0000 7f0faca00640 1 -- 172.21.15.72:0/1981578191 >> 172.21.15.72:0/1981578191 conn(0x7f0fa80fffe0 msgr2=0x7f0fa8102420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:56:03.552 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.551+0000 7f0faca00640 1 -- 172.21.15.72:0/1981578191 shutdown_connections 2024-04-26T21:56:03.552 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.551+0000 7f0faca00640 1 -- 172.21.15.72:0/1981578191 wait complete. 2024-04-26T21:56:03.554 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.552+0000 7f0faca00640 1 Processor -- start 2024-04-26T21:56:03.554 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.552+0000 7f0faca00640 1 -- start start 2024-04-26T21:56:03.554 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.552+0000 7f0faca00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0fa8104a70 0x7f0fa819ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:03.554 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.552+0000 7f0faca00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0fa8105c60 0x7f0fa819b290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:03.554 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.552+0000 7f0faca00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0fa8106620 0x7f0fa81a2300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:03.555 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.552+0000 7f0faca00640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f0fa8110590 con 0x7f0fa8104a70 2024-04-26T21:56:03.555 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.552+0000 7f0faca00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f0fa8110410 con 0x7f0fa8106620 2024-04-26T21:56:03.555 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.552+0000 7f0faca00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f0fa8110710 con 0x7f0fa8105c60 2024-04-26T21:56:03.555 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.553+0000 7f0fa7400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0fa8104a70 0x7f0fa819ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:03.555 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.553+0000 7f0fa7400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0fa8104a70 0x7f0fa819ad50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:37606/0 (socket says 172.21.15.72:37606) 2024-04-26T21:56:03.555 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.553+0000 7f0fa7400640 1 -- 172.21.15.72:0/445820685 learned_addr learned my addr 172.21.15.72:0/445820685 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:56:03.555 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.553+0000 7f0fa6a00640 1 --2- 172.21.15.72:0/445820685 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0fa8105c60 0x7f0fa819b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:03.555 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.553+0000 7f0fa7e00640 1 --2- 172.21.15.72:0/445820685 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0fa8106620 0x7f0fa81a2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:03.555 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.553+0000 7f0fa7400640 1 -- 172.21.15.72:0/445820685 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0fa8105c60 msgr2=0x7f0fa819b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:03.555 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.553+0000 7f0fa7400640 1 --2- 172.21.15.72:0/445820685 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0fa8105c60 0x7f0fa819b290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:03.555 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.553+0000 7f0fa7400640 1 -- 172.21.15.72:0/445820685 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0fa8106620 msgr2=0x7f0fa81a2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:03.555 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.553+0000 7f0fa7400640 1 --2- 172.21.15.72:0/445820685 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0fa8106620 0x7f0fa81a2300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:03.555 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.553+0000 7f0fa7400640 1 -- 172.21.15.72:0/445820685 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fa81a2970 con 0x7f0fa8104a70 2024-04-26T21:56:03.556 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.553+0000 7f0fa7e00640 1 --2- 172.21.15.72:0/445820685 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0fa8106620 0x7f0fa81a2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:56:03.556 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.554+0000 7f0fa7400640 1 --2- 172.21.15.72:0/445820685 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0fa8104a70 0x7f0fa819ad50 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f0f9000b450 tx=0x7f0f9000b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:56:03.556 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.554+0000 7f0f9fe00640 1 -- 172.21.15.72:0/445820685 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f90015020 con 0x7f0fa8104a70 2024-04-26T21:56:03.556 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.554+0000 7f0f9fe00640 1 -- 172.21.15.72:0/445820685 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0f90011d20 con 0x7f0fa8104a70 2024-04-26T21:56:03.556 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.554+0000 7f0f9fe00640 1 -- 172.21.15.72:0/445820685 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f90010e10 con 0x7f0fa8104a70 2024-04-26T21:56:03.556 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.554+0000 7f0faca00640 1 -- 172.21.15.72:0/445820685 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0fa81a2c60 con 0x7f0fa8104a70 2024-04-26T21:56:03.556 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.554+0000 7f0faca00640 1 -- 172.21.15.72:0/445820685 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0fa81a3180 con 0x7f0fa8104a70 2024-04-26T21:56:03.559 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.558+0000 7f0fb0e4c740 1 Processor -- start 2024-04-26T21:56:03.559 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.558+0000 7f0fb0e4c740 1 -- start start 2024-04-26T21:56:03.559 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.559+0000 7f0f9fe00640 1 -- 172.21.15.72:0/445820685 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f0f90018020 con 0x7f0fa8104a70 2024-04-26T21:56:03.559 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.559+0000 7f0fb0e4c740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559fcecda050 0x559fcecda470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:03.559 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.559+0000 7f0fb0e4c740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559fcecdaa40 0x559fcecdf540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:03.559 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.559+0000 7f0fb0e4c740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559fcecdfbb0 0x559fcece2040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:03.559 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.559+0000 7f0fb0e4c740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x559fcec3dc70 con 0x559fcecdaa40 2024-04-26T21:56:03.560 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.559+0000 7f0fb0e4c740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x559fcebf6980 con 0x559fcecdfbb0 2024-04-26T21:56:03.560 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.559+0000 7f0fb0e4c740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x559fcec39c30 con 0x559fcecda050 2024-04-26T21:56:03.560 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.559+0000 7f0fa7400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559fcecdaa40 0x559fcecdf540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:03.560 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.559+0000 7f0fa7400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559fcecdaa40 0x559fcecdf540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:37608/0 (socket says 172.21.15.72:37608) 2024-04-26T21:56:03.560 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.559+0000 7f0fa7400640 1 -- 172.21.15.72:0/2593266867 learned_addr learned my addr 172.21.15.72:0/2593266867 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:56:03.562 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.560+0000 7f0fa6a00640 1 --2- 172.21.15.72:0/2593266867 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559fcecdfbb0 0x559fcece2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:03.562 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.560+0000 7f0fa7400640 1 -- 172.21.15.72:0/2593266867 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559fcecda050 msgr2=0x559fcecda470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:56:03.562 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.560+0000 7f0fa7400640 1 --2- 172.21.15.72:0/2593266867 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559fcecda050 0x559fcecda470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:03.562 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.560+0000 7f0fa7400640 1 -- 172.21.15.72:0/2593266867 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559fcecdfbb0 msgr2=0x559fcece2040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:56:03.562 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.560+0000 7f0fa7400640 1 --2- 172.21.15.72:0/2593266867 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559fcecdfbb0 0x559fcece2040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:03.562 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.560+0000 7f0fa7400640 1 -- 172.21.15.72:0/2593266867 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559fcebce7b0 con 0x559fcecdaa40 2024-04-26T21:56:03.562 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.560+0000 7f0fa7400640 1 --2- 172.21.15.72:0/2593266867 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559fcecdaa40 0x559fcecdf540 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f0f900d3fd0 tx=0x7f0f900d86f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dbefdae3506b4a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:56:03.562 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.560+0000 7f0f97e00640 1 -- 172.21.15.72:0/2593266867 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f900e3070 con 0x559fcecdaa40 2024-04-26T21:56:03.563 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.560+0000 7f0f97e00640 1 -- 172.21.15.72:0/2593266867 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0f900d0af0 con 0x559fcecdaa40 2024-04-26T21:56:03.563 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.560+0000 7f0f97e00640 1 -- 172.21.15.72:0/2593266867 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f900de8a0 con 0x559fcecdaa40 2024-04-26T21:56:03.563 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.560+0000 7f0f9fe00640 1 --2- 172.21.15.72:0/445820685 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f0f8c0a3c40 0x7f0f8c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:03.563 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.561+0000 7f0f9fe00640 1 -- 172.21.15.72:0/445820685 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7f0f900d0070 con 0x7f0fa8104a70 2024-04-26T21:56:03.563 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.561+0000 7f0fa7e00640 1 --2- 172.21.15.72:0/445820685 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f0f8c0a3c40 0x7f0f8c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:03.563 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.561+0000 7f0fb0e4c740 1 -- 172.21.15.72:0/2593266867 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559fcecdaa40 msgr2=0x559fcecdf540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:03.563 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.561+0000 7f0fb0e4c740 1 --2- 172.21.15.72:0/2593266867 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559fcecdaa40 0x559fcecdf540 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f0f900d3fd0 tx=0x7f0f900d86f0 comp rx=0 tx=0).stop 2024-04-26T21:56:03.563 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.561+0000 7f0fb0e4c740 1 -- 172.21.15.72:0/2593266867 shutdown_connections 2024-04-26T21:56:03.563 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.561+0000 7f0fb0e4c740 1 --2- 172.21.15.72:0/2593266867 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559fcecdfbb0 0x559fcece2040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:03.563 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.561+0000 7f0fb0e4c740 1 --2- 172.21.15.72:0/2593266867 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559fcecdaa40 0x559fcecdf540 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:03.563 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.561+0000 7f0fb0e4c740 1 --2- 172.21.15.72:0/2593266867 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559fcecda050 0x559fcecda470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:03.563 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.561+0000 7f0fb0e4c740 1 -- 172.21.15.72:0/2593266867 >> 172.21.15.72:0/2593266867 conn(0x559fcecd56c0 msgr2=0x559fcecd5ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:56:03.563 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.561+0000 7f0fb0e4c740 1 -- 172.21.15.72:0/2593266867 shutdown_connections 2024-04-26T21:56:03.563 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.562+0000 7f0fb0e4c740 1 -- 172.21.15.72:0/2593266867 wait complete. 2024-04-26T21:56:03.564 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.562+0000 7f0fa7e00640 1 --2- 172.21.15.72:0/445820685 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f0f8c0a3c40 0x7f0f8c0a6100 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x559fcecdb2a0 tx=0x7f0fa0008330 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:56:03.564 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.562+0000 7f0fb0e4c740 1 -- start start 2024-04-26T21:56:03.564 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.562+0000 7f0fb0e4c740 1 Processor -- start 2024-04-26T21:56:03.567 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.563+0000 7f0fb0e4c740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559fcecda050 0x559fcece3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:03.567 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.563+0000 7f0fb0e4c740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559fcecdaa40 0x559fcece4500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:03.567 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.563+0000 7f0fb0e4c740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559fcecdfbb0 0x559fceceb540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:03.567 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.563+0000 7f0fb0e4c740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x559fcec3cf80 con 0x559fcecdaa40 2024-04-26T21:56:03.567 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.563+0000 7f0fb0e4c740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x559fcebea260 con 0x559fcecda050 2024-04-26T21:56:03.567 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.563+0000 7f0fb0e4c740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x559fcec422a0 con 0x559fcecdfbb0 2024-04-26T21:56:03.567 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.563+0000 7f0fb0e4c740 10 client.0 _ll_register_callbacks cb 0x559fcec34960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:56:03.567 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.563+0000 7f0fa6a00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559fcecdaa40 0x559fcece4500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:03.567 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.563+0000 7f0fa6a00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559fcecdaa40 0x559fcece4500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:37612/0 (socket says 172.21.15.72:37612) 2024-04-26T21:56:03.567 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.563+0000 7f0fa6a00640 1 -- 172.21.15.72:0/570330507 learned_addr learned my addr 172.21.15.72:0/570330507 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:56:03.567 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.563+0000 7f0fa7400640 1 --2- 172.21.15.72:0/570330507 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559fcecda050 0x559fcece3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:03.568 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.563+0000 7f0fa7e00640 1 --2- 172.21.15.72:0/570330507 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559fcecdfbb0 0x559fceceb540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:03.568 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.563+0000 7f0fa6a00640 1 -- 172.21.15.72:0/570330507 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559fcecdfbb0 msgr2=0x559fceceb540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:56:03.568 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.563+0000 7f0fa6a00640 1 --2- 172.21.15.72:0/570330507 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559fcecdfbb0 0x559fceceb540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:03.568 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.563+0000 7f0fa6a00640 1 -- 172.21.15.72:0/570330507 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559fcecda050 msgr2=0x559fcece3fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:56:03.568 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.564+0000 7f0fa6a00640 1 --2- 172.21.15.72:0/570330507 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559fcecda050 0x559fcece3fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:03.568 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.564+0000 7f0fa6a00640 1 -- 172.21.15.72:0/570330507 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559fcebcc1c0 con 0x559fcecdaa40 2024-04-26T21:56:03.568 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.564+0000 7f0fa7e00640 1 --2- 172.21.15.72:0/570330507 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559fcecdfbb0 0x559fceceb540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:56:03.568 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.564+0000 7f0fa6a00640 1 --2- 172.21.15.72:0/570330507 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559fcecdaa40 0x559fcece4500 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f0f98031460 tx=0x7f0f980045a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=153f4c34084afa7a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:56:03.568 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.564+0000 7f0f94c00640 10 client.0 ms_handle_connect on v2:172.21.15.72:3300/0 2024-04-26T21:56:03.568 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.564+0000 7f0f94c00640 1 -- 172.21.15.72:0/570330507 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f98035030 con 0x559fcecdaa40 2024-04-26T21:56:03.568 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.564+0000 7f0f94c00640 1 -- 172.21.15.72:0/570330507 <== mon.0 v2:172.21.15.72:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7f0f98003b00 con 0x559fcecdaa40 2024-04-26T21:56:03.568 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.564+0000 7f0fb0e4c740 1 -- 172.21.15.72:0/570330507 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0f900d0af0 con 0x559fcecdaa40 2024-04-26T21:56:03.568 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.564+0000 7f0f94c00640 1 -- 172.21.15.72:0/570330507 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0f98049030 con 0x559fcecdaa40 2024-04-26T21:56:03.568 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.565+0000 7f0fb0e4c740 10 client.15633 fetch_fsmap learned FSMap version 27 2024-04-26T21:56:03.569 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.565+0000 7f0fb0e4c740 1 -- 172.21.15.72:0/570330507 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559fce94f3f0 con 0x559fcecdaa40 2024-04-26T21:56:03.569 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.565+0000 7f0f94c00640 1 -- 172.21.15.72:0/570330507 <== mon.0 v2:172.21.15.72:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0f900d0af0 con 0x559fcecdaa40 2024-04-26T21:56:03.569 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.565+0000 7f0f94c00640 1 -- 172.21.15.72:0/570330507 <== mon.0 v2:172.21.15.72:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f0f9803c440 con 0x559fcecdaa40 2024-04-26T21:56:03.569 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.565+0000 7f0fb0e4c740 10 client.15633 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T21:56:03.569 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.565+0000 7f0fb0e4c740 10 client.15633 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:56:03.569 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.565+0000 7f0fb0e4c740 20 client.15633 populate_metadata read hostname 'smithi072' 2024-04-26T21:56:03.569 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.565+0000 7f0fb0e4c740 1 --2- 172.21.15.72:0/570330507 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x559fcecefb30 0x559fcecf1f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:03.569 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.565+0000 7f0fb0e4c740 4 client.15633 mds_command: new command op to 34517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:56:03.569 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.565+0000 7f0fb0e4c740 1 -- 172.21.15.72:0/570330507 --> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559fcec4b880 con 0x559fcecefb30 2024-04-26T21:56:03.569 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.566+0000 7f0fa7400640 1 --2- 172.21.15.72:0/570330507 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x559fcecefb30 0x559fcecf1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:03.569 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.567+0000 7f0fa7400640 1 --2- 172.21.15.72:0/570330507 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x559fcecefb30 0x559fcecf1f40 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ce6187cf90c609c server_cookie=ab53f9667ee8480f in_seq=0 out_seq=0 2024-04-26T21:56:03.569 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.567+0000 7f0f94c00640 10 client.15633 ms_handle_connect on v2:172.21.15.76:6838/327145636 2024-04-26T21:56:03.571 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.569+0000 7f0f94c00640 1 -- 172.21.15.72:0/570330507 <== mds.0 v2:172.21.15.76:6838/327145636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559fcec4b880 con 0x559fcecefb30 2024-04-26T21:56:03.571 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.569+0000 7f0f94c00640 10 client.15633 handle_command_reply: tid=0 2024-04-26T21:56:03.571 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.569+0000 7f0fb0e4c740 1 client.15633 shutdown 2024-04-26T21:56:03.571 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.569+0000 7f0fb0e4c740 1 -- 172.21.15.72:0/570330507 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559fcecdaa40 msgr2=0x559fcece4500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:03.571 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.569+0000 7f0fb0e4c740 1 --2- 172.21.15.72:0/570330507 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559fcecdaa40 0x559fcece4500 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f0f98031460 tx=0x7f0f980045a0 comp rx=0 tx=0).stop 2024-04-26T21:56:03.571 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.569+0000 7f0fb0e4c740 1 -- 172.21.15.72:0/570330507 shutdown_connections 2024-04-26T21:56:03.571 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.569+0000 7f0fb0e4c740 1 --2- 172.21.15.72:0/570330507 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x559fcecefb30 0x559fcecf1f40 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:03.571 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.569+0000 7f0fb0e4c740 1 --2- 172.21.15.72:0/570330507 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559fcecdfbb0 0x559fceceb540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:03.571 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.569+0000 7f0fb0e4c740 1 --2- 172.21.15.72:0/570330507 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559fcecdaa40 0x559fcece4500 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:03.571 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.569+0000 7f0fb0e4c740 1 --2- 172.21.15.72:0/570330507 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559fcecda050 0x559fcece3fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:03.571 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.569+0000 7f0fb0e4c740 1 -- 172.21.15.72:0/570330507 >> 172.21.15.72:0/570330507 conn(0x559fcecd56c0 msgr2=0x559fcecd47b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:56:03.571 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.569+0000 7f0f94c00640 0 client.15633 ms_handle_reset on v2:172.21.15.76:6838/327145636 2024-04-26T21:56:03.571 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.569+0000 7f0fb0e4c740 1 -- 172.21.15.72:0/570330507 shutdown_connections 2024-04-26T21:56:03.572 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.569+0000 7f0fb0e4c740 1 -- 172.21.15.72:0/570330507 wait complete. 2024-04-26T21:56:03.572 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.569+0000 7f0fb0e4c740 20 client.15633 trim_cache size 0 max 16384 2024-04-26T21:56:03.584 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.584+0000 7f0fb0e4c740 1 Processor -- start 2024-04-26T21:56:03.584 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.584+0000 7f0fb0e4c740 1 -- start start 2024-04-26T21:56:03.584 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.584+0000 7f0fb0e4c740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559fcecda050 0x559fcecd4630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:03.585 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.584+0000 7f0fb0e4c740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559fcecdaa40 0x559fceceb500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:03.585 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.584+0000 7f0fb0e4c740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559fcecdfbb0 0x559fcecdb4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:03.585 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.584+0000 7f0fb0e4c740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f0fa81ac040 con 0x559fcecdfbb0 2024-04-26T21:56:03.585 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.584+0000 7f0fb0e4c740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f0fa81adb80 con 0x559fcecdaa40 2024-04-26T21:56:03.585 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.584+0000 7f0fb0e4c740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f0fa81ac710 con 0x559fcecda050 2024-04-26T21:56:03.586 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.585+0000 7f0fa7e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559fcecdfbb0 0x559fcecdb4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:03.586 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.585+0000 7f0fa7e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559fcecdfbb0 0x559fcecdb4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:37622/0 (socket says 172.21.15.72:37622) 2024-04-26T21:56:03.586 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.585+0000 7f0fa7e00640 1 -- 172.21.15.72:0/3696603286 learned_addr learned my addr 172.21.15.72:0/3696603286 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:56:03.586 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.585+0000 7f0fa6a00640 1 --2- 172.21.15.72:0/3696603286 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559fcecdaa40 0x559fceceb500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:03.586 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.585+0000 7f0fa7400640 1 --2- 172.21.15.72:0/3696603286 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559fcecda050 0x559fcecd4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:03.586 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.585+0000 7f0fa7e00640 1 -- 172.21.15.72:0/3696603286 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559fcecda050 msgr2=0x559fcecd4630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:56:03.587 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.585+0000 7f0fa7e00640 1 --2- 172.21.15.72:0/3696603286 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559fcecda050 0x559fcecd4630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:03.587 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.585+0000 7f0fa7e00640 1 -- 172.21.15.72:0/3696603286 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559fcecdaa40 msgr2=0x559fceceb500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:56:03.587 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.585+0000 7f0fa7e00640 1 --2- 172.21.15.72:0/3696603286 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559fcecdaa40 0x559fceceb500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:03.587 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.585+0000 7f0fa7e00640 1 -- 172.21.15.72:0/3696603286 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559fcea197d0 con 0x559fcecdfbb0 2024-04-26T21:56:03.587 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.586+0000 7f0fa7e00640 1 --2- 172.21.15.72:0/3696603286 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559fcecdfbb0 0x559fcecdb4e0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f0fa00089e0 tx=0x7f0fa0010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d17c45b925a3617 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:56:03.587 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.586+0000 7f0f97400640 1 -- 172.21.15.72:0/3696603286 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fa0021070 con 0x559fcecdfbb0 2024-04-26T21:56:03.587 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.586+0000 7f0f97400640 1 -- 172.21.15.72:0/3696603286 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0fa0016040 con 0x559fcecdfbb0 2024-04-26T21:56:03.587 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.586+0000 7f0f97400640 1 -- 172.21.15.72:0/3696603286 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fa001dbb0 con 0x559fcecdfbb0 2024-04-26T21:56:03.591 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.587+0000 7f0fb0e4c740 1 -- 172.21.15.72:0/3696603286 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559fcecdfbb0 msgr2=0x559fcecdb4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:03.591 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.587+0000 7f0fb0e4c740 1 --2- 172.21.15.72:0/3696603286 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559fcecdfbb0 0x559fcecdb4e0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f0fa00089e0 tx=0x7f0fa0010000 comp rx=0 tx=0).stop 2024-04-26T21:56:03.591 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.587+0000 7f0fb0e4c740 1 -- 172.21.15.72:0/3696603286 shutdown_connections 2024-04-26T21:56:03.591 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.587+0000 7f0fb0e4c740 1 --2- 172.21.15.72:0/3696603286 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559fcecdfbb0 0x559fcecdb4e0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:03.591 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.587+0000 7f0fb0e4c740 1 --2- 172.21.15.72:0/3696603286 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559fcecdaa40 0x559fceceb500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:03.591 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.587+0000 7f0fb0e4c740 1 --2- 172.21.15.72:0/3696603286 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559fcecda050 0x559fcecd4630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:03.591 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.587+0000 7f0fb0e4c740 1 -- 172.21.15.72:0/3696603286 >> 172.21.15.72:0/3696603286 conn(0x559fcecd56c0 msgr2=0x559fcecd3f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:56:03.591 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.587+0000 7f0fb0e4c740 1 -- 172.21.15.72:0/3696603286 shutdown_connections 2024-04-26T21:56:03.591 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.587+0000 7f0fb0e4c740 1 -- 172.21.15.72:0/3696603286 wait complete. 2024-04-26T21:56:03.591 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.587+0000 7f0fb0e4c740 1 -- start start 2024-04-26T21:56:03.592 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.588+0000 7f0fb0e4c740 1 Processor -- start 2024-04-26T21:56:03.592 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.588+0000 7f0fb0e4c740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559fcecda050 0x559fcece8490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:03.592 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.588+0000 7f0fb0e4c740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559fcecdaa40 0x559fcece89d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:03.592 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.588+0000 7f0fb0e4c740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559fcecdfbb0 0x559fced5c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:03.592 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.588+0000 7f0fb0e4c740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f0fa81ad4b0 con 0x559fcecdfbb0 2024-04-26T21:56:03.592 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.588+0000 7f0fb0e4c740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f0fa81acde0 con 0x559fcecda050 2024-04-26T21:56:03.592 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.588+0000 7f0fb0e4c740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f0fa81aa020 con 0x559fcecdaa40 2024-04-26T21:56:03.592 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.588+0000 7f0fb0e4c740 10 client.0 _ll_register_callbacks cb 0x7f0f98009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:56:03.592 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.588+0000 7f0fa7e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559fcecdfbb0 0x559fced5c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:03.592 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.588+0000 7f0fa7e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559fcecdfbb0 0x559fced5c8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:37636/0 (socket says 172.21.15.72:37636) 2024-04-26T21:56:03.592 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.588+0000 7f0fa7e00640 1 -- 172.21.15.72:0/3556257199 learned_addr learned my addr 172.21.15.72:0/3556257199 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:56:03.592 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.589+0000 7f0fa6a00640 1 --2- 172.21.15.72:0/3556257199 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559fcecdaa40 0x559fcece89d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:03.592 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.589+0000 7f0fa7400640 1 --2- 172.21.15.72:0/3556257199 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559fcecda050 0x559fcece8490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:03.592 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.589+0000 7f0fa7e00640 1 -- 172.21.15.72:0/3556257199 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559fcecdaa40 msgr2=0x559fcece89d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:56:03.593 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.589+0000 7f0fa7e00640 1 --2- 172.21.15.72:0/3556257199 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559fcecdaa40 0x559fcece89d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:03.593 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.589+0000 7f0fa7e00640 1 -- 172.21.15.72:0/3556257199 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559fcecda050 msgr2=0x559fcece8490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:56:03.593 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.589+0000 7f0fa7e00640 1 --2- 172.21.15.72:0/3556257199 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559fcecda050 0x559fcece8490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:03.593 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.589+0000 7f0fa7e00640 1 -- 172.21.15.72:0/3556257199 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559fcebe42a0 con 0x559fcecdfbb0 2024-04-26T21:56:03.593 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.589+0000 7f0fa7400640 1 --2- 172.21.15.72:0/3556257199 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559fcecda050 0x559fcece8490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:56:03.593 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.589+0000 7f0fa6a00640 1 --2- 172.21.15.72:0/3556257199 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559fcecdaa40 0x559fcece89d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:56:03.593 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.589+0000 7f0fa7e00640 1 --2- 172.21.15.72:0/3556257199 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559fcecdfbb0 0x559fced5c8c0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f0fa0012040 tx=0x7f0fa001db60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fd45eb11fd8f895 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:56:03.593 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.589+0000 7f0f94c00640 10 client.0 ms_handle_connect on v2:172.21.15.72:3300/0 2024-04-26T21:56:03.593 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.589+0000 7f0f94c00640 1 -- 172.21.15.72:0/3556257199 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fa0021040 con 0x559fcecdfbb0 2024-04-26T21:56:03.593 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.589+0000 7f0f94c00640 1 -- 172.21.15.72:0/3556257199 <== mon.0 v2:172.21.15.72:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7f0fa000f9c0 con 0x559fcecdfbb0 2024-04-26T21:56:03.593 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.589+0000 7f0fb0e4c740 1 -- 172.21.15.72:0/3556257199 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0fa0016040 con 0x559fcecdfbb0 2024-04-26T21:56:03.593 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.590+0000 7f0f94c00640 1 -- 172.21.15.72:0/3556257199 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0fa001de10 con 0x559fcecdfbb0 2024-04-26T21:56:03.593 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.590+0000 7f0fb0e4c740 10 client.15639 fetch_fsmap learned FSMap version 27 2024-04-26T21:56:03.594 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.590+0000 7f0fb0e4c740 1 -- 172.21.15.72:0/3556257199 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559fced5df00 con 0x559fcecdfbb0 2024-04-26T21:56:03.594 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.590+0000 7f0f94c00640 1 -- 172.21.15.72:0/3556257199 <== mon.0 v2:172.21.15.72:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0fa0016040 con 0x559fcecdfbb0 2024-04-26T21:56:03.594 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.590+0000 7f0f94c00640 1 -- 172.21.15.72:0/3556257199 <== mon.0 v2:172.21.15.72:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f0fa0026b70 con 0x559fcecdfbb0 2024-04-26T21:56:03.594 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.590+0000 7f0fb0e4c740 10 client.15639 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T21:56:03.594 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.590+0000 7f0fb0e4c740 10 client.15639 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:56:03.594 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.590+0000 7f0fb0e4c740 20 client.15639 populate_metadata read hostname 'smithi072' 2024-04-26T21:56:03.594 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.590+0000 7f0fb0e4c740 1 --2- 172.21.15.72:0/3556257199 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x559fced5e8e0 0x559fced60d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:03.594 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.590+0000 7f0fb0e4c740 4 client.15639 mds_command: new command op to 34517 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-26T21:56:03.594 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.590+0000 7f0fb0e4c740 1 -- 172.21.15.72:0/3556257199 --> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559fcec456f0 con 0x559fced5e8e0 2024-04-26T21:56:03.594 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.591+0000 7f0fa7400640 1 --2- 172.21.15.72:0/3556257199 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x559fced5e8e0 0x559fced60d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:03.595 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.592+0000 7f0fa7400640 1 --2- 172.21.15.72:0/3556257199 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x559fced5e8e0 0x559fced60d00 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9da254d12256264c server_cookie=62584dbe1d50600b in_seq=0 out_seq=0 2024-04-26T21:56:03.596 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.592+0000 7f0f94c00640 10 client.15639 ms_handle_connect on v2:172.21.15.76:6838/327145636 2024-04-26T21:56:03.596 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.592+0000 7f0f94c00640 1 -- 172.21.15.72:0/3556257199 <== mds.0 v2:172.21.15.76:6838/327145636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559fcec456f0 con 0x559fced5e8e0 2024-04-26T21:56:03.596 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.592+0000 7f0f94c00640 10 client.15639 handle_command_reply: tid=0 2024-04-26T21:56:03.596 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.592+0000 7f0fb0e4c740 1 client.15639 shutdown 2024-04-26T21:56:03.596 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.592+0000 7f0fb0e4c740 1 -- 172.21.15.72:0/3556257199 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559fcecdfbb0 msgr2=0x559fced5c8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:03.596 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.592+0000 7f0fb0e4c740 1 --2- 172.21.15.72:0/3556257199 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559fcecdfbb0 0x559fced5c8c0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f0fa0012040 tx=0x7f0fa001db60 comp rx=0 tx=0).stop 2024-04-26T21:56:03.596 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.592+0000 7f0fb0e4c740 1 -- 172.21.15.72:0/3556257199 shutdown_connections 2024-04-26T21:56:03.596 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.592+0000 7f0fb0e4c740 1 --2- 172.21.15.72:0/3556257199 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x559fced5e8e0 0x559fced60d00 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:03.596 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.592+0000 7f0fb0e4c740 1 --2- 172.21.15.72:0/3556257199 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559fcecdfbb0 0x559fced5c8c0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:03.596 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.592+0000 7f0fb0e4c740 1 --2- 172.21.15.72:0/3556257199 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559fcecdaa40 0x559fcece89d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:03.596 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.592+0000 7f0f94c00640 0 client.15639 ms_handle_reset on v2:172.21.15.76:6838/327145636 2024-04-26T21:56:03.596 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.592+0000 7f0fb0e4c740 1 --2- 172.21.15.72:0/3556257199 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559fcecda050 0x559fcece8490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:03.596 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.592+0000 7f0fb0e4c740 1 -- 172.21.15.72:0/3556257199 >> 172.21.15.72:0/3556257199 conn(0x559fcecd56c0 msgr2=0x559fcecd47b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:56:03.596 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.593+0000 7f0fb0e4c740 1 -- 172.21.15.72:0/3556257199 shutdown_connections 2024-04-26T21:56:03.597 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.593+0000 7f0fb0e4c740 1 -- 172.21.15.72:0/3556257199 wait complete. 2024-04-26T21:56:03.597 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.593+0000 7f0fb0e4c740 20 client.15639 trim_cache size 0 max 16384 2024-04-26T21:56:03.597 INFO:teuthology.orchestra.run.smithi072.stdout:[] 2024-04-26T21:56:03.598 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.593+0000 7f0faca00640 1 -- 172.21.15.72:0/445820685 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f0f8c0a3c40 msgr2=0x7f0f8c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:03.598 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.593+0000 7f0faca00640 1 --2- 172.21.15.72:0/445820685 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f0f8c0a3c40 0x7f0f8c0a6100 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x559fcecdb2a0 tx=0x7f0fa0008330 comp rx=0 tx=0).stop 2024-04-26T21:56:03.598 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.593+0000 7f0faca00640 1 -- 172.21.15.72:0/445820685 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0fa8104a70 msgr2=0x7f0fa819ad50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:03.598 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.593+0000 7f0faca00640 1 --2- 172.21.15.72:0/445820685 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0fa8104a70 0x7f0fa819ad50 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f0f9000b450 tx=0x7f0f9000b910 comp rx=0 tx=0).stop 2024-04-26T21:56:03.598 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.593+0000 7f0faca00640 1 -- 172.21.15.72:0/445820685 shutdown_connections 2024-04-26T21:56:03.598 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.593+0000 7f0faca00640 1 --2- 172.21.15.72:0/445820685 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0fa8106620 0x7f0fa81a2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:03.598 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.593+0000 7f0faca00640 1 --2- 172.21.15.72:0/445820685 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f0f8c0a3c40 0x7f0f8c0a6100 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:03.598 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.593+0000 7f0faca00640 1 --2- 172.21.15.72:0/445820685 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0fa8105c60 0x7f0fa819b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:03.598 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.593+0000 7f0faca00640 1 --2- 172.21.15.72:0/445820685 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0fa8104a70 0x7f0fa819ad50 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:03.599 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.594+0000 7f0faca00640 1 -- 172.21.15.72:0/445820685 >> 172.21.15.72:0/445820685 conn(0x7f0fa80fffe0 msgr2=0x7f0fa8109070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:56:03.599 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.594+0000 7f0faca00640 1 -- 172.21.15.72:0/445820685 shutdown_connections 2024-04-26T21:56:03.599 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:03.594+0000 7f0faca00640 1 -- 172.21.15.72:0/445820685 wait complete. 2024-04-26T21:56:04.612 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-26T21:56:04.612 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:56:04.612 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-26T21:56:04.828 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.828+0000 7f28be400640 1 -- 172.21.15.72:0/2493632149 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f28b8105c60 msgr2=0x7f28b81060e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:04.828 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.828+0000 7f28be400640 1 --2- 172.21.15.72:0/2493632149 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f28b8105c60 0x7f28b81060e0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f28a4009900 tx=0x7f28a402a410 comp rx=0 tx=0).stop 2024-04-26T21:56:04.828 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.828+0000 7f28be400640 1 -- 172.21.15.72:0/2493632149 shutdown_connections 2024-04-26T21:56:04.828 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.828+0000 7f28be400640 1 --2- 172.21.15.72:0/2493632149 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f28b8106620 0x7f28b810ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:04.829 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.828+0000 7f28be400640 1 --2- 172.21.15.72:0/2493632149 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f28b8105c60 0x7f28b81060e0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:04.829 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.828+0000 7f28be400640 1 --2- 172.21.15.72:0/2493632149 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f28b8104a70 0x7f28b8104e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:04.829 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.828+0000 7f28be400640 1 -- 172.21.15.72:0/2493632149 >> 172.21.15.72:0/2493632149 conn(0x7f28b80fffe0 msgr2=0x7f28b8102420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:56:04.829 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.828+0000 7f28be400640 1 -- 172.21.15.72:0/2493632149 shutdown_connections 2024-04-26T21:56:04.829 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.828+0000 7f28be400640 1 -- 172.21.15.72:0/2493632149 wait complete. 2024-04-26T21:56:04.831 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.829+0000 7f28be400640 1 Processor -- start 2024-04-26T21:56:04.831 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.829+0000 7f28be400640 1 -- start start 2024-04-26T21:56:04.831 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.829+0000 7f28be400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f28b8104a70 0x7f28b819abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:04.831 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.829+0000 7f28be400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f28b8105c60 0x7f28b819b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:04.831 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.829+0000 7f28be400640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f28b8106620 0x7f28b81a2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:04.831 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.829+0000 7f28be400640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f28b81104a0 con 0x7f28b8104a70 2024-04-26T21:56:04.831 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.829+0000 7f28be400640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f28b8110320 con 0x7f28b8106620 2024-04-26T21:56:04.831 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.829+0000 7f28be400640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f28b8110620 con 0x7f28b8105c60 2024-04-26T21:56:04.832 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.830+0000 7f28bd000640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f28b8104a70 0x7f28b819abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:04.832 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.830+0000 7f28bd000640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f28b8104a70 0x7f28b819abc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:37660/0 (socket says 172.21.15.72:37660) 2024-04-26T21:56:04.832 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.830+0000 7f28bd000640 1 -- 172.21.15.72:0/2296475118 learned_addr learned my addr 172.21.15.72:0/2296475118 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:56:04.832 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.830+0000 7f28bda00640 1 --2- 172.21.15.72:0/2296475118 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f28b8106620 0x7f28b81a2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:04.832 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.830+0000 7f28bd000640 1 -- 172.21.15.72:0/2296475118 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f28b8105c60 msgr2=0x7f28b819b100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:04.832 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.830+0000 7f28bd000640 1 --2- 172.21.15.72:0/2296475118 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f28b8105c60 0x7f28b819b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:04.832 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.830+0000 7f28bd000640 1 -- 172.21.15.72:0/2296475118 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f28b8106620 msgr2=0x7f28b81a2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:04.832 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.830+0000 7f28bd000640 1 --2- 172.21.15.72:0/2296475118 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f28b8106620 0x7f28b81a2170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:04.832 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.830+0000 7f28bd000640 1 -- 172.21.15.72:0/2296475118 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28b81a27e0 con 0x7f28b8104a70 2024-04-26T21:56:04.832 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.830+0000 7f28bda00640 1 --2- 172.21.15.72:0/2296475118 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f28b8106620 0x7f28b81a2170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:56:04.832 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.830+0000 7f28bd000640 1 --2- 172.21.15.72:0/2296475118 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f28b8104a70 0x7f28b819abc0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f28ac00d770 tx=0x7f28ac00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:56:04.832 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.831+0000 7f28b5600640 1 -- 172.21.15.72:0/2296475118 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28ac017070 con 0x7f28b8104a70 2024-04-26T21:56:04.832 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.831+0000 7f28b5600640 1 -- 172.21.15.72:0/2296475118 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f28ac009c30 con 0x7f28b8104a70 2024-04-26T21:56:04.832 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.831+0000 7f28b5600640 1 -- 172.21.15.72:0/2296475118 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28ac003c50 con 0x7f28b8104a70 2024-04-26T21:56:04.832 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.831+0000 7f28be400640 1 -- 172.21.15.72:0/2296475118 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28b81a2ad0 con 0x7f28b8104a70 2024-04-26T21:56:04.833 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.831+0000 7f28be400640 1 -- 172.21.15.72:0/2296475118 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28b81a3070 con 0x7f28b8104a70 2024-04-26T21:56:04.834 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.834+0000 7f28c2877740 1 Processor -- start 2024-04-26T21:56:04.834 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.834+0000 7f28c2877740 1 -- start start 2024-04-26T21:56:04.834 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.834+0000 7f28b5600640 1 -- 172.21.15.72:0/2296475118 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f28ac003df0 con 0x7f28b8104a70 2024-04-26T21:56:04.834 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.834+0000 7f28c2877740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5616d9ee7a70 0x5616d9ee7e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:04.834 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.834+0000 7f28c2877740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5616d9ee8460 0x5616d9eecf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:04.835 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.834+0000 7f28c2877740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5616d9eed5d0 0x5616d9eefa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:04.835 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.834+0000 7f28c2877740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x5616d9e4c980 con 0x5616d9eed5d0 2024-04-26T21:56:04.835 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.834+0000 7f28c2877740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5616d9de9220 con 0x5616d9ee7a70 2024-04-26T21:56:04.835 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.834+0000 7f28c2877740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5616d9e48940 con 0x5616d9ee8460 2024-04-26T21:56:04.837 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.835+0000 7f28b7e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5616d9eed5d0 0x5616d9eefa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:04.837 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.835+0000 7f28b7e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5616d9eed5d0 0x5616d9eefa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:37666/0 (socket says 172.21.15.72:37666) 2024-04-26T21:56:04.837 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.835+0000 7f28b7e00640 1 -- 172.21.15.72:0/3415423171 learned_addr learned my addr 172.21.15.72:0/3415423171 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:56:04.838 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.835+0000 7f28bd000640 1 --2- 172.21.15.72:0/3415423171 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5616d9ee8460 0x5616d9eecf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:04.838 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.835+0000 7f28b7e00640 1 -- 172.21.15.72:0/3415423171 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5616d9ee8460 msgr2=0x5616d9eecf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:56:04.838 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.835+0000 7f28b7e00640 1 --2- 172.21.15.72:0/3415423171 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5616d9ee8460 0x5616d9eecf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:04.838 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.835+0000 7f28b7e00640 1 -- 172.21.15.72:0/3415423171 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5616d9ee7a70 msgr2=0x5616d9ee7e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:56:04.838 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.835+0000 7f28b7e00640 1 --2- 172.21.15.72:0/3415423171 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5616d9ee7a70 0x5616d9ee7e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:04.838 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.835+0000 7f28b7e00640 1 -- 172.21.15.72:0/3415423171 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5616d9c267e0 con 0x5616d9eed5d0 2024-04-26T21:56:04.838 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.835+0000 7f28b7e00640 1 --2- 172.21.15.72:0/3415423171 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5616d9eed5d0 0x5616d9eefa60 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f28a402aa00 tx=0x7f28a40040f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0ffa345ba752975 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:56:04.838 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.836+0000 7f28aa000640 1 -- 172.21.15.72:0/3415423171 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28a403f070 con 0x5616d9eed5d0 2024-04-26T21:56:04.838 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.836+0000 7f28b5600640 1 --2- 172.21.15.72:0/2296475118 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f28a00a3cc0 0x7f28a00a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:04.838 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.836+0000 7f28aa000640 1 -- 172.21.15.72:0/3415423171 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f28a4003a80 con 0x5616d9eed5d0 2024-04-26T21:56:04.838 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.836+0000 7f28aa000640 1 -- 172.21.15.72:0/3415423171 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28a4035030 con 0x5616d9eed5d0 2024-04-26T21:56:04.838 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.836+0000 7f28b5600640 1 -- 172.21.15.72:0/2296475118 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7f28ac0d0050 con 0x7f28b8104a70 2024-04-26T21:56:04.838 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.836+0000 7f28bda00640 1 --2- 172.21.15.72:0/2296475118 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f28a00a3cc0 0x7f28a00a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:04.839 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.837+0000 7f28c2877740 1 -- 172.21.15.72:0/3415423171 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5616d9eed5d0 msgr2=0x5616d9eefa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:04.839 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.837+0000 7f28c2877740 1 --2- 172.21.15.72:0/3415423171 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5616d9eed5d0 0x5616d9eefa60 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f28a402aa00 tx=0x7f28a40040f0 comp rx=0 tx=0).stop 2024-04-26T21:56:04.839 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.837+0000 7f28c2877740 1 -- 172.21.15.72:0/3415423171 shutdown_connections 2024-04-26T21:56:04.839 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.837+0000 7f28c2877740 1 --2- 172.21.15.72:0/3415423171 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5616d9eed5d0 0x5616d9eefa60 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:04.839 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.837+0000 7f28c2877740 1 --2- 172.21.15.72:0/3415423171 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5616d9ee8460 0x5616d9eecf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:04.839 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.837+0000 7f28bda00640 1 --2- 172.21.15.72:0/2296475118 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f28a00a3cc0 0x7f28a00a6180 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x5616d9ee8cc0 tx=0x7f28b0009440 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:56:04.839 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.837+0000 7f28c2877740 1 --2- 172.21.15.72:0/3415423171 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5616d9ee7a70 0x5616d9ee7e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:04.839 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.837+0000 7f28c2877740 1 -- 172.21.15.72:0/3415423171 >> 172.21.15.72:0/3415423171 conn(0x5616d9ee30e0 msgr2=0x5616d9ee34e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:56:04.839 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.837+0000 7f28c2877740 1 -- 172.21.15.72:0/3415423171 shutdown_connections 2024-04-26T21:56:04.839 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.837+0000 7f28c2877740 1 -- 172.21.15.72:0/3415423171 wait complete. 2024-04-26T21:56:04.839 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.837+0000 7f28c2877740 1 -- start start 2024-04-26T21:56:04.839 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.838+0000 7f28c2877740 1 Processor -- start 2024-04-26T21:56:04.842 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.838+0000 7f28c2877740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5616d9ee7a70 0x5616d9ef1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:04.843 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.838+0000 7f28c2877740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5616d9ee8460 0x5616d9ef2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:04.843 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.838+0000 7f28c2877740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5616d9eed5d0 0x5616d9ef9440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:04.843 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.838+0000 7f28c2877740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x5616d9ef0520 con 0x5616d9ee7a70 2024-04-26T21:56:04.843 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.838+0000 7f28c2877740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5616d9dd4170 con 0x5616d9ee8460 2024-04-26T21:56:04.843 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.838+0000 7f28c2877740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5616d9ef06a0 con 0x5616d9eed5d0 2024-04-26T21:56:04.843 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.838+0000 7f28c2877740 10 client.0 _ll_register_callbacks cb 0x5616d9e43770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:56:04.843 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.838+0000 7f28bd000640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5616d9ee7a70 0x5616d9ef1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:04.843 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.838+0000 7f28bd000640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5616d9ee7a70 0x5616d9ef1ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:37682/0 (socket says 172.21.15.72:37682) 2024-04-26T21:56:04.843 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.838+0000 7f28bd000640 1 -- 172.21.15.72:0/2623703687 learned_addr learned my addr 172.21.15.72:0/2623703687 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:56:04.843 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.839+0000 7f28bda00640 1 --2- 172.21.15.72:0/2623703687 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5616d9eed5d0 0x5616d9ef9440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:04.843 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.839+0000 7f28b7e00640 1 --2- 172.21.15.72:0/2623703687 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5616d9ee8460 0x5616d9ef2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:04.843 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.839+0000 7f28bd000640 1 -- 172.21.15.72:0/2623703687 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5616d9eed5d0 msgr2=0x5616d9ef9440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:56:04.843 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.839+0000 7f28bd000640 1 --2- 172.21.15.72:0/2623703687 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5616d9eed5d0 0x5616d9ef9440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:04.844 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.839+0000 7f28bd000640 1 -- 172.21.15.72:0/2623703687 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5616d9ee8460 msgr2=0x5616d9ef2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:56:04.844 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.839+0000 7f28bd000640 1 --2- 172.21.15.72:0/2623703687 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5616d9ee8460 0x5616d9ef2400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:04.844 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.839+0000 7f28bd000640 1 -- 172.21.15.72:0/2623703687 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5616d9c86f70 con 0x5616d9ee7a70 2024-04-26T21:56:04.844 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.839+0000 7f28b7e00640 1 --2- 172.21.15.72:0/2623703687 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5616d9ee8460 0x5616d9ef2400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:56:04.844 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.839+0000 7f28bd000640 1 --2- 172.21.15.72:0/2623703687 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5616d9ee7a70 0x5616d9ef1ec0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f28ac09be50 tx=0x7f28ac09bc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c59bb373cbd95512 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:56:04.844 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.839+0000 7f289ea00640 10 client.0 ms_handle_connect on v2:172.21.15.72:3300/0 2024-04-26T21:56:04.844 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.839+0000 7f289ea00640 1 -- 172.21.15.72:0/2623703687 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28ac020070 con 0x5616d9ee7a70 2024-04-26T21:56:04.844 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.839+0000 7f289ea00640 1 -- 172.21.15.72:0/2623703687 <== mon.0 v2:172.21.15.72:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7f28ac0a0120 con 0x5616d9ee7a70 2024-04-26T21:56:04.844 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.839+0000 7f28c2877740 1 -- 172.21.15.72:0/2623703687 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f28a4003a80 con 0x5616d9ee7a70 2024-04-26T21:56:04.844 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.840+0000 7f289ea00640 1 -- 172.21.15.72:0/2623703687 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f28ac009de0 con 0x5616d9ee7a70 2024-04-26T21:56:04.844 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.840+0000 7f28c2877740 10 client.15651 fetch_fsmap learned FSMap version 27 2024-04-26T21:56:04.844 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.840+0000 7f28c2877740 1 -- 172.21.15.72:0/2623703687 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5616d9b1e170 con 0x5616d9ee7a70 2024-04-26T21:56:04.844 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.840+0000 7f289ea00640 1 -- 172.21.15.72:0/2623703687 <== mon.0 v2:172.21.15.72:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f28a4003a80 con 0x5616d9ee7a70 2024-04-26T21:56:04.844 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.840+0000 7f289ea00640 1 -- 172.21.15.72:0/2623703687 <== mon.0 v2:172.21.15.72:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f28ac0d20d0 con 0x5616d9ee7a70 2024-04-26T21:56:04.844 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.840+0000 7f28c2877740 10 client.15651 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T21:56:04.845 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.840+0000 7f28c2877740 10 client.15651 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:56:04.845 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.840+0000 7f28c2877740 20 client.15651 populate_metadata read hostname 'smithi072' 2024-04-26T21:56:04.845 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.840+0000 7f28c2877740 1 --2- 172.21.15.72:0/2623703687 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x5616d9efd8a0 0x5616d9effd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:04.845 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.840+0000 7f28c2877740 4 client.15651 mds_command: new command op to 34517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:56:04.845 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.840+0000 7f28c2877740 1 -- 172.21.15.72:0/2623703687 --> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5616d9e5a590 con 0x5616d9efd8a0 2024-04-26T21:56:04.845 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.841+0000 7f28b7e00640 1 --2- 172.21.15.72:0/2623703687 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x5616d9efd8a0 0x5616d9effd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:04.845 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.842+0000 7f28b7e00640 1 --2- 172.21.15.72:0/2623703687 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x5616d9efd8a0 0x5616d9effd30 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f2949a4da9ee0ddd server_cookie=c002178392a66bf4 in_seq=0 out_seq=0 2024-04-26T21:56:04.845 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.842+0000 7f289ea00640 10 client.15651 ms_handle_connect on v2:172.21.15.76:6838/327145636 2024-04-26T21:56:04.846 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.844+0000 7f289ea00640 1 -- 172.21.15.72:0/2623703687 <== mds.0 v2:172.21.15.76:6838/327145636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5616d9e5a590 con 0x5616d9efd8a0 2024-04-26T21:56:04.846 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.844+0000 7f289ea00640 10 client.15651 handle_command_reply: tid=0 2024-04-26T21:56:04.846 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.844+0000 7f28c2877740 1 client.15651 shutdown 2024-04-26T21:56:04.846 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.844+0000 7f28c2877740 1 -- 172.21.15.72:0/2623703687 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5616d9ee7a70 msgr2=0x5616d9ef1ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:04.847 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.844+0000 7f28c2877740 1 --2- 172.21.15.72:0/2623703687 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5616d9ee7a70 0x5616d9ef1ec0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f28ac09be50 tx=0x7f28ac09bc90 comp rx=0 tx=0).stop 2024-04-26T21:56:04.847 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.844+0000 7f28c2877740 1 -- 172.21.15.72:0/2623703687 shutdown_connections 2024-04-26T21:56:04.847 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.844+0000 7f28c2877740 1 --2- 172.21.15.72:0/2623703687 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x5616d9efd8a0 0x5616d9effd30 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:04.847 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.844+0000 7f28c2877740 1 --2- 172.21.15.72:0/2623703687 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5616d9eed5d0 0x5616d9ef9440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:04.847 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.844+0000 7f28c2877740 1 --2- 172.21.15.72:0/2623703687 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5616d9ee8460 0x5616d9ef2400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:04.847 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.844+0000 7f28c2877740 1 --2- 172.21.15.72:0/2623703687 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5616d9ee7a70 0x5616d9ef1ec0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:04.847 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.844+0000 7f28c2877740 1 -- 172.21.15.72:0/2623703687 >> 172.21.15.72:0/2623703687 conn(0x5616d9ee30e0 msgr2=0x5616d9ee22e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:56:04.847 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.844+0000 7f289ea00640 0 client.15651 ms_handle_reset on v2:172.21.15.76:6838/327145636 2024-04-26T21:56:04.847 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.844+0000 7f28c2877740 1 -- 172.21.15.72:0/2623703687 shutdown_connections 2024-04-26T21:56:04.847 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.844+0000 7f28c2877740 1 -- 172.21.15.72:0/2623703687 wait complete. 2024-04-26T21:56:04.847 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.844+0000 7f28c2877740 20 client.15651 trim_cache size 0 max 16384 2024-04-26T21:56:04.863 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.863+0000 7f28c2877740 1 Processor -- start 2024-04-26T21:56:04.863 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.863+0000 7f28c2877740 1 -- start start 2024-04-26T21:56:04.864 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.864+0000 7f28c2877740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5616d9ee7a70 0x5616d9ef1180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:04.864 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.864+0000 7f28c2877740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5616d9ee8460 0x5616d9ef9400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:04.864 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.864+0000 7f28c2877740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5616d9eed5d0 0x5616d9ee6090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:04.864 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.864+0000 7f28c2877740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f28b81abf20 con 0x5616d9ee7a70 2024-04-26T21:56:04.864 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.864+0000 7f28c2877740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f28b81ada60 con 0x5616d9eed5d0 2024-04-26T21:56:04.864 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.864+0000 7f28c2877740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f28b81ac5f0 con 0x5616d9ee8460 2024-04-26T21:56:04.866 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.864+0000 7f28bd000640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5616d9ee7a70 0x5616d9ef1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:04.866 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.864+0000 7f28bd000640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5616d9ee7a70 0x5616d9ef1180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:37688/0 (socket says 172.21.15.72:37688) 2024-04-26T21:56:04.866 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.864+0000 7f28bd000640 1 -- 172.21.15.72:0/3829908996 learned_addr learned my addr 172.21.15.72:0/3829908996 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:56:04.866 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.864+0000 7f28b7e00640 1 --2- 172.21.15.72:0/3829908996 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5616d9ee8460 0x5616d9ef9400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:04.866 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.864+0000 7f28bd000640 1 -- 172.21.15.72:0/3829908996 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5616d9ee8460 msgr2=0x5616d9ef9400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:56:04.866 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.864+0000 7f28bd000640 1 --2- 172.21.15.72:0/3829908996 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5616d9ee8460 0x5616d9ef9400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:04.866 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.864+0000 7f28bd000640 1 -- 172.21.15.72:0/3829908996 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5616d9eed5d0 msgr2=0x5616d9ee6090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:56:04.866 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.864+0000 7f28bda00640 1 --2- 172.21.15.72:0/3829908996 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5616d9eed5d0 0x5616d9ee6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:04.867 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.864+0000 7f28bd000640 1 --2- 172.21.15.72:0/3829908996 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5616d9eed5d0 0x5616d9ee6090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:04.867 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.864+0000 7f28bd000640 1 -- 172.21.15.72:0/3829908996 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5616d9ee93b0 con 0x5616d9ee7a70 2024-04-26T21:56:04.867 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.864+0000 7f28bda00640 1 --2- 172.21.15.72:0/3829908996 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5616d9eed5d0 0x5616d9ee6090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:56:04.867 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.865+0000 7f28bd000640 1 --2- 172.21.15.72:0/3829908996 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5616d9ee7a70 0x5616d9ef1180 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f28ac098560 tx=0x7f28ac09bdc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d408d434cf27e04e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:56:04.867 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.865+0000 7f28a9600640 1 -- 172.21.15.72:0/3829908996 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28ac020050 con 0x5616d9ee7a70 2024-04-26T21:56:04.867 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.865+0000 7f28a9600640 1 -- 172.21.15.72:0/3829908996 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f28ac015020 con 0x5616d9ee7a70 2024-04-26T21:56:04.867 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.865+0000 7f28a9600640 1 -- 172.21.15.72:0/3829908996 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28ac0a0110 con 0x5616d9ee7a70 2024-04-26T21:56:04.867 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.865+0000 7f28c2877740 1 -- 172.21.15.72:0/3829908996 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5616d9ee7a70 msgr2=0x5616d9ef1180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:04.867 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.865+0000 7f28c2877740 1 --2- 172.21.15.72:0/3829908996 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5616d9ee7a70 0x5616d9ef1180 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f28ac098560 tx=0x7f28ac09bdc0 comp rx=0 tx=0).stop 2024-04-26T21:56:04.867 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.866+0000 7f28c2877740 1 -- 172.21.15.72:0/3829908996 shutdown_connections 2024-04-26T21:56:04.867 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.866+0000 7f28c2877740 1 --2- 172.21.15.72:0/3829908996 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5616d9eed5d0 0x5616d9ee6090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:04.867 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.866+0000 7f28c2877740 1 --2- 172.21.15.72:0/3829908996 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5616d9ee8460 0x5616d9ef9400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:04.867 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.866+0000 7f28c2877740 1 --2- 172.21.15.72:0/3829908996 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5616d9ee7a70 0x5616d9ef1180 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:04.867 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.866+0000 7f28c2877740 1 -- 172.21.15.72:0/3829908996 >> 172.21.15.72:0/3829908996 conn(0x5616d9ee30e0 msgr2=0x5616d9ef0a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:56:04.867 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.866+0000 7f28c2877740 1 -- 172.21.15.72:0/3829908996 shutdown_connections 2024-04-26T21:56:04.868 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.866+0000 7f28c2877740 1 -- 172.21.15.72:0/3829908996 wait complete. 2024-04-26T21:56:04.868 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.866+0000 7f28c2877740 1 -- start start 2024-04-26T21:56:04.868 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.866+0000 7f28c2877740 1 Processor -- start 2024-04-26T21:56:04.870 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.867+0000 7f28c2877740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5616d9ee7a70 0x5616d9ef6640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:04.870 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.867+0000 7f28c2877740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5616d9ee8460 0x5616d9ef6b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:04.870 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.867+0000 7f28c2877740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5616d9eed5d0 0x5616d9ef7b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:04.870 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.867+0000 7f28c2877740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f28b81ad390 con 0x5616d9ee7a70 2024-04-26T21:56:04.870 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.867+0000 7f28c2877740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f28b81accc0 con 0x5616d9ee8460 2024-04-26T21:56:04.871 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.867+0000 7f28c2877740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f28b81a9f00 con 0x5616d9eed5d0 2024-04-26T21:56:04.871 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.867+0000 7f28c2877740 10 client.0 _ll_register_callbacks cb 0x7f28ac0046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:56:04.871 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.867+0000 7f28bd000640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5616d9ee7a70 0x5616d9ef6640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:04.871 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.867+0000 7f28bd000640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5616d9ee7a70 0x5616d9ef6640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:37696/0 (socket says 172.21.15.72:37696) 2024-04-26T21:56:04.871 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.867+0000 7f28bd000640 1 -- 172.21.15.72:0/4178418658 learned_addr learned my addr 172.21.15.72:0/4178418658 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:56:04.871 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.867+0000 7f28b7e00640 1 --2- 172.21.15.72:0/4178418658 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5616d9ee8460 0x5616d9ef6b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:04.871 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.867+0000 7f28bd000640 1 -- 172.21.15.72:0/4178418658 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5616d9eed5d0 msgr2=0x5616d9ef7b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:56:04.871 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.867+0000 7f28bd000640 1 --2- 172.21.15.72:0/4178418658 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5616d9eed5d0 0x5616d9ef7b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:04.871 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.867+0000 7f28bd000640 1 -- 172.21.15.72:0/4178418658 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5616d9ee8460 msgr2=0x5616d9ef6b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:56:04.871 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.867+0000 7f28bd000640 1 --2- 172.21.15.72:0/4178418658 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5616d9ee8460 0x5616d9ef6b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:04.871 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.867+0000 7f28bd000640 1 -- 172.21.15.72:0/4178418658 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5616d9ef8180 con 0x5616d9ee7a70 2024-04-26T21:56:04.871 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.867+0000 7f28bd000640 1 --2- 172.21.15.72:0/4178418658 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5616d9ee7a70 0x5616d9ef6640 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f28ac0a4890 tx=0x7f28ac0cdde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=540bf27a688a23b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:56:04.871 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.867+0000 7f289ea00640 10 client.0 ms_handle_connect on v2:172.21.15.72:3300/0 2024-04-26T21:56:04.871 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.868+0000 7f289ea00640 1 -- 172.21.15.72:0/4178418658 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28ac020070 con 0x5616d9ee7a70 2024-04-26T21:56:04.871 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.868+0000 7f289ea00640 1 -- 172.21.15.72:0/4178418658 <== mon.0 v2:172.21.15.72:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7f28ac0d3140 con 0x5616d9ee7a70 2024-04-26T21:56:04.872 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.868+0000 7f28c2877740 1 -- 172.21.15.72:0/4178418658 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f28ac015020 con 0x5616d9ee7a70 2024-04-26T21:56:04.872 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.868+0000 7f289ea00640 1 -- 172.21.15.72:0/4178418658 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f28ac0a0500 con 0x5616d9ee7a70 2024-04-26T21:56:04.872 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.868+0000 7f28c2877740 10 client.15657 fetch_fsmap learned FSMap version 27 2024-04-26T21:56:04.872 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.868+0000 7f28c2877740 1 -- 172.21.15.72:0/4178418658 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5616d9ef9430 con 0x5616d9ee7a70 2024-04-26T21:56:04.872 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.868+0000 7f289ea00640 1 -- 172.21.15.72:0/4178418658 <== mon.0 v2:172.21.15.72:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f28ac015020 con 0x5616d9ee7a70 2024-04-26T21:56:04.872 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.868+0000 7f289ea00640 1 -- 172.21.15.72:0/4178418658 <== mon.0 v2:172.21.15.72:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f28ac004270 con 0x5616d9ee7a70 2024-04-26T21:56:04.872 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.868+0000 7f28c2877740 10 client.15657 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T21:56:04.872 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.868+0000 7f28c2877740 10 client.15657 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:56:04.872 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.868+0000 7f28c2877740 20 client.15657 populate_metadata read hostname 'smithi072' 2024-04-26T21:56:04.872 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.868+0000 7f28c2877740 1 --2- 172.21.15.72:0/4178418658 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x5616d9f640d0 0x5616d9f664f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:04.872 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.868+0000 7f28c2877740 4 client.15657 mds_command: new command op to 34517 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-26T21:56:04.872 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.868+0000 7f28c2877740 1 -- 172.21.15.72:0/4178418658 --> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5616d9e54400 con 0x5616d9f640d0 2024-04-26T21:56:04.872 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.869+0000 7f28b7e00640 1 --2- 172.21.15.72:0/4178418658 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x5616d9f640d0 0x5616d9f664f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:04.872 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.870+0000 7f28b7e00640 1 --2- 172.21.15.72:0/4178418658 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x5616d9f640d0 0x5616d9f664f0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a047201d3a00d0fa server_cookie=3d3e40a00f119204 in_seq=0 out_seq=0 2024-04-26T21:56:04.873 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.870+0000 7f289ea00640 10 client.15657 ms_handle_connect on v2:172.21.15.76:6838/327145636 2024-04-26T21:56:04.874 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.872+0000 7f289ea00640 1 -- 172.21.15.72:0/4178418658 <== mds.0 v2:172.21.15.76:6838/327145636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5616d9e54400 con 0x5616d9f640d0 2024-04-26T21:56:04.874 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.872+0000 7f289ea00640 10 client.15657 handle_command_reply: tid=0 2024-04-26T21:56:04.874 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.872+0000 7f28c2877740 1 client.15657 shutdown 2024-04-26T21:56:04.874 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.873+0000 7f28c2877740 1 -- 172.21.15.72:0/4178418658 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5616d9ee7a70 msgr2=0x5616d9ef6640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:04.874 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.873+0000 7f28c2877740 1 --2- 172.21.15.72:0/4178418658 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5616d9ee7a70 0x5616d9ef6640 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f28ac0a4890 tx=0x7f28ac0cdde0 comp rx=0 tx=0).stop 2024-04-26T21:56:04.874 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.873+0000 7f28c2877740 1 -- 172.21.15.72:0/4178418658 shutdown_connections 2024-04-26T21:56:04.874 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.873+0000 7f28c2877740 1 --2- 172.21.15.72:0/4178418658 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x5616d9f640d0 0x5616d9f664f0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:04.874 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.873+0000 7f28c2877740 1 --2- 172.21.15.72:0/4178418658 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5616d9eed5d0 0x5616d9ef7b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:04.874 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.873+0000 7f28c2877740 1 --2- 172.21.15.72:0/4178418658 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5616d9ee8460 0x5616d9ef6b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:04.874 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.873+0000 7f28c2877740 1 --2- 172.21.15.72:0/4178418658 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5616d9ee7a70 0x5616d9ef6640 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:04.874 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.873+0000 7f289ea00640 0 client.15657 ms_handle_reset on v2:172.21.15.76:6838/327145636 2024-04-26T21:56:04.874 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.873+0000 7f28c2877740 1 -- 172.21.15.72:0/4178418658 >> 172.21.15.72:0/4178418658 conn(0x5616d9ee30e0 msgr2=0x5616d9ee22e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:56:04.874 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.873+0000 7f28c2877740 1 -- 172.21.15.72:0/4178418658 shutdown_connections 2024-04-26T21:56:04.875 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.873+0000 7f28c2877740 1 -- 172.21.15.72:0/4178418658 wait complete. 2024-04-26T21:56:04.875 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.873+0000 7f28c2877740 20 client.15657 trim_cache size 0 max 16384 2024-04-26T21:56:04.875 INFO:teuthology.orchestra.run.smithi072.stdout:{ 2024-04-26T21:56:04.875 INFO:teuthology.orchestra.run.smithi072.stdout: "return_code": 0, 2024-04-26T21:56:04.875 INFO:teuthology.orchestra.run.smithi072.stdout: "scrub_tag": "98faa0cd-bf77-42fe-a1c6-bdb868ce4471", 2024-04-26T21:56:04.875 INFO:teuthology.orchestra.run.smithi072.stdout: "mode": "asynchronous" 2024-04-26T21:56:04.875 INFO:teuthology.orchestra.run.smithi072.stdout:} 2024-04-26T21:56:04.876 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.874+0000 7f28be400640 1 -- 172.21.15.72:0/2296475118 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f28a00a3cc0 msgr2=0x7f28a00a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:04.876 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.874+0000 7f28be400640 1 --2- 172.21.15.72:0/2296475118 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f28a00a3cc0 0x7f28a00a6180 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x5616d9ee8cc0 tx=0x7f28b0009440 comp rx=0 tx=0).stop 2024-04-26T21:56:04.876 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.874+0000 7f28be400640 1 -- 172.21.15.72:0/2296475118 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f28b8104a70 msgr2=0x7f28b819abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:04.876 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.874+0000 7f28be400640 1 --2- 172.21.15.72:0/2296475118 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f28b8104a70 0x7f28b819abc0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f28ac00d770 tx=0x7f28ac00dc30 comp rx=0 tx=0).stop 2024-04-26T21:56:04.876 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.874+0000 7f28be400640 1 -- 172.21.15.72:0/2296475118 shutdown_connections 2024-04-26T21:56:04.876 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.874+0000 7f28be400640 1 --2- 172.21.15.72:0/2296475118 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f28b8106620 0x7f28b81a2170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:04.876 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.874+0000 7f28be400640 1 --2- 172.21.15.72:0/2296475118 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f28a00a3cc0 0x7f28a00a6180 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:04.876 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.874+0000 7f28be400640 1 --2- 172.21.15.72:0/2296475118 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f28b8105c60 0x7f28b819b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:04.877 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.874+0000 7f28be400640 1 --2- 172.21.15.72:0/2296475118 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f28b8104a70 0x7f28b819abc0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:04.877 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.874+0000 7f28be400640 1 -- 172.21.15.72:0/2296475118 >> 172.21.15.72:0/2296475118 conn(0x7f28b80fffe0 msgr2=0x7f28b8109000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:56:04.877 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.874+0000 7f28be400640 1 -- 172.21.15.72:0/2296475118 shutdown_connections 2024-04-26T21:56:04.877 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:04.874+0000 7f28be400640 1 -- 172.21.15.72:0/2296475118 wait complete. 2024-04-26T21:56:04.892 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-26T21:56:04.892 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-26T21:56:05.102 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.101+0000 7fc9b9c00640 1 -- 172.21.15.72:0/1788758694 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc9b4109fc0 msgr2=0x7fc9b410a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:05.102 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.101+0000 7fc9b9c00640 1 --2- 172.21.15.72:0/1788758694 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc9b4109fc0 0x7fc9b410a440 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fc9a0009900 tx=0x7fc9a002a410 comp rx=0 tx=0).stop 2024-04-26T21:56:05.102 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.101+0000 7fc9b9c00640 1 -- 172.21.15.72:0/1788758694 shutdown_connections 2024-04-26T21:56:05.102 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.101+0000 7fc9b9c00640 1 --2- 172.21.15.72:0/1788758694 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc9b410a980 0x7fc9b41112a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:05.102 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.101+0000 7fc9b9c00640 1 --2- 172.21.15.72:0/1788758694 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc9b4109fc0 0x7fc9b410a440 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:05.102 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.101+0000 7fc9b9c00640 1 --2- 172.21.15.72:0/1788758694 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc9b4108e70 0x7fc9b4109290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:05.102 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.101+0000 7fc9b9c00640 1 -- 172.21.15.72:0/1788758694 >> 172.21.15.72:0/1788758694 conn(0x7fc9b4077e00 msgr2=0x7fc9b407a240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:56:05.102 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.102+0000 7fc9b9c00640 1 -- 172.21.15.72:0/1788758694 shutdown_connections 2024-04-26T21:56:05.102 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.102+0000 7fc9b9c00640 1 -- 172.21.15.72:0/1788758694 wait complete. 2024-04-26T21:56:05.104 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.103+0000 7fc9b9c00640 1 Processor -- start 2024-04-26T21:56:05.104 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.103+0000 7fc9b9c00640 1 -- start start 2024-04-26T21:56:05.105 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.103+0000 7fc9b9c00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc9b4108e70 0x7fc9b419ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:05.105 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.103+0000 7fc9b9c00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc9b4109fc0 0x7fc9b419f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:05.105 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.103+0000 7fc9b9c00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc9b410a980 0x7fc9b41a64e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:05.105 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.103+0000 7fc9b9c00640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fc9b4114850 con 0x7fc9b410a980 2024-04-26T21:56:05.105 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.103+0000 7fc9b9c00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fc9b41146d0 con 0x7fc9b4109fc0 2024-04-26T21:56:05.105 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.103+0000 7fc9b9c00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fc9b41149d0 con 0x7fc9b4108e70 2024-04-26T21:56:05.105 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.103+0000 7fc9b9200640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc9b410a980 0x7fc9b41a64e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:05.105 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.103+0000 7fc9b9200640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc9b410a980 0x7fc9b41a64e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:37714/0 (socket says 172.21.15.72:37714) 2024-04-26T21:56:05.105 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.103+0000 7fc9b9200640 1 -- 172.21.15.72:0/2190002952 learned_addr learned my addr 172.21.15.72:0/2190002952 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:56:05.105 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.104+0000 7fc9b3e00640 1 --2- 172.21.15.72:0/2190002952 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc9b4108e70 0x7fc9b419ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:05.105 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.104+0000 7fc9b3400640 1 --2- 172.21.15.72:0/2190002952 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc9b4109fc0 0x7fc9b419f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:05.105 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.104+0000 7fc9b9200640 1 -- 172.21.15.72:0/2190002952 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc9b4108e70 msgr2=0x7fc9b419ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:05.105 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.104+0000 7fc9b9200640 1 --2- 172.21.15.72:0/2190002952 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc9b4108e70 0x7fc9b419ef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:05.106 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.104+0000 7fc9b9200640 1 -- 172.21.15.72:0/2190002952 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc9b4109fc0 msgr2=0x7fc9b419f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:05.106 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.104+0000 7fc9b9200640 1 --2- 172.21.15.72:0/2190002952 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc9b4109fc0 0x7fc9b419f470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:05.106 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.104+0000 7fc9b9200640 1 -- 172.21.15.72:0/2190002952 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9b41a6b50 con 0x7fc9b410a980 2024-04-26T21:56:05.106 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.104+0000 7fc9b9200640 1 --2- 172.21.15.72:0/2190002952 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc9b410a980 0x7fc9b41a64e0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fc9ac00cce0 tx=0x7fc9ac00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:56:05.106 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.104+0000 7fc9b0c00640 1 -- 172.21.15.72:0/2190002952 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9ac00f450 con 0x7fc9b410a980 2024-04-26T21:56:05.106 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.104+0000 7fc9b0c00640 1 -- 172.21.15.72:0/2190002952 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc9ac00fa80 con 0x7fc9b410a980 2024-04-26T21:56:05.106 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.104+0000 7fc9b9c00640 1 -- 172.21.15.72:0/2190002952 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9b41a6e40 con 0x7fc9b410a980 2024-04-26T21:56:05.106 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.104+0000 7fc9b0c00640 1 -- 172.21.15.72:0/2190002952 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9ac018ad0 con 0x7fc9b410a980 2024-04-26T21:56:05.106 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.105+0000 7fc9b9c00640 1 -- 172.21.15.72:0/2190002952 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9b41a7300 con 0x7fc9b410a980 2024-04-26T21:56:05.108 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.108+0000 7fc9be1b9740 1 Processor -- start 2024-04-26T21:56:05.108 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.108+0000 7fc9b0c00640 1 -- 172.21.15.72:0/2190002952 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fc9ac00f5f0 con 0x7fc9b410a980 2024-04-26T21:56:05.108 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.108+0000 7fc9be1b9740 1 -- start start 2024-04-26T21:56:05.110 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.108+0000 7fc9be1b9740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556daa987960 0x556daa987d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:05.110 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.108+0000 7fc9be1b9740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556daa988350 0x556daa98ce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:05.110 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.108+0000 7fc9be1b9740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556daa98d4c0 0x556daa98f950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:05.110 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.108+0000 7fc9be1b9740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x556daa898030 con 0x556daa987960 2024-04-26T21:56:05.110 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.108+0000 7fc9be1b9740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x556daa8b37f0 con 0x556daa98d4c0 2024-04-26T21:56:05.110 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.108+0000 7fc9be1b9740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x556daa86e7e0 con 0x556daa988350 2024-04-26T21:56:05.110 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.109+0000 7fc9b3400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556daa987960 0x556daa987d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:05.110 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.109+0000 7fc9b3400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556daa987960 0x556daa987d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:37730/0 (socket says 172.21.15.72:37730) 2024-04-26T21:56:05.110 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.109+0000 7fc9b3400640 1 -- 172.21.15.72:0/370540831 learned_addr learned my addr 172.21.15.72:0/370540831 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:56:05.110 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.109+0000 7fc9b3e00640 1 --2- 172.21.15.72:0/370540831 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556daa988350 0x556daa98ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:05.110 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.109+0000 7fc9b3400640 1 -- 172.21.15.72:0/370540831 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556daa988350 msgr2=0x556daa98ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:56:05.110 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.109+0000 7fc9b3400640 1 --2- 172.21.15.72:0/370540831 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556daa988350 0x556daa98ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:05.111 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.109+0000 7fc9b3400640 1 -- 172.21.15.72:0/370540831 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556daa98d4c0 msgr2=0x556daa98f950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:56:05.111 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.109+0000 7fc9b3400640 1 --2- 172.21.15.72:0/370540831 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556daa98d4c0 0x556daa98f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:05.111 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.109+0000 7fc9b3400640 1 -- 172.21.15.72:0/370540831 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556daa87a050 con 0x556daa987960 2024-04-26T21:56:05.111 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.109+0000 7fc9b3400640 1 --2- 172.21.15.72:0/370540831 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556daa987960 0x556daa987d80 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fc9a0006010 tx=0x7fc9a0031620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8724283d910d7071 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:56:05.111 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.109+0000 7fc9b0c00640 1 --2- 172.21.15.72:0/2190002952 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fc9980a3cc0 0x7fc9980a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:05.111 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.110+0000 7fc9b0c00640 1 -- 172.21.15.72:0/2190002952 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7fc9ac0d0e80 con 0x7fc9b410a980 2024-04-26T21:56:05.111 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.110+0000 7fc9b3e00640 1 --2- 172.21.15.72:0/2190002952 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fc9980a3cc0 0x7fc9980a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:05.111 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.110+0000 7fc9a9600640 1 -- 172.21.15.72:0/370540831 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9a0035070 con 0x556daa987960 2024-04-26T21:56:05.111 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.110+0000 7fc9a9600640 1 -- 172.21.15.72:0/370540831 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc9a0004030 con 0x556daa987960 2024-04-26T21:56:05.111 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.110+0000 7fc9a9600640 1 -- 172.21.15.72:0/370540831 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9a0003ad0 con 0x556daa987960 2024-04-26T21:56:05.115 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.111+0000 7fc9be1b9740 1 -- 172.21.15.72:0/370540831 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556daa987960 msgr2=0x556daa987d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:05.115 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.111+0000 7fc9b3e00640 1 --2- 172.21.15.72:0/2190002952 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fc9980a3cc0 0x7fc9980a6180 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fc9a4005fd0 tx=0x7fc9a4005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:56:05.115 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.111+0000 7fc9be1b9740 1 --2- 172.21.15.72:0/370540831 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556daa987960 0x556daa987d80 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fc9a0006010 tx=0x7fc9a0031620 comp rx=0 tx=0).stop 2024-04-26T21:56:05.115 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.111+0000 7fc9be1b9740 1 -- 172.21.15.72:0/370540831 shutdown_connections 2024-04-26T21:56:05.115 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.111+0000 7fc9be1b9740 1 --2- 172.21.15.72:0/370540831 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556daa98d4c0 0x556daa98f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:05.115 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.111+0000 7fc9be1b9740 1 --2- 172.21.15.72:0/370540831 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556daa988350 0x556daa98ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:05.115 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.111+0000 7fc9be1b9740 1 --2- 172.21.15.72:0/370540831 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556daa987960 0x556daa987d80 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:05.115 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.111+0000 7fc9be1b9740 1 -- 172.21.15.72:0/370540831 >> 172.21.15.72:0/370540831 conn(0x556daa982fd0 msgr2=0x556daa9833d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:56:05.115 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.111+0000 7fc9be1b9740 1 -- 172.21.15.72:0/370540831 shutdown_connections 2024-04-26T21:56:05.115 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.111+0000 7fc9be1b9740 1 -- 172.21.15.72:0/370540831 wait complete. 2024-04-26T21:56:05.115 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.111+0000 7fc9be1b9740 1 -- start start 2024-04-26T21:56:05.115 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.111+0000 7fc9be1b9740 1 Processor -- start 2024-04-26T21:56:05.116 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.112+0000 7fc9be1b9740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556daa987960 0x556daa991780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:05.116 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.112+0000 7fc9be1b9740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556daa988350 0x556daa991cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:05.116 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.112+0000 7fc9be1b9740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556daa98d4c0 0x556daa998d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:05.116 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.112+0000 7fc9be1b9740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x556daa8f0200 con 0x556daa988350 2024-04-26T21:56:05.116 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.112+0000 7fc9be1b9740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x556daa8eaee0 con 0x556daa987960 2024-04-26T21:56:05.116 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.112+0000 7fc9be1b9740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x556daa8f7e70 con 0x556daa98d4c0 2024-04-26T21:56:05.116 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.112+0000 7fc9be1b9740 10 client.0 _ll_register_callbacks cb 0x556daa8e1270 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:56:05.116 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.112+0000 7fc9b3400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556daa988350 0x556daa991cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:05.116 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.112+0000 7fc9b3400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556daa988350 0x556daa991cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:37734/0 (socket says 172.21.15.72:37734) 2024-04-26T21:56:05.116 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.112+0000 7fc9b3400640 1 -- 172.21.15.72:0/892700032 learned_addr learned my addr 172.21.15.72:0/892700032 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:56:05.116 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.112+0000 7fc9b3e00640 1 --2- 172.21.15.72:0/892700032 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556daa987960 0x556daa991780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:05.116 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.113+0000 7fc9b9200640 1 --2- 172.21.15.72:0/892700032 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556daa98d4c0 0x556daa998d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:05.116 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.113+0000 7fc9b3400640 1 -- 172.21.15.72:0/892700032 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556daa98d4c0 msgr2=0x556daa998d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:56:05.116 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.113+0000 7fc9b3400640 1 --2- 172.21.15.72:0/892700032 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556daa98d4c0 0x556daa998d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:05.116 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.113+0000 7fc9b3400640 1 -- 172.21.15.72:0/892700032 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556daa987960 msgr2=0x556daa991780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:56:05.117 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.113+0000 7fc9b3400640 1 --2- 172.21.15.72:0/892700032 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556daa987960 0x556daa991780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:05.117 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.113+0000 7fc9b3400640 1 -- 172.21.15.72:0/892700032 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556daa6c77d0 con 0x556daa988350 2024-04-26T21:56:05.117 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.113+0000 7fc9b3400640 1 --2- 172.21.15.72:0/892700032 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556daa988350 0x556daa991cc0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fc9a00092a0 tx=0x7fc9a0004490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf2d55c095ffb025 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:56:05.117 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.113+0000 7fc99e000640 10 client.0 ms_handle_connect on v2:172.21.15.72:3300/0 2024-04-26T21:56:05.117 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.113+0000 7fc99e000640 1 -- 172.21.15.72:0/892700032 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9a0035070 con 0x556daa988350 2024-04-26T21:56:05.117 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.113+0000 7fc99e000640 1 -- 172.21.15.72:0/892700032 <== mon.0 v2:172.21.15.72:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7fc9a0003850 con 0x556daa988350 2024-04-26T21:56:05.117 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.113+0000 7fc9be1b9740 1 -- 172.21.15.72:0/892700032 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc9a0004030 con 0x556daa988350 2024-04-26T21:56:05.117 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.114+0000 7fc99e000640 1 -- 172.21.15.72:0/892700032 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc9a0057050 con 0x556daa988350 2024-04-26T21:56:05.117 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.114+0000 7fc9be1b9740 10 client.15669 fetch_fsmap learned FSMap version 27 2024-04-26T21:56:05.117 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.114+0000 7fc9be1b9740 1 -- 172.21.15.72:0/892700032 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556daa5fd3f0 con 0x556daa988350 2024-04-26T21:56:05.117 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.114+0000 7fc99e000640 1 -- 172.21.15.72:0/892700032 <== mon.0 v2:172.21.15.72:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc9a0004030 con 0x556daa988350 2024-04-26T21:56:05.117 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.114+0000 7fc99e000640 1 -- 172.21.15.72:0/892700032 <== mon.0 v2:172.21.15.72:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fc9a0047a20 con 0x556daa988350 2024-04-26T21:56:05.117 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.114+0000 7fc9be1b9740 10 client.15669 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T21:56:05.117 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.114+0000 7fc9be1b9740 10 client.15669 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:56:05.118 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.114+0000 7fc9be1b9740 20 client.15669 populate_metadata read hostname 'smithi072' 2024-04-26T21:56:05.118 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.114+0000 7fc9be1b9740 1 --2- 172.21.15.72:0/892700032 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x556daa99ce60 0x556daa99f250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:05.118 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.114+0000 7fc9be1b9740 4 client.15669 mds_command: new command op to 34517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:56:05.118 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.114+0000 7fc9be1b9740 1 -- 172.21.15.72:0/892700032 --> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556daa8f97e0 con 0x556daa99ce60 2024-04-26T21:56:05.119 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.115+0000 7fc9b3e00640 1 --2- 172.21.15.72:0/892700032 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x556daa99ce60 0x556daa99f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:05.119 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.116+0000 7fc9b3e00640 1 --2- 172.21.15.72:0/892700032 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x556daa99ce60 0x556daa99f250 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d1c1caeecce8928 server_cookie=eeea5d0b7d5f8f07 in_seq=0 out_seq=0 2024-04-26T21:56:05.119 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.116+0000 7fc99e000640 10 client.15669 ms_handle_connect on v2:172.21.15.76:6838/327145636 2024-04-26T21:56:05.119 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.118+0000 7fc99e000640 1 -- 172.21.15.72:0/892700032 <== mds.0 v2:172.21.15.76:6838/327145636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556daa8f97e0 con 0x556daa99ce60 2024-04-26T21:56:05.119 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.118+0000 7fc99e000640 10 client.15669 handle_command_reply: tid=0 2024-04-26T21:56:05.120 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.118+0000 7fc9be1b9740 1 client.15669 shutdown 2024-04-26T21:56:05.120 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.118+0000 7fc9be1b9740 1 -- 172.21.15.72:0/892700032 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556daa988350 msgr2=0x556daa991cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:05.120 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.118+0000 7fc9be1b9740 1 --2- 172.21.15.72:0/892700032 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556daa988350 0x556daa991cc0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fc9a00092a0 tx=0x7fc9a0004490 comp rx=0 tx=0).stop 2024-04-26T21:56:05.120 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.118+0000 7fc9be1b9740 1 -- 172.21.15.72:0/892700032 shutdown_connections 2024-04-26T21:56:05.120 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.118+0000 7fc9be1b9740 1 --2- 172.21.15.72:0/892700032 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x556daa99ce60 0x556daa99f250 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:05.120 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.118+0000 7fc9be1b9740 1 --2- 172.21.15.72:0/892700032 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556daa98d4c0 0x556daa998d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:05.120 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.118+0000 7fc99e000640 0 client.15669 ms_handle_reset on v2:172.21.15.76:6838/327145636 2024-04-26T21:56:05.120 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.118+0000 7fc9be1b9740 1 --2- 172.21.15.72:0/892700032 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556daa988350 0x556daa991cc0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:05.120 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.118+0000 7fc9be1b9740 1 --2- 172.21.15.72:0/892700032 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556daa987960 0x556daa991780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:05.120 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.118+0000 7fc9be1b9740 1 -- 172.21.15.72:0/892700032 >> 172.21.15.72:0/892700032 conn(0x556daa982fd0 msgr2=0x556daa9821f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:56:05.120 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.119+0000 7fc9be1b9740 1 -- 172.21.15.72:0/892700032 shutdown_connections 2024-04-26T21:56:05.120 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.119+0000 7fc9be1b9740 1 -- 172.21.15.72:0/892700032 wait complete. 2024-04-26T21:56:05.120 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.119+0000 7fc9be1b9740 20 client.15669 trim_cache size 0 max 16384 2024-04-26T21:56:05.138 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.137+0000 7fc9be1b9740 1 Processor -- start 2024-04-26T21:56:05.138 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.138+0000 7fc9be1b9740 1 -- start start 2024-04-26T21:56:05.138 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.138+0000 7fc9be1b9740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556daa987960 0x556daa9866e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:05.138 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.138+0000 7fc9be1b9740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556daa988350 0x556daa998cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:05.138 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.138+0000 7fc9be1b9740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556daa98d4c0 0x556daa988df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:05.138 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.138+0000 7fc9be1b9740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fc9b41b1cb0 con 0x556daa98d4c0 2024-04-26T21:56:05.138 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.138+0000 7fc9be1b9740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fc9b41b0840 con 0x556daa988350 2024-04-26T21:56:05.138 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.138+0000 7fc9be1b9740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fc9b41b0f10 con 0x556daa987960 2024-04-26T21:56:05.139 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.138+0000 7fc9b9200640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556daa98d4c0 0x556daa988df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:05.140 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.138+0000 7fc9b9200640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556daa98d4c0 0x556daa988df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:37748/0 (socket says 172.21.15.72:37748) 2024-04-26T21:56:05.140 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.138+0000 7fc9b9200640 1 -- 172.21.15.72:0/3847573082 learned_addr learned my addr 172.21.15.72:0/3847573082 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:56:05.140 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.139+0000 7fc9b3400640 1 --2- 172.21.15.72:0/3847573082 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556daa988350 0x556daa998cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:05.140 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.139+0000 7fc9b3e00640 1 --2- 172.21.15.72:0/3847573082 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556daa987960 0x556daa9866e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:05.140 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.139+0000 7fc9b9200640 1 -- 172.21.15.72:0/3847573082 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556daa987960 msgr2=0x556daa9866e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:56:05.140 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.139+0000 7fc9b9200640 1 --2- 172.21.15.72:0/3847573082 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556daa987960 0x556daa9866e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:05.140 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.139+0000 7fc9b9200640 1 -- 172.21.15.72:0/3847573082 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556daa988350 msgr2=0x556daa998cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:56:05.140 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.139+0000 7fc9b9200640 1 --2- 172.21.15.72:0/3847573082 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556daa988350 0x556daa998cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:05.140 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.139+0000 7fc9b9200640 1 -- 172.21.15.72:0/3847573082 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556daa97e850 con 0x556daa98d4c0 2024-04-26T21:56:05.140 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.139+0000 7fc9b9200640 1 --2- 172.21.15.72:0/3847573082 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556daa98d4c0 0x556daa988df0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fc9ac0998f0 tx=0x7fc9ac09d6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad8bbe60acd32899 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:56:05.140 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.139+0000 7fc9a8c00640 1 -- 172.21.15.72:0/3847573082 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9ac0b5070 con 0x556daa98d4c0 2024-04-26T21:56:05.140 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.139+0000 7fc9a8c00640 1 -- 172.21.15.72:0/3847573082 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc9ac0a5b90 con 0x556daa98d4c0 2024-04-26T21:56:05.140 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.139+0000 7fc9a8c00640 1 -- 172.21.15.72:0/3847573082 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9ac0b6260 con 0x556daa98d4c0 2024-04-26T21:56:05.144 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.140+0000 7fc9be1b9740 1 -- 172.21.15.72:0/3847573082 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556daa98d4c0 msgr2=0x556daa988df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:05.144 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.140+0000 7fc9be1b9740 1 --2- 172.21.15.72:0/3847573082 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556daa98d4c0 0x556daa988df0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fc9ac0998f0 tx=0x7fc9ac09d6e0 comp rx=0 tx=0).stop 2024-04-26T21:56:05.144 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.140+0000 7fc9be1b9740 1 -- 172.21.15.72:0/3847573082 shutdown_connections 2024-04-26T21:56:05.144 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.140+0000 7fc9be1b9740 1 --2- 172.21.15.72:0/3847573082 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556daa98d4c0 0x556daa988df0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:05.144 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.140+0000 7fc9be1b9740 1 --2- 172.21.15.72:0/3847573082 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556daa988350 0x556daa998cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:05.144 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.140+0000 7fc9be1b9740 1 --2- 172.21.15.72:0/3847573082 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556daa987960 0x556daa9866e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:05.144 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.140+0000 7fc9be1b9740 1 -- 172.21.15.72:0/3847573082 >> 172.21.15.72:0/3847573082 conn(0x556daa982fd0 msgr2=0x556daa985fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:56:05.144 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.140+0000 7fc9be1b9740 1 -- 172.21.15.72:0/3847573082 shutdown_connections 2024-04-26T21:56:05.144 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.140+0000 7fc9be1b9740 1 -- 172.21.15.72:0/3847573082 wait complete. 2024-04-26T21:56:05.145 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.140+0000 7fc9be1b9740 1 -- start start 2024-04-26T21:56:05.145 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.141+0000 7fc9be1b9740 1 Processor -- start 2024-04-26T21:56:05.145 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.141+0000 7fc9be1b9740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556daa987960 0x556daa996130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:05.145 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.141+0000 7fc9be1b9740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556daa988350 0x556daa996670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:05.145 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.141+0000 7fc9be1b9740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556daa98d4c0 0x556daa997680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:05.145 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.141+0000 7fc9be1b9740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fc9b41ae150 con 0x556daa98d4c0 2024-04-26T21:56:05.145 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.141+0000 7fc9be1b9740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fc9b41b15e0 con 0x556daa988350 2024-04-26T21:56:05.145 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.141+0000 7fc9be1b9740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fc9b41ae7c0 con 0x556daa987960 2024-04-26T21:56:05.145 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.141+0000 7fc9be1b9740 10 client.0 _ll_register_callbacks cb 0x7fc9a0031570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:56:05.145 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.141+0000 7fc9b9200640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556daa98d4c0 0x556daa997680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:05.145 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.142+0000 7fc9b9200640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556daa98d4c0 0x556daa997680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:37764/0 (socket says 172.21.15.72:37764) 2024-04-26T21:56:05.145 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.142+0000 7fc9b9200640 1 -- 172.21.15.72:0/2585273630 learned_addr learned my addr 172.21.15.72:0/2585273630 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:56:05.145 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.142+0000 7fc9b3e00640 1 --2- 172.21.15.72:0/2585273630 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556daa987960 0x556daa996130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:05.145 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.142+0000 7fc9b3400640 1 --2- 172.21.15.72:0/2585273630 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556daa988350 0x556daa996670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:05.146 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.142+0000 7fc9b9200640 1 -- 172.21.15.72:0/2585273630 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556daa987960 msgr2=0x556daa996130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:56:05.146 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.142+0000 7fc9b9200640 1 --2- 172.21.15.72:0/2585273630 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556daa987960 0x556daa996130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:05.146 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.142+0000 7fc9b9200640 1 -- 172.21.15.72:0/2585273630 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556daa988350 msgr2=0x556daa996670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:56:05.146 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.142+0000 7fc9b9200640 1 --2- 172.21.15.72:0/2585273630 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556daa988350 0x556daa996670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:05.146 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.142+0000 7fc9b9200640 1 -- 172.21.15.72:0/2585273630 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556daa5bd0f0 con 0x556daa98d4c0 2024-04-26T21:56:05.146 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.142+0000 7fc9b3e00640 1 --2- 172.21.15.72:0/2585273630 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556daa987960 0x556daa996130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:56:05.146 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.142+0000 7fc9b9200640 1 --2- 172.21.15.72:0/2585273630 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556daa98d4c0 0x556daa997680 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fc9ac09ceb0 tx=0x7fc9ac0cebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fa93056daa53256 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:56:05.146 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.142+0000 7fc99e000640 10 client.0 ms_handle_connect on v2:172.21.15.72:3300/0 2024-04-26T21:56:05.146 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.142+0000 7fc99e000640 1 -- 172.21.15.72:0/2585273630 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9ac07f020 con 0x556daa98d4c0 2024-04-26T21:56:05.146 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.142+0000 7fc99e000640 1 -- 172.21.15.72:0/2585273630 <== mon.0 v2:172.21.15.72:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7fc9ac0a5900 con 0x556daa98d4c0 2024-04-26T21:56:05.146 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.142+0000 7fc9be1b9740 1 -- 172.21.15.72:0/2585273630 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc9ac0a5b90 con 0x556daa98d4c0 2024-04-26T21:56:05.146 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.143+0000 7fc99e000640 1 -- 172.21.15.72:0/2585273630 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc9ac0b5030 con 0x556daa98d4c0 2024-04-26T21:56:05.146 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.143+0000 7fc9be1b9740 10 client.15675 fetch_fsmap learned FSMap version 27 2024-04-26T21:56:05.146 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.143+0000 7fc9be1b9740 1 -- 172.21.15.72:0/2585273630 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556daa8b3a30 con 0x556daa98d4c0 2024-04-26T21:56:05.146 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.143+0000 7fc99e000640 1 -- 172.21.15.72:0/2585273630 <== mon.0 v2:172.21.15.72:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc9ac0a5b90 con 0x556daa98d4c0 2024-04-26T21:56:05.147 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.143+0000 7fc99e000640 1 -- 172.21.15.72:0/2585273630 <== mon.0 v2:172.21.15.72:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fc9ac098e70 con 0x556daa98d4c0 2024-04-26T21:56:05.147 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.143+0000 7fc9be1b9740 10 client.15675 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T21:56:05.147 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.143+0000 7fc9be1b9740 10 client.15675 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:56:05.147 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.143+0000 7fc9be1b9740 20 client.15675 populate_metadata read hostname 'smithi072' 2024-04-26T21:56:05.147 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.143+0000 7fc9be1b9740 1 --2- 172.21.15.72:0/2585273630 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x556daaa035a0 0x556daaa059c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:05.147 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.143+0000 7fc9be1b9740 4 client.15675 mds_command: new command op to 34517 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-26T21:56:05.147 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.143+0000 7fc9be1b9740 1 -- 172.21.15.72:0/2585273630 --> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] -- command(tid 0: {"prefix": "scrub status"}) -- 0x556daa991570 con 0x556daaa035a0 2024-04-26T21:56:05.147 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.144+0000 7fc9b3e00640 1 --2- 172.21.15.72:0/2585273630 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x556daaa035a0 0x556daaa059c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:05.148 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.145+0000 7fc9b3e00640 1 --2- 172.21.15.72:0/2585273630 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x556daaa035a0 0x556daaa059c0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=440061cd5a29789b server_cookie=2f14b75035cba0b7 in_seq=0 out_seq=0 2024-04-26T21:56:05.148 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.145+0000 7fc99e000640 10 client.15675 ms_handle_connect on v2:172.21.15.76:6838/327145636 2024-04-26T21:56:05.148 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.145+0000 7fc99e000640 1 -- 172.21.15.72:0/2585273630 <== mds.0 v2:172.21.15.76:6838/327145636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x556daa991570 con 0x556daaa035a0 2024-04-26T21:56:05.149 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.145+0000 7fc99e000640 10 client.15675 handle_command_reply: tid=0 2024-04-26T21:56:05.149 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.145+0000 7fc9be1b9740 1 client.15675 shutdown 2024-04-26T21:56:05.149 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.146+0000 7fc9be1b9740 1 -- 172.21.15.72:0/2585273630 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556daa98d4c0 msgr2=0x556daa997680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:05.149 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.146+0000 7fc9be1b9740 1 --2- 172.21.15.72:0/2585273630 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556daa98d4c0 0x556daa997680 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fc9ac09ceb0 tx=0x7fc9ac0cebd0 comp rx=0 tx=0).stop 2024-04-26T21:56:05.149 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.146+0000 7fc9be1b9740 1 -- 172.21.15.72:0/2585273630 shutdown_connections 2024-04-26T21:56:05.149 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.146+0000 7fc9be1b9740 1 --2- 172.21.15.72:0/2585273630 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x556daaa035a0 0x556daaa059c0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:05.149 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.146+0000 7fc9be1b9740 1 --2- 172.21.15.72:0/2585273630 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556daa98d4c0 0x556daa997680 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:05.149 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.146+0000 7fc9be1b9740 1 --2- 172.21.15.72:0/2585273630 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556daa988350 0x556daa996670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:05.149 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.146+0000 7fc9be1b9740 1 --2- 172.21.15.72:0/2585273630 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556daa987960 0x556daa996130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:05.149 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.146+0000 7fc99e000640 0 client.15675 ms_handle_reset on v2:172.21.15.76:6838/327145636 2024-04-26T21:56:05.149 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.146+0000 7fc9be1b9740 1 -- 172.21.15.72:0/2585273630 >> 172.21.15.72:0/2585273630 conn(0x556daa982fd0 msgr2=0x556daa981960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:56:05.149 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.146+0000 7fc9be1b9740 1 -- 172.21.15.72:0/2585273630 shutdown_connections 2024-04-26T21:56:05.149 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.146+0000 7fc9be1b9740 1 -- 172.21.15.72:0/2585273630 wait complete. 2024-04-26T21:56:05.149 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.146+0000 7fc9be1b9740 20 client.15675 trim_cache size 0 max 16384 2024-04-26T21:56:05.150 INFO:teuthology.orchestra.run.smithi072.stdout:{ 2024-04-26T21:56:05.150 INFO:teuthology.orchestra.run.smithi072.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-26T21:56:05.150 INFO:teuthology.orchestra.run.smithi072.stdout: "scrubs": { 2024-04-26T21:56:05.150 INFO:teuthology.orchestra.run.smithi072.stdout: "98faa0cd-bf77-42fe-a1c6-bdb868ce4471": { 2024-04-26T21:56:05.150 INFO:teuthology.orchestra.run.smithi072.stdout: "path": "/", 2024-04-26T21:56:05.150 INFO:teuthology.orchestra.run.smithi072.stdout: "tag": "98faa0cd-bf77-42fe-a1c6-bdb868ce4471", 2024-04-26T21:56:05.150 INFO:teuthology.orchestra.run.smithi072.stdout: "options": "recursive,force" 2024-04-26T21:56:05.150 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:56:05.150 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:56:05.150 INFO:teuthology.orchestra.run.smithi072.stdout:} 2024-04-26T21:56:05.151 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.147+0000 7fc9b9c00640 1 -- 172.21.15.72:0/2190002952 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fc9980a3cc0 msgr2=0x7fc9980a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:05.151 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.147+0000 7fc9b9c00640 1 --2- 172.21.15.72:0/2190002952 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fc9980a3cc0 0x7fc9980a6180 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fc9a4005fd0 tx=0x7fc9a4005850 comp rx=0 tx=0).stop 2024-04-26T21:56:05.151 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.147+0000 7fc9b9c00640 1 -- 172.21.15.72:0/2190002952 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc9b410a980 msgr2=0x7fc9b41a64e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:05.151 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.147+0000 7fc9b9c00640 1 --2- 172.21.15.72:0/2190002952 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc9b410a980 0x7fc9b41a64e0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fc9ac00cce0 tx=0x7fc9ac00a620 comp rx=0 tx=0).stop 2024-04-26T21:56:05.151 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.147+0000 7fc9b9c00640 1 -- 172.21.15.72:0/2190002952 shutdown_connections 2024-04-26T21:56:05.151 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.147+0000 7fc9b9c00640 1 --2- 172.21.15.72:0/2190002952 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc9b410a980 0x7fc9b41a64e0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:05.151 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.147+0000 7fc9b9c00640 1 --2- 172.21.15.72:0/2190002952 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc9b4109fc0 0x7fc9b419f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:05.152 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.147+0000 7fc9b9c00640 1 --2- 172.21.15.72:0/2190002952 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fc9980a3cc0 0x7fc9980a6180 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:05.152 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.147+0000 7fc9b9c00640 1 --2- 172.21.15.72:0/2190002952 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc9b4108e70 0x7fc9b419ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:05.152 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.147+0000 7fc9b9c00640 1 -- 172.21.15.72:0/2190002952 >> 172.21.15.72:0/2190002952 conn(0x7fc9b4077e00 msgr2=0x7fc9b410d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:56:05.152 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.147+0000 7fc9b9c00640 1 -- 172.21.15.72:0/2190002952 shutdown_connections 2024-04-26T21:56:05.152 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:05.147+0000 7fc9b9c00640 1 -- 172.21.15.72:0/2190002952 wait complete. 2024-04-26T21:56:05.165 INFO:tasks.cephfs.filesystem:scrub status for tag:98faa0cd-bf77-42fe-a1c6-bdb868ce4471 - {'path': '/', 'tag': '98faa0cd-bf77-42fe-a1c6-bdb868ce4471', 'options': 'recursive,force'} 2024-04-26T21:56:05.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:56:04 smithi076 ceph-mon[31823]: pgmap v468: 97 pgs: 97 active+clean; 650 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-26T21:56:05.321 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:56:04 smithi072 ceph-mon[28138]: pgmap v468: 97 pgs: 97 active+clean; 650 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-26T21:56:05.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:56:04 smithi164 ceph-mon[31916]: pgmap v468: 97 pgs: 97 active+clean; 650 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-26T21:56:06.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:56:05 smithi076 ceph-mon[31823]: scrub queued for path: / 2024-04-26T21:56:06.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:56:05 smithi076 ceph-mon[31823]: scrub summary: idle+waiting paths [/] 2024-04-26T21:56:06.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:56:05 smithi076 ceph-mon[31823]: scrub summary: active paths [/] 2024-04-26T21:56:06.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:56:05 smithi076 ceph-mon[31823]: scrub summary: idle+waiting paths [/] 2024-04-26T21:56:06.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:56:05 smithi072 ceph-mon[28138]: scrub queued for path: / 2024-04-26T21:56:06.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:56:05 smithi072 ceph-mon[28138]: scrub summary: idle+waiting paths [/] 2024-04-26T21:56:06.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:56:05 smithi072 ceph-mon[28138]: scrub summary: active paths [/] 2024-04-26T21:56:06.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:56:05 smithi072 ceph-mon[28138]: scrub summary: idle+waiting paths [/] 2024-04-26T21:56:06.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:56:05 smithi164 ceph-mon[31916]: scrub queued for path: / 2024-04-26T21:56:06.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:56:05 smithi164 ceph-mon[31916]: scrub summary: idle+waiting paths [/] 2024-04-26T21:56:06.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:56:05 smithi164 ceph-mon[31916]: scrub summary: active paths [/] 2024-04-26T21:56:06.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:56:05 smithi164 ceph-mon[31916]: scrub summary: idle+waiting paths [/] 2024-04-26T21:56:07.217 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:56:06 smithi076 ceph-mon[31823]: pgmap v469: 97 pgs: 97 active+clean; 650 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-26T21:56:07.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:56:06 smithi072 ceph-mon[28138]: pgmap v469: 97 pgs: 97 active+clean; 650 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-26T21:56:07.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:56:06 smithi164 ceph-mon[31916]: pgmap v469: 97 pgs: 97 active+clean; 650 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-26T21:56:09.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:56:08 smithi076 ceph-mon[31823]: pgmap v470: 97 pgs: 97 active+clean; 650 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:56:09.321 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:56:08 smithi072 ceph-mon[28138]: pgmap v470: 97 pgs: 97 active+clean; 650 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:56:09.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:56:08 smithi164 ceph-mon[31916]: pgmap v470: 97 pgs: 97 active+clean; 650 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:56:10.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:56:09 smithi076 ceph-mon[31823]: scrub summary: idle 2024-04-26T21:56:10.321 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:56:09 smithi072 ceph-mon[28138]: scrub summary: idle 2024-04-26T21:56:10.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:56:09 smithi164 ceph-mon[31916]: scrub summary: idle 2024-04-26T21:56:11.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:56:10 smithi076 ceph-mon[31823]: pgmap v471: 97 pgs: 97 active+clean; 650 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-04-26T21:56:11.321 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:56:10 smithi072 ceph-mon[28138]: pgmap v471: 97 pgs: 97 active+clean; 650 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-04-26T21:56:11.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:56:10 smithi164 ceph-mon[31916]: pgmap v471: 97 pgs: 97 active+clean; 650 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-04-26T21:56:13.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:56:12 smithi076 ceph-mon[31823]: pgmap v472: 97 pgs: 97 active+clean; 650 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-26T21:56:13.321 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:56:12 smithi072 ceph-mon[28138]: pgmap v472: 97 pgs: 97 active+clean; 650 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-26T21:56:13.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:56:12 smithi164 ceph-mon[31916]: pgmap v472: 97 pgs: 97 active+clean; 650 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-26T21:56:15.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:56:15 smithi076 ceph-mon[31823]: pgmap v473: 97 pgs: 97 active+clean; 650 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 9 op/s 2024-04-26T21:56:15.321 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:56:14 smithi072 ceph-mon[28138]: pgmap v473: 97 pgs: 97 active+clean; 650 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 9 op/s 2024-04-26T21:56:15.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:56:15 smithi164 ceph-mon[31916]: pgmap v473: 97 pgs: 97 active+clean; 650 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 9 op/s 2024-04-26T21:56:17.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:56:17 smithi076 ceph-mon[31823]: pgmap v474: 97 pgs: 97 active+clean; 650 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 11 op/s 2024-04-26T21:56:17.321 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:56:17 smithi072 ceph-mon[28138]: pgmap v474: 97 pgs: 97 active+clean; 650 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 11 op/s 2024-04-26T21:56:17.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:56:17 smithi164 ceph-mon[31916]: pgmap v474: 97 pgs: 97 active+clean; 650 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 11 op/s 2024-04-26T21:56:19.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:56:19 smithi076 ceph-mon[31823]: pgmap v475: 97 pgs: 97 active+clean; 650 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-26T21:56:19.321 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:56:19 smithi072 ceph-mon[28138]: pgmap v475: 97 pgs: 97 active+clean; 650 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-26T21:56:19.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:56:19 smithi164 ceph-mon[31916]: pgmap v475: 97 pgs: 97 active+clean; 650 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-26T21:56:21.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:56:21 smithi076 ceph-mon[31823]: pgmap v476: 97 pgs: 97 active+clean; 650 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-04-26T21:56:21.321 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:56:21 smithi072 ceph-mon[28138]: pgmap v476: 97 pgs: 97 active+clean; 650 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-04-26T21:56:21.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:56:21 smithi164 ceph-mon[31916]: pgmap v476: 97 pgs: 97 active+clean; 650 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-04-26T21:56:23.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:56:23 smithi076 ceph-mon[31823]: pgmap v477: 97 pgs: 97 active+clean; 650 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:56:23.321 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:56:23 smithi072 ceph-mon[28138]: pgmap v477: 97 pgs: 97 active+clean; 650 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:56:23.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:56:23 smithi164 ceph-mon[31916]: pgmap v477: 97 pgs: 97 active+clean; 650 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:56:24.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:56:24 smithi164 ceph-mon[31916]: pgmap v478: 97 pgs: 97 active+clean; 650 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-26T21:56:24.564 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:56:24 smithi076 ceph-mon[31823]: pgmap v478: 97 pgs: 97 active+clean; 650 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-26T21:56:24.574 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:56:24 smithi072 ceph-mon[28138]: pgmap v478: 97 pgs: 97 active+clean; 650 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-26T21:56:27.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:56:26 smithi164 ceph-mon[31916]: pgmap v479: 97 pgs: 97 active+clean; 650 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-26T21:56:27.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:56:26 smithi076 ceph-mon[31823]: pgmap v479: 97 pgs: 97 active+clean; 650 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-26T21:56:27.321 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:56:26 smithi072 ceph-mon[28138]: pgmap v479: 97 pgs: 97 active+clean; 650 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-26T21:56:29.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:56:28 smithi164 ceph-mon[31916]: pgmap v480: 97 pgs: 97 active+clean; 650 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:56:29.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:56:28 smithi076 ceph-mon[31823]: pgmap v480: 97 pgs: 97 active+clean; 650 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:56:29.321 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:56:28 smithi072 ceph-mon[28138]: pgmap v480: 97 pgs: 97 active+clean; 650 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:56:31.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:56:30 smithi164 ceph-mon[31916]: pgmap v481: 97 pgs: 97 active+clean; 650 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-26T21:56:31.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:56:30 smithi076 ceph-mon[31823]: pgmap v481: 97 pgs: 97 active+clean; 650 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-26T21:56:31.321 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:56:30 smithi072 ceph-mon[28138]: pgmap v481: 97 pgs: 97 active+clean; 650 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-26T21:56:32.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:56:31 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:56:32.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:56:31 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:56:32.321 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:56:31 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:56:33.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:56:32 smithi076 ceph-mon[31823]: pgmap v482: 97 pgs: 97 active+clean; 650 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:56:33.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:56:32 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:56:33.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:56:32 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:56:33.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:56:32 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:56:33.321 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:56:32 smithi072 ceph-mon[28138]: pgmap v482: 97 pgs: 97 active+clean; 650 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:56:33.321 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:56:32 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:56:33.321 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:56:32 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:56:33.321 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:56:32 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:56:33.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:56:32 smithi164 ceph-mon[31916]: pgmap v482: 97 pgs: 97 active+clean; 650 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:56:33.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:56:32 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:56:33.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:56:32 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:56:33.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:56:32 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:56:35.166 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-26T21:56:35.166 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-26T21:56:35.226 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:56:34 smithi072 ceph-mon[28138]: pgmap v483: 97 pgs: 97 active+clean; 650 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-26T21:56:35.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:56:34 smithi076 ceph-mon[31823]: pgmap v483: 97 pgs: 97 active+clean; 650 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-26T21:56:35.343 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.342+0000 7f7b8f600640 1 -- 172.21.15.72:0/2853728017 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7b88104520 msgr2=0x7f7b8810adb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:35.343 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.342+0000 7f7b8f600640 1 --2- 172.21.15.72:0/2853728017 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7b88104520 0x7f7b8810adb0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f7b80009ca0 tx=0x7f7b8002a8c0 comp rx=0 tx=0).stop 2024-04-26T21:56:35.343 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.342+0000 7f7b8f600640 1 -- 172.21.15.72:0/2853728017 shutdown_connections 2024-04-26T21:56:35.343 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.343+0000 7f7b8f600640 1 --2- 172.21.15.72:0/2853728017 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7b88104520 0x7f7b8810adb0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:35.343 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.343+0000 7f7b8f600640 1 --2- 172.21.15.72:0/2853728017 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7b88103b60 0x7f7b88103fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:35.343 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.343+0000 7f7b8f600640 1 --2- 172.21.15.72:0/2853728017 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7b88102970 0x7f7b88102d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:35.343 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.343+0000 7f7b8f600640 1 -- 172.21.15.72:0/2853728017 >> 172.21.15.72:0/2853728017 conn(0x7f7b880fdee0 msgr2=0x7f7b88100320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:56:35.344 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.343+0000 7f7b8f600640 1 -- 172.21.15.72:0/2853728017 shutdown_connections 2024-04-26T21:56:35.344 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.343+0000 7f7b8f600640 1 -- 172.21.15.72:0/2853728017 wait complete. 2024-04-26T21:56:35.345 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.344+0000 7f7b8f600640 1 Processor -- start 2024-04-26T21:56:35.345 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.344+0000 7f7b8f600640 1 -- start start 2024-04-26T21:56:35.346 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.344+0000 7f7b8f600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7b88102970 0x7f7b88076690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:35.346 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.344+0000 7f7b8f600640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7b88103b60 0x7f7b88074d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:35.346 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.344+0000 7f7b8f600640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7b88104520 0x7f7b88075260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:35.346 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.344+0000 7f7b8f600640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f7b8810e3a0 con 0x7f7b88104520 2024-04-26T21:56:35.346 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.344+0000 7f7b8f600640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f7b8810e220 con 0x7f7b88103b60 2024-04-26T21:56:35.346 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.344+0000 7f7b8f600640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f7b8810e520 con 0x7f7b88102970 2024-04-26T21:56:35.346 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.345+0000 7f7b8d600640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7b88104520 0x7f7b88075260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:35.346 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.345+0000 7f7b8d600640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7b88104520 0x7f7b88075260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:58462/0 (socket says 172.21.15.72:58462) 2024-04-26T21:56:35.346 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.345+0000 7f7b8d600640 1 -- 172.21.15.72:0/351765875 learned_addr learned my addr 172.21.15.72:0/351765875 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:56:35.346 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.345+0000 7f7b8cc00640 1 --2- 172.21.15.72:0/351765875 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7b88102970 0x7f7b88076690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:35.346 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.345+0000 7f7b87e00640 1 --2- 172.21.15.72:0/351765875 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7b88103b60 0x7f7b88074d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:35.346 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.345+0000 7f7b8d600640 1 -- 172.21.15.72:0/351765875 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7b88102970 msgr2=0x7f7b88076690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:35.346 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.345+0000 7f7b8d600640 1 --2- 172.21.15.72:0/351765875 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7b88102970 0x7f7b88076690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:35.347 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.345+0000 7f7b8d600640 1 -- 172.21.15.72:0/351765875 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7b88103b60 msgr2=0x7f7b88074d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:35.347 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.345+0000 7f7b8d600640 1 --2- 172.21.15.72:0/351765875 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7b88103b60 0x7f7b88074d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:35.347 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.345+0000 7f7b8d600640 1 -- 172.21.15.72:0/351765875 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b88075b20 con 0x7f7b88104520 2024-04-26T21:56:35.347 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.346+0000 7f7b8d600640 1 --2- 172.21.15.72:0/351765875 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7b88104520 0x7f7b88075260 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f7b8002aeb0 tx=0x7f7b80003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:56:35.347 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.346+0000 7f7b85600640 1 -- 172.21.15.72:0/351765875 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b80032ed0 con 0x7f7b88104520 2024-04-26T21:56:35.347 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.346+0000 7f7b8f600640 1 -- 172.21.15.72:0/351765875 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b881a2bc0 con 0x7f7b88104520 2024-04-26T21:56:35.347 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.346+0000 7f7b85600640 1 -- 172.21.15.72:0/351765875 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7b80003e10 con 0x7f7b88104520 2024-04-26T21:56:35.348 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.346+0000 7f7b85600640 1 -- 172.21.15.72:0/351765875 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b80030cd0 con 0x7f7b88104520 2024-04-26T21:56:35.348 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.347+0000 7f7b8f600640 1 -- 172.21.15.72:0/351765875 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b881a3130 con 0x7f7b88104520 2024-04-26T21:56:35.350 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.351+0000 7f7b9243a740 1 Processor -- start 2024-04-26T21:56:35.351 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.351+0000 7f7b85600640 1 -- 172.21.15.72:0/351765875 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f7b80004240 con 0x7f7b88104520 2024-04-26T21:56:35.352 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.351+0000 7f7b9243a740 1 -- start start 2024-04-26T21:56:35.352 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.352+0000 7f7b9243a740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5651cf1c0790 0x5651cf1c0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:35.352 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.352+0000 7f7b9243a740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5651cf1c1180 0x5651cf1c5c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:35.352 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.352+0000 7f7b9243a740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5651cf1c62f0 0x5651cf1c8780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:35.352 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.352+0000 7f7b9243a740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x5651cf0d1030 con 0x5651cf1c62f0 2024-04-26T21:56:35.352 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.352+0000 7f7b9243a740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5651cf0ec7f0 con 0x5651cf1c0790 2024-04-26T21:56:35.353 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.352+0000 7f7b9243a740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5651cf0a77e0 con 0x5651cf1c1180 2024-04-26T21:56:35.353 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.353+0000 7f7b87e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5651cf1c62f0 0x5651cf1c8780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:35.353 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.353+0000 7f7b87e00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5651cf1c0790 0x5651cf1c0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:35.353 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.353+0000 7f7b87e00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5651cf1c0790 0x5651cf1c0bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.72:53562/0 (socket says 172.21.15.72:53562) 2024-04-26T21:56:35.354 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.353+0000 7f7b87e00640 1 -- 172.21.15.72:0/3408820404 learned_addr learned my addr 172.21.15.72:0/3408820404 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:56:35.354 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.354+0000 7f7b87e00640 1 -- 172.21.15.72:0/3408820404 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5651cf1c1180 msgr2=0x5651cf1c5c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:56:35.354 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.354+0000 7f7b87e00640 1 --2- 172.21.15.72:0/3408820404 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5651cf1c1180 0x5651cf1c5c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:35.354 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.354+0000 7f7b87e00640 1 -- 172.21.15.72:0/3408820404 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5651cf1c0790 msgr2=0x5651cf1c0bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:56:35.354 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.354+0000 7f7b87e00640 1 --2- 172.21.15.72:0/3408820404 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5651cf1c0790 0x5651cf1c0bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:35.355 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.354+0000 7f7b85600640 1 --2- 172.21.15.72:0/351765875 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f7b700a3cc0 0x7f7b700a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:35.355 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.355+0000 7f7b87e00640 1 -- 172.21.15.72:0/3408820404 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5651cf0b3050 con 0x5651cf1c62f0 2024-04-26T21:56:35.355 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.355+0000 7f7b85600640 1 -- 172.21.15.72:0/351765875 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7f7b800f1b60 con 0x7f7b88104520 2024-04-26T21:56:35.355 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.355+0000 7f7b87e00640 1 --2- 172.21.15.72:0/3408820404 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5651cf1c62f0 0x5651cf1c8780 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f7b74005d00 tx=0x7f7b7400b5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d9fd9893c8752d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:56:35.356 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.356+0000 7f7b8cc00640 1 --2- 172.21.15.72:0/351765875 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f7b700a3cc0 0x7f7b700a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:35.356 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.356+0000 7f7b79c00640 1 -- 172.21.15.72:0/3408820404 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b74005020 con 0x5651cf1c62f0 2024-04-26T21:56:35.356 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.356+0000 7f7b79c00640 1 -- 172.21.15.72:0/3408820404 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7b740047a0 con 0x5651cf1c62f0 2024-04-26T21:56:35.356 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.356+0000 7f7b79c00640 1 -- 172.21.15.72:0/3408820404 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b74005650 con 0x5651cf1c62f0 2024-04-26T21:56:35.357 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.357+0000 7f7b9243a740 1 -- 172.21.15.72:0/3408820404 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5651cf1c62f0 msgr2=0x5651cf1c8780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:35.357 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.357+0000 7f7b9243a740 1 --2- 172.21.15.72:0/3408820404 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5651cf1c62f0 0x5651cf1c8780 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f7b74005d00 tx=0x7f7b7400b5b0 comp rx=0 tx=0).stop 2024-04-26T21:56:35.357 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.357+0000 7f7b9243a740 1 -- 172.21.15.72:0/3408820404 shutdown_connections 2024-04-26T21:56:35.357 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.357+0000 7f7b9243a740 1 --2- 172.21.15.72:0/3408820404 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5651cf1c62f0 0x5651cf1c8780 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:35.357 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.357+0000 7f7b9243a740 1 --2- 172.21.15.72:0/3408820404 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5651cf1c1180 0x5651cf1c5c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:35.357 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.357+0000 7f7b9243a740 1 --2- 172.21.15.72:0/3408820404 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5651cf1c0790 0x5651cf1c0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:35.357 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.357+0000 7f7b9243a740 1 -- 172.21.15.72:0/3408820404 >> 172.21.15.72:0/3408820404 conn(0x5651cf1bbdb0 msgr2=0x5651cf1bc1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:56:35.357 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.357+0000 7f7b9243a740 1 -- 172.21.15.72:0/3408820404 shutdown_connections 2024-04-26T21:56:35.358 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.357+0000 7f7b8cc00640 1 --2- 172.21.15.72:0/351765875 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f7b700a3cc0 0x7f7b700a6180 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x5651cf1c1580 tx=0x7f7b7c005f30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:56:35.358 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.357+0000 7f7b9243a740 1 -- 172.21.15.72:0/3408820404 wait complete. 2024-04-26T21:56:35.358 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.358+0000 7f7b9243a740 1 -- start start 2024-04-26T21:56:35.358 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.358+0000 7f7b9243a740 1 Processor -- start 2024-04-26T21:56:35.361 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.359+0000 7f7b9243a740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5651cf1c0790 0x5651cf1d2f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:35.361 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.359+0000 7f7b9243a740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5651cf1c1180 0x5651cf1d3450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:35.361 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.359+0000 7f7b9243a740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5651cf1c62f0 0x5651cf1da490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:35.361 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.359+0000 7f7b9243a740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x5651cf129200 con 0x5651cf1c0790 2024-04-26T21:56:35.361 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.359+0000 7f7b9243a740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5651cf123ee0 con 0x5651cf1c1180 2024-04-26T21:56:35.361 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.359+0000 7f7b9243a740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5651cf130e70 con 0x5651cf1c62f0 2024-04-26T21:56:35.361 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.359+0000 7f7b9243a740 10 client.0 _ll_register_callbacks cb 0x5651cf11a270 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:56:35.362 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.359+0000 7f7b8cc00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5651cf1c0790 0x5651cf1d2f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:35.362 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.359+0000 7f7b8cc00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5651cf1c0790 0x5651cf1d2f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:58478/0 (socket says 172.21.15.72:58478) 2024-04-26T21:56:35.362 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.359+0000 7f7b8cc00640 1 -- 172.21.15.72:0/3012076388 learned_addr learned my addr 172.21.15.72:0/3012076388 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:56:35.362 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.359+0000 7f7b8d600640 1 --2- 172.21.15.72:0/3012076388 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5651cf1c62f0 0x5651cf1da490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:35.362 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.360+0000 7f7b87e00640 1 --2- 172.21.15.72:0/3012076388 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5651cf1c1180 0x5651cf1d3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:35.362 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.360+0000 7f7b8cc00640 1 -- 172.21.15.72:0/3012076388 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5651cf1c62f0 msgr2=0x5651cf1da490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:56:35.362 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.360+0000 7f7b8cc00640 1 --2- 172.21.15.72:0/3012076388 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5651cf1c62f0 0x5651cf1da490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:35.362 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.360+0000 7f7b8cc00640 1 -- 172.21.15.72:0/3012076388 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5651cf1c1180 msgr2=0x5651cf1d3450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:56:35.362 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.360+0000 7f7b8cc00640 1 --2- 172.21.15.72:0/3012076388 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5651cf1c1180 0x5651cf1d3450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:35.362 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.360+0000 7f7b8cc00640 1 -- 172.21.15.72:0/3012076388 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5651cef007d0 con 0x5651cf1c0790 2024-04-26T21:56:35.362 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.360+0000 7f7b8cc00640 1 --2- 172.21.15.72:0/3012076388 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5651cf1c0790 0x5651cf1d2f10 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f7b7c00ff70 tx=0x7f7b7c005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92dbcf7eeeba59e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:56:35.362 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.360+0000 7f7b6e000640 10 client.0 ms_handle_connect on v2:172.21.15.72:3300/0 2024-04-26T21:56:35.362 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.361+0000 7f7b6e000640 1 -- 172.21.15.72:0/3012076388 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b7c022070 con 0x5651cf1c0790 2024-04-26T21:56:35.362 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.361+0000 7f7b6e000640 1 -- 172.21.15.72:0/3012076388 <== mon.0 v2:172.21.15.72:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7f7b7c01e860 con 0x5651cf1c0790 2024-04-26T21:56:35.363 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.361+0000 7f7b9243a740 1 -- 172.21.15.72:0/3012076388 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b740047a0 con 0x5651cf1c0790 2024-04-26T21:56:35.363 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.361+0000 7f7b6e000640 1 -- 172.21.15.72:0/3012076388 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7b7c02e050 con 0x5651cf1c0790 2024-04-26T21:56:35.363 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.361+0000 7f7b9243a740 10 client.15687 fetch_fsmap learned FSMap version 27 2024-04-26T21:56:35.364 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.361+0000 7f7b9243a740 1 -- 172.21.15.72:0/3012076388 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5651cee363f0 con 0x5651cf1c0790 2024-04-26T21:56:35.365 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.361+0000 7f7b6e000640 1 -- 172.21.15.72:0/3012076388 <== mon.0 v2:172.21.15.72:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7b740047a0 con 0x5651cf1c0790 2024-04-26T21:56:35.365 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.362+0000 7f7b6e000640 1 -- 172.21.15.72:0/3012076388 <== mon.0 v2:172.21.15.72:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f7b7c010a30 con 0x5651cf1c0790 2024-04-26T21:56:35.365 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.362+0000 7f7b9243a740 10 client.15687 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T21:56:35.365 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.362+0000 7f7b9243a740 10 client.15687 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:56:35.365 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.362+0000 7f7b9243a740 20 client.15687 populate_metadata read hostname 'smithi072' 2024-04-26T21:56:35.365 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.362+0000 7f7b9243a740 1 --2- 172.21.15.72:0/3012076388 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x5651cf1deaa0 0x5651cf1e0f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:35.365 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.362+0000 7f7b9243a740 4 client.15687 mds_command: new command op to 34517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:56:35.365 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.362+0000 7f7b9243a740 1 -- 172.21.15.72:0/3012076388 --> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5651cf1327e0 con 0x5651cf1deaa0 2024-04-26T21:56:35.365 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.363+0000 7f7b87e00640 1 --2- 172.21.15.72:0/3012076388 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x5651cf1deaa0 0x5651cf1e0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:35.365 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.364+0000 7f7b6e000640 10 client.15687 ms_handle_connect on v2:172.21.15.76:6838/327145636 2024-04-26T21:56:35.366 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.364+0000 7f7b87e00640 1 --2- 172.21.15.72:0/3012076388 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x5651cf1deaa0 0x5651cf1e0f60 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8fad115297e69844 server_cookie=6233e854f2e612e6 in_seq=0 out_seq=0 2024-04-26T21:56:35.367 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.365+0000 7f7b6e000640 1 -- 172.21.15.72:0/3012076388 <== mds.0 v2:172.21.15.76:6838/327145636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5651cf1327e0 con 0x5651cf1deaa0 2024-04-26T21:56:35.367 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.365+0000 7f7b6e000640 10 client.15687 handle_command_reply: tid=0 2024-04-26T21:56:35.367 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.365+0000 7f7b9243a740 1 client.15687 shutdown 2024-04-26T21:56:35.367 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.366+0000 7f7b9243a740 1 -- 172.21.15.72:0/3012076388 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5651cf1c0790 msgr2=0x5651cf1d2f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:35.367 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.366+0000 7f7b9243a740 1 --2- 172.21.15.72:0/3012076388 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5651cf1c0790 0x5651cf1d2f10 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f7b7c00ff70 tx=0x7f7b7c005850 comp rx=0 tx=0).stop 2024-04-26T21:56:35.367 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.366+0000 7f7b9243a740 1 -- 172.21.15.72:0/3012076388 shutdown_connections 2024-04-26T21:56:35.367 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.366+0000 7f7b9243a740 1 --2- 172.21.15.72:0/3012076388 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x5651cf1deaa0 0x5651cf1e0f60 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:35.367 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.366+0000 7f7b9243a740 1 --2- 172.21.15.72:0/3012076388 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5651cf1c62f0 0x5651cf1da490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:35.367 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.366+0000 7f7b9243a740 1 --2- 172.21.15.72:0/3012076388 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5651cf1c1180 0x5651cf1d3450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:35.367 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.366+0000 7f7b9243a740 1 --2- 172.21.15.72:0/3012076388 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5651cf1c0790 0x5651cf1d2f10 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:35.368 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.366+0000 7f7b9243a740 1 -- 172.21.15.72:0/3012076388 >> 172.21.15.72:0/3012076388 conn(0x5651cf1bbdb0 msgr2=0x5651cf1bb040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:56:35.368 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.366+0000 7f7b6e000640 0 client.15687 ms_handle_reset on v2:172.21.15.76:6838/327145636 2024-04-26T21:56:35.368 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.366+0000 7f7b9243a740 1 -- 172.21.15.72:0/3012076388 shutdown_connections 2024-04-26T21:56:35.368 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.366+0000 7f7b9243a740 1 -- 172.21.15.72:0/3012076388 wait complete. 2024-04-26T21:56:35.368 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.366+0000 7f7b9243a740 20 client.15687 trim_cache size 0 max 16384 2024-04-26T21:56:35.385 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.385+0000 7f7b9243a740 1 Processor -- start 2024-04-26T21:56:35.386 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.385+0000 7f7b9243a740 1 -- start start 2024-04-26T21:56:35.386 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.386+0000 7f7b9243a740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5651cf1c0790 0x5651cf1d33b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:35.386 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.386+0000 7f7b9243a740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5651cf1c1180 0x5651cf1c1ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:35.386 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.386+0000 7f7b9243a740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5651cf1c62f0 0x5651cf1c20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:35.386 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.386+0000 7f7b9243a740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f7b881a9820 con 0x5651cf1c62f0 2024-04-26T21:56:35.386 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.386+0000 7f7b9243a740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f7b881a83b0 con 0x5651cf1c1180 2024-04-26T21:56:35.386 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.386+0000 7f7b9243a740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f7b881a8a80 con 0x5651cf1c0790 2024-04-26T21:56:35.388 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.386+0000 7f7b8d600640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5651cf1c62f0 0x5651cf1c20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:35.388 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.386+0000 7f7b8d600640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5651cf1c62f0 0x5651cf1c20e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:58482/0 (socket says 172.21.15.72:58482) 2024-04-26T21:56:35.388 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.386+0000 7f7b8d600640 1 -- 172.21.15.72:0/733483048 learned_addr learned my addr 172.21.15.72:0/733483048 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:56:35.388 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.386+0000 7f7b87e00640 1 --2- 172.21.15.72:0/733483048 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5651cf1c1180 0x5651cf1c1ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:35.388 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.386+0000 7f7b8cc00640 1 --2- 172.21.15.72:0/733483048 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5651cf1c0790 0x5651cf1d33b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:35.388 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.387+0000 7f7b8d600640 1 -- 172.21.15.72:0/733483048 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5651cf1c0790 msgr2=0x5651cf1d33b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:56:35.388 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.387+0000 7f7b8d600640 1 --2- 172.21.15.72:0/733483048 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5651cf1c0790 0x5651cf1d33b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:35.388 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.387+0000 7f7b8d600640 1 -- 172.21.15.72:0/733483048 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5651cf1c1180 msgr2=0x5651cf1c1ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:56:35.388 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.387+0000 7f7b8d600640 1 --2- 172.21.15.72:0/733483048 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5651cf1c1180 0x5651cf1c1ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:35.389 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.387+0000 7f7b8d600640 1 -- 172.21.15.72:0/733483048 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5651cedf60f0 con 0x5651cf1c62f0 2024-04-26T21:56:35.389 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.387+0000 7f7b8d600640 1 --2- 172.21.15.72:0/733483048 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5651cf1c62f0 0x5651cf1c20e0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f7b800bda10 tx=0x7f7b80030640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efe29d2fcdf4c813 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:56:35.389 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.387+0000 7f7b79200640 1 -- 172.21.15.72:0/733483048 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b800d7270 con 0x5651cf1c62f0 2024-04-26T21:56:35.389 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.387+0000 7f7b79200640 1 -- 172.21.15.72:0/733483048 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7b80036040 con 0x5651cf1c62f0 2024-04-26T21:56:35.389 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.387+0000 7f7b79200640 1 -- 172.21.15.72:0/733483048 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b800bc0f0 con 0x5651cf1c62f0 2024-04-26T21:56:35.392 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.388+0000 7f7b9243a740 1 -- 172.21.15.72:0/733483048 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5651cf1c62f0 msgr2=0x5651cf1c20e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:35.393 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.388+0000 7f7b9243a740 1 --2- 172.21.15.72:0/733483048 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5651cf1c62f0 0x5651cf1c20e0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f7b800bda10 tx=0x7f7b80030640 comp rx=0 tx=0).stop 2024-04-26T21:56:35.393 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.389+0000 7f7b9243a740 1 -- 172.21.15.72:0/733483048 shutdown_connections 2024-04-26T21:56:35.393 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.389+0000 7f7b9243a740 1 --2- 172.21.15.72:0/733483048 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5651cf1c62f0 0x5651cf1c20e0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:35.393 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.389+0000 7f7b9243a740 1 --2- 172.21.15.72:0/733483048 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5651cf1c1180 0x5651cf1c1ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:35.393 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.389+0000 7f7b9243a740 1 --2- 172.21.15.72:0/733483048 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5651cf1c0790 0x5651cf1d33b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:35.393 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.389+0000 7f7b9243a740 1 -- 172.21.15.72:0/733483048 >> 172.21.15.72:0/733483048 conn(0x5651cf1bbdb0 msgr2=0x5651cf1bc190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:56:35.393 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.389+0000 7f7b9243a740 1 -- 172.21.15.72:0/733483048 shutdown_connections 2024-04-26T21:56:35.393 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.389+0000 7f7b9243a740 1 -- 172.21.15.72:0/733483048 wait complete. 2024-04-26T21:56:35.393 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.390+0000 7f7b9243a740 1 -- start start 2024-04-26T21:56:35.393 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.390+0000 7f7b9243a740 1 Processor -- start 2024-04-26T21:56:35.393 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.391+0000 7f7b9243a740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5651cf1c0790 0x5651cf1d6d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:35.393 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.391+0000 7f7b9243a740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5651cf1c1180 0x5651cf1d72c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:35.393 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.391+0000 7f7b9243a740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5651cf1c62f0 0x5651cf1d82d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:35.394 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.391+0000 7f7b9243a740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f7b881a5ce0 con 0x5651cf1c0790 2024-04-26T21:56:35.394 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.391+0000 7f7b9243a740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f7b881a9150 con 0x5651cf1c1180 2024-04-26T21:56:35.394 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.391+0000 7f7b9243a740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f7b881a6330 con 0x5651cf1c62f0 2024-04-26T21:56:35.394 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.391+0000 7f7b9243a740 10 client.0 _ll_register_callbacks cb 0x7f7b7c015070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:56:35.394 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.391+0000 7f7b8cc00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5651cf1c0790 0x5651cf1d6d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:35.394 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.391+0000 7f7b8cc00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5651cf1c0790 0x5651cf1d6d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:58484/0 (socket says 172.21.15.72:58484) 2024-04-26T21:56:35.394 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.391+0000 7f7b8cc00640 1 -- 172.21.15.72:0/1163547563 learned_addr learned my addr 172.21.15.72:0/1163547563 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:56:35.394 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.391+0000 7f7b87e00640 1 --2- 172.21.15.72:0/1163547563 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5651cf1c1180 0x5651cf1d72c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:35.394 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.391+0000 7f7b8cc00640 1 -- 172.21.15.72:0/1163547563 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5651cf1c62f0 msgr2=0x5651cf1d82d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:56:35.394 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.391+0000 7f7b8cc00640 1 --2- 172.21.15.72:0/1163547563 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5651cf1c62f0 0x5651cf1d82d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:35.394 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.392+0000 7f7b8cc00640 1 -- 172.21.15.72:0/1163547563 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5651cf1c1180 msgr2=0x5651cf1d72c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:56:35.394 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.392+0000 7f7b8cc00640 1 --2- 172.21.15.72:0/1163547563 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5651cf1c1180 0x5651cf1d72c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:35.395 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.392+0000 7f7b8cc00640 1 -- 172.21.15.72:0/1163547563 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5651cf0eca30 con 0x5651cf1c0790 2024-04-26T21:56:35.395 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.392+0000 7f7b8cc00640 1 --2- 172.21.15.72:0/1163547563 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5651cf1c0790 0x5651cf1d6d80 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f7b7c005780 tx=0x7f7b7c026630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ada631a53d748801 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:56:35.395 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.392+0000 7f7b6e000640 10 client.0 ms_handle_connect on v2:172.21.15.72:3300/0 2024-04-26T21:56:35.395 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.392+0000 7f7b6e000640 1 -- 172.21.15.72:0/1163547563 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b7c022040 con 0x5651cf1c0790 2024-04-26T21:56:35.395 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.392+0000 7f7b9243a740 1 -- 172.21.15.72:0/1163547563 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b80036040 con 0x5651cf1c0790 2024-04-26T21:56:35.395 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.392+0000 7f7b6e000640 1 -- 172.21.15.72:0/1163547563 <== mon.0 v2:172.21.15.72:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7f7b7c026be0 con 0x5651cf1c0790 2024-04-26T21:56:35.395 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.393+0000 7f7b6e000640 1 -- 172.21.15.72:0/1163547563 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7b7c042050 con 0x5651cf1c0790 2024-04-26T21:56:35.395 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.393+0000 7f7b9243a740 10 client.15693 fetch_fsmap learned FSMap version 27 2024-04-26T21:56:35.395 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.393+0000 7f7b9243a740 1 -- 172.21.15.72:0/1163547563 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5651cf1d9a70 con 0x5651cf1c0790 2024-04-26T21:56:35.397 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.393+0000 7f7b6e000640 1 -- 172.21.15.72:0/1163547563 <== mon.0 v2:172.21.15.72:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7b80036040 con 0x5651cf1c0790 2024-04-26T21:56:35.397 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.393+0000 7f7b6e000640 1 -- 172.21.15.72:0/1163547563 <== mon.0 v2:172.21.15.72:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f7b7c01e660 con 0x5651cf1c0790 2024-04-26T21:56:35.397 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.393+0000 7f7b9243a740 10 client.15693 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T21:56:35.397 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.393+0000 7f7b9243a740 10 client.15693 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:56:35.398 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.393+0000 7f7b9243a740 20 client.15693 populate_metadata read hostname 'smithi072' 2024-04-26T21:56:35.398 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.393+0000 7f7b9243a740 1 --2- 172.21.15.72:0/1163547563 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x5651cf245650 0x5651cf247a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:35.398 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.393+0000 7f7b9243a740 4 client.15693 mds_command: new command op to 34517 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-26T21:56:35.398 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.393+0000 7f7b9243a740 1 -- 172.21.15.72:0/1163547563 --> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5651cf12c650 con 0x5651cf245650 2024-04-26T21:56:35.398 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.394+0000 7f7b87e00640 1 --2- 172.21.15.72:0/1163547563 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x5651cf245650 0x5651cf247a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:35.398 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.395+0000 7f7b87e00640 1 --2- 172.21.15.72:0/1163547563 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x5651cf245650 0x5651cf247a70 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f7254a678600a50 server_cookie=de7f3ecc6dee4154 in_seq=0 out_seq=0 2024-04-26T21:56:35.398 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.395+0000 7f7b6e000640 10 client.15693 ms_handle_connect on v2:172.21.15.76:6838/327145636 2024-04-26T21:56:35.398 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.396+0000 7f7b6e000640 1 -- 172.21.15.72:0/1163547563 <== mds.0 v2:172.21.15.76:6838/327145636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5651cf12c650 con 0x5651cf245650 2024-04-26T21:56:35.398 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.396+0000 7f7b6e000640 10 client.15693 handle_command_reply: tid=0 2024-04-26T21:56:35.398 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.396+0000 7f7b9243a740 1 client.15693 shutdown 2024-04-26T21:56:35.398 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.397+0000 7f7b9243a740 1 -- 172.21.15.72:0/1163547563 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5651cf1c0790 msgr2=0x5651cf1d6d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:35.398 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.397+0000 7f7b9243a740 1 --2- 172.21.15.72:0/1163547563 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5651cf1c0790 0x5651cf1d6d80 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f7b7c005780 tx=0x7f7b7c026630 comp rx=0 tx=0).stop 2024-04-26T21:56:35.398 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.397+0000 7f7b9243a740 1 -- 172.21.15.72:0/1163547563 shutdown_connections 2024-04-26T21:56:35.398 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.397+0000 7f7b9243a740 1 --2- 172.21.15.72:0/1163547563 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x5651cf245650 0x5651cf247a70 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:35.399 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.397+0000 7f7b9243a740 1 --2- 172.21.15.72:0/1163547563 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5651cf1c62f0 0x5651cf1d82d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:35.399 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.397+0000 7f7b9243a740 1 --2- 172.21.15.72:0/1163547563 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5651cf1c1180 0x5651cf1d72c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:35.399 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.397+0000 7f7b9243a740 1 --2- 172.21.15.72:0/1163547563 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5651cf1c0790 0x5651cf1d6d80 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:35.399 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.397+0000 7f7b9243a740 1 -- 172.21.15.72:0/1163547563 >> 172.21.15.72:0/1163547563 conn(0x5651cf1bbdb0 msgr2=0x5651cf1ba7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:56:35.399 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.397+0000 7f7b6e000640 0 client.15693 ms_handle_reset on v2:172.21.15.76:6838/327145636 2024-04-26T21:56:35.399 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.397+0000 7f7b9243a740 1 -- 172.21.15.72:0/1163547563 shutdown_connections 2024-04-26T21:56:35.399 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.397+0000 7f7b9243a740 1 -- 172.21.15.72:0/1163547563 wait complete. 2024-04-26T21:56:35.399 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.397+0000 7f7b9243a740 20 client.15693 trim_cache size 0 max 16384 2024-04-26T21:56:35.400 INFO:teuthology.orchestra.run.smithi072.stdout:{ 2024-04-26T21:56:35.400 INFO:teuthology.orchestra.run.smithi072.stdout: "status": "no active scrubs running", 2024-04-26T21:56:35.400 INFO:teuthology.orchestra.run.smithi072.stdout: "scrubs": {} 2024-04-26T21:56:35.400 INFO:teuthology.orchestra.run.smithi072.stdout:} 2024-04-26T21:56:35.401 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.398+0000 7f7b8f600640 1 -- 172.21.15.72:0/351765875 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f7b700a3cc0 msgr2=0x7f7b700a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:35.401 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.398+0000 7f7b8f600640 1 --2- 172.21.15.72:0/351765875 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f7b700a3cc0 0x7f7b700a6180 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x5651cf1c1580 tx=0x7f7b7c005f30 comp rx=0 tx=0).stop 2024-04-26T21:56:35.401 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.398+0000 7f7b8f600640 1 -- 172.21.15.72:0/351765875 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7b88104520 msgr2=0x7f7b88075260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:35.401 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.398+0000 7f7b8f600640 1 --2- 172.21.15.72:0/351765875 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7b88104520 0x7f7b88075260 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f7b8002aeb0 tx=0x7f7b80003c20 comp rx=0 tx=0).stop 2024-04-26T21:56:35.401 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.398+0000 7f7b8f600640 1 -- 172.21.15.72:0/351765875 shutdown_connections 2024-04-26T21:56:35.402 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.398+0000 7f7b8f600640 1 --2- 172.21.15.72:0/351765875 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7b88104520 0x7f7b88075260 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:35.402 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.398+0000 7f7b8f600640 1 --2- 172.21.15.72:0/351765875 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7b88103b60 0x7f7b88074d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:35.402 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.398+0000 7f7b8f600640 1 --2- 172.21.15.72:0/351765875 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f7b700a3cc0 0x7f7b700a6180 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:35.402 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.398+0000 7f7b8f600640 1 --2- 172.21.15.72:0/351765875 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7b88102970 0x7f7b88076690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:35.402 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.398+0000 7f7b8f600640 1 -- 172.21.15.72:0/351765875 >> 172.21.15.72:0/351765875 conn(0x7f7b880fdee0 msgr2=0x7f7b88106f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:56:35.402 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.399+0000 7f7b8f600640 1 -- 172.21.15.72:0/351765875 shutdown_connections 2024-04-26T21:56:35.402 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.399+0000 7f7b8f600640 1 -- 172.21.15.72:0/351765875 wait complete. 2024-04-26T21:56:35.421 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-26T21:56:35.422 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:56:35.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:56:34 smithi164 ceph-mon[31916]: pgmap v483: 97 pgs: 97 active+clean; 650 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-26T21:56:35.590 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.590+0000 7f9a07200640 1 -- 172.21.15.72:0/1267802561 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9a081031f0 msgr2=0x7f9a081055e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:35.590 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.590+0000 7f9a07200640 1 --2- 172.21.15.72:0/1267802561 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9a081031f0 0x7f9a081055e0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f99f8009900 tx=0x7f99f802a410 comp rx=0 tx=0).stop 2024-04-26T21:56:35.591 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.590+0000 7f9a07200640 1 -- 172.21.15.72:0/1267802561 shutdown_connections 2024-04-26T21:56:35.591 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.590+0000 7f9a07200640 1 --2- 172.21.15.72:0/1267802561 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9a08108450 0x7f9a0810a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:35.591 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.590+0000 7f9a07200640 1 --2- 172.21.15.72:0/1267802561 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9a08105b20 0x7f9a08107f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:35.591 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.590+0000 7f9a07200640 1 --2- 172.21.15.72:0/1267802561 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9a081031f0 0x7f9a081055e0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:35.591 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.590+0000 7f9a07200640 1 -- 172.21.15.72:0/1267802561 >> 172.21.15.72:0/1267802561 conn(0x7f9a080fcbc0 msgr2=0x7f9a080ff020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:56:35.591 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.590+0000 7f9a07200640 1 -- 172.21.15.72:0/1267802561 shutdown_connections 2024-04-26T21:56:35.591 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.590+0000 7f9a07200640 1 -- 172.21.15.72:0/1267802561 wait complete. 2024-04-26T21:56:35.592 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.591+0000 7f9a07200640 1 Processor -- start 2024-04-26T21:56:35.593 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.591+0000 7f9a07200640 1 -- start start 2024-04-26T21:56:35.593 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.592+0000 7f9a07200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9a081031f0 0x7f9a08196680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:35.593 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.592+0000 7f9a07200640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9a08105b20 0x7f9a08196bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:35.593 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.592+0000 7f9a07200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9a08108450 0x7f9a0819dc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:35.593 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.592+0000 7f9a07200640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f9a0810bfc0 con 0x7f9a08105b20 2024-04-26T21:56:35.593 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.592+0000 7f9a07200640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f9a0810be40 con 0x7f9a081031f0 2024-04-26T21:56:35.593 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.592+0000 7f9a07200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f9a0810c140 con 0x7f9a08108450 2024-04-26T21:56:35.593 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.592+0000 7f9a05400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9a08105b20 0x7f9a08196bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:35.593 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.592+0000 7f9a05400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9a08105b20 0x7f9a08196bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:58514/0 (socket says 172.21.15.72:58514) 2024-04-26T21:56:35.593 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.592+0000 7f9a05400640 1 -- 172.21.15.72:0/12066193 learned_addr learned my addr 172.21.15.72:0/12066193 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:56:35.593 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.592+0000 7f9a05e00640 1 --2- 172.21.15.72:0/12066193 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9a081031f0 0x7f9a08196680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:35.593 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.592+0000 7f9a06800640 1 --2- 172.21.15.72:0/12066193 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9a08108450 0x7f9a0819dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:35.593 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.592+0000 7f9a05400640 1 -- 172.21.15.72:0/12066193 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9a08108450 msgr2=0x7f9a0819dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:35.594 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.593+0000 7f9a05400640 1 --2- 172.21.15.72:0/12066193 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9a08108450 0x7f9a0819dc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:35.594 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.593+0000 7f9a05400640 1 -- 172.21.15.72:0/12066193 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9a081031f0 msgr2=0x7f9a08196680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:35.594 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.593+0000 7f9a05400640 1 --2- 172.21.15.72:0/12066193 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9a081031f0 0x7f9a08196680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:35.594 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.593+0000 7f9a05400640 1 -- 172.21.15.72:0/12066193 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a0819e2a0 con 0x7f9a08105b20 2024-04-26T21:56:35.595 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.593+0000 7f9a05400640 1 --2- 172.21.15.72:0/12066193 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9a08105b20 0x7f9a08196bc0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f99f4005fd0 tx=0x7f99f400dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:56:35.595 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.593+0000 7f99fea00640 1 -- 172.21.15.72:0/12066193 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99f4017070 con 0x7f9a08105b20 2024-04-26T21:56:35.595 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.593+0000 7f9a07200640 1 -- 172.21.15.72:0/12066193 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a0819e590 con 0x7f9a08105b20 2024-04-26T21:56:35.595 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.594+0000 7f9a07200640 1 -- 172.21.15.72:0/12066193 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a0819eb30 con 0x7f9a08105b20 2024-04-26T21:56:35.595 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.594+0000 7f99fea00640 1 -- 172.21.15.72:0/12066193 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f99f4009c30 con 0x7f9a08105b20 2024-04-26T21:56:35.595 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.594+0000 7f99fea00640 1 -- 172.21.15.72:0/12066193 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99f4003c70 con 0x7f9a08105b20 2024-04-26T21:56:35.595 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.595+0000 7f9a07200640 1 -- 172.21.15.72:0/12066193 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99e80052c0 con 0x7f9a08105b20 2024-04-26T21:56:35.598 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.598+0000 7f99fea00640 1 -- 172.21.15.72:0/12066193 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f99f4003e10 con 0x7f9a08105b20 2024-04-26T21:56:35.599 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.599+0000 7f99fea00640 1 --2- 172.21.15.72:0/12066193 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f99ec0a3cc0 0x7f99ec0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:35.600 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.599+0000 7f99fea00640 1 -- 172.21.15.72:0/12066193 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7f99f40d0ac0 con 0x7f9a08105b20 2024-04-26T21:56:35.600 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.600+0000 7f9a05e00640 1 --2- 172.21.15.72:0/12066193 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f99ec0a3cc0 0x7f99ec0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:35.601 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.601+0000 7f9a05e00640 1 --2- 172.21.15.72:0/12066193 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f99ec0a3cc0 0x7f99ec0a6180 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f99f8006010 tx=0x7f99f8031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:56:35.606 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.606+0000 7f99fea00640 1 -- 172.21.15.72:0/12066193 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f99f4020030 con 0x7f9a08105b20 2024-04-26T21:56:35.897 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.897+0000 7f9a07200640 1 -- 172.21.15.72:0/12066193 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f99e80055b0 con 0x7f9a08105b20 2024-04-26T21:56:35.898 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.898+0000 7f99fea00640 1 -- 172.21.15.72:0/12066193 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11931 (secure 0 0 0) 0x7f99f409d260 con 0x7f9a08105b20 2024-04-26T21:56:35.898 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T21:56:35.898 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6835/1320381959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6834","nonce":1320381959},{"type":"v1","addr":"172.21.15.72:6835","nonce":1320381959}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6837/2142200512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6836","nonce":2142200512},{"type":"v1","addr":"172.21.15.72:6837","nonce":2142200512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T21:52:50.567857+0000","modified":"2024-04-26T21:55:22.894383+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34517,"mds_1":34472,"mds_2":34442,"mds_3":34406,"mds_4":24463},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14652":{"gid":14652,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.72:6839/1312898281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6838","nonce":1312898281},{"type":"v1","addr":"172.21.15.72:6839","nonce":1312898281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14664":{"gid":14664,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.72:6841/3257692339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6840","nonce":3257692339},{"type":"v1","addr":"172.21.15.72:6841","nonce":3257692339}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24388":{"gid":24388,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6833/4236668465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":4236668465},{"type":"v1","addr":"172.21.15.164:6833","nonce":4236668465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6835/2391114169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":2391114169},{"type":"v1","addr":"172.21.15.164:6835","nonce":2391114169}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6837/3585829992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":3585829992},{"type":"v1","addr":"172.21.15.164:6837","nonce":3585829992}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.164:6839/1636226820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":1636226820},{"type":"v1","addr":"172.21.15.164:6839","nonce":1636226820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34406":{"gid":34406,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":15,"addr":"172.21.15.76:6833/3869483750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":3869483750},{"type":"v1","addr":"172.21.15.76:6833","nonce":3869483750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34442":{"gid":34442,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.76:6835/3517429435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":3517429435},{"type":"v1","addr":"172.21.15.76:6835","nonce":3517429435}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34472":{"gid":34472,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.76:6837/1150678991","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":1150678991},{"type":"v1","addr":"172.21.15.76:6837","nonce":1150678991}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34517":{"gid":34517,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.76:6839/327145636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6838","nonce":327145636},{"type":"v1","addr":"172.21.15.76:6839","nonce":327145636}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34517,"qdb_cluster":[34472,34517,34442,34406,24463]},"id":1}]} 2024-04-26T21:56:35.899 INFO:teuthology.orchestra.run.smithi072.stderr:dumped fsmap epoch 27 2024-04-26T21:56:35.900 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.900+0000 7f9a07200640 1 -- 172.21.15.72:0/12066193 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f99ec0a3cc0 msgr2=0x7f99ec0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:35.901 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.900+0000 7f9a07200640 1 --2- 172.21.15.72:0/12066193 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f99ec0a3cc0 0x7f99ec0a6180 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f99f8006010 tx=0x7f99f8031400 comp rx=0 tx=0).stop 2024-04-26T21:56:35.901 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.900+0000 7f9a07200640 1 -- 172.21.15.72:0/12066193 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9a08105b20 msgr2=0x7f9a08196bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:35.901 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.900+0000 7f9a07200640 1 --2- 172.21.15.72:0/12066193 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9a08105b20 0x7f9a08196bc0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f99f4005fd0 tx=0x7f99f400dde0 comp rx=0 tx=0).stop 2024-04-26T21:56:35.901 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.900+0000 7f9a07200640 1 -- 172.21.15.72:0/12066193 shutdown_connections 2024-04-26T21:56:35.901 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.900+0000 7f9a07200640 1 --2- 172.21.15.72:0/12066193 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f99ec0a3cc0 0x7f99ec0a6180 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:35.901 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.900+0000 7f9a07200640 1 --2- 172.21.15.72:0/12066193 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9a08108450 0x7f9a0819dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:35.901 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.900+0000 7f9a07200640 1 --2- 172.21.15.72:0/12066193 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9a08105b20 0x7f9a08196bc0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:35.901 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.900+0000 7f9a07200640 1 --2- 172.21.15.72:0/12066193 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9a081031f0 0x7f9a08196680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:35.901 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.900+0000 7f9a07200640 1 -- 172.21.15.72:0/12066193 >> 172.21.15.72:0/12066193 conn(0x7f9a080fcbc0 msgr2=0x7f9a08103cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:56:35.901 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.900+0000 7f9a07200640 1 -- 172.21.15.72:0/12066193 shutdown_connections 2024-04-26T21:56:35.901 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:35.900+0000 7f9a07200640 1 -- 172.21.15.72:0/12066193 wait complete. 2024-04-26T21:56:35.926 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-26T21:56:36.096 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.095+0000 7fdea2200640 1 -- 172.21.15.72:0/54568676 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fde9c101a70 msgr2=0x7fde9c101ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:36.096 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.095+0000 7fdea2200640 1 --2- 172.21.15.72:0/54568676 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fde9c101a70 0x7fde9c101ed0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fde88009900 tx=0x7fde8802a3f0 comp rx=0 tx=0).stop 2024-04-26T21:56:36.096 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.096+0000 7fdea2200640 1 -- 172.21.15.72:0/54568676 shutdown_connections 2024-04-26T21:56:36.096 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.096+0000 7fdea2200640 1 --2- 172.21.15.72:0/54568676 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fde9c102410 0x7fde9c102890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:36.096 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.096+0000 7fdea2200640 1 --2- 172.21.15.72:0/54568676 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fde9c101a70 0x7fde9c101ed0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:36.096 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.096+0000 7fdea2200640 1 --2- 172.21.15.72:0/54568676 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fde9c101110 0x7fde9c101530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:36.096 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.096+0000 7fdea2200640 1 -- 172.21.15.72:0/54568676 >> 172.21.15.72:0/54568676 conn(0x7fde9c0fccd0 msgr2=0x7fde9c0ff0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:56:36.096 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.096+0000 7fdea2200640 1 -- 172.21.15.72:0/54568676 shutdown_connections 2024-04-26T21:56:36.099 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.096+0000 7fdea2200640 1 -- 172.21.15.72:0/54568676 wait complete. 2024-04-26T21:56:36.099 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.097+0000 7fdea2200640 1 Processor -- start 2024-04-26T21:56:36.099 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.097+0000 7fdea2200640 1 -- start start 2024-04-26T21:56:36.099 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.098+0000 7fdea2200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fde9c101110 0x7fde9c19abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:36.099 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.098+0000 7fdea2200640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fde9c101a70 0x7fde9c19b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:36.100 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.098+0000 7fdea2200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fde9c102410 0x7fde9c1a2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:36.100 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.098+0000 7fdea2200640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fde9c1104b0 con 0x7fde9c101a70 2024-04-26T21:56:36.100 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.098+0000 7fdea2200640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fde9c110330 con 0x7fde9c102410 2024-04-26T21:56:36.100 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.098+0000 7fdea2200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fde9c110630 con 0x7fde9c101110 2024-04-26T21:56:36.100 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.098+0000 7fde9be00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fde9c101a70 0x7fde9c19b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:36.100 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.098+0000 7fde9be00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fde9c101a70 0x7fde9c19b100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:58534/0 (socket says 172.21.15.72:58534) 2024-04-26T21:56:36.100 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.098+0000 7fde9be00640 1 -- 172.21.15.72:0/3075758946 learned_addr learned my addr 172.21.15.72:0/3075758946 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:56:36.100 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.098+0000 7fdea0e00640 1 --2- 172.21.15.72:0/3075758946 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fde9c101110 0x7fde9c19abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:36.100 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.098+0000 7fdea1800640 1 --2- 172.21.15.72:0/3075758946 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fde9c102410 0x7fde9c1a2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:36.100 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.099+0000 7fde9be00640 1 -- 172.21.15.72:0/3075758946 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fde9c101110 msgr2=0x7fde9c19abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:36.100 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.099+0000 7fde9be00640 1 --2- 172.21.15.72:0/3075758946 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fde9c101110 0x7fde9c19abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:36.100 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.099+0000 7fde9be00640 1 -- 172.21.15.72:0/3075758946 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fde9c102410 msgr2=0x7fde9c1a2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:36.100 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.099+0000 7fde9be00640 1 --2- 172.21.15.72:0/3075758946 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fde9c102410 0x7fde9c1a2170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:36.100 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.099+0000 7fde9be00640 1 -- 172.21.15.72:0/3075758946 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde9c1a27e0 con 0x7fde9c101a70 2024-04-26T21:56:36.101 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.099+0000 7fdea1800640 1 --2- 172.21.15.72:0/3075758946 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fde9c102410 0x7fde9c1a2170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:56:36.101 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.099+0000 7fdea0e00640 1 --2- 172.21.15.72:0/3075758946 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fde9c101110 0x7fde9c19abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:56:36.101 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.099+0000 7fde9be00640 1 --2- 172.21.15.72:0/3075758946 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fde9c101a70 0x7fde9c19b100 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fde88031460 tx=0x7fde88003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:56:36.101 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.100+0000 7fde99600640 1 -- 172.21.15.72:0/3075758946 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde8803f070 con 0x7fde9c101a70 2024-04-26T21:56:36.102 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.100+0000 7fde99600640 1 -- 172.21.15.72:0/3075758946 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fde88031da0 con 0x7fde9c101a70 2024-04-26T21:56:36.102 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.100+0000 7fdea2200640 1 -- 172.21.15.72:0/3075758946 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde9c1a2a70 con 0x7fde9c101a70 2024-04-26T21:56:36.102 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.100+0000 7fde99600640 1 -- 172.21.15.72:0/3075758946 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde88035030 con 0x7fde9c101a70 2024-04-26T21:56:36.102 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.100+0000 7fdea2200640 1 -- 172.21.15.72:0/3075758946 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde9c1a2fb0 con 0x7fde9c101a70 2024-04-26T21:56:36.104 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.104+0000 7fdea677f740 1 Processor -- start 2024-04-26T21:56:36.104 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.105+0000 7fdea677f740 1 -- start start 2024-04-26T21:56:36.105 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.105+0000 7fdea677f740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562dbc048050 0x562dbc048470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:36.105 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.105+0000 7fdea677f740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x562dbc048a40 0x562dbc04d540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:36.105 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.105+0000 7fdea677f740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562dbc04dbb0 0x562dbc050040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:36.105 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.105+0000 7fdea677f740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x562dbbfabc70 con 0x562dbc048a40 2024-04-26T21:56:36.106 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.105+0000 7fdea677f740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x562dbbf64980 con 0x562dbc04dbb0 2024-04-26T21:56:36.106 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.105+0000 7fdea677f740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x562dbbfa7c30 con 0x562dbc048050 2024-04-26T21:56:36.106 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.105+0000 7fde99600640 1 -- 172.21.15.72:0/3075758946 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fde88042070 con 0x7fde9c101a70 2024-04-26T21:56:36.107 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.106+0000 7fdea0e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x562dbc048a40 0x562dbc04d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:36.108 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.106+0000 7fdea0e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x562dbc048a40 0x562dbc04d540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:58550/0 (socket says 172.21.15.72:58550) 2024-04-26T21:56:36.108 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.106+0000 7fdea0e00640 1 -- 172.21.15.72:0/3686838528 learned_addr learned my addr 172.21.15.72:0/3686838528 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:56:36.108 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.106+0000 7fdea1800640 1 --2- 172.21.15.72:0/3686838528 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562dbc048050 0x562dbc048470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:36.108 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.106+0000 7fdea0e00640 1 -- 172.21.15.72:0/3686838528 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562dbc048050 msgr2=0x562dbc048470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:56:36.108 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.106+0000 7fdea0e00640 1 --2- 172.21.15.72:0/3686838528 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562dbc048050 0x562dbc048470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:36.108 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.106+0000 7fdea0e00640 1 -- 172.21.15.72:0/3686838528 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562dbc04dbb0 msgr2=0x562dbc050040 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T21:56:36.108 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.106+0000 7fdea0e00640 1 --2- 172.21.15.72:0/3686838528 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562dbc04dbb0 0x562dbc050040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:36.108 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.106+0000 7fdea0e00640 1 -- 172.21.15.72:0/3686838528 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562dbbf3c7b0 con 0x562dbc048a40 2024-04-26T21:56:36.108 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.107+0000 7fdea0e00640 1 --2- 172.21.15.72:0/3686838528 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x562dbc048a40 0x562dbc04d540 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fde9000bd40 tx=0x7fde9000f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14969fd707558feb server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:56:36.108 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.107+0000 7fde8e000640 1 -- 172.21.15.72:0/3686838528 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde90015070 con 0x562dbc048a40 2024-04-26T21:56:36.108 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.107+0000 7fde8e000640 1 -- 172.21.15.72:0/3686838528 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fde9000be30 con 0x562dbc048a40 2024-04-26T21:56:36.110 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.107+0000 7fde8e000640 1 -- 172.21.15.72:0/3686838528 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde90011830 con 0x562dbc048a40 2024-04-26T21:56:36.110 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.108+0000 7fde99600640 1 --2- 172.21.15.72:0/3075758946 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fde840a3cc0 0x7fde840a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:36.110 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.108+0000 7fde99600640 1 -- 172.21.15.72:0/3075758946 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7fde880f0fc0 con 0x7fde9c101a70 2024-04-26T21:56:36.110 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.108+0000 7fdea1800640 1 --2- 172.21.15.72:0/3075758946 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fde840a3cc0 0x7fde840a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:36.110 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.108+0000 7fdea677f740 1 -- 172.21.15.72:0/3686838528 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x562dbc048a40 msgr2=0x562dbc04d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:36.110 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.108+0000 7fdea677f740 1 --2- 172.21.15.72:0/3686838528 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x562dbc048a40 0x562dbc04d540 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fde9000bd40 tx=0x7fde9000f730 comp rx=0 tx=0).stop 2024-04-26T21:56:36.110 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.109+0000 7fdea1800640 1 --2- 172.21.15.72:0/3075758946 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fde840a3cc0 0x7fde840a6180 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x562dbc0492a0 tx=0x7fde9400e040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:56:36.110 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.109+0000 7fdea677f740 1 -- 172.21.15.72:0/3686838528 shutdown_connections 2024-04-26T21:56:36.110 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.109+0000 7fdea677f740 1 --2- 172.21.15.72:0/3686838528 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562dbc04dbb0 0x562dbc050040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:36.110 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.109+0000 7fdea677f740 1 --2- 172.21.15.72:0/3686838528 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x562dbc048a40 0x562dbc04d540 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:36.110 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.109+0000 7fdea677f740 1 --2- 172.21.15.72:0/3686838528 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562dbc048050 0x562dbc048470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:36.111 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.109+0000 7fdea677f740 1 -- 172.21.15.72:0/3686838528 >> 172.21.15.72:0/3686838528 conn(0x562dbc0436c0 msgr2=0x562dbc043ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:56:36.111 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.109+0000 7fdea677f740 1 -- 172.21.15.72:0/3686838528 shutdown_connections 2024-04-26T21:56:36.111 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.109+0000 7fdea677f740 1 -- 172.21.15.72:0/3686838528 wait complete. 2024-04-26T21:56:36.111 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.110+0000 7fdea677f740 1 -- start start 2024-04-26T21:56:36.113 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.110+0000 7fdea677f740 1 Processor -- start 2024-04-26T21:56:36.113 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.111+0000 7fdea677f740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562dbc048050 0x562dbc063070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:36.113 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.111+0000 7fdea677f740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562dbc048a40 0x562dbc0635b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:36.113 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.111+0000 7fdea677f740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x562dbc04dbb0 0x562dbc06a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:36.113 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.111+0000 7fdea677f740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x562dbbfaaf80 con 0x562dbc04dbb0 2024-04-26T21:56:36.113 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.111+0000 7fdea677f740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x562dbbf58260 con 0x562dbc048a40 2024-04-26T21:56:36.113 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.111+0000 7fdea677f740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x562dbbfb02a0 con 0x562dbc048050 2024-04-26T21:56:36.114 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.111+0000 7fdea677f740 10 client.0 _ll_register_callbacks cb 0x562dbbfa2960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:56:36.114 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.111+0000 7fdea1800640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x562dbc04dbb0 0x562dbc06a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:36.114 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.111+0000 7fdea1800640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x562dbc04dbb0 0x562dbc06a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:58554/0 (socket says 172.21.15.72:58554) 2024-04-26T21:56:36.114 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.111+0000 7fdea1800640 1 -- 172.21.15.72:0/1590892143 learned_addr learned my addr 172.21.15.72:0/1590892143 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:56:36.114 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.112+0000 7fdea0e00640 1 --2- 172.21.15.72:0/1590892143 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562dbc048050 0x562dbc063070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:36.114 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.112+0000 7fdea1800640 1 -- 172.21.15.72:0/1590892143 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562dbc048050 msgr2=0x562dbc063070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:56:36.114 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.112+0000 7fdea1800640 1 --2- 172.21.15.72:0/1590892143 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562dbc048050 0x562dbc063070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:36.114 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.112+0000 7fde9be00640 1 --2- 172.21.15.72:0/1590892143 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562dbc048a40 0x562dbc0635b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:36.114 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.112+0000 7fdea1800640 1 -- 172.21.15.72:0/1590892143 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562dbc048a40 msgr2=0x562dbc0635b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:56:36.114 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.112+0000 7fdea1800640 1 --2- 172.21.15.72:0/1590892143 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562dbc048a40 0x562dbc0635b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:36.114 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.112+0000 7fdea1800640 1 -- 172.21.15.72:0/1590892143 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562dbbf3a1c0 con 0x562dbc04dbb0 2024-04-26T21:56:36.114 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.112+0000 7fdea1800640 1 --2- 172.21.15.72:0/1590892143 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x562dbc04dbb0 0x562dbc06a5f0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fde94007e00 tx=0x7fde94004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=381502ebf692590b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:56:36.114 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.112+0000 7fde82a00640 10 client.0 ms_handle_connect on v2:172.21.15.72:3300/0 2024-04-26T21:56:36.115 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.113+0000 7fde82a00640 1 -- 172.21.15.72:0/1590892143 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde94026020 con 0x562dbc04dbb0 2024-04-26T21:56:36.115 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.113+0000 7fdea677f740 1 -- 172.21.15.72:0/1590892143 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fde9000be30 con 0x562dbc04dbb0 2024-04-26T21:56:36.115 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.113+0000 7fde82a00640 1 -- 172.21.15.72:0/1590892143 <== mon.0 v2:172.21.15.72:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7fde940223f0 con 0x562dbc04dbb0 2024-04-26T21:56:36.115 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.113+0000 7fde82a00640 1 -- 172.21.15.72:0/1590892143 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fde9401e040 con 0x562dbc04dbb0 2024-04-26T21:56:36.115 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.113+0000 7fdea677f740 10 client.15711 fetch_fsmap learned FSMap version 27 2024-04-26T21:56:36.115 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.113+0000 7fdea677f740 1 -- 172.21.15.72:0/1590892143 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x562dbbcbd3f0 con 0x562dbc04dbb0 2024-04-26T21:56:36.115 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.113+0000 7fde82a00640 1 -- 172.21.15.72:0/1590892143 <== mon.0 v2:172.21.15.72:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fde9000be30 con 0x562dbc04dbb0 2024-04-26T21:56:36.116 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.114+0000 7fde82a00640 1 -- 172.21.15.72:0/1590892143 <== mon.0 v2:172.21.15.72:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fde94021030 con 0x562dbc04dbb0 2024-04-26T21:56:36.116 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.114+0000 7fdea677f740 10 client.15711 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T21:56:36.116 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.114+0000 7fdea677f740 10 client.15711 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-04-26T21:56:36.116 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.114+0000 7fdea677f740 20 client.15711 populate_metadata read hostname 'smithi072' 2024-04-26T21:56:36.116 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.114+0000 7fdea677f740 1 --2- 172.21.15.72:0/1590892143 >> [v2:172.21.15.164:6838/1636226820,v1:172.21.15.164:6839/1636226820] conn(0x562dbc06eca0 0x562dbc071120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:36.116 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.114+0000 7fdea677f740 4 client.15711 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:56:36.116 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.114+0000 7fdea677f740 1 -- 172.21.15.72:0/1590892143 --> [v2:172.21.15.164:6838/1636226820,v1:172.21.15.164:6839/1636226820] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562dbbfb9880 con 0x562dbc06eca0 2024-04-26T21:56:36.116 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.115+0000 7fdea0e00640 1 --2- 172.21.15.72:0/1590892143 >> [v2:172.21.15.164:6838/1636226820,v1:172.21.15.164:6839/1636226820] conn(0x562dbc06eca0 0x562dbc071120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:36.117 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.115+0000 7fdea0e00640 1 --2- 172.21.15.72:0/1590892143 >> [v2:172.21.15.164:6838/1636226820,v1:172.21.15.164:6839/1636226820] conn(0x562dbc06eca0 0x562dbc071120 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=89ad4aae79513ab5 server_cookie=eeae8274cb375bc9 in_seq=0 out_seq=0 2024-04-26T21:56:36.117 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.116+0000 7fde82a00640 10 client.15711 ms_handle_connect on v2:172.21.15.164:6838/1636226820 2024-04-26T21:56:36.117 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.117+0000 7fde82a00640 1 -- 172.21.15.72:0/1590892143 <== mds.4 v2:172.21.15.164:6838/1636226820 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562dbbfb9880 con 0x562dbc06eca0 2024-04-26T21:56:36.117 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.117+0000 7fde82a00640 10 client.15711 handle_command_reply: tid=0 2024-04-26T21:56:36.117 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.117+0000 7fdea677f740 1 client.15711 shutdown 2024-04-26T21:56:36.118 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.118+0000 7fdea677f740 1 -- 172.21.15.72:0/1590892143 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x562dbc04dbb0 msgr2=0x562dbc06a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:36.118 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.118+0000 7fdea677f740 1 --2- 172.21.15.72:0/1590892143 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x562dbc04dbb0 0x562dbc06a5f0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fde94007e00 tx=0x7fde94004a40 comp rx=0 tx=0).stop 2024-04-26T21:56:36.118 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.118+0000 7fdea677f740 1 -- 172.21.15.72:0/1590892143 shutdown_connections 2024-04-26T21:56:36.119 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.118+0000 7fdea677f740 1 --2- 172.21.15.72:0/1590892143 >> [v2:172.21.15.164:6838/1636226820,v1:172.21.15.164:6839/1636226820] conn(0x562dbc06eca0 0x562dbc071120 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:36.119 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.118+0000 7fde82a00640 0 client.15711 ms_handle_reset on v2:172.21.15.164:6838/1636226820 2024-04-26T21:56:36.119 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.118+0000 7fdea677f740 1 --2- 172.21.15.72:0/1590892143 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x562dbc04dbb0 0x562dbc06a5f0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:36.119 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.118+0000 7fdea677f740 1 --2- 172.21.15.72:0/1590892143 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562dbc048a40 0x562dbc0635b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:36.119 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.118+0000 7fdea677f740 1 --2- 172.21.15.72:0/1590892143 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562dbc048050 0x562dbc063070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:36.119 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.118+0000 7fdea677f740 1 -- 172.21.15.72:0/1590892143 >> 172.21.15.72:0/1590892143 conn(0x562dbc0436c0 msgr2=0x562dbc0427b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:56:36.119 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.118+0000 7fdea677f740 1 -- 172.21.15.72:0/1590892143 shutdown_connections 2024-04-26T21:56:36.119 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.118+0000 7fdea677f740 1 -- 172.21.15.72:0/1590892143 wait complete. 2024-04-26T21:56:36.119 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.118+0000 7fdea677f740 20 client.15711 trim_cache size 0 max 16384 2024-04-26T21:56:36.133 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.133+0000 7fdea677f740 1 Processor -- start 2024-04-26T21:56:36.133 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.133+0000 7fdea677f740 1 -- start start 2024-04-26T21:56:36.133 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.133+0000 7fdea677f740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x562dbc048050 0x562dbc046e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:36.133 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.133+0000 7fdea677f740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562dbc048a40 0x562dbc06a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:36.133 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.133+0000 7fdea677f740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562dbc04dbb0 0x562dbc042630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:36.133 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.133+0000 7fdea677f740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fde9c1a37d0 con 0x562dbc048050 2024-04-26T21:56:36.134 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.133+0000 7fdea677f740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fde9c1a5310 con 0x562dbc048a40 2024-04-26T21:56:36.134 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.133+0000 7fdea677f740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fde9c1a3ea0 con 0x562dbc04dbb0 2024-04-26T21:56:36.135 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.134+0000 7fdea0e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x562dbc048050 0x562dbc046e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:36.135 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.134+0000 7fdea0e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x562dbc048050 0x562dbc046e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:58562/0 (socket says 172.21.15.72:58562) 2024-04-26T21:56:36.135 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.134+0000 7fdea0e00640 1 -- 172.21.15.72:0/3410406643 learned_addr learned my addr 172.21.15.72:0/3410406643 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:56:36.135 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.134+0000 7fde9be00640 1 --2- 172.21.15.72:0/3410406643 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562dbc048a40 0x562dbc06a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:36.135 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.134+0000 7fdea0e00640 1 -- 172.21.15.72:0/3410406643 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562dbc04dbb0 msgr2=0x562dbc042630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:56:36.135 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.134+0000 7fdea0e00640 1 --2- 172.21.15.72:0/3410406643 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562dbc04dbb0 0x562dbc042630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:36.135 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.134+0000 7fdea0e00640 1 -- 172.21.15.72:0/3410406643 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562dbc048a40 msgr2=0x562dbc06a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:56:36.135 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.134+0000 7fdea0e00640 1 --2- 172.21.15.72:0/3410406643 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562dbc048a40 0x562dbc06a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:36.135 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.134+0000 7fdea0e00640 1 -- 172.21.15.72:0/3410406643 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562dbbd877d0 con 0x562dbc048050 2024-04-26T21:56:36.135 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.134+0000 7fde9be00640 1 --2- 172.21.15.72:0/3410406643 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562dbc048a40 0x562dbc06a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:56:36.136 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.134+0000 7fdea0e00640 1 --2- 172.21.15.72:0/3410406643 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x562dbc048050 0x562dbc046e60 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fde9000bc90 tx=0x7fde9000b660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f74f36d185f6dcdf server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:56:36.136 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.135+0000 7fde8d600640 1 -- 172.21.15.72:0/3410406643 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde90015040 con 0x562dbc048050 2024-04-26T21:56:36.136 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.135+0000 7fde8d600640 1 -- 172.21.15.72:0/3410406643 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fde900054d0 con 0x562dbc048050 2024-04-26T21:56:36.136 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.135+0000 7fde8d600640 1 -- 172.21.15.72:0/3410406643 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde900049b0 con 0x562dbc048050 2024-04-26T21:56:36.138 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.135+0000 7fdea677f740 1 -- 172.21.15.72:0/3410406643 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x562dbc048050 msgr2=0x562dbc046e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:36.138 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.135+0000 7fdea677f740 1 --2- 172.21.15.72:0/3410406643 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x562dbc048050 0x562dbc046e60 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fde9000bc90 tx=0x7fde9000b660 comp rx=0 tx=0).stop 2024-04-26T21:56:36.138 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.135+0000 7fdea677f740 1 -- 172.21.15.72:0/3410406643 shutdown_connections 2024-04-26T21:56:36.138 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.135+0000 7fdea677f740 1 --2- 172.21.15.72:0/3410406643 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562dbc04dbb0 0x562dbc042630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:36.138 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.135+0000 7fdea677f740 1 --2- 172.21.15.72:0/3410406643 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562dbc048a40 0x562dbc06a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:36.138 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.135+0000 7fdea677f740 1 --2- 172.21.15.72:0/3410406643 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x562dbc048050 0x562dbc046e60 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:36.138 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.135+0000 7fdea677f740 1 -- 172.21.15.72:0/3410406643 >> 172.21.15.72:0/3410406643 conn(0x562dbc0436c0 msgr2=0x562dbc041f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:56:36.138 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.136+0000 7fdea677f740 1 -- 172.21.15.72:0/3410406643 shutdown_connections 2024-04-26T21:56:36.139 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.136+0000 7fdea677f740 1 -- 172.21.15.72:0/3410406643 wait complete. 2024-04-26T21:56:36.139 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.136+0000 7fdea677f740 1 -- start start 2024-04-26T21:56:36.139 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.136+0000 7fdea677f740 1 Processor -- start 2024-04-26T21:56:36.139 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.137+0000 7fdea677f740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562dbc048a40 0x562dbc067ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:36.139 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.137+0000 7fdea677f740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x562dbc04dbb0 0x562dbc068020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:36.139 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.137+0000 7fdea677f740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562dbc068fe0 0x562dbc069460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:36.139 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.137+0000 7fdea677f740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fde9c1a4c40 con 0x562dbc04dbb0 2024-04-26T21:56:36.139 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.137+0000 7fdea677f740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fde9c1a4570 con 0x562dbc068fe0 2024-04-26T21:56:36.139 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.137+0000 7fdea677f740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fde9c073070 con 0x562dbc048a40 2024-04-26T21:56:36.139 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.137+0000 7fdea677f740 10 client.0 _ll_register_callbacks cb 0x7fde94009d90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:56:36.139 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.137+0000 7fde9be00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x562dbc04dbb0 0x562dbc068020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:36.139 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.137+0000 7fde9be00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x562dbc04dbb0 0x562dbc068020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:58572/0 (socket says 172.21.15.72:58572) 2024-04-26T21:56:36.140 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.137+0000 7fde9be00640 1 -- 172.21.15.72:0/4287761140 learned_addr learned my addr 172.21.15.72:0/4287761140 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:56:36.140 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.137+0000 7fdea0e00640 1 --2- 172.21.15.72:0/4287761140 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562dbc048a40 0x562dbc067ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:36.140 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.138+0000 7fdea1800640 1 --2- 172.21.15.72:0/4287761140 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562dbc068fe0 0x562dbc069460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:36.140 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.138+0000 7fde9be00640 1 -- 172.21.15.72:0/4287761140 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562dbc048a40 msgr2=0x562dbc067ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:56:36.140 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.138+0000 7fde9be00640 1 --2- 172.21.15.72:0/4287761140 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562dbc048a40 0x562dbc067ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:36.140 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.138+0000 7fde9be00640 1 -- 172.21.15.72:0/4287761140 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562dbc068fe0 msgr2=0x562dbc069460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:56:36.140 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.138+0000 7fde9be00640 1 --2- 172.21.15.72:0/4287761140 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562dbc068fe0 0x562dbc069460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:36.140 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.138+0000 7fde9be00640 1 -- 172.21.15.72:0/4287761140 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562dbbf522a0 con 0x562dbc04dbb0 2024-04-26T21:56:36.140 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.138+0000 7fde9be00640 1 --2- 172.21.15.72:0/4287761140 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x562dbc04dbb0 0x562dbc068020 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fde880c57a0 tx=0x7fde880bc9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59ab74842830de61 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:56:36.140 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.138+0000 7fde82a00640 10 client.0 ms_handle_connect on v2:172.21.15.72:3300/0 2024-04-26T21:56:36.143 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.138+0000 7fde82a00640 1 -- 172.21.15.72:0/4287761140 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde88049050 con 0x562dbc04dbb0 2024-04-26T21:56:36.143 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.138+0000 7fde82a00640 1 -- 172.21.15.72:0/4287761140 <== mon.0 v2:172.21.15.72:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7fde880eeae0 con 0x562dbc04dbb0 2024-04-26T21:56:36.143 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.138+0000 7fdea677f740 1 -- 172.21.15.72:0/4287761140 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fde900054d0 con 0x562dbc04dbb0 2024-04-26T21:56:36.143 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.139+0000 7fde82a00640 1 -- 172.21.15.72:0/4287761140 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fde880c1410 con 0x562dbc04dbb0 2024-04-26T21:56:36.143 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.139+0000 7fdea677f740 10 client.15717 fetch_fsmap learned FSMap version 27 2024-04-26T21:56:36.143 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.139+0000 7fdea677f740 1 -- 172.21.15.72:0/4287761140 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x562dbc0d5260 con 0x562dbc04dbb0 2024-04-26T21:56:36.143 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.139+0000 7fde82a00640 1 -- 172.21.15.72:0/4287761140 <== mon.0 v2:172.21.15.72:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fde900054d0 con 0x562dbc04dbb0 2024-04-26T21:56:36.143 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.139+0000 7fde82a00640 1 -- 172.21.15.72:0/4287761140 <== mon.0 v2:172.21.15.72:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fde880c5840 con 0x562dbc04dbb0 2024-04-26T21:56:36.143 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.139+0000 7fdea677f740 10 client.15717 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T21:56:36.143 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.139+0000 7fdea677f740 10 client.15717 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-04-26T21:56:36.143 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.139+0000 7fdea677f740 20 client.15717 populate_metadata read hostname 'smithi072' 2024-04-26T21:56:36.143 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.139+0000 7fdea677f740 1 --2- 172.21.15.72:0/4287761140 >> [v2:172.21.15.164:6838/1636226820,v1:172.21.15.164:6839/1636226820] conn(0x562dbc0d5b30 0x562dbc0d7f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:36.144 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.139+0000 7fdea677f740 4 client.15717 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-26T21:56:36.144 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.139+0000 7fdea677f740 1 -- 172.21.15.72:0/4287761140 --> [v2:172.21.15.164:6838/1636226820,v1:172.21.15.164:6839/1636226820] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562dbbfb36f0 con 0x562dbc0d5b30 2024-04-26T21:56:36.144 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.140+0000 7fdea0e00640 1 --2- 172.21.15.72:0/4287761140 >> [v2:172.21.15.164:6838/1636226820,v1:172.21.15.164:6839/1636226820] conn(0x562dbc0d5b30 0x562dbc0d7f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:36.144 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.141+0000 7fde82a00640 10 client.15717 ms_handle_connect on v2:172.21.15.164:6838/1636226820 2024-04-26T21:56:36.144 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.141+0000 7fdea0e00640 1 --2- 172.21.15.72:0/4287761140 >> [v2:172.21.15.164:6838/1636226820,v1:172.21.15.164:6839/1636226820] conn(0x562dbc0d5b30 0x562dbc0d7f50 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e98d3a90c7cea0f8 server_cookie=7b81c97d9da17f5b in_seq=0 out_seq=0 2024-04-26T21:56:36.144 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.142+0000 7fde82a00640 1 -- 172.21.15.72:0/4287761140 <== mds.4 v2:172.21.15.164:6838/1636226820 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562dbbfb36f0 con 0x562dbc0d5b30 2024-04-26T21:56:36.144 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.142+0000 7fde82a00640 10 client.15717 handle_command_reply: tid=0 2024-04-26T21:56:36.144 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.142+0000 7fdea677f740 1 client.15717 shutdown 2024-04-26T21:56:36.144 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.142+0000 7fdea677f740 1 -- 172.21.15.72:0/4287761140 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x562dbc04dbb0 msgr2=0x562dbc068020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:36.144 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.142+0000 7fdea677f740 1 --2- 172.21.15.72:0/4287761140 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x562dbc04dbb0 0x562dbc068020 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fde880c57a0 tx=0x7fde880bc9c0 comp rx=0 tx=0).stop 2024-04-26T21:56:36.144 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.142+0000 7fdea677f740 1 -- 172.21.15.72:0/4287761140 shutdown_connections 2024-04-26T21:56:36.144 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.142+0000 7fdea677f740 1 --2- 172.21.15.72:0/4287761140 >> [v2:172.21.15.164:6838/1636226820,v1:172.21.15.164:6839/1636226820] conn(0x562dbc0d5b30 0x562dbc0d7f50 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:36.144 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.142+0000 7fdea677f740 1 --2- 172.21.15.72:0/4287761140 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562dbc068fe0 0x562dbc069460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:36.144 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.142+0000 7fde82a00640 0 client.15717 ms_handle_reset on v2:172.21.15.164:6838/1636226820 2024-04-26T21:56:36.145 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.142+0000 7fdea677f740 1 --2- 172.21.15.72:0/4287761140 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x562dbc04dbb0 0x562dbc068020 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:36.145 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.142+0000 7fdea677f740 1 --2- 172.21.15.72:0/4287761140 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562dbc048a40 0x562dbc067ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:36.145 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.142+0000 7fdea677f740 1 -- 172.21.15.72:0/4287761140 >> 172.21.15.72:0/4287761140 conn(0x562dbc0436c0 msgr2=0x562dbc046ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:56:36.145 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.142+0000 7fdea677f740 1 -- 172.21.15.72:0/4287761140 shutdown_connections 2024-04-26T21:56:36.145 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.142+0000 7fdea677f740 1 -- 172.21.15.72:0/4287761140 wait complete. 2024-04-26T21:56:36.145 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.143+0000 7fdea677f740 20 client.15717 trim_cache size 0 max 16384 2024-04-26T21:56:36.145 INFO:teuthology.orchestra.run.smithi072.stdout:[] 2024-04-26T21:56:36.146 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.143+0000 7fdea2200640 1 -- 172.21.15.72:0/3075758946 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fde840a3cc0 msgr2=0x7fde840a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:36.146 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.143+0000 7fdea2200640 1 --2- 172.21.15.72:0/3075758946 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fde840a3cc0 0x7fde840a6180 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x562dbc0492a0 tx=0x7fde9400e040 comp rx=0 tx=0).stop 2024-04-26T21:56:36.146 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.143+0000 7fdea2200640 1 -- 172.21.15.72:0/3075758946 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fde9c101a70 msgr2=0x7fde9c19b100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:36.146 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.143+0000 7fdea2200640 1 --2- 172.21.15.72:0/3075758946 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fde9c101a70 0x7fde9c19b100 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fde88031460 tx=0x7fde88003c80 comp rx=0 tx=0).stop 2024-04-26T21:56:36.146 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.144+0000 7fdea2200640 1 -- 172.21.15.72:0/3075758946 shutdown_connections 2024-04-26T21:56:36.146 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.144+0000 7fdea2200640 1 --2- 172.21.15.72:0/3075758946 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fde9c102410 0x7fde9c1a2170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:36.147 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.144+0000 7fdea2200640 1 --2- 172.21.15.72:0/3075758946 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fde9c101a70 0x7fde9c19b100 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:36.147 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.144+0000 7fdea2200640 1 --2- 172.21.15.72:0/3075758946 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fde840a3cc0 0x7fde840a6180 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:36.147 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.144+0000 7fdea2200640 1 --2- 172.21.15.72:0/3075758946 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fde9c101110 0x7fde9c19abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:36.147 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.144+0000 7fdea2200640 1 -- 172.21.15.72:0/3075758946 >> 172.21.15.72:0/3075758946 conn(0x7fde9c0fccd0 msgr2=0x7fde9c0fe480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:56:36.147 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.144+0000 7fdea2200640 1 -- 172.21.15.72:0/3075758946 shutdown_connections 2024-04-26T21:56:36.147 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.144+0000 7fdea2200640 1 -- 172.21.15.72:0/3075758946 wait complete. 2024-04-26T21:56:36.159 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-26T21:56:36.210 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:56:35 smithi072 ceph-mon[28138]: from='client.? 172.21.15.72:0/12066193' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:56:36.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:56:35 smithi076 ceph-mon[31823]: from='client.? 172.21.15.72:0/12066193' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:56:36.326 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.326+0000 7fcee5a00640 1 -- 172.21.15.72:0/1503610534 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcee0101a70 msgr2=0x7fcee0101ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:36.326 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.326+0000 7fcee5a00640 1 --2- 172.21.15.72:0/1503610534 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcee0101a70 0x7fcee0101ed0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fcecc009900 tx=0x7fcecc02a410 comp rx=0 tx=0).stop 2024-04-26T21:56:36.327 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.326+0000 7fcee5a00640 1 -- 172.21.15.72:0/1503610534 shutdown_connections 2024-04-26T21:56:36.327 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.326+0000 7fcee5a00640 1 --2- 172.21.15.72:0/1503610534 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcee0102410 0x7fcee0102890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:36.327 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.326+0000 7fcee5a00640 1 --2- 172.21.15.72:0/1503610534 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcee0101a70 0x7fcee0101ed0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:36.327 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.326+0000 7fcee5a00640 1 --2- 172.21.15.72:0/1503610534 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcee0101110 0x7fcee0101530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:36.327 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.326+0000 7fcee5a00640 1 -- 172.21.15.72:0/1503610534 >> 172.21.15.72:0/1503610534 conn(0x7fcee00fccd0 msgr2=0x7fcee00ff0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:56:36.327 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.326+0000 7fcee5a00640 1 -- 172.21.15.72:0/1503610534 shutdown_connections 2024-04-26T21:56:36.327 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.326+0000 7fcee5a00640 1 -- 172.21.15.72:0/1503610534 wait complete. 2024-04-26T21:56:36.330 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.327+0000 7fcee5a00640 1 Processor -- start 2024-04-26T21:56:36.330 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.327+0000 7fcee5a00640 1 -- start start 2024-04-26T21:56:36.330 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.328+0000 7fcee5a00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcee0101110 0x7fcee019ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:36.330 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.328+0000 7fcee5a00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcee0101a70 0x7fcee019b160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:36.330 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.328+0000 7fcee5a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcee0102410 0x7fcee01a2180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:36.330 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.328+0000 7fcee5a00640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fcee0110480 con 0x7fcee0101a70 2024-04-26T21:56:36.330 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.328+0000 7fcee5a00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fcee0110300 con 0x7fcee0101110 2024-04-26T21:56:36.330 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.328+0000 7fcee5a00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fcee0110600 con 0x7fcee0102410 2024-04-26T21:56:36.331 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.328+0000 7fcedf400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcee0101a70 0x7fcee019b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:36.331 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.328+0000 7fcedf400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcee0101a70 0x7fcee019b160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:58594/0 (socket says 172.21.15.72:58594) 2024-04-26T21:56:36.331 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.328+0000 7fcedf400640 1 -- 172.21.15.72:0/474356277 learned_addr learned my addr 172.21.15.72:0/474356277 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:56:36.331 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.328+0000 7fcedfe00640 1 --2- 172.21.15.72:0/474356277 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcee0101110 0x7fcee019ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:36.331 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.328+0000 7fcee5000640 1 --2- 172.21.15.72:0/474356277 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcee0102410 0x7fcee01a2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:36.331 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.329+0000 7fcedf400640 1 -- 172.21.15.72:0/474356277 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcee0102410 msgr2=0x7fcee01a2180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:36.331 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.329+0000 7fcedf400640 1 --2- 172.21.15.72:0/474356277 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcee0102410 0x7fcee01a2180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:36.331 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.329+0000 7fcedf400640 1 -- 172.21.15.72:0/474356277 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcee0101110 msgr2=0x7fcee019ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:36.331 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.329+0000 7fcedf400640 1 --2- 172.21.15.72:0/474356277 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcee0101110 0x7fcee019ac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:36.331 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.329+0000 7fcedf400640 1 -- 172.21.15.72:0/474356277 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcee01a27f0 con 0x7fcee0101a70 2024-04-26T21:56:36.331 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.329+0000 7fcedf400640 1 --2- 172.21.15.72:0/474356277 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcee0101a70 0x7fcee019b160 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fcecc009230 tx=0x7fcecc0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:56:36.332 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.329+0000 7fcedcc00640 1 -- 172.21.15.72:0/474356277 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcecc03f030 con 0x7fcee0101a70 2024-04-26T21:56:36.332 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.329+0000 7fcedcc00640 1 -- 172.21.15.72:0/474356277 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcecc003c90 con 0x7fcee0101a70 2024-04-26T21:56:36.332 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.329+0000 7fcedcc00640 1 -- 172.21.15.72:0/474356277 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcecc035070 con 0x7fcee0101a70 2024-04-26T21:56:36.332 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.330+0000 7fcee5a00640 1 -- 172.21.15.72:0/474356277 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcee01a2a80 con 0x7fcee0101a70 2024-04-26T21:56:36.332 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.330+0000 7fcee5a00640 1 -- 172.21.15.72:0/474356277 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcee01a2fa0 con 0x7fcee0101a70 2024-04-26T21:56:36.333 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.333+0000 7fcedcc00640 1 -- 172.21.15.72:0/474356277 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fcecc042030 con 0x7fcee0101a70 2024-04-26T21:56:36.334 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.334+0000 7fcee9de1740 1 Processor -- start 2024-04-26T21:56:36.334 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.334+0000 7fcee9de1740 1 -- start start 2024-04-26T21:56:36.335 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.335+0000 7fcee9de1740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c95da7e050 0x55c95da7e470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:36.335 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.335+0000 7fcee9de1740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c95da7ea40 0x55c95da83540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:36.336 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.335+0000 7fcee9de1740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c95da83bb0 0x55c95da86040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:36.336 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.335+0000 7fcee9de1740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55c95d9e1c70 con 0x55c95da83bb0 2024-04-26T21:56:36.336 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.335+0000 7fcee9de1740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55c95d99a980 con 0x55c95da7ea40 2024-04-26T21:56:36.336 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.335+0000 7fcee9de1740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55c95d9ddc30 con 0x55c95da7e050 2024-04-26T21:56:36.336 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.335+0000 7fcedf400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c95da83bb0 0x55c95da86040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:36.336 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.335+0000 7fcedf400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c95da83bb0 0x55c95da86040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:58606/0 (socket says 172.21.15.72:58606) 2024-04-26T21:56:36.336 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.335+0000 7fcedf400640 1 -- 172.21.15.72:0/749188781 learned_addr learned my addr 172.21.15.72:0/749188781 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:56:36.336 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.336+0000 7fcee5000640 1 --2- 172.21.15.72:0/749188781 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c95da7e050 0x55c95da7e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:36.336 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.336+0000 7fcedcc00640 1 --2- 172.21.15.72:0/474356277 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fcec40a3cc0 0x7fcec40a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:36.338 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.336+0000 7fcedcc00640 1 -- 172.21.15.72:0/474356277 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7fcecc0f0e10 con 0x7fcee0101a70 2024-04-26T21:56:36.338 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.336+0000 7fcedf400640 1 -- 172.21.15.72:0/749188781 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c95da7e050 msgr2=0x55c95da7e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:56:36.338 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.336+0000 7fcedf400640 1 --2- 172.21.15.72:0/749188781 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c95da7e050 0x55c95da7e470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:36.339 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.336+0000 7fcedf400640 1 -- 172.21.15.72:0/749188781 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c95da7ea40 msgr2=0x55c95da83540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:56:36.339 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.336+0000 7fcedfe00640 1 --2- 172.21.15.72:0/749188781 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c95da7ea40 0x55c95da83540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:36.339 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.336+0000 7fcee5000640 1 --2- 172.21.15.72:0/474356277 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fcec40a3cc0 0x7fcec40a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:36.339 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.336+0000 7fcedf400640 1 --2- 172.21.15.72:0/749188781 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c95da7ea40 0x55c95da83540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:36.339 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.336+0000 7fcedf400640 1 -- 172.21.15.72:0/749188781 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c95d9727b0 con 0x55c95da83bb0 2024-04-26T21:56:36.339 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.336+0000 7fcedfe00640 1 --2- 172.21.15.72:0/749188781 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c95da7ea40 0x55c95da83540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:56:36.339 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.336+0000 7fcedf400640 1 --2- 172.21.15.72:0/749188781 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c95da83bb0 0x55c95da86040 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fcecc0ee620 tx=0x7fcecc02a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b7a8aa2265fa580 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:56:36.339 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.337+0000 7fced5600640 1 -- 172.21.15.72:0/749188781 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcecc049070 con 0x55c95da83bb0 2024-04-26T21:56:36.340 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.337+0000 7fced5600640 1 -- 172.21.15.72:0/749188781 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcecc003e40 con 0x55c95da83bb0 2024-04-26T21:56:36.340 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.337+0000 7fced5600640 1 -- 172.21.15.72:0/749188781 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcecc0c1320 con 0x55c95da83bb0 2024-04-26T21:56:36.340 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.338+0000 7fcee9de1740 1 -- 172.21.15.72:0/749188781 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c95da83bb0 msgr2=0x55c95da86040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:36.340 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.338+0000 7fcee5000640 1 --2- 172.21.15.72:0/474356277 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fcec40a3cc0 0x7fcec40a6180 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x55c95da7f2a0 tx=0x7fced8004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:56:36.340 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.338+0000 7fcee9de1740 1 --2- 172.21.15.72:0/749188781 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c95da83bb0 0x55c95da86040 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fcecc0ee620 tx=0x7fcecc02a410 comp rx=0 tx=0).stop 2024-04-26T21:56:36.340 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.338+0000 7fcee9de1740 1 -- 172.21.15.72:0/749188781 shutdown_connections 2024-04-26T21:56:36.340 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.338+0000 7fcee9de1740 1 --2- 172.21.15.72:0/749188781 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c95da83bb0 0x55c95da86040 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:36.340 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.338+0000 7fcee9de1740 1 --2- 172.21.15.72:0/749188781 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c95da7ea40 0x55c95da83540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:36.340 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.338+0000 7fcee9de1740 1 --2- 172.21.15.72:0/749188781 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c95da7e050 0x55c95da7e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:36.340 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.338+0000 7fcee9de1740 1 -- 172.21.15.72:0/749188781 >> 172.21.15.72:0/749188781 conn(0x55c95da796c0 msgr2=0x55c95da79ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:56:36.340 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.338+0000 7fcee9de1740 1 -- 172.21.15.72:0/749188781 shutdown_connections 2024-04-26T21:56:36.341 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.338+0000 7fcee9de1740 1 -- 172.21.15.72:0/749188781 wait complete. 2024-04-26T21:56:36.344 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.339+0000 7fcee9de1740 1 -- start start 2024-04-26T21:56:36.344 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.339+0000 7fcee9de1740 1 Processor -- start 2024-04-26T21:56:36.344 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.340+0000 7fcee9de1740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c95da7e050 0x55c95da87c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:36.344 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.340+0000 7fcee9de1740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c95da7ea40 0x55c95da88190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:36.344 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.340+0000 7fcee9de1740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c95da83bb0 0x55c95da8f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:36.344 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.340+0000 7fcee9de1740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55c95d9e0f80 con 0x55c95da7e050 2024-04-26T21:56:36.344 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.340+0000 7fcee9de1740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55c95d98e260 con 0x55c95da83bb0 2024-04-26T21:56:36.344 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.340+0000 7fcee9de1740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55c95d9e62a0 con 0x55c95da7ea40 2024-04-26T21:56:36.344 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.340+0000 7fcee9de1740 10 client.0 _ll_register_callbacks cb 0x55c95d9d8960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:56:36.344 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.340+0000 7fcedfe00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c95da7e050 0x55c95da87c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:36.345 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.340+0000 7fcedfe00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c95da7e050 0x55c95da87c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:58614/0 (socket says 172.21.15.72:58614) 2024-04-26T21:56:36.345 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.340+0000 7fcedfe00640 1 -- 172.21.15.72:0/3594747356 learned_addr learned my addr 172.21.15.72:0/3594747356 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:56:36.345 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.340+0000 7fcee5000640 1 --2- 172.21.15.72:0/3594747356 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c95da83bb0 0x55c95da8f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:36.345 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.340+0000 7fcedf400640 1 --2- 172.21.15.72:0/3594747356 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c95da7ea40 0x55c95da88190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:36.345 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.341+0000 7fcedfe00640 1 -- 172.21.15.72:0/3594747356 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c95da7ea40 msgr2=0x55c95da88190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:56:36.345 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.341+0000 7fcedfe00640 1 --2- 172.21.15.72:0/3594747356 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c95da7ea40 0x55c95da88190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:36.345 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.341+0000 7fcedfe00640 1 -- 172.21.15.72:0/3594747356 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c95da83bb0 msgr2=0x55c95da8f1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:56:36.345 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.341+0000 7fcedfe00640 1 --2- 172.21.15.72:0/3594747356 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c95da83bb0 0x55c95da8f1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:36.345 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.341+0000 7fcedfe00640 1 -- 172.21.15.72:0/3594747356 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c95d9701c0 con 0x55c95da7e050 2024-04-26T21:56:36.345 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.341+0000 7fcedfe00640 1 --2- 172.21.15.72:0/3594747356 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c95da7e050 0x55c95da87c50 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fced0006aa0 tx=0x7fced000e680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20127059b4d2ea91 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:56:36.345 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.341+0000 7fceca000640 10 client.0 ms_handle_connect on v2:172.21.15.72:3300/0 2024-04-26T21:56:36.346 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.341+0000 7fceca000640 1 -- 172.21.15.72:0/3594747356 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fced0017020 con 0x55c95da7e050 2024-04-26T21:56:36.346 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.342+0000 7fceca000640 1 -- 172.21.15.72:0/3594747356 <== mon.0 v2:172.21.15.72:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7fced0013410 con 0x55c95da7e050 2024-04-26T21:56:36.346 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.342+0000 7fcee9de1740 1 -- 172.21.15.72:0/3594747356 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55c95d9ef880 con 0x55c95da7e050 2024-04-26T21:56:36.346 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.342+0000 7fceca000640 1 -- 172.21.15.72:0/3594747356 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fced0022050 con 0x55c95da7e050 2024-04-26T21:56:36.346 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.342+0000 7fcee9de1740 10 client.15729 fetch_fsmap learned FSMap version 27 2024-04-26T21:56:36.346 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.342+0000 7fcee9de1740 1 -- 172.21.15.72:0/3594747356 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c95d6f33f0 con 0x55c95da7e050 2024-04-26T21:56:36.346 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.342+0000 7fceca000640 1 -- 172.21.15.72:0/3594747356 <== mon.0 v2:172.21.15.72:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x55c95d9ef880 con 0x55c95da7e050 2024-04-26T21:56:36.346 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.342+0000 7fceca000640 1 -- 172.21.15.72:0/3594747356 <== mon.0 v2:172.21.15.72:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fced0012050 con 0x55c95da7e050 2024-04-26T21:56:36.346 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.343+0000 7fcee9de1740 10 client.15729 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T21:56:36.347 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.343+0000 7fcee9de1740 10 client.15729 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-26T21:56:36.347 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.343+0000 7fcee9de1740 20 client.15729 populate_metadata read hostname 'smithi072' 2024-04-26T21:56:36.347 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.343+0000 7fcee9de1740 1 --2- 172.21.15.72:0/3594747356 >> [v2:172.21.15.76:6832/3869483750,v1:172.21.15.76:6833/3869483750] conn(0x55c95da93790 0x55c95da95be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:36.347 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.343+0000 7fcee9de1740 4 client.15729 mds_command: new command op to 34406 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:56:36.347 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.343+0000 7fcee9de1740 1 -- 172.21.15.72:0/3594747356 --> [v2:172.21.15.76:6832/3869483750,v1:172.21.15.76:6833/3869483750] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c95d9e96f0 con 0x55c95da93790 2024-04-26T21:56:36.347 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.343+0000 7fcedf400640 1 --2- 172.21.15.72:0/3594747356 >> [v2:172.21.15.76:6832/3869483750,v1:172.21.15.76:6833/3869483750] conn(0x55c95da93790 0x55c95da95be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:36.348 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.344+0000 7fceca000640 10 client.15729 ms_handle_connect on v2:172.21.15.76:6832/3869483750 2024-04-26T21:56:36.348 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.344+0000 7fcedf400640 1 --2- 172.21.15.72:0/3594747356 >> [v2:172.21.15.76:6832/3869483750,v1:172.21.15.76:6833/3869483750] conn(0x55c95da93790 0x55c95da95be0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=c1f28fbeafaf5fff server_cookie=39f91feebd5b46da in_seq=0 out_seq=0 2024-04-26T21:56:36.349 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.346+0000 7fceca000640 1 -- 172.21.15.72:0/3594747356 <== mds.3 v2:172.21.15.76:6832/3869483750 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c95d9e96f0 con 0x55c95da93790 2024-04-26T21:56:36.349 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.346+0000 7fceca000640 10 client.15729 handle_command_reply: tid=0 2024-04-26T21:56:36.349 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.346+0000 7fcee9de1740 1 client.15729 shutdown 2024-04-26T21:56:36.349 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.346+0000 7fcee9de1740 1 -- 172.21.15.72:0/3594747356 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c95da7e050 msgr2=0x55c95da87c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:36.349 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.346+0000 7fcee9de1740 1 --2- 172.21.15.72:0/3594747356 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c95da7e050 0x55c95da87c50 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fced0006aa0 tx=0x7fced000e680 comp rx=0 tx=0).stop 2024-04-26T21:56:36.349 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.347+0000 7fcee9de1740 1 -- 172.21.15.72:0/3594747356 shutdown_connections 2024-04-26T21:56:36.349 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.347+0000 7fcee9de1740 1 --2- 172.21.15.72:0/3594747356 >> [v2:172.21.15.76:6832/3869483750,v1:172.21.15.76:6833/3869483750] conn(0x55c95da93790 0x55c95da95be0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:36.349 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.347+0000 7fceca000640 0 client.15729 ms_handle_reset on v2:172.21.15.76:6832/3869483750 2024-04-26T21:56:36.349 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.347+0000 7fcee9de1740 1 --2- 172.21.15.72:0/3594747356 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c95da83bb0 0x55c95da8f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:36.349 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.347+0000 7fcee9de1740 1 --2- 172.21.15.72:0/3594747356 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c95da7ea40 0x55c95da88190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:36.349 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.347+0000 7fcee9de1740 1 --2- 172.21.15.72:0/3594747356 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c95da7e050 0x55c95da87c50 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:36.350 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.347+0000 7fcee9de1740 1 -- 172.21.15.72:0/3594747356 >> 172.21.15.72:0/3594747356 conn(0x55c95da796c0 msgr2=0x55c95da787b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:56:36.350 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.347+0000 7fcee9de1740 1 -- 172.21.15.72:0/3594747356 shutdown_connections 2024-04-26T21:56:36.350 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.347+0000 7fcee9de1740 1 -- 172.21.15.72:0/3594747356 wait complete. 2024-04-26T21:56:36.350 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.347+0000 7fcee9de1740 20 client.15729 trim_cache size 0 max 16384 2024-04-26T21:56:36.362 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.362+0000 7fcee9de1740 1 Processor -- start 2024-04-26T21:56:36.362 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.362+0000 7fcee9de1740 1 -- start start 2024-04-26T21:56:36.362 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.362+0000 7fcee9de1740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c95da7e050 0x55c95da77f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:36.362 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.362+0000 7fcee9de1740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c95da7ea40 0x55c95da8f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:36.363 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.362+0000 7fcee9de1740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c95da83bb0 0x55c95da9a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:36.364 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.362+0000 7fcee9de1740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fcee01abf60 con 0x55c95da7e050 2024-04-26T21:56:36.364 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.362+0000 7fcee9de1740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fcee01adaa0 con 0x55c95da83bb0 2024-04-26T21:56:36.364 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.362+0000 7fcee9de1740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fcee01ac630 con 0x55c95da7ea40 2024-04-26T21:56:36.364 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.362+0000 7fcedfe00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c95da7e050 0x55c95da77f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:36.364 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.363+0000 7fcedfe00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c95da7e050 0x55c95da77f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:58626/0 (socket says 172.21.15.72:58626) 2024-04-26T21:56:36.364 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.363+0000 7fcedfe00640 1 -- 172.21.15.72:0/3926599273 learned_addr learned my addr 172.21.15.72:0/3926599273 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:56:36.364 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.363+0000 7fcedf400640 1 --2- 172.21.15.72:0/3926599273 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c95da7ea40 0x55c95da8f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:36.364 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.363+0000 7fcee5000640 1 --2- 172.21.15.72:0/3926599273 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c95da83bb0 0x55c95da9a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:36.364 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.363+0000 7fcedfe00640 1 -- 172.21.15.72:0/3926599273 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c95da7ea40 msgr2=0x55c95da8f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:56:36.364 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.363+0000 7fcedfe00640 1 --2- 172.21.15.72:0/3926599273 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c95da7ea40 0x55c95da8f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:36.364 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.363+0000 7fcedfe00640 1 -- 172.21.15.72:0/3926599273 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c95da83bb0 msgr2=0x55c95da9a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:56:36.365 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.363+0000 7fcedfe00640 1 --2- 172.21.15.72:0/3926599273 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c95da83bb0 0x55c95da9a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:36.365 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.363+0000 7fcedfe00640 1 -- 172.21.15.72:0/3926599273 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c95d7bd7d0 con 0x55c95da7e050 2024-04-26T21:56:36.365 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.363+0000 7fcedfe00640 1 --2- 172.21.15.72:0/3926599273 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c95da7e050 0x55c95da77f20 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fced001b8b0 tx=0x7fced000eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5f8ef407c65f1fe server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:56:36.365 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.364+0000 7fced4c00640 1 -- 172.21.15.72:0/3926599273 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fced0017070 con 0x55c95da7e050 2024-04-26T21:56:36.365 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.364+0000 7fced4c00640 1 -- 172.21.15.72:0/3926599273 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fced002b070 con 0x55c95da7e050 2024-04-26T21:56:36.365 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.364+0000 7fced4c00640 1 -- 172.21.15.72:0/3926599273 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fced000f040 con 0x55c95da7e050 2024-04-26T21:56:36.367 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.365+0000 7fcee9de1740 1 -- 172.21.15.72:0/3926599273 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c95da7e050 msgr2=0x55c95da77f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:36.367 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.365+0000 7fcee9de1740 1 --2- 172.21.15.72:0/3926599273 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c95da7e050 0x55c95da77f20 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fced001b8b0 tx=0x7fced000eb80 comp rx=0 tx=0).stop 2024-04-26T21:56:36.367 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.365+0000 7fcee9de1740 1 -- 172.21.15.72:0/3926599273 shutdown_connections 2024-04-26T21:56:36.367 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.365+0000 7fcee9de1740 1 --2- 172.21.15.72:0/3926599273 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c95da83bb0 0x55c95da9a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:36.367 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.365+0000 7fcee9de1740 1 --2- 172.21.15.72:0/3926599273 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c95da7ea40 0x55c95da8f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:36.367 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.365+0000 7fcee9de1740 1 --2- 172.21.15.72:0/3926599273 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c95da7e050 0x55c95da77f20 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:36.368 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.365+0000 7fcee9de1740 1 -- 172.21.15.72:0/3926599273 >> 172.21.15.72:0/3926599273 conn(0x55c95da796c0 msgr2=0x55c95da7fcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:56:36.368 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.365+0000 7fcee9de1740 1 -- 172.21.15.72:0/3926599273 shutdown_connections 2024-04-26T21:56:36.368 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.365+0000 7fcee9de1740 1 -- 172.21.15.72:0/3926599273 wait complete. 2024-04-26T21:56:36.368 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.365+0000 7fcee9de1740 1 -- start start 2024-04-26T21:56:36.368 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.366+0000 7fcee9de1740 1 Processor -- start 2024-04-26T21:56:36.368 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.366+0000 7fcee9de1740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c95da7e050 0x55c95da8bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:36.368 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.366+0000 7fcee9de1740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c95da7ea40 0x55c95da8c230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:36.368 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.366+0000 7fcee9de1740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c95da83bb0 0x55c95db00560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:36.368 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.366+0000 7fcee9de1740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fcee01ad3d0 con 0x55c95da7e050 2024-04-26T21:56:36.368 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.366+0000 7fcee9de1740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fcee01acd00 con 0x55c95da83bb0 2024-04-26T21:56:36.368 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.366+0000 7fcee9de1740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fcee01a9f40 con 0x55c95da7ea40 2024-04-26T21:56:36.368 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.366+0000 7fcee9de1740 10 client.0 _ll_register_callbacks cb 0x7fced0009120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:56:36.369 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.367+0000 7fcedfe00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c95da7e050 0x55c95da8bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:36.369 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.367+0000 7fcedfe00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c95da7e050 0x55c95da8bcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:58632/0 (socket says 172.21.15.72:58632) 2024-04-26T21:56:36.369 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.367+0000 7fcedfe00640 1 -- 172.21.15.72:0/3685646904 learned_addr learned my addr 172.21.15.72:0/3685646904 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:56:36.369 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.367+0000 7fcee5000640 1 --2- 172.21.15.72:0/3685646904 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c95da83bb0 0x55c95db00560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:36.369 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.367+0000 7fcedf400640 1 --2- 172.21.15.72:0/3685646904 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c95da7ea40 0x55c95da8c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:36.369 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.367+0000 7fcedfe00640 1 -- 172.21.15.72:0/3685646904 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c95da7ea40 msgr2=0x55c95da8c230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:56:36.369 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.367+0000 7fcedfe00640 1 --2- 172.21.15.72:0/3685646904 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c95da7ea40 0x55c95da8c230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:36.369 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.367+0000 7fcedfe00640 1 -- 172.21.15.72:0/3685646904 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c95da83bb0 msgr2=0x55c95db00560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:56:36.369 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.367+0000 7fcedfe00640 1 --2- 172.21.15.72:0/3685646904 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c95da83bb0 0x55c95db00560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:36.369 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.367+0000 7fcedfe00640 1 -- 172.21.15.72:0/3685646904 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c95d9882a0 con 0x55c95da7e050 2024-04-26T21:56:36.372 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.368+0000 7fcedfe00640 1 --2- 172.21.15.72:0/3685646904 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c95da7e050 0x55c95da8bcf0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fced0028040 tx=0x7fced0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4362cc41094a6d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:56:36.372 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.368+0000 7fceca000640 10 client.0 ms_handle_connect on v2:172.21.15.72:3300/0 2024-04-26T21:56:36.372 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.368+0000 7fceca000640 1 -- 172.21.15.72:0/3685646904 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fced0017040 con 0x55c95da7e050 2024-04-26T21:56:36.372 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.368+0000 7fceca000640 1 -- 172.21.15.72:0/3685646904 <== mon.0 v2:172.21.15.72:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7fced002f650 con 0x55c95da7e050 2024-04-26T21:56:36.372 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.368+0000 7fcee9de1740 1 -- 172.21.15.72:0/3685646904 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fced002b070 con 0x55c95da7e050 2024-04-26T21:56:36.372 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.368+0000 7fceca000640 1 -- 172.21.15.72:0/3685646904 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fced001bc30 con 0x55c95da7e050 2024-04-26T21:56:36.373 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.368+0000 7fcee9de1740 10 client.15735 fetch_fsmap learned FSMap version 27 2024-04-26T21:56:36.373 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.368+0000 7fcee9de1740 1 -- 172.21.15.72:0/3685646904 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c95db01800 con 0x55c95da7e050 2024-04-26T21:56:36.373 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.369+0000 7fceca000640 1 -- 172.21.15.72:0/3685646904 <== mon.0 v2:172.21.15.72:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fced0034070 con 0x55c95da7e050 2024-04-26T21:56:36.373 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.369+0000 7fceca000640 1 -- 172.21.15.72:0/3685646904 <== mon.0 v2:172.21.15.72:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fced0012040 con 0x55c95da7e050 2024-04-26T21:56:36.373 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.369+0000 7fcee9de1740 10 client.15735 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T21:56:36.373 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.369+0000 7fcee9de1740 10 client.15735 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-26T21:56:36.373 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.369+0000 7fcee9de1740 20 client.15735 populate_metadata read hostname 'smithi072' 2024-04-26T21:56:36.373 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.369+0000 7fcee9de1740 1 --2- 172.21.15.72:0/3685646904 >> [v2:172.21.15.76:6832/3869483750,v1:172.21.15.76:6833/3869483750] conn(0x55c95db02090 0x55c95db044b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:36.373 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.369+0000 7fcee9de1740 4 client.15735 mds_command: new command op to 34406 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-26T21:56:36.373 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.369+0000 7fcee9de1740 1 -- 172.21.15.72:0/3685646904 --> [v2:172.21.15.76:6832/3869483750,v1:172.21.15.76:6833/3869483750] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fced002b070 con 0x55c95db02090 2024-04-26T21:56:36.373 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.370+0000 7fcedf400640 1 --2- 172.21.15.72:0/3685646904 >> [v2:172.21.15.76:6832/3869483750,v1:172.21.15.76:6833/3869483750] conn(0x55c95db02090 0x55c95db044b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:36.373 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.371+0000 7fcedf400640 1 --2- 172.21.15.72:0/3685646904 >> [v2:172.21.15.76:6832/3869483750,v1:172.21.15.76:6833/3869483750] conn(0x55c95db02090 0x55c95db044b0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=67ff13b596d345f server_cookie=8141f19b9bb4b994 in_seq=0 out_seq=0 2024-04-26T21:56:36.374 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.371+0000 7fceca000640 10 client.15735 ms_handle_connect on v2:172.21.15.76:6832/3869483750 2024-04-26T21:56:36.374 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.371+0000 7fceca000640 1 -- 172.21.15.72:0/3685646904 <== mds.3 v2:172.21.15.76:6832/3869483750 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fced002b070 con 0x55c95db02090 2024-04-26T21:56:36.374 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.371+0000 7fceca000640 10 client.15735 handle_command_reply: tid=0 2024-04-26T21:56:36.374 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.371+0000 7fcee9de1740 1 client.15735 shutdown 2024-04-26T21:56:36.374 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.371+0000 7fcee9de1740 1 -- 172.21.15.72:0/3685646904 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c95da7e050 msgr2=0x55c95da8bcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:36.374 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.371+0000 7fcee9de1740 1 --2- 172.21.15.72:0/3685646904 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c95da7e050 0x55c95da8bcf0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fced0028040 tx=0x7fced0009250 comp rx=0 tx=0).stop 2024-04-26T21:56:36.374 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.372+0000 7fcee9de1740 1 -- 172.21.15.72:0/3685646904 shutdown_connections 2024-04-26T21:56:36.374 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.372+0000 7fcee9de1740 1 --2- 172.21.15.72:0/3685646904 >> [v2:172.21.15.76:6832/3869483750,v1:172.21.15.76:6833/3869483750] conn(0x55c95db02090 0x55c95db044b0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:36.374 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.372+0000 7fcee9de1740 1 --2- 172.21.15.72:0/3685646904 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c95da83bb0 0x55c95db00560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:36.374 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.372+0000 7fcee9de1740 1 --2- 172.21.15.72:0/3685646904 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c95da7ea40 0x55c95da8c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:36.374 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.372+0000 7fcee9de1740 1 --2- 172.21.15.72:0/3685646904 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c95da7e050 0x55c95da8bcf0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:36.375 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.372+0000 7fcee9de1740 1 -- 172.21.15.72:0/3685646904 >> 172.21.15.72:0/3685646904 conn(0x55c95da796c0 msgr2=0x55c95da7c3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:56:36.375 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.372+0000 7fceca000640 0 client.15735 ms_handle_reset on v2:172.21.15.76:6832/3869483750 2024-04-26T21:56:36.375 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.372+0000 7fcee9de1740 1 -- 172.21.15.72:0/3685646904 shutdown_connections 2024-04-26T21:56:36.375 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.372+0000 7fcee9de1740 1 -- 172.21.15.72:0/3685646904 wait complete. 2024-04-26T21:56:36.375 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.372+0000 7fcee9de1740 20 client.15735 trim_cache size 0 max 16384 2024-04-26T21:56:36.375 INFO:teuthology.orchestra.run.smithi072.stdout:[] 2024-04-26T21:56:36.376 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.373+0000 7fcee5a00640 1 -- 172.21.15.72:0/474356277 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fcec40a3cc0 msgr2=0x7fcec40a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:36.376 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.373+0000 7fcee5a00640 1 --2- 172.21.15.72:0/474356277 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fcec40a3cc0 0x7fcec40a6180 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x55c95da7f2a0 tx=0x7fced8004a40 comp rx=0 tx=0).stop 2024-04-26T21:56:36.376 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.373+0000 7fcee5a00640 1 -- 172.21.15.72:0/474356277 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcee0101a70 msgr2=0x7fcee019b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:36.376 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.373+0000 7fcee5a00640 1 --2- 172.21.15.72:0/474356277 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcee0101a70 0x7fcee019b160 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fcecc009230 tx=0x7fcecc0042f0 comp rx=0 tx=0).stop 2024-04-26T21:56:36.376 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.373+0000 7fcee5a00640 1 -- 172.21.15.72:0/474356277 shutdown_connections 2024-04-26T21:56:36.376 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.373+0000 7fcee5a00640 1 --2- 172.21.15.72:0/474356277 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fcec40a3cc0 0x7fcec40a6180 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:36.376 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.373+0000 7fcee5a00640 1 --2- 172.21.15.72:0/474356277 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcee0102410 0x7fcee01a2180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:36.377 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.373+0000 7fcee5a00640 1 --2- 172.21.15.72:0/474356277 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcee0101a70 0x7fcee019b160 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:36.377 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.373+0000 7fcee5a00640 1 --2- 172.21.15.72:0/474356277 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcee0101110 0x7fcee019ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:36.377 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.373+0000 7fcee5a00640 1 -- 172.21.15.72:0/474356277 >> 172.21.15.72:0/474356277 conn(0x7fcee00fccd0 msgr2=0x7fcee00fe4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:56:36.377 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.374+0000 7fcee5a00640 1 -- 172.21.15.72:0/474356277 shutdown_connections 2024-04-26T21:56:36.377 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.374+0000 7fcee5a00640 1 -- 172.21.15.72:0/474356277 wait complete. 2024-04-26T21:56:36.397 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-26T21:56:36.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:56:35 smithi164 ceph-mon[31916]: from='client.? 172.21.15.72:0/12066193' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:56:36.567 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.567+0000 7eff72000640 1 -- 172.21.15.72:0/3956621691 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7eff6c101a70 msgr2=0x7eff6c101ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:36.568 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.567+0000 7eff72000640 1 --2- 172.21.15.72:0/3956621691 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7eff6c101a70 0x7eff6c101ed0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7eff58009900 tx=0x7eff5802a3f0 comp rx=0 tx=0).stop 2024-04-26T21:56:36.568 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.567+0000 7eff72000640 1 -- 172.21.15.72:0/3956621691 shutdown_connections 2024-04-26T21:56:36.568 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.567+0000 7eff72000640 1 --2- 172.21.15.72:0/3956621691 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7eff6c102410 0x7eff6c102890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:36.568 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.567+0000 7eff72000640 1 --2- 172.21.15.72:0/3956621691 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7eff6c101a70 0x7eff6c101ed0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:36.568 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.567+0000 7eff72000640 1 --2- 172.21.15.72:0/3956621691 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7eff6c101110 0x7eff6c101530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:36.568 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.567+0000 7eff72000640 1 -- 172.21.15.72:0/3956621691 >> 172.21.15.72:0/3956621691 conn(0x7eff6c0fccd0 msgr2=0x7eff6c0ff0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:56:36.568 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.567+0000 7eff72000640 1 -- 172.21.15.72:0/3956621691 shutdown_connections 2024-04-26T21:56:36.568 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.567+0000 7eff72000640 1 -- 172.21.15.72:0/3956621691 wait complete. 2024-04-26T21:56:36.569 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.569+0000 7eff72000640 1 Processor -- start 2024-04-26T21:56:36.570 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.569+0000 7eff72000640 1 -- start start 2024-04-26T21:56:36.570 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.569+0000 7eff72000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7eff6c101110 0x7eff6c0713d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:36.570 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.569+0000 7eff72000640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7eff6c101a70 0x7eff6c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:36.570 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.569+0000 7eff72000640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7eff6c102410 0x7eff6c1aabe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:36.570 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.570+0000 7eff72000640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7eff6c1104d0 con 0x7eff6c101a70 2024-04-26T21:56:36.570 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.570+0000 7eff72000640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7eff6c110350 con 0x7eff6c102410 2024-04-26T21:56:36.570 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.570+0000 7eff72000640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7eff6c110650 con 0x7eff6c101110 2024-04-26T21:56:36.570 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.570+0000 7eff6be00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7eff6c101a70 0x7eff6c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:36.570 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.570+0000 7eff6be00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7eff6c101a70 0x7eff6c071910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:58644/0 (socket says 172.21.15.72:58644) 2024-04-26T21:56:36.571 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.570+0000 7eff6be00640 1 -- 172.21.15.72:0/1164423712 learned_addr learned my addr 172.21.15.72:0/1164423712 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:56:36.571 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.570+0000 7eff70c00640 1 --2- 172.21.15.72:0/1164423712 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7eff6c101110 0x7eff6c0713d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:36.571 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.570+0000 7eff71600640 1 --2- 172.21.15.72:0/1164423712 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7eff6c102410 0x7eff6c1aabe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:36.572 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.570+0000 7eff6be00640 1 -- 172.21.15.72:0/1164423712 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7eff6c101110 msgr2=0x7eff6c0713d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:36.572 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.570+0000 7eff6be00640 1 --2- 172.21.15.72:0/1164423712 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7eff6c101110 0x7eff6c0713d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:36.572 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.570+0000 7eff6be00640 1 -- 172.21.15.72:0/1164423712 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7eff6c102410 msgr2=0x7eff6c1aabe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:36.572 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.570+0000 7eff6be00640 1 --2- 172.21.15.72:0/1164423712 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7eff6c102410 0x7eff6c1aabe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:36.572 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.571+0000 7eff6be00640 1 -- 172.21.15.72:0/1164423712 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff6c1ab120 con 0x7eff6c101a70 2024-04-26T21:56:36.572 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.571+0000 7eff6be00640 1 --2- 172.21.15.72:0/1164423712 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7eff6c101a70 0x7eff6c071910 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7eff58031460 tx=0x7eff58003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:56:36.572 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.571+0000 7eff69600640 1 -- 172.21.15.72:0/1164423712 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff5803f070 con 0x7eff6c101a70 2024-04-26T21:56:36.572 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.571+0000 7eff69600640 1 -- 172.21.15.72:0/1164423712 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7eff58031da0 con 0x7eff6c101a70 2024-04-26T21:56:36.572 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.571+0000 7eff69600640 1 -- 172.21.15.72:0/1164423712 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff58035030 con 0x7eff6c101a70 2024-04-26T21:56:36.572 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.572+0000 7eff72000640 1 -- 172.21.15.72:0/1164423712 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff6c1ab2f0 con 0x7eff6c101a70 2024-04-26T21:56:36.572 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.572+0000 7eff72000640 1 -- 172.21.15.72:0/1164423712 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff6c1ab7e0 con 0x7eff6c101a70 2024-04-26T21:56:36.575 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.576+0000 7eff69600640 1 -- 172.21.15.72:0/1164423712 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7eff58042070 con 0x7eff6c101a70 2024-04-26T21:56:36.579 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.577+0000 7eff76392740 1 Processor -- start 2024-04-26T21:56:36.580 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.577+0000 7eff76392740 1 -- start start 2024-04-26T21:56:36.580 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.577+0000 7eff76392740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55cb53b95050 0x55cb53b95470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:36.580 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.577+0000 7eff76392740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cb53b95a40 0x55cb53b9a540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:36.580 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.577+0000 7eff76392740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cb53b9abb0 0x55cb53b9d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:36.580 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.577+0000 7eff76392740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55cb53af8c70 con 0x55cb53b95050 2024-04-26T21:56:36.581 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.577+0000 7eff76392740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55cb53ab1980 con 0x55cb53b95a40 2024-04-26T21:56:36.581 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.577+0000 7eff76392740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55cb53af4c30 con 0x55cb53b9abb0 2024-04-26T21:56:36.581 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.578+0000 7eff71600640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55cb53b95050 0x55cb53b95470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:36.581 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.578+0000 7eff71600640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55cb53b95050 0x55cb53b95470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:58652/0 (socket says 172.21.15.72:58652) 2024-04-26T21:56:36.581 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.578+0000 7eff71600640 1 -- 172.21.15.72:0/2697737000 learned_addr learned my addr 172.21.15.72:0/2697737000 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:56:36.582 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.578+0000 7eff70c00640 1 --2- 172.21.15.72:0/2697737000 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cb53b95a40 0x55cb53b9a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:36.582 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.578+0000 7eff69600640 1 --2- 172.21.15.72:0/1164423712 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7eff540a3cc0 0x7eff540a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:36.582 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.578+0000 7eff6be00640 1 --2- 172.21.15.72:0/2697737000 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cb53b9abb0 0x55cb53b9d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:36.582 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.578+0000 7eff71600640 1 -- 172.21.15.72:0/2697737000 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cb53b9abb0 msgr2=0x55cb53b9d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:56:36.582 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.578+0000 7eff71600640 1 --2- 172.21.15.72:0/2697737000 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cb53b9abb0 0x55cb53b9d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:36.582 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.578+0000 7eff71600640 1 -- 172.21.15.72:0/2697737000 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cb53b95a40 msgr2=0x55cb53b9a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:56:36.583 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.578+0000 7eff71600640 1 --2- 172.21.15.72:0/2697737000 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cb53b95a40 0x55cb53b9a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:36.583 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.578+0000 7eff71600640 1 -- 172.21.15.72:0/2697737000 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cb53a897b0 con 0x55cb53b95050 2024-04-26T21:56:36.583 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.578+0000 7eff69600640 1 -- 172.21.15.72:0/1164423712 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7eff580f0ea0 con 0x7eff6c101a70 2024-04-26T21:56:36.583 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.578+0000 7eff71600640 1 --2- 172.21.15.72:0/1164423712 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7eff540a3cc0 0x7eff540a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:36.583 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.579+0000 7eff71600640 1 --2- 172.21.15.72:0/2697737000 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55cb53b95050 0x55cb53b95470 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7eff6400c4a0 tx=0x7eff6400c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30004f5c08b3b921 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:56:36.583 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.579+0000 7eff5dc00640 1 -- 172.21.15.72:0/2697737000 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff64018640 con 0x55cb53b95050 2024-04-26T21:56:36.583 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.579+0000 7eff5dc00640 1 -- 172.21.15.72:0/2697737000 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7eff6401e070 con 0x55cb53b95050 2024-04-26T21:56:36.584 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.579+0000 7eff5dc00640 1 -- 172.21.15.72:0/2697737000 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff6400a800 con 0x55cb53b95050 2024-04-26T21:56:36.584 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.579+0000 7eff71600640 1 --2- 172.21.15.72:0/1164423712 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7eff540a3cc0 0x7eff540a6180 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7eff6401d000 tx=0x7eff64017000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:56:36.591 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.580+0000 7eff76392740 1 -- 172.21.15.72:0/2697737000 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55cb53b95050 msgr2=0x55cb53b95470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:36.591 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.580+0000 7eff76392740 1 --2- 172.21.15.72:0/2697737000 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55cb53b95050 0x55cb53b95470 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7eff6400c4a0 tx=0x7eff6400c960 comp rx=0 tx=0).stop 2024-04-26T21:56:36.591 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.580+0000 7eff76392740 1 -- 172.21.15.72:0/2697737000 shutdown_connections 2024-04-26T21:56:36.591 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.580+0000 7eff76392740 1 --2- 172.21.15.72:0/2697737000 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cb53b9abb0 0x55cb53b9d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:36.591 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.580+0000 7eff76392740 1 --2- 172.21.15.72:0/2697737000 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cb53b95a40 0x55cb53b9a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:36.591 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.580+0000 7eff76392740 1 --2- 172.21.15.72:0/2697737000 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55cb53b95050 0x55cb53b95470 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:36.592 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.580+0000 7eff76392740 1 -- 172.21.15.72:0/2697737000 >> 172.21.15.72:0/2697737000 conn(0x55cb53b906c0 msgr2=0x55cb53b90ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:56:36.592 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.580+0000 7eff76392740 1 -- 172.21.15.72:0/2697737000 shutdown_connections 2024-04-26T21:56:36.592 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.581+0000 7eff76392740 1 -- 172.21.15.72:0/2697737000 wait complete. 2024-04-26T21:56:36.592 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.581+0000 7eff76392740 1 -- start start 2024-04-26T21:56:36.592 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.581+0000 7eff76392740 1 Processor -- start 2024-04-26T21:56:36.592 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.582+0000 7eff76392740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55cb53b95050 0x55cb53b9ef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:36.592 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.582+0000 7eff76392740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cb53b95a40 0x55cb53b9f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:36.593 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.582+0000 7eff76392740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cb53b9abb0 0x55cb53ba64e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:36.593 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.582+0000 7eff76392740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55cb53af7f80 con 0x55cb53b95050 2024-04-26T21:56:36.593 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.582+0000 7eff76392740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55cb53aa5260 con 0x55cb53b95a40 2024-04-26T21:56:36.593 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.582+0000 7eff76392740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55cb53afd2a0 con 0x55cb53b9abb0 2024-04-26T21:56:36.593 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.582+0000 7eff76392740 10 client.0 _ll_register_callbacks cb 0x55cb53aef960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:56:36.593 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.582+0000 7eff70c00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55cb53b95050 0x55cb53b9ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:36.594 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.582+0000 7eff70c00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55cb53b95050 0x55cb53b9ef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:58654/0 (socket says 172.21.15.72:58654) 2024-04-26T21:56:36.594 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.582+0000 7eff70c00640 1 -- 172.21.15.72:0/3899070931 learned_addr learned my addr 172.21.15.72:0/3899070931 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:56:36.594 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.583+0000 7eff71600640 1 --2- 172.21.15.72:0/3899070931 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cb53b9abb0 0x55cb53ba64e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:36.594 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.583+0000 7eff70c00640 1 -- 172.21.15.72:0/3899070931 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cb53b9abb0 msgr2=0x55cb53ba64e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:56:36.594 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.583+0000 7eff70c00640 1 --2- 172.21.15.72:0/3899070931 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cb53b9abb0 0x55cb53ba64e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:36.594 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.583+0000 7eff70c00640 1 -- 172.21.15.72:0/3899070931 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cb53b95a40 msgr2=0x55cb53b9f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:56:36.595 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.583+0000 7eff70c00640 1 --2- 172.21.15.72:0/3899070931 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cb53b95a40 0x55cb53b9f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:36.595 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.583+0000 7eff70c00640 1 -- 172.21.15.72:0/3899070931 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cb53a871c0 con 0x55cb53b95050 2024-04-26T21:56:36.595 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.583+0000 7eff70c00640 1 --2- 172.21.15.72:0/3899070931 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55cb53b95050 0x55cb53b9ef60 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7eff6000b990 tx=0x7eff6000be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dadeb85d3cc694d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:56:36.595 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.583+0000 7eff52000640 10 client.0 ms_handle_connect on v2:172.21.15.72:3300/0 2024-04-26T21:56:36.595 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.583+0000 7eff52000640 1 -- 172.21.15.72:0/3899070931 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff60003e60 con 0x55cb53b95050 2024-04-26T21:56:36.595 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.583+0000 7eff52000640 1 -- 172.21.15.72:0/3899070931 <== mon.0 v2:172.21.15.72:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7eff60005020 con 0x55cb53b95050 2024-04-26T21:56:36.596 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.584+0000 7eff76392740 1 -- 172.21.15.72:0/3899070931 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7eff6401e070 con 0x55cb53b95050 2024-04-26T21:56:36.596 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.584+0000 7eff52000640 1 -- 172.21.15.72:0/3899070931 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7eff60005220 con 0x55cb53b95050 2024-04-26T21:56:36.596 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.585+0000 7eff76392740 10 client.15747 fetch_fsmap learned FSMap version 27 2024-04-26T21:56:36.596 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.585+0000 7eff76392740 1 -- 172.21.15.72:0/3899070931 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55cb5380a3f0 con 0x55cb53b95050 2024-04-26T21:56:36.596 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.585+0000 7eff52000640 1 -- 172.21.15.72:0/3899070931 <== mon.0 v2:172.21.15.72:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7eff6401e070 con 0x55cb53b95050 2024-04-26T21:56:36.596 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.585+0000 7eff52000640 1 -- 172.21.15.72:0/3899070931 <== mon.0 v2:172.21.15.72:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7eff6001b470 con 0x55cb53b95050 2024-04-26T21:56:36.596 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.585+0000 7eff76392740 10 client.15747 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T21:56:36.596 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.585+0000 7eff76392740 10 client.15747 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-26T21:56:36.597 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.585+0000 7eff76392740 20 client.15747 populate_metadata read hostname 'smithi072' 2024-04-26T21:56:36.597 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.586+0000 7eff76392740 1 --2- 172.21.15.72:0/3899070931 >> [v2:172.21.15.76:6834/3517429435,v1:172.21.15.76:6835/3517429435] conn(0x55cb53baab50 0x55cb53bad010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:36.597 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.586+0000 7eff76392740 4 client.15747 mds_command: new command op to 34442 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:56:36.597 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.586+0000 7eff76392740 1 -- 172.21.15.72:0/3899070931 --> [v2:172.21.15.76:6834/3517429435,v1:172.21.15.76:6835/3517429435] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cb53b06880 con 0x55cb53baab50 2024-04-26T21:56:36.597 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.586+0000 7eff6be00640 1 --2- 172.21.15.72:0/3899070931 >> [v2:172.21.15.76:6834/3517429435,v1:172.21.15.76:6835/3517429435] conn(0x55cb53baab50 0x55cb53bad010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:36.597 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.587+0000 7eff6be00640 1 --2- 172.21.15.72:0/3899070931 >> [v2:172.21.15.76:6834/3517429435,v1:172.21.15.76:6835/3517429435] conn(0x55cb53baab50 0x55cb53bad010 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2ce0ced702d4807b server_cookie=fa5156017a8973d5 in_seq=0 out_seq=0 2024-04-26T21:56:36.598 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.587+0000 7eff52000640 10 client.15747 ms_handle_connect on v2:172.21.15.76:6834/3517429435 2024-04-26T21:56:36.598 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.589+0000 7eff52000640 1 -- 172.21.15.72:0/3899070931 <== mds.2 v2:172.21.15.76:6834/3517429435 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55cb53b06880 con 0x55cb53baab50 2024-04-26T21:56:36.598 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.589+0000 7eff52000640 10 client.15747 handle_command_reply: tid=0 2024-04-26T21:56:36.598 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.589+0000 7eff76392740 1 client.15747 shutdown 2024-04-26T21:56:36.598 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.590+0000 7eff76392740 1 -- 172.21.15.72:0/3899070931 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55cb53b95050 msgr2=0x55cb53b9ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:36.598 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.590+0000 7eff76392740 1 --2- 172.21.15.72:0/3899070931 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55cb53b95050 0x55cb53b9ef60 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7eff6000b990 tx=0x7eff6000be50 comp rx=0 tx=0).stop 2024-04-26T21:56:36.598 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.590+0000 7eff76392740 1 -- 172.21.15.72:0/3899070931 shutdown_connections 2024-04-26T21:56:36.599 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.590+0000 7eff76392740 1 --2- 172.21.15.72:0/3899070931 >> [v2:172.21.15.76:6834/3517429435,v1:172.21.15.76:6835/3517429435] conn(0x55cb53baab50 0x55cb53bad010 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:36.599 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.590+0000 7eff76392740 1 --2- 172.21.15.72:0/3899070931 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cb53b9abb0 0x55cb53ba64e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:36.599 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.590+0000 7eff76392740 1 --2- 172.21.15.72:0/3899070931 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cb53b95a40 0x55cb53b9f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:36.599 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.590+0000 7eff76392740 1 --2- 172.21.15.72:0/3899070931 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55cb53b95050 0x55cb53b9ef60 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:36.599 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.590+0000 7eff76392740 1 -- 172.21.15.72:0/3899070931 >> 172.21.15.72:0/3899070931 conn(0x55cb53b906c0 msgr2=0x55cb53b8f7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:56:36.599 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.590+0000 7eff52000640 0 client.15747 ms_handle_reset on v2:172.21.15.76:6834/3517429435 2024-04-26T21:56:36.600 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.590+0000 7eff76392740 1 -- 172.21.15.72:0/3899070931 shutdown_connections 2024-04-26T21:56:36.600 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.590+0000 7eff76392740 1 -- 172.21.15.72:0/3899070931 wait complete. 2024-04-26T21:56:36.600 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.591+0000 7eff76392740 20 client.15747 trim_cache size 0 max 16384 2024-04-26T21:56:36.630 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.618+0000 7eff76392740 1 Processor -- start 2024-04-26T21:56:36.630 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.618+0000 7eff76392740 1 -- start start 2024-04-26T21:56:36.631 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.618+0000 7eff76392740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cb53b95050 0x55cb53b933a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:36.631 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.618+0000 7eff76392740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cb53b95a40 0x55cb53ba64a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:36.631 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.618+0000 7eff76392740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55cb53b9abb0 0x55cb53bb18a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:36.631 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.618+0000 7eff76392740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7eff6c1b47b0 con 0x55cb53b9abb0 2024-04-26T21:56:36.631 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.618+0000 7eff76392740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7eff6c1b62f0 con 0x55cb53b95a40 2024-04-26T21:56:36.631 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.618+0000 7eff76392740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7eff6c1b4e80 con 0x55cb53b95050 2024-04-26T21:56:36.632 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.619+0000 7eff71600640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55cb53b9abb0 0x55cb53bb18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:36.632 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.619+0000 7eff71600640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55cb53b9abb0 0x55cb53bb18a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:58668/0 (socket says 172.21.15.72:58668) 2024-04-26T21:56:36.632 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.619+0000 7eff71600640 1 -- 172.21.15.72:0/3551066115 learned_addr learned my addr 172.21.15.72:0/3551066115 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:56:36.632 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.619+0000 7eff6be00640 1 --2- 172.21.15.72:0/3551066115 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cb53b95a40 0x55cb53ba64a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:36.632 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.619+0000 7eff70c00640 1 --2- 172.21.15.72:0/3551066115 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cb53b95050 0x55cb53b933a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:36.632 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.619+0000 7eff71600640 1 -- 172.21.15.72:0/3551066115 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cb53b95050 msgr2=0x55cb53b933a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:56:36.632 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.619+0000 7eff71600640 1 --2- 172.21.15.72:0/3551066115 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cb53b95050 0x55cb53b933a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:36.633 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.619+0000 7eff71600640 1 -- 172.21.15.72:0/3551066115 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cb53b95a40 msgr2=0x55cb53ba64a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:56:36.633 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.619+0000 7eff71600640 1 --2- 172.21.15.72:0/3551066115 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cb53b95a40 0x55cb53ba64a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:36.633 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.619+0000 7eff71600640 1 -- 172.21.15.72:0/3551066115 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cb538d47d0 con 0x55cb53b9abb0 2024-04-26T21:56:36.633 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.620+0000 7eff71600640 1 --2- 172.21.15.72:0/3551066115 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55cb53b9abb0 0x55cb53bb18a0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x55cb53b962a0 tx=0x7eff64011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5683d610f8a7f341 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:56:36.633 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.620+0000 7eff5d200640 1 -- 172.21.15.72:0/3551066115 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff6400ad20 con 0x55cb53b9abb0 2024-04-26T21:56:36.633 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.620+0000 7eff5d200640 1 -- 172.21.15.72:0/3551066115 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7eff6401e040 con 0x55cb53b9abb0 2024-04-26T21:56:36.634 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.620+0000 7eff5d200640 1 -- 172.21.15.72:0/3551066115 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff640186a0 con 0x55cb53b9abb0 2024-04-26T21:56:36.634 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.621+0000 7eff76392740 1 -- 172.21.15.72:0/3551066115 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55cb53b9abb0 msgr2=0x55cb53bb18a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:36.634 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.621+0000 7eff76392740 1 --2- 172.21.15.72:0/3551066115 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55cb53b9abb0 0x55cb53bb18a0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x55cb53b962a0 tx=0x7eff64011000 comp rx=0 tx=0).stop 2024-04-26T21:56:36.634 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.621+0000 7eff76392740 1 -- 172.21.15.72:0/3551066115 shutdown_connections 2024-04-26T21:56:36.634 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.621+0000 7eff76392740 1 --2- 172.21.15.72:0/3551066115 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55cb53b9abb0 0x55cb53bb18a0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:36.634 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.621+0000 7eff76392740 1 --2- 172.21.15.72:0/3551066115 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cb53b95a40 0x55cb53ba64a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:36.635 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.621+0000 7eff76392740 1 --2- 172.21.15.72:0/3551066115 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cb53b95050 0x55cb53b933a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:36.635 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.621+0000 7eff76392740 1 -- 172.21.15.72:0/3551066115 >> 172.21.15.72:0/3551066115 conn(0x55cb53b906c0 msgr2=0x55cb53b8f7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:56:36.635 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.621+0000 7eff76392740 1 -- 172.21.15.72:0/3551066115 shutdown_connections 2024-04-26T21:56:36.635 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.621+0000 7eff76392740 1 -- 172.21.15.72:0/3551066115 wait complete. 2024-04-26T21:56:36.635 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.622+0000 7eff76392740 1 -- start start 2024-04-26T21:56:36.635 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.622+0000 7eff76392740 1 Processor -- start 2024-04-26T21:56:36.637 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.623+0000 7eff76392740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55cb53b95050 0x55cb53ba3360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:36.637 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.623+0000 7eff76392740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cb53b95a40 0x55cb53ba38a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:36.638 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.623+0000 7eff76392740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cb53b9abb0 0x55cb53c0f110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:36.638 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.623+0000 7eff76392740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7eff6c1b5c20 con 0x55cb53b95050 2024-04-26T21:56:36.638 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.623+0000 7eff76392740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7eff6c1b5550 con 0x55cb53b95a40 2024-04-26T21:56:36.638 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.623+0000 7eff76392740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7eff6c1b2790 con 0x55cb53b9abb0 2024-04-26T21:56:36.638 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.623+0000 7eff76392740 10 client.0 _ll_register_callbacks cb 0x7eff600042b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:56:36.638 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.623+0000 7eff70c00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55cb53b95050 0x55cb53ba3360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:36.639 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.623+0000 7eff70c00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55cb53b95050 0x55cb53ba3360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:58670/0 (socket says 172.21.15.72:58670) 2024-04-26T21:56:36.639 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.623+0000 7eff70c00640 1 -- 172.21.15.72:0/3075708999 learned_addr learned my addr 172.21.15.72:0/3075708999 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:56:36.639 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.623+0000 7eff6be00640 1 --2- 172.21.15.72:0/3075708999 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cb53b95a40 0x55cb53ba38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:36.639 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.623+0000 7eff71600640 1 --2- 172.21.15.72:0/3075708999 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cb53b9abb0 0x55cb53c0f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:36.639 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.623+0000 7eff70c00640 1 -- 172.21.15.72:0/3075708999 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cb53b9abb0 msgr2=0x55cb53c0f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:56:36.639 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.624+0000 7eff70c00640 1 --2- 172.21.15.72:0/3075708999 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cb53b9abb0 0x55cb53c0f110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:36.640 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.624+0000 7eff70c00640 1 -- 172.21.15.72:0/3075708999 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cb53b95a40 msgr2=0x55cb53ba38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:56:36.640 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.624+0000 7eff70c00640 1 --2- 172.21.15.72:0/3075708999 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cb53b95a40 0x55cb53ba38a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:36.640 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.624+0000 7eff70c00640 1 -- 172.21.15.72:0/3075708999 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cb53a9f2a0 con 0x55cb53b95050 2024-04-26T21:56:36.640 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.624+0000 7eff71600640 1 --2- 172.21.15.72:0/3075708999 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cb53b9abb0 0x55cb53c0f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:56:36.640 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.624+0000 7eff70c00640 1 --2- 172.21.15.72:0/3075708999 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55cb53b95050 0x55cb53ba3360 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7eff60001eb0 tx=0x7eff60003fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=379a728c1b4506c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:56:36.640 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.624+0000 7eff52000640 10 client.0 ms_handle_connect on v2:172.21.15.72:3300/0 2024-04-26T21:56:36.641 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.624+0000 7eff52000640 1 -- 172.21.15.72:0/3075708999 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff600053c0 con 0x55cb53b95050 2024-04-26T21:56:36.641 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.624+0000 7eff52000640 1 -- 172.21.15.72:0/3075708999 <== mon.0 v2:172.21.15.72:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7eff60005560 con 0x55cb53b95050 2024-04-26T21:56:36.641 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.624+0000 7eff76392740 1 -- 172.21.15.72:0/3075708999 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7eff6401e040 con 0x55cb53b95050 2024-04-26T21:56:36.641 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.625+0000 7eff52000640 1 -- 172.21.15.72:0/3075708999 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7eff60011440 con 0x55cb53b95050 2024-04-26T21:56:36.641 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.625+0000 7eff76392740 10 client.15753 fetch_fsmap learned FSMap version 27 2024-04-26T21:56:36.641 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.625+0000 7eff76392740 1 -- 172.21.15.72:0/3075708999 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55cb53c10770 con 0x55cb53b95050 2024-04-26T21:56:36.642 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.625+0000 7eff52000640 1 -- 172.21.15.72:0/3075708999 <== mon.0 v2:172.21.15.72:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7eff6401e040 con 0x55cb53b95050 2024-04-26T21:56:36.642 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.625+0000 7eff52000640 1 -- 172.21.15.72:0/3075708999 <== mon.0 v2:172.21.15.72:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7eff6003a630 con 0x55cb53b95050 2024-04-26T21:56:36.642 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.625+0000 7eff76392740 10 client.15753 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T21:56:36.642 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.625+0000 7eff76392740 10 client.15753 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-26T21:56:36.642 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.625+0000 7eff76392740 20 client.15753 populate_metadata read hostname 'smithi072' 2024-04-26T21:56:36.642 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.625+0000 7eff76392740 1 --2- 172.21.15.72:0/3075708999 >> [v2:172.21.15.76:6834/3517429435,v1:172.21.15.76:6835/3517429435] conn(0x55cb53c11150 0x55cb53c13570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:36.642 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.625+0000 7eff76392740 4 client.15753 mds_command: new command op to 34442 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-26T21:56:36.643 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.625+0000 7eff76392740 1 -- 172.21.15.72:0/3075708999 --> [v2:172.21.15.76:6834/3517429435,v1:172.21.15.76:6835/3517429435] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55cb53b006f0 con 0x55cb53c11150 2024-04-26T21:56:36.643 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.626+0000 7eff6be00640 1 --2- 172.21.15.72:0/3075708999 >> [v2:172.21.15.76:6834/3517429435,v1:172.21.15.76:6835/3517429435] conn(0x55cb53c11150 0x55cb53c13570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:36.643 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.627+0000 7eff52000640 10 client.15753 ms_handle_connect on v2:172.21.15.76:6834/3517429435 2024-04-26T21:56:36.643 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.627+0000 7eff6be00640 1 --2- 172.21.15.72:0/3075708999 >> [v2:172.21.15.76:6834/3517429435,v1:172.21.15.76:6835/3517429435] conn(0x55cb53c11150 0x55cb53c13570 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=42a91d3bced930ac server_cookie=70e3d2c136b56ad4 in_seq=0 out_seq=0 2024-04-26T21:56:36.643 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.627+0000 7eff52000640 1 -- 172.21.15.72:0/3075708999 <== mds.2 v2:172.21.15.76:6834/3517429435 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55cb53b006f0 con 0x55cb53c11150 2024-04-26T21:56:36.643 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.627+0000 7eff52000640 10 client.15753 handle_command_reply: tid=0 2024-04-26T21:56:36.644 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.627+0000 7eff76392740 1 client.15753 shutdown 2024-04-26T21:56:36.644 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.627+0000 7eff76392740 1 -- 172.21.15.72:0/3075708999 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55cb53b95050 msgr2=0x55cb53ba3360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:36.644 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.627+0000 7eff76392740 1 --2- 172.21.15.72:0/3075708999 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55cb53b95050 0x55cb53ba3360 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7eff60001eb0 tx=0x7eff60003fa0 comp rx=0 tx=0).stop 2024-04-26T21:56:36.644 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.628+0000 7eff76392740 1 -- 172.21.15.72:0/3075708999 shutdown_connections 2024-04-26T21:56:36.644 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.628+0000 7eff76392740 1 --2- 172.21.15.72:0/3075708999 >> [v2:172.21.15.76:6834/3517429435,v1:172.21.15.76:6835/3517429435] conn(0x55cb53c11150 0x55cb53c13570 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:36.644 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.628+0000 7eff76392740 1 --2- 172.21.15.72:0/3075708999 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cb53b9abb0 0x55cb53c0f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:36.644 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.628+0000 7eff76392740 1 --2- 172.21.15.72:0/3075708999 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cb53b95a40 0x55cb53ba38a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:36.645 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.628+0000 7eff76392740 1 --2- 172.21.15.72:0/3075708999 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55cb53b95050 0x55cb53ba3360 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:36.645 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.628+0000 7eff76392740 1 -- 172.21.15.72:0/3075708999 >> 172.21.15.72:0/3075708999 conn(0x55cb53b906c0 msgr2=0x55cb53b96470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:56:36.645 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.628+0000 7eff52000640 0 client.15753 ms_handle_reset on v2:172.21.15.76:6834/3517429435 2024-04-26T21:56:36.645 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.629+0000 7eff76392740 1 -- 172.21.15.72:0/3075708999 shutdown_connections 2024-04-26T21:56:36.645 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.629+0000 7eff76392740 1 -- 172.21.15.72:0/3075708999 wait complete. 2024-04-26T21:56:36.645 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.629+0000 7eff76392740 20 client.15753 trim_cache size 0 max 16384 2024-04-26T21:56:36.646 INFO:teuthology.orchestra.run.smithi072.stdout:[] 2024-04-26T21:56:36.647 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.629+0000 7eff72000640 1 -- 172.21.15.72:0/1164423712 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7eff540a3cc0 msgr2=0x7eff540a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:36.647 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.630+0000 7eff72000640 1 --2- 172.21.15.72:0/1164423712 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7eff540a3cc0 0x7eff540a6180 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7eff6401d000 tx=0x7eff64017000 comp rx=0 tx=0).stop 2024-04-26T21:56:36.648 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.630+0000 7eff72000640 1 -- 172.21.15.72:0/1164423712 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7eff6c101a70 msgr2=0x7eff6c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:36.648 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.630+0000 7eff72000640 1 --2- 172.21.15.72:0/1164423712 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7eff6c101a70 0x7eff6c071910 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7eff58031460 tx=0x7eff58003c80 comp rx=0 tx=0).stop 2024-04-26T21:56:36.648 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.630+0000 7eff72000640 1 -- 172.21.15.72:0/1164423712 shutdown_connections 2024-04-26T21:56:36.648 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.630+0000 7eff72000640 1 --2- 172.21.15.72:0/1164423712 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7eff6c102410 0x7eff6c1aabe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:36.648 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.630+0000 7eff72000640 1 --2- 172.21.15.72:0/1164423712 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7eff6c101a70 0x7eff6c071910 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:36.648 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.630+0000 7eff72000640 1 --2- 172.21.15.72:0/1164423712 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7eff540a3cc0 0x7eff540a6180 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:36.648 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.630+0000 7eff72000640 1 --2- 172.21.15.72:0/1164423712 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7eff6c101110 0x7eff6c0713d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:36.649 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.630+0000 7eff72000640 1 -- 172.21.15.72:0/1164423712 >> 172.21.15.72:0/1164423712 conn(0x7eff6c0fccd0 msgr2=0x7eff6c0fe490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:56:36.649 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.630+0000 7eff72000640 1 -- 172.21.15.72:0/1164423712 shutdown_connections 2024-04-26T21:56:36.649 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.630+0000 7eff72000640 1 -- 172.21.15.72:0/1164423712 wait complete. 2024-04-26T21:56:36.653 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-26T21:56:36.823 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.823+0000 7f3642200640 1 -- 172.21.15.72:0/3712373906 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f363c101130 msgr2=0x7f363c101550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:36.823 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.823+0000 7f3642200640 1 --2- 172.21.15.72:0/3712373906 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f363c101130 0x7f363c101550 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f3630009900 tx=0x7f363002a480 comp rx=0 tx=0).stop 2024-04-26T21:56:36.823 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.823+0000 7f3642200640 1 -- 172.21.15.72:0/3712373906 shutdown_connections 2024-04-26T21:56:36.824 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.823+0000 7f3642200640 1 --2- 172.21.15.72:0/3712373906 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f363c102430 0x7f363c1028b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:36.824 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.823+0000 7f3642200640 1 --2- 172.21.15.72:0/3712373906 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f363c101a90 0x7f363c101ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:36.824 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.823+0000 7f3642200640 1 --2- 172.21.15.72:0/3712373906 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f363c101130 0x7f363c101550 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:36.824 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.823+0000 7f3642200640 1 -- 172.21.15.72:0/3712373906 >> 172.21.15.72:0/3712373906 conn(0x7f363c0fccd0 msgr2=0x7f363c0ff0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:56:36.824 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.823+0000 7f3642200640 1 -- 172.21.15.72:0/3712373906 shutdown_connections 2024-04-26T21:56:36.825 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.823+0000 7f3642200640 1 -- 172.21.15.72:0/3712373906 wait complete. 2024-04-26T21:56:36.825 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.824+0000 7f3642200640 1 Processor -- start 2024-04-26T21:56:36.825 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.824+0000 7f3642200640 1 -- start start 2024-04-26T21:56:36.825 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.825+0000 7f3642200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f363c101130 0x7f363c0713b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:36.825 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.825+0000 7f3642200640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f363c101a90 0x7f363c0718f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:36.825 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.825+0000 7f3642200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f363c102430 0x7f363c1aab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:36.825 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.825+0000 7f3642200640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f363c110440 con 0x7f363c101a90 2024-04-26T21:56:36.825 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.825+0000 7f3642200640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f363c1102c0 con 0x7f363c101130 2024-04-26T21:56:36.825 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.825+0000 7f3642200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f363c1105c0 con 0x7f363c102430 2024-04-26T21:56:36.827 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.825+0000 7f363be00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f363c101a90 0x7f363c0718f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:36.827 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.825+0000 7f363be00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f363c101a90 0x7f363c0718f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:58680/0 (socket says 172.21.15.72:58680) 2024-04-26T21:56:36.827 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.825+0000 7f363be00640 1 -- 172.21.15.72:0/2758013536 learned_addr learned my addr 172.21.15.72:0/2758013536 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:56:36.827 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.825+0000 7f3640e00640 1 --2- 172.21.15.72:0/2758013536 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f363c101130 0x7f363c0713b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:36.827 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.826+0000 7f3641800640 1 --2- 172.21.15.72:0/2758013536 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f363c102430 0x7f363c1aab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:36.827 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.826+0000 7f363be00640 1 -- 172.21.15.72:0/2758013536 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f363c102430 msgr2=0x7f363c1aab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:36.827 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.826+0000 7f363be00640 1 --2- 172.21.15.72:0/2758013536 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f363c102430 0x7f363c1aab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:36.827 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.826+0000 7f363be00640 1 -- 172.21.15.72:0/2758013536 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f363c101130 msgr2=0x7f363c0713b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:36.827 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.826+0000 7f363be00640 1 --2- 172.21.15.72:0/2758013536 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f363c101130 0x7f363c0713b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:36.828 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.826+0000 7f363be00640 1 -- 172.21.15.72:0/2758013536 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f363c1ab090 con 0x7f363c101a90 2024-04-26T21:56:36.828 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.826+0000 7f3640e00640 1 --2- 172.21.15.72:0/2758013536 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f363c101130 0x7f363c0713b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:56:36.828 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.826+0000 7f363be00640 1 --2- 172.21.15.72:0/2758013536 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f363c101a90 0x7f363c0718f0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f362800d950 tx=0x7f362800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:56:36.828 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.827+0000 7f3639600640 1 -- 172.21.15.72:0/2758013536 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3628017070 con 0x7f363c101a90 2024-04-26T21:56:36.828 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.827+0000 7f3642200640 1 -- 172.21.15.72:0/2758013536 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f363c1ab2c0 con 0x7f363c101a90 2024-04-26T21:56:36.828 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.827+0000 7f3639600640 1 -- 172.21.15.72:0/2758013536 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3628009c30 con 0x7f363c101a90 2024-04-26T21:56:36.828 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.827+0000 7f3639600640 1 -- 172.21.15.72:0/2758013536 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f362800f620 con 0x7f363c101a90 2024-04-26T21:56:36.828 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.827+0000 7f3642200640 1 -- 172.21.15.72:0/2758013536 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f363c1ab860 con 0x7f363c101a90 2024-04-26T21:56:36.831 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.831+0000 7f364663f740 1 Processor -- start 2024-04-26T21:56:36.831 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.832+0000 7f364663f740 1 -- start start 2024-04-26T21:56:36.832 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.832+0000 7f364663f740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559b574c5050 0x559b574c5470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:36.832 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.832+0000 7f364663f740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559b574c5a40 0x559b574ca540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:36.832 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.832+0000 7f364663f740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559b574cabb0 0x559b574cd040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:36.833 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.832+0000 7f364663f740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x559b57428c70 con 0x559b574c5a40 2024-04-26T21:56:36.833 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.832+0000 7f364663f740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x559b573e1980 con 0x559b574cabb0 2024-04-26T21:56:36.833 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.832+0000 7f364663f740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x559b57424c30 con 0x559b574c5050 2024-04-26T21:56:36.833 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.832+0000 7f3639600640 1 -- 172.21.15.72:0/2758013536 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f3628010460 con 0x7f363c101a90 2024-04-26T21:56:36.833 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.832+0000 7f3640e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559b574c5a40 0x559b574ca540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:36.833 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.832+0000 7f3640e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559b574c5a40 0x559b574ca540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:58686/0 (socket says 172.21.15.72:58686) 2024-04-26T21:56:36.833 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.833+0000 7f3640e00640 1 -- 172.21.15.72:0/1567245341 learned_addr learned my addr 172.21.15.72:0/1567245341 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:56:36.833 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.833+0000 7f3641800640 1 --2- 172.21.15.72:0/1567245341 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559b574c5050 0x559b574c5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:36.833 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.833+0000 7f363be00640 1 --2- 172.21.15.72:0/1567245341 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559b574cabb0 0x559b574cd040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:36.834 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.833+0000 7f3640e00640 1 -- 172.21.15.72:0/1567245341 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559b574c5050 msgr2=0x559b574c5470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:56:36.834 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.833+0000 7f3640e00640 1 --2- 172.21.15.72:0/1567245341 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559b574c5050 0x559b574c5470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:36.834 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.833+0000 7f3640e00640 1 -- 172.21.15.72:0/1567245341 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559b574cabb0 msgr2=0x559b574cd040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:56:36.834 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.833+0000 7f3640e00640 1 --2- 172.21.15.72:0/1567245341 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559b574cabb0 0x559b574cd040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:36.834 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.833+0000 7f3640e00640 1 -- 172.21.15.72:0/1567245341 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559b573b97b0 con 0x559b574c5a40 2024-04-26T21:56:36.834 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.834+0000 7f3640e00640 1 --2- 172.21.15.72:0/1567245341 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559b574c5a40 0x559b574ca540 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f363002aa70 tx=0x7f36300041f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ff0a0bdddd01356 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:56:36.835 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.834+0000 7f362e000640 1 -- 172.21.15.72:0/1567245341 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3630035030 con 0x559b574c5a40 2024-04-26T21:56:36.835 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.834+0000 7f362e000640 1 -- 172.21.15.72:0/1567245341 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3630003a80 con 0x559b574c5a40 2024-04-26T21:56:36.835 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.834+0000 7f362e000640 1 -- 172.21.15.72:0/1567245341 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3630031ad0 con 0x559b574c5a40 2024-04-26T21:56:36.835 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.835+0000 7f3639600640 1 --2- 172.21.15.72:0/2758013536 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f36240a3d40 0x7f36240a6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:36.835 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.835+0000 7f3639600640 1 -- 172.21.15.72:0/2758013536 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7f36280d0240 con 0x7f363c101a90 2024-04-26T21:56:36.835 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.835+0000 7f364663f740 1 -- 172.21.15.72:0/1567245341 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559b574c5a40 msgr2=0x559b574ca540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:36.835 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.835+0000 7f364663f740 1 --2- 172.21.15.72:0/1567245341 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559b574c5a40 0x559b574ca540 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f363002aa70 tx=0x7f36300041f0 comp rx=0 tx=0).stop 2024-04-26T21:56:36.836 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.835+0000 7f3641800640 1 --2- 172.21.15.72:0/2758013536 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f36240a3d40 0x7f36240a6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:36.837 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.836+0000 7f3641800640 1 --2- 172.21.15.72:0/2758013536 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f36240a3d40 0x7f36240a6200 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x559b574c62a0 tx=0x7f3634009600 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:56:36.837 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.836+0000 7f364663f740 1 -- 172.21.15.72:0/1567245341 shutdown_connections 2024-04-26T21:56:36.837 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.836+0000 7f364663f740 1 --2- 172.21.15.72:0/1567245341 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559b574cabb0 0x559b574cd040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:36.837 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.836+0000 7f364663f740 1 --2- 172.21.15.72:0/1567245341 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559b574c5a40 0x559b574ca540 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:36.837 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.837+0000 7f364663f740 1 --2- 172.21.15.72:0/1567245341 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559b574c5050 0x559b574c5470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:36.837 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.837+0000 7f364663f740 1 -- 172.21.15.72:0/1567245341 >> 172.21.15.72:0/1567245341 conn(0x559b574c06c0 msgr2=0x559b574c0ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:56:36.837 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.837+0000 7f364663f740 1 -- 172.21.15.72:0/1567245341 shutdown_connections 2024-04-26T21:56:36.837 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.837+0000 7f364663f740 1 -- 172.21.15.72:0/1567245341 wait complete. 2024-04-26T21:56:36.838 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.837+0000 7f364663f740 1 -- start start 2024-04-26T21:56:36.838 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.838+0000 7f364663f740 1 Processor -- start 2024-04-26T21:56:36.839 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.838+0000 7f364663f740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559b574c5050 0x559b574cef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:36.839 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.838+0000 7f364663f740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559b574c5a40 0x559b574cf4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:36.839 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.838+0000 7f364663f740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559b574cabb0 0x559b574d6510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:36.839 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.838+0000 7f364663f740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x559b57427f80 con 0x559b574c5a40 2024-04-26T21:56:36.839 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.838+0000 7f364663f740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x559b573d5260 con 0x559b574c5050 2024-04-26T21:56:36.839 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.838+0000 7f364663f740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x559b5742d2a0 con 0x559b574cabb0 2024-04-26T21:56:36.839 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.838+0000 7f364663f740 10 client.0 _ll_register_callbacks cb 0x559b5741f960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:56:36.839 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.839+0000 7f363be00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559b574c5a40 0x559b574cf4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:36.839 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.839+0000 7f363be00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559b574c5a40 0x559b574cf4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:58700/0 (socket says 172.21.15.72:58700) 2024-04-26T21:56:36.839 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.839+0000 7f363be00640 1 -- 172.21.15.72:0/668592809 learned_addr learned my addr 172.21.15.72:0/668592809 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:56:36.841 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.839+0000 7f3640e00640 1 --2- 172.21.15.72:0/668592809 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559b574c5050 0x559b574cef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:36.841 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.839+0000 7f363be00640 1 -- 172.21.15.72:0/668592809 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559b574cabb0 msgr2=0x559b574d6510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:56:36.841 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.839+0000 7f363be00640 1 --2- 172.21.15.72:0/668592809 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559b574cabb0 0x559b574d6510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:36.841 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.839+0000 7f363be00640 1 -- 172.21.15.72:0/668592809 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559b574c5050 msgr2=0x559b574cef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:56:36.841 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.839+0000 7f363be00640 1 --2- 172.21.15.72:0/668592809 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559b574c5050 0x559b574cef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:36.841 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.839+0000 7f363be00640 1 -- 172.21.15.72:0/668592809 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559b573b71c0 con 0x559b574c5a40 2024-04-26T21:56:36.841 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.839+0000 7f363be00640 1 --2- 172.21.15.72:0/668592809 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559b574c5a40 0x559b574cf4d0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f3628098740 tx=0x7f3628010f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82327c6d7846513f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:56:36.841 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.839+0000 7f3622a00640 10 client.0 ms_handle_connect on v2:172.21.15.72:3300/0 2024-04-26T21:56:36.842 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.840+0000 7f3622a00640 1 -- 172.21.15.72:0/668592809 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3628020040 con 0x559b574c5a40 2024-04-26T21:56:36.842 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.840+0000 7f3622a00640 1 -- 172.21.15.72:0/668592809 <== mon.0 v2:172.21.15.72:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7f362809b730 con 0x559b574c5a40 2024-04-26T21:56:36.842 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.840+0000 7f364663f740 1 -- 172.21.15.72:0/668592809 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3630003a80 con 0x559b574c5a40 2024-04-26T21:56:36.842 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.840+0000 7f3622a00640 1 -- 172.21.15.72:0/668592809 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f362809c9e0 con 0x559b574c5a40 2024-04-26T21:56:36.842 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.840+0000 7f364663f740 10 client.15765 fetch_fsmap learned FSMap version 27 2024-04-26T21:56:36.842 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.840+0000 7f364663f740 1 -- 172.21.15.72:0/668592809 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559b5713a3f0 con 0x559b574c5a40 2024-04-26T21:56:36.842 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.841+0000 7f3622a00640 1 -- 172.21.15.72:0/668592809 <== mon.0 v2:172.21.15.72:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3630003a80 con 0x559b574c5a40 2024-04-26T21:56:36.842 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.841+0000 7f3622a00640 1 -- 172.21.15.72:0/668592809 <== mon.0 v2:172.21.15.72:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f3628010750 con 0x559b574c5a40 2024-04-26T21:56:36.842 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.841+0000 7f364663f740 10 client.15765 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T21:56:36.842 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.841+0000 7f364663f740 10 client.15765 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-26T21:56:36.842 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.841+0000 7f364663f740 20 client.15765 populate_metadata read hostname 'smithi072' 2024-04-26T21:56:36.842 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.841+0000 7f364663f740 1 --2- 172.21.15.72:0/668592809 >> [v2:172.21.15.76:6836/1150678991,v1:172.21.15.76:6837/1150678991] conn(0x559b574daa20 0x559b574dcec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:36.842 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.841+0000 7f364663f740 4 client.15765 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:56:36.842 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.841+0000 7f364663f740 1 -- 172.21.15.72:0/668592809 --> [v2:172.21.15.76:6836/1150678991,v1:172.21.15.76:6837/1150678991] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559b57436880 con 0x559b574daa20 2024-04-26T21:56:36.843 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.842+0000 7f3640e00640 1 --2- 172.21.15.72:0/668592809 >> [v2:172.21.15.76:6836/1150678991,v1:172.21.15.76:6837/1150678991] conn(0x559b574daa20 0x559b574dcec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:36.843 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.843+0000 7f3640e00640 1 --2- 172.21.15.72:0/668592809 >> [v2:172.21.15.76:6836/1150678991,v1:172.21.15.76:6837/1150678991] conn(0x559b574daa20 0x559b574dcec0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=af480d2b8993d20f server_cookie=d10a5d20baa9f3ce in_seq=0 out_seq=0 2024-04-26T21:56:36.843 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.843+0000 7f3622a00640 10 client.15765 ms_handle_connect on v2:172.21.15.76:6836/1150678991 2024-04-26T21:56:36.844 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.845+0000 7f3622a00640 1 -- 172.21.15.72:0/668592809 <== mds.1 v2:172.21.15.76:6836/1150678991 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559b57436880 con 0x559b574daa20 2024-04-26T21:56:36.845 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.845+0000 7f3622a00640 10 client.15765 handle_command_reply: tid=0 2024-04-26T21:56:36.845 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.845+0000 7f364663f740 1 client.15765 shutdown 2024-04-26T21:56:36.845 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.845+0000 7f364663f740 1 -- 172.21.15.72:0/668592809 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559b574c5a40 msgr2=0x559b574cf4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:36.845 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.845+0000 7f364663f740 1 --2- 172.21.15.72:0/668592809 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559b574c5a40 0x559b574cf4d0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f3628098740 tx=0x7f3628010f90 comp rx=0 tx=0).stop 2024-04-26T21:56:36.846 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.845+0000 7f364663f740 1 -- 172.21.15.72:0/668592809 shutdown_connections 2024-04-26T21:56:36.846 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.845+0000 7f364663f740 1 --2- 172.21.15.72:0/668592809 >> [v2:172.21.15.76:6836/1150678991,v1:172.21.15.76:6837/1150678991] conn(0x559b574daa20 0x559b574dcec0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:36.846 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.845+0000 7f364663f740 1 --2- 172.21.15.72:0/668592809 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559b574cabb0 0x559b574d6510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:36.846 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.845+0000 7f364663f740 1 --2- 172.21.15.72:0/668592809 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559b574c5a40 0x559b574cf4d0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:36.846 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.845+0000 7f3622a00640 0 client.15765 ms_handle_reset on v2:172.21.15.76:6836/1150678991 2024-04-26T21:56:36.846 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.845+0000 7f364663f740 1 --2- 172.21.15.72:0/668592809 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559b574c5050 0x559b574cef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:36.846 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.845+0000 7f364663f740 1 -- 172.21.15.72:0/668592809 >> 172.21.15.72:0/668592809 conn(0x559b574c06c0 msgr2=0x559b574bf7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:56:36.846 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.845+0000 7f364663f740 1 -- 172.21.15.72:0/668592809 shutdown_connections 2024-04-26T21:56:36.846 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.846+0000 7f364663f740 1 -- 172.21.15.72:0/668592809 wait complete. 2024-04-26T21:56:36.846 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.846+0000 7f364663f740 20 client.15765 trim_cache size 0 max 16384 2024-04-26T21:56:36.873 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.873+0000 7f364663f740 1 Processor -- start 2024-04-26T21:56:36.874 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.873+0000 7f364663f740 1 -- start start 2024-04-26T21:56:36.874 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.874+0000 7f364663f740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559b574c5050 0x559b574ca2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:36.874 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.874+0000 7f364663f740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559b574c5a40 0x559b574d64d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:36.875 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.874+0000 7f364663f740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559b574cabb0 0x559b574c64e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:36.875 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.874+0000 7f364663f740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f363c1b4740 con 0x559b574c5a40 2024-04-26T21:56:36.875 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.874+0000 7f364663f740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f363c1b6280 con 0x559b574cabb0 2024-04-26T21:56:36.875 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.874+0000 7f364663f740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f363c1b4e10 con 0x559b574c5050 2024-04-26T21:56:36.875 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.874+0000 7f363be00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559b574c5a40 0x559b574d64d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:36.875 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.874+0000 7f363be00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559b574c5a40 0x559b574d64d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:58710/0 (socket says 172.21.15.72:58710) 2024-04-26T21:56:36.875 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.874+0000 7f363be00640 1 -- 172.21.15.72:0/704579473 learned_addr learned my addr 172.21.15.72:0/704579473 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:56:36.876 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.874+0000 7f3640e00640 1 --2- 172.21.15.72:0/704579473 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559b574c5050 0x559b574ca2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:36.876 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.875+0000 7f3641800640 1 --2- 172.21.15.72:0/704579473 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559b574cabb0 0x559b574c64e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:36.876 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.875+0000 7f363be00640 1 -- 172.21.15.72:0/704579473 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559b574c5050 msgr2=0x559b574ca2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:56:36.876 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.875+0000 7f363be00640 1 --2- 172.21.15.72:0/704579473 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559b574c5050 0x559b574ca2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:36.876 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.875+0000 7f363be00640 1 -- 172.21.15.72:0/704579473 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559b574cabb0 msgr2=0x559b574c64e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:56:36.876 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.875+0000 7f363be00640 1 --2- 172.21.15.72:0/704579473 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559b574cabb0 0x559b574c64e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:36.876 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.875+0000 7f363be00640 1 -- 172.21.15.72:0/704579473 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559b572047d0 con 0x559b574c5a40 2024-04-26T21:56:36.876 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.875+0000 7f363be00640 1 --2- 172.21.15.72:0/704579473 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559b574c5a40 0x559b574d64d0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f3628010af0 tx=0x7f36280cdad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5891766589a38b49 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:56:36.877 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.875+0000 7f362d600640 1 -- 172.21.15.72:0/704579473 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3628020070 con 0x559b574c5a40 2024-04-26T21:56:36.877 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.876+0000 7f362d600640 1 -- 172.21.15.72:0/704579473 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3628096e50 con 0x559b574c5a40 2024-04-26T21:56:36.877 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.876+0000 7f362d600640 1 -- 172.21.15.72:0/704579473 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36280d3ad0 con 0x559b574c5a40 2024-04-26T21:56:36.877 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.877+0000 7f364663f740 1 -- 172.21.15.72:0/704579473 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559b574c5a40 msgr2=0x559b574d64d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:36.877 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.877+0000 7f364663f740 1 --2- 172.21.15.72:0/704579473 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559b574c5a40 0x559b574d64d0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f3628010af0 tx=0x7f36280cdad0 comp rx=0 tx=0).stop 2024-04-26T21:56:36.877 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.877+0000 7f364663f740 1 -- 172.21.15.72:0/704579473 shutdown_connections 2024-04-26T21:56:36.877 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.877+0000 7f364663f740 1 --2- 172.21.15.72:0/704579473 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559b574cabb0 0x559b574c64e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:36.877 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.877+0000 7f364663f740 1 --2- 172.21.15.72:0/704579473 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559b574c5a40 0x559b574d64d0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:36.877 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.877+0000 7f364663f740 1 --2- 172.21.15.72:0/704579473 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559b574c5050 0x559b574ca2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:36.878 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.877+0000 7f364663f740 1 -- 172.21.15.72:0/704579473 >> 172.21.15.72:0/704579473 conn(0x559b574c06c0 msgr2=0x559b574bf7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:56:36.878 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.877+0000 7f364663f740 1 -- 172.21.15.72:0/704579473 shutdown_connections 2024-04-26T21:56:36.878 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.877+0000 7f364663f740 1 -- 172.21.15.72:0/704579473 wait complete. 2024-04-26T21:56:36.879 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.877+0000 7f364663f740 1 -- start start 2024-04-26T21:56:36.879 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.878+0000 7f364663f740 1 Processor -- start 2024-04-26T21:56:36.879 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.878+0000 7f364663f740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559b574c5050 0x559b574d3920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:36.879 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.878+0000 7f364663f740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559b574c5a40 0x559b574d3e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:36.879 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.878+0000 7f364663f740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559b574cabb0 0x559b574d4e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:36.879 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.878+0000 7f364663f740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f363c1b5bb0 con 0x559b574c5a40 2024-04-26T21:56:36.879 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.878+0000 7f364663f740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f363c1b54e0 con 0x559b574c5050 2024-04-26T21:56:36.879 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.878+0000 7f364663f740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f363c1b2720 con 0x559b574cabb0 2024-04-26T21:56:36.879 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.878+0000 7f364663f740 10 client.0 _ll_register_callbacks cb 0x7f3628098580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:56:36.880 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.879+0000 7f363be00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559b574c5a40 0x559b574d3e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:36.880 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.879+0000 7f3640e00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559b574c5050 0x559b574d3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:36.880 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.879+0000 7f3641800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559b574cabb0 0x559b574d4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:36.880 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.879+0000 7f3640e00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559b574c5050 0x559b574d3920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.72:53798/0 (socket says 172.21.15.72:53798) 2024-04-26T21:56:36.880 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.879+0000 7f3640e00640 1 -- 172.21.15.72:0/278522626 learned_addr learned my addr 172.21.15.72:0/278522626 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:56:36.880 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.880+0000 7f363be00640 1 -- 172.21.15.72:0/278522626 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559b574cabb0 msgr2=0x559b574d4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:56:36.880 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.880+0000 7f363be00640 1 --2- 172.21.15.72:0/278522626 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559b574cabb0 0x559b574d4e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:36.880 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.880+0000 7f363be00640 1 -- 172.21.15.72:0/278522626 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559b574c5050 msgr2=0x559b574d3920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:56:36.880 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.880+0000 7f363be00640 1 --2- 172.21.15.72:0/278522626 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559b574c5050 0x559b574d3920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:36.881 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.880+0000 7f363be00640 1 -- 172.21.15.72:0/278522626 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559b573cf2a0 con 0x559b574c5a40 2024-04-26T21:56:36.881 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.880+0000 7f3640e00640 1 --2- 172.21.15.72:0/278522626 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559b574c5050 0x559b574d3920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:56:36.881 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.880+0000 7f363be00640 1 --2- 172.21.15.72:0/278522626 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559b574c5a40 0x559b574d3e60 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f3628098cb0 tx=0x7f36280a0370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c4a33f4e60a24f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:56:36.881 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.880+0000 7f3622a00640 10 client.0 ms_handle_connect on v2:172.21.15.72:3300/0 2024-04-26T21:56:36.882 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.881+0000 7f3622a00640 1 -- 172.21.15.72:0/278522626 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3628020070 con 0x559b574c5a40 2024-04-26T21:56:36.882 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.881+0000 7f3622a00640 1 -- 172.21.15.72:0/278522626 <== mon.0 v2:172.21.15.72:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7f3628098230 con 0x559b574c5a40 2024-04-26T21:56:36.882 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.881+0000 7f364663f740 1 -- 172.21.15.72:0/278522626 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3628096e50 con 0x559b574c5a40 2024-04-26T21:56:36.882 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.881+0000 7f3622a00640 1 -- 172.21.15.72:0/278522626 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f36280a0690 con 0x559b574c5a40 2024-04-26T21:56:36.883 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.881+0000 7f364663f740 10 client.15771 fetch_fsmap learned FSMap version 27 2024-04-26T21:56:36.883 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.881+0000 7f364663f740 1 -- 172.21.15.72:0/278522626 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559b574d6590 con 0x559b574c5a40 2024-04-26T21:56:36.883 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.881+0000 7f3622a00640 1 -- 172.21.15.72:0/278522626 <== mon.0 v2:172.21.15.72:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3628096e50 con 0x559b574c5a40 2024-04-26T21:56:36.883 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.882+0000 7f3622a00640 1 -- 172.21.15.72:0/278522626 <== mon.0 v2:172.21.15.72:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f362809b1a0 con 0x559b574c5a40 2024-04-26T21:56:36.883 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.882+0000 7f364663f740 10 client.15771 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T21:56:36.883 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.882+0000 7f364663f740 10 client.15771 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-26T21:56:36.883 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.882+0000 7f364663f740 20 client.15771 populate_metadata read hostname 'smithi072' 2024-04-26T21:56:36.883 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.882+0000 7f364663f740 1 --2- 172.21.15.72:0/278522626 >> [v2:172.21.15.76:6836/1150678991,v1:172.21.15.76:6837/1150678991] conn(0x559b57549ab0 0x559b5754bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:36.883 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.882+0000 7f364663f740 4 client.15771 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-26T21:56:36.883 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.882+0000 7f364663f740 1 -- 172.21.15.72:0/278522626 --> [v2:172.21.15.76:6836/1150678991,v1:172.21.15.76:6837/1150678991] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559b574306f0 con 0x559b57549ab0 2024-04-26T21:56:36.883 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.882+0000 7f3640e00640 1 --2- 172.21.15.72:0/278522626 >> [v2:172.21.15.76:6836/1150678991,v1:172.21.15.76:6837/1150678991] conn(0x559b57549ab0 0x559b5754bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:36.884 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.883+0000 7f3640e00640 1 --2- 172.21.15.72:0/278522626 >> [v2:172.21.15.76:6836/1150678991,v1:172.21.15.76:6837/1150678991] conn(0x559b57549ab0 0x559b5754bea0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ca6f1845dd8c365f server_cookie=9654eb00e8d1a2a in_seq=0 out_seq=0 2024-04-26T21:56:36.884 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.883+0000 7f3622a00640 10 client.15771 ms_handle_connect on v2:172.21.15.76:6836/1150678991 2024-04-26T21:56:36.885 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.884+0000 7f3622a00640 1 -- 172.21.15.72:0/278522626 <== mds.1 v2:172.21.15.76:6836/1150678991 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559b574306f0 con 0x559b57549ab0 2024-04-26T21:56:36.885 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.884+0000 7f3622a00640 10 client.15771 handle_command_reply: tid=0 2024-04-26T21:56:36.885 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.884+0000 7f364663f740 1 client.15771 shutdown 2024-04-26T21:56:36.885 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.884+0000 7f364663f740 1 -- 172.21.15.72:0/278522626 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559b574c5a40 msgr2=0x559b574d3e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:36.885 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.884+0000 7f364663f740 1 --2- 172.21.15.72:0/278522626 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559b574c5a40 0x559b574d3e60 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f3628098cb0 tx=0x7f36280a0370 comp rx=0 tx=0).stop 2024-04-26T21:56:36.885 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.884+0000 7f364663f740 1 -- 172.21.15.72:0/278522626 shutdown_connections 2024-04-26T21:56:36.885 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.884+0000 7f364663f740 1 --2- 172.21.15.72:0/278522626 >> [v2:172.21.15.76:6836/1150678991,v1:172.21.15.76:6837/1150678991] conn(0x559b57549ab0 0x559b5754bea0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:36.885 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.884+0000 7f364663f740 1 --2- 172.21.15.72:0/278522626 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559b574cabb0 0x559b574d4e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:36.885 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.884+0000 7f364663f740 1 --2- 172.21.15.72:0/278522626 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559b574c5a40 0x559b574d3e60 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:36.885 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.884+0000 7f364663f740 1 --2- 172.21.15.72:0/278522626 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559b574c5050 0x559b574d3920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:36.885 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.884+0000 7f364663f740 1 -- 172.21.15.72:0/278522626 >> 172.21.15.72:0/278522626 conn(0x559b574c06c0 msgr2=0x559b574c35d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:56:36.885 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.884+0000 7f3622a00640 0 client.15771 ms_handle_reset on v2:172.21.15.76:6836/1150678991 2024-04-26T21:56:36.885 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.885+0000 7f364663f740 1 -- 172.21.15.72:0/278522626 shutdown_connections 2024-04-26T21:56:36.885 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.885+0000 7f364663f740 1 -- 172.21.15.72:0/278522626 wait complete. 2024-04-26T21:56:36.886 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.885+0000 7f364663f740 20 client.15771 trim_cache size 0 max 16384 2024-04-26T21:56:36.886 INFO:teuthology.orchestra.run.smithi072.stdout:[] 2024-04-26T21:56:36.887 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.886+0000 7f3642200640 1 -- 172.21.15.72:0/2758013536 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f36240a3d40 msgr2=0x7f36240a6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:36.887 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.886+0000 7f3642200640 1 --2- 172.21.15.72:0/2758013536 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f36240a3d40 0x7f36240a6200 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x559b574c62a0 tx=0x7f3634009600 comp rx=0 tx=0).stop 2024-04-26T21:56:36.887 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.886+0000 7f3642200640 1 -- 172.21.15.72:0/2758013536 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f363c101a90 msgr2=0x7f363c0718f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:36.887 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.886+0000 7f3642200640 1 --2- 172.21.15.72:0/2758013536 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f363c101a90 0x7f363c0718f0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f362800d950 tx=0x7f362800de10 comp rx=0 tx=0).stop 2024-04-26T21:56:36.887 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.886+0000 7f3642200640 1 -- 172.21.15.72:0/2758013536 shutdown_connections 2024-04-26T21:56:36.887 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.886+0000 7f3642200640 1 --2- 172.21.15.72:0/2758013536 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f36240a3d40 0x7f36240a6200 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:36.887 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.886+0000 7f3642200640 1 --2- 172.21.15.72:0/2758013536 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f363c102430 0x7f363c1aab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:36.888 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.886+0000 7f3642200640 1 --2- 172.21.15.72:0/2758013536 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f363c101a90 0x7f363c0718f0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:36.888 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.886+0000 7f3642200640 1 --2- 172.21.15.72:0/2758013536 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f363c101130 0x7f363c0713b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:36.888 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.886+0000 7f3642200640 1 -- 172.21.15.72:0/2758013536 >> 172.21.15.72:0/2758013536 conn(0x7f363c0fccd0 msgr2=0x7f363c0fe7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:56:36.888 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.886+0000 7f3642200640 1 -- 172.21.15.72:0/2758013536 shutdown_connections 2024-04-26T21:56:36.888 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:36.887+0000 7f3642200640 1 -- 172.21.15.72:0/2758013536 wait complete. 2024-04-26T21:56:36.910 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-26T21:56:37.094 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.093+0000 7f2a9ec00640 1 -- 172.21.15.72:0/1679819735 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2a98102410 msgr2=0x7f2a98102890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:37.094 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.093+0000 7f2a9ec00640 1 --2- 172.21.15.72:0/1679819735 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2a98102410 0x7f2a98102890 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f2a90009ca0 tx=0x7f2a9002a8c0 comp rx=0 tx=0).stop 2024-04-26T21:56:37.094 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.094+0000 7f2a9ec00640 1 -- 172.21.15.72:0/1679819735 shutdown_connections 2024-04-26T21:56:37.094 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.094+0000 7f2a9ec00640 1 --2- 172.21.15.72:0/1679819735 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2a98102410 0x7f2a98102890 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:37.094 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.094+0000 7f2a9ec00640 1 --2- 172.21.15.72:0/1679819735 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2a98101a70 0x7f2a98101ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:37.095 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.094+0000 7f2a9ec00640 1 --2- 172.21.15.72:0/1679819735 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2a98101110 0x7f2a98101530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:37.095 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.094+0000 7f2a9ec00640 1 -- 172.21.15.72:0/1679819735 >> 172.21.15.72:0/1679819735 conn(0x7f2a980fccd0 msgr2=0x7f2a980ff0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:56:37.095 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.094+0000 7f2a9ec00640 1 -- 172.21.15.72:0/1679819735 shutdown_connections 2024-04-26T21:56:37.095 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.094+0000 7f2a9ec00640 1 -- 172.21.15.72:0/1679819735 wait complete. 2024-04-26T21:56:37.096 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.095+0000 7f2a9ec00640 1 Processor -- start 2024-04-26T21:56:37.096 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.095+0000 7f2a9ec00640 1 -- start start 2024-04-26T21:56:37.096 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.095+0000 7f2a9ec00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2a98101110 0x7f2a9819ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:37.096 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.095+0000 7f2a9ec00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2a98101a70 0x7f2a9819b050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:37.096 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.095+0000 7f2a9ec00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2a98102410 0x7f2a981a20c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:37.097 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.095+0000 7f2a9ec00640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f2a98110460 con 0x7f2a98101a70 2024-04-26T21:56:37.097 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.095+0000 7f2a9ec00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f2a981102e0 con 0x7f2a98102410 2024-04-26T21:56:37.097 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.095+0000 7f2a9ec00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f2a981105e0 con 0x7f2a98101110 2024-04-26T21:56:37.097 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.096+0000 7f2a9ce00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2a98101a70 0x7f2a9819b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:37.097 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.096+0000 7f2a9ce00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2a98101a70 0x7f2a9819b050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:58744/0 (socket says 172.21.15.72:58744) 2024-04-26T21:56:37.097 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.096+0000 7f2a9ce00640 1 -- 172.21.15.72:0/679085284 learned_addr learned my addr 172.21.15.72:0/679085284 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:56:37.097 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.096+0000 7f2a9d800640 1 --2- 172.21.15.72:0/679085284 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2a98101110 0x7f2a9819ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:37.097 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.096+0000 7f2a9e200640 1 --2- 172.21.15.72:0/679085284 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2a98102410 0x7f2a981a20c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:37.097 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.096+0000 7f2a9ce00640 1 -- 172.21.15.72:0/679085284 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2a98101110 msgr2=0x7f2a9819ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:37.097 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.096+0000 7f2a9ce00640 1 --2- 172.21.15.72:0/679085284 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2a98101110 0x7f2a9819ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:37.097 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.096+0000 7f2a9ce00640 1 -- 172.21.15.72:0/679085284 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2a98102410 msgr2=0x7f2a981a20c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:37.097 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.096+0000 7f2a9ce00640 1 --2- 172.21.15.72:0/679085284 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2a98102410 0x7f2a981a20c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:37.097 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.096+0000 7f2a9ce00640 1 -- 172.21.15.72:0/679085284 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a981a2730 con 0x7f2a98101a70 2024-04-26T21:56:37.098 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.096+0000 7f2a9d800640 1 --2- 172.21.15.72:0/679085284 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2a98101110 0x7f2a9819ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:56:37.098 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.097+0000 7f2a9ce00640 1 --2- 172.21.15.72:0/679085284 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2a98101a70 0x7f2a9819b050 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f2a8400d770 tx=0x7f2a8400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:56:37.098 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.097+0000 7f2a96000640 1 -- 172.21.15.72:0/679085284 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a84017070 con 0x7f2a98101a70 2024-04-26T21:56:37.098 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.097+0000 7f2a96000640 1 -- 172.21.15.72:0/679085284 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2a840054b0 con 0x7f2a98101a70 2024-04-26T21:56:37.098 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.097+0000 7f2a96000640 1 -- 172.21.15.72:0/679085284 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a84004c60 con 0x7f2a98101a70 2024-04-26T21:56:37.099 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.097+0000 7f2a9ec00640 1 -- 172.21.15.72:0/679085284 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a981a2a20 con 0x7f2a98101a70 2024-04-26T21:56:37.099 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.097+0000 7f2a9ec00640 1 -- 172.21.15.72:0/679085284 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a981a2f20 con 0x7f2a98101a70 2024-04-26T21:56:37.102 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.102+0000 7f2aa3011740 1 Processor -- start 2024-04-26T21:56:37.103 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.103+0000 7f2a96000640 1 -- 172.21.15.72:0/679085284 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f2a84004e00 con 0x7f2a98101a70 2024-04-26T21:56:37.103 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.103+0000 7f2aa3011740 1 -- start start 2024-04-26T21:56:37.103 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.103+0000 7f2aa3011740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56321f2a7860 0x56321f2a7c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:37.103 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.103+0000 7f2aa3011740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56321f2a8250 0x56321f2acd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:37.103 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.103+0000 7f2aa3011740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56321f2ad3c0 0x56321f2af850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:37.103 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.103+0000 7f2aa3011740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x56321f202c70 con 0x56321f2a8250 2024-04-26T21:56:37.104 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.103+0000 7f2aa3011740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x56321f1bb980 con 0x56321f2a7860 2024-04-26T21:56:37.104 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.103+0000 7f2aa3011740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x56321f1fec30 con 0x56321f2ad3c0 2024-04-26T21:56:37.105 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.104+0000 7f2a9d800640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56321f2a8250 0x56321f2acd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:37.105 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.104+0000 7f2a9d800640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56321f2a8250 0x56321f2acd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:58750/0 (socket says 172.21.15.72:58750) 2024-04-26T21:56:37.105 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.104+0000 7f2a9d800640 1 -- 172.21.15.72:0/2952157680 learned_addr learned my addr 172.21.15.72:0/2952157680 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:56:37.105 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.104+0000 7f2a9e200640 1 --2- 172.21.15.72:0/2952157680 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56321f2a7860 0x56321f2a7c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:37.105 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.104+0000 7f2a9ce00640 1 --2- 172.21.15.72:0/2952157680 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56321f2ad3c0 0x56321f2af850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:37.105 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.104+0000 7f2a9d800640 1 -- 172.21.15.72:0/2952157680 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56321f2ad3c0 msgr2=0x56321f2af850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:56:37.105 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.104+0000 7f2a9d800640 1 --2- 172.21.15.72:0/2952157680 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56321f2ad3c0 0x56321f2af850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:37.105 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.104+0000 7f2a9d800640 1 -- 172.21.15.72:0/2952157680 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56321f2a7860 msgr2=0x56321f2a7c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:56:37.105 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.104+0000 7f2a9d800640 1 --2- 172.21.15.72:0/2952157680 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56321f2a7860 0x56321f2a7c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:37.105 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.104+0000 7f2a9d800640 1 -- 172.21.15.72:0/2952157680 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56321f1937b0 con 0x56321f2a8250 2024-04-26T21:56:37.105 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.104+0000 7f2a9ce00640 1 --2- 172.21.15.72:0/2952157680 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56321f2ad3c0 0x56321f2af850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:56:37.105 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.105+0000 7f2a9d800640 1 --2- 172.21.15.72:0/2952157680 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56321f2a8250 0x56321f2acd50 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f2a8c00bd40 tx=0x7f2a8c00f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4926a7656cd6a579 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:56:37.106 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.105+0000 7f2a96000640 1 --2- 172.21.15.72:0/679085284 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f2a800a3cc0 0x7f2a800a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:37.106 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.105+0000 7f2a8aa00640 1 -- 172.21.15.72:0/2952157680 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a8c015070 con 0x56321f2a8250 2024-04-26T21:56:37.106 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.105+0000 7f2a96000640 1 -- 172.21.15.72:0/679085284 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7f2a840d0080 con 0x7f2a98101a70 2024-04-26T21:56:37.106 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.105+0000 7f2a9e200640 1 --2- 172.21.15.72:0/679085284 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f2a800a3cc0 0x7f2a800a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:37.106 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.105+0000 7f2a8aa00640 1 -- 172.21.15.72:0/2952157680 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2a8c00be30 con 0x56321f2a8250 2024-04-26T21:56:37.107 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.105+0000 7f2a8aa00640 1 -- 172.21.15.72:0/2952157680 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a8c011900 con 0x56321f2a8250 2024-04-26T21:56:37.107 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.106+0000 7f2aa3011740 1 -- 172.21.15.72:0/2952157680 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56321f2a8250 msgr2=0x56321f2acd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:37.107 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.106+0000 7f2aa3011740 1 --2- 172.21.15.72:0/2952157680 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56321f2a8250 0x56321f2acd50 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f2a8c00bd40 tx=0x7f2a8c00f730 comp rx=0 tx=0).stop 2024-04-26T21:56:37.108 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.107+0000 7f2a9e200640 1 --2- 172.21.15.72:0/679085284 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f2a800a3cc0 0x7f2a800a6180 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x56321f2a8ab0 tx=0x7f2a90031980 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:56:37.108 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.107+0000 7f2aa3011740 1 -- 172.21.15.72:0/2952157680 shutdown_connections 2024-04-26T21:56:37.108 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.107+0000 7f2aa3011740 1 --2- 172.21.15.72:0/2952157680 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56321f2ad3c0 0x56321f2af850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:37.108 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.107+0000 7f2aa3011740 1 --2- 172.21.15.72:0/2952157680 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56321f2a8250 0x56321f2acd50 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:37.108 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.107+0000 7f2aa3011740 1 --2- 172.21.15.72:0/2952157680 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56321f2a7860 0x56321f2a7c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:37.108 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.107+0000 7f2aa3011740 1 -- 172.21.15.72:0/2952157680 >> 172.21.15.72:0/2952157680 conn(0x56321f2a2ed0 msgr2=0x56321f2a32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:56:37.108 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.107+0000 7f2aa3011740 1 -- 172.21.15.72:0/2952157680 shutdown_connections 2024-04-26T21:56:37.108 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.107+0000 7f2aa3011740 1 -- 172.21.15.72:0/2952157680 wait complete. 2024-04-26T21:56:37.108 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.107+0000 7f2aa3011740 1 -- start start 2024-04-26T21:56:37.108 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.108+0000 7f2aa3011740 1 Processor -- start 2024-04-26T21:56:37.108 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.108+0000 7f2aa3011740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56321f2a7860 0x56321f2b1800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:37.109 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.108+0000 7f2aa3011740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56321f2a8250 0x56321f2b1d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:37.109 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.108+0000 7f2aa3011740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56321f2ad3c0 0x56321f2b8d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:37.109 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.108+0000 7f2aa3011740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x56321f201f80 con 0x56321f2ad3c0 2024-04-26T21:56:37.110 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.108+0000 7f2aa3011740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x56321f1af260 con 0x56321f2a8250 2024-04-26T21:56:37.110 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.108+0000 7f2aa3011740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x56321f2072a0 con 0x56321f2a7860 2024-04-26T21:56:37.110 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.108+0000 7f2aa3011740 10 client.0 _ll_register_callbacks cb 0x56321f1f9960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:56:37.110 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.109+0000 7f2a9e200640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56321f2ad3c0 0x56321f2b8d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:37.110 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.109+0000 7f2a9e200640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56321f2ad3c0 0x56321f2b8d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:58752/0 (socket says 172.21.15.72:58752) 2024-04-26T21:56:37.110 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.109+0000 7f2a9e200640 1 -- 172.21.15.72:0/3126158853 learned_addr learned my addr 172.21.15.72:0/3126158853 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:56:37.110 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.109+0000 7f2a9d800640 1 --2- 172.21.15.72:0/3126158853 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56321f2a7860 0x56321f2b1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:37.110 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.109+0000 7f2a9ce00640 1 --2- 172.21.15.72:0/3126158853 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56321f2a8250 0x56321f2b1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:37.110 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.109+0000 7f2a9e200640 1 -- 172.21.15.72:0/3126158853 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56321f2a7860 msgr2=0x56321f2b1800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:56:37.110 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.109+0000 7f2a9e200640 1 --2- 172.21.15.72:0/3126158853 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56321f2a7860 0x56321f2b1800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:37.110 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.109+0000 7f2a9e200640 1 -- 172.21.15.72:0/3126158853 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56321f2a8250 msgr2=0x56321f2b1d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:56:37.110 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.109+0000 7f2a9e200640 1 --2- 172.21.15.72:0/3126158853 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56321f2a8250 0x56321f2b1d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:37.110 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.109+0000 7f2a9e200640 1 -- 172.21.15.72:0/3126158853 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56321f1911c0 con 0x56321f2ad3c0 2024-04-26T21:56:37.111 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.110+0000 7f2a9e200640 1 --2- 172.21.15.72:0/3126158853 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56321f2ad3c0 0x56321f2b8d10 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f2a90035000 tx=0x7f2a900095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd21598efc7b83a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:56:37.111 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.110+0000 7f2a7f400640 10 client.0 ms_handle_connect on v2:172.21.15.72:3300/0 2024-04-26T21:56:37.112 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.110+0000 7f2a7f400640 1 -- 172.21.15.72:0/3126158853 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a9004f070 con 0x56321f2ad3c0 2024-04-26T21:56:37.112 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.110+0000 7f2a7f400640 1 -- 172.21.15.72:0/3126158853 <== mon.0 v2:172.21.15.72:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7f2a90004510 con 0x56321f2ad3c0 2024-04-26T21:56:37.112 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.110+0000 7f2aa3011740 1 -- 172.21.15.72:0/3126158853 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2a8c00be30 con 0x56321f2ad3c0 2024-04-26T21:56:37.112 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.111+0000 7f2a7f400640 1 -- 172.21.15.72:0/3126158853 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2a9005b050 con 0x56321f2ad3c0 2024-04-26T21:56:37.112 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.111+0000 7f2aa3011740 10 client.15783 fetch_fsmap learned FSMap version 27 2024-04-26T21:56:37.112 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.111+0000 7f2aa3011740 1 -- 172.21.15.72:0/3126158853 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56321ef143f0 con 0x56321f2ad3c0 2024-04-26T21:56:37.112 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.111+0000 7f2a7f400640 1 -- 172.21.15.72:0/3126158853 <== mon.0 v2:172.21.15.72:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2a8c00be30 con 0x56321f2ad3c0 2024-04-26T21:56:37.113 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.112+0000 7f2a7f400640 1 -- 172.21.15.72:0/3126158853 <== mon.0 v2:172.21.15.72:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f2a90044040 con 0x56321f2ad3c0 2024-04-26T21:56:37.113 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.112+0000 7f2aa3011740 10 client.15783 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T21:56:37.113 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.112+0000 7f2aa3011740 10 client.15783 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:56:37.113 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.112+0000 7f2aa3011740 20 client.15783 populate_metadata read hostname 'smithi072' 2024-04-26T21:56:37.113 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.112+0000 7f2aa3011740 1 --2- 172.21.15.72:0/3126158853 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x56321f2bd1c0 0x56321f2bf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:37.113 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.112+0000 7f2aa3011740 4 client.15783 mds_command: new command op to 34517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:56:37.113 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.112+0000 7f2aa3011740 1 -- 172.21.15.72:0/3126158853 --> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56321f210880 con 0x56321f2bd1c0 2024-04-26T21:56:37.113 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.113+0000 7f2a9d800640 1 --2- 172.21.15.72:0/3126158853 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x56321f2bd1c0 0x56321f2bf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:37.114 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.114+0000 7f2a9d800640 1 --2- 172.21.15.72:0/3126158853 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x56321f2bd1c0 0x56321f2bf640 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a67b19831da620d3 server_cookie=37f237129e331f88 in_seq=0 out_seq=0 2024-04-26T21:56:37.114 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.114+0000 7f2a7f400640 10 client.15783 ms_handle_connect on v2:172.21.15.76:6838/327145636 2024-04-26T21:56:37.116 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.116+0000 7f2a7f400640 1 -- 172.21.15.72:0/3126158853 <== mds.0 v2:172.21.15.76:6838/327145636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56321f210880 con 0x56321f2bd1c0 2024-04-26T21:56:37.116 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.116+0000 7f2a7f400640 10 client.15783 handle_command_reply: tid=0 2024-04-26T21:56:37.116 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.116+0000 7f2aa3011740 1 client.15783 shutdown 2024-04-26T21:56:37.116 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.116+0000 7f2aa3011740 1 -- 172.21.15.72:0/3126158853 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56321f2ad3c0 msgr2=0x56321f2b8d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:37.116 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.116+0000 7f2aa3011740 1 --2- 172.21.15.72:0/3126158853 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56321f2ad3c0 0x56321f2b8d10 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f2a90035000 tx=0x7f2a900095c0 comp rx=0 tx=0).stop 2024-04-26T21:56:37.117 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.116+0000 7f2aa3011740 1 -- 172.21.15.72:0/3126158853 shutdown_connections 2024-04-26T21:56:37.117 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.116+0000 7f2aa3011740 1 --2- 172.21.15.72:0/3126158853 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x56321f2bd1c0 0x56321f2bf640 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:37.117 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.116+0000 7f2aa3011740 1 --2- 172.21.15.72:0/3126158853 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56321f2ad3c0 0x56321f2b8d10 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:37.117 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.116+0000 7f2aa3011740 1 --2- 172.21.15.72:0/3126158853 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56321f2a8250 0x56321f2b1d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:37.117 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.116+0000 7f2aa3011740 1 --2- 172.21.15.72:0/3126158853 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56321f2a7860 0x56321f2b1800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:37.117 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.116+0000 7f2aa3011740 1 -- 172.21.15.72:0/3126158853 >> 172.21.15.72:0/3126158853 conn(0x56321f2a2ed0 msgr2=0x56321f2997b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:56:37.117 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.116+0000 7f2a7f400640 0 client.15783 ms_handle_reset on v2:172.21.15.76:6838/327145636 2024-04-26T21:56:37.118 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.117+0000 7f2aa3011740 1 -- 172.21.15.72:0/3126158853 shutdown_connections 2024-04-26T21:56:37.118 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.117+0000 7f2aa3011740 1 -- 172.21.15.72:0/3126158853 wait complete. 2024-04-26T21:56:37.118 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.117+0000 7f2aa3011740 20 client.15783 trim_cache size 0 max 16384 2024-04-26T21:56:37.144 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.143+0000 7f2aa3011740 1 Processor -- start 2024-04-26T21:56:37.144 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.143+0000 7f2aa3011740 1 -- start start 2024-04-26T21:56:37.144 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.144+0000 7f2aa3011740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56321f2a7860 0x56321f2bf1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:37.144 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.144+0000 7f2aa3011740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56321f2a8250 0x56321f2b8cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:37.144 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.144+0000 7f2aa3011740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56321f2ad3c0 0x56321f2a64f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:37.144 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.144+0000 7f2aa3011740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f2a981a7aa0 con 0x56321f2a7860 2024-04-26T21:56:37.144 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.144+0000 7f2aa3011740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f2a981a95e0 con 0x56321f2a8250 2024-04-26T21:56:37.144 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.144+0000 7f2aa3011740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f2a981a8170 con 0x56321f2ad3c0 2024-04-26T21:56:37.145 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.144+0000 7f2a9d800640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56321f2a7860 0x56321f2bf1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:37.145 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.144+0000 7f2a9d800640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56321f2a7860 0x56321f2bf1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:58766/0 (socket says 172.21.15.72:58766) 2024-04-26T21:56:37.145 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.144+0000 7f2a9d800640 1 -- 172.21.15.72:0/187055597 learned_addr learned my addr 172.21.15.72:0/187055597 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:56:37.145 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.144+0000 7f2a9ce00640 1 --2- 172.21.15.72:0/187055597 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56321f2a8250 0x56321f2b8cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:37.145 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.145+0000 7f2a9e200640 1 --2- 172.21.15.72:0/187055597 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56321f2ad3c0 0x56321f2a64f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:37.146 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.145+0000 7f2a9d800640 1 -- 172.21.15.72:0/187055597 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56321f2ad3c0 msgr2=0x56321f2a64f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:56:37.146 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.145+0000 7f2a9d800640 1 --2- 172.21.15.72:0/187055597 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56321f2ad3c0 0x56321f2a64f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:37.146 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.145+0000 7f2a9d800640 1 -- 172.21.15.72:0/187055597 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56321f2a8250 msgr2=0x56321f2b8cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:56:37.146 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.145+0000 7f2a9d800640 1 --2- 172.21.15.72:0/187055597 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56321f2a8250 0x56321f2b8cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:37.146 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.145+0000 7f2a9d800640 1 -- 172.21.15.72:0/187055597 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56321efde7d0 con 0x56321f2a7860 2024-04-26T21:56:37.146 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.145+0000 7f2a9d800640 1 --2- 172.21.15.72:0/187055597 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56321f2a7860 0x56321f2bf1d0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f2a8c00bd40 tx=0x7f2a8c005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2d0ad4d59043a0c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:56:37.147 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.146+0000 7f2a8a000640 1 -- 172.21.15.72:0/187055597 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a8c015040 con 0x56321f2a7860 2024-04-26T21:56:37.147 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.146+0000 7f2a8a000640 1 -- 172.21.15.72:0/187055597 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2a8c004240 con 0x56321f2a7860 2024-04-26T21:56:37.147 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.146+0000 7f2a8a000640 1 -- 172.21.15.72:0/187055597 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a8c004c30 con 0x56321f2a7860 2024-04-26T21:56:37.147 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.147+0000 7f2aa3011740 1 -- 172.21.15.72:0/187055597 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56321f2a7860 msgr2=0x56321f2bf1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:37.147 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.147+0000 7f2aa3011740 1 --2- 172.21.15.72:0/187055597 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56321f2a7860 0x56321f2bf1d0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f2a8c00bd40 tx=0x7f2a8c005850 comp rx=0 tx=0).stop 2024-04-26T21:56:37.147 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.147+0000 7f2aa3011740 1 -- 172.21.15.72:0/187055597 shutdown_connections 2024-04-26T21:56:37.147 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.147+0000 7f2aa3011740 1 --2- 172.21.15.72:0/187055597 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56321f2ad3c0 0x56321f2a64f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:37.147 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.147+0000 7f2aa3011740 1 --2- 172.21.15.72:0/187055597 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56321f2a8250 0x56321f2b8cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:37.147 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.147+0000 7f2aa3011740 1 --2- 172.21.15.72:0/187055597 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56321f2a7860 0x56321f2bf1d0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:37.148 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.147+0000 7f2aa3011740 1 -- 172.21.15.72:0/187055597 >> 172.21.15.72:0/187055597 conn(0x56321f2a2ed0 msgr2=0x56321f2a5de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:56:37.148 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.147+0000 7f2aa3011740 1 -- 172.21.15.72:0/187055597 shutdown_connections 2024-04-26T21:56:37.148 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.147+0000 7f2aa3011740 1 -- 172.21.15.72:0/187055597 wait complete. 2024-04-26T21:56:37.149 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.147+0000 7f2aa3011740 1 -- start start 2024-04-26T21:56:37.149 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.148+0000 7f2aa3011740 1 Processor -- start 2024-04-26T21:56:37.149 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.148+0000 7f2aa3011740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56321f2a7860 0x56321f2b5e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:37.149 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.148+0000 7f2aa3011740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56321f2a8250 0x56321f2b63a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:37.149 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.148+0000 7f2aa3011740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56321f2ad3c0 0x56321f2b7340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:37.149 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.148+0000 7f2aa3011740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f2a981a8f10 con 0x56321f2a8250 2024-04-26T21:56:37.149 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.148+0000 7f2aa3011740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f2a981a8840 con 0x56321f2a7860 2024-04-26T21:56:37.149 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.148+0000 7f2aa3011740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f2a981a5a80 con 0x56321f2ad3c0 2024-04-26T21:56:37.149 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.148+0000 7f2aa3011740 10 client.0 _ll_register_callbacks cb 0x7f2a90009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:56:37.149 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.149+0000 7f2a9ce00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56321f2a8250 0x56321f2b63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:37.149 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.149+0000 7f2a9ce00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56321f2a8250 0x56321f2b63a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:58782/0 (socket says 172.21.15.72:58782) 2024-04-26T21:56:37.149 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.149+0000 7f2a9ce00640 1 -- 172.21.15.72:0/275648103 learned_addr learned my addr 172.21.15.72:0/275648103 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:56:37.151 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.149+0000 7f2a9d800640 1 --2- 172.21.15.72:0/275648103 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56321f2a7860 0x56321f2b5e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:37.151 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.149+0000 7f2a9ce00640 1 -- 172.21.15.72:0/275648103 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56321f2ad3c0 msgr2=0x56321f2b7340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:56:37.151 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.149+0000 7f2a9ce00640 1 --2- 172.21.15.72:0/275648103 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56321f2ad3c0 0x56321f2b7340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:37.151 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.149+0000 7f2a9ce00640 1 -- 172.21.15.72:0/275648103 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56321f2a7860 msgr2=0x56321f2b5e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:56:37.151 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.149+0000 7f2a9ce00640 1 --2- 172.21.15.72:0/275648103 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56321f2a7860 0x56321f2b5e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:37.151 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.149+0000 7f2a9ce00640 1 -- 172.21.15.72:0/275648103 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56321f1a92a0 con 0x56321f2a8250 2024-04-26T21:56:37.151 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.150+0000 7f2a9ce00640 1 --2- 172.21.15.72:0/275648103 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56321f2a8250 0x56321f2b63a0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f2a84021000 tx=0x7f2a840984e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1261024d7438a768 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:56:37.151 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.150+0000 7f2a7f400640 10 client.0 ms_handle_connect on v2:172.21.15.72:3300/0 2024-04-26T21:56:37.151 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.150+0000 7f2a7f400640 1 -- 172.21.15.72:0/275648103 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a84020040 con 0x56321f2a8250 2024-04-26T21:56:37.151 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.150+0000 7f2a7f400640 1 -- 172.21.15.72:0/275648103 <== mon.0 v2:172.21.15.72:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7f2a840987f0 con 0x56321f2a8250 2024-04-26T21:56:37.151 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.150+0000 7f2aa3011740 1 -- 172.21.15.72:0/275648103 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2a8c004240 con 0x56321f2a8250 2024-04-26T21:56:37.152 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.150+0000 7f2a7f400640 1 -- 172.21.15.72:0/275648103 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2a840a4df0 con 0x56321f2a8250 2024-04-26T21:56:37.152 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.151+0000 7f2aa3011740 10 client.15789 fetch_fsmap learned FSMap version 27 2024-04-26T21:56:37.152 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.151+0000 7f2aa3011740 1 -- 172.21.15.72:0/275648103 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56321f2b8ad0 con 0x56321f2a8250 2024-04-26T21:56:37.152 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.151+0000 7f2a7f400640 1 -- 172.21.15.72:0/275648103 <== mon.0 v2:172.21.15.72:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2a8c004240 con 0x56321f2a8250 2024-04-26T21:56:37.153 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.152+0000 7f2a7f400640 1 -- 172.21.15.72:0/275648103 <== mon.0 v2:172.21.15.72:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f2a84005180 con 0x56321f2a8250 2024-04-26T21:56:37.153 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.152+0000 7f2aa3011740 10 client.15789 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T21:56:37.153 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.152+0000 7f2aa3011740 10 client.15789 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:56:37.153 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.152+0000 7f2aa3011740 20 client.15789 populate_metadata read hostname 'smithi072' 2024-04-26T21:56:37.153 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.152+0000 7f2aa3011740 1 --2- 172.21.15.72:0/275648103 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x56321f323a70 0x56321f325e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:37.153 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.152+0000 7f2aa3011740 4 client.15789 mds_command: new command op to 34517 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-26T21:56:37.153 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.152+0000 7f2aa3011740 1 -- 172.21.15.72:0/275648103 --> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56321f20a6f0 con 0x56321f323a70 2024-04-26T21:56:37.153 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.153+0000 7f2a9d800640 1 --2- 172.21.15.72:0/275648103 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x56321f323a70 0x56321f325e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:37.154 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.153+0000 7f2a9d800640 1 --2- 172.21.15.72:0/275648103 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x56321f323a70 0x56321f325e60 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a97f7283b6731f90 server_cookie=42883aab7167c145 in_seq=0 out_seq=0 2024-04-26T21:56:37.154 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.153+0000 7f2a7f400640 10 client.15789 ms_handle_connect on v2:172.21.15.76:6838/327145636 2024-04-26T21:56:37.154 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.154+0000 7f2a7f400640 1 -- 172.21.15.72:0/275648103 <== mds.0 v2:172.21.15.76:6838/327145636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56321f20a6f0 con 0x56321f323a70 2024-04-26T21:56:37.154 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.154+0000 7f2a7f400640 10 client.15789 handle_command_reply: tid=0 2024-04-26T21:56:37.154 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.154+0000 7f2aa3011740 1 client.15789 shutdown 2024-04-26T21:56:37.154 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.154+0000 7f2aa3011740 1 -- 172.21.15.72:0/275648103 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56321f2a8250 msgr2=0x56321f2b63a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:37.155 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.154+0000 7f2aa3011740 1 --2- 172.21.15.72:0/275648103 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56321f2a8250 0x56321f2b63a0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f2a84021000 tx=0x7f2a840984e0 comp rx=0 tx=0).stop 2024-04-26T21:56:37.155 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.155+0000 7f2aa3011740 1 -- 172.21.15.72:0/275648103 shutdown_connections 2024-04-26T21:56:37.155 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.155+0000 7f2aa3011740 1 --2- 172.21.15.72:0/275648103 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x56321f323a70 0x56321f325e60 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:37.155 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.155+0000 7f2aa3011740 1 --2- 172.21.15.72:0/275648103 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56321f2ad3c0 0x56321f2b7340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:37.155 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.155+0000 7f2aa3011740 1 --2- 172.21.15.72:0/275648103 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56321f2a8250 0x56321f2b63a0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:37.155 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.155+0000 7f2aa3011740 1 --2- 172.21.15.72:0/275648103 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56321f2a7860 0x56321f2b5e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:37.155 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.155+0000 7f2aa3011740 1 -- 172.21.15.72:0/275648103 >> 172.21.15.72:0/275648103 conn(0x56321f2a2ed0 msgr2=0x56321f298f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:56:37.155 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.155+0000 7f2a7f400640 0 client.15789 ms_handle_reset on v2:172.21.15.76:6838/327145636 2024-04-26T21:56:37.157 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.157+0000 7f2aa3011740 1 -- 172.21.15.72:0/275648103 shutdown_connections 2024-04-26T21:56:37.157 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.157+0000 7f2aa3011740 1 -- 172.21.15.72:0/275648103 wait complete. 2024-04-26T21:56:37.158 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.159+0000 7f2aa3011740 20 client.15789 trim_cache size 0 max 16384 2024-04-26T21:56:37.159 INFO:teuthology.orchestra.run.smithi072.stdout:[] 2024-04-26T21:56:37.159 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.159+0000 7f2a9ec00640 1 -- 172.21.15.72:0/679085284 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f2a800a3cc0 msgr2=0x7f2a800a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:37.159 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.159+0000 7f2a9ec00640 1 --2- 172.21.15.72:0/679085284 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f2a800a3cc0 0x7f2a800a6180 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x56321f2a8ab0 tx=0x7f2a90031980 comp rx=0 tx=0).stop 2024-04-26T21:56:37.159 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.159+0000 7f2a9ec00640 1 -- 172.21.15.72:0/679085284 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2a98101a70 msgr2=0x7f2a9819b050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:37.160 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.159+0000 7f2a9ec00640 1 --2- 172.21.15.72:0/679085284 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2a98101a70 0x7f2a9819b050 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f2a8400d770 tx=0x7f2a8400dc30 comp rx=0 tx=0).stop 2024-04-26T21:56:37.161 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.161+0000 7f2a9ec00640 1 -- 172.21.15.72:0/679085284 shutdown_connections 2024-04-26T21:56:37.161 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.161+0000 7f2a9ec00640 1 --2- 172.21.15.72:0/679085284 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2a98102410 0x7f2a981a20c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:37.161 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.161+0000 7f2a9ec00640 1 --2- 172.21.15.72:0/679085284 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2a98101a70 0x7f2a9819b050 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:37.161 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.161+0000 7f2a9ec00640 1 --2- 172.21.15.72:0/679085284 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f2a800a3cc0 0x7f2a800a6180 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:37.161 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.161+0000 7f2a9ec00640 1 --2- 172.21.15.72:0/679085284 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2a98101110 0x7f2a9819ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:37.161 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.161+0000 7f2a9ec00640 1 -- 172.21.15.72:0/679085284 >> 172.21.15.72:0/679085284 conn(0x7f2a980fccd0 msgr2=0x7f2a980fe420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:56:37.161 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.161+0000 7f2a9ec00640 1 -- 172.21.15.72:0/679085284 shutdown_connections 2024-04-26T21:56:37.161 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:37.161+0000 7f2a9ec00640 1 -- 172.21.15.72:0/679085284 wait complete. 2024-04-26T21:56:37.181 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:56:36 smithi072 ceph-mon[28138]: pgmap v484: 97 pgs: 97 active+clean; 650 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-26T21:56:37.216 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:56:36 smithi076 ceph-mon[31823]: pgmap v484: 97 pgs: 97 active+clean; 650 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-26T21:56:37.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:56:36 smithi164 ceph-mon[31916]: pgmap v484: 97 pgs: 97 active+clean; 650 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-26T21:56:38.184 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-26T21:56:38.185 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:56:38.185 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-26T21:56:38.458 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.458+0000 7f5daa600640 1 -- 172.21.15.72:0/3854594187 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5da4101110 msgr2=0x7f5da4101530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:38.459 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.458+0000 7f5daa600640 1 --2- 172.21.15.72:0/3854594187 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5da4101110 0x7f5da4101530 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f5d98009900 tx=0x7f5d9802a410 comp rx=0 tx=0).stop 2024-04-26T21:56:38.459 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.458+0000 7f5daa600640 1 -- 172.21.15.72:0/3854594187 shutdown_connections 2024-04-26T21:56:38.459 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.458+0000 7f5daa600640 1 --2- 172.21.15.72:0/3854594187 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5da4102410 0x7f5da4102890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:38.459 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.458+0000 7f5daa600640 1 --2- 172.21.15.72:0/3854594187 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5da4101a70 0x7f5da4101ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:38.459 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.458+0000 7f5daa600640 1 --2- 172.21.15.72:0/3854594187 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5da4101110 0x7f5da4101530 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:38.459 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.458+0000 7f5daa600640 1 -- 172.21.15.72:0/3854594187 >> 172.21.15.72:0/3854594187 conn(0x7f5da40fccd0 msgr2=0x7f5da40ff0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:56:38.459 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.458+0000 7f5daa600640 1 -- 172.21.15.72:0/3854594187 shutdown_connections 2024-04-26T21:56:38.459 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.458+0000 7f5daa600640 1 -- 172.21.15.72:0/3854594187 wait complete. 2024-04-26T21:56:38.459 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.460+0000 7f5daa600640 1 Processor -- start 2024-04-26T21:56:38.460 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.460+0000 7f5daa600640 1 -- start start 2024-04-26T21:56:38.460 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.460+0000 7f5daa600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5da4101110 0x7f5da419abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:38.460 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.460+0000 7f5daa600640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5da4101a70 0x7f5da419b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:38.461 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.460+0000 7f5daa600640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5da4102410 0x7f5da41a2160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:38.461 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.460+0000 7f5daa600640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f5da4110400 con 0x7f5da4101a70 2024-04-26T21:56:38.461 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.460+0000 7f5daa600640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f5da4110280 con 0x7f5da4102410 2024-04-26T21:56:38.461 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.460+0000 7f5daa600640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f5da4110580 con 0x7f5da4101110 2024-04-26T21:56:38.462 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.461+0000 7f5da3e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5da4101a70 0x7f5da419b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:38.462 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.461+0000 7f5da3e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5da4101a70 0x7f5da419b0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:58800/0 (socket says 172.21.15.72:58800) 2024-04-26T21:56:38.462 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.461+0000 7f5da3e00640 1 -- 172.21.15.72:0/713869073 learned_addr learned my addr 172.21.15.72:0/713869073 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:56:38.462 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.461+0000 7f5da9200640 1 --2- 172.21.15.72:0/713869073 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5da4101110 0x7f5da419abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:38.462 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.461+0000 7f5da9c00640 1 --2- 172.21.15.72:0/713869073 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5da4102410 0x7f5da41a2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:38.462 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.462+0000 7f5da3e00640 1 -- 172.21.15.72:0/713869073 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5da4101110 msgr2=0x7f5da419abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:38.462 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.462+0000 7f5da3e00640 1 --2- 172.21.15.72:0/713869073 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5da4101110 0x7f5da419abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:38.462 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.462+0000 7f5da3e00640 1 -- 172.21.15.72:0/713869073 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5da4102410 msgr2=0x7f5da41a2160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:38.462 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.462+0000 7f5da3e00640 1 --2- 172.21.15.72:0/713869073 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5da4102410 0x7f5da41a2160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:38.462 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.462+0000 7f5da3e00640 1 -- 172.21.15.72:0/713869073 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5da41a27d0 con 0x7f5da4101a70 2024-04-26T21:56:38.463 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.462+0000 7f5da3e00640 1 --2- 172.21.15.72:0/713869073 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5da4101a70 0x7f5da419b0f0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f5d9000d770 tx=0x7f5d9000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:56:38.463 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.462+0000 7f5da1600640 1 -- 172.21.15.72:0/713869073 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d90017070 con 0x7f5da4101a70 2024-04-26T21:56:38.463 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.462+0000 7f5da1600640 1 -- 172.21.15.72:0/713869073 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5d900054b0 con 0x7f5da4101a70 2024-04-26T21:56:38.463 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.462+0000 7f5da1600640 1 -- 172.21.15.72:0/713869073 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d90004c60 con 0x7f5da4101a70 2024-04-26T21:56:38.463 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.463+0000 7f5daa600640 1 -- 172.21.15.72:0/713869073 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5da41a2ac0 con 0x7f5da4101a70 2024-04-26T21:56:38.464 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.463+0000 7f5daa600640 1 -- 172.21.15.72:0/713869073 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5da41a2f90 con 0x7f5da4101a70 2024-04-26T21:56:38.468 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.468+0000 7f5daebd2740 1 Processor -- start 2024-04-26T21:56:38.469 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.468+0000 7f5daebd2740 1 -- start start 2024-04-26T21:56:38.469 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.469+0000 7f5daebd2740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55da52776a70 0x55da52776e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:38.469 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.469+0000 7f5daebd2740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55da52777460 0x55da5277bf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:38.469 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.469+0000 7f5daebd2740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55da5277c5d0 0x55da5277ea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:38.469 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.469+0000 7f5daebd2740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55da526db980 con 0x55da52777460 2024-04-26T21:56:38.470 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.469+0000 7f5daebd2740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55da52678220 con 0x55da52776a70 2024-04-26T21:56:38.470 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.469+0000 7f5daebd2740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55da526d7940 con 0x55da5277c5d0 2024-04-26T21:56:38.471 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.469+0000 7f5da1600640 1 -- 172.21.15.72:0/713869073 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f5d90004e00 con 0x7f5da4101a70 2024-04-26T21:56:38.471 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.469+0000 7f5da9200640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55da52777460 0x55da5277bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:38.472 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.469+0000 7f5da9200640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55da52777460 0x55da5277bf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:58804/0 (socket says 172.21.15.72:58804) 2024-04-26T21:56:38.472 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.469+0000 7f5da9200640 1 -- 172.21.15.72:0/2619759187 learned_addr learned my addr 172.21.15.72:0/2619759187 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:56:38.472 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.469+0000 7f5da9c00640 1 --2- 172.21.15.72:0/2619759187 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55da52776a70 0x55da52776e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:38.472 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.470+0000 7f5da3e00640 1 --2- 172.21.15.72:0/2619759187 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55da5277c5d0 0x55da5277ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:38.472 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.470+0000 7f5da9200640 1 -- 172.21.15.72:0/2619759187 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55da5277c5d0 msgr2=0x55da5277ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:56:38.472 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.470+0000 7f5da9200640 1 --2- 172.21.15.72:0/2619759187 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55da5277c5d0 0x55da5277ea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:38.472 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.470+0000 7f5da9200640 1 -- 172.21.15.72:0/2619759187 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55da52776a70 msgr2=0x55da52776e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:56:38.472 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.470+0000 7f5da9200640 1 --2- 172.21.15.72:0/2619759187 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55da52776a70 0x55da52776e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:38.472 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.470+0000 7f5da9200640 1 -- 172.21.15.72:0/2619759187 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da524b57e0 con 0x55da52777460 2024-04-26T21:56:38.472 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.471+0000 7f5da9200640 1 --2- 172.21.15.72:0/2619759187 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55da52777460 0x55da5277bf60 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f5d98000c00 tx=0x7f5d98031580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3597ad481bbfa44 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:56:38.472 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.471+0000 7f5d96000640 1 -- 172.21.15.72:0/2619759187 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d98035030 con 0x55da52777460 2024-04-26T21:56:38.472 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.471+0000 7f5da1600640 1 --2- 172.21.15.72:0/713869073 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f5d8c0a3cc0 0x7f5d8c0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:38.472 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.471+0000 7f5d96000640 1 -- 172.21.15.72:0/2619759187 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5d98040070 con 0x55da52777460 2024-04-26T21:56:38.473 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.471+0000 7f5d96000640 1 -- 172.21.15.72:0/2619759187 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d98003850 con 0x55da52777460 2024-04-26T21:56:38.474 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.472+0000 7f5da1600640 1 -- 172.21.15.72:0/713869073 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7f5d900d0330 con 0x7f5da4101a70 2024-04-26T21:56:38.474 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.472+0000 7f5da9c00640 1 --2- 172.21.15.72:0/713869073 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f5d8c0a3cc0 0x7f5d8c0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:38.474 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.472+0000 7f5daebd2740 1 -- 172.21.15.72:0/2619759187 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55da52777460 msgr2=0x55da5277bf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:38.474 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.472+0000 7f5daebd2740 1 --2- 172.21.15.72:0/2619759187 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55da52777460 0x55da5277bf60 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f5d98000c00 tx=0x7f5d98031580 comp rx=0 tx=0).stop 2024-04-26T21:56:38.474 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.472+0000 7f5daebd2740 1 -- 172.21.15.72:0/2619759187 shutdown_connections 2024-04-26T21:56:38.474 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.472+0000 7f5daebd2740 1 --2- 172.21.15.72:0/2619759187 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55da5277c5d0 0x55da5277ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:38.474 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.472+0000 7f5daebd2740 1 --2- 172.21.15.72:0/2619759187 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55da52777460 0x55da5277bf60 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:38.474 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.472+0000 7f5daebd2740 1 --2- 172.21.15.72:0/2619759187 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55da52776a70 0x55da52776e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:38.474 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.472+0000 7f5daebd2740 1 -- 172.21.15.72:0/2619759187 >> 172.21.15.72:0/2619759187 conn(0x55da527720e0 msgr2=0x55da527724e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:56:38.474 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.473+0000 7f5daebd2740 1 -- 172.21.15.72:0/2619759187 shutdown_connections 2024-04-26T21:56:38.475 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.473+0000 7f5daebd2740 1 -- 172.21.15.72:0/2619759187 wait complete. 2024-04-26T21:56:38.475 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.473+0000 7f5daebd2740 1 -- start start 2024-04-26T21:56:38.475 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.473+0000 7f5da9c00640 1 --2- 172.21.15.72:0/713869073 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f5d8c0a3cc0 0x7f5d8c0a6180 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x55da52777cc0 tx=0x7f5d9c009600 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:56:38.475 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.473+0000 7f5daebd2740 1 Processor -- start 2024-04-26T21:56:38.475 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.474+0000 7f5daebd2740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55da52776a70 0x55da52780f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:38.475 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.474+0000 7f5daebd2740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55da52777460 0x55da52781470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:38.480 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.474+0000 7f5daebd2740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55da5277c5d0 0x55da527884b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:38.480 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.474+0000 7f5daebd2740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55da5277f520 con 0x55da52777460 2024-04-26T21:56:38.480 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.474+0000 7f5daebd2740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55da52663170 con 0x55da52776a70 2024-04-26T21:56:38.480 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.474+0000 7f5daebd2740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55da5277f6a0 con 0x55da5277c5d0 2024-04-26T21:56:38.480 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.474+0000 7f5daebd2740 10 client.0 _ll_register_callbacks cb 0x55da526d2770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:56:38.480 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.474+0000 7f5da3e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55da52777460 0x55da52781470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:38.480 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.475+0000 7f5da9200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55da52776a70 0x55da52780f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:38.480 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.475+0000 7f5da3e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55da52777460 0x55da52781470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:58812/0 (socket says 172.21.15.72:58812) 2024-04-26T21:56:38.481 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.475+0000 7f5da3e00640 1 -- 172.21.15.72:0/1120322312 learned_addr learned my addr 172.21.15.72:0/1120322312 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:56:38.481 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.475+0000 7f5da9c00640 1 --2- 172.21.15.72:0/1120322312 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55da5277c5d0 0x55da527884b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:38.481 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.475+0000 7f5da3e00640 1 -- 172.21.15.72:0/1120322312 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55da5277c5d0 msgr2=0x55da527884b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:56:38.481 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.475+0000 7f5da3e00640 1 --2- 172.21.15.72:0/1120322312 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55da5277c5d0 0x55da527884b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:38.481 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.475+0000 7f5da3e00640 1 -- 172.21.15.72:0/1120322312 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55da52776a70 msgr2=0x55da52780f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:56:38.481 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.475+0000 7f5da3e00640 1 --2- 172.21.15.72:0/1120322312 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55da52776a70 0x55da52780f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:38.481 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.475+0000 7f5da3e00640 1 -- 172.21.15.72:0/1120322312 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da52515f70 con 0x55da52777460 2024-04-26T21:56:38.481 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.475+0000 7f5da9c00640 1 --2- 172.21.15.72:0/1120322312 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55da5277c5d0 0x55da527884b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:56:38.481 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.476+0000 7f5da3e00640 1 --2- 172.21.15.72:0/1120322312 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55da52777460 0x55da52781470 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f5d90098830 tx=0x7f5d900056e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fbf6ab060eef3cd server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:56:38.481 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.476+0000 7f5d8aa00640 10 client.0 ms_handle_connect on v2:172.21.15.72:3300/0 2024-04-26T21:56:38.481 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.476+0000 7f5d8aa00640 1 -- 172.21.15.72:0/1120322312 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d9009bd30 con 0x55da52777460 2024-04-26T21:56:38.481 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.476+0000 7f5d8aa00640 1 -- 172.21.15.72:0/1120322312 <== mon.0 v2:172.21.15.72:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7f5d90098a40 con 0x55da52777460 2024-04-26T21:56:38.481 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.476+0000 7f5daebd2740 1 -- 172.21.15.72:0/1120322312 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5d98040070 con 0x55da52777460 2024-04-26T21:56:38.482 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.476+0000 7f5d8aa00640 1 -- 172.21.15.72:0/1120322312 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f5d900b5020 con 0x55da52777460 2024-04-26T21:56:38.482 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.476+0000 7f5daebd2740 10 client.15801 fetch_fsmap learned FSMap version 27 2024-04-26T21:56:38.482 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.476+0000 7f5daebd2740 1 -- 172.21.15.72:0/1120322312 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55da523ad170 con 0x55da52777460 2024-04-26T21:56:38.482 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.477+0000 7f5d8aa00640 1 -- 172.21.15.72:0/1120322312 <== mon.0 v2:172.21.15.72:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5d98040070 con 0x55da52777460 2024-04-26T21:56:38.482 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.477+0000 7f5d8aa00640 1 -- 172.21.15.72:0/1120322312 <== mon.0 v2:172.21.15.72:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f5d900050f0 con 0x55da52777460 2024-04-26T21:56:38.482 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.477+0000 7f5daebd2740 10 client.15801 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T21:56:38.482 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.477+0000 7f5daebd2740 10 client.15801 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:56:38.482 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.477+0000 7f5daebd2740 20 client.15801 populate_metadata read hostname 'smithi072' 2024-04-26T21:56:38.482 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.477+0000 7f5daebd2740 1 --2- 172.21.15.72:0/1120322312 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x55da52795260 0x55da527976c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:38.482 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.477+0000 7f5daebd2740 4 client.15801 mds_command: new command op to 34517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:56:38.482 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.477+0000 7f5daebd2740 1 -- 172.21.15.72:0/1120322312 --> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55da526e9590 con 0x55da52795260 2024-04-26T21:56:38.482 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.478+0000 7f5da9200640 1 --2- 172.21.15.72:0/1120322312 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x55da52795260 0x55da527976c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:38.482 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.479+0000 7f5da9200640 1 --2- 172.21.15.72:0/1120322312 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x55da52795260 0x55da527976c0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=33727cab74e48425 server_cookie=b0d69d3d7082aedc in_seq=0 out_seq=0 2024-04-26T21:56:38.483 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.479+0000 7f5d8aa00640 10 client.15801 ms_handle_connect on v2:172.21.15.76:6838/327145636 2024-04-26T21:56:38.484 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.480+0000 7f5d8aa00640 1 -- 172.21.15.72:0/1120322312 <== mds.0 v2:172.21.15.76:6838/327145636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55da526e9590 con 0x55da52795260 2024-04-26T21:56:38.484 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.480+0000 7f5d8aa00640 10 client.15801 handle_command_reply: tid=0 2024-04-26T21:56:38.484 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.481+0000 7f5daebd2740 1 client.15801 shutdown 2024-04-26T21:56:38.484 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.481+0000 7f5daebd2740 1 -- 172.21.15.72:0/1120322312 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55da52777460 msgr2=0x55da52781470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:38.484 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.481+0000 7f5daebd2740 1 --2- 172.21.15.72:0/1120322312 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55da52777460 0x55da52781470 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f5d90098830 tx=0x7f5d900056e0 comp rx=0 tx=0).stop 2024-04-26T21:56:38.484 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.481+0000 7f5daebd2740 1 -- 172.21.15.72:0/1120322312 shutdown_connections 2024-04-26T21:56:38.484 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.481+0000 7f5daebd2740 1 --2- 172.21.15.72:0/1120322312 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x55da52795260 0x55da527976c0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:38.484 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.481+0000 7f5daebd2740 1 --2- 172.21.15.72:0/1120322312 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55da5277c5d0 0x55da527884b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:38.484 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.481+0000 7f5daebd2740 1 --2- 172.21.15.72:0/1120322312 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55da52777460 0x55da52781470 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:38.484 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.481+0000 7f5daebd2740 1 --2- 172.21.15.72:0/1120322312 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55da52776a70 0x55da52780f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:38.484 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.481+0000 7f5daebd2740 1 -- 172.21.15.72:0/1120322312 >> 172.21.15.72:0/1120322312 conn(0x55da527720e0 msgr2=0x55da527712e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:56:38.484 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.481+0000 7f5d8aa00640 0 client.15801 ms_handle_reset on v2:172.21.15.76:6838/327145636 2024-04-26T21:56:38.484 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.481+0000 7f5daebd2740 1 -- 172.21.15.72:0/1120322312 shutdown_connections 2024-04-26T21:56:38.485 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.481+0000 7f5daebd2740 1 -- 172.21.15.72:0/1120322312 wait complete. 2024-04-26T21:56:38.485 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.481+0000 7f5daebd2740 20 client.15801 trim_cache size 0 max 16384 2024-04-26T21:56:38.507 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.506+0000 7f5daebd2740 1 Processor -- start 2024-04-26T21:56:38.507 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.506+0000 7f5daebd2740 1 -- start start 2024-04-26T21:56:38.507 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.506+0000 7f5daebd2740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55da52776a70 0x55da52775090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:38.507 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.506+0000 7f5daebd2740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55da5277c5d0 0x55da52788470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:38.507 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.506+0000 7f5daebd2740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55da52779070 0x55da527755d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:38.507 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.506+0000 7f5daebd2740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f5da41a7a60 con 0x55da5277c5d0 2024-04-26T21:56:38.507 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.506+0000 7f5daebd2740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f5da41a95a0 con 0x55da52776a70 2024-04-26T21:56:38.507 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.506+0000 7f5daebd2740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f5da41a8130 con 0x55da52779070 2024-04-26T21:56:38.507 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.506+0000 7f5da3e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55da5277c5d0 0x55da52788470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:38.507 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.507+0000 7f5da3e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55da5277c5d0 0x55da52788470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:58814/0 (socket says 172.21.15.72:58814) 2024-04-26T21:56:38.507 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.507+0000 7f5da3e00640 1 -- 172.21.15.72:0/2597136663 learned_addr learned my addr 172.21.15.72:0/2597136663 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:56:38.508 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.507+0000 7f5da9c00640 1 --2- 172.21.15.72:0/2597136663 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55da52779070 0x55da527755d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:38.508 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.507+0000 7f5da9200640 1 --2- 172.21.15.72:0/2597136663 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55da52776a70 0x55da52775090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:38.508 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.507+0000 7f5da3e00640 1 -- 172.21.15.72:0/2597136663 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55da52779070 msgr2=0x55da527755d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:56:38.508 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.507+0000 7f5da3e00640 1 --2- 172.21.15.72:0/2597136663 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55da52779070 0x55da527755d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:38.508 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.507+0000 7f5da3e00640 1 -- 172.21.15.72:0/2597136663 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55da52776a70 msgr2=0x55da52775090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:56:38.508 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.507+0000 7f5da3e00640 1 --2- 172.21.15.72:0/2597136663 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55da52776a70 0x55da52775090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:38.508 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.507+0000 7f5da3e00640 1 -- 172.21.15.72:0/2597136663 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da52777cf0 con 0x55da5277c5d0 2024-04-26T21:56:38.510 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.507+0000 7f5da3e00640 1 --2- 172.21.15.72:0/2597136663 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55da5277c5d0 0x55da52788470 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f5d90012040 tx=0x7f5d900a51a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe92807a0e1baa2d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:56:38.510 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.507+0000 7f5d95600640 1 -- 172.21.15.72:0/2597136663 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d9009bd30 con 0x55da5277c5d0 2024-04-26T21:56:38.510 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.508+0000 7f5d95600640 1 -- 172.21.15.72:0/2597136663 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5d9009cad0 con 0x55da5277c5d0 2024-04-26T21:56:38.510 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.508+0000 7f5d95600640 1 -- 172.21.15.72:0/2597136663 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d90098a40 con 0x55da5277c5d0 2024-04-26T21:56:38.510 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.508+0000 7f5daebd2740 1 -- 172.21.15.72:0/2597136663 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55da5277c5d0 msgr2=0x55da52788470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:38.511 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.508+0000 7f5daebd2740 1 --2- 172.21.15.72:0/2597136663 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55da5277c5d0 0x55da52788470 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f5d90012040 tx=0x7f5d900a51a0 comp rx=0 tx=0).stop 2024-04-26T21:56:38.511 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.508+0000 7f5daebd2740 1 -- 172.21.15.72:0/2597136663 shutdown_connections 2024-04-26T21:56:38.511 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.508+0000 7f5daebd2740 1 --2- 172.21.15.72:0/2597136663 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55da52779070 0x55da527755d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:38.511 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.508+0000 7f5daebd2740 1 --2- 172.21.15.72:0/2597136663 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55da5277c5d0 0x55da52788470 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:38.511 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.508+0000 7f5daebd2740 1 --2- 172.21.15.72:0/2597136663 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55da52776a70 0x55da52775090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:38.511 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.508+0000 7f5daebd2740 1 -- 172.21.15.72:0/2597136663 >> 172.21.15.72:0/2597136663 conn(0x55da527720e0 msgr2=0x55da52778960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:56:38.511 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.508+0000 7f5daebd2740 1 -- 172.21.15.72:0/2597136663 shutdown_connections 2024-04-26T21:56:38.511 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.509+0000 7f5daebd2740 1 -- 172.21.15.72:0/2597136663 wait complete. 2024-04-26T21:56:38.511 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.509+0000 7f5daebd2740 1 -- start start 2024-04-26T21:56:38.511 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.509+0000 7f5daebd2740 1 Processor -- start 2024-04-26T21:56:38.511 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.510+0000 7f5daebd2740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55da52776a70 0x55da52786270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:38.511 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.510+0000 7f5daebd2740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55da52779070 0x55da527867b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:38.511 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.510+0000 7f5daebd2740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55da5277c5d0 0x55da52787750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:38.511 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.510+0000 7f5daebd2740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f5da41a8ed0 con 0x55da52776a70 2024-04-26T21:56:38.511 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.510+0000 7f5daebd2740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f5da41a8800 con 0x55da52779070 2024-04-26T21:56:38.512 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.510+0000 7f5daebd2740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f5da41a5a40 con 0x55da5277c5d0 2024-04-26T21:56:38.512 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.510+0000 7f5daebd2740 10 client.0 _ll_register_callbacks cb 0x7f5d900d38a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:56:38.512 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.510+0000 7f5da9200640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55da52776a70 0x55da52786270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:38.512 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.510+0000 7f5da9200640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55da52776a70 0x55da52786270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:58826/0 (socket says 172.21.15.72:58826) 2024-04-26T21:56:38.512 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.510+0000 7f5da9200640 1 -- 172.21.15.72:0/1438016486 learned_addr learned my addr 172.21.15.72:0/1438016486 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:56:38.514 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.511+0000 7f5da3e00640 1 --2- 172.21.15.72:0/1438016486 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55da52779070 0x55da527867b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:38.514 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.511+0000 7f5da9200640 1 -- 172.21.15.72:0/1438016486 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55da5277c5d0 msgr2=0x55da52787750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:56:38.514 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.511+0000 7f5da9200640 1 --2- 172.21.15.72:0/1438016486 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55da5277c5d0 0x55da52787750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:38.514 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.511+0000 7f5da9200640 1 -- 172.21.15.72:0/1438016486 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55da52779070 msgr2=0x55da527867b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:56:38.514 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.511+0000 7f5da9200640 1 --2- 172.21.15.72:0/1438016486 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55da52779070 0x55da527867b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:38.514 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.511+0000 7f5da9200640 1 -- 172.21.15.72:0/1438016486 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da52787d40 con 0x55da52776a70 2024-04-26T21:56:38.514 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.511+0000 7f5da9200640 1 --2- 172.21.15.72:0/1438016486 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55da52776a70 0x55da52786270 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f5d98005940 tx=0x7f5d98031a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4eb1856b9699333 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:56:38.514 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.511+0000 7f5d8aa00640 10 client.0 ms_handle_connect on v2:172.21.15.72:3300/0 2024-04-26T21:56:38.514 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.511+0000 7f5d8aa00640 1 -- 172.21.15.72:0/1438016486 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d98035040 con 0x55da52776a70 2024-04-26T21:56:38.514 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.511+0000 7f5d8aa00640 1 -- 172.21.15.72:0/1438016486 <== mon.0 v2:172.21.15.72:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7f5d98051070 con 0x55da52776a70 2024-04-26T21:56:38.515 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.511+0000 7f5daebd2740 1 -- 172.21.15.72:0/1438016486 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5d9009cad0 con 0x55da52776a70 2024-04-26T21:56:38.515 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.512+0000 7f5d8aa00640 1 -- 172.21.15.72:0/1438016486 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f5d98051270 con 0x55da52776a70 2024-04-26T21:56:38.515 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.512+0000 7f5daebd2740 10 client.15807 fetch_fsmap learned FSMap version 27 2024-04-26T21:56:38.515 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.512+0000 7f5daebd2740 1 -- 172.21.15.72:0/1438016486 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55da527fbee0 con 0x55da52776a70 2024-04-26T21:56:38.515 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.512+0000 7f5d8aa00640 1 -- 172.21.15.72:0/1438016486 <== mon.0 v2:172.21.15.72:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5d9009cad0 con 0x55da52776a70 2024-04-26T21:56:38.515 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.512+0000 7f5d8aa00640 1 -- 172.21.15.72:0/1438016486 <== mon.0 v2:172.21.15.72:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f5d9803ca20 con 0x55da52776a70 2024-04-26T21:56:38.515 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.512+0000 7f5daebd2740 10 client.15807 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T21:56:38.515 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.512+0000 7f5daebd2740 10 client.15807 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:56:38.515 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.512+0000 7f5daebd2740 20 client.15807 populate_metadata read hostname 'smithi072' 2024-04-26T21:56:38.515 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.512+0000 7f5daebd2740 1 --2- 172.21.15.72:0/1438016486 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x55da527fc470 0x55da527fe890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:38.515 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.512+0000 7f5daebd2740 4 client.15807 mds_command: new command op to 34517 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-26T21:56:38.515 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.512+0000 7f5daebd2740 1 -- 172.21.15.72:0/1438016486 --> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55da526e3400 con 0x55da527fc470 2024-04-26T21:56:38.515 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.513+0000 7f5da3e00640 1 --2- 172.21.15.72:0/1438016486 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x55da527fc470 0x55da527fe890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:38.515 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.514+0000 7f5da3e00640 1 --2- 172.21.15.72:0/1438016486 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x55da527fc470 0x55da527fe890 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=94601a151122accf server_cookie=d525e0f67f1495e1 in_seq=0 out_seq=0 2024-04-26T21:56:38.516 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.514+0000 7f5d8aa00640 10 client.15807 ms_handle_connect on v2:172.21.15.76:6838/327145636 2024-04-26T21:56:38.516 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.517+0000 7f5d8aa00640 1 -- 172.21.15.72:0/1438016486 <== mds.0 v2:172.21.15.76:6838/327145636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55da526e3400 con 0x55da527fc470 2024-04-26T21:56:38.516 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.517+0000 7f5d8aa00640 10 client.15807 handle_command_reply: tid=0 2024-04-26T21:56:38.516 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.517+0000 7f5daebd2740 1 client.15807 shutdown 2024-04-26T21:56:38.517 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.517+0000 7f5daebd2740 1 -- 172.21.15.72:0/1438016486 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55da52776a70 msgr2=0x55da52786270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:38.518 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.517+0000 7f5daebd2740 1 --2- 172.21.15.72:0/1438016486 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55da52776a70 0x55da52786270 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f5d98005940 tx=0x7f5d98031a70 comp rx=0 tx=0).stop 2024-04-26T21:56:38.518 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.517+0000 7f5daebd2740 1 -- 172.21.15.72:0/1438016486 shutdown_connections 2024-04-26T21:56:38.518 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.517+0000 7f5daebd2740 1 --2- 172.21.15.72:0/1438016486 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x55da527fc470 0x55da527fe890 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:38.518 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.517+0000 7f5daebd2740 1 --2- 172.21.15.72:0/1438016486 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55da5277c5d0 0x55da52787750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:38.518 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.517+0000 7f5daebd2740 1 --2- 172.21.15.72:0/1438016486 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55da52779070 0x55da527867b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:38.518 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.517+0000 7f5daebd2740 1 --2- 172.21.15.72:0/1438016486 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55da52776a70 0x55da52786270 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:38.518 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.517+0000 7f5daebd2740 1 -- 172.21.15.72:0/1438016486 >> 172.21.15.72:0/1438016486 conn(0x55da527720e0 msgr2=0x55da52770a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:56:38.518 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.517+0000 7f5d8aa00640 0 client.15807 ms_handle_reset on v2:172.21.15.76:6838/327145636 2024-04-26T21:56:38.518 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.517+0000 7f5daebd2740 1 -- 172.21.15.72:0/1438016486 shutdown_connections 2024-04-26T21:56:38.518 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.517+0000 7f5daebd2740 1 -- 172.21.15.72:0/1438016486 wait complete. 2024-04-26T21:56:38.518 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.517+0000 7f5daebd2740 20 client.15807 trim_cache size 0 max 16384 2024-04-26T21:56:38.519 INFO:teuthology.orchestra.run.smithi072.stdout:{ 2024-04-26T21:56:38.519 INFO:teuthology.orchestra.run.smithi072.stdout: "return_code": 0, 2024-04-26T21:56:38.519 INFO:teuthology.orchestra.run.smithi072.stdout: "scrub_tag": "9fc595cc-50bd-4ca7-82b5-d50ff9582a99", 2024-04-26T21:56:38.519 INFO:teuthology.orchestra.run.smithi072.stdout: "mode": "asynchronous" 2024-04-26T21:56:38.519 INFO:teuthology.orchestra.run.smithi072.stdout:} 2024-04-26T21:56:38.520 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.518+0000 7f5daa600640 1 -- 172.21.15.72:0/713869073 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f5d8c0a3cc0 msgr2=0x7f5d8c0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:38.520 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.518+0000 7f5daa600640 1 --2- 172.21.15.72:0/713869073 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f5d8c0a3cc0 0x7f5d8c0a6180 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x55da52777cc0 tx=0x7f5d9c009600 comp rx=0 tx=0).stop 2024-04-26T21:56:38.520 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.518+0000 7f5daa600640 1 -- 172.21.15.72:0/713869073 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5da4101a70 msgr2=0x7f5da419b0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:38.520 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.518+0000 7f5daa600640 1 --2- 172.21.15.72:0/713869073 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5da4101a70 0x7f5da419b0f0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f5d9000d770 tx=0x7f5d9000dc30 comp rx=0 tx=0).stop 2024-04-26T21:56:38.520 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.518+0000 7f5daa600640 1 -- 172.21.15.72:0/713869073 shutdown_connections 2024-04-26T21:56:38.520 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.518+0000 7f5daa600640 1 --2- 172.21.15.72:0/713869073 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5da4102410 0x7f5da41a2160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:38.520 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.518+0000 7f5daa600640 1 --2- 172.21.15.72:0/713869073 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5da4101a70 0x7f5da419b0f0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:38.520 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.518+0000 7f5daa600640 1 --2- 172.21.15.72:0/713869073 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f5d8c0a3cc0 0x7f5d8c0a6180 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:38.520 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.518+0000 7f5daa600640 1 --2- 172.21.15.72:0/713869073 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5da4101110 0x7f5da419abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:38.520 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.518+0000 7f5daa600640 1 -- 172.21.15.72:0/713869073 >> 172.21.15.72:0/713869073 conn(0x7f5da40fccd0 msgr2=0x7f5da40fe510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:56:38.520 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.518+0000 7f5daa600640 1 -- 172.21.15.72:0/713869073 shutdown_connections 2024-04-26T21:56:38.520 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.519+0000 7f5daa600640 1 -- 172.21.15.72:0/713869073 wait complete. 2024-04-26T21:56:38.534 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-26T21:56:38.534 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-26T21:56:38.703 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.702+0000 7f6768a00640 1 -- 172.21.15.72:0/1935035158 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6764101a70 msgr2=0x7f6764101ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:38.703 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.702+0000 7f6768a00640 1 --2- 172.21.15.72:0/1935035158 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6764101a70 0x7f6764101ed0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f6750009900 tx=0x7f675002a410 comp rx=0 tx=0).stop 2024-04-26T21:56:38.703 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.702+0000 7f6768a00640 1 -- 172.21.15.72:0/1935035158 shutdown_connections 2024-04-26T21:56:38.703 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.702+0000 7f6768a00640 1 --2- 172.21.15.72:0/1935035158 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6764102410 0x7f6764102890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:38.703 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.702+0000 7f6768a00640 1 --2- 172.21.15.72:0/1935035158 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6764101a70 0x7f6764101ed0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:38.703 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.702+0000 7f6768a00640 1 --2- 172.21.15.72:0/1935035158 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6764101110 0x7f6764101530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:38.703 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.702+0000 7f6768a00640 1 -- 172.21.15.72:0/1935035158 >> 172.21.15.72:0/1935035158 conn(0x7f67640fccd0 msgr2=0x7f67640ff0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:56:38.703 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.702+0000 7f6768a00640 1 -- 172.21.15.72:0/1935035158 shutdown_connections 2024-04-26T21:56:38.704 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.702+0000 7f6768a00640 1 -- 172.21.15.72:0/1935035158 wait complete. 2024-04-26T21:56:38.705 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.703+0000 7f6768a00640 1 Processor -- start 2024-04-26T21:56:38.706 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.703+0000 7f6768a00640 1 -- start start 2024-04-26T21:56:38.706 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.704+0000 7f6768a00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6764101110 0x7f676419ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:38.706 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.704+0000 7f6768a00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6764101a70 0x7f676419b040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:38.706 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.704+0000 7f6768a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6764102410 0x7f67641a20b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:38.706 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.704+0000 7f6768a00640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f6764110450 con 0x7f6764101110 2024-04-26T21:56:38.706 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.704+0000 7f6768a00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f67641102d0 con 0x7f6764101a70 2024-04-26T21:56:38.706 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.704+0000 7f6768a00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f67641105d0 con 0x7f6764102410 2024-04-26T21:56:38.706 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.704+0000 7f6763400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6764101110 0x7f676419ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:38.706 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.704+0000 7f6763400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6764101110 0x7f676419ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:58840/0 (socket says 172.21.15.72:58840) 2024-04-26T21:56:38.706 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.704+0000 7f6763400640 1 -- 172.21.15.72:0/1471514208 learned_addr learned my addr 172.21.15.72:0/1471514208 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:56:38.706 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.704+0000 7f6762a00640 1 --2- 172.21.15.72:0/1471514208 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6764101a70 0x7f676419b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:38.706 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.704+0000 7f6763400640 1 -- 172.21.15.72:0/1471514208 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6764102410 msgr2=0x7f67641a20b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:38.706 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.704+0000 7f6763400640 1 --2- 172.21.15.72:0/1471514208 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6764102410 0x7f67641a20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:38.707 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.705+0000 7f6763400640 1 -- 172.21.15.72:0/1471514208 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6764101a70 msgr2=0x7f676419b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:38.707 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.705+0000 7f6763400640 1 --2- 172.21.15.72:0/1471514208 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6764101a70 0x7f676419b040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:38.707 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.705+0000 7f6763400640 1 -- 172.21.15.72:0/1471514208 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67641a2720 con 0x7f6764101110 2024-04-26T21:56:38.707 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.705+0000 7f6763400640 1 --2- 172.21.15.72:0/1471514208 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6764101110 0x7f676419ab00 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f675400c970 tx=0x7f675400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:56:38.707 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.705+0000 7f675be00640 1 -- 172.21.15.72:0/1471514208 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6754007bb0 con 0x7f6764101110 2024-04-26T21:56:38.707 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.705+0000 7f675be00640 1 -- 172.21.15.72:0/1471514208 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6754007d50 con 0x7f6764101110 2024-04-26T21:56:38.707 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.705+0000 7f675be00640 1 -- 172.21.15.72:0/1471514208 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6754004e40 con 0x7f6764101110 2024-04-26T21:56:38.707 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.705+0000 7f6768a00640 1 -- 172.21.15.72:0/1471514208 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67641a2a10 con 0x7f6764101110 2024-04-26T21:56:38.707 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.705+0000 7f6768a00640 1 -- 172.21.15.72:0/1471514208 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67641a2f10 con 0x7f6764101110 2024-04-26T21:56:38.709 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.708+0000 7f676cd9d740 1 Processor -- start 2024-04-26T21:56:38.709 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.709+0000 7f675be00640 1 -- 172.21.15.72:0/1471514208 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f6754005020 con 0x7f6764101110 2024-04-26T21:56:38.709 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.709+0000 7f676cd9d740 1 -- start start 2024-04-26T21:56:38.709 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.709+0000 7f676cd9d740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5649b0e7f170 0x5649b0e7f590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:38.710 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.709+0000 7f676cd9d740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5649b0e7fb60 0x5649b0e84660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:38.710 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.709+0000 7f676cd9d740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5649b0e84cd0 0x5649b0e87160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:38.710 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.709+0000 7f676cd9d740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x5649b0d87030 con 0x5649b0e84cd0 2024-04-26T21:56:38.710 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.709+0000 7f676cd9d740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5649b0da27f0 con 0x5649b0e7fb60 2024-04-26T21:56:38.710 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.709+0000 7f676cd9d740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5649b0d5d7e0 con 0x5649b0e7f170 2024-04-26T21:56:38.710 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.710+0000 7f6762a00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5649b0e84cd0 0x5649b0e87160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:38.711 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.710+0000 7f6762a00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5649b0e84cd0 0x5649b0e87160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:58842/0 (socket says 172.21.15.72:58842) 2024-04-26T21:56:38.711 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.710+0000 7f6762a00640 1 -- 172.21.15.72:0/1562449871 learned_addr learned my addr 172.21.15.72:0/1562449871 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:56:38.711 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.710+0000 7f6763e00640 1 --2- 172.21.15.72:0/1562449871 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5649b0e7f170 0x5649b0e7f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:38.711 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.710+0000 7f6763400640 1 --2- 172.21.15.72:0/1562449871 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5649b0e7fb60 0x5649b0e84660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:38.712 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.710+0000 7f6762a00640 1 -- 172.21.15.72:0/1562449871 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5649b0e7f170 msgr2=0x5649b0e7f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:56:38.712 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.710+0000 7f6762a00640 1 --2- 172.21.15.72:0/1562449871 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5649b0e7f170 0x5649b0e7f590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:38.712 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.710+0000 7f6762a00640 1 -- 172.21.15.72:0/1562449871 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5649b0e7fb60 msgr2=0x5649b0e84660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:56:38.712 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.710+0000 7f6762a00640 1 --2- 172.21.15.72:0/1562449871 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5649b0e7fb60 0x5649b0e84660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:38.712 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.710+0000 7f6762a00640 1 -- 172.21.15.72:0/1562449871 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5649b0d69050 con 0x5649b0e84cd0 2024-04-26T21:56:38.712 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.711+0000 7f675be00640 1 --2- 172.21.15.72:0/1471514208 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f67480a3cc0 0x7f67480a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:38.712 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.711+0000 7f675be00640 1 -- 172.21.15.72:0/1471514208 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7f67540d0450 con 0x7f6764101110 2024-04-26T21:56:38.712 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.711+0000 7f6762a00640 1 --2- 172.21.15.72:0/1562449871 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5649b0e84cd0 0x5649b0e87160 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f67500043c0 tx=0x7f67500043f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8b49767adb3053 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:56:38.712 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.711+0000 7f6763e00640 1 --2- 172.21.15.72:0/1471514208 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f67480a3cc0 0x7f67480a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:38.712 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.711+0000 7f674fe00640 1 -- 172.21.15.72:0/1562449871 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6750040070 con 0x5649b0e84cd0 2024-04-26T21:56:38.712 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.711+0000 7f674fe00640 1 -- 172.21.15.72:0/1562449871 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6750003c80 con 0x5649b0e84cd0 2024-04-26T21:56:38.713 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.712+0000 7f674fe00640 1 -- 172.21.15.72:0/1562449871 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6750035030 con 0x5649b0e84cd0 2024-04-26T21:56:38.715 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.712+0000 7f676cd9d740 1 -- 172.21.15.72:0/1562449871 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5649b0e84cd0 msgr2=0x5649b0e87160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:38.715 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.712+0000 7f676cd9d740 1 --2- 172.21.15.72:0/1562449871 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5649b0e84cd0 0x5649b0e87160 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f67500043c0 tx=0x7f67500043f0 comp rx=0 tx=0).stop 2024-04-26T21:56:38.715 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.712+0000 7f6763e00640 1 --2- 172.21.15.72:0/1471514208 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f67480a3cc0 0x7f67480a6180 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x5649b0e803c0 tx=0x7f675c007520 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:56:38.715 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.712+0000 7f676cd9d740 1 -- 172.21.15.72:0/1562449871 shutdown_connections 2024-04-26T21:56:38.715 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.712+0000 7f676cd9d740 1 --2- 172.21.15.72:0/1562449871 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5649b0e84cd0 0x5649b0e87160 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:38.715 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.712+0000 7f676cd9d740 1 --2- 172.21.15.72:0/1562449871 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5649b0e7fb60 0x5649b0e84660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:38.715 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.712+0000 7f676cd9d740 1 --2- 172.21.15.72:0/1562449871 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5649b0e7f170 0x5649b0e7f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:38.716 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.712+0000 7f676cd9d740 1 -- 172.21.15.72:0/1562449871 >> 172.21.15.72:0/1562449871 conn(0x5649b0e71fd0 msgr2=0x5649b0e723d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:56:38.716 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.712+0000 7f676cd9d740 1 -- 172.21.15.72:0/1562449871 shutdown_connections 2024-04-26T21:56:38.716 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.713+0000 7f676cd9d740 1 -- 172.21.15.72:0/1562449871 wait complete. 2024-04-26T21:56:38.716 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.713+0000 7f676cd9d740 1 -- start start 2024-04-26T21:56:38.716 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.713+0000 7f676cd9d740 1 Processor -- start 2024-04-26T21:56:38.716 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.714+0000 7f676cd9d740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5649b0e7f170 0x5649b0e88f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:38.716 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.714+0000 7f676cd9d740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5649b0e7fb60 0x5649b0e894b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:38.716 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.714+0000 7f676cd9d740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5649b0e84cd0 0x5649b0e904f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:38.716 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.714+0000 7f676cd9d740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x5649b0ddf200 con 0x5649b0e84cd0 2024-04-26T21:56:38.716 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.714+0000 7f676cd9d740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5649b0dd9ee0 con 0x5649b0e7fb60 2024-04-26T21:56:38.716 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.714+0000 7f676cd9d740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5649b0de6e70 con 0x5649b0e7f170 2024-04-26T21:56:38.716 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.714+0000 7f676cd9d740 10 client.0 _ll_register_callbacks cb 0x5649b0dd0270 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:56:38.716 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.714+0000 7f6763e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5649b0e84cd0 0x5649b0e904f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:38.717 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.714+0000 7f6763e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5649b0e84cd0 0x5649b0e904f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:58846/0 (socket says 172.21.15.72:58846) 2024-04-26T21:56:38.717 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.714+0000 7f6763e00640 1 -- 172.21.15.72:0/1983688925 learned_addr learned my addr 172.21.15.72:0/1983688925 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:56:38.717 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.714+0000 7f6762a00640 1 --2- 172.21.15.72:0/1983688925 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5649b0e7fb60 0x5649b0e894b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:38.717 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.714+0000 7f6763400640 1 --2- 172.21.15.72:0/1983688925 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5649b0e7f170 0x5649b0e88f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:38.717 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.715+0000 7f6763e00640 1 -- 172.21.15.72:0/1983688925 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5649b0e7f170 msgr2=0x5649b0e88f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:56:38.717 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.715+0000 7f6763e00640 1 --2- 172.21.15.72:0/1983688925 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5649b0e7f170 0x5649b0e88f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:38.717 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.715+0000 7f6763e00640 1 -- 172.21.15.72:0/1983688925 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5649b0e7fb60 msgr2=0x5649b0e894b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:56:38.717 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.715+0000 7f6763e00640 1 --2- 172.21.15.72:0/1983688925 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5649b0e7fb60 0x5649b0e894b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:38.717 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.715+0000 7f6763e00640 1 -- 172.21.15.72:0/1983688925 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5649b0bb67d0 con 0x5649b0e84cd0 2024-04-26T21:56:38.719 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.715+0000 7f6763e00640 1 --2- 172.21.15.72:0/1983688925 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5649b0e84cd0 0x5649b0e904f0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f675c007970 tx=0x7f675c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=851126eb3de1d4f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:56:38.719 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.715+0000 7f674cc00640 10 client.0 ms_handle_connect on v2:172.21.15.72:3300/0 2024-04-26T21:56:38.719 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.716+0000 7f674cc00640 1 -- 172.21.15.72:0/1983688925 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f675c026680 con 0x5649b0e84cd0 2024-04-26T21:56:38.719 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.716+0000 7f676cd9d740 1 -- 172.21.15.72:0/1983688925 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6750003c80 con 0x5649b0e84cd0 2024-04-26T21:56:38.719 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.716+0000 7f674cc00640 1 -- 172.21.15.72:0/1983688925 <== mon.0 v2:172.21.15.72:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7f675c026820 con 0x5649b0e84cd0 2024-04-26T21:56:38.719 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.716+0000 7f674cc00640 1 -- 172.21.15.72:0/1983688925 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f675c02b020 con 0x5649b0e84cd0 2024-04-26T21:56:38.719 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.716+0000 7f676cd9d740 10 client.15819 fetch_fsmap learned FSMap version 27 2024-04-26T21:56:38.719 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.716+0000 7f676cd9d740 1 -- 172.21.15.72:0/1983688925 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5649b0aec3f0 con 0x5649b0e84cd0 2024-04-26T21:56:38.720 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.716+0000 7f674cc00640 1 -- 172.21.15.72:0/1983688925 <== mon.0 v2:172.21.15.72:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6750003c80 con 0x5649b0e84cd0 2024-04-26T21:56:38.720 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.717+0000 7f674cc00640 1 -- 172.21.15.72:0/1983688925 <== mon.0 v2:172.21.15.72:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f675c0308c0 con 0x5649b0e84cd0 2024-04-26T21:56:38.720 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.717+0000 7f676cd9d740 10 client.15819 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T21:56:38.720 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.717+0000 7f676cd9d740 10 client.15819 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:56:38.720 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.717+0000 7f676cd9d740 20 client.15819 populate_metadata read hostname 'smithi072' 2024-04-26T21:56:38.720 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.717+0000 7f676cd9d740 1 --2- 172.21.15.72:0/1983688925 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x5649b0e94ac0 0x5649b0e96f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:38.720 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.717+0000 7f676cd9d740 4 client.15819 mds_command: new command op to 34517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:56:38.720 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.717+0000 7f676cd9d740 1 -- 172.21.15.72:0/1983688925 --> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5649b0de87e0 con 0x5649b0e94ac0 2024-04-26T21:56:38.720 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.717+0000 7f6763400640 1 --2- 172.21.15.72:0/1983688925 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x5649b0e94ac0 0x5649b0e96f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:38.720 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.718+0000 7f6763400640 1 --2- 172.21.15.72:0/1983688925 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x5649b0e94ac0 0x5649b0e96f80 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5cfa960d9b39f005 server_cookie=a51b73e4463fe274 in_seq=0 out_seq=0 2024-04-26T21:56:38.720 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.718+0000 7f674cc00640 10 client.15819 ms_handle_connect on v2:172.21.15.76:6838/327145636 2024-04-26T21:56:38.721 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.720+0000 7f674cc00640 1 -- 172.21.15.72:0/1983688925 <== mds.0 v2:172.21.15.76:6838/327145636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5649b0de87e0 con 0x5649b0e94ac0 2024-04-26T21:56:38.721 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.720+0000 7f674cc00640 10 client.15819 handle_command_reply: tid=0 2024-04-26T21:56:38.721 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.720+0000 7f676cd9d740 1 client.15819 shutdown 2024-04-26T21:56:38.721 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.720+0000 7f676cd9d740 1 -- 172.21.15.72:0/1983688925 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5649b0e84cd0 msgr2=0x5649b0e904f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:38.722 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.720+0000 7f676cd9d740 1 --2- 172.21.15.72:0/1983688925 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5649b0e84cd0 0x5649b0e904f0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f675c007970 tx=0x7f675c004a40 comp rx=0 tx=0).stop 2024-04-26T21:56:38.722 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.720+0000 7f676cd9d740 1 -- 172.21.15.72:0/1983688925 shutdown_connections 2024-04-26T21:56:38.722 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.721+0000 7f676cd9d740 1 --2- 172.21.15.72:0/1983688925 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x5649b0e94ac0 0x5649b0e96f80 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:38.722 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.721+0000 7f676cd9d740 1 --2- 172.21.15.72:0/1983688925 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5649b0e84cd0 0x5649b0e904f0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:38.722 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.721+0000 7f676cd9d740 1 --2- 172.21.15.72:0/1983688925 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5649b0e7fb60 0x5649b0e894b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:38.722 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.721+0000 7f676cd9d740 1 --2- 172.21.15.72:0/1983688925 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5649b0e7f170 0x5649b0e88f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:38.722 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.721+0000 7f674cc00640 0 client.15819 ms_handle_reset on v2:172.21.15.76:6838/327145636 2024-04-26T21:56:38.722 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.721+0000 7f676cd9d740 1 -- 172.21.15.72:0/1983688925 >> 172.21.15.72:0/1983688925 conn(0x5649b0e71fd0 msgr2=0x5649b0e711f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:56:38.722 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.721+0000 7f676cd9d740 1 -- 172.21.15.72:0/1983688925 shutdown_connections 2024-04-26T21:56:38.722 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.721+0000 7f676cd9d740 1 -- 172.21.15.72:0/1983688925 wait complete. 2024-04-26T21:56:38.722 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.721+0000 7f676cd9d740 20 client.15819 trim_cache size 0 max 16384 2024-04-26T21:56:38.740 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.739+0000 7f676cd9d740 1 Processor -- start 2024-04-26T21:56:38.740 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.740+0000 7f676cd9d740 1 -- start start 2024-04-26T21:56:38.740 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.740+0000 7f676cd9d740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5649b0e7f170 0x5649b0e88f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:38.740 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.740+0000 7f676cd9d740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5649b0e7fb60 0x5649b0e904b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:38.740 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.740+0000 7f676cd9d740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5649b0e84cd0 0x5649b0e71070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:38.740 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.740+0000 7f676cd9d740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f67641a9580 con 0x5649b0e84cd0 2024-04-26T21:56:38.740 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.740+0000 7f676cd9d740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f67641a8110 con 0x5649b0e7fb60 2024-04-26T21:56:38.741 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.740+0000 7f676cd9d740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f67641a87e0 con 0x5649b0e7f170 2024-04-26T21:56:38.744 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.740+0000 7f6763e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5649b0e84cd0 0x5649b0e71070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:38.744 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.740+0000 7f6763e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5649b0e84cd0 0x5649b0e71070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:58852/0 (socket says 172.21.15.72:58852) 2024-04-26T21:56:38.744 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.740+0000 7f6763e00640 1 -- 172.21.15.72:0/3813312858 learned_addr learned my addr 172.21.15.72:0/3813312858 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:56:38.744 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.740+0000 7f6762a00640 1 --2- 172.21.15.72:0/3813312858 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5649b0e7fb60 0x5649b0e904b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:38.744 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.740+0000 7f6763400640 1 --2- 172.21.15.72:0/3813312858 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5649b0e7f170 0x5649b0e88f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:38.744 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.741+0000 7f6763e00640 1 -- 172.21.15.72:0/3813312858 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5649b0e7f170 msgr2=0x5649b0e88f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:56:38.744 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.741+0000 7f6763e00640 1 --2- 172.21.15.72:0/3813312858 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5649b0e7f170 0x5649b0e88f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:38.744 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.741+0000 7f6763e00640 1 -- 172.21.15.72:0/3813312858 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5649b0e7fb60 msgr2=0x5649b0e904b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:56:38.745 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.741+0000 7f6763e00640 1 --2- 172.21.15.72:0/3813312858 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5649b0e7fb60 0x5649b0e904b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:38.745 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.741+0000 7f6763e00640 1 -- 172.21.15.72:0/3813312858 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5649b0aac0f0 con 0x5649b0e84cd0 2024-04-26T21:56:38.745 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.741+0000 7f6763400640 1 --2- 172.21.15.72:0/3813312858 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5649b0e7f170 0x5649b0e88f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:56:38.745 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.741+0000 7f6763e00640 1 --2- 172.21.15.72:0/3813312858 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5649b0e84cd0 0x5649b0e71070 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f675c0076d0 tx=0x7f675c007cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed4764101edae226 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:56:38.745 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.741+0000 7f674f400640 1 -- 172.21.15.72:0/3813312858 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f675c03e070 con 0x5649b0e84cd0 2024-04-26T21:56:38.745 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.741+0000 7f674f400640 1 -- 172.21.15.72:0/3813312858 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f675c010e20 con 0x5649b0e84cd0 2024-04-26T21:56:38.745 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.742+0000 7f674f400640 1 -- 172.21.15.72:0/3813312858 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f675c026c10 con 0x5649b0e84cd0 2024-04-26T21:56:38.745 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.742+0000 7f676cd9d740 1 -- 172.21.15.72:0/3813312858 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5649b0e84cd0 msgr2=0x5649b0e71070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:38.745 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.742+0000 7f676cd9d740 1 --2- 172.21.15.72:0/3813312858 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5649b0e84cd0 0x5649b0e71070 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f675c0076d0 tx=0x7f675c007cb0 comp rx=0 tx=0).stop 2024-04-26T21:56:38.745 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.742+0000 7f676cd9d740 1 -- 172.21.15.72:0/3813312858 shutdown_connections 2024-04-26T21:56:38.745 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.742+0000 7f676cd9d740 1 --2- 172.21.15.72:0/3813312858 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5649b0e84cd0 0x5649b0e71070 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:38.745 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.742+0000 7f676cd9d740 1 --2- 172.21.15.72:0/3813312858 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5649b0e7fb60 0x5649b0e904b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:38.745 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.742+0000 7f676cd9d740 1 --2- 172.21.15.72:0/3813312858 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5649b0e7f170 0x5649b0e88f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:38.746 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.742+0000 7f676cd9d740 1 -- 172.21.15.72:0/3813312858 >> 172.21.15.72:0/3813312858 conn(0x5649b0e71fd0 msgr2=0x5649b0e70960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:56:38.746 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.742+0000 7f676cd9d740 1 -- 172.21.15.72:0/3813312858 shutdown_connections 2024-04-26T21:56:38.746 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.742+0000 7f676cd9d740 1 -- 172.21.15.72:0/3813312858 wait complete. 2024-04-26T21:56:38.746 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.743+0000 7f676cd9d740 1 -- start start 2024-04-26T21:56:38.746 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.743+0000 7f676cd9d740 1 Processor -- start 2024-04-26T21:56:38.746 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.743+0000 7f676cd9d740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5649b0e7f170 0x5649b0e8de40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:38.746 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.743+0000 7f676cd9d740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5649b0e7fb60 0x5649b0e8e380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:38.746 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.743+0000 7f676cd9d740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5649b0e84cd0 0x5649b0e8f390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:38.746 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.743+0000 7f676cd9d740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f67641a5a20 con 0x5649b0e84cd0 2024-04-26T21:56:38.746 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.743+0000 7f676cd9d740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f67641a8eb0 con 0x5649b0e7f170 2024-04-26T21:56:38.746 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.743+0000 7f676cd9d740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f67641a6090 con 0x5649b0e7fb60 2024-04-26T21:56:38.746 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.744+0000 7f676cd9d740 10 client.0 _ll_register_callbacks cb 0x7f675c007ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:56:38.747 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.744+0000 7f6763e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5649b0e84cd0 0x5649b0e8f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:38.747 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.744+0000 7f6763e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5649b0e84cd0 0x5649b0e8f390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:58858/0 (socket says 172.21.15.72:58858) 2024-04-26T21:56:38.747 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.744+0000 7f6763e00640 1 -- 172.21.15.72:0/3505314201 learned_addr learned my addr 172.21.15.72:0/3505314201 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:56:38.747 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.744+0000 7f6763400640 1 --2- 172.21.15.72:0/3505314201 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5649b0e7f170 0x5649b0e8de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:38.749 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.744+0000 7f6763e00640 1 -- 172.21.15.72:0/3505314201 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5649b0e7fb60 msgr2=0x5649b0e8e380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:56:38.749 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.744+0000 7f6763e00640 1 --2- 172.21.15.72:0/3505314201 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5649b0e7fb60 0x5649b0e8e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:38.749 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.744+0000 7f6763e00640 1 -- 172.21.15.72:0/3505314201 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5649b0e7f170 msgr2=0x5649b0e8de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:56:38.750 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.744+0000 7f6763e00640 1 --2- 172.21.15.72:0/3505314201 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5649b0e7f170 0x5649b0e8de40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:38.750 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.744+0000 7f6763e00640 1 -- 172.21.15.72:0/3505314201 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5649b0da2a30 con 0x5649b0e84cd0 2024-04-26T21:56:38.750 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.744+0000 7f6763400640 1 --2- 172.21.15.72:0/3505314201 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5649b0e7f170 0x5649b0e8de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:56:38.750 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.745+0000 7f6763e00640 1 --2- 172.21.15.72:0/3505314201 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5649b0e84cd0 0x5649b0e8f390 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f675c008010 tx=0x7f675c004d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9175d29556d3c077 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:56:38.750 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.745+0000 7f674cc00640 10 client.0 ms_handle_connect on v2:172.21.15.72:3300/0 2024-04-26T21:56:38.750 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.745+0000 7f674cc00640 1 -- 172.21.15.72:0/3505314201 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f675c046060 con 0x5649b0e84cd0 2024-04-26T21:56:38.750 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.745+0000 7f676cd9d740 1 -- 172.21.15.72:0/3505314201 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f675c010e20 con 0x5649b0e84cd0 2024-04-26T21:56:38.750 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.745+0000 7f674cc00640 1 -- 172.21.15.72:0/3505314201 <== mon.0 v2:172.21.15.72:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7f675c026da0 con 0x5649b0e84cd0 2024-04-26T21:56:38.750 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.745+0000 7f674cc00640 1 -- 172.21.15.72:0/3505314201 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f675c03e070 con 0x5649b0e84cd0 2024-04-26T21:56:38.750 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.745+0000 7f676cd9d740 10 client.15825 fetch_fsmap learned FSMap version 27 2024-04-26T21:56:38.750 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.745+0000 7f676cd9d740 1 -- 172.21.15.72:0/3505314201 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5649b0efb0c0 con 0x5649b0e84cd0 2024-04-26T21:56:38.750 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.746+0000 7f674cc00640 1 -- 172.21.15.72:0/3505314201 <== mon.0 v2:172.21.15.72:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f675c010e20 con 0x5649b0e84cd0 2024-04-26T21:56:38.750 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.746+0000 7f674cc00640 1 -- 172.21.15.72:0/3505314201 <== mon.0 v2:172.21.15.72:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f675c0052d0 con 0x5649b0e84cd0 2024-04-26T21:56:38.751 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.746+0000 7f676cd9d740 10 client.15825 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T21:56:38.751 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.746+0000 7f676cd9d740 10 client.15825 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:56:38.751 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.746+0000 7f676cd9d740 20 client.15825 populate_metadata read hostname 'smithi072' 2024-04-26T21:56:38.751 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.746+0000 7f676cd9d740 1 --2- 172.21.15.72:0/3505314201 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x5649b0efb920 0x5649b0efdd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:38.751 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.746+0000 7f676cd9d740 4 client.15825 mds_command: new command op to 34517 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-26T21:56:38.751 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.746+0000 7f676cd9d740 1 -- 172.21.15.72:0/3505314201 --> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5649b0de2650 con 0x5649b0efb920 2024-04-26T21:56:38.751 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.747+0000 7f6763400640 1 --2- 172.21.15.72:0/3505314201 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x5649b0efb920 0x5649b0efdd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:38.751 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.747+0000 7f6763400640 1 --2- 172.21.15.72:0/3505314201 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x5649b0efb920 0x5649b0efdd40 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=38ed4febe47f9be9 server_cookie=bdbea2c5e6692f0e in_seq=0 out_seq=0 2024-04-26T21:56:38.751 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.747+0000 7f674cc00640 10 client.15825 ms_handle_connect on v2:172.21.15.76:6838/327145636 2024-04-26T21:56:38.751 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.748+0000 7f674cc00640 1 -- 172.21.15.72:0/3505314201 <== mds.0 v2:172.21.15.76:6838/327145636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5649b0de2650 con 0x5649b0efb920 2024-04-26T21:56:38.751 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.748+0000 7f674cc00640 10 client.15825 handle_command_reply: tid=0 2024-04-26T21:56:38.751 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.748+0000 7f676cd9d740 1 client.15825 shutdown 2024-04-26T21:56:38.751 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.748+0000 7f676cd9d740 1 -- 172.21.15.72:0/3505314201 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5649b0e84cd0 msgr2=0x5649b0e8f390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:38.751 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.748+0000 7f676cd9d740 1 --2- 172.21.15.72:0/3505314201 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5649b0e84cd0 0x5649b0e8f390 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f675c008010 tx=0x7f675c004d90 comp rx=0 tx=0).stop 2024-04-26T21:56:38.752 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.748+0000 7f676cd9d740 1 -- 172.21.15.72:0/3505314201 shutdown_connections 2024-04-26T21:56:38.752 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.748+0000 7f676cd9d740 1 --2- 172.21.15.72:0/3505314201 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x5649b0efb920 0x5649b0efdd40 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:38.752 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.748+0000 7f676cd9d740 1 --2- 172.21.15.72:0/3505314201 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5649b0e84cd0 0x5649b0e8f390 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:38.752 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.748+0000 7f676cd9d740 1 --2- 172.21.15.72:0/3505314201 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5649b0e7fb60 0x5649b0e8e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:38.752 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.748+0000 7f676cd9d740 1 --2- 172.21.15.72:0/3505314201 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5649b0e7f170 0x5649b0e8de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:38.752 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.748+0000 7f676cd9d740 1 -- 172.21.15.72:0/3505314201 >> 172.21.15.72:0/3505314201 conn(0x5649b0e71fd0 msgr2=0x5649b0e711f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:56:38.752 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.748+0000 7f674cc00640 0 client.15825 ms_handle_reset on v2:172.21.15.76:6838/327145636 2024-04-26T21:56:38.752 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.748+0000 7f676cd9d740 1 -- 172.21.15.72:0/3505314201 shutdown_connections 2024-04-26T21:56:38.752 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.748+0000 7f676cd9d740 1 -- 172.21.15.72:0/3505314201 wait complete. 2024-04-26T21:56:38.752 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.749+0000 7f676cd9d740 20 client.15825 trim_cache size 0 max 16384 2024-04-26T21:56:38.752 INFO:teuthology.orchestra.run.smithi072.stdout:{ 2024-04-26T21:56:38.753 INFO:teuthology.orchestra.run.smithi072.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-26T21:56:38.753 INFO:teuthology.orchestra.run.smithi072.stdout: "scrubs": { 2024-04-26T21:56:38.753 INFO:teuthology.orchestra.run.smithi072.stdout: "9fc595cc-50bd-4ca7-82b5-d50ff9582a99": { 2024-04-26T21:56:38.753 INFO:teuthology.orchestra.run.smithi072.stdout: "path": "/", 2024-04-26T21:56:38.753 INFO:teuthology.orchestra.run.smithi072.stdout: "tag": "9fc595cc-50bd-4ca7-82b5-d50ff9582a99", 2024-04-26T21:56:38.753 INFO:teuthology.orchestra.run.smithi072.stdout: "options": "recursive,force" 2024-04-26T21:56:38.753 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:56:38.753 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:56:38.753 INFO:teuthology.orchestra.run.smithi072.stdout:} 2024-04-26T21:56:38.754 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.749+0000 7f6768a00640 1 -- 172.21.15.72:0/1471514208 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f67480a3cc0 msgr2=0x7f67480a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:38.754 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.749+0000 7f6768a00640 1 --2- 172.21.15.72:0/1471514208 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f67480a3cc0 0x7f67480a6180 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x5649b0e803c0 tx=0x7f675c007520 comp rx=0 tx=0).stop 2024-04-26T21:56:38.754 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.749+0000 7f6768a00640 1 -- 172.21.15.72:0/1471514208 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6764101110 msgr2=0x7f676419ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:38.754 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.749+0000 7f6768a00640 1 --2- 172.21.15.72:0/1471514208 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6764101110 0x7f676419ab00 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f675400c970 tx=0x7f675400ce30 comp rx=0 tx=0).stop 2024-04-26T21:56:38.754 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.749+0000 7f6768a00640 1 -- 172.21.15.72:0/1471514208 shutdown_connections 2024-04-26T21:56:38.754 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.749+0000 7f6768a00640 1 --2- 172.21.15.72:0/1471514208 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f67480a3cc0 0x7f67480a6180 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:38.754 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.749+0000 7f6768a00640 1 --2- 172.21.15.72:0/1471514208 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6764102410 0x7f67641a20b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:38.755 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.749+0000 7f6768a00640 1 --2- 172.21.15.72:0/1471514208 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6764101a70 0x7f676419b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:38.755 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.749+0000 7f6768a00640 1 --2- 172.21.15.72:0/1471514208 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6764101110 0x7f676419ab00 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:38.755 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.749+0000 7f6768a00640 1 -- 172.21.15.72:0/1471514208 >> 172.21.15.72:0/1471514208 conn(0x7f67640fccd0 msgr2=0x7f67640fe3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:56:38.755 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.750+0000 7f6768a00640 1 -- 172.21.15.72:0/1471514208 shutdown_connections 2024-04-26T21:56:38.755 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:56:38.750+0000 7f6768a00640 1 -- 172.21.15.72:0/1471514208 wait complete. 2024-04-26T21:56:38.766 INFO:tasks.cephfs.filesystem:scrub status for tag:9fc595cc-50bd-4ca7-82b5-d50ff9582a99 - {'path': '/', 'tag': '9fc595cc-50bd-4ca7-82b5-d50ff9582a99', 'options': 'recursive,force'} 2024-04-26T21:56:39.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:56:38 smithi076 ceph-mon[31823]: pgmap v485: 97 pgs: 97 active+clean; 650 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:56:39.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:56:38 smithi076 ceph-mon[31823]: scrub queued for path: / 2024-04-26T21:56:39.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:56:38 smithi076 ceph-mon[31823]: scrub summary: idle+waiting paths [/] 2024-04-26T21:56:39.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:56:38 smithi076 ceph-mon[31823]: scrub summary: active paths [/] 2024-04-26T21:56:39.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:56:38 smithi076 ceph-mon[31823]: scrub summary: idle+waiting paths [/] 2024-04-26T21:56:39.321 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:56:38 smithi072 ceph-mon[28138]: pgmap v485: 97 pgs: 97 active+clean; 650 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:56:39.321 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:56:38 smithi072 ceph-mon[28138]: scrub queued for path: / 2024-04-26T21:56:39.321 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:56:38 smithi072 ceph-mon[28138]: scrub summary: idle+waiting paths [/] 2024-04-26T21:56:39.321 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:56:38 smithi072 ceph-mon[28138]: scrub summary: active paths [/] 2024-04-26T21:56:39.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:56:38 smithi072 ceph-mon[28138]: scrub summary: idle+waiting paths [/] 2024-04-26T21:56:39.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:56:38 smithi164 ceph-mon[31916]: pgmap v485: 97 pgs: 97 active+clean; 650 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:56:39.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:56:38 smithi164 ceph-mon[31916]: scrub queued for path: / 2024-04-26T21:56:39.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:56:38 smithi164 ceph-mon[31916]: scrub summary: idle+waiting paths [/] 2024-04-26T21:56:39.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:56:38 smithi164 ceph-mon[31916]: scrub summary: active paths [/] 2024-04-26T21:56:39.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:56:38 smithi164 ceph-mon[31916]: scrub summary: idle+waiting paths [/] 2024-04-26T21:56:40.317 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:56:39 smithi076 ceph-mon[31823]: scrub summary: idle 2024-04-26T21:56:40.321 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:56:39 smithi072 ceph-mon[28138]: scrub summary: idle 2024-04-26T21:56:40.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:56:39 smithi164 ceph-mon[31916]: scrub summary: idle 2024-04-26T21:56:41.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:56:40 smithi076 ceph-mon[31823]: pgmap v486: 97 pgs: 97 active+clean; 650 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-04-26T21:56:41.321 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:56:40 smithi072 ceph-mon[28138]: pgmap v486: 97 pgs: 97 active+clean; 650 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-04-26T21:56:41.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:56:40 smithi164 ceph-mon[31916]: pgmap v486: 97 pgs: 97 active+clean; 650 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-04-26T21:56:43.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:56:42 smithi076 ceph-mon[31823]: pgmap v487: 97 pgs: 97 active+clean; 650 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-26T21:56:43.321 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:56:42 smithi072 ceph-mon[28138]: pgmap v487: 97 pgs: 97 active+clean; 650 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-26T21:56:43.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:56:42 smithi164 ceph-mon[31916]: pgmap v487: 97 pgs: 97 active+clean; 650 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-26T21:56:45.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:56:44 smithi076 ceph-mon[31823]: pgmap v488: 97 pgs: 97 active+clean; 650 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 9 op/s 2024-04-26T21:56:45.321 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:56:44 smithi072 ceph-mon[28138]: pgmap v488: 97 pgs: 97 active+clean; 650 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 9 op/s 2024-04-26T21:56:45.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:56:44 smithi164 ceph-mon[31916]: pgmap v488: 97 pgs: 97 active+clean; 650 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 9 op/s 2024-04-26T21:56:47.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:56:46 smithi076 ceph-mon[31823]: pgmap v489: 97 pgs: 97 active+clean; 650 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 11 op/s 2024-04-26T21:56:47.321 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:56:46 smithi072 ceph-mon[28138]: pgmap v489: 97 pgs: 97 active+clean; 650 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 11 op/s 2024-04-26T21:56:47.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:56:46 smithi164 ceph-mon[31916]: pgmap v489: 97 pgs: 97 active+clean; 650 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 11 op/s 2024-04-26T21:56:49.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:56:48 smithi076 ceph-mon[31823]: pgmap v490: 97 pgs: 97 active+clean; 650 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-26T21:56:49.321 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:56:48 smithi072 ceph-mon[28138]: pgmap v490: 97 pgs: 97 active+clean; 650 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-26T21:56:49.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:56:48 smithi164 ceph-mon[31916]: pgmap v490: 97 pgs: 97 active+clean; 650 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-26T21:56:51.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:56:51 smithi076 ceph-mon[31823]: pgmap v491: 97 pgs: 97 active+clean; 650 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-04-26T21:56:51.321 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:56:51 smithi072 ceph-mon[28138]: pgmap v491: 97 pgs: 97 active+clean; 650 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-04-26T21:56:51.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:56:51 smithi164 ceph-mon[31916]: pgmap v491: 97 pgs: 97 active+clean; 650 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-04-26T21:56:53.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:56:53 smithi076 ceph-mon[31823]: pgmap v492: 97 pgs: 97 active+clean; 650 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:56:53.321 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:56:53 smithi072 ceph-mon[28138]: pgmap v492: 97 pgs: 97 active+clean; 650 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:56:53.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:56:53 smithi164 ceph-mon[31916]: pgmap v492: 97 pgs: 97 active+clean; 650 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:56:55.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:56:55 smithi076 ceph-mon[31823]: pgmap v493: 97 pgs: 97 active+clean; 650 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-26T21:56:55.321 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:56:55 smithi072 ceph-mon[28138]: pgmap v493: 97 pgs: 97 active+clean; 650 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-26T21:56:55.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:56:55 smithi164 ceph-mon[31916]: pgmap v493: 97 pgs: 97 active+clean; 650 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-26T21:56:57.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:56:57 smithi076 ceph-mon[31823]: pgmap v494: 97 pgs: 97 active+clean; 650 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-26T21:56:57.321 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:56:57 smithi072 ceph-mon[28138]: pgmap v494: 97 pgs: 97 active+clean; 650 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-26T21:56:57.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:56:57 smithi164 ceph-mon[31916]: pgmap v494: 97 pgs: 97 active+clean; 650 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-26T21:56:59.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:56:59 smithi076 ceph-mon[31823]: pgmap v495: 97 pgs: 97 active+clean; 650 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:56:59.321 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:56:59 smithi072 ceph-mon[28138]: pgmap v495: 97 pgs: 97 active+clean; 650 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:56:59.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:56:59 smithi164 ceph-mon[31916]: pgmap v495: 97 pgs: 97 active+clean; 650 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:57:00.917 INFO:tasks.workunit.client.0.smithi072.stderr:Updating files: 62% (7711/12271) Updating files: 63% (7731/12271) Updating files: 64% (7854/12271) Updating files: 65% (7977/12271) Updating files: 66% (8099/12271) Updating files: 67% (8222/12271) Updating files: 68% (8345/12271) Updating files: 69% (8467/12271) Updating files: 70% (8590/12271) Updating files: 71% (8713/12271) Updating files: 72% (8836/12271) Updating files: 73% (8958/12271) Updating files: 74% (9081/12271) Updating files: 75% (9204/12271) Updating files: 76% (9326/12271) Updating files: 77% (9449/12271) Updating files: 78% (9572/12271) Updating files: 79% (9695/12271) Updating files: 80% (9817/12271) Updating files: 81% (9940/12271) Updating files: 82% (10063/12271) Updating files: 83% (10185/12271) Updating files: 84% (10308/12271) Updating files: 85% (10431/12271) Updating files: 86% (10554/12271) Updating files: 87% (10676/12271) Updating files: 88% (10799/12271) Updating files: 89% (10922/12271) Updating files: 90% (11044/12271) Updating files: 91% (11167/12271) Updating files: 92% (11290/12271) Updating files: 93% (11413/12271) Updating files: 94% (11535/12271) Updating files: 95% (11658/12271) Updating files: 96% (11781/12271) Updating files: 97% (11903/12271) Updating files: 98% (12026/12271) Updating files: 99% (12149/12271) Updating files: 100% (12271/12271) Updating files: 100% (12271/12271), done. 2024-04-26T21:57:01.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:57:01 smithi076 ceph-mon[31823]: pgmap v496: 97 pgs: 97 active+clean; 650 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-26T21:57:01.321 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:57:01 smithi072 ceph-mon[28138]: pgmap v496: 97 pgs: 97 active+clean; 650 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-26T21:57:01.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:57:01 smithi164 ceph-mon[31916]: pgmap v496: 97 pgs: 97 active+clean; 650 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-26T21:57:01.919 INFO:tasks.workunit.client.0.smithi072.stderr:Note: switching to 'bc612ffb9c1d5bc27536bc917a705262a36d4387'. 2024-04-26T21:57:01.919 INFO:tasks.workunit.client.0.smithi072.stderr: 2024-04-26T21:57:01.919 INFO:tasks.workunit.client.0.smithi072.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-04-26T21:57:01.920 INFO:tasks.workunit.client.0.smithi072.stderr:changes and commit them, and you can discard any commits you make in this 2024-04-26T21:57:01.920 INFO:tasks.workunit.client.0.smithi072.stderr:state without impacting any branches by switching back to a branch. 2024-04-26T21:57:01.920 INFO:tasks.workunit.client.0.smithi072.stderr: 2024-04-26T21:57:01.920 INFO:tasks.workunit.client.0.smithi072.stderr:If you want to create a new branch to retain commits you create, you may 2024-04-26T21:57:01.920 INFO:tasks.workunit.client.0.smithi072.stderr:do so (now or later) by using -c with the switch command. Example: 2024-04-26T21:57:01.920 INFO:tasks.workunit.client.0.smithi072.stderr: 2024-04-26T21:57:01.921 INFO:tasks.workunit.client.0.smithi072.stderr: git switch -c 2024-04-26T21:57:01.921 INFO:tasks.workunit.client.0.smithi072.stderr: 2024-04-26T21:57:01.921 INFO:tasks.workunit.client.0.smithi072.stderr:Or undo this operation with: 2024-04-26T21:57:01.921 INFO:tasks.workunit.client.0.smithi072.stderr: 2024-04-26T21:57:01.921 INFO:tasks.workunit.client.0.smithi072.stderr: git switch - 2024-04-26T21:57:01.921 INFO:tasks.workunit.client.0.smithi072.stderr: 2024-04-26T21:57:01.921 INFO:tasks.workunit.client.0.smithi072.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-04-26T21:57:01.922 INFO:tasks.workunit.client.0.smithi072.stderr: 2024-04-26T21:57:01.922 INFO:tasks.workunit.client.0.smithi072.stderr:HEAD is now at bc612ffb9c1 Merge PR #55144 into wip-rishabh-testing-20240426.111959 2024-04-26T21:57:01.934 DEBUG:teuthology.orchestra.run.smithi072:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-04-26T21:57:01.996 INFO:tasks.workunit.client.0.smithi072.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-04-26T21:57:02.003 INFO:tasks.workunit.client.0.smithi072.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-26T21:57:02.004 INFO:tasks.workunit.client.0.smithi072.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-04-26T21:57:03.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:57:03 smithi076 ceph-mon[31823]: pgmap v497: 97 pgs: 97 active+clean; 650 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:57:03.321 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:57:03 smithi072 ceph-mon[28138]: pgmap v497: 97 pgs: 97 active+clean; 650 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:57:03.328 INFO:tasks.workunit.client.0.smithi072.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-04-26T21:57:03.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:57:03 smithi164 ceph-mon[31916]: pgmap v497: 97 pgs: 97 active+clean; 650 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:57:03.458 INFO:tasks.workunit.client.0.smithi072.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-04-26T21:57:03.539 INFO:tasks.workunit.client.0.smithi072.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-26T21:57:03.544 INFO:tasks.workunit.client.0.smithi072.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-26T21:57:03.544 INFO:tasks.workunit.client.0.smithi072.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-04-26T21:57:03.755 INFO:tasks.workunit.client.0.smithi072.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-26T21:57:03.766 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2024-04-26T21:57:03.766 DEBUG:teuthology.orchestra.run.smithi072:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-04-26T21:57:03.819 INFO:tasks.workunit:Running workunits matching suites/fsync-tester.sh on client.0... 2024-04-26T21:57:03.820 INFO:tasks.workunit:Running workunit suites/fsync-tester.sh... 2024-04-26T21:57:03.820 DEBUG:teuthology.orchestra.run.smithi072:workunit test suites/fsync-tester.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=bc612ffb9c1d5bc27536bc917a705262a36d4387 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsync-tester.sh 2024-04-26T21:57:03.910 INFO:tasks.workunit.client.0.smithi072.stderr:+ wget http://download.ceph.com/qa/fsync-tester.c 2024-04-26T21:57:03.950 INFO:tasks.workunit.client.0.smithi072.stderr:--2024-04-26 21:57:03-- http://download.ceph.com/qa/fsync-tester.c 2024-04-26T21:57:03.952 INFO:tasks.workunit.client.0.smithi072.stderr:Resolving download.ceph.com (download.ceph.com)... 158.69.68.124, 2607:5300:201:2000::3:58a1 2024-04-26T21:57:03.988 INFO:tasks.workunit.client.0.smithi072.stderr:Connecting to download.ceph.com (download.ceph.com)|158.69.68.124|:80... connected. 2024-04-26T21:57:04.030 INFO:tasks.workunit.client.0.smithi072.stderr:HTTP request sent, awaiting response... 200 OK 2024-04-26T21:57:04.031 INFO:tasks.workunit.client.0.smithi072.stderr:Length: 3024 (3.0K) [application/octet-stream] 2024-04-26T21:57:04.040 INFO:tasks.workunit.client.0.smithi072.stderr:Saving to: ‘fsync-tester.c’ 2024-04-26T21:57:04.049 INFO:tasks.workunit.client.0.smithi072.stderr: 2024-04-26T21:57:04.049 INFO:tasks.workunit.client.0.smithi072.stderr: 0K .. 100% 115M=0s 2024-04-26T21:57:04.050 INFO:tasks.workunit.client.0.smithi072.stderr: 2024-04-26T21:57:04.052 INFO:tasks.workunit.client.0.smithi072.stderr:2024-04-26 21:57:04 (115 MB/s) - ‘fsync-tester.c’ saved [3024/3024] 2024-04-26T21:57:04.053 INFO:tasks.workunit.client.0.smithi072.stderr: 2024-04-26T21:57:04.054 INFO:tasks.workunit.client.0.smithi072.stderr:+ gcc -D_GNU_SOURCE fsync-tester.c -o fsync-tester 2024-04-26T21:57:04.278 INFO:tasks.workunit.client.0.smithi072.stderr:+ ./fsync-tester 2024-04-26T21:57:04.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:57:04 smithi164 ceph-mon[31916]: pgmap v498: 97 pgs: 97 active+clean; 650 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-26T21:57:04.564 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:57:04 smithi076 ceph-mon[31823]: pgmap v498: 97 pgs: 97 active+clean; 650 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-26T21:57:04.571 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:57:04 smithi072 ceph-mon[28138]: pgmap v498: 97 pgs: 97 active+clean; 650 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-26T21:57:07.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:57:06 smithi164 ceph-mon[31916]: pgmap v499: 97 pgs: 97 active+clean; 668 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-04-26T21:57:07.268 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:57:06 smithi076 ceph-mon[31823]: pgmap v499: 97 pgs: 97 active+clean; 668 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-04-26T21:57:07.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:57:06 smithi072 ceph-mon[28138]: pgmap v499: 97 pgs: 97 active+clean; 668 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-04-26T21:57:08.767 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-26T21:57:08.767 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-26T21:57:09.004 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.004+0000 7f5f8ee00640 1 -- 172.21.15.72:0/23921146 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5f88103320 msgr2=0x7f5f88105710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:09.004 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.004+0000 7f5f8ee00640 1 --2- 172.21.15.72:0/23921146 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5f88103320 0x7f5f88105710 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f5f7c009900 tx=0x7f5f7c02a410 comp rx=0 tx=0).stop 2024-04-26T21:57:09.005 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.004+0000 7f5f8ee00640 1 -- 172.21.15.72:0/23921146 shutdown_connections 2024-04-26T21:57:09.005 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.004+0000 7f5f8ee00640 1 --2- 172.21.15.72:0/23921146 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5f88108580 0x7f5f8810a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:09.005 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.004+0000 7f5f8ee00640 1 --2- 172.21.15.72:0/23921146 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5f88105c50 0x7f5f88108040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:09.005 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.004+0000 7f5f8ee00640 1 --2- 172.21.15.72:0/23921146 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5f88103320 0x7f5f88105710 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:09.005 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.004+0000 7f5f8ee00640 1 -- 172.21.15.72:0/23921146 >> 172.21.15.72:0/23921146 conn(0x7f5f880fccd0 msgr2=0x7f5f880ff0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:57:09.005 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.004+0000 7f5f8ee00640 1 -- 172.21.15.72:0/23921146 shutdown_connections 2024-04-26T21:57:09.005 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.004+0000 7f5f8ee00640 1 -- 172.21.15.72:0/23921146 wait complete. 2024-04-26T21:57:09.005 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.006+0000 7f5f8ee00640 1 Processor -- start 2024-04-26T21:57:09.006 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.006+0000 7f5f8ee00640 1 -- start start 2024-04-26T21:57:09.007 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.006+0000 7f5f8ee00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5f88103320 0x7f5f88198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:09.007 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.006+0000 7f5f8ee00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5f88105c50 0x7f5f88198e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:09.007 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.006+0000 7f5f8ee00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5f88108580 0x7f5f8819fee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:09.007 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.006+0000 7f5f8ee00640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f5f8810e290 con 0x7f5f88105c50 2024-04-26T21:57:09.008 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.006+0000 7f5f8ee00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f5f8810e110 con 0x7f5f88108580 2024-04-26T21:57:09.008 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.006+0000 7f5f8ee00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f5f8810e410 con 0x7f5f88103320 2024-04-26T21:57:09.008 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.007+0000 7f5f8d000640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5f88105c50 0x7f5f88198e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:09.008 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.007+0000 7f5f8d000640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5f88105c50 0x7f5f88198e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:48238/0 (socket says 172.21.15.72:48238) 2024-04-26T21:57:09.008 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.007+0000 7f5f8d000640 1 -- 172.21.15.72:0/3692370053 learned_addr learned my addr 172.21.15.72:0/3692370053 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:57:09.008 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.007+0000 7f5f8da00640 1 --2- 172.21.15.72:0/3692370053 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5f88103320 0x7f5f88198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:09.008 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.007+0000 7f5f8e400640 1 --2- 172.21.15.72:0/3692370053 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5f88108580 0x7f5f8819fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:09.008 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.007+0000 7f5f8d000640 1 -- 172.21.15.72:0/3692370053 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5f88103320 msgr2=0x7f5f88198930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:09.008 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.007+0000 7f5f8d000640 1 --2- 172.21.15.72:0/3692370053 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5f88103320 0x7f5f88198930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:09.008 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.007+0000 7f5f8d000640 1 -- 172.21.15.72:0/3692370053 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5f88108580 msgr2=0x7f5f8819fee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:09.008 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.007+0000 7f5f8d000640 1 --2- 172.21.15.72:0/3692370053 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5f88108580 0x7f5f8819fee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:09.008 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.007+0000 7f5f8d000640 1 -- 172.21.15.72:0/3692370053 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f881a0550 con 0x7f5f88105c50 2024-04-26T21:57:09.008 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.007+0000 7f5f8d000640 1 --2- 172.21.15.72:0/3692370053 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5f88105c50 0x7f5f88198e70 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f5f7400c970 tx=0x7f5f7400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:57:09.009 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.008+0000 7f5f86000640 1 -- 172.21.15.72:0/3692370053 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f74014070 con 0x7f5f88105c50 2024-04-26T21:57:09.009 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.008+0000 7f5f86000640 1 -- 172.21.15.72:0/3692370053 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5f74004440 con 0x7f5f88105c50 2024-04-26T21:57:09.009 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.008+0000 7f5f8ee00640 1 -- 172.21.15.72:0/3692370053 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f881a0840 con 0x7f5f88105c50 2024-04-26T21:57:09.009 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.008+0000 7f5f8ee00640 1 -- 172.21.15.72:0/3692370053 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f881a0d90 con 0x7f5f88105c50 2024-04-26T21:57:09.009 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.008+0000 7f5f86000640 1 -- 172.21.15.72:0/3692370053 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f74003ce0 con 0x7f5f88105c50 2024-04-26T21:57:09.012 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.012+0000 7f5f9339d740 1 Processor -- start 2024-04-26T21:57:09.012 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.012+0000 7f5f9339d740 1 -- start start 2024-04-26T21:57:09.013 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.013+0000 7f5f9339d740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bb8324f960 0x55bb8324fd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:09.013 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.013+0000 7f5f86000640 1 -- 172.21.15.72:0/3692370053 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f5f74017070 con 0x7f5f88105c50 2024-04-26T21:57:09.013 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.013+0000 7f5f9339d740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55bb83250350 0x55bb83254e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:09.013 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.013+0000 7f5f9339d740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bb832554c0 0x55bb83257950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:09.013 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.013+0000 7f5f9339d740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55bb83160030 con 0x55bb83250350 2024-04-26T21:57:09.013 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.013+0000 7f5f9339d740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55bb8317b7f0 con 0x55bb8324f960 2024-04-26T21:57:09.013 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.013+0000 7f5f9339d740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55bb831367e0 con 0x55bb832554c0 2024-04-26T21:57:09.015 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.013+0000 7f5f8da00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55bb83250350 0x55bb83254e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:09.015 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.013+0000 7f5f8da00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55bb83250350 0x55bb83254e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:48246/0 (socket says 172.21.15.72:48246) 2024-04-26T21:57:09.015 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.013+0000 7f5f8da00640 1 -- 172.21.15.72:0/566257847 learned_addr learned my addr 172.21.15.72:0/566257847 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:57:09.015 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.013+0000 7f5f8e400640 1 --2- 172.21.15.72:0/566257847 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bb8324f960 0x55bb8324fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:09.015 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.013+0000 7f5f8d000640 1 --2- 172.21.15.72:0/566257847 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bb832554c0 0x55bb83257950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:09.015 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.014+0000 7f5f8da00640 1 -- 172.21.15.72:0/566257847 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bb832554c0 msgr2=0x55bb83257950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:09.015 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.014+0000 7f5f8da00640 1 --2- 172.21.15.72:0/566257847 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bb832554c0 0x55bb83257950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:09.015 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.014+0000 7f5f8da00640 1 -- 172.21.15.72:0/566257847 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bb8324f960 msgr2=0x55bb8324fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:09.015 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.014+0000 7f5f8da00640 1 --2- 172.21.15.72:0/566257847 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bb8324f960 0x55bb8324fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:09.015 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.014+0000 7f5f8da00640 1 -- 172.21.15.72:0/566257847 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb83142050 con 0x55bb83250350 2024-04-26T21:57:09.015 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.014+0000 7f5f8da00640 1 --2- 172.21.15.72:0/566257847 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55bb83250350 0x55bb83254e50 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f5f7c031460 tx=0x7f5f7c004390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62b918b64127f41e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:57:09.015 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.014+0000 7f5f7aa00640 1 -- 172.21.15.72:0/566257847 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f7c035030 con 0x55bb83250350 2024-04-26T21:57:09.015 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.015+0000 7f5f7aa00640 1 -- 172.21.15.72:0/566257847 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5f7c003c80 con 0x55bb83250350 2024-04-26T21:57:09.015 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.015+0000 7f5f7aa00640 1 -- 172.21.15.72:0/566257847 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f7c031b60 con 0x55bb83250350 2024-04-26T21:57:09.016 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.015+0000 7f5f86000640 1 --2- 172.21.15.72:0/3692370053 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f5f700a3c40 0x7f5f700a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:09.018 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.015+0000 7f5f86000640 1 -- 172.21.15.72:0/3692370053 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7f5f740d05c0 con 0x7f5f88105c50 2024-04-26T21:57:09.018 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.015+0000 7f5f8e400640 1 --2- 172.21.15.72:0/3692370053 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f5f700a3c40 0x7f5f700a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:09.019 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.015+0000 7f5f9339d740 1 -- 172.21.15.72:0/566257847 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55bb83250350 msgr2=0x55bb83254e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:09.019 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.015+0000 7f5f9339d740 1 --2- 172.21.15.72:0/566257847 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55bb83250350 0x55bb83254e50 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f5f7c031460 tx=0x7f5f7c004390 comp rx=0 tx=0).stop 2024-04-26T21:57:09.019 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.016+0000 7f5f9339d740 1 -- 172.21.15.72:0/566257847 shutdown_connections 2024-04-26T21:57:09.019 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.016+0000 7f5f9339d740 1 --2- 172.21.15.72:0/566257847 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bb832554c0 0x55bb83257950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:09.019 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.016+0000 7f5f9339d740 1 --2- 172.21.15.72:0/566257847 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55bb83250350 0x55bb83254e50 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:09.019 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.016+0000 7f5f9339d740 1 --2- 172.21.15.72:0/566257847 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bb8324f960 0x55bb8324fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:09.019 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.016+0000 7f5f9339d740 1 -- 172.21.15.72:0/566257847 >> 172.21.15.72:0/566257847 conn(0x55bb8324afd0 msgr2=0x55bb8324b3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:57:09.019 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.016+0000 7f5f9339d740 1 -- 172.21.15.72:0/566257847 shutdown_connections 2024-04-26T21:57:09.019 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.016+0000 7f5f9339d740 1 -- 172.21.15.72:0/566257847 wait complete. 2024-04-26T21:57:09.019 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.016+0000 7f5f8e400640 1 --2- 172.21.15.72:0/3692370053 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f5f700a3c40 0x7f5f700a6100 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x55bb83250bb0 tx=0x7f5f80009580 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:57:09.019 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.016+0000 7f5f9339d740 1 -- start start 2024-04-26T21:57:09.019 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.017+0000 7f5f9339d740 1 Processor -- start 2024-04-26T21:57:09.020 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.017+0000 7f5f9339d740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bb8324f960 0x55bb8325db50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:09.020 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.017+0000 7f5f9339d740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bb83250350 0x55bb8325e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:09.020 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.017+0000 7f5f9339d740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55bb832554c0 0x55bb832650d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:09.020 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.017+0000 7f5f9339d740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55bb831b8200 con 0x55bb832554c0 2024-04-26T21:57:09.020 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.017+0000 7f5f9339d740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55bb831b2ee0 con 0x55bb83250350 2024-04-26T21:57:09.020 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.017+0000 7f5f9339d740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55bb831bfe70 con 0x55bb8324f960 2024-04-26T21:57:09.020 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.017+0000 7f5f9339d740 10 client.0 _ll_register_callbacks cb 0x55bb831a9270 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:57:09.020 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.017+0000 7f5f8e400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55bb832554c0 0x55bb832650d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:09.020 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.018+0000 7f5f8e400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55bb832554c0 0x55bb832650d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:48260/0 (socket says 172.21.15.72:48260) 2024-04-26T21:57:09.020 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.018+0000 7f5f8e400640 1 -- 172.21.15.72:0/2994169668 learned_addr learned my addr 172.21.15.72:0/2994169668 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:57:09.020 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.018+0000 7f5f8da00640 1 --2- 172.21.15.72:0/2994169668 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bb8324f960 0x55bb8325db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:09.020 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.018+0000 7f5f8d000640 1 --2- 172.21.15.72:0/2994169668 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bb83250350 0x55bb8325e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:09.020 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.018+0000 7f5f8e400640 1 -- 172.21.15.72:0/2994169668 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bb8324f960 msgr2=0x55bb8325db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:09.020 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.018+0000 7f5f8e400640 1 --2- 172.21.15.72:0/2994169668 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bb8324f960 0x55bb8325db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:09.021 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.018+0000 7f5f8e400640 1 -- 172.21.15.72:0/2994169668 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bb83250350 msgr2=0x55bb8325e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:09.021 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.018+0000 7f5f8e400640 1 --2- 172.21.15.72:0/2994169668 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bb83250350 0x55bb8325e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:09.021 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.018+0000 7f5f8e400640 1 -- 172.21.15.72:0/2994169668 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb82f8f7d0 con 0x55bb832554c0 2024-04-26T21:57:09.021 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.018+0000 7f5f8d000640 1 --2- 172.21.15.72:0/2994169668 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bb83250350 0x55bb8325e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:57:09.021 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.018+0000 7f5f8e400640 1 --2- 172.21.15.72:0/2994169668 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55bb832554c0 0x55bb832650d0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f5f80012870 tx=0x7f5f80004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f53edf9f909f104d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:57:09.021 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.018+0000 7f5f6f400640 10 client.0 ms_handle_connect on v2:172.21.15.72:3300/0 2024-04-26T21:57:09.021 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.019+0000 7f5f6f400640 1 -- 172.21.15.72:0/2994169668 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f8000ece0 con 0x55bb832554c0 2024-04-26T21:57:09.021 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.019+0000 7f5f6f400640 1 -- 172.21.15.72:0/2994169668 <== mon.0 v2:172.21.15.72:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7f5f8001f420 con 0x55bb832554c0 2024-04-26T21:57:09.021 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.019+0000 7f5f9339d740 1 -- 172.21.15.72:0/2994169668 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f7c003c80 con 0x55bb832554c0 2024-04-26T21:57:09.023 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.019+0000 7f5f6f400640 1 -- 172.21.15.72:0/2994169668 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f5f8002c020 con 0x55bb832554c0 2024-04-26T21:57:09.024 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.019+0000 7f5f9339d740 10 client.15837 fetch_fsmap learned FSMap version 27 2024-04-26T21:57:09.024 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.019+0000 7f5f9339d740 1 -- 172.21.15.72:0/2994169668 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55bb82ec53f0 con 0x55bb832554c0 2024-04-26T21:57:09.024 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.019+0000 7f5f6f400640 1 -- 172.21.15.72:0/2994169668 <== mon.0 v2:172.21.15.72:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5f7c003c80 con 0x55bb832554c0 2024-04-26T21:57:09.024 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.019+0000 7f5f6f400640 1 -- 172.21.15.72:0/2994169668 <== mon.0 v2:172.21.15.72:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f5f8000f040 con 0x55bb832554c0 2024-04-26T21:57:09.024 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.020+0000 7f5f9339d740 10 client.15837 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T21:57:09.024 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.020+0000 7f5f9339d740 10 client.15837 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:57:09.024 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.020+0000 7f5f9339d740 20 client.15837 populate_metadata read hostname 'smithi072' 2024-04-26T21:57:09.024 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.020+0000 7f5f9339d740 1 --2- 172.21.15.72:0/2994169668 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x55bb83269770 0x55bb8326bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:09.024 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.020+0000 7f5f9339d740 4 client.15837 mds_command: new command op to 34517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:57:09.024 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.020+0000 7f5f9339d740 1 -- 172.21.15.72:0/2994169668 --> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bb831c17e0 con 0x55bb83269770 2024-04-26T21:57:09.024 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.020+0000 7f5f8da00640 1 --2- 172.21.15.72:0/2994169668 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x55bb83269770 0x55bb8326bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:09.024 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.021+0000 7f5f8da00640 1 --2- 172.21.15.72:0/2994169668 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x55bb83269770 0x55bb8326bbb0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b1fde6db6089d1b7 server_cookie=99bcf614228e08ab in_seq=0 out_seq=0 2024-04-26T21:57:09.024 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.021+0000 7f5f6f400640 10 client.15837 ms_handle_connect on v2:172.21.15.76:6838/327145636 2024-04-26T21:57:09.024 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.023+0000 7f5f6f400640 1 -- 172.21.15.72:0/2994169668 <== mds.0 v2:172.21.15.76:6838/327145636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55bb831c17e0 con 0x55bb83269770 2024-04-26T21:57:09.025 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.023+0000 7f5f6f400640 10 client.15837 handle_command_reply: tid=0 2024-04-26T21:57:09.025 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.023+0000 7f5f9339d740 1 client.15837 shutdown 2024-04-26T21:57:09.025 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.023+0000 7f5f9339d740 1 -- 172.21.15.72:0/2994169668 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55bb832554c0 msgr2=0x55bb832650d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:09.025 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.023+0000 7f5f9339d740 1 --2- 172.21.15.72:0/2994169668 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55bb832554c0 0x55bb832650d0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f5f80012870 tx=0x7f5f80004a40 comp rx=0 tx=0).stop 2024-04-26T21:57:09.025 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.023+0000 7f5f9339d740 1 -- 172.21.15.72:0/2994169668 shutdown_connections 2024-04-26T21:57:09.025 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.023+0000 7f5f9339d740 1 --2- 172.21.15.72:0/2994169668 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x55bb83269770 0x55bb8326bbb0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:09.025 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.023+0000 7f5f9339d740 1 --2- 172.21.15.72:0/2994169668 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55bb832554c0 0x55bb832650d0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:09.025 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.023+0000 7f5f9339d740 1 --2- 172.21.15.72:0/2994169668 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bb83250350 0x55bb8325e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:09.025 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.023+0000 7f5f9339d740 1 --2- 172.21.15.72:0/2994169668 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bb8324f960 0x55bb8325db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:09.025 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.023+0000 7f5f9339d740 1 -- 172.21.15.72:0/2994169668 >> 172.21.15.72:0/2994169668 conn(0x55bb8324afd0 msgr2=0x55bb8324a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:57:09.025 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.023+0000 7f5f6f400640 0 client.15837 ms_handle_reset on v2:172.21.15.76:6838/327145636 2024-04-26T21:57:09.025 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.023+0000 7f5f9339d740 1 -- 172.21.15.72:0/2994169668 shutdown_connections 2024-04-26T21:57:09.025 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.024+0000 7f5f9339d740 1 -- 172.21.15.72:0/2994169668 wait complete. 2024-04-26T21:57:09.025 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.024+0000 7f5f9339d740 20 client.15837 trim_cache size 0 max 16384 2024-04-26T21:57:09.048 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.048+0000 7f5f9339d740 1 Processor -- start 2024-04-26T21:57:09.048 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.048+0000 7f5f9339d740 1 -- start start 2024-04-26T21:57:09.048 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.048+0000 7f5f9339d740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bb8324f960 0x55bb8324b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:09.048 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.048+0000 7f5f9339d740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bb83250350 0x55bb83265090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:09.048 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.048+0000 7f5f9339d740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55bb832554c0 0x55bb8324df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:09.048 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.048+0000 7f5f9339d740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f5f881a9570 con 0x55bb832554c0 2024-04-26T21:57:09.049 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.048+0000 7f5f9339d740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f5f881a8100 con 0x55bb8324f960 2024-04-26T21:57:09.049 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.048+0000 7f5f9339d740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f5f881a87d0 con 0x55bb83250350 2024-04-26T21:57:09.051 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.049+0000 7f5f8e400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55bb832554c0 0x55bb8324df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:09.051 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.049+0000 7f5f8e400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55bb832554c0 0x55bb8324df80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:48272/0 (socket says 172.21.15.72:48272) 2024-04-26T21:57:09.051 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.049+0000 7f5f8e400640 1 -- 172.21.15.72:0/2399300070 learned_addr learned my addr 172.21.15.72:0/2399300070 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:57:09.051 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.049+0000 7f5f8da00640 1 --2- 172.21.15.72:0/2399300070 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bb8324f960 0x55bb8324b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:09.051 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.049+0000 7f5f8d000640 1 --2- 172.21.15.72:0/2399300070 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bb83250350 0x55bb83265090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:09.051 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.049+0000 7f5f8e400640 1 -- 172.21.15.72:0/2399300070 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bb83250350 msgr2=0x55bb83265090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:09.051 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.049+0000 7f5f8e400640 1 --2- 172.21.15.72:0/2399300070 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bb83250350 0x55bb83265090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:09.051 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.049+0000 7f5f8e400640 1 -- 172.21.15.72:0/2399300070 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bb8324f960 msgr2=0x55bb8324b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:09.051 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.049+0000 7f5f8e400640 1 --2- 172.21.15.72:0/2399300070 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bb8324f960 0x55bb8324b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:09.052 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.049+0000 7f5f8e400640 1 -- 172.21.15.72:0/2399300070 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb82e850f0 con 0x55bb832554c0 2024-04-26T21:57:09.052 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.050+0000 7f5f8e400640 1 --2- 172.21.15.72:0/2399300070 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55bb832554c0 0x55bb8324df80 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f5f80012eb0 tx=0x7f5f80012ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3221df01d312be6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:57:09.052 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.050+0000 7f5f7a000640 1 -- 172.21.15.72:0/2399300070 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f8000eba0 con 0x55bb832554c0 2024-04-26T21:57:09.052 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.050+0000 7f5f7a000640 1 -- 172.21.15.72:0/2399300070 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5f8000ed40 con 0x55bb832554c0 2024-04-26T21:57:09.052 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.050+0000 7f5f7a000640 1 -- 172.21.15.72:0/2399300070 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f80032490 con 0x55bb832554c0 2024-04-26T21:57:09.052 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.051+0000 7f5f9339d740 1 -- 172.21.15.72:0/2399300070 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55bb832554c0 msgr2=0x55bb8324df80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:09.052 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.051+0000 7f5f9339d740 1 --2- 172.21.15.72:0/2399300070 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55bb832554c0 0x55bb8324df80 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f5f80012eb0 tx=0x7f5f80012ee0 comp rx=0 tx=0).stop 2024-04-26T21:57:09.052 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.051+0000 7f5f9339d740 1 -- 172.21.15.72:0/2399300070 shutdown_connections 2024-04-26T21:57:09.052 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.051+0000 7f5f9339d740 1 --2- 172.21.15.72:0/2399300070 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55bb832554c0 0x55bb8324df80 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:09.052 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.051+0000 7f5f9339d740 1 --2- 172.21.15.72:0/2399300070 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bb83250350 0x55bb83265090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:09.052 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.051+0000 7f5f9339d740 1 --2- 172.21.15.72:0/2399300070 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bb8324f960 0x55bb8324b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:09.052 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.051+0000 7f5f9339d740 1 -- 172.21.15.72:0/2399300070 >> 172.21.15.72:0/2399300070 conn(0x55bb8324afd0 msgr2=0x55bb8324a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:57:09.052 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.051+0000 7f5f9339d740 1 -- 172.21.15.72:0/2399300070 shutdown_connections 2024-04-26T21:57:09.053 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.051+0000 7f5f9339d740 1 -- 172.21.15.72:0/2399300070 wait complete. 2024-04-26T21:57:09.056 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.051+0000 7f5f9339d740 1 -- start start 2024-04-26T21:57:09.056 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.052+0000 7f5f9339d740 1 Processor -- start 2024-04-26T21:57:09.056 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.052+0000 7f5f9339d740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55bb8324f960 0x55bb83262aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:09.056 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.052+0000 7f5f9339d740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bb83250350 0x55bb83262fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:09.056 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.052+0000 7f5f9339d740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bb832554c0 0x55bb83263f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:09.056 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.052+0000 7f5f9339d740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f5f881a5a10 con 0x55bb8324f960 2024-04-26T21:57:09.056 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.052+0000 7f5f9339d740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f5f881a8ea0 con 0x55bb83250350 2024-04-26T21:57:09.056 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.052+0000 7f5f9339d740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f5f881a6080 con 0x55bb832554c0 2024-04-26T21:57:09.056 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.052+0000 7f5f9339d740 10 client.0 _ll_register_callbacks cb 0x7f5f80009f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:57:09.056 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.052+0000 7f5f8da00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55bb8324f960 0x55bb83262aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:09.056 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.052+0000 7f5f8da00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55bb8324f960 0x55bb83262aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:48276/0 (socket says 172.21.15.72:48276) 2024-04-26T21:57:09.057 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.052+0000 7f5f8da00640 1 -- 172.21.15.72:0/360991605 learned_addr learned my addr 172.21.15.72:0/360991605 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:57:09.057 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.053+0000 7f5f8d000640 1 --2- 172.21.15.72:0/360991605 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bb83250350 0x55bb83262fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:09.057 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.053+0000 7f5f8e400640 1 --2- 172.21.15.72:0/360991605 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bb832554c0 0x55bb83263f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:09.057 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.053+0000 7f5f8da00640 1 -- 172.21.15.72:0/360991605 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bb832554c0 msgr2=0x55bb83263f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:09.057 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.053+0000 7f5f8da00640 1 --2- 172.21.15.72:0/360991605 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bb832554c0 0x55bb83263f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:09.057 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.053+0000 7f5f8da00640 1 -- 172.21.15.72:0/360991605 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bb83250350 msgr2=0x55bb83262fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:09.057 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.053+0000 7f5f8da00640 1 --2- 172.21.15.72:0/360991605 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bb83250350 0x55bb83262fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:09.057 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.053+0000 7f5f8da00640 1 -- 172.21.15.72:0/360991605 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb8317ba30 con 0x55bb8324f960 2024-04-26T21:57:09.057 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.053+0000 7f5f8d000640 1 --2- 172.21.15.72:0/360991605 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bb83250350 0x55bb83262fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:57:09.057 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.053+0000 7f5f8da00640 1 --2- 172.21.15.72:0/360991605 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55bb8324f960 0x55bb83262aa0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f5f7c02a450 tx=0x7f5f7c005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6e5b5d54a7546f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:57:09.057 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.053+0000 7f5f6f400640 10 client.0 ms_handle_connect on v2:172.21.15.72:3300/0 2024-04-26T21:57:09.057 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.053+0000 7f5f6f400640 1 -- 172.21.15.72:0/360991605 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f7c035040 con 0x55bb8324f960 2024-04-26T21:57:09.057 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.054+0000 7f5f6f400640 1 -- 172.21.15.72:0/360991605 <== mon.0 v2:172.21.15.72:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7f5f7c04e7b0 con 0x55bb8324f960 2024-04-26T21:57:09.057 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.054+0000 7f5f9339d740 1 -- 172.21.15.72:0/360991605 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f8000ed40 con 0x55bb8324f960 2024-04-26T21:57:09.057 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.054+0000 7f5f6f400640 1 -- 172.21.15.72:0/360991605 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f5f7c05b030 con 0x55bb8324f960 2024-04-26T21:57:09.058 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.054+0000 7f5f9339d740 10 client.15843 fetch_fsmap learned FSMap version 27 2024-04-26T21:57:09.058 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.054+0000 7f5f9339d740 1 -- 172.21.15.72:0/360991605 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55bb832cfcf0 con 0x55bb8324f960 2024-04-26T21:57:09.058 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.054+0000 7f5f6f400640 1 -- 172.21.15.72:0/360991605 <== mon.0 v2:172.21.15.72:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5f8000ed40 con 0x55bb8324f960 2024-04-26T21:57:09.058 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.054+0000 7f5f6f400640 1 -- 172.21.15.72:0/360991605 <== mon.0 v2:172.21.15.72:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f5f7c03c7b0 con 0x55bb8324f960 2024-04-26T21:57:09.058 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.054+0000 7f5f9339d740 10 client.15843 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T21:57:09.058 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.054+0000 7f5f9339d740 10 client.15843 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:57:09.058 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.054+0000 7f5f9339d740 20 client.15843 populate_metadata read hostname 'smithi072' 2024-04-26T21:57:09.058 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.054+0000 7f5f9339d740 1 --2- 172.21.15.72:0/360991605 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x55bb832d02c0 0x55bb832d26b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:09.058 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.054+0000 7f5f9339d740 4 client.15843 mds_command: new command op to 34517 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-26T21:57:09.058 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.055+0000 7f5f9339d740 1 -- 172.21.15.72:0/360991605 --> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55bb831bb650 con 0x55bb832d02c0 2024-04-26T21:57:09.059 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.055+0000 7f5f8d000640 1 --2- 172.21.15.72:0/360991605 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x55bb832d02c0 0x55bb832d26b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:09.059 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.056+0000 7f5f8d000640 1 --2- 172.21.15.72:0/360991605 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x55bb832d02c0 0x55bb832d26b0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ad4bc721dee745e server_cookie=65d70ea9fcacebd9 in_seq=0 out_seq=0 2024-04-26T21:57:09.059 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.056+0000 7f5f6f400640 10 client.15843 ms_handle_connect on v2:172.21.15.76:6838/327145636 2024-04-26T21:57:09.060 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.056+0000 7f5f6f400640 1 -- 172.21.15.72:0/360991605 <== mds.0 v2:172.21.15.76:6838/327145636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55bb831bb650 con 0x55bb832d02c0 2024-04-26T21:57:09.060 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.056+0000 7f5f6f400640 10 client.15843 handle_command_reply: tid=0 2024-04-26T21:57:09.061 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.057+0000 7f5f9339d740 1 client.15843 shutdown 2024-04-26T21:57:09.061 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.057+0000 7f5f9339d740 1 -- 172.21.15.72:0/360991605 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55bb8324f960 msgr2=0x55bb83262aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:09.061 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.057+0000 7f5f9339d740 1 --2- 172.21.15.72:0/360991605 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55bb8324f960 0x55bb83262aa0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f5f7c02a450 tx=0x7f5f7c005fd0 comp rx=0 tx=0).stop 2024-04-26T21:57:09.061 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.057+0000 7f5f9339d740 1 -- 172.21.15.72:0/360991605 shutdown_connections 2024-04-26T21:57:09.061 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.057+0000 7f5f9339d740 1 --2- 172.21.15.72:0/360991605 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x55bb832d02c0 0x55bb832d26b0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:09.061 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.057+0000 7f5f9339d740 1 --2- 172.21.15.72:0/360991605 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bb832554c0 0x55bb83263f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:09.062 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.057+0000 7f5f9339d740 1 --2- 172.21.15.72:0/360991605 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bb83250350 0x55bb83262fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:09.062 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.057+0000 7f5f9339d740 1 --2- 172.21.15.72:0/360991605 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55bb8324f960 0x55bb83262aa0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:09.062 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.057+0000 7f5f9339d740 1 -- 172.21.15.72:0/360991605 >> 172.21.15.72:0/360991605 conn(0x55bb8324afd0 msgr2=0x55bb83249960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:57:09.062 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.057+0000 7f5f6f400640 0 client.15843 ms_handle_reset on v2:172.21.15.76:6838/327145636 2024-04-26T21:57:09.062 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.057+0000 7f5f9339d740 1 -- 172.21.15.72:0/360991605 shutdown_connections 2024-04-26T21:57:09.062 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.057+0000 7f5f9339d740 1 -- 172.21.15.72:0/360991605 wait complete. 2024-04-26T21:57:09.062 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.057+0000 7f5f9339d740 20 client.15843 trim_cache size 0 max 16384 2024-04-26T21:57:09.062 INFO:teuthology.orchestra.run.smithi072.stdout:{ 2024-04-26T21:57:09.062 INFO:teuthology.orchestra.run.smithi072.stdout: "status": "no active scrubs running", 2024-04-26T21:57:09.062 INFO:teuthology.orchestra.run.smithi072.stdout: "scrubs": {} 2024-04-26T21:57:09.063 INFO:teuthology.orchestra.run.smithi072.stdout:} 2024-04-26T21:57:09.063 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.058+0000 7f5f8ee00640 1 -- 172.21.15.72:0/3692370053 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f5f700a3c40 msgr2=0x7f5f700a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:09.063 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.058+0000 7f5f8ee00640 1 --2- 172.21.15.72:0/3692370053 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f5f700a3c40 0x7f5f700a6100 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x55bb83250bb0 tx=0x7f5f80009580 comp rx=0 tx=0).stop 2024-04-26T21:57:09.063 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.058+0000 7f5f8ee00640 1 -- 172.21.15.72:0/3692370053 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5f88105c50 msgr2=0x7f5f88198e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:09.064 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.058+0000 7f5f8ee00640 1 --2- 172.21.15.72:0/3692370053 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5f88105c50 0x7f5f88198e70 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f5f7400c970 tx=0x7f5f7400ce30 comp rx=0 tx=0).stop 2024-04-26T21:57:09.064 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.058+0000 7f5f8ee00640 1 -- 172.21.15.72:0/3692370053 shutdown_connections 2024-04-26T21:57:09.064 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.058+0000 7f5f8ee00640 1 --2- 172.21.15.72:0/3692370053 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5f88108580 0x7f5f8819fee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:09.064 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.058+0000 7f5f8ee00640 1 --2- 172.21.15.72:0/3692370053 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5f88105c50 0x7f5f88198e70 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:09.064 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.058+0000 7f5f8ee00640 1 --2- 172.21.15.72:0/3692370053 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f5f700a3c40 0x7f5f700a6100 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:09.064 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.058+0000 7f5f8ee00640 1 --2- 172.21.15.72:0/3692370053 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5f88103320 0x7f5f88198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:09.064 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.058+0000 7f5f8ee00640 1 -- 172.21.15.72:0/3692370053 >> 172.21.15.72:0/3692370053 conn(0x7f5f880fccd0 msgr2=0x7f5f88103d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:57:09.064 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.058+0000 7f5f8ee00640 1 -- 172.21.15.72:0/3692370053 shutdown_connections 2024-04-26T21:57:09.064 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.058+0000 7f5f8ee00640 1 -- 172.21.15.72:0/3692370053 wait complete. 2024-04-26T21:57:09.079 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-26T21:57:09.079 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:57:09.141 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:57:08 smithi072 ceph-mon[28138]: pgmap v500: 97 pgs: 97 active+clean; 4.7 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 343 KiB/s wr, 10 op/s 2024-04-26T21:57:09.284 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.284+0000 7f02d7e00640 1 -- 172.21.15.72:0/3145117101 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f02d810a620 msgr2=0x7f02d810cac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:09.285 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.284+0000 7f02d7e00640 1 --2- 172.21.15.72:0/3145117101 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f02d810a620 0x7f02d810cac0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f02d0009ca0 tx=0x7f02d002a950 comp rx=0 tx=0).stop 2024-04-26T21:57:09.285 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.284+0000 7f02d7e00640 1 -- 172.21.15.72:0/3145117101 shutdown_connections 2024-04-26T21:57:09.285 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.284+0000 7f02d7e00640 1 --2- 172.21.15.72:0/3145117101 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f02d810a620 0x7f02d810cac0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:09.285 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.284+0000 7f02d7e00640 1 --2- 172.21.15.72:0/3145117101 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f02d8075330 0x7f02d8075790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:09.285 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.284+0000 7f02d7e00640 1 --2- 172.21.15.72:0/3145117101 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f02d8076910 0x7f02d8074d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:09.285 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.284+0000 7f02d7e00640 1 -- 172.21.15.72:0/3145117101 >> 172.21.15.72:0/3145117101 conn(0x7f02d80fdd00 msgr2=0x7f02d8100140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:57:09.285 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.284+0000 7f02d7e00640 1 -- 172.21.15.72:0/3145117101 shutdown_connections 2024-04-26T21:57:09.285 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.284+0000 7f02d7e00640 1 -- 172.21.15.72:0/3145117101 wait complete. 2024-04-26T21:57:09.288 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.285+0000 7f02d7e00640 1 Processor -- start 2024-04-26T21:57:09.288 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.285+0000 7f02d7e00640 1 -- start start 2024-04-26T21:57:09.288 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.286+0000 7f02d7e00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f02d8075330 0x7f02d819aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:09.288 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.286+0000 7f02d7e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f02d8076910 0x7f02d819afd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:09.288 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.286+0000 7f02d7e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f02d810a620 0x7f02d81a1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:09.288 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.286+0000 7f02d7e00640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f02d81103c0 con 0x7f02d8076910 2024-04-26T21:57:09.288 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.286+0000 7f02d7e00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f02d8110240 con 0x7f02d8075330 2024-04-26T21:57:09.289 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.286+0000 7f02d7e00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f02d8110540 con 0x7f02d810a620 2024-04-26T21:57:09.289 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.286+0000 7f02d6000640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f02d8076910 0x7f02d819afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:09.289 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.286+0000 7f02d6000640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f02d8076910 0x7f02d819afd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:48302/0 (socket says 172.21.15.72:48302) 2024-04-26T21:57:09.289 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.286+0000 7f02d6000640 1 -- 172.21.15.72:0/2675833015 learned_addr learned my addr 172.21.15.72:0/2675833015 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:57:09.289 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.286+0000 7f02d6a00640 1 --2- 172.21.15.72:0/2675833015 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f02d8075330 0x7f02d819aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:09.289 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.287+0000 7f02d7400640 1 --2- 172.21.15.72:0/2675833015 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f02d810a620 0x7f02d81a1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:09.289 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.287+0000 7f02d6000640 1 -- 172.21.15.72:0/2675833015 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f02d810a620 msgr2=0x7f02d81a1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:09.289 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.287+0000 7f02d6000640 1 --2- 172.21.15.72:0/2675833015 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f02d810a620 0x7f02d81a1ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:09.289 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.287+0000 7f02d6000640 1 -- 172.21.15.72:0/2675833015 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f02d8075330 msgr2=0x7f02d819aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:09.289 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.287+0000 7f02d6000640 1 --2- 172.21.15.72:0/2675833015 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f02d8075330 0x7f02d819aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:09.289 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.287+0000 7f02d6000640 1 -- 172.21.15.72:0/2675833015 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02d81a2660 con 0x7f02d8076910 2024-04-26T21:57:09.289 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.287+0000 7f02d7400640 1 --2- 172.21.15.72:0/2675833015 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f02d810a620 0x7f02d81a1ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:57:09.289 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.287+0000 7f02d6000640 1 --2- 172.21.15.72:0/2675833015 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f02d8076910 0x7f02d819afd0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f02c400d950 tx=0x7f02c400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:57:09.290 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.287+0000 7f02cf400640 1 -- 172.21.15.72:0/2675833015 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02c4017070 con 0x7f02d8076910 2024-04-26T21:57:09.290 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.287+0000 7f02cf400640 1 -- 172.21.15.72:0/2675833015 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f02c4009c30 con 0x7f02d8076910 2024-04-26T21:57:09.290 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.287+0000 7f02d7e00640 1 -- 172.21.15.72:0/2675833015 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02d81a2950 con 0x7f02d8076910 2024-04-26T21:57:09.290 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.287+0000 7f02cf400640 1 -- 172.21.15.72:0/2675833015 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02c400f620 con 0x7f02d8076910 2024-04-26T21:57:09.290 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.288+0000 7f02d7e00640 1 -- 172.21.15.72:0/2675833015 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02d81a2ef0 con 0x7f02d8076910 2024-04-26T21:57:09.290 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.289+0000 7f02d7e00640 1 -- 172.21.15.72:0/2675833015 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02d81093e0 con 0x7f02d8076910 2024-04-26T21:57:09.292 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.292+0000 7f02cf400640 1 -- 172.21.15.72:0/2675833015 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f02c4010460 con 0x7f02d8076910 2024-04-26T21:57:09.294 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.294+0000 7f02cf400640 1 --2- 172.21.15.72:0/2675833015 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f02bc0a3cc0 0x7f02bc0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:09.294 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.294+0000 7f02cf400640 1 -- 172.21.15.72:0/2675833015 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7f02c40d02b0 con 0x7f02d8076910 2024-04-26T21:57:09.294 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.294+0000 7f02d6a00640 1 --2- 172.21.15.72:0/2675833015 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f02bc0a3cc0 0x7f02bc0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:09.295 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.295+0000 7f02d6a00640 1 --2- 172.21.15.72:0/2675833015 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f02bc0a3cc0 0x7f02bc0a6180 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f02c8005fd0 tx=0x7f02c8005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:57:09.300 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.301+0000 7f02cf400640 1 -- 172.21.15.72:0/2675833015 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f02c409ca50 con 0x7f02d8076910 2024-04-26T21:57:09.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:57:08 smithi076 ceph-mon[31823]: pgmap v500: 97 pgs: 97 active+clean; 4.7 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 343 KiB/s wr, 10 op/s 2024-04-26T21:57:09.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:57:08 smithi164 ceph-mon[31916]: pgmap v500: 97 pgs: 97 active+clean; 4.7 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 343 KiB/s wr, 10 op/s 2024-04-26T21:57:09.586 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.586+0000 7f02d7e00640 1 -- 172.21.15.72:0/2675833015 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f02d802dad0 con 0x7f02d8076910 2024-04-26T21:57:09.588 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.588+0000 7f02cf400640 1 -- 172.21.15.72:0/2675833015 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11931 (secure 0 0 0) 0x7f02c4009de0 con 0x7f02d8076910 2024-04-26T21:57:09.588 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T21:57:09.588 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6835/1320381959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6834","nonce":1320381959},{"type":"v1","addr":"172.21.15.72:6835","nonce":1320381959}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6837/2142200512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6836","nonce":2142200512},{"type":"v1","addr":"172.21.15.72:6837","nonce":2142200512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T21:52:50.567857+0000","modified":"2024-04-26T21:55:22.894383+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34517,"mds_1":34472,"mds_2":34442,"mds_3":34406,"mds_4":24463},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14652":{"gid":14652,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.72:6839/1312898281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6838","nonce":1312898281},{"type":"v1","addr":"172.21.15.72:6839","nonce":1312898281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14664":{"gid":14664,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.72:6841/3257692339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6840","nonce":3257692339},{"type":"v1","addr":"172.21.15.72:6841","nonce":3257692339}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24388":{"gid":24388,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6833/4236668465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":4236668465},{"type":"v1","addr":"172.21.15.164:6833","nonce":4236668465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6835/2391114169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":2391114169},{"type":"v1","addr":"172.21.15.164:6835","nonce":2391114169}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6837/3585829992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":3585829992},{"type":"v1","addr":"172.21.15.164:6837","nonce":3585829992}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.164:6839/1636226820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":1636226820},{"type":"v1","addr":"172.21.15.164:6839","nonce":1636226820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34406":{"gid":34406,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":15,"addr":"172.21.15.76:6833/3869483750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":3869483750},{"type":"v1","addr":"172.21.15.76:6833","nonce":3869483750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34442":{"gid":34442,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.76:6835/3517429435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":3517429435},{"type":"v1","addr":"172.21.15.76:6835","nonce":3517429435}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34472":{"gid":34472,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.76:6837/1150678991","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":1150678991},{"type":"v1","addr":"172.21.15.76:6837","nonce":1150678991}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34517":{"gid":34517,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.76:6839/327145636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6838","nonce":327145636},{"type":"v1","addr":"172.21.15.76:6839","nonce":327145636}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34517,"qdb_cluster":[34472,34517,34442,34406,24463]},"id":1}]} 2024-04-26T21:57:09.588 INFO:teuthology.orchestra.run.smithi072.stderr:dumped fsmap epoch 27 2024-04-26T21:57:09.591 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.591+0000 7f02d7e00640 1 -- 172.21.15.72:0/2675833015 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f02bc0a3cc0 msgr2=0x7f02bc0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:09.592 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.591+0000 7f02d7e00640 1 --2- 172.21.15.72:0/2675833015 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f02bc0a3cc0 0x7f02bc0a6180 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f02c8005fd0 tx=0x7f02c8005850 comp rx=0 tx=0).stop 2024-04-26T21:57:09.592 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.591+0000 7f02d7e00640 1 -- 172.21.15.72:0/2675833015 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f02d8076910 msgr2=0x7f02d819afd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:09.592 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.591+0000 7f02d7e00640 1 --2- 172.21.15.72:0/2675833015 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f02d8076910 0x7f02d819afd0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f02c400d950 tx=0x7f02c400de10 comp rx=0 tx=0).stop 2024-04-26T21:57:09.592 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.591+0000 7f02d7e00640 1 -- 172.21.15.72:0/2675833015 shutdown_connections 2024-04-26T21:57:09.592 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.591+0000 7f02d7e00640 1 --2- 172.21.15.72:0/2675833015 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f02bc0a3cc0 0x7f02bc0a6180 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:09.592 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.591+0000 7f02d7e00640 1 --2- 172.21.15.72:0/2675833015 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f02d810a620 0x7f02d81a1ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:09.592 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.591+0000 7f02d7e00640 1 --2- 172.21.15.72:0/2675833015 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f02d8076910 0x7f02d819afd0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:09.592 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.591+0000 7f02d7e00640 1 --2- 172.21.15.72:0/2675833015 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f02d8075330 0x7f02d819aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:09.592 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.591+0000 7f02d7e00640 1 -- 172.21.15.72:0/2675833015 >> 172.21.15.72:0/2675833015 conn(0x7f02d80fdd00 msgr2=0x7f02d81065b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:57:09.592 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.592+0000 7f02d7e00640 1 -- 172.21.15.72:0/2675833015 shutdown_connections 2024-04-26T21:57:09.593 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.592+0000 7f02d7e00640 1 -- 172.21.15.72:0/2675833015 wait complete. 2024-04-26T21:57:09.614 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-26T21:57:09.820 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.819+0000 7f5fe2800640 1 -- 172.21.15.72:0/1178722623 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5fdc1028c0 msgr2=0x7f5fdc102ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:09.820 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.819+0000 7f5fe2800640 1 --2- 172.21.15.72:0/1178722623 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5fdc1028c0 0x7f5fdc102ce0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f5fd0009900 tx=0x7f5fd002a400 comp rx=0 tx=0).stop 2024-04-26T21:57:09.820 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.819+0000 7f5fe2800640 1 -- 172.21.15.72:0/1178722623 shutdown_connections 2024-04-26T21:57:09.820 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.819+0000 7f5fe2800640 1 --2- 172.21.15.72:0/1178722623 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5fdc104470 0x7f5fdc10ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:09.820 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.819+0000 7f5fe2800640 1 --2- 172.21.15.72:0/1178722623 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5fdc103ab0 0x7f5fdc103f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:09.820 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.819+0000 7f5fe2800640 1 --2- 172.21.15.72:0/1178722623 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5fdc1028c0 0x7f5fdc102ce0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:09.820 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.819+0000 7f5fe2800640 1 -- 172.21.15.72:0/1178722623 >> 172.21.15.72:0/1178722623 conn(0x7f5fdc0fde30 msgr2=0x7f5fdc100270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:57:09.820 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.819+0000 7f5fe2800640 1 -- 172.21.15.72:0/1178722623 shutdown_connections 2024-04-26T21:57:09.820 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.819+0000 7f5fe2800640 1 -- 172.21.15.72:0/1178722623 wait complete. 2024-04-26T21:57:09.822 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.820+0000 7f5fe2800640 1 Processor -- start 2024-04-26T21:57:09.822 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.821+0000 7f5fe2800640 1 -- start start 2024-04-26T21:57:09.822 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.821+0000 7f5fe2800640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5fdc1028c0 0x7f5fdc19ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:09.822 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.821+0000 7f5fe2800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5fdc103ab0 0x7f5fdc19b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:09.822 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.821+0000 7f5fe2800640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5fdc104470 0x7f5fdc1a20e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:09.822 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.821+0000 7f5fe2800640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f5fdc10e250 con 0x7f5fdc1028c0 2024-04-26T21:57:09.822 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.821+0000 7f5fe2800640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f5fdc10e0d0 con 0x7f5fdc104470 2024-04-26T21:57:09.823 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.821+0000 7f5fe2800640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f5fdc10e3d0 con 0x7f5fdc103ab0 2024-04-26T21:57:09.823 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.821+0000 7f5fe1400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5fdc1028c0 0x7f5fdc19ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:09.823 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.821+0000 7f5fe1400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5fdc1028c0 0x7f5fdc19ab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:48318/0 (socket says 172.21.15.72:48318) 2024-04-26T21:57:09.823 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.821+0000 7f5fe1400640 1 -- 172.21.15.72:0/3762423934 learned_addr learned my addr 172.21.15.72:0/3762423934 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:57:09.823 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.822+0000 7f5fe0a00640 1 --2- 172.21.15.72:0/3762423934 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5fdc103ab0 0x7f5fdc19b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:09.823 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.822+0000 7f5fe1e00640 1 --2- 172.21.15.72:0/3762423934 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5fdc104470 0x7f5fdc1a20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:09.823 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.822+0000 7f5fe1400640 1 -- 172.21.15.72:0/3762423934 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5fdc103ab0 msgr2=0x7f5fdc19b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:09.823 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.822+0000 7f5fe1400640 1 --2- 172.21.15.72:0/3762423934 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5fdc103ab0 0x7f5fdc19b070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:09.823 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.822+0000 7f5fe1400640 1 -- 172.21.15.72:0/3762423934 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5fdc104470 msgr2=0x7f5fdc1a20e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:09.823 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.822+0000 7f5fe1400640 1 --2- 172.21.15.72:0/3762423934 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5fdc104470 0x7f5fdc1a20e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:09.823 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.822+0000 7f5fe1400640 1 -- 172.21.15.72:0/3762423934 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fdc1a2750 con 0x7f5fdc1028c0 2024-04-26T21:57:09.823 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.822+0000 7f5fe1e00640 1 --2- 172.21.15.72:0/3762423934 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5fdc104470 0x7f5fdc1a20e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:57:09.823 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.822+0000 7f5fe1400640 1 --2- 172.21.15.72:0/3762423934 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5fdc1028c0 0x7f5fdc19ab30 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f5fd0031460 tx=0x7f5fd0003d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:57:09.824 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.823+0000 7f5fda000640 1 -- 172.21.15.72:0/3762423934 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fd003f070 con 0x7f5fdc1028c0 2024-04-26T21:57:09.824 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.823+0000 7f5fda000640 1 -- 172.21.15.72:0/3762423934 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5fd0031da0 con 0x7f5fdc1028c0 2024-04-26T21:57:09.824 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.823+0000 7f5fe2800640 1 -- 172.21.15.72:0/3762423934 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fdc1a29e0 con 0x7f5fdc1028c0 2024-04-26T21:57:09.825 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.823+0000 7f5fda000640 1 -- 172.21.15.72:0/3762423934 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fd0035030 con 0x7f5fdc1028c0 2024-04-26T21:57:09.825 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.823+0000 7f5fe2800640 1 -- 172.21.15.72:0/3762423934 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fdc1a2ed0 con 0x7f5fdc1028c0 2024-04-26T21:57:09.826 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.827+0000 7f5fe6bd9740 1 Processor -- start 2024-04-26T21:57:09.827 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.827+0000 7f5fe6bd9740 1 -- start start 2024-04-26T21:57:09.827 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.827+0000 7f5fe6bd9740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c3427f5050 0x55c3427f5470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:09.827 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.827+0000 7f5fe6bd9740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c3427f5a40 0x55c3427fa540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:09.828 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.827+0000 7f5fe6bd9740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c3427fabb0 0x55c3427fd040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:09.828 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.827+0000 7f5fe6bd9740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55c342758c70 con 0x55c3427f5a40 2024-04-26T21:57:09.828 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.827+0000 7f5fe6bd9740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55c342711980 con 0x55c3427fabb0 2024-04-26T21:57:09.828 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.827+0000 7f5fe6bd9740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55c342754c30 con 0x55c3427f5050 2024-04-26T21:57:09.828 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.828+0000 7f5fe1e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c3427f5050 0x55c3427f5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:09.829 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.828+0000 7f5fe1e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c3427f5050 0x55c3427f5470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.72:33636/0 (socket says 172.21.15.72:33636) 2024-04-26T21:57:09.829 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.828+0000 7f5fe1e00640 1 -- 172.21.15.72:0/436361311 learned_addr learned my addr 172.21.15.72:0/436361311 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:57:09.829 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.828+0000 7f5fe0a00640 1 --2- 172.21.15.72:0/436361311 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c3427fabb0 0x55c3427fd040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:09.829 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.829+0000 7f5fda000640 1 -- 172.21.15.72:0/3762423934 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f5fd0042070 con 0x7f5fdc1028c0 2024-04-26T21:57:09.829 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.829+0000 7f5fe1e00640 1 -- 172.21.15.72:0/436361311 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c3427fabb0 msgr2=0x55c3427fd040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:09.829 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.829+0000 7f5fe1e00640 1 --2- 172.21.15.72:0/436361311 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c3427fabb0 0x55c3427fd040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:09.829 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.829+0000 7f5fe1e00640 1 -- 172.21.15.72:0/436361311 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c3427f5a40 msgr2=0x55c3427fa540 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T21:57:09.830 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.829+0000 7f5fe1e00640 1 --2- 172.21.15.72:0/436361311 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c3427f5a40 0x55c3427fa540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:09.830 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.829+0000 7f5fe1e00640 1 -- 172.21.15.72:0/436361311 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c3426e97b0 con 0x55c3427f5050 2024-04-26T21:57:09.830 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.830+0000 7f5fe1e00640 1 --2- 172.21.15.72:0/436361311 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c3427f5050 0x55c3427f5470 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f5fd4008e60 tx=0x7f5fd400d710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7e34375db36ee4df server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:57:09.830 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.830+0000 7f5fce600640 1 -- 172.21.15.72:0/436361311 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fd400f830 con 0x55c3427f5050 2024-04-26T21:57:09.830 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.830+0000 7f5fce600640 1 -- 172.21.15.72:0/436361311 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5fd4018480 con 0x55c3427f5050 2024-04-26T21:57:09.830 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.830+0000 7f5fce600640 1 -- 172.21.15.72:0/436361311 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fd400fe60 con 0x55c3427f5050 2024-04-26T21:57:09.831 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.830+0000 7f5fda000640 1 --2- 172.21.15.72:0/3762423934 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f5fc40a3c40 0x7f5fc40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:09.831 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.830+0000 7f5fda000640 1 -- 172.21.15.72:0/3762423934 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7f5fd00f10c0 con 0x7f5fdc1028c0 2024-04-26T21:57:09.832 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.831+0000 7f5fe1400640 1 --2- 172.21.15.72:0/3762423934 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f5fc40a3c40 0x7f5fc40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:09.832 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.831+0000 7f5fe6bd9740 1 -- 172.21.15.72:0/436361311 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c3427f5050 msgr2=0x55c3427f5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:09.832 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.831+0000 7f5fe6bd9740 1 --2- 172.21.15.72:0/436361311 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c3427f5050 0x55c3427f5470 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f5fd4008e60 tx=0x7f5fd400d710 comp rx=0 tx=0).stop 2024-04-26T21:57:09.832 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.831+0000 7f5fe6bd9740 1 -- 172.21.15.72:0/436361311 shutdown_connections 2024-04-26T21:57:09.832 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.831+0000 7f5fe6bd9740 1 --2- 172.21.15.72:0/436361311 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c3427fabb0 0x55c3427fd040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:09.832 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.831+0000 7f5fe6bd9740 1 --2- 172.21.15.72:0/436361311 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c3427f5a40 0x55c3427fa540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:09.832 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.831+0000 7f5fe6bd9740 1 --2- 172.21.15.72:0/436361311 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c3427f5050 0x55c3427f5470 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:09.832 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.831+0000 7f5fe6bd9740 1 -- 172.21.15.72:0/436361311 >> 172.21.15.72:0/436361311 conn(0x55c3427f06c0 msgr2=0x55c3427f0ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:57:09.832 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.831+0000 7f5fe6bd9740 1 -- 172.21.15.72:0/436361311 shutdown_connections 2024-04-26T21:57:09.832 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.831+0000 7f5fe6bd9740 1 -- 172.21.15.72:0/436361311 wait complete. 2024-04-26T21:57:09.832 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.831+0000 7f5fe1400640 1 --2- 172.21.15.72:0/3762423934 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f5fc40a3c40 0x7f5fc40a6100 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x55c3427f5e40 tx=0x7f5fd002a400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:57:09.833 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.832+0000 7f5fe6bd9740 1 -- start start 2024-04-26T21:57:09.835 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.832+0000 7f5fe6bd9740 1 Processor -- start 2024-04-26T21:57:09.835 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.833+0000 7f5fe6bd9740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c3427f5050 0x55c3427fef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:09.835 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.833+0000 7f5fe6bd9740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c3427f5a40 0x55c3427ff490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:09.836 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.833+0000 7f5fe6bd9740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c3427fabb0 0x55c3428064d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:09.836 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.833+0000 7f5fe6bd9740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55c342757f80 con 0x55c3427f5a40 2024-04-26T21:57:09.836 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.833+0000 7f5fe6bd9740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55c342705260 con 0x55c3427f5050 2024-04-26T21:57:09.836 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.833+0000 7f5fe6bd9740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55c34275d2a0 con 0x55c3427fabb0 2024-04-26T21:57:09.836 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.833+0000 7f5fe6bd9740 10 client.0 _ll_register_callbacks cb 0x55c34274f960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:57:09.836 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.833+0000 7f5fe0a00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c3427f5a40 0x55c3427ff490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:09.836 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.833+0000 7f5fe0a00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c3427f5a40 0x55c3427ff490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:48332/0 (socket says 172.21.15.72:48332) 2024-04-26T21:57:09.836 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.833+0000 7f5fe0a00640 1 -- 172.21.15.72:0/235123560 learned_addr learned my addr 172.21.15.72:0/235123560 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:57:09.836 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.833+0000 7f5fe1e00640 1 --2- 172.21.15.72:0/235123560 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c3427f5050 0x55c3427fef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:09.836 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.833+0000 7f5fe1e00640 1 --2- 172.21.15.72:0/235123560 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c3427fabb0 0x55c3428064d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:09.836 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.834+0000 7f5fe0a00640 1 -- 172.21.15.72:0/235123560 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c3427fabb0 msgr2=0x55c3428064d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:09.836 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.834+0000 7f5fe0a00640 1 --2- 172.21.15.72:0/235123560 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c3427fabb0 0x55c3428064d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:09.837 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.834+0000 7f5fe0a00640 1 -- 172.21.15.72:0/235123560 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c3427f5050 msgr2=0x55c3427fef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:09.837 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.834+0000 7f5fe0a00640 1 --2- 172.21.15.72:0/235123560 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c3427f5050 0x55c3427fef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:09.837 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.834+0000 7f5fe0a00640 1 -- 172.21.15.72:0/235123560 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c3426e71c0 con 0x55c3427f5a40 2024-04-26T21:57:09.837 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.834+0000 7f5fe0a00640 1 --2- 172.21.15.72:0/235123560 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c3427f5a40 0x55c3427ff490 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f5fc800ce90 tx=0x7f5fc80039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d5344fa965fc7e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:57:09.837 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.834+0000 7f5fc2a00640 10 client.0 ms_handle_connect on v2:172.21.15.72:3300/0 2024-04-26T21:57:09.837 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.834+0000 7f5fc2a00640 1 -- 172.21.15.72:0/235123560 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fc80119f0 con 0x55c3427f5a40 2024-04-26T21:57:09.837 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.834+0000 7f5fe6bd9740 1 -- 172.21.15.72:0/235123560 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5fd4018480 con 0x55c3427f5a40 2024-04-26T21:57:09.837 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.834+0000 7f5fc2a00640 1 -- 172.21.15.72:0/235123560 <== mon.0 v2:172.21.15.72:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7f5fc8011b90 con 0x55c3427f5a40 2024-04-26T21:57:09.837 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.835+0000 7f5fc2a00640 1 -- 172.21.15.72:0/235123560 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f5fc8011d90 con 0x55c3427f5a40 2024-04-26T21:57:09.837 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.835+0000 7f5fe6bd9740 10 client.15858 fetch_fsmap learned FSMap version 27 2024-04-26T21:57:09.837 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.835+0000 7f5fe6bd9740 1 -- 172.21.15.72:0/235123560 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c34246a3f0 con 0x55c3427f5a40 2024-04-26T21:57:09.837 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.835+0000 7f5fc2a00640 1 -- 172.21.15.72:0/235123560 <== mon.0 v2:172.21.15.72:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5fd4018480 con 0x55c3427f5a40 2024-04-26T21:57:09.837 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.835+0000 7f5fc2a00640 1 -- 172.21.15.72:0/235123560 <== mon.0 v2:172.21.15.72:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f5fc8018070 con 0x55c3427f5a40 2024-04-26T21:57:09.837 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.835+0000 7f5fe6bd9740 10 client.15858 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T21:57:09.838 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.835+0000 7f5fe6bd9740 10 client.15858 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-04-26T21:57:09.838 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.835+0000 7f5fe6bd9740 20 client.15858 populate_metadata read hostname 'smithi072' 2024-04-26T21:57:09.838 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.835+0000 7f5fe6bd9740 1 --2- 172.21.15.72:0/235123560 >> [v2:172.21.15.164:6838/1636226820,v1:172.21.15.164:6839/1636226820] conn(0x55c34280aa20 0x55c34280cec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:09.838 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.835+0000 7f5fe6bd9740 4 client.15858 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:57:09.838 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.835+0000 7f5fe6bd9740 1 -- 172.21.15.72:0/235123560 --> [v2:172.21.15.164:6838/1636226820,v1:172.21.15.164:6839/1636226820] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c342766880 con 0x55c34280aa20 2024-04-26T21:57:09.839 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.836+0000 7f5fe1e00640 1 --2- 172.21.15.72:0/235123560 >> [v2:172.21.15.164:6838/1636226820,v1:172.21.15.164:6839/1636226820] conn(0x55c34280aa20 0x55c34280cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:09.839 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.837+0000 7f5fe1e00640 1 --2- 172.21.15.72:0/235123560 >> [v2:172.21.15.164:6838/1636226820,v1:172.21.15.164:6839/1636226820] conn(0x55c34280aa20 0x55c34280cec0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=34cf8967774e67e3 server_cookie=b4589a8878b34b23 in_seq=0 out_seq=0 2024-04-26T21:57:09.839 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.837+0000 7f5fc2a00640 10 client.15858 ms_handle_connect on v2:172.21.15.164:6838/1636226820 2024-04-26T21:57:09.839 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.839+0000 7f5fc2a00640 1 -- 172.21.15.72:0/235123560 <== mds.4 v2:172.21.15.164:6838/1636226820 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c342766880 con 0x55c34280aa20 2024-04-26T21:57:09.839 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.839+0000 7f5fc2a00640 10 client.15858 handle_command_reply: tid=0 2024-04-26T21:57:09.840 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.839+0000 7f5fe6bd9740 1 client.15858 shutdown 2024-04-26T21:57:09.840 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.839+0000 7f5fe6bd9740 1 -- 172.21.15.72:0/235123560 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c3427f5a40 msgr2=0x55c3427ff490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:09.840 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.839+0000 7f5fe6bd9740 1 --2- 172.21.15.72:0/235123560 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c3427f5a40 0x55c3427ff490 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f5fc800ce90 tx=0x7f5fc80039c0 comp rx=0 tx=0).stop 2024-04-26T21:57:09.840 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.839+0000 7f5fe6bd9740 1 -- 172.21.15.72:0/235123560 shutdown_connections 2024-04-26T21:57:09.840 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.839+0000 7f5fe6bd9740 1 --2- 172.21.15.72:0/235123560 >> [v2:172.21.15.164:6838/1636226820,v1:172.21.15.164:6839/1636226820] conn(0x55c34280aa20 0x55c34280cec0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:09.840 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.839+0000 7f5fe6bd9740 1 --2- 172.21.15.72:0/235123560 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c3427fabb0 0x55c3428064d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:09.840 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.839+0000 7f5fe6bd9740 1 --2- 172.21.15.72:0/235123560 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c3427f5a40 0x55c3427ff490 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:09.840 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.839+0000 7f5fc2a00640 0 client.15858 ms_handle_reset on v2:172.21.15.164:6838/1636226820 2024-04-26T21:57:09.840 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.839+0000 7f5fe6bd9740 1 --2- 172.21.15.72:0/235123560 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c3427f5050 0x55c3427fef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:09.840 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.839+0000 7f5fe6bd9740 1 -- 172.21.15.72:0/235123560 >> 172.21.15.72:0/235123560 conn(0x55c3427f06c0 msgr2=0x55c3427ef7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:57:09.841 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.840+0000 7f5fe6bd9740 1 -- 172.21.15.72:0/235123560 shutdown_connections 2024-04-26T21:57:09.841 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.840+0000 7f5fe6bd9740 1 -- 172.21.15.72:0/235123560 wait complete. 2024-04-26T21:57:09.841 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.840+0000 7f5fe6bd9740 20 client.15858 trim_cache size 0 max 16384 2024-04-26T21:57:09.864 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.863+0000 7f5fe6bd9740 1 Processor -- start 2024-04-26T21:57:09.864 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.863+0000 7f5fe6bd9740 1 -- start start 2024-04-26T21:57:09.864 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.863+0000 7f5fe6bd9740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c3427f5050 0x55c3427f6bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:09.864 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.863+0000 7f5fe6bd9740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c3427f5a40 0x55c342806490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:09.864 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.863+0000 7f5fe6bd9740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c3427fabb0 0x55c3427f7130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:09.864 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.863+0000 7f5fe6bd9740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f5fdc1abee0 con 0x55c3427f5050 2024-04-26T21:57:09.864 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.863+0000 7f5fe6bd9740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f5fdc1ada20 con 0x55c3427f5a40 2024-04-26T21:57:09.864 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.863+0000 7f5fe6bd9740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f5fdc1ac5b0 con 0x55c3427fabb0 2024-04-26T21:57:09.864 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.864+0000 7f5fe1e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c3427f5050 0x55c3427f6bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:09.865 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.864+0000 7f5fe1e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c3427f5050 0x55c3427f6bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:48334/0 (socket says 172.21.15.72:48334) 2024-04-26T21:57:09.865 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.864+0000 7f5fe1e00640 1 -- 172.21.15.72:0/2215352070 learned_addr learned my addr 172.21.15.72:0/2215352070 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:57:09.865 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.864+0000 7f5fe0a00640 1 --2- 172.21.15.72:0/2215352070 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c3427f5a40 0x55c342806490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:09.867 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.864+0000 7f5fe1e00640 1 -- 172.21.15.72:0/2215352070 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c3427fabb0 msgr2=0x55c3427f7130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:09.867 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.864+0000 7f5fe1e00640 1 --2- 172.21.15.72:0/2215352070 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c3427fabb0 0x55c3427f7130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:09.867 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.864+0000 7f5fe1e00640 1 -- 172.21.15.72:0/2215352070 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c3427f5a40 msgr2=0x55c342806490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:09.867 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.864+0000 7f5fe1e00640 1 --2- 172.21.15.72:0/2215352070 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c3427f5a40 0x55c342806490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:09.867 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.864+0000 7f5fe1e00640 1 -- 172.21.15.72:0/2215352070 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c3425347d0 con 0x55c3427f5050 2024-04-26T21:57:09.867 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.865+0000 7f5fe1e00640 1 --2- 172.21.15.72:0/2215352070 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c3427f5050 0x55c3427f6bf0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f5fd4000f80 tx=0x7f5fd4008270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f2990ade70e9691 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:57:09.867 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.865+0000 7f5fcdc00640 1 -- 172.21.15.72:0/2215352070 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fd4029020 con 0x55c3427f5050 2024-04-26T21:57:09.867 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.865+0000 7f5fcdc00640 1 -- 172.21.15.72:0/2215352070 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5fd4018910 con 0x55c3427f5050 2024-04-26T21:57:09.867 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.865+0000 7f5fcdc00640 1 -- 172.21.15.72:0/2215352070 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fd400f630 con 0x55c3427f5050 2024-04-26T21:57:09.867 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.866+0000 7f5fe6bd9740 1 -- 172.21.15.72:0/2215352070 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c3427f5050 msgr2=0x55c3427f6bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:09.867 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.866+0000 7f5fe6bd9740 1 --2- 172.21.15.72:0/2215352070 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c3427f5050 0x55c3427f6bf0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f5fd4000f80 tx=0x7f5fd4008270 comp rx=0 tx=0).stop 2024-04-26T21:57:09.868 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.866+0000 7f5fe6bd9740 1 -- 172.21.15.72:0/2215352070 shutdown_connections 2024-04-26T21:57:09.868 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.866+0000 7f5fe6bd9740 1 --2- 172.21.15.72:0/2215352070 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c3427fabb0 0x55c3427f7130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:09.868 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.866+0000 7f5fe6bd9740 1 --2- 172.21.15.72:0/2215352070 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c3427f5a40 0x55c342806490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:09.868 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.866+0000 7f5fe6bd9740 1 --2- 172.21.15.72:0/2215352070 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c3427f5050 0x55c3427f6bf0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:09.868 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.866+0000 7f5fe6bd9740 1 -- 172.21.15.72:0/2215352070 >> 172.21.15.72:0/2215352070 conn(0x55c3427f06c0 msgr2=0x55c3427f64e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:57:09.868 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.866+0000 7f5fe6bd9740 1 -- 172.21.15.72:0/2215352070 shutdown_connections 2024-04-26T21:57:09.868 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.866+0000 7f5fe6bd9740 1 -- 172.21.15.72:0/2215352070 wait complete. 2024-04-26T21:57:09.868 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.867+0000 7f5fe6bd9740 1 -- start start 2024-04-26T21:57:09.868 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.867+0000 7f5fe6bd9740 1 Processor -- start 2024-04-26T21:57:09.871 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.867+0000 7f5fe6bd9740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c3427f5050 0x55c342804150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:09.871 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.867+0000 7f5fe6bd9740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c3427f5a40 0x55c342804690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:09.871 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.867+0000 7f5fe6bd9740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c3427fabb0 0x55c3428056a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:09.871 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.867+0000 7f5fe6bd9740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f5fdc1ad350 con 0x55c3427fabb0 2024-04-26T21:57:09.871 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.867+0000 7f5fe6bd9740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f5fdc1acc80 con 0x55c3427f5a40 2024-04-26T21:57:09.871 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.867+0000 7f5fe6bd9740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f5fdc1a9ec0 con 0x55c3427f5050 2024-04-26T21:57:09.871 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.867+0000 7f5fe6bd9740 10 client.0 _ll_register_callbacks cb 0x7f5fc8004120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:57:09.871 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.868+0000 7f5fe1e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c3427fabb0 0x55c3428056a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:09.871 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.868+0000 7f5fe1e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c3427fabb0 0x55c3428056a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:48346/0 (socket says 172.21.15.72:48346) 2024-04-26T21:57:09.871 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.868+0000 7f5fe1e00640 1 -- 172.21.15.72:0/105890914 learned_addr learned my addr 172.21.15.72:0/105890914 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:57:09.872 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.868+0000 7f5fe0a00640 1 --2- 172.21.15.72:0/105890914 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c3427f5a40 0x55c342804690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:09.872 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.868+0000 7f5fe1e00640 1 --2- 172.21.15.72:0/105890914 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c3427f5050 0x55c342804150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:09.872 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.868+0000 7f5fe1e00640 1 -- 172.21.15.72:0/105890914 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c3427f5050 msgr2=0x55c342804150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:09.872 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.868+0000 7f5fe1e00640 1 --2- 172.21.15.72:0/105890914 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c3427f5050 0x55c342804150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:09.872 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.868+0000 7f5fe1e00640 1 -- 172.21.15.72:0/105890914 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c3427f5a40 msgr2=0x55c342804690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:09.872 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.868+0000 7f5fe1e00640 1 --2- 172.21.15.72:0/105890914 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c3427f5a40 0x55c342804690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:09.872 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.868+0000 7f5fe1e00640 1 -- 172.21.15.72:0/105890914 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c3426ff2a0 con 0x55c3427fabb0 2024-04-26T21:57:09.872 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.869+0000 7f5fe1e00640 1 --2- 172.21.15.72:0/105890914 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c3427fabb0 0x55c3428056a0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f5fd4008fd0 tx=0x7f5fd401b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45ad19de3847623d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:57:09.872 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.869+0000 7f5fc2a00640 10 client.0 ms_handle_connect on v2:172.21.15.72:3300/0 2024-04-26T21:57:09.872 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.869+0000 7f5fc2a00640 1 -- 172.21.15.72:0/105890914 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fd4039030 con 0x55c3427fabb0 2024-04-26T21:57:09.872 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.869+0000 7f5fc2a00640 1 -- 172.21.15.72:0/105890914 <== mon.0 v2:172.21.15.72:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7f5fd4018480 con 0x55c3427fabb0 2024-04-26T21:57:09.872 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.869+0000 7f5fe6bd9740 1 -- 172.21.15.72:0/105890914 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5fd4018910 con 0x55c3427fabb0 2024-04-26T21:57:09.872 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.869+0000 7f5fc2a00640 1 -- 172.21.15.72:0/105890914 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f5fd4043050 con 0x55c3427fabb0 2024-04-26T21:57:09.873 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.869+0000 7f5fe6bd9740 10 client.15864 fetch_fsmap learned FSMap version 27 2024-04-26T21:57:09.873 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.869+0000 7f5fe6bd9740 1 -- 172.21.15.72:0/105890914 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c3428713f0 con 0x55c3427fabb0 2024-04-26T21:57:09.873 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.870+0000 7f5fc2a00640 1 -- 172.21.15.72:0/105890914 <== mon.0 v2:172.21.15.72:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5fd4018910 con 0x55c3427fabb0 2024-04-26T21:57:09.873 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.870+0000 7f5fc2a00640 1 -- 172.21.15.72:0/105890914 <== mon.0 v2:172.21.15.72:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f5fd400faf0 con 0x55c3427fabb0 2024-04-26T21:57:09.873 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.870+0000 7f5fe6bd9740 10 client.15864 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T21:57:09.873 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.870+0000 7f5fe6bd9740 10 client.15864 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-04-26T21:57:09.873 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.870+0000 7f5fe6bd9740 20 client.15864 populate_metadata read hostname 'smithi072' 2024-04-26T21:57:09.873 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.870+0000 7f5fe6bd9740 1 --2- 172.21.15.72:0/105890914 >> [v2:172.21.15.164:6838/1636226820,v1:172.21.15.164:6839/1636226820] conn(0x55c342871b90 0x55c342873f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:09.873 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.870+0000 7f5fe6bd9740 4 client.15864 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-26T21:57:09.873 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.870+0000 7f5fe6bd9740 1 -- 172.21.15.72:0/105890914 --> [v2:172.21.15.164:6838/1636226820,v1:172.21.15.164:6839/1636226820] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c3427606f0 con 0x55c342871b90 2024-04-26T21:57:09.873 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.871+0000 7f5fe0a00640 1 --2- 172.21.15.72:0/105890914 >> [v2:172.21.15.164:6838/1636226820,v1:172.21.15.164:6839/1636226820] conn(0x55c342871b90 0x55c342873f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:09.875 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.872+0000 7f5fe0a00640 1 --2- 172.21.15.72:0/105890914 >> [v2:172.21.15.164:6838/1636226820,v1:172.21.15.164:6839/1636226820] conn(0x55c342871b90 0x55c342873f80 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=54ed545dfec7daf server_cookie=d9a62531ef12c632 in_seq=0 out_seq=0 2024-04-26T21:57:09.875 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.872+0000 7f5fc2a00640 10 client.15864 ms_handle_connect on v2:172.21.15.164:6838/1636226820 2024-04-26T21:57:09.875 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.872+0000 7f5fc2a00640 1 -- 172.21.15.72:0/105890914 <== mds.4 v2:172.21.15.164:6838/1636226820 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c3427606f0 con 0x55c342871b90 2024-04-26T21:57:09.875 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.872+0000 7f5fc2a00640 10 client.15864 handle_command_reply: tid=0 2024-04-26T21:57:09.875 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.872+0000 7f5fe6bd9740 1 client.15864 shutdown 2024-04-26T21:57:09.875 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.872+0000 7f5fe6bd9740 1 -- 172.21.15.72:0/105890914 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c3427fabb0 msgr2=0x55c3428056a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:09.875 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.872+0000 7f5fe6bd9740 1 --2- 172.21.15.72:0/105890914 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c3427fabb0 0x55c3428056a0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f5fd4008fd0 tx=0x7f5fd401b040 comp rx=0 tx=0).stop 2024-04-26T21:57:09.875 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.872+0000 7f5fe6bd9740 1 -- 172.21.15.72:0/105890914 shutdown_connections 2024-04-26T21:57:09.875 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.872+0000 7f5fe6bd9740 1 --2- 172.21.15.72:0/105890914 >> [v2:172.21.15.164:6838/1636226820,v1:172.21.15.164:6839/1636226820] conn(0x55c342871b90 0x55c342873f80 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:09.875 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.872+0000 7f5fe6bd9740 1 --2- 172.21.15.72:0/105890914 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c3427fabb0 0x55c3428056a0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:09.876 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.872+0000 7f5fe6bd9740 1 --2- 172.21.15.72:0/105890914 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c3427f5a40 0x55c342804690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:09.876 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.872+0000 7f5fe6bd9740 1 --2- 172.21.15.72:0/105890914 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c3427f5050 0x55c342804150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:09.876 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.872+0000 7f5fe6bd9740 1 -- 172.21.15.72:0/105890914 >> 172.21.15.72:0/105890914 conn(0x55c3427f06c0 msgr2=0x55c3427f6d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:57:09.876 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.872+0000 7f5fc2a00640 0 client.15864 ms_handle_reset on v2:172.21.15.164:6838/1636226820 2024-04-26T21:57:09.876 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.872+0000 7f5fe6bd9740 1 -- 172.21.15.72:0/105890914 shutdown_connections 2024-04-26T21:57:09.876 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.873+0000 7f5fe6bd9740 1 -- 172.21.15.72:0/105890914 wait complete. 2024-04-26T21:57:09.876 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.873+0000 7f5fe6bd9740 20 client.15864 trim_cache size 0 max 16384 2024-04-26T21:57:09.876 INFO:teuthology.orchestra.run.smithi072.stdout:[] 2024-04-26T21:57:09.877 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.873+0000 7f5fe2800640 1 -- 172.21.15.72:0/3762423934 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f5fc40a3c40 msgr2=0x7f5fc40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:09.877 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.873+0000 7f5fe2800640 1 --2- 172.21.15.72:0/3762423934 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f5fc40a3c40 0x7f5fc40a6100 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x55c3427f5e40 tx=0x7f5fd002a400 comp rx=0 tx=0).stop 2024-04-26T21:57:09.877 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.873+0000 7f5fe2800640 1 -- 172.21.15.72:0/3762423934 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5fdc1028c0 msgr2=0x7f5fdc19ab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:09.877 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.873+0000 7f5fe2800640 1 --2- 172.21.15.72:0/3762423934 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5fdc1028c0 0x7f5fdc19ab30 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f5fd0031460 tx=0x7f5fd0003d50 comp rx=0 tx=0).stop 2024-04-26T21:57:09.877 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.874+0000 7f5fe2800640 1 -- 172.21.15.72:0/3762423934 shutdown_connections 2024-04-26T21:57:09.877 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.874+0000 7f5fe2800640 1 --2- 172.21.15.72:0/3762423934 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5fdc104470 0x7f5fdc1a20e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:09.877 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.874+0000 7f5fe2800640 1 --2- 172.21.15.72:0/3762423934 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f5fc40a3c40 0x7f5fc40a6100 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:09.878 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.874+0000 7f5fe2800640 1 --2- 172.21.15.72:0/3762423934 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5fdc103ab0 0x7f5fdc19b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:09.878 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.874+0000 7f5fe2800640 1 --2- 172.21.15.72:0/3762423934 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5fdc1028c0 0x7f5fdc19ab30 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:09.878 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.874+0000 7f5fe2800640 1 -- 172.21.15.72:0/3762423934 >> 172.21.15.72:0/3762423934 conn(0x7f5fdc0fde30 msgr2=0x7f5fdc074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:57:09.878 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.874+0000 7f5fe2800640 1 -- 172.21.15.72:0/3762423934 shutdown_connections 2024-04-26T21:57:09.878 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:09.874+0000 7f5fe2800640 1 -- 172.21.15.72:0/3762423934 wait complete. 2024-04-26T21:57:09.893 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-26T21:57:10.120 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.119+0000 7f6954400640 1 -- 172.21.15.72:0/2719675192 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f694c103b70 msgr2=0x7f694c103ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:10.120 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.119+0000 7f6954400640 1 --2- 172.21.15.72:0/2719675192 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f694c103b70 0x7f694c103ff0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f6938009900 tx=0x7f693802a3f0 comp rx=0 tx=0).stop 2024-04-26T21:57:10.120 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.119+0000 7f6954400640 1 -- 172.21.15.72:0/2719675192 shutdown_connections 2024-04-26T21:57:10.120 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.119+0000 7f6954400640 1 --2- 172.21.15.72:0/2719675192 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f694c104530 0x7f694c10adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:10.120 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.119+0000 7f6954400640 1 --2- 172.21.15.72:0/2719675192 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f694c103b70 0x7f694c103ff0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:10.120 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.119+0000 7f6954400640 1 --2- 172.21.15.72:0/2719675192 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f694c102980 0x7f694c102da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:10.120 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.119+0000 7f6954400640 1 -- 172.21.15.72:0/2719675192 >> 172.21.15.72:0/2719675192 conn(0x7f694c0fdf10 msgr2=0x7f694c100330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:57:10.120 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.119+0000 7f6954400640 1 -- 172.21.15.72:0/2719675192 shutdown_connections 2024-04-26T21:57:10.120 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.119+0000 7f6954400640 1 -- 172.21.15.72:0/2719675192 wait complete. 2024-04-26T21:57:10.123 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.120+0000 7f6954400640 1 Processor -- start 2024-04-26T21:57:10.123 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.121+0000 7f6954400640 1 -- start start 2024-04-26T21:57:10.123 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.121+0000 7f6954400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f694c102980 0x7f694c19abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:10.123 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.121+0000 7f6954400640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f694c103b70 0x7f694c19b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:10.123 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.121+0000 7f6954400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f694c104530 0x7f694c1a21a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:10.123 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.121+0000 7f6954400640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f694c10e390 con 0x7f694c104530 2024-04-26T21:57:10.123 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.121+0000 7f6954400640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f694c10e210 con 0x7f694c103b70 2024-04-26T21:57:10.123 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.121+0000 7f6954400640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f694c10e510 con 0x7f694c102980 2024-04-26T21:57:10.123 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.121+0000 7f6952400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f694c104530 0x7f694c1a21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:10.123 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.121+0000 7f6952400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f694c104530 0x7f694c1a21a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:48368/0 (socket says 172.21.15.72:48368) 2024-04-26T21:57:10.123 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.121+0000 7f6952400640 1 -- 172.21.15.72:0/3639077620 learned_addr learned my addr 172.21.15.72:0/3639077620 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:57:10.124 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.121+0000 7f6951000640 1 --2- 172.21.15.72:0/3639077620 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f694c103b70 0x7f694c19b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:10.124 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.121+0000 7f6951a00640 1 --2- 172.21.15.72:0/3639077620 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f694c102980 0x7f694c19abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:10.124 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.122+0000 7f6952400640 1 -- 172.21.15.72:0/3639077620 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f694c102980 msgr2=0x7f694c19abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:10.124 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.122+0000 7f6952400640 1 --2- 172.21.15.72:0/3639077620 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f694c102980 0x7f694c19abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:10.124 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.122+0000 7f6952400640 1 -- 172.21.15.72:0/3639077620 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f694c103b70 msgr2=0x7f694c19b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:10.124 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.122+0000 7f6952400640 1 --2- 172.21.15.72:0/3639077620 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f694c103b70 0x7f694c19b130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:10.124 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.122+0000 7f6952400640 1 -- 172.21.15.72:0/3639077620 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f694c1a2810 con 0x7f694c104530 2024-04-26T21:57:10.124 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.122+0000 7f6951000640 1 --2- 172.21.15.72:0/3639077620 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f694c103b70 0x7f694c19b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:57:10.124 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.122+0000 7f6952400640 1 --2- 172.21.15.72:0/3639077620 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f694c104530 0x7f694c1a21a0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f694400cce0 tx=0x7f694400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:57:10.124 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.122+0000 7f694a000640 1 -- 172.21.15.72:0/3639077620 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f694400f450 con 0x7f694c104530 2024-04-26T21:57:10.124 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.122+0000 7f694a000640 1 -- 172.21.15.72:0/3639077620 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f694400fa80 con 0x7f694c104530 2024-04-26T21:57:10.124 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.122+0000 7f694a000640 1 -- 172.21.15.72:0/3639077620 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6944018ad0 con 0x7f694c104530 2024-04-26T21:57:10.124 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.122+0000 7f6954400640 1 -- 172.21.15.72:0/3639077620 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f694c1a2b00 con 0x7f694c104530 2024-04-26T21:57:10.124 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.123+0000 7f6954400640 1 -- 172.21.15.72:0/3639077620 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f694c1a3050 con 0x7f694c104530 2024-04-26T21:57:10.126 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.126+0000 7f694a000640 1 -- 172.21.15.72:0/3639077620 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f694400f5f0 con 0x7f694c104530 2024-04-26T21:57:10.127 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.127+0000 7f6957217740 1 Processor -- start 2024-04-26T21:57:10.127 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.127+0000 7f6957217740 1 -- start start 2024-04-26T21:57:10.128 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.127+0000 7f6957217740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558ad6c39ea0 0x558ad6c3a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:10.128 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.127+0000 7f6957217740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558ad6c3a890 0x558ad6c3f390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:10.128 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.127+0000 7f6957217740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558ad6c3fa00 0x558ad6c41e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:10.128 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.127+0000 7f6957217740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x558ad6b9dc70 con 0x558ad6c39ea0 2024-04-26T21:57:10.128 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.127+0000 7f6957217740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x558ad6b56980 con 0x558ad6c3fa00 2024-04-26T21:57:10.128 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.127+0000 7f6957217740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x558ad6b99c30 con 0x558ad6c3a890 2024-04-26T21:57:10.128 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.128+0000 7f6951000640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558ad6c39ea0 0x558ad6c3a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:10.129 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.128+0000 7f6951000640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558ad6c39ea0 0x558ad6c3a2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:48372/0 (socket says 172.21.15.72:48372) 2024-04-26T21:57:10.129 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.128+0000 7f6951000640 1 -- 172.21.15.72:0/1029416967 learned_addr learned my addr 172.21.15.72:0/1029416967 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:57:10.129 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.128+0000 7f6951a00640 1 --2- 172.21.15.72:0/1029416967 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558ad6c3a890 0x558ad6c3f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:10.129 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.128+0000 7f694a000640 1 --2- 172.21.15.72:0/3639077620 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f69340a3cc0 0x7f69340a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:10.129 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.128+0000 7f6951000640 1 -- 172.21.15.72:0/1029416967 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558ad6c3a890 msgr2=0x558ad6c3f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:10.129 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.128+0000 7f6951000640 1 --2- 172.21.15.72:0/1029416967 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558ad6c3a890 0x558ad6c3f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:10.129 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.128+0000 7f6951000640 1 -- 172.21.15.72:0/1029416967 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558ad6c3fa00 msgr2=0x558ad6c41e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:10.129 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.128+0000 7f6951000640 1 --2- 172.21.15.72:0/1029416967 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558ad6c3fa00 0x558ad6c41e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:10.129 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.128+0000 7f6951000640 1 -- 172.21.15.72:0/1029416967 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ad6b2e7b0 con 0x558ad6c39ea0 2024-04-26T21:57:10.130 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.129+0000 7f694a000640 1 -- 172.21.15.72:0/3639077620 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7f69440d0e20 con 0x7f694c104530 2024-04-26T21:57:10.131 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.129+0000 7f6952400640 1 --2- 172.21.15.72:0/3639077620 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f69340a3cc0 0x7f69340a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:10.131 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.129+0000 7f6951000640 1 --2- 172.21.15.72:0/1029416967 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558ad6c39ea0 0x558ad6c3a2c0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f6938005e80 tx=0x7f6938004310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d52c85b9e75040eb server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:57:10.131 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.129+0000 7f693ea00640 1 -- 172.21.15.72:0/1029416967 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6938041020 con 0x558ad6c39ea0 2024-04-26T21:57:10.131 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.129+0000 7f693ea00640 1 -- 172.21.15.72:0/1029416967 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6938003c80 con 0x558ad6c39ea0 2024-04-26T21:57:10.131 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.129+0000 7f693ea00640 1 -- 172.21.15.72:0/1029416967 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6938035070 con 0x558ad6c39ea0 2024-04-26T21:57:10.131 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.130+0000 7f6957217740 1 -- 172.21.15.72:0/1029416967 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558ad6c39ea0 msgr2=0x558ad6c3a2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:10.131 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.130+0000 7f6957217740 1 --2- 172.21.15.72:0/1029416967 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558ad6c39ea0 0x558ad6c3a2c0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f6938005e80 tx=0x7f6938004310 comp rx=0 tx=0).stop 2024-04-26T21:57:10.131 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.130+0000 7f6952400640 1 --2- 172.21.15.72:0/3639077620 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f69340a3cc0 0x7f69340a6180 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f6944099380 tx=0x7f694400ff30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:57:10.131 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.130+0000 7f6957217740 1 -- 172.21.15.72:0/1029416967 shutdown_connections 2024-04-26T21:57:10.131 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.130+0000 7f6957217740 1 --2- 172.21.15.72:0/1029416967 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558ad6c3fa00 0x558ad6c41e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:10.131 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.130+0000 7f6957217740 1 --2- 172.21.15.72:0/1029416967 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558ad6c3a890 0x558ad6c3f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:10.132 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.130+0000 7f6957217740 1 --2- 172.21.15.72:0/1029416967 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558ad6c39ea0 0x558ad6c3a2c0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:10.132 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.130+0000 7f6957217740 1 -- 172.21.15.72:0/1029416967 >> 172.21.15.72:0/1029416967 conn(0x558ad6c35510 msgr2=0x558ad6c35910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:57:10.132 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.130+0000 7f6957217740 1 -- 172.21.15.72:0/1029416967 shutdown_connections 2024-04-26T21:57:10.132 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.130+0000 7f6957217740 1 -- 172.21.15.72:0/1029416967 wait complete. 2024-04-26T21:57:10.132 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.131+0000 7f6957217740 1 -- start start 2024-04-26T21:57:10.135 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.131+0000 7f6957217740 1 Processor -- start 2024-04-26T21:57:10.135 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.132+0000 7f6957217740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558ad6c39ea0 0x558ad6c43e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:10.135 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.132+0000 7f6957217740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558ad6c3a890 0x558ad6c44390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:10.135 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.132+0000 7f6957217740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558ad6c3fa00 0x558ad6c4b360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:10.135 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.132+0000 7f6957217740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x558ad6b9cf80 con 0x558ad6c39ea0 2024-04-26T21:57:10.135 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.132+0000 7f6957217740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x558ad6b4a260 con 0x558ad6c3a890 2024-04-26T21:57:10.135 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.132+0000 7f6957217740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x558ad6ba22a0 con 0x558ad6c3fa00 2024-04-26T21:57:10.135 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.132+0000 7f6957217740 10 client.0 _ll_register_callbacks cb 0x558ad6b94960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:57:10.135 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.132+0000 7f6951000640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558ad6c39ea0 0x558ad6c43e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:10.135 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.132+0000 7f6951000640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558ad6c39ea0 0x558ad6c43e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:48386/0 (socket says 172.21.15.72:48386) 2024-04-26T21:57:10.135 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.132+0000 7f6951000640 1 -- 172.21.15.72:0/3150392155 learned_addr learned my addr 172.21.15.72:0/3150392155 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:57:10.135 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.132+0000 7f6951a00640 1 --2- 172.21.15.72:0/3150392155 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558ad6c3a890 0x558ad6c44390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:10.135 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.132+0000 7f6951000640 1 -- 172.21.15.72:0/3150392155 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558ad6c3fa00 msgr2=0x558ad6c4b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:10.136 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.132+0000 7f6951000640 1 --2- 172.21.15.72:0/3150392155 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558ad6c3fa00 0x558ad6c4b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:10.136 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.132+0000 7f6951000640 1 -- 172.21.15.72:0/3150392155 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558ad6c3a890 msgr2=0x558ad6c44390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:10.136 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.132+0000 7f6951000640 1 --2- 172.21.15.72:0/3150392155 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558ad6c3a890 0x558ad6c44390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:10.136 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.132+0000 7f6951000640 1 -- 172.21.15.72:0/3150392155 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ad6b2c1c0 con 0x558ad6c39ea0 2024-04-26T21:57:10.136 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.133+0000 7f6951000640 1 --2- 172.21.15.72:0/3150392155 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558ad6c39ea0 0x558ad6c43e50 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f6938031660 tx=0x7f6938031690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9d44ae839b51a74 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:57:10.136 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.133+0000 7f6933400640 10 client.0 ms_handle_connect on v2:172.21.15.72:3300/0 2024-04-26T21:57:10.136 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.133+0000 7f6933400640 1 -- 172.21.15.72:0/3150392155 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6938041030 con 0x558ad6c39ea0 2024-04-26T21:57:10.136 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.133+0000 7f6933400640 1 -- 172.21.15.72:0/3150392155 <== mon.0 v2:172.21.15.72:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7f69380035f0 con 0x558ad6c39ea0 2024-04-26T21:57:10.136 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.133+0000 7f6957217740 1 -- 172.21.15.72:0/3150392155 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6938003c80 con 0x558ad6c39ea0 2024-04-26T21:57:10.136 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.133+0000 7f6933400640 1 -- 172.21.15.72:0/3150392155 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6938054030 con 0x558ad6c39ea0 2024-04-26T21:57:10.136 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.133+0000 7f6957217740 10 client.15876 fetch_fsmap learned FSMap version 27 2024-04-26T21:57:10.136 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.133+0000 7f6957217740 1 -- 172.21.15.72:0/3150392155 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x558ad68af3f0 con 0x558ad6c39ea0 2024-04-26T21:57:10.136 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.134+0000 7f6933400640 1 -- 172.21.15.72:0/3150392155 <== mon.0 v2:172.21.15.72:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6938003c80 con 0x558ad6c39ea0 2024-04-26T21:57:10.136 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.134+0000 7f6933400640 1 -- 172.21.15.72:0/3150392155 <== mon.0 v2:172.21.15.72:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f693804fa20 con 0x558ad6c39ea0 2024-04-26T21:57:10.137 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.134+0000 7f6957217740 10 client.15876 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T21:57:10.137 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.134+0000 7f6957217740 10 client.15876 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-26T21:57:10.137 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.134+0000 7f6957217740 20 client.15876 populate_metadata read hostname 'smithi072' 2024-04-26T21:57:10.137 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.134+0000 7f6957217740 1 --2- 172.21.15.72:0/3150392155 >> [v2:172.21.15.76:6832/3869483750,v1:172.21.15.76:6833/3869483750] conn(0x558ad6c4f4c0 0x558ad6c518b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:10.137 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.134+0000 7f6957217740 4 client.15876 mds_command: new command op to 34406 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:57:10.137 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.134+0000 7f6957217740 1 -- 172.21.15.72:0/3150392155 --> [v2:172.21.15.76:6832/3869483750,v1:172.21.15.76:6833/3869483750] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558ad6bab880 con 0x558ad6c4f4c0 2024-04-26T21:57:10.137 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.135+0000 7f6951a00640 1 --2- 172.21.15.72:0/3150392155 >> [v2:172.21.15.76:6832/3869483750,v1:172.21.15.76:6833/3869483750] conn(0x558ad6c4f4c0 0x558ad6c518b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:10.138 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.136+0000 7f6951a00640 1 --2- 172.21.15.72:0/3150392155 >> [v2:172.21.15.76:6832/3869483750,v1:172.21.15.76:6833/3869483750] conn(0x558ad6c4f4c0 0x558ad6c518b0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=49f67d7c03780c4f server_cookie=999093cd5f6f4af7 in_seq=0 out_seq=0 2024-04-26T21:57:10.138 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.136+0000 7f6933400640 10 client.15876 ms_handle_connect on v2:172.21.15.76:6832/3869483750 2024-04-26T21:57:10.138 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.137+0000 7f6933400640 1 -- 172.21.15.72:0/3150392155 <== mds.3 v2:172.21.15.76:6832/3869483750 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558ad6bab880 con 0x558ad6c4f4c0 2024-04-26T21:57:10.139 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.137+0000 7f6933400640 10 client.15876 handle_command_reply: tid=0 2024-04-26T21:57:10.139 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.137+0000 7f6957217740 1 client.15876 shutdown 2024-04-26T21:57:10.139 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.138+0000 7f6957217740 1 -- 172.21.15.72:0/3150392155 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558ad6c39ea0 msgr2=0x558ad6c43e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:10.139 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.138+0000 7f6957217740 1 --2- 172.21.15.72:0/3150392155 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558ad6c39ea0 0x558ad6c43e50 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f6938031660 tx=0x7f6938031690 comp rx=0 tx=0).stop 2024-04-26T21:57:10.139 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.138+0000 7f6957217740 1 -- 172.21.15.72:0/3150392155 shutdown_connections 2024-04-26T21:57:10.139 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.138+0000 7f6957217740 1 --2- 172.21.15.72:0/3150392155 >> [v2:172.21.15.76:6832/3869483750,v1:172.21.15.76:6833/3869483750] conn(0x558ad6c4f4c0 0x558ad6c518b0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:10.139 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.138+0000 7f6957217740 1 --2- 172.21.15.72:0/3150392155 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558ad6c3fa00 0x558ad6c4b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:10.139 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.138+0000 7f6957217740 1 --2- 172.21.15.72:0/3150392155 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558ad6c3a890 0x558ad6c44390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:10.139 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.138+0000 7f6957217740 1 --2- 172.21.15.72:0/3150392155 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558ad6c39ea0 0x558ad6c43e50 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:10.139 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.138+0000 7f6957217740 1 -- 172.21.15.72:0/3150392155 >> 172.21.15.72:0/3150392155 conn(0x558ad6c35510 msgr2=0x558ad6c34690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:57:10.139 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.138+0000 7f6933400640 0 client.15876 ms_handle_reset on v2:172.21.15.76:6832/3869483750 2024-04-26T21:57:10.139 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.138+0000 7f6957217740 1 -- 172.21.15.72:0/3150392155 shutdown_connections 2024-04-26T21:57:10.139 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.138+0000 7f6957217740 1 -- 172.21.15.72:0/3150392155 wait complete. 2024-04-26T21:57:10.139 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.138+0000 7f6957217740 20 client.15876 trim_cache size 0 max 16384 2024-04-26T21:57:10.156 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.155+0000 7f6957217740 1 Processor -- start 2024-04-26T21:57:10.156 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.156+0000 7f6957217740 1 -- start start 2024-04-26T21:57:10.156 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.156+0000 7f6957217740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558ad6c39ea0 0x558ad6c442f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:10.156 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.156+0000 7f6957217740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558ad6c3a890 0x558ad6c4b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:10.156 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.156+0000 7f6957217740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558ad6c3fa00 0x558ad6c38b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:10.156 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.156+0000 7f6957217740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f694c1abf50 con 0x558ad6c39ea0 2024-04-26T21:57:10.156 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.156+0000 7f6957217740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f694c1ada90 con 0x558ad6c3a890 2024-04-26T21:57:10.156 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.156+0000 7f6957217740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f694c1ac620 con 0x558ad6c3fa00 2024-04-26T21:57:10.157 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.156+0000 7f6951000640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558ad6c39ea0 0x558ad6c442f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:10.158 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.156+0000 7f6951000640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558ad6c39ea0 0x558ad6c442f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:48402/0 (socket says 172.21.15.72:48402) 2024-04-26T21:57:10.158 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.156+0000 7f6951000640 1 -- 172.21.15.72:0/637928860 learned_addr learned my addr 172.21.15.72:0/637928860 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:57:10.158 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.157+0000 7f6951a00640 1 --2- 172.21.15.72:0/637928860 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558ad6c3a890 0x558ad6c4b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:10.158 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.157+0000 7f6951000640 1 -- 172.21.15.72:0/637928860 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558ad6c3fa00 msgr2=0x558ad6c38b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:10.158 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.157+0000 7f6951000640 1 --2- 172.21.15.72:0/637928860 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558ad6c3fa00 0x558ad6c38b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:10.158 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.157+0000 7f6951000640 1 -- 172.21.15.72:0/637928860 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558ad6c3a890 msgr2=0x558ad6c4b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:10.158 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.157+0000 7f6951000640 1 --2- 172.21.15.72:0/637928860 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558ad6c3a890 0x558ad6c4b2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:10.158 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.157+0000 7f6951000640 1 -- 172.21.15.72:0/637928860 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ad69797d0 con 0x558ad6c39ea0 2024-04-26T21:57:10.158 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.157+0000 7f6951000640 1 --2- 172.21.15.72:0/637928860 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558ad6c39ea0 0x558ad6c442f0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f6938040040 tx=0x7f6938005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac5f08f8f512c0fa server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:57:10.158 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.157+0000 7f693e000640 1 -- 172.21.15.72:0/637928860 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6938041040 con 0x558ad6c39ea0 2024-04-26T21:57:10.158 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.158+0000 7f693e000640 1 -- 172.21.15.72:0/637928860 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f693804a070 con 0x558ad6c39ea0 2024-04-26T21:57:10.158 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.158+0000 7f693e000640 1 -- 172.21.15.72:0/637928860 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6938035070 con 0x558ad6c39ea0 2024-04-26T21:57:10.160 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.158+0000 7f6957217740 1 -- 172.21.15.72:0/637928860 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558ad6c39ea0 msgr2=0x558ad6c442f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:10.160 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.158+0000 7f6957217740 1 --2- 172.21.15.72:0/637928860 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558ad6c39ea0 0x558ad6c442f0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f6938040040 tx=0x7f6938005fd0 comp rx=0 tx=0).stop 2024-04-26T21:57:10.160 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.158+0000 7f6957217740 1 -- 172.21.15.72:0/637928860 shutdown_connections 2024-04-26T21:57:10.160 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.158+0000 7f6957217740 1 --2- 172.21.15.72:0/637928860 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558ad6c3fa00 0x558ad6c38b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:10.161 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.158+0000 7f6957217740 1 --2- 172.21.15.72:0/637928860 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558ad6c3a890 0x558ad6c4b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:10.161 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.158+0000 7f6957217740 1 --2- 172.21.15.72:0/637928860 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558ad6c39ea0 0x558ad6c442f0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:10.161 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.158+0000 7f6957217740 1 -- 172.21.15.72:0/637928860 >> 172.21.15.72:0/637928860 conn(0x558ad6c35510 msgr2=0x558ad6c38420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:57:10.161 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.159+0000 7f6957217740 1 -- 172.21.15.72:0/637928860 shutdown_connections 2024-04-26T21:57:10.161 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.159+0000 7f6957217740 1 -- 172.21.15.72:0/637928860 wait complete. 2024-04-26T21:57:10.161 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.159+0000 7f6957217740 1 -- start start 2024-04-26T21:57:10.161 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.159+0000 7f6957217740 1 Processor -- start 2024-04-26T21:57:10.161 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.160+0000 7f6957217740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558ad6c39ea0 0x558ad6c4b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:10.161 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.160+0000 7f6957217740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558ad6c3a890 0x558ad6c47ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:10.161 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.160+0000 7f6957217740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558ad6c3fa00 0x558ad6c48220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:10.161 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.160+0000 7f6957217740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f694c1ad3c0 con 0x558ad6c39ea0 2024-04-26T21:57:10.161 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.160+0000 7f6957217740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f694c1accf0 con 0x558ad6c3fa00 2024-04-26T21:57:10.161 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.160+0000 7f6957217740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f694c1a9f30 con 0x558ad6c3a890 2024-04-26T21:57:10.161 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.160+0000 7f6957217740 10 client.0 _ll_register_callbacks cb 0x7f6938004260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:57:10.162 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.160+0000 7f6951000640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558ad6c39ea0 0x558ad6c4b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:10.162 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.160+0000 7f6951000640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558ad6c39ea0 0x558ad6c4b1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:48416/0 (socket says 172.21.15.72:48416) 2024-04-26T21:57:10.162 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.160+0000 7f6951000640 1 -- 172.21.15.72:0/4289302975 learned_addr learned my addr 172.21.15.72:0/4289302975 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:57:10.162 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.160+0000 7f6951000640 1 -- 172.21.15.72:0/4289302975 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558ad6c3a890 msgr2=0x558ad6c47ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:10.162 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.160+0000 7f6951000640 1 --2- 172.21.15.72:0/4289302975 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558ad6c3a890 0x558ad6c47ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:10.162 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.160+0000 7f6951000640 1 -- 172.21.15.72:0/4289302975 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558ad6c3fa00 msgr2=0x558ad6c48220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:10.162 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.160+0000 7f6951000640 1 --2- 172.21.15.72:0/4289302975 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558ad6c3fa00 0x558ad6c48220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:10.162 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.160+0000 7f6951000640 1 -- 172.21.15.72:0/4289302975 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ad6b442a0 con 0x558ad6c39ea0 2024-04-26T21:57:10.162 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.161+0000 7f6951000640 1 --2- 172.21.15.72:0/4289302975 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558ad6c39ea0 0x558ad6c4b1d0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f6938005a80 tx=0x7f6938000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12eae203298d32ec server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:57:10.162 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.161+0000 7f6933400640 10 client.0 ms_handle_connect on v2:172.21.15.72:3300/0 2024-04-26T21:57:10.164 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.161+0000 7f6933400640 1 -- 172.21.15.72:0/4289302975 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6938035040 con 0x558ad6c39ea0 2024-04-26T21:57:10.164 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.161+0000 7f6933400640 1 -- 172.21.15.72:0/4289302975 <== mon.0 v2:172.21.15.72:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7f6938059b10 con 0x558ad6c39ea0 2024-04-26T21:57:10.164 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.161+0000 7f6957217740 1 -- 172.21.15.72:0/4289302975 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f693804a070 con 0x558ad6c39ea0 2024-04-26T21:57:10.164 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.161+0000 7f6933400640 1 -- 172.21.15.72:0/4289302975 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6938041040 con 0x558ad6c39ea0 2024-04-26T21:57:10.164 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.161+0000 7f6957217740 10 client.15882 fetch_fsmap learned FSMap version 27 2024-04-26T21:57:10.164 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.161+0000 7f6957217740 1 -- 172.21.15.72:0/4289302975 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x558ad6cb48c0 con 0x558ad6c39ea0 2024-04-26T21:57:10.164 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.162+0000 7f6933400640 1 -- 172.21.15.72:0/4289302975 <== mon.0 v2:172.21.15.72:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f693804a070 con 0x558ad6c39ea0 2024-04-26T21:57:10.164 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.162+0000 7f6933400640 1 -- 172.21.15.72:0/4289302975 <== mon.0 v2:172.21.15.72:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f693804f590 con 0x558ad6c39ea0 2024-04-26T21:57:10.164 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.162+0000 7f6957217740 10 client.15882 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T21:57:10.164 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.162+0000 7f6957217740 10 client.15882 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-26T21:57:10.164 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.162+0000 7f6957217740 20 client.15882 populate_metadata read hostname 'smithi072' 2024-04-26T21:57:10.165 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.162+0000 7f6957217740 1 --2- 172.21.15.72:0/4289302975 >> [v2:172.21.15.76:6832/3869483750,v1:172.21.15.76:6833/3869483750] conn(0x558ad6cb5240 0x558ad6cb7660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:10.165 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.162+0000 7f6957217740 4 client.15882 mds_command: new command op to 34406 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-26T21:57:10.165 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.162+0000 7f6957217740 1 -- 172.21.15.72:0/4289302975 --> [v2:172.21.15.76:6832/3869483750,v1:172.21.15.76:6833/3869483750] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558ad6ba56f0 con 0x558ad6cb5240 2024-04-26T21:57:10.165 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.163+0000 7f6951a00640 1 --2- 172.21.15.72:0/4289302975 >> [v2:172.21.15.76:6832/3869483750,v1:172.21.15.76:6833/3869483750] conn(0x558ad6cb5240 0x558ad6cb7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:10.165 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.164+0000 7f6951a00640 1 --2- 172.21.15.72:0/4289302975 >> [v2:172.21.15.76:6832/3869483750,v1:172.21.15.76:6833/3869483750] conn(0x558ad6cb5240 0x558ad6cb7660 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=3b36083343d1796c server_cookie=d4d8318027d7c3c in_seq=0 out_seq=0 2024-04-26T21:57:10.165 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.164+0000 7f6933400640 10 client.15882 ms_handle_connect on v2:172.21.15.76:6832/3869483750 2024-04-26T21:57:10.166 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.164+0000 7f6933400640 1 -- 172.21.15.72:0/4289302975 <== mds.3 v2:172.21.15.76:6832/3869483750 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558ad6ba56f0 con 0x558ad6cb5240 2024-04-26T21:57:10.166 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.164+0000 7f6933400640 10 client.15882 handle_command_reply: tid=0 2024-04-26T21:57:10.166 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.164+0000 7f6957217740 1 client.15882 shutdown 2024-04-26T21:57:10.166 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.165+0000 7f6957217740 1 -- 172.21.15.72:0/4289302975 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558ad6c39ea0 msgr2=0x558ad6c4b1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:10.166 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.165+0000 7f6957217740 1 --2- 172.21.15.72:0/4289302975 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558ad6c39ea0 0x558ad6c4b1d0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f6938005a80 tx=0x7f6938000bc0 comp rx=0 tx=0).stop 2024-04-26T21:57:10.166 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.165+0000 7f6957217740 1 -- 172.21.15.72:0/4289302975 shutdown_connections 2024-04-26T21:57:10.166 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.165+0000 7f6957217740 1 --2- 172.21.15.72:0/4289302975 >> [v2:172.21.15.76:6832/3869483750,v1:172.21.15.76:6833/3869483750] conn(0x558ad6cb5240 0x558ad6cb7660 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:10.166 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.165+0000 7f6957217740 1 --2- 172.21.15.72:0/4289302975 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558ad6c3fa00 0x558ad6c48220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:10.166 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.165+0000 7f6957217740 1 --2- 172.21.15.72:0/4289302975 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558ad6c3a890 0x558ad6c47ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:10.167 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.165+0000 7f6957217740 1 --2- 172.21.15.72:0/4289302975 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558ad6c39ea0 0x558ad6c4b1d0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:10.167 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.165+0000 7f6933400640 0 client.15882 ms_handle_reset on v2:172.21.15.76:6832/3869483750 2024-04-26T21:57:10.167 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.165+0000 7f6957217740 1 -- 172.21.15.72:0/4289302975 >> 172.21.15.72:0/4289302975 conn(0x558ad6c35510 msgr2=0x558ad6c3b330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:57:10.167 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.165+0000 7f6957217740 1 -- 172.21.15.72:0/4289302975 shutdown_connections 2024-04-26T21:57:10.167 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.165+0000 7f6957217740 1 -- 172.21.15.72:0/4289302975 wait complete. 2024-04-26T21:57:10.167 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.165+0000 7f6957217740 20 client.15882 trim_cache size 0 max 16384 2024-04-26T21:57:10.167 INFO:teuthology.orchestra.run.smithi072.stdout:[] 2024-04-26T21:57:10.168 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.166+0000 7f6954400640 1 -- 172.21.15.72:0/3639077620 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f69340a3cc0 msgr2=0x7f69340a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:10.168 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.166+0000 7f6954400640 1 --2- 172.21.15.72:0/3639077620 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f69340a3cc0 0x7f69340a6180 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f6944099380 tx=0x7f694400ff30 comp rx=0 tx=0).stop 2024-04-26T21:57:10.168 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.166+0000 7f6954400640 1 -- 172.21.15.72:0/3639077620 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f694c104530 msgr2=0x7f694c1a21a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:10.168 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.166+0000 7f6954400640 1 --2- 172.21.15.72:0/3639077620 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f694c104530 0x7f694c1a21a0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f694400cce0 tx=0x7f694400a620 comp rx=0 tx=0).stop 2024-04-26T21:57:10.168 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.166+0000 7f6954400640 1 -- 172.21.15.72:0/3639077620 shutdown_connections 2024-04-26T21:57:10.168 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.166+0000 7f6954400640 1 --2- 172.21.15.72:0/3639077620 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f694c104530 0x7f694c1a21a0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:10.168 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.166+0000 7f6954400640 1 --2- 172.21.15.72:0/3639077620 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f694c103b70 0x7f694c19b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:10.169 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.166+0000 7f6954400640 1 --2- 172.21.15.72:0/3639077620 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f69340a3cc0 0x7f69340a6180 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:10.169 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.166+0000 7f6954400640 1 --2- 172.21.15.72:0/3639077620 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f694c102980 0x7f694c19abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:10.169 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.166+0000 7f6954400640 1 -- 172.21.15.72:0/3639077620 >> 172.21.15.72:0/3639077620 conn(0x7f694c0fdf10 msgr2=0x7f694c074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:57:10.169 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.166+0000 7f6954400640 1 -- 172.21.15.72:0/3639077620 shutdown_connections 2024-04-26T21:57:10.169 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.166+0000 7f6954400640 1 -- 172.21.15.72:0/3639077620 wait complete. 2024-04-26T21:57:10.184 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:57:09 smithi072 ceph-mon[28138]: from='client.? 172.21.15.72:0/2675833015' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:57:10.188 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-26T21:57:10.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:57:09 smithi076 ceph-mon[31823]: from='client.? 172.21.15.72:0/2675833015' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:57:10.397 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.397+0000 7f3317800640 1 -- 172.21.15.72:0/1402606145 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3310105d30 msgr2=0x7f3310108120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:10.397 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.397+0000 7f3317800640 1 --2- 172.21.15.72:0/1402606145 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3310105d30 0x7f3310108120 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f32fc009900 tx=0x7f32fc02a400 comp rx=0 tx=0).stop 2024-04-26T21:57:10.397 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.397+0000 7f3317800640 1 -- 172.21.15.72:0/1402606145 shutdown_connections 2024-04-26T21:57:10.397 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.397+0000 7f3317800640 1 --2- 172.21.15.72:0/1402606145 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3310108660 0x7f331010aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:10.397 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.397+0000 7f3317800640 1 --2- 172.21.15.72:0/1402606145 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3310105d30 0x7f3310108120 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:10.397 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.397+0000 7f3317800640 1 --2- 172.21.15.72:0/1402606145 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3310103400 0x7f33101057f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:10.398 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.397+0000 7f3317800640 1 -- 172.21.15.72:0/1402606145 >> 172.21.15.72:0/1402606145 conn(0x7f33100fcdf0 msgr2=0x7f33100ff230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:57:10.398 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.397+0000 7f3317800640 1 -- 172.21.15.72:0/1402606145 shutdown_connections 2024-04-26T21:57:10.398 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.397+0000 7f3317800640 1 -- 172.21.15.72:0/1402606145 wait complete. 2024-04-26T21:57:10.400 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.398+0000 7f3317800640 1 Processor -- start 2024-04-26T21:57:10.400 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.398+0000 7f3317800640 1 -- start start 2024-04-26T21:57:10.400 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.398+0000 7f3317800640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3310103400 0x7f3310198ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:10.400 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.398+0000 7f3317800640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3310105d30 0x7f3310199000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:10.400 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.398+0000 7f3317800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3310108660 0x7f33101a0070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:10.400 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.398+0000 7f3317800640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f331010e350 con 0x7f3310105d30 2024-04-26T21:57:10.400 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.398+0000 7f3317800640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f331010e1d0 con 0x7f3310103400 2024-04-26T21:57:10.400 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.398+0000 7f3317800640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f331010e4d0 con 0x7f3310108660 2024-04-26T21:57:10.400 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.399+0000 7f330fe00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3310105d30 0x7f3310199000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:10.401 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.399+0000 7f330fe00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3310105d30 0x7f3310199000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:48422/0 (socket says 172.21.15.72:48422) 2024-04-26T21:57:10.401 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.399+0000 7f330fe00640 1 -- 172.21.15.72:0/3327322494 learned_addr learned my addr 172.21.15.72:0/3327322494 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:57:10.401 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.399+0000 7f3314e00640 1 --2- 172.21.15.72:0/3327322494 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3310103400 0x7f3310198ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:10.401 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.399+0000 7f3315800640 1 --2- 172.21.15.72:0/3327322494 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3310108660 0x7f33101a0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:10.401 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.399+0000 7f330fe00640 1 -- 172.21.15.72:0/3327322494 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3310108660 msgr2=0x7f33101a0070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:10.401 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.399+0000 7f330fe00640 1 --2- 172.21.15.72:0/3327322494 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3310108660 0x7f33101a0070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:10.401 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.399+0000 7f330fe00640 1 -- 172.21.15.72:0/3327322494 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3310103400 msgr2=0x7f3310198ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:10.401 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.399+0000 7f330fe00640 1 --2- 172.21.15.72:0/3327322494 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3310103400 0x7f3310198ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:10.401 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.399+0000 7f330fe00640 1 -- 172.21.15.72:0/3327322494 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33101a06e0 con 0x7f3310105d30 2024-04-26T21:57:10.401 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.400+0000 7f330fe00640 1 --2- 172.21.15.72:0/3327322494 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3310105d30 0x7f3310199000 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f32fc005ab0 tx=0x7f32fc0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:57:10.401 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.400+0000 7f330d600640 1 -- 172.21.15.72:0/3327322494 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32fc03f030 con 0x7f3310105d30 2024-04-26T21:57:10.401 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.400+0000 7f330d600640 1 -- 172.21.15.72:0/3327322494 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f32fc003c90 con 0x7f3310105d30 2024-04-26T21:57:10.401 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.400+0000 7f3317800640 1 -- 172.21.15.72:0/3327322494 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33101a0970 con 0x7f3310105d30 2024-04-26T21:57:10.401 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.400+0000 7f330d600640 1 -- 172.21.15.72:0/3327322494 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32fc035070 con 0x7f3310105d30 2024-04-26T21:57:10.402 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.400+0000 7f3317800640 1 -- 172.21.15.72:0/3327322494 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33101a0eb0 con 0x7f3310105d30 2024-04-26T21:57:10.404 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.404+0000 7f331a79d740 1 Processor -- start 2024-04-26T21:57:10.404 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.404+0000 7f330d600640 1 -- 172.21.15.72:0/3327322494 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f32fc042030 con 0x7f3310105d30 2024-04-26T21:57:10.404 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.404+0000 7f331a79d740 1 -- start start 2024-04-26T21:57:10.405 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.405+0000 7f331a79d740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b4cd9fcea0 0x55b4cd9fd2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:10.405 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.405+0000 7f331a79d740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b4cd9fd890 0x55b4cda02390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:10.405 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.405+0000 7f331a79d740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55b4cda02a00 0x55b4cda04e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:10.405 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.405+0000 7f331a79d740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55b4cd960c70 con 0x55b4cda02a00 2024-04-26T21:57:10.405 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.405+0000 7f331a79d740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55b4cd919980 con 0x55b4cd9fcea0 2024-04-26T21:57:10.405 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.405+0000 7f331a79d740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55b4cd95cc30 con 0x55b4cd9fd890 2024-04-26T21:57:10.406 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.405+0000 7f330fe00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55b4cda02a00 0x55b4cda04e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:10.406 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.405+0000 7f330fe00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55b4cda02a00 0x55b4cda04e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:48432/0 (socket says 172.21.15.72:48432) 2024-04-26T21:57:10.406 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.405+0000 7f330fe00640 1 -- 172.21.15.72:0/3918461986 learned_addr learned my addr 172.21.15.72:0/3918461986 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:57:10.406 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.405+0000 7f3315800640 1 --2- 172.21.15.72:0/3918461986 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b4cd9fcea0 0x55b4cd9fd2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:10.406 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.405+0000 7f3314e00640 1 --2- 172.21.15.72:0/3918461986 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b4cd9fd890 0x55b4cda02390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:10.407 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.406+0000 7f330fe00640 1 -- 172.21.15.72:0/3918461986 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b4cd9fd890 msgr2=0x55b4cda02390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:10.407 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.406+0000 7f330fe00640 1 --2- 172.21.15.72:0/3918461986 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b4cd9fd890 0x55b4cda02390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:10.407 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.406+0000 7f330fe00640 1 -- 172.21.15.72:0/3918461986 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b4cd9fcea0 msgr2=0x55b4cd9fd2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:10.407 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.406+0000 7f330fe00640 1 --2- 172.21.15.72:0/3918461986 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b4cd9fcea0 0x55b4cd9fd2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:10.407 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.406+0000 7f330fe00640 1 -- 172.21.15.72:0/3918461986 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b4cd8f17b0 con 0x55b4cda02a00 2024-04-26T21:57:10.407 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.406+0000 7f330fe00640 1 --2- 172.21.15.72:0/3918461986 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55b4cda02a00 0x55b4cda04e90 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f32fc0ee5a0 tx=0x7f32fc0f4f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8514b63a1176549 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:57:10.407 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.406+0000 7f330d600640 1 --2- 172.21.15.72:0/3327322494 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f32f80a3cc0 0x7f32f80a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:10.407 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.406+0000 7f330d600640 1 -- 172.21.15.72:0/3327322494 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7f32fc0f0d30 con 0x7f3310105d30 2024-04-26T21:57:10.407 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.406+0000 7f3315800640 1 --2- 172.21.15.72:0/3327322494 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f32f80a3cc0 0x7f32f80a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:10.408 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.406+0000 7f3302000640 1 -- 172.21.15.72:0/3918461986 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32fc0b90d0 con 0x55b4cda02a00 2024-04-26T21:57:10.408 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.407+0000 7f3302000640 1 -- 172.21.15.72:0/3918461986 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f32fc003e40 con 0x55b4cda02a00 2024-04-26T21:57:10.408 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.407+0000 7f3302000640 1 -- 172.21.15.72:0/3918461986 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32fc0bc790 con 0x55b4cda02a00 2024-04-26T21:57:10.410 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.407+0000 7f331a79d740 1 -- 172.21.15.72:0/3918461986 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55b4cda02a00 msgr2=0x55b4cda04e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:10.410 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.407+0000 7f331a79d740 1 --2- 172.21.15.72:0/3918461986 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55b4cda02a00 0x55b4cda04e90 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f32fc0ee5a0 tx=0x7f32fc0f4f80 comp rx=0 tx=0).stop 2024-04-26T21:57:10.410 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.407+0000 7f3315800640 1 --2- 172.21.15.72:0/3327322494 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f32f80a3cc0 0x7f32f80a6180 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x55b4cd9fe0f0 tx=0x7f3308007680 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:57:10.410 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.408+0000 7f331a79d740 1 -- 172.21.15.72:0/3918461986 shutdown_connections 2024-04-26T21:57:10.410 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.408+0000 7f331a79d740 1 --2- 172.21.15.72:0/3918461986 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55b4cda02a00 0x55b4cda04e90 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:10.411 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.408+0000 7f331a79d740 1 --2- 172.21.15.72:0/3918461986 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b4cd9fd890 0x55b4cda02390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:10.411 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.408+0000 7f331a79d740 1 --2- 172.21.15.72:0/3918461986 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b4cd9fcea0 0x55b4cd9fd2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:10.411 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.408+0000 7f331a79d740 1 -- 172.21.15.72:0/3918461986 >> 172.21.15.72:0/3918461986 conn(0x55b4cd9f8510 msgr2=0x55b4cd9f8910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:57:10.411 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.408+0000 7f331a79d740 1 -- 172.21.15.72:0/3918461986 shutdown_connections 2024-04-26T21:57:10.411 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.408+0000 7f331a79d740 1 -- 172.21.15.72:0/3918461986 wait complete. 2024-04-26T21:57:10.411 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.408+0000 7f331a79d740 1 -- start start 2024-04-26T21:57:10.411 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.408+0000 7f331a79d740 1 Processor -- start 2024-04-26T21:57:10.411 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.409+0000 7f331a79d740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55b4cd9fcea0 0x55b4cda06af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:10.411 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.409+0000 7f331a79d740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b4cd9fd890 0x55b4cda07030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:10.411 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.409+0000 7f331a79d740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b4cda02a00 0x55b4cda0e070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:10.411 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.409+0000 7f331a79d740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55b4cd95ff80 con 0x55b4cd9fcea0 2024-04-26T21:57:10.411 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.409+0000 7f331a79d740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55b4cd90d260 con 0x55b4cd9fd890 2024-04-26T21:57:10.411 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.409+0000 7f331a79d740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55b4cd9652a0 con 0x55b4cda02a00 2024-04-26T21:57:10.411 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.409+0000 7f331a79d740 10 client.0 _ll_register_callbacks cb 0x55b4cd957960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:57:10.412 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.409+0000 7f3314e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55b4cd9fcea0 0x55b4cda06af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:10.412 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.409+0000 7f3314e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55b4cd9fcea0 0x55b4cda06af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:48446/0 (socket says 172.21.15.72:48446) 2024-04-26T21:57:10.412 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.409+0000 7f3314e00640 1 -- 172.21.15.72:0/373026110 learned_addr learned my addr 172.21.15.72:0/373026110 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:57:10.412 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.410+0000 7f330fe00640 1 --2- 172.21.15.72:0/373026110 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b4cd9fd890 0x55b4cda07030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:10.412 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.410+0000 7f3314e00640 1 -- 172.21.15.72:0/373026110 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b4cda02a00 msgr2=0x55b4cda0e070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:10.412 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.410+0000 7f3314e00640 1 --2- 172.21.15.72:0/373026110 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b4cda02a00 0x55b4cda0e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:10.412 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.410+0000 7f3314e00640 1 -- 172.21.15.72:0/373026110 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b4cd9fd890 msgr2=0x55b4cda07030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:10.412 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.410+0000 7f3314e00640 1 --2- 172.21.15.72:0/373026110 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b4cd9fd890 0x55b4cda07030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:10.412 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.410+0000 7f3314e00640 1 -- 172.21.15.72:0/373026110 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b4cd8ef1c0 con 0x55b4cd9fcea0 2024-04-26T21:57:10.412 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.410+0000 7f3314e00640 1 --2- 172.21.15.72:0/373026110 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55b4cd9fcea0 0x55b4cda06af0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f33040107c0 tx=0x7f3304010c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=337903edcdfea4e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:57:10.412 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.410+0000 7f32f6a00640 10 client.0 ms_handle_connect on v2:172.21.15.72:3300/0 2024-04-26T21:57:10.412 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.410+0000 7f32f6a00640 1 -- 172.21.15.72:0/373026110 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f330400cab0 con 0x55b4cd9fcea0 2024-04-26T21:57:10.412 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.410+0000 7f32f6a00640 1 -- 172.21.15.72:0/373026110 <== mon.0 v2:172.21.15.72:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7f330400cc50 con 0x55b4cd9fcea0 2024-04-26T21:57:10.412 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.410+0000 7f331a79d740 1 -- 172.21.15.72:0/373026110 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55b4cd96e880 con 0x55b4cd9fcea0 2024-04-26T21:57:10.414 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.411+0000 7f32f6a00640 1 -- 172.21.15.72:0/373026110 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f330400ce50 con 0x55b4cd9fcea0 2024-04-26T21:57:10.414 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.411+0000 7f331a79d740 10 client.15894 fetch_fsmap learned FSMap version 27 2024-04-26T21:57:10.414 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.411+0000 7f331a79d740 1 -- 172.21.15.72:0/373026110 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b4cd6723f0 con 0x55b4cd9fcea0 2024-04-26T21:57:10.414 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.411+0000 7f32f6a00640 1 -- 172.21.15.72:0/373026110 <== mon.0 v2:172.21.15.72:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x55b4cd96e880 con 0x55b4cd9fcea0 2024-04-26T21:57:10.414 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.411+0000 7f32f6a00640 1 -- 172.21.15.72:0/373026110 <== mon.0 v2:172.21.15.72:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f33040114d0 con 0x55b4cd9fcea0 2024-04-26T21:57:10.414 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.411+0000 7f331a79d740 10 client.15894 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T21:57:10.414 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.411+0000 7f331a79d740 10 client.15894 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-26T21:57:10.414 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.411+0000 7f331a79d740 20 client.15894 populate_metadata read hostname 'smithi072' 2024-04-26T21:57:10.414 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.411+0000 7f331a79d740 1 --2- 172.21.15.72:0/373026110 >> [v2:172.21.15.76:6834/3517429435,v1:172.21.15.76:6835/3517429435] conn(0x55b4cda12650 0x55b4cda14b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:10.414 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.411+0000 7f331a79d740 4 client.15894 mds_command: new command op to 34442 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:57:10.414 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.411+0000 7f331a79d740 1 -- 172.21.15.72:0/373026110 --> [v2:172.21.15.76:6834/3517429435,v1:172.21.15.76:6835/3517429435] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b4cd9686f0 con 0x55b4cda12650 2024-04-26T21:57:10.414 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.412+0000 7f330fe00640 1 --2- 172.21.15.72:0/373026110 >> [v2:172.21.15.76:6834/3517429435,v1:172.21.15.76:6835/3517429435] conn(0x55b4cda12650 0x55b4cda14b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:10.414 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.413+0000 7f330fe00640 1 --2- 172.21.15.72:0/373026110 >> [v2:172.21.15.76:6834/3517429435,v1:172.21.15.76:6835/3517429435] conn(0x55b4cda12650 0x55b4cda14b10 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2715d8e57a4ece3b server_cookie=91c65def256ad528 in_seq=0 out_seq=0 2024-04-26T21:57:10.415 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.413+0000 7f32f6a00640 10 client.15894 ms_handle_connect on v2:172.21.15.76:6834/3517429435 2024-04-26T21:57:10.416 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.415+0000 7f32f6a00640 1 -- 172.21.15.72:0/373026110 <== mds.2 v2:172.21.15.76:6834/3517429435 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b4cd9686f0 con 0x55b4cda12650 2024-04-26T21:57:10.416 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.415+0000 7f32f6a00640 10 client.15894 handle_command_reply: tid=0 2024-04-26T21:57:10.416 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.415+0000 7f331a79d740 1 client.15894 shutdown 2024-04-26T21:57:10.416 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.415+0000 7f331a79d740 1 -- 172.21.15.72:0/373026110 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55b4cd9fcea0 msgr2=0x55b4cda06af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:10.416 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.415+0000 7f331a79d740 1 --2- 172.21.15.72:0/373026110 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55b4cd9fcea0 0x55b4cda06af0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f33040107c0 tx=0x7f3304010c80 comp rx=0 tx=0).stop 2024-04-26T21:57:10.416 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.415+0000 7f331a79d740 1 -- 172.21.15.72:0/373026110 shutdown_connections 2024-04-26T21:57:10.416 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.415+0000 7f331a79d740 1 --2- 172.21.15.72:0/373026110 >> [v2:172.21.15.76:6834/3517429435,v1:172.21.15.76:6835/3517429435] conn(0x55b4cda12650 0x55b4cda14b10 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:10.416 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.415+0000 7f331a79d740 1 --2- 172.21.15.72:0/373026110 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b4cda02a00 0x55b4cda0e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:10.416 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.415+0000 7f331a79d740 1 --2- 172.21.15.72:0/373026110 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b4cd9fd890 0x55b4cda07030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:10.416 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.415+0000 7f331a79d740 1 --2- 172.21.15.72:0/373026110 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55b4cd9fcea0 0x55b4cda06af0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:10.416 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.415+0000 7f32f6a00640 0 client.15894 ms_handle_reset on v2:172.21.15.76:6834/3517429435 2024-04-26T21:57:10.417 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.415+0000 7f331a79d740 1 -- 172.21.15.72:0/373026110 >> 172.21.15.72:0/373026110 conn(0x55b4cd9f8510 msgr2=0x55b4cd9f7690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:57:10.417 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.415+0000 7f331a79d740 1 -- 172.21.15.72:0/373026110 shutdown_connections 2024-04-26T21:57:10.417 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.415+0000 7f331a79d740 1 -- 172.21.15.72:0/373026110 wait complete. 2024-04-26T21:57:10.417 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.416+0000 7f331a79d740 20 client.15894 trim_cache size 0 max 16384 2024-04-26T21:57:10.431 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.430+0000 7f331a79d740 1 Processor -- start 2024-04-26T21:57:10.431 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.430+0000 7f331a79d740 1 -- start start 2024-04-26T21:57:10.431 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.430+0000 7f331a79d740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b4cd9fcea0 0x55b4cda1a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:10.431 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.430+0000 7f331a79d740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b4cd9fd890 0x55b4cda1a920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:10.431 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.430+0000 7f331a79d740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55b4cda02a00 0x55b4cd9fea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:10.431 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.430+0000 7f331a79d740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f33101a9ee0 con 0x55b4cda02a00 2024-04-26T21:57:10.431 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.430+0000 7f331a79d740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f33101aba20 con 0x55b4cd9fd890 2024-04-26T21:57:10.431 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.430+0000 7f331a79d740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f33101aa5b0 con 0x55b4cd9fcea0 2024-04-26T21:57:10.431 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.431+0000 7f3315800640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55b4cda02a00 0x55b4cd9fea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:10.432 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.431+0000 7f3315800640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55b4cda02a00 0x55b4cd9fea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:48452/0 (socket says 172.21.15.72:48452) 2024-04-26T21:57:10.432 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.431+0000 7f3315800640 1 -- 172.21.15.72:0/3125094230 learned_addr learned my addr 172.21.15.72:0/3125094230 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:57:10.432 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.431+0000 7f330fe00640 1 --2- 172.21.15.72:0/3125094230 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b4cd9fd890 0x55b4cda1a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:10.432 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.431+0000 7f3314e00640 1 --2- 172.21.15.72:0/3125094230 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b4cd9fcea0 0x55b4cda1a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:10.432 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.431+0000 7f3315800640 1 -- 172.21.15.72:0/3125094230 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b4cd9fcea0 msgr2=0x55b4cda1a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:10.432 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.431+0000 7f3315800640 1 --2- 172.21.15.72:0/3125094230 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b4cd9fcea0 0x55b4cda1a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:10.432 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.431+0000 7f3315800640 1 -- 172.21.15.72:0/3125094230 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b4cd9fd890 msgr2=0x55b4cda1a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:10.432 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.431+0000 7f3315800640 1 --2- 172.21.15.72:0/3125094230 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b4cd9fd890 0x55b4cda1a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:10.432 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.431+0000 7f3315800640 1 -- 172.21.15.72:0/3125094230 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b4cd73c7d0 con 0x55b4cda02a00 2024-04-26T21:57:10.432 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.431+0000 7f330fe00640 1 --2- 172.21.15.72:0/3125094230 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b4cd9fd890 0x55b4cda1a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:57:10.435 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.431+0000 7f3315800640 1 --2- 172.21.15.72:0/3125094230 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55b4cda02a00 0x55b4cd9fea10 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f3308007fd0 tx=0x7f3308013040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8b5be21a4d7ca9c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:57:10.435 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.432+0000 7f3301600640 1 -- 172.21.15.72:0/3125094230 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f330800edc0 con 0x55b4cda02a00 2024-04-26T21:57:10.435 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.432+0000 7f3301600640 1 -- 172.21.15.72:0/3125094230 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f330801c8a0 con 0x55b4cda02a00 2024-04-26T21:57:10.435 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.432+0000 7f3301600640 1 -- 172.21.15.72:0/3125094230 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f330801d920 con 0x55b4cda02a00 2024-04-26T21:57:10.435 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.432+0000 7f331a79d740 1 -- 172.21.15.72:0/3125094230 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55b4cda02a00 msgr2=0x55b4cd9fea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:10.435 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.432+0000 7f331a79d740 1 --2- 172.21.15.72:0/3125094230 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55b4cda02a00 0x55b4cd9fea10 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f3308007fd0 tx=0x7f3308013040 comp rx=0 tx=0).stop 2024-04-26T21:57:10.435 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.433+0000 7f331a79d740 1 -- 172.21.15.72:0/3125094230 shutdown_connections 2024-04-26T21:57:10.435 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.433+0000 7f331a79d740 1 --2- 172.21.15.72:0/3125094230 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55b4cda02a00 0x55b4cd9fea10 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:10.435 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.433+0000 7f331a79d740 1 --2- 172.21.15.72:0/3125094230 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b4cd9fd890 0x55b4cda1a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:10.436 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.433+0000 7f331a79d740 1 --2- 172.21.15.72:0/3125094230 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b4cd9fcea0 0x55b4cda1a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:10.436 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.433+0000 7f331a79d740 1 -- 172.21.15.72:0/3125094230 >> 172.21.15.72:0/3125094230 conn(0x55b4cd9f8510 msgr2=0x55b4cd9fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:57:10.436 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.433+0000 7f331a79d740 1 -- 172.21.15.72:0/3125094230 shutdown_connections 2024-04-26T21:57:10.436 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.433+0000 7f331a79d740 1 -- 172.21.15.72:0/3125094230 wait complete. 2024-04-26T21:57:10.436 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.433+0000 7f331a79d740 1 -- start start 2024-04-26T21:57:10.436 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.433+0000 7f331a79d740 1 Processor -- start 2024-04-26T21:57:10.436 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.434+0000 7f331a79d740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55b4cd9fcea0 0x55b4cda73af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:10.436 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.434+0000 7f331a79d740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b4cd9fd890 0x55b4cda74030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:10.436 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.434+0000 7f331a79d740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b4cda02a00 0x55b4cda7b000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:10.436 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.434+0000 7f331a79d740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f33101ab350 con 0x55b4cd9fcea0 2024-04-26T21:57:10.436 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.434+0000 7f331a79d740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f33101aac80 con 0x55b4cda02a00 2024-04-26T21:57:10.436 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.434+0000 7f331a79d740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f33101a7ec0 con 0x55b4cd9fd890 2024-04-26T21:57:10.436 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.434+0000 7f331a79d740 10 client.0 _ll_register_callbacks cb 0x7f330400e3f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:57:10.436 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.434+0000 7f3314e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55b4cd9fcea0 0x55b4cda73af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:10.437 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.434+0000 7f3314e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55b4cd9fcea0 0x55b4cda73af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:48456/0 (socket says 172.21.15.72:48456) 2024-04-26T21:57:10.437 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.434+0000 7f3314e00640 1 -- 172.21.15.72:0/2245267953 learned_addr learned my addr 172.21.15.72:0/2245267953 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:57:10.437 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.434+0000 7f330fe00640 1 --2- 172.21.15.72:0/2245267953 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b4cd9fd890 0x55b4cda74030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:10.437 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.434+0000 7f3314e00640 1 -- 172.21.15.72:0/2245267953 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b4cd9fd890 msgr2=0x55b4cda74030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:10.437 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.434+0000 7f3314e00640 1 --2- 172.21.15.72:0/2245267953 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b4cd9fd890 0x55b4cda74030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:10.437 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.434+0000 7f3314e00640 1 -- 172.21.15.72:0/2245267953 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b4cda02a00 msgr2=0x55b4cda7b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:10.437 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.434+0000 7f3314e00640 1 --2- 172.21.15.72:0/2245267953 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b4cda02a00 0x55b4cda7b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:10.437 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.434+0000 7f3314e00640 1 -- 172.21.15.72:0/2245267953 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b4cd9072a0 con 0x55b4cd9fcea0 2024-04-26T21:57:10.437 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.435+0000 7f3314e00640 1 --2- 172.21.15.72:0/2245267953 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55b4cd9fcea0 0x55b4cda73af0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f330400e3c0 tx=0x7f3304011ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dae614ff035c9e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:57:10.437 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.435+0000 7f32f6a00640 10 client.0 ms_handle_connect on v2:172.21.15.72:3300/0 2024-04-26T21:57:10.437 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.435+0000 7f32f6a00640 1 -- 172.21.15.72:0/2245267953 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f330402a070 con 0x55b4cd9fcea0 2024-04-26T21:57:10.437 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.435+0000 7f32f6a00640 1 -- 172.21.15.72:0/2245267953 <== mon.0 v2:172.21.15.72:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7f330400c900 con 0x55b4cd9fcea0 2024-04-26T21:57:10.437 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.435+0000 7f331a79d740 1 -- 172.21.15.72:0/2245267953 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f330801c8a0 con 0x55b4cd9fcea0 2024-04-26T21:57:10.440 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.435+0000 7f32f6a00640 1 -- 172.21.15.72:0/2245267953 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3304035020 con 0x55b4cd9fcea0 2024-04-26T21:57:10.440 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.436+0000 7f331a79d740 10 client.15900 fetch_fsmap learned FSMap version 27 2024-04-26T21:57:10.440 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.436+0000 7f331a79d740 1 -- 172.21.15.72:0/2245267953 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b4cda7c700 con 0x55b4cd9fcea0 2024-04-26T21:57:10.440 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.436+0000 7f32f6a00640 1 -- 172.21.15.72:0/2245267953 <== mon.0 v2:172.21.15.72:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f330801c8a0 con 0x55b4cd9fcea0 2024-04-26T21:57:10.440 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.436+0000 7f32f6a00640 1 -- 172.21.15.72:0/2245267953 <== mon.0 v2:172.21.15.72:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f33040036f0 con 0x55b4cd9fcea0 2024-04-26T21:57:10.440 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.436+0000 7f331a79d740 10 client.15900 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T21:57:10.440 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.436+0000 7f331a79d740 10 client.15900 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-26T21:57:10.440 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.436+0000 7f331a79d740 20 client.15900 populate_metadata read hostname 'smithi072' 2024-04-26T21:57:10.440 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.436+0000 7f331a79d740 1 --2- 172.21.15.72:0/2245267953 >> [v2:172.21.15.76:6834/3517429435,v1:172.21.15.76:6835/3517429435] conn(0x55b4cda7cfc0 0x55b4cda7f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:10.440 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.436+0000 7f331a79d740 4 client.15900 mds_command: new command op to 34442 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-26T21:57:10.440 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.436+0000 7f331a79d740 1 -- 172.21.15.72:0/2245267953 --> [v2:172.21.15.76:6834/3517429435,v1:172.21.15.76:6835/3517429435] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b4cd8e34d0 con 0x55b4cda7cfc0 2024-04-26T21:57:10.440 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.437+0000 7f330fe00640 1 --2- 172.21.15.72:0/2245267953 >> [v2:172.21.15.76:6834/3517429435,v1:172.21.15.76:6835/3517429435] conn(0x55b4cda7cfc0 0x55b4cda7f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:10.441 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.438+0000 7f330fe00640 1 --2- 172.21.15.72:0/2245267953 >> [v2:172.21.15.76:6834/3517429435,v1:172.21.15.76:6835/3517429435] conn(0x55b4cda7cfc0 0x55b4cda7f3e0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4a16342b471e6805 server_cookie=dec4057233769f97 in_seq=0 out_seq=0 2024-04-26T21:57:10.441 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.438+0000 7f32f6a00640 10 client.15900 ms_handle_connect on v2:172.21.15.76:6834/3517429435 2024-04-26T21:57:10.441 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.438+0000 7f32f6a00640 1 -- 172.21.15.72:0/2245267953 <== mds.2 v2:172.21.15.76:6834/3517429435 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b4cd8e34d0 con 0x55b4cda7cfc0 2024-04-26T21:57:10.441 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.438+0000 7f32f6a00640 10 client.15900 handle_command_reply: tid=0 2024-04-26T21:57:10.441 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.438+0000 7f331a79d740 1 client.15900 shutdown 2024-04-26T21:57:10.441 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.438+0000 7f331a79d740 1 -- 172.21.15.72:0/2245267953 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55b4cd9fcea0 msgr2=0x55b4cda73af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:10.441 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.438+0000 7f331a79d740 1 --2- 172.21.15.72:0/2245267953 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55b4cd9fcea0 0x55b4cda73af0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f330400e3c0 tx=0x7f3304011ea0 comp rx=0 tx=0).stop 2024-04-26T21:57:10.441 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.438+0000 7f331a79d740 1 -- 172.21.15.72:0/2245267953 shutdown_connections 2024-04-26T21:57:10.441 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.438+0000 7f331a79d740 1 --2- 172.21.15.72:0/2245267953 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b4cda02a00 0x55b4cda7b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:10.441 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.438+0000 7f331a79d740 1 --2- 172.21.15.72:0/2245267953 >> [v2:172.21.15.76:6834/3517429435,v1:172.21.15.76:6835/3517429435] conn(0x55b4cda7cfc0 0x55b4cda7f3e0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:10.441 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.438+0000 7f331a79d740 1 --2- 172.21.15.72:0/2245267953 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b4cd9fd890 0x55b4cda74030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:10.441 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.438+0000 7f331a79d740 1 --2- 172.21.15.72:0/2245267953 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55b4cd9fcea0 0x55b4cda73af0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:10.441 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.438+0000 7f331a79d740 1 -- 172.21.15.72:0/2245267953 >> 172.21.15.72:0/2245267953 conn(0x55b4cd9f8510 msgr2=0x55b4cd9feb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:57:10.441 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.438+0000 7f32f6a00640 0 client.15900 ms_handle_reset on v2:172.21.15.76:6834/3517429435 2024-04-26T21:57:10.442 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.439+0000 7f331a79d740 1 -- 172.21.15.72:0/2245267953 shutdown_connections 2024-04-26T21:57:10.442 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.439+0000 7f331a79d740 1 -- 172.21.15.72:0/2245267953 wait complete. 2024-04-26T21:57:10.442 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.439+0000 7f331a79d740 20 client.15900 trim_cache size 0 max 16384 2024-04-26T21:57:10.442 INFO:teuthology.orchestra.run.smithi072.stdout:[] 2024-04-26T21:57:10.443 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.439+0000 7f3317800640 1 -- 172.21.15.72:0/3327322494 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f32f80a3cc0 msgr2=0x7f32f80a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:10.443 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.439+0000 7f3317800640 1 --2- 172.21.15.72:0/3327322494 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f32f80a3cc0 0x7f32f80a6180 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x55b4cd9fe0f0 tx=0x7f3308007680 comp rx=0 tx=0).stop 2024-04-26T21:57:10.443 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.439+0000 7f3317800640 1 -- 172.21.15.72:0/3327322494 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3310105d30 msgr2=0x7f3310199000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:10.443 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.439+0000 7f3317800640 1 --2- 172.21.15.72:0/3327322494 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3310105d30 0x7f3310199000 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f32fc005ab0 tx=0x7f32fc0042f0 comp rx=0 tx=0).stop 2024-04-26T21:57:10.443 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.440+0000 7f3317800640 1 -- 172.21.15.72:0/3327322494 shutdown_connections 2024-04-26T21:57:10.443 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.440+0000 7f3317800640 1 --2- 172.21.15.72:0/3327322494 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f32f80a3cc0 0x7f32f80a6180 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:10.443 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.440+0000 7f3317800640 1 --2- 172.21.15.72:0/3327322494 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3310108660 0x7f33101a0070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:10.443 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.440+0000 7f3317800640 1 --2- 172.21.15.72:0/3327322494 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3310105d30 0x7f3310199000 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:10.443 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.440+0000 7f3317800640 1 --2- 172.21.15.72:0/3327322494 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3310103400 0x7f3310198ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:10.443 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.440+0000 7f3317800640 1 -- 172.21.15.72:0/3327322494 >> 172.21.15.72:0/3327322494 conn(0x7f33100fcdf0 msgr2=0x7f3310103d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:57:10.444 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.440+0000 7f3317800640 1 -- 172.21.15.72:0/3327322494 shutdown_connections 2024-04-26T21:57:10.444 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.440+0000 7f3317800640 1 -- 172.21.15.72:0/3327322494 wait complete. 2024-04-26T21:57:10.444 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:57:09 smithi164 ceph-mon[31916]: from='client.? 172.21.15.72:0/2675833015' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:57:10.457 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-26T21:57:10.652 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.651+0000 7fcc45e00640 1 -- 172.21.15.72:0/3945255689 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcc401028c0 msgr2=0x7fcc40102ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:10.652 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.651+0000 7fcc45e00640 1 --2- 172.21.15.72:0/3945255689 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcc401028c0 0x7fcc40102ce0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fcc34009900 tx=0x7fcc3402a410 comp rx=0 tx=0).stop 2024-04-26T21:57:10.652 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.651+0000 7fcc45e00640 1 -- 172.21.15.72:0/3945255689 shutdown_connections 2024-04-26T21:57:10.652 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.651+0000 7fcc45e00640 1 --2- 172.21.15.72:0/3945255689 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcc40104470 0x7fcc4010ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:10.652 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.651+0000 7fcc45e00640 1 --2- 172.21.15.72:0/3945255689 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcc40103ab0 0x7fcc40103f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:10.652 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.651+0000 7fcc45e00640 1 --2- 172.21.15.72:0/3945255689 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcc401028c0 0x7fcc40102ce0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:10.652 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.651+0000 7fcc45e00640 1 -- 172.21.15.72:0/3945255689 >> 172.21.15.72:0/3945255689 conn(0x7fcc400fde30 msgr2=0x7fcc40100270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:57:10.652 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.652+0000 7fcc45e00640 1 -- 172.21.15.72:0/3945255689 shutdown_connections 2024-04-26T21:57:10.652 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.652+0000 7fcc45e00640 1 -- 172.21.15.72:0/3945255689 wait complete. 2024-04-26T21:57:10.653 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.653+0000 7fcc45e00640 1 Processor -- start 2024-04-26T21:57:10.653 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.653+0000 7fcc45e00640 1 -- start start 2024-04-26T21:57:10.653 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.653+0000 7fcc45e00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcc401028c0 0x7fcc4019abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:10.653 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.653+0000 7fcc45e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcc40103ab0 0x7fcc4019b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:10.654 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.653+0000 7fcc45e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcc40104470 0x7fcc401a21a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:10.654 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.653+0000 7fcc45e00640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fcc4010e250 con 0x7fcc40103ab0 2024-04-26T21:57:10.654 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.653+0000 7fcc45e00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fcc4010e0d0 con 0x7fcc401028c0 2024-04-26T21:57:10.654 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.653+0000 7fcc45e00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fcc4010e3d0 con 0x7fcc40104470 2024-04-26T21:57:10.655 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.654+0000 7fcc3fe00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcc40103ab0 0x7fcc4019b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:10.655 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.654+0000 7fcc3fe00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcc40103ab0 0x7fcc4019b130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:48482/0 (socket says 172.21.15.72:48482) 2024-04-26T21:57:10.655 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.654+0000 7fcc3fe00640 1 -- 172.21.15.72:0/1539690515 learned_addr learned my addr 172.21.15.72:0/1539690515 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:57:10.655 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.654+0000 7fcc44a00640 1 --2- 172.21.15.72:0/1539690515 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcc401028c0 0x7fcc4019abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:10.656 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.654+0000 7fcc45400640 1 --2- 172.21.15.72:0/1539690515 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcc40104470 0x7fcc401a21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:10.656 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.654+0000 7fcc3fe00640 1 -- 172.21.15.72:0/1539690515 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcc40104470 msgr2=0x7fcc401a21a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:10.656 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.654+0000 7fcc3fe00640 1 --2- 172.21.15.72:0/1539690515 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcc40104470 0x7fcc401a21a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:10.656 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.654+0000 7fcc3fe00640 1 -- 172.21.15.72:0/1539690515 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcc401028c0 msgr2=0x7fcc4019abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:10.656 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.654+0000 7fcc3fe00640 1 --2- 172.21.15.72:0/1539690515 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcc401028c0 0x7fcc4019abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:10.656 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.654+0000 7fcc3fe00640 1 -- 172.21.15.72:0/1539690515 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc401a2810 con 0x7fcc40103ab0 2024-04-26T21:57:10.656 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.654+0000 7fcc45400640 1 --2- 172.21.15.72:0/1539690515 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcc40104470 0x7fcc401a21a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:57:10.656 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.655+0000 7fcc3fe00640 1 --2- 172.21.15.72:0/1539690515 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcc40103ab0 0x7fcc4019b130 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fcc2c00d950 tx=0x7fcc2c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:57:10.656 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.655+0000 7fcc3d600640 1 -- 172.21.15.72:0/1539690515 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc2c017070 con 0x7fcc40103ab0 2024-04-26T21:57:10.656 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.655+0000 7fcc3d600640 1 -- 172.21.15.72:0/1539690515 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcc2c009c30 con 0x7fcc40103ab0 2024-04-26T21:57:10.656 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.655+0000 7fcc45e00640 1 -- 172.21.15.72:0/1539690515 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc401a2b00 con 0x7fcc40103ab0 2024-04-26T21:57:10.656 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.655+0000 7fcc3d600640 1 -- 172.21.15.72:0/1539690515 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc2c00f620 con 0x7fcc40103ab0 2024-04-26T21:57:10.656 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.655+0000 7fcc45e00640 1 -- 172.21.15.72:0/1539690515 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc401a3050 con 0x7fcc40103ab0 2024-04-26T21:57:10.659 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.659+0000 7fcc49f22740 1 Processor -- start 2024-04-26T21:57:10.660 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.659+0000 7fcc49f22740 1 -- start start 2024-04-26T21:57:10.660 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.659+0000 7fcc3d600640 1 -- 172.21.15.72:0/1539690515 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fcc2c010460 con 0x7fcc40103ab0 2024-04-26T21:57:10.660 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.659+0000 7fcc49f22740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5603b084d050 0x5603b084d470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:10.660 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.659+0000 7fcc49f22740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5603b084da40 0x5603b0852540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:10.660 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.659+0000 7fcc49f22740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5603b0852bb0 0x5603b0855040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:10.660 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.659+0000 7fcc49f22740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x5603b07b0c70 con 0x5603b0852bb0 2024-04-26T21:57:10.660 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.659+0000 7fcc49f22740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5603b0769980 con 0x5603b084da40 2024-04-26T21:57:10.660 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.659+0000 7fcc49f22740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5603b07acc30 con 0x5603b084d050 2024-04-26T21:57:10.660 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.660+0000 7fcc3fe00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5603b0852bb0 0x5603b0855040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:10.660 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.660+0000 7fcc3fe00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5603b0852bb0 0x5603b0855040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:48496/0 (socket says 172.21.15.72:48496) 2024-04-26T21:57:10.661 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.660+0000 7fcc3fe00640 1 -- 172.21.15.72:0/2678400634 learned_addr learned my addr 172.21.15.72:0/2678400634 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:57:10.661 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.660+0000 7fcc44a00640 1 --2- 172.21.15.72:0/2678400634 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5603b084da40 0x5603b0852540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:10.661 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.660+0000 7fcc45400640 1 --2- 172.21.15.72:0/2678400634 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5603b084d050 0x5603b084d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:10.662 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.660+0000 7fcc3fe00640 1 -- 172.21.15.72:0/2678400634 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5603b084d050 msgr2=0x5603b084d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:10.662 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.660+0000 7fcc3fe00640 1 --2- 172.21.15.72:0/2678400634 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5603b084d050 0x5603b084d470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:10.663 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.660+0000 7fcc3fe00640 1 -- 172.21.15.72:0/2678400634 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5603b084da40 msgr2=0x5603b0852540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:10.663 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.660+0000 7fcc3fe00640 1 --2- 172.21.15.72:0/2678400634 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5603b084da40 0x5603b0852540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:10.663 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.660+0000 7fcc3fe00640 1 -- 172.21.15.72:0/2678400634 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5603b07417b0 con 0x5603b0852bb0 2024-04-26T21:57:10.663 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.660+0000 7fcc44a00640 1 --2- 172.21.15.72:0/2678400634 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5603b084da40 0x5603b0852540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:57:10.663 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.661+0000 7fcc3fe00640 1 --2- 172.21.15.72:0/2678400634 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5603b0852bb0 0x5603b0855040 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fcc2c0cdb40 tx=0x7fcc2c0d89d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb061a7bcf543af0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:57:10.663 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.661+0000 7fcc3d600640 1 --2- 172.21.15.72:0/1539690515 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fcc280a3c40 0x7fcc280a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:10.663 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.661+0000 7fcc31c00640 1 -- 172.21.15.72:0/2678400634 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc2c0e4070 con 0x5603b0852bb0 2024-04-26T21:57:10.663 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.661+0000 7fcc31c00640 1 -- 172.21.15.72:0/2678400634 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcc2c0ab210 con 0x5603b0852bb0 2024-04-26T21:57:10.663 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.661+0000 7fcc31c00640 1 -- 172.21.15.72:0/2678400634 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc2c0b41c0 con 0x5603b0852bb0 2024-04-26T21:57:10.663 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.661+0000 7fcc3d600640 1 -- 172.21.15.72:0/1539690515 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7fcc2c0d02d0 con 0x7fcc40103ab0 2024-04-26T21:57:10.663 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.661+0000 7fcc45400640 1 --2- 172.21.15.72:0/1539690515 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fcc280a3c40 0x7fcc280a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:10.663 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.662+0000 7fcc49f22740 1 -- 172.21.15.72:0/2678400634 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5603b0852bb0 msgr2=0x5603b0855040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:10.663 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.662+0000 7fcc49f22740 1 --2- 172.21.15.72:0/2678400634 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5603b0852bb0 0x5603b0855040 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fcc2c0cdb40 tx=0x7fcc2c0d89d0 comp rx=0 tx=0).stop 2024-04-26T21:57:10.664 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.662+0000 7fcc45400640 1 --2- 172.21.15.72:0/1539690515 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fcc280a3c40 0x7fcc280a6100 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x5603b084e2a0 tx=0x7fcc38009590 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:57:10.664 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.662+0000 7fcc49f22740 1 -- 172.21.15.72:0/2678400634 shutdown_connections 2024-04-26T21:57:10.664 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.662+0000 7fcc49f22740 1 --2- 172.21.15.72:0/2678400634 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5603b0852bb0 0x5603b0855040 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:10.664 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.662+0000 7fcc49f22740 1 --2- 172.21.15.72:0/2678400634 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5603b084da40 0x5603b0852540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:10.664 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.662+0000 7fcc49f22740 1 --2- 172.21.15.72:0/2678400634 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5603b084d050 0x5603b084d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:10.664 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.662+0000 7fcc49f22740 1 -- 172.21.15.72:0/2678400634 >> 172.21.15.72:0/2678400634 conn(0x5603b08486c0 msgr2=0x5603b0848ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:57:10.664 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.662+0000 7fcc49f22740 1 -- 172.21.15.72:0/2678400634 shutdown_connections 2024-04-26T21:57:10.664 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.662+0000 7fcc49f22740 1 -- 172.21.15.72:0/2678400634 wait complete. 2024-04-26T21:57:10.664 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.663+0000 7fcc49f22740 1 -- start start 2024-04-26T21:57:10.667 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.663+0000 7fcc49f22740 1 Processor -- start 2024-04-26T21:57:10.667 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.663+0000 7fcc49f22740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5603b084d050 0x5603b0856eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:10.667 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.663+0000 7fcc49f22740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5603b084da40 0x5603b08573f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:10.667 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.663+0000 7fcc49f22740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5603b0852bb0 0x5603b085e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:10.667 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.663+0000 7fcc49f22740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x5603b07aff80 con 0x5603b0852bb0 2024-04-26T21:57:10.667 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.664+0000 7fcc49f22740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5603b075d260 con 0x5603b084da40 2024-04-26T21:57:10.667 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.664+0000 7fcc49f22740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5603b07b52a0 con 0x5603b084d050 2024-04-26T21:57:10.667 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.664+0000 7fcc49f22740 10 client.0 _ll_register_callbacks cb 0x5603b07a7960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:57:10.668 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.664+0000 7fcc45400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5603b0852bb0 0x5603b085e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:10.668 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.664+0000 7fcc45400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5603b0852bb0 0x5603b085e430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:48506/0 (socket says 172.21.15.72:48506) 2024-04-26T21:57:10.668 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.664+0000 7fcc45400640 1 -- 172.21.15.72:0/3622528828 learned_addr learned my addr 172.21.15.72:0/3622528828 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:57:10.668 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.664+0000 7fcc3fe00640 1 --2- 172.21.15.72:0/3622528828 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5603b084da40 0x5603b08573f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:10.668 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.664+0000 7fcc44a00640 1 --2- 172.21.15.72:0/3622528828 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5603b084d050 0x5603b0856eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:10.668 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.664+0000 7fcc45400640 1 -- 172.21.15.72:0/3622528828 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5603b084d050 msgr2=0x5603b0856eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:10.668 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.664+0000 7fcc45400640 1 --2- 172.21.15.72:0/3622528828 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5603b084d050 0x5603b0856eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:10.668 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.664+0000 7fcc45400640 1 -- 172.21.15.72:0/3622528828 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5603b084da40 msgr2=0x5603b08573f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:10.668 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.664+0000 7fcc45400640 1 --2- 172.21.15.72:0/3622528828 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5603b084da40 0x5603b08573f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:10.668 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.664+0000 7fcc45400640 1 -- 172.21.15.72:0/3622528828 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5603b073f1c0 con 0x5603b0852bb0 2024-04-26T21:57:10.668 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.664+0000 7fcc3fe00640 1 --2- 172.21.15.72:0/3622528828 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5603b084da40 0x5603b08573f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:57:10.668 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.665+0000 7fcc45400640 1 --2- 172.21.15.72:0/3622528828 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5603b0852bb0 0x5603b085e430 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fcc38005c00 tx=0x7fcc38004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcd26013bbfdd740 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:57:10.668 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.665+0000 7fcc26000640 10 client.0 ms_handle_connect on v2:172.21.15.72:3300/0 2024-04-26T21:57:10.668 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.665+0000 7fcc26000640 1 -- 172.21.15.72:0/3622528828 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc3800fd60 con 0x5603b0852bb0 2024-04-26T21:57:10.669 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.665+0000 7fcc49f22740 1 -- 172.21.15.72:0/3622528828 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcc2c0ab210 con 0x5603b0852bb0 2024-04-26T21:57:10.669 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.665+0000 7fcc26000640 1 -- 172.21.15.72:0/3622528828 <== mon.0 v2:172.21.15.72:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7fcc3801f420 con 0x5603b0852bb0 2024-04-26T21:57:10.669 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.665+0000 7fcc26000640 1 -- 172.21.15.72:0/3622528828 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fcc3802c030 con 0x5603b0852bb0 2024-04-26T21:57:10.669 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.666+0000 7fcc49f22740 10 client.15912 fetch_fsmap learned FSMap version 27 2024-04-26T21:57:10.669 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.666+0000 7fcc49f22740 1 -- 172.21.15.72:0/3622528828 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5603b04c23f0 con 0x5603b0852bb0 2024-04-26T21:57:10.669 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.666+0000 7fcc26000640 1 -- 172.21.15.72:0/3622528828 <== mon.0 v2:172.21.15.72:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcc2c0ab210 con 0x5603b0852bb0 2024-04-26T21:57:10.669 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.666+0000 7fcc26000640 1 -- 172.21.15.72:0/3622528828 <== mon.0 v2:172.21.15.72:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fcc38031450 con 0x5603b0852bb0 2024-04-26T21:57:10.669 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.666+0000 7fcc49f22740 10 client.15912 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T21:57:10.669 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.666+0000 7fcc49f22740 10 client.15912 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-26T21:57:10.669 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.666+0000 7fcc49f22740 20 client.15912 populate_metadata read hostname 'smithi072' 2024-04-26T21:57:10.669 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.666+0000 7fcc49f22740 1 --2- 172.21.15.72:0/3622528828 >> [v2:172.21.15.76:6836/1150678991,v1:172.21.15.76:6837/1150678991] conn(0x5603b0862a40 0x5603b0864ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:10.669 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.666+0000 7fcc49f22740 4 client.15912 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:57:10.669 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.666+0000 7fcc49f22740 1 -- 172.21.15.72:0/3622528828 --> [v2:172.21.15.76:6836/1150678991,v1:172.21.15.76:6837/1150678991] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5603b07be880 con 0x5603b0862a40 2024-04-26T21:57:10.669 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.667+0000 7fcc44a00640 1 --2- 172.21.15.72:0/3622528828 >> [v2:172.21.15.76:6836/1150678991,v1:172.21.15.76:6837/1150678991] conn(0x5603b0862a40 0x5603b0864ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:10.671 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.668+0000 7fcc44a00640 1 --2- 172.21.15.72:0/3622528828 >> [v2:172.21.15.76:6836/1150678991,v1:172.21.15.76:6837/1150678991] conn(0x5603b0862a40 0x5603b0864ec0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8a99da10cb11d3fb server_cookie=bf034021e0c4c0f3 in_seq=0 out_seq=0 2024-04-26T21:57:10.671 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.668+0000 7fcc26000640 10 client.15912 ms_handle_connect on v2:172.21.15.76:6836/1150678991 2024-04-26T21:57:10.671 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.670+0000 7fcc26000640 1 -- 172.21.15.72:0/3622528828 <== mds.1 v2:172.21.15.76:6836/1150678991 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5603b07be880 con 0x5603b0862a40 2024-04-26T21:57:10.671 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.670+0000 7fcc26000640 10 client.15912 handle_command_reply: tid=0 2024-04-26T21:57:10.671 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.670+0000 7fcc49f22740 1 client.15912 shutdown 2024-04-26T21:57:10.671 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.670+0000 7fcc49f22740 1 -- 172.21.15.72:0/3622528828 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5603b0852bb0 msgr2=0x5603b085e430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:10.671 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.670+0000 7fcc49f22740 1 --2- 172.21.15.72:0/3622528828 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5603b0852bb0 0x5603b085e430 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fcc38005c00 tx=0x7fcc38004a40 comp rx=0 tx=0).stop 2024-04-26T21:57:10.671 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.670+0000 7fcc49f22740 1 -- 172.21.15.72:0/3622528828 shutdown_connections 2024-04-26T21:57:10.671 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.670+0000 7fcc49f22740 1 --2- 172.21.15.72:0/3622528828 >> [v2:172.21.15.76:6836/1150678991,v1:172.21.15.76:6837/1150678991] conn(0x5603b0862a40 0x5603b0864ec0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:10.671 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.670+0000 7fcc49f22740 1 --2- 172.21.15.72:0/3622528828 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5603b0852bb0 0x5603b085e430 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:10.671 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.670+0000 7fcc49f22740 1 --2- 172.21.15.72:0/3622528828 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5603b084da40 0x5603b08573f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:10.672 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.670+0000 7fcc26000640 0 client.15912 ms_handle_reset on v2:172.21.15.76:6836/1150678991 2024-04-26T21:57:10.672 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.670+0000 7fcc49f22740 1 --2- 172.21.15.72:0/3622528828 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5603b084d050 0x5603b0856eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:10.672 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.670+0000 7fcc49f22740 1 -- 172.21.15.72:0/3622528828 >> 172.21.15.72:0/3622528828 conn(0x5603b08486c0 msgr2=0x5603b08477b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:57:10.672 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.670+0000 7fcc49f22740 1 -- 172.21.15.72:0/3622528828 shutdown_connections 2024-04-26T21:57:10.672 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.670+0000 7fcc49f22740 1 -- 172.21.15.72:0/3622528828 wait complete. 2024-04-26T21:57:10.672 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.671+0000 7fcc49f22740 20 client.15912 trim_cache size 0 max 16384 2024-04-26T21:57:10.685 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.685+0000 7fcc49f22740 1 Processor -- start 2024-04-26T21:57:10.686 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.685+0000 7fcc49f22740 1 -- start start 2024-04-26T21:57:10.686 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.686+0000 7fcc49f22740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5603b084d050 0x5603b084b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:10.686 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.686+0000 7fcc49f22740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5603b084da40 0x5603b085e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:10.686 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.686+0000 7fcc49f22740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5603b0852bb0 0x5603b084b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:10.686 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.686+0000 7fcc49f22740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fcc401ac010 con 0x5603b084d050 2024-04-26T21:57:10.686 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.686+0000 7fcc49f22740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fcc401adb50 con 0x5603b0852bb0 2024-04-26T21:57:10.686 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.686+0000 7fcc49f22740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fcc401ac6e0 con 0x5603b084da40 2024-04-26T21:57:10.687 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.686+0000 7fcc44a00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5603b084d050 0x5603b084b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:10.687 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.686+0000 7fcc44a00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5603b084d050 0x5603b084b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:48510/0 (socket says 172.21.15.72:48510) 2024-04-26T21:57:10.687 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.686+0000 7fcc44a00640 1 -- 172.21.15.72:0/422825633 learned_addr learned my addr 172.21.15.72:0/422825633 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:57:10.687 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.686+0000 7fcc3fe00640 1 --2- 172.21.15.72:0/422825633 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5603b084da40 0x5603b085e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:10.688 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.686+0000 7fcc45400640 1 --2- 172.21.15.72:0/422825633 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5603b0852bb0 0x5603b084b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:10.688 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.686+0000 7fcc44a00640 1 -- 172.21.15.72:0/422825633 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5603b084da40 msgr2=0x5603b085e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:10.688 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.686+0000 7fcc44a00640 1 --2- 172.21.15.72:0/422825633 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5603b084da40 0x5603b085e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:10.688 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.686+0000 7fcc44a00640 1 -- 172.21.15.72:0/422825633 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5603b0852bb0 msgr2=0x5603b084b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:10.688 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.686+0000 7fcc44a00640 1 --2- 172.21.15.72:0/422825633 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5603b0852bb0 0x5603b084b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:10.688 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.686+0000 7fcc44a00640 1 -- 172.21.15.72:0/422825633 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5603b058c7d0 con 0x5603b084d050 2024-04-26T21:57:10.688 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.687+0000 7fcc44a00640 1 --2- 172.21.15.72:0/422825633 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5603b084d050 0x5603b084b3a0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fcc3402a9d0 tx=0x7fcc34031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=174c746f5baf1048 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:57:10.688 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.687+0000 7fcc31200640 1 -- 172.21.15.72:0/422825633 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc34035040 con 0x5603b084d050 2024-04-26T21:57:10.688 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.687+0000 7fcc31200640 1 -- 172.21.15.72:0/422825633 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcc34049070 con 0x5603b084d050 2024-04-26T21:57:10.688 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.687+0000 7fcc31200640 1 -- 172.21.15.72:0/422825633 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc34003b00 con 0x5603b084d050 2024-04-26T21:57:10.691 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.688+0000 7fcc49f22740 1 -- 172.21.15.72:0/422825633 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5603b084d050 msgr2=0x5603b084b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:10.691 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.688+0000 7fcc49f22740 1 --2- 172.21.15.72:0/422825633 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5603b084d050 0x5603b084b3a0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fcc3402a9d0 tx=0x7fcc34031400 comp rx=0 tx=0).stop 2024-04-26T21:57:10.691 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.688+0000 7fcc49f22740 1 -- 172.21.15.72:0/422825633 shutdown_connections 2024-04-26T21:57:10.691 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.688+0000 7fcc49f22740 1 --2- 172.21.15.72:0/422825633 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5603b0852bb0 0x5603b084b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:10.691 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.688+0000 7fcc49f22740 1 --2- 172.21.15.72:0/422825633 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5603b084da40 0x5603b085e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:10.691 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.688+0000 7fcc49f22740 1 --2- 172.21.15.72:0/422825633 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5603b084d050 0x5603b084b3a0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:10.691 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.688+0000 7fcc49f22740 1 -- 172.21.15.72:0/422825633 >> 172.21.15.72:0/422825633 conn(0x5603b08486c0 msgr2=0x5603b08477b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:57:10.691 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.688+0000 7fcc49f22740 1 -- 172.21.15.72:0/422825633 shutdown_connections 2024-04-26T21:57:10.691 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.688+0000 7fcc49f22740 1 -- 172.21.15.72:0/422825633 wait complete. 2024-04-26T21:57:10.691 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.689+0000 7fcc49f22740 1 -- start start 2024-04-26T21:57:10.692 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.689+0000 7fcc49f22740 1 Processor -- start 2024-04-26T21:57:10.692 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.689+0000 7fcc49f22740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5603b084d050 0x5603b085b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:10.692 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.689+0000 7fcc49f22740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5603b084da40 0x5603b085bef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:10.692 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.689+0000 7fcc49f22740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5603b0852bb0 0x5603b085cf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:10.692 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.689+0000 7fcc49f22740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fcc401ad480 con 0x5603b084d050 2024-04-26T21:57:10.692 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.689+0000 7fcc49f22740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fcc401acdb0 con 0x5603b0852bb0 2024-04-26T21:57:10.692 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.689+0000 7fcc49f22740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fcc401a9ff0 con 0x5603b084da40 2024-04-26T21:57:10.692 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.689+0000 7fcc49f22740 10 client.0 _ll_register_callbacks cb 0x7fcc380096a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:57:10.692 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.689+0000 7fcc44a00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5603b084d050 0x5603b085b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:10.692 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.690+0000 7fcc44a00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5603b084d050 0x5603b085b9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:48518/0 (socket says 172.21.15.72:48518) 2024-04-26T21:57:10.692 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.690+0000 7fcc44a00640 1 -- 172.21.15.72:0/1581891319 learned_addr learned my addr 172.21.15.72:0/1581891319 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:57:10.692 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.690+0000 7fcc3fe00640 1 --2- 172.21.15.72:0/1581891319 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5603b084da40 0x5603b085bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:10.692 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.690+0000 7fcc45400640 1 --2- 172.21.15.72:0/1581891319 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5603b0852bb0 0x5603b085cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:10.692 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.690+0000 7fcc44a00640 1 -- 172.21.15.72:0/1581891319 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5603b084da40 msgr2=0x5603b085bef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:10.693 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.690+0000 7fcc44a00640 1 --2- 172.21.15.72:0/1581891319 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5603b084da40 0x5603b085bef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:10.693 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.690+0000 7fcc44a00640 1 -- 172.21.15.72:0/1581891319 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5603b0852bb0 msgr2=0x5603b085cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:10.693 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.690+0000 7fcc44a00640 1 --2- 172.21.15.72:0/1581891319 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5603b0852bb0 0x5603b085cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:10.693 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.690+0000 7fcc44a00640 1 -- 172.21.15.72:0/1581891319 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5603b07572a0 con 0x5603b084d050 2024-04-26T21:57:10.693 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.690+0000 7fcc44a00640 1 --2- 172.21.15.72:0/1581891319 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5603b084d050 0x5603b085b9b0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fcc34009850 tx=0x7fcc34009630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e9c505e3650ac67 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:57:10.693 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.690+0000 7fcc26000640 10 client.0 ms_handle_connect on v2:172.21.15.72:3300/0 2024-04-26T21:57:10.693 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.691+0000 7fcc26000640 1 -- 172.21.15.72:0/1581891319 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc34035040 con 0x5603b084d050 2024-04-26T21:57:10.693 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.691+0000 7fcc26000640 1 -- 172.21.15.72:0/1581891319 <== mon.0 v2:172.21.15.72:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7fcc34031500 con 0x5603b084d050 2024-04-26T21:57:10.693 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.691+0000 7fcc49f22740 1 -- 172.21.15.72:0/1581891319 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcc34049070 con 0x5603b084d050 2024-04-26T21:57:10.693 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.691+0000 7fcc26000640 1 -- 172.21.15.72:0/1581891319 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fcc3405b030 con 0x5603b084d050 2024-04-26T21:57:10.693 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.691+0000 7fcc49f22740 10 client.15918 fetch_fsmap learned FSMap version 27 2024-04-26T21:57:10.693 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.691+0000 7fcc49f22740 1 -- 172.21.15.72:0/1581891319 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5603b085e690 con 0x5603b084d050 2024-04-26T21:57:10.695 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.691+0000 7fcc26000640 1 -- 172.21.15.72:0/1581891319 <== mon.0 v2:172.21.15.72:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcc34049070 con 0x5603b084d050 2024-04-26T21:57:10.695 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.692+0000 7fcc26000640 1 -- 172.21.15.72:0/1581891319 <== mon.0 v2:172.21.15.72:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fcc340445b0 con 0x5603b084d050 2024-04-26T21:57:10.695 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.692+0000 7fcc49f22740 10 client.15918 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T21:57:10.695 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.692+0000 7fcc49f22740 10 client.15918 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-26T21:57:10.695 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.692+0000 7fcc49f22740 20 client.15918 populate_metadata read hostname 'smithi072' 2024-04-26T21:57:10.696 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.692+0000 7fcc49f22740 1 --2- 172.21.15.72:0/1581891319 >> [v2:172.21.15.76:6836/1150678991,v1:172.21.15.76:6837/1150678991] conn(0x5603b08c9670 0x5603b08cba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:10.696 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.692+0000 7fcc49f22740 4 client.15918 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-26T21:57:10.696 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.692+0000 7fcc49f22740 1 -- 172.21.15.72:0/1581891319 --> [v2:172.21.15.76:6836/1150678991,v1:172.21.15.76:6837/1150678991] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5603b07b86f0 con 0x5603b08c9670 2024-04-26T21:57:10.696 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.692+0000 7fcc3fe00640 1 --2- 172.21.15.72:0/1581891319 >> [v2:172.21.15.76:6836/1150678991,v1:172.21.15.76:6837/1150678991] conn(0x5603b08c9670 0x5603b08cba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:10.696 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.693+0000 7fcc3fe00640 1 --2- 172.21.15.72:0/1581891319 >> [v2:172.21.15.76:6836/1150678991,v1:172.21.15.76:6837/1150678991] conn(0x5603b08c9670 0x5603b08cba90 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6d3e35ace974c34f server_cookie=c45970e306a22483 in_seq=0 out_seq=0 2024-04-26T21:57:10.696 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.693+0000 7fcc26000640 10 client.15918 ms_handle_connect on v2:172.21.15.76:6836/1150678991 2024-04-26T21:57:10.696 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.694+0000 7fcc26000640 1 -- 172.21.15.72:0/1581891319 <== mds.1 v2:172.21.15.76:6836/1150678991 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5603b07b86f0 con 0x5603b08c9670 2024-04-26T21:57:10.696 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.694+0000 7fcc26000640 10 client.15918 handle_command_reply: tid=0 2024-04-26T21:57:10.696 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.694+0000 7fcc49f22740 1 client.15918 shutdown 2024-04-26T21:57:10.696 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.694+0000 7fcc49f22740 1 -- 172.21.15.72:0/1581891319 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5603b084d050 msgr2=0x5603b085b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:10.696 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.694+0000 7fcc49f22740 1 --2- 172.21.15.72:0/1581891319 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5603b084d050 0x5603b085b9b0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fcc34009850 tx=0x7fcc34009630 comp rx=0 tx=0).stop 2024-04-26T21:57:10.696 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.694+0000 7fcc49f22740 1 -- 172.21.15.72:0/1581891319 shutdown_connections 2024-04-26T21:57:10.696 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.694+0000 7fcc49f22740 1 --2- 172.21.15.72:0/1581891319 >> [v2:172.21.15.76:6836/1150678991,v1:172.21.15.76:6837/1150678991] conn(0x5603b08c9670 0x5603b08cba90 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:10.696 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.694+0000 7fcc49f22740 1 --2- 172.21.15.72:0/1581891319 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5603b0852bb0 0x5603b085cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:10.697 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.694+0000 7fcc49f22740 1 --2- 172.21.15.72:0/1581891319 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5603b084da40 0x5603b085bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:10.697 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.694+0000 7fcc49f22740 1 --2- 172.21.15.72:0/1581891319 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5603b084d050 0x5603b085b9b0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:10.697 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.694+0000 7fcc49f22740 1 -- 172.21.15.72:0/1581891319 >> 172.21.15.72:0/1581891319 conn(0x5603b08486c0 msgr2=0x5603b084e2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:57:10.697 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.694+0000 7fcc26000640 0 client.15918 ms_handle_reset on v2:172.21.15.76:6836/1150678991 2024-04-26T21:57:10.697 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.694+0000 7fcc49f22740 1 -- 172.21.15.72:0/1581891319 shutdown_connections 2024-04-26T21:57:10.697 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.694+0000 7fcc49f22740 1 -- 172.21.15.72:0/1581891319 wait complete. 2024-04-26T21:57:10.697 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.694+0000 7fcc49f22740 20 client.15918 trim_cache size 0 max 16384 2024-04-26T21:57:10.697 INFO:teuthology.orchestra.run.smithi072.stdout:[] 2024-04-26T21:57:10.698 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.695+0000 7fcc45e00640 1 -- 172.21.15.72:0/1539690515 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fcc280a3c40 msgr2=0x7fcc280a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:10.698 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.695+0000 7fcc45e00640 1 --2- 172.21.15.72:0/1539690515 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fcc280a3c40 0x7fcc280a6100 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x5603b084e2a0 tx=0x7fcc38009590 comp rx=0 tx=0).stop 2024-04-26T21:57:10.698 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.695+0000 7fcc45e00640 1 -- 172.21.15.72:0/1539690515 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcc40103ab0 msgr2=0x7fcc4019b130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:10.698 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.695+0000 7fcc45e00640 1 --2- 172.21.15.72:0/1539690515 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcc40103ab0 0x7fcc4019b130 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fcc2c00d950 tx=0x7fcc2c00de10 comp rx=0 tx=0).stop 2024-04-26T21:57:10.698 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.695+0000 7fcc45e00640 1 -- 172.21.15.72:0/1539690515 shutdown_connections 2024-04-26T21:57:10.698 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.695+0000 7fcc45e00640 1 --2- 172.21.15.72:0/1539690515 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fcc280a3c40 0x7fcc280a6100 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:10.698 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.695+0000 7fcc45e00640 1 --2- 172.21.15.72:0/1539690515 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcc40104470 0x7fcc401a21a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:10.698 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.695+0000 7fcc45e00640 1 --2- 172.21.15.72:0/1539690515 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcc40103ab0 0x7fcc4019b130 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:10.699 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.695+0000 7fcc45e00640 1 --2- 172.21.15.72:0/1539690515 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcc401028c0 0x7fcc4019abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:10.699 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.695+0000 7fcc45e00640 1 -- 172.21.15.72:0/1539690515 >> 172.21.15.72:0/1539690515 conn(0x7fcc400fde30 msgr2=0x7fcc40074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:57:10.699 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.695+0000 7fcc45e00640 1 -- 172.21.15.72:0/1539690515 shutdown_connections 2024-04-26T21:57:10.699 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.695+0000 7fcc45e00640 1 -- 172.21.15.72:0/1539690515 wait complete. 2024-04-26T21:57:10.716 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-26T21:57:10.910 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.910+0000 7f2cde200640 1 -- 172.21.15.72:0/452841324 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2cd8101130 msgr2=0x7f2cd8101550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:10.910 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.910+0000 7f2cde200640 1 --2- 172.21.15.72:0/452841324 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2cd8101130 0x7f2cd8101550 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f2ccc009900 tx=0x7f2ccc02a3f0 comp rx=0 tx=0).stop 2024-04-26T21:57:10.910 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.910+0000 7f2cde200640 1 -- 172.21.15.72:0/452841324 shutdown_connections 2024-04-26T21:57:10.911 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.910+0000 7f2cde200640 1 --2- 172.21.15.72:0/452841324 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2cd8102430 0x7f2cd81028b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:10.911 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.910+0000 7f2cde200640 1 --2- 172.21.15.72:0/452841324 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2cd8101a90 0x7f2cd8101ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:10.911 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.910+0000 7f2cde200640 1 --2- 172.21.15.72:0/452841324 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2cd8101130 0x7f2cd8101550 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:10.911 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.910+0000 7f2cde200640 1 -- 172.21.15.72:0/452841324 >> 172.21.15.72:0/452841324 conn(0x7f2cd80fccf0 msgr2=0x7f2cd80ff110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:57:10.911 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.910+0000 7f2cde200640 1 -- 172.21.15.72:0/452841324 shutdown_connections 2024-04-26T21:57:10.911 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.910+0000 7f2cde200640 1 -- 172.21.15.72:0/452841324 wait complete. 2024-04-26T21:57:10.913 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.911+0000 7f2cde200640 1 Processor -- start 2024-04-26T21:57:10.913 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.911+0000 7f2cde200640 1 -- start start 2024-04-26T21:57:10.913 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.911+0000 7f2cde200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2cd8101130 0x7f2cd819ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:10.913 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.911+0000 7f2cde200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2cd8101a90 0x7f2cd819b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:10.914 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.911+0000 7f2cde200640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2cd8102430 0x7f2cd81a2130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:10.914 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.911+0000 7f2cde200640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f2cd8110470 con 0x7f2cd8102430 2024-04-26T21:57:10.914 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.911+0000 7f2cde200640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f2cd81102f0 con 0x7f2cd8101130 2024-04-26T21:57:10.914 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.911+0000 7f2cde200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f2cd81105f0 con 0x7f2cd8101a90 2024-04-26T21:57:10.914 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.912+0000 7f2cdd800640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2cd8102430 0x7f2cd81a2130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:10.914 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.912+0000 7f2cdd800640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2cd8102430 0x7f2cd81a2130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:45530/0 (socket says 172.21.15.72:45530) 2024-04-26T21:57:10.914 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.912+0000 7f2cdd800640 1 -- 172.21.15.72:0/1602492719 learned_addr learned my addr 172.21.15.72:0/1602492719 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:57:10.914 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.912+0000 7f2cdce00640 1 --2- 172.21.15.72:0/1602492719 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2cd8101130 0x7f2cd819ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:10.914 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.912+0000 7f2cd7e00640 1 --2- 172.21.15.72:0/1602492719 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2cd8101a90 0x7f2cd819b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:10.914 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.912+0000 7f2cdd800640 1 -- 172.21.15.72:0/1602492719 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2cd8101a90 msgr2=0x7f2cd819b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:10.914 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.912+0000 7f2cdd800640 1 --2- 172.21.15.72:0/1602492719 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2cd8101a90 0x7f2cd819b0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:10.915 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.912+0000 7f2cdd800640 1 -- 172.21.15.72:0/1602492719 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2cd8101130 msgr2=0x7f2cd819ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:10.915 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.912+0000 7f2cdd800640 1 --2- 172.21.15.72:0/1602492719 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2cd8101130 0x7f2cd819ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:10.915 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.912+0000 7f2cdd800640 1 -- 172.21.15.72:0/1602492719 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cd81a27a0 con 0x7f2cd8102430 2024-04-26T21:57:10.915 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.913+0000 7f2cdd800640 1 --2- 172.21.15.72:0/1602492719 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2cd8102430 0x7f2cd81a2130 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f2cd000eaa0 tx=0x7f2cd000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:57:10.915 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.913+0000 7f2cd5600640 1 -- 172.21.15.72:0/1602492719 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cd000cdb0 con 0x7f2cd8102430 2024-04-26T21:57:10.915 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.913+0000 7f2cd5600640 1 -- 172.21.15.72:0/1602492719 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2cd0005020 con 0x7f2cd8102430 2024-04-26T21:57:10.915 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.913+0000 7f2cde200640 1 -- 172.21.15.72:0/1602492719 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2cd81a2a90 con 0x7f2cd8102430 2024-04-26T21:57:10.915 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.913+0000 7f2cd5600640 1 -- 172.21.15.72:0/1602492719 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cd00116d0 con 0x7f2cd8102430 2024-04-26T21:57:10.915 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.913+0000 7f2cde200640 1 -- 172.21.15.72:0/1602492719 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2cd81a2f90 con 0x7f2cd8102430 2024-04-26T21:57:10.916 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.917+0000 7f2ce25cf740 1 Processor -- start 2024-04-26T21:57:10.917 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.917+0000 7f2cd5600640 1 -- 172.21.15.72:0/1602492719 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f2cd00051d0 con 0x7f2cd8102430 2024-04-26T21:57:10.919 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.917+0000 7f2ce25cf740 1 -- start start 2024-04-26T21:57:10.919 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.918+0000 7f2ce25cf740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557056ba0050 0x557056ba0470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:10.919 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.918+0000 7f2ce25cf740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557056ba0a40 0x557056ba5540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:10.919 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.918+0000 7f2ce25cf740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557056ba5bb0 0x557056ba8040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:10.919 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.918+0000 7f2ce25cf740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x557056b03c70 con 0x557056ba5bb0 2024-04-26T21:57:10.919 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.918+0000 7f2ce25cf740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x557056abc980 con 0x557056ba0050 2024-04-26T21:57:10.920 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.918+0000 7f2ce25cf740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x557056affc30 con 0x557056ba0a40 2024-04-26T21:57:10.920 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.918+0000 7f2cd7e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557056ba5bb0 0x557056ba8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:10.920 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.918+0000 7f2cd7e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557056ba5bb0 0x557056ba8040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:45534/0 (socket says 172.21.15.72:45534) 2024-04-26T21:57:10.920 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.918+0000 7f2cd7e00640 1 -- 172.21.15.72:0/2041776071 learned_addr learned my addr 172.21.15.72:0/2041776071 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:57:10.920 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.918+0000 7f2cd7e00640 1 --2- 172.21.15.72:0/2041776071 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557056ba0050 0x557056ba0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:10.920 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.919+0000 7f2cdce00640 1 --2- 172.21.15.72:0/2041776071 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557056ba0a40 0x557056ba5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:10.920 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.919+0000 7f2cd5600640 1 --2- 172.21.15.72:0/1602492719 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f2cc00a3c40 0x7f2cc00a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:10.920 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.919+0000 7f2cd7e00640 1 -- 172.21.15.72:0/2041776071 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557056ba0a40 msgr2=0x557056ba5540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:10.920 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.919+0000 7f2cd7e00640 1 --2- 172.21.15.72:0/2041776071 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557056ba0a40 0x557056ba5540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:10.920 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.919+0000 7f2cd5600640 1 -- 172.21.15.72:0/1602492719 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7f2cd00d0910 con 0x7f2cd8102430 2024-04-26T21:57:10.920 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.919+0000 7f2cd7e00640 1 -- 172.21.15.72:0/2041776071 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557056ba0050 msgr2=0x557056ba0470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:10.920 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.919+0000 7f2cd7e00640 1 --2- 172.21.15.72:0/2041776071 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557056ba0050 0x557056ba0470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:10.920 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.919+0000 7f2cd7e00640 1 -- 172.21.15.72:0/2041776071 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557056a947b0 con 0x557056ba5bb0 2024-04-26T21:57:10.920 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.919+0000 7f2cdd800640 1 --2- 172.21.15.72:0/1602492719 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f2cc00a3c40 0x7f2cc00a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:10.923 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.920+0000 7f2cd7e00640 1 --2- 172.21.15.72:0/2041776071 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557056ba5bb0 0x557056ba8040 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f2cc40058c0 tx=0x7f2cc400c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa089332a9d1785a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:57:10.923 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.920+0000 7f2cdd800640 1 --2- 172.21.15.72:0/1602492719 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f2cc00a3c40 0x7f2cc00a6100 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f2cd00993e0 tx=0x7f2cd00057c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:57:10.923 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.920+0000 7f2cca000640 1 -- 172.21.15.72:0/2041776071 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cc40142d0 con 0x557056ba5bb0 2024-04-26T21:57:10.923 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.920+0000 7f2cca000640 1 -- 172.21.15.72:0/2041776071 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2cc40054b0 con 0x557056ba5bb0 2024-04-26T21:57:10.923 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.920+0000 7f2cca000640 1 -- 172.21.15.72:0/2041776071 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cc400f760 con 0x557056ba5bb0 2024-04-26T21:57:10.923 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.921+0000 7f2ce25cf740 1 -- 172.21.15.72:0/2041776071 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557056ba5bb0 msgr2=0x557056ba8040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:10.923 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.921+0000 7f2ce25cf740 1 --2- 172.21.15.72:0/2041776071 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557056ba5bb0 0x557056ba8040 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f2cc40058c0 tx=0x7f2cc400c4d0 comp rx=0 tx=0).stop 2024-04-26T21:57:10.923 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.921+0000 7f2ce25cf740 1 -- 172.21.15.72:0/2041776071 shutdown_connections 2024-04-26T21:57:10.923 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.921+0000 7f2ce25cf740 1 --2- 172.21.15.72:0/2041776071 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557056ba5bb0 0x557056ba8040 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:10.923 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.921+0000 7f2ce25cf740 1 --2- 172.21.15.72:0/2041776071 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557056ba0a40 0x557056ba5540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:10.923 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.921+0000 7f2ce25cf740 1 --2- 172.21.15.72:0/2041776071 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557056ba0050 0x557056ba0470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:10.923 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.921+0000 7f2ce25cf740 1 -- 172.21.15.72:0/2041776071 >> 172.21.15.72:0/2041776071 conn(0x557056b9b6c0 msgr2=0x557056b9bac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:57:10.923 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.921+0000 7f2ce25cf740 1 -- 172.21.15.72:0/2041776071 shutdown_connections 2024-04-26T21:57:10.924 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.921+0000 7f2ce25cf740 1 -- 172.21.15.72:0/2041776071 wait complete. 2024-04-26T21:57:10.924 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.922+0000 7f2ce25cf740 1 -- start start 2024-04-26T21:57:10.924 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.922+0000 7f2ce25cf740 1 Processor -- start 2024-04-26T21:57:10.924 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.922+0000 7f2ce25cf740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557056ba0050 0x557056ba9f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:10.924 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.922+0000 7f2ce25cf740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557056ba0a40 0x557056baa460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:10.924 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.922+0000 7f2ce25cf740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557056ba5bb0 0x557056bb14a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:10.924 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.922+0000 7f2ce25cf740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x557056b02f80 con 0x557056ba5bb0 2024-04-26T21:57:10.924 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.922+0000 7f2ce25cf740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x557056ab0260 con 0x557056ba0050 2024-04-26T21:57:10.924 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.922+0000 7f2ce25cf740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x557056b082a0 con 0x557056ba0a40 2024-04-26T21:57:10.924 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.922+0000 7f2ce25cf740 10 client.0 _ll_register_callbacks cb 0x557056afa960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:57:10.924 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.923+0000 7f2cd7e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557056ba5bb0 0x557056bb14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:10.924 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.923+0000 7f2cd7e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557056ba5bb0 0x557056bb14a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:45548/0 (socket says 172.21.15.72:45548) 2024-04-26T21:57:10.924 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.923+0000 7f2cd7e00640 1 -- 172.21.15.72:0/3674877265 learned_addr learned my addr 172.21.15.72:0/3674877265 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:57:10.924 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.923+0000 7f2cdce00640 1 --2- 172.21.15.72:0/3674877265 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557056ba0a40 0x557056baa460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:10.924 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.923+0000 7f2cd7e00640 1 --2- 172.21.15.72:0/3674877265 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557056ba0050 0x557056ba9f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:10.927 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.923+0000 7f2cd7e00640 1 -- 172.21.15.72:0/3674877265 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557056ba0a40 msgr2=0x557056baa460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:10.927 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.923+0000 7f2cd7e00640 1 --2- 172.21.15.72:0/3674877265 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557056ba0a40 0x557056baa460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:10.927 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.923+0000 7f2cd7e00640 1 -- 172.21.15.72:0/3674877265 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557056ba0050 msgr2=0x557056ba9f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:10.927 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.923+0000 7f2cd7e00640 1 --2- 172.21.15.72:0/3674877265 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557056ba0050 0x557056ba9f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:10.927 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.923+0000 7f2cd7e00640 1 -- 172.21.15.72:0/3674877265 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557056a921c0 con 0x557056ba5bb0 2024-04-26T21:57:10.928 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.924+0000 7f2cd7e00640 1 --2- 172.21.15.72:0/3674877265 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557056ba5bb0 0x557056bb14a0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f2cc4014fd0 tx=0x7f2cc4005c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63bf6171c430edde server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:57:10.928 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.924+0000 7f2cbea00640 10 client.0 ms_handle_connect on v2:172.21.15.72:3300/0 2024-04-26T21:57:10.928 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.924+0000 7f2cbea00640 1 -- 172.21.15.72:0/3674877265 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cc4004730 con 0x557056ba5bb0 2024-04-26T21:57:10.928 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.924+0000 7f2cbea00640 1 -- 172.21.15.72:0/3674877265 <== mon.0 v2:172.21.15.72:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7f2cc4005120 con 0x557056ba5bb0 2024-04-26T21:57:10.928 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.924+0000 7f2ce25cf740 1 -- 172.21.15.72:0/3674877265 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2cc40054b0 con 0x557056ba5bb0 2024-04-26T21:57:10.928 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.925+0000 7f2cbea00640 1 -- 172.21.15.72:0/3674877265 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2cc400f760 con 0x557056ba5bb0 2024-04-26T21:57:10.928 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.925+0000 7f2ce25cf740 10 client.15930 fetch_fsmap learned FSMap version 27 2024-04-26T21:57:10.928 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.925+0000 7f2ce25cf740 1 -- 172.21.15.72:0/3674877265 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5570568153f0 con 0x557056ba5bb0 2024-04-26T21:57:10.928 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.925+0000 7f2cbea00640 1 -- 172.21.15.72:0/3674877265 <== mon.0 v2:172.21.15.72:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2cc40054b0 con 0x557056ba5bb0 2024-04-26T21:57:10.928 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.925+0000 7f2cbea00640 1 -- 172.21.15.72:0/3674877265 <== mon.0 v2:172.21.15.72:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f2cc40206e0 con 0x557056ba5bb0 2024-04-26T21:57:10.928 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.925+0000 7f2ce25cf740 10 client.15930 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T21:57:10.928 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.925+0000 7f2ce25cf740 10 client.15930 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:57:10.928 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.925+0000 7f2ce25cf740 20 client.15930 populate_metadata read hostname 'smithi072' 2024-04-26T21:57:10.928 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.925+0000 7f2ce25cf740 1 --2- 172.21.15.72:0/3674877265 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x557056bb56b0 0x557056bb7aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:10.928 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.925+0000 7f2ce25cf740 4 client.15930 mds_command: new command op to 34517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:57:10.929 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.925+0000 7f2ce25cf740 1 -- 172.21.15.72:0/3674877265 --> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557056b11880 con 0x557056bb56b0 2024-04-26T21:57:10.929 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.926+0000 7f2cdce00640 1 --2- 172.21.15.72:0/3674877265 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x557056bb56b0 0x557056bb7aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:10.929 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.927+0000 7f2cdce00640 1 --2- 172.21.15.72:0/3674877265 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x557056bb56b0 0x557056bb7aa0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ee86c475e9a51b6 server_cookie=cb6a4430ad9fe8a9 in_seq=0 out_seq=0 2024-04-26T21:57:10.929 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.927+0000 7f2cbea00640 10 client.15930 ms_handle_connect on v2:172.21.15.76:6838/327145636 2024-04-26T21:57:10.930 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.929+0000 7f2cbea00640 1 -- 172.21.15.72:0/3674877265 <== mds.0 v2:172.21.15.76:6838/327145636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557056b11880 con 0x557056bb56b0 2024-04-26T21:57:10.930 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.929+0000 7f2cbea00640 10 client.15930 handle_command_reply: tid=0 2024-04-26T21:57:10.930 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.929+0000 7f2ce25cf740 1 client.15930 shutdown 2024-04-26T21:57:10.930 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.929+0000 7f2ce25cf740 1 -- 172.21.15.72:0/3674877265 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557056ba5bb0 msgr2=0x557056bb14a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:10.930 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.929+0000 7f2ce25cf740 1 --2- 172.21.15.72:0/3674877265 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557056ba5bb0 0x557056bb14a0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f2cc4014fd0 tx=0x7f2cc4005c50 comp rx=0 tx=0).stop 2024-04-26T21:57:10.930 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.929+0000 7f2ce25cf740 1 -- 172.21.15.72:0/3674877265 shutdown_connections 2024-04-26T21:57:10.930 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.929+0000 7f2ce25cf740 1 --2- 172.21.15.72:0/3674877265 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x557056bb56b0 0x557056bb7aa0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:10.930 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.929+0000 7f2ce25cf740 1 --2- 172.21.15.72:0/3674877265 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557056ba5bb0 0x557056bb14a0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:10.930 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.929+0000 7f2ce25cf740 1 --2- 172.21.15.72:0/3674877265 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557056ba0a40 0x557056baa460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:10.931 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.929+0000 7f2ce25cf740 1 --2- 172.21.15.72:0/3674877265 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557056ba0050 0x557056ba9f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:10.931 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.929+0000 7f2ce25cf740 1 -- 172.21.15.72:0/3674877265 >> 172.21.15.72:0/3674877265 conn(0x557056b9b6c0 msgr2=0x557056b9a7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:57:10.931 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.929+0000 7f2cbea00640 0 client.15930 ms_handle_reset on v2:172.21.15.76:6838/327145636 2024-04-26T21:57:10.931 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.929+0000 7f2ce25cf740 1 -- 172.21.15.72:0/3674877265 shutdown_connections 2024-04-26T21:57:10.931 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.930+0000 7f2ce25cf740 1 -- 172.21.15.72:0/3674877265 wait complete. 2024-04-26T21:57:10.931 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.930+0000 7f2ce25cf740 20 client.15930 trim_cache size 0 max 16384 2024-04-26T21:57:10.946 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.945+0000 7f2ce25cf740 1 Processor -- start 2024-04-26T21:57:10.947 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.945+0000 7f2ce25cf740 1 -- start start 2024-04-26T21:57:10.947 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.945+0000 7f2ce25cf740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557056ba0050 0x557056ba1130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:10.947 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.945+0000 7f2ce25cf740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557056ba0a40 0x557056bb13c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:10.947 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.945+0000 7f2ce25cf740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557056ba5bb0 0x557056ba1670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:10.947 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.946+0000 7f2ce25cf740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f2cd81abf30 con 0x557056ba0050 2024-04-26T21:57:10.947 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.946+0000 7f2ce25cf740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f2cd81ada70 con 0x557056ba5bb0 2024-04-26T21:57:10.947 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.946+0000 7f2ce25cf740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f2cd81ac600 con 0x557056ba0a40 2024-04-26T21:57:10.947 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.946+0000 7f2cd7e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557056ba0050 0x557056ba1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:10.947 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.946+0000 7f2cd7e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557056ba0050 0x557056ba1130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:45552/0 (socket says 172.21.15.72:45552) 2024-04-26T21:57:10.947 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.946+0000 7f2cd7e00640 1 -- 172.21.15.72:0/1102620484 learned_addr learned my addr 172.21.15.72:0/1102620484 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:57:10.947 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.946+0000 7f2cdce00640 1 --2- 172.21.15.72:0/1102620484 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557056ba0a40 0x557056bb13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:10.947 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.946+0000 7f2cd7e00640 1 -- 172.21.15.72:0/1102620484 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557056ba0a40 msgr2=0x557056bb13c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:10.947 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.946+0000 7f2cd7e00640 1 --2- 172.21.15.72:0/1102620484 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557056ba0a40 0x557056bb13c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:10.948 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.946+0000 7f2cd7e00640 1 -- 172.21.15.72:0/1102620484 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557056ba5bb0 msgr2=0x557056ba1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:10.948 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.946+0000 7f2cd7e00640 1 --2- 172.21.15.72:0/1102620484 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557056ba5bb0 0x557056ba1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:10.948 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.946+0000 7f2cd7e00640 1 -- 172.21.15.72:0/1102620484 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5570568df7d0 con 0x557056ba0050 2024-04-26T21:57:10.948 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.946+0000 7f2cdce00640 1 --2- 172.21.15.72:0/1102620484 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557056ba0a40 0x557056bb13c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:57:10.948 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.947+0000 7f2cd7e00640 1 --2- 172.21.15.72:0/1102620484 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557056ba0050 0x557056ba1130 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f2cc400c410 tx=0x7f2cc4005970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5101f6296bb40e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:57:10.950 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.947+0000 7f2cc9600640 1 -- 172.21.15.72:0/1102620484 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cc4026070 con 0x557056ba0050 2024-04-26T21:57:10.951 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.947+0000 7f2cc9600640 1 -- 172.21.15.72:0/1102620484 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2cc400c990 con 0x557056ba0050 2024-04-26T21:57:10.951 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.947+0000 7f2cc9600640 1 -- 172.21.15.72:0/1102620484 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cc4004730 con 0x557056ba0050 2024-04-26T21:57:10.951 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.948+0000 7f2ce25cf740 1 -- 172.21.15.72:0/1102620484 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557056ba0050 msgr2=0x557056ba1130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:10.951 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.948+0000 7f2ce25cf740 1 --2- 172.21.15.72:0/1102620484 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557056ba0050 0x557056ba1130 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f2cc400c410 tx=0x7f2cc4005970 comp rx=0 tx=0).stop 2024-04-26T21:57:10.951 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.948+0000 7f2ce25cf740 1 -- 172.21.15.72:0/1102620484 shutdown_connections 2024-04-26T21:57:10.951 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.948+0000 7f2ce25cf740 1 --2- 172.21.15.72:0/1102620484 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557056ba5bb0 0x557056ba1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:10.951 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.948+0000 7f2ce25cf740 1 --2- 172.21.15.72:0/1102620484 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557056ba0a40 0x557056bb13c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:10.951 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.948+0000 7f2ce25cf740 1 --2- 172.21.15.72:0/1102620484 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557056ba0050 0x557056ba1130 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:10.951 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.948+0000 7f2ce25cf740 1 -- 172.21.15.72:0/1102620484 >> 172.21.15.72:0/1102620484 conn(0x557056b9b6c0 msgr2=0x557056b9ef40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:57:10.951 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.948+0000 7f2ce25cf740 1 -- 172.21.15.72:0/1102620484 shutdown_connections 2024-04-26T21:57:10.951 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.948+0000 7f2ce25cf740 1 -- 172.21.15.72:0/1102620484 wait complete. 2024-04-26T21:57:10.951 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.948+0000 7f2ce25cf740 1 -- start start 2024-04-26T21:57:10.953 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.949+0000 7f2ce25cf740 1 Processor -- start 2024-04-26T21:57:10.953 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.949+0000 7f2ce25cf740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557056ba0050 0x557056bae800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:10.953 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.949+0000 7f2ce25cf740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557056ba0a40 0x557056baed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:10.953 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.949+0000 7f2ce25cf740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557056ba5bb0 0x557056bafd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:10.953 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.949+0000 7f2ce25cf740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f2cd81ad3a0 con 0x557056ba5bb0 2024-04-26T21:57:10.953 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.949+0000 7f2ce25cf740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f2cd81accd0 con 0x557056ba0050 2024-04-26T21:57:10.953 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.949+0000 7f2ce25cf740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f2cd81a9f10 con 0x557056ba0a40 2024-04-26T21:57:10.953 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.949+0000 7f2ce25cf740 10 client.0 _ll_register_callbacks cb 0x7f2cc400cf30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:57:10.953 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.949+0000 7f2cd7e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557056ba5bb0 0x557056bafd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:10.953 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.949+0000 7f2cd7e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557056ba5bb0 0x557056bafd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:45562/0 (socket says 172.21.15.72:45562) 2024-04-26T21:57:10.954 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.949+0000 7f2cd7e00640 1 -- 172.21.15.72:0/3254748258 learned_addr learned my addr 172.21.15.72:0/3254748258 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:57:10.954 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.950+0000 7f2cdce00640 1 --2- 172.21.15.72:0/3254748258 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557056ba0a40 0x557056baed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:10.954 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.950+0000 7f2cd7e00640 1 --2- 172.21.15.72:0/3254748258 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557056ba0050 0x557056bae800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:10.954 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.950+0000 7f2cd7e00640 1 -- 172.21.15.72:0/3254748258 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557056ba0a40 msgr2=0x557056baed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:10.954 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.950+0000 7f2cd7e00640 1 --2- 172.21.15.72:0/3254748258 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557056ba0a40 0x557056baed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:10.954 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.950+0000 7f2cd7e00640 1 -- 172.21.15.72:0/3254748258 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557056ba0050 msgr2=0x557056bae800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:10.954 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.950+0000 7f2cd7e00640 1 --2- 172.21.15.72:0/3254748258 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557056ba0050 0x557056bae800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:10.954 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.950+0000 7f2cd7e00640 1 -- 172.21.15.72:0/3254748258 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557056aaa2a0 con 0x557056ba5bb0 2024-04-26T21:57:10.954 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.950+0000 7f2cd7e00640 1 --2- 172.21.15.72:0/3254748258 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557056ba5bb0 0x557056bafd50 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f2cc4006010 tx=0x7f2cc4021fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49620a2be18cc697 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:57:10.954 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.950+0000 7f2cbea00640 10 client.0 ms_handle_connect on v2:172.21.15.72:3300/0 2024-04-26T21:57:10.954 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.950+0000 7f2cbea00640 1 -- 172.21.15.72:0/3254748258 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cc4026070 con 0x557056ba5bb0 2024-04-26T21:57:10.954 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.950+0000 7f2cbea00640 1 -- 172.21.15.72:0/3254748258 <== mon.0 v2:172.21.15.72:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7f2cc4005320 con 0x557056ba5bb0 2024-04-26T21:57:10.954 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.950+0000 7f2ce25cf740 1 -- 172.21.15.72:0/3254748258 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2cc400c990 con 0x557056ba5bb0 2024-04-26T21:57:10.954 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.951+0000 7f2cbea00640 1 -- 172.21.15.72:0/3254748258 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2cc4005520 con 0x557056ba5bb0 2024-04-26T21:57:10.955 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.951+0000 7f2ce25cf740 10 client.15936 fetch_fsmap learned FSMap version 27 2024-04-26T21:57:10.955 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.951+0000 7f2ce25cf740 1 -- 172.21.15.72:0/3254748258 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x557056bb1470 con 0x557056ba5bb0 2024-04-26T21:57:10.956 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.951+0000 7f2cbea00640 1 -- 172.21.15.72:0/3254748258 <== mon.0 v2:172.21.15.72:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2cc400c990 con 0x557056ba5bb0 2024-04-26T21:57:10.956 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.951+0000 7f2cbea00640 1 -- 172.21.15.72:0/3254748258 <== mon.0 v2:172.21.15.72:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f2cc4039020 con 0x557056ba5bb0 2024-04-26T21:57:10.956 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.951+0000 7f2ce25cf740 10 client.15936 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T21:57:10.956 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.951+0000 7f2ce25cf740 10 client.15936 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:57:10.957 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.951+0000 7f2ce25cf740 20 client.15936 populate_metadata read hostname 'smithi072' 2024-04-26T21:57:10.957 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.951+0000 7f2ce25cf740 1 --2- 172.21.15.72:0/3254748258 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x557056c1bf10 0x557056c1e300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:10.957 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.951+0000 7f2ce25cf740 4 client.15936 mds_command: new command op to 34517 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-26T21:57:10.957 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.951+0000 7f2ce25cf740 1 -- 172.21.15.72:0/3254748258 --> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557056b0b6f0 con 0x557056c1bf10 2024-04-26T21:57:10.957 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.952+0000 7f2cdce00640 1 --2- 172.21.15.72:0/3254748258 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x557056c1bf10 0x557056c1e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:10.957 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.953+0000 7f2cdce00640 1 --2- 172.21.15.72:0/3254748258 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x557056c1bf10 0x557056c1e300 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5d7587a9a8669b9b server_cookie=3ed9db1e261ab616 in_seq=0 out_seq=0 2024-04-26T21:57:10.957 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.953+0000 7f2cbea00640 10 client.15936 ms_handle_connect on v2:172.21.15.76:6838/327145636 2024-04-26T21:57:10.957 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.954+0000 7f2cbea00640 1 -- 172.21.15.72:0/3254748258 <== mds.0 v2:172.21.15.76:6838/327145636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557056b0b6f0 con 0x557056c1bf10 2024-04-26T21:57:10.957 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.954+0000 7f2cbea00640 10 client.15936 handle_command_reply: tid=0 2024-04-26T21:57:10.957 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.954+0000 7f2ce25cf740 1 client.15936 shutdown 2024-04-26T21:57:10.957 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.954+0000 7f2ce25cf740 1 -- 172.21.15.72:0/3254748258 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557056ba5bb0 msgr2=0x557056bafd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:10.957 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.954+0000 7f2ce25cf740 1 --2- 172.21.15.72:0/3254748258 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557056ba5bb0 0x557056bafd50 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f2cc4006010 tx=0x7f2cc4021fd0 comp rx=0 tx=0).stop 2024-04-26T21:57:10.957 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.954+0000 7f2ce25cf740 1 -- 172.21.15.72:0/3254748258 shutdown_connections 2024-04-26T21:57:10.957 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.954+0000 7f2ce25cf740 1 --2- 172.21.15.72:0/3254748258 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x557056c1bf10 0x557056c1e300 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:10.958 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.954+0000 7f2ce25cf740 1 --2- 172.21.15.72:0/3254748258 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557056ba5bb0 0x557056bafd50 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:10.958 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.954+0000 7f2ce25cf740 1 --2- 172.21.15.72:0/3254748258 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557056ba0a40 0x557056baed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:10.958 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.954+0000 7f2ce25cf740 1 --2- 172.21.15.72:0/3254748258 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557056ba0050 0x557056bae800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:10.958 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.954+0000 7f2cbea00640 0 client.15936 ms_handle_reset on v2:172.21.15.76:6838/327145636 2024-04-26T21:57:10.958 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.954+0000 7f2ce25cf740 1 -- 172.21.15.72:0/3254748258 >> 172.21.15.72:0/3254748258 conn(0x557056b9b6c0 msgr2=0x557056b99f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:57:10.958 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.954+0000 7f2ce25cf740 1 -- 172.21.15.72:0/3254748258 shutdown_connections 2024-04-26T21:57:10.958 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.954+0000 7f2ce25cf740 1 -- 172.21.15.72:0/3254748258 wait complete. 2024-04-26T21:57:10.958 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.954+0000 7f2ce25cf740 20 client.15936 trim_cache size 0 max 16384 2024-04-26T21:57:10.958 INFO:teuthology.orchestra.run.smithi072.stdout:[] 2024-04-26T21:57:10.959 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.955+0000 7f2cde200640 1 -- 172.21.15.72:0/1602492719 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f2cc00a3c40 msgr2=0x7f2cc00a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:10.959 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.955+0000 7f2cde200640 1 --2- 172.21.15.72:0/1602492719 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f2cc00a3c40 0x7f2cc00a6100 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f2cd00993e0 tx=0x7f2cd00057c0 comp rx=0 tx=0).stop 2024-04-26T21:57:10.960 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.955+0000 7f2cde200640 1 -- 172.21.15.72:0/1602492719 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2cd8102430 msgr2=0x7f2cd81a2130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:10.960 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.955+0000 7f2cde200640 1 --2- 172.21.15.72:0/1602492719 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2cd8102430 0x7f2cd81a2130 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f2cd000eaa0 tx=0x7f2cd000ef60 comp rx=0 tx=0).stop 2024-04-26T21:57:10.960 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.955+0000 7f2cde200640 1 -- 172.21.15.72:0/1602492719 shutdown_connections 2024-04-26T21:57:10.960 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.955+0000 7f2cde200640 1 --2- 172.21.15.72:0/1602492719 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2cd8102430 0x7f2cd81a2130 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:10.960 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.955+0000 7f2cde200640 1 --2- 172.21.15.72:0/1602492719 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f2cc00a3c40 0x7f2cc00a6100 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:10.960 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.955+0000 7f2cde200640 1 --2- 172.21.15.72:0/1602492719 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2cd8101a90 0x7f2cd819b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:10.960 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.955+0000 7f2cde200640 1 --2- 172.21.15.72:0/1602492719 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2cd8101130 0x7f2cd819ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:10.960 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.955+0000 7f2cde200640 1 -- 172.21.15.72:0/1602492719 >> 172.21.15.72:0/1602492719 conn(0x7f2cd80fccf0 msgr2=0x7f2cd8100030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:57:10.960 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.955+0000 7f2cde200640 1 -- 172.21.15.72:0/1602492719 shutdown_connections 2024-04-26T21:57:10.960 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:10.955+0000 7f2cde200640 1 -- 172.21.15.72:0/1602492719 wait complete. 2024-04-26T21:57:10.973 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:57:10 smithi072 ceph-mon[28138]: pgmap v501: 97 pgs: 97 active+clean; 17 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 1.3 MiB/s wr, 14 op/s 2024-04-26T21:57:11.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:57:10 smithi076 ceph-mon[31823]: pgmap v501: 97 pgs: 97 active+clean; 17 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 1.3 MiB/s wr, 14 op/s 2024-04-26T21:57:11.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:57:10 smithi164 ceph-mon[31916]: pgmap v501: 97 pgs: 97 active+clean; 17 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 1.3 MiB/s wr, 14 op/s 2024-04-26T21:57:11.977 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-26T21:57:11.977 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:57:11.977 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-26T21:57:12.201 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.201+0000 7fd073400640 1 -- 172.21.15.72:0/3727179245 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd06c103400 msgr2=0x7fd06c1057f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:12.202 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.201+0000 7fd073400640 1 --2- 172.21.15.72:0/3727179245 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd06c103400 0x7fd06c1057f0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fd060009900 tx=0x7fd06002a3f0 comp rx=0 tx=0).stop 2024-04-26T21:57:12.202 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.201+0000 7fd073400640 1 -- 172.21.15.72:0/3727179245 shutdown_connections 2024-04-26T21:57:12.202 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.201+0000 7fd073400640 1 --2- 172.21.15.72:0/3727179245 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd06c108660 0x7fd06c10aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:12.202 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.201+0000 7fd073400640 1 --2- 172.21.15.72:0/3727179245 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd06c105d30 0x7fd06c108120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:12.202 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.201+0000 7fd073400640 1 --2- 172.21.15.72:0/3727179245 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd06c103400 0x7fd06c1057f0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:12.202 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.201+0000 7fd073400640 1 -- 172.21.15.72:0/3727179245 >> 172.21.15.72:0/3727179245 conn(0x7fd06c0fcdf0 msgr2=0x7fd06c0ff230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:57:12.202 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.201+0000 7fd073400640 1 -- 172.21.15.72:0/3727179245 shutdown_connections 2024-04-26T21:57:12.202 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.201+0000 7fd073400640 1 -- 172.21.15.72:0/3727179245 wait complete. 2024-04-26T21:57:12.204 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.202+0000 7fd073400640 1 Processor -- start 2024-04-26T21:57:12.204 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.202+0000 7fd073400640 1 -- start start 2024-04-26T21:57:12.205 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.203+0000 7fd073400640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd06c103400 0x7fd06c19ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:12.205 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.203+0000 7fd073400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd06c105d30 0x7fd06c19b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:12.205 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.203+0000 7fd073400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd06c108660 0x7fd06c1a2310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:12.205 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.203+0000 7fd073400640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fd06c10e3b0 con 0x7fd06c105d30 2024-04-26T21:57:12.205 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.203+0000 7fd073400640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fd06c10e230 con 0x7fd06c103400 2024-04-26T21:57:12.205 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.203+0000 7fd073400640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd06c10e530 con 0x7fd06c108660 2024-04-26T21:57:12.205 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.203+0000 7fd06be00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd06c105d30 0x7fd06c19b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:12.205 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.203+0000 7fd06be00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd06c105d30 0x7fd06c19b2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:45574/0 (socket says 172.21.15.72:45574) 2024-04-26T21:57:12.205 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.203+0000 7fd06be00640 1 -- 172.21.15.72:0/4215050531 learned_addr learned my addr 172.21.15.72:0/4215050531 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:57:12.205 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.203+0000 7fd070a00640 1 --2- 172.21.15.72:0/4215050531 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd06c103400 0x7fd06c19ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:12.205 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.203+0000 7fd071400640 1 --2- 172.21.15.72:0/4215050531 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd06c108660 0x7fd06c1a2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:12.205 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.204+0000 7fd06be00640 1 -- 172.21.15.72:0/4215050531 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd06c108660 msgr2=0x7fd06c1a2310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:12.205 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.204+0000 7fd06be00640 1 --2- 172.21.15.72:0/4215050531 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd06c108660 0x7fd06c1a2310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:12.205 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.204+0000 7fd06be00640 1 -- 172.21.15.72:0/4215050531 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd06c103400 msgr2=0x7fd06c19ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:12.206 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.204+0000 7fd06be00640 1 --2- 172.21.15.72:0/4215050531 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd06c103400 0x7fd06c19ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:12.206 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.204+0000 7fd06be00640 1 -- 172.21.15.72:0/4215050531 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd06c1a2980 con 0x7fd06c105d30 2024-04-26T21:57:12.206 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.204+0000 7fd071400640 1 --2- 172.21.15.72:0/4215050531 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd06c108660 0x7fd06c1a2310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:57:12.206 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.204+0000 7fd06be00640 1 --2- 172.21.15.72:0/4215050531 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd06c105d30 0x7fd06c19b2a0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fd05800d770 tx=0x7fd05800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:57:12.206 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.204+0000 7fd069600640 1 -- 172.21.15.72:0/4215050531 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd058017070 con 0x7fd06c105d30 2024-04-26T21:57:12.206 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.204+0000 7fd069600640 1 -- 172.21.15.72:0/4215050531 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd058009c30 con 0x7fd06c105d30 2024-04-26T21:57:12.206 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.204+0000 7fd073400640 1 -- 172.21.15.72:0/4215050531 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd06c1a2c70 con 0x7fd06c105d30 2024-04-26T21:57:12.206 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.204+0000 7fd069600640 1 -- 172.21.15.72:0/4215050531 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd05800f620 con 0x7fd06c105d30 2024-04-26T21:57:12.206 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.204+0000 7fd073400640 1 -- 172.21.15.72:0/4215050531 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd06c1a31c0 con 0x7fd06c105d30 2024-04-26T21:57:12.208 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.208+0000 7fd076258740 1 Processor -- start 2024-04-26T21:57:12.208 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.208+0000 7fd069600640 1 -- 172.21.15.72:0/4215050531 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fd05801a070 con 0x7fd06c105d30 2024-04-26T21:57:12.208 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.208+0000 7fd076258740 1 -- start start 2024-04-26T21:57:12.210 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.208+0000 7fd076258740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x555f51fe8860 0x555f51fe8c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:12.210 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.208+0000 7fd076258740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555f51fe9250 0x555f51fedd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:12.210 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.209+0000 7fd076258740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555f51fee3c0 0x555f51ff0850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:12.210 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.209+0000 7fd076258740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x555f51f4d980 con 0x555f51fe8860 2024-04-26T21:57:12.210 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.209+0000 7fd076258740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x555f51eea220 con 0x555f51fee3c0 2024-04-26T21:57:12.210 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.209+0000 7fd076258740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x555f51f49940 con 0x555f51fe9250 2024-04-26T21:57:12.210 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.209+0000 7fd071400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x555f51fe8860 0x555f51fe8c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:12.210 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.209+0000 7fd071400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x555f51fe8860 0x555f51fe8c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:45590/0 (socket says 172.21.15.72:45590) 2024-04-26T21:57:12.210 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.209+0000 7fd071400640 1 -- 172.21.15.72:0/2410924171 learned_addr learned my addr 172.21.15.72:0/2410924171 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:57:12.210 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.209+0000 7fd070a00640 1 --2- 172.21.15.72:0/2410924171 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555f51fe9250 0x555f51fedd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:12.210 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.209+0000 7fd06be00640 1 --2- 172.21.15.72:0/2410924171 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555f51fee3c0 0x555f51ff0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:12.210 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.209+0000 7fd071400640 1 -- 172.21.15.72:0/2410924171 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555f51fe9250 msgr2=0x555f51fedd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:12.211 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.209+0000 7fd071400640 1 --2- 172.21.15.72:0/2410924171 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555f51fe9250 0x555f51fedd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:12.211 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.209+0000 7fd071400640 1 -- 172.21.15.72:0/2410924171 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555f51fee3c0 msgr2=0x555f51ff0850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:12.211 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.209+0000 7fd071400640 1 --2- 172.21.15.72:0/2410924171 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555f51fee3c0 0x555f51ff0850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:12.211 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.210+0000 7fd071400640 1 -- 172.21.15.72:0/2410924171 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f51d277e0 con 0x555f51fe8860 2024-04-26T21:57:12.211 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.210+0000 7fd069600640 1 --2- 172.21.15.72:0/4215050531 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fd0540a3c40 0x7fd0540a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:12.211 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.210+0000 7fd071400640 1 --2- 172.21.15.72:0/2410924171 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x555f51fe8860 0x555f51fe8c80 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fd064008dd0 tx=0x7fd06400d710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=855a734855257f24 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:57:12.213 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.210+0000 7fd069600640 1 -- 172.21.15.72:0/4215050531 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7fd0580d01d0 con 0x7fd06c105d30 2024-04-26T21:57:12.213 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.210+0000 7fd05dc00640 1 -- 172.21.15.72:0/2410924171 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd06400f830 con 0x555f51fe8860 2024-04-26T21:57:12.213 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.210+0000 7fd070a00640 1 --2- 172.21.15.72:0/4215050531 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fd0540a3c40 0x7fd0540a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:12.213 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.210+0000 7fd05dc00640 1 -- 172.21.15.72:0/2410924171 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd06400de20 con 0x555f51fe8860 2024-04-26T21:57:12.213 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.210+0000 7fd05dc00640 1 -- 172.21.15.72:0/2410924171 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd064018d30 con 0x555f51fe8860 2024-04-26T21:57:12.213 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.211+0000 7fd076258740 1 -- 172.21.15.72:0/2410924171 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x555f51fe8860 msgr2=0x555f51fe8c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:12.213 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.211+0000 7fd076258740 1 --2- 172.21.15.72:0/2410924171 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x555f51fe8860 0x555f51fe8c80 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fd064008dd0 tx=0x7fd06400d710 comp rx=0 tx=0).stop 2024-04-26T21:57:12.213 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.211+0000 7fd070a00640 1 --2- 172.21.15.72:0/4215050531 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fd0540a3c40 0x7fd0540a6100 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fd060006010 tx=0x7fd060031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:57:12.213 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.211+0000 7fd076258740 1 -- 172.21.15.72:0/2410924171 shutdown_connections 2024-04-26T21:57:12.214 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.211+0000 7fd076258740 1 --2- 172.21.15.72:0/2410924171 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555f51fee3c0 0x555f51ff0850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:12.214 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.211+0000 7fd076258740 1 --2- 172.21.15.72:0/2410924171 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555f51fe9250 0x555f51fedd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:12.214 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.211+0000 7fd076258740 1 --2- 172.21.15.72:0/2410924171 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x555f51fe8860 0x555f51fe8c80 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:12.214 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.211+0000 7fd076258740 1 -- 172.21.15.72:0/2410924171 >> 172.21.15.72:0/2410924171 conn(0x555f51fe3ed0 msgr2=0x555f51fe42d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:57:12.214 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.212+0000 7fd076258740 1 -- 172.21.15.72:0/2410924171 shutdown_connections 2024-04-26T21:57:12.214 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.212+0000 7fd076258740 1 -- 172.21.15.72:0/2410924171 wait complete. 2024-04-26T21:57:12.214 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.212+0000 7fd076258740 1 -- start start 2024-04-26T21:57:12.214 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.212+0000 7fd076258740 1 Processor -- start 2024-04-26T21:57:12.214 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.213+0000 7fd076258740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x555f51fe8860 0x555f51ff2d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:12.214 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.213+0000 7fd076258740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555f51fe9250 0x555f51ff32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:12.214 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.213+0000 7fd076258740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555f51fee3c0 0x555f51ffa300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:12.214 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.213+0000 7fd076258740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x555f51ff1370 con 0x555f51fe8860 2024-04-26T21:57:12.214 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.213+0000 7fd076258740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x555f51ed5170 con 0x555f51fe9250 2024-04-26T21:57:12.214 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.213+0000 7fd076258740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x555f51ff14f0 con 0x555f51fee3c0 2024-04-26T21:57:12.215 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.213+0000 7fd076258740 10 client.0 _ll_register_callbacks cb 0x555f51f44770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:57:12.215 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.213+0000 7fd070a00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x555f51fe8860 0x555f51ff2d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:12.215 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.213+0000 7fd070a00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x555f51fe8860 0x555f51ff2d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:45592/0 (socket says 172.21.15.72:45592) 2024-04-26T21:57:12.215 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.213+0000 7fd070a00640 1 -- 172.21.15.72:0/1513095168 learned_addr learned my addr 172.21.15.72:0/1513095168 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:57:12.217 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.213+0000 7fd06be00640 1 --2- 172.21.15.72:0/1513095168 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555f51fe9250 0x555f51ff32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:12.217 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.214+0000 7fd071400640 1 --2- 172.21.15.72:0/1513095168 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555f51fee3c0 0x555f51ffa300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:12.217 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.214+0000 7fd070a00640 1 -- 172.21.15.72:0/1513095168 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555f51fee3c0 msgr2=0x555f51ffa300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:12.217 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.214+0000 7fd070a00640 1 --2- 172.21.15.72:0/1513095168 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555f51fee3c0 0x555f51ffa300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:12.217 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.214+0000 7fd070a00640 1 -- 172.21.15.72:0/1513095168 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555f51fe9250 msgr2=0x555f51ff32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:12.217 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.214+0000 7fd070a00640 1 --2- 172.21.15.72:0/1513095168 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555f51fe9250 0x555f51ff32c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:12.217 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.214+0000 7fd070a00640 1 -- 172.21.15.72:0/1513095168 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f51d87f70 con 0x555f51fe8860 2024-04-26T21:57:12.217 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.214+0000 7fd071400640 1 --2- 172.21.15.72:0/1513095168 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555f51fee3c0 0x555f51ffa300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:57:12.217 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.214+0000 7fd070a00640 1 --2- 172.21.15.72:0/1513095168 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x555f51fe8860 0x555f51ff2d80 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fd06002a9e0 tx=0x7fd060009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5f8657315b9e3ad server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:57:12.217 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.214+0000 7fd052000640 10 client.0 ms_handle_connect on v2:172.21.15.72:3300/0 2024-04-26T21:57:12.217 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.215+0000 7fd052000640 1 -- 172.21.15.72:0/1513095168 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd06004d070 con 0x555f51fe8860 2024-04-26T21:57:12.217 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.215+0000 7fd052000640 1 -- 172.21.15.72:0/1513095168 <== mon.0 v2:172.21.15.72:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7fd060031910 con 0x555f51fe8860 2024-04-26T21:57:12.218 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.215+0000 7fd076258740 1 -- 172.21.15.72:0/1513095168 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd06400de20 con 0x555f51fe8860 2024-04-26T21:57:12.218 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.215+0000 7fd052000640 1 -- 172.21.15.72:0/1513095168 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd060059050 con 0x555f51fe8860 2024-04-26T21:57:12.218 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.215+0000 7fd076258740 10 client.15948 fetch_fsmap learned FSMap version 27 2024-04-26T21:57:12.218 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.215+0000 7fd076258740 1 -- 172.21.15.72:0/1513095168 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x555f51c1f170 con 0x555f51fe8860 2024-04-26T21:57:12.218 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.215+0000 7fd052000640 1 -- 172.21.15.72:0/1513095168 <== mon.0 v2:172.21.15.72:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd06400de20 con 0x555f51fe8860 2024-04-26T21:57:12.218 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.216+0000 7fd052000640 1 -- 172.21.15.72:0/1513095168 <== mon.0 v2:172.21.15.72:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fd060049c50 con 0x555f51fe8860 2024-04-26T21:57:12.218 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.216+0000 7fd076258740 10 client.15948 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T21:57:12.218 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.216+0000 7fd076258740 10 client.15948 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:57:12.218 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.216+0000 7fd076258740 20 client.15948 populate_metadata read hostname 'smithi072' 2024-04-26T21:57:12.218 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.216+0000 7fd076258740 1 --2- 172.21.15.72:0/1513095168 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x555f51ffe860 0x555f52000cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:12.218 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.216+0000 7fd076258740 4 client.15948 mds_command: new command op to 34517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:57:12.218 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.216+0000 7fd076258740 1 -- 172.21.15.72:0/1513095168 --> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555f51f5b590 con 0x555f51ffe860 2024-04-26T21:57:12.218 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.217+0000 7fd06be00640 1 --2- 172.21.15.72:0/1513095168 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x555f51ffe860 0x555f52000cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:12.219 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.217+0000 7fd06be00640 1 --2- 172.21.15.72:0/1513095168 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x555f51ffe860 0x555f52000cd0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa81cb9dd4ba88c4 server_cookie=d6c87062277650c7 in_seq=0 out_seq=0 2024-04-26T21:57:12.219 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.217+0000 7fd052000640 10 client.15948 ms_handle_connect on v2:172.21.15.76:6838/327145636 2024-04-26T21:57:12.219 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.219+0000 7fd052000640 1 -- 172.21.15.72:0/1513095168 <== mds.0 v2:172.21.15.76:6838/327145636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555f51f5b590 con 0x555f51ffe860 2024-04-26T21:57:12.219 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.219+0000 7fd052000640 10 client.15948 handle_command_reply: tid=0 2024-04-26T21:57:12.219 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.219+0000 7fd076258740 1 client.15948 shutdown 2024-04-26T21:57:12.220 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.219+0000 7fd076258740 1 -- 172.21.15.72:0/1513095168 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x555f51fe8860 msgr2=0x555f51ff2d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:12.220 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.219+0000 7fd076258740 1 --2- 172.21.15.72:0/1513095168 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x555f51fe8860 0x555f51ff2d80 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fd06002a9e0 tx=0x7fd060009260 comp rx=0 tx=0).stop 2024-04-26T21:57:12.220 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.220+0000 7fd076258740 1 -- 172.21.15.72:0/1513095168 shutdown_connections 2024-04-26T21:57:12.221 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.220+0000 7fd076258740 1 --2- 172.21.15.72:0/1513095168 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x555f51ffe860 0x555f52000cd0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:12.221 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.220+0000 7fd076258740 1 --2- 172.21.15.72:0/1513095168 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555f51fee3c0 0x555f51ffa300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:12.221 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.220+0000 7fd076258740 1 --2- 172.21.15.72:0/1513095168 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555f51fe9250 0x555f51ff32c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:12.221 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.220+0000 7fd076258740 1 --2- 172.21.15.72:0/1513095168 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x555f51fe8860 0x555f51ff2d80 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:12.221 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.220+0000 7fd076258740 1 -- 172.21.15.72:0/1513095168 >> 172.21.15.72:0/1513095168 conn(0x555f51fe3ed0 msgr2=0x555f51fe2ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:57:12.221 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.220+0000 7fd052000640 0 client.15948 ms_handle_reset on v2:172.21.15.76:6838/327145636 2024-04-26T21:57:12.221 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.220+0000 7fd076258740 1 -- 172.21.15.72:0/1513095168 shutdown_connections 2024-04-26T21:57:12.221 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.220+0000 7fd076258740 1 -- 172.21.15.72:0/1513095168 wait complete. 2024-04-26T21:57:12.221 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.220+0000 7fd076258740 20 client.15948 trim_cache size 0 max 16384 2024-04-26T21:57:12.244 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.241+0000 7fd076258740 1 Processor -- start 2024-04-26T21:57:12.244 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.241+0000 7fd076258740 1 -- start start 2024-04-26T21:57:12.244 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.241+0000 7fd076258740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555f51fe8860 0x555f51fe7650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:12.244 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.241+0000 7fd076258740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555f51fe9250 0x555f51ffa2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:12.244 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.241+0000 7fd076258740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x555f51fee3c0 0x555f51ff2990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:12.244 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.241+0000 7fd076258740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fd06c1ac090 con 0x555f51fee3c0 2024-04-26T21:57:12.244 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.241+0000 7fd076258740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fd06c1adbd0 con 0x555f51fe8860 2024-04-26T21:57:12.244 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.241+0000 7fd076258740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd06c1ac760 con 0x555f51fe9250 2024-04-26T21:57:12.244 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.242+0000 7fd071400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x555f51fee3c0 0x555f51ff2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:12.245 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.242+0000 7fd071400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x555f51fee3c0 0x555f51ff2990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:45596/0 (socket says 172.21.15.72:45596) 2024-04-26T21:57:12.245 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.242+0000 7fd071400640 1 -- 172.21.15.72:0/799582698 learned_addr learned my addr 172.21.15.72:0/799582698 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:57:12.245 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.242+0000 7fd070a00640 1 --2- 172.21.15.72:0/799582698 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555f51fe8860 0x555f51fe7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:12.245 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.242+0000 7fd06be00640 1 --2- 172.21.15.72:0/799582698 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555f51fe9250 0x555f51ffa2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:12.245 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.242+0000 7fd071400640 1 -- 172.21.15.72:0/799582698 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555f51fe9250 msgr2=0x555f51ffa2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:12.245 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.242+0000 7fd071400640 1 --2- 172.21.15.72:0/799582698 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555f51fe9250 0x555f51ffa2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:12.245 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.242+0000 7fd071400640 1 -- 172.21.15.72:0/799582698 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555f51fe8860 msgr2=0x555f51fe7650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:12.245 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.242+0000 7fd071400640 1 --2- 172.21.15.72:0/799582698 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555f51fe8860 0x555f51fe7650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:12.245 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.242+0000 7fd071400640 1 -- 172.21.15.72:0/799582698 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f51feae20 con 0x555f51fee3c0 2024-04-26T21:57:12.245 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.243+0000 7fd071400640 1 --2- 172.21.15.72:0/799582698 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x555f51fee3c0 0x555f51ff2990 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fd064008d70 tx=0x7fd06400dfb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d04b6058c8336a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:57:12.245 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.243+0000 7fd05d200640 1 -- 172.21.15.72:0/799582698 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd06400f830 con 0x555f51fee3c0 2024-04-26T21:57:12.245 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.243+0000 7fd05d200640 1 -- 172.21.15.72:0/799582698 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd064028070 con 0x555f51fee3c0 2024-04-26T21:57:12.245 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.243+0000 7fd05d200640 1 -- 172.21.15.72:0/799582698 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd064018ad0 con 0x555f51fee3c0 2024-04-26T21:57:12.245 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.244+0000 7fd076258740 1 -- 172.21.15.72:0/799582698 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x555f51fee3c0 msgr2=0x555f51ff2990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:12.245 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.244+0000 7fd076258740 1 --2- 172.21.15.72:0/799582698 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x555f51fee3c0 0x555f51ff2990 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fd064008d70 tx=0x7fd06400dfb0 comp rx=0 tx=0).stop 2024-04-26T21:57:12.246 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.244+0000 7fd076258740 1 -- 172.21.15.72:0/799582698 shutdown_connections 2024-04-26T21:57:12.246 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.244+0000 7fd076258740 1 --2- 172.21.15.72:0/799582698 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x555f51fee3c0 0x555f51ff2990 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:12.246 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.244+0000 7fd076258740 1 --2- 172.21.15.72:0/799582698 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555f51fe9250 0x555f51ffa2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:12.246 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.244+0000 7fd076258740 1 --2- 172.21.15.72:0/799582698 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555f51fe8860 0x555f51fe7650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:12.246 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.244+0000 7fd076258740 1 -- 172.21.15.72:0/799582698 >> 172.21.15.72:0/799582698 conn(0x555f51fe3ed0 msgr2=0x555f51fe6f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:57:12.246 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.244+0000 7fd076258740 1 -- 172.21.15.72:0/799582698 shutdown_connections 2024-04-26T21:57:12.246 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.244+0000 7fd076258740 1 -- 172.21.15.72:0/799582698 wait complete. 2024-04-26T21:57:12.249 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.245+0000 7fd076258740 1 -- start start 2024-04-26T21:57:12.249 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.245+0000 7fd076258740 1 Processor -- start 2024-04-26T21:57:12.249 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.245+0000 7fd076258740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555f51fe8860 0x555f51ff7960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:12.249 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.245+0000 7fd076258740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x555f51fe9250 0x555f51ff7ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:12.249 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.245+0000 7fd076258740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555f51fee3c0 0x555f51ff8eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:12.249 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.245+0000 7fd076258740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fd06c1ad500 con 0x555f51fe9250 2024-04-26T21:57:12.249 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.245+0000 7fd076258740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fd06c1ace30 con 0x555f51fee3c0 2024-04-26T21:57:12.249 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.245+0000 7fd076258740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd06c1aa070 con 0x555f51fe8860 2024-04-26T21:57:12.249 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.245+0000 7fd076258740 10 client.0 _ll_register_callbacks cb 0x7fd060009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:57:12.249 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.246+0000 7fd06be00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x555f51fe9250 0x555f51ff7ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:12.250 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.246+0000 7fd06be00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x555f51fe9250 0x555f51ff7ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:45606/0 (socket says 172.21.15.72:45606) 2024-04-26T21:57:12.250 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.246+0000 7fd06be00640 1 -- 172.21.15.72:0/1358612642 learned_addr learned my addr 172.21.15.72:0/1358612642 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:57:12.250 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.246+0000 7fd071400640 1 --2- 172.21.15.72:0/1358612642 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555f51fee3c0 0x555f51ff8eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:12.250 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.246+0000 7fd070a00640 1 --2- 172.21.15.72:0/1358612642 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555f51fe8860 0x555f51ff7960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:12.250 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.246+0000 7fd06be00640 1 -- 172.21.15.72:0/1358612642 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555f51fe8860 msgr2=0x555f51ff7960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:12.250 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.246+0000 7fd06be00640 1 --2- 172.21.15.72:0/1358612642 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555f51fe8860 0x555f51ff7960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:12.250 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.246+0000 7fd06be00640 1 -- 172.21.15.72:0/1358612642 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555f51fee3c0 msgr2=0x555f51ff8eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:12.250 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.246+0000 7fd06be00640 1 --2- 172.21.15.72:0/1358612642 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555f51fee3c0 0x555f51ff8eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:12.250 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.246+0000 7fd06be00640 1 -- 172.21.15.72:0/1358612642 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f52001040 con 0x555f51fe9250 2024-04-26T21:57:12.250 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.246+0000 7fd071400640 1 --2- 172.21.15.72:0/1358612642 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555f51fee3c0 0x555f51ff8eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:57:12.250 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.246+0000 7fd06be00640 1 --2- 172.21.15.72:0/1358612642 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x555f51fe9250 0x555f51ff7ea0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fd05809c260 tx=0x7fd058010f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db7f218380e96c26 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:57:12.250 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.246+0000 7fd052000640 10 client.0 ms_handle_connect on v2:172.21.15.72:3300/0 2024-04-26T21:57:12.250 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.247+0000 7fd052000640 1 -- 172.21.15.72:0/1358612642 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd058021040 con 0x555f51fe9250 2024-04-26T21:57:12.251 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.247+0000 7fd052000640 1 -- 172.21.15.72:0/1358612642 <== mon.0 v2:172.21.15.72:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7fd0580cdd50 con 0x555f51fe9250 2024-04-26T21:57:12.251 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.247+0000 7fd076258740 1 -- 172.21.15.72:0/1358612642 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd064028070 con 0x555f51fe9250 2024-04-26T21:57:12.251 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.247+0000 7fd052000640 1 -- 172.21.15.72:0/1358612642 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd05809c2d0 con 0x555f51fe9250 2024-04-26T21:57:12.251 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.247+0000 7fd076258740 10 client.15954 fetch_fsmap learned FSMap version 27 2024-04-26T21:57:12.251 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.247+0000 7fd076258740 1 -- 172.21.15.72:0/1358612642 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x555f52064e10 con 0x555f51fe9250 2024-04-26T21:57:12.251 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.247+0000 7fd052000640 1 -- 172.21.15.72:0/1358612642 <== mon.0 v2:172.21.15.72:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd064028070 con 0x555f51fe9250 2024-04-26T21:57:12.251 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.247+0000 7fd052000640 1 -- 172.21.15.72:0/1358612642 <== mon.0 v2:172.21.15.72:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fd05809b720 con 0x555f51fe9250 2024-04-26T21:57:12.251 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.247+0000 7fd076258740 10 client.15954 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T21:57:12.251 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.247+0000 7fd076258740 10 client.15954 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:57:12.251 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.247+0000 7fd076258740 20 client.15954 populate_metadata read hostname 'smithi072' 2024-04-26T21:57:12.251 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.247+0000 7fd076258740 1 --2- 172.21.15.72:0/1358612642 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x555f520656c0 0x555f52067ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:12.251 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.247+0000 7fd076258740 4 client.15954 mds_command: new command op to 34517 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-26T21:57:12.251 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.247+0000 7fd076258740 1 -- 172.21.15.72:0/1358612642 --> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x555f51f55400 con 0x555f520656c0 2024-04-26T21:57:12.251 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.248+0000 7fd070a00640 1 --2- 172.21.15.72:0/1358612642 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x555f520656c0 0x555f52067ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:12.251 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.249+0000 7fd052000640 10 client.15954 ms_handle_connect on v2:172.21.15.76:6838/327145636 2024-04-26T21:57:12.252 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.249+0000 7fd070a00640 1 --2- 172.21.15.72:0/1358612642 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x555f520656c0 0x555f52067ae0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b9336b33d8bb6431 server_cookie=c7b3da9e42df6901 in_seq=0 out_seq=0 2024-04-26T21:57:12.253 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.252+0000 7fd052000640 1 -- 172.21.15.72:0/1358612642 <== mds.0 v2:172.21.15.76:6838/327145636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x555f51f55400 con 0x555f520656c0 2024-04-26T21:57:12.253 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.252+0000 7fd052000640 10 client.15954 handle_command_reply: tid=0 2024-04-26T21:57:12.253 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.252+0000 7fd076258740 1 client.15954 shutdown 2024-04-26T21:57:12.253 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.252+0000 7fd076258740 1 -- 172.21.15.72:0/1358612642 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x555f51fe9250 msgr2=0x555f51ff7ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:12.253 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.252+0000 7fd076258740 1 --2- 172.21.15.72:0/1358612642 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x555f51fe9250 0x555f51ff7ea0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fd05809c260 tx=0x7fd058010f00 comp rx=0 tx=0).stop 2024-04-26T21:57:12.253 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.252+0000 7fd076258740 1 -- 172.21.15.72:0/1358612642 shutdown_connections 2024-04-26T21:57:12.253 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.252+0000 7fd076258740 1 --2- 172.21.15.72:0/1358612642 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x555f520656c0 0x555f52067ae0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:12.253 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.252+0000 7fd076258740 1 --2- 172.21.15.72:0/1358612642 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555f51fee3c0 0x555f51ff8eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:12.253 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.252+0000 7fd076258740 1 --2- 172.21.15.72:0/1358612642 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x555f51fe9250 0x555f51ff7ea0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:12.254 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.252+0000 7fd052000640 0 client.15954 ms_handle_reset on v2:172.21.15.76:6838/327145636 2024-04-26T21:57:12.254 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.252+0000 7fd076258740 1 --2- 172.21.15.72:0/1358612642 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555f51fe8860 0x555f51ff7960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:12.254 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.252+0000 7fd076258740 1 -- 172.21.15.72:0/1358612642 >> 172.21.15.72:0/1358612642 conn(0x555f51fe3ed0 msgr2=0x555f51ff2920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:57:12.254 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.252+0000 7fd076258740 1 -- 172.21.15.72:0/1358612642 shutdown_connections 2024-04-26T21:57:12.254 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.252+0000 7fd076258740 1 -- 172.21.15.72:0/1358612642 wait complete. 2024-04-26T21:57:12.254 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.252+0000 7fd076258740 20 client.15954 trim_cache size 0 max 16384 2024-04-26T21:57:12.254 INFO:teuthology.orchestra.run.smithi072.stdout:{ 2024-04-26T21:57:12.254 INFO:teuthology.orchestra.run.smithi072.stdout: "return_code": 0, 2024-04-26T21:57:12.254 INFO:teuthology.orchestra.run.smithi072.stdout: "scrub_tag": "224864bf-2b78-48ff-8722-ede585665410", 2024-04-26T21:57:12.255 INFO:teuthology.orchestra.run.smithi072.stdout: "mode": "asynchronous" 2024-04-26T21:57:12.255 INFO:teuthology.orchestra.run.smithi072.stdout:} 2024-04-26T21:57:12.255 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.253+0000 7fd073400640 1 -- 172.21.15.72:0/4215050531 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fd0540a3c40 msgr2=0x7fd0540a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:12.255 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.253+0000 7fd073400640 1 --2- 172.21.15.72:0/4215050531 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fd0540a3c40 0x7fd0540a6100 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fd060006010 tx=0x7fd060031400 comp rx=0 tx=0).stop 2024-04-26T21:57:12.255 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.253+0000 7fd073400640 1 -- 172.21.15.72:0/4215050531 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd06c105d30 msgr2=0x7fd06c19b2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:12.256 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.253+0000 7fd073400640 1 --2- 172.21.15.72:0/4215050531 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd06c105d30 0x7fd06c19b2a0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fd05800d770 tx=0x7fd05800dc30 comp rx=0 tx=0).stop 2024-04-26T21:57:12.256 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.253+0000 7fd073400640 1 -- 172.21.15.72:0/4215050531 shutdown_connections 2024-04-26T21:57:12.256 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.253+0000 7fd073400640 1 --2- 172.21.15.72:0/4215050531 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fd0540a3c40 0x7fd0540a6100 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:12.256 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.253+0000 7fd073400640 1 --2- 172.21.15.72:0/4215050531 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd06c108660 0x7fd06c1a2310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:12.256 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.253+0000 7fd073400640 1 --2- 172.21.15.72:0/4215050531 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd06c105d30 0x7fd06c19b2a0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:12.256 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.253+0000 7fd073400640 1 --2- 172.21.15.72:0/4215050531 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd06c103400 0x7fd06c19ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:12.256 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.253+0000 7fd073400640 1 -- 172.21.15.72:0/4215050531 >> 172.21.15.72:0/4215050531 conn(0x7fd06c0fcdf0 msgr2=0x7fd06c102ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:57:12.256 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.253+0000 7fd073400640 1 -- 172.21.15.72:0/4215050531 shutdown_connections 2024-04-26T21:57:12.256 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.253+0000 7fd073400640 1 -- 172.21.15.72:0/4215050531 wait complete. 2024-04-26T21:57:12.274 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-26T21:57:12.274 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-26T21:57:12.471 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.471+0000 7f2ee1000640 1 -- 172.21.15.72:0/4053598011 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2edc104410 msgr2=0x7f2edc10aca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:12.471 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.471+0000 7f2ee1000640 1 --2- 172.21.15.72:0/4053598011 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2edc104410 0x7f2edc10aca0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f2ed4009ca0 tx=0x7f2ed402a8c0 comp rx=0 tx=0).stop 2024-04-26T21:57:12.471 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.471+0000 7f2ee1000640 1 -- 172.21.15.72:0/4053598011 shutdown_connections 2024-04-26T21:57:12.472 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.471+0000 7f2ee1000640 1 --2- 172.21.15.72:0/4053598011 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2edc104410 0x7f2edc10aca0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:12.472 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.471+0000 7f2ee1000640 1 --2- 172.21.15.72:0/4053598011 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2edc103a50 0x7f2edc103ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:12.472 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.471+0000 7f2ee1000640 1 --2- 172.21.15.72:0/4053598011 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2edc102860 0x7f2edc102c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:12.472 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.471+0000 7f2ee1000640 1 -- 172.21.15.72:0/4053598011 >> 172.21.15.72:0/4053598011 conn(0x7f2edc0fddd0 msgr2=0x7f2edc100210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:57:12.472 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.471+0000 7f2ee1000640 1 -- 172.21.15.72:0/4053598011 shutdown_connections 2024-04-26T21:57:12.472 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.471+0000 7f2ee1000640 1 -- 172.21.15.72:0/4053598011 wait complete. 2024-04-26T21:57:12.473 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.472+0000 7f2ee1000640 1 Processor -- start 2024-04-26T21:57:12.473 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.472+0000 7f2ee1000640 1 -- start start 2024-04-26T21:57:12.474 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.472+0000 7f2ee1000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2edc102860 0x7f2edc198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:12.474 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.472+0000 7f2ee1000640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2edc103a50 0x7f2edc198e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:12.474 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.473+0000 7f2ee1000640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2edc104410 0x7f2edc19fed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:12.474 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.473+0000 7f2ee1000640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f2edc10e270 con 0x7f2edc104410 2024-04-26T21:57:12.474 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.473+0000 7f2ee1000640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f2edc10e0f0 con 0x7f2edc103a50 2024-04-26T21:57:12.474 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.473+0000 7f2ee1000640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f2edc10e3f0 con 0x7f2edc102860 2024-04-26T21:57:12.474 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.473+0000 7f2edbe00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2edc104410 0x7f2edc19fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:12.474 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.473+0000 7f2edbe00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2edc104410 0x7f2edc19fed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:45634/0 (socket says 172.21.15.72:45634) 2024-04-26T21:57:12.474 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.473+0000 7f2edbe00640 1 -- 172.21.15.72:0/2278728592 learned_addr learned my addr 172.21.15.72:0/2278728592 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:57:12.474 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.473+0000 7f2edb400640 1 --2- 172.21.15.72:0/2278728592 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2edc102860 0x7f2edc198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:12.474 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.473+0000 7f2edaa00640 1 --2- 172.21.15.72:0/2278728592 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2edc103a50 0x7f2edc198e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:12.474 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.474+0000 7f2edbe00640 1 -- 172.21.15.72:0/2278728592 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2edc102860 msgr2=0x7f2edc198920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:12.474 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.474+0000 7f2edbe00640 1 --2- 172.21.15.72:0/2278728592 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2edc102860 0x7f2edc198920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:12.475 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.474+0000 7f2edbe00640 1 -- 172.21.15.72:0/2278728592 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2edc103a50 msgr2=0x7f2edc198e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:12.475 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.474+0000 7f2edbe00640 1 --2- 172.21.15.72:0/2278728592 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2edc103a50 0x7f2edc198e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:12.475 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.474+0000 7f2edbe00640 1 -- 172.21.15.72:0/2278728592 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2edc1a0540 con 0x7f2edc104410 2024-04-26T21:57:12.475 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.474+0000 7f2edbe00640 1 --2- 172.21.15.72:0/2278728592 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2edc104410 0x7f2edc19fed0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f2ed4031040 tx=0x7f2ed4003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:57:12.476 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.474+0000 7f2ed3e00640 1 -- 172.21.15.72:0/2278728592 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ed4004040 con 0x7f2edc104410 2024-04-26T21:57:12.476 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.474+0000 7f2ed3e00640 1 -- 172.21.15.72:0/2278728592 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2ed4004670 con 0x7f2edc104410 2024-04-26T21:57:12.476 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.474+0000 7f2ee1000640 1 -- 172.21.15.72:0/2278728592 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2edc1a07d0 con 0x7f2edc104410 2024-04-26T21:57:12.476 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.474+0000 7f2ed3e00640 1 -- 172.21.15.72:0/2278728592 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ed4030d60 con 0x7f2edc104410 2024-04-26T21:57:12.476 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.475+0000 7f2ee1000640 1 -- 172.21.15.72:0/2278728592 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2edc1a0d20 con 0x7f2edc104410 2024-04-26T21:57:12.478 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.479+0000 7f2ee558d740 1 Processor -- start 2024-04-26T21:57:12.479 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.479+0000 7f2ed3e00640 1 -- 172.21.15.72:0/2278728592 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f2ed40305c0 con 0x7f2edc104410 2024-04-26T21:57:12.479 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.479+0000 7f2ee558d740 1 -- start start 2024-04-26T21:57:12.480 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.480+0000 7f2ee558d740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c3827a68e0 0x55c3827a6d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:12.480 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.480+0000 7f2ee558d740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c3827a72d0 0x55c3827a9dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:12.480 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.480+0000 7f2ee558d740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c3827aa430 0x55c3827b72a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:12.480 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.480+0000 7f2ee558d740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55c3826b7030 con 0x55c3827aa430 2024-04-26T21:57:12.481 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.480+0000 7f2ee558d740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55c3826d27f0 con 0x55c3827a68e0 2024-04-26T21:57:12.481 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.480+0000 7f2ee558d740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55c38268d7e0 con 0x55c3827a72d0 2024-04-26T21:57:12.481 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.480+0000 7f2edaa00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c3827aa430 0x55c3827b72a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:12.481 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.480+0000 7f2edaa00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c3827aa430 0x55c3827b72a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:45650/0 (socket says 172.21.15.72:45650) 2024-04-26T21:57:12.481 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.480+0000 7f2edaa00640 1 -- 172.21.15.72:0/4293205589 learned_addr learned my addr 172.21.15.72:0/4293205589 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:57:12.481 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.480+0000 7f2edb400640 1 --2- 172.21.15.72:0/4293205589 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c3827a72d0 0x55c3827a9dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:12.481 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.480+0000 7f2edaa00640 1 --2- 172.21.15.72:0/4293205589 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c3827a68e0 0x55c3827a6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:12.483 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.481+0000 7f2edaa00640 1 -- 172.21.15.72:0/4293205589 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c3827a72d0 msgr2=0x55c3827a9dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:12.483 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.481+0000 7f2edaa00640 1 --2- 172.21.15.72:0/4293205589 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c3827a72d0 0x55c3827a9dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:12.483 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.481+0000 7f2edaa00640 1 -- 172.21.15.72:0/4293205589 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c3827a68e0 msgr2=0x55c3827a6d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:12.483 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.481+0000 7f2edaa00640 1 --2- 172.21.15.72:0/4293205589 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c3827a68e0 0x55c3827a6d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:12.483 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.481+0000 7f2edaa00640 1 -- 172.21.15.72:0/4293205589 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c382699050 con 0x55c3827aa430 2024-04-26T21:57:12.483 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.481+0000 7f2ed3e00640 1 --2- 172.21.15.72:0/2278728592 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f2ec00a3c40 0x7f2ec00a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:12.483 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.481+0000 7f2edaa00640 1 --2- 172.21.15.72:0/4293205589 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c3827aa430 0x55c3827b72a0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f2ec800bfd0 tx=0x7f2ec800b650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f03751093e78d023 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:57:12.483 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.481+0000 7f2ed3e00640 1 -- 172.21.15.72:0/2278728592 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7f2ed40f1b50 con 0x7f2edc104410 2024-04-26T21:57:12.483 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.481+0000 7f2edb400640 1 --2- 172.21.15.72:0/2278728592 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f2ec00a3c40 0x7f2ec00a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:12.483 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.482+0000 7f2ed0c00640 1 -- 172.21.15.72:0/4293205589 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ec8012d00 con 0x55c3827aa430 2024-04-26T21:57:12.483 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.482+0000 7f2ed0c00640 1 -- 172.21.15.72:0/4293205589 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2ec8010740 con 0x55c3827aa430 2024-04-26T21:57:12.483 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.482+0000 7f2ed0c00640 1 -- 172.21.15.72:0/4293205589 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ec80037f0 con 0x55c3827aa430 2024-04-26T21:57:12.484 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.483+0000 7f2ee558d740 1 -- 172.21.15.72:0/4293205589 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c3827aa430 msgr2=0x55c3827b72a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:12.484 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.483+0000 7f2ee558d740 1 --2- 172.21.15.72:0/4293205589 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c3827aa430 0x55c3827b72a0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f2ec800bfd0 tx=0x7f2ec800b650 comp rx=0 tx=0).stop 2024-04-26T21:57:12.484 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.483+0000 7f2edb400640 1 --2- 172.21.15.72:0/2278728592 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f2ec00a3c40 0x7f2ec00a6100 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x55c3827a76d0 tx=0x7f2ecc00b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:57:12.484 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.483+0000 7f2ee558d740 1 -- 172.21.15.72:0/4293205589 shutdown_connections 2024-04-26T21:57:12.484 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.483+0000 7f2ee558d740 1 --2- 172.21.15.72:0/4293205589 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c3827aa430 0x55c3827b72a0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:12.484 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.483+0000 7f2ee558d740 1 --2- 172.21.15.72:0/4293205589 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c3827a72d0 0x55c3827a9dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:12.484 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.483+0000 7f2ee558d740 1 --2- 172.21.15.72:0/4293205589 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c3827a68e0 0x55c3827a6d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:12.484 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.483+0000 7f2ee558d740 1 -- 172.21.15.72:0/4293205589 >> 172.21.15.72:0/4293205589 conn(0x55c3827a1fd0 msgr2=0x55c3827a23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:57:12.484 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.483+0000 7f2ee558d740 1 -- 172.21.15.72:0/4293205589 shutdown_connections 2024-04-26T21:57:12.484 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.483+0000 7f2ee558d740 1 -- 172.21.15.72:0/4293205589 wait complete. 2024-04-26T21:57:12.487 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.483+0000 7f2ee558d740 1 -- start start 2024-04-26T21:57:12.487 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.484+0000 7f2ee558d740 1 Processor -- start 2024-04-26T21:57:12.487 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.484+0000 7f2ee558d740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c3827a68e0 0x55c3827b9080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:12.487 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.484+0000 7f2ee558d740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c3827a72d0 0x55c3827b95c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:12.487 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.484+0000 7f2ee558d740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c3827aa430 0x55c3827c0600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:12.487 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.484+0000 7f2ee558d740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55c38270f200 con 0x55c3827aa430 2024-04-26T21:57:12.488 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.484+0000 7f2ee558d740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55c382709ee0 con 0x55c3827a68e0 2024-04-26T21:57:12.488 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.484+0000 7f2ee558d740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55c382716e70 con 0x55c3827a72d0 2024-04-26T21:57:12.488 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.484+0000 7f2ee558d740 10 client.0 _ll_register_callbacks cb 0x55c382700270 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:57:12.488 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.484+0000 7f2edbe00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c3827aa430 0x55c3827c0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:12.488 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.484+0000 7f2edbe00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c3827aa430 0x55c3827c0600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:45664/0 (socket says 172.21.15.72:45664) 2024-04-26T21:57:12.488 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.484+0000 7f2edbe00640 1 -- 172.21.15.72:0/126801249 learned_addr learned my addr 172.21.15.72:0/126801249 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:57:12.488 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.485+0000 7f2edb400640 1 --2- 172.21.15.72:0/126801249 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c3827a68e0 0x55c3827b9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:12.488 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.485+0000 7f2edaa00640 1 --2- 172.21.15.72:0/126801249 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c3827a72d0 0x55c3827b95c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:12.488 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.485+0000 7f2edbe00640 1 -- 172.21.15.72:0/126801249 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c3827a72d0 msgr2=0x55c3827b95c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:12.488 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.485+0000 7f2edbe00640 1 --2- 172.21.15.72:0/126801249 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c3827a72d0 0x55c3827b95c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:12.488 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.485+0000 7f2edbe00640 1 -- 172.21.15.72:0/126801249 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c3827a68e0 msgr2=0x55c3827b9080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:12.488 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.485+0000 7f2edbe00640 1 --2- 172.21.15.72:0/126801249 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c3827a68e0 0x55c3827b9080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:12.488 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.485+0000 7f2edbe00640 1 -- 172.21.15.72:0/126801249 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c3824e67d0 con 0x55c3827aa430 2024-04-26T21:57:12.488 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.485+0000 7f2edb400640 1 --2- 172.21.15.72:0/126801249 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c3827a68e0 0x55c3827b9080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:57:12.489 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.485+0000 7f2edaa00640 1 --2- 172.21.15.72:0/126801249 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c3827a72d0 0x55c3827b95c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:57:12.489 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.485+0000 7f2edbe00640 1 --2- 172.21.15.72:0/126801249 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c3827aa430 0x55c3827c0600 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f2ed40c64a0 tx=0x7f2ed4004820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d53ad0c43bd85183 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:57:12.489 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.485+0000 7f2ec5600640 10 client.0 ms_handle_connect on v2:172.21.15.72:3300/0 2024-04-26T21:57:12.489 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.485+0000 7f2ec5600640 1 -- 172.21.15.72:0/126801249 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ed40d7060 con 0x55c3827aa430 2024-04-26T21:57:12.489 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.485+0000 7f2ec5600640 1 -- 172.21.15.72:0/126801249 <== mon.0 v2:172.21.15.72:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7f2ed40ba3d0 con 0x55c3827aa430 2024-04-26T21:57:12.489 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.485+0000 7f2ee558d740 1 -- 172.21.15.72:0/126801249 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2ec8010740 con 0x55c3827aa430 2024-04-26T21:57:12.489 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.486+0000 7f2ec5600640 1 -- 172.21.15.72:0/126801249 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2ed40bdd60 con 0x55c3827aa430 2024-04-26T21:57:12.489 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.486+0000 7f2ee558d740 10 client.15966 fetch_fsmap learned FSMap version 27 2024-04-26T21:57:12.489 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.486+0000 7f2ee558d740 1 -- 172.21.15.72:0/126801249 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c38241c3f0 con 0x55c3827aa430 2024-04-26T21:57:12.489 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.486+0000 7f2ec5600640 1 -- 172.21.15.72:0/126801249 <== mon.0 v2:172.21.15.72:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2ec8010740 con 0x55c3827aa430 2024-04-26T21:57:12.489 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.486+0000 7f2ec5600640 1 -- 172.21.15.72:0/126801249 <== mon.0 v2:172.21.15.72:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f2ed40a8040 con 0x55c3827aa430 2024-04-26T21:57:12.489 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.486+0000 7f2ee558d740 10 client.15966 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T21:57:12.489 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.486+0000 7f2ee558d740 10 client.15966 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:57:12.489 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.486+0000 7f2ee558d740 20 client.15966 populate_metadata read hostname 'smithi072' 2024-04-26T21:57:12.489 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.486+0000 7f2ee558d740 1 --2- 172.21.15.72:0/126801249 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x55c3827c4af0 0x55c3827c6f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:12.490 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.486+0000 7f2ee558d740 4 client.15966 mds_command: new command op to 34517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:57:12.490 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.486+0000 7f2ee558d740 1 -- 172.21.15.72:0/126801249 --> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c3827187e0 con 0x55c3827c4af0 2024-04-26T21:57:12.490 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.487+0000 7f2edb400640 1 --2- 172.21.15.72:0/126801249 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x55c3827c4af0 0x55c3827c6f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:12.491 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.488+0000 7f2edb400640 1 --2- 172.21.15.72:0/126801249 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x55c3827c4af0 0x55c3827c6f70 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb5d82aee2eb1726 server_cookie=91ad36fabf36b687 in_seq=0 out_seq=0 2024-04-26T21:57:12.491 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.488+0000 7f2ec5600640 10 client.15966 ms_handle_connect on v2:172.21.15.76:6838/327145636 2024-04-26T21:57:12.491 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.490+0000 7f2ec5600640 1 -- 172.21.15.72:0/126801249 <== mds.0 v2:172.21.15.76:6838/327145636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c3827187e0 con 0x55c3827c4af0 2024-04-26T21:57:12.491 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.490+0000 7f2ec5600640 10 client.15966 handle_command_reply: tid=0 2024-04-26T21:57:12.491 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.490+0000 7f2ee558d740 1 client.15966 shutdown 2024-04-26T21:57:12.491 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.490+0000 7f2ee558d740 1 -- 172.21.15.72:0/126801249 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c3827aa430 msgr2=0x55c3827c0600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:12.491 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.490+0000 7f2ee558d740 1 --2- 172.21.15.72:0/126801249 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c3827aa430 0x55c3827c0600 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f2ed40c64a0 tx=0x7f2ed4004820 comp rx=0 tx=0).stop 2024-04-26T21:57:12.491 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.490+0000 7f2ee558d740 1 -- 172.21.15.72:0/126801249 shutdown_connections 2024-04-26T21:57:12.491 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.490+0000 7f2ee558d740 1 --2- 172.21.15.72:0/126801249 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x55c3827c4af0 0x55c3827c6f70 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:12.491 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.490+0000 7f2ee558d740 1 --2- 172.21.15.72:0/126801249 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c3827aa430 0x55c3827c0600 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:12.492 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.490+0000 7f2ee558d740 1 --2- 172.21.15.72:0/126801249 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c3827a72d0 0x55c3827b95c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:12.492 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.490+0000 7f2ee558d740 1 --2- 172.21.15.72:0/126801249 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c3827a68e0 0x55c3827b9080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:12.492 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.490+0000 7f2ec5600640 0 client.15966 ms_handle_reset on v2:172.21.15.76:6838/327145636 2024-04-26T21:57:12.492 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.490+0000 7f2ee558d740 1 -- 172.21.15.72:0/126801249 >> 172.21.15.72:0/126801249 conn(0x55c3827a1fd0 msgr2=0x55c3827a11f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:57:12.492 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.490+0000 7f2ee558d740 1 -- 172.21.15.72:0/126801249 shutdown_connections 2024-04-26T21:57:12.492 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.490+0000 7f2ee558d740 1 -- 172.21.15.72:0/126801249 wait complete. 2024-04-26T21:57:12.492 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.490+0000 7f2ee558d740 20 client.15966 trim_cache size 0 max 16384 2024-04-26T21:57:12.509 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.509+0000 7f2ee558d740 1 Processor -- start 2024-04-26T21:57:12.509 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.510+0000 7f2ee558d740 1 -- start start 2024-04-26T21:57:12.511 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.510+0000 7f2ee558d740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c3827a68e0 0x55c3827a5770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:12.511 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.510+0000 7f2ee558d740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c3827a72d0 0x55c3827c05c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:12.511 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.510+0000 7f2ee558d740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c3827aa430 0x55c3827b3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:12.511 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.510+0000 7f2ee558d740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f2edc1a9470 con 0x55c3827aa430 2024-04-26T21:57:12.511 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.510+0000 7f2ee558d740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f2edc1a8000 con 0x55c3827a68e0 2024-04-26T21:57:12.511 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.510+0000 7f2ee558d740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f2edc1a86d0 con 0x55c3827a72d0 2024-04-26T21:57:12.511 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.510+0000 7f2edbe00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c3827aa430 0x55c3827b3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:12.511 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.510+0000 7f2edbe00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c3827aa430 0x55c3827b3280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:45674/0 (socket says 172.21.15.72:45674) 2024-04-26T21:57:12.511 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.510+0000 7f2edbe00640 1 -- 172.21.15.72:0/1824872148 learned_addr learned my addr 172.21.15.72:0/1824872148 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:57:12.511 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.511+0000 7f2edbe00640 1 -- 172.21.15.72:0/1824872148 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c3827a72d0 msgr2=0x55c3827c05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:12.511 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.511+0000 7f2edbe00640 1 --2- 172.21.15.72:0/1824872148 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c3827a72d0 0x55c3827c05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:12.511 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.511+0000 7f2edbe00640 1 -- 172.21.15.72:0/1824872148 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c3827a68e0 msgr2=0x55c3827a5770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:12.512 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.511+0000 7f2edb400640 1 --2- 172.21.15.72:0/1824872148 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c3827a68e0 0x55c3827a5770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:12.512 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.511+0000 7f2edbe00640 1 --2- 172.21.15.72:0/1824872148 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c3827a68e0 0x55c3827a5770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:12.512 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.511+0000 7f2edbe00640 1 -- 172.21.15.72:0/1824872148 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c3823dc0f0 con 0x55c3827aa430 2024-04-26T21:57:12.512 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.511+0000 7f2edb400640 1 --2- 172.21.15.72:0/1824872148 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c3827a68e0 0x55c3827a5770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:57:12.512 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.511+0000 7f2edbe00640 1 --2- 172.21.15.72:0/1824872148 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c3827aa430 0x55c3827b3280 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f2ed4005e30 tx=0x7f2ed40bd8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=927fd4f1c2ef0e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:57:12.515 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.511+0000 7f2ec7e00640 1 -- 172.21.15.72:0/1824872148 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ed40c2110 con 0x55c3827aa430 2024-04-26T21:57:12.516 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.511+0000 7f2ec7e00640 1 -- 172.21.15.72:0/1824872148 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2ed4036020 con 0x55c3827aa430 2024-04-26T21:57:12.516 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.511+0000 7f2ec7e00640 1 -- 172.21.15.72:0/1824872148 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ed40ba560 con 0x55c3827aa430 2024-04-26T21:57:12.516 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.512+0000 7f2ee558d740 1 -- 172.21.15.72:0/1824872148 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c3827aa430 msgr2=0x55c3827b3280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:12.516 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.512+0000 7f2ee558d740 1 --2- 172.21.15.72:0/1824872148 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c3827aa430 0x55c3827b3280 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f2ed4005e30 tx=0x7f2ed40bd8e0 comp rx=0 tx=0).stop 2024-04-26T21:57:12.516 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.512+0000 7f2ee558d740 1 -- 172.21.15.72:0/1824872148 shutdown_connections 2024-04-26T21:57:12.516 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.512+0000 7f2ee558d740 1 --2- 172.21.15.72:0/1824872148 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c3827aa430 0x55c3827b3280 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:12.516 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.512+0000 7f2ee558d740 1 --2- 172.21.15.72:0/1824872148 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c3827a72d0 0x55c3827c05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:12.516 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.512+0000 7f2ee558d740 1 --2- 172.21.15.72:0/1824872148 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c3827a68e0 0x55c3827a5770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:12.516 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.512+0000 7f2ee558d740 1 -- 172.21.15.72:0/1824872148 >> 172.21.15.72:0/1824872148 conn(0x55c3827a1fd0 msgr2=0x55c3827a5060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:57:12.516 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.512+0000 7f2ee558d740 1 -- 172.21.15.72:0/1824872148 shutdown_connections 2024-04-26T21:57:12.516 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.512+0000 7f2ee558d740 1 -- 172.21.15.72:0/1824872148 wait complete. 2024-04-26T21:57:12.516 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.513+0000 7f2ee558d740 1 -- start start 2024-04-26T21:57:12.516 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.513+0000 7f2ee558d740 1 Processor -- start 2024-04-26T21:57:12.517 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.513+0000 7f2ee558d740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c3827a68e0 0x55c3827bd8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:12.517 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.513+0000 7f2ee558d740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c3827a72d0 0x55c3827bdde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:12.517 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.513+0000 7f2ee558d740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c3827aa430 0x55c3827bed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:12.517 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.513+0000 7f2ee558d740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f2edc1a5910 con 0x55c3827aa430 2024-04-26T21:57:12.517 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.513+0000 7f2ee558d740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f2edc1a8da0 con 0x55c3827a68e0 2024-04-26T21:57:12.517 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.514+0000 7f2ee558d740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f2edc1a5f80 con 0x55c3827a72d0 2024-04-26T21:57:12.517 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.514+0000 7f2ee558d740 10 client.0 _ll_register_callbacks cb 0x7f2ed4005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:57:12.517 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.514+0000 7f2edbe00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c3827aa430 0x55c3827bed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:12.517 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.514+0000 7f2edbe00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c3827aa430 0x55c3827bed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:45682/0 (socket says 172.21.15.72:45682) 2024-04-26T21:57:12.517 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.514+0000 7f2edbe00640 1 -- 172.21.15.72:0/3655083912 learned_addr learned my addr 172.21.15.72:0/3655083912 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:57:12.517 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.514+0000 7f2edb400640 1 --2- 172.21.15.72:0/3655083912 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c3827a68e0 0x55c3827bd8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:12.517 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.514+0000 7f2edaa00640 1 --2- 172.21.15.72:0/3655083912 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c3827a72d0 0x55c3827bdde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:12.517 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.514+0000 7f2edbe00640 1 -- 172.21.15.72:0/3655083912 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c3827a72d0 msgr2=0x55c3827bdde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:12.517 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.514+0000 7f2edbe00640 1 --2- 172.21.15.72:0/3655083912 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c3827a72d0 0x55c3827bdde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:12.517 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.514+0000 7f2edbe00640 1 -- 172.21.15.72:0/3655083912 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c3827a68e0 msgr2=0x55c3827bd8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:12.518 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.514+0000 7f2edbe00640 1 --2- 172.21.15.72:0/3655083912 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c3827a68e0 0x55c3827bd8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:12.518 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.514+0000 7f2edbe00640 1 -- 172.21.15.72:0/3655083912 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c3826d2a30 con 0x55c3827aa430 2024-04-26T21:57:12.518 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.514+0000 7f2edaa00640 1 --2- 172.21.15.72:0/3655083912 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c3827a72d0 0x55c3827bdde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:57:12.518 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.515+0000 7f2edbe00640 1 --2- 172.21.15.72:0/3655083912 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c3827aa430 0x55c3827bed30 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f2ed402a900 tx=0x7f2ed40095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98d1d6070d2fdb52 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:57:12.518 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.515+0000 7f2ec5600640 10 client.0 ms_handle_connect on v2:172.21.15.72:3300/0 2024-04-26T21:57:12.518 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.515+0000 7f2ec5600640 1 -- 172.21.15.72:0/3655083912 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ed40ba540 con 0x55c3827aa430 2024-04-26T21:57:12.518 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.515+0000 7f2ec5600640 1 -- 172.21.15.72:0/3655083912 <== mon.0 v2:172.21.15.72:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7f2ed40041e0 con 0x55c3827aa430 2024-04-26T21:57:12.518 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.515+0000 7f2ee558d740 1 -- 172.21.15.72:0/3655083912 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2ed4036020 con 0x55c3827aa430 2024-04-26T21:57:12.518 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.515+0000 7f2ec5600640 1 -- 172.21.15.72:0/3655083912 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2ed40c2110 con 0x55c3827aa430 2024-04-26T21:57:12.518 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.515+0000 7f2ee558d740 10 client.15972 fetch_fsmap learned FSMap version 27 2024-04-26T21:57:12.518 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.515+0000 7f2ee558d740 1 -- 172.21.15.72:0/3655083912 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c3826d2a30 con 0x55c3827aa430 2024-04-26T21:57:12.518 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.515+0000 7f2ec5600640 1 -- 172.21.15.72:0/3655083912 <== mon.0 v2:172.21.15.72:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2ed4036020 con 0x55c3827aa430 2024-04-26T21:57:12.518 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.516+0000 7f2ec5600640 1 -- 172.21.15.72:0/3655083912 <== mon.0 v2:172.21.15.72:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f2ed40c8ce0 con 0x55c3827aa430 2024-04-26T21:57:12.519 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.516+0000 7f2ee558d740 10 client.15972 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T21:57:12.519 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.516+0000 7f2ee558d740 10 client.15972 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:57:12.519 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.516+0000 7f2ee558d740 20 client.15972 populate_metadata read hostname 'smithi072' 2024-04-26T21:57:12.519 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.516+0000 7f2ee558d740 1 --2- 172.21.15.72:0/3655083912 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x55c38282b0f0 0x55c38282d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:12.519 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.516+0000 7f2ee558d740 4 client.15972 mds_command: new command op to 34517 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-26T21:57:12.519 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.516+0000 7f2ee558d740 1 -- 172.21.15.72:0/3655083912 --> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c382712650 con 0x55c38282b0f0 2024-04-26T21:57:12.520 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.516+0000 7f2edb400640 1 --2- 172.21.15.72:0/3655083912 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x55c38282b0f0 0x55c38282d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:12.520 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.517+0000 7f2edb400640 1 --2- 172.21.15.72:0/3655083912 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x55c38282b0f0 0x55c38282d4e0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e6f47087d8419cb4 server_cookie=4ac70f53fa63bd95 in_seq=0 out_seq=0 2024-04-26T21:57:12.520 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.517+0000 7f2ec5600640 10 client.15972 ms_handle_connect on v2:172.21.15.76:6838/327145636 2024-04-26T21:57:12.520 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.518+0000 7f2ec5600640 1 -- 172.21.15.72:0/3655083912 <== mds.0 v2:172.21.15.76:6838/327145636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55c382712650 con 0x55c38282b0f0 2024-04-26T21:57:12.520 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.518+0000 7f2ec5600640 10 client.15972 handle_command_reply: tid=0 2024-04-26T21:57:12.521 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.518+0000 7f2ee558d740 1 client.15972 shutdown 2024-04-26T21:57:12.521 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.518+0000 7f2ee558d740 1 -- 172.21.15.72:0/3655083912 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c3827aa430 msgr2=0x55c3827bed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:12.521 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.518+0000 7f2ee558d740 1 --2- 172.21.15.72:0/3655083912 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c3827aa430 0x55c3827bed30 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f2ed402a900 tx=0x7f2ed40095c0 comp rx=0 tx=0).stop 2024-04-26T21:57:12.521 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.518+0000 7f2ee558d740 1 -- 172.21.15.72:0/3655083912 shutdown_connections 2024-04-26T21:57:12.521 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.518+0000 7f2ee558d740 1 --2- 172.21.15.72:0/3655083912 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x55c38282b0f0 0x55c38282d4e0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:12.521 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.518+0000 7f2ee558d740 1 --2- 172.21.15.72:0/3655083912 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c3827aa430 0x55c3827bed30 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:12.521 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.518+0000 7f2ee558d740 1 --2- 172.21.15.72:0/3655083912 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c3827a72d0 0x55c3827bdde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:12.521 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.518+0000 7f2ec5600640 0 client.15972 ms_handle_reset on v2:172.21.15.76:6838/327145636 2024-04-26T21:57:12.521 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.518+0000 7f2ee558d740 1 --2- 172.21.15.72:0/3655083912 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c3827a68e0 0x55c3827bd8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:12.521 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.518+0000 7f2ee558d740 1 -- 172.21.15.72:0/3655083912 >> 172.21.15.72:0/3655083912 conn(0x55c3827a1fd0 msgr2=0x55c3827a11f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:57:12.521 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.518+0000 7f2ee558d740 1 -- 172.21.15.72:0/3655083912 shutdown_connections 2024-04-26T21:57:12.521 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.518+0000 7f2ee558d740 1 -- 172.21.15.72:0/3655083912 wait complete. 2024-04-26T21:57:12.521 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.518+0000 7f2ee558d740 20 client.15972 trim_cache size 0 max 16384 2024-04-26T21:57:12.522 INFO:teuthology.orchestra.run.smithi072.stdout:{ 2024-04-26T21:57:12.522 INFO:teuthology.orchestra.run.smithi072.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-26T21:57:12.522 INFO:teuthology.orchestra.run.smithi072.stdout: "scrubs": { 2024-04-26T21:57:12.522 INFO:teuthology.orchestra.run.smithi072.stdout: "224864bf-2b78-48ff-8722-ede585665410": { 2024-04-26T21:57:12.522 INFO:teuthology.orchestra.run.smithi072.stdout: "path": "/", 2024-04-26T21:57:12.522 INFO:teuthology.orchestra.run.smithi072.stdout: "tag": "224864bf-2b78-48ff-8722-ede585665410", 2024-04-26T21:57:12.522 INFO:teuthology.orchestra.run.smithi072.stdout: "options": "recursive,force" 2024-04-26T21:57:12.522 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:57:12.522 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:57:12.522 INFO:teuthology.orchestra.run.smithi072.stdout:} 2024-04-26T21:57:12.523 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.519+0000 7f2ee1000640 1 -- 172.21.15.72:0/2278728592 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f2ec00a3c40 msgr2=0x7f2ec00a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:12.523 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.519+0000 7f2ee1000640 1 --2- 172.21.15.72:0/2278728592 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f2ec00a3c40 0x7f2ec00a6100 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x55c3827a76d0 tx=0x7f2ecc00b390 comp rx=0 tx=0).stop 2024-04-26T21:57:12.523 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.519+0000 7f2ee1000640 1 -- 172.21.15.72:0/2278728592 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2edc104410 msgr2=0x7f2edc19fed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:12.523 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.519+0000 7f2ee1000640 1 --2- 172.21.15.72:0/2278728592 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2edc104410 0x7f2edc19fed0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f2ed4031040 tx=0x7f2ed4003ba0 comp rx=0 tx=0).stop 2024-04-26T21:57:12.523 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.519+0000 7f2ee1000640 1 -- 172.21.15.72:0/2278728592 shutdown_connections 2024-04-26T21:57:12.523 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.519+0000 7f2ee1000640 1 --2- 172.21.15.72:0/2278728592 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2edc104410 0x7f2edc19fed0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:12.523 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.519+0000 7f2ee1000640 1 --2- 172.21.15.72:0/2278728592 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2edc103a50 0x7f2edc198e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:12.524 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.519+0000 7f2ee1000640 1 --2- 172.21.15.72:0/2278728592 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f2ec00a3c40 0x7f2ec00a6100 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:12.524 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.519+0000 7f2ee1000640 1 --2- 172.21.15.72:0/2278728592 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2edc102860 0x7f2edc198920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:12.524 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.519+0000 7f2ee1000640 1 -- 172.21.15.72:0/2278728592 >> 172.21.15.72:0/2278728592 conn(0x7f2edc0fddd0 msgr2=0x7f2edc106e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:57:12.524 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.519+0000 7f2ee1000640 1 -- 172.21.15.72:0/2278728592 shutdown_connections 2024-04-26T21:57:12.524 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:12.520+0000 7f2ee1000640 1 -- 172.21.15.72:0/2278728592 wait complete. 2024-04-26T21:57:12.539 INFO:tasks.cephfs.filesystem:scrub status for tag:224864bf-2b78-48ff-8722-ede585665410 - {'path': '/', 'tag': '224864bf-2b78-48ff-8722-ede585665410', 'options': 'recursive,force'} 2024-04-26T21:57:13.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:57:12 smithi076 ceph-mon[31823]: pgmap v502: 97 pgs: 97 active+clean; 25 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 2.0 MiB/s wr, 11 op/s 2024-04-26T21:57:13.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:57:12 smithi076 ceph-mon[31823]: scrub queued for path: / 2024-04-26T21:57:13.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:57:12 smithi076 ceph-mon[31823]: scrub summary: idle+waiting paths [/] 2024-04-26T21:57:13.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:57:12 smithi076 ceph-mon[31823]: scrub summary: active paths [/] 2024-04-26T21:57:13.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:57:12 smithi076 ceph-mon[31823]: scrub summary: idle+waiting paths [/] 2024-04-26T21:57:13.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:57:12 smithi072 ceph-mon[28138]: pgmap v502: 97 pgs: 97 active+clean; 25 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 2.0 MiB/s wr, 11 op/s 2024-04-26T21:57:13.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:57:12 smithi072 ceph-mon[28138]: scrub queued for path: / 2024-04-26T21:57:13.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:57:12 smithi072 ceph-mon[28138]: scrub summary: idle+waiting paths [/] 2024-04-26T21:57:13.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:57:12 smithi072 ceph-mon[28138]: scrub summary: active paths [/] 2024-04-26T21:57:13.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:57:12 smithi072 ceph-mon[28138]: scrub summary: idle+waiting paths [/] 2024-04-26T21:57:13.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:57:12 smithi164 ceph-mon[31916]: pgmap v502: 97 pgs: 97 active+clean; 25 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 2.0 MiB/s wr, 11 op/s 2024-04-26T21:57:13.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:57:12 smithi164 ceph-mon[31916]: scrub queued for path: / 2024-04-26T21:57:13.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:57:12 smithi164 ceph-mon[31916]: scrub summary: idle+waiting paths [/] 2024-04-26T21:57:13.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:57:12 smithi164 ceph-mon[31916]: scrub summary: active paths [/] 2024-04-26T21:57:13.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:57:12 smithi164 ceph-mon[31916]: scrub summary: idle+waiting paths [/] 2024-04-26T21:57:15.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:57:14 smithi076 ceph-mon[31823]: pgmap v503: 97 pgs: 97 active+clean; 37 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 3.0 MiB/s wr, 12 op/s 2024-04-26T21:57:15.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:57:14 smithi076 ceph-mon[31823]: scrub summary: idle 2024-04-26T21:57:15.321 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:57:14 smithi072 ceph-mon[28138]: pgmap v503: 97 pgs: 97 active+clean; 37 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 3.0 MiB/s wr, 12 op/s 2024-04-26T21:57:15.321 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:57:14 smithi072 ceph-mon[28138]: scrub summary: idle 2024-04-26T21:57:15.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:57:14 smithi164 ceph-mon[31916]: pgmap v503: 97 pgs: 97 active+clean; 37 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 3.0 MiB/s wr, 12 op/s 2024-04-26T21:57:15.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:57:14 smithi164 ceph-mon[31916]: scrub summary: idle 2024-04-26T21:57:17.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:57:16 smithi076 ceph-mon[31823]: pgmap v504: 97 pgs: 97 active+clean; 53 MiB data, 551 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.3 MiB/s wr, 15 op/s 2024-04-26T21:57:17.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:57:16 smithi072 ceph-mon[28138]: pgmap v504: 97 pgs: 97 active+clean; 53 MiB data, 551 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.3 MiB/s wr, 15 op/s 2024-04-26T21:57:17.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:57:16 smithi164 ceph-mon[31916]: pgmap v504: 97 pgs: 97 active+clean; 53 MiB data, 551 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.3 MiB/s wr, 15 op/s 2024-04-26T21:57:19.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:57:18 smithi076 ceph-mon[31823]: pgmap v505: 97 pgs: 97 active+clean; 65 MiB data, 575 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 5.3 MiB/s wr, 11 op/s 2024-04-26T21:57:19.321 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:57:18 smithi072 ceph-mon[28138]: pgmap v505: 97 pgs: 97 active+clean; 65 MiB data, 575 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 5.3 MiB/s wr, 11 op/s 2024-04-26T21:57:19.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:57:18 smithi164 ceph-mon[31916]: pgmap v505: 97 pgs: 97 active+clean; 65 MiB data, 575 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 5.3 MiB/s wr, 11 op/s 2024-04-26T21:57:21.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:57:20 smithi076 ceph-mon[31823]: pgmap v506: 97 pgs: 97 active+clean; 85 MiB data, 627 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.7 MiB/s wr, 15 op/s 2024-04-26T21:57:21.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:57:20 smithi072 ceph-mon[28138]: pgmap v506: 97 pgs: 97 active+clean; 85 MiB data, 627 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.7 MiB/s wr, 15 op/s 2024-04-26T21:57:21.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:57:20 smithi164 ceph-mon[31916]: pgmap v506: 97 pgs: 97 active+clean; 85 MiB data, 627 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.7 MiB/s wr, 15 op/s 2024-04-26T21:57:23.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:57:22 smithi076 ceph-mon[31823]: pgmap v507: 97 pgs: 97 active+clean; 85 MiB data, 631 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 5.7 MiB/s wr, 10 op/s 2024-04-26T21:57:23.321 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:57:22 smithi072 ceph-mon[28138]: pgmap v507: 97 pgs: 97 active+clean; 85 MiB data, 631 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 5.7 MiB/s wr, 10 op/s 2024-04-26T21:57:23.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:57:22 smithi164 ceph-mon[31916]: pgmap v507: 97 pgs: 97 active+clean; 85 MiB data, 631 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 5.7 MiB/s wr, 10 op/s 2024-04-26T21:57:25.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:57:25 smithi076 ceph-mon[31823]: pgmap v508: 97 pgs: 97 active+clean; 85 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 5.0 MiB/s wr, 11 op/s 2024-04-26T21:57:25.321 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:57:25 smithi072 ceph-mon[28138]: pgmap v508: 97 pgs: 97 active+clean; 85 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 5.0 MiB/s wr, 11 op/s 2024-04-26T21:57:25.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:57:25 smithi164 ceph-mon[31916]: pgmap v508: 97 pgs: 97 active+clean; 85 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 5.0 MiB/s wr, 11 op/s 2024-04-26T21:57:27.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:57:27 smithi076 ceph-mon[31823]: pgmap v509: 97 pgs: 97 active+clean; 105 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 5.7 MiB/s wr, 14 op/s 2024-04-26T21:57:27.321 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:57:27 smithi072 ceph-mon[28138]: pgmap v509: 97 pgs: 97 active+clean; 105 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 5.7 MiB/s wr, 14 op/s 2024-04-26T21:57:27.444 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:57:27 smithi164 ceph-mon[31916]: pgmap v509: 97 pgs: 97 active+clean; 105 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 5.7 MiB/s wr, 14 op/s 2024-04-26T21:57:29.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:57:29 smithi076 ceph-mon[31823]: pgmap v510: 97 pgs: 97 active+clean; 117 MiB data, 728 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 5.3 MiB/s wr, 10 op/s 2024-04-26T21:57:29.321 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:57:29 smithi072 ceph-mon[28138]: pgmap v510: 97 pgs: 97 active+clean; 117 MiB data, 728 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 5.3 MiB/s wr, 10 op/s 2024-04-26T21:57:29.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:57:29 smithi164 ceph-mon[31916]: pgmap v510: 97 pgs: 97 active+clean; 117 MiB data, 728 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 5.3 MiB/s wr, 10 op/s 2024-04-26T21:57:31.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:57:31 smithi076 ceph-mon[31823]: pgmap v511: 97 pgs: 97 active+clean; 125 MiB data, 768 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 5.0 MiB/s wr, 13 op/s 2024-04-26T21:57:31.321 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:57:31 smithi072 ceph-mon[28138]: pgmap v511: 97 pgs: 97 active+clean; 125 MiB data, 768 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 5.0 MiB/s wr, 13 op/s 2024-04-26T21:57:31.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:57:31 smithi164 ceph-mon[31916]: pgmap v511: 97 pgs: 97 active+clean; 125 MiB data, 768 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 5.0 MiB/s wr, 13 op/s 2024-04-26T21:57:33.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:57:33 smithi076 ceph-mon[31823]: pgmap v512: 97 pgs: 97 active+clean; 137 MiB data, 788 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 4.3 MiB/s wr, 9 op/s 2024-04-26T21:57:33.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:57:33 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:57:33.321 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:57:33 smithi072 ceph-mon[28138]: pgmap v512: 97 pgs: 97 active+clean; 137 MiB data, 788 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 4.3 MiB/s wr, 9 op/s 2024-04-26T21:57:33.321 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:57:33 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:57:33.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:57:33 smithi164 ceph-mon[31916]: pgmap v512: 97 pgs: 97 active+clean; 137 MiB data, 788 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 4.3 MiB/s wr, 9 op/s 2024-04-26T21:57:33.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:57:33 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:57:35.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:57:35 smithi076 ceph-mon[31823]: pgmap v513: 97 pgs: 97 active+clean; 157 MiB data, 812 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 6.0 MiB/s wr, 11 op/s 2024-04-26T21:57:35.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:57:35 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:57:35.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:57:35 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:57:35.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:57:35 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:57:35.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:57:35 smithi072 ceph-mon[28138]: pgmap v513: 97 pgs: 97 active+clean; 157 MiB data, 812 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 6.0 MiB/s wr, 11 op/s 2024-04-26T21:57:35.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:57:35 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:57:35.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:57:35 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:57:35.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:57:35 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:57:35.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:57:35 smithi164 ceph-mon[31916]: pgmap v513: 97 pgs: 97 active+clean; 157 MiB data, 812 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 6.0 MiB/s wr, 11 op/s 2024-04-26T21:57:35.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:57:35 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:57:35.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:57:35 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:57:35.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:57:35 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:57:36.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:57:36 smithi076 ceph-mon[31823]: pgmap v514: 97 pgs: 97 active+clean; 177 MiB data, 872 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 7.7 MiB/s wr, 13 op/s 2024-04-26T21:57:36.321 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:57:36 smithi072 ceph-mon[28138]: pgmap v514: 97 pgs: 97 active+clean; 177 MiB data, 872 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 7.7 MiB/s wr, 13 op/s 2024-04-26T21:57:36.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:57:36 smithi164 ceph-mon[31916]: pgmap v514: 97 pgs: 97 active+clean; 177 MiB data, 872 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 7.7 MiB/s wr, 13 op/s 2024-04-26T21:57:39.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:57:38 smithi076 ceph-mon[31823]: pgmap v515: 97 pgs: 97 active+clean; 181 MiB data, 892 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 6.3 MiB/s wr, 10 op/s 2024-04-26T21:57:39.321 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:57:38 smithi072 ceph-mon[28138]: pgmap v515: 97 pgs: 97 active+clean; 181 MiB data, 892 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 6.3 MiB/s wr, 10 op/s 2024-04-26T21:57:39.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:57:38 smithi164 ceph-mon[31916]: pgmap v515: 97 pgs: 97 active+clean; 181 MiB data, 892 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 6.3 MiB/s wr, 10 op/s 2024-04-26T21:57:41.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:57:40 smithi076 ceph-mon[31823]: pgmap v516: 97 pgs: 97 active+clean; 189 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 6.0 MiB/s wr, 13 op/s 2024-04-26T21:57:41.321 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:57:40 smithi072 ceph-mon[28138]: pgmap v516: 97 pgs: 97 active+clean; 189 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 6.0 MiB/s wr, 13 op/s 2024-04-26T21:57:41.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:57:40 smithi164 ceph-mon[31916]: pgmap v516: 97 pgs: 97 active+clean; 189 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 6.0 MiB/s wr, 13 op/s 2024-04-26T21:57:42.540 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-26T21:57:42.541 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-26T21:57:42.771 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.771+0000 7f71c3e00640 1 -- 172.21.15.72:0/1228630952 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f71c4100e90 msgr2=0x7f71c4107fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:42.771 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.771+0000 7f71c3e00640 1 --2- 172.21.15.72:0/1228630952 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f71c4100e90 0x7f71c4107fe0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f71ac009900 tx=0x7f71ac02a3f0 comp rx=0 tx=0).stop 2024-04-26T21:57:42.771 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.771+0000 7f71c3e00640 1 -- 172.21.15.72:0/1228630952 shutdown_connections 2024-04-26T21:57:42.771 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.771+0000 7f71c3e00640 1 --2- 172.21.15.72:0/1228630952 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f71c4108520 0x7f71c410a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:42.771 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.771+0000 7f71c3e00640 1 --2- 172.21.15.72:0/1228630952 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f71c4100e90 0x7f71c4107fe0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:42.772 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.771+0000 7f71c3e00640 1 --2- 172.21.15.72:0/1228630952 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f71c41004a0 0x7f71c41008c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:42.772 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.771+0000 7f71c3e00640 1 -- 172.21.15.72:0/1228630952 >> 172.21.15.72:0/1228630952 conn(0x7f71c40fbff0 msgr2=0x7f71c40fe430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:57:42.772 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.771+0000 7f71c3e00640 1 -- 172.21.15.72:0/1228630952 shutdown_connections 2024-04-26T21:57:42.772 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.771+0000 7f71c3e00640 1 -- 172.21.15.72:0/1228630952 wait complete. 2024-04-26T21:57:42.773 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.772+0000 7f71c3e00640 1 Processor -- start 2024-04-26T21:57:42.773 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.772+0000 7f71c3e00640 1 -- start start 2024-04-26T21:57:42.773 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.773+0000 7f71c3e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f71c41004a0 0x7f71c4103930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:42.773 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.773+0000 7f71c3e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f71c4100e90 0x7f71c4101f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:42.773 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.773+0000 7f71c3e00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f71c4108520 0x7f71c41024b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:42.773 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.773+0000 7f71c3e00640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f71c410e240 con 0x7f71c4100e90 2024-04-26T21:57:42.773 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.773+0000 7f71c3e00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f71c410e0c0 con 0x7f71c4108520 2024-04-26T21:57:42.773 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.773+0000 7f71c3e00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f71c410e3c0 con 0x7f71c41004a0 2024-04-26T21:57:42.775 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.773+0000 7f71bbe00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f71c4100e90 0x7f71c4101f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:42.775 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.773+0000 7f71bbe00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f71c4100e90 0x7f71c4101f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:47908/0 (socket says 172.21.15.72:47908) 2024-04-26T21:57:42.775 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.773+0000 7f71bbe00640 1 -- 172.21.15.72:0/1217030083 learned_addr learned my addr 172.21.15.72:0/1217030083 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:57:42.775 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.774+0000 7f71c3400640 1 --2- 172.21.15.72:0/1217030083 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f71c4108520 0x7f71c41024b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:42.775 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.774+0000 7f71c2a00640 1 --2- 172.21.15.72:0/1217030083 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f71c41004a0 0x7f71c4103930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:42.775 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.774+0000 7f71bbe00640 1 -- 172.21.15.72:0/1217030083 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f71c41004a0 msgr2=0x7f71c4103930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:42.775 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.774+0000 7f71bbe00640 1 --2- 172.21.15.72:0/1217030083 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f71c41004a0 0x7f71c4103930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:42.775 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.774+0000 7f71bbe00640 1 -- 172.21.15.72:0/1217030083 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f71c4108520 msgr2=0x7f71c41024b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:42.775 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.774+0000 7f71bbe00640 1 --2- 172.21.15.72:0/1217030083 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f71c4108520 0x7f71c41024b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:42.776 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.774+0000 7f71bbe00640 1 -- 172.21.15.72:0/1217030083 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71c4102d40 con 0x7f71c4100e90 2024-04-26T21:57:42.776 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.774+0000 7f71bbe00640 1 --2- 172.21.15.72:0/1217030083 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f71c4100e90 0x7f71c4101f70 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f71ac009230 tx=0x7f71ac0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:57:42.776 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.774+0000 7f71bb400640 1 -- 172.21.15.72:0/1217030083 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71ac03f030 con 0x7f71c4100e90 2024-04-26T21:57:42.776 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.774+0000 7f71bb400640 1 -- 172.21.15.72:0/1217030083 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f71ac003c90 con 0x7f71c4100e90 2024-04-26T21:57:42.776 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.774+0000 7f71bb400640 1 -- 172.21.15.72:0/1217030083 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71ac035070 con 0x7f71c4100e90 2024-04-26T21:57:42.776 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.775+0000 7f71c3e00640 1 -- 172.21.15.72:0/1217030083 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71c41a2b00 con 0x7f71c4100e90 2024-04-26T21:57:42.776 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.775+0000 7f71c3e00640 1 -- 172.21.15.72:0/1217030083 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71c41a3000 con 0x7f71c4100e90 2024-04-26T21:57:42.778 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.778+0000 7f71bb400640 1 -- 172.21.15.72:0/1217030083 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f71ac042030 con 0x7f71c4100e90 2024-04-26T21:57:42.778 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.779+0000 7f71cca42740 1 Processor -- start 2024-04-26T21:57:42.779 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.779+0000 7f71cca42740 1 -- start start 2024-04-26T21:57:42.780 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.779+0000 7f71cca42740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a30ada0960 0x55a30ada0d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:42.780 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.779+0000 7f71cca42740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a30ada1350 0x55a30ada5e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:42.780 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.779+0000 7f71cca42740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a30ada64c0 0x55a30ada8950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:42.780 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.779+0000 7f71cca42740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55a30acb1030 con 0x55a30ada1350 2024-04-26T21:57:42.780 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.779+0000 7f71cca42740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55a30accc7f0 con 0x55a30ada64c0 2024-04-26T21:57:42.780 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.779+0000 7f71cca42740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55a30ac877e0 con 0x55a30ada0960 2024-04-26T21:57:42.780 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.780+0000 7f71c2a00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a30ada1350 0x55a30ada5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:42.780 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.780+0000 7f71c2a00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a30ada1350 0x55a30ada5e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:47920/0 (socket says 172.21.15.72:47920) 2024-04-26T21:57:42.780 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.780+0000 7f71c2a00640 1 -- 172.21.15.72:0/3218510324 learned_addr learned my addr 172.21.15.72:0/3218510324 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:57:42.780 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.780+0000 7f71bbe00640 1 --2- 172.21.15.72:0/3218510324 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a30ada64c0 0x55a30ada8950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:42.782 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.780+0000 7f71c2a00640 1 -- 172.21.15.72:0/3218510324 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a30ada0960 msgr2=0x55a30ada0d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:42.782 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.780+0000 7f71c2a00640 1 --2- 172.21.15.72:0/3218510324 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a30ada0960 0x55a30ada0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:42.782 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.780+0000 7f71c2a00640 1 -- 172.21.15.72:0/3218510324 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a30ada64c0 msgr2=0x55a30ada8950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:42.782 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.780+0000 7f71c2a00640 1 --2- 172.21.15.72:0/3218510324 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a30ada64c0 0x55a30ada8950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:42.782 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.780+0000 7f71c2a00640 1 -- 172.21.15.72:0/3218510324 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a30ac93050 con 0x55a30ada1350 2024-04-26T21:57:42.782 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.780+0000 7f71bb400640 1 --2- 172.21.15.72:0/1217030083 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f71a80a3cc0 0x7f71a80a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:42.782 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.780+0000 7f71bb400640 1 -- 172.21.15.72:0/1217030083 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7f71ac0f0d90 con 0x7f71c4100e90 2024-04-26T21:57:42.782 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.781+0000 7f71c3400640 1 --2- 172.21.15.72:0/1217030083 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f71a80a3cc0 0x7f71a80a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:42.782 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.781+0000 7f71c2a00640 1 --2- 172.21.15.72:0/3218510324 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a30ada1350 0x55a30ada5e50 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f71b400bcf0 tx=0x7f71b400f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbd55bf699bf1a53 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:57:42.783 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.781+0000 7f71b3e00640 1 -- 172.21.15.72:0/3218510324 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71b4015070 con 0x55a30ada1350 2024-04-26T21:57:42.783 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.781+0000 7f71b3e00640 1 -- 172.21.15.72:0/3218510324 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f71b401a8f0 con 0x55a30ada1350 2024-04-26T21:57:42.783 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.781+0000 7f71b3e00640 1 -- 172.21.15.72:0/3218510324 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71b40118a0 con 0x55a30ada1350 2024-04-26T21:57:42.783 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.782+0000 7f71cca42740 1 -- 172.21.15.72:0/3218510324 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a30ada1350 msgr2=0x55a30ada5e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:42.783 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.782+0000 7f71cca42740 1 --2- 172.21.15.72:0/3218510324 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a30ada1350 0x55a30ada5e50 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f71b400bcf0 tx=0x7f71b400f730 comp rx=0 tx=0).stop 2024-04-26T21:57:42.783 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.782+0000 7f71c3400640 1 --2- 172.21.15.72:0/1217030083 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f71a80a3cc0 0x7f71a80a6180 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x55a30ada1bb0 tx=0x7f71bc0094a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:57:42.783 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.782+0000 7f71cca42740 1 -- 172.21.15.72:0/3218510324 shutdown_connections 2024-04-26T21:57:42.783 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.782+0000 7f71cca42740 1 --2- 172.21.15.72:0/3218510324 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a30ada64c0 0x55a30ada8950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:42.783 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.782+0000 7f71cca42740 1 --2- 172.21.15.72:0/3218510324 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a30ada1350 0x55a30ada5e50 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:42.783 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.782+0000 7f71cca42740 1 --2- 172.21.15.72:0/3218510324 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a30ada0960 0x55a30ada0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:42.783 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.782+0000 7f71cca42740 1 -- 172.21.15.72:0/3218510324 >> 172.21.15.72:0/3218510324 conn(0x55a30ad9bfd0 msgr2=0x55a30ad9c3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:57:42.783 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.782+0000 7f71cca42740 1 -- 172.21.15.72:0/3218510324 shutdown_connections 2024-04-26T21:57:42.783 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.782+0000 7f71cca42740 1 -- 172.21.15.72:0/3218510324 wait complete. 2024-04-26T21:57:42.786 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.783+0000 7f71cca42740 1 -- start start 2024-04-26T21:57:42.786 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.783+0000 7f71cca42740 1 Processor -- start 2024-04-26T21:57:42.787 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.783+0000 7f71cca42740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a30ada0960 0x55a30adaa810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:42.787 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.783+0000 7f71cca42740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a30ada1350 0x55a30adaad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:42.787 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.783+0000 7f71cca42740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a30ada64c0 0x55a30adb1d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:42.787 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.783+0000 7f71cca42740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55a30ad09200 con 0x55a30ada64c0 2024-04-26T21:57:42.787 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.783+0000 7f71cca42740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55a30ad03ee0 con 0x55a30ada1350 2024-04-26T21:57:42.787 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.783+0000 7f71cca42740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55a30ad10e70 con 0x55a30ada0960 2024-04-26T21:57:42.787 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.784+0000 7f71cca42740 10 client.0 _ll_register_callbacks cb 0x55a30acfa270 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:57:42.787 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.784+0000 7f71c3400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a30ada64c0 0x55a30adb1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:42.787 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.784+0000 7f71c3400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a30ada64c0 0x55a30adb1d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:47928/0 (socket says 172.21.15.72:47928) 2024-04-26T21:57:42.787 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.784+0000 7f71c3400640 1 -- 172.21.15.72:0/139207480 learned_addr learned my addr 172.21.15.72:0/139207480 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:57:42.787 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.784+0000 7f71c2a00640 1 --2- 172.21.15.72:0/139207480 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a30ada0960 0x55a30adaa810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:42.787 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.784+0000 7f71bbe00640 1 --2- 172.21.15.72:0/139207480 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a30ada1350 0x55a30adaad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:42.787 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.784+0000 7f71c3400640 1 -- 172.21.15.72:0/139207480 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a30ada0960 msgr2=0x55a30adaa810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:42.788 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.784+0000 7f71c3400640 1 --2- 172.21.15.72:0/139207480 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a30ada0960 0x55a30adaa810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:42.788 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.784+0000 7f71c3400640 1 -- 172.21.15.72:0/139207480 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a30ada1350 msgr2=0x55a30adaad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:42.788 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.784+0000 7f71c3400640 1 --2- 172.21.15.72:0/139207480 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a30ada1350 0x55a30adaad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:42.788 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.784+0000 7f71c3400640 1 -- 172.21.15.72:0/139207480 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a30aae07d0 con 0x55a30ada64c0 2024-04-26T21:57:42.788 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.785+0000 7f71c3400640 1 --2- 172.21.15.72:0/139207480 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a30ada64c0 0x55a30adb1d90 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f71bc0126d0 tx=0x7f71bc004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80f57f6ee72eb143 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:57:42.788 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.785+0000 7f71b0c00640 10 client.0 ms_handle_connect on v2:172.21.15.72:3300/0 2024-04-26T21:57:42.788 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.785+0000 7f71b0c00640 1 -- 172.21.15.72:0/139207480 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71bc022030 con 0x55a30ada64c0 2024-04-26T21:57:42.788 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.785+0000 7f71b0c00640 1 -- 172.21.15.72:0/139207480 <== mon.0 v2:172.21.15.72:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7f71bc011870 con 0x55a30ada64c0 2024-04-26T21:57:42.788 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.785+0000 7f71cca42740 1 -- 172.21.15.72:0/139207480 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f71b401a8f0 con 0x55a30ada64c0 2024-04-26T21:57:42.788 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.785+0000 7f71b0c00640 1 -- 172.21.15.72:0/139207480 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f71bc02c030 con 0x55a30ada64c0 2024-04-26T21:57:42.788 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.785+0000 7f71cca42740 10 client.15984 fetch_fsmap learned FSMap version 27 2024-04-26T21:57:42.788 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.785+0000 7f71cca42740 1 -- 172.21.15.72:0/139207480 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a30aa163f0 con 0x55a30ada64c0 2024-04-26T21:57:42.788 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.786+0000 7f71b0c00640 1 -- 172.21.15.72:0/139207480 <== mon.0 v2:172.21.15.72:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f71b401a8f0 con 0x55a30ada64c0 2024-04-26T21:57:42.788 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.786+0000 7f71b0c00640 1 -- 172.21.15.72:0/139207480 <== mon.0 v2:172.21.15.72:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f71bc00f040 con 0x55a30ada64c0 2024-04-26T21:57:42.788 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.786+0000 7f71cca42740 10 client.15984 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T21:57:42.789 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.786+0000 7f71cca42740 10 client.15984 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:57:42.789 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.786+0000 7f71cca42740 20 client.15984 populate_metadata read hostname 'smithi072' 2024-04-26T21:57:42.789 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.786+0000 7f71cca42740 1 --2- 172.21.15.72:0/139207480 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x55a30adb6310 0x55a30adb8790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:42.789 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.786+0000 7f71cca42740 4 client.15984 mds_command: new command op to 34517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:57:42.789 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.786+0000 7f71cca42740 1 -- 172.21.15.72:0/139207480 --> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a30ad127e0 con 0x55a30adb6310 2024-04-26T21:57:42.790 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.787+0000 7f71c2a00640 1 --2- 172.21.15.72:0/139207480 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x55a30adb6310 0x55a30adb8790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:42.790 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.788+0000 7f71c2a00640 1 --2- 172.21.15.72:0/139207480 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x55a30adb6310 0x55a30adb8790 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc74f570736ae63b server_cookie=74590f36b4894a7d in_seq=0 out_seq=0 2024-04-26T21:57:42.790 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.788+0000 7f71b0c00640 10 client.15984 ms_handle_connect on v2:172.21.15.76:6838/327145636 2024-04-26T21:57:42.791 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.790+0000 7f71b0c00640 1 -- 172.21.15.72:0/139207480 <== mds.0 v2:172.21.15.76:6838/327145636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a30ad127e0 con 0x55a30adb6310 2024-04-26T21:57:42.791 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.790+0000 7f71b0c00640 10 client.15984 handle_command_reply: tid=0 2024-04-26T21:57:42.791 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.790+0000 7f71cca42740 1 client.15984 shutdown 2024-04-26T21:57:42.791 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.790+0000 7f71cca42740 1 -- 172.21.15.72:0/139207480 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a30ada64c0 msgr2=0x55a30adb1d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:42.791 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.790+0000 7f71cca42740 1 --2- 172.21.15.72:0/139207480 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a30ada64c0 0x55a30adb1d90 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f71bc0126d0 tx=0x7f71bc004ab0 comp rx=0 tx=0).stop 2024-04-26T21:57:42.791 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.790+0000 7f71cca42740 1 -- 172.21.15.72:0/139207480 shutdown_connections 2024-04-26T21:57:42.791 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.790+0000 7f71cca42740 1 --2- 172.21.15.72:0/139207480 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x55a30adb6310 0x55a30adb8790 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:42.791 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.790+0000 7f71cca42740 1 --2- 172.21.15.72:0/139207480 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a30ada64c0 0x55a30adb1d90 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:42.791 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.790+0000 7f71cca42740 1 --2- 172.21.15.72:0/139207480 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a30ada1350 0x55a30adaad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:42.791 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.790+0000 7f71cca42740 1 --2- 172.21.15.72:0/139207480 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a30ada0960 0x55a30adaa810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:42.791 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.790+0000 7f71b0c00640 0 client.15984 ms_handle_reset on v2:172.21.15.76:6838/327145636 2024-04-26T21:57:42.791 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.790+0000 7f71cca42740 1 -- 172.21.15.72:0/139207480 >> 172.21.15.72:0/139207480 conn(0x55a30ad9bfd0 msgr2=0x55a30ad9b1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:57:42.791 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.790+0000 7f71cca42740 1 -- 172.21.15.72:0/139207480 shutdown_connections 2024-04-26T21:57:42.791 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.790+0000 7f71cca42740 1 -- 172.21.15.72:0/139207480 wait complete. 2024-04-26T21:57:42.792 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.790+0000 7f71cca42740 20 client.15984 trim_cache size 0 max 16384 2024-04-26T21:57:42.812 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.812+0000 7f71cca42740 1 Processor -- start 2024-04-26T21:57:42.812 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.812+0000 7f71cca42740 1 -- start start 2024-04-26T21:57:42.813 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.812+0000 7f71cca42740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a30ada0960 0x55a30ad9f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:42.813 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.812+0000 7f71cca42740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a30ada1350 0x55a30adb1d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:42.813 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.812+0000 7f71cca42740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a30ada64c0 0x55a30ada59e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:42.813 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.812+0000 7f71cca42740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f71c41adad0 con 0x55a30ada64c0 2024-04-26T21:57:42.813 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.812+0000 7f71cca42740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f71c41ac660 con 0x55a30ada0960 2024-04-26T21:57:42.813 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.812+0000 7f71cca42740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f71c41acd30 con 0x55a30ada1350 2024-04-26T21:57:42.814 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.813+0000 7f71c3400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a30ada64c0 0x55a30ada59e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:42.814 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.813+0000 7f71c3400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a30ada64c0 0x55a30ada59e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:47930/0 (socket says 172.21.15.72:47930) 2024-04-26T21:57:42.814 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.813+0000 7f71c3400640 1 -- 172.21.15.72:0/2574584086 learned_addr learned my addr 172.21.15.72:0/2574584086 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:57:42.814 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.813+0000 7f71c2a00640 1 --2- 172.21.15.72:0/2574584086 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a30ada0960 0x55a30ad9f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:42.814 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.813+0000 7f71bbe00640 1 --2- 172.21.15.72:0/2574584086 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a30ada1350 0x55a30adb1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:42.814 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.813+0000 7f71c3400640 1 -- 172.21.15.72:0/2574584086 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a30ada1350 msgr2=0x55a30adb1d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:42.814 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.813+0000 7f71c3400640 1 --2- 172.21.15.72:0/2574584086 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a30ada1350 0x55a30adb1d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:42.814 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.813+0000 7f71c3400640 1 -- 172.21.15.72:0/2574584086 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a30ada0960 msgr2=0x55a30ad9f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:42.814 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.813+0000 7f71c3400640 1 --2- 172.21.15.72:0/2574584086 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a30ada0960 0x55a30ad9f8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:42.814 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.813+0000 7f71c3400640 1 -- 172.21.15.72:0/2574584086 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a30a9d60f0 con 0x55a30ada64c0 2024-04-26T21:57:42.814 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.813+0000 7f71c2a00640 1 --2- 172.21.15.72:0/2574584086 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a30ada0960 0x55a30ad9f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:57:42.814 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.813+0000 7f71c3400640 1 --2- 172.21.15.72:0/2574584086 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a30ada64c0 0x55a30ada59e0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f71bc006010 tx=0x7f71bc009f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df3b40d7d952bbe8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:57:42.817 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.814+0000 7f71b3400640 1 -- 172.21.15.72:0/2574584086 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71bc022040 con 0x55a30ada64c0 2024-04-26T21:57:42.817 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.814+0000 7f71b3400640 1 -- 172.21.15.72:0/2574584086 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f71bc01e440 con 0x55a30ada64c0 2024-04-26T21:57:42.817 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.814+0000 7f71b3400640 1 -- 172.21.15.72:0/2574584086 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71bc011d70 con 0x55a30ada64c0 2024-04-26T21:57:42.817 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.815+0000 7f71cca42740 1 -- 172.21.15.72:0/2574584086 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a30ada64c0 msgr2=0x55a30ada59e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:42.817 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.815+0000 7f71cca42740 1 --2- 172.21.15.72:0/2574584086 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a30ada64c0 0x55a30ada59e0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f71bc006010 tx=0x7f71bc009f20 comp rx=0 tx=0).stop 2024-04-26T21:57:42.817 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.815+0000 7f71cca42740 1 -- 172.21.15.72:0/2574584086 shutdown_connections 2024-04-26T21:57:42.817 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.815+0000 7f71cca42740 1 --2- 172.21.15.72:0/2574584086 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a30ada64c0 0x55a30ada59e0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:42.817 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.815+0000 7f71cca42740 1 --2- 172.21.15.72:0/2574584086 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a30ada1350 0x55a30adb1d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:42.817 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.815+0000 7f71cca42740 1 --2- 172.21.15.72:0/2574584086 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a30ada0960 0x55a30ad9f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:42.817 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.815+0000 7f71cca42740 1 -- 172.21.15.72:0/2574584086 >> 172.21.15.72:0/2574584086 conn(0x55a30ad9bfd0 msgr2=0x55a30ad9a960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:57:42.817 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.815+0000 7f71cca42740 1 -- 172.21.15.72:0/2574584086 shutdown_connections 2024-04-26T21:57:42.817 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.815+0000 7f71cca42740 1 -- 172.21.15.72:0/2574584086 wait complete. 2024-04-26T21:57:42.817 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.815+0000 7f71cca42740 1 -- start start 2024-04-26T21:57:42.818 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.816+0000 7f71cca42740 1 Processor -- start 2024-04-26T21:57:42.818 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.816+0000 7f71cca42740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a30ada0960 0x55a30adaf9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:42.818 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.816+0000 7f71cca42740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a30ada1350 0x55a30adaff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:42.818 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.816+0000 7f71cca42740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a30ada64c0 0x55a30adb0f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:42.818 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.816+0000 7f71cca42740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f71c41a9f70 con 0x55a30ada0960 2024-04-26T21:57:42.818 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.816+0000 7f71cca42740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f71c41ad400 con 0x55a30ada1350 2024-04-26T21:57:42.818 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.816+0000 7f71cca42740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f71c41aa5e0 con 0x55a30ada64c0 2024-04-26T21:57:42.818 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.816+0000 7f71cca42740 10 client.0 _ll_register_callbacks cb 0x7f71bc016070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:57:42.818 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.816+0000 7f71c2a00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a30ada0960 0x55a30adaf9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:42.818 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.816+0000 7f71c2a00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a30ada0960 0x55a30adaf9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:47934/0 (socket says 172.21.15.72:47934) 2024-04-26T21:57:42.818 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.816+0000 7f71c2a00640 1 -- 172.21.15.72:0/1044010562 learned_addr learned my addr 172.21.15.72:0/1044010562 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:57:42.818 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.817+0000 7f71bbe00640 1 --2- 172.21.15.72:0/1044010562 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a30ada1350 0x55a30adaff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:42.818 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.817+0000 7f71c2a00640 1 -- 172.21.15.72:0/1044010562 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a30ada64c0 msgr2=0x55a30adb0f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:42.819 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.817+0000 7f71c2a00640 1 --2- 172.21.15.72:0/1044010562 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a30ada64c0 0x55a30adb0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:42.819 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.817+0000 7f71c2a00640 1 -- 172.21.15.72:0/1044010562 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a30ada1350 msgr2=0x55a30adaff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:42.819 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.817+0000 7f71c2a00640 1 --2- 172.21.15.72:0/1044010562 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a30ada1350 0x55a30adaff10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:42.819 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.817+0000 7f71c2a00640 1 -- 172.21.15.72:0/1044010562 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a30accca30 con 0x55a30ada0960 2024-04-26T21:57:42.820 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.817+0000 7f71c2a00640 1 --2- 172.21.15.72:0/1044010562 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a30ada0960 0x55a30adaf9d0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f71b40058c0 tx=0x7f71b400b450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5beb5b6d81d2db09 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:57:42.820 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.817+0000 7f71b0c00640 10 client.0 ms_handle_connect on v2:172.21.15.72:3300/0 2024-04-26T21:57:42.821 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.817+0000 7f71b0c00640 1 -- 172.21.15.72:0/1044010562 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71b4015040 con 0x55a30ada0960 2024-04-26T21:57:42.821 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.818+0000 7f71cca42740 1 -- 172.21.15.72:0/1044010562 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f71bc01e440 con 0x55a30ada0960 2024-04-26T21:57:42.821 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.818+0000 7f71b0c00640 1 -- 172.21.15.72:0/1044010562 <== mon.0 v2:172.21.15.72:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7f71b4005450 con 0x55a30ada0960 2024-04-26T21:57:42.821 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.818+0000 7f71b0c00640 1 -- 172.21.15.72:0/1044010562 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f71b4005650 con 0x55a30ada0960 2024-04-26T21:57:42.821 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.818+0000 7f71cca42740 10 client.15990 fetch_fsmap learned FSMap version 27 2024-04-26T21:57:42.821 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.818+0000 7f71cca42740 1 -- 172.21.15.72:0/1044010562 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a30ae1cc40 con 0x55a30ada0960 2024-04-26T21:57:42.821 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.818+0000 7f71b0c00640 1 -- 172.21.15.72:0/1044010562 <== mon.0 v2:172.21.15.72:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f71bc01e440 con 0x55a30ada0960 2024-04-26T21:57:42.821 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.818+0000 7f71b0c00640 1 -- 172.21.15.72:0/1044010562 <== mon.0 v2:172.21.15.72:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f71b4004980 con 0x55a30ada0960 2024-04-26T21:57:42.821 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.819+0000 7f71cca42740 10 client.15990 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T21:57:42.821 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.819+0000 7f71cca42740 10 client.15990 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:57:42.821 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.819+0000 7f71cca42740 20 client.15990 populate_metadata read hostname 'smithi072' 2024-04-26T21:57:42.821 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.819+0000 7f71cca42740 1 --2- 172.21.15.72:0/1044010562 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x55a30ae1d450 0x55a30ae1f870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:42.821 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.819+0000 7f71cca42740 4 client.15990 mds_command: new command op to 34517 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-26T21:57:42.821 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.819+0000 7f71cca42740 1 -- 172.21.15.72:0/1044010562 --> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a30ad0c650 con 0x55a30ae1d450 2024-04-26T21:57:42.822 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.819+0000 7f71bbe00640 1 --2- 172.21.15.72:0/1044010562 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x55a30ae1d450 0x55a30ae1f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:42.823 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.821+0000 7f71bbe00640 1 --2- 172.21.15.72:0/1044010562 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x55a30ae1d450 0x55a30ae1f870 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=597f99e5aaa490e3 server_cookie=7fbcc99607bc1447 in_seq=0 out_seq=0 2024-04-26T21:57:42.823 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.821+0000 7f71b0c00640 10 client.15990 ms_handle_connect on v2:172.21.15.76:6838/327145636 2024-04-26T21:57:42.823 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.822+0000 7f71b0c00640 1 -- 172.21.15.72:0/1044010562 <== mds.0 v2:172.21.15.76:6838/327145636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55a30ad0c650 con 0x55a30ae1d450 2024-04-26T21:57:42.823 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.822+0000 7f71b0c00640 10 client.15990 handle_command_reply: tid=0 2024-04-26T21:57:42.823 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.822+0000 7f71cca42740 1 client.15990 shutdown 2024-04-26T21:57:42.823 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.822+0000 7f71cca42740 1 -- 172.21.15.72:0/1044010562 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a30ada0960 msgr2=0x55a30adaf9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:42.823 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.822+0000 7f71cca42740 1 --2- 172.21.15.72:0/1044010562 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a30ada0960 0x55a30adaf9d0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f71b40058c0 tx=0x7f71b400b450 comp rx=0 tx=0).stop 2024-04-26T21:57:42.824 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.822+0000 7f71cca42740 1 -- 172.21.15.72:0/1044010562 shutdown_connections 2024-04-26T21:57:42.824 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.822+0000 7f71cca42740 1 --2- 172.21.15.72:0/1044010562 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x55a30ae1d450 0x55a30ae1f870 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:42.824 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.822+0000 7f71cca42740 1 --2- 172.21.15.72:0/1044010562 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a30ada64c0 0x55a30adb0f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:42.824 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.822+0000 7f71cca42740 1 --2- 172.21.15.72:0/1044010562 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a30ada1350 0x55a30adaff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:42.824 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.822+0000 7f71cca42740 1 --2- 172.21.15.72:0/1044010562 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a30ada0960 0x55a30adaf9d0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:42.824 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.822+0000 7f71cca42740 1 -- 172.21.15.72:0/1044010562 >> 172.21.15.72:0/1044010562 conn(0x55a30ad9bfd0 msgr2=0x55a30ad9b1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:57:42.824 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.822+0000 7f71b0c00640 0 client.15990 ms_handle_reset on v2:172.21.15.76:6838/327145636 2024-04-26T21:57:42.824 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.823+0000 7f71cca42740 1 -- 172.21.15.72:0/1044010562 shutdown_connections 2024-04-26T21:57:42.824 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.823+0000 7f71cca42740 1 -- 172.21.15.72:0/1044010562 wait complete. 2024-04-26T21:57:42.824 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.823+0000 7f71cca42740 20 client.15990 trim_cache size 0 max 16384 2024-04-26T21:57:42.824 INFO:teuthology.orchestra.run.smithi072.stdout:{ 2024-04-26T21:57:42.825 INFO:teuthology.orchestra.run.smithi072.stdout: "status": "no active scrubs running", 2024-04-26T21:57:42.825 INFO:teuthology.orchestra.run.smithi072.stdout: "scrubs": {} 2024-04-26T21:57:42.825 INFO:teuthology.orchestra.run.smithi072.stdout:} 2024-04-26T21:57:42.825 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.824+0000 7f71c3e00640 1 -- 172.21.15.72:0/1217030083 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f71a80a3cc0 msgr2=0x7f71a80a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:42.825 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.824+0000 7f71c3e00640 1 --2- 172.21.15.72:0/1217030083 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f71a80a3cc0 0x7f71a80a6180 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x55a30ada1bb0 tx=0x7f71bc0094a0 comp rx=0 tx=0).stop 2024-04-26T21:57:42.826 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.824+0000 7f71c3e00640 1 -- 172.21.15.72:0/1217030083 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f71c4100e90 msgr2=0x7f71c4101f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:42.826 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.824+0000 7f71c3e00640 1 --2- 172.21.15.72:0/1217030083 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f71c4100e90 0x7f71c4101f70 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f71ac009230 tx=0x7f71ac0042f0 comp rx=0 tx=0).stop 2024-04-26T21:57:42.826 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.824+0000 7f71c3e00640 1 -- 172.21.15.72:0/1217030083 shutdown_connections 2024-04-26T21:57:42.826 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.824+0000 7f71c3e00640 1 --2- 172.21.15.72:0/1217030083 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f71c4108520 0x7f71c41024b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:42.826 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.824+0000 7f71c3e00640 1 --2- 172.21.15.72:0/1217030083 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f71c4100e90 0x7f71c4101f70 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:42.826 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.824+0000 7f71c3e00640 1 --2- 172.21.15.72:0/1217030083 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f71a80a3cc0 0x7f71a80a6180 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:42.826 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.824+0000 7f71c3e00640 1 --2- 172.21.15.72:0/1217030083 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f71c41004a0 0x7f71c4103930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:42.826 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.824+0000 7f71c3e00640 1 -- 172.21.15.72:0/1217030083 >> 172.21.15.72:0/1217030083 conn(0x7f71c40fbff0 msgr2=0x7f71c4103fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:57:42.826 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.824+0000 7f71c3e00640 1 -- 172.21.15.72:0/1217030083 shutdown_connections 2024-04-26T21:57:42.826 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:42.824+0000 7f71c3e00640 1 -- 172.21.15.72:0/1217030083 wait complete. 2024-04-26T21:57:42.843 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-26T21:57:42.843 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:57:43.044 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.044+0000 7ff477600640 1 -- 172.21.15.72:0/2726035348 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff470103b70 msgr2=0x7ff470103ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:43.044 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.044+0000 7ff477600640 1 --2- 172.21.15.72:0/2726035348 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff470103b70 0x7ff470103ff0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7ff45c009900 tx=0x7ff45c02a410 comp rx=0 tx=0).stop 2024-04-26T21:57:43.044 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.044+0000 7ff477600640 1 -- 172.21.15.72:0/2726035348 shutdown_connections 2024-04-26T21:57:43.044 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.044+0000 7ff477600640 1 --2- 172.21.15.72:0/2726035348 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff470104530 0x7ff47010adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:43.044 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.044+0000 7ff477600640 1 --2- 172.21.15.72:0/2726035348 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff470103b70 0x7ff470103ff0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:43.044 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.044+0000 7ff477600640 1 --2- 172.21.15.72:0/2726035348 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff470102980 0x7ff470102da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:43.045 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.044+0000 7ff477600640 1 -- 172.21.15.72:0/2726035348 >> 172.21.15.72:0/2726035348 conn(0x7ff4700fdf10 msgr2=0x7ff470100330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:57:43.045 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.044+0000 7ff477600640 1 -- 172.21.15.72:0/2726035348 shutdown_connections 2024-04-26T21:57:43.045 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.044+0000 7ff477600640 1 -- 172.21.15.72:0/2726035348 wait complete. 2024-04-26T21:57:43.045 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.045+0000 7ff477600640 1 Processor -- start 2024-04-26T21:57:43.045 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.045+0000 7ff477600640 1 -- start start 2024-04-26T21:57:43.046 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.046+0000 7ff477600640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff470102980 0x7ff47019ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:43.047 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.046+0000 7ff477600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff470103b70 0x7ff47019b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:43.047 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.046+0000 7ff477600640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff470104530 0x7ff4701a22a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:43.047 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.046+0000 7ff477600640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7ff47010e340 con 0x7ff470102980 2024-04-26T21:57:43.047 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.046+0000 7ff477600640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7ff47010e1c0 con 0x7ff470104530 2024-04-26T21:57:43.047 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.046+0000 7ff477600640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ff47010e4c0 con 0x7ff470103b70 2024-04-26T21:57:43.047 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.046+0000 7ff474c00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff470102980 0x7ff47019ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:43.047 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.046+0000 7ff474c00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff470102980 0x7ff47019ad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:47954/0 (socket says 172.21.15.72:47954) 2024-04-26T21:57:43.047 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.046+0000 7ff474c00640 1 -- 172.21.15.72:0/969622021 learned_addr learned my addr 172.21.15.72:0/969622021 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:57:43.047 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.046+0000 7ff475600640 1 --2- 172.21.15.72:0/969622021 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff470104530 0x7ff4701a22a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:43.047 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.046+0000 7ff474c00640 1 -- 172.21.15.72:0/969622021 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff470103b70 msgr2=0x7ff47019b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:43.047 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.046+0000 7ff474c00640 1 --2- 172.21.15.72:0/969622021 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff470103b70 0x7ff47019b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:43.047 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.046+0000 7ff474c00640 1 -- 172.21.15.72:0/969622021 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff470104530 msgr2=0x7ff4701a22a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:43.047 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.046+0000 7ff474c00640 1 --2- 172.21.15.72:0/969622021 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff470104530 0x7ff4701a22a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:43.047 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.046+0000 7ff474c00640 1 -- 172.21.15.72:0/969622021 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4701a2940 con 0x7ff470102980 2024-04-26T21:57:43.048 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.047+0000 7ff474c00640 1 --2- 172.21.15.72:0/969622021 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff470102980 0x7ff47019ad30 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7ff46400d770 tx=0x7ff46400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:57:43.048 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.047+0000 7ff46d600640 1 -- 172.21.15.72:0/969622021 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff464017070 con 0x7ff470102980 2024-04-26T21:57:43.049 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.047+0000 7ff46d600640 1 -- 172.21.15.72:0/969622021 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff464009c30 con 0x7ff470102980 2024-04-26T21:57:43.049 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.047+0000 7ff477600640 1 -- 172.21.15.72:0/969622021 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4701a2c30 con 0x7ff470102980 2024-04-26T21:57:43.049 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.047+0000 7ff46d600640 1 -- 172.21.15.72:0/969622021 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff464003c50 con 0x7ff470102980 2024-04-26T21:57:43.049 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.047+0000 7ff477600640 1 -- 172.21.15.72:0/969622021 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4701a3180 con 0x7ff470102980 2024-04-26T21:57:43.049 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.049+0000 7ff477600640 1 -- 172.21.15.72:0/969622021 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4701a3350 con 0x7ff470102980 2024-04-26T21:57:43.051 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.051+0000 7ff46d600640 1 -- 172.21.15.72:0/969622021 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7ff464003df0 con 0x7ff470102980 2024-04-26T21:57:43.053 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.053+0000 7ff46d600640 1 --2- 172.21.15.72:0/969622021 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7ff4580a3c40 0x7ff4580a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:43.053 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.053+0000 7ff46d600640 1 -- 172.21.15.72:0/969622021 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7ff4640d01b0 con 0x7ff470102980 2024-04-26T21:57:43.053 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.053+0000 7ff46fe00640 1 --2- 172.21.15.72:0/969622021 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7ff4580a3c40 0x7ff4580a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:43.054 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.054+0000 7ff46fe00640 1 --2- 172.21.15.72:0/969622021 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7ff4580a3c40 0x7ff4580a6100 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7ff45c02a930 tx=0x7ff45c000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:57:43.058 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.058+0000 7ff46d600640 1 -- 172.21.15.72:0/969622021 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7ff46409c950 con 0x7ff470102980 2024-04-26T21:57:43.306 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.306+0000 7ff477600640 1 -- 172.21.15.72:0/969622021 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff4701a35c0 con 0x7ff470102980 2024-04-26T21:57:43.307 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.307+0000 7ff46d600640 1 -- 172.21.15.72:0/969622021 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11931 (secure 0 0 0) 0x7ff464009de0 con 0x7ff470102980 2024-04-26T21:57:43.307 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T21:57:43.307 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6835/1320381959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6834","nonce":1320381959},{"type":"v1","addr":"172.21.15.72:6835","nonce":1320381959}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6837/2142200512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6836","nonce":2142200512},{"type":"v1","addr":"172.21.15.72:6837","nonce":2142200512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T21:52:50.567857+0000","modified":"2024-04-26T21:55:22.894383+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34517,"mds_1":34472,"mds_2":34442,"mds_3":34406,"mds_4":24463},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14652":{"gid":14652,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.72:6839/1312898281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6838","nonce":1312898281},{"type":"v1","addr":"172.21.15.72:6839","nonce":1312898281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14664":{"gid":14664,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.72:6841/3257692339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6840","nonce":3257692339},{"type":"v1","addr":"172.21.15.72:6841","nonce":3257692339}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24388":{"gid":24388,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6833/4236668465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":4236668465},{"type":"v1","addr":"172.21.15.164:6833","nonce":4236668465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6835/2391114169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":2391114169},{"type":"v1","addr":"172.21.15.164:6835","nonce":2391114169}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6837/3585829992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":3585829992},{"type":"v1","addr":"172.21.15.164:6837","nonce":3585829992}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.164:6839/1636226820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":1636226820},{"type":"v1","addr":"172.21.15.164:6839","nonce":1636226820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34406":{"gid":34406,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":15,"addr":"172.21.15.76:6833/3869483750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":3869483750},{"type":"v1","addr":"172.21.15.76:6833","nonce":3869483750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34442":{"gid":34442,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.76:6835/3517429435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":3517429435},{"type":"v1","addr":"172.21.15.76:6835","nonce":3517429435}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34472":{"gid":34472,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.76:6837/1150678991","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":1150678991},{"type":"v1","addr":"172.21.15.76:6837","nonce":1150678991}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34517":{"gid":34517,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.76:6839/327145636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6838","nonce":327145636},{"type":"v1","addr":"172.21.15.76:6839","nonce":327145636}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34517,"qdb_cluster":[34472,34517,34442,34406,24463]},"id":1}]} 2024-04-26T21:57:43.308 INFO:teuthology.orchestra.run.smithi072.stderr:dumped fsmap epoch 27 2024-04-26T21:57:43.309 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.309+0000 7ff477600640 1 -- 172.21.15.72:0/969622021 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7ff4580a3c40 msgr2=0x7ff4580a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:43.310 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.309+0000 7ff477600640 1 --2- 172.21.15.72:0/969622021 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7ff4580a3c40 0x7ff4580a6100 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7ff45c02a930 tx=0x7ff45c000bc0 comp rx=0 tx=0).stop 2024-04-26T21:57:43.310 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.309+0000 7ff477600640 1 -- 172.21.15.72:0/969622021 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff470102980 msgr2=0x7ff47019ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:43.310 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.309+0000 7ff477600640 1 --2- 172.21.15.72:0/969622021 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff470102980 0x7ff47019ad30 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7ff46400d770 tx=0x7ff46400dc30 comp rx=0 tx=0).stop 2024-04-26T21:57:43.310 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.310+0000 7ff477600640 1 -- 172.21.15.72:0/969622021 shutdown_connections 2024-04-26T21:57:43.310 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.310+0000 7ff477600640 1 --2- 172.21.15.72:0/969622021 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff470104530 0x7ff4701a22a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:43.310 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.310+0000 7ff477600640 1 --2- 172.21.15.72:0/969622021 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7ff4580a3c40 0x7ff4580a6100 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:43.310 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.310+0000 7ff477600640 1 --2- 172.21.15.72:0/969622021 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff470103b70 0x7ff47019b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:43.310 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.310+0000 7ff477600640 1 --2- 172.21.15.72:0/969622021 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff470102980 0x7ff47019ad30 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:43.310 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.310+0000 7ff477600640 1 -- 172.21.15.72:0/969622021 >> 172.21.15.72:0/969622021 conn(0x7ff4700fdf10 msgr2=0x7ff470109360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:57:43.311 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.310+0000 7ff477600640 1 -- 172.21.15.72:0/969622021 shutdown_connections 2024-04-26T21:57:43.311 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.311+0000 7ff477600640 1 -- 172.21.15.72:0/969622021 wait complete. 2024-04-26T21:57:43.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:57:42 smithi076 ceph-mon[31823]: pgmap v517: 97 pgs: 97 active+clean; 201 MiB data, 984 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 6.3 MiB/s wr, 10 op/s 2024-04-26T21:57:43.321 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:57:42 smithi072 ceph-mon[28138]: pgmap v517: 97 pgs: 97 active+clean; 201 MiB data, 984 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 6.3 MiB/s wr, 10 op/s 2024-04-26T21:57:43.332 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-26T21:57:43.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:57:42 smithi164 ceph-mon[31916]: pgmap v517: 97 pgs: 97 active+clean; 201 MiB data, 984 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 6.3 MiB/s wr, 10 op/s 2024-04-26T21:57:43.536 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.535+0000 7f5befe00640 1 -- 172.21.15.72:0/753063150 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5bf0104410 msgr2=0x7f5bf010aca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:43.536 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.535+0000 7f5befe00640 1 --2- 172.21.15.72:0/753063150 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5bf0104410 0x7f5bf010aca0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f5be8009ca0 tx=0x7f5be802a8c0 comp rx=0 tx=0).stop 2024-04-26T21:57:43.536 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.535+0000 7f5befe00640 1 -- 172.21.15.72:0/753063150 shutdown_connections 2024-04-26T21:57:43.536 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.535+0000 7f5befe00640 1 --2- 172.21.15.72:0/753063150 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5bf0104410 0x7f5bf010aca0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:43.536 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.535+0000 7f5befe00640 1 --2- 172.21.15.72:0/753063150 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5bf0103a50 0x7f5bf0103ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:43.536 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.535+0000 7f5befe00640 1 --2- 172.21.15.72:0/753063150 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5bf0102860 0x7f5bf0102c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:43.536 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.535+0000 7f5befe00640 1 -- 172.21.15.72:0/753063150 >> 172.21.15.72:0/753063150 conn(0x7f5bf00fddd0 msgr2=0x7f5bf0100210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:57:43.536 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.535+0000 7f5befe00640 1 -- 172.21.15.72:0/753063150 shutdown_connections 2024-04-26T21:57:43.536 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.535+0000 7f5befe00640 1 -- 172.21.15.72:0/753063150 wait complete. 2024-04-26T21:57:43.537 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.537+0000 7f5befe00640 1 Processor -- start 2024-04-26T21:57:43.537 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.537+0000 7f5befe00640 1 -- start start 2024-04-26T21:57:43.537 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.537+0000 7f5befe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5bf0102860 0x7f5bf019ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:43.538 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.537+0000 7f5befe00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5bf0103a50 0x7f5bf019b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:43.538 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.537+0000 7f5befe00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5bf0104410 0x7f5bf01a20e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:43.538 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.537+0000 7f5befe00640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f5bf010e270 con 0x7f5bf0104410 2024-04-26T21:57:43.538 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.537+0000 7f5befe00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f5bf010e0f0 con 0x7f5bf0103a50 2024-04-26T21:57:43.538 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.537+0000 7f5befe00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f5bf010e3f0 con 0x7f5bf0102860 2024-04-26T21:57:43.538 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.537+0000 7f5bef400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5bf0104410 0x7f5bf01a20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:43.538 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.537+0000 7f5bef400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5bf0104410 0x7f5bf01a20e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:47966/0 (socket says 172.21.15.72:47966) 2024-04-26T21:57:43.538 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.538+0000 7f5bef400640 1 -- 172.21.15.72:0/1773397545 learned_addr learned my addr 172.21.15.72:0/1773397545 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:57:43.539 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.538+0000 7f5bee000640 1 --2- 172.21.15.72:0/1773397545 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5bf0103a50 0x7f5bf019b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:43.539 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.538+0000 7f5beea00640 1 --2- 172.21.15.72:0/1773397545 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5bf0102860 0x7f5bf019ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:43.539 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.538+0000 7f5bef400640 1 -- 172.21.15.72:0/1773397545 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5bf0102860 msgr2=0x7f5bf019ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:43.540 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.538+0000 7f5bef400640 1 --2- 172.21.15.72:0/1773397545 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5bf0102860 0x7f5bf019ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:43.540 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.538+0000 7f5bef400640 1 -- 172.21.15.72:0/1773397545 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5bf0103a50 msgr2=0x7f5bf019b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:43.540 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.538+0000 7f5bef400640 1 --2- 172.21.15.72:0/1773397545 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5bf0103a50 0x7f5bf019b0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:43.540 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.538+0000 7f5bef400640 1 -- 172.21.15.72:0/1773397545 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bf01a2750 con 0x7f5bf0104410 2024-04-26T21:57:43.540 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.538+0000 7f5bef400640 1 --2- 172.21.15.72:0/1773397545 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5bf0104410 0x7f5bf01a20e0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f5be8031040 tx=0x7f5be8003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:57:43.540 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.539+0000 7f5be7400640 1 -- 172.21.15.72:0/1773397545 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5be8003d40 con 0x7f5bf0104410 2024-04-26T21:57:43.540 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.539+0000 7f5befe00640 1 -- 172.21.15.72:0/1773397545 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5bf01a29e0 con 0x7f5bf0104410 2024-04-26T21:57:43.540 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.539+0000 7f5be7400640 1 -- 172.21.15.72:0/1773397545 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5be80044d0 con 0x7f5bf0104410 2024-04-26T21:57:43.540 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.539+0000 7f5be7400640 1 -- 172.21.15.72:0/1773397545 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5be8030ba0 con 0x7f5bf0104410 2024-04-26T21:57:43.540 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.539+0000 7f5befe00640 1 -- 172.21.15.72:0/1773397545 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5bf01a2f80 con 0x7f5bf0104410 2024-04-26T21:57:43.543 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.542+0000 7f5be7400640 1 -- 172.21.15.72:0/1773397545 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f5be8004680 con 0x7f5bf0104410 2024-04-26T21:57:43.543 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.542+0000 7f5bf89c0740 1 Processor -- start 2024-04-26T21:57:43.543 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.543+0000 7f5bf89c0740 1 -- start start 2024-04-26T21:57:43.543 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.543+0000 7f5bf89c0740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561a398cc050 0x561a398cc470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:43.543 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.543+0000 7f5bf89c0740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561a398cca40 0x561a398d1540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:43.543 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.543+0000 7f5bf89c0740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561a398d1bb0 0x561a398d4040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:43.543 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.543+0000 7f5bf89c0740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x561a3982fc70 con 0x561a398cca40 2024-04-26T21:57:43.543 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.543+0000 7f5bf89c0740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x561a397e8980 con 0x561a398cc050 2024-04-26T21:57:43.543 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.543+0000 7f5bf89c0740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x561a3982bc30 con 0x561a398d1bb0 2024-04-26T21:57:43.545 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.543+0000 7f5beea00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561a398cca40 0x561a398d1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:43.545 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.543+0000 7f5beea00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561a398cca40 0x561a398d1540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:47976/0 (socket says 172.21.15.72:47976) 2024-04-26T21:57:43.545 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.543+0000 7f5beea00640 1 -- 172.21.15.72:0/315185026 learned_addr learned my addr 172.21.15.72:0/315185026 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:57:43.545 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.544+0000 7f5bee000640 1 --2- 172.21.15.72:0/315185026 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561a398cc050 0x561a398cc470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:43.545 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.544+0000 7f5bee000640 1 --2- 172.21.15.72:0/315185026 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561a398d1bb0 0x561a398d4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:43.545 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.544+0000 7f5beea00640 1 -- 172.21.15.72:0/315185026 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561a398d1bb0 msgr2=0x561a398d4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:43.545 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.544+0000 7f5beea00640 1 --2- 172.21.15.72:0/315185026 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561a398d1bb0 0x561a398d4040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:43.545 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.544+0000 7f5beea00640 1 -- 172.21.15.72:0/315185026 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561a398cc050 msgr2=0x561a398cc470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:43.545 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.544+0000 7f5beea00640 1 --2- 172.21.15.72:0/315185026 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561a398cc050 0x561a398cc470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:43.545 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.544+0000 7f5beea00640 1 -- 172.21.15.72:0/315185026 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a397c07b0 con 0x561a398cca40 2024-04-26T21:57:43.546 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.544+0000 7f5bee000640 1 --2- 172.21.15.72:0/315185026 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561a398d1bb0 0x561a398d4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:57:43.546 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.544+0000 7f5beea00640 1 --2- 172.21.15.72:0/315185026 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561a398cca40 0x561a398d1540 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f5be0009c20 tx=0x7f5be00117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1afcb87a1f74603f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:57:43.546 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.544+0000 7f5be7400640 1 --2- 172.21.15.72:0/1773397545 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f5bd40a3c40 0x7f5bd40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:43.546 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.544+0000 7f5bdbe00640 1 -- 172.21.15.72:0/315185026 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5be0012450 con 0x561a398cca40 2024-04-26T21:57:43.546 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.544+0000 7f5be7400640 1 -- 172.21.15.72:0/1773397545 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7f5be80f19d0 con 0x7f5bf0104410 2024-04-26T21:57:43.546 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.545+0000 7f5bee000640 1 --2- 172.21.15.72:0/1773397545 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f5bd40a3c40 0x7f5bd40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:43.546 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.545+0000 7f5bdbe00640 1 -- 172.21.15.72:0/315185026 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5be0012a80 con 0x561a398cca40 2024-04-26T21:57:43.546 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.545+0000 7f5bdbe00640 1 -- 172.21.15.72:0/315185026 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5be000f990 con 0x561a398cca40 2024-04-26T21:57:43.549 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.546+0000 7f5bf89c0740 1 -- 172.21.15.72:0/315185026 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561a398cca40 msgr2=0x561a398d1540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:43.549 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.546+0000 7f5bee000640 1 --2- 172.21.15.72:0/1773397545 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f5bd40a3c40 0x7f5bd40a6100 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x561a398cd2a0 tx=0x7f5bdc004730 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:57:43.549 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.546+0000 7f5bf89c0740 1 --2- 172.21.15.72:0/315185026 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561a398cca40 0x561a398d1540 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f5be0009c20 tx=0x7f5be00117b0 comp rx=0 tx=0).stop 2024-04-26T21:57:43.549 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.546+0000 7f5bf89c0740 1 -- 172.21.15.72:0/315185026 shutdown_connections 2024-04-26T21:57:43.549 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.546+0000 7f5bf89c0740 1 --2- 172.21.15.72:0/315185026 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561a398d1bb0 0x561a398d4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:43.549 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.546+0000 7f5bf89c0740 1 --2- 172.21.15.72:0/315185026 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561a398cca40 0x561a398d1540 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:43.549 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.546+0000 7f5bf89c0740 1 --2- 172.21.15.72:0/315185026 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561a398cc050 0x561a398cc470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:43.549 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.546+0000 7f5bf89c0740 1 -- 172.21.15.72:0/315185026 >> 172.21.15.72:0/315185026 conn(0x561a398c76c0 msgr2=0x561a398c7ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:57:43.549 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.546+0000 7f5bf89c0740 1 -- 172.21.15.72:0/315185026 shutdown_connections 2024-04-26T21:57:43.549 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.546+0000 7f5bf89c0740 1 -- 172.21.15.72:0/315185026 wait complete. 2024-04-26T21:57:43.549 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.546+0000 7f5bf89c0740 1 -- start start 2024-04-26T21:57:43.549 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.547+0000 7f5bf89c0740 1 Processor -- start 2024-04-26T21:57:43.549 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.547+0000 7f5bf89c0740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561a398cc050 0x561a398d5f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:43.550 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.547+0000 7f5bf89c0740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561a398cca40 0x561a398d64a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:43.550 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.547+0000 7f5bf89c0740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561a398d1bb0 0x561a398dd4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:43.550 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.547+0000 7f5bf89c0740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x561a3982ef80 con 0x561a398cca40 2024-04-26T21:57:43.550 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.547+0000 7f5bf89c0740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x561a397dc260 con 0x561a398d1bb0 2024-04-26T21:57:43.550 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.547+0000 7f5bf89c0740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x561a398342a0 con 0x561a398cc050 2024-04-26T21:57:43.550 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.547+0000 7f5bf89c0740 10 client.0 _ll_register_callbacks cb 0x561a39826960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:57:43.550 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.547+0000 7f5bee000640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561a398cca40 0x561a398d64a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:43.550 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.548+0000 7f5bee000640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561a398cca40 0x561a398d64a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:47982/0 (socket says 172.21.15.72:47982) 2024-04-26T21:57:43.550 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.548+0000 7f5bee000640 1 -- 172.21.15.72:0/641905920 learned_addr learned my addr 172.21.15.72:0/641905920 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:57:43.550 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.548+0000 7f5bef400640 1 --2- 172.21.15.72:0/641905920 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561a398d1bb0 0x561a398dd4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:43.550 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.548+0000 7f5beea00640 1 --2- 172.21.15.72:0/641905920 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561a398cc050 0x561a398d5f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:43.550 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.548+0000 7f5bee000640 1 -- 172.21.15.72:0/641905920 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561a398cc050 msgr2=0x561a398d5f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:43.550 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.548+0000 7f5bee000640 1 --2- 172.21.15.72:0/641905920 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561a398cc050 0x561a398d5f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:43.551 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.548+0000 7f5bee000640 1 -- 172.21.15.72:0/641905920 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561a398d1bb0 msgr2=0x561a398dd4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:43.551 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.548+0000 7f5bee000640 1 --2- 172.21.15.72:0/641905920 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561a398d1bb0 0x561a398dd4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:43.551 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.548+0000 7f5bee000640 1 -- 172.21.15.72:0/641905920 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a397be1c0 con 0x561a398cca40 2024-04-26T21:57:43.551 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.548+0000 7f5bee000640 1 --2- 172.21.15.72:0/641905920 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561a398cca40 0x561a398d64a0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f5bdc01c660 tx=0x7f5bdc005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f6ff36c3feeffe4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:57:43.551 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.548+0000 7f5bd8c00640 10 client.0 ms_handle_connect on v2:172.21.15.72:3300/0 2024-04-26T21:57:43.551 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.549+0000 7f5bd8c00640 1 -- 172.21.15.72:0/641905920 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bdc02c060 con 0x561a398cca40 2024-04-26T21:57:43.551 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.549+0000 7f5bd8c00640 1 -- 172.21.15.72:0/641905920 <== mon.0 v2:172.21.15.72:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7f5bdc016030 con 0x561a398cca40 2024-04-26T21:57:43.551 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.549+0000 7f5bf89c0740 1 -- 172.21.15.72:0/641905920 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5be0012a80 con 0x561a398cca40 2024-04-26T21:57:43.552 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.549+0000 7f5bd8c00640 1 -- 172.21.15.72:0/641905920 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f5bdc01cd10 con 0x561a398cca40 2024-04-26T21:57:43.552 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.549+0000 7f5bf89c0740 10 client.16008 fetch_fsmap learned FSMap version 27 2024-04-26T21:57:43.552 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.549+0000 7f5bf89c0740 1 -- 172.21.15.72:0/641905920 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x561a395413f0 con 0x561a398cca40 2024-04-26T21:57:43.552 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.550+0000 7f5bd8c00640 1 -- 172.21.15.72:0/641905920 <== mon.0 v2:172.21.15.72:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5be0012a80 con 0x561a398cca40 2024-04-26T21:57:43.552 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.550+0000 7f5bd8c00640 1 -- 172.21.15.72:0/641905920 <== mon.0 v2:172.21.15.72:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f5bdc00f420 con 0x561a398cca40 2024-04-26T21:57:43.553 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.550+0000 7f5bf89c0740 10 client.16008 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T21:57:43.553 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.550+0000 7f5bf89c0740 10 client.16008 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-04-26T21:57:43.553 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.550+0000 7f5bf89c0740 20 client.16008 populate_metadata read hostname 'smithi072' 2024-04-26T21:57:43.553 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.550+0000 7f5bf89c0740 1 --2- 172.21.15.72:0/641905920 >> [v2:172.21.15.164:6838/1636226820,v1:172.21.15.164:6839/1636226820] conn(0x561a398e1af0 0x561a398e3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:43.553 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.550+0000 7f5bf89c0740 4 client.16008 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:57:43.553 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.550+0000 7f5bf89c0740 1 -- 172.21.15.72:0/641905920 --> [v2:172.21.15.164:6838/1636226820,v1:172.21.15.164:6839/1636226820] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561a3983d880 con 0x561a398e1af0 2024-04-26T21:57:43.553 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.551+0000 7f5beea00640 1 --2- 172.21.15.72:0/641905920 >> [v2:172.21.15.164:6838/1636226820,v1:172.21.15.164:6839/1636226820] conn(0x561a398e1af0 0x561a398e3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:43.553 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.552+0000 7f5beea00640 1 --2- 172.21.15.72:0/641905920 >> [v2:172.21.15.164:6838/1636226820,v1:172.21.15.164:6839/1636226820] conn(0x561a398e1af0 0x561a398e3f20 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=407c485e3cf64dea server_cookie=fd9c6cf35afbef2a in_seq=0 out_seq=0 2024-04-26T21:57:43.553 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.552+0000 7f5bd8c00640 10 client.16008 ms_handle_connect on v2:172.21.15.164:6838/1636226820 2024-04-26T21:57:43.554 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.553+0000 7f5bd8c00640 1 -- 172.21.15.72:0/641905920 <== mds.4 v2:172.21.15.164:6838/1636226820 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561a3983d880 con 0x561a398e1af0 2024-04-26T21:57:43.554 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.553+0000 7f5bd8c00640 10 client.16008 handle_command_reply: tid=0 2024-04-26T21:57:43.555 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.553+0000 7f5bf89c0740 1 client.16008 shutdown 2024-04-26T21:57:43.555 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.554+0000 7f5bf89c0740 1 -- 172.21.15.72:0/641905920 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561a398cca40 msgr2=0x561a398d64a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:43.555 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.554+0000 7f5bf89c0740 1 --2- 172.21.15.72:0/641905920 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561a398cca40 0x561a398d64a0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f5bdc01c660 tx=0x7f5bdc005f90 comp rx=0 tx=0).stop 2024-04-26T21:57:43.555 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.554+0000 7f5bf89c0740 1 -- 172.21.15.72:0/641905920 shutdown_connections 2024-04-26T21:57:43.555 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.554+0000 7f5bf89c0740 1 --2- 172.21.15.72:0/641905920 >> [v2:172.21.15.164:6838/1636226820,v1:172.21.15.164:6839/1636226820] conn(0x561a398e1af0 0x561a398e3f20 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:43.555 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.554+0000 7f5bf89c0740 1 --2- 172.21.15.72:0/641905920 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561a398d1bb0 0x561a398dd4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:43.555 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.554+0000 7f5bf89c0740 1 --2- 172.21.15.72:0/641905920 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561a398cca40 0x561a398d64a0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:43.555 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.554+0000 7f5bd8c00640 0 client.16008 ms_handle_reset on v2:172.21.15.164:6838/1636226820 2024-04-26T21:57:43.555 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.554+0000 7f5bf89c0740 1 --2- 172.21.15.72:0/641905920 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561a398cc050 0x561a398d5f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:43.555 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.554+0000 7f5bf89c0740 1 -- 172.21.15.72:0/641905920 >> 172.21.15.72:0/641905920 conn(0x561a398c76c0 msgr2=0x561a398c67b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:57:43.555 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.554+0000 7f5bf89c0740 1 -- 172.21.15.72:0/641905920 shutdown_connections 2024-04-26T21:57:43.555 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.554+0000 7f5bf89c0740 1 -- 172.21.15.72:0/641905920 wait complete. 2024-04-26T21:57:43.555 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.554+0000 7f5bf89c0740 20 client.16008 trim_cache size 0 max 16384 2024-04-26T21:57:43.576 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.574+0000 7f5bf89c0740 1 Processor -- start 2024-04-26T21:57:43.576 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.575+0000 7f5bf89c0740 1 -- start start 2024-04-26T21:57:43.576 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.575+0000 7f5bf89c0740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561a398cc050 0x561a398d1230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:43.576 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.575+0000 7f5bf89c0740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561a398cca40 0x561a398dd4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:43.576 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.575+0000 7f5bf89c0740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561a398d1bb0 0x561a398cd4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:43.576 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.575+0000 7f5bf89c0740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f5bf01abf00 con 0x561a398cc050 2024-04-26T21:57:43.576 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.575+0000 7f5bf89c0740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f5bf01ada40 con 0x561a398cca40 2024-04-26T21:57:43.576 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.575+0000 7f5bf89c0740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f5bf01ac5d0 con 0x561a398d1bb0 2024-04-26T21:57:43.576 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.575+0000 7f5beea00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561a398cc050 0x561a398d1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:43.576 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.575+0000 7f5beea00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561a398cc050 0x561a398d1230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:47984/0 (socket says 172.21.15.72:47984) 2024-04-26T21:57:43.577 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.575+0000 7f5beea00640 1 -- 172.21.15.72:0/3460417428 learned_addr learned my addr 172.21.15.72:0/3460417428 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:57:43.577 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.575+0000 7f5bee000640 1 --2- 172.21.15.72:0/3460417428 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561a398cca40 0x561a398dd4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:43.577 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.575+0000 7f5bef400640 1 --2- 172.21.15.72:0/3460417428 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561a398d1bb0 0x561a398cd4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:43.577 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.576+0000 7f5beea00640 1 -- 172.21.15.72:0/3460417428 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561a398d1bb0 msgr2=0x561a398cd4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:43.577 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.576+0000 7f5beea00640 1 --2- 172.21.15.72:0/3460417428 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561a398d1bb0 0x561a398cd4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:43.577 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.576+0000 7f5beea00640 1 -- 172.21.15.72:0/3460417428 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561a398cca40 msgr2=0x561a398dd4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:43.577 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.576+0000 7f5beea00640 1 --2- 172.21.15.72:0/3460417428 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561a398cca40 0x561a398dd4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:43.577 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.576+0000 7f5beea00640 1 -- 172.21.15.72:0/3460417428 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a3960b7d0 con 0x561a398cc050 2024-04-26T21:57:43.577 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.576+0000 7f5beea00640 1 --2- 172.21.15.72:0/3460417428 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561a398cc050 0x561a398d1230 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f5be0004600 tx=0x7f5be0009d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7718eedd2f7226f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:57:43.580 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.576+0000 7f5bdb400640 1 -- 172.21.15.72:0/3460417428 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5be0030070 con 0x561a398cc050 2024-04-26T21:57:43.581 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.576+0000 7f5bdb400640 1 -- 172.21.15.72:0/3460417428 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5be00129e0 con 0x561a398cc050 2024-04-26T21:57:43.581 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.576+0000 7f5bdb400640 1 -- 172.21.15.72:0/3460417428 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5be0004a00 con 0x561a398cc050 2024-04-26T21:57:43.581 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.577+0000 7f5bf89c0740 1 -- 172.21.15.72:0/3460417428 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561a398cc050 msgr2=0x561a398d1230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:43.581 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.577+0000 7f5bf89c0740 1 --2- 172.21.15.72:0/3460417428 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561a398cc050 0x561a398d1230 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f5be0004600 tx=0x7f5be0009d10 comp rx=0 tx=0).stop 2024-04-26T21:57:43.581 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.577+0000 7f5bf89c0740 1 -- 172.21.15.72:0/3460417428 shutdown_connections 2024-04-26T21:57:43.581 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.577+0000 7f5bf89c0740 1 --2- 172.21.15.72:0/3460417428 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561a398d1bb0 0x561a398cd4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:43.581 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.577+0000 7f5bf89c0740 1 --2- 172.21.15.72:0/3460417428 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561a398cca40 0x561a398dd4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:43.581 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.577+0000 7f5bf89c0740 1 --2- 172.21.15.72:0/3460417428 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561a398cc050 0x561a398d1230 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:43.581 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.577+0000 7f5bf89c0740 1 -- 172.21.15.72:0/3460417428 >> 172.21.15.72:0/3460417428 conn(0x561a398c76c0 msgr2=0x561a398cae60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:57:43.581 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.577+0000 7f5bf89c0740 1 -- 172.21.15.72:0/3460417428 shutdown_connections 2024-04-26T21:57:43.581 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.577+0000 7f5bf89c0740 1 -- 172.21.15.72:0/3460417428 wait complete. 2024-04-26T21:57:43.581 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.578+0000 7f5bf89c0740 1 -- start start 2024-04-26T21:57:43.581 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.578+0000 7f5bf89c0740 1 Processor -- start 2024-04-26T21:57:43.581 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.578+0000 7f5bf89c0740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561a398cc050 0x561a398dae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:43.582 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.578+0000 7f5bf89c0740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561a398cca40 0x561a398db340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:43.582 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.578+0000 7f5bf89c0740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561a398d1bb0 0x561a398dc350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:43.582 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.578+0000 7f5bf89c0740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f5bf01ad370 con 0x561a398cc050 2024-04-26T21:57:43.582 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.578+0000 7f5bf89c0740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f5bf01acca0 con 0x561a398cca40 2024-04-26T21:57:43.582 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.578+0000 7f5bf89c0740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f5bf01a9ee0 con 0x561a398d1bb0 2024-04-26T21:57:43.582 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.578+0000 7f5bf89c0740 10 client.0 _ll_register_callbacks cb 0x7f5bdc009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:57:43.582 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.579+0000 7f5beea00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561a398cc050 0x561a398dae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:43.582 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.579+0000 7f5beea00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561a398cc050 0x561a398dae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:47990/0 (socket says 172.21.15.72:47990) 2024-04-26T21:57:43.582 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.579+0000 7f5beea00640 1 -- 172.21.15.72:0/895447916 learned_addr learned my addr 172.21.15.72:0/895447916 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:57:43.582 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.579+0000 7f5bee000640 1 --2- 172.21.15.72:0/895447916 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561a398cca40 0x561a398db340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:43.582 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.579+0000 7f5bef400640 1 --2- 172.21.15.72:0/895447916 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561a398d1bb0 0x561a398dc350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:43.582 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.579+0000 7f5beea00640 1 -- 172.21.15.72:0/895447916 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561a398d1bb0 msgr2=0x561a398dc350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:43.582 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.579+0000 7f5beea00640 1 --2- 172.21.15.72:0/895447916 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561a398d1bb0 0x561a398dc350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:43.582 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.579+0000 7f5beea00640 1 -- 172.21.15.72:0/895447916 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561a398cca40 msgr2=0x561a398db340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:43.583 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.579+0000 7f5beea00640 1 --2- 172.21.15.72:0/895447916 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561a398cca40 0x561a398db340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:43.583 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.579+0000 7f5beea00640 1 -- 172.21.15.72:0/895447916 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a397d62a0 con 0x561a398cc050 2024-04-26T21:57:43.583 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.580+0000 7f5beea00640 1 --2- 172.21.15.72:0/895447916 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561a398cc050 0x561a398dae00 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f5be001c040 tx=0x7f5be00092e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a3933fb1a237839 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:57:43.583 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.580+0000 7f5bd8c00640 10 client.0 ms_handle_connect on v2:172.21.15.72:3300/0 2024-04-26T21:57:43.583 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.580+0000 7f5bd8c00640 1 -- 172.21.15.72:0/895447916 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5be0030070 con 0x561a398cc050 2024-04-26T21:57:43.583 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.580+0000 7f5bd8c00640 1 -- 172.21.15.72:0/895447916 <== mon.0 v2:172.21.15.72:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7f5be0024810 con 0x561a398cc050 2024-04-26T21:57:43.583 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.580+0000 7f5bf89c0740 1 -- 172.21.15.72:0/895447916 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5be00129e0 con 0x561a398cc050 2024-04-26T21:57:43.583 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.580+0000 7f5bd8c00640 1 -- 172.21.15.72:0/895447916 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f5be0024a10 con 0x561a398cc050 2024-04-26T21:57:43.583 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.580+0000 7f5bf89c0740 10 client.16014 fetch_fsmap learned FSMap version 27 2024-04-26T21:57:43.583 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.580+0000 7f5bf89c0740 1 -- 172.21.15.72:0/895447916 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x561a39948160 con 0x561a398cc050 2024-04-26T21:57:43.583 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.580+0000 7f5bd8c00640 1 -- 172.21.15.72:0/895447916 <== mon.0 v2:172.21.15.72:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5be00129e0 con 0x561a398cc050 2024-04-26T21:57:43.583 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.581+0000 7f5bd8c00640 1 -- 172.21.15.72:0/895447916 <== mon.0 v2:172.21.15.72:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f5be002b040 con 0x561a398cc050 2024-04-26T21:57:43.583 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.581+0000 7f5bf89c0740 10 client.16014 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T21:57:43.583 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.581+0000 7f5bf89c0740 10 client.16014 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-04-26T21:57:43.584 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.581+0000 7f5bf89c0740 20 client.16014 populate_metadata read hostname 'smithi072' 2024-04-26T21:57:43.584 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.581+0000 7f5bf89c0740 1 --2- 172.21.15.72:0/895447916 >> [v2:172.21.15.164:6838/1636226820,v1:172.21.15.164:6839/1636226820] conn(0x561a39948900 0x561a3994acf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:43.584 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.581+0000 7f5bf89c0740 4 client.16014 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-26T21:57:43.584 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.581+0000 7f5bf89c0740 1 -- 172.21.15.72:0/895447916 --> [v2:172.21.15.164:6838/1636226820,v1:172.21.15.164:6839/1636226820] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561a398376f0 con 0x561a39948900 2024-04-26T21:57:43.585 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.581+0000 7f5bee000640 1 --2- 172.21.15.72:0/895447916 >> [v2:172.21.15.164:6838/1636226820,v1:172.21.15.164:6839/1636226820] conn(0x561a39948900 0x561a3994acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:43.585 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.582+0000 7f5bee000640 1 --2- 172.21.15.72:0/895447916 >> [v2:172.21.15.164:6838/1636226820,v1:172.21.15.164:6839/1636226820] conn(0x561a39948900 0x561a3994acf0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ce8a8dd11aa255c3 server_cookie=a21bf0cb284e2f27 in_seq=0 out_seq=0 2024-04-26T21:57:43.585 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.582+0000 7f5bd8c00640 10 client.16014 ms_handle_connect on v2:172.21.15.164:6838/1636226820 2024-04-26T21:57:43.585 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.583+0000 7f5bd8c00640 1 -- 172.21.15.72:0/895447916 <== mds.4 v2:172.21.15.164:6838/1636226820 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561a398376f0 con 0x561a39948900 2024-04-26T21:57:43.585 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.583+0000 7f5bd8c00640 10 client.16014 handle_command_reply: tid=0 2024-04-26T21:57:43.585 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.583+0000 7f5bf89c0740 1 client.16014 shutdown 2024-04-26T21:57:43.586 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.583+0000 7f5bf89c0740 1 -- 172.21.15.72:0/895447916 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561a398cc050 msgr2=0x561a398dae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:43.586 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.583+0000 7f5bf89c0740 1 --2- 172.21.15.72:0/895447916 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561a398cc050 0x561a398dae00 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f5be001c040 tx=0x7f5be00092e0 comp rx=0 tx=0).stop 2024-04-26T21:57:43.586 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.583+0000 7f5bf89c0740 1 -- 172.21.15.72:0/895447916 shutdown_connections 2024-04-26T21:57:43.586 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.583+0000 7f5bf89c0740 1 --2- 172.21.15.72:0/895447916 >> [v2:172.21.15.164:6838/1636226820,v1:172.21.15.164:6839/1636226820] conn(0x561a39948900 0x561a3994acf0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:43.586 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.583+0000 7f5bf89c0740 1 --2- 172.21.15.72:0/895447916 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561a398d1bb0 0x561a398dc350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:43.586 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.583+0000 7f5bf89c0740 1 --2- 172.21.15.72:0/895447916 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561a398cca40 0x561a398db340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:43.586 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.583+0000 7f5bf89c0740 1 --2- 172.21.15.72:0/895447916 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561a398cc050 0x561a398dae00 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:43.586 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.583+0000 7f5bd8c00640 0 client.16014 ms_handle_reset on v2:172.21.15.164:6838/1636226820 2024-04-26T21:57:43.586 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.583+0000 7f5bf89c0740 1 -- 172.21.15.72:0/895447916 >> 172.21.15.72:0/895447916 conn(0x561a398c76c0 msgr2=0x561a398c5f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:57:43.586 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.583+0000 7f5bf89c0740 1 -- 172.21.15.72:0/895447916 shutdown_connections 2024-04-26T21:57:43.586 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.583+0000 7f5bf89c0740 1 -- 172.21.15.72:0/895447916 wait complete. 2024-04-26T21:57:43.586 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.583+0000 7f5bf89c0740 20 client.16014 trim_cache size 0 max 16384 2024-04-26T21:57:43.587 INFO:teuthology.orchestra.run.smithi072.stdout:[] 2024-04-26T21:57:43.587 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.584+0000 7f5befe00640 1 -- 172.21.15.72:0/1773397545 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f5bd40a3c40 msgr2=0x7f5bd40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:43.587 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.584+0000 7f5befe00640 1 --2- 172.21.15.72:0/1773397545 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f5bd40a3c40 0x7f5bd40a6100 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x561a398cd2a0 tx=0x7f5bdc004730 comp rx=0 tx=0).stop 2024-04-26T21:57:43.587 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.584+0000 7f5befe00640 1 -- 172.21.15.72:0/1773397545 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5bf0104410 msgr2=0x7f5bf01a20e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:43.588 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.584+0000 7f5befe00640 1 --2- 172.21.15.72:0/1773397545 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5bf0104410 0x7f5bf01a20e0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f5be8031040 tx=0x7f5be8003ba0 comp rx=0 tx=0).stop 2024-04-26T21:57:43.588 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.584+0000 7f5befe00640 1 -- 172.21.15.72:0/1773397545 shutdown_connections 2024-04-26T21:57:43.588 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.584+0000 7f5befe00640 1 --2- 172.21.15.72:0/1773397545 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5bf0104410 0x7f5bf01a20e0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:43.588 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.584+0000 7f5befe00640 1 --2- 172.21.15.72:0/1773397545 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5bf0103a50 0x7f5bf019b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:43.588 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.584+0000 7f5befe00640 1 --2- 172.21.15.72:0/1773397545 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f5bd40a3c40 0x7f5bd40a6100 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:43.588 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.584+0000 7f5befe00640 1 --2- 172.21.15.72:0/1773397545 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5bf0102860 0x7f5bf019ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:43.588 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.584+0000 7f5befe00640 1 -- 172.21.15.72:0/1773397545 >> 172.21.15.72:0/1773397545 conn(0x7f5bf00fddd0 msgr2=0x7f5bf0074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:57:43.588 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.584+0000 7f5befe00640 1 -- 172.21.15.72:0/1773397545 shutdown_connections 2024-04-26T21:57:43.588 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.584+0000 7f5befe00640 1 -- 172.21.15.72:0/1773397545 wait complete. 2024-04-26T21:57:43.610 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-26T21:57:43.838 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.837+0000 7f2d13e00640 1 -- 172.21.15.72:0/3821436413 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2d14105c50 msgr2=0x7f2d14108040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:43.838 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.837+0000 7f2d13e00640 1 --2- 172.21.15.72:0/3821436413 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2d14105c50 0x7f2d14108040 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f2d00009900 tx=0x7f2d0002a410 comp rx=0 tx=0).stop 2024-04-26T21:57:43.838 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.838+0000 7f2d13e00640 1 -- 172.21.15.72:0/3821436413 shutdown_connections 2024-04-26T21:57:43.838 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.838+0000 7f2d13e00640 1 --2- 172.21.15.72:0/3821436413 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2d14108580 0x7f2d1410a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:43.838 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.838+0000 7f2d13e00640 1 --2- 172.21.15.72:0/3821436413 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2d14105c50 0x7f2d14108040 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:43.838 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.838+0000 7f2d13e00640 1 --2- 172.21.15.72:0/3821436413 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2d14103320 0x7f2d14105710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:43.838 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.838+0000 7f2d13e00640 1 -- 172.21.15.72:0/3821436413 >> 172.21.15.72:0/3821436413 conn(0x7f2d140fccd0 msgr2=0x7f2d140ff0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:57:43.839 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.838+0000 7f2d13e00640 1 -- 172.21.15.72:0/3821436413 shutdown_connections 2024-04-26T21:57:43.839 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.838+0000 7f2d13e00640 1 -- 172.21.15.72:0/3821436413 wait complete. 2024-04-26T21:57:43.840 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.839+0000 7f2d13e00640 1 Processor -- start 2024-04-26T21:57:43.840 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.839+0000 7f2d13e00640 1 -- start start 2024-04-26T21:57:43.840 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.839+0000 7f2d13e00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2d14103320 0x7f2d1419ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:43.841 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.839+0000 7f2d13e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2d14105c50 0x7f2d1419b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:43.841 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.839+0000 7f2d13e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2d14108580 0x7f2d141a21c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:43.841 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.839+0000 7f2d13e00640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f2d1410e270 con 0x7f2d14105c50 2024-04-26T21:57:43.841 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.839+0000 7f2d13e00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f2d1410e0f0 con 0x7f2d14103320 2024-04-26T21:57:43.841 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.839+0000 7f2d13e00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f2d1410e3f0 con 0x7f2d14108580 2024-04-26T21:57:43.841 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.840+0000 7f2d12000640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2d14105c50 0x7f2d1419b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:43.841 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.840+0000 7f2d12000640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2d14105c50 0x7f2d1419b190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:48014/0 (socket says 172.21.15.72:48014) 2024-04-26T21:57:43.841 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.840+0000 7f2d12000640 1 -- 172.21.15.72:0/3872900454 learned_addr learned my addr 172.21.15.72:0/3872900454 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:57:43.841 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.840+0000 7f2d12a00640 1 --2- 172.21.15.72:0/3872900454 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2d14103320 0x7f2d1419ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:43.841 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.840+0000 7f2d12000640 1 -- 172.21.15.72:0/3872900454 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2d14108580 msgr2=0x7f2d141a21c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:43.841 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.840+0000 7f2d13400640 1 --2- 172.21.15.72:0/3872900454 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2d14108580 0x7f2d141a21c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:43.841 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.840+0000 7f2d12000640 1 --2- 172.21.15.72:0/3872900454 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2d14108580 0x7f2d141a21c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:43.841 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.840+0000 7f2d12000640 1 -- 172.21.15.72:0/3872900454 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2d14103320 msgr2=0x7f2d1419ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:43.841 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.840+0000 7f2d12000640 1 --2- 172.21.15.72:0/3872900454 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2d14103320 0x7f2d1419ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:43.842 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.840+0000 7f2d12000640 1 -- 172.21.15.72:0/3872900454 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d141a2860 con 0x7f2d14105c50 2024-04-26T21:57:43.842 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.840+0000 7f2d12a00640 1 --2- 172.21.15.72:0/3872900454 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2d14103320 0x7f2d1419ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:57:43.842 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.841+0000 7f2d12000640 1 --2- 172.21.15.72:0/3872900454 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2d14105c50 0x7f2d1419b190 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f2d00005ab0 tx=0x7f2d000042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:57:43.842 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.841+0000 7f2d0b400640 1 -- 172.21.15.72:0/3872900454 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d0003f030 con 0x7f2d14105c50 2024-04-26T21:57:43.843 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.841+0000 7f2d0b400640 1 -- 172.21.15.72:0/3872900454 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2d00003c90 con 0x7f2d14105c50 2024-04-26T21:57:43.843 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.841+0000 7f2d0b400640 1 -- 172.21.15.72:0/3872900454 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d00035070 con 0x7f2d14105c50 2024-04-26T21:57:43.843 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.841+0000 7f2d13e00640 1 -- 172.21.15.72:0/3872900454 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d141a2af0 con 0x7f2d14105c50 2024-04-26T21:57:43.843 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.841+0000 7f2d13e00640 1 -- 172.21.15.72:0/3872900454 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d141a2f90 con 0x7f2d14105c50 2024-04-26T21:57:43.844 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.844+0000 7f2d1c78d740 1 Processor -- start 2024-04-26T21:57:43.844 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.845+0000 7f2d1c78d740 1 -- start start 2024-04-26T21:57:43.845 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.845+0000 7f2d1c78d740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c2f88de050 0x55c2f88de470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:43.845 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.845+0000 7f2d1c78d740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c2f88dea40 0x55c2f88e3540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:43.845 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.845+0000 7f2d1c78d740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c2f88e3bb0 0x55c2f88e6040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:43.845 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.845+0000 7f2d1c78d740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55c2f8841c70 con 0x55c2f88e3bb0 2024-04-26T21:57:43.845 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.845+0000 7f2d1c78d740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55c2f87fa980 con 0x55c2f88dea40 2024-04-26T21:57:43.845 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.845+0000 7f2d1c78d740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55c2f883dc30 con 0x55c2f88de050 2024-04-26T21:57:43.846 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.846+0000 7f2d13400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c2f88de050 0x55c2f88de470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:43.847 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.846+0000 7f2d12a00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c2f88dea40 0x55c2f88e3540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:43.847 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.846+0000 7f2d13400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c2f88de050 0x55c2f88de470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.72:57532/0 (socket says 172.21.15.72:57532) 2024-04-26T21:57:43.847 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.846+0000 7f2d13400640 1 -- 172.21.15.72:0/2926723935 learned_addr learned my addr 172.21.15.72:0/2926723935 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:57:43.847 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.846+0000 7f2d0b400640 1 -- 172.21.15.72:0/3872900454 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f2d00042030 con 0x7f2d14105c50 2024-04-26T21:57:43.847 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.846+0000 7f2d12000640 1 --2- 172.21.15.72:0/2926723935 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c2f88e3bb0 0x55c2f88e6040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:43.847 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.846+0000 7f2d12a00640 1 -- 172.21.15.72:0/2926723935 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c2f88de050 msgr2=0x55c2f88de470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:43.847 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.846+0000 7f2d12a00640 1 --2- 172.21.15.72:0/2926723935 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c2f88de050 0x55c2f88de470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:43.847 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.846+0000 7f2d12a00640 1 -- 172.21.15.72:0/2926723935 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c2f88e3bb0 msgr2=0x55c2f88e6040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:43.847 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.846+0000 7f2d12a00640 1 --2- 172.21.15.72:0/2926723935 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c2f88e3bb0 0x55c2f88e6040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:43.847 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.846+0000 7f2d12a00640 1 -- 172.21.15.72:0/2926723935 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c2f87d27b0 con 0x55c2f88dea40 2024-04-26T21:57:43.849 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.847+0000 7f2d12a00640 1 --2- 172.21.15.72:0/2926723935 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c2f88dea40 0x55c2f88e3540 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f2d1419c120 tx=0x7f2d0400f730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=651d0e2318a3af14 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:57:43.849 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.847+0000 7f2d0b400640 1 --2- 172.21.15.72:0/3872900454 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f2cf80a3cc0 0x7f2cf80a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:43.849 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.847+0000 7f2cffe00640 1 -- 172.21.15.72:0/2926723935 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d04015070 con 0x55c2f88dea40 2024-04-26T21:57:43.849 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.848+0000 7f2cffe00640 1 -- 172.21.15.72:0/2926723935 <== mon.2 v2:172.21.15.76:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2d0401a8f0 con 0x55c2f88dea40 2024-04-26T21:57:43.849 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.848+0000 7f2cffe00640 1 -- 172.21.15.72:0/2926723935 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d040117c0 con 0x55c2f88dea40 2024-04-26T21:57:43.849 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.848+0000 7f2d0b400640 1 -- 172.21.15.72:0/3872900454 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7f2d000f10a0 con 0x7f2d14105c50 2024-04-26T21:57:43.849 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.848+0000 7f2d13400640 1 --2- 172.21.15.72:0/3872900454 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f2cf80a3cc0 0x7f2cf80a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:43.849 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.848+0000 7f2d1c78d740 1 -- 172.21.15.72:0/2926723935 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c2f88dea40 msgr2=0x55c2f88e3540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:43.849 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.848+0000 7f2d1c78d740 1 --2- 172.21.15.72:0/2926723935 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c2f88dea40 0x55c2f88e3540 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f2d1419c120 tx=0x7f2d0400f730 comp rx=0 tx=0).stop 2024-04-26T21:57:43.849 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.848+0000 7f2d13400640 1 --2- 172.21.15.72:0/3872900454 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f2cf80a3cc0 0x7f2cf80a6180 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x55c2f88df2a0 tx=0x7f2d0c007680 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:57:43.849 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.849+0000 7f2d1c78d740 1 -- 172.21.15.72:0/2926723935 shutdown_connections 2024-04-26T21:57:43.849 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.849+0000 7f2d1c78d740 1 --2- 172.21.15.72:0/2926723935 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c2f88e3bb0 0x55c2f88e6040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:43.849 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.849+0000 7f2d1c78d740 1 --2- 172.21.15.72:0/2926723935 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c2f88dea40 0x55c2f88e3540 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:43.849 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.849+0000 7f2d1c78d740 1 --2- 172.21.15.72:0/2926723935 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c2f88de050 0x55c2f88de470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:43.850 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.849+0000 7f2d1c78d740 1 -- 172.21.15.72:0/2926723935 >> 172.21.15.72:0/2926723935 conn(0x55c2f88d96c0 msgr2=0x55c2f88d9ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:57:43.850 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.849+0000 7f2d1c78d740 1 -- 172.21.15.72:0/2926723935 shutdown_connections 2024-04-26T21:57:43.850 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.849+0000 7f2d1c78d740 1 -- 172.21.15.72:0/2926723935 wait complete. 2024-04-26T21:57:43.852 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.849+0000 7f2d1c78d740 1 -- start start 2024-04-26T21:57:43.852 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.850+0000 7f2d1c78d740 1 Processor -- start 2024-04-26T21:57:43.852 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.850+0000 7f2d1c78d740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c2f88de050 0x55c2f88e7f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:43.852 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.850+0000 7f2d1c78d740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c2f88dea40 0x55c2f88e84d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:43.852 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.850+0000 7f2d1c78d740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c2f88e3bb0 0x55c2f88ef510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:43.852 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.850+0000 7f2d1c78d740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55c2f8840f80 con 0x55c2f88e3bb0 2024-04-26T21:57:43.852 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.850+0000 7f2d1c78d740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55c2f87ee260 con 0x55c2f88de050 2024-04-26T21:57:43.852 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.850+0000 7f2d1c78d740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55c2f88462a0 con 0x55c2f88dea40 2024-04-26T21:57:43.852 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.850+0000 7f2d1c78d740 10 client.0 _ll_register_callbacks cb 0x55c2f8838960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:57:43.853 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.850+0000 7f2d13400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c2f88e3bb0 0x55c2f88ef510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:43.853 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.850+0000 7f2d13400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c2f88e3bb0 0x55c2f88ef510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:48034/0 (socket says 172.21.15.72:48034) 2024-04-26T21:57:43.853 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.850+0000 7f2d13400640 1 -- 172.21.15.72:0/127306686 learned_addr learned my addr 172.21.15.72:0/127306686 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:57:43.853 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.851+0000 7f2d12000640 1 --2- 172.21.15.72:0/127306686 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c2f88dea40 0x55c2f88e84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:43.853 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.851+0000 7f2d12a00640 1 --2- 172.21.15.72:0/127306686 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c2f88de050 0x55c2f88e7f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:43.853 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.851+0000 7f2d13400640 1 -- 172.21.15.72:0/127306686 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c2f88dea40 msgr2=0x55c2f88e84d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:43.853 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.851+0000 7f2d13400640 1 --2- 172.21.15.72:0/127306686 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c2f88dea40 0x55c2f88e84d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:43.853 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.851+0000 7f2d13400640 1 -- 172.21.15.72:0/127306686 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c2f88de050 msgr2=0x55c2f88e7f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:43.853 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.851+0000 7f2d13400640 1 --2- 172.21.15.72:0/127306686 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c2f88de050 0x55c2f88e7f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:43.853 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.851+0000 7f2d13400640 1 -- 172.21.15.72:0/127306686 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c2f87d01c0 con 0x55c2f88e3bb0 2024-04-26T21:57:43.853 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.851+0000 7f2d13400640 1 --2- 172.21.15.72:0/127306686 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c2f88e3bb0 0x55c2f88ef510 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c007ab0 tx=0x7f2d0c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=640a13c8930b0051 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:57:43.853 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.851+0000 7f2cfcc00640 10 client.0 ms_handle_connect on v2:172.21.15.72:3300/0 2024-04-26T21:57:43.853 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.852+0000 7f2cfcc00640 1 -- 172.21.15.72:0/127306686 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d0c023020 con 0x55c2f88e3bb0 2024-04-26T21:57:43.853 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.852+0000 7f2cfcc00640 1 -- 172.21.15.72:0/127306686 <== mon.0 v2:172.21.15.72:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7f2d0c011cb0 con 0x55c2f88e3bb0 2024-04-26T21:57:43.853 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.852+0000 7f2d1c78d740 1 -- 172.21.15.72:0/127306686 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d0401a8f0 con 0x55c2f88e3bb0 2024-04-26T21:57:43.854 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.852+0000 7f2cfcc00640 1 -- 172.21.15.72:0/127306686 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2d0c02e050 con 0x55c2f88e3bb0 2024-04-26T21:57:43.854 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.852+0000 7f2d1c78d740 10 client.16026 fetch_fsmap learned FSMap version 27 2024-04-26T21:57:43.854 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.852+0000 7f2d1c78d740 1 -- 172.21.15.72:0/127306686 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c2f85533f0 con 0x55c2f88e3bb0 2024-04-26T21:57:43.855 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.852+0000 7f2cfcc00640 1 -- 172.21.15.72:0/127306686 <== mon.0 v2:172.21.15.72:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2d0401a8f0 con 0x55c2f88e3bb0 2024-04-26T21:57:43.855 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.853+0000 7f2cfcc00640 1 -- 172.21.15.72:0/127306686 <== mon.0 v2:172.21.15.72:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f2d0c011040 con 0x55c2f88e3bb0 2024-04-26T21:57:43.855 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.853+0000 7f2d1c78d740 10 client.16026 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T21:57:43.855 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.853+0000 7f2d1c78d740 10 client.16026 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-26T21:57:43.855 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.853+0000 7f2d1c78d740 20 client.16026 populate_metadata read hostname 'smithi072' 2024-04-26T21:57:43.855 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.853+0000 7f2d1c78d740 1 --2- 172.21.15.72:0/127306686 >> [v2:172.21.15.76:6832/3869483750,v1:172.21.15.76:6833/3869483750] conn(0x55c2f88f3b10 0x55c2f88f5fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:43.855 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.853+0000 7f2d1c78d740 4 client.16026 mds_command: new command op to 34406 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:57:43.855 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.853+0000 7f2d1c78d740 1 -- 172.21.15.72:0/127306686 --> [v2:172.21.15.76:6832/3869483750,v1:172.21.15.76:6833/3869483750] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c2f884f880 con 0x55c2f88f3b10 2024-04-26T21:57:43.855 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.854+0000 7f2d12a00640 1 --2- 172.21.15.72:0/127306686 >> [v2:172.21.15.76:6832/3869483750,v1:172.21.15.76:6833/3869483750] conn(0x55c2f88f3b10 0x55c2f88f5fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:43.855 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.855+0000 7f2d12a00640 1 --2- 172.21.15.72:0/127306686 >> [v2:172.21.15.76:6832/3869483750,v1:172.21.15.76:6833/3869483750] conn(0x55c2f88f3b10 0x55c2f88f5fd0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=beea331f07deb6a1 server_cookie=4ad9ee5227f42d14 in_seq=0 out_seq=0 2024-04-26T21:57:43.855 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.855+0000 7f2cfcc00640 10 client.16026 ms_handle_connect on v2:172.21.15.76:6832/3869483750 2024-04-26T21:57:43.857 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.856+0000 7f2cfcc00640 1 -- 172.21.15.72:0/127306686 <== mds.3 v2:172.21.15.76:6832/3869483750 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c2f884f880 con 0x55c2f88f3b10 2024-04-26T21:57:43.857 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.856+0000 7f2cfcc00640 10 client.16026 handle_command_reply: tid=0 2024-04-26T21:57:43.857 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.856+0000 7f2d1c78d740 1 client.16026 shutdown 2024-04-26T21:57:43.857 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.857+0000 7f2d1c78d740 1 -- 172.21.15.72:0/127306686 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c2f88e3bb0 msgr2=0x55c2f88ef510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:43.857 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.857+0000 7f2d1c78d740 1 --2- 172.21.15.72:0/127306686 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c2f88e3bb0 0x55c2f88ef510 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c007ab0 tx=0x7f2d0c004a40 comp rx=0 tx=0).stop 2024-04-26T21:57:43.857 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.857+0000 7f2d1c78d740 1 -- 172.21.15.72:0/127306686 shutdown_connections 2024-04-26T21:57:43.857 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.857+0000 7f2d1c78d740 1 --2- 172.21.15.72:0/127306686 >> [v2:172.21.15.76:6832/3869483750,v1:172.21.15.76:6833/3869483750] conn(0x55c2f88f3b10 0x55c2f88f5fd0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:43.857 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.857+0000 7f2d1c78d740 1 --2- 172.21.15.72:0/127306686 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c2f88e3bb0 0x55c2f88ef510 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:43.858 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.857+0000 7f2d1c78d740 1 --2- 172.21.15.72:0/127306686 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c2f88dea40 0x55c2f88e84d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:43.858 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.857+0000 7f2d1c78d740 1 --2- 172.21.15.72:0/127306686 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c2f88de050 0x55c2f88e7f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:43.858 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.857+0000 7f2cfcc00640 0 client.16026 ms_handle_reset on v2:172.21.15.76:6832/3869483750 2024-04-26T21:57:43.858 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.857+0000 7f2d1c78d740 1 -- 172.21.15.72:0/127306686 >> 172.21.15.72:0/127306686 conn(0x55c2f88d96c0 msgr2=0x55c2f88d87b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:57:43.858 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.857+0000 7f2d1c78d740 1 -- 172.21.15.72:0/127306686 shutdown_connections 2024-04-26T21:57:43.858 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.857+0000 7f2d1c78d740 1 -- 172.21.15.72:0/127306686 wait complete. 2024-04-26T21:57:43.858 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.857+0000 7f2d1c78d740 20 client.16026 trim_cache size 0 max 16384 2024-04-26T21:57:43.874 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.874+0000 7f2d1c78d740 1 Processor -- start 2024-04-26T21:57:43.874 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.874+0000 7f2d1c78d740 1 -- start start 2024-04-26T21:57:43.874 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.874+0000 7f2d1c78d740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c2f88de050 0x55c2f88dce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:43.874 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.874+0000 7f2d1c78d740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c2f88dea40 0x55c2f88ef4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:43.874 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.874+0000 7f2d1c78d740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c2f88e3bb0 0x55c2f88d8630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:43.874 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.874+0000 7f2d1c78d740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f2d141abff0 con 0x55c2f88e3bb0 2024-04-26T21:57:43.874 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.874+0000 7f2d1c78d740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f2d141adb30 con 0x55c2f88de050 2024-04-26T21:57:43.874 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.874+0000 7f2d1c78d740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f2d141ac6c0 con 0x55c2f88dea40 2024-04-26T21:57:43.876 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.874+0000 7f2d13400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c2f88e3bb0 0x55c2f88d8630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:43.876 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.874+0000 7f2d13400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c2f88e3bb0 0x55c2f88d8630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:48040/0 (socket says 172.21.15.72:48040) 2024-04-26T21:57:43.876 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.874+0000 7f2d13400640 1 -- 172.21.15.72:0/4255026862 learned_addr learned my addr 172.21.15.72:0/4255026862 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:57:43.876 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.875+0000 7f2d12a00640 1 --2- 172.21.15.72:0/4255026862 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c2f88de050 0x55c2f88dce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:43.876 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.875+0000 7f2d12000640 1 --2- 172.21.15.72:0/4255026862 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c2f88dea40 0x55c2f88ef4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:43.876 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.875+0000 7f2d13400640 1 -- 172.21.15.72:0/4255026862 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c2f88dea40 msgr2=0x55c2f88ef4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:43.876 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.875+0000 7f2d13400640 1 --2- 172.21.15.72:0/4255026862 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c2f88dea40 0x55c2f88ef4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:43.876 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.875+0000 7f2d13400640 1 -- 172.21.15.72:0/4255026862 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c2f88de050 msgr2=0x55c2f88dce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:43.876 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.875+0000 7f2d13400640 1 --2- 172.21.15.72:0/4255026862 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c2f88de050 0x55c2f88dce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:43.876 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.875+0000 7f2d13400640 1 -- 172.21.15.72:0/4255026862 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c2f861d7d0 con 0x55c2f88e3bb0 2024-04-26T21:57:43.876 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.875+0000 7f2d13400640 1 --2- 172.21.15.72:0/4255026862 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c2f88e3bb0 0x55c2f88d8630 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c014040 tx=0x7f2d0c012ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7328d86f30c5ad7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:57:43.876 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.876+0000 7f2cff400640 1 -- 172.21.15.72:0/4255026862 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d0c036070 con 0x55c2f88e3bb0 2024-04-26T21:57:43.876 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.876+0000 7f2cff400640 1 -- 172.21.15.72:0/4255026862 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2d0c018030 con 0x55c2f88e3bb0 2024-04-26T21:57:43.877 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.876+0000 7f2cff400640 1 -- 172.21.15.72:0/4255026862 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d0c023050 con 0x55c2f88e3bb0 2024-04-26T21:57:43.877 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.877+0000 7f2d1c78d740 1 -- 172.21.15.72:0/4255026862 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c2f88e3bb0 msgr2=0x55c2f88d8630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:43.878 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.877+0000 7f2d1c78d740 1 --2- 172.21.15.72:0/4255026862 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c2f88e3bb0 0x55c2f88d8630 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c014040 tx=0x7f2d0c012ae0 comp rx=0 tx=0).stop 2024-04-26T21:57:43.878 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.877+0000 7f2d1c78d740 1 -- 172.21.15.72:0/4255026862 shutdown_connections 2024-04-26T21:57:43.878 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.877+0000 7f2d1c78d740 1 --2- 172.21.15.72:0/4255026862 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c2f88e3bb0 0x55c2f88d8630 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:43.878 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.877+0000 7f2d1c78d740 1 --2- 172.21.15.72:0/4255026862 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c2f88dea40 0x55c2f88ef4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:43.878 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.877+0000 7f2d1c78d740 1 --2- 172.21.15.72:0/4255026862 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c2f88de050 0x55c2f88dce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:43.878 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.877+0000 7f2d1c78d740 1 -- 172.21.15.72:0/4255026862 >> 172.21.15.72:0/4255026862 conn(0x55c2f88d96c0 msgr2=0x55c2f88d7f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:57:43.878 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.877+0000 7f2d1c78d740 1 -- 172.21.15.72:0/4255026862 shutdown_connections 2024-04-26T21:57:43.878 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.877+0000 7f2d1c78d740 1 -- 172.21.15.72:0/4255026862 wait complete. 2024-04-26T21:57:43.878 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.877+0000 7f2d1c78d740 1 -- start start 2024-04-26T21:57:43.878 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.878+0000 7f2d1c78d740 1 Processor -- start 2024-04-26T21:57:43.880 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.878+0000 7f2d1c78d740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c2f88de050 0x55c2f88ec820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:43.881 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.878+0000 7f2d1c78d740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c2f88dea40 0x55c2f88ecd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:43.881 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.878+0000 7f2d1c78d740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c2f88e3bb0 0x55c2f88edd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:43.881 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.878+0000 7f2d1c78d740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f2d141ad460 con 0x55c2f88e3bb0 2024-04-26T21:57:43.881 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.878+0000 7f2d1c78d740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f2d141acd90 con 0x55c2f88de050 2024-04-26T21:57:43.881 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.878+0000 7f2d1c78d740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f2d141a9fd0 con 0x55c2f88dea40 2024-04-26T21:57:43.881 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.878+0000 7f2d1c78d740 10 client.0 _ll_register_callbacks cb 0x7f2d0c0071b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:57:43.881 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.878+0000 7f2d13400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c2f88e3bb0 0x55c2f88edd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:43.881 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.878+0000 7f2d13400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c2f88e3bb0 0x55c2f88edd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:48044/0 (socket says 172.21.15.72:48044) 2024-04-26T21:57:43.881 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.879+0000 7f2d13400640 1 -- 172.21.15.72:0/2024577414 learned_addr learned my addr 172.21.15.72:0/2024577414 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:57:43.881 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.879+0000 7f2d12a00640 1 --2- 172.21.15.72:0/2024577414 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c2f88de050 0x55c2f88ec820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:43.881 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.879+0000 7f2d12000640 1 --2- 172.21.15.72:0/2024577414 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c2f88dea40 0x55c2f88ecd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:43.881 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.879+0000 7f2d13400640 1 -- 172.21.15.72:0/2024577414 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c2f88dea40 msgr2=0x55c2f88ecd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:43.882 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.879+0000 7f2d13400640 1 --2- 172.21.15.72:0/2024577414 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c2f88dea40 0x55c2f88ecd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:43.882 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.879+0000 7f2d13400640 1 -- 172.21.15.72:0/2024577414 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c2f88de050 msgr2=0x55c2f88ec820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:43.882 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.879+0000 7f2d13400640 1 --2- 172.21.15.72:0/2024577414 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c2f88de050 0x55c2f88ec820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:43.882 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.879+0000 7f2d13400640 1 -- 172.21.15.72:0/2024577414 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c2f87e82a0 con 0x55c2f88e3bb0 2024-04-26T21:57:43.882 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.879+0000 7f2d12000640 1 --2- 172.21.15.72:0/2024577414 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c2f88dea40 0x55c2f88ecd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:57:43.882 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.879+0000 7f2d13400640 1 --2- 172.21.15.72:0/2024577414 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c2f88e3bb0 0x55c2f88edd70 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c01e4e0 tx=0x7f2d0c004a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d898c3fa48754493 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:57:43.882 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.879+0000 7f2cfcc00640 10 client.0 ms_handle_connect on v2:172.21.15.72:3300/0 2024-04-26T21:57:43.882 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.880+0000 7f2cfcc00640 1 -- 172.21.15.72:0/2024577414 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d0c03f070 con 0x55c2f88e3bb0 2024-04-26T21:57:43.882 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.880+0000 7f2cfcc00640 1 -- 172.21.15.72:0/2024577414 <== mon.0 v2:172.21.15.72:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7f2d0c011d00 con 0x55c2f88e3bb0 2024-04-26T21:57:43.882 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.880+0000 7f2d1c78d740 1 -- 172.21.15.72:0/2024577414 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d0c018030 con 0x55c2f88e3bb0 2024-04-26T21:57:43.882 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.880+0000 7f2cfcc00640 1 -- 172.21.15.72:0/2024577414 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2d0c04b050 con 0x55c2f88e3bb0 2024-04-26T21:57:43.882 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.880+0000 7f2d1c78d740 10 client.16032 fetch_fsmap learned FSMap version 27 2024-04-26T21:57:43.882 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.880+0000 7f2d1c78d740 1 -- 172.21.15.72:0/2024577414 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c2f88ef490 con 0x55c2f88e3bb0 2024-04-26T21:57:43.882 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.880+0000 7f2cfcc00640 1 -- 172.21.15.72:0/2024577414 <== mon.0 v2:172.21.15.72:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2d0c018030 con 0x55c2f88e3bb0 2024-04-26T21:57:43.883 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.881+0000 7f2cfcc00640 1 -- 172.21.15.72:0/2024577414 <== mon.0 v2:172.21.15.72:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f2d0c01fb10 con 0x55c2f88e3bb0 2024-04-26T21:57:43.883 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.881+0000 7f2d1c78d740 10 client.16032 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T21:57:43.883 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.881+0000 7f2d1c78d740 10 client.16032 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-26T21:57:43.883 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.881+0000 7f2d1c78d740 20 client.16032 populate_metadata read hostname 'smithi072' 2024-04-26T21:57:43.883 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.881+0000 7f2d1c78d740 1 --2- 172.21.15.72:0/2024577414 >> [v2:172.21.15.76:6832/3869483750,v1:172.21.15.76:6833/3869483750] conn(0x55c2f895a4d0 0x55c2f895c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:43.883 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.881+0000 7f2d1c78d740 4 client.16032 mds_command: new command op to 34406 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-26T21:57:43.883 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.881+0000 7f2d1c78d740 1 -- 172.21.15.72:0/2024577414 --> [v2:172.21.15.76:6832/3869483750,v1:172.21.15.76:6833/3869483750] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c2f88496f0 con 0x55c2f895a4d0 2024-04-26T21:57:43.884 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.881+0000 7f2d12a00640 1 --2- 172.21.15.72:0/2024577414 >> [v2:172.21.15.76:6832/3869483750,v1:172.21.15.76:6833/3869483750] conn(0x55c2f895a4d0 0x55c2f895c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:43.884 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.882+0000 7f2d12a00640 1 --2- 172.21.15.72:0/2024577414 >> [v2:172.21.15.76:6832/3869483750,v1:172.21.15.76:6833/3869483750] conn(0x55c2f895a4d0 0x55c2f895c8f0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=9d834e11b8ea3451 server_cookie=b57650947f720fc0 in_seq=0 out_seq=0 2024-04-26T21:57:43.884 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.882+0000 7f2cfcc00640 10 client.16032 ms_handle_connect on v2:172.21.15.76:6832/3869483750 2024-04-26T21:57:43.885 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.883+0000 7f2cfcc00640 1 -- 172.21.15.72:0/2024577414 <== mds.3 v2:172.21.15.76:6832/3869483750 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c2f88496f0 con 0x55c2f895a4d0 2024-04-26T21:57:43.885 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.883+0000 7f2cfcc00640 10 client.16032 handle_command_reply: tid=0 2024-04-26T21:57:43.885 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.883+0000 7f2d1c78d740 1 client.16032 shutdown 2024-04-26T21:57:43.885 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.883+0000 7f2d1c78d740 1 -- 172.21.15.72:0/2024577414 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c2f88e3bb0 msgr2=0x55c2f88edd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:43.885 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.883+0000 7f2d1c78d740 1 --2- 172.21.15.72:0/2024577414 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c2f88e3bb0 0x55c2f88edd70 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c01e4e0 tx=0x7f2d0c004a20 comp rx=0 tx=0).stop 2024-04-26T21:57:43.885 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.883+0000 7f2d1c78d740 1 -- 172.21.15.72:0/2024577414 shutdown_connections 2024-04-26T21:57:43.885 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.883+0000 7f2d1c78d740 1 --2- 172.21.15.72:0/2024577414 >> [v2:172.21.15.76:6832/3869483750,v1:172.21.15.76:6833/3869483750] conn(0x55c2f895a4d0 0x55c2f895c8f0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:43.885 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.883+0000 7f2d1c78d740 1 --2- 172.21.15.72:0/2024577414 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c2f88e3bb0 0x55c2f88edd70 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:43.885 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.883+0000 7f2d1c78d740 1 --2- 172.21.15.72:0/2024577414 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c2f88dea40 0x55c2f88ecd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:43.885 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.883+0000 7f2d1c78d740 1 --2- 172.21.15.72:0/2024577414 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c2f88de050 0x55c2f88ec820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:43.885 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.883+0000 7f2d1c78d740 1 -- 172.21.15.72:0/2024577414 >> 172.21.15.72:0/2024577414 conn(0x55c2f88d96c0 msgr2=0x55c2f88dcce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:57:43.885 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.883+0000 7f2cfcc00640 0 client.16032 ms_handle_reset on v2:172.21.15.76:6832/3869483750 2024-04-26T21:57:43.885 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.883+0000 7f2d1c78d740 1 -- 172.21.15.72:0/2024577414 shutdown_connections 2024-04-26T21:57:43.885 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.883+0000 7f2d1c78d740 1 -- 172.21.15.72:0/2024577414 wait complete. 2024-04-26T21:57:43.885 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.883+0000 7f2d1c78d740 20 client.16032 trim_cache size 0 max 16384 2024-04-26T21:57:43.886 INFO:teuthology.orchestra.run.smithi072.stdout:[] 2024-04-26T21:57:43.887 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.884+0000 7f2d13e00640 1 -- 172.21.15.72:0/3872900454 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f2cf80a3cc0 msgr2=0x7f2cf80a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:43.887 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.884+0000 7f2d13e00640 1 --2- 172.21.15.72:0/3872900454 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f2cf80a3cc0 0x7f2cf80a6180 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x55c2f88df2a0 tx=0x7f2d0c007680 comp rx=0 tx=0).stop 2024-04-26T21:57:43.887 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.884+0000 7f2d13e00640 1 -- 172.21.15.72:0/3872900454 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2d14105c50 msgr2=0x7f2d1419b190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:43.887 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.884+0000 7f2d13e00640 1 --2- 172.21.15.72:0/3872900454 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2d14105c50 0x7f2d1419b190 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f2d00005ab0 tx=0x7f2d000042f0 comp rx=0 tx=0).stop 2024-04-26T21:57:43.887 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.884+0000 7f2d13e00640 1 -- 172.21.15.72:0/3872900454 shutdown_connections 2024-04-26T21:57:43.887 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.884+0000 7f2d13e00640 1 --2- 172.21.15.72:0/3872900454 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f2cf80a3cc0 0x7f2cf80a6180 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:43.887 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.884+0000 7f2d13e00640 1 --2- 172.21.15.72:0/3872900454 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2d14108580 0x7f2d141a21c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:43.887 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.884+0000 7f2d13e00640 1 --2- 172.21.15.72:0/3872900454 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2d14105c50 0x7f2d1419b190 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:43.887 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.884+0000 7f2d13e00640 1 --2- 172.21.15.72:0/3872900454 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2d14103320 0x7f2d1419ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:43.887 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.884+0000 7f2d13e00640 1 -- 172.21.15.72:0/3872900454 >> 172.21.15.72:0/3872900454 conn(0x7f2d140fccd0 msgr2=0x7f2d14102b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:57:43.887 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.884+0000 7f2d13e00640 1 -- 172.21.15.72:0/3872900454 shutdown_connections 2024-04-26T21:57:43.887 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:43.885+0000 7f2d13e00640 1 -- 172.21.15.72:0/3872900454 wait complete. 2024-04-26T21:57:43.905 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-26T21:57:44.121 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.120+0000 7f1509000640 1 -- 172.21.15.72:0/2791742028 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1504108f90 msgr2=0x7f15041093b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:44.121 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.120+0000 7f1509000640 1 --2- 172.21.15.72:0/2791742028 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1504108f90 0x7f15041093b0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f14f4009900 tx=0x7f14f402a410 comp rx=0 tx=0).stop 2024-04-26T21:57:44.121 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.120+0000 7f1509000640 1 -- 172.21.15.72:0/2791742028 shutdown_connections 2024-04-26T21:57:44.121 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.120+0000 7f1509000640 1 --2- 172.21.15.72:0/2791742028 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f150410ab40 0x7f15041113d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:44.121 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.120+0000 7f1509000640 1 --2- 172.21.15.72:0/2791742028 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f150410a180 0x7f150410a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:44.121 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.120+0000 7f1509000640 1 --2- 172.21.15.72:0/2791742028 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1504108f90 0x7f15041093b0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:44.121 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.120+0000 7f1509000640 1 -- 172.21.15.72:0/2791742028 >> 172.21.15.72:0/2791742028 conn(0x7f1504077f30 msgr2=0x7f150407a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:57:44.121 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.121+0000 7f1509000640 1 -- 172.21.15.72:0/2791742028 shutdown_connections 2024-04-26T21:57:44.121 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.121+0000 7f1509000640 1 -- 172.21.15.72:0/2791742028 wait complete. 2024-04-26T21:57:44.124 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.122+0000 7f1509000640 1 Processor -- start 2024-04-26T21:57:44.124 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.122+0000 7f1509000640 1 -- start start 2024-04-26T21:57:44.124 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.122+0000 7f1509000640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1504108f90 0x7f150419f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:44.124 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.122+0000 7f1509000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f150410a180 0x7f150419f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:44.124 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.122+0000 7f1509000640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f150410ab40 0x7f15041a66e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:44.124 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.122+0000 7f1509000640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f1504114980 con 0x7f1504108f90 2024-04-26T21:57:44.124 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.122+0000 7f1509000640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f1504114800 con 0x7f150410ab40 2024-04-26T21:57:44.124 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.122+0000 7f1509000640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f1504114b00 con 0x7f150410a180 2024-04-26T21:57:44.124 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.122+0000 7f1501e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1504108f90 0x7f150419f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:44.124 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.122+0000 7f1501e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1504108f90 0x7f150419f130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:48074/0 (socket says 172.21.15.72:48074) 2024-04-26T21:57:44.124 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.123+0000 7f1501e00640 1 -- 172.21.15.72:0/40208498 learned_addr learned my addr 172.21.15.72:0/40208498 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:57:44.125 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.123+0000 7f1501400640 1 --2- 172.21.15.72:0/40208498 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f150410a180 0x7f150419f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:44.125 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.123+0000 7f1502800640 1 --2- 172.21.15.72:0/40208498 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f150410ab40 0x7f15041a66e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:44.125 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.123+0000 7f1501e00640 1 -- 172.21.15.72:0/40208498 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f150410a180 msgr2=0x7f150419f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:44.125 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.123+0000 7f1501e00640 1 --2- 172.21.15.72:0/40208498 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f150410a180 0x7f150419f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:44.125 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.123+0000 7f1501e00640 1 -- 172.21.15.72:0/40208498 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f150410ab40 msgr2=0x7f15041a66e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:44.125 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.123+0000 7f1501e00640 1 --2- 172.21.15.72:0/40208498 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f150410ab40 0x7f15041a66e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:44.125 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.123+0000 7f1501e00640 1 -- 172.21.15.72:0/40208498 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15041a6d50 con 0x7f1504108f90 2024-04-26T21:57:44.125 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.123+0000 7f1501e00640 1 --2- 172.21.15.72:0/40208498 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1504108f90 0x7f150419f130 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f14f4005ab0 tx=0x7f14f40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:57:44.125 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.124+0000 7f14faa00640 1 -- 172.21.15.72:0/40208498 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14f403f030 con 0x7f1504108f90 2024-04-26T21:57:44.125 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.124+0000 7f14faa00640 1 -- 172.21.15.72:0/40208498 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f14f4003c90 con 0x7f1504108f90 2024-04-26T21:57:44.125 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.124+0000 7f14faa00640 1 -- 172.21.15.72:0/40208498 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14f4035070 con 0x7f1504108f90 2024-04-26T21:57:44.125 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.124+0000 7f1509000640 1 -- 172.21.15.72:0/40208498 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15041a6fe0 con 0x7f1504108f90 2024-04-26T21:57:44.125 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.124+0000 7f1509000640 1 -- 172.21.15.72:0/40208498 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15041a73f0 con 0x7f1504108f90 2024-04-26T21:57:44.127 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.127+0000 7f150bfe0740 1 Processor -- start 2024-04-26T21:57:44.127 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.127+0000 7f14faa00640 1 -- 172.21.15.72:0/40208498 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f14f4042030 con 0x7f1504108f90 2024-04-26T21:57:44.128 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.128+0000 7f150bfe0740 1 -- start start 2024-04-26T21:57:44.128 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.128+0000 7f150bfe0740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56514c239ea0 0x56514c23a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:44.129 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.128+0000 7f150bfe0740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56514c23a890 0x56514c23f390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:44.129 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.128+0000 7f150bfe0740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56514c23fa00 0x56514c241e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:44.129 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.128+0000 7f150bfe0740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x56514c19dc70 con 0x56514c23a890 2024-04-26T21:57:44.129 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.128+0000 7f150bfe0740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x56514c156980 con 0x56514c239ea0 2024-04-26T21:57:44.129 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.128+0000 7f150bfe0740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x56514c199c30 con 0x56514c23fa00 2024-04-26T21:57:44.129 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.128+0000 7f1501e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56514c23a890 0x56514c23f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:44.129 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.129+0000 7f1501e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56514c23a890 0x56514c23f390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:48082/0 (socket says 172.21.15.72:48082) 2024-04-26T21:57:44.129 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.129+0000 7f1501e00640 1 -- 172.21.15.72:0/1237513961 learned_addr learned my addr 172.21.15.72:0/1237513961 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:57:44.129 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.129+0000 7f1502800640 1 --2- 172.21.15.72:0/1237513961 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56514c239ea0 0x56514c23a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:44.129 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.129+0000 7f1501400640 1 --2- 172.21.15.72:0/1237513961 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56514c23fa00 0x56514c241e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:44.130 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.129+0000 7f1501e00640 1 -- 172.21.15.72:0/1237513961 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56514c23fa00 msgr2=0x56514c241e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:44.130 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.129+0000 7f1501e00640 1 --2- 172.21.15.72:0/1237513961 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56514c23fa00 0x56514c241e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:44.131 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.129+0000 7f1501e00640 1 -- 172.21.15.72:0/1237513961 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56514c239ea0 msgr2=0x56514c23a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:44.131 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.129+0000 7f1501e00640 1 --2- 172.21.15.72:0/1237513961 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56514c239ea0 0x56514c23a2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:44.131 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.129+0000 7f1501e00640 1 -- 172.21.15.72:0/1237513961 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56514c12e7b0 con 0x56514c23a890 2024-04-26T21:57:44.131 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.129+0000 7f1501e00640 1 --2- 172.21.15.72:0/1237513961 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56514c23a890 0x56514c23f390 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f14f40f92f0 tx=0x7f14f40f93e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e53b04cf768ad9b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:57:44.131 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.130+0000 7f14faa00640 1 --2- 172.21.15.72:0/40208498 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f14e80a3c40 0x7f14e80a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:44.131 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.130+0000 7f14faa00640 1 -- 172.21.15.72:0/40208498 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7f14f40f10d0 con 0x7f1504108f90 2024-04-26T21:57:44.131 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.130+0000 7f1502800640 1 --2- 172.21.15.72:0/40208498 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f14e80a3c40 0x7f14e80a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:44.131 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.130+0000 7f14ef400640 1 -- 172.21.15.72:0/1237513961 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14f40f4c70 con 0x56514c23a890 2024-04-26T21:57:44.131 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.130+0000 7f14ef400640 1 -- 172.21.15.72:0/1237513961 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f14f40fe030 con 0x56514c23a890 2024-04-26T21:57:44.131 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.130+0000 7f14ef400640 1 -- 172.21.15.72:0/1237513961 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14f40bcad0 con 0x56514c23a890 2024-04-26T21:57:44.134 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.131+0000 7f1502800640 1 --2- 172.21.15.72:0/40208498 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f14e80a3c40 0x7f14e80a6100 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x56514c23b0f0 tx=0x7f14fc009740 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:57:44.134 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.131+0000 7f150bfe0740 1 -- 172.21.15.72:0/1237513961 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56514c23a890 msgr2=0x56514c23f390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:44.134 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.131+0000 7f150bfe0740 1 --2- 172.21.15.72:0/1237513961 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56514c23a890 0x56514c23f390 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f14f40f92f0 tx=0x7f14f40f93e0 comp rx=0 tx=0).stop 2024-04-26T21:57:44.134 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.131+0000 7f150bfe0740 1 -- 172.21.15.72:0/1237513961 shutdown_connections 2024-04-26T21:57:44.134 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.131+0000 7f150bfe0740 1 --2- 172.21.15.72:0/1237513961 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56514c23fa00 0x56514c241e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:44.134 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.131+0000 7f150bfe0740 1 --2- 172.21.15.72:0/1237513961 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56514c23a890 0x56514c23f390 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:44.134 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.131+0000 7f150bfe0740 1 --2- 172.21.15.72:0/1237513961 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56514c239ea0 0x56514c23a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:44.134 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.131+0000 7f150bfe0740 1 -- 172.21.15.72:0/1237513961 >> 172.21.15.72:0/1237513961 conn(0x56514c235510 msgr2=0x56514c235910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:57:44.134 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.131+0000 7f150bfe0740 1 -- 172.21.15.72:0/1237513961 shutdown_connections 2024-04-26T21:57:44.134 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.131+0000 7f150bfe0740 1 -- 172.21.15.72:0/1237513961 wait complete. 2024-04-26T21:57:44.134 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.132+0000 7f150bfe0740 1 -- start start 2024-04-26T21:57:44.134 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.132+0000 7f150bfe0740 1 Processor -- start 2024-04-26T21:57:44.135 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.133+0000 7f150bfe0740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56514c239ea0 0x56514c243ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:44.135 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.133+0000 7f150bfe0740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56514c23a890 0x56514c244020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:44.135 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.133+0000 7f150bfe0740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56514c23fa00 0x56514c24b060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:44.135 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.133+0000 7f150bfe0740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x56514c19cf80 con 0x56514c23a890 2024-04-26T21:57:44.135 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.133+0000 7f150bfe0740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x56514c14a260 con 0x56514c23fa00 2024-04-26T21:57:44.135 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.133+0000 7f150bfe0740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x56514c1a22a0 con 0x56514c239ea0 2024-04-26T21:57:44.135 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.133+0000 7f150bfe0740 10 client.0 _ll_register_callbacks cb 0x56514c194960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:57:44.135 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.133+0000 7f1501400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56514c23a890 0x56514c244020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:44.135 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.133+0000 7f1501400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56514c23a890 0x56514c244020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:48096/0 (socket says 172.21.15.72:48096) 2024-04-26T21:57:44.135 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.133+0000 7f1501400640 1 -- 172.21.15.72:0/1430661265 learned_addr learned my addr 172.21.15.72:0/1430661265 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:57:44.135 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.133+0000 7f1502800640 1 --2- 172.21.15.72:0/1430661265 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56514c23fa00 0x56514c24b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:44.135 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.133+0000 7f1501e00640 1 --2- 172.21.15.72:0/1430661265 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56514c239ea0 0x56514c243ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:44.135 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.133+0000 7f1501400640 1 -- 172.21.15.72:0/1430661265 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56514c239ea0 msgr2=0x56514c243ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:44.135 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.133+0000 7f1501400640 1 --2- 172.21.15.72:0/1430661265 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56514c239ea0 0x56514c243ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:44.136 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.133+0000 7f1501400640 1 -- 172.21.15.72:0/1430661265 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56514c23fa00 msgr2=0x56514c24b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:44.136 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.133+0000 7f1501400640 1 --2- 172.21.15.72:0/1430661265 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56514c23fa00 0x56514c24b060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:44.136 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.133+0000 7f1501400640 1 -- 172.21.15.72:0/1430661265 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56514c12c1c0 con 0x56514c23a890 2024-04-26T21:57:44.136 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.134+0000 7f1501400640 1 --2- 172.21.15.72:0/1430661265 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56514c23a890 0x56514c244020 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f14f0010740 tx=0x7f14f0010c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af7dbd2192239162 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:57:44.136 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.134+0000 7f14e3e00640 10 client.0 ms_handle_connect on v2:172.21.15.72:3300/0 2024-04-26T21:57:44.137 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.134+0000 7f14e3e00640 1 -- 172.21.15.72:0/1430661265 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14f000ccc0 con 0x56514c23a890 2024-04-26T21:57:44.137 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.134+0000 7f14e3e00640 1 -- 172.21.15.72:0/1430661265 <== mon.0 v2:172.21.15.72:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7f14f0011440 con 0x56514c23a890 2024-04-26T21:57:44.137 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.134+0000 7f150bfe0740 1 -- 172.21.15.72:0/1430661265 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f14f40fe030 con 0x56514c23a890 2024-04-26T21:57:44.137 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.135+0000 7f14e3e00640 1 -- 172.21.15.72:0/1430661265 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f14f0020030 con 0x56514c23a890 2024-04-26T21:57:44.137 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.135+0000 7f150bfe0740 10 client.16044 fetch_fsmap learned FSMap version 27 2024-04-26T21:57:44.137 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.135+0000 7f150bfe0740 1 -- 172.21.15.72:0/1430661265 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56514beaf3f0 con 0x56514c23a890 2024-04-26T21:57:44.138 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.135+0000 7f14e3e00640 1 -- 172.21.15.72:0/1430661265 <== mon.0 v2:172.21.15.72:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f14f40fe030 con 0x56514c23a890 2024-04-26T21:57:44.138 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.135+0000 7f14e3e00640 1 -- 172.21.15.72:0/1430661265 <== mon.0 v2:172.21.15.72:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f14f0025430 con 0x56514c23a890 2024-04-26T21:57:44.138 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.135+0000 7f150bfe0740 10 client.16044 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T21:57:44.138 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.135+0000 7f150bfe0740 10 client.16044 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-26T21:57:44.138 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.135+0000 7f150bfe0740 20 client.16044 populate_metadata read hostname 'smithi072' 2024-04-26T21:57:44.138 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.135+0000 7f150bfe0740 1 --2- 172.21.15.72:0/1430661265 >> [v2:172.21.15.76:6834/3517429435,v1:172.21.15.76:6835/3517429435] conn(0x56514c24f5f0 0x56514c251ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:44.138 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.135+0000 7f150bfe0740 4 client.16044 mds_command: new command op to 34442 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:57:44.138 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.135+0000 7f150bfe0740 1 -- 172.21.15.72:0/1430661265 --> [v2:172.21.15.76:6834/3517429435,v1:172.21.15.76:6835/3517429435] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56514c1ab880 con 0x56514c24f5f0 2024-04-26T21:57:44.138 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.136+0000 7f1501e00640 1 --2- 172.21.15.72:0/1430661265 >> [v2:172.21.15.76:6834/3517429435,v1:172.21.15.76:6835/3517429435] conn(0x56514c24f5f0 0x56514c251ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:44.138 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.137+0000 7f1501e00640 1 --2- 172.21.15.72:0/1430661265 >> [v2:172.21.15.76:6834/3517429435,v1:172.21.15.76:6835/3517429435] conn(0x56514c24f5f0 0x56514c251ab0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c19e56e3fe3f2100 server_cookie=2894197603d76b08 in_seq=0 out_seq=0 2024-04-26T21:57:44.138 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.137+0000 7f14e3e00640 10 client.16044 ms_handle_connect on v2:172.21.15.76:6834/3517429435 2024-04-26T21:57:44.139 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.138+0000 7f14e3e00640 1 -- 172.21.15.72:0/1430661265 <== mds.2 v2:172.21.15.76:6834/3517429435 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56514c1ab880 con 0x56514c24f5f0 2024-04-26T21:57:44.140 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.138+0000 7f14e3e00640 10 client.16044 handle_command_reply: tid=0 2024-04-26T21:57:44.140 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.139+0000 7f150bfe0740 1 client.16044 shutdown 2024-04-26T21:57:44.140 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.139+0000 7f150bfe0740 1 -- 172.21.15.72:0/1430661265 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56514c23a890 msgr2=0x56514c244020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:44.140 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.139+0000 7f150bfe0740 1 --2- 172.21.15.72:0/1430661265 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56514c23a890 0x56514c244020 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f14f0010740 tx=0x7f14f0010c00 comp rx=0 tx=0).stop 2024-04-26T21:57:44.140 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.139+0000 7f150bfe0740 1 -- 172.21.15.72:0/1430661265 shutdown_connections 2024-04-26T21:57:44.140 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.139+0000 7f150bfe0740 1 --2- 172.21.15.72:0/1430661265 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56514c23fa00 0x56514c24b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:44.140 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.139+0000 7f150bfe0740 1 --2- 172.21.15.72:0/1430661265 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56514c23a890 0x56514c244020 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:44.140 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.139+0000 7f150bfe0740 1 --2- 172.21.15.72:0/1430661265 >> [v2:172.21.15.76:6834/3517429435,v1:172.21.15.76:6835/3517429435] conn(0x56514c24f5f0 0x56514c251ab0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:44.140 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.139+0000 7f150bfe0740 1 --2- 172.21.15.72:0/1430661265 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56514c239ea0 0x56514c243ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:44.140 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.139+0000 7f150bfe0740 1 -- 172.21.15.72:0/1430661265 >> 172.21.15.72:0/1430661265 conn(0x56514c235510 msgr2=0x56514c234690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:57:44.140 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.139+0000 7f14e3e00640 0 client.16044 ms_handle_reset on v2:172.21.15.76:6834/3517429435 2024-04-26T21:57:44.140 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.139+0000 7f150bfe0740 1 -- 172.21.15.72:0/1430661265 shutdown_connections 2024-04-26T21:57:44.140 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.139+0000 7f150bfe0740 1 -- 172.21.15.72:0/1430661265 wait complete. 2024-04-26T21:57:44.141 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.139+0000 7f150bfe0740 20 client.16044 trim_cache size 0 max 16384 2024-04-26T21:57:44.154 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.154+0000 7f150bfe0740 1 Processor -- start 2024-04-26T21:57:44.154 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.154+0000 7f150bfe0740 1 -- start start 2024-04-26T21:57:44.155 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.154+0000 7f150bfe0740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56514c239ea0 0x56514c23b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:44.155 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.154+0000 7f150bfe0740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56514c23a890 0x56514c24b020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:44.155 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.154+0000 7f150bfe0740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56514c23fa00 0x56514c23b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:44.155 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.154+0000 7f150bfe0740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f15041b05a0 con 0x56514c239ea0 2024-04-26T21:57:44.155 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.154+0000 7f150bfe0740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f15041b20e0 con 0x56514c23fa00 2024-04-26T21:57:44.155 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.154+0000 7f150bfe0740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f15041b0c70 con 0x56514c23a890 2024-04-26T21:57:44.155 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.155+0000 7f1501e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56514c239ea0 0x56514c23b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:44.155 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.155+0000 7f1501e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56514c239ea0 0x56514c23b120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:48104/0 (socket says 172.21.15.72:48104) 2024-04-26T21:57:44.155 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.155+0000 7f1501e00640 1 -- 172.21.15.72:0/3823852913 learned_addr learned my addr 172.21.15.72:0/3823852913 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:57:44.157 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.155+0000 7f1502800640 1 --2- 172.21.15.72:0/3823852913 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56514c23fa00 0x56514c23b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:44.157 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.155+0000 7f1501400640 1 --2- 172.21.15.72:0/3823852913 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56514c23a890 0x56514c24b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:44.157 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.155+0000 7f1501e00640 1 -- 172.21.15.72:0/3823852913 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56514c23a890 msgr2=0x56514c24b020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:44.157 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.155+0000 7f1501e00640 1 --2- 172.21.15.72:0/3823852913 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56514c23a890 0x56514c24b020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:44.157 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.155+0000 7f1501e00640 1 -- 172.21.15.72:0/3823852913 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56514c23fa00 msgr2=0x56514c23b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:44.157 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.155+0000 7f1501e00640 1 --2- 172.21.15.72:0/3823852913 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56514c23fa00 0x56514c23b660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:44.157 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.155+0000 7f1501e00640 1 -- 172.21.15.72:0/3823852913 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56514bf797d0 con 0x56514c239ea0 2024-04-26T21:57:44.157 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.156+0000 7f1501e00640 1 --2- 172.21.15.72:0/3823852913 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56514c239ea0 0x56514c23b120 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f14f4034040 tx=0x7f14f4005c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b40642f2be147918 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:57:44.157 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.156+0000 7f14eea00640 1 -- 172.21.15.72:0/3823852913 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14f4049040 con 0x56514c239ea0 2024-04-26T21:57:44.157 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.156+0000 7f14eea00640 1 -- 172.21.15.72:0/3823852913 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f14f40fe040 con 0x56514c239ea0 2024-04-26T21:57:44.157 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.156+0000 7f14eea00640 1 -- 172.21.15.72:0/3823852913 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14f40bccb0 con 0x56514c239ea0 2024-04-26T21:57:44.157 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.157+0000 7f150bfe0740 1 -- 172.21.15.72:0/3823852913 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56514c239ea0 msgr2=0x56514c23b120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:44.157 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.157+0000 7f150bfe0740 1 --2- 172.21.15.72:0/3823852913 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56514c239ea0 0x56514c23b120 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f14f4034040 tx=0x7f14f4005c90 comp rx=0 tx=0).stop 2024-04-26T21:57:44.157 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.157+0000 7f150bfe0740 1 -- 172.21.15.72:0/3823852913 shutdown_connections 2024-04-26T21:57:44.158 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.157+0000 7f150bfe0740 1 --2- 172.21.15.72:0/3823852913 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56514c23fa00 0x56514c23b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:44.158 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.157+0000 7f150bfe0740 1 --2- 172.21.15.72:0/3823852913 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56514c23a890 0x56514c24b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:44.158 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.157+0000 7f150bfe0740 1 --2- 172.21.15.72:0/3823852913 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56514c239ea0 0x56514c23b120 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:44.158 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.157+0000 7f150bfe0740 1 -- 172.21.15.72:0/3823852913 >> 172.21.15.72:0/3823852913 conn(0x56514c235510 msgr2=0x56514c234690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:57:44.158 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.157+0000 7f150bfe0740 1 -- 172.21.15.72:0/3823852913 shutdown_connections 2024-04-26T21:57:44.158 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.157+0000 7f150bfe0740 1 -- 172.21.15.72:0/3823852913 wait complete. 2024-04-26T21:57:44.160 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.157+0000 7f150bfe0740 1 -- start start 2024-04-26T21:57:44.161 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.157+0000 7f150bfe0740 1 Processor -- start 2024-04-26T21:57:44.161 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.158+0000 7f150bfe0740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56514c239ea0 0x56514c247b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:44.161 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.158+0000 7f150bfe0740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56514c23a890 0x56514c2480a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:44.161 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.158+0000 7f150bfe0740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56514c23fa00 0x56514c2b3c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:44.161 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.158+0000 7f150bfe0740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f15041b1a10 con 0x56514c23a890 2024-04-26T21:57:44.161 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.158+0000 7f150bfe0740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f15041b1340 con 0x56514c23fa00 2024-04-26T21:57:44.161 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.158+0000 7f150bfe0740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f15041ae580 con 0x56514c239ea0 2024-04-26T21:57:44.161 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.158+0000 7f150bfe0740 10 client.0 _ll_register_callbacks cb 0x7f14f000e390 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:57:44.161 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.158+0000 7f1501400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56514c23a890 0x56514c2480a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:44.161 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.158+0000 7f1501400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56514c23a890 0x56514c2480a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:48118/0 (socket says 172.21.15.72:48118) 2024-04-26T21:57:44.161 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.158+0000 7f1501400640 1 -- 172.21.15.72:0/186756770 learned_addr learned my addr 172.21.15.72:0/186756770 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:57:44.161 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.158+0000 7f1501e00640 1 --2- 172.21.15.72:0/186756770 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56514c239ea0 0x56514c247b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:44.161 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.159+0000 7f1502800640 1 --2- 172.21.15.72:0/186756770 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56514c23fa00 0x56514c2b3c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:44.162 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.159+0000 7f1501400640 1 -- 172.21.15.72:0/186756770 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56514c239ea0 msgr2=0x56514c247b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:44.162 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.159+0000 7f1501400640 1 --2- 172.21.15.72:0/186756770 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56514c239ea0 0x56514c247b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:44.162 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.159+0000 7f1501400640 1 -- 172.21.15.72:0/186756770 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56514c23fa00 msgr2=0x56514c2b3c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:44.162 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.159+0000 7f1501400640 1 --2- 172.21.15.72:0/186756770 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56514c23fa00 0x56514c2b3c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:44.162 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.159+0000 7f1501400640 1 -- 172.21.15.72:0/186756770 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56514c1442a0 con 0x56514c23a890 2024-04-26T21:57:44.162 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.159+0000 7f1501e00640 1 --2- 172.21.15.72:0/186756770 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56514c239ea0 0x56514c247b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:57:44.162 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.159+0000 7f1501400640 1 --2- 172.21.15.72:0/186756770 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56514c23a890 0x56514c2480a0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f14f0010410 tx=0x7f14f0025dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be543f1ba0a86284 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:57:44.162 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.159+0000 7f14e3e00640 10 client.0 ms_handle_connect on v2:172.21.15.72:3300/0 2024-04-26T21:57:44.162 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.159+0000 7f14e3e00640 1 -- 172.21.15.72:0/186756770 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14f0034060 con 0x56514c23a890 2024-04-26T21:57:44.162 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.159+0000 7f14e3e00640 1 -- 172.21.15.72:0/186756770 <== mon.0 v2:172.21.15.72:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7f14f000ccd0 con 0x56514c23a890 2024-04-26T21:57:44.162 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.159+0000 7f150bfe0740 1 -- 172.21.15.72:0/186756770 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f14f40fe040 con 0x56514c23a890 2024-04-26T21:57:44.162 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.160+0000 7f14e3e00640 1 -- 172.21.15.72:0/186756770 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f14f002a050 con 0x56514c23a890 2024-04-26T21:57:44.162 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.160+0000 7f150bfe0740 10 client.16050 fetch_fsmap learned FSMap version 27 2024-04-26T21:57:44.162 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.160+0000 7f150bfe0740 1 -- 172.21.15.72:0/186756770 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56514c2b4ed0 con 0x56514c23a890 2024-04-26T21:57:44.163 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.160+0000 7f14e3e00640 1 -- 172.21.15.72:0/186756770 <== mon.0 v2:172.21.15.72:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f14f40fe040 con 0x56514c23a890 2024-04-26T21:57:44.163 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.160+0000 7f14e3e00640 1 -- 172.21.15.72:0/186756770 <== mon.0 v2:172.21.15.72:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f14f00046f0 con 0x56514c23a890 2024-04-26T21:57:44.163 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.160+0000 7f150bfe0740 10 client.16050 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T21:57:44.163 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.160+0000 7f150bfe0740 10 client.16050 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-26T21:57:44.163 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.160+0000 7f150bfe0740 20 client.16050 populate_metadata read hostname 'smithi072' 2024-04-26T21:57:44.163 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.160+0000 7f150bfe0740 1 --2- 172.21.15.72:0/186756770 >> [v2:172.21.15.76:6834/3517429435,v1:172.21.15.76:6835/3517429435] conn(0x56514c2b5900 0x56514c2b7d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:44.163 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.160+0000 7f150bfe0740 4 client.16050 mds_command: new command op to 34442 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-26T21:57:44.163 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.160+0000 7f150bfe0740 1 -- 172.21.15.72:0/186756770 --> [v2:172.21.15.76:6834/3517429435,v1:172.21.15.76:6835/3517429435] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56514c1a56f0 con 0x56514c2b5900 2024-04-26T21:57:44.163 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.161+0000 7f1501e00640 1 --2- 172.21.15.72:0/186756770 >> [v2:172.21.15.76:6834/3517429435,v1:172.21.15.76:6835/3517429435] conn(0x56514c2b5900 0x56514c2b7d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:44.164 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.162+0000 7f1501e00640 1 --2- 172.21.15.72:0/186756770 >> [v2:172.21.15.76:6834/3517429435,v1:172.21.15.76:6835/3517429435] conn(0x56514c2b5900 0x56514c2b7d20 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d4e4bda0172b1b1d server_cookie=195fd77fe0fb26ae in_seq=0 out_seq=0 2024-04-26T21:57:44.164 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.162+0000 7f14e3e00640 10 client.16050 ms_handle_connect on v2:172.21.15.76:6834/3517429435 2024-04-26T21:57:44.164 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.162+0000 7f14e3e00640 1 -- 172.21.15.72:0/186756770 <== mds.2 v2:172.21.15.76:6834/3517429435 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56514c1a56f0 con 0x56514c2b5900 2024-04-26T21:57:44.165 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.162+0000 7f14e3e00640 10 client.16050 handle_command_reply: tid=0 2024-04-26T21:57:44.165 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.162+0000 7f150bfe0740 1 client.16050 shutdown 2024-04-26T21:57:44.165 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.162+0000 7f150bfe0740 1 -- 172.21.15.72:0/186756770 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56514c23a890 msgr2=0x56514c2480a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:44.165 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.162+0000 7f150bfe0740 1 --2- 172.21.15.72:0/186756770 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56514c23a890 0x56514c2480a0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f14f0010410 tx=0x7f14f0025dc0 comp rx=0 tx=0).stop 2024-04-26T21:57:44.165 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.162+0000 7f150bfe0740 1 -- 172.21.15.72:0/186756770 shutdown_connections 2024-04-26T21:57:44.165 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.162+0000 7f150bfe0740 1 --2- 172.21.15.72:0/186756770 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56514c23fa00 0x56514c2b3c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:44.165 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.162+0000 7f150bfe0740 1 --2- 172.21.15.72:0/186756770 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56514c23a890 0x56514c2480a0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:44.165 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.162+0000 7f150bfe0740 1 --2- 172.21.15.72:0/186756770 >> [v2:172.21.15.76:6834/3517429435,v1:172.21.15.76:6835/3517429435] conn(0x56514c2b5900 0x56514c2b7d20 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:44.165 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.163+0000 7f150bfe0740 1 --2- 172.21.15.72:0/186756770 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56514c239ea0 0x56514c247b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:44.165 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.163+0000 7f14e3e00640 0 client.16050 ms_handle_reset on v2:172.21.15.76:6834/3517429435 2024-04-26T21:57:44.165 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.163+0000 7f150bfe0740 1 -- 172.21.15.72:0/186756770 >> 172.21.15.72:0/186756770 conn(0x56514c235510 msgr2=0x56514c2383f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:57:44.165 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.163+0000 7f150bfe0740 1 -- 172.21.15.72:0/186756770 shutdown_connections 2024-04-26T21:57:44.165 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.163+0000 7f150bfe0740 1 -- 172.21.15.72:0/186756770 wait complete. 2024-04-26T21:57:44.165 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.163+0000 7f150bfe0740 20 client.16050 trim_cache size 0 max 16384 2024-04-26T21:57:44.166 INFO:teuthology.orchestra.run.smithi072.stdout:[] 2024-04-26T21:57:44.166 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.164+0000 7f1509000640 1 -- 172.21.15.72:0/40208498 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f14e80a3c40 msgr2=0x7f14e80a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:44.166 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.164+0000 7f1509000640 1 --2- 172.21.15.72:0/40208498 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f14e80a3c40 0x7f14e80a6100 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x56514c23b0f0 tx=0x7f14fc009740 comp rx=0 tx=0).stop 2024-04-26T21:57:44.167 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.164+0000 7f1509000640 1 -- 172.21.15.72:0/40208498 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1504108f90 msgr2=0x7f150419f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:44.167 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.164+0000 7f1509000640 1 --2- 172.21.15.72:0/40208498 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1504108f90 0x7f150419f130 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f14f4005ab0 tx=0x7f14f40042f0 comp rx=0 tx=0).stop 2024-04-26T21:57:44.167 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.164+0000 7f1509000640 1 -- 172.21.15.72:0/40208498 shutdown_connections 2024-04-26T21:57:44.167 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.164+0000 7f1509000640 1 --2- 172.21.15.72:0/40208498 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f150410ab40 0x7f15041a66e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:44.167 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.164+0000 7f1509000640 1 --2- 172.21.15.72:0/40208498 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f14e80a3c40 0x7f14e80a6100 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:44.167 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.164+0000 7f1509000640 1 --2- 172.21.15.72:0/40208498 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f150410a180 0x7f150419f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:44.167 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.164+0000 7f1509000640 1 --2- 172.21.15.72:0/40208498 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1504108f90 0x7f150419f130 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:44.167 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.164+0000 7f1509000640 1 -- 172.21.15.72:0/40208498 >> 172.21.15.72:0/40208498 conn(0x7f1504077f30 msgr2=0x7f150410d6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:57:44.167 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.164+0000 7f1509000640 1 -- 172.21.15.72:0/40208498 shutdown_connections 2024-04-26T21:57:44.167 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.164+0000 7f1509000640 1 -- 172.21.15.72:0/40208498 wait complete. 2024-04-26T21:57:44.181 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:57:43 smithi072 ceph-mon[28138]: from='client.? 172.21.15.72:0/969622021' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:57:44.183 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-26T21:57:44.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:57:43 smithi076 ceph-mon[31823]: from='client.? 172.21.15.72:0/969622021' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:57:44.394 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.394+0000 7ff481400640 1 -- 172.21.15.72:0/3024844002 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff47c108f90 msgr2=0x7ff47c1093b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:44.394 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.394+0000 7ff481400640 1 --2- 172.21.15.72:0/3024844002 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff47c108f90 0x7ff47c1093b0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7ff46c009900 tx=0x7ff46c02a410 comp rx=0 tx=0).stop 2024-04-26T21:57:44.395 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.394+0000 7ff481400640 1 -- 172.21.15.72:0/3024844002 shutdown_connections 2024-04-26T21:57:44.395 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.394+0000 7ff481400640 1 --2- 172.21.15.72:0/3024844002 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff47c10ab40 0x7ff47c1113d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:44.395 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.394+0000 7ff481400640 1 --2- 172.21.15.72:0/3024844002 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff47c10a180 0x7ff47c10a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:44.395 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.394+0000 7ff481400640 1 --2- 172.21.15.72:0/3024844002 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff47c108f90 0x7ff47c1093b0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:44.395 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.394+0000 7ff481400640 1 -- 172.21.15.72:0/3024844002 >> 172.21.15.72:0/3024844002 conn(0x7ff47c077f30 msgr2=0x7ff47c07a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:57:44.395 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.394+0000 7ff481400640 1 -- 172.21.15.72:0/3024844002 shutdown_connections 2024-04-26T21:57:44.395 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.394+0000 7ff481400640 1 -- 172.21.15.72:0/3024844002 wait complete. 2024-04-26T21:57:44.397 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.395+0000 7ff481400640 1 Processor -- start 2024-04-26T21:57:44.397 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.395+0000 7ff481400640 1 -- start start 2024-04-26T21:57:44.397 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.396+0000 7ff481400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff47c108f90 0x7ff47c19f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:44.397 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.396+0000 7ff481400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff47c10a180 0x7ff47c19f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:44.398 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.396+0000 7ff481400640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff47c10ab40 0x7ff47c1a66c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:44.398 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.396+0000 7ff481400640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7ff47c1149d0 con 0x7ff47c10a180 2024-04-26T21:57:44.398 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.396+0000 7ff481400640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7ff47c114850 con 0x7ff47c10ab40 2024-04-26T21:57:44.398 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.396+0000 7ff481400640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ff47c114b50 con 0x7ff47c108f90 2024-04-26T21:57:44.398 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.396+0000 7ff479e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff47c10a180 0x7ff47c19f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:44.398 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.396+0000 7ff479e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff47c10a180 0x7ff47c19f650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:48132/0 (socket says 172.21.15.72:48132) 2024-04-26T21:57:44.398 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.396+0000 7ff479e00640 1 -- 172.21.15.72:0/975189001 learned_addr learned my addr 172.21.15.72:0/975189001 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:57:44.398 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.396+0000 7ff47b200640 1 --2- 172.21.15.72:0/975189001 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff47c10ab40 0x7ff47c1a66c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:44.398 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.396+0000 7ff47a800640 1 --2- 172.21.15.72:0/975189001 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff47c108f90 0x7ff47c19f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:44.398 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.397+0000 7ff479e00640 1 -- 172.21.15.72:0/975189001 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff47c108f90 msgr2=0x7ff47c19f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:44.398 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.397+0000 7ff479e00640 1 --2- 172.21.15.72:0/975189001 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff47c108f90 0x7ff47c19f110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:44.398 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.397+0000 7ff479e00640 1 -- 172.21.15.72:0/975189001 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff47c10ab40 msgr2=0x7ff47c1a66c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:44.398 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.397+0000 7ff479e00640 1 --2- 172.21.15.72:0/975189001 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff47c10ab40 0x7ff47c1a66c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:44.398 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.397+0000 7ff479e00640 1 -- 172.21.15.72:0/975189001 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff47c1a6d30 con 0x7ff47c10a180 2024-04-26T21:57:44.399 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.397+0000 7ff47a800640 1 --2- 172.21.15.72:0/975189001 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff47c108f90 0x7ff47c19f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:57:44.399 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.397+0000 7ff479e00640 1 --2- 172.21.15.72:0/975189001 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff47c10a180 0x7ff47c19f650 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7ff46400d950 tx=0x7ff46400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:57:44.399 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.397+0000 7ff473400640 1 -- 172.21.15.72:0/975189001 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff464017030 con 0x7ff47c10a180 2024-04-26T21:57:44.399 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.397+0000 7ff473400640 1 -- 172.21.15.72:0/975189001 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff464004020 con 0x7ff47c10a180 2024-04-26T21:57:44.399 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.397+0000 7ff473400640 1 -- 172.21.15.72:0/975189001 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4640055d0 con 0x7ff47c10a180 2024-04-26T21:57:44.399 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.397+0000 7ff481400640 1 -- 172.21.15.72:0/975189001 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff47c1a6fc0 con 0x7ff47c10a180 2024-04-26T21:57:44.399 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.397+0000 7ff481400640 1 -- 172.21.15.72:0/975189001 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff47c1a74e0 con 0x7ff47c10a180 2024-04-26T21:57:44.402 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.402+0000 7ff48436a740 1 Processor -- start 2024-04-26T21:57:44.402 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.402+0000 7ff48436a740 1 -- start start 2024-04-26T21:57:44.403 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.403+0000 7ff48436a740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5595c11daea0 0x5595c11db2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:44.403 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.403+0000 7ff48436a740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5595c11db890 0x5595c11e0390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:44.403 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.403+0000 7ff48436a740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5595c11e0a00 0x5595c11e2e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:44.404 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.403+0000 7ff48436a740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x5595c113ec70 con 0x5595c11daea0 2024-04-26T21:57:44.404 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.403+0000 7ff48436a740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5595c10f7980 con 0x5595c11db890 2024-04-26T21:57:44.404 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.403+0000 7ff48436a740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5595c113ac30 con 0x5595c11e0a00 2024-04-26T21:57:44.404 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.403+0000 7ff473400640 1 -- 172.21.15.72:0/975189001 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7ff464009a30 con 0x7ff47c10a180 2024-04-26T21:57:44.404 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.403+0000 7ff47b200640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5595c11daea0 0x5595c11db2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:44.404 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.403+0000 7ff47b200640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5595c11daea0 0x5595c11db2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:48140/0 (socket says 172.21.15.72:48140) 2024-04-26T21:57:44.404 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.403+0000 7ff47b200640 1 -- 172.21.15.72:0/634881556 learned_addr learned my addr 172.21.15.72:0/634881556 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:57:44.404 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.403+0000 7ff47b200640 1 -- 172.21.15.72:0/634881556 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5595c11e0a00 msgr2=0x5595c11e2e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:44.404 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.403+0000 7ff47b200640 1 --2- 172.21.15.72:0/634881556 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5595c11e0a00 0x5595c11e2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:44.404 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.403+0000 7ff47b200640 1 -- 172.21.15.72:0/634881556 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5595c11db890 msgr2=0x5595c11e0390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:44.404 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.403+0000 7ff47b200640 1 --2- 172.21.15.72:0/634881556 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5595c11db890 0x5595c11e0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:44.404 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.404+0000 7ff47b200640 1 -- 172.21.15.72:0/634881556 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5595c10cf7b0 con 0x5595c11daea0 2024-04-26T21:57:44.405 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.404+0000 7ff47b200640 1 --2- 172.21.15.72:0/634881556 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5595c11daea0 0x5595c11db2c0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7ff474009f40 tx=0x7ff474010860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acfa03ecf04a2821 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:57:44.405 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.404+0000 7ff46b400640 1 -- 172.21.15.72:0/634881556 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff474003b70 con 0x5595c11daea0 2024-04-26T21:57:44.405 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.404+0000 7ff46b400640 1 -- 172.21.15.72:0/634881556 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff474003d10 con 0x5595c11daea0 2024-04-26T21:57:44.405 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.404+0000 7ff46b400640 1 -- 172.21.15.72:0/634881556 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4740118c0 con 0x5595c11daea0 2024-04-26T21:57:44.406 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.405+0000 7ff48436a740 1 -- 172.21.15.72:0/634881556 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5595c11daea0 msgr2=0x5595c11db2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:44.407 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.405+0000 7ff48436a740 1 --2- 172.21.15.72:0/634881556 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5595c11daea0 0x5595c11db2c0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7ff474009f40 tx=0x7ff474010860 comp rx=0 tx=0).stop 2024-04-26T21:57:44.407 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.406+0000 7ff48436a740 1 -- 172.21.15.72:0/634881556 shutdown_connections 2024-04-26T21:57:44.407 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.406+0000 7ff48436a740 1 --2- 172.21.15.72:0/634881556 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5595c11e0a00 0x5595c11e2e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:44.407 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.406+0000 7ff48436a740 1 --2- 172.21.15.72:0/634881556 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5595c11db890 0x5595c11e0390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:44.407 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.406+0000 7ff48436a740 1 --2- 172.21.15.72:0/634881556 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5595c11daea0 0x5595c11db2c0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:44.407 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.406+0000 7ff48436a740 1 -- 172.21.15.72:0/634881556 >> 172.21.15.72:0/634881556 conn(0x5595c11d6510 msgr2=0x5595c11d6910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:57:44.407 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.406+0000 7ff48436a740 1 -- 172.21.15.72:0/634881556 shutdown_connections 2024-04-26T21:57:44.407 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.406+0000 7ff48436a740 1 -- 172.21.15.72:0/634881556 wait complete. 2024-04-26T21:57:44.407 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.406+0000 7ff473400640 1 --2- 172.21.15.72:0/975189001 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7ff4600a3b50 0x7ff4600a6010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:44.407 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.407+0000 7ff48436a740 1 -- start start 2024-04-26T21:57:44.407 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.407+0000 7ff473400640 1 -- 172.21.15.72:0/975189001 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7ff4640d0170 con 0x7ff47c10a180 2024-04-26T21:57:44.407 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.407+0000 7ff47b200640 1 --2- 172.21.15.72:0/975189001 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7ff4600a3b50 0x7ff4600a6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:44.407 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.407+0000 7ff48436a740 1 Processor -- start 2024-04-26T21:57:44.409 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.408+0000 7ff48436a740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5595c11daea0 0x5595c11e4e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:44.409 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.408+0000 7ff48436a740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5595c11db890 0x5595c11e5360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:44.409 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.408+0000 7ff48436a740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5595c11e0a00 0x5595c11ec3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:44.409 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.408+0000 7ff48436a740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x5595c113df80 con 0x5595c11db890 2024-04-26T21:57:44.409 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.408+0000 7ff48436a740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5595c10eb260 con 0x5595c11daea0 2024-04-26T21:57:44.409 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.408+0000 7ff48436a740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5595c11432a0 con 0x5595c11e0a00 2024-04-26T21:57:44.409 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.408+0000 7ff48436a740 10 client.0 _ll_register_callbacks cb 0x5595c1135960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:57:44.409 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.408+0000 7ff47b200640 1 --2- 172.21.15.72:0/975189001 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7ff4600a3b50 0x7ff4600a6010 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7ff47400a010 tx=0x7ff4740046c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:57:44.409 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.408+0000 7ff479e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5595c11db890 0x5595c11e5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:44.409 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.408+0000 7ff479e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5595c11db890 0x5595c11e5360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:48144/0 (socket says 172.21.15.72:48144) 2024-04-26T21:57:44.409 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.408+0000 7ff479e00640 1 -- 172.21.15.72:0/2407352395 learned_addr learned my addr 172.21.15.72:0/2407352395 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:57:44.409 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.408+0000 7ff47a800640 1 --2- 172.21.15.72:0/2407352395 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5595c11daea0 0x5595c11e4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:44.409 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.408+0000 7ff47b200640 1 --2- 172.21.15.72:0/2407352395 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5595c11e0a00 0x5595c11ec3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:44.409 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.409+0000 7ff479e00640 1 -- 172.21.15.72:0/2407352395 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5595c11e0a00 msgr2=0x5595c11ec3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:44.410 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.409+0000 7ff479e00640 1 --2- 172.21.15.72:0/2407352395 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5595c11e0a00 0x5595c11ec3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:44.410 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.409+0000 7ff479e00640 1 -- 172.21.15.72:0/2407352395 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5595c11daea0 msgr2=0x5595c11e4e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:44.410 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.409+0000 7ff479e00640 1 --2- 172.21.15.72:0/2407352395 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5595c11daea0 0x5595c11e4e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:44.410 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.409+0000 7ff479e00640 1 -- 172.21.15.72:0/2407352395 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5595c10cd1c0 con 0x5595c11db890 2024-04-26T21:57:44.411 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.409+0000 7ff479e00640 1 --2- 172.21.15.72:0/2407352395 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5595c11db890 0x5595c11e5360 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7ff4640a4950 tx=0x7ff46409bca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26901233def6483d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:57:44.411 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.409+0000 7ff45fe00640 10 client.0 ms_handle_connect on v2:172.21.15.72:3300/0 2024-04-26T21:57:44.412 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.409+0000 7ff45fe00640 1 -- 172.21.15.72:0/2407352395 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4640a4e80 con 0x5595c11db890 2024-04-26T21:57:44.412 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.409+0000 7ff45fe00640 1 -- 172.21.15.72:0/2407352395 <== mon.0 v2:172.21.15.72:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7ff464098880 con 0x5595c11db890 2024-04-26T21:57:44.412 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.409+0000 7ff48436a740 1 -- 172.21.15.72:0/2407352395 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff474003d10 con 0x5595c11db890 2024-04-26T21:57:44.412 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.410+0000 7ff45fe00640 1 -- 172.21.15.72:0/2407352395 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff4640b5020 con 0x5595c11db890 2024-04-26T21:57:44.412 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.410+0000 7ff48436a740 10 client.16062 fetch_fsmap learned FSMap version 27 2024-04-26T21:57:44.412 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.410+0000 7ff48436a740 1 -- 172.21.15.72:0/2407352395 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5595c0e503f0 con 0x5595c11db890 2024-04-26T21:57:44.412 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.410+0000 7ff45fe00640 1 -- 172.21.15.72:0/2407352395 <== mon.0 v2:172.21.15.72:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff474003d10 con 0x5595c11db890 2024-04-26T21:57:44.412 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.410+0000 7ff45fe00640 1 -- 172.21.15.72:0/2407352395 <== mon.0 v2:172.21.15.72:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7ff4640a0120 con 0x5595c11db890 2024-04-26T21:57:44.412 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.410+0000 7ff48436a740 10 client.16062 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T21:57:44.412 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.410+0000 7ff48436a740 10 client.16062 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-26T21:57:44.412 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.410+0000 7ff48436a740 20 client.16062 populate_metadata read hostname 'smithi072' 2024-04-26T21:57:44.412 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.410+0000 7ff48436a740 1 --2- 172.21.15.72:0/2407352395 >> [v2:172.21.15.76:6836/1150678991,v1:172.21.15.76:6837/1150678991] conn(0x5595c11f0830 0x5595c11f2c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:44.412 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.410+0000 7ff48436a740 4 client.16062 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:57:44.412 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.410+0000 7ff48436a740 1 -- 172.21.15.72:0/2407352395 --> [v2:172.21.15.76:6836/1150678991,v1:172.21.15.76:6837/1150678991] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5595c114c880 con 0x5595c11f0830 2024-04-26T21:57:44.413 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.411+0000 7ff47a800640 1 --2- 172.21.15.72:0/2407352395 >> [v2:172.21.15.76:6836/1150678991,v1:172.21.15.76:6837/1150678991] conn(0x5595c11f0830 0x5595c11f2c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:44.413 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.412+0000 7ff47a800640 1 --2- 172.21.15.72:0/2407352395 >> [v2:172.21.15.76:6836/1150678991,v1:172.21.15.76:6837/1150678991] conn(0x5595c11f0830 0x5595c11f2c20 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a653ac20e5471ab1 server_cookie=51e16a590495a350 in_seq=0 out_seq=0 2024-04-26T21:57:44.413 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.412+0000 7ff45fe00640 10 client.16062 ms_handle_connect on v2:172.21.15.76:6836/1150678991 2024-04-26T21:57:44.414 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.414+0000 7ff45fe00640 1 -- 172.21.15.72:0/2407352395 <== mds.1 v2:172.21.15.76:6836/1150678991 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5595c114c880 con 0x5595c11f0830 2024-04-26T21:57:44.414 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.414+0000 7ff45fe00640 10 client.16062 handle_command_reply: tid=0 2024-04-26T21:57:44.414 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.414+0000 7ff48436a740 1 client.16062 shutdown 2024-04-26T21:57:44.415 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.415+0000 7ff48436a740 1 -- 172.21.15.72:0/2407352395 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5595c11db890 msgr2=0x5595c11e5360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:44.416 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.415+0000 7ff48436a740 1 --2- 172.21.15.72:0/2407352395 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5595c11db890 0x5595c11e5360 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7ff4640a4950 tx=0x7ff46409bca0 comp rx=0 tx=0).stop 2024-04-26T21:57:44.416 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.415+0000 7ff48436a740 1 -- 172.21.15.72:0/2407352395 shutdown_connections 2024-04-26T21:57:44.416 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.415+0000 7ff48436a740 1 --2- 172.21.15.72:0/2407352395 >> [v2:172.21.15.76:6836/1150678991,v1:172.21.15.76:6837/1150678991] conn(0x5595c11f0830 0x5595c11f2c20 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:44.416 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.415+0000 7ff48436a740 1 --2- 172.21.15.72:0/2407352395 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5595c11e0a00 0x5595c11ec3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:44.416 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.415+0000 7ff48436a740 1 --2- 172.21.15.72:0/2407352395 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5595c11db890 0x5595c11e5360 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:44.416 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.415+0000 7ff48436a740 1 --2- 172.21.15.72:0/2407352395 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5595c11daea0 0x5595c11e4e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:44.416 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.415+0000 7ff48436a740 1 -- 172.21.15.72:0/2407352395 >> 172.21.15.72:0/2407352395 conn(0x5595c11d6510 msgr2=0x5595c11d5690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:57:44.416 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.415+0000 7ff45fe00640 0 client.16062 ms_handle_reset on v2:172.21.15.76:6836/1150678991 2024-04-26T21:57:44.416 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.415+0000 7ff48436a740 1 -- 172.21.15.72:0/2407352395 shutdown_connections 2024-04-26T21:57:44.416 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.415+0000 7ff48436a740 1 -- 172.21.15.72:0/2407352395 wait complete. 2024-04-26T21:57:44.416 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.415+0000 7ff48436a740 20 client.16062 trim_cache size 0 max 16384 2024-04-26T21:57:44.431 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.431+0000 7ff48436a740 1 Processor -- start 2024-04-26T21:57:44.431 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.431+0000 7ff48436a740 1 -- start start 2024-04-26T21:57:44.431 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.431+0000 7ff48436a740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5595c11daea0 0x5595c11d9cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:44.432 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.432+0000 7ff48436a740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5595c11db890 0x5595c11ec360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:44.432 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.432+0000 7ff48436a740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5595c11e0a00 0x5595c11e4e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:44.432 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.432+0000 7ff48436a740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7ff47c1b06a0 con 0x5595c11e0a00 2024-04-26T21:57:44.432 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.432+0000 7ff48436a740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7ff47c1b21e0 con 0x5595c11db890 2024-04-26T21:57:44.432 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.432+0000 7ff48436a740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ff47c1b0d70 con 0x5595c11daea0 2024-04-26T21:57:44.433 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.432+0000 7ff47b200640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5595c11e0a00 0x5595c11e4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:44.433 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.432+0000 7ff47b200640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5595c11e0a00 0x5595c11e4e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:48146/0 (socket says 172.21.15.72:48146) 2024-04-26T21:57:44.433 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.432+0000 7ff47b200640 1 -- 172.21.15.72:0/1560963060 learned_addr learned my addr 172.21.15.72:0/1560963060 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:57:44.433 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.432+0000 7ff47a800640 1 --2- 172.21.15.72:0/1560963060 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5595c11daea0 0x5595c11d9cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:44.434 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.432+0000 7ff479e00640 1 --2- 172.21.15.72:0/1560963060 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5595c11db890 0x5595c11ec360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:44.434 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.432+0000 7ff47b200640 1 -- 172.21.15.72:0/1560963060 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5595c11daea0 msgr2=0x5595c11d9cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:44.434 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.432+0000 7ff47b200640 1 --2- 172.21.15.72:0/1560963060 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5595c11daea0 0x5595c11d9cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:44.434 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.432+0000 7ff47b200640 1 -- 172.21.15.72:0/1560963060 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5595c11db890 msgr2=0x5595c11ec360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:44.434 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.432+0000 7ff47b200640 1 --2- 172.21.15.72:0/1560963060 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5595c11db890 0x5595c11ec360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:44.434 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.432+0000 7ff47b200640 1 -- 172.21.15.72:0/1560963060 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5595c0f1a7d0 con 0x5595c11e0a00 2024-04-26T21:57:44.434 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.433+0000 7ff47b200640 1 --2- 172.21.15.72:0/1560963060 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5595c11e0a00 0x5595c11e4e20 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7ff474009ee0 tx=0x7ff474021000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e434427e35256ea3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:57:44.434 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.433+0000 7ff46aa00640 1 -- 172.21.15.72:0/1560963060 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff474004190 con 0x5595c11e0a00 2024-04-26T21:57:44.434 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.433+0000 7ff46aa00640 1 -- 172.21.15.72:0/1560963060 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff474004330 con 0x5595c11e0a00 2024-04-26T21:57:44.434 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.433+0000 7ff46aa00640 1 -- 172.21.15.72:0/1560963060 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff474005040 con 0x5595c11e0a00 2024-04-26T21:57:44.436 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.434+0000 7ff48436a740 1 -- 172.21.15.72:0/1560963060 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5595c11e0a00 msgr2=0x5595c11e4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:44.437 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.434+0000 7ff48436a740 1 --2- 172.21.15.72:0/1560963060 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5595c11e0a00 0x5595c11e4e20 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7ff474009ee0 tx=0x7ff474021000 comp rx=0 tx=0).stop 2024-04-26T21:57:44.437 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.434+0000 7ff48436a740 1 -- 172.21.15.72:0/1560963060 shutdown_connections 2024-04-26T21:57:44.437 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.434+0000 7ff48436a740 1 --2- 172.21.15.72:0/1560963060 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5595c11e0a00 0x5595c11e4e20 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:44.437 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.434+0000 7ff48436a740 1 --2- 172.21.15.72:0/1560963060 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5595c11db890 0x5595c11ec360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:44.437 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.434+0000 7ff48436a740 1 --2- 172.21.15.72:0/1560963060 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5595c11daea0 0x5595c11d9cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:44.437 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.434+0000 7ff48436a740 1 -- 172.21.15.72:0/1560963060 >> 172.21.15.72:0/1560963060 conn(0x5595c11d6510 msgr2=0x5595c11d4e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:57:44.437 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.434+0000 7ff48436a740 1 -- 172.21.15.72:0/1560963060 shutdown_connections 2024-04-26T21:57:44.437 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.434+0000 7ff48436a740 1 -- 172.21.15.72:0/1560963060 wait complete. 2024-04-26T21:57:44.437 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.435+0000 7ff48436a740 1 -- start start 2024-04-26T21:57:44.437 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.435+0000 7ff48436a740 1 Processor -- start 2024-04-26T21:57:44.437 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.435+0000 7ff48436a740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5595c11daea0 0x5595c11e9c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:44.437 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.435+0000 7ff48436a740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5595c11db890 0x5595c11ea140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:44.437 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.435+0000 7ff48436a740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5595c11e0a00 0x5595c11eb150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:44.437 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.435+0000 7ff48436a740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7ff47c1b1b10 con 0x5595c11daea0 2024-04-26T21:57:44.438 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.435+0000 7ff48436a740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7ff47c1b1440 con 0x5595c11e0a00 2024-04-26T21:57:44.438 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.435+0000 7ff48436a740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ff47c1ae680 con 0x5595c11db890 2024-04-26T21:57:44.438 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.436+0000 7ff48436a740 10 client.0 _ll_register_callbacks cb 0x7ff464005770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:57:44.438 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.436+0000 7ff47a800640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5595c11daea0 0x5595c11e9c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:44.438 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.436+0000 7ff47a800640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5595c11daea0 0x5595c11e9c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:48158/0 (socket says 172.21.15.72:48158) 2024-04-26T21:57:44.438 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.436+0000 7ff47a800640 1 -- 172.21.15.72:0/2855706068 learned_addr learned my addr 172.21.15.72:0/2855706068 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:57:44.438 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.436+0000 7ff479e00640 1 --2- 172.21.15.72:0/2855706068 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5595c11db890 0x5595c11ea140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:44.438 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.436+0000 7ff47a800640 1 -- 172.21.15.72:0/2855706068 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5595c11db890 msgr2=0x5595c11ea140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:44.438 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.436+0000 7ff47a800640 1 --2- 172.21.15.72:0/2855706068 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5595c11db890 0x5595c11ea140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:44.438 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.436+0000 7ff47a800640 1 -- 172.21.15.72:0/2855706068 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5595c11e0a00 msgr2=0x5595c11eb150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:44.438 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.436+0000 7ff47a800640 1 --2- 172.21.15.72:0/2855706068 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5595c11e0a00 0x5595c11eb150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:44.438 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.436+0000 7ff47a800640 1 -- 172.21.15.72:0/2855706068 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5595c10e52a0 con 0x5595c11daea0 2024-04-26T21:57:44.439 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.436+0000 7ff479e00640 1 --2- 172.21.15.72:0/2855706068 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5595c11db890 0x5595c11ea140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:57:44.439 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.436+0000 7ff47a800640 1 --2- 172.21.15.72:0/2855706068 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5595c11daea0 0x5595c11e9c00 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7ff46c009900 tx=0x7ff46c0043a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d3bc7620a4f27f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:57:44.439 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.436+0000 7ff45fe00640 10 client.0 ms_handle_connect on v2:172.21.15.72:3300/0 2024-04-26T21:57:44.439 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.437+0000 7ff45fe00640 1 -- 172.21.15.72:0/2855706068 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff46c035040 con 0x5595c11daea0 2024-04-26T21:57:44.439 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.437+0000 7ff45fe00640 1 -- 172.21.15.72:0/2855706068 <== mon.0 v2:172.21.15.72:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7ff46c031910 con 0x5595c11daea0 2024-04-26T21:57:44.439 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.437+0000 7ff48436a740 1 -- 172.21.15.72:0/2855706068 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff474004330 con 0x5595c11daea0 2024-04-26T21:57:44.441 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.437+0000 7ff45fe00640 1 -- 172.21.15.72:0/2855706068 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff46c052050 con 0x5595c11daea0 2024-04-26T21:57:44.441 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.437+0000 7ff48436a740 10 client.16068 fetch_fsmap learned FSMap version 27 2024-04-26T21:57:44.441 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.437+0000 7ff48436a740 1 -- 172.21.15.72:0/2855706068 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5595c1256d30 con 0x5595c11daea0 2024-04-26T21:57:44.441 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.437+0000 7ff45fe00640 1 -- 172.21.15.72:0/2855706068 <== mon.0 v2:172.21.15.72:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff474004330 con 0x5595c11daea0 2024-04-26T21:57:44.441 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.438+0000 7ff45fe00640 1 -- 172.21.15.72:0/2855706068 <== mon.0 v2:172.21.15.72:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7ff46c003cb0 con 0x5595c11daea0 2024-04-26T21:57:44.441 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.438+0000 7ff48436a740 10 client.16068 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T21:57:44.441 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.438+0000 7ff48436a740 10 client.16068 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-26T21:57:44.441 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.438+0000 7ff48436a740 20 client.16068 populate_metadata read hostname 'smithi072' 2024-04-26T21:57:44.441 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.438+0000 7ff48436a740 1 --2- 172.21.15.72:0/2855706068 >> [v2:172.21.15.76:6836/1150678991,v1:172.21.15.76:6837/1150678991] conn(0x5595c1257710 0x5595c1259b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:44.441 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.438+0000 7ff48436a740 4 client.16068 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-26T21:57:44.441 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.438+0000 7ff48436a740 1 -- 172.21.15.72:0/2855706068 --> [v2:172.21.15.76:6836/1150678991,v1:172.21.15.76:6837/1150678991] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5595c11466f0 con 0x5595c1257710 2024-04-26T21:57:44.442 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.438+0000 7ff479e00640 1 --2- 172.21.15.72:0/2855706068 >> [v2:172.21.15.76:6836/1150678991,v1:172.21.15.76:6837/1150678991] conn(0x5595c1257710 0x5595c1259b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:44.442 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.439+0000 7ff479e00640 1 --2- 172.21.15.72:0/2855706068 >> [v2:172.21.15.76:6836/1150678991,v1:172.21.15.76:6837/1150678991] conn(0x5595c1257710 0x5595c1259b30 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d4c4ca3e3463dc4e server_cookie=73d9bf69e9f053 in_seq=0 out_seq=0 2024-04-26T21:57:44.442 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.439+0000 7ff45fe00640 10 client.16068 ms_handle_connect on v2:172.21.15.76:6836/1150678991 2024-04-26T21:57:44.442 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.440+0000 7ff45fe00640 1 -- 172.21.15.72:0/2855706068 <== mds.1 v2:172.21.15.76:6836/1150678991 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5595c11466f0 con 0x5595c1257710 2024-04-26T21:57:44.442 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.440+0000 7ff45fe00640 10 client.16068 handle_command_reply: tid=0 2024-04-26T21:57:44.442 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.440+0000 7ff48436a740 1 client.16068 shutdown 2024-04-26T21:57:44.442 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.440+0000 7ff48436a740 1 -- 172.21.15.72:0/2855706068 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5595c11daea0 msgr2=0x5595c11e9c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:44.442 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.440+0000 7ff48436a740 1 --2- 172.21.15.72:0/2855706068 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5595c11daea0 0x5595c11e9c00 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7ff46c009900 tx=0x7ff46c0043a0 comp rx=0 tx=0).stop 2024-04-26T21:57:44.442 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.440+0000 7ff48436a740 1 -- 172.21.15.72:0/2855706068 shutdown_connections 2024-04-26T21:57:44.442 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.440+0000 7ff48436a740 1 --2- 172.21.15.72:0/2855706068 >> [v2:172.21.15.76:6836/1150678991,v1:172.21.15.76:6837/1150678991] conn(0x5595c1257710 0x5595c1259b30 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:44.442 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.440+0000 7ff48436a740 1 --2- 172.21.15.72:0/2855706068 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5595c11e0a00 0x5595c11eb150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:44.442 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.440+0000 7ff48436a740 1 --2- 172.21.15.72:0/2855706068 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5595c11db890 0x5595c11ea140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:44.443 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.440+0000 7ff48436a740 1 --2- 172.21.15.72:0/2855706068 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5595c11daea0 0x5595c11e9c00 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:44.443 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.440+0000 7ff45fe00640 0 client.16068 ms_handle_reset on v2:172.21.15.76:6836/1150678991 2024-04-26T21:57:44.443 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.440+0000 7ff48436a740 1 -- 172.21.15.72:0/2855706068 >> 172.21.15.72:0/2855706068 conn(0x5595c11d6510 msgr2=0x5595c11d9b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:57:44.443 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.440+0000 7ff48436a740 1 -- 172.21.15.72:0/2855706068 shutdown_connections 2024-04-26T21:57:44.443 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.440+0000 7ff48436a740 1 -- 172.21.15.72:0/2855706068 wait complete. 2024-04-26T21:57:44.443 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.441+0000 7ff48436a740 20 client.16068 trim_cache size 0 max 16384 2024-04-26T21:57:44.443 INFO:teuthology.orchestra.run.smithi072.stdout:[] 2024-04-26T21:57:44.444 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.441+0000 7ff481400640 1 -- 172.21.15.72:0/975189001 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7ff4600a3b50 msgr2=0x7ff4600a6010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:44.444 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.441+0000 7ff481400640 1 --2- 172.21.15.72:0/975189001 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7ff4600a3b50 0x7ff4600a6010 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7ff47400a010 tx=0x7ff4740046c0 comp rx=0 tx=0).stop 2024-04-26T21:57:44.444 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.441+0000 7ff481400640 1 -- 172.21.15.72:0/975189001 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff47c10a180 msgr2=0x7ff47c19f650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:44.444 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.441+0000 7ff481400640 1 --2- 172.21.15.72:0/975189001 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff47c10a180 0x7ff47c19f650 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7ff46400d950 tx=0x7ff46400de10 comp rx=0 tx=0).stop 2024-04-26T21:57:44.444 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.441+0000 7ff481400640 1 -- 172.21.15.72:0/975189001 shutdown_connections 2024-04-26T21:57:44.444 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.441+0000 7ff481400640 1 --2- 172.21.15.72:0/975189001 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff47c10ab40 0x7ff47c1a66c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:44.445 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.441+0000 7ff481400640 1 --2- 172.21.15.72:0/975189001 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff47c10a180 0x7ff47c19f650 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:44.445 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.441+0000 7ff481400640 1 --2- 172.21.15.72:0/975189001 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7ff4600a3b50 0x7ff4600a6010 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:44.445 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.441+0000 7ff481400640 1 --2- 172.21.15.72:0/975189001 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff47c108f90 0x7ff47c19f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:44.445 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.441+0000 7ff481400640 1 -- 172.21.15.72:0/975189001 >> 172.21.15.72:0/975189001 conn(0x7ff47c077f30 msgr2=0x7ff47c10d540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:57:44.445 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.442+0000 7ff481400640 1 -- 172.21.15.72:0/975189001 shutdown_connections 2024-04-26T21:57:44.445 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.442+0000 7ff481400640 1 -- 172.21.15.72:0/975189001 wait complete. 2024-04-26T21:57:44.445 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:57:43 smithi164 ceph-mon[31916]: from='client.? 172.21.15.72:0/969622021' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:57:44.459 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-26T21:57:44.666 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.665+0000 7f4cbf400640 1 -- 172.21.15.72:0/1606632508 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4cb8104b90 msgr2=0x7f4cb8104fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:44.670 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.665+0000 7f4cbf400640 1 --2- 172.21.15.72:0/1606632508 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4cb8104b90 0x7f4cb8104fb0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f4cac009900 tx=0x7f4cac02a400 comp rx=0 tx=0).stop 2024-04-26T21:57:44.670 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.666+0000 7f4cbf400640 1 -- 172.21.15.72:0/1606632508 shutdown_connections 2024-04-26T21:57:44.670 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.666+0000 7f4cbf400640 1 --2- 172.21.15.72:0/1606632508 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4cb8106740 0x7f4cb810cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:44.670 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.666+0000 7f4cbf400640 1 --2- 172.21.15.72:0/1606632508 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4cb8105d80 0x7f4cb8106200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:44.671 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.666+0000 7f4cbf400640 1 --2- 172.21.15.72:0/1606632508 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4cb8104b90 0x7f4cb8104fb0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:44.671 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.666+0000 7f4cbf400640 1 -- 172.21.15.72:0/1606632508 >> 172.21.15.72:0/1606632508 conn(0x7f4cb81000e0 msgr2=0x7f4cb8102540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:57:44.671 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.666+0000 7f4cbf400640 1 -- 172.21.15.72:0/1606632508 shutdown_connections 2024-04-26T21:57:44.671 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.666+0000 7f4cbf400640 1 -- 172.21.15.72:0/1606632508 wait complete. 2024-04-26T21:57:44.673 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.667+0000 7f4cbf400640 1 Processor -- start 2024-04-26T21:57:44.673 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.667+0000 7f4cbf400640 1 -- start start 2024-04-26T21:57:44.673 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.668+0000 7f4cbf400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4cb8104b90 0x7f4cb819ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:44.673 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.668+0000 7f4cbf400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4cb8105d80 0x7f4cb819b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:44.673 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.668+0000 7f4cbf400640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4cb8106740 0x7f4cb81a21c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:44.673 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.668+0000 7f4cbf400640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f4cb8110590 con 0x7f4cb8104b90 2024-04-26T21:57:44.673 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.668+0000 7f4cbf400640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f4cb8110410 con 0x7f4cb8106740 2024-04-26T21:57:44.673 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.668+0000 7f4cbf400640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f4cb8110710 con 0x7f4cb8105d80 2024-04-26T21:57:44.673 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.668+0000 7f4cbca00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4cb8104b90 0x7f4cb819ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:44.673 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.668+0000 7f4cbca00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4cb8104b90 0x7f4cb819ac50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:48168/0 (socket says 172.21.15.72:48168) 2024-04-26T21:57:44.673 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.668+0000 7f4cbca00640 1 -- 172.21.15.72:0/1121575878 learned_addr learned my addr 172.21.15.72:0/1121575878 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:57:44.673 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.668+0000 7f4cbd400640 1 --2- 172.21.15.72:0/1121575878 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4cb8106740 0x7f4cb81a21c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:44.674 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.668+0000 7f4cbca00640 1 -- 172.21.15.72:0/1121575878 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4cb8105d80 msgr2=0x7f4cb819b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:44.674 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.668+0000 7f4cbca00640 1 --2- 172.21.15.72:0/1121575878 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4cb8105d80 0x7f4cb819b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:44.674 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.668+0000 7f4cbca00640 1 -- 172.21.15.72:0/1121575878 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4cb8106740 msgr2=0x7f4cb81a21c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:44.674 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.668+0000 7f4cbca00640 1 --2- 172.21.15.72:0/1121575878 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4cb8106740 0x7f4cb81a21c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:44.674 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.668+0000 7f4cbca00640 1 -- 172.21.15.72:0/1121575878 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cb81a2860 con 0x7f4cb8104b90 2024-04-26T21:57:44.674 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.668+0000 7f4cbd400640 1 --2- 172.21.15.72:0/1121575878 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4cb8106740 0x7f4cb81a21c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:57:44.674 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.669+0000 7f4cbca00640 1 --2- 172.21.15.72:0/1121575878 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4cb8104b90 0x7f4cb819ac50 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f4cac031460 tx=0x7f4cac003d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:57:44.675 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.669+0000 7f4cb5600640 1 -- 172.21.15.72:0/1121575878 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cac03f070 con 0x7f4cb8104b90 2024-04-26T21:57:44.675 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.669+0000 7f4cb5600640 1 -- 172.21.15.72:0/1121575878 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4cac031da0 con 0x7f4cb8104b90 2024-04-26T21:57:44.675 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.669+0000 7f4cbf400640 1 -- 172.21.15.72:0/1121575878 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4cb81a2af0 con 0x7f4cb8104b90 2024-04-26T21:57:44.675 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.669+0000 7f4cb5600640 1 -- 172.21.15.72:0/1121575878 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cac035030 con 0x7f4cb8104b90 2024-04-26T21:57:44.675 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.669+0000 7f4cbf400640 1 -- 172.21.15.72:0/1121575878 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4cb81a2fe0 con 0x7f4cb8104b90 2024-04-26T21:57:44.677 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.674+0000 7f4cc23c1740 1 Processor -- start 2024-04-26T21:57:44.678 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.674+0000 7f4cc23c1740 1 -- start start 2024-04-26T21:57:44.678 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.674+0000 7f4cb5600640 1 -- 172.21.15.72:0/1121575878 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f4cac042070 con 0x7f4cb8104b90 2024-04-26T21:57:44.678 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.674+0000 7f4cc23c1740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556f75924ea0 0x556f759252c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:44.678 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.674+0000 7f4cc23c1740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556f75925890 0x556f7592a390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:44.678 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.674+0000 7f4cc23c1740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556f7592aa00 0x556f7592ce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:44.678 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.674+0000 7f4cc23c1740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x556f75888c70 con 0x556f75924ea0 2024-04-26T21:57:44.678 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.674+0000 7f4cc23c1740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x556f75841980 con 0x556f75925890 2024-04-26T21:57:44.678 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.674+0000 7f4cc23c1740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x556f75884c30 con 0x556f7592aa00 2024-04-26T21:57:44.678 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.675+0000 7f4cbd400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556f75924ea0 0x556f759252c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:44.678 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.675+0000 7f4cbd400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556f75924ea0 0x556f759252c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:48176/0 (socket says 172.21.15.72:48176) 2024-04-26T21:57:44.678 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.675+0000 7f4cbd400640 1 -- 172.21.15.72:0/2707477473 learned_addr learned my addr 172.21.15.72:0/2707477473 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:57:44.678 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.675+0000 7f4cbca00640 1 --2- 172.21.15.72:0/2707477473 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556f75925890 0x556f7592a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:44.678 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.675+0000 7f4cb7e00640 1 --2- 172.21.15.72:0/2707477473 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556f7592aa00 0x556f7592ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:44.679 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.675+0000 7f4cbd400640 1 -- 172.21.15.72:0/2707477473 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556f7592aa00 msgr2=0x556f7592ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:44.679 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.675+0000 7f4cbd400640 1 --2- 172.21.15.72:0/2707477473 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556f7592aa00 0x556f7592ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:44.679 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.675+0000 7f4cbd400640 1 -- 172.21.15.72:0/2707477473 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556f75925890 msgr2=0x556f7592a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:44.679 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.675+0000 7f4cbd400640 1 --2- 172.21.15.72:0/2707477473 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556f75925890 0x556f7592a390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:44.679 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.675+0000 7f4cbd400640 1 -- 172.21.15.72:0/2707477473 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f758197b0 con 0x556f75924ea0 2024-04-26T21:57:44.679 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.676+0000 7f4cbd400640 1 --2- 172.21.15.72:0/2707477473 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556f75924ea0 0x556f759252c0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f4cb0008e60 tx=0x7f4cb000d710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71feb4b34a19d390 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:57:44.679 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.676+0000 7f4ca9c00640 1 -- 172.21.15.72:0/2707477473 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cb000f830 con 0x556f75924ea0 2024-04-26T21:57:44.679 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.676+0000 7f4ca9c00640 1 -- 172.21.15.72:0/2707477473 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4cb0018480 con 0x556f75924ea0 2024-04-26T21:57:44.679 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.676+0000 7f4ca9c00640 1 -- 172.21.15.72:0/2707477473 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cb000fe60 con 0x556f75924ea0 2024-04-26T21:57:44.679 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.676+0000 7f4cb5600640 1 --2- 172.21.15.72:0/1121575878 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f4ca00a3c40 0x7f4ca00a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:44.679 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.676+0000 7f4cb5600640 1 -- 172.21.15.72:0/1121575878 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7f4cac0f10f0 con 0x7f4cb8104b90 2024-04-26T21:57:44.679 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.677+0000 7f4cbca00640 1 --2- 172.21.15.72:0/1121575878 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f4ca00a3c40 0x7f4ca00a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:44.679 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.677+0000 7f4cc23c1740 1 -- 172.21.15.72:0/2707477473 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556f75924ea0 msgr2=0x556f759252c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:44.679 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.677+0000 7f4cc23c1740 1 --2- 172.21.15.72:0/2707477473 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556f75924ea0 0x556f759252c0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f4cb0008e60 tx=0x7f4cb000d710 comp rx=0 tx=0).stop 2024-04-26T21:57:44.680 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.677+0000 7f4cc23c1740 1 -- 172.21.15.72:0/2707477473 shutdown_connections 2024-04-26T21:57:44.680 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.677+0000 7f4cc23c1740 1 --2- 172.21.15.72:0/2707477473 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556f7592aa00 0x556f7592ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:44.680 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.677+0000 7f4cc23c1740 1 --2- 172.21.15.72:0/2707477473 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556f75925890 0x556f7592a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:44.680 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.677+0000 7f4cc23c1740 1 --2- 172.21.15.72:0/2707477473 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556f75924ea0 0x556f759252c0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:44.680 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.677+0000 7f4cc23c1740 1 -- 172.21.15.72:0/2707477473 >> 172.21.15.72:0/2707477473 conn(0x556f75920510 msgr2=0x556f75920910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:57:44.680 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.677+0000 7f4cbca00640 1 --2- 172.21.15.72:0/1121575878 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f4ca00a3c40 0x7f4ca00a6100 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x556f75926300 tx=0x7f4cac000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:57:44.680 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.678+0000 7f4cc23c1740 1 -- 172.21.15.72:0/2707477473 shutdown_connections 2024-04-26T21:57:44.680 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.678+0000 7f4cc23c1740 1 -- 172.21.15.72:0/2707477473 wait complete. 2024-04-26T21:57:44.683 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.678+0000 7f4cc23c1740 1 -- start start 2024-04-26T21:57:44.683 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.678+0000 7f4cc23c1740 1 Processor -- start 2024-04-26T21:57:44.683 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.679+0000 7f4cc23c1740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556f75924ea0 0x556f7592ede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:44.683 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.679+0000 7f4cc23c1740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556f75925890 0x556f7592f320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:44.683 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.679+0000 7f4cc23c1740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556f7592aa00 0x556f75936360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:44.683 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.679+0000 7f4cc23c1740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x556f75887f80 con 0x556f75925890 2024-04-26T21:57:44.683 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.679+0000 7f4cc23c1740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x556f75835260 con 0x556f75924ea0 2024-04-26T21:57:44.683 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.679+0000 7f4cc23c1740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x556f7588d2a0 con 0x556f7592aa00 2024-04-26T21:57:44.683 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.679+0000 7f4cc23c1740 10 client.0 _ll_register_callbacks cb 0x556f7587f960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:57:44.683 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.679+0000 7f4cb7e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556f75925890 0x556f7592f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:44.684 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.679+0000 7f4cb7e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556f75925890 0x556f7592f320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:48190/0 (socket says 172.21.15.72:48190) 2024-04-26T21:57:44.684 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.679+0000 7f4cb7e00640 1 -- 172.21.15.72:0/498805090 learned_addr learned my addr 172.21.15.72:0/498805090 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:57:44.684 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.679+0000 7f4cbd400640 1 --2- 172.21.15.72:0/498805090 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556f75924ea0 0x556f7592ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:44.684 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.680+0000 7f4cbd400640 1 --2- 172.21.15.72:0/498805090 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556f7592aa00 0x556f75936360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:44.684 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.680+0000 7f4cb7e00640 1 -- 172.21.15.72:0/498805090 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556f7592aa00 msgr2=0x556f75936360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:44.684 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.680+0000 7f4cb7e00640 1 --2- 172.21.15.72:0/498805090 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556f7592aa00 0x556f75936360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:44.684 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.680+0000 7f4cb7e00640 1 -- 172.21.15.72:0/498805090 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556f75924ea0 msgr2=0x556f7592ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:44.684 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.680+0000 7f4cb7e00640 1 --2- 172.21.15.72:0/498805090 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556f75924ea0 0x556f7592ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:44.684 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.680+0000 7f4cb7e00640 1 -- 172.21.15.72:0/498805090 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556f758171c0 con 0x556f75925890 2024-04-26T21:57:44.684 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.680+0000 7f4cb7e00640 1 --2- 172.21.15.72:0/498805090 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556f75925890 0x556f7592f320 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f4ca400e680 tx=0x7f4ca400eb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21008c2bd1e36c39 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:57:44.684 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.680+0000 7f4c9e000640 10 client.0 ms_handle_connect on v2:172.21.15.72:3300/0 2024-04-26T21:57:44.684 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.680+0000 7f4c9e000640 1 -- 172.21.15.72:0/498805090 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ca4004220 con 0x556f75925890 2024-04-26T21:57:44.684 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.680+0000 7f4cc23c1740 1 -- 172.21.15.72:0/498805090 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4cb0018480 con 0x556f75925890 2024-04-26T21:57:44.684 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.681+0000 7f4c9e000640 1 -- 172.21.15.72:0/498805090 <== mon.0 v2:172.21.15.72:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7f4ca40043c0 con 0x556f75925890 2024-04-26T21:57:44.685 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.681+0000 7f4c9e000640 1 -- 172.21.15.72:0/498805090 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4ca401f030 con 0x556f75925890 2024-04-26T21:57:44.685 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.681+0000 7f4cc23c1740 10 client.16080 fetch_fsmap learned FSMap version 27 2024-04-26T21:57:44.685 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.681+0000 7f4cc23c1740 1 -- 172.21.15.72:0/498805090 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556f7559a3f0 con 0x556f75925890 2024-04-26T21:57:44.685 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.681+0000 7f4c9e000640 1 -- 172.21.15.72:0/498805090 <== mon.0 v2:172.21.15.72:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4cb0018480 con 0x556f75925890 2024-04-26T21:57:44.685 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.681+0000 7f4c9e000640 1 -- 172.21.15.72:0/498805090 <== mon.0 v2:172.21.15.72:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f4ca4003ad0 con 0x556f75925890 2024-04-26T21:57:44.685 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.681+0000 7f4cc23c1740 10 client.16080 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T21:57:44.685 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.681+0000 7f4cc23c1740 10 client.16080 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:57:44.685 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.681+0000 7f4cc23c1740 20 client.16080 populate_metadata read hostname 'smithi072' 2024-04-26T21:57:44.685 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.681+0000 7f4cc23c1740 1 --2- 172.21.15.72:0/498805090 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x556f7593a900 0x556f7593cd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:44.685 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.681+0000 7f4cc23c1740 4 client.16080 mds_command: new command op to 34517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:57:44.685 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.681+0000 7f4cc23c1740 1 -- 172.21.15.72:0/498805090 --> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556f75896880 con 0x556f7593a900 2024-04-26T21:57:44.685 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.682+0000 7f4cbd400640 1 --2- 172.21.15.72:0/498805090 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x556f7593a900 0x556f7593cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:44.685 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.683+0000 7f4c9e000640 10 client.16080 ms_handle_connect on v2:172.21.15.76:6838/327145636 2024-04-26T21:57:44.685 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.683+0000 7f4cbd400640 1 --2- 172.21.15.72:0/498805090 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x556f7593a900 0x556f7593cd30 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b219fc7ac7b8b06f server_cookie=fb5d92438ae8a5fc in_seq=0 out_seq=0 2024-04-26T21:57:44.686 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.685+0000 7f4c9e000640 1 -- 172.21.15.72:0/498805090 <== mds.0 v2:172.21.15.76:6838/327145636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556f75896880 con 0x556f7593a900 2024-04-26T21:57:44.687 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.685+0000 7f4c9e000640 10 client.16080 handle_command_reply: tid=0 2024-04-26T21:57:44.687 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.685+0000 7f4cc23c1740 1 client.16080 shutdown 2024-04-26T21:57:44.687 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.685+0000 7f4cc23c1740 1 -- 172.21.15.72:0/498805090 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556f75925890 msgr2=0x556f7592f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:44.687 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.685+0000 7f4cc23c1740 1 --2- 172.21.15.72:0/498805090 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556f75925890 0x556f7592f320 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f4ca400e680 tx=0x7f4ca400eb40 comp rx=0 tx=0).stop 2024-04-26T21:57:44.687 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.685+0000 7f4cc23c1740 1 -- 172.21.15.72:0/498805090 shutdown_connections 2024-04-26T21:57:44.687 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.685+0000 7f4cc23c1740 1 --2- 172.21.15.72:0/498805090 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x556f7593a900 0x556f7593cd30 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:44.687 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.686+0000 7f4cc23c1740 1 --2- 172.21.15.72:0/498805090 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556f7592aa00 0x556f75936360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:44.687 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.686+0000 7f4c9e000640 0 client.16080 ms_handle_reset on v2:172.21.15.76:6838/327145636 2024-04-26T21:57:44.687 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.686+0000 7f4cc23c1740 1 --2- 172.21.15.72:0/498805090 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556f75925890 0x556f7592f320 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:44.687 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.686+0000 7f4cc23c1740 1 --2- 172.21.15.72:0/498805090 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556f75924ea0 0x556f7592ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:44.687 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.686+0000 7f4cc23c1740 1 -- 172.21.15.72:0/498805090 >> 172.21.15.72:0/498805090 conn(0x556f75920510 msgr2=0x556f7591f690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:57:44.687 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.686+0000 7f4cc23c1740 1 -- 172.21.15.72:0/498805090 shutdown_connections 2024-04-26T21:57:44.687 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.686+0000 7f4cc23c1740 1 -- 172.21.15.72:0/498805090 wait complete. 2024-04-26T21:57:44.687 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.686+0000 7f4cc23c1740 20 client.16080 trim_cache size 0 max 16384 2024-04-26T21:57:44.703 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.702+0000 7f4cc23c1740 1 Processor -- start 2024-04-26T21:57:44.704 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.703+0000 7f4cc23c1740 1 -- start start 2024-04-26T21:57:44.704 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.703+0000 7f4cc23c1740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556f75924ea0 0x556f759231f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:44.704 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.703+0000 7f4cc23c1740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556f75925890 0x556f75936320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:44.704 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.703+0000 7f4cc23c1740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556f7592aa00 0x556f75923730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:44.704 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.703+0000 7f4cc23c1740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f4cb81ac050 con 0x556f75925890 2024-04-26T21:57:44.704 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.703+0000 7f4cc23c1740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f4cb81adb90 con 0x556f7592aa00 2024-04-26T21:57:44.704 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.703+0000 7f4cc23c1740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f4cb81ac720 con 0x556f75924ea0 2024-04-26T21:57:44.704 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.703+0000 7f4cb7e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556f75925890 0x556f75936320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:44.704 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.703+0000 7f4cb7e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556f75925890 0x556f75936320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:48206/0 (socket says 172.21.15.72:48206) 2024-04-26T21:57:44.704 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.703+0000 7f4cb7e00640 1 -- 172.21.15.72:0/4265691147 learned_addr learned my addr 172.21.15.72:0/4265691147 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:57:44.704 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.704+0000 7f4cb7e00640 1 -- 172.21.15.72:0/4265691147 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556f75924ea0 msgr2=0x556f759231f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:44.704 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.704+0000 7f4cbd400640 1 --2- 172.21.15.72:0/4265691147 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556f75924ea0 0x556f759231f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:44.705 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.704+0000 7f4cb7e00640 1 --2- 172.21.15.72:0/4265691147 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556f75924ea0 0x556f759231f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:44.705 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.704+0000 7f4cb7e00640 1 -- 172.21.15.72:0/4265691147 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556f7592aa00 msgr2=0x556f75923730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:44.705 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.704+0000 7f4cb7e00640 1 --2- 172.21.15.72:0/4265691147 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556f7592aa00 0x556f75923730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:44.705 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.704+0000 7f4cb7e00640 1 -- 172.21.15.72:0/4265691147 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f756647d0 con 0x556f75925890 2024-04-26T21:57:44.705 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.704+0000 7f4cbd400640 1 --2- 172.21.15.72:0/4265691147 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556f75924ea0 0x556f759231f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:57:44.708 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.704+0000 7f4cb7e00640 1 --2- 172.21.15.72:0/4265691147 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556f75925890 0x556f75936320 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f4ca4000c00 tx=0x7f4ca4004660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae104ad7a704a887 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:57:44.708 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.704+0000 7f4ca9200640 1 -- 172.21.15.72:0/4265691147 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ca4029070 con 0x556f75925890 2024-04-26T21:57:44.708 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.705+0000 7f4ca9200640 1 -- 172.21.15.72:0/4265691147 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4ca40246f0 con 0x556f75925890 2024-04-26T21:57:44.708 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.705+0000 7f4ca9200640 1 -- 172.21.15.72:0/4265691147 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ca4005280 con 0x556f75925890 2024-04-26T21:57:44.708 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.705+0000 7f4cc23c1740 1 -- 172.21.15.72:0/4265691147 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556f75925890 msgr2=0x556f75936320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:44.708 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.705+0000 7f4cc23c1740 1 --2- 172.21.15.72:0/4265691147 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556f75925890 0x556f75936320 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f4ca4000c00 tx=0x7f4ca4004660 comp rx=0 tx=0).stop 2024-04-26T21:57:44.708 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.705+0000 7f4cc23c1740 1 -- 172.21.15.72:0/4265691147 shutdown_connections 2024-04-26T21:57:44.709 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.705+0000 7f4cc23c1740 1 --2- 172.21.15.72:0/4265691147 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556f7592aa00 0x556f75923730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:44.709 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.705+0000 7f4cc23c1740 1 --2- 172.21.15.72:0/4265691147 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556f75925890 0x556f75936320 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:44.709 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.705+0000 7f4cc23c1740 1 --2- 172.21.15.72:0/4265691147 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556f75924ea0 0x556f759231f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:44.709 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.705+0000 7f4cc23c1740 1 -- 172.21.15.72:0/4265691147 >> 172.21.15.72:0/4265691147 conn(0x556f75920510 msgr2=0x556f7591f690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:57:44.709 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.706+0000 7f4cc23c1740 1 -- 172.21.15.72:0/4265691147 shutdown_connections 2024-04-26T21:57:44.709 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.706+0000 7f4cc23c1740 1 -- 172.21.15.72:0/4265691147 wait complete. 2024-04-26T21:57:44.709 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.706+0000 7f4cc23c1740 1 -- start start 2024-04-26T21:57:44.709 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.706+0000 7f4cc23c1740 1 Processor -- start 2024-04-26T21:57:44.709 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.706+0000 7f4cc23c1740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556f75924ea0 0x556f75933ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:44.709 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.707+0000 7f4cc23c1740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556f75925890 0x556f75934000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:44.709 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.707+0000 7f4cc23c1740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556f7592aa00 0x556f75935010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:44.709 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.707+0000 7f4cc23c1740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f4cb81ad4c0 con 0x556f7592aa00 2024-04-26T21:57:44.709 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.707+0000 7f4cc23c1740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f4cb81acdf0 con 0x556f75925890 2024-04-26T21:57:44.710 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.707+0000 7f4cc23c1740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f4cb81aa030 con 0x556f75924ea0 2024-04-26T21:57:44.710 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.707+0000 7f4cc23c1740 10 client.0 _ll_register_callbacks cb 0x7f4ca400c330 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:57:44.710 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.707+0000 7f4cbd400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556f7592aa00 0x556f75935010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:44.710 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.707+0000 7f4cbd400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556f7592aa00 0x556f75935010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:48212/0 (socket says 172.21.15.72:48212) 2024-04-26T21:57:44.710 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.707+0000 7f4cbd400640 1 -- 172.21.15.72:0/316608371 learned_addr learned my addr 172.21.15.72:0/316608371 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:57:44.710 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.707+0000 7f4cb7e00640 1 --2- 172.21.15.72:0/316608371 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556f75925890 0x556f75934000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:44.710 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.707+0000 7f4cbd400640 1 --2- 172.21.15.72:0/316608371 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556f75924ea0 0x556f75933ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:44.710 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.707+0000 7f4cbd400640 1 -- 172.21.15.72:0/316608371 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556f75924ea0 msgr2=0x556f75933ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:44.710 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.707+0000 7f4cbd400640 1 --2- 172.21.15.72:0/316608371 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556f75924ea0 0x556f75933ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:44.710 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.707+0000 7f4cbd400640 1 -- 172.21.15.72:0/316608371 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556f75925890 msgr2=0x556f75934000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:44.710 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.707+0000 7f4cbd400640 1 --2- 172.21.15.72:0/316608371 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556f75925890 0x556f75934000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:44.710 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.707+0000 7f4cbd400640 1 -- 172.21.15.72:0/316608371 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556f7582f2a0 con 0x556f7592aa00 2024-04-26T21:57:44.710 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.708+0000 7f4cbd400640 1 --2- 172.21.15.72:0/316608371 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556f7592aa00 0x556f75935010 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f4cb0008e30 tx=0x7f4cb0008730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a02c230862b1e9cc server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:57:44.710 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.708+0000 7f4c9e000640 10 client.0 ms_handle_connect on v2:172.21.15.72:3300/0 2024-04-26T21:57:44.711 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.708+0000 7f4c9e000640 1 -- 172.21.15.72:0/316608371 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cb0018cb0 con 0x556f7592aa00 2024-04-26T21:57:44.711 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.708+0000 7f4c9e000640 1 -- 172.21.15.72:0/316608371 <== mon.0 v2:172.21.15.72:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7f4cb0018480 con 0x556f7592aa00 2024-04-26T21:57:44.711 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.708+0000 7f4cc23c1740 1 -- 172.21.15.72:0/316608371 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ca40246f0 con 0x556f7592aa00 2024-04-26T21:57:44.711 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.708+0000 7f4c9e000640 1 -- 172.21.15.72:0/316608371 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4cb0018680 con 0x556f7592aa00 2024-04-26T21:57:44.711 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.708+0000 7f4cc23c1740 10 client.16086 fetch_fsmap learned FSMap version 27 2024-04-26T21:57:44.711 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.708+0000 7f4cc23c1740 1 -- 172.21.15.72:0/316608371 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556f759a0eb0 con 0x556f7592aa00 2024-04-26T21:57:44.713 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.709+0000 7f4c9e000640 1 -- 172.21.15.72:0/316608371 <== mon.0 v2:172.21.15.72:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4ca40246f0 con 0x556f7592aa00 2024-04-26T21:57:44.713 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.709+0000 7f4c9e000640 1 -- 172.21.15.72:0/316608371 <== mon.0 v2:172.21.15.72:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f4cb000eca0 con 0x556f7592aa00 2024-04-26T21:57:44.713 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.709+0000 7f4cc23c1740 10 client.16086 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T21:57:44.713 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.709+0000 7f4cc23c1740 10 client.16086 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:57:44.713 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.709+0000 7f4cc23c1740 20 client.16086 populate_metadata read hostname 'smithi072' 2024-04-26T21:57:44.713 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.709+0000 7f4cc23c1740 1 --2- 172.21.15.72:0/316608371 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x556f759a15d0 0x556f759a39f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:44.713 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.709+0000 7f4cc23c1740 4 client.16086 mds_command: new command op to 34517 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-26T21:57:44.713 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.709+0000 7f4cc23c1740 1 -- 172.21.15.72:0/316608371 --> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556f758906f0 con 0x556f759a15d0 2024-04-26T21:57:44.713 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.710+0000 7f4cb7e00640 1 --2- 172.21.15.72:0/316608371 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x556f759a15d0 0x556f759a39f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:44.713 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.711+0000 7f4cb7e00640 1 --2- 172.21.15.72:0/316608371 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x556f759a15d0 0x556f759a39f0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d4896c8642459ce1 server_cookie=60b7eccd135905b6 in_seq=0 out_seq=0 2024-04-26T21:57:44.713 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.711+0000 7f4c9e000640 10 client.16086 ms_handle_connect on v2:172.21.15.76:6838/327145636 2024-04-26T21:57:44.714 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.711+0000 7f4c9e000640 1 -- 172.21.15.72:0/316608371 <== mds.0 v2:172.21.15.76:6838/327145636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556f758906f0 con 0x556f759a15d0 2024-04-26T21:57:44.714 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.711+0000 7f4c9e000640 10 client.16086 handle_command_reply: tid=0 2024-04-26T21:57:44.714 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.711+0000 7f4cc23c1740 1 client.16086 shutdown 2024-04-26T21:57:44.714 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.711+0000 7f4cc23c1740 1 -- 172.21.15.72:0/316608371 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556f7592aa00 msgr2=0x556f75935010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:44.714 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.711+0000 7f4cc23c1740 1 --2- 172.21.15.72:0/316608371 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556f7592aa00 0x556f75935010 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f4cb0008e30 tx=0x7f4cb0008730 comp rx=0 tx=0).stop 2024-04-26T21:57:44.714 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.712+0000 7f4cc23c1740 1 -- 172.21.15.72:0/316608371 shutdown_connections 2024-04-26T21:57:44.714 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.712+0000 7f4cc23c1740 1 --2- 172.21.15.72:0/316608371 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x556f759a15d0 0x556f759a39f0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:44.714 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.712+0000 7f4cc23c1740 1 --2- 172.21.15.72:0/316608371 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556f7592aa00 0x556f75935010 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:44.714 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.712+0000 7f4cc23c1740 1 --2- 172.21.15.72:0/316608371 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556f75925890 0x556f75934000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:44.714 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.712+0000 7f4cc23c1740 1 --2- 172.21.15.72:0/316608371 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556f75924ea0 0x556f75933ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:44.714 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.712+0000 7f4cc23c1740 1 -- 172.21.15.72:0/316608371 >> 172.21.15.72:0/316608371 conn(0x556f75920510 msgr2=0x556f75926330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:57:44.714 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.712+0000 7f4c9e000640 0 client.16086 ms_handle_reset on v2:172.21.15.76:6838/327145636 2024-04-26T21:57:44.714 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.712+0000 7f4cc23c1740 1 -- 172.21.15.72:0/316608371 shutdown_connections 2024-04-26T21:57:44.715 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.712+0000 7f4cc23c1740 1 -- 172.21.15.72:0/316608371 wait complete. 2024-04-26T21:57:44.715 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.712+0000 7f4cc23c1740 20 client.16086 trim_cache size 0 max 16384 2024-04-26T21:57:44.715 INFO:teuthology.orchestra.run.smithi072.stdout:[] 2024-04-26T21:57:44.716 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.713+0000 7f4cbf400640 1 -- 172.21.15.72:0/1121575878 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f4ca00a3c40 msgr2=0x7f4ca00a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:44.716 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.713+0000 7f4cbf400640 1 --2- 172.21.15.72:0/1121575878 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f4ca00a3c40 0x7f4ca00a6100 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x556f75926300 tx=0x7f4cac000bc0 comp rx=0 tx=0).stop 2024-04-26T21:57:44.716 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.713+0000 7f4cbf400640 1 -- 172.21.15.72:0/1121575878 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4cb8104b90 msgr2=0x7f4cb819ac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:44.716 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.713+0000 7f4cbf400640 1 --2- 172.21.15.72:0/1121575878 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4cb8104b90 0x7f4cb819ac50 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f4cac031460 tx=0x7f4cac003d50 comp rx=0 tx=0).stop 2024-04-26T21:57:44.716 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.713+0000 7f4cbf400640 1 -- 172.21.15.72:0/1121575878 shutdown_connections 2024-04-26T21:57:44.716 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.713+0000 7f4cbf400640 1 --2- 172.21.15.72:0/1121575878 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4cb8106740 0x7f4cb81a21c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:44.716 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.713+0000 7f4cbf400640 1 --2- 172.21.15.72:0/1121575878 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f4ca00a3c40 0x7f4ca00a6100 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:44.716 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.713+0000 7f4cbf400640 1 --2- 172.21.15.72:0/1121575878 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4cb8105d80 0x7f4cb819b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:44.716 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.713+0000 7f4cbf400640 1 --2- 172.21.15.72:0/1121575878 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4cb8104b90 0x7f4cb819ac50 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:44.716 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.713+0000 7f4cbf400640 1 -- 172.21.15.72:0/1121575878 >> 172.21.15.72:0/1121575878 conn(0x7f4cb81000e0 msgr2=0x7f4cb81090a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:57:44.716 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.713+0000 7f4cbf400640 1 -- 172.21.15.72:0/1121575878 shutdown_connections 2024-04-26T21:57:44.716 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:44.713+0000 7f4cbf400640 1 -- 172.21.15.72:0/1121575878 wait complete. 2024-04-26T21:57:45.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:57:44 smithi072 ceph-mon[28138]: pgmap v518: 97 pgs: 97 active+clean; 209 MiB data, 1004 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 6.0 MiB/s wr, 10 op/s 2024-04-26T21:57:45.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:57:44 smithi076 ceph-mon[31823]: pgmap v518: 97 pgs: 97 active+clean; 209 MiB data, 1004 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 6.0 MiB/s wr, 10 op/s 2024-04-26T21:57:45.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:57:44 smithi164 ceph-mon[31916]: pgmap v518: 97 pgs: 97 active+clean; 209 MiB data, 1004 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 6.0 MiB/s wr, 10 op/s 2024-04-26T21:57:45.735 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-26T21:57:45.735 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:57:45.735 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-26T21:57:45.960 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:45.960+0000 7f3711800640 1 -- 172.21.15.72:0/2576489064 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f370c10a980 msgr2=0x7f370c1112a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:45.961 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:45.960+0000 7f3711800640 1 --2- 172.21.15.72:0/2576489064 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f370c10a980 0x7f370c1112a0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f3704009860 tx=0x7f370402a340 comp rx=0 tx=0).stop 2024-04-26T21:57:45.961 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:45.960+0000 7f3711800640 1 -- 172.21.15.72:0/2576489064 shutdown_connections 2024-04-26T21:57:45.961 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:45.960+0000 7f3711800640 1 --2- 172.21.15.72:0/2576489064 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f370c10a980 0x7f370c1112a0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:45.961 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:45.960+0000 7f3711800640 1 --2- 172.21.15.72:0/2576489064 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f370c109fc0 0x7f370c10a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:45.961 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:45.960+0000 7f3711800640 1 --2- 172.21.15.72:0/2576489064 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f370c108e70 0x7f370c109290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:45.961 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:45.960+0000 7f3711800640 1 -- 172.21.15.72:0/2576489064 >> 172.21.15.72:0/2576489064 conn(0x7f370c077e00 msgr2=0x7f370c07a240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:57:45.961 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:45.960+0000 7f3711800640 1 -- 172.21.15.72:0/2576489064 shutdown_connections 2024-04-26T21:57:45.961 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:45.960+0000 7f3711800640 1 -- 172.21.15.72:0/2576489064 wait complete. 2024-04-26T21:57:45.963 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:45.961+0000 7f3711800640 1 Processor -- start 2024-04-26T21:57:45.963 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:45.961+0000 7f3711800640 1 -- start start 2024-04-26T21:57:45.964 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:45.962+0000 7f3711800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f370c108e70 0x7f370c19ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:45.964 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:45.962+0000 7f3711800640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f370c109fc0 0x7f370c19f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:45.964 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:45.962+0000 7f3711800640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f370c10a980 0x7f370c1a64d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:45.964 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:45.962+0000 7f3711800640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f370c114870 con 0x7f370c10a980 2024-04-26T21:57:45.964 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:45.962+0000 7f3711800640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f370c1146f0 con 0x7f370c109fc0 2024-04-26T21:57:45.964 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:45.962+0000 7f3711800640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f370c1149f0 con 0x7f370c108e70 2024-04-26T21:57:45.964 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:45.962+0000 7f3710e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f370c10a980 0x7f370c1a64d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:45.964 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:45.962+0000 7f3710e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f370c10a980 0x7f370c1a64d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:48242/0 (socket says 172.21.15.72:48242) 2024-04-26T21:57:45.964 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:45.962+0000 7f3710e00640 1 -- 172.21.15.72:0/2150962044 learned_addr learned my addr 172.21.15.72:0/2150962044 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:57:45.964 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:45.962+0000 7f370b400640 1 --2- 172.21.15.72:0/2150962044 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f370c109fc0 0x7f370c19f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:45.964 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:45.962+0000 7f370be00640 1 --2- 172.21.15.72:0/2150962044 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f370c108e70 0x7f370c19ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:45.964 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:45.963+0000 7f3710e00640 1 -- 172.21.15.72:0/2150962044 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f370c108e70 msgr2=0x7f370c19ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:45.964 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:45.963+0000 7f3710e00640 1 --2- 172.21.15.72:0/2150962044 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f370c108e70 0x7f370c19ef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:45.965 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:45.963+0000 7f3710e00640 1 -- 172.21.15.72:0/2150962044 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f370c109fc0 msgr2=0x7f370c19f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:45.965 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:45.963+0000 7f3710e00640 1 --2- 172.21.15.72:0/2150962044 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f370c109fc0 0x7f370c19f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:45.965 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:45.963+0000 7f3710e00640 1 -- 172.21.15.72:0/2150962044 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f370c1a6b40 con 0x7f370c10a980 2024-04-26T21:57:45.965 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:45.963+0000 7f370b400640 1 --2- 172.21.15.72:0/2150962044 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f370c109fc0 0x7f370c19f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:57:45.965 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:45.963+0000 7f3710e00640 1 --2- 172.21.15.72:0/2150962044 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f370c10a980 0x7f370c1a64d0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f3704009830 tx=0x7f370402ad70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:57:45.965 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:45.963+0000 7f3708c00640 1 -- 172.21.15.72:0/2150962044 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3704037040 con 0x7f370c10a980 2024-04-26T21:57:45.965 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:45.963+0000 7f3708c00640 1 -- 172.21.15.72:0/2150962044 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3704041070 con 0x7f370c10a980 2024-04-26T21:57:45.965 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:45.963+0000 7f3711800640 1 -- 172.21.15.72:0/2150962044 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f370c1a6dd0 con 0x7f370c10a980 2024-04-26T21:57:45.965 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:45.963+0000 7f3708c00640 1 -- 172.21.15.72:0/2150962044 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3704005290 con 0x7f370c10a980 2024-04-26T21:57:45.965 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:45.964+0000 7f3711800640 1 -- 172.21.15.72:0/2150962044 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f370c1a7280 con 0x7f370c10a980 2024-04-26T21:57:45.967 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:45.967+0000 7f3708c00640 1 -- 172.21.15.72:0/2150962044 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f3704005430 con 0x7f370c10a980 2024-04-26T21:57:45.967 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:45.967+0000 7f3715c1c740 1 Processor -- start 2024-04-26T21:57:45.968 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:45.967+0000 7f3715c1c740 1 -- start start 2024-04-26T21:57:45.968 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:45.968+0000 7f3715c1c740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56295fe60a70 0x56295fe60e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:45.968 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:45.968+0000 7f3715c1c740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56295fe61460 0x56295fe65f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:45.968 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:45.968+0000 7f3715c1c740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56295fe665d0 0x56295fe68a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:45.968 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:45.968+0000 7f3715c1c740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x56295fdc5980 con 0x56295fe60a70 2024-04-26T21:57:45.968 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:45.968+0000 7f3715c1c740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x56295fd62220 con 0x56295fe61460 2024-04-26T21:57:45.968 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:45.968+0000 7f3715c1c740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x56295fdc1940 con 0x56295fe665d0 2024-04-26T21:57:45.968 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:45.968+0000 7f370b400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56295fe60a70 0x56295fe60e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:45.969 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:45.968+0000 7f370b400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56295fe60a70 0x56295fe60e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:48248/0 (socket says 172.21.15.72:48248) 2024-04-26T21:57:45.969 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:45.968+0000 7f370b400640 1 -- 172.21.15.72:0/2126869111 learned_addr learned my addr 172.21.15.72:0/2126869111 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:57:45.970 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:45.969+0000 7f370be00640 1 --2- 172.21.15.72:0/2126869111 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56295fe61460 0x56295fe65f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:45.970 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:45.969+0000 7f3708c00640 1 --2- 172.21.15.72:0/2150962044 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f36f00a3cc0 0x7f36f00a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:45.970 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:45.969+0000 7f370b400640 1 --2- 172.21.15.72:0/2126869111 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56295fe665d0 0x56295fe68a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:45.970 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:45.969+0000 7f370b400640 1 -- 172.21.15.72:0/2126869111 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56295fe665d0 msgr2=0x56295fe68a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:45.970 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:45.969+0000 7f370b400640 1 --2- 172.21.15.72:0/2126869111 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56295fe665d0 0x56295fe68a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:45.970 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:45.969+0000 7f370b400640 1 -- 172.21.15.72:0/2126869111 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56295fe61460 msgr2=0x56295fe65f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:45.971 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:45.969+0000 7f370b400640 1 --2- 172.21.15.72:0/2126869111 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56295fe61460 0x56295fe65f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:45.971 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:45.969+0000 7f370b400640 1 -- 172.21.15.72:0/2126869111 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56295fb9f7e0 con 0x56295fe60a70 2024-04-26T21:57:45.971 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:45.969+0000 7f3708c00640 1 -- 172.21.15.72:0/2150962044 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7f37040f0540 con 0x7f370c10a980 2024-04-26T21:57:45.971 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:45.969+0000 7f3710e00640 1 --2- 172.21.15.72:0/2150962044 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f36f00a3cc0 0x7f36f00a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:45.971 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:45.969+0000 7f370be00640 1 --2- 172.21.15.72:0/2126869111 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56295fe61460 0x56295fe65f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:57:45.971 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:45.969+0000 7f370b400640 1 --2- 172.21.15.72:0/2126869111 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56295fe60a70 0x56295fe60e90 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f36f8009e50 tx=0x7f36f80096b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4ddd2a62aed92e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:57:45.971 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:45.970+0000 7f3701600640 1 -- 172.21.15.72:0/2126869111 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36f8012070 con 0x56295fe60a70 2024-04-26T21:57:45.971 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:45.970+0000 7f3701600640 1 -- 172.21.15.72:0/2126869111 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f36f80054b0 con 0x56295fe60a70 2024-04-26T21:57:45.971 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:45.970+0000 7f3701600640 1 -- 172.21.15.72:0/2126869111 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36f8004e20 con 0x56295fe60a70 2024-04-26T21:57:45.974 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:45.971+0000 7f3715c1c740 1 -- 172.21.15.72:0/2126869111 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56295fe60a70 msgr2=0x56295fe60e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:45.974 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:45.971+0000 7f3710e00640 1 --2- 172.21.15.72:0/2150962044 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f36f00a3cc0 0x7f36f00a6180 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f37040edf50 tx=0x7f37040058d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:57:45.974 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:45.971+0000 7f3715c1c740 1 --2- 172.21.15.72:0/2126869111 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56295fe60a70 0x56295fe60e90 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f36f8009e50 tx=0x7f36f80096b0 comp rx=0 tx=0).stop 2024-04-26T21:57:45.974 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:45.971+0000 7f3715c1c740 1 -- 172.21.15.72:0/2126869111 shutdown_connections 2024-04-26T21:57:45.974 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:45.971+0000 7f3715c1c740 1 --2- 172.21.15.72:0/2126869111 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56295fe665d0 0x56295fe68a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:45.974 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:45.971+0000 7f3715c1c740 1 --2- 172.21.15.72:0/2126869111 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56295fe61460 0x56295fe65f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:45.974 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:45.971+0000 7f3715c1c740 1 --2- 172.21.15.72:0/2126869111 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56295fe60a70 0x56295fe60e90 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:45.974 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:45.971+0000 7f3715c1c740 1 -- 172.21.15.72:0/2126869111 >> 172.21.15.72:0/2126869111 conn(0x56295fe5c0e0 msgr2=0x56295fe5c4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:57:45.974 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:45.971+0000 7f3715c1c740 1 -- 172.21.15.72:0/2126869111 shutdown_connections 2024-04-26T21:57:45.975 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:45.971+0000 7f3715c1c740 1 -- 172.21.15.72:0/2126869111 wait complete. 2024-04-26T21:57:45.975 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:45.972+0000 7f3715c1c740 1 -- start start 2024-04-26T21:57:45.975 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:45.972+0000 7f3715c1c740 1 Processor -- start 2024-04-26T21:57:45.975 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:45.972+0000 7f3715c1c740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56295fe60a70 0x56295fe6afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:45.975 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:45.973+0000 7f3715c1c740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56295fe61460 0x56295fe6b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:45.975 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:45.973+0000 7f3715c1c740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56295fe665d0 0x56295fe724c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:45.975 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:45.973+0000 7f3715c1c740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x56295fe69580 con 0x56295fe61460 2024-04-26T21:57:45.975 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:45.973+0000 7f3715c1c740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x56295fd4d170 con 0x56295fe60a70 2024-04-26T21:57:45.975 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:45.973+0000 7f3715c1c740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x56295fe69700 con 0x56295fe665d0 2024-04-26T21:57:45.975 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:45.973+0000 7f3715c1c740 10 client.0 _ll_register_callbacks cb 0x56295fdbc770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:57:45.975 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:45.973+0000 7f370be00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56295fe61460 0x56295fe6b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:45.975 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:45.973+0000 7f370be00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56295fe61460 0x56295fe6b4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:48252/0 (socket says 172.21.15.72:48252) 2024-04-26T21:57:45.975 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:45.973+0000 7f370be00640 1 -- 172.21.15.72:0/1276854347 learned_addr learned my addr 172.21.15.72:0/1276854347 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:57:45.975 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:45.973+0000 7f370b400640 1 --2- 172.21.15.72:0/1276854347 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56295fe60a70 0x56295fe6afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:45.976 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:45.973+0000 7f370be00640 1 -- 172.21.15.72:0/1276854347 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56295fe665d0 msgr2=0x56295fe724c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:45.976 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:45.973+0000 7f370be00640 1 --2- 172.21.15.72:0/1276854347 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56295fe665d0 0x56295fe724c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:45.976 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:45.973+0000 7f370be00640 1 -- 172.21.15.72:0/1276854347 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56295fe60a70 msgr2=0x56295fe6afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:45.976 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:45.973+0000 7f370be00640 1 --2- 172.21.15.72:0/1276854347 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56295fe60a70 0x56295fe6afb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:45.976 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:45.973+0000 7f370be00640 1 -- 172.21.15.72:0/1276854347 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56295fbfff70 con 0x56295fe61460 2024-04-26T21:57:45.976 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:45.973+0000 7f370b400640 1 --2- 172.21.15.72:0/1276854347 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56295fe60a70 0x56295fe6afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:57:45.976 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:45.974+0000 7f370be00640 1 --2- 172.21.15.72:0/1276854347 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56295fe61460 0x56295fe6b4f0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f36fc009c10 tx=0x7f36fc004410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccb783cc95812117 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:57:45.976 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:45.974+0000 7f36f6000640 10 client.0 ms_handle_connect on v2:172.21.15.72:3300/0 2024-04-26T21:57:45.976 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:45.974+0000 7f36f6000640 1 -- 172.21.15.72:0/1276854347 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36fc01b670 con 0x56295fe61460 2024-04-26T21:57:45.976 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:45.974+0000 7f36f6000640 1 -- 172.21.15.72:0/1276854347 <== mon.0 v2:172.21.15.72:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7f36fc01b810 con 0x56295fe61460 2024-04-26T21:57:45.976 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:45.974+0000 7f3715c1c740 1 -- 172.21.15.72:0/1276854347 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f36f80054b0 con 0x56295fe61460 2024-04-26T21:57:45.977 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:45.974+0000 7f36f6000640 1 -- 172.21.15.72:0/1276854347 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f36fc020020 con 0x56295fe61460 2024-04-26T21:57:45.977 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:45.974+0000 7f3715c1c740 10 client.16098 fetch_fsmap learned FSMap version 27 2024-04-26T21:57:45.978 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:45.974+0000 7f3715c1c740 1 -- 172.21.15.72:0/1276854347 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56295fa97170 con 0x56295fe61460 2024-04-26T21:57:45.978 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:45.975+0000 7f36f6000640 1 -- 172.21.15.72:0/1276854347 <== mon.0 v2:172.21.15.72:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f36f80054b0 con 0x56295fe61460 2024-04-26T21:57:45.978 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:45.975+0000 7f36f6000640 1 -- 172.21.15.72:0/1276854347 <== mon.0 v2:172.21.15.72:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f36fc012ba0 con 0x56295fe61460 2024-04-26T21:57:45.978 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:45.975+0000 7f3715c1c740 10 client.16098 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T21:57:45.978 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:45.975+0000 7f3715c1c740 10 client.16098 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:57:45.978 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:45.975+0000 7f3715c1c740 20 client.16098 populate_metadata read hostname 'smithi072' 2024-04-26T21:57:45.978 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:45.975+0000 7f3715c1c740 1 --2- 172.21.15.72:0/1276854347 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x56295fe76a30 0x56295fe78eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:45.978 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:45.975+0000 7f3715c1c740 4 client.16098 mds_command: new command op to 34517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:57:45.978 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:45.975+0000 7f3715c1c740 1 -- 172.21.15.72:0/1276854347 --> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56295fdd3590 con 0x56295fe76a30 2024-04-26T21:57:45.978 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:45.976+0000 7f370b400640 1 --2- 172.21.15.72:0/1276854347 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x56295fe76a30 0x56295fe78eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:45.978 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:45.977+0000 7f36f6000640 10 client.16098 ms_handle_connect on v2:172.21.15.76:6838/327145636 2024-04-26T21:57:45.978 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:45.977+0000 7f370b400640 1 --2- 172.21.15.72:0/1276854347 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x56295fe76a30 0x56295fe78eb0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df1a158bd69453ce server_cookie=f06c6107f4feed10 in_seq=0 out_seq=0 2024-04-26T21:57:45.980 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:45.979+0000 7f36f6000640 1 -- 172.21.15.72:0/1276854347 <== mds.0 v2:172.21.15.76:6838/327145636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56295fdd3590 con 0x56295fe76a30 2024-04-26T21:57:45.980 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:45.979+0000 7f36f6000640 10 client.16098 handle_command_reply: tid=0 2024-04-26T21:57:45.980 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:45.979+0000 7f3715c1c740 1 client.16098 shutdown 2024-04-26T21:57:45.980 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:45.979+0000 7f3715c1c740 1 -- 172.21.15.72:0/1276854347 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56295fe61460 msgr2=0x56295fe6b4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:45.980 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:45.979+0000 7f3715c1c740 1 --2- 172.21.15.72:0/1276854347 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56295fe61460 0x56295fe6b4f0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f36fc009c10 tx=0x7f36fc004410 comp rx=0 tx=0).stop 2024-04-26T21:57:45.980 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:45.980+0000 7f3715c1c740 1 -- 172.21.15.72:0/1276854347 shutdown_connections 2024-04-26T21:57:45.980 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:45.980+0000 7f3715c1c740 1 --2- 172.21.15.72:0/1276854347 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x56295fe76a30 0x56295fe78eb0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:45.980 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:45.980+0000 7f3715c1c740 1 --2- 172.21.15.72:0/1276854347 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56295fe665d0 0x56295fe724c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:45.980 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:45.980+0000 7f3715c1c740 1 --2- 172.21.15.72:0/1276854347 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56295fe61460 0x56295fe6b4f0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:45.980 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:45.980+0000 7f36f6000640 0 client.16098 ms_handle_reset on v2:172.21.15.76:6838/327145636 2024-04-26T21:57:45.980 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:45.980+0000 7f3715c1c740 1 --2- 172.21.15.72:0/1276854347 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56295fe60a70 0x56295fe6afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:45.980 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:45.980+0000 7f3715c1c740 1 -- 172.21.15.72:0/1276854347 >> 172.21.15.72:0/1276854347 conn(0x56295fe5c0e0 msgr2=0x56295fe5b2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:57:45.981 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:45.980+0000 7f3715c1c740 1 -- 172.21.15.72:0/1276854347 shutdown_connections 2024-04-26T21:57:45.981 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:45.980+0000 7f3715c1c740 1 -- 172.21.15.72:0/1276854347 wait complete. 2024-04-26T21:57:45.981 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:45.980+0000 7f3715c1c740 20 client.16098 trim_cache size 0 max 16384 2024-04-26T21:57:46.007 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.007+0000 7f3715c1c740 1 Processor -- start 2024-04-26T21:57:46.007 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.007+0000 7f3715c1c740 1 -- start start 2024-04-26T21:57:46.007 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.007+0000 7f3715c1c740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56295fe60a70 0x56295fe5f060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:46.007 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.007+0000 7f3715c1c740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56295fe61460 0x56295fe72480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:46.007 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.007+0000 7f3715c1c740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56295fe665d0 0x56295fe5f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:46.008 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.007+0000 7f3715c1c740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f370c1b0240 con 0x56295fe61460 2024-04-26T21:57:46.008 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.007+0000 7f3715c1c740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f370c1b1d80 con 0x56295fe60a70 2024-04-26T21:57:46.008 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.007+0000 7f3715c1c740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f370c1b0910 con 0x56295fe665d0 2024-04-26T21:57:46.012 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.008+0000 7f370be00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56295fe61460 0x56295fe72480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:46.012 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.008+0000 7f370be00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56295fe61460 0x56295fe72480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:48254/0 (socket says 172.21.15.72:48254) 2024-04-26T21:57:46.012 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.008+0000 7f370be00640 1 -- 172.21.15.72:0/941015686 learned_addr learned my addr 172.21.15.72:0/941015686 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:57:46.012 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.008+0000 7f370b400640 1 --2- 172.21.15.72:0/941015686 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56295fe665d0 0x56295fe5f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:46.012 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.008+0000 7f370be00640 1 -- 172.21.15.72:0/941015686 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56295fe665d0 msgr2=0x56295fe5f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:46.012 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.008+0000 7f370be00640 1 --2- 172.21.15.72:0/941015686 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56295fe665d0 0x56295fe5f5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:46.012 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.008+0000 7f370b400640 1 --2- 172.21.15.72:0/941015686 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56295fe60a70 0x56295fe5f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:46.012 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.008+0000 7f370be00640 1 -- 172.21.15.72:0/941015686 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56295fe60a70 msgr2=0x56295fe5f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:46.012 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.008+0000 7f370be00640 1 --2- 172.21.15.72:0/941015686 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56295fe60a70 0x56295fe5f060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:46.013 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.008+0000 7f370be00640 1 -- 172.21.15.72:0/941015686 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56295fe6a070 con 0x56295fe61460 2024-04-26T21:57:46.013 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.009+0000 7f370be00640 1 --2- 172.21.15.72:0/941015686 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56295fe61460 0x56295fe72480 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f36fc00a010 tx=0x7f36fc009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e35b7dfd89eedf20 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:57:46.013 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.009+0000 7f3700c00640 1 -- 172.21.15.72:0/941015686 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36fc025670 con 0x56295fe61460 2024-04-26T21:57:46.013 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.009+0000 7f3700c00640 1 -- 172.21.15.72:0/941015686 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f36fc016070 con 0x56295fe61460 2024-04-26T21:57:46.013 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.009+0000 7f3700c00640 1 -- 172.21.15.72:0/941015686 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36fc0129a0 con 0x56295fe61460 2024-04-26T21:57:46.013 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.010+0000 7f3715c1c740 1 -- 172.21.15.72:0/941015686 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56295fe61460 msgr2=0x56295fe72480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:46.013 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.010+0000 7f3715c1c740 1 --2- 172.21.15.72:0/941015686 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56295fe61460 0x56295fe72480 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f36fc00a010 tx=0x7f36fc009b80 comp rx=0 tx=0).stop 2024-04-26T21:57:46.013 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.010+0000 7f3715c1c740 1 -- 172.21.15.72:0/941015686 shutdown_connections 2024-04-26T21:57:46.013 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.010+0000 7f3715c1c740 1 --2- 172.21.15.72:0/941015686 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56295fe665d0 0x56295fe5f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:46.013 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.010+0000 7f3715c1c740 1 --2- 172.21.15.72:0/941015686 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56295fe61460 0x56295fe72480 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:46.013 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.010+0000 7f3715c1c740 1 --2- 172.21.15.72:0/941015686 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56295fe60a70 0x56295fe5f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:46.013 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.010+0000 7f3715c1c740 1 -- 172.21.15.72:0/941015686 >> 172.21.15.72:0/941015686 conn(0x56295fe5c0e0 msgr2=0x56295fe6afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:57:46.013 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.010+0000 7f3715c1c740 1 -- 172.21.15.72:0/941015686 shutdown_connections 2024-04-26T21:57:46.013 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.010+0000 7f3715c1c740 1 -- 172.21.15.72:0/941015686 wait complete. 2024-04-26T21:57:46.014 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.010+0000 7f3715c1c740 1 -- start start 2024-04-26T21:57:46.014 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.011+0000 7f3715c1c740 1 Processor -- start 2024-04-26T21:57:46.014 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.011+0000 7f3715c1c740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56295fe60a70 0x56295fe6fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:46.014 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.011+0000 7f3715c1c740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56295fe61460 0x56295fe702f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:46.014 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.011+0000 7f3715c1c740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56295fe665d0 0x56295fe71300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:46.014 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.011+0000 7f3715c1c740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f370c1b16b0 con 0x56295fe60a70 2024-04-26T21:57:46.014 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.011+0000 7f3715c1c740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f370c1b0fe0 con 0x56295fe61460 2024-04-26T21:57:46.014 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.011+0000 7f3715c1c740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f370c1ae220 con 0x56295fe665d0 2024-04-26T21:57:46.014 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.011+0000 7f3715c1c740 10 client.0 _ll_register_callbacks cb 0x7f36fc003970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:57:46.014 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.011+0000 7f370b400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56295fe60a70 0x56295fe6fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:46.014 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.011+0000 7f370b400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56295fe60a70 0x56295fe6fdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:48268/0 (socket says 172.21.15.72:48268) 2024-04-26T21:57:46.014 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.011+0000 7f370b400640 1 -- 172.21.15.72:0/4151077229 learned_addr learned my addr 172.21.15.72:0/4151077229 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:57:46.014 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.012+0000 7f370be00640 1 --2- 172.21.15.72:0/4151077229 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56295fe61460 0x56295fe702f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:46.014 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.012+0000 7f370b400640 1 -- 172.21.15.72:0/4151077229 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56295fe665d0 msgr2=0x56295fe71300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:46.015 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.012+0000 7f370b400640 1 --2- 172.21.15.72:0/4151077229 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56295fe665d0 0x56295fe71300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:46.015 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.012+0000 7f370b400640 1 -- 172.21.15.72:0/4151077229 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56295fe61460 msgr2=0x56295fe702f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:46.015 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.012+0000 7f370b400640 1 --2- 172.21.15.72:0/4151077229 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56295fe61460 0x56295fe702f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:46.015 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.012+0000 7f370b400640 1 -- 172.21.15.72:0/4151077229 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56295fe718f0 con 0x56295fe60a70 2024-04-26T21:57:46.015 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.012+0000 7f370be00640 1 --2- 172.21.15.72:0/4151077229 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56295fe61460 0x56295fe702f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:57:46.015 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.012+0000 7f370b400640 1 --2- 172.21.15.72:0/4151077229 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56295fe60a70 0x56295fe6fdb0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f36f8009d70 tx=0x7f36f8005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10fdfa8d60a1287c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:57:46.015 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.012+0000 7f36f6000640 10 client.0 ms_handle_connect on v2:172.21.15.72:3300/0 2024-04-26T21:57:46.017 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.012+0000 7f36f6000640 1 -- 172.21.15.72:0/4151077229 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36f8029320 con 0x56295fe60a70 2024-04-26T21:57:46.017 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.013+0000 7f36f6000640 1 -- 172.21.15.72:0/4151077229 <== mon.0 v2:172.21.15.72:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7f36f80294c0 con 0x56295fe60a70 2024-04-26T21:57:46.017 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.013+0000 7f3715c1c740 1 -- 172.21.15.72:0/4151077229 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f36fc016070 con 0x56295fe60a70 2024-04-26T21:57:46.017 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.013+0000 7f36f6000640 1 -- 172.21.15.72:0/4151077229 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f36f80296e0 con 0x56295fe60a70 2024-04-26T21:57:46.018 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.013+0000 7f3715c1c740 10 client.16104 fetch_fsmap learned FSMap version 27 2024-04-26T21:57:46.018 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.013+0000 7f3715c1c740 1 -- 172.21.15.72:0/4151077229 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56295fedd290 con 0x56295fe60a70 2024-04-26T21:57:46.018 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.013+0000 7f36f6000640 1 -- 172.21.15.72:0/4151077229 <== mon.0 v2:172.21.15.72:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f36fc016070 con 0x56295fe60a70 2024-04-26T21:57:46.018 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.013+0000 7f36f6000640 1 -- 172.21.15.72:0/4151077229 <== mon.0 v2:172.21.15.72:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f36f800f040 con 0x56295fe60a70 2024-04-26T21:57:46.018 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.013+0000 7f3715c1c740 10 client.16104 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T21:57:46.018 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.013+0000 7f3715c1c740 10 client.16104 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:57:46.018 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.013+0000 7f3715c1c740 20 client.16104 populate_metadata read hostname 'smithi072' 2024-04-26T21:57:46.018 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.013+0000 7f3715c1c740 1 --2- 172.21.15.72:0/4151077229 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x56295fedd9b0 0x56295fedfda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:46.018 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.013+0000 7f3715c1c740 4 client.16104 mds_command: new command op to 34517 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-26T21:57:46.018 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.013+0000 7f3715c1c740 1 -- 172.21.15.72:0/4151077229 --> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x56295fdcd400 con 0x56295fedd9b0 2024-04-26T21:57:46.018 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.014+0000 7f370be00640 1 --2- 172.21.15.72:0/4151077229 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x56295fedd9b0 0x56295fedfda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:46.018 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.015+0000 7f370be00640 1 --2- 172.21.15.72:0/4151077229 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x56295fedd9b0 0x56295fedfda0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d127b2da4602e9b1 server_cookie=ba166511c0343946 in_seq=0 out_seq=0 2024-04-26T21:57:46.018 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.015+0000 7f36f6000640 10 client.16104 ms_handle_connect on v2:172.21.15.76:6838/327145636 2024-04-26T21:57:46.020 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.019+0000 7f36f6000640 1 -- 172.21.15.72:0/4151077229 <== mds.0 v2:172.21.15.76:6838/327145636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56295fdcd400 con 0x56295fedd9b0 2024-04-26T21:57:46.020 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.019+0000 7f36f6000640 10 client.16104 handle_command_reply: tid=0 2024-04-26T21:57:46.020 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.019+0000 7f3715c1c740 1 client.16104 shutdown 2024-04-26T21:57:46.020 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.019+0000 7f3715c1c740 1 -- 172.21.15.72:0/4151077229 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56295fe60a70 msgr2=0x56295fe6fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:46.020 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.019+0000 7f3715c1c740 1 --2- 172.21.15.72:0/4151077229 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56295fe60a70 0x56295fe6fdb0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f36f8009d70 tx=0x7f36f8005850 comp rx=0 tx=0).stop 2024-04-26T21:57:46.020 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.019+0000 7f3715c1c740 1 -- 172.21.15.72:0/4151077229 shutdown_connections 2024-04-26T21:57:46.020 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.019+0000 7f3715c1c740 1 --2- 172.21.15.72:0/4151077229 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x56295fedd9b0 0x56295fedfda0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:46.020 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.019+0000 7f3715c1c740 1 --2- 172.21.15.72:0/4151077229 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56295fe665d0 0x56295fe71300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:46.020 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.019+0000 7f3715c1c740 1 --2- 172.21.15.72:0/4151077229 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56295fe61460 0x56295fe702f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:46.020 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.019+0000 7f3715c1c740 1 --2- 172.21.15.72:0/4151077229 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56295fe60a70 0x56295fe6fdb0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:46.020 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.019+0000 7f36f6000640 0 client.16104 ms_handle_reset on v2:172.21.15.76:6838/327145636 2024-04-26T21:57:46.020 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.019+0000 7f3715c1c740 1 -- 172.21.15.72:0/4151077229 >> 172.21.15.72:0/4151077229 conn(0x56295fe5c0e0 msgr2=0x56295fe5aa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:57:46.020 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.019+0000 7f3715c1c740 1 -- 172.21.15.72:0/4151077229 shutdown_connections 2024-04-26T21:57:46.021 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.020+0000 7f3715c1c740 1 -- 172.21.15.72:0/4151077229 wait complete. 2024-04-26T21:57:46.021 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.020+0000 7f3715c1c740 20 client.16104 trim_cache size 0 max 16384 2024-04-26T21:57:46.021 INFO:teuthology.orchestra.run.smithi072.stdout:{ 2024-04-26T21:57:46.021 INFO:teuthology.orchestra.run.smithi072.stdout: "return_code": 0, 2024-04-26T21:57:46.021 INFO:teuthology.orchestra.run.smithi072.stdout: "scrub_tag": "a27e9422-b783-4d1a-b0ed-348169cd7034", 2024-04-26T21:57:46.021 INFO:teuthology.orchestra.run.smithi072.stdout: "mode": "asynchronous" 2024-04-26T21:57:46.021 INFO:teuthology.orchestra.run.smithi072.stdout:} 2024-04-26T21:57:46.022 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.020+0000 7f3711800640 1 -- 172.21.15.72:0/2150962044 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f36f00a3cc0 msgr2=0x7f36f00a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:46.022 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.020+0000 7f3711800640 1 --2- 172.21.15.72:0/2150962044 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f36f00a3cc0 0x7f36f00a6180 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f37040edf50 tx=0x7f37040058d0 comp rx=0 tx=0).stop 2024-04-26T21:57:46.022 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.020+0000 7f3711800640 1 -- 172.21.15.72:0/2150962044 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f370c10a980 msgr2=0x7f370c1a64d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:46.022 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.020+0000 7f3711800640 1 --2- 172.21.15.72:0/2150962044 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f370c10a980 0x7f370c1a64d0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f3704009830 tx=0x7f370402ad70 comp rx=0 tx=0).stop 2024-04-26T21:57:46.022 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.021+0000 7f3711800640 1 -- 172.21.15.72:0/2150962044 shutdown_connections 2024-04-26T21:57:46.022 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.021+0000 7f3711800640 1 --2- 172.21.15.72:0/2150962044 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f370c10a980 0x7f370c1a64d0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:46.022 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.021+0000 7f3711800640 1 --2- 172.21.15.72:0/2150962044 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f370c109fc0 0x7f370c19f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:46.023 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.021+0000 7f3711800640 1 --2- 172.21.15.72:0/2150962044 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f36f00a3cc0 0x7f36f00a6180 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:46.023 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.021+0000 7f3711800640 1 --2- 172.21.15.72:0/2150962044 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f370c108e70 0x7f370c19ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:46.023 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.021+0000 7f3711800640 1 -- 172.21.15.72:0/2150962044 >> 172.21.15.72:0/2150962044 conn(0x7f370c077e00 msgr2=0x7f370c10d420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:57:46.023 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.021+0000 7f3711800640 1 -- 172.21.15.72:0/2150962044 shutdown_connections 2024-04-26T21:57:46.023 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.021+0000 7f3711800640 1 -- 172.21.15.72:0/2150962044 wait complete. 2024-04-26T21:57:46.039 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-26T21:57:46.039 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-26T21:57:46.249 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.249+0000 7efea6a00640 1 -- 172.21.15.72:0/2066272656 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efea0103b00 msgr2=0x7efea0103f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:46.249 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.249+0000 7efea6a00640 1 --2- 172.21.15.72:0/2066272656 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efea0103b00 0x7efea0103f80 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7efe8c009930 tx=0x7efe8c02a420 comp rx=0 tx=0).stop 2024-04-26T21:57:46.250 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.249+0000 7efea6a00640 1 -- 172.21.15.72:0/2066272656 shutdown_connections 2024-04-26T21:57:46.250 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.249+0000 7efea6a00640 1 --2- 172.21.15.72:0/2066272656 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efea01044c0 0x7efea010ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:46.250 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.249+0000 7efea6a00640 1 --2- 172.21.15.72:0/2066272656 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efea0103b00 0x7efea0103f80 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:46.250 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.249+0000 7efea6a00640 1 --2- 172.21.15.72:0/2066272656 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efea0100b30 0x7efea0100f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:46.250 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.249+0000 7efea6a00640 1 -- 172.21.15.72:0/2066272656 >> 172.21.15.72:0/2066272656 conn(0x7efea00fc460 msgr2=0x7efea00fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:57:46.250 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.249+0000 7efea6a00640 1 -- 172.21.15.72:0/2066272656 shutdown_connections 2024-04-26T21:57:46.250 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.249+0000 7efea6a00640 1 -- 172.21.15.72:0/2066272656 wait complete. 2024-04-26T21:57:46.252 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.250+0000 7efea6a00640 1 Processor -- start 2024-04-26T21:57:46.252 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.250+0000 7efea6a00640 1 -- start start 2024-04-26T21:57:46.252 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.251+0000 7efea6a00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efea0100b30 0x7efea0103470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:46.252 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.251+0000 7efea6a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efea0103b00 0x7efea0101ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:46.252 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.251+0000 7efea6a00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efea01044c0 0x7efea0101ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:46.252 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.251+0000 7efea6a00640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7efea010e2a0 con 0x7efea0100b30 2024-04-26T21:57:46.253 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.251+0000 7efea6a00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7efea010e120 con 0x7efea01044c0 2024-04-26T21:57:46.253 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.251+0000 7efea6a00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7efea010e420 con 0x7efea0103b00 2024-04-26T21:57:46.253 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.251+0000 7efea5600640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efea0100b30 0x7efea0103470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:46.253 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.251+0000 7efea5600640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efea0100b30 0x7efea0103470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:48278/0 (socket says 172.21.15.72:48278) 2024-04-26T21:57:46.253 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.251+0000 7efea5600640 1 -- 172.21.15.72:0/3582021018 learned_addr learned my addr 172.21.15.72:0/3582021018 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:57:46.253 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.251+0000 7efea6000640 1 --2- 172.21.15.72:0/3582021018 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efea01044c0 0x7efea0101ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:46.253 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.251+0000 7efea5600640 1 -- 172.21.15.72:0/3582021018 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efea0103b00 msgr2=0x7efea0101ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:46.253 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.251+0000 7efea4c00640 1 --2- 172.21.15.72:0/3582021018 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efea0103b00 0x7efea0101ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:46.253 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.251+0000 7efea5600640 1 --2- 172.21.15.72:0/3582021018 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efea0103b00 0x7efea0101ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:46.253 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.251+0000 7efea5600640 1 -- 172.21.15.72:0/3582021018 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efea01044c0 msgr2=0x7efea0101ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:46.253 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.251+0000 7efea5600640 1 --2- 172.21.15.72:0/3582021018 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efea01044c0 0x7efea0101ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:46.253 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.251+0000 7efea5600640 1 -- 172.21.15.72:0/3582021018 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efea0102880 con 0x7efea0100b30 2024-04-26T21:57:46.253 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.251+0000 7efea6000640 1 --2- 172.21.15.72:0/3582021018 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efea01044c0 0x7efea0101ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:57:46.253 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.252+0000 7efea5600640 1 --2- 172.21.15.72:0/3582021018 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efea0100b30 0x7efea0103470 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7efe9400d770 tx=0x7efe9400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:57:46.253 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.252+0000 7efe9e000640 1 -- 172.21.15.72:0/3582021018 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe94017070 con 0x7efea0100b30 2024-04-26T21:57:46.254 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.252+0000 7efe9e000640 1 -- 172.21.15.72:0/3582021018 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efe94009c30 con 0x7efea0100b30 2024-04-26T21:57:46.254 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.252+0000 7efea6a00640 1 -- 172.21.15.72:0/3582021018 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efea01a2b60 con 0x7efea0100b30 2024-04-26T21:57:46.254 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.252+0000 7efe9e000640 1 -- 172.21.15.72:0/3582021018 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe94003c50 con 0x7efea0100b30 2024-04-26T21:57:46.254 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.252+0000 7efea6a00640 1 -- 172.21.15.72:0/3582021018 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efea01a3100 con 0x7efea0100b30 2024-04-26T21:57:46.255 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.256+0000 7efeaaf6e740 1 Processor -- start 2024-04-26T21:57:46.256 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.256+0000 7efeaaf6e740 1 -- start start 2024-04-26T21:57:46.257 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.256+0000 7efeaaf6e740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5587468ed960 0x5587468edd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:46.257 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.256+0000 7efeaaf6e740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5587468ee350 0x5587468f2e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:46.258 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.256+0000 7efeaaf6e740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5587468f34c0 0x5587468f5950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:46.258 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.256+0000 7efeaaf6e740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x5587467fe030 con 0x5587468ed960 2024-04-26T21:57:46.258 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.256+0000 7efeaaf6e740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5587468197f0 con 0x5587468f34c0 2024-04-26T21:57:46.258 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.256+0000 7efeaaf6e740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5587467d47e0 con 0x5587468ee350 2024-04-26T21:57:46.258 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.256+0000 7efe9e000640 1 -- 172.21.15.72:0/3582021018 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7efe94003df0 con 0x7efea0100b30 2024-04-26T21:57:46.258 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.256+0000 7efea6000640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5587468ed960 0x5587468edd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:46.258 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.256+0000 7efea6000640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5587468ed960 0x5587468edd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:48282/0 (socket says 172.21.15.72:48282) 2024-04-26T21:57:46.258 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.256+0000 7efea6000640 1 -- 172.21.15.72:0/2855257392 learned_addr learned my addr 172.21.15.72:0/2855257392 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:57:46.258 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.257+0000 7efea4c00640 1 --2- 172.21.15.72:0/2855257392 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5587468f34c0 0x5587468f5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:46.258 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.257+0000 7efea6000640 1 -- 172.21.15.72:0/2855257392 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5587468ee350 msgr2=0x5587468f2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:46.258 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.257+0000 7efea6000640 1 --2- 172.21.15.72:0/2855257392 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5587468ee350 0x5587468f2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:46.258 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.257+0000 7efea6000640 1 -- 172.21.15.72:0/2855257392 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5587468f34c0 msgr2=0x5587468f5950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:46.258 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.257+0000 7efea6000640 1 --2- 172.21.15.72:0/2855257392 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5587468f34c0 0x5587468f5950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:46.259 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.257+0000 7efea6000640 1 -- 172.21.15.72:0/2855257392 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5587467e0050 con 0x5587468ed960 2024-04-26T21:57:46.259 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.257+0000 7efea6000640 1 --2- 172.21.15.72:0/2855257392 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5587468ed960 0x5587468edd80 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7efe98008dd0 tx=0x7efe9800d710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1aa2c1f17c4b0148 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:57:46.259 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.258+0000 7efe92600640 1 -- 172.21.15.72:0/2855257392 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe9800f830 con 0x5587468ed960 2024-04-26T21:57:46.259 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.258+0000 7efe92600640 1 -- 172.21.15.72:0/2855257392 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efe9800de20 con 0x5587468ed960 2024-04-26T21:57:46.259 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.258+0000 7efe92600640 1 -- 172.21.15.72:0/2855257392 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe98018d50 con 0x5587468ed960 2024-04-26T21:57:46.263 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.258+0000 7efe9e000640 1 --2- 172.21.15.72:0/3582021018 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7efe880a3cc0 0x7efe880a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:46.263 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.258+0000 7efeaaf6e740 1 -- 172.21.15.72:0/2855257392 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5587468ed960 msgr2=0x5587468edd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:46.263 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.258+0000 7efeaaf6e740 1 --2- 172.21.15.72:0/2855257392 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5587468ed960 0x5587468edd80 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7efe98008dd0 tx=0x7efe9800d710 comp rx=0 tx=0).stop 2024-04-26T21:57:46.263 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.259+0000 7efe9e000640 1 -- 172.21.15.72:0/3582021018 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7efe940d00e0 con 0x7efea0100b30 2024-04-26T21:57:46.263 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.259+0000 7efeaaf6e740 1 -- 172.21.15.72:0/2855257392 shutdown_connections 2024-04-26T21:57:46.263 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.259+0000 7efeaaf6e740 1 --2- 172.21.15.72:0/2855257392 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5587468f34c0 0x5587468f5950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:46.263 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.259+0000 7efeaaf6e740 1 --2- 172.21.15.72:0/2855257392 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5587468ee350 0x5587468f2e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:46.263 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.259+0000 7efeaaf6e740 1 --2- 172.21.15.72:0/2855257392 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5587468ed960 0x5587468edd80 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:46.263 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.259+0000 7efeaaf6e740 1 -- 172.21.15.72:0/2855257392 >> 172.21.15.72:0/2855257392 conn(0x5587468e8fd0 msgr2=0x5587468e93d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:57:46.263 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.259+0000 7efea5600640 1 --2- 172.21.15.72:0/3582021018 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7efe880a3cc0 0x7efe880a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:46.263 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.259+0000 7efeaaf6e740 1 -- 172.21.15.72:0/2855257392 shutdown_connections 2024-04-26T21:57:46.263 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.259+0000 7efeaaf6e740 1 -- 172.21.15.72:0/2855257392 wait complete. 2024-04-26T21:57:46.263 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.259+0000 7efea5600640 1 --2- 172.21.15.72:0/3582021018 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7efe880a3cc0 0x7efe880a6180 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x5587468ee750 tx=0x7efe9400de80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:57:46.263 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.259+0000 7efeaaf6e740 1 -- start start 2024-04-26T21:57:46.264 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.260+0000 7efeaaf6e740 1 Processor -- start 2024-04-26T21:57:46.264 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.260+0000 7efeaaf6e740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5587468ed960 0x5587468f77a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:46.264 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.260+0000 7efeaaf6e740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5587468ee350 0x5587468f7ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:46.264 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.260+0000 7efeaaf6e740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5587468f34c0 0x5587468fed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:46.264 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.260+0000 7efeaaf6e740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x558746856200 con 0x5587468ee350 2024-04-26T21:57:46.264 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.260+0000 7efeaaf6e740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x558746850ee0 con 0x5587468f34c0 2024-04-26T21:57:46.264 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.260+0000 7efeaaf6e740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55874685de70 con 0x5587468ed960 2024-04-26T21:57:46.264 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.260+0000 7efeaaf6e740 10 client.0 _ll_register_callbacks cb 0x558746847270 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:57:46.264 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.260+0000 7efea4c00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5587468ee350 0x5587468f7ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:46.264 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.261+0000 7efea4c00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5587468ee350 0x5587468f7ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:48294/0 (socket says 172.21.15.72:48294) 2024-04-26T21:57:46.264 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.261+0000 7efea4c00640 1 -- 172.21.15.72:0/2787069685 learned_addr learned my addr 172.21.15.72:0/2787069685 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:57:46.264 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.261+0000 7efea6000640 1 --2- 172.21.15.72:0/2787069685 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5587468ed960 0x5587468f77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:46.264 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.261+0000 7efea4c00640 1 -- 172.21.15.72:0/2787069685 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5587468ed960 msgr2=0x5587468f77a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:46.264 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.261+0000 7efea4c00640 1 --2- 172.21.15.72:0/2787069685 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5587468ed960 0x5587468f77a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:46.265 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.261+0000 7efea6000640 1 --2- 172.21.15.72:0/2787069685 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5587468f34c0 0x5587468fed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:46.265 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.261+0000 7efea4c00640 1 -- 172.21.15.72:0/2787069685 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5587468f34c0 msgr2=0x5587468fed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:46.265 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.261+0000 7efea4c00640 1 --2- 172.21.15.72:0/2787069685 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5587468f34c0 0x5587468fed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:46.265 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.261+0000 7efea4c00640 1 -- 172.21.15.72:0/2787069685 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55874662d7d0 con 0x5587468ee350 2024-04-26T21:57:46.265 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.261+0000 7efea6000640 1 --2- 172.21.15.72:0/2787069685 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5587468f34c0 0x5587468fed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:57:46.265 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.261+0000 7efea4c00640 1 --2- 172.21.15.72:0/2787069685 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5587468ee350 0x5587468f7ce0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7efe8c031460 tx=0x7efe8c0043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=635603a53abc2ad5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:57:46.265 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.261+0000 7efe86a00640 10 client.0 ms_handle_connect on v2:172.21.15.72:3300/0 2024-04-26T21:57:46.265 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.262+0000 7efe86a00640 1 -- 172.21.15.72:0/2787069685 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe8c035030 con 0x5587468ee350 2024-04-26T21:57:46.265 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.262+0000 7efe86a00640 1 -- 172.21.15.72:0/2787069685 <== mon.0 v2:172.21.15.72:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7efe8c003a00 con 0x5587468ee350 2024-04-26T21:57:46.265 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.262+0000 7efeaaf6e740 1 -- 172.21.15.72:0/2787069685 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efe9800de20 con 0x5587468ee350 2024-04-26T21:57:46.265 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.262+0000 7efe86a00640 1 -- 172.21.15.72:0/2787069685 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7efe8c04b050 con 0x5587468ee350 2024-04-26T21:57:46.265 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.262+0000 7efeaaf6e740 10 client.16116 fetch_fsmap learned FSMap version 27 2024-04-26T21:57:46.265 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.262+0000 7efeaaf6e740 1 -- 172.21.15.72:0/2787069685 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5587465633f0 con 0x5587468ee350 2024-04-26T21:57:46.265 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.262+0000 7efe86a00640 1 -- 172.21.15.72:0/2787069685 <== mon.0 v2:172.21.15.72:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efe9800de20 con 0x5587468ee350 2024-04-26T21:57:46.265 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.262+0000 7efe86a00640 1 -- 172.21.15.72:0/2787069685 <== mon.0 v2:172.21.15.72:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7efe8c031ce0 con 0x5587468ee350 2024-04-26T21:57:46.266 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.263+0000 7efeaaf6e740 10 client.16116 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T21:57:46.266 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.263+0000 7efeaaf6e740 10 client.16116 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:57:46.266 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.263+0000 7efeaaf6e740 20 client.16116 populate_metadata read hostname 'smithi072' 2024-04-26T21:57:46.266 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.263+0000 7efeaaf6e740 1 --2- 172.21.15.72:0/2787069685 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x558746903280 0x5587469056c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:46.266 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.263+0000 7efeaaf6e740 4 client.16116 mds_command: new command op to 34517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:57:46.266 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.263+0000 7efeaaf6e740 1 -- 172.21.15.72:0/2787069685 --> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55874685f7e0 con 0x558746903280 2024-04-26T21:57:46.266 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.263+0000 7efea6000640 1 --2- 172.21.15.72:0/2787069685 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x558746903280 0x5587469056c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:46.267 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.264+0000 7efea6000640 1 --2- 172.21.15.72:0/2787069685 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x558746903280 0x5587469056c0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=14cd4a6a106c17a1 server_cookie=95fc3c376dbc33ef in_seq=0 out_seq=0 2024-04-26T21:57:46.267 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.264+0000 7efe86a00640 10 client.16116 ms_handle_connect on v2:172.21.15.76:6838/327145636 2024-04-26T21:57:46.268 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.267+0000 7efe86a00640 1 -- 172.21.15.72:0/2787069685 <== mds.0 v2:172.21.15.76:6838/327145636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55874685f7e0 con 0x558746903280 2024-04-26T21:57:46.268 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.267+0000 7efe86a00640 10 client.16116 handle_command_reply: tid=0 2024-04-26T21:57:46.268 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.267+0000 7efeaaf6e740 1 client.16116 shutdown 2024-04-26T21:57:46.268 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.268+0000 7efeaaf6e740 1 -- 172.21.15.72:0/2787069685 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5587468ee350 msgr2=0x5587468f7ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:46.268 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.268+0000 7efeaaf6e740 1 --2- 172.21.15.72:0/2787069685 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5587468ee350 0x5587468f7ce0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7efe8c031460 tx=0x7efe8c0043b0 comp rx=0 tx=0).stop 2024-04-26T21:57:46.268 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.268+0000 7efeaaf6e740 1 -- 172.21.15.72:0/2787069685 shutdown_connections 2024-04-26T21:57:46.268 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.268+0000 7efeaaf6e740 1 --2- 172.21.15.72:0/2787069685 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x558746903280 0x5587469056c0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:46.268 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.268+0000 7efeaaf6e740 1 --2- 172.21.15.72:0/2787069685 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5587468f34c0 0x5587468fed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:46.268 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.268+0000 7efeaaf6e740 1 --2- 172.21.15.72:0/2787069685 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5587468ee350 0x5587468f7ce0 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:46.268 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.268+0000 7efeaaf6e740 1 --2- 172.21.15.72:0/2787069685 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5587468ed960 0x5587468f77a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:46.269 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.268+0000 7efeaaf6e740 1 -- 172.21.15.72:0/2787069685 >> 172.21.15.72:0/2787069685 conn(0x5587468e8fd0 msgr2=0x5587468e81f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:57:46.269 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.268+0000 7efe86a00640 0 client.16116 ms_handle_reset on v2:172.21.15.76:6838/327145636 2024-04-26T21:57:46.269 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.268+0000 7efeaaf6e740 1 -- 172.21.15.72:0/2787069685 shutdown_connections 2024-04-26T21:57:46.269 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.268+0000 7efeaaf6e740 1 -- 172.21.15.72:0/2787069685 wait complete. 2024-04-26T21:57:46.269 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.268+0000 7efeaaf6e740 20 client.16116 trim_cache size 0 max 16384 2024-04-26T21:57:46.291 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.291+0000 7efeaaf6e740 1 Processor -- start 2024-04-26T21:57:46.291 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.291+0000 7efeaaf6e740 1 -- start start 2024-04-26T21:57:46.291 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.291+0000 7efeaaf6e740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5587468ed960 0x5587468f6dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:46.291 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.291+0000 7efeaaf6e740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5587468ee350 0x5587468fece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:46.291 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.291+0000 7efeaaf6e740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5587468f34c0 0x5587468e8070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:46.291 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.291+0000 7efeaaf6e740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7efea01adb60 con 0x5587468ee350 2024-04-26T21:57:46.291 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.291+0000 7efeaaf6e740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7efea01ac6f0 con 0x5587468ed960 2024-04-26T21:57:46.292 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.291+0000 7efeaaf6e740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7efea01acdc0 con 0x5587468f34c0 2024-04-26T21:57:46.293 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.291+0000 7efea4c00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5587468ee350 0x5587468fece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:46.293 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.292+0000 7efea4c00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5587468ee350 0x5587468fece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:48298/0 (socket says 172.21.15.72:48298) 2024-04-26T21:57:46.293 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.292+0000 7efea4c00640 1 -- 172.21.15.72:0/2201594894 learned_addr learned my addr 172.21.15.72:0/2201594894 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:57:46.293 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.292+0000 7efea6000640 1 --2- 172.21.15.72:0/2201594894 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5587468ed960 0x5587468f6dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:46.293 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.292+0000 7efea6000640 1 --2- 172.21.15.72:0/2201594894 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5587468f34c0 0x5587468e8070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:46.293 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.292+0000 7efea4c00640 1 -- 172.21.15.72:0/2201594894 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5587468f34c0 msgr2=0x5587468e8070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:46.293 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.292+0000 7efea4c00640 1 --2- 172.21.15.72:0/2201594894 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5587468f34c0 0x5587468e8070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:46.293 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.292+0000 7efea4c00640 1 -- 172.21.15.72:0/2201594894 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5587468ed960 msgr2=0x5587468f6dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:46.293 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.292+0000 7efea4c00640 1 --2- 172.21.15.72:0/2201594894 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5587468ed960 0x5587468f6dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:46.293 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.292+0000 7efea4c00640 1 -- 172.21.15.72:0/2201594894 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5587465230f0 con 0x5587468ee350 2024-04-26T21:57:46.293 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.292+0000 7efea4c00640 1 --2- 172.21.15.72:0/2201594894 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5587468ee350 0x5587468fece0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7efe8c009850 tx=0x7efe8c02a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57fca077b5252b8a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:57:46.293 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.293+0000 7efe91c00640 1 -- 172.21.15.72:0/2201594894 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe8c035040 con 0x5587468ee350 2024-04-26T21:57:46.294 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.293+0000 7efe91c00640 1 -- 172.21.15.72:0/2201594894 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efe8c03c8b0 con 0x5587468ee350 2024-04-26T21:57:46.294 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.293+0000 7efe91c00640 1 -- 172.21.15.72:0/2201594894 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe8c031500 con 0x5587468ee350 2024-04-26T21:57:46.295 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.294+0000 7efeaaf6e740 1 -- 172.21.15.72:0/2201594894 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5587468ee350 msgr2=0x5587468fece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:46.295 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.294+0000 7efeaaf6e740 1 --2- 172.21.15.72:0/2201594894 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5587468ee350 0x5587468fece0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7efe8c009850 tx=0x7efe8c02a420 comp rx=0 tx=0).stop 2024-04-26T21:57:46.296 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.294+0000 7efeaaf6e740 1 -- 172.21.15.72:0/2201594894 shutdown_connections 2024-04-26T21:57:46.296 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.294+0000 7efeaaf6e740 1 --2- 172.21.15.72:0/2201594894 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5587468f34c0 0x5587468e8070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:46.296 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.294+0000 7efeaaf6e740 1 --2- 172.21.15.72:0/2201594894 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5587468ee350 0x5587468fece0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:46.296 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.294+0000 7efeaaf6e740 1 --2- 172.21.15.72:0/2201594894 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5587468ed960 0x5587468f6dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:46.296 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.294+0000 7efeaaf6e740 1 -- 172.21.15.72:0/2201594894 >> 172.21.15.72:0/2201594894 conn(0x5587468e8fd0 msgr2=0x5587468e7960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:57:46.296 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.294+0000 7efeaaf6e740 1 -- 172.21.15.72:0/2201594894 shutdown_connections 2024-04-26T21:57:46.296 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.294+0000 7efeaaf6e740 1 -- 172.21.15.72:0/2201594894 wait complete. 2024-04-26T21:57:46.296 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.294+0000 7efeaaf6e740 1 -- start start 2024-04-26T21:57:46.296 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.294+0000 7efeaaf6e740 1 Processor -- start 2024-04-26T21:57:46.296 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.295+0000 7efeaaf6e740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5587468ed960 0x5587468fc1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:46.296 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.295+0000 7efeaaf6e740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5587468ee350 0x5587468fc700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:46.296 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.295+0000 7efeaaf6e740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5587468f34c0 0x5587468fd710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:46.296 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.295+0000 7efeaaf6e740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7efea01aa000 con 0x5587468f34c0 2024-04-26T21:57:46.296 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.295+0000 7efeaaf6e740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7efea01ad490 con 0x5587468ed960 2024-04-26T21:57:46.297 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.295+0000 7efeaaf6e740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7efea01aa670 con 0x5587468ee350 2024-04-26T21:57:46.297 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.295+0000 7efeaaf6e740 10 client.0 _ll_register_callbacks cb 0x7efe8c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:57:46.297 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.295+0000 7efea6000640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5587468f34c0 0x5587468fd710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:46.297 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.295+0000 7efea6000640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5587468f34c0 0x5587468fd710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:48306/0 (socket says 172.21.15.72:48306) 2024-04-26T21:57:46.297 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.295+0000 7efea6000640 1 -- 172.21.15.72:0/3261361836 learned_addr learned my addr 172.21.15.72:0/3261361836 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:57:46.297 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.296+0000 7efea6000640 1 --2- 172.21.15.72:0/3261361836 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5587468ed960 0x5587468fc1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:46.297 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.296+0000 7efea4c00640 1 --2- 172.21.15.72:0/3261361836 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5587468ee350 0x5587468fc700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:46.299 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.296+0000 7efea6000640 1 -- 172.21.15.72:0/3261361836 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5587468ee350 msgr2=0x5587468fc700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:46.299 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.296+0000 7efea6000640 1 --2- 172.21.15.72:0/3261361836 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5587468ee350 0x5587468fc700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:46.299 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.296+0000 7efea6000640 1 -- 172.21.15.72:0/3261361836 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5587468ed960 msgr2=0x5587468fc1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:46.299 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.296+0000 7efea6000640 1 --2- 172.21.15.72:0/3261361836 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5587468ed960 0x5587468fc1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:46.299 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.296+0000 7efea6000640 1 -- 172.21.15.72:0/3261361836 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558746819a30 con 0x5587468f34c0 2024-04-26T21:57:46.299 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.296+0000 7efea6000640 1 --2- 172.21.15.72:0/3261361836 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5587468f34c0 0x5587468fd710 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7efe98006010 tx=0x7efe980086e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4279d1078d2e8d74 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:57:46.299 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.296+0000 7efe86a00640 10 client.0 ms_handle_connect on v2:172.21.15.72:3300/0 2024-04-26T21:57:46.299 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.297+0000 7efe86a00640 1 -- 172.21.15.72:0/3261361836 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe9802a190 con 0x5587468f34c0 2024-04-26T21:57:46.299 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.297+0000 7efe86a00640 1 -- 172.21.15.72:0/3261361836 <== mon.0 v2:172.21.15.72:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7efe98020040 con 0x5587468f34c0 2024-04-26T21:57:46.299 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.297+0000 7efeaaf6e740 1 -- 172.21.15.72:0/3261361836 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efe8c03c8b0 con 0x5587468f34c0 2024-04-26T21:57:46.299 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.297+0000 7efe86a00640 1 -- 172.21.15.72:0/3261361836 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7efe9802a6b0 con 0x5587468f34c0 2024-04-26T21:57:46.299 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.297+0000 7efeaaf6e740 10 client.16122 fetch_fsmap learned FSMap version 27 2024-04-26T21:57:46.300 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.297+0000 7efeaaf6e740 1 -- 172.21.15.72:0/3261361836 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5587468fee40 con 0x5587468f34c0 2024-04-26T21:57:46.300 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.297+0000 7efe86a00640 1 -- 172.21.15.72:0/3261361836 <== mon.0 v2:172.21.15.72:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efe8c03c8b0 con 0x5587468f34c0 2024-04-26T21:57:46.300 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.298+0000 7efe86a00640 1 -- 172.21.15.72:0/3261361836 <== mon.0 v2:172.21.15.72:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7efe98028430 con 0x5587468f34c0 2024-04-26T21:57:46.300 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.298+0000 7efeaaf6e740 10 client.16122 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T21:57:46.300 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.298+0000 7efeaaf6e740 10 client.16122 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:57:46.300 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.298+0000 7efeaaf6e740 20 client.16122 populate_metadata read hostname 'smithi072' 2024-04-26T21:57:46.300 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.298+0000 7efeaaf6e740 1 --2- 172.21.15.72:0/3261361836 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x558746969b40 0x55874696bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:46.300 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.298+0000 7efeaaf6e740 4 client.16122 mds_command: new command op to 34517 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-26T21:57:46.300 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.298+0000 7efeaaf6e740 1 -- 172.21.15.72:0/3261361836 --> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5587468f7590 con 0x558746969b40 2024-04-26T21:57:46.300 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.298+0000 7efea4c00640 1 --2- 172.21.15.72:0/3261361836 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x558746969b40 0x55874696bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:46.301 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.299+0000 7efe86a00640 10 client.16122 ms_handle_connect on v2:172.21.15.76:6838/327145636 2024-04-26T21:57:46.302 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.299+0000 7efea4c00640 1 --2- 172.21.15.72:0/3261361836 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x558746969b40 0x55874696bf30 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8aa7b9bf688b09a0 server_cookie=fc5804847bc4f0c7 in_seq=0 out_seq=0 2024-04-26T21:57:46.302 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.300+0000 7efe86a00640 1 -- 172.21.15.72:0/3261361836 <== mds.0 v2:172.21.15.76:6838/327145636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5587468f7590 con 0x558746969b40 2024-04-26T21:57:46.302 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.300+0000 7efe86a00640 10 client.16122 handle_command_reply: tid=0 2024-04-26T21:57:46.302 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.300+0000 7efeaaf6e740 1 client.16122 shutdown 2024-04-26T21:57:46.302 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.300+0000 7efeaaf6e740 1 -- 172.21.15.72:0/3261361836 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5587468f34c0 msgr2=0x5587468fd710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:46.302 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.300+0000 7efeaaf6e740 1 --2- 172.21.15.72:0/3261361836 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5587468f34c0 0x5587468fd710 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7efe98006010 tx=0x7efe980086e0 comp rx=0 tx=0).stop 2024-04-26T21:57:46.302 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.300+0000 7efeaaf6e740 1 -- 172.21.15.72:0/3261361836 shutdown_connections 2024-04-26T21:57:46.302 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.300+0000 7efeaaf6e740 1 --2- 172.21.15.72:0/3261361836 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x558746969b40 0x55874696bf30 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:46.302 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.300+0000 7efeaaf6e740 1 --2- 172.21.15.72:0/3261361836 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5587468f34c0 0x5587468fd710 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:46.302 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.300+0000 7efeaaf6e740 1 --2- 172.21.15.72:0/3261361836 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5587468ee350 0x5587468fc700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:46.302 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.300+0000 7efe86a00640 0 client.16122 ms_handle_reset on v2:172.21.15.76:6838/327145636 2024-04-26T21:57:46.302 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.300+0000 7efeaaf6e740 1 --2- 172.21.15.72:0/3261361836 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5587468ed960 0x5587468fc1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:46.303 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.300+0000 7efeaaf6e740 1 -- 172.21.15.72:0/3261361836 >> 172.21.15.72:0/3261361836 conn(0x5587468e8fd0 msgr2=0x5587468e81f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:57:46.303 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.301+0000 7efeaaf6e740 1 -- 172.21.15.72:0/3261361836 shutdown_connections 2024-04-26T21:57:46.303 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.301+0000 7efeaaf6e740 1 -- 172.21.15.72:0/3261361836 wait complete. 2024-04-26T21:57:46.303 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.301+0000 7efeaaf6e740 20 client.16122 trim_cache size 0 max 16384 2024-04-26T21:57:46.303 INFO:teuthology.orchestra.run.smithi072.stdout:{ 2024-04-26T21:57:46.303 INFO:teuthology.orchestra.run.smithi072.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-26T21:57:46.303 INFO:teuthology.orchestra.run.smithi072.stdout: "scrubs": { 2024-04-26T21:57:46.303 INFO:teuthology.orchestra.run.smithi072.stdout: "a27e9422-b783-4d1a-b0ed-348169cd7034": { 2024-04-26T21:57:46.303 INFO:teuthology.orchestra.run.smithi072.stdout: "path": "/", 2024-04-26T21:57:46.304 INFO:teuthology.orchestra.run.smithi072.stdout: "tag": "a27e9422-b783-4d1a-b0ed-348169cd7034", 2024-04-26T21:57:46.304 INFO:teuthology.orchestra.run.smithi072.stdout: "options": "recursive,force" 2024-04-26T21:57:46.304 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:57:46.304 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:57:46.304 INFO:teuthology.orchestra.run.smithi072.stdout:} 2024-04-26T21:57:46.304 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.301+0000 7efea6a00640 1 -- 172.21.15.72:0/3582021018 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7efe880a3cc0 msgr2=0x7efe880a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:46.304 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.301+0000 7efea6a00640 1 --2- 172.21.15.72:0/3582021018 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7efe880a3cc0 0x7efe880a6180 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x5587468ee750 tx=0x7efe9400de80 comp rx=0 tx=0).stop 2024-04-26T21:57:46.305 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.301+0000 7efea6a00640 1 -- 172.21.15.72:0/3582021018 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efea0100b30 msgr2=0x7efea0103470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:46.305 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.301+0000 7efea6a00640 1 --2- 172.21.15.72:0/3582021018 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efea0100b30 0x7efea0103470 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7efe9400d770 tx=0x7efe9400dc30 comp rx=0 tx=0).stop 2024-04-26T21:57:46.305 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.302+0000 7efea6a00640 1 -- 172.21.15.72:0/3582021018 shutdown_connections 2024-04-26T21:57:46.305 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.302+0000 7efea6a00640 1 --2- 172.21.15.72:0/3582021018 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efea01044c0 0x7efea0101ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:46.305 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.302+0000 7efea6a00640 1 --2- 172.21.15.72:0/3582021018 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7efe880a3cc0 0x7efe880a6180 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:46.305 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.302+0000 7efea6a00640 1 --2- 172.21.15.72:0/3582021018 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efea0103b00 0x7efea0101ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:46.305 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.302+0000 7efea6a00640 1 --2- 172.21.15.72:0/3582021018 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efea0100b30 0x7efea0103470 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:46.305 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.302+0000 7efea6a00640 1 -- 172.21.15.72:0/3582021018 >> 172.21.15.72:0/3582021018 conn(0x7efea00fc460 msgr2=0x7efea0106f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:57:46.305 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.302+0000 7efea6a00640 1 -- 172.21.15.72:0/3582021018 shutdown_connections 2024-04-26T21:57:46.305 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:57:46.302+0000 7efea6a00640 1 -- 172.21.15.72:0/3582021018 wait complete. 2024-04-26T21:57:46.319 INFO:tasks.cephfs.filesystem:scrub status for tag:a27e9422-b783-4d1a-b0ed-348169cd7034 - {'path': '/', 'tag': 'a27e9422-b783-4d1a-b0ed-348169cd7034', 'options': 'recursive,force'} 2024-04-26T21:57:47.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:57:46 smithi076 ceph-mon[31823]: pgmap v519: 97 pgs: 97 active+clean; 221 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 5.3 MiB/s wr, 13 op/s 2024-04-26T21:57:47.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:57:46 smithi076 ceph-mon[31823]: scrub queued for path: / 2024-04-26T21:57:47.316 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:57:46 smithi076 ceph-mon[31823]: scrub summary: idle+waiting paths [/] 2024-04-26T21:57:47.316 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:57:46 smithi076 ceph-mon[31823]: scrub summary: active paths [/] 2024-04-26T21:57:47.316 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:57:46 smithi076 ceph-mon[31823]: scrub summary: idle+waiting paths [/] 2024-04-26T21:57:47.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:57:46 smithi072 ceph-mon[28138]: pgmap v519: 97 pgs: 97 active+clean; 221 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 5.3 MiB/s wr, 13 op/s 2024-04-26T21:57:47.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:57:46 smithi072 ceph-mon[28138]: scrub queued for path: / 2024-04-26T21:57:47.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:57:46 smithi072 ceph-mon[28138]: scrub summary: idle+waiting paths [/] 2024-04-26T21:57:47.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:57:46 smithi072 ceph-mon[28138]: scrub summary: active paths [/] 2024-04-26T21:57:47.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:57:46 smithi072 ceph-mon[28138]: scrub summary: idle+waiting paths [/] 2024-04-26T21:57:47.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:57:46 smithi164 ceph-mon[31916]: pgmap v519: 97 pgs: 97 active+clean; 221 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 5.3 MiB/s wr, 13 op/s 2024-04-26T21:57:47.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:57:46 smithi164 ceph-mon[31916]: scrub queued for path: / 2024-04-26T21:57:47.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:57:46 smithi164 ceph-mon[31916]: scrub summary: idle+waiting paths [/] 2024-04-26T21:57:47.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:57:46 smithi164 ceph-mon[31916]: scrub summary: active paths [/] 2024-04-26T21:57:47.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:57:46 smithi164 ceph-mon[31916]: scrub summary: idle+waiting paths [/] 2024-04-26T21:57:49.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:57:48 smithi076 ceph-mon[31823]: pgmap v520: 97 pgs: 97 active+clean; 237 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 5.0 MiB/s wr, 11 op/s 2024-04-26T21:57:49.321 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:57:48 smithi072 ceph-mon[28138]: pgmap v520: 97 pgs: 97 active+clean; 237 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 5.0 MiB/s wr, 11 op/s 2024-04-26T21:57:49.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:57:48 smithi164 ceph-mon[31916]: pgmap v520: 97 pgs: 97 active+clean; 237 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 5.0 MiB/s wr, 11 op/s 2024-04-26T21:57:50.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:57:50 smithi076 ceph-mon[31823]: scrub summary: idle 2024-04-26T21:57:50.321 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:57:50 smithi072 ceph-mon[28138]: scrub summary: idle 2024-04-26T21:57:50.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:57:50 smithi164 ceph-mon[31916]: scrub summary: idle 2024-04-26T21:57:51.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:57:51 smithi076 ceph-mon[31823]: pgmap v521: 97 pgs: 97 active+clean; 250 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 6.1 MiB/s wr, 60 op/s 2024-04-26T21:57:51.321 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:57:51 smithi072 ceph-mon[28138]: pgmap v521: 97 pgs: 97 active+clean; 250 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 6.1 MiB/s wr, 60 op/s 2024-04-26T21:57:51.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:57:51 smithi164 ceph-mon[31916]: pgmap v521: 97 pgs: 97 active+clean; 250 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 6.1 MiB/s wr, 60 op/s 2024-04-26T21:57:53.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:57:53 smithi076 ceph-mon[31823]: pgmap v522: 97 pgs: 97 active+clean; 258 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 6.5 MiB/s wr, 159 op/s 2024-04-26T21:57:53.321 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:57:53 smithi072 ceph-mon[28138]: pgmap v522: 97 pgs: 97 active+clean; 258 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 6.5 MiB/s wr, 159 op/s 2024-04-26T21:57:53.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:57:53 smithi164 ceph-mon[31916]: pgmap v522: 97 pgs: 97 active+clean; 258 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 6.5 MiB/s wr, 159 op/s 2024-04-26T21:57:55.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:57:55 smithi076 ceph-mon[31823]: pgmap v523: 97 pgs: 97 active+clean; 258 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 7.0 MiB/s wr, 519 op/s 2024-04-26T21:57:55.321 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:57:55 smithi072 ceph-mon[28138]: pgmap v523: 97 pgs: 97 active+clean; 258 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 7.0 MiB/s wr, 519 op/s 2024-04-26T21:57:55.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:57:55 smithi164 ceph-mon[31916]: pgmap v523: 97 pgs: 97 active+clean; 258 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 7.0 MiB/s wr, 519 op/s 2024-04-26T21:57:57.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:57:57 smithi076 ceph-mon[31823]: pgmap v524: 97 pgs: 97 active+clean; 258 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.4 MiB/s wr, 914 op/s 2024-04-26T21:57:57.321 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:57:57 smithi072 ceph-mon[28138]: pgmap v524: 97 pgs: 97 active+clean; 258 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.4 MiB/s wr, 914 op/s 2024-04-26T21:57:57.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:57:57 smithi164 ceph-mon[31916]: pgmap v524: 97 pgs: 97 active+clean; 258 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.4 MiB/s wr, 914 op/s 2024-04-26T21:57:59.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:57:59 smithi076 ceph-mon[31823]: pgmap v525: 97 pgs: 97 active+clean; 258 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 8.7 MiB/s wr, 1.24k op/s 2024-04-26T21:57:59.321 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:57:59 smithi072 ceph-mon[28138]: pgmap v525: 97 pgs: 97 active+clean; 258 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 8.7 MiB/s wr, 1.24k op/s 2024-04-26T21:57:59.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:57:59 smithi164 ceph-mon[31916]: pgmap v525: 97 pgs: 97 active+clean; 258 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 8.7 MiB/s wr, 1.24k op/s 2024-04-26T21:58:01.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:58:01 smithi076 ceph-mon[31823]: pgmap v526: 97 pgs: 97 active+clean; 258 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.4 MiB/s wr, 1.62k op/s 2024-04-26T21:58:01.321 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:58:01 smithi072 ceph-mon[28138]: pgmap v526: 97 pgs: 97 active+clean; 258 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.4 MiB/s wr, 1.62k op/s 2024-04-26T21:58:01.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:58:01 smithi164 ceph-mon[31916]: pgmap v526: 97 pgs: 97 active+clean; 258 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.4 MiB/s wr, 1.62k op/s 2024-04-26T21:58:03.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:58:03 smithi076 ceph-mon[31823]: pgmap v527: 97 pgs: 97 active+clean; 258 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 9.3 MiB/s wr, 1.88k op/s 2024-04-26T21:58:03.321 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:58:03 smithi072 ceph-mon[28138]: pgmap v527: 97 pgs: 97 active+clean; 258 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 9.3 MiB/s wr, 1.88k op/s 2024-04-26T21:58:03.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:58:03 smithi164 ceph-mon[31916]: pgmap v527: 97 pgs: 97 active+clean; 258 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 9.3 MiB/s wr, 1.88k op/s 2024-04-26T21:58:04.940 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:58:04 smithi164 ceph-mon[31916]: pgmap v528: 97 pgs: 97 active+clean; 258 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 10 MiB/s wr, 2.25k op/s 2024-04-26T21:58:05.064 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:58:04 smithi076 ceph-mon[31823]: pgmap v528: 97 pgs: 97 active+clean; 258 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 10 MiB/s wr, 2.25k op/s 2024-04-26T21:58:05.071 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:58:04 smithi072 ceph-mon[28138]: pgmap v528: 97 pgs: 97 active+clean; 258 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 10 MiB/s wr, 2.25k op/s 2024-04-26T21:58:07.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:58:06 smithi076 ceph-mon[31823]: pgmap v529: 97 pgs: 97 active+clean; 258 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 MiB/s wr, 2.40k op/s 2024-04-26T21:58:07.321 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:58:06 smithi072 ceph-mon[28138]: pgmap v529: 97 pgs: 97 active+clean; 258 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 MiB/s wr, 2.40k op/s 2024-04-26T21:58:07.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:58:06 smithi164 ceph-mon[31916]: pgmap v529: 97 pgs: 97 active+clean; 258 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 MiB/s wr, 2.40k op/s 2024-04-26T21:58:09.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:58:08 smithi076 ceph-mon[31823]: pgmap v530: 97 pgs: 97 active+clean; 258 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 11 MiB/s wr, 2.42k op/s 2024-04-26T21:58:09.321 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:58:08 smithi072 ceph-mon[28138]: pgmap v530: 97 pgs: 97 active+clean; 258 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 11 MiB/s wr, 2.42k op/s 2024-04-26T21:58:09.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:58:08 smithi164 ceph-mon[31916]: pgmap v530: 97 pgs: 97 active+clean; 258 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 11 MiB/s wr, 2.42k op/s 2024-04-26T21:58:11.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:58:10 smithi076 ceph-mon[31823]: pgmap v531: 97 pgs: 97 active+clean; 258 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 MiB/s wr, 2.55k op/s 2024-04-26T21:58:11.321 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:58:10 smithi072 ceph-mon[28138]: pgmap v531: 97 pgs: 97 active+clean; 258 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 MiB/s wr, 2.55k op/s 2024-04-26T21:58:11.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:58:10 smithi164 ceph-mon[31916]: pgmap v531: 97 pgs: 97 active+clean; 258 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 MiB/s wr, 2.55k op/s 2024-04-26T21:58:13.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:58:12 smithi076 ceph-mon[31823]: pgmap v532: 97 pgs: 97 active+clean; 258 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 11 MiB/s wr, 2.49k op/s 2024-04-26T21:58:13.321 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:58:12 smithi072 ceph-mon[28138]: pgmap v532: 97 pgs: 97 active+clean; 258 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 11 MiB/s wr, 2.49k op/s 2024-04-26T21:58:13.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:58:12 smithi164 ceph-mon[31916]: pgmap v532: 97 pgs: 97 active+clean; 258 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 11 MiB/s wr, 2.49k op/s 2024-04-26T21:58:15.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:58:14 smithi076 ceph-mon[31823]: pgmap v533: 97 pgs: 97 active+clean; 258 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 MiB/s wr, 2.65k op/s 2024-04-26T21:58:15.321 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:58:14 smithi072 ceph-mon[28138]: pgmap v533: 97 pgs: 97 active+clean; 258 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 MiB/s wr, 2.65k op/s 2024-04-26T21:58:15.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:58:14 smithi164 ceph-mon[31916]: pgmap v533: 97 pgs: 97 active+clean; 258 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 MiB/s wr, 2.65k op/s 2024-04-26T21:58:16.321 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-26T21:58:16.321 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-26T21:58:16.568 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.567+0000 7f85e7600640 1 -- 172.21.15.72:0/506855635 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f85e010a810 msgr2=0x7f85e010ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:16.568 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.567+0000 7f85e7600640 1 --2- 172.21.15.72:0/506855635 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f85e010a810 0x7f85e010ccb0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f85d8009ca0 tx=0x7f85d802a950 comp rx=0 tx=0).stop 2024-04-26T21:58:16.568 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.568+0000 7f85e7600640 1 -- 172.21.15.72:0/506855635 shutdown_connections 2024-04-26T21:58:16.568 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.568+0000 7f85e7600640 1 --2- 172.21.15.72:0/506855635 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f85e010a810 0x7f85e010ccb0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:16.568 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.568+0000 7f85e7600640 1 --2- 172.21.15.72:0/506855635 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f85e0075330 0x7f85e0075790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:16.568 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.568+0000 7f85e7600640 1 --2- 172.21.15.72:0/506855635 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f85e0076910 0x7f85e0074d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:16.568 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.568+0000 7f85e7600640 1 -- 172.21.15.72:0/506855635 >> 172.21.15.72:0/506855635 conn(0x7f85e00fdf10 msgr2=0x7f85e0100330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:58:16.568 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.568+0000 7f85e7600640 1 -- 172.21.15.72:0/506855635 shutdown_connections 2024-04-26T21:58:16.568 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.568+0000 7f85e7600640 1 -- 172.21.15.72:0/506855635 wait complete. 2024-04-26T21:58:16.569 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.569+0000 7f85e7600640 1 Processor -- start 2024-04-26T21:58:16.570 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.569+0000 7f85e7600640 1 -- start start 2024-04-26T21:58:16.570 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.569+0000 7f85e7600640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f85e0075330 0x7f85e019ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:16.570 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.569+0000 7f85e7600640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f85e0076910 0x7f85e019b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:16.570 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.570+0000 7f85e7600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f85e010a810 0x7f85e01a2210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:16.570 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.570+0000 7f85e7600640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f85e0110540 con 0x7f85e0076910 2024-04-26T21:58:16.570 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.570+0000 7f85e7600640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f85e01103c0 con 0x7f85e0075330 2024-04-26T21:58:16.570 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.570+0000 7f85e7600640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f85e01106c0 con 0x7f85e010a810 2024-04-26T21:58:16.570 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.570+0000 7f85e4c00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f85e0075330 0x7f85e019ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:16.572 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.570+0000 7f85e4c00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f85e0075330 0x7f85e019ac70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.72:48564/0 (socket says 172.21.15.72:48564) 2024-04-26T21:58:16.572 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.570+0000 7f85e4c00640 1 -- 172.21.15.72:0/3462852931 learned_addr learned my addr 172.21.15.72:0/3462852931 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:58:16.572 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.570+0000 7f85dfe00640 1 --2- 172.21.15.72:0/3462852931 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f85e0076910 0x7f85e019b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:16.572 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.571+0000 7f85e5600640 1 --2- 172.21.15.72:0/3462852931 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f85e010a810 0x7f85e01a2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:16.572 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.571+0000 7f85e4c00640 1 -- 172.21.15.72:0/3462852931 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f85e010a810 msgr2=0x7f85e01a2210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:16.572 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.571+0000 7f85e4c00640 1 --2- 172.21.15.72:0/3462852931 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f85e010a810 0x7f85e01a2210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:16.572 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.571+0000 7f85e4c00640 1 -- 172.21.15.72:0/3462852931 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f85e0076910 msgr2=0x7f85e019b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:16.572 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.571+0000 7f85e4c00640 1 --2- 172.21.15.72:0/3462852931 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f85e0076910 0x7f85e019b1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:16.572 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.571+0000 7f85e4c00640 1 -- 172.21.15.72:0/3462852931 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85e006c480 con 0x7f85e0075330 2024-04-26T21:58:16.572 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.571+0000 7f85e5600640 1 --2- 172.21.15.72:0/3462852931 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f85e010a810 0x7f85e01a2210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:58:16.572 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.571+0000 7f85dfe00640 1 --2- 172.21.15.72:0/3462852931 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f85e0076910 0x7f85e019b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:58:16.572 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.572+0000 7f85e4c00640 1 --2- 172.21.15.72:0/3462852931 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f85e0075330 0x7f85e019ac70 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f85d400e9f0 tx=0x7f85d400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:58:16.573 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.572+0000 7f85dd600640 1 -- 172.21.15.72:0/3462852931 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85d400cd80 con 0x7f85e0075330 2024-04-26T21:58:16.573 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.572+0000 7f85dd600640 1 -- 172.21.15.72:0/3462852931 <== mon.2 v2:172.21.15.76:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f85d40108c0 con 0x7f85e0075330 2024-04-26T21:58:16.573 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.572+0000 7f85e7600640 1 -- 172.21.15.72:0/3462852931 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85e006c770 con 0x7f85e0075330 2024-04-26T21:58:16.573 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.572+0000 7f85e7600640 1 -- 172.21.15.72:0/3462852931 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85e006cd10 con 0x7f85e0075330 2024-04-26T21:58:16.573 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.572+0000 7f85dd600640 1 -- 172.21.15.72:0/3462852931 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85d40187c0 con 0x7f85e0075330 2024-04-26T21:58:16.576 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.576+0000 7f85ea5e2740 1 Processor -- start 2024-04-26T21:58:16.577 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.576+0000 7f85ea5e2740 1 -- start start 2024-04-26T21:58:16.577 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.577+0000 7f85ea5e2740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5629b7697fa0 0x5629b76983c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:16.577 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.577+0000 7f85ea5e2740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5629b7698990 0x5629b769d490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:16.577 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.577+0000 7f85ea5e2740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5629b769db00 0x5629b769ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:16.577 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.577+0000 7f85ea5e2740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x5629b75a0030 con 0x5629b769db00 2024-04-26T21:58:16.577 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.577+0000 7f85ea5e2740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5629b75bb7f0 con 0x5629b7697fa0 2024-04-26T21:58:16.577 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.577+0000 7f85ea5e2740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5629b75767e0 con 0x5629b7698990 2024-04-26T21:58:16.579 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.578+0000 7f85e5600640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5629b7697fa0 0x5629b76983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:16.579 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.578+0000 7f85e5600640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5629b7697fa0 0x5629b76983c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.72:48578/0 (socket says 172.21.15.72:48578) 2024-04-26T21:58:16.579 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.578+0000 7f85e5600640 1 -- 172.21.15.72:0/2175780514 learned_addr learned my addr 172.21.15.72:0/2175780514 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:58:16.579 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.578+0000 7f85dd600640 1 -- 172.21.15.72:0/3462852931 <== mon.2 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f85d4018960 con 0x7f85e0075330 2024-04-26T21:58:16.580 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.578+0000 7f85dfe00640 1 --2- 172.21.15.72:0/2175780514 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5629b769db00 0x5629b769ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:16.580 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.579+0000 7f85e5600640 1 -- 172.21.15.72:0/2175780514 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5629b7698990 msgr2=0x5629b769d490 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T21:58:16.580 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.579+0000 7f85e5600640 1 --2- 172.21.15.72:0/2175780514 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5629b7698990 0x5629b769d490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:16.580 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.579+0000 7f85e5600640 1 -- 172.21.15.72:0/2175780514 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5629b769db00 msgr2=0x5629b769ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:58:16.580 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.580+0000 7f85e5600640 1 --2- 172.21.15.72:0/2175780514 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5629b769db00 0x5629b769ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:16.580 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.580+0000 7f85e5600640 1 -- 172.21.15.72:0/2175780514 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5629b7582050 con 0x5629b7697fa0 2024-04-26T21:58:16.580 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.580+0000 7f85dfe00640 1 --2- 172.21.15.72:0/2175780514 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5629b769db00 0x5629b769ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:58:16.581 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.581+0000 7f85e5600640 1 --2- 172.21.15.72:0/2175780514 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5629b7697fa0 0x5629b76983c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f85d8009990 tx=0x7f85d80099c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ebc1f16864a542 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:58:16.582 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.582+0000 7f85d1c00640 1 -- 172.21.15.72:0/2175780514 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85d8034050 con 0x5629b7697fa0 2024-04-26T21:58:16.582 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.582+0000 7f85d1c00640 1 -- 172.21.15.72:0/2175780514 <== mon.2 v2:172.21.15.76:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f85d8003e00 con 0x5629b7697fa0 2024-04-26T21:58:16.582 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.582+0000 7f85d1c00640 1 -- 172.21.15.72:0/2175780514 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85d80042a0 con 0x5629b7697fa0 2024-04-26T21:58:16.583 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.583+0000 7f85ea5e2740 1 -- 172.21.15.72:0/2175780514 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5629b7697fa0 msgr2=0x5629b76983c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:16.583 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.583+0000 7f85ea5e2740 1 --2- 172.21.15.72:0/2175780514 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5629b7697fa0 0x5629b76983c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f85d8009990 tx=0x7f85d80099c0 comp rx=0 tx=0).stop 2024-04-26T21:58:16.583 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.583+0000 7f85ea5e2740 1 -- 172.21.15.72:0/2175780514 shutdown_connections 2024-04-26T21:58:16.584 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.583+0000 7f85ea5e2740 1 --2- 172.21.15.72:0/2175780514 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5629b769db00 0x5629b769ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:16.584 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.583+0000 7f85ea5e2740 1 --2- 172.21.15.72:0/2175780514 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5629b7698990 0x5629b769d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:16.584 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.583+0000 7f85ea5e2740 1 --2- 172.21.15.72:0/2175780514 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5629b7697fa0 0x5629b76983c0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:16.584 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.583+0000 7f85ea5e2740 1 -- 172.21.15.72:0/2175780514 >> 172.21.15.72:0/2175780514 conn(0x5629b768adb0 msgr2=0x5629b768b1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:58:16.584 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.583+0000 7f85dd600640 1 --2- 172.21.15.72:0/3462852931 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f85c80a3d80 0x7f85c80a6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:16.584 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.584+0000 7f85ea5e2740 1 -- 172.21.15.72:0/2175780514 shutdown_connections 2024-04-26T21:58:16.584 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.584+0000 7f85ea5e2740 1 -- 172.21.15.72:0/2175780514 wait complete. 2024-04-26T21:58:16.585 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.584+0000 7f85dd600640 1 -- 172.21.15.72:0/3462852931 <== mon.2 v2:172.21.15.76:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7f85d40d0a10 con 0x7f85e0075330 2024-04-26T21:58:16.585 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.584+0000 7f85e5600640 1 --2- 172.21.15.72:0/3462852931 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f85c80a3d80 0x7f85c80a6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:16.585 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.584+0000 7f85ea5e2740 1 -- start start 2024-04-26T21:58:16.585 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.585+0000 7f85ea5e2740 1 Processor -- start 2024-04-26T21:58:16.586 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.585+0000 7f85ea5e2740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5629b7697fa0 0x5629b7690bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:16.586 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.585+0000 7f85ea5e2740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5629b7698990 0x5629b7691130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:16.586 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.585+0000 7f85ea5e2740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5629b769db00 0x5629b7696180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:16.586 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.585+0000 7f85ea5e2740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x5629b75f8200 con 0x5629b7698990 2024-04-26T21:58:16.586 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.585+0000 7f85ea5e2740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5629b75f2ee0 con 0x5629b7697fa0 2024-04-26T21:58:16.586 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.585+0000 7f85ea5e2740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5629b75ffe70 con 0x5629b769db00 2024-04-26T21:58:16.586 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.585+0000 7f85ea5e2740 10 client.0 _ll_register_callbacks cb 0x5629b75e9270 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:58:16.586 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.585+0000 7f85e5600640 1 --2- 172.21.15.72:0/3462852931 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f85c80a3d80 0x7f85c80a6240 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f85d8009ca0 tx=0x7f85d802ae50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:58:16.586 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.586+0000 7f85e5600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5629b769db00 0x5629b7696180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:16.586 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.586+0000 7f85e5600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5629b769db00 0x5629b7696180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.72:55722/0 (socket says 172.21.15.72:55722) 2024-04-26T21:58:16.586 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.586+0000 7f85e5600640 1 -- 172.21.15.72:0/2233015863 learned_addr learned my addr 172.21.15.72:0/2233015863 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:58:16.586 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.586+0000 7f85e4c00640 1 --2- 172.21.15.72:0/2233015863 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5629b7697fa0 0x5629b7690bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:16.587 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.586+0000 7f85dfe00640 1 --2- 172.21.15.72:0/2233015863 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5629b7698990 0x5629b7691130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:16.588 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.586+0000 7f85e5600640 1 -- 172.21.15.72:0/2233015863 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5629b7697fa0 msgr2=0x5629b7690bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:58:16.588 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.586+0000 7f85e5600640 1 --2- 172.21.15.72:0/2233015863 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5629b7697fa0 0x5629b7690bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:16.588 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.586+0000 7f85e5600640 1 -- 172.21.15.72:0/2233015863 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5629b7698990 msgr2=0x5629b7691130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:58:16.588 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.586+0000 7f85e5600640 1 --2- 172.21.15.72:0/2233015863 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5629b7698990 0x5629b7691130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:16.588 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.586+0000 7f85e5600640 1 -- 172.21.15.72:0/2233015863 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5629b73cf7d0 con 0x5629b769db00 2024-04-26T21:58:16.588 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.587+0000 7f85dfe00640 1 --2- 172.21.15.72:0/2233015863 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5629b7698990 0x5629b7691130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:58:16.588 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.587+0000 7f85c6000640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-04-26T21:58:16.588 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.587+0000 7f85e5600640 1 --2- 172.21.15.72:0/2233015863 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5629b769db00 0x5629b7696180 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f85d8043000 tx=0x7f85d80099c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=38037808f76e0a5b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:58:16.588 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.588+0000 7f85c6000640 1 -- 172.21.15.72:0/2233015863 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85d805d060 con 0x5629b769db00 2024-04-26T21:58:16.588 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.588+0000 7f85ea5e2740 1 -- 172.21.15.72:0/2233015863 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f85d8003e00 con 0x5629b769db00 2024-04-26T21:58:16.589 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.588+0000 7f85c6000640 1 -- 172.21.15.72:0/2233015863 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7f85d8054030 con 0x5629b769db00 2024-04-26T21:58:16.589 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.588+0000 7f85c6000640 1 -- 172.21.15.72:0/2233015863 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f85d805d060 con 0x5629b769db00 2024-04-26T21:58:16.589 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.588+0000 7f85ea5e2740 10 client.25282 fetch_fsmap learned FSMap version 27 2024-04-26T21:58:16.589 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.588+0000 7f85ea5e2740 1 -- 172.21.15.72:0/2233015863 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5629b73053f0 con 0x5629b769db00 2024-04-26T21:58:16.589 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.589+0000 7f85c6000640 1 -- 172.21.15.72:0/2233015863 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f85d8003e00 con 0x5629b769db00 2024-04-26T21:58:16.590 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.589+0000 7f85c6000640 1 -- 172.21.15.72:0/2233015863 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f85d804c070 con 0x5629b769db00 2024-04-26T21:58:16.590 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.589+0000 7f85ea5e2740 10 client.25282 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T21:58:16.590 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.589+0000 7f85ea5e2740 10 client.25282 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:58:16.590 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.589+0000 7f85ea5e2740 20 client.25282 populate_metadata read hostname 'smithi072' 2024-04-26T21:58:16.590 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.590+0000 7f85ea5e2740 1 --2- 172.21.15.72:0/2233015863 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x5629b76b60b0 0x5629b76b84d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:16.590 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.590+0000 7f85ea5e2740 4 client.25282 mds_command: new command op to 34517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:58:16.590 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.590+0000 7f85ea5e2740 1 -- 172.21.15.72:0/2233015863 --> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5629b76017e0 con 0x5629b76b60b0 2024-04-26T21:58:16.591 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.590+0000 7f85e4c00640 1 --2- 172.21.15.72:0/2233015863 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x5629b76b60b0 0x5629b76b84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:16.591 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.591+0000 7f85e4c00640 1 --2- 172.21.15.72:0/2233015863 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x5629b76b60b0 0x5629b76b84d0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f1c1ed2c1e6f17f server_cookie=6faaefb4c014e5d in_seq=0 out_seq=0 2024-04-26T21:58:16.591 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.591+0000 7f85c6000640 10 client.25282 ms_handle_connect on v2:172.21.15.76:6838/327145636 2024-04-26T21:58:16.592 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.592+0000 7f85c6000640 1 -- 172.21.15.72:0/2233015863 <== mds.0 v2:172.21.15.76:6838/327145636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5629b76017e0 con 0x5629b76b60b0 2024-04-26T21:58:16.592 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.592+0000 7f85c6000640 10 client.25282 handle_command_reply: tid=0 2024-04-26T21:58:16.592 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.592+0000 7f85ea5e2740 1 client.25282 shutdown 2024-04-26T21:58:16.592 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.592+0000 7f85ea5e2740 1 -- 172.21.15.72:0/2233015863 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5629b769db00 msgr2=0x5629b7696180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:16.592 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.592+0000 7f85ea5e2740 1 --2- 172.21.15.72:0/2233015863 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5629b769db00 0x5629b7696180 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f85d8043000 tx=0x7f85d80099c0 comp rx=0 tx=0).stop 2024-04-26T21:58:16.592 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.592+0000 7f85ea5e2740 1 -- 172.21.15.72:0/2233015863 shutdown_connections 2024-04-26T21:58:16.592 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.592+0000 7f85ea5e2740 1 --2- 172.21.15.72:0/2233015863 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x5629b76b60b0 0x5629b76b84d0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:16.592 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.592+0000 7f85ea5e2740 1 --2- 172.21.15.72:0/2233015863 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5629b769db00 0x5629b7696180 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:16.593 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.592+0000 7f85ea5e2740 1 --2- 172.21.15.72:0/2233015863 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5629b7698990 0x5629b7691130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:16.593 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.592+0000 7f85ea5e2740 1 --2- 172.21.15.72:0/2233015863 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5629b7697fa0 0x5629b7690bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:16.593 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.592+0000 7f85ea5e2740 1 -- 172.21.15.72:0/2233015863 >> 172.21.15.72:0/2233015863 conn(0x5629b768adb0 msgr2=0x5629b768a040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:58:16.593 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.592+0000 7f85c6000640 0 client.25282 ms_handle_reset on v2:172.21.15.76:6838/327145636 2024-04-26T21:58:16.593 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.593+0000 7f85ea5e2740 1 -- 172.21.15.72:0/2233015863 shutdown_connections 2024-04-26T21:58:16.593 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.593+0000 7f85ea5e2740 1 -- 172.21.15.72:0/2233015863 wait complete. 2024-04-26T21:58:16.593 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.593+0000 7f85ea5e2740 20 client.25282 trim_cache size 0 max 16384 2024-04-26T21:58:16.623 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.622+0000 7f85ea5e2740 1 Processor -- start 2024-04-26T21:58:16.623 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.623+0000 7f85ea5e2740 1 -- start start 2024-04-26T21:58:16.623 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.623+0000 7f85ea5e2740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5629b7697fa0 0x5629b76901a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:16.623 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.623+0000 7f85ea5e2740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5629b7698990 0x5629b769cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:16.623 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.623+0000 7f85ea5e2740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5629b769db00 0x5629b76910b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:16.624 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.623+0000 7f85ea5e2740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f85c0010250 con 0x5629b7698990 2024-04-26T21:58:16.624 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.623+0000 7f85ea5e2740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f85c000ede0 con 0x5629b7697fa0 2024-04-26T21:58:16.624 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.623+0000 7f85ea5e2740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f85c000f4b0 con 0x5629b769db00 2024-04-26T21:58:16.624 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.623+0000 7f85dfe00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5629b7698990 0x5629b769cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:16.624 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.624+0000 7f85e4c00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5629b7697fa0 0x5629b76901a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:16.624 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.624+0000 7f85dfe00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5629b7698990 0x5629b769cfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:45908/0 (socket says 172.21.15.72:45908) 2024-04-26T21:58:16.625 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.624+0000 7f85dfe00640 1 -- 172.21.15.72:0/2828400541 learned_addr learned my addr 172.21.15.72:0/2828400541 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:58:16.625 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.624+0000 7f85e5600640 1 --2- 172.21.15.72:0/2828400541 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5629b769db00 0x5629b76910b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:16.625 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.624+0000 7f85dfe00640 1 -- 172.21.15.72:0/2828400541 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5629b769db00 msgr2=0x5629b76910b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:58:16.625 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.624+0000 7f85dfe00640 1 --2- 172.21.15.72:0/2828400541 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5629b769db00 0x5629b76910b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:16.625 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.624+0000 7f85dfe00640 1 -- 172.21.15.72:0/2828400541 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5629b7697fa0 msgr2=0x5629b76901a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:58:16.625 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.624+0000 7f85dfe00640 1 --2- 172.21.15.72:0/2828400541 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5629b7697fa0 0x5629b76901a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:16.625 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.624+0000 7f85dfe00640 1 -- 172.21.15.72:0/2828400541 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5629b72c50f0 con 0x5629b7698990 2024-04-26T21:58:16.626 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.625+0000 7f85dfe00640 1 --2- 172.21.15.72:0/2828400541 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5629b7698990 0x5629b769cfb0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f85cc009c60 tx=0x7f85cc013830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d65b55feb36bc514 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:58:16.626 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.626+0000 7f85d1200640 1 -- 172.21.15.72:0/2828400541 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85cc021070 con 0x5629b7698990 2024-04-26T21:58:16.626 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.626+0000 7f85d1200640 1 -- 172.21.15.72:0/2828400541 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f85cc009d50 con 0x5629b7698990 2024-04-26T21:58:16.626 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.626+0000 7f85d1200640 1 -- 172.21.15.72:0/2828400541 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85cc011640 con 0x5629b7698990 2024-04-26T21:58:16.626 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.627+0000 7f85ea5e2740 1 -- 172.21.15.72:0/2828400541 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5629b7698990 msgr2=0x5629b769cfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:16.626 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.627+0000 7f85ea5e2740 1 --2- 172.21.15.72:0/2828400541 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5629b7698990 0x5629b769cfb0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f85cc009c60 tx=0x7f85cc013830 comp rx=0 tx=0).stop 2024-04-26T21:58:16.627 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.627+0000 7f85ea5e2740 1 -- 172.21.15.72:0/2828400541 shutdown_connections 2024-04-26T21:58:16.627 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.627+0000 7f85ea5e2740 1 --2- 172.21.15.72:0/2828400541 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5629b769db00 0x5629b76910b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:16.627 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.627+0000 7f85ea5e2740 1 --2- 172.21.15.72:0/2828400541 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5629b7698990 0x5629b769cfb0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:16.628 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.627+0000 7f85ea5e2740 1 --2- 172.21.15.72:0/2828400541 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5629b7697fa0 0x5629b76901a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:16.628 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.627+0000 7f85ea5e2740 1 -- 172.21.15.72:0/2828400541 >> 172.21.15.72:0/2828400541 conn(0x5629b768adb0 msgr2=0x5629b768ddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:58:16.628 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.628+0000 7f85ea5e2740 1 -- 172.21.15.72:0/2828400541 shutdown_connections 2024-04-26T21:58:16.628 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.628+0000 7f85ea5e2740 1 -- 172.21.15.72:0/2828400541 wait complete. 2024-04-26T21:58:16.628 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.629+0000 7f85ea5e2740 1 -- start start 2024-04-26T21:58:16.629 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.629+0000 7f85ea5e2740 1 Processor -- start 2024-04-26T21:58:16.629 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.629+0000 7f85ea5e2740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5629b7697fa0 0x5629b76c6d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:16.630 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.629+0000 7f85ea5e2740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5629b7698990 0x5629b76c72c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:16.630 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.629+0000 7f85ea5e2740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5629b769db00 0x5629b76c8280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:16.631 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.629+0000 7f85ea5e2740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f85c000c6f0 con 0x5629b7698990 2024-04-26T21:58:16.631 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.629+0000 7f85ea5e2740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f85c000fb80 con 0x5629b7697fa0 2024-04-26T21:58:16.631 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.629+0000 7f85ea5e2740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f85c000cd60 con 0x5629b769db00 2024-04-26T21:58:16.631 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.629+0000 7f85ea5e2740 10 client.0 _ll_register_callbacks cb 0x7f85cc0095f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:58:16.631 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.630+0000 7f85e4c00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5629b7697fa0 0x5629b76c6d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:16.631 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.630+0000 7f85e4c00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5629b7697fa0 0x5629b76c6d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.72:48600/0 (socket says 172.21.15.72:48600) 2024-04-26T21:58:16.631 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.630+0000 7f85e4c00640 1 -- 172.21.15.72:0/2172496308 learned_addr learned my addr 172.21.15.72:0/2172496308 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:58:16.631 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.630+0000 7f85e5600640 1 --2- 172.21.15.72:0/2172496308 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5629b769db00 0x5629b76c8280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:16.631 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.630+0000 7f85dfe00640 1 --2- 172.21.15.72:0/2172496308 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5629b7698990 0x5629b76c72c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:16.631 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.631+0000 7f85e4c00640 1 -- 172.21.15.72:0/2172496308 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5629b769db00 msgr2=0x5629b76c8280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:58:16.631 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.631+0000 7f85e4c00640 1 --2- 172.21.15.72:0/2172496308 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5629b769db00 0x5629b76c8280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:16.631 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.631+0000 7f85e4c00640 1 -- 172.21.15.72:0/2172496308 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5629b7698990 msgr2=0x5629b76c72c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:58:16.631 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.631+0000 7f85e4c00640 1 --2- 172.21.15.72:0/2172496308 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5629b7698990 0x5629b76c72c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:16.632 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.631+0000 7f85e4c00640 1 -- 172.21.15.72:0/2172496308 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5629b75bba30 con 0x5629b7697fa0 2024-04-26T21:58:16.632 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.631+0000 7f85dfe00640 1 --2- 172.21.15.72:0/2172496308 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5629b7698990 0x5629b76c72c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:58:16.633 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.631+0000 7f85e4c00640 1 --2- 172.21.15.72:0/2172496308 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5629b7697fa0 0x5629b76c6d80 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f85d400cfd0 tx=0x7f85d4009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b9ed03e80785a4f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:58:16.633 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.631+0000 7f85c6000640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-04-26T21:58:16.633 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.631+0000 7f85c6000640 1 -- 172.21.15.72:0/2172496308 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85d40a20b0 con 0x5629b7697fa0 2024-04-26T21:58:16.634 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.631+0000 7f85ea5e2740 1 -- 172.21.15.72:0/2172496308 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f85cc009d50 con 0x5629b7697fa0 2024-04-26T21:58:16.634 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.631+0000 7f85c6000640 1 -- 172.21.15.72:0/2172496308 <== mon.2 v2:172.21.15.76:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7f85d40a2250 con 0x5629b7697fa0 2024-04-26T21:58:16.634 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.632+0000 7f85c6000640 1 -- 172.21.15.72:0/2172496308 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f85d409c0c0 con 0x5629b7697fa0 2024-04-26T21:58:16.634 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.632+0000 7f85ea5e2740 10 client.35357 fetch_fsmap learned FSMap version 27 2024-04-26T21:58:16.634 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.632+0000 7f85ea5e2740 1 -- 172.21.15.72:0/2172496308 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5629b76c9a20 con 0x5629b7697fa0 2024-04-26T21:58:16.634 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.632+0000 7f85c6000640 1 -- 172.21.15.72:0/2172496308 <== mon.2 v2:172.21.15.76:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f85cc009d50 con 0x5629b7697fa0 2024-04-26T21:58:16.634 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.632+0000 7f85c6000640 1 -- 172.21.15.72:0/2172496308 <== mon.2 v2:172.21.15.76:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f85d4098b50 con 0x5629b7697fa0 2024-04-26T21:58:16.634 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.632+0000 7f85ea5e2740 10 client.35357 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T21:58:16.634 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.632+0000 7f85ea5e2740 10 client.35357 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:58:16.634 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.632+0000 7f85ea5e2740 20 client.35357 populate_metadata read hostname 'smithi072' 2024-04-26T21:58:16.634 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.632+0000 7f85ea5e2740 1 --2- 172.21.15.72:0/2172496308 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x5629b771c6a0 0x5629b771eac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:16.634 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.632+0000 7f85ea5e2740 4 client.35357 mds_command: new command op to 34517 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-26T21:58:16.634 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.632+0000 7f85ea5e2740 1 -- 172.21.15.72:0/2172496308 --> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5629b75fb650 con 0x5629b771c6a0 2024-04-26T21:58:16.634 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.633+0000 7f85dfe00640 1 --2- 172.21.15.72:0/2172496308 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x5629b771c6a0 0x5629b771eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:16.635 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.633+0000 7f85dfe00640 1 --2- 172.21.15.72:0/2172496308 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x5629b771c6a0 0x5629b771eac0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=300ab1f1dc26fa88 server_cookie=831dd43aaa3b9a72 in_seq=0 out_seq=0 2024-04-26T21:58:16.635 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.633+0000 7f85c6000640 10 client.35357 ms_handle_connect on v2:172.21.15.76:6838/327145636 2024-04-26T21:58:16.635 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.633+0000 7f85c6000640 1 -- 172.21.15.72:0/2172496308 <== mds.0 v2:172.21.15.76:6838/327145636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5629b75fb650 con 0x5629b771c6a0 2024-04-26T21:58:16.635 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.633+0000 7f85c6000640 10 client.35357 handle_command_reply: tid=0 2024-04-26T21:58:16.635 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.634+0000 7f85ea5e2740 1 client.35357 shutdown 2024-04-26T21:58:16.635 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.634+0000 7f85ea5e2740 1 -- 172.21.15.72:0/2172496308 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5629b7697fa0 msgr2=0x5629b76c6d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:16.635 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.634+0000 7f85ea5e2740 1 --2- 172.21.15.72:0/2172496308 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5629b7697fa0 0x5629b76c6d80 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f85d400cfd0 tx=0x7f85d4009310 comp rx=0 tx=0).stop 2024-04-26T21:58:16.636 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.634+0000 7f85ea5e2740 1 -- 172.21.15.72:0/2172496308 shutdown_connections 2024-04-26T21:58:16.636 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.634+0000 7f85ea5e2740 1 --2- 172.21.15.72:0/2172496308 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x5629b771c6a0 0x5629b771eac0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:16.636 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.634+0000 7f85ea5e2740 1 --2- 172.21.15.72:0/2172496308 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5629b769db00 0x5629b76c8280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:16.636 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.634+0000 7f85ea5e2740 1 --2- 172.21.15.72:0/2172496308 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5629b7698990 0x5629b76c72c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:16.636 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.634+0000 7f85ea5e2740 1 --2- 172.21.15.72:0/2172496308 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5629b7697fa0 0x5629b76c6d80 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:16.636 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.634+0000 7f85c6000640 0 client.35357 ms_handle_reset on v2:172.21.15.76:6838/327145636 2024-04-26T21:58:16.636 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.634+0000 7f85ea5e2740 1 -- 172.21.15.72:0/2172496308 >> 172.21.15.72:0/2172496308 conn(0x5629b768adb0 msgr2=0x5629b7690e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:58:16.636 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.634+0000 7f85ea5e2740 1 -- 172.21.15.72:0/2172496308 shutdown_connections 2024-04-26T21:58:16.636 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.634+0000 7f85ea5e2740 1 -- 172.21.15.72:0/2172496308 wait complete. 2024-04-26T21:58:16.636 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.634+0000 7f85ea5e2740 20 client.35357 trim_cache size 0 max 16384 2024-04-26T21:58:16.637 INFO:teuthology.orchestra.run.smithi072.stdout:{ 2024-04-26T21:58:16.637 INFO:teuthology.orchestra.run.smithi072.stdout: "status": "no active scrubs running", 2024-04-26T21:58:16.637 INFO:teuthology.orchestra.run.smithi072.stdout: "scrubs": {} 2024-04-26T21:58:16.637 INFO:teuthology.orchestra.run.smithi072.stdout:} 2024-04-26T21:58:16.638 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.635+0000 7f85e7600640 1 -- 172.21.15.72:0/3462852931 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f85c80a3d80 msgr2=0x7f85c80a6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:16.638 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.635+0000 7f85e7600640 1 --2- 172.21.15.72:0/3462852931 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f85c80a3d80 0x7f85c80a6240 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f85d8009ca0 tx=0x7f85d802ae50 comp rx=0 tx=0).stop 2024-04-26T21:58:16.638 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.635+0000 7f85e7600640 1 -- 172.21.15.72:0/3462852931 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f85e0075330 msgr2=0x7f85e019ac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:16.638 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.635+0000 7f85e7600640 1 --2- 172.21.15.72:0/3462852931 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f85e0075330 0x7f85e019ac70 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f85d400e9f0 tx=0x7f85d400eeb0 comp rx=0 tx=0).stop 2024-04-26T21:58:16.638 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.635+0000 7f85e7600640 1 -- 172.21.15.72:0/3462852931 shutdown_connections 2024-04-26T21:58:16.638 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.635+0000 7f85e7600640 1 --2- 172.21.15.72:0/3462852931 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f85c80a3d80 0x7f85c80a6240 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:16.638 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.635+0000 7f85e7600640 1 --2- 172.21.15.72:0/3462852931 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f85e010a810 0x7f85e01a2210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:16.638 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.635+0000 7f85e7600640 1 --2- 172.21.15.72:0/3462852931 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f85e0076910 0x7f85e019b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:16.638 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.635+0000 7f85e7600640 1 --2- 172.21.15.72:0/3462852931 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f85e0075330 0x7f85e019ac70 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:16.638 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.635+0000 7f85e7600640 1 -- 172.21.15.72:0/3462852931 >> 172.21.15.72:0/3462852931 conn(0x7f85e00fdf10 msgr2=0x7f85e01067a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:58:16.638 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.635+0000 7f85e7600640 1 -- 172.21.15.72:0/3462852931 shutdown_connections 2024-04-26T21:58:16.638 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.635+0000 7f85e7600640 1 -- 172.21.15.72:0/3462852931 wait complete. 2024-04-26T21:58:16.653 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-26T21:58:16.653 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:58:16.879 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.879+0000 7f299d200640 1 -- 172.21.15.72:0/4045942443 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2998103b40 msgr2=0x7f2998103fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:16.879 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.879+0000 7f299d200640 1 --2- 172.21.15.72:0/4045942443 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2998103b40 0x7f2998103fc0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f2984009860 tx=0x7f298402a340 comp rx=0 tx=0).stop 2024-04-26T21:58:16.879 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.879+0000 7f299d200640 1 -- 172.21.15.72:0/4045942443 shutdown_connections 2024-04-26T21:58:16.879 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.879+0000 7f299d200640 1 --2- 172.21.15.72:0/4045942443 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2998104500 0x7f299810ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:16.879 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.879+0000 7f299d200640 1 --2- 172.21.15.72:0/4045942443 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2998103b40 0x7f2998103fc0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:16.879 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.879+0000 7f299d200640 1 --2- 172.21.15.72:0/4045942443 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2998102950 0x7f2998102d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:16.879 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.879+0000 7f299d200640 1 -- 172.21.15.72:0/4045942443 >> 172.21.15.72:0/4045942443 conn(0x7f29980fdee0 msgr2=0x7f2998100300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:58:16.880 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.879+0000 7f299d200640 1 -- 172.21.15.72:0/4045942443 shutdown_connections 2024-04-26T21:58:16.880 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.880+0000 7f299d200640 1 -- 172.21.15.72:0/4045942443 wait complete. 2024-04-26T21:58:16.881 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.881+0000 7f299d200640 1 Processor -- start 2024-04-26T21:58:16.881 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.881+0000 7f299d200640 1 -- start start 2024-04-26T21:58:16.881 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.882+0000 7f299d200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2998102950 0x7f29981086d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:16.882 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.882+0000 7f299d200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2998103b40 0x7f2998108c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:16.882 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.882+0000 7f299d200640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2998104500 0x7f2998105240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:16.882 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.882+0000 7f299d200640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f299810e380 con 0x7f2998104500 2024-04-26T21:58:16.882 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.882+0000 7f299d200640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f299810e200 con 0x7f2998103b40 2024-04-26T21:58:16.882 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.882+0000 7f299d200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f299810e500 con 0x7f2998102950 2024-04-26T21:58:16.883 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.882+0000 7f2995e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2998102950 0x7f29981086d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:16.883 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.882+0000 7f2995e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2998102950 0x7f29981086d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.72:55760/0 (socket says 172.21.15.72:55760) 2024-04-26T21:58:16.883 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.882+0000 7f2995e00640 1 -- 172.21.15.72:0/1914470993 learned_addr learned my addr 172.21.15.72:0/1914470993 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:58:16.883 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.882+0000 7f2996800640 1 --2- 172.21.15.72:0/1914470993 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2998104500 0x7f2998105240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:16.883 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.882+0000 7f2995400640 1 --2- 172.21.15.72:0/1914470993 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2998103b40 0x7f2998108c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:16.883 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.883+0000 7f2996800640 1 -- 172.21.15.72:0/1914470993 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2998102950 msgr2=0x7f29981086d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:16.883 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.883+0000 7f2996800640 1 --2- 172.21.15.72:0/1914470993 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2998102950 0x7f29981086d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:16.883 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.883+0000 7f2996800640 1 -- 172.21.15.72:0/1914470993 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2998103b40 msgr2=0x7f2998108c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:16.884 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.883+0000 7f2996800640 1 --2- 172.21.15.72:0/1914470993 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2998103b40 0x7f2998108c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:16.884 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.883+0000 7f2996800640 1 -- 172.21.15.72:0/1914470993 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2998105b00 con 0x7f2998104500 2024-04-26T21:58:16.884 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.883+0000 7f2995e00640 1 --2- 172.21.15.72:0/1914470993 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2998102950 0x7f29981086d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:58:16.884 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.883+0000 7f2996800640 1 --2- 172.21.15.72:0/1914470993 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2998104500 0x7f2998105240 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f2990009fd0 tx=0x7f299000ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:58:16.884 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.883+0000 7f298ea00640 1 -- 172.21.15.72:0/1914470993 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f299000ce20 con 0x7f2998104500 2024-04-26T21:58:16.884 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.884+0000 7f298ea00640 1 -- 172.21.15.72:0/1914470993 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2990005020 con 0x7f2998104500 2024-04-26T21:58:16.884 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.884+0000 7f299d200640 1 -- 172.21.15.72:0/1914470993 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29981a7150 con 0x7f2998104500 2024-04-26T21:58:16.884 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.884+0000 7f298ea00640 1 -- 172.21.15.72:0/1914470993 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2990010690 con 0x7f2998104500 2024-04-26T21:58:16.885 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.884+0000 7f299d200640 1 -- 172.21.15.72:0/1914470993 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29981a7550 con 0x7f2998104500 2024-04-26T21:58:16.887 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.887+0000 7f299d200640 1 -- 172.21.15.72:0/1914470993 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2978005290 con 0x7f2998104500 2024-04-26T21:58:16.890 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.888+0000 7f298ea00640 1 -- 172.21.15.72:0/1914470993 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f2990004a40 con 0x7f2998104500 2024-04-26T21:58:16.890 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.890+0000 7f298ea00640 1 --2- 172.21.15.72:0/1914470993 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f297c0a3c80 0x7f297c0a6140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:16.890 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.890+0000 7f298ea00640 1 -- 172.21.15.72:0/1914470993 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7f29900d0be0 con 0x7f2998104500 2024-04-26T21:58:16.891 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.890+0000 7f2995e00640 1 --2- 172.21.15.72:0/1914470993 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f297c0a3c80 0x7f297c0a6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:16.891 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.892+0000 7f2995e00640 1 --2- 172.21.15.72:0/1914470993 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f297c0a3c80 0x7f297c0a6140 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f29880096c0 tx=0x7f2988009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:58:16.896 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:16.896+0000 7f298ea00640 1 -- 172.21.15.72:0/1914470993 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f299009d380 con 0x7f2998104500 2024-04-26T21:58:17.272 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.272+0000 7f299d200640 1 -- 172.21.15.72:0/1914470993 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2978005850 con 0x7f2998104500 2024-04-26T21:58:17.273 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.273+0000 7f298ea00640 1 -- 172.21.15.72:0/1914470993 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11931 (secure 0 0 0) 0x7f299009cc80 con 0x7f2998104500 2024-04-26T21:58:17.273 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T21:58:17.273 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6835/1320381959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6834","nonce":1320381959},{"type":"v1","addr":"172.21.15.72:6835","nonce":1320381959}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6837/2142200512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6836","nonce":2142200512},{"type":"v1","addr":"172.21.15.72:6837","nonce":2142200512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T21:52:50.567857+0000","modified":"2024-04-26T21:55:22.894383+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34517,"mds_1":34472,"mds_2":34442,"mds_3":34406,"mds_4":24463},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14652":{"gid":14652,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.72:6839/1312898281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6838","nonce":1312898281},{"type":"v1","addr":"172.21.15.72:6839","nonce":1312898281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14664":{"gid":14664,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.72:6841/3257692339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6840","nonce":3257692339},{"type":"v1","addr":"172.21.15.72:6841","nonce":3257692339}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24388":{"gid":24388,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6833/4236668465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":4236668465},{"type":"v1","addr":"172.21.15.164:6833","nonce":4236668465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6835/2391114169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":2391114169},{"type":"v1","addr":"172.21.15.164:6835","nonce":2391114169}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6837/3585829992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":3585829992},{"type":"v1","addr":"172.21.15.164:6837","nonce":3585829992}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.164:6839/1636226820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":1636226820},{"type":"v1","addr":"172.21.15.164:6839","nonce":1636226820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34406":{"gid":34406,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":15,"addr":"172.21.15.76:6833/3869483750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":3869483750},{"type":"v1","addr":"172.21.15.76:6833","nonce":3869483750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34442":{"gid":34442,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.76:6835/3517429435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":3517429435},{"type":"v1","addr":"172.21.15.76:6835","nonce":3517429435}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34472":{"gid":34472,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.76:6837/1150678991","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":1150678991},{"type":"v1","addr":"172.21.15.76:6837","nonce":1150678991}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34517":{"gid":34517,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.76:6839/327145636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6838","nonce":327145636},{"type":"v1","addr":"172.21.15.76:6839","nonce":327145636}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34517,"qdb_cluster":[34472,34517,34442,34406,24463]},"id":1}]} 2024-04-26T21:58:17.273 INFO:teuthology.orchestra.run.smithi072.stderr:dumped fsmap epoch 27 2024-04-26T21:58:17.276 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.275+0000 7f299d200640 1 -- 172.21.15.72:0/1914470993 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f297c0a3c80 msgr2=0x7f297c0a6140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:17.276 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.275+0000 7f299d200640 1 --2- 172.21.15.72:0/1914470993 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f297c0a3c80 0x7f297c0a6140 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f29880096c0 tx=0x7f2988009260 comp rx=0 tx=0).stop 2024-04-26T21:58:17.276 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.275+0000 7f299d200640 1 -- 172.21.15.72:0/1914470993 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2998104500 msgr2=0x7f2998105240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:17.276 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.275+0000 7f299d200640 1 --2- 172.21.15.72:0/1914470993 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2998104500 0x7f2998105240 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f2990009fd0 tx=0x7f299000ef10 comp rx=0 tx=0).stop 2024-04-26T21:58:17.276 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.276+0000 7f299d200640 1 -- 172.21.15.72:0/1914470993 shutdown_connections 2024-04-26T21:58:17.276 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.276+0000 7f299d200640 1 --2- 172.21.15.72:0/1914470993 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2998104500 0x7f2998105240 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:17.276 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.276+0000 7f299d200640 1 --2- 172.21.15.72:0/1914470993 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2998103b40 0x7f2998108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:17.276 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.276+0000 7f299d200640 1 --2- 172.21.15.72:0/1914470993 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f297c0a3c80 0x7f297c0a6140 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:17.276 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.276+0000 7f299d200640 1 --2- 172.21.15.72:0/1914470993 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2998102950 0x7f29981086d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:17.277 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.276+0000 7f299d200640 1 -- 172.21.15.72:0/1914470993 >> 172.21.15.72:0/1914470993 conn(0x7f29980fdee0 msgr2=0x7f2998074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:58:17.277 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.276+0000 7f299d200640 1 -- 172.21.15.72:0/1914470993 shutdown_connections 2024-04-26T21:58:17.277 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.276+0000 7f299d200640 1 -- 172.21.15.72:0/1914470993 wait complete. 2024-04-26T21:58:17.295 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:58:16 smithi072 ceph-mon[28138]: pgmap v534: 97 pgs: 97 active+clean; 258 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 MiB/s wr, 2.70k op/s 2024-04-26T21:58:17.298 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-26T21:58:17.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:58:16 smithi076 ceph-mon[31823]: pgmap v534: 97 pgs: 97 active+clean; 258 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 MiB/s wr, 2.70k op/s 2024-04-26T21:58:17.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:58:16 smithi164 ceph-mon[31916]: pgmap v534: 97 pgs: 97 active+clean; 258 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 MiB/s wr, 2.70k op/s 2024-04-26T21:58:17.545 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.545+0000 7fcaa3200640 1 -- 172.21.15.72:0/3725676425 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcaa4106620 msgr2=0x7fcaa410ceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:17.545 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.545+0000 7fcaa3200640 1 --2- 172.21.15.72:0/3725676425 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcaa4106620 0x7fcaa410ceb0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fca9c009ca0 tx=0x7fca9c02a8b0 comp rx=0 tx=0).stop 2024-04-26T21:58:17.545 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.545+0000 7fcaa3200640 1 -- 172.21.15.72:0/3725676425 shutdown_connections 2024-04-26T21:58:17.545 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.545+0000 7fcaa3200640 1 --2- 172.21.15.72:0/3725676425 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcaa4106620 0x7fcaa410ceb0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:17.545 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.545+0000 7fcaa3200640 1 --2- 172.21.15.72:0/3725676425 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcaa4105c60 0x7fcaa41060e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:17.545 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.545+0000 7fcaa3200640 1 --2- 172.21.15.72:0/3725676425 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcaa4104a70 0x7fcaa4104e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:17.546 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.545+0000 7fcaa3200640 1 -- 172.21.15.72:0/3725676425 >> 172.21.15.72:0/3725676425 conn(0x7fcaa40fffe0 msgr2=0x7fcaa4102420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:58:17.546 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.545+0000 7fcaa3200640 1 -- 172.21.15.72:0/3725676425 shutdown_connections 2024-04-26T21:58:17.546 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.545+0000 7fcaa3200640 1 -- 172.21.15.72:0/3725676425 wait complete. 2024-04-26T21:58:17.547 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.546+0000 7fcaa3200640 1 Processor -- start 2024-04-26T21:58:17.547 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.546+0000 7fcaa3200640 1 -- start start 2024-04-26T21:58:17.547 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.546+0000 7fcaa3200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcaa4104a70 0x7fcaa419ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:17.547 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.546+0000 7fcaa3200640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcaa4105c60 0x7fcaa419b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:17.547 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.547+0000 7fcaa3200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcaa4106620 0x7fcaa41a20e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:17.547 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.547+0000 7fcaa3200640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fcaa4110450 con 0x7fcaa4105c60 2024-04-26T21:58:17.547 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.547+0000 7fcaa3200640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fcaa41102d0 con 0x7fcaa4104a70 2024-04-26T21:58:17.548 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.547+0000 7fcaa3200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fcaa41105d0 con 0x7fcaa4106620 2024-04-26T21:58:17.548 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.547+0000 7fcaa1400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcaa4105c60 0x7fcaa419b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:17.548 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.547+0000 7fcaa1e00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcaa4104a70 0x7fcaa419ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:17.548 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.547+0000 7fcaa1e00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcaa4104a70 0x7fcaa419ab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.72:48624/0 (socket says 172.21.15.72:48624) 2024-04-26T21:58:17.548 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.547+0000 7fcaa1e00640 1 -- 172.21.15.72:0/3090874690 learned_addr learned my addr 172.21.15.72:0/3090874690 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:58:17.548 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.547+0000 7fcaa2800640 1 --2- 172.21.15.72:0/3090874690 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcaa4106620 0x7fcaa41a20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:17.549 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.548+0000 7fcaa1e00640 1 -- 172.21.15.72:0/3090874690 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcaa4106620 msgr2=0x7fcaa41a20e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:17.549 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.548+0000 7fcaa1e00640 1 --2- 172.21.15.72:0/3090874690 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcaa4106620 0x7fcaa41a20e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:17.549 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.548+0000 7fcaa1e00640 1 -- 172.21.15.72:0/3090874690 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcaa4105c60 msgr2=0x7fcaa419b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:17.549 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.548+0000 7fcaa1e00640 1 --2- 172.21.15.72:0/3090874690 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcaa4105c60 0x7fcaa419b070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:17.549 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.548+0000 7fcaa1e00640 1 -- 172.21.15.72:0/3090874690 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcaa41a2750 con 0x7fcaa4104a70 2024-04-26T21:58:17.549 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.548+0000 7fcaa1e00640 1 --2- 172.21.15.72:0/3090874690 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcaa4104a70 0x7fcaa419ab30 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fca9400d950 tx=0x7fca9400de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:58:17.549 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.549+0000 7fca9aa00640 1 -- 172.21.15.72:0/3090874690 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca94017070 con 0x7fcaa4104a70 2024-04-26T21:58:17.550 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.549+0000 7fcaa3200640 1 -- 172.21.15.72:0/3090874690 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcaa41a2a40 con 0x7fcaa4104a70 2024-04-26T21:58:17.550 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.549+0000 7fcaa3200640 1 -- 172.21.15.72:0/3090874690 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcaa41a2f90 con 0x7fcaa4104a70 2024-04-26T21:58:17.550 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.549+0000 7fca9aa00640 1 -- 172.21.15.72:0/3090874690 <== mon.2 v2:172.21.15.76:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fca94009bb0 con 0x7fcaa4104a70 2024-04-26T21:58:17.550 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.549+0000 7fca9aa00640 1 -- 172.21.15.72:0/3090874690 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca9400f620 con 0x7fcaa4104a70 2024-04-26T21:58:17.554 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.553+0000 7fcaabd6c740 1 Processor -- start 2024-04-26T21:58:17.554 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.553+0000 7fcaabd6c740 1 -- start start 2024-04-26T21:58:17.554 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.553+0000 7fca9aa00640 1 -- 172.21.15.72:0/3090874690 <== mon.2 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fca9401a070 con 0x7fcaa4104a70 2024-04-26T21:58:17.554 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.554+0000 7fcaabd6c740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561b3a724050 0x561b3a724470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:17.555 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.554+0000 7fcaabd6c740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561b3a724a40 0x561b3a729540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:17.555 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.554+0000 7fcaabd6c740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561b3a729bb0 0x561b3a72c040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:17.555 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.554+0000 7fcaabd6c740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x561b3a687c70 con 0x561b3a724050 2024-04-26T21:58:17.555 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.554+0000 7fcaabd6c740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x561b3a640980 con 0x561b3a729bb0 2024-04-26T21:58:17.555 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.554+0000 7fcaabd6c740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x561b3a683c30 con 0x561b3a724a40 2024-04-26T21:58:17.555 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.554+0000 7fcaa2800640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561b3a724050 0x561b3a724470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:17.555 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.554+0000 7fcaa2800640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561b3a724050 0x561b3a724470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:45936/0 (socket says 172.21.15.72:45936) 2024-04-26T21:58:17.555 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.554+0000 7fcaa2800640 1 -- 172.21.15.72:0/3406557112 learned_addr learned my addr 172.21.15.72:0/3406557112 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:58:17.555 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.554+0000 7fcaa1e00640 1 --2- 172.21.15.72:0/3406557112 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561b3a724a40 0x561b3a729540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:17.556 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.555+0000 7fcaa2800640 1 -- 172.21.15.72:0/3406557112 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561b3a724a40 msgr2=0x561b3a729540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:58:17.556 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.555+0000 7fcaa2800640 1 --2- 172.21.15.72:0/3406557112 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561b3a724a40 0x561b3a729540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:17.556 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.555+0000 7fcaa2800640 1 -- 172.21.15.72:0/3406557112 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561b3a729bb0 msgr2=0x561b3a72c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:58:17.556 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.556+0000 7fcaa1400640 1 --2- 172.21.15.72:0/3406557112 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561b3a729bb0 0x561b3a72c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:17.556 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.556+0000 7fcaa2800640 1 --2- 172.21.15.72:0/3406557112 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561b3a729bb0 0x561b3a72c040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:17.556 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.556+0000 7fcaa2800640 1 -- 172.21.15.72:0/3406557112 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561b3a6187b0 con 0x561b3a724050 2024-04-26T21:58:17.556 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.556+0000 7fcaa1400640 1 --2- 172.21.15.72:0/3406557112 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561b3a729bb0 0x561b3a72c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:58:17.556 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.556+0000 7fca9aa00640 1 --2- 172.21.15.72:0/3090874690 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fca880a3c80 0x7fca880a6140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:17.557 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.556+0000 7fcaa2800640 1 --2- 172.21.15.72:0/3406557112 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561b3a724050 0x561b3a724470 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fca9c035000 tx=0x7fca9c0098b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=800d756ce6fcc5c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:58:17.557 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.556+0000 7fca9aa00640 1 -- 172.21.15.72:0/3090874690 <== mon.2 v2:172.21.15.76:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7fca940cfed0 con 0x7fcaa4104a70 2024-04-26T21:58:17.557 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.556+0000 7fcaa1e00640 1 --2- 172.21.15.72:0/3090874690 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fca880a3c80 0x7fca880a6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:17.557 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.556+0000 7fca8f400640 1 -- 172.21.15.72:0/3406557112 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca9c0308b0 con 0x561b3a724050 2024-04-26T21:58:17.558 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.557+0000 7fca8f400640 1 -- 172.21.15.72:0/3406557112 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fca9c036030 con 0x561b3a724050 2024-04-26T21:58:17.558 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.557+0000 7fca8f400640 1 -- 172.21.15.72:0/3406557112 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca9c0043c0 con 0x561b3a724050 2024-04-26T21:58:17.558 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.557+0000 7fcaabd6c740 1 -- 172.21.15.72:0/3406557112 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561b3a724050 msgr2=0x561b3a724470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:17.558 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.557+0000 7fcaabd6c740 1 --2- 172.21.15.72:0/3406557112 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561b3a724050 0x561b3a724470 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fca9c035000 tx=0x7fca9c0098b0 comp rx=0 tx=0).stop 2024-04-26T21:58:17.558 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.557+0000 7fcaa1e00640 1 --2- 172.21.15.72:0/3090874690 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fca880a3c80 0x7fca880a6140 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x561b3a7254b0 tx=0x7fca940d3a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:58:17.558 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.558+0000 7fcaabd6c740 1 -- 172.21.15.72:0/3406557112 shutdown_connections 2024-04-26T21:58:17.558 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.558+0000 7fcaabd6c740 1 --2- 172.21.15.72:0/3406557112 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561b3a729bb0 0x561b3a72c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:17.558 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.558+0000 7fcaabd6c740 1 --2- 172.21.15.72:0/3406557112 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561b3a724a40 0x561b3a729540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:17.558 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.558+0000 7fcaabd6c740 1 --2- 172.21.15.72:0/3406557112 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561b3a724050 0x561b3a724470 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:17.558 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.558+0000 7fcaabd6c740 1 -- 172.21.15.72:0/3406557112 >> 172.21.15.72:0/3406557112 conn(0x561b3a71f6c0 msgr2=0x561b3a71fac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:58:17.558 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.558+0000 7fcaabd6c740 1 -- 172.21.15.72:0/3406557112 shutdown_connections 2024-04-26T21:58:17.558 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.558+0000 7fcaabd6c740 1 -- 172.21.15.72:0/3406557112 wait complete. 2024-04-26T21:58:17.559 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.558+0000 7fcaabd6c740 1 -- start start 2024-04-26T21:58:17.559 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.558+0000 7fcaabd6c740 1 Processor -- start 2024-04-26T21:58:17.559 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.560+0000 7fcaabd6c740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561b3a724050 0x561b3a72df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:17.559 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.560+0000 7fcaabd6c740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561b3a724a40 0x561b3a72e470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:17.560 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.560+0000 7fcaabd6c740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561b3a729bb0 0x561b3a7354b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:17.560 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.560+0000 7fcaabd6c740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x561b3a686f80 con 0x561b3a724050 2024-04-26T21:58:17.560 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.560+0000 7fcaabd6c740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x561b3a634260 con 0x561b3a724a40 2024-04-26T21:58:17.560 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.560+0000 7fcaabd6c740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x561b3a68c2a0 con 0x561b3a729bb0 2024-04-26T21:58:17.560 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.560+0000 7fcaabd6c740 10 client.0 _ll_register_callbacks cb 0x561b3a67e960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:58:17.560 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.560+0000 7fcaa1400640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561b3a724a40 0x561b3a72e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:17.560 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.560+0000 7fcaa1400640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561b3a724a40 0x561b3a72e470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.72:48632/0 (socket says 172.21.15.72:48632) 2024-04-26T21:58:17.560 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.560+0000 7fcaa2800640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561b3a724050 0x561b3a72df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:17.561 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.560+0000 7fcaa2800640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561b3a724050 0x561b3a72df30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:45948/0 (socket says 172.21.15.72:45948) 2024-04-26T21:58:17.561 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.560+0000 7fcaa2800640 1 -- 172.21.15.72:0/3505742732 learned_addr learned my addr 172.21.15.72:0/3505742732 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:58:17.561 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.561+0000 7fcaa2800640 1 --2- 172.21.15.72:0/3505742732 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561b3a729bb0 0x561b3a7354b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:17.561 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.561+0000 7fcaa1400640 1 -- 172.21.15.72:0/3505742732 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561b3a729bb0 msgr2=0x561b3a7354b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:58:17.561 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.561+0000 7fcaa1400640 1 --2- 172.21.15.72:0/3505742732 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561b3a729bb0 0x561b3a7354b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:17.561 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.561+0000 7fcaa1400640 1 -- 172.21.15.72:0/3505742732 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561b3a724050 msgr2=0x561b3a72df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:58:17.562 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.561+0000 7fcaa1400640 1 --2- 172.21.15.72:0/3505742732 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561b3a724050 0x561b3a72df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:17.562 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.561+0000 7fcaa1400640 1 -- 172.21.15.72:0/3505742732 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561b3a6161c0 con 0x561b3a724a40 2024-04-26T21:58:17.562 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.561+0000 7fcaa2800640 1 --2- 172.21.15.72:0/3505742732 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561b3a724050 0x561b3a72df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:58:17.562 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.562+0000 7fcaa1400640 1 --2- 172.21.15.72:0/3505742732 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561b3a724a40 0x561b3a72e470 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x561b3a720590 tx=0x7fca90013b70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ae79f4495744f9e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:58:17.562 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.562+0000 7fca83e00640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-04-26T21:58:17.563 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.562+0000 7fca83e00640 1 -- 172.21.15.72:0/3505742732 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca90004440 con 0x561b3a724a40 2024-04-26T21:58:17.563 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.562+0000 7fcaabd6c740 1 -- 172.21.15.72:0/3505742732 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fca9c036030 con 0x561b3a724a40 2024-04-26T21:58:17.563 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.562+0000 7fca83e00640 1 -- 172.21.15.72:0/3505742732 <== mon.2 v2:172.21.15.76:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7fca90011990 con 0x561b3a724a40 2024-04-26T21:58:17.563 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.562+0000 7fca83e00640 1 -- 172.21.15.72:0/3505742732 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fca90011b90 con 0x561b3a724a40 2024-04-26T21:58:17.563 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.562+0000 7fcaabd6c740 10 client.35372 fetch_fsmap learned FSMap version 27 2024-04-26T21:58:17.563 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.562+0000 7fcaabd6c740 1 -- 172.21.15.72:0/3505742732 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x561b3a3993f0 con 0x561b3a724a40 2024-04-26T21:58:17.563 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.563+0000 7fca83e00640 1 -- 172.21.15.72:0/3505742732 <== mon.2 v2:172.21.15.76:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fca9c036030 con 0x561b3a724a40 2024-04-26T21:58:17.563 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.563+0000 7fca83e00640 1 -- 172.21.15.72:0/3505742732 <== mon.2 v2:172.21.15.76:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fca9001a070 con 0x561b3a724a40 2024-04-26T21:58:17.564 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.563+0000 7fcaabd6c740 10 client.35372 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T21:58:17.564 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.563+0000 7fcaabd6c740 10 client.35372 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-04-26T21:58:17.564 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.563+0000 7fcaabd6c740 20 client.35372 populate_metadata read hostname 'smithi072' 2024-04-26T21:58:17.564 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.563+0000 7fcaabd6c740 1 --2- 172.21.15.72:0/3505742732 >> [v2:172.21.15.164:6838/1636226820,v1:172.21.15.164:6839/1636226820] conn(0x561b3a7399b0 0x561b3a73bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:17.564 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.563+0000 7fcaabd6c740 4 client.35372 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:58:17.564 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.563+0000 7fcaabd6c740 1 -- 172.21.15.72:0/3505742732 --> [v2:172.21.15.164:6838/1636226820,v1:172.21.15.164:6839/1636226820] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561b3a695880 con 0x561b3a7399b0 2024-04-26T21:58:17.564 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.564+0000 7fcaa2800640 1 --2- 172.21.15.72:0/3505742732 >> [v2:172.21.15.164:6838/1636226820,v1:172.21.15.164:6839/1636226820] conn(0x561b3a7399b0 0x561b3a73bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:17.565 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.565+0000 7fcaa2800640 1 --2- 172.21.15.72:0/3505742732 >> [v2:172.21.15.164:6838/1636226820,v1:172.21.15.164:6839/1636226820] conn(0x561b3a7399b0 0x561b3a73bdd0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=9d2eecac72af1c23 server_cookie=2df0c6ec7d653849 in_seq=0 out_seq=0 2024-04-26T21:58:17.565 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.565+0000 7fca83e00640 10 client.35372 ms_handle_connect on v2:172.21.15.164:6838/1636226820 2024-04-26T21:58:17.567 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.567+0000 7fca83e00640 1 -- 172.21.15.72:0/3505742732 <== mds.4 v2:172.21.15.164:6838/1636226820 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561b3a695880 con 0x561b3a7399b0 2024-04-26T21:58:17.568 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.567+0000 7fca83e00640 10 client.35372 handle_command_reply: tid=0 2024-04-26T21:58:17.568 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.567+0000 7fcaabd6c740 1 client.35372 shutdown 2024-04-26T21:58:17.568 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.567+0000 7fcaabd6c740 1 -- 172.21.15.72:0/3505742732 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561b3a724a40 msgr2=0x561b3a72e470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:17.568 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.567+0000 7fcaabd6c740 1 --2- 172.21.15.72:0/3505742732 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561b3a724a40 0x561b3a72e470 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x561b3a720590 tx=0x7fca90013b70 comp rx=0 tx=0).stop 2024-04-26T21:58:17.568 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.567+0000 7fcaabd6c740 1 -- 172.21.15.72:0/3505742732 shutdown_connections 2024-04-26T21:58:17.568 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.567+0000 7fcaabd6c740 1 --2- 172.21.15.72:0/3505742732 >> [v2:172.21.15.164:6838/1636226820,v1:172.21.15.164:6839/1636226820] conn(0x561b3a7399b0 0x561b3a73bdd0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:17.568 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.567+0000 7fcaabd6c740 1 --2- 172.21.15.72:0/3505742732 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561b3a729bb0 0x561b3a7354b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:17.568 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.567+0000 7fcaabd6c740 1 --2- 172.21.15.72:0/3505742732 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561b3a724a40 0x561b3a72e470 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:17.568 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.567+0000 7fcaabd6c740 1 --2- 172.21.15.72:0/3505742732 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561b3a724050 0x561b3a72df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:17.568 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.567+0000 7fcaabd6c740 1 -- 172.21.15.72:0/3505742732 >> 172.21.15.72:0/3505742732 conn(0x561b3a71f6c0 msgr2=0x561b3a71e7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:58:17.568 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.567+0000 7fca83e00640 0 client.35372 ms_handle_reset on v2:172.21.15.164:6838/1636226820 2024-04-26T21:58:17.568 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.568+0000 7fcaabd6c740 1 -- 172.21.15.72:0/3505742732 shutdown_connections 2024-04-26T21:58:17.568 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.568+0000 7fcaabd6c740 1 -- 172.21.15.72:0/3505742732 wait complete. 2024-04-26T21:58:17.568 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.568+0000 7fcaabd6c740 20 client.35372 trim_cache size 0 max 16384 2024-04-26T21:58:17.592 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.591+0000 7fcaabd6c740 1 Processor -- start 2024-04-26T21:58:17.592 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.591+0000 7fcaabd6c740 1 -- start start 2024-04-26T21:58:17.592 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.591+0000 7fcaabd6c740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561b3a724050 0x561b3a722680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:17.592 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.591+0000 7fcaabd6c740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561b3a724a40 0x561b3a735470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:17.592 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.591+0000 7fcaabd6c740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561b3a729bb0 0x561b3a722bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:17.592 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.591+0000 7fcaabd6c740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fca8400e6b0 con 0x561b3a729bb0 2024-04-26T21:58:17.592 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.591+0000 7fcaabd6c740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fca840101f0 con 0x561b3a724050 2024-04-26T21:58:17.593 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.591+0000 7fcaabd6c740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fca8400ed80 con 0x561b3a724a40 2024-04-26T21:58:17.593 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.592+0000 7fcaa2800640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561b3a729bb0 0x561b3a722bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:17.593 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.592+0000 7fcaa2800640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561b3a729bb0 0x561b3a722bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:45964/0 (socket says 172.21.15.72:45964) 2024-04-26T21:58:17.593 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.592+0000 7fcaa2800640 1 -- 172.21.15.72:0/502720522 learned_addr learned my addr 172.21.15.72:0/502720522 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:58:17.593 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.592+0000 7fcaa2800640 1 --2- 172.21.15.72:0/502720522 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561b3a724050 0x561b3a722680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:17.593 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.592+0000 7fcaa1400640 1 --2- 172.21.15.72:0/502720522 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561b3a724a40 0x561b3a735470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:17.593 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.592+0000 7fcaa2800640 1 -- 172.21.15.72:0/502720522 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561b3a724a40 msgr2=0x561b3a735470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:58:17.593 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.592+0000 7fcaa2800640 1 --2- 172.21.15.72:0/502720522 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561b3a724a40 0x561b3a735470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:17.593 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.592+0000 7fcaa2800640 1 -- 172.21.15.72:0/502720522 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561b3a724050 msgr2=0x561b3a722680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:58:17.593 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.592+0000 7fcaa2800640 1 --2- 172.21.15.72:0/502720522 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561b3a724050 0x561b3a722680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:17.593 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.592+0000 7fcaa2800640 1 -- 172.21.15.72:0/502720522 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561b3a4637d0 con 0x561b3a729bb0 2024-04-26T21:58:17.593 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.592+0000 7fcaa1400640 1 --2- 172.21.15.72:0/502720522 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561b3a724a40 0x561b3a735470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:58:17.595 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.592+0000 7fcaa2800640 1 --2- 172.21.15.72:0/502720522 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561b3a729bb0 0x561b3a722bc0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fca9c000f80 tx=0x7fca9c02a8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33ccbe8d3eb235d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:58:17.595 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.593+0000 7fca8ea00640 1 -- 172.21.15.72:0/502720522 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca9c030a20 con 0x561b3a729bb0 2024-04-26T21:58:17.595 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.593+0000 7fca8ea00640 1 -- 172.21.15.72:0/502720522 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fca9c036040 con 0x561b3a729bb0 2024-04-26T21:58:17.595 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.593+0000 7fca8ea00640 1 -- 172.21.15.72:0/502720522 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca9c052440 con 0x561b3a729bb0 2024-04-26T21:58:17.595 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.593+0000 7fcaabd6c740 1 -- 172.21.15.72:0/502720522 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561b3a729bb0 msgr2=0x561b3a722bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:17.595 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.593+0000 7fcaabd6c740 1 --2- 172.21.15.72:0/502720522 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561b3a729bb0 0x561b3a722bc0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fca9c000f80 tx=0x7fca9c02a8b0 comp rx=0 tx=0).stop 2024-04-26T21:58:17.595 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.594+0000 7fcaabd6c740 1 -- 172.21.15.72:0/502720522 shutdown_connections 2024-04-26T21:58:17.595 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.594+0000 7fcaabd6c740 1 --2- 172.21.15.72:0/502720522 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561b3a729bb0 0x561b3a722bc0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:17.595 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.594+0000 7fcaabd6c740 1 --2- 172.21.15.72:0/502720522 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561b3a724a40 0x561b3a735470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:17.596 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.594+0000 7fcaabd6c740 1 --2- 172.21.15.72:0/502720522 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561b3a724050 0x561b3a722680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:17.596 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.594+0000 7fcaabd6c740 1 -- 172.21.15.72:0/502720522 >> 172.21.15.72:0/502720522 conn(0x561b3a71f6c0 msgr2=0x561b3a72cd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:58:17.596 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.594+0000 7fcaabd6c740 1 -- 172.21.15.72:0/502720522 shutdown_connections 2024-04-26T21:58:17.596 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.594+0000 7fcaabd6c740 1 -- 172.21.15.72:0/502720522 wait complete. 2024-04-26T21:58:17.596 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.594+0000 7fcaabd6c740 1 -- start start 2024-04-26T21:58:17.596 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.594+0000 7fcaabd6c740 1 Processor -- start 2024-04-26T21:58:17.596 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.595+0000 7fcaabd6c740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561b3a724050 0x561b3a732330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:17.596 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.595+0000 7fcaabd6c740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561b3a724a40 0x561b3a732870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:17.596 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.595+0000 7fcaabd6c740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561b3a729bb0 0x561b3a7a6750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:17.596 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.595+0000 7fcaabd6c740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fca8400fb20 con 0x561b3a729bb0 2024-04-26T21:58:17.596 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.595+0000 7fcaabd6c740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fca8400f450 con 0x561b3a724a40 2024-04-26T21:58:17.596 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.595+0000 7fcaabd6c740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fca8400c690 con 0x561b3a724050 2024-04-26T21:58:17.596 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.595+0000 7fcaabd6c740 10 client.0 _ll_register_callbacks cb 0x7fca900097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:58:17.596 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.595+0000 7fcaa2800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561b3a724050 0x561b3a732330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:17.598 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.595+0000 7fcaa2800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561b3a724050 0x561b3a732330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.72:55810/0 (socket says 172.21.15.72:55810) 2024-04-26T21:58:17.598 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.595+0000 7fcaa2800640 1 -- 172.21.15.72:0/3577247039 learned_addr learned my addr 172.21.15.72:0/3577247039 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:58:17.599 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.595+0000 7fcaa2800640 1 --2- 172.21.15.72:0/3577247039 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561b3a729bb0 0x561b3a7a6750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:17.599 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.595+0000 7fcaa1400640 1 --2- 172.21.15.72:0/3577247039 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561b3a724a40 0x561b3a732870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:17.599 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.596+0000 7fcaa1400640 1 -- 172.21.15.72:0/3577247039 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561b3a724050 msgr2=0x561b3a732330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:58:17.599 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.596+0000 7fcaa1400640 1 --2- 172.21.15.72:0/3577247039 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561b3a724050 0x561b3a732330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:17.599 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.596+0000 7fcaa1400640 1 -- 172.21.15.72:0/3577247039 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561b3a729bb0 msgr2=0x561b3a7a6750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:58:17.599 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.596+0000 7fcaa1400640 1 --2- 172.21.15.72:0/3577247039 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561b3a729bb0 0x561b3a7a6750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:17.599 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.596+0000 7fcaa1400640 1 -- 172.21.15.72:0/3577247039 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561b3a62e2a0 con 0x561b3a724a40 2024-04-26T21:58:17.599 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.596+0000 7fcaa2800640 1 --2- 172.21.15.72:0/3577247039 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561b3a729bb0 0x561b3a7a6750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:58:17.599 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.597+0000 7fcaa1400640 1 --2- 172.21.15.72:0/3577247039 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561b3a724a40 0x561b3a732870 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fca90026700 tx=0x7fca90011f60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cdbaa487fa288ad4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:58:17.599 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.597+0000 7fca83e00640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-04-26T21:58:17.599 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.597+0000 7fca83e00640 1 -- 172.21.15.72:0/3577247039 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca900054b0 con 0x561b3a724a40 2024-04-26T21:58:17.599 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.597+0000 7fcaabd6c740 1 -- 172.21.15.72:0/3577247039 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fca9c036040 con 0x561b3a724a40 2024-04-26T21:58:17.599 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.597+0000 7fca83e00640 1 -- 172.21.15.72:0/3577247039 <== mon.2 v2:172.21.15.76:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7fca900035f0 con 0x561b3a724a40 2024-04-26T21:58:17.599 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.597+0000 7fca83e00640 1 -- 172.21.15.72:0/3577247039 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fca90005650 con 0x561b3a724a40 2024-04-26T21:58:17.600 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.597+0000 7fcaabd6c740 10 client.35378 fetch_fsmap learned FSMap version 27 2024-04-26T21:58:17.600 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.597+0000 7fcaabd6c740 1 -- 172.21.15.72:0/3577247039 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x561b3a7a7db0 con 0x561b3a724a40 2024-04-26T21:58:17.600 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.598+0000 7fca83e00640 1 -- 172.21.15.72:0/3577247039 <== mon.2 v2:172.21.15.76:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fca9c036040 con 0x561b3a724a40 2024-04-26T21:58:17.600 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.598+0000 7fca83e00640 1 -- 172.21.15.72:0/3577247039 <== mon.2 v2:172.21.15.76:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fca9002e070 con 0x561b3a724a40 2024-04-26T21:58:17.600 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.598+0000 7fcaabd6c740 10 client.35378 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T21:58:17.600 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.598+0000 7fcaabd6c740 10 client.35378 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-04-26T21:58:17.600 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.598+0000 7fcaabd6c740 20 client.35378 populate_metadata read hostname 'smithi072' 2024-04-26T21:58:17.600 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.598+0000 7fcaabd6c740 1 --2- 172.21.15.72:0/3577247039 >> [v2:172.21.15.164:6838/1636226820,v1:172.21.15.164:6839/1636226820] conn(0x561b3a7a8700 0x561b3a7aab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:17.600 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.598+0000 7fcaabd6c740 4 client.35378 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-26T21:58:17.600 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.598+0000 7fcaabd6c740 1 -- 172.21.15.72:0/3577247039 --> [v2:172.21.15.164:6838/1636226820,v1:172.21.15.164:6839/1636226820] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561b3a68f6f0 con 0x561b3a7a8700 2024-04-26T21:58:17.600 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.599+0000 7fcaa2800640 1 --2- 172.21.15.72:0/3577247039 >> [v2:172.21.15.164:6838/1636226820,v1:172.21.15.164:6839/1636226820] conn(0x561b3a7a8700 0x561b3a7aab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:17.601 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.600+0000 7fcaa2800640 1 --2- 172.21.15.72:0/3577247039 >> [v2:172.21.15.164:6838/1636226820,v1:172.21.15.164:6839/1636226820] conn(0x561b3a7a8700 0x561b3a7aab20 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=6ae8c82285b53761 server_cookie=b4680ccce4a36080 in_seq=0 out_seq=0 2024-04-26T21:58:17.601 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.600+0000 7fca83e00640 10 client.35378 ms_handle_connect on v2:172.21.15.164:6838/1636226820 2024-04-26T21:58:17.601 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.601+0000 7fca83e00640 1 -- 172.21.15.72:0/3577247039 <== mds.4 v2:172.21.15.164:6838/1636226820 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561b3a68f6f0 con 0x561b3a7a8700 2024-04-26T21:58:17.602 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.601+0000 7fca83e00640 10 client.35378 handle_command_reply: tid=0 2024-04-26T21:58:17.602 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.601+0000 7fcaabd6c740 1 client.35378 shutdown 2024-04-26T21:58:17.602 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.601+0000 7fcaabd6c740 1 -- 172.21.15.72:0/3577247039 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561b3a724a40 msgr2=0x561b3a732870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:17.602 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.601+0000 7fcaabd6c740 1 --2- 172.21.15.72:0/3577247039 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561b3a724a40 0x561b3a732870 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fca90026700 tx=0x7fca90011f60 comp rx=0 tx=0).stop 2024-04-26T21:58:17.602 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.601+0000 7fcaabd6c740 1 -- 172.21.15.72:0/3577247039 shutdown_connections 2024-04-26T21:58:17.602 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.601+0000 7fcaabd6c740 1 --2- 172.21.15.72:0/3577247039 >> [v2:172.21.15.164:6838/1636226820,v1:172.21.15.164:6839/1636226820] conn(0x561b3a7a8700 0x561b3a7aab20 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:17.602 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.601+0000 7fca83e00640 0 client.35378 ms_handle_reset on v2:172.21.15.164:6838/1636226820 2024-04-26T21:58:17.602 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.601+0000 7fcaabd6c740 1 --2- 172.21.15.72:0/3577247039 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561b3a729bb0 0x561b3a7a6750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:17.602 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.601+0000 7fcaabd6c740 1 --2- 172.21.15.72:0/3577247039 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561b3a724a40 0x561b3a732870 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:17.602 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.601+0000 7fcaabd6c740 1 --2- 172.21.15.72:0/3577247039 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561b3a724050 0x561b3a732330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:17.602 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.601+0000 7fcaabd6c740 1 -- 172.21.15.72:0/3577247039 >> 172.21.15.72:0/3577247039 conn(0x561b3a71f6c0 msgr2=0x561b3a722680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:58:17.602 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.601+0000 7fcaabd6c740 1 -- 172.21.15.72:0/3577247039 shutdown_connections 2024-04-26T21:58:17.602 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.601+0000 7fcaabd6c740 1 -- 172.21.15.72:0/3577247039 wait complete. 2024-04-26T21:58:17.602 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.601+0000 7fcaabd6c740 20 client.35378 trim_cache size 0 max 16384 2024-04-26T21:58:17.603 INFO:teuthology.orchestra.run.smithi072.stdout:[] 2024-04-26T21:58:17.603 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.602+0000 7fcaa3200640 1 -- 172.21.15.72:0/3090874690 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fca880a3c80 msgr2=0x7fca880a6140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:17.603 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.602+0000 7fcaa3200640 1 --2- 172.21.15.72:0/3090874690 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fca880a3c80 0x7fca880a6140 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x561b3a7254b0 tx=0x7fca940d3a20 comp rx=0 tx=0).stop 2024-04-26T21:58:17.604 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.602+0000 7fcaa3200640 1 -- 172.21.15.72:0/3090874690 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcaa4104a70 msgr2=0x7fcaa419ab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:17.604 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.602+0000 7fcaa3200640 1 --2- 172.21.15.72:0/3090874690 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcaa4104a70 0x7fcaa419ab30 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fca9400d950 tx=0x7fca9400de10 comp rx=0 tx=0).stop 2024-04-26T21:58:17.604 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.602+0000 7fcaa3200640 1 -- 172.21.15.72:0/3090874690 shutdown_connections 2024-04-26T21:58:17.604 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.602+0000 7fcaa3200640 1 --2- 172.21.15.72:0/3090874690 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fca880a3c80 0x7fca880a6140 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:17.604 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.602+0000 7fcaa3200640 1 --2- 172.21.15.72:0/3090874690 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcaa4106620 0x7fcaa41a20e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:17.604 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.602+0000 7fcaa3200640 1 --2- 172.21.15.72:0/3090874690 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcaa4105c60 0x7fcaa419b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:17.604 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.602+0000 7fcaa3200640 1 --2- 172.21.15.72:0/3090874690 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcaa4104a70 0x7fcaa419ab30 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:17.604 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.602+0000 7fcaa3200640 1 -- 172.21.15.72:0/3090874690 >> 172.21.15.72:0/3090874690 conn(0x7fcaa40fffe0 msgr2=0x7fcaa4108fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:58:17.604 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.603+0000 7fcaa3200640 1 -- 172.21.15.72:0/3090874690 shutdown_connections 2024-04-26T21:58:17.604 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.603+0000 7fcaa3200640 1 -- 172.21.15.72:0/3090874690 wait complete. 2024-04-26T21:58:17.623 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-26T21:58:17.847 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.846+0000 7fa22ca00640 1 -- 172.21.15.72:0/791178045 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa22810a810 msgr2=0x7fa22810ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:17.847 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.846+0000 7fa22ca00640 1 --2- 172.21.15.72:0/791178045 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa22810a810 0x7fa22810ccb0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fa220009ca0 tx=0x7fa22002a950 comp rx=0 tx=0).stop 2024-04-26T21:58:17.847 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.847+0000 7fa22ca00640 1 -- 172.21.15.72:0/791178045 shutdown_connections 2024-04-26T21:58:17.847 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.847+0000 7fa22ca00640 1 --2- 172.21.15.72:0/791178045 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa22810a810 0x7fa22810ccb0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:17.847 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.847+0000 7fa22ca00640 1 --2- 172.21.15.72:0/791178045 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa228075330 0x7fa228075790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:17.847 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.847+0000 7fa22ca00640 1 --2- 172.21.15.72:0/791178045 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa228076910 0x7fa228074d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:17.847 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.847+0000 7fa22ca00640 1 -- 172.21.15.72:0/791178045 >> 172.21.15.72:0/791178045 conn(0x7fa2280fdf10 msgr2=0x7fa228100330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:58:17.847 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.847+0000 7fa22ca00640 1 -- 172.21.15.72:0/791178045 shutdown_connections 2024-04-26T21:58:17.847 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.847+0000 7fa22ca00640 1 -- 172.21.15.72:0/791178045 wait complete. 2024-04-26T21:58:17.848 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.848+0000 7fa22ca00640 1 Processor -- start 2024-04-26T21:58:17.848 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.848+0000 7fa22ca00640 1 -- start start 2024-04-26T21:58:17.849 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.849+0000 7fa22ca00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa228075330 0x7fa22819f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:17.849 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.849+0000 7fa22ca00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa228076910 0x7fa22819f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:17.849 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.849+0000 7fa22ca00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa22810a810 0x7fa2281a6640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:17.849 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.849+0000 7fa22ca00640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fa228103440 con 0x7fa228075330 2024-04-26T21:58:17.850 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.849+0000 7fa22ca00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fa2281032c0 con 0x7fa228076910 2024-04-26T21:58:17.850 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.849+0000 7fa22ca00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa2281035c0 con 0x7fa22810a810 2024-04-26T21:58:17.850 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.849+0000 7fa225e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa228075330 0x7fa22819f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:17.850 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.849+0000 7fa225e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa228075330 0x7fa22819f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:45994/0 (socket says 172.21.15.72:45994) 2024-04-26T21:58:17.850 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.849+0000 7fa225e00640 1 -- 172.21.15.72:0/2446429437 learned_addr learned my addr 172.21.15.72:0/2446429437 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:58:17.850 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.849+0000 7fa226800640 1 --2- 172.21.15.72:0/2446429437 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa22810a810 0x7fa2281a6640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:17.850 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.849+0000 7fa225400640 1 --2- 172.21.15.72:0/2446429437 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa228076910 0x7fa22819f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:17.851 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.850+0000 7fa225e00640 1 -- 172.21.15.72:0/2446429437 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa22810a810 msgr2=0x7fa2281a6640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:17.851 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.850+0000 7fa225e00640 1 --2- 172.21.15.72:0/2446429437 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa22810a810 0x7fa2281a6640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:17.851 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.850+0000 7fa225e00640 1 -- 172.21.15.72:0/2446429437 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa228076910 msgr2=0x7fa22819f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:17.851 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.850+0000 7fa225e00640 1 --2- 172.21.15.72:0/2446429437 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa228076910 0x7fa22819f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:17.851 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.850+0000 7fa225e00640 1 -- 172.21.15.72:0/2446429437 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2281a6cb0 con 0x7fa228075330 2024-04-26T21:58:17.851 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.850+0000 7fa225e00640 1 --2- 172.21.15.72:0/2446429437 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa228075330 0x7fa22819f090 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fa21800d950 tx=0x7fa21800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:58:17.851 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.851+0000 7fa21ea00640 1 -- 172.21.15.72:0/2446429437 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa218017070 con 0x7fa228075330 2024-04-26T21:58:17.851 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.851+0000 7fa21ea00640 1 -- 172.21.15.72:0/2446429437 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa218009c30 con 0x7fa228075330 2024-04-26T21:58:17.851 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.851+0000 7fa21ea00640 1 -- 172.21.15.72:0/2446429437 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa21800f620 con 0x7fa228075330 2024-04-26T21:58:17.851 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.851+0000 7fa22ca00640 1 -- 172.21.15.72:0/2446429437 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2281a6fa0 con 0x7fa228075330 2024-04-26T21:58:17.851 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.851+0000 7fa22ca00640 1 -- 172.21.15.72:0/2446429437 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2281a7410 con 0x7fa228075330 2024-04-26T21:58:17.856 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.855+0000 7fa22f7e4740 1 Processor -- start 2024-04-26T21:58:17.857 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.855+0000 7fa22f7e4740 1 -- start start 2024-04-26T21:58:17.857 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.855+0000 7fa22f7e4740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d87339beb0 0x55d87339c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:17.857 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.855+0000 7fa22f7e4740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d87339c8a0 0x55d8733a13a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:17.857 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.855+0000 7fa22f7e4740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d8733a1a10 0x55d8733a3ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:17.857 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.855+0000 7fa22f7e4740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55d8732eec70 con 0x55d87339c8a0 2024-04-26T21:58:17.857 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.855+0000 7fa22f7e4740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55d8732a7980 con 0x55d87339beb0 2024-04-26T21:58:17.857 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.855+0000 7fa22f7e4740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55d8732eac30 con 0x55d8733a1a10 2024-04-26T21:58:17.857 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.856+0000 7fa226800640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d87339beb0 0x55d87339c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:17.857 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.856+0000 7fa226800640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d87339beb0 0x55d87339c2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.72:48682/0 (socket says 172.21.15.72:48682) 2024-04-26T21:58:17.857 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.856+0000 7fa226800640 1 -- 172.21.15.72:0/2065843341 learned_addr learned my addr 172.21.15.72:0/2065843341 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:58:17.857 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.856+0000 7fa225400640 1 --2- 172.21.15.72:0/2065843341 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d8733a1a10 0x55d8733a3ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:17.857 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.857+0000 7fa21ea00640 1 -- 172.21.15.72:0/2446429437 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fa218010460 con 0x7fa228075330 2024-04-26T21:58:17.857 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.857+0000 7fa226800640 1 -- 172.21.15.72:0/2065843341 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d8733a1a10 msgr2=0x55d8733a3ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:58:17.858 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.857+0000 7fa226800640 1 --2- 172.21.15.72:0/2065843341 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d8733a1a10 0x55d8733a3ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:17.858 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.857+0000 7fa226800640 1 -- 172.21.15.72:0/2065843341 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d87339c8a0 msgr2=0x55d8733a13a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:58:17.858 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.857+0000 7fa226800640 1 --2- 172.21.15.72:0/2065843341 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d87339c8a0 0x55d8733a13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:17.858 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.857+0000 7fa226800640 1 -- 172.21.15.72:0/2065843341 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d87327f7b0 con 0x55d87339beb0 2024-04-26T21:58:17.859 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.857+0000 7fa226800640 1 --2- 172.21.15.72:0/2065843341 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d87339beb0 0x55d87339c2d0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fa220035000 tx=0x7fa22002f450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e39581884a83c4e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:58:17.859 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.857+0000 7fa213000640 1 -- 172.21.15.72:0/2065843341 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa22006a070 con 0x55d87339beb0 2024-04-26T21:58:17.859 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.858+0000 7fa213000640 1 -- 172.21.15.72:0/2065843341 <== mon.2 v2:172.21.15.76:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa220003e00 con 0x55d87339beb0 2024-04-26T21:58:17.859 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.858+0000 7fa213000640 1 -- 172.21.15.72:0/2065843341 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa220034040 con 0x55d87339beb0 2024-04-26T21:58:17.859 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.858+0000 7fa22f7e4740 1 -- 172.21.15.72:0/2065843341 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d87339beb0 msgr2=0x55d87339c2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:17.859 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.858+0000 7fa22f7e4740 1 --2- 172.21.15.72:0/2065843341 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d87339beb0 0x55d87339c2d0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fa220035000 tx=0x7fa22002f450 comp rx=0 tx=0).stop 2024-04-26T21:58:17.859 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.859+0000 7fa22f7e4740 1 -- 172.21.15.72:0/2065843341 shutdown_connections 2024-04-26T21:58:17.859 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.859+0000 7fa22f7e4740 1 --2- 172.21.15.72:0/2065843341 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d8733a1a10 0x55d8733a3ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:17.859 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.859+0000 7fa22f7e4740 1 --2- 172.21.15.72:0/2065843341 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d87339c8a0 0x55d8733a13a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:17.859 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.859+0000 7fa22f7e4740 1 --2- 172.21.15.72:0/2065843341 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d87339beb0 0x55d87339c2d0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:17.859 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.859+0000 7fa22f7e4740 1 -- 172.21.15.72:0/2065843341 >> 172.21.15.72:0/2065843341 conn(0x55d873386510 msgr2=0x55d873386910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:58:17.860 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.859+0000 7fa22f7e4740 1 -- 172.21.15.72:0/2065843341 shutdown_connections 2024-04-26T21:58:17.860 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.859+0000 7fa22f7e4740 1 -- 172.21.15.72:0/2065843341 wait complete. 2024-04-26T21:58:17.861 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.859+0000 7fa21ea00640 1 --2- 172.21.15.72:0/2446429437 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fa20c0a3d20 0x7fa20c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:17.861 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.859+0000 7fa22f7e4740 1 -- start start 2024-04-26T21:58:17.861 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.859+0000 7fa21ea00640 1 -- 172.21.15.72:0/2446429437 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7fa2180d00d0 con 0x7fa228075330 2024-04-26T21:58:17.861 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.860+0000 7fa22f7e4740 1 Processor -- start 2024-04-26T21:58:17.861 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.860+0000 7fa225400640 1 --2- 172.21.15.72:0/2446429437 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fa20c0a3d20 0x7fa20c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:17.861 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.860+0000 7fa22f7e4740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d87339beb0 0x55d87338c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:17.861 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.860+0000 7fa22f7e4740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d87339c8a0 0x55d87338c780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:17.861 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.860+0000 7fa22f7e4740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d8733a1a10 0x55d8733937e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:17.861 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.860+0000 7fa22f7e4740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55d8732edf80 con 0x55d87339c8a0 2024-04-26T21:58:17.861 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.860+0000 7fa22f7e4740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55d87329b260 con 0x55d87339beb0 2024-04-26T21:58:17.861 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.860+0000 7fa22f7e4740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55d8732f32a0 con 0x55d8733a1a10 2024-04-26T21:58:17.861 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.860+0000 7fa22f7e4740 10 client.0 _ll_register_callbacks cb 0x55d8732e5960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:58:17.862 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.861+0000 7fa225e00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d87339beb0 0x55d87338c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:17.862 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.861+0000 7fa225400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d87339c8a0 0x55d87338c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:17.862 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.861+0000 7fa225e00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d87339beb0 0x55d87338c240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.72:48698/0 (socket says 172.21.15.72:48698) 2024-04-26T21:58:17.862 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.861+0000 7fa225e00640 1 -- 172.21.15.72:0/3800097418 learned_addr learned my addr 172.21.15.72:0/3800097418 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:58:17.862 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.861+0000 7fa226800640 1 --2- 172.21.15.72:0/3800097418 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d8733a1a10 0x55d8733937e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:17.862 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.861+0000 7fa225e00640 1 -- 172.21.15.72:0/3800097418 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d8733a1a10 msgr2=0x55d8733937e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:58:17.862 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.861+0000 7fa225e00640 1 --2- 172.21.15.72:0/3800097418 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d8733a1a10 0x55d8733937e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:17.862 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.861+0000 7fa225e00640 1 -- 172.21.15.72:0/3800097418 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d87339c8a0 msgr2=0x55d87338c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:58:17.862 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.861+0000 7fa225e00640 1 --2- 172.21.15.72:0/3800097418 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d87339c8a0 0x55d87338c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:17.862 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.861+0000 7fa225e00640 1 -- 172.21.15.72:0/3800097418 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d87327d1c0 con 0x55d87339beb0 2024-04-26T21:58:17.863 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.861+0000 7fa225e00640 1 --2- 172.21.15.72:0/3800097418 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d87339beb0 0x55d87338c240 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa218098680 tx=0x7fa21809bc70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e7f0a729311e0ce6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:58:17.864 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.861+0000 7fa207400640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-04-26T21:58:17.864 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.862+0000 7fa207400640 1 -- 172.21.15.72:0/3800097418 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa218020040 con 0x55d87339beb0 2024-04-26T21:58:17.864 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.862+0000 7fa207400640 1 -- 172.21.15.72:0/3800097418 <== mon.2 v2:172.21.15.76:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7fa2180a0120 con 0x55d87339beb0 2024-04-26T21:58:17.864 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.862+0000 7fa22f7e4740 1 -- 172.21.15.72:0/3800097418 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa220003e00 con 0x55d87339beb0 2024-04-26T21:58:17.864 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.862+0000 7fa225400640 1 --2- 172.21.15.72:0/2446429437 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fa20c0a3d20 0x7fa20c0a61e0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x55d8733873e0 tx=0x7fa214012930 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:58:17.864 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.862+0000 7fa207400640 1 -- 172.21.15.72:0/3800097418 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa2180b5030 con 0x55d87339beb0 2024-04-26T21:58:17.864 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.862+0000 7fa22f7e4740 10 client.35390 fetch_fsmap learned FSMap version 27 2024-04-26T21:58:17.864 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.862+0000 7fa22f7e4740 1 -- 172.21.15.72:0/3800097418 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d8730003f0 con 0x55d87339beb0 2024-04-26T21:58:17.864 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.863+0000 7fa207400640 1 -- 172.21.15.72:0/3800097418 <== mon.2 v2:172.21.15.76:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa220003e00 con 0x55d87339beb0 2024-04-26T21:58:17.864 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.863+0000 7fa207400640 1 -- 172.21.15.72:0/3800097418 <== mon.2 v2:172.21.15.76:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fa2180d20e0 con 0x55d87339beb0 2024-04-26T21:58:17.864 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.863+0000 7fa22f7e4740 10 client.35390 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T21:58:17.864 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.863+0000 7fa22f7e4740 10 client.35390 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-26T21:58:17.864 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.863+0000 7fa22f7e4740 20 client.35390 populate_metadata read hostname 'smithi072' 2024-04-26T21:58:17.864 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.863+0000 7fa22f7e4740 1 --2- 172.21.15.72:0/3800097418 >> [v2:172.21.15.76:6832/3869483750,v1:172.21.15.76:6833/3869483750] conn(0x55d873397cc0 0x55d87339a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:17.865 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.863+0000 7fa22f7e4740 4 client.35390 mds_command: new command op to 34406 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:58:17.865 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.863+0000 7fa22f7e4740 1 -- 172.21.15.72:0/3800097418 --> [v2:172.21.15.76:6832/3869483750,v1:172.21.15.76:6833/3869483750] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d8732fc880 con 0x55d873397cc0 2024-04-26T21:58:17.865 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.864+0000 7fa225400640 1 --2- 172.21.15.72:0/3800097418 >> [v2:172.21.15.76:6832/3869483750,v1:172.21.15.76:6833/3869483750] conn(0x55d873397cc0 0x55d87339a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:17.865 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.865+0000 7fa225400640 1 --2- 172.21.15.72:0/3800097418 >> [v2:172.21.15.76:6832/3869483750,v1:172.21.15.76:6833/3869483750] conn(0x55d873397cc0 0x55d87339a0b0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=a310a882a90e31ad server_cookie=a59167e4c434ac92 in_seq=0 out_seq=0 2024-04-26T21:58:17.865 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.865+0000 7fa207400640 10 client.35390 ms_handle_connect on v2:172.21.15.76:6832/3869483750 2024-04-26T21:58:17.866 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.866+0000 7fa207400640 1 -- 172.21.15.72:0/3800097418 <== mds.3 v2:172.21.15.76:6832/3869483750 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d8732fc880 con 0x55d873397cc0 2024-04-26T21:58:17.866 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.866+0000 7fa207400640 10 client.35390 handle_command_reply: tid=0 2024-04-26T21:58:17.866 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.866+0000 7fa22f7e4740 1 client.35390 shutdown 2024-04-26T21:58:17.867 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.867+0000 7fa22f7e4740 1 -- 172.21.15.72:0/3800097418 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d87339beb0 msgr2=0x55d87338c240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:17.867 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.867+0000 7fa22f7e4740 1 --2- 172.21.15.72:0/3800097418 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d87339beb0 0x55d87338c240 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa218098680 tx=0x7fa21809bc70 comp rx=0 tx=0).stop 2024-04-26T21:58:17.867 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.867+0000 7fa22f7e4740 1 -- 172.21.15.72:0/3800097418 shutdown_connections 2024-04-26T21:58:17.867 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.867+0000 7fa22f7e4740 1 --2- 172.21.15.72:0/3800097418 >> [v2:172.21.15.76:6832/3869483750,v1:172.21.15.76:6833/3869483750] conn(0x55d873397cc0 0x55d87339a0b0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:17.867 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.867+0000 7fa22f7e4740 1 --2- 172.21.15.72:0/3800097418 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d8733a1a10 0x55d8733937e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:17.867 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.867+0000 7fa22f7e4740 1 --2- 172.21.15.72:0/3800097418 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d87339c8a0 0x55d87338c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:17.867 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.867+0000 7fa22f7e4740 1 --2- 172.21.15.72:0/3800097418 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d87339beb0 0x55d87338c240 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:17.867 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.867+0000 7fa22f7e4740 1 -- 172.21.15.72:0/3800097418 >> 172.21.15.72:0/3800097418 conn(0x55d873386510 msgr2=0x55d873385690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:58:17.867 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.867+0000 7fa207400640 0 client.35390 ms_handle_reset on v2:172.21.15.76:6832/3869483750 2024-04-26T21:58:17.868 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.867+0000 7fa22f7e4740 1 -- 172.21.15.72:0/3800097418 shutdown_connections 2024-04-26T21:58:17.868 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.867+0000 7fa22f7e4740 1 -- 172.21.15.72:0/3800097418 wait complete. 2024-04-26T21:58:17.868 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.867+0000 7fa22f7e4740 20 client.35390 trim_cache size 0 max 16384 2024-04-26T21:58:17.887 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.887+0000 7fa22f7e4740 1 Processor -- start 2024-04-26T21:58:17.887 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.887+0000 7fa22f7e4740 1 -- start start 2024-04-26T21:58:17.888 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.888+0000 7fa22f7e4740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d873386510 0x55d87339d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:17.888 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.888+0000 7fa22f7e4740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d87339beb0 0x55d87339d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:17.888 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.888+0000 7fa22f7e4740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d87339c8a0 0x55d87339de20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:17.888 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.888+0000 7fa22f7e4740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fa2281a7b60 con 0x55d87339c8a0 2024-04-26T21:58:17.888 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.888+0000 7fa22f7e4740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fa2281a96a0 con 0x55d873386510 2024-04-26T21:58:17.888 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.888+0000 7fa22f7e4740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa2281a8230 con 0x55d87339beb0 2024-04-26T21:58:17.889 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.888+0000 7fa226800640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d87339c8a0 0x55d87339de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:17.889 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.888+0000 7fa226800640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d87339c8a0 0x55d87339de20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:46026/0 (socket says 172.21.15.72:46026) 2024-04-26T21:58:17.889 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.888+0000 7fa226800640 1 -- 172.21.15.72:0/3411804597 learned_addr learned my addr 172.21.15.72:0/3411804597 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:58:17.889 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.889+0000 7fa225400640 1 --2- 172.21.15.72:0/3411804597 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d87339beb0 0x55d87339d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:17.889 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.889+0000 7fa225e00640 1 --2- 172.21.15.72:0/3411804597 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d873386510 0x55d87339d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:17.889 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.889+0000 7fa226800640 1 -- 172.21.15.72:0/3411804597 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d87339beb0 msgr2=0x55d87339d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:58:17.889 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.889+0000 7fa226800640 1 --2- 172.21.15.72:0/3411804597 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d87339beb0 0x55d87339d8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:17.889 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.889+0000 7fa226800640 1 -- 172.21.15.72:0/3411804597 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d873386510 msgr2=0x55d87339d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:58:17.890 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.889+0000 7fa226800640 1 --2- 172.21.15.72:0/3411804597 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d873386510 0x55d87339d3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:17.890 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.889+0000 7fa226800640 1 -- 172.21.15.72:0/3411804597 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d87339a420 con 0x55d87339c8a0 2024-04-26T21:58:17.890 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.889+0000 7fa226800640 1 --2- 172.21.15.72:0/3411804597 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d87339c8a0 0x55d87339de20 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fa22002f4c0 tx=0x7fa220005fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=524599e0e5ddf907 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:58:17.890 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.890+0000 7fa212600640 1 -- 172.21.15.72:0/3411804597 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa22006a030 con 0x55d87339c8a0 2024-04-26T21:58:17.890 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.890+0000 7fa212600640 1 -- 172.21.15.72:0/3411804597 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa22006c070 con 0x55d87339c8a0 2024-04-26T21:58:17.891 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.890+0000 7fa212600640 1 -- 172.21.15.72:0/3411804597 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa220034050 con 0x55d87339c8a0 2024-04-26T21:58:17.891 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.891+0000 7fa22f7e4740 1 -- 172.21.15.72:0/3411804597 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d87339c8a0 msgr2=0x55d87339de20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:17.891 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.891+0000 7fa22f7e4740 1 --2- 172.21.15.72:0/3411804597 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d87339c8a0 0x55d87339de20 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fa22002f4c0 tx=0x7fa220005fc0 comp rx=0 tx=0).stop 2024-04-26T21:58:17.892 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.892+0000 7fa22f7e4740 1 -- 172.21.15.72:0/3411804597 shutdown_connections 2024-04-26T21:58:17.892 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.892+0000 7fa22f7e4740 1 --2- 172.21.15.72:0/3411804597 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d87339c8a0 0x55d87339de20 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:17.892 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.892+0000 7fa22f7e4740 1 --2- 172.21.15.72:0/3411804597 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d87339beb0 0x55d87339d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:17.892 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.892+0000 7fa22f7e4740 1 --2- 172.21.15.72:0/3411804597 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d873386510 0x55d87339d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:17.892 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.892+0000 7fa22f7e4740 1 -- 172.21.15.72:0/3411804597 >> 172.21.15.72:0/3411804597 conn(0x55d873386130 msgr2=0x55d873385690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:58:17.892 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.892+0000 7fa22f7e4740 1 -- 172.21.15.72:0/3411804597 shutdown_connections 2024-04-26T21:58:17.892 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.892+0000 7fa22f7e4740 1 -- 172.21.15.72:0/3411804597 wait complete. 2024-04-26T21:58:17.894 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.892+0000 7fa22f7e4740 1 -- start start 2024-04-26T21:58:17.894 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.892+0000 7fa22f7e4740 1 Processor -- start 2024-04-26T21:58:17.894 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.893+0000 7fa22f7e4740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d873386510 0x55d8733a9c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:17.895 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.893+0000 7fa22f7e4740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d87339beb0 0x55d8733a82a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:17.895 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.893+0000 7fa22f7e4740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d87339c8a0 0x55d8733a8860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:17.895 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.893+0000 7fa22f7e4740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fa2281a8fd0 con 0x55d87339beb0 2024-04-26T21:58:17.895 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.893+0000 7fa22f7e4740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fa2281a8900 con 0x55d87339c8a0 2024-04-26T21:58:17.895 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.893+0000 7fa22f7e4740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa228116120 con 0x55d873386510 2024-04-26T21:58:17.895 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.893+0000 7fa22f7e4740 10 client.0 _ll_register_callbacks cb 0x7fa21800df40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:58:17.895 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.893+0000 7fa225400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d87339beb0 0x55d8733a82a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:17.895 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.893+0000 7fa225400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d87339beb0 0x55d8733a82a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:46038/0 (socket says 172.21.15.72:46038) 2024-04-26T21:58:17.895 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.893+0000 7fa225400640 1 -- 172.21.15.72:0/2586012665 learned_addr learned my addr 172.21.15.72:0/2586012665 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:58:17.895 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.893+0000 7fa226800640 1 --2- 172.21.15.72:0/2586012665 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d87339c8a0 0x55d8733a8860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:17.895 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.893+0000 7fa225e00640 1 --2- 172.21.15.72:0/2586012665 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d873386510 0x55d8733a9c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:17.895 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.894+0000 7fa225400640 1 -- 172.21.15.72:0/2586012665 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d873386510 msgr2=0x55d8733a9c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:58:17.895 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.894+0000 7fa225400640 1 --2- 172.21.15.72:0/2586012665 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d873386510 0x55d8733a9c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:17.896 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.894+0000 7fa225400640 1 -- 172.21.15.72:0/2586012665 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d87339c8a0 msgr2=0x55d8733a8860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:58:17.896 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.894+0000 7fa225400640 1 --2- 172.21.15.72:0/2586012665 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d87339c8a0 0x55d8733a8860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:17.896 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.894+0000 7fa225400640 1 -- 172.21.15.72:0/2586012665 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d8730ca7d0 con 0x55d87339beb0 2024-04-26T21:58:17.896 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.894+0000 7fa226800640 1 --2- 172.21.15.72:0/2586012665 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d87339c8a0 0x55d8733a8860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:58:17.896 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.894+0000 7fa225400640 1 --2- 172.21.15.72:0/2586012665 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d87339beb0 0x55d8733a82a0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fa214012fd0 tx=0x7fa21401e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2161926f94240845 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:58:17.896 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.894+0000 7fa207400640 10 client.0 ms_handle_connect on v2:172.21.15.72:3300/0 2024-04-26T21:58:17.896 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.894+0000 7fa207400640 1 -- 172.21.15.72:0/2586012665 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa21402f070 con 0x55d87339beb0 2024-04-26T21:58:17.896 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.894+0000 7fa22f7e4740 1 -- 172.21.15.72:0/2586012665 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa22006c070 con 0x55d87339beb0 2024-04-26T21:58:17.896 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.894+0000 7fa207400640 1 -- 172.21.15.72:0/2586012665 <== mon.0 v2:172.21.15.72:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7fa21402bd50 con 0x55d87339beb0 2024-04-26T21:58:17.896 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.895+0000 7fa207400640 1 -- 172.21.15.72:0/2586012665 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa21403a050 con 0x55d87339beb0 2024-04-26T21:58:17.896 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.895+0000 7fa22f7e4740 10 client.16176 fetch_fsmap learned FSMap version 27 2024-04-26T21:58:17.896 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.895+0000 7fa22f7e4740 1 -- 172.21.15.72:0/2586012665 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d8732952a0 con 0x55d87339beb0 2024-04-26T21:58:17.898 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.895+0000 7fa207400640 1 -- 172.21.15.72:0/2586012665 <== mon.0 v2:172.21.15.72:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa22006c070 con 0x55d87339beb0 2024-04-26T21:58:17.898 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.895+0000 7fa207400640 1 -- 172.21.15.72:0/2586012665 <== mon.0 v2:172.21.15.72:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fa2140050b0 con 0x55d87339beb0 2024-04-26T21:58:17.898 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.895+0000 7fa22f7e4740 10 client.16176 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T21:58:17.898 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.895+0000 7fa22f7e4740 10 client.16176 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-26T21:58:17.898 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.895+0000 7fa22f7e4740 20 client.16176 populate_metadata read hostname 'smithi072' 2024-04-26T21:58:17.898 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.895+0000 7fa22f7e4740 1 --2- 172.21.15.72:0/2586012665 >> [v2:172.21.15.76:6832/3869483750,v1:172.21.15.76:6833/3869483750] conn(0x55d8733a1a10 0x55d873409820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:17.898 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.895+0000 7fa22f7e4740 4 client.16176 mds_command: new command op to 34406 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-26T21:58:17.898 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.895+0000 7fa22f7e4740 1 -- 172.21.15.72:0/2586012665 --> [v2:172.21.15.76:6832/3869483750,v1:172.21.15.76:6833/3869483750] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d8732f66f0 con 0x55d8733a1a10 2024-04-26T21:58:17.898 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.896+0000 7fa225e00640 1 --2- 172.21.15.72:0/2586012665 >> [v2:172.21.15.76:6832/3869483750,v1:172.21.15.76:6833/3869483750] conn(0x55d8733a1a10 0x55d873409820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:17.899 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.896+0000 7fa207400640 10 client.16176 ms_handle_connect on v2:172.21.15.76:6832/3869483750 2024-04-26T21:58:17.899 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.896+0000 7fa225e00640 1 --2- 172.21.15.72:0/2586012665 >> [v2:172.21.15.76:6832/3869483750,v1:172.21.15.76:6833/3869483750] conn(0x55d8733a1a10 0x55d873409820 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=96404599d234522e server_cookie=3e5f6d78dc9b5341 in_seq=0 out_seq=0 2024-04-26T21:58:17.899 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.897+0000 7fa207400640 1 -- 172.21.15.72:0/2586012665 <== mds.3 v2:172.21.15.76:6832/3869483750 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d8732f66f0 con 0x55d8733a1a10 2024-04-26T21:58:17.899 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.897+0000 7fa207400640 10 client.16176 handle_command_reply: tid=0 2024-04-26T21:58:17.899 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.897+0000 7fa22f7e4740 1 client.16176 shutdown 2024-04-26T21:58:17.899 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.897+0000 7fa22f7e4740 1 -- 172.21.15.72:0/2586012665 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d87339beb0 msgr2=0x55d8733a82a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:17.899 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.897+0000 7fa22f7e4740 1 --2- 172.21.15.72:0/2586012665 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d87339beb0 0x55d8733a82a0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fa214012fd0 tx=0x7fa21401e000 comp rx=0 tx=0).stop 2024-04-26T21:58:17.899 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.897+0000 7fa22f7e4740 1 -- 172.21.15.72:0/2586012665 shutdown_connections 2024-04-26T21:58:17.899 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.897+0000 7fa22f7e4740 1 --2- 172.21.15.72:0/2586012665 >> [v2:172.21.15.76:6832/3869483750,v1:172.21.15.76:6833/3869483750] conn(0x55d8733a1a10 0x55d873409820 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:17.899 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.897+0000 7fa22f7e4740 1 --2- 172.21.15.72:0/2586012665 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d87339c8a0 0x55d8733a8860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:17.899 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.897+0000 7fa22f7e4740 1 --2- 172.21.15.72:0/2586012665 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d87339beb0 0x55d8733a82a0 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:17.899 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.897+0000 7fa22f7e4740 1 --2- 172.21.15.72:0/2586012665 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d873386510 0x55d8733a9c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:17.899 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.897+0000 7fa207400640 0 client.16176 ms_handle_reset on v2:172.21.15.76:6832/3869483750 2024-04-26T21:58:17.899 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.897+0000 7fa22f7e4740 1 -- 172.21.15.72:0/2586012665 >> 172.21.15.72:0/2586012665 conn(0x55d873386130 msgr2=0x55d873384e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:58:17.900 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.897+0000 7fa22f7e4740 1 -- 172.21.15.72:0/2586012665 shutdown_connections 2024-04-26T21:58:17.900 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.898+0000 7fa22f7e4740 1 -- 172.21.15.72:0/2586012665 wait complete. 2024-04-26T21:58:17.900 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.898+0000 7fa22f7e4740 20 client.16176 trim_cache size 0 max 16384 2024-04-26T21:58:17.900 INFO:teuthology.orchestra.run.smithi072.stdout:[] 2024-04-26T21:58:17.901 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.898+0000 7fa22ca00640 1 -- 172.21.15.72:0/2446429437 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fa20c0a3d20 msgr2=0x7fa20c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:17.901 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.898+0000 7fa22ca00640 1 --2- 172.21.15.72:0/2446429437 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fa20c0a3d20 0x7fa20c0a61e0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x55d8733873e0 tx=0x7fa214012930 comp rx=0 tx=0).stop 2024-04-26T21:58:17.901 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.898+0000 7fa22ca00640 1 -- 172.21.15.72:0/2446429437 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa228075330 msgr2=0x7fa22819f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:17.901 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.898+0000 7fa22ca00640 1 --2- 172.21.15.72:0/2446429437 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa228075330 0x7fa22819f090 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fa21800d950 tx=0x7fa21800de10 comp rx=0 tx=0).stop 2024-04-26T21:58:17.901 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.899+0000 7fa22ca00640 1 -- 172.21.15.72:0/2446429437 shutdown_connections 2024-04-26T21:58:17.901 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.899+0000 7fa22ca00640 1 --2- 172.21.15.72:0/2446429437 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fa20c0a3d20 0x7fa20c0a61e0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:17.901 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.899+0000 7fa22ca00640 1 --2- 172.21.15.72:0/2446429437 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa22810a810 0x7fa2281a6640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:17.901 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.899+0000 7fa22ca00640 1 --2- 172.21.15.72:0/2446429437 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa228076910 0x7fa22819f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:17.901 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.899+0000 7fa22ca00640 1 --2- 172.21.15.72:0/2446429437 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa228075330 0x7fa22819f090 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:17.902 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.899+0000 7fa22ca00640 1 -- 172.21.15.72:0/2446429437 >> 172.21.15.72:0/2446429437 conn(0x7fa2280fdf10 msgr2=0x7fa228103e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:58:17.902 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.899+0000 7fa22ca00640 1 -- 172.21.15.72:0/2446429437 shutdown_connections 2024-04-26T21:58:17.902 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:17.899+0000 7fa22ca00640 1 -- 172.21.15.72:0/2446429437 wait complete. 2024-04-26T21:58:17.921 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-26T21:58:18.158 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.157+0000 7f5f00e00640 1 -- 172.21.15.72:0/554104004 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5efc104b90 msgr2=0x7f5efc104fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:18.158 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.157+0000 7f5f00e00640 1 --2- 172.21.15.72:0/554104004 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5efc104b90 0x7f5efc104fb0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f5ef4009be0 tx=0x7f5ef402a6d0 comp rx=0 tx=0).stop 2024-04-26T21:58:18.158 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.157+0000 7f5f00e00640 1 -- 172.21.15.72:0/554104004 shutdown_connections 2024-04-26T21:58:18.158 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.157+0000 7f5f00e00640 1 --2- 172.21.15.72:0/554104004 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5efc106740 0x7f5efc10cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:18.158 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.157+0000 7f5f00e00640 1 --2- 172.21.15.72:0/554104004 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5efc105d80 0x7f5efc106200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:18.158 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.158+0000 7f5f00e00640 1 --2- 172.21.15.72:0/554104004 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5efc104b90 0x7f5efc104fb0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:18.158 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.158+0000 7f5f00e00640 1 -- 172.21.15.72:0/554104004 >> 172.21.15.72:0/554104004 conn(0x7f5efc1000e0 msgr2=0x7f5efc102540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:58:18.159 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.158+0000 7f5f00e00640 1 -- 172.21.15.72:0/554104004 shutdown_connections 2024-04-26T21:58:18.159 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.158+0000 7f5f00e00640 1 -- 172.21.15.72:0/554104004 wait complete. 2024-04-26T21:58:18.159 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.159+0000 7f5f00e00640 1 Processor -- start 2024-04-26T21:58:18.159 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.159+0000 7f5f00e00640 1 -- start start 2024-04-26T21:58:18.159 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.159+0000 7f5f00e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5efc104b90 0x7f5efc19f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:18.160 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.159+0000 7f5f00e00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5efc105d80 0x7f5efc19f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:18.160 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.159+0000 7f5f00e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5efc106740 0x7f5efc1a66a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:18.160 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.159+0000 7f5f00e00640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f5efc110540 con 0x7f5efc106740 2024-04-26T21:58:18.160 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.159+0000 7f5f00e00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f5efc1103c0 con 0x7f5efc105d80 2024-04-26T21:58:18.160 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.159+0000 7f5f00e00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f5efc1106c0 con 0x7f5efc104b90 2024-04-26T21:58:18.161 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.160+0000 7f5ef9e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5efc104b90 0x7f5efc19f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:18.161 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.160+0000 7f5ef9e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5efc104b90 0x7f5efc19f0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.72:55890/0 (socket says 172.21.15.72:55890) 2024-04-26T21:58:18.161 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.160+0000 7f5ef9e00640 1 -- 172.21.15.72:0/392432732 learned_addr learned my addr 172.21.15.72:0/392432732 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:58:18.161 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.160+0000 7f5ef9400640 1 --2- 172.21.15.72:0/392432732 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5efc105d80 0x7f5efc19f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:18.161 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.160+0000 7f5efa800640 1 --2- 172.21.15.72:0/392432732 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5efc106740 0x7f5efc1a66a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:18.161 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.161+0000 7f5ef9e00640 1 -- 172.21.15.72:0/392432732 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5efc105d80 msgr2=0x7f5efc19f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:18.161 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.161+0000 7f5ef9e00640 1 --2- 172.21.15.72:0/392432732 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5efc105d80 0x7f5efc19f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:18.161 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.161+0000 7f5ef9e00640 1 -- 172.21.15.72:0/392432732 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5efc106740 msgr2=0x7f5efc1a66a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:18.161 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.161+0000 7f5ef9e00640 1 --2- 172.21.15.72:0/392432732 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5efc106740 0x7f5efc1a66a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:18.162 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.161+0000 7f5ef9e00640 1 -- 172.21.15.72:0/392432732 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5efc1a6d10 con 0x7f5efc104b90 2024-04-26T21:58:18.162 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.161+0000 7f5ef9e00640 1 --2- 172.21.15.72:0/392432732 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5efc104b90 0x7f5efc19f0f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f5ef402acc0 tx=0x7f5ef4030be0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:58:18.163 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.162+0000 7f5ef2a00640 1 -- 172.21.15.72:0/392432732 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ef4041020 con 0x7f5efc104b90 2024-04-26T21:58:18.163 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.162+0000 7f5f00e00640 1 -- 172.21.15.72:0/392432732 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5efc1a6fa0 con 0x7f5efc104b90 2024-04-26T21:58:18.163 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.162+0000 7f5ef2a00640 1 -- 172.21.15.72:0/392432732 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5ef4037040 con 0x7f5efc104b90 2024-04-26T21:58:18.163 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.162+0000 7f5f00e00640 1 -- 172.21.15.72:0/392432732 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5efc1a73b0 con 0x7f5efc104b90 2024-04-26T21:58:18.163 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.162+0000 7f5ef2a00640 1 -- 172.21.15.72:0/392432732 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ef4036070 con 0x7f5efc104b90 2024-04-26T21:58:18.166 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.166+0000 7f5f03d7f740 1 Processor -- start 2024-04-26T21:58:18.166 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.166+0000 7f5f03d7f740 1 -- start start 2024-04-26T21:58:18.167 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.167+0000 7f5f03d7f740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5563ba7e2ea0 0x5563ba7e32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:18.167 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.167+0000 7f5f03d7f740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5563ba7e3890 0x5563ba7e8390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:18.167 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.167+0000 7f5f03d7f740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5563ba7e8a00 0x5563ba7eae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:18.168 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.167+0000 7f5f03d7f740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x5563ba746c70 con 0x5563ba7e2ea0 2024-04-26T21:58:18.168 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.167+0000 7f5f03d7f740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5563ba6ff980 con 0x5563ba7e8a00 2024-04-26T21:58:18.168 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.167+0000 7f5f03d7f740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5563ba742c30 con 0x5563ba7e3890 2024-04-26T21:58:18.168 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.167+0000 7f5ef2a00640 1 -- 172.21.15.72:0/392432732 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f5ef4044020 con 0x7f5efc104b90 2024-04-26T21:58:18.168 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.167+0000 7f5efa800640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5563ba7e2ea0 0x5563ba7e32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:18.168 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.168+0000 7f5ef9400640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5563ba7e8a00 0x5563ba7eae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:18.168 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.168+0000 7f5efa800640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5563ba7e2ea0 0x5563ba7e32c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:46066/0 (socket says 172.21.15.72:46066) 2024-04-26T21:58:18.168 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.168+0000 7f5ef9400640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5563ba7e8a00 0x5563ba7eae90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.72:48750/0 (socket says 172.21.15.72:48750) 2024-04-26T21:58:18.168 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.168+0000 7f5efa800640 1 -- 172.21.15.72:0/2278818292 learned_addr learned my addr 172.21.15.72:0/2278818292 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:58:18.169 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.168+0000 7f5ef9e00640 1 --2- 172.21.15.72:0/2278818292 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5563ba7e3890 0x5563ba7e8390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:18.169 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.168+0000 7f5efa800640 1 -- 172.21.15.72:0/2278818292 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5563ba7e3890 msgr2=0x5563ba7e8390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:58:18.169 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.168+0000 7f5efa800640 1 --2- 172.21.15.72:0/2278818292 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5563ba7e3890 0x5563ba7e8390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:18.169 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.168+0000 7f5efa800640 1 -- 172.21.15.72:0/2278818292 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5563ba7e8a00 msgr2=0x5563ba7eae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:58:18.170 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.168+0000 7f5efa800640 1 --2- 172.21.15.72:0/2278818292 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5563ba7e8a00 0x5563ba7eae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:18.170 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.168+0000 7f5efa800640 1 -- 172.21.15.72:0/2278818292 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5563ba6d77b0 con 0x5563ba7e2ea0 2024-04-26T21:58:18.170 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.168+0000 7f5ef9e00640 1 --2- 172.21.15.72:0/2278818292 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5563ba7e3890 0x5563ba7e8390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:58:18.170 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.168+0000 7f5ef9400640 1 --2- 172.21.15.72:0/2278818292 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5563ba7e8a00 0x5563ba7eae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:58:18.170 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.169+0000 7f5efa800640 1 --2- 172.21.15.72:0/2278818292 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5563ba7e2ea0 0x5563ba7e32c0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f5ee8009ca0 tx=0x7f5ee8011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a508cd4735b2839 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:58:18.170 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.169+0000 7f5ef2a00640 1 --2- 172.21.15.72:0/392432732 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f5ee00a3c40 0x7f5ee00a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:18.170 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.169+0000 7f5ee7400640 1 -- 172.21.15.72:0/2278818292 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ee801e070 con 0x5563ba7e2ea0 2024-04-26T21:58:18.170 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.169+0000 7f5ef2a00640 1 -- 172.21.15.72:0/392432732 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7f5ef40f0e30 con 0x7f5efc104b90 2024-04-26T21:58:18.170 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.169+0000 7f5ee7400640 1 -- 172.21.15.72:0/2278818292 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5ee8009e50 con 0x5563ba7e2ea0 2024-04-26T21:58:18.170 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.169+0000 7f5ee7400640 1 -- 172.21.15.72:0/2278818292 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ee800e940 con 0x5563ba7e2ea0 2024-04-26T21:58:18.170 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.169+0000 7f5ef9e00640 1 --2- 172.21.15.72:0/392432732 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f5ee00a3c40 0x7f5ee00a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:18.171 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.171+0000 7f5f03d7f740 1 -- 172.21.15.72:0/2278818292 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5563ba7e2ea0 msgr2=0x5563ba7e32c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:18.172 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.171+0000 7f5f03d7f740 1 --2- 172.21.15.72:0/2278818292 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5563ba7e2ea0 0x5563ba7e32c0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f5ee8009ca0 tx=0x7f5ee8011810 comp rx=0 tx=0).stop 2024-04-26T21:58:18.172 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.171+0000 7f5ef9e00640 1 --2- 172.21.15.72:0/392432732 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f5ee00a3c40 0x7f5ee00a6100 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x5563ba7e4300 tx=0x7f5ef402a6d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:58:18.172 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.171+0000 7f5f03d7f740 1 -- 172.21.15.72:0/2278818292 shutdown_connections 2024-04-26T21:58:18.172 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.171+0000 7f5f03d7f740 1 --2- 172.21.15.72:0/2278818292 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5563ba7e8a00 0x5563ba7eae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:18.172 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.171+0000 7f5f03d7f740 1 --2- 172.21.15.72:0/2278818292 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5563ba7e3890 0x5563ba7e8390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:18.172 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.171+0000 7f5f03d7f740 1 --2- 172.21.15.72:0/2278818292 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5563ba7e2ea0 0x5563ba7e32c0 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:18.172 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.171+0000 7f5f03d7f740 1 -- 172.21.15.72:0/2278818292 >> 172.21.15.72:0/2278818292 conn(0x5563ba7de510 msgr2=0x5563ba7de910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:58:18.172 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.171+0000 7f5f03d7f740 1 -- 172.21.15.72:0/2278818292 shutdown_connections 2024-04-26T21:58:18.172 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.171+0000 7f5f03d7f740 1 -- 172.21.15.72:0/2278818292 wait complete. 2024-04-26T21:58:18.172 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.172+0000 7f5f03d7f740 1 -- start start 2024-04-26T21:58:18.172 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.172+0000 7f5f03d7f740 1 Processor -- start 2024-04-26T21:58:18.173 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.172+0000 7f5f03d7f740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5563ba7e2ea0 0x5563ba7ece50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:18.174 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.172+0000 7f5f03d7f740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5563ba7e3890 0x5563ba7ed390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:18.174 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.172+0000 7f5f03d7f740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5563ba7e8a00 0x5563ba7f4360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:18.174 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.172+0000 7f5f03d7f740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x5563ba745f80 con 0x5563ba7e8a00 2024-04-26T21:58:18.174 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.172+0000 7f5f03d7f740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5563ba6f3260 con 0x5563ba7e2ea0 2024-04-26T21:58:18.174 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.172+0000 7f5f03d7f740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5563ba74b2a0 con 0x5563ba7e3890 2024-04-26T21:58:18.174 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.172+0000 7f5f03d7f740 10 client.0 _ll_register_callbacks cb 0x5563ba73d960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:58:18.174 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.173+0000 7f5ef9400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5563ba7e3890 0x5563ba7ed390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:18.174 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.173+0000 7f5ef9400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5563ba7e3890 0x5563ba7ed390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.72:55914/0 (socket says 172.21.15.72:55914) 2024-04-26T21:58:18.174 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.173+0000 7f5ef9400640 1 -- 172.21.15.72:0/465925304 learned_addr learned my addr 172.21.15.72:0/465925304 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:58:18.174 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.173+0000 7f5efa800640 1 --2- 172.21.15.72:0/465925304 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5563ba7e2ea0 0x5563ba7ece50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:18.174 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.173+0000 7f5efa800640 1 --2- 172.21.15.72:0/465925304 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5563ba7e8a00 0x5563ba7f4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:18.174 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.173+0000 7f5ef9400640 1 -- 172.21.15.72:0/465925304 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5563ba7e2ea0 msgr2=0x5563ba7ece50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:58:18.174 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.173+0000 7f5ef9400640 1 --2- 172.21.15.72:0/465925304 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5563ba7e2ea0 0x5563ba7ece50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:18.175 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.173+0000 7f5ef9400640 1 -- 172.21.15.72:0/465925304 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5563ba7e8a00 msgr2=0x5563ba7f4360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:58:18.175 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.174+0000 7f5ef9400640 1 --2- 172.21.15.72:0/465925304 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5563ba7e8a00 0x5563ba7f4360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:18.175 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.174+0000 7f5ef9400640 1 -- 172.21.15.72:0/465925304 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5563ba6d51c0 con 0x5563ba7e3890 2024-04-26T21:58:18.175 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.174+0000 7f5efa800640 1 --2- 172.21.15.72:0/465925304 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5563ba7e8a00 0x5563ba7f4360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:58:18.176 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.174+0000 7f5ef9400640 1 --2- 172.21.15.72:0/465925304 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5563ba7e3890 0x5563ba7ed390 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f5eec0116d0 tx=0x7f5eec011b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b17b8e9107ae2c28 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:58:18.176 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.174+0000 7f5edbe00640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-04-26T21:58:18.176 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.175+0000 7f5edbe00640 1 -- 172.21.15.72:0/465925304 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eec01e020 con 0x5563ba7e3890 2024-04-26T21:58:18.176 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.175+0000 7f5edbe00640 1 -- 172.21.15.72:0/465925304 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7f5eec00fb90 con 0x5563ba7e3890 2024-04-26T21:58:18.176 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.175+0000 7f5f03d7f740 1 -- 172.21.15.72:0/465925304 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5ee8009e50 con 0x5563ba7e3890 2024-04-26T21:58:18.176 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.175+0000 7f5edbe00640 1 -- 172.21.15.72:0/465925304 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f5eec00fd90 con 0x5563ba7e3890 2024-04-26T21:58:18.176 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.175+0000 7f5f03d7f740 10 client.25336 fetch_fsmap learned FSMap version 27 2024-04-26T21:58:18.176 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.175+0000 7f5f03d7f740 1 -- 172.21.15.72:0/465925304 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5563ba4583f0 con 0x5563ba7e3890 2024-04-26T21:58:18.176 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.176+0000 7f5edbe00640 1 -- 172.21.15.72:0/465925304 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5ee8009e50 con 0x5563ba7e3890 2024-04-26T21:58:18.177 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.177+0000 7f5edbe00640 1 -- 172.21.15.72:0/465925304 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f5eec019070 con 0x5563ba7e3890 2024-04-26T21:58:18.177 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.177+0000 7f5f03d7f740 10 client.25336 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T21:58:18.177 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.177+0000 7f5f03d7f740 10 client.25336 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-26T21:58:18.177 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.177+0000 7f5f03d7f740 20 client.25336 populate_metadata read hostname 'smithi072' 2024-04-26T21:58:18.177 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.177+0000 7f5f03d7f740 1 --2- 172.21.15.72:0/465925304 >> [v2:172.21.15.76:6834/3517429435,v1:172.21.15.76:6835/3517429435] conn(0x5563ba7f8970 0x5563ba7fad60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:18.177 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.177+0000 7f5f03d7f740 4 client.25336 mds_command: new command op to 34442 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:58:18.177 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.177+0000 7f5f03d7f740 1 -- 172.21.15.72:0/465925304 --> [v2:172.21.15.76:6834/3517429435,v1:172.21.15.76:6835/3517429435] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5563ba754880 con 0x5563ba7f8970 2024-04-26T21:58:18.178 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.178+0000 7f5efa800640 1 --2- 172.21.15.72:0/465925304 >> [v2:172.21.15.76:6834/3517429435,v1:172.21.15.76:6835/3517429435] conn(0x5563ba7f8970 0x5563ba7fad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:18.178 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.178+0000 7f5efa800640 1 --2- 172.21.15.72:0/465925304 >> [v2:172.21.15.76:6834/3517429435,v1:172.21.15.76:6835/3517429435] conn(0x5563ba7f8970 0x5563ba7fad60 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8f2e6e40a5d93e8e server_cookie=370fccd5806331fe in_seq=0 out_seq=0 2024-04-26T21:58:18.178 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.179+0000 7f5edbe00640 10 client.25336 ms_handle_connect on v2:172.21.15.76:6834/3517429435 2024-04-26T21:58:18.179 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.180+0000 7f5edbe00640 1 -- 172.21.15.72:0/465925304 <== mds.2 v2:172.21.15.76:6834/3517429435 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5563ba754880 con 0x5563ba7f8970 2024-04-26T21:58:18.180 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.180+0000 7f5edbe00640 10 client.25336 handle_command_reply: tid=0 2024-04-26T21:58:18.180 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.180+0000 7f5f03d7f740 1 client.25336 shutdown 2024-04-26T21:58:18.180 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.180+0000 7f5f03d7f740 1 -- 172.21.15.72:0/465925304 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5563ba7e3890 msgr2=0x5563ba7ed390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:18.181 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.180+0000 7f5f03d7f740 1 --2- 172.21.15.72:0/465925304 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5563ba7e3890 0x5563ba7ed390 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f5eec0116d0 tx=0x7f5eec011b90 comp rx=0 tx=0).stop 2024-04-26T21:58:18.181 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.180+0000 7f5f03d7f740 1 -- 172.21.15.72:0/465925304 shutdown_connections 2024-04-26T21:58:18.181 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.180+0000 7f5f03d7f740 1 --2- 172.21.15.72:0/465925304 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5563ba7e8a00 0x5563ba7f4360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:18.181 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.180+0000 7f5f03d7f740 1 --2- 172.21.15.72:0/465925304 >> [v2:172.21.15.76:6834/3517429435,v1:172.21.15.76:6835/3517429435] conn(0x5563ba7f8970 0x5563ba7fad60 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:18.181 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.180+0000 7f5f03d7f740 1 --2- 172.21.15.72:0/465925304 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5563ba7e3890 0x5563ba7ed390 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:18.181 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.180+0000 7f5f03d7f740 1 --2- 172.21.15.72:0/465925304 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5563ba7e2ea0 0x5563ba7ece50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:18.181 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.180+0000 7f5f03d7f740 1 -- 172.21.15.72:0/465925304 >> 172.21.15.72:0/465925304 conn(0x5563ba7de510 msgr2=0x5563ba7dd690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:58:18.181 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.180+0000 7f5edbe00640 0 client.25336 ms_handle_reset on v2:172.21.15.76:6834/3517429435 2024-04-26T21:58:18.181 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.180+0000 7f5f03d7f740 1 -- 172.21.15.72:0/465925304 shutdown_connections 2024-04-26T21:58:18.181 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.181+0000 7f5f03d7f740 1 -- 172.21.15.72:0/465925304 wait complete. 2024-04-26T21:58:18.181 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.181+0000 7f5f03d7f740 20 client.25336 trim_cache size 0 max 16384 2024-04-26T21:58:18.202 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.202+0000 7f5f03d7f740 1 Processor -- start 2024-04-26T21:58:18.202 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.202+0000 7f5f03d7f740 1 -- start start 2024-04-26T21:58:18.203 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.202+0000 7f5f03d7f740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5563ba7e2ea0 0x5563ba7ece50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:18.203 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.202+0000 7f5f03d7f740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5563ba7e3890 0x5563ba7ed390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:18.203 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.202+0000 7f5f03d7f740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5563ba7e8a00 0x5563ba7f4320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:18.203 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.202+0000 7f5f03d7f740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f5efc1b0610 con 0x5563ba7e8a00 2024-04-26T21:58:18.203 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.202+0000 7f5f03d7f740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f5efc1b2150 con 0x5563ba7e3890 2024-04-26T21:58:18.203 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.202+0000 7f5f03d7f740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f5efc1b0ce0 con 0x5563ba7e2ea0 2024-04-26T21:58:18.203 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.203+0000 7f5efa800640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5563ba7e8a00 0x5563ba7f4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:18.203 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.203+0000 7f5efa800640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5563ba7e8a00 0x5563ba7f4320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:46076/0 (socket says 172.21.15.72:46076) 2024-04-26T21:58:18.203 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.203+0000 7f5efa800640 1 -- 172.21.15.72:0/3633828991 learned_addr learned my addr 172.21.15.72:0/3633828991 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:58:18.203 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.203+0000 7f5efa800640 1 --2- 172.21.15.72:0/3633828991 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5563ba7e2ea0 0x5563ba7ece50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:18.204 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.203+0000 7f5efa800640 1 -- 172.21.15.72:0/3633828991 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5563ba7e2ea0 msgr2=0x5563ba7ece50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:58:18.204 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.203+0000 7f5efa800640 1 --2- 172.21.15.72:0/3633828991 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5563ba7e2ea0 0x5563ba7ece50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:18.204 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.203+0000 7f5efa800640 1 -- 172.21.15.72:0/3633828991 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5563ba7e3890 msgr2=0x5563ba7ed390 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T21:58:18.205 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.203+0000 7f5efa800640 1 --2- 172.21.15.72:0/3633828991 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5563ba7e3890 0x5563ba7ed390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:18.205 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.203+0000 7f5efa800640 1 -- 172.21.15.72:0/3633828991 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5563ba5227d0 con 0x5563ba7e8a00 2024-04-26T21:58:18.205 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.204+0000 7f5efa800640 1 --2- 172.21.15.72:0/3633828991 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5563ba7e8a00 0x5563ba7f4320 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f5ee8011d10 tx=0x7f5ee8000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=633490e2935776fa server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:58:18.205 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.204+0000 7f5ee6a00640 1 -- 172.21.15.72:0/3633828991 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ee801e040 con 0x5563ba7e8a00 2024-04-26T21:58:18.205 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.204+0000 7f5ee6a00640 1 -- 172.21.15.72:0/3633828991 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5ee80050d0 con 0x5563ba7e8a00 2024-04-26T21:58:18.205 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.204+0000 7f5ee6a00640 1 -- 172.21.15.72:0/3633828991 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ee802e3b0 con 0x5563ba7e8a00 2024-04-26T21:58:18.206 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.205+0000 7f5f03d7f740 1 -- 172.21.15.72:0/3633828991 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5563ba7e8a00 msgr2=0x5563ba7f4320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:18.206 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.205+0000 7f5f03d7f740 1 --2- 172.21.15.72:0/3633828991 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5563ba7e8a00 0x5563ba7f4320 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f5ee8011d10 tx=0x7f5ee8000bc0 comp rx=0 tx=0).stop 2024-04-26T21:58:18.206 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.205+0000 7f5f03d7f740 1 -- 172.21.15.72:0/3633828991 shutdown_connections 2024-04-26T21:58:18.206 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.205+0000 7f5f03d7f740 1 --2- 172.21.15.72:0/3633828991 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5563ba7e8a00 0x5563ba7f4320 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:18.206 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.205+0000 7f5f03d7f740 1 --2- 172.21.15.72:0/3633828991 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5563ba7e3890 0x5563ba7ed390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:18.206 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.205+0000 7f5f03d7f740 1 --2- 172.21.15.72:0/3633828991 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5563ba7e2ea0 0x5563ba7ece50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:18.206 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.205+0000 7f5f03d7f740 1 -- 172.21.15.72:0/3633828991 >> 172.21.15.72:0/3633828991 conn(0x5563ba7de510 msgr2=0x5563ba7ebbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:58:18.207 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.205+0000 7f5f03d7f740 1 -- 172.21.15.72:0/3633828991 shutdown_connections 2024-04-26T21:58:18.207 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.205+0000 7f5f03d7f740 1 -- 172.21.15.72:0/3633828991 wait complete. 2024-04-26T21:58:18.207 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.206+0000 7f5f03d7f740 1 -- start start 2024-04-26T21:58:18.207 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.206+0000 7f5f03d7f740 1 Processor -- start 2024-04-26T21:58:18.208 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.206+0000 7f5f03d7f740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5563ba7e2ea0 0x5563ba7f4380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:18.208 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.206+0000 7f5f03d7f740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5563ba7e3890 0x5563ba7f0e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:18.208 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.207+0000 7f5f03d7f740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5563ba7e8a00 0x5563ba7f1430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:18.208 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.207+0000 7f5f03d7f740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f5efc1b1a80 con 0x5563ba7e2ea0 2024-04-26T21:58:18.208 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.207+0000 7f5f03d7f740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f5efc1b13b0 con 0x5563ba7e3890 2024-04-26T21:58:18.209 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.207+0000 7f5f03d7f740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f5efc1ae5f0 con 0x5563ba7e8a00 2024-04-26T21:58:18.209 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.207+0000 7f5f03d7f740 10 client.0 _ll_register_callbacks cb 0x7f5eec009890 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:58:18.209 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.207+0000 7f5efa800640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5563ba7e2ea0 0x5563ba7f4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:18.209 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.207+0000 7f5ef9400640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5563ba7e3890 0x5563ba7f0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:18.209 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.207+0000 7f5efa800640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5563ba7e2ea0 0x5563ba7f4380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:46086/0 (socket says 172.21.15.72:46086) 2024-04-26T21:58:18.209 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.207+0000 7f5efa800640 1 -- 172.21.15.72:0/2651173699 learned_addr learned my addr 172.21.15.72:0/2651173699 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:58:18.209 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.208+0000 7f5efa800640 1 -- 172.21.15.72:0/2651173699 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5563ba7e8a00 msgr2=0x5563ba7f1430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:58:18.209 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.208+0000 7f5efa800640 1 --2- 172.21.15.72:0/2651173699 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5563ba7e8a00 0x5563ba7f1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:18.209 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.208+0000 7f5efa800640 1 -- 172.21.15.72:0/2651173699 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5563ba7e3890 msgr2=0x5563ba7f0e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:58:18.209 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.208+0000 7f5efa800640 1 --2- 172.21.15.72:0/2651173699 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5563ba7e3890 0x5563ba7f0e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:18.209 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.208+0000 7f5efa800640 1 -- 172.21.15.72:0/2651173699 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5563ba6ed2a0 con 0x5563ba7e2ea0 2024-04-26T21:58:18.209 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.208+0000 7f5ef9400640 1 --2- 172.21.15.72:0/2651173699 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5563ba7e3890 0x5563ba7f0e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:58:18.209 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.208+0000 7f5efa800640 1 --2- 172.21.15.72:0/2651173699 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5563ba7e2ea0 0x5563ba7f4380 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f5ee8009250 tx=0x7f5ee8009fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5571a5575a76f009 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:58:18.209 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.208+0000 7f5edbe00640 10 client.0 ms_handle_connect on v2:172.21.15.72:3300/0 2024-04-26T21:58:18.210 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.208+0000 7f5edbe00640 1 -- 172.21.15.72:0/2651173699 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ee801e030 con 0x5563ba7e2ea0 2024-04-26T21:58:18.210 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.208+0000 7f5f03d7f740 1 -- 172.21.15.72:0/2651173699 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5ee80050d0 con 0x5563ba7e2ea0 2024-04-26T21:58:18.210 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.208+0000 7f5edbe00640 1 -- 172.21.15.72:0/2651173699 <== mon.0 v2:172.21.15.72:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7f5ee8005280 con 0x5563ba7e2ea0 2024-04-26T21:58:18.210 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.208+0000 7f5edbe00640 1 -- 172.21.15.72:0/2651173699 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f5ee8005480 con 0x5563ba7e2ea0 2024-04-26T21:58:18.210 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.209+0000 7f5f03d7f740 10 client.16194 fetch_fsmap learned FSMap version 27 2024-04-26T21:58:18.210 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.209+0000 7f5f03d7f740 1 -- 172.21.15.72:0/2651173699 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5563ba85e010 con 0x5563ba7e2ea0 2024-04-26T21:58:18.211 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.209+0000 7f5edbe00640 1 -- 172.21.15.72:0/2651173699 <== mon.0 v2:172.21.15.72:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5ee80050d0 con 0x5563ba7e2ea0 2024-04-26T21:58:18.211 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.209+0000 7f5edbe00640 1 -- 172.21.15.72:0/2651173699 <== mon.0 v2:172.21.15.72:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f5ee8024040 con 0x5563ba7e2ea0 2024-04-26T21:58:18.211 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.209+0000 7f5f03d7f740 10 client.16194 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T21:58:18.211 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.209+0000 7f5f03d7f740 10 client.16194 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-26T21:58:18.211 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.209+0000 7f5f03d7f740 20 client.16194 populate_metadata read hostname 'smithi072' 2024-04-26T21:58:18.211 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.209+0000 7f5f03d7f740 1 --2- 172.21.15.72:0/2651173699 >> [v2:172.21.15.76:6834/3517429435,v1:172.21.15.76:6835/3517429435] conn(0x5563ba85e700 0x5563ba860b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:18.211 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.209+0000 7f5f03d7f740 4 client.16194 mds_command: new command op to 34442 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-26T21:58:18.211 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.209+0000 7f5f03d7f740 1 -- 172.21.15.72:0/2651173699 --> [v2:172.21.15.76:6834/3517429435,v1:172.21.15.76:6835/3517429435] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5563ba74e6f0 con 0x5563ba85e700 2024-04-26T21:58:18.211 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.210+0000 7f5ef9400640 1 --2- 172.21.15.72:0/2651173699 >> [v2:172.21.15.76:6834/3517429435,v1:172.21.15.76:6835/3517429435] conn(0x5563ba85e700 0x5563ba860b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:18.211 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.211+0000 7f5ef9400640 1 --2- 172.21.15.72:0/2651173699 >> [v2:172.21.15.76:6834/3517429435,v1:172.21.15.76:6835/3517429435] conn(0x5563ba85e700 0x5563ba860b20 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c8265187f6cf871b server_cookie=50f51c625461f59d in_seq=0 out_seq=0 2024-04-26T21:58:18.211 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.211+0000 7f5edbe00640 10 client.16194 ms_handle_connect on v2:172.21.15.76:6834/3517429435 2024-04-26T21:58:18.213 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.211+0000 7f5edbe00640 1 -- 172.21.15.72:0/2651173699 <== mds.2 v2:172.21.15.76:6834/3517429435 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5563ba74e6f0 con 0x5563ba85e700 2024-04-26T21:58:18.213 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.211+0000 7f5edbe00640 10 client.16194 handle_command_reply: tid=0 2024-04-26T21:58:18.213 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.211+0000 7f5f03d7f740 1 client.16194 shutdown 2024-04-26T21:58:18.213 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.212+0000 7f5f03d7f740 1 -- 172.21.15.72:0/2651173699 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5563ba7e2ea0 msgr2=0x5563ba7f4380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:18.213 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.212+0000 7f5f03d7f740 1 --2- 172.21.15.72:0/2651173699 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5563ba7e2ea0 0x5563ba7f4380 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f5ee8009250 tx=0x7f5ee8009fb0 comp rx=0 tx=0).stop 2024-04-26T21:58:18.213 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.212+0000 7f5f03d7f740 1 -- 172.21.15.72:0/2651173699 shutdown_connections 2024-04-26T21:58:18.213 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.212+0000 7f5f03d7f740 1 --2- 172.21.15.72:0/2651173699 >> [v2:172.21.15.76:6834/3517429435,v1:172.21.15.76:6835/3517429435] conn(0x5563ba85e700 0x5563ba860b20 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:18.213 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.212+0000 7f5f03d7f740 1 --2- 172.21.15.72:0/2651173699 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5563ba7e8a00 0x5563ba7f1430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:18.213 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.212+0000 7f5f03d7f740 1 --2- 172.21.15.72:0/2651173699 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5563ba7e3890 0x5563ba7f0e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:18.213 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.212+0000 7f5f03d7f740 1 --2- 172.21.15.72:0/2651173699 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5563ba7e2ea0 0x5563ba7f4380 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:18.213 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.212+0000 7f5f03d7f740 1 -- 172.21.15.72:0/2651173699 >> 172.21.15.72:0/2651173699 conn(0x5563ba7de510 msgr2=0x5563ba7ecd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:58:18.213 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.212+0000 7f5edbe00640 0 client.16194 ms_handle_reset on v2:172.21.15.76:6834/3517429435 2024-04-26T21:58:18.213 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.212+0000 7f5f03d7f740 1 -- 172.21.15.72:0/2651173699 shutdown_connections 2024-04-26T21:58:18.214 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.212+0000 7f5f03d7f740 1 -- 172.21.15.72:0/2651173699 wait complete. 2024-04-26T21:58:18.214 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.212+0000 7f5f03d7f740 20 client.16194 trim_cache size 0 max 16384 2024-04-26T21:58:18.214 INFO:teuthology.orchestra.run.smithi072.stdout:[] 2024-04-26T21:58:18.214 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.213+0000 7f5f00e00640 1 -- 172.21.15.72:0/392432732 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f5ee00a3c40 msgr2=0x7f5ee00a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:18.215 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.213+0000 7f5f00e00640 1 --2- 172.21.15.72:0/392432732 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f5ee00a3c40 0x7f5ee00a6100 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x5563ba7e4300 tx=0x7f5ef402a6d0 comp rx=0 tx=0).stop 2024-04-26T21:58:18.215 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.213+0000 7f5f00e00640 1 -- 172.21.15.72:0/392432732 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5efc104b90 msgr2=0x7f5efc19f0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:18.215 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.213+0000 7f5f00e00640 1 --2- 172.21.15.72:0/392432732 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5efc104b90 0x7f5efc19f0f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f5ef402acc0 tx=0x7f5ef4030be0 comp rx=0 tx=0).stop 2024-04-26T21:58:18.216 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.213+0000 7f5f00e00640 1 -- 172.21.15.72:0/392432732 shutdown_connections 2024-04-26T21:58:18.216 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.213+0000 7f5f00e00640 1 --2- 172.21.15.72:0/392432732 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5efc106740 0x7f5efc1a66a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:18.216 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.213+0000 7f5f00e00640 1 --2- 172.21.15.72:0/392432732 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5efc105d80 0x7f5efc19f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:18.216 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.213+0000 7f5f00e00640 1 --2- 172.21.15.72:0/392432732 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f5ee00a3c40 0x7f5ee00a6100 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:18.216 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.213+0000 7f5f00e00640 1 --2- 172.21.15.72:0/392432732 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5efc104b90 0x7f5efc19f0f0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:18.216 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.213+0000 7f5f00e00640 1 -- 172.21.15.72:0/392432732 >> 172.21.15.72:0/392432732 conn(0x7f5efc1000e0 msgr2=0x7f5efc078d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:58:18.216 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.214+0000 7f5f00e00640 1 -- 172.21.15.72:0/392432732 shutdown_connections 2024-04-26T21:58:18.216 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.214+0000 7f5f00e00640 1 -- 172.21.15.72:0/392432732 wait complete. 2024-04-26T21:58:18.233 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:58:17 smithi072 ceph-mon[28138]: from='client.? 172.21.15.72:0/1914470993' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:58:18.236 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-26T21:58:18.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:58:17 smithi076 ceph-mon[31823]: from='client.? 172.21.15.72:0/1914470993' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:58:18.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:58:17 smithi164 ceph-mon[31916]: from='client.? 172.21.15.72:0/1914470993' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:58:18.474 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.474+0000 7f344fe00640 1 -- 172.21.15.72:0/3941890860 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3450101740 msgr2=0x7f3450101b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:18.474 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.474+0000 7f344fe00640 1 --2- 172.21.15.72:0/3941890860 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3450101740 0x7f3450101b60 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f3440009900 tx=0x7f344002e800 comp rx=0 tx=0).stop 2024-04-26T21:58:18.474 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.474+0000 7f344fe00640 1 -- 172.21.15.72:0/3941890860 shutdown_connections 2024-04-26T21:58:18.474 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.474+0000 7f344fe00640 1 --2- 172.21.15.72:0/3941890860 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f34501032f0 0x7f3450109b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:18.474 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.474+0000 7f344fe00640 1 --2- 172.21.15.72:0/3941890860 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3450102930 0x7f3450102db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:18.474 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.474+0000 7f344fe00640 1 --2- 172.21.15.72:0/3941890860 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3450101740 0x7f3450101b60 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:18.475 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.474+0000 7f344fe00640 1 -- 172.21.15.72:0/3941890860 >> 172.21.15.72:0/3941890860 conn(0x7f34500fccd0 msgr2=0x7f34500ff0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:58:18.475 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.474+0000 7f344fe00640 1 -- 172.21.15.72:0/3941890860 shutdown_connections 2024-04-26T21:58:18.475 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.474+0000 7f344fe00640 1 -- 172.21.15.72:0/3941890860 wait complete. 2024-04-26T21:58:18.475 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.475+0000 7f344fe00640 1 Processor -- start 2024-04-26T21:58:18.475 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.475+0000 7f344fe00640 1 -- start start 2024-04-26T21:58:18.475 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.476+0000 7f344fe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3450101740 0x7f34501967d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:18.476 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.476+0000 7f344fe00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3450102930 0x7f3450196d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:18.476 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.476+0000 7f344fe00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f34501032f0 0x7f345019dd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:18.476 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.476+0000 7f344fe00640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f345010c070 con 0x7f3450102930 2024-04-26T21:58:18.476 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.476+0000 7f344fe00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f345010bef0 con 0x7f34501032f0 2024-04-26T21:58:18.476 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.476+0000 7f344fe00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f345010c1f0 con 0x7f3450101740 2024-04-26T21:58:18.476 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.476+0000 7f344f400640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f34501032f0 0x7f345019dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:18.476 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.476+0000 7f344f400640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f34501032f0 0x7f345019dd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.72:48790/0 (socket says 172.21.15.72:48790) 2024-04-26T21:58:18.477 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.476+0000 7f344f400640 1 -- 172.21.15.72:0/34999851 learned_addr learned my addr 172.21.15.72:0/34999851 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:58:18.477 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.477+0000 7f344ea00640 1 --2- 172.21.15.72:0/34999851 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3450101740 0x7f34501967d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:18.477 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.477+0000 7f344e000640 1 --2- 172.21.15.72:0/34999851 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3450102930 0x7f3450196d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:18.477 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.477+0000 7f344f400640 1 -- 172.21.15.72:0/34999851 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3450101740 msgr2=0x7f34501967d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:18.477 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.477+0000 7f344f400640 1 --2- 172.21.15.72:0/34999851 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3450101740 0x7f34501967d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:18.478 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.477+0000 7f344f400640 1 -- 172.21.15.72:0/34999851 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3450102930 msgr2=0x7f3450196d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:18.478 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.477+0000 7f344f400640 1 --2- 172.21.15.72:0/34999851 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3450102930 0x7f3450196d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:18.478 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.477+0000 7f344f400640 1 -- 172.21.15.72:0/34999851 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f345019e3f0 con 0x7f34501032f0 2024-04-26T21:58:18.478 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.478+0000 7f344f400640 1 --2- 172.21.15.72:0/34999851 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f34501032f0 0x7f345019dd80 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f344800eaa0 tx=0x7f344800ef60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:58:18.478 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.478+0000 7f3447400640 1 -- 172.21.15.72:0/34999851 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f344800cdb0 con 0x7f34501032f0 2024-04-26T21:58:18.478 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.478+0000 7f344fe00640 1 -- 172.21.15.72:0/34999851 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f345019e6e0 con 0x7f34501032f0 2024-04-26T21:58:18.479 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.478+0000 7f3447400640 1 -- 172.21.15.72:0/34999851 <== mon.2 v2:172.21.15.76:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3448005020 con 0x7f34501032f0 2024-04-26T21:58:18.479 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.478+0000 7f3447400640 1 -- 172.21.15.72:0/34999851 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34480116d0 con 0x7f34501032f0 2024-04-26T21:58:18.479 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.478+0000 7f344fe00640 1 -- 172.21.15.72:0/34999851 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f345019ec30 con 0x7f34501032f0 2024-04-26T21:58:18.481 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.482+0000 7f34589b5740 1 Processor -- start 2024-04-26T21:58:18.483 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.483+0000 7f3447400640 1 -- 172.21.15.72:0/34999851 <== mon.2 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f34480051d0 con 0x7f34501032f0 2024-04-26T21:58:18.483 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.483+0000 7f34589b5740 1 -- start start 2024-04-26T21:58:18.483 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.483+0000 7f34589b5740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558a037bf050 0x558a037bf470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:18.484 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.483+0000 7f34589b5740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558a037bfa40 0x558a037c4540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:18.484 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.483+0000 7f34589b5740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558a037c4bb0 0x558a037c7040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:18.484 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.483+0000 7f34589b5740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x558a03722c70 con 0x558a037bf050 2024-04-26T21:58:18.484 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.483+0000 7f34589b5740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x558a036db980 con 0x558a037bfa40 2024-04-26T21:58:18.484 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.483+0000 7f34589b5740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x558a0371ec30 con 0x558a037c4bb0 2024-04-26T21:58:18.484 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.484+0000 7f344ea00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558a037bfa40 0x558a037c4540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:18.484 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.484+0000 7f344ea00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558a037bfa40 0x558a037c4540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.72:48806/0 (socket says 172.21.15.72:48806) 2024-04-26T21:58:18.485 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.484+0000 7f344e000640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558a037bf050 0x558a037bf470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:18.485 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.484+0000 7f344ea00640 1 -- 172.21.15.72:0/2217423967 learned_addr learned my addr 172.21.15.72:0/2217423967 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:58:18.485 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.485+0000 7f3447400640 1 --2- 172.21.15.72:0/34999851 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f34340a3d00 0x7f34340a61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:18.485 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.485+0000 7f344e000640 1 --2- 172.21.15.72:0/2217423967 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558a037c4bb0 0x558a037c7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:18.486 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.486+0000 7f3447400640 1 -- 172.21.15.72:0/34999851 <== mon.2 v2:172.21.15.76:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7f34480d0aa0 con 0x7f34501032f0 2024-04-26T21:58:18.486 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.486+0000 7f344e000640 1 -- 172.21.15.72:0/2217423967 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558a037c4bb0 msgr2=0x558a037c7040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:58:18.486 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.486+0000 7f344e000640 1 --2- 172.21.15.72:0/2217423967 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558a037c4bb0 0x558a037c7040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:18.486 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.486+0000 7f344e000640 1 -- 172.21.15.72:0/2217423967 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558a037bfa40 msgr2=0x558a037c4540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:58:18.486 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.486+0000 7f344e000640 1 --2- 172.21.15.72:0/2217423967 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558a037bfa40 0x558a037c4540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:18.486 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.486+0000 7f344f400640 1 --2- 172.21.15.72:0/34999851 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f34340a3d00 0x7f34340a61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:18.486 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.486+0000 7f344e000640 1 -- 172.21.15.72:0/2217423967 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a036b37b0 con 0x558a037bf050 2024-04-26T21:58:18.487 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.486+0000 7f344e000640 1 --2- 172.21.15.72:0/2217423967 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558a037bf050 0x558a037bf470 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f3450197ce0 tx=0x7f343c009870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fec187feac9ed94 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:58:18.487 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.487+0000 7f344f400640 1 --2- 172.21.15.72:0/34999851 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f34340a3d00 0x7f34340a61c0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f344809cb80 tx=0x7f3448005b40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:58:18.487 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.487+0000 7f343be00640 1 -- 172.21.15.72:0/2217423967 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f343c01a030 con 0x558a037bf050 2024-04-26T21:58:18.488 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.487+0000 7f343be00640 1 -- 172.21.15.72:0/2217423967 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f343c005020 con 0x558a037bf050 2024-04-26T21:58:18.488 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.487+0000 7f343be00640 1 -- 172.21.15.72:0/2217423967 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f343c004ae0 con 0x558a037bf050 2024-04-26T21:58:18.488 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.488+0000 7f34589b5740 1 -- 172.21.15.72:0/2217423967 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558a037bf050 msgr2=0x558a037bf470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:18.488 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.488+0000 7f34589b5740 1 --2- 172.21.15.72:0/2217423967 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558a037bf050 0x558a037bf470 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f3450197ce0 tx=0x7f343c009870 comp rx=0 tx=0).stop 2024-04-26T21:58:18.489 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.488+0000 7f34589b5740 1 -- 172.21.15.72:0/2217423967 shutdown_connections 2024-04-26T21:58:18.489 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.488+0000 7f34589b5740 1 --2- 172.21.15.72:0/2217423967 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558a037c4bb0 0x558a037c7040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:18.489 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.488+0000 7f34589b5740 1 --2- 172.21.15.72:0/2217423967 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558a037bfa40 0x558a037c4540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:18.489 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.488+0000 7f34589b5740 1 --2- 172.21.15.72:0/2217423967 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558a037bf050 0x558a037bf470 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:18.489 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.488+0000 7f34589b5740 1 -- 172.21.15.72:0/2217423967 >> 172.21.15.72:0/2217423967 conn(0x558a037ba6c0 msgr2=0x558a037baac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:58:18.489 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.488+0000 7f34589b5740 1 -- 172.21.15.72:0/2217423967 shutdown_connections 2024-04-26T21:58:18.489 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.489+0000 7f34589b5740 1 -- 172.21.15.72:0/2217423967 wait complete. 2024-04-26T21:58:18.491 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.489+0000 7f34589b5740 1 -- start start 2024-04-26T21:58:18.491 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.489+0000 7f34589b5740 1 Processor -- start 2024-04-26T21:58:18.491 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.490+0000 7f34589b5740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558a037bf050 0x558a037d17a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:18.491 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.490+0000 7f34589b5740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558a037bfa40 0x558a037d1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:18.491 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.490+0000 7f34589b5740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558a037c4bb0 0x558a037d8d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:18.491 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.490+0000 7f34589b5740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x558a03721f80 con 0x558a037bfa40 2024-04-26T21:58:18.491 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.490+0000 7f34589b5740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x558a036cf260 con 0x558a037c4bb0 2024-04-26T21:58:18.491 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.490+0000 7f34589b5740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x558a037272a0 con 0x558a037bf050 2024-04-26T21:58:18.492 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.490+0000 7f34589b5740 10 client.0 _ll_register_callbacks cb 0x558a03719960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:58:18.492 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.490+0000 7f344ea00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558a037bfa40 0x558a037d1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:18.492 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.490+0000 7f344e000640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558a037c4bb0 0x558a037d8d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:18.492 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.490+0000 7f344ea00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558a037bfa40 0x558a037d1ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:46138/0 (socket says 172.21.15.72:46138) 2024-04-26T21:58:18.492 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.490+0000 7f344ea00640 1 -- 172.21.15.72:0/1634681219 learned_addr learned my addr 172.21.15.72:0/1634681219 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:58:18.492 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.490+0000 7f344e000640 1 --2- 172.21.15.72:0/1634681219 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558a037bf050 0x558a037d17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:18.492 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.491+0000 7f344ea00640 1 -- 172.21.15.72:0/1634681219 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558a037bf050 msgr2=0x558a037d17a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:58:18.492 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.491+0000 7f344ea00640 1 --2- 172.21.15.72:0/1634681219 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558a037bf050 0x558a037d17a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:18.492 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.491+0000 7f344ea00640 1 -- 172.21.15.72:0/1634681219 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558a037c4bb0 msgr2=0x558a037d8d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:58:18.492 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.491+0000 7f344ea00640 1 --2- 172.21.15.72:0/1634681219 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558a037c4bb0 0x558a037d8d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:18.492 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.491+0000 7f344ea00640 1 -- 172.21.15.72:0/1634681219 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a036b11c0 con 0x558a037bfa40 2024-04-26T21:58:18.492 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.491+0000 7f344e000640 1 --2- 172.21.15.72:0/1634681219 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558a037c4bb0 0x558a037d8d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:58:18.492 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.491+0000 7f344ea00640 1 --2- 172.21.15.72:0/1634681219 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558a037bfa40 0x558a037d1ce0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f3440009230 tx=0x7f34400044f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e4f242de264b219 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:58:18.492 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.491+0000 7f3438c00640 10 client.0 ms_handle_connect on v2:172.21.15.72:3300/0 2024-04-26T21:58:18.494 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.491+0000 7f3438c00640 1 -- 172.21.15.72:0/1634681219 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f344003a030 con 0x558a037bfa40 2024-04-26T21:58:18.494 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.492+0000 7f3438c00640 1 -- 172.21.15.72:0/1634681219 <== mon.0 v2:172.21.15.72:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7f3440003a00 con 0x558a037bfa40 2024-04-26T21:58:18.494 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.492+0000 7f34589b5740 1 -- 172.21.15.72:0/1634681219 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f343c005020 con 0x558a037bfa40 2024-04-26T21:58:18.494 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.492+0000 7f3438c00640 1 -- 172.21.15.72:0/1634681219 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f344004e030 con 0x558a037bfa40 2024-04-26T21:58:18.494 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.492+0000 7f34589b5740 10 client.16206 fetch_fsmap learned FSMap version 27 2024-04-26T21:58:18.494 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.492+0000 7f34589b5740 1 -- 172.21.15.72:0/1634681219 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x558a034343f0 con 0x558a037bfa40 2024-04-26T21:58:18.494 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.492+0000 7f3438c00640 1 -- 172.21.15.72:0/1634681219 <== mon.0 v2:172.21.15.72:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f343c005020 con 0x558a037bfa40 2024-04-26T21:58:18.494 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.492+0000 7f3438c00640 1 -- 172.21.15.72:0/1634681219 <== mon.0 v2:172.21.15.72:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f3440035070 con 0x558a037bfa40 2024-04-26T21:58:18.495 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.493+0000 7f34589b5740 10 client.16206 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T21:58:18.495 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.493+0000 7f34589b5740 10 client.16206 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-26T21:58:18.495 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.493+0000 7f34589b5740 20 client.16206 populate_metadata read hostname 'smithi072' 2024-04-26T21:58:18.495 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.493+0000 7f34589b5740 1 --2- 172.21.15.72:0/1634681219 >> [v2:172.21.15.76:6836/1150678991,v1:172.21.15.76:6837/1150678991] conn(0x558a037dd350 0x558a037df770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:18.495 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.493+0000 7f34589b5740 4 client.16206 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:58:18.495 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.493+0000 7f34589b5740 1 -- 172.21.15.72:0/1634681219 --> [v2:172.21.15.76:6836/1150678991,v1:172.21.15.76:6837/1150678991] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558a03730880 con 0x558a037dd350 2024-04-26T21:58:18.495 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.493+0000 7f344e000640 1 --2- 172.21.15.72:0/1634681219 >> [v2:172.21.15.76:6836/1150678991,v1:172.21.15.76:6837/1150678991] conn(0x558a037dd350 0x558a037df770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:18.495 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.494+0000 7f344e000640 1 --2- 172.21.15.72:0/1634681219 >> [v2:172.21.15.76:6836/1150678991,v1:172.21.15.76:6837/1150678991] conn(0x558a037dd350 0x558a037df770 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a0e9e006b140c7a0 server_cookie=f0bc46d96eb19071 in_seq=0 out_seq=0 2024-04-26T21:58:18.495 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.494+0000 7f3438c00640 10 client.16206 ms_handle_connect on v2:172.21.15.76:6836/1150678991 2024-04-26T21:58:18.496 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.495+0000 7f3438c00640 1 -- 172.21.15.72:0/1634681219 <== mds.1 v2:172.21.15.76:6836/1150678991 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558a03730880 con 0x558a037dd350 2024-04-26T21:58:18.496 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.495+0000 7f3438c00640 10 client.16206 handle_command_reply: tid=0 2024-04-26T21:58:18.496 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.495+0000 7f34589b5740 1 client.16206 shutdown 2024-04-26T21:58:18.496 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.495+0000 7f34589b5740 1 -- 172.21.15.72:0/1634681219 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558a037bfa40 msgr2=0x558a037d1ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:18.496 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.495+0000 7f34589b5740 1 --2- 172.21.15.72:0/1634681219 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558a037bfa40 0x558a037d1ce0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f3440009230 tx=0x7f34400044f0 comp rx=0 tx=0).stop 2024-04-26T21:58:18.497 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.495+0000 7f34589b5740 1 -- 172.21.15.72:0/1634681219 shutdown_connections 2024-04-26T21:58:18.497 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.495+0000 7f34589b5740 1 --2- 172.21.15.72:0/1634681219 >> [v2:172.21.15.76:6836/1150678991,v1:172.21.15.76:6837/1150678991] conn(0x558a037dd350 0x558a037df770 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:18.497 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.495+0000 7f34589b5740 1 --2- 172.21.15.72:0/1634681219 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558a037c4bb0 0x558a037d8d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:18.497 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.495+0000 7f34589b5740 1 --2- 172.21.15.72:0/1634681219 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558a037bfa40 0x558a037d1ce0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:18.497 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.495+0000 7f34589b5740 1 --2- 172.21.15.72:0/1634681219 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558a037bf050 0x558a037d17a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:18.497 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.495+0000 7f34589b5740 1 -- 172.21.15.72:0/1634681219 >> 172.21.15.72:0/1634681219 conn(0x558a037ba6c0 msgr2=0x558a037b97b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:58:18.497 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.495+0000 7f3438c00640 0 client.16206 ms_handle_reset on v2:172.21.15.76:6836/1150678991 2024-04-26T21:58:18.497 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.496+0000 7f34589b5740 1 -- 172.21.15.72:0/1634681219 shutdown_connections 2024-04-26T21:58:18.497 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.496+0000 7f34589b5740 1 -- 172.21.15.72:0/1634681219 wait complete. 2024-04-26T21:58:18.497 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.496+0000 7f34589b5740 20 client.16206 trim_cache size 0 max 16384 2024-04-26T21:58:18.518 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.518+0000 7f34589b5740 1 Processor -- start 2024-04-26T21:58:18.518 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.518+0000 7f34589b5740 1 -- start start 2024-04-26T21:58:18.518 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.518+0000 7f34589b5740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558a037bf050 0x558a037c4300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:18.518 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.518+0000 7f34589b5740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558a037c4bb0 0x558a037d8ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:18.518 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.518+0000 7f34589b5740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558a037b9630 0x558a037c04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:18.519 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.518+0000 7f34589b5740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f34501a7b30 con 0x558a037bf050 2024-04-26T21:58:18.519 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.518+0000 7f34589b5740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f34501a9670 con 0x558a037c4bb0 2024-04-26T21:58:18.519 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.518+0000 7f34589b5740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f34501a8200 con 0x558a037b9630 2024-04-26T21:58:18.520 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.519+0000 7f344ea00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558a037c4bb0 0x558a037d8ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:18.520 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.519+0000 7f344e000640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558a037bf050 0x558a037c4300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:18.520 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.519+0000 7f344ea00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558a037c4bb0 0x558a037d8ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.72:48820/0 (socket says 172.21.15.72:48820) 2024-04-26T21:58:18.520 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.519+0000 7f344ea00640 1 -- 172.21.15.72:0/3216684578 learned_addr learned my addr 172.21.15.72:0/3216684578 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:58:18.520 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.519+0000 7f344e000640 1 --2- 172.21.15.72:0/3216684578 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558a037b9630 0x558a037c04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:18.520 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.519+0000 7f344ea00640 1 -- 172.21.15.72:0/3216684578 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558a037b9630 msgr2=0x558a037c04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:58:18.520 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.519+0000 7f344ea00640 1 --2- 172.21.15.72:0/3216684578 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558a037b9630 0x558a037c04b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:18.520 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.519+0000 7f344ea00640 1 -- 172.21.15.72:0/3216684578 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558a037bf050 msgr2=0x558a037c4300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:58:18.520 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.519+0000 7f344ea00640 1 --2- 172.21.15.72:0/3216684578 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558a037bf050 0x558a037c4300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:18.520 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.519+0000 7f344ea00640 1 -- 172.21.15.72:0/3216684578 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a034fe7d0 con 0x558a037c4bb0 2024-04-26T21:58:18.521 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.520+0000 7f344e000640 1 --2- 172.21.15.72:0/3216684578 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558a037bf050 0x558a037c4300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:58:18.521 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.520+0000 7f344ea00640 1 --2- 172.21.15.72:0/3216684578 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558a037c4bb0 0x558a037d8ce0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f3440036e10 tx=0x7f344002e800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2469725c7c02553 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:58:18.521 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.520+0000 7f343b400640 1 -- 172.21.15.72:0/3216684578 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3440000e60 con 0x558a037c4bb0 2024-04-26T21:58:18.521 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.521+0000 7f343b400640 1 -- 172.21.15.72:0/3216684578 <== mon.2 v2:172.21.15.76:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3440036980 con 0x558a037c4bb0 2024-04-26T21:58:18.521 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.521+0000 7f343b400640 1 -- 172.21.15.72:0/3216684578 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3440041c80 con 0x558a037c4bb0 2024-04-26T21:58:18.522 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.521+0000 7f34589b5740 1 -- 172.21.15.72:0/3216684578 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558a037c4bb0 msgr2=0x558a037d8ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:18.522 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.521+0000 7f34589b5740 1 --2- 172.21.15.72:0/3216684578 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558a037c4bb0 0x558a037d8ce0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f3440036e10 tx=0x7f344002e800 comp rx=0 tx=0).stop 2024-04-26T21:58:18.522 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.522+0000 7f34589b5740 1 -- 172.21.15.72:0/3216684578 shutdown_connections 2024-04-26T21:58:18.522 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.522+0000 7f34589b5740 1 --2- 172.21.15.72:0/3216684578 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558a037b9630 0x558a037c04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:18.522 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.522+0000 7f34589b5740 1 --2- 172.21.15.72:0/3216684578 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558a037c4bb0 0x558a037d8ce0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:18.522 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.522+0000 7f34589b5740 1 --2- 172.21.15.72:0/3216684578 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558a037bf050 0x558a037c4300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:18.522 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.522+0000 7f34589b5740 1 -- 172.21.15.72:0/3216684578 >> 172.21.15.72:0/3216684578 conn(0x558a037ba6c0 msgr2=0x558a037b8f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:58:18.523 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.522+0000 7f34589b5740 1 -- 172.21.15.72:0/3216684578 shutdown_connections 2024-04-26T21:58:18.523 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.522+0000 7f34589b5740 1 -- 172.21.15.72:0/3216684578 wait complete. 2024-04-26T21:58:18.523 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.523+0000 7f34589b5740 1 -- start start 2024-04-26T21:58:18.523 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.523+0000 7f34589b5740 1 Processor -- start 2024-04-26T21:58:18.523 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.523+0000 7f34589b5740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558a037b9630 0x558a037d6690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:18.524 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.523+0000 7f34589b5740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558a037bf050 0x558a037d6bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:18.524 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.524+0000 7f34589b5740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558a037c4bb0 0x558a037d7be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:18.524 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.524+0000 7f34589b5740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f34501a8fa0 con 0x558a037bf050 2024-04-26T21:58:18.524 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.524+0000 7f34589b5740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f34501a88d0 con 0x558a037c4bb0 2024-04-26T21:58:18.524 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.524+0000 7f34589b5740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f34501a5b10 con 0x558a037b9630 2024-04-26T21:58:18.524 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.524+0000 7f34589b5740 10 client.0 _ll_register_callbacks cb 0x7f3440009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:58:18.525 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.524+0000 7f344e000640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558a037c4bb0 0x558a037d7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:18.525 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.524+0000 7f344e000640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558a037c4bb0 0x558a037d7be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.72:48834/0 (socket says 172.21.15.72:48834) 2024-04-26T21:58:18.525 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.524+0000 7f344e000640 1 -- 172.21.15.72:0/4015363499 learned_addr learned my addr 172.21.15.72:0/4015363499 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:58:18.525 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.524+0000 7f344e000640 1 --2- 172.21.15.72:0/4015363499 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558a037b9630 0x558a037d6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:18.525 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.525+0000 7f344e000640 1 -- 172.21.15.72:0/4015363499 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558a037b9630 msgr2=0x558a037d6690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:58:18.525 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.525+0000 7f344e000640 1 --2- 172.21.15.72:0/4015363499 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558a037b9630 0x558a037d6690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:18.525 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.525+0000 7f344e000640 1 -- 172.21.15.72:0/4015363499 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558a037bf050 msgr2=0x558a037d6bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:58:18.525 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.525+0000 7f344e000640 1 --2- 172.21.15.72:0/4015363499 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558a037bf050 0x558a037d6bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:18.526 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.525+0000 7f344e000640 1 -- 172.21.15.72:0/4015363499 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a036c92a0 con 0x558a037c4bb0 2024-04-26T21:58:18.526 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.525+0000 7f344e000640 1 --2- 172.21.15.72:0/4015363499 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558a037c4bb0 0x558a037d7be0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f343c0058c0 tx=0x7f343c011f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2900320673a876bc server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:58:18.526 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.525+0000 7f3438c00640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-04-26T21:58:18.527 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.526+0000 7f3438c00640 1 -- 172.21.15.72:0/4015363499 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f343c01a040 con 0x558a037c4bb0 2024-04-26T21:58:18.527 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.526+0000 7f34589b5740 1 -- 172.21.15.72:0/4015363499 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3440036980 con 0x558a037c4bb0 2024-04-26T21:58:18.527 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.526+0000 7f3438c00640 1 -- 172.21.15.72:0/4015363499 <== mon.2 v2:172.21.15.76:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7f343c02dae0 con 0x558a037c4bb0 2024-04-26T21:58:18.527 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.526+0000 7f3438c00640 1 -- 172.21.15.72:0/4015363499 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f343c005600 con 0x558a037c4bb0 2024-04-26T21:58:18.527 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.526+0000 7f34589b5740 10 client.35426 fetch_fsmap learned FSMap version 27 2024-04-26T21:58:18.527 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.527+0000 7f34589b5740 1 -- 172.21.15.72:0/4015363499 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x558a038439a0 con 0x558a037c4bb0 2024-04-26T21:58:18.527 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.527+0000 7f3438c00640 1 -- 172.21.15.72:0/4015363499 <== mon.2 v2:172.21.15.76:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3440036980 con 0x558a037c4bb0 2024-04-26T21:58:18.528 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.527+0000 7f3438c00640 1 -- 172.21.15.72:0/4015363499 <== mon.2 v2:172.21.15.76:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f343c02dce0 con 0x558a037c4bb0 2024-04-26T21:58:18.528 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.527+0000 7f34589b5740 10 client.35426 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T21:58:18.528 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.527+0000 7f34589b5740 10 client.35426 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-26T21:58:18.528 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.527+0000 7f34589b5740 20 client.35426 populate_metadata read hostname 'smithi072' 2024-04-26T21:58:18.528 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.527+0000 7f34589b5740 1 --2- 172.21.15.72:0/4015363499 >> [v2:172.21.15.76:6836/1150678991,v1:172.21.15.76:6837/1150678991] conn(0x558a03844160 0x558a03846580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:18.528 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.527+0000 7f34589b5740 4 client.35426 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-26T21:58:18.528 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.527+0000 7f34589b5740 1 -- 172.21.15.72:0/4015363499 --> [v2:172.21.15.76:6836/1150678991,v1:172.21.15.76:6837/1150678991] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558a0372a6f0 con 0x558a03844160 2024-04-26T21:58:18.528 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.528+0000 7f344ea00640 1 --2- 172.21.15.72:0/4015363499 >> [v2:172.21.15.76:6836/1150678991,v1:172.21.15.76:6837/1150678991] conn(0x558a03844160 0x558a03846580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:18.529 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.529+0000 7f344ea00640 1 --2- 172.21.15.72:0/4015363499 >> [v2:172.21.15.76:6836/1150678991,v1:172.21.15.76:6837/1150678991] conn(0x558a03844160 0x558a03846580 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a945dc8c2e101f4a server_cookie=4e858e6f8a7f91df in_seq=0 out_seq=0 2024-04-26T21:58:18.530 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.529+0000 7f3438c00640 10 client.35426 ms_handle_connect on v2:172.21.15.76:6836/1150678991 2024-04-26T21:58:18.530 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.529+0000 7f3438c00640 1 -- 172.21.15.72:0/4015363499 <== mds.1 v2:172.21.15.76:6836/1150678991 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558a0372a6f0 con 0x558a03844160 2024-04-26T21:58:18.530 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.529+0000 7f3438c00640 10 client.35426 handle_command_reply: tid=0 2024-04-26T21:58:18.530 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.529+0000 7f34589b5740 1 client.35426 shutdown 2024-04-26T21:58:18.530 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.530+0000 7f34589b5740 1 -- 172.21.15.72:0/4015363499 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558a037c4bb0 msgr2=0x558a037d7be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:18.530 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.530+0000 7f34589b5740 1 --2- 172.21.15.72:0/4015363499 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558a037c4bb0 0x558a037d7be0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f343c0058c0 tx=0x7f343c011f90 comp rx=0 tx=0).stop 2024-04-26T21:58:18.530 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.530+0000 7f34589b5740 1 -- 172.21.15.72:0/4015363499 shutdown_connections 2024-04-26T21:58:18.530 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.530+0000 7f34589b5740 1 --2- 172.21.15.72:0/4015363499 >> [v2:172.21.15.76:6836/1150678991,v1:172.21.15.76:6837/1150678991] conn(0x558a03844160 0x558a03846580 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:18.530 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.530+0000 7f34589b5740 1 --2- 172.21.15.72:0/4015363499 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558a037c4bb0 0x558a037d7be0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:18.530 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.530+0000 7f34589b5740 1 --2- 172.21.15.72:0/4015363499 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558a037bf050 0x558a037d6bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:18.530 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.530+0000 7f34589b5740 1 --2- 172.21.15.72:0/4015363499 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558a037b9630 0x558a037d6690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:18.530 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.530+0000 7f34589b5740 1 -- 172.21.15.72:0/4015363499 >> 172.21.15.72:0/4015363499 conn(0x558a037ba6c0 msgr2=0x558a037bd6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:58:18.531 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.530+0000 7f3438c00640 0 client.35426 ms_handle_reset on v2:172.21.15.76:6836/1150678991 2024-04-26T21:58:18.531 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.531+0000 7f34589b5740 1 -- 172.21.15.72:0/4015363499 shutdown_connections 2024-04-26T21:58:18.531 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.531+0000 7f34589b5740 1 -- 172.21.15.72:0/4015363499 wait complete. 2024-04-26T21:58:18.531 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.532+0000 7f34589b5740 20 client.35426 trim_cache size 0 max 16384 2024-04-26T21:58:18.532 INFO:teuthology.orchestra.run.smithi072.stdout:[] 2024-04-26T21:58:18.532 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.532+0000 7f344fe00640 1 -- 172.21.15.72:0/34999851 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f34340a3d00 msgr2=0x7f34340a61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:18.532 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.532+0000 7f344fe00640 1 --2- 172.21.15.72:0/34999851 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f34340a3d00 0x7f34340a61c0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f344809cb80 tx=0x7f3448005b40 comp rx=0 tx=0).stop 2024-04-26T21:58:18.532 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.532+0000 7f344fe00640 1 -- 172.21.15.72:0/34999851 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f34501032f0 msgr2=0x7f345019dd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:18.532 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.532+0000 7f344fe00640 1 --2- 172.21.15.72:0/34999851 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f34501032f0 0x7f345019dd80 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f344800eaa0 tx=0x7f344800ef60 comp rx=0 tx=0).stop 2024-04-26T21:58:18.533 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.533+0000 7f344fe00640 1 -- 172.21.15.72:0/34999851 shutdown_connections 2024-04-26T21:58:18.533 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.533+0000 7f344fe00640 1 --2- 172.21.15.72:0/34999851 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f34501032f0 0x7f345019dd80 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:18.533 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.533+0000 7f344fe00640 1 --2- 172.21.15.72:0/34999851 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3450102930 0x7f3450196d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:18.533 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.533+0000 7f344fe00640 1 --2- 172.21.15.72:0/34999851 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f34340a3d00 0x7f34340a61c0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:18.533 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.533+0000 7f344fe00640 1 --2- 172.21.15.72:0/34999851 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3450101740 0x7f34501967d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:18.534 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.533+0000 7f344fe00640 1 -- 172.21.15.72:0/34999851 >> 172.21.15.72:0/34999851 conn(0x7f34500fccd0 msgr2=0x7f3450105d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:58:18.534 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.533+0000 7f344fe00640 1 -- 172.21.15.72:0/34999851 shutdown_connections 2024-04-26T21:58:18.534 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.533+0000 7f344fe00640 1 -- 172.21.15.72:0/34999851 wait complete. 2024-04-26T21:58:18.553 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-26T21:58:18.784 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.784+0000 7f58cbe00640 1 -- 172.21.15.72:0/3267622688 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f58cc104a70 msgr2=0x7f58cc104e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:18.785 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.784+0000 7f58cbe00640 1 --2- 172.21.15.72:0/3267622688 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f58cc104a70 0x7f58cc104e90 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f58bc009900 tx=0x7f58bc02a410 comp rx=0 tx=0).stop 2024-04-26T21:58:18.785 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.784+0000 7f58cbe00640 1 -- 172.21.15.72:0/3267622688 shutdown_connections 2024-04-26T21:58:18.785 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.784+0000 7f58cbe00640 1 --2- 172.21.15.72:0/3267622688 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f58cc106620 0x7f58cc10ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:18.785 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.784+0000 7f58cbe00640 1 --2- 172.21.15.72:0/3267622688 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f58cc105c60 0x7f58cc1060e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:18.785 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.784+0000 7f58cbe00640 1 --2- 172.21.15.72:0/3267622688 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f58cc104a70 0x7f58cc104e90 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:18.785 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.784+0000 7f58cbe00640 1 -- 172.21.15.72:0/3267622688 >> 172.21.15.72:0/3267622688 conn(0x7f58cc0fffe0 msgr2=0x7f58cc102420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:58:18.785 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.784+0000 7f58cbe00640 1 -- 172.21.15.72:0/3267622688 shutdown_connections 2024-04-26T21:58:18.785 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.784+0000 7f58cbe00640 1 -- 172.21.15.72:0/3267622688 wait complete. 2024-04-26T21:58:18.787 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.785+0000 7f58cbe00640 1 Processor -- start 2024-04-26T21:58:18.787 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.785+0000 7f58cbe00640 1 -- start start 2024-04-26T21:58:18.787 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.786+0000 7f58cbe00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f58cc104a70 0x7f58cc19f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:18.787 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.786+0000 7f58cbe00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f58cc105c60 0x7f58cc19f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:18.787 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.786+0000 7f58cbe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f58cc106620 0x7f58cc1a6600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:18.787 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.786+0000 7f58cbe00640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f58cc1104f0 con 0x7f58cc104a70 2024-04-26T21:58:18.787 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.786+0000 7f58cbe00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f58cc110370 con 0x7f58cc105c60 2024-04-26T21:58:18.787 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.786+0000 7f58cbe00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f58cc110670 con 0x7f58cc106620 2024-04-26T21:58:18.787 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.786+0000 7f58caa00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f58cc104a70 0x7f58cc19f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:18.787 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.786+0000 7f58caa00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f58cc104a70 0x7f58cc19f0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:46156/0 (socket says 172.21.15.72:46156) 2024-04-26T21:58:18.788 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.786+0000 7f58caa00640 1 -- 172.21.15.72:0/1479193791 learned_addr learned my addr 172.21.15.72:0/1479193791 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:58:18.788 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.786+0000 7f58ca000640 1 --2- 172.21.15.72:0/1479193791 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f58cc105c60 0x7f58cc19f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:18.788 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.787+0000 7f58cb400640 1 --2- 172.21.15.72:0/1479193791 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f58cc106620 0x7f58cc1a6600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:18.788 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.787+0000 7f58caa00640 1 -- 172.21.15.72:0/1479193791 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f58cc106620 msgr2=0x7f58cc1a6600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:18.788 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.787+0000 7f58caa00640 1 --2- 172.21.15.72:0/1479193791 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f58cc106620 0x7f58cc1a6600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:18.788 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.787+0000 7f58caa00640 1 -- 172.21.15.72:0/1479193791 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f58cc105c60 msgr2=0x7f58cc19f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:18.788 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.787+0000 7f58caa00640 1 --2- 172.21.15.72:0/1479193791 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f58cc105c60 0x7f58cc19f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:18.788 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.787+0000 7f58caa00640 1 -- 172.21.15.72:0/1479193791 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58cc1a6c70 con 0x7f58cc104a70 2024-04-26T21:58:18.788 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.787+0000 7f58cb400640 1 --2- 172.21.15.72:0/1479193791 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f58cc106620 0x7f58cc1a6600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:58:18.788 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.787+0000 7f58caa00640 1 --2- 172.21.15.72:0/1479193791 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f58cc104a70 0x7f58cc19f0a0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f58bc031460 tx=0x7f58bc003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:58:18.789 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.788+0000 7f58c3400640 1 -- 172.21.15.72:0/1479193791 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58bc03f070 con 0x7f58cc104a70 2024-04-26T21:58:18.789 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.788+0000 7f58c3400640 1 -- 172.21.15.72:0/1479193791 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f58bc031da0 con 0x7f58cc104a70 2024-04-26T21:58:18.789 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.788+0000 7f58c3400640 1 -- 172.21.15.72:0/1479193791 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58bc035030 con 0x7f58cc104a70 2024-04-26T21:58:18.789 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.788+0000 7f58cbe00640 1 -- 172.21.15.72:0/1479193791 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58cc1a6f00 con 0x7f58cc104a70 2024-04-26T21:58:18.789 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.788+0000 7f58cbe00640 1 -- 172.21.15.72:0/1479193791 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58cc1a7390 con 0x7f58cc104a70 2024-04-26T21:58:18.792 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.792+0000 7f58d45e6740 1 Processor -- start 2024-04-26T21:58:18.792 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.792+0000 7f58d45e6740 1 -- start start 2024-04-26T21:58:18.793 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.792+0000 7f58d45e6740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561e8085e050 0x561e8085e470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:18.793 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.792+0000 7f58d45e6740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561e8085ea40 0x561e80863540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:18.793 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.792+0000 7f58d45e6740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561e80863bb0 0x561e80866040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:18.793 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.792+0000 7f58d45e6740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x561e807c1c70 con 0x561e8085e050 2024-04-26T21:58:18.793 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.792+0000 7f58d45e6740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x561e8077a980 con 0x561e8085ea40 2024-04-26T21:58:18.793 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.792+0000 7f58d45e6740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x561e807bdc30 con 0x561e80863bb0 2024-04-26T21:58:18.793 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.793+0000 7f58c3400640 1 -- 172.21.15.72:0/1479193791 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f58bc042070 con 0x7f58cc104a70 2024-04-26T21:58:18.793 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.793+0000 7f58cb400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561e8085e050 0x561e8085e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:18.793 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.793+0000 7f58cb400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561e8085e050 0x561e8085e470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:46172/0 (socket says 172.21.15.72:46172) 2024-04-26T21:58:18.793 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.793+0000 7f58cb400640 1 -- 172.21.15.72:0/3994599012 learned_addr learned my addr 172.21.15.72:0/3994599012 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:58:18.794 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.793+0000 7f58caa00640 1 --2- 172.21.15.72:0/3994599012 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561e8085ea40 0x561e80863540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:18.794 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.794+0000 7f58cb400640 1 -- 172.21.15.72:0/3994599012 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561e80863bb0 msgr2=0x561e80866040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:58:18.794 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.794+0000 7f58cb400640 1 --2- 172.21.15.72:0/3994599012 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561e80863bb0 0x561e80866040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:18.794 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.794+0000 7f58cb400640 1 -- 172.21.15.72:0/3994599012 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561e8085ea40 msgr2=0x561e80863540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:58:18.795 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.794+0000 7f58cb400640 1 --2- 172.21.15.72:0/3994599012 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561e8085ea40 0x561e80863540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:18.795 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.794+0000 7f58cb400640 1 -- 172.21.15.72:0/3994599012 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e807527b0 con 0x561e8085e050 2024-04-26T21:58:18.795 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.794+0000 7f58cb400640 1 --2- 172.21.15.72:0/3994599012 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561e8085e050 0x561e8085e470 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f58c4007f30 tx=0x7f58c400c750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f349336fc87efa10 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:58:18.795 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.795+0000 7f58b7400640 1 -- 172.21.15.72:0/3994599012 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58c4014070 con 0x561e8085e050 2024-04-26T21:58:18.795 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.795+0000 7f58b7400640 1 -- 172.21.15.72:0/3994599012 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f58c400fce0 con 0x561e8085e050 2024-04-26T21:58:18.795 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.795+0000 7f58b7400640 1 -- 172.21.15.72:0/3994599012 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58c400ec00 con 0x561e8085e050 2024-04-26T21:58:18.795 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.795+0000 7f58c3400640 1 --2- 172.21.15.72:0/1479193791 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f58b00a3d40 0x7f58b00a6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:18.796 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.796+0000 7f58c3400640 1 -- 172.21.15.72:0/1479193791 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7f58bc0f0ec0 con 0x7f58cc104a70 2024-04-26T21:58:18.796 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.796+0000 7f58caa00640 1 --2- 172.21.15.72:0/1479193791 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f58b00a3d40 0x7f58b00a6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:18.796 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.796+0000 7f58d45e6740 1 -- 172.21.15.72:0/3994599012 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561e8085e050 msgr2=0x561e8085e470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:18.796 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.796+0000 7f58d45e6740 1 --2- 172.21.15.72:0/3994599012 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561e8085e050 0x561e8085e470 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f58c4007f30 tx=0x7f58c400c750 comp rx=0 tx=0).stop 2024-04-26T21:58:18.797 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.797+0000 7f58d45e6740 1 -- 172.21.15.72:0/3994599012 shutdown_connections 2024-04-26T21:58:18.797 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.797+0000 7f58d45e6740 1 --2- 172.21.15.72:0/3994599012 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561e80863bb0 0x561e80866040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:18.797 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.797+0000 7f58d45e6740 1 --2- 172.21.15.72:0/3994599012 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561e8085ea40 0x561e80863540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:18.798 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.797+0000 7f58d45e6740 1 --2- 172.21.15.72:0/3994599012 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561e8085e050 0x561e8085e470 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:18.798 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.797+0000 7f58d45e6740 1 -- 172.21.15.72:0/3994599012 >> 172.21.15.72:0/3994599012 conn(0x561e808596c0 msgr2=0x561e80859ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:58:18.798 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.797+0000 7f58caa00640 1 --2- 172.21.15.72:0/1479193791 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f58b00a3d40 0x7f58b00a6200 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x561e8085ee40 tx=0x7f58bc02a410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:58:18.798 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.797+0000 7f58d45e6740 1 -- 172.21.15.72:0/3994599012 shutdown_connections 2024-04-26T21:58:18.798 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.797+0000 7f58d45e6740 1 -- 172.21.15.72:0/3994599012 wait complete. 2024-04-26T21:58:18.798 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.798+0000 7f58d45e6740 1 -- start start 2024-04-26T21:58:18.798 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.798+0000 7f58d45e6740 1 Processor -- start 2024-04-26T21:58:18.799 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.799+0000 7f58d45e6740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561e8085e050 0x561e80867fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:18.799 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.799+0000 7f58d45e6740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561e8085ea40 0x561e808684e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:18.799 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.799+0000 7f58d45e6740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561e80863bb0 0x561e8086f520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:18.799 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.799+0000 7f58d45e6740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x561e807c0f80 con 0x561e80863bb0 2024-04-26T21:58:18.799 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.799+0000 7f58d45e6740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x561e8076e260 con 0x561e8085ea40 2024-04-26T21:58:18.800 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.799+0000 7f58d45e6740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x561e807c62a0 con 0x561e8085e050 2024-04-26T21:58:18.800 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.799+0000 7f58d45e6740 10 client.0 _ll_register_callbacks cb 0x561e807b8960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:58:18.800 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.799+0000 7f58ca000640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561e8085ea40 0x561e808684e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:18.800 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.799+0000 7f58cb400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561e80863bb0 0x561e8086f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:18.800 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.799+0000 7f58ca000640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561e8085ea40 0x561e808684e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.72:48850/0 (socket says 172.21.15.72:48850) 2024-04-26T21:58:18.800 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.799+0000 7f58ca000640 1 -- 172.21.15.72:0/259830814 learned_addr learned my addr 172.21.15.72:0/259830814 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:58:18.801 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.800+0000 7f58cb400640 1 --2- 172.21.15.72:0/259830814 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561e8085e050 0x561e80867fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:18.801 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.800+0000 7f58cb400640 1 -- 172.21.15.72:0/259830814 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561e8085e050 msgr2=0x561e80867fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:58:18.801 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.800+0000 7f58cb400640 1 --2- 172.21.15.72:0/259830814 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561e8085e050 0x561e80867fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:18.801 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.800+0000 7f58cb400640 1 -- 172.21.15.72:0/259830814 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561e8085ea40 msgr2=0x561e808684e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:58:18.801 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.800+0000 7f58cb400640 1 --2- 172.21.15.72:0/259830814 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561e8085ea40 0x561e808684e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:18.802 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.800+0000 7f58cb400640 1 -- 172.21.15.72:0/259830814 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e807501c0 con 0x561e80863bb0 2024-04-26T21:58:18.802 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.800+0000 7f58ca000640 1 --2- 172.21.15.72:0/259830814 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561e8085ea40 0x561e808684e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:58:18.802 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.800+0000 7f58cb400640 1 --2- 172.21.15.72:0/259830814 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561e80863bb0 0x561e8086f520 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f58c4007f30 tx=0x7f58c4004b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9366ef9bc86079d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:58:18.802 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.800+0000 7f58abe00640 10 client.0 ms_handle_connect on v2:172.21.15.72:3300/0 2024-04-26T21:58:18.802 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.801+0000 7f58abe00640 1 -- 172.21.15.72:0/259830814 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58c4014050 con 0x561e80863bb0 2024-04-26T21:58:18.802 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.801+0000 7f58abe00640 1 -- 172.21.15.72:0/259830814 <== mon.0 v2:172.21.15.72:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7f58c400f860 con 0x561e80863bb0 2024-04-26T21:58:18.802 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.801+0000 7f58d45e6740 1 -- 172.21.15.72:0/259830814 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f58c400fce0 con 0x561e80863bb0 2024-04-26T21:58:18.802 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.801+0000 7f58abe00640 1 -- 172.21.15.72:0/259830814 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f58c400fa80 con 0x561e80863bb0 2024-04-26T21:58:18.802 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.801+0000 7f58d45e6740 10 client.16221 fetch_fsmap learned FSMap version 27 2024-04-26T21:58:18.802 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.801+0000 7f58d45e6740 1 -- 172.21.15.72:0/259830814 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x561e804d33f0 con 0x561e80863bb0 2024-04-26T21:58:18.803 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.802+0000 7f58abe00640 1 -- 172.21.15.72:0/259830814 <== mon.0 v2:172.21.15.72:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f58c400fce0 con 0x561e80863bb0 2024-04-26T21:58:18.803 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.802+0000 7f58abe00640 1 -- 172.21.15.72:0/259830814 <== mon.0 v2:172.21.15.72:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f58c4019cc0 con 0x561e80863bb0 2024-04-26T21:58:18.803 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.802+0000 7f58d45e6740 10 client.16221 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T21:58:18.803 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.802+0000 7f58d45e6740 10 client.16221 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:58:18.803 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.802+0000 7f58d45e6740 20 client.16221 populate_metadata read hostname 'smithi072' 2024-04-26T21:58:18.803 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.802+0000 7f58d45e6740 1 --2- 172.21.15.72:0/259830814 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x561e80873840 0x561e80875c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:18.803 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.802+0000 7f58d45e6740 4 client.16221 mds_command: new command op to 34517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:58:18.804 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.802+0000 7f58d45e6740 1 -- 172.21.15.72:0/259830814 --> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561e807cf880 con 0x561e80873840 2024-04-26T21:58:18.804 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.803+0000 7f58ca000640 1 --2- 172.21.15.72:0/259830814 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x561e80873840 0x561e80875c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:18.804 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.804+0000 7f58ca000640 1 --2- 172.21.15.72:0/259830814 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x561e80873840 0x561e80875c30 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d760e2bed60cb5a0 server_cookie=4a47156e5885103b in_seq=0 out_seq=0 2024-04-26T21:58:18.804 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.804+0000 7f58abe00640 10 client.16221 ms_handle_connect on v2:172.21.15.76:6838/327145636 2024-04-26T21:58:18.805 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.805+0000 7f58abe00640 1 -- 172.21.15.72:0/259830814 <== mds.0 v2:172.21.15.76:6838/327145636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561e807cf880 con 0x561e80873840 2024-04-26T21:58:18.805 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.805+0000 7f58abe00640 10 client.16221 handle_command_reply: tid=0 2024-04-26T21:58:18.805 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.805+0000 7f58d45e6740 1 client.16221 shutdown 2024-04-26T21:58:18.806 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.805+0000 7f58d45e6740 1 -- 172.21.15.72:0/259830814 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561e80863bb0 msgr2=0x561e8086f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:18.806 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.805+0000 7f58d45e6740 1 --2- 172.21.15.72:0/259830814 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561e80863bb0 0x561e8086f520 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f58c4007f30 tx=0x7f58c4004b50 comp rx=0 tx=0).stop 2024-04-26T21:58:18.806 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.806+0000 7f58d45e6740 1 -- 172.21.15.72:0/259830814 shutdown_connections 2024-04-26T21:58:18.806 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.806+0000 7f58d45e6740 1 --2- 172.21.15.72:0/259830814 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x561e80873840 0x561e80875c30 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:18.806 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.806+0000 7f58d45e6740 1 --2- 172.21.15.72:0/259830814 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561e80863bb0 0x561e8086f520 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:18.806 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.806+0000 7f58d45e6740 1 --2- 172.21.15.72:0/259830814 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561e8085ea40 0x561e808684e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:18.806 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.806+0000 7f58d45e6740 1 --2- 172.21.15.72:0/259830814 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561e8085e050 0x561e80867fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:18.806 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.806+0000 7f58abe00640 0 client.16221 ms_handle_reset on v2:172.21.15.76:6838/327145636 2024-04-26T21:58:18.806 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.806+0000 7f58d45e6740 1 -- 172.21.15.72:0/259830814 >> 172.21.15.72:0/259830814 conn(0x561e808596c0 msgr2=0x561e808587b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:58:18.806 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.806+0000 7f58d45e6740 1 -- 172.21.15.72:0/259830814 shutdown_connections 2024-04-26T21:58:18.806 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.806+0000 7f58d45e6740 1 -- 172.21.15.72:0/259830814 wait complete. 2024-04-26T21:58:18.806 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.806+0000 7f58d45e6740 20 client.16221 trim_cache size 0 max 16384 2024-04-26T21:58:18.826 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.826+0000 7f58d45e6740 1 Processor -- start 2024-04-26T21:58:18.826 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.826+0000 7f58d45e6740 1 -- start start 2024-04-26T21:58:18.826 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.826+0000 7f58d45e6740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561e8085e050 0x561e8085cf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:18.826 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.826+0000 7f58d45e6740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561e8085ea40 0x561e8086f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:18.826 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.826+0000 7f58d45e6740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561e80863bb0 0x561e80854640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:18.826 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.826+0000 7f58d45e6740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f58cc1b03c0 con 0x561e80863bb0 2024-04-26T21:58:18.826 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.826+0000 7f58d45e6740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f58cc1b1f00 con 0x561e8085e050 2024-04-26T21:58:18.826 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.826+0000 7f58d45e6740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f58cc1b0a90 con 0x561e8085ea40 2024-04-26T21:58:18.827 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.826+0000 7f58cb400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561e80863bb0 0x561e80854640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:18.827 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.827+0000 7f58ca000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561e8085ea40 0x561e8086f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:18.827 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.827+0000 7f58cb400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561e80863bb0 0x561e80854640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:46198/0 (socket says 172.21.15.72:46198) 2024-04-26T21:58:18.827 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.827+0000 7f58cb400640 1 -- 172.21.15.72:0/227200687 learned_addr learned my addr 172.21.15.72:0/227200687 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:58:18.827 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.827+0000 7f58cb400640 1 --2- 172.21.15.72:0/227200687 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561e8085e050 0x561e8085cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:18.828 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.827+0000 7f58cb400640 1 -- 172.21.15.72:0/227200687 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561e8085ea40 msgr2=0x561e8086f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:58:18.828 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.827+0000 7f58cb400640 1 --2- 172.21.15.72:0/227200687 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561e8085ea40 0x561e8086f4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:18.828 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.827+0000 7f58cb400640 1 -- 172.21.15.72:0/227200687 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561e8085e050 msgr2=0x561e8085cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:58:18.828 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.827+0000 7f58cb400640 1 --2- 172.21.15.72:0/227200687 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561e8085e050 0x561e8085cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:18.828 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.827+0000 7f58cb400640 1 -- 172.21.15.72:0/227200687 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e80875fa0 con 0x561e80863bb0 2024-04-26T21:58:18.828 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.828+0000 7f58cb400640 1 --2- 172.21.15.72:0/227200687 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561e80863bb0 0x561e80854640 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f58c400ccd0 tx=0x7f58c400feb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f14cc66409ba4e8e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:58:18.829 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.829+0000 7f58b6a00640 1 -- 172.21.15.72:0/227200687 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58c4014070 con 0x561e80863bb0 2024-04-26T21:58:18.829 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.829+0000 7f58b6a00640 1 -- 172.21.15.72:0/227200687 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f58c400ed20 con 0x561e80863bb0 2024-04-26T21:58:18.829 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.829+0000 7f58b6a00640 1 -- 172.21.15.72:0/227200687 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58c4027690 con 0x561e80863bb0 2024-04-26T21:58:18.830 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.830+0000 7f58d45e6740 1 -- 172.21.15.72:0/227200687 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561e80863bb0 msgr2=0x561e80854640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:18.830 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.830+0000 7f58d45e6740 1 --2- 172.21.15.72:0/227200687 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561e80863bb0 0x561e80854640 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f58c400ccd0 tx=0x7f58c400feb0 comp rx=0 tx=0).stop 2024-04-26T21:58:18.831 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.830+0000 7f58d45e6740 1 -- 172.21.15.72:0/227200687 shutdown_connections 2024-04-26T21:58:18.831 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.830+0000 7f58d45e6740 1 --2- 172.21.15.72:0/227200687 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561e80863bb0 0x561e80854640 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:18.831 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.830+0000 7f58d45e6740 1 --2- 172.21.15.72:0/227200687 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561e8085ea40 0x561e8086f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:18.831 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.830+0000 7f58d45e6740 1 --2- 172.21.15.72:0/227200687 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561e8085e050 0x561e8085cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:18.831 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.830+0000 7f58d45e6740 1 -- 172.21.15.72:0/227200687 >> 172.21.15.72:0/227200687 conn(0x561e808596c0 msgr2=0x561e8085f4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:58:18.831 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.831+0000 7f58d45e6740 1 -- 172.21.15.72:0/227200687 shutdown_connections 2024-04-26T21:58:18.831 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.831+0000 7f58d45e6740 1 -- 172.21.15.72:0/227200687 wait complete. 2024-04-26T21:58:18.831 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.831+0000 7f58d45e6740 1 -- start start 2024-04-26T21:58:18.831 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.831+0000 7f58d45e6740 1 Processor -- start 2024-04-26T21:58:18.832 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.832+0000 7f58d45e6740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561e8085e050 0x561e8086ca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:18.832 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.832+0000 7f58d45e6740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561e8085ea40 0x561e8086cf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:18.832 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.832+0000 7f58d45e6740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561e80863bb0 0x561e8086dee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:18.832 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.832+0000 7f58d45e6740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f58cc1b1830 con 0x561e8085ea40 2024-04-26T21:58:18.832 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.832+0000 7f58d45e6740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f58cc1b1160 con 0x561e80863bb0 2024-04-26T21:58:18.832 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.832+0000 7f58d45e6740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f58cc1ae3a0 con 0x561e8085e050 2024-04-26T21:58:18.832 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.832+0000 7f58d45e6740 10 client.0 _ll_register_callbacks cb 0x7f58c4007d80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:58:18.833 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.833+0000 7f58cb400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561e8085e050 0x561e8086ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:18.833 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.833+0000 7f58ca000640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561e8085ea40 0x561e8086cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:18.833 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.833+0000 7f58cb400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561e8085e050 0x561e8086ca00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.72:56014/0 (socket says 172.21.15.72:56014) 2024-04-26T21:58:18.833 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.833+0000 7f58cb400640 1 -- 172.21.15.72:0/3800908550 learned_addr learned my addr 172.21.15.72:0/3800908550 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:58:18.833 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.833+0000 7f58cb400640 1 --2- 172.21.15.72:0/3800908550 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561e80863bb0 0x561e8086dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:18.834 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.834+0000 7f58cb400640 1 -- 172.21.15.72:0/3800908550 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561e80863bb0 msgr2=0x561e8086dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:58:18.834 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.834+0000 7f58cb400640 1 --2- 172.21.15.72:0/3800908550 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561e80863bb0 0x561e8086dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:18.834 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.834+0000 7f58cb400640 1 -- 172.21.15.72:0/3800908550 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561e8085ea40 msgr2=0x561e8086cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:58:18.834 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.834+0000 7f58cb400640 1 --2- 172.21.15.72:0/3800908550 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561e8085ea40 0x561e8086cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:18.835 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.834+0000 7f58cb400640 1 -- 172.21.15.72:0/3800908550 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e8059d7d0 con 0x561e8085e050 2024-04-26T21:58:18.835 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.834+0000 7f58ca000640 1 --2- 172.21.15.72:0/3800908550 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561e8085ea40 0x561e8086cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:58:18.835 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.834+0000 7f58cb400640 1 --2- 172.21.15.72:0/3800908550 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561e8085e050 0x561e8086ca00 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f58c4007720 tx=0x7f58c4004b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ea93d23e73548452 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:58:18.835 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.834+0000 7f58abe00640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-04-26T21:58:18.836 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.835+0000 7f58abe00640 1 -- 172.21.15.72:0/3800908550 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58c4014030 con 0x561e8085e050 2024-04-26T21:58:18.836 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.835+0000 7f58d45e6740 1 -- 172.21.15.72:0/3800908550 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f58c400ed20 con 0x561e8085e050 2024-04-26T21:58:18.836 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.835+0000 7f58abe00640 1 -- 172.21.15.72:0/3800908550 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7f58c4027af0 con 0x561e8085e050 2024-04-26T21:58:18.836 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.836+0000 7f58abe00640 1 -- 172.21.15.72:0/3800908550 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f58c4027cf0 con 0x561e8085e050 2024-04-26T21:58:18.836 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.836+0000 7f58d45e6740 10 client.25366 fetch_fsmap learned FSMap version 27 2024-04-26T21:58:18.836 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.836+0000 7f58d45e6740 1 -- 172.21.15.72:0/3800908550 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x561e807682a0 con 0x561e8085e050 2024-04-26T21:58:18.837 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.837+0000 7f58abe00640 1 -- 172.21.15.72:0/3800908550 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f58c400ed20 con 0x561e8085e050 2024-04-26T21:58:18.837 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.837+0000 7f58abe00640 1 -- 172.21.15.72:0/3800908550 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f58c4019450 con 0x561e8085e050 2024-04-26T21:58:18.837 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.837+0000 7f58d45e6740 10 client.25366 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T21:58:18.837 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.837+0000 7f58d45e6740 10 client.25366 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:58:18.837 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.837+0000 7f58d45e6740 20 client.25366 populate_metadata read hostname 'smithi072' 2024-04-26T21:58:18.837 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.837+0000 7f58d45e6740 1 --2- 172.21.15.72:0/3800908550 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x561e808d9f70 0x561e808dc390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:18.837 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.837+0000 7f58d45e6740 4 client.25366 mds_command: new command op to 34517 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-26T21:58:18.838 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.837+0000 7f58d45e6740 1 -- 172.21.15.72:0/3800908550 --> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561e807c96f0 con 0x561e808d9f70 2024-04-26T21:58:18.838 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.838+0000 7f58ca000640 1 --2- 172.21.15.72:0/3800908550 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x561e808d9f70 0x561e808dc390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:18.839 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.839+0000 7f58ca000640 1 --2- 172.21.15.72:0/3800908550 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x561e808d9f70 0x561e808dc390 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=92579f1923c029e1 server_cookie=3434285dc7bc7e05 in_seq=0 out_seq=0 2024-04-26T21:58:18.839 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.839+0000 7f58abe00640 10 client.25366 ms_handle_connect on v2:172.21.15.76:6838/327145636 2024-04-26T21:58:18.840 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.840+0000 7f58abe00640 1 -- 172.21.15.72:0/3800908550 <== mds.0 v2:172.21.15.76:6838/327145636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561e807c96f0 con 0x561e808d9f70 2024-04-26T21:58:18.840 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.840+0000 7f58abe00640 10 client.25366 handle_command_reply: tid=0 2024-04-26T21:58:18.841 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.840+0000 7f58d45e6740 1 client.25366 shutdown 2024-04-26T21:58:18.841 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.840+0000 7f58d45e6740 1 -- 172.21.15.72:0/3800908550 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561e8085e050 msgr2=0x561e8086ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:18.841 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.840+0000 7f58d45e6740 1 --2- 172.21.15.72:0/3800908550 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561e8085e050 0x561e8086ca00 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f58c4007720 tx=0x7f58c4004b50 comp rx=0 tx=0).stop 2024-04-26T21:58:18.841 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.840+0000 7f58d45e6740 1 -- 172.21.15.72:0/3800908550 shutdown_connections 2024-04-26T21:58:18.841 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.840+0000 7f58d45e6740 1 --2- 172.21.15.72:0/3800908550 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x561e808d9f70 0x561e808dc390 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:18.841 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.840+0000 7f58d45e6740 1 --2- 172.21.15.72:0/3800908550 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561e80863bb0 0x561e8086dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:18.841 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.840+0000 7f58d45e6740 1 --2- 172.21.15.72:0/3800908550 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561e8085ea40 0x561e8086cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:18.841 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.840+0000 7f58d45e6740 1 --2- 172.21.15.72:0/3800908550 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561e8085e050 0x561e8086ca00 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:18.841 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.840+0000 7f58d45e6740 1 -- 172.21.15.72:0/3800908550 >> 172.21.15.72:0/3800908550 conn(0x561e808596c0 msgr2=0x561e8085cda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:58:18.841 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.840+0000 7f58abe00640 0 client.25366 ms_handle_reset on v2:172.21.15.76:6838/327145636 2024-04-26T21:58:18.841 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.841+0000 7f58d45e6740 1 -- 172.21.15.72:0/3800908550 shutdown_connections 2024-04-26T21:58:18.841 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.841+0000 7f58d45e6740 1 -- 172.21.15.72:0/3800908550 wait complete. 2024-04-26T21:58:18.842 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.841+0000 7f58d45e6740 20 client.25366 trim_cache size 0 max 16384 2024-04-26T21:58:18.842 INFO:teuthology.orchestra.run.smithi072.stdout:[] 2024-04-26T21:58:18.843 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.842+0000 7f58cbe00640 1 -- 172.21.15.72:0/1479193791 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f58b00a3d40 msgr2=0x7f58b00a6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:18.843 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.842+0000 7f58cbe00640 1 --2- 172.21.15.72:0/1479193791 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f58b00a3d40 0x7f58b00a6200 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x561e8085ee40 tx=0x7f58bc02a410 comp rx=0 tx=0).stop 2024-04-26T21:58:18.843 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.842+0000 7f58cbe00640 1 -- 172.21.15.72:0/1479193791 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f58cc104a70 msgr2=0x7f58cc19f0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:18.843 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.842+0000 7f58cbe00640 1 --2- 172.21.15.72:0/1479193791 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f58cc104a70 0x7f58cc19f0a0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f58bc031460 tx=0x7f58bc003c80 comp rx=0 tx=0).stop 2024-04-26T21:58:18.843 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.842+0000 7f58cbe00640 1 -- 172.21.15.72:0/1479193791 shutdown_connections 2024-04-26T21:58:18.843 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.842+0000 7f58cbe00640 1 --2- 172.21.15.72:0/1479193791 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f58b00a3d40 0x7f58b00a6200 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:18.843 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.842+0000 7f58cbe00640 1 --2- 172.21.15.72:0/1479193791 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f58cc106620 0x7f58cc1a6600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:18.843 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.842+0000 7f58cbe00640 1 --2- 172.21.15.72:0/1479193791 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f58cc105c60 0x7f58cc19f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:18.843 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.842+0000 7f58cbe00640 1 --2- 172.21.15.72:0/1479193791 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f58cc104a70 0x7f58cc19f0a0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:18.843 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.842+0000 7f58cbe00640 1 -- 172.21.15.72:0/1479193791 >> 172.21.15.72:0/1479193791 conn(0x7f58cc0fffe0 msgr2=0x7f58cc078d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:58:18.844 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.844+0000 7f58cbe00640 1 -- 172.21.15.72:0/1479193791 shutdown_connections 2024-04-26T21:58:18.844 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:18.844+0000 7f58cbe00640 1 -- 172.21.15.72:0/1479193791 wait complete. 2024-04-26T21:58:19.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:58:18 smithi076 ceph-mon[31823]: pgmap v535: 97 pgs: 97 active+clean; 258 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 MiB/s wr, 2.61k op/s 2024-04-26T21:58:19.321 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:58:18 smithi072 ceph-mon[28138]: pgmap v535: 97 pgs: 97 active+clean; 258 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 MiB/s wr, 2.61k op/s 2024-04-26T21:58:19.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:58:18 smithi164 ceph-mon[31916]: pgmap v535: 97 pgs: 97 active+clean; 258 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 MiB/s wr, 2.61k op/s 2024-04-26T21:58:19.867 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-26T21:58:19.867 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:58:19.868 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-26T21:58:20.123 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.123+0000 7f3007c00640 1 -- 172.21.15.72:0/1503579644 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3000104bb0 msgr2=0x7f3000104fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:20.123 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.123+0000 7f3007c00640 1 --2- 172.21.15.72:0/1503579644 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3000104bb0 0x7f3000104fd0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f2ff8009ca0 tx=0x7f2ff802a8c0 comp rx=0 tx=0).stop 2024-04-26T21:58:20.123 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.123+0000 7f3007c00640 1 -- 172.21.15.72:0/1503579644 shutdown_connections 2024-04-26T21:58:20.123 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.123+0000 7f3007c00640 1 --2- 172.21.15.72:0/1503579644 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3000106760 0x7f300010cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:20.123 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.123+0000 7f3007c00640 1 --2- 172.21.15.72:0/1503579644 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3000105da0 0x7f3000106220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:20.123 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.123+0000 7f3007c00640 1 --2- 172.21.15.72:0/1503579644 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3000104bb0 0x7f3000104fd0 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:20.123 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.123+0000 7f3007c00640 1 -- 172.21.15.72:0/1503579644 >> 172.21.15.72:0/1503579644 conn(0x7f3000100120 msgr2=0x7f3000102560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:58:20.123 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.123+0000 7f3007c00640 1 -- 172.21.15.72:0/1503579644 shutdown_connections 2024-04-26T21:58:20.124 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.123+0000 7f3007c00640 1 -- 172.21.15.72:0/1503579644 wait complete. 2024-04-26T21:58:20.125 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.124+0000 7f3007c00640 1 Processor -- start 2024-04-26T21:58:20.125 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.124+0000 7f3007c00640 1 -- start start 2024-04-26T21:58:20.125 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.124+0000 7f3007c00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3000104bb0 0x7f300019adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:20.125 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.124+0000 7f3007c00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3000105da0 0x7f300019b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:20.125 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.125+0000 7f3007c00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3000106760 0x7f30001a2330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:20.125 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.125+0000 7f3007c00640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f3000110630 con 0x7f3000105da0 2024-04-26T21:58:20.125 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.125+0000 7f3007c00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f30001104b0 con 0x7f3000104bb0 2024-04-26T21:58:20.125 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.125+0000 7f3007c00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f30001107b0 con 0x7f3000106760 2024-04-26T21:58:20.125 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.125+0000 7f3005200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3000104bb0 0x7f300019adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:20.126 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.125+0000 7f3005200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3000104bb0 0x7f300019adc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.72:48898/0 (socket says 172.21.15.72:48898) 2024-04-26T21:58:20.126 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.125+0000 7f3005200640 1 -- 172.21.15.72:0/2356902837 learned_addr learned my addr 172.21.15.72:0/2356902837 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:58:20.127 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.125+0000 7f2fffe00640 1 --2- 172.21.15.72:0/2356902837 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3000105da0 0x7f300019b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:20.127 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.125+0000 7f3005c00640 1 --2- 172.21.15.72:0/2356902837 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3000106760 0x7f30001a2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:20.127 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.126+0000 7f2fffe00640 1 -- 172.21.15.72:0/2356902837 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3000106760 msgr2=0x7f30001a2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:20.127 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.126+0000 7f2fffe00640 1 --2- 172.21.15.72:0/2356902837 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3000106760 0x7f30001a2330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:20.127 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.126+0000 7f2fffe00640 1 -- 172.21.15.72:0/2356902837 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3000104bb0 msgr2=0x7f300019adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:20.127 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.126+0000 7f2fffe00640 1 --2- 172.21.15.72:0/2356902837 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3000104bb0 0x7f300019adc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:20.127 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.126+0000 7f2fffe00640 1 -- 172.21.15.72:0/2356902837 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30001a29d0 con 0x7f3000105da0 2024-04-26T21:58:20.128 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.126+0000 7f3005c00640 1 --2- 172.21.15.72:0/2356902837 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3000106760 0x7f30001a2330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:58:20.128 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.126+0000 7f2fffe00640 1 --2- 172.21.15.72:0/2356902837 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3000105da0 0x7f300019b300 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f2fe8005fd0 tx=0x7f2fe800dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:58:20.128 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.127+0000 7f2ffd600640 1 -- 172.21.15.72:0/2356902837 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fe8009c30 con 0x7f3000105da0 2024-04-26T21:58:20.128 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.127+0000 7f2ffd600640 1 -- 172.21.15.72:0/2356902837 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2fe80044c0 con 0x7f3000105da0 2024-04-26T21:58:20.128 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.127+0000 7f3007c00640 1 -- 172.21.15.72:0/2356902837 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30001a2cc0 con 0x7f3000105da0 2024-04-26T21:58:20.128 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.127+0000 7f2ffd600640 1 -- 172.21.15.72:0/2356902837 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fe8009c30 con 0x7f3000105da0 2024-04-26T21:58:20.128 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.127+0000 7f3007c00640 1 -- 172.21.15.72:0/2356902837 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30001a31c0 con 0x7f3000105da0 2024-04-26T21:58:20.131 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.131+0000 7f2ffd600640 1 -- 172.21.15.72:0/2356902837 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f2fe8003b70 con 0x7f3000105da0 2024-04-26T21:58:20.131 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.132+0000 7f300a9f0740 1 Processor -- start 2024-04-26T21:58:20.132 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.132+0000 7f300a9f0740 1 -- start start 2024-04-26T21:58:20.132 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.132+0000 7f300a9f0740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f85f660070 0x55f85f660490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:20.132 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.132+0000 7f300a9f0740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f85f660a60 0x55f85f665560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:20.133 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.132+0000 7f300a9f0740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f85f665bd0 0x55f85f668060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:20.133 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.132+0000 7f300a9f0740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55f85f5bc980 con 0x55f85f665bd0 2024-04-26T21:58:20.133 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.132+0000 7f300a9f0740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55f85f559220 con 0x55f85f660a60 2024-04-26T21:58:20.133 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.132+0000 7f300a9f0740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55f85f5b8940 con 0x55f85f660070 2024-04-26T21:58:20.135 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.133+0000 7f2ffd600640 1 --2- 172.21.15.72:0/2356902837 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f2fe40a3e40 0x7f2fe40a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:20.135 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.133+0000 7f2ffd600640 1 -- 172.21.15.72:0/2356902837 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7f2fe80d01a0 con 0x7f3000105da0 2024-04-26T21:58:20.135 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.133+0000 7f3005200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f85f660a60 0x55f85f665560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:20.135 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.133+0000 7f2fffe00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f85f665bd0 0x55f85f668060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:20.135 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.133+0000 7f3005200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f85f660a60 0x55f85f665560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.72:48908/0 (socket says 172.21.15.72:48908) 2024-04-26T21:58:20.135 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.133+0000 7f3005200640 1 -- 172.21.15.72:0/3820070001 learned_addr learned my addr 172.21.15.72:0/3820070001 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:58:20.135 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.133+0000 7f3005c00640 1 --2- 172.21.15.72:0/3820070001 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f85f660070 0x55f85f660490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:20.135 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.133+0000 7f3005200640 1 -- 172.21.15.72:0/3820070001 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f85f660070 msgr2=0x55f85f660490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:58:20.135 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.133+0000 7f3005200640 1 --2- 172.21.15.72:0/3820070001 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f85f660070 0x55f85f660490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:20.135 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.133+0000 7f3005200640 1 -- 172.21.15.72:0/3820070001 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f85f665bd0 msgr2=0x55f85f668060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:58:20.135 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.133+0000 7f3005200640 1 --2- 172.21.15.72:0/3820070001 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f85f665bd0 0x55f85f668060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:20.136 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.133+0000 7f3005200640 1 -- 172.21.15.72:0/3820070001 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f85f3967e0 con 0x55f85f660a60 2024-04-26T21:58:20.136 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.134+0000 7f3005200640 1 --2- 172.21.15.72:0/3820070001 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f85f660a60 0x55f85f665560 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f2ff8031500 tx=0x7f2ff8009a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=acfd3c42970c315b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:58:20.136 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.134+0000 7f3005c00640 1 --2- 172.21.15.72:0/2356902837 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f2fe40a3e40 0x7f2fe40a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:20.136 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.134+0000 7f2ff6000640 1 -- 172.21.15.72:0/3820070001 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ff8031c80 con 0x55f85f660a60 2024-04-26T21:58:20.136 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.134+0000 7f2ff6000640 1 -- 172.21.15.72:0/3820070001 <== mon.2 v2:172.21.15.76:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2ff8036030 con 0x55f85f660a60 2024-04-26T21:58:20.136 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.134+0000 7f2ff6000640 1 -- 172.21.15.72:0/3820070001 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ff80045c0 con 0x55f85f660a60 2024-04-26T21:58:20.137 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.136+0000 7f300a9f0740 1 -- 172.21.15.72:0/3820070001 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f85f660a60 msgr2=0x55f85f665560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:20.137 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.136+0000 7f3005c00640 1 --2- 172.21.15.72:0/2356902837 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f2fe40a3e40 0x7f2fe40a6300 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x55f85f6612c0 tx=0x7f2ff00046c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:58:20.137 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.136+0000 7f300a9f0740 1 --2- 172.21.15.72:0/3820070001 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f85f660a60 0x55f85f665560 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f2ff8031500 tx=0x7f2ff8009a20 comp rx=0 tx=0).stop 2024-04-26T21:58:20.137 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.136+0000 7f300a9f0740 1 -- 172.21.15.72:0/3820070001 shutdown_connections 2024-04-26T21:58:20.137 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.136+0000 7f300a9f0740 1 --2- 172.21.15.72:0/3820070001 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f85f665bd0 0x55f85f668060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:20.137 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.136+0000 7f300a9f0740 1 --2- 172.21.15.72:0/3820070001 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f85f660a60 0x55f85f665560 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:20.137 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.136+0000 7f300a9f0740 1 --2- 172.21.15.72:0/3820070001 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f85f660070 0x55f85f660490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:20.137 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.136+0000 7f300a9f0740 1 -- 172.21.15.72:0/3820070001 >> 172.21.15.72:0/3820070001 conn(0x55f85f652ed0 msgr2=0x55f85f6532d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:58:20.138 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.136+0000 7f300a9f0740 1 -- 172.21.15.72:0/3820070001 shutdown_connections 2024-04-26T21:58:20.138 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.136+0000 7f300a9f0740 1 -- 172.21.15.72:0/3820070001 wait complete. 2024-04-26T21:58:20.138 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.137+0000 7f300a9f0740 1 -- start start 2024-04-26T21:58:20.138 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.138+0000 7f300a9f0740 1 Processor -- start 2024-04-26T21:58:20.139 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.139+0000 7f300a9f0740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f85f660070 0x55f85f66a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:20.141 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.139+0000 7f300a9f0740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f85f660a60 0x55f85f66a820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:20.141 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.139+0000 7f300a9f0740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f85f665bd0 0x55f85f6717f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:20.141 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.139+0000 7f300a9f0740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55f85f6689a0 con 0x55f85f660070 2024-04-26T21:58:20.141 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.139+0000 7f300a9f0740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55f85f544170 con 0x55f85f660a60 2024-04-26T21:58:20.141 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.139+0000 7f300a9f0740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55f85f668b20 con 0x55f85f665bd0 2024-04-26T21:58:20.142 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.139+0000 7f3005200640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f85f660070 0x55f85f66a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:20.142 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.139+0000 7f3005200640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f85f660070 0x55f85f66a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:46252/0 (socket says 172.21.15.72:46252) 2024-04-26T21:58:20.142 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.139+0000 7f3005200640 1 -- 172.21.15.72:0/1339919519 learned_addr learned my addr 172.21.15.72:0/1339919519 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:58:20.142 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.140+0000 7f300a9f0740 10 client.0 _ll_register_callbacks cb 0x55f85f5b3770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:58:20.142 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.140+0000 7f2fffe00640 1 --2- 172.21.15.72:0/1339919519 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f85f660a60 0x55f85f66a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:20.142 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.140+0000 7f3005c00640 1 --2- 172.21.15.72:0/1339919519 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f85f665bd0 0x55f85f6717f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:20.142 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.140+0000 7f3005200640 1 -- 172.21.15.72:0/1339919519 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f85f665bd0 msgr2=0x55f85f6717f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:58:20.142 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.140+0000 7f3005200640 1 --2- 172.21.15.72:0/1339919519 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f85f665bd0 0x55f85f6717f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:20.142 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.140+0000 7f3005200640 1 -- 172.21.15.72:0/1339919519 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f85f660a60 msgr2=0x55f85f66a820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:58:20.142 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.140+0000 7f3005200640 1 --2- 172.21.15.72:0/1339919519 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f85f660a60 0x55f85f66a820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:20.143 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.140+0000 7f3005200640 1 -- 172.21.15.72:0/1339919519 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f85f3f6f70 con 0x55f85f660070 2024-04-26T21:58:20.143 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.140+0000 7f2fffe00640 1 --2- 172.21.15.72:0/1339919519 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f85f660a60 0x55f85f66a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:58:20.143 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.140+0000 7f3005c00640 1 --2- 172.21.15.72:0/1339919519 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f85f665bd0 0x55f85f6717f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:58:20.143 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.141+0000 7f3005200640 1 --2- 172.21.15.72:0/1339919519 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f85f660070 0x55f85f66a2e0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f2ff80314d0 tx=0x7f2ff8003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19e07781580a2c21 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:58:20.143 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.141+0000 7f2feea00640 10 client.0 ms_handle_connect on v2:172.21.15.72:3300/0 2024-04-26T21:58:20.143 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.141+0000 7f2feea00640 1 -- 172.21.15.72:0/1339919519 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ff804a040 con 0x55f85f660070 2024-04-26T21:58:20.143 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.141+0000 7f2feea00640 1 -- 172.21.15.72:0/1339919519 <== mon.0 v2:172.21.15.72:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7f2ff8004340 con 0x55f85f660070 2024-04-26T21:58:20.144 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.141+0000 7f300a9f0740 1 -- 172.21.15.72:0/1339919519 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2ff8036030 con 0x55f85f660070 2024-04-26T21:58:20.146 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.142+0000 7f2feea00640 1 -- 172.21.15.72:0/1339919519 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2ff8004540 con 0x55f85f660070 2024-04-26T21:58:20.147 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.142+0000 7f300a9f0740 10 client.16239 fetch_fsmap learned FSMap version 27 2024-04-26T21:58:20.147 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.142+0000 7f300a9f0740 1 -- 172.21.15.72:0/1339919519 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f85f28e170 con 0x55f85f660070 2024-04-26T21:58:20.147 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.142+0000 7f2feea00640 1 -- 172.21.15.72:0/1339919519 <== mon.0 v2:172.21.15.72:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2ff8036030 con 0x55f85f660070 2024-04-26T21:58:20.147 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.142+0000 7f2feea00640 1 -- 172.21.15.72:0/1339919519 <== mon.0 v2:172.21.15.72:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f2ff804d030 con 0x55f85f660070 2024-04-26T21:58:20.147 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.142+0000 7f300a9f0740 10 client.16239 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T21:58:20.148 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.142+0000 7f300a9f0740 10 client.16239 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:58:20.148 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.142+0000 7f300a9f0740 20 client.16239 populate_metadata read hostname 'smithi072' 2024-04-26T21:58:20.148 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.142+0000 7f300a9f0740 1 --2- 172.21.15.72:0/1339919519 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x55f85f675950 0x55f85f677d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:20.148 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.142+0000 7f300a9f0740 4 client.16239 mds_command: new command op to 34517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:58:20.148 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.142+0000 7f300a9f0740 1 -- 172.21.15.72:0/1339919519 --> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f85f5ca590 con 0x55f85f675950 2024-04-26T21:58:20.148 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.143+0000 7f2fffe00640 1 --2- 172.21.15.72:0/1339919519 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x55f85f675950 0x55f85f677d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:20.148 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.144+0000 7f2feea00640 10 client.16239 ms_handle_connect on v2:172.21.15.76:6838/327145636 2024-04-26T21:58:20.149 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.144+0000 7f2fffe00640 1 --2- 172.21.15.72:0/1339919519 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x55f85f675950 0x55f85f677d60 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f91a94b61bcc31e9 server_cookie=eeb113d79c5ac9c4 in_seq=0 out_seq=0 2024-04-26T21:58:20.149 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.145+0000 7f2feea00640 1 -- 172.21.15.72:0/1339919519 <== mds.0 v2:172.21.15.76:6838/327145636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f85f5ca590 con 0x55f85f675950 2024-04-26T21:58:20.149 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.145+0000 7f2feea00640 10 client.16239 handle_command_reply: tid=0 2024-04-26T21:58:20.149 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.145+0000 7f300a9f0740 1 client.16239 shutdown 2024-04-26T21:58:20.149 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.145+0000 7f300a9f0740 1 -- 172.21.15.72:0/1339919519 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f85f660070 msgr2=0x55f85f66a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:20.149 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.145+0000 7f300a9f0740 1 --2- 172.21.15.72:0/1339919519 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f85f660070 0x55f85f66a2e0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f2ff80314d0 tx=0x7f2ff8003ba0 comp rx=0 tx=0).stop 2024-04-26T21:58:20.149 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.145+0000 7f300a9f0740 1 -- 172.21.15.72:0/1339919519 shutdown_connections 2024-04-26T21:58:20.149 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.146+0000 7f300a9f0740 1 --2- 172.21.15.72:0/1339919519 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x55f85f675950 0x55f85f677d60 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:20.149 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.146+0000 7f300a9f0740 1 --2- 172.21.15.72:0/1339919519 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f85f665bd0 0x55f85f6717f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:20.150 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.146+0000 7f300a9f0740 1 --2- 172.21.15.72:0/1339919519 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f85f660a60 0x55f85f66a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:20.150 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.146+0000 7f300a9f0740 1 --2- 172.21.15.72:0/1339919519 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f85f660070 0x55f85f66a2e0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:20.150 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.146+0000 7f300a9f0740 1 -- 172.21.15.72:0/1339919519 >> 172.21.15.72:0/1339919519 conn(0x55f85f652ed0 msgr2=0x55f85f651ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:58:20.150 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.146+0000 7f2feea00640 0 client.16239 ms_handle_reset on v2:172.21.15.76:6838/327145636 2024-04-26T21:58:20.150 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.146+0000 7f300a9f0740 1 -- 172.21.15.72:0/1339919519 shutdown_connections 2024-04-26T21:58:20.150 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.146+0000 7f300a9f0740 1 -- 172.21.15.72:0/1339919519 wait complete. 2024-04-26T21:58:20.150 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.146+0000 7f300a9f0740 20 client.16239 trim_cache size 0 max 16384 2024-04-26T21:58:20.179 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.178+0000 7f300a9f0740 1 Processor -- start 2024-04-26T21:58:20.179 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.178+0000 7f300a9f0740 1 -- start start 2024-04-26T21:58:20.179 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.178+0000 7f300a9f0740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f85f660070 0x55f85f669900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:20.180 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.178+0000 7f300a9f0740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f85f660a60 0x55f85f671710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:20.180 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.178+0000 7f300a9f0740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f85f665bd0 0x55f85f669e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:20.180 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.178+0000 7f300a9f0740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f30001a7d40 con 0x55f85f660070 2024-04-26T21:58:20.180 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.178+0000 7f300a9f0740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f30001a9880 con 0x55f85f665bd0 2024-04-26T21:58:20.180 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.178+0000 7f300a9f0740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f30001a8410 con 0x55f85f660a60 2024-04-26T21:58:20.180 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.179+0000 7f3005200640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f85f660070 0x55f85f669900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:20.180 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.179+0000 7f3005200640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f85f660070 0x55f85f669900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:46266/0 (socket says 172.21.15.72:46266) 2024-04-26T21:58:20.180 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.179+0000 7f3005200640 1 -- 172.21.15.72:0/3441542219 learned_addr learned my addr 172.21.15.72:0/3441542219 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:58:20.180 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.179+0000 7f3005c00640 1 --2- 172.21.15.72:0/3441542219 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f85f665bd0 0x55f85f669e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:20.180 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.179+0000 7f3005200640 1 -- 172.21.15.72:0/3441542219 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f85f660a60 msgr2=0x55f85f671710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:58:20.180 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.179+0000 7f3005200640 1 --2- 172.21.15.72:0/3441542219 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f85f660a60 0x55f85f671710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:20.180 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.179+0000 7f3005200640 1 -- 172.21.15.72:0/3441542219 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f85f665bd0 msgr2=0x55f85f669e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:58:20.180 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.179+0000 7f3005200640 1 --2- 172.21.15.72:0/3441542219 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f85f665bd0 0x55f85f669e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:20.180 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.179+0000 7f3005200640 1 -- 172.21.15.72:0/3441542219 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f85f652a10 con 0x55f85f660070 2024-04-26T21:58:20.180 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.179+0000 7f3005200640 1 --2- 172.21.15.72:0/3441542219 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f85f660070 0x55f85f669900 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f2ff805c040 tx=0x7f2ff8009880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67d88e71d57ef412 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:58:20.182 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.180+0000 7f2ff5600640 1 -- 172.21.15.72:0/3441542219 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ff80046e0 con 0x55f85f660070 2024-04-26T21:58:20.182 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.180+0000 7f2ff5600640 1 -- 172.21.15.72:0/3441542219 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2ff8036070 con 0x55f85f660070 2024-04-26T21:58:20.182 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.180+0000 7f2ff5600640 1 -- 172.21.15.72:0/3441542219 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ff8003e70 con 0x55f85f660070 2024-04-26T21:58:20.182 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.181+0000 7f300a9f0740 1 -- 172.21.15.72:0/3441542219 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f85f660070 msgr2=0x55f85f669900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:20.182 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.181+0000 7f300a9f0740 1 --2- 172.21.15.72:0/3441542219 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f85f660070 0x55f85f669900 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f2ff805c040 tx=0x7f2ff8009880 comp rx=0 tx=0).stop 2024-04-26T21:58:20.182 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.181+0000 7f300a9f0740 1 -- 172.21.15.72:0/3441542219 shutdown_connections 2024-04-26T21:58:20.182 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.181+0000 7f300a9f0740 1 --2- 172.21.15.72:0/3441542219 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f85f665bd0 0x55f85f669e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:20.182 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.181+0000 7f300a9f0740 1 --2- 172.21.15.72:0/3441542219 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f85f660a60 0x55f85f671710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:20.182 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.181+0000 7f300a9f0740 1 --2- 172.21.15.72:0/3441542219 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f85f660070 0x55f85f669900 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:20.182 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.181+0000 7f300a9f0740 1 -- 172.21.15.72:0/3441542219 >> 172.21.15.72:0/3441542219 conn(0x55f85f652ed0 msgr2=0x55f85f651ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:58:20.182 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.181+0000 7f300a9f0740 1 -- 172.21.15.72:0/3441542219 shutdown_connections 2024-04-26T21:58:20.182 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.181+0000 7f300a9f0740 1 -- 172.21.15.72:0/3441542219 wait complete. 2024-04-26T21:58:20.183 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.181+0000 7f300a9f0740 1 -- start start 2024-04-26T21:58:20.183 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.182+0000 7f300a9f0740 1 Processor -- start 2024-04-26T21:58:20.185 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.182+0000 7f300a9f0740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f85f660070 0x55f85f66ef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:20.185 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.182+0000 7f300a9f0740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f85f660a60 0x55f85f66f480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:20.185 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.182+0000 7f300a9f0740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f85f665bd0 0x55f85f670490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:20.185 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.182+0000 7f300a9f0740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f30001a91b0 con 0x55f85f660a60 2024-04-26T21:58:20.185 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.182+0000 7f300a9f0740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f30001a8ae0 con 0x55f85f665bd0 2024-04-26T21:58:20.185 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.182+0000 7f300a9f0740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f30001a5d20 con 0x55f85f660070 2024-04-26T21:58:20.186 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.182+0000 7f300a9f0740 10 client.0 _ll_register_callbacks cb 0x7f2ff8031e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:58:20.186 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.183+0000 7f2fffe00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f85f660a60 0x55f85f66f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:20.186 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.183+0000 7f2fffe00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f85f660a60 0x55f85f66f480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:46270/0 (socket says 172.21.15.72:46270) 2024-04-26T21:58:20.186 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.183+0000 7f2fffe00640 1 -- 172.21.15.72:0/4227297164 learned_addr learned my addr 172.21.15.72:0/4227297164 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:58:20.186 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.183+0000 7f3005c00640 1 --2- 172.21.15.72:0/4227297164 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f85f665bd0 0x55f85f670490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:20.186 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.183+0000 7f2fffe00640 1 -- 172.21.15.72:0/4227297164 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f85f660070 msgr2=0x55f85f66ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:58:20.186 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.183+0000 7f2fffe00640 1 --2- 172.21.15.72:0/4227297164 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f85f660070 0x55f85f66ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:20.186 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.183+0000 7f2fffe00640 1 -- 172.21.15.72:0/4227297164 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f85f665bd0 msgr2=0x55f85f670490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:58:20.186 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.183+0000 7f2fffe00640 1 --2- 172.21.15.72:0/4227297164 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f85f665bd0 0x55f85f670490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:20.186 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.183+0000 7f2fffe00640 1 -- 172.21.15.72:0/4227297164 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f85f670a80 con 0x55f85f660a60 2024-04-26T21:58:20.186 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.183+0000 7f3005c00640 1 --2- 172.21.15.72:0/4227297164 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f85f665bd0 0x55f85f670490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:58:20.186 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.183+0000 7f2fffe00640 1 --2- 172.21.15.72:0/4227297164 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f85f660a60 0x55f85f66f480 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f2fe800dfd0 tx=0x7f2fe809caa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34e1c7765256054c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:58:20.187 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.183+0000 7f2feea00640 10 client.0 ms_handle_connect on v2:172.21.15.72:3300/0 2024-04-26T21:58:20.187 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.183+0000 7f2feea00640 1 -- 172.21.15.72:0/4227297164 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fe8020040 con 0x55f85f660a60 2024-04-26T21:58:20.187 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.184+0000 7f2feea00640 1 -- 172.21.15.72:0/4227297164 <== mon.0 v2:172.21.15.72:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7f2fe8098880 con 0x55f85f660a60 2024-04-26T21:58:20.187 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.184+0000 7f300a9f0740 1 -- 172.21.15.72:0/4227297164 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2ff8036070 con 0x55f85f660a60 2024-04-26T21:58:20.187 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.184+0000 7f2feea00640 1 -- 172.21.15.72:0/4227297164 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2fe80a4f10 con 0x55f85f660a60 2024-04-26T21:58:20.187 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.184+0000 7f300a9f0740 10 client.16245 fetch_fsmap learned FSMap version 27 2024-04-26T21:58:20.187 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.184+0000 7f300a9f0740 1 -- 172.21.15.72:0/4227297164 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f85f6dbfb0 con 0x55f85f660a60 2024-04-26T21:58:20.187 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.184+0000 7f2feea00640 1 -- 172.21.15.72:0/4227297164 <== mon.0 v2:172.21.15.72:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2ff8036070 con 0x55f85f660a60 2024-04-26T21:58:20.187 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.185+0000 7f2feea00640 1 -- 172.21.15.72:0/4227297164 <== mon.0 v2:172.21.15.72:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f2fe8081b90 con 0x55f85f660a60 2024-04-26T21:58:20.187 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.185+0000 7f300a9f0740 10 client.16245 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T21:58:20.187 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.185+0000 7f300a9f0740 10 client.16245 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:58:20.187 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.185+0000 7f300a9f0740 20 client.16245 populate_metadata read hostname 'smithi072' 2024-04-26T21:58:20.188 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.185+0000 7f300a9f0740 1 --2- 172.21.15.72:0/4227297164 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x55f85f6dca00 0x55f85f6dee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:20.188 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.185+0000 7f300a9f0740 4 client.16245 mds_command: new command op to 34517 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-26T21:58:20.188 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.185+0000 7f300a9f0740 1 -- 172.21.15.72:0/4227297164 --> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55f85f5c4400 con 0x55f85f6dca00 2024-04-26T21:58:20.188 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.185+0000 7f3005200640 1 --2- 172.21.15.72:0/4227297164 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x55f85f6dca00 0x55f85f6dee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:20.188 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.186+0000 7f3005200640 1 --2- 172.21.15.72:0/4227297164 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x55f85f6dca00 0x55f85f6dee20 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9554a458fa937172 server_cookie=94c8a4b42e4a5dd6 in_seq=0 out_seq=0 2024-04-26T21:58:20.188 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.186+0000 7f2feea00640 10 client.16245 ms_handle_connect on v2:172.21.15.76:6838/327145636 2024-04-26T21:58:20.189 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.189+0000 7f2feea00640 1 -- 172.21.15.72:0/4227297164 <== mds.0 v2:172.21.15.76:6838/327145636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55f85f5c4400 con 0x55f85f6dca00 2024-04-26T21:58:20.189 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.189+0000 7f2feea00640 10 client.16245 handle_command_reply: tid=0 2024-04-26T21:58:20.189 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.189+0000 7f300a9f0740 1 client.16245 shutdown 2024-04-26T21:58:20.189 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.189+0000 7f300a9f0740 1 -- 172.21.15.72:0/4227297164 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f85f660a60 msgr2=0x55f85f66f480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:20.189 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.189+0000 7f300a9f0740 1 --2- 172.21.15.72:0/4227297164 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f85f660a60 0x55f85f66f480 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f2fe800dfd0 tx=0x7f2fe809caa0 comp rx=0 tx=0).stop 2024-04-26T21:58:20.190 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.190+0000 7f300a9f0740 1 -- 172.21.15.72:0/4227297164 shutdown_connections 2024-04-26T21:58:20.190 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.190+0000 7f300a9f0740 1 --2- 172.21.15.72:0/4227297164 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x55f85f6dca00 0x55f85f6dee20 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:20.190 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.190+0000 7f2feea00640 0 client.16245 ms_handle_reset on v2:172.21.15.76:6838/327145636 2024-04-26T21:58:20.190 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.190+0000 7f300a9f0740 1 --2- 172.21.15.72:0/4227297164 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f85f665bd0 0x55f85f670490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:20.190 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.190+0000 7f300a9f0740 1 --2- 172.21.15.72:0/4227297164 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f85f660a60 0x55f85f66f480 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:20.190 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.190+0000 7f300a9f0740 1 --2- 172.21.15.72:0/4227297164 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f85f660070 0x55f85f66ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:20.190 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.190+0000 7f300a9f0740 1 -- 172.21.15.72:0/4227297164 >> 172.21.15.72:0/4227297164 conn(0x55f85f652ed0 msgr2=0x55f85f655c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:58:20.190 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.190+0000 7f300a9f0740 1 -- 172.21.15.72:0/4227297164 shutdown_connections 2024-04-26T21:58:20.191 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.190+0000 7f300a9f0740 1 -- 172.21.15.72:0/4227297164 wait complete. 2024-04-26T21:58:20.191 INFO:teuthology.orchestra.run.smithi072.stdout:{ 2024-04-26T21:58:20.191 INFO:teuthology.orchestra.run.smithi072.stdout: "return_code": 0, 2024-04-26T21:58:20.191 INFO:teuthology.orchestra.run.smithi072.stdout: "scrub_tag": "5f8c5706-4d99-48db-9dae-ed7fd91e84e4", 2024-04-26T21:58:20.191 INFO:teuthology.orchestra.run.smithi072.stdout: "mode": "asynchronous" 2024-04-26T21:58:20.191 INFO:teuthology.orchestra.run.smithi072.stdout:} 2024-04-26T21:58:20.192 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.190+0000 7f300a9f0740 20 client.16245 trim_cache size 0 max 16384 2024-04-26T21:58:20.192 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.191+0000 7f3007c00640 1 -- 172.21.15.72:0/2356902837 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f2fe40a3e40 msgr2=0x7f2fe40a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:20.192 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.191+0000 7f3007c00640 1 --2- 172.21.15.72:0/2356902837 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f2fe40a3e40 0x7f2fe40a6300 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x55f85f6612c0 tx=0x7f2ff00046c0 comp rx=0 tx=0).stop 2024-04-26T21:58:20.192 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.191+0000 7f3007c00640 1 -- 172.21.15.72:0/2356902837 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3000105da0 msgr2=0x7f300019b300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:20.192 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.191+0000 7f3007c00640 1 --2- 172.21.15.72:0/2356902837 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3000105da0 0x7f300019b300 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f2fe8005fd0 tx=0x7f2fe800dde0 comp rx=0 tx=0).stop 2024-04-26T21:58:20.193 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.191+0000 7f3007c00640 1 -- 172.21.15.72:0/2356902837 shutdown_connections 2024-04-26T21:58:20.193 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.191+0000 7f3007c00640 1 --2- 172.21.15.72:0/2356902837 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f2fe40a3e40 0x7f2fe40a6300 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:20.193 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.191+0000 7f3007c00640 1 --2- 172.21.15.72:0/2356902837 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3000106760 0x7f30001a2330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:20.193 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.191+0000 7f3007c00640 1 --2- 172.21.15.72:0/2356902837 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3000105da0 0x7f300019b300 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:20.193 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.191+0000 7f3007c00640 1 --2- 172.21.15.72:0/2356902837 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3000104bb0 0x7f300019adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:20.193 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.191+0000 7f3007c00640 1 -- 172.21.15.72:0/2356902837 >> 172.21.15.72:0/2356902837 conn(0x7f3000100120 msgr2=0x7f3000109110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:58:20.193 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.191+0000 7f3007c00640 1 -- 172.21.15.72:0/2356902837 shutdown_connections 2024-04-26T21:58:20.193 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.191+0000 7f3007c00640 1 -- 172.21.15.72:0/2356902837 wait complete. 2024-04-26T21:58:20.211 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-26T21:58:20.211 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-26T21:58:20.447 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.446+0000 7f61aa200640 1 -- 172.21.15.72:0/652154682 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f61a4105c60 msgr2=0x7f61a41060e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:20.447 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.446+0000 7f61aa200640 1 --2- 172.21.15.72:0/652154682 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f61a4105c60 0x7f61a41060e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6190009900 tx=0x7f619002a3f0 comp rx=0 tx=0).stop 2024-04-26T21:58:20.447 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.446+0000 7f61aa200640 1 -- 172.21.15.72:0/652154682 shutdown_connections 2024-04-26T21:58:20.447 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.446+0000 7f61aa200640 1 --2- 172.21.15.72:0/652154682 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f61a4106620 0x7f61a410ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:20.447 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.446+0000 7f61aa200640 1 --2- 172.21.15.72:0/652154682 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f61a4105c60 0x7f61a41060e0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:20.448 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.446+0000 7f61aa200640 1 --2- 172.21.15.72:0/652154682 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f61a4104a70 0x7f61a4104e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:20.448 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.446+0000 7f61aa200640 1 -- 172.21.15.72:0/652154682 >> 172.21.15.72:0/652154682 conn(0x7f61a40fffe0 msgr2=0x7f61a4102420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:58:20.448 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.446+0000 7f61aa200640 1 -- 172.21.15.72:0/652154682 shutdown_connections 2024-04-26T21:58:20.448 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.446+0000 7f61aa200640 1 -- 172.21.15.72:0/652154682 wait complete. 2024-04-26T21:58:20.451 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.448+0000 7f61aa200640 1 Processor -- start 2024-04-26T21:58:20.451 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.448+0000 7f61aa200640 1 -- start start 2024-04-26T21:58:20.451 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.448+0000 7f61aa200640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f61a4104a70 0x7f61a419efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:20.452 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.448+0000 7f61aa200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f61a4105c60 0x7f61a419f510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:20.452 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.448+0000 7f61aa200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f61a4106620 0x7f61a41a6580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:20.452 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.448+0000 7f61aa200640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f61a4077b80 con 0x7f61a4104a70 2024-04-26T21:58:20.452 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.448+0000 7f61aa200640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f61a4077a00 con 0x7f61a4106620 2024-04-26T21:58:20.452 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.448+0000 7f61aa200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f61a4077d00 con 0x7f61a4105c60 2024-04-26T21:58:20.452 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.449+0000 7f61a3e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f61a4105c60 0x7f61a419f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:20.453 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.449+0000 7f61a3e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f61a4105c60 0x7f61a419f510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.72:56092/0 (socket says 172.21.15.72:56092) 2024-04-26T21:58:20.453 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.449+0000 7f61a3e00640 1 -- 172.21.15.72:0/2271084683 learned_addr learned my addr 172.21.15.72:0/2271084683 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:58:20.453 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.449+0000 7f61a8e00640 1 --2- 172.21.15.72:0/2271084683 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f61a4104a70 0x7f61a419efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:20.453 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.449+0000 7f61a9800640 1 --2- 172.21.15.72:0/2271084683 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f61a4106620 0x7f61a41a6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:20.453 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.449+0000 7f61a8e00640 1 -- 172.21.15.72:0/2271084683 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f61a4105c60 msgr2=0x7f61a419f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:20.453 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.449+0000 7f61a8e00640 1 --2- 172.21.15.72:0/2271084683 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f61a4105c60 0x7f61a419f510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:20.453 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.450+0000 7f61a8e00640 1 -- 172.21.15.72:0/2271084683 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f61a4106620 msgr2=0x7f61a41a6580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:20.453 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.450+0000 7f61a8e00640 1 --2- 172.21.15.72:0/2271084683 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f61a4106620 0x7f61a41a6580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:20.453 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.450+0000 7f61a8e00640 1 -- 172.21.15.72:0/2271084683 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61a41a6bf0 con 0x7f61a4104a70 2024-04-26T21:58:20.453 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.450+0000 7f61a3e00640 1 --2- 172.21.15.72:0/2271084683 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f61a4105c60 0x7f61a419f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:58:20.453 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.450+0000 7f61a8e00640 1 --2- 172.21.15.72:0/2271084683 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f61a4104a70 0x7f61a419efd0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f619800d6a0 tx=0x7f619800daa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:58:20.453 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.450+0000 7f61a1600640 1 -- 172.21.15.72:0/2271084683 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6198017070 con 0x7f61a4104a70 2024-04-26T21:58:20.453 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.450+0000 7f61a1600640 1 -- 172.21.15.72:0/2271084683 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6198009be0 con 0x7f61a4104a70 2024-04-26T21:58:20.454 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.450+0000 7f61a1600640 1 -- 172.21.15.72:0/2271084683 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61980039b0 con 0x7f61a4104a70 2024-04-26T21:58:20.454 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.450+0000 7f61aa200640 1 -- 172.21.15.72:0/2271084683 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61a41a6ee0 con 0x7f61a4104a70 2024-04-26T21:58:20.454 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.451+0000 7f61aa200640 1 -- 172.21.15.72:0/2271084683 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61a41a72f0 con 0x7f61a4104a70 2024-04-26T21:58:20.455 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.455+0000 7f61ae622740 1 Processor -- start 2024-04-26T21:58:20.455 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.455+0000 7f61a1600640 1 -- 172.21.15.72:0/2271084683 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f6198004220 con 0x7f61a4104a70 2024-04-26T21:58:20.456 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.456+0000 7f61ae622740 1 -- start start 2024-04-26T21:58:20.457 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.456+0000 7f61ae622740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557177c04960 0x557177c04d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:20.457 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.456+0000 7f61ae622740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557177c05350 0x557177c09e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:20.457 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.456+0000 7f61ae622740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557177c0a4c0 0x557177c0c950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:20.457 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.456+0000 7f61ae622740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x557177b15030 con 0x557177c0a4c0 2024-04-26T21:58:20.457 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.456+0000 7f61ae622740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x557177b307f0 con 0x557177c04960 2024-04-26T21:58:20.457 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.456+0000 7f61ae622740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x557177aeb7e0 con 0x557177c05350 2024-04-26T21:58:20.457 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.456+0000 7f61a9800640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557177c04960 0x557177c04d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:20.457 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.456+0000 7f61a9800640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557177c04960 0x557177c04d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.72:48944/0 (socket says 172.21.15.72:48944) 2024-04-26T21:58:20.457 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.456+0000 7f61a9800640 1 -- 172.21.15.72:0/1299759342 learned_addr learned my addr 172.21.15.72:0/1299759342 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:58:20.458 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.456+0000 7f61a3e00640 1 --2- 172.21.15.72:0/1299759342 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557177c0a4c0 0x557177c0c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:20.458 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.457+0000 7f61a8e00640 1 --2- 172.21.15.72:0/1299759342 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557177c05350 0x557177c09e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:20.458 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.457+0000 7f61a3e00640 1 -- 172.21.15.72:0/1299759342 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557177c05350 msgr2=0x557177c09e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:58:20.458 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.457+0000 7f61a3e00640 1 --2- 172.21.15.72:0/1299759342 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557177c05350 0x557177c09e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:20.458 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.457+0000 7f61a3e00640 1 -- 172.21.15.72:0/1299759342 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557177c04960 msgr2=0x557177c04d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:58:20.458 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.457+0000 7f61a3e00640 1 --2- 172.21.15.72:0/1299759342 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557177c04960 0x557177c04d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:20.458 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.457+0000 7f61a3e00640 1 -- 172.21.15.72:0/1299759342 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557177af7050 con 0x557177c0a4c0 2024-04-26T21:58:20.458 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.457+0000 7f61a9800640 1 --2- 172.21.15.72:0/1299759342 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557177c04960 0x557177c04d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:58:20.458 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.457+0000 7f61a1600640 1 --2- 172.21.15.72:0/2271084683 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f618c0a3cc0 0x7f618c0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:20.460 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.458+0000 7f61a3e00640 1 --2- 172.21.15.72:0/1299759342 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557177c0a4c0 0x557177c0c950 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f6190000c00 tx=0x7f6190031930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc9815aa570f516d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:58:20.460 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.458+0000 7f61a1600640 1 -- 172.21.15.72:0/2271084683 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7f61980cfdc0 con 0x7f61a4104a70 2024-04-26T21:58:20.460 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.458+0000 7f61a9800640 1 --2- 172.21.15.72:0/2271084683 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f618c0a3cc0 0x7f618c0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:20.460 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.458+0000 7f6196000640 1 -- 172.21.15.72:0/1299759342 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6190035030 con 0x557177c0a4c0 2024-04-26T21:58:20.460 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.458+0000 7f6196000640 1 -- 172.21.15.72:0/1299759342 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6190040070 con 0x557177c0a4c0 2024-04-26T21:58:20.460 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.458+0000 7f6196000640 1 -- 172.21.15.72:0/1299759342 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6190003ca0 con 0x557177c0a4c0 2024-04-26T21:58:20.460 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.459+0000 7f61ae622740 1 -- 172.21.15.72:0/1299759342 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557177c0a4c0 msgr2=0x557177c0c950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:20.460 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.459+0000 7f61ae622740 1 --2- 172.21.15.72:0/1299759342 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557177c0a4c0 0x557177c0c950 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f6190000c00 tx=0x7f6190031930 comp rx=0 tx=0).stop 2024-04-26T21:58:20.460 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.459+0000 7f61a9800640 1 --2- 172.21.15.72:0/2271084683 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f618c0a3cc0 0x7f618c0a6180 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x557177c05bb0 tx=0x7f619c009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:58:20.460 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.460+0000 7f61ae622740 1 -- 172.21.15.72:0/1299759342 shutdown_connections 2024-04-26T21:58:20.460 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.460+0000 7f61ae622740 1 --2- 172.21.15.72:0/1299759342 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557177c0a4c0 0x557177c0c950 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:20.460 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.460+0000 7f61ae622740 1 --2- 172.21.15.72:0/1299759342 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557177c05350 0x557177c09e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:20.461 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.460+0000 7f61ae622740 1 --2- 172.21.15.72:0/1299759342 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557177c04960 0x557177c04d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:20.461 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.460+0000 7f61ae622740 1 -- 172.21.15.72:0/1299759342 >> 172.21.15.72:0/1299759342 conn(0x557177bfffd0 msgr2=0x557177c003d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:58:20.461 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.460+0000 7f61ae622740 1 -- 172.21.15.72:0/1299759342 shutdown_connections 2024-04-26T21:58:20.461 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.460+0000 7f61ae622740 1 -- 172.21.15.72:0/1299759342 wait complete. 2024-04-26T21:58:20.463 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.460+0000 7f61ae622740 1 -- start start 2024-04-26T21:58:20.463 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.461+0000 7f61ae622740 1 Processor -- start 2024-04-26T21:58:20.463 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.461+0000 7f61ae622740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557177c04960 0x557177c1f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:20.463 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.461+0000 7f61ae622740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557177c05350 0x557177c1fc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:20.464 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.461+0000 7f61ae622740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557177c0a4c0 0x557177c26c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:20.464 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.461+0000 7f61ae622740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x557177b6d200 con 0x557177c0a4c0 2024-04-26T21:58:20.464 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.461+0000 7f61ae622740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x557177b67ee0 con 0x557177c05350 2024-04-26T21:58:20.464 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.461+0000 7f61ae622740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x557177b74e70 con 0x557177c04960 2024-04-26T21:58:20.464 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.461+0000 7f61ae622740 10 client.0 _ll_register_callbacks cb 0x557177b5e270 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:58:20.464 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.462+0000 7f61a9800640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557177c0a4c0 0x557177c26c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:20.464 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.462+0000 7f61a8e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557177c04960 0x557177c1f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:20.464 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.462+0000 7f61a8e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557177c04960 0x557177c1f740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.72:56112/0 (socket says 172.21.15.72:56112) 2024-04-26T21:58:20.464 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.462+0000 7f61a8e00640 1 -- 172.21.15.72:0/2314040852 learned_addr learned my addr 172.21.15.72:0/2314040852 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:58:20.464 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.462+0000 7f61a3e00640 1 --2- 172.21.15.72:0/2314040852 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557177c05350 0x557177c1fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:20.464 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.462+0000 7f61a8e00640 1 -- 172.21.15.72:0/2314040852 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557177c05350 msgr2=0x557177c1fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:58:20.464 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.462+0000 7f61a8e00640 1 --2- 172.21.15.72:0/2314040852 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557177c05350 0x557177c1fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:20.464 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.462+0000 7f61a8e00640 1 -- 172.21.15.72:0/2314040852 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557177c0a4c0 msgr2=0x557177c26c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:58:20.465 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.463+0000 7f61a8e00640 1 --2- 172.21.15.72:0/2314040852 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557177c0a4c0 0x557177c26c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:20.465 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.463+0000 7f61a8e00640 1 -- 172.21.15.72:0/2314040852 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5571779447d0 con 0x557177c04960 2024-04-26T21:58:20.465 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.463+0000 7f61a9800640 1 --2- 172.21.15.72:0/2314040852 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557177c0a4c0 0x557177c26c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:58:20.465 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.463+0000 7f61a8e00640 1 --2- 172.21.15.72:0/2314040852 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557177c04960 0x557177c1f740 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f619800dfd0 tx=0x7f619809bd50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=16953d7fe59c9fd1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:58:20.465 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.463+0000 7f618aa00640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-04-26T21:58:20.465 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.463+0000 7f618aa00640 1 -- 172.21.15.72:0/2314040852 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6198020070 con 0x557177c04960 2024-04-26T21:58:20.465 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.463+0000 7f61ae622740 1 -- 172.21.15.72:0/2314040852 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6190040070 con 0x557177c04960 2024-04-26T21:58:20.465 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.463+0000 7f618aa00640 1 -- 172.21.15.72:0/2314040852 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7f6198098530 con 0x557177c04960 2024-04-26T21:58:20.466 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.464+0000 7f618aa00640 1 -- 172.21.15.72:0/2314040852 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f619809bec0 con 0x557177c04960 2024-04-26T21:58:20.466 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.464+0000 7f61ae622740 10 client.25384 fetch_fsmap learned FSMap version 27 2024-04-26T21:58:20.467 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.464+0000 7f61ae622740 1 -- 172.21.15.72:0/2314040852 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55717787a3f0 con 0x557177c04960 2024-04-26T21:58:20.467 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.464+0000 7f618aa00640 1 -- 172.21.15.72:0/2314040852 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6190040070 con 0x557177c04960 2024-04-26T21:58:20.467 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.465+0000 7f618aa00640 1 -- 172.21.15.72:0/2314040852 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f6198009870 con 0x557177c04960 2024-04-26T21:58:20.467 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.465+0000 7f61ae622740 10 client.25384 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T21:58:20.467 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.465+0000 7f61ae622740 10 client.25384 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:58:20.467 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.465+0000 7f61ae622740 20 client.25384 populate_metadata read hostname 'smithi072' 2024-04-26T21:58:20.467 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.465+0000 7f61ae622740 1 --2- 172.21.15.72:0/2314040852 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x557177c2b250 0x557177c2d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:20.467 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.465+0000 7f61ae622740 4 client.25384 mds_command: new command op to 34517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:58:20.467 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.465+0000 7f61ae622740 1 -- 172.21.15.72:0/2314040852 --> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557177b767e0 con 0x557177c2b250 2024-04-26T21:58:20.467 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.466+0000 7f61a3e00640 1 --2- 172.21.15.72:0/2314040852 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x557177c2b250 0x557177c2d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:20.467 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.467+0000 7f618aa00640 10 client.25384 ms_handle_connect on v2:172.21.15.76:6838/327145636 2024-04-26T21:58:20.467 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.467+0000 7f61a3e00640 1 --2- 172.21.15.72:0/2314040852 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x557177c2b250 0x557177c2d710 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9aa9e788269a3126 server_cookie=1cb33b4783251842 in_seq=0 out_seq=0 2024-04-26T21:58:20.468 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.468+0000 7f618aa00640 1 -- 172.21.15.72:0/2314040852 <== mds.0 v2:172.21.15.76:6838/327145636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557177b767e0 con 0x557177c2b250 2024-04-26T21:58:20.468 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.468+0000 7f618aa00640 10 client.25384 handle_command_reply: tid=0 2024-04-26T21:58:20.468 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.468+0000 7f61ae622740 1 client.25384 shutdown 2024-04-26T21:58:20.469 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.468+0000 7f61ae622740 1 -- 172.21.15.72:0/2314040852 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557177c04960 msgr2=0x557177c1f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:20.469 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.468+0000 7f61ae622740 1 --2- 172.21.15.72:0/2314040852 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557177c04960 0x557177c1f740 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f619800dfd0 tx=0x7f619809bd50 comp rx=0 tx=0).stop 2024-04-26T21:58:20.469 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.468+0000 7f61ae622740 1 -- 172.21.15.72:0/2314040852 shutdown_connections 2024-04-26T21:58:20.469 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.468+0000 7f61ae622740 1 --2- 172.21.15.72:0/2314040852 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x557177c2b250 0x557177c2d710 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:20.469 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.469+0000 7f61ae622740 1 --2- 172.21.15.72:0/2314040852 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557177c0a4c0 0x557177c26c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:20.469 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.469+0000 7f61ae622740 1 --2- 172.21.15.72:0/2314040852 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557177c05350 0x557177c1fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:20.469 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.469+0000 7f61ae622740 1 --2- 172.21.15.72:0/2314040852 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557177c04960 0x557177c1f740 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:20.469 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.469+0000 7f61ae622740 1 -- 172.21.15.72:0/2314040852 >> 172.21.15.72:0/2314040852 conn(0x557177bfffd0 msgr2=0x557177bff1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:58:20.469 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.469+0000 7f618aa00640 0 client.25384 ms_handle_reset on v2:172.21.15.76:6838/327145636 2024-04-26T21:58:20.470 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.469+0000 7f61ae622740 1 -- 172.21.15.72:0/2314040852 shutdown_connections 2024-04-26T21:58:20.470 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.469+0000 7f61ae622740 1 -- 172.21.15.72:0/2314040852 wait complete. 2024-04-26T21:58:20.470 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.470+0000 7f61ae622740 20 client.25384 trim_cache size 0 max 16384 2024-04-26T21:58:20.501 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.499+0000 7f61ae622740 1 Processor -- start 2024-04-26T21:58:20.501 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.499+0000 7f61ae622740 1 -- start start 2024-04-26T21:58:20.501 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.500+0000 7f61ae622740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557177c04960 0x557177c03730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:20.501 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.500+0000 7f61ae622740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557177c05350 0x557177c26c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:20.501 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.500+0000 7f61ae622740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557177c0a4c0 0x557177c05dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:20.501 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.500+0000 7f61ae622740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f61a41a9640 con 0x557177c05350 2024-04-26T21:58:20.501 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.500+0000 7f61ae622740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f61a41a81d0 con 0x557177c0a4c0 2024-04-26T21:58:20.501 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.500+0000 7f61ae622740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f61a41a88a0 con 0x557177c04960 2024-04-26T21:58:20.502 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.500+0000 7f61a3e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557177c05350 0x557177c26c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:20.502 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.500+0000 7f61a3e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557177c05350 0x557177c26c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:46320/0 (socket says 172.21.15.72:46320) 2024-04-26T21:58:20.502 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.500+0000 7f61a3e00640 1 -- 172.21.15.72:0/2654871019 learned_addr learned my addr 172.21.15.72:0/2654871019 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:58:20.502 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.501+0000 7f61a8e00640 1 --2- 172.21.15.72:0/2654871019 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557177c04960 0x557177c03730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:20.502 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.501+0000 7f61a9800640 1 --2- 172.21.15.72:0/2654871019 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557177c0a4c0 0x557177c05dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:20.502 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.501+0000 7f61a3e00640 1 -- 172.21.15.72:0/2654871019 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557177c04960 msgr2=0x557177c03730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:58:20.502 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.501+0000 7f61a3e00640 1 --2- 172.21.15.72:0/2654871019 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557177c04960 0x557177c03730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:20.502 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.501+0000 7f61a3e00640 1 -- 172.21.15.72:0/2654871019 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557177c0a4c0 msgr2=0x557177c05dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:58:20.502 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.501+0000 7f61a3e00640 1 --2- 172.21.15.72:0/2654871019 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557177c0a4c0 0x557177c05dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:20.502 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.501+0000 7f61a3e00640 1 -- 172.21.15.72:0/2654871019 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55717783a0f0 con 0x557177c05350 2024-04-26T21:58:20.503 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.501+0000 7f61a3e00640 1 --2- 172.21.15.72:0/2654871019 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557177c05350 0x557177c26c10 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f6190009a20 tx=0x7f619002a3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ef29082521257af server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:58:20.503 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.502+0000 7f6195600640 1 -- 172.21.15.72:0/2654871019 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6190035040 con 0x557177c05350 2024-04-26T21:58:20.503 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.502+0000 7f6195600640 1 -- 172.21.15.72:0/2654871019 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6190056070 con 0x557177c05350 2024-04-26T21:58:20.504 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.502+0000 7f6195600640 1 -- 172.21.15.72:0/2654871019 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6190003ca0 con 0x557177c05350 2024-04-26T21:58:20.504 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.503+0000 7f61ae622740 1 -- 172.21.15.72:0/2654871019 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557177c05350 msgr2=0x557177c26c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:20.504 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.503+0000 7f61ae622740 1 --2- 172.21.15.72:0/2654871019 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557177c05350 0x557177c26c10 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f6190009a20 tx=0x7f619002a3f0 comp rx=0 tx=0).stop 2024-04-26T21:58:20.504 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.503+0000 7f61ae622740 1 -- 172.21.15.72:0/2654871019 shutdown_connections 2024-04-26T21:58:20.504 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.503+0000 7f61ae622740 1 --2- 172.21.15.72:0/2654871019 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557177c0a4c0 0x557177c05dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:20.504 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.503+0000 7f61ae622740 1 --2- 172.21.15.72:0/2654871019 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557177c05350 0x557177c26c10 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:20.504 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.503+0000 7f61ae622740 1 --2- 172.21.15.72:0/2654871019 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557177c04960 0x557177c03730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:20.504 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.503+0000 7f61ae622740 1 -- 172.21.15.72:0/2654871019 >> 172.21.15.72:0/2654871019 conn(0x557177bfffd0 msgr2=0x557177c03020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:58:20.504 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.503+0000 7f61ae622740 1 -- 172.21.15.72:0/2654871019 shutdown_connections 2024-04-26T21:58:20.505 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.504+0000 7f61ae622740 1 -- 172.21.15.72:0/2654871019 wait complete. 2024-04-26T21:58:20.505 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.504+0000 7f61ae622740 1 -- start start 2024-04-26T21:58:20.505 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.504+0000 7f61ae622740 1 Processor -- start 2024-04-26T21:58:20.505 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.505+0000 7f61ae622740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557177c04960 0x557177c23e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:20.505 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.505+0000 7f61ae622740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557177c05350 0x557177c243b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:20.505 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.505+0000 7f61ae622740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557177c0a4c0 0x557177c253c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:20.505 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.505+0000 7f61a3e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557177c05350 0x557177c243b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:20.506 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.505+0000 7f61ae622740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f61a41163e0 con 0x557177c05350 2024-04-26T21:58:20.506 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.505+0000 7f61a8e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557177c04960 0x557177c23e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:20.506 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.505+0000 7f61a3e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557177c05350 0x557177c243b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:46332/0 (socket says 172.21.15.72:46332) 2024-04-26T21:58:20.506 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.505+0000 7f61a8e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557177c04960 0x557177c23e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.72:56134/0 (socket says 172.21.15.72:56134) 2024-04-26T21:58:20.506 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.505+0000 7f61a3e00640 1 -- 172.21.15.72:0/1450259901 learned_addr learned my addr 172.21.15.72:0/1450259901 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:58:20.506 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.505+0000 7f61a9800640 1 --2- 172.21.15.72:0/1450259901 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557177c0a4c0 0x557177c253c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:20.506 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.505+0000 7f61ae622740 1 -- 172.21.15.72:0/1450259901 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f61a41a8f70 con 0x557177c0a4c0 2024-04-26T21:58:20.506 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.505+0000 7f61ae622740 1 -- 172.21.15.72:0/1450259901 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f61a4116a50 con 0x557177c04960 2024-04-26T21:58:20.506 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.506+0000 7f61ae622740 10 client.0 _ll_register_callbacks cb 0x7f61980046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:58:20.507 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.506+0000 7f61a9800640 1 -- 172.21.15.72:0/1450259901 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557177c04960 msgr2=0x557177c23e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:58:20.507 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.506+0000 7f61a9800640 1 --2- 172.21.15.72:0/1450259901 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557177c04960 0x557177c23e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:20.507 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.506+0000 7f61a9800640 1 -- 172.21.15.72:0/1450259901 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557177c05350 msgr2=0x557177c243b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:58:20.507 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.506+0000 7f61a9800640 1 --2- 172.21.15.72:0/1450259901 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557177c05350 0x557177c243b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:20.507 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.507+0000 7f61a9800640 1 -- 172.21.15.72:0/1450259901 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557177b30a30 con 0x557177c0a4c0 2024-04-26T21:58:20.508 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.507+0000 7f61a8e00640 1 --2- 172.21.15.72:0/1450259901 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557177c04960 0x557177c23e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:58:20.508 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.507+0000 7f61a3e00640 1 --2- 172.21.15.72:0/1450259901 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557177c05350 0x557177c243b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:58:20.508 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.507+0000 7f61a9800640 1 --2- 172.21.15.72:0/1450259901 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557177c0a4c0 0x557177c253c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x557177b74f30 tx=0x7f619c004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f5a930a716d15365 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:58:20.508 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.507+0000 7f618aa00640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-04-26T21:58:20.508 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.508+0000 7f618aa00640 1 -- 172.21.15.72:0/1450259901 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f619c02f060 con 0x557177c0a4c0 2024-04-26T21:58:20.509 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.508+0000 7f61ae622740 1 -- 172.21.15.72:0/1450259901 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6190056070 con 0x557177c0a4c0 2024-04-26T21:58:20.509 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.508+0000 7f618aa00640 1 -- 172.21.15.72:0/1450259901 <== mon.2 v2:172.21.15.76:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7f619c018030 con 0x557177c0a4c0 2024-04-26T21:58:20.509 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.508+0000 7f618aa00640 1 -- 172.21.15.72:0/1450259901 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f619c02f060 con 0x557177c0a4c0 2024-04-26T21:58:20.509 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.509+0000 7f61ae622740 10 client.35477 fetch_fsmap learned FSMap version 27 2024-04-26T21:58:20.509 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.509+0000 7f61ae622740 1 -- 172.21.15.72:0/1450259901 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x557177c26be0 con 0x557177c0a4c0 2024-04-26T21:58:20.510 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.509+0000 7f618aa00640 1 -- 172.21.15.72:0/1450259901 <== mon.2 v2:172.21.15.76:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6190056070 con 0x557177c0a4c0 2024-04-26T21:58:20.510 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.509+0000 7f618aa00640 1 -- 172.21.15.72:0/1450259901 <== mon.2 v2:172.21.15.76:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f619c01e420 con 0x557177c0a4c0 2024-04-26T21:58:20.510 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.509+0000 7f61ae622740 10 client.35477 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T21:58:20.510 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.509+0000 7f61ae622740 10 client.35477 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:58:20.510 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.509+0000 7f61ae622740 20 client.35477 populate_metadata read hostname 'smithi072' 2024-04-26T21:58:20.510 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.510+0000 7f61ae622740 1 --2- 172.21.15.72:0/1450259901 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x557177c91c90 0x557177c940b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:20.510 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.510+0000 7f61ae622740 4 client.35477 mds_command: new command op to 34517 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-26T21:58:20.510 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.510+0000 7f61ae622740 1 -- 172.21.15.72:0/1450259901 --> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] -- command(tid 0: {"prefix": "scrub status"}) -- 0x557177b70650 con 0x557177c91c90 2024-04-26T21:58:20.510 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.510+0000 7f61a8e00640 1 --2- 172.21.15.72:0/1450259901 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x557177c91c90 0x557177c940b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:20.511 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.511+0000 7f61a8e00640 1 --2- 172.21.15.72:0/1450259901 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x557177c91c90 0x557177c940b0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=beec546a9f85f8f6 server_cookie=f5e93b76eedce3c7 in_seq=0 out_seq=0 2024-04-26T21:58:20.511 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.511+0000 7f618aa00640 10 client.35477 ms_handle_connect on v2:172.21.15.76:6838/327145636 2024-04-26T21:58:20.511 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.511+0000 7f618aa00640 1 -- 172.21.15.72:0/1450259901 <== mds.0 v2:172.21.15.76:6838/327145636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x557177b70650 con 0x557177c91c90 2024-04-26T21:58:20.511 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.511+0000 7f618aa00640 10 client.35477 handle_command_reply: tid=0 2024-04-26T21:58:20.511 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.511+0000 7f61ae622740 1 client.35477 shutdown 2024-04-26T21:58:20.511 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.511+0000 7f61ae622740 1 -- 172.21.15.72:0/1450259901 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557177c0a4c0 msgr2=0x557177c253c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:20.511 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.511+0000 7f61ae622740 1 --2- 172.21.15.72:0/1450259901 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557177c0a4c0 0x557177c253c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x557177b74f30 tx=0x7f619c004a40 comp rx=0 tx=0).stop 2024-04-26T21:58:20.512 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.511+0000 7f61ae622740 1 -- 172.21.15.72:0/1450259901 shutdown_connections 2024-04-26T21:58:20.512 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.511+0000 7f61ae622740 1 --2- 172.21.15.72:0/1450259901 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x557177c91c90 0x557177c940b0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:20.512 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.511+0000 7f61ae622740 1 --2- 172.21.15.72:0/1450259901 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557177c0a4c0 0x557177c253c0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:20.512 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.511+0000 7f61ae622740 1 --2- 172.21.15.72:0/1450259901 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557177c05350 0x557177c243b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:20.512 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.511+0000 7f61ae622740 1 --2- 172.21.15.72:0/1450259901 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557177c04960 0x557177c23e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:20.513 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.511+0000 7f61ae622740 1 -- 172.21.15.72:0/1450259901 >> 172.21.15.72:0/1450259901 conn(0x557177bfffd0 msgr2=0x557177c038b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:58:20.513 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.512+0000 7f618aa00640 0 client.35477 ms_handle_reset on v2:172.21.15.76:6838/327145636 2024-04-26T21:58:20.513 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.512+0000 7f61ae622740 1 -- 172.21.15.72:0/1450259901 shutdown_connections 2024-04-26T21:58:20.513 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.512+0000 7f61ae622740 1 -- 172.21.15.72:0/1450259901 wait complete. 2024-04-26T21:58:20.513 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.512+0000 7f61ae622740 20 client.35477 trim_cache size 0 max 16384 2024-04-26T21:58:20.513 INFO:teuthology.orchestra.run.smithi072.stdout:{ 2024-04-26T21:58:20.513 INFO:teuthology.orchestra.run.smithi072.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-26T21:58:20.513 INFO:teuthology.orchestra.run.smithi072.stdout: "scrubs": { 2024-04-26T21:58:20.513 INFO:teuthology.orchestra.run.smithi072.stdout: "5f8c5706-4d99-48db-9dae-ed7fd91e84e4": { 2024-04-26T21:58:20.513 INFO:teuthology.orchestra.run.smithi072.stdout: "path": "/", 2024-04-26T21:58:20.513 INFO:teuthology.orchestra.run.smithi072.stdout: "tag": "5f8c5706-4d99-48db-9dae-ed7fd91e84e4", 2024-04-26T21:58:20.514 INFO:teuthology.orchestra.run.smithi072.stdout: "options": "recursive,force" 2024-04-26T21:58:20.514 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:20.514 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:20.514 INFO:teuthology.orchestra.run.smithi072.stdout:} 2024-04-26T21:58:20.514 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.513+0000 7f61aa200640 1 -- 172.21.15.72:0/2271084683 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f618c0a3cc0 msgr2=0x7f618c0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:20.514 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.513+0000 7f61aa200640 1 --2- 172.21.15.72:0/2271084683 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f618c0a3cc0 0x7f618c0a6180 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x557177c05bb0 tx=0x7f619c009500 comp rx=0 tx=0).stop 2024-04-26T21:58:20.514 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.513+0000 7f61aa200640 1 -- 172.21.15.72:0/2271084683 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f61a4104a70 msgr2=0x7f61a419efd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:20.514 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.513+0000 7f61aa200640 1 --2- 172.21.15.72:0/2271084683 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f61a4104a70 0x7f61a419efd0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f619800d6a0 tx=0x7f619800daa0 comp rx=0 tx=0).stop 2024-04-26T21:58:20.514 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.513+0000 7f61aa200640 1 -- 172.21.15.72:0/2271084683 shutdown_connections 2024-04-26T21:58:20.514 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.513+0000 7f61aa200640 1 --2- 172.21.15.72:0/2271084683 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f61a4106620 0x7f61a41a6580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:20.515 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.513+0000 7f61aa200640 1 --2- 172.21.15.72:0/2271084683 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f618c0a3cc0 0x7f618c0a6180 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:20.515 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.513+0000 7f61aa200640 1 --2- 172.21.15.72:0/2271084683 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f61a4105c60 0x7f61a419f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:20.515 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.513+0000 7f61aa200640 1 --2- 172.21.15.72:0/2271084683 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f61a4104a70 0x7f61a419efd0 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:20.515 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.513+0000 7f61aa200640 1 -- 172.21.15.72:0/2271084683 >> 172.21.15.72:0/2271084683 conn(0x7f61a40fffe0 msgr2=0x7f61a407a750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:58:20.515 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.513+0000 7f61aa200640 1 -- 172.21.15.72:0/2271084683 shutdown_connections 2024-04-26T21:58:20.515 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:20.513+0000 7f61aa200640 1 -- 172.21.15.72:0/2271084683 wait complete. 2024-04-26T21:58:20.536 INFO:tasks.cephfs.filesystem:scrub status for tag:5f8c5706-4d99-48db-9dae-ed7fd91e84e4 - {'path': '/', 'tag': '5f8c5706-4d99-48db-9dae-ed7fd91e84e4', 'options': 'recursive,force'} 2024-04-26T21:58:21.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:58:21 smithi076 ceph-mon[31823]: pgmap v536: 97 pgs: 97 active+clean; 258 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 MiB/s wr, 2.66k op/s 2024-04-26T21:58:21.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:58:21 smithi076 ceph-mon[31823]: scrub queued for path: / 2024-04-26T21:58:21.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:58:21 smithi076 ceph-mon[31823]: scrub summary: idle+waiting paths [/] 2024-04-26T21:58:21.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:58:21 smithi076 ceph-mon[31823]: scrub summary: active paths [/] 2024-04-26T21:58:21.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:58:21 smithi076 ceph-mon[31823]: scrub summary: idle+waiting paths [/] 2024-04-26T21:58:21.321 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:58:21 smithi072 ceph-mon[28138]: pgmap v536: 97 pgs: 97 active+clean; 258 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 MiB/s wr, 2.66k op/s 2024-04-26T21:58:21.321 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:58:21 smithi072 ceph-mon[28138]: scrub queued for path: / 2024-04-26T21:58:21.321 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:58:21 smithi072 ceph-mon[28138]: scrub summary: idle+waiting paths [/] 2024-04-26T21:58:21.321 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:58:21 smithi072 ceph-mon[28138]: scrub summary: active paths [/] 2024-04-26T21:58:21.321 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:58:21 smithi072 ceph-mon[28138]: scrub summary: idle+waiting paths [/] 2024-04-26T21:58:21.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:58:21 smithi164 ceph-mon[31916]: pgmap v536: 97 pgs: 97 active+clean; 258 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 MiB/s wr, 2.66k op/s 2024-04-26T21:58:21.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:58:21 smithi164 ceph-mon[31916]: scrub queued for path: / 2024-04-26T21:58:21.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:58:21 smithi164 ceph-mon[31916]: scrub summary: idle+waiting paths [/] 2024-04-26T21:58:21.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:58:21 smithi164 ceph-mon[31916]: scrub summary: active paths [/] 2024-04-26T21:58:21.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:58:21 smithi164 ceph-mon[31916]: scrub summary: idle+waiting paths [/] 2024-04-26T21:58:23.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:58:23 smithi076 ceph-mon[31823]: pgmap v537: 97 pgs: 97 active+clean; 258 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 11 MiB/s wr, 2.53k op/s 2024-04-26T21:58:23.321 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:58:23 smithi072 ceph-mon[28138]: pgmap v537: 97 pgs: 97 active+clean; 258 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 11 MiB/s wr, 2.53k op/s 2024-04-26T21:58:23.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:58:23 smithi164 ceph-mon[31916]: pgmap v537: 97 pgs: 97 active+clean; 258 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 11 MiB/s wr, 2.53k op/s 2024-04-26T21:58:25.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:58:25 smithi076 ceph-mon[31823]: pgmap v538: 97 pgs: 97 active+clean; 258 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 MiB/s wr, 2.70k op/s 2024-04-26T21:58:25.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:58:25 smithi076 ceph-mon[31823]: scrub summary: idle 2024-04-26T21:58:25.321 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:58:25 smithi072 ceph-mon[28138]: pgmap v538: 97 pgs: 97 active+clean; 258 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 MiB/s wr, 2.70k op/s 2024-04-26T21:58:25.321 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:58:25 smithi072 ceph-mon[28138]: scrub summary: idle 2024-04-26T21:58:25.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:58:25 smithi164 ceph-mon[31916]: pgmap v538: 97 pgs: 97 active+clean; 258 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 MiB/s wr, 2.70k op/s 2024-04-26T21:58:25.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:58:25 smithi164 ceph-mon[31916]: scrub summary: idle 2024-04-26T21:58:27.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:58:27 smithi076 ceph-mon[31823]: pgmap v539: 97 pgs: 97 active+clean; 258 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 MiB/s wr, 2.74k op/s 2024-04-26T21:58:27.321 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:58:27 smithi072 ceph-mon[28138]: pgmap v539: 97 pgs: 97 active+clean; 258 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 MiB/s wr, 2.74k op/s 2024-04-26T21:58:27.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:58:27 smithi164 ceph-mon[31916]: pgmap v539: 97 pgs: 97 active+clean; 258 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 MiB/s wr, 2.74k op/s 2024-04-26T21:58:29.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:58:29 smithi076 ceph-mon[31823]: pgmap v540: 97 pgs: 97 active+clean; 258 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 11 MiB/s wr, 2.56k op/s 2024-04-26T21:58:29.321 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:58:29 smithi072 ceph-mon[28138]: pgmap v540: 97 pgs: 97 active+clean; 258 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 11 MiB/s wr, 2.56k op/s 2024-04-26T21:58:29.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:58:29 smithi164 ceph-mon[31916]: pgmap v540: 97 pgs: 97 active+clean; 258 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 11 MiB/s wr, 2.56k op/s 2024-04-26T21:58:31.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:58:31 smithi076 ceph-mon[31823]: pgmap v541: 97 pgs: 97 active+clean; 258 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 MiB/s wr, 2.55k op/s 2024-04-26T21:58:31.321 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:58:31 smithi072 ceph-mon[28138]: pgmap v541: 97 pgs: 97 active+clean; 258 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 MiB/s wr, 2.55k op/s 2024-04-26T21:58:31.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:58:31 smithi164 ceph-mon[31916]: pgmap v541: 97 pgs: 97 active+clean; 258 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 MiB/s wr, 2.55k op/s 2024-04-26T21:58:33.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:58:33 smithi076 ceph-mon[31823]: pgmap v542: 97 pgs: 97 active+clean; 258 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 11 MiB/s wr, 2.42k op/s 2024-04-26T21:58:33.321 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:58:33 smithi072 ceph-mon[28138]: pgmap v542: 97 pgs: 97 active+clean; 258 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 11 MiB/s wr, 2.42k op/s 2024-04-26T21:58:33.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:58:33 smithi164 ceph-mon[31916]: pgmap v542: 97 pgs: 97 active+clean; 258 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 11 MiB/s wr, 2.42k op/s 2024-04-26T21:58:35.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:58:35 smithi076 ceph-mon[31823]: pgmap v543: 97 pgs: 97 active+clean; 258 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 11 MiB/s wr, 2.59k op/s 2024-04-26T21:58:35.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:58:35 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:58:35.321 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:58:35 smithi072 ceph-mon[28138]: pgmap v543: 97 pgs: 97 active+clean; 258 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 11 MiB/s wr, 2.59k op/s 2024-04-26T21:58:35.321 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:58:35 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:58:35.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:58:35 smithi164 ceph-mon[31916]: pgmap v543: 97 pgs: 97 active+clean; 258 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 11 MiB/s wr, 2.59k op/s 2024-04-26T21:58:35.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:58:35 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:58:36.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:58:36 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:58:36.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:58:36 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:58:36.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:58:36 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:58:36.321 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:58:36 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:58:36.321 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:58:36 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:58:36.321 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:58:36 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:58:36.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:58:36 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:58:36.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:58:36 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:58:36.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:58:36 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:58:37.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:58:37 smithi076 ceph-mon[31823]: pgmap v544: 97 pgs: 97 active+clean; 258 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 MiB/s wr, 2.62k op/s 2024-04-26T21:58:37.321 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:58:37 smithi072 ceph-mon[28138]: pgmap v544: 97 pgs: 97 active+clean; 258 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 MiB/s wr, 2.62k op/s 2024-04-26T21:58:37.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:58:37 smithi164 ceph-mon[31916]: pgmap v544: 97 pgs: 97 active+clean; 258 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 MiB/s wr, 2.62k op/s 2024-04-26T21:58:38.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:58:38 smithi076 ceph-mon[31823]: pgmap v545: 97 pgs: 97 active+clean; 258 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 11 MiB/s wr, 2.53k op/s 2024-04-26T21:58:38.321 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:58:38 smithi072 ceph-mon[28138]: pgmap v545: 97 pgs: 97 active+clean; 258 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 11 MiB/s wr, 2.53k op/s 2024-04-26T21:58:38.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:58:38 smithi164 ceph-mon[31916]: pgmap v545: 97 pgs: 97 active+clean; 258 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 11 MiB/s wr, 2.53k op/s 2024-04-26T21:58:41.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:58:40 smithi076 ceph-mon[31823]: pgmap v546: 97 pgs: 97 active+clean; 258 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 MiB/s wr, 2.66k op/s 2024-04-26T21:58:41.321 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:58:40 smithi072 ceph-mon[28138]: pgmap v546: 97 pgs: 97 active+clean; 258 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 MiB/s wr, 2.66k op/s 2024-04-26T21:58:41.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:58:40 smithi164 ceph-mon[31916]: pgmap v546: 97 pgs: 97 active+clean; 258 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 MiB/s wr, 2.66k op/s 2024-04-26T21:58:43.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:58:42 smithi076 ceph-mon[31823]: pgmap v547: 97 pgs: 97 active+clean; 258 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 11 MiB/s wr, 2.57k op/s 2024-04-26T21:58:43.321 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:58:42 smithi072 ceph-mon[28138]: pgmap v547: 97 pgs: 97 active+clean; 258 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 11 MiB/s wr, 2.57k op/s 2024-04-26T21:58:43.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:58:42 smithi164 ceph-mon[31916]: pgmap v547: 97 pgs: 97 active+clean; 258 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 11 MiB/s wr, 2.57k op/s 2024-04-26T21:58:45.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:58:44 smithi076 ceph-mon[31823]: pgmap v548: 97 pgs: 97 active+clean; 258 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 12 MiB/s wr, 2.69k op/s 2024-04-26T21:58:45.321 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:58:44 smithi072 ceph-mon[28138]: pgmap v548: 97 pgs: 97 active+clean; 258 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 12 MiB/s wr, 2.69k op/s 2024-04-26T21:58:45.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:58:44 smithi164 ceph-mon[31916]: pgmap v548: 97 pgs: 97 active+clean; 258 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 12 MiB/s wr, 2.69k op/s 2024-04-26T21:58:46.646 INFO:tasks.workunit.client.0.smithi072.stdout:setting up random write file 2024-04-26T21:58:46.646 INFO:tasks.workunit.client.0.smithi072.stdout:done setting up random write file 2024-04-26T21:58:46.646 INFO:tasks.workunit.client.0.smithi072.stdout:starting fsync run 2024-04-26T21:58:46.647 INFO:tasks.workunit.client.0.smithi072.stdout:write time: 0.0084s fsync time: 0.0233s 2024-04-26T21:58:46.647 INFO:tasks.workunit.client.0.smithi072.stdout:write time: 0.0048s fsync time: 0.0245s 2024-04-26T21:58:46.647 INFO:tasks.workunit.client.0.smithi072.stdout:write time: 0.0071s fsync time: 0.0831s 2024-04-26T21:58:46.647 INFO:tasks.workunit.client.0.smithi072.stdout:write time: 0.0048s fsync time: 0.0447s 2024-04-26T21:58:46.647 INFO:tasks.workunit.client.0.smithi072.stdout:write time: 0.0024s fsync time: 0.0362s 2024-04-26T21:58:46.647 INFO:tasks.workunit.client.0.smithi072.stdout:write time: 0.0049s fsync time: 0.0507s 2024-04-26T21:58:46.647 INFO:tasks.workunit.client.0.smithi072.stdout:write time: 0.0033s fsync time: 0.0325s 2024-04-26T21:58:46.647 INFO:tasks.workunit.client.0.smithi072.stdout:write time: 0.0046s fsync time: 0.0330s 2024-04-26T21:58:46.647 INFO:tasks.workunit.client.0.smithi072.stdout:write time: 0.0065s fsync time: 0.0349s 2024-04-26T21:58:46.647 INFO:tasks.workunit.client.0.smithi072.stdout:write time: 0.0029s fsync time: 0.0390s 2024-04-26T21:58:46.647 INFO:tasks.workunit.client.0.smithi072.stdout:write time: 0.0043s fsync time: 0.0510s 2024-04-26T21:58:46.647 INFO:tasks.workunit.client.0.smithi072.stdout:write time: 0.0061s fsync time: 0.0532s 2024-04-26T21:58:46.647 INFO:tasks.workunit.client.0.smithi072.stdout:write time: 0.0041s fsync time: 0.0472s 2024-04-26T21:58:46.647 INFO:tasks.workunit.client.0.smithi072.stdout:write time: 0.0041s fsync time: 0.0445s 2024-04-26T21:58:46.647 INFO:tasks.workunit.client.0.smithi072.stdout:write time: 0.0029s fsync time: 0.0274s 2024-04-26T21:58:46.648 INFO:tasks.workunit.client.0.smithi072.stdout:write time: 0.0042s fsync time: 0.0253s 2024-04-26T21:58:46.648 INFO:tasks.workunit.client.0.smithi072.stdout:write time: 0.0031s fsync time: 0.0498s 2024-04-26T21:58:46.648 INFO:tasks.workunit.client.0.smithi072.stdout:write time: 0.0034s fsync time: 0.0427s 2024-04-26T21:58:46.648 INFO:tasks.workunit.client.0.smithi072.stdout:write time: 0.0028s fsync time: 0.0284s 2024-04-26T21:58:46.648 INFO:tasks.workunit.client.0.smithi072.stdout:write time: 0.0041s fsync time: 0.0458s 2024-04-26T21:58:46.648 INFO:tasks.workunit.client.0.smithi072.stdout:write time: 0.0029s fsync time: 0.0238s 2024-04-26T21:58:46.648 INFO:tasks.workunit.client.0.smithi072.stdout:write time: 0.0035s fsync time: 0.0332s 2024-04-26T21:58:46.648 INFO:tasks.workunit.client.0.smithi072.stdout:write time: 0.0038s fsync time: 0.0432s 2024-04-26T21:58:46.648 INFO:tasks.workunit.client.0.smithi072.stdout:write time: 0.0050s fsync time: 0.0638s 2024-04-26T21:58:46.648 INFO:tasks.workunit.client.0.smithi072.stdout:write time: 0.0030s fsync time: 0.0484s 2024-04-26T21:58:46.648 INFO:tasks.workunit.client.0.smithi072.stdout:write time: 0.0025s fsync time: 0.0425s 2024-04-26T21:58:46.648 INFO:tasks.workunit.client.0.smithi072.stdout:write time: 0.0078s fsync time: 0.0466s 2024-04-26T21:58:46.648 INFO:tasks.workunit.client.0.smithi072.stdout:write time: 0.0029s fsync time: 0.0355s 2024-04-26T21:58:46.648 INFO:tasks.workunit.client.0.smithi072.stdout:write time: 0.0037s fsync time: 0.0534s 2024-04-26T21:58:46.648 INFO:tasks.workunit.client.0.smithi072.stdout:write time: 0.0061s fsync time: 0.0673s 2024-04-26T21:58:46.649 INFO:tasks.workunit.client.0.smithi072.stdout:write time: 0.0024s fsync time: 0.0454s 2024-04-26T21:58:46.649 INFO:tasks.workunit.client.0.smithi072.stdout:write time: 0.0052s fsync time: 0.0375s 2024-04-26T21:58:46.649 INFO:tasks.workunit.client.0.smithi072.stdout:write time: 0.0033s fsync time: 0.0675s 2024-04-26T21:58:46.649 INFO:tasks.workunit.client.0.smithi072.stdout:write time: 0.0030s fsync time: 0.0378s 2024-04-26T21:58:46.649 INFO:tasks.workunit.client.0.smithi072.stdout:write time: 0.0028s fsync time: 0.0644s 2024-04-26T21:58:46.649 INFO:tasks.workunit.client.0.smithi072.stdout:write time: 0.0063s fsync time: 0.0443s 2024-04-26T21:58:46.649 INFO:tasks.workunit.client.0.smithi072.stdout:write time: 0.5711s fsync time: 0.0497s 2024-04-26T21:58:46.649 INFO:tasks.workunit.client.0.smithi072.stdout:write time: 0.0034s fsync time: 0.0377s 2024-04-26T21:58:46.649 INFO:tasks.workunit.client.0.smithi072.stdout:write time: 0.0037s fsync time: 0.0200s 2024-04-26T21:58:46.649 INFO:tasks.workunit.client.0.smithi072.stdout:write time: 0.0031s fsync time: 0.0322s 2024-04-26T21:58:46.649 INFO:tasks.workunit.client.0.smithi072.stdout:write time: 0.0063s fsync time: 0.0499s 2024-04-26T21:58:46.649 INFO:tasks.workunit.client.0.smithi072.stdout:write time: 0.0035s fsync time: 0.0371s 2024-04-26T21:58:46.649 INFO:tasks.workunit.client.0.smithi072.stdout:write time: 0.0042s fsync time: 0.0574s 2024-04-26T21:58:46.649 INFO:tasks.workunit.client.0.smithi072.stdout:write time: 0.0029s fsync time: 0.0365s 2024-04-26T21:58:46.649 INFO:tasks.workunit.client.0.smithi072.stdout:write time: 0.0033s fsync time: 0.0499s 2024-04-26T21:58:46.649 INFO:tasks.workunit.client.0.smithi072.stdout:write time: 0.0032s fsync time: 0.0370s 2024-04-26T21:58:46.650 INFO:tasks.workunit.client.0.smithi072.stdout:write time: 0.0046s fsync time: 0.0496s 2024-04-26T21:58:46.650 INFO:tasks.workunit.client.0.smithi072.stdout:write time: 0.0034s fsync time: 0.0613s 2024-04-26T21:58:46.650 INFO:tasks.workunit.client.0.smithi072.stdout:write time: 0.0026s fsync time: 0.0505s 2024-04-26T21:58:46.650 INFO:tasks.workunit.client.0.smithi072.stdout:write time: 0.0039s fsync time: 0.0569s 2024-04-26T21:58:46.650 INFO:tasks.workunit.client.0.smithi072.stdout:write time: 0.0026s fsync time: 0.0592s 2024-04-26T21:58:46.650 INFO:tasks.workunit.client.0.smithi072.stdout:write time: 0.0027s fsync time: 0.0579s 2024-04-26T21:58:46.650 INFO:tasks.workunit.client.0.smithi072.stdout:write time: 0.0041s fsync time: 0.0529s 2024-04-26T21:58:46.650 INFO:tasks.workunit.client.0.smithi072.stdout:write time: 0.0029s fsync time: 0.0442s 2024-04-26T21:58:46.650 INFO:tasks.workunit.client.0.smithi072.stdout:write time: 0.0025s fsync time: 0.0469s 2024-04-26T21:58:46.650 INFO:tasks.workunit.client.0.smithi072.stdout:write time: 0.0052s fsync time: 0.0575s 2024-04-26T21:58:46.650 INFO:tasks.workunit.client.0.smithi072.stdout:write time: 0.0050s fsync time: 0.0259s 2024-04-26T21:58:46.650 INFO:tasks.workunit.client.0.smithi072.stdout:write time: 0.0060s fsync time: 0.0275s 2024-04-26T21:58:46.650 INFO:tasks.workunit.client.0.smithi072.stdout:run done 58 fsyncs total, killing random writer 2024-04-26T21:58:46.653 INFO:tasks.workunit.client.0.smithi072.stderr:+ echo /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin 2024-04-26T21:58:46.653 INFO:tasks.workunit.client.0.smithi072.stdout:/home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin 2024-04-26T21:58:46.653 INFO:tasks.workunit.client.0.smithi072.stderr:+ whereis lsof 2024-04-26T21:58:47.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:58:46 smithi076 ceph-mon[31823]: pgmap v549: 97 pgs: 97 active+clean; 258 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 MiB/s wr, 2.67k op/s 2024-04-26T21:58:47.321 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:58:46 smithi072 ceph-mon[28138]: pgmap v549: 97 pgs: 97 active+clean; 258 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 MiB/s wr, 2.67k op/s 2024-04-26T21:58:47.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:58:46 smithi164 ceph-mon[31916]: pgmap v549: 97 pgs: 97 active+clean; 258 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 MiB/s wr, 2.67k op/s 2024-04-26T21:58:48.386 INFO:tasks.workunit.client.0.smithi072.stdout:lsof: /usr/bin/lsof /usr/share/man/man1/lsof.1.gz 2024-04-26T21:58:48.387 INFO:tasks.workunit.client.0.smithi072.stderr:+ lsof 2024-04-26T21:58:48.416 INFO:tasks.workunit.client.0.smithi072.stderr:lsof: WARNING: can't stat() overlay file system /var/lib/containers/storage/overlay/fc10633a6d40068493c7dfddd017320787b69b18da25fde4000fafcf27d247e0/merged 2024-04-26T21:58:48.416 INFO:tasks.workunit.client.0.smithi072.stderr: Output information may be incomplete. 2024-04-26T21:58:48.416 INFO:tasks.workunit.client.0.smithi072.stderr:lsof: WARNING: can't stat() overlay file system /var/lib/containers/storage/overlay/2ec2fde413c8408d1bc1f16441821759667c84aa089da73c28bd2bd433c0a260/merged 2024-04-26T21:58:48.416 INFO:tasks.workunit.client.0.smithi072.stderr: Output information may be incomplete. 2024-04-26T21:58:48.421 INFO:tasks.workunit.client.0.smithi072.stderr:lsof: WARNING: can't stat() overlay file system /var/lib/containers/storage/overlay/30c53f8127e8dc9f9ac2105b666f786acc8b70a23c7de14ef9db58f7906cc494/merged 2024-04-26T21:58:48.421 INFO:tasks.workunit.client.0.smithi072.stderr: Output information may be incomplete. 2024-04-26T21:58:48.421 INFO:tasks.workunit.client.0.smithi072.stderr:lsof: WARNING: can't stat() overlay file system /var/lib/containers/storage/overlay/1995c4009fccdcd981ce20c39a62beaf018ffb29f1f07cbf8a63d752d7d49a45/merged 2024-04-26T21:58:48.421 INFO:tasks.workunit.client.0.smithi072.stderr: Output information may be incomplete. 2024-04-26T21:58:48.421 INFO:tasks.workunit.client.0.smithi072.stderr:lsof: WARNING: can't stat() overlay file system /var/lib/containers/storage/overlay/508b2729647d93be927b6dbbdefe8ed0f6195d21fd27badf6591b6bacb50411c/merged 2024-04-26T21:58:48.421 INFO:tasks.workunit.client.0.smithi072.stderr: Output information may be incomplete. 2024-04-26T21:58:48.421 INFO:tasks.workunit.client.0.smithi072.stderr:lsof: WARNING: can't stat() overlay file system /var/lib/containers/storage/overlay/16de90379b1bac42cc1e6c1e40aae28eb42126ee4f66d77f3a0cd69828a69dc4/merged 2024-04-26T21:58:48.421 INFO:tasks.workunit.client.0.smithi072.stderr: Output information may be incomplete. 2024-04-26T21:58:48.421 INFO:tasks.workunit.client.0.smithi072.stderr:lsof: WARNING: can't stat() overlay file system /var/lib/containers/storage/overlay/970b64e22f7265dc4f12c18b19afa4d7aa53184fd4a59546adb4ae8d4e975587/merged 2024-04-26T21:58:48.422 INFO:tasks.workunit.client.0.smithi072.stderr: Output information may be incomplete. 2024-04-26T21:58:48.422 INFO:tasks.workunit.client.0.smithi072.stderr:lsof: WARNING: can't stat() overlay file system /var/lib/containers/storage/overlay/4204b3eb66b7de0f9298c94fc5d0fcd9ae3237b1e1221f068db9278a0b120594/merged 2024-04-26T21:58:48.422 INFO:tasks.workunit.client.0.smithi072.stderr: Output information may be incomplete. 2024-04-26T21:58:48.422 INFO:tasks.workunit.client.0.smithi072.stderr:lsof: WARNING: can't stat() overlay file system /var/lib/containers/storage/overlay/aba746a21fc88dc5f32e1172cd29925901bb88caf596beede12f956fe6bfe4d6/merged 2024-04-26T21:58:48.422 INFO:tasks.workunit.client.0.smithi072.stderr: Output information may be incomplete. 2024-04-26T21:58:48.422 INFO:tasks.workunit.client.0.smithi072.stderr:lsof: WARNING: can't stat() overlay file system /var/lib/containers/storage/overlay/02f3eceff14af239489105a4d3c82a344f42133958d4e5859d3a36424d74ac76/merged 2024-04-26T21:58:48.422 INFO:tasks.workunit.client.0.smithi072.stderr: Output information may be incomplete. 2024-04-26T21:58:48.577 INFO:tasks.workunit.client.0.smithi072.stdout:COMMAND PID TID TASKCMD USER FD TYPE DEVICE SIZE/OFF NODE NAME 2024-04-26T21:58:48.577 INFO:tasks.workunit.client.0.smithi072.stdout:systemd 1 root cwd unknown /proc/1/cwd (readlink: Permission denied) 2024-04-26T21:58:48.577 INFO:tasks.workunit.client.0.smithi072.stdout:systemd 1 root rtd unknown /proc/1/root (readlink: Permission denied) 2024-04-26T21:58:48.577 INFO:tasks.workunit.client.0.smithi072.stdout:systemd 1 root txt unknown /proc/1/exe (readlink: Permission denied) 2024-04-26T21:58:48.577 INFO:tasks.workunit.client.0.smithi072.stdout:systemd 1 root NOFD /proc/1/fd (opendir: Permission denied) 2024-04-26T21:58:48.577 INFO:tasks.workunit.client.0.smithi072.stdout:kthreadd 2 root cwd unknown /proc/2/cwd (readlink: Permission denied) 2024-04-26T21:58:48.577 INFO:tasks.workunit.client.0.smithi072.stdout:kthreadd 2 root rtd unknown /proc/2/root (readlink: Permission denied) 2024-04-26T21:58:48.577 INFO:tasks.workunit.client.0.smithi072.stdout:kthreadd 2 root txt unknown /proc/2/exe (readlink: Permission denied) 2024-04-26T21:58:48.577 INFO:tasks.workunit.client.0.smithi072.stdout:kthreadd 2 root NOFD /proc/2/fd (opendir: Permission denied) 2024-04-26T21:58:48.577 INFO:tasks.workunit.client.0.smithi072.stdout:pool_work 3 root cwd unknown /proc/3/cwd (readlink: Permission denied) 2024-04-26T21:58:48.577 INFO:tasks.workunit.client.0.smithi072.stdout:pool_work 3 root rtd unknown /proc/3/root (readlink: Permission denied) 2024-04-26T21:58:48.578 INFO:tasks.workunit.client.0.smithi072.stdout:pool_work 3 root txt unknown /proc/3/exe (readlink: Permission denied) 2024-04-26T21:58:48.578 INFO:tasks.workunit.client.0.smithi072.stdout:pool_work 3 root NOFD /proc/3/fd (opendir: Permission denied) 2024-04-26T21:58:48.578 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 4 root cwd unknown /proc/4/cwd (readlink: Permission denied) 2024-04-26T21:58:48.578 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 4 root rtd unknown /proc/4/root (readlink: Permission denied) 2024-04-26T21:58:48.578 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 4 root txt unknown /proc/4/exe (readlink: Permission denied) 2024-04-26T21:58:48.578 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 4 root NOFD /proc/4/fd (opendir: Permission denied) 2024-04-26T21:58:48.578 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 5 root cwd unknown /proc/5/cwd (readlink: Permission denied) 2024-04-26T21:58:48.578 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 5 root rtd unknown /proc/5/root (readlink: Permission denied) 2024-04-26T21:58:48.578 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 5 root txt unknown /proc/5/exe (readlink: Permission denied) 2024-04-26T21:58:48.578 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 5 root NOFD /proc/5/fd (opendir: Permission denied) 2024-04-26T21:58:48.578 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 6 root cwd unknown /proc/6/cwd (readlink: Permission denied) 2024-04-26T21:58:48.579 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 6 root rtd unknown /proc/6/root (readlink: Permission denied) 2024-04-26T21:58:48.579 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 6 root txt unknown /proc/6/exe (readlink: Permission denied) 2024-04-26T21:58:48.579 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 6 root NOFD /proc/6/fd (opendir: Permission denied) 2024-04-26T21:58:48.579 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/0 9 root cwd unknown /proc/9/cwd (readlink: Permission denied) 2024-04-26T21:58:48.579 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/0 9 root rtd unknown /proc/9/root (readlink: Permission denied) 2024-04-26T21:58:48.579 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/0 9 root txt unknown /proc/9/exe (readlink: Permission denied) 2024-04-26T21:58:48.579 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/0 9 root NOFD /proc/9/fd (opendir: Permission denied) 2024-04-26T21:58:48.579 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/u 10 root cwd unknown /proc/10/cwd (readlink: Permission denied) 2024-04-26T21:58:48.579 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/u 10 root rtd unknown /proc/10/root (readlink: Permission denied) 2024-04-26T21:58:48.579 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/u 10 root txt unknown /proc/10/exe (readlink: Permission denied) 2024-04-26T21:58:48.579 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/u 10 root NOFD /proc/10/fd (opendir: Permission denied) 2024-04-26T21:58:48.579 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 11 root cwd unknown /proc/11/cwd (readlink: Permission denied) 2024-04-26T21:58:48.579 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 11 root rtd unknown /proc/11/root (readlink: Permission denied) 2024-04-26T21:58:48.579 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 11 root txt unknown /proc/11/exe (readlink: Permission denied) 2024-04-26T21:58:48.580 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 11 root NOFD /proc/11/fd (opendir: Permission denied) 2024-04-26T21:58:48.580 INFO:tasks.workunit.client.0.smithi072.stdout:rcu_tasks 12 root cwd unknown /proc/12/cwd (readlink: Permission denied) 2024-04-26T21:58:48.580 INFO:tasks.workunit.client.0.smithi072.stdout:rcu_tasks 12 root rtd unknown /proc/12/root (readlink: Permission denied) 2024-04-26T21:58:48.580 INFO:tasks.workunit.client.0.smithi072.stdout:rcu_tasks 12 root txt unknown /proc/12/exe (readlink: Permission denied) 2024-04-26T21:58:48.580 INFO:tasks.workunit.client.0.smithi072.stdout:rcu_tasks 12 root NOFD /proc/12/fd (opendir: Permission denied) 2024-04-26T21:58:48.580 INFO:tasks.workunit.client.0.smithi072.stdout:rcu_tasks 13 root cwd unknown /proc/13/cwd (readlink: Permission denied) 2024-04-26T21:58:48.580 INFO:tasks.workunit.client.0.smithi072.stdout:rcu_tasks 13 root rtd unknown /proc/13/root (readlink: Permission denied) 2024-04-26T21:58:48.580 INFO:tasks.workunit.client.0.smithi072.stdout:rcu_tasks 13 root txt unknown /proc/13/exe (readlink: Permission denied) 2024-04-26T21:58:48.580 INFO:tasks.workunit.client.0.smithi072.stdout:rcu_tasks 13 root NOFD /proc/13/fd (opendir: Permission denied) 2024-04-26T21:58:48.580 INFO:tasks.workunit.client.0.smithi072.stdout:rcu_tasks 14 root cwd unknown /proc/14/cwd (readlink: Permission denied) 2024-04-26T21:58:48.580 INFO:tasks.workunit.client.0.smithi072.stdout:rcu_tasks 14 root rtd unknown /proc/14/root (readlink: Permission denied) 2024-04-26T21:58:48.580 INFO:tasks.workunit.client.0.smithi072.stdout:rcu_tasks 14 root txt unknown /proc/14/exe (readlink: Permission denied) 2024-04-26T21:58:48.581 INFO:tasks.workunit.client.0.smithi072.stdout:rcu_tasks 14 root NOFD /proc/14/fd (opendir: Permission denied) 2024-04-26T21:58:48.581 INFO:tasks.workunit.client.0.smithi072.stdout:ksoftirqd 15 root cwd unknown /proc/15/cwd (readlink: Permission denied) 2024-04-26T21:58:48.581 INFO:tasks.workunit.client.0.smithi072.stdout:ksoftirqd 15 root rtd unknown /proc/15/root (readlink: Permission denied) 2024-04-26T21:58:48.581 INFO:tasks.workunit.client.0.smithi072.stdout:ksoftirqd 15 root txt unknown /proc/15/exe (readlink: Permission denied) 2024-04-26T21:58:48.581 INFO:tasks.workunit.client.0.smithi072.stdout:ksoftirqd 15 root NOFD /proc/15/fd (opendir: Permission denied) 2024-04-26T21:58:48.581 INFO:tasks.workunit.client.0.smithi072.stdout:rcu_preem 16 root cwd unknown /proc/16/cwd (readlink: Permission denied) 2024-04-26T21:58:48.581 INFO:tasks.workunit.client.0.smithi072.stdout:rcu_preem 16 root rtd unknown /proc/16/root (readlink: Permission denied) 2024-04-26T21:58:48.581 INFO:tasks.workunit.client.0.smithi072.stdout:rcu_preem 16 root txt unknown /proc/16/exe (readlink: Permission denied) 2024-04-26T21:58:48.581 INFO:tasks.workunit.client.0.smithi072.stdout:rcu_preem 16 root NOFD /proc/16/fd (opendir: Permission denied) 2024-04-26T21:58:48.581 INFO:tasks.workunit.client.0.smithi072.stdout:rcu_exp_p 17 root cwd unknown /proc/17/cwd (readlink: Permission denied) 2024-04-26T21:58:48.581 INFO:tasks.workunit.client.0.smithi072.stdout:rcu_exp_p 17 root rtd unknown /proc/17/root (readlink: Permission denied) 2024-04-26T21:58:48.581 INFO:tasks.workunit.client.0.smithi072.stdout:rcu_exp_p 17 root txt unknown /proc/17/exe (readlink: Permission denied) 2024-04-26T21:58:48.581 INFO:tasks.workunit.client.0.smithi072.stdout:rcu_exp_p 17 root NOFD /proc/17/fd (opendir: Permission denied) 2024-04-26T21:58:48.582 INFO:tasks.workunit.client.0.smithi072.stdout:rcu_exp_g 18 root cwd unknown /proc/18/cwd (readlink: Permission denied) 2024-04-26T21:58:48.582 INFO:tasks.workunit.client.0.smithi072.stdout:rcu_exp_g 18 root rtd unknown /proc/18/root (readlink: Permission denied) 2024-04-26T21:58:48.582 INFO:tasks.workunit.client.0.smithi072.stdout:rcu_exp_g 18 root txt unknown /proc/18/exe (readlink: Permission denied) 2024-04-26T21:58:48.582 INFO:tasks.workunit.client.0.smithi072.stdout:rcu_exp_g 18 root NOFD /proc/18/fd (opendir: Permission denied) 2024-04-26T21:58:48.582 INFO:tasks.workunit.client.0.smithi072.stdout:migration 19 root cwd unknown /proc/19/cwd (readlink: Permission denied) 2024-04-26T21:58:48.582 INFO:tasks.workunit.client.0.smithi072.stdout:migration 19 root rtd unknown /proc/19/root (readlink: Permission denied) 2024-04-26T21:58:48.582 INFO:tasks.workunit.client.0.smithi072.stdout:migration 19 root txt unknown /proc/19/exe (readlink: Permission denied) 2024-04-26T21:58:48.582 INFO:tasks.workunit.client.0.smithi072.stdout:migration 19 root NOFD /proc/19/fd (opendir: Permission denied) 2024-04-26T21:58:48.582 INFO:tasks.workunit.client.0.smithi072.stdout:cpuhp/0 20 root cwd unknown /proc/20/cwd (readlink: Permission denied) 2024-04-26T21:58:48.582 INFO:tasks.workunit.client.0.smithi072.stdout:cpuhp/0 20 root rtd unknown /proc/20/root (readlink: Permission denied) 2024-04-26T21:58:48.583 INFO:tasks.workunit.client.0.smithi072.stdout:cpuhp/0 20 root txt unknown /proc/20/exe (readlink: Permission denied) 2024-04-26T21:58:48.583 INFO:tasks.workunit.client.0.smithi072.stdout:cpuhp/0 20 root NOFD /proc/20/fd (opendir: Permission denied) 2024-04-26T21:58:48.583 INFO:tasks.workunit.client.0.smithi072.stdout:cpuhp/1 21 root cwd unknown /proc/21/cwd (readlink: Permission denied) 2024-04-26T21:58:48.583 INFO:tasks.workunit.client.0.smithi072.stdout:cpuhp/1 21 root rtd unknown /proc/21/root (readlink: Permission denied) 2024-04-26T21:58:48.583 INFO:tasks.workunit.client.0.smithi072.stdout:cpuhp/1 21 root txt unknown /proc/21/exe (readlink: Permission denied) 2024-04-26T21:58:48.583 INFO:tasks.workunit.client.0.smithi072.stdout:cpuhp/1 21 root NOFD /proc/21/fd (opendir: Permission denied) 2024-04-26T21:58:48.583 INFO:tasks.workunit.client.0.smithi072.stdout:migration 22 root cwd unknown /proc/22/cwd (readlink: Permission denied) 2024-04-26T21:58:48.583 INFO:tasks.workunit.client.0.smithi072.stdout:migration 22 root rtd unknown /proc/22/root (readlink: Permission denied) 2024-04-26T21:58:48.583 INFO:tasks.workunit.client.0.smithi072.stdout:migration 22 root txt unknown /proc/22/exe (readlink: Permission denied) 2024-04-26T21:58:48.583 INFO:tasks.workunit.client.0.smithi072.stdout:migration 22 root NOFD /proc/22/fd (opendir: Permission denied) 2024-04-26T21:58:48.583 INFO:tasks.workunit.client.0.smithi072.stdout:ksoftirqd 23 root cwd unknown /proc/23/cwd (readlink: Permission denied) 2024-04-26T21:58:48.583 INFO:tasks.workunit.client.0.smithi072.stdout:ksoftirqd 23 root rtd unknown /proc/23/root (readlink: Permission denied) 2024-04-26T21:58:48.584 INFO:tasks.workunit.client.0.smithi072.stdout:ksoftirqd 23 root txt unknown /proc/23/exe (readlink: Permission denied) 2024-04-26T21:58:48.584 INFO:tasks.workunit.client.0.smithi072.stdout:ksoftirqd 23 root NOFD /proc/23/fd (opendir: Permission denied) 2024-04-26T21:58:48.584 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/1 25 root cwd unknown /proc/25/cwd (readlink: Permission denied) 2024-04-26T21:58:48.584 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/1 25 root rtd unknown /proc/25/root (readlink: Permission denied) 2024-04-26T21:58:48.584 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/1 25 root txt unknown /proc/25/exe (readlink: Permission denied) 2024-04-26T21:58:48.584 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/1 25 root NOFD /proc/25/fd (opendir: Permission denied) 2024-04-26T21:58:48.584 INFO:tasks.workunit.client.0.smithi072.stdout:cpuhp/2 26 root cwd unknown /proc/26/cwd (readlink: Permission denied) 2024-04-26T21:58:48.584 INFO:tasks.workunit.client.0.smithi072.stdout:cpuhp/2 26 root rtd unknown /proc/26/root (readlink: Permission denied) 2024-04-26T21:58:48.584 INFO:tasks.workunit.client.0.smithi072.stdout:cpuhp/2 26 root txt unknown /proc/26/exe (readlink: Permission denied) 2024-04-26T21:58:48.584 INFO:tasks.workunit.client.0.smithi072.stdout:cpuhp/2 26 root NOFD /proc/26/fd (opendir: Permission denied) 2024-04-26T21:58:48.585 INFO:tasks.workunit.client.0.smithi072.stdout:migration 27 root cwd unknown /proc/27/cwd (readlink: Permission denied) 2024-04-26T21:58:48.585 INFO:tasks.workunit.client.0.smithi072.stdout:migration 27 root rtd unknown /proc/27/root (readlink: Permission denied) 2024-04-26T21:58:48.585 INFO:tasks.workunit.client.0.smithi072.stdout:migration 27 root txt unknown /proc/27/exe (readlink: Permission denied) 2024-04-26T21:58:48.585 INFO:tasks.workunit.client.0.smithi072.stdout:migration 27 root NOFD /proc/27/fd (opendir: Permission denied) 2024-04-26T21:58:48.585 INFO:tasks.workunit.client.0.smithi072.stdout:ksoftirqd 28 root cwd unknown /proc/28/cwd (readlink: Permission denied) 2024-04-26T21:58:48.585 INFO:tasks.workunit.client.0.smithi072.stdout:ksoftirqd 28 root rtd unknown /proc/28/root (readlink: Permission denied) 2024-04-26T21:58:48.585 INFO:tasks.workunit.client.0.smithi072.stdout:ksoftirqd 28 root txt unknown /proc/28/exe (readlink: Permission denied) 2024-04-26T21:58:48.585 INFO:tasks.workunit.client.0.smithi072.stdout:ksoftirqd 28 root NOFD /proc/28/fd (opendir: Permission denied) 2024-04-26T21:58:48.585 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/2 30 root cwd unknown /proc/30/cwd (readlink: Permission denied) 2024-04-26T21:58:48.585 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/2 30 root rtd unknown /proc/30/root (readlink: Permission denied) 2024-04-26T21:58:48.585 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/2 30 root txt unknown /proc/30/exe (readlink: Permission denied) 2024-04-26T21:58:48.585 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/2 30 root NOFD /proc/30/fd (opendir: Permission denied) 2024-04-26T21:58:48.585 INFO:tasks.workunit.client.0.smithi072.stdout:cpuhp/3 31 root cwd unknown /proc/31/cwd (readlink: Permission denied) 2024-04-26T21:58:48.585 INFO:tasks.workunit.client.0.smithi072.stdout:cpuhp/3 31 root rtd unknown /proc/31/root (readlink: Permission denied) 2024-04-26T21:58:48.586 INFO:tasks.workunit.client.0.smithi072.stdout:cpuhp/3 31 root txt unknown /proc/31/exe (readlink: Permission denied) 2024-04-26T21:58:48.586 INFO:tasks.workunit.client.0.smithi072.stdout:cpuhp/3 31 root NOFD /proc/31/fd (opendir: Permission denied) 2024-04-26T21:58:48.586 INFO:tasks.workunit.client.0.smithi072.stdout:migration 32 root cwd unknown /proc/32/cwd (readlink: Permission denied) 2024-04-26T21:58:48.586 INFO:tasks.workunit.client.0.smithi072.stdout:migration 32 root rtd unknown /proc/32/root (readlink: Permission denied) 2024-04-26T21:58:48.586 INFO:tasks.workunit.client.0.smithi072.stdout:migration 32 root txt unknown /proc/32/exe (readlink: Permission denied) 2024-04-26T21:58:48.586 INFO:tasks.workunit.client.0.smithi072.stdout:migration 32 root NOFD /proc/32/fd (opendir: Permission denied) 2024-04-26T21:58:48.586 INFO:tasks.workunit.client.0.smithi072.stdout:ksoftirqd 33 root cwd unknown /proc/33/cwd (readlink: Permission denied) 2024-04-26T21:58:48.586 INFO:tasks.workunit.client.0.smithi072.stdout:ksoftirqd 33 root rtd unknown /proc/33/root (readlink: Permission denied) 2024-04-26T21:58:48.586 INFO:tasks.workunit.client.0.smithi072.stdout:ksoftirqd 33 root txt unknown /proc/33/exe (readlink: Permission denied) 2024-04-26T21:58:48.586 INFO:tasks.workunit.client.0.smithi072.stdout:ksoftirqd 33 root NOFD /proc/33/fd (opendir: Permission denied) 2024-04-26T21:58:48.586 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/3 35 root cwd unknown /proc/35/cwd (readlink: Permission denied) 2024-04-26T21:58:48.587 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/3 35 root rtd unknown /proc/35/root (readlink: Permission denied) 2024-04-26T21:58:48.587 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/3 35 root txt unknown /proc/35/exe (readlink: Permission denied) 2024-04-26T21:58:48.587 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/3 35 root NOFD /proc/35/fd (opendir: Permission denied) 2024-04-26T21:58:48.587 INFO:tasks.workunit.client.0.smithi072.stdout:cpuhp/4 36 root cwd unknown /proc/36/cwd (readlink: Permission denied) 2024-04-26T21:58:48.587 INFO:tasks.workunit.client.0.smithi072.stdout:cpuhp/4 36 root rtd unknown /proc/36/root (readlink: Permission denied) 2024-04-26T21:58:48.587 INFO:tasks.workunit.client.0.smithi072.stdout:cpuhp/4 36 root txt unknown /proc/36/exe (readlink: Permission denied) 2024-04-26T21:58:48.587 INFO:tasks.workunit.client.0.smithi072.stdout:cpuhp/4 36 root NOFD /proc/36/fd (opendir: Permission denied) 2024-04-26T21:58:48.587 INFO:tasks.workunit.client.0.smithi072.stdout:migration 37 root cwd unknown /proc/37/cwd (readlink: Permission denied) 2024-04-26T21:58:48.587 INFO:tasks.workunit.client.0.smithi072.stdout:migration 37 root rtd unknown /proc/37/root (readlink: Permission denied) 2024-04-26T21:58:48.587 INFO:tasks.workunit.client.0.smithi072.stdout:migration 37 root txt unknown /proc/37/exe (readlink: Permission denied) 2024-04-26T21:58:48.587 INFO:tasks.workunit.client.0.smithi072.stdout:migration 37 root NOFD /proc/37/fd (opendir: Permission denied) 2024-04-26T21:58:48.588 INFO:tasks.workunit.client.0.smithi072.stdout:ksoftirqd 38 root cwd unknown /proc/38/cwd (readlink: Permission denied) 2024-04-26T21:58:48.588 INFO:tasks.workunit.client.0.smithi072.stdout:ksoftirqd 38 root rtd unknown /proc/38/root (readlink: Permission denied) 2024-04-26T21:58:48.588 INFO:tasks.workunit.client.0.smithi072.stdout:ksoftirqd 38 root txt unknown /proc/38/exe (readlink: Permission denied) 2024-04-26T21:58:48.588 INFO:tasks.workunit.client.0.smithi072.stdout:ksoftirqd 38 root NOFD /proc/38/fd (opendir: Permission denied) 2024-04-26T21:58:48.588 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/4 40 root cwd unknown /proc/40/cwd (readlink: Permission denied) 2024-04-26T21:58:48.588 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/4 40 root rtd unknown /proc/40/root (readlink: Permission denied) 2024-04-26T21:58:48.588 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/4 40 root txt unknown /proc/40/exe (readlink: Permission denied) 2024-04-26T21:58:48.588 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/4 40 root NOFD /proc/40/fd (opendir: Permission denied) 2024-04-26T21:58:48.588 INFO:tasks.workunit.client.0.smithi072.stdout:cpuhp/5 41 root cwd unknown /proc/41/cwd (readlink: Permission denied) 2024-04-26T21:58:48.588 INFO:tasks.workunit.client.0.smithi072.stdout:cpuhp/5 41 root rtd unknown /proc/41/root (readlink: Permission denied) 2024-04-26T21:58:48.588 INFO:tasks.workunit.client.0.smithi072.stdout:cpuhp/5 41 root txt unknown /proc/41/exe (readlink: Permission denied) 2024-04-26T21:58:48.588 INFO:tasks.workunit.client.0.smithi072.stdout:cpuhp/5 41 root NOFD /proc/41/fd (opendir: Permission denied) 2024-04-26T21:58:48.589 INFO:tasks.workunit.client.0.smithi072.stdout:migration 42 root cwd unknown /proc/42/cwd (readlink: Permission denied) 2024-04-26T21:58:48.589 INFO:tasks.workunit.client.0.smithi072.stdout:migration 42 root rtd unknown /proc/42/root (readlink: Permission denied) 2024-04-26T21:58:48.589 INFO:tasks.workunit.client.0.smithi072.stdout:migration 42 root txt unknown /proc/42/exe (readlink: Permission denied) 2024-04-26T21:58:48.589 INFO:tasks.workunit.client.0.smithi072.stdout:migration 42 root NOFD /proc/42/fd (opendir: Permission denied) 2024-04-26T21:58:48.589 INFO:tasks.workunit.client.0.smithi072.stdout:ksoftirqd 43 root cwd unknown /proc/43/cwd (readlink: Permission denied) 2024-04-26T21:58:48.589 INFO:tasks.workunit.client.0.smithi072.stdout:ksoftirqd 43 root rtd unknown /proc/43/root (readlink: Permission denied) 2024-04-26T21:58:48.589 INFO:tasks.workunit.client.0.smithi072.stdout:ksoftirqd 43 root txt unknown /proc/43/exe (readlink: Permission denied) 2024-04-26T21:58:48.589 INFO:tasks.workunit.client.0.smithi072.stdout:ksoftirqd 43 root NOFD /proc/43/fd (opendir: Permission denied) 2024-04-26T21:58:48.589 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/5 45 root cwd unknown /proc/45/cwd (readlink: Permission denied) 2024-04-26T21:58:48.589 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/5 45 root rtd unknown /proc/45/root (readlink: Permission denied) 2024-04-26T21:58:48.589 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/5 45 root txt unknown /proc/45/exe (readlink: Permission denied) 2024-04-26T21:58:48.589 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/5 45 root NOFD /proc/45/fd (opendir: Permission denied) 2024-04-26T21:58:48.590 INFO:tasks.workunit.client.0.smithi072.stdout:cpuhp/6 46 root cwd unknown /proc/46/cwd (readlink: Permission denied) 2024-04-26T21:58:48.590 INFO:tasks.workunit.client.0.smithi072.stdout:cpuhp/6 46 root rtd unknown /proc/46/root (readlink: Permission denied) 2024-04-26T21:58:48.590 INFO:tasks.workunit.client.0.smithi072.stdout:cpuhp/6 46 root txt unknown /proc/46/exe (readlink: Permission denied) 2024-04-26T21:58:48.590 INFO:tasks.workunit.client.0.smithi072.stdout:cpuhp/6 46 root NOFD /proc/46/fd (opendir: Permission denied) 2024-04-26T21:58:48.590 INFO:tasks.workunit.client.0.smithi072.stdout:migration 47 root cwd unknown /proc/47/cwd (readlink: Permission denied) 2024-04-26T21:58:48.590 INFO:tasks.workunit.client.0.smithi072.stdout:migration 47 root rtd unknown /proc/47/root (readlink: Permission denied) 2024-04-26T21:58:48.590 INFO:tasks.workunit.client.0.smithi072.stdout:migration 47 root txt unknown /proc/47/exe (readlink: Permission denied) 2024-04-26T21:58:48.590 INFO:tasks.workunit.client.0.smithi072.stdout:migration 47 root NOFD /proc/47/fd (opendir: Permission denied) 2024-04-26T21:58:48.590 INFO:tasks.workunit.client.0.smithi072.stdout:ksoftirqd 48 root cwd unknown /proc/48/cwd (readlink: Permission denied) 2024-04-26T21:58:48.590 INFO:tasks.workunit.client.0.smithi072.stdout:ksoftirqd 48 root rtd unknown /proc/48/root (readlink: Permission denied) 2024-04-26T21:58:48.591 INFO:tasks.workunit.client.0.smithi072.stdout:ksoftirqd 48 root txt unknown /proc/48/exe (readlink: Permission denied) 2024-04-26T21:58:48.591 INFO:tasks.workunit.client.0.smithi072.stdout:ksoftirqd 48 root NOFD /proc/48/fd (opendir: Permission denied) 2024-04-26T21:58:48.591 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/6 50 root cwd unknown /proc/50/cwd (readlink: Permission denied) 2024-04-26T21:58:48.591 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/6 50 root rtd unknown /proc/50/root (readlink: Permission denied) 2024-04-26T21:58:48.591 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/6 50 root txt unknown /proc/50/exe (readlink: Permission denied) 2024-04-26T21:58:48.591 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/6 50 root NOFD /proc/50/fd (opendir: Permission denied) 2024-04-26T21:58:48.591 INFO:tasks.workunit.client.0.smithi072.stdout:cpuhp/7 51 root cwd unknown /proc/51/cwd (readlink: Permission denied) 2024-04-26T21:58:48.591 INFO:tasks.workunit.client.0.smithi072.stdout:cpuhp/7 51 root rtd unknown /proc/51/root (readlink: Permission denied) 2024-04-26T21:58:48.591 INFO:tasks.workunit.client.0.smithi072.stdout:cpuhp/7 51 root txt unknown /proc/51/exe (readlink: Permission denied) 2024-04-26T21:58:48.591 INFO:tasks.workunit.client.0.smithi072.stdout:cpuhp/7 51 root NOFD /proc/51/fd (opendir: Permission denied) 2024-04-26T21:58:48.591 INFO:tasks.workunit.client.0.smithi072.stdout:migration 52 root cwd unknown /proc/52/cwd (readlink: Permission denied) 2024-04-26T21:58:48.591 INFO:tasks.workunit.client.0.smithi072.stdout:migration 52 root rtd unknown /proc/52/root (readlink: Permission denied) 2024-04-26T21:58:48.591 INFO:tasks.workunit.client.0.smithi072.stdout:migration 52 root txt unknown /proc/52/exe (readlink: Permission denied) 2024-04-26T21:58:48.592 INFO:tasks.workunit.client.0.smithi072.stdout:migration 52 root NOFD /proc/52/fd (opendir: Permission denied) 2024-04-26T21:58:48.592 INFO:tasks.workunit.client.0.smithi072.stdout:ksoftirqd 53 root cwd unknown /proc/53/cwd (readlink: Permission denied) 2024-04-26T21:58:48.592 INFO:tasks.workunit.client.0.smithi072.stdout:ksoftirqd 53 root rtd unknown /proc/53/root (readlink: Permission denied) 2024-04-26T21:58:48.592 INFO:tasks.workunit.client.0.smithi072.stdout:ksoftirqd 53 root txt unknown /proc/53/exe (readlink: Permission denied) 2024-04-26T21:58:48.592 INFO:tasks.workunit.client.0.smithi072.stdout:ksoftirqd 53 root NOFD /proc/53/fd (opendir: Permission denied) 2024-04-26T21:58:48.592 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/7 55 root cwd unknown /proc/55/cwd (readlink: Permission denied) 2024-04-26T21:58:48.592 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/7 55 root rtd unknown /proc/55/root (readlink: Permission denied) 2024-04-26T21:58:48.592 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/7 55 root txt unknown /proc/55/exe (readlink: Permission denied) 2024-04-26T21:58:48.592 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/7 55 root NOFD /proc/55/fd (opendir: Permission denied) 2024-04-26T21:58:48.592 INFO:tasks.workunit.client.0.smithi072.stdout:kdevtmpfs 56 root cwd unknown /proc/56/cwd (readlink: Permission denied) 2024-04-26T21:58:48.592 INFO:tasks.workunit.client.0.smithi072.stdout:kdevtmpfs 56 root rtd unknown /proc/56/root (readlink: Permission denied) 2024-04-26T21:58:48.592 INFO:tasks.workunit.client.0.smithi072.stdout:kdevtmpfs 56 root txt unknown /proc/56/exe (readlink: Permission denied) 2024-04-26T21:58:48.592 INFO:tasks.workunit.client.0.smithi072.stdout:kdevtmpfs 56 root NOFD /proc/56/fd (opendir: Permission denied) 2024-04-26T21:58:48.592 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 57 root cwd unknown /proc/57/cwd (readlink: Permission denied) 2024-04-26T21:58:48.593 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 57 root rtd unknown /proc/57/root (readlink: Permission denied) 2024-04-26T21:58:48.593 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 57 root txt unknown /proc/57/exe (readlink: Permission denied) 2024-04-26T21:58:48.593 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 57 root NOFD /proc/57/fd (opendir: Permission denied) 2024-04-26T21:58:48.593 INFO:tasks.workunit.client.0.smithi072.stdout:kauditd 58 root cwd unknown /proc/58/cwd (readlink: Permission denied) 2024-04-26T21:58:48.593 INFO:tasks.workunit.client.0.smithi072.stdout:kauditd 58 root rtd unknown /proc/58/root (readlink: Permission denied) 2024-04-26T21:58:48.593 INFO:tasks.workunit.client.0.smithi072.stdout:kauditd 58 root txt unknown /proc/58/exe (readlink: Permission denied) 2024-04-26T21:58:48.593 INFO:tasks.workunit.client.0.smithi072.stdout:kauditd 58 root NOFD /proc/58/fd (opendir: Permission denied) 2024-04-26T21:58:48.593 INFO:tasks.workunit.client.0.smithi072.stdout:khungtask 59 root cwd unknown /proc/59/cwd (readlink: Permission denied) 2024-04-26T21:58:48.593 INFO:tasks.workunit.client.0.smithi072.stdout:khungtask 59 root rtd unknown /proc/59/root (readlink: Permission denied) 2024-04-26T21:58:48.593 INFO:tasks.workunit.client.0.smithi072.stdout:khungtask 59 root txt unknown /proc/59/exe (readlink: Permission denied) 2024-04-26T21:58:48.593 INFO:tasks.workunit.client.0.smithi072.stdout:khungtask 59 root NOFD /proc/59/fd (opendir: Permission denied) 2024-04-26T21:58:48.593 INFO:tasks.workunit.client.0.smithi072.stdout:oom_reape 61 root cwd unknown /proc/61/cwd (readlink: Permission denied) 2024-04-26T21:58:48.593 INFO:tasks.workunit.client.0.smithi072.stdout:oom_reape 61 root rtd unknown /proc/61/root (readlink: Permission denied) 2024-04-26T21:58:48.594 INFO:tasks.workunit.client.0.smithi072.stdout:oom_reape 61 root txt unknown /proc/61/exe (readlink: Permission denied) 2024-04-26T21:58:48.594 INFO:tasks.workunit.client.0.smithi072.stdout:oom_reape 61 root NOFD /proc/61/fd (opendir: Permission denied) 2024-04-26T21:58:48.594 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 63 root cwd unknown /proc/63/cwd (readlink: Permission denied) 2024-04-26T21:58:48.594 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 63 root rtd unknown /proc/63/root (readlink: Permission denied) 2024-04-26T21:58:48.594 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 63 root txt unknown /proc/63/exe (readlink: Permission denied) 2024-04-26T21:58:48.594 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 63 root NOFD /proc/63/fd (opendir: Permission denied) 2024-04-26T21:58:48.594 INFO:tasks.workunit.client.0.smithi072.stdout:kcompactd 64 root cwd unknown /proc/64/cwd (readlink: Permission denied) 2024-04-26T21:58:48.594 INFO:tasks.workunit.client.0.smithi072.stdout:kcompactd 64 root rtd unknown /proc/64/root (readlink: Permission denied) 2024-04-26T21:58:48.594 INFO:tasks.workunit.client.0.smithi072.stdout:kcompactd 64 root txt unknown /proc/64/exe (readlink: Permission denied) 2024-04-26T21:58:48.594 INFO:tasks.workunit.client.0.smithi072.stdout:kcompactd 64 root NOFD /proc/64/fd (opendir: Permission denied) 2024-04-26T21:58:48.594 INFO:tasks.workunit.client.0.smithi072.stdout:ksmd 65 root cwd unknown /proc/65/cwd (readlink: Permission denied) 2024-04-26T21:58:48.594 INFO:tasks.workunit.client.0.smithi072.stdout:ksmd 65 root rtd unknown /proc/65/root (readlink: Permission denied) 2024-04-26T21:58:48.594 INFO:tasks.workunit.client.0.smithi072.stdout:ksmd 65 root txt unknown /proc/65/exe (readlink: Permission denied) 2024-04-26T21:58:48.594 INFO:tasks.workunit.client.0.smithi072.stdout:ksmd 65 root NOFD /proc/65/fd (opendir: Permission denied) 2024-04-26T21:58:48.595 INFO:tasks.workunit.client.0.smithi072.stdout:khugepage 66 root cwd unknown /proc/66/cwd (readlink: Permission denied) 2024-04-26T21:58:48.595 INFO:tasks.workunit.client.0.smithi072.stdout:khugepage 66 root rtd unknown /proc/66/root (readlink: Permission denied) 2024-04-26T21:58:48.595 INFO:tasks.workunit.client.0.smithi072.stdout:khugepage 66 root txt unknown /proc/66/exe (readlink: Permission denied) 2024-04-26T21:58:48.595 INFO:tasks.workunit.client.0.smithi072.stdout:khugepage 66 root NOFD /proc/66/fd (opendir: Permission denied) 2024-04-26T21:58:48.595 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 67 root cwd unknown /proc/67/cwd (readlink: Permission denied) 2024-04-26T21:58:48.595 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 67 root rtd unknown /proc/67/root (readlink: Permission denied) 2024-04-26T21:58:48.595 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 67 root txt unknown /proc/67/exe (readlink: Permission denied) 2024-04-26T21:58:48.595 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 67 root NOFD /proc/67/fd (opendir: Permission denied) 2024-04-26T21:58:48.595 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 68 root cwd unknown /proc/68/cwd (readlink: Permission denied) 2024-04-26T21:58:48.595 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 68 root rtd unknown /proc/68/root (readlink: Permission denied) 2024-04-26T21:58:48.595 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 68 root txt unknown /proc/68/exe (readlink: Permission denied) 2024-04-26T21:58:48.595 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 68 root NOFD /proc/68/fd (opendir: Permission denied) 2024-04-26T21:58:48.595 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 69 root cwd unknown /proc/69/cwd (readlink: Permission denied) 2024-04-26T21:58:48.596 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 69 root rtd unknown /proc/69/root (readlink: Permission denied) 2024-04-26T21:58:48.596 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 69 root txt unknown /proc/69/exe (readlink: Permission denied) 2024-04-26T21:58:48.596 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 69 root NOFD /proc/69/fd (opendir: Permission denied) 2024-04-26T21:58:48.596 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 70 root cwd unknown /proc/70/cwd (readlink: Permission denied) 2024-04-26T21:58:48.596 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 70 root rtd unknown /proc/70/root (readlink: Permission denied) 2024-04-26T21:58:48.596 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 70 root txt unknown /proc/70/exe (readlink: Permission denied) 2024-04-26T21:58:48.596 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 70 root NOFD /proc/70/fd (opendir: Permission denied) 2024-04-26T21:58:48.596 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/4 74 root cwd unknown /proc/74/cwd (readlink: Permission denied) 2024-04-26T21:58:48.596 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/4 74 root rtd unknown /proc/74/root (readlink: Permission denied) 2024-04-26T21:58:48.596 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/4 74 root txt unknown /proc/74/exe (readlink: Permission denied) 2024-04-26T21:58:48.596 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/4 74 root NOFD /proc/74/fd (opendir: Permission denied) 2024-04-26T21:58:48.596 INFO:tasks.workunit.client.0.smithi072.stdout:irq/9-acp 78 root cwd unknown /proc/78/cwd (readlink: Permission denied) 2024-04-26T21:58:48.596 INFO:tasks.workunit.client.0.smithi072.stdout:irq/9-acp 78 root rtd unknown /proc/78/root (readlink: Permission denied) 2024-04-26T21:58:48.597 INFO:tasks.workunit.client.0.smithi072.stdout:irq/9-acp 78 root txt unknown /proc/78/exe (readlink: Permission denied) 2024-04-26T21:58:48.597 INFO:tasks.workunit.client.0.smithi072.stdout:irq/9-acp 78 root NOFD /proc/78/fd (opendir: Permission denied) 2024-04-26T21:58:48.597 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 79 root cwd unknown /proc/79/cwd (readlink: Permission denied) 2024-04-26T21:58:48.597 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 79 root rtd unknown /proc/79/root (readlink: Permission denied) 2024-04-26T21:58:48.597 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 79 root txt unknown /proc/79/exe (readlink: Permission denied) 2024-04-26T21:58:48.597 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 79 root NOFD /proc/79/fd (opendir: Permission denied) 2024-04-26T21:58:48.597 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 80 root cwd unknown /proc/80/cwd (readlink: Permission denied) 2024-04-26T21:58:48.597 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 80 root rtd unknown /proc/80/root (readlink: Permission denied) 2024-04-26T21:58:48.597 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 80 root txt unknown /proc/80/exe (readlink: Permission denied) 2024-04-26T21:58:48.597 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 80 root NOFD /proc/80/fd (opendir: Permission denied) 2024-04-26T21:58:48.597 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 81 root cwd unknown /proc/81/cwd (readlink: Permission denied) 2024-04-26T21:58:48.597 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 81 root rtd unknown /proc/81/root (readlink: Permission denied) 2024-04-26T21:58:48.597 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 81 root txt unknown /proc/81/exe (readlink: Permission denied) 2024-04-26T21:58:48.597 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 81 root NOFD /proc/81/fd (opendir: Permission denied) 2024-04-26T21:58:48.598 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 82 root cwd unknown /proc/82/cwd (readlink: Permission denied) 2024-04-26T21:58:48.598 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 82 root rtd unknown /proc/82/root (readlink: Permission denied) 2024-04-26T21:58:48.598 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 82 root txt unknown /proc/82/exe (readlink: Permission denied) 2024-04-26T21:58:48.598 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 82 root NOFD /proc/82/fd (opendir: Permission denied) 2024-04-26T21:58:48.598 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 83 root cwd unknown /proc/83/cwd (readlink: Permission denied) 2024-04-26T21:58:48.598 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 83 root rtd unknown /proc/83/root (readlink: Permission denied) 2024-04-26T21:58:48.598 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 83 root txt unknown /proc/83/exe (readlink: Permission denied) 2024-04-26T21:58:48.598 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 83 root NOFD /proc/83/fd (opendir: Permission denied) 2024-04-26T21:58:48.598 INFO:tasks.workunit.client.0.smithi072.stdout:watchdogd 84 root cwd unknown /proc/84/cwd (readlink: Permission denied) 2024-04-26T21:58:48.598 INFO:tasks.workunit.client.0.smithi072.stdout:watchdogd 84 root rtd unknown /proc/84/root (readlink: Permission denied) 2024-04-26T21:58:48.598 INFO:tasks.workunit.client.0.smithi072.stdout:watchdogd 84 root txt unknown /proc/84/exe (readlink: Permission denied) 2024-04-26T21:58:48.598 INFO:tasks.workunit.client.0.smithi072.stdout:watchdogd 84 root NOFD /proc/84/fd (opendir: Permission denied) 2024-04-26T21:58:48.598 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/0 85 root cwd unknown /proc/85/cwd (readlink: Permission denied) 2024-04-26T21:58:48.599 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/0 85 root rtd unknown /proc/85/root (readlink: Permission denied) 2024-04-26T21:58:48.599 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/0 85 root txt unknown /proc/85/exe (readlink: Permission denied) 2024-04-26T21:58:48.599 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/0 85 root NOFD /proc/85/fd (opendir: Permission denied) 2024-04-26T21:58:48.599 INFO:tasks.workunit.client.0.smithi072.stdout:kswapd0 86 root cwd unknown /proc/86/cwd (readlink: Permission denied) 2024-04-26T21:58:48.599 INFO:tasks.workunit.client.0.smithi072.stdout:kswapd0 86 root rtd unknown /proc/86/root (readlink: Permission denied) 2024-04-26T21:58:48.599 INFO:tasks.workunit.client.0.smithi072.stdout:kswapd0 86 root txt unknown /proc/86/exe (readlink: Permission denied) 2024-04-26T21:58:48.599 INFO:tasks.workunit.client.0.smithi072.stdout:kswapd0 86 root NOFD /proc/86/fd (opendir: Permission denied) 2024-04-26T21:58:48.599 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 88 root cwd unknown /proc/88/cwd (readlink: Permission denied) 2024-04-26T21:58:48.599 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 88 root rtd unknown /proc/88/root (readlink: Permission denied) 2024-04-26T21:58:48.599 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 88 root txt unknown /proc/88/exe (readlink: Permission denied) 2024-04-26T21:58:48.599 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 88 root NOFD /proc/88/fd (opendir: Permission denied) 2024-04-26T21:58:48.599 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 91 root cwd unknown /proc/91/cwd (readlink: Permission denied) 2024-04-26T21:58:48.599 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 91 root rtd unknown /proc/91/root (readlink: Permission denied) 2024-04-26T21:58:48.600 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 91 root txt unknown /proc/91/exe (readlink: Permission denied) 2024-04-26T21:58:48.600 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 91 root NOFD /proc/91/fd (opendir: Permission denied) 2024-04-26T21:58:48.600 INFO:tasks.workunit.client.0.smithi072.stdout:scsi_eh_0 93 root cwd unknown /proc/93/cwd (readlink: Permission denied) 2024-04-26T21:58:48.600 INFO:tasks.workunit.client.0.smithi072.stdout:scsi_eh_0 93 root rtd unknown /proc/93/root (readlink: Permission denied) 2024-04-26T21:58:48.600 INFO:tasks.workunit.client.0.smithi072.stdout:scsi_eh_0 93 root txt unknown /proc/93/exe (readlink: Permission denied) 2024-04-26T21:58:48.600 INFO:tasks.workunit.client.0.smithi072.stdout:scsi_eh_0 93 root NOFD /proc/93/fd (opendir: Permission denied) 2024-04-26T21:58:48.600 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 94 root cwd unknown /proc/94/cwd (readlink: Permission denied) 2024-04-26T21:58:48.600 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 94 root rtd unknown /proc/94/root (readlink: Permission denied) 2024-04-26T21:58:48.600 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 94 root txt unknown /proc/94/exe (readlink: Permission denied) 2024-04-26T21:58:48.600 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 94 root NOFD /proc/94/fd (opendir: Permission denied) 2024-04-26T21:58:48.600 INFO:tasks.workunit.client.0.smithi072.stdout:scsi_eh_1 95 root cwd unknown /proc/95/cwd (readlink: Permission denied) 2024-04-26T21:58:48.600 INFO:tasks.workunit.client.0.smithi072.stdout:scsi_eh_1 95 root rtd unknown /proc/95/root (readlink: Permission denied) 2024-04-26T21:58:48.600 INFO:tasks.workunit.client.0.smithi072.stdout:scsi_eh_1 95 root txt unknown /proc/95/exe (readlink: Permission denied) 2024-04-26T21:58:48.601 INFO:tasks.workunit.client.0.smithi072.stdout:scsi_eh_1 95 root NOFD /proc/95/fd (opendir: Permission denied) 2024-04-26T21:58:48.601 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 96 root cwd unknown /proc/96/cwd (readlink: Permission denied) 2024-04-26T21:58:48.601 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 96 root rtd unknown /proc/96/root (readlink: Permission denied) 2024-04-26T21:58:48.601 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 96 root txt unknown /proc/96/exe (readlink: Permission denied) 2024-04-26T21:58:48.601 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 96 root NOFD /proc/96/fd (opendir: Permission denied) 2024-04-26T21:58:48.601 INFO:tasks.workunit.client.0.smithi072.stdout:scsi_eh_2 97 root cwd unknown /proc/97/cwd (readlink: Permission denied) 2024-04-26T21:58:48.601 INFO:tasks.workunit.client.0.smithi072.stdout:scsi_eh_2 97 root rtd unknown /proc/97/root (readlink: Permission denied) 2024-04-26T21:58:48.601 INFO:tasks.workunit.client.0.smithi072.stdout:scsi_eh_2 97 root txt unknown /proc/97/exe (readlink: Permission denied) 2024-04-26T21:58:48.601 INFO:tasks.workunit.client.0.smithi072.stdout:scsi_eh_2 97 root NOFD /proc/97/fd (opendir: Permission denied) 2024-04-26T21:58:48.601 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 98 root cwd unknown /proc/98/cwd (readlink: Permission denied) 2024-04-26T21:58:48.601 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 98 root rtd unknown /proc/98/root (readlink: Permission denied) 2024-04-26T21:58:48.601 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 98 root txt unknown /proc/98/exe (readlink: Permission denied) 2024-04-26T21:58:48.601 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 98 root NOFD /proc/98/fd (opendir: Permission denied) 2024-04-26T21:58:48.601 INFO:tasks.workunit.client.0.smithi072.stdout:scsi_eh_3 99 root cwd unknown /proc/99/cwd (readlink: Permission denied) 2024-04-26T21:58:48.602 INFO:tasks.workunit.client.0.smithi072.stdout:scsi_eh_3 99 root rtd unknown /proc/99/root (readlink: Permission denied) 2024-04-26T21:58:48.602 INFO:tasks.workunit.client.0.smithi072.stdout:scsi_eh_3 99 root txt unknown /proc/99/exe (readlink: Permission denied) 2024-04-26T21:58:48.602 INFO:tasks.workunit.client.0.smithi072.stdout:scsi_eh_3 99 root NOFD /proc/99/fd (opendir: Permission denied) 2024-04-26T21:58:48.602 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 100 root cwd unknown /proc/100/cwd (readlink: Permission denied) 2024-04-26T21:58:48.602 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 100 root rtd unknown /proc/100/root (readlink: Permission denied) 2024-04-26T21:58:48.602 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 100 root txt unknown /proc/100/exe (readlink: Permission denied) 2024-04-26T21:58:48.602 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 100 root NOFD /proc/100/fd (opendir: Permission denied) 2024-04-26T21:58:48.602 INFO:tasks.workunit.client.0.smithi072.stdout:scsi_eh_4 104 root cwd unknown /proc/104/cwd (readlink: Permission denied) 2024-04-26T21:58:48.602 INFO:tasks.workunit.client.0.smithi072.stdout:scsi_eh_4 104 root rtd unknown /proc/104/root (readlink: Permission denied) 2024-04-26T21:58:48.602 INFO:tasks.workunit.client.0.smithi072.stdout:scsi_eh_4 104 root txt unknown /proc/104/exe (readlink: Permission denied) 2024-04-26T21:58:48.602 INFO:tasks.workunit.client.0.smithi072.stdout:scsi_eh_4 104 root NOFD /proc/104/fd (opendir: Permission denied) 2024-04-26T21:58:48.602 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 105 root cwd unknown /proc/105/cwd (readlink: Permission denied) 2024-04-26T21:58:48.602 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 105 root rtd unknown /proc/105/root (readlink: Permission denied) 2024-04-26T21:58:48.602 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 105 root txt unknown /proc/105/exe (readlink: Permission denied) 2024-04-26T21:58:48.603 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 105 root NOFD /proc/105/fd (opendir: Permission denied) 2024-04-26T21:58:48.603 INFO:tasks.workunit.client.0.smithi072.stdout:scsi_eh_5 106 root cwd unknown /proc/106/cwd (readlink: Permission denied) 2024-04-26T21:58:48.603 INFO:tasks.workunit.client.0.smithi072.stdout:scsi_eh_5 106 root rtd unknown /proc/106/root (readlink: Permission denied) 2024-04-26T21:58:48.603 INFO:tasks.workunit.client.0.smithi072.stdout:scsi_eh_5 106 root txt unknown /proc/106/exe (readlink: Permission denied) 2024-04-26T21:58:48.603 INFO:tasks.workunit.client.0.smithi072.stdout:scsi_eh_5 106 root NOFD /proc/106/fd (opendir: Permission denied) 2024-04-26T21:58:48.603 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 107 root cwd unknown /proc/107/cwd (readlink: Permission denied) 2024-04-26T21:58:48.603 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 107 root rtd unknown /proc/107/root (readlink: Permission denied) 2024-04-26T21:58:48.603 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 107 root txt unknown /proc/107/exe (readlink: Permission denied) 2024-04-26T21:58:48.603 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 107 root NOFD /proc/107/fd (opendir: Permission denied) 2024-04-26T21:58:48.603 INFO:tasks.workunit.client.0.smithi072.stdout:scsi_eh_6 108 root cwd unknown /proc/108/cwd (readlink: Permission denied) 2024-04-26T21:58:48.603 INFO:tasks.workunit.client.0.smithi072.stdout:scsi_eh_6 108 root rtd unknown /proc/108/root (readlink: Permission denied) 2024-04-26T21:58:48.603 INFO:tasks.workunit.client.0.smithi072.stdout:scsi_eh_6 108 root txt unknown /proc/108/exe (readlink: Permission denied) 2024-04-26T21:58:48.604 INFO:tasks.workunit.client.0.smithi072.stdout:scsi_eh_6 108 root NOFD /proc/108/fd (opendir: Permission denied) 2024-04-26T21:58:48.604 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 109 root cwd unknown /proc/109/cwd (readlink: Permission denied) 2024-04-26T21:58:48.604 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 109 root rtd unknown /proc/109/root (readlink: Permission denied) 2024-04-26T21:58:48.604 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 109 root txt unknown /proc/109/exe (readlink: Permission denied) 2024-04-26T21:58:48.604 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 109 root NOFD /proc/109/fd (opendir: Permission denied) 2024-04-26T21:58:48.604 INFO:tasks.workunit.client.0.smithi072.stdout:scsi_eh_7 110 root cwd unknown /proc/110/cwd (readlink: Permission denied) 2024-04-26T21:58:48.604 INFO:tasks.workunit.client.0.smithi072.stdout:scsi_eh_7 110 root rtd unknown /proc/110/root (readlink: Permission denied) 2024-04-26T21:58:48.604 INFO:tasks.workunit.client.0.smithi072.stdout:scsi_eh_7 110 root txt unknown /proc/110/exe (readlink: Permission denied) 2024-04-26T21:58:48.604 INFO:tasks.workunit.client.0.smithi072.stdout:scsi_eh_7 110 root NOFD /proc/110/fd (opendir: Permission denied) 2024-04-26T21:58:48.604 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 111 root cwd unknown /proc/111/cwd (readlink: Permission denied) 2024-04-26T21:58:48.604 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 111 root rtd unknown /proc/111/root (readlink: Permission denied) 2024-04-26T21:58:48.604 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 111 root txt unknown /proc/111/exe (readlink: Permission denied) 2024-04-26T21:58:48.604 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 111 root NOFD /proc/111/fd (opendir: Permission denied) 2024-04-26T21:58:48.605 INFO:tasks.workunit.client.0.smithi072.stdout:scsi_eh_8 112 root cwd unknown /proc/112/cwd (readlink: Permission denied) 2024-04-26T21:58:48.605 INFO:tasks.workunit.client.0.smithi072.stdout:scsi_eh_8 112 root rtd unknown /proc/112/root (readlink: Permission denied) 2024-04-26T21:58:48.605 INFO:tasks.workunit.client.0.smithi072.stdout:scsi_eh_8 112 root txt unknown /proc/112/exe (readlink: Permission denied) 2024-04-26T21:58:48.605 INFO:tasks.workunit.client.0.smithi072.stdout:scsi_eh_8 112 root NOFD /proc/112/fd (opendir: Permission denied) 2024-04-26T21:58:48.605 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 113 root cwd unknown /proc/113/cwd (readlink: Permission denied) 2024-04-26T21:58:48.605 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 113 root rtd unknown /proc/113/root (readlink: Permission denied) 2024-04-26T21:58:48.605 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 113 root txt unknown /proc/113/exe (readlink: Permission denied) 2024-04-26T21:58:48.605 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 113 root NOFD /proc/113/fd (opendir: Permission denied) 2024-04-26T21:58:48.605 INFO:tasks.workunit.client.0.smithi072.stdout:scsi_eh_9 114 root cwd unknown /proc/114/cwd (readlink: Permission denied) 2024-04-26T21:58:48.605 INFO:tasks.workunit.client.0.smithi072.stdout:scsi_eh_9 114 root rtd unknown /proc/114/root (readlink: Permission denied) 2024-04-26T21:58:48.605 INFO:tasks.workunit.client.0.smithi072.stdout:scsi_eh_9 114 root txt unknown /proc/114/exe (readlink: Permission denied) 2024-04-26T21:58:48.605 INFO:tasks.workunit.client.0.smithi072.stdout:scsi_eh_9 114 root NOFD /proc/114/fd (opendir: Permission denied) 2024-04-26T21:58:48.605 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 115 root cwd unknown /proc/115/cwd (readlink: Permission denied) 2024-04-26T21:58:48.605 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 115 root rtd unknown /proc/115/root (readlink: Permission denied) 2024-04-26T21:58:48.606 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 115 root txt unknown /proc/115/exe (readlink: Permission denied) 2024-04-26T21:58:48.606 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 115 root NOFD /proc/115/fd (opendir: Permission denied) 2024-04-26T21:58:48.606 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/u 116 root cwd unknown /proc/116/cwd (readlink: Permission denied) 2024-04-26T21:58:48.606 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/u 116 root rtd unknown /proc/116/root (readlink: Permission denied) 2024-04-26T21:58:48.606 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/u 116 root txt unknown /proc/116/exe (readlink: Permission denied) 2024-04-26T21:58:48.606 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/u 116 root NOFD /proc/116/fd (opendir: Permission denied) 2024-04-26T21:58:48.606 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/u 119 root cwd unknown /proc/119/cwd (readlink: Permission denied) 2024-04-26T21:58:48.606 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/u 119 root rtd unknown /proc/119/root (readlink: Permission denied) 2024-04-26T21:58:48.606 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/u 119 root txt unknown /proc/119/exe (readlink: Permission denied) 2024-04-26T21:58:48.606 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/u 119 root NOFD /proc/119/fd (opendir: Permission denied) 2024-04-26T21:58:48.606 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/0 126 root cwd unknown /proc/126/cwd (readlink: Permission denied) 2024-04-26T21:58:48.606 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/0 126 root rtd unknown /proc/126/root (readlink: Permission denied) 2024-04-26T21:58:48.606 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/0 126 root txt unknown /proc/126/exe (readlink: Permission denied) 2024-04-26T21:58:48.606 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/0 126 root NOFD /proc/126/fd (opendir: Permission denied) 2024-04-26T21:58:48.606 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/7 128 root cwd unknown /proc/128/cwd (readlink: Permission denied) 2024-04-26T21:58:48.607 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/7 128 root rtd unknown /proc/128/root (readlink: Permission denied) 2024-04-26T21:58:48.607 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/7 128 root txt unknown /proc/128/exe (readlink: Permission denied) 2024-04-26T21:58:48.607 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/7 128 root NOFD /proc/128/fd (opendir: Permission denied) 2024-04-26T21:58:48.607 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 129 root cwd unknown /proc/129/cwd (readlink: Permission denied) 2024-04-26T21:58:48.607 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 129 root rtd unknown /proc/129/root (readlink: Permission denied) 2024-04-26T21:58:48.607 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 129 root txt unknown /proc/129/exe (readlink: Permission denied) 2024-04-26T21:58:48.607 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 129 root NOFD /proc/129/fd (opendir: Permission denied) 2024-04-26T21:58:48.607 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 131 root cwd unknown /proc/131/cwd (readlink: Permission denied) 2024-04-26T21:58:48.607 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 131 root rtd unknown /proc/131/root (readlink: Permission denied) 2024-04-26T21:58:48.607 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 131 root txt unknown /proc/131/exe (readlink: Permission denied) 2024-04-26T21:58:48.607 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 131 root NOFD /proc/131/fd (opendir: Permission denied) 2024-04-26T21:58:48.607 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 132 root cwd unknown /proc/132/cwd (readlink: Permission denied) 2024-04-26T21:58:48.607 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 132 root rtd unknown /proc/132/root (readlink: Permission denied) 2024-04-26T21:58:48.608 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 132 root txt unknown /proc/132/exe (readlink: Permission denied) 2024-04-26T21:58:48.608 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 132 root NOFD /proc/132/fd (opendir: Permission denied) 2024-04-26T21:58:48.608 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/u 139 root cwd unknown /proc/139/cwd (readlink: Permission denied) 2024-04-26T21:58:48.608 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/u 139 root rtd unknown /proc/139/root (readlink: Permission denied) 2024-04-26T21:58:48.608 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/u 139 root txt unknown /proc/139/exe (readlink: Permission denied) 2024-04-26T21:58:48.608 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/u 139 root NOFD /proc/139/fd (opendir: Permission denied) 2024-04-26T21:58:48.608 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/1 209 root cwd unknown /proc/209/cwd (readlink: Permission denied) 2024-04-26T21:58:48.608 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/1 209 root rtd unknown /proc/209/root (readlink: Permission denied) 2024-04-26T21:58:48.608 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/1 209 root txt unknown /proc/209/exe (readlink: Permission denied) 2024-04-26T21:58:48.608 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/1 209 root NOFD /proc/209/fd (opendir: Permission denied) 2024-04-26T21:58:48.608 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/2 249 root cwd unknown /proc/249/cwd (readlink: Permission denied) 2024-04-26T21:58:48.608 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/2 249 root rtd unknown /proc/249/root (readlink: Permission denied) 2024-04-26T21:58:48.608 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/2 249 root txt unknown /proc/249/exe (readlink: Permission denied) 2024-04-26T21:58:48.608 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/2 249 root NOFD /proc/249/fd (opendir: Permission denied) 2024-04-26T21:58:48.609 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/5 267 root cwd unknown /proc/267/cwd (readlink: Permission denied) 2024-04-26T21:58:48.609 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/5 267 root rtd unknown /proc/267/root (readlink: Permission denied) 2024-04-26T21:58:48.609 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/5 267 root txt unknown /proc/267/exe (readlink: Permission denied) 2024-04-26T21:58:48.609 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/5 267 root NOFD /proc/267/fd (opendir: Permission denied) 2024-04-26T21:58:48.609 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/3 270 root cwd unknown /proc/270/cwd (readlink: Permission denied) 2024-04-26T21:58:48.609 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/3 270 root rtd unknown /proc/270/root (readlink: Permission denied) 2024-04-26T21:58:48.609 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/3 270 root txt unknown /proc/270/exe (readlink: Permission denied) 2024-04-26T21:58:48.609 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/3 270 root NOFD /proc/270/fd (opendir: Permission denied) 2024-04-26T21:58:48.609 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/6 304 root cwd unknown /proc/304/cwd (readlink: Permission denied) 2024-04-26T21:58:48.609 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/6 304 root rtd unknown /proc/304/root (readlink: Permission denied) 2024-04-26T21:58:48.609 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/6 304 root txt unknown /proc/304/exe (readlink: Permission denied) 2024-04-26T21:58:48.609 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/6 304 root NOFD /proc/304/fd (opendir: Permission denied) 2024-04-26T21:58:48.609 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/4 366 root cwd unknown /proc/366/cwd (readlink: Permission denied) 2024-04-26T21:58:48.609 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/4 366 root rtd unknown /proc/366/root (readlink: Permission denied) 2024-04-26T21:58:48.610 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/4 366 root txt unknown /proc/366/exe (readlink: Permission denied) 2024-04-26T21:58:48.610 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/4 366 root NOFD /proc/366/fd (opendir: Permission denied) 2024-04-26T21:58:48.610 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 402 root cwd unknown /proc/402/cwd (readlink: Permission denied) 2024-04-26T21:58:48.610 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 402 root rtd unknown /proc/402/root (readlink: Permission denied) 2024-04-26T21:58:48.610 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 402 root txt unknown /proc/402/exe (readlink: Permission denied) 2024-04-26T21:58:48.610 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 402 root NOFD /proc/402/fd (opendir: Permission denied) 2024-04-26T21:58:48.610 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 403 root cwd unknown /proc/403/cwd (readlink: Permission denied) 2024-04-26T21:58:48.610 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 403 root rtd unknown /proc/403/root (readlink: Permission denied) 2024-04-26T21:58:48.610 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 403 root txt unknown /proc/403/exe (readlink: Permission denied) 2024-04-26T21:58:48.610 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 403 root NOFD /proc/403/fd (opendir: Permission denied) 2024-04-26T21:58:48.610 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 404 root cwd unknown /proc/404/cwd (readlink: Permission denied) 2024-04-26T21:58:48.610 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 404 root rtd unknown /proc/404/root (readlink: Permission denied) 2024-04-26T21:58:48.610 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 404 root txt unknown /proc/404/exe (readlink: Permission denied) 2024-04-26T21:58:48.610 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 404 root NOFD /proc/404/fd (opendir: Permission denied) 2024-04-26T21:58:48.610 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 435 root cwd unknown /proc/435/cwd (readlink: Permission denied) 2024-04-26T21:58:48.611 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 435 root rtd unknown /proc/435/root (readlink: Permission denied) 2024-04-26T21:58:48.611 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 435 root txt unknown /proc/435/exe (readlink: Permission denied) 2024-04-26T21:58:48.611 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 435 root NOFD /proc/435/fd (opendir: Permission denied) 2024-04-26T21:58:48.611 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/3 446 root cwd unknown /proc/446/cwd (readlink: Permission denied) 2024-04-26T21:58:48.611 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/3 446 root rtd unknown /proc/446/root (readlink: Permission denied) 2024-04-26T21:58:48.611 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/3 446 root txt unknown /proc/446/exe (readlink: Permission denied) 2024-04-26T21:58:48.611 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/3 446 root NOFD /proc/446/fd (opendir: Permission denied) 2024-04-26T21:58:48.611 INFO:tasks.workunit.client.0.smithi072.stdout:jbd2/sda1 447 root cwd unknown /proc/447/cwd (readlink: Permission denied) 2024-04-26T21:58:48.611 INFO:tasks.workunit.client.0.smithi072.stdout:jbd2/sda1 447 root rtd unknown /proc/447/root (readlink: Permission denied) 2024-04-26T21:58:48.611 INFO:tasks.workunit.client.0.smithi072.stdout:jbd2/sda1 447 root txt unknown /proc/447/exe (readlink: Permission denied) 2024-04-26T21:58:48.611 INFO:tasks.workunit.client.0.smithi072.stdout:jbd2/sda1 447 root NOFD /proc/447/fd (opendir: Permission denied) 2024-04-26T21:58:48.611 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 448 root cwd unknown /proc/448/cwd (readlink: Permission denied) 2024-04-26T21:58:48.611 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 448 root rtd unknown /proc/448/root (readlink: Permission denied) 2024-04-26T21:58:48.612 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 448 root txt unknown /proc/448/exe (readlink: Permission denied) 2024-04-26T21:58:48.612 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 448 root NOFD /proc/448/fd (opendir: Permission denied) 2024-04-26T21:58:48.612 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/5 509 root cwd unknown /proc/509/cwd (readlink: Permission denied) 2024-04-26T21:58:48.612 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/5 509 root rtd unknown /proc/509/root (readlink: Permission denied) 2024-04-26T21:58:48.612 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/5 509 root txt unknown /proc/509/exe (readlink: Permission denied) 2024-04-26T21:58:48.612 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/5 509 root NOFD /proc/509/fd (opendir: Permission denied) 2024-04-26T21:58:48.612 INFO:tasks.workunit.client.0.smithi072.stdout:systemd-j 524 root cwd unknown /proc/524/cwd (readlink: Permission denied) 2024-04-26T21:58:48.612 INFO:tasks.workunit.client.0.smithi072.stdout:systemd-j 524 root rtd unknown /proc/524/root (readlink: Permission denied) 2024-04-26T21:58:48.612 INFO:tasks.workunit.client.0.smithi072.stdout:systemd-j 524 root txt unknown /proc/524/exe (readlink: Permission denied) 2024-04-26T21:58:48.612 INFO:tasks.workunit.client.0.smithi072.stdout:systemd-j 524 root NOFD /proc/524/fd (opendir: Permission denied) 2024-04-26T21:58:48.612 INFO:tasks.workunit.client.0.smithi072.stdout:ext4lazyi 530 root cwd unknown /proc/530/cwd (readlink: Permission denied) 2024-04-26T21:58:48.613 INFO:tasks.workunit.client.0.smithi072.stdout:ext4lazyi 530 root rtd unknown /proc/530/root (readlink: Permission denied) 2024-04-26T21:58:48.613 INFO:tasks.workunit.client.0.smithi072.stdout:ext4lazyi 530 root txt unknown /proc/530/exe (readlink: Permission denied) 2024-04-26T21:58:48.613 INFO:tasks.workunit.client.0.smithi072.stdout:ext4lazyi 530 root NOFD /proc/530/fd (opendir: Permission denied) 2024-04-26T21:58:48.613 INFO:tasks.workunit.client.0.smithi072.stdout:systemd-u 539 root cwd unknown /proc/539/cwd (readlink: Permission denied) 2024-04-26T21:58:48.613 INFO:tasks.workunit.client.0.smithi072.stdout:systemd-u 539 root rtd unknown /proc/539/root (readlink: Permission denied) 2024-04-26T21:58:48.613 INFO:tasks.workunit.client.0.smithi072.stdout:systemd-u 539 root txt unknown /proc/539/exe (readlink: Permission denied) 2024-04-26T21:58:48.613 INFO:tasks.workunit.client.0.smithi072.stdout:systemd-u 539 root NOFD /proc/539/fd (opendir: Permission denied) 2024-04-26T21:58:48.613 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 599 root cwd unknown /proc/599/cwd (readlink: Permission denied) 2024-04-26T21:58:48.613 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 599 root rtd unknown /proc/599/root (readlink: Permission denied) 2024-04-26T21:58:48.613 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 599 root txt unknown /proc/599/exe (readlink: Permission denied) 2024-04-26T21:58:48.613 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 599 root NOFD /proc/599/fd (opendir: Permission denied) 2024-04-26T21:58:48.613 INFO:tasks.workunit.client.0.smithi072.stdout:kipmi0 601 root cwd unknown /proc/601/cwd (readlink: Permission denied) 2024-04-26T21:58:48.613 INFO:tasks.workunit.client.0.smithi072.stdout:kipmi0 601 root rtd unknown /proc/601/root (readlink: Permission denied) 2024-04-26T21:58:48.613 INFO:tasks.workunit.client.0.smithi072.stdout:kipmi0 601 root txt unknown /proc/601/exe (readlink: Permission denied) 2024-04-26T21:58:48.614 INFO:tasks.workunit.client.0.smithi072.stdout:kipmi0 601 root NOFD /proc/601/fd (opendir: Permission denied) 2024-04-26T21:58:48.614 INFO:tasks.workunit.client.0.smithi072.stdout:rpcbind 611 rpc cwd unknown /proc/611/cwd (readlink: Permission denied) 2024-04-26T21:58:48.614 INFO:tasks.workunit.client.0.smithi072.stdout:rpcbind 611 rpc rtd unknown /proc/611/root (readlink: Permission denied) 2024-04-26T21:58:48.614 INFO:tasks.workunit.client.0.smithi072.stdout:rpcbind 611 rpc txt unknown /proc/611/exe (readlink: Permission denied) 2024-04-26T21:58:48.614 INFO:tasks.workunit.client.0.smithi072.stdout:rpcbind 611 rpc NOFD /proc/611/fd (opendir: Permission denied) 2024-04-26T21:58:48.614 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 613 root cwd unknown /proc/613/cwd (readlink: Permission denied) 2024-04-26T21:58:48.614 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 613 root rtd unknown /proc/613/root (readlink: Permission denied) 2024-04-26T21:58:48.614 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 613 root txt unknown /proc/613/exe (readlink: Permission denied) 2024-04-26T21:58:48.614 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 613 root NOFD /proc/613/fd (opendir: Permission denied) 2024-04-26T21:58:48.614 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 614 root cwd unknown /proc/614/cwd (readlink: Permission denied) 2024-04-26T21:58:48.614 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 614 root rtd unknown /proc/614/root (readlink: Permission denied) 2024-04-26T21:58:48.615 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 614 root txt unknown /proc/614/exe (readlink: Permission denied) 2024-04-26T21:58:48.615 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 614 root NOFD /proc/614/fd (opendir: Permission denied) 2024-04-26T21:58:48.615 INFO:tasks.workunit.client.0.smithi072.stdout:auditd 631 root cwd unknown /proc/631/cwd (readlink: Permission denied) 2024-04-26T21:58:48.615 INFO:tasks.workunit.client.0.smithi072.stdout:auditd 631 root rtd unknown /proc/631/root (readlink: Permission denied) 2024-04-26T21:58:48.615 INFO:tasks.workunit.client.0.smithi072.stdout:auditd 631 root txt unknown /proc/631/exe (readlink: Permission denied) 2024-04-26T21:58:48.615 INFO:tasks.workunit.client.0.smithi072.stdout:auditd 631 root NOFD /proc/631/fd (opendir: Permission denied) 2024-04-26T21:58:48.615 INFO:tasks.workunit.client.0.smithi072.stdout:auditd 631 632 auditd root cwd unknown /proc/631/task/632/cwd (readlink: Permission denied) 2024-04-26T21:58:48.615 INFO:tasks.workunit.client.0.smithi072.stdout:auditd 631 632 auditd root rtd unknown /proc/631/task/632/root (readlink: Permission denied) 2024-04-26T21:58:48.615 INFO:tasks.workunit.client.0.smithi072.stdout:auditd 631 632 auditd root txt unknown /proc/631/task/632/exe (readlink: Permission denied) 2024-04-26T21:58:48.615 INFO:tasks.workunit.client.0.smithi072.stdout:auditd 631 632 auditd root NOFD /proc/631/task/632/fd (opendir: Permission denied) 2024-04-26T21:58:48.615 INFO:tasks.workunit.client.0.smithi072.stdout:auditd 631 634 auditd root cwd unknown /proc/631/task/634/cwd (readlink: Permission denied) 2024-04-26T21:58:48.615 INFO:tasks.workunit.client.0.smithi072.stdout:auditd 631 634 auditd root rtd unknown /proc/631/task/634/root (readlink: Permission denied) 2024-04-26T21:58:48.615 INFO:tasks.workunit.client.0.smithi072.stdout:auditd 631 634 auditd root txt unknown /proc/631/task/634/exe (readlink: Permission denied) 2024-04-26T21:58:48.615 INFO:tasks.workunit.client.0.smithi072.stdout:auditd 631 634 auditd root NOFD /proc/631/task/634/fd (opendir: Permission denied) 2024-04-26T21:58:48.616 INFO:tasks.workunit.client.0.smithi072.stdout:sedispatc 633 root cwd unknown /proc/633/cwd (readlink: Permission denied) 2024-04-26T21:58:48.616 INFO:tasks.workunit.client.0.smithi072.stdout:sedispatc 633 root rtd unknown /proc/633/root (readlink: Permission denied) 2024-04-26T21:58:48.616 INFO:tasks.workunit.client.0.smithi072.stdout:sedispatc 633 root txt unknown /proc/633/exe (readlink: Permission denied) 2024-04-26T21:58:48.616 INFO:tasks.workunit.client.0.smithi072.stdout:sedispatc 633 root NOFD /proc/633/fd (opendir: Permission denied) 2024-04-26T21:58:48.616 INFO:tasks.workunit.client.0.smithi072.stdout:dbus-brok 654 dbus cwd unknown /proc/654/cwd (readlink: Permission denied) 2024-04-26T21:58:48.616 INFO:tasks.workunit.client.0.smithi072.stdout:dbus-brok 654 dbus rtd unknown /proc/654/root (readlink: Permission denied) 2024-04-26T21:58:48.616 INFO:tasks.workunit.client.0.smithi072.stdout:dbus-brok 654 dbus txt unknown /proc/654/exe (readlink: Permission denied) 2024-04-26T21:58:48.616 INFO:tasks.workunit.client.0.smithi072.stdout:dbus-brok 654 dbus NOFD /proc/654/fd (opendir: Permission denied) 2024-04-26T21:58:48.616 INFO:tasks.workunit.client.0.smithi072.stdout:dbus-brok 655 dbus cwd unknown /proc/655/cwd (readlink: Permission denied) 2024-04-26T21:58:48.616 INFO:tasks.workunit.client.0.smithi072.stdout:dbus-brok 655 dbus rtd unknown /proc/655/root (readlink: Permission denied) 2024-04-26T21:58:48.616 INFO:tasks.workunit.client.0.smithi072.stdout:dbus-brok 655 dbus txt unknown /proc/655/exe (readlink: Permission denied) 2024-04-26T21:58:48.616 INFO:tasks.workunit.client.0.smithi072.stdout:dbus-brok 655 dbus NOFD /proc/655/fd (opendir: Permission denied) 2024-04-26T21:58:48.616 INFO:tasks.workunit.client.0.smithi072.stdout:NetworkMa 656 root cwd unknown /proc/656/cwd (readlink: Permission denied) 2024-04-26T21:58:48.616 INFO:tasks.workunit.client.0.smithi072.stdout:NetworkMa 656 root rtd unknown /proc/656/root (readlink: Permission denied) 2024-04-26T21:58:48.617 INFO:tasks.workunit.client.0.smithi072.stdout:NetworkMa 656 root txt unknown /proc/656/exe (readlink: Permission denied) 2024-04-26T21:58:48.617 INFO:tasks.workunit.client.0.smithi072.stdout:NetworkMa 656 root NOFD /proc/656/fd (opendir: Permission denied) 2024-04-26T21:58:48.617 INFO:tasks.workunit.client.0.smithi072.stdout:NetworkMa 656 672 gmain root cwd unknown /proc/656/task/672/cwd (readlink: Permission denied) 2024-04-26T21:58:48.617 INFO:tasks.workunit.client.0.smithi072.stdout:NetworkMa 656 672 gmain root rtd unknown /proc/656/task/672/root (readlink: Permission denied) 2024-04-26T21:58:48.617 INFO:tasks.workunit.client.0.smithi072.stdout:NetworkMa 656 672 gmain root txt unknown /proc/656/task/672/exe (readlink: Permission denied) 2024-04-26T21:58:48.617 INFO:tasks.workunit.client.0.smithi072.stdout:NetworkMa 656 672 gmain root NOFD /proc/656/task/672/fd (opendir: Permission denied) 2024-04-26T21:58:48.617 INFO:tasks.workunit.client.0.smithi072.stdout:NetworkMa 656 673 gdbus root cwd unknown /proc/656/task/673/cwd (readlink: Permission denied) 2024-04-26T21:58:48.617 INFO:tasks.workunit.client.0.smithi072.stdout:NetworkMa 656 673 gdbus root rtd unknown /proc/656/task/673/root (readlink: Permission denied) 2024-04-26T21:58:48.617 INFO:tasks.workunit.client.0.smithi072.stdout:NetworkMa 656 673 gdbus root txt unknown /proc/656/task/673/exe (readlink: Permission denied) 2024-04-26T21:58:48.617 INFO:tasks.workunit.client.0.smithi072.stdout:NetworkMa 656 673 gdbus root NOFD /proc/656/task/673/fd (opendir: Permission denied) 2024-04-26T21:58:48.617 INFO:tasks.workunit.client.0.smithi072.stdout:irqbalanc 659 root cwd unknown /proc/659/cwd (readlink: Permission denied) 2024-04-26T21:58:48.617 INFO:tasks.workunit.client.0.smithi072.stdout:irqbalanc 659 root rtd unknown /proc/659/root (readlink: Permission denied) 2024-04-26T21:58:48.617 INFO:tasks.workunit.client.0.smithi072.stdout:irqbalanc 659 root txt unknown /proc/659/exe (readlink: Permission denied) 2024-04-26T21:58:48.617 INFO:tasks.workunit.client.0.smithi072.stdout:irqbalanc 659 root NOFD /proc/659/fd (opendir: Permission denied) 2024-04-26T21:58:48.617 INFO:tasks.workunit.client.0.smithi072.stdout:irqbalanc 659 662 gmain root cwd unknown /proc/659/task/662/cwd (readlink: Permission denied) 2024-04-26T21:58:48.618 INFO:tasks.workunit.client.0.smithi072.stdout:irqbalanc 659 662 gmain root rtd unknown /proc/659/task/662/root (readlink: Permission denied) 2024-04-26T21:58:48.618 INFO:tasks.workunit.client.0.smithi072.stdout:irqbalanc 659 662 gmain root txt unknown /proc/659/task/662/exe (readlink: Permission denied) 2024-04-26T21:58:48.618 INFO:tasks.workunit.client.0.smithi072.stdout:irqbalanc 659 662 gmain root NOFD /proc/659/task/662/fd (opendir: Permission denied) 2024-04-26T21:58:48.618 INFO:tasks.workunit.client.0.smithi072.stdout:lsmd 660 libstoragemgmt cwd unknown /proc/660/cwd (readlink: Permission denied) 2024-04-26T21:58:48.618 INFO:tasks.workunit.client.0.smithi072.stdout:lsmd 660 libstoragemgmt rtd unknown /proc/660/root (readlink: Permission denied) 2024-04-26T21:58:48.618 INFO:tasks.workunit.client.0.smithi072.stdout:lsmd 660 libstoragemgmt txt unknown /proc/660/exe (readlink: Permission denied) 2024-04-26T21:58:48.618 INFO:tasks.workunit.client.0.smithi072.stdout:lsmd 660 libstoragemgmt NOFD /proc/660/fd (opendir: Permission denied) 2024-04-26T21:58:48.618 INFO:tasks.workunit.client.0.smithi072.stdout:smartd 661 root cwd unknown /proc/661/cwd (readlink: Permission denied) 2024-04-26T21:58:48.618 INFO:tasks.workunit.client.0.smithi072.stdout:smartd 661 root rtd unknown /proc/661/root (readlink: Permission denied) 2024-04-26T21:58:48.618 INFO:tasks.workunit.client.0.smithi072.stdout:smartd 661 root txt unknown /proc/661/exe (readlink: Permission denied) 2024-04-26T21:58:48.618 INFO:tasks.workunit.client.0.smithi072.stdout:smartd 661 root NOFD /proc/661/fd (opendir: Permission denied) 2024-04-26T21:58:48.618 INFO:tasks.workunit.client.0.smithi072.stdout:systemd-l 664 root cwd unknown /proc/664/cwd (readlink: Permission denied) 2024-04-26T21:58:48.618 INFO:tasks.workunit.client.0.smithi072.stdout:systemd-l 664 root rtd unknown /proc/664/root (readlink: Permission denied) 2024-04-26T21:58:48.619 INFO:tasks.workunit.client.0.smithi072.stdout:systemd-l 664 root txt unknown /proc/664/exe (readlink: Permission denied) 2024-04-26T21:58:48.619 INFO:tasks.workunit.client.0.smithi072.stdout:systemd-l 664 root NOFD /proc/664/fd (opendir: Permission denied) 2024-04-26T21:58:48.619 INFO:tasks.workunit.client.0.smithi072.stdout:sshd 707 root cwd unknown /proc/707/cwd (readlink: Permission denied) 2024-04-26T21:58:48.619 INFO:tasks.workunit.client.0.smithi072.stdout:sshd 707 root rtd unknown /proc/707/root (readlink: Permission denied) 2024-04-26T21:58:48.619 INFO:tasks.workunit.client.0.smithi072.stdout:sshd 707 root txt unknown /proc/707/exe (readlink: Permission denied) 2024-04-26T21:58:48.619 INFO:tasks.workunit.client.0.smithi072.stdout:sshd 707 root NOFD /proc/707/fd (opendir: Permission denied) 2024-04-26T21:58:48.619 INFO:tasks.workunit.client.0.smithi072.stdout:gssproxy 711 root cwd unknown /proc/711/cwd (readlink: Permission denied) 2024-04-26T21:58:48.619 INFO:tasks.workunit.client.0.smithi072.stdout:gssproxy 711 root rtd unknown /proc/711/root (readlink: Permission denied) 2024-04-26T21:58:48.619 INFO:tasks.workunit.client.0.smithi072.stdout:gssproxy 711 root txt unknown /proc/711/exe (readlink: Permission denied) 2024-04-26T21:58:48.619 INFO:tasks.workunit.client.0.smithi072.stdout:gssproxy 711 root NOFD /proc/711/fd (opendir: Permission denied) 2024-04-26T21:58:48.619 INFO:tasks.workunit.client.0.smithi072.stdout:gssproxy 711 712 gssproxy root cwd unknown /proc/711/task/712/cwd (readlink: Permission denied) 2024-04-26T21:58:48.620 INFO:tasks.workunit.client.0.smithi072.stdout:gssproxy 711 712 gssproxy root rtd unknown /proc/711/task/712/root (readlink: Permission denied) 2024-04-26T21:58:48.620 INFO:tasks.workunit.client.0.smithi072.stdout:gssproxy 711 712 gssproxy root txt unknown /proc/711/task/712/exe (readlink: Permission denied) 2024-04-26T21:58:48.620 INFO:tasks.workunit.client.0.smithi072.stdout:gssproxy 711 712 gssproxy root NOFD /proc/711/task/712/fd (opendir: Permission denied) 2024-04-26T21:58:48.620 INFO:tasks.workunit.client.0.smithi072.stdout:gssproxy 711 713 gssproxy root cwd unknown /proc/711/task/713/cwd (readlink: Permission denied) 2024-04-26T21:58:48.620 INFO:tasks.workunit.client.0.smithi072.stdout:gssproxy 711 713 gssproxy root rtd unknown /proc/711/task/713/root (readlink: Permission denied) 2024-04-26T21:58:48.620 INFO:tasks.workunit.client.0.smithi072.stdout:gssproxy 711 713 gssproxy root txt unknown /proc/711/task/713/exe (readlink: Permission denied) 2024-04-26T21:58:48.620 INFO:tasks.workunit.client.0.smithi072.stdout:gssproxy 711 713 gssproxy root NOFD /proc/711/task/713/fd (opendir: Permission denied) 2024-04-26T21:58:48.620 INFO:tasks.workunit.client.0.smithi072.stdout:gssproxy 711 714 gssproxy root cwd unknown /proc/711/task/714/cwd (readlink: Permission denied) 2024-04-26T21:58:48.620 INFO:tasks.workunit.client.0.smithi072.stdout:gssproxy 711 714 gssproxy root rtd unknown /proc/711/task/714/root (readlink: Permission denied) 2024-04-26T21:58:48.621 INFO:tasks.workunit.client.0.smithi072.stdout:gssproxy 711 714 gssproxy root txt unknown /proc/711/task/714/exe (readlink: Permission denied) 2024-04-26T21:58:48.621 INFO:tasks.workunit.client.0.smithi072.stdout:gssproxy 711 714 gssproxy root NOFD /proc/711/task/714/fd (opendir: Permission denied) 2024-04-26T21:58:48.621 INFO:tasks.workunit.client.0.smithi072.stdout:gssproxy 711 715 gssproxy root cwd unknown /proc/711/task/715/cwd (readlink: Permission denied) 2024-04-26T21:58:48.621 INFO:tasks.workunit.client.0.smithi072.stdout:gssproxy 711 715 gssproxy root rtd unknown /proc/711/task/715/root (readlink: Permission denied) 2024-04-26T21:58:48.621 INFO:tasks.workunit.client.0.smithi072.stdout:gssproxy 711 715 gssproxy root txt unknown /proc/711/task/715/exe (readlink: Permission denied) 2024-04-26T21:58:48.621 INFO:tasks.workunit.client.0.smithi072.stdout:gssproxy 711 715 gssproxy root NOFD /proc/711/task/715/fd (opendir: Permission denied) 2024-04-26T21:58:48.621 INFO:tasks.workunit.client.0.smithi072.stdout:gssproxy 711 716 gssproxy root cwd unknown /proc/711/task/716/cwd (readlink: Permission denied) 2024-04-26T21:58:48.621 INFO:tasks.workunit.client.0.smithi072.stdout:gssproxy 711 716 gssproxy root rtd unknown /proc/711/task/716/root (readlink: Permission denied) 2024-04-26T21:58:48.621 INFO:tasks.workunit.client.0.smithi072.stdout:gssproxy 711 716 gssproxy root txt unknown /proc/711/task/716/exe (readlink: Permission denied) 2024-04-26T21:58:48.621 INFO:tasks.workunit.client.0.smithi072.stdout:gssproxy 711 716 gssproxy root NOFD /proc/711/task/716/fd (opendir: Permission denied) 2024-04-26T21:58:48.621 INFO:tasks.workunit.client.0.smithi072.stdout:atd 718 root cwd unknown /proc/718/cwd (readlink: Permission denied) 2024-04-26T21:58:48.621 INFO:tasks.workunit.client.0.smithi072.stdout:atd 718 root rtd unknown /proc/718/root (readlink: Permission denied) 2024-04-26T21:58:48.622 INFO:tasks.workunit.client.0.smithi072.stdout:atd 718 root txt unknown /proc/718/exe (readlink: Permission denied) 2024-04-26T21:58:48.622 INFO:tasks.workunit.client.0.smithi072.stdout:atd 718 root NOFD /proc/718/fd (opendir: Permission denied) 2024-04-26T21:58:48.622 INFO:tasks.workunit.client.0.smithi072.stdout:crond 719 root cwd unknown /proc/719/cwd (readlink: Permission denied) 2024-04-26T21:58:48.622 INFO:tasks.workunit.client.0.smithi072.stdout:crond 719 root rtd unknown /proc/719/root (readlink: Permission denied) 2024-04-26T21:58:48.622 INFO:tasks.workunit.client.0.smithi072.stdout:crond 719 root txt unknown /proc/719/exe (readlink: Permission denied) 2024-04-26T21:58:48.622 INFO:tasks.workunit.client.0.smithi072.stdout:crond 719 root NOFD /proc/719/fd (opendir: Permission denied) 2024-04-26T21:58:48.622 INFO:tasks.workunit.client.0.smithi072.stdout:nrpe 737 nrpe cwd unknown /proc/737/cwd (readlink: Permission denied) 2024-04-26T21:58:48.622 INFO:tasks.workunit.client.0.smithi072.stdout:nrpe 737 nrpe rtd unknown /proc/737/root (readlink: Permission denied) 2024-04-26T21:58:48.622 INFO:tasks.workunit.client.0.smithi072.stdout:nrpe 737 nrpe txt unknown /proc/737/exe (readlink: Permission denied) 2024-04-26T21:58:48.622 INFO:tasks.workunit.client.0.smithi072.stdout:nrpe 737 nrpe NOFD /proc/737/fd (opendir: Permission denied) 2024-04-26T21:58:48.622 INFO:tasks.workunit.client.0.smithi072.stdout:agetty 1063 root cwd unknown /proc/1063/cwd (readlink: Permission denied) 2024-04-26T21:58:48.623 INFO:tasks.workunit.client.0.smithi072.stdout:agetty 1063 root rtd unknown /proc/1063/root (readlink: Permission denied) 2024-04-26T21:58:48.623 INFO:tasks.workunit.client.0.smithi072.stdout:agetty 1063 root txt unknown /proc/1063/exe (readlink: Permission denied) 2024-04-26T21:58:48.623 INFO:tasks.workunit.client.0.smithi072.stdout:agetty 1063 root NOFD /proc/1063/fd (opendir: Permission denied) 2024-04-26T21:58:48.623 INFO:tasks.workunit.client.0.smithi072.stdout:agetty 1064 root cwd unknown /proc/1064/cwd (readlink: Permission denied) 2024-04-26T21:58:48.623 INFO:tasks.workunit.client.0.smithi072.stdout:agetty 1064 root rtd unknown /proc/1064/root (readlink: Permission denied) 2024-04-26T21:58:48.623 INFO:tasks.workunit.client.0.smithi072.stdout:agetty 1064 root txt unknown /proc/1064/exe (readlink: Permission denied) 2024-04-26T21:58:48.623 INFO:tasks.workunit.client.0.smithi072.stdout:agetty 1064 root NOFD /proc/1064/fd (opendir: Permission denied) 2024-04-26T21:58:48.623 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/5 4130 root cwd unknown /proc/4130/cwd (readlink: Permission denied) 2024-04-26T21:58:48.623 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/5 4130 root rtd unknown /proc/4130/root (readlink: Permission denied) 2024-04-26T21:58:48.624 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/5 4130 root txt unknown /proc/4130/exe (readlink: Permission denied) 2024-04-26T21:58:48.624 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/5 4130 root NOFD /proc/4130/fd (opendir: Permission denied) 2024-04-26T21:58:48.624 INFO:tasks.workunit.client.0.smithi072.stdout:systemd 4208 ubuntu cwd DIR 8,1 4096 2 / 2024-04-26T21:58:48.624 INFO:tasks.workunit.client.0.smithi072.stdout:systemd 4208 ubuntu rtd DIR 8,1 4096 2 / 2024-04-26T21:58:48.624 INFO:tasks.workunit.client.0.smithi072.stdout:systemd 4208 ubuntu txt REG 8,1 98216 18583 /usr/lib/systemd/systemd 2024-04-26T21:58:48.624 INFO:tasks.workunit.client.0.smithi072.stdout:systemd 4208 ubuntu DEL REG 8,1 61581 /etc/selinux/targeted/contexts/files/file_contexts.bin 2024-04-26T21:58:48.624 INFO:tasks.workunit.client.0.smithi072.stdout:systemd 4208 ubuntu mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-26T21:58:48.624 INFO:tasks.workunit.client.0.smithi072.stdout:systemd 4208 ubuntu mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-26T21:58:48.624 INFO:tasks.workunit.client.0.smithi072.stdout:systemd 4208 ubuntu mem REG 8,1 882376 7182 /usr/lib64/libzstd.so.1.5.1 2024-04-26T21:58:48.624 INFO:tasks.workunit.client.0.smithi072.stdout:systemd 4208 ubuntu mem REG 8,1 1714208 7271 /usr/lib64/libp11-kit.so.0.3.1 2024-04-26T21:58:48.624 INFO:tasks.workunit.client.0.smithi072.stdout:systemd 4208 ubuntu DEL REG 8,1 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-26T21:58:48.625 INFO:tasks.workunit.client.0.smithi072.stdout:systemd 4208 ubuntu mem REG 8,1 1288504 7366 /usr/lib64/libgcrypt.so.20.4.0 2024-04-26T21:58:48.625 INFO:tasks.workunit.client.0.smithi072.stdout:systemd 4208 ubuntu mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-26T21:58:48.625 INFO:tasks.workunit.client.0.smithi072.stdout:systemd 4208 ubuntu DEL REG 8,1 62167 /etc/selinux/targeted/contexts/files/file_contexts.homedirs.bin 2024-04-26T21:58:48.625 INFO:tasks.workunit.client.0.smithi072.stdout:systemd 4208 ubuntu mem REG 8,1 44784 7268 /usr/lib64/libffi.so.8.1.0 2024-04-26T21:58:48.625 INFO:tasks.workunit.client.0.smithi072.stdout:systemd 4208 ubuntu mem REG 8,1 153600 7292 /usr/lib64/libgpg-error.so.0.32.0 2024-04-26T21:58:48.625 INFO:tasks.workunit.client.0.smithi072.stdout:systemd 4208 ubuntu mem REG 8,1 28552 7287 /usr/lib64/libattr.so.1.1.2501 2024-04-26T21:58:48.625 INFO:tasks.workunit.client.0.smithi072.stdout:systemd 4208 ubuntu mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-26T21:58:48.625 INFO:tasks.workunit.client.0.smithi072.stdout:systemd 4208 ubuntu mem REG 8,1 32528 7206 /usr/lib64/libcap-ng.so.0.0.0 2024-04-26T21:58:48.625 INFO:tasks.workunit.client.0.smithi072.stdout:systemd 4208 ubuntu mem REG 8,1 40424 8439 /usr/lib64/libeconf.so.0.4.1 2024-04-26T21:58:48.625 INFO:tasks.workunit.client.0.smithi072.stdout:systemd 4208 ubuntu mem REG 8,1 178728 7177 /usr/lib64/liblzma.so.5.2.5 2024-04-26T21:58:48.626 INFO:tasks.workunit.client.0.smithi072.stdout:systemd 4208 ubuntu mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-26T21:58:48.626 INFO:tasks.workunit.client.0.smithi072.stdout:systemd 4208 ubuntu mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-26T21:58:48.626 INFO:tasks.workunit.client.0.smithi072.stdout:systemd 4208 ubuntu mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-26T21:58:48.626 INFO:tasks.workunit.client.0.smithi072.stdout:systemd 4208 ubuntu mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-26T21:58:48.626 INFO:tasks.workunit.client.0.smithi072.stdout:systemd 4208 ubuntu mem REG 8,1 40496 7288 /usr/lib64/libacl.so.1.1.2301 2024-04-26T21:58:48.626 INFO:tasks.workunit.client.0.smithi072.stdout:systemd 4208 ubuntu mem REG 8,1 291512 7639 /usr/lib64/libmount.so.1.1.0 2024-04-26T21:58:48.626 INFO:tasks.workunit.client.0.smithi072.stdout:systemd 4208 ubuntu mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-26T21:58:48.626 INFO:tasks.workunit.client.0.smithi072.stdout:systemd 4208 ubuntu mem REG 8,1 138848 7210 /usr/lib64/libaudit.so.1.0.0 2024-04-26T21:58:48.626 INFO:tasks.workunit.client.0.smithi072.stdout:systemd 4208 ubuntu mem REG 8,1 3785856 46432 /usr/lib64/systemd/libsystemd-shared-252.so 2024-04-26T21:58:48.626 INFO:tasks.workunit.client.0.smithi072.stdout:systemd 4208 ubuntu mem REG 8,1 107424 7663 /usr/lib64/libkmod.so.2.3.6 2024-04-26T21:58:48.627 INFO:tasks.workunit.client.0.smithi072.stdout:systemd 4208 ubuntu mem REG 8,1 69592 8448 /usr/lib64/libpam.so.0.85.1 2024-04-26T21:58:48.627 INFO:tasks.workunit.client.0.smithi072.stdout:systemd 4208 ubuntu mem REG 8,1 127680 7455 /usr/lib64/libseccomp.so.2.5.2 2024-04-26T21:58:48.627 INFO:tasks.workunit.client.0.smithi072.stdout:systemd 4208 ubuntu mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-26T21:58:48.627 INFO:tasks.workunit.client.0.smithi072.stdout:systemd 4208 ubuntu mem REG 8,1 2157904 46429 /usr/lib64/systemd/libsystemd-core-252.so 2024-04-26T21:58:48.627 INFO:tasks.workunit.client.0.smithi072.stdout:systemd 4208 ubuntu mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-26T21:58:48.627 INFO:tasks.workunit.client.0.smithi072.stdout:systemd 4208 ubuntu mem REG 0,20 0 19 /sys/fs/selinux/status 2024-04-26T21:58:48.628 INFO:tasks.workunit.client.0.smithi072.stdout:systemd 4208 ubuntu 0r CHR 1,3 0t0 4 /dev/null 2024-04-26T21:58:48.628 INFO:tasks.workunit.client.0.smithi072.stdout:systemd 4208 ubuntu 1u unix 0x0000000000000000 0t0 12322 type=STREAM (CONNECTED) 2024-04-26T21:58:48.628 INFO:tasks.workunit.client.0.smithi072.stdout:systemd 4208 ubuntu 2u unix 0x0000000000000000 0t0 12322 type=STREAM (CONNECTED) 2024-04-26T21:58:48.628 INFO:tasks.workunit.client.0.smithi072.stdout:systemd 4208 ubuntu 3u unix 0x0000000000000000 0t0 12350 type=DGRAM (CONNECTED) 2024-04-26T21:58:48.628 INFO:tasks.workunit.client.0.smithi072.stdout:systemd 4208 ubuntu 4u a_inode 0,15 0 50 [eventpoll:5,6,8,9,11,12,14,15,16,17,19,20,21] 2024-04-26T21:58:48.628 INFO:tasks.workunit.client.0.smithi072.stdout:systemd 4208 ubuntu 5u a_inode 0,15 0 50 [signalfd] 2024-04-26T21:58:48.628 INFO:tasks.workunit.client.0.smithi072.stdout:systemd 4208 ubuntu 6r a_inode 0,15 0 50 inotify 2024-04-26T21:58:48.628 INFO:tasks.workunit.client.0.smithi072.stdout:systemd 4208 ubuntu 7r DIR 0,26 0 3581 /sys/fs/cgroup/user.slice/user-1000.slice/user@1000.service 2024-04-26T21:58:48.628 INFO:tasks.workunit.client.0.smithi072.stdout:systemd 4208 ubuntu 8u a_inode 0,15 0 50 [timerfd] 2024-04-26T21:58:48.628 INFO:tasks.workunit.client.0.smithi072.stdout:systemd 4208 ubuntu 9u a_inode 0,15 0 50 [eventpoll:10,13] 2024-04-26T21:58:48.628 INFO:tasks.workunit.client.0.smithi072.stdout:systemd 4208 ubuntu 10r REG 0,21 0 12339 /proc/4208/mountinfo 2024-04-26T21:58:48.628 INFO:tasks.workunit.client.0.smithi072.stdout:systemd 4208 ubuntu 11r a_inode 0,15 0 50 inotify 2024-04-26T21:58:48.629 INFO:tasks.workunit.client.0.smithi072.stdout:systemd 4208 ubuntu 12u unix 0x0000000000000000 0t0 12372 /run/user/1000/bus type=STREAM (LISTEN) 2024-04-26T21:58:48.629 INFO:tasks.workunit.client.0.smithi072.stdout:systemd 4208 ubuntu 13r a_inode 0,15 0 50 inotify 2024-04-26T21:58:48.629 INFO:tasks.workunit.client.0.smithi072.stdout:systemd 4208 ubuntu 14r REG 0,21 0 4026532078 /proc/swaps 2024-04-26T21:58:48.629 INFO:tasks.workunit.client.0.smithi072.stdout:systemd 4208 ubuntu 15u netlink 0t0 12354 KOBJECT_UEVENT 2024-04-26T21:58:48.629 INFO:tasks.workunit.client.0.smithi072.stdout:systemd 4208 ubuntu 16u unix 0x0000000000000000 0t0 12359 /run/user/1000/systemd/notify type=DGRAM (UNCONNECTED) 2024-04-26T21:58:48.629 INFO:tasks.workunit.client.0.smithi072.stdout:systemd 4208 ubuntu 17u unix 0x0000000000000000 0t0 12360 type=DGRAM (CONNECTED) 2024-04-26T21:58:48.629 INFO:tasks.workunit.client.0.smithi072.stdout:systemd 4208 ubuntu 18u unix 0x0000000000000000 0t0 12361 type=DGRAM (CONNECTED) 2024-04-26T21:58:48.629 INFO:tasks.workunit.client.0.smithi072.stdout:systemd 4208 ubuntu 19u unix 0x0000000000000000 0t0 12362 /run/user/1000/systemd/private type=STREAM (LISTEN) 2024-04-26T21:58:48.629 INFO:tasks.workunit.client.0.smithi072.stdout:systemd 4208 ubuntu 20u unix 0x0000000000000000 0t0 12364 type=STREAM (CONNECTED) 2024-04-26T21:58:48.629 INFO:tasks.workunit.client.0.smithi072.stdout:systemd 4208 ubuntu 21u a_inode 0,15 0 50 [timerfd] 2024-04-26T21:58:48.629 INFO:tasks.workunit.client.0.smithi072.stdout:(sd-pam) 4210 ubuntu cwd unknown /proc/4210/cwd (readlink: Permission denied) 2024-04-26T21:58:48.629 INFO:tasks.workunit.client.0.smithi072.stdout:(sd-pam) 4210 ubuntu rtd unknown /proc/4210/root (readlink: Permission denied) 2024-04-26T21:58:48.629 INFO:tasks.workunit.client.0.smithi072.stdout:(sd-pam) 4210 ubuntu txt unknown /proc/4210/exe (readlink: Permission denied) 2024-04-26T21:58:48.629 INFO:tasks.workunit.client.0.smithi072.stdout:(sd-pam) 4210 ubuntu NOFD /proc/4210/fd (opendir: Permission denied) 2024-04-26T21:58:48.630 INFO:tasks.workunit.client.0.smithi072.stdout:rsyslogd 4473 root cwd unknown /proc/4473/cwd (readlink: Permission denied) 2024-04-26T21:58:48.630 INFO:tasks.workunit.client.0.smithi072.stdout:rsyslogd 4473 root rtd unknown /proc/4473/root (readlink: Permission denied) 2024-04-26T21:58:48.630 INFO:tasks.workunit.client.0.smithi072.stdout:rsyslogd 4473 root txt unknown /proc/4473/exe (readlink: Permission denied) 2024-04-26T21:58:48.630 INFO:tasks.workunit.client.0.smithi072.stdout:rsyslogd 4473 root NOFD /proc/4473/fd (opendir: Permission denied) 2024-04-26T21:58:48.630 INFO:tasks.workunit.client.0.smithi072.stdout:rsyslogd 4473 4475 in:imjour root cwd unknown /proc/4473/task/4475/cwd (readlink: Permission denied) 2024-04-26T21:58:48.630 INFO:tasks.workunit.client.0.smithi072.stdout:rsyslogd 4473 4475 in:imjour root rtd unknown /proc/4473/task/4475/root (readlink: Permission denied) 2024-04-26T21:58:48.630 INFO:tasks.workunit.client.0.smithi072.stdout:rsyslogd 4473 4475 in:imjour root txt unknown /proc/4473/task/4475/exe (readlink: Permission denied) 2024-04-26T21:58:48.630 INFO:tasks.workunit.client.0.smithi072.stdout:rsyslogd 4473 4475 in:imjour root NOFD /proc/4473/task/4475/fd (opendir: Permission denied) 2024-04-26T21:58:48.630 INFO:tasks.workunit.client.0.smithi072.stdout:rsyslogd 4473 4476 rs:main root cwd unknown /proc/4473/task/4476/cwd (readlink: Permission denied) 2024-04-26T21:58:48.630 INFO:tasks.workunit.client.0.smithi072.stdout:rsyslogd 4473 4476 rs:main root rtd unknown /proc/4473/task/4476/root (readlink: Permission denied) 2024-04-26T21:58:48.630 INFO:tasks.workunit.client.0.smithi072.stdout:rsyslogd 4473 4476 rs:main root txt unknown /proc/4473/task/4476/exe (readlink: Permission denied) 2024-04-26T21:58:48.630 INFO:tasks.workunit.client.0.smithi072.stdout:rsyslogd 4473 4476 rs:main root NOFD /proc/4473/task/4476/fd (opendir: Permission denied) 2024-04-26T21:58:48.630 INFO:tasks.workunit.client.0.smithi072.stdout:polkitd 17431 polkitd cwd unknown /proc/17431/cwd (readlink: Permission denied) 2024-04-26T21:58:48.630 INFO:tasks.workunit.client.0.smithi072.stdout:polkitd 17431 polkitd rtd unknown /proc/17431/root (readlink: Permission denied) 2024-04-26T21:58:48.631 INFO:tasks.workunit.client.0.smithi072.stdout:polkitd 17431 polkitd txt unknown /proc/17431/exe (readlink: Permission denied) 2024-04-26T21:58:48.631 INFO:tasks.workunit.client.0.smithi072.stdout:polkitd 17431 polkitd NOFD /proc/17431/fd (opendir: Permission denied) 2024-04-26T21:58:48.631 INFO:tasks.workunit.client.0.smithi072.stdout:polkitd 17431 17433 gmain polkitd cwd unknown /proc/17431/task/17433/cwd (readlink: Permission denied) 2024-04-26T21:58:48.631 INFO:tasks.workunit.client.0.smithi072.stdout:polkitd 17431 17433 gmain polkitd rtd unknown /proc/17431/task/17433/root (readlink: Permission denied) 2024-04-26T21:58:48.631 INFO:tasks.workunit.client.0.smithi072.stdout:polkitd 17431 17433 gmain polkitd txt unknown /proc/17431/task/17433/exe (readlink: Permission denied) 2024-04-26T21:58:48.631 INFO:tasks.workunit.client.0.smithi072.stdout:polkitd 17431 17433 gmain polkitd NOFD /proc/17431/task/17433/fd (opendir: Permission denied) 2024-04-26T21:58:48.631 INFO:tasks.workunit.client.0.smithi072.stdout:polkitd 17431 17434 gdbus polkitd cwd unknown /proc/17431/task/17434/cwd (readlink: Permission denied) 2024-04-26T21:58:48.631 INFO:tasks.workunit.client.0.smithi072.stdout:polkitd 17431 17434 gdbus polkitd rtd unknown /proc/17431/task/17434/root (readlink: Permission denied) 2024-04-26T21:58:48.631 INFO:tasks.workunit.client.0.smithi072.stdout:polkitd 17431 17434 gdbus polkitd txt unknown /proc/17431/task/17434/exe (readlink: Permission denied) 2024-04-26T21:58:48.631 INFO:tasks.workunit.client.0.smithi072.stdout:polkitd 17431 17434 gdbus polkitd NOFD /proc/17431/task/17434/fd (opendir: Permission denied) 2024-04-26T21:58:48.631 INFO:tasks.workunit.client.0.smithi072.stdout:polkitd 17431 17435 JS\x20Hel polkitd cwd unknown /proc/17431/task/17435/cwd (readlink: Permission denied) 2024-04-26T21:58:48.631 INFO:tasks.workunit.client.0.smithi072.stdout:polkitd 17431 17435 JS\x20Hel polkitd rtd unknown /proc/17431/task/17435/root (readlink: Permission denied) 2024-04-26T21:58:48.631 INFO:tasks.workunit.client.0.smithi072.stdout:polkitd 17431 17435 JS\x20Hel polkitd txt unknown /proc/17431/task/17435/exe (readlink: Permission denied) 2024-04-26T21:58:48.631 INFO:tasks.workunit.client.0.smithi072.stdout:polkitd 17431 17435 JS\x20Hel polkitd NOFD /proc/17431/task/17435/fd (opendir: Permission denied) 2024-04-26T21:58:48.632 INFO:tasks.workunit.client.0.smithi072.stdout:polkitd 17431 17436 JS\x20Hel polkitd cwd unknown /proc/17431/task/17436/cwd (readlink: Permission denied) 2024-04-26T21:58:48.632 INFO:tasks.workunit.client.0.smithi072.stdout:polkitd 17431 17436 JS\x20Hel polkitd rtd unknown /proc/17431/task/17436/root (readlink: Permission denied) 2024-04-26T21:58:48.632 INFO:tasks.workunit.client.0.smithi072.stdout:polkitd 17431 17436 JS\x20Hel polkitd txt unknown /proc/17431/task/17436/exe (readlink: Permission denied) 2024-04-26T21:58:48.632 INFO:tasks.workunit.client.0.smithi072.stdout:polkitd 17431 17436 JS\x20Hel polkitd NOFD /proc/17431/task/17436/fd (opendir: Permission denied) 2024-04-26T21:58:48.632 INFO:tasks.workunit.client.0.smithi072.stdout:polkitd 17431 17437 JS\x20Hel polkitd cwd unknown /proc/17431/task/17437/cwd (readlink: Permission denied) 2024-04-26T21:58:48.632 INFO:tasks.workunit.client.0.smithi072.stdout:polkitd 17431 17437 JS\x20Hel polkitd rtd unknown /proc/17431/task/17437/root (readlink: Permission denied) 2024-04-26T21:58:48.632 INFO:tasks.workunit.client.0.smithi072.stdout:polkitd 17431 17437 JS\x20Hel polkitd txt unknown /proc/17431/task/17437/exe (readlink: Permission denied) 2024-04-26T21:58:48.632 INFO:tasks.workunit.client.0.smithi072.stdout:polkitd 17431 17437 JS\x20Hel polkitd NOFD /proc/17431/task/17437/fd (opendir: Permission denied) 2024-04-26T21:58:48.632 INFO:tasks.workunit.client.0.smithi072.stdout:polkitd 17431 17438 JS\x20Hel polkitd cwd unknown /proc/17431/task/17438/cwd (readlink: Permission denied) 2024-04-26T21:58:48.632 INFO:tasks.workunit.client.0.smithi072.stdout:polkitd 17431 17438 JS\x20Hel polkitd rtd unknown /proc/17431/task/17438/root (readlink: Permission denied) 2024-04-26T21:58:48.632 INFO:tasks.workunit.client.0.smithi072.stdout:polkitd 17431 17438 JS\x20Hel polkitd txt unknown /proc/17431/task/17438/exe (readlink: Permission denied) 2024-04-26T21:58:48.632 INFO:tasks.workunit.client.0.smithi072.stdout:polkitd 17431 17438 JS\x20Hel polkitd NOFD /proc/17431/task/17438/fd (opendir: Permission denied) 2024-04-26T21:58:48.632 INFO:tasks.workunit.client.0.smithi072.stdout:polkitd 17431 17439 JS\x20Hel polkitd cwd unknown /proc/17431/task/17439/cwd (readlink: Permission denied) 2024-04-26T21:58:48.632 INFO:tasks.workunit.client.0.smithi072.stdout:polkitd 17431 17439 JS\x20Hel polkitd rtd unknown /proc/17431/task/17439/root (readlink: Permission denied) 2024-04-26T21:58:48.632 INFO:tasks.workunit.client.0.smithi072.stdout:polkitd 17431 17439 JS\x20Hel polkitd txt unknown /proc/17431/task/17439/exe (readlink: Permission denied) 2024-04-26T21:58:48.633 INFO:tasks.workunit.client.0.smithi072.stdout:polkitd 17431 17439 JS\x20Hel polkitd NOFD /proc/17431/task/17439/fd (opendir: Permission denied) 2024-04-26T21:58:48.633 INFO:tasks.workunit.client.0.smithi072.stdout:polkitd 17431 17440 JS\x20Hel polkitd cwd unknown /proc/17431/task/17440/cwd (readlink: Permission denied) 2024-04-26T21:58:48.633 INFO:tasks.workunit.client.0.smithi072.stdout:polkitd 17431 17440 JS\x20Hel polkitd rtd unknown /proc/17431/task/17440/root (readlink: Permission denied) 2024-04-26T21:58:48.633 INFO:tasks.workunit.client.0.smithi072.stdout:polkitd 17431 17440 JS\x20Hel polkitd txt unknown /proc/17431/task/17440/exe (readlink: Permission denied) 2024-04-26T21:58:48.633 INFO:tasks.workunit.client.0.smithi072.stdout:polkitd 17431 17440 JS\x20Hel polkitd NOFD /proc/17431/task/17440/fd (opendir: Permission denied) 2024-04-26T21:58:48.633 INFO:tasks.workunit.client.0.smithi072.stdout:polkitd 17431 17441 JS\x20Hel polkitd cwd unknown /proc/17431/task/17441/cwd (readlink: Permission denied) 2024-04-26T21:58:48.633 INFO:tasks.workunit.client.0.smithi072.stdout:polkitd 17431 17441 JS\x20Hel polkitd rtd unknown /proc/17431/task/17441/root (readlink: Permission denied) 2024-04-26T21:58:48.633 INFO:tasks.workunit.client.0.smithi072.stdout:polkitd 17431 17441 JS\x20Hel polkitd txt unknown /proc/17431/task/17441/exe (readlink: Permission denied) 2024-04-26T21:58:48.633 INFO:tasks.workunit.client.0.smithi072.stdout:polkitd 17431 17441 JS\x20Hel polkitd NOFD /proc/17431/task/17441/fd (opendir: Permission denied) 2024-04-26T21:58:48.633 INFO:tasks.workunit.client.0.smithi072.stdout:polkitd 17431 17442 JS\x20Hel polkitd cwd unknown /proc/17431/task/17442/cwd (readlink: Permission denied) 2024-04-26T21:58:48.633 INFO:tasks.workunit.client.0.smithi072.stdout:polkitd 17431 17442 JS\x20Hel polkitd rtd unknown /proc/17431/task/17442/root (readlink: Permission denied) 2024-04-26T21:58:48.633 INFO:tasks.workunit.client.0.smithi072.stdout:polkitd 17431 17442 JS\x20Hel polkitd txt unknown /proc/17431/task/17442/exe (readlink: Permission denied) 2024-04-26T21:58:48.633 INFO:tasks.workunit.client.0.smithi072.stdout:polkitd 17431 17442 JS\x20Hel polkitd NOFD /proc/17431/task/17442/fd (opendir: Permission denied) 2024-04-26T21:58:48.633 INFO:tasks.workunit.client.0.smithi072.stdout:polkitd 17431 17443 polkitd polkitd cwd unknown /proc/17431/task/17443/cwd (readlink: Permission denied) 2024-04-26T21:58:48.634 INFO:tasks.workunit.client.0.smithi072.stdout:polkitd 17431 17443 polkitd polkitd rtd unknown /proc/17431/task/17443/root (readlink: Permission denied) 2024-04-26T21:58:48.634 INFO:tasks.workunit.client.0.smithi072.stdout:polkitd 17431 17443 polkitd polkitd txt unknown /proc/17431/task/17443/exe (readlink: Permission denied) 2024-04-26T21:58:48.634 INFO:tasks.workunit.client.0.smithi072.stdout:polkitd 17431 17443 polkitd polkitd NOFD /proc/17431/task/17443/fd (opendir: Permission denied) 2024-04-26T21:58:48.634 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 17883 root cwd unknown /proc/17883/cwd (readlink: Permission denied) 2024-04-26T21:58:48.634 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 17883 root rtd unknown /proc/17883/root (readlink: Permission denied) 2024-04-26T21:58:48.634 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 17883 root txt unknown /proc/17883/exe (readlink: Permission denied) 2024-04-26T21:58:48.634 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 17883 root NOFD /proc/17883/fd (opendir: Permission denied) 2024-04-26T21:58:48.634 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 17913 root cwd unknown /proc/17913/cwd (readlink: Permission denied) 2024-04-26T21:58:48.634 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 17913 root rtd unknown /proc/17913/root (readlink: Permission denied) 2024-04-26T21:58:48.634 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 17913 root txt unknown /proc/17913/exe (readlink: Permission denied) 2024-04-26T21:58:48.634 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 17913 root NOFD /proc/17913/fd (opendir: Permission denied) 2024-04-26T21:58:48.634 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 17943 root cwd unknown /proc/17943/cwd (readlink: Permission denied) 2024-04-26T21:58:48.634 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 17943 root rtd unknown /proc/17943/root (readlink: Permission denied) 2024-04-26T21:58:48.634 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 17943 root txt unknown /proc/17943/exe (readlink: Permission denied) 2024-04-26T21:58:48.635 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 17943 root NOFD /proc/17943/fd (opendir: Permission denied) 2024-04-26T21:58:48.635 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 17972 root cwd unknown /proc/17972/cwd (readlink: Permission denied) 2024-04-26T21:58:48.635 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 17972 root rtd unknown /proc/17972/root (readlink: Permission denied) 2024-04-26T21:58:48.635 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 17972 root txt unknown /proc/17972/exe (readlink: Permission denied) 2024-04-26T21:58:48.635 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 17972 root NOFD /proc/17972/fd (opendir: Permission denied) 2024-04-26T21:58:48.635 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 18001 root cwd unknown /proc/18001/cwd (readlink: Permission denied) 2024-04-26T21:58:48.635 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 18001 root rtd unknown /proc/18001/root (readlink: Permission denied) 2024-04-26T21:58:48.635 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 18001 root txt unknown /proc/18001/exe (readlink: Permission denied) 2024-04-26T21:58:48.635 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 18001 root NOFD /proc/18001/fd (opendir: Permission denied) 2024-04-26T21:58:48.635 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 18165 root cwd unknown /proc/18165/cwd (readlink: Permission denied) 2024-04-26T21:58:48.635 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 18165 root rtd unknown /proc/18165/root (readlink: Permission denied) 2024-04-26T21:58:48.635 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 18165 root txt unknown /proc/18165/exe (readlink: Permission denied) 2024-04-26T21:58:48.635 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 18165 root NOFD /proc/18165/fd (opendir: Permission denied) 2024-04-26T21:58:48.635 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 18166 root cwd unknown /proc/18166/cwd (readlink: Permission denied) 2024-04-26T21:58:48.636 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 18166 root rtd unknown /proc/18166/root (readlink: Permission denied) 2024-04-26T21:58:48.636 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 18166 root txt unknown /proc/18166/exe (readlink: Permission denied) 2024-04-26T21:58:48.636 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 18166 root NOFD /proc/18166/fd (opendir: Permission denied) 2024-04-26T21:58:48.636 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 18167 root cwd unknown /proc/18167/cwd (readlink: Permission denied) 2024-04-26T21:58:48.636 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 18167 root rtd unknown /proc/18167/root (readlink: Permission denied) 2024-04-26T21:58:48.636 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 18167 root txt unknown /proc/18167/exe (readlink: Permission denied) 2024-04-26T21:58:48.636 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 18167 root NOFD /proc/18167/fd (opendir: Permission denied) 2024-04-26T21:58:48.636 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 18168 root cwd unknown /proc/18168/cwd (readlink: Permission denied) 2024-04-26T21:58:48.636 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 18168 root rtd unknown /proc/18168/root (readlink: Permission denied) 2024-04-26T21:58:48.636 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 18168 root txt unknown /proc/18168/exe (readlink: Permission denied) 2024-04-26T21:58:48.636 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 18168 root NOFD /proc/18168/fd (opendir: Permission denied) 2024-04-26T21:58:48.636 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 18169 root cwd unknown /proc/18169/cwd (readlink: Permission denied) 2024-04-26T21:58:48.636 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 18169 root rtd unknown /proc/18169/root (readlink: Permission denied) 2024-04-26T21:58:48.636 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 18169 root txt unknown /proc/18169/exe (readlink: Permission denied) 2024-04-26T21:58:48.637 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 18169 root NOFD /proc/18169/fd (opendir: Permission denied) 2024-04-26T21:58:48.637 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 18170 root cwd unknown /proc/18170/cwd (readlink: Permission denied) 2024-04-26T21:58:48.637 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 18170 root rtd unknown /proc/18170/root (readlink: Permission denied) 2024-04-26T21:58:48.637 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 18170 root txt unknown /proc/18170/exe (readlink: Permission denied) 2024-04-26T21:58:48.637 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 18170 root NOFD /proc/18170/fd (opendir: Permission denied) 2024-04-26T21:58:48.637 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 18171 root cwd unknown /proc/18171/cwd (readlink: Permission denied) 2024-04-26T21:58:48.637 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 18171 root rtd unknown /proc/18171/root (readlink: Permission denied) 2024-04-26T21:58:48.637 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 18171 root txt unknown /proc/18171/exe (readlink: Permission denied) 2024-04-26T21:58:48.637 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 18171 root NOFD /proc/18171/fd (opendir: Permission denied) 2024-04-26T21:58:48.637 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/1 18172 root cwd unknown /proc/18172/cwd (readlink: Permission denied) 2024-04-26T21:58:48.637 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/1 18172 root rtd unknown /proc/18172/root (readlink: Permission denied) 2024-04-26T21:58:48.637 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/1 18172 root txt unknown /proc/18172/exe (readlink: Permission denied) 2024-04-26T21:58:48.637 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/1 18172 root NOFD /proc/18172/fd (opendir: Permission denied) 2024-04-26T21:58:48.637 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 18173 root cwd unknown /proc/18173/cwd (readlink: Permission denied) 2024-04-26T21:58:48.637 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 18173 root rtd unknown /proc/18173/root (readlink: Permission denied) 2024-04-26T21:58:48.638 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 18173 root txt unknown /proc/18173/exe (readlink: Permission denied) 2024-04-26T21:58:48.638 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 18173 root NOFD /proc/18173/fd (opendir: Permission denied) 2024-04-26T21:58:48.638 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 18174 root cwd unknown /proc/18174/cwd (readlink: Permission denied) 2024-04-26T21:58:48.638 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 18174 root rtd unknown /proc/18174/root (readlink: Permission denied) 2024-04-26T21:58:48.638 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 18174 root txt unknown /proc/18174/exe (readlink: Permission denied) 2024-04-26T21:58:48.638 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/R 18174 root NOFD /proc/18174/fd (opendir: Permission denied) 2024-04-26T21:58:48.638 INFO:tasks.workunit.client.0.smithi072.stdout:xfsaild/d 18175 root cwd unknown /proc/18175/cwd (readlink: Permission denied) 2024-04-26T21:58:48.638 INFO:tasks.workunit.client.0.smithi072.stdout:xfsaild/d 18175 root rtd unknown /proc/18175/root (readlink: Permission denied) 2024-04-26T21:58:48.638 INFO:tasks.workunit.client.0.smithi072.stdout:xfsaild/d 18175 root txt unknown /proc/18175/exe (readlink: Permission denied) 2024-04-26T21:58:48.638 INFO:tasks.workunit.client.0.smithi072.stdout:xfsaild/d 18175 root NOFD /proc/18175/fd (opendir: Permission denied) 2024-04-26T21:58:48.638 INFO:tasks.workunit.client.0.smithi072.stdout:sshd 19105 root cwd unknown /proc/19105/cwd (readlink: Permission denied) 2024-04-26T21:58:48.638 INFO:tasks.workunit.client.0.smithi072.stdout:sshd 19105 root rtd unknown /proc/19105/root (readlink: Permission denied) 2024-04-26T21:58:48.638 INFO:tasks.workunit.client.0.smithi072.stdout:sshd 19105 root txt unknown /proc/19105/exe (readlink: Permission denied) 2024-04-26T21:58:48.639 INFO:tasks.workunit.client.0.smithi072.stdout:sshd 19105 root NOFD /proc/19105/fd (opendir: Permission denied) 2024-04-26T21:58:48.639 INFO:tasks.workunit.client.0.smithi072.stdout:sshd 19108 ubuntu cwd unknown /proc/19108/cwd (readlink: Permission denied) 2024-04-26T21:58:48.639 INFO:tasks.workunit.client.0.smithi072.stdout:sshd 19108 ubuntu rtd unknown /proc/19108/root (readlink: Permission denied) 2024-04-26T21:58:48.639 INFO:tasks.workunit.client.0.smithi072.stdout:sshd 19108 ubuntu txt unknown /proc/19108/exe (readlink: Permission denied) 2024-04-26T21:58:48.639 INFO:tasks.workunit.client.0.smithi072.stdout:sshd 19108 ubuntu NOFD /proc/19108/fd (opendir: Permission denied) 2024-04-26T21:58:48.639 INFO:tasks.workunit.client.0.smithi072.stdout:chronyd 19169 chrony cwd unknown /proc/19169/cwd (readlink: Permission denied) 2024-04-26T21:58:48.639 INFO:tasks.workunit.client.0.smithi072.stdout:chronyd 19169 chrony rtd unknown /proc/19169/root (readlink: Permission denied) 2024-04-26T21:58:48.639 INFO:tasks.workunit.client.0.smithi072.stdout:chronyd 19169 chrony txt unknown /proc/19169/exe (readlink: Permission denied) 2024-04-26T21:58:48.639 INFO:tasks.workunit.client.0.smithi072.stdout:chronyd 19169 chrony NOFD /proc/19169/fd (opendir: Permission denied) 2024-04-26T21:58:48.639 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-cras 19556 ceph cwd unknown /proc/19556/cwd (readlink: Permission denied) 2024-04-26T21:58:48.639 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-cras 19556 ceph rtd unknown /proc/19556/root (readlink: Permission denied) 2024-04-26T21:58:48.639 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-cras 19556 ceph txt unknown /proc/19556/exe (readlink: Permission denied) 2024-04-26T21:58:48.639 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-cras 19556 ceph NOFD /proc/19556/fd (opendir: Permission denied) 2024-04-26T21:58:48.639 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/7 20405 root cwd unknown /proc/20405/cwd (readlink: Permission denied) 2024-04-26T21:58:48.639 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/7 20405 root rtd unknown /proc/20405/root (readlink: Permission denied) 2024-04-26T21:58:48.640 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/7 20405 root txt unknown /proc/20405/exe (readlink: Permission denied) 2024-04-26T21:58:48.640 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/7 20405 root NOFD /proc/20405/fd (opendir: Permission denied) 2024-04-26T21:58:48.640 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/7 20408 root cwd unknown /proc/20408/cwd (readlink: Permission denied) 2024-04-26T21:58:48.640 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/7 20408 root rtd unknown /proc/20408/root (readlink: Permission denied) 2024-04-26T21:58:48.640 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/7 20408 root txt unknown /proc/20408/exe (readlink: Permission denied) 2024-04-26T21:58:48.640 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/7 20408 root NOFD /proc/20408/fd (opendir: Permission denied) 2024-04-26T21:58:48.640 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/2 20442 root cwd unknown /proc/20442/cwd (readlink: Permission denied) 2024-04-26T21:58:48.640 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/2 20442 root rtd unknown /proc/20442/root (readlink: Permission denied) 2024-04-26T21:58:48.640 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/2 20442 root txt unknown /proc/20442/exe (readlink: Permission denied) 2024-04-26T21:58:48.640 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/2 20442 root NOFD /proc/20442/fd (opendir: Permission denied) 2024-04-26T21:58:48.640 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/6 20449 root cwd unknown /proc/20449/cwd (readlink: Permission denied) 2024-04-26T21:58:48.640 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/6 20449 root rtd unknown /proc/20449/root (readlink: Permission denied) 2024-04-26T21:58:48.640 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/6 20449 root txt unknown /proc/20449/exe (readlink: Permission denied) 2024-04-26T21:58:48.640 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/6 20449 root NOFD /proc/20449/fd (opendir: Permission denied) 2024-04-26T21:58:48.640 INFO:tasks.workunit.client.0.smithi072.stdout:sudo 27130 root cwd unknown /proc/27130/cwd (readlink: Permission denied) 2024-04-26T21:58:48.641 INFO:tasks.workunit.client.0.smithi072.stdout:sudo 27130 root rtd unknown /proc/27130/root (readlink: Permission denied) 2024-04-26T21:58:48.641 INFO:tasks.workunit.client.0.smithi072.stdout:sudo 27130 root txt unknown /proc/27130/exe (readlink: Permission denied) 2024-04-26T21:58:48.641 INFO:tasks.workunit.client.0.smithi072.stdout:sudo 27130 root NOFD /proc/27130/fd (opendir: Permission denied) 2024-04-26T21:58:48.641 INFO:tasks.workunit.client.0.smithi072.stdout:sudo 27146 root cwd unknown /proc/27146/cwd (readlink: Permission denied) 2024-04-26T21:58:48.641 INFO:tasks.workunit.client.0.smithi072.stdout:sudo 27146 root rtd unknown /proc/27146/root (readlink: Permission denied) 2024-04-26T21:58:48.641 INFO:tasks.workunit.client.0.smithi072.stdout:sudo 27146 root txt unknown /proc/27146/exe (readlink: Permission denied) 2024-04-26T21:58:48.641 INFO:tasks.workunit.client.0.smithi072.stdout:sudo 27146 root NOFD /proc/27146/fd (opendir: Permission denied) 2024-04-26T21:58:48.641 INFO:tasks.workunit.client.0.smithi072.stdout:journalct 27158 root cwd unknown /proc/27158/cwd (readlink: Permission denied) 2024-04-26T21:58:48.641 INFO:tasks.workunit.client.0.smithi072.stdout:journalct 27158 root rtd unknown /proc/27158/root (readlink: Permission denied) 2024-04-26T21:58:48.641 INFO:tasks.workunit.client.0.smithi072.stdout:journalct 27158 root txt unknown /proc/27158/exe (readlink: Permission denied) 2024-04-26T21:58:48.641 INFO:tasks.workunit.client.0.smithi072.stdout:journalct 27158 root NOFD /proc/27158/fd (opendir: Permission denied) 2024-04-26T21:58:48.641 INFO:tasks.workunit.client.0.smithi072.stdout:journalct 27171 root cwd unknown /proc/27171/cwd (readlink: Permission denied) 2024-04-26T21:58:48.641 INFO:tasks.workunit.client.0.smithi072.stdout:journalct 27171 root rtd unknown /proc/27171/root (readlink: Permission denied) 2024-04-26T21:58:48.642 INFO:tasks.workunit.client.0.smithi072.stdout:journalct 27171 root txt unknown /proc/27171/exe (readlink: Permission denied) 2024-04-26T21:58:48.642 INFO:tasks.workunit.client.0.smithi072.stdout:journalct 27171 root NOFD /proc/27171/fd (opendir: Permission denied) 2024-04-26T21:58:48.642 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/0 27407 root cwd unknown /proc/27407/cwd (readlink: Permission denied) 2024-04-26T21:58:48.642 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/0 27407 root rtd unknown /proc/27407/root (readlink: Permission denied) 2024-04-26T21:58:48.642 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/0 27407 root txt unknown /proc/27407/exe (readlink: Permission denied) 2024-04-26T21:58:48.642 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/0 27407 root NOFD /proc/27407/fd (opendir: Permission denied) 2024-04-26T21:58:48.642 INFO:tasks.workunit.client.0.smithi072.stdout:conmon 28134 root cwd unknown /proc/28134/cwd (readlink: Permission denied) 2024-04-26T21:58:48.642 INFO:tasks.workunit.client.0.smithi072.stdout:conmon 28134 root rtd unknown /proc/28134/root (readlink: Permission denied) 2024-04-26T21:58:48.642 INFO:tasks.workunit.client.0.smithi072.stdout:conmon 28134 root txt unknown /proc/28134/exe (readlink: Permission denied) 2024-04-26T21:58:48.642 INFO:tasks.workunit.client.0.smithi072.stdout:conmon 28134 root NOFD /proc/28134/fd (opendir: Permission denied) 2024-04-26T21:58:48.642 INFO:tasks.workunit.client.0.smithi072.stdout:podman-in 28136 root cwd unknown /proc/28136/cwd (readlink: Permission denied) 2024-04-26T21:58:48.642 INFO:tasks.workunit.client.0.smithi072.stdout:podman-in 28136 root rtd unknown /proc/28136/root (readlink: Permission denied) 2024-04-26T21:58:48.643 INFO:tasks.workunit.client.0.smithi072.stdout:podman-in 28136 root txt unknown /proc/28136/exe (readlink: Permission denied) 2024-04-26T21:58:48.643 INFO:tasks.workunit.client.0.smithi072.stdout:podman-in 28136 root NOFD /proc/28136/fd (opendir: Permission denied) 2024-04-26T21:58:48.643 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mon 28138 ceph cwd unknown /proc/28138/cwd (readlink: Permission denied) 2024-04-26T21:58:48.643 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mon 28138 ceph rtd unknown /proc/28138/root (readlink: Permission denied) 2024-04-26T21:58:48.643 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mon 28138 ceph txt unknown /proc/28138/exe (readlink: Permission denied) 2024-04-26T21:58:48.643 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mon 28138 ceph NOFD /proc/28138/fd (opendir: Permission denied) 2024-04-26T21:58:48.643 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mon 28138 28139 log ceph cwd unknown /proc/28138/task/28139/cwd (readlink: Permission denied) 2024-04-26T21:58:48.643 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mon 28138 28139 log ceph rtd unknown /proc/28138/task/28139/root (readlink: Permission denied) 2024-04-26T21:58:48.643 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mon 28138 28139 log ceph txt unknown /proc/28138/task/28139/exe (readlink: Permission denied) 2024-04-26T21:58:48.643 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mon 28138 28139 log ceph NOFD /proc/28138/task/28139/fd (opendir: Permission denied) 2024-04-26T21:58:48.643 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mon 28138 28140 service ceph cwd unknown /proc/28138/task/28140/cwd (readlink: Permission denied) 2024-04-26T21:58:48.643 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mon 28138 28140 service ceph rtd unknown /proc/28138/task/28140/root (readlink: Permission denied) 2024-04-26T21:58:48.643 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mon 28138 28140 service ceph txt unknown /proc/28138/task/28140/exe (readlink: Permission denied) 2024-04-26T21:58:48.644 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mon 28138 28140 service ceph NOFD /proc/28138/task/28140/fd (opendir: Permission denied) 2024-04-26T21:58:48.644 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mon 28138 28141 admin_soc ceph cwd unknown /proc/28138/task/28141/cwd (readlink: Permission denied) 2024-04-26T21:58:48.644 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mon 28138 28141 admin_soc ceph rtd unknown /proc/28138/task/28141/root (readlink: Permission denied) 2024-04-26T21:58:48.644 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mon 28138 28141 admin_soc ceph txt unknown /proc/28138/task/28141/exe (readlink: Permission denied) 2024-04-26T21:58:48.644 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mon 28138 28141 admin_soc ceph NOFD /proc/28138/task/28141/fd (opendir: Permission denied) 2024-04-26T21:58:48.644 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mon 28138 28142 signal_ha ceph cwd unknown /proc/28138/task/28142/cwd (readlink: Permission denied) 2024-04-26T21:58:48.644 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mon 28138 28142 signal_ha ceph rtd unknown /proc/28138/task/28142/root (readlink: Permission denied) 2024-04-26T21:58:48.644 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mon 28138 28142 signal_ha ceph txt unknown /proc/28138/task/28142/exe (readlink: Permission denied) 2024-04-26T21:58:48.644 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mon 28138 28142 signal_ha ceph NOFD /proc/28138/task/28142/fd (opendir: Permission denied) 2024-04-26T21:58:48.644 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mon 28138 28143 rocksdb:l ceph cwd unknown /proc/28138/task/28143/cwd (readlink: Permission denied) 2024-04-26T21:58:48.644 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mon 28138 28143 rocksdb:l ceph rtd unknown /proc/28138/task/28143/root (readlink: Permission denied) 2024-04-26T21:58:48.644 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mon 28138 28143 rocksdb:l ceph txt unknown /proc/28138/task/28143/exe (readlink: Permission denied) 2024-04-26T21:58:48.644 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mon 28138 28143 rocksdb:l ceph NOFD /proc/28138/task/28143/fd (opendir: Permission denied) 2024-04-26T21:58:48.645 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mon 28138 28144 rocksdb:h ceph cwd unknown /proc/28138/task/28144/cwd (readlink: Permission denied) 2024-04-26T21:58:48.645 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mon 28138 28144 rocksdb:h ceph rtd unknown /proc/28138/task/28144/root (readlink: Permission denied) 2024-04-26T21:58:48.645 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mon 28138 28144 rocksdb:h ceph txt unknown /proc/28138/task/28144/exe (readlink: Permission denied) 2024-04-26T21:58:48.645 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mon 28138 28144 rocksdb:h ceph NOFD /proc/28138/task/28144/fd (opendir: Permission denied) 2024-04-26T21:58:48.645 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mon 28138 28160 ceph-mon ceph cwd unknown /proc/28138/task/28160/cwd (readlink: Permission denied) 2024-04-26T21:58:48.645 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mon 28138 28160 ceph-mon ceph rtd unknown /proc/28138/task/28160/root (readlink: Permission denied) 2024-04-26T21:58:48.645 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mon 28138 28160 ceph-mon ceph txt unknown /proc/28138/task/28160/exe (readlink: Permission denied) 2024-04-26T21:58:48.645 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mon 28138 28160 ceph-mon ceph NOFD /proc/28138/task/28160/fd (opendir: Permission denied) 2024-04-26T21:58:48.645 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mon 28138 28161 fn_monsto ceph cwd unknown /proc/28138/task/28161/cwd (readlink: Permission denied) 2024-04-26T21:58:48.645 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mon 28138 28161 fn_monsto ceph rtd unknown /proc/28138/task/28161/root (readlink: Permission denied) 2024-04-26T21:58:48.645 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mon 28138 28161 fn_monsto ceph txt unknown /proc/28138/task/28161/exe (readlink: Permission denied) 2024-04-26T21:58:48.645 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mon 28138 28161 fn_monsto ceph NOFD /proc/28138/task/28161/fd (opendir: Permission denied) 2024-04-26T21:58:48.645 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mon 28138 28162 msgr-work ceph cwd unknown /proc/28138/task/28162/cwd (readlink: Permission denied) 2024-04-26T21:58:48.645 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mon 28138 28162 msgr-work ceph rtd unknown /proc/28138/task/28162/root (readlink: Permission denied) 2024-04-26T21:58:48.646 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mon 28138 28162 msgr-work ceph txt unknown /proc/28138/task/28162/exe (readlink: Permission denied) 2024-04-26T21:58:48.646 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mon 28138 28162 msgr-work ceph NOFD /proc/28138/task/28162/fd (opendir: Permission denied) 2024-04-26T21:58:48.646 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mon 28138 28163 msgr-work ceph cwd unknown /proc/28138/task/28163/cwd (readlink: Permission denied) 2024-04-26T21:58:48.646 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mon 28138 28163 msgr-work ceph rtd unknown /proc/28138/task/28163/root (readlink: Permission denied) 2024-04-26T21:58:48.646 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mon 28138 28163 msgr-work ceph txt unknown /proc/28138/task/28163/exe (readlink: Permission denied) 2024-04-26T21:58:48.646 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mon 28138 28163 msgr-work ceph NOFD /proc/28138/task/28163/fd (opendir: Permission denied) 2024-04-26T21:58:48.646 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mon 28138 28164 msgr-work ceph cwd unknown /proc/28138/task/28164/cwd (readlink: Permission denied) 2024-04-26T21:58:48.646 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mon 28138 28164 msgr-work ceph rtd unknown /proc/28138/task/28164/root (readlink: Permission denied) 2024-04-26T21:58:48.646 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mon 28138 28164 msgr-work ceph txt unknown /proc/28138/task/28164/exe (readlink: Permission denied) 2024-04-26T21:58:48.646 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mon 28138 28164 msgr-work ceph NOFD /proc/28138/task/28164/fd (opendir: Permission denied) 2024-04-26T21:58:48.646 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mon 28138 28165 OpHistory ceph cwd unknown /proc/28138/task/28165/cwd (readlink: Permission denied) 2024-04-26T21:58:48.646 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mon 28138 28165 OpHistory ceph rtd unknown /proc/28138/task/28165/root (readlink: Permission denied) 2024-04-26T21:58:48.646 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mon 28138 28165 OpHistory ceph txt unknown /proc/28138/task/28165/exe (readlink: Permission denied) 2024-04-26T21:58:48.646 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mon 28138 28165 OpHistory ceph NOFD /proc/28138/task/28165/fd (opendir: Permission denied) 2024-04-26T21:58:48.647 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mon 28138 28166 fin ceph cwd unknown /proc/28138/task/28166/cwd (readlink: Permission denied) 2024-04-26T21:58:48.647 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mon 28138 28166 fin ceph rtd unknown /proc/28138/task/28166/root (readlink: Permission denied) 2024-04-26T21:58:48.647 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mon 28138 28166 fin ceph txt unknown /proc/28138/task/28166/exe (readlink: Permission denied) 2024-04-26T21:58:48.647 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mon 28138 28166 fin ceph NOFD /proc/28138/task/28166/fd (opendir: Permission denied) 2024-04-26T21:58:48.647 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mon 28138 28167 safe_time ceph cwd unknown /proc/28138/task/28167/cwd (readlink: Permission denied) 2024-04-26T21:58:48.647 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mon 28138 28167 safe_time ceph rtd unknown /proc/28138/task/28167/root (readlink: Permission denied) 2024-04-26T21:58:48.647 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mon 28138 28167 safe_time ceph txt unknown /proc/28138/task/28167/exe (readlink: Permission denied) 2024-04-26T21:58:48.647 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mon 28138 28167 safe_time ceph NOFD /proc/28138/task/28167/fd (opendir: Permission denied) 2024-04-26T21:58:48.647 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mon 28138 28168 cpu_tp ceph cwd unknown /proc/28138/task/28168/cwd (readlink: Permission denied) 2024-04-26T21:58:48.647 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mon 28138 28168 cpu_tp ceph rtd unknown /proc/28138/task/28168/root (readlink: Permission denied) 2024-04-26T21:58:48.647 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mon 28138 28168 cpu_tp ceph txt unknown /proc/28138/task/28168/exe (readlink: Permission denied) 2024-04-26T21:58:48.647 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mon 28138 28168 cpu_tp ceph NOFD /proc/28138/task/28168/fd (opendir: Permission denied) 2024-04-26T21:58:48.647 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mon 28138 28169 cpu_tp ceph cwd unknown /proc/28138/task/28169/cwd (readlink: Permission denied) 2024-04-26T21:58:48.648 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mon 28138 28169 cpu_tp ceph rtd unknown /proc/28138/task/28169/root (readlink: Permission denied) 2024-04-26T21:58:48.648 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mon 28138 28169 cpu_tp ceph txt unknown /proc/28138/task/28169/exe (readlink: Permission denied) 2024-04-26T21:58:48.648 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mon 28138 28169 cpu_tp ceph NOFD /proc/28138/task/28169/fd (opendir: Permission denied) 2024-04-26T21:58:48.648 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mon 28138 28170 cpu_tp ceph cwd unknown /proc/28138/task/28170/cwd (readlink: Permission denied) 2024-04-26T21:58:48.648 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mon 28138 28170 cpu_tp ceph rtd unknown /proc/28138/task/28170/root (readlink: Permission denied) 2024-04-26T21:58:48.648 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mon 28138 28170 cpu_tp ceph txt unknown /proc/28138/task/28170/exe (readlink: Permission denied) 2024-04-26T21:58:48.648 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mon 28138 28170 cpu_tp ceph NOFD /proc/28138/task/28170/fd (opendir: Permission denied) 2024-04-26T21:58:48.648 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mon 28138 28171 cpu_tp ceph cwd unknown /proc/28138/task/28171/cwd (readlink: Permission denied) 2024-04-26T21:58:48.648 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mon 28138 28171 cpu_tp ceph rtd unknown /proc/28138/task/28171/root (readlink: Permission denied) 2024-04-26T21:58:48.648 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mon 28138 28171 cpu_tp ceph txt unknown /proc/28138/task/28171/exe (readlink: Permission denied) 2024-04-26T21:58:48.648 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mon 28138 28171 cpu_tp ceph NOFD /proc/28138/task/28171/fd (opendir: Permission denied) 2024-04-26T21:58:48.649 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mon 28138 28172 ms_dispat ceph cwd unknown /proc/28138/task/28172/cwd (readlink: Permission denied) 2024-04-26T21:58:48.649 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mon 28138 28172 ms_dispat ceph rtd unknown /proc/28138/task/28172/root (readlink: Permission denied) 2024-04-26T21:58:48.649 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mon 28138 28172 ms_dispat ceph txt unknown /proc/28138/task/28172/exe (readlink: Permission denied) 2024-04-26T21:58:48.649 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mon 28138 28172 ms_dispat ceph NOFD /proc/28138/task/28172/fd (opendir: Permission denied) 2024-04-26T21:58:48.649 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mon 28138 28173 ms_local ceph cwd unknown /proc/28138/task/28173/cwd (readlink: Permission denied) 2024-04-26T21:58:48.649 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mon 28138 28173 ms_local ceph rtd unknown /proc/28138/task/28173/root (readlink: Permission denied) 2024-04-26T21:58:48.649 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mon 28138 28173 ms_local ceph txt unknown /proc/28138/task/28173/exe (readlink: Permission denied) 2024-04-26T21:58:48.649 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mon 28138 28173 ms_local ceph NOFD /proc/28138/task/28173/fd (opendir: Permission denied) 2024-04-26T21:58:48.649 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mon 28138 28174 safe_time ceph cwd unknown /proc/28138/task/28174/cwd (readlink: Permission denied) 2024-04-26T21:58:48.649 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mon 28138 28174 safe_time ceph rtd unknown /proc/28138/task/28174/root (readlink: Permission denied) 2024-04-26T21:58:48.649 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mon 28138 28174 safe_time ceph txt unknown /proc/28138/task/28174/exe (readlink: Permission denied) 2024-04-26T21:58:48.649 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mon 28138 28174 safe_time ceph NOFD /proc/28138/task/28174/fd (opendir: Permission denied) 2024-04-26T21:58:48.649 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mon 28138 28175 ms_dispat ceph cwd unknown /proc/28138/task/28175/cwd (readlink: Permission denied) 2024-04-26T21:58:48.649 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mon 28138 28175 ms_dispat ceph rtd unknown /proc/28138/task/28175/root (readlink: Permission denied) 2024-04-26T21:58:48.650 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mon 28138 28175 ms_dispat ceph txt unknown /proc/28138/task/28175/exe (readlink: Permission denied) 2024-04-26T21:58:48.650 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mon 28138 28175 ms_dispat ceph NOFD /proc/28138/task/28175/fd (opendir: Permission denied) 2024-04-26T21:58:48.650 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mon 28138 28176 ms_local ceph cwd unknown /proc/28138/task/28176/cwd (readlink: Permission denied) 2024-04-26T21:58:48.650 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mon 28138 28176 ms_local ceph rtd unknown /proc/28138/task/28176/root (readlink: Permission denied) 2024-04-26T21:58:48.650 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mon 28138 28176 ms_local ceph txt unknown /proc/28138/task/28176/exe (readlink: Permission denied) 2024-04-26T21:58:48.650 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mon 28138 28176 ms_local ceph NOFD /proc/28138/task/28176/fd (opendir: Permission denied) 2024-04-26T21:58:48.650 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mon 28138 61100 rstore_co ceph cwd unknown /proc/28138/task/61100/cwd (readlink: Permission denied) 2024-04-26T21:58:48.650 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mon 28138 61100 rstore_co ceph rtd unknown /proc/28138/task/61100/root (readlink: Permission denied) 2024-04-26T21:58:48.650 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mon 28138 61100 rstore_co ceph txt unknown /proc/28138/task/61100/exe (readlink: Permission denied) 2024-04-26T21:58:48.650 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mon 28138 61100 rstore_co ceph NOFD /proc/28138/task/61100/fd (opendir: Permission denied) 2024-04-26T21:58:48.651 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/3 28303 root cwd unknown /proc/28303/cwd (readlink: Permission denied) 2024-04-26T21:58:48.651 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/3 28303 root rtd unknown /proc/28303/root (readlink: Permission denied) 2024-04-26T21:58:48.651 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/3 28303 root txt unknown /proc/28303/exe (readlink: Permission denied) 2024-04-26T21:58:48.651 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/3 28303 root NOFD /proc/28303/fd (opendir: Permission denied) 2024-04-26T21:58:48.651 INFO:tasks.workunit.client.0.smithi072.stdout:conmon 28363 root cwd unknown /proc/28363/cwd (readlink: Permission denied) 2024-04-26T21:58:48.651 INFO:tasks.workunit.client.0.smithi072.stdout:conmon 28363 root rtd unknown /proc/28363/root (readlink: Permission denied) 2024-04-26T21:58:48.651 INFO:tasks.workunit.client.0.smithi072.stdout:conmon 28363 root txt unknown /proc/28363/exe (readlink: Permission denied) 2024-04-26T21:58:48.651 INFO:tasks.workunit.client.0.smithi072.stdout:conmon 28363 root NOFD /proc/28363/fd (opendir: Permission denied) 2024-04-26T21:58:48.651 INFO:tasks.workunit.client.0.smithi072.stdout:podman-in 28365 root cwd unknown /proc/28365/cwd (readlink: Permission denied) 2024-04-26T21:58:48.651 INFO:tasks.workunit.client.0.smithi072.stdout:podman-in 28365 root rtd unknown /proc/28365/root (readlink: Permission denied) 2024-04-26T21:58:48.651 INFO:tasks.workunit.client.0.smithi072.stdout:podman-in 28365 root txt unknown /proc/28365/exe (readlink: Permission denied) 2024-04-26T21:58:48.651 INFO:tasks.workunit.client.0.smithi072.stdout:podman-in 28365 root NOFD /proc/28365/fd (opendir: Permission denied) 2024-04-26T21:58:48.651 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 ceph cwd unknown /proc/28367/cwd (readlink: Permission denied) 2024-04-26T21:58:48.651 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 ceph rtd unknown /proc/28367/root (readlink: Permission denied) 2024-04-26T21:58:48.652 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 ceph txt unknown /proc/28367/exe (readlink: Permission denied) 2024-04-26T21:58:48.652 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 ceph NOFD /proc/28367/fd (opendir: Permission denied) 2024-04-26T21:58:48.652 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 30873 log ceph cwd unknown /proc/28367/task/30873/cwd (readlink: Permission denied) 2024-04-26T21:58:48.652 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 30873 log ceph rtd unknown /proc/28367/task/30873/root (readlink: Permission denied) 2024-04-26T21:58:48.652 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 30873 log ceph txt unknown /proc/28367/task/30873/exe (readlink: Permission denied) 2024-04-26T21:58:48.652 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 30873 log ceph NOFD /proc/28367/task/30873/fd (opendir: Permission denied) 2024-04-26T21:58:48.652 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 30875 msgr-work ceph cwd unknown /proc/28367/task/30875/cwd (readlink: Permission denied) 2024-04-26T21:58:48.652 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 30875 msgr-work ceph rtd unknown /proc/28367/task/30875/root (readlink: Permission denied) 2024-04-26T21:58:48.652 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 30875 msgr-work ceph txt unknown /proc/28367/task/30875/exe (readlink: Permission denied) 2024-04-26T21:58:48.652 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 30875 msgr-work ceph NOFD /proc/28367/task/30875/fd (opendir: Permission denied) 2024-04-26T21:58:48.652 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 30876 msgr-work ceph cwd unknown /proc/28367/task/30876/cwd (readlink: Permission denied) 2024-04-26T21:58:48.653 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 30876 msgr-work ceph rtd unknown /proc/28367/task/30876/root (readlink: Permission denied) 2024-04-26T21:58:48.653 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 30876 msgr-work ceph txt unknown /proc/28367/task/30876/exe (readlink: Permission denied) 2024-04-26T21:58:48.653 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 30876 msgr-work ceph NOFD /proc/28367/task/30876/fd (opendir: Permission denied) 2024-04-26T21:58:48.653 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 30877 msgr-work ceph cwd unknown /proc/28367/task/30877/cwd (readlink: Permission denied) 2024-04-26T21:58:48.653 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 30877 msgr-work ceph rtd unknown /proc/28367/task/30877/root (readlink: Permission denied) 2024-04-26T21:58:48.653 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 30877 msgr-work ceph txt unknown /proc/28367/task/30877/exe (readlink: Permission denied) 2024-04-26T21:58:48.653 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 30877 msgr-work ceph NOFD /proc/28367/task/30877/fd (opendir: Permission denied) 2024-04-26T21:58:48.653 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 30881 service ceph cwd unknown /proc/28367/task/30881/cwd (readlink: Permission denied) 2024-04-26T21:58:48.653 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 30881 service ceph rtd unknown /proc/28367/task/30881/root (readlink: Permission denied) 2024-04-26T21:58:48.653 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 30881 service ceph txt unknown /proc/28367/task/30881/exe (readlink: Permission denied) 2024-04-26T21:58:48.653 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 30881 service ceph NOFD /proc/28367/task/30881/fd (opendir: Permission denied) 2024-04-26T21:58:48.653 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 30882 admin_soc ceph cwd unknown /proc/28367/task/30882/cwd (readlink: Permission denied) 2024-04-26T21:58:48.653 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 30882 admin_soc ceph rtd unknown /proc/28367/task/30882/root (readlink: Permission denied) 2024-04-26T21:58:48.653 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 30882 admin_soc ceph txt unknown /proc/28367/task/30882/exe (readlink: Permission denied) 2024-04-26T21:58:48.654 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 30882 admin_soc ceph NOFD /proc/28367/task/30882/fd (opendir: Permission denied) 2024-04-26T21:58:48.654 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 30883 ceph_time ceph cwd unknown /proc/28367/task/30883/cwd (readlink: Permission denied) 2024-04-26T21:58:48.654 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 30883 ceph_time ceph rtd unknown /proc/28367/task/30883/root (readlink: Permission denied) 2024-04-26T21:58:48.654 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 30883 ceph_time ceph txt unknown /proc/28367/task/30883/exe (readlink: Permission denied) 2024-04-26T21:58:48.654 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 30883 ceph_time ceph NOFD /proc/28367/task/30883/fd (opendir: Permission denied) 2024-04-26T21:58:48.654 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 30884 fn_anonym ceph cwd unknown /proc/28367/task/30884/cwd (readlink: Permission denied) 2024-04-26T21:58:48.654 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 30884 fn_anonym ceph rtd unknown /proc/28367/task/30884/root (readlink: Permission denied) 2024-04-26T21:58:48.654 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 30884 fn_anonym ceph txt unknown /proc/28367/task/30884/exe (readlink: Permission denied) 2024-04-26T21:58:48.654 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 30884 fn_anonym ceph NOFD /proc/28367/task/30884/fd (opendir: Permission denied) 2024-04-26T21:58:48.655 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 30885 signal_ha ceph cwd unknown /proc/28367/task/30885/cwd (readlink: Permission denied) 2024-04-26T21:58:48.655 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 30885 signal_ha ceph rtd unknown /proc/28367/task/30885/root (readlink: Permission denied) 2024-04-26T21:58:48.655 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 30885 signal_ha ceph txt unknown /proc/28367/task/30885/exe (readlink: Permission denied) 2024-04-26T21:58:48.655 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 30885 signal_ha ceph NOFD /proc/28367/task/30885/fd (opendir: Permission denied) 2024-04-26T21:58:48.655 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 30886 mgrsb-fin ceph cwd unknown /proc/28367/task/30886/cwd (readlink: Permission denied) 2024-04-26T21:58:48.655 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 30886 mgrsb-fin ceph rtd unknown /proc/28367/task/30886/root (readlink: Permission denied) 2024-04-26T21:58:48.655 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 30886 mgrsb-fin ceph txt unknown /proc/28367/task/30886/exe (readlink: Permission denied) 2024-04-26T21:58:48.655 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 30886 mgrsb-fin ceph NOFD /proc/28367/task/30886/fd (opendir: Permission denied) 2024-04-26T21:58:48.655 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 30887 ms_dispat ceph cwd unknown /proc/28367/task/30887/cwd (readlink: Permission denied) 2024-04-26T21:58:48.655 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 30887 ms_dispat ceph rtd unknown /proc/28367/task/30887/root (readlink: Permission denied) 2024-04-26T21:58:48.655 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 30887 ms_dispat ceph txt unknown /proc/28367/task/30887/exe (readlink: Permission denied) 2024-04-26T21:58:48.656 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 30887 ms_dispat ceph NOFD /proc/28367/task/30887/fd (opendir: Permission denied) 2024-04-26T21:58:48.656 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 30888 ms_local ceph cwd unknown /proc/28367/task/30888/cwd (readlink: Permission denied) 2024-04-26T21:58:48.656 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 30888 ms_local ceph rtd unknown /proc/28367/task/30888/root (readlink: Permission denied) 2024-04-26T21:58:48.656 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 30888 ms_local ceph txt unknown /proc/28367/task/30888/exe (readlink: Permission denied) 2024-04-26T21:58:48.656 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 30888 ms_local ceph NOFD /proc/28367/task/30888/fd (opendir: Permission denied) 2024-04-26T21:58:48.656 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 30889 io_contex ceph cwd unknown /proc/28367/task/30889/cwd (readlink: Permission denied) 2024-04-26T21:58:48.656 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 30889 io_contex ceph rtd unknown /proc/28367/task/30889/root (readlink: Permission denied) 2024-04-26T21:58:48.656 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 30889 io_contex ceph txt unknown /proc/28367/task/30889/exe (readlink: Permission denied) 2024-04-26T21:58:48.656 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 30889 io_contex ceph NOFD /proc/28367/task/30889/fd (opendir: Permission denied) 2024-04-26T21:58:48.656 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 30890 io_contex ceph cwd unknown /proc/28367/task/30890/cwd (readlink: Permission denied) 2024-04-26T21:58:48.656 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 30890 io_contex ceph rtd unknown /proc/28367/task/30890/root (readlink: Permission denied) 2024-04-26T21:58:48.656 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 30890 io_contex ceph txt unknown /proc/28367/task/30890/exe (readlink: Permission denied) 2024-04-26T21:58:48.657 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 30890 io_contex ceph NOFD /proc/28367/task/30890/fd (opendir: Permission denied) 2024-04-26T21:58:48.657 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 30891 safe_time ceph cwd unknown /proc/28367/task/30891/cwd (readlink: Permission denied) 2024-04-26T21:58:48.657 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 30891 safe_time ceph rtd unknown /proc/28367/task/30891/root (readlink: Permission denied) 2024-04-26T21:58:48.657 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 30891 safe_time ceph txt unknown /proc/28367/task/30891/exe (readlink: Permission denied) 2024-04-26T21:58:48.657 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 30891 safe_time ceph NOFD /proc/28367/task/30891/fd (opendir: Permission denied) 2024-04-26T21:58:48.657 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 30892 safe_time ceph cwd unknown /proc/28367/task/30892/cwd (readlink: Permission denied) 2024-04-26T21:58:48.657 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 30892 safe_time ceph rtd unknown /proc/28367/task/30892/root (readlink: Permission denied) 2024-04-26T21:58:48.657 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 30892 safe_time ceph txt unknown /proc/28367/task/30892/exe (readlink: Permission denied) 2024-04-26T21:58:48.657 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 30892 safe_time ceph NOFD /proc/28367/task/30892/fd (opendir: Permission denied) 2024-04-26T21:58:48.657 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 30893 safe_time ceph cwd unknown /proc/28367/task/30893/cwd (readlink: Permission denied) 2024-04-26T21:58:48.657 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 30893 safe_time ceph rtd unknown /proc/28367/task/30893/root (readlink: Permission denied) 2024-04-26T21:58:48.657 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 30893 safe_time ceph txt unknown /proc/28367/task/30893/exe (readlink: Permission denied) 2024-04-26T21:58:48.657 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 30893 safe_time ceph NOFD /proc/28367/task/30893/fd (opendir: Permission denied) 2024-04-26T21:58:48.658 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 30894 fn_anonym ceph cwd unknown /proc/28367/task/30894/cwd (readlink: Permission denied) 2024-04-26T21:58:48.658 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 30894 fn_anonym ceph rtd unknown /proc/28367/task/30894/root (readlink: Permission denied) 2024-04-26T21:58:48.658 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 30894 fn_anonym ceph txt unknown /proc/28367/task/30894/exe (readlink: Permission denied) 2024-04-26T21:58:48.658 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 30894 fn_anonym ceph NOFD /proc/28367/task/30894/fd (opendir: Permission denied) 2024-04-26T21:58:48.658 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 30895 flusher ceph cwd unknown /proc/28367/task/30895/cwd (readlink: Permission denied) 2024-04-26T21:58:48.658 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 30895 flusher ceph rtd unknown /proc/28367/task/30895/root (readlink: Permission denied) 2024-04-26T21:58:48.658 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 30895 flusher ceph txt unknown /proc/28367/task/30895/exe (readlink: Permission denied) 2024-04-26T21:58:48.658 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 30895 flusher ceph NOFD /proc/28367/task/30895/fd (opendir: Permission denied) 2024-04-26T21:58:48.658 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 30896 safe_time ceph cwd unknown /proc/28367/task/30896/cwd (readlink: Permission denied) 2024-04-26T21:58:48.659 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 30896 safe_time ceph rtd unknown /proc/28367/task/30896/root (readlink: Permission denied) 2024-04-26T21:58:48.659 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 30896 safe_time ceph txt unknown /proc/28367/task/30896/exe (readlink: Permission denied) 2024-04-26T21:58:48.659 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 30896 safe_time ceph NOFD /proc/28367/task/30896/fd (opendir: Permission denied) 2024-04-26T21:58:48.659 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31076 OpHistory ceph cwd unknown /proc/28367/task/31076/cwd (readlink: Permission denied) 2024-04-26T21:58:48.659 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31076 OpHistory ceph rtd unknown /proc/28367/task/31076/root (readlink: Permission denied) 2024-04-26T21:58:48.659 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31076 OpHistory ceph txt unknown /proc/28367/task/31076/exe (readlink: Permission denied) 2024-04-26T21:58:48.659 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31076 OpHistory ceph NOFD /proc/28367/task/31076/fd (opendir: Permission denied) 2024-04-26T21:58:48.659 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31077 mgr-fin ceph cwd unknown /proc/28367/task/31077/cwd (readlink: Permission denied) 2024-04-26T21:58:48.659 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31077 mgr-fin ceph rtd unknown /proc/28367/task/31077/root (readlink: Permission denied) 2024-04-26T21:58:48.659 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31077 mgr-fin ceph txt unknown /proc/28367/task/31077/exe (readlink: Permission denied) 2024-04-26T21:58:48.659 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31077 mgr-fin ceph NOFD /proc/28367/task/31077/fd (opendir: Permission denied) 2024-04-26T21:58:48.660 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31078 ms_dispat ceph cwd unknown /proc/28367/task/31078/cwd (readlink: Permission denied) 2024-04-26T21:58:48.660 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31078 ms_dispat ceph rtd unknown /proc/28367/task/31078/root (readlink: Permission denied) 2024-04-26T21:58:48.660 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31078 ms_dispat ceph txt unknown /proc/28367/task/31078/exe (readlink: Permission denied) 2024-04-26T21:58:48.660 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31078 ms_dispat ceph NOFD /proc/28367/task/31078/fd (opendir: Permission denied) 2024-04-26T21:58:48.660 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31079 ms_local ceph cwd unknown /proc/28367/task/31079/cwd (readlink: Permission denied) 2024-04-26T21:58:48.660 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31079 ms_local ceph rtd unknown /proc/28367/task/31079/root (readlink: Permission denied) 2024-04-26T21:58:48.660 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31079 ms_local ceph txt unknown /proc/28367/task/31079/exe (readlink: Permission denied) 2024-04-26T21:58:48.660 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31079 ms_local ceph NOFD /proc/28367/task/31079/fd (opendir: Permission denied) 2024-04-26T21:58:48.660 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31080 safe_time ceph cwd unknown /proc/28367/task/31080/cwd (readlink: Permission denied) 2024-04-26T21:58:48.660 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31080 safe_time ceph rtd unknown /proc/28367/task/31080/root (readlink: Permission denied) 2024-04-26T21:58:48.660 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31080 safe_time ceph txt unknown /proc/28367/task/31080/exe (readlink: Permission denied) 2024-04-26T21:58:48.660 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31080 safe_time ceph NOFD /proc/28367/task/31080/fd (opendir: Permission denied) 2024-04-26T21:58:48.660 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31081 cmdfin ceph cwd unknown /proc/28367/task/31081/cwd (readlink: Permission denied) 2024-04-26T21:58:48.661 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31081 cmdfin ceph rtd unknown /proc/28367/task/31081/root (readlink: Permission denied) 2024-04-26T21:58:48.661 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31081 cmdfin ceph txt unknown /proc/28367/task/31081/exe (readlink: Permission denied) 2024-04-26T21:58:48.661 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31081 cmdfin ceph NOFD /proc/28367/task/31081/fd (opendir: Permission denied) 2024-04-26T21:58:48.661 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31085 io_contex ceph cwd unknown /proc/28367/task/31085/cwd (readlink: Permission denied) 2024-04-26T21:58:48.661 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31085 io_contex ceph rtd unknown /proc/28367/task/31085/root (readlink: Permission denied) 2024-04-26T21:58:48.661 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31085 io_contex ceph txt unknown /proc/28367/task/31085/exe (readlink: Permission denied) 2024-04-26T21:58:48.661 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31085 io_contex ceph NOFD /proc/28367/task/31085/fd (opendir: Permission denied) 2024-04-26T21:58:48.661 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31086 io_contex ceph cwd unknown /proc/28367/task/31086/cwd (readlink: Permission denied) 2024-04-26T21:58:48.661 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31086 io_contex ceph rtd unknown /proc/28367/task/31086/root (readlink: Permission denied) 2024-04-26T21:58:48.661 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31086 io_contex ceph txt unknown /proc/28367/task/31086/exe (readlink: Permission denied) 2024-04-26T21:58:48.661 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31086 io_contex ceph NOFD /proc/28367/task/31086/fd (opendir: Permission denied) 2024-04-26T21:58:48.661 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31087 ceph_time ceph cwd unknown /proc/28367/task/31087/cwd (readlink: Permission denied) 2024-04-26T21:58:48.661 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31087 ceph_time ceph rtd unknown /proc/28367/task/31087/root (readlink: Permission denied) 2024-04-26T21:58:48.661 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31087 ceph_time ceph txt unknown /proc/28367/task/31087/exe (readlink: Permission denied) 2024-04-26T21:58:48.661 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31087 ceph_time ceph NOFD /proc/28367/task/31087/fd (opendir: Permission denied) 2024-04-26T21:58:48.662 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31088 ms_dispat ceph cwd unknown /proc/28367/task/31088/cwd (readlink: Permission denied) 2024-04-26T21:58:48.662 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31088 ms_dispat ceph rtd unknown /proc/28367/task/31088/root (readlink: Permission denied) 2024-04-26T21:58:48.662 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31088 ms_dispat ceph txt unknown /proc/28367/task/31088/exe (readlink: Permission denied) 2024-04-26T21:58:48.662 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31088 ms_dispat ceph NOFD /proc/28367/task/31088/fd (opendir: Permission denied) 2024-04-26T21:58:48.662 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31089 ms_local ceph cwd unknown /proc/28367/task/31089/cwd (readlink: Permission denied) 2024-04-26T21:58:48.662 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31089 ms_local ceph rtd unknown /proc/28367/task/31089/root (readlink: Permission denied) 2024-04-26T21:58:48.662 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31089 ms_local ceph txt unknown /proc/28367/task/31089/exe (readlink: Permission denied) 2024-04-26T21:58:48.662 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31089 ms_local ceph NOFD /proc/28367/task/31089/fd (opendir: Permission denied) 2024-04-26T21:58:48.662 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31090 safe_time ceph cwd unknown /proc/28367/task/31090/cwd (readlink: Permission denied) 2024-04-26T21:58:48.662 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31090 safe_time ceph rtd unknown /proc/28367/task/31090/root (readlink: Permission denied) 2024-04-26T21:58:48.662 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31090 safe_time ceph txt unknown /proc/28367/task/31090/exe (readlink: Permission denied) 2024-04-26T21:58:48.662 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31090 safe_time ceph NOFD /proc/28367/task/31090/fd (opendir: Permission denied) 2024-04-26T21:58:48.662 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31091 safe_time ceph cwd unknown /proc/28367/task/31091/cwd (readlink: Permission denied) 2024-04-26T21:58:48.663 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31091 safe_time ceph rtd unknown /proc/28367/task/31091/root (readlink: Permission denied) 2024-04-26T21:58:48.663 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31091 safe_time ceph txt unknown /proc/28367/task/31091/exe (readlink: Permission denied) 2024-04-26T21:58:48.663 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31091 safe_time ceph NOFD /proc/28367/task/31091/fd (opendir: Permission denied) 2024-04-26T21:58:48.663 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31092 balancer ceph cwd unknown /proc/28367/task/31092/cwd (readlink: Permission denied) 2024-04-26T21:58:48.663 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31092 balancer ceph rtd unknown /proc/28367/task/31092/root (readlink: Permission denied) 2024-04-26T21:58:48.663 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31092 balancer ceph txt unknown /proc/28367/task/31092/exe (readlink: Permission denied) 2024-04-26T21:58:48.663 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31092 balancer ceph NOFD /proc/28367/task/31092/fd (opendir: Permission denied) 2024-04-26T21:58:48.663 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31093 m-fin-bal ceph cwd unknown /proc/28367/task/31093/cwd (readlink: Permission denied) 2024-04-26T21:58:48.663 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31093 m-fin-bal ceph rtd unknown /proc/28367/task/31093/root (readlink: Permission denied) 2024-04-26T21:58:48.663 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31093 m-fin-bal ceph txt unknown /proc/28367/task/31093/exe (readlink: Permission denied) 2024-04-26T21:58:48.663 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31093 m-fin-bal ceph NOFD /proc/28367/task/31093/fd (opendir: Permission denied) 2024-04-26T21:58:48.663 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31094 mgr-fin ceph cwd unknown /proc/28367/task/31094/cwd (readlink: Permission denied) 2024-04-26T21:58:48.663 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31094 mgr-fin ceph rtd unknown /proc/28367/task/31094/root (readlink: Permission denied) 2024-04-26T21:58:48.663 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31094 mgr-fin ceph txt unknown /proc/28367/task/31094/exe (readlink: Permission denied) 2024-04-26T21:58:48.663 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31094 mgr-fin ceph NOFD /proc/28367/task/31094/fd (opendir: Permission denied) 2024-04-26T21:58:48.664 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31095 mgr-fin ceph cwd unknown /proc/28367/task/31095/cwd (readlink: Permission denied) 2024-04-26T21:58:48.664 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31095 mgr-fin ceph rtd unknown /proc/28367/task/31095/root (readlink: Permission denied) 2024-04-26T21:58:48.664 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31095 mgr-fin ceph txt unknown /proc/28367/task/31095/exe (readlink: Permission denied) 2024-04-26T21:58:48.664 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31095 mgr-fin ceph NOFD /proc/28367/task/31095/fd (opendir: Permission denied) 2024-04-26T21:58:48.664 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31096 mgr-fin ceph cwd unknown /proc/28367/task/31096/cwd (readlink: Permission denied) 2024-04-26T21:58:48.664 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31096 mgr-fin ceph rtd unknown /proc/28367/task/31096/root (readlink: Permission denied) 2024-04-26T21:58:48.664 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31096 mgr-fin ceph txt unknown /proc/28367/task/31096/exe (readlink: Permission denied) 2024-04-26T21:58:48.664 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31096 mgr-fin ceph NOFD /proc/28367/task/31096/fd (opendir: Permission denied) 2024-04-26T21:58:48.664 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31097 mgr-fin ceph cwd unknown /proc/28367/task/31097/cwd (readlink: Permission denied) 2024-04-26T21:58:48.664 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31097 mgr-fin ceph rtd unknown /proc/28367/task/31097/root (readlink: Permission denied) 2024-04-26T21:58:48.664 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31097 mgr-fin ceph txt unknown /proc/28367/task/31097/exe (readlink: Permission denied) 2024-04-26T21:58:48.664 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31097 mgr-fin ceph NOFD /proc/28367/task/31097/fd (opendir: Permission denied) 2024-04-26T21:58:48.664 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31098 mgr-fin ceph cwd unknown /proc/28367/task/31098/cwd (readlink: Permission denied) 2024-04-26T21:58:48.664 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31098 mgr-fin ceph rtd unknown /proc/28367/task/31098/root (readlink: Permission denied) 2024-04-26T21:58:48.664 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31098 mgr-fin ceph txt unknown /proc/28367/task/31098/exe (readlink: Permission denied) 2024-04-26T21:58:48.665 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31098 mgr-fin ceph NOFD /proc/28367/task/31098/fd (opendir: Permission denied) 2024-04-26T21:58:48.665 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31099 mgr-fin ceph cwd unknown /proc/28367/task/31099/cwd (readlink: Permission denied) 2024-04-26T21:58:48.665 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31099 mgr-fin ceph rtd unknown /proc/28367/task/31099/root (readlink: Permission denied) 2024-04-26T21:58:48.665 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31099 mgr-fin ceph txt unknown /proc/28367/task/31099/exe (readlink: Permission denied) 2024-04-26T21:58:48.665 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31099 mgr-fin ceph NOFD /proc/28367/task/31099/fd (opendir: Permission denied) 2024-04-26T21:58:48.665 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31100 mgr-fin ceph cwd unknown /proc/28367/task/31100/cwd (readlink: Permission denied) 2024-04-26T21:58:48.665 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31100 mgr-fin ceph rtd unknown /proc/28367/task/31100/root (readlink: Permission denied) 2024-04-26T21:58:48.665 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31100 mgr-fin ceph txt unknown /proc/28367/task/31100/exe (readlink: Permission denied) 2024-04-26T21:58:48.665 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31100 mgr-fin ceph NOFD /proc/28367/task/31100/fd (opendir: Permission denied) 2024-04-26T21:58:48.665 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31101 mgr-fin ceph cwd unknown /proc/28367/task/31101/cwd (readlink: Permission denied) 2024-04-26T21:58:48.665 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31101 mgr-fin ceph rtd unknown /proc/28367/task/31101/root (readlink: Permission denied) 2024-04-26T21:58:48.666 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31101 mgr-fin ceph txt unknown /proc/28367/task/31101/exe (readlink: Permission denied) 2024-04-26T21:58:48.666 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31101 mgr-fin ceph NOFD /proc/28367/task/31101/fd (opendir: Permission denied) 2024-04-26T21:58:48.666 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31102 mgr-fin ceph cwd unknown /proc/28367/task/31102/cwd (readlink: Permission denied) 2024-04-26T21:58:48.666 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31102 mgr-fin ceph rtd unknown /proc/28367/task/31102/root (readlink: Permission denied) 2024-04-26T21:58:48.666 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31102 mgr-fin ceph txt unknown /proc/28367/task/31102/exe (readlink: Permission denied) 2024-04-26T21:58:48.666 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31102 mgr-fin ceph NOFD /proc/28367/task/31102/fd (opendir: Permission denied) 2024-04-26T21:58:48.666 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31103 mgr-fin ceph cwd unknown /proc/28367/task/31103/cwd (readlink: Permission denied) 2024-04-26T21:58:48.666 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31103 mgr-fin ceph rtd unknown /proc/28367/task/31103/root (readlink: Permission denied) 2024-04-26T21:58:48.666 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31103 mgr-fin ceph txt unknown /proc/28367/task/31103/exe (readlink: Permission denied) 2024-04-26T21:58:48.666 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31103 mgr-fin ceph NOFD /proc/28367/task/31103/fd (opendir: Permission denied) 2024-04-26T21:58:48.666 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31104 mgr-fin ceph cwd unknown /proc/28367/task/31104/cwd (readlink: Permission denied) 2024-04-26T21:58:48.666 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31104 mgr-fin ceph rtd unknown /proc/28367/task/31104/root (readlink: Permission denied) 2024-04-26T21:58:48.666 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31104 mgr-fin ceph txt unknown /proc/28367/task/31104/exe (readlink: Permission denied) 2024-04-26T21:58:48.667 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31104 mgr-fin ceph NOFD /proc/28367/task/31104/fd (opendir: Permission denied) 2024-04-26T21:58:48.667 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31105 mgr-fin ceph cwd unknown /proc/28367/task/31105/cwd (readlink: Permission denied) 2024-04-26T21:58:48.667 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31105 mgr-fin ceph rtd unknown /proc/28367/task/31105/root (readlink: Permission denied) 2024-04-26T21:58:48.667 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31105 mgr-fin ceph txt unknown /proc/28367/task/31105/exe (readlink: Permission denied) 2024-04-26T21:58:48.667 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31105 mgr-fin ceph NOFD /proc/28367/task/31105/fd (opendir: Permission denied) 2024-04-26T21:58:48.667 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31106 mgr-fin ceph cwd unknown /proc/28367/task/31106/cwd (readlink: Permission denied) 2024-04-26T21:58:48.667 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31106 mgr-fin ceph rtd unknown /proc/28367/task/31106/root (readlink: Permission denied) 2024-04-26T21:58:48.667 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31106 mgr-fin ceph txt unknown /proc/28367/task/31106/exe (readlink: Permission denied) 2024-04-26T21:58:48.667 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31106 mgr-fin ceph NOFD /proc/28367/task/31106/fd (opendir: Permission denied) 2024-04-26T21:58:48.667 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31107 mgr-fin ceph cwd unknown /proc/28367/task/31107/cwd (readlink: Permission denied) 2024-04-26T21:58:48.667 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31107 mgr-fin ceph rtd unknown /proc/28367/task/31107/root (readlink: Permission denied) 2024-04-26T21:58:48.667 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31107 mgr-fin ceph txt unknown /proc/28367/task/31107/exe (readlink: Permission denied) 2024-04-26T21:58:48.667 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31107 mgr-fin ceph NOFD /proc/28367/task/31107/fd (opendir: Permission denied) 2024-04-26T21:58:48.667 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31108 mgr-fin ceph cwd unknown /proc/28367/task/31108/cwd (readlink: Permission denied) 2024-04-26T21:58:48.668 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31108 mgr-fin ceph rtd unknown /proc/28367/task/31108/root (readlink: Permission denied) 2024-04-26T21:58:48.668 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31108 mgr-fin ceph txt unknown /proc/28367/task/31108/exe (readlink: Permission denied) 2024-04-26T21:58:48.668 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31108 mgr-fin ceph NOFD /proc/28367/task/31108/fd (opendir: Permission denied) 2024-04-26T21:58:48.668 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31109 mgr-fin ceph cwd unknown /proc/28367/task/31109/cwd (readlink: Permission denied) 2024-04-26T21:58:48.668 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31109 mgr-fin ceph rtd unknown /proc/28367/task/31109/root (readlink: Permission denied) 2024-04-26T21:58:48.668 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31109 mgr-fin ceph txt unknown /proc/28367/task/31109/exe (readlink: Permission denied) 2024-04-26T21:58:48.668 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31109 mgr-fin ceph NOFD /proc/28367/task/31109/fd (opendir: Permission denied) 2024-04-26T21:58:48.668 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31110 cephadm ceph cwd unknown /proc/28367/task/31110/cwd (readlink: Permission denied) 2024-04-26T21:58:48.668 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31110 cephadm ceph rtd unknown /proc/28367/task/31110/root (readlink: Permission denied) 2024-04-26T21:58:48.668 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31110 cephadm ceph txt unknown /proc/28367/task/31110/exe (readlink: Permission denied) 2024-04-26T21:58:48.668 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31110 cephadm ceph NOFD /proc/28367/task/31110/fd (opendir: Permission denied) 2024-04-26T21:58:48.668 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31111 mgr-fin ceph cwd unknown /proc/28367/task/31111/cwd (readlink: Permission denied) 2024-04-26T21:58:48.668 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31111 mgr-fin ceph rtd unknown /proc/28367/task/31111/root (readlink: Permission denied) 2024-04-26T21:58:48.668 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31111 mgr-fin ceph txt unknown /proc/28367/task/31111/exe (readlink: Permission denied) 2024-04-26T21:58:48.669 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31111 mgr-fin ceph NOFD /proc/28367/task/31111/fd (opendir: Permission denied) 2024-04-26T21:58:48.669 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31112 m-fin-cep ceph cwd unknown /proc/28367/task/31112/cwd (readlink: Permission denied) 2024-04-26T21:58:48.669 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31112 m-fin-cep ceph rtd unknown /proc/28367/task/31112/root (readlink: Permission denied) 2024-04-26T21:58:48.669 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31112 m-fin-cep ceph txt unknown /proc/28367/task/31112/exe (readlink: Permission denied) 2024-04-26T21:58:48.669 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31112 m-fin-cep ceph NOFD /proc/28367/task/31112/fd (opendir: Permission denied) 2024-04-26T21:58:48.669 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31113 crash ceph cwd unknown /proc/28367/task/31113/cwd (readlink: Permission denied) 2024-04-26T21:58:48.669 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31113 crash ceph rtd unknown /proc/28367/task/31113/root (readlink: Permission denied) 2024-04-26T21:58:48.669 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31113 crash ceph txt unknown /proc/28367/task/31113/exe (readlink: Permission denied) 2024-04-26T21:58:48.669 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31113 crash ceph NOFD /proc/28367/task/31113/fd (opendir: Permission denied) 2024-04-26T21:58:48.669 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31114 m-fin-cra ceph cwd unknown /proc/28367/task/31114/cwd (readlink: Permission denied) 2024-04-26T21:58:48.669 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31114 m-fin-cra ceph rtd unknown /proc/28367/task/31114/root (readlink: Permission denied) 2024-04-26T21:58:48.669 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31114 m-fin-cra ceph txt unknown /proc/28367/task/31114/exe (readlink: Permission denied) 2024-04-26T21:58:48.669 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31114 m-fin-cra ceph NOFD /proc/28367/task/31114/fd (opendir: Permission denied) 2024-04-26T21:58:48.669 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31115 mgr-fin ceph cwd unknown /proc/28367/task/31115/cwd (readlink: Permission denied) 2024-04-26T21:58:48.670 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31115 mgr-fin ceph rtd unknown /proc/28367/task/31115/root (readlink: Permission denied) 2024-04-26T21:58:48.670 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31115 mgr-fin ceph txt unknown /proc/28367/task/31115/exe (readlink: Permission denied) 2024-04-26T21:58:48.670 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31115 mgr-fin ceph NOFD /proc/28367/task/31115/fd (opendir: Permission denied) 2024-04-26T21:58:48.670 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31116 cephadm ceph cwd unknown /proc/28367/task/31116/cwd (readlink: Permission denied) 2024-04-26T21:58:48.670 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31116 cephadm ceph rtd unknown /proc/28367/task/31116/root (readlink: Permission denied) 2024-04-26T21:58:48.670 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31116 cephadm ceph txt unknown /proc/28367/task/31116/exe (readlink: Permission denied) 2024-04-26T21:58:48.670 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31116 cephadm ceph NOFD /proc/28367/task/31116/fd (opendir: Permission denied) 2024-04-26T21:58:48.670 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31117 mgr-fin ceph cwd unknown /proc/28367/task/31117/cwd (readlink: Permission denied) 2024-04-26T21:58:48.670 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31117 mgr-fin ceph rtd unknown /proc/28367/task/31117/root (readlink: Permission denied) 2024-04-26T21:58:48.670 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31117 mgr-fin ceph txt unknown /proc/28367/task/31117/exe (readlink: Permission denied) 2024-04-26T21:58:48.670 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31117 mgr-fin ceph NOFD /proc/28367/task/31117/fd (opendir: Permission denied) 2024-04-26T21:58:48.670 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31118 mgr-fin ceph cwd unknown /proc/28367/task/31118/cwd (readlink: Permission denied) 2024-04-26T21:58:48.670 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31118 mgr-fin ceph rtd unknown /proc/28367/task/31118/root (readlink: Permission denied) 2024-04-26T21:58:48.670 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31118 mgr-fin ceph txt unknown /proc/28367/task/31118/exe (readlink: Permission denied) 2024-04-26T21:58:48.671 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31118 mgr-fin ceph NOFD /proc/28367/task/31118/fd (opendir: Permission denied) 2024-04-26T21:58:48.671 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31119 mgr-fin ceph cwd unknown /proc/28367/task/31119/cwd (readlink: Permission denied) 2024-04-26T21:58:48.671 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31119 mgr-fin ceph rtd unknown /proc/28367/task/31119/root (readlink: Permission denied) 2024-04-26T21:58:48.671 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31119 mgr-fin ceph txt unknown /proc/28367/task/31119/exe (readlink: Permission denied) 2024-04-26T21:58:48.671 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31119 mgr-fin ceph NOFD /proc/28367/task/31119/fd (opendir: Permission denied) 2024-04-26T21:58:48.671 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31120 dashboard ceph cwd unknown /proc/28367/task/31120/cwd (readlink: Permission denied) 2024-04-26T21:58:48.671 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31120 dashboard ceph rtd unknown /proc/28367/task/31120/root (readlink: Permission denied) 2024-04-26T21:58:48.671 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31120 dashboard ceph txt unknown /proc/28367/task/31120/exe (readlink: Permission denied) 2024-04-26T21:58:48.671 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31120 dashboard ceph NOFD /proc/28367/task/31120/fd (opendir: Permission denied) 2024-04-26T21:58:48.671 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31121 mgr-fin ceph cwd unknown /proc/28367/task/31121/cwd (readlink: Permission denied) 2024-04-26T21:58:48.671 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31121 mgr-fin ceph rtd unknown /proc/28367/task/31121/root (readlink: Permission denied) 2024-04-26T21:58:48.671 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31121 mgr-fin ceph txt unknown /proc/28367/task/31121/exe (readlink: Permission denied) 2024-04-26T21:58:48.671 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31121 mgr-fin ceph NOFD /proc/28367/task/31121/fd (opendir: Permission denied) 2024-04-26T21:58:48.672 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31122 m-fin-das ceph cwd unknown /proc/28367/task/31122/cwd (readlink: Permission denied) 2024-04-26T21:58:48.672 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31122 m-fin-das ceph rtd unknown /proc/28367/task/31122/root (readlink: Permission denied) 2024-04-26T21:58:48.672 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31122 m-fin-das ceph txt unknown /proc/28367/task/31122/exe (readlink: Permission denied) 2024-04-26T21:58:48.672 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31122 m-fin-das ceph NOFD /proc/28367/task/31122/fd (opendir: Permission denied) 2024-04-26T21:58:48.672 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31123 mgr-fin ceph cwd unknown /proc/28367/task/31123/cwd (readlink: Permission denied) 2024-04-26T21:58:48.672 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31123 mgr-fin ceph rtd unknown /proc/28367/task/31123/root (readlink: Permission denied) 2024-04-26T21:58:48.672 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31123 mgr-fin ceph txt unknown /proc/28367/task/31123/exe (readlink: Permission denied) 2024-04-26T21:58:48.672 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31123 mgr-fin ceph NOFD /proc/28367/task/31123/fd (opendir: Permission denied) 2024-04-26T21:58:48.672 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31124 devicehea ceph cwd unknown /proc/28367/task/31124/cwd (readlink: Permission denied) 2024-04-26T21:58:48.672 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31124 devicehea ceph rtd unknown /proc/28367/task/31124/root (readlink: Permission denied) 2024-04-26T21:58:48.672 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31124 devicehea ceph txt unknown /proc/28367/task/31124/exe (readlink: Permission denied) 2024-04-26T21:58:48.672 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31124 devicehea ceph NOFD /proc/28367/task/31124/fd (opendir: Permission denied) 2024-04-26T21:58:48.672 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31125 mgr-fin ceph cwd unknown /proc/28367/task/31125/cwd (readlink: Permission denied) 2024-04-26T21:58:48.672 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31125 mgr-fin ceph rtd unknown /proc/28367/task/31125/root (readlink: Permission denied) 2024-04-26T21:58:48.672 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31125 mgr-fin ceph txt unknown /proc/28367/task/31125/exe (readlink: Permission denied) 2024-04-26T21:58:48.673 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31125 mgr-fin ceph NOFD /proc/28367/task/31125/fd (opendir: Permission denied) 2024-04-26T21:58:48.673 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31126 m-fin-dev ceph cwd unknown /proc/28367/task/31126/cwd (readlink: Permission denied) 2024-04-26T21:58:48.673 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31126 m-fin-dev ceph rtd unknown /proc/28367/task/31126/root (readlink: Permission denied) 2024-04-26T21:58:48.673 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31126 m-fin-dev ceph txt unknown /proc/28367/task/31126/exe (readlink: Permission denied) 2024-04-26T21:58:48.673 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31126 m-fin-dev ceph NOFD /proc/28367/task/31126/fd (opendir: Permission denied) 2024-04-26T21:58:48.673 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31128 m-fin-ios ceph cwd unknown /proc/28367/task/31128/cwd (readlink: Permission denied) 2024-04-26T21:58:48.673 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31128 m-fin-ios ceph rtd unknown /proc/28367/task/31128/root (readlink: Permission denied) 2024-04-26T21:58:48.673 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31128 m-fin-ios ceph txt unknown /proc/28367/task/31128/exe (readlink: Permission denied) 2024-04-26T21:58:48.673 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31128 m-fin-ios ceph NOFD /proc/28367/task/31128/fd (opendir: Permission denied) 2024-04-26T21:58:48.673 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31129 mgr-fin ceph cwd unknown /proc/28367/task/31129/cwd (readlink: Permission denied) 2024-04-26T21:58:48.673 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31129 mgr-fin ceph rtd unknown /proc/28367/task/31129/root (readlink: Permission denied) 2024-04-26T21:58:48.673 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31129 mgr-fin ceph txt unknown /proc/28367/task/31129/exe (readlink: Permission denied) 2024-04-26T21:58:48.673 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31129 mgr-fin ceph NOFD /proc/28367/task/31129/fd (opendir: Permission denied) 2024-04-26T21:58:48.673 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31130 mgr-fin ceph cwd unknown /proc/28367/task/31130/cwd (readlink: Permission denied) 2024-04-26T21:58:48.674 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31130 mgr-fin ceph rtd unknown /proc/28367/task/31130/root (readlink: Permission denied) 2024-04-26T21:58:48.674 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31130 mgr-fin ceph txt unknown /proc/28367/task/31130/exe (readlink: Permission denied) 2024-04-26T21:58:48.674 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31130 mgr-fin ceph NOFD /proc/28367/task/31130/fd (opendir: Permission denied) 2024-04-26T21:58:48.674 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31131 mgr-fin ceph cwd unknown /proc/28367/task/31131/cwd (readlink: Permission denied) 2024-04-26T21:58:48.674 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31131 mgr-fin ceph rtd unknown /proc/28367/task/31131/root (readlink: Permission denied) 2024-04-26T21:58:48.674 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31131 mgr-fin ceph txt unknown /proc/28367/task/31131/exe (readlink: Permission denied) 2024-04-26T21:58:48.674 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31131 mgr-fin ceph NOFD /proc/28367/task/31131/fd (opendir: Permission denied) 2024-04-26T21:58:48.674 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31133 mgr-fin ceph cwd unknown /proc/28367/task/31133/cwd (readlink: Permission denied) 2024-04-26T21:58:48.674 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31133 mgr-fin ceph rtd unknown /proc/28367/task/31133/root (readlink: Permission denied) 2024-04-26T21:58:48.674 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31133 mgr-fin ceph txt unknown /proc/28367/task/31133/exe (readlink: Permission denied) 2024-04-26T21:58:48.674 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31133 mgr-fin ceph NOFD /proc/28367/task/31133/fd (opendir: Permission denied) 2024-04-26T21:58:48.674 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31134 m-fin-nfs ceph cwd unknown /proc/28367/task/31134/cwd (readlink: Permission denied) 2024-04-26T21:58:48.674 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31134 m-fin-nfs ceph rtd unknown /proc/28367/task/31134/root (readlink: Permission denied) 2024-04-26T21:58:48.675 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31134 m-fin-nfs ceph txt unknown /proc/28367/task/31134/exe (readlink: Permission denied) 2024-04-26T21:58:48.675 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31134 m-fin-nfs ceph NOFD /proc/28367/task/31134/fd (opendir: Permission denied) 2024-04-26T21:58:48.675 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31136 m-fin-orc ceph cwd unknown /proc/28367/task/31136/cwd (readlink: Permission denied) 2024-04-26T21:58:48.675 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31136 m-fin-orc ceph rtd unknown /proc/28367/task/31136/root (readlink: Permission denied) 2024-04-26T21:58:48.675 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31136 m-fin-orc ceph txt unknown /proc/28367/task/31136/exe (readlink: Permission denied) 2024-04-26T21:58:48.675 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31136 m-fin-orc ceph NOFD /proc/28367/task/31136/fd (opendir: Permission denied) 2024-04-26T21:58:48.675 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31137 pg_autosc ceph cwd unknown /proc/28367/task/31137/cwd (readlink: Permission denied) 2024-04-26T21:58:48.675 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31137 pg_autosc ceph rtd unknown /proc/28367/task/31137/root (readlink: Permission denied) 2024-04-26T21:58:48.675 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31137 pg_autosc ceph txt unknown /proc/28367/task/31137/exe (readlink: Permission denied) 2024-04-26T21:58:48.675 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31137 pg_autosc ceph NOFD /proc/28367/task/31137/fd (opendir: Permission denied) 2024-04-26T21:58:48.675 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31138 m-fin-pg_ ceph cwd unknown /proc/28367/task/31138/cwd (readlink: Permission denied) 2024-04-26T21:58:48.675 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31138 m-fin-pg_ ceph rtd unknown /proc/28367/task/31138/root (readlink: Permission denied) 2024-04-26T21:58:48.675 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31138 m-fin-pg_ ceph txt unknown /proc/28367/task/31138/exe (readlink: Permission denied) 2024-04-26T21:58:48.675 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31138 m-fin-pg_ ceph NOFD /proc/28367/task/31138/fd (opendir: Permission denied) 2024-04-26T21:58:48.675 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31139 progress ceph cwd unknown /proc/28367/task/31139/cwd (readlink: Permission denied) 2024-04-26T21:58:48.676 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31139 progress ceph rtd unknown /proc/28367/task/31139/root (readlink: Permission denied) 2024-04-26T21:58:48.676 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31139 progress ceph txt unknown /proc/28367/task/31139/exe (readlink: Permission denied) 2024-04-26T21:58:48.676 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31139 progress ceph NOFD /proc/28367/task/31139/fd (opendir: Permission denied) 2024-04-26T21:58:48.676 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31140 m-fin-pro ceph cwd unknown /proc/28367/task/31140/cwd (readlink: Permission denied) 2024-04-26T21:58:48.676 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31140 m-fin-pro ceph rtd unknown /proc/28367/task/31140/root (readlink: Permission denied) 2024-04-26T21:58:48.676 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31140 m-fin-pro ceph txt unknown /proc/28367/task/31140/exe (readlink: Permission denied) 2024-04-26T21:58:48.676 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31140 m-fin-pro ceph NOFD /proc/28367/task/31140/fd (opendir: Permission denied) 2024-04-26T21:58:48.676 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31144 io_contex ceph cwd unknown /proc/28367/task/31144/cwd (readlink: Permission denied) 2024-04-26T21:58:48.676 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31144 io_contex ceph rtd unknown /proc/28367/task/31144/root (readlink: Permission denied) 2024-04-26T21:58:48.676 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31144 io_contex ceph txt unknown /proc/28367/task/31144/exe (readlink: Permission denied) 2024-04-26T21:58:48.676 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31144 io_contex ceph NOFD /proc/28367/task/31144/fd (opendir: Permission denied) 2024-04-26T21:58:48.676 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31145 io_contex ceph cwd unknown /proc/28367/task/31145/cwd (readlink: Permission denied) 2024-04-26T21:58:48.676 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31145 io_contex ceph rtd unknown /proc/28367/task/31145/root (readlink: Permission denied) 2024-04-26T21:58:48.676 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31145 io_contex ceph txt unknown /proc/28367/task/31145/exe (readlink: Permission denied) 2024-04-26T21:58:48.676 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31145 io_contex ceph NOFD /proc/28367/task/31145/fd (opendir: Permission denied) 2024-04-26T21:58:48.677 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31146 ceph_time ceph cwd unknown /proc/28367/task/31146/cwd (readlink: Permission denied) 2024-04-26T21:58:48.677 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31146 ceph_time ceph rtd unknown /proc/28367/task/31146/root (readlink: Permission denied) 2024-04-26T21:58:48.677 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31146 ceph_time ceph txt unknown /proc/28367/task/31146/exe (readlink: Permission denied) 2024-04-26T21:58:48.677 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31146 ceph_time ceph NOFD /proc/28367/task/31146/fd (opendir: Permission denied) 2024-04-26T21:58:48.677 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31147 ms_dispat ceph cwd unknown /proc/28367/task/31147/cwd (readlink: Permission denied) 2024-04-26T21:58:48.677 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31147 ms_dispat ceph rtd unknown /proc/28367/task/31147/root (readlink: Permission denied) 2024-04-26T21:58:48.677 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31147 ms_dispat ceph txt unknown /proc/28367/task/31147/exe (readlink: Permission denied) 2024-04-26T21:58:48.677 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31147 ms_dispat ceph NOFD /proc/28367/task/31147/fd (opendir: Permission denied) 2024-04-26T21:58:48.677 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31148 ms_local ceph cwd unknown /proc/28367/task/31148/cwd (readlink: Permission denied) 2024-04-26T21:58:48.677 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31148 ms_local ceph rtd unknown /proc/28367/task/31148/root (readlink: Permission denied) 2024-04-26T21:58:48.677 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31148 ms_local ceph txt unknown /proc/28367/task/31148/exe (readlink: Permission denied) 2024-04-26T21:58:48.677 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31148 ms_local ceph NOFD /proc/28367/task/31148/fd (opendir: Permission denied) 2024-04-26T21:58:48.677 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31149 safe_time ceph cwd unknown /proc/28367/task/31149/cwd (readlink: Permission denied) 2024-04-26T21:58:48.678 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31149 safe_time ceph rtd unknown /proc/28367/task/31149/root (readlink: Permission denied) 2024-04-26T21:58:48.678 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31149 safe_time ceph txt unknown /proc/28367/task/31149/exe (readlink: Permission denied) 2024-04-26T21:58:48.678 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31149 safe_time ceph NOFD /proc/28367/task/31149/fd (opendir: Permission denied) 2024-04-26T21:58:48.678 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31150 safe_time ceph cwd unknown /proc/28367/task/31150/cwd (readlink: Permission denied) 2024-04-26T21:58:48.678 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31150 safe_time ceph rtd unknown /proc/28367/task/31150/root (readlink: Permission denied) 2024-04-26T21:58:48.678 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31150 safe_time ceph txt unknown /proc/28367/task/31150/exe (readlink: Permission denied) 2024-04-26T21:58:48.678 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31150 safe_time ceph NOFD /proc/28367/task/31150/fd (opendir: Permission denied) 2024-04-26T21:58:48.678 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31151 mgr-fin ceph cwd unknown /proc/28367/task/31151/cwd (readlink: Permission denied) 2024-04-26T21:58:48.678 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31151 mgr-fin ceph rtd unknown /proc/28367/task/31151/root (readlink: Permission denied) 2024-04-26T21:58:48.678 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31151 mgr-fin ceph txt unknown /proc/28367/task/31151/exe (readlink: Permission denied) 2024-04-26T21:58:48.678 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31151 mgr-fin ceph NOFD /proc/28367/task/31151/fd (opendir: Permission denied) 2024-04-26T21:58:48.678 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31153 m-fin-rbd ceph cwd unknown /proc/28367/task/31153/cwd (readlink: Permission denied) 2024-04-26T21:58:48.678 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31153 m-fin-rbd ceph rtd unknown /proc/28367/task/31153/root (readlink: Permission denied) 2024-04-26T21:58:48.678 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31153 m-fin-rbd ceph txt unknown /proc/28367/task/31153/exe (readlink: Permission denied) 2024-04-26T21:58:48.679 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31153 m-fin-rbd ceph NOFD /proc/28367/task/31153/fd (opendir: Permission denied) 2024-04-26T21:58:48.679 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31154 restful ceph cwd unknown /proc/28367/task/31154/cwd (readlink: Permission denied) 2024-04-26T21:58:48.679 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31154 restful ceph rtd unknown /proc/28367/task/31154/root (readlink: Permission denied) 2024-04-26T21:58:48.679 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31154 restful ceph txt unknown /proc/28367/task/31154/exe (readlink: Permission denied) 2024-04-26T21:58:48.679 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31154 restful ceph NOFD /proc/28367/task/31154/fd (opendir: Permission denied) 2024-04-26T21:58:48.679 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31155 m-fin-res ceph cwd unknown /proc/28367/task/31155/cwd (readlink: Permission denied) 2024-04-26T21:58:48.679 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31155 m-fin-res ceph rtd unknown /proc/28367/task/31155/root (readlink: Permission denied) 2024-04-26T21:58:48.679 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31155 m-fin-res ceph txt unknown /proc/28367/task/31155/exe (readlink: Permission denied) 2024-04-26T21:58:48.679 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31155 m-fin-res ceph NOFD /proc/28367/task/31155/fd (opendir: Permission denied) 2024-04-26T21:58:48.679 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31157 m-fin-sta ceph cwd unknown /proc/28367/task/31157/cwd (readlink: Permission denied) 2024-04-26T21:58:48.679 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31157 m-fin-sta ceph rtd unknown /proc/28367/task/31157/root (readlink: Permission denied) 2024-04-26T21:58:48.679 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31157 m-fin-sta ceph txt unknown /proc/28367/task/31157/exe (readlink: Permission denied) 2024-04-26T21:58:48.679 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31157 m-fin-sta ceph NOFD /proc/28367/task/31157/fd (opendir: Permission denied) 2024-04-26T21:58:48.679 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31158 mgr-fin ceph cwd unknown /proc/28367/task/31158/cwd (readlink: Permission denied) 2024-04-26T21:58:48.680 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31158 mgr-fin ceph rtd unknown /proc/28367/task/31158/root (readlink: Permission denied) 2024-04-26T21:58:48.680 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31158 mgr-fin ceph txt unknown /proc/28367/task/31158/exe (readlink: Permission denied) 2024-04-26T21:58:48.680 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31158 mgr-fin ceph NOFD /proc/28367/task/31158/fd (opendir: Permission denied) 2024-04-26T21:58:48.680 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31159 telemetry ceph cwd unknown /proc/28367/task/31159/cwd (readlink: Permission denied) 2024-04-26T21:58:48.680 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31159 telemetry ceph rtd unknown /proc/28367/task/31159/root (readlink: Permission denied) 2024-04-26T21:58:48.680 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31159 telemetry ceph txt unknown /proc/28367/task/31159/exe (readlink: Permission denied) 2024-04-26T21:58:48.680 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31159 telemetry ceph NOFD /proc/28367/task/31159/fd (opendir: Permission denied) 2024-04-26T21:58:48.680 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31160 mgr-fin ceph cwd unknown /proc/28367/task/31160/cwd (readlink: Permission denied) 2024-04-26T21:58:48.680 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31160 mgr-fin ceph rtd unknown /proc/28367/task/31160/root (readlink: Permission denied) 2024-04-26T21:58:48.680 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31160 mgr-fin ceph txt unknown /proc/28367/task/31160/exe (readlink: Permission denied) 2024-04-26T21:58:48.680 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31160 mgr-fin ceph NOFD /proc/28367/task/31160/fd (opendir: Permission denied) 2024-04-26T21:58:48.680 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31161 m-fin-tel ceph cwd unknown /proc/28367/task/31161/cwd (readlink: Permission denied) 2024-04-26T21:58:48.680 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31161 m-fin-tel ceph rtd unknown /proc/28367/task/31161/root (readlink: Permission denied) 2024-04-26T21:58:48.680 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31161 m-fin-tel ceph txt unknown /proc/28367/task/31161/exe (readlink: Permission denied) 2024-04-26T21:58:48.681 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31161 m-fin-tel ceph NOFD /proc/28367/task/31161/fd (opendir: Permission denied) 2024-04-26T21:58:48.681 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31162 mgr-fin ceph cwd unknown /proc/28367/task/31162/cwd (readlink: Permission denied) 2024-04-26T21:58:48.681 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31162 mgr-fin ceph rtd unknown /proc/28367/task/31162/root (readlink: Permission denied) 2024-04-26T21:58:48.681 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31162 mgr-fin ceph txt unknown /proc/28367/task/31162/exe (readlink: Permission denied) 2024-04-26T21:58:48.681 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31162 mgr-fin ceph NOFD /proc/28367/task/31162/fd (opendir: Permission denied) 2024-04-26T21:58:48.681 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31163 mgr-fin ceph cwd unknown /proc/28367/task/31163/cwd (readlink: Permission denied) 2024-04-26T21:58:48.681 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31163 mgr-fin ceph rtd unknown /proc/28367/task/31163/root (readlink: Permission denied) 2024-04-26T21:58:48.681 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31163 mgr-fin ceph txt unknown /proc/28367/task/31163/exe (readlink: Permission denied) 2024-04-26T21:58:48.681 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31163 mgr-fin ceph NOFD /proc/28367/task/31163/fd (opendir: Permission denied) 2024-04-26T21:58:48.681 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31164 mgr-fin ceph cwd unknown /proc/28367/task/31164/cwd (readlink: Permission denied) 2024-04-26T21:58:48.682 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31164 mgr-fin ceph rtd unknown /proc/28367/task/31164/root (readlink: Permission denied) 2024-04-26T21:58:48.682 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31164 mgr-fin ceph txt unknown /proc/28367/task/31164/exe (readlink: Permission denied) 2024-04-26T21:58:48.682 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31164 mgr-fin ceph NOFD /proc/28367/task/31164/fd (opendir: Permission denied) 2024-04-26T21:58:48.682 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31169 io_contex ceph cwd unknown /proc/28367/task/31169/cwd (readlink: Permission denied) 2024-04-26T21:58:48.682 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31169 io_contex ceph rtd unknown /proc/28367/task/31169/root (readlink: Permission denied) 2024-04-26T21:58:48.682 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31169 io_contex ceph txt unknown /proc/28367/task/31169/exe (readlink: Permission denied) 2024-04-26T21:58:48.682 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31169 io_contex ceph NOFD /proc/28367/task/31169/fd (opendir: Permission denied) 2024-04-26T21:58:48.682 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31170 io_contex ceph cwd unknown /proc/28367/task/31170/cwd (readlink: Permission denied) 2024-04-26T21:58:48.682 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31170 io_contex ceph rtd unknown /proc/28367/task/31170/root (readlink: Permission denied) 2024-04-26T21:58:48.682 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31170 io_contex ceph txt unknown /proc/28367/task/31170/exe (readlink: Permission denied) 2024-04-26T21:58:48.682 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31170 io_contex ceph NOFD /proc/28367/task/31170/fd (opendir: Permission denied) 2024-04-26T21:58:48.682 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31171 ceph_time ceph cwd unknown /proc/28367/task/31171/cwd (readlink: Permission denied) 2024-04-26T21:58:48.682 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31171 ceph_time ceph rtd unknown /proc/28367/task/31171/root (readlink: Permission denied) 2024-04-26T21:58:48.682 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31171 ceph_time ceph txt unknown /proc/28367/task/31171/exe (readlink: Permission denied) 2024-04-26T21:58:48.683 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31171 ceph_time ceph NOFD /proc/28367/task/31171/fd (opendir: Permission denied) 2024-04-26T21:58:48.683 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31172 ms_dispat ceph cwd unknown /proc/28367/task/31172/cwd (readlink: Permission denied) 2024-04-26T21:58:48.683 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31172 ms_dispat ceph rtd unknown /proc/28367/task/31172/root (readlink: Permission denied) 2024-04-26T21:58:48.683 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31172 ms_dispat ceph txt unknown /proc/28367/task/31172/exe (readlink: Permission denied) 2024-04-26T21:58:48.683 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31172 ms_dispat ceph NOFD /proc/28367/task/31172/fd (opendir: Permission denied) 2024-04-26T21:58:48.683 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31173 ms_local ceph cwd unknown /proc/28367/task/31173/cwd (readlink: Permission denied) 2024-04-26T21:58:48.683 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31173 ms_local ceph rtd unknown /proc/28367/task/31173/root (readlink: Permission denied) 2024-04-26T21:58:48.683 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31173 ms_local ceph txt unknown /proc/28367/task/31173/exe (readlink: Permission denied) 2024-04-26T21:58:48.683 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31173 ms_local ceph NOFD /proc/28367/task/31173/fd (opendir: Permission denied) 2024-04-26T21:58:48.683 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31174 safe_time ceph cwd unknown /proc/28367/task/31174/cwd (readlink: Permission denied) 2024-04-26T21:58:48.683 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31174 safe_time ceph rtd unknown /proc/28367/task/31174/root (readlink: Permission denied) 2024-04-26T21:58:48.683 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31174 safe_time ceph txt unknown /proc/28367/task/31174/exe (readlink: Permission denied) 2024-04-26T21:58:48.684 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31174 safe_time ceph NOFD /proc/28367/task/31174/fd (opendir: Permission denied) 2024-04-26T21:58:48.684 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31175 safe_time ceph cwd unknown /proc/28367/task/31175/cwd (readlink: Permission denied) 2024-04-26T21:58:48.684 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31175 safe_time ceph rtd unknown /proc/28367/task/31175/root (readlink: Permission denied) 2024-04-26T21:58:48.684 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31175 safe_time ceph txt unknown /proc/28367/task/31175/exe (readlink: Permission denied) 2024-04-26T21:58:48.684 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31175 safe_time ceph NOFD /proc/28367/task/31175/fd (opendir: Permission denied) 2024-04-26T21:58:48.684 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31176 mgr-fin ceph cwd unknown /proc/28367/task/31176/cwd (readlink: Permission denied) 2024-04-26T21:58:48.684 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31176 mgr-fin ceph rtd unknown /proc/28367/task/31176/root (readlink: Permission denied) 2024-04-26T21:58:48.684 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31176 mgr-fin ceph txt unknown /proc/28367/task/31176/exe (readlink: Permission denied) 2024-04-26T21:58:48.684 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31176 mgr-fin ceph NOFD /proc/28367/task/31176/fd (opendir: Permission denied) 2024-04-26T21:58:48.684 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31177 mgr-fin ceph cwd unknown /proc/28367/task/31177/cwd (readlink: Permission denied) 2024-04-26T21:58:48.684 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31177 mgr-fin ceph rtd unknown /proc/28367/task/31177/root (readlink: Permission denied) 2024-04-26T21:58:48.684 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31177 mgr-fin ceph txt unknown /proc/28367/task/31177/exe (readlink: Permission denied) 2024-04-26T21:58:48.685 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31177 mgr-fin ceph NOFD /proc/28367/task/31177/fd (opendir: Permission denied) 2024-04-26T21:58:48.685 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31178 mgr-fin ceph cwd unknown /proc/28367/task/31178/cwd (readlink: Permission denied) 2024-04-26T21:58:48.685 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31178 mgr-fin ceph rtd unknown /proc/28367/task/31178/root (readlink: Permission denied) 2024-04-26T21:58:48.685 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31178 mgr-fin ceph txt unknown /proc/28367/task/31178/exe (readlink: Permission denied) 2024-04-26T21:58:48.685 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31178 mgr-fin ceph NOFD /proc/28367/task/31178/fd (opendir: Permission denied) 2024-04-26T21:58:48.685 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31179 mgr-fin ceph cwd unknown /proc/28367/task/31179/cwd (readlink: Permission denied) 2024-04-26T21:58:48.685 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31179 mgr-fin ceph rtd unknown /proc/28367/task/31179/root (readlink: Permission denied) 2024-04-26T21:58:48.685 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31179 mgr-fin ceph txt unknown /proc/28367/task/31179/exe (readlink: Permission denied) 2024-04-26T21:58:48.685 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31179 mgr-fin ceph NOFD /proc/28367/task/31179/fd (opendir: Permission denied) 2024-04-26T21:58:48.685 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31180 mgr-fin ceph cwd unknown /proc/28367/task/31180/cwd (readlink: Permission denied) 2024-04-26T21:58:48.685 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31180 mgr-fin ceph rtd unknown /proc/28367/task/31180/root (readlink: Permission denied) 2024-04-26T21:58:48.685 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31180 mgr-fin ceph txt unknown /proc/28367/task/31180/exe (readlink: Permission denied) 2024-04-26T21:58:48.686 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31180 mgr-fin ceph NOFD /proc/28367/task/31180/fd (opendir: Permission denied) 2024-04-26T21:58:48.686 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31181 mgr-fin ceph cwd unknown /proc/28367/task/31181/cwd (readlink: Permission denied) 2024-04-26T21:58:48.686 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31181 mgr-fin ceph rtd unknown /proc/28367/task/31181/root (readlink: Permission denied) 2024-04-26T21:58:48.686 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31181 mgr-fin ceph txt unknown /proc/28367/task/31181/exe (readlink: Permission denied) 2024-04-26T21:58:48.686 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31181 mgr-fin ceph NOFD /proc/28367/task/31181/fd (opendir: Permission denied) 2024-04-26T21:58:48.686 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31182 mgr-fin ceph cwd unknown /proc/28367/task/31182/cwd (readlink: Permission denied) 2024-04-26T21:58:48.686 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31182 mgr-fin ceph rtd unknown /proc/28367/task/31182/root (readlink: Permission denied) 2024-04-26T21:58:48.686 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31182 mgr-fin ceph txt unknown /proc/28367/task/31182/exe (readlink: Permission denied) 2024-04-26T21:58:48.686 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31182 mgr-fin ceph NOFD /proc/28367/task/31182/fd (opendir: Permission denied) 2024-04-26T21:58:48.686 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31183 mgr-fin ceph cwd unknown /proc/28367/task/31183/cwd (readlink: Permission denied) 2024-04-26T21:58:48.687 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31183 mgr-fin ceph rtd unknown /proc/28367/task/31183/root (readlink: Permission denied) 2024-04-26T21:58:48.687 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31183 mgr-fin ceph txt unknown /proc/28367/task/31183/exe (readlink: Permission denied) 2024-04-26T21:58:48.687 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31183 mgr-fin ceph NOFD /proc/28367/task/31183/fd (opendir: Permission denied) 2024-04-26T21:58:48.687 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31184 mgr-fin ceph cwd unknown /proc/28367/task/31184/cwd (readlink: Permission denied) 2024-04-26T21:58:48.687 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31184 mgr-fin ceph rtd unknown /proc/28367/task/31184/root (readlink: Permission denied) 2024-04-26T21:58:48.687 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31184 mgr-fin ceph txt unknown /proc/28367/task/31184/exe (readlink: Permission denied) 2024-04-26T21:58:48.687 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31184 mgr-fin ceph NOFD /proc/28367/task/31184/fd (opendir: Permission denied) 2024-04-26T21:58:48.687 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31185 mgr-fin ceph cwd unknown /proc/28367/task/31185/cwd (readlink: Permission denied) 2024-04-26T21:58:48.687 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31185 mgr-fin ceph rtd unknown /proc/28367/task/31185/root (readlink: Permission denied) 2024-04-26T21:58:48.687 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31185 mgr-fin ceph txt unknown /proc/28367/task/31185/exe (readlink: Permission denied) 2024-04-26T21:58:48.687 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31185 mgr-fin ceph NOFD /proc/28367/task/31185/fd (opendir: Permission denied) 2024-04-26T21:58:48.687 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31186 mgr-fin ceph cwd unknown /proc/28367/task/31186/cwd (readlink: Permission denied) 2024-04-26T21:58:48.688 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31186 mgr-fin ceph rtd unknown /proc/28367/task/31186/root (readlink: Permission denied) 2024-04-26T21:58:48.688 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31186 mgr-fin ceph txt unknown /proc/28367/task/31186/exe (readlink: Permission denied) 2024-04-26T21:58:48.688 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31186 mgr-fin ceph NOFD /proc/28367/task/31186/fd (opendir: Permission denied) 2024-04-26T21:58:48.688 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31187 mgr-fin ceph cwd unknown /proc/28367/task/31187/cwd (readlink: Permission denied) 2024-04-26T21:58:48.688 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31187 mgr-fin ceph rtd unknown /proc/28367/task/31187/root (readlink: Permission denied) 2024-04-26T21:58:48.688 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31187 mgr-fin ceph txt unknown /proc/28367/task/31187/exe (readlink: Permission denied) 2024-04-26T21:58:48.688 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31187 mgr-fin ceph NOFD /proc/28367/task/31187/fd (opendir: Permission denied) 2024-04-26T21:58:48.688 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31189 m-fin-vol ceph cwd unknown /proc/28367/task/31189/cwd (readlink: Permission denied) 2024-04-26T21:58:48.688 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31189 m-fin-vol ceph rtd unknown /proc/28367/task/31189/root (readlink: Permission denied) 2024-04-26T21:58:48.688 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31189 m-fin-vol ceph txt unknown /proc/28367/task/31189/exe (readlink: Permission denied) 2024-04-26T21:58:48.688 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31189 m-fin-vol ceph NOFD /proc/28367/task/31189/fd (opendir: Permission denied) 2024-04-26T21:58:48.688 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31205 mgr-fin ceph cwd unknown /proc/28367/task/31205/cwd (readlink: Permission denied) 2024-04-26T21:58:48.688 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31205 mgr-fin ceph rtd unknown /proc/28367/task/31205/root (readlink: Permission denied) 2024-04-26T21:58:48.688 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31205 mgr-fin ceph txt unknown /proc/28367/task/31205/exe (readlink: Permission denied) 2024-04-26T21:58:48.689 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31205 mgr-fin ceph NOFD /proc/28367/task/31205/fd (opendir: Permission denied) 2024-04-26T21:58:48.689 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31206 mgr-fin ceph cwd unknown /proc/28367/task/31206/cwd (readlink: Permission denied) 2024-04-26T21:58:48.689 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31206 mgr-fin ceph rtd unknown /proc/28367/task/31206/root (readlink: Permission denied) 2024-04-26T21:58:48.689 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31206 mgr-fin ceph txt unknown /proc/28367/task/31206/exe (readlink: Permission denied) 2024-04-26T21:58:48.689 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31206 mgr-fin ceph NOFD /proc/28367/task/31206/fd (opendir: Permission denied) 2024-04-26T21:58:48.689 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31208 mgr-fin ceph cwd unknown /proc/28367/task/31208/cwd (readlink: Permission denied) 2024-04-26T21:58:48.689 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31208 mgr-fin ceph rtd unknown /proc/28367/task/31208/root (readlink: Permission denied) 2024-04-26T21:58:48.689 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31208 mgr-fin ceph txt unknown /proc/28367/task/31208/exe (readlink: Permission denied) 2024-04-26T21:58:48.689 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31208 mgr-fin ceph NOFD /proc/28367/task/31208/fd (opendir: Permission denied) 2024-04-26T21:58:48.689 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31209 mgr-fin ceph cwd unknown /proc/28367/task/31209/cwd (readlink: Permission denied) 2024-04-26T21:58:48.689 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31209 mgr-fin ceph rtd unknown /proc/28367/task/31209/root (readlink: Permission denied) 2024-04-26T21:58:48.689 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31209 mgr-fin ceph txt unknown /proc/28367/task/31209/exe (readlink: Permission denied) 2024-04-26T21:58:48.689 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31209 mgr-fin ceph NOFD /proc/28367/task/31209/fd (opendir: Permission denied) 2024-04-26T21:58:48.690 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31210 mgr-fin ceph cwd unknown /proc/28367/task/31210/cwd (readlink: Permission denied) 2024-04-26T21:58:48.690 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31210 mgr-fin ceph rtd unknown /proc/28367/task/31210/root (readlink: Permission denied) 2024-04-26T21:58:48.690 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31210 mgr-fin ceph txt unknown /proc/28367/task/31210/exe (readlink: Permission denied) 2024-04-26T21:58:48.690 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31210 mgr-fin ceph NOFD /proc/28367/task/31210/fd (opendir: Permission denied) 2024-04-26T21:58:48.690 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31211 mgr-fin ceph cwd unknown /proc/28367/task/31211/cwd (readlink: Permission denied) 2024-04-26T21:58:48.690 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31211 mgr-fin ceph rtd unknown /proc/28367/task/31211/root (readlink: Permission denied) 2024-04-26T21:58:48.690 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31211 mgr-fin ceph txt unknown /proc/28367/task/31211/exe (readlink: Permission denied) 2024-04-26T21:58:48.690 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31211 mgr-fin ceph NOFD /proc/28367/task/31211/fd (opendir: Permission denied) 2024-04-26T21:58:48.690 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31213 mgr-fin ceph cwd unknown /proc/28367/task/31213/cwd (readlink: Permission denied) 2024-04-26T21:58:48.690 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31213 mgr-fin ceph rtd unknown /proc/28367/task/31213/root (readlink: Permission denied) 2024-04-26T21:58:48.690 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31213 mgr-fin ceph txt unknown /proc/28367/task/31213/exe (readlink: Permission denied) 2024-04-26T21:58:48.690 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31213 mgr-fin ceph NOFD /proc/28367/task/31213/fd (opendir: Permission denied) 2024-04-26T21:58:48.690 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31214 mgr-fin ceph cwd unknown /proc/28367/task/31214/cwd (readlink: Permission denied) 2024-04-26T21:58:48.690 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31214 mgr-fin ceph rtd unknown /proc/28367/task/31214/root (readlink: Permission denied) 2024-04-26T21:58:48.691 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31214 mgr-fin ceph txt unknown /proc/28367/task/31214/exe (readlink: Permission denied) 2024-04-26T21:58:48.691 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31214 mgr-fin ceph NOFD /proc/28367/task/31214/fd (opendir: Permission denied) 2024-04-26T21:58:48.691 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31215 mgr-fin ceph cwd unknown /proc/28367/task/31215/cwd (readlink: Permission denied) 2024-04-26T21:58:48.691 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31215 mgr-fin ceph rtd unknown /proc/28367/task/31215/root (readlink: Permission denied) 2024-04-26T21:58:48.691 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31215 mgr-fin ceph txt unknown /proc/28367/task/31215/exe (readlink: Permission denied) 2024-04-26T21:58:48.691 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31215 mgr-fin ceph NOFD /proc/28367/task/31215/fd (opendir: Permission denied) 2024-04-26T21:58:48.691 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31216 mgr-fin ceph cwd unknown /proc/28367/task/31216/cwd (readlink: Permission denied) 2024-04-26T21:58:48.691 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31216 mgr-fin ceph rtd unknown /proc/28367/task/31216/root (readlink: Permission denied) 2024-04-26T21:58:48.691 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31216 mgr-fin ceph txt unknown /proc/28367/task/31216/exe (readlink: Permission denied) 2024-04-26T21:58:48.691 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31216 mgr-fin ceph NOFD /proc/28367/task/31216/fd (opendir: Permission denied) 2024-04-26T21:58:48.691 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31217 mgr-fin ceph cwd unknown /proc/28367/task/31217/cwd (readlink: Permission denied) 2024-04-26T21:58:48.691 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31217 mgr-fin ceph rtd unknown /proc/28367/task/31217/root (readlink: Permission denied) 2024-04-26T21:58:48.691 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31217 mgr-fin ceph txt unknown /proc/28367/task/31217/exe (readlink: Permission denied) 2024-04-26T21:58:48.692 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31217 mgr-fin ceph NOFD /proc/28367/task/31217/fd (opendir: Permission denied) 2024-04-26T21:58:48.692 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31255 mgr-fin ceph cwd unknown /proc/28367/task/31255/cwd (readlink: Permission denied) 2024-04-26T21:58:48.692 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31255 mgr-fin ceph rtd unknown /proc/28367/task/31255/root (readlink: Permission denied) 2024-04-26T21:58:48.692 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31255 mgr-fin ceph txt unknown /proc/28367/task/31255/exe (readlink: Permission denied) 2024-04-26T21:58:48.692 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31255 mgr-fin ceph NOFD /proc/28367/task/31255/fd (opendir: Permission denied) 2024-04-26T21:58:48.692 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31256 mgr-fin ceph cwd unknown /proc/28367/task/31256/cwd (readlink: Permission denied) 2024-04-26T21:58:48.692 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31256 mgr-fin ceph rtd unknown /proc/28367/task/31256/root (readlink: Permission denied) 2024-04-26T21:58:48.692 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31256 mgr-fin ceph txt unknown /proc/28367/task/31256/exe (readlink: Permission denied) 2024-04-26T21:58:48.692 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31256 mgr-fin ceph NOFD /proc/28367/task/31256/fd (opendir: Permission denied) 2024-04-26T21:58:48.692 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31257 mgr-fin ceph cwd unknown /proc/28367/task/31257/cwd (readlink: Permission denied) 2024-04-26T21:58:48.692 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31257 mgr-fin ceph rtd unknown /proc/28367/task/31257/root (readlink: Permission denied) 2024-04-26T21:58:48.692 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31257 mgr-fin ceph txt unknown /proc/28367/task/31257/exe (readlink: Permission denied) 2024-04-26T21:58:48.692 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31257 mgr-fin ceph NOFD /proc/28367/task/31257/fd (opendir: Permission denied) 2024-04-26T21:58:48.692 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31258 mgr-fin ceph cwd unknown /proc/28367/task/31258/cwd (readlink: Permission denied) 2024-04-26T21:58:48.693 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31258 mgr-fin ceph rtd unknown /proc/28367/task/31258/root (readlink: Permission denied) 2024-04-26T21:58:48.693 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31258 mgr-fin ceph txt unknown /proc/28367/task/31258/exe (readlink: Permission denied) 2024-04-26T21:58:48.693 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31258 mgr-fin ceph NOFD /proc/28367/task/31258/fd (opendir: Permission denied) 2024-04-26T21:58:48.693 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31259 mgr-fin ceph cwd unknown /proc/28367/task/31259/cwd (readlink: Permission denied) 2024-04-26T21:58:48.693 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31259 mgr-fin ceph rtd unknown /proc/28367/task/31259/root (readlink: Permission denied) 2024-04-26T21:58:48.693 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31259 mgr-fin ceph txt unknown /proc/28367/task/31259/exe (readlink: Permission denied) 2024-04-26T21:58:48.693 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31259 mgr-fin ceph NOFD /proc/28367/task/31259/fd (opendir: Permission denied) 2024-04-26T21:58:48.693 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31260 mgr-fin ceph cwd unknown /proc/28367/task/31260/cwd (readlink: Permission denied) 2024-04-26T21:58:48.693 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31260 mgr-fin ceph rtd unknown /proc/28367/task/31260/root (readlink: Permission denied) 2024-04-26T21:58:48.693 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31260 mgr-fin ceph txt unknown /proc/28367/task/31260/exe (readlink: Permission denied) 2024-04-26T21:58:48.694 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31260 mgr-fin ceph NOFD /proc/28367/task/31260/fd (opendir: Permission denied) 2024-04-26T21:58:48.694 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31261 mgr-fin ceph cwd unknown /proc/28367/task/31261/cwd (readlink: Permission denied) 2024-04-26T21:58:48.694 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31261 mgr-fin ceph rtd unknown /proc/28367/task/31261/root (readlink: Permission denied) 2024-04-26T21:58:48.694 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31261 mgr-fin ceph txt unknown /proc/28367/task/31261/exe (readlink: Permission denied) 2024-04-26T21:58:48.694 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31261 mgr-fin ceph NOFD /proc/28367/task/31261/fd (opendir: Permission denied) 2024-04-26T21:58:48.694 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31262 mgr-fin ceph cwd unknown /proc/28367/task/31262/cwd (readlink: Permission denied) 2024-04-26T21:58:48.694 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31262 mgr-fin ceph rtd unknown /proc/28367/task/31262/root (readlink: Permission denied) 2024-04-26T21:58:48.694 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31262 mgr-fin ceph txt unknown /proc/28367/task/31262/exe (readlink: Permission denied) 2024-04-26T21:58:48.694 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31262 mgr-fin ceph NOFD /proc/28367/task/31262/fd (opendir: Permission denied) 2024-04-26T21:58:48.694 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31263 mgr-fin ceph cwd unknown /proc/28367/task/31263/cwd (readlink: Permission denied) 2024-04-26T21:58:48.695 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31263 mgr-fin ceph rtd unknown /proc/28367/task/31263/root (readlink: Permission denied) 2024-04-26T21:58:48.695 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31263 mgr-fin ceph txt unknown /proc/28367/task/31263/exe (readlink: Permission denied) 2024-04-26T21:58:48.695 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31263 mgr-fin ceph NOFD /proc/28367/task/31263/fd (opendir: Permission denied) 2024-04-26T21:58:48.695 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31264 mgr-fin ceph cwd unknown /proc/28367/task/31264/cwd (readlink: Permission denied) 2024-04-26T21:58:48.695 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31264 mgr-fin ceph rtd unknown /proc/28367/task/31264/root (readlink: Permission denied) 2024-04-26T21:58:48.695 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31264 mgr-fin ceph txt unknown /proc/28367/task/31264/exe (readlink: Permission denied) 2024-04-26T21:58:48.695 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31264 mgr-fin ceph NOFD /proc/28367/task/31264/fd (opendir: Permission denied) 2024-04-26T21:58:48.695 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31265 mgr-fin ceph cwd unknown /proc/28367/task/31265/cwd (readlink: Permission denied) 2024-04-26T21:58:48.695 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31265 mgr-fin ceph rtd unknown /proc/28367/task/31265/root (readlink: Permission denied) 2024-04-26T21:58:48.695 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31265 mgr-fin ceph txt unknown /proc/28367/task/31265/exe (readlink: Permission denied) 2024-04-26T21:58:48.696 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31265 mgr-fin ceph NOFD /proc/28367/task/31265/fd (opendir: Permission denied) 2024-04-26T21:58:48.696 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31766 dashboard ceph cwd unknown /proc/28367/task/31766/cwd (readlink: Permission denied) 2024-04-26T21:58:48.696 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31766 dashboard ceph rtd unknown /proc/28367/task/31766/root (readlink: Permission denied) 2024-04-26T21:58:48.696 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31766 dashboard ceph txt unknown /proc/28367/task/31766/exe (readlink: Permission denied) 2024-04-26T21:58:48.696 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31766 dashboard ceph NOFD /proc/28367/task/31766/fd (opendir: Permission denied) 2024-04-26T21:58:48.696 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31767 dashboard ceph cwd unknown /proc/28367/task/31767/cwd (readlink: Permission denied) 2024-04-26T21:58:48.696 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31767 dashboard ceph rtd unknown /proc/28367/task/31767/root (readlink: Permission denied) 2024-04-26T21:58:48.696 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31767 dashboard ceph txt unknown /proc/28367/task/31767/exe (readlink: Permission denied) 2024-04-26T21:58:48.696 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31767 dashboard ceph NOFD /proc/28367/task/31767/fd (opendir: Permission denied) 2024-04-26T21:58:48.696 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31768 dashboard ceph cwd unknown /proc/28367/task/31768/cwd (readlink: Permission denied) 2024-04-26T21:58:48.697 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31768 dashboard ceph rtd unknown /proc/28367/task/31768/root (readlink: Permission denied) 2024-04-26T21:58:48.697 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31768 dashboard ceph txt unknown /proc/28367/task/31768/exe (readlink: Permission denied) 2024-04-26T21:58:48.697 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31768 dashboard ceph NOFD /proc/28367/task/31768/fd (opendir: Permission denied) 2024-04-26T21:58:48.697 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31769 dashboard ceph cwd unknown /proc/28367/task/31769/cwd (readlink: Permission denied) 2024-04-26T21:58:48.697 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31769 dashboard ceph rtd unknown /proc/28367/task/31769/root (readlink: Permission denied) 2024-04-26T21:58:48.697 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31769 dashboard ceph txt unknown /proc/28367/task/31769/exe (readlink: Permission denied) 2024-04-26T21:58:48.697 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31769 dashboard ceph NOFD /proc/28367/task/31769/fd (opendir: Permission denied) 2024-04-26T21:58:48.697 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31770 dashboard ceph cwd unknown /proc/28367/task/31770/cwd (readlink: Permission denied) 2024-04-26T21:58:48.697 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31770 dashboard ceph rtd unknown /proc/28367/task/31770/root (readlink: Permission denied) 2024-04-26T21:58:48.697 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31770 dashboard ceph txt unknown /proc/28367/task/31770/exe (readlink: Permission denied) 2024-04-26T21:58:48.697 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31770 dashboard ceph NOFD /proc/28367/task/31770/fd (opendir: Permission denied) 2024-04-26T21:58:48.697 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31771 dashboard ceph cwd unknown /proc/28367/task/31771/cwd (readlink: Permission denied) 2024-04-26T21:58:48.697 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31771 dashboard ceph rtd unknown /proc/28367/task/31771/root (readlink: Permission denied) 2024-04-26T21:58:48.698 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31771 dashboard ceph txt unknown /proc/28367/task/31771/exe (readlink: Permission denied) 2024-04-26T21:58:48.698 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31771 dashboard ceph NOFD /proc/28367/task/31771/fd (opendir: Permission denied) 2024-04-26T21:58:48.698 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31772 dashboard ceph cwd unknown /proc/28367/task/31772/cwd (readlink: Permission denied) 2024-04-26T21:58:48.698 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31772 dashboard ceph rtd unknown /proc/28367/task/31772/root (readlink: Permission denied) 2024-04-26T21:58:48.698 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31772 dashboard ceph txt unknown /proc/28367/task/31772/exe (readlink: Permission denied) 2024-04-26T21:58:48.698 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31772 dashboard ceph NOFD /proc/28367/task/31772/fd (opendir: Permission denied) 2024-04-26T21:58:48.698 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31773 dashboard ceph cwd unknown /proc/28367/task/31773/cwd (readlink: Permission denied) 2024-04-26T21:58:48.698 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31773 dashboard ceph rtd unknown /proc/28367/task/31773/root (readlink: Permission denied) 2024-04-26T21:58:48.698 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31773 dashboard ceph txt unknown /proc/28367/task/31773/exe (readlink: Permission denied) 2024-04-26T21:58:48.698 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31773 dashboard ceph NOFD /proc/28367/task/31773/fd (opendir: Permission denied) 2024-04-26T21:58:48.698 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31774 dashboard ceph cwd unknown /proc/28367/task/31774/cwd (readlink: Permission denied) 2024-04-26T21:58:48.698 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31774 dashboard ceph rtd unknown /proc/28367/task/31774/root (readlink: Permission denied) 2024-04-26T21:58:48.698 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31774 dashboard ceph txt unknown /proc/28367/task/31774/exe (readlink: Permission denied) 2024-04-26T21:58:48.699 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31774 dashboard ceph NOFD /proc/28367/task/31774/fd (opendir: Permission denied) 2024-04-26T21:58:48.699 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31775 dashboard ceph cwd unknown /proc/28367/task/31775/cwd (readlink: Permission denied) 2024-04-26T21:58:48.699 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31775 dashboard ceph rtd unknown /proc/28367/task/31775/root (readlink: Permission denied) 2024-04-26T21:58:48.699 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31775 dashboard ceph txt unknown /proc/28367/task/31775/exe (readlink: Permission denied) 2024-04-26T21:58:48.699 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31775 dashboard ceph NOFD /proc/28367/task/31775/fd (opendir: Permission denied) 2024-04-26T21:58:48.699 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31776 dashboard ceph cwd unknown /proc/28367/task/31776/cwd (readlink: Permission denied) 2024-04-26T21:58:48.699 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31776 dashboard ceph rtd unknown /proc/28367/task/31776/root (readlink: Permission denied) 2024-04-26T21:58:48.699 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31776 dashboard ceph txt unknown /proc/28367/task/31776/exe (readlink: Permission denied) 2024-04-26T21:58:48.699 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31776 dashboard ceph NOFD /proc/28367/task/31776/fd (opendir: Permission denied) 2024-04-26T21:58:48.699 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31802 dashboard ceph cwd unknown /proc/28367/task/31802/cwd (readlink: Permission denied) 2024-04-26T21:58:48.699 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31802 dashboard ceph rtd unknown /proc/28367/task/31802/root (readlink: Permission denied) 2024-04-26T21:58:48.699 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31802 dashboard ceph txt unknown /proc/28367/task/31802/exe (readlink: Permission denied) 2024-04-26T21:58:48.699 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 31802 dashboard ceph NOFD /proc/28367/task/31802/fd (opendir: Permission denied) 2024-04-26T21:58:48.699 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 38548 cephadm ceph cwd unknown /proc/28367/task/38548/cwd (readlink: Permission denied) 2024-04-26T21:58:48.700 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 38548 cephadm ceph rtd unknown /proc/28367/task/38548/root (readlink: Permission denied) 2024-04-26T21:58:48.700 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 38548 cephadm ceph txt unknown /proc/28367/task/38548/exe (readlink: Permission denied) 2024-04-26T21:58:48.700 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 38548 cephadm ceph NOFD /proc/28367/task/38548/fd (opendir: Permission denied) 2024-04-26T21:58:48.700 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 38549 cephadm ceph cwd unknown /proc/28367/task/38549/cwd (readlink: Permission denied) 2024-04-26T21:58:48.700 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 38549 cephadm ceph rtd unknown /proc/28367/task/38549/root (readlink: Permission denied) 2024-04-26T21:58:48.700 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 38549 cephadm ceph txt unknown /proc/28367/task/38549/exe (readlink: Permission denied) 2024-04-26T21:58:48.700 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 38549 cephadm ceph NOFD /proc/28367/task/38549/fd (opendir: Permission denied) 2024-04-26T21:58:48.700 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 38550 cephadm ceph cwd unknown /proc/28367/task/38550/cwd (readlink: Permission denied) 2024-04-26T21:58:48.700 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 38550 cephadm ceph rtd unknown /proc/28367/task/38550/root (readlink: Permission denied) 2024-04-26T21:58:48.700 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 38550 cephadm ceph txt unknown /proc/28367/task/38550/exe (readlink: Permission denied) 2024-04-26T21:58:48.700 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 38550 cephadm ceph NOFD /proc/28367/task/38550/fd (opendir: Permission denied) 2024-04-26T21:58:48.700 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 38551 cephadm ceph cwd unknown /proc/28367/task/38551/cwd (readlink: Permission denied) 2024-04-26T21:58:48.700 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 38551 cephadm ceph rtd unknown /proc/28367/task/38551/root (readlink: Permission denied) 2024-04-26T21:58:48.700 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 38551 cephadm ceph txt unknown /proc/28367/task/38551/exe (readlink: Permission denied) 2024-04-26T21:58:48.700 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 38551 cephadm ceph NOFD /proc/28367/task/38551/fd (opendir: Permission denied) 2024-04-26T21:58:48.701 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 38552 cephadm ceph cwd unknown /proc/28367/task/38552/cwd (readlink: Permission denied) 2024-04-26T21:58:48.701 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 38552 cephadm ceph rtd unknown /proc/28367/task/38552/root (readlink: Permission denied) 2024-04-26T21:58:48.701 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 38552 cephadm ceph txt unknown /proc/28367/task/38552/exe (readlink: Permission denied) 2024-04-26T21:58:48.701 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 38552 cephadm ceph NOFD /proc/28367/task/38552/fd (opendir: Permission denied) 2024-04-26T21:58:48.701 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 38553 cephadm ceph cwd unknown /proc/28367/task/38553/cwd (readlink: Permission denied) 2024-04-26T21:58:48.701 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 38553 cephadm ceph rtd unknown /proc/28367/task/38553/root (readlink: Permission denied) 2024-04-26T21:58:48.701 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 38553 cephadm ceph txt unknown /proc/28367/task/38553/exe (readlink: Permission denied) 2024-04-26T21:58:48.701 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 38553 cephadm ceph NOFD /proc/28367/task/38553/fd (opendir: Permission denied) 2024-04-26T21:58:48.701 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 38554 cephadm ceph cwd unknown /proc/28367/task/38554/cwd (readlink: Permission denied) 2024-04-26T21:58:48.701 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 38554 cephadm ceph rtd unknown /proc/28367/task/38554/root (readlink: Permission denied) 2024-04-26T21:58:48.701 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 38554 cephadm ceph txt unknown /proc/28367/task/38554/exe (readlink: Permission denied) 2024-04-26T21:58:48.701 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 38554 cephadm ceph NOFD /proc/28367/task/38554/fd (opendir: Permission denied) 2024-04-26T21:58:48.701 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 38555 cephadm ceph cwd unknown /proc/28367/task/38555/cwd (readlink: Permission denied) 2024-04-26T21:58:48.701 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 38555 cephadm ceph rtd unknown /proc/28367/task/38555/root (readlink: Permission denied) 2024-04-26T21:58:48.701 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 38555 cephadm ceph txt unknown /proc/28367/task/38555/exe (readlink: Permission denied) 2024-04-26T21:58:48.702 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 38555 cephadm ceph NOFD /proc/28367/task/38555/fd (opendir: Permission denied) 2024-04-26T21:58:48.702 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 38556 cephadm ceph cwd unknown /proc/28367/task/38556/cwd (readlink: Permission denied) 2024-04-26T21:58:48.702 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 38556 cephadm ceph rtd unknown /proc/28367/task/38556/root (readlink: Permission denied) 2024-04-26T21:58:48.702 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 38556 cephadm ceph txt unknown /proc/28367/task/38556/exe (readlink: Permission denied) 2024-04-26T21:58:48.702 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 38556 cephadm ceph NOFD /proc/28367/task/38556/fd (opendir: Permission denied) 2024-04-26T21:58:48.702 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 38557 cephadm ceph cwd unknown /proc/28367/task/38557/cwd (readlink: Permission denied) 2024-04-26T21:58:48.702 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 38557 cephadm ceph rtd unknown /proc/28367/task/38557/root (readlink: Permission denied) 2024-04-26T21:58:48.702 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 38557 cephadm ceph txt unknown /proc/28367/task/38557/exe (readlink: Permission denied) 2024-04-26T21:58:48.702 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 38557 cephadm ceph NOFD /proc/28367/task/38557/fd (opendir: Permission denied) 2024-04-26T21:58:48.702 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 38868 cephadm ceph cwd unknown /proc/28367/task/38868/cwd (readlink: Permission denied) 2024-04-26T21:58:48.702 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 38868 cephadm ceph rtd unknown /proc/28367/task/38868/root (readlink: Permission denied) 2024-04-26T21:58:48.702 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 38868 cephadm ceph txt unknown /proc/28367/task/38868/exe (readlink: Permission denied) 2024-04-26T21:58:48.702 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 38868 cephadm ceph NOFD /proc/28367/task/38868/fd (opendir: Permission denied) 2024-04-26T21:58:48.703 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 38869 cephadm ceph cwd unknown /proc/28367/task/38869/cwd (readlink: Permission denied) 2024-04-26T21:58:48.703 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 38869 cephadm ceph rtd unknown /proc/28367/task/38869/root (readlink: Permission denied) 2024-04-26T21:58:48.703 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 38869 cephadm ceph txt unknown /proc/28367/task/38869/exe (readlink: Permission denied) 2024-04-26T21:58:48.703 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 38869 cephadm ceph NOFD /proc/28367/task/38869/fd (opendir: Permission denied) 2024-04-26T21:58:48.703 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 38870 cephadm ceph cwd unknown /proc/28367/task/38870/cwd (readlink: Permission denied) 2024-04-26T21:58:48.703 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 38870 cephadm ceph rtd unknown /proc/28367/task/38870/root (readlink: Permission denied) 2024-04-26T21:58:48.703 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 38870 cephadm ceph txt unknown /proc/28367/task/38870/exe (readlink: Permission denied) 2024-04-26T21:58:48.703 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 38870 cephadm ceph NOFD /proc/28367/task/38870/fd (opendir: Permission denied) 2024-04-26T21:58:48.703 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 38871 cephadm ceph cwd unknown /proc/28367/task/38871/cwd (readlink: Permission denied) 2024-04-26T21:58:48.703 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 38871 cephadm ceph rtd unknown /proc/28367/task/38871/root (readlink: Permission denied) 2024-04-26T21:58:48.703 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 38871 cephadm ceph txt unknown /proc/28367/task/38871/exe (readlink: Permission denied) 2024-04-26T21:58:48.703 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 38871 cephadm ceph NOFD /proc/28367/task/38871/fd (opendir: Permission denied) 2024-04-26T21:58:48.703 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 38872 cephadm ceph cwd unknown /proc/28367/task/38872/cwd (readlink: Permission denied) 2024-04-26T21:58:48.703 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 38872 cephadm ceph rtd unknown /proc/28367/task/38872/root (readlink: Permission denied) 2024-04-26T21:58:48.703 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 38872 cephadm ceph txt unknown /proc/28367/task/38872/exe (readlink: Permission denied) 2024-04-26T21:58:48.704 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 38872 cephadm ceph NOFD /proc/28367/task/38872/fd (opendir: Permission denied) 2024-04-26T21:58:48.704 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 38873 cephadm ceph cwd unknown /proc/28367/task/38873/cwd (readlink: Permission denied) 2024-04-26T21:58:48.704 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 38873 cephadm ceph rtd unknown /proc/28367/task/38873/root (readlink: Permission denied) 2024-04-26T21:58:48.704 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 38873 cephadm ceph txt unknown /proc/28367/task/38873/exe (readlink: Permission denied) 2024-04-26T21:58:48.704 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 38873 cephadm ceph NOFD /proc/28367/task/38873/fd (opendir: Permission denied) 2024-04-26T21:58:48.704 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 38874 cephadm ceph cwd unknown /proc/28367/task/38874/cwd (readlink: Permission denied) 2024-04-26T21:58:48.704 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 38874 cephadm ceph rtd unknown /proc/28367/task/38874/root (readlink: Permission denied) 2024-04-26T21:58:48.704 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 38874 cephadm ceph txt unknown /proc/28367/task/38874/exe (readlink: Permission denied) 2024-04-26T21:58:48.704 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 38874 cephadm ceph NOFD /proc/28367/task/38874/fd (opendir: Permission denied) 2024-04-26T21:58:48.704 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 38875 cephadm ceph cwd unknown /proc/28367/task/38875/cwd (readlink: Permission denied) 2024-04-26T21:58:48.704 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 38875 cephadm ceph rtd unknown /proc/28367/task/38875/root (readlink: Permission denied) 2024-04-26T21:58:48.704 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 38875 cephadm ceph txt unknown /proc/28367/task/38875/exe (readlink: Permission denied) 2024-04-26T21:58:48.704 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 38875 cephadm ceph NOFD /proc/28367/task/38875/fd (opendir: Permission denied) 2024-04-26T21:58:48.704 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 38876 cephadm ceph cwd unknown /proc/28367/task/38876/cwd (readlink: Permission denied) 2024-04-26T21:58:48.705 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 38876 cephadm ceph rtd unknown /proc/28367/task/38876/root (readlink: Permission denied) 2024-04-26T21:58:48.705 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 38876 cephadm ceph txt unknown /proc/28367/task/38876/exe (readlink: Permission denied) 2024-04-26T21:58:48.705 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 38876 cephadm ceph NOFD /proc/28367/task/38876/fd (opendir: Permission denied) 2024-04-26T21:58:48.705 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 38877 cephadm ceph cwd unknown /proc/28367/task/38877/cwd (readlink: Permission denied) 2024-04-26T21:58:48.705 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 38877 cephadm ceph rtd unknown /proc/28367/task/38877/root (readlink: Permission denied) 2024-04-26T21:58:48.705 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 38877 cephadm ceph txt unknown /proc/28367/task/38877/exe (readlink: Permission denied) 2024-04-26T21:58:48.705 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 38877 cephadm ceph NOFD /proc/28367/task/38877/fd (opendir: Permission denied) 2024-04-26T21:58:48.705 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 47574 devicehea ceph cwd unknown /proc/28367/task/47574/cwd (readlink: Permission denied) 2024-04-26T21:58:48.705 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 47574 devicehea ceph rtd unknown /proc/28367/task/47574/root (readlink: Permission denied) 2024-04-26T21:58:48.705 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 47574 devicehea ceph txt unknown /proc/28367/task/47574/exe (readlink: Permission denied) 2024-04-26T21:58:48.705 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 47574 devicehea ceph NOFD /proc/28367/task/47574/fd (opendir: Permission denied) 2024-04-26T21:58:48.705 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 47632 io_contex ceph cwd unknown /proc/28367/task/47632/cwd (readlink: Permission denied) 2024-04-26T21:58:48.705 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 47632 io_contex ceph rtd unknown /proc/28367/task/47632/root (readlink: Permission denied) 2024-04-26T21:58:48.705 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 47632 io_contex ceph txt unknown /proc/28367/task/47632/exe (readlink: Permission denied) 2024-04-26T21:58:48.706 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 47632 io_contex ceph NOFD /proc/28367/task/47632/fd (opendir: Permission denied) 2024-04-26T21:58:48.706 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 47633 io_contex ceph cwd unknown /proc/28367/task/47633/cwd (readlink: Permission denied) 2024-04-26T21:58:48.706 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 47633 io_contex ceph rtd unknown /proc/28367/task/47633/root (readlink: Permission denied) 2024-04-26T21:58:48.706 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 47633 io_contex ceph txt unknown /proc/28367/task/47633/exe (readlink: Permission denied) 2024-04-26T21:58:48.706 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 47633 io_contex ceph NOFD /proc/28367/task/47633/fd (opendir: Permission denied) 2024-04-26T21:58:48.706 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 47634 ceph_time ceph cwd unknown /proc/28367/task/47634/cwd (readlink: Permission denied) 2024-04-26T21:58:48.706 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 47634 ceph_time ceph rtd unknown /proc/28367/task/47634/root (readlink: Permission denied) 2024-04-26T21:58:48.706 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 47634 ceph_time ceph txt unknown /proc/28367/task/47634/exe (readlink: Permission denied) 2024-04-26T21:58:48.706 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 47634 ceph_time ceph NOFD /proc/28367/task/47634/fd (opendir: Permission denied) 2024-04-26T21:58:48.706 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 47635 ms_dispat ceph cwd unknown /proc/28367/task/47635/cwd (readlink: Permission denied) 2024-04-26T21:58:48.706 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 47635 ms_dispat ceph rtd unknown /proc/28367/task/47635/root (readlink: Permission denied) 2024-04-26T21:58:48.706 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 47635 ms_dispat ceph txt unknown /proc/28367/task/47635/exe (readlink: Permission denied) 2024-04-26T21:58:48.706 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 47635 ms_dispat ceph NOFD /proc/28367/task/47635/fd (opendir: Permission denied) 2024-04-26T21:58:48.706 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 47636 ms_local ceph cwd unknown /proc/28367/task/47636/cwd (readlink: Permission denied) 2024-04-26T21:58:48.707 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 47636 ms_local ceph rtd unknown /proc/28367/task/47636/root (readlink: Permission denied) 2024-04-26T21:58:48.707 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 47636 ms_local ceph txt unknown /proc/28367/task/47636/exe (readlink: Permission denied) 2024-04-26T21:58:48.707 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 47636 ms_local ceph NOFD /proc/28367/task/47636/fd (opendir: Permission denied) 2024-04-26T21:58:48.707 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 47637 safe_time ceph cwd unknown /proc/28367/task/47637/cwd (readlink: Permission denied) 2024-04-26T21:58:48.707 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 47637 safe_time ceph rtd unknown /proc/28367/task/47637/root (readlink: Permission denied) 2024-04-26T21:58:48.707 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 47637 safe_time ceph txt unknown /proc/28367/task/47637/exe (readlink: Permission denied) 2024-04-26T21:58:48.707 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 47637 safe_time ceph NOFD /proc/28367/task/47637/fd (opendir: Permission denied) 2024-04-26T21:58:48.707 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 47638 safe_time ceph cwd unknown /proc/28367/task/47638/cwd (readlink: Permission denied) 2024-04-26T21:58:48.707 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 47638 safe_time ceph rtd unknown /proc/28367/task/47638/root (readlink: Permission denied) 2024-04-26T21:58:48.707 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 47638 safe_time ceph txt unknown /proc/28367/task/47638/exe (readlink: Permission denied) 2024-04-26T21:58:48.707 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 47638 safe_time ceph NOFD /proc/28367/task/47638/fd (opendir: Permission denied) 2024-04-26T21:58:48.707 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 72097 io_contex ceph cwd unknown /proc/28367/task/72097/cwd (readlink: Permission denied) 2024-04-26T21:58:48.707 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 72097 io_contex ceph rtd unknown /proc/28367/task/72097/root (readlink: Permission denied) 2024-04-26T21:58:48.707 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 72097 io_contex ceph txt unknown /proc/28367/task/72097/exe (readlink: Permission denied) 2024-04-26T21:58:48.708 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 72097 io_contex ceph NOFD /proc/28367/task/72097/fd (opendir: Permission denied) 2024-04-26T21:58:48.708 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 72098 io_contex ceph cwd unknown /proc/28367/task/72098/cwd (readlink: Permission denied) 2024-04-26T21:58:48.708 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 72098 io_contex ceph rtd unknown /proc/28367/task/72098/root (readlink: Permission denied) 2024-04-26T21:58:48.708 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 72098 io_contex ceph txt unknown /proc/28367/task/72098/exe (readlink: Permission denied) 2024-04-26T21:58:48.708 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 72098 io_contex ceph NOFD /proc/28367/task/72098/fd (opendir: Permission denied) 2024-04-26T21:58:48.708 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 72102 ceph_time ceph cwd unknown /proc/28367/task/72102/cwd (readlink: Permission denied) 2024-04-26T21:58:48.708 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 72102 ceph_time ceph rtd unknown /proc/28367/task/72102/root (readlink: Permission denied) 2024-04-26T21:58:48.708 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 72102 ceph_time ceph txt unknown /proc/28367/task/72102/exe (readlink: Permission denied) 2024-04-26T21:58:48.708 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 72102 ceph_time ceph NOFD /proc/28367/task/72102/fd (opendir: Permission denied) 2024-04-26T21:58:48.708 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 72103 fn_anonym ceph cwd unknown /proc/28367/task/72103/cwd (readlink: Permission denied) 2024-04-26T21:58:48.708 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 72103 fn_anonym ceph rtd unknown /proc/28367/task/72103/root (readlink: Permission denied) 2024-04-26T21:58:48.708 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 72103 fn_anonym ceph txt unknown /proc/28367/task/72103/exe (readlink: Permission denied) 2024-04-26T21:58:48.708 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 72103 fn_anonym ceph NOFD /proc/28367/task/72103/fd (opendir: Permission denied) 2024-04-26T21:58:48.709 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 72104 safe_time ceph cwd unknown /proc/28367/task/72104/cwd (readlink: Permission denied) 2024-04-26T21:58:48.709 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 72104 safe_time ceph rtd unknown /proc/28367/task/72104/root (readlink: Permission denied) 2024-04-26T21:58:48.709 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 72104 safe_time ceph txt unknown /proc/28367/task/72104/exe (readlink: Permission denied) 2024-04-26T21:58:48.709 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 72104 safe_time ceph NOFD /proc/28367/task/72104/fd (opendir: Permission denied) 2024-04-26T21:58:48.709 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 72105 fn_anonym ceph cwd unknown /proc/28367/task/72105/cwd (readlink: Permission denied) 2024-04-26T21:58:48.709 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 72105 fn_anonym ceph rtd unknown /proc/28367/task/72105/root (readlink: Permission denied) 2024-04-26T21:58:48.709 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 72105 fn_anonym ceph txt unknown /proc/28367/task/72105/exe (readlink: Permission denied) 2024-04-26T21:58:48.709 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 72105 fn_anonym ceph NOFD /proc/28367/task/72105/fd (opendir: Permission denied) 2024-04-26T21:58:48.709 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 72106 flusher ceph cwd unknown /proc/28367/task/72106/cwd (readlink: Permission denied) 2024-04-26T21:58:48.709 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 72106 flusher ceph rtd unknown /proc/28367/task/72106/root (readlink: Permission denied) 2024-04-26T21:58:48.709 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 72106 flusher ceph txt unknown /proc/28367/task/72106/exe (readlink: Permission denied) 2024-04-26T21:58:48.709 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 72106 flusher ceph NOFD /proc/28367/task/72106/fd (opendir: Permission denied) 2024-04-26T21:58:48.709 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 72107 ms_dispat ceph cwd unknown /proc/28367/task/72107/cwd (readlink: Permission denied) 2024-04-26T21:58:48.709 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 72107 ms_dispat ceph rtd unknown /proc/28367/task/72107/root (readlink: Permission denied) 2024-04-26T21:58:48.710 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 72107 ms_dispat ceph txt unknown /proc/28367/task/72107/exe (readlink: Permission denied) 2024-04-26T21:58:48.710 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 72107 ms_dispat ceph NOFD /proc/28367/task/72107/fd (opendir: Permission denied) 2024-04-26T21:58:48.710 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 72108 ms_local ceph cwd unknown /proc/28367/task/72108/cwd (readlink: Permission denied) 2024-04-26T21:58:48.710 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 72108 ms_local ceph rtd unknown /proc/28367/task/72108/root (readlink: Permission denied) 2024-04-26T21:58:48.710 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 72108 ms_local ceph txt unknown /proc/28367/task/72108/exe (readlink: Permission denied) 2024-04-26T21:58:48.710 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 72108 ms_local ceph NOFD /proc/28367/task/72108/fd (opendir: Permission denied) 2024-04-26T21:58:48.710 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 72109 safe_time ceph cwd unknown /proc/28367/task/72109/cwd (readlink: Permission denied) 2024-04-26T21:58:48.710 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 72109 safe_time ceph rtd unknown /proc/28367/task/72109/root (readlink: Permission denied) 2024-04-26T21:58:48.710 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 72109 safe_time ceph txt unknown /proc/28367/task/72109/exe (readlink: Permission denied) 2024-04-26T21:58:48.710 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 72109 safe_time ceph NOFD /proc/28367/task/72109/fd (opendir: Permission denied) 2024-04-26T21:58:48.710 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 72110 m-fin-vol ceph cwd unknown /proc/28367/task/72110/cwd (readlink: Permission denied) 2024-04-26T21:58:48.710 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 72110 m-fin-vol ceph rtd unknown /proc/28367/task/72110/root (readlink: Permission denied) 2024-04-26T21:58:48.710 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 72110 m-fin-vol ceph txt unknown /proc/28367/task/72110/exe (readlink: Permission denied) 2024-04-26T21:58:48.710 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mgr 28367 72110 m-fin-vol ceph NOFD /proc/28367/task/72110/fd (opendir: Permission denied) 2024-04-26T21:58:48.711 INFO:tasks.workunit.client.0.smithi072.stdout:systemd 29507 root cwd unknown /proc/29507/cwd (readlink: Permission denied) 2024-04-26T21:58:48.711 INFO:tasks.workunit.client.0.smithi072.stdout:systemd 29507 root rtd unknown /proc/29507/root (readlink: Permission denied) 2024-04-26T21:58:48.711 INFO:tasks.workunit.client.0.smithi072.stdout:systemd 29507 root txt unknown /proc/29507/exe (readlink: Permission denied) 2024-04-26T21:58:48.711 INFO:tasks.workunit.client.0.smithi072.stdout:systemd 29507 root NOFD /proc/29507/fd (opendir: Permission denied) 2024-04-26T21:58:48.711 INFO:tasks.workunit.client.0.smithi072.stdout:(sd-pam) 29509 root cwd unknown /proc/29509/cwd (readlink: Permission denied) 2024-04-26T21:58:48.711 INFO:tasks.workunit.client.0.smithi072.stdout:(sd-pam) 29509 root rtd unknown /proc/29509/root (readlink: Permission denied) 2024-04-26T21:58:48.711 INFO:tasks.workunit.client.0.smithi072.stdout:(sd-pam) 29509 root txt unknown /proc/29509/exe (readlink: Permission denied) 2024-04-26T21:58:48.711 INFO:tasks.workunit.client.0.smithi072.stdout:(sd-pam) 29509 root NOFD /proc/29509/fd (opendir: Permission denied) 2024-04-26T21:58:48.711 INFO:tasks.workunit.client.0.smithi072.stdout:sshd 31168 root cwd unknown /proc/31168/cwd (readlink: Permission denied) 2024-04-26T21:58:48.711 INFO:tasks.workunit.client.0.smithi072.stdout:sshd 31168 root rtd unknown /proc/31168/root (readlink: Permission denied) 2024-04-26T21:58:48.711 INFO:tasks.workunit.client.0.smithi072.stdout:sshd 31168 root txt unknown /proc/31168/exe (readlink: Permission denied) 2024-04-26T21:58:48.711 INFO:tasks.workunit.client.0.smithi072.stdout:sshd 31168 root NOFD /proc/31168/fd (opendir: Permission denied) 2024-04-26T21:58:48.711 INFO:tasks.workunit.client.0.smithi072.stdout:sshd 31192 root cwd unknown /proc/31192/cwd (readlink: Permission denied) 2024-04-26T21:58:48.711 INFO:tasks.workunit.client.0.smithi072.stdout:sshd 31192 root rtd unknown /proc/31192/root (readlink: Permission denied) 2024-04-26T21:58:48.712 INFO:tasks.workunit.client.0.smithi072.stdout:sshd 31192 root txt unknown /proc/31192/exe (readlink: Permission denied) 2024-04-26T21:58:48.712 INFO:tasks.workunit.client.0.smithi072.stdout:sshd 31192 root NOFD /proc/31192/fd (opendir: Permission denied) 2024-04-26T21:58:48.712 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/4 39249 root cwd unknown /proc/39249/cwd (readlink: Permission denied) 2024-04-26T21:58:48.712 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/4 39249 root rtd unknown /proc/39249/root (readlink: Permission denied) 2024-04-26T21:58:48.712 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/4 39249 root txt unknown /proc/39249/exe (readlink: Permission denied) 2024-04-26T21:58:48.712 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/4 39249 root NOFD /proc/39249/fd (opendir: Permission denied) 2024-04-26T21:58:48.712 INFO:tasks.workunit.client.0.smithi072.stdout:conmon 43226 root cwd unknown /proc/43226/cwd (readlink: Permission denied) 2024-04-26T21:58:48.712 INFO:tasks.workunit.client.0.smithi072.stdout:conmon 43226 root rtd unknown /proc/43226/root (readlink: Permission denied) 2024-04-26T21:58:48.712 INFO:tasks.workunit.client.0.smithi072.stdout:conmon 43226 root txt unknown /proc/43226/exe (readlink: Permission denied) 2024-04-26T21:58:48.712 INFO:tasks.workunit.client.0.smithi072.stdout:conmon 43226 root NOFD /proc/43226/fd (opendir: Permission denied) 2024-04-26T21:58:48.712 INFO:tasks.workunit.client.0.smithi072.stdout:podman-in 43228 root cwd unknown /proc/43228/cwd (readlink: Permission denied) 2024-04-26T21:58:48.712 INFO:tasks.workunit.client.0.smithi072.stdout:podman-in 43228 root rtd unknown /proc/43228/root (readlink: Permission denied) 2024-04-26T21:58:48.712 INFO:tasks.workunit.client.0.smithi072.stdout:podman-in 43228 root txt unknown /proc/43228/exe (readlink: Permission denied) 2024-04-26T21:58:48.713 INFO:tasks.workunit.client.0.smithi072.stdout:podman-in 43228 root NOFD /proc/43228/fd (opendir: Permission denied) 2024-04-26T21:58:48.713 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 ceph cwd unknown /proc/43230/cwd (readlink: Permission denied) 2024-04-26T21:58:48.713 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 ceph rtd unknown /proc/43230/root (readlink: Permission denied) 2024-04-26T21:58:48.713 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 ceph txt unknown /proc/43230/exe (readlink: Permission denied) 2024-04-26T21:58:48.713 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 ceph NOFD /proc/43230/fd (opendir: Permission denied) 2024-04-26T21:58:48.713 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 43231 log ceph cwd unknown /proc/43230/task/43231/cwd (readlink: Permission denied) 2024-04-26T21:58:48.713 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 43231 log ceph rtd unknown /proc/43230/task/43231/root (readlink: Permission denied) 2024-04-26T21:58:48.713 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 43231 log ceph txt unknown /proc/43230/task/43231/exe (readlink: Permission denied) 2024-04-26T21:58:48.713 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 43231 log ceph NOFD /proc/43230/task/43231/fd (opendir: Permission denied) 2024-04-26T21:58:48.713 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 43233 msgr-work ceph cwd unknown /proc/43230/task/43233/cwd (readlink: Permission denied) 2024-04-26T21:58:48.713 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 43233 msgr-work ceph rtd unknown /proc/43230/task/43233/root (readlink: Permission denied) 2024-04-26T21:58:48.713 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 43233 msgr-work ceph txt unknown /proc/43230/task/43233/exe (readlink: Permission denied) 2024-04-26T21:58:48.713 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 43233 msgr-work ceph NOFD /proc/43230/task/43233/fd (opendir: Permission denied) 2024-04-26T21:58:48.713 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 43234 msgr-work ceph cwd unknown /proc/43230/task/43234/cwd (readlink: Permission denied) 2024-04-26T21:58:48.714 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 43234 msgr-work ceph rtd unknown /proc/43230/task/43234/root (readlink: Permission denied) 2024-04-26T21:58:48.714 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 43234 msgr-work ceph txt unknown /proc/43230/task/43234/exe (readlink: Permission denied) 2024-04-26T21:58:48.714 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 43234 msgr-work ceph NOFD /proc/43230/task/43234/fd (opendir: Permission denied) 2024-04-26T21:58:48.714 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 43235 msgr-work ceph cwd unknown /proc/43230/task/43235/cwd (readlink: Permission denied) 2024-04-26T21:58:48.714 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 43235 msgr-work ceph rtd unknown /proc/43230/task/43235/root (readlink: Permission denied) 2024-04-26T21:58:48.714 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 43235 msgr-work ceph txt unknown /proc/43230/task/43235/exe (readlink: Permission denied) 2024-04-26T21:58:48.714 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 43235 msgr-work ceph NOFD /proc/43230/task/43235/fd (opendir: Permission denied) 2024-04-26T21:58:48.714 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 43239 service ceph cwd unknown /proc/43230/task/43239/cwd (readlink: Permission denied) 2024-04-26T21:58:48.714 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 43239 service ceph rtd unknown /proc/43230/task/43239/root (readlink: Permission denied) 2024-04-26T21:58:48.714 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 43239 service ceph txt unknown /proc/43230/task/43239/exe (readlink: Permission denied) 2024-04-26T21:58:48.714 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 43239 service ceph NOFD /proc/43230/task/43239/fd (opendir: Permission denied) 2024-04-26T21:58:48.714 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 43240 admin_soc ceph cwd unknown /proc/43230/task/43240/cwd (readlink: Permission denied) 2024-04-26T21:58:48.715 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 43240 admin_soc ceph rtd unknown /proc/43230/task/43240/root (readlink: Permission denied) 2024-04-26T21:58:48.715 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 43240 admin_soc ceph txt unknown /proc/43230/task/43240/exe (readlink: Permission denied) 2024-04-26T21:58:48.715 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 43240 admin_soc ceph NOFD /proc/43230/task/43240/fd (opendir: Permission denied) 2024-04-26T21:58:48.715 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 43384 signal_ha ceph cwd unknown /proc/43230/task/43384/cwd (readlink: Permission denied) 2024-04-26T21:58:48.715 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 43384 signal_ha ceph rtd unknown /proc/43230/task/43384/root (readlink: Permission denied) 2024-04-26T21:58:48.715 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 43384 signal_ha ceph txt unknown /proc/43230/task/43384/exe (readlink: Permission denied) 2024-04-26T21:58:48.715 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 43384 signal_ha ceph NOFD /proc/43230/task/43384/fd (opendir: Permission denied) 2024-04-26T21:58:48.715 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 43385 io_contex ceph cwd unknown /proc/43230/task/43385/cwd (readlink: Permission denied) 2024-04-26T21:58:48.715 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 43385 io_contex ceph rtd unknown /proc/43230/task/43385/root (readlink: Permission denied) 2024-04-26T21:58:48.715 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 43385 io_contex ceph txt unknown /proc/43230/task/43385/exe (readlink: Permission denied) 2024-04-26T21:58:48.715 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 43385 io_contex ceph NOFD /proc/43230/task/43385/fd (opendir: Permission denied) 2024-04-26T21:58:48.715 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 43386 io_contex ceph cwd unknown /proc/43230/task/43386/cwd (readlink: Permission denied) 2024-04-26T21:58:48.715 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 43386 io_contex ceph rtd unknown /proc/43230/task/43386/root (readlink: Permission denied) 2024-04-26T21:58:48.716 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 43386 io_contex ceph txt unknown /proc/43230/task/43386/exe (readlink: Permission denied) 2024-04-26T21:58:48.716 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 43386 io_contex ceph NOFD /proc/43230/task/43386/fd (opendir: Permission denied) 2024-04-26T21:58:48.716 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 43425 OpHistory ceph cwd unknown /proc/43230/task/43425/cwd (readlink: Permission denied) 2024-04-26T21:58:48.716 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 43425 OpHistory ceph rtd unknown /proc/43230/task/43425/root (readlink: Permission denied) 2024-04-26T21:58:48.716 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 43425 OpHistory ceph txt unknown /proc/43230/task/43425/exe (readlink: Permission denied) 2024-04-26T21:58:48.716 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 43425 OpHistory ceph NOFD /proc/43230/task/43425/fd (opendir: Permission denied) 2024-04-26T21:58:48.716 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 43426 ceph_time ceph cwd unknown /proc/43230/task/43426/cwd (readlink: Permission denied) 2024-04-26T21:58:48.716 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 43426 ceph_time ceph rtd unknown /proc/43230/task/43426/root (readlink: Permission denied) 2024-04-26T21:58:48.716 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 43426 ceph_time ceph txt unknown /proc/43230/task/43426/exe (readlink: Permission denied) 2024-04-26T21:58:48.716 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 43426 ceph_time ceph NOFD /proc/43230/task/43426/fd (opendir: Permission denied) 2024-04-26T21:58:48.716 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 43499 ceph-osd ceph cwd unknown /proc/43230/task/43499/cwd (readlink: Permission denied) 2024-04-26T21:58:48.716 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 43499 ceph-osd ceph rtd unknown /proc/43230/task/43499/root (readlink: Permission denied) 2024-04-26T21:58:48.716 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 43499 ceph-osd ceph txt unknown /proc/43230/task/43499/exe (readlink: Permission denied) 2024-04-26T21:58:48.716 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 43499 ceph-osd ceph NOFD /proc/43230/task/43499/fd (opendir: Permission denied) 2024-04-26T21:58:48.716 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 43537 ceph-osd ceph cwd unknown /proc/43230/task/43537/cwd (readlink: Permission denied) 2024-04-26T21:58:48.717 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 43537 ceph-osd ceph rtd unknown /proc/43230/task/43537/root (readlink: Permission denied) 2024-04-26T21:58:48.717 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 43537 ceph-osd ceph txt unknown /proc/43230/task/43537/exe (readlink: Permission denied) 2024-04-26T21:58:48.717 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 43537 ceph-osd ceph NOFD /proc/43230/task/43537/fd (opendir: Permission denied) 2024-04-26T21:58:48.717 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 43541 ceph-osd ceph cwd unknown /proc/43230/task/43541/cwd (readlink: Permission denied) 2024-04-26T21:58:48.717 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 43541 ceph-osd ceph rtd unknown /proc/43230/task/43541/root (readlink: Permission denied) 2024-04-26T21:58:48.717 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 43541 ceph-osd ceph txt unknown /proc/43230/task/43541/exe (readlink: Permission denied) 2024-04-26T21:58:48.717 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 43541 ceph-osd ceph NOFD /proc/43230/task/43541/fd (opendir: Permission denied) 2024-04-26T21:58:48.717 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 43563 ceph-osd ceph cwd unknown /proc/43230/task/43563/cwd (readlink: Permission denied) 2024-04-26T21:58:48.717 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 43563 ceph-osd ceph rtd unknown /proc/43230/task/43563/root (readlink: Permission denied) 2024-04-26T21:58:48.717 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 43563 ceph-osd ceph txt unknown /proc/43230/task/43563/exe (readlink: Permission denied) 2024-04-26T21:58:48.717 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 43563 ceph-osd ceph NOFD /proc/43230/task/43563/fd (opendir: Permission denied) 2024-04-26T21:58:48.717 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 43567 ceph-osd ceph cwd unknown /proc/43230/task/43567/cwd (readlink: Permission denied) 2024-04-26T21:58:48.717 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 43567 ceph-osd ceph rtd unknown /proc/43230/task/43567/root (readlink: Permission denied) 2024-04-26T21:58:48.717 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 43567 ceph-osd ceph txt unknown /proc/43230/task/43567/exe (readlink: Permission denied) 2024-04-26T21:58:48.717 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 43567 ceph-osd ceph NOFD /proc/43230/task/43567/fd (opendir: Permission denied) 2024-04-26T21:58:48.718 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 43596 ceph-osd ceph cwd unknown /proc/43230/task/43596/cwd (readlink: Permission denied) 2024-04-26T21:58:48.718 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 43596 ceph-osd ceph rtd unknown /proc/43230/task/43596/root (readlink: Permission denied) 2024-04-26T21:58:48.718 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 43596 ceph-osd ceph txt unknown /proc/43230/task/43596/exe (readlink: Permission denied) 2024-04-26T21:58:48.718 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 43596 ceph-osd ceph NOFD /proc/43230/task/43596/fd (opendir: Permission denied) 2024-04-26T21:58:48.718 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 43614 ceph-osd ceph cwd unknown /proc/43230/task/43614/cwd (readlink: Permission denied) 2024-04-26T21:58:48.718 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 43614 ceph-osd ceph rtd unknown /proc/43230/task/43614/root (readlink: Permission denied) 2024-04-26T21:58:48.718 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 43614 ceph-osd ceph txt unknown /proc/43230/task/43614/exe (readlink: Permission denied) 2024-04-26T21:58:48.718 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 43614 ceph-osd ceph NOFD /proc/43230/task/43614/fd (opendir: Permission denied) 2024-04-26T21:58:48.718 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 43625 ceph-osd ceph cwd unknown /proc/43230/task/43625/cwd (readlink: Permission denied) 2024-04-26T21:58:48.718 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 43625 ceph-osd ceph rtd unknown /proc/43230/task/43625/root (readlink: Permission denied) 2024-04-26T21:58:48.719 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 43625 ceph-osd ceph txt unknown /proc/43230/task/43625/exe (readlink: Permission denied) 2024-04-26T21:58:48.719 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 43625 ceph-osd ceph NOFD /proc/43230/task/43625/fd (opendir: Permission denied) 2024-04-26T21:58:48.719 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 43626 ceph-osd ceph cwd unknown /proc/43230/task/43626/cwd (readlink: Permission denied) 2024-04-26T21:58:48.719 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 43626 ceph-osd ceph rtd unknown /proc/43230/task/43626/root (readlink: Permission denied) 2024-04-26T21:58:48.719 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 43626 ceph-osd ceph txt unknown /proc/43230/task/43626/exe (readlink: Permission denied) 2024-04-26T21:58:48.719 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 43626 ceph-osd ceph NOFD /proc/43230/task/43626/fd (opendir: Permission denied) 2024-04-26T21:58:48.719 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 43627 safe_time ceph cwd unknown /proc/43230/task/43627/cwd (readlink: Permission denied) 2024-04-26T21:58:48.719 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 43627 safe_time ceph rtd unknown /proc/43230/task/43627/root (readlink: Permission denied) 2024-04-26T21:58:48.719 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 43627 safe_time ceph txt unknown /proc/43230/task/43627/exe (readlink: Permission denied) 2024-04-26T21:58:48.719 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 43627 safe_time ceph NOFD /proc/43230/task/43627/fd (opendir: Permission denied) 2024-04-26T21:58:48.719 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 43628 safe_time ceph cwd unknown /proc/43230/task/43628/cwd (readlink: Permission denied) 2024-04-26T21:58:48.719 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 43628 safe_time ceph rtd unknown /proc/43230/task/43628/root (readlink: Permission denied) 2024-04-26T21:58:48.719 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 43628 safe_time ceph txt unknown /proc/43230/task/43628/exe (readlink: Permission denied) 2024-04-26T21:58:48.719 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 43628 safe_time ceph NOFD /proc/43230/task/43628/fd (opendir: Permission denied) 2024-04-26T21:58:48.719 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 43629 safe_time ceph cwd unknown /proc/43230/task/43629/cwd (readlink: Permission denied) 2024-04-26T21:58:48.719 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 43629 safe_time ceph rtd unknown /proc/43230/task/43629/root (readlink: Permission denied) 2024-04-26T21:58:48.720 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 43629 safe_time ceph txt unknown /proc/43230/task/43629/exe (readlink: Permission denied) 2024-04-26T21:58:48.720 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 43629 safe_time ceph NOFD /proc/43230/task/43629/fd (opendir: Permission denied) 2024-04-26T21:58:48.720 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 43630 safe_time ceph cwd unknown /proc/43230/task/43630/cwd (readlink: Permission denied) 2024-04-26T21:58:48.720 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 43630 safe_time ceph rtd unknown /proc/43230/task/43630/root (readlink: Permission denied) 2024-04-26T21:58:48.720 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 43630 safe_time ceph txt unknown /proc/43230/task/43630/exe (readlink: Permission denied) 2024-04-26T21:58:48.720 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 43630 safe_time ceph NOFD /proc/43230/task/43630/fd (opendir: Permission denied) 2024-04-26T21:58:48.720 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 43631 fn_anonym ceph cwd unknown /proc/43230/task/43631/cwd (readlink: Permission denied) 2024-04-26T21:58:48.720 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 43631 fn_anonym ceph rtd unknown /proc/43230/task/43631/root (readlink: Permission denied) 2024-04-26T21:58:48.720 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 43631 fn_anonym ceph txt unknown /proc/43230/task/43631/exe (readlink: Permission denied) 2024-04-26T21:58:48.721 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 43631 fn_anonym ceph NOFD /proc/43230/task/43631/fd (opendir: Permission denied) 2024-04-26T21:58:48.721 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 43636 rocksdb:l ceph cwd unknown /proc/43230/task/43636/cwd (readlink: Permission denied) 2024-04-26T21:58:48.721 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 43636 rocksdb:l ceph rtd unknown /proc/43230/task/43636/root (readlink: Permission denied) 2024-04-26T21:58:48.721 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 43636 rocksdb:l ceph txt unknown /proc/43230/task/43636/exe (readlink: Permission denied) 2024-04-26T21:58:48.721 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 43636 rocksdb:l ceph NOFD /proc/43230/task/43636/fd (opendir: Permission denied) 2024-04-26T21:58:48.721 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 43637 rocksdb:l ceph cwd unknown /proc/43230/task/43637/cwd (readlink: Permission denied) 2024-04-26T21:58:48.721 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 43637 rocksdb:l ceph rtd unknown /proc/43230/task/43637/root (readlink: Permission denied) 2024-04-26T21:58:48.721 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 43637 rocksdb:l ceph txt unknown /proc/43230/task/43637/exe (readlink: Permission denied) 2024-04-26T21:58:48.721 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 43637 rocksdb:l ceph NOFD /proc/43230/task/43637/fd (opendir: Permission denied) 2024-04-26T21:58:48.721 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 43638 rocksdb:l ceph cwd unknown /proc/43230/task/43638/cwd (readlink: Permission denied) 2024-04-26T21:58:48.721 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 43638 rocksdb:l ceph rtd unknown /proc/43230/task/43638/root (readlink: Permission denied) 2024-04-26T21:58:48.721 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 43638 rocksdb:l ceph txt unknown /proc/43230/task/43638/exe (readlink: Permission denied) 2024-04-26T21:58:48.721 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 43638 rocksdb:l ceph NOFD /proc/43230/task/43638/fd (opendir: Permission denied) 2024-04-26T21:58:48.721 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 43639 rocksdb:h ceph cwd unknown /proc/43230/task/43639/cwd (readlink: Permission denied) 2024-04-26T21:58:48.722 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 43639 rocksdb:h ceph rtd unknown /proc/43230/task/43639/root (readlink: Permission denied) 2024-04-26T21:58:48.722 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 43639 rocksdb:h ceph txt unknown /proc/43230/task/43639/exe (readlink: Permission denied) 2024-04-26T21:58:48.722 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 43639 rocksdb:h ceph NOFD /proc/43230/task/43639/fd (opendir: Permission denied) 2024-04-26T21:58:48.722 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 44063 bstore_ai ceph cwd unknown /proc/43230/task/44063/cwd (readlink: Permission denied) 2024-04-26T21:58:48.722 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 44063 bstore_ai ceph rtd unknown /proc/43230/task/44063/root (readlink: Permission denied) 2024-04-26T21:58:48.722 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 44063 bstore_ai ceph txt unknown /proc/43230/task/44063/exe (readlink: Permission denied) 2024-04-26T21:58:48.722 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 44063 bstore_ai ceph NOFD /proc/43230/task/44063/fd (opendir: Permission denied) 2024-04-26T21:58:48.722 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 44291 bstore_ai ceph cwd unknown /proc/43230/task/44291/cwd (readlink: Permission denied) 2024-04-26T21:58:48.723 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 44291 bstore_ai ceph rtd unknown /proc/43230/task/44291/root (readlink: Permission denied) 2024-04-26T21:58:48.723 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 44291 bstore_ai ceph txt unknown /proc/43230/task/44291/exe (readlink: Permission denied) 2024-04-26T21:58:48.723 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 44291 bstore_ai ceph NOFD /proc/43230/task/44291/fd (opendir: Permission denied) 2024-04-26T21:58:48.723 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 44476 ceph-osd ceph cwd unknown /proc/43230/task/44476/cwd (readlink: Permission denied) 2024-04-26T21:58:48.723 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 44476 ceph-osd ceph rtd unknown /proc/43230/task/44476/root (readlink: Permission denied) 2024-04-26T21:58:48.723 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 44476 ceph-osd ceph txt unknown /proc/43230/task/44476/exe (readlink: Permission denied) 2024-04-26T21:58:48.723 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 44476 ceph-osd ceph NOFD /proc/43230/task/44476/fd (opendir: Permission denied) 2024-04-26T21:58:48.723 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 44477 cfin ceph cwd unknown /proc/43230/task/44477/cwd (readlink: Permission denied) 2024-04-26T21:58:48.723 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 44477 cfin ceph rtd unknown /proc/43230/task/44477/root (readlink: Permission denied) 2024-04-26T21:58:48.723 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 44477 cfin ceph txt unknown /proc/43230/task/44477/exe (readlink: Permission denied) 2024-04-26T21:58:48.723 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 44477 cfin ceph NOFD /proc/43230/task/44477/fd (opendir: Permission denied) 2024-04-26T21:58:48.723 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 44478 bstore_kv ceph cwd unknown /proc/43230/task/44478/cwd (readlink: Permission denied) 2024-04-26T21:58:48.723 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 44478 bstore_kv ceph rtd unknown /proc/43230/task/44478/root (readlink: Permission denied) 2024-04-26T21:58:48.723 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 44478 bstore_kv ceph txt unknown /proc/43230/task/44478/exe (readlink: Permission denied) 2024-04-26T21:58:48.723 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 44478 bstore_kv ceph NOFD /proc/43230/task/44478/fd (opendir: Permission denied) 2024-04-26T21:58:48.724 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 44479 bstore_kv ceph cwd unknown /proc/43230/task/44479/cwd (readlink: Permission denied) 2024-04-26T21:58:48.724 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 44479 bstore_kv ceph rtd unknown /proc/43230/task/44479/root (readlink: Permission denied) 2024-04-26T21:58:48.724 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 44479 bstore_kv ceph txt unknown /proc/43230/task/44479/exe (readlink: Permission denied) 2024-04-26T21:58:48.724 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 44479 bstore_kv ceph NOFD /proc/43230/task/44479/fd (opendir: Permission denied) 2024-04-26T21:58:48.724 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 44480 bstore_me ceph cwd unknown /proc/43230/task/44480/cwd (readlink: Permission denied) 2024-04-26T21:58:48.724 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 44480 bstore_me ceph rtd unknown /proc/43230/task/44480/root (readlink: Permission denied) 2024-04-26T21:58:48.724 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 44480 bstore_me ceph txt unknown /proc/43230/task/44480/exe (readlink: Permission denied) 2024-04-26T21:58:48.724 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 44480 bstore_me ceph NOFD /proc/43230/task/44480/fd (opendir: Permission denied) 2024-04-26T21:58:48.724 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 44481 ms_dispat ceph cwd unknown /proc/43230/task/44481/cwd (readlink: Permission denied) 2024-04-26T21:58:48.724 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 44481 ms_dispat ceph rtd unknown /proc/43230/task/44481/root (readlink: Permission denied) 2024-04-26T21:58:48.725 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 44481 ms_dispat ceph txt unknown /proc/43230/task/44481/exe (readlink: Permission denied) 2024-04-26T21:58:48.725 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 44481 ms_dispat ceph NOFD /proc/43230/task/44481/fd (opendir: Permission denied) 2024-04-26T21:58:48.725 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 44482 ms_local ceph cwd unknown /proc/43230/task/44482/cwd (readlink: Permission denied) 2024-04-26T21:58:48.725 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 44482 ms_local ceph rtd unknown /proc/43230/task/44482/root (readlink: Permission denied) 2024-04-26T21:58:48.725 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 44482 ms_local ceph txt unknown /proc/43230/task/44482/exe (readlink: Permission denied) 2024-04-26T21:58:48.725 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 44482 ms_local ceph NOFD /proc/43230/task/44482/fd (opendir: Permission denied) 2024-04-26T21:58:48.725 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 44483 safe_time ceph cwd unknown /proc/43230/task/44483/cwd (readlink: Permission denied) 2024-04-26T21:58:48.725 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 44483 safe_time ceph rtd unknown /proc/43230/task/44483/root (readlink: Permission denied) 2024-04-26T21:58:48.725 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 44483 safe_time ceph txt unknown /proc/43230/task/44483/exe (readlink: Permission denied) 2024-04-26T21:58:48.725 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 44483 safe_time ceph NOFD /proc/43230/task/44483/fd (opendir: Permission denied) 2024-04-26T21:58:48.725 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 44484 safe_time ceph cwd unknown /proc/43230/task/44484/cwd (readlink: Permission denied) 2024-04-26T21:58:48.726 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 44484 safe_time ceph rtd unknown /proc/43230/task/44484/root (readlink: Permission denied) 2024-04-26T21:58:48.726 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 44484 safe_time ceph txt unknown /proc/43230/task/44484/exe (readlink: Permission denied) 2024-04-26T21:58:48.726 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 44484 safe_time ceph NOFD /proc/43230/task/44484/fd (opendir: Permission denied) 2024-04-26T21:58:48.726 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 44485 ms_dispat ceph cwd unknown /proc/43230/task/44485/cwd (readlink: Permission denied) 2024-04-26T21:58:48.726 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 44485 ms_dispat ceph rtd unknown /proc/43230/task/44485/root (readlink: Permission denied) 2024-04-26T21:58:48.726 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 44485 ms_dispat ceph txt unknown /proc/43230/task/44485/exe (readlink: Permission denied) 2024-04-26T21:58:48.726 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 44485 ms_dispat ceph NOFD /proc/43230/task/44485/fd (opendir: Permission denied) 2024-04-26T21:58:48.726 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 44486 ms_local ceph cwd unknown /proc/43230/task/44486/cwd (readlink: Permission denied) 2024-04-26T21:58:48.726 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 44486 ms_local ceph rtd unknown /proc/43230/task/44486/root (readlink: Permission denied) 2024-04-26T21:58:48.726 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 44486 ms_local ceph txt unknown /proc/43230/task/44486/exe (readlink: Permission denied) 2024-04-26T21:58:48.727 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 44486 ms_local ceph NOFD /proc/43230/task/44486/fd (opendir: Permission denied) 2024-04-26T21:58:48.727 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 44487 ms_dispat ceph cwd unknown /proc/43230/task/44487/cwd (readlink: Permission denied) 2024-04-26T21:58:48.727 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 44487 ms_dispat ceph rtd unknown /proc/43230/task/44487/root (readlink: Permission denied) 2024-04-26T21:58:48.727 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 44487 ms_dispat ceph txt unknown /proc/43230/task/44487/exe (readlink: Permission denied) 2024-04-26T21:58:48.727 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 44487 ms_dispat ceph NOFD /proc/43230/task/44487/fd (opendir: Permission denied) 2024-04-26T21:58:48.727 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 44488 ms_local ceph cwd unknown /proc/43230/task/44488/cwd (readlink: Permission denied) 2024-04-26T21:58:48.727 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 44488 ms_local ceph rtd unknown /proc/43230/task/44488/root (readlink: Permission denied) 2024-04-26T21:58:48.727 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 44488 ms_local ceph txt unknown /proc/43230/task/44488/exe (readlink: Permission denied) 2024-04-26T21:58:48.727 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 44488 ms_local ceph NOFD /proc/43230/task/44488/fd (opendir: Permission denied) 2024-04-26T21:58:48.727 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 44489 ms_dispat ceph cwd unknown /proc/43230/task/44489/cwd (readlink: Permission denied) 2024-04-26T21:58:48.727 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 44489 ms_dispat ceph rtd unknown /proc/43230/task/44489/root (readlink: Permission denied) 2024-04-26T21:58:48.727 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 44489 ms_dispat ceph txt unknown /proc/43230/task/44489/exe (readlink: Permission denied) 2024-04-26T21:58:48.728 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 44489 ms_dispat ceph NOFD /proc/43230/task/44489/fd (opendir: Permission denied) 2024-04-26T21:58:48.728 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 44490 ms_local ceph cwd unknown /proc/43230/task/44490/cwd (readlink: Permission denied) 2024-04-26T21:58:48.728 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 44490 ms_local ceph rtd unknown /proc/43230/task/44490/root (readlink: Permission denied) 2024-04-26T21:58:48.728 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 44490 ms_local ceph txt unknown /proc/43230/task/44490/exe (readlink: Permission denied) 2024-04-26T21:58:48.728 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 44490 ms_local ceph NOFD /proc/43230/task/44490/fd (opendir: Permission denied) 2024-04-26T21:58:48.728 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 44491 ms_dispat ceph cwd unknown /proc/43230/task/44491/cwd (readlink: Permission denied) 2024-04-26T21:58:48.728 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 44491 ms_dispat ceph rtd unknown /proc/43230/task/44491/root (readlink: Permission denied) 2024-04-26T21:58:48.728 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 44491 ms_dispat ceph txt unknown /proc/43230/task/44491/exe (readlink: Permission denied) 2024-04-26T21:58:48.728 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 44491 ms_dispat ceph NOFD /proc/43230/task/44491/fd (opendir: Permission denied) 2024-04-26T21:58:48.728 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 44492 ms_local ceph cwd unknown /proc/43230/task/44492/cwd (readlink: Permission denied) 2024-04-26T21:58:48.728 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 44492 ms_local ceph rtd unknown /proc/43230/task/44492/root (readlink: Permission denied) 2024-04-26T21:58:48.729 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 44492 ms_local ceph txt unknown /proc/43230/task/44492/exe (readlink: Permission denied) 2024-04-26T21:58:48.729 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 44492 ms_local ceph NOFD /proc/43230/task/44492/fd (opendir: Permission denied) 2024-04-26T21:58:48.729 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 44493 ms_dispat ceph cwd unknown /proc/43230/task/44493/cwd (readlink: Permission denied) 2024-04-26T21:58:48.729 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 44493 ms_dispat ceph rtd unknown /proc/43230/task/44493/root (readlink: Permission denied) 2024-04-26T21:58:48.729 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 44493 ms_dispat ceph txt unknown /proc/43230/task/44493/exe (readlink: Permission denied) 2024-04-26T21:58:48.729 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 44493 ms_dispat ceph NOFD /proc/43230/task/44493/fd (opendir: Permission denied) 2024-04-26T21:58:48.729 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 44494 ms_local ceph cwd unknown /proc/43230/task/44494/cwd (readlink: Permission denied) 2024-04-26T21:58:48.729 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 44494 ms_local ceph rtd unknown /proc/43230/task/44494/root (readlink: Permission denied) 2024-04-26T21:58:48.729 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 44494 ms_local ceph txt unknown /proc/43230/task/44494/exe (readlink: Permission denied) 2024-04-26T21:58:48.729 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 44494 ms_local ceph NOFD /proc/43230/task/44494/fd (opendir: Permission denied) 2024-04-26T21:58:48.730 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 44495 ms_dispat ceph cwd unknown /proc/43230/task/44495/cwd (readlink: Permission denied) 2024-04-26T21:58:48.730 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 44495 ms_dispat ceph rtd unknown /proc/43230/task/44495/root (readlink: Permission denied) 2024-04-26T21:58:48.730 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 44495 ms_dispat ceph txt unknown /proc/43230/task/44495/exe (readlink: Permission denied) 2024-04-26T21:58:48.730 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 44495 ms_dispat ceph NOFD /proc/43230/task/44495/fd (opendir: Permission denied) 2024-04-26T21:58:48.730 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 44496 ms_local ceph cwd unknown /proc/43230/task/44496/cwd (readlink: Permission denied) 2024-04-26T21:58:48.730 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 44496 ms_local ceph rtd unknown /proc/43230/task/44496/root (readlink: Permission denied) 2024-04-26T21:58:48.730 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 44496 ms_local ceph txt unknown /proc/43230/task/44496/exe (readlink: Permission denied) 2024-04-26T21:58:48.730 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 44496 ms_local ceph NOFD /proc/43230/task/44496/fd (opendir: Permission denied) 2024-04-26T21:58:48.730 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 44497 fn_anonym ceph cwd unknown /proc/43230/task/44497/cwd (readlink: Permission denied) 2024-04-26T21:58:48.730 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 44497 fn_anonym ceph rtd unknown /proc/43230/task/44497/root (readlink: Permission denied) 2024-04-26T21:58:48.730 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 44497 fn_anonym ceph txt unknown /proc/43230/task/44497/exe (readlink: Permission denied) 2024-04-26T21:58:48.730 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 44497 fn_anonym ceph NOFD /proc/43230/task/44497/fd (opendir: Permission denied) 2024-04-26T21:58:48.730 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 44498 finisher ceph cwd unknown /proc/43230/task/44498/cwd (readlink: Permission denied) 2024-04-26T21:58:48.731 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 44498 finisher ceph rtd unknown /proc/43230/task/44498/root (readlink: Permission denied) 2024-04-26T21:58:48.731 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 44498 finisher ceph txt unknown /proc/43230/task/44498/exe (readlink: Permission denied) 2024-04-26T21:58:48.731 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 44498 finisher ceph NOFD /proc/43230/task/44498/fd (opendir: Permission denied) 2024-04-26T21:58:48.731 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 44499 safe_time ceph cwd unknown /proc/43230/task/44499/cwd (readlink: Permission denied) 2024-04-26T21:58:48.731 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 44499 safe_time ceph rtd unknown /proc/43230/task/44499/root (readlink: Permission denied) 2024-04-26T21:58:48.731 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 44499 safe_time ceph txt unknown /proc/43230/task/44499/exe (readlink: Permission denied) 2024-04-26T21:58:48.731 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 44499 safe_time ceph NOFD /proc/43230/task/44499/fd (opendir: Permission denied) 2024-04-26T21:58:48.731 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 44500 safe_time ceph cwd unknown /proc/43230/task/44500/cwd (readlink: Permission denied) 2024-04-26T21:58:48.731 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 44500 safe_time ceph rtd unknown /proc/43230/task/44500/root (readlink: Permission denied) 2024-04-26T21:58:48.731 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 44500 safe_time ceph txt unknown /proc/43230/task/44500/exe (readlink: Permission denied) 2024-04-26T21:58:48.731 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 44500 safe_time ceph NOFD /proc/43230/task/44500/fd (opendir: Permission denied) 2024-04-26T21:58:48.731 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 44501 ceph-osd ceph cwd unknown /proc/43230/task/44501/cwd (readlink: Permission denied) 2024-04-26T21:58:48.731 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 44501 ceph-osd ceph rtd unknown /proc/43230/task/44501/root (readlink: Permission denied) 2024-04-26T21:58:48.731 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 44501 ceph-osd ceph txt unknown /proc/43230/task/44501/exe (readlink: Permission denied) 2024-04-26T21:58:48.731 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 44501 ceph-osd ceph NOFD /proc/43230/task/44501/fd (opendir: Permission denied) 2024-04-26T21:58:48.732 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 44502 osd_srv_a ceph cwd unknown /proc/43230/task/44502/cwd (readlink: Permission denied) 2024-04-26T21:58:48.732 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 44502 osd_srv_a ceph rtd unknown /proc/43230/task/44502/root (readlink: Permission denied) 2024-04-26T21:58:48.732 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 44502 osd_srv_a ceph txt unknown /proc/43230/task/44502/exe (readlink: Permission denied) 2024-04-26T21:58:48.732 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 44502 osd_srv_a ceph NOFD /proc/43230/task/44502/fd (opendir: Permission denied) 2024-04-26T21:58:48.732 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 44503 tp_osd_tp ceph cwd unknown /proc/43230/task/44503/cwd (readlink: Permission denied) 2024-04-26T21:58:48.732 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 44503 tp_osd_tp ceph rtd unknown /proc/43230/task/44503/root (readlink: Permission denied) 2024-04-26T21:58:48.732 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 44503 tp_osd_tp ceph txt unknown /proc/43230/task/44503/exe (readlink: Permission denied) 2024-04-26T21:58:48.732 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 44503 tp_osd_tp ceph NOFD /proc/43230/task/44503/fd (opendir: Permission denied) 2024-04-26T21:58:48.732 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 44504 tp_osd_tp ceph cwd unknown /proc/43230/task/44504/cwd (readlink: Permission denied) 2024-04-26T21:58:48.732 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 44504 tp_osd_tp ceph rtd unknown /proc/43230/task/44504/root (readlink: Permission denied) 2024-04-26T21:58:48.732 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 44504 tp_osd_tp ceph txt unknown /proc/43230/task/44504/exe (readlink: Permission denied) 2024-04-26T21:58:48.732 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 44504 tp_osd_tp ceph NOFD /proc/43230/task/44504/fd (opendir: Permission denied) 2024-04-26T21:58:48.732 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 44505 tp_osd_tp ceph cwd unknown /proc/43230/task/44505/cwd (readlink: Permission denied) 2024-04-26T21:58:48.732 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 44505 tp_osd_tp ceph rtd unknown /proc/43230/task/44505/root (readlink: Permission denied) 2024-04-26T21:58:48.733 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 44505 tp_osd_tp ceph txt unknown /proc/43230/task/44505/exe (readlink: Permission denied) 2024-04-26T21:58:48.733 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 44505 tp_osd_tp ceph NOFD /proc/43230/task/44505/fd (opendir: Permission denied) 2024-04-26T21:58:48.733 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 44506 tp_osd_tp ceph cwd unknown /proc/43230/task/44506/cwd (readlink: Permission denied) 2024-04-26T21:58:48.733 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 44506 tp_osd_tp ceph rtd unknown /proc/43230/task/44506/root (readlink: Permission denied) 2024-04-26T21:58:48.733 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 44506 tp_osd_tp ceph txt unknown /proc/43230/task/44506/exe (readlink: Permission denied) 2024-04-26T21:58:48.733 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 44506 tp_osd_tp ceph NOFD /proc/43230/task/44506/fd (opendir: Permission denied) 2024-04-26T21:58:48.733 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 44507 tp_osd_tp ceph cwd unknown /proc/43230/task/44507/cwd (readlink: Permission denied) 2024-04-26T21:58:48.733 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 44507 tp_osd_tp ceph rtd unknown /proc/43230/task/44507/root (readlink: Permission denied) 2024-04-26T21:58:48.733 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 44507 tp_osd_tp ceph txt unknown /proc/43230/task/44507/exe (readlink: Permission denied) 2024-04-26T21:58:48.733 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 44507 tp_osd_tp ceph NOFD /proc/43230/task/44507/fd (opendir: Permission denied) 2024-04-26T21:58:48.733 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 44508 tp_osd_tp ceph cwd unknown /proc/43230/task/44508/cwd (readlink: Permission denied) 2024-04-26T21:58:48.733 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 44508 tp_osd_tp ceph rtd unknown /proc/43230/task/44508/root (readlink: Permission denied) 2024-04-26T21:58:48.733 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 44508 tp_osd_tp ceph txt unknown /proc/43230/task/44508/exe (readlink: Permission denied) 2024-04-26T21:58:48.733 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 44508 tp_osd_tp ceph NOFD /proc/43230/task/44508/fd (opendir: Permission denied) 2024-04-26T21:58:48.734 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 44509 tp_osd_tp ceph cwd unknown /proc/43230/task/44509/cwd (readlink: Permission denied) 2024-04-26T21:58:48.734 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 44509 tp_osd_tp ceph rtd unknown /proc/43230/task/44509/root (readlink: Permission denied) 2024-04-26T21:58:48.734 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 44509 tp_osd_tp ceph txt unknown /proc/43230/task/44509/exe (readlink: Permission denied) 2024-04-26T21:58:48.734 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 44509 tp_osd_tp ceph NOFD /proc/43230/task/44509/fd (opendir: Permission denied) 2024-04-26T21:58:48.734 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 44510 tp_osd_tp ceph cwd unknown /proc/43230/task/44510/cwd (readlink: Permission denied) 2024-04-26T21:58:48.734 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 44510 tp_osd_tp ceph rtd unknown /proc/43230/task/44510/root (readlink: Permission denied) 2024-04-26T21:58:48.734 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 44510 tp_osd_tp ceph txt unknown /proc/43230/task/44510/exe (readlink: Permission denied) 2024-04-26T21:58:48.734 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 44510 tp_osd_tp ceph NOFD /proc/43230/task/44510/fd (opendir: Permission denied) 2024-04-26T21:58:48.734 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 44511 tp_osd_tp ceph cwd unknown /proc/43230/task/44511/cwd (readlink: Permission denied) 2024-04-26T21:58:48.734 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 44511 tp_osd_tp ceph rtd unknown /proc/43230/task/44511/root (readlink: Permission denied) 2024-04-26T21:58:48.734 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 44511 tp_osd_tp ceph txt unknown /proc/43230/task/44511/exe (readlink: Permission denied) 2024-04-26T21:58:48.734 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 44511 tp_osd_tp ceph NOFD /proc/43230/task/44511/fd (opendir: Permission denied) 2024-04-26T21:58:48.734 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 44512 tp_osd_tp ceph cwd unknown /proc/43230/task/44512/cwd (readlink: Permission denied) 2024-04-26T21:58:48.735 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 44512 tp_osd_tp ceph rtd unknown /proc/43230/task/44512/root (readlink: Permission denied) 2024-04-26T21:58:48.735 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 44512 tp_osd_tp ceph txt unknown /proc/43230/task/44512/exe (readlink: Permission denied) 2024-04-26T21:58:48.735 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 44512 tp_osd_tp ceph NOFD /proc/43230/task/44512/fd (opendir: Permission denied) 2024-04-26T21:58:48.735 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 44513 tp_osd_tp ceph cwd unknown /proc/43230/task/44513/cwd (readlink: Permission denied) 2024-04-26T21:58:48.735 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 44513 tp_osd_tp ceph rtd unknown /proc/43230/task/44513/root (readlink: Permission denied) 2024-04-26T21:58:48.735 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 44513 tp_osd_tp ceph txt unknown /proc/43230/task/44513/exe (readlink: Permission denied) 2024-04-26T21:58:48.735 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 44513 tp_osd_tp ceph NOFD /proc/43230/task/44513/fd (opendir: Permission denied) 2024-04-26T21:58:48.735 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 44514 tp_osd_tp ceph cwd unknown /proc/43230/task/44514/cwd (readlink: Permission denied) 2024-04-26T21:58:48.735 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 44514 tp_osd_tp ceph rtd unknown /proc/43230/task/44514/root (readlink: Permission denied) 2024-04-26T21:58:48.735 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 44514 tp_osd_tp ceph txt unknown /proc/43230/task/44514/exe (readlink: Permission denied) 2024-04-26T21:58:48.735 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 44514 tp_osd_tp ceph NOFD /proc/43230/task/44514/fd (opendir: Permission denied) 2024-04-26T21:58:48.735 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 44515 tp_osd_tp ceph cwd unknown /proc/43230/task/44515/cwd (readlink: Permission denied) 2024-04-26T21:58:48.735 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 44515 tp_osd_tp ceph rtd unknown /proc/43230/task/44515/root (readlink: Permission denied) 2024-04-26T21:58:48.735 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 44515 tp_osd_tp ceph txt unknown /proc/43230/task/44515/exe (readlink: Permission denied) 2024-04-26T21:58:48.735 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 44515 tp_osd_tp ceph NOFD /proc/43230/task/44515/fd (opendir: Permission denied) 2024-04-26T21:58:48.735 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 44516 tp_osd_tp ceph cwd unknown /proc/43230/task/44516/cwd (readlink: Permission denied) 2024-04-26T21:58:48.736 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 44516 tp_osd_tp ceph rtd unknown /proc/43230/task/44516/root (readlink: Permission denied) 2024-04-26T21:58:48.736 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 44516 tp_osd_tp ceph txt unknown /proc/43230/task/44516/exe (readlink: Permission denied) 2024-04-26T21:58:48.736 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 44516 tp_osd_tp ceph NOFD /proc/43230/task/44516/fd (opendir: Permission denied) 2024-04-26T21:58:48.736 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 44517 tp_osd_tp ceph cwd unknown /proc/43230/task/44517/cwd (readlink: Permission denied) 2024-04-26T21:58:48.736 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 44517 tp_osd_tp ceph rtd unknown /proc/43230/task/44517/root (readlink: Permission denied) 2024-04-26T21:58:48.736 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 44517 tp_osd_tp ceph txt unknown /proc/43230/task/44517/exe (readlink: Permission denied) 2024-04-26T21:58:48.736 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 44517 tp_osd_tp ceph NOFD /proc/43230/task/44517/fd (opendir: Permission denied) 2024-04-26T21:58:48.736 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 44518 tp_osd_tp ceph cwd unknown /proc/43230/task/44518/cwd (readlink: Permission denied) 2024-04-26T21:58:48.736 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 44518 tp_osd_tp ceph rtd unknown /proc/43230/task/44518/root (readlink: Permission denied) 2024-04-26T21:58:48.736 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 44518 tp_osd_tp ceph txt unknown /proc/43230/task/44518/exe (readlink: Permission denied) 2024-04-26T21:58:48.736 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 44518 tp_osd_tp ceph NOFD /proc/43230/task/44518/fd (opendir: Permission denied) 2024-04-26T21:58:48.736 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 44519 osd_srv_h ceph cwd unknown /proc/43230/task/44519/cwd (readlink: Permission denied) 2024-04-26T21:58:48.736 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 44519 osd_srv_h ceph rtd unknown /proc/43230/task/44519/root (readlink: Permission denied) 2024-04-26T21:58:48.736 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 44519 osd_srv_h ceph txt unknown /proc/43230/task/44519/exe (readlink: Permission denied) 2024-04-26T21:58:48.737 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 43230 44519 osd_srv_h ceph NOFD /proc/43230/task/44519/fd (opendir: Permission denied) 2024-04-26T21:58:48.737 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/1 43821 root cwd unknown /proc/43821/cwd (readlink: Permission denied) 2024-04-26T21:58:48.737 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/1 43821 root rtd unknown /proc/43821/root (readlink: Permission denied) 2024-04-26T21:58:48.737 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/1 43821 root txt unknown /proc/43821/exe (readlink: Permission denied) 2024-04-26T21:58:48.737 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/1 43821 root NOFD /proc/43821/fd (opendir: Permission denied) 2024-04-26T21:58:48.737 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/0 43822 root cwd unknown /proc/43822/cwd (readlink: Permission denied) 2024-04-26T21:58:48.737 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/0 43822 root rtd unknown /proc/43822/root (readlink: Permission denied) 2024-04-26T21:58:48.737 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/0 43822 root txt unknown /proc/43822/exe (readlink: Permission denied) 2024-04-26T21:58:48.737 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/0 43822 root NOFD /proc/43822/fd (opendir: Permission denied) 2024-04-26T21:58:48.737 INFO:tasks.workunit.client.0.smithi072.stdout:sudo 44651 root cwd unknown /proc/44651/cwd (readlink: Permission denied) 2024-04-26T21:58:48.737 INFO:tasks.workunit.client.0.smithi072.stdout:sudo 44651 root rtd unknown /proc/44651/root (readlink: Permission denied) 2024-04-26T21:58:48.737 INFO:tasks.workunit.client.0.smithi072.stdout:sudo 44651 root txt unknown /proc/44651/exe (readlink: Permission denied) 2024-04-26T21:58:48.737 INFO:tasks.workunit.client.0.smithi072.stdout:sudo 44651 root NOFD /proc/44651/fd (opendir: Permission denied) 2024-04-26T21:58:48.737 INFO:tasks.workunit.client.0.smithi072.stdout:journalct 44679 root cwd unknown /proc/44679/cwd (readlink: Permission denied) 2024-04-26T21:58:48.737 INFO:tasks.workunit.client.0.smithi072.stdout:journalct 44679 root rtd unknown /proc/44679/root (readlink: Permission denied) 2024-04-26T21:58:48.738 INFO:tasks.workunit.client.0.smithi072.stdout:journalct 44679 root txt unknown /proc/44679/exe (readlink: Permission denied) 2024-04-26T21:58:48.738 INFO:tasks.workunit.client.0.smithi072.stdout:journalct 44679 root NOFD /proc/44679/fd (opendir: Permission denied) 2024-04-26T21:58:48.738 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/2 47106 root cwd unknown /proc/47106/cwd (readlink: Permission denied) 2024-04-26T21:58:48.738 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/2 47106 root rtd unknown /proc/47106/root (readlink: Permission denied) 2024-04-26T21:58:48.738 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/2 47106 root txt unknown /proc/47106/exe (readlink: Permission denied) 2024-04-26T21:58:48.738 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/2 47106 root NOFD /proc/47106/fd (opendir: Permission denied) 2024-04-26T21:58:48.738 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/5 49172 root cwd unknown /proc/49172/cwd (readlink: Permission denied) 2024-04-26T21:58:48.738 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/5 49172 root rtd unknown /proc/49172/root (readlink: Permission denied) 2024-04-26T21:58:48.738 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/5 49172 root txt unknown /proc/49172/exe (readlink: Permission denied) 2024-04-26T21:58:48.738 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/5 49172 root NOFD /proc/49172/fd (opendir: Permission denied) 2024-04-26T21:58:48.738 INFO:tasks.workunit.client.0.smithi072.stdout:conmon 50018 root cwd unknown /proc/50018/cwd (readlink: Permission denied) 2024-04-26T21:58:48.738 INFO:tasks.workunit.client.0.smithi072.stdout:conmon 50018 root rtd unknown /proc/50018/root (readlink: Permission denied) 2024-04-26T21:58:48.738 INFO:tasks.workunit.client.0.smithi072.stdout:conmon 50018 root txt unknown /proc/50018/exe (readlink: Permission denied) 2024-04-26T21:58:48.738 INFO:tasks.workunit.client.0.smithi072.stdout:conmon 50018 root NOFD /proc/50018/fd (opendir: Permission denied) 2024-04-26T21:58:48.739 INFO:tasks.workunit.client.0.smithi072.stdout:podman-in 50020 root cwd unknown /proc/50020/cwd (readlink: Permission denied) 2024-04-26T21:58:48.739 INFO:tasks.workunit.client.0.smithi072.stdout:podman-in 50020 root rtd unknown /proc/50020/root (readlink: Permission denied) 2024-04-26T21:58:48.739 INFO:tasks.workunit.client.0.smithi072.stdout:podman-in 50020 root txt unknown /proc/50020/exe (readlink: Permission denied) 2024-04-26T21:58:48.739 INFO:tasks.workunit.client.0.smithi072.stdout:podman-in 50020 root NOFD /proc/50020/fd (opendir: Permission denied) 2024-04-26T21:58:48.739 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 ceph cwd unknown /proc/50022/cwd (readlink: Permission denied) 2024-04-26T21:58:48.739 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 ceph rtd unknown /proc/50022/root (readlink: Permission denied) 2024-04-26T21:58:48.739 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 ceph txt unknown /proc/50022/exe (readlink: Permission denied) 2024-04-26T21:58:48.739 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 ceph NOFD /proc/50022/fd (opendir: Permission denied) 2024-04-26T21:58:48.739 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 50023 log ceph cwd unknown /proc/50022/task/50023/cwd (readlink: Permission denied) 2024-04-26T21:58:48.739 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 50023 log ceph rtd unknown /proc/50022/task/50023/root (readlink: Permission denied) 2024-04-26T21:58:48.739 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 50023 log ceph txt unknown /proc/50022/task/50023/exe (readlink: Permission denied) 2024-04-26T21:58:48.739 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 50023 log ceph NOFD /proc/50022/task/50023/fd (opendir: Permission denied) 2024-04-26T21:58:48.739 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 50025 msgr-work ceph cwd unknown /proc/50022/task/50025/cwd (readlink: Permission denied) 2024-04-26T21:58:48.739 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 50025 msgr-work ceph rtd unknown /proc/50022/task/50025/root (readlink: Permission denied) 2024-04-26T21:58:48.739 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 50025 msgr-work ceph txt unknown /proc/50022/task/50025/exe (readlink: Permission denied) 2024-04-26T21:58:48.740 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 50025 msgr-work ceph NOFD /proc/50022/task/50025/fd (opendir: Permission denied) 2024-04-26T21:58:48.740 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 50026 msgr-work ceph cwd unknown /proc/50022/task/50026/cwd (readlink: Permission denied) 2024-04-26T21:58:48.740 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 50026 msgr-work ceph rtd unknown /proc/50022/task/50026/root (readlink: Permission denied) 2024-04-26T21:58:48.740 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 50026 msgr-work ceph txt unknown /proc/50022/task/50026/exe (readlink: Permission denied) 2024-04-26T21:58:48.740 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 50026 msgr-work ceph NOFD /proc/50022/task/50026/fd (opendir: Permission denied) 2024-04-26T21:58:48.740 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 50027 msgr-work ceph cwd unknown /proc/50022/task/50027/cwd (readlink: Permission denied) 2024-04-26T21:58:48.740 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 50027 msgr-work ceph rtd unknown /proc/50022/task/50027/root (readlink: Permission denied) 2024-04-26T21:58:48.740 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 50027 msgr-work ceph txt unknown /proc/50022/task/50027/exe (readlink: Permission denied) 2024-04-26T21:58:48.740 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 50027 msgr-work ceph NOFD /proc/50022/task/50027/fd (opendir: Permission denied) 2024-04-26T21:58:48.740 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 50031 service ceph cwd unknown /proc/50022/task/50031/cwd (readlink: Permission denied) 2024-04-26T21:58:48.740 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 50031 service ceph rtd unknown /proc/50022/task/50031/root (readlink: Permission denied) 2024-04-26T21:58:48.740 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 50031 service ceph txt unknown /proc/50022/task/50031/exe (readlink: Permission denied) 2024-04-26T21:58:48.740 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 50031 service ceph NOFD /proc/50022/task/50031/fd (opendir: Permission denied) 2024-04-26T21:58:48.740 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 50032 admin_soc ceph cwd unknown /proc/50022/task/50032/cwd (readlink: Permission denied) 2024-04-26T21:58:48.741 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 50032 admin_soc ceph rtd unknown /proc/50022/task/50032/root (readlink: Permission denied) 2024-04-26T21:58:48.741 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 50032 admin_soc ceph txt unknown /proc/50022/task/50032/exe (readlink: Permission denied) 2024-04-26T21:58:48.741 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 50032 admin_soc ceph NOFD /proc/50022/task/50032/fd (opendir: Permission denied) 2024-04-26T21:58:48.741 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 50177 signal_ha ceph cwd unknown /proc/50022/task/50177/cwd (readlink: Permission denied) 2024-04-26T21:58:48.741 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 50177 signal_ha ceph rtd unknown /proc/50022/task/50177/root (readlink: Permission denied) 2024-04-26T21:58:48.741 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 50177 signal_ha ceph txt unknown /proc/50022/task/50177/exe (readlink: Permission denied) 2024-04-26T21:58:48.741 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 50177 signal_ha ceph NOFD /proc/50022/task/50177/fd (opendir: Permission denied) 2024-04-26T21:58:48.741 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 50178 io_contex ceph cwd unknown /proc/50022/task/50178/cwd (readlink: Permission denied) 2024-04-26T21:58:48.741 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 50178 io_contex ceph rtd unknown /proc/50022/task/50178/root (readlink: Permission denied) 2024-04-26T21:58:48.741 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 50178 io_contex ceph txt unknown /proc/50022/task/50178/exe (readlink: Permission denied) 2024-04-26T21:58:48.741 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 50178 io_contex ceph NOFD /proc/50022/task/50178/fd (opendir: Permission denied) 2024-04-26T21:58:48.741 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 50179 io_contex ceph cwd unknown /proc/50022/task/50179/cwd (readlink: Permission denied) 2024-04-26T21:58:48.741 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 50179 io_contex ceph rtd unknown /proc/50022/task/50179/root (readlink: Permission denied) 2024-04-26T21:58:48.741 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 50179 io_contex ceph txt unknown /proc/50022/task/50179/exe (readlink: Permission denied) 2024-04-26T21:58:48.742 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 50179 io_contex ceph NOFD /proc/50022/task/50179/fd (opendir: Permission denied) 2024-04-26T21:58:48.742 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 50201 OpHistory ceph cwd unknown /proc/50022/task/50201/cwd (readlink: Permission denied) 2024-04-26T21:58:48.742 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 50201 OpHistory ceph rtd unknown /proc/50022/task/50201/root (readlink: Permission denied) 2024-04-26T21:58:48.742 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 50201 OpHistory ceph txt unknown /proc/50022/task/50201/exe (readlink: Permission denied) 2024-04-26T21:58:48.742 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 50201 OpHistory ceph NOFD /proc/50022/task/50201/fd (opendir: Permission denied) 2024-04-26T21:58:48.742 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 50203 ceph_time ceph cwd unknown /proc/50022/task/50203/cwd (readlink: Permission denied) 2024-04-26T21:58:48.742 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 50203 ceph_time ceph rtd unknown /proc/50022/task/50203/root (readlink: Permission denied) 2024-04-26T21:58:48.742 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 50203 ceph_time ceph txt unknown /proc/50022/task/50203/exe (readlink: Permission denied) 2024-04-26T21:58:48.742 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 50203 ceph_time ceph NOFD /proc/50022/task/50203/fd (opendir: Permission denied) 2024-04-26T21:58:48.742 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 50400 ceph-osd ceph cwd unknown /proc/50022/task/50400/cwd (readlink: Permission denied) 2024-04-26T21:58:48.742 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 50400 ceph-osd ceph rtd unknown /proc/50022/task/50400/root (readlink: Permission denied) 2024-04-26T21:58:48.742 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 50400 ceph-osd ceph txt unknown /proc/50022/task/50400/exe (readlink: Permission denied) 2024-04-26T21:58:48.742 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 50400 ceph-osd ceph NOFD /proc/50022/task/50400/fd (opendir: Permission denied) 2024-04-26T21:58:48.742 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 50401 safe_time ceph cwd unknown /proc/50022/task/50401/cwd (readlink: Permission denied) 2024-04-26T21:58:48.743 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 50401 safe_time ceph rtd unknown /proc/50022/task/50401/root (readlink: Permission denied) 2024-04-26T21:58:48.743 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 50401 safe_time ceph txt unknown /proc/50022/task/50401/exe (readlink: Permission denied) 2024-04-26T21:58:48.743 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 50401 safe_time ceph NOFD /proc/50022/task/50401/fd (opendir: Permission denied) 2024-04-26T21:58:48.743 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 50402 safe_time ceph cwd unknown /proc/50022/task/50402/cwd (readlink: Permission denied) 2024-04-26T21:58:48.743 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 50402 safe_time ceph rtd unknown /proc/50022/task/50402/root (readlink: Permission denied) 2024-04-26T21:58:48.743 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 50402 safe_time ceph txt unknown /proc/50022/task/50402/exe (readlink: Permission denied) 2024-04-26T21:58:48.743 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 50402 safe_time ceph NOFD /proc/50022/task/50402/fd (opendir: Permission denied) 2024-04-26T21:58:48.743 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 50403 safe_time ceph cwd unknown /proc/50022/task/50403/cwd (readlink: Permission denied) 2024-04-26T21:58:48.743 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 50403 safe_time ceph rtd unknown /proc/50022/task/50403/root (readlink: Permission denied) 2024-04-26T21:58:48.743 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 50403 safe_time ceph txt unknown /proc/50022/task/50403/exe (readlink: Permission denied) 2024-04-26T21:58:48.743 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 50403 safe_time ceph NOFD /proc/50022/task/50403/fd (opendir: Permission denied) 2024-04-26T21:58:48.743 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 50404 safe_time ceph cwd unknown /proc/50022/task/50404/cwd (readlink: Permission denied) 2024-04-26T21:58:48.743 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 50404 safe_time ceph rtd unknown /proc/50022/task/50404/root (readlink: Permission denied) 2024-04-26T21:58:48.744 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 50404 safe_time ceph txt unknown /proc/50022/task/50404/exe (readlink: Permission denied) 2024-04-26T21:58:48.744 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 50404 safe_time ceph NOFD /proc/50022/task/50404/fd (opendir: Permission denied) 2024-04-26T21:58:48.744 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 50405 fn_anonym ceph cwd unknown /proc/50022/task/50405/cwd (readlink: Permission denied) 2024-04-26T21:58:48.744 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 50405 fn_anonym ceph rtd unknown /proc/50022/task/50405/root (readlink: Permission denied) 2024-04-26T21:58:48.744 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 50405 fn_anonym ceph txt unknown /proc/50022/task/50405/exe (readlink: Permission denied) 2024-04-26T21:58:48.744 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 50405 fn_anonym ceph NOFD /proc/50022/task/50405/fd (opendir: Permission denied) 2024-04-26T21:58:48.744 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 50410 rocksdb:l ceph cwd unknown /proc/50022/task/50410/cwd (readlink: Permission denied) 2024-04-26T21:58:48.744 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 50410 rocksdb:l ceph rtd unknown /proc/50022/task/50410/root (readlink: Permission denied) 2024-04-26T21:58:48.744 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 50410 rocksdb:l ceph txt unknown /proc/50022/task/50410/exe (readlink: Permission denied) 2024-04-26T21:58:48.744 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 50410 rocksdb:l ceph NOFD /proc/50022/task/50410/fd (opendir: Permission denied) 2024-04-26T21:58:48.744 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 50411 rocksdb:l ceph cwd unknown /proc/50022/task/50411/cwd (readlink: Permission denied) 2024-04-26T21:58:48.744 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 50411 rocksdb:l ceph rtd unknown /proc/50022/task/50411/root (readlink: Permission denied) 2024-04-26T21:58:48.744 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 50411 rocksdb:l ceph txt unknown /proc/50022/task/50411/exe (readlink: Permission denied) 2024-04-26T21:58:48.745 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 50411 rocksdb:l ceph NOFD /proc/50022/task/50411/fd (opendir: Permission denied) 2024-04-26T21:58:48.745 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 50412 rocksdb:l ceph cwd unknown /proc/50022/task/50412/cwd (readlink: Permission denied) 2024-04-26T21:58:48.745 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 50412 rocksdb:l ceph rtd unknown /proc/50022/task/50412/root (readlink: Permission denied) 2024-04-26T21:58:48.745 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 50412 rocksdb:l ceph txt unknown /proc/50022/task/50412/exe (readlink: Permission denied) 2024-04-26T21:58:48.745 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 50412 rocksdb:l ceph NOFD /proc/50022/task/50412/fd (opendir: Permission denied) 2024-04-26T21:58:48.745 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 50413 rocksdb:h ceph cwd unknown /proc/50022/task/50413/cwd (readlink: Permission denied) 2024-04-26T21:58:48.745 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 50413 rocksdb:h ceph rtd unknown /proc/50022/task/50413/root (readlink: Permission denied) 2024-04-26T21:58:48.745 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 50413 rocksdb:h ceph txt unknown /proc/50022/task/50413/exe (readlink: Permission denied) 2024-04-26T21:58:48.745 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 50413 rocksdb:h ceph NOFD /proc/50022/task/50413/fd (opendir: Permission denied) 2024-04-26T21:58:48.745 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 50844 bstore_ai ceph cwd unknown /proc/50022/task/50844/cwd (readlink: Permission denied) 2024-04-26T21:58:48.745 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 50844 bstore_ai ceph rtd unknown /proc/50022/task/50844/root (readlink: Permission denied) 2024-04-26T21:58:48.745 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 50844 bstore_ai ceph txt unknown /proc/50022/task/50844/exe (readlink: Permission denied) 2024-04-26T21:58:48.745 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 50844 bstore_ai ceph NOFD /proc/50022/task/50844/fd (opendir: Permission denied) 2024-04-26T21:58:48.745 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 51096 bstore_ai ceph cwd unknown /proc/50022/task/51096/cwd (readlink: Permission denied) 2024-04-26T21:58:48.745 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 51096 bstore_ai ceph rtd unknown /proc/50022/task/51096/root (readlink: Permission denied) 2024-04-26T21:58:48.746 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 51096 bstore_ai ceph txt unknown /proc/50022/task/51096/exe (readlink: Permission denied) 2024-04-26T21:58:48.746 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 51096 bstore_ai ceph NOFD /proc/50022/task/51096/fd (opendir: Permission denied) 2024-04-26T21:58:48.746 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 51280 ceph-osd ceph cwd unknown /proc/50022/task/51280/cwd (readlink: Permission denied) 2024-04-26T21:58:48.746 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 51280 ceph-osd ceph rtd unknown /proc/50022/task/51280/root (readlink: Permission denied) 2024-04-26T21:58:48.746 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 51280 ceph-osd ceph txt unknown /proc/50022/task/51280/exe (readlink: Permission denied) 2024-04-26T21:58:48.746 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 51280 ceph-osd ceph NOFD /proc/50022/task/51280/fd (opendir: Permission denied) 2024-04-26T21:58:48.746 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 51281 cfin ceph cwd unknown /proc/50022/task/51281/cwd (readlink: Permission denied) 2024-04-26T21:58:48.746 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 51281 cfin ceph rtd unknown /proc/50022/task/51281/root (readlink: Permission denied) 2024-04-26T21:58:48.746 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 51281 cfin ceph txt unknown /proc/50022/task/51281/exe (readlink: Permission denied) 2024-04-26T21:58:48.746 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 51281 cfin ceph NOFD /proc/50022/task/51281/fd (opendir: Permission denied) 2024-04-26T21:58:48.746 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 51282 bstore_kv ceph cwd unknown /proc/50022/task/51282/cwd (readlink: Permission denied) 2024-04-26T21:58:48.746 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 51282 bstore_kv ceph rtd unknown /proc/50022/task/51282/root (readlink: Permission denied) 2024-04-26T21:58:48.746 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 51282 bstore_kv ceph txt unknown /proc/50022/task/51282/exe (readlink: Permission denied) 2024-04-26T21:58:48.746 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 51282 bstore_kv ceph NOFD /proc/50022/task/51282/fd (opendir: Permission denied) 2024-04-26T21:58:48.747 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 51283 bstore_kv ceph cwd unknown /proc/50022/task/51283/cwd (readlink: Permission denied) 2024-04-26T21:58:48.747 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 51283 bstore_kv ceph rtd unknown /proc/50022/task/51283/root (readlink: Permission denied) 2024-04-26T21:58:48.747 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 51283 bstore_kv ceph txt unknown /proc/50022/task/51283/exe (readlink: Permission denied) 2024-04-26T21:58:48.747 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 51283 bstore_kv ceph NOFD /proc/50022/task/51283/fd (opendir: Permission denied) 2024-04-26T21:58:48.747 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 51284 bstore_me ceph cwd unknown /proc/50022/task/51284/cwd (readlink: Permission denied) 2024-04-26T21:58:48.747 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 51284 bstore_me ceph rtd unknown /proc/50022/task/51284/root (readlink: Permission denied) 2024-04-26T21:58:48.747 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 51284 bstore_me ceph txt unknown /proc/50022/task/51284/exe (readlink: Permission denied) 2024-04-26T21:58:48.747 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 51284 bstore_me ceph NOFD /proc/50022/task/51284/fd (opendir: Permission denied) 2024-04-26T21:58:48.747 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 51285 ms_dispat ceph cwd unknown /proc/50022/task/51285/cwd (readlink: Permission denied) 2024-04-26T21:58:48.747 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 51285 ms_dispat ceph rtd unknown /proc/50022/task/51285/root (readlink: Permission denied) 2024-04-26T21:58:48.747 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 51285 ms_dispat ceph txt unknown /proc/50022/task/51285/exe (readlink: Permission denied) 2024-04-26T21:58:48.747 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 51285 ms_dispat ceph NOFD /proc/50022/task/51285/fd (opendir: Permission denied) 2024-04-26T21:58:48.747 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 51286 ms_local ceph cwd unknown /proc/50022/task/51286/cwd (readlink: Permission denied) 2024-04-26T21:58:48.747 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 51286 ms_local ceph rtd unknown /proc/50022/task/51286/root (readlink: Permission denied) 2024-04-26T21:58:48.748 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 51286 ms_local ceph txt unknown /proc/50022/task/51286/exe (readlink: Permission denied) 2024-04-26T21:58:48.748 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 51286 ms_local ceph NOFD /proc/50022/task/51286/fd (opendir: Permission denied) 2024-04-26T21:58:48.748 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 51287 safe_time ceph cwd unknown /proc/50022/task/51287/cwd (readlink: Permission denied) 2024-04-26T21:58:48.748 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 51287 safe_time ceph rtd unknown /proc/50022/task/51287/root (readlink: Permission denied) 2024-04-26T21:58:48.748 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 51287 safe_time ceph txt unknown /proc/50022/task/51287/exe (readlink: Permission denied) 2024-04-26T21:58:48.748 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 51287 safe_time ceph NOFD /proc/50022/task/51287/fd (opendir: Permission denied) 2024-04-26T21:58:48.748 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 51288 safe_time ceph cwd unknown /proc/50022/task/51288/cwd (readlink: Permission denied) 2024-04-26T21:58:48.748 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 51288 safe_time ceph rtd unknown /proc/50022/task/51288/root (readlink: Permission denied) 2024-04-26T21:58:48.748 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 51288 safe_time ceph txt unknown /proc/50022/task/51288/exe (readlink: Permission denied) 2024-04-26T21:58:48.748 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 51288 safe_time ceph NOFD /proc/50022/task/51288/fd (opendir: Permission denied) 2024-04-26T21:58:48.748 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 51289 ms_dispat ceph cwd unknown /proc/50022/task/51289/cwd (readlink: Permission denied) 2024-04-26T21:58:48.748 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 51289 ms_dispat ceph rtd unknown /proc/50022/task/51289/root (readlink: Permission denied) 2024-04-26T21:58:48.748 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 51289 ms_dispat ceph txt unknown /proc/50022/task/51289/exe (readlink: Permission denied) 2024-04-26T21:58:48.749 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 51289 ms_dispat ceph NOFD /proc/50022/task/51289/fd (opendir: Permission denied) 2024-04-26T21:58:48.749 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 51290 ms_local ceph cwd unknown /proc/50022/task/51290/cwd (readlink: Permission denied) 2024-04-26T21:58:48.749 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 51290 ms_local ceph rtd unknown /proc/50022/task/51290/root (readlink: Permission denied) 2024-04-26T21:58:48.749 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 51290 ms_local ceph txt unknown /proc/50022/task/51290/exe (readlink: Permission denied) 2024-04-26T21:58:48.749 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 51290 ms_local ceph NOFD /proc/50022/task/51290/fd (opendir: Permission denied) 2024-04-26T21:58:48.749 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 51291 ms_dispat ceph cwd unknown /proc/50022/task/51291/cwd (readlink: Permission denied) 2024-04-26T21:58:48.749 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 51291 ms_dispat ceph rtd unknown /proc/50022/task/51291/root (readlink: Permission denied) 2024-04-26T21:58:48.749 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 51291 ms_dispat ceph txt unknown /proc/50022/task/51291/exe (readlink: Permission denied) 2024-04-26T21:58:48.749 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 51291 ms_dispat ceph NOFD /proc/50022/task/51291/fd (opendir: Permission denied) 2024-04-26T21:58:48.749 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 51292 ms_local ceph cwd unknown /proc/50022/task/51292/cwd (readlink: Permission denied) 2024-04-26T21:58:48.749 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 51292 ms_local ceph rtd unknown /proc/50022/task/51292/root (readlink: Permission denied) 2024-04-26T21:58:48.750 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 51292 ms_local ceph txt unknown /proc/50022/task/51292/exe (readlink: Permission denied) 2024-04-26T21:58:48.750 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 51292 ms_local ceph NOFD /proc/50022/task/51292/fd (opendir: Permission denied) 2024-04-26T21:58:48.750 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 51293 ms_dispat ceph cwd unknown /proc/50022/task/51293/cwd (readlink: Permission denied) 2024-04-26T21:58:48.750 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 51293 ms_dispat ceph rtd unknown /proc/50022/task/51293/root (readlink: Permission denied) 2024-04-26T21:58:48.750 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 51293 ms_dispat ceph txt unknown /proc/50022/task/51293/exe (readlink: Permission denied) 2024-04-26T21:58:48.750 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 51293 ms_dispat ceph NOFD /proc/50022/task/51293/fd (opendir: Permission denied) 2024-04-26T21:58:48.750 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 51294 ms_local ceph cwd unknown /proc/50022/task/51294/cwd (readlink: Permission denied) 2024-04-26T21:58:48.750 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 51294 ms_local ceph rtd unknown /proc/50022/task/51294/root (readlink: Permission denied) 2024-04-26T21:58:48.750 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 51294 ms_local ceph txt unknown /proc/50022/task/51294/exe (readlink: Permission denied) 2024-04-26T21:58:48.750 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 51294 ms_local ceph NOFD /proc/50022/task/51294/fd (opendir: Permission denied) 2024-04-26T21:58:48.750 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 51295 ms_dispat ceph cwd unknown /proc/50022/task/51295/cwd (readlink: Permission denied) 2024-04-26T21:58:48.750 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 51295 ms_dispat ceph rtd unknown /proc/50022/task/51295/root (readlink: Permission denied) 2024-04-26T21:58:48.750 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 51295 ms_dispat ceph txt unknown /proc/50022/task/51295/exe (readlink: Permission denied) 2024-04-26T21:58:48.750 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 51295 ms_dispat ceph NOFD /proc/50022/task/51295/fd (opendir: Permission denied) 2024-04-26T21:58:48.751 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 51296 ms_local ceph cwd unknown /proc/50022/task/51296/cwd (readlink: Permission denied) 2024-04-26T21:58:48.751 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 51296 ms_local ceph rtd unknown /proc/50022/task/51296/root (readlink: Permission denied) 2024-04-26T21:58:48.751 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 51296 ms_local ceph txt unknown /proc/50022/task/51296/exe (readlink: Permission denied) 2024-04-26T21:58:48.751 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 51296 ms_local ceph NOFD /proc/50022/task/51296/fd (opendir: Permission denied) 2024-04-26T21:58:48.751 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 51297 ms_dispat ceph cwd unknown /proc/50022/task/51297/cwd (readlink: Permission denied) 2024-04-26T21:58:48.751 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 51297 ms_dispat ceph rtd unknown /proc/50022/task/51297/root (readlink: Permission denied) 2024-04-26T21:58:48.751 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 51297 ms_dispat ceph txt unknown /proc/50022/task/51297/exe (readlink: Permission denied) 2024-04-26T21:58:48.751 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 51297 ms_dispat ceph NOFD /proc/50022/task/51297/fd (opendir: Permission denied) 2024-04-26T21:58:48.751 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 51298 ms_local ceph cwd unknown /proc/50022/task/51298/cwd (readlink: Permission denied) 2024-04-26T21:58:48.751 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 51298 ms_local ceph rtd unknown /proc/50022/task/51298/root (readlink: Permission denied) 2024-04-26T21:58:48.752 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 51298 ms_local ceph txt unknown /proc/50022/task/51298/exe (readlink: Permission denied) 2024-04-26T21:58:48.752 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 51298 ms_local ceph NOFD /proc/50022/task/51298/fd (opendir: Permission denied) 2024-04-26T21:58:48.752 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 51299 ms_dispat ceph cwd unknown /proc/50022/task/51299/cwd (readlink: Permission denied) 2024-04-26T21:58:48.752 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 51299 ms_dispat ceph rtd unknown /proc/50022/task/51299/root (readlink: Permission denied) 2024-04-26T21:58:48.752 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 51299 ms_dispat ceph txt unknown /proc/50022/task/51299/exe (readlink: Permission denied) 2024-04-26T21:58:48.752 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 51299 ms_dispat ceph NOFD /proc/50022/task/51299/fd (opendir: Permission denied) 2024-04-26T21:58:48.752 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 51300 ms_local ceph cwd unknown /proc/50022/task/51300/cwd (readlink: Permission denied) 2024-04-26T21:58:48.752 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 51300 ms_local ceph rtd unknown /proc/50022/task/51300/root (readlink: Permission denied) 2024-04-26T21:58:48.752 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 51300 ms_local ceph txt unknown /proc/50022/task/51300/exe (readlink: Permission denied) 2024-04-26T21:58:48.752 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 51300 ms_local ceph NOFD /proc/50022/task/51300/fd (opendir: Permission denied) 2024-04-26T21:58:48.752 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 51301 fn_anonym ceph cwd unknown /proc/50022/task/51301/cwd (readlink: Permission denied) 2024-04-26T21:58:48.752 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 51301 fn_anonym ceph rtd unknown /proc/50022/task/51301/root (readlink: Permission denied) 2024-04-26T21:58:48.752 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 51301 fn_anonym ceph txt unknown /proc/50022/task/51301/exe (readlink: Permission denied) 2024-04-26T21:58:48.752 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 51301 fn_anonym ceph NOFD /proc/50022/task/51301/fd (opendir: Permission denied) 2024-04-26T21:58:48.753 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 51302 finisher ceph cwd unknown /proc/50022/task/51302/cwd (readlink: Permission denied) 2024-04-26T21:58:48.753 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 51302 finisher ceph rtd unknown /proc/50022/task/51302/root (readlink: Permission denied) 2024-04-26T21:58:48.753 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 51302 finisher ceph txt unknown /proc/50022/task/51302/exe (readlink: Permission denied) 2024-04-26T21:58:48.753 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 51302 finisher ceph NOFD /proc/50022/task/51302/fd (opendir: Permission denied) 2024-04-26T21:58:48.753 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 51303 safe_time ceph cwd unknown /proc/50022/task/51303/cwd (readlink: Permission denied) 2024-04-26T21:58:48.753 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 51303 safe_time ceph rtd unknown /proc/50022/task/51303/root (readlink: Permission denied) 2024-04-26T21:58:48.753 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 51303 safe_time ceph txt unknown /proc/50022/task/51303/exe (readlink: Permission denied) 2024-04-26T21:58:48.753 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 51303 safe_time ceph NOFD /proc/50022/task/51303/fd (opendir: Permission denied) 2024-04-26T21:58:48.753 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 51304 safe_time ceph cwd unknown /proc/50022/task/51304/cwd (readlink: Permission denied) 2024-04-26T21:58:48.753 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 51304 safe_time ceph rtd unknown /proc/50022/task/51304/root (readlink: Permission denied) 2024-04-26T21:58:48.753 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 51304 safe_time ceph txt unknown /proc/50022/task/51304/exe (readlink: Permission denied) 2024-04-26T21:58:48.754 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 51304 safe_time ceph NOFD /proc/50022/task/51304/fd (opendir: Permission denied) 2024-04-26T21:58:48.754 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 51305 ceph-osd ceph cwd unknown /proc/50022/task/51305/cwd (readlink: Permission denied) 2024-04-26T21:58:48.754 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 51305 ceph-osd ceph rtd unknown /proc/50022/task/51305/root (readlink: Permission denied) 2024-04-26T21:58:48.754 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 51305 ceph-osd ceph txt unknown /proc/50022/task/51305/exe (readlink: Permission denied) 2024-04-26T21:58:48.754 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 51305 ceph-osd ceph NOFD /proc/50022/task/51305/fd (opendir: Permission denied) 2024-04-26T21:58:48.754 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 51306 osd_srv_a ceph cwd unknown /proc/50022/task/51306/cwd (readlink: Permission denied) 2024-04-26T21:58:48.754 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 51306 osd_srv_a ceph rtd unknown /proc/50022/task/51306/root (readlink: Permission denied) 2024-04-26T21:58:48.754 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 51306 osd_srv_a ceph txt unknown /proc/50022/task/51306/exe (readlink: Permission denied) 2024-04-26T21:58:48.754 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 51306 osd_srv_a ceph NOFD /proc/50022/task/51306/fd (opendir: Permission denied) 2024-04-26T21:58:48.754 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 51307 tp_osd_tp ceph cwd unknown /proc/50022/task/51307/cwd (readlink: Permission denied) 2024-04-26T21:58:48.754 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 51307 tp_osd_tp ceph rtd unknown /proc/50022/task/51307/root (readlink: Permission denied) 2024-04-26T21:58:48.754 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 51307 tp_osd_tp ceph txt unknown /proc/50022/task/51307/exe (readlink: Permission denied) 2024-04-26T21:58:48.754 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 51307 tp_osd_tp ceph NOFD /proc/50022/task/51307/fd (opendir: Permission denied) 2024-04-26T21:58:48.755 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 51308 tp_osd_tp ceph cwd unknown /proc/50022/task/51308/cwd (readlink: Permission denied) 2024-04-26T21:58:48.755 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 51308 tp_osd_tp ceph rtd unknown /proc/50022/task/51308/root (readlink: Permission denied) 2024-04-26T21:58:48.755 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 51308 tp_osd_tp ceph txt unknown /proc/50022/task/51308/exe (readlink: Permission denied) 2024-04-26T21:58:48.755 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 51308 tp_osd_tp ceph NOFD /proc/50022/task/51308/fd (opendir: Permission denied) 2024-04-26T21:58:48.755 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 51309 tp_osd_tp ceph cwd unknown /proc/50022/task/51309/cwd (readlink: Permission denied) 2024-04-26T21:58:48.755 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 51309 tp_osd_tp ceph rtd unknown /proc/50022/task/51309/root (readlink: Permission denied) 2024-04-26T21:58:48.755 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 51309 tp_osd_tp ceph txt unknown /proc/50022/task/51309/exe (readlink: Permission denied) 2024-04-26T21:58:48.755 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 51309 tp_osd_tp ceph NOFD /proc/50022/task/51309/fd (opendir: Permission denied) 2024-04-26T21:58:48.755 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 51310 tp_osd_tp ceph cwd unknown /proc/50022/task/51310/cwd (readlink: Permission denied) 2024-04-26T21:58:48.755 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 51310 tp_osd_tp ceph rtd unknown /proc/50022/task/51310/root (readlink: Permission denied) 2024-04-26T21:58:48.755 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 51310 tp_osd_tp ceph txt unknown /proc/50022/task/51310/exe (readlink: Permission denied) 2024-04-26T21:58:48.756 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 51310 tp_osd_tp ceph NOFD /proc/50022/task/51310/fd (opendir: Permission denied) 2024-04-26T21:58:48.756 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 51311 tp_osd_tp ceph cwd unknown /proc/50022/task/51311/cwd (readlink: Permission denied) 2024-04-26T21:58:48.756 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 51311 tp_osd_tp ceph rtd unknown /proc/50022/task/51311/root (readlink: Permission denied) 2024-04-26T21:58:48.756 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 51311 tp_osd_tp ceph txt unknown /proc/50022/task/51311/exe (readlink: Permission denied) 2024-04-26T21:58:48.756 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 51311 tp_osd_tp ceph NOFD /proc/50022/task/51311/fd (opendir: Permission denied) 2024-04-26T21:58:48.756 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 51312 tp_osd_tp ceph cwd unknown /proc/50022/task/51312/cwd (readlink: Permission denied) 2024-04-26T21:58:48.756 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 51312 tp_osd_tp ceph rtd unknown /proc/50022/task/51312/root (readlink: Permission denied) 2024-04-26T21:58:48.756 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 51312 tp_osd_tp ceph txt unknown /proc/50022/task/51312/exe (readlink: Permission denied) 2024-04-26T21:58:48.756 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 51312 tp_osd_tp ceph NOFD /proc/50022/task/51312/fd (opendir: Permission denied) 2024-04-26T21:58:48.756 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 51313 tp_osd_tp ceph cwd unknown /proc/50022/task/51313/cwd (readlink: Permission denied) 2024-04-26T21:58:48.756 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 51313 tp_osd_tp ceph rtd unknown /proc/50022/task/51313/root (readlink: Permission denied) 2024-04-26T21:58:48.757 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 51313 tp_osd_tp ceph txt unknown /proc/50022/task/51313/exe (readlink: Permission denied) 2024-04-26T21:58:48.757 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 51313 tp_osd_tp ceph NOFD /proc/50022/task/51313/fd (opendir: Permission denied) 2024-04-26T21:58:48.757 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 51314 tp_osd_tp ceph cwd unknown /proc/50022/task/51314/cwd (readlink: Permission denied) 2024-04-26T21:58:48.757 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 51314 tp_osd_tp ceph rtd unknown /proc/50022/task/51314/root (readlink: Permission denied) 2024-04-26T21:58:48.757 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 51314 tp_osd_tp ceph txt unknown /proc/50022/task/51314/exe (readlink: Permission denied) 2024-04-26T21:58:48.757 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 51314 tp_osd_tp ceph NOFD /proc/50022/task/51314/fd (opendir: Permission denied) 2024-04-26T21:58:48.757 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 51315 tp_osd_tp ceph cwd unknown /proc/50022/task/51315/cwd (readlink: Permission denied) 2024-04-26T21:58:48.757 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 51315 tp_osd_tp ceph rtd unknown /proc/50022/task/51315/root (readlink: Permission denied) 2024-04-26T21:58:48.757 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 51315 tp_osd_tp ceph txt unknown /proc/50022/task/51315/exe (readlink: Permission denied) 2024-04-26T21:58:48.757 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 51315 tp_osd_tp ceph NOFD /proc/50022/task/51315/fd (opendir: Permission denied) 2024-04-26T21:58:48.757 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 51316 tp_osd_tp ceph cwd unknown /proc/50022/task/51316/cwd (readlink: Permission denied) 2024-04-26T21:58:48.757 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 51316 tp_osd_tp ceph rtd unknown /proc/50022/task/51316/root (readlink: Permission denied) 2024-04-26T21:58:48.757 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 51316 tp_osd_tp ceph txt unknown /proc/50022/task/51316/exe (readlink: Permission denied) 2024-04-26T21:58:48.757 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 51316 tp_osd_tp ceph NOFD /proc/50022/task/51316/fd (opendir: Permission denied) 2024-04-26T21:58:48.758 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 51317 tp_osd_tp ceph cwd unknown /proc/50022/task/51317/cwd (readlink: Permission denied) 2024-04-26T21:58:48.758 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 51317 tp_osd_tp ceph rtd unknown /proc/50022/task/51317/root (readlink: Permission denied) 2024-04-26T21:58:48.758 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 51317 tp_osd_tp ceph txt unknown /proc/50022/task/51317/exe (readlink: Permission denied) 2024-04-26T21:58:48.758 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 51317 tp_osd_tp ceph NOFD /proc/50022/task/51317/fd (opendir: Permission denied) 2024-04-26T21:58:48.758 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 51318 tp_osd_tp ceph cwd unknown /proc/50022/task/51318/cwd (readlink: Permission denied) 2024-04-26T21:58:48.758 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 51318 tp_osd_tp ceph rtd unknown /proc/50022/task/51318/root (readlink: Permission denied) 2024-04-26T21:58:48.758 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 51318 tp_osd_tp ceph txt unknown /proc/50022/task/51318/exe (readlink: Permission denied) 2024-04-26T21:58:48.758 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 51318 tp_osd_tp ceph NOFD /proc/50022/task/51318/fd (opendir: Permission denied) 2024-04-26T21:58:48.758 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 51319 tp_osd_tp ceph cwd unknown /proc/50022/task/51319/cwd (readlink: Permission denied) 2024-04-26T21:58:48.758 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 51319 tp_osd_tp ceph rtd unknown /proc/50022/task/51319/root (readlink: Permission denied) 2024-04-26T21:58:48.758 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 51319 tp_osd_tp ceph txt unknown /proc/50022/task/51319/exe (readlink: Permission denied) 2024-04-26T21:58:48.758 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 51319 tp_osd_tp ceph NOFD /proc/50022/task/51319/fd (opendir: Permission denied) 2024-04-26T21:58:48.758 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 51320 tp_osd_tp ceph cwd unknown /proc/50022/task/51320/cwd (readlink: Permission denied) 2024-04-26T21:58:48.758 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 51320 tp_osd_tp ceph rtd unknown /proc/50022/task/51320/root (readlink: Permission denied) 2024-04-26T21:58:48.758 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 51320 tp_osd_tp ceph txt unknown /proc/50022/task/51320/exe (readlink: Permission denied) 2024-04-26T21:58:48.759 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 51320 tp_osd_tp ceph NOFD /proc/50022/task/51320/fd (opendir: Permission denied) 2024-04-26T21:58:48.759 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 51321 tp_osd_tp ceph cwd unknown /proc/50022/task/51321/cwd (readlink: Permission denied) 2024-04-26T21:58:48.759 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 51321 tp_osd_tp ceph rtd unknown /proc/50022/task/51321/root (readlink: Permission denied) 2024-04-26T21:58:48.759 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 51321 tp_osd_tp ceph txt unknown /proc/50022/task/51321/exe (readlink: Permission denied) 2024-04-26T21:58:48.759 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 51321 tp_osd_tp ceph NOFD /proc/50022/task/51321/fd (opendir: Permission denied) 2024-04-26T21:58:48.759 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 51322 tp_osd_tp ceph cwd unknown /proc/50022/task/51322/cwd (readlink: Permission denied) 2024-04-26T21:58:48.759 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 51322 tp_osd_tp ceph rtd unknown /proc/50022/task/51322/root (readlink: Permission denied) 2024-04-26T21:58:48.759 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 51322 tp_osd_tp ceph txt unknown /proc/50022/task/51322/exe (readlink: Permission denied) 2024-04-26T21:58:48.759 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 51322 tp_osd_tp ceph NOFD /proc/50022/task/51322/fd (opendir: Permission denied) 2024-04-26T21:58:48.759 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 51323 osd_srv_h ceph cwd unknown /proc/50022/task/51323/cwd (readlink: Permission denied) 2024-04-26T21:58:48.759 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 51323 osd_srv_h ceph rtd unknown /proc/50022/task/51323/root (readlink: Permission denied) 2024-04-26T21:58:48.759 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 51323 osd_srv_h ceph txt unknown /proc/50022/task/51323/exe (readlink: Permission denied) 2024-04-26T21:58:48.759 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 50022 51323 osd_srv_h ceph NOFD /proc/50022/task/51323/fd (opendir: Permission denied) 2024-04-26T21:58:48.759 INFO:tasks.workunit.client.0.smithi072.stdout:sudo 51552 root cwd unknown /proc/51552/cwd (readlink: Permission denied) 2024-04-26T21:58:48.760 INFO:tasks.workunit.client.0.smithi072.stdout:sudo 51552 root rtd unknown /proc/51552/root (readlink: Permission denied) 2024-04-26T21:58:48.760 INFO:tasks.workunit.client.0.smithi072.stdout:sudo 51552 root txt unknown /proc/51552/exe (readlink: Permission denied) 2024-04-26T21:58:48.760 INFO:tasks.workunit.client.0.smithi072.stdout:sudo 51552 root NOFD /proc/51552/fd (opendir: Permission denied) 2024-04-26T21:58:48.760 INFO:tasks.workunit.client.0.smithi072.stdout:journalct 51580 root cwd unknown /proc/51580/cwd (readlink: Permission denied) 2024-04-26T21:58:48.760 INFO:tasks.workunit.client.0.smithi072.stdout:journalct 51580 root rtd unknown /proc/51580/root (readlink: Permission denied) 2024-04-26T21:58:48.760 INFO:tasks.workunit.client.0.smithi072.stdout:journalct 51580 root txt unknown /proc/51580/exe (readlink: Permission denied) 2024-04-26T21:58:48.760 INFO:tasks.workunit.client.0.smithi072.stdout:journalct 51580 root NOFD /proc/51580/fd (opendir: Permission denied) 2024-04-26T21:58:48.760 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/u 54147 root cwd unknown /proc/54147/cwd (readlink: Permission denied) 2024-04-26T21:58:48.760 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/u 54147 root rtd unknown /proc/54147/root (readlink: Permission denied) 2024-04-26T21:58:48.760 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/u 54147 root txt unknown /proc/54147/exe (readlink: Permission denied) 2024-04-26T21:58:48.760 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/u 54147 root NOFD /proc/54147/fd (opendir: Permission denied) 2024-04-26T21:58:48.760 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/6 56533 root cwd unknown /proc/56533/cwd (readlink: Permission denied) 2024-04-26T21:58:48.760 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/6 56533 root rtd unknown /proc/56533/root (readlink: Permission denied) 2024-04-26T21:58:48.760 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/6 56533 root txt unknown /proc/56533/exe (readlink: Permission denied) 2024-04-26T21:58:48.760 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/6 56533 root NOFD /proc/56533/fd (opendir: Permission denied) 2024-04-26T21:58:48.761 INFO:tasks.workunit.client.0.smithi072.stdout:conmon 57091 root cwd unknown /proc/57091/cwd (readlink: Permission denied) 2024-04-26T21:58:48.761 INFO:tasks.workunit.client.0.smithi072.stdout:conmon 57091 root rtd unknown /proc/57091/root (readlink: Permission denied) 2024-04-26T21:58:48.761 INFO:tasks.workunit.client.0.smithi072.stdout:conmon 57091 root txt unknown /proc/57091/exe (readlink: Permission denied) 2024-04-26T21:58:48.761 INFO:tasks.workunit.client.0.smithi072.stdout:conmon 57091 root NOFD /proc/57091/fd (opendir: Permission denied) 2024-04-26T21:58:48.761 INFO:tasks.workunit.client.0.smithi072.stdout:podman-in 57093 root cwd unknown /proc/57093/cwd (readlink: Permission denied) 2024-04-26T21:58:48.761 INFO:tasks.workunit.client.0.smithi072.stdout:podman-in 57093 root rtd unknown /proc/57093/root (readlink: Permission denied) 2024-04-26T21:58:48.761 INFO:tasks.workunit.client.0.smithi072.stdout:podman-in 57093 root txt unknown /proc/57093/exe (readlink: Permission denied) 2024-04-26T21:58:48.761 INFO:tasks.workunit.client.0.smithi072.stdout:podman-in 57093 root NOFD /proc/57093/fd (opendir: Permission denied) 2024-04-26T21:58:48.761 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 ceph cwd unknown /proc/57095/cwd (readlink: Permission denied) 2024-04-26T21:58:48.761 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 ceph rtd unknown /proc/57095/root (readlink: Permission denied) 2024-04-26T21:58:48.761 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 ceph txt unknown /proc/57095/exe (readlink: Permission denied) 2024-04-26T21:58:48.762 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 ceph NOFD /proc/57095/fd (opendir: Permission denied) 2024-04-26T21:58:48.762 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 57096 log ceph cwd unknown /proc/57095/task/57096/cwd (readlink: Permission denied) 2024-04-26T21:58:48.762 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 57096 log ceph rtd unknown /proc/57095/task/57096/root (readlink: Permission denied) 2024-04-26T21:58:48.762 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 57096 log ceph txt unknown /proc/57095/task/57096/exe (readlink: Permission denied) 2024-04-26T21:58:48.762 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 57096 log ceph NOFD /proc/57095/task/57096/fd (opendir: Permission denied) 2024-04-26T21:58:48.762 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 57098 msgr-work ceph cwd unknown /proc/57095/task/57098/cwd (readlink: Permission denied) 2024-04-26T21:58:48.762 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 57098 msgr-work ceph rtd unknown /proc/57095/task/57098/root (readlink: Permission denied) 2024-04-26T21:58:48.762 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 57098 msgr-work ceph txt unknown /proc/57095/task/57098/exe (readlink: Permission denied) 2024-04-26T21:58:48.762 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 57098 msgr-work ceph NOFD /proc/57095/task/57098/fd (opendir: Permission denied) 2024-04-26T21:58:48.762 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 57099 msgr-work ceph cwd unknown /proc/57095/task/57099/cwd (readlink: Permission denied) 2024-04-26T21:58:48.763 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 57099 msgr-work ceph rtd unknown /proc/57095/task/57099/root (readlink: Permission denied) 2024-04-26T21:58:48.763 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 57099 msgr-work ceph txt unknown /proc/57095/task/57099/exe (readlink: Permission denied) 2024-04-26T21:58:48.763 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 57099 msgr-work ceph NOFD /proc/57095/task/57099/fd (opendir: Permission denied) 2024-04-26T21:58:48.763 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 57100 msgr-work ceph cwd unknown /proc/57095/task/57100/cwd (readlink: Permission denied) 2024-04-26T21:58:48.763 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 57100 msgr-work ceph rtd unknown /proc/57095/task/57100/root (readlink: Permission denied) 2024-04-26T21:58:48.763 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 57100 msgr-work ceph txt unknown /proc/57095/task/57100/exe (readlink: Permission denied) 2024-04-26T21:58:48.763 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 57100 msgr-work ceph NOFD /proc/57095/task/57100/fd (opendir: Permission denied) 2024-04-26T21:58:48.763 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 57104 service ceph cwd unknown /proc/57095/task/57104/cwd (readlink: Permission denied) 2024-04-26T21:58:48.763 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 57104 service ceph rtd unknown /proc/57095/task/57104/root (readlink: Permission denied) 2024-04-26T21:58:48.763 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 57104 service ceph txt unknown /proc/57095/task/57104/exe (readlink: Permission denied) 2024-04-26T21:58:48.763 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 57104 service ceph NOFD /proc/57095/task/57104/fd (opendir: Permission denied) 2024-04-26T21:58:48.764 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 57105 admin_soc ceph cwd unknown /proc/57095/task/57105/cwd (readlink: Permission denied) 2024-04-26T21:58:48.764 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 57105 admin_soc ceph rtd unknown /proc/57095/task/57105/root (readlink: Permission denied) 2024-04-26T21:58:48.764 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 57105 admin_soc ceph txt unknown /proc/57095/task/57105/exe (readlink: Permission denied) 2024-04-26T21:58:48.764 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 57105 admin_soc ceph NOFD /proc/57095/task/57105/fd (opendir: Permission denied) 2024-04-26T21:58:48.764 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 57249 signal_ha ceph cwd unknown /proc/57095/task/57249/cwd (readlink: Permission denied) 2024-04-26T21:58:48.764 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 57249 signal_ha ceph rtd unknown /proc/57095/task/57249/root (readlink: Permission denied) 2024-04-26T21:58:48.764 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 57249 signal_ha ceph txt unknown /proc/57095/task/57249/exe (readlink: Permission denied) 2024-04-26T21:58:48.764 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 57249 signal_ha ceph NOFD /proc/57095/task/57249/fd (opendir: Permission denied) 2024-04-26T21:58:48.764 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 57250 io_contex ceph cwd unknown /proc/57095/task/57250/cwd (readlink: Permission denied) 2024-04-26T21:58:48.764 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 57250 io_contex ceph rtd unknown /proc/57095/task/57250/root (readlink: Permission denied) 2024-04-26T21:58:48.765 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 57250 io_contex ceph txt unknown /proc/57095/task/57250/exe (readlink: Permission denied) 2024-04-26T21:58:48.765 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 57250 io_contex ceph NOFD /proc/57095/task/57250/fd (opendir: Permission denied) 2024-04-26T21:58:48.765 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 57251 io_contex ceph cwd unknown /proc/57095/task/57251/cwd (readlink: Permission denied) 2024-04-26T21:58:48.765 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 57251 io_contex ceph rtd unknown /proc/57095/task/57251/root (readlink: Permission denied) 2024-04-26T21:58:48.765 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 57251 io_contex ceph txt unknown /proc/57095/task/57251/exe (readlink: Permission denied) 2024-04-26T21:58:48.765 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 57251 io_contex ceph NOFD /proc/57095/task/57251/fd (opendir: Permission denied) 2024-04-26T21:58:48.765 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 57268 OpHistory ceph cwd unknown /proc/57095/task/57268/cwd (readlink: Permission denied) 2024-04-26T21:58:48.765 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 57268 OpHistory ceph rtd unknown /proc/57095/task/57268/root (readlink: Permission denied) 2024-04-26T21:58:48.765 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 57268 OpHistory ceph txt unknown /proc/57095/task/57268/exe (readlink: Permission denied) 2024-04-26T21:58:48.765 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 57268 OpHistory ceph NOFD /proc/57095/task/57268/fd (opendir: Permission denied) 2024-04-26T21:58:48.765 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 57269 ceph_time ceph cwd unknown /proc/57095/task/57269/cwd (readlink: Permission denied) 2024-04-26T21:58:48.765 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 57269 ceph_time ceph rtd unknown /proc/57095/task/57269/root (readlink: Permission denied) 2024-04-26T21:58:48.765 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 57269 ceph_time ceph txt unknown /proc/57095/task/57269/exe (readlink: Permission denied) 2024-04-26T21:58:48.765 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 57269 ceph_time ceph NOFD /proc/57095/task/57269/fd (opendir: Permission denied) 2024-04-26T21:58:48.765 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 57366 ceph-osd ceph cwd unknown /proc/57095/task/57366/cwd (readlink: Permission denied) 2024-04-26T21:58:48.766 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 57366 ceph-osd ceph rtd unknown /proc/57095/task/57366/root (readlink: Permission denied) 2024-04-26T21:58:48.766 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 57366 ceph-osd ceph txt unknown /proc/57095/task/57366/exe (readlink: Permission denied) 2024-04-26T21:58:48.766 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 57366 ceph-osd ceph NOFD /proc/57095/task/57366/fd (opendir: Permission denied) 2024-04-26T21:58:48.766 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 57388 ceph-osd ceph cwd unknown /proc/57095/task/57388/cwd (readlink: Permission denied) 2024-04-26T21:58:48.766 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 57388 ceph-osd ceph rtd unknown /proc/57095/task/57388/root (readlink: Permission denied) 2024-04-26T21:58:48.766 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 57388 ceph-osd ceph txt unknown /proc/57095/task/57388/exe (readlink: Permission denied) 2024-04-26T21:58:48.766 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 57388 ceph-osd ceph NOFD /proc/57095/task/57388/fd (opendir: Permission denied) 2024-04-26T21:58:48.766 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 57407 ceph-osd ceph cwd unknown /proc/57095/task/57407/cwd (readlink: Permission denied) 2024-04-26T21:58:48.766 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 57407 ceph-osd ceph rtd unknown /proc/57095/task/57407/root (readlink: Permission denied) 2024-04-26T21:58:48.766 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 57407 ceph-osd ceph txt unknown /proc/57095/task/57407/exe (readlink: Permission denied) 2024-04-26T21:58:48.766 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 57407 ceph-osd ceph NOFD /proc/57095/task/57407/fd (opendir: Permission denied) 2024-04-26T21:58:48.766 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 57419 ceph-osd ceph cwd unknown /proc/57095/task/57419/cwd (readlink: Permission denied) 2024-04-26T21:58:48.766 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 57419 ceph-osd ceph rtd unknown /proc/57095/task/57419/root (readlink: Permission denied) 2024-04-26T21:58:48.766 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 57419 ceph-osd ceph txt unknown /proc/57095/task/57419/exe (readlink: Permission denied) 2024-04-26T21:58:48.766 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 57419 ceph-osd ceph NOFD /proc/57095/task/57419/fd (opendir: Permission denied) 2024-04-26T21:58:48.767 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 57435 ceph-osd ceph cwd unknown /proc/57095/task/57435/cwd (readlink: Permission denied) 2024-04-26T21:58:48.767 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 57435 ceph-osd ceph rtd unknown /proc/57095/task/57435/root (readlink: Permission denied) 2024-04-26T21:58:48.767 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 57435 ceph-osd ceph txt unknown /proc/57095/task/57435/exe (readlink: Permission denied) 2024-04-26T21:58:48.767 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 57435 ceph-osd ceph NOFD /proc/57095/task/57435/fd (opendir: Permission denied) 2024-04-26T21:58:48.767 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 57457 ceph-osd ceph cwd unknown /proc/57095/task/57457/cwd (readlink: Permission denied) 2024-04-26T21:58:48.767 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 57457 ceph-osd ceph rtd unknown /proc/57095/task/57457/root (readlink: Permission denied) 2024-04-26T21:58:48.767 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 57457 ceph-osd ceph txt unknown /proc/57095/task/57457/exe (readlink: Permission denied) 2024-04-26T21:58:48.767 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 57457 ceph-osd ceph NOFD /proc/57095/task/57457/fd (opendir: Permission denied) 2024-04-26T21:58:48.767 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 57462 ceph-osd ceph cwd unknown /proc/57095/task/57462/cwd (readlink: Permission denied) 2024-04-26T21:58:48.767 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 57462 ceph-osd ceph rtd unknown /proc/57095/task/57462/root (readlink: Permission denied) 2024-04-26T21:58:48.767 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 57462 ceph-osd ceph txt unknown /proc/57095/task/57462/exe (readlink: Permission denied) 2024-04-26T21:58:48.767 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 57462 ceph-osd ceph NOFD /proc/57095/task/57462/fd (opendir: Permission denied) 2024-04-26T21:58:48.767 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 57485 ceph-osd ceph cwd unknown /proc/57095/task/57485/cwd (readlink: Permission denied) 2024-04-26T21:58:48.767 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 57485 ceph-osd ceph rtd unknown /proc/57095/task/57485/root (readlink: Permission denied) 2024-04-26T21:58:48.768 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 57485 ceph-osd ceph txt unknown /proc/57095/task/57485/exe (readlink: Permission denied) 2024-04-26T21:58:48.768 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 57485 ceph-osd ceph NOFD /proc/57095/task/57485/fd (opendir: Permission denied) 2024-04-26T21:58:48.768 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 57486 ceph-osd ceph cwd unknown /proc/57095/task/57486/cwd (readlink: Permission denied) 2024-04-26T21:58:48.768 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 57486 ceph-osd ceph rtd unknown /proc/57095/task/57486/root (readlink: Permission denied) 2024-04-26T21:58:48.768 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 57486 ceph-osd ceph txt unknown /proc/57095/task/57486/exe (readlink: Permission denied) 2024-04-26T21:58:48.768 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 57486 ceph-osd ceph NOFD /proc/57095/task/57486/fd (opendir: Permission denied) 2024-04-26T21:58:48.768 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 57487 safe_time ceph cwd unknown /proc/57095/task/57487/cwd (readlink: Permission denied) 2024-04-26T21:58:48.768 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 57487 safe_time ceph rtd unknown /proc/57095/task/57487/root (readlink: Permission denied) 2024-04-26T21:58:48.768 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 57487 safe_time ceph txt unknown /proc/57095/task/57487/exe (readlink: Permission denied) 2024-04-26T21:58:48.768 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 57487 safe_time ceph NOFD /proc/57095/task/57487/fd (opendir: Permission denied) 2024-04-26T21:58:48.768 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 57488 safe_time ceph cwd unknown /proc/57095/task/57488/cwd (readlink: Permission denied) 2024-04-26T21:58:48.768 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 57488 safe_time ceph rtd unknown /proc/57095/task/57488/root (readlink: Permission denied) 2024-04-26T21:58:48.768 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 57488 safe_time ceph txt unknown /proc/57095/task/57488/exe (readlink: Permission denied) 2024-04-26T21:58:48.768 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 57488 safe_time ceph NOFD /proc/57095/task/57488/fd (opendir: Permission denied) 2024-04-26T21:58:48.769 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 57489 safe_time ceph cwd unknown /proc/57095/task/57489/cwd (readlink: Permission denied) 2024-04-26T21:58:48.769 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 57489 safe_time ceph rtd unknown /proc/57095/task/57489/root (readlink: Permission denied) 2024-04-26T21:58:48.769 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 57489 safe_time ceph txt unknown /proc/57095/task/57489/exe (readlink: Permission denied) 2024-04-26T21:58:48.769 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 57489 safe_time ceph NOFD /proc/57095/task/57489/fd (opendir: Permission denied) 2024-04-26T21:58:48.769 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 57490 safe_time ceph cwd unknown /proc/57095/task/57490/cwd (readlink: Permission denied) 2024-04-26T21:58:48.769 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 57490 safe_time ceph rtd unknown /proc/57095/task/57490/root (readlink: Permission denied) 2024-04-26T21:58:48.769 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 57490 safe_time ceph txt unknown /proc/57095/task/57490/exe (readlink: Permission denied) 2024-04-26T21:58:48.769 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 57490 safe_time ceph NOFD /proc/57095/task/57490/fd (opendir: Permission denied) 2024-04-26T21:58:48.769 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 57491 fn_anonym ceph cwd unknown /proc/57095/task/57491/cwd (readlink: Permission denied) 2024-04-26T21:58:48.769 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 57491 fn_anonym ceph rtd unknown /proc/57095/task/57491/root (readlink: Permission denied) 2024-04-26T21:58:48.769 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 57491 fn_anonym ceph txt unknown /proc/57095/task/57491/exe (readlink: Permission denied) 2024-04-26T21:58:48.769 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 57491 fn_anonym ceph NOFD /proc/57095/task/57491/fd (opendir: Permission denied) 2024-04-26T21:58:48.769 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 57495 rocksdb:l ceph cwd unknown /proc/57095/task/57495/cwd (readlink: Permission denied) 2024-04-26T21:58:48.769 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 57495 rocksdb:l ceph rtd unknown /proc/57095/task/57495/root (readlink: Permission denied) 2024-04-26T21:58:48.769 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 57495 rocksdb:l ceph txt unknown /proc/57095/task/57495/exe (readlink: Permission denied) 2024-04-26T21:58:48.770 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 57495 rocksdb:l ceph NOFD /proc/57095/task/57495/fd (opendir: Permission denied) 2024-04-26T21:58:48.770 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 57497 rocksdb:l ceph cwd unknown /proc/57095/task/57497/cwd (readlink: Permission denied) 2024-04-26T21:58:48.770 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 57497 rocksdb:l ceph rtd unknown /proc/57095/task/57497/root (readlink: Permission denied) 2024-04-26T21:58:48.770 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 57497 rocksdb:l ceph txt unknown /proc/57095/task/57497/exe (readlink: Permission denied) 2024-04-26T21:58:48.770 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 57497 rocksdb:l ceph NOFD /proc/57095/task/57497/fd (opendir: Permission denied) 2024-04-26T21:58:48.770 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 57498 rocksdb:l ceph cwd unknown /proc/57095/task/57498/cwd (readlink: Permission denied) 2024-04-26T21:58:48.770 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 57498 rocksdb:l ceph rtd unknown /proc/57095/task/57498/root (readlink: Permission denied) 2024-04-26T21:58:48.770 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 57498 rocksdb:l ceph txt unknown /proc/57095/task/57498/exe (readlink: Permission denied) 2024-04-26T21:58:48.770 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 57498 rocksdb:l ceph NOFD /proc/57095/task/57498/fd (opendir: Permission denied) 2024-04-26T21:58:48.770 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 57499 rocksdb:h ceph cwd unknown /proc/57095/task/57499/cwd (readlink: Permission denied) 2024-04-26T21:58:48.770 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 57499 rocksdb:h ceph rtd unknown /proc/57095/task/57499/root (readlink: Permission denied) 2024-04-26T21:58:48.770 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 57499 rocksdb:h ceph txt unknown /proc/57095/task/57499/exe (readlink: Permission denied) 2024-04-26T21:58:48.770 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 57499 rocksdb:h ceph NOFD /proc/57095/task/57499/fd (opendir: Permission denied) 2024-04-26T21:58:48.770 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 57949 bstore_ai ceph cwd unknown /proc/57095/task/57949/cwd (readlink: Permission denied) 2024-04-26T21:58:48.770 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 57949 bstore_ai ceph rtd unknown /proc/57095/task/57949/root (readlink: Permission denied) 2024-04-26T21:58:48.771 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 57949 bstore_ai ceph txt unknown /proc/57095/task/57949/exe (readlink: Permission denied) 2024-04-26T21:58:48.771 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 57949 bstore_ai ceph NOFD /proc/57095/task/57949/fd (opendir: Permission denied) 2024-04-26T21:58:48.771 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 58178 bstore_ai ceph cwd unknown /proc/57095/task/58178/cwd (readlink: Permission denied) 2024-04-26T21:58:48.771 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 58178 bstore_ai ceph rtd unknown /proc/57095/task/58178/root (readlink: Permission denied) 2024-04-26T21:58:48.771 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 58178 bstore_ai ceph txt unknown /proc/57095/task/58178/exe (readlink: Permission denied) 2024-04-26T21:58:48.771 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 58178 bstore_ai ceph NOFD /proc/57095/task/58178/fd (opendir: Permission denied) 2024-04-26T21:58:48.771 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 58365 ceph-osd ceph cwd unknown /proc/57095/task/58365/cwd (readlink: Permission denied) 2024-04-26T21:58:48.771 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 58365 ceph-osd ceph rtd unknown /proc/57095/task/58365/root (readlink: Permission denied) 2024-04-26T21:58:48.771 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 58365 ceph-osd ceph txt unknown /proc/57095/task/58365/exe (readlink: Permission denied) 2024-04-26T21:58:48.771 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 58365 ceph-osd ceph NOFD /proc/57095/task/58365/fd (opendir: Permission denied) 2024-04-26T21:58:48.771 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 58366 cfin ceph cwd unknown /proc/57095/task/58366/cwd (readlink: Permission denied) 2024-04-26T21:58:48.771 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 58366 cfin ceph rtd unknown /proc/57095/task/58366/root (readlink: Permission denied) 2024-04-26T21:58:48.771 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 58366 cfin ceph txt unknown /proc/57095/task/58366/exe (readlink: Permission denied) 2024-04-26T21:58:48.772 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 58366 cfin ceph NOFD /proc/57095/task/58366/fd (opendir: Permission denied) 2024-04-26T21:58:48.772 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 58367 bstore_kv ceph cwd unknown /proc/57095/task/58367/cwd (readlink: Permission denied) 2024-04-26T21:58:48.772 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 58367 bstore_kv ceph rtd unknown /proc/57095/task/58367/root (readlink: Permission denied) 2024-04-26T21:58:48.772 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 58367 bstore_kv ceph txt unknown /proc/57095/task/58367/exe (readlink: Permission denied) 2024-04-26T21:58:48.772 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 58367 bstore_kv ceph NOFD /proc/57095/task/58367/fd (opendir: Permission denied) 2024-04-26T21:58:48.772 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 58368 bstore_kv ceph cwd unknown /proc/57095/task/58368/cwd (readlink: Permission denied) 2024-04-26T21:58:48.772 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 58368 bstore_kv ceph rtd unknown /proc/57095/task/58368/root (readlink: Permission denied) 2024-04-26T21:58:48.772 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 58368 bstore_kv ceph txt unknown /proc/57095/task/58368/exe (readlink: Permission denied) 2024-04-26T21:58:48.772 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 58368 bstore_kv ceph NOFD /proc/57095/task/58368/fd (opendir: Permission denied) 2024-04-26T21:58:48.772 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 58369 bstore_me ceph cwd unknown /proc/57095/task/58369/cwd (readlink: Permission denied) 2024-04-26T21:58:48.772 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 58369 bstore_me ceph rtd unknown /proc/57095/task/58369/root (readlink: Permission denied) 2024-04-26T21:58:48.772 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 58369 bstore_me ceph txt unknown /proc/57095/task/58369/exe (readlink: Permission denied) 2024-04-26T21:58:48.772 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 58369 bstore_me ceph NOFD /proc/57095/task/58369/fd (opendir: Permission denied) 2024-04-26T21:58:48.772 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 58373 ms_dispat ceph cwd unknown /proc/57095/task/58373/cwd (readlink: Permission denied) 2024-04-26T21:58:48.772 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 58373 ms_dispat ceph rtd unknown /proc/57095/task/58373/root (readlink: Permission denied) 2024-04-26T21:58:48.773 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 58373 ms_dispat ceph txt unknown /proc/57095/task/58373/exe (readlink: Permission denied) 2024-04-26T21:58:48.773 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 58373 ms_dispat ceph NOFD /proc/57095/task/58373/fd (opendir: Permission denied) 2024-04-26T21:58:48.773 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 58374 ms_local ceph cwd unknown /proc/57095/task/58374/cwd (readlink: Permission denied) 2024-04-26T21:58:48.773 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 58374 ms_local ceph rtd unknown /proc/57095/task/58374/root (readlink: Permission denied) 2024-04-26T21:58:48.773 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 58374 ms_local ceph txt unknown /proc/57095/task/58374/exe (readlink: Permission denied) 2024-04-26T21:58:48.773 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 58374 ms_local ceph NOFD /proc/57095/task/58374/fd (opendir: Permission denied) 2024-04-26T21:58:48.773 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 58375 safe_time ceph cwd unknown /proc/57095/task/58375/cwd (readlink: Permission denied) 2024-04-26T21:58:48.773 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 58375 safe_time ceph rtd unknown /proc/57095/task/58375/root (readlink: Permission denied) 2024-04-26T21:58:48.773 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 58375 safe_time ceph txt unknown /proc/57095/task/58375/exe (readlink: Permission denied) 2024-04-26T21:58:48.773 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 58375 safe_time ceph NOFD /proc/57095/task/58375/fd (opendir: Permission denied) 2024-04-26T21:58:48.773 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 58376 safe_time ceph cwd unknown /proc/57095/task/58376/cwd (readlink: Permission denied) 2024-04-26T21:58:48.773 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 58376 safe_time ceph rtd unknown /proc/57095/task/58376/root (readlink: Permission denied) 2024-04-26T21:58:48.773 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 58376 safe_time ceph txt unknown /proc/57095/task/58376/exe (readlink: Permission denied) 2024-04-26T21:58:48.773 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 58376 safe_time ceph NOFD /proc/57095/task/58376/fd (opendir: Permission denied) 2024-04-26T21:58:48.774 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 58377 ms_dispat ceph cwd unknown /proc/57095/task/58377/cwd (readlink: Permission denied) 2024-04-26T21:58:48.774 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 58377 ms_dispat ceph rtd unknown /proc/57095/task/58377/root (readlink: Permission denied) 2024-04-26T21:58:48.774 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 58377 ms_dispat ceph txt unknown /proc/57095/task/58377/exe (readlink: Permission denied) 2024-04-26T21:58:48.774 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 58377 ms_dispat ceph NOFD /proc/57095/task/58377/fd (opendir: Permission denied) 2024-04-26T21:58:48.774 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 58378 ms_local ceph cwd unknown /proc/57095/task/58378/cwd (readlink: Permission denied) 2024-04-26T21:58:48.774 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 58378 ms_local ceph rtd unknown /proc/57095/task/58378/root (readlink: Permission denied) 2024-04-26T21:58:48.774 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 58378 ms_local ceph txt unknown /proc/57095/task/58378/exe (readlink: Permission denied) 2024-04-26T21:58:48.774 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 58378 ms_local ceph NOFD /proc/57095/task/58378/fd (opendir: Permission denied) 2024-04-26T21:58:48.774 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 58379 ms_dispat ceph cwd unknown /proc/57095/task/58379/cwd (readlink: Permission denied) 2024-04-26T21:58:48.774 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 58379 ms_dispat ceph rtd unknown /proc/57095/task/58379/root (readlink: Permission denied) 2024-04-26T21:58:48.774 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 58379 ms_dispat ceph txt unknown /proc/57095/task/58379/exe (readlink: Permission denied) 2024-04-26T21:58:48.774 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 58379 ms_dispat ceph NOFD /proc/57095/task/58379/fd (opendir: Permission denied) 2024-04-26T21:58:48.774 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 58380 ms_local ceph cwd unknown /proc/57095/task/58380/cwd (readlink: Permission denied) 2024-04-26T21:58:48.774 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 58380 ms_local ceph rtd unknown /proc/57095/task/58380/root (readlink: Permission denied) 2024-04-26T21:58:48.774 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 58380 ms_local ceph txt unknown /proc/57095/task/58380/exe (readlink: Permission denied) 2024-04-26T21:58:48.775 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 58380 ms_local ceph NOFD /proc/57095/task/58380/fd (opendir: Permission denied) 2024-04-26T21:58:48.775 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 58381 ms_dispat ceph cwd unknown /proc/57095/task/58381/cwd (readlink: Permission denied) 2024-04-26T21:58:48.775 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 58381 ms_dispat ceph rtd unknown /proc/57095/task/58381/root (readlink: Permission denied) 2024-04-26T21:58:48.775 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 58381 ms_dispat ceph txt unknown /proc/57095/task/58381/exe (readlink: Permission denied) 2024-04-26T21:58:48.775 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 58381 ms_dispat ceph NOFD /proc/57095/task/58381/fd (opendir: Permission denied) 2024-04-26T21:58:48.775 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 58382 ms_local ceph cwd unknown /proc/57095/task/58382/cwd (readlink: Permission denied) 2024-04-26T21:58:48.775 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 58382 ms_local ceph rtd unknown /proc/57095/task/58382/root (readlink: Permission denied) 2024-04-26T21:58:48.775 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 58382 ms_local ceph txt unknown /proc/57095/task/58382/exe (readlink: Permission denied) 2024-04-26T21:58:48.775 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 58382 ms_local ceph NOFD /proc/57095/task/58382/fd (opendir: Permission denied) 2024-04-26T21:58:48.775 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 58383 ms_dispat ceph cwd unknown /proc/57095/task/58383/cwd (readlink: Permission denied) 2024-04-26T21:58:48.775 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 58383 ms_dispat ceph rtd unknown /proc/57095/task/58383/root (readlink: Permission denied) 2024-04-26T21:58:48.775 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 58383 ms_dispat ceph txt unknown /proc/57095/task/58383/exe (readlink: Permission denied) 2024-04-26T21:58:48.775 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 58383 ms_dispat ceph NOFD /proc/57095/task/58383/fd (opendir: Permission denied) 2024-04-26T21:58:48.776 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 58384 ms_local ceph cwd unknown /proc/57095/task/58384/cwd (readlink: Permission denied) 2024-04-26T21:58:48.776 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 58384 ms_local ceph rtd unknown /proc/57095/task/58384/root (readlink: Permission denied) 2024-04-26T21:58:48.776 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 58384 ms_local ceph txt unknown /proc/57095/task/58384/exe (readlink: Permission denied) 2024-04-26T21:58:48.776 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 58384 ms_local ceph NOFD /proc/57095/task/58384/fd (opendir: Permission denied) 2024-04-26T21:58:48.776 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 58385 ms_dispat ceph cwd unknown /proc/57095/task/58385/cwd (readlink: Permission denied) 2024-04-26T21:58:48.776 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 58385 ms_dispat ceph rtd unknown /proc/57095/task/58385/root (readlink: Permission denied) 2024-04-26T21:58:48.776 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 58385 ms_dispat ceph txt unknown /proc/57095/task/58385/exe (readlink: Permission denied) 2024-04-26T21:58:48.776 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 58385 ms_dispat ceph NOFD /proc/57095/task/58385/fd (opendir: Permission denied) 2024-04-26T21:58:48.776 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 58386 ms_local ceph cwd unknown /proc/57095/task/58386/cwd (readlink: Permission denied) 2024-04-26T21:58:48.776 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 58386 ms_local ceph rtd unknown /proc/57095/task/58386/root (readlink: Permission denied) 2024-04-26T21:58:48.776 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 58386 ms_local ceph txt unknown /proc/57095/task/58386/exe (readlink: Permission denied) 2024-04-26T21:58:48.776 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 58386 ms_local ceph NOFD /proc/57095/task/58386/fd (opendir: Permission denied) 2024-04-26T21:58:48.776 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 58387 ms_dispat ceph cwd unknown /proc/57095/task/58387/cwd (readlink: Permission denied) 2024-04-26T21:58:48.776 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 58387 ms_dispat ceph rtd unknown /proc/57095/task/58387/root (readlink: Permission denied) 2024-04-26T21:58:48.776 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 58387 ms_dispat ceph txt unknown /proc/57095/task/58387/exe (readlink: Permission denied) 2024-04-26T21:58:48.777 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 58387 ms_dispat ceph NOFD /proc/57095/task/58387/fd (opendir: Permission denied) 2024-04-26T21:58:48.777 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 58388 ms_local ceph cwd unknown /proc/57095/task/58388/cwd (readlink: Permission denied) 2024-04-26T21:58:48.777 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 58388 ms_local ceph rtd unknown /proc/57095/task/58388/root (readlink: Permission denied) 2024-04-26T21:58:48.777 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 58388 ms_local ceph txt unknown /proc/57095/task/58388/exe (readlink: Permission denied) 2024-04-26T21:58:48.777 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 58388 ms_local ceph NOFD /proc/57095/task/58388/fd (opendir: Permission denied) 2024-04-26T21:58:48.777 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 58389 fn_anonym ceph cwd unknown /proc/57095/task/58389/cwd (readlink: Permission denied) 2024-04-26T21:58:48.777 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 58389 fn_anonym ceph rtd unknown /proc/57095/task/58389/root (readlink: Permission denied) 2024-04-26T21:58:48.777 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 58389 fn_anonym ceph txt unknown /proc/57095/task/58389/exe (readlink: Permission denied) 2024-04-26T21:58:48.777 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 58389 fn_anonym ceph NOFD /proc/57095/task/58389/fd (opendir: Permission denied) 2024-04-26T21:58:48.777 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 58390 finisher ceph cwd unknown /proc/57095/task/58390/cwd (readlink: Permission denied) 2024-04-26T21:58:48.777 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 58390 finisher ceph rtd unknown /proc/57095/task/58390/root (readlink: Permission denied) 2024-04-26T21:58:48.777 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 58390 finisher ceph txt unknown /proc/57095/task/58390/exe (readlink: Permission denied) 2024-04-26T21:58:48.777 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 58390 finisher ceph NOFD /proc/57095/task/58390/fd (opendir: Permission denied) 2024-04-26T21:58:48.777 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 58391 safe_time ceph cwd unknown /proc/57095/task/58391/cwd (readlink: Permission denied) 2024-04-26T21:58:48.777 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 58391 safe_time ceph rtd unknown /proc/57095/task/58391/root (readlink: Permission denied) 2024-04-26T21:58:48.778 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 58391 safe_time ceph txt unknown /proc/57095/task/58391/exe (readlink: Permission denied) 2024-04-26T21:58:48.778 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 58391 safe_time ceph NOFD /proc/57095/task/58391/fd (opendir: Permission denied) 2024-04-26T21:58:48.778 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 58392 safe_time ceph cwd unknown /proc/57095/task/58392/cwd (readlink: Permission denied) 2024-04-26T21:58:48.778 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 58392 safe_time ceph rtd unknown /proc/57095/task/58392/root (readlink: Permission denied) 2024-04-26T21:58:48.778 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 58392 safe_time ceph txt unknown /proc/57095/task/58392/exe (readlink: Permission denied) 2024-04-26T21:58:48.778 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 58392 safe_time ceph NOFD /proc/57095/task/58392/fd (opendir: Permission denied) 2024-04-26T21:58:48.778 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 58393 ceph-osd ceph cwd unknown /proc/57095/task/58393/cwd (readlink: Permission denied) 2024-04-26T21:58:48.778 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 58393 ceph-osd ceph rtd unknown /proc/57095/task/58393/root (readlink: Permission denied) 2024-04-26T21:58:48.778 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 58393 ceph-osd ceph txt unknown /proc/57095/task/58393/exe (readlink: Permission denied) 2024-04-26T21:58:48.778 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 58393 ceph-osd ceph NOFD /proc/57095/task/58393/fd (opendir: Permission denied) 2024-04-26T21:58:48.778 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 58394 osd_srv_a ceph cwd unknown /proc/57095/task/58394/cwd (readlink: Permission denied) 2024-04-26T21:58:48.778 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 58394 osd_srv_a ceph rtd unknown /proc/57095/task/58394/root (readlink: Permission denied) 2024-04-26T21:58:48.778 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 58394 osd_srv_a ceph txt unknown /proc/57095/task/58394/exe (readlink: Permission denied) 2024-04-26T21:58:48.778 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 58394 osd_srv_a ceph NOFD /proc/57095/task/58394/fd (opendir: Permission denied) 2024-04-26T21:58:48.779 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 58395 tp_osd_tp ceph cwd unknown /proc/57095/task/58395/cwd (readlink: Permission denied) 2024-04-26T21:58:48.779 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 58395 tp_osd_tp ceph rtd unknown /proc/57095/task/58395/root (readlink: Permission denied) 2024-04-26T21:58:48.779 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 58395 tp_osd_tp ceph txt unknown /proc/57095/task/58395/exe (readlink: Permission denied) 2024-04-26T21:58:48.779 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 58395 tp_osd_tp ceph NOFD /proc/57095/task/58395/fd (opendir: Permission denied) 2024-04-26T21:58:48.779 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 58396 tp_osd_tp ceph cwd unknown /proc/57095/task/58396/cwd (readlink: Permission denied) 2024-04-26T21:58:48.779 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 58396 tp_osd_tp ceph rtd unknown /proc/57095/task/58396/root (readlink: Permission denied) 2024-04-26T21:58:48.779 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 58396 tp_osd_tp ceph txt unknown /proc/57095/task/58396/exe (readlink: Permission denied) 2024-04-26T21:58:48.779 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 58396 tp_osd_tp ceph NOFD /proc/57095/task/58396/fd (opendir: Permission denied) 2024-04-26T21:58:48.779 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 58397 tp_osd_tp ceph cwd unknown /proc/57095/task/58397/cwd (readlink: Permission denied) 2024-04-26T21:58:48.779 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 58397 tp_osd_tp ceph rtd unknown /proc/57095/task/58397/root (readlink: Permission denied) 2024-04-26T21:58:48.779 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 58397 tp_osd_tp ceph txt unknown /proc/57095/task/58397/exe (readlink: Permission denied) 2024-04-26T21:58:48.779 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 58397 tp_osd_tp ceph NOFD /proc/57095/task/58397/fd (opendir: Permission denied) 2024-04-26T21:58:48.779 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 58398 tp_osd_tp ceph cwd unknown /proc/57095/task/58398/cwd (readlink: Permission denied) 2024-04-26T21:58:48.779 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 58398 tp_osd_tp ceph rtd unknown /proc/57095/task/58398/root (readlink: Permission denied) 2024-04-26T21:58:48.780 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 58398 tp_osd_tp ceph txt unknown /proc/57095/task/58398/exe (readlink: Permission denied) 2024-04-26T21:58:48.780 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 58398 tp_osd_tp ceph NOFD /proc/57095/task/58398/fd (opendir: Permission denied) 2024-04-26T21:58:48.780 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 58399 tp_osd_tp ceph cwd unknown /proc/57095/task/58399/cwd (readlink: Permission denied) 2024-04-26T21:58:48.780 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 58399 tp_osd_tp ceph rtd unknown /proc/57095/task/58399/root (readlink: Permission denied) 2024-04-26T21:58:48.780 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 58399 tp_osd_tp ceph txt unknown /proc/57095/task/58399/exe (readlink: Permission denied) 2024-04-26T21:58:48.780 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 58399 tp_osd_tp ceph NOFD /proc/57095/task/58399/fd (opendir: Permission denied) 2024-04-26T21:58:48.780 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 58400 tp_osd_tp ceph cwd unknown /proc/57095/task/58400/cwd (readlink: Permission denied) 2024-04-26T21:58:48.780 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 58400 tp_osd_tp ceph rtd unknown /proc/57095/task/58400/root (readlink: Permission denied) 2024-04-26T21:58:48.780 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 58400 tp_osd_tp ceph txt unknown /proc/57095/task/58400/exe (readlink: Permission denied) 2024-04-26T21:58:48.780 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 58400 tp_osd_tp ceph NOFD /proc/57095/task/58400/fd (opendir: Permission denied) 2024-04-26T21:58:48.780 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 58401 tp_osd_tp ceph cwd unknown /proc/57095/task/58401/cwd (readlink: Permission denied) 2024-04-26T21:58:48.780 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 58401 tp_osd_tp ceph rtd unknown /proc/57095/task/58401/root (readlink: Permission denied) 2024-04-26T21:58:48.780 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 58401 tp_osd_tp ceph txt unknown /proc/57095/task/58401/exe (readlink: Permission denied) 2024-04-26T21:58:48.780 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 58401 tp_osd_tp ceph NOFD /proc/57095/task/58401/fd (opendir: Permission denied) 2024-04-26T21:58:48.781 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 58402 tp_osd_tp ceph cwd unknown /proc/57095/task/58402/cwd (readlink: Permission denied) 2024-04-26T21:58:48.781 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 58402 tp_osd_tp ceph rtd unknown /proc/57095/task/58402/root (readlink: Permission denied) 2024-04-26T21:58:48.781 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 58402 tp_osd_tp ceph txt unknown /proc/57095/task/58402/exe (readlink: Permission denied) 2024-04-26T21:58:48.781 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 58402 tp_osd_tp ceph NOFD /proc/57095/task/58402/fd (opendir: Permission denied) 2024-04-26T21:58:48.781 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 58403 tp_osd_tp ceph cwd unknown /proc/57095/task/58403/cwd (readlink: Permission denied) 2024-04-26T21:58:48.781 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 58403 tp_osd_tp ceph rtd unknown /proc/57095/task/58403/root (readlink: Permission denied) 2024-04-26T21:58:48.781 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 58403 tp_osd_tp ceph txt unknown /proc/57095/task/58403/exe (readlink: Permission denied) 2024-04-26T21:58:48.781 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 58403 tp_osd_tp ceph NOFD /proc/57095/task/58403/fd (opendir: Permission denied) 2024-04-26T21:58:48.781 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 58404 tp_osd_tp ceph cwd unknown /proc/57095/task/58404/cwd (readlink: Permission denied) 2024-04-26T21:58:48.781 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 58404 tp_osd_tp ceph rtd unknown /proc/57095/task/58404/root (readlink: Permission denied) 2024-04-26T21:58:48.781 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 58404 tp_osd_tp ceph txt unknown /proc/57095/task/58404/exe (readlink: Permission denied) 2024-04-26T21:58:48.781 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 58404 tp_osd_tp ceph NOFD /proc/57095/task/58404/fd (opendir: Permission denied) 2024-04-26T21:58:48.781 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 58405 tp_osd_tp ceph cwd unknown /proc/57095/task/58405/cwd (readlink: Permission denied) 2024-04-26T21:58:48.781 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 58405 tp_osd_tp ceph rtd unknown /proc/57095/task/58405/root (readlink: Permission denied) 2024-04-26T21:58:48.781 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 58405 tp_osd_tp ceph txt unknown /proc/57095/task/58405/exe (readlink: Permission denied) 2024-04-26T21:58:48.782 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 58405 tp_osd_tp ceph NOFD /proc/57095/task/58405/fd (opendir: Permission denied) 2024-04-26T21:58:48.782 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 58406 tp_osd_tp ceph cwd unknown /proc/57095/task/58406/cwd (readlink: Permission denied) 2024-04-26T21:58:48.782 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 58406 tp_osd_tp ceph rtd unknown /proc/57095/task/58406/root (readlink: Permission denied) 2024-04-26T21:58:48.782 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 58406 tp_osd_tp ceph txt unknown /proc/57095/task/58406/exe (readlink: Permission denied) 2024-04-26T21:58:48.782 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 58406 tp_osd_tp ceph NOFD /proc/57095/task/58406/fd (opendir: Permission denied) 2024-04-26T21:58:48.782 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 58407 tp_osd_tp ceph cwd unknown /proc/57095/task/58407/cwd (readlink: Permission denied) 2024-04-26T21:58:48.782 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 58407 tp_osd_tp ceph rtd unknown /proc/57095/task/58407/root (readlink: Permission denied) 2024-04-26T21:58:48.782 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 58407 tp_osd_tp ceph txt unknown /proc/57095/task/58407/exe (readlink: Permission denied) 2024-04-26T21:58:48.782 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 58407 tp_osd_tp ceph NOFD /proc/57095/task/58407/fd (opendir: Permission denied) 2024-04-26T21:58:48.782 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 58408 tp_osd_tp ceph cwd unknown /proc/57095/task/58408/cwd (readlink: Permission denied) 2024-04-26T21:58:48.782 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 58408 tp_osd_tp ceph rtd unknown /proc/57095/task/58408/root (readlink: Permission denied) 2024-04-26T21:58:48.782 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 58408 tp_osd_tp ceph txt unknown /proc/57095/task/58408/exe (readlink: Permission denied) 2024-04-26T21:58:48.783 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 58408 tp_osd_tp ceph NOFD /proc/57095/task/58408/fd (opendir: Permission denied) 2024-04-26T21:58:48.783 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 58409 tp_osd_tp ceph cwd unknown /proc/57095/task/58409/cwd (readlink: Permission denied) 2024-04-26T21:58:48.783 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 58409 tp_osd_tp ceph rtd unknown /proc/57095/task/58409/root (readlink: Permission denied) 2024-04-26T21:58:48.783 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 58409 tp_osd_tp ceph txt unknown /proc/57095/task/58409/exe (readlink: Permission denied) 2024-04-26T21:58:48.783 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 58409 tp_osd_tp ceph NOFD /proc/57095/task/58409/fd (opendir: Permission denied) 2024-04-26T21:58:48.783 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 58410 tp_osd_tp ceph cwd unknown /proc/57095/task/58410/cwd (readlink: Permission denied) 2024-04-26T21:58:48.783 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 58410 tp_osd_tp ceph rtd unknown /proc/57095/task/58410/root (readlink: Permission denied) 2024-04-26T21:58:48.783 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 58410 tp_osd_tp ceph txt unknown /proc/57095/task/58410/exe (readlink: Permission denied) 2024-04-26T21:58:48.783 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 58410 tp_osd_tp ceph NOFD /proc/57095/task/58410/fd (opendir: Permission denied) 2024-04-26T21:58:48.783 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 58411 osd_srv_h ceph cwd unknown /proc/57095/task/58411/cwd (readlink: Permission denied) 2024-04-26T21:58:48.783 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 58411 osd_srv_h ceph rtd unknown /proc/57095/task/58411/root (readlink: Permission denied) 2024-04-26T21:58:48.784 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 58411 osd_srv_h ceph txt unknown /proc/57095/task/58411/exe (readlink: Permission denied) 2024-04-26T21:58:48.784 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 57095 58411 osd_srv_h ceph NOFD /proc/57095/task/58411/fd (opendir: Permission denied) 2024-04-26T21:58:48.784 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/u 57406 root cwd unknown /proc/57406/cwd (readlink: Permission denied) 2024-04-26T21:58:48.784 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/u 57406 root rtd unknown /proc/57406/root (readlink: Permission denied) 2024-04-26T21:58:48.784 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/u 57406 root txt unknown /proc/57406/exe (readlink: Permission denied) 2024-04-26T21:58:48.784 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/u 57406 root NOFD /proc/57406/fd (opendir: Permission denied) 2024-04-26T21:58:48.784 INFO:tasks.workunit.client.0.smithi072.stdout:sudo 58546 root cwd unknown /proc/58546/cwd (readlink: Permission denied) 2024-04-26T21:58:48.784 INFO:tasks.workunit.client.0.smithi072.stdout:sudo 58546 root rtd unknown /proc/58546/root (readlink: Permission denied) 2024-04-26T21:58:48.784 INFO:tasks.workunit.client.0.smithi072.stdout:sudo 58546 root txt unknown /proc/58546/exe (readlink: Permission denied) 2024-04-26T21:58:48.784 INFO:tasks.workunit.client.0.smithi072.stdout:sudo 58546 root NOFD /proc/58546/fd (opendir: Permission denied) 2024-04-26T21:58:48.784 INFO:tasks.workunit.client.0.smithi072.stdout:journalct 58575 root cwd unknown /proc/58575/cwd (readlink: Permission denied) 2024-04-26T21:58:48.784 INFO:tasks.workunit.client.0.smithi072.stdout:journalct 58575 root rtd unknown /proc/58575/root (readlink: Permission denied) 2024-04-26T21:58:48.784 INFO:tasks.workunit.client.0.smithi072.stdout:journalct 58575 root txt unknown /proc/58575/exe (readlink: Permission denied) 2024-04-26T21:58:48.785 INFO:tasks.workunit.client.0.smithi072.stdout:journalct 58575 root NOFD /proc/58575/fd (opendir: Permission denied) 2024-04-26T21:58:48.785 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/3 61348 root cwd unknown /proc/61348/cwd (readlink: Permission denied) 2024-04-26T21:58:48.785 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/3 61348 root rtd unknown /proc/61348/root (readlink: Permission denied) 2024-04-26T21:58:48.785 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/3 61348 root txt unknown /proc/61348/exe (readlink: Permission denied) 2024-04-26T21:58:48.785 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/3 61348 root NOFD /proc/61348/fd (opendir: Permission denied) 2024-04-26T21:58:48.785 INFO:tasks.workunit.client.0.smithi072.stdout:conmon 63797 root cwd unknown /proc/63797/cwd (readlink: Permission denied) 2024-04-26T21:58:48.785 INFO:tasks.workunit.client.0.smithi072.stdout:conmon 63797 root rtd unknown /proc/63797/root (readlink: Permission denied) 2024-04-26T21:58:48.785 INFO:tasks.workunit.client.0.smithi072.stdout:conmon 63797 root txt unknown /proc/63797/exe (readlink: Permission denied) 2024-04-26T21:58:48.785 INFO:tasks.workunit.client.0.smithi072.stdout:conmon 63797 root NOFD /proc/63797/fd (opendir: Permission denied) 2024-04-26T21:58:48.785 INFO:tasks.workunit.client.0.smithi072.stdout:podman-in 63799 root cwd unknown /proc/63799/cwd (readlink: Permission denied) 2024-04-26T21:58:48.785 INFO:tasks.workunit.client.0.smithi072.stdout:podman-in 63799 root rtd unknown /proc/63799/root (readlink: Permission denied) 2024-04-26T21:58:48.785 INFO:tasks.workunit.client.0.smithi072.stdout:podman-in 63799 root txt unknown /proc/63799/exe (readlink: Permission denied) 2024-04-26T21:58:48.785 INFO:tasks.workunit.client.0.smithi072.stdout:podman-in 63799 root NOFD /proc/63799/fd (opendir: Permission denied) 2024-04-26T21:58:48.785 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 ceph cwd unknown /proc/63801/cwd (readlink: Permission denied) 2024-04-26T21:58:48.785 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 ceph rtd unknown /proc/63801/root (readlink: Permission denied) 2024-04-26T21:58:48.786 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 ceph txt unknown /proc/63801/exe (readlink: Permission denied) 2024-04-26T21:58:48.786 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 ceph NOFD /proc/63801/fd (opendir: Permission denied) 2024-04-26T21:58:48.786 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 63802 log ceph cwd unknown /proc/63801/task/63802/cwd (readlink: Permission denied) 2024-04-26T21:58:48.786 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 63802 log ceph rtd unknown /proc/63801/task/63802/root (readlink: Permission denied) 2024-04-26T21:58:48.786 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 63802 log ceph txt unknown /proc/63801/task/63802/exe (readlink: Permission denied) 2024-04-26T21:58:48.786 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 63802 log ceph NOFD /proc/63801/task/63802/fd (opendir: Permission denied) 2024-04-26T21:58:48.786 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 63804 msgr-work ceph cwd unknown /proc/63801/task/63804/cwd (readlink: Permission denied) 2024-04-26T21:58:48.786 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 63804 msgr-work ceph rtd unknown /proc/63801/task/63804/root (readlink: Permission denied) 2024-04-26T21:58:48.786 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 63804 msgr-work ceph txt unknown /proc/63801/task/63804/exe (readlink: Permission denied) 2024-04-26T21:58:48.787 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 63804 msgr-work ceph NOFD /proc/63801/task/63804/fd (opendir: Permission denied) 2024-04-26T21:58:48.787 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 63805 msgr-work ceph cwd unknown /proc/63801/task/63805/cwd (readlink: Permission denied) 2024-04-26T21:58:48.787 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 63805 msgr-work ceph rtd unknown /proc/63801/task/63805/root (readlink: Permission denied) 2024-04-26T21:58:48.787 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 63805 msgr-work ceph txt unknown /proc/63801/task/63805/exe (readlink: Permission denied) 2024-04-26T21:58:48.787 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 63805 msgr-work ceph NOFD /proc/63801/task/63805/fd (opendir: Permission denied) 2024-04-26T21:58:48.787 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 63806 msgr-work ceph cwd unknown /proc/63801/task/63806/cwd (readlink: Permission denied) 2024-04-26T21:58:48.787 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 63806 msgr-work ceph rtd unknown /proc/63801/task/63806/root (readlink: Permission denied) 2024-04-26T21:58:48.787 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 63806 msgr-work ceph txt unknown /proc/63801/task/63806/exe (readlink: Permission denied) 2024-04-26T21:58:48.787 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 63806 msgr-work ceph NOFD /proc/63801/task/63806/fd (opendir: Permission denied) 2024-04-26T21:58:48.787 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 63810 service ceph cwd unknown /proc/63801/task/63810/cwd (readlink: Permission denied) 2024-04-26T21:58:48.787 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 63810 service ceph rtd unknown /proc/63801/task/63810/root (readlink: Permission denied) 2024-04-26T21:58:48.787 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 63810 service ceph txt unknown /proc/63801/task/63810/exe (readlink: Permission denied) 2024-04-26T21:58:48.787 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 63810 service ceph NOFD /proc/63801/task/63810/fd (opendir: Permission denied) 2024-04-26T21:58:48.787 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 63811 admin_soc ceph cwd unknown /proc/63801/task/63811/cwd (readlink: Permission denied) 2024-04-26T21:58:48.787 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 63811 admin_soc ceph rtd unknown /proc/63801/task/63811/root (readlink: Permission denied) 2024-04-26T21:58:48.788 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 63811 admin_soc ceph txt unknown /proc/63801/task/63811/exe (readlink: Permission denied) 2024-04-26T21:58:48.788 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 63811 admin_soc ceph NOFD /proc/63801/task/63811/fd (opendir: Permission denied) 2024-04-26T21:58:48.788 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 63956 signal_ha ceph cwd unknown /proc/63801/task/63956/cwd (readlink: Permission denied) 2024-04-26T21:58:48.788 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 63956 signal_ha ceph rtd unknown /proc/63801/task/63956/root (readlink: Permission denied) 2024-04-26T21:58:48.788 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 63956 signal_ha ceph txt unknown /proc/63801/task/63956/exe (readlink: Permission denied) 2024-04-26T21:58:48.788 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 63956 signal_ha ceph NOFD /proc/63801/task/63956/fd (opendir: Permission denied) 2024-04-26T21:58:48.788 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 63957 io_contex ceph cwd unknown /proc/63801/task/63957/cwd (readlink: Permission denied) 2024-04-26T21:58:48.788 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 63957 io_contex ceph rtd unknown /proc/63801/task/63957/root (readlink: Permission denied) 2024-04-26T21:58:48.788 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 63957 io_contex ceph txt unknown /proc/63801/task/63957/exe (readlink: Permission denied) 2024-04-26T21:58:48.788 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 63957 io_contex ceph NOFD /proc/63801/task/63957/fd (opendir: Permission denied) 2024-04-26T21:58:48.788 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 63958 io_contex ceph cwd unknown /proc/63801/task/63958/cwd (readlink: Permission denied) 2024-04-26T21:58:48.789 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 63958 io_contex ceph rtd unknown /proc/63801/task/63958/root (readlink: Permission denied) 2024-04-26T21:58:48.789 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 63958 io_contex ceph txt unknown /proc/63801/task/63958/exe (readlink: Permission denied) 2024-04-26T21:58:48.789 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 63958 io_contex ceph NOFD /proc/63801/task/63958/fd (opendir: Permission denied) 2024-04-26T21:58:48.789 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 63961 OpHistory ceph cwd unknown /proc/63801/task/63961/cwd (readlink: Permission denied) 2024-04-26T21:58:48.789 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 63961 OpHistory ceph rtd unknown /proc/63801/task/63961/root (readlink: Permission denied) 2024-04-26T21:58:48.789 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 63961 OpHistory ceph txt unknown /proc/63801/task/63961/exe (readlink: Permission denied) 2024-04-26T21:58:48.789 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 63961 OpHistory ceph NOFD /proc/63801/task/63961/fd (opendir: Permission denied) 2024-04-26T21:58:48.789 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 63962 ceph_time ceph cwd unknown /proc/63801/task/63962/cwd (readlink: Permission denied) 2024-04-26T21:58:48.789 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 63962 ceph_time ceph rtd unknown /proc/63801/task/63962/root (readlink: Permission denied) 2024-04-26T21:58:48.789 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 63962 ceph_time ceph txt unknown /proc/63801/task/63962/exe (readlink: Permission denied) 2024-04-26T21:58:48.789 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 63962 ceph_time ceph NOFD /proc/63801/task/63962/fd (opendir: Permission denied) 2024-04-26T21:58:48.789 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 64072 ceph-osd ceph cwd unknown /proc/63801/task/64072/cwd (readlink: Permission denied) 2024-04-26T21:58:48.789 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 64072 ceph-osd ceph rtd unknown /proc/63801/task/64072/root (readlink: Permission denied) 2024-04-26T21:58:48.790 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 64072 ceph-osd ceph txt unknown /proc/63801/task/64072/exe (readlink: Permission denied) 2024-04-26T21:58:48.790 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 64072 ceph-osd ceph NOFD /proc/63801/task/64072/fd (opendir: Permission denied) 2024-04-26T21:58:48.790 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 64091 ceph-osd ceph cwd unknown /proc/63801/task/64091/cwd (readlink: Permission denied) 2024-04-26T21:58:48.790 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 64091 ceph-osd ceph rtd unknown /proc/63801/task/64091/root (readlink: Permission denied) 2024-04-26T21:58:48.790 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 64091 ceph-osd ceph txt unknown /proc/63801/task/64091/exe (readlink: Permission denied) 2024-04-26T21:58:48.790 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 64091 ceph-osd ceph NOFD /proc/63801/task/64091/fd (opendir: Permission denied) 2024-04-26T21:58:48.790 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 64122 ceph-osd ceph cwd unknown /proc/63801/task/64122/cwd (readlink: Permission denied) 2024-04-26T21:58:48.790 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 64122 ceph-osd ceph rtd unknown /proc/63801/task/64122/root (readlink: Permission denied) 2024-04-26T21:58:48.790 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 64122 ceph-osd ceph txt unknown /proc/63801/task/64122/exe (readlink: Permission denied) 2024-04-26T21:58:48.790 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 64122 ceph-osd ceph NOFD /proc/63801/task/64122/fd (opendir: Permission denied) 2024-04-26T21:58:48.790 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 64130 ceph-osd ceph cwd unknown /proc/63801/task/64130/cwd (readlink: Permission denied) 2024-04-26T21:58:48.790 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 64130 ceph-osd ceph rtd unknown /proc/63801/task/64130/root (readlink: Permission denied) 2024-04-26T21:58:48.791 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 64130 ceph-osd ceph txt unknown /proc/63801/task/64130/exe (readlink: Permission denied) 2024-04-26T21:58:48.791 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 64130 ceph-osd ceph NOFD /proc/63801/task/64130/fd (opendir: Permission denied) 2024-04-26T21:58:48.791 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 64137 ceph-osd ceph cwd unknown /proc/63801/task/64137/cwd (readlink: Permission denied) 2024-04-26T21:58:48.791 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 64137 ceph-osd ceph rtd unknown /proc/63801/task/64137/root (readlink: Permission denied) 2024-04-26T21:58:48.791 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 64137 ceph-osd ceph txt unknown /proc/63801/task/64137/exe (readlink: Permission denied) 2024-04-26T21:58:48.791 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 64137 ceph-osd ceph NOFD /proc/63801/task/64137/fd (opendir: Permission denied) 2024-04-26T21:58:48.791 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 64144 ceph-osd ceph cwd unknown /proc/63801/task/64144/cwd (readlink: Permission denied) 2024-04-26T21:58:48.791 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 64144 ceph-osd ceph rtd unknown /proc/63801/task/64144/root (readlink: Permission denied) 2024-04-26T21:58:48.791 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 64144 ceph-osd ceph txt unknown /proc/63801/task/64144/exe (readlink: Permission denied) 2024-04-26T21:58:48.791 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 64144 ceph-osd ceph NOFD /proc/63801/task/64144/fd (opendir: Permission denied) 2024-04-26T21:58:48.791 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 64183 ceph-osd ceph cwd unknown /proc/63801/task/64183/cwd (readlink: Permission denied) 2024-04-26T21:58:48.792 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 64183 ceph-osd ceph rtd unknown /proc/63801/task/64183/root (readlink: Permission denied) 2024-04-26T21:58:48.792 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 64183 ceph-osd ceph txt unknown /proc/63801/task/64183/exe (readlink: Permission denied) 2024-04-26T21:58:48.792 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 64183 ceph-osd ceph NOFD /proc/63801/task/64183/fd (opendir: Permission denied) 2024-04-26T21:58:48.792 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 64192 ceph-osd ceph cwd unknown /proc/63801/task/64192/cwd (readlink: Permission denied) 2024-04-26T21:58:48.792 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 64192 ceph-osd ceph rtd unknown /proc/63801/task/64192/root (readlink: Permission denied) 2024-04-26T21:58:48.792 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 64192 ceph-osd ceph txt unknown /proc/63801/task/64192/exe (readlink: Permission denied) 2024-04-26T21:58:48.792 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 64192 ceph-osd ceph NOFD /proc/63801/task/64192/fd (opendir: Permission denied) 2024-04-26T21:58:48.792 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 64193 ceph-osd ceph cwd unknown /proc/63801/task/64193/cwd (readlink: Permission denied) 2024-04-26T21:58:48.792 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 64193 ceph-osd ceph rtd unknown /proc/63801/task/64193/root (readlink: Permission denied) 2024-04-26T21:58:48.792 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 64193 ceph-osd ceph txt unknown /proc/63801/task/64193/exe (readlink: Permission denied) 2024-04-26T21:58:48.792 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 64193 ceph-osd ceph NOFD /proc/63801/task/64193/fd (opendir: Permission denied) 2024-04-26T21:58:48.793 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 64194 safe_time ceph cwd unknown /proc/63801/task/64194/cwd (readlink: Permission denied) 2024-04-26T21:58:48.793 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 64194 safe_time ceph rtd unknown /proc/63801/task/64194/root (readlink: Permission denied) 2024-04-26T21:58:48.793 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 64194 safe_time ceph txt unknown /proc/63801/task/64194/exe (readlink: Permission denied) 2024-04-26T21:58:48.793 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 64194 safe_time ceph NOFD /proc/63801/task/64194/fd (opendir: Permission denied) 2024-04-26T21:58:48.793 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 64195 safe_time ceph cwd unknown /proc/63801/task/64195/cwd (readlink: Permission denied) 2024-04-26T21:58:48.793 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 64195 safe_time ceph rtd unknown /proc/63801/task/64195/root (readlink: Permission denied) 2024-04-26T21:58:48.793 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 64195 safe_time ceph txt unknown /proc/63801/task/64195/exe (readlink: Permission denied) 2024-04-26T21:58:48.793 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 64195 safe_time ceph NOFD /proc/63801/task/64195/fd (opendir: Permission denied) 2024-04-26T21:58:48.793 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 64196 safe_time ceph cwd unknown /proc/63801/task/64196/cwd (readlink: Permission denied) 2024-04-26T21:58:48.793 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 64196 safe_time ceph rtd unknown /proc/63801/task/64196/root (readlink: Permission denied) 2024-04-26T21:58:48.793 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 64196 safe_time ceph txt unknown /proc/63801/task/64196/exe (readlink: Permission denied) 2024-04-26T21:58:48.794 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 64196 safe_time ceph NOFD /proc/63801/task/64196/fd (opendir: Permission denied) 2024-04-26T21:58:48.794 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 64197 safe_time ceph cwd unknown /proc/63801/task/64197/cwd (readlink: Permission denied) 2024-04-26T21:58:48.794 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 64197 safe_time ceph rtd unknown /proc/63801/task/64197/root (readlink: Permission denied) 2024-04-26T21:58:48.794 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 64197 safe_time ceph txt unknown /proc/63801/task/64197/exe (readlink: Permission denied) 2024-04-26T21:58:48.794 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 64197 safe_time ceph NOFD /proc/63801/task/64197/fd (opendir: Permission denied) 2024-04-26T21:58:48.794 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 64198 fn_anonym ceph cwd unknown /proc/63801/task/64198/cwd (readlink: Permission denied) 2024-04-26T21:58:48.794 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 64198 fn_anonym ceph rtd unknown /proc/63801/task/64198/root (readlink: Permission denied) 2024-04-26T21:58:48.794 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 64198 fn_anonym ceph txt unknown /proc/63801/task/64198/exe (readlink: Permission denied) 2024-04-26T21:58:48.794 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 64198 fn_anonym ceph NOFD /proc/63801/task/64198/fd (opendir: Permission denied) 2024-04-26T21:58:48.794 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 64202 rocksdb:l ceph cwd unknown /proc/63801/task/64202/cwd (readlink: Permission denied) 2024-04-26T21:58:48.794 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 64202 rocksdb:l ceph rtd unknown /proc/63801/task/64202/root (readlink: Permission denied) 2024-04-26T21:58:48.795 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 64202 rocksdb:l ceph txt unknown /proc/63801/task/64202/exe (readlink: Permission denied) 2024-04-26T21:58:48.795 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 64202 rocksdb:l ceph NOFD /proc/63801/task/64202/fd (opendir: Permission denied) 2024-04-26T21:58:48.795 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 64203 rocksdb:l ceph cwd unknown /proc/63801/task/64203/cwd (readlink: Permission denied) 2024-04-26T21:58:48.795 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 64203 rocksdb:l ceph rtd unknown /proc/63801/task/64203/root (readlink: Permission denied) 2024-04-26T21:58:48.795 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 64203 rocksdb:l ceph txt unknown /proc/63801/task/64203/exe (readlink: Permission denied) 2024-04-26T21:58:48.795 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 64203 rocksdb:l ceph NOFD /proc/63801/task/64203/fd (opendir: Permission denied) 2024-04-26T21:58:48.795 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 64204 rocksdb:l ceph cwd unknown /proc/63801/task/64204/cwd (readlink: Permission denied) 2024-04-26T21:58:48.795 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 64204 rocksdb:l ceph rtd unknown /proc/63801/task/64204/root (readlink: Permission denied) 2024-04-26T21:58:48.795 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 64204 rocksdb:l ceph txt unknown /proc/63801/task/64204/exe (readlink: Permission denied) 2024-04-26T21:58:48.795 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 64204 rocksdb:l ceph NOFD /proc/63801/task/64204/fd (opendir: Permission denied) 2024-04-26T21:58:48.795 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 64205 rocksdb:h ceph cwd unknown /proc/63801/task/64205/cwd (readlink: Permission denied) 2024-04-26T21:58:48.796 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 64205 rocksdb:h ceph rtd unknown /proc/63801/task/64205/root (readlink: Permission denied) 2024-04-26T21:58:48.796 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 64205 rocksdb:h ceph txt unknown /proc/63801/task/64205/exe (readlink: Permission denied) 2024-04-26T21:58:48.796 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 64205 rocksdb:h ceph NOFD /proc/63801/task/64205/fd (opendir: Permission denied) 2024-04-26T21:58:48.796 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 64664 bstore_ai ceph cwd unknown /proc/63801/task/64664/cwd (readlink: Permission denied) 2024-04-26T21:58:48.796 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 64664 bstore_ai ceph rtd unknown /proc/63801/task/64664/root (readlink: Permission denied) 2024-04-26T21:58:48.796 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 64664 bstore_ai ceph txt unknown /proc/63801/task/64664/exe (readlink: Permission denied) 2024-04-26T21:58:48.796 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 64664 bstore_ai ceph NOFD /proc/63801/task/64664/fd (opendir: Permission denied) 2024-04-26T21:58:48.796 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 64939 bstore_ai ceph cwd unknown /proc/63801/task/64939/cwd (readlink: Permission denied) 2024-04-26T21:58:48.796 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 64939 bstore_ai ceph rtd unknown /proc/63801/task/64939/root (readlink: Permission denied) 2024-04-26T21:58:48.796 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 64939 bstore_ai ceph txt unknown /proc/63801/task/64939/exe (readlink: Permission denied) 2024-04-26T21:58:48.796 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 64939 bstore_ai ceph NOFD /proc/63801/task/64939/fd (opendir: Permission denied) 2024-04-26T21:58:48.796 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 65136 ceph-osd ceph cwd unknown /proc/63801/task/65136/cwd (readlink: Permission denied) 2024-04-26T21:58:48.796 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 65136 ceph-osd ceph rtd unknown /proc/63801/task/65136/root (readlink: Permission denied) 2024-04-26T21:58:48.797 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 65136 ceph-osd ceph txt unknown /proc/63801/task/65136/exe (readlink: Permission denied) 2024-04-26T21:58:48.797 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 65136 ceph-osd ceph NOFD /proc/63801/task/65136/fd (opendir: Permission denied) 2024-04-26T21:58:48.797 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 65137 cfin ceph cwd unknown /proc/63801/task/65137/cwd (readlink: Permission denied) 2024-04-26T21:58:48.797 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 65137 cfin ceph rtd unknown /proc/63801/task/65137/root (readlink: Permission denied) 2024-04-26T21:58:48.797 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 65137 cfin ceph txt unknown /proc/63801/task/65137/exe (readlink: Permission denied) 2024-04-26T21:58:48.797 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 65137 cfin ceph NOFD /proc/63801/task/65137/fd (opendir: Permission denied) 2024-04-26T21:58:48.797 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 65138 bstore_kv ceph cwd unknown /proc/63801/task/65138/cwd (readlink: Permission denied) 2024-04-26T21:58:48.797 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 65138 bstore_kv ceph rtd unknown /proc/63801/task/65138/root (readlink: Permission denied) 2024-04-26T21:58:48.797 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 65138 bstore_kv ceph txt unknown /proc/63801/task/65138/exe (readlink: Permission denied) 2024-04-26T21:58:48.797 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 65138 bstore_kv ceph NOFD /proc/63801/task/65138/fd (opendir: Permission denied) 2024-04-26T21:58:48.797 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 65139 bstore_kv ceph cwd unknown /proc/63801/task/65139/cwd (readlink: Permission denied) 2024-04-26T21:58:48.797 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 65139 bstore_kv ceph rtd unknown /proc/63801/task/65139/root (readlink: Permission denied) 2024-04-26T21:58:48.797 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 65139 bstore_kv ceph txt unknown /proc/63801/task/65139/exe (readlink: Permission denied) 2024-04-26T21:58:48.797 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 65139 bstore_kv ceph NOFD /proc/63801/task/65139/fd (opendir: Permission denied) 2024-04-26T21:58:48.798 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 65140 bstore_me ceph cwd unknown /proc/63801/task/65140/cwd (readlink: Permission denied) 2024-04-26T21:58:48.798 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 65140 bstore_me ceph rtd unknown /proc/63801/task/65140/root (readlink: Permission denied) 2024-04-26T21:58:48.798 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 65140 bstore_me ceph txt unknown /proc/63801/task/65140/exe (readlink: Permission denied) 2024-04-26T21:58:48.798 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 65140 bstore_me ceph NOFD /proc/63801/task/65140/fd (opendir: Permission denied) 2024-04-26T21:58:48.798 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 65143 ms_dispat ceph cwd unknown /proc/63801/task/65143/cwd (readlink: Permission denied) 2024-04-26T21:58:48.798 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 65143 ms_dispat ceph rtd unknown /proc/63801/task/65143/root (readlink: Permission denied) 2024-04-26T21:58:48.798 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 65143 ms_dispat ceph txt unknown /proc/63801/task/65143/exe (readlink: Permission denied) 2024-04-26T21:58:48.798 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 65143 ms_dispat ceph NOFD /proc/63801/task/65143/fd (opendir: Permission denied) 2024-04-26T21:58:48.798 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 65144 ms_local ceph cwd unknown /proc/63801/task/65144/cwd (readlink: Permission denied) 2024-04-26T21:58:48.798 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 65144 ms_local ceph rtd unknown /proc/63801/task/65144/root (readlink: Permission denied) 2024-04-26T21:58:48.798 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 65144 ms_local ceph txt unknown /proc/63801/task/65144/exe (readlink: Permission denied) 2024-04-26T21:58:48.798 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 65144 ms_local ceph NOFD /proc/63801/task/65144/fd (opendir: Permission denied) 2024-04-26T21:58:48.798 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 65145 safe_time ceph cwd unknown /proc/63801/task/65145/cwd (readlink: Permission denied) 2024-04-26T21:58:48.799 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 65145 safe_time ceph rtd unknown /proc/63801/task/65145/root (readlink: Permission denied) 2024-04-26T21:58:48.799 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 65145 safe_time ceph txt unknown /proc/63801/task/65145/exe (readlink: Permission denied) 2024-04-26T21:58:48.799 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 65145 safe_time ceph NOFD /proc/63801/task/65145/fd (opendir: Permission denied) 2024-04-26T21:58:48.799 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 65146 safe_time ceph cwd unknown /proc/63801/task/65146/cwd (readlink: Permission denied) 2024-04-26T21:58:48.799 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 65146 safe_time ceph rtd unknown /proc/63801/task/65146/root (readlink: Permission denied) 2024-04-26T21:58:48.799 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 65146 safe_time ceph txt unknown /proc/63801/task/65146/exe (readlink: Permission denied) 2024-04-26T21:58:48.799 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 65146 safe_time ceph NOFD /proc/63801/task/65146/fd (opendir: Permission denied) 2024-04-26T21:58:48.799 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 65147 ms_dispat ceph cwd unknown /proc/63801/task/65147/cwd (readlink: Permission denied) 2024-04-26T21:58:48.799 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 65147 ms_dispat ceph rtd unknown /proc/63801/task/65147/root (readlink: Permission denied) 2024-04-26T21:58:48.799 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 65147 ms_dispat ceph txt unknown /proc/63801/task/65147/exe (readlink: Permission denied) 2024-04-26T21:58:48.799 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 65147 ms_dispat ceph NOFD /proc/63801/task/65147/fd (opendir: Permission denied) 2024-04-26T21:58:48.799 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 65148 ms_local ceph cwd unknown /proc/63801/task/65148/cwd (readlink: Permission denied) 2024-04-26T21:58:48.799 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 65148 ms_local ceph rtd unknown /proc/63801/task/65148/root (readlink: Permission denied) 2024-04-26T21:58:48.800 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 65148 ms_local ceph txt unknown /proc/63801/task/65148/exe (readlink: Permission denied) 2024-04-26T21:58:48.800 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 65148 ms_local ceph NOFD /proc/63801/task/65148/fd (opendir: Permission denied) 2024-04-26T21:58:48.800 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 65149 ms_dispat ceph cwd unknown /proc/63801/task/65149/cwd (readlink: Permission denied) 2024-04-26T21:58:48.800 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 65149 ms_dispat ceph rtd unknown /proc/63801/task/65149/root (readlink: Permission denied) 2024-04-26T21:58:48.800 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 65149 ms_dispat ceph txt unknown /proc/63801/task/65149/exe (readlink: Permission denied) 2024-04-26T21:58:48.800 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 65149 ms_dispat ceph NOFD /proc/63801/task/65149/fd (opendir: Permission denied) 2024-04-26T21:58:48.800 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 65150 ms_local ceph cwd unknown /proc/63801/task/65150/cwd (readlink: Permission denied) 2024-04-26T21:58:48.800 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 65150 ms_local ceph rtd unknown /proc/63801/task/65150/root (readlink: Permission denied) 2024-04-26T21:58:48.800 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 65150 ms_local ceph txt unknown /proc/63801/task/65150/exe (readlink: Permission denied) 2024-04-26T21:58:48.800 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 65150 ms_local ceph NOFD /proc/63801/task/65150/fd (opendir: Permission denied) 2024-04-26T21:58:48.800 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 65151 ms_dispat ceph cwd unknown /proc/63801/task/65151/cwd (readlink: Permission denied) 2024-04-26T21:58:48.800 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 65151 ms_dispat ceph rtd unknown /proc/63801/task/65151/root (readlink: Permission denied) 2024-04-26T21:58:48.800 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 65151 ms_dispat ceph txt unknown /proc/63801/task/65151/exe (readlink: Permission denied) 2024-04-26T21:58:48.801 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 65151 ms_dispat ceph NOFD /proc/63801/task/65151/fd (opendir: Permission denied) 2024-04-26T21:58:48.801 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 65152 ms_local ceph cwd unknown /proc/63801/task/65152/cwd (readlink: Permission denied) 2024-04-26T21:58:48.801 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 65152 ms_local ceph rtd unknown /proc/63801/task/65152/root (readlink: Permission denied) 2024-04-26T21:58:48.801 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 65152 ms_local ceph txt unknown /proc/63801/task/65152/exe (readlink: Permission denied) 2024-04-26T21:58:48.801 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 65152 ms_local ceph NOFD /proc/63801/task/65152/fd (opendir: Permission denied) 2024-04-26T21:58:48.801 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 65153 ms_dispat ceph cwd unknown /proc/63801/task/65153/cwd (readlink: Permission denied) 2024-04-26T21:58:48.801 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 65153 ms_dispat ceph rtd unknown /proc/63801/task/65153/root (readlink: Permission denied) 2024-04-26T21:58:48.801 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 65153 ms_dispat ceph txt unknown /proc/63801/task/65153/exe (readlink: Permission denied) 2024-04-26T21:58:48.801 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 65153 ms_dispat ceph NOFD /proc/63801/task/65153/fd (opendir: Permission denied) 2024-04-26T21:58:48.801 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 65154 ms_local ceph cwd unknown /proc/63801/task/65154/cwd (readlink: Permission denied) 2024-04-26T21:58:48.801 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 65154 ms_local ceph rtd unknown /proc/63801/task/65154/root (readlink: Permission denied) 2024-04-26T21:58:48.801 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 65154 ms_local ceph txt unknown /proc/63801/task/65154/exe (readlink: Permission denied) 2024-04-26T21:58:48.801 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 65154 ms_local ceph NOFD /proc/63801/task/65154/fd (opendir: Permission denied) 2024-04-26T21:58:48.801 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 65155 ms_dispat ceph cwd unknown /proc/63801/task/65155/cwd (readlink: Permission denied) 2024-04-26T21:58:48.802 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 65155 ms_dispat ceph rtd unknown /proc/63801/task/65155/root (readlink: Permission denied) 2024-04-26T21:58:48.802 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 65155 ms_dispat ceph txt unknown /proc/63801/task/65155/exe (readlink: Permission denied) 2024-04-26T21:58:48.802 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 65155 ms_dispat ceph NOFD /proc/63801/task/65155/fd (opendir: Permission denied) 2024-04-26T21:58:48.802 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 65156 ms_local ceph cwd unknown /proc/63801/task/65156/cwd (readlink: Permission denied) 2024-04-26T21:58:48.802 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 65156 ms_local ceph rtd unknown /proc/63801/task/65156/root (readlink: Permission denied) 2024-04-26T21:58:48.802 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 65156 ms_local ceph txt unknown /proc/63801/task/65156/exe (readlink: Permission denied) 2024-04-26T21:58:48.802 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 65156 ms_local ceph NOFD /proc/63801/task/65156/fd (opendir: Permission denied) 2024-04-26T21:58:48.802 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 65157 ms_dispat ceph cwd unknown /proc/63801/task/65157/cwd (readlink: Permission denied) 2024-04-26T21:58:48.802 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 65157 ms_dispat ceph rtd unknown /proc/63801/task/65157/root (readlink: Permission denied) 2024-04-26T21:58:48.802 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 65157 ms_dispat ceph txt unknown /proc/63801/task/65157/exe (readlink: Permission denied) 2024-04-26T21:58:48.802 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 65157 ms_dispat ceph NOFD /proc/63801/task/65157/fd (opendir: Permission denied) 2024-04-26T21:58:48.802 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 65158 ms_local ceph cwd unknown /proc/63801/task/65158/cwd (readlink: Permission denied) 2024-04-26T21:58:48.803 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 65158 ms_local ceph rtd unknown /proc/63801/task/65158/root (readlink: Permission denied) 2024-04-26T21:58:48.803 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 65158 ms_local ceph txt unknown /proc/63801/task/65158/exe (readlink: Permission denied) 2024-04-26T21:58:48.803 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 65158 ms_local ceph NOFD /proc/63801/task/65158/fd (opendir: Permission denied) 2024-04-26T21:58:48.803 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 65159 fn_anonym ceph cwd unknown /proc/63801/task/65159/cwd (readlink: Permission denied) 2024-04-26T21:58:48.803 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 65159 fn_anonym ceph rtd unknown /proc/63801/task/65159/root (readlink: Permission denied) 2024-04-26T21:58:48.803 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 65159 fn_anonym ceph txt unknown /proc/63801/task/65159/exe (readlink: Permission denied) 2024-04-26T21:58:48.803 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 65159 fn_anonym ceph NOFD /proc/63801/task/65159/fd (opendir: Permission denied) 2024-04-26T21:58:48.803 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 65160 finisher ceph cwd unknown /proc/63801/task/65160/cwd (readlink: Permission denied) 2024-04-26T21:58:48.803 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 65160 finisher ceph rtd unknown /proc/63801/task/65160/root (readlink: Permission denied) 2024-04-26T21:58:48.803 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 65160 finisher ceph txt unknown /proc/63801/task/65160/exe (readlink: Permission denied) 2024-04-26T21:58:48.803 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 65160 finisher ceph NOFD /proc/63801/task/65160/fd (opendir: Permission denied) 2024-04-26T21:58:48.803 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 65161 safe_time ceph cwd unknown /proc/63801/task/65161/cwd (readlink: Permission denied) 2024-04-26T21:58:48.804 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 65161 safe_time ceph rtd unknown /proc/63801/task/65161/root (readlink: Permission denied) 2024-04-26T21:58:48.804 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 65161 safe_time ceph txt unknown /proc/63801/task/65161/exe (readlink: Permission denied) 2024-04-26T21:58:48.804 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 65161 safe_time ceph NOFD /proc/63801/task/65161/fd (opendir: Permission denied) 2024-04-26T21:58:48.804 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 65162 safe_time ceph cwd unknown /proc/63801/task/65162/cwd (readlink: Permission denied) 2024-04-26T21:58:48.804 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 65162 safe_time ceph rtd unknown /proc/63801/task/65162/root (readlink: Permission denied) 2024-04-26T21:58:48.804 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 65162 safe_time ceph txt unknown /proc/63801/task/65162/exe (readlink: Permission denied) 2024-04-26T21:58:48.804 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 65162 safe_time ceph NOFD /proc/63801/task/65162/fd (opendir: Permission denied) 2024-04-26T21:58:48.804 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 65163 ceph-osd ceph cwd unknown /proc/63801/task/65163/cwd (readlink: Permission denied) 2024-04-26T21:58:48.804 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 65163 ceph-osd ceph rtd unknown /proc/63801/task/65163/root (readlink: Permission denied) 2024-04-26T21:58:48.804 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 65163 ceph-osd ceph txt unknown /proc/63801/task/65163/exe (readlink: Permission denied) 2024-04-26T21:58:48.804 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 65163 ceph-osd ceph NOFD /proc/63801/task/65163/fd (opendir: Permission denied) 2024-04-26T21:58:48.804 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 65164 osd_srv_a ceph cwd unknown /proc/63801/task/65164/cwd (readlink: Permission denied) 2024-04-26T21:58:48.804 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 65164 osd_srv_a ceph rtd unknown /proc/63801/task/65164/root (readlink: Permission denied) 2024-04-26T21:58:48.805 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 65164 osd_srv_a ceph txt unknown /proc/63801/task/65164/exe (readlink: Permission denied) 2024-04-26T21:58:48.805 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 65164 osd_srv_a ceph NOFD /proc/63801/task/65164/fd (opendir: Permission denied) 2024-04-26T21:58:48.805 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 65165 tp_osd_tp ceph cwd unknown /proc/63801/task/65165/cwd (readlink: Permission denied) 2024-04-26T21:58:48.805 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 65165 tp_osd_tp ceph rtd unknown /proc/63801/task/65165/root (readlink: Permission denied) 2024-04-26T21:58:48.805 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 65165 tp_osd_tp ceph txt unknown /proc/63801/task/65165/exe (readlink: Permission denied) 2024-04-26T21:58:48.805 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 65165 tp_osd_tp ceph NOFD /proc/63801/task/65165/fd (opendir: Permission denied) 2024-04-26T21:58:48.805 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 65166 tp_osd_tp ceph cwd unknown /proc/63801/task/65166/cwd (readlink: Permission denied) 2024-04-26T21:58:48.805 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 65166 tp_osd_tp ceph rtd unknown /proc/63801/task/65166/root (readlink: Permission denied) 2024-04-26T21:58:48.805 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 65166 tp_osd_tp ceph txt unknown /proc/63801/task/65166/exe (readlink: Permission denied) 2024-04-26T21:58:48.805 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 65166 tp_osd_tp ceph NOFD /proc/63801/task/65166/fd (opendir: Permission denied) 2024-04-26T21:58:48.805 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 65167 tp_osd_tp ceph cwd unknown /proc/63801/task/65167/cwd (readlink: Permission denied) 2024-04-26T21:58:48.805 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 65167 tp_osd_tp ceph rtd unknown /proc/63801/task/65167/root (readlink: Permission denied) 2024-04-26T21:58:48.805 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 65167 tp_osd_tp ceph txt unknown /proc/63801/task/65167/exe (readlink: Permission denied) 2024-04-26T21:58:48.805 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 65167 tp_osd_tp ceph NOFD /proc/63801/task/65167/fd (opendir: Permission denied) 2024-04-26T21:58:48.806 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 65168 tp_osd_tp ceph cwd unknown /proc/63801/task/65168/cwd (readlink: Permission denied) 2024-04-26T21:58:48.806 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 65168 tp_osd_tp ceph rtd unknown /proc/63801/task/65168/root (readlink: Permission denied) 2024-04-26T21:58:48.806 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 65168 tp_osd_tp ceph txt unknown /proc/63801/task/65168/exe (readlink: Permission denied) 2024-04-26T21:58:48.806 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 65168 tp_osd_tp ceph NOFD /proc/63801/task/65168/fd (opendir: Permission denied) 2024-04-26T21:58:48.806 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 65169 tp_osd_tp ceph cwd unknown /proc/63801/task/65169/cwd (readlink: Permission denied) 2024-04-26T21:58:48.806 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 65169 tp_osd_tp ceph rtd unknown /proc/63801/task/65169/root (readlink: Permission denied) 2024-04-26T21:58:48.806 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 65169 tp_osd_tp ceph txt unknown /proc/63801/task/65169/exe (readlink: Permission denied) 2024-04-26T21:58:48.806 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 65169 tp_osd_tp ceph NOFD /proc/63801/task/65169/fd (opendir: Permission denied) 2024-04-26T21:58:48.806 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 65170 tp_osd_tp ceph cwd unknown /proc/63801/task/65170/cwd (readlink: Permission denied) 2024-04-26T21:58:48.806 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 65170 tp_osd_tp ceph rtd unknown /proc/63801/task/65170/root (readlink: Permission denied) 2024-04-26T21:58:48.806 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 65170 tp_osd_tp ceph txt unknown /proc/63801/task/65170/exe (readlink: Permission denied) 2024-04-26T21:58:48.806 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 65170 tp_osd_tp ceph NOFD /proc/63801/task/65170/fd (opendir: Permission denied) 2024-04-26T21:58:48.806 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 65171 tp_osd_tp ceph cwd unknown /proc/63801/task/65171/cwd (readlink: Permission denied) 2024-04-26T21:58:48.806 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 65171 tp_osd_tp ceph rtd unknown /proc/63801/task/65171/root (readlink: Permission denied) 2024-04-26T21:58:48.807 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 65171 tp_osd_tp ceph txt unknown /proc/63801/task/65171/exe (readlink: Permission denied) 2024-04-26T21:58:48.807 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 65171 tp_osd_tp ceph NOFD /proc/63801/task/65171/fd (opendir: Permission denied) 2024-04-26T21:58:48.807 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 65172 tp_osd_tp ceph cwd unknown /proc/63801/task/65172/cwd (readlink: Permission denied) 2024-04-26T21:58:48.807 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 65172 tp_osd_tp ceph rtd unknown /proc/63801/task/65172/root (readlink: Permission denied) 2024-04-26T21:58:48.807 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 65172 tp_osd_tp ceph txt unknown /proc/63801/task/65172/exe (readlink: Permission denied) 2024-04-26T21:58:48.807 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 65172 tp_osd_tp ceph NOFD /proc/63801/task/65172/fd (opendir: Permission denied) 2024-04-26T21:58:48.807 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 65173 tp_osd_tp ceph cwd unknown /proc/63801/task/65173/cwd (readlink: Permission denied) 2024-04-26T21:58:48.807 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 65173 tp_osd_tp ceph rtd unknown /proc/63801/task/65173/root (readlink: Permission denied) 2024-04-26T21:58:48.807 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 65173 tp_osd_tp ceph txt unknown /proc/63801/task/65173/exe (readlink: Permission denied) 2024-04-26T21:58:48.807 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 65173 tp_osd_tp ceph NOFD /proc/63801/task/65173/fd (opendir: Permission denied) 2024-04-26T21:58:48.807 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 65174 tp_osd_tp ceph cwd unknown /proc/63801/task/65174/cwd (readlink: Permission denied) 2024-04-26T21:58:48.807 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 65174 tp_osd_tp ceph rtd unknown /proc/63801/task/65174/root (readlink: Permission denied) 2024-04-26T21:58:48.807 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 65174 tp_osd_tp ceph txt unknown /proc/63801/task/65174/exe (readlink: Permission denied) 2024-04-26T21:58:48.807 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 65174 tp_osd_tp ceph NOFD /proc/63801/task/65174/fd (opendir: Permission denied) 2024-04-26T21:58:48.808 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 65175 tp_osd_tp ceph cwd unknown /proc/63801/task/65175/cwd (readlink: Permission denied) 2024-04-26T21:58:48.808 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 65175 tp_osd_tp ceph rtd unknown /proc/63801/task/65175/root (readlink: Permission denied) 2024-04-26T21:58:48.808 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 65175 tp_osd_tp ceph txt unknown /proc/63801/task/65175/exe (readlink: Permission denied) 2024-04-26T21:58:48.808 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 65175 tp_osd_tp ceph NOFD /proc/63801/task/65175/fd (opendir: Permission denied) 2024-04-26T21:58:48.808 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 65176 tp_osd_tp ceph cwd unknown /proc/63801/task/65176/cwd (readlink: Permission denied) 2024-04-26T21:58:48.808 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 65176 tp_osd_tp ceph rtd unknown /proc/63801/task/65176/root (readlink: Permission denied) 2024-04-26T21:58:48.808 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 65176 tp_osd_tp ceph txt unknown /proc/63801/task/65176/exe (readlink: Permission denied) 2024-04-26T21:58:48.808 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 65176 tp_osd_tp ceph NOFD /proc/63801/task/65176/fd (opendir: Permission denied) 2024-04-26T21:58:48.808 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 65177 tp_osd_tp ceph cwd unknown /proc/63801/task/65177/cwd (readlink: Permission denied) 2024-04-26T21:58:48.808 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 65177 tp_osd_tp ceph rtd unknown /proc/63801/task/65177/root (readlink: Permission denied) 2024-04-26T21:58:48.808 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 65177 tp_osd_tp ceph txt unknown /proc/63801/task/65177/exe (readlink: Permission denied) 2024-04-26T21:58:48.808 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 65177 tp_osd_tp ceph NOFD /proc/63801/task/65177/fd (opendir: Permission denied) 2024-04-26T21:58:48.808 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 65178 tp_osd_tp ceph cwd unknown /proc/63801/task/65178/cwd (readlink: Permission denied) 2024-04-26T21:58:48.808 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 65178 tp_osd_tp ceph rtd unknown /proc/63801/task/65178/root (readlink: Permission denied) 2024-04-26T21:58:48.809 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 65178 tp_osd_tp ceph txt unknown /proc/63801/task/65178/exe (readlink: Permission denied) 2024-04-26T21:58:48.809 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 65178 tp_osd_tp ceph NOFD /proc/63801/task/65178/fd (opendir: Permission denied) 2024-04-26T21:58:48.809 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 65179 tp_osd_tp ceph cwd unknown /proc/63801/task/65179/cwd (readlink: Permission denied) 2024-04-26T21:58:48.809 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 65179 tp_osd_tp ceph rtd unknown /proc/63801/task/65179/root (readlink: Permission denied) 2024-04-26T21:58:48.809 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 65179 tp_osd_tp ceph txt unknown /proc/63801/task/65179/exe (readlink: Permission denied) 2024-04-26T21:58:48.809 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 65179 tp_osd_tp ceph NOFD /proc/63801/task/65179/fd (opendir: Permission denied) 2024-04-26T21:58:48.809 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 65180 tp_osd_tp ceph cwd unknown /proc/63801/task/65180/cwd (readlink: Permission denied) 2024-04-26T21:58:48.809 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 65180 tp_osd_tp ceph rtd unknown /proc/63801/task/65180/root (readlink: Permission denied) 2024-04-26T21:58:48.809 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 65180 tp_osd_tp ceph txt unknown /proc/63801/task/65180/exe (readlink: Permission denied) 2024-04-26T21:58:48.809 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 65180 tp_osd_tp ceph NOFD /proc/63801/task/65180/fd (opendir: Permission denied) 2024-04-26T21:58:48.809 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 65181 osd_srv_h ceph cwd unknown /proc/63801/task/65181/cwd (readlink: Permission denied) 2024-04-26T21:58:48.809 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 65181 osd_srv_h ceph rtd unknown /proc/63801/task/65181/root (readlink: Permission denied) 2024-04-26T21:58:48.809 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 65181 osd_srv_h ceph txt unknown /proc/63801/task/65181/exe (readlink: Permission denied) 2024-04-26T21:58:48.809 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-osd 63801 65181 osd_srv_h ceph NOFD /proc/63801/task/65181/fd (opendir: Permission denied) 2024-04-26T21:58:48.810 INFO:tasks.workunit.client.0.smithi072.stdout:sudo 65315 root cwd unknown /proc/65315/cwd (readlink: Permission denied) 2024-04-26T21:58:48.810 INFO:tasks.workunit.client.0.smithi072.stdout:sudo 65315 root rtd unknown /proc/65315/root (readlink: Permission denied) 2024-04-26T21:58:48.810 INFO:tasks.workunit.client.0.smithi072.stdout:sudo 65315 root txt unknown /proc/65315/exe (readlink: Permission denied) 2024-04-26T21:58:48.810 INFO:tasks.workunit.client.0.smithi072.stdout:sudo 65315 root NOFD /proc/65315/fd (opendir: Permission denied) 2024-04-26T21:58:48.810 INFO:tasks.workunit.client.0.smithi072.stdout:journalct 65343 root cwd unknown /proc/65343/cwd (readlink: Permission denied) 2024-04-26T21:58:48.810 INFO:tasks.workunit.client.0.smithi072.stdout:journalct 65343 root rtd unknown /proc/65343/root (readlink: Permission denied) 2024-04-26T21:58:48.810 INFO:tasks.workunit.client.0.smithi072.stdout:journalct 65343 root txt unknown /proc/65343/exe (readlink: Permission denied) 2024-04-26T21:58:48.810 INFO:tasks.workunit.client.0.smithi072.stdout:journalct 65343 root NOFD /proc/65343/fd (opendir: Permission denied) 2024-04-26T21:58:48.810 INFO:tasks.workunit.client.0.smithi072.stdout:sudo 68483 root cwd unknown /proc/68483/cwd (readlink: Permission denied) 2024-04-26T21:58:48.810 INFO:tasks.workunit.client.0.smithi072.stdout:sudo 68483 root rtd unknown /proc/68483/root (readlink: Permission denied) 2024-04-26T21:58:48.810 INFO:tasks.workunit.client.0.smithi072.stdout:sudo 68483 root txt unknown /proc/68483/exe (readlink: Permission denied) 2024-04-26T21:58:48.810 INFO:tasks.workunit.client.0.smithi072.stdout:sudo 68483 root NOFD /proc/68483/fd (opendir: Permission denied) 2024-04-26T21:58:48.810 INFO:tasks.workunit.client.0.smithi072.stdout:sudo 68499 root cwd unknown /proc/68499/cwd (readlink: Permission denied) 2024-04-26T21:58:48.811 INFO:tasks.workunit.client.0.smithi072.stdout:sudo 68499 root rtd unknown /proc/68499/root (readlink: Permission denied) 2024-04-26T21:58:48.811 INFO:tasks.workunit.client.0.smithi072.stdout:sudo 68499 root txt unknown /proc/68499/exe (readlink: Permission denied) 2024-04-26T21:58:48.811 INFO:tasks.workunit.client.0.smithi072.stdout:sudo 68499 root NOFD /proc/68499/fd (opendir: Permission denied) 2024-04-26T21:58:48.811 INFO:tasks.workunit.client.0.smithi072.stdout:sudo 68505 root cwd unknown /proc/68505/cwd (readlink: Permission denied) 2024-04-26T21:58:48.811 INFO:tasks.workunit.client.0.smithi072.stdout:sudo 68505 root rtd unknown /proc/68505/root (readlink: Permission denied) 2024-04-26T21:58:48.811 INFO:tasks.workunit.client.0.smithi072.stdout:sudo 68505 root txt unknown /proc/68505/exe (readlink: Permission denied) 2024-04-26T21:58:48.811 INFO:tasks.workunit.client.0.smithi072.stdout:sudo 68505 root NOFD /proc/68505/fd (opendir: Permission denied) 2024-04-26T21:58:48.811 INFO:tasks.workunit.client.0.smithi072.stdout:journalct 68518 root cwd unknown /proc/68518/cwd (readlink: Permission denied) 2024-04-26T21:58:48.811 INFO:tasks.workunit.client.0.smithi072.stdout:journalct 68518 root rtd unknown /proc/68518/root (readlink: Permission denied) 2024-04-26T21:58:48.811 INFO:tasks.workunit.client.0.smithi072.stdout:journalct 68518 root txt unknown /proc/68518/exe (readlink: Permission denied) 2024-04-26T21:58:48.811 INFO:tasks.workunit.client.0.smithi072.stdout:journalct 68518 root NOFD /proc/68518/fd (opendir: Permission denied) 2024-04-26T21:58:48.811 INFO:tasks.workunit.client.0.smithi072.stdout:sudo 68534 root cwd unknown /proc/68534/cwd (readlink: Permission denied) 2024-04-26T21:58:48.811 INFO:tasks.workunit.client.0.smithi072.stdout:sudo 68534 root rtd unknown /proc/68534/root (readlink: Permission denied) 2024-04-26T21:58:48.812 INFO:tasks.workunit.client.0.smithi072.stdout:sudo 68534 root txt unknown /proc/68534/exe (readlink: Permission denied) 2024-04-26T21:58:48.812 INFO:tasks.workunit.client.0.smithi072.stdout:sudo 68534 root NOFD /proc/68534/fd (opendir: Permission denied) 2024-04-26T21:58:48.812 INFO:tasks.workunit.client.0.smithi072.stdout:journalct 68535 root cwd unknown /proc/68535/cwd (readlink: Permission denied) 2024-04-26T21:58:48.812 INFO:tasks.workunit.client.0.smithi072.stdout:journalct 68535 root rtd unknown /proc/68535/root (readlink: Permission denied) 2024-04-26T21:58:48.812 INFO:tasks.workunit.client.0.smithi072.stdout:journalct 68535 root txt unknown /proc/68535/exe (readlink: Permission denied) 2024-04-26T21:58:48.812 INFO:tasks.workunit.client.0.smithi072.stdout:journalct 68535 root NOFD /proc/68535/fd (opendir: Permission denied) 2024-04-26T21:58:48.812 INFO:tasks.workunit.client.0.smithi072.stdout:journalct 68542 root cwd unknown /proc/68542/cwd (readlink: Permission denied) 2024-04-26T21:58:48.812 INFO:tasks.workunit.client.0.smithi072.stdout:journalct 68542 root rtd unknown /proc/68542/root (readlink: Permission denied) 2024-04-26T21:58:48.812 INFO:tasks.workunit.client.0.smithi072.stdout:journalct 68542 root txt unknown /proc/68542/exe (readlink: Permission denied) 2024-04-26T21:58:48.812 INFO:tasks.workunit.client.0.smithi072.stdout:journalct 68542 root NOFD /proc/68542/fd (opendir: Permission denied) 2024-04-26T21:58:48.812 INFO:tasks.workunit.client.0.smithi072.stdout:journalct 68554 root cwd unknown /proc/68554/cwd (readlink: Permission denied) 2024-04-26T21:58:48.812 INFO:tasks.workunit.client.0.smithi072.stdout:journalct 68554 root rtd unknown /proc/68554/root (readlink: Permission denied) 2024-04-26T21:58:48.812 INFO:tasks.workunit.client.0.smithi072.stdout:journalct 68554 root txt unknown /proc/68554/exe (readlink: Permission denied) 2024-04-26T21:58:48.812 INFO:tasks.workunit.client.0.smithi072.stdout:journalct 68554 root NOFD /proc/68554/fd (opendir: Permission denied) 2024-04-26T21:58:48.812 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/4 68929 root cwd unknown /proc/68929/cwd (readlink: Permission denied) 2024-04-26T21:58:48.813 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/4 68929 root rtd unknown /proc/68929/root (readlink: Permission denied) 2024-04-26T21:58:48.813 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/4 68929 root txt unknown /proc/68929/exe (readlink: Permission denied) 2024-04-26T21:58:48.813 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/4 68929 root NOFD /proc/68929/fd (opendir: Permission denied) 2024-04-26T21:58:48.813 INFO:tasks.workunit.client.0.smithi072.stdout:conmon 69091 root cwd unknown /proc/69091/cwd (readlink: Permission denied) 2024-04-26T21:58:48.813 INFO:tasks.workunit.client.0.smithi072.stdout:conmon 69091 root rtd unknown /proc/69091/root (readlink: Permission denied) 2024-04-26T21:58:48.813 INFO:tasks.workunit.client.0.smithi072.stdout:conmon 69091 root txt unknown /proc/69091/exe (readlink: Permission denied) 2024-04-26T21:58:48.813 INFO:tasks.workunit.client.0.smithi072.stdout:conmon 69091 root NOFD /proc/69091/fd (opendir: Permission denied) 2024-04-26T21:58:48.813 INFO:tasks.workunit.client.0.smithi072.stdout:podman-in 69093 root cwd unknown /proc/69093/cwd (readlink: Permission denied) 2024-04-26T21:58:48.813 INFO:tasks.workunit.client.0.smithi072.stdout:podman-in 69093 root rtd unknown /proc/69093/root (readlink: Permission denied) 2024-04-26T21:58:48.813 INFO:tasks.workunit.client.0.smithi072.stdout:podman-in 69093 root txt unknown /proc/69093/exe (readlink: Permission denied) 2024-04-26T21:58:48.813 INFO:tasks.workunit.client.0.smithi072.stdout:podman-in 69093 root NOFD /proc/69093/fd (opendir: Permission denied) 2024-04-26T21:58:48.813 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 69095 ceph cwd unknown /proc/69095/cwd (readlink: Permission denied) 2024-04-26T21:58:48.813 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 69095 ceph rtd unknown /proc/69095/root (readlink: Permission denied) 2024-04-26T21:58:48.813 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 69095 ceph txt unknown /proc/69095/exe (readlink: Permission denied) 2024-04-26T21:58:48.813 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 69095 ceph NOFD /proc/69095/fd (opendir: Permission denied) 2024-04-26T21:58:48.814 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 69095 69103 log ceph cwd unknown /proc/69095/task/69103/cwd (readlink: Permission denied) 2024-04-26T21:58:48.814 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 69095 69103 log ceph rtd unknown /proc/69095/task/69103/root (readlink: Permission denied) 2024-04-26T21:58:48.814 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 69095 69103 log ceph txt unknown /proc/69095/task/69103/exe (readlink: Permission denied) 2024-04-26T21:58:48.814 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 69095 69103 log ceph NOFD /proc/69095/task/69103/fd (opendir: Permission denied) 2024-04-26T21:58:48.814 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 69095 69106 msgr-work ceph cwd unknown /proc/69095/task/69106/cwd (readlink: Permission denied) 2024-04-26T21:58:48.814 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 69095 69106 msgr-work ceph rtd unknown /proc/69095/task/69106/root (readlink: Permission denied) 2024-04-26T21:58:48.814 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 69095 69106 msgr-work ceph txt unknown /proc/69095/task/69106/exe (readlink: Permission denied) 2024-04-26T21:58:48.814 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 69095 69106 msgr-work ceph NOFD /proc/69095/task/69106/fd (opendir: Permission denied) 2024-04-26T21:58:48.814 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 69095 69107 msgr-work ceph cwd unknown /proc/69095/task/69107/cwd (readlink: Permission denied) 2024-04-26T21:58:48.814 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 69095 69107 msgr-work ceph rtd unknown /proc/69095/task/69107/root (readlink: Permission denied) 2024-04-26T21:58:48.814 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 69095 69107 msgr-work ceph txt unknown /proc/69095/task/69107/exe (readlink: Permission denied) 2024-04-26T21:58:48.814 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 69095 69107 msgr-work ceph NOFD /proc/69095/task/69107/fd (opendir: Permission denied) 2024-04-26T21:58:48.814 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 69095 69108 msgr-work ceph cwd unknown /proc/69095/task/69108/cwd (readlink: Permission denied) 2024-04-26T21:58:48.815 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 69095 69108 msgr-work ceph rtd unknown /proc/69095/task/69108/root (readlink: Permission denied) 2024-04-26T21:58:48.815 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 69095 69108 msgr-work ceph txt unknown /proc/69095/task/69108/exe (readlink: Permission denied) 2024-04-26T21:58:48.815 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 69095 69108 msgr-work ceph NOFD /proc/69095/task/69108/fd (opendir: Permission denied) 2024-04-26T21:58:48.815 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 69095 69115 service ceph cwd unknown /proc/69095/task/69115/cwd (readlink: Permission denied) 2024-04-26T21:58:48.815 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 69095 69115 service ceph rtd unknown /proc/69095/task/69115/root (readlink: Permission denied) 2024-04-26T21:58:48.815 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 69095 69115 service ceph txt unknown /proc/69095/task/69115/exe (readlink: Permission denied) 2024-04-26T21:58:48.815 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 69095 69115 service ceph NOFD /proc/69095/task/69115/fd (opendir: Permission denied) 2024-04-26T21:58:48.815 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 69095 69116 admin_soc ceph cwd unknown /proc/69095/task/69116/cwd (readlink: Permission denied) 2024-04-26T21:58:48.815 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 69095 69116 admin_soc ceph rtd unknown /proc/69095/task/69116/root (readlink: Permission denied) 2024-04-26T21:58:48.815 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 69095 69116 admin_soc ceph txt unknown /proc/69095/task/69116/exe (readlink: Permission denied) 2024-04-26T21:58:48.815 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 69095 69116 admin_soc ceph NOFD /proc/69095/task/69116/fd (opendir: Permission denied) 2024-04-26T21:58:48.815 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 69095 69119 signal_ha ceph cwd unknown /proc/69095/task/69119/cwd (readlink: Permission denied) 2024-04-26T21:58:48.815 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 69095 69119 signal_ha ceph rtd unknown /proc/69095/task/69119/root (readlink: Permission denied) 2024-04-26T21:58:48.816 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 69095 69119 signal_ha ceph txt unknown /proc/69095/task/69119/exe (readlink: Permission denied) 2024-04-26T21:58:48.816 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 69095 69119 signal_ha ceph NOFD /proc/69095/task/69119/fd (opendir: Permission denied) 2024-04-26T21:58:48.816 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 69095 69120 io_contex ceph cwd unknown /proc/69095/task/69120/cwd (readlink: Permission denied) 2024-04-26T21:58:48.816 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 69095 69120 io_contex ceph rtd unknown /proc/69095/task/69120/root (readlink: Permission denied) 2024-04-26T21:58:48.816 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 69095 69120 io_contex ceph txt unknown /proc/69095/task/69120/exe (readlink: Permission denied) 2024-04-26T21:58:48.816 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 69095 69120 io_contex ceph NOFD /proc/69095/task/69120/fd (opendir: Permission denied) 2024-04-26T21:58:48.816 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 69095 69121 io_contex ceph cwd unknown /proc/69095/task/69121/cwd (readlink: Permission denied) 2024-04-26T21:58:48.816 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 69095 69121 io_contex ceph rtd unknown /proc/69095/task/69121/root (readlink: Permission denied) 2024-04-26T21:58:48.816 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 69095 69121 io_contex ceph txt unknown /proc/69095/task/69121/exe (readlink: Permission denied) 2024-04-26T21:58:48.816 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 69095 69121 io_contex ceph NOFD /proc/69095/task/69121/fd (opendir: Permission denied) 2024-04-26T21:58:48.816 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 69095 69123 ms_dispat ceph cwd unknown /proc/69095/task/69123/cwd (readlink: Permission denied) 2024-04-26T21:58:48.816 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 69095 69123 ms_dispat ceph rtd unknown /proc/69095/task/69123/root (readlink: Permission denied) 2024-04-26T21:58:48.816 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 69095 69123 ms_dispat ceph txt unknown /proc/69095/task/69123/exe (readlink: Permission denied) 2024-04-26T21:58:48.816 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 69095 69123 ms_dispat ceph NOFD /proc/69095/task/69123/fd (opendir: Permission denied) 2024-04-26T21:58:48.817 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 69095 69124 ms_local ceph cwd unknown /proc/69095/task/69124/cwd (readlink: Permission denied) 2024-04-26T21:58:48.817 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 69095 69124 ms_local ceph rtd unknown /proc/69095/task/69124/root (readlink: Permission denied) 2024-04-26T21:58:48.817 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 69095 69124 ms_local ceph txt unknown /proc/69095/task/69124/exe (readlink: Permission denied) 2024-04-26T21:58:48.817 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 69095 69124 ms_local ceph NOFD /proc/69095/task/69124/fd (opendir: Permission denied) 2024-04-26T21:58:48.817 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 69095 69125 safe_time ceph cwd unknown /proc/69095/task/69125/cwd (readlink: Permission denied) 2024-04-26T21:58:48.817 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 69095 69125 safe_time ceph rtd unknown /proc/69095/task/69125/root (readlink: Permission denied) 2024-04-26T21:58:48.817 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 69095 69125 safe_time ceph txt unknown /proc/69095/task/69125/exe (readlink: Permission denied) 2024-04-26T21:58:48.817 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 69095 69125 safe_time ceph NOFD /proc/69095/task/69125/fd (opendir: Permission denied) 2024-04-26T21:58:48.817 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 69095 69126 safe_time ceph cwd unknown /proc/69095/task/69126/cwd (readlink: Permission denied) 2024-04-26T21:58:48.817 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 69095 69126 safe_time ceph rtd unknown /proc/69095/task/69126/root (readlink: Permission denied) 2024-04-26T21:58:48.818 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 69095 69126 safe_time ceph txt unknown /proc/69095/task/69126/exe (readlink: Permission denied) 2024-04-26T21:58:48.818 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 69095 69126 safe_time ceph NOFD /proc/69095/task/69126/fd (opendir: Permission denied) 2024-04-26T21:58:48.818 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 69095 69127 ceph-mds ceph cwd unknown /proc/69095/task/69127/cwd (readlink: Permission denied) 2024-04-26T21:58:48.818 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 69095 69127 ceph-mds ceph rtd unknown /proc/69095/task/69127/root (readlink: Permission denied) 2024-04-26T21:58:48.818 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 69095 69127 ceph-mds ceph txt unknown /proc/69095/task/69127/exe (readlink: Permission denied) 2024-04-26T21:58:48.818 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 69095 69127 ceph-mds ceph NOFD /proc/69095/task/69127/fd (opendir: Permission denied) 2024-04-26T21:58:48.818 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 69095 69223 safe_time ceph cwd unknown /proc/69095/task/69223/cwd (readlink: Permission denied) 2024-04-26T21:58:48.818 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 69095 69223 safe_time ceph rtd unknown /proc/69095/task/69223/root (readlink: Permission denied) 2024-04-26T21:58:48.818 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 69095 69223 safe_time ceph txt unknown /proc/69095/task/69223/exe (readlink: Permission denied) 2024-04-26T21:58:48.818 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 69095 69223 safe_time ceph NOFD /proc/69095/task/69223/fd (opendir: Permission denied) 2024-04-26T21:58:48.818 INFO:tasks.workunit.client.0.smithi072.stdout:conmon 69480 root cwd unknown /proc/69480/cwd (readlink: Permission denied) 2024-04-26T21:58:48.818 INFO:tasks.workunit.client.0.smithi072.stdout:conmon 69480 root rtd unknown /proc/69480/root (readlink: Permission denied) 2024-04-26T21:58:48.819 INFO:tasks.workunit.client.0.smithi072.stdout:conmon 69480 root txt unknown /proc/69480/exe (readlink: Permission denied) 2024-04-26T21:58:48.819 INFO:tasks.workunit.client.0.smithi072.stdout:conmon 69480 root NOFD /proc/69480/fd (opendir: Permission denied) 2024-04-26T21:58:48.819 INFO:tasks.workunit.client.0.smithi072.stdout:podman-in 69482 root cwd unknown /proc/69482/cwd (readlink: Permission denied) 2024-04-26T21:58:48.819 INFO:tasks.workunit.client.0.smithi072.stdout:podman-in 69482 root rtd unknown /proc/69482/root (readlink: Permission denied) 2024-04-26T21:58:48.819 INFO:tasks.workunit.client.0.smithi072.stdout:podman-in 69482 root txt unknown /proc/69482/exe (readlink: Permission denied) 2024-04-26T21:58:48.819 INFO:tasks.workunit.client.0.smithi072.stdout:podman-in 69482 root NOFD /proc/69482/fd (opendir: Permission denied) 2024-04-26T21:58:48.819 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 69484 ceph cwd unknown /proc/69484/cwd (readlink: Permission denied) 2024-04-26T21:58:48.819 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 69484 ceph rtd unknown /proc/69484/root (readlink: Permission denied) 2024-04-26T21:58:48.819 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 69484 ceph txt unknown /proc/69484/exe (readlink: Permission denied) 2024-04-26T21:58:48.819 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 69484 ceph NOFD /proc/69484/fd (opendir: Permission denied) 2024-04-26T21:58:48.819 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 69484 69485 log ceph cwd unknown /proc/69484/task/69485/cwd (readlink: Permission denied) 2024-04-26T21:58:48.819 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 69484 69485 log ceph rtd unknown /proc/69484/task/69485/root (readlink: Permission denied) 2024-04-26T21:58:48.819 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 69484 69485 log ceph txt unknown /proc/69484/task/69485/exe (readlink: Permission denied) 2024-04-26T21:58:48.820 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 69484 69485 log ceph NOFD /proc/69484/task/69485/fd (opendir: Permission denied) 2024-04-26T21:58:48.820 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 69484 69487 msgr-work ceph cwd unknown /proc/69484/task/69487/cwd (readlink: Permission denied) 2024-04-26T21:58:48.820 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 69484 69487 msgr-work ceph rtd unknown /proc/69484/task/69487/root (readlink: Permission denied) 2024-04-26T21:58:48.820 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 69484 69487 msgr-work ceph txt unknown /proc/69484/task/69487/exe (readlink: Permission denied) 2024-04-26T21:58:48.820 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 69484 69487 msgr-work ceph NOFD /proc/69484/task/69487/fd (opendir: Permission denied) 2024-04-26T21:58:48.820 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 69484 69488 msgr-work ceph cwd unknown /proc/69484/task/69488/cwd (readlink: Permission denied) 2024-04-26T21:58:48.820 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 69484 69488 msgr-work ceph rtd unknown /proc/69484/task/69488/root (readlink: Permission denied) 2024-04-26T21:58:48.820 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 69484 69488 msgr-work ceph txt unknown /proc/69484/task/69488/exe (readlink: Permission denied) 2024-04-26T21:58:48.820 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 69484 69488 msgr-work ceph NOFD /proc/69484/task/69488/fd (opendir: Permission denied) 2024-04-26T21:58:48.820 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 69484 69489 msgr-work ceph cwd unknown /proc/69484/task/69489/cwd (readlink: Permission denied) 2024-04-26T21:58:48.820 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 69484 69489 msgr-work ceph rtd unknown /proc/69484/task/69489/root (readlink: Permission denied) 2024-04-26T21:58:48.820 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 69484 69489 msgr-work ceph txt unknown /proc/69484/task/69489/exe (readlink: Permission denied) 2024-04-26T21:58:48.821 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 69484 69489 msgr-work ceph NOFD /proc/69484/task/69489/fd (opendir: Permission denied) 2024-04-26T21:58:48.821 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 69484 69493 service ceph cwd unknown /proc/69484/task/69493/cwd (readlink: Permission denied) 2024-04-26T21:58:48.821 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 69484 69493 service ceph rtd unknown /proc/69484/task/69493/root (readlink: Permission denied) 2024-04-26T21:58:48.821 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 69484 69493 service ceph txt unknown /proc/69484/task/69493/exe (readlink: Permission denied) 2024-04-26T21:58:48.821 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 69484 69493 service ceph NOFD /proc/69484/task/69493/fd (opendir: Permission denied) 2024-04-26T21:58:48.821 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 69484 69494 admin_soc ceph cwd unknown /proc/69484/task/69494/cwd (readlink: Permission denied) 2024-04-26T21:58:48.821 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 69484 69494 admin_soc ceph rtd unknown /proc/69484/task/69494/root (readlink: Permission denied) 2024-04-26T21:58:48.821 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 69484 69494 admin_soc ceph txt unknown /proc/69484/task/69494/exe (readlink: Permission denied) 2024-04-26T21:58:48.821 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 69484 69494 admin_soc ceph NOFD /proc/69484/task/69494/fd (opendir: Permission denied) 2024-04-26T21:58:48.821 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 69484 69495 signal_ha ceph cwd unknown /proc/69484/task/69495/cwd (readlink: Permission denied) 2024-04-26T21:58:48.821 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 69484 69495 signal_ha ceph rtd unknown /proc/69484/task/69495/root (readlink: Permission denied) 2024-04-26T21:58:48.822 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 69484 69495 signal_ha ceph txt unknown /proc/69484/task/69495/exe (readlink: Permission denied) 2024-04-26T21:58:48.822 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 69484 69495 signal_ha ceph NOFD /proc/69484/task/69495/fd (opendir: Permission denied) 2024-04-26T21:58:48.822 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 69484 69496 io_contex ceph cwd unknown /proc/69484/task/69496/cwd (readlink: Permission denied) 2024-04-26T21:58:48.822 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 69484 69496 io_contex ceph rtd unknown /proc/69484/task/69496/root (readlink: Permission denied) 2024-04-26T21:58:48.822 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 69484 69496 io_contex ceph txt unknown /proc/69484/task/69496/exe (readlink: Permission denied) 2024-04-26T21:58:48.822 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 69484 69496 io_contex ceph NOFD /proc/69484/task/69496/fd (opendir: Permission denied) 2024-04-26T21:58:48.822 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 69484 69497 io_contex ceph cwd unknown /proc/69484/task/69497/cwd (readlink: Permission denied) 2024-04-26T21:58:48.822 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 69484 69497 io_contex ceph rtd unknown /proc/69484/task/69497/root (readlink: Permission denied) 2024-04-26T21:58:48.822 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 69484 69497 io_contex ceph txt unknown /proc/69484/task/69497/exe (readlink: Permission denied) 2024-04-26T21:58:48.823 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 69484 69497 io_contex ceph NOFD /proc/69484/task/69497/fd (opendir: Permission denied) 2024-04-26T21:58:48.823 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 69484 69498 ms_dispat ceph cwd unknown /proc/69484/task/69498/cwd (readlink: Permission denied) 2024-04-26T21:58:48.823 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 69484 69498 ms_dispat ceph rtd unknown /proc/69484/task/69498/root (readlink: Permission denied) 2024-04-26T21:58:48.823 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 69484 69498 ms_dispat ceph txt unknown /proc/69484/task/69498/exe (readlink: Permission denied) 2024-04-26T21:58:48.823 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 69484 69498 ms_dispat ceph NOFD /proc/69484/task/69498/fd (opendir: Permission denied) 2024-04-26T21:58:48.823 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 69484 69499 ms_local ceph cwd unknown /proc/69484/task/69499/cwd (readlink: Permission denied) 2024-04-26T21:58:48.823 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 69484 69499 ms_local ceph rtd unknown /proc/69484/task/69499/root (readlink: Permission denied) 2024-04-26T21:58:48.823 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 69484 69499 ms_local ceph txt unknown /proc/69484/task/69499/exe (readlink: Permission denied) 2024-04-26T21:58:48.823 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 69484 69499 ms_local ceph NOFD /proc/69484/task/69499/fd (opendir: Permission denied) 2024-04-26T21:58:48.823 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 69484 69500 safe_time ceph cwd unknown /proc/69484/task/69500/cwd (readlink: Permission denied) 2024-04-26T21:58:48.823 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 69484 69500 safe_time ceph rtd unknown /proc/69484/task/69500/root (readlink: Permission denied) 2024-04-26T21:58:48.824 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 69484 69500 safe_time ceph txt unknown /proc/69484/task/69500/exe (readlink: Permission denied) 2024-04-26T21:58:48.824 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 69484 69500 safe_time ceph NOFD /proc/69484/task/69500/fd (opendir: Permission denied) 2024-04-26T21:58:48.824 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 69484 69501 safe_time ceph cwd unknown /proc/69484/task/69501/cwd (readlink: Permission denied) 2024-04-26T21:58:48.824 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 69484 69501 safe_time ceph rtd unknown /proc/69484/task/69501/root (readlink: Permission denied) 2024-04-26T21:58:48.824 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 69484 69501 safe_time ceph txt unknown /proc/69484/task/69501/exe (readlink: Permission denied) 2024-04-26T21:58:48.824 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 69484 69501 safe_time ceph NOFD /proc/69484/task/69501/fd (opendir: Permission denied) 2024-04-26T21:58:48.824 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 69484 69502 ceph-mds ceph cwd unknown /proc/69484/task/69502/cwd (readlink: Permission denied) 2024-04-26T21:58:48.824 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 69484 69502 ceph-mds ceph rtd unknown /proc/69484/task/69502/root (readlink: Permission denied) 2024-04-26T21:58:48.824 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 69484 69502 ceph-mds ceph txt unknown /proc/69484/task/69502/exe (readlink: Permission denied) 2024-04-26T21:58:48.824 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 69484 69502 ceph-mds ceph NOFD /proc/69484/task/69502/fd (opendir: Permission denied) 2024-04-26T21:58:48.824 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 69484 69507 safe_time ceph cwd unknown /proc/69484/task/69507/cwd (readlink: Permission denied) 2024-04-26T21:58:48.825 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 69484 69507 safe_time ceph rtd unknown /proc/69484/task/69507/root (readlink: Permission denied) 2024-04-26T21:58:48.825 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 69484 69507 safe_time ceph txt unknown /proc/69484/task/69507/exe (readlink: Permission denied) 2024-04-26T21:58:48.825 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 69484 69507 safe_time ceph NOFD /proc/69484/task/69507/fd (opendir: Permission denied) 2024-04-26T21:58:48.825 INFO:tasks.workunit.client.0.smithi072.stdout:conmon 70185 root cwd unknown /proc/70185/cwd (readlink: Permission denied) 2024-04-26T21:58:48.825 INFO:tasks.workunit.client.0.smithi072.stdout:conmon 70185 root rtd unknown /proc/70185/root (readlink: Permission denied) 2024-04-26T21:58:48.825 INFO:tasks.workunit.client.0.smithi072.stdout:conmon 70185 root txt unknown /proc/70185/exe (readlink: Permission denied) 2024-04-26T21:58:48.825 INFO:tasks.workunit.client.0.smithi072.stdout:conmon 70185 root NOFD /proc/70185/fd (opendir: Permission denied) 2024-04-26T21:58:48.825 INFO:tasks.workunit.client.0.smithi072.stdout:podman-in 70205 root cwd unknown /proc/70205/cwd (readlink: Permission denied) 2024-04-26T21:58:48.825 INFO:tasks.workunit.client.0.smithi072.stdout:podman-in 70205 root rtd unknown /proc/70205/root (readlink: Permission denied) 2024-04-26T21:58:48.825 INFO:tasks.workunit.client.0.smithi072.stdout:podman-in 70205 root txt unknown /proc/70205/exe (readlink: Permission denied) 2024-04-26T21:58:48.825 INFO:tasks.workunit.client.0.smithi072.stdout:podman-in 70205 root NOFD /proc/70205/fd (opendir: Permission denied) 2024-04-26T21:58:48.826 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70207 ceph cwd unknown /proc/70207/cwd (readlink: Permission denied) 2024-04-26T21:58:48.826 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70207 ceph rtd unknown /proc/70207/root (readlink: Permission denied) 2024-04-26T21:58:48.826 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70207 ceph txt unknown /proc/70207/exe (readlink: Permission denied) 2024-04-26T21:58:48.826 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70207 ceph NOFD /proc/70207/fd (opendir: Permission denied) 2024-04-26T21:58:48.826 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70207 70208 log ceph cwd unknown /proc/70207/task/70208/cwd (readlink: Permission denied) 2024-04-26T21:58:48.826 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70207 70208 log ceph rtd unknown /proc/70207/task/70208/root (readlink: Permission denied) 2024-04-26T21:58:48.826 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70207 70208 log ceph txt unknown /proc/70207/task/70208/exe (readlink: Permission denied) 2024-04-26T21:58:48.826 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70207 70208 log ceph NOFD /proc/70207/task/70208/fd (opendir: Permission denied) 2024-04-26T21:58:48.826 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70207 70210 msgr-work ceph cwd unknown /proc/70207/task/70210/cwd (readlink: Permission denied) 2024-04-26T21:58:48.827 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70207 70210 msgr-work ceph rtd unknown /proc/70207/task/70210/root (readlink: Permission denied) 2024-04-26T21:58:48.827 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70207 70210 msgr-work ceph txt unknown /proc/70207/task/70210/exe (readlink: Permission denied) 2024-04-26T21:58:48.827 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70207 70210 msgr-work ceph NOFD /proc/70207/task/70210/fd (opendir: Permission denied) 2024-04-26T21:58:48.827 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70207 70211 msgr-work ceph cwd unknown /proc/70207/task/70211/cwd (readlink: Permission denied) 2024-04-26T21:58:48.827 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70207 70211 msgr-work ceph rtd unknown /proc/70207/task/70211/root (readlink: Permission denied) 2024-04-26T21:58:48.827 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70207 70211 msgr-work ceph txt unknown /proc/70207/task/70211/exe (readlink: Permission denied) 2024-04-26T21:58:48.827 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70207 70211 msgr-work ceph NOFD /proc/70207/task/70211/fd (opendir: Permission denied) 2024-04-26T21:58:48.827 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70207 70212 msgr-work ceph cwd unknown /proc/70207/task/70212/cwd (readlink: Permission denied) 2024-04-26T21:58:48.827 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70207 70212 msgr-work ceph rtd unknown /proc/70207/task/70212/root (readlink: Permission denied) 2024-04-26T21:58:48.827 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70207 70212 msgr-work ceph txt unknown /proc/70207/task/70212/exe (readlink: Permission denied) 2024-04-26T21:58:48.827 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70207 70212 msgr-work ceph NOFD /proc/70207/task/70212/fd (opendir: Permission denied) 2024-04-26T21:58:48.828 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70207 70216 service ceph cwd unknown /proc/70207/task/70216/cwd (readlink: Permission denied) 2024-04-26T21:58:48.828 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70207 70216 service ceph rtd unknown /proc/70207/task/70216/root (readlink: Permission denied) 2024-04-26T21:58:48.828 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70207 70216 service ceph txt unknown /proc/70207/task/70216/exe (readlink: Permission denied) 2024-04-26T21:58:48.828 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70207 70216 service ceph NOFD /proc/70207/task/70216/fd (opendir: Permission denied) 2024-04-26T21:58:48.828 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70207 70217 admin_soc ceph cwd unknown /proc/70207/task/70217/cwd (readlink: Permission denied) 2024-04-26T21:58:48.828 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70207 70217 admin_soc ceph rtd unknown /proc/70207/task/70217/root (readlink: Permission denied) 2024-04-26T21:58:48.828 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70207 70217 admin_soc ceph txt unknown /proc/70207/task/70217/exe (readlink: Permission denied) 2024-04-26T21:58:48.828 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70207 70217 admin_soc ceph NOFD /proc/70207/task/70217/fd (opendir: Permission denied) 2024-04-26T21:58:48.828 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70207 70218 signal_ha ceph cwd unknown /proc/70207/task/70218/cwd (readlink: Permission denied) 2024-04-26T21:58:48.828 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70207 70218 signal_ha ceph rtd unknown /proc/70207/task/70218/root (readlink: Permission denied) 2024-04-26T21:58:48.828 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70207 70218 signal_ha ceph txt unknown /proc/70207/task/70218/exe (readlink: Permission denied) 2024-04-26T21:58:48.828 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70207 70218 signal_ha ceph NOFD /proc/70207/task/70218/fd (opendir: Permission denied) 2024-04-26T21:58:48.829 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70207 70219 io_contex ceph cwd unknown /proc/70207/task/70219/cwd (readlink: Permission denied) 2024-04-26T21:58:48.829 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70207 70219 io_contex ceph rtd unknown /proc/70207/task/70219/root (readlink: Permission denied) 2024-04-26T21:58:48.829 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70207 70219 io_contex ceph txt unknown /proc/70207/task/70219/exe (readlink: Permission denied) 2024-04-26T21:58:48.829 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70207 70219 io_contex ceph NOFD /proc/70207/task/70219/fd (opendir: Permission denied) 2024-04-26T21:58:48.829 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70207 70220 io_contex ceph cwd unknown /proc/70207/task/70220/cwd (readlink: Permission denied) 2024-04-26T21:58:48.829 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70207 70220 io_contex ceph rtd unknown /proc/70207/task/70220/root (readlink: Permission denied) 2024-04-26T21:58:48.829 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70207 70220 io_contex ceph txt unknown /proc/70207/task/70220/exe (readlink: Permission denied) 2024-04-26T21:58:48.829 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70207 70220 io_contex ceph NOFD /proc/70207/task/70220/fd (opendir: Permission denied) 2024-04-26T21:58:48.829 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70207 70221 ms_dispat ceph cwd unknown /proc/70207/task/70221/cwd (readlink: Permission denied) 2024-04-26T21:58:48.829 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70207 70221 ms_dispat ceph rtd unknown /proc/70207/task/70221/root (readlink: Permission denied) 2024-04-26T21:58:48.829 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70207 70221 ms_dispat ceph txt unknown /proc/70207/task/70221/exe (readlink: Permission denied) 2024-04-26T21:58:48.829 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70207 70221 ms_dispat ceph NOFD /proc/70207/task/70221/fd (opendir: Permission denied) 2024-04-26T21:58:48.829 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70207 70222 ms_local ceph cwd unknown /proc/70207/task/70222/cwd (readlink: Permission denied) 2024-04-26T21:58:48.829 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70207 70222 ms_local ceph rtd unknown /proc/70207/task/70222/root (readlink: Permission denied) 2024-04-26T21:58:48.830 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70207 70222 ms_local ceph txt unknown /proc/70207/task/70222/exe (readlink: Permission denied) 2024-04-26T21:58:48.830 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70207 70222 ms_local ceph NOFD /proc/70207/task/70222/fd (opendir: Permission denied) 2024-04-26T21:58:48.830 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70207 70223 safe_time ceph cwd unknown /proc/70207/task/70223/cwd (readlink: Permission denied) 2024-04-26T21:58:48.830 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70207 70223 safe_time ceph rtd unknown /proc/70207/task/70223/root (readlink: Permission denied) 2024-04-26T21:58:48.830 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70207 70223 safe_time ceph txt unknown /proc/70207/task/70223/exe (readlink: Permission denied) 2024-04-26T21:58:48.830 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70207 70223 safe_time ceph NOFD /proc/70207/task/70223/fd (opendir: Permission denied) 2024-04-26T21:58:48.830 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70207 70224 safe_time ceph cwd unknown /proc/70207/task/70224/cwd (readlink: Permission denied) 2024-04-26T21:58:48.830 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70207 70224 safe_time ceph rtd unknown /proc/70207/task/70224/root (readlink: Permission denied) 2024-04-26T21:58:48.830 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70207 70224 safe_time ceph txt unknown /proc/70207/task/70224/exe (readlink: Permission denied) 2024-04-26T21:58:48.830 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70207 70224 safe_time ceph NOFD /proc/70207/task/70224/fd (opendir: Permission denied) 2024-04-26T21:58:48.830 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70207 70225 ceph-mds ceph cwd unknown /proc/70207/task/70225/cwd (readlink: Permission denied) 2024-04-26T21:58:48.830 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70207 70225 ceph-mds ceph rtd unknown /proc/70207/task/70225/root (readlink: Permission denied) 2024-04-26T21:58:48.830 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70207 70225 ceph-mds ceph txt unknown /proc/70207/task/70225/exe (readlink: Permission denied) 2024-04-26T21:58:48.830 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70207 70225 ceph-mds ceph NOFD /proc/70207/task/70225/fd (opendir: Permission denied) 2024-04-26T21:58:48.831 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70207 70255 safe_time ceph cwd unknown /proc/70207/task/70255/cwd (readlink: Permission denied) 2024-04-26T21:58:48.831 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70207 70255 safe_time ceph rtd unknown /proc/70207/task/70255/root (readlink: Permission denied) 2024-04-26T21:58:48.831 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70207 70255 safe_time ceph txt unknown /proc/70207/task/70255/exe (readlink: Permission denied) 2024-04-26T21:58:48.831 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70207 70255 safe_time ceph NOFD /proc/70207/task/70255/fd (opendir: Permission denied) 2024-04-26T21:58:48.831 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70207 72111 ceph_time ceph cwd unknown /proc/70207/task/72111/cwd (readlink: Permission denied) 2024-04-26T21:58:48.831 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70207 72111 ceph_time ceph rtd unknown /proc/70207/task/72111/root (readlink: Permission denied) 2024-04-26T21:58:48.831 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70207 72111 ceph_time ceph txt unknown /proc/70207/task/72111/exe (readlink: Permission denied) 2024-04-26T21:58:48.831 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70207 72111 ceph_time ceph NOFD /proc/70207/task/72111/fd (opendir: Permission denied) 2024-04-26T21:58:48.831 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70207 72113 OpHistory ceph cwd unknown /proc/70207/task/72113/cwd (readlink: Permission denied) 2024-04-26T21:58:48.831 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70207 72113 OpHistory ceph rtd unknown /proc/70207/task/72113/root (readlink: Permission denied) 2024-04-26T21:58:48.831 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70207 72113 OpHistory ceph txt unknown /proc/70207/task/72113/exe (readlink: Permission denied) 2024-04-26T21:58:48.832 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70207 72113 OpHistory ceph NOFD /proc/70207/task/72113/fd (opendir: Permission denied) 2024-04-26T21:58:48.832 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70207 72115 ms_dispat ceph cwd unknown /proc/70207/task/72115/cwd (readlink: Permission denied) 2024-04-26T21:58:48.832 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70207 72115 ms_dispat ceph rtd unknown /proc/70207/task/72115/root (readlink: Permission denied) 2024-04-26T21:58:48.832 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70207 72115 ms_dispat ceph txt unknown /proc/70207/task/72115/exe (readlink: Permission denied) 2024-04-26T21:58:48.832 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70207 72115 ms_dispat ceph NOFD /proc/70207/task/72115/fd (opendir: Permission denied) 2024-04-26T21:58:48.832 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70207 72117 ms_dispat ceph cwd unknown /proc/70207/task/72117/cwd (readlink: Permission denied) 2024-04-26T21:58:48.832 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70207 72117 ms_dispat ceph rtd unknown /proc/70207/task/72117/root (readlink: Permission denied) 2024-04-26T21:58:48.832 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70207 72117 ms_dispat ceph txt unknown /proc/70207/task/72117/exe (readlink: Permission denied) 2024-04-26T21:58:48.833 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70207 72117 ms_dispat ceph NOFD /proc/70207/task/72117/fd (opendir: Permission denied) 2024-04-26T21:58:48.833 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70207 72123 mds_rank_ ceph cwd unknown /proc/70207/task/72123/cwd (readlink: Permission denied) 2024-04-26T21:58:48.833 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70207 72123 mds_rank_ ceph rtd unknown /proc/70207/task/72123/root (readlink: Permission denied) 2024-04-26T21:58:48.833 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70207 72123 mds_rank_ ceph txt unknown /proc/70207/task/72123/exe (readlink: Permission denied) 2024-04-26T21:58:48.833 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70207 72123 mds_rank_ ceph NOFD /proc/70207/task/72123/fd (opendir: Permission denied) 2024-04-26T21:58:48.833 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70207 72124 PQ_Finish ceph cwd unknown /proc/70207/task/72124/cwd (readlink: Permission denied) 2024-04-26T21:58:48.833 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70207 72124 PQ_Finish ceph rtd unknown /proc/70207/task/72124/root (readlink: Permission denied) 2024-04-26T21:58:48.833 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70207 72124 PQ_Finish ceph txt unknown /proc/70207/task/72124/exe (readlink: Permission denied) 2024-04-26T21:58:48.833 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70207 72124 PQ_Finish ceph NOFD /proc/70207/task/72124/fd (opendir: Permission denied) 2024-04-26T21:58:48.833 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70207 72125 safe_time ceph cwd unknown /proc/70207/task/72125/cwd (readlink: Permission denied) 2024-04-26T21:58:48.833 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70207 72125 safe_time ceph rtd unknown /proc/70207/task/72125/root (readlink: Permission denied) 2024-04-26T21:58:48.833 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70207 72125 safe_time ceph txt unknown /proc/70207/task/72125/exe (readlink: Permission denied) 2024-04-26T21:58:48.833 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70207 72125 safe_time ceph NOFD /proc/70207/task/72125/fd (opendir: Permission denied) 2024-04-26T21:58:48.833 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70207 72127 MR_Finish ceph cwd unknown /proc/70207/task/72127/cwd (readlink: Permission denied) 2024-04-26T21:58:48.834 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70207 72127 MR_Finish ceph rtd unknown /proc/70207/task/72127/root (readlink: Permission denied) 2024-04-26T21:58:48.834 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70207 72127 MR_Finish ceph txt unknown /proc/70207/task/72127/exe (readlink: Permission denied) 2024-04-26T21:58:48.834 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70207 72127 MR_Finish ceph NOFD /proc/70207/task/72127/fd (opendir: Permission denied) 2024-04-26T21:58:48.834 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70207 72130 md_submit ceph cwd unknown /proc/70207/task/72130/cwd (readlink: Permission denied) 2024-04-26T21:58:48.834 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70207 72130 md_submit ceph rtd unknown /proc/70207/task/72130/root (readlink: Permission denied) 2024-04-26T21:58:48.834 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70207 72130 md_submit ceph txt unknown /proc/70207/task/72130/exe (readlink: Permission denied) 2024-04-26T21:58:48.834 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70207 72130 md_submit ceph NOFD /proc/70207/task/72130/fd (opendir: Permission denied) 2024-04-26T21:58:48.834 INFO:tasks.workunit.client.0.smithi072.stdout:conmon 70549 root cwd unknown /proc/70549/cwd (readlink: Permission denied) 2024-04-26T21:58:48.834 INFO:tasks.workunit.client.0.smithi072.stdout:conmon 70549 root rtd unknown /proc/70549/root (readlink: Permission denied) 2024-04-26T21:58:48.834 INFO:tasks.workunit.client.0.smithi072.stdout:conmon 70549 root txt unknown /proc/70549/exe (readlink: Permission denied) 2024-04-26T21:58:48.834 INFO:tasks.workunit.client.0.smithi072.stdout:conmon 70549 root NOFD /proc/70549/fd (opendir: Permission denied) 2024-04-26T21:58:48.834 INFO:tasks.workunit.client.0.smithi072.stdout:podman-in 70551 root cwd unknown /proc/70551/cwd (readlink: Permission denied) 2024-04-26T21:58:48.834 INFO:tasks.workunit.client.0.smithi072.stdout:podman-in 70551 root rtd unknown /proc/70551/root (readlink: Permission denied) 2024-04-26T21:58:48.834 INFO:tasks.workunit.client.0.smithi072.stdout:podman-in 70551 root txt unknown /proc/70551/exe (readlink: Permission denied) 2024-04-26T21:58:48.835 INFO:tasks.workunit.client.0.smithi072.stdout:podman-in 70551 root NOFD /proc/70551/fd (opendir: Permission denied) 2024-04-26T21:58:48.835 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70553 ceph cwd unknown /proc/70553/cwd (readlink: Permission denied) 2024-04-26T21:58:48.835 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70553 ceph rtd unknown /proc/70553/root (readlink: Permission denied) 2024-04-26T21:58:48.835 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70553 ceph txt unknown /proc/70553/exe (readlink: Permission denied) 2024-04-26T21:58:48.835 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70553 ceph NOFD /proc/70553/fd (opendir: Permission denied) 2024-04-26T21:58:48.835 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70553 70554 log ceph cwd unknown /proc/70553/task/70554/cwd (readlink: Permission denied) 2024-04-26T21:58:48.835 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70553 70554 log ceph rtd unknown /proc/70553/task/70554/root (readlink: Permission denied) 2024-04-26T21:58:48.835 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70553 70554 log ceph txt unknown /proc/70553/task/70554/exe (readlink: Permission denied) 2024-04-26T21:58:48.835 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70553 70554 log ceph NOFD /proc/70553/task/70554/fd (opendir: Permission denied) 2024-04-26T21:58:48.835 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70553 70556 msgr-work ceph cwd unknown /proc/70553/task/70556/cwd (readlink: Permission denied) 2024-04-26T21:58:48.835 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70553 70556 msgr-work ceph rtd unknown /proc/70553/task/70556/root (readlink: Permission denied) 2024-04-26T21:58:48.835 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70553 70556 msgr-work ceph txt unknown /proc/70553/task/70556/exe (readlink: Permission denied) 2024-04-26T21:58:48.835 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70553 70556 msgr-work ceph NOFD /proc/70553/task/70556/fd (opendir: Permission denied) 2024-04-26T21:58:48.835 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70553 70557 msgr-work ceph cwd unknown /proc/70553/task/70557/cwd (readlink: Permission denied) 2024-04-26T21:58:48.835 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70553 70557 msgr-work ceph rtd unknown /proc/70553/task/70557/root (readlink: Permission denied) 2024-04-26T21:58:48.836 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70553 70557 msgr-work ceph txt unknown /proc/70553/task/70557/exe (readlink: Permission denied) 2024-04-26T21:58:48.836 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70553 70557 msgr-work ceph NOFD /proc/70553/task/70557/fd (opendir: Permission denied) 2024-04-26T21:58:48.836 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70553 70558 msgr-work ceph cwd unknown /proc/70553/task/70558/cwd (readlink: Permission denied) 2024-04-26T21:58:48.836 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70553 70558 msgr-work ceph rtd unknown /proc/70553/task/70558/root (readlink: Permission denied) 2024-04-26T21:58:48.836 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70553 70558 msgr-work ceph txt unknown /proc/70553/task/70558/exe (readlink: Permission denied) 2024-04-26T21:58:48.836 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70553 70558 msgr-work ceph NOFD /proc/70553/task/70558/fd (opendir: Permission denied) 2024-04-26T21:58:48.836 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70553 70562 service ceph cwd unknown /proc/70553/task/70562/cwd (readlink: Permission denied) 2024-04-26T21:58:48.836 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70553 70562 service ceph rtd unknown /proc/70553/task/70562/root (readlink: Permission denied) 2024-04-26T21:58:48.836 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70553 70562 service ceph txt unknown /proc/70553/task/70562/exe (readlink: Permission denied) 2024-04-26T21:58:48.836 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70553 70562 service ceph NOFD /proc/70553/task/70562/fd (opendir: Permission denied) 2024-04-26T21:58:48.836 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70553 70563 admin_soc ceph cwd unknown /proc/70553/task/70563/cwd (readlink: Permission denied) 2024-04-26T21:58:48.836 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70553 70563 admin_soc ceph rtd unknown /proc/70553/task/70563/root (readlink: Permission denied) 2024-04-26T21:58:48.836 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70553 70563 admin_soc ceph txt unknown /proc/70553/task/70563/exe (readlink: Permission denied) 2024-04-26T21:58:48.836 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70553 70563 admin_soc ceph NOFD /proc/70553/task/70563/fd (opendir: Permission denied) 2024-04-26T21:58:48.837 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70553 70564 signal_ha ceph cwd unknown /proc/70553/task/70564/cwd (readlink: Permission denied) 2024-04-26T21:58:48.837 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70553 70564 signal_ha ceph rtd unknown /proc/70553/task/70564/root (readlink: Permission denied) 2024-04-26T21:58:48.837 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70553 70564 signal_ha ceph txt unknown /proc/70553/task/70564/exe (readlink: Permission denied) 2024-04-26T21:58:48.837 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70553 70564 signal_ha ceph NOFD /proc/70553/task/70564/fd (opendir: Permission denied) 2024-04-26T21:58:48.837 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70553 70565 io_contex ceph cwd unknown /proc/70553/task/70565/cwd (readlink: Permission denied) 2024-04-26T21:58:48.837 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70553 70565 io_contex ceph rtd unknown /proc/70553/task/70565/root (readlink: Permission denied) 2024-04-26T21:58:48.837 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70553 70565 io_contex ceph txt unknown /proc/70553/task/70565/exe (readlink: Permission denied) 2024-04-26T21:58:48.837 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70553 70565 io_contex ceph NOFD /proc/70553/task/70565/fd (opendir: Permission denied) 2024-04-26T21:58:48.837 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70553 70566 io_contex ceph cwd unknown /proc/70553/task/70566/cwd (readlink: Permission denied) 2024-04-26T21:58:48.837 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70553 70566 io_contex ceph rtd unknown /proc/70553/task/70566/root (readlink: Permission denied) 2024-04-26T21:58:48.837 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70553 70566 io_contex ceph txt unknown /proc/70553/task/70566/exe (readlink: Permission denied) 2024-04-26T21:58:48.837 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70553 70566 io_contex ceph NOFD /proc/70553/task/70566/fd (opendir: Permission denied) 2024-04-26T21:58:48.837 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70553 70567 ms_dispat ceph cwd unknown /proc/70553/task/70567/cwd (readlink: Permission denied) 2024-04-26T21:58:48.838 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70553 70567 ms_dispat ceph rtd unknown /proc/70553/task/70567/root (readlink: Permission denied) 2024-04-26T21:58:48.838 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70553 70567 ms_dispat ceph txt unknown /proc/70553/task/70567/exe (readlink: Permission denied) 2024-04-26T21:58:48.838 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70553 70567 ms_dispat ceph NOFD /proc/70553/task/70567/fd (opendir: Permission denied) 2024-04-26T21:58:48.838 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70553 70568 ms_local ceph cwd unknown /proc/70553/task/70568/cwd (readlink: Permission denied) 2024-04-26T21:58:48.838 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70553 70568 ms_local ceph rtd unknown /proc/70553/task/70568/root (readlink: Permission denied) 2024-04-26T21:58:48.838 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70553 70568 ms_local ceph txt unknown /proc/70553/task/70568/exe (readlink: Permission denied) 2024-04-26T21:58:48.838 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70553 70568 ms_local ceph NOFD /proc/70553/task/70568/fd (opendir: Permission denied) 2024-04-26T21:58:48.838 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70553 70569 safe_time ceph cwd unknown /proc/70553/task/70569/cwd (readlink: Permission denied) 2024-04-26T21:58:48.838 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70553 70569 safe_time ceph rtd unknown /proc/70553/task/70569/root (readlink: Permission denied) 2024-04-26T21:58:48.838 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70553 70569 safe_time ceph txt unknown /proc/70553/task/70569/exe (readlink: Permission denied) 2024-04-26T21:58:48.838 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70553 70569 safe_time ceph NOFD /proc/70553/task/70569/fd (opendir: Permission denied) 2024-04-26T21:58:48.838 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70553 70570 safe_time ceph cwd unknown /proc/70553/task/70570/cwd (readlink: Permission denied) 2024-04-26T21:58:48.838 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70553 70570 safe_time ceph rtd unknown /proc/70553/task/70570/root (readlink: Permission denied) 2024-04-26T21:58:48.838 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70553 70570 safe_time ceph txt unknown /proc/70553/task/70570/exe (readlink: Permission denied) 2024-04-26T21:58:48.838 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70553 70570 safe_time ceph NOFD /proc/70553/task/70570/fd (opendir: Permission denied) 2024-04-26T21:58:48.839 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70553 70571 ceph-mds ceph cwd unknown /proc/70553/task/70571/cwd (readlink: Permission denied) 2024-04-26T21:58:48.839 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70553 70571 ceph-mds ceph rtd unknown /proc/70553/task/70571/root (readlink: Permission denied) 2024-04-26T21:58:48.839 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70553 70571 ceph-mds ceph txt unknown /proc/70553/task/70571/exe (readlink: Permission denied) 2024-04-26T21:58:48.839 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70553 70571 ceph-mds ceph NOFD /proc/70553/task/70571/fd (opendir: Permission denied) 2024-04-26T21:58:48.839 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70553 70576 safe_time ceph cwd unknown /proc/70553/task/70576/cwd (readlink: Permission denied) 2024-04-26T21:58:48.839 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70553 70576 safe_time ceph rtd unknown /proc/70553/task/70576/root (readlink: Permission denied) 2024-04-26T21:58:48.839 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70553 70576 safe_time ceph txt unknown /proc/70553/task/70576/exe (readlink: Permission denied) 2024-04-26T21:58:48.839 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70553 70576 safe_time ceph NOFD /proc/70553/task/70576/fd (opendir: Permission denied) 2024-04-26T21:58:48.839 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70553 72112 ceph_time ceph cwd unknown /proc/70553/task/72112/cwd (readlink: Permission denied) 2024-04-26T21:58:48.839 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70553 72112 ceph_time ceph rtd unknown /proc/70553/task/72112/root (readlink: Permission denied) 2024-04-26T21:58:48.839 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70553 72112 ceph_time ceph txt unknown /proc/70553/task/72112/exe (readlink: Permission denied) 2024-04-26T21:58:48.839 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70553 72112 ceph_time ceph NOFD /proc/70553/task/72112/fd (opendir: Permission denied) 2024-04-26T21:58:48.839 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70553 72114 OpHistory ceph cwd unknown /proc/70553/task/72114/cwd (readlink: Permission denied) 2024-04-26T21:58:48.839 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70553 72114 OpHistory ceph rtd unknown /proc/70553/task/72114/root (readlink: Permission denied) 2024-04-26T21:58:48.840 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70553 72114 OpHistory ceph txt unknown /proc/70553/task/72114/exe (readlink: Permission denied) 2024-04-26T21:58:48.840 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70553 72114 OpHistory ceph NOFD /proc/70553/task/72114/fd (opendir: Permission denied) 2024-04-26T21:58:48.840 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70553 72116 ms_dispat ceph cwd unknown /proc/70553/task/72116/cwd (readlink: Permission denied) 2024-04-26T21:58:48.840 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70553 72116 ms_dispat ceph rtd unknown /proc/70553/task/72116/root (readlink: Permission denied) 2024-04-26T21:58:48.840 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70553 72116 ms_dispat ceph txt unknown /proc/70553/task/72116/exe (readlink: Permission denied) 2024-04-26T21:58:48.840 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70553 72116 ms_dispat ceph NOFD /proc/70553/task/72116/fd (opendir: Permission denied) 2024-04-26T21:58:48.840 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70553 72118 ms_dispat ceph cwd unknown /proc/70553/task/72118/cwd (readlink: Permission denied) 2024-04-26T21:58:48.840 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70553 72118 ms_dispat ceph rtd unknown /proc/70553/task/72118/root (readlink: Permission denied) 2024-04-26T21:58:48.840 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70553 72118 ms_dispat ceph txt unknown /proc/70553/task/72118/exe (readlink: Permission denied) 2024-04-26T21:58:48.840 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70553 72118 ms_dispat ceph NOFD /proc/70553/task/72118/fd (opendir: Permission denied) 2024-04-26T21:58:48.840 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70553 72119 mds_rank_ ceph cwd unknown /proc/70553/task/72119/cwd (readlink: Permission denied) 2024-04-26T21:58:48.840 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70553 72119 mds_rank_ ceph rtd unknown /proc/70553/task/72119/root (readlink: Permission denied) 2024-04-26T21:58:48.840 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70553 72119 mds_rank_ ceph txt unknown /proc/70553/task/72119/exe (readlink: Permission denied) 2024-04-26T21:58:48.841 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70553 72119 mds_rank_ ceph NOFD /proc/70553/task/72119/fd (opendir: Permission denied) 2024-04-26T21:58:48.841 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70553 72120 PQ_Finish ceph cwd unknown /proc/70553/task/72120/cwd (readlink: Permission denied) 2024-04-26T21:58:48.841 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70553 72120 PQ_Finish ceph rtd unknown /proc/70553/task/72120/root (readlink: Permission denied) 2024-04-26T21:58:48.841 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70553 72120 PQ_Finish ceph txt unknown /proc/70553/task/72120/exe (readlink: Permission denied) 2024-04-26T21:58:48.841 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70553 72120 PQ_Finish ceph NOFD /proc/70553/task/72120/fd (opendir: Permission denied) 2024-04-26T21:58:48.841 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70553 72121 safe_time ceph cwd unknown /proc/70553/task/72121/cwd (readlink: Permission denied) 2024-04-26T21:58:48.841 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70553 72121 safe_time ceph rtd unknown /proc/70553/task/72121/root (readlink: Permission denied) 2024-04-26T21:58:48.841 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70553 72121 safe_time ceph txt unknown /proc/70553/task/72121/exe (readlink: Permission denied) 2024-04-26T21:58:48.841 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70553 72121 safe_time ceph NOFD /proc/70553/task/72121/fd (opendir: Permission denied) 2024-04-26T21:58:48.841 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70553 72122 MR_Finish ceph cwd unknown /proc/70553/task/72122/cwd (readlink: Permission denied) 2024-04-26T21:58:48.841 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70553 72122 MR_Finish ceph rtd unknown /proc/70553/task/72122/root (readlink: Permission denied) 2024-04-26T21:58:48.841 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70553 72122 MR_Finish ceph txt unknown /proc/70553/task/72122/exe (readlink: Permission denied) 2024-04-26T21:58:48.841 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70553 72122 MR_Finish ceph NOFD /proc/70553/task/72122/fd (opendir: Permission denied) 2024-04-26T21:58:48.841 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70553 72128 md_submit ceph cwd unknown /proc/70553/task/72128/cwd (readlink: Permission denied) 2024-04-26T21:58:48.841 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70553 72128 md_submit ceph rtd unknown /proc/70553/task/72128/root (readlink: Permission denied) 2024-04-26T21:58:48.842 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70553 72128 md_submit ceph txt unknown /proc/70553/task/72128/exe (readlink: Permission denied) 2024-04-26T21:58:48.842 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-mds 70553 72128 md_submit ceph NOFD /proc/70553/task/72128/fd (opendir: Permission denied) 2024-04-26T21:58:48.842 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/5 75255 root cwd unknown /proc/75255/cwd (readlink: Permission denied) 2024-04-26T21:58:48.842 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/5 75255 root rtd unknown /proc/75255/root (readlink: Permission denied) 2024-04-26T21:58:48.842 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/5 75255 root txt unknown /proc/75255/exe (readlink: Permission denied) 2024-04-26T21:58:48.842 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/5 75255 root NOFD /proc/75255/fd (opendir: Permission denied) 2024-04-26T21:58:48.842 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/2 76683 root cwd unknown /proc/76683/cwd (readlink: Permission denied) 2024-04-26T21:58:48.842 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/2 76683 root rtd unknown /proc/76683/root (readlink: Permission denied) 2024-04-26T21:58:48.842 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/2 76683 root txt unknown /proc/76683/exe (readlink: Permission denied) 2024-04-26T21:58:48.842 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/2 76683 root NOFD /proc/76683/fd (opendir: Permission denied) 2024-04-26T21:58:48.842 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/2 76827 root cwd unknown /proc/76827/cwd (readlink: Permission denied) 2024-04-26T21:58:48.842 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/2 76827 root rtd unknown /proc/76827/root (readlink: Permission denied) 2024-04-26T21:58:48.842 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/2 76827 root txt unknown /proc/76827/exe (readlink: Permission denied) 2024-04-26T21:58:48.842 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/2 76827 root NOFD /proc/76827/fd (opendir: Permission denied) 2024-04-26T21:58:48.842 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/4 79208 root cwd unknown /proc/79208/cwd (readlink: Permission denied) 2024-04-26T21:58:48.843 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/4 79208 root rtd unknown /proc/79208/root (readlink: Permission denied) 2024-04-26T21:58:48.843 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/4 79208 root txt unknown /proc/79208/exe (readlink: Permission denied) 2024-04-26T21:58:48.843 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/4 79208 root NOFD /proc/79208/fd (opendir: Permission denied) 2024-04-26T21:58:48.843 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/0 80545 root cwd unknown /proc/80545/cwd (readlink: Permission denied) 2024-04-26T21:58:48.843 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/0 80545 root rtd unknown /proc/80545/root (readlink: Permission denied) 2024-04-26T21:58:48.843 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/0 80545 root txt unknown /proc/80545/exe (readlink: Permission denied) 2024-04-26T21:58:48.843 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/0 80545 root NOFD /proc/80545/fd (opendir: Permission denied) 2024-04-26T21:58:48.843 INFO:tasks.workunit.client.0.smithi072.stdout:bash 81080 ubuntu cwd DIR 8,1 4096 61580 /home/ubuntu 2024-04-26T21:58:48.843 INFO:tasks.workunit.client.0.smithi072.stdout:bash 81080 ubuntu rtd DIR 8,1 4096 2 / 2024-04-26T21:58:48.843 INFO:tasks.workunit.client.0.smithi072.stdout:bash 81080 ubuntu txt REG 8,1 1389072 7149 /usr/bin/bash 2024-04-26T21:58:48.843 INFO:tasks.workunit.client.0.smithi072.stdout:bash 81080 ubuntu mem REG 8,1 2586930 16078 /usr/lib/locale/en_US.utf8/LC_COLLATE 2024-04-26T21:58:48.843 INFO:tasks.workunit.client.0.smithi072.stdout:bash 81080 ubuntu mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-26T21:58:48.843 INFO:tasks.workunit.client.0.smithi072.stdout:bash 81080 ubuntu mem REG 8,1 346132 28859 /usr/lib/locale/C.utf8/LC_CTYPE 2024-04-26T21:58:48.843 INFO:tasks.workunit.client.0.smithi072.stdout:bash 81080 ubuntu mem REG 8,1 54 16081 /usr/lib/locale/en_US.utf8/LC_NUMERIC 2024-04-26T21:58:48.844 INFO:tasks.workunit.client.0.smithi072.stdout:bash 81080 ubuntu mem REG 8,1 3284 28786 /usr/lib/locale/en_US.utf8/LC_TIME 2024-04-26T21:58:48.844 INFO:tasks.workunit.client.0.smithi072.stdout:bash 81080 ubuntu mem REG 8,1 286 28783 /usr/lib/locale/en_US.utf8/LC_MONETARY 2024-04-26T21:58:48.844 INFO:tasks.workunit.client.0.smithi072.stdout:bash 81080 ubuntu mem REG 8,1 57 26536 /usr/lib/locale/en_US.utf8/LC_MESSAGES/SYS_LC_MESSAGES 2024-04-26T21:58:48.844 INFO:tasks.workunit.client.0.smithi072.stdout:bash 81080 ubuntu mem REG 8,1 34 28836 /usr/lib/locale/en_US.utf8/LC_PAPER 2024-04-26T21:58:48.844 INFO:tasks.workunit.client.0.smithi072.stdout:bash 81080 ubuntu mem REG 8,1 77 16080 /usr/lib/locale/en_US.utf8/LC_NAME 2024-04-26T21:58:48.844 INFO:tasks.workunit.client.0.smithi072.stdout:bash 81080 ubuntu mem REG 8,1 167 28778 /usr/lib/locale/en_US.utf8/LC_ADDRESS 2024-04-26T21:58:48.844 INFO:tasks.workunit.client.0.smithi072.stdout:bash 81080 ubuntu mem REG 8,1 59 28785 /usr/lib/locale/en_US.utf8/LC_TELEPHONE 2024-04-26T21:58:48.844 INFO:tasks.workunit.client.0.smithi072.stdout:bash 81080 ubuntu mem REG 8,1 195080 7147 /usr/lib64/libtinfo.so.6.2 2024-04-26T21:58:48.844 INFO:tasks.workunit.client.0.smithi072.stdout:bash 81080 ubuntu mem REG 8,1 23 28782 /usr/lib/locale/en_US.utf8/LC_MEASUREMENT 2024-04-26T21:58:48.844 INFO:tasks.workunit.client.0.smithi072.stdout:bash 81080 ubuntu mem REG 8,1 26988 27963 /usr/lib64/gconv/gconv-modules.cache 2024-04-26T21:58:48.844 INFO:tasks.workunit.client.0.smithi072.stdout:bash 81080 ubuntu mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-26T21:58:48.844 INFO:tasks.workunit.client.0.smithi072.stdout:bash 81080 ubuntu mem REG 8,1 369 28781 /usr/lib/locale/en_US.utf8/LC_IDENTIFICATION 2024-04-26T21:58:48.844 INFO:tasks.workunit.client.0.smithi072.stdout:bash 81080 ubuntu 0r FIFO 0,14 0t0 200326 pipe 2024-04-26T21:58:48.844 INFO:tasks.workunit.client.0.smithi072.stdout:bash 81080 ubuntu 1w FIFO 0,14 0t0 200327 pipe 2024-04-26T21:58:48.845 INFO:tasks.workunit.client.0.smithi072.stdout:bash 81080 ubuntu 2w FIFO 0,14 0t0 200328 pipe 2024-04-26T21:58:48.845 INFO:tasks.workunit.client.0.smithi072.stdout:sudo 81110 root cwd unknown /proc/81110/cwd (readlink: Permission denied) 2024-04-26T21:58:48.845 INFO:tasks.workunit.client.0.smithi072.stdout:sudo 81110 root rtd unknown /proc/81110/root (readlink: Permission denied) 2024-04-26T21:58:48.845 INFO:tasks.workunit.client.0.smithi072.stdout:sudo 81110 root txt unknown /proc/81110/exe (readlink: Permission denied) 2024-04-26T21:58:48.845 INFO:tasks.workunit.client.0.smithi072.stdout:sudo 81110 root NOFD /proc/81110/fd (opendir: Permission denied) 2024-04-26T21:58:48.845 INFO:tasks.workunit.client.0.smithi072.stdout:sudo 81114 root cwd unknown /proc/81114/cwd (readlink: Permission denied) 2024-04-26T21:58:48.845 INFO:tasks.workunit.client.0.smithi072.stdout:sudo 81114 root rtd unknown /proc/81114/root (readlink: Permission denied) 2024-04-26T21:58:48.845 INFO:tasks.workunit.client.0.smithi072.stdout:sudo 81114 root txt unknown /proc/81114/exe (readlink: Permission denied) 2024-04-26T21:58:48.845 INFO:tasks.workunit.client.0.smithi072.stdout:sudo 81114 root NOFD /proc/81114/fd (opendir: Permission denied) 2024-04-26T21:58:48.845 INFO:tasks.workunit.client.0.smithi072.stdout:daemon-he 81124 root cwd unknown /proc/81124/cwd (readlink: Permission denied) 2024-04-26T21:58:48.845 INFO:tasks.workunit.client.0.smithi072.stdout:daemon-he 81124 root rtd unknown /proc/81124/root (readlink: Permission denied) 2024-04-26T21:58:48.845 INFO:tasks.workunit.client.0.smithi072.stdout:daemon-he 81124 root txt unknown /proc/81124/exe (readlink: Permission denied) 2024-04-26T21:58:48.845 INFO:tasks.workunit.client.0.smithi072.stdout:daemon-he 81124 root NOFD /proc/81124/fd (opendir: Permission denied) 2024-04-26T21:58:48.845 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-fuse 81138 root cwd unknown /proc/81138/cwd (readlink: Permission denied) 2024-04-26T21:58:48.845 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-fuse 81138 root rtd unknown /proc/81138/root (readlink: Permission denied) 2024-04-26T21:58:48.846 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-fuse 81138 root txt unknown /proc/81138/exe (readlink: Permission denied) 2024-04-26T21:58:48.846 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-fuse 81138 root NOFD /proc/81138/fd (opendir: Permission denied) 2024-04-26T21:58:48.846 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-fuse 81138 81147 log root cwd unknown /proc/81138/task/81147/cwd (readlink: Permission denied) 2024-04-26T21:58:48.846 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-fuse 81138 81147 log root rtd unknown /proc/81138/task/81147/root (readlink: Permission denied) 2024-04-26T21:58:48.846 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-fuse 81138 81147 log root txt unknown /proc/81138/task/81147/exe (readlink: Permission denied) 2024-04-26T21:58:48.846 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-fuse 81138 81147 log root NOFD /proc/81138/task/81147/fd (opendir: Permission denied) 2024-04-26T21:58:48.846 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-fuse 81138 81150 msgr-work root cwd unknown /proc/81138/task/81150/cwd (readlink: Permission denied) 2024-04-26T21:58:48.846 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-fuse 81138 81150 msgr-work root rtd unknown /proc/81138/task/81150/root (readlink: Permission denied) 2024-04-26T21:58:48.846 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-fuse 81138 81150 msgr-work root txt unknown /proc/81138/task/81150/exe (readlink: Permission denied) 2024-04-26T21:58:48.846 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-fuse 81138 81150 msgr-work root NOFD /proc/81138/task/81150/fd (opendir: Permission denied) 2024-04-26T21:58:48.846 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-fuse 81138 81151 msgr-work root cwd unknown /proc/81138/task/81151/cwd (readlink: Permission denied) 2024-04-26T21:58:48.846 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-fuse 81138 81151 msgr-work root rtd unknown /proc/81138/task/81151/root (readlink: Permission denied) 2024-04-26T21:58:48.846 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-fuse 81138 81151 msgr-work root txt unknown /proc/81138/task/81151/exe (readlink: Permission denied) 2024-04-26T21:58:48.846 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-fuse 81138 81151 msgr-work root NOFD /proc/81138/task/81151/fd (opendir: Permission denied) 2024-04-26T21:58:48.847 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-fuse 81138 81152 msgr-work root cwd unknown /proc/81138/task/81152/cwd (readlink: Permission denied) 2024-04-26T21:58:48.847 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-fuse 81138 81152 msgr-work root rtd unknown /proc/81138/task/81152/root (readlink: Permission denied) 2024-04-26T21:58:48.847 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-fuse 81138 81152 msgr-work root txt unknown /proc/81138/task/81152/exe (readlink: Permission denied) 2024-04-26T21:58:48.847 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-fuse 81138 81152 msgr-work root NOFD /proc/81138/task/81152/fd (opendir: Permission denied) 2024-04-26T21:58:48.847 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-fuse 81138 81161 service root cwd unknown /proc/81138/task/81161/cwd (readlink: Permission denied) 2024-04-26T21:58:48.847 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-fuse 81138 81161 service root rtd unknown /proc/81138/task/81161/root (readlink: Permission denied) 2024-04-26T21:58:48.847 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-fuse 81138 81161 service root txt unknown /proc/81138/task/81161/exe (readlink: Permission denied) 2024-04-26T21:58:48.847 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-fuse 81138 81161 service root NOFD /proc/81138/task/81161/fd (opendir: Permission denied) 2024-04-26T21:58:48.847 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-fuse 81138 81162 admin_soc root cwd unknown /proc/81138/task/81162/cwd (readlink: Permission denied) 2024-04-26T21:58:48.847 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-fuse 81138 81162 admin_soc root rtd unknown /proc/81138/task/81162/root (readlink: Permission denied) 2024-04-26T21:58:48.847 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-fuse 81138 81162 admin_soc root txt unknown /proc/81138/task/81162/exe (readlink: Permission denied) 2024-04-26T21:58:48.847 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-fuse 81138 81162 admin_soc root NOFD /proc/81138/task/81162/fd (opendir: Permission denied) 2024-04-26T21:58:48.847 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-fuse 81138 81163 signal_ha root cwd unknown /proc/81138/task/81163/cwd (readlink: Permission denied) 2024-04-26T21:58:48.848 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-fuse 81138 81163 signal_ha root rtd unknown /proc/81138/task/81163/root (readlink: Permission denied) 2024-04-26T21:58:48.848 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-fuse 81138 81163 signal_ha root txt unknown /proc/81138/task/81163/exe (readlink: Permission denied) 2024-04-26T21:58:48.848 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-fuse 81138 81163 signal_ha root NOFD /proc/81138/task/81163/fd (opendir: Permission denied) 2024-04-26T21:58:48.848 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-fuse 81138 81164 io_contex root cwd unknown /proc/81138/task/81164/cwd (readlink: Permission denied) 2024-04-26T21:58:48.848 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-fuse 81138 81164 io_contex root rtd unknown /proc/81138/task/81164/root (readlink: Permission denied) 2024-04-26T21:58:48.848 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-fuse 81138 81164 io_contex root txt unknown /proc/81138/task/81164/exe (readlink: Permission denied) 2024-04-26T21:58:48.848 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-fuse 81138 81164 io_contex root NOFD /proc/81138/task/81164/fd (opendir: Permission denied) 2024-04-26T21:58:48.848 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-fuse 81138 81165 io_contex root cwd unknown /proc/81138/task/81165/cwd (readlink: Permission denied) 2024-04-26T21:58:48.848 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-fuse 81138 81165 io_contex root rtd unknown /proc/81138/task/81165/root (readlink: Permission denied) 2024-04-26T21:58:48.848 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-fuse 81138 81165 io_contex root txt unknown /proc/81138/task/81165/exe (readlink: Permission denied) 2024-04-26T21:58:48.848 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-fuse 81138 81165 io_contex root NOFD /proc/81138/task/81165/fd (opendir: Permission denied) 2024-04-26T21:58:48.848 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-fuse 81138 81166 ceph_time root cwd unknown /proc/81138/task/81166/cwd (readlink: Permission denied) 2024-04-26T21:58:48.848 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-fuse 81138 81166 ceph_time root rtd unknown /proc/81138/task/81166/root (readlink: Permission denied) 2024-04-26T21:58:48.849 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-fuse 81138 81166 ceph_time root txt unknown /proc/81138/task/81166/exe (readlink: Permission denied) 2024-04-26T21:58:48.849 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-fuse 81138 81166 ceph_time root NOFD /proc/81138/task/81166/fd (opendir: Permission denied) 2024-04-26T21:58:48.849 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-fuse 81138 81167 fn_anonym root cwd unknown /proc/81138/task/81167/cwd (readlink: Permission denied) 2024-04-26T21:58:48.849 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-fuse 81138 81167 fn_anonym root rtd unknown /proc/81138/task/81167/root (readlink: Permission denied) 2024-04-26T21:58:48.849 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-fuse 81138 81167 fn_anonym root txt unknown /proc/81138/task/81167/exe (readlink: Permission denied) 2024-04-26T21:58:48.849 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-fuse 81138 81167 fn_anonym root NOFD /proc/81138/task/81167/fd (opendir: Permission denied) 2024-04-26T21:58:48.849 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-fuse 81138 81169 fn_anonym root cwd unknown /proc/81138/task/81169/cwd (readlink: Permission denied) 2024-04-26T21:58:48.849 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-fuse 81138 81169 fn_anonym root rtd unknown /proc/81138/task/81169/root (readlink: Permission denied) 2024-04-26T21:58:48.849 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-fuse 81138 81169 fn_anonym root txt unknown /proc/81138/task/81169/exe (readlink: Permission denied) 2024-04-26T21:58:48.849 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-fuse 81138 81169 fn_anonym root NOFD /proc/81138/task/81169/fd (opendir: Permission denied) 2024-04-26T21:58:48.849 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-fuse 81138 81170 fn_anonym root cwd unknown /proc/81138/task/81170/cwd (readlink: Permission denied) 2024-04-26T21:58:48.849 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-fuse 81138 81170 fn_anonym root rtd unknown /proc/81138/task/81170/root (readlink: Permission denied) 2024-04-26T21:58:48.849 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-fuse 81138 81170 fn_anonym root txt unknown /proc/81138/task/81170/exe (readlink: Permission denied) 2024-04-26T21:58:48.849 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-fuse 81138 81170 fn_anonym root NOFD /proc/81138/task/81170/fd (opendir: Permission denied) 2024-04-26T21:58:48.849 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-fuse 81138 81171 fn_anonym root cwd unknown /proc/81138/task/81171/cwd (readlink: Permission denied) 2024-04-26T21:58:48.850 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-fuse 81138 81171 fn_anonym root rtd unknown /proc/81138/task/81171/root (readlink: Permission denied) 2024-04-26T21:58:48.850 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-fuse 81138 81171 fn_anonym root txt unknown /proc/81138/task/81171/exe (readlink: Permission denied) 2024-04-26T21:58:48.850 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-fuse 81138 81171 fn_anonym root NOFD /proc/81138/task/81171/fd (opendir: Permission denied) 2024-04-26T21:58:48.850 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-fuse 81138 81172 fn_anonym root cwd unknown /proc/81138/task/81172/cwd (readlink: Permission denied) 2024-04-26T21:58:48.850 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-fuse 81138 81172 fn_anonym root rtd unknown /proc/81138/task/81172/root (readlink: Permission denied) 2024-04-26T21:58:48.850 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-fuse 81138 81172 fn_anonym root txt unknown /proc/81138/task/81172/exe (readlink: Permission denied) 2024-04-26T21:58:48.850 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-fuse 81138 81172 fn_anonym root NOFD /proc/81138/task/81172/fd (opendir: Permission denied) 2024-04-26T21:58:48.850 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-fuse 81138 81173 safe_time root cwd unknown /proc/81138/task/81173/cwd (readlink: Permission denied) 2024-04-26T21:58:48.850 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-fuse 81138 81173 safe_time root rtd unknown /proc/81138/task/81173/root (readlink: Permission denied) 2024-04-26T21:58:48.850 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-fuse 81138 81173 safe_time root txt unknown /proc/81138/task/81173/exe (readlink: Permission denied) 2024-04-26T21:58:48.850 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-fuse 81138 81173 safe_time root NOFD /proc/81138/task/81173/fd (opendir: Permission denied) 2024-04-26T21:58:48.850 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-fuse 81138 81174 fn_anonym root cwd unknown /proc/81138/task/81174/cwd (readlink: Permission denied) 2024-04-26T21:58:48.850 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-fuse 81138 81174 fn_anonym root rtd unknown /proc/81138/task/81174/root (readlink: Permission denied) 2024-04-26T21:58:48.850 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-fuse 81138 81174 fn_anonym root txt unknown /proc/81138/task/81174/exe (readlink: Permission denied) 2024-04-26T21:58:48.850 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-fuse 81138 81174 fn_anonym root NOFD /proc/81138/task/81174/fd (opendir: Permission denied) 2024-04-26T21:58:48.851 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-fuse 81138 81175 flusher root cwd unknown /proc/81138/task/81175/cwd (readlink: Permission denied) 2024-04-26T21:58:48.851 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-fuse 81138 81175 flusher root rtd unknown /proc/81138/task/81175/root (readlink: Permission denied) 2024-04-26T21:58:48.851 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-fuse 81138 81175 flusher root txt unknown /proc/81138/task/81175/exe (readlink: Permission denied) 2024-04-26T21:58:48.851 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-fuse 81138 81175 flusher root NOFD /proc/81138/task/81175/fd (opendir: Permission denied) 2024-04-26T21:58:48.851 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-fuse 81138 81177 ms_dispat root cwd unknown /proc/81138/task/81177/cwd (readlink: Permission denied) 2024-04-26T21:58:48.851 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-fuse 81138 81177 ms_dispat root rtd unknown /proc/81138/task/81177/root (readlink: Permission denied) 2024-04-26T21:58:48.851 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-fuse 81138 81177 ms_dispat root txt unknown /proc/81138/task/81177/exe (readlink: Permission denied) 2024-04-26T21:58:48.851 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-fuse 81138 81177 ms_dispat root NOFD /proc/81138/task/81177/fd (opendir: Permission denied) 2024-04-26T21:58:48.851 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-fuse 81138 81178 ms_local root cwd unknown /proc/81138/task/81178/cwd (readlink: Permission denied) 2024-04-26T21:58:48.851 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-fuse 81138 81178 ms_local root rtd unknown /proc/81138/task/81178/root (readlink: Permission denied) 2024-04-26T21:58:48.851 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-fuse 81138 81178 ms_local root txt unknown /proc/81138/task/81178/exe (readlink: Permission denied) 2024-04-26T21:58:48.851 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-fuse 81138 81178 ms_local root NOFD /proc/81138/task/81178/fd (opendir: Permission denied) 2024-04-26T21:58:48.851 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-fuse 81138 81179 safe_time root cwd unknown /proc/81138/task/81179/cwd (readlink: Permission denied) 2024-04-26T21:58:48.851 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-fuse 81138 81179 safe_time root rtd unknown /proc/81138/task/81179/root (readlink: Permission denied) 2024-04-26T21:58:48.852 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-fuse 81138 81179 safe_time root txt unknown /proc/81138/task/81179/exe (readlink: Permission denied) 2024-04-26T21:58:48.852 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-fuse 81138 81179 safe_time root NOFD /proc/81138/task/81179/fd (opendir: Permission denied) 2024-04-26T21:58:48.854 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-fuse 81138 81180 ceph-fuse root cwd unknown /proc/81138/task/81180/cwd (readlink: Permission denied) 2024-04-26T21:58:48.854 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-fuse 81138 81180 ceph-fuse root rtd unknown /proc/81138/task/81180/root (readlink: Permission denied) 2024-04-26T21:58:48.854 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-fuse 81138 81180 ceph-fuse root txt unknown /proc/81138/task/81180/exe (readlink: Permission denied) 2024-04-26T21:58:48.854 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-fuse 81138 81180 ceph-fuse root NOFD /proc/81138/task/81180/fd (opendir: Permission denied) 2024-04-26T21:58:48.854 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-fuse 81138 81182 ceph-fuse root cwd unknown /proc/81138/task/81182/cwd (readlink: Permission denied) 2024-04-26T21:58:48.854 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-fuse 81138 81182 ceph-fuse root rtd unknown /proc/81138/task/81182/root (readlink: Permission denied) 2024-04-26T21:58:48.855 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-fuse 81138 81182 ceph-fuse root txt unknown /proc/81138/task/81182/exe (readlink: Permission denied) 2024-04-26T21:58:48.855 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-fuse 81138 81182 ceph-fuse root NOFD /proc/81138/task/81182/fd (opendir: Permission denied) 2024-04-26T21:58:48.855 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-fuse 81138 81184 ceph-fuse root cwd unknown /proc/81138/task/81184/cwd (readlink: Permission denied) 2024-04-26T21:58:48.855 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-fuse 81138 81184 ceph-fuse root rtd unknown /proc/81138/task/81184/root (readlink: Permission denied) 2024-04-26T21:58:48.855 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-fuse 81138 81184 ceph-fuse root txt unknown /proc/81138/task/81184/exe (readlink: Permission denied) 2024-04-26T21:58:48.855 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-fuse 81138 81184 ceph-fuse root NOFD /proc/81138/task/81184/fd (opendir: Permission denied) 2024-04-26T21:58:48.855 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-fuse 81138 84122 ceph-fuse root cwd unknown /proc/81138/task/84122/cwd (readlink: Permission denied) 2024-04-26T21:58:48.855 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-fuse 81138 84122 ceph-fuse root rtd unknown /proc/81138/task/84122/root (readlink: Permission denied) 2024-04-26T21:58:48.855 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-fuse 81138 84122 ceph-fuse root txt unknown /proc/81138/task/84122/exe (readlink: Permission denied) 2024-04-26T21:58:48.856 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-fuse 81138 84122 ceph-fuse root NOFD /proc/81138/task/84122/fd (opendir: Permission denied) 2024-04-26T21:58:48.856 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-fuse 81138 85382 ceph-fuse root cwd unknown /proc/81138/task/85382/cwd (readlink: Permission denied) 2024-04-26T21:58:48.856 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-fuse 81138 85382 ceph-fuse root rtd unknown /proc/81138/task/85382/root (readlink: Permission denied) 2024-04-26T21:58:48.856 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-fuse 81138 85382 ceph-fuse root txt unknown /proc/81138/task/85382/exe (readlink: Permission denied) 2024-04-26T21:58:48.856 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-fuse 81138 85382 ceph-fuse root NOFD /proc/81138/task/85382/fd (opendir: Permission denied) 2024-04-26T21:58:48.856 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-fuse 81138 85383 ceph-fuse root cwd unknown /proc/81138/task/85383/cwd (readlink: Permission denied) 2024-04-26T21:58:48.856 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-fuse 81138 85383 ceph-fuse root rtd unknown /proc/81138/task/85383/root (readlink: Permission denied) 2024-04-26T21:58:48.857 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-fuse 81138 85383 ceph-fuse root txt unknown /proc/81138/task/85383/exe (readlink: Permission denied) 2024-04-26T21:58:48.857 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-fuse 81138 85383 ceph-fuse root NOFD /proc/81138/task/85383/fd (opendir: Permission denied) 2024-04-26T21:58:48.857 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-fuse 81138 86076 ceph-fuse root cwd unknown /proc/81138/task/86076/cwd (readlink: Permission denied) 2024-04-26T21:58:48.857 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-fuse 81138 86076 ceph-fuse root rtd unknown /proc/81138/task/86076/root (readlink: Permission denied) 2024-04-26T21:58:48.857 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-fuse 81138 86076 ceph-fuse root txt unknown /proc/81138/task/86076/exe (readlink: Permission denied) 2024-04-26T21:58:48.857 INFO:tasks.workunit.client.0.smithi072.stdout:ceph-fuse 81138 86076 ceph-fuse root NOFD /proc/81138/task/86076/fd (opendir: Permission denied) 2024-04-26T21:58:48.857 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/1 81916 root cwd unknown /proc/81916/cwd (readlink: Permission denied) 2024-04-26T21:58:48.857 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/1 81916 root rtd unknown /proc/81916/root (readlink: Permission denied) 2024-04-26T21:58:48.858 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/1 81916 root txt unknown /proc/81916/exe (readlink: Permission denied) 2024-04-26T21:58:48.858 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/1 81916 root NOFD /proc/81916/fd (opendir: Permission denied) 2024-04-26T21:58:48.858 INFO:tasks.workunit.client.0.smithi072.stdout:timeout 84102 ubuntu cwd DIR 0,114 194241720 1099511629283 /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-04-26T21:58:48.858 INFO:tasks.workunit.client.0.smithi072.stdout:timeout 84102 ubuntu rtd DIR 8,1 4096 2 / 2024-04-26T21:58:48.858 INFO:tasks.workunit.client.0.smithi072.stdout:timeout 84102 ubuntu txt REG 8,1 41144 7580 /usr/bin/timeout 2024-04-26T21:58:48.858 INFO:tasks.workunit.client.0.smithi072.stdout:timeout 84102 ubuntu mem REG 8,1 2586930 16078 /usr/lib/locale/en_US.utf8/LC_COLLATE 2024-04-26T21:58:48.858 INFO:tasks.workunit.client.0.smithi072.stdout:timeout 84102 ubuntu mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-26T21:58:48.858 INFO:tasks.workunit.client.0.smithi072.stdout:timeout 84102 ubuntu mem REG 8,1 346132 28859 /usr/lib/locale/C.utf8/LC_CTYPE 2024-04-26T21:58:48.859 INFO:tasks.workunit.client.0.smithi072.stdout:timeout 84102 ubuntu mem REG 8,1 54 16081 /usr/lib/locale/en_US.utf8/LC_NUMERIC 2024-04-26T21:58:48.859 INFO:tasks.workunit.client.0.smithi072.stdout:timeout 84102 ubuntu mem REG 8,1 3284 28786 /usr/lib/locale/en_US.utf8/LC_TIME 2024-04-26T21:58:48.859 INFO:tasks.workunit.client.0.smithi072.stdout:timeout 84102 ubuntu mem REG 8,1 286 28783 /usr/lib/locale/en_US.utf8/LC_MONETARY 2024-04-26T21:58:48.859 INFO:tasks.workunit.client.0.smithi072.stdout:timeout 84102 ubuntu mem REG 8,1 57 26536 /usr/lib/locale/en_US.utf8/LC_MESSAGES/SYS_LC_MESSAGES 2024-04-26T21:58:48.859 INFO:tasks.workunit.client.0.smithi072.stdout:timeout 84102 ubuntu mem REG 8,1 34 28836 /usr/lib/locale/en_US.utf8/LC_PAPER 2024-04-26T21:58:48.859 INFO:tasks.workunit.client.0.smithi072.stdout:timeout 84102 ubuntu mem REG 8,1 77 16080 /usr/lib/locale/en_US.utf8/LC_NAME 2024-04-26T21:58:48.859 INFO:tasks.workunit.client.0.smithi072.stdout:timeout 84102 ubuntu mem REG 8,1 167 28778 /usr/lib/locale/en_US.utf8/LC_ADDRESS 2024-04-26T21:58:48.860 INFO:tasks.workunit.client.0.smithi072.stdout:timeout 84102 ubuntu mem REG 8,1 59 28785 /usr/lib/locale/en_US.utf8/LC_TELEPHONE 2024-04-26T21:58:48.860 INFO:tasks.workunit.client.0.smithi072.stdout:timeout 84102 ubuntu mem REG 8,1 23 28782 /usr/lib/locale/en_US.utf8/LC_MEASUREMENT 2024-04-26T21:58:48.860 INFO:tasks.workunit.client.0.smithi072.stdout:timeout 84102 ubuntu mem REG 8,1 26988 27963 /usr/lib64/gconv/gconv-modules.cache 2024-04-26T21:58:48.860 INFO:tasks.workunit.client.0.smithi072.stdout:timeout 84102 ubuntu mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-26T21:58:48.860 INFO:tasks.workunit.client.0.smithi072.stdout:timeout 84102 ubuntu mem REG 8,1 369 28781 /usr/lib/locale/en_US.utf8/LC_IDENTIFICATION 2024-04-26T21:58:48.860 INFO:tasks.workunit.client.0.smithi072.stdout:timeout 84102 ubuntu 0r FIFO 0,14 0t0 211270 pipe 2024-04-26T21:58:48.860 INFO:tasks.workunit.client.0.smithi072.stdout:timeout 84102 ubuntu 1w FIFO 0,14 0t0 211271 pipe 2024-04-26T21:58:48.860 INFO:tasks.workunit.client.0.smithi072.stdout:timeout 84102 ubuntu 2w FIFO 0,14 0t0 211272 pipe 2024-04-26T21:58:48.860 INFO:tasks.workunit.client.0.smithi072.stdout:fsync-tes 84120 ubuntu cwd DIR 0,114 194241720 1099511629283 /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-04-26T21:58:48.860 INFO:tasks.workunit.client.0.smithi072.stdout:fsync-tes 84120 ubuntu rtd DIR 8,1 4096 2 / 2024-04-26T21:58:48.860 INFO:tasks.workunit.client.0.smithi072.stdout:fsync-tes 84120 ubuntu txt REG 8,1 1389072 7149 /usr/bin/bash 2024-04-26T21:58:48.860 INFO:tasks.workunit.client.0.smithi072.stdout:fsync-tes 84120 ubuntu mem REG 8,1 2586930 16078 /usr/lib/locale/en_US.utf8/LC_COLLATE 2024-04-26T21:58:48.860 INFO:tasks.workunit.client.0.smithi072.stdout:fsync-tes 84120 ubuntu mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-26T21:58:48.860 INFO:tasks.workunit.client.0.smithi072.stdout:fsync-tes 84120 ubuntu mem REG 8,1 346132 28859 /usr/lib/locale/C.utf8/LC_CTYPE 2024-04-26T21:58:48.861 INFO:tasks.workunit.client.0.smithi072.stdout:fsync-tes 84120 ubuntu mem REG 8,1 54 16081 /usr/lib/locale/en_US.utf8/LC_NUMERIC 2024-04-26T21:58:48.861 INFO:tasks.workunit.client.0.smithi072.stdout:fsync-tes 84120 ubuntu mem REG 8,1 3284 28786 /usr/lib/locale/en_US.utf8/LC_TIME 2024-04-26T21:58:48.861 INFO:tasks.workunit.client.0.smithi072.stdout:fsync-tes 84120 ubuntu mem REG 8,1 286 28783 /usr/lib/locale/en_US.utf8/LC_MONETARY 2024-04-26T21:58:48.861 INFO:tasks.workunit.client.0.smithi072.stdout:fsync-tes 84120 ubuntu mem REG 8,1 57 26536 /usr/lib/locale/en_US.utf8/LC_MESSAGES/SYS_LC_MESSAGES 2024-04-26T21:58:48.861 INFO:tasks.workunit.client.0.smithi072.stdout:fsync-tes 84120 ubuntu mem REG 8,1 34 28836 /usr/lib/locale/en_US.utf8/LC_PAPER 2024-04-26T21:58:48.861 INFO:tasks.workunit.client.0.smithi072.stdout:fsync-tes 84120 ubuntu mem REG 8,1 77 16080 /usr/lib/locale/en_US.utf8/LC_NAME 2024-04-26T21:58:48.861 INFO:tasks.workunit.client.0.smithi072.stdout:fsync-tes 84120 ubuntu mem REG 8,1 167 28778 /usr/lib/locale/en_US.utf8/LC_ADDRESS 2024-04-26T21:58:48.861 INFO:tasks.workunit.client.0.smithi072.stdout:fsync-tes 84120 ubuntu mem REG 8,1 59 28785 /usr/lib/locale/en_US.utf8/LC_TELEPHONE 2024-04-26T21:58:48.861 INFO:tasks.workunit.client.0.smithi072.stdout:fsync-tes 84120 ubuntu mem REG 8,1 195080 7147 /usr/lib64/libtinfo.so.6.2 2024-04-26T21:58:48.861 INFO:tasks.workunit.client.0.smithi072.stdout:fsync-tes 84120 ubuntu mem REG 8,1 23 28782 /usr/lib/locale/en_US.utf8/LC_MEASUREMENT 2024-04-26T21:58:48.861 INFO:tasks.workunit.client.0.smithi072.stdout:fsync-tes 84120 ubuntu mem REG 8,1 26988 27963 /usr/lib64/gconv/gconv-modules.cache 2024-04-26T21:58:48.861 INFO:tasks.workunit.client.0.smithi072.stdout:fsync-tes 84120 ubuntu mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-26T21:58:48.861 INFO:tasks.workunit.client.0.smithi072.stdout:fsync-tes 84120 ubuntu mem REG 8,1 369 28781 /usr/lib/locale/en_US.utf8/LC_IDENTIFICATION 2024-04-26T21:58:48.861 INFO:tasks.workunit.client.0.smithi072.stdout:fsync-tes 84120 ubuntu 0r FIFO 0,14 0t0 211270 pipe 2024-04-26T21:58:48.862 INFO:tasks.workunit.client.0.smithi072.stdout:fsync-tes 84120 ubuntu 1w FIFO 0,14 0t0 211271 pipe 2024-04-26T21:58:48.862 INFO:tasks.workunit.client.0.smithi072.stdout:fsync-tes 84120 ubuntu 2w FIFO 0,14 0t0 211272 pipe 2024-04-26T21:58:48.862 INFO:tasks.workunit.client.0.smithi072.stdout:fsync-tes 84120 ubuntu 255r REG 8,1 207 792363 /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsync-tester.sh 2024-04-26T21:58:48.862 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/u 84819 root cwd unknown /proc/84819/cwd (readlink: Permission denied) 2024-04-26T21:58:48.862 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/u 84819 root rtd unknown /proc/84819/root (readlink: Permission denied) 2024-04-26T21:58:48.862 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/u 84819 root txt unknown /proc/84819/exe (readlink: Permission denied) 2024-04-26T21:58:48.862 INFO:tasks.workunit.client.0.smithi072.stdout:kworker/u 84819 root NOFD /proc/84819/fd (opendir: Permission denied) 2024-04-26T21:58:48.862 INFO:tasks.workunit.client.0.smithi072.stdout:lsof 86078 ubuntu cwd DIR 0,114 194241720 1099511629283 /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-04-26T21:58:48.862 INFO:tasks.workunit.client.0.smithi072.stdout:lsof 86078 ubuntu rtd DIR 8,1 4096 2 / 2024-04-26T21:58:48.862 INFO:tasks.workunit.client.0.smithi072.stdout:lsof 86078 ubuntu txt REG 8,1 179232 9797 /usr/bin/lsof 2024-04-26T21:58:48.862 INFO:tasks.workunit.client.0.smithi072.stdout:lsof 86078 ubuntu mem REG 8,1 346132 28859 /usr/lib/locale/C.utf8/LC_CTYPE 2024-04-26T21:58:48.863 INFO:tasks.workunit.client.0.smithi072.stdout:lsof 86078 ubuntu mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-26T21:58:48.863 INFO:tasks.workunit.client.0.smithi072.stdout:lsof 86078 ubuntu mem REG 8,1 5431656 62278 /usr/lib64/libcrypto.so.3.2.1 2024-04-26T21:58:48.863 INFO:tasks.workunit.client.0.smithi072.stdout:lsof 86078 ubuntu mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-26T21:58:48.863 INFO:tasks.workunit.client.0.smithi072.stdout:lsof 86078 ubuntu mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-26T21:58:48.863 INFO:tasks.workunit.client.0.smithi072.stdout:lsof 86078 ubuntu mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-26T21:58:48.863 INFO:tasks.workunit.client.0.smithi072.stdout:lsof 86078 ubuntu mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-26T21:58:48.863 INFO:tasks.workunit.client.0.smithi072.stdout:lsof 86078 ubuntu mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-26T21:58:48.863 INFO:tasks.workunit.client.0.smithi072.stdout:lsof 86078 ubuntu mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-26T21:58:48.863 INFO:tasks.workunit.client.0.smithi072.stdout:lsof 86078 ubuntu mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-26T21:58:48.863 INFO:tasks.workunit.client.0.smithi072.stdout:lsof 86078 ubuntu mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-26T21:58:48.863 INFO:tasks.workunit.client.0.smithi072.stdout:lsof 86078 ubuntu mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-26T21:58:48.864 INFO:tasks.workunit.client.0.smithi072.stdout:lsof 86078 ubuntu mem REG 8,1 196224 8914 /usr/lib64/libtirpc.so.3.0.0 2024-04-26T21:58:48.864 INFO:tasks.workunit.client.0.smithi072.stdout:lsof 86078 ubuntu mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-26T21:58:48.864 INFO:tasks.workunit.client.0.smithi072.stdout:lsof 86078 ubuntu mem REG 8,1 26988 27963 /usr/lib64/gconv/gconv-modules.cache 2024-04-26T21:58:48.864 INFO:tasks.workunit.client.0.smithi072.stdout:lsof 86078 ubuntu mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-26T21:58:48.864 INFO:tasks.workunit.client.0.smithi072.stdout:lsof 86078 ubuntu 0r FIFO 0,14 0t0 211270 pipe 2024-04-26T21:58:48.864 INFO:tasks.workunit.client.0.smithi072.stdout:lsof 86078 ubuntu 1w FIFO 0,14 0t0 211271 pipe 2024-04-26T21:58:48.864 INFO:tasks.workunit.client.0.smithi072.stdout:lsof 86078 ubuntu 2w FIFO 0,14 0t0 211272 pipe 2024-04-26T21:58:48.864 INFO:tasks.workunit.client.0.smithi072.stdout:lsof 86078 ubuntu 3r DIR 0,21 0 1 /proc 2024-04-26T21:58:48.864 INFO:tasks.workunit.client.0.smithi072.stdout:lsof 86078 ubuntu 4r DIR 0,21 7 207858 /proc/86078/fd 2024-04-26T21:58:48.864 INFO:tasks.workunit.client.0.smithi072.stdout:lsof 86078 ubuntu 5w FIFO 0,14 0t0 207863 pipe 2024-04-26T21:58:48.864 INFO:tasks.workunit.client.0.smithi072.stdout:lsof 86078 ubuntu 6r FIFO 0,14 0t0 207864 pipe 2024-04-26T21:58:48.864 INFO:tasks.workunit.client.0.smithi072.stdout:lsof 86079 ubuntu cwd DIR 0,114 194241720 1099511629283 /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-04-26T21:58:48.865 INFO:tasks.workunit.client.0.smithi072.stdout:lsof 86079 ubuntu rtd DIR 8,1 4096 2 / 2024-04-26T21:58:48.865 INFO:tasks.workunit.client.0.smithi072.stdout:lsof 86079 ubuntu txt REG 8,1 179232 9797 /usr/bin/lsof 2024-04-26T21:58:48.865 INFO:tasks.workunit.client.0.smithi072.stdout:lsof 86079 ubuntu mem REG 8,1 346132 28859 /usr/lib/locale/C.utf8/LC_CTYPE 2024-04-26T21:58:48.865 INFO:tasks.workunit.client.0.smithi072.stdout:lsof 86079 ubuntu mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-26T21:58:48.865 INFO:tasks.workunit.client.0.smithi072.stdout:lsof 86079 ubuntu mem REG 8,1 5431656 62278 /usr/lib64/libcrypto.so.3.2.1 2024-04-26T21:58:48.865 INFO:tasks.workunit.client.0.smithi072.stdout:lsof 86079 ubuntu mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-26T21:58:48.865 INFO:tasks.workunit.client.0.smithi072.stdout:lsof 86079 ubuntu mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-26T21:58:48.865 INFO:tasks.workunit.client.0.smithi072.stdout:lsof 86079 ubuntu mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-26T21:58:48.865 INFO:tasks.workunit.client.0.smithi072.stdout:lsof 86079 ubuntu mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-26T21:58:48.865 INFO:tasks.workunit.client.0.smithi072.stdout:lsof 86079 ubuntu mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-26T21:58:48.865 INFO:tasks.workunit.client.0.smithi072.stdout:lsof 86079 ubuntu mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-26T21:58:48.866 INFO:tasks.workunit.client.0.smithi072.stdout:lsof 86079 ubuntu mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-26T21:58:48.866 INFO:tasks.workunit.client.0.smithi072.stdout:lsof 86079 ubuntu mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-26T21:58:48.866 INFO:tasks.workunit.client.0.smithi072.stdout:lsof 86079 ubuntu mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-26T21:58:48.866 INFO:tasks.workunit.client.0.smithi072.stdout:lsof 86079 ubuntu mem REG 8,1 196224 8914 /usr/lib64/libtirpc.so.3.0.0 2024-04-26T21:58:48.866 INFO:tasks.workunit.client.0.smithi072.stdout:lsof 86079 ubuntu mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-26T21:58:48.866 INFO:tasks.workunit.client.0.smithi072.stdout:lsof 86079 ubuntu mem REG 8,1 26988 27963 /usr/lib64/gconv/gconv-modules.cache 2024-04-26T21:58:48.866 INFO:tasks.workunit.client.0.smithi072.stdout:lsof 86079 ubuntu mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-26T21:58:48.866 INFO:tasks.workunit.client.0.smithi072.stdout:lsof 86079 ubuntu 4r FIFO 0,14 0t0 207863 pipe 2024-04-26T21:58:48.866 INFO:tasks.workunit.client.0.smithi072.stdout:lsof 86079 ubuntu 7w FIFO 0,14 0t0 207864 pipe 2024-04-26T21:58:48.869 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-04-26T21:58:48.869 DEBUG:teuthology.orchestra.run.smithi072:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-04-26T21:58:49.080 INFO:tasks.workunit:Stopping ['suites/fsync-tester.sh'] on client.0... 2024-04-26T21:58:49.081 DEBUG:teuthology.orchestra.run.smithi072:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-04-26T21:58:49.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:58:48 smithi076 ceph-mon[31823]: pgmap v550: 97 pgs: 97 active+clean; 258 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 11 MiB/s wr, 2.53k op/s 2024-04-26T21:58:49.321 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:58:48 smithi072 ceph-mon[28138]: pgmap v550: 97 pgs: 97 active+clean; 258 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 11 MiB/s wr, 2.53k op/s 2024-04-26T21:58:49.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:58:48 smithi164 ceph-mon[31916]: pgmap v550: 97 pgs: 97 active+clean; 258 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 11 MiB/s wr, 2.53k op/s 2024-04-26T21:58:50.140 DEBUG:teuthology.parallel:result is None 2024-04-26T21:58:50.140 DEBUG:teuthology.orchestra.run.smithi072:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-04-26T21:58:50.237 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-04-26T21:58:50.237 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-04-26T21:58:50.250 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-04-26T21:58:50.537 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-26T21:58:50.537 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-26T21:58:50.750 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.750+0000 7fccf8e00640 1 -- 172.21.15.72:0/284795599 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fccf4108e70 msgr2=0x7fccf4109290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:50.751 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.750+0000 7fccf8e00640 1 --2- 172.21.15.72:0/284795599 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fccf4108e70 0x7fccf4109290 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fcce4009900 tx=0x7fcce402a410 comp rx=0 tx=0).stop 2024-04-26T21:58:50.751 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.750+0000 7fccf8e00640 1 -- 172.21.15.72:0/284795599 shutdown_connections 2024-04-26T21:58:50.751 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.750+0000 7fccf8e00640 1 --2- 172.21.15.72:0/284795599 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fccf410a980 0x7fccf41112a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:50.751 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.750+0000 7fccf8e00640 1 --2- 172.21.15.72:0/284795599 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fccf4109fc0 0x7fccf410a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:50.751 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.750+0000 7fccf8e00640 1 --2- 172.21.15.72:0/284795599 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fccf4108e70 0x7fccf4109290 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:50.751 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.750+0000 7fccf8e00640 1 -- 172.21.15.72:0/284795599 >> 172.21.15.72:0/284795599 conn(0x7fccf4077e00 msgr2=0x7fccf407a240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:58:50.751 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.750+0000 7fccf8e00640 1 -- 172.21.15.72:0/284795599 shutdown_connections 2024-04-26T21:58:50.751 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.750+0000 7fccf8e00640 1 -- 172.21.15.72:0/284795599 wait complete. 2024-04-26T21:58:50.753 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.751+0000 7fccf8e00640 1 Processor -- start 2024-04-26T21:58:50.753 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.751+0000 7fccf8e00640 1 -- start start 2024-04-26T21:58:50.754 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.752+0000 7fccf8e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fccf4108e70 0x7fccf419ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:50.754 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.752+0000 7fccf8e00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fccf4109fc0 0x7fccf419f460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:50.754 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.752+0000 7fccf8e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fccf410a980 0x7fccf41a64d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:50.754 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.752+0000 7fccf8e00640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fccf4114850 con 0x7fccf4108e70 2024-04-26T21:58:50.754 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.752+0000 7fccf8e00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fccf41146d0 con 0x7fccf4109fc0 2024-04-26T21:58:50.754 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.752+0000 7fccf8e00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fccf41149d0 con 0x7fccf410a980 2024-04-26T21:58:50.754 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.752+0000 7fccf3400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fccf4108e70 0x7fccf419ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:50.754 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.752+0000 7fccf3400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fccf4108e70 0x7fccf419ef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:40690/0 (socket says 172.21.15.72:40690) 2024-04-26T21:58:50.754 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.752+0000 7fccf3400640 1 -- 172.21.15.72:0/1163237315 learned_addr learned my addr 172.21.15.72:0/1163237315 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:58:50.754 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.752+0000 7fccebe00640 1 --2- 172.21.15.72:0/1163237315 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fccf4109fc0 0x7fccf419f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:50.754 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.752+0000 7fccf3400640 1 -- 172.21.15.72:0/1163237315 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fccf410a980 msgr2=0x7fccf41a64d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:50.754 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.752+0000 7fccf3400640 1 --2- 172.21.15.72:0/1163237315 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fccf410a980 0x7fccf41a64d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:50.754 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.752+0000 7fccf3400640 1 -- 172.21.15.72:0/1163237315 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fccf4109fc0 msgr2=0x7fccf419f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:50.755 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.752+0000 7fccf3400640 1 --2- 172.21.15.72:0/1163237315 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fccf4109fc0 0x7fccf419f460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:50.755 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.752+0000 7fccf3400640 1 -- 172.21.15.72:0/1163237315 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccf41a6b40 con 0x7fccf4108e70 2024-04-26T21:58:50.755 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.753+0000 7fccf3400640 1 --2- 172.21.15.72:0/1163237315 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fccf4108e70 0x7fccf419ef20 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fcce4009230 tx=0x7fcce40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:58:50.755 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.753+0000 7fccf0c00640 1 -- 172.21.15.72:0/1163237315 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcce403f030 con 0x7fccf4108e70 2024-04-26T21:58:50.755 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.753+0000 7fccf0c00640 1 -- 172.21.15.72:0/1163237315 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcce4003c90 con 0x7fccf4108e70 2024-04-26T21:58:50.755 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.753+0000 7fccf0c00640 1 -- 172.21.15.72:0/1163237315 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcce4035070 con 0x7fccf4108e70 2024-04-26T21:58:50.755 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.753+0000 7fccf8e00640 1 -- 172.21.15.72:0/1163237315 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fccf41a6dd0 con 0x7fccf4108e70 2024-04-26T21:58:50.755 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.753+0000 7fccf8e00640 1 -- 172.21.15.72:0/1163237315 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fccf41a7200 con 0x7fccf4108e70 2024-04-26T21:58:50.757 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.756+0000 7fccfd390740 1 Processor -- start 2024-04-26T21:58:50.757 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.756+0000 7fccfd390740 1 -- start start 2024-04-26T21:58:50.757 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.756+0000 7fccf0c00640 1 -- 172.21.15.72:0/1163237315 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fcce4042030 con 0x7fccf4108e70 2024-04-26T21:58:50.758 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.757+0000 7fccfd390740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fea6c6a960 0x55fea6c6ad80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:50.758 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.757+0000 7fccfd390740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55fea6c6b350 0x55fea6c6fe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:50.758 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.757+0000 7fccfd390740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fea6c704c0 0x55fea6c72950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:50.758 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.757+0000 7fccfd390740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55fea6b7b030 con 0x55fea6c6b350 2024-04-26T21:58:50.758 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.757+0000 7fccfd390740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55fea6b967f0 con 0x55fea6c704c0 2024-04-26T21:58:50.758 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.757+0000 7fccfd390740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55fea6b517e0 con 0x55fea6c6a960 2024-04-26T21:58:50.758 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.757+0000 7fccf3400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55fea6c6b350 0x55fea6c6fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:50.758 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.757+0000 7fccf3400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55fea6c6b350 0x55fea6c6fe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:40692/0 (socket says 172.21.15.72:40692) 2024-04-26T21:58:50.758 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.757+0000 7fccf3400640 1 -- 172.21.15.72:0/4239219138 learned_addr learned my addr 172.21.15.72:0/4239219138 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:58:50.758 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.757+0000 7fccebe00640 1 --2- 172.21.15.72:0/4239219138 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fea6c704c0 0x55fea6c72950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:50.759 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.757+0000 7fccf3e00640 1 --2- 172.21.15.72:0/4239219138 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fea6c6a960 0x55fea6c6ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:50.759 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.757+0000 7fccf3400640 1 -- 172.21.15.72:0/4239219138 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fea6c6a960 msgr2=0x55fea6c6ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:58:50.759 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.757+0000 7fccf3400640 1 --2- 172.21.15.72:0/4239219138 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fea6c6a960 0x55fea6c6ad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:50.759 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.758+0000 7fccf3400640 1 -- 172.21.15.72:0/4239219138 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fea6c704c0 msgr2=0x55fea6c72950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:58:50.759 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.758+0000 7fccf3400640 1 --2- 172.21.15.72:0/4239219138 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fea6c704c0 0x55fea6c72950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:50.759 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.758+0000 7fccf3400640 1 -- 172.21.15.72:0/4239219138 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fea6b5d050 con 0x55fea6c6b350 2024-04-26T21:58:50.763 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.758+0000 7fccf3400640 1 --2- 172.21.15.72:0/4239219138 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55fea6c6b350 0x55fea6c6fe50 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fcce40f9160 tx=0x7fcce40f9250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80f62e2a40cb25f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:58:50.763 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.758+0000 7fccf0c00640 1 --2- 172.21.15.72:0/1163237315 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fccd40a3c40 0x7fccd40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:50.763 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.758+0000 7fcce8c00640 1 -- 172.21.15.72:0/4239219138 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcce40f9ac0 con 0x55fea6c6b350 2024-04-26T21:58:50.763 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.758+0000 7fcce8c00640 1 -- 172.21.15.72:0/4239219138 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcce40fe070 con 0x55fea6c6b350 2024-04-26T21:58:50.764 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.758+0000 7fcce8c00640 1 -- 172.21.15.72:0/4239219138 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcce40b92c0 con 0x55fea6c6b350 2024-04-26T21:58:50.764 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.758+0000 7fccf0c00640 1 -- 172.21.15.72:0/1163237315 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7fcce40f0ec0 con 0x7fccf4108e70 2024-04-26T21:58:50.764 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.759+0000 7fccf3e00640 1 --2- 172.21.15.72:0/1163237315 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fccd40a3c40 0x7fccd40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:50.764 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.759+0000 7fccfd390740 1 -- 172.21.15.72:0/4239219138 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55fea6c6b350 msgr2=0x55fea6c6fe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:50.764 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.759+0000 7fccfd390740 1 --2- 172.21.15.72:0/4239219138 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55fea6c6b350 0x55fea6c6fe50 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fcce40f9160 tx=0x7fcce40f9250 comp rx=0 tx=0).stop 2024-04-26T21:58:50.764 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.759+0000 7fccfd390740 1 -- 172.21.15.72:0/4239219138 shutdown_connections 2024-04-26T21:58:50.764 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.759+0000 7fccfd390740 1 --2- 172.21.15.72:0/4239219138 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fea6c704c0 0x55fea6c72950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:50.764 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.759+0000 7fccfd390740 1 --2- 172.21.15.72:0/4239219138 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55fea6c6b350 0x55fea6c6fe50 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:50.764 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.759+0000 7fccfd390740 1 --2- 172.21.15.72:0/4239219138 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fea6c6a960 0x55fea6c6ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:50.764 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.759+0000 7fccfd390740 1 -- 172.21.15.72:0/4239219138 >> 172.21.15.72:0/4239219138 conn(0x55fea6c65fd0 msgr2=0x55fea6c663d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:58:50.764 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.759+0000 7fccfd390740 1 -- 172.21.15.72:0/4239219138 shutdown_connections 2024-04-26T21:58:50.764 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.759+0000 7fccf3e00640 1 --2- 172.21.15.72:0/1163237315 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fccd40a3c40 0x7fccd40a6100 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x55fea6c6bbb0 tx=0x7fccec0073a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:58:50.764 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.759+0000 7fccfd390740 1 -- 172.21.15.72:0/4239219138 wait complete. 2024-04-26T21:58:50.764 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.760+0000 7fccfd390740 1 -- start start 2024-04-26T21:58:50.764 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.760+0000 7fccfd390740 1 Processor -- start 2024-04-26T21:58:50.765 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.760+0000 7fccfd390740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fea6c6a960 0x55fea6c744e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:50.765 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.760+0000 7fccfd390740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fea6c6b350 0x55fea6c74a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:50.765 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.760+0000 7fccfd390740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55fea6c704c0 0x55fea6c7ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:50.765 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.760+0000 7fccfd390740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55fea6bd3200 con 0x55fea6c704c0 2024-04-26T21:58:50.765 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.760+0000 7fccfd390740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55fea6bcdee0 con 0x55fea6c6b350 2024-04-26T21:58:50.765 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.760+0000 7fccfd390740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55fea6bdae70 con 0x55fea6c6a960 2024-04-26T21:58:50.765 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.760+0000 7fccfd390740 10 client.0 _ll_register_callbacks cb 0x55fea6bc4270 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:58:50.765 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.761+0000 7fccf3e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55fea6c704c0 0x55fea6c7ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:50.765 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.761+0000 7fccf3e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55fea6c704c0 0x55fea6c7ba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:40700/0 (socket says 172.21.15.72:40700) 2024-04-26T21:58:50.765 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.761+0000 7fccf3e00640 1 -- 172.21.15.72:0/164535362 learned_addr learned my addr 172.21.15.72:0/164535362 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:58:50.765 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.761+0000 7fccebe00640 1 --2- 172.21.15.72:0/164535362 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fea6c6b350 0x55fea6c74a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:50.765 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.761+0000 7fccf3400640 1 --2- 172.21.15.72:0/164535362 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fea6c6a960 0x55fea6c744e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:50.765 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.761+0000 7fccf3e00640 1 -- 172.21.15.72:0/164535362 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fea6c6a960 msgr2=0x55fea6c744e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:58:50.765 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.761+0000 7fccf3e00640 1 --2- 172.21.15.72:0/164535362 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fea6c6a960 0x55fea6c744e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:50.766 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.761+0000 7fccf3e00640 1 -- 172.21.15.72:0/164535362 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fea6c6b350 msgr2=0x55fea6c74a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:58:50.766 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.761+0000 7fccf3e00640 1 --2- 172.21.15.72:0/164535362 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fea6c6b350 0x55fea6c74a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:50.766 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.761+0000 7fccf3e00640 1 -- 172.21.15.72:0/164535362 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fea69aa7d0 con 0x55fea6c704c0 2024-04-26T21:58:50.766 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.761+0000 7fccf3400640 1 --2- 172.21.15.72:0/164535362 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fea6c6a960 0x55fea6c744e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:58:50.766 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.761+0000 7fccf3e00640 1 --2- 172.21.15.72:0/164535362 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55fea6c704c0 0x55fea6c7ba60 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fccec0040b0 tx=0x7fccec005e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29e2e7ef38470659 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:58:50.766 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.761+0000 7fcce1600640 10 client.0 ms_handle_connect on v2:172.21.15.72:3300/0 2024-04-26T21:58:50.766 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.762+0000 7fcce1600640 1 -- 172.21.15.72:0/164535362 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccec010a80 con 0x55fea6c704c0 2024-04-26T21:58:50.766 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.762+0000 7fcce1600640 1 -- 172.21.15.72:0/164535362 <== mon.0 v2:172.21.15.72:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7fccec010c20 con 0x55fea6c704c0 2024-04-26T21:58:50.766 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.762+0000 7fccfd390740 1 -- 172.21.15.72:0/164535362 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcce40fe070 con 0x55fea6c704c0 2024-04-26T21:58:50.766 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.762+0000 7fcce1600640 1 -- 172.21.15.72:0/164535362 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fccec010e20 con 0x55fea6c704c0 2024-04-26T21:58:50.766 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.762+0000 7fccfd390740 10 client.16275 fetch_fsmap learned FSMap version 27 2024-04-26T21:58:50.766 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.762+0000 7fccfd390740 1 -- 172.21.15.72:0/164535362 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55fea68e03f0 con 0x55fea6c704c0 2024-04-26T21:58:50.766 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.762+0000 7fcce1600640 1 -- 172.21.15.72:0/164535362 <== mon.0 v2:172.21.15.72:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcce40fe070 con 0x55fea6c704c0 2024-04-26T21:58:50.767 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.762+0000 7fcce1600640 1 -- 172.21.15.72:0/164535362 <== mon.0 v2:172.21.15.72:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fccec0275c0 con 0x55fea6c704c0 2024-04-26T21:58:50.767 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.763+0000 7fccfd390740 10 client.16275 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T21:58:50.767 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.763+0000 7fccfd390740 10 client.16275 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:58:50.767 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.763+0000 7fccfd390740 20 client.16275 populate_metadata read hostname 'smithi072' 2024-04-26T21:58:50.767 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.763+0000 7fccfd390740 1 --2- 172.21.15.72:0/164535362 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x55fea6c7ffc0 0x55fea6c82480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:50.767 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.763+0000 7fccfd390740 4 client.16275 mds_command: new command op to 34517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:58:50.767 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.763+0000 7fccfd390740 1 -- 172.21.15.72:0/164535362 --> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fea6bdc7e0 con 0x55fea6c7ffc0 2024-04-26T21:58:50.767 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.763+0000 7fccf3400640 1 --2- 172.21.15.72:0/164535362 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x55fea6c7ffc0 0x55fea6c82480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:50.768 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.764+0000 7fccf3400640 1 --2- 172.21.15.72:0/164535362 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x55fea6c7ffc0 0x55fea6c82480 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ebabe748970eb855 server_cookie=9f2fd49cfda23b36 in_seq=0 out_seq=0 2024-04-26T21:58:50.768 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.764+0000 7fcce1600640 10 client.16275 ms_handle_connect on v2:172.21.15.76:6838/327145636 2024-04-26T21:58:50.769 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.766+0000 7fcce1600640 1 -- 172.21.15.72:0/164535362 <== mds.0 v2:172.21.15.76:6838/327145636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55fea6bdc7e0 con 0x55fea6c7ffc0 2024-04-26T21:58:50.769 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.766+0000 7fcce1600640 10 client.16275 handle_command_reply: tid=0 2024-04-26T21:58:50.769 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.766+0000 7fccfd390740 1 client.16275 shutdown 2024-04-26T21:58:50.769 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.766+0000 7fccfd390740 1 -- 172.21.15.72:0/164535362 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55fea6c704c0 msgr2=0x55fea6c7ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:50.769 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.766+0000 7fccfd390740 1 --2- 172.21.15.72:0/164535362 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55fea6c704c0 0x55fea6c7ba60 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fccec0040b0 tx=0x7fccec005e90 comp rx=0 tx=0).stop 2024-04-26T21:58:50.769 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.766+0000 7fccfd390740 1 -- 172.21.15.72:0/164535362 shutdown_connections 2024-04-26T21:58:50.769 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.766+0000 7fccfd390740 1 --2- 172.21.15.72:0/164535362 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x55fea6c7ffc0 0x55fea6c82480 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:50.769 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.766+0000 7fccfd390740 1 --2- 172.21.15.72:0/164535362 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55fea6c704c0 0x55fea6c7ba60 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:50.769 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.766+0000 7fccfd390740 1 --2- 172.21.15.72:0/164535362 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fea6c6b350 0x55fea6c74a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:50.769 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.766+0000 7fccfd390740 1 --2- 172.21.15.72:0/164535362 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fea6c6a960 0x55fea6c744e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:50.769 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.766+0000 7fccfd390740 1 -- 172.21.15.72:0/164535362 >> 172.21.15.72:0/164535362 conn(0x55fea6c65fd0 msgr2=0x55fea6c651f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:58:50.769 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.766+0000 7fcce1600640 0 client.16275 ms_handle_reset on v2:172.21.15.76:6838/327145636 2024-04-26T21:58:50.769 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.766+0000 7fccfd390740 1 -- 172.21.15.72:0/164535362 shutdown_connections 2024-04-26T21:58:50.769 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.766+0000 7fccfd390740 1 -- 172.21.15.72:0/164535362 wait complete. 2024-04-26T21:58:50.769 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.767+0000 7fccfd390740 20 client.16275 trim_cache size 0 max 16384 2024-04-26T21:58:50.786 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.786+0000 7fccfd390740 1 Processor -- start 2024-04-26T21:58:50.786 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.786+0000 7fccfd390740 1 -- start start 2024-04-26T21:58:50.786 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.786+0000 7fccfd390740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fea6c6a960 0x55fea6c65070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:50.786 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.786+0000 7fccfd390740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fea6c6b350 0x55fea6c655b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:50.786 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.786+0000 7fccfd390740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55fea6c704c0 0x55fea6c7ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:50.786 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.786+0000 7fccfd390740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fccf41b1d90 con 0x55fea6c704c0 2024-04-26T21:58:50.786 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.786+0000 7fccfd390740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fccf41b0920 con 0x55fea6c6b350 2024-04-26T21:58:50.786 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.786+0000 7fccfd390740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fccf41b0ff0 con 0x55fea6c6a960 2024-04-26T21:58:50.789 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.786+0000 7fccf3e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55fea6c704c0 0x55fea6c7ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:50.789 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.786+0000 7fccf3e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55fea6c704c0 0x55fea6c7ba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:57494/0 (socket says 172.21.15.72:57494) 2024-04-26T21:58:50.789 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.786+0000 7fccf3e00640 1 -- 172.21.15.72:0/845286583 learned_addr learned my addr 172.21.15.72:0/845286583 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:58:50.789 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.787+0000 7fccf3e00640 1 -- 172.21.15.72:0/845286583 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fea6c6a960 msgr2=0x55fea6c65070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:58:50.789 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.787+0000 7fccf3e00640 1 --2- 172.21.15.72:0/845286583 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fea6c6a960 0x55fea6c65070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:50.789 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.787+0000 7fccebe00640 1 --2- 172.21.15.72:0/845286583 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fea6c6b350 0x55fea6c655b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:50.789 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.787+0000 7fccf3e00640 1 -- 172.21.15.72:0/845286583 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fea6c6b350 msgr2=0x55fea6c655b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:58:50.789 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.787+0000 7fccf3e00640 1 --2- 172.21.15.72:0/845286583 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fea6c6b350 0x55fea6c655b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:50.789 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.787+0000 7fccf3e00640 1 -- 172.21.15.72:0/845286583 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fea68a00f0 con 0x55fea6c704c0 2024-04-26T21:58:50.789 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.787+0000 7fccf3e00640 1 --2- 172.21.15.72:0/845286583 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55fea6c704c0 0x55fea6c7ba20 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fccec02a040 tx=0x7fccec007550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf68f7278ad3bb18 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:58:50.790 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.787+0000 7fcce3e00640 1 -- 172.21.15.72:0/845286583 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccec004d60 con 0x55fea6c704c0 2024-04-26T21:58:50.790 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.787+0000 7fcce3e00640 1 -- 172.21.15.72:0/845286583 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fccec038070 con 0x55fea6c704c0 2024-04-26T21:58:50.790 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.787+0000 7fcce3e00640 1 -- 172.21.15.72:0/845286583 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccec005290 con 0x55fea6c704c0 2024-04-26T21:58:50.790 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.788+0000 7fccfd390740 1 -- 172.21.15.72:0/845286583 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55fea6c704c0 msgr2=0x55fea6c7ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:50.790 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.788+0000 7fccfd390740 1 --2- 172.21.15.72:0/845286583 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55fea6c704c0 0x55fea6c7ba20 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fccec02a040 tx=0x7fccec007550 comp rx=0 tx=0).stop 2024-04-26T21:58:50.790 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.788+0000 7fccfd390740 1 -- 172.21.15.72:0/845286583 shutdown_connections 2024-04-26T21:58:50.790 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.788+0000 7fccfd390740 1 --2- 172.21.15.72:0/845286583 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55fea6c704c0 0x55fea6c7ba20 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:50.790 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.788+0000 7fccfd390740 1 --2- 172.21.15.72:0/845286583 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fea6c6b350 0x55fea6c655b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:50.790 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.788+0000 7fccfd390740 1 --2- 172.21.15.72:0/845286583 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fea6c6a960 0x55fea6c65070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:50.790 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.788+0000 7fccfd390740 1 -- 172.21.15.72:0/845286583 >> 172.21.15.72:0/845286583 conn(0x55fea6c65fd0 msgr2=0x55fea6c64960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:58:50.791 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.788+0000 7fccfd390740 1 -- 172.21.15.72:0/845286583 shutdown_connections 2024-04-26T21:58:50.791 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.788+0000 7fccfd390740 1 -- 172.21.15.72:0/845286583 wait complete. 2024-04-26T21:58:50.791 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.789+0000 7fccfd390740 1 -- start start 2024-04-26T21:58:50.791 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.789+0000 7fccfd390740 1 Processor -- start 2024-04-26T21:58:50.795 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.789+0000 7fccfd390740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fea6c6a960 0x55fea6c789f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:50.795 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.789+0000 7fccfd390740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55fea6c6b350 0x55fea6c78f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:50.795 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.789+0000 7fccfd390740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fea6c704c0 0x55fea6ce45b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:50.795 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.789+0000 7fccfd390740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fccf41ae230 con 0x55fea6c6b350 2024-04-26T21:58:50.796 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.789+0000 7fccfd390740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fccf41b16c0 con 0x55fea6c704c0 2024-04-26T21:58:50.796 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.790+0000 7fccfd390740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fccf41ae8a0 con 0x55fea6c6a960 2024-04-26T21:58:50.796 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.790+0000 7fccfd390740 10 client.0 _ll_register_callbacks cb 0x7fccec014070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:58:50.796 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.790+0000 7fccebe00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55fea6c6b350 0x55fea6c78f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:50.796 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.790+0000 7fccebe00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55fea6c6b350 0x55fea6c78f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:57508/0 (socket says 172.21.15.72:57508) 2024-04-26T21:58:50.796 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.790+0000 7fccebe00640 1 -- 172.21.15.72:0/299033607 learned_addr learned my addr 172.21.15.72:0/299033607 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:58:50.796 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.790+0000 7fccf3e00640 1 --2- 172.21.15.72:0/299033607 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fea6c704c0 0x55fea6ce45b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:50.796 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.790+0000 7fccf3400640 1 --2- 172.21.15.72:0/299033607 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fea6c6a960 0x55fea6c789f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:50.796 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.790+0000 7fccebe00640 1 -- 172.21.15.72:0/299033607 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fea6c6a960 msgr2=0x55fea6c789f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:58:50.796 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.790+0000 7fccebe00640 1 --2- 172.21.15.72:0/299033607 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fea6c6a960 0x55fea6c789f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:50.796 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.790+0000 7fccebe00640 1 -- 172.21.15.72:0/299033607 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fea6c704c0 msgr2=0x55fea6ce45b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:58:50.797 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.790+0000 7fccebe00640 1 --2- 172.21.15.72:0/299033607 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fea6c704c0 0x55fea6ce45b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:50.797 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.790+0000 7fccebe00640 1 -- 172.21.15.72:0/299033607 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fea6b96a30 con 0x55fea6c6b350 2024-04-26T21:58:50.797 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.790+0000 7fccf3400640 1 --2- 172.21.15.72:0/299033607 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fea6c6a960 0x55fea6c789f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:58:50.797 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.790+0000 7fccf3e00640 1 --2- 172.21.15.72:0/299033607 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fea6c704c0 0x55fea6ce45b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:58:50.797 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.791+0000 7fccebe00640 1 --2- 172.21.15.72:0/299033607 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55fea6c6b350 0x55fea6c78f30 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fccf41a0430 tx=0x7fccdc0137d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75109a6a55d2e29c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:58:50.797 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.791+0000 7fcce1600640 10 client.0 ms_handle_connect on v2:172.21.15.72:3300/0 2024-04-26T21:58:50.797 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.791+0000 7fcce1600640 1 -- 172.21.15.72:0/299033607 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccdc020060 con 0x55fea6c6b350 2024-04-26T21:58:50.797 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.791+0000 7fcce1600640 1 -- 172.21.15.72:0/299033607 <== mon.0 v2:172.21.15.72:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7fccdc01b410 con 0x55fea6c6b350 2024-04-26T21:58:50.797 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.791+0000 7fccfd390740 1 -- 172.21.15.72:0/299033607 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fccec038070 con 0x55fea6c6b350 2024-04-26T21:58:50.797 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.791+0000 7fcce1600640 1 -- 172.21.15.72:0/299033607 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fccdc01b610 con 0x55fea6c6b350 2024-04-26T21:58:50.797 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.791+0000 7fccfd390740 10 client.16281 fetch_fsmap learned FSMap version 27 2024-04-26T21:58:50.797 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.791+0000 7fccfd390740 1 -- 172.21.15.72:0/299033607 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55fea6ce5ce0 con 0x55fea6c6b350 2024-04-26T21:58:50.797 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.791+0000 7fcce1600640 1 -- 172.21.15.72:0/299033607 <== mon.0 v2:172.21.15.72:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fccec038070 con 0x55fea6c6b350 2024-04-26T21:58:50.798 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.792+0000 7fcce1600640 1 -- 172.21.15.72:0/299033607 <== mon.0 v2:172.21.15.72:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fccdc00f040 con 0x55fea6c6b350 2024-04-26T21:58:50.798 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.792+0000 7fccfd390740 10 client.16281 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T21:58:50.798 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.792+0000 7fccfd390740 10 client.16281 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:58:50.798 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.792+0000 7fccfd390740 20 client.16281 populate_metadata read hostname 'smithi072' 2024-04-26T21:58:50.798 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.792+0000 7fccfd390740 1 --2- 172.21.15.72:0/299033607 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x55fea6ce66f0 0x55fea6ce8b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:50.798 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.792+0000 7fccfd390740 4 client.16281 mds_command: new command op to 34517 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-26T21:58:50.798 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.792+0000 7fccfd390740 1 -- 172.21.15.72:0/299033607 --> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55fea6bd6650 con 0x55fea6ce66f0 2024-04-26T21:58:50.798 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.792+0000 7fccf3400640 1 --2- 172.21.15.72:0/299033607 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x55fea6ce66f0 0x55fea6ce8b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:50.798 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.793+0000 7fccf3400640 1 --2- 172.21.15.72:0/299033607 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x55fea6ce66f0 0x55fea6ce8b10 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cef494967abf5bc7 server_cookie=796adeda48efff73 in_seq=0 out_seq=0 2024-04-26T21:58:50.799 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.793+0000 7fcce1600640 10 client.16281 ms_handle_connect on v2:172.21.15.76:6838/327145636 2024-04-26T21:58:50.799 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.794+0000 7fcce1600640 1 -- 172.21.15.72:0/299033607 <== mds.0 v2:172.21.15.76:6838/327145636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55fea6bd6650 con 0x55fea6ce66f0 2024-04-26T21:58:50.799 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.794+0000 7fcce1600640 10 client.16281 handle_command_reply: tid=0 2024-04-26T21:58:50.799 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.794+0000 7fccfd390740 1 client.16281 shutdown 2024-04-26T21:58:50.799 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.794+0000 7fccfd390740 1 -- 172.21.15.72:0/299033607 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55fea6c6b350 msgr2=0x55fea6c78f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:50.799 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.794+0000 7fccfd390740 1 --2- 172.21.15.72:0/299033607 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55fea6c6b350 0x55fea6c78f30 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fccf41a0430 tx=0x7fccdc0137d0 comp rx=0 tx=0).stop 2024-04-26T21:58:50.799 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.794+0000 7fccfd390740 1 -- 172.21.15.72:0/299033607 shutdown_connections 2024-04-26T21:58:50.799 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.794+0000 7fccfd390740 1 --2- 172.21.15.72:0/299033607 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x55fea6ce66f0 0x55fea6ce8b10 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:50.799 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.794+0000 7fccfd390740 1 --2- 172.21.15.72:0/299033607 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fea6c704c0 0x55fea6ce45b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:50.799 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.794+0000 7fccfd390740 1 --2- 172.21.15.72:0/299033607 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55fea6c6b350 0x55fea6c78f30 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:50.799 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.794+0000 7fccfd390740 1 --2- 172.21.15.72:0/299033607 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fea6c6a960 0x55fea6c789f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:50.799 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.794+0000 7fcce1600640 0 client.16281 ms_handle_reset on v2:172.21.15.76:6838/327145636 2024-04-26T21:58:50.799 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.794+0000 7fccfd390740 1 -- 172.21.15.72:0/299033607 >> 172.21.15.72:0/299033607 conn(0x55fea6c65fd0 msgr2=0x55fea6c651f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:58:50.800 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.794+0000 7fccfd390740 1 -- 172.21.15.72:0/299033607 shutdown_connections 2024-04-26T21:58:50.800 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.794+0000 7fccfd390740 1 -- 172.21.15.72:0/299033607 wait complete. 2024-04-26T21:58:50.800 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.795+0000 7fccfd390740 20 client.16281 trim_cache size 0 max 16384 2024-04-26T21:58:50.800 INFO:teuthology.orchestra.run.smithi072.stdout:{ 2024-04-26T21:58:50.800 INFO:teuthology.orchestra.run.smithi072.stdout: "status": "no active scrubs running", 2024-04-26T21:58:50.800 INFO:teuthology.orchestra.run.smithi072.stdout: "scrubs": {} 2024-04-26T21:58:50.800 INFO:teuthology.orchestra.run.smithi072.stdout:} 2024-04-26T21:58:50.801 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.795+0000 7fccf8e00640 1 -- 172.21.15.72:0/1163237315 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fccd40a3c40 msgr2=0x7fccd40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:50.801 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.795+0000 7fccf8e00640 1 --2- 172.21.15.72:0/1163237315 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fccd40a3c40 0x7fccd40a6100 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x55fea6c6bbb0 tx=0x7fccec0073a0 comp rx=0 tx=0).stop 2024-04-26T21:58:50.801 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.795+0000 7fccf8e00640 1 -- 172.21.15.72:0/1163237315 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fccf4108e70 msgr2=0x7fccf419ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:50.801 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.795+0000 7fccf8e00640 1 --2- 172.21.15.72:0/1163237315 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fccf4108e70 0x7fccf419ef20 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fcce4009230 tx=0x7fcce40042f0 comp rx=0 tx=0).stop 2024-04-26T21:58:50.802 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.795+0000 7fccf8e00640 1 -- 172.21.15.72:0/1163237315 shutdown_connections 2024-04-26T21:58:50.802 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.795+0000 7fccf8e00640 1 --2- 172.21.15.72:0/1163237315 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fccd40a3c40 0x7fccd40a6100 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:50.802 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.795+0000 7fccf8e00640 1 --2- 172.21.15.72:0/1163237315 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fccf410a980 0x7fccf41a64d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:50.802 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.795+0000 7fccf8e00640 1 --2- 172.21.15.72:0/1163237315 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fccf4109fc0 0x7fccf419f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:50.802 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.795+0000 7fccf8e00640 1 --2- 172.21.15.72:0/1163237315 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fccf4108e70 0x7fccf419ef20 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:50.802 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.795+0000 7fccf8e00640 1 -- 172.21.15.72:0/1163237315 >> 172.21.15.72:0/1163237315 conn(0x7fccf4077e00 msgr2=0x7fccf410d3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:58:50.802 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.796+0000 7fccf8e00640 1 -- 172.21.15.72:0/1163237315 shutdown_connections 2024-04-26T21:58:50.802 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:50.796+0000 7fccf8e00640 1 -- 172.21.15.72:0/1163237315 wait complete. 2024-04-26T21:58:50.815 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-26T21:58:50.816 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:58:51.008 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.008+0000 7f2a6a000640 1 -- 172.21.15.72:0/3794364890 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2a64103b60 msgr2=0x7f2a64103fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:51.009 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.008+0000 7f2a6a000640 1 --2- 172.21.15.72:0/3794364890 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2a64103b60 0x7f2a64103fe0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f2a4c009900 tx=0x7f2a4c02a410 comp rx=0 tx=0).stop 2024-04-26T21:58:51.009 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.008+0000 7f2a6a000640 1 -- 172.21.15.72:0/3794364890 shutdown_connections 2024-04-26T21:58:51.009 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.008+0000 7f2a6a000640 1 --2- 172.21.15.72:0/3794364890 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2a64104520 0x7f2a6410adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:51.009 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.008+0000 7f2a6a000640 1 --2- 172.21.15.72:0/3794364890 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2a64103b60 0x7f2a64103fe0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:51.009 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.008+0000 7f2a6a000640 1 --2- 172.21.15.72:0/3794364890 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2a64102970 0x7f2a64102d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:51.009 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.008+0000 7f2a6a000640 1 -- 172.21.15.72:0/3794364890 >> 172.21.15.72:0/3794364890 conn(0x7f2a640fdee0 msgr2=0x7f2a64100320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:58:51.009 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.008+0000 7f2a6a000640 1 -- 172.21.15.72:0/3794364890 shutdown_connections 2024-04-26T21:58:51.009 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.008+0000 7f2a6a000640 1 -- 172.21.15.72:0/3794364890 wait complete. 2024-04-26T21:58:51.011 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.009+0000 7f2a6a000640 1 Processor -- start 2024-04-26T21:58:51.011 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.009+0000 7f2a6a000640 1 -- start start 2024-04-26T21:58:51.011 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.010+0000 7f2a6a000640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2a64102970 0x7f2a6419ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:51.011 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.010+0000 7f2a6a000640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2a64103b60 0x7f2a6419b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:51.011 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.010+0000 7f2a6a000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2a64104520 0x7f2a641a22a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:51.011 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.010+0000 7f2a6a000640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f2a6410e380 con 0x7f2a64103b60 2024-04-26T21:58:51.012 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.010+0000 7f2a6a000640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f2a6410e200 con 0x7f2a64102970 2024-04-26T21:58:51.012 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.010+0000 7f2a6a000640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f2a6410e500 con 0x7f2a64104520 2024-04-26T21:58:51.012 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.010+0000 7f2a5be00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2a64103b60 0x7f2a6419b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:51.012 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.010+0000 7f2a5be00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2a64103b60 0x7f2a6419b270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:57514/0 (socket says 172.21.15.72:57514) 2024-04-26T21:58:51.012 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.010+0000 7f2a5be00640 1 -- 172.21.15.72:0/4079986962 learned_addr learned my addr 172.21.15.72:0/4079986962 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:58:51.012 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.010+0000 7f2a63400640 1 --2- 172.21.15.72:0/4079986962 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2a64102970 0x7f2a6419ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:51.012 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.010+0000 7f2a5be00640 1 -- 172.21.15.72:0/4079986962 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2a64104520 msgr2=0x7f2a641a22a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:51.012 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.010+0000 7f2a5be00640 1 --2- 172.21.15.72:0/4079986962 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2a64104520 0x7f2a641a22a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:51.012 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.010+0000 7f2a5be00640 1 -- 172.21.15.72:0/4079986962 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2a64102970 msgr2=0x7f2a6419ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:51.012 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.010+0000 7f2a5be00640 1 --2- 172.21.15.72:0/4079986962 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2a64102970 0x7f2a6419ad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:51.012 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.010+0000 7f2a5be00640 1 -- 172.21.15.72:0/4079986962 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a641a2940 con 0x7f2a64103b60 2024-04-26T21:58:51.012 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.011+0000 7f2a5be00640 1 --2- 172.21.15.72:0/4079986962 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2a64103b60 0x7f2a6419b270 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f2a4c031460 tx=0x7f2a4c003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:58:51.012 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.011+0000 7f2a60c00640 1 -- 172.21.15.72:0/4079986962 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a4c03f070 con 0x7f2a64103b60 2024-04-26T21:58:51.013 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.011+0000 7f2a60c00640 1 -- 172.21.15.72:0/4079986962 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2a4c031da0 con 0x7f2a64103b60 2024-04-26T21:58:51.013 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.011+0000 7f2a6a000640 1 -- 172.21.15.72:0/4079986962 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a641a2bd0 con 0x7f2a64103b60 2024-04-26T21:58:51.013 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.011+0000 7f2a60c00640 1 -- 172.21.15.72:0/4079986962 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a4c035030 con 0x7f2a64103b60 2024-04-26T21:58:51.013 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.011+0000 7f2a6a000640 1 -- 172.21.15.72:0/4079986962 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a641a30c0 con 0x7f2a64103b60 2024-04-26T21:58:51.013 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.012+0000 7f2a6a000640 1 -- 172.21.15.72:0/4079986962 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a641a3290 con 0x7f2a64103b60 2024-04-26T21:58:51.014 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.014+0000 7f2a60c00640 1 -- 172.21.15.72:0/4079986962 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f2a4c042070 con 0x7f2a64103b60 2024-04-26T21:58:51.015 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.016+0000 7f2a60c00640 1 --2- 172.21.15.72:0/4079986962 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f2a480a3c40 0x7f2a480a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:51.016 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.016+0000 7f2a60c00640 1 -- 172.21.15.72:0/4079986962 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7f2a4c0f10a0 con 0x7f2a64103b60 2024-04-26T21:58:51.016 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.016+0000 7f2a63400640 1 --2- 172.21.15.72:0/4079986962 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f2a480a3c40 0x7f2a480a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:51.016 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.016+0000 7f2a63400640 1 --2- 172.21.15.72:0/4079986962 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f2a480a3c40 0x7f2a480a6100 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f2a54005fd0 tx=0x7f2a54005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:58:51.020 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.020+0000 7f2a60c00640 1 -- 172.21.15.72:0/4079986962 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f2a4c0bd840 con 0x7f2a64103b60 2024-04-26T21:58:51.266 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.266+0000 7f2a6a000640 1 -- 172.21.15.72:0/4079986962 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2a641a35f0 con 0x7f2a64103b60 2024-04-26T21:58:51.267 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.267+0000 7f2a60c00640 1 -- 172.21.15.72:0/4079986962 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11931 (secure 0 0 0) 0x7f2a4c0bd140 con 0x7f2a64103b60 2024-04-26T21:58:51.268 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T21:58:51.268 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6835/1320381959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6834","nonce":1320381959},{"type":"v1","addr":"172.21.15.72:6835","nonce":1320381959}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6837/2142200512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6836","nonce":2142200512},{"type":"v1","addr":"172.21.15.72:6837","nonce":2142200512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T21:52:50.567857+0000","modified":"2024-04-26T21:55:22.894383+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34517,"mds_1":34472,"mds_2":34442,"mds_3":34406,"mds_4":24463},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14652":{"gid":14652,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.72:6839/1312898281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6838","nonce":1312898281},{"type":"v1","addr":"172.21.15.72:6839","nonce":1312898281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14664":{"gid":14664,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.72:6841/3257692339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6840","nonce":3257692339},{"type":"v1","addr":"172.21.15.72:6841","nonce":3257692339}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24388":{"gid":24388,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6833/4236668465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":4236668465},{"type":"v1","addr":"172.21.15.164:6833","nonce":4236668465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6835/2391114169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":2391114169},{"type":"v1","addr":"172.21.15.164:6835","nonce":2391114169}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6837/3585829992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":3585829992},{"type":"v1","addr":"172.21.15.164:6837","nonce":3585829992}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.164:6839/1636226820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":1636226820},{"type":"v1","addr":"172.21.15.164:6839","nonce":1636226820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34406":{"gid":34406,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":15,"addr":"172.21.15.76:6833/3869483750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":3869483750},{"type":"v1","addr":"172.21.15.76:6833","nonce":3869483750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34442":{"gid":34442,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.76:6835/3517429435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":3517429435},{"type":"v1","addr":"172.21.15.76:6835","nonce":3517429435}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34472":{"gid":34472,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.76:6837/1150678991","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":1150678991},{"type":"v1","addr":"172.21.15.76:6837","nonce":1150678991}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34517":{"gid":34517,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.76:6839/327145636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6838","nonce":327145636},{"type":"v1","addr":"172.21.15.76:6839","nonce":327145636}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34517,"qdb_cluster":[34472,34517,34442,34406,24463]},"id":1}]} 2024-04-26T21:58:51.268 INFO:teuthology.orchestra.run.smithi072.stderr:dumped fsmap epoch 27 2024-04-26T21:58:51.270 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.270+0000 7f2a6a000640 1 -- 172.21.15.72:0/4079986962 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f2a480a3c40 msgr2=0x7f2a480a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:51.270 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.270+0000 7f2a6a000640 1 --2- 172.21.15.72:0/4079986962 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f2a480a3c40 0x7f2a480a6100 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f2a54005fd0 tx=0x7f2a54005850 comp rx=0 tx=0).stop 2024-04-26T21:58:51.270 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.270+0000 7f2a6a000640 1 -- 172.21.15.72:0/4079986962 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2a64103b60 msgr2=0x7f2a6419b270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:51.270 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.270+0000 7f2a6a000640 1 --2- 172.21.15.72:0/4079986962 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2a64103b60 0x7f2a6419b270 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f2a4c031460 tx=0x7f2a4c003c80 comp rx=0 tx=0).stop 2024-04-26T21:58:51.270 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.270+0000 7f2a6a000640 1 -- 172.21.15.72:0/4079986962 shutdown_connections 2024-04-26T21:58:51.270 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.270+0000 7f2a6a000640 1 --2- 172.21.15.72:0/4079986962 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f2a480a3c40 0x7f2a480a6100 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:51.270 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.270+0000 7f2a6a000640 1 --2- 172.21.15.72:0/4079986962 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2a64104520 0x7f2a641a22a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:51.270 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.270+0000 7f2a6a000640 1 --2- 172.21.15.72:0/4079986962 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2a64103b60 0x7f2a6419b270 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:51.270 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.270+0000 7f2a6a000640 1 --2- 172.21.15.72:0/4079986962 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2a64102970 0x7f2a6419ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:51.271 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.270+0000 7f2a6a000640 1 -- 172.21.15.72:0/4079986962 >> 172.21.15.72:0/4079986962 conn(0x7f2a640fdee0 msgr2=0x7f2a64074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:58:51.271 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.270+0000 7f2a6a000640 1 -- 172.21.15.72:0/4079986962 shutdown_connections 2024-04-26T21:58:51.271 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.270+0000 7f2a6a000640 1 -- 172.21.15.72:0/4079986962 wait complete. 2024-04-26T21:58:51.286 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:58:50 smithi072 ceph-mon[28138]: pgmap v551: 97 pgs: 97 active+clean; 258 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 MiB/s wr, 2.43k op/s 2024-04-26T21:58:51.289 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-26T21:58:51.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:58:50 smithi076 ceph-mon[31823]: pgmap v551: 97 pgs: 97 active+clean; 258 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 MiB/s wr, 2.43k op/s 2024-04-26T21:58:51.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:58:50 smithi164 ceph-mon[31916]: pgmap v551: 97 pgs: 97 active+clean; 258 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 MiB/s wr, 2.43k op/s 2024-04-26T21:58:51.492 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.491+0000 7f1363200640 1 -- 172.21.15.72:0/1536269303 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f136410a980 msgr2=0x7f13641112a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:51.492 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.491+0000 7f1363200640 1 --2- 172.21.15.72:0/1536269303 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f136410a980 0x7f13641112a0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f135c009ca0 tx=0x7f135c02a8b0 comp rx=0 tx=0).stop 2024-04-26T21:58:51.492 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.491+0000 7f1363200640 1 -- 172.21.15.72:0/1536269303 shutdown_connections 2024-04-26T21:58:51.492 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.491+0000 7f1363200640 1 --2- 172.21.15.72:0/1536269303 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f136410a980 0x7f13641112a0 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:51.492 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.492+0000 7f1363200640 1 --2- 172.21.15.72:0/1536269303 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1364109fc0 0x7f136410a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:51.492 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.492+0000 7f1363200640 1 --2- 172.21.15.72:0/1536269303 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1364108e70 0x7f1364109290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:51.492 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.492+0000 7f1363200640 1 -- 172.21.15.72:0/1536269303 >> 172.21.15.72:0/1536269303 conn(0x7f1364077e00 msgr2=0x7f136407a240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:58:51.493 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.492+0000 7f1363200640 1 -- 172.21.15.72:0/1536269303 shutdown_connections 2024-04-26T21:58:51.493 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.492+0000 7f1363200640 1 -- 172.21.15.72:0/1536269303 wait complete. 2024-04-26T21:58:51.495 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.492+0000 7f1363200640 1 Processor -- start 2024-04-26T21:58:51.495 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.493+0000 7f1363200640 1 -- start start 2024-04-26T21:58:51.495 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.493+0000 7f1363200640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1364108e70 0x7f136419eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:51.495 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.493+0000 7f1363200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1364109fc0 0x7f136419f420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:51.495 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.493+0000 7f1363200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f136410a980 0x7f13641a6490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:51.495 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.493+0000 7f1363200640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f13641148d0 con 0x7f1364108e70 2024-04-26T21:58:51.495 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.493+0000 7f1363200640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f1364114750 con 0x7f136410a980 2024-04-26T21:58:51.495 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.493+0000 7f1363200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f1364114a50 con 0x7f1364109fc0 2024-04-26T21:58:51.495 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.493+0000 7f1361e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1364108e70 0x7f136419eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:51.496 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.493+0000 7f1361e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1364108e70 0x7f136419eee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:57524/0 (socket says 172.21.15.72:57524) 2024-04-26T21:58:51.496 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.493+0000 7f1361e00640 1 -- 172.21.15.72:0/344549754 learned_addr learned my addr 172.21.15.72:0/344549754 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:58:51.496 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.494+0000 7f135be00640 1 --2- 172.21.15.72:0/344549754 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1364109fc0 0x7f136419f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:51.496 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.494+0000 7f1362800640 1 --2- 172.21.15.72:0/344549754 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f136410a980 0x7f13641a6490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:51.496 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.494+0000 7f1361e00640 1 -- 172.21.15.72:0/344549754 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1364109fc0 msgr2=0x7f136419f420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:51.496 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.494+0000 7f1361e00640 1 --2- 172.21.15.72:0/344549754 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1364109fc0 0x7f136419f420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:51.496 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.494+0000 7f1361e00640 1 -- 172.21.15.72:0/344549754 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f136410a980 msgr2=0x7f13641a6490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:51.496 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.494+0000 7f1361e00640 1 --2- 172.21.15.72:0/344549754 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f136410a980 0x7f13641a6490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:51.496 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.494+0000 7f1361e00640 1 -- 172.21.15.72:0/344549754 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13641a6b00 con 0x7f1364108e70 2024-04-26T21:58:51.496 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.494+0000 7f1362800640 1 --2- 172.21.15.72:0/344549754 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f136410a980 0x7f13641a6490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:58:51.496 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.494+0000 7f1361e00640 1 --2- 172.21.15.72:0/344549754 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1364108e70 0x7f136419eee0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f135400b4a0 tx=0x7f135400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:58:51.496 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.494+0000 7f135aa00640 1 -- 172.21.15.72:0/344549754 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1354014070 con 0x7f1364108e70 2024-04-26T21:58:51.496 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.494+0000 7f135aa00640 1 -- 172.21.15.72:0/344549754 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1354010aa0 con 0x7f1364108e70 2024-04-26T21:58:51.496 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.494+0000 7f135aa00640 1 -- 172.21.15.72:0/344549754 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1354017b70 con 0x7f1364108e70 2024-04-26T21:58:51.497 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.494+0000 7f1363200640 1 -- 172.21.15.72:0/344549754 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13641a6df0 con 0x7f1364108e70 2024-04-26T21:58:51.497 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.494+0000 7f1363200640 1 -- 172.21.15.72:0/344549754 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13641a7280 con 0x7f1364108e70 2024-04-26T21:58:51.498 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.498+0000 7f136bdf3740 1 Processor -- start 2024-04-26T21:58:51.498 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.498+0000 7f135aa00640 1 -- 172.21.15.72:0/344549754 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f1354017070 con 0x7f1364108e70 2024-04-26T21:58:51.498 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.498+0000 7f136bdf3740 1 -- start start 2024-04-26T21:58:51.501 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.498+0000 7f136bdf3740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5630ea568050 0x5630ea568470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:51.501 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.498+0000 7f136bdf3740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5630ea568a40 0x5630ea56d540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:51.501 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.498+0000 7f136bdf3740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5630ea56dbb0 0x5630ea570040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:51.501 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.498+0000 7f136bdf3740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x5630ea4cbc70 con 0x5630ea56dbb0 2024-04-26T21:58:51.501 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.498+0000 7f136bdf3740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5630ea484980 con 0x5630ea568050 2024-04-26T21:58:51.501 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.498+0000 7f136bdf3740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5630ea4c7c30 con 0x5630ea568a40 2024-04-26T21:58:51.501 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.499+0000 7f135be00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5630ea56dbb0 0x5630ea570040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:51.501 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.499+0000 7f135be00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5630ea56dbb0 0x5630ea570040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:57540/0 (socket says 172.21.15.72:57540) 2024-04-26T21:58:51.502 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.499+0000 7f135be00640 1 -- 172.21.15.72:0/1428525819 learned_addr learned my addr 172.21.15.72:0/1428525819 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:58:51.502 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.499+0000 7f1362800640 1 --2- 172.21.15.72:0/1428525819 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5630ea568050 0x5630ea568470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:51.502 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.499+0000 7f1361e00640 1 --2- 172.21.15.72:0/1428525819 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5630ea568a40 0x5630ea56d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:51.502 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.499+0000 7f135be00640 1 -- 172.21.15.72:0/1428525819 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5630ea568a40 msgr2=0x5630ea56d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:58:51.502 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.499+0000 7f135be00640 1 --2- 172.21.15.72:0/1428525819 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5630ea568a40 0x5630ea56d540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:51.502 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.499+0000 7f135be00640 1 -- 172.21.15.72:0/1428525819 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5630ea568050 msgr2=0x5630ea568470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:58:51.502 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.499+0000 7f135be00640 1 --2- 172.21.15.72:0/1428525819 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5630ea568050 0x5630ea568470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:51.502 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.499+0000 7f135be00640 1 -- 172.21.15.72:0/1428525819 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5630ea45c7b0 con 0x5630ea56dbb0 2024-04-26T21:58:51.502 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.499+0000 7f135aa00640 1 --2- 172.21.15.72:0/344549754 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f13480a3c40 0x7f13480a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:51.502 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.499+0000 7f135aa00640 1 -- 172.21.15.72:0/344549754 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7f13540cf960 con 0x7f1364108e70 2024-04-26T21:58:51.503 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.499+0000 7f135be00640 1 --2- 172.21.15.72:0/1428525819 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5630ea56dbb0 0x5630ea570040 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f134c00cc00 tx=0x7f134c00e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad7f85c9685587e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:58:51.503 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.500+0000 7f1362800640 1 --2- 172.21.15.72:0/344549754 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f13480a3c40 0x7f13480a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:51.503 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.500+0000 7f1353400640 1 -- 172.21.15.72:0/1428525819 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f134c004220 con 0x5630ea56dbb0 2024-04-26T21:58:51.503 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.500+0000 7f1353400640 1 -- 172.21.15.72:0/1428525819 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f134c0043c0 con 0x5630ea56dbb0 2024-04-26T21:58:51.503 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.500+0000 7f1353400640 1 -- 172.21.15.72:0/1428525819 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f134c004cd0 con 0x5630ea56dbb0 2024-04-26T21:58:51.503 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.500+0000 7f136bdf3740 1 -- 172.21.15.72:0/1428525819 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5630ea56dbb0 msgr2=0x5630ea570040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:51.503 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.501+0000 7f136bdf3740 1 --2- 172.21.15.72:0/1428525819 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5630ea56dbb0 0x5630ea570040 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f134c00cc00 tx=0x7f134c00e720 comp rx=0 tx=0).stop 2024-04-26T21:58:51.503 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.500+0000 7f1362800640 1 --2- 172.21.15.72:0/344549754 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f13480a3c40 0x7f13480a6100 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f135c009a60 tx=0x7f135c030f00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:58:51.503 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.501+0000 7f136bdf3740 1 -- 172.21.15.72:0/1428525819 shutdown_connections 2024-04-26T21:58:51.503 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.501+0000 7f136bdf3740 1 --2- 172.21.15.72:0/1428525819 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5630ea56dbb0 0x5630ea570040 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:51.503 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.501+0000 7f136bdf3740 1 --2- 172.21.15.72:0/1428525819 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5630ea568a40 0x5630ea56d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:51.503 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.501+0000 7f136bdf3740 1 --2- 172.21.15.72:0/1428525819 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5630ea568050 0x5630ea568470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:51.503 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.501+0000 7f136bdf3740 1 -- 172.21.15.72:0/1428525819 >> 172.21.15.72:0/1428525819 conn(0x5630ea5636c0 msgr2=0x5630ea563ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:58:51.504 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.501+0000 7f136bdf3740 1 -- 172.21.15.72:0/1428525819 shutdown_connections 2024-04-26T21:58:51.504 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.501+0000 7f136bdf3740 1 -- 172.21.15.72:0/1428525819 wait complete. 2024-04-26T21:58:51.504 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.501+0000 7f136bdf3740 1 -- start start 2024-04-26T21:58:51.504 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.501+0000 7f136bdf3740 1 Processor -- start 2024-04-26T21:58:51.509 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.502+0000 7f136bdf3740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5630ea568050 0x5630ea571ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:51.510 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.502+0000 7f136bdf3740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5630ea568a40 0x5630ea572420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:51.510 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.502+0000 7f136bdf3740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5630ea56dbb0 0x5630ea579460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:51.510 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.502+0000 7f136bdf3740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x5630ea4caf80 con 0x5630ea56dbb0 2024-04-26T21:58:51.510 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.502+0000 7f136bdf3740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5630ea478260 con 0x5630ea568050 2024-04-26T21:58:51.510 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.502+0000 7f136bdf3740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5630ea4d02a0 con 0x5630ea568a40 2024-04-26T21:58:51.510 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.502+0000 7f136bdf3740 10 client.0 _ll_register_callbacks cb 0x5630ea4c2960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:58:51.510 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.502+0000 7f1362800640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5630ea56dbb0 0x5630ea579460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:51.510 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.502+0000 7f1362800640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5630ea56dbb0 0x5630ea579460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:57556/0 (socket says 172.21.15.72:57556) 2024-04-26T21:58:51.511 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.502+0000 7f1362800640 1 -- 172.21.15.72:0/2519783224 learned_addr learned my addr 172.21.15.72:0/2519783224 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:58:51.511 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.502+0000 7f1361e00640 1 --2- 172.21.15.72:0/2519783224 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5630ea568050 0x5630ea571ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:51.511 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.502+0000 7f135be00640 1 --2- 172.21.15.72:0/2519783224 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5630ea568a40 0x5630ea572420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:51.511 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.503+0000 7f1362800640 1 -- 172.21.15.72:0/2519783224 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5630ea568a40 msgr2=0x5630ea572420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:58:51.511 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.503+0000 7f1362800640 1 --2- 172.21.15.72:0/2519783224 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5630ea568a40 0x5630ea572420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:51.511 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.503+0000 7f1362800640 1 -- 172.21.15.72:0/2519783224 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5630ea568050 msgr2=0x5630ea571ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:58:51.511 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.503+0000 7f1362800640 1 --2- 172.21.15.72:0/2519783224 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5630ea568050 0x5630ea571ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:51.511 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.503+0000 7f1362800640 1 -- 172.21.15.72:0/2519783224 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5630ea45a1c0 con 0x5630ea56dbb0 2024-04-26T21:58:51.511 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.503+0000 7f1362800640 1 --2- 172.21.15.72:0/2519783224 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5630ea56dbb0 0x5630ea579460 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f135c009420 tx=0x7f135c0095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a292ebb42d2e831 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:58:51.511 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.503+0000 7f1347e00640 10 client.0 ms_handle_connect on v2:172.21.15.72:3300/0 2024-04-26T21:58:51.511 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.503+0000 7f1347e00640 1 -- 172.21.15.72:0/2519783224 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f135c004510 con 0x5630ea56dbb0 2024-04-26T21:58:51.511 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.503+0000 7f1347e00640 1 -- 172.21.15.72:0/2519783224 <== mon.0 v2:172.21.15.72:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7f135c0046b0 con 0x5630ea56dbb0 2024-04-26T21:58:51.511 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.503+0000 7f136bdf3740 1 -- 172.21.15.72:0/2519783224 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f134c0043c0 con 0x5630ea56dbb0 2024-04-26T21:58:51.511 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.504+0000 7f1347e00640 1 -- 172.21.15.72:0/2519783224 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f135c055030 con 0x5630ea56dbb0 2024-04-26T21:58:51.511 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.504+0000 7f136bdf3740 10 client.16299 fetch_fsmap learned FSMap version 27 2024-04-26T21:58:51.512 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.504+0000 7f136bdf3740 1 -- 172.21.15.72:0/2519783224 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5630ea1dd3f0 con 0x5630ea56dbb0 2024-04-26T21:58:51.512 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.504+0000 7f1347e00640 1 -- 172.21.15.72:0/2519783224 <== mon.0 v2:172.21.15.72:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f134c0043c0 con 0x5630ea56dbb0 2024-04-26T21:58:51.512 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.504+0000 7f1347e00640 1 -- 172.21.15.72:0/2519783224 <== mon.0 v2:172.21.15.72:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f135c04e020 con 0x5630ea56dbb0 2024-04-26T21:58:51.512 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.504+0000 7f136bdf3740 10 client.16299 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T21:58:51.512 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.504+0000 7f136bdf3740 10 client.16299 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-04-26T21:58:51.512 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.504+0000 7f136bdf3740 20 client.16299 populate_metadata read hostname 'smithi072' 2024-04-26T21:58:51.512 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.504+0000 7f136bdf3740 1 --2- 172.21.15.72:0/2519783224 >> [v2:172.21.15.164:6838/1636226820,v1:172.21.15.164:6839/1636226820] conn(0x5630ea57d9d0 0x5630ea57fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:51.512 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.504+0000 7f136bdf3740 4 client.16299 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:58:51.512 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.504+0000 7f136bdf3740 1 -- 172.21.15.72:0/2519783224 --> [v2:172.21.15.164:6838/1636226820,v1:172.21.15.164:6839/1636226820] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5630ea4d9880 con 0x5630ea57d9d0 2024-04-26T21:58:51.512 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.505+0000 7f1361e00640 1 --2- 172.21.15.72:0/2519783224 >> [v2:172.21.15.164:6838/1636226820,v1:172.21.15.164:6839/1636226820] conn(0x5630ea57d9d0 0x5630ea57fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:51.512 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.506+0000 7f1361e00640 1 --2- 172.21.15.72:0/2519783224 >> [v2:172.21.15.164:6838/1636226820,v1:172.21.15.164:6839/1636226820] conn(0x5630ea57d9d0 0x5630ea57fe70 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=7aeef93cd2a008fa server_cookie=26873325d241a428 in_seq=0 out_seq=0 2024-04-26T21:58:51.513 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.506+0000 7f1347e00640 10 client.16299 ms_handle_connect on v2:172.21.15.164:6838/1636226820 2024-04-26T21:58:51.514 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.508+0000 7f1347e00640 1 -- 172.21.15.72:0/2519783224 <== mds.4 v2:172.21.15.164:6838/1636226820 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5630ea4d9880 con 0x5630ea57d9d0 2024-04-26T21:58:51.514 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.508+0000 7f1347e00640 10 client.16299 handle_command_reply: tid=0 2024-04-26T21:58:51.514 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.508+0000 7f136bdf3740 1 client.16299 shutdown 2024-04-26T21:58:51.515 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.508+0000 7f136bdf3740 1 -- 172.21.15.72:0/2519783224 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5630ea56dbb0 msgr2=0x5630ea579460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:51.515 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.508+0000 7f136bdf3740 1 --2- 172.21.15.72:0/2519783224 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5630ea56dbb0 0x5630ea579460 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f135c009420 tx=0x7f135c0095c0 comp rx=0 tx=0).stop 2024-04-26T21:58:51.515 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.509+0000 7f136bdf3740 1 -- 172.21.15.72:0/2519783224 shutdown_connections 2024-04-26T21:58:51.515 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.509+0000 7f136bdf3740 1 --2- 172.21.15.72:0/2519783224 >> [v2:172.21.15.164:6838/1636226820,v1:172.21.15.164:6839/1636226820] conn(0x5630ea57d9d0 0x5630ea57fe70 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:51.515 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.509+0000 7f136bdf3740 1 --2- 172.21.15.72:0/2519783224 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5630ea56dbb0 0x5630ea579460 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:51.515 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.509+0000 7f136bdf3740 1 --2- 172.21.15.72:0/2519783224 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5630ea568a40 0x5630ea572420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:51.515 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.509+0000 7f1347e00640 0 client.16299 ms_handle_reset on v2:172.21.15.164:6838/1636226820 2024-04-26T21:58:51.515 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.509+0000 7f136bdf3740 1 --2- 172.21.15.72:0/2519783224 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5630ea568050 0x5630ea571ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:51.515 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.509+0000 7f136bdf3740 1 -- 172.21.15.72:0/2519783224 >> 172.21.15.72:0/2519783224 conn(0x5630ea5636c0 msgr2=0x5630ea5627b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:58:51.515 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.509+0000 7f136bdf3740 1 -- 172.21.15.72:0/2519783224 shutdown_connections 2024-04-26T21:58:51.516 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.509+0000 7f136bdf3740 1 -- 172.21.15.72:0/2519783224 wait complete. 2024-04-26T21:58:51.516 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.509+0000 7f136bdf3740 20 client.16299 trim_cache size 0 max 16384 2024-04-26T21:58:51.524 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.523+0000 7f136bdf3740 1 Processor -- start 2024-04-26T21:58:51.524 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.523+0000 7f136bdf3740 1 -- start start 2024-04-26T21:58:51.524 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.524+0000 7f136bdf3740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5630ea568050 0x5630ea55e640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:51.524 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.524+0000 7f136bdf3740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5630ea568a40 0x5630ea579420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:51.524 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.524+0000 7f136bdf3740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5630ea56dbb0 0x5630ea5694b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:51.524 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.524+0000 7f136bdf3740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f13641b0190 con 0x5630ea568050 2024-04-26T21:58:51.524 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.524+0000 7f136bdf3740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f13641b1cd0 con 0x5630ea56dbb0 2024-04-26T21:58:51.524 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.524+0000 7f136bdf3740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f13641b0860 con 0x5630ea568a40 2024-04-26T21:58:51.525 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.524+0000 7f1361e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5630ea568050 0x5630ea55e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:51.526 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.525+0000 7f135be00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5630ea568a40 0x5630ea579420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:51.526 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.525+0000 7f1361e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5630ea568050 0x5630ea55e640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:57568/0 (socket says 172.21.15.72:57568) 2024-04-26T21:58:51.526 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.525+0000 7f1361e00640 1 -- 172.21.15.72:0/3838234043 learned_addr learned my addr 172.21.15.72:0/3838234043 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:58:51.526 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.525+0000 7f1362800640 1 --2- 172.21.15.72:0/3838234043 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5630ea56dbb0 0x5630ea5694b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:51.526 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.525+0000 7f1361e00640 1 -- 172.21.15.72:0/3838234043 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5630ea568a40 msgr2=0x5630ea579420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:58:51.526 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.525+0000 7f1361e00640 1 --2- 172.21.15.72:0/3838234043 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5630ea568a40 0x5630ea579420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:51.526 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.525+0000 7f1361e00640 1 -- 172.21.15.72:0/3838234043 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5630ea56dbb0 msgr2=0x5630ea5694b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:58:51.526 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.525+0000 7f1361e00640 1 --2- 172.21.15.72:0/3838234043 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5630ea56dbb0 0x5630ea5694b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:51.526 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.525+0000 7f1361e00640 1 -- 172.21.15.72:0/3838234043 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5630ea2a77d0 con 0x5630ea568050 2024-04-26T21:58:51.527 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.525+0000 7f1361e00640 1 --2- 172.21.15.72:0/3838234043 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5630ea568050 0x5630ea55e640 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f1354097fd0 tx=0x7f13540bc040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f5aa87194687fc6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:58:51.527 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.525+0000 7f1352a00640 1 -- 172.21.15.72:0/3838234043 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f135401e040 con 0x5630ea568050 2024-04-26T21:58:51.527 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.525+0000 7f1352a00640 1 -- 172.21.15.72:0/3838234043 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f13540a47e0 con 0x5630ea568050 2024-04-26T21:58:51.527 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.525+0000 7f1352a00640 1 -- 172.21.15.72:0/3838234043 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1354010c50 con 0x5630ea568050 2024-04-26T21:58:51.530 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.526+0000 7f136bdf3740 1 -- 172.21.15.72:0/3838234043 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5630ea568050 msgr2=0x5630ea55e640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:51.531 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.526+0000 7f136bdf3740 1 --2- 172.21.15.72:0/3838234043 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5630ea568050 0x5630ea55e640 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f1354097fd0 tx=0x7f13540bc040 comp rx=0 tx=0).stop 2024-04-26T21:58:51.531 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.526+0000 7f136bdf3740 1 -- 172.21.15.72:0/3838234043 shutdown_connections 2024-04-26T21:58:51.531 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.526+0000 7f136bdf3740 1 --2- 172.21.15.72:0/3838234043 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5630ea56dbb0 0x5630ea5694b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:51.531 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.526+0000 7f136bdf3740 1 --2- 172.21.15.72:0/3838234043 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5630ea568a40 0x5630ea579420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:51.531 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.526+0000 7f136bdf3740 1 --2- 172.21.15.72:0/3838234043 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5630ea568050 0x5630ea55e640 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:51.531 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.526+0000 7f136bdf3740 1 -- 172.21.15.72:0/3838234043 >> 172.21.15.72:0/3838234043 conn(0x5630ea5636c0 msgr2=0x5630ea5627b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:58:51.531 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.526+0000 7f136bdf3740 1 -- 172.21.15.72:0/3838234043 shutdown_connections 2024-04-26T21:58:51.531 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.526+0000 7f136bdf3740 1 -- 172.21.15.72:0/3838234043 wait complete. 2024-04-26T21:58:51.531 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.527+0000 7f136bdf3740 1 -- start start 2024-04-26T21:58:51.531 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.527+0000 7f136bdf3740 1 Processor -- start 2024-04-26T21:58:51.532 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.527+0000 7f136bdf3740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5630ea568050 0x5630ea5772b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:51.532 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.527+0000 7f136bdf3740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5630ea568a40 0x5630ea5777f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:51.532 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.527+0000 7f136bdf3740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5630ea56dbb0 0x5630ea578800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:51.532 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.527+0000 7f136bdf3740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f13641b1600 con 0x5630ea568a40 2024-04-26T21:58:51.532 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.527+0000 7f136bdf3740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f13641b0f30 con 0x5630ea56dbb0 2024-04-26T21:58:51.532 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.527+0000 7f136bdf3740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f13641ae170 con 0x5630ea568050 2024-04-26T21:58:51.532 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.527+0000 7f136bdf3740 10 client.0 _ll_register_callbacks cb 0x7f135c009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:58:51.532 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.527+0000 7f135be00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5630ea568a40 0x5630ea5777f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:51.532 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.528+0000 7f135be00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5630ea568a40 0x5630ea5777f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:57578/0 (socket says 172.21.15.72:57578) 2024-04-26T21:58:51.532 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.528+0000 7f135be00640 1 -- 172.21.15.72:0/1500100509 learned_addr learned my addr 172.21.15.72:0/1500100509 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:58:51.532 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.528+0000 7f1361e00640 1 --2- 172.21.15.72:0/1500100509 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5630ea568050 0x5630ea5772b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:51.532 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.528+0000 7f135be00640 1 -- 172.21.15.72:0/1500100509 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5630ea568050 msgr2=0x5630ea5772b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:58:51.532 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.528+0000 7f135be00640 1 --2- 172.21.15.72:0/1500100509 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5630ea568050 0x5630ea5772b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:51.533 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.528+0000 7f1362800640 1 --2- 172.21.15.72:0/1500100509 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5630ea56dbb0 0x5630ea578800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:51.533 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.528+0000 7f135be00640 1 -- 172.21.15.72:0/1500100509 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5630ea56dbb0 msgr2=0x5630ea578800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:58:51.533 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.528+0000 7f135be00640 1 --2- 172.21.15.72:0/1500100509 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5630ea56dbb0 0x5630ea578800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:51.533 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.528+0000 7f135be00640 1 -- 172.21.15.72:0/1500100509 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5630ea4722a0 con 0x5630ea568a40 2024-04-26T21:58:51.533 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.528+0000 7f135be00640 1 --2- 172.21.15.72:0/1500100509 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5630ea568a40 0x5630ea5777f0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x5630ea564590 tx=0x7f134c01ab40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52dc8d5450147385 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:58:51.533 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.528+0000 7f1347e00640 10 client.0 ms_handle_connect on v2:172.21.15.72:3300/0 2024-04-26T21:58:51.533 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.528+0000 7f1347e00640 1 -- 172.21.15.72:0/1500100509 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f134c02b060 con 0x5630ea568a40 2024-04-26T21:58:51.533 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.529+0000 7f1347e00640 1 -- 172.21.15.72:0/1500100509 <== mon.0 v2:172.21.15.72:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7f134c005070 con 0x5630ea568a40 2024-04-26T21:58:51.533 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.529+0000 7f136bdf3740 1 -- 172.21.15.72:0/1500100509 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f13540a47e0 con 0x5630ea568a40 2024-04-26T21:58:51.533 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.529+0000 7f1347e00640 1 -- 172.21.15.72:0/1500100509 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f134c0052b0 con 0x5630ea568a40 2024-04-26T21:58:51.533 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.529+0000 7f136bdf3740 10 client.16305 fetch_fsmap learned FSMap version 27 2024-04-26T21:58:51.533 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.529+0000 7f136bdf3740 1 -- 172.21.15.72:0/1500100509 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5630ea5e4570 con 0x5630ea568a40 2024-04-26T21:58:51.533 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.529+0000 7f1347e00640 1 -- 172.21.15.72:0/1500100509 <== mon.0 v2:172.21.15.72:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f13540a47e0 con 0x5630ea568a40 2024-04-26T21:58:51.533 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.529+0000 7f1347e00640 1 -- 172.21.15.72:0/1500100509 <== mon.0 v2:172.21.15.72:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f134c010430 con 0x5630ea568a40 2024-04-26T21:58:51.533 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.529+0000 7f136bdf3740 10 client.16305 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T21:58:51.534 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.529+0000 7f136bdf3740 10 client.16305 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-04-26T21:58:51.534 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.529+0000 7f136bdf3740 20 client.16305 populate_metadata read hostname 'smithi072' 2024-04-26T21:58:51.534 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.529+0000 7f136bdf3740 1 --2- 172.21.15.72:0/1500100509 >> [v2:172.21.15.164:6838/1636226820,v1:172.21.15.164:6839/1636226820] conn(0x5630ea5e4c80 0x5630ea5e70a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:51.534 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.529+0000 7f136bdf3740 4 client.16305 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-26T21:58:51.534 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.529+0000 7f136bdf3740 1 -- 172.21.15.72:0/1500100509 --> [v2:172.21.15.164:6838/1636226820,v1:172.21.15.164:6839/1636226820] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5630ea4d36f0 con 0x5630ea5e4c80 2024-04-26T21:58:51.534 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.530+0000 7f1361e00640 1 --2- 172.21.15.72:0/1500100509 >> [v2:172.21.15.164:6838/1636226820,v1:172.21.15.164:6839/1636226820] conn(0x5630ea5e4c80 0x5630ea5e70a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:51.534 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.531+0000 7f1361e00640 1 --2- 172.21.15.72:0/1500100509 >> [v2:172.21.15.164:6838/1636226820,v1:172.21.15.164:6839/1636226820] conn(0x5630ea5e4c80 0x5630ea5e70a0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a3fbe0ba4ad0d929 server_cookie=5c9a694ec887436c in_seq=0 out_seq=0 2024-04-26T21:58:51.534 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.531+0000 7f1347e00640 10 client.16305 ms_handle_connect on v2:172.21.15.164:6838/1636226820 2024-04-26T21:58:51.535 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.531+0000 7f1347e00640 1 -- 172.21.15.72:0/1500100509 <== mds.4 v2:172.21.15.164:6838/1636226820 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5630ea4d36f0 con 0x5630ea5e4c80 2024-04-26T21:58:51.535 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.531+0000 7f1347e00640 10 client.16305 handle_command_reply: tid=0 2024-04-26T21:58:51.535 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.531+0000 7f136bdf3740 1 client.16305 shutdown 2024-04-26T21:58:51.536 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.532+0000 7f136bdf3740 1 -- 172.21.15.72:0/1500100509 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5630ea568a40 msgr2=0x5630ea5777f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:51.536 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.532+0000 7f136bdf3740 1 --2- 172.21.15.72:0/1500100509 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5630ea568a40 0x5630ea5777f0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x5630ea564590 tx=0x7f134c01ab40 comp rx=0 tx=0).stop 2024-04-26T21:58:51.536 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.532+0000 7f136bdf3740 1 -- 172.21.15.72:0/1500100509 shutdown_connections 2024-04-26T21:58:51.536 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.532+0000 7f136bdf3740 1 --2- 172.21.15.72:0/1500100509 >> [v2:172.21.15.164:6838/1636226820,v1:172.21.15.164:6839/1636226820] conn(0x5630ea5e4c80 0x5630ea5e70a0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:51.536 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.532+0000 7f136bdf3740 1 --2- 172.21.15.72:0/1500100509 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5630ea56dbb0 0x5630ea578800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:51.536 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.532+0000 7f1347e00640 0 client.16305 ms_handle_reset on v2:172.21.15.164:6838/1636226820 2024-04-26T21:58:51.536 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.532+0000 7f136bdf3740 1 --2- 172.21.15.72:0/1500100509 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5630ea568a40 0x5630ea5777f0 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:51.536 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.532+0000 7f136bdf3740 1 --2- 172.21.15.72:0/1500100509 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5630ea568050 0x5630ea5772b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:51.536 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.532+0000 7f136bdf3740 1 -- 172.21.15.72:0/1500100509 >> 172.21.15.72:0/1500100509 conn(0x5630ea5636c0 msgr2=0x5630ea5665d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:58:51.536 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.532+0000 7f136bdf3740 1 -- 172.21.15.72:0/1500100509 shutdown_connections 2024-04-26T21:58:51.536 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.532+0000 7f136bdf3740 1 -- 172.21.15.72:0/1500100509 wait complete. 2024-04-26T21:58:51.536 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.532+0000 7f136bdf3740 20 client.16305 trim_cache size 0 max 16384 2024-04-26T21:58:51.537 INFO:teuthology.orchestra.run.smithi072.stdout:[] 2024-04-26T21:58:51.537 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.533+0000 7f1363200640 1 -- 172.21.15.72:0/344549754 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f13480a3c40 msgr2=0x7f13480a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:51.537 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.533+0000 7f1363200640 1 --2- 172.21.15.72:0/344549754 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f13480a3c40 0x7f13480a6100 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f135c009a60 tx=0x7f135c030f00 comp rx=0 tx=0).stop 2024-04-26T21:58:51.538 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.533+0000 7f1363200640 1 -- 172.21.15.72:0/344549754 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1364108e70 msgr2=0x7f136419eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:51.538 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.533+0000 7f1363200640 1 --2- 172.21.15.72:0/344549754 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1364108e70 0x7f136419eee0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f135400b4a0 tx=0x7f135400b960 comp rx=0 tx=0).stop 2024-04-26T21:58:51.538 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.533+0000 7f1363200640 1 -- 172.21.15.72:0/344549754 shutdown_connections 2024-04-26T21:58:51.538 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.533+0000 7f1363200640 1 --2- 172.21.15.72:0/344549754 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f136410a980 0x7f13641a6490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:51.538 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.533+0000 7f1363200640 1 --2- 172.21.15.72:0/344549754 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f13480a3c40 0x7f13480a6100 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:51.538 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.533+0000 7f1363200640 1 --2- 172.21.15.72:0/344549754 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1364109fc0 0x7f136419f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:51.538 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.533+0000 7f1363200640 1 --2- 172.21.15.72:0/344549754 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1364108e70 0x7f136419eee0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:51.538 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.533+0000 7f1363200640 1 -- 172.21.15.72:0/344549754 >> 172.21.15.72:0/344549754 conn(0x7f1364077e00 msgr2=0x7f136410d2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:58:51.538 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.533+0000 7f1363200640 1 -- 172.21.15.72:0/344549754 shutdown_connections 2024-04-26T21:58:51.538 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.533+0000 7f1363200640 1 -- 172.21.15.72:0/344549754 wait complete. 2024-04-26T21:58:51.551 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-26T21:58:51.740 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.740+0000 7f8f31000640 1 -- 172.21.15.72:0/3009878322 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8f2c104a70 msgr2=0x7f8f2c104e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:51.740 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.740+0000 7f8f31000640 1 --2- 172.21.15.72:0/3009878322 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8f2c104a70 0x7f8f2c104e90 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f8f1c009900 tx=0x7f8f1c02a410 comp rx=0 tx=0).stop 2024-04-26T21:58:51.740 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.740+0000 7f8f31000640 1 -- 172.21.15.72:0/3009878322 shutdown_connections 2024-04-26T21:58:51.740 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.740+0000 7f8f31000640 1 --2- 172.21.15.72:0/3009878322 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8f2c106620 0x7f8f2c10ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:51.740 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.740+0000 7f8f31000640 1 --2- 172.21.15.72:0/3009878322 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8f2c105c60 0x7f8f2c1060e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:51.740 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.740+0000 7f8f31000640 1 --2- 172.21.15.72:0/3009878322 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8f2c104a70 0x7f8f2c104e90 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:51.740 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.740+0000 7f8f31000640 1 -- 172.21.15.72:0/3009878322 >> 172.21.15.72:0/3009878322 conn(0x7f8f2c0fffe0 msgr2=0x7f8f2c102420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:58:51.741 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.740+0000 7f8f31000640 1 -- 172.21.15.72:0/3009878322 shutdown_connections 2024-04-26T21:58:51.741 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.740+0000 7f8f31000640 1 -- 172.21.15.72:0/3009878322 wait complete. 2024-04-26T21:58:51.742 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.741+0000 7f8f31000640 1 Processor -- start 2024-04-26T21:58:51.742 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.741+0000 7f8f31000640 1 -- start start 2024-04-26T21:58:51.742 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.741+0000 7f8f31000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8f2c104a70 0x7f8f2c19abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:51.742 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.741+0000 7f8f31000640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8f2c105c60 0x7f8f2c19b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:51.742 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.741+0000 7f8f31000640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8f2c106620 0x7f8f2c1a2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:51.743 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.742+0000 7f8f31000640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f8f2c110420 con 0x7f8f2c106620 2024-04-26T21:58:51.743 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.742+0000 7f8f31000640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f8f2c1102a0 con 0x7f8f2c105c60 2024-04-26T21:58:51.743 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.742+0000 7f8f31000640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f8f2c1105a0 con 0x7f8f2c104a70 2024-04-26T21:58:51.743 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.742+0000 7f8f2be00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8f2c106620 0x7f8f2c1a2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:51.743 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.742+0000 7f8f2be00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8f2c106620 0x7f8f2c1a2120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:57600/0 (socket says 172.21.15.72:57600) 2024-04-26T21:58:51.743 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.742+0000 7f8f2be00640 1 -- 172.21.15.72:0/3713050848 learned_addr learned my addr 172.21.15.72:0/3713050848 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:58:51.743 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.742+0000 7f8f2b400640 1 --2- 172.21.15.72:0/3713050848 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8f2c104a70 0x7f8f2c19abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:51.743 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.742+0000 7f8f2aa00640 1 --2- 172.21.15.72:0/3713050848 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8f2c105c60 0x7f8f2c19b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:51.743 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.742+0000 7f8f2be00640 1 -- 172.21.15.72:0/3713050848 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8f2c104a70 msgr2=0x7f8f2c19abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:51.743 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.742+0000 7f8f2be00640 1 --2- 172.21.15.72:0/3713050848 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8f2c104a70 0x7f8f2c19abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:51.743 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.742+0000 7f8f2be00640 1 -- 172.21.15.72:0/3713050848 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8f2c105c60 msgr2=0x7f8f2c19b100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:51.743 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.742+0000 7f8f2be00640 1 --2- 172.21.15.72:0/3713050848 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8f2c105c60 0x7f8f2c19b100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:51.743 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.742+0000 7f8f2be00640 1 -- 172.21.15.72:0/3713050848 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f2c1a2790 con 0x7f8f2c106620 2024-04-26T21:58:51.743 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.742+0000 7f8f2b400640 1 --2- 172.21.15.72:0/3713050848 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8f2c104a70 0x7f8f2c19abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:58:51.743 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.743+0000 7f8f2be00640 1 --2- 172.21.15.72:0/3713050848 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8f2c106620 0x7f8f2c1a2120 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f8f2400cce0 tx=0x7f8f2400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:58:51.744 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.743+0000 7f8f23e00640 1 -- 172.21.15.72:0/3713050848 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f2400f450 con 0x7f8f2c106620 2024-04-26T21:58:51.744 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.743+0000 7f8f31000640 1 -- 172.21.15.72:0/3713050848 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f2c1a2a80 con 0x7f8f2c106620 2024-04-26T21:58:51.744 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.743+0000 7f8f23e00640 1 -- 172.21.15.72:0/3713050848 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8f2400fa80 con 0x7f8f2c106620 2024-04-26T21:58:51.744 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.743+0000 7f8f23e00640 1 -- 172.21.15.72:0/3713050848 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f24018ad0 con 0x7f8f2c106620 2024-04-26T21:58:51.744 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.743+0000 7f8f31000640 1 -- 172.21.15.72:0/3713050848 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f2c1a2f80 con 0x7f8f2c106620 2024-04-26T21:58:51.746 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.746+0000 7f8f35427740 1 Processor -- start 2024-04-26T21:58:51.746 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.746+0000 7f8f23e00640 1 -- 172.21.15.72:0/3713050848 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f8f2400f5f0 con 0x7f8f2c106620 2024-04-26T21:58:51.747 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.747+0000 7f8f35427740 1 -- start start 2024-04-26T21:58:51.747 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.747+0000 7f8f35427740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561db6bc5050 0x561db6bc5470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:51.747 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.747+0000 7f8f35427740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561db6bc5a40 0x561db6bca540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:51.747 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.747+0000 7f8f35427740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561db6bcabb0 0x561db6bcd040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:51.747 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.747+0000 7f8f35427740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x561db6b28c70 con 0x561db6bc5050 2024-04-26T21:58:51.748 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.747+0000 7f8f35427740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x561db6ae1980 con 0x561db6bc5a40 2024-04-26T21:58:51.748 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.747+0000 7f8f35427740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x561db6b24c30 con 0x561db6bcabb0 2024-04-26T21:58:51.748 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.747+0000 7f8f2aa00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561db6bc5050 0x561db6bc5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:51.748 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.748+0000 7f8f2aa00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561db6bc5050 0x561db6bc5470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:57616/0 (socket says 172.21.15.72:57616) 2024-04-26T21:58:51.748 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.748+0000 7f8f2b400640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561db6bc5a40 0x561db6bca540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:51.748 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.748+0000 7f8f2aa00640 1 -- 172.21.15.72:0/3180426251 learned_addr learned my addr 172.21.15.72:0/3180426251 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:58:51.749 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.748+0000 7f8f2aa00640 1 --2- 172.21.15.72:0/3180426251 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561db6bcabb0 0x561db6bcd040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:51.749 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.748+0000 7f8f2aa00640 1 -- 172.21.15.72:0/3180426251 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561db6bcabb0 msgr2=0x561db6bcd040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:58:51.749 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.748+0000 7f8f2aa00640 1 --2- 172.21.15.72:0/3180426251 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561db6bcabb0 0x561db6bcd040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:51.749 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.748+0000 7f8f2aa00640 1 -- 172.21.15.72:0/3180426251 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561db6bc5a40 msgr2=0x561db6bca540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:58:51.749 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.748+0000 7f8f2aa00640 1 --2- 172.21.15.72:0/3180426251 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561db6bc5a40 0x561db6bca540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:51.749 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.748+0000 7f8f2aa00640 1 -- 172.21.15.72:0/3180426251 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561db6ab97b0 con 0x561db6bc5050 2024-04-26T21:58:51.749 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.748+0000 7f8f2aa00640 1 --2- 172.21.15.72:0/3180426251 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561db6bc5050 0x561db6bc5470 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f8f18013520 tx=0x7f8f1800b6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7022afcc2a3512b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:58:51.750 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.749+0000 7f8f23e00640 1 --2- 172.21.15.72:0/3713050848 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f8f100a3c40 0x7f8f100a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:51.750 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.749+0000 7f8f20c00640 1 -- 172.21.15.72:0/3180426251 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f18019030 con 0x561db6bc5050 2024-04-26T21:58:51.750 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.749+0000 7f8f20c00640 1 -- 172.21.15.72:0/3180426251 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8f18011950 con 0x561db6bc5050 2024-04-26T21:58:51.750 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.749+0000 7f8f20c00640 1 -- 172.21.15.72:0/3180426251 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f1801d6b0 con 0x561db6bc5050 2024-04-26T21:58:51.750 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.749+0000 7f8f23e00640 1 -- 172.21.15.72:0/3713050848 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7f8f240d0ee0 con 0x7f8f2c106620 2024-04-26T21:58:51.750 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.749+0000 7f8f2b400640 1 --2- 172.21.15.72:0/3713050848 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f8f100a3c40 0x7f8f100a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:51.752 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.750+0000 7f8f35427740 1 -- 172.21.15.72:0/3180426251 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561db6bc5050 msgr2=0x561db6bc5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:51.752 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.750+0000 7f8f35427740 1 --2- 172.21.15.72:0/3180426251 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561db6bc5050 0x561db6bc5470 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f8f18013520 tx=0x7f8f1800b6b0 comp rx=0 tx=0).stop 2024-04-26T21:58:51.752 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.750+0000 7f8f2b400640 1 --2- 172.21.15.72:0/3713050848 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f8f100a3c40 0x7f8f100a6100 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f8f1c006010 tx=0x7f8f1c005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:58:51.752 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.750+0000 7f8f35427740 1 -- 172.21.15.72:0/3180426251 shutdown_connections 2024-04-26T21:58:51.752 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.750+0000 7f8f35427740 1 --2- 172.21.15.72:0/3180426251 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561db6bcabb0 0x561db6bcd040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:51.752 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.750+0000 7f8f35427740 1 --2- 172.21.15.72:0/3180426251 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561db6bc5a40 0x561db6bca540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:51.752 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.750+0000 7f8f35427740 1 --2- 172.21.15.72:0/3180426251 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561db6bc5050 0x561db6bc5470 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:51.752 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.750+0000 7f8f35427740 1 -- 172.21.15.72:0/3180426251 >> 172.21.15.72:0/3180426251 conn(0x561db6bc06c0 msgr2=0x561db6bc0ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:58:51.753 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.750+0000 7f8f35427740 1 -- 172.21.15.72:0/3180426251 shutdown_connections 2024-04-26T21:58:51.753 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.750+0000 7f8f35427740 1 -- 172.21.15.72:0/3180426251 wait complete. 2024-04-26T21:58:51.753 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.750+0000 7f8f35427740 1 -- start start 2024-04-26T21:58:51.753 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.751+0000 7f8f35427740 1 Processor -- start 2024-04-26T21:58:51.753 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.751+0000 7f8f35427740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561db6bc5050 0x561db6bcef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:51.753 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.751+0000 7f8f35427740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561db6bc5a40 0x561db6bcf450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:51.753 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.751+0000 7f8f35427740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561db6bcabb0 0x561db6bd6490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:51.753 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.751+0000 7f8f35427740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x561db6b27f80 con 0x561db6bc5050 2024-04-26T21:58:51.753 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.751+0000 7f8f35427740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x561db6ad5260 con 0x561db6bcabb0 2024-04-26T21:58:51.753 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.751+0000 7f8f35427740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x561db6b2d2a0 con 0x561db6bc5a40 2024-04-26T21:58:51.753 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.751+0000 7f8f35427740 10 client.0 _ll_register_callbacks cb 0x561db6b1f960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:58:51.753 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.751+0000 7f8f2b400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561db6bc5050 0x561db6bcef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:51.753 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.751+0000 7f8f2b400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561db6bc5050 0x561db6bcef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:57626/0 (socket says 172.21.15.72:57626) 2024-04-26T21:58:51.753 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.751+0000 7f8f2b400640 1 -- 172.21.15.72:0/311826332 learned_addr learned my addr 172.21.15.72:0/311826332 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:58:51.753 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.752+0000 7f8f2aa00640 1 --2- 172.21.15.72:0/311826332 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561db6bc5a40 0x561db6bcf450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:51.753 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.752+0000 7f8f2be00640 1 --2- 172.21.15.72:0/311826332 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561db6bcabb0 0x561db6bd6490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:51.754 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.752+0000 7f8f2b400640 1 -- 172.21.15.72:0/311826332 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561db6bc5a40 msgr2=0x561db6bcf450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:58:51.754 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.752+0000 7f8f2b400640 1 --2- 172.21.15.72:0/311826332 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561db6bc5a40 0x561db6bcf450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:51.754 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.752+0000 7f8f2b400640 1 -- 172.21.15.72:0/311826332 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561db6bcabb0 msgr2=0x561db6bd6490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:58:51.754 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.752+0000 7f8f2b400640 1 --2- 172.21.15.72:0/311826332 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561db6bcabb0 0x561db6bd6490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:51.754 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.752+0000 7f8f2b400640 1 -- 172.21.15.72:0/311826332 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561db6ab71c0 con 0x561db6bc5050 2024-04-26T21:58:51.754 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.752+0000 7f8f2b400640 1 --2- 172.21.15.72:0/311826332 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561db6bc5050 0x561db6bcef10 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f8f2c105ac0 tx=0x7f8f1c02a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77b368dae8862355 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:58:51.754 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.752+0000 7f8f15600640 10 client.0 ms_handle_connect on v2:172.21.15.72:3300/0 2024-04-26T21:58:51.756 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.752+0000 7f8f15600640 1 -- 172.21.15.72:0/311826332 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f1c04d070 con 0x561db6bc5050 2024-04-26T21:58:51.756 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.753+0000 7f8f15600640 1 -- 172.21.15.72:0/311826332 <== mon.0 v2:172.21.15.72:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7f8f1c004480 con 0x561db6bc5050 2024-04-26T21:58:51.756 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.753+0000 7f8f35427740 1 -- 172.21.15.72:0/311826332 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f18011950 con 0x561db6bc5050 2024-04-26T21:58:51.756 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.753+0000 7f8f15600640 1 -- 172.21.15.72:0/311826332 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f8f1c059050 con 0x561db6bc5050 2024-04-26T21:58:51.756 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.753+0000 7f8f35427740 10 client.16317 fetch_fsmap learned FSMap version 27 2024-04-26T21:58:51.756 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.753+0000 7f8f35427740 1 -- 172.21.15.72:0/311826332 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x561db683a3f0 con 0x561db6bc5050 2024-04-26T21:58:51.756 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.753+0000 7f8f15600640 1 -- 172.21.15.72:0/311826332 <== mon.0 v2:172.21.15.72:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8f18011950 con 0x561db6bc5050 2024-04-26T21:58:51.756 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.753+0000 7f8f15600640 1 -- 172.21.15.72:0/311826332 <== mon.0 v2:172.21.15.72:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f8f1c03a040 con 0x561db6bc5050 2024-04-26T21:58:51.756 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.753+0000 7f8f35427740 10 client.16317 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T21:58:51.756 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.753+0000 7f8f35427740 10 client.16317 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-26T21:58:51.756 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.753+0000 7f8f35427740 20 client.16317 populate_metadata read hostname 'smithi072' 2024-04-26T21:58:51.756 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.753+0000 7f8f35427740 1 --2- 172.21.15.72:0/311826332 >> [v2:172.21.15.76:6832/3869483750,v1:172.21.15.76:6833/3869483750] conn(0x561db6bda9c0 0x561db6bdce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:51.756 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.754+0000 7f8f35427740 4 client.16317 mds_command: new command op to 34406 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:58:51.757 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.754+0000 7f8f35427740 1 -- 172.21.15.72:0/311826332 --> [v2:172.21.15.76:6832/3869483750,v1:172.21.15.76:6833/3869483750] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561db6b36880 con 0x561db6bda9c0 2024-04-26T21:58:51.757 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.754+0000 7f8f2aa00640 1 --2- 172.21.15.72:0/311826332 >> [v2:172.21.15.76:6832/3869483750,v1:172.21.15.76:6833/3869483750] conn(0x561db6bda9c0 0x561db6bdce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:51.757 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.755+0000 7f8f2aa00640 1 --2- 172.21.15.72:0/311826332 >> [v2:172.21.15.76:6832/3869483750,v1:172.21.15.76:6833/3869483750] conn(0x561db6bda9c0 0x561db6bdce60 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=569ff462e44cad12 server_cookie=2dc8d72d022aec45 in_seq=0 out_seq=0 2024-04-26T21:58:51.757 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.755+0000 7f8f15600640 10 client.16317 ms_handle_connect on v2:172.21.15.76:6832/3869483750 2024-04-26T21:58:51.758 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.757+0000 7f8f15600640 1 -- 172.21.15.72:0/311826332 <== mds.3 v2:172.21.15.76:6832/3869483750 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561db6b36880 con 0x561db6bda9c0 2024-04-26T21:58:51.758 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.757+0000 7f8f15600640 10 client.16317 handle_command_reply: tid=0 2024-04-26T21:58:51.758 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.757+0000 7f8f35427740 1 client.16317 shutdown 2024-04-26T21:58:51.758 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.757+0000 7f8f35427740 1 -- 172.21.15.72:0/311826332 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561db6bc5050 msgr2=0x561db6bcef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:51.758 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.757+0000 7f8f35427740 1 --2- 172.21.15.72:0/311826332 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561db6bc5050 0x561db6bcef10 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f8f2c105ac0 tx=0x7f8f1c02a410 comp rx=0 tx=0).stop 2024-04-26T21:58:51.758 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.757+0000 7f8f35427740 1 -- 172.21.15.72:0/311826332 shutdown_connections 2024-04-26T21:58:51.758 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.757+0000 7f8f35427740 1 --2- 172.21.15.72:0/311826332 >> [v2:172.21.15.76:6832/3869483750,v1:172.21.15.76:6833/3869483750] conn(0x561db6bda9c0 0x561db6bdce60 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:51.758 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.757+0000 7f8f35427740 1 --2- 172.21.15.72:0/311826332 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561db6bcabb0 0x561db6bd6490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:51.758 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.757+0000 7f8f35427740 1 --2- 172.21.15.72:0/311826332 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561db6bc5a40 0x561db6bcf450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:51.758 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.757+0000 7f8f15600640 0 client.16317 ms_handle_reset on v2:172.21.15.76:6832/3869483750 2024-04-26T21:58:51.759 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.757+0000 7f8f35427740 1 --2- 172.21.15.72:0/311826332 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561db6bc5050 0x561db6bcef10 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:51.759 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.757+0000 7f8f35427740 1 -- 172.21.15.72:0/311826332 >> 172.21.15.72:0/311826332 conn(0x561db6bc06c0 msgr2=0x561db6bbf7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:58:51.759 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.757+0000 7f8f35427740 1 -- 172.21.15.72:0/311826332 shutdown_connections 2024-04-26T21:58:51.759 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.757+0000 7f8f35427740 1 -- 172.21.15.72:0/311826332 wait complete. 2024-04-26T21:58:51.759 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.758+0000 7f8f35427740 20 client.16317 trim_cache size 0 max 16384 2024-04-26T21:58:51.772 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.772+0000 7f8f35427740 1 Processor -- start 2024-04-26T21:58:51.773 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.772+0000 7f8f35427740 1 -- start start 2024-04-26T21:58:51.773 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.772+0000 7f8f35427740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561db6bc5050 0x561db6bcf3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:51.773 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.772+0000 7f8f35427740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561db6bc5a40 0x561db6bd6450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:51.773 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.772+0000 7f8f35427740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561db6bcabb0 0x561db6bbf630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:51.773 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.772+0000 7f8f35427740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f8f2c1abe40 con 0x561db6bc5050 2024-04-26T21:58:51.773 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.772+0000 7f8f35427740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f8f2c1ad980 con 0x561db6bc5a40 2024-04-26T21:58:51.773 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.772+0000 7f8f35427740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f8f2c1ac510 con 0x561db6bcabb0 2024-04-26T21:58:51.773 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.773+0000 7f8f2b400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561db6bc5050 0x561db6bcf3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:51.773 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.773+0000 7f8f2b400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561db6bc5050 0x561db6bcf3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:57638/0 (socket says 172.21.15.72:57638) 2024-04-26T21:58:51.773 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.773+0000 7f8f2b400640 1 -- 172.21.15.72:0/2704342265 learned_addr learned my addr 172.21.15.72:0/2704342265 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:58:51.775 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.773+0000 7f8f2aa00640 1 --2- 172.21.15.72:0/2704342265 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561db6bc5a40 0x561db6bd6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:51.775 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.773+0000 7f8f2b400640 1 -- 172.21.15.72:0/2704342265 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561db6bcabb0 msgr2=0x561db6bbf630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:58:51.775 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.773+0000 7f8f2b400640 1 --2- 172.21.15.72:0/2704342265 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561db6bcabb0 0x561db6bbf630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:51.775 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.773+0000 7f8f2b400640 1 -- 172.21.15.72:0/2704342265 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561db6bc5a40 msgr2=0x561db6bd6450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:58:51.775 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.773+0000 7f8f2b400640 1 --2- 172.21.15.72:0/2704342265 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561db6bc5a40 0x561db6bd6450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:51.775 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.773+0000 7f8f2b400640 1 -- 172.21.15.72:0/2704342265 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561db69047d0 con 0x561db6bc5050 2024-04-26T21:58:51.775 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.773+0000 7f8f2b400640 1 --2- 172.21.15.72:0/2704342265 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561db6bc5050 0x561db6bcf3b0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f8f1c03d810 tx=0x7f8f1c032ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4dd33f7398c16e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:58:51.775 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.774+0000 7f8f17e00640 1 -- 172.21.15.72:0/2704342265 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f1c04d040 con 0x561db6bc5050 2024-04-26T21:58:51.775 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.774+0000 7f8f17e00640 1 -- 172.21.15.72:0/2704342265 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8f1c03d450 con 0x561db6bc5050 2024-04-26T21:58:51.775 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.774+0000 7f8f17e00640 1 -- 172.21.15.72:0/2704342265 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f1c035040 con 0x561db6bc5050 2024-04-26T21:58:51.775 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.774+0000 7f8f35427740 1 -- 172.21.15.72:0/2704342265 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561db6bc5050 msgr2=0x561db6bcf3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:51.776 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.774+0000 7f8f35427740 1 --2- 172.21.15.72:0/2704342265 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561db6bc5050 0x561db6bcf3b0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f8f1c03d810 tx=0x7f8f1c032ea0 comp rx=0 tx=0).stop 2024-04-26T21:58:51.776 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.774+0000 7f8f35427740 1 -- 172.21.15.72:0/2704342265 shutdown_connections 2024-04-26T21:58:51.776 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.774+0000 7f8f35427740 1 --2- 172.21.15.72:0/2704342265 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561db6bcabb0 0x561db6bbf630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:51.776 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.774+0000 7f8f35427740 1 --2- 172.21.15.72:0/2704342265 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561db6bc5a40 0x561db6bd6450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:51.776 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.774+0000 7f8f35427740 1 --2- 172.21.15.72:0/2704342265 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561db6bc5050 0x561db6bcf3b0 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:51.776 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.775+0000 7f8f35427740 1 -- 172.21.15.72:0/2704342265 >> 172.21.15.72:0/2704342265 conn(0x561db6bc06c0 msgr2=0x561db6bbef20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:58:51.776 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.775+0000 7f8f35427740 1 -- 172.21.15.72:0/2704342265 shutdown_connections 2024-04-26T21:58:51.776 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.775+0000 7f8f35427740 1 -- 172.21.15.72:0/2704342265 wait complete. 2024-04-26T21:58:51.776 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.775+0000 7f8f35427740 1 -- start start 2024-04-26T21:58:51.776 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.775+0000 7f8f35427740 1 Processor -- start 2024-04-26T21:58:51.779 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.776+0000 7f8f35427740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561db6bc5050 0x561db6bd38c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:51.779 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.776+0000 7f8f35427740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561db6bc5a40 0x561db6bd3e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:51.779 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.776+0000 7f8f35427740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561db6bcabb0 0x561db6bd4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:51.779 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.776+0000 7f8f35427740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f8f2c1ad2b0 con 0x561db6bcabb0 2024-04-26T21:58:51.779 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.776+0000 7f8f35427740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f8f2c1acbe0 con 0x561db6bc5a40 2024-04-26T21:58:51.779 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.776+0000 7f8f35427740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f8f2c1a9e20 con 0x561db6bc5050 2024-04-26T21:58:51.779 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.776+0000 7f8f35427740 10 client.0 _ll_register_callbacks cb 0x7f8f1c041070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:58:51.779 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.776+0000 7f8f2be00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561db6bcabb0 0x561db6bd4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:51.779 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.776+0000 7f8f2be00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561db6bcabb0 0x561db6bd4dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:57644/0 (socket says 172.21.15.72:57644) 2024-04-26T21:58:51.779 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.776+0000 7f8f2be00640 1 -- 172.21.15.72:0/2061069531 learned_addr learned my addr 172.21.15.72:0/2061069531 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:58:51.779 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.776+0000 7f8f2aa00640 1 --2- 172.21.15.72:0/2061069531 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561db6bc5a40 0x561db6bd3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:51.780 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.776+0000 7f8f2b400640 1 --2- 172.21.15.72:0/2061069531 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561db6bc5050 0x561db6bd38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:51.780 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.776+0000 7f8f2be00640 1 -- 172.21.15.72:0/2061069531 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561db6bc5050 msgr2=0x561db6bd38c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:58:51.780 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.776+0000 7f8f2be00640 1 --2- 172.21.15.72:0/2061069531 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561db6bc5050 0x561db6bd38c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:51.780 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.776+0000 7f8f2be00640 1 -- 172.21.15.72:0/2061069531 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561db6bc5a40 msgr2=0x561db6bd3e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:58:51.780 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.776+0000 7f8f2be00640 1 --2- 172.21.15.72:0/2061069531 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561db6bc5a40 0x561db6bd3e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:51.780 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.776+0000 7f8f2be00640 1 -- 172.21.15.72:0/2061069531 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561db6acf2a0 con 0x561db6bcabb0 2024-04-26T21:58:51.780 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.777+0000 7f8f2be00640 1 --2- 172.21.15.72:0/2061069531 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561db6bcabb0 0x561db6bd4dc0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f8f24099950 tx=0x7f8f2400f9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=faa90e22771d90b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:58:51.780 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.777+0000 7f8f15600640 10 client.0 ms_handle_connect on v2:172.21.15.72:3300/0 2024-04-26T21:58:51.780 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.777+0000 7f8f15600640 1 -- 172.21.15.72:0/2061069531 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f240b5070 con 0x561db6bcabb0 2024-04-26T21:58:51.780 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.777+0000 7f8f15600640 1 -- 172.21.15.72:0/2061069531 <== mon.0 v2:172.21.15.72:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7f8f2400fc30 con 0x561db6bcabb0 2024-04-26T21:58:51.780 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.777+0000 7f8f35427740 1 -- 172.21.15.72:0/2061069531 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f1c03d450 con 0x561db6bcabb0 2024-04-26T21:58:51.780 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.777+0000 7f8f15600640 1 -- 172.21.15.72:0/2061069531 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f8f240a1330 con 0x561db6bcabb0 2024-04-26T21:58:51.780 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.777+0000 7f8f35427740 10 client.16323 fetch_fsmap learned FSMap version 27 2024-04-26T21:58:51.780 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.777+0000 7f8f35427740 1 -- 172.21.15.72:0/2061069531 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x561db6bd6550 con 0x561db6bcabb0 2024-04-26T21:58:51.780 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.778+0000 7f8f15600640 1 -- 172.21.15.72:0/2061069531 <== mon.0 v2:172.21.15.72:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8f1c03d450 con 0x561db6bcabb0 2024-04-26T21:58:51.781 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.778+0000 7f8f15600640 1 -- 172.21.15.72:0/2061069531 <== mon.0 v2:172.21.15.72:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f8f240b6260 con 0x561db6bcabb0 2024-04-26T21:58:51.781 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.778+0000 7f8f35427740 10 client.16323 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T21:58:51.781 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.778+0000 7f8f35427740 10 client.16323 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-26T21:58:51.781 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.778+0000 7f8f35427740 20 client.16323 populate_metadata read hostname 'smithi072' 2024-04-26T21:58:51.781 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.778+0000 7f8f35427740 1 --2- 172.21.15.72:0/2061069531 >> [v2:172.21.15.76:6832/3869483750,v1:172.21.15.76:6833/3869483750] conn(0x561db6c415b0 0x561db6c439d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:51.781 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.778+0000 7f8f35427740 4 client.16323 mds_command: new command op to 34406 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-26T21:58:51.781 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.778+0000 7f8f35427740 1 -- 172.21.15.72:0/2061069531 --> [v2:172.21.15.76:6832/3869483750,v1:172.21.15.76:6833/3869483750] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561db6b306f0 con 0x561db6c415b0 2024-04-26T21:58:51.781 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.779+0000 7f8f2b400640 1 --2- 172.21.15.72:0/2061069531 >> [v2:172.21.15.76:6832/3869483750,v1:172.21.15.76:6833/3869483750] conn(0x561db6c415b0 0x561db6c439d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:51.782 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.779+0000 7f8f2b400640 1 --2- 172.21.15.72:0/2061069531 >> [v2:172.21.15.76:6832/3869483750,v1:172.21.15.76:6833/3869483750] conn(0x561db6c415b0 0x561db6c439d0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=88c591b522ae1528 server_cookie=b44ba25251efe97a in_seq=0 out_seq=0 2024-04-26T21:58:51.782 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.779+0000 7f8f15600640 10 client.16323 ms_handle_connect on v2:172.21.15.76:6832/3869483750 2024-04-26T21:58:51.782 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.780+0000 7f8f15600640 1 -- 172.21.15.72:0/2061069531 <== mds.3 v2:172.21.15.76:6832/3869483750 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561db6b306f0 con 0x561db6c415b0 2024-04-26T21:58:51.782 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.780+0000 7f8f15600640 10 client.16323 handle_command_reply: tid=0 2024-04-26T21:58:51.782 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.780+0000 7f8f35427740 1 client.16323 shutdown 2024-04-26T21:58:51.783 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.780+0000 7f8f35427740 1 -- 172.21.15.72:0/2061069531 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561db6bcabb0 msgr2=0x561db6bd4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:51.783 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.780+0000 7f8f35427740 1 --2- 172.21.15.72:0/2061069531 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561db6bcabb0 0x561db6bd4dc0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f8f24099950 tx=0x7f8f2400f9a0 comp rx=0 tx=0).stop 2024-04-26T21:58:51.783 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.780+0000 7f8f35427740 1 -- 172.21.15.72:0/2061069531 shutdown_connections 2024-04-26T21:58:51.783 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.780+0000 7f8f35427740 1 --2- 172.21.15.72:0/2061069531 >> [v2:172.21.15.76:6832/3869483750,v1:172.21.15.76:6833/3869483750] conn(0x561db6c415b0 0x561db6c439d0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:51.783 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.780+0000 7f8f35427740 1 --2- 172.21.15.72:0/2061069531 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561db6bcabb0 0x561db6bd4dc0 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:51.783 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.780+0000 7f8f35427740 1 --2- 172.21.15.72:0/2061069531 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561db6bc5a40 0x561db6bd3e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:51.783 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.780+0000 7f8f35427740 1 --2- 172.21.15.72:0/2061069531 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561db6bc5050 0x561db6bd38c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:51.783 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.780+0000 7f8f35427740 1 -- 172.21.15.72:0/2061069531 >> 172.21.15.72:0/2061069531 conn(0x561db6bc06c0 msgr2=0x561db6bbf7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:58:51.783 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.780+0000 7f8f15600640 0 client.16323 ms_handle_reset on v2:172.21.15.76:6832/3869483750 2024-04-26T21:58:51.783 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.780+0000 7f8f35427740 1 -- 172.21.15.72:0/2061069531 shutdown_connections 2024-04-26T21:58:51.783 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.780+0000 7f8f35427740 1 -- 172.21.15.72:0/2061069531 wait complete. 2024-04-26T21:58:51.783 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.780+0000 7f8f35427740 20 client.16323 trim_cache size 0 max 16384 2024-04-26T21:58:51.784 INFO:teuthology.orchestra.run.smithi072.stdout:[] 2024-04-26T21:58:51.784 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.781+0000 7f8f31000640 1 -- 172.21.15.72:0/3713050848 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f8f100a3c40 msgr2=0x7f8f100a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:51.784 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.781+0000 7f8f31000640 1 --2- 172.21.15.72:0/3713050848 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f8f100a3c40 0x7f8f100a6100 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f8f1c006010 tx=0x7f8f1c005a20 comp rx=0 tx=0).stop 2024-04-26T21:58:51.784 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.781+0000 7f8f31000640 1 -- 172.21.15.72:0/3713050848 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8f2c106620 msgr2=0x7f8f2c1a2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:51.784 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.781+0000 7f8f31000640 1 --2- 172.21.15.72:0/3713050848 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8f2c106620 0x7f8f2c1a2120 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f8f2400cce0 tx=0x7f8f2400a620 comp rx=0 tx=0).stop 2024-04-26T21:58:51.784 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.781+0000 7f8f31000640 1 -- 172.21.15.72:0/3713050848 shutdown_connections 2024-04-26T21:58:51.785 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.781+0000 7f8f31000640 1 --2- 172.21.15.72:0/3713050848 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8f2c106620 0x7f8f2c1a2120 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:51.785 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.781+0000 7f8f31000640 1 --2- 172.21.15.72:0/3713050848 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8f2c105c60 0x7f8f2c19b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:51.785 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.781+0000 7f8f31000640 1 --2- 172.21.15.72:0/3713050848 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f8f100a3c40 0x7f8f100a6100 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:51.785 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.781+0000 7f8f31000640 1 --2- 172.21.15.72:0/3713050848 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8f2c104a70 0x7f8f2c19abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:51.785 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.781+0000 7f8f31000640 1 -- 172.21.15.72:0/3713050848 >> 172.21.15.72:0/3713050848 conn(0x7f8f2c0fffe0 msgr2=0x7f8f2c1090d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:58:51.785 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.781+0000 7f8f31000640 1 -- 172.21.15.72:0/3713050848 shutdown_connections 2024-04-26T21:58:51.785 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.781+0000 7f8f31000640 1 -- 172.21.15.72:0/3713050848 wait complete. 2024-04-26T21:58:51.798 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-26T21:58:51.987 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.986+0000 7fa0be200640 1 -- 172.21.15.72:0/1328119443 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa0b8104530 msgr2=0x7fa0b810adc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:51.987 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.986+0000 7fa0be200640 1 --2- 172.21.15.72:0/1328119443 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa0b8104530 0x7fa0b810adc0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fa0b0009ca0 tx=0x7fa0b002a8c0 comp rx=0 tx=0).stop 2024-04-26T21:58:51.987 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.986+0000 7fa0be200640 1 -- 172.21.15.72:0/1328119443 shutdown_connections 2024-04-26T21:58:51.987 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.986+0000 7fa0be200640 1 --2- 172.21.15.72:0/1328119443 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa0b8104530 0x7fa0b810adc0 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:51.987 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.986+0000 7fa0be200640 1 --2- 172.21.15.72:0/1328119443 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa0b8103b70 0x7fa0b8103ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:51.987 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.986+0000 7fa0be200640 1 --2- 172.21.15.72:0/1328119443 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa0b8102980 0x7fa0b8102da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:51.987 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.986+0000 7fa0be200640 1 -- 172.21.15.72:0/1328119443 >> 172.21.15.72:0/1328119443 conn(0x7fa0b80fdf10 msgr2=0x7fa0b8100330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:58:51.987 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.986+0000 7fa0be200640 1 -- 172.21.15.72:0/1328119443 shutdown_connections 2024-04-26T21:58:51.987 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.986+0000 7fa0be200640 1 -- 172.21.15.72:0/1328119443 wait complete. 2024-04-26T21:58:51.989 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.987+0000 7fa0be200640 1 Processor -- start 2024-04-26T21:58:51.990 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.987+0000 7fa0be200640 1 -- start start 2024-04-26T21:58:51.990 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.988+0000 7fa0be200640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa0b8102980 0x7fa0b819acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:51.990 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.988+0000 7fa0be200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa0b8103b70 0x7fa0b819b200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:51.990 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.988+0000 7fa0be200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa0b8104530 0x7fa0b81a2260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:51.990 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.988+0000 7fa0be200640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fa0b810e3b0 con 0x7fa0b8102980 2024-04-26T21:58:51.990 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.988+0000 7fa0be200640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fa0b810e230 con 0x7fa0b8103b70 2024-04-26T21:58:51.990 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.988+0000 7fa0be200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa0b810e530 con 0x7fa0b8104530 2024-04-26T21:58:51.990 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.988+0000 7fa0b7400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa0b8102980 0x7fa0b819acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:51.990 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.988+0000 7fa0b7400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa0b8102980 0x7fa0b819acc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:57662/0 (socket says 172.21.15.72:57662) 2024-04-26T21:58:51.990 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.988+0000 7fa0b7400640 1 -- 172.21.15.72:0/2292936388 learned_addr learned my addr 172.21.15.72:0/2292936388 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:58:51.990 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.988+0000 7fa0b6a00640 1 --2- 172.21.15.72:0/2292936388 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa0b8103b70 0x7fa0b819b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:51.990 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.988+0000 7fa0b7e00640 1 --2- 172.21.15.72:0/2292936388 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa0b8104530 0x7fa0b81a2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:51.991 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.988+0000 7fa0b7400640 1 -- 172.21.15.72:0/2292936388 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa0b8104530 msgr2=0x7fa0b81a2260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:51.991 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.988+0000 7fa0b7400640 1 --2- 172.21.15.72:0/2292936388 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa0b8104530 0x7fa0b81a2260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:51.991 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.988+0000 7fa0b7400640 1 -- 172.21.15.72:0/2292936388 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa0b8103b70 msgr2=0x7fa0b819b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:51.991 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.988+0000 7fa0b7400640 1 --2- 172.21.15.72:0/2292936388 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa0b8103b70 0x7fa0b819b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:51.991 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.988+0000 7fa0b7400640 1 -- 172.21.15.72:0/2292936388 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0b81a2900 con 0x7fa0b8102980 2024-04-26T21:58:51.991 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.989+0000 7fa0b6a00640 1 --2- 172.21.15.72:0/2292936388 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa0b8103b70 0x7fa0b819b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:58:51.991 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.989+0000 7fa0b7e00640 1 --2- 172.21.15.72:0/2292936388 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa0b8104530 0x7fa0b81a2260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:58:51.991 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.989+0000 7fa0b7400640 1 --2- 172.21.15.72:0/2292936388 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa0b8102980 0x7fa0b819acc0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fa0a800e9c0 tx=0x7fa0a800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:58:51.991 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.989+0000 7fa0afe00640 1 -- 172.21.15.72:0/2292936388 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0a800cd80 con 0x7fa0b8102980 2024-04-26T21:58:51.991 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.989+0000 7fa0afe00640 1 -- 172.21.15.72:0/2292936388 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa0a80044b0 con 0x7fa0b8102980 2024-04-26T21:58:51.991 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.989+0000 7fa0be200640 1 -- 172.21.15.72:0/2292936388 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0b81a2bf0 con 0x7fa0b8102980 2024-04-26T21:58:51.991 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.989+0000 7fa0afe00640 1 -- 172.21.15.72:0/2292936388 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0a8010430 con 0x7fa0b8102980 2024-04-26T21:58:51.991 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.989+0000 7fa0be200640 1 -- 172.21.15.72:0/2292936388 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0b81a3140 con 0x7fa0b8102980 2024-04-26T21:58:51.993 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.993+0000 7fa0c1036740 1 Processor -- start 2024-04-26T21:58:51.993 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.993+0000 7fa0c1036740 1 -- start start 2024-04-26T21:58:51.993 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.993+0000 7fa0afe00640 1 -- 172.21.15.72:0/2292936388 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fa0a8003f00 con 0x7fa0b8102980 2024-04-26T21:58:51.993 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.993+0000 7fa0c1036740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bf79424ea0 0x55bf794252c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:51.993 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.993+0000 7fa0c1036740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bf79425890 0x55bf7942a390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:51.993 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.993+0000 7fa0c1036740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55bf7942aa00 0x55bf7942ce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:51.993 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.993+0000 7fa0c1036740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55bf79388c70 con 0x55bf7942aa00 2024-04-26T21:58:51.993 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.993+0000 7fa0c1036740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55bf79341980 con 0x55bf79424ea0 2024-04-26T21:58:51.993 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.993+0000 7fa0c1036740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55bf79384c30 con 0x55bf79425890 2024-04-26T21:58:51.995 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.993+0000 7fa0b6a00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55bf7942aa00 0x55bf7942ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:51.995 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.993+0000 7fa0b6a00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55bf7942aa00 0x55bf7942ce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:57672/0 (socket says 172.21.15.72:57672) 2024-04-26T21:58:51.995 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.993+0000 7fa0b6a00640 1 -- 172.21.15.72:0/2320406166 learned_addr learned my addr 172.21.15.72:0/2320406166 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:58:51.995 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.994+0000 7fa0b7e00640 1 --2- 172.21.15.72:0/2320406166 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bf79424ea0 0x55bf794252c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:51.995 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.994+0000 7fa0b7400640 1 --2- 172.21.15.72:0/2320406166 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bf79425890 0x55bf7942a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:51.995 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.994+0000 7fa0b6a00640 1 -- 172.21.15.72:0/2320406166 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bf79425890 msgr2=0x55bf7942a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:58:51.995 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.994+0000 7fa0b6a00640 1 --2- 172.21.15.72:0/2320406166 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bf79425890 0x55bf7942a390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:51.995 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.994+0000 7fa0b6a00640 1 -- 172.21.15.72:0/2320406166 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bf79424ea0 msgr2=0x55bf794252c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:58:51.996 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.994+0000 7fa0b6a00640 1 --2- 172.21.15.72:0/2320406166 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bf79424ea0 0x55bf794252c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:51.996 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.994+0000 7fa0b6a00640 1 -- 172.21.15.72:0/2320406166 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf793197b0 con 0x55bf7942aa00 2024-04-26T21:58:51.996 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.994+0000 7fa0afe00640 1 --2- 172.21.15.72:0/2292936388 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fa09c0a3c40 0x7fa09c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:51.996 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.994+0000 7fa0b6a00640 1 --2- 172.21.15.72:0/2320406166 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55bf7942aa00 0x55bf7942ce90 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fa0a4009c20 tx=0x7fa0a400e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7474f46002dd8373 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:58:51.996 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.995+0000 7fa0afe00640 1 -- 172.21.15.72:0/2292936388 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7fa0a80d0b70 con 0x7fa0b8102980 2024-04-26T21:58:51.996 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.995+0000 7fa0b7e00640 1 --2- 172.21.15.72:0/2292936388 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fa09c0a3c40 0x7fa09c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:51.996 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.995+0000 7fa0a3e00640 1 -- 172.21.15.72:0/2320406166 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0a4016070 con 0x55bf7942aa00 2024-04-26T21:58:51.996 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.995+0000 7fa0a3e00640 1 -- 172.21.15.72:0/2320406166 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa0a4011900 con 0x55bf7942aa00 2024-04-26T21:58:51.996 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.995+0000 7fa0a3e00640 1 -- 172.21.15.72:0/2320406166 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0a4010730 con 0x55bf7942aa00 2024-04-26T21:58:51.999 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.996+0000 7fa0c1036740 1 -- 172.21.15.72:0/2320406166 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55bf7942aa00 msgr2=0x55bf7942ce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:51.999 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.996+0000 7fa0b7e00640 1 --2- 172.21.15.72:0/2292936388 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fa09c0a3c40 0x7fa09c0a6100 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fa0b0004af0 tx=0x7fa0b0005d80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:58:51.999 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.996+0000 7fa0c1036740 1 --2- 172.21.15.72:0/2320406166 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55bf7942aa00 0x55bf7942ce90 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fa0a4009c20 tx=0x7fa0a400e6c0 comp rx=0 tx=0).stop 2024-04-26T21:58:51.999 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.996+0000 7fa0c1036740 1 -- 172.21.15.72:0/2320406166 shutdown_connections 2024-04-26T21:58:51.999 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.996+0000 7fa0c1036740 1 --2- 172.21.15.72:0/2320406166 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55bf7942aa00 0x55bf7942ce90 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:51.999 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.996+0000 7fa0c1036740 1 --2- 172.21.15.72:0/2320406166 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bf79425890 0x55bf7942a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:51.999 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.996+0000 7fa0c1036740 1 --2- 172.21.15.72:0/2320406166 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bf79424ea0 0x55bf794252c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:51.999 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.996+0000 7fa0c1036740 1 -- 172.21.15.72:0/2320406166 >> 172.21.15.72:0/2320406166 conn(0x55bf79420510 msgr2=0x55bf79420910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:58:51.999 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.996+0000 7fa0c1036740 1 -- 172.21.15.72:0/2320406166 shutdown_connections 2024-04-26T21:58:51.999 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.996+0000 7fa0c1036740 1 -- 172.21.15.72:0/2320406166 wait complete. 2024-04-26T21:58:51.999 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.997+0000 7fa0c1036740 1 -- start start 2024-04-26T21:58:51.999 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.997+0000 7fa0c1036740 1 Processor -- start 2024-04-26T21:58:52.000 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.997+0000 7fa0c1036740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55bf79424ea0 0x55bf7942edb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:52.000 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.997+0000 7fa0c1036740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bf79425890 0x55bf7942f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:52.000 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.997+0000 7fa0c1036740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bf7942aa00 0x55bf79436330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:52.000 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.997+0000 7fa0c1036740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55bf79387f80 con 0x55bf79424ea0 2024-04-26T21:58:52.000 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.998+0000 7fa0c1036740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55bf79335260 con 0x55bf7942aa00 2024-04-26T21:58:52.000 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.998+0000 7fa0c1036740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55bf7938d2a0 con 0x55bf79425890 2024-04-26T21:58:52.000 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.998+0000 7fa0c1036740 10 client.0 _ll_register_callbacks cb 0x55bf7937f960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:58:52.000 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.998+0000 7fa0b7400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55bf79424ea0 0x55bf7942edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:52.000 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.998+0000 7fa0b7400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55bf79424ea0 0x55bf7942edb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:57688/0 (socket says 172.21.15.72:57688) 2024-04-26T21:58:52.000 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.998+0000 7fa0b7400640 1 -- 172.21.15.72:0/2421757111 learned_addr learned my addr 172.21.15.72:0/2421757111 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:58:52.000 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.998+0000 7fa0b7e00640 1 --2- 172.21.15.72:0/2421757111 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bf7942aa00 0x55bf79436330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:52.000 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.998+0000 7fa0b6a00640 1 --2- 172.21.15.72:0/2421757111 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bf79425890 0x55bf7942f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:52.000 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.998+0000 7fa0b7400640 1 -- 172.21.15.72:0/2421757111 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bf79425890 msgr2=0x55bf7942f2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:58:52.000 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.998+0000 7fa0b7400640 1 --2- 172.21.15.72:0/2421757111 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bf79425890 0x55bf7942f2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:52.001 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.998+0000 7fa0b7400640 1 -- 172.21.15.72:0/2421757111 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bf7942aa00 msgr2=0x55bf79436330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:58:52.001 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.998+0000 7fa0b7400640 1 --2- 172.21.15.72:0/2421757111 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bf7942aa00 0x55bf79436330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:52.001 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.998+0000 7fa0b7400640 1 -- 172.21.15.72:0/2421757111 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf793171c0 con 0x55bf79424ea0 2024-04-26T21:58:52.001 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.998+0000 7fa0b6a00640 1 --2- 172.21.15.72:0/2421757111 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bf79425890 0x55bf7942f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:58:52.001 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.999+0000 7fa0b7400640 1 --2- 172.21.15.72:0/2421757111 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55bf79424ea0 0x55bf7942edb0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fa0a80ce470 tx=0x7fa0a809c570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74b2f88a50ec777c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:58:52.001 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:51.999+0000 7fa0a0c00640 10 client.0 ms_handle_connect on v2:172.21.15.72:3300/0 2024-04-26T21:58:52.003 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.002+0000 7fa0a0c00640 1 -- 172.21.15.72:0/2421757111 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0a80a0fc0 con 0x55bf79424ea0 2024-04-26T21:58:52.003 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.002+0000 7fa0a0c00640 1 -- 172.21.15.72:0/2421757111 <== mon.0 v2:172.21.15.72:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7fa0a801fdc0 con 0x55bf79424ea0 2024-04-26T21:58:52.003 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.002+0000 7fa0c1036740 1 -- 172.21.15.72:0/2421757111 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa0a4011900 con 0x55bf79424ea0 2024-04-26T21:58:52.003 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.002+0000 7fa0a0c00640 1 -- 172.21.15.72:0/2421757111 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa0a80a5880 con 0x55bf79424ea0 2024-04-26T21:58:52.003 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.002+0000 7fa0c1036740 10 client.16335 fetch_fsmap learned FSMap version 27 2024-04-26T21:58:52.003 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.002+0000 7fa0c1036740 1 -- 172.21.15.72:0/2421757111 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55bf7909a3f0 con 0x55bf79424ea0 2024-04-26T21:58:52.003 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.003+0000 7fa0a0c00640 1 -- 172.21.15.72:0/2421757111 <== mon.0 v2:172.21.15.72:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa0a4011900 con 0x55bf79424ea0 2024-04-26T21:58:52.003 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.003+0000 7fa0a0c00640 1 -- 172.21.15.72:0/2421757111 <== mon.0 v2:172.21.15.72:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fa0a80a53f0 con 0x55bf79424ea0 2024-04-26T21:58:52.003 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.003+0000 7fa0c1036740 10 client.16335 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T21:58:52.003 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.003+0000 7fa0c1036740 10 client.16335 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-26T21:58:52.003 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.003+0000 7fa0c1036740 20 client.16335 populate_metadata read hostname 'smithi072' 2024-04-26T21:58:52.003 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.003+0000 7fa0c1036740 1 --2- 172.21.15.72:0/2421757111 >> [v2:172.21.15.76:6834/3517429435,v1:172.21.15.76:6835/3517429435] conn(0x55bf7943a890 0x55bf7943cd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:52.003 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.003+0000 7fa0c1036740 4 client.16335 mds_command: new command op to 34442 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:58:52.004 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.003+0000 7fa0c1036740 1 -- 172.21.15.72:0/2421757111 --> [v2:172.21.15.76:6834/3517429435,v1:172.21.15.76:6835/3517429435] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bf79396880 con 0x55bf7943a890 2024-04-26T21:58:52.004 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.003+0000 7fa0b6a00640 1 --2- 172.21.15.72:0/2421757111 >> [v2:172.21.15.76:6834/3517429435,v1:172.21.15.76:6835/3517429435] conn(0x55bf7943a890 0x55bf7943cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:52.004 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.004+0000 7fa0b6a00640 1 --2- 172.21.15.72:0/2421757111 >> [v2:172.21.15.76:6834/3517429435,v1:172.21.15.76:6835/3517429435] conn(0x55bf7943a890 0x55bf7943cd30 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=85bda93cb66f35b7 server_cookie=47dc183ae606825d in_seq=0 out_seq=0 2024-04-26T21:58:52.004 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.004+0000 7fa0a0c00640 10 client.16335 ms_handle_connect on v2:172.21.15.76:6834/3517429435 2024-04-26T21:58:52.006 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.005+0000 7fa0a0c00640 1 -- 172.21.15.72:0/2421757111 <== mds.2 v2:172.21.15.76:6834/3517429435 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55bf79396880 con 0x55bf7943a890 2024-04-26T21:58:52.006 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.005+0000 7fa0a0c00640 10 client.16335 handle_command_reply: tid=0 2024-04-26T21:58:52.006 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.005+0000 7fa0c1036740 1 client.16335 shutdown 2024-04-26T21:58:52.006 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.006+0000 7fa0c1036740 1 -- 172.21.15.72:0/2421757111 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55bf79424ea0 msgr2=0x55bf7942edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:52.006 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.006+0000 7fa0c1036740 1 --2- 172.21.15.72:0/2421757111 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55bf79424ea0 0x55bf7942edb0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fa0a80ce470 tx=0x7fa0a809c570 comp rx=0 tx=0).stop 2024-04-26T21:58:52.006 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.006+0000 7fa0c1036740 1 -- 172.21.15.72:0/2421757111 shutdown_connections 2024-04-26T21:58:52.006 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.006+0000 7fa0c1036740 1 --2- 172.21.15.72:0/2421757111 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bf7942aa00 0x55bf79436330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:52.006 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.006+0000 7fa0c1036740 1 --2- 172.21.15.72:0/2421757111 >> [v2:172.21.15.76:6834/3517429435,v1:172.21.15.76:6835/3517429435] conn(0x55bf7943a890 0x55bf7943cd30 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:52.006 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.006+0000 7fa0c1036740 1 --2- 172.21.15.72:0/2421757111 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bf79425890 0x55bf7942f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:52.006 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.006+0000 7fa0c1036740 1 --2- 172.21.15.72:0/2421757111 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55bf79424ea0 0x55bf7942edb0 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:52.006 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.006+0000 7fa0c1036740 1 -- 172.21.15.72:0/2421757111 >> 172.21.15.72:0/2421757111 conn(0x55bf79420510 msgr2=0x55bf7941f690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:58:52.007 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.006+0000 7fa0a0c00640 0 client.16335 ms_handle_reset on v2:172.21.15.76:6834/3517429435 2024-04-26T21:58:52.007 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.006+0000 7fa0c1036740 1 -- 172.21.15.72:0/2421757111 shutdown_connections 2024-04-26T21:58:52.007 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.006+0000 7fa0c1036740 1 -- 172.21.15.72:0/2421757111 wait complete. 2024-04-26T21:58:52.007 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.006+0000 7fa0c1036740 20 client.16335 trim_cache size 0 max 16384 2024-04-26T21:58:52.022 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.021+0000 7fa0c1036740 1 Processor -- start 2024-04-26T21:58:52.022 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.021+0000 7fa0c1036740 1 -- start start 2024-04-26T21:58:52.022 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.021+0000 7fa0c1036740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55bf79424ea0 0x55bf7942f250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:52.022 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.021+0000 7fa0c1036740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bf79425890 0x55bf794362f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:52.022 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.021+0000 7fa0c1036740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bf7942aa00 0x55bf7941f510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:52.023 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.021+0000 7fa0c1036740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fa0b81ac0d0 con 0x55bf79424ea0 2024-04-26T21:58:52.023 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.021+0000 7fa0c1036740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fa0b81adc10 con 0x55bf7942aa00 2024-04-26T21:58:52.023 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.021+0000 7fa0c1036740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa0b81ac7a0 con 0x55bf79425890 2024-04-26T21:58:52.023 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.022+0000 7fa0b7400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55bf79424ea0 0x55bf7942f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:52.023 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.022+0000 7fa0b7400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55bf79424ea0 0x55bf7942f250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:57704/0 (socket says 172.21.15.72:57704) 2024-04-26T21:58:52.023 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.022+0000 7fa0b7400640 1 -- 172.21.15.72:0/4012469874 learned_addr learned my addr 172.21.15.72:0/4012469874 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:58:52.023 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.022+0000 7fa0b6a00640 1 --2- 172.21.15.72:0/4012469874 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bf79425890 0x55bf794362f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:52.023 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.022+0000 7fa0b7e00640 1 --2- 172.21.15.72:0/4012469874 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bf7942aa00 0x55bf7941f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:52.024 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.022+0000 7fa0b7400640 1 -- 172.21.15.72:0/4012469874 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bf79425890 msgr2=0x55bf794362f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:58:52.024 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.022+0000 7fa0b7400640 1 --2- 172.21.15.72:0/4012469874 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bf79425890 0x55bf794362f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:52.024 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.022+0000 7fa0b7400640 1 -- 172.21.15.72:0/4012469874 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bf7942aa00 msgr2=0x55bf7941f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:58:52.024 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.022+0000 7fa0b7400640 1 --2- 172.21.15.72:0/4012469874 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bf7942aa00 0x55bf7941f510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:52.024 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.022+0000 7fa0b7400640 1 -- 172.21.15.72:0/4012469874 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf791647d0 con 0x55bf79424ea0 2024-04-26T21:58:52.024 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.022+0000 7fa0b7e00640 1 --2- 172.21.15.72:0/4012469874 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bf7942aa00 0x55bf7941f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:58:52.028 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.022+0000 7fa0b7400640 1 --2- 172.21.15.72:0/4012469874 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55bf79424ea0 0x55bf7942f250 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fa0a80b3040 tx=0x7fa0a809cd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7a9a12b248546c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:58:52.028 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.023+0000 7fa0a3400640 1 -- 172.21.15.72:0/4012469874 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0a809c9e0 con 0x55bf79424ea0 2024-04-26T21:58:52.028 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.023+0000 7fa0a3400640 1 -- 172.21.15.72:0/4012469874 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa0a809d310 con 0x55bf79424ea0 2024-04-26T21:58:52.028 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.023+0000 7fa0a3400640 1 -- 172.21.15.72:0/4012469874 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0a80041f0 con 0x55bf79424ea0 2024-04-26T21:58:52.028 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.023+0000 7fa0c1036740 1 -- 172.21.15.72:0/4012469874 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55bf79424ea0 msgr2=0x55bf7942f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:52.028 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.023+0000 7fa0c1036740 1 --2- 172.21.15.72:0/4012469874 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55bf79424ea0 0x55bf7942f250 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fa0a80b3040 tx=0x7fa0a809cd90 comp rx=0 tx=0).stop 2024-04-26T21:58:52.028 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.024+0000 7fa0c1036740 1 -- 172.21.15.72:0/4012469874 shutdown_connections 2024-04-26T21:58:52.028 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.024+0000 7fa0c1036740 1 --2- 172.21.15.72:0/4012469874 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bf7942aa00 0x55bf7941f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:52.028 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.024+0000 7fa0c1036740 1 --2- 172.21.15.72:0/4012469874 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bf79425890 0x55bf794362f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:52.029 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.024+0000 7fa0c1036740 1 --2- 172.21.15.72:0/4012469874 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55bf79424ea0 0x55bf7942f250 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:52.029 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.024+0000 7fa0c1036740 1 -- 172.21.15.72:0/4012469874 >> 172.21.15.72:0/4012469874 conn(0x55bf79420510 msgr2=0x55bf7941ee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:58:52.029 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.024+0000 7fa0c1036740 1 -- 172.21.15.72:0/4012469874 shutdown_connections 2024-04-26T21:58:52.029 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.024+0000 7fa0c1036740 1 -- 172.21.15.72:0/4012469874 wait complete. 2024-04-26T21:58:52.029 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.024+0000 7fa0c1036740 1 -- start start 2024-04-26T21:58:52.029 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.024+0000 7fa0c1036740 1 Processor -- start 2024-04-26T21:58:52.029 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.025+0000 7fa0c1036740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55bf79424ea0 0x55bf79433380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:52.029 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.025+0000 7fa0c1036740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bf79425890 0x55bf794338c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:52.029 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.025+0000 7fa0c1036740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bf7942aa00 0x55bf7949ee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:52.029 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.025+0000 7fa0c1036740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fa0b81ad540 con 0x55bf79424ea0 2024-04-26T21:58:52.029 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.025+0000 7fa0c1036740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fa0b81ace70 con 0x55bf79425890 2024-04-26T21:58:52.029 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.025+0000 7fa0c1036740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa0b81aa0b0 con 0x55bf7942aa00 2024-04-26T21:58:52.029 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.025+0000 7fa0c1036740 10 client.0 _ll_register_callbacks cb 0x7fa0a80d4150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:58:52.029 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.025+0000 7fa0b7400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55bf79424ea0 0x55bf79433380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:52.030 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.025+0000 7fa0b7400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55bf79424ea0 0x55bf79433380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:57706/0 (socket says 172.21.15.72:57706) 2024-04-26T21:58:52.030 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.025+0000 7fa0b7400640 1 -- 172.21.15.72:0/3977030676 learned_addr learned my addr 172.21.15.72:0/3977030676 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:58:52.030 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.025+0000 7fa0b6a00640 1 --2- 172.21.15.72:0/3977030676 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bf79425890 0x55bf794338c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:52.030 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.025+0000 7fa0b7400640 1 -- 172.21.15.72:0/3977030676 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bf7942aa00 msgr2=0x55bf7949ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:58:52.030 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.025+0000 7fa0b7e00640 1 --2- 172.21.15.72:0/3977030676 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bf7942aa00 0x55bf7949ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:52.030 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.025+0000 7fa0b7400640 1 --2- 172.21.15.72:0/3977030676 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bf7942aa00 0x55bf7949ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:52.030 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.025+0000 7fa0b7400640 1 -- 172.21.15.72:0/3977030676 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bf79425890 msgr2=0x55bf794338c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:58:52.030 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.025+0000 7fa0b7400640 1 --2- 172.21.15.72:0/3977030676 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bf79425890 0x55bf794338c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:52.030 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.025+0000 7fa0b7400640 1 -- 172.21.15.72:0/3977030676 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf7932f2a0 con 0x55bf79424ea0 2024-04-26T21:58:52.030 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.026+0000 7fa0b7e00640 1 --2- 172.21.15.72:0/3977030676 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bf7942aa00 0x55bf7949ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:58:52.030 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.026+0000 7fa0b6a00640 1 --2- 172.21.15.72:0/3977030676 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bf79425890 0x55bf794338c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:58:52.030 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.026+0000 7fa0b7400640 1 --2- 172.21.15.72:0/3977030676 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55bf79424ea0 0x55bf79433380 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fa0a809cba0 tx=0x7fa0a809b120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bec301376e009168 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:58:52.030 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.026+0000 7fa0a0c00640 10 client.0 ms_handle_connect on v2:172.21.15.72:3300/0 2024-04-26T21:58:52.031 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.026+0000 7fa0a0c00640 1 -- 172.21.15.72:0/3977030676 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0a809c9e0 con 0x55bf79424ea0 2024-04-26T21:58:52.031 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.026+0000 7fa0a0c00640 1 -- 172.21.15.72:0/3977030676 <== mon.0 v2:172.21.15.72:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7fa0a8061be0 con 0x55bf79424ea0 2024-04-26T21:58:52.031 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.026+0000 7fa0c1036740 1 -- 172.21.15.72:0/3977030676 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa0a809d310 con 0x55bf79424ea0 2024-04-26T21:58:52.031 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.026+0000 7fa0a0c00640 1 -- 172.21.15.72:0/3977030676 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa0a80036f0 con 0x55bf79424ea0 2024-04-26T21:58:52.031 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.026+0000 7fa0c1036740 10 client.16341 fetch_fsmap learned FSMap version 27 2024-04-26T21:58:52.031 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.026+0000 7fa0c1036740 1 -- 172.21.15.72:0/3977030676 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55bf7932f2a0 con 0x55bf79424ea0 2024-04-26T21:58:52.031 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.027+0000 7fa0a0c00640 1 -- 172.21.15.72:0/3977030676 <== mon.0 v2:172.21.15.72:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa0a809d310 con 0x55bf79424ea0 2024-04-26T21:58:52.031 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.027+0000 7fa0a0c00640 1 -- 172.21.15.72:0/3977030676 <== mon.0 v2:172.21.15.72:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fa0a80a22e0 con 0x55bf79424ea0 2024-04-26T21:58:52.031 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.027+0000 7fa0c1036740 10 client.16341 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T21:58:52.031 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.027+0000 7fa0c1036740 10 client.16341 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-26T21:58:52.031 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.027+0000 7fa0c1036740 20 client.16341 populate_metadata read hostname 'smithi072' 2024-04-26T21:58:52.031 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.027+0000 7fa0c1036740 1 --2- 172.21.15.72:0/3977030676 >> [v2:172.21.15.76:6834/3517429435,v1:172.21.15.76:6835/3517429435] conn(0x55bf794a0a20 0x55bf794a2e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:52.031 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.027+0000 7fa0c1036740 4 client.16341 mds_command: new command op to 34442 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-26T21:58:52.031 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.027+0000 7fa0c1036740 1 -- 172.21.15.72:0/3977030676 --> [v2:172.21.15.76:6834/3517429435,v1:172.21.15.76:6835/3517429435] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bf793906f0 con 0x55bf794a0a20 2024-04-26T21:58:52.032 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.027+0000 7fa0b6a00640 1 --2- 172.21.15.72:0/3977030676 >> [v2:172.21.15.76:6834/3517429435,v1:172.21.15.76:6835/3517429435] conn(0x55bf794a0a20 0x55bf794a2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:52.032 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.028+0000 7fa0b6a00640 1 --2- 172.21.15.72:0/3977030676 >> [v2:172.21.15.76:6834/3517429435,v1:172.21.15.76:6835/3517429435] conn(0x55bf794a0a20 0x55bf794a2e40 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d315607b74907f35 server_cookie=b3c0b79213a78bc6 in_seq=0 out_seq=0 2024-04-26T21:58:52.032 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.028+0000 7fa0a0c00640 10 client.16341 ms_handle_connect on v2:172.21.15.76:6834/3517429435 2024-04-26T21:58:52.033 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.029+0000 7fa0a0c00640 1 -- 172.21.15.72:0/3977030676 <== mds.2 v2:172.21.15.76:6834/3517429435 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bf793906f0 con 0x55bf794a0a20 2024-04-26T21:58:52.033 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.029+0000 7fa0a0c00640 10 client.16341 handle_command_reply: tid=0 2024-04-26T21:58:52.033 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.029+0000 7fa0c1036740 1 client.16341 shutdown 2024-04-26T21:58:52.033 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.029+0000 7fa0c1036740 1 -- 172.21.15.72:0/3977030676 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55bf79424ea0 msgr2=0x55bf79433380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:52.033 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.029+0000 7fa0c1036740 1 --2- 172.21.15.72:0/3977030676 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55bf79424ea0 0x55bf79433380 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fa0a809cba0 tx=0x7fa0a809b120 comp rx=0 tx=0).stop 2024-04-26T21:58:52.033 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.029+0000 7fa0c1036740 1 -- 172.21.15.72:0/3977030676 shutdown_connections 2024-04-26T21:58:52.033 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.029+0000 7fa0c1036740 1 --2- 172.21.15.72:0/3977030676 >> [v2:172.21.15.76:6834/3517429435,v1:172.21.15.76:6835/3517429435] conn(0x55bf794a0a20 0x55bf794a2e40 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:52.033 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.029+0000 7fa0c1036740 1 --2- 172.21.15.72:0/3977030676 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bf7942aa00 0x55bf7949ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:52.033 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.029+0000 7fa0c1036740 1 --2- 172.21.15.72:0/3977030676 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bf79425890 0x55bf794338c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:52.033 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.029+0000 7fa0a0c00640 0 client.16341 ms_handle_reset on v2:172.21.15.76:6834/3517429435 2024-04-26T21:58:52.033 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.029+0000 7fa0c1036740 1 --2- 172.21.15.72:0/3977030676 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55bf79424ea0 0x55bf79433380 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:52.034 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.029+0000 7fa0c1036740 1 -- 172.21.15.72:0/3977030676 >> 172.21.15.72:0/3977030676 conn(0x55bf79420510 msgr2=0x55bf79423cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:58:52.034 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.030+0000 7fa0c1036740 1 -- 172.21.15.72:0/3977030676 shutdown_connections 2024-04-26T21:58:52.034 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.030+0000 7fa0c1036740 1 -- 172.21.15.72:0/3977030676 wait complete. 2024-04-26T21:58:52.034 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.030+0000 7fa0c1036740 20 client.16341 trim_cache size 0 max 16384 2024-04-26T21:58:52.034 INFO:teuthology.orchestra.run.smithi072.stdout:[] 2024-04-26T21:58:52.035 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.030+0000 7fa0be200640 1 -- 172.21.15.72:0/2292936388 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fa09c0a3c40 msgr2=0x7fa09c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:52.035 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.030+0000 7fa0be200640 1 --2- 172.21.15.72:0/2292936388 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fa09c0a3c40 0x7fa09c0a6100 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fa0b0004af0 tx=0x7fa0b0005d80 comp rx=0 tx=0).stop 2024-04-26T21:58:52.035 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.030+0000 7fa0be200640 1 -- 172.21.15.72:0/2292936388 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa0b8102980 msgr2=0x7fa0b819acc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:52.035 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.030+0000 7fa0be200640 1 --2- 172.21.15.72:0/2292936388 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa0b8102980 0x7fa0b819acc0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fa0a800e9c0 tx=0x7fa0a800ee80 comp rx=0 tx=0).stop 2024-04-26T21:58:52.035 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.030+0000 7fa0be200640 1 -- 172.21.15.72:0/2292936388 shutdown_connections 2024-04-26T21:58:52.035 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.030+0000 7fa0be200640 1 --2- 172.21.15.72:0/2292936388 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fa09c0a3c40 0x7fa09c0a6100 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:52.035 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.030+0000 7fa0be200640 1 --2- 172.21.15.72:0/2292936388 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa0b8104530 0x7fa0b81a2260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:52.035 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.030+0000 7fa0be200640 1 --2- 172.21.15.72:0/2292936388 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa0b8103b70 0x7fa0b819b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:52.035 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.030+0000 7fa0be200640 1 --2- 172.21.15.72:0/2292936388 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa0b8102980 0x7fa0b819acc0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:52.035 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.030+0000 7fa0be200640 1 -- 172.21.15.72:0/2292936388 >> 172.21.15.72:0/2292936388 conn(0x7fa0b80fdf10 msgr2=0x7fa0b8074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:58:52.035 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.031+0000 7fa0be200640 1 -- 172.21.15.72:0/2292936388 shutdown_connections 2024-04-26T21:58:52.035 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.031+0000 7fa0be200640 1 -- 172.21.15.72:0/2292936388 wait complete. 2024-04-26T21:58:52.049 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-26T21:58:52.105 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:58:51 smithi072 ceph-mon[28138]: from='client.? 172.21.15.72:0/4079986962' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:58:52.231 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.230+0000 7f9186e00640 1 -- 172.21.15.72:0/3781333526 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9180105d60 msgr2=0x7f91801061e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:52.231 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.230+0000 7f9186e00640 1 --2- 172.21.15.72:0/3781333526 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9180105d60 0x7f91801061e0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f916c009900 tx=0x7f916c02a3f0 comp rx=0 tx=0).stop 2024-04-26T21:58:52.231 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.231+0000 7f9186e00640 1 -- 172.21.15.72:0/3781333526 shutdown_connections 2024-04-26T21:58:52.231 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.231+0000 7f9186e00640 1 --2- 172.21.15.72:0/3781333526 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9180106720 0x7f918010cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:52.231 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.231+0000 7f9186e00640 1 --2- 172.21.15.72:0/3781333526 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9180105d60 0x7f91801061e0 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:52.231 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.231+0000 7f9186e00640 1 --2- 172.21.15.72:0/3781333526 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9180104b70 0x7f9180104f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:52.231 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.231+0000 7f9186e00640 1 -- 172.21.15.72:0/3781333526 >> 172.21.15.72:0/3781333526 conn(0x7f91801000e0 msgr2=0x7f9180102520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:58:52.232 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.231+0000 7f9186e00640 1 -- 172.21.15.72:0/3781333526 shutdown_connections 2024-04-26T21:58:52.232 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.231+0000 7f9186e00640 1 -- 172.21.15.72:0/3781333526 wait complete. 2024-04-26T21:58:52.234 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.232+0000 7f9186e00640 1 Processor -- start 2024-04-26T21:58:52.234 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.232+0000 7f9186e00640 1 -- start start 2024-04-26T21:58:52.234 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.232+0000 7f9186e00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9180104b70 0x7f918019ad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:52.234 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.232+0000 7f9186e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9180105d60 0x7f918019b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:52.234 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.232+0000 7f9186e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9180106720 0x7f91801a22b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:52.234 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.232+0000 7f9186e00640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f9180110530 con 0x7f9180106720 2024-04-26T21:58:52.234 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.232+0000 7f9186e00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f91801103b0 con 0x7f9180104b70 2024-04-26T21:58:52.234 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.232+0000 7f9186e00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f91801106b0 con 0x7f9180105d60 2024-04-26T21:58:52.234 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.232+0000 7f9184e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9180106720 0x7f91801a22b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:52.235 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.232+0000 7f9184e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9180106720 0x7f91801a22b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:57734/0 (socket says 172.21.15.72:57734) 2024-04-26T21:58:52.235 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.232+0000 7f9184e00640 1 -- 172.21.15.72:0/371186508 learned_addr learned my addr 172.21.15.72:0/371186508 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:58:52.235 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.233+0000 7f917fe00640 1 --2- 172.21.15.72:0/371186508 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9180104b70 0x7f918019ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:52.235 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.233+0000 7f917f400640 1 --2- 172.21.15.72:0/371186508 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9180105d60 0x7f918019b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:52.235 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.233+0000 7f9184e00640 1 -- 172.21.15.72:0/371186508 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9180105d60 msgr2=0x7f918019b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:52.235 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.233+0000 7f9184e00640 1 --2- 172.21.15.72:0/371186508 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9180105d60 0x7f918019b2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:52.235 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.233+0000 7f9184e00640 1 -- 172.21.15.72:0/371186508 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9180104b70 msgr2=0x7f918019ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:52.235 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.233+0000 7f9184e00640 1 --2- 172.21.15.72:0/371186508 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9180104b70 0x7f918019ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:52.235 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.233+0000 7f9184e00640 1 -- 172.21.15.72:0/371186508 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91801a2950 con 0x7f9180106720 2024-04-26T21:58:52.235 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.233+0000 7f917f400640 1 --2- 172.21.15.72:0/371186508 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9180105d60 0x7f918019b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:58:52.235 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.233+0000 7f917fe00640 1 --2- 172.21.15.72:0/371186508 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9180104b70 0x7f918019ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:58:52.235 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.233+0000 7f9184e00640 1 --2- 172.21.15.72:0/371186508 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9180106720 0x7f91801a22b0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f917800eaa0 tx=0x7f917800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:58:52.235 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.233+0000 7f917cc00640 1 -- 172.21.15.72:0/371186508 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f917800cdb0 con 0x7f9180106720 2024-04-26T21:58:52.235 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.234+0000 7f9186e00640 1 -- 172.21.15.72:0/371186508 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91801a2c40 con 0x7f9180106720 2024-04-26T21:58:52.236 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.234+0000 7f917cc00640 1 -- 172.21.15.72:0/371186508 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9178005020 con 0x7f9180106720 2024-04-26T21:58:52.236 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.234+0000 7f917cc00640 1 -- 172.21.15.72:0/371186508 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91780116d0 con 0x7f9180106720 2024-04-26T21:58:52.236 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.234+0000 7f9186e00640 1 -- 172.21.15.72:0/371186508 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91801a3140 con 0x7f9180106720 2024-04-26T21:58:52.237 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.237+0000 7f9189b79740 1 Processor -- start 2024-04-26T21:58:52.237 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.237+0000 7f917cc00640 1 -- 172.21.15.72:0/371186508 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f91780051d0 con 0x7f9180106720 2024-04-26T21:58:52.237 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.238+0000 7f9189b79740 1 -- start start 2024-04-26T21:58:52.238 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.238+0000 7f9189b79740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f38e95aea0 0x55f38e95b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:52.238 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.238+0000 7f9189b79740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f38e95b890 0x55f38e960390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:52.238 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.238+0000 7f9189b79740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f38e960a00 0x55f38e962e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:52.239 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.238+0000 7f9189b79740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55f38e8bec70 con 0x55f38e95aea0 2024-04-26T21:58:52.239 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.238+0000 7f9189b79740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55f38e877980 con 0x55f38e960a00 2024-04-26T21:58:52.239 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.238+0000 7f9189b79740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55f38e8bac30 con 0x55f38e95b890 2024-04-26T21:58:52.239 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.238+0000 7f917f400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f38e95aea0 0x55f38e95b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:52.239 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.238+0000 7f917f400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f38e95aea0 0x55f38e95b2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:57744/0 (socket says 172.21.15.72:57744) 2024-04-26T21:58:52.239 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.238+0000 7f917f400640 1 -- 172.21.15.72:0/1504473456 learned_addr learned my addr 172.21.15.72:0/1504473456 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:58:52.239 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.239+0000 7f917fe00640 1 --2- 172.21.15.72:0/1504473456 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f38e95b890 0x55f38e960390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:52.239 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.239+0000 7f917f400640 1 --2- 172.21.15.72:0/1504473456 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f38e960a00 0x55f38e962e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:52.241 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.239+0000 7f917f400640 1 -- 172.21.15.72:0/1504473456 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f38e95b890 msgr2=0x55f38e960390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:58:52.241 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.239+0000 7f917f400640 1 --2- 172.21.15.72:0/1504473456 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f38e95b890 0x55f38e960390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:52.241 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.239+0000 7f917f400640 1 -- 172.21.15.72:0/1504473456 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f38e960a00 msgr2=0x55f38e962e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:58:52.242 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.239+0000 7f917f400640 1 --2- 172.21.15.72:0/1504473456 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f38e960a00 0x55f38e962e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:52.242 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.239+0000 7f917f400640 1 -- 172.21.15.72:0/1504473456 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f38e84f7b0 con 0x55f38e95aea0 2024-04-26T21:58:52.242 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.239+0000 7f917f400640 1 --2- 172.21.15.72:0/1504473456 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f38e95aea0 0x55f38e95b2c0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f916c02a910 tx=0x7f916c005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=511b7f1703fcdb71 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:58:52.242 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.240+0000 7f917cc00640 1 --2- 172.21.15.72:0/371186508 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f91640a3c40 0x7f91640a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:52.242 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.240+0000 7f9175600640 1 -- 172.21.15.72:0/1504473456 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f916c035070 con 0x55f38e95aea0 2024-04-26T21:58:52.242 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.240+0000 7f917cc00640 1 -- 172.21.15.72:0/371186508 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7f91780d08d0 con 0x7f9180106720 2024-04-26T21:58:52.242 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.240+0000 7f9175600640 1 -- 172.21.15.72:0/1504473456 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f916c045070 con 0x55f38e95aea0 2024-04-26T21:58:52.242 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.240+0000 7f9175600640 1 -- 172.21.15.72:0/1504473456 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f916c003bf0 con 0x55f38e95aea0 2024-04-26T21:58:52.242 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.240+0000 7f917fe00640 1 --2- 172.21.15.72:0/371186508 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f91640a3c40 0x7f91640a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:52.243 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.241+0000 7f9189b79740 1 -- 172.21.15.72:0/1504473456 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f38e95aea0 msgr2=0x55f38e95b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:52.243 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.241+0000 7f9189b79740 1 --2- 172.21.15.72:0/1504473456 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f38e95aea0 0x55f38e95b2c0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f916c02a910 tx=0x7f916c005b50 comp rx=0 tx=0).stop 2024-04-26T21:58:52.243 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.241+0000 7f917fe00640 1 --2- 172.21.15.72:0/371186508 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f91640a3c40 0x7f91640a6100 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x55f38e95bc90 tx=0x7f917000b600 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:58:52.243 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.241+0000 7f9189b79740 1 -- 172.21.15.72:0/1504473456 shutdown_connections 2024-04-26T21:58:52.243 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.241+0000 7f9189b79740 1 --2- 172.21.15.72:0/1504473456 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f38e960a00 0x55f38e962e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:52.243 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.241+0000 7f9189b79740 1 --2- 172.21.15.72:0/1504473456 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f38e95b890 0x55f38e960390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:52.243 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.241+0000 7f9189b79740 1 --2- 172.21.15.72:0/1504473456 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f38e95aea0 0x55f38e95b2c0 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:52.243 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.241+0000 7f9189b79740 1 -- 172.21.15.72:0/1504473456 >> 172.21.15.72:0/1504473456 conn(0x55f38e956510 msgr2=0x55f38e956910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:58:52.243 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.241+0000 7f9189b79740 1 -- 172.21.15.72:0/1504473456 shutdown_connections 2024-04-26T21:58:52.243 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.241+0000 7f9189b79740 1 -- 172.21.15.72:0/1504473456 wait complete. 2024-04-26T21:58:52.243 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.241+0000 7f9189b79740 1 -- start start 2024-04-26T21:58:52.243 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.241+0000 7f9189b79740 1 Processor -- start 2024-04-26T21:58:52.246 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.242+0000 7f9189b79740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f38e95aea0 0x55f38e964dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:52.246 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.242+0000 7f9189b79740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f38e95b890 0x55f38e965300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:52.246 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.242+0000 7f9189b79740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f38e960a00 0x55f38e96c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:52.246 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.242+0000 7f9189b79740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55f38e8bdf80 con 0x55f38e960a00 2024-04-26T21:58:52.247 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.242+0000 7f9189b79740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55f38e86b260 con 0x55f38e95b890 2024-04-26T21:58:52.247 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.242+0000 7f9189b79740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55f38e8c32a0 con 0x55f38e95aea0 2024-04-26T21:58:52.247 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.242+0000 7f9189b79740 10 client.0 _ll_register_callbacks cb 0x55f38e8b5960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:58:52.247 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.242+0000 7f9184e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f38e960a00 0x55f38e96c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:52.247 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.242+0000 7f9184e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f38e960a00 0x55f38e96c340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:57754/0 (socket says 172.21.15.72:57754) 2024-04-26T21:58:52.247 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.242+0000 7f9184e00640 1 -- 172.21.15.72:0/4125580694 learned_addr learned my addr 172.21.15.72:0/4125580694 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:58:52.247 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.242+0000 7f917fe00640 1 --2- 172.21.15.72:0/4125580694 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f38e95aea0 0x55f38e964dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:52.247 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.243+0000 7f917f400640 1 --2- 172.21.15.72:0/4125580694 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f38e95b890 0x55f38e965300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:52.247 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.243+0000 7f9184e00640 1 -- 172.21.15.72:0/4125580694 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f38e95aea0 msgr2=0x55f38e964dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:58:52.247 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.243+0000 7f9184e00640 1 --2- 172.21.15.72:0/4125580694 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f38e95aea0 0x55f38e964dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:52.247 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.243+0000 7f9184e00640 1 -- 172.21.15.72:0/4125580694 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f38e95b890 msgr2=0x55f38e965300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:58:52.247 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.243+0000 7f9184e00640 1 --2- 172.21.15.72:0/4125580694 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f38e95b890 0x55f38e965300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:52.247 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.243+0000 7f9184e00640 1 -- 172.21.15.72:0/4125580694 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f38e84d1c0 con 0x55f38e960a00 2024-04-26T21:58:52.248 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.243+0000 7f9184e00640 1 --2- 172.21.15.72:0/4125580694 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f38e960a00 0x55f38e96c340 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f9178099310 tx=0x7f9178005720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32edcf60e9dd4c94 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:58:52.248 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.243+0000 7f916a000640 10 client.0 ms_handle_connect on v2:172.21.15.72:3300/0 2024-04-26T21:58:52.248 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.243+0000 7f916a000640 1 -- 172.21.15.72:0/4125580694 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f917801f040 con 0x55f38e960a00 2024-04-26T21:58:52.248 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.243+0000 7f916a000640 1 -- 172.21.15.72:0/4125580694 <== mon.0 v2:172.21.15.72:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7f91780bbb90 con 0x55f38e960a00 2024-04-26T21:58:52.248 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.243+0000 7f9189b79740 1 -- 172.21.15.72:0/4125580694 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f916c045070 con 0x55f38e960a00 2024-04-26T21:58:52.248 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.244+0000 7f916a000640 1 -- 172.21.15.72:0/4125580694 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f9178004c40 con 0x55f38e960a00 2024-04-26T21:58:52.248 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.244+0000 7f9189b79740 10 client.16353 fetch_fsmap learned FSMap version 27 2024-04-26T21:58:52.248 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.244+0000 7f9189b79740 1 -- 172.21.15.72:0/4125580694 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f38e5d03f0 con 0x55f38e960a00 2024-04-26T21:58:52.248 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.244+0000 7f916a000640 1 -- 172.21.15.72:0/4125580694 <== mon.0 v2:172.21.15.72:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f916c045070 con 0x55f38e960a00 2024-04-26T21:58:52.248 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.244+0000 7f916a000640 1 -- 172.21.15.72:0/4125580694 <== mon.0 v2:172.21.15.72:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f917800f040 con 0x55f38e960a00 2024-04-26T21:58:52.248 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.244+0000 7f9189b79740 10 client.16353 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T21:58:52.248 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.244+0000 7f9189b79740 10 client.16353 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-26T21:58:52.248 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.244+0000 7f9189b79740 20 client.16353 populate_metadata read hostname 'smithi072' 2024-04-26T21:58:52.248 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.244+0000 7f9189b79740 1 --2- 172.21.15.72:0/4125580694 >> [v2:172.21.15.76:6836/1150678991,v1:172.21.15.76:6837/1150678991] conn(0x55f38e9708c0 0x55f38e972d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:52.249 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.244+0000 7f9189b79740 4 client.16353 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:58:52.249 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.244+0000 7f9189b79740 1 -- 172.21.15.72:0/4125580694 --> [v2:172.21.15.76:6836/1150678991,v1:172.21.15.76:6837/1150678991] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f38e8cc880 con 0x55f38e9708c0 2024-04-26T21:58:52.249 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.245+0000 7f917fe00640 1 --2- 172.21.15.72:0/4125580694 >> [v2:172.21.15.76:6836/1150678991,v1:172.21.15.76:6837/1150678991] conn(0x55f38e9708c0 0x55f38e972d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:52.249 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.245+0000 7f917fe00640 1 --2- 172.21.15.72:0/4125580694 >> [v2:172.21.15.76:6836/1150678991,v1:172.21.15.76:6837/1150678991] conn(0x55f38e9708c0 0x55f38e972d60 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8888cc12f7fb0e16 server_cookie=a9615800462e2aee in_seq=0 out_seq=0 2024-04-26T21:58:52.249 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.245+0000 7f916a000640 10 client.16353 ms_handle_connect on v2:172.21.15.76:6836/1150678991 2024-04-26T21:58:52.249 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.246+0000 7f916a000640 1 -- 172.21.15.72:0/4125580694 <== mds.1 v2:172.21.15.76:6836/1150678991 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f38e8cc880 con 0x55f38e9708c0 2024-04-26T21:58:52.249 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.246+0000 7f916a000640 10 client.16353 handle_command_reply: tid=0 2024-04-26T21:58:52.249 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.247+0000 7f9189b79740 1 client.16353 shutdown 2024-04-26T21:58:52.250 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.247+0000 7f9189b79740 1 -- 172.21.15.72:0/4125580694 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f38e960a00 msgr2=0x55f38e96c340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:52.250 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.247+0000 7f9189b79740 1 --2- 172.21.15.72:0/4125580694 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f38e960a00 0x55f38e96c340 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f9178099310 tx=0x7f9178005720 comp rx=0 tx=0).stop 2024-04-26T21:58:52.250 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.247+0000 7f9189b79740 1 -- 172.21.15.72:0/4125580694 shutdown_connections 2024-04-26T21:58:52.250 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.247+0000 7f9189b79740 1 --2- 172.21.15.72:0/4125580694 >> [v2:172.21.15.76:6836/1150678991,v1:172.21.15.76:6837/1150678991] conn(0x55f38e9708c0 0x55f38e972d60 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:52.250 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.247+0000 7f9189b79740 1 --2- 172.21.15.72:0/4125580694 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f38e960a00 0x55f38e96c340 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:52.250 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.247+0000 7f916a000640 0 client.16353 ms_handle_reset on v2:172.21.15.76:6836/1150678991 2024-04-26T21:58:52.250 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.247+0000 7f9189b79740 1 --2- 172.21.15.72:0/4125580694 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f38e95b890 0x55f38e965300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:52.251 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.247+0000 7f9189b79740 1 --2- 172.21.15.72:0/4125580694 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f38e95aea0 0x55f38e964dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:52.251 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.247+0000 7f9189b79740 1 -- 172.21.15.72:0/4125580694 >> 172.21.15.72:0/4125580694 conn(0x55f38e956510 msgr2=0x55f38e955690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:58:52.251 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.247+0000 7f9189b79740 1 -- 172.21.15.72:0/4125580694 shutdown_connections 2024-04-26T21:58:52.251 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.247+0000 7f9189b79740 1 -- 172.21.15.72:0/4125580694 wait complete. 2024-04-26T21:58:52.251 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.247+0000 7f9189b79740 20 client.16353 trim_cache size 0 max 16384 2024-04-26T21:58:52.263 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.262+0000 7f9189b79740 1 Processor -- start 2024-04-26T21:58:52.263 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.262+0000 7f9189b79740 1 -- start start 2024-04-26T21:58:52.263 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.262+0000 7f9189b79740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f38e95aea0 0x55f38e955690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:52.263 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.262+0000 7f9189b79740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f38e95b890 0x55f38e96c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:52.263 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.262+0000 7f9189b79740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f38e960a00 0x55f38e977620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:52.263 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.262+0000 7f9189b79740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f91801ac020 con 0x55f38e95aea0 2024-04-26T21:58:52.263 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.262+0000 7f9189b79740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f91801adb60 con 0x55f38e95b890 2024-04-26T21:58:52.263 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.262+0000 7f9189b79740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f91801ac6f0 con 0x55f38e960a00 2024-04-26T21:58:52.263 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.263+0000 7f917fe00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f38e95aea0 0x55f38e955690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:52.263 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.263+0000 7f917fe00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f38e95aea0 0x55f38e955690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:57766/0 (socket says 172.21.15.72:57766) 2024-04-26T21:58:52.263 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.263+0000 7f917fe00640 1 -- 172.21.15.72:0/996893952 learned_addr learned my addr 172.21.15.72:0/996893952 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:58:52.263 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.263+0000 7f917f400640 1 --2- 172.21.15.72:0/996893952 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f38e95b890 0x55f38e96c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:52.264 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.263+0000 7f9184e00640 1 --2- 172.21.15.72:0/996893952 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f38e960a00 0x55f38e977620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:52.264 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.263+0000 7f917fe00640 1 -- 172.21.15.72:0/996893952 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f38e960a00 msgr2=0x55f38e977620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:58:52.264 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.263+0000 7f917fe00640 1 --2- 172.21.15.72:0/996893952 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f38e960a00 0x55f38e977620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:52.264 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.263+0000 7f917fe00640 1 -- 172.21.15.72:0/996893952 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f38e95b890 msgr2=0x55f38e96c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:58:52.264 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.263+0000 7f917fe00640 1 --2- 172.21.15.72:0/996893952 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f38e95b890 0x55f38e96c300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:52.264 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.263+0000 7f917fe00640 1 -- 172.21.15.72:0/996893952 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f38e69a7d0 con 0x55f38e95aea0 2024-04-26T21:58:52.266 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.263+0000 7f917fe00640 1 --2- 172.21.15.72:0/996893952 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f38e95aea0 0x55f38e955690 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f918019c210 tx=0x7f9170015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c516d6faa89fae90 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:58:52.266 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.264+0000 7f9174c00640 1 -- 172.21.15.72:0/996893952 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f917001fe80 con 0x55f38e95aea0 2024-04-26T21:58:52.266 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.264+0000 7f9174c00640 1 -- 172.21.15.72:0/996893952 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f917002f070 con 0x55f38e95aea0 2024-04-26T21:58:52.267 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.264+0000 7f9174c00640 1 -- 172.21.15.72:0/996893952 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9170020bc0 con 0x55f38e95aea0 2024-04-26T21:58:52.267 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.264+0000 7f9189b79740 1 -- 172.21.15.72:0/996893952 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f38e95aea0 msgr2=0x55f38e955690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:52.267 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.264+0000 7f9189b79740 1 --2- 172.21.15.72:0/996893952 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f38e95aea0 0x55f38e955690 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f918019c210 tx=0x7f9170015000 comp rx=0 tx=0).stop 2024-04-26T21:58:52.267 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.265+0000 7f9189b79740 1 -- 172.21.15.72:0/996893952 shutdown_connections 2024-04-26T21:58:52.267 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.265+0000 7f9189b79740 1 --2- 172.21.15.72:0/996893952 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f38e960a00 0x55f38e977620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:52.267 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.265+0000 7f9189b79740 1 --2- 172.21.15.72:0/996893952 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f38e95b890 0x55f38e96c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:52.267 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.265+0000 7f9189b79740 1 --2- 172.21.15.72:0/996893952 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f38e95aea0 0x55f38e955690 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:52.267 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.265+0000 7f9189b79740 1 -- 172.21.15.72:0/996893952 >> 172.21.15.72:0/996893952 conn(0x55f38e956510 msgr2=0x55f38e959b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:58:52.267 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.265+0000 7f9189b79740 1 -- 172.21.15.72:0/996893952 shutdown_connections 2024-04-26T21:58:52.267 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.265+0000 7f9189b79740 1 -- 172.21.15.72:0/996893952 wait complete. 2024-04-26T21:58:52.267 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.265+0000 7f9189b79740 1 -- start start 2024-04-26T21:58:52.267 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.265+0000 7f9189b79740 1 Processor -- start 2024-04-26T21:58:52.267 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.266+0000 7f9189b79740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f38e95aea0 0x55f38e969ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:52.268 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.266+0000 7f9189b79740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f38e95b890 0x55f38e96a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:52.268 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.266+0000 7f9189b79740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f38e960a00 0x55f38e96b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:52.268 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.266+0000 7f9189b79740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f91801ad490 con 0x55f38e95b890 2024-04-26T21:58:52.268 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.266+0000 7f9189b79740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f91801acdc0 con 0x55f38e95aea0 2024-04-26T21:58:52.268 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.266+0000 7f9189b79740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f91801aa000 con 0x55f38e960a00 2024-04-26T21:58:52.268 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.266+0000 7f9189b79740 10 client.0 _ll_register_callbacks cb 0x7f9178098be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:58:52.268 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.266+0000 7f917f400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f38e95b890 0x55f38e96a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:52.268 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.266+0000 7f917f400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f38e95b890 0x55f38e96a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:57774/0 (socket says 172.21.15.72:57774) 2024-04-26T21:58:52.268 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.266+0000 7f917f400640 1 -- 172.21.15.72:0/561799026 learned_addr learned my addr 172.21.15.72:0/561799026 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:58:52.268 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.266+0000 7f9184e00640 1 --2- 172.21.15.72:0/561799026 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f38e960a00 0x55f38e96b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:52.268 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.266+0000 7f917fe00640 1 --2- 172.21.15.72:0/561799026 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f38e95aea0 0x55f38e969ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:52.268 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.266+0000 7f917f400640 1 -- 172.21.15.72:0/561799026 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f38e960a00 msgr2=0x55f38e96b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:58:52.268 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.266+0000 7f917f400640 1 --2- 172.21.15.72:0/561799026 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f38e960a00 0x55f38e96b3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:52.268 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.266+0000 7f917f400640 1 -- 172.21.15.72:0/561799026 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f38e95aea0 msgr2=0x55f38e969ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:58:52.268 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.266+0000 7f917f400640 1 --2- 172.21.15.72:0/561799026 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f38e95aea0 0x55f38e969ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:52.269 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.266+0000 7f917f400640 1 -- 172.21.15.72:0/561799026 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f38e8652a0 con 0x55f38e95b890 2024-04-26T21:58:52.269 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.267+0000 7f917f400640 1 --2- 172.21.15.72:0/561799026 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f38e95b890 0x55f38e96a3e0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f916c02a910 tx=0x7f916c031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f2c96b794047d4a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:58:52.269 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.267+0000 7f916a000640 10 client.0 ms_handle_connect on v2:172.21.15.72:3300/0 2024-04-26T21:58:52.271 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.267+0000 7f916a000640 1 -- 172.21.15.72:0/561799026 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f916c04d070 con 0x55f38e95b890 2024-04-26T21:58:52.271 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.267+0000 7f916a000640 1 -- 172.21.15.72:0/561799026 <== mon.0 v2:172.21.15.72:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7f916c004240 con 0x55f38e95b890 2024-04-26T21:58:52.271 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.267+0000 7f9189b79740 1 -- 172.21.15.72:0/561799026 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f917002f070 con 0x55f38e95b890 2024-04-26T21:58:52.271 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.267+0000 7f916a000640 1 -- 172.21.15.72:0/561799026 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f916c059050 con 0x55f38e95b890 2024-04-26T21:58:52.271 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.267+0000 7f9189b79740 10 client.16359 fetch_fsmap learned FSMap version 27 2024-04-26T21:58:52.271 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.267+0000 7f9189b79740 1 -- 172.21.15.72:0/561799026 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f38e9d71a0 con 0x55f38e95b890 2024-04-26T21:58:52.271 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.268+0000 7f916a000640 1 -- 172.21.15.72:0/561799026 <== mon.0 v2:172.21.15.72:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f917002f070 con 0x55f38e95b890 2024-04-26T21:58:52.272 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.268+0000 7f916a000640 1 -- 172.21.15.72:0/561799026 <== mon.0 v2:172.21.15.72:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f916c03d480 con 0x55f38e95b890 2024-04-26T21:58:52.272 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.268+0000 7f9189b79740 10 client.16359 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T21:58:52.272 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.268+0000 7f9189b79740 10 client.16359 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-26T21:58:52.272 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.268+0000 7f9189b79740 20 client.16359 populate_metadata read hostname 'smithi072' 2024-04-26T21:58:52.272 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.268+0000 7f9189b79740 1 --2- 172.21.15.72:0/561799026 >> [v2:172.21.15.76:6836/1150678991,v1:172.21.15.76:6837/1150678991] conn(0x55f38e9d7660 0x55f38e9d9a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:52.272 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.268+0000 7f9189b79740 4 client.16359 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-26T21:58:52.272 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.268+0000 7f9189b79740 1 -- 172.21.15.72:0/561799026 --> [v2:172.21.15.76:6836/1150678991,v1:172.21.15.76:6837/1150678991] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f38e8c66f0 con 0x55f38e9d7660 2024-04-26T21:58:52.272 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.268+0000 7f917fe00640 1 --2- 172.21.15.72:0/561799026 >> [v2:172.21.15.76:6836/1150678991,v1:172.21.15.76:6837/1150678991] conn(0x55f38e9d7660 0x55f38e9d9a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:52.272 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.269+0000 7f917fe00640 1 --2- 172.21.15.72:0/561799026 >> [v2:172.21.15.76:6836/1150678991,v1:172.21.15.76:6837/1150678991] conn(0x55f38e9d7660 0x55f38e9d9a50 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3e605a9ed63a523e server_cookie=e57e5b467f3f336b in_seq=0 out_seq=0 2024-04-26T21:58:52.272 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.269+0000 7f916a000640 10 client.16359 ms_handle_connect on v2:172.21.15.76:6836/1150678991 2024-04-26T21:58:52.273 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.269+0000 7f916a000640 1 -- 172.21.15.72:0/561799026 <== mds.1 v2:172.21.15.76:6836/1150678991 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f38e8c66f0 con 0x55f38e9d7660 2024-04-26T21:58:52.273 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.269+0000 7f916a000640 10 client.16359 handle_command_reply: tid=0 2024-04-26T21:58:52.273 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.269+0000 7f9189b79740 1 client.16359 shutdown 2024-04-26T21:58:52.273 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.270+0000 7f9189b79740 1 -- 172.21.15.72:0/561799026 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f38e95b890 msgr2=0x55f38e96a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:52.273 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.270+0000 7f9189b79740 1 --2- 172.21.15.72:0/561799026 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f38e95b890 0x55f38e96a3e0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f916c02a910 tx=0x7f916c031400 comp rx=0 tx=0).stop 2024-04-26T21:58:52.273 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.270+0000 7f9189b79740 1 -- 172.21.15.72:0/561799026 shutdown_connections 2024-04-26T21:58:52.273 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.270+0000 7f9189b79740 1 --2- 172.21.15.72:0/561799026 >> [v2:172.21.15.76:6836/1150678991,v1:172.21.15.76:6837/1150678991] conn(0x55f38e9d7660 0x55f38e9d9a50 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:52.273 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.270+0000 7f9189b79740 1 --2- 172.21.15.72:0/561799026 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f38e960a00 0x55f38e96b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:52.273 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.270+0000 7f9189b79740 1 --2- 172.21.15.72:0/561799026 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f38e95b890 0x55f38e96a3e0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:52.273 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.270+0000 7f916a000640 0 client.16359 ms_handle_reset on v2:172.21.15.76:6836/1150678991 2024-04-26T21:58:52.273 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.270+0000 7f9189b79740 1 --2- 172.21.15.72:0/561799026 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f38e95aea0 0x55f38e969ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:52.273 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.270+0000 7f9189b79740 1 -- 172.21.15.72:0/561799026 >> 172.21.15.72:0/561799026 conn(0x55f38e956510 msgr2=0x55f38e955690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:58:52.273 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.270+0000 7f9189b79740 1 -- 172.21.15.72:0/561799026 shutdown_connections 2024-04-26T21:58:52.273 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.270+0000 7f9189b79740 1 -- 172.21.15.72:0/561799026 wait complete. 2024-04-26T21:58:52.274 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.270+0000 7f9189b79740 20 client.16359 trim_cache size 0 max 16384 2024-04-26T21:58:52.274 INFO:teuthology.orchestra.run.smithi072.stdout:[] 2024-04-26T21:58:52.275 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.271+0000 7f9186e00640 1 -- 172.21.15.72:0/371186508 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f91640a3c40 msgr2=0x7f91640a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:52.275 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.271+0000 7f9186e00640 1 --2- 172.21.15.72:0/371186508 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f91640a3c40 0x7f91640a6100 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x55f38e95bc90 tx=0x7f917000b600 comp rx=0 tx=0).stop 2024-04-26T21:58:52.275 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.271+0000 7f9186e00640 1 -- 172.21.15.72:0/371186508 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9180106720 msgr2=0x7f91801a22b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:52.275 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.271+0000 7f9186e00640 1 --2- 172.21.15.72:0/371186508 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9180106720 0x7f91801a22b0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f917800eaa0 tx=0x7f917800ef60 comp rx=0 tx=0).stop 2024-04-26T21:58:52.275 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.271+0000 7f9186e00640 1 -- 172.21.15.72:0/371186508 shutdown_connections 2024-04-26T21:58:52.275 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.271+0000 7f9186e00640 1 --2- 172.21.15.72:0/371186508 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9180106720 0x7f91801a22b0 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:52.275 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.271+0000 7f9186e00640 1 --2- 172.21.15.72:0/371186508 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f91640a3c40 0x7f91640a6100 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:52.275 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.271+0000 7f9186e00640 1 --2- 172.21.15.72:0/371186508 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9180105d60 0x7f918019b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:52.275 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.271+0000 7f9186e00640 1 --2- 172.21.15.72:0/371186508 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9180104b70 0x7f918019ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:52.275 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.271+0000 7f9186e00640 1 -- 172.21.15.72:0/371186508 >> 172.21.15.72:0/371186508 conn(0x7f91801000e0 msgr2=0x7f9180109220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:58:52.275 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.271+0000 7f9186e00640 1 -- 172.21.15.72:0/371186508 shutdown_connections 2024-04-26T21:58:52.276 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.271+0000 7f9186e00640 1 -- 172.21.15.72:0/371186508 wait complete. 2024-04-26T21:58:52.290 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-26T21:58:52.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:58:51 smithi076 ceph-mon[31823]: from='client.? 172.21.15.72:0/4079986962' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:58:52.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:58:51 smithi164 ceph-mon[31916]: from='client.? 172.21.15.72:0/4079986962' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:58:52.480 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.479+0000 7f422a200640 1 -- 172.21.15.72:0/1893891787 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f42241032f0 msgr2=0x7f4224109b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:52.480 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.479+0000 7f422a200640 1 --2- 172.21.15.72:0/1893891787 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f42241032f0 0x7f4224109b80 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f421c009ca0 tx=0x7f421c02a8c0 comp rx=0 tx=0).stop 2024-04-26T21:58:52.480 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.479+0000 7f422a200640 1 -- 172.21.15.72:0/1893891787 shutdown_connections 2024-04-26T21:58:52.480 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.479+0000 7f422a200640 1 --2- 172.21.15.72:0/1893891787 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f42241032f0 0x7f4224109b80 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:52.480 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.479+0000 7f422a200640 1 --2- 172.21.15.72:0/1893891787 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4224102930 0x7f4224102db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:52.480 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.479+0000 7f422a200640 1 --2- 172.21.15.72:0/1893891787 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4224101740 0x7f4224101b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:52.480 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.479+0000 7f422a200640 1 -- 172.21.15.72:0/1893891787 >> 172.21.15.72:0/1893891787 conn(0x7f42240fccd0 msgr2=0x7f42240ff0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:58:52.480 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.480+0000 7f422a200640 1 -- 172.21.15.72:0/1893891787 shutdown_connections 2024-04-26T21:58:52.480 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.480+0000 7f422a200640 1 -- 172.21.15.72:0/1893891787 wait complete. 2024-04-26T21:58:52.483 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.481+0000 7f422a200640 1 Processor -- start 2024-04-26T21:58:52.483 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.481+0000 7f422a200640 1 -- start start 2024-04-26T21:58:52.483 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.481+0000 7f422a200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4224101740 0x7f422419ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:52.483 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.481+0000 7f422a200640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4224102930 0x7f422419b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:52.483 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.481+0000 7f422a200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f42241032f0 0x7f42241a20e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:52.483 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.481+0000 7f422a200640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f422410d150 con 0x7f4224102930 2024-04-26T21:58:52.483 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.481+0000 7f422a200640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f422410cfd0 con 0x7f4224101740 2024-04-26T21:58:52.483 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.481+0000 7f422a200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f422410d2d0 con 0x7f42241032f0 2024-04-26T21:58:52.483 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.481+0000 7f4223e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4224102930 0x7f422419b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:52.483 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.481+0000 7f4223e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4224102930 0x7f422419b0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:57796/0 (socket says 172.21.15.72:57796) 2024-04-26T21:58:52.483 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.482+0000 7f4223e00640 1 -- 172.21.15.72:0/2878031241 learned_addr learned my addr 172.21.15.72:0/2878031241 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:58:52.483 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.482+0000 7f4228e00640 1 --2- 172.21.15.72:0/2878031241 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4224101740 0x7f422419ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:52.483 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.482+0000 7f4229800640 1 --2- 172.21.15.72:0/2878031241 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f42241032f0 0x7f42241a20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:52.484 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.482+0000 7f4223e00640 1 -- 172.21.15.72:0/2878031241 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f42241032f0 msgr2=0x7f42241a20e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:52.484 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.482+0000 7f4223e00640 1 --2- 172.21.15.72:0/2878031241 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f42241032f0 0x7f42241a20e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:52.484 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.482+0000 7f4223e00640 1 -- 172.21.15.72:0/2878031241 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4224101740 msgr2=0x7f422419ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:52.484 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.482+0000 7f4223e00640 1 --2- 172.21.15.72:0/2878031241 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4224101740 0x7f422419ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:52.484 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.482+0000 7f4223e00640 1 -- 172.21.15.72:0/2878031241 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42241a2750 con 0x7f4224102930 2024-04-26T21:58:52.484 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.482+0000 7f4228e00640 1 --2- 172.21.15.72:0/2878031241 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4224101740 0x7f422419ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:58:52.484 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.482+0000 7f4223e00640 1 --2- 172.21.15.72:0/2878031241 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4224102930 0x7f422419b0c0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f421000d950 tx=0x7f421000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:58:52.484 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.483+0000 7f4221600640 1 -- 172.21.15.72:0/2878031241 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4210017070 con 0x7f4224102930 2024-04-26T21:58:52.484 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.483+0000 7f422a200640 1 -- 172.21.15.72:0/2878031241 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42241a2a40 con 0x7f4224102930 2024-04-26T21:58:52.484 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.483+0000 7f4221600640 1 -- 172.21.15.72:0/2878031241 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4210009c30 con 0x7f4224102930 2024-04-26T21:58:52.484 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.483+0000 7f4221600640 1 -- 172.21.15.72:0/2878031241 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4210020060 con 0x7f4224102930 2024-04-26T21:58:52.484 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.483+0000 7f422a200640 1 -- 172.21.15.72:0/2878031241 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42241a2f90 con 0x7f4224102930 2024-04-26T21:58:52.486 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.486+0000 7f422e5b0740 1 Processor -- start 2024-04-26T21:58:52.486 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.486+0000 7f422e5b0740 1 -- start start 2024-04-26T21:58:52.487 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.487+0000 7f422e5b0740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558bf5690050 0x558bf5690470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:52.487 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.487+0000 7f422e5b0740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558bf5690a40 0x558bf5695540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:52.488 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.487+0000 7f422e5b0740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558bf5695bb0 0x558bf5698040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:52.488 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.487+0000 7f422e5b0740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x558bf55f3c70 con 0x558bf5690050 2024-04-26T21:58:52.488 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.487+0000 7f422e5b0740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x558bf55ac980 con 0x558bf5695bb0 2024-04-26T21:58:52.488 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.487+0000 7f422e5b0740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x558bf55efc30 con 0x558bf5690a40 2024-04-26T21:58:52.488 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.487+0000 7f4229800640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558bf5690050 0x558bf5690470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:52.488 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.487+0000 7f4229800640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558bf5690050 0x558bf5690470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:57804/0 (socket says 172.21.15.72:57804) 2024-04-26T21:58:52.488 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.487+0000 7f4229800640 1 -- 172.21.15.72:0/3766044266 learned_addr learned my addr 172.21.15.72:0/3766044266 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:58:52.488 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.487+0000 7f4221600640 1 -- 172.21.15.72:0/2878031241 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f4210005070 con 0x7f4224102930 2024-04-26T21:58:52.488 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.487+0000 7f4228e00640 1 --2- 172.21.15.72:0/3766044266 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558bf5690a40 0x558bf5695540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:52.488 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.487+0000 7f4229800640 1 -- 172.21.15.72:0/3766044266 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558bf5690a40 msgr2=0x558bf5695540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:58:52.488 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.487+0000 7f4229800640 1 --2- 172.21.15.72:0/3766044266 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558bf5690a40 0x558bf5695540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:52.488 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.487+0000 7f4229800640 1 -- 172.21.15.72:0/3766044266 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558bf5695bb0 msgr2=0x558bf5698040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:58:52.488 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.487+0000 7f4229800640 1 --2- 172.21.15.72:0/3766044266 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558bf5695bb0 0x558bf5698040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:52.489 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.487+0000 7f4229800640 1 -- 172.21.15.72:0/3766044266 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558bf55847b0 con 0x558bf5690050 2024-04-26T21:58:52.489 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.488+0000 7f4229800640 1 --2- 172.21.15.72:0/3766044266 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558bf5690050 0x558bf5690470 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f421c005cc0 tx=0x7f421c009a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=551a5e4497ddab31 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:58:52.490 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.488+0000 7f4216000640 1 -- 172.21.15.72:0/3766044266 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f421c031a80 con 0x558bf5690050 2024-04-26T21:58:52.490 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.488+0000 7f4216000640 1 -- 172.21.15.72:0/3766044266 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f421c036040 con 0x558bf5690050 2024-04-26T21:58:52.490 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.488+0000 7f4216000640 1 -- 172.21.15.72:0/3766044266 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f421c004460 con 0x558bf5690050 2024-04-26T21:58:52.490 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.489+0000 7f4221600640 1 --2- 172.21.15.72:0/2878031241 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f420c0a3c40 0x7f420c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:52.490 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.489+0000 7f422e5b0740 1 -- 172.21.15.72:0/3766044266 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558bf5690050 msgr2=0x558bf5690470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:52.490 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.489+0000 7f422e5b0740 1 --2- 172.21.15.72:0/3766044266 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558bf5690050 0x558bf5690470 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f421c005cc0 tx=0x7f421c009a20 comp rx=0 tx=0).stop 2024-04-26T21:58:52.490 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.489+0000 7f4221600640 1 -- 172.21.15.72:0/2878031241 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7f42100cfe50 con 0x7f4224102930 2024-04-26T21:58:52.491 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.489+0000 7f422e5b0740 1 -- 172.21.15.72:0/3766044266 shutdown_connections 2024-04-26T21:58:52.491 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.489+0000 7f422e5b0740 1 --2- 172.21.15.72:0/3766044266 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558bf5695bb0 0x558bf5698040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:52.491 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.489+0000 7f422e5b0740 1 --2- 172.21.15.72:0/3766044266 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558bf5690a40 0x558bf5695540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:52.491 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.489+0000 7f422e5b0740 1 --2- 172.21.15.72:0/3766044266 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558bf5690050 0x558bf5690470 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:52.491 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.489+0000 7f422e5b0740 1 -- 172.21.15.72:0/3766044266 >> 172.21.15.72:0/3766044266 conn(0x558bf568b6c0 msgr2=0x558bf568bac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:58:52.491 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.489+0000 7f4228e00640 1 --2- 172.21.15.72:0/2878031241 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f420c0a3c40 0x7f420c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:52.491 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.490+0000 7f422e5b0740 1 -- 172.21.15.72:0/3766044266 shutdown_connections 2024-04-26T21:58:52.491 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.490+0000 7f422e5b0740 1 -- 172.21.15.72:0/3766044266 wait complete. 2024-04-26T21:58:52.491 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.490+0000 7f422e5b0740 1 -- start start 2024-04-26T21:58:52.491 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.490+0000 7f4228e00640 1 --2- 172.21.15.72:0/2878031241 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f420c0a3c40 0x7f420c0a6100 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x558bf568c590 tx=0x7f421800b5a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:58:52.491 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.490+0000 7f422e5b0740 1 Processor -- start 2024-04-26T21:58:52.494 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.491+0000 7f422e5b0740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558bf5690050 0x558bf5699f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:52.494 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.491+0000 7f422e5b0740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558bf5690a40 0x558bf569a470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:52.494 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.491+0000 7f422e5b0740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558bf5695bb0 0x558bf56a14b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:52.494 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.491+0000 7f422e5b0740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x558bf55f2f80 con 0x558bf5695bb0 2024-04-26T21:58:52.494 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.491+0000 7f422e5b0740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x558bf55a0260 con 0x558bf5690050 2024-04-26T21:58:52.494 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.491+0000 7f422e5b0740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x558bf55f82a0 con 0x558bf5690a40 2024-04-26T21:58:52.494 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.491+0000 7f422e5b0740 10 client.0 _ll_register_callbacks cb 0x558bf55ea960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:58:52.494 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.491+0000 7f4229800640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558bf5695bb0 0x558bf56a14b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:52.494 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.491+0000 7f4229800640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558bf5695bb0 0x558bf56a14b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:57806/0 (socket says 172.21.15.72:57806) 2024-04-26T21:58:52.495 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.491+0000 7f4229800640 1 -- 172.21.15.72:0/4294113149 learned_addr learned my addr 172.21.15.72:0/4294113149 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:58:52.495 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.491+0000 7f4228e00640 1 --2- 172.21.15.72:0/4294113149 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558bf5690050 0x558bf5699f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:52.495 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.491+0000 7f4223e00640 1 --2- 172.21.15.72:0/4294113149 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558bf5690a40 0x558bf569a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:52.495 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.491+0000 7f4229800640 1 -- 172.21.15.72:0/4294113149 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558bf5690a40 msgr2=0x558bf569a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:58:52.495 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.491+0000 7f4229800640 1 --2- 172.21.15.72:0/4294113149 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558bf5690a40 0x558bf569a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:52.495 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.491+0000 7f4229800640 1 -- 172.21.15.72:0/4294113149 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558bf5690050 msgr2=0x558bf5699f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:58:52.495 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.491+0000 7f4229800640 1 --2- 172.21.15.72:0/4294113149 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558bf5690050 0x558bf5699f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:52.495 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.492+0000 7f4229800640 1 -- 172.21.15.72:0/4294113149 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558bf55821c0 con 0x558bf5695bb0 2024-04-26T21:58:52.495 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.492+0000 7f4229800640 1 --2- 172.21.15.72:0/4294113149 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558bf5695bb0 0x558bf56a14b0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f421c009430 tx=0x7f421c0316e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f2b0364a52c1c3b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:58:52.495 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.492+0000 7f420aa00640 10 client.0 ms_handle_connect on v2:172.21.15.72:3300/0 2024-04-26T21:58:52.495 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.492+0000 7f420aa00640 1 -- 172.21.15.72:0/4294113149 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f421c053030 con 0x558bf5695bb0 2024-04-26T21:58:52.495 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.492+0000 7f420aa00640 1 -- 172.21.15.72:0/4294113149 <== mon.0 v2:172.21.15.72:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7f421c04a050 con 0x558bf5695bb0 2024-04-26T21:58:52.495 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.492+0000 7f422e5b0740 1 -- 172.21.15.72:0/4294113149 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f421c036040 con 0x558bf5695bb0 2024-04-26T21:58:52.495 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.492+0000 7f420aa00640 1 -- 172.21.15.72:0/4294113149 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f421c05d050 con 0x558bf5695bb0 2024-04-26T21:58:52.496 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.492+0000 7f422e5b0740 10 client.16371 fetch_fsmap learned FSMap version 27 2024-04-26T21:58:52.496 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.492+0000 7f422e5b0740 1 -- 172.21.15.72:0/4294113149 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x558bf53053f0 con 0x558bf5695bb0 2024-04-26T21:58:52.496 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.493+0000 7f420aa00640 1 -- 172.21.15.72:0/4294113149 <== mon.0 v2:172.21.15.72:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f421c036040 con 0x558bf5695bb0 2024-04-26T21:58:52.496 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.493+0000 7f420aa00640 1 -- 172.21.15.72:0/4294113149 <== mon.0 v2:172.21.15.72:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f421c0043e0 con 0x558bf5695bb0 2024-04-26T21:58:52.496 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.493+0000 7f422e5b0740 10 client.16371 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T21:58:52.496 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.493+0000 7f422e5b0740 10 client.16371 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:58:52.496 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.493+0000 7f422e5b0740 20 client.16371 populate_metadata read hostname 'smithi072' 2024-04-26T21:58:52.496 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.493+0000 7f422e5b0740 1 --2- 172.21.15.72:0/4294113149 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x558bf56a5610 0x558bf56a7a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:52.496 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.493+0000 7f422e5b0740 4 client.16371 mds_command: new command op to 34517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:58:52.496 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.493+0000 7f422e5b0740 1 -- 172.21.15.72:0/4294113149 --> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558bf5601880 con 0x558bf56a5610 2024-04-26T21:58:52.496 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.494+0000 7f4228e00640 1 --2- 172.21.15.72:0/4294113149 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x558bf56a5610 0x558bf56a7a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:52.497 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.494+0000 7f4228e00640 1 --2- 172.21.15.72:0/4294113149 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x558bf56a5610 0x558bf56a7a20 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d0921d570f55dd5 server_cookie=5fc44cf2221fbe6e in_seq=0 out_seq=0 2024-04-26T21:58:52.498 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.494+0000 7f420aa00640 10 client.16371 ms_handle_connect on v2:172.21.15.76:6838/327145636 2024-04-26T21:58:52.498 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.495+0000 7f420aa00640 1 -- 172.21.15.72:0/4294113149 <== mds.0 v2:172.21.15.76:6838/327145636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558bf5601880 con 0x558bf56a5610 2024-04-26T21:58:52.498 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.495+0000 7f420aa00640 10 client.16371 handle_command_reply: tid=0 2024-04-26T21:58:52.498 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.495+0000 7f422e5b0740 1 client.16371 shutdown 2024-04-26T21:58:52.498 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.496+0000 7f422e5b0740 1 -- 172.21.15.72:0/4294113149 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558bf5695bb0 msgr2=0x558bf56a14b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:52.498 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.496+0000 7f422e5b0740 1 --2- 172.21.15.72:0/4294113149 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558bf5695bb0 0x558bf56a14b0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f421c009430 tx=0x7f421c0316e0 comp rx=0 tx=0).stop 2024-04-26T21:58:52.498 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.496+0000 7f422e5b0740 1 -- 172.21.15.72:0/4294113149 shutdown_connections 2024-04-26T21:58:52.498 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.496+0000 7f422e5b0740 1 --2- 172.21.15.72:0/4294113149 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x558bf56a5610 0x558bf56a7a20 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:52.498 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.496+0000 7f422e5b0740 1 --2- 172.21.15.72:0/4294113149 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558bf5695bb0 0x558bf56a14b0 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:52.498 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.496+0000 7f422e5b0740 1 --2- 172.21.15.72:0/4294113149 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558bf5690a40 0x558bf569a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:52.498 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.496+0000 7f420aa00640 0 client.16371 ms_handle_reset on v2:172.21.15.76:6838/327145636 2024-04-26T21:58:52.498 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.496+0000 7f422e5b0740 1 --2- 172.21.15.72:0/4294113149 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558bf5690050 0x558bf5699f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:52.498 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.496+0000 7f422e5b0740 1 -- 172.21.15.72:0/4294113149 >> 172.21.15.72:0/4294113149 conn(0x558bf568b6c0 msgr2=0x558bf568a7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:58:52.498 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.496+0000 7f422e5b0740 1 -- 172.21.15.72:0/4294113149 shutdown_connections 2024-04-26T21:58:52.499 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.496+0000 7f422e5b0740 1 -- 172.21.15.72:0/4294113149 wait complete. 2024-04-26T21:58:52.499 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.496+0000 7f422e5b0740 20 client.16371 trim_cache size 0 max 16384 2024-04-26T21:58:52.512 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.511+0000 7f422e5b0740 1 Processor -- start 2024-04-26T21:58:52.512 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.511+0000 7f422e5b0740 1 -- start start 2024-04-26T21:58:52.512 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.511+0000 7f422e5b0740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558bf5690050 0x558bf568a630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:52.512 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.511+0000 7f422e5b0740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558bf5690a40 0x558bf568ab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:52.512 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.511+0000 7f422e5b0740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558bf5695bb0 0x558bf56a1470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:52.513 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.511+0000 7f422e5b0740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f42241abf20 con 0x558bf5690a40 2024-04-26T21:58:52.513 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.511+0000 7f422e5b0740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f42241ada60 con 0x558bf5690050 2024-04-26T21:58:52.513 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.511+0000 7f422e5b0740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f42241ac5f0 con 0x558bf5695bb0 2024-04-26T21:58:52.513 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.511+0000 7f4223e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558bf5690a40 0x558bf568ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:52.513 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.511+0000 7f4223e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558bf5690a40 0x558bf568ab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:57810/0 (socket says 172.21.15.72:57810) 2024-04-26T21:58:52.513 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.511+0000 7f4223e00640 1 -- 172.21.15.72:0/912272222 learned_addr learned my addr 172.21.15.72:0/912272222 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:58:52.513 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.512+0000 7f4228e00640 1 --2- 172.21.15.72:0/912272222 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558bf5690050 0x558bf568a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:52.513 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.512+0000 7f4229800640 1 --2- 172.21.15.72:0/912272222 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558bf5695bb0 0x558bf56a1470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:52.513 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.512+0000 7f4223e00640 1 -- 172.21.15.72:0/912272222 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558bf5695bb0 msgr2=0x558bf56a1470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:58:52.513 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.512+0000 7f4223e00640 1 --2- 172.21.15.72:0/912272222 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558bf5695bb0 0x558bf56a1470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:52.513 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.512+0000 7f4223e00640 1 -- 172.21.15.72:0/912272222 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558bf5690050 msgr2=0x558bf568a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:58:52.513 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.512+0000 7f4223e00640 1 --2- 172.21.15.72:0/912272222 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558bf5690050 0x558bf568a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:52.513 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.512+0000 7f4223e00640 1 -- 172.21.15.72:0/912272222 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558bf53cf7d0 con 0x558bf5690a40 2024-04-26T21:58:52.513 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.512+0000 7f4229800640 1 --2- 172.21.15.72:0/912272222 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558bf5695bb0 0x558bf56a1470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:58:52.513 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.512+0000 7f4223e00640 1 --2- 172.21.15.72:0/912272222 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558bf5690a40 0x558bf568ab70 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f42100cdd00 tx=0x7f421009c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af8103df4048fcac server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:58:52.514 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.512+0000 7f4215600640 1 -- 172.21.15.72:0/912272222 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42100b4020 con 0x558bf5690a40 2024-04-26T21:58:52.514 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.512+0000 7f4215600640 1 -- 172.21.15.72:0/912272222 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f42100a0300 con 0x558bf5690a40 2024-04-26T21:58:52.514 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.512+0000 7f4215600640 1 -- 172.21.15.72:0/912272222 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4210015070 con 0x558bf5690a40 2024-04-26T21:58:52.517 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.513+0000 7f422e5b0740 1 -- 172.21.15.72:0/912272222 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558bf5690a40 msgr2=0x558bf568ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:52.517 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.513+0000 7f422e5b0740 1 --2- 172.21.15.72:0/912272222 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558bf5690a40 0x558bf568ab70 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f42100cdd00 tx=0x7f421009c650 comp rx=0 tx=0).stop 2024-04-26T21:58:52.517 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.513+0000 7f422e5b0740 1 -- 172.21.15.72:0/912272222 shutdown_connections 2024-04-26T21:58:52.517 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.513+0000 7f422e5b0740 1 --2- 172.21.15.72:0/912272222 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558bf5695bb0 0x558bf56a1470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:52.517 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.513+0000 7f422e5b0740 1 --2- 172.21.15.72:0/912272222 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558bf5690a40 0x558bf568ab70 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:52.517 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.513+0000 7f422e5b0740 1 --2- 172.21.15.72:0/912272222 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558bf5690050 0x558bf568a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:52.517 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.513+0000 7f422e5b0740 1 -- 172.21.15.72:0/912272222 >> 172.21.15.72:0/912272222 conn(0x558bf568b6c0 msgr2=0x558bf5689f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:58:52.517 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.513+0000 7f422e5b0740 1 -- 172.21.15.72:0/912272222 shutdown_connections 2024-04-26T21:58:52.517 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.513+0000 7f422e5b0740 1 -- 172.21.15.72:0/912272222 wait complete. 2024-04-26T21:58:52.517 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.514+0000 7f422e5b0740 1 -- start start 2024-04-26T21:58:52.518 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.514+0000 7f422e5b0740 1 Processor -- start 2024-04-26T21:58:52.518 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.514+0000 7f422e5b0740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558bf5690050 0x558bf569ee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:52.518 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.514+0000 7f422e5b0740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558bf5690a40 0x558bf569f370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:52.518 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.514+0000 7f422e5b0740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558bf5695bb0 0x558bf56a0380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:52.518 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.514+0000 7f422e5b0740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f42241ad390 con 0x558bf5690a40 2024-04-26T21:58:52.518 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.514+0000 7f422e5b0740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f42241accc0 con 0x558bf5695bb0 2024-04-26T21:58:52.518 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.514+0000 7f422e5b0740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f42241a9f00 con 0x558bf5690050 2024-04-26T21:58:52.518 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.514+0000 7f422e5b0740 10 client.0 _ll_register_callbacks cb 0x7f421800b440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:58:52.518 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.514+0000 7f4223e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558bf5690a40 0x558bf569f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:52.518 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.514+0000 7f4223e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558bf5690a40 0x558bf569f370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:57814/0 (socket says 172.21.15.72:57814) 2024-04-26T21:58:52.518 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.515+0000 7f4223e00640 1 -- 172.21.15.72:0/1027106119 learned_addr learned my addr 172.21.15.72:0/1027106119 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:58:52.518 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.515+0000 7f4228e00640 1 --2- 172.21.15.72:0/1027106119 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558bf5690050 0x558bf569ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:52.518 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.515+0000 7f4229800640 1 --2- 172.21.15.72:0/1027106119 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558bf5695bb0 0x558bf56a0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:52.519 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.515+0000 7f4223e00640 1 -- 172.21.15.72:0/1027106119 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558bf5690050 msgr2=0x558bf569ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:58:52.519 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.515+0000 7f4223e00640 1 --2- 172.21.15.72:0/1027106119 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558bf5690050 0x558bf569ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:52.519 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.515+0000 7f4223e00640 1 -- 172.21.15.72:0/1027106119 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558bf5695bb0 msgr2=0x558bf56a0380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:58:52.519 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.515+0000 7f4223e00640 1 --2- 172.21.15.72:0/1027106119 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558bf5695bb0 0x558bf56a0380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:52.519 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.515+0000 7f4223e00640 1 -- 172.21.15.72:0/1027106119 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558bf559a2a0 con 0x558bf5690a40 2024-04-26T21:58:52.519 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.515+0000 7f4223e00640 1 --2- 172.21.15.72:0/1027106119 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558bf5690a40 0x558bf569f370 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f421009bee0 tx=0x7f4210005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e588f5896f5621a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:58:52.519 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.515+0000 7f420aa00640 10 client.0 ms_handle_connect on v2:172.21.15.72:3300/0 2024-04-26T21:58:52.519 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.515+0000 7f420aa00640 1 -- 172.21.15.72:0/1027106119 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42100b4030 con 0x558bf5690a40 2024-04-26T21:58:52.519 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.515+0000 7f420aa00640 1 -- 172.21.15.72:0/1027106119 <== mon.0 v2:172.21.15.72:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7f4210005360 con 0x558bf5690a40 2024-04-26T21:58:52.519 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.515+0000 7f422e5b0740 1 -- 172.21.15.72:0/1027106119 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f42100a0300 con 0x558bf5690a40 2024-04-26T21:58:52.519 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.516+0000 7f420aa00640 1 -- 172.21.15.72:0/1027106119 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4210015070 con 0x558bf5690a40 2024-04-26T21:58:52.519 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.516+0000 7f422e5b0740 10 client.16377 fetch_fsmap learned FSMap version 27 2024-04-26T21:58:52.519 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.516+0000 7f422e5b0740 1 -- 172.21.15.72:0/1027106119 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x558bf570bc10 con 0x558bf5690a40 2024-04-26T21:58:52.519 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.516+0000 7f420aa00640 1 -- 172.21.15.72:0/1027106119 <== mon.0 v2:172.21.15.72:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f42100a0300 con 0x558bf5690a40 2024-04-26T21:58:52.519 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.516+0000 7f420aa00640 1 -- 172.21.15.72:0/1027106119 <== mon.0 v2:172.21.15.72:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f4210096ac0 con 0x558bf5690a40 2024-04-26T21:58:52.520 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.516+0000 7f422e5b0740 10 client.16377 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T21:58:52.520 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.516+0000 7f422e5b0740 10 client.16377 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:58:52.520 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.516+0000 7f422e5b0740 20 client.16377 populate_metadata read hostname 'smithi072' 2024-04-26T21:58:52.520 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.516+0000 7f422e5b0740 1 --2- 172.21.15.72:0/1027106119 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x558bf570c580 0x558bf570e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:52.520 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.516+0000 7f422e5b0740 4 client.16377 mds_command: new command op to 34517 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-26T21:58:52.520 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.516+0000 7f422e5b0740 1 -- 172.21.15.72:0/1027106119 --> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558bf55fb6f0 con 0x558bf570c580 2024-04-26T21:58:52.520 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.517+0000 7f4228e00640 1 --2- 172.21.15.72:0/1027106119 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x558bf570c580 0x558bf570e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:52.520 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.517+0000 7f4228e00640 1 --2- 172.21.15.72:0/1027106119 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x558bf570c580 0x558bf570e9a0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aabc5e1dbe9f0959 server_cookie=c3da6f0163f5f95 in_seq=0 out_seq=0 2024-04-26T21:58:52.520 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.517+0000 7f420aa00640 10 client.16377 ms_handle_connect on v2:172.21.15.76:6838/327145636 2024-04-26T21:58:52.521 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.518+0000 7f420aa00640 1 -- 172.21.15.72:0/1027106119 <== mds.0 v2:172.21.15.76:6838/327145636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558bf55fb6f0 con 0x558bf570c580 2024-04-26T21:58:52.521 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.518+0000 7f420aa00640 10 client.16377 handle_command_reply: tid=0 2024-04-26T21:58:52.522 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.518+0000 7f422e5b0740 1 client.16377 shutdown 2024-04-26T21:58:52.522 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.518+0000 7f422e5b0740 1 -- 172.21.15.72:0/1027106119 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558bf5690a40 msgr2=0x558bf569f370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:52.522 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.518+0000 7f422e5b0740 1 --2- 172.21.15.72:0/1027106119 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558bf5690a40 0x558bf569f370 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f421009bee0 tx=0x7f4210005880 comp rx=0 tx=0).stop 2024-04-26T21:58:52.522 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.518+0000 7f422e5b0740 1 -- 172.21.15.72:0/1027106119 shutdown_connections 2024-04-26T21:58:52.522 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.518+0000 7f422e5b0740 1 --2- 172.21.15.72:0/1027106119 >> [v2:172.21.15.76:6838/327145636,v1:172.21.15.76:6839/327145636] conn(0x558bf570c580 0x558bf570e9a0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:52.522 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.518+0000 7f422e5b0740 1 --2- 172.21.15.72:0/1027106119 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558bf5695bb0 0x558bf56a0380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:52.522 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.518+0000 7f422e5b0740 1 --2- 172.21.15.72:0/1027106119 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558bf5690a40 0x558bf569f370 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:52.522 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.518+0000 7f420aa00640 0 client.16377 ms_handle_reset on v2:172.21.15.76:6838/327145636 2024-04-26T21:58:52.522 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.518+0000 7f422e5b0740 1 --2- 172.21.15.72:0/1027106119 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558bf5690050 0x558bf569ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:52.522 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.518+0000 7f422e5b0740 1 -- 172.21.15.72:0/1027106119 >> 172.21.15.72:0/1027106119 conn(0x558bf568b6c0 msgr2=0x558bf568a7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:58:52.522 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.518+0000 7f422e5b0740 1 -- 172.21.15.72:0/1027106119 shutdown_connections 2024-04-26T21:58:52.522 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.518+0000 7f422e5b0740 1 -- 172.21.15.72:0/1027106119 wait complete. 2024-04-26T21:58:52.522 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.518+0000 7f422e5b0740 20 client.16377 trim_cache size 0 max 16384 2024-04-26T21:58:52.523 INFO:teuthology.orchestra.run.smithi072.stdout:[] 2024-04-26T21:58:52.523 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.519+0000 7f422a200640 1 -- 172.21.15.72:0/2878031241 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f420c0a3c40 msgr2=0x7f420c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:52.523 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.519+0000 7f422a200640 1 --2- 172.21.15.72:0/2878031241 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f420c0a3c40 0x7f420c0a6100 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x558bf568c590 tx=0x7f421800b5a0 comp rx=0 tx=0).stop 2024-04-26T21:58:52.523 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.519+0000 7f422a200640 1 -- 172.21.15.72:0/2878031241 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4224102930 msgr2=0x7f422419b0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:52.524 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.519+0000 7f422a200640 1 --2- 172.21.15.72:0/2878031241 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4224102930 0x7f422419b0c0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f421000d950 tx=0x7f421000de10 comp rx=0 tx=0).stop 2024-04-26T21:58:52.524 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.519+0000 7f422a200640 1 -- 172.21.15.72:0/2878031241 shutdown_connections 2024-04-26T21:58:52.524 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.519+0000 7f422a200640 1 --2- 172.21.15.72:0/2878031241 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f420c0a3c40 0x7f420c0a6100 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:52.524 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.519+0000 7f422a200640 1 --2- 172.21.15.72:0/2878031241 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f42241032f0 0x7f42241a20e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:52.524 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.519+0000 7f422a200640 1 --2- 172.21.15.72:0/2878031241 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4224102930 0x7f422419b0c0 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:52.524 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.519+0000 7f422a200640 1 --2- 172.21.15.72:0/2878031241 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4224101740 0x7f422419ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:52.524 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.519+0000 7f422a200640 1 -- 172.21.15.72:0/2878031241 >> 172.21.15.72:0/2878031241 conn(0x7f42240fccd0 msgr2=0x7f42240fe7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:58:52.524 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.519+0000 7f422a200640 1 -- 172.21.15.72:0/2878031241 shutdown_connections 2024-04-26T21:58:52.524 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:52.519+0000 7f422a200640 1 -- 172.21.15.72:0/2878031241 wait complete. 2024-04-26T21:58:53.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:58:53 smithi076 ceph-mon[31823]: pgmap v552: 97 pgs: 97 active+clean; 258 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 9.2 MiB/s wr, 2.06k op/s 2024-04-26T21:58:53.321 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:58:53 smithi072 ceph-mon[28138]: pgmap v552: 97 pgs: 97 active+clean; 258 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 9.2 MiB/s wr, 2.06k op/s 2024-04-26T21:58:53.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:58:53 smithi164 ceph-mon[31916]: pgmap v552: 97 pgs: 97 active+clean; 258 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 9.2 MiB/s wr, 2.06k op/s 2024-04-26T21:58:53.537 INFO:tasks.fwd_scrub.fs.[cephfs]:end scrubbing fs: cephfs 2024-04-26T21:58:53.537 INFO:tasks.fwd_scrub:done joining 2024-04-26T21:58:53.537 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-04-26T21:58:53.548 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:58:53.744 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:53.743+0000 7f6ec7e00640 1 -- 172.21.15.72:0/592086677 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6ec0104520 msgr2=0x7f6ec010adb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:53.744 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:53.743+0000 7f6ec7e00640 1 --2- 172.21.15.72:0/592086677 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6ec0104520 0x7f6ec010adb0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f6eb8009900 tx=0x7f6eb802a3f0 comp rx=0 tx=0).stop 2024-04-26T21:58:53.744 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:53.743+0000 7f6ec7e00640 1 -- 172.21.15.72:0/592086677 shutdown_connections 2024-04-26T21:58:53.744 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:53.743+0000 7f6ec7e00640 1 --2- 172.21.15.72:0/592086677 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6ec0104520 0x7f6ec010adb0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:53.744 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:53.743+0000 7f6ec7e00640 1 --2- 172.21.15.72:0/592086677 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6ec0103b60 0x7f6ec0103fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:53.744 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:53.743+0000 7f6ec7e00640 1 --2- 172.21.15.72:0/592086677 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6ec0102970 0x7f6ec0102d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:53.744 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:53.743+0000 7f6ec7e00640 1 -- 172.21.15.72:0/592086677 >> 172.21.15.72:0/592086677 conn(0x7f6ec00fdee0 msgr2=0x7f6ec0100320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:58:53.744 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:53.744+0000 7f6ec7e00640 1 -- 172.21.15.72:0/592086677 shutdown_connections 2024-04-26T21:58:53.744 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:53.744+0000 7f6ec7e00640 1 -- 172.21.15.72:0/592086677 wait complete. 2024-04-26T21:58:53.747 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:53.745+0000 7f6ec7e00640 1 Processor -- start 2024-04-26T21:58:53.747 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:53.745+0000 7f6ec7e00640 1 -- start start 2024-04-26T21:58:53.747 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:53.745+0000 7f6ec7e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6ec0102970 0x7f6ec00766e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:53.747 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:53.745+0000 7f6ec7e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6ec0103b60 0x7f6ec0074d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:53.747 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:53.745+0000 7f6ec7e00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6ec0104520 0x7f6ec0075260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:53.747 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:53.745+0000 7f6ec7e00640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f6ec010e350 con 0x7f6ec0102970 2024-04-26T21:58:53.747 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:53.745+0000 7f6ec7e00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f6ec010e1d0 con 0x7f6ec0104520 2024-04-26T21:58:53.747 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:53.745+0000 7f6ec7e00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f6ec010e4d0 con 0x7f6ec0103b60 2024-04-26T21:58:53.747 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:53.745+0000 7f6ec5400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6ec0102970 0x7f6ec00766e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:53.747 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:53.745+0000 7f6ec5400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6ec0102970 0x7f6ec00766e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:57822/0 (socket says 172.21.15.72:57822) 2024-04-26T21:58:53.747 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:53.746+0000 7f6ec5400640 1 -- 172.21.15.72:0/3358546226 learned_addr learned my addr 172.21.15.72:0/3358546226 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:58:53.748 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:53.746+0000 7f6ec4a00640 1 --2- 172.21.15.72:0/3358546226 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6ec0103b60 0x7f6ec0074d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:53.748 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:53.746+0000 7f6ec5e00640 1 --2- 172.21.15.72:0/3358546226 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6ec0104520 0x7f6ec0075260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:53.748 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:53.746+0000 7f6ec5400640 1 -- 172.21.15.72:0/3358546226 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6ec0103b60 msgr2=0x7f6ec0074d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:53.748 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:53.746+0000 7f6ec5400640 1 --2- 172.21.15.72:0/3358546226 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6ec0103b60 0x7f6ec0074d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:53.748 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:53.746+0000 7f6ec5400640 1 -- 172.21.15.72:0/3358546226 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6ec0104520 msgr2=0x7f6ec0075260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:53.748 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:53.746+0000 7f6ec5400640 1 --2- 172.21.15.72:0/3358546226 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6ec0104520 0x7f6ec0075260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:53.748 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:53.746+0000 7f6ec5400640 1 -- 172.21.15.72:0/3358546226 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ec0075af0 con 0x7f6ec0102970 2024-04-26T21:58:53.748 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:53.746+0000 7f6ec5400640 1 --2- 172.21.15.72:0/3358546226 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6ec0102970 0x7f6ec00766e0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f6eb400ea80 tx=0x7f6eb400ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:58:53.748 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:53.747+0000 7f6ebe000640 1 -- 172.21.15.72:0/3358546226 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eb400ce40 con 0x7f6ec0102970 2024-04-26T21:58:53.748 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:53.747+0000 7f6ec7e00640 1 -- 172.21.15.72:0/3358546226 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ec01a2b90 con 0x7f6ec0102970 2024-04-26T21:58:53.748 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:53.747+0000 7f6ebe000640 1 -- 172.21.15.72:0/3358546226 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6eb4011900 con 0x7f6ec0102970 2024-04-26T21:58:53.748 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:53.747+0000 7f6ebe000640 1 -- 172.21.15.72:0/3358546226 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eb401a9e0 con 0x7f6ec0102970 2024-04-26T21:58:53.748 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:53.747+0000 7f6ec7e00640 1 -- 172.21.15.72:0/3358546226 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ec01a30e0 con 0x7f6ec0102970 2024-04-26T21:58:53.749 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:53.748+0000 7f6ec7e00640 1 -- 172.21.15.72:0/3358546226 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ec0109db0 con 0x7f6ec0102970 2024-04-26T21:58:53.750 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:53.750+0000 7f6ebe000640 1 -- 172.21.15.72:0/3358546226 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f6eb401ab80 con 0x7f6ec0102970 2024-04-26T21:58:53.751 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:53.751+0000 7f6ebe000640 1 --2- 172.21.15.72:0/3358546226 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f6ea80a3c40 0x7f6ea80a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:53.751 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:53.752+0000 7f6ebe000640 1 -- 172.21.15.72:0/3358546226 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7f6eb40d0cc0 con 0x7f6ec0102970 2024-04-26T21:58:53.752 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:53.752+0000 7f6ec4a00640 1 --2- 172.21.15.72:0/3358546226 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f6ea80a3c40 0x7f6ea80a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:53.752 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:53.752+0000 7f6ec4a00640 1 --2- 172.21.15.72:0/3358546226 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f6ea80a3c40 0x7f6ea80a6100 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f6ec0076450 tx=0x7f6eac00b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:58:53.756 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:53.756+0000 7f6ebe000640 1 -- 172.21.15.72:0/3358546226 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f6eb40d7080 con 0x7f6ec0102970 2024-04-26T21:58:53.970 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:53.968+0000 7f6ec7e00640 1 -- 172.21.15.72:0/3358546226 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6ec002dd30 con 0x7f6ec0102970 2024-04-26T21:58:53.971 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:53.970+0000 7f6ebe000640 1 -- 172.21.15.72:0/3358546226 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+22783 (secure 0 0 0) 0x7f6eb4016030 con 0x7f6ec0102970 2024-04-26T21:58:53.971 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T21:58:53.972 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":91,"fsid":"a1780950-0415-11ef-bc93-c7b262605968","created":"2024-04-26T21:42:01.504173+0000","modified":"2024-04-26T21:54:11.892872+0000","last_up_change":"2024-04-26T21:51:43.482437+0000","last_in_change":"2024-04-26T21:51:18.468587+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:46:29.785427+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:52:48.320391+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T21:52:49.517493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"04e06884-b355-4762-9b29-9034b7201e9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6802","nonce":1416765179},{"type":"v1","addr":"172.21.15.72:6803","nonce":1416765179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6804","nonce":1416765179},{"type":"v1","addr":"172.21.15.72:6805","nonce":1416765179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6808","nonce":1416765179},{"type":"v1","addr":"172.21.15.72:6809","nonce":1416765179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6806","nonce":1416765179},{"type":"v1","addr":"172.21.15.72:6807","nonce":1416765179}]},"public_addr":"172.21.15.72:6803/1416765179","cluster_addr":"172.21.15.72:6805/1416765179","heartbeat_back_addr":"172.21.15.72:6809/1416765179","heartbeat_front_addr":"172.21.15.72:6807/1416765179","state":["exists","up"]},{"osd":1,"uuid":"d75a479a-4a2e-48e9-a01e-2b1c9617416a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":755426949},{"type":"v1","addr":"172.21.15.76:6801","nonce":755426949}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":755426949},{"type":"v1","addr":"172.21.15.76:6803","nonce":755426949}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":755426949},{"type":"v1","addr":"172.21.15.76:6807","nonce":755426949}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":755426949},{"type":"v1","addr":"172.21.15.76:6805","nonce":755426949}]},"public_addr":"172.21.15.76:6801/755426949","cluster_addr":"172.21.15.76:6803/755426949","heartbeat_back_addr":"172.21.15.76:6807/755426949","heartbeat_front_addr":"172.21.15.76:6805/755426949","state":["exists","up"]},{"osd":2,"uuid":"b6f8da58-4428-4e8c-a944-7ea5645d6d92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":852265973},{"type":"v1","addr":"172.21.15.164:6801","nonce":852265973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":852265973},{"type":"v1","addr":"172.21.15.164:6803","nonce":852265973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":852265973},{"type":"v1","addr":"172.21.15.164:6807","nonce":852265973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":852265973},{"type":"v1","addr":"172.21.15.164:6805","nonce":852265973}]},"public_addr":"172.21.15.164:6801/852265973","cluster_addr":"172.21.15.164:6803/852265973","heartbeat_back_addr":"172.21.15.164:6807/852265973","heartbeat_front_addr":"172.21.15.164:6805/852265973","state":["exists","up"]},{"osd":3,"uuid":"76d83409-649e-41e5-b4f4-43bbd5b00ea5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6810","nonce":4027763789},{"type":"v1","addr":"172.21.15.72:6811","nonce":4027763789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6812","nonce":4027763789},{"type":"v1","addr":"172.21.15.72:6813","nonce":4027763789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6816","nonce":4027763789},{"type":"v1","addr":"172.21.15.72:6817","nonce":4027763789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6814","nonce":4027763789},{"type":"v1","addr":"172.21.15.72:6815","nonce":4027763789}]},"public_addr":"172.21.15.72:6811/4027763789","cluster_addr":"172.21.15.72:6813/4027763789","heartbeat_back_addr":"172.21.15.72:6817/4027763789","heartbeat_front_addr":"172.21.15.72:6815/4027763789","state":["exists","up"]},{"osd":4,"uuid":"d2cfeea7-1661-4156-bdcd-f770a8d1a878","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":3718867827},{"type":"v1","addr":"172.21.15.76:6809","nonce":3718867827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":3718867827},{"type":"v1","addr":"172.21.15.76:6811","nonce":3718867827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":3718867827},{"type":"v1","addr":"172.21.15.76:6815","nonce":3718867827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":3718867827},{"type":"v1","addr":"172.21.15.76:6813","nonce":3718867827}]},"public_addr":"172.21.15.76:6809/3718867827","cluster_addr":"172.21.15.76:6811/3718867827","heartbeat_back_addr":"172.21.15.76:6815/3718867827","heartbeat_front_addr":"172.21.15.76:6813/3718867827","state":["exists","up"]},{"osd":5,"uuid":"f3896ad6-a040-4ea3-a431-ff9b7fc05c36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":2543954772},{"type":"v1","addr":"172.21.15.164:6809","nonce":2543954772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":2543954772},{"type":"v1","addr":"172.21.15.164:6811","nonce":2543954772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":2543954772},{"type":"v1","addr":"172.21.15.164:6815","nonce":2543954772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":2543954772},{"type":"v1","addr":"172.21.15.164:6813","nonce":2543954772}]},"public_addr":"172.21.15.164:6809/2543954772","cluster_addr":"172.21.15.164:6811/2543954772","heartbeat_back_addr":"172.21.15.164:6815/2543954772","heartbeat_front_addr":"172.21.15.164:6813/2543954772","state":["exists","up"]},{"osd":6,"uuid":"d5d56650-be7e-4ba5-ba3a-d7d473294dab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6818","nonce":2080659027},{"type":"v1","addr":"172.21.15.72:6819","nonce":2080659027}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6820","nonce":2080659027},{"type":"v1","addr":"172.21.15.72:6821","nonce":2080659027}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6824","nonce":2080659027},{"type":"v1","addr":"172.21.15.72:6825","nonce":2080659027}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6822","nonce":2080659027},{"type":"v1","addr":"172.21.15.72:6823","nonce":2080659027}]},"public_addr":"172.21.15.72:6819/2080659027","cluster_addr":"172.21.15.72:6821/2080659027","heartbeat_back_addr":"172.21.15.72:6825/2080659027","heartbeat_front_addr":"172.21.15.72:6823/2080659027","state":["exists","up"]},{"osd":7,"uuid":"18975b5a-9309-4a0f-9f37-3ce09c2e745d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":3144631323},{"type":"v1","addr":"172.21.15.76:6817","nonce":3144631323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":3144631323},{"type":"v1","addr":"172.21.15.76:6819","nonce":3144631323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":3144631323},{"type":"v1","addr":"172.21.15.76:6823","nonce":3144631323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":3144631323},{"type":"v1","addr":"172.21.15.76:6821","nonce":3144631323}]},"public_addr":"172.21.15.76:6817/3144631323","cluster_addr":"172.21.15.76:6819/3144631323","heartbeat_back_addr":"172.21.15.76:6823/3144631323","heartbeat_front_addr":"172.21.15.76:6821/3144631323","state":["exists","up"]},{"osd":8,"uuid":"779e9c1e-8e97-48d4-a854-68ca13f38e0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":627888883},{"type":"v1","addr":"172.21.15.164:6817","nonce":627888883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":627888883},{"type":"v1","addr":"172.21.15.164:6819","nonce":627888883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":627888883},{"type":"v1","addr":"172.21.15.164:6823","nonce":627888883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":627888883},{"type":"v1","addr":"172.21.15.164:6821","nonce":627888883}]},"public_addr":"172.21.15.164:6817/627888883","cluster_addr":"172.21.15.164:6819/627888883","heartbeat_back_addr":"172.21.15.164:6823/627888883","heartbeat_front_addr":"172.21.15.164:6821/627888883","state":["exists","up"]},{"osd":9,"uuid":"d0317850-7b2d-4850-b2cf-0b919505c1fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6826","nonce":1273118485},{"type":"v1","addr":"172.21.15.72:6827","nonce":1273118485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6828","nonce":1273118485},{"type":"v1","addr":"172.21.15.72:6829","nonce":1273118485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6832","nonce":1273118485},{"type":"v1","addr":"172.21.15.72:6833","nonce":1273118485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6830","nonce":1273118485},{"type":"v1","addr":"172.21.15.72:6831","nonce":1273118485}]},"public_addr":"172.21.15.72:6827/1273118485","cluster_addr":"172.21.15.72:6829/1273118485","heartbeat_back_addr":"172.21.15.72:6833/1273118485","heartbeat_front_addr":"172.21.15.72:6831/1273118485","state":["exists","up"]},{"osd":10,"uuid":"e0aa38d8-6011-4d2a-b67b-5bff6d999e47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":2808954459},{"type":"v1","addr":"172.21.15.76:6825","nonce":2808954459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":2808954459},{"type":"v1","addr":"172.21.15.76:6827","nonce":2808954459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":2808954459},{"type":"v1","addr":"172.21.15.76:6831","nonce":2808954459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":2808954459},{"type":"v1","addr":"172.21.15.76:6829","nonce":2808954459}]},"public_addr":"172.21.15.76:6825/2808954459","cluster_addr":"172.21.15.76:6827/2808954459","heartbeat_back_addr":"172.21.15.76:6831/2808954459","heartbeat_front_addr":"172.21.15.76:6829/2808954459","state":["exists","up"]},{"osd":11,"uuid":"981b3d4d-916e-4937-9916-7cc21a891e43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":2354951784},{"type":"v1","addr":"172.21.15.164:6825","nonce":2354951784}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":2354951784},{"type":"v1","addr":"172.21.15.164:6827","nonce":2354951784}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":2354951784},{"type":"v1","addr":"172.21.15.164:6831","nonce":2354951784}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":2354951784},{"type":"v1","addr":"172.21.15.164:6829","nonce":2354951784}]},"public_addr":"172.21.15.164:6825/2354951784","cluster_addr":"172.21.15.164:6827/2354951784","heartbeat_back_addr":"172.21.15.164:6831/2354951784","heartbeat_front_addr":"172.21.15.164:6829/2354951784","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:18.997924+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:52.439081+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:46:25.893288+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:47:00.112807+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:47:35.116874+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:48:09.216344+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:48:45.028853+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:49:19.905397+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:49:55.030536+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:50:30.767662+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:51:06.423267+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:51:41.631211+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.1a","mappings":[{"from":5,"to":7}]},{"pgid":"3.1b","mappings":[{"from":4,"to":6}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.28","mappings":[{"from":5,"to":6}]},{"pgid":"3.2a","mappings":[{"from":3,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.72:0/416277784":"2024-04-27T21:43:09.602791+0000","172.21.15.72:6800/4010358672":"2024-04-27T21:43:09.602791+0000","172.21.15.72:6801/1012617371":"2024-04-27T21:42:34.684837+0000","172.21.15.72:6801/4010358672":"2024-04-27T21:43:09.602791+0000","172.21.15.72:0/3555448179":"2024-04-27T21:43:09.602791+0000","172.21.15.72:6800/1012617371":"2024-04-27T21:42:34.684837+0000","172.21.15.72:0/2920378761":"2024-04-27T21:42:34.684837+0000","172.21.15.72:0/1933740041":"2024-04-27T21:43:09.602791+0000","172.21.15.72:0/3225295775":"2024-04-27T21:42:34.684837+0000","172.21.15.72:0/2326016180":"2024-04-27T21:42:34.684837+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:58:53.972 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:53.972+0000 7f6ec7e00640 1 -- 172.21.15.72:0/3358546226 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f6ea80a3c40 msgr2=0x7f6ea80a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:53.973 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:53.972+0000 7f6ec7e00640 1 --2- 172.21.15.72:0/3358546226 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f6ea80a3c40 0x7f6ea80a6100 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f6ec0076450 tx=0x7f6eac00b390 comp rx=0 tx=0).stop 2024-04-26T21:58:53.973 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:53.972+0000 7f6ec7e00640 1 -- 172.21.15.72:0/3358546226 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6ec0102970 msgr2=0x7f6ec00766e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:53.973 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:53.972+0000 7f6ec7e00640 1 --2- 172.21.15.72:0/3358546226 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6ec0102970 0x7f6ec00766e0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f6eb400ea80 tx=0x7f6eb400ef40 comp rx=0 tx=0).stop 2024-04-26T21:58:53.973 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:53.973+0000 7f6ec7e00640 1 -- 172.21.15.72:0/3358546226 shutdown_connections 2024-04-26T21:58:53.973 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:53.973+0000 7f6ec7e00640 1 --2- 172.21.15.72:0/3358546226 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6ec0104520 0x7f6ec0075260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:53.973 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:53.973+0000 7f6ec7e00640 1 --2- 172.21.15.72:0/3358546226 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f6ea80a3c40 0x7f6ea80a6100 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:53.973 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:53.973+0000 7f6ec7e00640 1 --2- 172.21.15.72:0/3358546226 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6ec0103b60 0x7f6ec0074d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:53.973 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:53.973+0000 7f6ec7e00640 1 --2- 172.21.15.72:0/3358546226 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6ec0102970 0x7f6ec00766e0 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:53.973 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:53.973+0000 7f6ec7e00640 1 -- 172.21.15.72:0/3358546226 >> 172.21.15.72:0/3358546226 conn(0x7f6ec00fdee0 msgr2=0x7f6ec0106f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:58:53.973 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:53.973+0000 7f6ec7e00640 1 -- 172.21.15.72:0/3358546226 shutdown_connections 2024-04-26T21:58:53.973 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:53.973+0000 7f6ec7e00640 1 -- 172.21.15.72:0/3358546226 wait complete. 2024-04-26T21:58:53.991 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:46:29.785427+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T21:52:48.320391+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T21:52:49.517493+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:58:53.991 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:58:54.184 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:54.184+0000 7fdfe2a00640 1 -- 172.21.15.72:0/2112064022 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdfdc102860 msgr2=0x7fdfdc102c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:54.185 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:54.184+0000 7fdfe2a00640 1 --2- 172.21.15.72:0/2112064022 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdfdc102860 0x7fdfdc102c80 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fdfd0009900 tx=0x7fdfd002a410 comp rx=0 tx=0).stop 2024-04-26T21:58:54.185 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:54.184+0000 7fdfe2a00640 1 -- 172.21.15.72:0/2112064022 shutdown_connections 2024-04-26T21:58:54.185 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:54.184+0000 7fdfe2a00640 1 --2- 172.21.15.72:0/2112064022 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdfdc104410 0x7fdfdc10aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:54.185 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:54.184+0000 7fdfe2a00640 1 --2- 172.21.15.72:0/2112064022 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdfdc103a50 0x7fdfdc103ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:54.185 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:54.184+0000 7fdfe2a00640 1 --2- 172.21.15.72:0/2112064022 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdfdc102860 0x7fdfdc102c80 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:54.185 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:54.184+0000 7fdfe2a00640 1 -- 172.21.15.72:0/2112064022 >> 172.21.15.72:0/2112064022 conn(0x7fdfdc0fddd0 msgr2=0x7fdfdc100210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:58:54.185 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:54.184+0000 7fdfe2a00640 1 -- 172.21.15.72:0/2112064022 shutdown_connections 2024-04-26T21:58:54.185 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:54.184+0000 7fdfe2a00640 1 -- 172.21.15.72:0/2112064022 wait complete. 2024-04-26T21:58:54.187 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:54.185+0000 7fdfe2a00640 1 Processor -- start 2024-04-26T21:58:54.187 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:54.186+0000 7fdfe2a00640 1 -- start start 2024-04-26T21:58:54.187 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:54.186+0000 7fdfe2a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdfdc102860 0x7fdfdc19ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:54.187 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:54.186+0000 7fdfe2a00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdfdc103a50 0x7fdfdc19b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:54.187 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:54.186+0000 7fdfe2a00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdfdc104410 0x7fdfdc1a21a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:54.187 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:54.186+0000 7fdfe2a00640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fdfdc10e240 con 0x7fdfdc104410 2024-04-26T21:58:54.187 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:54.186+0000 7fdfe2a00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fdfdc10e0c0 con 0x7fdfdc103a50 2024-04-26T21:58:54.187 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:54.186+0000 7fdfe2a00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fdfdc10e3c0 con 0x7fdfdc102860 2024-04-26T21:58:54.187 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:54.186+0000 7fdfe2000640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdfdc104410 0x7fdfdc1a21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:54.187 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:54.186+0000 7fdfe2000640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdfdc104410 0x7fdfdc1a21a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:57832/0 (socket says 172.21.15.72:57832) 2024-04-26T21:58:54.187 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:54.186+0000 7fdfe2000640 1 -- 172.21.15.72:0/657593458 learned_addr learned my addr 172.21.15.72:0/657593458 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:58:54.188 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:54.187+0000 7fdfe1600640 1 --2- 172.21.15.72:0/657593458 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdfdc102860 0x7fdfdc19ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:54.188 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:54.187+0000 7fdfe0c00640 1 --2- 172.21.15.72:0/657593458 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdfdc103a50 0x7fdfdc19b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:54.188 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:54.187+0000 7fdfe2000640 1 -- 172.21.15.72:0/657593458 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdfdc102860 msgr2=0x7fdfdc19ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:54.188 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:54.187+0000 7fdfe2000640 1 --2- 172.21.15.72:0/657593458 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdfdc102860 0x7fdfdc19ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:54.188 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:54.187+0000 7fdfe2000640 1 -- 172.21.15.72:0/657593458 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdfdc103a50 msgr2=0x7fdfdc19b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:54.188 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:54.187+0000 7fdfe2000640 1 --2- 172.21.15.72:0/657593458 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdfdc103a50 0x7fdfdc19b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:54.188 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:54.187+0000 7fdfe2000640 1 -- 172.21.15.72:0/657593458 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfdc1a2840 con 0x7fdfdc104410 2024-04-26T21:58:54.188 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:54.187+0000 7fdfe2000640 1 --2- 172.21.15.72:0/657593458 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdfdc104410 0x7fdfdc1a21a0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fdfd400eaa0 tx=0x7fdfd400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:58:54.189 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:54.187+0000 7fdfda000640 1 -- 172.21.15.72:0/657593458 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfd400cdb0 con 0x7fdfdc104410 2024-04-26T21:58:54.189 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:54.187+0000 7fdfda000640 1 -- 172.21.15.72:0/657593458 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdfd4005020 con 0x7fdfdc104410 2024-04-26T21:58:54.189 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:54.187+0000 7fdfda000640 1 -- 172.21.15.72:0/657593458 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfd40116d0 con 0x7fdfdc104410 2024-04-26T21:58:54.189 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:54.187+0000 7fdfe2a00640 1 -- 172.21.15.72:0/657593458 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfdc1a2b30 con 0x7fdfdc104410 2024-04-26T21:58:54.189 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:54.188+0000 7fdfe2a00640 1 -- 172.21.15.72:0/657593458 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfdc1a3080 con 0x7fdfdc104410 2024-04-26T21:58:54.189 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:54.189+0000 7fdfe2a00640 1 -- 172.21.15.72:0/657593458 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdfdc1a32b0 con 0x7fdfdc104410 2024-04-26T21:58:54.191 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:54.191+0000 7fdfda000640 1 -- 172.21.15.72:0/657593458 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fdfd40051d0 con 0x7fdfdc104410 2024-04-26T21:58:54.193 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:54.193+0000 7fdfda000640 1 --2- 172.21.15.72:0/657593458 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fdfc40a3c40 0x7fdfc40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:54.193 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:54.194+0000 7fdfda000640 1 -- 172.21.15.72:0/657593458 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7fdfd40d09a0 con 0x7fdfdc104410 2024-04-26T21:58:54.193 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:54.194+0000 7fdfe1600640 1 --2- 172.21.15.72:0/657593458 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fdfc40a3c40 0x7fdfc40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:54.194 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:54.194+0000 7fdfe1600640 1 --2- 172.21.15.72:0/657593458 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fdfc40a3c40 0x7fdfc40a6100 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fdfd00057c0 tx=0x7fdfd00095f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:58:54.198 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:54.198+0000 7fdfda000640 1 -- 172.21.15.72:0/657593458 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fdfd4016030 con 0x7fdfdc104410 2024-04-26T21:58:54.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:58:54 smithi076 ceph-mon[31823]: from='client.? 172.21.15.72:0/3358546226' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:58:54.321 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:58:54 smithi072 ceph-mon[28138]: from='client.? 172.21.15.72:0/3358546226' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:58:54.418 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:54.418+0000 7fdfe2a00640 1 -- 172.21.15.72:0/657593458 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fdfdc1a3480 con 0x7fdfdc104410 2024-04-26T21:58:54.418 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:54.418+0000 7fdfda000640 1 -- 172.21.15.72:0/657593458 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) ==== 93+0+10 (secure 0 0 0) 0x7fdfd409d140 con 0x7fdfdc104410 2024-04-26T21:58:54.418 INFO:teuthology.orchestra.run.smithi072.stdout:pg_num: 1 2024-04-26T21:58:54.421 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:54.420+0000 7fdfe2a00640 1 -- 172.21.15.72:0/657593458 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fdfc40a3c40 msgr2=0x7fdfc40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:54.421 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:54.420+0000 7fdfe2a00640 1 --2- 172.21.15.72:0/657593458 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fdfc40a3c40 0x7fdfc40a6100 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fdfd00057c0 tx=0x7fdfd00095f0 comp rx=0 tx=0).stop 2024-04-26T21:58:54.421 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:54.421+0000 7fdfe2a00640 1 -- 172.21.15.72:0/657593458 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdfdc104410 msgr2=0x7fdfdc1a21a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:54.421 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:54.421+0000 7fdfe2a00640 1 --2- 172.21.15.72:0/657593458 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdfdc104410 0x7fdfdc1a21a0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fdfd400eaa0 tx=0x7fdfd400ef60 comp rx=0 tx=0).stop 2024-04-26T21:58:54.421 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:54.421+0000 7fdfe2a00640 1 -- 172.21.15.72:0/657593458 shutdown_connections 2024-04-26T21:58:54.421 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:54.421+0000 7fdfe2a00640 1 --2- 172.21.15.72:0/657593458 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdfdc104410 0x7fdfdc1a21a0 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:54.421 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:54.421+0000 7fdfe2a00640 1 --2- 172.21.15.72:0/657593458 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdfdc103a50 0x7fdfdc19b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:54.421 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:54.421+0000 7fdfe2a00640 1 --2- 172.21.15.72:0/657593458 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fdfc40a3c40 0x7fdfc40a6100 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:54.421 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:54.421+0000 7fdfe2a00640 1 --2- 172.21.15.72:0/657593458 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdfdc102860 0x7fdfdc19ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:54.421 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:54.421+0000 7fdfe2a00640 1 -- 172.21.15.72:0/657593458 >> 172.21.15.72:0/657593458 conn(0x7fdfdc0fddd0 msgr2=0x7fdfdc1091f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:58:54.421 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:54.421+0000 7fdfe2a00640 1 -- 172.21.15.72:0/657593458 shutdown_connections 2024-04-26T21:58:54.422 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:54.421+0000 7fdfe2a00640 1 -- 172.21.15.72:0/657593458 wait complete. 2024-04-26T21:58:54.440 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T21:58:54.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:58:54 smithi164 ceph-mon[31916]: from='client.? 172.21.15.72:0/3358546226' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:58:54.629 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:54.629+0000 7f7546800640 1 -- 172.21.15.72:0/3187518421 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7540104470 msgr2=0x7f754010ad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:54.629 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:54.629+0000 7f7546800640 1 --2- 172.21.15.72:0/3187518421 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7540104470 0x7f754010ad00 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f7538009ca0 tx=0x7f753802a8c0 comp rx=0 tx=0).stop 2024-04-26T21:58:54.630 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:54.629+0000 7f7546800640 1 -- 172.21.15.72:0/3187518421 shutdown_connections 2024-04-26T21:58:54.630 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:54.629+0000 7f7546800640 1 --2- 172.21.15.72:0/3187518421 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7540104470 0x7f754010ad00 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:54.630 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:54.629+0000 7f7546800640 1 --2- 172.21.15.72:0/3187518421 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7540103ab0 0x7f7540103f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:54.630 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:54.629+0000 7f7546800640 1 --2- 172.21.15.72:0/3187518421 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f75401028c0 0x7f7540102ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:54.630 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:54.629+0000 7f7546800640 1 -- 172.21.15.72:0/3187518421 >> 172.21.15.72:0/3187518421 conn(0x7f75400fde30 msgr2=0x7f7540100270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:58:54.630 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:54.629+0000 7f7546800640 1 -- 172.21.15.72:0/3187518421 shutdown_connections 2024-04-26T21:58:54.630 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:54.629+0000 7f7546800640 1 -- 172.21.15.72:0/3187518421 wait complete. 2024-04-26T21:58:54.632 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:54.630+0000 7f7546800640 1 Processor -- start 2024-04-26T21:58:54.632 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:54.630+0000 7f7546800640 1 -- start start 2024-04-26T21:58:54.632 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:54.631+0000 7f7546800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f75401028c0 0x7f75400766e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:54.632 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:54.631+0000 7f7546800640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7540103ab0 0x7f7540074d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:54.632 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:54.631+0000 7f7546800640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7540104470 0x7f7540075260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:54.632 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:54.631+0000 7f7546800640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f754010e260 con 0x7f7540103ab0 2024-04-26T21:58:54.632 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:54.631+0000 7f7546800640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f754010e0e0 con 0x7f7540104470 2024-04-26T21:58:54.632 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:54.631+0000 7f7546800640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f754010e3e0 con 0x7f75401028c0 2024-04-26T21:58:54.632 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:54.631+0000 7f7544a00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7540103ab0 0x7f7540074d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:54.632 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:54.631+0000 7f7544a00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7540103ab0 0x7f7540074d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:57846/0 (socket says 172.21.15.72:57846) 2024-04-26T21:58:54.632 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:54.631+0000 7f7544a00640 1 -- 172.21.15.72:0/3081735041 learned_addr learned my addr 172.21.15.72:0/3081735041 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:58:54.632 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:54.631+0000 7f7545400640 1 --2- 172.21.15.72:0/3081735041 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f75401028c0 0x7f75400766e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:54.632 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:54.631+0000 7f7545e00640 1 --2- 172.21.15.72:0/3081735041 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7540104470 0x7f7540075260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:54.632 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:54.631+0000 7f7544a00640 1 -- 172.21.15.72:0/3081735041 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f75401028c0 msgr2=0x7f75400766e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:54.633 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:54.631+0000 7f7544a00640 1 --2- 172.21.15.72:0/3081735041 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f75401028c0 0x7f75400766e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:54.633 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:54.632+0000 7f7544a00640 1 -- 172.21.15.72:0/3081735041 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7540104470 msgr2=0x7f7540075260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:54.633 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:54.632+0000 7f7544a00640 1 --2- 172.21.15.72:0/3081735041 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7540104470 0x7f7540075260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:54.633 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:54.632+0000 7f7544a00640 1 -- 172.21.15.72:0/3081735041 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7540075af0 con 0x7f7540103ab0 2024-04-26T21:58:54.633 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:54.632+0000 7f7544a00640 1 --2- 172.21.15.72:0/3081735041 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7540103ab0 0x7f7540074d20 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f752c00c920 tx=0x7f752c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:58:54.634 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:54.632+0000 7f753e000640 1 -- 172.21.15.72:0/3081735041 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f752c007c70 con 0x7f7540103ab0 2024-04-26T21:58:54.634 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:54.632+0000 7f753e000640 1 -- 172.21.15.72:0/3081735041 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f752c007e10 con 0x7f7540103ab0 2024-04-26T21:58:54.634 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:54.632+0000 7f7546800640 1 -- 172.21.15.72:0/3081735041 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75401a29d0 con 0x7f7540103ab0 2024-04-26T21:58:54.634 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:54.632+0000 7f753e000640 1 -- 172.21.15.72:0/3081735041 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f752c018710 con 0x7f7540103ab0 2024-04-26T21:58:54.634 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:54.632+0000 7f7546800640 1 -- 172.21.15.72:0/3081735041 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75401a2f20 con 0x7f7540103ab0 2024-04-26T21:58:54.634 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:54.634+0000 7f7546800640 1 -- 172.21.15.72:0/3081735041 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7540109be0 con 0x7f7540103ab0 2024-04-26T21:58:54.635 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:54.636+0000 7f753e000640 1 -- 172.21.15.72:0/3081735041 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f752c00f450 con 0x7f7540103ab0 2024-04-26T21:58:54.637 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:54.637+0000 7f753e000640 1 --2- 172.21.15.72:0/3081735041 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f75280a3c40 0x7f75280a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:54.637 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:54.637+0000 7f753e000640 1 -- 172.21.15.72:0/3081735041 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7f752c0d0910 con 0x7f7540103ab0 2024-04-26T21:58:54.637 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:54.637+0000 7f7545400640 1 --2- 172.21.15.72:0/3081735041 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f75280a3c40 0x7f75280a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:54.638 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:54.638+0000 7f7545400640 1 --2- 172.21.15.72:0/3081735041 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f75280a3c40 0x7f75280a6100 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f7534005fd0 tx=0x7f7534005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:58:54.641 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:54.641+0000 7f753e000640 1 -- 172.21.15.72:0/3081735041 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f752c014030 con 0x7f7540103ab0 2024-04-26T21:58:54.860 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:54.861+0000 7f7546800640 1 -- 172.21.15.72:0/3081735041 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f754002dd10 con 0x7f7540103ab0 2024-04-26T21:58:54.861 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:54.861+0000 7f753e000640 1 -- 172.21.15.72:0/3081735041 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v91) ==== 104+0+11 (secure 0 0 0) 0x7f752c09d0b0 con 0x7f7540103ab0 2024-04-26T21:58:54.861 INFO:teuthology.orchestra.run.smithi072.stdout:pg_num: 32 2024-04-26T21:58:54.865 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:54.864+0000 7f7546800640 1 -- 172.21.15.72:0/3081735041 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f75280a3c40 msgr2=0x7f75280a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:54.865 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:54.864+0000 7f7546800640 1 --2- 172.21.15.72:0/3081735041 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f75280a3c40 0x7f75280a6100 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f7534005fd0 tx=0x7f7534005850 comp rx=0 tx=0).stop 2024-04-26T21:58:54.865 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:54.865+0000 7f7546800640 1 -- 172.21.15.72:0/3081735041 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7540103ab0 msgr2=0x7f7540074d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:54.865 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:54.865+0000 7f7546800640 1 --2- 172.21.15.72:0/3081735041 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7540103ab0 0x7f7540074d20 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f752c00c920 tx=0x7f752c00cde0 comp rx=0 tx=0).stop 2024-04-26T21:58:54.865 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:54.865+0000 7f7546800640 1 -- 172.21.15.72:0/3081735041 shutdown_connections 2024-04-26T21:58:54.865 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:54.865+0000 7f7546800640 1 --2- 172.21.15.72:0/3081735041 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7540104470 0x7f7540075260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:54.865 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:54.865+0000 7f7546800640 1 --2- 172.21.15.72:0/3081735041 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7540103ab0 0x7f7540074d20 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:54.865 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:54.865+0000 7f7546800640 1 --2- 172.21.15.72:0/3081735041 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f75280a3c40 0x7f75280a6100 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:54.865 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:54.865+0000 7f7546800640 1 --2- 172.21.15.72:0/3081735041 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f75401028c0 0x7f75400766e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:54.865 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:54.865+0000 7f7546800640 1 -- 172.21.15.72:0/3081735041 >> 172.21.15.72:0/3081735041 conn(0x7f75400fde30 msgr2=0x7f7540106db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:58:54.865 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:54.865+0000 7f7546800640 1 -- 172.21.15.72:0/3081735041 shutdown_connections 2024-04-26T21:58:54.865 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:54.865+0000 7f7546800640 1 -- 172.21.15.72:0/3081735041 wait complete. 2024-04-26T21:58:54.884 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T21:58:55.076 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:55.076+0000 7f3315400640 1 -- 172.21.15.72:0/3474635403 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3310103b50 msgr2=0x7f3310103fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:55.077 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:55.076+0000 7f3315400640 1 --2- 172.21.15.72:0/3474635403 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3310103b50 0x7f3310103fd0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f32fc009900 tx=0x7f32fc02a3f0 comp rx=0 tx=0).stop 2024-04-26T21:58:55.077 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:55.076+0000 7f3315400640 1 -- 172.21.15.72:0/3474635403 shutdown_connections 2024-04-26T21:58:55.077 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:55.076+0000 7f3315400640 1 --2- 172.21.15.72:0/3474635403 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3310104510 0x7f331010ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:55.077 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:55.076+0000 7f3315400640 1 --2- 172.21.15.72:0/3474635403 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3310103b50 0x7f3310103fd0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:55.077 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:55.076+0000 7f3315400640 1 --2- 172.21.15.72:0/3474635403 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3310102960 0x7f3310102d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:55.077 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:55.076+0000 7f3315400640 1 -- 172.21.15.72:0/3474635403 >> 172.21.15.72:0/3474635403 conn(0x7f33100fdef0 msgr2=0x7f3310100310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:58:55.077 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:55.076+0000 7f3315400640 1 -- 172.21.15.72:0/3474635403 shutdown_connections 2024-04-26T21:58:55.077 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:55.076+0000 7f3315400640 1 -- 172.21.15.72:0/3474635403 wait complete. 2024-04-26T21:58:55.079 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:55.077+0000 7f3315400640 1 Processor -- start 2024-04-26T21:58:55.079 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:55.077+0000 7f3315400640 1 -- start start 2024-04-26T21:58:55.079 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:55.078+0000 7f3315400640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3310102960 0x7f331019ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:55.080 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:55.078+0000 7f3315400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3310103b50 0x7f331019b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:55.081 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:55.078+0000 7f3315400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3310104510 0x7f33101a23f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:55.081 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:55.078+0000 7f3315400640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f331010e430 con 0x7f3310104510 2024-04-26T21:58:55.081 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:55.078+0000 7f3315400640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f331010e2b0 con 0x7f3310102960 2024-04-26T21:58:55.081 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:55.078+0000 7f3315400640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f331010e5b0 con 0x7f3310103b50 2024-04-26T21:58:55.081 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:55.078+0000 7f330f200640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3310104510 0x7f33101a23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:55.082 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:55.078+0000 7f330f200640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3310104510 0x7f33101a23f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:57862/0 (socket says 172.21.15.72:57862) 2024-04-26T21:58:55.082 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:55.078+0000 7f330f200640 1 -- 172.21.15.72:0/3994328810 learned_addr learned my addr 172.21.15.72:0/3994328810 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:58:55.082 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:55.078+0000 7f330de00640 1 --2- 172.21.15.72:0/3994328810 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3310103b50 0x7f331019b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:55.083 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:55.078+0000 7f330e800640 1 --2- 172.21.15.72:0/3994328810 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3310102960 0x7f331019ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:55.083 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:55.079+0000 7f330f200640 1 -- 172.21.15.72:0/3994328810 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3310103b50 msgr2=0x7f331019b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:55.083 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:55.079+0000 7f330f200640 1 --2- 172.21.15.72:0/3994328810 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3310103b50 0x7f331019b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:55.083 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:55.079+0000 7f330f200640 1 -- 172.21.15.72:0/3994328810 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3310102960 msgr2=0x7f331019ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:55.083 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:55.079+0000 7f330f200640 1 --2- 172.21.15.72:0/3994328810 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3310102960 0x7f331019ae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:55.083 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:55.079+0000 7f330f200640 1 -- 172.21.15.72:0/3994328810 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33101a2a60 con 0x7f3310104510 2024-04-26T21:58:55.083 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:55.079+0000 7f330f200640 1 --2- 172.21.15.72:0/3994328810 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3310104510 0x7f33101a23f0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f330800b970 tx=0x7f330800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:58:55.084 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:55.079+0000 7f3307400640 1 -- 172.21.15.72:0/3994328810 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3308019070 con 0x7f3310104510 2024-04-26T21:58:55.084 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:55.079+0000 7f3307400640 1 -- 172.21.15.72:0/3994328810 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3308014440 con 0x7f3310104510 2024-04-26T21:58:55.084 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:55.079+0000 7f3307400640 1 -- 172.21.15.72:0/3994328810 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33080075d0 con 0x7f3310104510 2024-04-26T21:58:55.084 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:55.079+0000 7f3315400640 1 -- 172.21.15.72:0/3994328810 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33101a2d50 con 0x7f3310104510 2024-04-26T21:58:55.084 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:55.079+0000 7f3315400640 1 -- 172.21.15.72:0/3994328810 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33101a3240 con 0x7f3310104510 2024-04-26T21:58:55.088 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:55.080+0000 7f3315400640 1 -- 172.21.15.72:0/3994328810 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33101a34d0 con 0x7f3310104510 2024-04-26T21:58:55.089 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:55.082+0000 7f3307400640 1 -- 172.21.15.72:0/3994328810 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f330801c070 con 0x7f3310104510 2024-04-26T21:58:55.089 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:55.085+0000 7f3307400640 1 --2- 172.21.15.72:0/3994328810 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f32f40a3cc0 0x7f32f40a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:55.089 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:55.085+0000 7f3307400640 1 -- 172.21.15.72:0/3994328810 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7f33080cfa30 con 0x7f3310104510 2024-04-26T21:58:55.089 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:55.085+0000 7f330e800640 1 --2- 172.21.15.72:0/3994328810 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f32f40a3cc0 0x7f32f40a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:55.089 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:55.086+0000 7f330e800640 1 --2- 172.21.15.72:0/3994328810 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f32f40a3cc0 0x7f32f40a6180 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f33101039b0 tx=0x7f3300008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:58:55.091 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:55.089+0000 7f3307400640 1 -- 172.21.15.72:0/3994328810 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f330809bc90 con 0x7f3310104510 2024-04-26T21:58:55.291 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:55.291+0000 7f3315400640 1 -- 172.21.15.72:0/3994328810 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f33101a3870 con 0x7f3310104510 2024-04-26T21:58:55.292 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:55.292+0000 7f3307400640 1 -- 172.21.15.72:0/3994328810 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v91) ==== 100+0+11 (secure 0 0 0) 0x7f330809ff90 con 0x7f3310104510 2024-04-26T21:58:55.292 INFO:teuthology.orchestra.run.smithi072.stdout:pg_num: 64 2024-04-26T21:58:55.295 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:55.294+0000 7f3315400640 1 -- 172.21.15.72:0/3994328810 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f32f40a3cc0 msgr2=0x7f32f40a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:55.295 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:55.294+0000 7f3315400640 1 --2- 172.21.15.72:0/3994328810 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f32f40a3cc0 0x7f32f40a6180 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f33101039b0 tx=0x7f3300008040 comp rx=0 tx=0).stop 2024-04-26T21:58:55.295 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:55.294+0000 7f3315400640 1 -- 172.21.15.72:0/3994328810 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3310104510 msgr2=0x7f33101a23f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:55.295 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:55.294+0000 7f3315400640 1 --2- 172.21.15.72:0/3994328810 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3310104510 0x7f33101a23f0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f330800b970 tx=0x7f330800be30 comp rx=0 tx=0).stop 2024-04-26T21:58:55.295 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:55.294+0000 7f3315400640 1 -- 172.21.15.72:0/3994328810 shutdown_connections 2024-04-26T21:58:55.295 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:55.294+0000 7f3315400640 1 --2- 172.21.15.72:0/3994328810 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3310104510 0x7f33101a23f0 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:55.295 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:55.294+0000 7f3315400640 1 --2- 172.21.15.72:0/3994328810 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f32f40a3cc0 0x7f32f40a6180 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:55.295 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:55.294+0000 7f3315400640 1 --2- 172.21.15.72:0/3994328810 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3310103b50 0x7f331019b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:55.295 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:55.294+0000 7f3315400640 1 --2- 172.21.15.72:0/3994328810 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3310102960 0x7f331019ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:55.295 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:55.295+0000 7f3315400640 1 -- 172.21.15.72:0/3994328810 >> 172.21.15.72:0/3994328810 conn(0x7f33100fdef0 msgr2=0x7f3310109360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:58:55.295 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:55.295+0000 7f3315400640 1 -- 172.21.15.72:0/3994328810 shutdown_connections 2024-04-26T21:58:55.295 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:55.295+0000 7f3315400640 1 -- 172.21.15.72:0/3994328810 wait complete. 2024-04-26T21:58:55.309 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:58:55 smithi072 ceph-mon[28138]: pgmap v553: 97 pgs: 97 active+clean; 238 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 7.9 MiB/s wr, 1.75k op/s 2024-04-26T21:58:55.309 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:58:55 smithi072 ceph-mon[28138]: from='client.? 172.21.15.72:0/657593458' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T21:58:55.309 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:58:55 smithi072 ceph-mon[28138]: from='client.? 172.21.15.72:0/3081735041' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T21:58:55.312 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-04-26T21:58:55.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:58:55 smithi076 ceph-mon[31823]: pgmap v553: 97 pgs: 97 active+clean; 238 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 7.9 MiB/s wr, 1.75k op/s 2024-04-26T21:58:55.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:58:55 smithi076 ceph-mon[31823]: from='client.? 172.21.15.72:0/657593458' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T21:58:55.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:58:55 smithi076 ceph-mon[31823]: from='client.? 172.21.15.72:0/3081735041' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T21:58:55.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:58:55 smithi164 ceph-mon[31916]: pgmap v553: 97 pgs: 97 active+clean; 238 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 7.9 MiB/s wr, 1.75k op/s 2024-04-26T21:58:55.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:58:55 smithi164 ceph-mon[31916]: from='client.? 172.21.15.72:0/657593458' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T21:58:55.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:58:55 smithi164 ceph-mon[31916]: from='client.? 172.21.15.72:0/3081735041' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T21:58:55.494 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:55.493+0000 7f3267200640 1 -- 172.21.15.72:0/1411888598 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f32681049d0 msgr2=0x7f3268104df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:55.494 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:55.493+0000 7f3267200640 1 --2- 172.21.15.72:0/1411888598 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f32681049d0 0x7f3268104df0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f3258009900 tx=0x7f325802a3f0 comp rx=0 tx=0).stop 2024-04-26T21:58:55.494 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:55.493+0000 7f3267200640 1 -- 172.21.15.72:0/1411888598 shutdown_connections 2024-04-26T21:58:55.494 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:55.493+0000 7f3267200640 1 --2- 172.21.15.72:0/1411888598 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3268106580 0x7f326810ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:55.494 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:55.493+0000 7f3267200640 1 --2- 172.21.15.72:0/1411888598 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3268105bc0 0x7f3268106040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:55.494 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:55.493+0000 7f3267200640 1 --2- 172.21.15.72:0/1411888598 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f32681049d0 0x7f3268104df0 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:55.494 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:55.493+0000 7f3267200640 1 -- 172.21.15.72:0/1411888598 >> 172.21.15.72:0/1411888598 conn(0x7f32680fff40 msgr2=0x7f3268102380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:58:55.494 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:55.494+0000 7f3267200640 1 -- 172.21.15.72:0/1411888598 shutdown_connections 2024-04-26T21:58:55.494 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:55.494+0000 7f3267200640 1 -- 172.21.15.72:0/1411888598 wait complete. 2024-04-26T21:58:55.497 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:55.494+0000 7f3267200640 1 Processor -- start 2024-04-26T21:58:55.497 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:55.495+0000 7f3267200640 1 -- start start 2024-04-26T21:58:55.497 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:55.495+0000 7f3267200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f32681049d0 0x7f326819abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:55.497 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:55.495+0000 7f3267200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3268105bc0 0x7f326819b110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:55.497 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:55.495+0000 7f3267200640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3268106580 0x7f32681a2180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:55.497 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:55.495+0000 7f3267200640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f3268110450 con 0x7f3268106580 2024-04-26T21:58:55.497 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:55.495+0000 7f3267200640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f32681102d0 con 0x7f32681049d0 2024-04-26T21:58:55.497 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:55.495+0000 7f3267200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f32681105d0 con 0x7f3268105bc0 2024-04-26T21:58:55.497 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:55.495+0000 7f3266800640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3268106580 0x7f32681a2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:55.497 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:55.495+0000 7f3266800640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3268106580 0x7f32681a2180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:57892/0 (socket says 172.21.15.72:57892) 2024-04-26T21:58:55.497 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:55.495+0000 7f3266800640 1 -- 172.21.15.72:0/2492863644 learned_addr learned my addr 172.21.15.72:0/2492863644 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:58:55.497 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:55.496+0000 7f3265e00640 1 --2- 172.21.15.72:0/2492863644 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f32681049d0 0x7f326819abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:55.497 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:55.496+0000 7f3266800640 1 -- 172.21.15.72:0/2492863644 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3268105bc0 msgr2=0x7f326819b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:55.498 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:55.496+0000 7f3266800640 1 --2- 172.21.15.72:0/2492863644 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3268105bc0 0x7f326819b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:55.498 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:55.496+0000 7f3266800640 1 -- 172.21.15.72:0/2492863644 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f32681049d0 msgr2=0x7f326819abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:55.498 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:55.496+0000 7f3266800640 1 --2- 172.21.15.72:0/2492863644 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f32681049d0 0x7f326819abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:55.498 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:55.496+0000 7f3266800640 1 -- 172.21.15.72:0/2492863644 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32681a27f0 con 0x7f3268106580 2024-04-26T21:58:55.498 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:55.496+0000 7f3266800640 1 --2- 172.21.15.72:0/2492863644 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3268106580 0x7f32681a2180 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f326000cad0 tx=0x7f326000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:58:55.498 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:55.496+0000 7f325ea00640 1 -- 172.21.15.72:0/2492863644 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f326000ae20 con 0x7f3268106580 2024-04-26T21:58:55.498 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:55.496+0000 7f325ea00640 1 -- 172.21.15.72:0/2492863644 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3260005020 con 0x7f3268106580 2024-04-26T21:58:55.498 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:55.496+0000 7f3267200640 1 -- 172.21.15.72:0/2492863644 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32681a2ae0 con 0x7f3268106580 2024-04-26T21:58:55.498 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:55.496+0000 7f325ea00640 1 -- 172.21.15.72:0/2492863644 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f326000f6b0 con 0x7f3268106580 2024-04-26T21:58:55.498 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:55.497+0000 7f3267200640 1 -- 172.21.15.72:0/2492863644 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32681a3060 con 0x7f3268106580 2024-04-26T21:58:55.499 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:55.498+0000 7f3267200640 1 -- 172.21.15.72:0/2492863644 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f326810bdb0 con 0x7f3268106580 2024-04-26T21:58:55.501 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:55.501+0000 7f325ea00640 1 -- 172.21.15.72:0/2492863644 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f32600051d0 con 0x7f3268106580 2024-04-26T21:58:55.503 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:55.503+0000 7f325ea00640 1 --2- 172.21.15.72:0/2492863644 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f324c0a3c40 0x7f324c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:55.503 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:55.503+0000 7f325ea00640 1 -- 172.21.15.72:0/2492863644 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7f32600d0a60 con 0x7f3268106580 2024-04-26T21:58:55.503 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:55.503+0000 7f3265e00640 1 --2- 172.21.15.72:0/2492863644 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f324c0a3c40 0x7f324c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:55.504 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:55.504+0000 7f3265e00640 1 --2- 172.21.15.72:0/2492863644 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f324c0a3c40 0x7f324c0a6100 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f32580057c0 tx=0x7f3258000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:58:55.507 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:55.507+0000 7f325ea00640 1 -- 172.21.15.72:0/2492863644 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f3260014030 con 0x7f3268106580 2024-04-26T21:58:55.761 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:55.761+0000 7f3267200640 1 -- 172.21.15.72:0/2492863644 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7f326802dcd0 con 0x7f3268106580 2024-04-26T21:58:55.773 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:55.773+0000 7f325ea00640 1 -- 172.21.15.72:0/2492863644 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v17) ==== 81+0+745773 (secure 0 0 0) 0x7f326009d260 con 0x7f3268106580 2024-04-26T21:58:55.786 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T21:58:55.786 INFO:teuthology.orchestra.run.smithi072.stdout:{ 2024-04-26T21:58:55.786 INFO:teuthology.orchestra.run.smithi072.stdout: "epoch": 17, 2024-04-26T21:58:55.786 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:55.786 INFO:teuthology.orchestra.run.smithi072.stdout: "active_gid": 14152, 2024-04-26T21:58:55.787 INFO:teuthology.orchestra.run.smithi072.stdout: "active_name": "x", 2024-04-26T21:58:55.787 INFO:teuthology.orchestra.run.smithi072.stdout: "active_addrs": { 2024-04-26T21:58:55.787 INFO:teuthology.orchestra.run.smithi072.stdout: "addrvec": [ 2024-04-26T21:58:55.787 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-26T21:58:55.787 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "v2", 2024-04-26T21:58:55.787 INFO:teuthology.orchestra.run.smithi072.stdout: "addr": "172.21.15.72:6800", 2024-04-26T21:58:55.787 INFO:teuthology.orchestra.run.smithi072.stdout: "nonce": 588150172 2024-04-26T21:58:55.787 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.787 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-26T21:58:55.787 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "v1", 2024-04-26T21:58:55.787 INFO:teuthology.orchestra.run.smithi072.stdout: "addr": "172.21.15.72:6801", 2024-04-26T21:58:55.787 INFO:teuthology.orchestra.run.smithi072.stdout: "nonce": 588150172 2024-04-26T21:58:55.787 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:55.787 INFO:teuthology.orchestra.run.smithi072.stdout: ] 2024-04-26T21:58:55.787 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.788 INFO:teuthology.orchestra.run.smithi072.stdout: "active_addr": "172.21.15.72:6801/588150172", 2024-04-26T21:58:55.788 INFO:teuthology.orchestra.run.smithi072.stdout: "active_change": "2024-04-26T21:43:09.603138+0000", 2024-04-26T21:58:55.788 INFO:teuthology.orchestra.run.smithi072.stdout: "active_mgr_features": 4540701547738038271, 2024-04-26T21:58:55.788 INFO:teuthology.orchestra.run.smithi072.stdout: "available": true, 2024-04-26T21:58:55.788 INFO:teuthology.orchestra.run.smithi072.stdout: "standbys": [ 2024-04-26T21:58:55.788 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-26T21:58:55.788 INFO:teuthology.orchestra.run.smithi072.stdout: "gid": 24109, 2024-04-26T21:58:55.788 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "z", 2024-04-26T21:58:55.788 INFO:teuthology.orchestra.run.smithi072.stdout: "mgr_features": 4540701547738038271, 2024-04-26T21:58:55.788 INFO:teuthology.orchestra.run.smithi072.stdout: "available_modules": [ 2024-04-26T21:58:55.789 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-26T21:58:55.789 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "alerts", 2024-04-26T21:58:55.789 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-26T21:58:55.789 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-26T21:58:55.789 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-26T21:58:55.789 INFO:teuthology.orchestra.run.smithi072.stdout: "interval": { 2024-04-26T21:58:55.790 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "interval", 2024-04-26T21:58:55.790 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "secs", 2024-04-26T21:58:55.790 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.790 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:55.790 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "60", 2024-04-26T21:58:55.791 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:55.791 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.791 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:55.791 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "How frequently to reexamine health status", 2024-04-26T21:58:55.791 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:55.791 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.791 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.791 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.791 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-26T21:58:55.791 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-26T21:58:55.791 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:55.791 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.791 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:55.791 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:55.792 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:55.792 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.792 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:55.792 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:55.792 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:55.792 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:55.792 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:55.792 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:55.792 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:55.792 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:55.792 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:55.792 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:55.792 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.792 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.793 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.793 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-26T21:58:55.793 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-26T21:58:55.793 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:55.793 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.793 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:55.793 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:55.793 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:55.793 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.793 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:55.793 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:55.793 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:55.793 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.793 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.793 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.794 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-26T21:58:55.794 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-26T21:58:55.794 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:55.794 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.794 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:55.794 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-26T21:58:55.794 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:55.794 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.794 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:55.794 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:55.794 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:55.794 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:55.794 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:55.795 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:55.795 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:55.795 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:55.795 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:55.795 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:55.795 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.795 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.795 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.795 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-26T21:58:55.795 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-26T21:58:55.795 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:55.796 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.796 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:55.796 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:55.796 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:55.796 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.796 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:55.796 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:55.796 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:55.796 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.796 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.796 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.796 INFO:teuthology.orchestra.run.smithi072.stdout: "smtp_destination": { 2024-04-26T21:58:55.796 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "smtp_destination", 2024-04-26T21:58:55.796 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:55.796 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.797 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:55.797 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:55.797 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:55.797 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.797 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:55.797 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Email address to send alerts to", 2024-04-26T21:58:55.797 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:55.797 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.797 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.797 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.797 INFO:teuthology.orchestra.run.smithi072.stdout: "smtp_from_name": { 2024-04-26T21:58:55.797 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "smtp_from_name", 2024-04-26T21:58:55.797 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:55.797 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.797 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:55.798 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "Ceph", 2024-04-26T21:58:55.798 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:55.798 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.798 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:55.798 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Email From: name", 2024-04-26T21:58:55.798 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:55.798 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.798 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.798 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.798 INFO:teuthology.orchestra.run.smithi072.stdout: "smtp_host": { 2024-04-26T21:58:55.798 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "smtp_host", 2024-04-26T21:58:55.798 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:55.798 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.798 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:55.798 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:55.799 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:55.799 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.799 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:55.799 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "SMTP server", 2024-04-26T21:58:55.799 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:55.799 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.799 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.799 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.799 INFO:teuthology.orchestra.run.smithi072.stdout: "smtp_password": { 2024-04-26T21:58:55.799 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "smtp_password", 2024-04-26T21:58:55.799 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:55.800 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.800 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:55.800 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:55.800 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:55.800 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.800 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:55.800 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Password to authenticate with", 2024-04-26T21:58:55.800 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:55.800 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.800 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.800 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.800 INFO:teuthology.orchestra.run.smithi072.stdout: "smtp_port": { 2024-04-26T21:58:55.800 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "smtp_port", 2024-04-26T21:58:55.800 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-26T21:58:55.801 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.801 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:55.801 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "465", 2024-04-26T21:58:55.801 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:55.801 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.801 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:55.801 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "SMTP port", 2024-04-26T21:58:55.801 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:55.801 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.801 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.801 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.801 INFO:teuthology.orchestra.run.smithi072.stdout: "smtp_sender": { 2024-04-26T21:58:55.801 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "smtp_sender", 2024-04-26T21:58:55.801 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:55.801 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.802 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:55.802 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:55.802 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:55.802 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.802 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:55.802 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "SMTP envelope sender", 2024-04-26T21:58:55.802 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:55.802 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.802 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.802 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.802 INFO:teuthology.orchestra.run.smithi072.stdout: "smtp_ssl": { 2024-04-26T21:58:55.802 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "smtp_ssl", 2024-04-26T21:58:55.803 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:55.803 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.803 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:55.803 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-26T21:58:55.803 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:55.803 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.803 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:55.803 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-26T21:58:55.803 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:55.803 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.803 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.803 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.803 INFO:teuthology.orchestra.run.smithi072.stdout: "smtp_user": { 2024-04-26T21:58:55.803 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "smtp_user", 2024-04-26T21:58:55.803 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:55.804 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.804 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:55.804 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:55.804 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:55.804 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.804 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:55.804 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "User to authenticate as", 2024-04-26T21:58:55.804 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:55.804 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.804 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.804 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:55.805 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:55.805 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.805 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-26T21:58:55.805 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "balancer", 2024-04-26T21:58:55.805 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-26T21:58:55.805 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-26T21:58:55.805 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-26T21:58:55.805 INFO:teuthology.orchestra.run.smithi072.stdout: "active": { 2024-04-26T21:58:55.805 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "active", 2024-04-26T21:58:55.805 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:55.805 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.805 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:55.805 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-26T21:58:55.805 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:55.805 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.806 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:55.806 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "automatically balance PGs across cluster", 2024-04-26T21:58:55.806 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:55.806 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.806 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.806 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.806 INFO:teuthology.orchestra.run.smithi072.stdout: "begin_time": { 2024-04-26T21:58:55.806 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "begin_time", 2024-04-26T21:58:55.806 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:55.806 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.806 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:55.806 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "0000", 2024-04-26T21:58:55.806 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:55.806 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.807 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:55.807 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "beginning time of day to automatically balance", 2024-04-26T21:58:55.807 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-26T21:58:55.807 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.807 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.807 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.807 INFO:teuthology.orchestra.run.smithi072.stdout: "begin_weekday": { 2024-04-26T21:58:55.807 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "begin_weekday", 2024-04-26T21:58:55.807 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "uint", 2024-04-26T21:58:55.807 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.807 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:55.807 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "0", 2024-04-26T21:58:55.807 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "0", 2024-04-26T21:58:55.807 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "6", 2024-04-26T21:58:55.808 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:55.808 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-26T21:58:55.808 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-26T21:58:55.808 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.808 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.808 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.808 INFO:teuthology.orchestra.run.smithi072.stdout: "crush_compat_max_iterations": { 2024-04-26T21:58:55.808 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "crush_compat_max_iterations", 2024-04-26T21:58:55.808 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "uint", 2024-04-26T21:58:55.808 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.808 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:55.808 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "25", 2024-04-26T21:58:55.808 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "1", 2024-04-26T21:58:55.808 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "250", 2024-04-26T21:58:55.809 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:55.809 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-26T21:58:55.809 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:55.809 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.809 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.809 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.809 INFO:teuthology.orchestra.run.smithi072.stdout: "crush_compat_metrics": { 2024-04-26T21:58:55.809 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "crush_compat_metrics", 2024-04-26T21:58:55.809 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:55.809 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.809 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:55.809 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "pgs,objects,bytes", 2024-04-26T21:58:55.809 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:55.809 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.809 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:55.810 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-26T21:58:55.810 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-26T21:58:55.810 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.810 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.810 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.810 INFO:teuthology.orchestra.run.smithi072.stdout: "crush_compat_step": { 2024-04-26T21:58:55.810 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "crush_compat_step", 2024-04-26T21:58:55.810 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "float", 2024-04-26T21:58:55.810 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.810 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:55.810 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "0.5", 2024-04-26T21:58:55.810 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "0.001", 2024-04-26T21:58:55.810 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "0.999", 2024-04-26T21:58:55.811 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:55.811 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "aggressiveness of optimization", 2024-04-26T21:58:55.811 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-26T21:58:55.811 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.814 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.814 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.814 INFO:teuthology.orchestra.run.smithi072.stdout: "end_time": { 2024-04-26T21:58:55.814 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "end_time", 2024-04-26T21:58:55.814 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:55.814 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.814 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:55.814 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "2359", 2024-04-26T21:58:55.814 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:55.814 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.814 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:55.814 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "ending time of day to automatically balance", 2024-04-26T21:58:55.814 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-26T21:58:55.815 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.815 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.815 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.815 INFO:teuthology.orchestra.run.smithi072.stdout: "end_weekday": { 2024-04-26T21:58:55.815 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "end_weekday", 2024-04-26T21:58:55.815 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "uint", 2024-04-26T21:58:55.815 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.815 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:55.815 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "0", 2024-04-26T21:58:55.815 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "0", 2024-04-26T21:58:55.815 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "6", 2024-04-26T21:58:55.815 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:55.815 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-26T21:58:55.815 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-26T21:58:55.816 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.816 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.816 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.816 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-26T21:58:55.816 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-26T21:58:55.816 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:55.816 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.816 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:55.816 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:55.816 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:55.817 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.817 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:55.817 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:55.817 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:55.817 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:55.817 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:55.817 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:55.817 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:55.817 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:55.817 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:55.817 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:55.817 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.817 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.817 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.817 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-26T21:58:55.818 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-26T21:58:55.818 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:55.818 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.818 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:55.818 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:55.818 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:55.818 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.818 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:55.818 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:55.818 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:55.818 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.818 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.819 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.819 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-26T21:58:55.819 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-26T21:58:55.819 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:55.819 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.819 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:55.819 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-26T21:58:55.819 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:55.819 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.819 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:55.819 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:55.819 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:55.819 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:55.819 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:55.819 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:55.820 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:55.820 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:55.820 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:55.820 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:55.820 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.820 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.820 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.820 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-26T21:58:55.820 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-26T21:58:55.820 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:55.820 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.820 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:55.820 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:55.820 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:55.820 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.821 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:55.821 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:55.821 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:55.821 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.821 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.821 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.821 INFO:teuthology.orchestra.run.smithi072.stdout: "min_score": { 2024-04-26T21:58:55.821 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "min_score", 2024-04-26T21:58:55.821 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "float", 2024-04-26T21:58:55.821 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.821 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:55.821 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "0", 2024-04-26T21:58:55.821 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:55.821 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.821 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:55.822 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-26T21:58:55.822 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:55.822 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.822 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.822 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.822 INFO:teuthology.orchestra.run.smithi072.stdout: "mode": { 2024-04-26T21:58:55.822 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "mode", 2024-04-26T21:58:55.822 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:55.822 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.822 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:55.822 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "upmap", 2024-04-26T21:58:55.822 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:55.822 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.822 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:55.823 INFO:teuthology.orchestra.run.smithi072.stdout: "crush-compat", 2024-04-26T21:58:55.823 INFO:teuthology.orchestra.run.smithi072.stdout: "none", 2024-04-26T21:58:55.823 INFO:teuthology.orchestra.run.smithi072.stdout: "read", 2024-04-26T21:58:55.823 INFO:teuthology.orchestra.run.smithi072.stdout: "upmap", 2024-04-26T21:58:55.823 INFO:teuthology.orchestra.run.smithi072.stdout: "upmap-read" 2024-04-26T21:58:55.823 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:55.823 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Balancer mode", 2024-04-26T21:58:55.823 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:55.823 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.823 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.823 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.823 INFO:teuthology.orchestra.run.smithi072.stdout: "pool_ids": { 2024-04-26T21:58:55.823 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "pool_ids", 2024-04-26T21:58:55.823 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:55.824 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.824 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:55.824 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:55.824 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:55.824 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.824 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:55.824 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-26T21:58:55.824 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:55.824 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.824 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.824 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.824 INFO:teuthology.orchestra.run.smithi072.stdout: "sleep_interval": { 2024-04-26T21:58:55.824 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "sleep_interval", 2024-04-26T21:58:55.824 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "secs", 2024-04-26T21:58:55.824 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.824 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:55.825 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "60", 2024-04-26T21:58:55.825 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:55.825 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.825 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:55.825 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-26T21:58:55.825 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:55.825 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.825 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.825 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.825 INFO:teuthology.orchestra.run.smithi072.stdout: "upmap_max_deviation": { 2024-04-26T21:58:55.825 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "upmap_max_deviation", 2024-04-26T21:58:55.825 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-26T21:58:55.825 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.825 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:55.825 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "5", 2024-04-26T21:58:55.826 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "1", 2024-04-26T21:58:55.826 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.826 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:55.826 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-26T21:58:55.826 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-26T21:58:55.826 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.826 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.826 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.826 INFO:teuthology.orchestra.run.smithi072.stdout: "upmap_max_optimizations": { 2024-04-26T21:58:55.826 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "upmap_max_optimizations", 2024-04-26T21:58:55.826 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "uint", 2024-04-26T21:58:55.826 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.826 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:55.826 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "10", 2024-04-26T21:58:55.827 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:55.827 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.827 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:55.827 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-26T21:58:55.827 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:55.827 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.827 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.827 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:55.827 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:55.827 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.827 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-26T21:58:55.827 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "cephadm", 2024-04-26T21:58:55.827 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-26T21:58:55.827 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-26T21:58:55.827 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-26T21:58:55.827 INFO:teuthology.orchestra.run.smithi072.stdout: "agent_down_multiplier": { 2024-04-26T21:58:55.828 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "agent_down_multiplier", 2024-04-26T21:58:55.828 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "float", 2024-04-26T21:58:55.828 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.828 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:55.828 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "3.0", 2024-04-26T21:58:55.828 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:55.828 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.828 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:55.828 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-26T21:58:55.828 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:55.828 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.828 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.828 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.828 INFO:teuthology.orchestra.run.smithi072.stdout: "agent_refresh_rate": { 2024-04-26T21:58:55.828 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "agent_refresh_rate", 2024-04-26T21:58:55.829 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "secs", 2024-04-26T21:58:55.829 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.829 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:55.829 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "20", 2024-04-26T21:58:55.829 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:55.829 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.829 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:55.829 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-26T21:58:55.829 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:55.829 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.829 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.829 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.829 INFO:teuthology.orchestra.run.smithi072.stdout: "agent_starting_port": { 2024-04-26T21:58:55.829 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "agent_starting_port", 2024-04-26T21:58:55.829 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-26T21:58:55.829 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.830 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:55.830 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "4721", 2024-04-26T21:58:55.830 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:55.830 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.830 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:55.830 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-26T21:58:55.830 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:55.830 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.830 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.830 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.830 INFO:teuthology.orchestra.run.smithi072.stdout: "allow_ptrace": { 2024-04-26T21:58:55.830 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "allow_ptrace", 2024-04-26T21:58:55.830 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:55.831 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.831 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:55.831 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:55.831 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:55.831 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.831 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:55.831 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-26T21:58:55.831 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-26T21:58:55.831 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.831 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.831 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.831 INFO:teuthology.orchestra.run.smithi072.stdout: "autotune_interval": { 2024-04-26T21:58:55.831 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "autotune_interval", 2024-04-26T21:58:55.831 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "secs", 2024-04-26T21:58:55.831 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.832 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:55.832 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "600", 2024-04-26T21:58:55.832 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:55.832 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.832 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:55.832 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-26T21:58:55.832 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:55.832 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.832 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.832 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.832 INFO:teuthology.orchestra.run.smithi072.stdout: "autotune_memory_target_ratio": { 2024-04-26T21:58:55.832 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "autotune_memory_target_ratio", 2024-04-26T21:58:55.832 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "float", 2024-04-26T21:58:55.832 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.833 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:55.833 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "0.7", 2024-04-26T21:58:55.833 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:55.833 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.833 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:55.833 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-26T21:58:55.833 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:55.833 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.833 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.833 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.833 INFO:teuthology.orchestra.run.smithi072.stdout: "cephadm_log_destination": { 2024-04-26T21:58:55.833 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "cephadm_log_destination", 2024-04-26T21:58:55.833 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:55.833 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.833 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:55.833 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:55.834 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:55.834 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.834 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:55.834 INFO:teuthology.orchestra.run.smithi072.stdout: "file", 2024-04-26T21:58:55.834 INFO:teuthology.orchestra.run.smithi072.stdout: "file,syslog", 2024-04-26T21:58:55.834 INFO:teuthology.orchestra.run.smithi072.stdout: "syslog" 2024-04-26T21:58:55.834 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:55.834 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-26T21:58:55.834 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:55.834 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.834 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.834 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.834 INFO:teuthology.orchestra.run.smithi072.stdout: "cgroups_split": { 2024-04-26T21:58:55.834 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "cgroups_split", 2024-04-26T21:58:55.835 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:55.835 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.835 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:55.835 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-26T21:58:55.835 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:55.835 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.835 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:55.835 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-26T21:58:55.835 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:55.835 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.835 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.835 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.835 INFO:teuthology.orchestra.run.smithi072.stdout: "config_checks_enabled": { 2024-04-26T21:58:55.835 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "config_checks_enabled", 2024-04-26T21:58:55.835 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:55.836 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.836 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:55.836 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:55.836 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:55.836 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.836 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:55.836 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-26T21:58:55.836 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:55.836 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.836 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.836 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.836 INFO:teuthology.orchestra.run.smithi072.stdout: "config_dashboard": { 2024-04-26T21:58:55.836 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "config_dashboard", 2024-04-26T21:58:55.836 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:55.836 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.837 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:55.837 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-26T21:58:55.837 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:55.837 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.837 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:55.837 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-26T21:58:55.837 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:55.837 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.837 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.837 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.837 INFO:teuthology.orchestra.run.smithi072.stdout: "container_image_alertmanager": { 2024-04-26T21:58:55.837 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "container_image_alertmanager", 2024-04-26T21:58:55.837 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:55.837 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.838 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:55.838 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-26T21:58:55.838 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:55.838 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.838 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:55.838 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Prometheus container image", 2024-04-26T21:58:55.838 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:55.838 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.838 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.838 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.838 INFO:teuthology.orchestra.run.smithi072.stdout: "container_image_base": { 2024-04-26T21:58:55.838 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "container_image_base", 2024-04-26T21:58:55.838 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:55.838 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.838 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:55.839 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-26T21:58:55.839 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:55.839 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.839 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:55.839 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Container image name, without the tag", 2024-04-26T21:58:55.839 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:55.839 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.839 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.839 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.839 INFO:teuthology.orchestra.run.smithi072.stdout: "container_image_elasticsearch": { 2024-04-26T21:58:55.839 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "container_image_elasticsearch", 2024-04-26T21:58:55.839 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:55.839 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.839 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:55.840 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-26T21:58:55.840 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:55.840 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.840 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:55.840 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "elasticsearch container image", 2024-04-26T21:58:55.840 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:55.840 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.840 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.840 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.840 INFO:teuthology.orchestra.run.smithi072.stdout: "container_image_grafana": { 2024-04-26T21:58:55.840 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "container_image_grafana", 2024-04-26T21:58:55.840 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:55.840 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.840 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:55.840 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-26T21:58:55.841 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:55.841 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.841 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:55.841 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Prometheus container image", 2024-04-26T21:58:55.841 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:55.841 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.841 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.841 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.841 INFO:teuthology.orchestra.run.smithi072.stdout: "container_image_haproxy": { 2024-04-26T21:58:55.841 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "container_image_haproxy", 2024-04-26T21:58:55.841 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:55.841 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.841 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:55.841 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-26T21:58:55.842 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:55.842 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.842 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:55.842 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "HAproxy container image", 2024-04-26T21:58:55.842 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:55.842 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.842 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.842 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.842 INFO:teuthology.orchestra.run.smithi072.stdout: "container_image_jaeger_agent": { 2024-04-26T21:58:55.842 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "container_image_jaeger_agent", 2024-04-26T21:58:55.842 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:55.842 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.842 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:55.842 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-26T21:58:55.843 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:55.843 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.843 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:55.843 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Jaeger agent container image", 2024-04-26T21:58:55.843 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:55.843 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.843 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.843 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.843 INFO:teuthology.orchestra.run.smithi072.stdout: "container_image_jaeger_collector": { 2024-04-26T21:58:55.843 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "container_image_jaeger_collector", 2024-04-26T21:58:55.843 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:55.843 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.843 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:55.843 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-26T21:58:55.843 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:55.844 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.844 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:55.844 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Jaeger collector container image", 2024-04-26T21:58:55.844 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:55.844 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.844 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.844 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.844 INFO:teuthology.orchestra.run.smithi072.stdout: "container_image_jaeger_query": { 2024-04-26T21:58:55.844 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "container_image_jaeger_query", 2024-04-26T21:58:55.844 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:55.844 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.844 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:55.844 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-26T21:58:55.844 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:55.844 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.845 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:55.845 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Jaeger query container image", 2024-04-26T21:58:55.845 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:55.845 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.845 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.845 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.845 INFO:teuthology.orchestra.run.smithi072.stdout: "container_image_keepalived": { 2024-04-26T21:58:55.845 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "container_image_keepalived", 2024-04-26T21:58:55.845 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:55.845 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.845 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:55.845 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-26T21:58:55.845 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:55.845 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.846 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:55.846 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Keepalived container image", 2024-04-26T21:58:55.846 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:55.846 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.846 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.846 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.846 INFO:teuthology.orchestra.run.smithi072.stdout: "container_image_loki": { 2024-04-26T21:58:55.846 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "container_image_loki", 2024-04-26T21:58:55.846 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:55.846 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.846 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:55.846 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-26T21:58:55.846 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:55.846 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.847 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:55.847 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Loki container image", 2024-04-26T21:58:55.847 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:55.847 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.847 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.847 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.847 INFO:teuthology.orchestra.run.smithi072.stdout: "container_image_node_exporter": { 2024-04-26T21:58:55.847 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "container_image_node_exporter", 2024-04-26T21:58:55.847 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:55.847 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.847 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:55.847 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-26T21:58:55.847 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:55.847 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.847 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:55.847 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Prometheus container image", 2024-04-26T21:58:55.848 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:55.848 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.848 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.848 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.848 INFO:teuthology.orchestra.run.smithi072.stdout: "container_image_nvmeof": { 2024-04-26T21:58:55.848 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "container_image_nvmeof", 2024-04-26T21:58:55.848 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:55.848 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.848 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:55.848 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-26T21:58:55.848 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:55.848 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.848 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:55.848 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Nvme-of container image", 2024-04-26T21:58:55.848 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:55.849 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.849 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.849 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.849 INFO:teuthology.orchestra.run.smithi072.stdout: "container_image_prometheus": { 2024-04-26T21:58:55.849 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "container_image_prometheus", 2024-04-26T21:58:55.849 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:55.849 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.849 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:55.849 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-26T21:58:55.849 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:55.849 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.849 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:55.849 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Prometheus container image", 2024-04-26T21:58:55.849 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:55.849 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.849 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.850 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.850 INFO:teuthology.orchestra.run.smithi072.stdout: "container_image_promtail": { 2024-04-26T21:58:55.850 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "container_image_promtail", 2024-04-26T21:58:55.850 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:55.850 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.850 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:55.850 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-26T21:58:55.850 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:55.850 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.850 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:55.850 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Promtail container image", 2024-04-26T21:58:55.850 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:55.850 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.850 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.851 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.851 INFO:teuthology.orchestra.run.smithi072.stdout: "container_image_samba": { 2024-04-26T21:58:55.851 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "container_image_samba", 2024-04-26T21:58:55.851 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:55.851 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.851 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:55.851 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-26T21:58:55.851 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:55.851 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.851 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:55.851 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Samba/SMB container image", 2024-04-26T21:58:55.851 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:55.851 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.851 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.852 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.852 INFO:teuthology.orchestra.run.smithi072.stdout: "container_image_snmp_gateway": { 2024-04-26T21:58:55.852 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "container_image_snmp_gateway", 2024-04-26T21:58:55.852 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:55.852 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.852 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:55.852 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-26T21:58:55.852 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:55.852 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.852 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:55.852 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "SNMP Gateway container image", 2024-04-26T21:58:55.852 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:55.852 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.852 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.852 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.852 INFO:teuthology.orchestra.run.smithi072.stdout: "container_init": { 2024-04-26T21:58:55.853 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "container_init", 2024-04-26T21:58:55.853 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:55.853 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.853 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:55.853 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-26T21:58:55.853 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:55.853 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.853 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:55.853 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Run podman/docker with `--init`", 2024-04-26T21:58:55.853 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:55.853 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.853 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.853 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.854 INFO:teuthology.orchestra.run.smithi072.stdout: "daemon_cache_timeout": { 2024-04-26T21:58:55.854 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "daemon_cache_timeout", 2024-04-26T21:58:55.854 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "secs", 2024-04-26T21:58:55.854 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.854 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:55.854 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "600", 2024-04-26T21:58:55.854 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:55.854 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.854 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:55.854 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-26T21:58:55.854 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:55.854 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.854 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.854 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.854 INFO:teuthology.orchestra.run.smithi072.stdout: "default_cephadm_command_timeout": { 2024-04-26T21:58:55.855 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "default_cephadm_command_timeout", 2024-04-26T21:58:55.855 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-26T21:58:55.855 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.855 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:55.855 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "900", 2024-04-26T21:58:55.855 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:55.855 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.855 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:55.855 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-26T21:58:55.855 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:55.855 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.855 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.855 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.855 INFO:teuthology.orchestra.run.smithi072.stdout: "default_registry": { 2024-04-26T21:58:55.856 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "default_registry", 2024-04-26T21:58:55.856 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:55.856 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.856 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:55.856 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "docker.io", 2024-04-26T21:58:55.856 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:55.856 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.856 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:55.856 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-26T21:58:55.856 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:55.856 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.856 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.856 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.856 INFO:teuthology.orchestra.run.smithi072.stdout: "device_cache_timeout": { 2024-04-26T21:58:55.856 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "device_cache_timeout", 2024-04-26T21:58:55.857 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "secs", 2024-04-26T21:58:55.857 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.857 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:55.857 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "1800", 2024-04-26T21:58:55.857 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:55.857 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.857 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:55.857 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "seconds to cache device inventory", 2024-04-26T21:58:55.857 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:55.857 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.857 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.857 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.857 INFO:teuthology.orchestra.run.smithi072.stdout: "device_enhanced_scan": { 2024-04-26T21:58:55.857 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "device_enhanced_scan", 2024-04-26T21:58:55.858 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:55.858 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.858 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:55.858 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:55.858 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:55.858 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.858 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:55.858 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-26T21:58:55.858 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:55.858 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.858 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.858 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.858 INFO:teuthology.orchestra.run.smithi072.stdout: "facts_cache_timeout": { 2024-04-26T21:58:55.858 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "facts_cache_timeout", 2024-04-26T21:58:55.859 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "secs", 2024-04-26T21:58:55.859 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.859 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:55.859 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "60", 2024-04-26T21:58:55.859 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:55.859 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.859 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:55.859 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "seconds to cache host facts data", 2024-04-26T21:58:55.859 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:55.859 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.859 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.859 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.859 INFO:teuthology.orchestra.run.smithi072.stdout: "grafana_dashboards_path": { 2024-04-26T21:58:55.859 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "grafana_dashboards_path", 2024-04-26T21:58:55.859 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:55.859 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.860 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:55.860 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-26T21:58:55.860 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:55.860 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.860 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:55.860 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-26T21:58:55.860 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:55.860 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.860 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.860 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.860 INFO:teuthology.orchestra.run.smithi072.stdout: "host_check_interval": { 2024-04-26T21:58:55.860 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "host_check_interval", 2024-04-26T21:58:55.860 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "secs", 2024-04-26T21:58:55.860 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.861 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:55.861 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "600", 2024-04-26T21:58:55.861 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:55.861 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.861 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:55.861 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "how frequently to perform a host check", 2024-04-26T21:58:55.861 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:55.861 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.861 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.861 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.861 INFO:teuthology.orchestra.run.smithi072.stdout: "hw_monitoring": { 2024-04-26T21:58:55.861 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "hw_monitoring", 2024-04-26T21:58:55.861 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:55.861 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.861 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:55.862 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:55.862 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:55.862 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.862 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:55.862 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-26T21:58:55.862 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:55.862 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.862 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.862 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.862 INFO:teuthology.orchestra.run.smithi072.stdout: "inventory_list_all": { 2024-04-26T21:58:55.862 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "inventory_list_all", 2024-04-26T21:58:55.862 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:55.862 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.862 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:55.862 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:55.863 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:55.863 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.863 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:55.863 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-26T21:58:55.863 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:55.863 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.863 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.863 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.863 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-26T21:58:55.863 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-26T21:58:55.863 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:55.863 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.863 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:55.863 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:55.863 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:55.864 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.864 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:55.864 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:55.864 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:55.864 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:55.864 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:55.864 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:55.864 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:55.864 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:55.864 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:55.864 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:55.864 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.864 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.864 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.864 INFO:teuthology.orchestra.run.smithi072.stdout: "log_refresh_metadata": { 2024-04-26T21:58:55.864 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_refresh_metadata", 2024-04-26T21:58:55.865 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:55.865 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.865 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:55.865 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:55.865 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:55.865 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.865 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:55.865 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-26T21:58:55.865 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:55.865 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.865 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.865 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.865 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-26T21:58:55.865 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-26T21:58:55.866 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:55.866 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.866 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:55.866 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-26T21:58:55.866 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:55.866 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.866 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:55.866 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-26T21:58:55.866 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:55.866 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.866 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.866 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.866 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-26T21:58:55.866 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-26T21:58:55.867 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:55.867 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.867 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:55.867 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-26T21:58:55.867 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:55.867 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.867 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:55.867 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:55.867 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:55.867 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:55.867 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:55.867 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:55.867 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:55.867 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:55.867 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:55.868 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:55.868 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.868 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.868 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.868 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-26T21:58:55.868 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-26T21:58:55.868 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:55.868 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.868 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:55.868 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:55.868 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:55.868 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.868 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:55.868 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:55.868 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:55.868 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.869 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.869 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.869 INFO:teuthology.orchestra.run.smithi072.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-26T21:58:55.869 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-26T21:58:55.869 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:55.869 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.869 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:55.869 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:55.869 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:55.869 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.869 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:55.869 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-26T21:58:55.869 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:55.869 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.869 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.870 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.870 INFO:teuthology.orchestra.run.smithi072.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-26T21:58:55.870 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-26T21:58:55.870 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:55.870 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.870 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:55.870 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "*", 2024-04-26T21:58:55.870 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:55.870 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.870 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:55.870 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-26T21:58:55.870 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:55.870 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.870 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.871 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.871 INFO:teuthology.orchestra.run.smithi072.stdout: "max_count_per_host": { 2024-04-26T21:58:55.871 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "max_count_per_host", 2024-04-26T21:58:55.871 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-26T21:58:55.871 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.871 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:55.871 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "10", 2024-04-26T21:58:55.871 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:55.871 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.871 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:55.871 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "max number of daemons per service per host", 2024-04-26T21:58:55.871 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:55.871 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.871 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.871 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.871 INFO:teuthology.orchestra.run.smithi072.stdout: "max_osd_draining_count": { 2024-04-26T21:58:55.872 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "max_osd_draining_count", 2024-04-26T21:58:55.872 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-26T21:58:55.872 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.872 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:55.872 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "10", 2024-04-26T21:58:55.872 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:55.872 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.872 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:55.872 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-26T21:58:55.872 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:55.872 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.872 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.872 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.872 INFO:teuthology.orchestra.run.smithi072.stdout: "migration_current": { 2024-04-26T21:58:55.872 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "migration_current", 2024-04-26T21:58:55.873 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-26T21:58:55.873 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.873 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:55.873 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:55.873 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:55.873 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.873 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:55.873 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "internal - do not modify", 2024-04-26T21:58:55.873 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:55.873 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.873 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.873 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.873 INFO:teuthology.orchestra.run.smithi072.stdout: "mode": { 2024-04-26T21:58:55.873 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "mode", 2024-04-26T21:58:55.873 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:55.874 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.874 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:55.874 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "root", 2024-04-26T21:58:55.874 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:55.874 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.874 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:55.874 INFO:teuthology.orchestra.run.smithi072.stdout: "cephadm-package", 2024-04-26T21:58:55.874 INFO:teuthology.orchestra.run.smithi072.stdout: "root" 2024-04-26T21:58:55.874 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:55.874 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "mode for remote execution of cephadm", 2024-04-26T21:58:55.874 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:55.874 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.874 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.874 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.874 INFO:teuthology.orchestra.run.smithi072.stdout: "oob_default_addr": { 2024-04-26T21:58:55.875 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "oob_default_addr", 2024-04-26T21:58:55.875 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:55.875 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.875 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:55.875 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "169.254.1.1", 2024-04-26T21:58:55.875 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:55.875 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.875 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:55.875 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-26T21:58:55.875 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:55.875 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.875 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.875 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.875 INFO:teuthology.orchestra.run.smithi072.stdout: "prometheus_alerts_path": { 2024-04-26T21:58:55.875 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "prometheus_alerts_path", 2024-04-26T21:58:55.875 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:55.875 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.876 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:55.876 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-26T21:58:55.876 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:55.876 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.876 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:55.876 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-26T21:58:55.876 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:55.876 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.876 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.876 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.876 INFO:teuthology.orchestra.run.smithi072.stdout: "registry_insecure": { 2024-04-26T21:58:55.876 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "registry_insecure", 2024-04-26T21:58:55.876 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:55.876 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.877 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:55.877 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:55.877 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:55.877 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.877 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:55.877 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-26T21:58:55.877 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:55.877 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.877 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.877 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.877 INFO:teuthology.orchestra.run.smithi072.stdout: "registry_password": { 2024-04-26T21:58:55.877 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "registry_password", 2024-04-26T21:58:55.877 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:55.877 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.878 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:55.878 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:55.878 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:55.878 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.878 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:55.878 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-26T21:58:55.878 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:55.878 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.878 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.878 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.878 INFO:teuthology.orchestra.run.smithi072.stdout: "registry_url": { 2024-04-26T21:58:55.878 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "registry_url", 2024-04-26T21:58:55.878 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:55.878 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.878 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:55.879 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:55.879 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:55.879 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.879 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:55.879 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-26T21:58:55.879 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:55.879 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.879 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.879 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.879 INFO:teuthology.orchestra.run.smithi072.stdout: "registry_username": { 2024-04-26T21:58:55.879 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "registry_username", 2024-04-26T21:58:55.879 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:55.879 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.879 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:55.879 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:55.880 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:55.880 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.880 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:55.880 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-26T21:58:55.880 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:55.880 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.880 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.880 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.880 INFO:teuthology.orchestra.run.smithi072.stdout: "secure_monitoring_stack": { 2024-04-26T21:58:55.880 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "secure_monitoring_stack", 2024-04-26T21:58:55.880 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:55.880 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.880 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:55.880 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:55.880 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:55.881 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.881 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:55.881 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-26T21:58:55.881 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:55.881 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.881 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.881 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.881 INFO:teuthology.orchestra.run.smithi072.stdout: "service_discovery_port": { 2024-04-26T21:58:55.881 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "service_discovery_port", 2024-04-26T21:58:55.881 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-26T21:58:55.881 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.881 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:55.881 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "8765", 2024-04-26T21:58:55.881 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:55.881 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.882 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:55.882 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "cephadm service discovery port", 2024-04-26T21:58:55.882 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:55.882 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.882 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.882 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.882 INFO:teuthology.orchestra.run.smithi072.stdout: "ssh_config_file": { 2024-04-26T21:58:55.882 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "ssh_config_file", 2024-04-26T21:58:55.882 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:55.882 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.882 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:55.882 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:55.882 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:55.882 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.883 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:55.883 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-26T21:58:55.883 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:55.883 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.883 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.883 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.883 INFO:teuthology.orchestra.run.smithi072.stdout: "use_agent": { 2024-04-26T21:58:55.883 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "use_agent", 2024-04-26T21:58:55.883 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:55.883 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.883 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:55.883 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:55.883 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:55.883 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.884 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:55.884 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-26T21:58:55.884 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:55.884 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.884 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.884 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.884 INFO:teuthology.orchestra.run.smithi072.stdout: "use_repo_digest": { 2024-04-26T21:58:55.884 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "use_repo_digest", 2024-04-26T21:58:55.884 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:55.884 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.884 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:55.884 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-26T21:58:55.884 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:55.885 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.885 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:55.885 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-26T21:58:55.885 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:55.885 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.885 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.885 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.885 INFO:teuthology.orchestra.run.smithi072.stdout: "warn_on_failed_host_check": { 2024-04-26T21:58:55.885 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "warn_on_failed_host_check", 2024-04-26T21:58:55.885 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:55.885 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.885 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:55.885 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-26T21:58:55.885 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:55.885 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.885 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:55.886 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "raise a health warning if the host check fails", 2024-04-26T21:58:55.886 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:55.886 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.886 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.886 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.886 INFO:teuthology.orchestra.run.smithi072.stdout: "warn_on_stray_daemons": { 2024-04-26T21:58:55.886 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "warn_on_stray_daemons", 2024-04-26T21:58:55.886 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:55.886 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.886 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:55.886 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-26T21:58:55.886 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:55.886 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.886 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:55.887 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-26T21:58:55.887 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:55.887 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.887 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.887 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.887 INFO:teuthology.orchestra.run.smithi072.stdout: "warn_on_stray_hosts": { 2024-04-26T21:58:55.887 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "warn_on_stray_hosts", 2024-04-26T21:58:55.887 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:55.887 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.887 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:55.887 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-26T21:58:55.887 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:55.887 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.887 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:55.888 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-26T21:58:55.888 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:55.888 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.888 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.888 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:55.888 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:55.888 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.888 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-26T21:58:55.888 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "crash", 2024-04-26T21:58:55.888 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-26T21:58:55.888 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-26T21:58:55.888 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-26T21:58:55.888 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-26T21:58:55.888 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-26T21:58:55.888 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:55.889 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.889 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:55.889 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:55.889 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:55.889 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.889 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:55.889 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:55.889 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:55.889 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:55.889 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:55.889 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:55.889 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:55.889 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:55.889 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:55.889 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:55.890 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.890 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.890 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.890 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-26T21:58:55.890 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-26T21:58:55.890 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:55.890 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.890 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:55.890 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:55.890 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:55.890 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.890 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:55.890 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:55.890 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:55.890 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.891 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.891 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.891 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-26T21:58:55.891 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-26T21:58:55.891 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:55.891 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.891 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:55.891 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-26T21:58:55.891 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:55.891 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.891 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:55.891 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:55.891 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:55.891 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:55.891 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:55.892 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:55.892 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:55.892 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:55.892 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:55.892 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:55.892 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.892 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.892 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.892 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-26T21:58:55.892 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-26T21:58:55.892 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:55.892 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.892 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:55.892 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:55.892 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:55.893 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.893 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:55.893 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:55.893 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:55.893 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.893 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.893 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.893 INFO:teuthology.orchestra.run.smithi072.stdout: "retain_interval": { 2024-04-26T21:58:55.893 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "retain_interval", 2024-04-26T21:58:55.893 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "secs", 2024-04-26T21:58:55.893 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.893 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:55.893 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "31536000", 2024-04-26T21:58:55.893 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:55.893 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.893 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:55.894 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-26T21:58:55.894 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:55.894 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.894 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.894 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.894 INFO:teuthology.orchestra.run.smithi072.stdout: "warn_recent_interval": { 2024-04-26T21:58:55.894 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "warn_recent_interval", 2024-04-26T21:58:55.894 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "secs", 2024-04-26T21:58:55.894 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.894 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:55.894 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "1209600", 2024-04-26T21:58:55.894 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:55.894 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.894 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:55.895 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-26T21:58:55.895 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:55.895 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.895 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.895 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:55.895 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:55.895 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.895 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-26T21:58:55.895 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "dashboard", 2024-04-26T21:58:55.895 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-26T21:58:55.895 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-26T21:58:55.895 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-26T21:58:55.895 INFO:teuthology.orchestra.run.smithi072.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-26T21:58:55.895 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-26T21:58:55.895 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-26T21:58:55.896 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.896 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:55.896 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "10", 2024-04-26T21:58:55.896 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:55.896 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.896 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:55.896 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:55.896 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:55.896 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.896 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.896 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.896 INFO:teuthology.orchestra.run.smithi072.stdout: "ALERTMANAGER_API_HOST": { 2024-04-26T21:58:55.896 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-26T21:58:55.896 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:55.896 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.896 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:55.897 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:55.897 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:55.897 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.897 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:55.897 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:55.897 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:55.897 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.897 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.897 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.897 INFO:teuthology.orchestra.run.smithi072.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-26T21:58:55.897 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-26T21:58:55.897 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:55.897 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.897 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:55.897 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-26T21:58:55.898 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:55.898 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.898 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:55.898 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:55.898 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:55.898 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.898 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.898 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.898 INFO:teuthology.orchestra.run.smithi072.stdout: "AUDIT_API_ENABLED": { 2024-04-26T21:58:55.898 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "AUDIT_API_ENABLED", 2024-04-26T21:58:55.898 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:55.898 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.898 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:55.899 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:55.899 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:55.899 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.899 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:55.899 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:55.899 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:55.899 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.899 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.899 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.899 INFO:teuthology.orchestra.run.smithi072.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-26T21:58:55.899 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-26T21:58:55.899 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:55.899 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.899 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:55.899 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-26T21:58:55.899 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:55.900 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.900 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:55.900 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:55.900 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:55.900 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.900 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.900 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.900 INFO:teuthology.orchestra.run.smithi072.stdout: "ENABLE_BROWSABLE_API": { 2024-04-26T21:58:55.900 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-26T21:58:55.900 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:55.900 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.900 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:55.900 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-26T21:58:55.900 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:55.901 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.901 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:55.901 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:55.901 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:55.901 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.901 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.901 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.901 INFO:teuthology.orchestra.run.smithi072.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-26T21:58:55.901 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-26T21:58:55.901 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:55.901 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.901 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:55.901 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-26T21:58:55.902 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:55.902 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.902 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:55.902 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:55.902 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:55.902 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.902 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.902 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.902 INFO:teuthology.orchestra.run.smithi072.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-26T21:58:55.902 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-26T21:58:55.902 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:55.902 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.902 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:55.902 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-26T21:58:55.903 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:55.903 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.903 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:55.903 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:55.903 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:55.903 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.903 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.903 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.903 INFO:teuthology.orchestra.run.smithi072.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-26T21:58:55.903 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-26T21:58:55.903 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:55.903 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.903 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:55.903 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-26T21:58:55.903 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:55.903 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.904 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:55.904 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:55.904 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:55.904 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.904 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.904 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.904 INFO:teuthology.orchestra.run.smithi072.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-26T21:58:55.904 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-26T21:58:55.904 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:55.904 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.904 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:55.904 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-26T21:58:55.904 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:55.904 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.904 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:55.904 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:55.905 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:55.905 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.905 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.905 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.905 INFO:teuthology.orchestra.run.smithi072.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-26T21:58:55.905 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-26T21:58:55.905 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:55.905 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.905 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:55.905 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-26T21:58:55.905 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:55.905 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.905 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:55.905 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:55.905 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:55.906 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.906 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.906 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.906 INFO:teuthology.orchestra.run.smithi072.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-26T21:58:55.906 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-26T21:58:55.906 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:55.906 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.906 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:55.906 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-26T21:58:55.906 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:55.906 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.906 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:55.906 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:55.906 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:55.907 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.907 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.907 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.907 INFO:teuthology.orchestra.run.smithi072.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-26T21:58:55.907 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-26T21:58:55.907 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:55.907 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.907 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:55.907 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-26T21:58:55.907 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:55.907 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.907 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:55.907 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:55.907 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:55.907 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.907 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.908 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.908 INFO:teuthology.orchestra.run.smithi072.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-26T21:58:55.908 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-26T21:58:55.908 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:55.908 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.908 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:55.908 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:55.908 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:55.908 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.908 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:55.908 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:55.908 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:55.908 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.908 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.908 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.909 INFO:teuthology.orchestra.run.smithi072.stdout: "GRAFANA_API_PASSWORD": { 2024-04-26T21:58:55.909 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-26T21:58:55.909 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:55.909 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.909 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:55.909 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "admin", 2024-04-26T21:58:55.909 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:55.909 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.909 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:55.909 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:55.909 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:55.909 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.909 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.909 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.909 INFO:teuthology.orchestra.run.smithi072.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-26T21:58:55.909 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-26T21:58:55.910 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:55.910 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.910 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:55.910 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-26T21:58:55.910 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:55.910 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.910 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:55.910 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:55.910 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:55.910 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.910 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.910 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.910 INFO:teuthology.orchestra.run.smithi072.stdout: "GRAFANA_API_URL": { 2024-04-26T21:58:55.910 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "GRAFANA_API_URL", 2024-04-26T21:58:55.911 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:55.911 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.911 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:55.911 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:55.911 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:55.911 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.911 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:55.911 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:55.911 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:55.911 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.911 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.911 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.911 INFO:teuthology.orchestra.run.smithi072.stdout: "GRAFANA_API_USERNAME": { 2024-04-26T21:58:55.911 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-26T21:58:55.911 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:55.911 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.911 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:55.912 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "admin", 2024-04-26T21:58:55.912 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:55.912 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.912 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:55.912 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:55.912 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:55.912 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.912 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.912 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.912 INFO:teuthology.orchestra.run.smithi072.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-26T21:58:55.912 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-26T21:58:55.912 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:55.912 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.912 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:55.912 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:55.913 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:55.913 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.913 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:55.913 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:55.913 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:55.913 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.913 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.913 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.913 INFO:teuthology.orchestra.run.smithi072.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-26T21:58:55.913 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-26T21:58:55.913 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:55.913 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.913 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:55.913 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:55.914 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:55.914 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.914 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:55.914 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:55.914 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:55.914 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.914 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.914 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.914 INFO:teuthology.orchestra.run.smithi072.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-26T21:58:55.914 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-26T21:58:55.914 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:55.914 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.914 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:55.914 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-26T21:58:55.915 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:55.915 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.915 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:55.915 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:55.915 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:55.915 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.915 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.915 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.915 INFO:teuthology.orchestra.run.smithi072.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-26T21:58:55.915 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-26T21:58:55.915 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:55.915 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.915 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:55.915 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:55.916 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:55.916 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.916 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:55.916 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:55.916 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:55.916 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.916 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.916 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.916 INFO:teuthology.orchestra.run.smithi072.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-26T21:58:55.916 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-26T21:58:55.916 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:55.916 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.916 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:55.916 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "[]", 2024-04-26T21:58:55.917 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:55.917 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.917 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:55.917 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:55.917 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:55.917 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.917 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.917 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.917 INFO:teuthology.orchestra.run.smithi072.stdout: "MULTICLUSTER_CONFIG": { 2024-04-26T21:58:55.917 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-26T21:58:55.917 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:55.917 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.917 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:55.917 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "{}", 2024-04-26T21:58:55.917 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:55.918 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.918 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:55.918 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:55.918 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:55.918 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.918 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.918 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.918 INFO:teuthology.orchestra.run.smithi072.stdout: "PROMETHEUS_API_HOST": { 2024-04-26T21:58:55.918 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-26T21:58:55.918 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:55.918 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.918 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:55.918 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:55.918 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:55.919 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.919 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:55.919 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:55.919 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:55.919 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.919 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.919 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.919 INFO:teuthology.orchestra.run.smithi072.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-26T21:58:55.919 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-26T21:58:55.919 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:55.919 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.919 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:55.919 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-26T21:58:55.919 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:55.919 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.920 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:55.920 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:55.920 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:55.920 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.920 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.920 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.920 INFO:teuthology.orchestra.run.smithi072.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-26T21:58:55.920 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-26T21:58:55.920 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:55.920 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.920 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:55.920 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:55.920 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:55.920 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.920 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:55.921 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:55.921 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:55.921 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.921 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.921 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.921 INFO:teuthology.orchestra.run.smithi072.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-26T21:58:55.921 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-26T21:58:55.921 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:55.921 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.921 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:55.921 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:55.921 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:55.921 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.921 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:55.921 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:55.922 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:55.922 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.922 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.922 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.922 INFO:teuthology.orchestra.run.smithi072.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-26T21:58:55.922 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-26T21:58:55.922 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:55.922 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.922 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:55.922 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-26T21:58:55.922 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:55.922 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.922 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:55.922 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:55.922 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:55.923 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.923 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.923 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.923 INFO:teuthology.orchestra.run.smithi072.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-26T21:58:55.923 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-26T21:58:55.923 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:55.923 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.923 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:55.923 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-26T21:58:55.923 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:55.923 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.923 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:55.923 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:55.923 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:55.923 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.924 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.924 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.924 INFO:teuthology.orchestra.run.smithi072.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-26T21:58:55.924 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-26T21:58:55.924 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:55.924 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.924 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:55.924 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:55.924 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:55.924 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.924 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:55.924 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:55.924 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:55.924 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.924 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.925 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.925 INFO:teuthology.orchestra.run.smithi072.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-26T21:58:55.925 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-26T21:58:55.925 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:55.925 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.925 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:55.925 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:55.925 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:55.925 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.925 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:55.925 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:55.925 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:55.925 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.925 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.925 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.926 INFO:teuthology.orchestra.run.smithi072.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-26T21:58:55.926 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-26T21:58:55.926 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:55.926 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.926 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:55.926 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:55.926 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:55.926 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.926 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:55.926 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:55.926 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:55.926 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.926 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.927 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.927 INFO:teuthology.orchestra.run.smithi072.stdout: "PWD_POLICY_ENABLED": { 2024-04-26T21:58:55.927 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-26T21:58:55.927 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:55.927 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.927 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:55.927 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-26T21:58:55.927 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:55.927 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.927 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:55.927 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:55.927 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:55.927 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.927 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.927 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.928 INFO:teuthology.orchestra.run.smithi072.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-26T21:58:55.928 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-26T21:58:55.928 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:55.928 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.928 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:55.928 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-26T21:58:55.928 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:55.928 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.928 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:55.928 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:55.928 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:55.928 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.928 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.928 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.928 INFO:teuthology.orchestra.run.smithi072.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-26T21:58:55.929 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-26T21:58:55.929 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-26T21:58:55.929 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.929 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:55.929 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "10", 2024-04-26T21:58:55.929 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:55.929 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.929 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:55.929 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:55.929 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:55.929 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.929 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.929 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.929 INFO:teuthology.orchestra.run.smithi072.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-26T21:58:55.929 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-26T21:58:55.930 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-26T21:58:55.930 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.930 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:55.930 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "8", 2024-04-26T21:58:55.930 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:55.930 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.930 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:55.930 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:55.930 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:55.930 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.930 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.930 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.930 INFO:teuthology.orchestra.run.smithi072.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-26T21:58:55.930 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-26T21:58:55.931 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-26T21:58:55.931 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.931 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:55.931 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "45", 2024-04-26T21:58:55.931 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:55.931 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.931 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:55.931 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:55.931 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:55.931 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.931 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.931 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.931 INFO:teuthology.orchestra.run.smithi072.stdout: "RGW_API_ACCESS_KEY": { 2024-04-26T21:58:55.931 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-26T21:58:55.931 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:55.931 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.932 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:55.932 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:55.932 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:55.932 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.932 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:55.932 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:55.932 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:55.932 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.932 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.932 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.932 INFO:teuthology.orchestra.run.smithi072.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-26T21:58:55.932 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-26T21:58:55.932 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:55.932 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.932 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:55.933 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "admin", 2024-04-26T21:58:55.933 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:55.933 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.933 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:55.933 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:55.933 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:55.933 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.933 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.933 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.933 INFO:teuthology.orchestra.run.smithi072.stdout: "RGW_API_SECRET_KEY": { 2024-04-26T21:58:55.933 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-26T21:58:55.933 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:55.933 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.933 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:55.933 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:55.934 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:55.934 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.934 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:55.934 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:55.934 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:55.934 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.934 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.934 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.934 INFO:teuthology.orchestra.run.smithi072.stdout: "RGW_API_SSL_VERIFY": { 2024-04-26T21:58:55.934 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-26T21:58:55.934 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:55.934 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.934 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:55.934 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-26T21:58:55.935 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:55.935 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.935 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:55.935 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:55.935 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:55.935 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.935 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.935 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.935 INFO:teuthology.orchestra.run.smithi072.stdout: "UNSAFE_TLS_v1_2": { 2024-04-26T21:58:55.935 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-26T21:58:55.935 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:55.935 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.935 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:55.935 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:55.935 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:55.936 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.936 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:55.936 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:55.936 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:55.936 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.936 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.936 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.936 INFO:teuthology.orchestra.run.smithi072.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-26T21:58:55.936 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-26T21:58:55.936 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-26T21:58:55.936 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.936 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:55.936 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "0", 2024-04-26T21:58:55.936 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:55.936 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.937 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:55.937 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:55.937 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:55.937 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.937 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.937 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.937 INFO:teuthology.orchestra.run.smithi072.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-26T21:58:55.937 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-26T21:58:55.937 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-26T21:58:55.937 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.937 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:55.937 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "10", 2024-04-26T21:58:55.937 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:55.937 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.937 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:55.937 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:55.938 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:55.938 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.938 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.938 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.938 INFO:teuthology.orchestra.run.smithi072.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-26T21:58:55.938 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-26T21:58:55.938 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-26T21:58:55.938 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.938 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:55.938 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "5", 2024-04-26T21:58:55.938 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:55.938 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.938 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:55.939 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:55.939 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:55.939 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.939 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.939 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.939 INFO:teuthology.orchestra.run.smithi072.stdout: "cross_origin_url": { 2024-04-26T21:58:55.939 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "cross_origin_url", 2024-04-26T21:58:55.939 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:55.939 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.939 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:55.939 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:55.939 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:55.939 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.939 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:55.939 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:55.940 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:55.940 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.940 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.940 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.940 INFO:teuthology.orchestra.run.smithi072.stdout: "crt_file": { 2024-04-26T21:58:55.940 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "crt_file", 2024-04-26T21:58:55.940 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:55.940 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.940 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:55.940 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:55.940 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:55.940 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.940 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:55.940 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:55.940 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:55.941 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.941 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.941 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.941 INFO:teuthology.orchestra.run.smithi072.stdout: "debug": { 2024-04-26T21:58:55.941 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "debug", 2024-04-26T21:58:55.941 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:55.941 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.941 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:55.941 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:55.941 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:55.941 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.941 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:55.941 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Enable/disable debug options", 2024-04-26T21:58:55.941 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:55.941 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.942 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.942 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.942 INFO:teuthology.orchestra.run.smithi072.stdout: "jwt_token_ttl": { 2024-04-26T21:58:55.942 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "jwt_token_ttl", 2024-04-26T21:58:55.942 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-26T21:58:55.942 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.942 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:55.942 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "28800", 2024-04-26T21:58:55.942 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:55.942 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.942 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:55.942 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:55.942 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:55.942 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.943 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.943 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.943 INFO:teuthology.orchestra.run.smithi072.stdout: "key_file": { 2024-04-26T21:58:55.943 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "key_file", 2024-04-26T21:58:55.943 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:55.943 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.943 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:55.943 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:55.943 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:55.943 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.943 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:55.943 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:55.943 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:55.943 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.943 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.944 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.944 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-26T21:58:55.944 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-26T21:58:55.944 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:55.944 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.944 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:55.944 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:55.944 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:55.944 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.944 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:55.944 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:55.944 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:55.944 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:55.944 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:55.944 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:55.945 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:55.945 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:55.945 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:55.945 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:55.945 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.945 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.945 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.945 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-26T21:58:55.945 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-26T21:58:55.945 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:55.945 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.945 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:55.945 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:55.945 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:55.945 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.946 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:55.946 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:55.946 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:55.946 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.946 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.946 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.946 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-26T21:58:55.946 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-26T21:58:55.946 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:55.946 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.946 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:55.946 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-26T21:58:55.946 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:55.946 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.946 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:55.947 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:55.947 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:55.947 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:55.947 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:55.947 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:55.947 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:55.947 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:55.947 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:55.947 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:55.947 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.947 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.947 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.947 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-26T21:58:55.947 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-26T21:58:55.947 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:55.947 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.948 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:55.948 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:55.948 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:55.948 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.948 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:55.948 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:55.948 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:55.948 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.948 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.948 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.948 INFO:teuthology.orchestra.run.smithi072.stdout: "motd": { 2024-04-26T21:58:55.948 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "motd", 2024-04-26T21:58:55.948 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:55.948 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.949 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:55.949 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:55.949 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:55.949 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.949 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:55.949 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "The message of the day", 2024-04-26T21:58:55.949 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:55.949 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.949 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.949 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.949 INFO:teuthology.orchestra.run.smithi072.stdout: "redirect_resolve_ip_addr": { 2024-04-26T21:58:55.949 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "redirect_resolve_ip_addr", 2024-04-26T21:58:55.949 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:55.949 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.949 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:55.949 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:55.950 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:55.950 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.950 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:55.950 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:55.950 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:55.950 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.950 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.950 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.950 INFO:teuthology.orchestra.run.smithi072.stdout: "server_addr": { 2024-04-26T21:58:55.950 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "server_addr", 2024-04-26T21:58:55.950 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:55.950 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.950 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:55.950 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "::", 2024-04-26T21:58:55.951 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:55.951 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.951 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:55.951 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:55.951 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:55.951 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.951 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.951 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.951 INFO:teuthology.orchestra.run.smithi072.stdout: "server_port": { 2024-04-26T21:58:55.951 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "server_port", 2024-04-26T21:58:55.951 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-26T21:58:55.951 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.951 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:55.952 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "8080", 2024-04-26T21:58:55.952 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:55.952 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.952 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:55.952 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:55.952 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:55.952 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.952 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.952 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.952 INFO:teuthology.orchestra.run.smithi072.stdout: "ssl": { 2024-04-26T21:58:55.952 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "ssl", 2024-04-26T21:58:55.952 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:55.952 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.952 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:55.952 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-26T21:58:55.953 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:55.953 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.953 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:55.953 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:55.953 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:55.953 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.953 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.953 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.953 INFO:teuthology.orchestra.run.smithi072.stdout: "ssl_server_port": { 2024-04-26T21:58:55.953 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "ssl_server_port", 2024-04-26T21:58:55.953 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-26T21:58:55.953 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.953 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:55.953 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "8443", 2024-04-26T21:58:55.954 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:55.954 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.954 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:55.954 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:55.954 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:55.954 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.954 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.954 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.954 INFO:teuthology.orchestra.run.smithi072.stdout: "standby_behaviour": { 2024-04-26T21:58:55.954 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "standby_behaviour", 2024-04-26T21:58:55.954 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:55.954 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.954 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:55.954 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "redirect", 2024-04-26T21:58:55.954 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:55.955 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.955 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:55.955 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:55.955 INFO:teuthology.orchestra.run.smithi072.stdout: "redirect" 2024-04-26T21:58:55.955 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:55.955 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:55.955 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:55.955 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.955 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.955 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.955 INFO:teuthology.orchestra.run.smithi072.stdout: "standby_error_status_code": { 2024-04-26T21:58:55.955 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "standby_error_status_code", 2024-04-26T21:58:55.955 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-26T21:58:55.955 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.955 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:55.956 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "500", 2024-04-26T21:58:55.956 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "400", 2024-04-26T21:58:55.956 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "599", 2024-04-26T21:58:55.956 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:55.956 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:55.956 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:55.956 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.956 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.956 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.956 INFO:teuthology.orchestra.run.smithi072.stdout: "url_prefix": { 2024-04-26T21:58:55.956 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "url_prefix", 2024-04-26T21:58:55.956 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:55.956 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.956 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:55.956 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:55.957 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:55.957 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.957 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:55.957 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:55.957 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:55.957 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.957 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.957 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:55.957 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:55.957 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.957 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-26T21:58:55.957 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "devicehealth", 2024-04-26T21:58:55.957 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-26T21:58:55.957 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-26T21:58:55.957 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-26T21:58:55.958 INFO:teuthology.orchestra.run.smithi072.stdout: "enable_monitoring": { 2024-04-26T21:58:55.958 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "enable_monitoring", 2024-04-26T21:58:55.958 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:55.958 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.958 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:55.958 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-26T21:58:55.958 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:55.958 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.958 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:55.958 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "monitor device health metrics", 2024-04-26T21:58:55.958 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:55.958 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.958 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.958 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.959 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-26T21:58:55.959 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-26T21:58:55.959 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:55.959 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.959 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:55.959 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:55.959 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:55.959 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.959 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:55.959 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:55.959 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:55.959 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:55.959 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:55.959 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:55.959 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:55.960 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:55.960 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:55.960 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:55.960 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.960 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.960 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.960 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-26T21:58:55.960 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-26T21:58:55.960 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:55.960 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.960 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:55.960 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:55.960 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:55.960 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.960 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:55.961 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:55.961 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:55.961 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.961 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.961 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.961 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-26T21:58:55.961 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-26T21:58:55.961 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:55.961 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.961 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:55.961 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-26T21:58:55.961 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:55.961 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.961 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:55.961 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:55.962 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:55.962 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:55.962 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:55.962 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:55.962 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:55.962 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:55.962 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:55.962 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:55.962 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.962 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.962 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.962 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-26T21:58:55.962 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-26T21:58:55.962 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:55.962 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.963 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:55.963 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:55.963 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:55.963 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.963 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:55.963 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:55.963 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:55.963 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.963 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.963 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.963 INFO:teuthology.orchestra.run.smithi072.stdout: "mark_out_threshold": { 2024-04-26T21:58:55.963 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "mark_out_threshold", 2024-04-26T21:58:55.963 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "secs", 2024-04-26T21:58:55.963 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.963 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:55.964 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "2419200", 2024-04-26T21:58:55.964 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:55.964 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.964 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:55.964 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-26T21:58:55.964 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:55.964 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.964 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.964 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.964 INFO:teuthology.orchestra.run.smithi072.stdout: "pool_name": { 2024-04-26T21:58:55.964 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "pool_name", 2024-04-26T21:58:55.964 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:55.964 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.964 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:55.965 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "device_health_metrics", 2024-04-26T21:58:55.965 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:55.965 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.965 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:55.965 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-26T21:58:55.965 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:55.965 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.965 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.965 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.965 INFO:teuthology.orchestra.run.smithi072.stdout: "retention_period": { 2024-04-26T21:58:55.965 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "retention_period", 2024-04-26T21:58:55.965 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "secs", 2024-04-26T21:58:55.965 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.965 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:55.965 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "15552000", 2024-04-26T21:58:55.965 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:55.966 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.966 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:55.966 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "how long to retain device health metrics", 2024-04-26T21:58:55.966 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:55.966 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.966 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.966 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.966 INFO:teuthology.orchestra.run.smithi072.stdout: "scrape_frequency": { 2024-04-26T21:58:55.966 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "scrape_frequency", 2024-04-26T21:58:55.966 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "secs", 2024-04-26T21:58:55.966 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.966 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:55.966 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "86400", 2024-04-26T21:58:55.966 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:55.966 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.967 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:55.967 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-26T21:58:55.967 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:55.967 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.967 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.967 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.967 INFO:teuthology.orchestra.run.smithi072.stdout: "self_heal": { 2024-04-26T21:58:55.967 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "self_heal", 2024-04-26T21:58:55.967 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:55.967 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.967 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:55.967 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-26T21:58:55.967 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:55.967 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.967 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:55.967 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-26T21:58:55.968 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:55.968 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.968 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.968 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.968 INFO:teuthology.orchestra.run.smithi072.stdout: "sleep_interval": { 2024-04-26T21:58:55.968 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "sleep_interval", 2024-04-26T21:58:55.968 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "secs", 2024-04-26T21:58:55.968 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.968 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:55.968 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "600", 2024-04-26T21:58:55.968 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:55.968 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.968 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:55.968 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "how frequently to wake up and check device health", 2024-04-26T21:58:55.968 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:55.969 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.969 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.969 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.969 INFO:teuthology.orchestra.run.smithi072.stdout: "warn_threshold": { 2024-04-26T21:58:55.969 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "warn_threshold", 2024-04-26T21:58:55.969 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "secs", 2024-04-26T21:58:55.969 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.969 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:55.969 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "7257600", 2024-04-26T21:58:55.969 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:55.969 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.969 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:55.969 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-26T21:58:55.969 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:55.969 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.969 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.970 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:55.970 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:55.970 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.970 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-26T21:58:55.970 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "diskprediction_local", 2024-04-26T21:58:55.970 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-26T21:58:55.970 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-26T21:58:55.970 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-26T21:58:55.970 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-26T21:58:55.970 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-26T21:58:55.970 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:55.970 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.970 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:55.970 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:55.970 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:55.971 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.971 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:55.971 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:55.971 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:55.971 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:55.971 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:55.971 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:55.971 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:55.971 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:55.971 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:55.971 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:55.971 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.971 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.971 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.971 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-26T21:58:55.972 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-26T21:58:55.972 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:55.972 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.972 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:55.972 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:55.972 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:55.972 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.972 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:55.972 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:55.972 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:55.972 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.972 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.972 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.972 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-26T21:58:55.972 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-26T21:58:55.972 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:55.973 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.973 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:55.973 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-26T21:58:55.973 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:55.973 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.973 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:55.973 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:55.973 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:55.973 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:55.973 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:55.973 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:55.973 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:55.973 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:55.973 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:55.973 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:55.973 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.974 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.974 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.974 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-26T21:58:55.974 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-26T21:58:55.974 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:55.974 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.974 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:55.974 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:55.974 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:55.974 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.974 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:55.974 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:55.974 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:55.974 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.975 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.975 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.975 INFO:teuthology.orchestra.run.smithi072.stdout: "predict_interval": { 2024-04-26T21:58:55.975 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "predict_interval", 2024-04-26T21:58:55.975 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:55.975 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.975 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:55.975 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "86400", 2024-04-26T21:58:55.975 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:55.975 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.975 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:55.975 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:55.975 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:55.975 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.975 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.975 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.976 INFO:teuthology.orchestra.run.smithi072.stdout: "predictor_model": { 2024-04-26T21:58:55.976 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "predictor_model", 2024-04-26T21:58:55.976 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:55.976 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.976 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:55.976 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "prophetstor", 2024-04-26T21:58:55.976 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:55.976 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.976 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:55.976 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:55.976 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:55.976 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.976 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.977 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.977 INFO:teuthology.orchestra.run.smithi072.stdout: "sleep_interval": { 2024-04-26T21:58:55.977 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "sleep_interval", 2024-04-26T21:58:55.977 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:55.977 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.977 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:55.977 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "600", 2024-04-26T21:58:55.977 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:55.977 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.977 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:55.977 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:55.977 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:55.977 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.977 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.977 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:55.978 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:55.978 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.978 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-26T21:58:55.978 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "influx", 2024-04-26T21:58:55.978 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": false, 2024-04-26T21:58:55.978 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "influxdb python module not found", 2024-04-26T21:58:55.978 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-26T21:58:55.978 INFO:teuthology.orchestra.run.smithi072.stdout: "batch_size": { 2024-04-26T21:58:55.978 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "batch_size", 2024-04-26T21:58:55.978 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-26T21:58:55.978 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.978 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:55.978 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "5000", 2024-04-26T21:58:55.978 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:55.979 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.979 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:55.979 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-26T21:58:55.979 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:55.979 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.979 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.979 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.979 INFO:teuthology.orchestra.run.smithi072.stdout: "database": { 2024-04-26T21:58:55.979 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "database", 2024-04-26T21:58:55.979 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:55.979 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.979 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:55.979 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "ceph", 2024-04-26T21:58:55.979 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:55.980 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.980 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:55.980 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-26T21:58:55.980 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:55.980 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.980 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.980 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.980 INFO:teuthology.orchestra.run.smithi072.stdout: "hostname": { 2024-04-26T21:58:55.980 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "hostname", 2024-04-26T21:58:55.980 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:55.980 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.980 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:55.980 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:55.980 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:55.981 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.981 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:55.981 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "InfluxDB server hostname", 2024-04-26T21:58:55.981 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:55.981 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.981 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.981 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.981 INFO:teuthology.orchestra.run.smithi072.stdout: "interval": { 2024-04-26T21:58:55.981 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "interval", 2024-04-26T21:58:55.981 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "secs", 2024-04-26T21:58:55.981 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.981 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:55.981 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "30", 2024-04-26T21:58:55.981 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "5", 2024-04-26T21:58:55.981 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.981 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:55.982 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-26T21:58:55.982 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:55.982 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.982 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.982 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.982 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-26T21:58:55.982 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-26T21:58:55.982 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:55.982 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.982 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:55.982 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:55.982 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:55.982 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.982 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:55.983 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:55.983 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:55.983 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:55.983 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:55.983 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:55.983 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:55.983 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:55.983 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:55.983 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:55.983 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.983 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.983 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.983 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-26T21:58:55.983 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-26T21:58:55.984 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:55.984 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.984 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:55.984 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:55.984 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:55.984 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.984 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:55.984 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:55.984 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:55.984 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.984 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.984 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.984 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-26T21:58:55.984 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-26T21:58:55.984 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:55.985 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.985 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:55.985 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-26T21:58:55.985 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:55.985 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.985 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:55.985 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:55.985 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:55.985 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:55.985 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:55.985 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:55.985 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:55.985 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:55.985 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:55.985 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:55.985 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.986 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.986 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.986 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-26T21:58:55.986 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-26T21:58:55.986 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:55.986 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.986 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:55.986 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:55.986 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:55.986 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.986 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:55.986 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:55.986 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:55.986 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.987 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.987 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.987 INFO:teuthology.orchestra.run.smithi072.stdout: "password": { 2024-04-26T21:58:55.987 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "password", 2024-04-26T21:58:55.987 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:55.987 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.987 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:55.987 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:55.987 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:55.987 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.987 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:55.987 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "password of InfluxDB server user", 2024-04-26T21:58:55.987 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:55.987 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.987 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.988 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.988 INFO:teuthology.orchestra.run.smithi072.stdout: "port": { 2024-04-26T21:58:55.988 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "port", 2024-04-26T21:58:55.988 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-26T21:58:55.988 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.988 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:55.988 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "8086", 2024-04-26T21:58:55.988 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:55.988 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.988 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:55.988 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "InfluxDB server port", 2024-04-26T21:58:55.988 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:55.988 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.988 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.988 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.989 INFO:teuthology.orchestra.run.smithi072.stdout: "ssl": { 2024-04-26T21:58:55.989 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "ssl", 2024-04-26T21:58:55.989 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:55.989 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.989 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:55.989 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "false", 2024-04-26T21:58:55.989 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:55.989 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.989 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:55.989 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-26T21:58:55.989 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:55.989 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.989 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.990 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.990 INFO:teuthology.orchestra.run.smithi072.stdout: "threads": { 2024-04-26T21:58:55.990 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "threads", 2024-04-26T21:58:55.990 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-26T21:58:55.990 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.990 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:55.990 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "5", 2024-04-26T21:58:55.990 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "1", 2024-04-26T21:58:55.990 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "32", 2024-04-26T21:58:55.990 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:55.990 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-26T21:58:55.990 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:55.990 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.990 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.991 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.991 INFO:teuthology.orchestra.run.smithi072.stdout: "username": { 2024-04-26T21:58:55.991 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "username", 2024-04-26T21:58:55.991 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:55.991 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.991 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:55.991 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:55.991 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:55.991 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.991 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:55.991 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "username of InfluxDB server user", 2024-04-26T21:58:55.991 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:55.991 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.991 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.991 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.992 INFO:teuthology.orchestra.run.smithi072.stdout: "verify_ssl": { 2024-04-26T21:58:55.992 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "verify_ssl", 2024-04-26T21:58:55.992 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:55.992 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.992 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:55.992 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "true", 2024-04-26T21:58:55.992 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:55.992 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.992 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:55.992 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-26T21:58:55.992 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:55.992 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.992 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.992 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:55.992 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:55.993 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.993 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-26T21:58:55.993 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "insights", 2024-04-26T21:58:55.993 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-26T21:58:55.993 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-26T21:58:55.993 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-26T21:58:55.993 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-26T21:58:55.993 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-26T21:58:55.993 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:55.993 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.993 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:55.993 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:55.993 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:55.993 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.994 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:55.994 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:55.994 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:55.994 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:55.994 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:55.994 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:55.994 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:55.994 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:55.994 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:55.994 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:55.994 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.994 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.994 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.994 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-26T21:58:55.995 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-26T21:58:55.995 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:55.995 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.995 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:55.995 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:55.995 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:55.995 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.995 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:55.995 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:55.995 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:55.995 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.995 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.995 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.995 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-26T21:58:55.995 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-26T21:58:55.996 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:55.996 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.996 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:55.996 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-26T21:58:55.996 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:55.996 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.996 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:55.996 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:55.996 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:55.996 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:55.996 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:55.996 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:55.996 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:55.996 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:55.996 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:55.996 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:55.997 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.997 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.997 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.997 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-26T21:58:55.997 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-26T21:58:55.997 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:55.997 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.997 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:55.997 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:55.997 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:55.997 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.997 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:55.997 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:55.997 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:55.997 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.998 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:55.998 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:55.998 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:55.998 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:55.998 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-26T21:58:55.998 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "iostat", 2024-04-26T21:58:55.998 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-26T21:58:55.998 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-26T21:58:55.998 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-26T21:58:55.998 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-26T21:58:55.998 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-26T21:58:55.998 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:55.998 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:55.998 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:55.999 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:55.999 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:55.999 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:55.999 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:55.999 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:55.999 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:55.999 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:55.999 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:55.999 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:55.999 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:55.999 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:55.999 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:55.999 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:55.999 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:55.999 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.000 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.000 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-26T21:58:56.000 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-26T21:58:56.000 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.000 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.000 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.000 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.000 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.000 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.000 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.000 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.000 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.000 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.000 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.000 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.001 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-26T21:58:56.001 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-26T21:58:56.001 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.001 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.001 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.001 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-26T21:58:56.001 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.001 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.001 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:56.001 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:56.001 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:56.001 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:56.001 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:56.001 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:56.001 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:56.002 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.002 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.002 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.002 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.002 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.002 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.002 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-26T21:58:56.002 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-26T21:58:56.002 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.002 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.002 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.002 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.002 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.002 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.003 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.003 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.003 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.003 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.003 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.003 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:56.003 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:56.003 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.003 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-26T21:58:56.003 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "k8sevents", 2024-04-26T21:58:56.003 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-26T21:58:56.003 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-26T21:58:56.003 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-26T21:58:56.003 INFO:teuthology.orchestra.run.smithi072.stdout: "ceph_event_retention_days": { 2024-04-26T21:58:56.003 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "ceph_event_retention_days", 2024-04-26T21:58:56.004 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-26T21:58:56.004 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.004 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.004 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "7", 2024-04-26T21:58:56.004 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.004 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.004 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.004 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-26T21:58:56.004 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.004 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.004 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.004 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.004 INFO:teuthology.orchestra.run.smithi072.stdout: "config_check_secs": { 2024-04-26T21:58:56.004 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "config_check_secs", 2024-04-26T21:58:56.004 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-26T21:58:56.005 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.005 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.005 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "10", 2024-04-26T21:58:56.005 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "10", 2024-04-26T21:58:56.005 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.005 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.005 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-26T21:58:56.005 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.005 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.005 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.005 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.005 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-26T21:58:56.005 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-26T21:58:56.005 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.005 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.006 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.006 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.006 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.006 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.006 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:56.006 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:56.006 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:56.006 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:56.006 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:56.006 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:56.006 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:56.006 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.006 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.006 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.007 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.007 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.007 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.007 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-26T21:58:56.007 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-26T21:58:56.007 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.007 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.007 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.007 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.007 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.007 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.007 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.007 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.007 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.007 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.008 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.008 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.008 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-26T21:58:56.008 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-26T21:58:56.008 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.008 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.008 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.008 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-26T21:58:56.008 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.008 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.008 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:56.008 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:56.008 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:56.008 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:56.008 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:56.009 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:56.009 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:56.009 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.009 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.009 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.009 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.009 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.009 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.009 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-26T21:58:56.009 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-26T21:58:56.009 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.009 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.009 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.009 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.009 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.010 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.010 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.010 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.010 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.010 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.010 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.010 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:56.010 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:56.010 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.010 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-26T21:58:56.010 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "localpool", 2024-04-26T21:58:56.010 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-26T21:58:56.010 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-26T21:58:56.010 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-26T21:58:56.011 INFO:teuthology.orchestra.run.smithi072.stdout: "failure_domain": { 2024-04-26T21:58:56.011 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "failure_domain", 2024-04-26T21:58:56.011 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.011 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.011 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.011 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "host", 2024-04-26T21:58:56.011 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.011 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.011 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.011 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "failure domain for any created local pool", 2024-04-26T21:58:56.011 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-26T21:58:56.011 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.011 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.011 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.011 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-26T21:58:56.012 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-26T21:58:56.012 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.012 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.012 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.012 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.012 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.012 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.012 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:56.012 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:56.012 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:56.012 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:56.012 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:56.012 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:56.012 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:56.013 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.013 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.013 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.013 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.013 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.013 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.013 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-26T21:58:56.013 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-26T21:58:56.013 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.013 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.013 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.013 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.013 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.013 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.013 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.013 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.014 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.014 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.014 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.014 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.014 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-26T21:58:56.014 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-26T21:58:56.014 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.014 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.014 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.014 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-26T21:58:56.014 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.014 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.014 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:56.014 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:56.015 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:56.015 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:56.015 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:56.015 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:56.015 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:56.015 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.015 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.015 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.015 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.015 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.015 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.015 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-26T21:58:56.015 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-26T21:58:56.016 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.016 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.016 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.016 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.016 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.016 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.016 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.016 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.016 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.016 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.016 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.016 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.016 INFO:teuthology.orchestra.run.smithi072.stdout: "min_size": { 2024-04-26T21:58:56.016 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "min_size", 2024-04-26T21:58:56.016 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-26T21:58:56.017 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.017 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.017 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.017 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.017 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.017 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.017 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "default min_size for any created local pool", 2024-04-26T21:58:56.017 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-26T21:58:56.017 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.017 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.017 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.017 INFO:teuthology.orchestra.run.smithi072.stdout: "num_rep": { 2024-04-26T21:58:56.017 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "num_rep", 2024-04-26T21:58:56.017 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-26T21:58:56.017 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.017 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.018 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "3", 2024-04-26T21:58:56.018 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.018 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.018 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.018 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "default replica count for any created local pool", 2024-04-26T21:58:56.018 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.018 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.018 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.018 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.018 INFO:teuthology.orchestra.run.smithi072.stdout: "pg_num": { 2024-04-26T21:58:56.018 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "pg_num", 2024-04-26T21:58:56.018 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-26T21:58:56.018 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.019 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.019 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "128", 2024-04-26T21:58:56.019 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.019 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.019 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.019 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "default pg_num for any created local pool", 2024-04-26T21:58:56.019 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.019 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.019 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.019 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.019 INFO:teuthology.orchestra.run.smithi072.stdout: "prefix": { 2024-04-26T21:58:56.019 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "prefix", 2024-04-26T21:58:56.019 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.019 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.019 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.019 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.020 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.020 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.020 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.020 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "name prefix for any created local pool", 2024-04-26T21:58:56.020 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.020 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.020 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.020 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.020 INFO:teuthology.orchestra.run.smithi072.stdout: "subtree": { 2024-04-26T21:58:56.020 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "subtree", 2024-04-26T21:58:56.020 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.020 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.020 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.020 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "rack", 2024-04-26T21:58:56.020 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.020 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.021 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.021 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-26T21:58:56.021 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-26T21:58:56.021 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.021 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.021 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:56.021 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:56.021 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.021 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-26T21:58:56.021 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "mds_autoscaler", 2024-04-26T21:58:56.021 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-26T21:58:56.021 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-26T21:58:56.021 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-26T21:58:56.021 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-26T21:58:56.021 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-26T21:58:56.022 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.022 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.022 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.022 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.022 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.022 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.022 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:56.022 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:56.022 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:56.022 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:56.022 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:56.022 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:56.022 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:56.022 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.023 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.023 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.023 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.023 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.023 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.023 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-26T21:58:56.023 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-26T21:58:56.023 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.023 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.023 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.023 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.023 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.023 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.023 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.023 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.023 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.024 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.024 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.024 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.024 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-26T21:58:56.024 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-26T21:58:56.024 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.024 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.024 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.024 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-26T21:58:56.024 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.024 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.024 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:56.024 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:56.024 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:56.024 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:56.024 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:56.025 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:56.025 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:56.025 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.025 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.025 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.025 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.025 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.025 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.025 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-26T21:58:56.025 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-26T21:58:56.025 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.025 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.025 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.025 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.025 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.025 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.026 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.026 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.026 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.026 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.026 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.026 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:56.026 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:56.026 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.026 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-26T21:58:56.026 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "mirroring", 2024-04-26T21:58:56.026 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-26T21:58:56.026 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-26T21:58:56.026 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-26T21:58:56.026 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-26T21:58:56.026 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-26T21:58:56.027 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.027 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.027 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.027 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.027 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.027 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.027 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:56.027 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:56.027 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:56.027 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:56.027 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:56.027 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:56.027 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:56.027 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.027 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.028 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.028 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.028 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.028 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.028 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-26T21:58:56.028 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-26T21:58:56.028 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.028 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.028 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.028 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.028 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.028 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.028 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.028 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.029 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.029 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.029 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.029 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.029 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-26T21:58:56.029 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-26T21:58:56.029 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.029 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.029 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.029 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-26T21:58:56.029 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.029 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.029 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:56.029 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:56.029 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:56.029 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:56.030 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:56.030 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:56.030 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:56.030 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.030 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.030 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.030 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.030 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.030 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.030 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-26T21:58:56.030 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-26T21:58:56.030 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.030 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.030 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.031 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.031 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.031 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.031 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.031 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.031 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.031 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.031 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.031 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:56.031 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:56.031 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.031 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-26T21:58:56.031 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "nfs", 2024-04-26T21:58:56.031 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-26T21:58:56.032 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-26T21:58:56.032 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-26T21:58:56.032 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-26T21:58:56.032 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-26T21:58:56.032 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.032 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.032 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.032 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.032 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.032 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.032 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:56.032 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:56.032 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:56.032 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:56.032 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:56.032 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:56.033 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:56.033 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.033 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.033 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.033 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.033 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.033 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.033 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-26T21:58:56.033 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-26T21:58:56.033 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.033 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.033 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.033 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.033 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.034 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.034 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.034 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.034 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.034 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.034 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.034 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.034 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-26T21:58:56.034 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-26T21:58:56.034 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.034 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.034 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.034 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-26T21:58:56.034 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.035 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.035 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:56.035 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:56.035 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:56.035 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:56.035 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:56.035 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:56.035 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:56.035 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.035 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.035 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.035 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.035 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.035 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.035 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-26T21:58:56.036 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-26T21:58:56.036 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.036 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.036 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.036 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.036 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.036 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.036 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.036 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.036 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.036 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.036 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.036 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:56.036 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:56.036 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.037 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-26T21:58:56.037 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "orchestrator", 2024-04-26T21:58:56.037 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-26T21:58:56.037 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-26T21:58:56.037 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-26T21:58:56.037 INFO:teuthology.orchestra.run.smithi072.stdout: "fail_fs": { 2024-04-26T21:58:56.037 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "fail_fs", 2024-04-26T21:58:56.037 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.037 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.037 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.037 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.037 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.037 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.037 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.037 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-26T21:58:56.038 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.038 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.038 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.038 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.038 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-26T21:58:56.038 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-26T21:58:56.038 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.038 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.038 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.038 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.038 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.038 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.038 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:56.038 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:56.038 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:56.039 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:56.039 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:56.039 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:56.039 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:56.039 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.039 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.039 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.039 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.039 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.039 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.039 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-26T21:58:56.039 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-26T21:58:56.039 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.039 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.039 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.040 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.040 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.040 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.040 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.040 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.040 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.040 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.040 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.040 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.040 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-26T21:58:56.040 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-26T21:58:56.040 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.040 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.040 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.040 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-26T21:58:56.041 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.041 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.041 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:56.041 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:56.041 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:56.041 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:56.041 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:56.041 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:56.041 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:56.041 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.041 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.041 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.041 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.041 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.042 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.042 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-26T21:58:56.042 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-26T21:58:56.042 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.042 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.042 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.042 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.042 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.042 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.042 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.042 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.042 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.042 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.042 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.043 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.043 INFO:teuthology.orchestra.run.smithi072.stdout: "orchestrator": { 2024-04-26T21:58:56.043 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "orchestrator", 2024-04-26T21:58:56.043 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.043 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.043 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.043 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.043 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.043 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.043 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:56.043 INFO:teuthology.orchestra.run.smithi072.stdout: "cephadm", 2024-04-26T21:58:56.043 INFO:teuthology.orchestra.run.smithi072.stdout: "rook", 2024-04-26T21:58:56.043 INFO:teuthology.orchestra.run.smithi072.stdout: "test_orchestrator" 2024-04-26T21:58:56.043 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.043 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Orchestrator backend", 2024-04-26T21:58:56.044 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.044 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.044 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.044 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:56.044 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:56.044 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.044 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-26T21:58:56.044 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "osd_perf_query", 2024-04-26T21:58:56.044 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-26T21:58:56.044 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-26T21:58:56.044 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-26T21:58:56.044 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-26T21:58:56.044 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-26T21:58:56.044 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.044 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.045 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.045 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.045 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.045 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.045 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:56.045 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:56.045 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:56.045 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:56.045 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:56.045 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:56.045 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:56.045 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.045 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.045 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.046 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.046 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.046 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.046 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-26T21:58:56.046 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-26T21:58:56.046 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.046 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.046 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.046 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.046 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.046 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.046 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.046 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.046 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.047 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.047 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.047 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.047 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-26T21:58:56.047 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-26T21:58:56.047 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.047 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.047 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.047 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-26T21:58:56.047 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.047 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.047 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:56.047 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:56.047 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:56.047 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:56.048 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:56.048 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:56.048 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:56.048 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.048 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.048 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.048 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.048 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.048 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.048 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-26T21:58:56.048 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-26T21:58:56.048 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.048 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.048 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.048 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.049 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.049 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.049 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.049 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.049 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.049 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.049 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.049 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:56.049 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:56.049 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.049 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-26T21:58:56.049 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "osd_support", 2024-04-26T21:58:56.049 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-26T21:58:56.049 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-26T21:58:56.049 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-26T21:58:56.050 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-26T21:58:56.050 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-26T21:58:56.050 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.050 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.050 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.050 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.050 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.050 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.050 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:56.050 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:56.050 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:56.050 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:56.050 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:56.050 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:56.050 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:56.051 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.051 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.051 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.051 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.051 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.051 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.051 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-26T21:58:56.051 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-26T21:58:56.051 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.051 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.051 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.051 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.051 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.051 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.051 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.052 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.052 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.052 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.052 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.052 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.052 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-26T21:58:56.052 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-26T21:58:56.052 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.052 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.052 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.052 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-26T21:58:56.052 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.052 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.052 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:56.052 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:56.053 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:56.053 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:56.053 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:56.053 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:56.053 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:56.053 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.053 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.053 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.053 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.053 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.053 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.053 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-26T21:58:56.053 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-26T21:58:56.053 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.053 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.054 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.054 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.054 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.054 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.054 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.054 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.054 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.054 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.054 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.054 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:56.054 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:56.054 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.054 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-26T21:58:56.055 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "pg_autoscaler", 2024-04-26T21:58:56.055 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-26T21:58:56.055 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-26T21:58:56.055 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-26T21:58:56.055 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-26T21:58:56.055 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-26T21:58:56.055 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.055 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.055 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.055 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.055 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.055 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.055 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:56.055 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:56.055 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:56.055 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:56.056 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:56.056 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:56.056 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:56.056 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.056 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.056 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.056 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.056 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.056 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.056 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-26T21:58:56.056 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-26T21:58:56.056 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.056 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.056 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.056 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.056 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.057 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.057 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.057 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.057 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.057 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.057 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.057 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.057 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-26T21:58:56.057 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-26T21:58:56.057 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.057 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.057 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.057 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-26T21:58:56.057 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.057 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.058 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:56.058 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:56.058 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:56.058 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:56.058 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:56.058 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:56.058 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:56.058 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.058 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.058 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.058 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.058 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.058 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.058 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-26T21:58:56.059 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-26T21:58:56.059 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.059 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.059 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.059 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.059 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.059 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.059 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.059 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.059 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.059 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.059 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.059 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.059 INFO:teuthology.orchestra.run.smithi072.stdout: "sleep_interval": { 2024-04-26T21:58:56.059 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "sleep_interval", 2024-04-26T21:58:56.059 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "secs", 2024-04-26T21:58:56.060 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.060 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.060 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "60", 2024-04-26T21:58:56.060 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.060 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.060 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.060 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.060 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.060 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.060 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.060 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.060 INFO:teuthology.orchestra.run.smithi072.stdout: "threshold": { 2024-04-26T21:58:56.060 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "threshold", 2024-04-26T21:58:56.060 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "float", 2024-04-26T21:58:56.060 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.060 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.061 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "3.0", 2024-04-26T21:58:56.061 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "1.0", 2024-04-26T21:58:56.061 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.061 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.061 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "scaling threshold", 2024-04-26T21:58:56.061 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-26T21:58:56.061 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.061 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.061 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:56.061 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:56.061 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.061 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-26T21:58:56.061 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "progress", 2024-04-26T21:58:56.061 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-26T21:58:56.061 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-26T21:58:56.062 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-26T21:58:56.062 INFO:teuthology.orchestra.run.smithi072.stdout: "allow_pg_recovery_event": { 2024-04-26T21:58:56.062 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "allow_pg_recovery_event", 2024-04-26T21:58:56.062 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.062 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.062 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.062 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.062 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.062 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.062 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.062 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-26T21:58:56.062 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.062 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.062 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.063 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.063 INFO:teuthology.orchestra.run.smithi072.stdout: "enabled": { 2024-04-26T21:58:56.063 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "enabled", 2024-04-26T21:58:56.063 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.063 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.063 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.063 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-26T21:58:56.063 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.063 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.063 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.063 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.063 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.063 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.063 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.063 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.063 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-26T21:58:56.063 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-26T21:58:56.064 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.064 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.064 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.064 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.064 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.064 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.064 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:56.064 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:56.064 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:56.064 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:56.064 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:56.064 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:56.064 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:56.064 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.064 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.064 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.065 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.065 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.065 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.065 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-26T21:58:56.065 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-26T21:58:56.065 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.065 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.065 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.065 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.065 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.065 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.065 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.065 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.065 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.065 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.065 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.066 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.066 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-26T21:58:56.066 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-26T21:58:56.066 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.066 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.066 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.066 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-26T21:58:56.066 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.066 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.066 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:56.066 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:56.066 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:56.066 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:56.066 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:56.066 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:56.067 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:56.067 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.067 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.067 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.067 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.067 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.067 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.067 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-26T21:58:56.067 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-26T21:58:56.067 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.067 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.067 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.067 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.067 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.067 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.068 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.068 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.068 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.068 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.068 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.068 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.068 INFO:teuthology.orchestra.run.smithi072.stdout: "max_completed_events": { 2024-04-26T21:58:56.068 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "max_completed_events", 2024-04-26T21:58:56.068 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-26T21:58:56.068 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.068 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.068 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "50", 2024-04-26T21:58:56.068 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.068 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.068 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.068 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "number of past completed events to remember", 2024-04-26T21:58:56.069 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.069 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.069 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.069 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.069 INFO:teuthology.orchestra.run.smithi072.stdout: "sleep_interval": { 2024-04-26T21:58:56.069 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "sleep_interval", 2024-04-26T21:58:56.069 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "secs", 2024-04-26T21:58:56.069 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.069 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.069 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "5", 2024-04-26T21:58:56.069 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.069 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.069 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.069 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "how long the module is going to sleep", 2024-04-26T21:58:56.069 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.069 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.070 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.070 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:56.070 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:56.070 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.070 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-26T21:58:56.070 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "prometheus", 2024-04-26T21:58:56.070 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-26T21:58:56.070 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-26T21:58:56.070 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-26T21:58:56.070 INFO:teuthology.orchestra.run.smithi072.stdout: "cache": { 2024-04-26T21:58:56.070 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "cache", 2024-04-26T21:58:56.070 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.070 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.070 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.070 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-26T21:58:56.071 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.071 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.071 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.071 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.071 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.071 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.071 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.071 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.071 INFO:teuthology.orchestra.run.smithi072.stdout: "exclude_perf_counters": { 2024-04-26T21:58:56.071 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "exclude_perf_counters", 2024-04-26T21:58:56.071 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.071 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.071 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.071 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-26T21:58:56.071 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.071 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.072 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.072 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-26T21:58:56.072 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-26T21:58:56.072 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.072 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.072 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.072 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-26T21:58:56.072 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-26T21:58:56.072 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.072 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.072 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.072 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.072 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.072 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.072 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:56.073 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:56.073 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:56.073 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:56.073 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:56.073 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:56.073 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:56.073 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.073 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.073 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.073 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.073 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.073 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.073 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-26T21:58:56.073 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-26T21:58:56.073 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.074 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.074 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.074 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.074 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.074 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.074 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.074 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.074 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.074 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.074 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.074 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.074 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-26T21:58:56.074 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-26T21:58:56.074 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.075 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.075 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.075 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-26T21:58:56.075 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.075 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.075 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:56.075 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:56.075 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:56.075 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:56.075 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:56.075 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:56.075 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:56.075 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.075 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.075 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.076 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.076 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.076 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.076 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-26T21:58:56.076 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-26T21:58:56.076 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.076 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.076 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.076 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.076 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.076 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.076 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.076 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.076 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.076 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.077 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.077 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.077 INFO:teuthology.orchestra.run.smithi072.stdout: "rbd_stats_pools": { 2024-04-26T21:58:56.077 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "rbd_stats_pools", 2024-04-26T21:58:56.077 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.077 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.077 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.077 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.077 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.077 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.077 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.077 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.077 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.077 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.077 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.078 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.078 INFO:teuthology.orchestra.run.smithi072.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-26T21:58:56.078 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-26T21:58:56.078 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-26T21:58:56.078 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.078 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.078 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "300", 2024-04-26T21:58:56.078 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.078 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.078 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.078 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.078 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.078 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.079 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.079 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.079 INFO:teuthology.orchestra.run.smithi072.stdout: "scrape_interval": { 2024-04-26T21:58:56.079 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "scrape_interval", 2024-04-26T21:58:56.079 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "float", 2024-04-26T21:58:56.079 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.079 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.079 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "15.0", 2024-04-26T21:58:56.079 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.079 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.079 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.079 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.079 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.079 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.080 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.080 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.080 INFO:teuthology.orchestra.run.smithi072.stdout: "server_addr": { 2024-04-26T21:58:56.080 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "server_addr", 2024-04-26T21:58:56.080 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.080 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.080 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.080 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "::", 2024-04-26T21:58:56.080 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.080 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.080 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.080 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-26T21:58:56.080 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.080 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.080 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.081 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.081 INFO:teuthology.orchestra.run.smithi072.stdout: "server_port": { 2024-04-26T21:58:56.081 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "server_port", 2024-04-26T21:58:56.081 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-26T21:58:56.081 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.081 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.081 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "9283", 2024-04-26T21:58:56.081 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.081 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.081 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.081 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-26T21:58:56.081 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.081 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.081 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.082 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.082 INFO:teuthology.orchestra.run.smithi072.stdout: "stale_cache_strategy": { 2024-04-26T21:58:56.082 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "stale_cache_strategy", 2024-04-26T21:58:56.082 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.082 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.082 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.082 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "log", 2024-04-26T21:58:56.082 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.082 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.082 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.082 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.082 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.082 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.082 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.083 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.083 INFO:teuthology.orchestra.run.smithi072.stdout: "standby_behaviour": { 2024-04-26T21:58:56.083 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "standby_behaviour", 2024-04-26T21:58:56.083 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.083 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.083 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.083 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "default", 2024-04-26T21:58:56.083 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.083 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.083 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:56.083 INFO:teuthology.orchestra.run.smithi072.stdout: "default", 2024-04-26T21:58:56.083 INFO:teuthology.orchestra.run.smithi072.stdout: "error" 2024-04-26T21:58:56.083 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.083 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.083 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.084 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.084 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.084 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.084 INFO:teuthology.orchestra.run.smithi072.stdout: "standby_error_status_code": { 2024-04-26T21:58:56.084 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "standby_error_status_code", 2024-04-26T21:58:56.084 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-26T21:58:56.084 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.084 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.084 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "500", 2024-04-26T21:58:56.084 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "400", 2024-04-26T21:58:56.084 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "599", 2024-04-26T21:58:56.084 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.084 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.084 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.084 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.085 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.085 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:56.085 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:56.085 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.085 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-26T21:58:56.085 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "rbd_support", 2024-04-26T21:58:56.085 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-26T21:58:56.085 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-26T21:58:56.085 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-26T21:58:56.085 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-26T21:58:56.085 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-26T21:58:56.085 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.085 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.085 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.085 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.086 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.086 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.086 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:56.086 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:56.086 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:56.086 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:56.086 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:56.086 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:56.086 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:56.086 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.086 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.086 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.086 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.086 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.087 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.087 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-26T21:58:56.087 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-26T21:58:56.087 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.087 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.087 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.087 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.087 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.087 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.087 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.087 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.087 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.087 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.087 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.087 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.088 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-26T21:58:56.088 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-26T21:58:56.088 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.088 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.088 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.088 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-26T21:58:56.088 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.088 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.088 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:56.088 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:56.088 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:56.088 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:56.088 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:56.088 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:56.088 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:56.089 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.089 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.089 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.089 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.089 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.089 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.089 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-26T21:58:56.089 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-26T21:58:56.089 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.089 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.089 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.089 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.089 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.089 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.089 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.090 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.090 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.090 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.090 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.090 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.090 INFO:teuthology.orchestra.run.smithi072.stdout: "max_concurrent_snap_create": { 2024-04-26T21:58:56.090 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "max_concurrent_snap_create", 2024-04-26T21:58:56.090 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-26T21:58:56.090 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.090 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.090 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "10", 2024-04-26T21:58:56.090 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.090 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.090 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.090 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.091 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.091 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.091 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.091 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.091 INFO:teuthology.orchestra.run.smithi072.stdout: "mirror_snapshot_schedule": { 2024-04-26T21:58:56.091 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "mirror_snapshot_schedule", 2024-04-26T21:58:56.091 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.091 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.091 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.091 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.091 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.091 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.091 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.091 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.091 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.092 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.092 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.092 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.092 INFO:teuthology.orchestra.run.smithi072.stdout: "trash_purge_schedule": { 2024-04-26T21:58:56.092 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "trash_purge_schedule", 2024-04-26T21:58:56.092 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.092 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.092 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.092 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.092 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.092 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.093 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.093 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.093 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.093 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.093 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.093 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:56.093 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:56.093 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.093 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-26T21:58:56.093 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "restful", 2024-04-26T21:58:56.093 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-26T21:58:56.093 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-26T21:58:56.093 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-26T21:58:56.094 INFO:teuthology.orchestra.run.smithi072.stdout: "enable_auth": { 2024-04-26T21:58:56.094 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "enable_auth", 2024-04-26T21:58:56.094 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.094 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.094 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.094 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-26T21:58:56.094 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.094 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.094 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.094 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.094 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.094 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.094 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.094 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.095 INFO:teuthology.orchestra.run.smithi072.stdout: "key_file": { 2024-04-26T21:58:56.095 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "key_file", 2024-04-26T21:58:56.095 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.095 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.095 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.095 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.095 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.095 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.095 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.095 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.095 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.095 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.095 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.095 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.095 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-26T21:58:56.096 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-26T21:58:56.096 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.096 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.096 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.096 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.096 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.096 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.096 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:56.096 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:56.096 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:56.096 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:56.096 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:56.096 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:56.096 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:56.096 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.097 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.097 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.097 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.097 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.097 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.097 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-26T21:58:56.097 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-26T21:58:56.097 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.097 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.097 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.097 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.097 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.097 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.097 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.097 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.098 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.098 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.098 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.098 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.098 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-26T21:58:56.098 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-26T21:58:56.098 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.098 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.098 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.098 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-26T21:58:56.098 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.098 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.098 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:56.098 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:56.098 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:56.099 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:56.099 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:56.099 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:56.099 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:56.099 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.099 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.099 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.099 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.099 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.099 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.099 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-26T21:58:56.099 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-26T21:58:56.099 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.099 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.099 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.100 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.100 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.100 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.100 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.100 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.100 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.100 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.100 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.100 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.100 INFO:teuthology.orchestra.run.smithi072.stdout: "server_addr": { 2024-04-26T21:58:56.100 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "server_addr", 2024-04-26T21:58:56.100 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.100 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.100 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.101 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.101 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.101 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.101 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.101 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.101 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.101 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.101 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.101 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.101 INFO:teuthology.orchestra.run.smithi072.stdout: "server_port": { 2024-04-26T21:58:56.101 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "server_port", 2024-04-26T21:58:56.101 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.101 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.101 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.101 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.102 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.102 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.102 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.102 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.102 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.102 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.102 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.102 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:56.102 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:56.102 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.102 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-26T21:58:56.102 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "rgw", 2024-04-26T21:58:56.102 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-26T21:58:56.102 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-26T21:58:56.102 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-26T21:58:56.103 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-26T21:58:56.103 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-26T21:58:56.103 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.103 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.103 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.103 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.103 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.103 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.103 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:56.103 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:56.103 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:56.103 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:56.103 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:56.103 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:56.103 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:56.103 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.104 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.104 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.104 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.104 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.104 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.104 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-26T21:58:56.104 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-26T21:58:56.104 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.104 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.104 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.104 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.104 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.104 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.104 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.104 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.105 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.105 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.105 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.105 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.105 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-26T21:58:56.105 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-26T21:58:56.105 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.105 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.105 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.105 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-26T21:58:56.105 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.105 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.105 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:56.105 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:56.105 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:56.106 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:56.106 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:56.106 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:56.106 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:56.106 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.106 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.106 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.106 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.106 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.106 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.106 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-26T21:58:56.106 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-26T21:58:56.106 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.106 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.107 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.107 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.107 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.107 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.107 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.107 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.107 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.107 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.107 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.107 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:56.107 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:56.107 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.107 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-26T21:58:56.107 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "rook", 2024-04-26T21:58:56.107 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-26T21:58:56.108 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-26T21:58:56.108 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-26T21:58:56.108 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-26T21:58:56.108 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-26T21:58:56.108 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.108 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.108 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.108 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.108 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.108 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.108 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:56.108 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:56.108 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:56.108 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:56.108 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:56.109 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:56.109 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:56.109 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.109 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.109 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.109 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.109 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.109 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.109 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-26T21:58:56.109 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-26T21:58:56.109 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.109 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.109 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.109 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.109 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.110 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.110 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.110 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.110 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.110 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.110 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.110 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.110 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-26T21:58:56.110 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-26T21:58:56.110 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.110 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.110 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.110 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-26T21:58:56.110 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.111 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.111 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:56.111 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:56.111 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:56.111 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:56.111 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:56.111 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:56.111 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:56.111 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.111 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.111 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.111 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.111 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.111 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.112 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-26T21:58:56.112 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-26T21:58:56.112 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.112 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.112 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.112 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.112 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.112 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.112 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.112 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.112 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.112 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.112 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.112 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.112 INFO:teuthology.orchestra.run.smithi072.stdout: "storage_class": { 2024-04-26T21:58:56.113 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "storage_class", 2024-04-26T21:58:56.113 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.113 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.113 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.113 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "local", 2024-04-26T21:58:56.113 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.113 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.113 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.113 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-26T21:58:56.113 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.113 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.113 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.113 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:56.113 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:56.113 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.113 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-26T21:58:56.114 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "selftest", 2024-04-26T21:58:56.114 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-26T21:58:56.114 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-26T21:58:56.114 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-26T21:58:56.114 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-26T21:58:56.114 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-26T21:58:56.114 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.114 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.114 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.114 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.114 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.114 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.114 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:56.114 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:56.114 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:56.115 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:56.115 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:56.115 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:56.115 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:56.115 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.115 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.115 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.115 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.115 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.115 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.115 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-26T21:58:56.115 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-26T21:58:56.115 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.115 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.116 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.116 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.116 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.116 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.116 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.116 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.116 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.116 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.116 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.116 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.116 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-26T21:58:56.116 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-26T21:58:56.116 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.116 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.116 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.117 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-26T21:58:56.117 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.117 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.117 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:56.117 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:56.117 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:56.117 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:56.117 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:56.117 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:56.117 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:56.117 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.117 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.117 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.117 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.117 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.117 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.118 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-26T21:58:56.118 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-26T21:58:56.118 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.118 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.118 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.118 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.118 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.118 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.118 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.118 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.118 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.118 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.118 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.118 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.118 INFO:teuthology.orchestra.run.smithi072.stdout: "roption1": { 2024-04-26T21:58:56.119 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "roption1", 2024-04-26T21:58:56.119 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.119 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.119 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.119 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.119 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.119 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.119 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.119 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.119 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.119 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.119 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.119 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.119 INFO:teuthology.orchestra.run.smithi072.stdout: "roption2": { 2024-04-26T21:58:56.120 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "roption2", 2024-04-26T21:58:56.120 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.120 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.120 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.120 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "xyz", 2024-04-26T21:58:56.120 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.120 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.120 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.120 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.120 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.120 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.120 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.120 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.120 INFO:teuthology.orchestra.run.smithi072.stdout: "rwoption1": { 2024-04-26T21:58:56.120 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "rwoption1", 2024-04-26T21:58:56.121 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.121 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.121 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.121 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.121 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.121 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.121 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.121 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.121 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.121 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.121 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.121 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.121 INFO:teuthology.orchestra.run.smithi072.stdout: "rwoption2": { 2024-04-26T21:58:56.121 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "rwoption2", 2024-04-26T21:58:56.121 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-26T21:58:56.122 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.122 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.122 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.122 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.122 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.122 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.122 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.122 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.122 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.122 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.122 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.122 INFO:teuthology.orchestra.run.smithi072.stdout: "rwoption3": { 2024-04-26T21:58:56.122 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "rwoption3", 2024-04-26T21:58:56.122 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "float", 2024-04-26T21:58:56.122 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.123 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.123 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.123 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.123 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.123 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.123 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.123 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.123 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.123 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.123 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.123 INFO:teuthology.orchestra.run.smithi072.stdout: "rwoption4": { 2024-04-26T21:58:56.123 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "rwoption4", 2024-04-26T21:58:56.123 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.123 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.123 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.124 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.124 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.124 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.124 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.124 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.124 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.124 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.124 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.124 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.124 INFO:teuthology.orchestra.run.smithi072.stdout: "rwoption5": { 2024-04-26T21:58:56.124 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "rwoption5", 2024-04-26T21:58:56.124 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.124 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.124 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.124 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.125 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.125 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.125 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.125 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.125 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.125 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.125 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.125 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.125 INFO:teuthology.orchestra.run.smithi072.stdout: "rwoption6": { 2024-04-26T21:58:56.125 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "rwoption6", 2024-04-26T21:58:56.125 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.125 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.125 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.125 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-26T21:58:56.125 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.126 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.126 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.126 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.126 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.126 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.126 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.126 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.126 INFO:teuthology.orchestra.run.smithi072.stdout: "rwoption7": { 2024-04-26T21:58:56.126 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "rwoption7", 2024-04-26T21:58:56.126 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-26T21:58:56.126 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.126 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.126 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.126 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "1", 2024-04-26T21:58:56.126 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "42", 2024-04-26T21:58:56.127 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.127 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.127 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.127 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.127 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.127 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.127 INFO:teuthology.orchestra.run.smithi072.stdout: "testkey": { 2024-04-26T21:58:56.127 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "testkey", 2024-04-26T21:58:56.127 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.127 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.127 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.127 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.127 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.127 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.127 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.128 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.128 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.128 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.128 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.128 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.128 INFO:teuthology.orchestra.run.smithi072.stdout: "testlkey": { 2024-04-26T21:58:56.128 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "testlkey", 2024-04-26T21:58:56.128 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.128 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.128 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.128 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.128 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.128 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.128 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.128 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.129 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.129 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.129 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.129 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.129 INFO:teuthology.orchestra.run.smithi072.stdout: "testnewline": { 2024-04-26T21:58:56.129 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "testnewline", 2024-04-26T21:58:56.129 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.129 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.129 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.129 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.129 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.129 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.129 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.129 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.129 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.130 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.130 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.130 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:56.130 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:56.130 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.130 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-26T21:58:56.130 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "snap_schedule", 2024-04-26T21:58:56.130 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-26T21:58:56.130 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-26T21:58:56.130 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-26T21:58:56.130 INFO:teuthology.orchestra.run.smithi072.stdout: "allow_m_granularity": { 2024-04-26T21:58:56.130 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "allow_m_granularity", 2024-04-26T21:58:56.130 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.130 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.130 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.131 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.131 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.131 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.131 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.131 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "allow minute scheduled snapshots", 2024-04-26T21:58:56.131 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.131 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.131 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.131 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.131 INFO:teuthology.orchestra.run.smithi072.stdout: "dump_on_update": { 2024-04-26T21:58:56.131 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "dump_on_update", 2024-04-26T21:58:56.131 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.131 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.131 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.131 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.132 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.132 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.132 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.132 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "dump database to debug log on update", 2024-04-26T21:58:56.132 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.132 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.132 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.132 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.132 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-26T21:58:56.132 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-26T21:58:56.132 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.132 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.132 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.133 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.133 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.133 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.133 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:56.133 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:56.133 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:56.133 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:56.133 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:56.133 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:56.133 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:56.133 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.133 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.133 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.133 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.133 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.134 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.134 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-26T21:58:56.134 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-26T21:58:56.134 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.134 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.134 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.134 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.134 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.134 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.134 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.134 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.134 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.134 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.134 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.134 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.135 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-26T21:58:56.135 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-26T21:58:56.135 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.135 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.135 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.135 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-26T21:58:56.135 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.135 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.135 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:56.135 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:56.135 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:56.135 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:56.135 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:56.135 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:56.135 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:56.136 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.136 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.136 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.136 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.136 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.136 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.136 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-26T21:58:56.136 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-26T21:58:56.136 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.136 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.136 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.136 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.136 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.136 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.137 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.137 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.137 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.137 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.137 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.137 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:56.137 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:56.137 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.137 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-26T21:58:56.137 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "stats", 2024-04-26T21:58:56.137 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-26T21:58:56.137 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-26T21:58:56.137 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-26T21:58:56.137 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-26T21:58:56.137 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-26T21:58:56.138 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.138 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.138 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.138 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.138 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.138 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.138 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:56.138 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:56.138 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:56.138 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:56.138 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:56.138 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:56.138 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:56.138 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.139 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.139 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.139 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.139 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.139 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.139 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-26T21:58:56.139 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-26T21:58:56.139 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.139 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.139 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.139 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.139 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.139 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.139 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.139 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.140 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.140 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.140 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.140 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.140 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-26T21:58:56.140 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-26T21:58:56.140 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.140 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.140 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.140 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-26T21:58:56.140 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.140 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.140 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:56.140 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:56.140 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:56.141 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:56.141 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:56.141 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:56.141 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:56.141 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.141 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.141 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.141 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.141 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.141 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.141 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-26T21:58:56.141 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-26T21:58:56.141 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.141 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.142 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.142 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.142 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.142 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.142 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.142 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.142 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.142 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.142 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.142 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:56.142 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:56.142 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.142 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-26T21:58:56.142 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "status", 2024-04-26T21:58:56.142 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-26T21:58:56.143 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-26T21:58:56.143 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-26T21:58:56.143 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-26T21:58:56.143 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-26T21:58:56.143 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.143 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.143 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.143 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.143 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.143 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.143 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:56.143 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:56.143 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:56.143 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:56.143 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:56.144 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:56.144 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:56.144 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.144 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.144 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.144 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.144 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.144 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.144 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-26T21:58:56.144 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-26T21:58:56.144 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.144 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.144 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.144 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.144 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.145 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.145 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.145 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.145 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.145 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.145 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.145 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.145 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-26T21:58:56.145 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-26T21:58:56.145 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.145 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.145 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.145 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-26T21:58:56.146 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.146 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.146 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:56.146 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:56.146 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:56.146 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:56.146 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:56.146 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:56.146 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:56.146 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.146 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.146 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.146 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.146 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.146 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.147 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-26T21:58:56.147 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-26T21:58:56.147 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.147 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.147 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.147 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.147 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.147 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.147 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.147 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.147 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.147 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.147 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.147 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:56.147 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:56.148 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.148 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-26T21:58:56.148 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "telegraf", 2024-04-26T21:58:56.148 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-26T21:58:56.148 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-26T21:58:56.148 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-26T21:58:56.148 INFO:teuthology.orchestra.run.smithi072.stdout: "address": { 2024-04-26T21:58:56.148 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "address", 2024-04-26T21:58:56.148 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.148 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.148 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.148 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-26T21:58:56.148 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.148 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.148 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.149 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.149 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.149 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.149 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.149 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.149 INFO:teuthology.orchestra.run.smithi072.stdout: "interval": { 2024-04-26T21:58:56.149 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "interval", 2024-04-26T21:58:56.149 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "secs", 2024-04-26T21:58:56.149 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.149 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.149 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "15", 2024-04-26T21:58:56.149 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.149 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.149 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.149 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.150 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.150 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.150 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.150 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.150 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-26T21:58:56.150 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-26T21:58:56.150 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.150 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.150 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.150 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.150 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.150 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.150 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:56.150 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:56.150 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:56.151 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:56.151 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:56.151 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:56.151 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:56.151 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.151 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.151 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.151 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.151 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.151 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.151 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-26T21:58:56.151 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-26T21:58:56.151 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.151 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.151 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.152 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.152 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.152 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.152 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.152 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.152 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.152 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.152 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.152 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.152 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-26T21:58:56.152 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-26T21:58:56.152 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.152 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.152 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.152 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-26T21:58:56.153 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.153 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.153 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:56.153 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:56.153 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:56.153 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:56.153 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:56.153 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:56.153 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:56.153 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.153 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.153 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.153 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.153 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.153 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.154 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-26T21:58:56.154 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-26T21:58:56.154 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.154 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.154 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.154 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.154 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.154 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.154 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.154 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.154 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.154 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.154 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.154 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:56.154 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:56.155 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.155 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-26T21:58:56.155 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "telemetry", 2024-04-26T21:58:56.155 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-26T21:58:56.155 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-26T21:58:56.155 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-26T21:58:56.155 INFO:teuthology.orchestra.run.smithi072.stdout: "channel_basic": { 2024-04-26T21:58:56.155 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "channel_basic", 2024-04-26T21:58:56.155 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.155 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.155 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.155 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-26T21:58:56.155 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.155 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.156 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.156 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-26T21:58:56.156 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.156 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.156 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.156 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.156 INFO:teuthology.orchestra.run.smithi072.stdout: "channel_crash": { 2024-04-26T21:58:56.156 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "channel_crash", 2024-04-26T21:58:56.156 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.156 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.156 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.156 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-26T21:58:56.156 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.156 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.156 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.157 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-26T21:58:56.157 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.157 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.157 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.157 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.157 INFO:teuthology.orchestra.run.smithi072.stdout: "channel_device": { 2024-04-26T21:58:56.157 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "channel_device", 2024-04-26T21:58:56.157 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.157 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.157 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.157 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-26T21:58:56.157 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.157 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.157 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.157 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-26T21:58:56.158 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.158 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.158 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.158 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.158 INFO:teuthology.orchestra.run.smithi072.stdout: "channel_ident": { 2024-04-26T21:58:56.158 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "channel_ident", 2024-04-26T21:58:56.158 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.158 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.158 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.158 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.158 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.158 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.158 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.159 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-26T21:58:56.159 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.159 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.159 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.159 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.159 INFO:teuthology.orchestra.run.smithi072.stdout: "channel_perf": { 2024-04-26T21:58:56.159 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "channel_perf", 2024-04-26T21:58:56.159 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.159 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.159 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.159 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.159 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.159 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.159 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.159 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-26T21:58:56.160 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.160 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.160 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.160 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.160 INFO:teuthology.orchestra.run.smithi072.stdout: "contact": { 2024-04-26T21:58:56.160 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "contact", 2024-04-26T21:58:56.160 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.160 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.160 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.160 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.160 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.160 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.160 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.160 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.160 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.161 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.161 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.161 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.161 INFO:teuthology.orchestra.run.smithi072.stdout: "description": { 2024-04-26T21:58:56.161 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "description", 2024-04-26T21:58:56.161 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.161 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.161 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.161 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.161 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.161 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.161 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.161 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.161 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.161 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.162 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.162 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.162 INFO:teuthology.orchestra.run.smithi072.stdout: "device_url": { 2024-04-26T21:58:56.162 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "device_url", 2024-04-26T21:58:56.162 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.162 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.162 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.162 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-26T21:58:56.162 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.162 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.162 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.162 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.162 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.162 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.162 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.163 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.163 INFO:teuthology.orchestra.run.smithi072.stdout: "enabled": { 2024-04-26T21:58:56.163 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "enabled", 2024-04-26T21:58:56.163 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.163 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.163 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.163 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.163 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.163 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.163 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.163 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.163 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.163 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.163 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.163 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.164 INFO:teuthology.orchestra.run.smithi072.stdout: "interval": { 2024-04-26T21:58:56.164 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "interval", 2024-04-26T21:58:56.164 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-26T21:58:56.164 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.164 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.164 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "24", 2024-04-26T21:58:56.164 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "8", 2024-04-26T21:58:56.164 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.164 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.164 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.164 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.164 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.164 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.164 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.164 INFO:teuthology.orchestra.run.smithi072.stdout: "last_opt_revision": { 2024-04-26T21:58:56.165 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "last_opt_revision", 2024-04-26T21:58:56.165 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-26T21:58:56.165 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.165 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.165 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "1", 2024-04-26T21:58:56.165 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.165 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.165 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.165 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.165 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.165 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.165 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.165 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.165 INFO:teuthology.orchestra.run.smithi072.stdout: "leaderboard": { 2024-04-26T21:58:56.166 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "leaderboard", 2024-04-26T21:58:56.166 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.166 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.166 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.166 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.166 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.166 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.166 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.166 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.166 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.166 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.166 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.166 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.166 INFO:teuthology.orchestra.run.smithi072.stdout: "leaderboard_description": { 2024-04-26T21:58:56.166 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "leaderboard_description", 2024-04-26T21:58:56.167 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.167 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.167 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.167 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.167 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.167 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.167 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.167 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.167 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.167 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.167 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.167 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.167 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-26T21:58:56.167 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-26T21:58:56.167 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.168 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.168 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.168 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.168 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.168 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.168 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:56.168 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:56.168 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:56.168 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:56.168 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:56.168 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:56.168 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:56.168 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.168 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.168 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.169 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.169 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.169 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.169 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-26T21:58:56.169 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-26T21:58:56.169 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.169 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.169 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.169 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.169 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.169 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.169 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.169 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.169 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.169 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.170 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.170 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.170 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-26T21:58:56.170 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-26T21:58:56.170 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.170 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.170 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.170 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-26T21:58:56.170 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.170 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.170 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:56.170 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:56.170 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:56.170 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:56.170 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:56.171 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:56.171 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:56.171 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.171 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.171 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.171 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.171 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.171 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.171 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-26T21:58:56.171 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-26T21:58:56.171 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.171 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.171 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.172 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.172 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.172 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.172 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.172 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.172 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.172 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.172 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.172 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.172 INFO:teuthology.orchestra.run.smithi072.stdout: "organization": { 2024-04-26T21:58:56.172 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "organization", 2024-04-26T21:58:56.172 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.172 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.172 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.173 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.173 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.173 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.173 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.173 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.173 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.173 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.173 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.173 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.173 INFO:teuthology.orchestra.run.smithi072.stdout: "proxy": { 2024-04-26T21:58:56.173 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "proxy", 2024-04-26T21:58:56.173 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.173 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.173 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.173 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.174 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.174 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.174 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.174 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.174 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.174 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.174 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.174 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.174 INFO:teuthology.orchestra.run.smithi072.stdout: "url": { 2024-04-26T21:58:56.174 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "url", 2024-04-26T21:58:56.174 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.174 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.174 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.174 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-26T21:58:56.175 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.175 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.175 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.175 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.175 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.175 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.175 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.175 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:56.175 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:56.175 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.175 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-26T21:58:56.175 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "test_orchestrator", 2024-04-26T21:58:56.175 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-26T21:58:56.175 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-26T21:58:56.175 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-26T21:58:56.176 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-26T21:58:56.176 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-26T21:58:56.176 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.176 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.176 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.176 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.176 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.176 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.176 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:56.176 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:56.176 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:56.176 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:56.176 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:56.176 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:56.176 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:56.177 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.177 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.177 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.177 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.177 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.177 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.177 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-26T21:58:56.177 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-26T21:58:56.177 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.177 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.177 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.177 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.177 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.177 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.178 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.178 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.178 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.178 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.178 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.178 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.178 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-26T21:58:56.178 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-26T21:58:56.178 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.178 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.178 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.178 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-26T21:58:56.178 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.178 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.178 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:56.179 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:56.179 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:56.179 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:56.179 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:56.179 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:56.179 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:56.179 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.179 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.179 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.179 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.179 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.179 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.179 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-26T21:58:56.179 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-26T21:58:56.179 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.180 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.180 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.180 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.180 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.180 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.180 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.180 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.180 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.180 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.180 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.180 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:56.180 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:56.180 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.180 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-26T21:58:56.180 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "volumes", 2024-04-26T21:58:56.181 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-26T21:58:56.181 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-26T21:58:56.181 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-26T21:58:56.181 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-26T21:58:56.181 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-26T21:58:56.181 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.181 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.181 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.181 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.181 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.181 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.181 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:56.181 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:56.181 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:56.181 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:56.182 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:56.182 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:56.182 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:56.182 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.182 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.182 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.182 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.182 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.182 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.182 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-26T21:58:56.182 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-26T21:58:56.182 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.182 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.182 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.182 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.183 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.183 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.183 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.183 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.183 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.183 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.183 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.183 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.183 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-26T21:58:56.183 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-26T21:58:56.183 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.183 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.183 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.183 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-26T21:58:56.183 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.184 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.184 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:56.184 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:56.184 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:56.184 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:56.184 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:56.184 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:56.184 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:56.184 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.184 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.184 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.184 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.184 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.184 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.185 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-26T21:58:56.185 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-26T21:58:56.185 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.185 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.185 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.185 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.185 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.185 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.185 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.185 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.185 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.185 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.185 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.185 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.185 INFO:teuthology.orchestra.run.smithi072.stdout: "max_concurrent_clones": { 2024-04-26T21:58:56.186 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "max_concurrent_clones", 2024-04-26T21:58:56.186 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-26T21:58:56.186 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.186 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.186 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "4", 2024-04-26T21:58:56.186 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.186 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.186 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.186 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-26T21:58:56.186 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.186 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.186 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.186 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.186 INFO:teuthology.orchestra.run.smithi072.stdout: "periodic_async_work": { 2024-04-26T21:58:56.187 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "periodic_async_work", 2024-04-26T21:58:56.187 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.187 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.187 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.187 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.187 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.187 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.187 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.187 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Periodically check for async work", 2024-04-26T21:58:56.187 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.187 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.187 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.187 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.187 INFO:teuthology.orchestra.run.smithi072.stdout: "snapshot_clone_delay": { 2024-04-26T21:58:56.187 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "snapshot_clone_delay", 2024-04-26T21:58:56.188 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-26T21:58:56.188 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.188 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.188 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "0", 2024-04-26T21:58:56.188 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.188 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.188 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.188 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-26T21:58:56.188 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.188 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.188 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.188 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.188 INFO:teuthology.orchestra.run.smithi072.stdout: "snapshot_clone_no_wait": { 2024-04-26T21:58:56.188 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "snapshot_clone_no_wait", 2024-04-26T21:58:56.188 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.189 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.189 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.189 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-26T21:58:56.189 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.189 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.189 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.189 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-26T21:58:56.189 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.189 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.189 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.189 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:56.189 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:56.189 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.189 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-26T21:58:56.189 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "zabbix", 2024-04-26T21:58:56.190 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-26T21:58:56.190 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-26T21:58:56.190 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-26T21:58:56.190 INFO:teuthology.orchestra.run.smithi072.stdout: "discovery_interval": { 2024-04-26T21:58:56.190 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "discovery_interval", 2024-04-26T21:58:56.190 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "uint", 2024-04-26T21:58:56.190 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.190 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.190 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "100", 2024-04-26T21:58:56.190 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.190 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.190 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.190 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.190 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.191 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.191 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.191 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.191 INFO:teuthology.orchestra.run.smithi072.stdout: "identifier": { 2024-04-26T21:58:56.191 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "identifier", 2024-04-26T21:58:56.191 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.191 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.191 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.191 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.191 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.191 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.191 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.191 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.191 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.191 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.192 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.192 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.192 INFO:teuthology.orchestra.run.smithi072.stdout: "interval": { 2024-04-26T21:58:56.192 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "interval", 2024-04-26T21:58:56.192 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "secs", 2024-04-26T21:58:56.192 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.192 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.192 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "60", 2024-04-26T21:58:56.192 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.192 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.192 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.192 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.192 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.192 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.192 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.193 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.193 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-26T21:58:56.193 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-26T21:58:56.193 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.193 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.193 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.193 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.193 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.193 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.193 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:56.193 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:56.193 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:56.193 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:56.193 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:56.193 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:56.194 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:56.194 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.194 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.194 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.194 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.194 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.194 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.194 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-26T21:58:56.194 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-26T21:58:56.194 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.194 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.194 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.194 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.194 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.195 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.195 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.195 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.195 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.195 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.195 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.195 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.195 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-26T21:58:56.195 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-26T21:58:56.195 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.195 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.195 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.195 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-26T21:58:56.195 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.195 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.196 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:56.196 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:56.196 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:56.196 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:56.196 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:56.196 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:56.196 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:56.196 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.196 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.196 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.196 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.196 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.196 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.196 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-26T21:58:56.196 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-26T21:58:56.196 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.197 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.197 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.197 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.197 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.197 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.197 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.197 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.197 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.197 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.197 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.197 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.197 INFO:teuthology.orchestra.run.smithi072.stdout: "zabbix_host": { 2024-04-26T21:58:56.197 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "zabbix_host", 2024-04-26T21:58:56.198 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.198 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.198 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.198 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.198 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.198 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.198 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.198 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.198 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.198 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.198 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.198 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.198 INFO:teuthology.orchestra.run.smithi072.stdout: "zabbix_port": { 2024-04-26T21:58:56.198 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "zabbix_port", 2024-04-26T21:58:56.199 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-26T21:58:56.199 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.199 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.199 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "10051", 2024-04-26T21:58:56.199 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.199 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.199 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.199 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.199 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.199 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.199 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.199 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.199 INFO:teuthology.orchestra.run.smithi072.stdout: "zabbix_sender": { 2024-04-26T21:58:56.199 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "zabbix_sender", 2024-04-26T21:58:56.200 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.200 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.200 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.200 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-26T21:58:56.200 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.200 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.200 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.200 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.200 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.200 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.200 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.200 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:56.200 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:56.200 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:56.201 INFO:teuthology.orchestra.run.smithi072.stdout: ] 2024-04-26T21:58:56.201 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.201 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-26T21:58:56.201 INFO:teuthology.orchestra.run.smithi072.stdout: "gid": 34109, 2024-04-26T21:58:56.201 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "y", 2024-04-26T21:58:56.201 INFO:teuthology.orchestra.run.smithi072.stdout: "mgr_features": 4540701547738038271, 2024-04-26T21:58:56.201 INFO:teuthology.orchestra.run.smithi072.stdout: "available_modules": [ 2024-04-26T21:58:56.201 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-26T21:58:56.201 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "alerts", 2024-04-26T21:58:56.201 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-26T21:58:56.201 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-26T21:58:56.201 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-26T21:58:56.201 INFO:teuthology.orchestra.run.smithi072.stdout: "interval": { 2024-04-26T21:58:56.201 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "interval", 2024-04-26T21:58:56.202 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "secs", 2024-04-26T21:58:56.202 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.202 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.202 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "60", 2024-04-26T21:58:56.202 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.202 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.202 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.202 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "How frequently to reexamine health status", 2024-04-26T21:58:56.202 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.202 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.202 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.202 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.202 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-26T21:58:56.203 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-26T21:58:56.203 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.203 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.203 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.203 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.203 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.203 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.203 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:56.203 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:56.203 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:56.203 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:56.203 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:56.203 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:56.203 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:56.203 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.204 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.204 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.204 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.204 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.204 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.204 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-26T21:58:56.204 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-26T21:58:56.204 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.204 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.204 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.204 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.204 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.204 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.204 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.205 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.205 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.205 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.205 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.205 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.205 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-26T21:58:56.205 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-26T21:58:56.205 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.205 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.205 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.205 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-26T21:58:56.205 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.205 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.205 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:56.205 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:56.205 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:56.206 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:56.206 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:56.206 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:56.206 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:56.206 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.206 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.206 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.206 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.206 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.206 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.206 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-26T21:58:56.206 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-26T21:58:56.206 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.206 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.207 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.207 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.207 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.207 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.207 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.207 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.207 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.207 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.207 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.207 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.207 INFO:teuthology.orchestra.run.smithi072.stdout: "smtp_destination": { 2024-04-26T21:58:56.207 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "smtp_destination", 2024-04-26T21:58:56.207 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.207 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.207 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.208 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.208 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.208 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.208 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.208 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Email address to send alerts to", 2024-04-26T21:58:56.208 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.208 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.208 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.208 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.208 INFO:teuthology.orchestra.run.smithi072.stdout: "smtp_from_name": { 2024-04-26T21:58:56.208 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "smtp_from_name", 2024-04-26T21:58:56.208 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.208 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.208 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.208 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "Ceph", 2024-04-26T21:58:56.209 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.209 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.209 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.209 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Email From: name", 2024-04-26T21:58:56.209 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.209 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.209 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.209 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.209 INFO:teuthology.orchestra.run.smithi072.stdout: "smtp_host": { 2024-04-26T21:58:56.209 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "smtp_host", 2024-04-26T21:58:56.209 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.209 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.209 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.209 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.209 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.209 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.210 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.210 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "SMTP server", 2024-04-26T21:58:56.210 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.210 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.210 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.210 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.210 INFO:teuthology.orchestra.run.smithi072.stdout: "smtp_password": { 2024-04-26T21:58:56.210 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "smtp_password", 2024-04-26T21:58:56.210 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.210 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.210 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.210 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.210 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.210 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.210 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.211 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Password to authenticate with", 2024-04-26T21:58:56.211 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.211 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.211 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.211 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.211 INFO:teuthology.orchestra.run.smithi072.stdout: "smtp_port": { 2024-04-26T21:58:56.211 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "smtp_port", 2024-04-26T21:58:56.211 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-26T21:58:56.211 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.211 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.211 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "465", 2024-04-26T21:58:56.211 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.211 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.211 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.212 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "SMTP port", 2024-04-26T21:58:56.212 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.212 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.212 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.212 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.212 INFO:teuthology.orchestra.run.smithi072.stdout: "smtp_sender": { 2024-04-26T21:58:56.212 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "smtp_sender", 2024-04-26T21:58:56.212 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.212 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.212 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.212 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.212 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.212 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.212 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.212 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "SMTP envelope sender", 2024-04-26T21:58:56.213 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.213 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.213 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.213 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.213 INFO:teuthology.orchestra.run.smithi072.stdout: "smtp_ssl": { 2024-04-26T21:58:56.213 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "smtp_ssl", 2024-04-26T21:58:56.213 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.213 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.213 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.213 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-26T21:58:56.213 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.213 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.213 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.213 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-26T21:58:56.214 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.214 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.214 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.214 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.214 INFO:teuthology.orchestra.run.smithi072.stdout: "smtp_user": { 2024-04-26T21:58:56.214 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "smtp_user", 2024-04-26T21:58:56.214 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.214 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.214 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.214 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.214 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.214 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.214 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.214 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "User to authenticate as", 2024-04-26T21:58:56.214 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.215 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.215 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.215 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:56.215 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:56.215 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.215 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-26T21:58:56.215 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "balancer", 2024-04-26T21:58:56.215 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-26T21:58:56.215 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-26T21:58:56.215 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-26T21:58:56.215 INFO:teuthology.orchestra.run.smithi072.stdout: "active": { 2024-04-26T21:58:56.215 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "active", 2024-04-26T21:58:56.215 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.215 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.215 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.216 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-26T21:58:56.216 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.216 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.216 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.216 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "automatically balance PGs across cluster", 2024-04-26T21:58:56.216 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.216 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.216 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.216 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.216 INFO:teuthology.orchestra.run.smithi072.stdout: "begin_time": { 2024-04-26T21:58:56.216 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "begin_time", 2024-04-26T21:58:56.216 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.216 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.216 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.216 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "0000", 2024-04-26T21:58:56.217 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.217 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.217 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.217 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "beginning time of day to automatically balance", 2024-04-26T21:58:56.217 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-26T21:58:56.217 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.217 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.217 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.217 INFO:teuthology.orchestra.run.smithi072.stdout: "begin_weekday": { 2024-04-26T21:58:56.217 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "begin_weekday", 2024-04-26T21:58:56.217 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "uint", 2024-04-26T21:58:56.217 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.217 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.217 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "0", 2024-04-26T21:58:56.217 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "0", 2024-04-26T21:58:56.218 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "6", 2024-04-26T21:58:56.218 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.218 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-26T21:58:56.218 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-26T21:58:56.218 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.218 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.218 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.218 INFO:teuthology.orchestra.run.smithi072.stdout: "crush_compat_max_iterations": { 2024-04-26T21:58:56.218 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "crush_compat_max_iterations", 2024-04-26T21:58:56.218 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "uint", 2024-04-26T21:58:56.218 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.218 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.218 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "25", 2024-04-26T21:58:56.218 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "1", 2024-04-26T21:58:56.218 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "250", 2024-04-26T21:58:56.219 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.219 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-26T21:58:56.219 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.219 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.219 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.219 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.219 INFO:teuthology.orchestra.run.smithi072.stdout: "crush_compat_metrics": { 2024-04-26T21:58:56.219 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "crush_compat_metrics", 2024-04-26T21:58:56.219 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.219 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.219 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.219 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "pgs,objects,bytes", 2024-04-26T21:58:56.219 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.219 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.219 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.220 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-26T21:58:56.220 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-26T21:58:56.220 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.220 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.220 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.220 INFO:teuthology.orchestra.run.smithi072.stdout: "crush_compat_step": { 2024-04-26T21:58:56.220 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "crush_compat_step", 2024-04-26T21:58:56.220 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "float", 2024-04-26T21:58:56.220 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.220 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.220 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "0.5", 2024-04-26T21:58:56.220 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "0.001", 2024-04-26T21:58:56.220 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "0.999", 2024-04-26T21:58:56.220 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.220 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "aggressiveness of optimization", 2024-04-26T21:58:56.221 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-26T21:58:56.221 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.221 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.221 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.221 INFO:teuthology.orchestra.run.smithi072.stdout: "end_time": { 2024-04-26T21:58:56.221 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "end_time", 2024-04-26T21:58:56.221 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.221 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.221 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.221 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "2359", 2024-04-26T21:58:56.221 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.221 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.221 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.221 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "ending time of day to automatically balance", 2024-04-26T21:58:56.222 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-26T21:58:56.222 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.222 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.222 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.222 INFO:teuthology.orchestra.run.smithi072.stdout: "end_weekday": { 2024-04-26T21:58:56.222 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "end_weekday", 2024-04-26T21:58:56.222 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "uint", 2024-04-26T21:58:56.222 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.222 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.222 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "0", 2024-04-26T21:58:56.222 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "0", 2024-04-26T21:58:56.222 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "6", 2024-04-26T21:58:56.222 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.222 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-26T21:58:56.223 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-26T21:58:56.223 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.223 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.223 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.223 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-26T21:58:56.223 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-26T21:58:56.223 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.223 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.223 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.223 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.223 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.223 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.223 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:56.223 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:56.223 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:56.224 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:56.224 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:56.224 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:56.224 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:56.224 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.224 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.224 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.224 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.224 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.224 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.224 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-26T21:58:56.224 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-26T21:58:56.224 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.224 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.224 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.225 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.225 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.225 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.225 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.225 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.225 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.225 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.225 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.225 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.225 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-26T21:58:56.225 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-26T21:58:56.225 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.225 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.225 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.226 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-26T21:58:56.226 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.226 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.226 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:56.226 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:56.226 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:56.226 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:56.226 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:56.226 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:56.226 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:56.226 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.226 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.226 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.226 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.226 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.227 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.227 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-26T21:58:56.227 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-26T21:58:56.229 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.230 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.230 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.230 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.230 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.230 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.230 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.230 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.231 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.231 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.231 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.231 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.231 INFO:teuthology.orchestra.run.smithi072.stdout: "min_score": { 2024-04-26T21:58:56.231 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "min_score", 2024-04-26T21:58:56.231 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "float", 2024-04-26T21:58:56.231 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.231 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.231 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "0", 2024-04-26T21:58:56.231 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.231 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.231 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.231 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-26T21:58:56.231 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.232 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.232 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.232 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.232 INFO:teuthology.orchestra.run.smithi072.stdout: "mode": { 2024-04-26T21:58:56.232 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "mode", 2024-04-26T21:58:56.232 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.232 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.232 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.232 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "upmap", 2024-04-26T21:58:56.232 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.232 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.232 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:56.232 INFO:teuthology.orchestra.run.smithi072.stdout: "crush-compat", 2024-04-26T21:58:56.232 INFO:teuthology.orchestra.run.smithi072.stdout: "none", 2024-04-26T21:58:56.232 INFO:teuthology.orchestra.run.smithi072.stdout: "read", 2024-04-26T21:58:56.233 INFO:teuthology.orchestra.run.smithi072.stdout: "upmap", 2024-04-26T21:58:56.233 INFO:teuthology.orchestra.run.smithi072.stdout: "upmap-read" 2024-04-26T21:58:56.233 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.233 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Balancer mode", 2024-04-26T21:58:56.233 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.233 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.233 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.233 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.233 INFO:teuthology.orchestra.run.smithi072.stdout: "pool_ids": { 2024-04-26T21:58:56.233 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "pool_ids", 2024-04-26T21:58:56.233 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.233 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.233 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.233 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.233 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.234 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.234 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.234 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-26T21:58:56.234 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.234 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.234 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.234 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.234 INFO:teuthology.orchestra.run.smithi072.stdout: "sleep_interval": { 2024-04-26T21:58:56.234 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "sleep_interval", 2024-04-26T21:58:56.234 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "secs", 2024-04-26T21:58:56.234 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.234 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.234 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "60", 2024-04-26T21:58:56.234 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.235 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.235 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.235 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-26T21:58:56.235 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.235 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.235 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.235 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.235 INFO:teuthology.orchestra.run.smithi072.stdout: "upmap_max_deviation": { 2024-04-26T21:58:56.235 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "upmap_max_deviation", 2024-04-26T21:58:56.235 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-26T21:58:56.235 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.235 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.235 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "5", 2024-04-26T21:58:56.235 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "1", 2024-04-26T21:58:56.235 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.235 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.236 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-26T21:58:56.236 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-26T21:58:56.236 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.236 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.236 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.236 INFO:teuthology.orchestra.run.smithi072.stdout: "upmap_max_optimizations": { 2024-04-26T21:58:56.236 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "upmap_max_optimizations", 2024-04-26T21:58:56.236 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "uint", 2024-04-26T21:58:56.236 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.236 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.236 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "10", 2024-04-26T21:58:56.236 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.236 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.236 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.237 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-26T21:58:56.237 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.237 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.237 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.237 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:56.237 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:56.237 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.237 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-26T21:58:56.237 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "cephadm", 2024-04-26T21:58:56.237 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-26T21:58:56.237 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-26T21:58:56.237 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-26T21:58:56.237 INFO:teuthology.orchestra.run.smithi072.stdout: "agent_down_multiplier": { 2024-04-26T21:58:56.237 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "agent_down_multiplier", 2024-04-26T21:58:56.237 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "float", 2024-04-26T21:58:56.238 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.238 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.238 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "3.0", 2024-04-26T21:58:56.238 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.238 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.238 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.238 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-26T21:58:56.238 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.238 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.239 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.239 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.239 INFO:teuthology.orchestra.run.smithi072.stdout: "agent_refresh_rate": { 2024-04-26T21:58:56.239 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "agent_refresh_rate", 2024-04-26T21:58:56.239 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "secs", 2024-04-26T21:58:56.239 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.239 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.239 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "20", 2024-04-26T21:58:56.239 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.239 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.239 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.240 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-26T21:58:56.240 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.240 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.240 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.240 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.240 INFO:teuthology.orchestra.run.smithi072.stdout: "agent_starting_port": { 2024-04-26T21:58:56.240 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "agent_starting_port", 2024-04-26T21:58:56.240 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-26T21:58:56.240 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.240 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.240 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "4721", 2024-04-26T21:58:56.240 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.240 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.240 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.240 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-26T21:58:56.241 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.241 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.241 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.241 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.241 INFO:teuthology.orchestra.run.smithi072.stdout: "allow_ptrace": { 2024-04-26T21:58:56.241 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "allow_ptrace", 2024-04-26T21:58:56.241 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.241 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.241 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.241 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.241 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.241 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.241 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.241 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-26T21:58:56.241 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-26T21:58:56.242 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.242 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.242 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.242 INFO:teuthology.orchestra.run.smithi072.stdout: "autotune_interval": { 2024-04-26T21:58:56.242 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "autotune_interval", 2024-04-26T21:58:56.242 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "secs", 2024-04-26T21:58:56.242 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.242 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.242 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "600", 2024-04-26T21:58:56.242 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.242 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.242 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.242 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-26T21:58:56.243 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.243 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.243 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.243 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.243 INFO:teuthology.orchestra.run.smithi072.stdout: "autotune_memory_target_ratio": { 2024-04-26T21:58:56.243 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "autotune_memory_target_ratio", 2024-04-26T21:58:56.243 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "float", 2024-04-26T21:58:56.243 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.243 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.243 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "0.7", 2024-04-26T21:58:56.243 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.243 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.243 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.243 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-26T21:58:56.243 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.244 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.244 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.244 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.244 INFO:teuthology.orchestra.run.smithi072.stdout: "cephadm_log_destination": { 2024-04-26T21:58:56.244 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "cephadm_log_destination", 2024-04-26T21:58:56.244 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.244 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.244 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.244 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.244 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.244 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.244 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:56.244 INFO:teuthology.orchestra.run.smithi072.stdout: "file", 2024-04-26T21:58:56.244 INFO:teuthology.orchestra.run.smithi072.stdout: "file,syslog", 2024-04-26T21:58:56.245 INFO:teuthology.orchestra.run.smithi072.stdout: "syslog" 2024-04-26T21:58:56.245 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.245 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-26T21:58:56.245 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.245 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.245 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.245 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.245 INFO:teuthology.orchestra.run.smithi072.stdout: "cgroups_split": { 2024-04-26T21:58:56.245 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "cgroups_split", 2024-04-26T21:58:56.245 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.245 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.245 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.245 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-26T21:58:56.245 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.245 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.246 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.246 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-26T21:58:56.246 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.246 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.246 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.246 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.246 INFO:teuthology.orchestra.run.smithi072.stdout: "config_checks_enabled": { 2024-04-26T21:58:56.246 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "config_checks_enabled", 2024-04-26T21:58:56.246 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.246 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.246 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.246 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.246 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.246 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.246 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.247 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-26T21:58:56.247 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.247 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.247 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.247 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.247 INFO:teuthology.orchestra.run.smithi072.stdout: "config_dashboard": { 2024-04-26T21:58:56.247 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "config_dashboard", 2024-04-26T21:58:56.247 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.247 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.247 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.247 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-26T21:58:56.247 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.247 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.247 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.247 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-26T21:58:56.248 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.248 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.248 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.248 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.248 INFO:teuthology.orchestra.run.smithi072.stdout: "container_image_alertmanager": { 2024-04-26T21:58:56.248 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "container_image_alertmanager", 2024-04-26T21:58:56.248 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.248 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.248 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.248 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-26T21:58:56.248 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.248 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.248 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.248 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Prometheus container image", 2024-04-26T21:58:56.248 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.248 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.249 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.249 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.249 INFO:teuthology.orchestra.run.smithi072.stdout: "container_image_base": { 2024-04-26T21:58:56.249 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "container_image_base", 2024-04-26T21:58:56.249 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.249 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.249 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.249 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-26T21:58:56.249 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.249 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.249 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.249 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Container image name, without the tag", 2024-04-26T21:58:56.249 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.249 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.249 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.250 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.250 INFO:teuthology.orchestra.run.smithi072.stdout: "container_image_elasticsearch": { 2024-04-26T21:58:56.250 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "container_image_elasticsearch", 2024-04-26T21:58:56.250 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.250 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.250 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.250 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-26T21:58:56.250 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.250 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.250 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.250 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "elasticsearch container image", 2024-04-26T21:58:56.250 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.250 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.250 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.251 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.251 INFO:teuthology.orchestra.run.smithi072.stdout: "container_image_grafana": { 2024-04-26T21:58:56.251 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "container_image_grafana", 2024-04-26T21:58:56.251 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.251 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.251 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.251 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-26T21:58:56.251 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.251 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.251 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.251 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Prometheus container image", 2024-04-26T21:58:56.251 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.251 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.251 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.251 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.252 INFO:teuthology.orchestra.run.smithi072.stdout: "container_image_haproxy": { 2024-04-26T21:58:56.252 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "container_image_haproxy", 2024-04-26T21:58:56.252 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.252 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.252 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.252 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-26T21:58:56.252 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.252 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.252 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.252 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "HAproxy container image", 2024-04-26T21:58:56.252 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.252 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.252 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.252 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.252 INFO:teuthology.orchestra.run.smithi072.stdout: "container_image_jaeger_agent": { 2024-04-26T21:58:56.253 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "container_image_jaeger_agent", 2024-04-26T21:58:56.253 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.253 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.253 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.253 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-26T21:58:56.253 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.253 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.253 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.253 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Jaeger agent container image", 2024-04-26T21:58:56.253 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.253 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.253 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.253 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.253 INFO:teuthology.orchestra.run.smithi072.stdout: "container_image_jaeger_collector": { 2024-04-26T21:58:56.254 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "container_image_jaeger_collector", 2024-04-26T21:58:56.254 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.254 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.254 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.254 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-26T21:58:56.254 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.254 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.254 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.254 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Jaeger collector container image", 2024-04-26T21:58:56.254 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.254 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.254 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.254 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.254 INFO:teuthology.orchestra.run.smithi072.stdout: "container_image_jaeger_query": { 2024-04-26T21:58:56.254 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "container_image_jaeger_query", 2024-04-26T21:58:56.255 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.255 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.255 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.255 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-26T21:58:56.255 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.255 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.255 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.255 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Jaeger query container image", 2024-04-26T21:58:56.255 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.255 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.255 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.255 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.255 INFO:teuthology.orchestra.run.smithi072.stdout: "container_image_keepalived": { 2024-04-26T21:58:56.255 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "container_image_keepalived", 2024-04-26T21:58:56.255 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.256 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.256 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.256 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-26T21:58:56.256 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.256 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.256 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.256 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Keepalived container image", 2024-04-26T21:58:56.256 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.256 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.256 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.256 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.256 INFO:teuthology.orchestra.run.smithi072.stdout: "container_image_loki": { 2024-04-26T21:58:56.256 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "container_image_loki", 2024-04-26T21:58:56.256 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.256 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.257 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.257 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-26T21:58:56.257 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.257 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.257 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.257 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Loki container image", 2024-04-26T21:58:56.257 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.257 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.257 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.257 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.257 INFO:teuthology.orchestra.run.smithi072.stdout: "container_image_node_exporter": { 2024-04-26T21:58:56.257 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "container_image_node_exporter", 2024-04-26T21:58:56.257 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.257 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.257 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.258 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-26T21:58:56.258 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.258 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.258 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.258 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Prometheus container image", 2024-04-26T21:58:56.258 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.258 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.258 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.258 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.258 INFO:teuthology.orchestra.run.smithi072.stdout: "container_image_nvmeof": { 2024-04-26T21:58:56.258 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "container_image_nvmeof", 2024-04-26T21:58:56.258 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.258 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.258 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.259 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-26T21:58:56.259 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.259 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.259 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.259 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Nvme-of container image", 2024-04-26T21:58:56.259 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.259 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.259 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.259 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.259 INFO:teuthology.orchestra.run.smithi072.stdout: "container_image_prometheus": { 2024-04-26T21:58:56.259 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "container_image_prometheus", 2024-04-26T21:58:56.259 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.259 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.259 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.259 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-26T21:58:56.259 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.260 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.260 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.260 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Prometheus container image", 2024-04-26T21:58:56.260 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.260 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.260 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.260 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.260 INFO:teuthology.orchestra.run.smithi072.stdout: "container_image_promtail": { 2024-04-26T21:58:56.260 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "container_image_promtail", 2024-04-26T21:58:56.260 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.260 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.260 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.260 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-26T21:58:56.260 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.260 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.261 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.261 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Promtail container image", 2024-04-26T21:58:56.261 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.261 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.261 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.261 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.261 INFO:teuthology.orchestra.run.smithi072.stdout: "container_image_samba": { 2024-04-26T21:58:56.261 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "container_image_samba", 2024-04-26T21:58:56.261 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.261 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.261 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.261 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-26T21:58:56.261 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.261 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.262 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.262 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Samba/SMB container image", 2024-04-26T21:58:56.262 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.262 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.262 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.262 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.262 INFO:teuthology.orchestra.run.smithi072.stdout: "container_image_snmp_gateway": { 2024-04-26T21:58:56.262 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "container_image_snmp_gateway", 2024-04-26T21:58:56.262 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.262 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.262 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.262 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-26T21:58:56.262 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.262 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.263 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.263 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "SNMP Gateway container image", 2024-04-26T21:58:56.263 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.263 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.263 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.263 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.263 INFO:teuthology.orchestra.run.smithi072.stdout: "container_init": { 2024-04-26T21:58:56.263 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "container_init", 2024-04-26T21:58:56.263 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.263 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.263 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.263 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-26T21:58:56.263 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.263 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.263 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.264 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Run podman/docker with `--init`", 2024-04-26T21:58:56.264 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.264 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.264 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.264 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.264 INFO:teuthology.orchestra.run.smithi072.stdout: "daemon_cache_timeout": { 2024-04-26T21:58:56.264 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "daemon_cache_timeout", 2024-04-26T21:58:56.264 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "secs", 2024-04-26T21:58:56.264 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.264 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.264 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "600", 2024-04-26T21:58:56.264 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.264 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.264 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.264 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-26T21:58:56.265 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.265 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.265 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.265 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.265 INFO:teuthology.orchestra.run.smithi072.stdout: "default_cephadm_command_timeout": { 2024-04-26T21:58:56.265 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "default_cephadm_command_timeout", 2024-04-26T21:58:56.265 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-26T21:58:56.265 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.265 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.265 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "900", 2024-04-26T21:58:56.265 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.265 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.265 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.265 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-26T21:58:56.265 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.266 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.266 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.266 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.266 INFO:teuthology.orchestra.run.smithi072.stdout: "default_registry": { 2024-04-26T21:58:56.266 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "default_registry", 2024-04-26T21:58:56.266 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.266 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.266 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.266 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "docker.io", 2024-04-26T21:58:56.266 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.266 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.266 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.266 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-26T21:58:56.266 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.266 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.267 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.267 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.267 INFO:teuthology.orchestra.run.smithi072.stdout: "device_cache_timeout": { 2024-04-26T21:58:56.267 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "device_cache_timeout", 2024-04-26T21:58:56.267 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "secs", 2024-04-26T21:58:56.267 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.267 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.267 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "1800", 2024-04-26T21:58:56.267 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.267 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.267 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.267 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "seconds to cache device inventory", 2024-04-26T21:58:56.267 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.267 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.268 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.268 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.268 INFO:teuthology.orchestra.run.smithi072.stdout: "device_enhanced_scan": { 2024-04-26T21:58:56.268 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "device_enhanced_scan", 2024-04-26T21:58:56.268 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.268 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.268 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.268 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.268 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.268 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.268 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.268 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-26T21:58:56.268 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.268 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.268 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.268 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.269 INFO:teuthology.orchestra.run.smithi072.stdout: "facts_cache_timeout": { 2024-04-26T21:58:56.269 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "facts_cache_timeout", 2024-04-26T21:58:56.269 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "secs", 2024-04-26T21:58:56.269 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.269 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.269 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "60", 2024-04-26T21:58:56.269 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.269 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.269 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.269 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "seconds to cache host facts data", 2024-04-26T21:58:56.269 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.269 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.269 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.269 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.269 INFO:teuthology.orchestra.run.smithi072.stdout: "grafana_dashboards_path": { 2024-04-26T21:58:56.270 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "grafana_dashboards_path", 2024-04-26T21:58:56.270 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.270 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.270 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.270 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-26T21:58:56.270 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.270 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.270 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.270 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-26T21:58:56.270 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.270 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.270 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.270 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.270 INFO:teuthology.orchestra.run.smithi072.stdout: "host_check_interval": { 2024-04-26T21:58:56.270 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "host_check_interval", 2024-04-26T21:58:56.271 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "secs", 2024-04-26T21:58:56.271 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.271 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.271 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "600", 2024-04-26T21:58:56.271 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.271 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.271 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.271 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "how frequently to perform a host check", 2024-04-26T21:58:56.271 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.271 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.271 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.271 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.271 INFO:teuthology.orchestra.run.smithi072.stdout: "hw_monitoring": { 2024-04-26T21:58:56.271 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "hw_monitoring", 2024-04-26T21:58:56.271 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.271 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.272 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.272 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.272 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.272 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.272 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.272 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-26T21:58:56.272 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.272 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.272 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.272 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.272 INFO:teuthology.orchestra.run.smithi072.stdout: "inventory_list_all": { 2024-04-26T21:58:56.272 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "inventory_list_all", 2024-04-26T21:58:56.272 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.272 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.272 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.273 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.273 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.273 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.273 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.273 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-26T21:58:56.273 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.273 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.273 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.273 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.273 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-26T21:58:56.273 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-26T21:58:56.273 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.273 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.273 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.274 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.274 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.274 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.274 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:56.274 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:56.274 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:56.274 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:56.274 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:56.274 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:56.274 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:56.274 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.274 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.274 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.274 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.275 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.275 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.275 INFO:teuthology.orchestra.run.smithi072.stdout: "log_refresh_metadata": { 2024-04-26T21:58:56.275 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_refresh_metadata", 2024-04-26T21:58:56.275 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.275 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.275 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.275 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.275 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.275 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.275 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.275 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-26T21:58:56.275 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.275 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.275 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.276 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.276 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-26T21:58:56.276 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-26T21:58:56.276 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.276 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.276 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.276 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-26T21:58:56.276 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.276 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.276 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.276 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-26T21:58:56.276 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.276 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.276 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.276 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.276 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-26T21:58:56.277 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-26T21:58:56.277 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.277 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.277 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.277 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-26T21:58:56.277 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.277 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.277 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:56.277 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:56.277 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:56.277 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:56.277 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:56.277 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:56.277 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:56.277 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.278 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.278 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.278 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.278 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.278 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.278 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-26T21:58:56.278 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-26T21:58:56.278 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.278 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.278 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.278 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.278 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.278 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.278 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.279 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.279 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.279 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.279 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.279 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.279 INFO:teuthology.orchestra.run.smithi072.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-26T21:58:56.279 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-26T21:58:56.279 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.279 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.279 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.279 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.279 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.279 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.279 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.279 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-26T21:58:56.280 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.280 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.280 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.280 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.280 INFO:teuthology.orchestra.run.smithi072.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-26T21:58:56.280 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-26T21:58:56.280 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.280 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.280 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.280 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "*", 2024-04-26T21:58:56.280 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.280 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.280 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.280 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-26T21:58:56.280 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.280 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.281 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.281 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.281 INFO:teuthology.orchestra.run.smithi072.stdout: "max_count_per_host": { 2024-04-26T21:58:56.281 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "max_count_per_host", 2024-04-26T21:58:56.281 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-26T21:58:56.281 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.281 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.281 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "10", 2024-04-26T21:58:56.281 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.281 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.281 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.281 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "max number of daemons per service per host", 2024-04-26T21:58:56.281 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.281 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.281 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.282 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.282 INFO:teuthology.orchestra.run.smithi072.stdout: "max_osd_draining_count": { 2024-04-26T21:58:56.282 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "max_osd_draining_count", 2024-04-26T21:58:56.282 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-26T21:58:56.282 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.282 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.282 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "10", 2024-04-26T21:58:56.282 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.282 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.282 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.282 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-26T21:58:56.282 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.282 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.282 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.283 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.283 INFO:teuthology.orchestra.run.smithi072.stdout: "migration_current": { 2024-04-26T21:58:56.283 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "migration_current", 2024-04-26T21:58:56.283 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-26T21:58:56.283 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.283 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.283 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.283 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.283 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.283 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.283 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "internal - do not modify", 2024-04-26T21:58:56.283 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.283 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.283 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.283 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.284 INFO:teuthology.orchestra.run.smithi072.stdout: "mode": { 2024-04-26T21:58:56.284 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "mode", 2024-04-26T21:58:56.284 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.284 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.284 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.284 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "root", 2024-04-26T21:58:56.284 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.284 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.284 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:56.284 INFO:teuthology.orchestra.run.smithi072.stdout: "cephadm-package", 2024-04-26T21:58:56.284 INFO:teuthology.orchestra.run.smithi072.stdout: "root" 2024-04-26T21:58:56.284 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.284 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "mode for remote execution of cephadm", 2024-04-26T21:58:56.284 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.284 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.284 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.285 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.285 INFO:teuthology.orchestra.run.smithi072.stdout: "oob_default_addr": { 2024-04-26T21:58:56.285 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "oob_default_addr", 2024-04-26T21:58:56.285 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.285 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.285 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.285 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "169.254.1.1", 2024-04-26T21:58:56.285 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.285 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.285 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.285 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-26T21:58:56.285 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.285 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.285 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.286 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.286 INFO:teuthology.orchestra.run.smithi072.stdout: "prometheus_alerts_path": { 2024-04-26T21:58:56.286 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "prometheus_alerts_path", 2024-04-26T21:58:56.286 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.286 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.286 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.286 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-26T21:58:56.286 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.286 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.286 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.286 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-26T21:58:56.286 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.286 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.286 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.287 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.287 INFO:teuthology.orchestra.run.smithi072.stdout: "registry_insecure": { 2024-04-26T21:58:56.287 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "registry_insecure", 2024-04-26T21:58:56.287 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.287 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.287 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.287 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.287 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.287 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.287 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.287 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-26T21:58:56.287 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.287 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.287 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.287 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.288 INFO:teuthology.orchestra.run.smithi072.stdout: "registry_password": { 2024-04-26T21:58:56.288 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "registry_password", 2024-04-26T21:58:56.288 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.288 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.288 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.288 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.288 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.288 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.288 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.288 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-26T21:58:56.288 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.288 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.288 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.288 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.288 INFO:teuthology.orchestra.run.smithi072.stdout: "registry_url": { 2024-04-26T21:58:56.289 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "registry_url", 2024-04-26T21:58:56.289 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.289 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.289 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.289 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.289 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.289 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.289 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.289 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-26T21:58:56.289 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.289 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.289 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.289 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.289 INFO:teuthology.orchestra.run.smithi072.stdout: "registry_username": { 2024-04-26T21:58:56.290 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "registry_username", 2024-04-26T21:58:56.290 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.290 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.290 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.290 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.290 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.290 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.290 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.290 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-26T21:58:56.290 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.290 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.290 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.290 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.290 INFO:teuthology.orchestra.run.smithi072.stdout: "secure_monitoring_stack": { 2024-04-26T21:58:56.291 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "secure_monitoring_stack", 2024-04-26T21:58:56.291 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.291 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.291 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.291 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.291 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.291 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.291 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.291 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-26T21:58:56.291 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.291 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.291 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.291 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.291 INFO:teuthology.orchestra.run.smithi072.stdout: "service_discovery_port": { 2024-04-26T21:58:56.291 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "service_discovery_port", 2024-04-26T21:58:56.292 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-26T21:58:56.292 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.292 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.292 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "8765", 2024-04-26T21:58:56.292 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.292 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.292 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.292 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "cephadm service discovery port", 2024-04-26T21:58:56.292 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.292 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.292 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.292 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.292 INFO:teuthology.orchestra.run.smithi072.stdout: "ssh_config_file": { 2024-04-26T21:58:56.292 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "ssh_config_file", 2024-04-26T21:58:56.292 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.293 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.293 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.293 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.293 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.293 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.293 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.293 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-26T21:58:56.293 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.293 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.293 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.293 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.293 INFO:teuthology.orchestra.run.smithi072.stdout: "use_agent": { 2024-04-26T21:58:56.293 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "use_agent", 2024-04-26T21:58:56.293 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.293 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.294 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.294 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.294 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.294 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.294 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.294 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-26T21:58:56.294 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.294 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.294 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.294 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.294 INFO:teuthology.orchestra.run.smithi072.stdout: "use_repo_digest": { 2024-04-26T21:58:56.294 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "use_repo_digest", 2024-04-26T21:58:56.294 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.294 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.294 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.295 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-26T21:58:56.295 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.295 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.295 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.295 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-26T21:58:56.295 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.295 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.295 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.295 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.295 INFO:teuthology.orchestra.run.smithi072.stdout: "warn_on_failed_host_check": { 2024-04-26T21:58:56.295 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "warn_on_failed_host_check", 2024-04-26T21:58:56.295 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.295 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.295 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.295 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-26T21:58:56.296 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.296 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.296 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.296 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "raise a health warning if the host check fails", 2024-04-26T21:58:56.296 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.296 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.296 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.296 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.296 INFO:teuthology.orchestra.run.smithi072.stdout: "warn_on_stray_daemons": { 2024-04-26T21:58:56.296 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "warn_on_stray_daemons", 2024-04-26T21:58:56.296 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.296 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.296 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.296 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-26T21:58:56.297 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.297 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.297 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.297 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-26T21:58:56.297 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.297 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.297 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.297 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.297 INFO:teuthology.orchestra.run.smithi072.stdout: "warn_on_stray_hosts": { 2024-04-26T21:58:56.297 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "warn_on_stray_hosts", 2024-04-26T21:58:56.297 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.297 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.297 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.297 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-26T21:58:56.297 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.298 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.298 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.298 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-26T21:58:56.298 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.298 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.298 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.298 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:56.298 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:56.298 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.298 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-26T21:58:56.298 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "crash", 2024-04-26T21:58:56.298 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-26T21:58:56.298 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-26T21:58:56.298 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-26T21:58:56.298 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-26T21:58:56.299 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-26T21:58:56.299 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.299 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.299 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.299 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.299 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.299 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.299 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:56.299 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:56.299 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:56.299 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:56.299 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:56.299 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:56.299 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:56.299 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.300 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.300 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.300 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.300 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.300 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.300 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-26T21:58:56.300 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-26T21:58:56.300 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.300 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.300 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.300 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.300 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.300 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.300 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.300 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.301 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.301 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.301 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.301 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.301 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-26T21:58:56.301 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-26T21:58:56.301 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.301 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.301 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.301 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-26T21:58:56.301 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.301 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.301 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:56.301 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:56.301 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:56.302 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:56.302 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:56.302 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:56.302 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:56.302 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.302 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.302 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.302 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.302 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.302 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.302 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-26T21:58:56.302 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-26T21:58:56.302 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.302 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.303 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.303 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.303 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.303 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.303 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.303 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.303 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.303 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.303 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.303 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.303 INFO:teuthology.orchestra.run.smithi072.stdout: "retain_interval": { 2024-04-26T21:58:56.303 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "retain_interval", 2024-04-26T21:58:56.303 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "secs", 2024-04-26T21:58:56.303 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.303 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.303 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "31536000", 2024-04-26T21:58:56.304 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.304 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.304 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.304 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-26T21:58:56.304 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.304 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.304 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.304 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.304 INFO:teuthology.orchestra.run.smithi072.stdout: "warn_recent_interval": { 2024-04-26T21:58:56.304 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "warn_recent_interval", 2024-04-26T21:58:56.304 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "secs", 2024-04-26T21:58:56.304 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.304 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.304 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "1209600", 2024-04-26T21:58:56.304 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.305 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.305 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.305 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-26T21:58:56.305 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.305 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.305 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.305 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:56.305 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:56.305 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.305 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-26T21:58:56.305 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "dashboard", 2024-04-26T21:58:56.305 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-26T21:58:56.305 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-26T21:58:56.305 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-26T21:58:56.305 INFO:teuthology.orchestra.run.smithi072.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-26T21:58:56.306 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-26T21:58:56.306 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-26T21:58:56.306 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.306 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.306 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "10", 2024-04-26T21:58:56.306 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.306 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.306 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.306 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.306 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.306 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.306 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.306 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.306 INFO:teuthology.orchestra.run.smithi072.stdout: "ALERTMANAGER_API_HOST": { 2024-04-26T21:58:56.306 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-26T21:58:56.307 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.307 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.307 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.307 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.307 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.307 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.307 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.307 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.307 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.307 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.307 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.307 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.307 INFO:teuthology.orchestra.run.smithi072.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-26T21:58:56.307 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-26T21:58:56.308 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.308 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.308 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.308 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-26T21:58:56.308 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.308 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.308 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.308 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.308 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.308 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.308 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.308 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.308 INFO:teuthology.orchestra.run.smithi072.stdout: "AUDIT_API_ENABLED": { 2024-04-26T21:58:56.308 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "AUDIT_API_ENABLED", 2024-04-26T21:58:56.308 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.308 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.309 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.309 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.309 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.309 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.309 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.309 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.309 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.309 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.309 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.309 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.309 INFO:teuthology.orchestra.run.smithi072.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-26T21:58:56.309 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-26T21:58:56.309 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.309 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.310 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.310 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-26T21:58:56.310 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.310 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.310 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.310 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.310 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.310 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.310 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.310 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.310 INFO:teuthology.orchestra.run.smithi072.stdout: "ENABLE_BROWSABLE_API": { 2024-04-26T21:58:56.310 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-26T21:58:56.310 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.310 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.311 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.311 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-26T21:58:56.311 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.311 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.311 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.311 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.311 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.311 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.311 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.311 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.311 INFO:teuthology.orchestra.run.smithi072.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-26T21:58:56.311 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-26T21:58:56.311 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.311 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.311 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.312 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-26T21:58:56.312 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.312 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.312 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.312 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.312 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.312 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.312 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.312 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.312 INFO:teuthology.orchestra.run.smithi072.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-26T21:58:56.312 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-26T21:58:56.312 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.312 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.312 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.313 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-26T21:58:56.313 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.313 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.313 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.313 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.313 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.313 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.313 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.313 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.313 INFO:teuthology.orchestra.run.smithi072.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-26T21:58:56.313 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-26T21:58:56.313 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.313 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.313 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.313 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-26T21:58:56.314 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.314 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.314 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.314 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.314 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.314 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.314 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.314 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.314 INFO:teuthology.orchestra.run.smithi072.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-26T21:58:56.314 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-26T21:58:56.314 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.314 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.314 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.314 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-26T21:58:56.315 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.315 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.315 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.315 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.315 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.315 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.315 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.315 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.315 INFO:teuthology.orchestra.run.smithi072.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-26T21:58:56.315 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-26T21:58:56.315 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.315 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.315 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.315 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-26T21:58:56.316 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.316 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.316 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.316 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.316 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.316 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.316 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.316 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.316 INFO:teuthology.orchestra.run.smithi072.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-26T21:58:56.316 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-26T21:58:56.316 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.316 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.316 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.316 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-26T21:58:56.316 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.317 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.317 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.317 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.317 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.317 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.317 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.317 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.317 INFO:teuthology.orchestra.run.smithi072.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-26T21:58:56.317 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-26T21:58:56.317 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.317 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.317 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.317 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-26T21:58:56.317 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.318 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.318 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.318 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.318 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.318 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.318 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.318 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.318 INFO:teuthology.orchestra.run.smithi072.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-26T21:58:56.318 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-26T21:58:56.318 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.318 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.318 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.318 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.318 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.318 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.319 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.319 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.319 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.319 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.319 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.319 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.319 INFO:teuthology.orchestra.run.smithi072.stdout: "GRAFANA_API_PASSWORD": { 2024-04-26T21:58:56.319 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-26T21:58:56.319 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.319 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.319 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.319 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "admin", 2024-04-26T21:58:56.319 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.319 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.319 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.320 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.320 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.320 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.320 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.320 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.320 INFO:teuthology.orchestra.run.smithi072.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-26T21:58:56.320 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-26T21:58:56.320 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.320 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.320 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.320 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-26T21:58:56.320 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.320 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.320 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.320 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.321 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.321 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.321 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.321 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.321 INFO:teuthology.orchestra.run.smithi072.stdout: "GRAFANA_API_URL": { 2024-04-26T21:58:56.321 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "GRAFANA_API_URL", 2024-04-26T21:58:56.321 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.321 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.321 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.321 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.321 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.321 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.321 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.321 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.321 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.321 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.322 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.322 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.322 INFO:teuthology.orchestra.run.smithi072.stdout: "GRAFANA_API_USERNAME": { 2024-04-26T21:58:56.322 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-26T21:58:56.322 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.322 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.322 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.322 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "admin", 2024-04-26T21:58:56.322 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.322 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.322 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.322 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.322 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.323 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.323 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.323 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.323 INFO:teuthology.orchestra.run.smithi072.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-26T21:58:56.323 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-26T21:58:56.323 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.323 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.323 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.323 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.323 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.323 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.323 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.323 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.323 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.323 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.324 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.324 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.324 INFO:teuthology.orchestra.run.smithi072.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-26T21:58:56.324 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-26T21:58:56.324 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.324 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.324 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.324 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.324 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.324 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.324 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.324 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.324 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.324 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.324 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.325 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.325 INFO:teuthology.orchestra.run.smithi072.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-26T21:58:56.325 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-26T21:58:56.325 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.325 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.325 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.325 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-26T21:58:56.325 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.325 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.325 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.325 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.325 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.325 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.325 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.325 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.326 INFO:teuthology.orchestra.run.smithi072.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-26T21:58:56.326 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-26T21:58:56.326 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.326 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.326 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.326 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.326 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.326 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.326 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.326 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.326 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.326 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.326 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.326 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.327 INFO:teuthology.orchestra.run.smithi072.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-26T21:58:56.327 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-26T21:58:56.327 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.327 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.327 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.327 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "[]", 2024-04-26T21:58:56.327 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.327 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.327 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.327 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.327 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.327 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.327 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.327 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.327 INFO:teuthology.orchestra.run.smithi072.stdout: "MULTICLUSTER_CONFIG": { 2024-04-26T21:58:56.328 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-26T21:58:56.328 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.328 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.328 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.328 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "{}", 2024-04-26T21:58:56.328 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.328 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.328 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.328 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.328 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.328 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.328 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.328 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.328 INFO:teuthology.orchestra.run.smithi072.stdout: "PROMETHEUS_API_HOST": { 2024-04-26T21:58:56.328 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-26T21:58:56.329 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.329 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.329 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.329 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.329 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.329 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.329 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.329 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.329 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.329 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.329 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.329 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.329 INFO:teuthology.orchestra.run.smithi072.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-26T21:58:56.329 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-26T21:58:56.329 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.330 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.330 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.330 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-26T21:58:56.330 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.330 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.330 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.330 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.330 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.330 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.330 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.330 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.330 INFO:teuthology.orchestra.run.smithi072.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-26T21:58:56.330 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-26T21:58:56.330 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.331 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.331 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.331 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.331 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.331 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.331 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.331 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.331 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.331 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.331 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.331 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.331 INFO:teuthology.orchestra.run.smithi072.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-26T21:58:56.331 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-26T21:58:56.331 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.331 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.332 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.332 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.332 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.332 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.332 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.332 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.332 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.332 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.332 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.332 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.332 INFO:teuthology.orchestra.run.smithi072.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-26T21:58:56.332 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-26T21:58:56.332 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.332 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.332 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.333 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-26T21:58:56.333 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.333 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.333 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.333 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.333 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.333 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.333 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.333 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.333 INFO:teuthology.orchestra.run.smithi072.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-26T21:58:56.333 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-26T21:58:56.333 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.333 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.333 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.333 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-26T21:58:56.334 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.334 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.334 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.334 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.334 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.334 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.334 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.334 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.334 INFO:teuthology.orchestra.run.smithi072.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-26T21:58:56.334 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-26T21:58:56.334 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.334 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.334 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.334 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.335 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.335 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.335 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.335 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.335 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.335 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.335 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.335 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.335 INFO:teuthology.orchestra.run.smithi072.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-26T21:58:56.335 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-26T21:58:56.335 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.335 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.335 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.335 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.336 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.336 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.336 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.336 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.336 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.336 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.336 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.336 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.336 INFO:teuthology.orchestra.run.smithi072.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-26T21:58:56.336 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-26T21:58:56.336 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.336 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.336 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.336 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.336 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.337 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.337 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.337 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.337 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.337 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.337 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.337 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.337 INFO:teuthology.orchestra.run.smithi072.stdout: "PWD_POLICY_ENABLED": { 2024-04-26T21:58:56.337 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-26T21:58:56.337 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.337 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.337 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.337 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-26T21:58:56.337 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.337 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.338 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.338 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.338 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.338 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.338 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.338 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.338 INFO:teuthology.orchestra.run.smithi072.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-26T21:58:56.338 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-26T21:58:56.338 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.338 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.338 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.338 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-26T21:58:56.338 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.338 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.339 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.339 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.339 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.339 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.339 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.339 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.339 INFO:teuthology.orchestra.run.smithi072.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-26T21:58:56.339 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-26T21:58:56.339 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-26T21:58:56.339 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.339 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.339 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "10", 2024-04-26T21:58:56.339 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.339 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.339 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.340 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.340 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.340 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.340 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.340 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.340 INFO:teuthology.orchestra.run.smithi072.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-26T21:58:56.340 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-26T21:58:56.340 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-26T21:58:56.340 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.340 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.340 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "8", 2024-04-26T21:58:56.340 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.340 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.340 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.340 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.341 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.341 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.341 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.341 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.341 INFO:teuthology.orchestra.run.smithi072.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-26T21:58:56.341 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-26T21:58:56.341 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-26T21:58:56.341 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.341 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.341 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "45", 2024-04-26T21:58:56.341 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.341 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.341 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.341 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.341 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.342 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.342 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.342 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.342 INFO:teuthology.orchestra.run.smithi072.stdout: "RGW_API_ACCESS_KEY": { 2024-04-26T21:58:56.342 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-26T21:58:56.342 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.342 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.342 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.342 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.342 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.342 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.342 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.342 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.342 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.343 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.343 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.343 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.343 INFO:teuthology.orchestra.run.smithi072.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-26T21:58:56.343 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-26T21:58:56.343 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.343 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.343 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.343 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "admin", 2024-04-26T21:58:56.343 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.343 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.343 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.343 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.343 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.343 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.344 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.344 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.344 INFO:teuthology.orchestra.run.smithi072.stdout: "RGW_API_SECRET_KEY": { 2024-04-26T21:58:56.344 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-26T21:58:56.344 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.344 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.344 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.344 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.344 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.344 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.344 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.344 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.344 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.344 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.344 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.345 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.345 INFO:teuthology.orchestra.run.smithi072.stdout: "RGW_API_SSL_VERIFY": { 2024-04-26T21:58:56.345 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-26T21:58:56.345 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.345 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.345 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.345 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-26T21:58:56.345 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.345 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.345 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.345 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.345 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.345 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.345 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.345 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.346 INFO:teuthology.orchestra.run.smithi072.stdout: "UNSAFE_TLS_v1_2": { 2024-04-26T21:58:56.346 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-26T21:58:56.346 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.346 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.346 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.346 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.346 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.346 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.346 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.346 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.346 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.346 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.346 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.346 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.347 INFO:teuthology.orchestra.run.smithi072.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-26T21:58:56.347 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-26T21:58:56.347 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-26T21:58:56.347 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.347 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.347 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "0", 2024-04-26T21:58:56.347 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.347 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.347 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.347 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.347 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.347 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.347 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.347 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.348 INFO:teuthology.orchestra.run.smithi072.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-26T21:58:56.348 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-26T21:58:56.348 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-26T21:58:56.348 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.348 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.348 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "10", 2024-04-26T21:58:56.348 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.348 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.348 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.348 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.348 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.348 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.348 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.348 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.349 INFO:teuthology.orchestra.run.smithi072.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-26T21:58:56.349 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-26T21:58:56.349 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-26T21:58:56.349 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.349 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.349 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "5", 2024-04-26T21:58:56.349 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.349 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.349 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.349 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.349 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.349 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.349 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.349 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.349 INFO:teuthology.orchestra.run.smithi072.stdout: "cross_origin_url": { 2024-04-26T21:58:56.350 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "cross_origin_url", 2024-04-26T21:58:56.350 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.350 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.350 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.350 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.350 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.350 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.350 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.350 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.350 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.350 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.350 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.350 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.350 INFO:teuthology.orchestra.run.smithi072.stdout: "crt_file": { 2024-04-26T21:58:56.351 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "crt_file", 2024-04-26T21:58:56.351 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.354 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.354 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.354 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.354 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.354 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.354 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.354 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.354 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.354 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.354 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.354 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.355 INFO:teuthology.orchestra.run.smithi072.stdout: "debug": { 2024-04-26T21:58:56.355 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "debug", 2024-04-26T21:58:56.355 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.355 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.355 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.355 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.355 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.355 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.355 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.355 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Enable/disable debug options", 2024-04-26T21:58:56.355 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.355 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.355 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.355 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.355 INFO:teuthology.orchestra.run.smithi072.stdout: "jwt_token_ttl": { 2024-04-26T21:58:56.356 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "jwt_token_ttl", 2024-04-26T21:58:56.356 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-26T21:58:56.356 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.356 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.356 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "28800", 2024-04-26T21:58:56.356 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.356 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.356 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.356 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.356 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.356 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.356 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.356 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.356 INFO:teuthology.orchestra.run.smithi072.stdout: "key_file": { 2024-04-26T21:58:56.356 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "key_file", 2024-04-26T21:58:56.357 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.357 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.357 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.357 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.357 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.357 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.357 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.357 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.357 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.357 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.357 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.357 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.357 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-26T21:58:56.357 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-26T21:58:56.357 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.358 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.358 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.358 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.358 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.358 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.358 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:56.358 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:56.358 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:56.358 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:56.358 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:56.358 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:56.358 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:56.358 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.358 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.359 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.359 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.359 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.359 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.359 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-26T21:58:56.359 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-26T21:58:56.359 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.359 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.359 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.359 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.359 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.359 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.359 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.359 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.359 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.360 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.360 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.360 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.360 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-26T21:58:56.360 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-26T21:58:56.360 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.360 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.360 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.360 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-26T21:58:56.360 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.360 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.360 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:56.360 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:56.360 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:56.360 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:56.361 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:56.361 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:56.361 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:56.361 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.361 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.361 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.361 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.361 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.361 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.361 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-26T21:58:56.361 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-26T21:58:56.361 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.361 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.361 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.361 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.362 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.362 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.362 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.362 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.362 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.362 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.362 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.362 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.362 INFO:teuthology.orchestra.run.smithi072.stdout: "motd": { 2024-04-26T21:58:56.362 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "motd", 2024-04-26T21:58:56.362 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.362 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.362 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.362 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.362 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.363 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.363 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.363 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "The message of the day", 2024-04-26T21:58:56.363 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.363 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.363 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.363 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.363 INFO:teuthology.orchestra.run.smithi072.stdout: "redirect_resolve_ip_addr": { 2024-04-26T21:58:56.363 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "redirect_resolve_ip_addr", 2024-04-26T21:58:56.363 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.363 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.363 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.363 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.364 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.364 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.364 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.364 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.364 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.364 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.364 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.364 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.364 INFO:teuthology.orchestra.run.smithi072.stdout: "server_addr": { 2024-04-26T21:58:56.364 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "server_addr", 2024-04-26T21:58:56.364 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.364 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.364 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.364 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "::", 2024-04-26T21:58:56.364 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.365 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.365 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.365 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.365 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.365 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.365 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.365 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.365 INFO:teuthology.orchestra.run.smithi072.stdout: "server_port": { 2024-04-26T21:58:56.365 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "server_port", 2024-04-26T21:58:56.365 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-26T21:58:56.365 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.365 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.365 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "8080", 2024-04-26T21:58:56.365 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.365 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.366 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.366 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.366 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.366 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.366 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.366 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.366 INFO:teuthology.orchestra.run.smithi072.stdout: "ssl": { 2024-04-26T21:58:56.366 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "ssl", 2024-04-26T21:58:56.366 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.366 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.366 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.366 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-26T21:58:56.366 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.366 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.367 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.367 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.367 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.367 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.367 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.367 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.367 INFO:teuthology.orchestra.run.smithi072.stdout: "ssl_server_port": { 2024-04-26T21:58:56.367 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "ssl_server_port", 2024-04-26T21:58:56.367 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-26T21:58:56.367 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.367 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.367 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "8443", 2024-04-26T21:58:56.367 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.367 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.367 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.368 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.368 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.368 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.368 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.368 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.368 INFO:teuthology.orchestra.run.smithi072.stdout: "standby_behaviour": { 2024-04-26T21:58:56.368 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "standby_behaviour", 2024-04-26T21:58:56.368 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.368 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.368 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.368 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "redirect", 2024-04-26T21:58:56.368 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.368 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.368 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:56.368 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:56.369 INFO:teuthology.orchestra.run.smithi072.stdout: "redirect" 2024-04-26T21:58:56.369 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.369 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.369 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.369 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.369 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.369 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.369 INFO:teuthology.orchestra.run.smithi072.stdout: "standby_error_status_code": { 2024-04-26T21:58:56.369 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "standby_error_status_code", 2024-04-26T21:58:56.369 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-26T21:58:56.369 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.369 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.369 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "500", 2024-04-26T21:58:56.369 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "400", 2024-04-26T21:58:56.369 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "599", 2024-04-26T21:58:56.370 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.370 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.370 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.370 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.370 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.370 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.370 INFO:teuthology.orchestra.run.smithi072.stdout: "url_prefix": { 2024-04-26T21:58:56.370 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "url_prefix", 2024-04-26T21:58:56.370 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.370 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.370 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.370 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.370 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.370 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.370 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.371 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.371 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.371 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.371 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.371 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:56.371 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:56.371 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.371 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-26T21:58:56.371 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "devicehealth", 2024-04-26T21:58:56.371 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-26T21:58:56.371 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-26T21:58:56.371 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-26T21:58:56.371 INFO:teuthology.orchestra.run.smithi072.stdout: "enable_monitoring": { 2024-04-26T21:58:56.371 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "enable_monitoring", 2024-04-26T21:58:56.371 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.372 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.372 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.372 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-26T21:58:56.372 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.372 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.372 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.372 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "monitor device health metrics", 2024-04-26T21:58:56.372 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.372 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.372 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.372 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.372 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-26T21:58:56.372 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-26T21:58:56.372 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.372 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.373 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.373 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.373 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.373 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.373 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:56.373 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:56.373 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:56.373 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:56.373 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:56.373 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:56.373 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:56.373 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.373 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.373 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.373 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.374 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.374 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.374 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-26T21:58:56.374 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-26T21:58:56.374 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.374 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.374 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.374 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.374 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.374 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.374 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.374 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.374 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.374 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.374 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.375 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.375 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-26T21:58:56.375 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-26T21:58:56.375 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.375 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.375 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.375 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-26T21:58:56.375 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.375 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.375 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:56.375 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:56.375 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:56.375 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:56.375 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:56.375 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:56.376 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:56.376 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.376 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.376 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.376 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.376 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.376 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.376 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-26T21:58:56.376 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-26T21:58:56.376 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.376 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.376 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.376 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.377 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.377 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.377 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.377 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.377 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.377 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.377 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.377 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.377 INFO:teuthology.orchestra.run.smithi072.stdout: "mark_out_threshold": { 2024-04-26T21:58:56.377 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "mark_out_threshold", 2024-04-26T21:58:56.377 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "secs", 2024-04-26T21:58:56.377 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.377 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.377 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "2419200", 2024-04-26T21:58:56.378 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.378 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.378 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.378 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-26T21:58:56.378 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.378 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.378 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.378 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.378 INFO:teuthology.orchestra.run.smithi072.stdout: "pool_name": { 2024-04-26T21:58:56.378 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "pool_name", 2024-04-26T21:58:56.378 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.378 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.378 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.378 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "device_health_metrics", 2024-04-26T21:58:56.379 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.379 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.379 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.379 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-26T21:58:56.379 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.379 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.379 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.379 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.379 INFO:teuthology.orchestra.run.smithi072.stdout: "retention_period": { 2024-04-26T21:58:56.379 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "retention_period", 2024-04-26T21:58:56.379 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "secs", 2024-04-26T21:58:56.379 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.379 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.379 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "15552000", 2024-04-26T21:58:56.379 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.380 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.380 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.380 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "how long to retain device health metrics", 2024-04-26T21:58:56.380 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.380 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.380 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.380 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.380 INFO:teuthology.orchestra.run.smithi072.stdout: "scrape_frequency": { 2024-04-26T21:58:56.380 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "scrape_frequency", 2024-04-26T21:58:56.380 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "secs", 2024-04-26T21:58:56.380 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.380 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.380 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "86400", 2024-04-26T21:58:56.381 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.381 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.381 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.381 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-26T21:58:56.381 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.381 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.381 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.381 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.381 INFO:teuthology.orchestra.run.smithi072.stdout: "self_heal": { 2024-04-26T21:58:56.381 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "self_heal", 2024-04-26T21:58:56.381 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.381 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.381 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.381 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-26T21:58:56.381 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.382 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.382 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.382 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-26T21:58:56.382 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.382 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.382 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.382 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.382 INFO:teuthology.orchestra.run.smithi072.stdout: "sleep_interval": { 2024-04-26T21:58:56.382 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "sleep_interval", 2024-04-26T21:58:56.382 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "secs", 2024-04-26T21:58:56.382 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.382 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.382 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "600", 2024-04-26T21:58:56.382 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.382 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.383 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.383 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "how frequently to wake up and check device health", 2024-04-26T21:58:56.383 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.383 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.383 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.383 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.383 INFO:teuthology.orchestra.run.smithi072.stdout: "warn_threshold": { 2024-04-26T21:58:56.383 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "warn_threshold", 2024-04-26T21:58:56.383 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "secs", 2024-04-26T21:58:56.383 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.383 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.383 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "7257600", 2024-04-26T21:58:56.383 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.383 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.384 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.384 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-26T21:58:56.384 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.384 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.384 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.384 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:56.384 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:56.384 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.384 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-26T21:58:56.384 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "diskprediction_local", 2024-04-26T21:58:56.384 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-26T21:58:56.384 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-26T21:58:56.384 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-26T21:58:56.384 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-26T21:58:56.385 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-26T21:58:56.385 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.385 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.385 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.385 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.385 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.385 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.385 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:56.385 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:56.385 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:56.385 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:56.385 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:56.385 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:56.385 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:56.385 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.386 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.386 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.386 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.386 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.386 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.386 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-26T21:58:56.386 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-26T21:58:56.386 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.386 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.386 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.386 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.386 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.386 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.386 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.387 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.387 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.387 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.387 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.387 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.387 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-26T21:58:56.387 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-26T21:58:56.387 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.387 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.387 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.387 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-26T21:58:56.387 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.387 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.387 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:56.387 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:56.388 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:56.388 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:56.388 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:56.388 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:56.388 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:56.388 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.388 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.388 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.388 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.388 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.388 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.388 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-26T21:58:56.388 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-26T21:58:56.388 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.388 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.389 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.389 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.389 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.389 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.389 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.389 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.389 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.389 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.389 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.389 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.389 INFO:teuthology.orchestra.run.smithi072.stdout: "predict_interval": { 2024-04-26T21:58:56.389 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "predict_interval", 2024-04-26T21:58:56.389 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.389 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.390 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.390 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "86400", 2024-04-26T21:58:56.390 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.390 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.390 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.390 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.390 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.390 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.390 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.390 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.390 INFO:teuthology.orchestra.run.smithi072.stdout: "predictor_model": { 2024-04-26T21:58:56.390 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "predictor_model", 2024-04-26T21:58:56.390 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.390 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.391 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.391 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "prophetstor", 2024-04-26T21:58:56.391 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.391 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.391 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.391 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.391 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.391 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.391 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.391 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.391 INFO:teuthology.orchestra.run.smithi072.stdout: "sleep_interval": { 2024-04-26T21:58:56.391 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "sleep_interval", 2024-04-26T21:58:56.391 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.391 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.392 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.392 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "600", 2024-04-26T21:58:56.392 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.392 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.392 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.392 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.392 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.392 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.392 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.392 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:56.392 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:56.392 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.392 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-26T21:58:56.392 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "influx", 2024-04-26T21:58:56.392 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": false, 2024-04-26T21:58:56.393 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "influxdb python module not found", 2024-04-26T21:58:56.393 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-26T21:58:56.393 INFO:teuthology.orchestra.run.smithi072.stdout: "batch_size": { 2024-04-26T21:58:56.393 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "batch_size", 2024-04-26T21:58:56.393 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-26T21:58:56.393 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.393 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.393 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "5000", 2024-04-26T21:58:56.393 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.393 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.393 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.393 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-26T21:58:56.393 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.393 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.393 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.394 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.394 INFO:teuthology.orchestra.run.smithi072.stdout: "database": { 2024-04-26T21:58:56.394 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "database", 2024-04-26T21:58:56.394 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.394 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.394 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.394 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "ceph", 2024-04-26T21:58:56.394 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.394 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.394 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.394 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-26T21:58:56.394 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.394 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.394 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.395 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.395 INFO:teuthology.orchestra.run.smithi072.stdout: "hostname": { 2024-04-26T21:58:56.395 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "hostname", 2024-04-26T21:58:56.395 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.395 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.395 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.395 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.395 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.395 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.395 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.395 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "InfluxDB server hostname", 2024-04-26T21:58:56.395 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.395 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.395 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.395 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.396 INFO:teuthology.orchestra.run.smithi072.stdout: "interval": { 2024-04-26T21:58:56.396 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "interval", 2024-04-26T21:58:56.396 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "secs", 2024-04-26T21:58:56.396 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.396 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.396 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "30", 2024-04-26T21:58:56.396 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "5", 2024-04-26T21:58:56.396 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.396 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.396 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-26T21:58:56.396 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.396 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.396 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.396 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.396 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-26T21:58:56.397 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-26T21:58:56.397 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.397 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.397 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.397 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.397 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.397 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.397 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:56.397 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:56.397 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:56.397 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:56.397 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:56.397 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:56.397 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:56.397 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.398 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.398 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.398 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.398 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.398 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.398 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-26T21:58:56.398 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-26T21:58:56.398 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.398 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.398 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.398 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.398 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.398 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.398 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.398 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.399 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.399 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.399 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.399 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.399 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-26T21:58:56.399 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-26T21:58:56.399 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.399 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.399 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.399 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-26T21:58:56.399 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.399 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.399 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:56.399 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:56.399 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:56.400 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:56.400 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:56.400 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:56.400 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:56.400 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.400 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.400 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.400 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.400 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.400 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.400 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-26T21:58:56.400 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-26T21:58:56.400 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.400 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.400 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.401 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.401 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.401 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.401 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.401 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.401 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.401 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.401 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.401 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.401 INFO:teuthology.orchestra.run.smithi072.stdout: "password": { 2024-04-26T21:58:56.401 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "password", 2024-04-26T21:58:56.401 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.401 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.401 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.402 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.402 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.402 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.402 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.402 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "password of InfluxDB server user", 2024-04-26T21:58:56.402 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.402 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.402 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.402 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.402 INFO:teuthology.orchestra.run.smithi072.stdout: "port": { 2024-04-26T21:58:56.402 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "port", 2024-04-26T21:58:56.402 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-26T21:58:56.402 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.402 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.402 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "8086", 2024-04-26T21:58:56.403 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.403 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.403 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.403 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "InfluxDB server port", 2024-04-26T21:58:56.403 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.403 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.403 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.403 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.403 INFO:teuthology.orchestra.run.smithi072.stdout: "ssl": { 2024-04-26T21:58:56.403 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "ssl", 2024-04-26T21:58:56.403 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.403 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.403 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.403 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "false", 2024-04-26T21:58:56.403 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.404 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.404 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.404 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-26T21:58:56.404 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.404 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.404 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.404 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.404 INFO:teuthology.orchestra.run.smithi072.stdout: "threads": { 2024-04-26T21:58:56.404 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "threads", 2024-04-26T21:58:56.404 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-26T21:58:56.404 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.404 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.404 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "5", 2024-04-26T21:58:56.404 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "1", 2024-04-26T21:58:56.404 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "32", 2024-04-26T21:58:56.405 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.405 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-26T21:58:56.405 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.405 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.405 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.405 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.405 INFO:teuthology.orchestra.run.smithi072.stdout: "username": { 2024-04-26T21:58:56.405 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "username", 2024-04-26T21:58:56.405 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.405 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.405 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.405 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.405 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.405 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.406 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.406 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "username of InfluxDB server user", 2024-04-26T21:58:56.406 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.406 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.406 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.406 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.406 INFO:teuthology.orchestra.run.smithi072.stdout: "verify_ssl": { 2024-04-26T21:58:56.406 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "verify_ssl", 2024-04-26T21:58:56.406 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.406 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.406 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.406 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "true", 2024-04-26T21:58:56.406 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.406 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.407 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.407 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-26T21:58:56.407 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.407 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.407 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.407 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:56.407 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:56.407 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.407 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-26T21:58:56.407 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "insights", 2024-04-26T21:58:56.407 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-26T21:58:56.407 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-26T21:58:56.407 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-26T21:58:56.407 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-26T21:58:56.407 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-26T21:58:56.408 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.408 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.408 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.408 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.408 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.408 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.408 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:56.408 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:56.408 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:56.408 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:56.408 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:56.408 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:56.408 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:56.408 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.408 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.408 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.409 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.409 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.409 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.409 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-26T21:58:56.409 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-26T21:58:56.409 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.409 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.409 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.409 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.409 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.409 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.409 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.409 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.409 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.409 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.410 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.410 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.410 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-26T21:58:56.410 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-26T21:58:56.410 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.410 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.410 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.410 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-26T21:58:56.410 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.410 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.410 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:56.410 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:56.410 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:56.410 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:56.411 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:56.411 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:56.411 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:56.411 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.411 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.411 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.411 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.411 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.411 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.411 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-26T21:58:56.411 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-26T21:58:56.411 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.411 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.411 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.411 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.411 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.412 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.412 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.412 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.412 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.412 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.412 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.412 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:56.412 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:56.412 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.412 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-26T21:58:56.412 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "iostat", 2024-04-26T21:58:56.412 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-26T21:58:56.412 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-26T21:58:56.412 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-26T21:58:56.412 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-26T21:58:56.413 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-26T21:58:56.413 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.413 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.413 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.413 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.413 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.413 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.413 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:56.413 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:56.413 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:56.413 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:56.413 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:56.413 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:56.413 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:56.413 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.413 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.414 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.414 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.414 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.414 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.414 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-26T21:58:56.414 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-26T21:58:56.414 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.414 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.414 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.414 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.414 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.414 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.414 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.415 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.415 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.415 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.415 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.415 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.415 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-26T21:58:56.415 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-26T21:58:56.415 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.415 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.415 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.415 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-26T21:58:56.415 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.415 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.415 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:56.415 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:56.416 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:56.416 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:56.416 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:56.416 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:56.416 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:56.416 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.416 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.416 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.416 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.416 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.416 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.416 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-26T21:58:56.416 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-26T21:58:56.416 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.416 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.417 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.417 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.417 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.417 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.417 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.417 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.417 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.417 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.417 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.417 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:56.417 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:56.417 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.417 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-26T21:58:56.417 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "k8sevents", 2024-04-26T21:58:56.418 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-26T21:58:56.418 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-26T21:58:56.418 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-26T21:58:56.418 INFO:teuthology.orchestra.run.smithi072.stdout: "ceph_event_retention_days": { 2024-04-26T21:58:56.418 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "ceph_event_retention_days", 2024-04-26T21:58:56.418 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-26T21:58:56.418 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.418 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.418 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "7", 2024-04-26T21:58:56.418 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.418 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.418 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.418 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-26T21:58:56.418 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.419 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.419 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.419 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.419 INFO:teuthology.orchestra.run.smithi072.stdout: "config_check_secs": { 2024-04-26T21:58:56.419 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "config_check_secs", 2024-04-26T21:58:56.419 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-26T21:58:56.419 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.419 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.419 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "10", 2024-04-26T21:58:56.419 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "10", 2024-04-26T21:58:56.419 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.419 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.419 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-26T21:58:56.419 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.419 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.420 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.420 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.420 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-26T21:58:56.420 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-26T21:58:56.420 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.420 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.420 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.420 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.420 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.420 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.420 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:56.420 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:56.420 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:56.420 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:56.420 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:56.421 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:56.421 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:56.421 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.421 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.421 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.421 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.421 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.421 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.421 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-26T21:58:56.421 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-26T21:58:56.421 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.421 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.421 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.421 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.421 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.422 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.422 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.422 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.422 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.422 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.422 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.422 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.422 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-26T21:58:56.422 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-26T21:58:56.422 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.422 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.422 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.422 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-26T21:58:56.422 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.423 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.423 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:56.423 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:56.423 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:56.423 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:56.423 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:56.423 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:56.423 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:56.423 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.423 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.423 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.423 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.423 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.423 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.423 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-26T21:58:56.424 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-26T21:58:56.424 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.424 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.424 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.424 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.424 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.424 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.424 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.424 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.424 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.424 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.424 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.424 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:56.424 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:56.424 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.425 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-26T21:58:56.425 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "localpool", 2024-04-26T21:58:56.425 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-26T21:58:56.425 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-26T21:58:56.425 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-26T21:58:56.425 INFO:teuthology.orchestra.run.smithi072.stdout: "failure_domain": { 2024-04-26T21:58:56.425 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "failure_domain", 2024-04-26T21:58:56.425 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.425 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.425 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.425 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "host", 2024-04-26T21:58:56.425 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.425 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.425 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.425 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "failure domain for any created local pool", 2024-04-26T21:58:56.426 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-26T21:58:56.426 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.426 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.426 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.426 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-26T21:58:56.426 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-26T21:58:56.426 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.426 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.426 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.426 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.426 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.426 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.426 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:56.426 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:56.426 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:56.427 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:56.427 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:56.427 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:56.427 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:56.427 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.427 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.427 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.427 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.427 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.427 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.427 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-26T21:58:56.428 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-26T21:58:56.428 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.428 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.428 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.428 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.428 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.428 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.428 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.428 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.428 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.428 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.428 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.428 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.428 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-26T21:58:56.428 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-26T21:58:56.429 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.429 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.429 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.429 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-26T21:58:56.429 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.429 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.429 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:56.429 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:56.429 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:56.429 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:56.429 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:56.429 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:56.429 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:56.429 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.429 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.430 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.430 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.430 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.430 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.430 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-26T21:58:56.430 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-26T21:58:56.430 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.430 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.430 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.430 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.430 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.430 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.430 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.430 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.431 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.431 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.431 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.431 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.431 INFO:teuthology.orchestra.run.smithi072.stdout: "min_size": { 2024-04-26T21:58:56.431 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "min_size", 2024-04-26T21:58:56.431 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-26T21:58:56.431 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.431 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.431 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.431 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.431 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.431 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.431 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "default min_size for any created local pool", 2024-04-26T21:58:56.432 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-26T21:58:56.432 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.432 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.432 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.432 INFO:teuthology.orchestra.run.smithi072.stdout: "num_rep": { 2024-04-26T21:58:56.432 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "num_rep", 2024-04-26T21:58:56.432 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-26T21:58:56.432 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.432 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.432 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "3", 2024-04-26T21:58:56.432 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.432 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.432 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.432 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "default replica count for any created local pool", 2024-04-26T21:58:56.433 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.433 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.433 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.433 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.433 INFO:teuthology.orchestra.run.smithi072.stdout: "pg_num": { 2024-04-26T21:58:56.433 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "pg_num", 2024-04-26T21:58:56.433 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-26T21:58:56.433 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.433 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.433 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "128", 2024-04-26T21:58:56.433 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.433 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.433 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.433 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "default pg_num for any created local pool", 2024-04-26T21:58:56.433 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.434 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.434 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.434 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.434 INFO:teuthology.orchestra.run.smithi072.stdout: "prefix": { 2024-04-26T21:58:56.434 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "prefix", 2024-04-26T21:58:56.434 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.434 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.434 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.434 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.434 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.434 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.434 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.434 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "name prefix for any created local pool", 2024-04-26T21:58:56.434 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.435 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.435 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.435 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.435 INFO:teuthology.orchestra.run.smithi072.stdout: "subtree": { 2024-04-26T21:58:56.435 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "subtree", 2024-04-26T21:58:56.435 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.435 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.435 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.435 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "rack", 2024-04-26T21:58:56.435 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.435 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.435 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.435 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-26T21:58:56.435 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-26T21:58:56.435 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.436 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.436 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:56.436 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:56.436 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.436 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-26T21:58:56.436 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "mds_autoscaler", 2024-04-26T21:58:56.436 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-26T21:58:56.436 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-26T21:58:56.436 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-26T21:58:56.436 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-26T21:58:56.436 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-26T21:58:56.436 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.436 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.436 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.436 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.436 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.437 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.437 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:56.437 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:56.437 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:56.437 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:56.437 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:56.437 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:56.437 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:56.437 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.437 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.437 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.437 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.437 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.437 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.437 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-26T21:58:56.438 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-26T21:58:56.438 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.438 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.438 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.438 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.438 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.438 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.438 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.438 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.438 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.438 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.438 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.438 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.438 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-26T21:58:56.439 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-26T21:58:56.439 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.439 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.439 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.439 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-26T21:58:56.439 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.439 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.439 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:56.439 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:56.439 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:56.439 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:56.439 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:56.439 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:56.439 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:56.439 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.440 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.440 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.440 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.440 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.440 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.440 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-26T21:58:56.440 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-26T21:58:56.440 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.440 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.440 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.440 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.440 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.440 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.440 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.441 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.441 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.441 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.441 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.441 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:56.441 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:56.441 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.441 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-26T21:58:56.441 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "mirroring", 2024-04-26T21:58:56.441 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-26T21:58:56.441 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-26T21:58:56.441 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-26T21:58:56.441 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-26T21:58:56.441 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-26T21:58:56.441 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.442 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.442 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.442 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.442 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.442 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.442 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:56.442 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:56.442 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:56.442 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:56.442 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:56.442 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:56.442 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:56.442 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.442 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.442 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.443 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.443 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.443 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.443 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-26T21:58:56.443 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-26T21:58:56.443 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.443 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.443 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.443 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.443 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.443 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.443 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.443 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.443 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.443 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.444 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.444 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.444 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-26T21:58:56.444 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-26T21:58:56.444 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.444 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.444 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.444 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-26T21:58:56.444 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.444 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.444 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:56.444 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:56.444 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:56.444 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:56.444 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:56.445 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:56.445 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:56.445 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.445 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.445 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.445 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.445 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.445 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.445 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-26T21:58:56.445 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-26T21:58:56.445 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.445 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.445 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.445 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.446 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.446 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.446 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.446 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.446 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.446 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.446 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.446 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:56.446 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:56.446 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.446 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-26T21:58:56.446 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "nfs", 2024-04-26T21:58:56.446 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-26T21:58:56.446 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-26T21:58:56.446 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-26T21:58:56.447 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-26T21:58:56.447 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-26T21:58:56.447 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.447 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.447 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.447 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.447 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.447 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.447 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:56.447 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:56.447 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:56.447 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:56.447 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:56.447 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:56.447 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:56.448 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.448 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.448 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.448 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.448 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.448 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.448 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-26T21:58:56.448 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-26T21:58:56.448 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.448 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.448 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.448 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.448 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.448 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.448 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.449 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.449 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.449 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.449 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.449 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.449 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-26T21:58:56.449 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-26T21:58:56.449 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.449 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.449 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.449 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-26T21:58:56.449 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.449 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.449 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:56.450 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:56.450 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:56.450 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:56.450 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:56.450 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:56.450 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:56.450 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.450 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.450 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.450 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.450 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.450 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.450 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-26T21:58:56.450 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-26T21:58:56.450 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.451 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.451 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.451 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.451 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.451 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.451 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.451 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.451 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.451 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.451 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.451 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:56.451 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:56.451 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.451 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-26T21:58:56.451 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "orchestrator", 2024-04-26T21:58:56.451 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-26T21:58:56.452 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-26T21:58:56.452 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-26T21:58:56.452 INFO:teuthology.orchestra.run.smithi072.stdout: "fail_fs": { 2024-04-26T21:58:56.452 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "fail_fs", 2024-04-26T21:58:56.452 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.452 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.452 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.452 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.452 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.452 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.452 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.452 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-26T21:58:56.452 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.452 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.453 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.453 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.453 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-26T21:58:56.453 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-26T21:58:56.453 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.453 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.453 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.453 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.453 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.453 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.453 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:56.453 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:56.453 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:56.453 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:56.453 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:56.453 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:56.454 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:56.454 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.454 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.454 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.454 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.454 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.454 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.454 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-26T21:58:56.454 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-26T21:58:56.454 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.454 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.454 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.454 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.454 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.455 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.455 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.455 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.455 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.455 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.455 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.455 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.455 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-26T21:58:56.455 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-26T21:58:56.455 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.455 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.455 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.455 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-26T21:58:56.455 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.456 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.456 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:56.456 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:56.456 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:56.456 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:56.456 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:56.456 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:56.456 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:56.456 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.456 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.456 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.456 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.456 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.456 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.456 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-26T21:58:56.457 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-26T21:58:56.457 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.457 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.457 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.457 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.457 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.457 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.457 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.457 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.457 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.457 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.457 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.457 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.457 INFO:teuthology.orchestra.run.smithi072.stdout: "orchestrator": { 2024-04-26T21:58:56.457 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "orchestrator", 2024-04-26T21:58:56.458 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.458 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.458 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.458 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.458 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.458 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.458 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:56.458 INFO:teuthology.orchestra.run.smithi072.stdout: "cephadm", 2024-04-26T21:58:56.458 INFO:teuthology.orchestra.run.smithi072.stdout: "rook", 2024-04-26T21:58:56.458 INFO:teuthology.orchestra.run.smithi072.stdout: "test_orchestrator" 2024-04-26T21:58:56.458 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.458 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Orchestrator backend", 2024-04-26T21:58:56.458 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.458 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.459 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.459 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:56.459 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:56.459 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.459 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-26T21:58:56.459 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "osd_perf_query", 2024-04-26T21:58:56.459 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-26T21:58:56.459 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-26T21:58:56.459 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-26T21:58:56.459 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-26T21:58:56.459 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-26T21:58:56.459 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.459 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.459 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.460 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.460 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.460 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.460 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:56.460 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:56.460 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:56.460 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:56.460 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:56.460 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:56.460 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:56.460 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.460 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.460 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.460 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.460 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.461 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.461 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-26T21:58:56.461 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-26T21:58:56.461 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.461 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.461 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.461 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.461 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.461 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.461 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.461 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.461 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.461 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.461 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.461 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.462 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-26T21:58:56.462 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-26T21:58:56.462 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.462 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.462 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.462 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-26T21:58:56.462 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.462 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.462 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:56.462 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:56.462 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:56.462 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:56.462 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:56.462 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:56.462 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:56.463 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.463 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.463 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.463 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.463 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.463 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.463 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-26T21:58:56.463 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-26T21:58:56.463 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.463 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.463 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.463 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.463 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.463 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.463 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.464 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.464 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.464 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.464 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.464 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:56.464 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:56.464 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.464 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-26T21:58:56.464 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "osd_support", 2024-04-26T21:58:56.464 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-26T21:58:56.464 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-26T21:58:56.464 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-26T21:58:56.464 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-26T21:58:56.464 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-26T21:58:56.465 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.465 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.465 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.465 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.465 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.465 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.465 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:56.465 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:56.465 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:56.465 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:56.465 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:56.465 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:56.465 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:56.465 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.465 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.466 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.466 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.466 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.466 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.466 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-26T21:58:56.466 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-26T21:58:56.466 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.466 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.466 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.466 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.466 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.466 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.466 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.466 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.466 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.467 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.467 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.467 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.467 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-26T21:58:56.467 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-26T21:58:56.467 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.467 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.467 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.467 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-26T21:58:56.467 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.467 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.467 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:56.467 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:56.467 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:56.467 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:56.468 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:56.468 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:56.468 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:56.468 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.468 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.468 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.468 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.468 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.468 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.468 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-26T21:58:56.468 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-26T21:58:56.468 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.468 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.468 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.468 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.469 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.469 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.469 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.469 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.469 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.469 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.469 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.469 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:56.469 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:56.469 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.469 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-26T21:58:56.469 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "pg_autoscaler", 2024-04-26T21:58:56.469 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-26T21:58:56.469 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-26T21:58:56.469 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-26T21:58:56.470 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-26T21:58:56.470 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-26T21:58:56.470 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.470 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.470 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.470 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.470 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.470 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.470 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:56.470 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:56.470 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:56.470 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:56.470 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:56.471 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:56.471 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:56.471 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.471 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.471 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.471 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.471 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.471 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.471 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-26T21:58:56.471 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-26T21:58:56.471 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.471 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.471 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.471 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.471 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.472 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.472 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.472 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.472 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.472 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.472 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.472 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.472 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-26T21:58:56.472 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-26T21:58:56.472 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.472 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.472 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.472 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-26T21:58:56.472 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.472 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.473 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:56.473 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:56.473 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:56.473 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:56.473 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:56.473 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:56.473 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:56.473 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.473 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.473 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.473 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.473 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.473 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.473 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-26T21:58:56.474 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-26T21:58:56.474 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.474 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.474 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.474 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.474 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.474 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.474 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.474 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.474 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.474 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.474 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.474 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.474 INFO:teuthology.orchestra.run.smithi072.stdout: "sleep_interval": { 2024-04-26T21:58:56.475 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "sleep_interval", 2024-04-26T21:58:56.475 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "secs", 2024-04-26T21:58:56.475 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.475 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.475 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "60", 2024-04-26T21:58:56.475 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.475 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.475 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.475 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.475 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.475 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.475 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.475 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.475 INFO:teuthology.orchestra.run.smithi072.stdout: "threshold": { 2024-04-26T21:58:56.475 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "threshold", 2024-04-26T21:58:56.476 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "float", 2024-04-26T21:58:56.476 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.476 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.476 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "3.0", 2024-04-26T21:58:56.476 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "1.0", 2024-04-26T21:58:56.476 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.476 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.476 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "scaling threshold", 2024-04-26T21:58:56.476 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-26T21:58:56.476 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.476 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.476 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:56.476 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:56.476 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.476 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-26T21:58:56.476 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "progress", 2024-04-26T21:58:56.477 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-26T21:58:56.477 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-26T21:58:56.477 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-26T21:58:56.477 INFO:teuthology.orchestra.run.smithi072.stdout: "allow_pg_recovery_event": { 2024-04-26T21:58:56.477 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "allow_pg_recovery_event", 2024-04-26T21:58:56.477 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.477 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.477 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.477 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.477 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.477 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.477 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.477 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-26T21:58:56.477 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.477 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.478 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.478 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.478 INFO:teuthology.orchestra.run.smithi072.stdout: "enabled": { 2024-04-26T21:58:56.478 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "enabled", 2024-04-26T21:58:56.478 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.478 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.478 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.478 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-26T21:58:56.478 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.478 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.478 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.478 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.478 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.478 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.478 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.479 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.479 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-26T21:58:56.479 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-26T21:58:56.479 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.479 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.479 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.479 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.479 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.479 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.479 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:56.479 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:56.479 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:56.479 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:56.479 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:56.480 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:56.480 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:56.480 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.480 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.480 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.480 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.480 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.480 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.480 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-26T21:58:56.480 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-26T21:58:56.480 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.480 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.480 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.480 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.481 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.481 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.481 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.481 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.481 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.481 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.481 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.481 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.481 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-26T21:58:56.481 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-26T21:58:56.481 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.481 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.481 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.481 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-26T21:58:56.481 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.482 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.482 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:56.482 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:56.482 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:56.482 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:56.482 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:56.482 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:56.482 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:56.482 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.482 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.482 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.482 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.482 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.482 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.483 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-26T21:58:56.483 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-26T21:58:56.483 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.483 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.483 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.483 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.483 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.483 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.483 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.483 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.483 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.483 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.483 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.483 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.483 INFO:teuthology.orchestra.run.smithi072.stdout: "max_completed_events": { 2024-04-26T21:58:56.484 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "max_completed_events", 2024-04-26T21:58:56.484 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-26T21:58:56.484 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.484 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.484 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "50", 2024-04-26T21:58:56.484 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.484 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.484 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.484 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "number of past completed events to remember", 2024-04-26T21:58:56.484 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.484 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.484 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.484 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.484 INFO:teuthology.orchestra.run.smithi072.stdout: "sleep_interval": { 2024-04-26T21:58:56.484 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "sleep_interval", 2024-04-26T21:58:56.485 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "secs", 2024-04-26T21:58:56.485 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.485 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.485 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "5", 2024-04-26T21:58:56.485 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.485 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.485 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.485 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "how long the module is going to sleep", 2024-04-26T21:58:56.485 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.485 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.485 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.485 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:56.485 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:56.485 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.486 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-26T21:58:56.486 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "prometheus", 2024-04-26T21:58:56.486 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-26T21:58:56.486 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-26T21:58:56.486 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-26T21:58:56.486 INFO:teuthology.orchestra.run.smithi072.stdout: "cache": { 2024-04-26T21:58:56.486 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "cache", 2024-04-26T21:58:56.486 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.486 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.486 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.486 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-26T21:58:56.486 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.486 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.486 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.487 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.487 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.487 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.487 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.487 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.487 INFO:teuthology.orchestra.run.smithi072.stdout: "exclude_perf_counters": { 2024-04-26T21:58:56.487 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "exclude_perf_counters", 2024-04-26T21:58:56.487 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.487 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.487 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.487 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-26T21:58:56.487 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.487 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.487 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.487 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-26T21:58:56.488 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-26T21:58:56.488 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.488 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.488 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.488 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-26T21:58:56.488 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-26T21:58:56.488 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.488 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.488 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.488 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.488 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.488 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.489 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:56.489 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:56.489 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:56.489 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:56.489 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:56.489 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:56.489 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:56.489 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.489 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.489 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.489 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.489 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.489 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.489 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-26T21:58:56.489 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-26T21:58:56.490 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.490 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.490 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.490 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.490 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.490 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.490 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.490 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.490 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.490 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.490 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.490 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.490 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-26T21:58:56.490 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-26T21:58:56.491 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.491 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.491 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.491 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-26T21:58:56.491 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.491 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.491 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:56.491 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:56.491 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:56.491 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:56.491 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:56.491 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:56.491 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:56.491 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.491 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.491 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.492 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.492 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.492 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.492 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-26T21:58:56.492 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-26T21:58:56.492 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.492 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.492 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.492 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.492 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.492 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.492 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.492 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.492 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.493 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.493 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.493 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.493 INFO:teuthology.orchestra.run.smithi072.stdout: "rbd_stats_pools": { 2024-04-26T21:58:56.493 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "rbd_stats_pools", 2024-04-26T21:58:56.493 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.493 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.493 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.493 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.493 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.493 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.493 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.493 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.493 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.494 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.494 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.494 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.494 INFO:teuthology.orchestra.run.smithi072.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-26T21:58:56.494 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-26T21:58:56.494 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-26T21:58:56.494 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.494 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.494 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "300", 2024-04-26T21:58:56.494 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.494 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.494 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.494 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.494 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.495 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.495 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.495 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.495 INFO:teuthology.orchestra.run.smithi072.stdout: "scrape_interval": { 2024-04-26T21:58:56.495 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "scrape_interval", 2024-04-26T21:58:56.495 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "float", 2024-04-26T21:58:56.495 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.495 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.495 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "15.0", 2024-04-26T21:58:56.495 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.495 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.495 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.495 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.495 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.495 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.496 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.496 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.496 INFO:teuthology.orchestra.run.smithi072.stdout: "server_addr": { 2024-04-26T21:58:56.496 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "server_addr", 2024-04-26T21:58:56.496 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.496 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.496 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.496 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "::", 2024-04-26T21:58:56.496 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.496 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.496 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.496 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-26T21:58:56.496 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.496 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.496 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.497 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.497 INFO:teuthology.orchestra.run.smithi072.stdout: "server_port": { 2024-04-26T21:58:56.497 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "server_port", 2024-04-26T21:58:56.497 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-26T21:58:56.497 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.497 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.497 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "9283", 2024-04-26T21:58:56.497 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.497 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.497 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.497 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-26T21:58:56.497 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.497 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.497 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.497 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.498 INFO:teuthology.orchestra.run.smithi072.stdout: "stale_cache_strategy": { 2024-04-26T21:58:56.498 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "stale_cache_strategy", 2024-04-26T21:58:56.498 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.498 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.498 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.498 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "log", 2024-04-26T21:58:56.498 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.498 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.498 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.498 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.498 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.498 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.498 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.498 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.499 INFO:teuthology.orchestra.run.smithi072.stdout: "standby_behaviour": { 2024-04-26T21:58:56.499 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "standby_behaviour", 2024-04-26T21:58:56.499 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.499 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.499 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.499 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "default", 2024-04-26T21:58:56.499 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.499 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.499 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:56.499 INFO:teuthology.orchestra.run.smithi072.stdout: "default", 2024-04-26T21:58:56.499 INFO:teuthology.orchestra.run.smithi072.stdout: "error" 2024-04-26T21:58:56.499 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.499 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.499 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.499 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.500 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.500 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.500 INFO:teuthology.orchestra.run.smithi072.stdout: "standby_error_status_code": { 2024-04-26T21:58:56.500 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "standby_error_status_code", 2024-04-26T21:58:56.500 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-26T21:58:56.500 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.500 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.500 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "500", 2024-04-26T21:58:56.500 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "400", 2024-04-26T21:58:56.500 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "599", 2024-04-26T21:58:56.500 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.500 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.500 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.500 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.500 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.501 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:56.501 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:56.501 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.501 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-26T21:58:56.501 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "rbd_support", 2024-04-26T21:58:56.501 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-26T21:58:56.501 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-26T21:58:56.501 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-26T21:58:56.501 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-26T21:58:56.501 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-26T21:58:56.501 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.501 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.501 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.501 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.501 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.502 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.502 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:56.502 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:56.502 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:56.502 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:56.502 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:56.502 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:56.502 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:56.502 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.502 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.502 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.502 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.502 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.502 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.502 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-26T21:58:56.503 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-26T21:58:56.503 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.503 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.503 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.503 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.503 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.503 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.503 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.503 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.503 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.503 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.503 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.503 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.503 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-26T21:58:56.503 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-26T21:58:56.504 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.504 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.504 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.504 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-26T21:58:56.504 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.504 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.504 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:56.504 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:56.504 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:56.504 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:56.504 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:56.504 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:56.504 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:56.504 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.504 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.504 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.505 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.505 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.505 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.505 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-26T21:58:56.505 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-26T21:58:56.505 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.505 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.505 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.505 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.505 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.505 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.505 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.505 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.505 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.505 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.506 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.506 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.506 INFO:teuthology.orchestra.run.smithi072.stdout: "max_concurrent_snap_create": { 2024-04-26T21:58:56.506 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "max_concurrent_snap_create", 2024-04-26T21:58:56.506 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-26T21:58:56.506 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.506 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.506 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "10", 2024-04-26T21:58:56.506 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.506 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.506 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.506 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.506 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.507 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.507 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.507 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.507 INFO:teuthology.orchestra.run.smithi072.stdout: "mirror_snapshot_schedule": { 2024-04-26T21:58:56.507 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "mirror_snapshot_schedule", 2024-04-26T21:58:56.507 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.507 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.507 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.507 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.507 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.507 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.507 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.507 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.507 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.507 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.508 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.508 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.508 INFO:teuthology.orchestra.run.smithi072.stdout: "trash_purge_schedule": { 2024-04-26T21:58:56.508 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "trash_purge_schedule", 2024-04-26T21:58:56.508 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.508 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.508 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.508 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.508 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.508 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.508 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.508 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.508 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.508 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.508 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.509 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:56.509 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:56.509 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.509 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-26T21:58:56.509 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "restful", 2024-04-26T21:58:56.509 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-26T21:58:56.509 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-26T21:58:56.509 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-26T21:58:56.509 INFO:teuthology.orchestra.run.smithi072.stdout: "enable_auth": { 2024-04-26T21:58:56.509 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "enable_auth", 2024-04-26T21:58:56.509 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.509 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.509 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.509 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-26T21:58:56.510 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.510 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.510 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.510 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.510 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.510 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.510 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.510 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.510 INFO:teuthology.orchestra.run.smithi072.stdout: "key_file": { 2024-04-26T21:58:56.510 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "key_file", 2024-04-26T21:58:56.510 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.510 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.510 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.510 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.510 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.511 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.511 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.511 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.511 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.511 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.511 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.511 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.511 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-26T21:58:56.511 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-26T21:58:56.511 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.511 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.511 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.511 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.511 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.512 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.512 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:56.512 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:56.512 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:56.512 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:56.512 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:56.512 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:56.512 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:56.512 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.512 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.512 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.512 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.512 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.512 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.512 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-26T21:58:56.513 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-26T21:58:56.513 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.513 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.513 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.513 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.513 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.513 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.513 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.513 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.513 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.513 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.513 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.513 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.513 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-26T21:58:56.513 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-26T21:58:56.514 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.514 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.514 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.514 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-26T21:58:56.514 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.514 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.514 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:56.514 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:56.514 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:56.514 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:56.514 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:56.514 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:56.514 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:56.514 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.515 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.515 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.515 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.515 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.515 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.515 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-26T21:58:56.515 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-26T21:58:56.515 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.515 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.515 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.515 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.515 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.515 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.515 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.515 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.516 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.516 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.516 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.516 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.516 INFO:teuthology.orchestra.run.smithi072.stdout: "server_addr": { 2024-04-26T21:58:56.516 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "server_addr", 2024-04-26T21:58:56.516 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.516 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.516 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.516 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.516 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.516 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.516 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.516 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.516 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.516 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.517 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.517 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.517 INFO:teuthology.orchestra.run.smithi072.stdout: "server_port": { 2024-04-26T21:58:56.517 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "server_port", 2024-04-26T21:58:56.517 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.517 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.517 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.517 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.517 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.517 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.517 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.517 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.517 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.517 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.517 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.518 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:56.518 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:56.518 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.518 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-26T21:58:56.518 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "rgw", 2024-04-26T21:58:56.518 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-26T21:58:56.518 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-26T21:58:56.518 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-26T21:58:56.518 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-26T21:58:56.518 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-26T21:58:56.518 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.518 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.518 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.518 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.518 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.519 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.519 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:56.519 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:56.519 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:56.519 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:56.519 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:56.519 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:56.519 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:56.519 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.519 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.519 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.519 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.519 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.520 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.520 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-26T21:58:56.520 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-26T21:58:56.520 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.520 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.520 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.520 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.520 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.520 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.520 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.520 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.520 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.520 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.520 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.520 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.521 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-26T21:58:56.521 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-26T21:58:56.521 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.521 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.521 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.521 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-26T21:58:56.521 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.521 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.521 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:56.521 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:56.521 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:56.521 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:56.521 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:56.521 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:56.521 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:56.522 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.522 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.522 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.522 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.522 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.522 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.522 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-26T21:58:56.522 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-26T21:58:56.522 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.522 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.522 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.522 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.522 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.522 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.523 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.523 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.523 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.523 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.523 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.523 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:56.523 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:56.523 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.523 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-26T21:58:56.523 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "rook", 2024-04-26T21:58:56.523 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-26T21:58:56.523 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-26T21:58:56.523 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-26T21:58:56.523 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-26T21:58:56.524 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-26T21:58:56.524 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.524 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.524 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.524 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.524 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.524 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.524 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:56.524 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:56.524 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:56.524 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:56.524 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:56.524 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:56.524 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:56.524 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.524 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.525 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.525 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.525 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.525 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.525 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-26T21:58:56.525 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-26T21:58:56.525 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.525 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.525 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.525 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.525 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.525 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.525 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.525 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.525 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.526 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.526 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.526 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.526 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-26T21:58:56.526 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-26T21:58:56.526 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.526 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.526 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.526 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-26T21:58:56.526 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.526 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.526 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:56.526 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:56.526 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:56.527 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:56.527 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:56.527 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:56.527 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:56.527 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.527 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.527 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.527 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.527 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.527 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.527 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-26T21:58:56.527 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-26T21:58:56.527 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.527 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.528 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.528 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.528 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.528 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.528 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.528 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.528 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.528 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.528 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.528 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.528 INFO:teuthology.orchestra.run.smithi072.stdout: "storage_class": { 2024-04-26T21:58:56.528 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "storage_class", 2024-04-26T21:58:56.528 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.528 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.528 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.528 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "local", 2024-04-26T21:58:56.529 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.529 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.529 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.529 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-26T21:58:56.529 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.529 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.529 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.529 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:56.529 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:56.529 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.529 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-26T21:58:56.529 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "selftest", 2024-04-26T21:58:56.529 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-26T21:58:56.529 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-26T21:58:56.529 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-26T21:58:56.530 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-26T21:58:56.530 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-26T21:58:56.530 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.530 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.530 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.530 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.530 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.530 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.530 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:56.530 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:56.530 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:56.530 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:56.530 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:56.530 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:56.531 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:56.531 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.531 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.531 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.531 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.531 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.531 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.531 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-26T21:58:56.531 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-26T21:58:56.531 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.531 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.531 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.531 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.531 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.531 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.532 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.532 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.532 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.532 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.532 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.532 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.532 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-26T21:58:56.532 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-26T21:58:56.532 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.532 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.532 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.532 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-26T21:58:56.532 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.532 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.533 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:56.533 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:56.533 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:56.533 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:56.533 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:56.533 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:56.533 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:56.533 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.533 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.533 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.533 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.533 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.533 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.533 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-26T21:58:56.533 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-26T21:58:56.534 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.534 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.534 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.534 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.534 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.534 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.534 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.534 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.534 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.534 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.534 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.534 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.534 INFO:teuthology.orchestra.run.smithi072.stdout: "roption1": { 2024-04-26T21:58:56.534 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "roption1", 2024-04-26T21:58:56.535 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.535 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.535 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.535 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.535 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.535 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.535 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.535 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.535 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.535 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.535 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.535 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.535 INFO:teuthology.orchestra.run.smithi072.stdout: "roption2": { 2024-04-26T21:58:56.535 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "roption2", 2024-04-26T21:58:56.535 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.536 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.536 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.536 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "xyz", 2024-04-26T21:58:56.536 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.536 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.536 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.536 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.536 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.536 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.536 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.536 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.536 INFO:teuthology.orchestra.run.smithi072.stdout: "rwoption1": { 2024-04-26T21:58:56.536 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "rwoption1", 2024-04-26T21:58:56.536 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.536 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.537 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.537 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.537 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.537 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.537 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.537 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.537 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.537 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.537 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.537 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.537 INFO:teuthology.orchestra.run.smithi072.stdout: "rwoption2": { 2024-04-26T21:58:56.537 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "rwoption2", 2024-04-26T21:58:56.537 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-26T21:58:56.537 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.537 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.538 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.538 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.538 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.538 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.538 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.538 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.538 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.538 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.538 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.538 INFO:teuthology.orchestra.run.smithi072.stdout: "rwoption3": { 2024-04-26T21:58:56.538 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "rwoption3", 2024-04-26T21:58:56.538 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "float", 2024-04-26T21:58:56.538 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.538 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.539 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.539 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.539 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.539 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.539 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.539 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.539 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.539 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.539 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.539 INFO:teuthology.orchestra.run.smithi072.stdout: "rwoption4": { 2024-04-26T21:58:56.539 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "rwoption4", 2024-04-26T21:58:56.539 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.539 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.539 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.539 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.540 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.540 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.540 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.540 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.540 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.540 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.540 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.540 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.540 INFO:teuthology.orchestra.run.smithi072.stdout: "rwoption5": { 2024-04-26T21:58:56.540 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "rwoption5", 2024-04-26T21:58:56.540 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.540 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.540 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.540 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.540 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.541 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.541 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.541 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.541 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.541 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.541 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.541 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.541 INFO:teuthology.orchestra.run.smithi072.stdout: "rwoption6": { 2024-04-26T21:58:56.541 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "rwoption6", 2024-04-26T21:58:56.541 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.541 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.541 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.541 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-26T21:58:56.541 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.541 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.542 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.542 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.542 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.542 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.542 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.542 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.542 INFO:teuthology.orchestra.run.smithi072.stdout: "rwoption7": { 2024-04-26T21:58:56.542 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "rwoption7", 2024-04-26T21:58:56.542 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-26T21:58:56.542 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.542 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.542 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.542 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "1", 2024-04-26T21:58:56.542 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "42", 2024-04-26T21:58:56.542 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.543 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.543 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.543 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.543 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.543 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.543 INFO:teuthology.orchestra.run.smithi072.stdout: "testkey": { 2024-04-26T21:58:56.543 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "testkey", 2024-04-26T21:58:56.543 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.543 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.543 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.543 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.543 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.543 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.543 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.543 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.544 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.544 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.544 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.544 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.544 INFO:teuthology.orchestra.run.smithi072.stdout: "testlkey": { 2024-04-26T21:58:56.544 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "testlkey", 2024-04-26T21:58:56.544 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.544 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.544 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.544 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.544 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.544 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.544 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.544 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.544 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.545 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.545 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.545 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.545 INFO:teuthology.orchestra.run.smithi072.stdout: "testnewline": { 2024-04-26T21:58:56.545 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "testnewline", 2024-04-26T21:58:56.545 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.545 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.545 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.545 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.545 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.545 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.545 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.545 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.545 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.545 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.546 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.546 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:56.546 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:56.546 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.546 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-26T21:58:56.546 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "snap_schedule", 2024-04-26T21:58:56.546 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-26T21:58:56.546 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-26T21:58:56.546 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-26T21:58:56.546 INFO:teuthology.orchestra.run.smithi072.stdout: "allow_m_granularity": { 2024-04-26T21:58:56.546 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "allow_m_granularity", 2024-04-26T21:58:56.546 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.546 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.546 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.547 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.547 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.547 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.547 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.547 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "allow minute scheduled snapshots", 2024-04-26T21:58:56.547 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.547 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.547 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.547 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.547 INFO:teuthology.orchestra.run.smithi072.stdout: "dump_on_update": { 2024-04-26T21:58:56.547 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "dump_on_update", 2024-04-26T21:58:56.547 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.547 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.547 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.547 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.548 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.548 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.548 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.548 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "dump database to debug log on update", 2024-04-26T21:58:56.548 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.548 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.548 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.548 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.548 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-26T21:58:56.548 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-26T21:58:56.548 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.548 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.548 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.548 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.548 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.549 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.549 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:56.549 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:56.549 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:56.549 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:56.549 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:56.549 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:56.549 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:56.549 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.549 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.549 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.549 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.549 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.549 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.549 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-26T21:58:56.550 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-26T21:58:56.550 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.550 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.550 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.550 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.550 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.550 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.550 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.550 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.550 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.550 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.550 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.550 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.550 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-26T21:58:56.551 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-26T21:58:56.551 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.551 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.551 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.551 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-26T21:58:56.551 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.551 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.551 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:56.551 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:56.551 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:56.551 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:56.551 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:56.551 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:56.551 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:56.551 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.552 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.552 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.552 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.552 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.552 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.552 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-26T21:58:56.552 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-26T21:58:56.552 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.552 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.552 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.552 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.552 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.552 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.552 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.552 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.553 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.553 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.553 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.553 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:56.553 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:56.553 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.553 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-26T21:58:56.553 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "stats", 2024-04-26T21:58:56.553 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-26T21:58:56.553 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-26T21:58:56.553 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-26T21:58:56.553 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-26T21:58:56.553 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-26T21:58:56.553 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.554 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.554 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.554 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.554 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.554 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.554 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:56.554 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:56.554 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:56.554 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:56.554 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:56.554 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:56.554 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:56.554 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.554 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.554 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.555 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.555 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.555 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.555 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-26T21:58:56.555 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-26T21:58:56.555 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.555 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.555 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.555 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.555 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.555 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.555 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.555 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.555 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.555 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.556 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.556 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.556 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-26T21:58:56.556 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-26T21:58:56.556 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.556 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.556 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.556 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-26T21:58:56.556 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.556 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.556 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:56.556 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:56.556 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:56.556 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:56.556 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:56.557 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:56.557 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:56.557 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.557 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.557 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.557 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.557 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.557 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.557 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-26T21:58:56.557 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-26T21:58:56.557 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.557 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.557 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.557 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.558 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.558 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.558 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.558 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.558 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.558 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.558 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.558 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:56.558 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:56.558 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.558 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-26T21:58:56.558 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "status", 2024-04-26T21:58:56.558 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-26T21:58:56.558 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-26T21:58:56.558 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-26T21:58:56.559 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-26T21:58:56.559 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-26T21:58:56.559 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.559 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.559 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.559 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.559 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.559 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.559 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:56.559 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:56.559 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:56.559 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:56.559 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:56.559 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:56.560 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:56.560 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.560 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.560 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.560 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.560 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.560 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.560 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-26T21:58:56.560 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-26T21:58:56.560 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.560 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.560 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.560 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.560 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.560 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.561 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.561 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.561 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.561 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.561 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.561 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.561 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-26T21:58:56.561 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-26T21:58:56.561 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.561 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.561 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.561 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-26T21:58:56.561 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.561 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.562 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:56.562 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:56.562 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:56.562 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:56.562 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:56.562 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:56.562 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:56.562 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.562 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.562 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.562 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.562 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.562 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.562 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-26T21:58:56.563 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-26T21:58:56.563 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.563 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.563 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.563 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.563 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.563 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.563 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.563 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.563 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.563 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.563 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.563 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:56.563 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:56.563 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.564 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-26T21:58:56.564 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "telegraf", 2024-04-26T21:58:56.564 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-26T21:58:56.564 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-26T21:58:56.564 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-26T21:58:56.564 INFO:teuthology.orchestra.run.smithi072.stdout: "address": { 2024-04-26T21:58:56.564 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "address", 2024-04-26T21:58:56.564 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.564 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.564 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.564 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-26T21:58:56.564 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.564 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.564 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.564 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.565 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.565 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.565 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.565 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.565 INFO:teuthology.orchestra.run.smithi072.stdout: "interval": { 2024-04-26T21:58:56.565 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "interval", 2024-04-26T21:58:56.565 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "secs", 2024-04-26T21:58:56.565 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.565 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.565 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "15", 2024-04-26T21:58:56.565 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.565 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.565 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.565 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.565 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.566 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.566 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.566 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.566 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-26T21:58:56.566 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-26T21:58:56.566 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.566 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.566 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.566 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.566 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.566 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.566 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:56.566 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:56.566 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:56.567 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:56.567 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:56.567 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:56.567 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:56.567 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.567 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.567 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.567 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.567 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.567 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.567 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-26T21:58:56.567 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-26T21:58:56.567 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.567 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.567 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.568 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.568 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.568 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.568 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.568 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.568 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.568 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.568 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.568 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.568 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-26T21:58:56.568 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-26T21:58:56.568 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.568 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.568 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.568 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-26T21:58:56.568 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.569 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.569 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:56.569 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:56.569 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:56.569 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:56.569 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:56.569 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:56.569 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:56.569 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.569 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.569 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.569 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.569 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.569 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.569 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-26T21:58:56.570 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-26T21:58:56.570 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.570 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.570 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.570 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.570 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.570 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.570 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.570 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.570 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.570 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.570 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.570 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:56.570 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:56.571 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.571 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-26T21:58:56.571 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "telemetry", 2024-04-26T21:58:56.571 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-26T21:58:56.571 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-26T21:58:56.571 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-26T21:58:56.571 INFO:teuthology.orchestra.run.smithi072.stdout: "channel_basic": { 2024-04-26T21:58:56.571 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "channel_basic", 2024-04-26T21:58:56.571 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.571 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.571 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.571 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-26T21:58:56.571 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.571 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.571 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.571 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-26T21:58:56.572 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.572 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.572 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.572 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.572 INFO:teuthology.orchestra.run.smithi072.stdout: "channel_crash": { 2024-04-26T21:58:56.572 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "channel_crash", 2024-04-26T21:58:56.572 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.572 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.572 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.572 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-26T21:58:56.572 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.572 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.572 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.572 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-26T21:58:56.573 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.573 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.573 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.573 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.573 INFO:teuthology.orchestra.run.smithi072.stdout: "channel_device": { 2024-04-26T21:58:56.573 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "channel_device", 2024-04-26T21:58:56.573 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.573 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.573 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.573 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-26T21:58:56.573 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.573 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.573 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.573 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-26T21:58:56.573 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.574 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.574 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.574 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.574 INFO:teuthology.orchestra.run.smithi072.stdout: "channel_ident": { 2024-04-26T21:58:56.574 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "channel_ident", 2024-04-26T21:58:56.574 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.574 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.574 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.574 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.574 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.574 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.574 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.574 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-26T21:58:56.574 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.575 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.575 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.575 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.575 INFO:teuthology.orchestra.run.smithi072.stdout: "channel_perf": { 2024-04-26T21:58:56.575 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "channel_perf", 2024-04-26T21:58:56.575 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.575 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.575 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.575 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.575 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.575 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.575 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.575 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-26T21:58:56.575 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.576 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.576 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.576 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.576 INFO:teuthology.orchestra.run.smithi072.stdout: "contact": { 2024-04-26T21:58:56.576 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "contact", 2024-04-26T21:58:56.576 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.576 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.576 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.576 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.576 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.576 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.576 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.576 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.576 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.577 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.577 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.577 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.577 INFO:teuthology.orchestra.run.smithi072.stdout: "description": { 2024-04-26T21:58:56.577 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "description", 2024-04-26T21:58:56.577 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.577 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.577 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.577 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.577 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.577 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.577 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.577 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.577 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.578 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.578 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.578 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.578 INFO:teuthology.orchestra.run.smithi072.stdout: "device_url": { 2024-04-26T21:58:56.578 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "device_url", 2024-04-26T21:58:56.578 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.578 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.578 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.578 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-26T21:58:56.578 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.578 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.578 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.578 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.578 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.579 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.579 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.579 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.579 INFO:teuthology.orchestra.run.smithi072.stdout: "enabled": { 2024-04-26T21:58:56.579 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "enabled", 2024-04-26T21:58:56.579 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.579 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.579 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.579 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.579 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.579 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.579 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.579 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.579 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.580 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.580 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.580 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.580 INFO:teuthology.orchestra.run.smithi072.stdout: "interval": { 2024-04-26T21:58:56.580 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "interval", 2024-04-26T21:58:56.580 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-26T21:58:56.580 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.580 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.580 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "24", 2024-04-26T21:58:56.580 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "8", 2024-04-26T21:58:56.580 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.580 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.580 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.580 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.581 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.581 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.581 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.581 INFO:teuthology.orchestra.run.smithi072.stdout: "last_opt_revision": { 2024-04-26T21:58:56.581 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "last_opt_revision", 2024-04-26T21:58:56.581 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-26T21:58:56.581 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.581 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.581 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "1", 2024-04-26T21:58:56.581 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.581 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.581 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.581 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.581 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.581 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.582 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.582 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.582 INFO:teuthology.orchestra.run.smithi072.stdout: "leaderboard": { 2024-04-26T21:58:56.582 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "leaderboard", 2024-04-26T21:58:56.582 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.582 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.582 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.582 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.582 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.582 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.582 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.582 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.582 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.582 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.582 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.583 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.583 INFO:teuthology.orchestra.run.smithi072.stdout: "leaderboard_description": { 2024-04-26T21:58:56.583 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "leaderboard_description", 2024-04-26T21:58:56.583 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.583 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.583 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.583 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.583 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.583 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.583 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.583 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.583 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.583 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.583 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.584 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.584 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-26T21:58:56.584 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-26T21:58:56.584 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.584 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.584 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.584 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.584 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.584 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.584 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:56.584 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:56.584 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:56.584 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:56.584 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:56.584 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:56.585 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:56.585 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.585 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.585 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.585 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.585 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.585 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.585 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-26T21:58:56.585 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-26T21:58:56.585 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.585 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.585 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.585 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.585 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.586 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.586 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.586 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.586 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.586 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.586 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.586 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.586 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-26T21:58:56.586 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-26T21:58:56.586 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.586 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.586 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.586 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-26T21:58:56.586 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.587 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.587 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:56.587 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:56.587 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:56.587 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:56.587 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:56.587 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:56.587 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:56.587 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.587 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.587 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.587 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.587 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.587 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.587 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-26T21:58:56.588 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-26T21:58:56.588 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.588 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.588 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.588 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.588 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.588 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.588 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.588 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.588 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.588 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.588 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.588 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.588 INFO:teuthology.orchestra.run.smithi072.stdout: "organization": { 2024-04-26T21:58:56.588 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "organization", 2024-04-26T21:58:56.589 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.589 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.589 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.589 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.589 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.589 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.589 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.589 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.589 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.589 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.589 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.589 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.589 INFO:teuthology.orchestra.run.smithi072.stdout: "proxy": { 2024-04-26T21:58:56.589 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "proxy", 2024-04-26T21:58:56.590 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.590 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.590 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.590 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.590 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.590 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.590 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.590 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.590 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.590 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.590 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.590 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.590 INFO:teuthology.orchestra.run.smithi072.stdout: "url": { 2024-04-26T21:58:56.590 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "url", 2024-04-26T21:58:56.591 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.591 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.591 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.591 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-26T21:58:56.591 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.591 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.591 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.591 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.591 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.591 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.591 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.591 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:56.591 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:56.591 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.591 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-26T21:58:56.592 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "test_orchestrator", 2024-04-26T21:58:56.592 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-26T21:58:56.592 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-26T21:58:56.592 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-26T21:58:56.592 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-26T21:58:56.592 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-26T21:58:56.592 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.592 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.592 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.592 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.592 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.592 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.592 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:56.592 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:56.592 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:56.593 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:56.593 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:56.593 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:56.593 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:56.593 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.593 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.593 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.593 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.593 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.593 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.593 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-26T21:58:56.593 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-26T21:58:56.593 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.593 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.593 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.594 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.594 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.594 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.594 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.594 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.594 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.594 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.594 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.594 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.594 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-26T21:58:56.594 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-26T21:58:56.594 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.594 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.594 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.595 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-26T21:58:56.595 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.595 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.595 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:56.595 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:56.595 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:56.595 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:56.595 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:56.595 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:56.595 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:56.595 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.595 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.595 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.595 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.595 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.595 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.596 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-26T21:58:56.596 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-26T21:58:56.596 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.596 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.596 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.596 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.596 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.596 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.596 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.596 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.596 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.596 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.596 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.596 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:56.596 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:56.597 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.597 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-26T21:58:56.597 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "volumes", 2024-04-26T21:58:56.597 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-26T21:58:56.597 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-26T21:58:56.597 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-26T21:58:56.597 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-26T21:58:56.597 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-26T21:58:56.597 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.597 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.597 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.597 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.597 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.597 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.597 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:56.597 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:56.598 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:56.598 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:56.598 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:56.598 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:56.598 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:56.598 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.598 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.598 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.598 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.598 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.598 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.598 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-26T21:58:56.598 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-26T21:58:56.598 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.599 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.599 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.599 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.599 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.599 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.599 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.599 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.599 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.599 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.599 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.599 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.599 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-26T21:58:56.599 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-26T21:58:56.599 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.599 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.600 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.600 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-26T21:58:56.600 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.600 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.600 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:56.600 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:56.600 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:56.600 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:56.600 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:56.600 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:56.600 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:56.600 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.600 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.600 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.600 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.601 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.601 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.601 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-26T21:58:56.601 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-26T21:58:56.601 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.601 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.601 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.601 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.601 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.601 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.601 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.601 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.601 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.601 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.601 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.602 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.602 INFO:teuthology.orchestra.run.smithi072.stdout: "max_concurrent_clones": { 2024-04-26T21:58:56.602 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "max_concurrent_clones", 2024-04-26T21:58:56.602 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-26T21:58:56.602 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.602 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.602 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "4", 2024-04-26T21:58:56.602 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.602 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.602 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.602 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-26T21:58:56.602 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.602 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.602 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.603 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.603 INFO:teuthology.orchestra.run.smithi072.stdout: "periodic_async_work": { 2024-04-26T21:58:56.603 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "periodic_async_work", 2024-04-26T21:58:56.603 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.603 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.603 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.603 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.603 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.603 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.603 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.603 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Periodically check for async work", 2024-04-26T21:58:56.603 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.603 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.603 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.603 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.604 INFO:teuthology.orchestra.run.smithi072.stdout: "snapshot_clone_delay": { 2024-04-26T21:58:56.604 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "snapshot_clone_delay", 2024-04-26T21:58:56.604 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-26T21:58:56.604 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.604 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.604 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "0", 2024-04-26T21:58:56.604 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.604 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.604 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.604 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-26T21:58:56.604 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.604 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.604 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.604 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.605 INFO:teuthology.orchestra.run.smithi072.stdout: "snapshot_clone_no_wait": { 2024-04-26T21:58:56.605 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "snapshot_clone_no_wait", 2024-04-26T21:58:56.605 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.605 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.605 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.605 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-26T21:58:56.605 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.605 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.605 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.605 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-26T21:58:56.605 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.605 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.605 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.605 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:56.605 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:56.606 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.606 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-26T21:58:56.606 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "zabbix", 2024-04-26T21:58:56.606 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-26T21:58:56.606 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-26T21:58:56.606 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-26T21:58:56.606 INFO:teuthology.orchestra.run.smithi072.stdout: "discovery_interval": { 2024-04-26T21:58:56.606 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "discovery_interval", 2024-04-26T21:58:56.606 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "uint", 2024-04-26T21:58:56.606 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.606 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.606 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "100", 2024-04-26T21:58:56.606 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.606 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.607 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.607 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.607 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.607 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.607 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.607 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.607 INFO:teuthology.orchestra.run.smithi072.stdout: "identifier": { 2024-04-26T21:58:56.607 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "identifier", 2024-04-26T21:58:56.607 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.607 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.607 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.607 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.607 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.607 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.607 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.608 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.608 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.608 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.608 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.608 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.608 INFO:teuthology.orchestra.run.smithi072.stdout: "interval": { 2024-04-26T21:58:56.608 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "interval", 2024-04-26T21:58:56.608 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "secs", 2024-04-26T21:58:56.608 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.608 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.608 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "60", 2024-04-26T21:58:56.608 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.608 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.608 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.608 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.609 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.609 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.609 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.609 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.609 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-26T21:58:56.609 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-26T21:58:56.609 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.609 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.609 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.609 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.609 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.609 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.609 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:56.609 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:56.609 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:56.609 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:56.610 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:56.610 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:56.610 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:56.610 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.610 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.610 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.610 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.610 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.610 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.610 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-26T21:58:56.610 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-26T21:58:56.610 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.610 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.610 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.611 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.611 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.611 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.611 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.611 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.611 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.611 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.611 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.611 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.611 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-26T21:58:56.611 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-26T21:58:56.611 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.611 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.611 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.611 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-26T21:58:56.612 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.612 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.612 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:56.612 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:56.612 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:56.612 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:56.612 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:56.612 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:56.612 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:56.612 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.612 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.612 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.612 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.612 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.613 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.613 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-26T21:58:56.613 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-26T21:58:56.613 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.613 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.613 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.613 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.613 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.613 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.613 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.613 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.613 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.613 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.613 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.614 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.614 INFO:teuthology.orchestra.run.smithi072.stdout: "zabbix_host": { 2024-04-26T21:58:56.614 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "zabbix_host", 2024-04-26T21:58:56.614 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.614 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.614 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.614 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.614 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.614 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.614 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.614 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.614 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.614 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.614 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.615 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.615 INFO:teuthology.orchestra.run.smithi072.stdout: "zabbix_port": { 2024-04-26T21:58:56.615 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "zabbix_port", 2024-04-26T21:58:56.615 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-26T21:58:56.615 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.615 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.615 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "10051", 2024-04-26T21:58:56.615 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.615 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.615 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.615 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.615 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.615 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.615 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.615 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.615 INFO:teuthology.orchestra.run.smithi072.stdout: "zabbix_sender": { 2024-04-26T21:58:56.616 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "zabbix_sender", 2024-04-26T21:58:56.616 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.616 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.616 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.616 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-26T21:58:56.616 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.616 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.616 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.616 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.616 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.616 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.616 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.616 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:56.616 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:56.616 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:56.617 INFO:teuthology.orchestra.run.smithi072.stdout: ] 2024-04-26T21:58:56.617 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:56.617 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.617 INFO:teuthology.orchestra.run.smithi072.stdout: "modules": [ 2024-04-26T21:58:56.617 INFO:teuthology.orchestra.run.smithi072.stdout: "cephadm", 2024-04-26T21:58:56.617 INFO:teuthology.orchestra.run.smithi072.stdout: "dashboard", 2024-04-26T21:58:56.617 INFO:teuthology.orchestra.run.smithi072.stdout: "iostat", 2024-04-26T21:58:56.617 INFO:teuthology.orchestra.run.smithi072.stdout: "nfs", 2024-04-26T21:58:56.617 INFO:teuthology.orchestra.run.smithi072.stdout: "restful" 2024-04-26T21:58:56.617 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.617 INFO:teuthology.orchestra.run.smithi072.stdout: "available_modules": [ 2024-04-26T21:58:56.617 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-26T21:58:56.617 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "alerts", 2024-04-26T21:58:56.617 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-26T21:58:56.617 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-26T21:58:56.618 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-26T21:58:56.618 INFO:teuthology.orchestra.run.smithi072.stdout: "interval": { 2024-04-26T21:58:56.618 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "interval", 2024-04-26T21:58:56.618 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "secs", 2024-04-26T21:58:56.618 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.618 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.618 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "60", 2024-04-26T21:58:56.618 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.618 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.618 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.618 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "How frequently to reexamine health status", 2024-04-26T21:58:56.618 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.618 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.618 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.619 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.619 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-26T21:58:56.619 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-26T21:58:56.619 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.619 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.619 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.619 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.619 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.619 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.619 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:56.619 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:56.619 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:56.619 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:56.619 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:56.620 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:56.620 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:56.620 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.620 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.620 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.620 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.620 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.620 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.620 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-26T21:58:56.620 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-26T21:58:56.620 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.620 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.620 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.620 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.620 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.620 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.621 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.621 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.621 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.621 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.621 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.621 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.621 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-26T21:58:56.621 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-26T21:58:56.621 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.621 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.621 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.621 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-26T21:58:56.621 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.621 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.621 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:56.621 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:56.622 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:56.622 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:56.622 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:56.622 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:56.622 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:56.622 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.622 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.622 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.622 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.622 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.622 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.622 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-26T21:58:56.622 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-26T21:58:56.622 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.623 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.623 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.623 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.623 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.623 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.623 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.623 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.623 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.623 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.623 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.623 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.623 INFO:teuthology.orchestra.run.smithi072.stdout: "smtp_destination": { 2024-04-26T21:58:56.623 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "smtp_destination", 2024-04-26T21:58:56.623 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.624 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.624 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.624 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.624 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.624 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.624 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.624 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Email address to send alerts to", 2024-04-26T21:58:56.624 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.624 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.624 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.624 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.624 INFO:teuthology.orchestra.run.smithi072.stdout: "smtp_from_name": { 2024-04-26T21:58:56.624 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "smtp_from_name", 2024-04-26T21:58:56.624 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.624 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.625 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.625 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "Ceph", 2024-04-26T21:58:56.625 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.625 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.625 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.625 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Email From: name", 2024-04-26T21:58:56.625 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.625 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.625 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.625 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.625 INFO:teuthology.orchestra.run.smithi072.stdout: "smtp_host": { 2024-04-26T21:58:56.625 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "smtp_host", 2024-04-26T21:58:56.625 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.625 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.625 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.625 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.626 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.626 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.626 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.626 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "SMTP server", 2024-04-26T21:58:56.626 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.626 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.626 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.626 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.626 INFO:teuthology.orchestra.run.smithi072.stdout: "smtp_password": { 2024-04-26T21:58:56.626 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "smtp_password", 2024-04-26T21:58:56.626 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.626 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.626 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.626 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.627 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.627 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.627 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.627 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Password to authenticate with", 2024-04-26T21:58:56.627 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.627 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.627 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.627 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.627 INFO:teuthology.orchestra.run.smithi072.stdout: "smtp_port": { 2024-04-26T21:58:56.627 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "smtp_port", 2024-04-26T21:58:56.627 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-26T21:58:56.627 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.627 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.627 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "465", 2024-04-26T21:58:56.627 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.628 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.628 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.628 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "SMTP port", 2024-04-26T21:58:56.628 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.628 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.628 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.628 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.628 INFO:teuthology.orchestra.run.smithi072.stdout: "smtp_sender": { 2024-04-26T21:58:56.628 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "smtp_sender", 2024-04-26T21:58:56.628 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.628 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.628 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.628 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.628 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.628 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.628 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.629 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "SMTP envelope sender", 2024-04-26T21:58:56.629 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.629 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.629 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.629 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.629 INFO:teuthology.orchestra.run.smithi072.stdout: "smtp_ssl": { 2024-04-26T21:58:56.629 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "smtp_ssl", 2024-04-26T21:58:56.629 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.629 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.629 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.629 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-26T21:58:56.629 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.629 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.629 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.629 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-26T21:58:56.630 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.630 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.630 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.630 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.630 INFO:teuthology.orchestra.run.smithi072.stdout: "smtp_user": { 2024-04-26T21:58:56.630 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "smtp_user", 2024-04-26T21:58:56.630 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.630 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.630 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.630 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.630 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.630 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.630 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.630 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "User to authenticate as", 2024-04-26T21:58:56.631 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.631 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.631 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.631 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:56.631 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:56.631 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.631 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-26T21:58:56.631 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "balancer", 2024-04-26T21:58:56.631 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-26T21:58:56.631 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-26T21:58:56.631 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-26T21:58:56.631 INFO:teuthology.orchestra.run.smithi072.stdout: "active": { 2024-04-26T21:58:56.631 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "active", 2024-04-26T21:58:56.631 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.631 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.631 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.632 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-26T21:58:56.632 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.632 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.632 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.632 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "automatically balance PGs across cluster", 2024-04-26T21:58:56.632 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.632 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.632 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.632 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.632 INFO:teuthology.orchestra.run.smithi072.stdout: "begin_time": { 2024-04-26T21:58:56.632 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "begin_time", 2024-04-26T21:58:56.632 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.632 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.632 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.632 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "0000", 2024-04-26T21:58:56.633 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.633 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.633 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.633 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "beginning time of day to automatically balance", 2024-04-26T21:58:56.633 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-26T21:58:56.633 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.633 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.633 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.633 INFO:teuthology.orchestra.run.smithi072.stdout: "begin_weekday": { 2024-04-26T21:58:56.633 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "begin_weekday", 2024-04-26T21:58:56.633 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "uint", 2024-04-26T21:58:56.633 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.633 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.633 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "0", 2024-04-26T21:58:56.634 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "0", 2024-04-26T21:58:56.634 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "6", 2024-04-26T21:58:56.634 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.634 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-26T21:58:56.634 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-26T21:58:56.634 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.634 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.634 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.634 INFO:teuthology.orchestra.run.smithi072.stdout: "crush_compat_max_iterations": { 2024-04-26T21:58:56.634 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "crush_compat_max_iterations", 2024-04-26T21:58:56.634 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "uint", 2024-04-26T21:58:56.634 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.634 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.634 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "25", 2024-04-26T21:58:56.634 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "1", 2024-04-26T21:58:56.635 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "250", 2024-04-26T21:58:56.635 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.637 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-26T21:58:56.638 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.638 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.638 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.638 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.638 INFO:teuthology.orchestra.run.smithi072.stdout: "crush_compat_metrics": { 2024-04-26T21:58:56.638 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "crush_compat_metrics", 2024-04-26T21:58:56.638 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.638 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.638 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.638 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "pgs,objects,bytes", 2024-04-26T21:58:56.638 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.638 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.638 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.638 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-26T21:58:56.639 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-26T21:58:56.639 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.639 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.639 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.639 INFO:teuthology.orchestra.run.smithi072.stdout: "crush_compat_step": { 2024-04-26T21:58:56.639 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "crush_compat_step", 2024-04-26T21:58:56.639 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "float", 2024-04-26T21:58:56.639 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.639 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.639 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "0.5", 2024-04-26T21:58:56.639 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "0.001", 2024-04-26T21:58:56.639 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "0.999", 2024-04-26T21:58:56.639 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.639 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "aggressiveness of optimization", 2024-04-26T21:58:56.640 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-26T21:58:56.640 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.640 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.640 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.640 INFO:teuthology.orchestra.run.smithi072.stdout: "end_time": { 2024-04-26T21:58:56.640 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "end_time", 2024-04-26T21:58:56.640 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.640 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.640 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.640 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "2359", 2024-04-26T21:58:56.640 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.640 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.640 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.640 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "ending time of day to automatically balance", 2024-04-26T21:58:56.640 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-26T21:58:56.641 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.641 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.641 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.641 INFO:teuthology.orchestra.run.smithi072.stdout: "end_weekday": { 2024-04-26T21:58:56.641 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "end_weekday", 2024-04-26T21:58:56.641 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "uint", 2024-04-26T21:58:56.641 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.641 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.641 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "0", 2024-04-26T21:58:56.641 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "0", 2024-04-26T21:58:56.641 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "6", 2024-04-26T21:58:56.641 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.641 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-26T21:58:56.641 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-26T21:58:56.642 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.642 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.642 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.642 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-26T21:58:56.642 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-26T21:58:56.642 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.642 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.642 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.642 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.642 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.642 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.642 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:56.642 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:56.642 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:56.642 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:56.642 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:56.643 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:56.643 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:56.643 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.643 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.643 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.643 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.643 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.643 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.643 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-26T21:58:56.643 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-26T21:58:56.643 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.643 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.643 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.643 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.643 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.644 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.644 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.644 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.644 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.644 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.644 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.644 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.644 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-26T21:58:56.644 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-26T21:58:56.644 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.644 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.644 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.644 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-26T21:58:56.644 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.645 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.645 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:56.645 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:56.645 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:56.645 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:56.645 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:56.645 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:56.645 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:56.645 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.645 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.645 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.645 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.645 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.645 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.645 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-26T21:58:56.646 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-26T21:58:56.646 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.646 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.646 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.646 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.646 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.646 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.646 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.646 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.646 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.646 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.646 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.646 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.646 INFO:teuthology.orchestra.run.smithi072.stdout: "min_score": { 2024-04-26T21:58:56.647 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "min_score", 2024-04-26T21:58:56.647 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "float", 2024-04-26T21:58:56.647 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.647 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.647 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "0", 2024-04-26T21:58:56.647 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.647 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.647 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.647 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-26T21:58:56.647 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.647 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.647 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.647 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.647 INFO:teuthology.orchestra.run.smithi072.stdout: "mode": { 2024-04-26T21:58:56.648 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "mode", 2024-04-26T21:58:56.648 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.648 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.648 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.648 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "upmap", 2024-04-26T21:58:56.648 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.648 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.648 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:56.648 INFO:teuthology.orchestra.run.smithi072.stdout: "crush-compat", 2024-04-26T21:58:56.648 INFO:teuthology.orchestra.run.smithi072.stdout: "none", 2024-04-26T21:58:56.648 INFO:teuthology.orchestra.run.smithi072.stdout: "read", 2024-04-26T21:58:56.648 INFO:teuthology.orchestra.run.smithi072.stdout: "upmap", 2024-04-26T21:58:56.648 INFO:teuthology.orchestra.run.smithi072.stdout: "upmap-read" 2024-04-26T21:58:56.648 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.648 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Balancer mode", 2024-04-26T21:58:56.648 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.649 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.649 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.649 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.649 INFO:teuthology.orchestra.run.smithi072.stdout: "pool_ids": { 2024-04-26T21:58:56.649 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "pool_ids", 2024-04-26T21:58:56.649 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.649 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.649 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.649 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.649 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.649 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.649 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.649 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-26T21:58:56.649 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.649 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.650 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.650 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.650 INFO:teuthology.orchestra.run.smithi072.stdout: "sleep_interval": { 2024-04-26T21:58:56.650 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "sleep_interval", 2024-04-26T21:58:56.650 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "secs", 2024-04-26T21:58:56.650 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.650 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.650 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "60", 2024-04-26T21:58:56.650 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.650 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.650 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.650 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-26T21:58:56.650 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.650 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.651 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.651 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.651 INFO:teuthology.orchestra.run.smithi072.stdout: "upmap_max_deviation": { 2024-04-26T21:58:56.651 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "upmap_max_deviation", 2024-04-26T21:58:56.651 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-26T21:58:56.651 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.651 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.651 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "5", 2024-04-26T21:58:56.651 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "1", 2024-04-26T21:58:56.651 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.651 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.651 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-26T21:58:56.651 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-26T21:58:56.651 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.651 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.651 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.652 INFO:teuthology.orchestra.run.smithi072.stdout: "upmap_max_optimizations": { 2024-04-26T21:58:56.652 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "upmap_max_optimizations", 2024-04-26T21:58:56.652 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "uint", 2024-04-26T21:58:56.652 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.652 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.652 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "10", 2024-04-26T21:58:56.652 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.652 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.652 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.652 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-26T21:58:56.652 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.652 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.652 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.652 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:56.652 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:56.652 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.653 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-26T21:58:56.653 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "cephadm", 2024-04-26T21:58:56.653 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-26T21:58:56.653 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-26T21:58:56.653 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-26T21:58:56.653 INFO:teuthology.orchestra.run.smithi072.stdout: "agent_down_multiplier": { 2024-04-26T21:58:56.653 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "agent_down_multiplier", 2024-04-26T21:58:56.653 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "float", 2024-04-26T21:58:56.653 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.653 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.653 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "3.0", 2024-04-26T21:58:56.653 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.653 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.653 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.653 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-26T21:58:56.653 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.653 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.654 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.654 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.654 INFO:teuthology.orchestra.run.smithi072.stdout: "agent_refresh_rate": { 2024-04-26T21:58:56.654 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "agent_refresh_rate", 2024-04-26T21:58:56.654 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "secs", 2024-04-26T21:58:56.654 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.654 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.654 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "20", 2024-04-26T21:58:56.654 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.654 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.654 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.654 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-26T21:58:56.654 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.654 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.654 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.655 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.655 INFO:teuthology.orchestra.run.smithi072.stdout: "agent_starting_port": { 2024-04-26T21:58:56.655 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "agent_starting_port", 2024-04-26T21:58:56.655 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-26T21:58:56.655 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.655 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.655 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "4721", 2024-04-26T21:58:56.655 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.655 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.655 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.655 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-26T21:58:56.655 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.655 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.655 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.655 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.656 INFO:teuthology.orchestra.run.smithi072.stdout: "allow_ptrace": { 2024-04-26T21:58:56.656 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "allow_ptrace", 2024-04-26T21:58:56.656 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.656 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.656 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.656 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.656 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.656 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.656 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.656 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-26T21:58:56.656 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-26T21:58:56.656 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.656 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.656 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.656 INFO:teuthology.orchestra.run.smithi072.stdout: "autotune_interval": { 2024-04-26T21:58:56.656 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "autotune_interval", 2024-04-26T21:58:56.657 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "secs", 2024-04-26T21:58:56.657 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.657 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.657 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "600", 2024-04-26T21:58:56.657 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.657 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.657 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.657 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-26T21:58:56.657 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.657 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.657 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.657 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.657 INFO:teuthology.orchestra.run.smithi072.stdout: "autotune_memory_target_ratio": { 2024-04-26T21:58:56.657 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "autotune_memory_target_ratio", 2024-04-26T21:58:56.657 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "float", 2024-04-26T21:58:56.658 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.658 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.658 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "0.7", 2024-04-26T21:58:56.658 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.658 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.658 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.658 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-26T21:58:56.658 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.658 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.658 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.658 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.658 INFO:teuthology.orchestra.run.smithi072.stdout: "cephadm_log_destination": { 2024-04-26T21:58:56.658 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "cephadm_log_destination", 2024-04-26T21:58:56.659 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.659 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.659 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.659 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.659 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.659 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.659 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:56.659 INFO:teuthology.orchestra.run.smithi072.stdout: "file", 2024-04-26T21:58:56.659 INFO:teuthology.orchestra.run.smithi072.stdout: "file,syslog", 2024-04-26T21:58:56.659 INFO:teuthology.orchestra.run.smithi072.stdout: "syslog" 2024-04-26T21:58:56.659 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.659 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-26T21:58:56.659 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.659 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.659 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.660 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.660 INFO:teuthology.orchestra.run.smithi072.stdout: "cgroups_split": { 2024-04-26T21:58:56.660 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "cgroups_split", 2024-04-26T21:58:56.660 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.660 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.660 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.660 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-26T21:58:56.660 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.660 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.660 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.660 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-26T21:58:56.660 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.660 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.660 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.661 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.661 INFO:teuthology.orchestra.run.smithi072.stdout: "config_checks_enabled": { 2024-04-26T21:58:56.661 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "config_checks_enabled", 2024-04-26T21:58:56.661 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.661 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.661 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.661 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.661 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.661 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.661 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.661 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-26T21:58:56.661 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.661 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.661 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.662 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.662 INFO:teuthology.orchestra.run.smithi072.stdout: "config_dashboard": { 2024-04-26T21:58:56.662 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "config_dashboard", 2024-04-26T21:58:56.662 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.662 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.662 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.662 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-26T21:58:56.662 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.662 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.662 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.662 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-26T21:58:56.662 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.662 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.662 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.663 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.663 INFO:teuthology.orchestra.run.smithi072.stdout: "container_image_alertmanager": { 2024-04-26T21:58:56.663 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "container_image_alertmanager", 2024-04-26T21:58:56.663 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.663 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.663 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.663 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-26T21:58:56.663 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.663 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.663 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.663 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Prometheus container image", 2024-04-26T21:58:56.663 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.663 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.663 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.664 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.664 INFO:teuthology.orchestra.run.smithi072.stdout: "container_image_base": { 2024-04-26T21:58:56.664 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "container_image_base", 2024-04-26T21:58:56.664 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.664 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.664 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.664 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-26T21:58:56.664 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.664 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.664 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.664 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Container image name, without the tag", 2024-04-26T21:58:56.664 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.664 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.664 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.664 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.665 INFO:teuthology.orchestra.run.smithi072.stdout: "container_image_elasticsearch": { 2024-04-26T21:58:56.665 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "container_image_elasticsearch", 2024-04-26T21:58:56.665 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.665 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.665 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.665 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-26T21:58:56.665 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.665 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.665 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.665 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "elasticsearch container image", 2024-04-26T21:58:56.665 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.665 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.665 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.665 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.666 INFO:teuthology.orchestra.run.smithi072.stdout: "container_image_grafana": { 2024-04-26T21:58:56.666 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "container_image_grafana", 2024-04-26T21:58:56.666 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.666 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.666 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.666 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-26T21:58:56.666 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.666 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.666 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.666 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Prometheus container image", 2024-04-26T21:58:56.666 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.666 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.666 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.666 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.667 INFO:teuthology.orchestra.run.smithi072.stdout: "container_image_haproxy": { 2024-04-26T21:58:56.667 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "container_image_haproxy", 2024-04-26T21:58:56.667 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.667 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.667 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.667 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-26T21:58:56.667 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.667 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.667 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.667 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "HAproxy container image", 2024-04-26T21:58:56.667 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.667 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.667 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.667 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.667 INFO:teuthology.orchestra.run.smithi072.stdout: "container_image_jaeger_agent": { 2024-04-26T21:58:56.668 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "container_image_jaeger_agent", 2024-04-26T21:58:56.668 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.668 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.668 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.668 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-26T21:58:56.668 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.668 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.668 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.668 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Jaeger agent container image", 2024-04-26T21:58:56.668 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.668 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.668 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.668 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.668 INFO:teuthology.orchestra.run.smithi072.stdout: "container_image_jaeger_collector": { 2024-04-26T21:58:56.668 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "container_image_jaeger_collector", 2024-04-26T21:58:56.668 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.669 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.669 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.669 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-26T21:58:56.669 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.669 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.669 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.669 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Jaeger collector container image", 2024-04-26T21:58:56.669 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.669 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.669 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.669 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.669 INFO:teuthology.orchestra.run.smithi072.stdout: "container_image_jaeger_query": { 2024-04-26T21:58:56.669 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "container_image_jaeger_query", 2024-04-26T21:58:56.669 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.669 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.670 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.670 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-26T21:58:56.670 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.670 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.670 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.670 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Jaeger query container image", 2024-04-26T21:58:56.670 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.670 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.670 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.670 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.670 INFO:teuthology.orchestra.run.smithi072.stdout: "container_image_keepalived": { 2024-04-26T21:58:56.670 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "container_image_keepalived", 2024-04-26T21:58:56.670 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.671 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.671 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.671 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-26T21:58:56.671 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.671 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.671 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.671 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Keepalived container image", 2024-04-26T21:58:56.671 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.671 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.671 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.671 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.671 INFO:teuthology.orchestra.run.smithi072.stdout: "container_image_loki": { 2024-04-26T21:58:56.671 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "container_image_loki", 2024-04-26T21:58:56.671 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.672 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.672 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.672 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-26T21:58:56.672 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.672 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.672 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.672 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Loki container image", 2024-04-26T21:58:56.672 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.672 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.672 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.672 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.672 INFO:teuthology.orchestra.run.smithi072.stdout: "container_image_node_exporter": { 2024-04-26T21:58:56.672 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "container_image_node_exporter", 2024-04-26T21:58:56.672 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.672 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.673 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.673 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-26T21:58:56.673 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.673 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.673 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.673 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Prometheus container image", 2024-04-26T21:58:56.673 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.673 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.673 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.673 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.673 INFO:teuthology.orchestra.run.smithi072.stdout: "container_image_nvmeof": { 2024-04-26T21:58:56.673 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "container_image_nvmeof", 2024-04-26T21:58:56.673 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.673 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.674 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.674 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-26T21:58:56.674 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.674 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.674 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.674 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Nvme-of container image", 2024-04-26T21:58:56.674 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.674 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.674 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.674 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.674 INFO:teuthology.orchestra.run.smithi072.stdout: "container_image_prometheus": { 2024-04-26T21:58:56.674 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "container_image_prometheus", 2024-04-26T21:58:56.674 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.674 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.675 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.675 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-26T21:58:56.675 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.675 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.675 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.675 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Prometheus container image", 2024-04-26T21:58:56.675 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.675 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.675 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.675 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.675 INFO:teuthology.orchestra.run.smithi072.stdout: "container_image_promtail": { 2024-04-26T21:58:56.675 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "container_image_promtail", 2024-04-26T21:58:56.675 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.675 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.676 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.676 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-26T21:58:56.676 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.676 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.676 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.676 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Promtail container image", 2024-04-26T21:58:56.676 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.676 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.676 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.676 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.676 INFO:teuthology.orchestra.run.smithi072.stdout: "container_image_samba": { 2024-04-26T21:58:56.676 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "container_image_samba", 2024-04-26T21:58:56.676 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.676 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.676 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.677 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-26T21:58:56.677 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.677 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.677 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.677 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Samba/SMB container image", 2024-04-26T21:58:56.677 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.677 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.677 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.677 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.677 INFO:teuthology.orchestra.run.smithi072.stdout: "container_image_snmp_gateway": { 2024-04-26T21:58:56.677 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "container_image_snmp_gateway", 2024-04-26T21:58:56.677 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.677 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.677 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.677 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-26T21:58:56.678 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.678 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.678 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.678 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "SNMP Gateway container image", 2024-04-26T21:58:56.678 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.678 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.678 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.678 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.678 INFO:teuthology.orchestra.run.smithi072.stdout: "container_init": { 2024-04-26T21:58:56.678 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "container_init", 2024-04-26T21:58:56.678 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.678 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.678 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.678 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-26T21:58:56.678 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.679 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.679 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.679 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Run podman/docker with `--init`", 2024-04-26T21:58:56.679 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.679 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.679 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.679 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.679 INFO:teuthology.orchestra.run.smithi072.stdout: "daemon_cache_timeout": { 2024-04-26T21:58:56.679 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "daemon_cache_timeout", 2024-04-26T21:58:56.679 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "secs", 2024-04-26T21:58:56.679 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.679 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.679 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "600", 2024-04-26T21:58:56.679 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.680 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.680 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.680 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-26T21:58:56.680 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.680 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.680 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.680 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.680 INFO:teuthology.orchestra.run.smithi072.stdout: "default_cephadm_command_timeout": { 2024-04-26T21:58:56.680 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "default_cephadm_command_timeout", 2024-04-26T21:58:56.680 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-26T21:58:56.680 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.680 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.680 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "900", 2024-04-26T21:58:56.680 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.681 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.681 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.681 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-26T21:58:56.681 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.681 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.681 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.681 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.681 INFO:teuthology.orchestra.run.smithi072.stdout: "default_registry": { 2024-04-26T21:58:56.681 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "default_registry", 2024-04-26T21:58:56.681 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.681 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.681 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.681 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "docker.io", 2024-04-26T21:58:56.681 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.681 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.682 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.682 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-26T21:58:56.682 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.682 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.682 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.682 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.682 INFO:teuthology.orchestra.run.smithi072.stdout: "device_cache_timeout": { 2024-04-26T21:58:56.682 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "device_cache_timeout", 2024-04-26T21:58:56.682 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "secs", 2024-04-26T21:58:56.682 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.682 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.682 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "1800", 2024-04-26T21:58:56.682 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.682 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.683 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.683 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "seconds to cache device inventory", 2024-04-26T21:58:56.683 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.683 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.683 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.683 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.683 INFO:teuthology.orchestra.run.smithi072.stdout: "device_enhanced_scan": { 2024-04-26T21:58:56.683 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "device_enhanced_scan", 2024-04-26T21:58:56.683 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.683 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.683 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.683 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.683 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.683 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.683 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.684 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-26T21:58:56.684 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.684 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.684 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.684 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.684 INFO:teuthology.orchestra.run.smithi072.stdout: "facts_cache_timeout": { 2024-04-26T21:58:56.684 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "facts_cache_timeout", 2024-04-26T21:58:56.684 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "secs", 2024-04-26T21:58:56.684 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.684 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.684 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "60", 2024-04-26T21:58:56.684 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.684 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.684 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.685 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "seconds to cache host facts data", 2024-04-26T21:58:56.685 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.685 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.685 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.685 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.685 INFO:teuthology.orchestra.run.smithi072.stdout: "grafana_dashboards_path": { 2024-04-26T21:58:56.685 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "grafana_dashboards_path", 2024-04-26T21:58:56.685 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.685 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.685 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.685 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-26T21:58:56.685 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.685 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.685 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.685 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-26T21:58:56.686 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.686 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.686 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.686 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.686 INFO:teuthology.orchestra.run.smithi072.stdout: "host_check_interval": { 2024-04-26T21:58:56.686 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "host_check_interval", 2024-04-26T21:58:56.686 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "secs", 2024-04-26T21:58:56.686 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.686 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.686 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "600", 2024-04-26T21:58:56.686 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.686 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.686 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.686 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "how frequently to perform a host check", 2024-04-26T21:58:56.687 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.687 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.687 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.687 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.687 INFO:teuthology.orchestra.run.smithi072.stdout: "hw_monitoring": { 2024-04-26T21:58:56.687 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "hw_monitoring", 2024-04-26T21:58:56.687 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.687 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.687 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.687 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.687 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.687 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.687 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.687 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-26T21:58:56.688 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.688 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.688 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.688 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.688 INFO:teuthology.orchestra.run.smithi072.stdout: "inventory_list_all": { 2024-04-26T21:58:56.688 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "inventory_list_all", 2024-04-26T21:58:56.688 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.688 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.688 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.688 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.688 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.688 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.688 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.688 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-26T21:58:56.688 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.689 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.689 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.689 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.689 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-26T21:58:56.689 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-26T21:58:56.689 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.689 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.689 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.689 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.689 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.689 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.689 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:56.689 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:56.689 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:56.690 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:56.690 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:56.690 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:56.690 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:56.690 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.690 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.690 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.690 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.690 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.690 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.690 INFO:teuthology.orchestra.run.smithi072.stdout: "log_refresh_metadata": { 2024-04-26T21:58:56.690 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_refresh_metadata", 2024-04-26T21:58:56.690 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.690 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.691 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.691 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.691 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.691 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.691 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.691 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-26T21:58:56.691 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.691 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.691 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.691 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.691 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-26T21:58:56.691 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-26T21:58:56.691 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.691 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.691 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.692 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-26T21:58:56.692 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.692 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.692 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.692 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-26T21:58:56.692 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.692 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.692 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.692 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.692 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-26T21:58:56.692 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-26T21:58:56.692 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.692 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.692 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.692 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-26T21:58:56.693 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.693 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.693 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:56.693 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:56.693 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:56.693 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:56.693 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:56.693 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:56.693 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:56.693 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.693 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.693 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.693 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.693 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.694 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.694 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-26T21:58:56.694 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-26T21:58:56.694 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.694 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.694 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.694 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.694 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.694 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.694 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.694 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.694 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.694 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.694 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.694 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.695 INFO:teuthology.orchestra.run.smithi072.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-26T21:58:56.695 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-26T21:58:56.695 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.695 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.695 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.695 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.695 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.695 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.695 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.695 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-26T21:58:56.695 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.695 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.695 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.695 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.695 INFO:teuthology.orchestra.run.smithi072.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-26T21:58:56.696 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-26T21:58:56.696 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.696 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.696 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.696 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "*", 2024-04-26T21:58:56.696 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.696 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.696 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.696 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-26T21:58:56.696 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.696 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.696 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.696 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.696 INFO:teuthology.orchestra.run.smithi072.stdout: "max_count_per_host": { 2024-04-26T21:58:56.696 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "max_count_per_host", 2024-04-26T21:58:56.697 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-26T21:58:56.697 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.697 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.697 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "10", 2024-04-26T21:58:56.697 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.697 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.697 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.697 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "max number of daemons per service per host", 2024-04-26T21:58:56.697 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.697 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.697 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.697 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.697 INFO:teuthology.orchestra.run.smithi072.stdout: "max_osd_draining_count": { 2024-04-26T21:58:56.697 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "max_osd_draining_count", 2024-04-26T21:58:56.697 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-26T21:58:56.698 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.698 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.698 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "10", 2024-04-26T21:58:56.698 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.698 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.698 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.698 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-26T21:58:56.698 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.698 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.698 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.698 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.698 INFO:teuthology.orchestra.run.smithi072.stdout: "migration_current": { 2024-04-26T21:58:56.698 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "migration_current", 2024-04-26T21:58:56.698 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-26T21:58:56.699 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.699 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.699 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.699 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.699 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.699 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.699 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "internal - do not modify", 2024-04-26T21:58:56.699 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.699 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.699 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.699 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.699 INFO:teuthology.orchestra.run.smithi072.stdout: "mode": { 2024-04-26T21:58:56.699 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "mode", 2024-04-26T21:58:56.699 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.699 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.700 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.700 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "root", 2024-04-26T21:58:56.700 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.700 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.700 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:56.700 INFO:teuthology.orchestra.run.smithi072.stdout: "cephadm-package", 2024-04-26T21:58:56.700 INFO:teuthology.orchestra.run.smithi072.stdout: "root" 2024-04-26T21:58:56.700 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.700 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "mode for remote execution of cephadm", 2024-04-26T21:58:56.700 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.700 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.700 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.700 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.700 INFO:teuthology.orchestra.run.smithi072.stdout: "oob_default_addr": { 2024-04-26T21:58:56.700 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "oob_default_addr", 2024-04-26T21:58:56.701 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.701 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.701 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.701 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "169.254.1.1", 2024-04-26T21:58:56.701 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.701 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.701 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.701 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-26T21:58:56.701 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.701 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.701 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.701 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.701 INFO:teuthology.orchestra.run.smithi072.stdout: "prometheus_alerts_path": { 2024-04-26T21:58:56.701 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "prometheus_alerts_path", 2024-04-26T21:58:56.701 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.702 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.702 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.702 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-26T21:58:56.702 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.702 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.702 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.702 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-26T21:58:56.702 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.702 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.702 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.702 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.702 INFO:teuthology.orchestra.run.smithi072.stdout: "registry_insecure": { 2024-04-26T21:58:56.702 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "registry_insecure", 2024-04-26T21:58:56.702 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.703 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.703 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.703 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.703 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.703 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.703 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.703 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-26T21:58:56.703 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.703 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.703 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.703 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.703 INFO:teuthology.orchestra.run.smithi072.stdout: "registry_password": { 2024-04-26T21:58:56.703 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "registry_password", 2024-04-26T21:58:56.703 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.703 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.703 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.704 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.704 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.704 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.704 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.704 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-26T21:58:56.704 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.704 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.704 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.704 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.704 INFO:teuthology.orchestra.run.smithi072.stdout: "registry_url": { 2024-04-26T21:58:56.704 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "registry_url", 2024-04-26T21:58:56.704 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.704 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.704 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.704 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.704 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.705 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.705 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.705 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-26T21:58:56.705 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.705 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.705 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.705 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.705 INFO:teuthology.orchestra.run.smithi072.stdout: "registry_username": { 2024-04-26T21:58:56.705 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "registry_username", 2024-04-26T21:58:56.705 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.705 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.705 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.705 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.705 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.705 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.706 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.706 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-26T21:58:56.706 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.706 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.706 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.706 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.706 INFO:teuthology.orchestra.run.smithi072.stdout: "secure_monitoring_stack": { 2024-04-26T21:58:56.706 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "secure_monitoring_stack", 2024-04-26T21:58:56.706 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.706 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.706 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.706 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.706 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.706 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.706 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.707 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-26T21:58:56.707 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.707 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.707 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.707 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.707 INFO:teuthology.orchestra.run.smithi072.stdout: "service_discovery_port": { 2024-04-26T21:58:56.707 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "service_discovery_port", 2024-04-26T21:58:56.707 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-26T21:58:56.707 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.707 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.707 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "8765", 2024-04-26T21:58:56.707 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.707 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.707 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.707 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "cephadm service discovery port", 2024-04-26T21:58:56.707 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.708 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.708 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.708 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.708 INFO:teuthology.orchestra.run.smithi072.stdout: "ssh_config_file": { 2024-04-26T21:58:56.708 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "ssh_config_file", 2024-04-26T21:58:56.708 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.708 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.708 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.708 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.708 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.708 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.708 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.708 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-26T21:58:56.708 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.708 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.708 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.708 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.709 INFO:teuthology.orchestra.run.smithi072.stdout: "use_agent": { 2024-04-26T21:58:56.709 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "use_agent", 2024-04-26T21:58:56.709 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.709 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.709 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.709 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.709 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.709 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.709 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.709 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-26T21:58:56.709 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.709 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.709 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.709 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.709 INFO:teuthology.orchestra.run.smithi072.stdout: "use_repo_digest": { 2024-04-26T21:58:56.709 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "use_repo_digest", 2024-04-26T21:58:56.710 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.710 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.710 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.710 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-26T21:58:56.710 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.710 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.710 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.710 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-26T21:58:56.710 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.710 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.710 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.710 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.710 INFO:teuthology.orchestra.run.smithi072.stdout: "warn_on_failed_host_check": { 2024-04-26T21:58:56.710 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "warn_on_failed_host_check", 2024-04-26T21:58:56.710 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.711 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.711 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.711 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-26T21:58:56.711 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.711 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.711 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.711 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "raise a health warning if the host check fails", 2024-04-26T21:58:56.711 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.711 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.711 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.711 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.711 INFO:teuthology.orchestra.run.smithi072.stdout: "warn_on_stray_daemons": { 2024-04-26T21:58:56.711 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "warn_on_stray_daemons", 2024-04-26T21:58:56.711 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.711 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.711 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.711 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-26T21:58:56.712 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.712 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.712 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.712 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-26T21:58:56.712 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.712 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.712 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.712 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.712 INFO:teuthology.orchestra.run.smithi072.stdout: "warn_on_stray_hosts": { 2024-04-26T21:58:56.712 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "warn_on_stray_hosts", 2024-04-26T21:58:56.712 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.712 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.712 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.712 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-26T21:58:56.712 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.712 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.713 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.713 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-26T21:58:56.713 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.713 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.713 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.713 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:56.713 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:56.713 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.713 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-26T21:58:56.713 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "crash", 2024-04-26T21:58:56.713 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-26T21:58:56.713 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-26T21:58:56.713 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-26T21:58:56.713 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-26T21:58:56.713 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-26T21:58:56.713 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.713 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.714 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.714 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.714 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.714 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.714 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:56.714 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:56.714 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:56.714 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:56.714 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:56.714 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:56.714 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:56.714 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.714 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.714 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.715 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.715 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.715 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.715 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-26T21:58:56.715 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-26T21:58:56.715 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.715 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.715 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.715 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.715 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.715 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.715 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.715 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.715 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.715 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.715 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.716 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.716 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-26T21:58:56.716 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-26T21:58:56.716 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.716 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.716 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.716 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-26T21:58:56.716 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.716 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.716 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:56.716 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:56.716 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:56.716 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:56.716 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:56.716 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:56.716 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:56.717 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.717 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.717 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.717 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.717 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.717 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.717 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-26T21:58:56.717 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-26T21:58:56.717 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.717 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.717 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.717 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.717 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.717 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.717 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.717 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.718 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.718 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.718 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.718 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.718 INFO:teuthology.orchestra.run.smithi072.stdout: "retain_interval": { 2024-04-26T21:58:56.718 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "retain_interval", 2024-04-26T21:58:56.718 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "secs", 2024-04-26T21:58:56.718 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.718 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.718 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "31536000", 2024-04-26T21:58:56.718 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.718 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.718 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.718 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-26T21:58:56.719 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.719 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.719 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.719 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.719 INFO:teuthology.orchestra.run.smithi072.stdout: "warn_recent_interval": { 2024-04-26T21:58:56.719 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "warn_recent_interval", 2024-04-26T21:58:56.719 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "secs", 2024-04-26T21:58:56.719 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.719 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.719 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "1209600", 2024-04-26T21:58:56.719 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.719 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.719 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.719 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-26T21:58:56.720 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.720 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.720 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.720 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:56.720 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:56.720 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.720 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-26T21:58:56.720 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "dashboard", 2024-04-26T21:58:56.720 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-26T21:58:56.720 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-26T21:58:56.720 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-26T21:58:56.720 INFO:teuthology.orchestra.run.smithi072.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-26T21:58:56.720 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-26T21:58:56.720 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-26T21:58:56.721 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.721 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.721 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "10", 2024-04-26T21:58:56.721 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.721 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.721 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.721 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.721 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.721 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.721 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.721 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.721 INFO:teuthology.orchestra.run.smithi072.stdout: "ALERTMANAGER_API_HOST": { 2024-04-26T21:58:56.721 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-26T21:58:56.721 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.721 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.722 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.722 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.722 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.722 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.722 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.722 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.722 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.722 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.722 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.722 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.722 INFO:teuthology.orchestra.run.smithi072.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-26T21:58:56.722 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-26T21:58:56.722 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.722 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.723 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.723 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-26T21:58:56.723 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.723 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.723 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.723 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.723 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.723 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.723 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.723 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.723 INFO:teuthology.orchestra.run.smithi072.stdout: "AUDIT_API_ENABLED": { 2024-04-26T21:58:56.723 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "AUDIT_API_ENABLED", 2024-04-26T21:58:56.723 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.723 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.723 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.724 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.724 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.724 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.724 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.724 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.724 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.724 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.724 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.724 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.724 INFO:teuthology.orchestra.run.smithi072.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-26T21:58:56.724 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-26T21:58:56.724 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.724 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.724 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.724 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-26T21:58:56.725 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.725 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.725 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.725 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.725 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.725 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.725 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.725 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.725 INFO:teuthology.orchestra.run.smithi072.stdout: "ENABLE_BROWSABLE_API": { 2024-04-26T21:58:56.725 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-26T21:58:56.725 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.725 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.725 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.725 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-26T21:58:56.726 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.726 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.726 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.726 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.726 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.726 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.726 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.726 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.726 INFO:teuthology.orchestra.run.smithi072.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-26T21:58:56.726 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-26T21:58:56.726 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.726 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.726 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.726 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-26T21:58:56.726 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.727 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.727 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.727 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.727 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.727 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.727 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.727 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.727 INFO:teuthology.orchestra.run.smithi072.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-26T21:58:56.727 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-26T21:58:56.727 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.727 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.727 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.727 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-26T21:58:56.727 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.727 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.727 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.728 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.728 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.728 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.728 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.728 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.728 INFO:teuthology.orchestra.run.smithi072.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-26T21:58:56.728 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-26T21:58:56.728 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.728 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.728 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.728 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-26T21:58:56.728 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.728 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.728 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.728 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.728 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.728 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.729 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.729 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.729 INFO:teuthology.orchestra.run.smithi072.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-26T21:58:56.729 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-26T21:58:56.729 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.729 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.729 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.729 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-26T21:58:56.729 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.729 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.729 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.729 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.729 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.729 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.729 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.729 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.730 INFO:teuthology.orchestra.run.smithi072.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-26T21:58:56.730 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-26T21:58:56.730 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.730 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.730 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.730 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-26T21:58:56.730 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.730 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.730 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.730 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.730 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.730 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.730 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.730 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.731 INFO:teuthology.orchestra.run.smithi072.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-26T21:58:56.731 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-26T21:58:56.731 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.731 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.731 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.731 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-26T21:58:56.731 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.731 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.731 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.731 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.731 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.731 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.731 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.731 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.731 INFO:teuthology.orchestra.run.smithi072.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-26T21:58:56.731 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-26T21:58:56.732 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.732 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.732 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.732 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-26T21:58:56.732 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.732 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.732 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.732 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.732 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.732 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.732 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.732 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.732 INFO:teuthology.orchestra.run.smithi072.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-26T21:58:56.732 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-26T21:58:56.732 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.732 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.732 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.733 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.733 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.733 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.733 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.733 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.733 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.733 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.733 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.733 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.733 INFO:teuthology.orchestra.run.smithi072.stdout: "GRAFANA_API_PASSWORD": { 2024-04-26T21:58:56.733 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-26T21:58:56.733 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.733 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.733 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.734 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "admin", 2024-04-26T21:58:56.734 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.734 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.734 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.734 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.734 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.734 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.734 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.734 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.734 INFO:teuthology.orchestra.run.smithi072.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-26T21:58:56.734 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-26T21:58:56.734 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.734 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.734 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.734 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-26T21:58:56.735 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.735 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.735 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.735 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.735 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.735 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.735 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.735 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.735 INFO:teuthology.orchestra.run.smithi072.stdout: "GRAFANA_API_URL": { 2024-04-26T21:58:56.735 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "GRAFANA_API_URL", 2024-04-26T21:58:56.735 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.735 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.735 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.735 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.735 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.736 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.736 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.736 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.736 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.736 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.736 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.736 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.736 INFO:teuthology.orchestra.run.smithi072.stdout: "GRAFANA_API_USERNAME": { 2024-04-26T21:58:56.736 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-26T21:58:56.736 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.736 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.736 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.736 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "admin", 2024-04-26T21:58:56.736 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.736 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.736 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.736 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.737 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.737 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.737 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.737 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.737 INFO:teuthology.orchestra.run.smithi072.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-26T21:58:56.737 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-26T21:58:56.737 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.737 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.737 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.737 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.737 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.737 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.737 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.737 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.737 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.738 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.738 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.738 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.738 INFO:teuthology.orchestra.run.smithi072.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-26T21:58:56.738 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-26T21:58:56.738 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.738 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.738 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.738 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.738 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.738 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.738 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.738 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.738 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.739 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.739 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.739 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.739 INFO:teuthology.orchestra.run.smithi072.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-26T21:58:56.739 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-26T21:58:56.739 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.739 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.739 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.739 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-26T21:58:56.739 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.739 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.739 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.739 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.739 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.739 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.739 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.740 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.740 INFO:teuthology.orchestra.run.smithi072.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-26T21:58:56.740 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-26T21:58:56.740 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.740 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.740 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.740 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.740 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.740 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.740 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.740 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.740 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.740 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.740 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.740 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.741 INFO:teuthology.orchestra.run.smithi072.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-26T21:58:56.741 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-26T21:58:56.741 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.741 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.741 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.741 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "[]", 2024-04-26T21:58:56.741 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.741 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.741 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.741 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.741 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.741 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.741 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.741 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.741 INFO:teuthology.orchestra.run.smithi072.stdout: "MULTICLUSTER_CONFIG": { 2024-04-26T21:58:56.741 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-26T21:58:56.741 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.742 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.742 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.742 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "{}", 2024-04-26T21:58:56.742 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.742 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.742 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.742 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.742 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.742 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.742 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.742 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.742 INFO:teuthology.orchestra.run.smithi072.stdout: "PROMETHEUS_API_HOST": { 2024-04-26T21:58:56.742 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-26T21:58:56.742 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.742 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.743 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.743 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.743 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.743 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.743 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.743 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.743 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.743 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.743 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.743 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.743 INFO:teuthology.orchestra.run.smithi072.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-26T21:58:56.743 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-26T21:58:56.743 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.743 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.743 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.743 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-26T21:58:56.744 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.744 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.744 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.744 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.744 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.744 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.744 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.744 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.744 INFO:teuthology.orchestra.run.smithi072.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-26T21:58:56.744 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-26T21:58:56.744 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.744 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.744 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.744 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.744 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.744 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.745 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.745 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.745 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.745 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.745 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.745 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.745 INFO:teuthology.orchestra.run.smithi072.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-26T21:58:56.745 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-26T21:58:56.745 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.745 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.745 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.745 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.745 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.745 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.745 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.746 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.746 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.746 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.746 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.746 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.746 INFO:teuthology.orchestra.run.smithi072.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-26T21:58:56.746 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-26T21:58:56.746 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.746 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.746 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.746 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-26T21:58:56.746 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.746 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.747 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.747 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.747 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.747 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.747 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.747 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.747 INFO:teuthology.orchestra.run.smithi072.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-26T21:58:56.747 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-26T21:58:56.747 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.747 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.747 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.747 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-26T21:58:56.747 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.747 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.747 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.748 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.748 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.748 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.748 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.748 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.748 INFO:teuthology.orchestra.run.smithi072.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-26T21:58:56.748 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-26T21:58:56.748 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.748 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.748 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.748 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.748 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.748 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.748 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.748 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.748 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.749 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.749 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.749 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.749 INFO:teuthology.orchestra.run.smithi072.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-26T21:58:56.749 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-26T21:58:56.749 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.749 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.749 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.749 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.749 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.749 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.749 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.749 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.749 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.749 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.750 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.750 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.750 INFO:teuthology.orchestra.run.smithi072.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-26T21:58:56.750 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-26T21:58:56.750 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.750 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.750 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.750 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.750 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.750 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.750 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.750 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.750 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.750 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.751 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.751 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.751 INFO:teuthology.orchestra.run.smithi072.stdout: "PWD_POLICY_ENABLED": { 2024-04-26T21:58:56.751 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-26T21:58:56.751 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.751 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.751 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.751 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-26T21:58:56.751 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.751 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.751 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.751 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.751 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.751 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.751 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.752 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.752 INFO:teuthology.orchestra.run.smithi072.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-26T21:58:56.752 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-26T21:58:56.752 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.752 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.752 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.752 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-26T21:58:56.752 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.752 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.752 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.752 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.752 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.752 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.752 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.752 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.752 INFO:teuthology.orchestra.run.smithi072.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-26T21:58:56.753 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-26T21:58:56.753 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-26T21:58:56.753 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.753 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.753 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "10", 2024-04-26T21:58:56.753 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.753 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.753 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.753 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.753 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.753 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.753 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.753 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.753 INFO:teuthology.orchestra.run.smithi072.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-26T21:58:56.753 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-26T21:58:56.754 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-26T21:58:56.754 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.754 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.754 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "8", 2024-04-26T21:58:56.754 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.754 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.754 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.754 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.754 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.754 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.754 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.754 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.754 INFO:teuthology.orchestra.run.smithi072.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-26T21:58:56.754 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-26T21:58:56.755 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-26T21:58:56.755 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.755 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.755 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "45", 2024-04-26T21:58:56.755 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.755 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.755 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.755 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.755 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.755 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.755 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.755 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.755 INFO:teuthology.orchestra.run.smithi072.stdout: "RGW_API_ACCESS_KEY": { 2024-04-26T21:58:56.755 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-26T21:58:56.756 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.756 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.756 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.756 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.756 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.756 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.756 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.756 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.756 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.756 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.756 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.756 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.756 INFO:teuthology.orchestra.run.smithi072.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-26T21:58:56.756 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-26T21:58:56.756 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.757 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.757 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.757 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "admin", 2024-04-26T21:58:56.757 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.757 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.757 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.757 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.757 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.757 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.757 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.757 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.757 INFO:teuthology.orchestra.run.smithi072.stdout: "RGW_API_SECRET_KEY": { 2024-04-26T21:58:56.757 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-26T21:58:56.757 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.757 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.758 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.758 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.758 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.758 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.758 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.758 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.758 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.758 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.758 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.758 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.758 INFO:teuthology.orchestra.run.smithi072.stdout: "RGW_API_SSL_VERIFY": { 2024-04-26T21:58:56.758 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-26T21:58:56.758 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.759 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.759 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.759 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-26T21:58:56.759 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.759 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.759 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.759 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.759 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.759 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.759 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.759 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.759 INFO:teuthology.orchestra.run.smithi072.stdout: "UNSAFE_TLS_v1_2": { 2024-04-26T21:58:56.759 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-26T21:58:56.759 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.759 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.760 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.760 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.760 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.760 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.760 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.760 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.760 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.760 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.760 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.760 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.760 INFO:teuthology.orchestra.run.smithi072.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-26T21:58:56.760 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-26T21:58:56.760 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-26T21:58:56.760 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.761 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.761 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "0", 2024-04-26T21:58:56.761 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.761 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.761 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.761 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.761 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.761 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.761 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.761 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.761 INFO:teuthology.orchestra.run.smithi072.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-26T21:58:56.761 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-26T21:58:56.761 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-26T21:58:56.761 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.761 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.762 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "10", 2024-04-26T21:58:56.762 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.762 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.762 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.762 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.762 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.762 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.762 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.762 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.762 INFO:teuthology.orchestra.run.smithi072.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-26T21:58:56.762 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-26T21:58:56.762 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-26T21:58:56.762 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.763 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.763 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "5", 2024-04-26T21:58:56.763 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.763 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.763 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.763 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.763 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.763 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.763 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.763 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.763 INFO:teuthology.orchestra.run.smithi072.stdout: "cross_origin_url": { 2024-04-26T21:58:56.763 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "cross_origin_url", 2024-04-26T21:58:56.763 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.763 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.764 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.764 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.764 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.764 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.764 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.764 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.764 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.764 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.764 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.764 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.764 INFO:teuthology.orchestra.run.smithi072.stdout: "crt_file": { 2024-04-26T21:58:56.764 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "crt_file", 2024-04-26T21:58:56.764 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.764 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.765 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.765 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.765 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.765 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.765 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.765 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.765 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.765 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.765 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.765 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.765 INFO:teuthology.orchestra.run.smithi072.stdout: "debug": { 2024-04-26T21:58:56.765 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "debug", 2024-04-26T21:58:56.765 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.765 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.765 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.766 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.766 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.766 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.766 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.766 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Enable/disable debug options", 2024-04-26T21:58:56.766 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.766 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.766 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.766 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.766 INFO:teuthology.orchestra.run.smithi072.stdout: "jwt_token_ttl": { 2024-04-26T21:58:56.766 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "jwt_token_ttl", 2024-04-26T21:58:56.766 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-26T21:58:56.766 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.766 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.767 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "28800", 2024-04-26T21:58:56.767 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.767 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.767 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.767 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.767 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.767 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.767 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.767 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.767 INFO:teuthology.orchestra.run.smithi072.stdout: "key_file": { 2024-04-26T21:58:56.767 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "key_file", 2024-04-26T21:58:56.767 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.767 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.767 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.768 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.768 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.768 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.768 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.768 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.768 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.768 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.768 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.768 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.768 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-26T21:58:56.768 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-26T21:58:56.768 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.768 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.768 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.768 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.768 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.769 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.769 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:56.769 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:56.769 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:56.769 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:56.769 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:56.769 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:56.769 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:56.769 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.769 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.769 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.769 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.769 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.769 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.769 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-26T21:58:56.769 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-26T21:58:56.770 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.770 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.770 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.770 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.770 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.770 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.770 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.770 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.770 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.770 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.770 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.770 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.770 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-26T21:58:56.770 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-26T21:58:56.770 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.771 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.771 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.771 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-26T21:58:56.771 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.771 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.771 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:56.771 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:56.771 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:56.771 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:56.771 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:56.771 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:56.771 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:56.771 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.771 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.771 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.772 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.772 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.772 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.772 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-26T21:58:56.772 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-26T21:58:56.772 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.772 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.772 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.772 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.772 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.772 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.772 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.772 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.772 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.772 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.772 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.773 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.773 INFO:teuthology.orchestra.run.smithi072.stdout: "motd": { 2024-04-26T21:58:56.773 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "motd", 2024-04-26T21:58:56.773 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.773 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.773 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.773 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.773 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.773 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.773 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.773 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "The message of the day", 2024-04-26T21:58:56.773 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.773 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.773 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.773 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.774 INFO:teuthology.orchestra.run.smithi072.stdout: "redirect_resolve_ip_addr": { 2024-04-26T21:58:56.774 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "redirect_resolve_ip_addr", 2024-04-26T21:58:56.774 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.774 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.774 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.774 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.774 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.774 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.774 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.774 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.774 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.774 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.774 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.774 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.775 INFO:teuthology.orchestra.run.smithi072.stdout: "server_addr": { 2024-04-26T21:58:56.775 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "server_addr", 2024-04-26T21:58:56.775 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.775 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.775 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.775 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "::", 2024-04-26T21:58:56.775 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.775 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.775 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.775 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.775 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.775 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.775 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.775 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.775 INFO:teuthology.orchestra.run.smithi072.stdout: "server_port": { 2024-04-26T21:58:56.776 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "server_port", 2024-04-26T21:58:56.776 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-26T21:58:56.776 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.776 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.776 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "8080", 2024-04-26T21:58:56.776 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.776 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.776 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.776 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.776 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.776 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.776 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.776 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.776 INFO:teuthology.orchestra.run.smithi072.stdout: "ssl": { 2024-04-26T21:58:56.777 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "ssl", 2024-04-26T21:58:56.777 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.777 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.777 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.777 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-26T21:58:56.777 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.777 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.777 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.777 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.777 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.777 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.777 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.777 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.777 INFO:teuthology.orchestra.run.smithi072.stdout: "ssl_server_port": { 2024-04-26T21:58:56.777 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "ssl_server_port", 2024-04-26T21:58:56.778 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-26T21:58:56.778 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.778 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.778 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "8443", 2024-04-26T21:58:56.778 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.778 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.778 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.778 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.778 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.778 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.778 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.778 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.778 INFO:teuthology.orchestra.run.smithi072.stdout: "standby_behaviour": { 2024-04-26T21:58:56.778 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "standby_behaviour", 2024-04-26T21:58:56.779 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.779 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.779 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.779 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "redirect", 2024-04-26T21:58:56.779 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.779 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.779 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:56.779 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:56.779 INFO:teuthology.orchestra.run.smithi072.stdout: "redirect" 2024-04-26T21:58:56.779 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.779 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.779 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.779 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.779 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.780 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.780 INFO:teuthology.orchestra.run.smithi072.stdout: "standby_error_status_code": { 2024-04-26T21:58:56.780 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "standby_error_status_code", 2024-04-26T21:58:56.780 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-26T21:58:56.780 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.780 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.780 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "500", 2024-04-26T21:58:56.780 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "400", 2024-04-26T21:58:56.780 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "599", 2024-04-26T21:58:56.780 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.780 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.780 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.780 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.780 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.780 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.781 INFO:teuthology.orchestra.run.smithi072.stdout: "url_prefix": { 2024-04-26T21:58:56.781 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "url_prefix", 2024-04-26T21:58:56.781 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.781 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.781 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.781 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.781 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.781 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.781 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.781 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.781 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.781 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.781 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.781 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:56.781 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:56.782 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.782 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-26T21:58:56.782 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "devicehealth", 2024-04-26T21:58:56.782 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-26T21:58:56.782 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-26T21:58:56.782 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-26T21:58:56.782 INFO:teuthology.orchestra.run.smithi072.stdout: "enable_monitoring": { 2024-04-26T21:58:56.782 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "enable_monitoring", 2024-04-26T21:58:56.782 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.782 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.782 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.782 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-26T21:58:56.782 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.782 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.782 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.783 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "monitor device health metrics", 2024-04-26T21:58:56.783 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.783 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.783 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.783 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.783 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-26T21:58:56.783 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-26T21:58:56.783 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.783 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.783 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.783 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.783 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.783 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.783 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:56.784 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:56.784 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:56.784 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:56.784 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:56.784 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:56.784 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:56.784 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.784 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.784 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.784 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.784 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.784 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.784 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-26T21:58:56.784 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-26T21:58:56.784 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.785 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.785 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.785 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.785 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.785 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.785 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.785 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.785 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.785 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.785 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.785 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.785 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-26T21:58:56.785 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-26T21:58:56.785 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.785 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.786 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.786 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-26T21:58:56.786 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.786 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.786 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:56.786 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:56.786 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:56.786 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:56.786 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:56.786 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:56.786 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:56.786 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.786 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.786 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.787 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.787 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.787 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.787 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-26T21:58:56.787 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-26T21:58:56.787 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.787 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.787 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.787 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.787 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.787 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.787 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.787 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.787 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.788 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.788 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.788 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.788 INFO:teuthology.orchestra.run.smithi072.stdout: "mark_out_threshold": { 2024-04-26T21:58:56.788 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "mark_out_threshold", 2024-04-26T21:58:56.788 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "secs", 2024-04-26T21:58:56.788 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.788 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.788 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "2419200", 2024-04-26T21:58:56.788 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.788 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.788 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.788 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-26T21:58:56.788 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.788 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.789 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.789 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.789 INFO:teuthology.orchestra.run.smithi072.stdout: "pool_name": { 2024-04-26T21:58:56.789 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "pool_name", 2024-04-26T21:58:56.789 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.789 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.789 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.789 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "device_health_metrics", 2024-04-26T21:58:56.789 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.789 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.789 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.789 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-26T21:58:56.789 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.789 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.789 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.790 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.790 INFO:teuthology.orchestra.run.smithi072.stdout: "retention_period": { 2024-04-26T21:58:56.790 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "retention_period", 2024-04-26T21:58:56.790 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "secs", 2024-04-26T21:58:56.790 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.790 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.790 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "15552000", 2024-04-26T21:58:56.790 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.790 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.790 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.791 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "how long to retain device health metrics", 2024-04-26T21:58:56.791 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.791 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.791 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.791 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.791 INFO:teuthology.orchestra.run.smithi072.stdout: "scrape_frequency": { 2024-04-26T21:58:56.791 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "scrape_frequency", 2024-04-26T21:58:56.791 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "secs", 2024-04-26T21:58:56.791 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.791 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.791 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "86400", 2024-04-26T21:58:56.791 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.791 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.792 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.792 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-26T21:58:56.792 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.792 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.792 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.792 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.792 INFO:teuthology.orchestra.run.smithi072.stdout: "self_heal": { 2024-04-26T21:58:56.792 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "self_heal", 2024-04-26T21:58:56.792 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.792 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.792 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.792 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-26T21:58:56.792 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.792 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.792 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.793 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-26T21:58:56.793 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.793 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.793 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.793 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.793 INFO:teuthology.orchestra.run.smithi072.stdout: "sleep_interval": { 2024-04-26T21:58:56.793 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "sleep_interval", 2024-04-26T21:58:56.793 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "secs", 2024-04-26T21:58:56.793 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.793 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.793 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "600", 2024-04-26T21:58:56.793 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.793 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.793 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.793 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "how frequently to wake up and check device health", 2024-04-26T21:58:56.793 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.794 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.794 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.794 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.794 INFO:teuthology.orchestra.run.smithi072.stdout: "warn_threshold": { 2024-04-26T21:58:56.794 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "warn_threshold", 2024-04-26T21:58:56.794 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "secs", 2024-04-26T21:58:56.794 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.794 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.794 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "7257600", 2024-04-26T21:58:56.794 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.794 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.794 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.794 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-26T21:58:56.794 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.795 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.795 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.795 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:56.795 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:56.795 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.795 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-26T21:58:56.795 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "diskprediction_local", 2024-04-26T21:58:56.795 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-26T21:58:56.795 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-26T21:58:56.795 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-26T21:58:56.795 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-26T21:58:56.795 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-26T21:58:56.795 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.795 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.795 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.796 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.796 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.796 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.796 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:56.796 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:56.796 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:56.796 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:56.796 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:56.796 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:56.796 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:56.796 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.796 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.796 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.796 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.797 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.797 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.797 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-26T21:58:56.797 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-26T21:58:56.797 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.797 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.797 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.797 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.797 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.797 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.797 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.797 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.797 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.797 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.797 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.797 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.798 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-26T21:58:56.798 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-26T21:58:56.798 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.798 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.798 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.798 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-26T21:58:56.798 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.798 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.798 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:56.798 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:56.798 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:56.798 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:56.798 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:56.798 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:56.798 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:56.799 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.799 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.799 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.799 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.799 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.799 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.799 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-26T21:58:56.799 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-26T21:58:56.799 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.799 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.799 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.799 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.799 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.799 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.799 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.799 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.800 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.800 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.800 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.800 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.800 INFO:teuthology.orchestra.run.smithi072.stdout: "predict_interval": { 2024-04-26T21:58:56.800 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "predict_interval", 2024-04-26T21:58:56.800 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.800 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.800 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.800 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "86400", 2024-04-26T21:58:56.800 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.800 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.800 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.800 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.800 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.801 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.801 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.801 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.801 INFO:teuthology.orchestra.run.smithi072.stdout: "predictor_model": { 2024-04-26T21:58:56.801 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "predictor_model", 2024-04-26T21:58:56.801 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.801 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.801 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.801 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "prophetstor", 2024-04-26T21:58:56.801 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.801 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.801 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.801 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.801 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.801 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.801 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.801 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.802 INFO:teuthology.orchestra.run.smithi072.stdout: "sleep_interval": { 2024-04-26T21:58:56.802 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "sleep_interval", 2024-04-26T21:58:56.802 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.802 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.802 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.802 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "600", 2024-04-26T21:58:56.802 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.802 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.802 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.802 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.802 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.802 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.802 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.802 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:56.802 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:56.803 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.803 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-26T21:58:56.803 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "influx", 2024-04-26T21:58:56.803 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": false, 2024-04-26T21:58:56.803 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "influxdb python module not found", 2024-04-26T21:58:56.803 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-26T21:58:56.803 INFO:teuthology.orchestra.run.smithi072.stdout: "batch_size": { 2024-04-26T21:58:56.803 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "batch_size", 2024-04-26T21:58:56.803 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-26T21:58:56.803 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.803 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.803 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "5000", 2024-04-26T21:58:56.803 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.803 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.803 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.803 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-26T21:58:56.803 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.804 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.804 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.804 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.804 INFO:teuthology.orchestra.run.smithi072.stdout: "database": { 2024-04-26T21:58:56.804 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "database", 2024-04-26T21:58:56.804 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.804 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.804 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.804 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "ceph", 2024-04-26T21:58:56.804 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.804 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.804 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.804 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-26T21:58:56.804 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.804 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.804 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.805 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.805 INFO:teuthology.orchestra.run.smithi072.stdout: "hostname": { 2024-04-26T21:58:56.805 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "hostname", 2024-04-26T21:58:56.805 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.805 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.805 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.805 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.805 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.805 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.805 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.805 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "InfluxDB server hostname", 2024-04-26T21:58:56.805 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.805 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.805 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.805 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.805 INFO:teuthology.orchestra.run.smithi072.stdout: "interval": { 2024-04-26T21:58:56.805 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "interval", 2024-04-26T21:58:56.806 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "secs", 2024-04-26T21:58:56.806 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.806 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.806 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "30", 2024-04-26T21:58:56.806 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "5", 2024-04-26T21:58:56.806 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.806 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.806 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-26T21:58:56.806 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.806 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.806 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.806 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.806 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-26T21:58:56.806 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-26T21:58:56.807 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.807 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.807 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.807 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.807 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.807 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.807 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:56.807 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:56.807 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:56.807 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:56.807 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:56.807 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:56.807 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:56.807 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.807 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.808 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.808 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.808 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.808 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.808 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-26T21:58:56.808 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-26T21:58:56.808 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.808 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.808 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.808 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.808 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.808 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.808 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.808 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.808 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.809 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.809 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.809 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.809 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-26T21:58:56.809 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-26T21:58:56.809 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.809 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.809 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.809 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-26T21:58:56.809 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.809 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.809 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:56.809 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:56.809 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:56.809 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:56.809 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:56.810 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:56.810 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:56.810 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.810 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.810 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.810 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.810 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.810 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.810 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-26T21:58:56.810 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-26T21:58:56.810 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.810 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.810 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.810 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.810 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.810 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.811 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.811 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.811 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.811 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.811 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.811 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.811 INFO:teuthology.orchestra.run.smithi072.stdout: "password": { 2024-04-26T21:58:56.811 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "password", 2024-04-26T21:58:56.811 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.811 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.811 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.811 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.811 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.811 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.811 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.811 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "password of InfluxDB server user", 2024-04-26T21:58:56.812 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.812 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.812 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.812 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.812 INFO:teuthology.orchestra.run.smithi072.stdout: "port": { 2024-04-26T21:58:56.812 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "port", 2024-04-26T21:58:56.812 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-26T21:58:56.812 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.812 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.812 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "8086", 2024-04-26T21:58:56.812 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.812 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.812 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.812 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "InfluxDB server port", 2024-04-26T21:58:56.812 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.812 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.813 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.813 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.813 INFO:teuthology.orchestra.run.smithi072.stdout: "ssl": { 2024-04-26T21:58:56.813 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "ssl", 2024-04-26T21:58:56.813 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.813 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.813 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.813 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "false", 2024-04-26T21:58:56.813 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.813 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.813 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.813 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-26T21:58:56.813 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.813 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.813 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.813 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.813 INFO:teuthology.orchestra.run.smithi072.stdout: "threads": { 2024-04-26T21:58:56.814 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "threads", 2024-04-26T21:58:56.814 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-26T21:58:56.814 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.814 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.814 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "5", 2024-04-26T21:58:56.814 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "1", 2024-04-26T21:58:56.814 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "32", 2024-04-26T21:58:56.814 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.814 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-26T21:58:56.814 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.814 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.814 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.814 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.814 INFO:teuthology.orchestra.run.smithi072.stdout: "username": { 2024-04-26T21:58:56.814 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "username", 2024-04-26T21:58:56.815 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.815 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.815 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.815 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.815 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.815 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.815 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.815 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "username of InfluxDB server user", 2024-04-26T21:58:56.815 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.815 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.815 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.815 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.815 INFO:teuthology.orchestra.run.smithi072.stdout: "verify_ssl": { 2024-04-26T21:58:56.815 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "verify_ssl", 2024-04-26T21:58:56.815 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.815 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.815 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.816 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "true", 2024-04-26T21:58:56.816 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.816 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.816 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.816 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-26T21:58:56.816 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.816 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.816 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.816 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:56.816 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:56.816 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.816 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-26T21:58:56.816 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "insights", 2024-04-26T21:58:56.816 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-26T21:58:56.816 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-26T21:58:56.816 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-26T21:58:56.816 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-26T21:58:56.817 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-26T21:58:56.817 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.817 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.817 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.817 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.817 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.817 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.817 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:56.817 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:56.817 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:56.817 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:56.817 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:56.817 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:56.817 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:56.817 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.817 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.818 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.818 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.818 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.818 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.818 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-26T21:58:56.818 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-26T21:58:56.818 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.818 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.818 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.818 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.818 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.818 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.818 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.818 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.818 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.819 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.819 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.819 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.819 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-26T21:58:56.819 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-26T21:58:56.819 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.819 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.819 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.819 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-26T21:58:56.819 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.819 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.819 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:56.819 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:56.819 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:56.819 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:56.820 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:56.820 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:56.820 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:56.820 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.820 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.820 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.820 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.820 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.820 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.820 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-26T21:58:56.820 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-26T21:58:56.820 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.820 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.820 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.821 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.821 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.821 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.821 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.821 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.821 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.821 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.821 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.821 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:56.821 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:56.821 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.821 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-26T21:58:56.821 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "iostat", 2024-04-26T21:58:56.821 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-26T21:58:56.821 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-26T21:58:56.822 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-26T21:58:56.822 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-26T21:58:56.822 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-26T21:58:56.822 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.822 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.822 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.822 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.822 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.822 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.822 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:56.822 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:56.822 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:56.822 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:56.822 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:56.822 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:56.823 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:56.823 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.823 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.823 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.823 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.823 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.823 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.823 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-26T21:58:56.823 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-26T21:58:56.823 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.823 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.823 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.823 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.823 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.823 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.824 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.824 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.824 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.824 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.824 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.824 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.824 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-26T21:58:56.824 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-26T21:58:56.824 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.824 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.824 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.824 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-26T21:58:56.824 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.824 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.825 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:56.825 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:56.825 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:56.825 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:56.825 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:56.825 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:56.825 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:56.825 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.825 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.825 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.825 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.825 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.825 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.825 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-26T21:58:56.825 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-26T21:58:56.826 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.826 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.826 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.826 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.826 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.826 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.826 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.826 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.826 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.826 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.826 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.826 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:56.826 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:56.826 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.826 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-26T21:58:56.827 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "k8sevents", 2024-04-26T21:58:56.827 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-26T21:58:56.827 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-26T21:58:56.827 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-26T21:58:56.827 INFO:teuthology.orchestra.run.smithi072.stdout: "ceph_event_retention_days": { 2024-04-26T21:58:56.827 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "ceph_event_retention_days", 2024-04-26T21:58:56.827 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-26T21:58:56.827 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.827 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.827 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "7", 2024-04-26T21:58:56.827 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.827 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.827 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.827 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-26T21:58:56.827 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.827 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.828 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.828 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.828 INFO:teuthology.orchestra.run.smithi072.stdout: "config_check_secs": { 2024-04-26T21:58:56.828 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "config_check_secs", 2024-04-26T21:58:56.828 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-26T21:58:56.828 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.828 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.828 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "10", 2024-04-26T21:58:56.828 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "10", 2024-04-26T21:58:56.828 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.828 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.828 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-26T21:58:56.828 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.828 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.829 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.829 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.829 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-26T21:58:56.829 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-26T21:58:56.829 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.829 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.829 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.829 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.829 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.829 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.829 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:56.829 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:56.829 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:56.829 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:56.829 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:56.829 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:56.830 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:56.830 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.830 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.830 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.830 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.830 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.830 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.830 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-26T21:58:56.830 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-26T21:58:56.830 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.830 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.830 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.830 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.830 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.830 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.831 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.831 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.831 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.831 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.831 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.831 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.831 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-26T21:58:56.831 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-26T21:58:56.831 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.831 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.831 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.831 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-26T21:58:56.831 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.831 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.831 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:56.831 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:56.832 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:56.832 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:56.832 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:56.832 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:56.832 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:56.832 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.832 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.832 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.832 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.832 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.832 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.832 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-26T21:58:56.832 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-26T21:58:56.832 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.832 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.833 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.833 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.833 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.833 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.833 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.833 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.833 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.833 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.833 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.833 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:56.833 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:56.833 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.833 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-26T21:58:56.833 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "localpool", 2024-04-26T21:58:56.833 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-26T21:58:56.834 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-26T21:58:56.834 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-26T21:58:56.834 INFO:teuthology.orchestra.run.smithi072.stdout: "failure_domain": { 2024-04-26T21:58:56.834 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "failure_domain", 2024-04-26T21:58:56.834 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.834 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.834 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.834 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "host", 2024-04-26T21:58:56.834 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.834 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.834 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.834 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "failure domain for any created local pool", 2024-04-26T21:58:56.834 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-26T21:58:56.834 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.834 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.835 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.835 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-26T21:58:56.835 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-26T21:58:56.835 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.835 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.835 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.835 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.835 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.835 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.835 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:56.835 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:56.835 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:56.835 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:56.835 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:56.835 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:56.835 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:56.836 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.836 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.836 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.836 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.836 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.836 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.836 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-26T21:58:56.836 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-26T21:58:56.836 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.836 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.836 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.836 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.836 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.836 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.836 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.837 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.837 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.837 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.837 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.837 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.837 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-26T21:58:56.837 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-26T21:58:56.837 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.837 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.837 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.837 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-26T21:58:56.837 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.837 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.837 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:56.837 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:56.837 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:56.838 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:56.838 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:56.838 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:56.838 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:56.838 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.838 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.838 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.838 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.838 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.838 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.838 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-26T21:58:56.838 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-26T21:58:56.838 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.839 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.839 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.839 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.839 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.839 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.839 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.839 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.839 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.839 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.839 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.839 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.839 INFO:teuthology.orchestra.run.smithi072.stdout: "min_size": { 2024-04-26T21:58:56.839 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "min_size", 2024-04-26T21:58:56.839 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-26T21:58:56.839 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.840 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.840 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.840 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.840 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.840 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.840 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "default min_size for any created local pool", 2024-04-26T21:58:56.840 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-26T21:58:56.840 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.840 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.840 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.840 INFO:teuthology.orchestra.run.smithi072.stdout: "num_rep": { 2024-04-26T21:58:56.840 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "num_rep", 2024-04-26T21:58:56.840 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-26T21:58:56.840 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.840 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.841 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "3", 2024-04-26T21:58:56.841 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.841 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.841 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.841 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "default replica count for any created local pool", 2024-04-26T21:58:56.841 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.841 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.841 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.841 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.841 INFO:teuthology.orchestra.run.smithi072.stdout: "pg_num": { 2024-04-26T21:58:56.841 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "pg_num", 2024-04-26T21:58:56.841 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-26T21:58:56.841 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.841 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.841 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "128", 2024-04-26T21:58:56.842 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.842 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.842 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.842 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "default pg_num for any created local pool", 2024-04-26T21:58:56.842 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.842 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.842 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.842 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.842 INFO:teuthology.orchestra.run.smithi072.stdout: "prefix": { 2024-04-26T21:58:56.842 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "prefix", 2024-04-26T21:58:56.842 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.842 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.842 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.842 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.843 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.843 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.843 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.845 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "name prefix for any created local pool", 2024-04-26T21:58:56.845 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.846 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.846 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.846 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.846 INFO:teuthology.orchestra.run.smithi072.stdout: "subtree": { 2024-04-26T21:58:56.846 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "subtree", 2024-04-26T21:58:56.846 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.846 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.846 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.846 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "rack", 2024-04-26T21:58:56.846 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.846 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.846 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.846 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-26T21:58:56.846 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-26T21:58:56.847 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.847 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.847 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:56.847 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:56.847 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.847 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-26T21:58:56.847 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "mds_autoscaler", 2024-04-26T21:58:56.847 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-26T21:58:56.847 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-26T21:58:56.847 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-26T21:58:56.847 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-26T21:58:56.847 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-26T21:58:56.847 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.847 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.847 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.848 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.848 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.848 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.848 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:56.848 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:56.848 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:56.848 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:56.848 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:56.848 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:56.848 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:56.848 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.848 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.848 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.848 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.848 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.849 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.849 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-26T21:58:56.849 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-26T21:58:56.849 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.849 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.849 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.849 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.849 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.849 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.849 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.849 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.849 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.849 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.849 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.849 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.850 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-26T21:58:56.850 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-26T21:58:56.850 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.850 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.850 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.850 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-26T21:58:56.850 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.850 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.850 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:56.850 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:56.850 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:56.850 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:56.850 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:56.850 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:56.850 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:56.851 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.851 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.851 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.851 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.851 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.851 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.851 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-26T21:58:56.851 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-26T21:58:56.851 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.851 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.851 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.851 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.851 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.851 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.851 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.852 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.852 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.852 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.852 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.852 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:56.852 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:56.852 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.852 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-26T21:58:56.852 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "mirroring", 2024-04-26T21:58:56.852 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-26T21:58:56.852 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-26T21:58:56.852 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-26T21:58:56.852 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-26T21:58:56.852 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-26T21:58:56.852 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.853 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.853 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.853 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.853 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.853 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.853 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:56.853 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:56.853 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:56.853 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:56.853 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:56.853 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:56.853 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:56.853 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.853 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.854 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.854 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.854 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.854 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.854 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-26T21:58:56.854 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-26T21:58:56.854 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.854 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.854 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.854 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.854 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.854 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.854 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.854 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.855 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.855 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.855 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.855 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.855 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-26T21:58:56.855 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-26T21:58:56.855 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.855 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.855 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.855 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-26T21:58:56.855 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.855 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.855 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:56.856 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:56.856 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:56.856 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:56.856 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:56.856 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:56.856 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:56.856 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.856 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.856 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.857 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.857 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.857 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.857 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-26T21:58:56.857 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-26T21:58:56.857 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.857 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.857 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.857 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.857 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.857 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.857 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.858 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.858 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.858 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.858 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.858 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:56.858 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:56.858 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.858 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-26T21:58:56.858 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "nfs", 2024-04-26T21:58:56.858 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-26T21:58:56.858 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-26T21:58:56.858 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-26T21:58:56.858 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-26T21:58:56.858 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-26T21:58:56.859 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.859 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.859 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.859 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.859 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.859 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.859 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:56.859 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:56.859 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:56.859 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:56.859 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:56.859 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:56.859 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:56.859 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.859 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.860 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.860 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.860 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.860 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.860 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-26T21:58:56.860 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-26T21:58:56.860 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.860 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.860 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.860 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.860 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.860 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.860 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.860 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.860 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.861 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.861 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.861 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.861 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-26T21:58:56.861 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-26T21:58:56.861 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.861 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.861 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.861 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-26T21:58:56.861 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.861 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.861 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:56.861 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:56.861 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:56.862 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:56.862 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:56.862 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:56.862 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:56.862 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.862 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.862 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.862 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.862 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.862 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.862 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-26T21:58:56.862 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-26T21:58:56.862 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.862 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.862 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.863 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.863 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.863 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.863 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.863 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.863 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.863 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.863 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.863 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:56.863 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:56.863 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.863 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-26T21:58:56.863 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "orchestrator", 2024-04-26T21:58:56.863 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-26T21:58:56.863 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-26T21:58:56.864 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-26T21:58:56.864 INFO:teuthology.orchestra.run.smithi072.stdout: "fail_fs": { 2024-04-26T21:58:56.864 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "fail_fs", 2024-04-26T21:58:56.864 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.864 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.864 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.864 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.864 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.864 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.864 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.864 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-26T21:58:56.864 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.864 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.864 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.864 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.865 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-26T21:58:56.865 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-26T21:58:56.865 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.865 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.865 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.865 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.865 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.865 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.865 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:56.865 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:56.865 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:56.865 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:56.865 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:56.865 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:56.865 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:56.866 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.866 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.866 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.866 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.866 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.866 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.866 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-26T21:58:56.866 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-26T21:58:56.866 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.866 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.866 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.866 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.866 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.866 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.867 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.867 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.867 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.867 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.867 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.867 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.867 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-26T21:58:56.867 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-26T21:58:56.867 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.867 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.867 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.867 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-26T21:58:56.867 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.867 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.867 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:56.868 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:56.868 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:56.868 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:56.868 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:56.868 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:56.868 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:56.868 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.868 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.868 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.868 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.868 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.868 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.868 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-26T21:58:56.868 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-26T21:58:56.868 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.868 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.869 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.869 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.869 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.869 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.869 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.869 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.869 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.869 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.869 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.869 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.869 INFO:teuthology.orchestra.run.smithi072.stdout: "orchestrator": { 2024-04-26T21:58:56.869 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "orchestrator", 2024-04-26T21:58:56.869 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.869 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.869 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.870 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.870 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.870 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.870 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:56.870 INFO:teuthology.orchestra.run.smithi072.stdout: "cephadm", 2024-04-26T21:58:56.870 INFO:teuthology.orchestra.run.smithi072.stdout: "rook", 2024-04-26T21:58:56.870 INFO:teuthology.orchestra.run.smithi072.stdout: "test_orchestrator" 2024-04-26T21:58:56.870 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.870 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Orchestrator backend", 2024-04-26T21:58:56.870 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.870 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.870 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.870 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:56.870 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:56.870 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.871 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-26T21:58:56.871 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "osd_perf_query", 2024-04-26T21:58:56.871 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-26T21:58:56.871 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-26T21:58:56.871 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-26T21:58:56.871 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-26T21:58:56.871 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-26T21:58:56.871 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.871 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.871 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.871 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.871 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.871 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.871 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:56.871 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:56.871 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:56.872 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:56.872 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:56.872 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:56.872 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:56.872 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.872 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.872 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.872 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.872 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.872 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.872 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-26T21:58:56.872 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-26T21:58:56.872 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.872 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.872 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.872 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.873 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.873 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.873 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.873 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.873 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.873 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.873 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.873 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.873 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-26T21:58:56.873 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-26T21:58:56.873 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.873 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.873 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.873 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-26T21:58:56.873 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.874 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.874 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:56.874 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:56.874 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:56.874 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:56.874 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:56.874 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:56.874 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:56.874 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.874 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.874 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.874 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.874 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.874 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.875 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-26T21:58:56.875 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-26T21:58:56.875 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.875 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.875 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.875 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.875 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.875 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.875 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.875 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.875 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.875 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.875 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.875 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:56.875 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:56.876 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.876 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-26T21:58:56.876 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "osd_support", 2024-04-26T21:58:56.876 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-26T21:58:56.876 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-26T21:58:56.876 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-26T21:58:56.876 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-26T21:58:56.876 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-26T21:58:56.876 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.876 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.876 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.876 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.876 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.876 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.876 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:56.877 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:56.877 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:56.877 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:56.877 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:56.877 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:56.877 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:56.877 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.877 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.877 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.877 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.877 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.877 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.877 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-26T21:58:56.877 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-26T21:58:56.877 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.878 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.878 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.878 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.878 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.878 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.878 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.878 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.878 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.878 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.878 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.878 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.878 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-26T21:58:56.878 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-26T21:58:56.878 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.878 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.879 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.879 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-26T21:58:56.879 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.879 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.879 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:56.879 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:56.879 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:56.879 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:56.879 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:56.879 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:56.879 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:56.879 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.879 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.879 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.879 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.880 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.880 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.880 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-26T21:58:56.880 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-26T21:58:56.880 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.880 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.880 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.880 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.880 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.880 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.880 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.880 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.880 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.880 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.881 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.881 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:56.881 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:56.881 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.881 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-26T21:58:56.881 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "pg_autoscaler", 2024-04-26T21:58:56.881 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-26T21:58:56.881 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-26T21:58:56.881 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-26T21:58:56.881 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-26T21:58:56.881 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-26T21:58:56.881 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.881 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.881 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.881 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.882 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.882 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.882 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:56.882 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:56.882 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:56.882 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:56.882 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:56.882 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:56.882 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:56.882 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.882 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.882 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.882 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.882 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.882 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.883 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-26T21:58:56.883 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-26T21:58:56.883 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.883 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.883 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.883 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.883 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.883 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.883 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.883 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.883 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.883 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.883 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.883 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.883 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-26T21:58:56.884 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-26T21:58:56.884 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.884 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.884 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.884 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-26T21:58:56.884 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.884 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.884 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:56.884 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:56.884 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:56.884 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:56.884 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:56.884 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:56.884 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:56.884 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.885 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.885 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.885 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.885 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.885 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.885 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-26T21:58:56.885 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-26T21:58:56.885 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.885 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.885 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.885 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.885 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.885 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.885 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.885 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.886 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.886 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.886 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.886 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.886 INFO:teuthology.orchestra.run.smithi072.stdout: "sleep_interval": { 2024-04-26T21:58:56.886 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "sleep_interval", 2024-04-26T21:58:56.886 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "secs", 2024-04-26T21:58:56.886 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.886 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.886 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "60", 2024-04-26T21:58:56.886 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.886 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.886 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.886 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.886 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.887 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.887 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.887 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.887 INFO:teuthology.orchestra.run.smithi072.stdout: "threshold": { 2024-04-26T21:58:56.887 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "threshold", 2024-04-26T21:58:56.887 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "float", 2024-04-26T21:58:56.887 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.887 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.887 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "3.0", 2024-04-26T21:58:56.887 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "1.0", 2024-04-26T21:58:56.887 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.887 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.887 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "scaling threshold", 2024-04-26T21:58:56.887 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-26T21:58:56.888 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.888 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.888 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:56.888 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:56.888 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.888 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-26T21:58:56.888 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "progress", 2024-04-26T21:58:56.888 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-26T21:58:56.888 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-26T21:58:56.888 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-26T21:58:56.888 INFO:teuthology.orchestra.run.smithi072.stdout: "allow_pg_recovery_event": { 2024-04-26T21:58:56.888 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "allow_pg_recovery_event", 2024-04-26T21:58:56.888 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.888 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.888 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.889 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.889 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.889 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.889 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.889 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-26T21:58:56.889 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.889 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.889 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.889 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.889 INFO:teuthology.orchestra.run.smithi072.stdout: "enabled": { 2024-04-26T21:58:56.889 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "enabled", 2024-04-26T21:58:56.889 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.889 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.889 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.889 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-26T21:58:56.890 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.890 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.890 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.890 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.890 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.890 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.890 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.890 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.890 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-26T21:58:56.890 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-26T21:58:56.890 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.890 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.890 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.890 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.890 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.891 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.891 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:56.891 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:56.891 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:56.891 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:56.891 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:56.891 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:56.891 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:56.891 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.891 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.891 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.891 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.891 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.891 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.891 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-26T21:58:56.892 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-26T21:58:56.892 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.892 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.892 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.892 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.892 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.892 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.892 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.892 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.892 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.892 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.892 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.892 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.892 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-26T21:58:56.892 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-26T21:58:56.893 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.893 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.893 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.893 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-26T21:58:56.893 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.893 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.893 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:56.893 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:56.893 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:56.893 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:56.893 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:56.893 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:56.893 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:56.893 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.893 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.894 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.894 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.894 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.894 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.894 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-26T21:58:56.894 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-26T21:58:56.894 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.894 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.894 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.894 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.894 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.894 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.894 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.894 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.895 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.895 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.895 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.895 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.895 INFO:teuthology.orchestra.run.smithi072.stdout: "max_completed_events": { 2024-04-26T21:58:56.895 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "max_completed_events", 2024-04-26T21:58:56.895 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-26T21:58:56.895 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.895 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.895 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "50", 2024-04-26T21:58:56.895 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.895 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.895 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.895 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "number of past completed events to remember", 2024-04-26T21:58:56.895 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.896 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.896 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.896 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.896 INFO:teuthology.orchestra.run.smithi072.stdout: "sleep_interval": { 2024-04-26T21:58:56.896 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "sleep_interval", 2024-04-26T21:58:56.896 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "secs", 2024-04-26T21:58:56.896 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.896 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.896 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "5", 2024-04-26T21:58:56.896 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.896 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.896 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.896 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "how long the module is going to sleep", 2024-04-26T21:58:56.896 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.896 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.896 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.897 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:56.897 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:56.897 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.897 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-26T21:58:56.897 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "prometheus", 2024-04-26T21:58:56.897 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-26T21:58:56.897 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-26T21:58:56.897 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-26T21:58:56.897 INFO:teuthology.orchestra.run.smithi072.stdout: "cache": { 2024-04-26T21:58:56.897 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "cache", 2024-04-26T21:58:56.897 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.897 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.897 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.897 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-26T21:58:56.897 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.898 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.898 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.898 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.898 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.898 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.898 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.898 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.898 INFO:teuthology.orchestra.run.smithi072.stdout: "exclude_perf_counters": { 2024-04-26T21:58:56.898 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "exclude_perf_counters", 2024-04-26T21:58:56.898 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.898 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.898 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.898 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-26T21:58:56.898 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.898 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.899 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.899 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-26T21:58:56.899 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-26T21:58:56.899 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.899 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.899 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.899 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-26T21:58:56.899 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-26T21:58:56.899 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.899 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.899 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.899 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.899 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.899 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.899 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:56.900 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:56.900 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:56.900 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:56.900 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:56.900 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:56.900 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:56.900 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.900 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.900 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.900 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.900 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.900 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.900 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-26T21:58:56.900 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-26T21:58:56.900 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.900 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.901 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.901 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.901 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.901 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.901 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.901 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.901 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.901 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.901 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.901 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.901 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-26T21:58:56.901 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-26T21:58:56.901 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.901 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.901 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.902 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-26T21:58:56.902 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.902 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.902 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:56.902 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:56.902 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:56.902 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:56.902 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:56.902 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:56.902 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:56.902 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.902 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.902 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.902 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.902 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.903 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.903 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-26T21:58:56.903 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-26T21:58:56.903 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.903 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.903 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.903 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.903 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.903 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.903 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.903 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.903 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.903 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.903 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.904 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.904 INFO:teuthology.orchestra.run.smithi072.stdout: "rbd_stats_pools": { 2024-04-26T21:58:56.904 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "rbd_stats_pools", 2024-04-26T21:58:56.904 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.904 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.904 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.904 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.904 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.904 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.904 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.904 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.904 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.904 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.904 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.904 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.905 INFO:teuthology.orchestra.run.smithi072.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-26T21:58:56.905 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-26T21:58:56.905 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-26T21:58:56.905 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.905 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.905 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "300", 2024-04-26T21:58:56.905 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.905 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.905 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.905 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.905 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.905 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.905 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.905 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.905 INFO:teuthology.orchestra.run.smithi072.stdout: "scrape_interval": { 2024-04-26T21:58:56.906 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "scrape_interval", 2024-04-26T21:58:56.906 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "float", 2024-04-26T21:58:56.906 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.906 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.906 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "15.0", 2024-04-26T21:58:56.906 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.906 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.906 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.906 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.906 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.906 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.906 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.906 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.906 INFO:teuthology.orchestra.run.smithi072.stdout: "server_addr": { 2024-04-26T21:58:56.907 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "server_addr", 2024-04-26T21:58:56.907 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.907 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.907 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.907 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "::", 2024-04-26T21:58:56.907 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.907 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.907 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.907 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-26T21:58:56.907 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.907 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.907 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.907 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.907 INFO:teuthology.orchestra.run.smithi072.stdout: "server_port": { 2024-04-26T21:58:56.907 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "server_port", 2024-04-26T21:58:56.908 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-26T21:58:56.908 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.908 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.908 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "9283", 2024-04-26T21:58:56.908 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.908 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.908 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.908 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-26T21:58:56.908 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.908 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.908 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.908 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.908 INFO:teuthology.orchestra.run.smithi072.stdout: "stale_cache_strategy": { 2024-04-26T21:58:56.908 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "stale_cache_strategy", 2024-04-26T21:58:56.908 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.908 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.909 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.909 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "log", 2024-04-26T21:58:56.909 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.909 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.909 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.909 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.909 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.909 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.909 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.909 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.909 INFO:teuthology.orchestra.run.smithi072.stdout: "standby_behaviour": { 2024-04-26T21:58:56.909 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "standby_behaviour", 2024-04-26T21:58:56.909 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.909 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.909 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.910 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "default", 2024-04-26T21:58:56.910 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.910 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.910 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:56.910 INFO:teuthology.orchestra.run.smithi072.stdout: "default", 2024-04-26T21:58:56.910 INFO:teuthology.orchestra.run.smithi072.stdout: "error" 2024-04-26T21:58:56.910 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.910 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.910 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.910 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.910 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.910 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.910 INFO:teuthology.orchestra.run.smithi072.stdout: "standby_error_status_code": { 2024-04-26T21:58:56.910 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "standby_error_status_code", 2024-04-26T21:58:56.910 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-26T21:58:56.911 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.911 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.911 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "500", 2024-04-26T21:58:56.911 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "400", 2024-04-26T21:58:56.911 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "599", 2024-04-26T21:58:56.911 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.911 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.911 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.911 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.911 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.911 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:56.911 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:56.911 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.911 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-26T21:58:56.911 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "rbd_support", 2024-04-26T21:58:56.912 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-26T21:58:56.912 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-26T21:58:56.912 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-26T21:58:56.912 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-26T21:58:56.912 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-26T21:58:56.912 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.912 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.912 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.912 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.912 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.912 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.912 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:56.912 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:56.912 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:56.912 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:56.913 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:56.913 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:56.913 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:56.913 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.913 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.913 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.913 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.913 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.913 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.913 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-26T21:58:56.913 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-26T21:58:56.913 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.913 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.913 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.913 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.913 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.914 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.914 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.914 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.914 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.914 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.914 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.914 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.914 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-26T21:58:56.914 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-26T21:58:56.914 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.914 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.914 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.914 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-26T21:58:56.914 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.915 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.915 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:56.915 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:56.915 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:56.915 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:56.915 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:56.915 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:56.915 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:56.915 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.915 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.915 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.915 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.915 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.915 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.915 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-26T21:58:56.915 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-26T21:58:56.916 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.916 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.916 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.916 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.916 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.916 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.916 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.916 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.916 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.916 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.916 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.916 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.916 INFO:teuthology.orchestra.run.smithi072.stdout: "max_concurrent_snap_create": { 2024-04-26T21:58:56.916 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "max_concurrent_snap_create", 2024-04-26T21:58:56.916 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-26T21:58:56.917 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.917 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.917 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "10", 2024-04-26T21:58:56.917 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.917 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.917 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.917 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.917 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.917 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.917 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.917 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.917 INFO:teuthology.orchestra.run.smithi072.stdout: "mirror_snapshot_schedule": { 2024-04-26T21:58:56.917 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "mirror_snapshot_schedule", 2024-04-26T21:58:56.917 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.917 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.917 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.918 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.918 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.918 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.918 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.918 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.918 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.918 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.918 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.918 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.918 INFO:teuthology.orchestra.run.smithi072.stdout: "trash_purge_schedule": { 2024-04-26T21:58:56.918 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "trash_purge_schedule", 2024-04-26T21:58:56.918 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.918 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.918 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.919 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.919 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.919 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.919 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.919 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.919 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.919 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.919 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.919 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:56.919 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:56.919 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.919 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-26T21:58:56.919 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "restful", 2024-04-26T21:58:56.919 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-26T21:58:56.919 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-26T21:58:56.920 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-26T21:58:56.920 INFO:teuthology.orchestra.run.smithi072.stdout: "enable_auth": { 2024-04-26T21:58:56.920 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "enable_auth", 2024-04-26T21:58:56.920 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.920 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.920 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.920 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-26T21:58:56.920 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.920 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.920 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.920 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.920 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.920 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.920 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.920 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.921 INFO:teuthology.orchestra.run.smithi072.stdout: "key_file": { 2024-04-26T21:58:56.921 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "key_file", 2024-04-26T21:58:56.921 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.921 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.921 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.921 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.921 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.921 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.921 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.921 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.921 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.921 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.921 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.921 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.921 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-26T21:58:56.922 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-26T21:58:56.922 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.922 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.922 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.922 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.922 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.922 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.922 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:56.922 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:56.922 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:56.922 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:56.922 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:56.922 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:56.922 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:56.923 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.923 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.923 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.923 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.923 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.923 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.923 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-26T21:58:56.923 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-26T21:58:56.923 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.923 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.923 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.923 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.923 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.923 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.923 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.924 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.924 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.924 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.924 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.924 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.924 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-26T21:58:56.924 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-26T21:58:56.924 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.924 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.924 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.924 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-26T21:58:56.924 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.924 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.924 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:56.924 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:56.924 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:56.925 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:56.925 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:56.925 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:56.925 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:56.925 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.925 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.925 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.925 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.925 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.925 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.925 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-26T21:58:56.925 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-26T21:58:56.925 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.925 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.925 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.926 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.926 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.926 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.926 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.926 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.926 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.926 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.926 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.926 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.926 INFO:teuthology.orchestra.run.smithi072.stdout: "server_addr": { 2024-04-26T21:58:56.926 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "server_addr", 2024-04-26T21:58:56.926 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.926 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.926 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.926 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.927 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.927 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.927 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.927 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.927 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.927 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.927 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.927 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.927 INFO:teuthology.orchestra.run.smithi072.stdout: "server_port": { 2024-04-26T21:58:56.927 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "server_port", 2024-04-26T21:58:56.927 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.927 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.927 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.927 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.927 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.928 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.928 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.928 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.928 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.928 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.928 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.928 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:56.928 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:56.928 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.928 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-26T21:58:56.928 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "rgw", 2024-04-26T21:58:56.928 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-26T21:58:56.928 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-26T21:58:56.928 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-26T21:58:56.928 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-26T21:58:56.929 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-26T21:58:56.929 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.929 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.929 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.929 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.929 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.929 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.929 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:56.929 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:56.929 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:56.929 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:56.929 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:56.929 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:56.929 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:56.929 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.930 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.930 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.930 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.930 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.930 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.930 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-26T21:58:56.930 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-26T21:58:56.930 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.930 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.930 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.930 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.930 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.930 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.930 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.931 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.931 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.931 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.931 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.931 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.931 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-26T21:58:56.931 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-26T21:58:56.931 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.931 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.931 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.931 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-26T21:58:56.931 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.931 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.931 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:56.931 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:56.931 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:56.932 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:56.932 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:56.932 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:56.932 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:56.932 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.932 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.932 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.932 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.932 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.932 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.932 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-26T21:58:56.932 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-26T21:58:56.932 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.932 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.932 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.933 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.933 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.933 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.933 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.933 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.933 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.933 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.933 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.933 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:56.933 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:56.933 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.933 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-26T21:58:56.933 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "rook", 2024-04-26T21:58:56.933 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-26T21:58:56.933 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-26T21:58:56.934 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-26T21:58:56.934 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-26T21:58:56.934 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-26T21:58:56.934 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.934 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.934 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.934 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.934 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.934 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.934 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:56.934 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:56.934 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:56.934 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:56.934 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:56.934 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:56.935 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:56.935 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.935 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.935 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.935 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.935 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.935 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.935 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-26T21:58:56.935 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-26T21:58:56.935 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.935 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.935 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.935 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.935 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.935 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.936 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.936 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.936 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.936 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.936 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.936 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.936 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-26T21:58:56.936 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-26T21:58:56.936 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.936 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.936 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.936 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-26T21:58:56.936 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.936 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.936 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:56.937 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:56.937 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:56.937 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:56.937 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:56.937 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:56.937 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:56.937 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.937 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.937 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.937 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.937 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.937 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.937 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-26T21:58:56.937 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-26T21:58:56.937 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.938 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.938 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.938 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.938 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.938 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.938 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.938 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.938 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.938 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.938 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.938 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.938 INFO:teuthology.orchestra.run.smithi072.stdout: "storage_class": { 2024-04-26T21:58:56.938 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "storage_class", 2024-04-26T21:58:56.938 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.939 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.939 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.939 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "local", 2024-04-26T21:58:56.939 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.939 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.939 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.939 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-26T21:58:56.939 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.939 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.939 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.939 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:56.939 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:56.939 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.939 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-26T21:58:56.939 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "selftest", 2024-04-26T21:58:56.940 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-26T21:58:56.940 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-26T21:58:56.940 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-26T21:58:56.940 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-26T21:58:56.940 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-26T21:58:56.940 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.940 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.940 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.940 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.940 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.940 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.940 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:56.940 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:56.940 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:56.940 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:56.941 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:56.941 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:56.941 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:56.941 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.941 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.941 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.941 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.941 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.941 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.941 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-26T21:58:56.941 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-26T21:58:56.941 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.941 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.941 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.941 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.942 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.942 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.942 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.942 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.942 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.942 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.942 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.942 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.942 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-26T21:58:56.942 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-26T21:58:56.942 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.942 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.942 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.942 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-26T21:58:56.943 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.943 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.943 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:56.943 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:56.943 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:56.943 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:56.943 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:56.943 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:56.943 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:56.943 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.943 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.943 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.943 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.943 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.944 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.944 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-26T21:58:56.944 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-26T21:58:56.944 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.944 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.944 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.944 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.944 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.944 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.944 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.944 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.944 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.944 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.944 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.944 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.944 INFO:teuthology.orchestra.run.smithi072.stdout: "roption1": { 2024-04-26T21:58:56.945 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "roption1", 2024-04-26T21:58:56.945 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.945 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.945 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.945 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.945 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.945 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.945 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.945 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.945 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.945 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.945 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.945 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.945 INFO:teuthology.orchestra.run.smithi072.stdout: "roption2": { 2024-04-26T21:58:56.945 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "roption2", 2024-04-26T21:58:56.946 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.946 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.946 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.946 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "xyz", 2024-04-26T21:58:56.946 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.946 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.946 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.946 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.946 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.946 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.946 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.946 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.946 INFO:teuthology.orchestra.run.smithi072.stdout: "rwoption1": { 2024-04-26T21:58:56.946 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "rwoption1", 2024-04-26T21:58:56.946 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.947 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.947 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.947 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.947 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.947 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.947 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.947 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.947 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.947 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.947 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.947 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.947 INFO:teuthology.orchestra.run.smithi072.stdout: "rwoption2": { 2024-04-26T21:58:56.947 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "rwoption2", 2024-04-26T21:58:56.947 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-26T21:58:56.947 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.948 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.948 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.948 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.948 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.948 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.948 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.948 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.948 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.948 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.948 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.948 INFO:teuthology.orchestra.run.smithi072.stdout: "rwoption3": { 2024-04-26T21:58:56.948 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "rwoption3", 2024-04-26T21:58:56.948 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "float", 2024-04-26T21:58:56.948 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.948 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.949 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.949 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.949 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.949 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.949 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.949 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.949 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.949 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.949 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.949 INFO:teuthology.orchestra.run.smithi072.stdout: "rwoption4": { 2024-04-26T21:58:56.949 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "rwoption4", 2024-04-26T21:58:56.949 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.949 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.949 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.949 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.950 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.950 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.950 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.950 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.950 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.950 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.950 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.950 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.950 INFO:teuthology.orchestra.run.smithi072.stdout: "rwoption5": { 2024-04-26T21:58:56.950 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "rwoption5", 2024-04-26T21:58:56.950 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.950 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.950 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.950 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.950 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.951 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.951 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.951 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.951 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.951 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.951 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.951 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.951 INFO:teuthology.orchestra.run.smithi072.stdout: "rwoption6": { 2024-04-26T21:58:56.951 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "rwoption6", 2024-04-26T21:58:56.951 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.951 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.951 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.951 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-26T21:58:56.951 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.952 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.952 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.952 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.952 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.952 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.952 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.952 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.952 INFO:teuthology.orchestra.run.smithi072.stdout: "rwoption7": { 2024-04-26T21:58:56.952 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "rwoption7", 2024-04-26T21:58:56.952 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-26T21:58:56.952 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.952 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.952 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.952 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "1", 2024-04-26T21:58:56.952 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "42", 2024-04-26T21:58:56.953 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.953 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.953 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.953 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.953 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.953 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.953 INFO:teuthology.orchestra.run.smithi072.stdout: "testkey": { 2024-04-26T21:58:56.953 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "testkey", 2024-04-26T21:58:56.953 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.953 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.953 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.953 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.953 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.953 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.954 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.954 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.954 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.954 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.954 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.954 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.954 INFO:teuthology.orchestra.run.smithi072.stdout: "testlkey": { 2024-04-26T21:58:56.954 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "testlkey", 2024-04-26T21:58:56.954 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.954 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.954 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.954 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.954 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.954 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.954 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.955 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.955 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.955 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.955 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.955 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.955 INFO:teuthology.orchestra.run.smithi072.stdout: "testnewline": { 2024-04-26T21:58:56.955 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "testnewline", 2024-04-26T21:58:56.955 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.955 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.955 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.955 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.955 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.955 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.955 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.955 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.956 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.956 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.956 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.956 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:56.956 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:56.956 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.956 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-26T21:58:56.956 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "snap_schedule", 2024-04-26T21:58:56.956 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-26T21:58:56.956 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-26T21:58:56.956 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-26T21:58:56.956 INFO:teuthology.orchestra.run.smithi072.stdout: "allow_m_granularity": { 2024-04-26T21:58:56.956 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "allow_m_granularity", 2024-04-26T21:58:56.956 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.956 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.957 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.957 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.957 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.957 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.957 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.957 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "allow minute scheduled snapshots", 2024-04-26T21:58:56.957 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.957 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.957 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.957 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.957 INFO:teuthology.orchestra.run.smithi072.stdout: "dump_on_update": { 2024-04-26T21:58:56.957 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "dump_on_update", 2024-04-26T21:58:56.957 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.957 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.958 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.958 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.958 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.958 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.958 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.958 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "dump database to debug log on update", 2024-04-26T21:58:56.958 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.958 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.958 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.958 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.958 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-26T21:58:56.958 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-26T21:58:56.958 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.958 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.958 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.959 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.959 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.959 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.959 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:56.959 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:56.959 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:56.959 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:56.959 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:56.959 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:56.959 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:56.959 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.959 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.959 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.959 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.959 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.960 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.960 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-26T21:58:56.960 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-26T21:58:56.960 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.960 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.960 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.960 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.960 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.960 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.960 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.960 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.960 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.960 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.960 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.960 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.961 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-26T21:58:56.961 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-26T21:58:56.961 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.961 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.961 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.961 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-26T21:58:56.961 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.961 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.961 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:56.961 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:56.961 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:56.961 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:56.961 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:56.961 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:56.961 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:56.962 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.962 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.962 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.962 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.962 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.962 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.962 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-26T21:58:56.962 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-26T21:58:56.962 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.962 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.962 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.962 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.962 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.962 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.962 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.963 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.963 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.963 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.963 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.963 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:56.963 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:56.963 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.963 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-26T21:58:56.963 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "stats", 2024-04-26T21:58:56.963 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-26T21:58:56.963 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-26T21:58:56.963 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-26T21:58:56.963 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-26T21:58:56.963 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-26T21:58:56.963 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.964 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.964 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.964 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.964 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.964 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.964 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:56.964 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:56.964 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:56.964 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:56.964 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:56.964 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:56.964 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:56.964 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.964 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.964 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.965 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.965 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.965 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.965 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-26T21:58:56.965 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-26T21:58:56.965 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.965 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.965 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.965 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.965 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.965 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.965 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.965 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.965 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.965 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.966 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.966 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.966 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-26T21:58:56.966 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-26T21:58:56.966 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.966 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.966 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.966 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-26T21:58:56.966 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.966 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.966 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:56.966 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:56.966 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:56.966 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:56.966 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:56.967 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:56.967 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:56.967 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.967 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.967 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.967 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.967 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.967 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.967 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-26T21:58:56.967 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-26T21:58:56.967 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.967 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.967 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.967 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.967 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.968 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.968 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.968 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.968 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.968 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.968 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.968 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:56.968 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:56.968 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.968 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-26T21:58:56.968 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "status", 2024-04-26T21:58:56.968 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-26T21:58:56.968 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-26T21:58:56.968 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-26T21:58:56.968 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-26T21:58:56.969 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-26T21:58:56.969 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.969 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.969 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.969 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.969 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.969 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.969 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:56.969 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:56.969 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:56.969 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:56.969 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:56.969 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:56.969 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:56.969 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.970 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.970 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.970 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.970 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.970 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.970 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-26T21:58:56.970 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-26T21:58:56.970 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.970 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.970 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.970 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.970 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.970 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.970 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.971 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.971 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.971 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.971 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.971 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.971 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-26T21:58:56.971 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-26T21:58:56.971 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.971 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.971 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.971 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-26T21:58:56.971 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.971 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.972 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:56.972 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:56.972 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:56.972 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:56.972 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:56.972 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:56.972 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:56.972 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.972 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.972 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.972 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.972 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.972 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.972 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-26T21:58:56.972 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-26T21:58:56.972 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.973 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.973 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.973 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.973 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.973 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.973 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.973 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.973 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.973 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.973 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.973 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:56.973 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:56.973 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.973 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-26T21:58:56.973 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "telegraf", 2024-04-26T21:58:56.973 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-26T21:58:56.974 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-26T21:58:56.974 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-26T21:58:56.974 INFO:teuthology.orchestra.run.smithi072.stdout: "address": { 2024-04-26T21:58:56.974 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "address", 2024-04-26T21:58:56.974 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.974 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.974 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.974 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-26T21:58:56.974 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.974 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.974 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.974 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.974 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.974 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.975 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.975 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.975 INFO:teuthology.orchestra.run.smithi072.stdout: "interval": { 2024-04-26T21:58:56.975 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "interval", 2024-04-26T21:58:56.975 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "secs", 2024-04-26T21:58:56.975 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.975 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.975 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "15", 2024-04-26T21:58:56.975 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.975 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.975 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.975 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.975 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.975 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.975 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.975 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.976 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-26T21:58:56.976 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-26T21:58:56.976 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.976 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.976 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.976 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.976 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.976 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.976 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:56.976 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:56.976 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:56.976 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:56.976 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:56.976 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:56.976 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:56.976 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.976 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.977 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.977 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.977 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.977 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.977 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-26T21:58:56.977 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-26T21:58:56.977 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.977 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.977 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.977 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.977 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.977 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.977 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.977 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.977 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.978 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.978 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.978 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.978 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-26T21:58:56.978 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-26T21:58:56.978 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.978 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.978 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.978 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-26T21:58:56.978 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.978 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.978 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:56.978 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:56.978 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:56.978 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:56.979 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:56.979 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:56.979 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:56.979 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.979 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.979 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.979 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.979 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.979 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.979 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-26T21:58:56.979 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-26T21:58:56.979 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.979 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.979 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.979 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.979 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.980 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.980 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.980 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.980 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.980 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.980 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.980 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:56.980 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:56.980 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.980 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-26T21:58:56.980 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "telemetry", 2024-04-26T21:58:56.980 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-26T21:58:56.980 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-26T21:58:56.980 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-26T21:58:56.980 INFO:teuthology.orchestra.run.smithi072.stdout: "channel_basic": { 2024-04-26T21:58:56.981 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "channel_basic", 2024-04-26T21:58:56.981 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.981 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.981 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.981 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-26T21:58:56.981 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.981 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.981 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.981 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-26T21:58:56.981 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.981 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.981 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.981 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.981 INFO:teuthology.orchestra.run.smithi072.stdout: "channel_crash": { 2024-04-26T21:58:56.981 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "channel_crash", 2024-04-26T21:58:56.982 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.982 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.982 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.982 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-26T21:58:56.982 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.982 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.982 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.982 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-26T21:58:56.982 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.982 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.982 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.982 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.982 INFO:teuthology.orchestra.run.smithi072.stdout: "channel_device": { 2024-04-26T21:58:56.982 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "channel_device", 2024-04-26T21:58:56.983 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.983 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.983 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.983 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-26T21:58:56.983 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.983 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.983 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.983 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-26T21:58:56.983 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.983 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.983 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.983 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.983 INFO:teuthology.orchestra.run.smithi072.stdout: "channel_ident": { 2024-04-26T21:58:56.983 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "channel_ident", 2024-04-26T21:58:56.984 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.984 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.984 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.984 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.984 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.984 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.984 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.984 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-26T21:58:56.984 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.984 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.984 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.984 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.984 INFO:teuthology.orchestra.run.smithi072.stdout: "channel_perf": { 2024-04-26T21:58:56.984 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "channel_perf", 2024-04-26T21:58:56.985 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.985 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.985 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.985 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.985 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.985 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.985 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.985 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-26T21:58:56.985 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.985 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.985 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.985 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.985 INFO:teuthology.orchestra.run.smithi072.stdout: "contact": { 2024-04-26T21:58:56.985 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "contact", 2024-04-26T21:58:56.985 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.986 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.986 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.986 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.986 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.986 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.986 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.986 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.986 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.986 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.986 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.986 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.986 INFO:teuthology.orchestra.run.smithi072.stdout: "description": { 2024-04-26T21:58:56.986 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "description", 2024-04-26T21:58:56.986 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.986 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.987 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.987 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.987 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.987 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.987 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.987 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.987 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.987 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.987 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.987 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.987 INFO:teuthology.orchestra.run.smithi072.stdout: "device_url": { 2024-04-26T21:58:56.987 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "device_url", 2024-04-26T21:58:56.987 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.987 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.987 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.988 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-26T21:58:56.988 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.988 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.988 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.988 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.988 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.988 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.988 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.988 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.988 INFO:teuthology.orchestra.run.smithi072.stdout: "enabled": { 2024-04-26T21:58:56.988 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "enabled", 2024-04-26T21:58:56.988 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.988 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.988 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.988 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.988 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.989 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.989 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.989 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.989 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.989 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.989 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.989 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.989 INFO:teuthology.orchestra.run.smithi072.stdout: "interval": { 2024-04-26T21:58:56.989 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "interval", 2024-04-26T21:58:56.989 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-26T21:58:56.989 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.989 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.989 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "24", 2024-04-26T21:58:56.989 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "8", 2024-04-26T21:58:56.989 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.990 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.990 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.990 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.990 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.990 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.990 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.990 INFO:teuthology.orchestra.run.smithi072.stdout: "last_opt_revision": { 2024-04-26T21:58:56.990 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "last_opt_revision", 2024-04-26T21:58:56.990 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-26T21:58:56.990 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.990 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.990 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "1", 2024-04-26T21:58:56.990 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.990 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.990 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.991 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.991 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.991 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.991 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.991 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.991 INFO:teuthology.orchestra.run.smithi072.stdout: "leaderboard": { 2024-04-26T21:58:56.991 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "leaderboard", 2024-04-26T21:58:56.991 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.991 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.991 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.991 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.991 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.991 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.991 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.991 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.991 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.992 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.992 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.992 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.992 INFO:teuthology.orchestra.run.smithi072.stdout: "leaderboard_description": { 2024-04-26T21:58:56.992 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "leaderboard_description", 2024-04-26T21:58:56.992 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.992 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.992 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.992 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.992 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.992 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.992 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.992 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.992 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.992 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.992 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.993 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.993 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-26T21:58:56.993 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-26T21:58:56.993 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.993 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.993 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.993 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.993 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.993 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.993 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:56.993 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:56.993 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:56.993 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:56.993 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:56.993 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:56.994 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:56.994 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.994 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.994 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.994 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.994 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.994 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.994 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-26T21:58:56.994 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-26T21:58:56.994 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.994 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.994 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.994 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.994 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.995 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.995 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.995 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.995 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.995 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.995 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.995 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.995 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-26T21:58:56.995 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-26T21:58:56.995 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.995 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.995 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.995 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-26T21:58:56.995 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.995 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.996 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:56.996 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:56.996 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:56.996 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:56.996 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:56.996 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:56.996 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:56.996 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:56.996 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.996 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.996 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.996 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.996 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.996 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-26T21:58:56.996 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-26T21:58:56.996 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:56.997 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.997 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:56.997 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:56.997 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.997 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.997 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.997 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.997 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.997 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.997 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.997 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.997 INFO:teuthology.orchestra.run.smithi072.stdout: "organization": { 2024-04-26T21:58:56.997 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "organization", 2024-04-26T21:58:56.997 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.997 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.998 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.998 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.998 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.998 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.998 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.998 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.998 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.998 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.998 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.998 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.998 INFO:teuthology.orchestra.run.smithi072.stdout: "proxy": { 2024-04-26T21:58:56.998 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "proxy", 2024-04-26T21:58:56.998 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.998 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.999 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.999 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:56.999 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:56.999 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:56.999 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:56.999 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:56.999 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:56.999 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:56.999 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:56.999 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:56.999 INFO:teuthology.orchestra.run.smithi072.stdout: "url": { 2024-04-26T21:58:56.999 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "url", 2024-04-26T21:58:56.999 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:56.999 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:56.999 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:56.999 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-26T21:58:57.000 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:57.000 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:57.000 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:57.000 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:57.000 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:57.000 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:57.000 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:57.000 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:57.000 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:57.000 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:57.000 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-26T21:58:57.000 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "test_orchestrator", 2024-04-26T21:58:57.000 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-26T21:58:57.000 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-26T21:58:57.000 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-26T21:58:57.000 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-26T21:58:57.001 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-26T21:58:57.001 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:57.001 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:57.001 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:57.001 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:57.001 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:57.001 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:57.001 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:57.001 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:57.001 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:57.001 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:57.001 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:57.001 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:57.001 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:57.001 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:57.001 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:57.002 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:57.002 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:57.002 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:57.002 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:57.002 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-26T21:58:57.002 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-26T21:58:57.002 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:57.002 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:57.002 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:57.002 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:57.002 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:57.002 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:57.002 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:57.002 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:57.003 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:57.003 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:57.003 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:57.003 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:57.003 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-26T21:58:57.003 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-26T21:58:57.003 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:57.003 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:57.003 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:57.003 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-26T21:58:57.003 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:57.003 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:57.003 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:57.003 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:57.003 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:57.003 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:57.003 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:57.004 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:57.004 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:57.004 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:57.004 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:57.004 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:57.004 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:57.004 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:57.004 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:57.004 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-26T21:58:57.004 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-26T21:58:57.004 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:57.004 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:57.004 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:57.004 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:57.004 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:57.005 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:57.005 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:57.005 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:57.005 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:57.005 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:57.005 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:57.005 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:57.005 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:57.005 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:57.005 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-26T21:58:57.005 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "volumes", 2024-04-26T21:58:57.005 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-26T21:58:57.005 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-26T21:58:57.005 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-26T21:58:57.005 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-26T21:58:57.006 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-26T21:58:57.006 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:57.006 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:57.006 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:57.006 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:57.006 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:57.006 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:57.006 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:57.006 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:57.006 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:57.006 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:57.006 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:57.006 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:57.006 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:57.006 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:57.007 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:57.007 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:57.007 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:57.007 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:57.007 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:57.007 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-26T21:58:57.007 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-26T21:58:57.007 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:57.007 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:57.007 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:57.007 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:57.007 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:57.007 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:57.007 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:57.008 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:57.008 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:57.008 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:57.008 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:57.008 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:57.008 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-26T21:58:57.008 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-26T21:58:57.008 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:57.008 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:57.008 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:57.008 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-26T21:58:57.008 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:57.008 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:57.008 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:57.008 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:57.008 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:57.008 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:57.009 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:57.009 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:57.009 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:57.009 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:57.009 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:57.009 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:57.009 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:57.009 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:57.009 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:57.009 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-26T21:58:57.009 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-26T21:58:57.009 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:57.009 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:57.009 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:57.009 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:57.010 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:57.010 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:57.010 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:57.010 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:57.010 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:57.010 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:57.010 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:57.010 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:57.010 INFO:teuthology.orchestra.run.smithi072.stdout: "max_concurrent_clones": { 2024-04-26T21:58:57.010 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "max_concurrent_clones", 2024-04-26T21:58:57.010 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-26T21:58:57.010 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:57.010 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:57.010 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "4", 2024-04-26T21:58:57.010 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:57.011 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:57.011 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:57.011 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-26T21:58:57.011 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:57.011 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:57.011 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:57.011 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:57.011 INFO:teuthology.orchestra.run.smithi072.stdout: "periodic_async_work": { 2024-04-26T21:58:57.011 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "periodic_async_work", 2024-04-26T21:58:57.011 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:57.011 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:57.011 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:57.011 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:57.011 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:57.011 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:57.011 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:57.012 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Periodically check for async work", 2024-04-26T21:58:57.012 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:57.012 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:57.012 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:57.012 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:57.012 INFO:teuthology.orchestra.run.smithi072.stdout: "snapshot_clone_delay": { 2024-04-26T21:58:57.012 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "snapshot_clone_delay", 2024-04-26T21:58:57.012 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-26T21:58:57.012 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:57.012 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:57.012 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "0", 2024-04-26T21:58:57.012 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:57.012 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:57.012 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:57.012 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-26T21:58:57.013 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:57.013 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:57.013 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:57.013 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:57.013 INFO:teuthology.orchestra.run.smithi072.stdout: "snapshot_clone_no_wait": { 2024-04-26T21:58:57.013 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "snapshot_clone_no_wait", 2024-04-26T21:58:57.013 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:57.013 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:57.013 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:57.013 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "True", 2024-04-26T21:58:57.013 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:57.013 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:57.013 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:57.013 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-26T21:58:57.013 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:57.013 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:57.014 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:57.014 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:57.014 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:57.014 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:57.014 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-26T21:58:57.014 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "zabbix", 2024-04-26T21:58:57.014 INFO:teuthology.orchestra.run.smithi072.stdout: "can_run": true, 2024-04-26T21:58:57.014 INFO:teuthology.orchestra.run.smithi072.stdout: "error_string": "", 2024-04-26T21:58:57.014 INFO:teuthology.orchestra.run.smithi072.stdout: "module_options": { 2024-04-26T21:58:57.014 INFO:teuthology.orchestra.run.smithi072.stdout: "discovery_interval": { 2024-04-26T21:58:57.014 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "discovery_interval", 2024-04-26T21:58:57.014 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "uint", 2024-04-26T21:58:57.014 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:57.014 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:57.015 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "100", 2024-04-26T21:58:57.015 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:57.015 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:57.015 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:57.015 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:57.015 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:57.015 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:57.015 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:57.015 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:57.015 INFO:teuthology.orchestra.run.smithi072.stdout: "identifier": { 2024-04-26T21:58:57.015 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "identifier", 2024-04-26T21:58:57.015 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:57.015 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:57.015 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:57.015 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:57.016 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:57.016 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:57.016 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:57.016 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:57.016 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:57.016 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:57.016 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:57.016 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:57.016 INFO:teuthology.orchestra.run.smithi072.stdout: "interval": { 2024-04-26T21:58:57.016 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "interval", 2024-04-26T21:58:57.016 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "secs", 2024-04-26T21:58:57.016 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:57.016 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:57.016 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "60", 2024-04-26T21:58:57.016 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:57.016 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:57.017 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:57.017 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:57.017 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:57.017 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:57.017 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:57.017 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:57.017 INFO:teuthology.orchestra.run.smithi072.stdout: "log_level": { 2024-04-26T21:58:57.017 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_level", 2024-04-26T21:58:57.017 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:57.017 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:57.017 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:57.017 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:57.017 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:57.017 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:57.017 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:57.017 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:57.018 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:57.018 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:57.018 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:57.018 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:57.018 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:57.018 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:57.018 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:57.018 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:57.018 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:57.018 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:57.018 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:57.018 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster": { 2024-04-26T21:58:57.018 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster", 2024-04-26T21:58:57.018 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:57.019 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:57.019 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:57.019 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:57.019 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:57.019 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:57.019 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:57.019 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:57.019 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:57.019 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:57.019 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:57.019 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:57.019 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_cluster_level": { 2024-04-26T21:58:57.019 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_cluster_level", 2024-04-26T21:58:57.019 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:57.019 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:57.020 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:57.020 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "info", 2024-04-26T21:58:57.020 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:57.020 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:57.020 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [ 2024-04-26T21:58:57.020 INFO:teuthology.orchestra.run.smithi072.stdout: "", 2024-04-26T21:58:57.020 INFO:teuthology.orchestra.run.smithi072.stdout: "critical", 2024-04-26T21:58:57.020 INFO:teuthology.orchestra.run.smithi072.stdout: "debug", 2024-04-26T21:58:57.020 INFO:teuthology.orchestra.run.smithi072.stdout: "error", 2024-04-26T21:58:57.020 INFO:teuthology.orchestra.run.smithi072.stdout: "info", 2024-04-26T21:58:57.020 INFO:teuthology.orchestra.run.smithi072.stdout: "warning" 2024-04-26T21:58:57.020 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:57.020 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:57.020 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:57.020 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:57.021 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:57.021 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:57.021 INFO:teuthology.orchestra.run.smithi072.stdout: "log_to_file": { 2024-04-26T21:58:57.021 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "log_to_file", 2024-04-26T21:58:57.021 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "bool", 2024-04-26T21:58:57.021 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:57.021 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 1, 2024-04-26T21:58:57.021 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "False", 2024-04-26T21:58:57.021 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:57.021 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:57.021 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:57.021 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:57.021 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:57.021 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:57.021 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:57.021 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:57.022 INFO:teuthology.orchestra.run.smithi072.stdout: "zabbix_host": { 2024-04-26T21:58:57.022 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "zabbix_host", 2024-04-26T21:58:57.022 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:57.022 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:57.022 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:57.022 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "", 2024-04-26T21:58:57.022 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:57.022 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:57.022 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:57.022 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:57.022 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:57.022 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:57.022 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:57.022 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:57.022 INFO:teuthology.orchestra.run.smithi072.stdout: "zabbix_port": { 2024-04-26T21:58:57.023 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "zabbix_port", 2024-04-26T21:58:57.023 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "int", 2024-04-26T21:58:57.023 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:57.023 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:57.023 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "10051", 2024-04-26T21:58:57.023 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:57.023 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:57.023 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:57.023 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:57.023 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:57.023 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:57.023 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:57.023 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:57.023 INFO:teuthology.orchestra.run.smithi072.stdout: "zabbix_sender": { 2024-04-26T21:58:57.023 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "zabbix_sender", 2024-04-26T21:58:57.024 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "str", 2024-04-26T21:58:57.024 INFO:teuthology.orchestra.run.smithi072.stdout: "level": "advanced", 2024-04-26T21:58:57.024 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": 0, 2024-04-26T21:58:57.024 INFO:teuthology.orchestra.run.smithi072.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-26T21:58:57.024 INFO:teuthology.orchestra.run.smithi072.stdout: "min": "", 2024-04-26T21:58:57.024 INFO:teuthology.orchestra.run.smithi072.stdout: "max": "", 2024-04-26T21:58:57.024 INFO:teuthology.orchestra.run.smithi072.stdout: "enum_allowed": [], 2024-04-26T21:58:57.024 INFO:teuthology.orchestra.run.smithi072.stdout: "desc": "", 2024-04-26T21:58:57.024 INFO:teuthology.orchestra.run.smithi072.stdout: "long_desc": "", 2024-04-26T21:58:57.024 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2024-04-26T21:58:57.024 INFO:teuthology.orchestra.run.smithi072.stdout: "see_also": [] 2024-04-26T21:58:57.024 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:57.024 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:57.024 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:57.024 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:57.024 INFO:teuthology.orchestra.run.smithi072.stdout: "services": { 2024-04-26T21:58:57.025 INFO:teuthology.orchestra.run.smithi072.stdout: "dashboard": "https://172.21.15.72:8443/" 2024-04-26T21:58:57.025 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:57.025 INFO:teuthology.orchestra.run.smithi072.stdout: "always_on_modules": { 2024-04-26T21:58:57.025 INFO:teuthology.orchestra.run.smithi072.stdout: "octopus": [ 2024-04-26T21:58:57.025 INFO:teuthology.orchestra.run.smithi072.stdout: "balancer", 2024-04-26T21:58:57.025 INFO:teuthology.orchestra.run.smithi072.stdout: "crash", 2024-04-26T21:58:57.025 INFO:teuthology.orchestra.run.smithi072.stdout: "devicehealth", 2024-04-26T21:58:57.025 INFO:teuthology.orchestra.run.smithi072.stdout: "orchestrator", 2024-04-26T21:58:57.025 INFO:teuthology.orchestra.run.smithi072.stdout: "pg_autoscaler", 2024-04-26T21:58:57.025 INFO:teuthology.orchestra.run.smithi072.stdout: "progress", 2024-04-26T21:58:57.025 INFO:teuthology.orchestra.run.smithi072.stdout: "rbd_support", 2024-04-26T21:58:57.025 INFO:teuthology.orchestra.run.smithi072.stdout: "status", 2024-04-26T21:58:57.025 INFO:teuthology.orchestra.run.smithi072.stdout: "telemetry", 2024-04-26T21:58:57.025 INFO:teuthology.orchestra.run.smithi072.stdout: "volumes" 2024-04-26T21:58:57.025 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:57.025 INFO:teuthology.orchestra.run.smithi072.stdout: "pacific": [ 2024-04-26T21:58:57.026 INFO:teuthology.orchestra.run.smithi072.stdout: "balancer", 2024-04-26T21:58:57.026 INFO:teuthology.orchestra.run.smithi072.stdout: "crash", 2024-04-26T21:58:57.026 INFO:teuthology.orchestra.run.smithi072.stdout: "devicehealth", 2024-04-26T21:58:57.026 INFO:teuthology.orchestra.run.smithi072.stdout: "orchestrator", 2024-04-26T21:58:57.026 INFO:teuthology.orchestra.run.smithi072.stdout: "pg_autoscaler", 2024-04-26T21:58:57.026 INFO:teuthology.orchestra.run.smithi072.stdout: "progress", 2024-04-26T21:58:57.026 INFO:teuthology.orchestra.run.smithi072.stdout: "rbd_support", 2024-04-26T21:58:57.026 INFO:teuthology.orchestra.run.smithi072.stdout: "status", 2024-04-26T21:58:57.026 INFO:teuthology.orchestra.run.smithi072.stdout: "telemetry", 2024-04-26T21:58:57.026 INFO:teuthology.orchestra.run.smithi072.stdout: "volumes" 2024-04-26T21:58:57.026 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:57.026 INFO:teuthology.orchestra.run.smithi072.stdout: "quincy": [ 2024-04-26T21:58:57.026 INFO:teuthology.orchestra.run.smithi072.stdout: "balancer", 2024-04-26T21:58:57.026 INFO:teuthology.orchestra.run.smithi072.stdout: "crash", 2024-04-26T21:58:57.026 INFO:teuthology.orchestra.run.smithi072.stdout: "devicehealth", 2024-04-26T21:58:57.027 INFO:teuthology.orchestra.run.smithi072.stdout: "orchestrator", 2024-04-26T21:58:57.027 INFO:teuthology.orchestra.run.smithi072.stdout: "pg_autoscaler", 2024-04-26T21:58:57.027 INFO:teuthology.orchestra.run.smithi072.stdout: "progress", 2024-04-26T21:58:57.027 INFO:teuthology.orchestra.run.smithi072.stdout: "rbd_support", 2024-04-26T21:58:57.027 INFO:teuthology.orchestra.run.smithi072.stdout: "status", 2024-04-26T21:58:57.027 INFO:teuthology.orchestra.run.smithi072.stdout: "telemetry", 2024-04-26T21:58:57.027 INFO:teuthology.orchestra.run.smithi072.stdout: "volumes" 2024-04-26T21:58:57.027 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:57.027 INFO:teuthology.orchestra.run.smithi072.stdout: "reef": [ 2024-04-26T21:58:57.027 INFO:teuthology.orchestra.run.smithi072.stdout: "balancer", 2024-04-26T21:58:57.027 INFO:teuthology.orchestra.run.smithi072.stdout: "crash", 2024-04-26T21:58:57.027 INFO:teuthology.orchestra.run.smithi072.stdout: "devicehealth", 2024-04-26T21:58:57.027 INFO:teuthology.orchestra.run.smithi072.stdout: "orchestrator", 2024-04-26T21:58:57.027 INFO:teuthology.orchestra.run.smithi072.stdout: "pg_autoscaler", 2024-04-26T21:58:57.027 INFO:teuthology.orchestra.run.smithi072.stdout: "progress", 2024-04-26T21:58:57.027 INFO:teuthology.orchestra.run.smithi072.stdout: "rbd_support", 2024-04-26T21:58:57.028 INFO:teuthology.orchestra.run.smithi072.stdout: "status", 2024-04-26T21:58:57.028 INFO:teuthology.orchestra.run.smithi072.stdout: "telemetry", 2024-04-26T21:58:57.028 INFO:teuthology.orchestra.run.smithi072.stdout: "volumes" 2024-04-26T21:58:57.028 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T21:58:57.028 INFO:teuthology.orchestra.run.smithi072.stdout: "squid": [ 2024-04-26T21:58:57.028 INFO:teuthology.orchestra.run.smithi072.stdout: "balancer", 2024-04-26T21:58:57.028 INFO:teuthology.orchestra.run.smithi072.stdout: "crash", 2024-04-26T21:58:57.028 INFO:teuthology.orchestra.run.smithi072.stdout: "devicehealth", 2024-04-26T21:58:57.028 INFO:teuthology.orchestra.run.smithi072.stdout: "orchestrator", 2024-04-26T21:58:57.028 INFO:teuthology.orchestra.run.smithi072.stdout: "pg_autoscaler", 2024-04-26T21:58:57.028 INFO:teuthology.orchestra.run.smithi072.stdout: "progress", 2024-04-26T21:58:57.028 INFO:teuthology.orchestra.run.smithi072.stdout: "rbd_support", 2024-04-26T21:58:57.028 INFO:teuthology.orchestra.run.smithi072.stdout: "status", 2024-04-26T21:58:57.028 INFO:teuthology.orchestra.run.smithi072.stdout: "telemetry", 2024-04-26T21:58:57.028 INFO:teuthology.orchestra.run.smithi072.stdout: "volumes" 2024-04-26T21:58:57.029 INFO:teuthology.orchestra.run.smithi072.stdout: ] 2024-04-26T21:58:57.029 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:57.029 INFO:teuthology.orchestra.run.smithi072.stdout: "last_failure_osd_epoch": 3, 2024-04-26T21:58:57.029 INFO:teuthology.orchestra.run.smithi072.stdout: "active_clients": [ 2024-04-26T21:58:57.029 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-26T21:58:57.029 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "devicehealth", 2024-04-26T21:58:57.029 INFO:teuthology.orchestra.run.smithi072.stdout: "addrvec": [ 2024-04-26T21:58:57.029 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-26T21:58:57.029 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "v2", 2024-04-26T21:58:57.029 INFO:teuthology.orchestra.run.smithi072.stdout: "addr": "172.21.15.72:0", 2024-04-26T21:58:57.029 INFO:teuthology.orchestra.run.smithi072.stdout: "nonce": 834906530 2024-04-26T21:58:57.029 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:57.029 INFO:teuthology.orchestra.run.smithi072.stdout: ] 2024-04-26T21:58:57.029 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:57.029 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-26T21:58:57.029 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "libcephsqlite", 2024-04-26T21:58:57.030 INFO:teuthology.orchestra.run.smithi072.stdout: "addrvec": [ 2024-04-26T21:58:57.030 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-26T21:58:57.030 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "v2", 2024-04-26T21:58:57.030 INFO:teuthology.orchestra.run.smithi072.stdout: "addr": "172.21.15.72:0", 2024-04-26T21:58:57.030 INFO:teuthology.orchestra.run.smithi072.stdout: "nonce": 355605634 2024-04-26T21:58:57.030 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:57.030 INFO:teuthology.orchestra.run.smithi072.stdout: ] 2024-04-26T21:58:57.030 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:57.030 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-26T21:58:57.030 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "rbd_support", 2024-04-26T21:58:57.030 INFO:teuthology.orchestra.run.smithi072.stdout: "addrvec": [ 2024-04-26T21:58:57.030 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-26T21:58:57.030 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "v2", 2024-04-26T21:58:57.030 INFO:teuthology.orchestra.run.smithi072.stdout: "addr": "172.21.15.72:0", 2024-04-26T21:58:57.030 INFO:teuthology.orchestra.run.smithi072.stdout: "nonce": 2799657309 2024-04-26T21:58:57.031 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:57.031 INFO:teuthology.orchestra.run.smithi072.stdout: ] 2024-04-26T21:58:57.031 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:57.031 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-26T21:58:57.031 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "volumes", 2024-04-26T21:58:57.031 INFO:teuthology.orchestra.run.smithi072.stdout: "addrvec": [ 2024-04-26T21:58:57.031 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-26T21:58:57.031 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "v2", 2024-04-26T21:58:57.031 INFO:teuthology.orchestra.run.smithi072.stdout: "addr": "172.21.15.72:0", 2024-04-26T21:58:57.031 INFO:teuthology.orchestra.run.smithi072.stdout: "nonce": 4234165608 2024-04-26T21:58:57.031 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:57.031 INFO:teuthology.orchestra.run.smithi072.stdout: ] 2024-04-26T21:58:57.031 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:57.031 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-26T21:58:57.031 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "volumes", 2024-04-26T21:58:57.031 INFO:teuthology.orchestra.run.smithi072.stdout: "addrvec": [ 2024-04-26T21:58:57.032 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-26T21:58:57.032 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "v2", 2024-04-26T21:58:57.032 INFO:teuthology.orchestra.run.smithi072.stdout: "addr": "172.21.15.72:0", 2024-04-26T21:58:57.032 INFO:teuthology.orchestra.run.smithi072.stdout: "nonce": 2253420320 2024-04-26T21:58:57.032 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:57.032 INFO:teuthology.orchestra.run.smithi072.stdout: ] 2024-04-26T21:58:57.032 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:57.032 INFO:teuthology.orchestra.run.smithi072.stdout: ] 2024-04-26T21:58:57.032 INFO:teuthology.orchestra.run.smithi072.stdout:} 2024-04-26T21:58:57.033 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:58:56 smithi164 ceph-mon[31916]: from='client.? 172.21.15.72:0/3994328810' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T21:58:57.033 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:58:56 smithi164 ceph-mon[31916]: from='client.? 172.21.15.72:0/2492863644' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-26T21:58:57.034 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:58:56 smithi076 ceph-mon[31823]: from='client.? 172.21.15.72:0/3994328810' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T21:58:57.034 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:58:56 smithi076 ceph-mon[31823]: from='client.? 172.21.15.72:0/2492863644' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-26T21:58:57.034 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:55.782+0000 7f3267200640 1 -- 172.21.15.72:0/2492863644 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f324c0a3c40 msgr2=0x7f324c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:57.034 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:55.782+0000 7f3267200640 1 --2- 172.21.15.72:0/2492863644 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f324c0a3c40 0x7f324c0a6100 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f32580057c0 tx=0x7f3258000bc0 comp rx=0 tx=0).stop 2024-04-26T21:58:57.035 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:55.782+0000 7f3267200640 1 -- 172.21.15.72:0/2492863644 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3268106580 msgr2=0x7f32681a2180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:57.035 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:55.782+0000 7f3267200640 1 --2- 172.21.15.72:0/2492863644 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3268106580 0x7f32681a2180 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f326000cad0 tx=0x7f326000cf90 comp rx=0 tx=0).stop 2024-04-26T21:58:57.035 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:55.783+0000 7f3267200640 1 -- 172.21.15.72:0/2492863644 shutdown_connections 2024-04-26T21:58:57.035 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:55.783+0000 7f3267200640 1 --2- 172.21.15.72:0/2492863644 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3268106580 0x7f32681a2180 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:57.035 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:55.783+0000 7f3267200640 1 --2- 172.21.15.72:0/2492863644 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f324c0a3c40 0x7f324c0a6100 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:57.035 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:55.783+0000 7f3267200640 1 --2- 172.21.15.72:0/2492863644 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3268105bc0 0x7f326819b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:57.035 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:55.783+0000 7f3267200640 1 --2- 172.21.15.72:0/2492863644 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f32681049d0 0x7f326819abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:57.035 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:55.783+0000 7f3267200640 1 -- 172.21.15.72:0/2492863644 >> 172.21.15.72:0/2492863644 conn(0x7f32680fff40 msgr2=0x7f3268108f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:58:57.035 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:55.783+0000 7f3267200640 1 -- 172.21.15.72:0/2492863644 shutdown_connections 2024-04-26T21:58:57.035 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:58:55.783+0000 7f3267200640 1 -- 172.21.15.72:0/2492863644 wait complete. 2024-04-26T21:58:57.041 DEBUG:tasks.check_counter:Getting stats from a 2024-04-26T21:58:57.041 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid a1780950-0415-11ef-bc93-c7b262605968 -- ceph daemon mds.a perf dump 2024-04-26T21:58:57.041 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:58:56 smithi072 ceph-mon[28138]: from='client.? 172.21.15.72:0/3994328810' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T21:58:57.041 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:58:56 smithi072 ceph-mon[28138]: from='client.? 172.21.15.72:0/2492863644' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-26T21:58:57.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:58:57 smithi076 ceph-mon[31823]: pgmap v554: 97 pgs: 97 active+clean; 193 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.0 MiB/s wr, 1.30k op/s 2024-04-26T21:58:57.321 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:58:57 smithi072 ceph-mon[28138]: pgmap v554: 97 pgs: 97 active+clean; 193 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.0 MiB/s wr, 1.30k op/s 2024-04-26T21:58:57.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:58:57 smithi164 ceph-mon[31916]: pgmap v554: 97 pgs: 97 active+clean; 193 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.0 MiB/s wr, 1.30k op/s 2024-04-26T21:58:57.458 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/a1780950-0415-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:58:58.321 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:58:58 smithi072 ceph-mon[28138]: pgmap v555: 97 pgs: 97 active+clean; 137 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 3.6 MiB/s wr, 822 op/s 2024-04-26T21:58:58.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:58:58 smithi164 ceph-mon[31916]: pgmap v555: 97 pgs: 97 active+clean; 137 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 3.6 MiB/s wr, 822 op/s 2024-04-26T21:58:58.564 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:58:58 smithi076 ceph-mon[31823]: pgmap v555: 97 pgs: 97 active+clean; 137 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 3.6 MiB/s wr, 822 op/s 2024-04-26T21:58:59.896 INFO:teuthology.orchestra.run.smithi072.stdout:{ 2024-04-26T21:58:59.896 INFO:teuthology.orchestra.run.smithi072.stdout: "AsyncMessenger::Worker-0": { 2024-04-26T21:58:59.896 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_recv_messages": 482, 2024-04-26T21:58:59.896 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_messages": 477, 2024-04-26T21:58:59.897 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_recv_bytes": 458592, 2024-04-26T21:58:59.897 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_bytes": 180190, 2024-04-26T21:58:59.897 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_created_connections": 5, 2024-04-26T21:58:59.897 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_active_connections": 1, 2024-04-26T21:58:59.897 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_total_time": 0.105729810, 2024-04-26T21:58:59.897 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_send_time": 0.039917517, 2024-04-26T21:58:59.897 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_recv_time": 0.035680321, 2024-04-26T21:58:59.897 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_fast_dispatch_time": 0.022107929, 2024-04-26T21:58:59.897 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T21:58:59.897 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 477, 2024-04-26T21:58:59.897 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.041760043, 2024-04-26T21:58:59.897 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000087547 2024-04-26T21:58:59.897 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:59.898 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_handle_ack_lat": { 2024-04-26T21:58:59.898 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-04-26T21:58:59.898 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-04-26T21:58:59.898 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-04-26T21:58:59.898 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:59.898 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_recv_encrypted_bytes": 337296, 2024-04-26T21:58:59.898 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_encrypted_bytes": 65392 2024-04-26T21:58:59.898 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:59.898 INFO:teuthology.orchestra.run.smithi072.stdout: "AsyncMessenger::Worker-1": { 2024-04-26T21:58:59.898 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_recv_messages": 223, 2024-04-26T21:58:59.898 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_messages": 301, 2024-04-26T21:58:59.898 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_recv_bytes": 55424, 2024-04-26T21:58:59.898 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_bytes": 288568, 2024-04-26T21:58:59.899 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_created_connections": 5, 2024-04-26T21:58:59.899 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_active_connections": 4, 2024-04-26T21:58:59.899 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_total_time": 0.053091167, 2024-04-26T21:58:59.899 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_send_time": 0.023157055, 2024-04-26T21:58:59.899 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_recv_time": 0.012397717, 2024-04-26T21:58:59.899 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_fast_dispatch_time": 0.012353434, 2024-04-26T21:58:59.899 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T21:58:59.899 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 299, 2024-04-26T21:58:59.899 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.023387685, 2024-04-26T21:58:59.899 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000078219 2024-04-26T21:58:59.899 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:59.899 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_handle_ack_lat": { 2024-04-26T21:58:59.899 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-04-26T21:58:59.899 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-04-26T21:58:59.900 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-04-26T21:58:59.900 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:59.900 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_recv_encrypted_bytes": 160, 2024-04-26T21:58:59.900 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_encrypted_bytes": 217904 2024-04-26T21:58:59.900 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:59.900 INFO:teuthology.orchestra.run.smithi072.stdout: "AsyncMessenger::Worker-2": { 2024-04-26T21:58:59.900 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_recv_messages": 365, 2024-04-26T21:58:59.900 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_messages": 367, 2024-04-26T21:58:59.900 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_recv_bytes": 97095, 2024-04-26T21:58:59.900 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_bytes": 116071, 2024-04-26T21:58:59.900 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_created_connections": 4, 2024-04-26T21:58:59.900 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_active_connections": 3, 2024-04-26T21:58:59.900 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_total_time": 0.062040336, 2024-04-26T21:58:59.900 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_send_time": 0.022445460, 2024-04-26T21:58:59.901 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_recv_time": 0.016806697, 2024-04-26T21:58:59.901 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_fast_dispatch_time": 0.015924966, 2024-04-26T21:58:59.901 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T21:58:59.901 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 365, 2024-04-26T21:58:59.901 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.028283259, 2024-04-26T21:58:59.901 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000077488 2024-04-26T21:58:59.901 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:59.901 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_handle_ack_lat": { 2024-04-26T21:58:59.901 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-04-26T21:58:59.901 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-04-26T21:58:59.901 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-04-26T21:58:59.901 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:59.901 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-26T21:58:59.902 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_encrypted_bytes": 0 2024-04-26T21:58:59.902 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:59.902 INFO:teuthology.orchestra.run.smithi072.stdout: "cct": { 2024-04-26T21:58:59.902 INFO:teuthology.orchestra.run.smithi072.stdout: "total_workers": 1, 2024-04-26T21:58:59.902 INFO:teuthology.orchestra.run.smithi072.stdout: "unhealthy_workers": 0 2024-04-26T21:58:59.902 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:59.902 INFO:teuthology.orchestra.run.smithi072.stdout: "finisher-MDSRank": { 2024-04-26T21:58:59.902 INFO:teuthology.orchestra.run.smithi072.stdout: "queue_len": 0, 2024-04-26T21:58:59.902 INFO:teuthology.orchestra.run.smithi072.stdout: "complete_latency": { 2024-04-26T21:58:59.902 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 1634, 2024-04-26T21:58:59.902 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.074373039, 2024-04-26T21:58:59.902 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000045515 2024-04-26T21:58:59.902 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:59.903 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:59.903 INFO:teuthology.orchestra.run.smithi072.stdout: "finisher-PurgeQueue": { 2024-04-26T21:58:59.903 INFO:teuthology.orchestra.run.smithi072.stdout: "queue_len": 0, 2024-04-26T21:58:59.903 INFO:teuthology.orchestra.run.smithi072.stdout: "complete_latency": { 2024-04-26T21:58:59.903 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-04-26T21:58:59.903 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-04-26T21:58:59.903 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-04-26T21:58:59.903 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:59.903 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:59.903 INFO:teuthology.orchestra.run.smithi072.stdout: "mds": { 2024-04-26T21:58:59.903 INFO:teuthology.orchestra.run.smithi072.stdout: "request": 0, 2024-04-26T21:58:59.903 INFO:teuthology.orchestra.run.smithi072.stdout: "reply": 0, 2024-04-26T21:58:59.903 INFO:teuthology.orchestra.run.smithi072.stdout: "reply_latency": { 2024-04-26T21:58:59.903 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-04-26T21:58:59.904 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-04-26T21:58:59.904 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-04-26T21:58:59.904 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:59.904 INFO:teuthology.orchestra.run.smithi072.stdout: "slow_reply": 0, 2024-04-26T21:58:59.904 INFO:teuthology.orchestra.run.smithi072.stdout: "forward": 0, 2024-04-26T21:58:59.904 INFO:teuthology.orchestra.run.smithi072.stdout: "dir_fetch_complete": 0, 2024-04-26T21:58:59.904 INFO:teuthology.orchestra.run.smithi072.stdout: "dir_fetch_keys": 0, 2024-04-26T21:58:59.904 INFO:teuthology.orchestra.run.smithi072.stdout: "dir_commit": 0, 2024-04-26T21:58:59.904 INFO:teuthology.orchestra.run.smithi072.stdout: "dir_split": 0, 2024-04-26T21:58:59.904 INFO:teuthology.orchestra.run.smithi072.stdout: "dir_merge": 0, 2024-04-26T21:58:59.904 INFO:teuthology.orchestra.run.smithi072.stdout: "inodes": 0, 2024-04-26T21:58:59.906 INFO:teuthology.orchestra.run.smithi072.stdout: "inodes_top": 0, 2024-04-26T21:58:59.906 INFO:teuthology.orchestra.run.smithi072.stdout: "inodes_bottom": 0, 2024-04-26T21:58:59.906 INFO:teuthology.orchestra.run.smithi072.stdout: "inodes_pin_tail": 0, 2024-04-26T21:58:59.906 INFO:teuthology.orchestra.run.smithi072.stdout: "inodes_pinned": 0, 2024-04-26T21:58:59.906 INFO:teuthology.orchestra.run.smithi072.stdout: "inodes_expired": 0, 2024-04-26T21:58:59.906 INFO:teuthology.orchestra.run.smithi072.stdout: "inodes_with_caps": 0, 2024-04-26T21:58:59.906 INFO:teuthology.orchestra.run.smithi072.stdout: "caps": 0, 2024-04-26T21:58:59.906 INFO:teuthology.orchestra.run.smithi072.stdout: "subtrees": 1, 2024-04-26T21:58:59.906 INFO:teuthology.orchestra.run.smithi072.stdout: "traverse": 0, 2024-04-26T21:58:59.906 INFO:teuthology.orchestra.run.smithi072.stdout: "traverse_hit": 0, 2024-04-26T21:58:59.906 INFO:teuthology.orchestra.run.smithi072.stdout: "traverse_forward": 0, 2024-04-26T21:58:59.907 INFO:teuthology.orchestra.run.smithi072.stdout: "traverse_discover": 0, 2024-04-26T21:58:59.907 INFO:teuthology.orchestra.run.smithi072.stdout: "traverse_dir_fetch": 0, 2024-04-26T21:58:59.907 INFO:teuthology.orchestra.run.smithi072.stdout: "traverse_remote_ino": 0, 2024-04-26T21:58:59.907 INFO:teuthology.orchestra.run.smithi072.stdout: "traverse_lock": 0, 2024-04-26T21:58:59.907 INFO:teuthology.orchestra.run.smithi072.stdout: "load_cent": 0, 2024-04-26T21:58:59.907 INFO:teuthology.orchestra.run.smithi072.stdout: "q": 0, 2024-04-26T21:58:59.907 INFO:teuthology.orchestra.run.smithi072.stdout: "exported": 0, 2024-04-26T21:58:59.907 INFO:teuthology.orchestra.run.smithi072.stdout: "exported_inodes": 0, 2024-04-26T21:58:59.907 INFO:teuthology.orchestra.run.smithi072.stdout: "imported": 0, 2024-04-26T21:58:59.907 INFO:teuthology.orchestra.run.smithi072.stdout: "imported_inodes": 0, 2024-04-26T21:58:59.907 INFO:teuthology.orchestra.run.smithi072.stdout: "openino_dir_fetch": 0, 2024-04-26T21:58:59.907 INFO:teuthology.orchestra.run.smithi072.stdout: "openino_backtrace_fetch": 0, 2024-04-26T21:58:59.907 INFO:teuthology.orchestra.run.smithi072.stdout: "openino_peer_discover": 0, 2024-04-26T21:58:59.907 INFO:teuthology.orchestra.run.smithi072.stdout: "root_rfiles": 0, 2024-04-26T21:58:59.907 INFO:teuthology.orchestra.run.smithi072.stdout: "root_rbytes": 0, 2024-04-26T21:58:59.908 INFO:teuthology.orchestra.run.smithi072.stdout: "root_rsnaps": 0, 2024-04-26T21:58:59.908 INFO:teuthology.orchestra.run.smithi072.stdout: "scrub_backtrace_fetch": 0, 2024-04-26T21:58:59.908 INFO:teuthology.orchestra.run.smithi072.stdout: "scrub_set_tag": 0, 2024-04-26T21:58:59.908 INFO:teuthology.orchestra.run.smithi072.stdout: "scrub_backtrace_repaired": 0, 2024-04-26T21:58:59.908 INFO:teuthology.orchestra.run.smithi072.stdout: "scrub_inotable_repaired": 0, 2024-04-26T21:58:59.908 INFO:teuthology.orchestra.run.smithi072.stdout: "scrub_dir_inodes": 0, 2024-04-26T21:58:59.908 INFO:teuthology.orchestra.run.smithi072.stdout: "scrub_dir_base_inodes": 0, 2024-04-26T21:58:59.908 INFO:teuthology.orchestra.run.smithi072.stdout: "scrub_dirfrag_rstats": 0, 2024-04-26T21:58:59.908 INFO:teuthology.orchestra.run.smithi072.stdout: "scrub_file_inodes": 0, 2024-04-26T21:58:59.908 INFO:teuthology.orchestra.run.smithi072.stdout: "handle_inode_file_caps": 0, 2024-04-26T21:58:59.908 INFO:teuthology.orchestra.run.smithi072.stdout: "ceph_cap_op_revoke": 0, 2024-04-26T21:58:59.908 INFO:teuthology.orchestra.run.smithi072.stdout: "ceph_cap_op_grant": 0, 2024-04-26T21:58:59.908 INFO:teuthology.orchestra.run.smithi072.stdout: "ceph_cap_op_trunc": 0, 2024-04-26T21:58:59.908 INFO:teuthology.orchestra.run.smithi072.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-26T21:58:59.909 INFO:teuthology.orchestra.run.smithi072.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-26T21:58:59.909 INFO:teuthology.orchestra.run.smithi072.stdout: "handle_client_caps": 0, 2024-04-26T21:58:59.909 INFO:teuthology.orchestra.run.smithi072.stdout: "handle_client_caps_dirty": 0, 2024-04-26T21:58:59.909 INFO:teuthology.orchestra.run.smithi072.stdout: "handle_client_cap_release": 0, 2024-04-26T21:58:59.909 INFO:teuthology.orchestra.run.smithi072.stdout: "process_request_cap_release": 0 2024-04-26T21:58:59.909 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:59.909 INFO:teuthology.orchestra.run.smithi072.stdout: "mds_cache": { 2024-04-26T21:58:59.909 INFO:teuthology.orchestra.run.smithi072.stdout: "dir_update": 0, 2024-04-26T21:58:59.909 INFO:teuthology.orchestra.run.smithi072.stdout: "dir_update_receipt": 0, 2024-04-26T21:58:59.909 INFO:teuthology.orchestra.run.smithi072.stdout: "dir_try_discover": 0, 2024-04-26T21:58:59.909 INFO:teuthology.orchestra.run.smithi072.stdout: "dir_send_discover": 0, 2024-04-26T21:58:59.909 INFO:teuthology.orchestra.run.smithi072.stdout: "dir_handle_discover": 0, 2024-04-26T21:58:59.909 INFO:teuthology.orchestra.run.smithi072.stdout: "num_strays": 0, 2024-04-26T21:58:59.909 INFO:teuthology.orchestra.run.smithi072.stdout: "num_strays_delayed": 0, 2024-04-26T21:58:59.909 INFO:teuthology.orchestra.run.smithi072.stdout: "num_strays_enqueuing": 0, 2024-04-26T21:58:59.909 INFO:teuthology.orchestra.run.smithi072.stdout: "strays_created": 0, 2024-04-26T21:58:59.910 INFO:teuthology.orchestra.run.smithi072.stdout: "strays_enqueued": 0, 2024-04-26T21:58:59.910 INFO:teuthology.orchestra.run.smithi072.stdout: "strays_reintegrated": 0, 2024-04-26T21:58:59.910 INFO:teuthology.orchestra.run.smithi072.stdout: "strays_migrated": 0, 2024-04-26T21:58:59.910 INFO:teuthology.orchestra.run.smithi072.stdout: "num_recovering_processing": 0, 2024-04-26T21:58:59.910 INFO:teuthology.orchestra.run.smithi072.stdout: "num_recovering_enqueued": 0, 2024-04-26T21:58:59.910 INFO:teuthology.orchestra.run.smithi072.stdout: "num_recovering_prioritized": 0, 2024-04-26T21:58:59.910 INFO:teuthology.orchestra.run.smithi072.stdout: "recovery_started": 0, 2024-04-26T21:58:59.910 INFO:teuthology.orchestra.run.smithi072.stdout: "recovery_completed": 0, 2024-04-26T21:58:59.910 INFO:teuthology.orchestra.run.smithi072.stdout: "ireq_quiesce_path": 0, 2024-04-26T21:58:59.910 INFO:teuthology.orchestra.run.smithi072.stdout: "ireq_quiesce_inode": 0, 2024-04-26T21:58:59.910 INFO:teuthology.orchestra.run.smithi072.stdout: "ireq_enqueue_scrub": 0, 2024-04-26T21:58:59.910 INFO:teuthology.orchestra.run.smithi072.stdout: "ireq_exportdir": 0, 2024-04-26T21:58:59.910 INFO:teuthology.orchestra.run.smithi072.stdout: "ireq_flush": 0, 2024-04-26T21:58:59.910 INFO:teuthology.orchestra.run.smithi072.stdout: "ireq_fragmentdir": 0, 2024-04-26T21:58:59.910 INFO:teuthology.orchestra.run.smithi072.stdout: "ireq_fragstats": 0, 2024-04-26T21:58:59.911 INFO:teuthology.orchestra.run.smithi072.stdout: "ireq_inodestats": 0 2024-04-26T21:58:59.911 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:59.911 INFO:teuthology.orchestra.run.smithi072.stdout: "mds_log": { 2024-04-26T21:58:59.911 INFO:teuthology.orchestra.run.smithi072.stdout: "evlrg": 0, 2024-04-26T21:58:59.911 INFO:teuthology.orchestra.run.smithi072.stdout: "evadd": 0, 2024-04-26T21:58:59.911 INFO:teuthology.orchestra.run.smithi072.stdout: "evex": 0, 2024-04-26T21:58:59.911 INFO:teuthology.orchestra.run.smithi072.stdout: "evtrm": 0, 2024-04-26T21:58:59.911 INFO:teuthology.orchestra.run.smithi072.stdout: "ev": 4, 2024-04-26T21:58:59.911 INFO:teuthology.orchestra.run.smithi072.stdout: "evexg": 0, 2024-04-26T21:58:59.911 INFO:teuthology.orchestra.run.smithi072.stdout: "evexd": 0, 2024-04-26T21:58:59.911 INFO:teuthology.orchestra.run.smithi072.stdout: "segadd": 0, 2024-04-26T21:58:59.911 INFO:teuthology.orchestra.run.smithi072.stdout: "segex": 0, 2024-04-26T21:58:59.911 INFO:teuthology.orchestra.run.smithi072.stdout: "segtrm": 0, 2024-04-26T21:58:59.911 INFO:teuthology.orchestra.run.smithi072.stdout: "seg": 2, 2024-04-26T21:58:59.912 INFO:teuthology.orchestra.run.smithi072.stdout: "segmjr": 2, 2024-04-26T21:58:59.912 INFO:teuthology.orchestra.run.smithi072.stdout: "segexg": 0, 2024-04-26T21:58:59.912 INFO:teuthology.orchestra.run.smithi072.stdout: "segexd": 0, 2024-04-26T21:58:59.912 INFO:teuthology.orchestra.run.smithi072.stdout: "expos": 4194304, 2024-04-26T21:58:59.912 INFO:teuthology.orchestra.run.smithi072.stdout: "wrpos": 4195776, 2024-04-26T21:58:59.912 INFO:teuthology.orchestra.run.smithi072.stdout: "rdpos": 4195283, 2024-04-26T21:58:59.912 INFO:teuthology.orchestra.run.smithi072.stdout: "jlat": { 2024-04-26T21:58:59.912 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-04-26T21:58:59.912 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-04-26T21:58:59.912 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-04-26T21:58:59.912 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:59.913 INFO:teuthology.orchestra.run.smithi072.stdout: "replayed": 4 2024-04-26T21:58:59.913 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:59.913 INFO:teuthology.orchestra.run.smithi072.stdout: "mds_mem": { 2024-04-26T21:58:59.913 INFO:teuthology.orchestra.run.smithi072.stdout: "ino": 3, 2024-04-26T21:58:59.913 INFO:teuthology.orchestra.run.smithi072.stdout: "ino+": 3, 2024-04-26T21:58:59.913 INFO:teuthology.orchestra.run.smithi072.stdout: "ino-": 0, 2024-04-26T21:58:59.913 INFO:teuthology.orchestra.run.smithi072.stdout: "dir": 1, 2024-04-26T21:58:59.913 INFO:teuthology.orchestra.run.smithi072.stdout: "dir+": 1, 2024-04-26T21:58:59.913 INFO:teuthology.orchestra.run.smithi072.stdout: "dir-": 0, 2024-04-26T21:58:59.913 INFO:teuthology.orchestra.run.smithi072.stdout: "dn": 0, 2024-04-26T21:58:59.913 INFO:teuthology.orchestra.run.smithi072.stdout: "dn+": 0, 2024-04-26T21:58:59.913 INFO:teuthology.orchestra.run.smithi072.stdout: "dn-": 0, 2024-04-26T21:58:59.913 INFO:teuthology.orchestra.run.smithi072.stdout: "cap": 0, 2024-04-26T21:58:59.913 INFO:teuthology.orchestra.run.smithi072.stdout: "cap+": 0, 2024-04-26T21:58:59.913 INFO:teuthology.orchestra.run.smithi072.stdout: "cap-": 0, 2024-04-26T21:58:59.913 INFO:teuthology.orchestra.run.smithi072.stdout: "rss": 48256, 2024-04-26T21:58:59.914 INFO:teuthology.orchestra.run.smithi072.stdout: "heap": 215416 2024-04-26T21:58:59.914 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:59.914 INFO:teuthology.orchestra.run.smithi072.stdout: "mds_server": { 2024-04-26T21:58:59.914 INFO:teuthology.orchestra.run.smithi072.stdout: "dispatch_client_request": 0, 2024-04-26T21:58:59.914 INFO:teuthology.orchestra.run.smithi072.stdout: "dispatch_server_request": 0, 2024-04-26T21:58:59.914 INFO:teuthology.orchestra.run.smithi072.stdout: "handle_client_request": 0, 2024-04-26T21:58:59.914 INFO:teuthology.orchestra.run.smithi072.stdout: "handle_client_session": 0, 2024-04-26T21:58:59.914 INFO:teuthology.orchestra.run.smithi072.stdout: "handle_peer_request": 0, 2024-04-26T21:58:59.914 INFO:teuthology.orchestra.run.smithi072.stdout: "req_create_latency": { 2024-04-26T21:58:59.914 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-04-26T21:58:59.914 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-04-26T21:58:59.914 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-04-26T21:58:59.914 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:59.915 INFO:teuthology.orchestra.run.smithi072.stdout: "req_getattr_latency": { 2024-04-26T21:58:59.915 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-04-26T21:58:59.915 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-04-26T21:58:59.915 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-04-26T21:58:59.915 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:59.915 INFO:teuthology.orchestra.run.smithi072.stdout: "req_getfilelock_latency": { 2024-04-26T21:58:59.915 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-04-26T21:58:59.915 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-04-26T21:58:59.915 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-04-26T21:58:59.915 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:59.915 INFO:teuthology.orchestra.run.smithi072.stdout: "req_link_latency": { 2024-04-26T21:58:59.915 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-04-26T21:58:59.915 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-04-26T21:58:59.915 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-04-26T21:58:59.915 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:59.916 INFO:teuthology.orchestra.run.smithi072.stdout: "req_lookup_latency": { 2024-04-26T21:58:59.916 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-04-26T21:58:59.916 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-04-26T21:58:59.916 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-04-26T21:58:59.916 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:59.916 INFO:teuthology.orchestra.run.smithi072.stdout: "req_lookuphash_latency": { 2024-04-26T21:58:59.916 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-04-26T21:58:59.916 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-04-26T21:58:59.916 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-04-26T21:58:59.916 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:59.916 INFO:teuthology.orchestra.run.smithi072.stdout: "req_lookupino_latency": { 2024-04-26T21:58:59.916 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-04-26T21:58:59.916 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-04-26T21:58:59.916 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-04-26T21:58:59.917 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:59.917 INFO:teuthology.orchestra.run.smithi072.stdout: "req_lookupname_latency": { 2024-04-26T21:58:59.917 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-04-26T21:58:59.917 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-04-26T21:58:59.917 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-04-26T21:58:59.917 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:59.917 INFO:teuthology.orchestra.run.smithi072.stdout: "req_lookupparent_latency": { 2024-04-26T21:58:59.917 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-04-26T21:58:59.917 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-04-26T21:58:59.917 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-04-26T21:58:59.917 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:59.917 INFO:teuthology.orchestra.run.smithi072.stdout: "req_lookupsnap_latency": { 2024-04-26T21:58:59.917 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-04-26T21:58:59.917 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-04-26T21:58:59.917 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-04-26T21:58:59.918 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:59.918 INFO:teuthology.orchestra.run.smithi072.stdout: "req_lssnap_latency": { 2024-04-26T21:58:59.918 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-04-26T21:58:59.918 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-04-26T21:58:59.918 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-04-26T21:58:59.918 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:59.918 INFO:teuthology.orchestra.run.smithi072.stdout: "req_mkdir_latency": { 2024-04-26T21:58:59.918 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-04-26T21:58:59.919 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-04-26T21:58:59.919 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-04-26T21:58:59.919 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:59.919 INFO:teuthology.orchestra.run.smithi072.stdout: "req_mknod_latency": { 2024-04-26T21:58:59.919 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-04-26T21:58:59.919 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-04-26T21:58:59.919 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-04-26T21:58:59.919 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:59.919 INFO:teuthology.orchestra.run.smithi072.stdout: "req_mksnap_latency": { 2024-04-26T21:58:59.919 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-04-26T21:58:59.919 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-04-26T21:58:59.920 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-04-26T21:58:59.920 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:59.920 INFO:teuthology.orchestra.run.smithi072.stdout: "req_open_latency": { 2024-04-26T21:58:59.920 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-04-26T21:58:59.920 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-04-26T21:58:59.920 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-04-26T21:58:59.920 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:59.920 INFO:teuthology.orchestra.run.smithi072.stdout: "req_readdir_latency": { 2024-04-26T21:58:59.920 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-04-26T21:58:59.920 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-04-26T21:58:59.920 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-04-26T21:58:59.920 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:59.920 INFO:teuthology.orchestra.run.smithi072.stdout: "req_rename_latency": { 2024-04-26T21:58:59.921 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-04-26T21:58:59.921 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-04-26T21:58:59.921 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-04-26T21:58:59.921 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:59.921 INFO:teuthology.orchestra.run.smithi072.stdout: "req_renamesnap_latency": { 2024-04-26T21:58:59.921 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-04-26T21:58:59.921 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-04-26T21:58:59.921 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-04-26T21:58:59.921 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:59.921 INFO:teuthology.orchestra.run.smithi072.stdout: "req_snapdiff_latency": { 2024-04-26T21:58:59.921 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-04-26T21:58:59.921 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-04-26T21:58:59.921 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-04-26T21:58:59.921 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:59.922 INFO:teuthology.orchestra.run.smithi072.stdout: "req_rmdir_latency": { 2024-04-26T21:58:59.922 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-04-26T21:58:59.922 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-04-26T21:58:59.922 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-04-26T21:58:59.922 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:59.922 INFO:teuthology.orchestra.run.smithi072.stdout: "req_rmsnap_latency": { 2024-04-26T21:58:59.922 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-04-26T21:58:59.922 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-04-26T21:58:59.922 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-04-26T21:58:59.922 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:59.922 INFO:teuthology.orchestra.run.smithi072.stdout: "req_rmxattr_latency": { 2024-04-26T21:58:59.922 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-04-26T21:58:59.922 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-04-26T21:58:59.923 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-04-26T21:58:59.923 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:59.923 INFO:teuthology.orchestra.run.smithi072.stdout: "req_setattr_latency": { 2024-04-26T21:58:59.923 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-04-26T21:58:59.923 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-04-26T21:58:59.923 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-04-26T21:58:59.923 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:59.923 INFO:teuthology.orchestra.run.smithi072.stdout: "req_setdirlayout_latency": { 2024-04-26T21:58:59.923 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-04-26T21:58:59.923 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-04-26T21:58:59.923 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-04-26T21:58:59.923 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:59.923 INFO:teuthology.orchestra.run.smithi072.stdout: "req_setfilelock_latency": { 2024-04-26T21:58:59.924 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-04-26T21:58:59.924 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-04-26T21:58:59.924 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-04-26T21:58:59.924 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:59.924 INFO:teuthology.orchestra.run.smithi072.stdout: "req_setlayout_latency": { 2024-04-26T21:58:59.924 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-04-26T21:58:59.924 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-04-26T21:58:59.924 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-04-26T21:58:59.924 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:59.924 INFO:teuthology.orchestra.run.smithi072.stdout: "req_setxattr_latency": { 2024-04-26T21:58:59.924 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-04-26T21:58:59.924 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-04-26T21:58:59.924 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-04-26T21:58:59.925 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:59.925 INFO:teuthology.orchestra.run.smithi072.stdout: "req_symlink_latency": { 2024-04-26T21:58:59.925 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-04-26T21:58:59.925 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-04-26T21:58:59.925 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-04-26T21:58:59.925 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:59.925 INFO:teuthology.orchestra.run.smithi072.stdout: "req_unlink_latency": { 2024-04-26T21:58:59.925 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-04-26T21:58:59.925 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-04-26T21:58:59.925 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-04-26T21:58:59.925 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:59.925 INFO:teuthology.orchestra.run.smithi072.stdout: "cap_revoke_eviction": 0, 2024-04-26T21:58:59.925 INFO:teuthology.orchestra.run.smithi072.stdout: "cap_acquisition_throttle": 0, 2024-04-26T21:58:59.925 INFO:teuthology.orchestra.run.smithi072.stdout: "req_getvxattr_latency": { 2024-04-26T21:58:59.926 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-04-26T21:58:59.926 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-04-26T21:58:59.926 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-04-26T21:58:59.926 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:59.926 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:59.926 INFO:teuthology.orchestra.run.smithi072.stdout: "mds_sessions": { 2024-04-26T21:58:59.926 INFO:teuthology.orchestra.run.smithi072.stdout: "session_count": 2, 2024-04-26T21:58:59.926 INFO:teuthology.orchestra.run.smithi072.stdout: "session_add": 2, 2024-04-26T21:58:59.926 INFO:teuthology.orchestra.run.smithi072.stdout: "session_remove": 0, 2024-04-26T21:58:59.926 INFO:teuthology.orchestra.run.smithi072.stdout: "sessions_open": 2, 2024-04-26T21:58:59.926 INFO:teuthology.orchestra.run.smithi072.stdout: "sessions_stale": 0, 2024-04-26T21:58:59.926 INFO:teuthology.orchestra.run.smithi072.stdout: "total_load": 0, 2024-04-26T21:58:59.926 INFO:teuthology.orchestra.run.smithi072.stdout: "average_load": 0, 2024-04-26T21:58:59.926 INFO:teuthology.orchestra.run.smithi072.stdout: "avg_session_uptime": 1657, 2024-04-26T21:58:59.927 INFO:teuthology.orchestra.run.smithi072.stdout: "mdthresh_evicted": 0 2024-04-26T21:58:59.927 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:59.927 INFO:teuthology.orchestra.run.smithi072.stdout: "mempool": { 2024-04-26T21:58:59.927 INFO:teuthology.orchestra.run.smithi072.stdout: "bloom_filter_bytes": 0, 2024-04-26T21:58:59.927 INFO:teuthology.orchestra.run.smithi072.stdout: "bloom_filter_items": 0, 2024-04-26T21:58:59.927 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_alloc_bytes": 0, 2024-04-26T21:58:59.927 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_alloc_items": 0, 2024-04-26T21:58:59.927 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_data_bytes": 0, 2024-04-26T21:58:59.927 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_data_items": 0, 2024-04-26T21:58:59.927 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-26T21:58:59.927 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_onode_items": 0, 2024-04-26T21:58:59.927 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-26T21:58:59.927 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_meta_items": 0, 2024-04-26T21:58:59.927 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_other_bytes": 0, 2024-04-26T21:58:59.927 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_other_items": 0, 2024-04-26T21:58:59.928 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-26T21:58:59.928 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_buffer_items": 0, 2024-04-26T21:58:59.928 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_extent_bytes": 0, 2024-04-26T21:58:59.928 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_extent_items": 0, 2024-04-26T21:58:59.928 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_blob_bytes": 0, 2024-04-26T21:58:59.928 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_blob_items": 0, 2024-04-26T21:58:59.928 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-26T21:58:59.928 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_shared_blob_items": 0, 2024-04-26T21:58:59.928 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-26T21:58:59.928 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_inline_bl_items": 0, 2024-04-26T21:58:59.928 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_fsck_bytes": 0, 2024-04-26T21:58:59.928 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_fsck_items": 0, 2024-04-26T21:58:59.928 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_txc_bytes": 0, 2024-04-26T21:58:59.928 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_txc_items": 0, 2024-04-26T21:58:59.928 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-26T21:58:59.929 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_writing_deferred_items": 0, 2024-04-26T21:58:59.929 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_writing_bytes": 0, 2024-04-26T21:58:59.929 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_writing_items": 0, 2024-04-26T21:58:59.929 INFO:teuthology.orchestra.run.smithi072.stdout: "bluefs_bytes": 0, 2024-04-26T21:58:59.929 INFO:teuthology.orchestra.run.smithi072.stdout: "bluefs_items": 0, 2024-04-26T21:58:59.929 INFO:teuthology.orchestra.run.smithi072.stdout: "bluefs_file_reader_bytes": 0, 2024-04-26T21:58:59.929 INFO:teuthology.orchestra.run.smithi072.stdout: "bluefs_file_reader_items": 0, 2024-04-26T21:58:59.929 INFO:teuthology.orchestra.run.smithi072.stdout: "bluefs_file_writer_bytes": 0, 2024-04-26T21:58:59.929 INFO:teuthology.orchestra.run.smithi072.stdout: "bluefs_file_writer_items": 0, 2024-04-26T21:58:59.929 INFO:teuthology.orchestra.run.smithi072.stdout: "buffer_anon_bytes": 189319, 2024-04-26T21:58:59.929 INFO:teuthology.orchestra.run.smithi072.stdout: "buffer_anon_items": 50, 2024-04-26T21:58:59.929 INFO:teuthology.orchestra.run.smithi072.stdout: "buffer_meta_bytes": 0, 2024-04-26T21:58:59.929 INFO:teuthology.orchestra.run.smithi072.stdout: "buffer_meta_items": 0, 2024-04-26T21:58:59.929 INFO:teuthology.orchestra.run.smithi072.stdout: "osd_bytes": 0, 2024-04-26T21:58:59.929 INFO:teuthology.orchestra.run.smithi072.stdout: "osd_items": 0, 2024-04-26T21:58:59.930 INFO:teuthology.orchestra.run.smithi072.stdout: "osd_mapbl_bytes": 0, 2024-04-26T21:58:59.930 INFO:teuthology.orchestra.run.smithi072.stdout: "osd_mapbl_items": 0, 2024-04-26T21:58:59.930 INFO:teuthology.orchestra.run.smithi072.stdout: "osd_pglog_bytes": 0, 2024-04-26T21:58:59.930 INFO:teuthology.orchestra.run.smithi072.stdout: "osd_pglog_items": 0, 2024-04-26T21:58:59.930 INFO:teuthology.orchestra.run.smithi072.stdout: "osdmap_bytes": 8008, 2024-04-26T21:58:59.930 INFO:teuthology.orchestra.run.smithi072.stdout: "osdmap_items": 168, 2024-04-26T21:58:59.930 INFO:teuthology.orchestra.run.smithi072.stdout: "osdmap_mapping_bytes": 0, 2024-04-26T21:58:59.930 INFO:teuthology.orchestra.run.smithi072.stdout: "osdmap_mapping_items": 0, 2024-04-26T21:58:59.930 INFO:teuthology.orchestra.run.smithi072.stdout: "pgmap_bytes": 0, 2024-04-26T21:58:59.930 INFO:teuthology.orchestra.run.smithi072.stdout: "pgmap_items": 0, 2024-04-26T21:58:59.930 INFO:teuthology.orchestra.run.smithi072.stdout: "mds_co_bytes": 8512, 2024-04-26T21:58:59.930 INFO:teuthology.orchestra.run.smithi072.stdout: "mds_co_items": 13, 2024-04-26T21:58:59.930 INFO:teuthology.orchestra.run.smithi072.stdout: "unittest_1_bytes": 0, 2024-04-26T21:58:59.930 INFO:teuthology.orchestra.run.smithi072.stdout: "unittest_1_items": 0, 2024-04-26T21:58:59.930 INFO:teuthology.orchestra.run.smithi072.stdout: "unittest_2_bytes": 0, 2024-04-26T21:58:59.931 INFO:teuthology.orchestra.run.smithi072.stdout: "unittest_2_items": 0 2024-04-26T21:58:59.931 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:59.931 INFO:teuthology.orchestra.run.smithi072.stdout: "objecter": { 2024-04-26T21:58:59.931 INFO:teuthology.orchestra.run.smithi072.stdout: "op_active": 0, 2024-04-26T21:58:59.931 INFO:teuthology.orchestra.run.smithi072.stdout: "op_laggy": 0, 2024-04-26T21:58:59.931 INFO:teuthology.orchestra.run.smithi072.stdout: "op_send": 948, 2024-04-26T21:58:59.931 INFO:teuthology.orchestra.run.smithi072.stdout: "op_send_bytes": 21, 2024-04-26T21:58:59.931 INFO:teuthology.orchestra.run.smithi072.stdout: "op_resend": 3, 2024-04-26T21:58:59.931 INFO:teuthology.orchestra.run.smithi072.stdout: "op_reply": 948, 2024-04-26T21:58:59.931 INFO:teuthology.orchestra.run.smithi072.stdout: "op_latency": { 2024-04-26T21:58:59.931 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 948, 2024-04-26T21:58:59.931 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 1.137977140, 2024-04-26T21:58:59.931 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.001200397 2024-04-26T21:58:59.931 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:59.931 INFO:teuthology.orchestra.run.smithi072.stdout: "op_inflight": 0, 2024-04-26T21:58:59.932 INFO:teuthology.orchestra.run.smithi072.stdout: "oplen_avg": { 2024-04-26T21:58:59.932 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 948, 2024-04-26T21:58:59.932 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 949 2024-04-26T21:58:59.932 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:59.932 INFO:teuthology.orchestra.run.smithi072.stdout: "op": 948, 2024-04-26T21:58:59.932 INFO:teuthology.orchestra.run.smithi072.stdout: "op_r": 948, 2024-04-26T21:58:59.932 INFO:teuthology.orchestra.run.smithi072.stdout: "op_w": 0, 2024-04-26T21:58:59.932 INFO:teuthology.orchestra.run.smithi072.stdout: "op_rmw": 0, 2024-04-26T21:58:59.932 INFO:teuthology.orchestra.run.smithi072.stdout: "op_pg": 0, 2024-04-26T21:58:59.932 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_stat": 580, 2024-04-26T21:58:59.932 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_create": 0, 2024-04-26T21:58:59.932 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_read": 366, 2024-04-26T21:58:59.932 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_write": 0, 2024-04-26T21:58:59.932 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_writefull": 0, 2024-04-26T21:58:59.932 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_writesame": 0, 2024-04-26T21:58:59.933 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_append": 0, 2024-04-26T21:58:59.933 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_zero": 0, 2024-04-26T21:58:59.933 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_truncate": 0, 2024-04-26T21:58:59.933 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_delete": 0, 2024-04-26T21:58:59.933 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_mapext": 0, 2024-04-26T21:58:59.933 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_sparse_read": 0, 2024-04-26T21:58:59.933 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_clonerange": 0, 2024-04-26T21:58:59.933 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_getxattr": 1, 2024-04-26T21:58:59.933 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_setxattr": 0, 2024-04-26T21:58:59.933 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_cmpxattr": 0, 2024-04-26T21:58:59.933 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_rmxattr": 0, 2024-04-26T21:58:59.933 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_resetxattrs": 0, 2024-04-26T21:58:59.933 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_call": 0, 2024-04-26T21:58:59.933 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_watch": 0, 2024-04-26T21:58:59.933 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_notify": 0, 2024-04-26T21:58:59.934 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_src_cmpxattr": 0, 2024-04-26T21:58:59.934 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_pgls": 0, 2024-04-26T21:58:59.934 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_pgls_filter": 0, 2024-04-26T21:58:59.934 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_other": 0, 2024-04-26T21:58:59.934 INFO:teuthology.orchestra.run.smithi072.stdout: "linger_active": 0, 2024-04-26T21:58:59.934 INFO:teuthology.orchestra.run.smithi072.stdout: "linger_send": 0, 2024-04-26T21:58:59.934 INFO:teuthology.orchestra.run.smithi072.stdout: "linger_resend": 0, 2024-04-26T21:58:59.934 INFO:teuthology.orchestra.run.smithi072.stdout: "linger_ping": 0, 2024-04-26T21:58:59.934 INFO:teuthology.orchestra.run.smithi072.stdout: "poolop_active": 0, 2024-04-26T21:58:59.934 INFO:teuthology.orchestra.run.smithi072.stdout: "poolop_send": 0, 2024-04-26T21:58:59.934 INFO:teuthology.orchestra.run.smithi072.stdout: "poolop_resend": 0, 2024-04-26T21:58:59.934 INFO:teuthology.orchestra.run.smithi072.stdout: "poolstat_active": 0, 2024-04-26T21:58:59.934 INFO:teuthology.orchestra.run.smithi072.stdout: "poolstat_send": 0, 2024-04-26T21:58:59.935 INFO:teuthology.orchestra.run.smithi072.stdout: "poolstat_resend": 0, 2024-04-26T21:58:59.935 INFO:teuthology.orchestra.run.smithi072.stdout: "statfs_active": 0, 2024-04-26T21:58:59.935 INFO:teuthology.orchestra.run.smithi072.stdout: "statfs_send": 0, 2024-04-26T21:58:59.935 INFO:teuthology.orchestra.run.smithi072.stdout: "statfs_resend": 0, 2024-04-26T21:58:59.935 INFO:teuthology.orchestra.run.smithi072.stdout: "command_active": 0, 2024-04-26T21:58:59.935 INFO:teuthology.orchestra.run.smithi072.stdout: "command_send": 0, 2024-04-26T21:58:59.935 INFO:teuthology.orchestra.run.smithi072.stdout: "command_resend": 0, 2024-04-26T21:58:59.935 INFO:teuthology.orchestra.run.smithi072.stdout: "map_epoch": 91, 2024-04-26T21:58:59.935 INFO:teuthology.orchestra.run.smithi072.stdout: "map_full": 0, 2024-04-26T21:58:59.935 INFO:teuthology.orchestra.run.smithi072.stdout: "map_inc": 4, 2024-04-26T21:58:59.935 INFO:teuthology.orchestra.run.smithi072.stdout: "osd_sessions": 5, 2024-04-26T21:58:59.935 INFO:teuthology.orchestra.run.smithi072.stdout: "osd_session_open": 5, 2024-04-26T21:58:59.935 INFO:teuthology.orchestra.run.smithi072.stdout: "osd_session_close": 0, 2024-04-26T21:58:59.935 INFO:teuthology.orchestra.run.smithi072.stdout: "osd_laggy": 0, 2024-04-26T21:58:59.935 INFO:teuthology.orchestra.run.smithi072.stdout: "omap_wr": 0, 2024-04-26T21:58:59.936 INFO:teuthology.orchestra.run.smithi072.stdout: "omap_rd": 2, 2024-04-26T21:58:59.936 INFO:teuthology.orchestra.run.smithi072.stdout: "omap_del": 0 2024-04-26T21:58:59.936 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:59.936 INFO:teuthology.orchestra.run.smithi072.stdout: "oft": { 2024-04-26T21:58:59.936 INFO:teuthology.orchestra.run.smithi072.stdout: "omap_total_objs": 0, 2024-04-26T21:58:59.936 INFO:teuthology.orchestra.run.smithi072.stdout: "omap_total_kv_pairs": 0, 2024-04-26T21:58:59.936 INFO:teuthology.orchestra.run.smithi072.stdout: "omap_total_updates": 0, 2024-04-26T21:58:59.936 INFO:teuthology.orchestra.run.smithi072.stdout: "omap_total_removes": 0 2024-04-26T21:58:59.936 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:59.936 INFO:teuthology.orchestra.run.smithi072.stdout: "osd-slow-ops": { 2024-04-26T21:58:59.936 INFO:teuthology.orchestra.run.smithi072.stdout: "slow_ops_count": 0 2024-04-26T21:58:59.936 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:59.936 INFO:teuthology.orchestra.run.smithi072.stdout: "purge_queue": { 2024-04-26T21:58:59.936 INFO:teuthology.orchestra.run.smithi072.stdout: "pq_executing_ops": 0, 2024-04-26T21:58:59.936 INFO:teuthology.orchestra.run.smithi072.stdout: "pq_executing_ops_high_water": 0, 2024-04-26T21:58:59.936 INFO:teuthology.orchestra.run.smithi072.stdout: "pq_executing": 0, 2024-04-26T21:58:59.937 INFO:teuthology.orchestra.run.smithi072.stdout: "pq_executing_high_water": 0, 2024-04-26T21:58:59.937 INFO:teuthology.orchestra.run.smithi072.stdout: "pq_executed_ops": 0, 2024-04-26T21:58:59.937 INFO:teuthology.orchestra.run.smithi072.stdout: "pq_executed": 0, 2024-04-26T21:58:59.937 INFO:teuthology.orchestra.run.smithi072.stdout: "pq_item_in_journal": 0 2024-04-26T21:58:59.937 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:59.937 INFO:teuthology.orchestra.run.smithi072.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-26T21:58:59.937 INFO:teuthology.orchestra.run.smithi072.stdout: "val": 0, 2024-04-26T21:58:59.937 INFO:teuthology.orchestra.run.smithi072.stdout: "max": 104857600, 2024-04-26T21:58:59.937 INFO:teuthology.orchestra.run.smithi072.stdout: "get_started": 0, 2024-04-26T21:58:59.937 INFO:teuthology.orchestra.run.smithi072.stdout: "get": 1066, 2024-04-26T21:58:59.937 INFO:teuthology.orchestra.run.smithi072.stdout: "get_sum": 506266, 2024-04-26T21:58:59.937 INFO:teuthology.orchestra.run.smithi072.stdout: "get_or_fail_fail": 0, 2024-04-26T21:58:59.937 INFO:teuthology.orchestra.run.smithi072.stdout: "get_or_fail_success": 1066, 2024-04-26T21:58:59.937 INFO:teuthology.orchestra.run.smithi072.stdout: "take": 0, 2024-04-26T21:58:59.937 INFO:teuthology.orchestra.run.smithi072.stdout: "take_sum": 0, 2024-04-26T21:58:59.938 INFO:teuthology.orchestra.run.smithi072.stdout: "put": 1066, 2024-04-26T21:58:59.938 INFO:teuthology.orchestra.run.smithi072.stdout: "put_sum": 506266, 2024-04-26T21:58:59.938 INFO:teuthology.orchestra.run.smithi072.stdout: "wait": { 2024-04-26T21:58:59.938 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-04-26T21:58:59.938 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-04-26T21:58:59.938 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-04-26T21:58:59.938 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:59.938 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:59.938 INFO:teuthology.orchestra.run.smithi072.stdout: "throttle-objecter_bytes": { 2024-04-26T21:58:59.938 INFO:teuthology.orchestra.run.smithi072.stdout: "val": 0, 2024-04-26T21:58:59.938 INFO:teuthology.orchestra.run.smithi072.stdout: "max": 104857600, 2024-04-26T21:58:59.938 INFO:teuthology.orchestra.run.smithi072.stdout: "get_started": 0, 2024-04-26T21:58:59.938 INFO:teuthology.orchestra.run.smithi072.stdout: "get": 0, 2024-04-26T21:58:59.939 INFO:teuthology.orchestra.run.smithi072.stdout: "get_sum": 0, 2024-04-26T21:58:59.939 INFO:teuthology.orchestra.run.smithi072.stdout: "get_or_fail_fail": 0, 2024-04-26T21:58:59.939 INFO:teuthology.orchestra.run.smithi072.stdout: "get_or_fail_success": 0, 2024-04-26T21:58:59.939 INFO:teuthology.orchestra.run.smithi072.stdout: "take": 948, 2024-04-26T21:58:59.939 INFO:teuthology.orchestra.run.smithi072.stdout: "take_sum": 1477, 2024-04-26T21:58:59.939 INFO:teuthology.orchestra.run.smithi072.stdout: "put": 948, 2024-04-26T21:58:59.939 INFO:teuthology.orchestra.run.smithi072.stdout: "put_sum": 1477, 2024-04-26T21:58:59.939 INFO:teuthology.orchestra.run.smithi072.stdout: "wait": { 2024-04-26T21:58:59.939 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-04-26T21:58:59.939 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-04-26T21:58:59.939 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-04-26T21:58:59.939 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:59.939 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:59.940 INFO:teuthology.orchestra.run.smithi072.stdout: "throttle-objecter_ops": { 2024-04-26T21:58:59.940 INFO:teuthology.orchestra.run.smithi072.stdout: "val": 0, 2024-04-26T21:58:59.940 INFO:teuthology.orchestra.run.smithi072.stdout: "max": 1024, 2024-04-26T21:58:59.940 INFO:teuthology.orchestra.run.smithi072.stdout: "get_started": 0, 2024-04-26T21:58:59.940 INFO:teuthology.orchestra.run.smithi072.stdout: "get": 0, 2024-04-26T21:58:59.940 INFO:teuthology.orchestra.run.smithi072.stdout: "get_sum": 0, 2024-04-26T21:58:59.940 INFO:teuthology.orchestra.run.smithi072.stdout: "get_or_fail_fail": 0, 2024-04-26T21:58:59.940 INFO:teuthology.orchestra.run.smithi072.stdout: "get_or_fail_success": 0, 2024-04-26T21:58:59.940 INFO:teuthology.orchestra.run.smithi072.stdout: "take": 948, 2024-04-26T21:58:59.940 INFO:teuthology.orchestra.run.smithi072.stdout: "take_sum": 948, 2024-04-26T21:58:59.940 INFO:teuthology.orchestra.run.smithi072.stdout: "put": 948, 2024-04-26T21:58:59.940 INFO:teuthology.orchestra.run.smithi072.stdout: "put_sum": 948, 2024-04-26T21:58:59.940 INFO:teuthology.orchestra.run.smithi072.stdout: "wait": { 2024-04-26T21:58:59.940 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-04-26T21:58:59.941 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-04-26T21:58:59.941 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-04-26T21:58:59.941 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:59.941 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:59.941 INFO:teuthology.orchestra.run.smithi072.stdout: "throttle-write_buf_throttle": { 2024-04-26T21:58:59.941 INFO:teuthology.orchestra.run.smithi072.stdout: "val": 0, 2024-04-26T21:58:59.941 INFO:teuthology.orchestra.run.smithi072.stdout: "max": 3758096384, 2024-04-26T21:58:59.941 INFO:teuthology.orchestra.run.smithi072.stdout: "get_started": 0, 2024-04-26T21:58:59.941 INFO:teuthology.orchestra.run.smithi072.stdout: "get": 0, 2024-04-26T21:58:59.941 INFO:teuthology.orchestra.run.smithi072.stdout: "get_sum": 0, 2024-04-26T21:58:59.941 INFO:teuthology.orchestra.run.smithi072.stdout: "get_or_fail_fail": 0, 2024-04-26T21:58:59.941 INFO:teuthology.orchestra.run.smithi072.stdout: "get_or_fail_success": 0, 2024-04-26T21:58:59.941 INFO:teuthology.orchestra.run.smithi072.stdout: "take": 0, 2024-04-26T21:58:59.942 INFO:teuthology.orchestra.run.smithi072.stdout: "take_sum": 0, 2024-04-26T21:58:59.942 INFO:teuthology.orchestra.run.smithi072.stdout: "put": 0, 2024-04-26T21:58:59.942 INFO:teuthology.orchestra.run.smithi072.stdout: "put_sum": 0, 2024-04-26T21:58:59.942 INFO:teuthology.orchestra.run.smithi072.stdout: "wait": { 2024-04-26T21:58:59.942 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-04-26T21:58:59.942 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-04-26T21:58:59.942 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-04-26T21:58:59.942 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:59.942 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:58:59.942 INFO:teuthology.orchestra.run.smithi072.stdout: "throttle-write_buf_throttle-0x5639db60e780": { 2024-04-26T21:58:59.942 INFO:teuthology.orchestra.run.smithi072.stdout: "val": 0, 2024-04-26T21:58:59.942 INFO:teuthology.orchestra.run.smithi072.stdout: "max": 3758096384, 2024-04-26T21:58:59.942 INFO:teuthology.orchestra.run.smithi072.stdout: "get_started": 0, 2024-04-26T21:58:59.943 INFO:teuthology.orchestra.run.smithi072.stdout: "get": 0, 2024-04-26T21:58:59.943 INFO:teuthology.orchestra.run.smithi072.stdout: "get_sum": 0, 2024-04-26T21:58:59.943 INFO:teuthology.orchestra.run.smithi072.stdout: "get_or_fail_fail": 0, 2024-04-26T21:58:59.943 INFO:teuthology.orchestra.run.smithi072.stdout: "get_or_fail_success": 0, 2024-04-26T21:58:59.943 INFO:teuthology.orchestra.run.smithi072.stdout: "take": 0, 2024-04-26T21:58:59.943 INFO:teuthology.orchestra.run.smithi072.stdout: "take_sum": 0, 2024-04-26T21:58:59.943 INFO:teuthology.orchestra.run.smithi072.stdout: "put": 0, 2024-04-26T21:58:59.943 INFO:teuthology.orchestra.run.smithi072.stdout: "put_sum": 0, 2024-04-26T21:58:59.943 INFO:teuthology.orchestra.run.smithi072.stdout: "wait": { 2024-04-26T21:58:59.943 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-04-26T21:58:59.943 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-04-26T21:58:59.943 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-04-26T21:58:59.943 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:59.943 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:58:59.943 INFO:teuthology.orchestra.run.smithi072.stdout:} 2024-04-26T21:59:01.020 INFO:tasks.check_counter:Daemon mds.a mds.exported=0 2024-04-26T21:59:01.021 INFO:tasks.check_counter:Daemon mds.a mds.imported=0 2024-04-26T21:59:01.021 DEBUG:tasks.check_counter:Getting stats from d 2024-04-26T21:59:01.021 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid a1780950-0415-11ef-bc93-c7b262605968 -- ceph daemon mds.d perf dump 2024-04-26T21:59:01.210 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:59:00 smithi072 ceph-mon[28138]: pgmap v556: 97 pgs: 97 active+clean; 77 MiB data, 876 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 2.0 MiB/s wr, 437 op/s 2024-04-26T21:59:01.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:59:00 smithi076 ceph-mon[31823]: pgmap v556: 97 pgs: 97 active+clean; 77 MiB data, 876 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 2.0 MiB/s wr, 437 op/s 2024-04-26T21:59:01.437 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/a1780950-0415-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:59:01.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:59:00 smithi164 ceph-mon[31916]: pgmap v556: 97 pgs: 97 active+clean; 77 MiB data, 876 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 2.0 MiB/s wr, 437 op/s 2024-04-26T21:59:03.310 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:59:02 smithi072 ceph-mon[28138]: pgmap v557: 97 pgs: 97 active+clean; 37 MiB data, 816 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 476 KiB/s wr, 127 op/s 2024-04-26T21:59:03.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:59:02 smithi076 ceph-mon[31823]: pgmap v557: 97 pgs: 97 active+clean; 37 MiB data, 816 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 476 KiB/s wr, 127 op/s 2024-04-26T21:59:03.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:59:02 smithi164 ceph-mon[31916]: pgmap v557: 97 pgs: 97 active+clean; 37 MiB data, 816 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 476 KiB/s wr, 127 op/s 2024-04-26T21:59:03.918 INFO:teuthology.orchestra.run.smithi072.stdout:{ 2024-04-26T21:59:03.918 INFO:teuthology.orchestra.run.smithi072.stdout: "AsyncMessenger::Worker-0": { 2024-04-26T21:59:03.919 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_recv_messages": 0, 2024-04-26T21:59:03.919 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_messages": 2, 2024-04-26T21:59:03.919 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_recv_bytes": 0, 2024-04-26T21:59:03.919 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_bytes": 0, 2024-04-26T21:59:03.919 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_created_connections": 4, 2024-04-26T21:59:03.919 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_active_connections": 0, 2024-04-26T21:59:03.919 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_total_time": 0.003637584, 2024-04-26T21:59:03.919 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_send_time": 0.000000000, 2024-04-26T21:59:03.919 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_recv_time": 0.000632684, 2024-04-26T21:59:03.919 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-26T21:59:03.919 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T21:59:03.919 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-04-26T21:59:03.919 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-04-26T21:59:03.919 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-04-26T21:59:03.919 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:59:03.920 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_handle_ack_lat": { 2024-04-26T21:59:03.920 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-04-26T21:59:03.920 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-04-26T21:59:03.920 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-04-26T21:59:03.920 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:59:03.920 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-26T21:59:03.920 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_encrypted_bytes": 0 2024-04-26T21:59:03.920 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:59:03.920 INFO:teuthology.orchestra.run.smithi072.stdout: "AsyncMessenger::Worker-1": { 2024-04-26T21:59:03.920 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_recv_messages": 115, 2024-04-26T21:59:03.920 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_messages": 113, 2024-04-26T21:59:03.920 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_recv_bytes": 312336, 2024-04-26T21:59:03.920 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_bytes": 68528, 2024-04-26T21:59:03.920 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_created_connections": 2, 2024-04-26T21:59:03.921 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_active_connections": 1, 2024-04-26T21:59:03.921 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_total_time": 0.041690493, 2024-04-26T21:59:03.921 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_send_time": 0.015200077, 2024-04-26T21:59:03.921 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_recv_time": 0.027986530, 2024-04-26T21:59:03.921 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_fast_dispatch_time": 0.004737278, 2024-04-26T21:59:03.921 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T21:59:03.921 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 112, 2024-04-26T21:59:03.921 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.009837198, 2024-04-26T21:59:03.921 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000087832 2024-04-26T21:59:03.921 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:59:03.921 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_handle_ack_lat": { 2024-04-26T21:59:03.921 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-04-26T21:59:03.921 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-04-26T21:59:03.921 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-04-26T21:59:03.921 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:59:03.922 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_recv_encrypted_bytes": 312336, 2024-04-26T21:59:03.922 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_encrypted_bytes": 68528 2024-04-26T21:59:03.922 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:59:03.922 INFO:teuthology.orchestra.run.smithi072.stdout: "AsyncMessenger::Worker-2": { 2024-04-26T21:59:03.922 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_recv_messages": 5, 2024-04-26T21:59:03.922 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_messages": 88, 2024-04-26T21:59:03.922 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_recv_bytes": 3072, 2024-04-26T21:59:03.922 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_bytes": 93952, 2024-04-26T21:59:03.922 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_created_connections": 3, 2024-04-26T21:59:03.922 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_active_connections": 2, 2024-04-26T21:59:03.922 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_total_time": 0.015308437, 2024-04-26T21:59:03.922 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_send_time": 0.011073606, 2024-04-26T21:59:03.922 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_recv_time": 0.002661644, 2024-04-26T21:59:03.922 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-26T21:59:03.922 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T21:59:03.922 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 87, 2024-04-26T21:59:03.923 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.008755663, 2024-04-26T21:59:03.923 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000100639 2024-04-26T21:59:03.923 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:59:03.923 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_handle_ack_lat": { 2024-04-26T21:59:03.923 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-04-26T21:59:03.923 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-04-26T21:59:03.923 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-04-26T21:59:03.923 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:59:03.923 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_recv_encrypted_bytes": 3072, 2024-04-26T21:59:03.923 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_encrypted_bytes": 93952 2024-04-26T21:59:03.923 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:59:03.923 INFO:teuthology.orchestra.run.smithi072.stdout: "cct": { 2024-04-26T21:59:03.923 INFO:teuthology.orchestra.run.smithi072.stdout: "total_workers": 0, 2024-04-26T21:59:03.923 INFO:teuthology.orchestra.run.smithi072.stdout: "unhealthy_workers": 0 2024-04-26T21:59:03.923 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:59:03.924 INFO:teuthology.orchestra.run.smithi072.stdout: "mempool": { 2024-04-26T21:59:03.924 INFO:teuthology.orchestra.run.smithi072.stdout: "bloom_filter_bytes": 0, 2024-04-26T21:59:03.924 INFO:teuthology.orchestra.run.smithi072.stdout: "bloom_filter_items": 0, 2024-04-26T21:59:03.924 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_alloc_bytes": 0, 2024-04-26T21:59:03.924 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_alloc_items": 0, 2024-04-26T21:59:03.924 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_data_bytes": 0, 2024-04-26T21:59:03.924 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_data_items": 0, 2024-04-26T21:59:03.924 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-26T21:59:03.924 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_onode_items": 0, 2024-04-26T21:59:03.924 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-26T21:59:03.924 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_meta_items": 0, 2024-04-26T21:59:03.924 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_other_bytes": 0, 2024-04-26T21:59:03.924 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_other_items": 0, 2024-04-26T21:59:03.924 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-26T21:59:03.925 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_buffer_items": 0, 2024-04-26T21:59:03.925 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_extent_bytes": 0, 2024-04-26T21:59:03.925 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_extent_items": 0, 2024-04-26T21:59:03.925 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_blob_bytes": 0, 2024-04-26T21:59:03.925 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_blob_items": 0, 2024-04-26T21:59:03.925 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-26T21:59:03.925 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_shared_blob_items": 0, 2024-04-26T21:59:03.925 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-26T21:59:03.925 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_inline_bl_items": 0, 2024-04-26T21:59:03.925 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_fsck_bytes": 0, 2024-04-26T21:59:03.925 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_fsck_items": 0, 2024-04-26T21:59:03.925 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_txc_bytes": 0, 2024-04-26T21:59:03.925 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_txc_items": 0, 2024-04-26T21:59:03.925 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-26T21:59:03.925 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_writing_deferred_items": 0, 2024-04-26T21:59:03.925 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_writing_bytes": 0, 2024-04-26T21:59:03.926 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_writing_items": 0, 2024-04-26T21:59:03.926 INFO:teuthology.orchestra.run.smithi072.stdout: "bluefs_bytes": 0, 2024-04-26T21:59:03.926 INFO:teuthology.orchestra.run.smithi072.stdout: "bluefs_items": 0, 2024-04-26T21:59:03.926 INFO:teuthology.orchestra.run.smithi072.stdout: "bluefs_file_reader_bytes": 0, 2024-04-26T21:59:03.926 INFO:teuthology.orchestra.run.smithi072.stdout: "bluefs_file_reader_items": 0, 2024-04-26T21:59:03.926 INFO:teuthology.orchestra.run.smithi072.stdout: "bluefs_file_writer_bytes": 0, 2024-04-26T21:59:03.926 INFO:teuthology.orchestra.run.smithi072.stdout: "bluefs_file_writer_items": 0, 2024-04-26T21:59:03.926 INFO:teuthology.orchestra.run.smithi072.stdout: "buffer_anon_bytes": 148840, 2024-04-26T21:59:03.926 INFO:teuthology.orchestra.run.smithi072.stdout: "buffer_anon_items": 28, 2024-04-26T21:59:03.926 INFO:teuthology.orchestra.run.smithi072.stdout: "buffer_meta_bytes": 0, 2024-04-26T21:59:03.926 INFO:teuthology.orchestra.run.smithi072.stdout: "buffer_meta_items": 0, 2024-04-26T21:59:03.926 INFO:teuthology.orchestra.run.smithi072.stdout: "osd_bytes": 0, 2024-04-26T21:59:03.926 INFO:teuthology.orchestra.run.smithi072.stdout: "osd_items": 0, 2024-04-26T21:59:03.926 INFO:teuthology.orchestra.run.smithi072.stdout: "osd_mapbl_bytes": 0, 2024-04-26T21:59:03.926 INFO:teuthology.orchestra.run.smithi072.stdout: "osd_mapbl_items": 0, 2024-04-26T21:59:03.926 INFO:teuthology.orchestra.run.smithi072.stdout: "osd_pglog_bytes": 0, 2024-04-26T21:59:03.927 INFO:teuthology.orchestra.run.smithi072.stdout: "osd_pglog_items": 0, 2024-04-26T21:59:03.927 INFO:teuthology.orchestra.run.smithi072.stdout: "osdmap_bytes": 0, 2024-04-26T21:59:03.927 INFO:teuthology.orchestra.run.smithi072.stdout: "osdmap_items": 0, 2024-04-26T21:59:03.927 INFO:teuthology.orchestra.run.smithi072.stdout: "osdmap_mapping_bytes": 0, 2024-04-26T21:59:03.927 INFO:teuthology.orchestra.run.smithi072.stdout: "osdmap_mapping_items": 0, 2024-04-26T21:59:03.927 INFO:teuthology.orchestra.run.smithi072.stdout: "pgmap_bytes": 0, 2024-04-26T21:59:03.927 INFO:teuthology.orchestra.run.smithi072.stdout: "pgmap_items": 0, 2024-04-26T21:59:03.927 INFO:teuthology.orchestra.run.smithi072.stdout: "mds_co_bytes": 928, 2024-04-26T21:59:03.927 INFO:teuthology.orchestra.run.smithi072.stdout: "mds_co_items": 2, 2024-04-26T21:59:03.927 INFO:teuthology.orchestra.run.smithi072.stdout: "unittest_1_bytes": 0, 2024-04-26T21:59:03.927 INFO:teuthology.orchestra.run.smithi072.stdout: "unittest_1_items": 0, 2024-04-26T21:59:03.927 INFO:teuthology.orchestra.run.smithi072.stdout: "unittest_2_bytes": 0, 2024-04-26T21:59:03.927 INFO:teuthology.orchestra.run.smithi072.stdout: "unittest_2_items": 0 2024-04-26T21:59:03.927 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:59:03.927 INFO:teuthology.orchestra.run.smithi072.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-26T21:59:03.927 INFO:teuthology.orchestra.run.smithi072.stdout: "val": 0, 2024-04-26T21:59:03.927 INFO:teuthology.orchestra.run.smithi072.stdout: "max": 104857600, 2024-04-26T21:59:03.928 INFO:teuthology.orchestra.run.smithi072.stdout: "get_started": 0, 2024-04-26T21:59:03.928 INFO:teuthology.orchestra.run.smithi072.stdout: "get": 116, 2024-04-26T21:59:03.928 INFO:teuthology.orchestra.run.smithi072.stdout: "get_sum": 296109, 2024-04-26T21:59:03.928 INFO:teuthology.orchestra.run.smithi072.stdout: "get_or_fail_fail": 0, 2024-04-26T21:59:03.928 INFO:teuthology.orchestra.run.smithi072.stdout: "get_or_fail_success": 116, 2024-04-26T21:59:03.928 INFO:teuthology.orchestra.run.smithi072.stdout: "take": 0, 2024-04-26T21:59:03.928 INFO:teuthology.orchestra.run.smithi072.stdout: "take_sum": 0, 2024-04-26T21:59:03.928 INFO:teuthology.orchestra.run.smithi072.stdout: "put": 116, 2024-04-26T21:59:03.928 INFO:teuthology.orchestra.run.smithi072.stdout: "put_sum": 296109, 2024-04-26T21:59:03.928 INFO:teuthology.orchestra.run.smithi072.stdout: "wait": { 2024-04-26T21:59:03.928 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-04-26T21:59:03.928 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-04-26T21:59:03.928 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-04-26T21:59:03.928 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:59:03.928 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:59:03.929 INFO:teuthology.orchestra.run.smithi072.stdout:} 2024-04-26T21:59:05.155 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-04-26T21:59:05.155 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-04-26T21:59:05.155 DEBUG:tasks.check_counter:Getting stats from g 2024-04-26T21:59:05.155 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid a1780950-0415-11ef-bc93-c7b262605968 -- ceph daemon mds.g perf dump 2024-04-26T21:59:05.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:59:04 smithi076 ceph-mon[31823]: pgmap v558: 97 pgs: 97 active+clean; 9.1 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 4.7 KiB/s wr, 18 op/s 2024-04-26T21:59:05.323 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:59:04 smithi072 ceph-mon[28138]: pgmap v558: 97 pgs: 97 active+clean; 9.1 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 4.7 KiB/s wr, 18 op/s 2024-04-26T21:59:05.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:59:04 smithi164 ceph-mon[31916]: pgmap v558: 97 pgs: 97 active+clean; 9.1 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 4.7 KiB/s wr, 18 op/s 2024-04-26T21:59:05.597 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/a1780950-0415-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:59:07.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:59:06 smithi076 ceph-mon[31823]: pgmap v559: 97 pgs: 97 active+clean; 1.1 MiB data, 656 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.6 KiB/s wr, 19 op/s 2024-04-26T21:59:07.321 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:59:06 smithi072 ceph-mon[28138]: pgmap v559: 97 pgs: 97 active+clean; 1.1 MiB data, 656 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.6 KiB/s wr, 19 op/s 2024-04-26T21:59:07.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:59:06 smithi164 ceph-mon[31916]: pgmap v559: 97 pgs: 97 active+clean; 1.1 MiB data, 656 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.6 KiB/s wr, 19 op/s 2024-04-26T21:59:07.887 INFO:teuthology.orchestra.run.smithi072.stdout:{ 2024-04-26T21:59:07.887 INFO:teuthology.orchestra.run.smithi072.stdout: "AsyncMessenger::Worker-0": { 2024-04-26T21:59:07.887 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_recv_messages": 489, 2024-04-26T21:59:07.887 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_messages": 486, 2024-04-26T21:59:07.887 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_recv_bytes": 459184, 2024-04-26T21:59:07.887 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_bytes": 184206, 2024-04-26T21:59:07.887 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_created_connections": 5, 2024-04-26T21:59:07.887 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_active_connections": 1, 2024-04-26T21:59:07.887 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_total_time": 0.121886921, 2024-04-26T21:59:07.887 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_send_time": 0.052667371, 2024-04-26T21:59:07.888 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_recv_time": 0.039015886, 2024-04-26T21:59:07.888 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_fast_dispatch_time": 0.023284516, 2024-04-26T21:59:07.888 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T21:59:07.888 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 485, 2024-04-26T21:59:07.888 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.041017539, 2024-04-26T21:59:07.888 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000084572 2024-04-26T21:59:07.888 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:59:07.888 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_handle_ack_lat": { 2024-04-26T21:59:07.888 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-04-26T21:59:07.888 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-04-26T21:59:07.888 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-04-26T21:59:07.888 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:59:07.889 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_recv_encrypted_bytes": 335200, 2024-04-26T21:59:07.889 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_encrypted_bytes": 66864 2024-04-26T21:59:07.889 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:59:07.889 INFO:teuthology.orchestra.run.smithi072.stdout: "AsyncMessenger::Worker-1": { 2024-04-26T21:59:07.889 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_recv_messages": 604, 2024-04-26T21:59:07.889 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_messages": 683, 2024-04-26T21:59:07.889 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_recv_bytes": 157475, 2024-04-26T21:59:07.889 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_bytes": 412651, 2024-04-26T21:59:07.889 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_created_connections": 5, 2024-04-26T21:59:07.889 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_active_connections": 4, 2024-04-26T21:59:07.889 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_total_time": 0.101828522, 2024-04-26T21:59:07.889 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_send_time": 0.040929902, 2024-04-26T21:59:07.890 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_recv_time": 0.024660308, 2024-04-26T21:59:07.890 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_fast_dispatch_time": 0.027216331, 2024-04-26T21:59:07.890 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T21:59:07.890 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 682, 2024-04-26T21:59:07.890 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.054228983, 2024-04-26T21:59:07.890 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000079514 2024-04-26T21:59:07.890 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:59:07.890 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_handle_ack_lat": { 2024-04-26T21:59:07.890 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-04-26T21:59:07.890 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-04-26T21:59:07.890 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-04-26T21:59:07.890 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:59:07.890 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_recv_encrypted_bytes": 3072, 2024-04-26T21:59:07.891 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_encrypted_bytes": 222112 2024-04-26T21:59:07.891 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:59:07.891 INFO:teuthology.orchestra.run.smithi072.stdout: "AsyncMessenger::Worker-2": { 2024-04-26T21:59:07.891 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_recv_messages": 4, 2024-04-26T21:59:07.891 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_messages": 6, 2024-04-26T21:59:07.891 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_recv_bytes": 1629, 2024-04-26T21:59:07.891 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_bytes": 1284, 2024-04-26T21:59:07.891 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_created_connections": 5, 2024-04-26T21:59:07.891 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_active_connections": 4, 2024-04-26T21:59:07.891 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_total_time": 0.002820908, 2024-04-26T21:59:07.891 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_send_time": 0.000150769, 2024-04-26T21:59:07.892 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_recv_time": 0.002322536, 2024-04-26T21:59:07.892 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_fast_dispatch_time": 0.000126602, 2024-04-26T21:59:07.892 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T21:59:07.892 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 4, 2024-04-26T21:59:07.892 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.006686234, 2024-04-26T21:59:07.892 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.001671558 2024-04-26T21:59:07.892 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:59:07.892 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_handle_ack_lat": { 2024-04-26T21:59:07.892 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-04-26T21:59:07.892 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-04-26T21:59:07.892 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-04-26T21:59:07.892 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:59:07.892 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-26T21:59:07.892 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_encrypted_bytes": 0 2024-04-26T21:59:07.893 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:59:07.893 INFO:teuthology.orchestra.run.smithi072.stdout: "cct": { 2024-04-26T21:59:07.893 INFO:teuthology.orchestra.run.smithi072.stdout: "total_workers": 1, 2024-04-26T21:59:07.893 INFO:teuthology.orchestra.run.smithi072.stdout: "unhealthy_workers": 0 2024-04-26T21:59:07.893 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:59:07.893 INFO:teuthology.orchestra.run.smithi072.stdout: "finisher-MDSRank": { 2024-04-26T21:59:07.893 INFO:teuthology.orchestra.run.smithi072.stdout: "queue_len": 0, 2024-04-26T21:59:07.893 INFO:teuthology.orchestra.run.smithi072.stdout: "complete_latency": { 2024-04-26T21:59:07.893 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 1700, 2024-04-26T21:59:07.893 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.076346729, 2024-04-26T21:59:07.893 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000044909 2024-04-26T21:59:07.893 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:59:07.893 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:59:07.894 INFO:teuthology.orchestra.run.smithi072.stdout: "finisher-PurgeQueue": { 2024-04-26T21:59:07.894 INFO:teuthology.orchestra.run.smithi072.stdout: "queue_len": 0, 2024-04-26T21:59:07.894 INFO:teuthology.orchestra.run.smithi072.stdout: "complete_latency": { 2024-04-26T21:59:07.894 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-04-26T21:59:07.894 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-04-26T21:59:07.894 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-04-26T21:59:07.894 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:59:07.894 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:59:07.894 INFO:teuthology.orchestra.run.smithi072.stdout: "mds": { 2024-04-26T21:59:07.894 INFO:teuthology.orchestra.run.smithi072.stdout: "request": 0, 2024-04-26T21:59:07.894 INFO:teuthology.orchestra.run.smithi072.stdout: "reply": 0, 2024-04-26T21:59:07.894 INFO:teuthology.orchestra.run.smithi072.stdout: "reply_latency": { 2024-04-26T21:59:07.895 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-04-26T21:59:07.895 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-04-26T21:59:07.895 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-04-26T21:59:07.895 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:59:07.895 INFO:teuthology.orchestra.run.smithi072.stdout: "slow_reply": 0, 2024-04-26T21:59:07.895 INFO:teuthology.orchestra.run.smithi072.stdout: "forward": 0, 2024-04-26T21:59:07.895 INFO:teuthology.orchestra.run.smithi072.stdout: "dir_fetch_complete": 0, 2024-04-26T21:59:07.895 INFO:teuthology.orchestra.run.smithi072.stdout: "dir_fetch_keys": 0, 2024-04-26T21:59:07.895 INFO:teuthology.orchestra.run.smithi072.stdout: "dir_commit": 0, 2024-04-26T21:59:07.895 INFO:teuthology.orchestra.run.smithi072.stdout: "dir_split": 0, 2024-04-26T21:59:07.895 INFO:teuthology.orchestra.run.smithi072.stdout: "dir_merge": 0, 2024-04-26T21:59:07.895 INFO:teuthology.orchestra.run.smithi072.stdout: "inodes": 0, 2024-04-26T21:59:07.895 INFO:teuthology.orchestra.run.smithi072.stdout: "inodes_top": 0, 2024-04-26T21:59:07.895 INFO:teuthology.orchestra.run.smithi072.stdout: "inodes_bottom": 0, 2024-04-26T21:59:07.895 INFO:teuthology.orchestra.run.smithi072.stdout: "inodes_pin_tail": 0, 2024-04-26T21:59:07.896 INFO:teuthology.orchestra.run.smithi072.stdout: "inodes_pinned": 0, 2024-04-26T21:59:07.896 INFO:teuthology.orchestra.run.smithi072.stdout: "inodes_expired": 0, 2024-04-26T21:59:07.896 INFO:teuthology.orchestra.run.smithi072.stdout: "inodes_with_caps": 0, 2024-04-26T21:59:07.896 INFO:teuthology.orchestra.run.smithi072.stdout: "caps": 0, 2024-04-26T21:59:07.896 INFO:teuthology.orchestra.run.smithi072.stdout: "subtrees": 1, 2024-04-26T21:59:07.896 INFO:teuthology.orchestra.run.smithi072.stdout: "traverse": 0, 2024-04-26T21:59:07.896 INFO:teuthology.orchestra.run.smithi072.stdout: "traverse_hit": 0, 2024-04-26T21:59:07.896 INFO:teuthology.orchestra.run.smithi072.stdout: "traverse_forward": 0, 2024-04-26T21:59:07.896 INFO:teuthology.orchestra.run.smithi072.stdout: "traverse_discover": 0, 2024-04-26T21:59:07.896 INFO:teuthology.orchestra.run.smithi072.stdout: "traverse_dir_fetch": 0, 2024-04-26T21:59:07.896 INFO:teuthology.orchestra.run.smithi072.stdout: "traverse_remote_ino": 0, 2024-04-26T21:59:07.897 INFO:teuthology.orchestra.run.smithi072.stdout: "traverse_lock": 0, 2024-04-26T21:59:07.897 INFO:teuthology.orchestra.run.smithi072.stdout: "load_cent": 0, 2024-04-26T21:59:07.897 INFO:teuthology.orchestra.run.smithi072.stdout: "q": 0, 2024-04-26T21:59:07.897 INFO:teuthology.orchestra.run.smithi072.stdout: "exported": 0, 2024-04-26T21:59:07.897 INFO:teuthology.orchestra.run.smithi072.stdout: "exported_inodes": 0, 2024-04-26T21:59:07.897 INFO:teuthology.orchestra.run.smithi072.stdout: "imported": 0, 2024-04-26T21:59:07.897 INFO:teuthology.orchestra.run.smithi072.stdout: "imported_inodes": 0, 2024-04-26T21:59:07.897 INFO:teuthology.orchestra.run.smithi072.stdout: "openino_dir_fetch": 0, 2024-04-26T21:59:07.897 INFO:teuthology.orchestra.run.smithi072.stdout: "openino_backtrace_fetch": 0, 2024-04-26T21:59:07.897 INFO:teuthology.orchestra.run.smithi072.stdout: "openino_peer_discover": 0, 2024-04-26T21:59:07.897 INFO:teuthology.orchestra.run.smithi072.stdout: "root_rfiles": 0, 2024-04-26T21:59:07.897 INFO:teuthology.orchestra.run.smithi072.stdout: "root_rbytes": 0, 2024-04-26T21:59:07.898 INFO:teuthology.orchestra.run.smithi072.stdout: "root_rsnaps": 0, 2024-04-26T21:59:07.898 INFO:teuthology.orchestra.run.smithi072.stdout: "scrub_backtrace_fetch": 0, 2024-04-26T21:59:07.898 INFO:teuthology.orchestra.run.smithi072.stdout: "scrub_set_tag": 0, 2024-04-26T21:59:07.898 INFO:teuthology.orchestra.run.smithi072.stdout: "scrub_backtrace_repaired": 0, 2024-04-26T21:59:07.898 INFO:teuthology.orchestra.run.smithi072.stdout: "scrub_inotable_repaired": 0, 2024-04-26T21:59:07.898 INFO:teuthology.orchestra.run.smithi072.stdout: "scrub_dir_inodes": 0, 2024-04-26T21:59:07.898 INFO:teuthology.orchestra.run.smithi072.stdout: "scrub_dir_base_inodes": 0, 2024-04-26T21:59:07.898 INFO:teuthology.orchestra.run.smithi072.stdout: "scrub_dirfrag_rstats": 0, 2024-04-26T21:59:07.898 INFO:teuthology.orchestra.run.smithi072.stdout: "scrub_file_inodes": 0, 2024-04-26T21:59:07.898 INFO:teuthology.orchestra.run.smithi072.stdout: "handle_inode_file_caps": 0, 2024-04-26T21:59:07.898 INFO:teuthology.orchestra.run.smithi072.stdout: "ceph_cap_op_revoke": 0, 2024-04-26T21:59:07.898 INFO:teuthology.orchestra.run.smithi072.stdout: "ceph_cap_op_grant": 0, 2024-04-26T21:59:07.898 INFO:teuthology.orchestra.run.smithi072.stdout: "ceph_cap_op_trunc": 0, 2024-04-26T21:59:07.899 INFO:teuthology.orchestra.run.smithi072.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-26T21:59:07.899 INFO:teuthology.orchestra.run.smithi072.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-26T21:59:07.899 INFO:teuthology.orchestra.run.smithi072.stdout: "handle_client_caps": 0, 2024-04-26T21:59:07.899 INFO:teuthology.orchestra.run.smithi072.stdout: "handle_client_caps_dirty": 0, 2024-04-26T21:59:07.899 INFO:teuthology.orchestra.run.smithi072.stdout: "handle_client_cap_release": 0, 2024-04-26T21:59:07.899 INFO:teuthology.orchestra.run.smithi072.stdout: "process_request_cap_release": 0 2024-04-26T21:59:07.899 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:59:07.899 INFO:teuthology.orchestra.run.smithi072.stdout: "mds_cache": { 2024-04-26T21:59:07.899 INFO:teuthology.orchestra.run.smithi072.stdout: "dir_update": 0, 2024-04-26T21:59:07.899 INFO:teuthology.orchestra.run.smithi072.stdout: "dir_update_receipt": 0, 2024-04-26T21:59:07.899 INFO:teuthology.orchestra.run.smithi072.stdout: "dir_try_discover": 0, 2024-04-26T21:59:07.899 INFO:teuthology.orchestra.run.smithi072.stdout: "dir_send_discover": 0, 2024-04-26T21:59:07.899 INFO:teuthology.orchestra.run.smithi072.stdout: "dir_handle_discover": 0, 2024-04-26T21:59:07.900 INFO:teuthology.orchestra.run.smithi072.stdout: "num_strays": 0, 2024-04-26T21:59:07.900 INFO:teuthology.orchestra.run.smithi072.stdout: "num_strays_delayed": 0, 2024-04-26T21:59:07.900 INFO:teuthology.orchestra.run.smithi072.stdout: "num_strays_enqueuing": 0, 2024-04-26T21:59:07.900 INFO:teuthology.orchestra.run.smithi072.stdout: "strays_created": 0, 2024-04-26T21:59:07.900 INFO:teuthology.orchestra.run.smithi072.stdout: "strays_enqueued": 0, 2024-04-26T21:59:07.900 INFO:teuthology.orchestra.run.smithi072.stdout: "strays_reintegrated": 0, 2024-04-26T21:59:07.900 INFO:teuthology.orchestra.run.smithi072.stdout: "strays_migrated": 0, 2024-04-26T21:59:07.900 INFO:teuthology.orchestra.run.smithi072.stdout: "num_recovering_processing": 0, 2024-04-26T21:59:07.900 INFO:teuthology.orchestra.run.smithi072.stdout: "num_recovering_enqueued": 0, 2024-04-26T21:59:07.900 INFO:teuthology.orchestra.run.smithi072.stdout: "num_recovering_prioritized": 0, 2024-04-26T21:59:07.900 INFO:teuthology.orchestra.run.smithi072.stdout: "recovery_started": 0, 2024-04-26T21:59:07.900 INFO:teuthology.orchestra.run.smithi072.stdout: "recovery_completed": 0, 2024-04-26T21:59:07.900 INFO:teuthology.orchestra.run.smithi072.stdout: "ireq_quiesce_path": 0, 2024-04-26T21:59:07.900 INFO:teuthology.orchestra.run.smithi072.stdout: "ireq_quiesce_inode": 0, 2024-04-26T21:59:07.900 INFO:teuthology.orchestra.run.smithi072.stdout: "ireq_enqueue_scrub": 0, 2024-04-26T21:59:07.901 INFO:teuthology.orchestra.run.smithi072.stdout: "ireq_exportdir": 0, 2024-04-26T21:59:07.901 INFO:teuthology.orchestra.run.smithi072.stdout: "ireq_flush": 0, 2024-04-26T21:59:07.901 INFO:teuthology.orchestra.run.smithi072.stdout: "ireq_fragmentdir": 0, 2024-04-26T21:59:07.901 INFO:teuthology.orchestra.run.smithi072.stdout: "ireq_fragstats": 0, 2024-04-26T21:59:07.901 INFO:teuthology.orchestra.run.smithi072.stdout: "ireq_inodestats": 0 2024-04-26T21:59:07.901 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:59:07.901 INFO:teuthology.orchestra.run.smithi072.stdout: "mds_log": { 2024-04-26T21:59:07.901 INFO:teuthology.orchestra.run.smithi072.stdout: "evlrg": 0, 2024-04-26T21:59:07.901 INFO:teuthology.orchestra.run.smithi072.stdout: "evadd": 0, 2024-04-26T21:59:07.901 INFO:teuthology.orchestra.run.smithi072.stdout: "evex": 0, 2024-04-26T21:59:07.901 INFO:teuthology.orchestra.run.smithi072.stdout: "evtrm": 0, 2024-04-26T21:59:07.901 INFO:teuthology.orchestra.run.smithi072.stdout: "ev": 3, 2024-04-26T21:59:07.901 INFO:teuthology.orchestra.run.smithi072.stdout: "evexg": 0, 2024-04-26T21:59:07.901 INFO:teuthology.orchestra.run.smithi072.stdout: "evexd": 0, 2024-04-26T21:59:07.902 INFO:teuthology.orchestra.run.smithi072.stdout: "segadd": 0, 2024-04-26T21:59:07.902 INFO:teuthology.orchestra.run.smithi072.stdout: "segex": 0, 2024-04-26T21:59:07.902 INFO:teuthology.orchestra.run.smithi072.stdout: "segtrm": 0, 2024-04-26T21:59:07.902 INFO:teuthology.orchestra.run.smithi072.stdout: "seg": 2, 2024-04-26T21:59:07.902 INFO:teuthology.orchestra.run.smithi072.stdout: "segmjr": 2, 2024-04-26T21:59:07.902 INFO:teuthology.orchestra.run.smithi072.stdout: "segexg": 0, 2024-04-26T21:59:07.902 INFO:teuthology.orchestra.run.smithi072.stdout: "segexd": 0, 2024-04-26T21:59:07.902 INFO:teuthology.orchestra.run.smithi072.stdout: "expos": 4194304, 2024-04-26T21:59:07.902 INFO:teuthology.orchestra.run.smithi072.stdout: "wrpos": 4195385, 2024-04-26T21:59:07.902 INFO:teuthology.orchestra.run.smithi072.stdout: "rdpos": 4194892, 2024-04-26T21:59:07.902 INFO:teuthology.orchestra.run.smithi072.stdout: "jlat": { 2024-04-26T21:59:07.902 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-04-26T21:59:07.902 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-04-26T21:59:07.903 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-04-26T21:59:07.903 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:59:07.903 INFO:teuthology.orchestra.run.smithi072.stdout: "replayed": 3 2024-04-26T21:59:07.903 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:59:07.903 INFO:teuthology.orchestra.run.smithi072.stdout: "mds_mem": { 2024-04-26T21:59:07.903 INFO:teuthology.orchestra.run.smithi072.stdout: "ino": 3, 2024-04-26T21:59:07.903 INFO:teuthology.orchestra.run.smithi072.stdout: "ino+": 3, 2024-04-26T21:59:07.903 INFO:teuthology.orchestra.run.smithi072.stdout: "ino-": 0, 2024-04-26T21:59:07.903 INFO:teuthology.orchestra.run.smithi072.stdout: "dir": 1, 2024-04-26T21:59:07.903 INFO:teuthology.orchestra.run.smithi072.stdout: "dir+": 1, 2024-04-26T21:59:07.903 INFO:teuthology.orchestra.run.smithi072.stdout: "dir-": 0, 2024-04-26T21:59:07.903 INFO:teuthology.orchestra.run.smithi072.stdout: "dn": 0, 2024-04-26T21:59:07.903 INFO:teuthology.orchestra.run.smithi072.stdout: "dn+": 0, 2024-04-26T21:59:07.903 INFO:teuthology.orchestra.run.smithi072.stdout: "dn-": 0, 2024-04-26T21:59:07.903 INFO:teuthology.orchestra.run.smithi072.stdout: "cap": 0, 2024-04-26T21:59:07.904 INFO:teuthology.orchestra.run.smithi072.stdout: "cap+": 0, 2024-04-26T21:59:07.904 INFO:teuthology.orchestra.run.smithi072.stdout: "cap-": 0, 2024-04-26T21:59:07.904 INFO:teuthology.orchestra.run.smithi072.stdout: "rss": 48256, 2024-04-26T21:59:07.904 INFO:teuthology.orchestra.run.smithi072.stdout: "heap": 215416 2024-04-26T21:59:07.904 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:59:07.904 INFO:teuthology.orchestra.run.smithi072.stdout: "mds_server": { 2024-04-26T21:59:07.904 INFO:teuthology.orchestra.run.smithi072.stdout: "dispatch_client_request": 0, 2024-04-26T21:59:07.904 INFO:teuthology.orchestra.run.smithi072.stdout: "dispatch_server_request": 0, 2024-04-26T21:59:07.904 INFO:teuthology.orchestra.run.smithi072.stdout: "handle_client_request": 0, 2024-04-26T21:59:07.904 INFO:teuthology.orchestra.run.smithi072.stdout: "handle_client_session": 0, 2024-04-26T21:59:07.904 INFO:teuthology.orchestra.run.smithi072.stdout: "handle_peer_request": 0, 2024-04-26T21:59:07.904 INFO:teuthology.orchestra.run.smithi072.stdout: "req_create_latency": { 2024-04-26T21:59:07.904 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-04-26T21:59:07.905 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-04-26T21:59:07.905 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-04-26T21:59:07.905 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:59:07.905 INFO:teuthology.orchestra.run.smithi072.stdout: "req_getattr_latency": { 2024-04-26T21:59:07.905 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-04-26T21:59:07.905 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-04-26T21:59:07.905 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-04-26T21:59:07.905 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:59:07.905 INFO:teuthology.orchestra.run.smithi072.stdout: "req_getfilelock_latency": { 2024-04-26T21:59:07.905 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-04-26T21:59:07.905 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-04-26T21:59:07.905 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-04-26T21:59:07.905 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:59:07.906 INFO:teuthology.orchestra.run.smithi072.stdout: "req_link_latency": { 2024-04-26T21:59:07.906 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-04-26T21:59:07.906 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-04-26T21:59:07.906 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-04-26T21:59:07.906 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:59:07.906 INFO:teuthology.orchestra.run.smithi072.stdout: "req_lookup_latency": { 2024-04-26T21:59:07.906 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-04-26T21:59:07.906 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-04-26T21:59:07.906 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-04-26T21:59:07.906 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:59:07.906 INFO:teuthology.orchestra.run.smithi072.stdout: "req_lookuphash_latency": { 2024-04-26T21:59:07.906 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-04-26T21:59:07.906 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-04-26T21:59:07.907 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-04-26T21:59:07.907 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:59:07.907 INFO:teuthology.orchestra.run.smithi072.stdout: "req_lookupino_latency": { 2024-04-26T21:59:07.907 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-04-26T21:59:07.907 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-04-26T21:59:07.907 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-04-26T21:59:07.907 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:59:07.907 INFO:teuthology.orchestra.run.smithi072.stdout: "req_lookupname_latency": { 2024-04-26T21:59:07.907 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-04-26T21:59:07.907 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-04-26T21:59:07.907 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-04-26T21:59:07.907 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:59:07.907 INFO:teuthology.orchestra.run.smithi072.stdout: "req_lookupparent_latency": { 2024-04-26T21:59:07.907 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-04-26T21:59:07.908 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-04-26T21:59:07.908 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-04-26T21:59:07.908 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:59:07.908 INFO:teuthology.orchestra.run.smithi072.stdout: "req_lookupsnap_latency": { 2024-04-26T21:59:07.908 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-04-26T21:59:07.908 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-04-26T21:59:07.908 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-04-26T21:59:07.908 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:59:07.908 INFO:teuthology.orchestra.run.smithi072.stdout: "req_lssnap_latency": { 2024-04-26T21:59:07.908 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-04-26T21:59:07.908 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-04-26T21:59:07.908 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-04-26T21:59:07.908 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:59:07.908 INFO:teuthology.orchestra.run.smithi072.stdout: "req_mkdir_latency": { 2024-04-26T21:59:07.908 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-04-26T21:59:07.909 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-04-26T21:59:07.909 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-04-26T21:59:07.909 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:59:07.909 INFO:teuthology.orchestra.run.smithi072.stdout: "req_mknod_latency": { 2024-04-26T21:59:07.909 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-04-26T21:59:07.909 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-04-26T21:59:07.909 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-04-26T21:59:07.910 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:59:07.910 INFO:teuthology.orchestra.run.smithi072.stdout: "req_mksnap_latency": { 2024-04-26T21:59:07.910 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-04-26T21:59:07.910 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-04-26T21:59:07.910 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-04-26T21:59:07.910 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:59:07.910 INFO:teuthology.orchestra.run.smithi072.stdout: "req_open_latency": { 2024-04-26T21:59:07.910 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-04-26T21:59:07.910 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-04-26T21:59:07.910 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-04-26T21:59:07.910 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:59:07.910 INFO:teuthology.orchestra.run.smithi072.stdout: "req_readdir_latency": { 2024-04-26T21:59:07.910 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-04-26T21:59:07.910 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-04-26T21:59:07.911 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-04-26T21:59:07.911 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:59:07.911 INFO:teuthology.orchestra.run.smithi072.stdout: "req_rename_latency": { 2024-04-26T21:59:07.911 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-04-26T21:59:07.911 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-04-26T21:59:07.911 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-04-26T21:59:07.911 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:59:07.911 INFO:teuthology.orchestra.run.smithi072.stdout: "req_renamesnap_latency": { 2024-04-26T21:59:07.911 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-04-26T21:59:07.911 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-04-26T21:59:07.911 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-04-26T21:59:07.911 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:59:07.911 INFO:teuthology.orchestra.run.smithi072.stdout: "req_snapdiff_latency": { 2024-04-26T21:59:07.911 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-04-26T21:59:07.912 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-04-26T21:59:07.912 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-04-26T21:59:07.912 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:59:07.912 INFO:teuthology.orchestra.run.smithi072.stdout: "req_rmdir_latency": { 2024-04-26T21:59:07.912 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-04-26T21:59:07.912 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-04-26T21:59:07.912 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-04-26T21:59:07.912 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:59:07.912 INFO:teuthology.orchestra.run.smithi072.stdout: "req_rmsnap_latency": { 2024-04-26T21:59:07.912 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-04-26T21:59:07.912 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-04-26T21:59:07.912 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-04-26T21:59:07.912 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:59:07.912 INFO:teuthology.orchestra.run.smithi072.stdout: "req_rmxattr_latency": { 2024-04-26T21:59:07.913 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-04-26T21:59:07.913 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-04-26T21:59:07.913 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-04-26T21:59:07.913 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:59:07.913 INFO:teuthology.orchestra.run.smithi072.stdout: "req_setattr_latency": { 2024-04-26T21:59:07.913 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-04-26T21:59:07.913 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-04-26T21:59:07.913 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-04-26T21:59:07.913 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:59:07.913 INFO:teuthology.orchestra.run.smithi072.stdout: "req_setdirlayout_latency": { 2024-04-26T21:59:07.913 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-04-26T21:59:07.913 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-04-26T21:59:07.913 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-04-26T21:59:07.914 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:59:07.914 INFO:teuthology.orchestra.run.smithi072.stdout: "req_setfilelock_latency": { 2024-04-26T21:59:07.914 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-04-26T21:59:07.914 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-04-26T21:59:07.914 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-04-26T21:59:07.914 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:59:07.914 INFO:teuthology.orchestra.run.smithi072.stdout: "req_setlayout_latency": { 2024-04-26T21:59:07.914 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-04-26T21:59:07.914 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-04-26T21:59:07.914 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-04-26T21:59:07.914 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:59:07.914 INFO:teuthology.orchestra.run.smithi072.stdout: "req_setxattr_latency": { 2024-04-26T21:59:07.914 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-04-26T21:59:07.915 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-04-26T21:59:07.915 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-04-26T21:59:07.915 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:59:07.915 INFO:teuthology.orchestra.run.smithi072.stdout: "req_symlink_latency": { 2024-04-26T21:59:07.915 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-04-26T21:59:07.915 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-04-26T21:59:07.915 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-04-26T21:59:07.915 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:59:07.915 INFO:teuthology.orchestra.run.smithi072.stdout: "req_unlink_latency": { 2024-04-26T21:59:07.915 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-04-26T21:59:07.915 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-04-26T21:59:07.915 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-04-26T21:59:07.915 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:59:07.915 INFO:teuthology.orchestra.run.smithi072.stdout: "cap_revoke_eviction": 0, 2024-04-26T21:59:07.916 INFO:teuthology.orchestra.run.smithi072.stdout: "cap_acquisition_throttle": 0, 2024-04-26T21:59:07.916 INFO:teuthology.orchestra.run.smithi072.stdout: "req_getvxattr_latency": { 2024-04-26T21:59:07.916 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-04-26T21:59:07.916 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-04-26T21:59:07.916 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-04-26T21:59:07.916 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:59:07.916 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:59:07.916 INFO:teuthology.orchestra.run.smithi072.stdout: "mds_sessions": { 2024-04-26T21:59:07.916 INFO:teuthology.orchestra.run.smithi072.stdout: "session_count": 1, 2024-04-26T21:59:07.916 INFO:teuthology.orchestra.run.smithi072.stdout: "session_add": 1, 2024-04-26T21:59:07.916 INFO:teuthology.orchestra.run.smithi072.stdout: "session_remove": 0, 2024-04-26T21:59:07.916 INFO:teuthology.orchestra.run.smithi072.stdout: "sessions_open": 1, 2024-04-26T21:59:07.916 INFO:teuthology.orchestra.run.smithi072.stdout: "sessions_stale": 0, 2024-04-26T21:59:07.916 INFO:teuthology.orchestra.run.smithi072.stdout: "total_load": 0, 2024-04-26T21:59:07.917 INFO:teuthology.orchestra.run.smithi072.stdout: "average_load": 0, 2024-04-26T21:59:07.917 INFO:teuthology.orchestra.run.smithi072.stdout: "avg_session_uptime": 1668, 2024-04-26T21:59:07.917 INFO:teuthology.orchestra.run.smithi072.stdout: "mdthresh_evicted": 0 2024-04-26T21:59:07.917 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:59:07.917 INFO:teuthology.orchestra.run.smithi072.stdout: "mempool": { 2024-04-26T21:59:07.917 INFO:teuthology.orchestra.run.smithi072.stdout: "bloom_filter_bytes": 0, 2024-04-26T21:59:07.917 INFO:teuthology.orchestra.run.smithi072.stdout: "bloom_filter_items": 0, 2024-04-26T21:59:07.917 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_alloc_bytes": 0, 2024-04-26T21:59:07.917 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_alloc_items": 0, 2024-04-26T21:59:07.917 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_data_bytes": 0, 2024-04-26T21:59:07.917 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_data_items": 0, 2024-04-26T21:59:07.917 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-26T21:59:07.917 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_onode_items": 0, 2024-04-26T21:59:07.917 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-26T21:59:07.918 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_meta_items": 0, 2024-04-26T21:59:07.918 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_other_bytes": 0, 2024-04-26T21:59:07.918 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_other_items": 0, 2024-04-26T21:59:07.918 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-26T21:59:07.918 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_buffer_items": 0, 2024-04-26T21:59:07.918 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_extent_bytes": 0, 2024-04-26T21:59:07.918 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_extent_items": 0, 2024-04-26T21:59:07.918 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_blob_bytes": 0, 2024-04-26T21:59:07.918 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_blob_items": 0, 2024-04-26T21:59:07.918 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-26T21:59:07.918 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_shared_blob_items": 0, 2024-04-26T21:59:07.918 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-26T21:59:07.918 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_inline_bl_items": 0, 2024-04-26T21:59:07.919 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_fsck_bytes": 0, 2024-04-26T21:59:07.919 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_fsck_items": 0, 2024-04-26T21:59:07.919 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_txc_bytes": 0, 2024-04-26T21:59:07.919 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_txc_items": 0, 2024-04-26T21:59:07.919 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-26T21:59:07.919 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_writing_deferred_items": 0, 2024-04-26T21:59:07.919 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_writing_bytes": 0, 2024-04-26T21:59:07.919 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_writing_items": 0, 2024-04-26T21:59:07.919 INFO:teuthology.orchestra.run.smithi072.stdout: "bluefs_bytes": 0, 2024-04-26T21:59:07.919 INFO:teuthology.orchestra.run.smithi072.stdout: "bluefs_items": 0, 2024-04-26T21:59:07.919 INFO:teuthology.orchestra.run.smithi072.stdout: "bluefs_file_reader_bytes": 0, 2024-04-26T21:59:07.919 INFO:teuthology.orchestra.run.smithi072.stdout: "bluefs_file_reader_items": 0, 2024-04-26T21:59:07.919 INFO:teuthology.orchestra.run.smithi072.stdout: "bluefs_file_writer_bytes": 0, 2024-04-26T21:59:07.919 INFO:teuthology.orchestra.run.smithi072.stdout: "bluefs_file_writer_items": 0, 2024-04-26T21:59:07.920 INFO:teuthology.orchestra.run.smithi072.stdout: "buffer_anon_bytes": 197336, 2024-04-26T21:59:07.920 INFO:teuthology.orchestra.run.smithi072.stdout: "buffer_anon_items": 53, 2024-04-26T21:59:07.920 INFO:teuthology.orchestra.run.smithi072.stdout: "buffer_meta_bytes": 0, 2024-04-26T21:59:07.920 INFO:teuthology.orchestra.run.smithi072.stdout: "buffer_meta_items": 0, 2024-04-26T21:59:07.920 INFO:teuthology.orchestra.run.smithi072.stdout: "osd_bytes": 0, 2024-04-26T21:59:07.920 INFO:teuthology.orchestra.run.smithi072.stdout: "osd_items": 0, 2024-04-26T21:59:07.920 INFO:teuthology.orchestra.run.smithi072.stdout: "osd_mapbl_bytes": 0, 2024-04-26T21:59:07.920 INFO:teuthology.orchestra.run.smithi072.stdout: "osd_mapbl_items": 0, 2024-04-26T21:59:07.920 INFO:teuthology.orchestra.run.smithi072.stdout: "osd_pglog_bytes": 0, 2024-04-26T21:59:07.920 INFO:teuthology.orchestra.run.smithi072.stdout: "osd_pglog_items": 0, 2024-04-26T21:59:07.920 INFO:teuthology.orchestra.run.smithi072.stdout: "osdmap_bytes": 8008, 2024-04-26T21:59:07.920 INFO:teuthology.orchestra.run.smithi072.stdout: "osdmap_items": 168, 2024-04-26T21:59:07.920 INFO:teuthology.orchestra.run.smithi072.stdout: "osdmap_mapping_bytes": 0, 2024-04-26T21:59:07.920 INFO:teuthology.orchestra.run.smithi072.stdout: "osdmap_mapping_items": 0, 2024-04-26T21:59:07.920 INFO:teuthology.orchestra.run.smithi072.stdout: "pgmap_bytes": 0, 2024-04-26T21:59:07.921 INFO:teuthology.orchestra.run.smithi072.stdout: "pgmap_items": 0, 2024-04-26T21:59:07.921 INFO:teuthology.orchestra.run.smithi072.stdout: "mds_co_bytes": 8512, 2024-04-26T21:59:07.921 INFO:teuthology.orchestra.run.smithi072.stdout: "mds_co_items": 13, 2024-04-26T21:59:07.921 INFO:teuthology.orchestra.run.smithi072.stdout: "unittest_1_bytes": 0, 2024-04-26T21:59:07.921 INFO:teuthology.orchestra.run.smithi072.stdout: "unittest_1_items": 0, 2024-04-26T21:59:07.921 INFO:teuthology.orchestra.run.smithi072.stdout: "unittest_2_bytes": 0, 2024-04-26T21:59:07.921 INFO:teuthology.orchestra.run.smithi072.stdout: "unittest_2_items": 0 2024-04-26T21:59:07.921 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:59:07.921 INFO:teuthology.orchestra.run.smithi072.stdout: "objecter": { 2024-04-26T21:59:07.921 INFO:teuthology.orchestra.run.smithi072.stdout: "op_active": 0, 2024-04-26T21:59:07.921 INFO:teuthology.orchestra.run.smithi072.stdout: "op_laggy": 0, 2024-04-26T21:59:07.921 INFO:teuthology.orchestra.run.smithi072.stdout: "op_send": 972, 2024-04-26T21:59:07.921 INFO:teuthology.orchestra.run.smithi072.stdout: "op_send_bytes": 21, 2024-04-26T21:59:07.921 INFO:teuthology.orchestra.run.smithi072.stdout: "op_resend": 3, 2024-04-26T21:59:07.921 INFO:teuthology.orchestra.run.smithi072.stdout: "op_reply": 972, 2024-04-26T21:59:07.922 INFO:teuthology.orchestra.run.smithi072.stdout: "op_latency": { 2024-04-26T21:59:07.922 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 972, 2024-04-26T21:59:07.922 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 1.052978865, 2024-04-26T21:59:07.922 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.001083311 2024-04-26T21:59:07.922 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:59:07.922 INFO:teuthology.orchestra.run.smithi072.stdout: "op_inflight": 0, 2024-04-26T21:59:07.922 INFO:teuthology.orchestra.run.smithi072.stdout: "oplen_avg": { 2024-04-26T21:59:07.922 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 972, 2024-04-26T21:59:07.922 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 973 2024-04-26T21:59:07.922 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:59:07.922 INFO:teuthology.orchestra.run.smithi072.stdout: "op": 972, 2024-04-26T21:59:07.922 INFO:teuthology.orchestra.run.smithi072.stdout: "op_r": 972, 2024-04-26T21:59:07.922 INFO:teuthology.orchestra.run.smithi072.stdout: "op_w": 0, 2024-04-26T21:59:07.923 INFO:teuthology.orchestra.run.smithi072.stdout: "op_rmw": 0, 2024-04-26T21:59:07.923 INFO:teuthology.orchestra.run.smithi072.stdout: "op_pg": 0, 2024-04-26T21:59:07.923 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_stat": 596, 2024-04-26T21:59:07.923 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_create": 0, 2024-04-26T21:59:07.923 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_read": 374, 2024-04-26T21:59:07.923 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_write": 0, 2024-04-26T21:59:07.923 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_writefull": 0, 2024-04-26T21:59:07.923 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_writesame": 0, 2024-04-26T21:59:07.923 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_append": 0, 2024-04-26T21:59:07.923 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_zero": 0, 2024-04-26T21:59:07.923 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_truncate": 0, 2024-04-26T21:59:07.923 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_delete": 0, 2024-04-26T21:59:07.923 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_mapext": 0, 2024-04-26T21:59:07.923 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_sparse_read": 0, 2024-04-26T21:59:07.924 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_clonerange": 0, 2024-04-26T21:59:07.924 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_getxattr": 1, 2024-04-26T21:59:07.924 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_setxattr": 0, 2024-04-26T21:59:07.924 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_cmpxattr": 0, 2024-04-26T21:59:07.924 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_rmxattr": 0, 2024-04-26T21:59:07.924 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_resetxattrs": 0, 2024-04-26T21:59:07.924 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_call": 0, 2024-04-26T21:59:07.924 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_watch": 0, 2024-04-26T21:59:07.924 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_notify": 0, 2024-04-26T21:59:07.924 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_src_cmpxattr": 0, 2024-04-26T21:59:07.924 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_pgls": 0, 2024-04-26T21:59:07.924 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_pgls_filter": 0, 2024-04-26T21:59:07.924 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_other": 0, 2024-04-26T21:59:07.924 INFO:teuthology.orchestra.run.smithi072.stdout: "linger_active": 0, 2024-04-26T21:59:07.925 INFO:teuthology.orchestra.run.smithi072.stdout: "linger_send": 0, 2024-04-26T21:59:07.925 INFO:teuthology.orchestra.run.smithi072.stdout: "linger_resend": 0, 2024-04-26T21:59:07.925 INFO:teuthology.orchestra.run.smithi072.stdout: "linger_ping": 0, 2024-04-26T21:59:07.925 INFO:teuthology.orchestra.run.smithi072.stdout: "poolop_active": 0, 2024-04-26T21:59:07.925 INFO:teuthology.orchestra.run.smithi072.stdout: "poolop_send": 0, 2024-04-26T21:59:07.925 INFO:teuthology.orchestra.run.smithi072.stdout: "poolop_resend": 0, 2024-04-26T21:59:07.925 INFO:teuthology.orchestra.run.smithi072.stdout: "poolstat_active": 0, 2024-04-26T21:59:07.925 INFO:teuthology.orchestra.run.smithi072.stdout: "poolstat_send": 0, 2024-04-26T21:59:07.925 INFO:teuthology.orchestra.run.smithi072.stdout: "poolstat_resend": 0, 2024-04-26T21:59:07.925 INFO:teuthology.orchestra.run.smithi072.stdout: "statfs_active": 0, 2024-04-26T21:59:07.925 INFO:teuthology.orchestra.run.smithi072.stdout: "statfs_send": 0, 2024-04-26T21:59:07.925 INFO:teuthology.orchestra.run.smithi072.stdout: "statfs_resend": 0, 2024-04-26T21:59:07.925 INFO:teuthology.orchestra.run.smithi072.stdout: "command_active": 0, 2024-04-26T21:59:07.926 INFO:teuthology.orchestra.run.smithi072.stdout: "command_send": 0, 2024-04-26T21:59:07.926 INFO:teuthology.orchestra.run.smithi072.stdout: "command_resend": 0, 2024-04-26T21:59:07.926 INFO:teuthology.orchestra.run.smithi072.stdout: "map_epoch": 91, 2024-04-26T21:59:07.926 INFO:teuthology.orchestra.run.smithi072.stdout: "map_full": 0, 2024-04-26T21:59:07.926 INFO:teuthology.orchestra.run.smithi072.stdout: "map_inc": 4, 2024-04-26T21:59:07.926 INFO:teuthology.orchestra.run.smithi072.stdout: "osd_sessions": 6, 2024-04-26T21:59:07.926 INFO:teuthology.orchestra.run.smithi072.stdout: "osd_session_open": 6, 2024-04-26T21:59:07.926 INFO:teuthology.orchestra.run.smithi072.stdout: "osd_session_close": 0, 2024-04-26T21:59:07.926 INFO:teuthology.orchestra.run.smithi072.stdout: "osd_laggy": 0, 2024-04-26T21:59:07.926 INFO:teuthology.orchestra.run.smithi072.stdout: "omap_wr": 0, 2024-04-26T21:59:07.926 INFO:teuthology.orchestra.run.smithi072.stdout: "omap_rd": 2, 2024-04-26T21:59:07.926 INFO:teuthology.orchestra.run.smithi072.stdout: "omap_del": 0 2024-04-26T21:59:07.926 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:59:07.926 INFO:teuthology.orchestra.run.smithi072.stdout: "oft": { 2024-04-26T21:59:07.927 INFO:teuthology.orchestra.run.smithi072.stdout: "omap_total_objs": 0, 2024-04-26T21:59:07.927 INFO:teuthology.orchestra.run.smithi072.stdout: "omap_total_kv_pairs": 0, 2024-04-26T21:59:07.927 INFO:teuthology.orchestra.run.smithi072.stdout: "omap_total_updates": 0, 2024-04-26T21:59:07.927 INFO:teuthology.orchestra.run.smithi072.stdout: "omap_total_removes": 0 2024-04-26T21:59:07.927 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:59:07.927 INFO:teuthology.orchestra.run.smithi072.stdout: "osd-slow-ops": { 2024-04-26T21:59:07.927 INFO:teuthology.orchestra.run.smithi072.stdout: "slow_ops_count": 0 2024-04-26T21:59:07.927 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:59:07.927 INFO:teuthology.orchestra.run.smithi072.stdout: "purge_queue": { 2024-04-26T21:59:07.927 INFO:teuthology.orchestra.run.smithi072.stdout: "pq_executing_ops": 0, 2024-04-26T21:59:07.927 INFO:teuthology.orchestra.run.smithi072.stdout: "pq_executing_ops_high_water": 0, 2024-04-26T21:59:07.927 INFO:teuthology.orchestra.run.smithi072.stdout: "pq_executing": 0, 2024-04-26T21:59:07.927 INFO:teuthology.orchestra.run.smithi072.stdout: "pq_executing_high_water": 0, 2024-04-26T21:59:07.927 INFO:teuthology.orchestra.run.smithi072.stdout: "pq_executed_ops": 0, 2024-04-26T21:59:07.927 INFO:teuthology.orchestra.run.smithi072.stdout: "pq_executed": 0, 2024-04-26T21:59:07.928 INFO:teuthology.orchestra.run.smithi072.stdout: "pq_item_in_journal": 0 2024-04-26T21:59:07.928 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:59:07.928 INFO:teuthology.orchestra.run.smithi072.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-26T21:59:07.928 INFO:teuthology.orchestra.run.smithi072.stdout: "val": 0, 2024-04-26T21:59:07.928 INFO:teuthology.orchestra.run.smithi072.stdout: "max": 104857600, 2024-04-26T21:59:07.928 INFO:teuthology.orchestra.run.smithi072.stdout: "get_started": 0, 2024-04-26T21:59:07.928 INFO:teuthology.orchestra.run.smithi072.stdout: "get": 1093, 2024-04-26T21:59:07.928 INFO:teuthology.orchestra.run.smithi072.stdout: "get_sum": 510857, 2024-04-26T21:59:07.928 INFO:teuthology.orchestra.run.smithi072.stdout: "get_or_fail_fail": 0, 2024-04-26T21:59:07.928 INFO:teuthology.orchestra.run.smithi072.stdout: "get_or_fail_success": 1093, 2024-04-26T21:59:07.928 INFO:teuthology.orchestra.run.smithi072.stdout: "take": 0, 2024-04-26T21:59:07.928 INFO:teuthology.orchestra.run.smithi072.stdout: "take_sum": 0, 2024-04-26T21:59:07.928 INFO:teuthology.orchestra.run.smithi072.stdout: "put": 1093, 2024-04-26T21:59:07.928 INFO:teuthology.orchestra.run.smithi072.stdout: "put_sum": 510857, 2024-04-26T21:59:07.929 INFO:teuthology.orchestra.run.smithi072.stdout: "wait": { 2024-04-26T21:59:07.929 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-04-26T21:59:07.929 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-04-26T21:59:07.929 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-04-26T21:59:07.929 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:59:07.929 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:59:07.929 INFO:teuthology.orchestra.run.smithi072.stdout: "throttle-objecter_bytes": { 2024-04-26T21:59:07.929 INFO:teuthology.orchestra.run.smithi072.stdout: "val": 0, 2024-04-26T21:59:07.929 INFO:teuthology.orchestra.run.smithi072.stdout: "max": 104857600, 2024-04-26T21:59:07.929 INFO:teuthology.orchestra.run.smithi072.stdout: "get_started": 0, 2024-04-26T21:59:07.929 INFO:teuthology.orchestra.run.smithi072.stdout: "get": 0, 2024-04-26T21:59:07.929 INFO:teuthology.orchestra.run.smithi072.stdout: "get_sum": 0, 2024-04-26T21:59:07.929 INFO:teuthology.orchestra.run.smithi072.stdout: "get_or_fail_fail": 0, 2024-04-26T21:59:07.929 INFO:teuthology.orchestra.run.smithi072.stdout: "get_or_fail_success": 0, 2024-04-26T21:59:07.930 INFO:teuthology.orchestra.run.smithi072.stdout: "take": 972, 2024-04-26T21:59:07.930 INFO:teuthology.orchestra.run.smithi072.stdout: "take_sum": 1086, 2024-04-26T21:59:07.930 INFO:teuthology.orchestra.run.smithi072.stdout: "put": 972, 2024-04-26T21:59:07.930 INFO:teuthology.orchestra.run.smithi072.stdout: "put_sum": 1086, 2024-04-26T21:59:07.930 INFO:teuthology.orchestra.run.smithi072.stdout: "wait": { 2024-04-26T21:59:07.930 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-04-26T21:59:07.930 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-04-26T21:59:07.930 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-04-26T21:59:07.930 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:59:07.930 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:59:07.930 INFO:teuthology.orchestra.run.smithi072.stdout: "throttle-objecter_ops": { 2024-04-26T21:59:07.930 INFO:teuthology.orchestra.run.smithi072.stdout: "val": 0, 2024-04-26T21:59:07.930 INFO:teuthology.orchestra.run.smithi072.stdout: "max": 1024, 2024-04-26T21:59:07.930 INFO:teuthology.orchestra.run.smithi072.stdout: "get_started": 0, 2024-04-26T21:59:07.930 INFO:teuthology.orchestra.run.smithi072.stdout: "get": 0, 2024-04-26T21:59:07.931 INFO:teuthology.orchestra.run.smithi072.stdout: "get_sum": 0, 2024-04-26T21:59:07.931 INFO:teuthology.orchestra.run.smithi072.stdout: "get_or_fail_fail": 0, 2024-04-26T21:59:07.931 INFO:teuthology.orchestra.run.smithi072.stdout: "get_or_fail_success": 0, 2024-04-26T21:59:07.931 INFO:teuthology.orchestra.run.smithi072.stdout: "take": 972, 2024-04-26T21:59:07.931 INFO:teuthology.orchestra.run.smithi072.stdout: "take_sum": 972, 2024-04-26T21:59:07.931 INFO:teuthology.orchestra.run.smithi072.stdout: "put": 972, 2024-04-26T21:59:07.931 INFO:teuthology.orchestra.run.smithi072.stdout: "put_sum": 972, 2024-04-26T21:59:07.931 INFO:teuthology.orchestra.run.smithi072.stdout: "wait": { 2024-04-26T21:59:07.931 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-04-26T21:59:07.931 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-04-26T21:59:07.931 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-04-26T21:59:07.931 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:59:07.931 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:59:07.931 INFO:teuthology.orchestra.run.smithi072.stdout: "throttle-write_buf_throttle": { 2024-04-26T21:59:07.931 INFO:teuthology.orchestra.run.smithi072.stdout: "val": 0, 2024-04-26T21:59:07.932 INFO:teuthology.orchestra.run.smithi072.stdout: "max": 3758096384, 2024-04-26T21:59:07.932 INFO:teuthology.orchestra.run.smithi072.stdout: "get_started": 0, 2024-04-26T21:59:07.932 INFO:teuthology.orchestra.run.smithi072.stdout: "get": 0, 2024-04-26T21:59:07.932 INFO:teuthology.orchestra.run.smithi072.stdout: "get_sum": 0, 2024-04-26T21:59:07.932 INFO:teuthology.orchestra.run.smithi072.stdout: "get_or_fail_fail": 0, 2024-04-26T21:59:07.932 INFO:teuthology.orchestra.run.smithi072.stdout: "get_or_fail_success": 0, 2024-04-26T21:59:07.932 INFO:teuthology.orchestra.run.smithi072.stdout: "take": 0, 2024-04-26T21:59:07.932 INFO:teuthology.orchestra.run.smithi072.stdout: "take_sum": 0, 2024-04-26T21:59:07.932 INFO:teuthology.orchestra.run.smithi072.stdout: "put": 0, 2024-04-26T21:59:07.932 INFO:teuthology.orchestra.run.smithi072.stdout: "put_sum": 0, 2024-04-26T21:59:07.932 INFO:teuthology.orchestra.run.smithi072.stdout: "wait": { 2024-04-26T21:59:07.932 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-04-26T21:59:07.932 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-04-26T21:59:07.932 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-04-26T21:59:07.933 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:59:07.933 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:59:07.933 INFO:teuthology.orchestra.run.smithi072.stdout: "throttle-write_buf_throttle-0x565171b6a780": { 2024-04-26T21:59:07.933 INFO:teuthology.orchestra.run.smithi072.stdout: "val": 0, 2024-04-26T21:59:07.933 INFO:teuthology.orchestra.run.smithi072.stdout: "max": 3758096384, 2024-04-26T21:59:07.933 INFO:teuthology.orchestra.run.smithi072.stdout: "get_started": 0, 2024-04-26T21:59:07.933 INFO:teuthology.orchestra.run.smithi072.stdout: "get": 0, 2024-04-26T21:59:07.933 INFO:teuthology.orchestra.run.smithi072.stdout: "get_sum": 0, 2024-04-26T21:59:07.933 INFO:teuthology.orchestra.run.smithi072.stdout: "get_or_fail_fail": 0, 2024-04-26T21:59:07.933 INFO:teuthology.orchestra.run.smithi072.stdout: "get_or_fail_success": 0, 2024-04-26T21:59:07.933 INFO:teuthology.orchestra.run.smithi072.stdout: "take": 0, 2024-04-26T21:59:07.933 INFO:teuthology.orchestra.run.smithi072.stdout: "take_sum": 0, 2024-04-26T21:59:07.933 INFO:teuthology.orchestra.run.smithi072.stdout: "put": 0, 2024-04-26T21:59:07.933 INFO:teuthology.orchestra.run.smithi072.stdout: "put_sum": 0, 2024-04-26T21:59:07.934 INFO:teuthology.orchestra.run.smithi072.stdout: "wait": { 2024-04-26T21:59:07.934 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-04-26T21:59:07.934 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-04-26T21:59:07.934 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-04-26T21:59:07.934 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:59:07.934 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:59:07.934 INFO:teuthology.orchestra.run.smithi072.stdout:} 2024-04-26T21:59:08.975 INFO:tasks.check_counter:Daemon mds.g mds.exported=0 2024-04-26T21:59:08.975 INFO:tasks.check_counter:Daemon mds.g mds.imported=0 2024-04-26T21:59:08.975 DEBUG:tasks.check_counter:Getting stats from j 2024-04-26T21:59:08.976 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid a1780950-0415-11ef-bc93-c7b262605968 -- ceph daemon mds.j perf dump 2024-04-26T21:59:09.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:59:09 smithi076 ceph-mon[31823]: pgmap v560: 97 pgs: 97 active+clean; 1.1 MiB data, 656 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 2.0 KiB/s wr, 13 op/s 2024-04-26T21:59:09.321 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:59:09 smithi072 ceph-mon[28138]: pgmap v560: 97 pgs: 97 active+clean; 1.1 MiB data, 656 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 2.0 KiB/s wr, 13 op/s 2024-04-26T21:59:09.413 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/a1780950-0415-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:59:09.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:59:09 smithi164 ceph-mon[31916]: pgmap v560: 97 pgs: 97 active+clean; 1.1 MiB data, 656 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 2.0 KiB/s wr, 13 op/s 2024-04-26T21:59:11.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:59:11 smithi076 ceph-mon[31823]: pgmap v561: 97 pgs: 97 active+clean; 1.1 MiB data, 656 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 2.2 KiB/s wr, 16 op/s 2024-04-26T21:59:11.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:59:11 smithi072 ceph-mon[28138]: pgmap v561: 97 pgs: 97 active+clean; 1.1 MiB data, 656 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 2.2 KiB/s wr, 16 op/s 2024-04-26T21:59:11.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:59:11 smithi164 ceph-mon[31916]: pgmap v561: 97 pgs: 97 active+clean; 1.1 MiB data, 656 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 2.2 KiB/s wr, 16 op/s 2024-04-26T21:59:11.948 INFO:teuthology.orchestra.run.smithi072.stdout:{ 2024-04-26T21:59:11.948 INFO:teuthology.orchestra.run.smithi072.stdout: "AsyncMessenger::Worker-0": { 2024-04-26T21:59:11.948 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_recv_messages": 0, 2024-04-26T21:59:11.948 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_messages": 2, 2024-04-26T21:59:11.948 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_recv_bytes": 0, 2024-04-26T21:59:11.948 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_bytes": 0, 2024-04-26T21:59:11.948 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_created_connections": 4, 2024-04-26T21:59:11.948 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_active_connections": 0, 2024-04-26T21:59:11.948 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_total_time": 0.003864795, 2024-04-26T21:59:11.948 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_send_time": 0.000000000, 2024-04-26T21:59:11.949 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_recv_time": 0.000468567, 2024-04-26T21:59:11.949 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-26T21:59:11.949 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T21:59:11.949 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-04-26T21:59:11.949 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-04-26T21:59:11.949 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-04-26T21:59:11.949 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:59:11.949 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_handle_ack_lat": { 2024-04-26T21:59:11.949 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-04-26T21:59:11.949 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-04-26T21:59:11.949 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-04-26T21:59:11.949 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:59:11.949 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-26T21:59:11.949 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_encrypted_bytes": 0 2024-04-26T21:59:11.950 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:59:11.950 INFO:teuthology.orchestra.run.smithi072.stdout: "AsyncMessenger::Worker-1": { 2024-04-26T21:59:11.950 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_recv_messages": 5, 2024-04-26T21:59:11.950 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_messages": 90, 2024-04-26T21:59:11.950 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_recv_bytes": 3072, 2024-04-26T21:59:11.950 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_bytes": 94304, 2024-04-26T21:59:11.950 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_created_connections": 3, 2024-04-26T21:59:11.950 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_active_connections": 2, 2024-04-26T21:59:11.950 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_total_time": 0.015251387, 2024-04-26T21:59:11.950 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_send_time": 0.011134640, 2024-04-26T21:59:11.950 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_recv_time": 0.002361715, 2024-04-26T21:59:11.950 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-26T21:59:11.950 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T21:59:11.951 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 89, 2024-04-26T21:59:11.951 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.008562877, 2024-04-26T21:59:11.951 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000096212 2024-04-26T21:59:11.951 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:59:11.951 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_handle_ack_lat": { 2024-04-26T21:59:11.951 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-04-26T21:59:11.951 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-04-26T21:59:11.951 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-04-26T21:59:11.951 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:59:11.951 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_recv_encrypted_bytes": 3072, 2024-04-26T21:59:11.951 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_encrypted_bytes": 94304 2024-04-26T21:59:11.951 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:59:11.951 INFO:teuthology.orchestra.run.smithi072.stdout: "AsyncMessenger::Worker-2": { 2024-04-26T21:59:11.951 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_recv_messages": 118, 2024-04-26T21:59:11.951 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_messages": 116, 2024-04-26T21:59:11.952 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_recv_bytes": 313152, 2024-04-26T21:59:11.952 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_bytes": 70448, 2024-04-26T21:59:11.952 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_created_connections": 2, 2024-04-26T21:59:11.952 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_active_connections": 1, 2024-04-26T21:59:11.952 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_total_time": 0.041370383, 2024-04-26T21:59:11.952 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_send_time": 0.016400232, 2024-04-26T21:59:11.952 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_recv_time": 0.023667885, 2024-04-26T21:59:11.952 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_fast_dispatch_time": 0.004824584, 2024-04-26T21:59:11.952 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T21:59:11.952 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 115, 2024-04-26T21:59:11.952 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.010805210, 2024-04-26T21:59:11.952 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000093958 2024-04-26T21:59:11.952 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:59:11.952 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_handle_ack_lat": { 2024-04-26T21:59:11.952 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-04-26T21:59:11.953 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-04-26T21:59:11.953 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-04-26T21:59:11.953 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:59:11.953 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_recv_encrypted_bytes": 313152, 2024-04-26T21:59:11.953 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_encrypted_bytes": 70448 2024-04-26T21:59:11.953 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:59:11.953 INFO:teuthology.orchestra.run.smithi072.stdout: "cct": { 2024-04-26T21:59:11.953 INFO:teuthology.orchestra.run.smithi072.stdout: "total_workers": 0, 2024-04-26T21:59:11.953 INFO:teuthology.orchestra.run.smithi072.stdout: "unhealthy_workers": 0 2024-04-26T21:59:11.953 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:59:11.953 INFO:teuthology.orchestra.run.smithi072.stdout: "mempool": { 2024-04-26T21:59:11.953 INFO:teuthology.orchestra.run.smithi072.stdout: "bloom_filter_bytes": 0, 2024-04-26T21:59:11.953 INFO:teuthology.orchestra.run.smithi072.stdout: "bloom_filter_items": 0, 2024-04-26T21:59:11.953 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_alloc_bytes": 0, 2024-04-26T21:59:11.953 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_alloc_items": 0, 2024-04-26T21:59:11.954 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_data_bytes": 0, 2024-04-26T21:59:11.954 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_data_items": 0, 2024-04-26T21:59:11.954 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-26T21:59:11.954 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_onode_items": 0, 2024-04-26T21:59:11.954 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-26T21:59:11.954 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_meta_items": 0, 2024-04-26T21:59:11.954 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_other_bytes": 0, 2024-04-26T21:59:11.954 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_other_items": 0, 2024-04-26T21:59:11.954 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-26T21:59:11.954 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_buffer_items": 0, 2024-04-26T21:59:11.954 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_extent_bytes": 0, 2024-04-26T21:59:11.954 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_extent_items": 0, 2024-04-26T21:59:11.954 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_blob_bytes": 0, 2024-04-26T21:59:11.955 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_blob_items": 0, 2024-04-26T21:59:11.955 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-26T21:59:11.955 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_shared_blob_items": 0, 2024-04-26T21:59:11.955 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-26T21:59:11.955 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_inline_bl_items": 0, 2024-04-26T21:59:11.955 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_fsck_bytes": 0, 2024-04-26T21:59:11.955 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_fsck_items": 0, 2024-04-26T21:59:11.955 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_txc_bytes": 0, 2024-04-26T21:59:11.955 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_txc_items": 0, 2024-04-26T21:59:11.955 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-26T21:59:11.955 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_writing_deferred_items": 0, 2024-04-26T21:59:11.955 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_writing_bytes": 0, 2024-04-26T21:59:11.955 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_writing_items": 0, 2024-04-26T21:59:11.955 INFO:teuthology.orchestra.run.smithi072.stdout: "bluefs_bytes": 0, 2024-04-26T21:59:11.955 INFO:teuthology.orchestra.run.smithi072.stdout: "bluefs_items": 0, 2024-04-26T21:59:11.956 INFO:teuthology.orchestra.run.smithi072.stdout: "bluefs_file_reader_bytes": 0, 2024-04-26T21:59:11.956 INFO:teuthology.orchestra.run.smithi072.stdout: "bluefs_file_reader_items": 0, 2024-04-26T21:59:11.956 INFO:teuthology.orchestra.run.smithi072.stdout: "bluefs_file_writer_bytes": 0, 2024-04-26T21:59:11.956 INFO:teuthology.orchestra.run.smithi072.stdout: "bluefs_file_writer_items": 0, 2024-04-26T21:59:11.956 INFO:teuthology.orchestra.run.smithi072.stdout: "buffer_anon_bytes": 148857, 2024-04-26T21:59:11.956 INFO:teuthology.orchestra.run.smithi072.stdout: "buffer_anon_items": 29, 2024-04-26T21:59:11.956 INFO:teuthology.orchestra.run.smithi072.stdout: "buffer_meta_bytes": 0, 2024-04-26T21:59:11.956 INFO:teuthology.orchestra.run.smithi072.stdout: "buffer_meta_items": 0, 2024-04-26T21:59:11.956 INFO:teuthology.orchestra.run.smithi072.stdout: "osd_bytes": 0, 2024-04-26T21:59:11.956 INFO:teuthology.orchestra.run.smithi072.stdout: "osd_items": 0, 2024-04-26T21:59:11.956 INFO:teuthology.orchestra.run.smithi072.stdout: "osd_mapbl_bytes": 0, 2024-04-26T21:59:11.956 INFO:teuthology.orchestra.run.smithi072.stdout: "osd_mapbl_items": 0, 2024-04-26T21:59:11.956 INFO:teuthology.orchestra.run.smithi072.stdout: "osd_pglog_bytes": 0, 2024-04-26T21:59:11.957 INFO:teuthology.orchestra.run.smithi072.stdout: "osd_pglog_items": 0, 2024-04-26T21:59:11.957 INFO:teuthology.orchestra.run.smithi072.stdout: "osdmap_bytes": 0, 2024-04-26T21:59:11.957 INFO:teuthology.orchestra.run.smithi072.stdout: "osdmap_items": 0, 2024-04-26T21:59:11.957 INFO:teuthology.orchestra.run.smithi072.stdout: "osdmap_mapping_bytes": 0, 2024-04-26T21:59:11.957 INFO:teuthology.orchestra.run.smithi072.stdout: "osdmap_mapping_items": 0, 2024-04-26T21:59:11.957 INFO:teuthology.orchestra.run.smithi072.stdout: "pgmap_bytes": 0, 2024-04-26T21:59:11.957 INFO:teuthology.orchestra.run.smithi072.stdout: "pgmap_items": 0, 2024-04-26T21:59:11.957 INFO:teuthology.orchestra.run.smithi072.stdout: "mds_co_bytes": 928, 2024-04-26T21:59:11.957 INFO:teuthology.orchestra.run.smithi072.stdout: "mds_co_items": 2, 2024-04-26T21:59:11.957 INFO:teuthology.orchestra.run.smithi072.stdout: "unittest_1_bytes": 0, 2024-04-26T21:59:11.957 INFO:teuthology.orchestra.run.smithi072.stdout: "unittest_1_items": 0, 2024-04-26T21:59:11.957 INFO:teuthology.orchestra.run.smithi072.stdout: "unittest_2_bytes": 0, 2024-04-26T21:59:11.958 INFO:teuthology.orchestra.run.smithi072.stdout: "unittest_2_items": 0 2024-04-26T21:59:11.958 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T21:59:11.958 INFO:teuthology.orchestra.run.smithi072.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-26T21:59:11.958 INFO:teuthology.orchestra.run.smithi072.stdout: "val": 0, 2024-04-26T21:59:11.958 INFO:teuthology.orchestra.run.smithi072.stdout: "max": 104857600, 2024-04-26T21:59:11.958 INFO:teuthology.orchestra.run.smithi072.stdout: "get_started": 0, 2024-04-26T21:59:11.958 INFO:teuthology.orchestra.run.smithi072.stdout: "get": 119, 2024-04-26T21:59:11.958 INFO:teuthology.orchestra.run.smithi072.stdout: "get_sum": 296499, 2024-04-26T21:59:11.958 INFO:teuthology.orchestra.run.smithi072.stdout: "get_or_fail_fail": 0, 2024-04-26T21:59:11.958 INFO:teuthology.orchestra.run.smithi072.stdout: "get_or_fail_success": 119, 2024-04-26T21:59:11.958 INFO:teuthology.orchestra.run.smithi072.stdout: "take": 0, 2024-04-26T21:59:11.958 INFO:teuthology.orchestra.run.smithi072.stdout: "take_sum": 0, 2024-04-26T21:59:11.958 INFO:teuthology.orchestra.run.smithi072.stdout: "put": 119, 2024-04-26T21:59:11.958 INFO:teuthology.orchestra.run.smithi072.stdout: "put_sum": 296499, 2024-04-26T21:59:11.959 INFO:teuthology.orchestra.run.smithi072.stdout: "wait": { 2024-04-26T21:59:11.959 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-04-26T21:59:11.959 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-04-26T21:59:11.959 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-04-26T21:59:11.959 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:59:11.959 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T21:59:11.959 INFO:teuthology.orchestra.run.smithi072.stdout:} 2024-04-26T21:59:12.919 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-04-26T21:59:12.919 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-04-26T21:59:12.919 DEBUG:tasks.check_counter:Getting stats from b 2024-04-26T21:59:12.920 DEBUG:teuthology.orchestra.run.smithi076:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid a1780950-0415-11ef-bc93-c7b262605968 -- ceph daemon mds.b perf dump 2024-04-26T21:59:13.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:59:13 smithi076 ceph-mon[31823]: pgmap v562: 97 pgs: 97 active+clean; 1.1 MiB data, 656 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 426 B/s wr, 10 op/s 2024-04-26T21:59:13.321 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:59:13 smithi072 ceph-mon[28138]: pgmap v562: 97 pgs: 97 active+clean; 1.1 MiB data, 656 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 426 B/s wr, 10 op/s 2024-04-26T21:59:13.368 INFO:teuthology.orchestra.run.smithi076.stderr:Inferring config /var/lib/ceph/a1780950-0415-11ef-bc93-c7b262605968/mon.b/config 2024-04-26T21:59:13.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:59:13 smithi164 ceph-mon[31916]: pgmap v562: 97 pgs: 97 active+clean; 1.1 MiB data, 656 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 426 B/s wr, 10 op/s 2024-04-26T21:59:15.203 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:59:15 smithi076 ceph-mon[31823]: pgmap v563: 97 pgs: 97 active+clean; 1.1 MiB data, 656 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 426 B/s wr, 12 op/s 2024-04-26T21:59:15.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:59:15 smithi072 ceph-mon[28138]: pgmap v563: 97 pgs: 97 active+clean; 1.1 MiB data, 656 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 426 B/s wr, 12 op/s 2024-04-26T21:59:15.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:59:15 smithi164 ceph-mon[31916]: pgmap v563: 97 pgs: 97 active+clean; 1.1 MiB data, 656 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 426 B/s wr, 12 op/s 2024-04-26T21:59:15.847 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-04-26T21:59:15.847 INFO:teuthology.orchestra.run.smithi076.stdout: "AsyncMessenger::Worker-0": { 2024-04-26T21:59:15.847 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_messages": 724, 2024-04-26T21:59:15.847 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_messages": 528, 2024-04-26T21:59:15.847 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_bytes": 237146, 2024-04-26T21:59:15.847 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_bytes": 767619, 2024-04-26T21:59:15.847 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_created_connections": 11, 2024-04-26T21:59:15.847 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_active_connections": 4, 2024-04-26T21:59:15.847 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_total_time": 0.194474396, 2024-04-26T21:59:15.847 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_send_time": 0.077817334, 2024-04-26T21:59:15.848 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_recv_time": 0.064581667, 2024-04-26T21:59:15.848 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_fast_dispatch_time": 0.033301835, 2024-04-26T21:59:15.848 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T21:59:15.848 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 527, 2024-04-26T21:59:15.848 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.100891025, 2024-04-26T21:59:15.848 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000191444 2024-04-26T21:59:15.848 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:15.848 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_handle_ack_lat": { 2024-04-26T21:59:15.848 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 799, 2024-04-26T21:59:15.848 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000314033, 2024-04-26T21:59:15.848 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000393 2024-04-26T21:59:15.848 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:15.848 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_encrypted_bytes": 2912, 2024-04-26T21:59:15.848 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_encrypted_bytes": 448 2024-04-26T21:59:15.849 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:15.849 INFO:teuthology.orchestra.run.smithi076.stdout: "AsyncMessenger::Worker-1": { 2024-04-26T21:59:15.849 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_messages": 883, 2024-04-26T21:59:15.849 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_messages": 591, 2024-04-26T21:59:15.849 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_bytes": 679632, 2024-04-26T21:59:15.849 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_bytes": 501563, 2024-04-26T21:59:15.849 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_created_connections": 33, 2024-04-26T21:59:15.849 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_active_connections": 6, 2024-04-26T21:59:15.849 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_total_time": 0.249601129, 2024-04-26T21:59:15.849 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_send_time": 0.091060612, 2024-04-26T21:59:15.849 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_recv_time": 0.126287144, 2024-04-26T21:59:15.849 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_fast_dispatch_time": 0.041243447, 2024-04-26T21:59:15.849 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T21:59:15.849 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 590, 2024-04-26T21:59:15.849 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.091046877, 2024-04-26T21:59:15.850 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000154316 2024-04-26T21:59:15.850 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:15.850 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_handle_ack_lat": { 2024-04-26T21:59:15.850 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 947, 2024-04-26T21:59:15.850 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000390338, 2024-04-26T21:59:15.850 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000412 2024-04-26T21:59:15.850 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:15.850 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_encrypted_bytes": 368544, 2024-04-26T21:59:15.850 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_encrypted_bytes": 80032 2024-04-26T21:59:15.850 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:15.850 INFO:teuthology.orchestra.run.smithi076.stdout: "AsyncMessenger::Worker-2": { 2024-04-26T21:59:15.850 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_messages": 796, 2024-04-26T21:59:15.850 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_messages": 409, 2024-04-26T21:59:15.851 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_bytes": 305509, 2024-04-26T21:59:15.851 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_bytes": 630331, 2024-04-26T21:59:15.851 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_created_connections": 38, 2024-04-26T21:59:15.851 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_active_connections": 8, 2024-04-26T21:59:15.853 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_total_time": 0.238216050, 2024-04-26T21:59:15.854 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_send_time": 0.083766154, 2024-04-26T21:59:15.854 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_recv_time": 0.101295336, 2024-04-26T21:59:15.854 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_fast_dispatch_time": 0.048893309, 2024-04-26T21:59:15.854 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T21:59:15.854 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 407, 2024-04-26T21:59:15.854 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.064586635, 2024-04-26T21:59:15.854 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000158689 2024-04-26T21:59:15.854 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:15.854 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_handle_ack_lat": { 2024-04-26T21:59:15.854 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 939, 2024-04-26T21:59:15.854 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000652331, 2024-04-26T21:59:15.854 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000694 2024-04-26T21:59:15.854 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:15.855 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_encrypted_bytes": 191824, 2024-04-26T21:59:15.855 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_encrypted_bytes": 225552 2024-04-26T21:59:15.855 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:15.855 INFO:teuthology.orchestra.run.smithi076.stdout: "cct": { 2024-04-26T21:59:15.855 INFO:teuthology.orchestra.run.smithi076.stdout: "total_workers": 1, 2024-04-26T21:59:15.855 INFO:teuthology.orchestra.run.smithi076.stdout: "unhealthy_workers": 0 2024-04-26T21:59:15.855 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:15.855 INFO:teuthology.orchestra.run.smithi076.stdout: "finisher-MDSRank": { 2024-04-26T21:59:15.855 INFO:teuthology.orchestra.run.smithi076.stdout: "queue_len": 0, 2024-04-26T21:59:15.855 INFO:teuthology.orchestra.run.smithi076.stdout: "complete_latency": { 2024-04-26T21:59:15.855 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 340, 2024-04-26T21:59:15.855 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.214769036, 2024-04-26T21:59:15.855 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000631673 2024-04-26T21:59:15.855 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-26T21:59:15.856 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:15.856 INFO:teuthology.orchestra.run.smithi076.stdout: "finisher-PurgeQueue": { 2024-04-26T21:59:15.856 INFO:teuthology.orchestra.run.smithi076.stdout: "queue_len": 0, 2024-04-26T21:59:15.856 INFO:teuthology.orchestra.run.smithi076.stdout: "complete_latency": { 2024-04-26T21:59:15.856 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 90, 2024-04-26T21:59:15.856 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.008995985, 2024-04-26T21:59:15.856 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000099955 2024-04-26T21:59:15.856 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-26T21:59:15.856 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:15.856 INFO:teuthology.orchestra.run.smithi076.stdout: "mds": { 2024-04-26T21:59:15.856 INFO:teuthology.orchestra.run.smithi076.stdout: "request": 50, 2024-04-26T21:59:15.856 INFO:teuthology.orchestra.run.smithi076.stdout: "reply": 50, 2024-04-26T21:59:15.856 INFO:teuthology.orchestra.run.smithi076.stdout: "reply_latency": { 2024-04-26T21:59:15.856 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 50, 2024-04-26T21:59:15.856 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.182994286, 2024-04-26T21:59:15.857 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.003659885 2024-04-26T21:59:15.857 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:15.857 INFO:teuthology.orchestra.run.smithi076.stdout: "slow_reply": 0, 2024-04-26T21:59:15.857 INFO:teuthology.orchestra.run.smithi076.stdout: "forward": 0, 2024-04-26T21:59:15.857 INFO:teuthology.orchestra.run.smithi076.stdout: "dir_fetch_complete": 0, 2024-04-26T21:59:15.857 INFO:teuthology.orchestra.run.smithi076.stdout: "dir_fetch_keys": 0, 2024-04-26T21:59:15.857 INFO:teuthology.orchestra.run.smithi076.stdout: "dir_commit": 12, 2024-04-26T21:59:15.857 INFO:teuthology.orchestra.run.smithi076.stdout: "dir_split": 0, 2024-04-26T21:59:15.857 INFO:teuthology.orchestra.run.smithi076.stdout: "dir_merge": 0, 2024-04-26T21:59:15.857 INFO:teuthology.orchestra.run.smithi076.stdout: "inodes": 24, 2024-04-26T21:59:15.857 INFO:teuthology.orchestra.run.smithi076.stdout: "inodes_top": 0, 2024-04-26T21:59:15.857 INFO:teuthology.orchestra.run.smithi076.stdout: "inodes_bottom": 24, 2024-04-26T21:59:15.858 INFO:teuthology.orchestra.run.smithi076.stdout: "inodes_pin_tail": 0, 2024-04-26T21:59:15.858 INFO:teuthology.orchestra.run.smithi076.stdout: "inodes_pinned": 24, 2024-04-26T21:59:15.858 INFO:teuthology.orchestra.run.smithi076.stdout: "inodes_expired": 0, 2024-04-26T21:59:15.858 INFO:teuthology.orchestra.run.smithi076.stdout: "inodes_with_caps": 4, 2024-04-26T21:59:15.858 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": 4, 2024-04-26T21:59:15.858 INFO:teuthology.orchestra.run.smithi076.stdout: "subtrees": 2, 2024-04-26T21:59:15.858 INFO:teuthology.orchestra.run.smithi076.stdout: "traverse": 74, 2024-04-26T21:59:15.858 INFO:teuthology.orchestra.run.smithi076.stdout: "traverse_hit": 66, 2024-04-26T21:59:15.858 INFO:teuthology.orchestra.run.smithi076.stdout: "traverse_forward": 0, 2024-04-26T21:59:15.858 INFO:teuthology.orchestra.run.smithi076.stdout: "traverse_discover": 0, 2024-04-26T21:59:15.858 INFO:teuthology.orchestra.run.smithi076.stdout: "traverse_dir_fetch": 0, 2024-04-26T21:59:15.858 INFO:teuthology.orchestra.run.smithi076.stdout: "traverse_remote_ino": 0, 2024-04-26T21:59:15.859 INFO:teuthology.orchestra.run.smithi076.stdout: "traverse_lock": 0, 2024-04-26T21:59:15.859 INFO:teuthology.orchestra.run.smithi076.stdout: "load_cent": 53, 2024-04-26T21:59:15.859 INFO:teuthology.orchestra.run.smithi076.stdout: "q": 0, 2024-04-26T21:59:15.859 INFO:teuthology.orchestra.run.smithi076.stdout: "exported": 0, 2024-04-26T21:59:15.859 INFO:teuthology.orchestra.run.smithi076.stdout: "exported_inodes": 0, 2024-04-26T21:59:15.859 INFO:teuthology.orchestra.run.smithi076.stdout: "imported": 0, 2024-04-26T21:59:15.859 INFO:teuthology.orchestra.run.smithi076.stdout: "imported_inodes": 0, 2024-04-26T21:59:15.859 INFO:teuthology.orchestra.run.smithi076.stdout: "openino_dir_fetch": 0, 2024-04-26T21:59:15.859 INFO:teuthology.orchestra.run.smithi076.stdout: "openino_backtrace_fetch": 85, 2024-04-26T21:59:15.859 INFO:teuthology.orchestra.run.smithi076.stdout: "openino_peer_discover": 0, 2024-04-26T21:59:15.859 INFO:teuthology.orchestra.run.smithi076.stdout: "root_rfiles": 4, 2024-04-26T21:59:15.859 INFO:teuthology.orchestra.run.smithi076.stdout: "root_rbytes": 234, 2024-04-26T21:59:15.859 INFO:teuthology.orchestra.run.smithi076.stdout: "root_rsnaps": 0, 2024-04-26T21:59:15.859 INFO:teuthology.orchestra.run.smithi076.stdout: "scrub_backtrace_fetch": 85, 2024-04-26T21:59:15.860 INFO:teuthology.orchestra.run.smithi076.stdout: "scrub_set_tag": 0, 2024-04-26T21:59:15.860 INFO:teuthology.orchestra.run.smithi076.stdout: "scrub_backtrace_repaired": 0, 2024-04-26T21:59:15.860 INFO:teuthology.orchestra.run.smithi076.stdout: "scrub_inotable_repaired": 0, 2024-04-26T21:59:15.860 INFO:teuthology.orchestra.run.smithi076.stdout: "scrub_dir_inodes": 51, 2024-04-26T21:59:15.860 INFO:teuthology.orchestra.run.smithi076.stdout: "scrub_dir_base_inodes": 6, 2024-04-26T21:59:15.860 INFO:teuthology.orchestra.run.smithi076.stdout: "scrub_dirfrag_rstats": 45, 2024-04-26T21:59:15.860 INFO:teuthology.orchestra.run.smithi076.stdout: "scrub_file_inodes": 34, 2024-04-26T21:59:15.860 INFO:teuthology.orchestra.run.smithi076.stdout: "handle_inode_file_caps": 0, 2024-04-26T21:59:15.860 INFO:teuthology.orchestra.run.smithi076.stdout: "ceph_cap_op_revoke": 20, 2024-04-26T21:59:15.860 INFO:teuthology.orchestra.run.smithi076.stdout: "ceph_cap_op_grant": 30, 2024-04-26T21:59:15.860 INFO:teuthology.orchestra.run.smithi076.stdout: "ceph_cap_op_trunc": 0, 2024-04-26T21:59:15.860 INFO:teuthology.orchestra.run.smithi076.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-26T21:59:15.860 INFO:teuthology.orchestra.run.smithi076.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-26T21:59:15.860 INFO:teuthology.orchestra.run.smithi076.stdout: "handle_client_caps": 95, 2024-04-26T21:59:15.860 INFO:teuthology.orchestra.run.smithi076.stdout: "handle_client_caps_dirty": 74, 2024-04-26T21:59:15.861 INFO:teuthology.orchestra.run.smithi076.stdout: "handle_client_cap_release": 4, 2024-04-26T21:59:15.861 INFO:teuthology.orchestra.run.smithi076.stdout: "process_request_cap_release": 33 2024-04-26T21:59:15.861 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:15.861 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_cache": { 2024-04-26T21:59:15.861 INFO:teuthology.orchestra.run.smithi076.stdout: "dir_update": 0, 2024-04-26T21:59:15.861 INFO:teuthology.orchestra.run.smithi076.stdout: "dir_update_receipt": 0, 2024-04-26T21:59:15.861 INFO:teuthology.orchestra.run.smithi076.stdout: "dir_try_discover": 0, 2024-04-26T21:59:15.861 INFO:teuthology.orchestra.run.smithi076.stdout: "dir_send_discover": 0, 2024-04-26T21:59:15.861 INFO:teuthology.orchestra.run.smithi076.stdout: "dir_handle_discover": 7, 2024-04-26T21:59:15.861 INFO:teuthology.orchestra.run.smithi076.stdout: "num_strays": 1, 2024-04-26T21:59:15.861 INFO:teuthology.orchestra.run.smithi076.stdout: "num_strays_delayed": 0, 2024-04-26T21:59:15.861 INFO:teuthology.orchestra.run.smithi076.stdout: "num_strays_enqueuing": 0, 2024-04-26T21:59:15.861 INFO:teuthology.orchestra.run.smithi076.stdout: "strays_created": 6, 2024-04-26T21:59:15.861 INFO:teuthology.orchestra.run.smithi076.stdout: "strays_enqueued": 5, 2024-04-26T21:59:15.861 INFO:teuthology.orchestra.run.smithi076.stdout: "strays_reintegrated": 0, 2024-04-26T21:59:15.861 INFO:teuthology.orchestra.run.smithi076.stdout: "strays_migrated": 0, 2024-04-26T21:59:15.862 INFO:teuthology.orchestra.run.smithi076.stdout: "num_recovering_processing": 0, 2024-04-26T21:59:15.862 INFO:teuthology.orchestra.run.smithi076.stdout: "num_recovering_enqueued": 0, 2024-04-26T21:59:15.862 INFO:teuthology.orchestra.run.smithi076.stdout: "num_recovering_prioritized": 0, 2024-04-26T21:59:15.862 INFO:teuthology.orchestra.run.smithi076.stdout: "recovery_started": 0, 2024-04-26T21:59:15.862 INFO:teuthology.orchestra.run.smithi076.stdout: "recovery_completed": 0, 2024-04-26T21:59:15.862 INFO:teuthology.orchestra.run.smithi076.stdout: "ireq_quiesce_path": 0, 2024-04-26T21:59:15.862 INFO:teuthology.orchestra.run.smithi076.stdout: "ireq_quiesce_inode": 0, 2024-04-26T21:59:15.862 INFO:teuthology.orchestra.run.smithi076.stdout: "ireq_enqueue_scrub": 6, 2024-04-26T21:59:15.862 INFO:teuthology.orchestra.run.smithi076.stdout: "ireq_exportdir": 0, 2024-04-26T21:59:15.862 INFO:teuthology.orchestra.run.smithi076.stdout: "ireq_flush": 0, 2024-04-26T21:59:15.862 INFO:teuthology.orchestra.run.smithi076.stdout: "ireq_fragmentdir": 0, 2024-04-26T21:59:15.862 INFO:teuthology.orchestra.run.smithi076.stdout: "ireq_fragstats": 0, 2024-04-26T21:59:15.862 INFO:teuthology.orchestra.run.smithi076.stdout: "ireq_inodestats": 0 2024-04-26T21:59:15.862 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:15.863 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_log": { 2024-04-26T21:59:15.863 INFO:teuthology.orchestra.run.smithi076.stdout: "evlrg": 0, 2024-04-26T21:59:15.863 INFO:teuthology.orchestra.run.smithi076.stdout: "evadd": 135, 2024-04-26T21:59:15.863 INFO:teuthology.orchestra.run.smithi076.stdout: "evex": 0, 2024-04-26T21:59:15.863 INFO:teuthology.orchestra.run.smithi076.stdout: "evtrm": 0, 2024-04-26T21:59:15.863 INFO:teuthology.orchestra.run.smithi076.stdout: "ev": 135, 2024-04-26T21:59:15.863 INFO:teuthology.orchestra.run.smithi076.stdout: "evexg": 0, 2024-04-26T21:59:15.863 INFO:teuthology.orchestra.run.smithi076.stdout: "evexd": 0, 2024-04-26T21:59:15.863 INFO:teuthology.orchestra.run.smithi076.stdout: "segadd": 2, 2024-04-26T21:59:15.863 INFO:teuthology.orchestra.run.smithi076.stdout: "segex": 0, 2024-04-26T21:59:15.863 INFO:teuthology.orchestra.run.smithi076.stdout: "segtrm": 0, 2024-04-26T21:59:15.863 INFO:teuthology.orchestra.run.smithi076.stdout: "seg": 2, 2024-04-26T21:59:15.863 INFO:teuthology.orchestra.run.smithi076.stdout: "segmjr": 2, 2024-04-26T21:59:15.863 INFO:teuthology.orchestra.run.smithi076.stdout: "segexg": 0, 2024-04-26T21:59:15.863 INFO:teuthology.orchestra.run.smithi076.stdout: "segexd": 0, 2024-04-26T21:59:15.863 INFO:teuthology.orchestra.run.smithi076.stdout: "expos": 4194304, 2024-04-26T21:59:15.864 INFO:teuthology.orchestra.run.smithi076.stdout: "wrpos": 4799575, 2024-04-26T21:59:15.864 INFO:teuthology.orchestra.run.smithi076.stdout: "rdpos": 0, 2024-04-26T21:59:15.864 INFO:teuthology.orchestra.run.smithi076.stdout: "jlat": { 2024-04-26T21:59:15.864 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 110, 2024-04-26T21:59:15.864 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 7.170057645, 2024-04-26T21:59:15.864 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.065182342 2024-04-26T21:59:15.864 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:15.864 INFO:teuthology.orchestra.run.smithi076.stdout: "replayed": 0 2024-04-26T21:59:15.864 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:15.864 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_mem": { 2024-04-26T21:59:15.864 INFO:teuthology.orchestra.run.smithi076.stdout: "ino": 24, 2024-04-26T21:59:15.864 INFO:teuthology.orchestra.run.smithi076.stdout: "ino+": 35, 2024-04-26T21:59:15.864 INFO:teuthology.orchestra.run.smithi076.stdout: "ino-": 11, 2024-04-26T21:59:15.864 INFO:teuthology.orchestra.run.smithi076.stdout: "dir": 19, 2024-04-26T21:59:15.864 INFO:teuthology.orchestra.run.smithi076.stdout: "dir+": 20, 2024-04-26T21:59:15.864 INFO:teuthology.orchestra.run.smithi076.stdout: "dir-": 1, 2024-04-26T21:59:15.865 INFO:teuthology.orchestra.run.smithi076.stdout: "dn": 24, 2024-04-26T21:59:15.865 INFO:teuthology.orchestra.run.smithi076.stdout: "dn+": 34, 2024-04-26T21:59:15.865 INFO:teuthology.orchestra.run.smithi076.stdout: "dn-": 10, 2024-04-26T21:59:15.865 INFO:teuthology.orchestra.run.smithi076.stdout: "cap": 4, 2024-04-26T21:59:15.865 INFO:teuthology.orchestra.run.smithi076.stdout: "cap+": 19, 2024-04-26T21:59:15.865 INFO:teuthology.orchestra.run.smithi076.stdout: "cap-": 15, 2024-04-26T21:59:15.865 INFO:teuthology.orchestra.run.smithi076.stdout: "rss": 53636, 2024-04-26T21:59:15.865 INFO:teuthology.orchestra.run.smithi076.stdout: "heap": 231800 2024-04-26T21:59:15.865 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:15.865 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_server": { 2024-04-26T21:59:15.865 INFO:teuthology.orchestra.run.smithi076.stdout: "dispatch_client_request": 71, 2024-04-26T21:59:15.865 INFO:teuthology.orchestra.run.smithi076.stdout: "dispatch_server_request": 1, 2024-04-26T21:59:15.865 INFO:teuthology.orchestra.run.smithi076.stdout: "handle_client_request": 50, 2024-04-26T21:59:15.865 INFO:teuthology.orchestra.run.smithi076.stdout: "handle_client_session": 14, 2024-04-26T21:59:15.866 INFO:teuthology.orchestra.run.smithi076.stdout: "handle_peer_request": 2, 2024-04-26T21:59:15.866 INFO:teuthology.orchestra.run.smithi076.stdout: "req_create_latency": { 2024-04-26T21:59:15.866 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 8, 2024-04-26T21:59:15.866 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.022410253, 2024-04-26T21:59:15.866 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.002801281 2024-04-26T21:59:15.866 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:15.866 INFO:teuthology.orchestra.run.smithi076.stdout: "req_getattr_latency": { 2024-04-26T21:59:15.866 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 16, 2024-04-26T21:59:15.866 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.009915843, 2024-04-26T21:59:15.866 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000619740 2024-04-26T21:59:15.866 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:15.866 INFO:teuthology.orchestra.run.smithi076.stdout: "req_getfilelock_latency": { 2024-04-26T21:59:15.866 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-04-26T21:59:15.866 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-04-26T21:59:15.867 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-04-26T21:59:15.867 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:15.867 INFO:teuthology.orchestra.run.smithi076.stdout: "req_link_latency": { 2024-04-26T21:59:15.867 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-04-26T21:59:15.867 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-04-26T21:59:15.867 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-04-26T21:59:15.867 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:15.867 INFO:teuthology.orchestra.run.smithi076.stdout: "req_lookup_latency": { 2024-04-26T21:59:15.867 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-04-26T21:59:15.867 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-04-26T21:59:15.867 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-04-26T21:59:15.867 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:15.867 INFO:teuthology.orchestra.run.smithi076.stdout: "req_lookuphash_latency": { 2024-04-26T21:59:15.867 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-04-26T21:59:15.867 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-04-26T21:59:15.867 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-04-26T21:59:15.868 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:15.868 INFO:teuthology.orchestra.run.smithi076.stdout: "req_lookupino_latency": { 2024-04-26T21:59:15.868 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-04-26T21:59:15.868 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-04-26T21:59:15.868 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-04-26T21:59:15.868 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:15.868 INFO:teuthology.orchestra.run.smithi076.stdout: "req_lookupname_latency": { 2024-04-26T21:59:15.868 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-04-26T21:59:15.868 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-04-26T21:59:15.868 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-04-26T21:59:15.868 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:15.868 INFO:teuthology.orchestra.run.smithi076.stdout: "req_lookupparent_latency": { 2024-04-26T21:59:15.868 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-04-26T21:59:15.868 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-04-26T21:59:15.868 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-04-26T21:59:15.869 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:15.869 INFO:teuthology.orchestra.run.smithi076.stdout: "req_lookupsnap_latency": { 2024-04-26T21:59:15.869 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-04-26T21:59:15.869 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-04-26T21:59:15.869 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-04-26T21:59:15.869 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:15.869 INFO:teuthology.orchestra.run.smithi076.stdout: "req_lssnap_latency": { 2024-04-26T21:59:15.869 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-04-26T21:59:15.869 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-04-26T21:59:15.869 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-04-26T21:59:15.869 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:15.869 INFO:teuthology.orchestra.run.smithi076.stdout: "req_mkdir_latency": { 2024-04-26T21:59:15.870 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 8, 2024-04-26T21:59:15.870 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.043422118, 2024-04-26T21:59:15.870 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.005427764 2024-04-26T21:59:15.870 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:15.870 INFO:teuthology.orchestra.run.smithi076.stdout: "req_mknod_latency": { 2024-04-26T21:59:15.870 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-04-26T21:59:15.870 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-04-26T21:59:15.871 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-04-26T21:59:15.871 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:15.871 INFO:teuthology.orchestra.run.smithi076.stdout: "req_mksnap_latency": { 2024-04-26T21:59:15.871 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-04-26T21:59:15.871 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-04-26T21:59:15.871 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-04-26T21:59:15.871 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:15.871 INFO:teuthology.orchestra.run.smithi076.stdout: "req_open_latency": { 2024-04-26T21:59:15.871 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-04-26T21:59:15.871 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-04-26T21:59:15.871 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-04-26T21:59:15.871 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:15.872 INFO:teuthology.orchestra.run.smithi076.stdout: "req_readdir_latency": { 2024-04-26T21:59:15.872 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 2, 2024-04-26T21:59:15.872 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.004214570, 2024-04-26T21:59:15.872 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.002107285 2024-04-26T21:59:15.872 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:15.872 INFO:teuthology.orchestra.run.smithi076.stdout: "req_rename_latency": { 2024-04-26T21:59:15.872 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 2, 2024-04-26T21:59:15.872 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.006029992, 2024-04-26T21:59:15.872 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.003014996 2024-04-26T21:59:15.872 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:15.872 INFO:teuthology.orchestra.run.smithi076.stdout: "req_renamesnap_latency": { 2024-04-26T21:59:15.872 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-04-26T21:59:15.873 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-04-26T21:59:15.873 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-04-26T21:59:15.873 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:15.873 INFO:teuthology.orchestra.run.smithi076.stdout: "req_snapdiff_latency": { 2024-04-26T21:59:15.873 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-04-26T21:59:15.873 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-04-26T21:59:15.873 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-04-26T21:59:15.873 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:15.873 INFO:teuthology.orchestra.run.smithi076.stdout: "req_rmdir_latency": { 2024-04-26T21:59:15.873 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 2, 2024-04-26T21:59:15.873 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.030971916, 2024-04-26T21:59:15.873 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.015485958 2024-04-26T21:59:15.873 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:15.874 INFO:teuthology.orchestra.run.smithi076.stdout: "req_rmsnap_latency": { 2024-04-26T21:59:15.874 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-04-26T21:59:15.874 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-04-26T21:59:15.874 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-04-26T21:59:15.874 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:15.874 INFO:teuthology.orchestra.run.smithi076.stdout: "req_rmxattr_latency": { 2024-04-26T21:59:15.874 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-04-26T21:59:15.874 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-04-26T21:59:15.874 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-04-26T21:59:15.874 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:15.874 INFO:teuthology.orchestra.run.smithi076.stdout: "req_setattr_latency": { 2024-04-26T21:59:15.874 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 1, 2024-04-26T21:59:15.875 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.003069462, 2024-04-26T21:59:15.875 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.003069462 2024-04-26T21:59:15.875 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:15.875 INFO:teuthology.orchestra.run.smithi076.stdout: "req_setdirlayout_latency": { 2024-04-26T21:59:15.875 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-04-26T21:59:15.875 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-04-26T21:59:15.875 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-04-26T21:59:15.875 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:15.875 INFO:teuthology.orchestra.run.smithi076.stdout: "req_setfilelock_latency": { 2024-04-26T21:59:15.875 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-04-26T21:59:15.875 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-04-26T21:59:15.876 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-04-26T21:59:15.876 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:15.876 INFO:teuthology.orchestra.run.smithi076.stdout: "req_setlayout_latency": { 2024-04-26T21:59:15.876 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-04-26T21:59:15.876 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-04-26T21:59:15.876 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-04-26T21:59:15.876 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:15.876 INFO:teuthology.orchestra.run.smithi076.stdout: "req_setxattr_latency": { 2024-04-26T21:59:15.876 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 7, 2024-04-26T21:59:15.876 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.045915103, 2024-04-26T21:59:15.876 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.006559300 2024-04-26T21:59:15.876 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:15.876 INFO:teuthology.orchestra.run.smithi076.stdout: "req_symlink_latency": { 2024-04-26T21:59:15.876 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-04-26T21:59:15.877 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-04-26T21:59:15.877 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-04-26T21:59:15.877 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:15.877 INFO:teuthology.orchestra.run.smithi076.stdout: "req_unlink_latency": { 2024-04-26T21:59:15.877 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 4, 2024-04-26T21:59:15.877 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.017045029, 2024-04-26T21:59:15.877 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.004261257 2024-04-26T21:59:15.877 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:15.877 INFO:teuthology.orchestra.run.smithi076.stdout: "cap_revoke_eviction": 0, 2024-04-26T21:59:15.877 INFO:teuthology.orchestra.run.smithi076.stdout: "cap_acquisition_throttle": 0, 2024-04-26T21:59:15.877 INFO:teuthology.orchestra.run.smithi076.stdout: "req_getvxattr_latency": { 2024-04-26T21:59:15.877 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-04-26T21:59:15.877 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-04-26T21:59:15.878 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-04-26T21:59:15.878 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-26T21:59:15.878 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:15.878 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_sessions": { 2024-04-26T21:59:15.878 INFO:teuthology.orchestra.run.smithi076.stdout: "session_count": 2, 2024-04-26T21:59:15.878 INFO:teuthology.orchestra.run.smithi076.stdout: "session_add": 2, 2024-04-26T21:59:15.878 INFO:teuthology.orchestra.run.smithi076.stdout: "session_remove": 0, 2024-04-26T21:59:15.878 INFO:teuthology.orchestra.run.smithi076.stdout: "sessions_open": 2, 2024-04-26T21:59:15.878 INFO:teuthology.orchestra.run.smithi076.stdout: "sessions_stale": 0, 2024-04-26T21:59:15.878 INFO:teuthology.orchestra.run.smithi076.stdout: "total_load": 16, 2024-04-26T21:59:15.878 INFO:teuthology.orchestra.run.smithi076.stdout: "average_load": 8, 2024-04-26T21:59:15.878 INFO:teuthology.orchestra.run.smithi076.stdout: "avg_session_uptime": 305, 2024-04-26T21:59:15.878 INFO:teuthology.orchestra.run.smithi076.stdout: "mdthresh_evicted": 0 2024-04-26T21:59:15.879 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:15.879 INFO:teuthology.orchestra.run.smithi076.stdout: "mempool": { 2024-04-26T21:59:15.879 INFO:teuthology.orchestra.run.smithi076.stdout: "bloom_filter_bytes": 0, 2024-04-26T21:59:15.879 INFO:teuthology.orchestra.run.smithi076.stdout: "bloom_filter_items": 0, 2024-04-26T21:59:15.879 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_alloc_bytes": 0, 2024-04-26T21:59:15.879 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_alloc_items": 0, 2024-04-26T21:59:15.879 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_data_bytes": 0, 2024-04-26T21:59:15.879 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_data_items": 0, 2024-04-26T21:59:15.879 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-26T21:59:15.879 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_onode_items": 0, 2024-04-26T21:59:15.879 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-26T21:59:15.879 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_meta_items": 0, 2024-04-26T21:59:15.879 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_other_bytes": 0, 2024-04-26T21:59:15.879 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_other_items": 0, 2024-04-26T21:59:15.880 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-26T21:59:15.880 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_buffer_items": 0, 2024-04-26T21:59:15.880 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_extent_bytes": 0, 2024-04-26T21:59:15.880 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_extent_items": 0, 2024-04-26T21:59:15.880 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_blob_bytes": 0, 2024-04-26T21:59:15.880 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_blob_items": 0, 2024-04-26T21:59:15.880 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-26T21:59:15.880 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_shared_blob_items": 0, 2024-04-26T21:59:15.880 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-26T21:59:15.880 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_inline_bl_items": 0, 2024-04-26T21:59:15.880 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_fsck_bytes": 0, 2024-04-26T21:59:15.881 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_fsck_items": 0, 2024-04-26T21:59:15.881 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_txc_bytes": 0, 2024-04-26T21:59:15.881 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_txc_items": 0, 2024-04-26T21:59:15.881 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-26T21:59:15.881 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_writing_deferred_items": 0, 2024-04-26T21:59:15.881 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_writing_bytes": 0, 2024-04-26T21:59:15.881 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_writing_items": 0, 2024-04-26T21:59:15.881 INFO:teuthology.orchestra.run.smithi076.stdout: "bluefs_bytes": 0, 2024-04-26T21:59:15.881 INFO:teuthology.orchestra.run.smithi076.stdout: "bluefs_items": 0, 2024-04-26T21:59:15.881 INFO:teuthology.orchestra.run.smithi076.stdout: "bluefs_file_reader_bytes": 0, 2024-04-26T21:59:15.881 INFO:teuthology.orchestra.run.smithi076.stdout: "bluefs_file_reader_items": 0, 2024-04-26T21:59:15.881 INFO:teuthology.orchestra.run.smithi076.stdout: "bluefs_file_writer_bytes": 0, 2024-04-26T21:59:15.881 INFO:teuthology.orchestra.run.smithi076.stdout: "bluefs_file_writer_items": 0, 2024-04-26T21:59:15.881 INFO:teuthology.orchestra.run.smithi076.stdout: "buffer_anon_bytes": 316789, 2024-04-26T21:59:15.882 INFO:teuthology.orchestra.run.smithi076.stdout: "buffer_anon_items": 132, 2024-04-26T21:59:15.882 INFO:teuthology.orchestra.run.smithi076.stdout: "buffer_meta_bytes": 0, 2024-04-26T21:59:15.882 INFO:teuthology.orchestra.run.smithi076.stdout: "buffer_meta_items": 0, 2024-04-26T21:59:15.882 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_bytes": 0, 2024-04-26T21:59:15.882 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_items": 0, 2024-04-26T21:59:15.882 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_mapbl_bytes": 0, 2024-04-26T21:59:15.882 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_mapbl_items": 0, 2024-04-26T21:59:15.882 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_pglog_bytes": 0, 2024-04-26T21:59:15.882 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_pglog_items": 0, 2024-04-26T21:59:15.882 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_bytes": 8008, 2024-04-26T21:59:15.882 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_items": 168, 2024-04-26T21:59:15.882 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_mapping_bytes": 0, 2024-04-26T21:59:15.882 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_mapping_items": 0, 2024-04-26T21:59:15.882 INFO:teuthology.orchestra.run.smithi076.stdout: "pgmap_bytes": 0, 2024-04-26T21:59:15.883 INFO:teuthology.orchestra.run.smithi076.stdout: "pgmap_items": 0, 2024-04-26T21:59:15.883 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_co_bytes": 99257, 2024-04-26T21:59:15.883 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_co_items": 707, 2024-04-26T21:59:15.883 INFO:teuthology.orchestra.run.smithi076.stdout: "unittest_1_bytes": 0, 2024-04-26T21:59:15.883 INFO:teuthology.orchestra.run.smithi076.stdout: "unittest_1_items": 0, 2024-04-26T21:59:15.883 INFO:teuthology.orchestra.run.smithi076.stdout: "unittest_2_bytes": 0, 2024-04-26T21:59:15.883 INFO:teuthology.orchestra.run.smithi076.stdout: "unittest_2_items": 0 2024-04-26T21:59:15.883 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:15.883 INFO:teuthology.orchestra.run.smithi076.stdout: "objecter": { 2024-04-26T21:59:15.883 INFO:teuthology.orchestra.run.smithi076.stdout: "op_active": 0, 2024-04-26T21:59:15.883 INFO:teuthology.orchestra.run.smithi076.stdout: "op_laggy": 0, 2024-04-26T21:59:15.883 INFO:teuthology.orchestra.run.smithi076.stdout: "op_send": 354, 2024-04-26T21:59:15.883 INFO:teuthology.orchestra.run.smithi076.stdout: "op_send_bytes": 621961, 2024-04-26T21:59:15.883 INFO:teuthology.orchestra.run.smithi076.stdout: "op_resend": 37, 2024-04-26T21:59:15.883 INFO:teuthology.orchestra.run.smithi076.stdout: "op_reply": 354, 2024-04-26T21:59:15.884 INFO:teuthology.orchestra.run.smithi076.stdout: "op_latency": { 2024-04-26T21:59:15.884 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 354, 2024-04-26T21:59:15.884 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 95.332168298, 2024-04-26T21:59:15.884 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.269299910 2024-04-26T21:59:15.884 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:15.884 INFO:teuthology.orchestra.run.smithi076.stdout: "op_inflight": 0, 2024-04-26T21:59:15.884 INFO:teuthology.orchestra.run.smithi076.stdout: "oplen_avg": { 2024-04-26T21:59:15.884 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 354, 2024-04-26T21:59:15.884 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 402 2024-04-26T21:59:15.884 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:15.884 INFO:teuthology.orchestra.run.smithi076.stdout: "op": 354, 2024-04-26T21:59:15.884 INFO:teuthology.orchestra.run.smithi076.stdout: "op_r": 99, 2024-04-26T21:59:15.884 INFO:teuthology.orchestra.run.smithi076.stdout: "op_w": 255, 2024-04-26T21:59:15.884 INFO:teuthology.orchestra.run.smithi076.stdout: "op_rmw": 0, 2024-04-26T21:59:15.885 INFO:teuthology.orchestra.run.smithi076.stdout: "op_pg": 0, 2024-04-26T21:59:15.885 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_stat": 0, 2024-04-26T21:59:15.885 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_create": 11, 2024-04-26T21:59:15.885 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_read": 8, 2024-04-26T21:59:15.885 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_write": 112, 2024-04-26T21:59:15.885 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_writefull": 21, 2024-04-26T21:59:15.885 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_writesame": 0, 2024-04-26T21:59:15.885 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_append": 0, 2024-04-26T21:59:15.885 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_zero": 0, 2024-04-26T21:59:15.885 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_truncate": 0, 2024-04-26T21:59:15.885 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_delete": 80, 2024-04-26T21:59:15.886 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_mapext": 0, 2024-04-26T21:59:15.886 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_sparse_read": 0, 2024-04-26T21:59:15.886 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_clonerange": 0, 2024-04-26T21:59:15.886 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_getxattr": 91, 2024-04-26T21:59:15.886 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_setxattr": 22, 2024-04-26T21:59:15.886 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_cmpxattr": 0, 2024-04-26T21:59:15.886 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_rmxattr": 0, 2024-04-26T21:59:15.886 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_resetxattrs": 0, 2024-04-26T21:59:15.886 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_call": 0, 2024-04-26T21:59:15.886 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_watch": 0, 2024-04-26T21:59:15.886 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_notify": 0, 2024-04-26T21:59:15.886 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_src_cmpxattr": 0, 2024-04-26T21:59:15.886 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_pgls": 0, 2024-04-26T21:59:15.887 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_pgls_filter": 0, 2024-04-26T21:59:15.887 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_other": 0, 2024-04-26T21:59:15.887 INFO:teuthology.orchestra.run.smithi076.stdout: "linger_active": 0, 2024-04-26T21:59:15.887 INFO:teuthology.orchestra.run.smithi076.stdout: "linger_send": 0, 2024-04-26T21:59:15.887 INFO:teuthology.orchestra.run.smithi076.stdout: "linger_resend": 0, 2024-04-26T21:59:15.887 INFO:teuthology.orchestra.run.smithi076.stdout: "linger_ping": 0, 2024-04-26T21:59:15.887 INFO:teuthology.orchestra.run.smithi076.stdout: "poolop_active": 0, 2024-04-26T21:59:15.887 INFO:teuthology.orchestra.run.smithi076.stdout: "poolop_send": 0, 2024-04-26T21:59:15.887 INFO:teuthology.orchestra.run.smithi076.stdout: "poolop_resend": 0, 2024-04-26T21:59:15.887 INFO:teuthology.orchestra.run.smithi076.stdout: "poolstat_active": 0, 2024-04-26T21:59:15.887 INFO:teuthology.orchestra.run.smithi076.stdout: "poolstat_send": 0, 2024-04-26T21:59:15.887 INFO:teuthology.orchestra.run.smithi076.stdout: "poolstat_resend": 0, 2024-04-26T21:59:15.887 INFO:teuthology.orchestra.run.smithi076.stdout: "statfs_active": 0, 2024-04-26T21:59:15.888 INFO:teuthology.orchestra.run.smithi076.stdout: "statfs_send": 0, 2024-04-26T21:59:15.888 INFO:teuthology.orchestra.run.smithi076.stdout: "statfs_resend": 0, 2024-04-26T21:59:15.888 INFO:teuthology.orchestra.run.smithi076.stdout: "command_active": 0, 2024-04-26T21:59:15.888 INFO:teuthology.orchestra.run.smithi076.stdout: "command_send": 0, 2024-04-26T21:59:15.888 INFO:teuthology.orchestra.run.smithi076.stdout: "command_resend": 0, 2024-04-26T21:59:15.888 INFO:teuthology.orchestra.run.smithi076.stdout: "map_epoch": 91, 2024-04-26T21:59:15.888 INFO:teuthology.orchestra.run.smithi076.stdout: "map_full": 0, 2024-04-26T21:59:15.888 INFO:teuthology.orchestra.run.smithi076.stdout: "map_inc": 5, 2024-04-26T21:59:15.888 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_sessions": 12, 2024-04-26T21:59:15.888 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_session_open": 12, 2024-04-26T21:59:15.888 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_session_close": 0, 2024-04-26T21:59:15.888 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_laggy": 0, 2024-04-26T21:59:15.888 INFO:teuthology.orchestra.run.smithi076.stdout: "omap_wr": 47, 2024-04-26T21:59:15.888 INFO:teuthology.orchestra.run.smithi076.stdout: "omap_rd": 0, 2024-04-26T21:59:15.889 INFO:teuthology.orchestra.run.smithi076.stdout: "omap_del": 10 2024-04-26T21:59:15.889 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:15.889 INFO:teuthology.orchestra.run.smithi076.stdout: "oft": { 2024-04-26T21:59:15.889 INFO:teuthology.orchestra.run.smithi076.stdout: "omap_total_objs": 1, 2024-04-26T21:59:15.889 INFO:teuthology.orchestra.run.smithi076.stdout: "omap_total_kv_pairs": 0, 2024-04-26T21:59:15.889 INFO:teuthology.orchestra.run.smithi076.stdout: "omap_total_updates": 0, 2024-04-26T21:59:15.889 INFO:teuthology.orchestra.run.smithi076.stdout: "omap_total_removes": 0 2024-04-26T21:59:15.889 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:15.889 INFO:teuthology.orchestra.run.smithi076.stdout: "osd-slow-ops": { 2024-04-26T21:59:15.889 INFO:teuthology.orchestra.run.smithi076.stdout: "slow_ops_count": 0 2024-04-26T21:59:15.889 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:15.889 INFO:teuthology.orchestra.run.smithi076.stdout: "purge_queue": { 2024-04-26T21:59:15.889 INFO:teuthology.orchestra.run.smithi076.stdout: "pq_executing_ops": 0, 2024-04-26T21:59:15.890 INFO:teuthology.orchestra.run.smithi076.stdout: "pq_executing_ops_high_water": 67, 2024-04-26T21:59:15.890 INFO:teuthology.orchestra.run.smithi076.stdout: "pq_executing": 0, 2024-04-26T21:59:15.890 INFO:teuthology.orchestra.run.smithi076.stdout: "pq_executing_high_water": 4, 2024-04-26T21:59:15.890 INFO:teuthology.orchestra.run.smithi076.stdout: "pq_executed_ops": 68, 2024-04-26T21:59:15.890 INFO:teuthology.orchestra.run.smithi076.stdout: "pq_executed": 5, 2024-04-26T21:59:15.890 INFO:teuthology.orchestra.run.smithi076.stdout: "pq_item_in_journal": 0 2024-04-26T21:59:15.890 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:15.890 INFO:teuthology.orchestra.run.smithi076.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-26T21:59:15.890 INFO:teuthology.orchestra.run.smithi076.stdout: "val": 0, 2024-04-26T21:59:15.890 INFO:teuthology.orchestra.run.smithi076.stdout: "max": 104857600, 2024-04-26T21:59:15.890 INFO:teuthology.orchestra.run.smithi076.stdout: "get_started": 0, 2024-04-26T21:59:15.890 INFO:teuthology.orchestra.run.smithi076.stdout: "get": 2399, 2024-04-26T21:59:15.890 INFO:teuthology.orchestra.run.smithi076.stdout: "get_sum": 974947, 2024-04-26T21:59:15.890 INFO:teuthology.orchestra.run.smithi076.stdout: "get_or_fail_fail": 0, 2024-04-26T21:59:15.891 INFO:teuthology.orchestra.run.smithi076.stdout: "get_or_fail_success": 2399, 2024-04-26T21:59:15.891 INFO:teuthology.orchestra.run.smithi076.stdout: "take": 0, 2024-04-26T21:59:15.891 INFO:teuthology.orchestra.run.smithi076.stdout: "take_sum": 0, 2024-04-26T21:59:15.891 INFO:teuthology.orchestra.run.smithi076.stdout: "put": 2399, 2024-04-26T21:59:15.891 INFO:teuthology.orchestra.run.smithi076.stdout: "put_sum": 974947, 2024-04-26T21:59:15.891 INFO:teuthology.orchestra.run.smithi076.stdout: "wait": { 2024-04-26T21:59:15.891 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-04-26T21:59:15.891 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-04-26T21:59:15.891 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-04-26T21:59:15.891 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-26T21:59:15.891 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:15.891 INFO:teuthology.orchestra.run.smithi076.stdout: "throttle-objecter_bytes": { 2024-04-26T21:59:15.891 INFO:teuthology.orchestra.run.smithi076.stdout: "val": 0, 2024-04-26T21:59:15.891 INFO:teuthology.orchestra.run.smithi076.stdout: "max": 104857600, 2024-04-26T21:59:15.891 INFO:teuthology.orchestra.run.smithi076.stdout: "get_started": 0, 2024-04-26T21:59:15.892 INFO:teuthology.orchestra.run.smithi076.stdout: "get": 0, 2024-04-26T21:59:15.892 INFO:teuthology.orchestra.run.smithi076.stdout: "get_sum": 0, 2024-04-26T21:59:15.892 INFO:teuthology.orchestra.run.smithi076.stdout: "get_or_fail_fail": 0, 2024-04-26T21:59:15.892 INFO:teuthology.orchestra.run.smithi076.stdout: "get_or_fail_success": 0, 2024-04-26T21:59:15.892 INFO:teuthology.orchestra.run.smithi076.stdout: "take": 354, 2024-04-26T21:59:15.892 INFO:teuthology.orchestra.run.smithi076.stdout: "take_sum": 622466, 2024-04-26T21:59:15.892 INFO:teuthology.orchestra.run.smithi076.stdout: "put": 354, 2024-04-26T21:59:15.892 INFO:teuthology.orchestra.run.smithi076.stdout: "put_sum": 622466, 2024-04-26T21:59:15.892 INFO:teuthology.orchestra.run.smithi076.stdout: "wait": { 2024-04-26T21:59:15.892 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-04-26T21:59:15.892 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-04-26T21:59:15.892 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-04-26T21:59:15.892 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-26T21:59:15.892 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:15.893 INFO:teuthology.orchestra.run.smithi076.stdout: "throttle-objecter_ops": { 2024-04-26T21:59:15.893 INFO:teuthology.orchestra.run.smithi076.stdout: "val": 0, 2024-04-26T21:59:15.893 INFO:teuthology.orchestra.run.smithi076.stdout: "max": 1024, 2024-04-26T21:59:15.893 INFO:teuthology.orchestra.run.smithi076.stdout: "get_started": 0, 2024-04-26T21:59:15.893 INFO:teuthology.orchestra.run.smithi076.stdout: "get": 0, 2024-04-26T21:59:15.893 INFO:teuthology.orchestra.run.smithi076.stdout: "get_sum": 0, 2024-04-26T21:59:15.893 INFO:teuthology.orchestra.run.smithi076.stdout: "get_or_fail_fail": 0, 2024-04-26T21:59:15.893 INFO:teuthology.orchestra.run.smithi076.stdout: "get_or_fail_success": 0, 2024-04-26T21:59:15.893 INFO:teuthology.orchestra.run.smithi076.stdout: "take": 354, 2024-04-26T21:59:15.893 INFO:teuthology.orchestra.run.smithi076.stdout: "take_sum": 354, 2024-04-26T21:59:15.893 INFO:teuthology.orchestra.run.smithi076.stdout: "put": 354, 2024-04-26T21:59:15.893 INFO:teuthology.orchestra.run.smithi076.stdout: "put_sum": 354, 2024-04-26T21:59:15.893 INFO:teuthology.orchestra.run.smithi076.stdout: "wait": { 2024-04-26T21:59:15.893 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-04-26T21:59:15.894 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-04-26T21:59:15.894 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-04-26T21:59:15.894 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-26T21:59:15.894 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:15.894 INFO:teuthology.orchestra.run.smithi076.stdout: "throttle-write_buf_throttle": { 2024-04-26T21:59:15.894 INFO:teuthology.orchestra.run.smithi076.stdout: "val": 0, 2024-04-26T21:59:15.894 INFO:teuthology.orchestra.run.smithi076.stdout: "max": 3758096384, 2024-04-26T21:59:15.894 INFO:teuthology.orchestra.run.smithi076.stdout: "get_started": 0, 2024-04-26T21:59:15.894 INFO:teuthology.orchestra.run.smithi076.stdout: "get": 5, 2024-04-26T21:59:15.894 INFO:teuthology.orchestra.run.smithi076.stdout: "get_sum": 505, 2024-04-26T21:59:15.894 INFO:teuthology.orchestra.run.smithi076.stdout: "get_or_fail_fail": 0, 2024-04-26T21:59:15.894 INFO:teuthology.orchestra.run.smithi076.stdout: "get_or_fail_success": 5, 2024-04-26T21:59:15.894 INFO:teuthology.orchestra.run.smithi076.stdout: "take": 0, 2024-04-26T21:59:15.894 INFO:teuthology.orchestra.run.smithi076.stdout: "take_sum": 0, 2024-04-26T21:59:15.895 INFO:teuthology.orchestra.run.smithi076.stdout: "put": 2, 2024-04-26T21:59:15.895 INFO:teuthology.orchestra.run.smithi076.stdout: "put_sum": 505, 2024-04-26T21:59:15.895 INFO:teuthology.orchestra.run.smithi076.stdout: "wait": { 2024-04-26T21:59:15.895 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-04-26T21:59:15.895 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-04-26T21:59:15.895 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-04-26T21:59:15.895 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-26T21:59:15.895 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:15.895 INFO:teuthology.orchestra.run.smithi076.stdout: "throttle-write_buf_throttle-0x563fdafec500": { 2024-04-26T21:59:15.895 INFO:teuthology.orchestra.run.smithi076.stdout: "val": 0, 2024-04-26T21:59:15.895 INFO:teuthology.orchestra.run.smithi076.stdout: "max": 3758096384, 2024-04-26T21:59:15.895 INFO:teuthology.orchestra.run.smithi076.stdout: "get_started": 0, 2024-04-26T21:59:15.895 INFO:teuthology.orchestra.run.smithi076.stdout: "get": 135, 2024-04-26T21:59:15.895 INFO:teuthology.orchestra.run.smithi076.stdout: "get_sum": 605271, 2024-04-26T21:59:15.895 INFO:teuthology.orchestra.run.smithi076.stdout: "get_or_fail_fail": 0, 2024-04-26T21:59:15.896 INFO:teuthology.orchestra.run.smithi076.stdout: "get_or_fail_success": 135, 2024-04-26T21:59:15.896 INFO:teuthology.orchestra.run.smithi076.stdout: "take": 0, 2024-04-26T21:59:15.896 INFO:teuthology.orchestra.run.smithi076.stdout: "take_sum": 0, 2024-04-26T21:59:15.896 INFO:teuthology.orchestra.run.smithi076.stdout: "put": 110, 2024-04-26T21:59:15.896 INFO:teuthology.orchestra.run.smithi076.stdout: "put_sum": 605271, 2024-04-26T21:59:15.896 INFO:teuthology.orchestra.run.smithi076.stdout: "wait": { 2024-04-26T21:59:15.896 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-04-26T21:59:15.896 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-04-26T21:59:15.896 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-04-26T21:59:15.896 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-26T21:59:15.896 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-26T21:59:15.896 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-04-26T21:59:16.784 INFO:tasks.check_counter:Daemon mds.b mds.exported=0 2024-04-26T21:59:16.784 INFO:tasks.check_counter:Daemon mds.b mds.imported=0 2024-04-26T21:59:16.784 DEBUG:tasks.check_counter:Getting stats from e 2024-04-26T21:59:16.785 DEBUG:teuthology.orchestra.run.smithi076:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid a1780950-0415-11ef-bc93-c7b262605968 -- ceph daemon mds.e perf dump 2024-04-26T21:59:17.233 INFO:teuthology.orchestra.run.smithi076.stderr:Inferring config /var/lib/ceph/a1780950-0415-11ef-bc93-c7b262605968/mon.b/config 2024-04-26T21:59:17.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:59:17 smithi076 ceph-mon[31823]: pgmap v564: 97 pgs: 97 active+clean; 1.1 MiB data, 656 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 170 B/s wr, 10 op/s 2024-04-26T21:59:17.321 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:59:17 smithi072 ceph-mon[28138]: pgmap v564: 97 pgs: 97 active+clean; 1.1 MiB data, 656 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 170 B/s wr, 10 op/s 2024-04-26T21:59:17.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:59:17 smithi164 ceph-mon[31916]: pgmap v564: 97 pgs: 97 active+clean; 1.1 MiB data, 656 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 170 B/s wr, 10 op/s 2024-04-26T21:59:19.117 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:59:19 smithi076 ceph-mon[31823]: pgmap v565: 97 pgs: 97 active+clean; 1.1 MiB data, 656 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 8 op/s 2024-04-26T21:59:19.321 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:59:19 smithi072 ceph-mon[28138]: pgmap v565: 97 pgs: 97 active+clean; 1.1 MiB data, 656 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 8 op/s 2024-04-26T21:59:19.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:59:19 smithi164 ceph-mon[31916]: pgmap v565: 97 pgs: 97 active+clean; 1.1 MiB data, 656 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 8 op/s 2024-04-26T21:59:19.452 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-04-26T21:59:19.452 INFO:teuthology.orchestra.run.smithi076.stdout: "AsyncMessenger::Worker-0": { 2024-04-26T21:59:19.452 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_messages": 348, 2024-04-26T21:59:19.452 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_messages": 439, 2024-04-26T21:59:19.452 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_bytes": 407450, 2024-04-26T21:59:19.452 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_bytes": 317118, 2024-04-26T21:59:19.452 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_created_connections": 21, 2024-04-26T21:59:19.452 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_active_connections": 4, 2024-04-26T21:59:19.452 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_total_time": 0.147284426, 2024-04-26T21:59:19.452 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_send_time": 0.063085241, 2024-04-26T21:59:19.452 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_recv_time": 0.073652984, 2024-04-26T21:59:19.453 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_fast_dispatch_time": 0.009589533, 2024-04-26T21:59:19.453 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T21:59:19.453 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 440, 2024-04-26T21:59:19.453 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.091125005, 2024-04-26T21:59:19.453 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000207102 2024-04-26T21:59:19.453 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:19.453 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_handle_ack_lat": { 2024-04-26T21:59:19.453 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 424, 2024-04-26T21:59:19.453 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000199283, 2024-04-26T21:59:19.453 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000470 2024-04-26T21:59:19.454 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:19.454 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_encrypted_bytes": 357616, 2024-04-26T21:59:19.454 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_encrypted_bytes": 76784 2024-04-26T21:59:19.454 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:19.454 INFO:teuthology.orchestra.run.smithi076.stdout: "AsyncMessenger::Worker-1": { 2024-04-26T21:59:19.454 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_messages": 56, 2024-04-26T21:59:19.454 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_messages": 141, 2024-04-26T21:59:19.454 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_bytes": 28560, 2024-04-26T21:59:19.454 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_bytes": 231884, 2024-04-26T21:59:19.454 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_created_connections": 8, 2024-04-26T21:59:19.454 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_active_connections": 5, 2024-04-26T21:59:19.454 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_total_time": 0.029812167, 2024-04-26T21:59:19.454 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_send_time": 0.013851913, 2024-04-26T21:59:19.455 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_recv_time": 0.014170829, 2024-04-26T21:59:19.455 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_fast_dispatch_time": 0.000541798, 2024-04-26T21:59:19.455 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T21:59:19.455 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 141, 2024-04-26T21:59:19.455 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.084787300, 2024-04-26T21:59:19.455 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000601328 2024-04-26T21:59:19.455 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:19.455 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_handle_ack_lat": { 2024-04-26T21:59:19.455 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 74, 2024-04-26T21:59:19.455 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000028440, 2024-04-26T21:59:19.455 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000384 2024-04-26T21:59:19.455 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:19.455 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_encrypted_bytes": 160, 2024-04-26T21:59:19.455 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_encrypted_bytes": 195008 2024-04-26T21:59:19.456 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:19.456 INFO:teuthology.orchestra.run.smithi076.stdout: "AsyncMessenger::Worker-2": { 2024-04-26T21:59:19.456 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_messages": 45, 2024-04-26T21:59:19.456 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_messages": 49, 2024-04-26T21:59:19.456 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_bytes": 19926, 2024-04-26T21:59:19.456 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_bytes": 66840, 2024-04-26T21:59:19.456 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_created_connections": 13, 2024-04-26T21:59:19.456 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_active_connections": 5, 2024-04-26T21:59:19.456 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_total_time": 0.023516794, 2024-04-26T21:59:19.456 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_send_time": 0.006473830, 2024-04-26T21:59:19.456 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_recv_time": 0.016791866, 2024-04-26T21:59:19.456 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_fast_dispatch_time": 0.000398047, 2024-04-26T21:59:19.456 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T21:59:19.457 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 44, 2024-04-26T21:59:19.457 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.057519924, 2024-04-26T21:59:19.457 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.001307271 2024-04-26T21:59:19.457 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:19.457 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_handle_ack_lat": { 2024-04-26T21:59:19.457 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 51, 2024-04-26T21:59:19.457 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000022914, 2024-04-26T21:59:19.457 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000449 2024-04-26T21:59:19.457 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:19.457 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_encrypted_bytes": 2912, 2024-04-26T21:59:19.457 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_encrypted_bytes": 448 2024-04-26T21:59:19.457 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:19.457 INFO:teuthology.orchestra.run.smithi076.stdout: "cct": { 2024-04-26T21:59:19.458 INFO:teuthology.orchestra.run.smithi076.stdout: "total_workers": 1, 2024-04-26T21:59:19.458 INFO:teuthology.orchestra.run.smithi076.stdout: "unhealthy_workers": 0 2024-04-26T21:59:19.458 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:19.458 INFO:teuthology.orchestra.run.smithi076.stdout: "finisher-MDSRank": { 2024-04-26T21:59:19.458 INFO:teuthology.orchestra.run.smithi076.stdout: "queue_len": 0, 2024-04-26T21:59:19.458 INFO:teuthology.orchestra.run.smithi076.stdout: "complete_latency": { 2024-04-26T21:59:19.458 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 51, 2024-04-26T21:59:19.458 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.002268297, 2024-04-26T21:59:19.458 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000044476 2024-04-26T21:59:19.458 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-26T21:59:19.458 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:19.459 INFO:teuthology.orchestra.run.smithi076.stdout: "finisher-PurgeQueue": { 2024-04-26T21:59:19.459 INFO:teuthology.orchestra.run.smithi076.stdout: "queue_len": 0, 2024-04-26T21:59:19.459 INFO:teuthology.orchestra.run.smithi076.stdout: "complete_latency": { 2024-04-26T21:59:19.459 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 2, 2024-04-26T21:59:19.459 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000010402, 2024-04-26T21:59:19.459 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000005201 2024-04-26T21:59:19.459 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-26T21:59:19.459 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:19.459 INFO:teuthology.orchestra.run.smithi076.stdout: "mds": { 2024-04-26T21:59:19.459 INFO:teuthology.orchestra.run.smithi076.stdout: "request": 1, 2024-04-26T21:59:19.459 INFO:teuthology.orchestra.run.smithi076.stdout: "reply": 1, 2024-04-26T21:59:19.459 INFO:teuthology.orchestra.run.smithi076.stdout: "reply_latency": { 2024-04-26T21:59:19.459 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 1, 2024-04-26T21:59:19.459 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.001698218, 2024-04-26T21:59:19.460 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.001698218 2024-04-26T21:59:19.460 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:19.460 INFO:teuthology.orchestra.run.smithi076.stdout: "slow_reply": 0, 2024-04-26T21:59:19.460 INFO:teuthology.orchestra.run.smithi076.stdout: "forward": 0, 2024-04-26T21:59:19.460 INFO:teuthology.orchestra.run.smithi076.stdout: "dir_fetch_complete": 0, 2024-04-26T21:59:19.460 INFO:teuthology.orchestra.run.smithi076.stdout: "dir_fetch_keys": 0, 2024-04-26T21:59:19.460 INFO:teuthology.orchestra.run.smithi076.stdout: "dir_commit": 11, 2024-04-26T21:59:19.460 INFO:teuthology.orchestra.run.smithi076.stdout: "dir_split": 0, 2024-04-26T21:59:19.460 INFO:teuthology.orchestra.run.smithi076.stdout: "dir_merge": 0, 2024-04-26T21:59:19.460 INFO:teuthology.orchestra.run.smithi076.stdout: "inodes": 11, 2024-04-26T21:59:19.460 INFO:teuthology.orchestra.run.smithi076.stdout: "inodes_top": 0, 2024-04-26T21:59:19.460 INFO:teuthology.orchestra.run.smithi076.stdout: "inodes_bottom": 11, 2024-04-26T21:59:19.460 INFO:teuthology.orchestra.run.smithi076.stdout: "inodes_pin_tail": 0, 2024-04-26T21:59:19.460 INFO:teuthology.orchestra.run.smithi076.stdout: "inodes_pinned": 11, 2024-04-26T21:59:19.461 INFO:teuthology.orchestra.run.smithi076.stdout: "inodes_expired": 0, 2024-04-26T21:59:19.461 INFO:teuthology.orchestra.run.smithi076.stdout: "inodes_with_caps": 1, 2024-04-26T21:59:19.461 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": 1, 2024-04-26T21:59:19.461 INFO:teuthology.orchestra.run.smithi076.stdout: "subtrees": 2, 2024-04-26T21:59:19.461 INFO:teuthology.orchestra.run.smithi076.stdout: "traverse": 2, 2024-04-26T21:59:19.461 INFO:teuthology.orchestra.run.smithi076.stdout: "traverse_hit": 1, 2024-04-26T21:59:19.461 INFO:teuthology.orchestra.run.smithi076.stdout: "traverse_forward": 0, 2024-04-26T21:59:19.461 INFO:teuthology.orchestra.run.smithi076.stdout: "traverse_discover": 1, 2024-04-26T21:59:19.461 INFO:teuthology.orchestra.run.smithi076.stdout: "traverse_dir_fetch": 0, 2024-04-26T21:59:19.461 INFO:teuthology.orchestra.run.smithi076.stdout: "traverse_remote_ino": 0, 2024-04-26T21:59:19.461 INFO:teuthology.orchestra.run.smithi076.stdout: "traverse_lock": 0, 2024-04-26T21:59:19.461 INFO:teuthology.orchestra.run.smithi076.stdout: "load_cent": 0, 2024-04-26T21:59:19.461 INFO:teuthology.orchestra.run.smithi076.stdout: "q": 0, 2024-04-26T21:59:19.461 INFO:teuthology.orchestra.run.smithi076.stdout: "exported": 0, 2024-04-26T21:59:19.461 INFO:teuthology.orchestra.run.smithi076.stdout: "exported_inodes": 0, 2024-04-26T21:59:19.461 INFO:teuthology.orchestra.run.smithi076.stdout: "imported": 0, 2024-04-26T21:59:19.462 INFO:teuthology.orchestra.run.smithi076.stdout: "imported_inodes": 0, 2024-04-26T21:59:19.462 INFO:teuthology.orchestra.run.smithi076.stdout: "openino_dir_fetch": 0, 2024-04-26T21:59:19.462 INFO:teuthology.orchestra.run.smithi076.stdout: "openino_backtrace_fetch": 0, 2024-04-26T21:59:19.462 INFO:teuthology.orchestra.run.smithi076.stdout: "openino_peer_discover": 0, 2024-04-26T21:59:19.462 INFO:teuthology.orchestra.run.smithi076.stdout: "root_rfiles": 8, 2024-04-26T21:59:19.462 INFO:teuthology.orchestra.run.smithi076.stdout: "root_rbytes": 195290530, 2024-04-26T21:59:19.462 INFO:teuthology.orchestra.run.smithi076.stdout: "root_rsnaps": 0, 2024-04-26T21:59:19.462 INFO:teuthology.orchestra.run.smithi076.stdout: "scrub_backtrace_fetch": 0, 2024-04-26T21:59:19.462 INFO:teuthology.orchestra.run.smithi076.stdout: "scrub_set_tag": 0, 2024-04-26T21:59:19.462 INFO:teuthology.orchestra.run.smithi076.stdout: "scrub_backtrace_repaired": 0, 2024-04-26T21:59:19.462 INFO:teuthology.orchestra.run.smithi076.stdout: "scrub_inotable_repaired": 0, 2024-04-26T21:59:19.462 INFO:teuthology.orchestra.run.smithi076.stdout: "scrub_dir_inodes": 0, 2024-04-26T21:59:19.463 INFO:teuthology.orchestra.run.smithi076.stdout: "scrub_dir_base_inodes": 0, 2024-04-26T21:59:19.463 INFO:teuthology.orchestra.run.smithi076.stdout: "scrub_dirfrag_rstats": 0, 2024-04-26T21:59:19.463 INFO:teuthology.orchestra.run.smithi076.stdout: "scrub_file_inodes": 0, 2024-04-26T21:59:19.463 INFO:teuthology.orchestra.run.smithi076.stdout: "handle_inode_file_caps": 0, 2024-04-26T21:59:19.463 INFO:teuthology.orchestra.run.smithi076.stdout: "ceph_cap_op_revoke": 0, 2024-04-26T21:59:19.463 INFO:teuthology.orchestra.run.smithi076.stdout: "ceph_cap_op_grant": 0, 2024-04-26T21:59:19.463 INFO:teuthology.orchestra.run.smithi076.stdout: "ceph_cap_op_trunc": 0, 2024-04-26T21:59:19.463 INFO:teuthology.orchestra.run.smithi076.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-26T21:59:19.463 INFO:teuthology.orchestra.run.smithi076.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-26T21:59:19.463 INFO:teuthology.orchestra.run.smithi076.stdout: "handle_client_caps": 0, 2024-04-26T21:59:19.463 INFO:teuthology.orchestra.run.smithi076.stdout: "handle_client_caps_dirty": 0, 2024-04-26T21:59:19.463 INFO:teuthology.orchestra.run.smithi076.stdout: "handle_client_cap_release": 0, 2024-04-26T21:59:19.463 INFO:teuthology.orchestra.run.smithi076.stdout: "process_request_cap_release": 0 2024-04-26T21:59:19.463 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:19.464 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_cache": { 2024-04-26T21:59:19.464 INFO:teuthology.orchestra.run.smithi076.stdout: "dir_update": 0, 2024-04-26T21:59:19.464 INFO:teuthology.orchestra.run.smithi076.stdout: "dir_update_receipt": 0, 2024-04-26T21:59:19.464 INFO:teuthology.orchestra.run.smithi076.stdout: "dir_try_discover": 0, 2024-04-26T21:59:19.464 INFO:teuthology.orchestra.run.smithi076.stdout: "dir_send_discover": 2, 2024-04-26T21:59:19.464 INFO:teuthology.orchestra.run.smithi076.stdout: "dir_handle_discover": 0, 2024-04-26T21:59:19.464 INFO:teuthology.orchestra.run.smithi076.stdout: "num_strays": 0, 2024-04-26T21:59:19.464 INFO:teuthology.orchestra.run.smithi076.stdout: "num_strays_delayed": 0, 2024-04-26T21:59:19.464 INFO:teuthology.orchestra.run.smithi076.stdout: "num_strays_enqueuing": 0, 2024-04-26T21:59:19.464 INFO:teuthology.orchestra.run.smithi076.stdout: "strays_created": 0, 2024-04-26T21:59:19.464 INFO:teuthology.orchestra.run.smithi076.stdout: "strays_enqueued": 0, 2024-04-26T21:59:19.464 INFO:teuthology.orchestra.run.smithi076.stdout: "strays_reintegrated": 0, 2024-04-26T21:59:19.464 INFO:teuthology.orchestra.run.smithi076.stdout: "strays_migrated": 0, 2024-04-26T21:59:19.464 INFO:teuthology.orchestra.run.smithi076.stdout: "num_recovering_processing": 0, 2024-04-26T21:59:19.465 INFO:teuthology.orchestra.run.smithi076.stdout: "num_recovering_enqueued": 0, 2024-04-26T21:59:19.465 INFO:teuthology.orchestra.run.smithi076.stdout: "num_recovering_prioritized": 0, 2024-04-26T21:59:19.465 INFO:teuthology.orchestra.run.smithi076.stdout: "recovery_started": 0, 2024-04-26T21:59:19.465 INFO:teuthology.orchestra.run.smithi076.stdout: "recovery_completed": 0, 2024-04-26T21:59:19.465 INFO:teuthology.orchestra.run.smithi076.stdout: "ireq_quiesce_path": 0, 2024-04-26T21:59:19.465 INFO:teuthology.orchestra.run.smithi076.stdout: "ireq_quiesce_inode": 0, 2024-04-26T21:59:19.465 INFO:teuthology.orchestra.run.smithi076.stdout: "ireq_enqueue_scrub": 0, 2024-04-26T21:59:19.465 INFO:teuthology.orchestra.run.smithi076.stdout: "ireq_exportdir": 0, 2024-04-26T21:59:19.465 INFO:teuthology.orchestra.run.smithi076.stdout: "ireq_flush": 0, 2024-04-26T21:59:19.465 INFO:teuthology.orchestra.run.smithi076.stdout: "ireq_fragmentdir": 0, 2024-04-26T21:59:19.465 INFO:teuthology.orchestra.run.smithi076.stdout: "ireq_fragstats": 0, 2024-04-26T21:59:19.465 INFO:teuthology.orchestra.run.smithi076.stdout: "ireq_inodestats": 0 2024-04-26T21:59:19.465 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:19.465 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_log": { 2024-04-26T21:59:19.465 INFO:teuthology.orchestra.run.smithi076.stdout: "evlrg": 0, 2024-04-26T21:59:19.466 INFO:teuthology.orchestra.run.smithi076.stdout: "evadd": 3, 2024-04-26T21:59:19.466 INFO:teuthology.orchestra.run.smithi076.stdout: "evex": 0, 2024-04-26T21:59:19.466 INFO:teuthology.orchestra.run.smithi076.stdout: "evtrm": 0, 2024-04-26T21:59:19.466 INFO:teuthology.orchestra.run.smithi076.stdout: "ev": 3, 2024-04-26T21:59:19.466 INFO:teuthology.orchestra.run.smithi076.stdout: "evexg": 0, 2024-04-26T21:59:19.466 INFO:teuthology.orchestra.run.smithi076.stdout: "evexd": 0, 2024-04-26T21:59:19.466 INFO:teuthology.orchestra.run.smithi076.stdout: "segadd": 2, 2024-04-26T21:59:19.466 INFO:teuthology.orchestra.run.smithi076.stdout: "segex": 0, 2024-04-26T21:59:19.466 INFO:teuthology.orchestra.run.smithi076.stdout: "segtrm": 0, 2024-04-26T21:59:19.466 INFO:teuthology.orchestra.run.smithi076.stdout: "seg": 2, 2024-04-26T21:59:19.466 INFO:teuthology.orchestra.run.smithi076.stdout: "segmjr": 2, 2024-04-26T21:59:19.466 INFO:teuthology.orchestra.run.smithi076.stdout: "segexg": 0, 2024-04-26T21:59:19.466 INFO:teuthology.orchestra.run.smithi076.stdout: "segexd": 0, 2024-04-26T21:59:19.467 INFO:teuthology.orchestra.run.smithi076.stdout: "expos": 4194304, 2024-04-26T21:59:19.467 INFO:teuthology.orchestra.run.smithi076.stdout: "wrpos": 4195385, 2024-04-26T21:59:19.467 INFO:teuthology.orchestra.run.smithi076.stdout: "rdpos": 0, 2024-04-26T21:59:19.467 INFO:teuthology.orchestra.run.smithi076.stdout: "jlat": { 2024-04-26T21:59:19.467 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 2, 2024-04-26T21:59:19.467 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.007748885, 2024-04-26T21:59:19.467 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.003874442 2024-04-26T21:59:19.467 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:19.467 INFO:teuthology.orchestra.run.smithi076.stdout: "replayed": 0 2024-04-26T21:59:19.467 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:19.467 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_mem": { 2024-04-26T21:59:19.467 INFO:teuthology.orchestra.run.smithi076.stdout: "ino": 14, 2024-04-26T21:59:19.467 INFO:teuthology.orchestra.run.smithi076.stdout: "ino+": 14, 2024-04-26T21:59:19.468 INFO:teuthology.orchestra.run.smithi076.stdout: "ino-": 0, 2024-04-26T21:59:19.468 INFO:teuthology.orchestra.run.smithi076.stdout: "dir": 12, 2024-04-26T21:59:19.468 INFO:teuthology.orchestra.run.smithi076.stdout: "dir+": 13, 2024-04-26T21:59:19.468 INFO:teuthology.orchestra.run.smithi076.stdout: "dir-": 1, 2024-04-26T21:59:19.468 INFO:teuthology.orchestra.run.smithi076.stdout: "dn": 11, 2024-04-26T21:59:19.468 INFO:teuthology.orchestra.run.smithi076.stdout: "dn+": 11, 2024-04-26T21:59:19.468 INFO:teuthology.orchestra.run.smithi076.stdout: "dn-": 0, 2024-04-26T21:59:19.469 INFO:teuthology.orchestra.run.smithi076.stdout: "cap": 1, 2024-04-26T21:59:19.469 INFO:teuthology.orchestra.run.smithi076.stdout: "cap+": 1, 2024-04-26T21:59:19.469 INFO:teuthology.orchestra.run.smithi076.stdout: "cap-": 0, 2024-04-26T21:59:19.469 INFO:teuthology.orchestra.run.smithi076.stdout: "rss": 50432, 2024-04-26T21:59:19.469 INFO:teuthology.orchestra.run.smithi076.stdout: "heap": 223608 2024-04-26T21:59:19.469 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:19.470 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_server": { 2024-04-26T21:59:19.470 INFO:teuthology.orchestra.run.smithi076.stdout: "dispatch_client_request": 2, 2024-04-26T21:59:19.470 INFO:teuthology.orchestra.run.smithi076.stdout: "dispatch_server_request": 0, 2024-04-26T21:59:19.470 INFO:teuthology.orchestra.run.smithi076.stdout: "handle_client_request": 1, 2024-04-26T21:59:19.470 INFO:teuthology.orchestra.run.smithi076.stdout: "handle_client_session": 5, 2024-04-26T21:59:19.471 INFO:teuthology.orchestra.run.smithi076.stdout: "handle_peer_request": 0, 2024-04-26T21:59:19.471 INFO:teuthology.orchestra.run.smithi076.stdout: "req_create_latency": { 2024-04-26T21:59:19.471 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-04-26T21:59:19.471 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-04-26T21:59:19.471 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-04-26T21:59:19.471 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:19.471 INFO:teuthology.orchestra.run.smithi076.stdout: "req_getattr_latency": { 2024-04-26T21:59:19.471 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 1, 2024-04-26T21:59:19.471 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.001698218, 2024-04-26T21:59:19.472 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.001698218 2024-04-26T21:59:19.472 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:19.472 INFO:teuthology.orchestra.run.smithi076.stdout: "req_getfilelock_latency": { 2024-04-26T21:59:19.472 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-04-26T21:59:19.472 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-04-26T21:59:19.472 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-04-26T21:59:19.472 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:19.472 INFO:teuthology.orchestra.run.smithi076.stdout: "req_link_latency": { 2024-04-26T21:59:19.472 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-04-26T21:59:19.472 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-04-26T21:59:19.472 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-04-26T21:59:19.472 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:19.472 INFO:teuthology.orchestra.run.smithi076.stdout: "req_lookup_latency": { 2024-04-26T21:59:19.472 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-04-26T21:59:19.472 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-04-26T21:59:19.473 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-04-26T21:59:19.473 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:19.473 INFO:teuthology.orchestra.run.smithi076.stdout: "req_lookuphash_latency": { 2024-04-26T21:59:19.473 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-04-26T21:59:19.473 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-04-26T21:59:19.473 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-04-26T21:59:19.473 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:19.473 INFO:teuthology.orchestra.run.smithi076.stdout: "req_lookupino_latency": { 2024-04-26T21:59:19.473 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-04-26T21:59:19.473 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-04-26T21:59:19.473 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-04-26T21:59:19.473 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:19.473 INFO:teuthology.orchestra.run.smithi076.stdout: "req_lookupname_latency": { 2024-04-26T21:59:19.473 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-04-26T21:59:19.473 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-04-26T21:59:19.474 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-04-26T21:59:19.474 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:19.474 INFO:teuthology.orchestra.run.smithi076.stdout: "req_lookupparent_latency": { 2024-04-26T21:59:19.474 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-04-26T21:59:19.474 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-04-26T21:59:19.474 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-04-26T21:59:19.474 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:19.474 INFO:teuthology.orchestra.run.smithi076.stdout: "req_lookupsnap_latency": { 2024-04-26T21:59:19.474 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-04-26T21:59:19.475 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-04-26T21:59:19.475 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-04-26T21:59:19.475 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:19.475 INFO:teuthology.orchestra.run.smithi076.stdout: "req_lssnap_latency": { 2024-04-26T21:59:19.475 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-04-26T21:59:19.475 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-04-26T21:59:19.476 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-04-26T21:59:19.476 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:19.476 INFO:teuthology.orchestra.run.smithi076.stdout: "req_mkdir_latency": { 2024-04-26T21:59:19.476 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-04-26T21:59:19.476 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-04-26T21:59:19.476 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-04-26T21:59:19.477 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:19.477 INFO:teuthology.orchestra.run.smithi076.stdout: "req_mknod_latency": { 2024-04-26T21:59:19.477 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-04-26T21:59:19.477 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-04-26T21:59:19.477 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-04-26T21:59:19.477 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:19.477 INFO:teuthology.orchestra.run.smithi076.stdout: "req_mksnap_latency": { 2024-04-26T21:59:19.477 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-04-26T21:59:19.477 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-04-26T21:59:19.477 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-04-26T21:59:19.477 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:19.477 INFO:teuthology.orchestra.run.smithi076.stdout: "req_open_latency": { 2024-04-26T21:59:19.477 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-04-26T21:59:19.477 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-04-26T21:59:19.478 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-04-26T21:59:19.478 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:19.478 INFO:teuthology.orchestra.run.smithi076.stdout: "req_readdir_latency": { 2024-04-26T21:59:19.478 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-04-26T21:59:19.478 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-04-26T21:59:19.478 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-04-26T21:59:19.478 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:19.478 INFO:teuthology.orchestra.run.smithi076.stdout: "req_rename_latency": { 2024-04-26T21:59:19.478 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-04-26T21:59:19.478 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-04-26T21:59:19.478 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-04-26T21:59:19.478 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:19.478 INFO:teuthology.orchestra.run.smithi076.stdout: "req_renamesnap_latency": { 2024-04-26T21:59:19.479 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-04-26T21:59:19.479 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-04-26T21:59:19.479 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-04-26T21:59:19.479 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:19.479 INFO:teuthology.orchestra.run.smithi076.stdout: "req_snapdiff_latency": { 2024-04-26T21:59:19.479 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-04-26T21:59:19.479 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-04-26T21:59:19.479 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-04-26T21:59:19.479 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:19.479 INFO:teuthology.orchestra.run.smithi076.stdout: "req_rmdir_latency": { 2024-04-26T21:59:19.479 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-04-26T21:59:19.479 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-04-26T21:59:19.479 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-04-26T21:59:19.479 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:19.480 INFO:teuthology.orchestra.run.smithi076.stdout: "req_rmsnap_latency": { 2024-04-26T21:59:19.480 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-04-26T21:59:19.480 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-04-26T21:59:19.480 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-04-26T21:59:19.480 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:19.480 INFO:teuthology.orchestra.run.smithi076.stdout: "req_rmxattr_latency": { 2024-04-26T21:59:19.480 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-04-26T21:59:19.480 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-04-26T21:59:19.480 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-04-26T21:59:19.480 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:19.480 INFO:teuthology.orchestra.run.smithi076.stdout: "req_setattr_latency": { 2024-04-26T21:59:19.480 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-04-26T21:59:19.480 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-04-26T21:59:19.480 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-04-26T21:59:19.481 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:19.481 INFO:teuthology.orchestra.run.smithi076.stdout: "req_setdirlayout_latency": { 2024-04-26T21:59:19.481 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-04-26T21:59:19.481 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-04-26T21:59:19.481 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-04-26T21:59:19.481 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:19.481 INFO:teuthology.orchestra.run.smithi076.stdout: "req_setfilelock_latency": { 2024-04-26T21:59:19.481 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-04-26T21:59:19.481 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-04-26T21:59:19.481 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-04-26T21:59:19.481 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:19.482 INFO:teuthology.orchestra.run.smithi076.stdout: "req_setlayout_latency": { 2024-04-26T21:59:19.482 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-04-26T21:59:19.482 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-04-26T21:59:19.482 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-04-26T21:59:19.482 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:19.482 INFO:teuthology.orchestra.run.smithi076.stdout: "req_setxattr_latency": { 2024-04-26T21:59:19.482 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-04-26T21:59:19.482 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-04-26T21:59:19.482 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-04-26T21:59:19.483 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:19.483 INFO:teuthology.orchestra.run.smithi076.stdout: "req_symlink_latency": { 2024-04-26T21:59:19.483 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-04-26T21:59:19.483 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-04-26T21:59:19.483 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-04-26T21:59:19.483 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:19.483 INFO:teuthology.orchestra.run.smithi076.stdout: "req_unlink_latency": { 2024-04-26T21:59:19.483 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-04-26T21:59:19.483 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-04-26T21:59:19.483 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-04-26T21:59:19.483 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:19.483 INFO:teuthology.orchestra.run.smithi076.stdout: "cap_revoke_eviction": 0, 2024-04-26T21:59:19.483 INFO:teuthology.orchestra.run.smithi076.stdout: "cap_acquisition_throttle": 0, 2024-04-26T21:59:19.483 INFO:teuthology.orchestra.run.smithi076.stdout: "req_getvxattr_latency": { 2024-04-26T21:59:19.483 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-04-26T21:59:19.484 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-04-26T21:59:19.484 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-04-26T21:59:19.484 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-26T21:59:19.484 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:19.484 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_sessions": { 2024-04-26T21:59:19.484 INFO:teuthology.orchestra.run.smithi076.stdout: "session_count": 1, 2024-04-26T21:59:19.484 INFO:teuthology.orchestra.run.smithi076.stdout: "session_add": 1, 2024-04-26T21:59:19.484 INFO:teuthology.orchestra.run.smithi076.stdout: "session_remove": 0, 2024-04-26T21:59:19.484 INFO:teuthology.orchestra.run.smithi076.stdout: "sessions_open": 1, 2024-04-26T21:59:19.484 INFO:teuthology.orchestra.run.smithi076.stdout: "sessions_stale": 0, 2024-04-26T21:59:19.484 INFO:teuthology.orchestra.run.smithi076.stdout: "total_load": 1, 2024-04-26T21:59:19.485 INFO:teuthology.orchestra.run.smithi076.stdout: "average_load": 1, 2024-04-26T21:59:19.485 INFO:teuthology.orchestra.run.smithi076.stdout: "avg_session_uptime": 235, 2024-04-26T21:59:19.485 INFO:teuthology.orchestra.run.smithi076.stdout: "mdthresh_evicted": 0 2024-04-26T21:59:19.485 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:19.485 INFO:teuthology.orchestra.run.smithi076.stdout: "mempool": { 2024-04-26T21:59:19.485 INFO:teuthology.orchestra.run.smithi076.stdout: "bloom_filter_bytes": 0, 2024-04-26T21:59:19.485 INFO:teuthology.orchestra.run.smithi076.stdout: "bloom_filter_items": 0, 2024-04-26T21:59:19.485 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_alloc_bytes": 0, 2024-04-26T21:59:19.485 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_alloc_items": 0, 2024-04-26T21:59:19.485 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_data_bytes": 0, 2024-04-26T21:59:19.485 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_data_items": 0, 2024-04-26T21:59:19.485 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-26T21:59:19.485 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_onode_items": 0, 2024-04-26T21:59:19.485 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-26T21:59:19.486 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_meta_items": 0, 2024-04-26T21:59:19.486 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_other_bytes": 0, 2024-04-26T21:59:19.486 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_other_items": 0, 2024-04-26T21:59:19.486 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-26T21:59:19.486 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_buffer_items": 0, 2024-04-26T21:59:19.486 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_extent_bytes": 0, 2024-04-26T21:59:19.486 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_extent_items": 0, 2024-04-26T21:59:19.486 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_blob_bytes": 0, 2024-04-26T21:59:19.486 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_blob_items": 0, 2024-04-26T21:59:19.486 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-26T21:59:19.486 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_shared_blob_items": 0, 2024-04-26T21:59:19.486 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-26T21:59:19.486 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_inline_bl_items": 0, 2024-04-26T21:59:19.487 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_fsck_bytes": 0, 2024-04-26T21:59:19.487 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_fsck_items": 0, 2024-04-26T21:59:19.487 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_txc_bytes": 0, 2024-04-26T21:59:19.487 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_txc_items": 0, 2024-04-26T21:59:19.487 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-26T21:59:19.487 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_writing_deferred_items": 0, 2024-04-26T21:59:19.487 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_writing_bytes": 0, 2024-04-26T21:59:19.487 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_writing_items": 0, 2024-04-26T21:59:19.487 INFO:teuthology.orchestra.run.smithi076.stdout: "bluefs_bytes": 0, 2024-04-26T21:59:19.487 INFO:teuthology.orchestra.run.smithi076.stdout: "bluefs_items": 0, 2024-04-26T21:59:19.487 INFO:teuthology.orchestra.run.smithi076.stdout: "bluefs_file_reader_bytes": 0, 2024-04-26T21:59:19.487 INFO:teuthology.orchestra.run.smithi076.stdout: "bluefs_file_reader_items": 0, 2024-04-26T21:59:19.487 INFO:teuthology.orchestra.run.smithi076.stdout: "bluefs_file_writer_bytes": 0, 2024-04-26T21:59:19.487 INFO:teuthology.orchestra.run.smithi076.stdout: "bluefs_file_writer_items": 0, 2024-04-26T21:59:19.487 INFO:teuthology.orchestra.run.smithi076.stdout: "buffer_anon_bytes": 247078, 2024-04-26T21:59:19.488 INFO:teuthology.orchestra.run.smithi076.stdout: "buffer_anon_items": 82, 2024-04-26T21:59:19.488 INFO:teuthology.orchestra.run.smithi076.stdout: "buffer_meta_bytes": 0, 2024-04-26T21:59:19.488 INFO:teuthology.orchestra.run.smithi076.stdout: "buffer_meta_items": 0, 2024-04-26T21:59:19.488 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_bytes": 0, 2024-04-26T21:59:19.488 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_items": 0, 2024-04-26T21:59:19.488 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_mapbl_bytes": 0, 2024-04-26T21:59:19.488 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_mapbl_items": 0, 2024-04-26T21:59:19.488 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_pglog_bytes": 0, 2024-04-26T21:59:19.488 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_pglog_items": 0, 2024-04-26T21:59:19.488 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_bytes": 8008, 2024-04-26T21:59:19.488 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_items": 168, 2024-04-26T21:59:19.488 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_mapping_bytes": 0, 2024-04-26T21:59:19.488 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_mapping_items": 0, 2024-04-26T21:59:19.488 INFO:teuthology.orchestra.run.smithi076.stdout: "pgmap_bytes": 0, 2024-04-26T21:59:19.488 INFO:teuthology.orchestra.run.smithi076.stdout: "pgmap_items": 0, 2024-04-26T21:59:19.489 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_co_bytes": 55448, 2024-04-26T21:59:19.489 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_co_items": 215, 2024-04-26T21:59:19.489 INFO:teuthology.orchestra.run.smithi076.stdout: "unittest_1_bytes": 0, 2024-04-26T21:59:19.489 INFO:teuthology.orchestra.run.smithi076.stdout: "unittest_1_items": 0, 2024-04-26T21:59:19.489 INFO:teuthology.orchestra.run.smithi076.stdout: "unittest_2_bytes": 0, 2024-04-26T21:59:19.489 INFO:teuthology.orchestra.run.smithi076.stdout: "unittest_2_items": 0 2024-04-26T21:59:19.489 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:19.489 INFO:teuthology.orchestra.run.smithi076.stdout: "objecter": { 2024-04-26T21:59:19.489 INFO:teuthology.orchestra.run.smithi076.stdout: "op_active": 0, 2024-04-26T21:59:19.489 INFO:teuthology.orchestra.run.smithi076.stdout: "op_laggy": 0, 2024-04-26T21:59:19.489 INFO:teuthology.orchestra.run.smithi076.stdout: "op_send": 38, 2024-04-26T21:59:19.489 INFO:teuthology.orchestra.run.smithi076.stdout: "op_send_bytes": 11387, 2024-04-26T21:59:19.489 INFO:teuthology.orchestra.run.smithi076.stdout: "op_resend": 33, 2024-04-26T21:59:19.489 INFO:teuthology.orchestra.run.smithi076.stdout: "op_reply": 38, 2024-04-26T21:59:19.489 INFO:teuthology.orchestra.run.smithi076.stdout: "op_latency": { 2024-04-26T21:59:19.490 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 38, 2024-04-26T21:59:19.490 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.508984970, 2024-04-26T21:59:19.490 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.013394341 2024-04-26T21:59:19.490 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:19.490 INFO:teuthology.orchestra.run.smithi076.stdout: "op_inflight": 0, 2024-04-26T21:59:19.490 INFO:teuthology.orchestra.run.smithi076.stdout: "oplen_avg": { 2024-04-26T21:59:19.490 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 38, 2024-04-26T21:59:19.490 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 61 2024-04-26T21:59:19.490 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:19.490 INFO:teuthology.orchestra.run.smithi076.stdout: "op": 38, 2024-04-26T21:59:19.490 INFO:teuthology.orchestra.run.smithi076.stdout: "op_r": 0, 2024-04-26T21:59:19.490 INFO:teuthology.orchestra.run.smithi076.stdout: "op_w": 38, 2024-04-26T21:59:19.490 INFO:teuthology.orchestra.run.smithi076.stdout: "op_rmw": 0, 2024-04-26T21:59:19.490 INFO:teuthology.orchestra.run.smithi076.stdout: "op_pg": 0, 2024-04-26T21:59:19.491 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_stat": 0, 2024-04-26T21:59:19.491 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_create": 10, 2024-04-26T21:59:19.491 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_read": 0, 2024-04-26T21:59:19.491 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_write": 2, 2024-04-26T21:59:19.491 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_writefull": 6, 2024-04-26T21:59:19.491 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_writesame": 0, 2024-04-26T21:59:19.491 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_append": 0, 2024-04-26T21:59:19.491 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_zero": 0, 2024-04-26T21:59:19.491 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_truncate": 0, 2024-04-26T21:59:19.491 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_delete": 6, 2024-04-26T21:59:19.491 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_mapext": 0, 2024-04-26T21:59:19.491 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_sparse_read": 0, 2024-04-26T21:59:19.491 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_clonerange": 0, 2024-04-26T21:59:19.491 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_getxattr": 0, 2024-04-26T21:59:19.491 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_setxattr": 20, 2024-04-26T21:59:19.492 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_cmpxattr": 0, 2024-04-26T21:59:19.492 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_rmxattr": 0, 2024-04-26T21:59:19.492 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_resetxattrs": 0, 2024-04-26T21:59:19.492 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_call": 0, 2024-04-26T21:59:19.492 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_watch": 0, 2024-04-26T21:59:19.492 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_notify": 0, 2024-04-26T21:59:19.492 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_src_cmpxattr": 0, 2024-04-26T21:59:19.492 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_pgls": 0, 2024-04-26T21:59:19.492 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_pgls_filter": 0, 2024-04-26T21:59:19.492 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_other": 0, 2024-04-26T21:59:19.492 INFO:teuthology.orchestra.run.smithi076.stdout: "linger_active": 0, 2024-04-26T21:59:19.492 INFO:teuthology.orchestra.run.smithi076.stdout: "linger_send": 0, 2024-04-26T21:59:19.492 INFO:teuthology.orchestra.run.smithi076.stdout: "linger_resend": 0, 2024-04-26T21:59:19.492 INFO:teuthology.orchestra.run.smithi076.stdout: "linger_ping": 0, 2024-04-26T21:59:19.492 INFO:teuthology.orchestra.run.smithi076.stdout: "poolop_active": 0, 2024-04-26T21:59:19.493 INFO:teuthology.orchestra.run.smithi076.stdout: "poolop_send": 0, 2024-04-26T21:59:19.493 INFO:teuthology.orchestra.run.smithi076.stdout: "poolop_resend": 0, 2024-04-26T21:59:19.493 INFO:teuthology.orchestra.run.smithi076.stdout: "poolstat_active": 0, 2024-04-26T21:59:19.493 INFO:teuthology.orchestra.run.smithi076.stdout: "poolstat_send": 0, 2024-04-26T21:59:19.493 INFO:teuthology.orchestra.run.smithi076.stdout: "poolstat_resend": 0, 2024-04-26T21:59:19.493 INFO:teuthology.orchestra.run.smithi076.stdout: "statfs_active": 0, 2024-04-26T21:59:19.493 INFO:teuthology.orchestra.run.smithi076.stdout: "statfs_send": 0, 2024-04-26T21:59:19.493 INFO:teuthology.orchestra.run.smithi076.stdout: "statfs_resend": 0, 2024-04-26T21:59:19.493 INFO:teuthology.orchestra.run.smithi076.stdout: "command_active": 0, 2024-04-26T21:59:19.493 INFO:teuthology.orchestra.run.smithi076.stdout: "command_send": 0, 2024-04-26T21:59:19.493 INFO:teuthology.orchestra.run.smithi076.stdout: "command_resend": 0, 2024-04-26T21:59:19.493 INFO:teuthology.orchestra.run.smithi076.stdout: "map_epoch": 91, 2024-04-26T21:59:19.493 INFO:teuthology.orchestra.run.smithi076.stdout: "map_full": 0, 2024-04-26T21:59:19.493 INFO:teuthology.orchestra.run.smithi076.stdout: "map_inc": 4, 2024-04-26T21:59:19.493 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_sessions": 8, 2024-04-26T21:59:19.494 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_session_open": 8, 2024-04-26T21:59:19.494 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_session_close": 0, 2024-04-26T21:59:19.494 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_laggy": 0, 2024-04-26T21:59:19.494 INFO:teuthology.orchestra.run.smithi076.stdout: "omap_wr": 16, 2024-04-26T21:59:19.494 INFO:teuthology.orchestra.run.smithi076.stdout: "omap_rd": 0, 2024-04-26T21:59:19.494 INFO:teuthology.orchestra.run.smithi076.stdout: "omap_del": 1 2024-04-26T21:59:19.494 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:19.494 INFO:teuthology.orchestra.run.smithi076.stdout: "oft": { 2024-04-26T21:59:19.494 INFO:teuthology.orchestra.run.smithi076.stdout: "omap_total_objs": 1, 2024-04-26T21:59:19.494 INFO:teuthology.orchestra.run.smithi076.stdout: "omap_total_kv_pairs": 0, 2024-04-26T21:59:19.494 INFO:teuthology.orchestra.run.smithi076.stdout: "omap_total_updates": 0, 2024-04-26T21:59:19.494 INFO:teuthology.orchestra.run.smithi076.stdout: "omap_total_removes": 0 2024-04-26T21:59:19.494 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:19.494 INFO:teuthology.orchestra.run.smithi076.stdout: "osd-slow-ops": { 2024-04-26T21:59:19.495 INFO:teuthology.orchestra.run.smithi076.stdout: "slow_ops_count": 0 2024-04-26T21:59:19.495 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:19.495 INFO:teuthology.orchestra.run.smithi076.stdout: "purge_queue": { 2024-04-26T21:59:19.495 INFO:teuthology.orchestra.run.smithi076.stdout: "pq_executing_ops": 0, 2024-04-26T21:59:19.495 INFO:teuthology.orchestra.run.smithi076.stdout: "pq_executing_ops_high_water": 0, 2024-04-26T21:59:19.495 INFO:teuthology.orchestra.run.smithi076.stdout: "pq_executing": 0, 2024-04-26T21:59:19.495 INFO:teuthology.orchestra.run.smithi076.stdout: "pq_executing_high_water": 0, 2024-04-26T21:59:19.495 INFO:teuthology.orchestra.run.smithi076.stdout: "pq_executed_ops": 0, 2024-04-26T21:59:19.495 INFO:teuthology.orchestra.run.smithi076.stdout: "pq_executed": 0, 2024-04-26T21:59:19.495 INFO:teuthology.orchestra.run.smithi076.stdout: "pq_item_in_journal": 0 2024-04-26T21:59:19.495 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:19.495 INFO:teuthology.orchestra.run.smithi076.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-26T21:59:19.495 INFO:teuthology.orchestra.run.smithi076.stdout: "val": 0, 2024-04-26T21:59:19.495 INFO:teuthology.orchestra.run.smithi076.stdout: "max": 104857600, 2024-04-26T21:59:19.495 INFO:teuthology.orchestra.run.smithi076.stdout: "get_started": 0, 2024-04-26T21:59:19.495 INFO:teuthology.orchestra.run.smithi076.stdout: "get": 445, 2024-04-26T21:59:19.496 INFO:teuthology.orchestra.run.smithi076.stdout: "get_sum": 404797, 2024-04-26T21:59:19.496 INFO:teuthology.orchestra.run.smithi076.stdout: "get_or_fail_fail": 0, 2024-04-26T21:59:19.496 INFO:teuthology.orchestra.run.smithi076.stdout: "get_or_fail_success": 445, 2024-04-26T21:59:19.496 INFO:teuthology.orchestra.run.smithi076.stdout: "take": 0, 2024-04-26T21:59:19.496 INFO:teuthology.orchestra.run.smithi076.stdout: "take_sum": 0, 2024-04-26T21:59:19.496 INFO:teuthology.orchestra.run.smithi076.stdout: "put": 445, 2024-04-26T21:59:19.496 INFO:teuthology.orchestra.run.smithi076.stdout: "put_sum": 404797, 2024-04-26T21:59:19.496 INFO:teuthology.orchestra.run.smithi076.stdout: "wait": { 2024-04-26T21:59:19.496 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-04-26T21:59:19.496 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-04-26T21:59:19.496 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-04-26T21:59:19.496 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-26T21:59:19.496 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:19.496 INFO:teuthology.orchestra.run.smithi076.stdout: "throttle-objecter_bytes": { 2024-04-26T21:59:19.497 INFO:teuthology.orchestra.run.smithi076.stdout: "val": 0, 2024-04-26T21:59:19.497 INFO:teuthology.orchestra.run.smithi076.stdout: "max": 104857600, 2024-04-26T21:59:19.497 INFO:teuthology.orchestra.run.smithi076.stdout: "get_started": 0, 2024-04-26T21:59:19.497 INFO:teuthology.orchestra.run.smithi076.stdout: "get": 0, 2024-04-26T21:59:19.497 INFO:teuthology.orchestra.run.smithi076.stdout: "get_sum": 0, 2024-04-26T21:59:19.497 INFO:teuthology.orchestra.run.smithi076.stdout: "get_or_fail_fail": 0, 2024-04-26T21:59:19.497 INFO:teuthology.orchestra.run.smithi076.stdout: "get_or_fail_success": 0, 2024-04-26T21:59:19.497 INFO:teuthology.orchestra.run.smithi076.stdout: "take": 38, 2024-04-26T21:59:19.497 INFO:teuthology.orchestra.run.smithi076.stdout: "take_sum": 11387, 2024-04-26T21:59:19.497 INFO:teuthology.orchestra.run.smithi076.stdout: "put": 38, 2024-04-26T21:59:19.497 INFO:teuthology.orchestra.run.smithi076.stdout: "put_sum": 11387, 2024-04-26T21:59:19.497 INFO:teuthology.orchestra.run.smithi076.stdout: "wait": { 2024-04-26T21:59:19.497 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-04-26T21:59:19.497 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-04-26T21:59:19.498 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-04-26T21:59:19.498 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-26T21:59:19.498 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:19.498 INFO:teuthology.orchestra.run.smithi076.stdout: "throttle-objecter_ops": { 2024-04-26T21:59:19.498 INFO:teuthology.orchestra.run.smithi076.stdout: "val": 0, 2024-04-26T21:59:19.498 INFO:teuthology.orchestra.run.smithi076.stdout: "max": 1024, 2024-04-26T21:59:19.498 INFO:teuthology.orchestra.run.smithi076.stdout: "get_started": 0, 2024-04-26T21:59:19.498 INFO:teuthology.orchestra.run.smithi076.stdout: "get": 0, 2024-04-26T21:59:19.498 INFO:teuthology.orchestra.run.smithi076.stdout: "get_sum": 0, 2024-04-26T21:59:19.498 INFO:teuthology.orchestra.run.smithi076.stdout: "get_or_fail_fail": 0, 2024-04-26T21:59:19.498 INFO:teuthology.orchestra.run.smithi076.stdout: "get_or_fail_success": 0, 2024-04-26T21:59:19.498 INFO:teuthology.orchestra.run.smithi076.stdout: "take": 38, 2024-04-26T21:59:19.498 INFO:teuthology.orchestra.run.smithi076.stdout: "take_sum": 38, 2024-04-26T21:59:19.498 INFO:teuthology.orchestra.run.smithi076.stdout: "put": 38, 2024-04-26T21:59:19.499 INFO:teuthology.orchestra.run.smithi076.stdout: "put_sum": 38, 2024-04-26T21:59:19.499 INFO:teuthology.orchestra.run.smithi076.stdout: "wait": { 2024-04-26T21:59:19.499 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-04-26T21:59:19.499 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-04-26T21:59:19.499 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-04-26T21:59:19.499 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-26T21:59:19.499 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:19.499 INFO:teuthology.orchestra.run.smithi076.stdout: "throttle-write_buf_throttle": { 2024-04-26T21:59:19.499 INFO:teuthology.orchestra.run.smithi076.stdout: "val": 0, 2024-04-26T21:59:19.499 INFO:teuthology.orchestra.run.smithi076.stdout: "max": 3758096384, 2024-04-26T21:59:19.499 INFO:teuthology.orchestra.run.smithi076.stdout: "get_started": 0, 2024-04-26T21:59:19.499 INFO:teuthology.orchestra.run.smithi076.stdout: "get": 0, 2024-04-26T21:59:19.499 INFO:teuthology.orchestra.run.smithi076.stdout: "get_sum": 0, 2024-04-26T21:59:19.499 INFO:teuthology.orchestra.run.smithi076.stdout: "get_or_fail_fail": 0, 2024-04-26T21:59:19.500 INFO:teuthology.orchestra.run.smithi076.stdout: "get_or_fail_success": 0, 2024-04-26T21:59:19.500 INFO:teuthology.orchestra.run.smithi076.stdout: "take": 0, 2024-04-26T21:59:19.500 INFO:teuthology.orchestra.run.smithi076.stdout: "take_sum": 0, 2024-04-26T21:59:19.500 INFO:teuthology.orchestra.run.smithi076.stdout: "put": 0, 2024-04-26T21:59:19.500 INFO:teuthology.orchestra.run.smithi076.stdout: "put_sum": 0, 2024-04-26T21:59:19.500 INFO:teuthology.orchestra.run.smithi076.stdout: "wait": { 2024-04-26T21:59:19.500 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-04-26T21:59:19.500 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-04-26T21:59:19.500 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-04-26T21:59:19.500 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-26T21:59:19.500 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:19.500 INFO:teuthology.orchestra.run.smithi076.stdout: "throttle-write_buf_throttle-0x55d55360c500": { 2024-04-26T21:59:19.500 INFO:teuthology.orchestra.run.smithi076.stdout: "val": 0, 2024-04-26T21:59:19.500 INFO:teuthology.orchestra.run.smithi076.stdout: "max": 3758096384, 2024-04-26T21:59:19.500 INFO:teuthology.orchestra.run.smithi076.stdout: "get_started": 0, 2024-04-26T21:59:19.501 INFO:teuthology.orchestra.run.smithi076.stdout: "get": 3, 2024-04-26T21:59:19.501 INFO:teuthology.orchestra.run.smithi076.stdout: "get_sum": 1081, 2024-04-26T21:59:19.501 INFO:teuthology.orchestra.run.smithi076.stdout: "get_or_fail_fail": 0, 2024-04-26T21:59:19.501 INFO:teuthology.orchestra.run.smithi076.stdout: "get_or_fail_success": 3, 2024-04-26T21:59:19.501 INFO:teuthology.orchestra.run.smithi076.stdout: "take": 0, 2024-04-26T21:59:19.501 INFO:teuthology.orchestra.run.smithi076.stdout: "take_sum": 0, 2024-04-26T21:59:19.501 INFO:teuthology.orchestra.run.smithi076.stdout: "put": 2, 2024-04-26T21:59:19.501 INFO:teuthology.orchestra.run.smithi076.stdout: "put_sum": 1081, 2024-04-26T21:59:19.501 INFO:teuthology.orchestra.run.smithi076.stdout: "wait": { 2024-04-26T21:59:19.501 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-04-26T21:59:19.501 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-04-26T21:59:19.501 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-04-26T21:59:19.501 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-26T21:59:19.502 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-26T21:59:19.502 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-04-26T21:59:20.447 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-04-26T21:59:20.447 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-04-26T21:59:20.447 DEBUG:tasks.check_counter:Getting stats from h 2024-04-26T21:59:20.447 DEBUG:teuthology.orchestra.run.smithi076:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid a1780950-0415-11ef-bc93-c7b262605968 -- ceph daemon mds.h perf dump 2024-04-26T21:59:20.892 INFO:teuthology.orchestra.run.smithi076.stderr:Inferring config /var/lib/ceph/a1780950-0415-11ef-bc93-c7b262605968/mon.b/config 2024-04-26T21:59:21.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:59:21 smithi076 ceph-mon[31823]: pgmap v566: 97 pgs: 97 active+clean; 1.1 MiB data, 656 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 170 B/s wr, 12 op/s 2024-04-26T21:59:21.321 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:59:21 smithi072 ceph-mon[28138]: pgmap v566: 97 pgs: 97 active+clean; 1.1 MiB data, 656 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 170 B/s wr, 12 op/s 2024-04-26T21:59:21.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:59:21 smithi164 ceph-mon[31916]: pgmap v566: 97 pgs: 97 active+clean; 1.1 MiB data, 656 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 170 B/s wr, 12 op/s 2024-04-26T21:59:23.254 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-04-26T21:59:23.254 INFO:teuthology.orchestra.run.smithi076.stdout: "AsyncMessenger::Worker-0": { 2024-04-26T21:59:23.255 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_messages": 220, 2024-04-26T21:59:23.255 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_messages": 214, 2024-04-26T21:59:23.255 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_bytes": 380652, 2024-04-26T21:59:23.255 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_bytes": 285064, 2024-04-26T21:59:23.255 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_created_connections": 26, 2024-04-26T21:59:23.255 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_active_connections": 7, 2024-04-26T21:59:23.255 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_total_time": 0.087228530, 2024-04-26T21:59:23.255 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_send_time": 0.022248982, 2024-04-26T21:59:23.255 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_recv_time": 0.056605000, 2024-04-26T21:59:23.255 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_fast_dispatch_time": 0.005878639, 2024-04-26T21:59:23.255 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T21:59:23.255 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 209, 2024-04-26T21:59:23.255 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.059401800, 2024-04-26T21:59:23.255 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000284219 2024-04-26T21:59:23.256 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:23.256 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_handle_ack_lat": { 2024-04-26T21:59:23.256 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 68, 2024-04-26T21:59:23.256 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000027358, 2024-04-26T21:59:23.256 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000402 2024-04-26T21:59:23.256 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:23.256 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_encrypted_bytes": 361312, 2024-04-26T21:59:23.256 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_encrypted_bytes": 76144 2024-04-26T21:59:23.256 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:23.256 INFO:teuthology.orchestra.run.smithi076.stdout: "AsyncMessenger::Worker-1": { 2024-04-26T21:59:23.256 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_messages": 200, 2024-04-26T21:59:23.256 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_messages": 385, 2024-04-26T21:59:23.256 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_bytes": 62039, 2024-04-26T21:59:23.256 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_bytes": 303647, 2024-04-26T21:59:23.256 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_created_connections": 9, 2024-04-26T21:59:23.257 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_active_connections": 4, 2024-04-26T21:59:23.257 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_total_time": 0.104654152, 2024-04-26T21:59:23.257 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_send_time": 0.056249141, 2024-04-26T21:59:23.257 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_recv_time": 0.041219448, 2024-04-26T21:59:23.257 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_fast_dispatch_time": 0.004113564, 2024-04-26T21:59:23.257 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T21:59:23.257 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 387, 2024-04-26T21:59:23.257 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.095886180, 2024-04-26T21:59:23.257 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000247767 2024-04-26T21:59:23.257 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:23.257 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_handle_ack_lat": { 2024-04-26T21:59:23.257 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 461, 2024-04-26T21:59:23.257 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000209084, 2024-04-26T21:59:23.257 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000453 2024-04-26T21:59:23.258 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:23.258 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_encrypted_bytes": 160, 2024-04-26T21:59:23.258 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_encrypted_bytes": 194656 2024-04-26T21:59:23.258 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:23.258 INFO:teuthology.orchestra.run.smithi076.stdout: "AsyncMessenger::Worker-2": { 2024-04-26T21:59:23.258 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_messages": 39, 2024-04-26T21:59:23.258 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_messages": 39, 2024-04-26T21:59:23.258 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_bytes": 19234, 2024-04-26T21:59:23.258 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_bytes": 25177, 2024-04-26T21:59:23.258 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_created_connections": 9, 2024-04-26T21:59:23.258 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_active_connections": 6, 2024-04-26T21:59:23.258 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_total_time": 0.018671508, 2024-04-26T21:59:23.258 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_send_time": 0.004537149, 2024-04-26T21:59:23.258 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_recv_time": 0.014007620, 2024-04-26T21:59:23.259 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_fast_dispatch_time": 0.000596248, 2024-04-26T21:59:23.259 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T21:59:23.259 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 38, 2024-04-26T21:59:23.259 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.059248067, 2024-04-26T21:59:23.259 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.001559159 2024-04-26T21:59:23.259 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:23.259 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_handle_ack_lat": { 2024-04-26T21:59:23.259 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 32, 2024-04-26T21:59:23.259 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000015782, 2024-04-26T21:59:23.259 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000493 2024-04-26T21:59:23.259 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:23.259 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_encrypted_bytes": 2912, 2024-04-26T21:59:23.259 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_encrypted_bytes": 448 2024-04-26T21:59:23.259 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:23.259 INFO:teuthology.orchestra.run.smithi076.stdout: "cct": { 2024-04-26T21:59:23.260 INFO:teuthology.orchestra.run.smithi076.stdout: "total_workers": 1, 2024-04-26T21:59:23.260 INFO:teuthology.orchestra.run.smithi076.stdout: "unhealthy_workers": 0 2024-04-26T21:59:23.260 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:23.260 INFO:teuthology.orchestra.run.smithi076.stdout: "finisher-MDSRank": { 2024-04-26T21:59:23.260 INFO:teuthology.orchestra.run.smithi076.stdout: "queue_len": 0, 2024-04-26T21:59:23.260 INFO:teuthology.orchestra.run.smithi076.stdout: "complete_latency": { 2024-04-26T21:59:23.260 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 51, 2024-04-26T21:59:23.260 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.002328813, 2024-04-26T21:59:23.260 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000045663 2024-04-26T21:59:23.260 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-26T21:59:23.260 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:23.260 INFO:teuthology.orchestra.run.smithi076.stdout: "finisher-PurgeQueue": { 2024-04-26T21:59:23.260 INFO:teuthology.orchestra.run.smithi076.stdout: "queue_len": 0, 2024-04-26T21:59:23.260 INFO:teuthology.orchestra.run.smithi076.stdout: "complete_latency": { 2024-04-26T21:59:23.260 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 2, 2024-04-26T21:59:23.261 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000013384, 2024-04-26T21:59:23.261 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000006692 2024-04-26T21:59:23.261 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-26T21:59:23.261 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:23.261 INFO:teuthology.orchestra.run.smithi076.stdout: "mds": { 2024-04-26T21:59:23.261 INFO:teuthology.orchestra.run.smithi076.stdout: "request": 2, 2024-04-26T21:59:23.261 INFO:teuthology.orchestra.run.smithi076.stdout: "reply": 2, 2024-04-26T21:59:23.261 INFO:teuthology.orchestra.run.smithi076.stdout: "reply_latency": { 2024-04-26T21:59:23.261 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 2, 2024-04-26T21:59:23.261 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.006917119, 2024-04-26T21:59:23.261 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.003458559 2024-04-26T21:59:23.261 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:23.261 INFO:teuthology.orchestra.run.smithi076.stdout: "slow_reply": 0, 2024-04-26T21:59:23.262 INFO:teuthology.orchestra.run.smithi076.stdout: "forward": 0, 2024-04-26T21:59:23.262 INFO:teuthology.orchestra.run.smithi076.stdout: "dir_fetch_complete": 0, 2024-04-26T21:59:23.262 INFO:teuthology.orchestra.run.smithi076.stdout: "dir_fetch_keys": 0, 2024-04-26T21:59:23.262 INFO:teuthology.orchestra.run.smithi076.stdout: "dir_commit": 11, 2024-04-26T21:59:23.262 INFO:teuthology.orchestra.run.smithi076.stdout: "dir_split": 0, 2024-04-26T21:59:23.262 INFO:teuthology.orchestra.run.smithi076.stdout: "dir_merge": 0, 2024-04-26T21:59:23.262 INFO:teuthology.orchestra.run.smithi076.stdout: "inodes": 12, 2024-04-26T21:59:23.262 INFO:teuthology.orchestra.run.smithi076.stdout: "inodes_top": 0, 2024-04-26T21:59:23.262 INFO:teuthology.orchestra.run.smithi076.stdout: "inodes_bottom": 12, 2024-04-26T21:59:23.262 INFO:teuthology.orchestra.run.smithi076.stdout: "inodes_pin_tail": 0, 2024-04-26T21:59:23.262 INFO:teuthology.orchestra.run.smithi076.stdout: "inodes_pinned": 12, 2024-04-26T21:59:23.262 INFO:teuthology.orchestra.run.smithi076.stdout: "inodes_expired": 0, 2024-04-26T21:59:23.262 INFO:teuthology.orchestra.run.smithi076.stdout: "inodes_with_caps": 2, 2024-04-26T21:59:23.263 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": 2, 2024-04-26T21:59:23.263 INFO:teuthology.orchestra.run.smithi076.stdout: "subtrees": 2, 2024-04-26T21:59:23.263 INFO:teuthology.orchestra.run.smithi076.stdout: "traverse": 4, 2024-04-26T21:59:23.263 INFO:teuthology.orchestra.run.smithi076.stdout: "traverse_hit": 3, 2024-04-26T21:59:23.263 INFO:teuthology.orchestra.run.smithi076.stdout: "traverse_forward": 0, 2024-04-26T21:59:23.263 INFO:teuthology.orchestra.run.smithi076.stdout: "traverse_discover": 1, 2024-04-26T21:59:23.263 INFO:teuthology.orchestra.run.smithi076.stdout: "traverse_dir_fetch": 0, 2024-04-26T21:59:23.263 INFO:teuthology.orchestra.run.smithi076.stdout: "traverse_remote_ino": 0, 2024-04-26T21:59:23.263 INFO:teuthology.orchestra.run.smithi076.stdout: "traverse_lock": 0, 2024-04-26T21:59:23.263 INFO:teuthology.orchestra.run.smithi076.stdout: "load_cent": 0, 2024-04-26T21:59:23.263 INFO:teuthology.orchestra.run.smithi076.stdout: "q": 0, 2024-04-26T21:59:23.263 INFO:teuthology.orchestra.run.smithi076.stdout: "exported": 0, 2024-04-26T21:59:23.263 INFO:teuthology.orchestra.run.smithi076.stdout: "exported_inodes": 0, 2024-04-26T21:59:23.263 INFO:teuthology.orchestra.run.smithi076.stdout: "imported": 0, 2024-04-26T21:59:23.263 INFO:teuthology.orchestra.run.smithi076.stdout: "imported_inodes": 0, 2024-04-26T21:59:23.264 INFO:teuthology.orchestra.run.smithi076.stdout: "openino_dir_fetch": 0, 2024-04-26T21:59:23.264 INFO:teuthology.orchestra.run.smithi076.stdout: "openino_backtrace_fetch": 0, 2024-04-26T21:59:23.264 INFO:teuthology.orchestra.run.smithi076.stdout: "openino_peer_discover": 0, 2024-04-26T21:59:23.264 INFO:teuthology.orchestra.run.smithi076.stdout: "root_rfiles": 8, 2024-04-26T21:59:23.264 INFO:teuthology.orchestra.run.smithi076.stdout: "root_rbytes": 195290530, 2024-04-26T21:59:23.264 INFO:teuthology.orchestra.run.smithi076.stdout: "root_rsnaps": 0, 2024-04-26T21:59:23.264 INFO:teuthology.orchestra.run.smithi076.stdout: "scrub_backtrace_fetch": 0, 2024-04-26T21:59:23.264 INFO:teuthology.orchestra.run.smithi076.stdout: "scrub_set_tag": 0, 2024-04-26T21:59:23.264 INFO:teuthology.orchestra.run.smithi076.stdout: "scrub_backtrace_repaired": 0, 2024-04-26T21:59:23.264 INFO:teuthology.orchestra.run.smithi076.stdout: "scrub_inotable_repaired": 0, 2024-04-26T21:59:23.264 INFO:teuthology.orchestra.run.smithi076.stdout: "scrub_dir_inodes": 0, 2024-04-26T21:59:23.264 INFO:teuthology.orchestra.run.smithi076.stdout: "scrub_dir_base_inodes": 0, 2024-04-26T21:59:23.264 INFO:teuthology.orchestra.run.smithi076.stdout: "scrub_dirfrag_rstats": 0, 2024-04-26T21:59:23.264 INFO:teuthology.orchestra.run.smithi076.stdout: "scrub_file_inodes": 0, 2024-04-26T21:59:23.264 INFO:teuthology.orchestra.run.smithi076.stdout: "handle_inode_file_caps": 0, 2024-04-26T21:59:23.265 INFO:teuthology.orchestra.run.smithi076.stdout: "ceph_cap_op_revoke": 0, 2024-04-26T21:59:23.265 INFO:teuthology.orchestra.run.smithi076.stdout: "ceph_cap_op_grant": 0, 2024-04-26T21:59:23.265 INFO:teuthology.orchestra.run.smithi076.stdout: "ceph_cap_op_trunc": 0, 2024-04-26T21:59:23.265 INFO:teuthology.orchestra.run.smithi076.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-26T21:59:23.265 INFO:teuthology.orchestra.run.smithi076.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-26T21:59:23.265 INFO:teuthology.orchestra.run.smithi076.stdout: "handle_client_caps": 0, 2024-04-26T21:59:23.265 INFO:teuthology.orchestra.run.smithi076.stdout: "handle_client_caps_dirty": 0, 2024-04-26T21:59:23.265 INFO:teuthology.orchestra.run.smithi076.stdout: "handle_client_cap_release": 0, 2024-04-26T21:59:23.265 INFO:teuthology.orchestra.run.smithi076.stdout: "process_request_cap_release": 0 2024-04-26T21:59:23.265 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:23.265 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_cache": { 2024-04-26T21:59:23.265 INFO:teuthology.orchestra.run.smithi076.stdout: "dir_update": 0, 2024-04-26T21:59:23.265 INFO:teuthology.orchestra.run.smithi076.stdout: "dir_update_receipt": 0, 2024-04-26T21:59:23.265 INFO:teuthology.orchestra.run.smithi076.stdout: "dir_try_discover": 0, 2024-04-26T21:59:23.265 INFO:teuthology.orchestra.run.smithi076.stdout: "dir_send_discover": 2, 2024-04-26T21:59:23.266 INFO:teuthology.orchestra.run.smithi076.stdout: "dir_handle_discover": 0, 2024-04-26T21:59:23.266 INFO:teuthology.orchestra.run.smithi076.stdout: "num_strays": 0, 2024-04-26T21:59:23.266 INFO:teuthology.orchestra.run.smithi076.stdout: "num_strays_delayed": 0, 2024-04-26T21:59:23.266 INFO:teuthology.orchestra.run.smithi076.stdout: "num_strays_enqueuing": 0, 2024-04-26T21:59:23.266 INFO:teuthology.orchestra.run.smithi076.stdout: "strays_created": 0, 2024-04-26T21:59:23.266 INFO:teuthology.orchestra.run.smithi076.stdout: "strays_enqueued": 0, 2024-04-26T21:59:23.266 INFO:teuthology.orchestra.run.smithi076.stdout: "strays_reintegrated": 0, 2024-04-26T21:59:23.266 INFO:teuthology.orchestra.run.smithi076.stdout: "strays_migrated": 0, 2024-04-26T21:59:23.266 INFO:teuthology.orchestra.run.smithi076.stdout: "num_recovering_processing": 0, 2024-04-26T21:59:23.266 INFO:teuthology.orchestra.run.smithi076.stdout: "num_recovering_enqueued": 0, 2024-04-26T21:59:23.266 INFO:teuthology.orchestra.run.smithi076.stdout: "num_recovering_prioritized": 0, 2024-04-26T21:59:23.266 INFO:teuthology.orchestra.run.smithi076.stdout: "recovery_started": 0, 2024-04-26T21:59:23.267 INFO:teuthology.orchestra.run.smithi076.stdout: "recovery_completed": 0, 2024-04-26T21:59:23.267 INFO:teuthology.orchestra.run.smithi076.stdout: "ireq_quiesce_path": 0, 2024-04-26T21:59:23.267 INFO:teuthology.orchestra.run.smithi076.stdout: "ireq_quiesce_inode": 0, 2024-04-26T21:59:23.267 INFO:teuthology.orchestra.run.smithi076.stdout: "ireq_enqueue_scrub": 0, 2024-04-26T21:59:23.267 INFO:teuthology.orchestra.run.smithi076.stdout: "ireq_exportdir": 0, 2024-04-26T21:59:23.267 INFO:teuthology.orchestra.run.smithi076.stdout: "ireq_flush": 0, 2024-04-26T21:59:23.267 INFO:teuthology.orchestra.run.smithi076.stdout: "ireq_fragmentdir": 0, 2024-04-26T21:59:23.267 INFO:teuthology.orchestra.run.smithi076.stdout: "ireq_fragstats": 0, 2024-04-26T21:59:23.267 INFO:teuthology.orchestra.run.smithi076.stdout: "ireq_inodestats": 0 2024-04-26T21:59:23.267 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:23.267 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_log": { 2024-04-26T21:59:23.267 INFO:teuthology.orchestra.run.smithi076.stdout: "evlrg": 0, 2024-04-26T21:59:23.267 INFO:teuthology.orchestra.run.smithi076.stdout: "evadd": 3, 2024-04-26T21:59:23.267 INFO:teuthology.orchestra.run.smithi076.stdout: "evex": 0, 2024-04-26T21:59:23.267 INFO:teuthology.orchestra.run.smithi076.stdout: "evtrm": 0, 2024-04-26T21:59:23.267 INFO:teuthology.orchestra.run.smithi076.stdout: "ev": 3, 2024-04-26T21:59:23.268 INFO:teuthology.orchestra.run.smithi076.stdout: "evexg": 0, 2024-04-26T21:59:23.268 INFO:teuthology.orchestra.run.smithi076.stdout: "evexd": 0, 2024-04-26T21:59:23.268 INFO:teuthology.orchestra.run.smithi076.stdout: "segadd": 2, 2024-04-26T21:59:23.268 INFO:teuthology.orchestra.run.smithi076.stdout: "segex": 0, 2024-04-26T21:59:23.268 INFO:teuthology.orchestra.run.smithi076.stdout: "segtrm": 0, 2024-04-26T21:59:23.268 INFO:teuthology.orchestra.run.smithi076.stdout: "seg": 2, 2024-04-26T21:59:23.268 INFO:teuthology.orchestra.run.smithi076.stdout: "segmjr": 2, 2024-04-26T21:59:23.268 INFO:teuthology.orchestra.run.smithi076.stdout: "segexg": 0, 2024-04-26T21:59:23.268 INFO:teuthology.orchestra.run.smithi076.stdout: "segexd": 0, 2024-04-26T21:59:23.268 INFO:teuthology.orchestra.run.smithi076.stdout: "expos": 4194304, 2024-04-26T21:59:23.268 INFO:teuthology.orchestra.run.smithi076.stdout: "wrpos": 4195385, 2024-04-26T21:59:23.268 INFO:teuthology.orchestra.run.smithi076.stdout: "rdpos": 0, 2024-04-26T21:59:23.268 INFO:teuthology.orchestra.run.smithi076.stdout: "jlat": { 2024-04-26T21:59:23.268 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 2, 2024-04-26T21:59:23.268 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.008843613, 2024-04-26T21:59:23.269 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.004421806 2024-04-26T21:59:23.269 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:23.269 INFO:teuthology.orchestra.run.smithi076.stdout: "replayed": 0 2024-04-26T21:59:23.269 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:23.269 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_mem": { 2024-04-26T21:59:23.269 INFO:teuthology.orchestra.run.smithi076.stdout: "ino": 15, 2024-04-26T21:59:23.269 INFO:teuthology.orchestra.run.smithi076.stdout: "ino+": 15, 2024-04-26T21:59:23.269 INFO:teuthology.orchestra.run.smithi076.stdout: "ino-": 0, 2024-04-26T21:59:23.269 INFO:teuthology.orchestra.run.smithi076.stdout: "dir": 13, 2024-04-26T21:59:23.269 INFO:teuthology.orchestra.run.smithi076.stdout: "dir+": 14, 2024-04-26T21:59:23.269 INFO:teuthology.orchestra.run.smithi076.stdout: "dir-": 1, 2024-04-26T21:59:23.269 INFO:teuthology.orchestra.run.smithi076.stdout: "dn": 12, 2024-04-26T21:59:23.269 INFO:teuthology.orchestra.run.smithi076.stdout: "dn+": 12, 2024-04-26T21:59:23.269 INFO:teuthology.orchestra.run.smithi076.stdout: "dn-": 0, 2024-04-26T21:59:23.269 INFO:teuthology.orchestra.run.smithi076.stdout: "cap": 2, 2024-04-26T21:59:23.270 INFO:teuthology.orchestra.run.smithi076.stdout: "cap+": 2, 2024-04-26T21:59:23.270 INFO:teuthology.orchestra.run.smithi076.stdout: "cap-": 0, 2024-04-26T21:59:23.270 INFO:teuthology.orchestra.run.smithi076.stdout: "rss": 50816, 2024-04-26T21:59:23.270 INFO:teuthology.orchestra.run.smithi076.stdout: "heap": 223608 2024-04-26T21:59:23.270 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:23.270 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_server": { 2024-04-26T21:59:23.270 INFO:teuthology.orchestra.run.smithi076.stdout: "dispatch_client_request": 6, 2024-04-26T21:59:23.270 INFO:teuthology.orchestra.run.smithi076.stdout: "dispatch_server_request": 0, 2024-04-26T21:59:23.270 INFO:teuthology.orchestra.run.smithi076.stdout: "handle_client_request": 2, 2024-04-26T21:59:23.270 INFO:teuthology.orchestra.run.smithi076.stdout: "handle_client_session": 5, 2024-04-26T21:59:23.270 INFO:teuthology.orchestra.run.smithi076.stdout: "handle_peer_request": 1, 2024-04-26T21:59:23.270 INFO:teuthology.orchestra.run.smithi076.stdout: "req_create_latency": { 2024-04-26T21:59:23.270 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-04-26T21:59:23.270 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-04-26T21:59:23.271 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-04-26T21:59:23.271 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:23.271 INFO:teuthology.orchestra.run.smithi076.stdout: "req_getattr_latency": { 2024-04-26T21:59:23.271 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 2, 2024-04-26T21:59:23.271 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.006917119, 2024-04-26T21:59:23.271 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.003458559 2024-04-26T21:59:23.271 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:23.271 INFO:teuthology.orchestra.run.smithi076.stdout: "req_getfilelock_latency": { 2024-04-26T21:59:23.271 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-04-26T21:59:23.271 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-04-26T21:59:23.271 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-04-26T21:59:23.271 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:23.271 INFO:teuthology.orchestra.run.smithi076.stdout: "req_link_latency": { 2024-04-26T21:59:23.271 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-04-26T21:59:23.272 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-04-26T21:59:23.272 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-04-26T21:59:23.272 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:23.272 INFO:teuthology.orchestra.run.smithi076.stdout: "req_lookup_latency": { 2024-04-26T21:59:23.272 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-04-26T21:59:23.272 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-04-26T21:59:23.272 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-04-26T21:59:23.272 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:23.272 INFO:teuthology.orchestra.run.smithi076.stdout: "req_lookuphash_latency": { 2024-04-26T21:59:23.272 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-04-26T21:59:23.272 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-04-26T21:59:23.272 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-04-26T21:59:23.272 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:23.272 INFO:teuthology.orchestra.run.smithi076.stdout: "req_lookupino_latency": { 2024-04-26T21:59:23.272 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-04-26T21:59:23.273 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-04-26T21:59:23.273 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-04-26T21:59:23.273 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:23.273 INFO:teuthology.orchestra.run.smithi076.stdout: "req_lookupname_latency": { 2024-04-26T21:59:23.273 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-04-26T21:59:23.273 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-04-26T21:59:23.273 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-04-26T21:59:23.273 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:23.273 INFO:teuthology.orchestra.run.smithi076.stdout: "req_lookupparent_latency": { 2024-04-26T21:59:23.273 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-04-26T21:59:23.273 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-04-26T21:59:23.273 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-04-26T21:59:23.273 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:23.273 INFO:teuthology.orchestra.run.smithi076.stdout: "req_lookupsnap_latency": { 2024-04-26T21:59:23.273 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-04-26T21:59:23.274 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-04-26T21:59:23.274 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-04-26T21:59:23.274 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:23.274 INFO:teuthology.orchestra.run.smithi076.stdout: "req_lssnap_latency": { 2024-04-26T21:59:23.274 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-04-26T21:59:23.274 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-04-26T21:59:23.274 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-04-26T21:59:23.274 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:23.274 INFO:teuthology.orchestra.run.smithi076.stdout: "req_mkdir_latency": { 2024-04-26T21:59:23.274 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-04-26T21:59:23.275 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-04-26T21:59:23.275 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-04-26T21:59:23.275 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:23.275 INFO:teuthology.orchestra.run.smithi076.stdout: "req_mknod_latency": { 2024-04-26T21:59:23.275 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-04-26T21:59:23.275 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-04-26T21:59:23.275 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-04-26T21:59:23.275 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:23.275 INFO:teuthology.orchestra.run.smithi076.stdout: "req_mksnap_latency": { 2024-04-26T21:59:23.275 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-04-26T21:59:23.275 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-04-26T21:59:23.276 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-04-26T21:59:23.276 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:23.276 INFO:teuthology.orchestra.run.smithi076.stdout: "req_open_latency": { 2024-04-26T21:59:23.276 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-04-26T21:59:23.276 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-04-26T21:59:23.276 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-04-26T21:59:23.276 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:23.276 INFO:teuthology.orchestra.run.smithi076.stdout: "req_readdir_latency": { 2024-04-26T21:59:23.276 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-04-26T21:59:23.276 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-04-26T21:59:23.276 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-04-26T21:59:23.276 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:23.276 INFO:teuthology.orchestra.run.smithi076.stdout: "req_rename_latency": { 2024-04-26T21:59:23.276 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-04-26T21:59:23.276 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-04-26T21:59:23.277 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-04-26T21:59:23.277 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:23.277 INFO:teuthology.orchestra.run.smithi076.stdout: "req_renamesnap_latency": { 2024-04-26T21:59:23.277 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-04-26T21:59:23.277 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-04-26T21:59:23.277 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-04-26T21:59:23.277 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:23.277 INFO:teuthology.orchestra.run.smithi076.stdout: "req_snapdiff_latency": { 2024-04-26T21:59:23.277 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-04-26T21:59:23.277 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-04-26T21:59:23.277 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-04-26T21:59:23.277 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:23.277 INFO:teuthology.orchestra.run.smithi076.stdout: "req_rmdir_latency": { 2024-04-26T21:59:23.277 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-04-26T21:59:23.277 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-04-26T21:59:23.278 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-04-26T21:59:23.278 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:23.278 INFO:teuthology.orchestra.run.smithi076.stdout: "req_rmsnap_latency": { 2024-04-26T21:59:23.278 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-04-26T21:59:23.278 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-04-26T21:59:23.278 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-04-26T21:59:23.278 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:23.278 INFO:teuthology.orchestra.run.smithi076.stdout: "req_rmxattr_latency": { 2024-04-26T21:59:23.278 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-04-26T21:59:23.278 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-04-26T21:59:23.278 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-04-26T21:59:23.278 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:23.278 INFO:teuthology.orchestra.run.smithi076.stdout: "req_setattr_latency": { 2024-04-26T21:59:23.279 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-04-26T21:59:23.279 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-04-26T21:59:23.279 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-04-26T21:59:23.279 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:23.279 INFO:teuthology.orchestra.run.smithi076.stdout: "req_setdirlayout_latency": { 2024-04-26T21:59:23.279 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-04-26T21:59:23.279 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-04-26T21:59:23.279 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-04-26T21:59:23.279 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:23.279 INFO:teuthology.orchestra.run.smithi076.stdout: "req_setfilelock_latency": { 2024-04-26T21:59:23.279 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-04-26T21:59:23.279 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-04-26T21:59:23.279 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-04-26T21:59:23.279 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:23.279 INFO:teuthology.orchestra.run.smithi076.stdout: "req_setlayout_latency": { 2024-04-26T21:59:23.280 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-04-26T21:59:23.280 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-04-26T21:59:23.280 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-04-26T21:59:23.280 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:23.280 INFO:teuthology.orchestra.run.smithi076.stdout: "req_setxattr_latency": { 2024-04-26T21:59:23.280 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-04-26T21:59:23.280 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-04-26T21:59:23.280 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-04-26T21:59:23.280 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:23.280 INFO:teuthology.orchestra.run.smithi076.stdout: "req_symlink_latency": { 2024-04-26T21:59:23.280 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-04-26T21:59:23.280 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-04-26T21:59:23.280 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-04-26T21:59:23.280 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:23.281 INFO:teuthology.orchestra.run.smithi076.stdout: "req_unlink_latency": { 2024-04-26T21:59:23.281 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-04-26T21:59:23.281 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-04-26T21:59:23.281 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-04-26T21:59:23.281 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:23.281 INFO:teuthology.orchestra.run.smithi076.stdout: "cap_revoke_eviction": 0, 2024-04-26T21:59:23.281 INFO:teuthology.orchestra.run.smithi076.stdout: "cap_acquisition_throttle": 0, 2024-04-26T21:59:23.281 INFO:teuthology.orchestra.run.smithi076.stdout: "req_getvxattr_latency": { 2024-04-26T21:59:23.281 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-04-26T21:59:23.281 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-04-26T21:59:23.281 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-04-26T21:59:23.281 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-26T21:59:23.281 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:23.281 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_sessions": { 2024-04-26T21:59:23.282 INFO:teuthology.orchestra.run.smithi076.stdout: "session_count": 1, 2024-04-26T21:59:23.282 INFO:teuthology.orchestra.run.smithi076.stdout: "session_add": 1, 2024-04-26T21:59:23.282 INFO:teuthology.orchestra.run.smithi076.stdout: "session_remove": 0, 2024-04-26T21:59:23.282 INFO:teuthology.orchestra.run.smithi076.stdout: "sessions_open": 1, 2024-04-26T21:59:23.282 INFO:teuthology.orchestra.run.smithi076.stdout: "sessions_stale": 0, 2024-04-26T21:59:23.282 INFO:teuthology.orchestra.run.smithi076.stdout: "total_load": 1, 2024-04-26T21:59:23.282 INFO:teuthology.orchestra.run.smithi076.stdout: "average_load": 1, 2024-04-26T21:59:23.282 INFO:teuthology.orchestra.run.smithi076.stdout: "avg_session_uptime": 237, 2024-04-26T21:59:23.282 INFO:teuthology.orchestra.run.smithi076.stdout: "mdthresh_evicted": 0 2024-04-26T21:59:23.282 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:23.282 INFO:teuthology.orchestra.run.smithi076.stdout: "mempool": { 2024-04-26T21:59:23.282 INFO:teuthology.orchestra.run.smithi076.stdout: "bloom_filter_bytes": 0, 2024-04-26T21:59:23.282 INFO:teuthology.orchestra.run.smithi076.stdout: "bloom_filter_items": 0, 2024-04-26T21:59:23.283 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_alloc_bytes": 0, 2024-04-26T21:59:23.283 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_alloc_items": 0, 2024-04-26T21:59:23.283 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_data_bytes": 0, 2024-04-26T21:59:23.283 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_data_items": 0, 2024-04-26T21:59:23.283 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-26T21:59:23.283 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_onode_items": 0, 2024-04-26T21:59:23.283 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-26T21:59:23.283 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_meta_items": 0, 2024-04-26T21:59:23.283 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_other_bytes": 0, 2024-04-26T21:59:23.283 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_other_items": 0, 2024-04-26T21:59:23.283 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-26T21:59:23.283 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_buffer_items": 0, 2024-04-26T21:59:23.283 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_extent_bytes": 0, 2024-04-26T21:59:23.283 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_extent_items": 0, 2024-04-26T21:59:23.284 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_blob_bytes": 0, 2024-04-26T21:59:23.284 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_blob_items": 0, 2024-04-26T21:59:23.284 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-26T21:59:23.284 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_shared_blob_items": 0, 2024-04-26T21:59:23.284 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-26T21:59:23.284 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_inline_bl_items": 0, 2024-04-26T21:59:23.284 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_fsck_bytes": 0, 2024-04-26T21:59:23.284 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_fsck_items": 0, 2024-04-26T21:59:23.284 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_txc_bytes": 0, 2024-04-26T21:59:23.284 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_txc_items": 0, 2024-04-26T21:59:23.284 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-26T21:59:23.284 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_writing_deferred_items": 0, 2024-04-26T21:59:23.284 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_writing_bytes": 0, 2024-04-26T21:59:23.285 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_writing_items": 0, 2024-04-26T21:59:23.285 INFO:teuthology.orchestra.run.smithi076.stdout: "bluefs_bytes": 0, 2024-04-26T21:59:23.285 INFO:teuthology.orchestra.run.smithi076.stdout: "bluefs_items": 0, 2024-04-26T21:59:23.285 INFO:teuthology.orchestra.run.smithi076.stdout: "bluefs_file_reader_bytes": 0, 2024-04-26T21:59:23.285 INFO:teuthology.orchestra.run.smithi076.stdout: "bluefs_file_reader_items": 0, 2024-04-26T21:59:23.285 INFO:teuthology.orchestra.run.smithi076.stdout: "bluefs_file_writer_bytes": 0, 2024-04-26T21:59:23.285 INFO:teuthology.orchestra.run.smithi076.stdout: "bluefs_file_writer_items": 0, 2024-04-26T21:59:23.285 INFO:teuthology.orchestra.run.smithi076.stdout: "buffer_anon_bytes": 254593, 2024-04-26T21:59:23.285 INFO:teuthology.orchestra.run.smithi076.stdout: "buffer_anon_items": 87, 2024-04-26T21:59:23.285 INFO:teuthology.orchestra.run.smithi076.stdout: "buffer_meta_bytes": 0, 2024-04-26T21:59:23.285 INFO:teuthology.orchestra.run.smithi076.stdout: "buffer_meta_items": 0, 2024-04-26T21:59:23.285 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_bytes": 0, 2024-04-26T21:59:23.285 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_items": 0, 2024-04-26T21:59:23.285 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_mapbl_bytes": 0, 2024-04-26T21:59:23.285 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_mapbl_items": 0, 2024-04-26T21:59:23.286 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_pglog_bytes": 0, 2024-04-26T21:59:23.286 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_pglog_items": 0, 2024-04-26T21:59:23.286 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_bytes": 8008, 2024-04-26T21:59:23.286 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_items": 168, 2024-04-26T21:59:23.286 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_mapping_bytes": 0, 2024-04-26T21:59:23.286 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_mapping_items": 0, 2024-04-26T21:59:23.286 INFO:teuthology.orchestra.run.smithi076.stdout: "pgmap_bytes": 0, 2024-04-26T21:59:23.286 INFO:teuthology.orchestra.run.smithi076.stdout: "pgmap_items": 0, 2024-04-26T21:59:23.286 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_co_bytes": 60032, 2024-04-26T21:59:23.286 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_co_items": 233, 2024-04-26T21:59:23.286 INFO:teuthology.orchestra.run.smithi076.stdout: "unittest_1_bytes": 0, 2024-04-26T21:59:23.286 INFO:teuthology.orchestra.run.smithi076.stdout: "unittest_1_items": 0, 2024-04-26T21:59:23.286 INFO:teuthology.orchestra.run.smithi076.stdout: "unittest_2_bytes": 0, 2024-04-26T21:59:23.286 INFO:teuthology.orchestra.run.smithi076.stdout: "unittest_2_items": 0 2024-04-26T21:59:23.287 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:23.287 INFO:teuthology.orchestra.run.smithi076.stdout: "objecter": { 2024-04-26T21:59:23.287 INFO:teuthology.orchestra.run.smithi076.stdout: "op_active": 0, 2024-04-26T21:59:23.287 INFO:teuthology.orchestra.run.smithi076.stdout: "op_laggy": 0, 2024-04-26T21:59:23.287 INFO:teuthology.orchestra.run.smithi076.stdout: "op_send": 38, 2024-04-26T21:59:23.287 INFO:teuthology.orchestra.run.smithi076.stdout: "op_send_bytes": 11448, 2024-04-26T21:59:23.287 INFO:teuthology.orchestra.run.smithi076.stdout: "op_resend": 33, 2024-04-26T21:59:23.287 INFO:teuthology.orchestra.run.smithi076.stdout: "op_reply": 38, 2024-04-26T21:59:23.287 INFO:teuthology.orchestra.run.smithi076.stdout: "op_latency": { 2024-04-26T21:59:23.287 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 38, 2024-04-26T21:59:23.287 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.552983676, 2024-04-26T21:59:23.287 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.014552202 2024-04-26T21:59:23.287 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:23.287 INFO:teuthology.orchestra.run.smithi076.stdout: "op_inflight": 0, 2024-04-26T21:59:23.287 INFO:teuthology.orchestra.run.smithi076.stdout: "oplen_avg": { 2024-04-26T21:59:23.288 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 38, 2024-04-26T21:59:23.288 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 61 2024-04-26T21:59:23.288 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:23.288 INFO:teuthology.orchestra.run.smithi076.stdout: "op": 38, 2024-04-26T21:59:23.288 INFO:teuthology.orchestra.run.smithi076.stdout: "op_r": 0, 2024-04-26T21:59:23.288 INFO:teuthology.orchestra.run.smithi076.stdout: "op_w": 38, 2024-04-26T21:59:23.288 INFO:teuthology.orchestra.run.smithi076.stdout: "op_rmw": 0, 2024-04-26T21:59:23.288 INFO:teuthology.orchestra.run.smithi076.stdout: "op_pg": 0, 2024-04-26T21:59:23.288 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_stat": 0, 2024-04-26T21:59:23.288 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_create": 10, 2024-04-26T21:59:23.288 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_read": 0, 2024-04-26T21:59:23.288 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_write": 2, 2024-04-26T21:59:23.288 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_writefull": 6, 2024-04-26T21:59:23.288 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_writesame": 0, 2024-04-26T21:59:23.288 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_append": 0, 2024-04-26T21:59:23.289 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_zero": 0, 2024-04-26T21:59:23.289 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_truncate": 0, 2024-04-26T21:59:23.289 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_delete": 6, 2024-04-26T21:59:23.289 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_mapext": 0, 2024-04-26T21:59:23.289 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_sparse_read": 0, 2024-04-26T21:59:23.289 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_clonerange": 0, 2024-04-26T21:59:23.289 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_getxattr": 0, 2024-04-26T21:59:23.289 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_setxattr": 20, 2024-04-26T21:59:23.289 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_cmpxattr": 0, 2024-04-26T21:59:23.289 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_rmxattr": 0, 2024-04-26T21:59:23.289 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_resetxattrs": 0, 2024-04-26T21:59:23.289 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_call": 0, 2024-04-26T21:59:23.289 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_watch": 0, 2024-04-26T21:59:23.289 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_notify": 0, 2024-04-26T21:59:23.289 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_src_cmpxattr": 0, 2024-04-26T21:59:23.290 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_pgls": 0, 2024-04-26T21:59:23.290 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_pgls_filter": 0, 2024-04-26T21:59:23.290 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_other": 0, 2024-04-26T21:59:23.290 INFO:teuthology.orchestra.run.smithi076.stdout: "linger_active": 0, 2024-04-26T21:59:23.290 INFO:teuthology.orchestra.run.smithi076.stdout: "linger_send": 0, 2024-04-26T21:59:23.290 INFO:teuthology.orchestra.run.smithi076.stdout: "linger_resend": 0, 2024-04-26T21:59:23.290 INFO:teuthology.orchestra.run.smithi076.stdout: "linger_ping": 0, 2024-04-26T21:59:23.290 INFO:teuthology.orchestra.run.smithi076.stdout: "poolop_active": 0, 2024-04-26T21:59:23.290 INFO:teuthology.orchestra.run.smithi076.stdout: "poolop_send": 0, 2024-04-26T21:59:23.290 INFO:teuthology.orchestra.run.smithi076.stdout: "poolop_resend": 0, 2024-04-26T21:59:23.290 INFO:teuthology.orchestra.run.smithi076.stdout: "poolstat_active": 0, 2024-04-26T21:59:23.290 INFO:teuthology.orchestra.run.smithi076.stdout: "poolstat_send": 0, 2024-04-26T21:59:23.290 INFO:teuthology.orchestra.run.smithi076.stdout: "poolstat_resend": 0, 2024-04-26T21:59:23.291 INFO:teuthology.orchestra.run.smithi076.stdout: "statfs_active": 0, 2024-04-26T21:59:23.291 INFO:teuthology.orchestra.run.smithi076.stdout: "statfs_send": 0, 2024-04-26T21:59:23.291 INFO:teuthology.orchestra.run.smithi076.stdout: "statfs_resend": 0, 2024-04-26T21:59:23.291 INFO:teuthology.orchestra.run.smithi076.stdout: "command_active": 0, 2024-04-26T21:59:23.291 INFO:teuthology.orchestra.run.smithi076.stdout: "command_send": 0, 2024-04-26T21:59:23.291 INFO:teuthology.orchestra.run.smithi076.stdout: "command_resend": 0, 2024-04-26T21:59:23.291 INFO:teuthology.orchestra.run.smithi076.stdout: "map_epoch": 91, 2024-04-26T21:59:23.291 INFO:teuthology.orchestra.run.smithi076.stdout: "map_full": 0, 2024-04-26T21:59:23.291 INFO:teuthology.orchestra.run.smithi076.stdout: "map_inc": 4, 2024-04-26T21:59:23.291 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_sessions": 10, 2024-04-26T21:59:23.291 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_session_open": 10, 2024-04-26T21:59:23.291 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_session_close": 0, 2024-04-26T21:59:23.291 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_laggy": 0, 2024-04-26T21:59:23.292 INFO:teuthology.orchestra.run.smithi076.stdout: "omap_wr": 16, 2024-04-26T21:59:23.292 INFO:teuthology.orchestra.run.smithi076.stdout: "omap_rd": 0, 2024-04-26T21:59:23.292 INFO:teuthology.orchestra.run.smithi076.stdout: "omap_del": 1 2024-04-26T21:59:23.292 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:23.292 INFO:teuthology.orchestra.run.smithi076.stdout: "oft": { 2024-04-26T21:59:23.292 INFO:teuthology.orchestra.run.smithi076.stdout: "omap_total_objs": 1, 2024-04-26T21:59:23.292 INFO:teuthology.orchestra.run.smithi076.stdout: "omap_total_kv_pairs": 0, 2024-04-26T21:59:23.292 INFO:teuthology.orchestra.run.smithi076.stdout: "omap_total_updates": 0, 2024-04-26T21:59:23.292 INFO:teuthology.orchestra.run.smithi076.stdout: "omap_total_removes": 0 2024-04-26T21:59:23.292 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:23.292 INFO:teuthology.orchestra.run.smithi076.stdout: "osd-slow-ops": { 2024-04-26T21:59:23.292 INFO:teuthology.orchestra.run.smithi076.stdout: "slow_ops_count": 0 2024-04-26T21:59:23.292 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:23.293 INFO:teuthology.orchestra.run.smithi076.stdout: "purge_queue": { 2024-04-26T21:59:23.293 INFO:teuthology.orchestra.run.smithi076.stdout: "pq_executing_ops": 0, 2024-04-26T21:59:23.293 INFO:teuthology.orchestra.run.smithi076.stdout: "pq_executing_ops_high_water": 0, 2024-04-26T21:59:23.293 INFO:teuthology.orchestra.run.smithi076.stdout: "pq_executing": 0, 2024-04-26T21:59:23.293 INFO:teuthology.orchestra.run.smithi076.stdout: "pq_executing_high_water": 0, 2024-04-26T21:59:23.293 INFO:teuthology.orchestra.run.smithi076.stdout: "pq_executed_ops": 0, 2024-04-26T21:59:23.293 INFO:teuthology.orchestra.run.smithi076.stdout: "pq_executed": 0, 2024-04-26T21:59:23.293 INFO:teuthology.orchestra.run.smithi076.stdout: "pq_item_in_journal": 0 2024-04-26T21:59:23.293 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:23.293 INFO:teuthology.orchestra.run.smithi076.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-26T21:59:23.293 INFO:teuthology.orchestra.run.smithi076.stdout: "val": 0, 2024-04-26T21:59:23.293 INFO:teuthology.orchestra.run.smithi076.stdout: "max": 104857600, 2024-04-26T21:59:23.293 INFO:teuthology.orchestra.run.smithi076.stdout: "get_started": 0, 2024-04-26T21:59:23.294 INFO:teuthology.orchestra.run.smithi076.stdout: "get": 455, 2024-04-26T21:59:23.294 INFO:teuthology.orchestra.run.smithi076.stdout: "get_sum": 409848, 2024-04-26T21:59:23.294 INFO:teuthology.orchestra.run.smithi076.stdout: "get_or_fail_fail": 0, 2024-04-26T21:59:23.294 INFO:teuthology.orchestra.run.smithi076.stdout: "get_or_fail_success": 455, 2024-04-26T21:59:23.294 INFO:teuthology.orchestra.run.smithi076.stdout: "take": 0, 2024-04-26T21:59:23.294 INFO:teuthology.orchestra.run.smithi076.stdout: "take_sum": 0, 2024-04-26T21:59:23.294 INFO:teuthology.orchestra.run.smithi076.stdout: "put": 455, 2024-04-26T21:59:23.294 INFO:teuthology.orchestra.run.smithi076.stdout: "put_sum": 409848, 2024-04-26T21:59:23.294 INFO:teuthology.orchestra.run.smithi076.stdout: "wait": { 2024-04-26T21:59:23.294 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-04-26T21:59:23.294 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-04-26T21:59:23.294 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-04-26T21:59:23.294 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-26T21:59:23.295 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:23.295 INFO:teuthology.orchestra.run.smithi076.stdout: "throttle-objecter_bytes": { 2024-04-26T21:59:23.295 INFO:teuthology.orchestra.run.smithi076.stdout: "val": 0, 2024-04-26T21:59:23.295 INFO:teuthology.orchestra.run.smithi076.stdout: "max": 104857600, 2024-04-26T21:59:23.295 INFO:teuthology.orchestra.run.smithi076.stdout: "get_started": 0, 2024-04-26T21:59:23.295 INFO:teuthology.orchestra.run.smithi076.stdout: "get": 0, 2024-04-26T21:59:23.295 INFO:teuthology.orchestra.run.smithi076.stdout: "get_sum": 0, 2024-04-26T21:59:23.295 INFO:teuthology.orchestra.run.smithi076.stdout: "get_or_fail_fail": 0, 2024-04-26T21:59:23.295 INFO:teuthology.orchestra.run.smithi076.stdout: "get_or_fail_success": 0, 2024-04-26T21:59:23.295 INFO:teuthology.orchestra.run.smithi076.stdout: "take": 38, 2024-04-26T21:59:23.295 INFO:teuthology.orchestra.run.smithi076.stdout: "take_sum": 11448, 2024-04-26T21:59:23.295 INFO:teuthology.orchestra.run.smithi076.stdout: "put": 38, 2024-04-26T21:59:23.295 INFO:teuthology.orchestra.run.smithi076.stdout: "put_sum": 11448, 2024-04-26T21:59:23.295 INFO:teuthology.orchestra.run.smithi076.stdout: "wait": { 2024-04-26T21:59:23.296 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-04-26T21:59:23.296 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-04-26T21:59:23.296 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-04-26T21:59:23.296 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-26T21:59:23.296 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:23.296 INFO:teuthology.orchestra.run.smithi076.stdout: "throttle-objecter_ops": { 2024-04-26T21:59:23.296 INFO:teuthology.orchestra.run.smithi076.stdout: "val": 0, 2024-04-26T21:59:23.296 INFO:teuthology.orchestra.run.smithi076.stdout: "max": 1024, 2024-04-26T21:59:23.296 INFO:teuthology.orchestra.run.smithi076.stdout: "get_started": 0, 2024-04-26T21:59:23.296 INFO:teuthology.orchestra.run.smithi076.stdout: "get": 0, 2024-04-26T21:59:23.296 INFO:teuthology.orchestra.run.smithi076.stdout: "get_sum": 0, 2024-04-26T21:59:23.296 INFO:teuthology.orchestra.run.smithi076.stdout: "get_or_fail_fail": 0, 2024-04-26T21:59:23.296 INFO:teuthology.orchestra.run.smithi076.stdout: "get_or_fail_success": 0, 2024-04-26T21:59:23.297 INFO:teuthology.orchestra.run.smithi076.stdout: "take": 38, 2024-04-26T21:59:23.297 INFO:teuthology.orchestra.run.smithi076.stdout: "take_sum": 38, 2024-04-26T21:59:23.297 INFO:teuthology.orchestra.run.smithi076.stdout: "put": 38, 2024-04-26T21:59:23.297 INFO:teuthology.orchestra.run.smithi076.stdout: "put_sum": 38, 2024-04-26T21:59:23.297 INFO:teuthology.orchestra.run.smithi076.stdout: "wait": { 2024-04-26T21:59:23.297 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-04-26T21:59:23.297 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-04-26T21:59:23.297 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-04-26T21:59:23.297 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-26T21:59:23.297 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:23.297 INFO:teuthology.orchestra.run.smithi076.stdout: "throttle-write_buf_throttle": { 2024-04-26T21:59:23.297 INFO:teuthology.orchestra.run.smithi076.stdout: "val": 0, 2024-04-26T21:59:23.297 INFO:teuthology.orchestra.run.smithi076.stdout: "max": 3758096384, 2024-04-26T21:59:23.298 INFO:teuthology.orchestra.run.smithi076.stdout: "get_started": 0, 2024-04-26T21:59:23.298 INFO:teuthology.orchestra.run.smithi076.stdout: "get": 0, 2024-04-26T21:59:23.298 INFO:teuthology.orchestra.run.smithi076.stdout: "get_sum": 0, 2024-04-26T21:59:23.298 INFO:teuthology.orchestra.run.smithi076.stdout: "get_or_fail_fail": 0, 2024-04-26T21:59:23.298 INFO:teuthology.orchestra.run.smithi076.stdout: "get_or_fail_success": 0, 2024-04-26T21:59:23.298 INFO:teuthology.orchestra.run.smithi076.stdout: "take": 0, 2024-04-26T21:59:23.298 INFO:teuthology.orchestra.run.smithi076.stdout: "take_sum": 0, 2024-04-26T21:59:23.298 INFO:teuthology.orchestra.run.smithi076.stdout: "put": 0, 2024-04-26T21:59:23.298 INFO:teuthology.orchestra.run.smithi076.stdout: "put_sum": 0, 2024-04-26T21:59:23.298 INFO:teuthology.orchestra.run.smithi076.stdout: "wait": { 2024-04-26T21:59:23.298 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-04-26T21:59:23.298 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-04-26T21:59:23.298 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-04-26T21:59:23.298 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-26T21:59:23.299 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:23.299 INFO:teuthology.orchestra.run.smithi076.stdout: "throttle-write_buf_throttle-0x5602df02c500": { 2024-04-26T21:59:23.299 INFO:teuthology.orchestra.run.smithi076.stdout: "val": 0, 2024-04-26T21:59:23.299 INFO:teuthology.orchestra.run.smithi076.stdout: "max": 3758096384, 2024-04-26T21:59:23.299 INFO:teuthology.orchestra.run.smithi076.stdout: "get_started": 0, 2024-04-26T21:59:23.299 INFO:teuthology.orchestra.run.smithi076.stdout: "get": 3, 2024-04-26T21:59:23.299 INFO:teuthology.orchestra.run.smithi076.stdout: "get_sum": 1081, 2024-04-26T21:59:23.299 INFO:teuthology.orchestra.run.smithi076.stdout: "get_or_fail_fail": 0, 2024-04-26T21:59:23.299 INFO:teuthology.orchestra.run.smithi076.stdout: "get_or_fail_success": 3, 2024-04-26T21:59:23.299 INFO:teuthology.orchestra.run.smithi076.stdout: "take": 0, 2024-04-26T21:59:23.299 INFO:teuthology.orchestra.run.smithi076.stdout: "take_sum": 0, 2024-04-26T21:59:23.299 INFO:teuthology.orchestra.run.smithi076.stdout: "put": 2, 2024-04-26T21:59:23.299 INFO:teuthology.orchestra.run.smithi076.stdout: "put_sum": 1081, 2024-04-26T21:59:23.299 INFO:teuthology.orchestra.run.smithi076.stdout: "wait": { 2024-04-26T21:59:23.300 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-04-26T21:59:23.300 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-04-26T21:59:23.300 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-04-26T21:59:23.300 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-26T21:59:23.300 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-26T21:59:23.300 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-04-26T21:59:23.300 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:59:23 smithi076 ceph-mon[31823]: pgmap v567: 97 pgs: 97 active+clean; 1.1 MiB data, 656 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:59:23.321 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:59:23 smithi072 ceph-mon[28138]: pgmap v567: 97 pgs: 97 active+clean; 1.1 MiB data, 656 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:59:23.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:59:23 smithi164 ceph-mon[31916]: pgmap v567: 97 pgs: 97 active+clean; 1.1 MiB data, 656 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:59:24.227 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2024-04-26T21:59:24.227 INFO:tasks.check_counter:Daemon mds.h mds.imported=0 2024-04-26T21:59:24.227 DEBUG:tasks.check_counter:Getting stats from k 2024-04-26T21:59:24.227 DEBUG:teuthology.orchestra.run.smithi076:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid a1780950-0415-11ef-bc93-c7b262605968 -- ceph daemon mds.k perf dump 2024-04-26T21:59:24.662 INFO:teuthology.orchestra.run.smithi076.stderr:Inferring config /var/lib/ceph/a1780950-0415-11ef-bc93-c7b262605968/mon.b/config 2024-04-26T21:59:25.064 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:59:25 smithi076 ceph-mon[31823]: pgmap v568: 97 pgs: 97 active+clean; 1.1 MiB data, 656 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:59:25.321 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:59:25 smithi072 ceph-mon[28138]: pgmap v568: 97 pgs: 97 active+clean; 1.1 MiB data, 656 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:59:25.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:59:25 smithi164 ceph-mon[31916]: pgmap v568: 97 pgs: 97 active+clean; 1.1 MiB data, 656 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:59:26.276 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:59:26 smithi076 ceph-mon[31823]: pgmap v569: 97 pgs: 97 active+clean; 1.1 MiB data, 656 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-26T21:59:26.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:59:26 smithi164 ceph-mon[31916]: pgmap v569: 97 pgs: 97 active+clean; 1.1 MiB data, 656 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-26T21:59:26.571 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:59:26 smithi072 ceph-mon[28138]: pgmap v569: 97 pgs: 97 active+clean; 1.1 MiB data, 656 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-26T21:59:26.872 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-04-26T21:59:26.872 INFO:teuthology.orchestra.run.smithi076.stdout: "AsyncMessenger::Worker-0": { 2024-04-26T21:59:26.872 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_messages": 66, 2024-04-26T21:59:26.872 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_messages": 65, 2024-04-26T21:59:26.872 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_bytes": 24544, 2024-04-26T21:59:26.872 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_bytes": 165726, 2024-04-26T21:59:26.872 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_created_connections": 22, 2024-04-26T21:59:26.872 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_active_connections": 6, 2024-04-26T21:59:26.872 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_total_time": 0.034419700, 2024-04-26T21:59:26.872 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_send_time": 0.004605817, 2024-04-26T21:59:26.872 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_recv_time": 0.021333019, 2024-04-26T21:59:26.873 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_fast_dispatch_time": 0.000779338, 2024-04-26T21:59:26.873 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T21:59:26.873 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 64, 2024-04-26T21:59:26.873 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.045183601, 2024-04-26T21:59:26.873 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000705993 2024-04-26T21:59:26.873 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:26.873 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_handle_ack_lat": { 2024-04-26T21:59:26.873 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 67, 2024-04-26T21:59:26.873 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000024625, 2024-04-26T21:59:26.873 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000367 2024-04-26T21:59:26.873 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:26.873 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_encrypted_bytes": 4976, 2024-04-26T21:59:26.874 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_encrypted_bytes": 2208 2024-04-26T21:59:26.874 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:26.874 INFO:teuthology.orchestra.run.smithi076.stdout: "AsyncMessenger::Worker-1": { 2024-04-26T21:59:26.874 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_messages": 389, 2024-04-26T21:59:26.874 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_messages": 460, 2024-04-26T21:59:26.874 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_bytes": 425447, 2024-04-26T21:59:26.874 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_bytes": 256915, 2024-04-26T21:59:26.874 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_created_connections": 8, 2024-04-26T21:59:26.874 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_active_connections": 5, 2024-04-26T21:59:26.874 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_total_time": 0.137188002, 2024-04-26T21:59:26.874 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_send_time": 0.065807700, 2024-04-26T21:59:26.875 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_recv_time": 0.064840854, 2024-04-26T21:59:26.875 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_fast_dispatch_time": 0.009857636, 2024-04-26T21:59:26.875 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T21:59:26.875 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 463, 2024-04-26T21:59:26.875 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.105489062, 2024-04-26T21:59:26.875 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000227838 2024-04-26T21:59:26.875 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:26.875 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_handle_ack_lat": { 2024-04-26T21:59:26.875 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 484, 2024-04-26T21:59:26.875 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000223284, 2024-04-26T21:59:26.875 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000461 2024-04-26T21:59:26.875 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:26.875 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_encrypted_bytes": 354048, 2024-04-26T21:59:26.876 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_encrypted_bytes": 80624 2024-04-26T21:59:26.876 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:26.876 INFO:teuthology.orchestra.run.smithi076.stdout: "AsyncMessenger::Worker-2": { 2024-04-26T21:59:26.876 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_messages": 41, 2024-04-26T21:59:26.876 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_messages": 136, 2024-04-26T21:59:26.876 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_bytes": 17025, 2024-04-26T21:59:26.876 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_bytes": 268208, 2024-04-26T21:59:26.876 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_created_connections": 14, 2024-04-26T21:59:26.876 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_active_connections": 7, 2024-04-26T21:59:26.876 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_total_time": 0.034748861, 2024-04-26T21:59:26.876 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_send_time": 0.014522642, 2024-04-26T21:59:26.877 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_recv_time": 0.017764360, 2024-04-26T21:59:26.877 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_fast_dispatch_time": 0.000181894, 2024-04-26T21:59:26.877 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T21:59:26.877 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 130, 2024-04-26T21:59:26.877 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.068037625, 2024-04-26T21:59:26.877 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000523366 2024-04-26T21:59:26.877 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:26.877 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_handle_ack_lat": { 2024-04-26T21:59:26.877 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 50, 2024-04-26T21:59:26.877 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000606195, 2024-04-26T21:59:26.877 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000012123 2024-04-26T21:59:26.877 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:26.878 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_encrypted_bytes": 160, 2024-04-26T21:59:26.878 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_encrypted_bytes": 197328 2024-04-26T21:59:26.878 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:26.878 INFO:teuthology.orchestra.run.smithi076.stdout: "cct": { 2024-04-26T21:59:26.878 INFO:teuthology.orchestra.run.smithi076.stdout: "total_workers": 1, 2024-04-26T21:59:26.878 INFO:teuthology.orchestra.run.smithi076.stdout: "unhealthy_workers": 0 2024-04-26T21:59:26.878 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:26.878 INFO:teuthology.orchestra.run.smithi076.stdout: "finisher-MDSRank": { 2024-04-26T21:59:26.878 INFO:teuthology.orchestra.run.smithi076.stdout: "queue_len": 0, 2024-04-26T21:59:26.878 INFO:teuthology.orchestra.run.smithi076.stdout: "complete_latency": { 2024-04-26T21:59:26.878 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 52, 2024-04-26T21:59:26.879 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.001611521, 2024-04-26T21:59:26.879 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000030990 2024-04-26T21:59:26.879 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-26T21:59:26.879 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:26.879 INFO:teuthology.orchestra.run.smithi076.stdout: "finisher-PurgeQueue": { 2024-04-26T21:59:26.879 INFO:teuthology.orchestra.run.smithi076.stdout: "queue_len": 0, 2024-04-26T21:59:26.879 INFO:teuthology.orchestra.run.smithi076.stdout: "complete_latency": { 2024-04-26T21:59:26.879 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 2, 2024-04-26T21:59:26.879 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000008652, 2024-04-26T21:59:26.879 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000004326 2024-04-26T21:59:26.879 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-26T21:59:26.879 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:26.880 INFO:teuthology.orchestra.run.smithi076.stdout: "mds": { 2024-04-26T21:59:26.880 INFO:teuthology.orchestra.run.smithi076.stdout: "request": 4, 2024-04-26T21:59:26.880 INFO:teuthology.orchestra.run.smithi076.stdout: "reply": 2, 2024-04-26T21:59:26.880 INFO:teuthology.orchestra.run.smithi076.stdout: "reply_latency": { 2024-04-26T21:59:26.880 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 2, 2024-04-26T21:59:26.880 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.002943336, 2024-04-26T21:59:26.880 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.001471668 2024-04-26T21:59:26.880 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:26.880 INFO:teuthology.orchestra.run.smithi076.stdout: "slow_reply": 0, 2024-04-26T21:59:26.880 INFO:teuthology.orchestra.run.smithi076.stdout: "forward": 2, 2024-04-26T21:59:26.880 INFO:teuthology.orchestra.run.smithi076.stdout: "dir_fetch_complete": 0, 2024-04-26T21:59:26.881 INFO:teuthology.orchestra.run.smithi076.stdout: "dir_fetch_keys": 0, 2024-04-26T21:59:26.881 INFO:teuthology.orchestra.run.smithi076.stdout: "dir_commit": 11, 2024-04-26T21:59:26.881 INFO:teuthology.orchestra.run.smithi076.stdout: "dir_split": 0, 2024-04-26T21:59:26.881 INFO:teuthology.orchestra.run.smithi076.stdout: "dir_merge": 0, 2024-04-26T21:59:26.881 INFO:teuthology.orchestra.run.smithi076.stdout: "inodes": 14, 2024-04-26T21:59:26.881 INFO:teuthology.orchestra.run.smithi076.stdout: "inodes_top": 0, 2024-04-26T21:59:26.881 INFO:teuthology.orchestra.run.smithi076.stdout: "inodes_bottom": 14, 2024-04-26T21:59:26.881 INFO:teuthology.orchestra.run.smithi076.stdout: "inodes_pin_tail": 0, 2024-04-26T21:59:26.881 INFO:teuthology.orchestra.run.smithi076.stdout: "inodes_pinned": 14, 2024-04-26T21:59:26.881 INFO:teuthology.orchestra.run.smithi076.stdout: "inodes_expired": 0, 2024-04-26T21:59:26.881 INFO:teuthology.orchestra.run.smithi076.stdout: "inodes_with_caps": 1, 2024-04-26T21:59:26.881 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": 1, 2024-04-26T21:59:26.881 INFO:teuthology.orchestra.run.smithi076.stdout: "subtrees": 2, 2024-04-26T21:59:26.882 INFO:teuthology.orchestra.run.smithi076.stdout: "traverse": 6, 2024-04-26T21:59:26.882 INFO:teuthology.orchestra.run.smithi076.stdout: "traverse_hit": 3, 2024-04-26T21:59:26.882 INFO:teuthology.orchestra.run.smithi076.stdout: "traverse_forward": 0, 2024-04-26T21:59:26.882 INFO:teuthology.orchestra.run.smithi076.stdout: "traverse_discover": 1, 2024-04-26T21:59:26.882 INFO:teuthology.orchestra.run.smithi076.stdout: "traverse_dir_fetch": 0, 2024-04-26T21:59:26.882 INFO:teuthology.orchestra.run.smithi076.stdout: "traverse_remote_ino": 0, 2024-04-26T21:59:26.882 INFO:teuthology.orchestra.run.smithi076.stdout: "traverse_lock": 0, 2024-04-26T21:59:26.882 INFO:teuthology.orchestra.run.smithi076.stdout: "load_cent": 1000, 2024-04-26T21:59:26.882 INFO:teuthology.orchestra.run.smithi076.stdout: "q": 1, 2024-04-26T21:59:26.882 INFO:teuthology.orchestra.run.smithi076.stdout: "exported": 0, 2024-04-26T21:59:26.882 INFO:teuthology.orchestra.run.smithi076.stdout: "exported_inodes": 0, 2024-04-26T21:59:26.883 INFO:teuthology.orchestra.run.smithi076.stdout: "imported": 0, 2024-04-26T21:59:26.883 INFO:teuthology.orchestra.run.smithi076.stdout: "imported_inodes": 0, 2024-04-26T21:59:26.883 INFO:teuthology.orchestra.run.smithi076.stdout: "openino_dir_fetch": 0, 2024-04-26T21:59:26.883 INFO:teuthology.orchestra.run.smithi076.stdout: "openino_backtrace_fetch": 0, 2024-04-26T21:59:26.883 INFO:teuthology.orchestra.run.smithi076.stdout: "openino_peer_discover": 0, 2024-04-26T21:59:26.883 INFO:teuthology.orchestra.run.smithi076.stdout: "root_rfiles": 8, 2024-04-26T21:59:26.883 INFO:teuthology.orchestra.run.smithi076.stdout: "root_rbytes": 195290530, 2024-04-26T21:59:26.883 INFO:teuthology.orchestra.run.smithi076.stdout: "root_rsnaps": 0, 2024-04-26T21:59:26.883 INFO:teuthology.orchestra.run.smithi076.stdout: "scrub_backtrace_fetch": 0, 2024-04-26T21:59:26.883 INFO:teuthology.orchestra.run.smithi076.stdout: "scrub_set_tag": 0, 2024-04-26T21:59:26.883 INFO:teuthology.orchestra.run.smithi076.stdout: "scrub_backtrace_repaired": 0, 2024-04-26T21:59:26.884 INFO:teuthology.orchestra.run.smithi076.stdout: "scrub_inotable_repaired": 0, 2024-04-26T21:59:26.884 INFO:teuthology.orchestra.run.smithi076.stdout: "scrub_dir_inodes": 0, 2024-04-26T21:59:26.884 INFO:teuthology.orchestra.run.smithi076.stdout: "scrub_dir_base_inodes": 0, 2024-04-26T21:59:26.884 INFO:teuthology.orchestra.run.smithi076.stdout: "scrub_dirfrag_rstats": 0, 2024-04-26T21:59:26.884 INFO:teuthology.orchestra.run.smithi076.stdout: "scrub_file_inodes": 0, 2024-04-26T21:59:26.884 INFO:teuthology.orchestra.run.smithi076.stdout: "handle_inode_file_caps": 0, 2024-04-26T21:59:26.884 INFO:teuthology.orchestra.run.smithi076.stdout: "ceph_cap_op_revoke": 1, 2024-04-26T21:59:26.884 INFO:teuthology.orchestra.run.smithi076.stdout: "ceph_cap_op_grant": 0, 2024-04-26T21:59:26.884 INFO:teuthology.orchestra.run.smithi076.stdout: "ceph_cap_op_trunc": 0, 2024-04-26T21:59:26.884 INFO:teuthology.orchestra.run.smithi076.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-26T21:59:26.884 INFO:teuthology.orchestra.run.smithi076.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-26T21:59:26.885 INFO:teuthology.orchestra.run.smithi076.stdout: "handle_client_caps": 1, 2024-04-26T21:59:26.885 INFO:teuthology.orchestra.run.smithi076.stdout: "handle_client_caps_dirty": 0, 2024-04-26T21:59:26.885 INFO:teuthology.orchestra.run.smithi076.stdout: "handle_client_cap_release": 1, 2024-04-26T21:59:26.885 INFO:teuthology.orchestra.run.smithi076.stdout: "process_request_cap_release": 2 2024-04-26T21:59:26.885 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:26.885 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_cache": { 2024-04-26T21:59:26.885 INFO:teuthology.orchestra.run.smithi076.stdout: "dir_update": 0, 2024-04-26T21:59:26.885 INFO:teuthology.orchestra.run.smithi076.stdout: "dir_update_receipt": 0, 2024-04-26T21:59:26.885 INFO:teuthology.orchestra.run.smithi076.stdout: "dir_try_discover": 0, 2024-04-26T21:59:26.885 INFO:teuthology.orchestra.run.smithi076.stdout: "dir_send_discover": 2, 2024-04-26T21:59:26.885 INFO:teuthology.orchestra.run.smithi076.stdout: "dir_handle_discover": 0, 2024-04-26T21:59:26.885 INFO:teuthology.orchestra.run.smithi076.stdout: "num_strays": 0, 2024-04-26T21:59:26.886 INFO:teuthology.orchestra.run.smithi076.stdout: "num_strays_delayed": 0, 2024-04-26T21:59:26.886 INFO:teuthology.orchestra.run.smithi076.stdout: "num_strays_enqueuing": 0, 2024-04-26T21:59:26.886 INFO:teuthology.orchestra.run.smithi076.stdout: "strays_created": 0, 2024-04-26T21:59:26.886 INFO:teuthology.orchestra.run.smithi076.stdout: "strays_enqueued": 0, 2024-04-26T21:59:26.886 INFO:teuthology.orchestra.run.smithi076.stdout: "strays_reintegrated": 0, 2024-04-26T21:59:26.886 INFO:teuthology.orchestra.run.smithi076.stdout: "strays_migrated": 0, 2024-04-26T21:59:26.886 INFO:teuthology.orchestra.run.smithi076.stdout: "num_recovering_processing": 0, 2024-04-26T21:59:26.886 INFO:teuthology.orchestra.run.smithi076.stdout: "num_recovering_enqueued": 0, 2024-04-26T21:59:26.886 INFO:teuthology.orchestra.run.smithi076.stdout: "num_recovering_prioritized": 0, 2024-04-26T21:59:26.886 INFO:teuthology.orchestra.run.smithi076.stdout: "recovery_started": 0, 2024-04-26T21:59:26.887 INFO:teuthology.orchestra.run.smithi076.stdout: "recovery_completed": 0, 2024-04-26T21:59:26.887 INFO:teuthology.orchestra.run.smithi076.stdout: "ireq_quiesce_path": 0, 2024-04-26T21:59:26.887 INFO:teuthology.orchestra.run.smithi076.stdout: "ireq_quiesce_inode": 0, 2024-04-26T21:59:26.887 INFO:teuthology.orchestra.run.smithi076.stdout: "ireq_enqueue_scrub": 0, 2024-04-26T21:59:26.887 INFO:teuthology.orchestra.run.smithi076.stdout: "ireq_exportdir": 0, 2024-04-26T21:59:26.887 INFO:teuthology.orchestra.run.smithi076.stdout: "ireq_flush": 0, 2024-04-26T21:59:26.887 INFO:teuthology.orchestra.run.smithi076.stdout: "ireq_fragmentdir": 0, 2024-04-26T21:59:26.887 INFO:teuthology.orchestra.run.smithi076.stdout: "ireq_fragstats": 0, 2024-04-26T21:59:26.887 INFO:teuthology.orchestra.run.smithi076.stdout: "ireq_inodestats": 0 2024-04-26T21:59:26.887 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:26.887 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_log": { 2024-04-26T21:59:26.887 INFO:teuthology.orchestra.run.smithi076.stdout: "evlrg": 0, 2024-04-26T21:59:26.887 INFO:teuthology.orchestra.run.smithi076.stdout: "evadd": 4, 2024-04-26T21:59:26.888 INFO:teuthology.orchestra.run.smithi076.stdout: "evex": 0, 2024-04-26T21:59:26.888 INFO:teuthology.orchestra.run.smithi076.stdout: "evtrm": 0, 2024-04-26T21:59:26.888 INFO:teuthology.orchestra.run.smithi076.stdout: "ev": 4, 2024-04-26T21:59:26.888 INFO:teuthology.orchestra.run.smithi076.stdout: "evexg": 0, 2024-04-26T21:59:26.888 INFO:teuthology.orchestra.run.smithi076.stdout: "evexd": 0, 2024-04-26T21:59:26.888 INFO:teuthology.orchestra.run.smithi076.stdout: "segadd": 2, 2024-04-26T21:59:26.888 INFO:teuthology.orchestra.run.smithi076.stdout: "segex": 0, 2024-04-26T21:59:26.888 INFO:teuthology.orchestra.run.smithi076.stdout: "segtrm": 0, 2024-04-26T21:59:26.888 INFO:teuthology.orchestra.run.smithi076.stdout: "seg": 2, 2024-04-26T21:59:26.888 INFO:teuthology.orchestra.run.smithi076.stdout: "segmjr": 2, 2024-04-26T21:59:26.888 INFO:teuthology.orchestra.run.smithi076.stdout: "segexg": 0, 2024-04-26T21:59:26.888 INFO:teuthology.orchestra.run.smithi076.stdout: "segexd": 0, 2024-04-26T21:59:26.889 INFO:teuthology.orchestra.run.smithi076.stdout: "expos": 4194304, 2024-04-26T21:59:26.889 INFO:teuthology.orchestra.run.smithi076.stdout: "wrpos": 4195776, 2024-04-26T21:59:26.889 INFO:teuthology.orchestra.run.smithi076.stdout: "rdpos": 0, 2024-04-26T21:59:26.889 INFO:teuthology.orchestra.run.smithi076.stdout: "jlat": { 2024-04-26T21:59:26.889 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 3, 2024-04-26T21:59:26.889 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.013098151, 2024-04-26T21:59:26.889 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.004366050 2024-04-26T21:59:26.889 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:26.889 INFO:teuthology.orchestra.run.smithi076.stdout: "replayed": 0 2024-04-26T21:59:26.889 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:26.889 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_mem": { 2024-04-26T21:59:26.889 INFO:teuthology.orchestra.run.smithi076.stdout: "ino": 17, 2024-04-26T21:59:26.890 INFO:teuthology.orchestra.run.smithi076.stdout: "ino+": 17, 2024-04-26T21:59:26.890 INFO:teuthology.orchestra.run.smithi076.stdout: "ino-": 0, 2024-04-26T21:59:26.890 INFO:teuthology.orchestra.run.smithi076.stdout: "dir": 15, 2024-04-26T21:59:26.890 INFO:teuthology.orchestra.run.smithi076.stdout: "dir+": 16, 2024-04-26T21:59:26.890 INFO:teuthology.orchestra.run.smithi076.stdout: "dir-": 1, 2024-04-26T21:59:26.890 INFO:teuthology.orchestra.run.smithi076.stdout: "dn": 14, 2024-04-26T21:59:26.890 INFO:teuthology.orchestra.run.smithi076.stdout: "dn+": 14, 2024-04-26T21:59:26.890 INFO:teuthology.orchestra.run.smithi076.stdout: "dn-": 0, 2024-04-26T21:59:26.890 INFO:teuthology.orchestra.run.smithi076.stdout: "cap": 1, 2024-04-26T21:59:26.890 INFO:teuthology.orchestra.run.smithi076.stdout: "cap+": 2, 2024-04-26T21:59:26.890 INFO:teuthology.orchestra.run.smithi076.stdout: "cap-": 1, 2024-04-26T21:59:26.891 INFO:teuthology.orchestra.run.smithi076.stdout: "rss": 50432, 2024-04-26T21:59:26.891 INFO:teuthology.orchestra.run.smithi076.stdout: "heap": 223608 2024-04-26T21:59:26.891 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:26.891 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_server": { 2024-04-26T21:59:26.891 INFO:teuthology.orchestra.run.smithi076.stdout: "dispatch_client_request": 5, 2024-04-26T21:59:26.891 INFO:teuthology.orchestra.run.smithi076.stdout: "dispatch_server_request": 0, 2024-04-26T21:59:26.891 INFO:teuthology.orchestra.run.smithi076.stdout: "handle_client_request": 4, 2024-04-26T21:59:26.891 INFO:teuthology.orchestra.run.smithi076.stdout: "handle_client_session": 11, 2024-04-26T21:59:26.891 INFO:teuthology.orchestra.run.smithi076.stdout: "handle_peer_request": 0, 2024-04-26T21:59:26.891 INFO:teuthology.orchestra.run.smithi076.stdout: "req_create_latency": { 2024-04-26T21:59:26.891 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-04-26T21:59:26.891 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-04-26T21:59:26.891 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-04-26T21:59:26.892 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:26.892 INFO:teuthology.orchestra.run.smithi076.stdout: "req_getattr_latency": { 2024-04-26T21:59:26.892 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 2, 2024-04-26T21:59:26.892 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.002943336, 2024-04-26T21:59:26.892 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.001471668 2024-04-26T21:59:26.892 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:26.892 INFO:teuthology.orchestra.run.smithi076.stdout: "req_getfilelock_latency": { 2024-04-26T21:59:26.892 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-04-26T21:59:26.892 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-04-26T21:59:26.892 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-04-26T21:59:26.892 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:26.892 INFO:teuthology.orchestra.run.smithi076.stdout: "req_link_latency": { 2024-04-26T21:59:26.893 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-04-26T21:59:26.893 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-04-26T21:59:26.893 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-04-26T21:59:26.893 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:26.893 INFO:teuthology.orchestra.run.smithi076.stdout: "req_lookup_latency": { 2024-04-26T21:59:26.893 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-04-26T21:59:26.893 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-04-26T21:59:26.893 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-04-26T21:59:26.893 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:26.893 INFO:teuthology.orchestra.run.smithi076.stdout: "req_lookuphash_latency": { 2024-04-26T21:59:26.893 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-04-26T21:59:26.893 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-04-26T21:59:26.894 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-04-26T21:59:26.894 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:26.894 INFO:teuthology.orchestra.run.smithi076.stdout: "req_lookupino_latency": { 2024-04-26T21:59:26.894 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-04-26T21:59:26.894 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-04-26T21:59:26.894 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-04-26T21:59:26.894 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:26.894 INFO:teuthology.orchestra.run.smithi076.stdout: "req_lookupname_latency": { 2024-04-26T21:59:26.894 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-04-26T21:59:26.894 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-04-26T21:59:26.894 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-04-26T21:59:26.894 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:26.895 INFO:teuthology.orchestra.run.smithi076.stdout: "req_lookupparent_latency": { 2024-04-26T21:59:26.895 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-04-26T21:59:26.895 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-04-26T21:59:26.895 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-04-26T21:59:26.895 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:26.895 INFO:teuthology.orchestra.run.smithi076.stdout: "req_lookupsnap_latency": { 2024-04-26T21:59:26.895 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-04-26T21:59:26.895 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-04-26T21:59:26.895 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-04-26T21:59:26.895 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:26.895 INFO:teuthology.orchestra.run.smithi076.stdout: "req_lssnap_latency": { 2024-04-26T21:59:26.895 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-04-26T21:59:26.896 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-04-26T21:59:26.896 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-04-26T21:59:26.896 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:26.896 INFO:teuthology.orchestra.run.smithi076.stdout: "req_mkdir_latency": { 2024-04-26T21:59:26.896 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-04-26T21:59:26.897 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-04-26T21:59:26.897 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-04-26T21:59:26.897 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:26.897 INFO:teuthology.orchestra.run.smithi076.stdout: "req_mknod_latency": { 2024-04-26T21:59:26.897 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-04-26T21:59:26.897 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-04-26T21:59:26.897 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-04-26T21:59:26.897 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:26.898 INFO:teuthology.orchestra.run.smithi076.stdout: "req_mksnap_latency": { 2024-04-26T21:59:26.898 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-04-26T21:59:26.898 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-04-26T21:59:26.898 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-04-26T21:59:26.898 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:26.898 INFO:teuthology.orchestra.run.smithi076.stdout: "req_open_latency": { 2024-04-26T21:59:26.898 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-04-26T21:59:26.898 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-04-26T21:59:26.898 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-04-26T21:59:26.898 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:26.898 INFO:teuthology.orchestra.run.smithi076.stdout: "req_readdir_latency": { 2024-04-26T21:59:26.898 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-04-26T21:59:26.899 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-04-26T21:59:26.899 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-04-26T21:59:26.899 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:26.899 INFO:teuthology.orchestra.run.smithi076.stdout: "req_rename_latency": { 2024-04-26T21:59:26.899 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-04-26T21:59:26.899 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-04-26T21:59:26.899 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-04-26T21:59:26.899 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:26.899 INFO:teuthology.orchestra.run.smithi076.stdout: "req_renamesnap_latency": { 2024-04-26T21:59:26.899 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-04-26T21:59:26.899 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-04-26T21:59:26.900 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-04-26T21:59:26.900 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:26.900 INFO:teuthology.orchestra.run.smithi076.stdout: "req_snapdiff_latency": { 2024-04-26T21:59:26.900 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-04-26T21:59:26.900 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-04-26T21:59:26.900 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-04-26T21:59:26.900 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:26.900 INFO:teuthology.orchestra.run.smithi076.stdout: "req_rmdir_latency": { 2024-04-26T21:59:26.900 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-04-26T21:59:26.900 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-04-26T21:59:26.900 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-04-26T21:59:26.900 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:26.901 INFO:teuthology.orchestra.run.smithi076.stdout: "req_rmsnap_latency": { 2024-04-26T21:59:26.901 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-04-26T21:59:26.901 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-04-26T21:59:26.901 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-04-26T21:59:26.901 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:26.901 INFO:teuthology.orchestra.run.smithi076.stdout: "req_rmxattr_latency": { 2024-04-26T21:59:26.901 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-04-26T21:59:26.901 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-04-26T21:59:26.901 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-04-26T21:59:26.901 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:26.901 INFO:teuthology.orchestra.run.smithi076.stdout: "req_setattr_latency": { 2024-04-26T21:59:26.901 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-04-26T21:59:26.902 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-04-26T21:59:26.902 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-04-26T21:59:26.902 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:26.902 INFO:teuthology.orchestra.run.smithi076.stdout: "req_setdirlayout_latency": { 2024-04-26T21:59:26.902 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-04-26T21:59:26.902 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-04-26T21:59:26.902 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-04-26T21:59:26.902 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:26.902 INFO:teuthology.orchestra.run.smithi076.stdout: "req_setfilelock_latency": { 2024-04-26T21:59:26.902 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-04-26T21:59:26.902 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-04-26T21:59:26.902 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-04-26T21:59:26.903 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:26.903 INFO:teuthology.orchestra.run.smithi076.stdout: "req_setlayout_latency": { 2024-04-26T21:59:26.903 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-04-26T21:59:26.903 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-04-26T21:59:26.903 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-04-26T21:59:26.903 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:26.903 INFO:teuthology.orchestra.run.smithi076.stdout: "req_setxattr_latency": { 2024-04-26T21:59:26.903 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-04-26T21:59:26.903 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-04-26T21:59:26.903 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-04-26T21:59:26.903 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:26.903 INFO:teuthology.orchestra.run.smithi076.stdout: "req_symlink_latency": { 2024-04-26T21:59:26.903 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-04-26T21:59:26.904 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-04-26T21:59:26.904 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-04-26T21:59:26.904 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:26.904 INFO:teuthology.orchestra.run.smithi076.stdout: "req_unlink_latency": { 2024-04-26T21:59:26.904 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-04-26T21:59:26.904 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-04-26T21:59:26.904 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-04-26T21:59:26.904 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:26.904 INFO:teuthology.orchestra.run.smithi076.stdout: "cap_revoke_eviction": 0, 2024-04-26T21:59:26.904 INFO:teuthology.orchestra.run.smithi076.stdout: "cap_acquisition_throttle": 0, 2024-04-26T21:59:26.904 INFO:teuthology.orchestra.run.smithi076.stdout: "req_getvxattr_latency": { 2024-04-26T21:59:26.904 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-04-26T21:59:26.905 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-04-26T21:59:26.905 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-04-26T21:59:26.905 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-26T21:59:26.905 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:26.905 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_sessions": { 2024-04-26T21:59:26.905 INFO:teuthology.orchestra.run.smithi076.stdout: "session_count": 2, 2024-04-26T21:59:26.905 INFO:teuthology.orchestra.run.smithi076.stdout: "session_add": 2, 2024-04-26T21:59:26.905 INFO:teuthology.orchestra.run.smithi076.stdout: "session_remove": 0, 2024-04-26T21:59:26.905 INFO:teuthology.orchestra.run.smithi076.stdout: "sessions_open": 2, 2024-04-26T21:59:26.905 INFO:teuthology.orchestra.run.smithi076.stdout: "sessions_stale": 0, 2024-04-26T21:59:26.905 INFO:teuthology.orchestra.run.smithi076.stdout: "total_load": 1, 2024-04-26T21:59:26.906 INFO:teuthology.orchestra.run.smithi076.stdout: "average_load": 0, 2024-04-26T21:59:26.906 INFO:teuthology.orchestra.run.smithi076.stdout: "avg_session_uptime": 315, 2024-04-26T21:59:26.906 INFO:teuthology.orchestra.run.smithi076.stdout: "mdthresh_evicted": 0 2024-04-26T21:59:26.906 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:26.906 INFO:teuthology.orchestra.run.smithi076.stdout: "mempool": { 2024-04-26T21:59:26.906 INFO:teuthology.orchestra.run.smithi076.stdout: "bloom_filter_bytes": 0, 2024-04-26T21:59:26.906 INFO:teuthology.orchestra.run.smithi076.stdout: "bloom_filter_items": 0, 2024-04-26T21:59:26.906 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_alloc_bytes": 0, 2024-04-26T21:59:26.906 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_alloc_items": 0, 2024-04-26T21:59:26.906 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_data_bytes": 0, 2024-04-26T21:59:26.906 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_data_items": 0, 2024-04-26T21:59:26.906 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-26T21:59:26.907 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_onode_items": 0, 2024-04-26T21:59:26.907 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-26T21:59:26.907 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_meta_items": 0, 2024-04-26T21:59:26.907 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_other_bytes": 0, 2024-04-26T21:59:26.907 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_other_items": 0, 2024-04-26T21:59:26.907 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-26T21:59:26.907 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_buffer_items": 0, 2024-04-26T21:59:26.907 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_extent_bytes": 0, 2024-04-26T21:59:26.907 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_extent_items": 0, 2024-04-26T21:59:26.907 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_blob_bytes": 0, 2024-04-26T21:59:26.907 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_blob_items": 0, 2024-04-26T21:59:26.907 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-26T21:59:26.908 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_shared_blob_items": 0, 2024-04-26T21:59:26.908 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-26T21:59:26.908 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_inline_bl_items": 0, 2024-04-26T21:59:26.908 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_fsck_bytes": 0, 2024-04-26T21:59:26.908 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_fsck_items": 0, 2024-04-26T21:59:26.908 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_txc_bytes": 0, 2024-04-26T21:59:26.908 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_txc_items": 0, 2024-04-26T21:59:26.908 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-26T21:59:26.908 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_writing_deferred_items": 0, 2024-04-26T21:59:26.908 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_writing_bytes": 0, 2024-04-26T21:59:26.908 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_writing_items": 0, 2024-04-26T21:59:26.909 INFO:teuthology.orchestra.run.smithi076.stdout: "bluefs_bytes": 0, 2024-04-26T21:59:26.909 INFO:teuthology.orchestra.run.smithi076.stdout: "bluefs_items": 0, 2024-04-26T21:59:26.909 INFO:teuthology.orchestra.run.smithi076.stdout: "bluefs_file_reader_bytes": 0, 2024-04-26T21:59:26.909 INFO:teuthology.orchestra.run.smithi076.stdout: "bluefs_file_reader_items": 0, 2024-04-26T21:59:26.909 INFO:teuthology.orchestra.run.smithi076.stdout: "bluefs_file_writer_bytes": 0, 2024-04-26T21:59:26.909 INFO:teuthology.orchestra.run.smithi076.stdout: "bluefs_file_writer_items": 0, 2024-04-26T21:59:26.909 INFO:teuthology.orchestra.run.smithi076.stdout: "buffer_anon_bytes": 247967, 2024-04-26T21:59:26.909 INFO:teuthology.orchestra.run.smithi076.stdout: "buffer_anon_items": 88, 2024-04-26T21:59:26.909 INFO:teuthology.orchestra.run.smithi076.stdout: "buffer_meta_bytes": 0, 2024-04-26T21:59:26.909 INFO:teuthology.orchestra.run.smithi076.stdout: "buffer_meta_items": 0, 2024-04-26T21:59:26.909 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_bytes": 0, 2024-04-26T21:59:26.909 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_items": 0, 2024-04-26T21:59:26.910 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_mapbl_bytes": 0, 2024-04-26T21:59:26.910 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_mapbl_items": 0, 2024-04-26T21:59:26.910 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_pglog_bytes": 0, 2024-04-26T21:59:26.910 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_pglog_items": 0, 2024-04-26T21:59:26.910 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_bytes": 8008, 2024-04-26T21:59:26.910 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_items": 168, 2024-04-26T21:59:26.910 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_mapping_bytes": 0, 2024-04-26T21:59:26.910 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_mapping_items": 0, 2024-04-26T21:59:26.910 INFO:teuthology.orchestra.run.smithi076.stdout: "pgmap_bytes": 0, 2024-04-26T21:59:26.910 INFO:teuthology.orchestra.run.smithi076.stdout: "pgmap_items": 0, 2024-04-26T21:59:26.910 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_co_bytes": 68053, 2024-04-26T21:59:26.910 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_co_items": 290, 2024-04-26T21:59:26.910 INFO:teuthology.orchestra.run.smithi076.stdout: "unittest_1_bytes": 0, 2024-04-26T21:59:26.911 INFO:teuthology.orchestra.run.smithi076.stdout: "unittest_1_items": 0, 2024-04-26T21:59:26.911 INFO:teuthology.orchestra.run.smithi076.stdout: "unittest_2_bytes": 0, 2024-04-26T21:59:26.911 INFO:teuthology.orchestra.run.smithi076.stdout: "unittest_2_items": 0 2024-04-26T21:59:26.911 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:26.911 INFO:teuthology.orchestra.run.smithi076.stdout: "objecter": { 2024-04-26T21:59:26.911 INFO:teuthology.orchestra.run.smithi076.stdout: "op_active": 0, 2024-04-26T21:59:26.911 INFO:teuthology.orchestra.run.smithi076.stdout: "op_laggy": 0, 2024-04-26T21:59:26.911 INFO:teuthology.orchestra.run.smithi076.stdout: "op_send": 39, 2024-04-26T21:59:26.911 INFO:teuthology.orchestra.run.smithi076.stdout: "op_send_bytes": 11953, 2024-04-26T21:59:26.911 INFO:teuthology.orchestra.run.smithi076.stdout: "op_resend": 33, 2024-04-26T21:59:26.911 INFO:teuthology.orchestra.run.smithi076.stdout: "op_reply": 39, 2024-04-26T21:59:26.911 INFO:teuthology.orchestra.run.smithi076.stdout: "op_latency": { 2024-04-26T21:59:26.911 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 39, 2024-04-26T21:59:26.912 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.437987067, 2024-04-26T21:59:26.912 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.011230437 2024-04-26T21:59:26.912 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:26.912 INFO:teuthology.orchestra.run.smithi076.stdout: "op_inflight": 0, 2024-04-26T21:59:26.912 INFO:teuthology.orchestra.run.smithi076.stdout: "oplen_avg": { 2024-04-26T21:59:26.912 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 39, 2024-04-26T21:59:26.912 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 62 2024-04-26T21:59:26.912 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:26.912 INFO:teuthology.orchestra.run.smithi076.stdout: "op": 39, 2024-04-26T21:59:26.912 INFO:teuthology.orchestra.run.smithi076.stdout: "op_r": 0, 2024-04-26T21:59:26.912 INFO:teuthology.orchestra.run.smithi076.stdout: "op_w": 39, 2024-04-26T21:59:26.913 INFO:teuthology.orchestra.run.smithi076.stdout: "op_rmw": 0, 2024-04-26T21:59:26.913 INFO:teuthology.orchestra.run.smithi076.stdout: "op_pg": 0, 2024-04-26T21:59:26.913 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_stat": 0, 2024-04-26T21:59:26.913 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_create": 10, 2024-04-26T21:59:26.913 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_read": 0, 2024-04-26T21:59:26.913 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_write": 3, 2024-04-26T21:59:26.913 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_writefull": 6, 2024-04-26T21:59:26.913 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_writesame": 0, 2024-04-26T21:59:26.913 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_append": 0, 2024-04-26T21:59:26.913 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_zero": 0, 2024-04-26T21:59:26.913 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_truncate": 0, 2024-04-26T21:59:26.913 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_delete": 6, 2024-04-26T21:59:26.913 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_mapext": 0, 2024-04-26T21:59:26.914 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_sparse_read": 0, 2024-04-26T21:59:26.914 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_clonerange": 0, 2024-04-26T21:59:26.914 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_getxattr": 0, 2024-04-26T21:59:26.914 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_setxattr": 20, 2024-04-26T21:59:26.914 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_cmpxattr": 0, 2024-04-26T21:59:26.914 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_rmxattr": 0, 2024-04-26T21:59:26.914 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_resetxattrs": 0, 2024-04-26T21:59:26.914 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_call": 0, 2024-04-26T21:59:26.914 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_watch": 0, 2024-04-26T21:59:26.914 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_notify": 0, 2024-04-26T21:59:26.914 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_src_cmpxattr": 0, 2024-04-26T21:59:26.914 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_pgls": 0, 2024-04-26T21:59:26.915 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_pgls_filter": 0, 2024-04-26T21:59:26.915 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_other": 0, 2024-04-26T21:59:26.915 INFO:teuthology.orchestra.run.smithi076.stdout: "linger_active": 0, 2024-04-26T21:59:26.915 INFO:teuthology.orchestra.run.smithi076.stdout: "linger_send": 0, 2024-04-26T21:59:26.915 INFO:teuthology.orchestra.run.smithi076.stdout: "linger_resend": 0, 2024-04-26T21:59:26.915 INFO:teuthology.orchestra.run.smithi076.stdout: "linger_ping": 0, 2024-04-26T21:59:26.915 INFO:teuthology.orchestra.run.smithi076.stdout: "poolop_active": 0, 2024-04-26T21:59:26.915 INFO:teuthology.orchestra.run.smithi076.stdout: "poolop_send": 0, 2024-04-26T21:59:26.915 INFO:teuthology.orchestra.run.smithi076.stdout: "poolop_resend": 0, 2024-04-26T21:59:26.915 INFO:teuthology.orchestra.run.smithi076.stdout: "poolstat_active": 0, 2024-04-26T21:59:26.915 INFO:teuthology.orchestra.run.smithi076.stdout: "poolstat_send": 0, 2024-04-26T21:59:26.916 INFO:teuthology.orchestra.run.smithi076.stdout: "poolstat_resend": 0, 2024-04-26T21:59:26.916 INFO:teuthology.orchestra.run.smithi076.stdout: "statfs_active": 0, 2024-04-26T21:59:26.916 INFO:teuthology.orchestra.run.smithi076.stdout: "statfs_send": 0, 2024-04-26T21:59:26.916 INFO:teuthology.orchestra.run.smithi076.stdout: "statfs_resend": 0, 2024-04-26T21:59:26.916 INFO:teuthology.orchestra.run.smithi076.stdout: "command_active": 0, 2024-04-26T21:59:26.916 INFO:teuthology.orchestra.run.smithi076.stdout: "command_send": 0, 2024-04-26T21:59:26.916 INFO:teuthology.orchestra.run.smithi076.stdout: "command_resend": 0, 2024-04-26T21:59:26.916 INFO:teuthology.orchestra.run.smithi076.stdout: "map_epoch": 91, 2024-04-26T21:59:26.916 INFO:teuthology.orchestra.run.smithi076.stdout: "map_full": 0, 2024-04-26T21:59:26.916 INFO:teuthology.orchestra.run.smithi076.stdout: "map_inc": 4, 2024-04-26T21:59:26.916 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_sessions": 9, 2024-04-26T21:59:26.916 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_session_open": 9, 2024-04-26T21:59:26.916 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_session_close": 0, 2024-04-26T21:59:26.916 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_laggy": 0, 2024-04-26T21:59:26.917 INFO:teuthology.orchestra.run.smithi076.stdout: "omap_wr": 16, 2024-04-26T21:59:26.917 INFO:teuthology.orchestra.run.smithi076.stdout: "omap_rd": 0, 2024-04-26T21:59:26.917 INFO:teuthology.orchestra.run.smithi076.stdout: "omap_del": 1 2024-04-26T21:59:26.917 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:26.917 INFO:teuthology.orchestra.run.smithi076.stdout: "oft": { 2024-04-26T21:59:26.917 INFO:teuthology.orchestra.run.smithi076.stdout: "omap_total_objs": 1, 2024-04-26T21:59:26.917 INFO:teuthology.orchestra.run.smithi076.stdout: "omap_total_kv_pairs": 0, 2024-04-26T21:59:26.917 INFO:teuthology.orchestra.run.smithi076.stdout: "omap_total_updates": 0, 2024-04-26T21:59:26.917 INFO:teuthology.orchestra.run.smithi076.stdout: "omap_total_removes": 0 2024-04-26T21:59:26.917 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:26.917 INFO:teuthology.orchestra.run.smithi076.stdout: "osd-slow-ops": { 2024-04-26T21:59:26.917 INFO:teuthology.orchestra.run.smithi076.stdout: "slow_ops_count": 0 2024-04-26T21:59:26.918 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:26.918 INFO:teuthology.orchestra.run.smithi076.stdout: "purge_queue": { 2024-04-26T21:59:26.918 INFO:teuthology.orchestra.run.smithi076.stdout: "pq_executing_ops": 0, 2024-04-26T21:59:26.918 INFO:teuthology.orchestra.run.smithi076.stdout: "pq_executing_ops_high_water": 0, 2024-04-26T21:59:26.918 INFO:teuthology.orchestra.run.smithi076.stdout: "pq_executing": 0, 2024-04-26T21:59:26.918 INFO:teuthology.orchestra.run.smithi076.stdout: "pq_executing_high_water": 0, 2024-04-26T21:59:26.918 INFO:teuthology.orchestra.run.smithi076.stdout: "pq_executed_ops": 0, 2024-04-26T21:59:26.918 INFO:teuthology.orchestra.run.smithi076.stdout: "pq_executed": 0, 2024-04-26T21:59:26.918 INFO:teuthology.orchestra.run.smithi076.stdout: "pq_item_in_journal": 0 2024-04-26T21:59:26.918 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:26.918 INFO:teuthology.orchestra.run.smithi076.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-26T21:59:26.918 INFO:teuthology.orchestra.run.smithi076.stdout: "val": 0, 2024-04-26T21:59:26.919 INFO:teuthology.orchestra.run.smithi076.stdout: "max": 104857600, 2024-04-26T21:59:26.919 INFO:teuthology.orchestra.run.smithi076.stdout: "get_started": 0, 2024-04-26T21:59:26.919 INFO:teuthology.orchestra.run.smithi076.stdout: "get": 492, 2024-04-26T21:59:26.919 INFO:teuthology.orchestra.run.smithi076.stdout: "get_sum": 411048, 2024-04-26T21:59:26.919 INFO:teuthology.orchestra.run.smithi076.stdout: "get_or_fail_fail": 0, 2024-04-26T21:59:26.919 INFO:teuthology.orchestra.run.smithi076.stdout: "get_or_fail_success": 492, 2024-04-26T21:59:26.919 INFO:teuthology.orchestra.run.smithi076.stdout: "take": 0, 2024-04-26T21:59:26.919 INFO:teuthology.orchestra.run.smithi076.stdout: "take_sum": 0, 2024-04-26T21:59:26.919 INFO:teuthology.orchestra.run.smithi076.stdout: "put": 492, 2024-04-26T21:59:26.919 INFO:teuthology.orchestra.run.smithi076.stdout: "put_sum": 411048, 2024-04-26T21:59:26.919 INFO:teuthology.orchestra.run.smithi076.stdout: "wait": { 2024-04-26T21:59:26.919 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-04-26T21:59:26.920 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-04-26T21:59:26.920 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-04-26T21:59:26.920 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-26T21:59:26.920 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:26.920 INFO:teuthology.orchestra.run.smithi076.stdout: "throttle-objecter_bytes": { 2024-04-26T21:59:26.920 INFO:teuthology.orchestra.run.smithi076.stdout: "val": 0, 2024-04-26T21:59:26.920 INFO:teuthology.orchestra.run.smithi076.stdout: "max": 104857600, 2024-04-26T21:59:26.920 INFO:teuthology.orchestra.run.smithi076.stdout: "get_started": 0, 2024-04-26T21:59:26.920 INFO:teuthology.orchestra.run.smithi076.stdout: "get": 0, 2024-04-26T21:59:26.920 INFO:teuthology.orchestra.run.smithi076.stdout: "get_sum": 0, 2024-04-26T21:59:26.920 INFO:teuthology.orchestra.run.smithi076.stdout: "get_or_fail_fail": 0, 2024-04-26T21:59:26.920 INFO:teuthology.orchestra.run.smithi076.stdout: "get_or_fail_success": 0, 2024-04-26T21:59:26.920 INFO:teuthology.orchestra.run.smithi076.stdout: "take": 39, 2024-04-26T21:59:26.921 INFO:teuthology.orchestra.run.smithi076.stdout: "take_sum": 11953, 2024-04-26T21:59:26.921 INFO:teuthology.orchestra.run.smithi076.stdout: "put": 39, 2024-04-26T21:59:26.921 INFO:teuthology.orchestra.run.smithi076.stdout: "put_sum": 11953, 2024-04-26T21:59:26.921 INFO:teuthology.orchestra.run.smithi076.stdout: "wait": { 2024-04-26T21:59:26.921 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-04-26T21:59:26.921 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-04-26T21:59:26.921 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-04-26T21:59:26.921 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-26T21:59:26.921 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:26.921 INFO:teuthology.orchestra.run.smithi076.stdout: "throttle-objecter_ops": { 2024-04-26T21:59:26.921 INFO:teuthology.orchestra.run.smithi076.stdout: "val": 0, 2024-04-26T21:59:26.921 INFO:teuthology.orchestra.run.smithi076.stdout: "max": 1024, 2024-04-26T21:59:26.921 INFO:teuthology.orchestra.run.smithi076.stdout: "get_started": 0, 2024-04-26T21:59:26.922 INFO:teuthology.orchestra.run.smithi076.stdout: "get": 0, 2024-04-26T21:59:26.922 INFO:teuthology.orchestra.run.smithi076.stdout: "get_sum": 0, 2024-04-26T21:59:26.922 INFO:teuthology.orchestra.run.smithi076.stdout: "get_or_fail_fail": 0, 2024-04-26T21:59:26.922 INFO:teuthology.orchestra.run.smithi076.stdout: "get_or_fail_success": 0, 2024-04-26T21:59:26.922 INFO:teuthology.orchestra.run.smithi076.stdout: "take": 39, 2024-04-26T21:59:26.922 INFO:teuthology.orchestra.run.smithi076.stdout: "take_sum": 39, 2024-04-26T21:59:26.922 INFO:teuthology.orchestra.run.smithi076.stdout: "put": 39, 2024-04-26T21:59:26.922 INFO:teuthology.orchestra.run.smithi076.stdout: "put_sum": 39, 2024-04-26T21:59:26.922 INFO:teuthology.orchestra.run.smithi076.stdout: "wait": { 2024-04-26T21:59:26.922 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-04-26T21:59:26.922 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-04-26T21:59:26.922 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-04-26T21:59:26.923 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-26T21:59:26.923 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:26.923 INFO:teuthology.orchestra.run.smithi076.stdout: "throttle-write_buf_throttle": { 2024-04-26T21:59:26.923 INFO:teuthology.orchestra.run.smithi076.stdout: "val": 0, 2024-04-26T21:59:26.923 INFO:teuthology.orchestra.run.smithi076.stdout: "max": 3758096384, 2024-04-26T21:59:26.923 INFO:teuthology.orchestra.run.smithi076.stdout: "get_started": 0, 2024-04-26T21:59:26.923 INFO:teuthology.orchestra.run.smithi076.stdout: "get": 0, 2024-04-26T21:59:26.923 INFO:teuthology.orchestra.run.smithi076.stdout: "get_sum": 0, 2024-04-26T21:59:26.923 INFO:teuthology.orchestra.run.smithi076.stdout: "get_or_fail_fail": 0, 2024-04-26T21:59:26.923 INFO:teuthology.orchestra.run.smithi076.stdout: "get_or_fail_success": 0, 2024-04-26T21:59:26.923 INFO:teuthology.orchestra.run.smithi076.stdout: "take": 0, 2024-04-26T21:59:26.923 INFO:teuthology.orchestra.run.smithi076.stdout: "take_sum": 0, 2024-04-26T21:59:26.923 INFO:teuthology.orchestra.run.smithi076.stdout: "put": 0, 2024-04-26T21:59:26.924 INFO:teuthology.orchestra.run.smithi076.stdout: "put_sum": 0, 2024-04-26T21:59:26.925 INFO:teuthology.orchestra.run.smithi076.stdout: "wait": { 2024-04-26T21:59:26.925 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-04-26T21:59:26.925 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-04-26T21:59:26.925 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-04-26T21:59:26.925 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-26T21:59:26.925 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-26T21:59:26.925 INFO:teuthology.orchestra.run.smithi076.stdout: "throttle-write_buf_throttle-0x55dda3eca500": { 2024-04-26T21:59:26.925 INFO:teuthology.orchestra.run.smithi076.stdout: "val": 0, 2024-04-26T21:59:26.926 INFO:teuthology.orchestra.run.smithi076.stdout: "max": 3758096384, 2024-04-26T21:59:26.926 INFO:teuthology.orchestra.run.smithi076.stdout: "get_started": 0, 2024-04-26T21:59:26.926 INFO:teuthology.orchestra.run.smithi076.stdout: "get": 4, 2024-04-26T21:59:26.926 INFO:teuthology.orchestra.run.smithi076.stdout: "get_sum": 1472, 2024-04-26T21:59:26.926 INFO:teuthology.orchestra.run.smithi076.stdout: "get_or_fail_fail": 0, 2024-04-26T21:59:26.926 INFO:teuthology.orchestra.run.smithi076.stdout: "get_or_fail_success": 4, 2024-04-26T21:59:26.926 INFO:teuthology.orchestra.run.smithi076.stdout: "take": 0, 2024-04-26T21:59:26.926 INFO:teuthology.orchestra.run.smithi076.stdout: "take_sum": 0, 2024-04-26T21:59:26.926 INFO:teuthology.orchestra.run.smithi076.stdout: "put": 3, 2024-04-26T21:59:26.926 INFO:teuthology.orchestra.run.smithi076.stdout: "put_sum": 1472, 2024-04-26T21:59:26.926 INFO:teuthology.orchestra.run.smithi076.stdout: "wait": { 2024-04-26T21:59:26.926 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-04-26T21:59:26.926 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-04-26T21:59:26.927 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-04-26T21:59:26.927 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-26T21:59:26.927 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-26T21:59:26.927 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-04-26T21:59:27.808 INFO:tasks.check_counter:Daemon mds.k mds.exported=0 2024-04-26T21:59:27.808 INFO:tasks.check_counter:Daemon mds.k mds.imported=0 2024-04-26T21:59:27.808 DEBUG:tasks.check_counter:Getting stats from c 2024-04-26T21:59:27.809 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid a1780950-0415-11ef-bc93-c7b262605968 -- ceph daemon mds.c perf dump 2024-04-26T21:59:28.263 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/a1780950-0415-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T21:59:29.187 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:59:28 smithi164 ceph-mon[31916]: pgmap v570: 97 pgs: 97 active+clean; 1.1 MiB data, 656 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:59:29.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:59:28 smithi076 ceph-mon[31823]: pgmap v570: 97 pgs: 97 active+clean; 1.1 MiB data, 656 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:59:29.321 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:59:28 smithi072 ceph-mon[28138]: pgmap v570: 97 pgs: 97 active+clean; 1.1 MiB data, 656 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:59:30.625 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-04-26T21:59:30.625 INFO:teuthology.orchestra.run.smithi164.stdout: "AsyncMessenger::Worker-0": { 2024-04-26T21:59:30.625 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_messages": 864, 2024-04-26T21:59:30.625 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_messages": 866, 2024-04-26T21:59:30.625 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_bytes": 859670, 2024-04-26T21:59:30.625 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_bytes": 274762, 2024-04-26T21:59:30.625 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_created_connections": 6, 2024-04-26T21:59:30.625 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_active_connections": 2, 2024-04-26T21:59:30.625 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_total_time": 0.181239197, 2024-04-26T21:59:30.625 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_send_time": 0.058293153, 2024-04-26T21:59:30.625 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_recv_time": 0.051551271, 2024-04-26T21:59:30.625 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_fast_dispatch_time": 0.050042223, 2024-04-26T21:59:30.625 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T21:59:30.626 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 864, 2024-04-26T21:59:30.626 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.079255326, 2024-04-26T21:59:30.626 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000091730 2024-04-26T21:59:30.626 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:30.626 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_handle_ack_lat": { 2024-04-26T21:59:30.626 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-26T21:59:30.626 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-26T21:59:30.626 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-26T21:59:30.626 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:30.626 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-26T21:59:30.626 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_encrypted_bytes": 0 2024-04-26T21:59:30.626 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:30.626 INFO:teuthology.orchestra.run.smithi164.stdout: "AsyncMessenger::Worker-1": { 2024-04-26T21:59:30.626 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_messages": 5, 2024-04-26T21:59:30.627 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_messages": 93, 2024-04-26T21:59:30.627 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_bytes": 1816, 2024-04-26T21:59:30.627 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_bytes": 199754, 2024-04-26T21:59:30.627 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_created_connections": 5, 2024-04-26T21:59:30.627 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_active_connections": 4, 2024-04-26T21:59:30.627 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_total_time": 0.015252618, 2024-04-26T21:59:30.627 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_send_time": 0.008409005, 2024-04-26T21:59:30.627 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_recv_time": 0.004515329, 2024-04-26T21:59:30.627 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_fast_dispatch_time": 0.000137625, 2024-04-26T21:59:30.627 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T21:59:30.627 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 91, 2024-04-26T21:59:30.627 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.013731993, 2024-04-26T21:59:30.627 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000150901 2024-04-26T21:59:30.627 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:30.628 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_handle_ack_lat": { 2024-04-26T21:59:30.628 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-26T21:59:30.628 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-26T21:59:30.628 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-26T21:59:30.628 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:30.628 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_encrypted_bytes": 160, 2024-04-26T21:59:30.628 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_encrypted_bytes": 198416 2024-04-26T21:59:30.628 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:30.628 INFO:teuthology.orchestra.run.smithi164.stdout: "AsyncMessenger::Worker-2": { 2024-04-26T21:59:30.628 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_messages": 508, 2024-04-26T21:59:30.628 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_messages": 502, 2024-04-26T21:59:30.628 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_bytes": 432951, 2024-04-26T21:59:30.628 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_bytes": 191847, 2024-04-26T21:59:30.628 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_created_connections": 3, 2024-04-26T21:59:30.629 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_active_connections": 2, 2024-04-26T21:59:30.629 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_total_time": 0.140498475, 2024-04-26T21:59:30.629 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_send_time": 0.055983124, 2024-04-26T21:59:30.629 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_recv_time": 0.050628817, 2024-04-26T21:59:30.629 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_fast_dispatch_time": 0.031403602, 2024-04-26T21:59:30.629 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T21:59:30.629 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 502, 2024-04-26T21:59:30.629 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.050238419, 2024-04-26T21:59:30.629 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000100076 2024-04-26T21:59:30.629 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:30.629 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_handle_ack_lat": { 2024-04-26T21:59:30.629 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-26T21:59:30.629 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-26T21:59:30.629 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-26T21:59:30.629 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:30.630 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_encrypted_bytes": 340720, 2024-04-26T21:59:30.630 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_encrypted_bytes": 73232 2024-04-26T21:59:30.630 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:30.630 INFO:teuthology.orchestra.run.smithi164.stdout: "cct": { 2024-04-26T21:59:30.630 INFO:teuthology.orchestra.run.smithi164.stdout: "total_workers": 1, 2024-04-26T21:59:30.630 INFO:teuthology.orchestra.run.smithi164.stdout: "unhealthy_workers": 0 2024-04-26T21:59:30.630 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:30.630 INFO:teuthology.orchestra.run.smithi164.stdout: "finisher-MDSRank": { 2024-04-26T21:59:30.630 INFO:teuthology.orchestra.run.smithi164.stdout: "queue_len": 0, 2024-04-26T21:59:30.630 INFO:teuthology.orchestra.run.smithi164.stdout: "complete_latency": { 2024-04-26T21:59:30.630 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 2047, 2024-04-26T21:59:30.630 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.111110959, 2024-04-26T21:59:30.630 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000054279 2024-04-26T21:59:30.630 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-04-26T21:59:30.631 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:30.631 INFO:teuthology.orchestra.run.smithi164.stdout: "finisher-PurgeQueue": { 2024-04-26T21:59:30.631 INFO:teuthology.orchestra.run.smithi164.stdout: "queue_len": 0, 2024-04-26T21:59:30.631 INFO:teuthology.orchestra.run.smithi164.stdout: "complete_latency": { 2024-04-26T21:59:30.631 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-26T21:59:30.631 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-26T21:59:30.631 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-26T21:59:30.631 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-04-26T21:59:30.631 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:30.631 INFO:teuthology.orchestra.run.smithi164.stdout: "mds": { 2024-04-26T21:59:30.631 INFO:teuthology.orchestra.run.smithi164.stdout: "request": 0, 2024-04-26T21:59:30.631 INFO:teuthology.orchestra.run.smithi164.stdout: "reply": 0, 2024-04-26T21:59:30.631 INFO:teuthology.orchestra.run.smithi164.stdout: "reply_latency": { 2024-04-26T21:59:30.631 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-26T21:59:30.631 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-26T21:59:30.632 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-26T21:59:30.632 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:30.632 INFO:teuthology.orchestra.run.smithi164.stdout: "slow_reply": 0, 2024-04-26T21:59:30.632 INFO:teuthology.orchestra.run.smithi164.stdout: "forward": 0, 2024-04-26T21:59:30.632 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_fetch_complete": 0, 2024-04-26T21:59:30.632 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_fetch_keys": 0, 2024-04-26T21:59:30.632 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_commit": 0, 2024-04-26T21:59:30.632 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_split": 0, 2024-04-26T21:59:30.632 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_merge": 0, 2024-04-26T21:59:30.632 INFO:teuthology.orchestra.run.smithi164.stdout: "inodes": 21, 2024-04-26T21:59:30.632 INFO:teuthology.orchestra.run.smithi164.stdout: "inodes_top": 0, 2024-04-26T21:59:30.632 INFO:teuthology.orchestra.run.smithi164.stdout: "inodes_bottom": 21, 2024-04-26T21:59:30.632 INFO:teuthology.orchestra.run.smithi164.stdout: "inodes_pin_tail": 0, 2024-04-26T21:59:30.632 INFO:teuthology.orchestra.run.smithi164.stdout: "inodes_pinned": 21, 2024-04-26T21:59:30.632 INFO:teuthology.orchestra.run.smithi164.stdout: "inodes_expired": 0, 2024-04-26T21:59:30.632 INFO:teuthology.orchestra.run.smithi164.stdout: "inodes_with_caps": 0, 2024-04-26T21:59:30.633 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": 0, 2024-04-26T21:59:30.633 INFO:teuthology.orchestra.run.smithi164.stdout: "subtrees": 2, 2024-04-26T21:59:30.633 INFO:teuthology.orchestra.run.smithi164.stdout: "traverse": 0, 2024-04-26T21:59:30.633 INFO:teuthology.orchestra.run.smithi164.stdout: "traverse_hit": 0, 2024-04-26T21:59:30.633 INFO:teuthology.orchestra.run.smithi164.stdout: "traverse_forward": 0, 2024-04-26T21:59:30.633 INFO:teuthology.orchestra.run.smithi164.stdout: "traverse_discover": 0, 2024-04-26T21:59:30.633 INFO:teuthology.orchestra.run.smithi164.stdout: "traverse_dir_fetch": 0, 2024-04-26T21:59:30.633 INFO:teuthology.orchestra.run.smithi164.stdout: "traverse_remote_ino": 0, 2024-04-26T21:59:30.633 INFO:teuthology.orchestra.run.smithi164.stdout: "traverse_lock": 0, 2024-04-26T21:59:30.633 INFO:teuthology.orchestra.run.smithi164.stdout: "load_cent": 0, 2024-04-26T21:59:30.633 INFO:teuthology.orchestra.run.smithi164.stdout: "q": 0, 2024-04-26T21:59:30.633 INFO:teuthology.orchestra.run.smithi164.stdout: "exported": 0, 2024-04-26T21:59:30.633 INFO:teuthology.orchestra.run.smithi164.stdout: "exported_inodes": 0, 2024-04-26T21:59:30.633 INFO:teuthology.orchestra.run.smithi164.stdout: "imported": 0, 2024-04-26T21:59:30.634 INFO:teuthology.orchestra.run.smithi164.stdout: "imported_inodes": 0, 2024-04-26T21:59:30.634 INFO:teuthology.orchestra.run.smithi164.stdout: "openino_dir_fetch": 0, 2024-04-26T21:59:30.634 INFO:teuthology.orchestra.run.smithi164.stdout: "openino_backtrace_fetch": 0, 2024-04-26T21:59:30.634 INFO:teuthology.orchestra.run.smithi164.stdout: "openino_peer_discover": 0, 2024-04-26T21:59:30.634 INFO:teuthology.orchestra.run.smithi164.stdout: "root_rfiles": 4, 2024-04-26T21:59:30.634 INFO:teuthology.orchestra.run.smithi164.stdout: "root_rbytes": 234, 2024-04-26T21:59:30.634 INFO:teuthology.orchestra.run.smithi164.stdout: "root_rsnaps": 0, 2024-04-26T21:59:30.634 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_backtrace_fetch": 0, 2024-04-26T21:59:30.634 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_set_tag": 0, 2024-04-26T21:59:30.634 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_backtrace_repaired": 0, 2024-04-26T21:59:30.634 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_inotable_repaired": 0, 2024-04-26T21:59:30.634 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_dir_inodes": 0, 2024-04-26T21:59:30.634 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_dir_base_inodes": 0, 2024-04-26T21:59:30.634 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_dirfrag_rstats": 0, 2024-04-26T21:59:30.635 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_file_inodes": 0, 2024-04-26T21:59:30.635 INFO:teuthology.orchestra.run.smithi164.stdout: "handle_inode_file_caps": 0, 2024-04-26T21:59:30.635 INFO:teuthology.orchestra.run.smithi164.stdout: "ceph_cap_op_revoke": 0, 2024-04-26T21:59:30.635 INFO:teuthology.orchestra.run.smithi164.stdout: "ceph_cap_op_grant": 0, 2024-04-26T21:59:30.635 INFO:teuthology.orchestra.run.smithi164.stdout: "ceph_cap_op_trunc": 0, 2024-04-26T21:59:30.635 INFO:teuthology.orchestra.run.smithi164.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-26T21:59:30.635 INFO:teuthology.orchestra.run.smithi164.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-26T21:59:30.635 INFO:teuthology.orchestra.run.smithi164.stdout: "handle_client_caps": 0, 2024-04-26T21:59:30.635 INFO:teuthology.orchestra.run.smithi164.stdout: "handle_client_caps_dirty": 0, 2024-04-26T21:59:30.635 INFO:teuthology.orchestra.run.smithi164.stdout: "handle_client_cap_release": 0, 2024-04-26T21:59:30.635 INFO:teuthology.orchestra.run.smithi164.stdout: "process_request_cap_release": 0 2024-04-26T21:59:30.635 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:30.635 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_cache": { 2024-04-26T21:59:30.635 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_update": 0, 2024-04-26T21:59:30.635 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_update_receipt": 0, 2024-04-26T21:59:30.636 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_try_discover": 0, 2024-04-26T21:59:30.636 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_send_discover": 0, 2024-04-26T21:59:30.636 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_handle_discover": 0, 2024-04-26T21:59:30.636 INFO:teuthology.orchestra.run.smithi164.stdout: "num_strays": 0, 2024-04-26T21:59:30.636 INFO:teuthology.orchestra.run.smithi164.stdout: "num_strays_delayed": 0, 2024-04-26T21:59:30.636 INFO:teuthology.orchestra.run.smithi164.stdout: "num_strays_enqueuing": 0, 2024-04-26T21:59:30.636 INFO:teuthology.orchestra.run.smithi164.stdout: "strays_created": 0, 2024-04-26T21:59:30.636 INFO:teuthology.orchestra.run.smithi164.stdout: "strays_enqueued": 0, 2024-04-26T21:59:30.636 INFO:teuthology.orchestra.run.smithi164.stdout: "strays_reintegrated": 0, 2024-04-26T21:59:30.636 INFO:teuthology.orchestra.run.smithi164.stdout: "strays_migrated": 0, 2024-04-26T21:59:30.636 INFO:teuthology.orchestra.run.smithi164.stdout: "num_recovering_processing": 0, 2024-04-26T21:59:30.636 INFO:teuthology.orchestra.run.smithi164.stdout: "num_recovering_enqueued": 0, 2024-04-26T21:59:30.636 INFO:teuthology.orchestra.run.smithi164.stdout: "num_recovering_prioritized": 0, 2024-04-26T21:59:30.636 INFO:teuthology.orchestra.run.smithi164.stdout: "recovery_started": 0, 2024-04-26T21:59:30.637 INFO:teuthology.orchestra.run.smithi164.stdout: "recovery_completed": 0, 2024-04-26T21:59:30.637 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_quiesce_path": 0, 2024-04-26T21:59:30.637 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_quiesce_inode": 0, 2024-04-26T21:59:30.637 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_enqueue_scrub": 0, 2024-04-26T21:59:30.637 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_exportdir": 0, 2024-04-26T21:59:30.637 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_flush": 0, 2024-04-26T21:59:30.637 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_fragmentdir": 0, 2024-04-26T21:59:30.637 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_fragstats": 0, 2024-04-26T21:59:30.637 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_inodestats": 0 2024-04-26T21:59:30.637 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:30.637 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_log": { 2024-04-26T21:59:30.637 INFO:teuthology.orchestra.run.smithi164.stdout: "evlrg": 0, 2024-04-26T21:59:30.637 INFO:teuthology.orchestra.run.smithi164.stdout: "evadd": 0, 2024-04-26T21:59:30.637 INFO:teuthology.orchestra.run.smithi164.stdout: "evex": 0, 2024-04-26T21:59:30.637 INFO:teuthology.orchestra.run.smithi164.stdout: "evtrm": 0, 2024-04-26T21:59:30.638 INFO:teuthology.orchestra.run.smithi164.stdout: "ev": 135, 2024-04-26T21:59:30.638 INFO:teuthology.orchestra.run.smithi164.stdout: "evexg": 0, 2024-04-26T21:59:30.638 INFO:teuthology.orchestra.run.smithi164.stdout: "evexd": 0, 2024-04-26T21:59:30.638 INFO:teuthology.orchestra.run.smithi164.stdout: "segadd": 0, 2024-04-26T21:59:30.638 INFO:teuthology.orchestra.run.smithi164.stdout: "segex": 0, 2024-04-26T21:59:30.638 INFO:teuthology.orchestra.run.smithi164.stdout: "segtrm": 0, 2024-04-26T21:59:30.638 INFO:teuthology.orchestra.run.smithi164.stdout: "seg": 2, 2024-04-26T21:59:30.638 INFO:teuthology.orchestra.run.smithi164.stdout: "segmjr": 2, 2024-04-26T21:59:30.638 INFO:teuthology.orchestra.run.smithi164.stdout: "segexg": 0, 2024-04-26T21:59:30.638 INFO:teuthology.orchestra.run.smithi164.stdout: "segexd": 0, 2024-04-26T21:59:30.638 INFO:teuthology.orchestra.run.smithi164.stdout: "expos": 4194304, 2024-04-26T21:59:30.638 INFO:teuthology.orchestra.run.smithi164.stdout: "wrpos": 4799575, 2024-04-26T21:59:30.638 INFO:teuthology.orchestra.run.smithi164.stdout: "rdpos": 4797924, 2024-04-26T21:59:30.638 INFO:teuthology.orchestra.run.smithi164.stdout: "jlat": { 2024-04-26T21:59:30.638 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-26T21:59:30.639 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-26T21:59:30.639 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-26T21:59:30.639 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:30.639 INFO:teuthology.orchestra.run.smithi164.stdout: "replayed": 135 2024-04-26T21:59:30.639 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:30.639 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_mem": { 2024-04-26T21:59:30.639 INFO:teuthology.orchestra.run.smithi164.stdout: "ino": 15, 2024-04-26T21:59:30.639 INFO:teuthology.orchestra.run.smithi164.stdout: "ino+": 20, 2024-04-26T21:59:30.639 INFO:teuthology.orchestra.run.smithi164.stdout: "ino-": 5, 2024-04-26T21:59:30.639 INFO:teuthology.orchestra.run.smithi164.stdout: "dir": 10, 2024-04-26T21:59:30.639 INFO:teuthology.orchestra.run.smithi164.stdout: "dir+": 11, 2024-04-26T21:59:30.639 INFO:teuthology.orchestra.run.smithi164.stdout: "dir-": 1, 2024-04-26T21:59:30.639 INFO:teuthology.orchestra.run.smithi164.stdout: "dn": 21, 2024-04-26T21:59:30.639 INFO:teuthology.orchestra.run.smithi164.stdout: "dn+": 25, 2024-04-26T21:59:30.639 INFO:teuthology.orchestra.run.smithi164.stdout: "dn-": 4, 2024-04-26T21:59:30.640 INFO:teuthology.orchestra.run.smithi164.stdout: "cap": 0, 2024-04-26T21:59:30.640 INFO:teuthology.orchestra.run.smithi164.stdout: "cap+": 0, 2024-04-26T21:59:30.640 INFO:teuthology.orchestra.run.smithi164.stdout: "cap-": 0, 2024-04-26T21:59:30.640 INFO:teuthology.orchestra.run.smithi164.stdout: "rss": 49664, 2024-04-26T21:59:30.640 INFO:teuthology.orchestra.run.smithi164.stdout: "heap": 215416 2024-04-26T21:59:30.640 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:30.640 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_server": { 2024-04-26T21:59:30.640 INFO:teuthology.orchestra.run.smithi164.stdout: "dispatch_client_request": 0, 2024-04-26T21:59:30.640 INFO:teuthology.orchestra.run.smithi164.stdout: "dispatch_server_request": 0, 2024-04-26T21:59:30.640 INFO:teuthology.orchestra.run.smithi164.stdout: "handle_client_request": 0, 2024-04-26T21:59:30.640 INFO:teuthology.orchestra.run.smithi164.stdout: "handle_client_session": 0, 2024-04-26T21:59:30.640 INFO:teuthology.orchestra.run.smithi164.stdout: "handle_peer_request": 0, 2024-04-26T21:59:30.640 INFO:teuthology.orchestra.run.smithi164.stdout: "req_create_latency": { 2024-04-26T21:59:30.640 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-26T21:59:30.640 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-26T21:59:30.641 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-26T21:59:30.641 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:30.641 INFO:teuthology.orchestra.run.smithi164.stdout: "req_getattr_latency": { 2024-04-26T21:59:30.641 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-26T21:59:30.641 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-26T21:59:30.641 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-26T21:59:30.641 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:30.641 INFO:teuthology.orchestra.run.smithi164.stdout: "req_getfilelock_latency": { 2024-04-26T21:59:30.641 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-26T21:59:30.641 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-26T21:59:30.641 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-26T21:59:30.641 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:30.641 INFO:teuthology.orchestra.run.smithi164.stdout: "req_link_latency": { 2024-04-26T21:59:30.641 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-26T21:59:30.642 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-26T21:59:30.642 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-26T21:59:30.642 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:30.642 INFO:teuthology.orchestra.run.smithi164.stdout: "req_lookup_latency": { 2024-04-26T21:59:30.642 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-26T21:59:30.642 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-26T21:59:30.642 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-26T21:59:30.642 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:30.642 INFO:teuthology.orchestra.run.smithi164.stdout: "req_lookuphash_latency": { 2024-04-26T21:59:30.642 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-26T21:59:30.642 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-26T21:59:30.642 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-26T21:59:30.642 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:30.642 INFO:teuthology.orchestra.run.smithi164.stdout: "req_lookupino_latency": { 2024-04-26T21:59:30.642 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-26T21:59:30.643 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-26T21:59:30.643 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-26T21:59:30.643 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:30.643 INFO:teuthology.orchestra.run.smithi164.stdout: "req_lookupname_latency": { 2024-04-26T21:59:30.643 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-26T21:59:30.643 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-26T21:59:30.643 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-26T21:59:30.643 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:30.643 INFO:teuthology.orchestra.run.smithi164.stdout: "req_lookupparent_latency": { 2024-04-26T21:59:30.643 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-26T21:59:30.643 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-26T21:59:30.643 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-26T21:59:30.643 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:30.643 INFO:teuthology.orchestra.run.smithi164.stdout: "req_lookupsnap_latency": { 2024-04-26T21:59:30.643 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-26T21:59:30.643 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-26T21:59:30.644 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-26T21:59:30.644 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:30.644 INFO:teuthology.orchestra.run.smithi164.stdout: "req_lssnap_latency": { 2024-04-26T21:59:30.644 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-26T21:59:30.644 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-26T21:59:30.644 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-26T21:59:30.644 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:30.644 INFO:teuthology.orchestra.run.smithi164.stdout: "req_mkdir_latency": { 2024-04-26T21:59:30.644 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-26T21:59:30.645 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-26T21:59:30.645 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-26T21:59:30.645 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:30.645 INFO:teuthology.orchestra.run.smithi164.stdout: "req_mknod_latency": { 2024-04-26T21:59:30.645 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-26T21:59:30.645 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-26T21:59:30.645 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-26T21:59:30.645 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:30.645 INFO:teuthology.orchestra.run.smithi164.stdout: "req_mksnap_latency": { 2024-04-26T21:59:30.645 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-26T21:59:30.645 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-26T21:59:30.645 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-26T21:59:30.646 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:30.646 INFO:teuthology.orchestra.run.smithi164.stdout: "req_open_latency": { 2024-04-26T21:59:30.646 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-26T21:59:30.646 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-26T21:59:30.646 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-26T21:59:30.646 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:30.646 INFO:teuthology.orchestra.run.smithi164.stdout: "req_readdir_latency": { 2024-04-26T21:59:30.646 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-26T21:59:30.646 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-26T21:59:30.646 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-26T21:59:30.646 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:30.646 INFO:teuthology.orchestra.run.smithi164.stdout: "req_rename_latency": { 2024-04-26T21:59:30.646 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-26T21:59:30.647 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-26T21:59:30.647 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-26T21:59:30.647 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:30.647 INFO:teuthology.orchestra.run.smithi164.stdout: "req_renamesnap_latency": { 2024-04-26T21:59:30.647 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-26T21:59:30.647 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-26T21:59:30.647 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-26T21:59:30.647 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:30.647 INFO:teuthology.orchestra.run.smithi164.stdout: "req_snapdiff_latency": { 2024-04-26T21:59:30.647 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-26T21:59:30.647 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-26T21:59:30.647 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-26T21:59:30.647 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:30.647 INFO:teuthology.orchestra.run.smithi164.stdout: "req_rmdir_latency": { 2024-04-26T21:59:30.648 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-26T21:59:30.648 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-26T21:59:30.648 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-26T21:59:30.648 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:30.648 INFO:teuthology.orchestra.run.smithi164.stdout: "req_rmsnap_latency": { 2024-04-26T21:59:30.648 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-26T21:59:30.648 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-26T21:59:30.648 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-26T21:59:30.648 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:30.648 INFO:teuthology.orchestra.run.smithi164.stdout: "req_rmxattr_latency": { 2024-04-26T21:59:30.648 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-26T21:59:30.648 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-26T21:59:30.648 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-26T21:59:30.648 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:30.648 INFO:teuthology.orchestra.run.smithi164.stdout: "req_setattr_latency": { 2024-04-26T21:59:30.649 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-26T21:59:30.649 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-26T21:59:30.649 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-26T21:59:30.649 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:30.649 INFO:teuthology.orchestra.run.smithi164.stdout: "req_setdirlayout_latency": { 2024-04-26T21:59:30.649 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-26T21:59:30.649 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-26T21:59:30.649 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-26T21:59:30.649 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:30.649 INFO:teuthology.orchestra.run.smithi164.stdout: "req_setfilelock_latency": { 2024-04-26T21:59:30.649 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-26T21:59:30.649 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-26T21:59:30.649 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-26T21:59:30.649 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:30.650 INFO:teuthology.orchestra.run.smithi164.stdout: "req_setlayout_latency": { 2024-04-26T21:59:30.650 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-26T21:59:30.650 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-26T21:59:30.650 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-26T21:59:30.650 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:30.650 INFO:teuthology.orchestra.run.smithi164.stdout: "req_setxattr_latency": { 2024-04-26T21:59:30.650 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-26T21:59:30.650 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-26T21:59:30.650 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-26T21:59:30.650 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:30.650 INFO:teuthology.orchestra.run.smithi164.stdout: "req_symlink_latency": { 2024-04-26T21:59:30.650 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-26T21:59:30.650 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-26T21:59:30.650 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-26T21:59:30.651 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:30.651 INFO:teuthology.orchestra.run.smithi164.stdout: "req_unlink_latency": { 2024-04-26T21:59:30.651 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-26T21:59:30.651 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-26T21:59:30.651 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-26T21:59:30.651 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:30.651 INFO:teuthology.orchestra.run.smithi164.stdout: "cap_revoke_eviction": 0, 2024-04-26T21:59:30.651 INFO:teuthology.orchestra.run.smithi164.stdout: "cap_acquisition_throttle": 0, 2024-04-26T21:59:30.651 INFO:teuthology.orchestra.run.smithi164.stdout: "req_getvxattr_latency": { 2024-04-26T21:59:30.651 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-26T21:59:30.651 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-26T21:59:30.651 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-26T21:59:30.651 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-04-26T21:59:30.651 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:30.651 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_sessions": { 2024-04-26T21:59:30.652 INFO:teuthology.orchestra.run.smithi164.stdout: "session_count": 2, 2024-04-26T21:59:30.652 INFO:teuthology.orchestra.run.smithi164.stdout: "session_add": 2, 2024-04-26T21:59:30.652 INFO:teuthology.orchestra.run.smithi164.stdout: "session_remove": 0, 2024-04-26T21:59:30.652 INFO:teuthology.orchestra.run.smithi164.stdout: "sessions_open": 2, 2024-04-26T21:59:30.652 INFO:teuthology.orchestra.run.smithi164.stdout: "sessions_stale": 0, 2024-04-26T21:59:30.652 INFO:teuthology.orchestra.run.smithi164.stdout: "total_load": 0, 2024-04-26T21:59:30.652 INFO:teuthology.orchestra.run.smithi164.stdout: "average_load": 0, 2024-04-26T21:59:30.652 INFO:teuthology.orchestra.run.smithi164.stdout: "avg_session_uptime": 1684, 2024-04-26T21:59:30.652 INFO:teuthology.orchestra.run.smithi164.stdout: "mdthresh_evicted": 0 2024-04-26T21:59:30.652 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:30.652 INFO:teuthology.orchestra.run.smithi164.stdout: "mempool": { 2024-04-26T21:59:30.652 INFO:teuthology.orchestra.run.smithi164.stdout: "bloom_filter_bytes": 0, 2024-04-26T21:59:30.652 INFO:teuthology.orchestra.run.smithi164.stdout: "bloom_filter_items": 0, 2024-04-26T21:59:30.652 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_alloc_bytes": 0, 2024-04-26T21:59:30.652 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_alloc_items": 0, 2024-04-26T21:59:30.653 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_data_bytes": 0, 2024-04-26T21:59:30.653 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_data_items": 0, 2024-04-26T21:59:30.653 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-26T21:59:30.653 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_onode_items": 0, 2024-04-26T21:59:30.653 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-26T21:59:30.653 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_meta_items": 0, 2024-04-26T21:59:30.653 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_other_bytes": 0, 2024-04-26T21:59:30.653 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_other_items": 0, 2024-04-26T21:59:30.653 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-26T21:59:30.653 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_buffer_items": 0, 2024-04-26T21:59:30.653 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_extent_bytes": 0, 2024-04-26T21:59:30.653 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_extent_items": 0, 2024-04-26T21:59:30.653 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_blob_bytes": 0, 2024-04-26T21:59:30.653 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_blob_items": 0, 2024-04-26T21:59:30.653 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-26T21:59:30.653 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_shared_blob_items": 0, 2024-04-26T21:59:30.654 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-26T21:59:30.654 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_inline_bl_items": 0, 2024-04-26T21:59:30.654 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_fsck_bytes": 0, 2024-04-26T21:59:30.654 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_fsck_items": 0, 2024-04-26T21:59:30.654 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_txc_bytes": 0, 2024-04-26T21:59:30.654 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_txc_items": 0, 2024-04-26T21:59:30.654 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-26T21:59:30.654 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_writing_deferred_items": 0, 2024-04-26T21:59:30.654 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_writing_bytes": 0, 2024-04-26T21:59:30.654 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_writing_items": 0, 2024-04-26T21:59:30.654 INFO:teuthology.orchestra.run.smithi164.stdout: "bluefs_bytes": 0, 2024-04-26T21:59:30.654 INFO:teuthology.orchestra.run.smithi164.stdout: "bluefs_items": 0, 2024-04-26T21:59:30.654 INFO:teuthology.orchestra.run.smithi164.stdout: "bluefs_file_reader_bytes": 0, 2024-04-26T21:59:30.654 INFO:teuthology.orchestra.run.smithi164.stdout: "bluefs_file_reader_items": 0, 2024-04-26T21:59:30.655 INFO:teuthology.orchestra.run.smithi164.stdout: "bluefs_file_writer_bytes": 0, 2024-04-26T21:59:30.655 INFO:teuthology.orchestra.run.smithi164.stdout: "bluefs_file_writer_items": 0, 2024-04-26T21:59:30.655 INFO:teuthology.orchestra.run.smithi164.stdout: "buffer_anon_bytes": 189256, 2024-04-26T21:59:30.655 INFO:teuthology.orchestra.run.smithi164.stdout: "buffer_anon_items": 49, 2024-04-26T21:59:30.655 INFO:teuthology.orchestra.run.smithi164.stdout: "buffer_meta_bytes": 0, 2024-04-26T21:59:30.655 INFO:teuthology.orchestra.run.smithi164.stdout: "buffer_meta_items": 0, 2024-04-26T21:59:30.655 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_bytes": 0, 2024-04-26T21:59:30.655 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_items": 0, 2024-04-26T21:59:30.655 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_mapbl_bytes": 0, 2024-04-26T21:59:30.655 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_mapbl_items": 0, 2024-04-26T21:59:30.655 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_pglog_bytes": 0, 2024-04-26T21:59:30.655 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_pglog_items": 0, 2024-04-26T21:59:30.655 INFO:teuthology.orchestra.run.smithi164.stdout: "osdmap_bytes": 8008, 2024-04-26T21:59:30.655 INFO:teuthology.orchestra.run.smithi164.stdout: "osdmap_items": 168, 2024-04-26T21:59:30.655 INFO:teuthology.orchestra.run.smithi164.stdout: "osdmap_mapping_bytes": 0, 2024-04-26T21:59:30.655 INFO:teuthology.orchestra.run.smithi164.stdout: "osdmap_mapping_items": 0, 2024-04-26T21:59:30.655 INFO:teuthology.orchestra.run.smithi164.stdout: "pgmap_bytes": 0, 2024-04-26T21:59:30.656 INFO:teuthology.orchestra.run.smithi164.stdout: "pgmap_items": 0, 2024-04-26T21:59:30.656 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_co_bytes": 61321, 2024-04-26T21:59:30.656 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_co_items": 348, 2024-04-26T21:59:30.656 INFO:teuthology.orchestra.run.smithi164.stdout: "unittest_1_bytes": 0, 2024-04-26T21:59:30.656 INFO:teuthology.orchestra.run.smithi164.stdout: "unittest_1_items": 0, 2024-04-26T21:59:30.656 INFO:teuthology.orchestra.run.smithi164.stdout: "unittest_2_bytes": 0, 2024-04-26T21:59:30.656 INFO:teuthology.orchestra.run.smithi164.stdout: "unittest_2_items": 0 2024-04-26T21:59:30.656 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:30.656 INFO:teuthology.orchestra.run.smithi164.stdout: "objecter": { 2024-04-26T21:59:30.656 INFO:teuthology.orchestra.run.smithi164.stdout: "op_active": 0, 2024-04-26T21:59:30.656 INFO:teuthology.orchestra.run.smithi164.stdout: "op_laggy": 0, 2024-04-26T21:59:30.656 INFO:teuthology.orchestra.run.smithi164.stdout: "op_send": 1241, 2024-04-26T21:59:30.656 INFO:teuthology.orchestra.run.smithi164.stdout: "op_send_bytes": 26, 2024-04-26T21:59:30.656 INFO:teuthology.orchestra.run.smithi164.stdout: "op_resend": 4, 2024-04-26T21:59:30.656 INFO:teuthology.orchestra.run.smithi164.stdout: "op_reply": 1241, 2024-04-26T21:59:30.657 INFO:teuthology.orchestra.run.smithi164.stdout: "op_latency": { 2024-04-26T21:59:30.657 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 1241, 2024-04-26T21:59:30.657 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 3.393941611, 2024-04-26T21:59:30.657 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.002734844 2024-04-26T21:59:30.657 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:30.657 INFO:teuthology.orchestra.run.smithi164.stdout: "op_inflight": 0, 2024-04-26T21:59:30.657 INFO:teuthology.orchestra.run.smithi164.stdout: "oplen_avg": { 2024-04-26T21:59:30.657 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 1241, 2024-04-26T21:59:30.657 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 1242 2024-04-26T21:59:30.657 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:30.657 INFO:teuthology.orchestra.run.smithi164.stdout: "op": 1241, 2024-04-26T21:59:30.657 INFO:teuthology.orchestra.run.smithi164.stdout: "op_r": 1241, 2024-04-26T21:59:30.657 INFO:teuthology.orchestra.run.smithi164.stdout: "op_w": 0, 2024-04-26T21:59:30.657 INFO:teuthology.orchestra.run.smithi164.stdout: "op_rmw": 0, 2024-04-26T21:59:30.657 INFO:teuthology.orchestra.run.smithi164.stdout: "op_pg": 0, 2024-04-26T21:59:30.657 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_stat": 761, 2024-04-26T21:59:30.657 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_create": 0, 2024-04-26T21:59:30.658 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_read": 477, 2024-04-26T21:59:30.658 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_write": 0, 2024-04-26T21:59:30.658 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_writefull": 0, 2024-04-26T21:59:30.658 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_writesame": 0, 2024-04-26T21:59:30.658 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_append": 0, 2024-04-26T21:59:30.658 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_zero": 0, 2024-04-26T21:59:30.658 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_truncate": 0, 2024-04-26T21:59:30.658 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_delete": 0, 2024-04-26T21:59:30.658 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_mapext": 0, 2024-04-26T21:59:30.658 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_sparse_read": 0, 2024-04-26T21:59:30.658 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_clonerange": 0, 2024-04-26T21:59:30.658 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_getxattr": 2, 2024-04-26T21:59:30.658 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_setxattr": 0, 2024-04-26T21:59:30.658 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_cmpxattr": 0, 2024-04-26T21:59:30.659 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_rmxattr": 0, 2024-04-26T21:59:30.659 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_resetxattrs": 0, 2024-04-26T21:59:30.659 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_call": 0, 2024-04-26T21:59:30.659 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_watch": 0, 2024-04-26T21:59:30.659 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_notify": 0, 2024-04-26T21:59:30.659 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_src_cmpxattr": 0, 2024-04-26T21:59:30.659 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_pgls": 0, 2024-04-26T21:59:30.659 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_pgls_filter": 0, 2024-04-26T21:59:30.659 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_other": 0, 2024-04-26T21:59:30.659 INFO:teuthology.orchestra.run.smithi164.stdout: "linger_active": 0, 2024-04-26T21:59:30.659 INFO:teuthology.orchestra.run.smithi164.stdout: "linger_send": 0, 2024-04-26T21:59:30.659 INFO:teuthology.orchestra.run.smithi164.stdout: "linger_resend": 0, 2024-04-26T21:59:30.659 INFO:teuthology.orchestra.run.smithi164.stdout: "linger_ping": 0, 2024-04-26T21:59:30.659 INFO:teuthology.orchestra.run.smithi164.stdout: "poolop_active": 0, 2024-04-26T21:59:30.659 INFO:teuthology.orchestra.run.smithi164.stdout: "poolop_send": 0, 2024-04-26T21:59:30.660 INFO:teuthology.orchestra.run.smithi164.stdout: "poolop_resend": 0, 2024-04-26T21:59:30.660 INFO:teuthology.orchestra.run.smithi164.stdout: "poolstat_active": 0, 2024-04-26T21:59:30.660 INFO:teuthology.orchestra.run.smithi164.stdout: "poolstat_send": 0, 2024-04-26T21:59:30.660 INFO:teuthology.orchestra.run.smithi164.stdout: "poolstat_resend": 0, 2024-04-26T21:59:30.660 INFO:teuthology.orchestra.run.smithi164.stdout: "statfs_active": 0, 2024-04-26T21:59:30.660 INFO:teuthology.orchestra.run.smithi164.stdout: "statfs_send": 0, 2024-04-26T21:59:30.660 INFO:teuthology.orchestra.run.smithi164.stdout: "statfs_resend": 0, 2024-04-26T21:59:30.660 INFO:teuthology.orchestra.run.smithi164.stdout: "command_active": 0, 2024-04-26T21:59:30.660 INFO:teuthology.orchestra.run.smithi164.stdout: "command_send": 0, 2024-04-26T21:59:30.660 INFO:teuthology.orchestra.run.smithi164.stdout: "command_resend": 0, 2024-04-26T21:59:30.660 INFO:teuthology.orchestra.run.smithi164.stdout: "map_epoch": 91, 2024-04-26T21:59:30.660 INFO:teuthology.orchestra.run.smithi164.stdout: "map_full": 0, 2024-04-26T21:59:30.660 INFO:teuthology.orchestra.run.smithi164.stdout: "map_inc": 4, 2024-04-26T21:59:30.660 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_sessions": 5, 2024-04-26T21:59:30.660 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_session_open": 5, 2024-04-26T21:59:30.660 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_session_close": 0, 2024-04-26T21:59:30.661 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_laggy": 0, 2024-04-26T21:59:30.661 INFO:teuthology.orchestra.run.smithi164.stdout: "omap_wr": 0, 2024-04-26T21:59:30.661 INFO:teuthology.orchestra.run.smithi164.stdout: "omap_rd": 2, 2024-04-26T21:59:30.661 INFO:teuthology.orchestra.run.smithi164.stdout: "omap_del": 0 2024-04-26T21:59:30.661 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:30.661 INFO:teuthology.orchestra.run.smithi164.stdout: "oft": { 2024-04-26T21:59:30.661 INFO:teuthology.orchestra.run.smithi164.stdout: "omap_total_objs": 0, 2024-04-26T21:59:30.661 INFO:teuthology.orchestra.run.smithi164.stdout: "omap_total_kv_pairs": 0, 2024-04-26T21:59:30.661 INFO:teuthology.orchestra.run.smithi164.stdout: "omap_total_updates": 0, 2024-04-26T21:59:30.661 INFO:teuthology.orchestra.run.smithi164.stdout: "omap_total_removes": 0 2024-04-26T21:59:30.661 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:30.661 INFO:teuthology.orchestra.run.smithi164.stdout: "osd-slow-ops": { 2024-04-26T21:59:30.661 INFO:teuthology.orchestra.run.smithi164.stdout: "slow_ops_count": 0 2024-04-26T21:59:30.661 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:30.661 INFO:teuthology.orchestra.run.smithi164.stdout: "purge_queue": { 2024-04-26T21:59:30.661 INFO:teuthology.orchestra.run.smithi164.stdout: "pq_executing_ops": 0, 2024-04-26T21:59:30.662 INFO:teuthology.orchestra.run.smithi164.stdout: "pq_executing_ops_high_water": 0, 2024-04-26T21:59:30.662 INFO:teuthology.orchestra.run.smithi164.stdout: "pq_executing": 0, 2024-04-26T21:59:30.662 INFO:teuthology.orchestra.run.smithi164.stdout: "pq_executing_high_water": 0, 2024-04-26T21:59:30.662 INFO:teuthology.orchestra.run.smithi164.stdout: "pq_executed_ops": 0, 2024-04-26T21:59:30.662 INFO:teuthology.orchestra.run.smithi164.stdout: "pq_executed": 0, 2024-04-26T21:59:30.662 INFO:teuthology.orchestra.run.smithi164.stdout: "pq_item_in_journal": 0 2024-04-26T21:59:30.662 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:30.662 INFO:teuthology.orchestra.run.smithi164.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-26T21:59:30.662 INFO:teuthology.orchestra.run.smithi164.stdout: "val": 0, 2024-04-26T21:59:30.662 INFO:teuthology.orchestra.run.smithi164.stdout: "max": 104857600, 2024-04-26T21:59:30.662 INFO:teuthology.orchestra.run.smithi164.stdout: "get_started": 0, 2024-04-26T21:59:30.662 INFO:teuthology.orchestra.run.smithi164.stdout: "get": 1373, 2024-04-26T21:59:30.662 INFO:teuthology.orchestra.run.smithi164.stdout: "get_sum": 1161292, 2024-04-26T21:59:30.662 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_fail": 0, 2024-04-26T21:59:30.663 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_success": 1373, 2024-04-26T21:59:30.663 INFO:teuthology.orchestra.run.smithi164.stdout: "take": 0, 2024-04-26T21:59:30.663 INFO:teuthology.orchestra.run.smithi164.stdout: "take_sum": 0, 2024-04-26T21:59:30.663 INFO:teuthology.orchestra.run.smithi164.stdout: "put": 1373, 2024-04-26T21:59:30.663 INFO:teuthology.orchestra.run.smithi164.stdout: "put_sum": 1161292, 2024-04-26T21:59:30.663 INFO:teuthology.orchestra.run.smithi164.stdout: "wait": { 2024-04-26T21:59:30.663 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-26T21:59:30.663 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-26T21:59:30.663 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-26T21:59:30.663 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-04-26T21:59:30.663 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:30.663 INFO:teuthology.orchestra.run.smithi164.stdout: "throttle-objecter_bytes": { 2024-04-26T21:59:30.663 INFO:teuthology.orchestra.run.smithi164.stdout: "val": 0, 2024-04-26T21:59:30.663 INFO:teuthology.orchestra.run.smithi164.stdout: "max": 104857600, 2024-04-26T21:59:30.663 INFO:teuthology.orchestra.run.smithi164.stdout: "get_started": 0, 2024-04-26T21:59:30.663 INFO:teuthology.orchestra.run.smithi164.stdout: "get": 0, 2024-04-26T21:59:30.664 INFO:teuthology.orchestra.run.smithi164.stdout: "get_sum": 0, 2024-04-26T21:59:30.664 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_fail": 0, 2024-04-26T21:59:30.664 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_success": 0, 2024-04-26T21:59:30.664 INFO:teuthology.orchestra.run.smithi164.stdout: "take": 1241, 2024-04-26T21:59:30.664 INFO:teuthology.orchestra.run.smithi164.stdout: "take_sum": 605281, 2024-04-26T21:59:30.664 INFO:teuthology.orchestra.run.smithi164.stdout: "put": 1241, 2024-04-26T21:59:30.664 INFO:teuthology.orchestra.run.smithi164.stdout: "put_sum": 605281, 2024-04-26T21:59:30.664 INFO:teuthology.orchestra.run.smithi164.stdout: "wait": { 2024-04-26T21:59:30.664 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-26T21:59:30.664 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-26T21:59:30.664 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-26T21:59:30.664 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-04-26T21:59:30.664 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:30.664 INFO:teuthology.orchestra.run.smithi164.stdout: "throttle-objecter_ops": { 2024-04-26T21:59:30.664 INFO:teuthology.orchestra.run.smithi164.stdout: "val": 0, 2024-04-26T21:59:30.664 INFO:teuthology.orchestra.run.smithi164.stdout: "max": 1024, 2024-04-26T21:59:30.665 INFO:teuthology.orchestra.run.smithi164.stdout: "get_started": 0, 2024-04-26T21:59:30.665 INFO:teuthology.orchestra.run.smithi164.stdout: "get": 0, 2024-04-26T21:59:30.665 INFO:teuthology.orchestra.run.smithi164.stdout: "get_sum": 0, 2024-04-26T21:59:30.665 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_fail": 0, 2024-04-26T21:59:30.665 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_success": 0, 2024-04-26T21:59:30.665 INFO:teuthology.orchestra.run.smithi164.stdout: "take": 1241, 2024-04-26T21:59:30.665 INFO:teuthology.orchestra.run.smithi164.stdout: "take_sum": 1241, 2024-04-26T21:59:30.665 INFO:teuthology.orchestra.run.smithi164.stdout: "put": 1241, 2024-04-26T21:59:30.665 INFO:teuthology.orchestra.run.smithi164.stdout: "put_sum": 1241, 2024-04-26T21:59:30.665 INFO:teuthology.orchestra.run.smithi164.stdout: "wait": { 2024-04-26T21:59:30.665 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-26T21:59:30.665 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-26T21:59:30.665 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-26T21:59:30.665 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-04-26T21:59:30.665 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:30.665 INFO:teuthology.orchestra.run.smithi164.stdout: "throttle-write_buf_throttle": { 2024-04-26T21:59:30.666 INFO:teuthology.orchestra.run.smithi164.stdout: "val": 0, 2024-04-26T21:59:30.666 INFO:teuthology.orchestra.run.smithi164.stdout: "max": 3758096384, 2024-04-26T21:59:30.666 INFO:teuthology.orchestra.run.smithi164.stdout: "get_started": 0, 2024-04-26T21:59:30.666 INFO:teuthology.orchestra.run.smithi164.stdout: "get": 0, 2024-04-26T21:59:30.666 INFO:teuthology.orchestra.run.smithi164.stdout: "get_sum": 0, 2024-04-26T21:59:30.666 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_fail": 0, 2024-04-26T21:59:30.666 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_success": 0, 2024-04-26T21:59:30.666 INFO:teuthology.orchestra.run.smithi164.stdout: "take": 0, 2024-04-26T21:59:30.666 INFO:teuthology.orchestra.run.smithi164.stdout: "take_sum": 0, 2024-04-26T21:59:30.666 INFO:teuthology.orchestra.run.smithi164.stdout: "put": 0, 2024-04-26T21:59:30.666 INFO:teuthology.orchestra.run.smithi164.stdout: "put_sum": 0, 2024-04-26T21:59:30.666 INFO:teuthology.orchestra.run.smithi164.stdout: "wait": { 2024-04-26T21:59:30.666 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-26T21:59:30.666 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-26T21:59:30.666 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-26T21:59:30.667 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-04-26T21:59:30.667 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:30.667 INFO:teuthology.orchestra.run.smithi164.stdout: "throttle-write_buf_throttle-0x564c78d3c780": { 2024-04-26T21:59:30.667 INFO:teuthology.orchestra.run.smithi164.stdout: "val": 0, 2024-04-26T21:59:30.667 INFO:teuthology.orchestra.run.smithi164.stdout: "max": 3758096384, 2024-04-26T21:59:30.667 INFO:teuthology.orchestra.run.smithi164.stdout: "get_started": 0, 2024-04-26T21:59:30.667 INFO:teuthology.orchestra.run.smithi164.stdout: "get": 0, 2024-04-26T21:59:30.667 INFO:teuthology.orchestra.run.smithi164.stdout: "get_sum": 0, 2024-04-26T21:59:30.667 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_fail": 0, 2024-04-26T21:59:30.667 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_success": 0, 2024-04-26T21:59:30.667 INFO:teuthology.orchestra.run.smithi164.stdout: "take": 0, 2024-04-26T21:59:30.667 INFO:teuthology.orchestra.run.smithi164.stdout: "take_sum": 0, 2024-04-26T21:59:30.667 INFO:teuthology.orchestra.run.smithi164.stdout: "put": 0, 2024-04-26T21:59:30.667 INFO:teuthology.orchestra.run.smithi164.stdout: "put_sum": 0, 2024-04-26T21:59:30.667 INFO:teuthology.orchestra.run.smithi164.stdout: "wait": { 2024-04-26T21:59:30.668 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-26T21:59:30.668 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-26T21:59:30.668 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-26T21:59:30.668 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-04-26T21:59:30.668 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-04-26T21:59:30.668 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-04-26T21:59:31.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:59:30 smithi076 ceph-mon[31823]: pgmap v571: 97 pgs: 97 active+clean; 1.1 MiB data, 656 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-26T21:59:31.321 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:59:30 smithi072 ceph-mon[28138]: pgmap v571: 97 pgs: 97 active+clean; 1.1 MiB data, 656 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-26T21:59:31.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:59:30 smithi164 ceph-mon[31916]: pgmap v571: 97 pgs: 97 active+clean; 1.1 MiB data, 656 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-26T21:59:31.645 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-04-26T21:59:31.645 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-04-26T21:59:31.645 DEBUG:tasks.check_counter:Getting stats from f 2024-04-26T21:59:31.646 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid a1780950-0415-11ef-bc93-c7b262605968 -- ceph daemon mds.f perf dump 2024-04-26T21:59:32.075 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/a1780950-0415-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T21:59:33.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:59:33 smithi076 ceph-mon[31823]: pgmap v572: 97 pgs: 97 active+clean; 1.1 MiB data, 656 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:59:33.321 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:59:33 smithi072 ceph-mon[28138]: pgmap v572: 97 pgs: 97 active+clean; 1.1 MiB data, 656 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:59:33.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:59:33 smithi164 ceph-mon[31916]: pgmap v572: 97 pgs: 97 active+clean; 1.1 MiB data, 656 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:59:34.344 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-04-26T21:59:34.345 INFO:teuthology.orchestra.run.smithi164.stdout: "AsyncMessenger::Worker-0": { 2024-04-26T21:59:34.345 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_messages": 1048, 2024-04-26T21:59:34.345 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_messages": 1049, 2024-04-26T21:59:34.345 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_bytes": 301538, 2024-04-26T21:59:34.345 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_bytes": 332953, 2024-04-26T21:59:34.345 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_created_connections": 6, 2024-04-26T21:59:34.345 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_active_connections": 2, 2024-04-26T21:59:34.345 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_total_time": 0.209199694, 2024-04-26T21:59:34.345 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_send_time": 0.075256897, 2024-04-26T21:59:34.345 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_recv_time": 0.053023226, 2024-04-26T21:59:34.345 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_fast_dispatch_time": 0.060768360, 2024-04-26T21:59:34.345 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T21:59:34.346 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 1047, 2024-04-26T21:59:34.346 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.090657946, 2024-04-26T21:59:34.346 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000086588 2024-04-26T21:59:34.346 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:34.346 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_handle_ack_lat": { 2024-04-26T21:59:34.346 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-26T21:59:34.346 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-26T21:59:34.346 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-26T21:59:34.346 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:34.346 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-26T21:59:34.346 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_encrypted_bytes": 0 2024-04-26T21:59:34.346 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:34.346 INFO:teuthology.orchestra.run.smithi164.stdout: "AsyncMessenger::Worker-1": { 2024-04-26T21:59:34.347 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_messages": 7, 2024-04-26T21:59:34.347 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_messages": 94, 2024-04-26T21:59:34.347 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_bytes": 3628, 2024-04-26T21:59:34.347 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_bytes": 199738, 2024-04-26T21:59:34.347 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_created_connections": 4, 2024-04-26T21:59:34.347 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_active_connections": 3, 2024-04-26T21:59:34.347 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_total_time": 0.014722460, 2024-04-26T21:59:34.347 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_send_time": 0.009018034, 2024-04-26T21:59:34.347 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_recv_time": 0.003500177, 2024-04-26T21:59:34.347 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_fast_dispatch_time": 0.000082480, 2024-04-26T21:59:34.347 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T21:59:34.347 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 93, 2024-04-26T21:59:34.347 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.013974894, 2024-04-26T21:59:34.347 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000150267 2024-04-26T21:59:34.348 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:34.348 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_handle_ack_lat": { 2024-04-26T21:59:34.348 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-26T21:59:34.348 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-26T21:59:34.348 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-26T21:59:34.348 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:34.348 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_encrypted_bytes": 3072, 2024-04-26T21:59:34.348 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_encrypted_bytes": 199040 2024-04-26T21:59:34.348 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:34.348 INFO:teuthology.orchestra.run.smithi164.stdout: "AsyncMessenger::Worker-2": { 2024-04-26T21:59:34.348 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_messages": 132, 2024-04-26T21:59:34.348 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_messages": 129, 2024-04-26T21:59:34.348 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_bytes": 338460, 2024-04-26T21:59:34.348 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_bytes": 74222, 2024-04-26T21:59:34.349 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_created_connections": 3, 2024-04-26T21:59:34.349 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_active_connections": 2, 2024-04-26T21:59:34.349 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_total_time": 0.050327068, 2024-04-26T21:59:34.349 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_send_time": 0.017917300, 2024-04-26T21:59:34.349 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_recv_time": 0.032461296, 2024-04-26T21:59:34.349 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_fast_dispatch_time": 0.005773184, 2024-04-26T21:59:34.349 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T21:59:34.349 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 128, 2024-04-26T21:59:34.349 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.015246949, 2024-04-26T21:59:34.350 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000119116 2024-04-26T21:59:34.350 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:34.350 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_handle_ack_lat": { 2024-04-26T21:59:34.350 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-26T21:59:34.350 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-26T21:59:34.350 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-26T21:59:34.350 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:34.350 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_encrypted_bytes": 338192, 2024-04-26T21:59:34.350 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_encrypted_bytes": 73904 2024-04-26T21:59:34.350 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:34.350 INFO:teuthology.orchestra.run.smithi164.stdout: "cct": { 2024-04-26T21:59:34.350 INFO:teuthology.orchestra.run.smithi164.stdout: "total_workers": 1, 2024-04-26T21:59:34.350 INFO:teuthology.orchestra.run.smithi164.stdout: "unhealthy_workers": 0 2024-04-26T21:59:34.351 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:34.351 INFO:teuthology.orchestra.run.smithi164.stdout: "finisher-MDSRank": { 2024-04-26T21:59:34.351 INFO:teuthology.orchestra.run.smithi164.stdout: "queue_len": 0, 2024-04-26T21:59:34.351 INFO:teuthology.orchestra.run.smithi164.stdout: "complete_latency": { 2024-04-26T21:59:34.351 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 1840, 2024-04-26T21:59:34.351 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.093270314, 2024-04-26T21:59:34.351 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000050690 2024-04-26T21:59:34.351 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-04-26T21:59:34.351 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:34.351 INFO:teuthology.orchestra.run.smithi164.stdout: "finisher-PurgeQueue": { 2024-04-26T21:59:34.351 INFO:teuthology.orchestra.run.smithi164.stdout: "queue_len": 0, 2024-04-26T21:59:34.351 INFO:teuthology.orchestra.run.smithi164.stdout: "complete_latency": { 2024-04-26T21:59:34.351 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-26T21:59:34.351 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-26T21:59:34.351 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-26T21:59:34.352 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-04-26T21:59:34.352 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:34.352 INFO:teuthology.orchestra.run.smithi164.stdout: "mds": { 2024-04-26T21:59:34.352 INFO:teuthology.orchestra.run.smithi164.stdout: "request": 0, 2024-04-26T21:59:34.352 INFO:teuthology.orchestra.run.smithi164.stdout: "reply": 0, 2024-04-26T21:59:34.352 INFO:teuthology.orchestra.run.smithi164.stdout: "reply_latency": { 2024-04-26T21:59:34.352 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-26T21:59:34.352 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-26T21:59:34.352 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-26T21:59:34.352 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:34.352 INFO:teuthology.orchestra.run.smithi164.stdout: "slow_reply": 0, 2024-04-26T21:59:34.352 INFO:teuthology.orchestra.run.smithi164.stdout: "forward": 0, 2024-04-26T21:59:34.352 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_fetch_complete": 0, 2024-04-26T21:59:34.352 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_fetch_keys": 0, 2024-04-26T21:59:34.353 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_commit": 0, 2024-04-26T21:59:34.353 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_split": 0, 2024-04-26T21:59:34.353 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_merge": 0, 2024-04-26T21:59:34.353 INFO:teuthology.orchestra.run.smithi164.stdout: "inodes": 0, 2024-04-26T21:59:34.353 INFO:teuthology.orchestra.run.smithi164.stdout: "inodes_top": 0, 2024-04-26T21:59:34.353 INFO:teuthology.orchestra.run.smithi164.stdout: "inodes_bottom": 0, 2024-04-26T21:59:34.353 INFO:teuthology.orchestra.run.smithi164.stdout: "inodes_pin_tail": 0, 2024-04-26T21:59:34.353 INFO:teuthology.orchestra.run.smithi164.stdout: "inodes_pinned": 0, 2024-04-26T21:59:34.353 INFO:teuthology.orchestra.run.smithi164.stdout: "inodes_expired": 0, 2024-04-26T21:59:34.353 INFO:teuthology.orchestra.run.smithi164.stdout: "inodes_with_caps": 0, 2024-04-26T21:59:34.354 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": 0, 2024-04-26T21:59:34.354 INFO:teuthology.orchestra.run.smithi164.stdout: "subtrees": 1, 2024-04-26T21:59:34.354 INFO:teuthology.orchestra.run.smithi164.stdout: "traverse": 0, 2024-04-26T21:59:34.354 INFO:teuthology.orchestra.run.smithi164.stdout: "traverse_hit": 0, 2024-04-26T21:59:34.354 INFO:teuthology.orchestra.run.smithi164.stdout: "traverse_forward": 0, 2024-04-26T21:59:34.354 INFO:teuthology.orchestra.run.smithi164.stdout: "traverse_discover": 0, 2024-04-26T21:59:34.354 INFO:teuthology.orchestra.run.smithi164.stdout: "traverse_dir_fetch": 0, 2024-04-26T21:59:34.354 INFO:teuthology.orchestra.run.smithi164.stdout: "traverse_remote_ino": 0, 2024-04-26T21:59:34.354 INFO:teuthology.orchestra.run.smithi164.stdout: "traverse_lock": 0, 2024-04-26T21:59:34.354 INFO:teuthology.orchestra.run.smithi164.stdout: "load_cent": 0, 2024-04-26T21:59:34.354 INFO:teuthology.orchestra.run.smithi164.stdout: "q": 0, 2024-04-26T21:59:34.354 INFO:teuthology.orchestra.run.smithi164.stdout: "exported": 0, 2024-04-26T21:59:34.355 INFO:teuthology.orchestra.run.smithi164.stdout: "exported_inodes": 0, 2024-04-26T21:59:34.355 INFO:teuthology.orchestra.run.smithi164.stdout: "imported": 0, 2024-04-26T21:59:34.355 INFO:teuthology.orchestra.run.smithi164.stdout: "imported_inodes": 0, 2024-04-26T21:59:34.355 INFO:teuthology.orchestra.run.smithi164.stdout: "openino_dir_fetch": 0, 2024-04-26T21:59:34.355 INFO:teuthology.orchestra.run.smithi164.stdout: "openino_backtrace_fetch": 0, 2024-04-26T21:59:34.355 INFO:teuthology.orchestra.run.smithi164.stdout: "openino_peer_discover": 0, 2024-04-26T21:59:34.355 INFO:teuthology.orchestra.run.smithi164.stdout: "root_rfiles": 0, 2024-04-26T21:59:34.355 INFO:teuthology.orchestra.run.smithi164.stdout: "root_rbytes": 0, 2024-04-26T21:59:34.355 INFO:teuthology.orchestra.run.smithi164.stdout: "root_rsnaps": 0, 2024-04-26T21:59:34.355 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_backtrace_fetch": 0, 2024-04-26T21:59:34.355 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_set_tag": 0, 2024-04-26T21:59:34.355 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_backtrace_repaired": 0, 2024-04-26T21:59:34.355 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_inotable_repaired": 0, 2024-04-26T21:59:34.356 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_dir_inodes": 0, 2024-04-26T21:59:34.356 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_dir_base_inodes": 0, 2024-04-26T21:59:34.356 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_dirfrag_rstats": 0, 2024-04-26T21:59:34.356 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_file_inodes": 0, 2024-04-26T21:59:34.356 INFO:teuthology.orchestra.run.smithi164.stdout: "handle_inode_file_caps": 0, 2024-04-26T21:59:34.356 INFO:teuthology.orchestra.run.smithi164.stdout: "ceph_cap_op_revoke": 0, 2024-04-26T21:59:34.356 INFO:teuthology.orchestra.run.smithi164.stdout: "ceph_cap_op_grant": 0, 2024-04-26T21:59:34.356 INFO:teuthology.orchestra.run.smithi164.stdout: "ceph_cap_op_trunc": 0, 2024-04-26T21:59:34.356 INFO:teuthology.orchestra.run.smithi164.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-26T21:59:34.356 INFO:teuthology.orchestra.run.smithi164.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-26T21:59:34.356 INFO:teuthology.orchestra.run.smithi164.stdout: "handle_client_caps": 0, 2024-04-26T21:59:34.356 INFO:teuthology.orchestra.run.smithi164.stdout: "handle_client_caps_dirty": 0, 2024-04-26T21:59:34.356 INFO:teuthology.orchestra.run.smithi164.stdout: "handle_client_cap_release": 0, 2024-04-26T21:59:34.356 INFO:teuthology.orchestra.run.smithi164.stdout: "process_request_cap_release": 0 2024-04-26T21:59:34.356 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:34.357 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_cache": { 2024-04-26T21:59:34.357 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_update": 0, 2024-04-26T21:59:34.357 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_update_receipt": 0, 2024-04-26T21:59:34.357 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_try_discover": 0, 2024-04-26T21:59:34.357 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_send_discover": 0, 2024-04-26T21:59:34.357 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_handle_discover": 0, 2024-04-26T21:59:34.357 INFO:teuthology.orchestra.run.smithi164.stdout: "num_strays": 0, 2024-04-26T21:59:34.357 INFO:teuthology.orchestra.run.smithi164.stdout: "num_strays_delayed": 0, 2024-04-26T21:59:34.357 INFO:teuthology.orchestra.run.smithi164.stdout: "num_strays_enqueuing": 0, 2024-04-26T21:59:34.357 INFO:teuthology.orchestra.run.smithi164.stdout: "strays_created": 0, 2024-04-26T21:59:34.357 INFO:teuthology.orchestra.run.smithi164.stdout: "strays_enqueued": 0, 2024-04-26T21:59:34.357 INFO:teuthology.orchestra.run.smithi164.stdout: "strays_reintegrated": 0, 2024-04-26T21:59:34.357 INFO:teuthology.orchestra.run.smithi164.stdout: "strays_migrated": 0, 2024-04-26T21:59:34.357 INFO:teuthology.orchestra.run.smithi164.stdout: "num_recovering_processing": 0, 2024-04-26T21:59:34.357 INFO:teuthology.orchestra.run.smithi164.stdout: "num_recovering_enqueued": 0, 2024-04-26T21:59:34.357 INFO:teuthology.orchestra.run.smithi164.stdout: "num_recovering_prioritized": 0, 2024-04-26T21:59:34.358 INFO:teuthology.orchestra.run.smithi164.stdout: "recovery_started": 0, 2024-04-26T21:59:34.358 INFO:teuthology.orchestra.run.smithi164.stdout: "recovery_completed": 0, 2024-04-26T21:59:34.358 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_quiesce_path": 0, 2024-04-26T21:59:34.358 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_quiesce_inode": 0, 2024-04-26T21:59:34.358 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_enqueue_scrub": 0, 2024-04-26T21:59:34.358 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_exportdir": 0, 2024-04-26T21:59:34.358 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_flush": 0, 2024-04-26T21:59:34.358 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_fragmentdir": 0, 2024-04-26T21:59:34.358 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_fragstats": 0, 2024-04-26T21:59:34.358 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_inodestats": 0 2024-04-26T21:59:34.358 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:34.358 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_log": { 2024-04-26T21:59:34.358 INFO:teuthology.orchestra.run.smithi164.stdout: "evlrg": 0, 2024-04-26T21:59:34.359 INFO:teuthology.orchestra.run.smithi164.stdout: "evadd": 0, 2024-04-26T21:59:34.359 INFO:teuthology.orchestra.run.smithi164.stdout: "evex": 0, 2024-04-26T21:59:34.359 INFO:teuthology.orchestra.run.smithi164.stdout: "evtrm": 0, 2024-04-26T21:59:34.359 INFO:teuthology.orchestra.run.smithi164.stdout: "ev": 3, 2024-04-26T21:59:34.359 INFO:teuthology.orchestra.run.smithi164.stdout: "evexg": 0, 2024-04-26T21:59:34.359 INFO:teuthology.orchestra.run.smithi164.stdout: "evexd": 0, 2024-04-26T21:59:34.359 INFO:teuthology.orchestra.run.smithi164.stdout: "segadd": 0, 2024-04-26T21:59:34.359 INFO:teuthology.orchestra.run.smithi164.stdout: "segex": 0, 2024-04-26T21:59:34.359 INFO:teuthology.orchestra.run.smithi164.stdout: "segtrm": 0, 2024-04-26T21:59:34.359 INFO:teuthology.orchestra.run.smithi164.stdout: "seg": 2, 2024-04-26T21:59:34.359 INFO:teuthology.orchestra.run.smithi164.stdout: "segmjr": 2, 2024-04-26T21:59:34.359 INFO:teuthology.orchestra.run.smithi164.stdout: "segexg": 0, 2024-04-26T21:59:34.359 INFO:teuthology.orchestra.run.smithi164.stdout: "segexd": 0, 2024-04-26T21:59:34.359 INFO:teuthology.orchestra.run.smithi164.stdout: "expos": 4194304, 2024-04-26T21:59:34.359 INFO:teuthology.orchestra.run.smithi164.stdout: "wrpos": 4195385, 2024-04-26T21:59:34.360 INFO:teuthology.orchestra.run.smithi164.stdout: "rdpos": 4194892, 2024-04-26T21:59:34.360 INFO:teuthology.orchestra.run.smithi164.stdout: "jlat": { 2024-04-26T21:59:34.360 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-26T21:59:34.360 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-26T21:59:34.360 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-26T21:59:34.360 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:34.360 INFO:teuthology.orchestra.run.smithi164.stdout: "replayed": 3 2024-04-26T21:59:34.360 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:34.360 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_mem": { 2024-04-26T21:59:34.360 INFO:teuthology.orchestra.run.smithi164.stdout: "ino": 3, 2024-04-26T21:59:34.360 INFO:teuthology.orchestra.run.smithi164.stdout: "ino+": 3, 2024-04-26T21:59:34.360 INFO:teuthology.orchestra.run.smithi164.stdout: "ino-": 0, 2024-04-26T21:59:34.361 INFO:teuthology.orchestra.run.smithi164.stdout: "dir": 1, 2024-04-26T21:59:34.361 INFO:teuthology.orchestra.run.smithi164.stdout: "dir+": 1, 2024-04-26T21:59:34.361 INFO:teuthology.orchestra.run.smithi164.stdout: "dir-": 0, 2024-04-26T21:59:34.361 INFO:teuthology.orchestra.run.smithi164.stdout: "dn": 0, 2024-04-26T21:59:34.361 INFO:teuthology.orchestra.run.smithi164.stdout: "dn+": 0, 2024-04-26T21:59:34.361 INFO:teuthology.orchestra.run.smithi164.stdout: "dn-": 0, 2024-04-26T21:59:34.361 INFO:teuthology.orchestra.run.smithi164.stdout: "cap": 0, 2024-04-26T21:59:34.361 INFO:teuthology.orchestra.run.smithi164.stdout: "cap+": 0, 2024-04-26T21:59:34.361 INFO:teuthology.orchestra.run.smithi164.stdout: "cap-": 0, 2024-04-26T21:59:34.361 INFO:teuthology.orchestra.run.smithi164.stdout: "rss": 48128, 2024-04-26T21:59:34.361 INFO:teuthology.orchestra.run.smithi164.stdout: "heap": 215416 2024-04-26T21:59:34.361 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:34.361 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_server": { 2024-04-26T21:59:34.361 INFO:teuthology.orchestra.run.smithi164.stdout: "dispatch_client_request": 0, 2024-04-26T21:59:34.362 INFO:teuthology.orchestra.run.smithi164.stdout: "dispatch_server_request": 0, 2024-04-26T21:59:34.362 INFO:teuthology.orchestra.run.smithi164.stdout: "handle_client_request": 0, 2024-04-26T21:59:34.362 INFO:teuthology.orchestra.run.smithi164.stdout: "handle_client_session": 0, 2024-04-26T21:59:34.362 INFO:teuthology.orchestra.run.smithi164.stdout: "handle_peer_request": 0, 2024-04-26T21:59:34.362 INFO:teuthology.orchestra.run.smithi164.stdout: "req_create_latency": { 2024-04-26T21:59:34.362 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-26T21:59:34.362 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-26T21:59:34.362 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-26T21:59:34.362 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:34.362 INFO:teuthology.orchestra.run.smithi164.stdout: "req_getattr_latency": { 2024-04-26T21:59:34.362 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-26T21:59:34.362 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-26T21:59:34.363 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-26T21:59:34.363 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:34.363 INFO:teuthology.orchestra.run.smithi164.stdout: "req_getfilelock_latency": { 2024-04-26T21:59:34.363 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-26T21:59:34.363 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-26T21:59:34.363 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-26T21:59:34.363 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:34.363 INFO:teuthology.orchestra.run.smithi164.stdout: "req_link_latency": { 2024-04-26T21:59:34.363 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-26T21:59:34.363 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-26T21:59:34.363 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-26T21:59:34.363 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:34.363 INFO:teuthology.orchestra.run.smithi164.stdout: "req_lookup_latency": { 2024-04-26T21:59:34.363 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-26T21:59:34.364 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-26T21:59:34.364 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-26T21:59:34.364 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:34.364 INFO:teuthology.orchestra.run.smithi164.stdout: "req_lookuphash_latency": { 2024-04-26T21:59:34.364 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-26T21:59:34.364 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-26T21:59:34.364 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-26T21:59:34.364 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:34.364 INFO:teuthology.orchestra.run.smithi164.stdout: "req_lookupino_latency": { 2024-04-26T21:59:34.364 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-26T21:59:34.364 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-26T21:59:34.364 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-26T21:59:34.364 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:34.365 INFO:teuthology.orchestra.run.smithi164.stdout: "req_lookupname_latency": { 2024-04-26T21:59:34.365 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-26T21:59:34.365 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-26T21:59:34.365 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-26T21:59:34.365 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:34.365 INFO:teuthology.orchestra.run.smithi164.stdout: "req_lookupparent_latency": { 2024-04-26T21:59:34.365 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-26T21:59:34.365 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-26T21:59:34.365 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-26T21:59:34.365 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:34.365 INFO:teuthology.orchestra.run.smithi164.stdout: "req_lookupsnap_latency": { 2024-04-26T21:59:34.365 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-26T21:59:34.365 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-26T21:59:34.366 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-26T21:59:34.366 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:34.366 INFO:teuthology.orchestra.run.smithi164.stdout: "req_lssnap_latency": { 2024-04-26T21:59:34.366 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-26T21:59:34.366 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-26T21:59:34.366 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-26T21:59:34.366 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:34.366 INFO:teuthology.orchestra.run.smithi164.stdout: "req_mkdir_latency": { 2024-04-26T21:59:34.366 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-26T21:59:34.367 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-26T21:59:34.367 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-26T21:59:34.367 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:34.367 INFO:teuthology.orchestra.run.smithi164.stdout: "req_mknod_latency": { 2024-04-26T21:59:34.367 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-26T21:59:34.367 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-26T21:59:34.367 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-26T21:59:34.367 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:34.367 INFO:teuthology.orchestra.run.smithi164.stdout: "req_mksnap_latency": { 2024-04-26T21:59:34.367 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-26T21:59:34.368 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-26T21:59:34.368 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-26T21:59:34.368 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:34.368 INFO:teuthology.orchestra.run.smithi164.stdout: "req_open_latency": { 2024-04-26T21:59:34.368 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-26T21:59:34.368 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-26T21:59:34.368 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-26T21:59:34.368 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:34.368 INFO:teuthology.orchestra.run.smithi164.stdout: "req_readdir_latency": { 2024-04-26T21:59:34.368 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-26T21:59:34.368 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-26T21:59:34.368 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-26T21:59:34.368 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:34.368 INFO:teuthology.orchestra.run.smithi164.stdout: "req_rename_latency": { 2024-04-26T21:59:34.369 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-26T21:59:34.369 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-26T21:59:34.369 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-26T21:59:34.369 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:34.369 INFO:teuthology.orchestra.run.smithi164.stdout: "req_renamesnap_latency": { 2024-04-26T21:59:34.369 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-26T21:59:34.369 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-26T21:59:34.369 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-26T21:59:34.369 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:34.369 INFO:teuthology.orchestra.run.smithi164.stdout: "req_snapdiff_latency": { 2024-04-26T21:59:34.369 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-26T21:59:34.369 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-26T21:59:34.369 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-26T21:59:34.369 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:34.369 INFO:teuthology.orchestra.run.smithi164.stdout: "req_rmdir_latency": { 2024-04-26T21:59:34.370 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-26T21:59:34.370 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-26T21:59:34.370 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-26T21:59:34.370 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:34.370 INFO:teuthology.orchestra.run.smithi164.stdout: "req_rmsnap_latency": { 2024-04-26T21:59:34.370 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-26T21:59:34.370 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-26T21:59:34.370 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-26T21:59:34.370 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:34.370 INFO:teuthology.orchestra.run.smithi164.stdout: "req_rmxattr_latency": { 2024-04-26T21:59:34.370 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-26T21:59:34.370 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-26T21:59:34.370 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-26T21:59:34.371 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:34.371 INFO:teuthology.orchestra.run.smithi164.stdout: "req_setattr_latency": { 2024-04-26T21:59:34.371 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-26T21:59:34.371 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-26T21:59:34.371 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-26T21:59:34.371 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:34.371 INFO:teuthology.orchestra.run.smithi164.stdout: "req_setdirlayout_latency": { 2024-04-26T21:59:34.371 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-26T21:59:34.371 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-26T21:59:34.371 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-26T21:59:34.371 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:34.371 INFO:teuthology.orchestra.run.smithi164.stdout: "req_setfilelock_latency": { 2024-04-26T21:59:34.371 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-26T21:59:34.371 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-26T21:59:34.372 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-26T21:59:34.372 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:34.372 INFO:teuthology.orchestra.run.smithi164.stdout: "req_setlayout_latency": { 2024-04-26T21:59:34.372 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-26T21:59:34.372 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-26T21:59:34.372 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-26T21:59:34.372 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:34.372 INFO:teuthology.orchestra.run.smithi164.stdout: "req_setxattr_latency": { 2024-04-26T21:59:34.372 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-26T21:59:34.372 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-26T21:59:34.372 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-26T21:59:34.372 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:34.372 INFO:teuthology.orchestra.run.smithi164.stdout: "req_symlink_latency": { 2024-04-26T21:59:34.372 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-26T21:59:34.373 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-26T21:59:34.373 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-26T21:59:34.373 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:34.373 INFO:teuthology.orchestra.run.smithi164.stdout: "req_unlink_latency": { 2024-04-26T21:59:34.373 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-26T21:59:34.373 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-26T21:59:34.373 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-26T21:59:34.373 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:34.373 INFO:teuthology.orchestra.run.smithi164.stdout: "cap_revoke_eviction": 0, 2024-04-26T21:59:34.373 INFO:teuthology.orchestra.run.smithi164.stdout: "cap_acquisition_throttle": 0, 2024-04-26T21:59:34.373 INFO:teuthology.orchestra.run.smithi164.stdout: "req_getvxattr_latency": { 2024-04-26T21:59:34.373 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-26T21:59:34.373 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-26T21:59:34.373 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-26T21:59:34.374 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-04-26T21:59:34.374 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:34.374 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_sessions": { 2024-04-26T21:59:34.374 INFO:teuthology.orchestra.run.smithi164.stdout: "session_count": 1, 2024-04-26T21:59:34.374 INFO:teuthology.orchestra.run.smithi164.stdout: "session_add": 1, 2024-04-26T21:59:34.374 INFO:teuthology.orchestra.run.smithi164.stdout: "session_remove": 0, 2024-04-26T21:59:34.374 INFO:teuthology.orchestra.run.smithi164.stdout: "sessions_open": 1, 2024-04-26T21:59:34.374 INFO:teuthology.orchestra.run.smithi164.stdout: "sessions_stale": 0, 2024-04-26T21:59:34.374 INFO:teuthology.orchestra.run.smithi164.stdout: "total_load": 0, 2024-04-26T21:59:34.374 INFO:teuthology.orchestra.run.smithi164.stdout: "average_load": 0, 2024-04-26T21:59:34.374 INFO:teuthology.orchestra.run.smithi164.stdout: "avg_session_uptime": 1685, 2024-04-26T21:59:34.374 INFO:teuthology.orchestra.run.smithi164.stdout: "mdthresh_evicted": 0 2024-04-26T21:59:34.374 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:34.375 INFO:teuthology.orchestra.run.smithi164.stdout: "mempool": { 2024-04-26T21:59:34.375 INFO:teuthology.orchestra.run.smithi164.stdout: "bloom_filter_bytes": 0, 2024-04-26T21:59:34.375 INFO:teuthology.orchestra.run.smithi164.stdout: "bloom_filter_items": 0, 2024-04-26T21:59:34.375 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_alloc_bytes": 0, 2024-04-26T21:59:34.375 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_alloc_items": 0, 2024-04-26T21:59:34.375 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_data_bytes": 0, 2024-04-26T21:59:34.375 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_data_items": 0, 2024-04-26T21:59:34.375 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-26T21:59:34.375 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_onode_items": 0, 2024-04-26T21:59:34.375 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-26T21:59:34.375 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_meta_items": 0, 2024-04-26T21:59:34.375 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_other_bytes": 0, 2024-04-26T21:59:34.375 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_other_items": 0, 2024-04-26T21:59:34.375 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-26T21:59:34.375 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_buffer_items": 0, 2024-04-26T21:59:34.376 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_extent_bytes": 0, 2024-04-26T21:59:34.376 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_extent_items": 0, 2024-04-26T21:59:34.376 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_blob_bytes": 0, 2024-04-26T21:59:34.376 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_blob_items": 0, 2024-04-26T21:59:34.376 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-26T21:59:34.376 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_shared_blob_items": 0, 2024-04-26T21:59:34.376 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-26T21:59:34.376 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_inline_bl_items": 0, 2024-04-26T21:59:34.376 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_fsck_bytes": 0, 2024-04-26T21:59:34.376 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_fsck_items": 0, 2024-04-26T21:59:34.376 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_txc_bytes": 0, 2024-04-26T21:59:34.376 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_txc_items": 0, 2024-04-26T21:59:34.377 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-26T21:59:34.377 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_writing_deferred_items": 0, 2024-04-26T21:59:34.377 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_writing_bytes": 0, 2024-04-26T21:59:34.377 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_writing_items": 0, 2024-04-26T21:59:34.377 INFO:teuthology.orchestra.run.smithi164.stdout: "bluefs_bytes": 0, 2024-04-26T21:59:34.377 INFO:teuthology.orchestra.run.smithi164.stdout: "bluefs_items": 0, 2024-04-26T21:59:34.377 INFO:teuthology.orchestra.run.smithi164.stdout: "bluefs_file_reader_bytes": 0, 2024-04-26T21:59:34.377 INFO:teuthology.orchestra.run.smithi164.stdout: "bluefs_file_reader_items": 0, 2024-04-26T21:59:34.377 INFO:teuthology.orchestra.run.smithi164.stdout: "bluefs_file_writer_bytes": 0, 2024-04-26T21:59:34.377 INFO:teuthology.orchestra.run.smithi164.stdout: "bluefs_file_writer_items": 0, 2024-04-26T21:59:34.377 INFO:teuthology.orchestra.run.smithi164.stdout: "buffer_anon_bytes": 181199, 2024-04-26T21:59:34.377 INFO:teuthology.orchestra.run.smithi164.stdout: "buffer_anon_items": 45, 2024-04-26T21:59:34.377 INFO:teuthology.orchestra.run.smithi164.stdout: "buffer_meta_bytes": 0, 2024-04-26T21:59:34.378 INFO:teuthology.orchestra.run.smithi164.stdout: "buffer_meta_items": 0, 2024-04-26T21:59:34.378 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_bytes": 0, 2024-04-26T21:59:34.378 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_items": 0, 2024-04-26T21:59:34.378 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_mapbl_bytes": 0, 2024-04-26T21:59:34.378 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_mapbl_items": 0, 2024-04-26T21:59:34.378 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_pglog_bytes": 0, 2024-04-26T21:59:34.378 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_pglog_items": 0, 2024-04-26T21:59:34.378 INFO:teuthology.orchestra.run.smithi164.stdout: "osdmap_bytes": 8008, 2024-04-26T21:59:34.378 INFO:teuthology.orchestra.run.smithi164.stdout: "osdmap_items": 168, 2024-04-26T21:59:34.378 INFO:teuthology.orchestra.run.smithi164.stdout: "osdmap_mapping_bytes": 0, 2024-04-26T21:59:34.378 INFO:teuthology.orchestra.run.smithi164.stdout: "osdmap_mapping_items": 0, 2024-04-26T21:59:34.378 INFO:teuthology.orchestra.run.smithi164.stdout: "pgmap_bytes": 0, 2024-04-26T21:59:34.378 INFO:teuthology.orchestra.run.smithi164.stdout: "pgmap_items": 0, 2024-04-26T21:59:34.378 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_co_bytes": 8512, 2024-04-26T21:59:34.378 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_co_items": 13, 2024-04-26T21:59:34.379 INFO:teuthology.orchestra.run.smithi164.stdout: "unittest_1_bytes": 0, 2024-04-26T21:59:34.379 INFO:teuthology.orchestra.run.smithi164.stdout: "unittest_1_items": 0, 2024-04-26T21:59:34.379 INFO:teuthology.orchestra.run.smithi164.stdout: "unittest_2_bytes": 0, 2024-04-26T21:59:34.379 INFO:teuthology.orchestra.run.smithi164.stdout: "unittest_2_items": 0 2024-04-26T21:59:34.379 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:34.379 INFO:teuthology.orchestra.run.smithi164.stdout: "objecter": { 2024-04-26T21:59:34.379 INFO:teuthology.orchestra.run.smithi164.stdout: "op_active": 0, 2024-04-26T21:59:34.379 INFO:teuthology.orchestra.run.smithi164.stdout: "op_laggy": 0, 2024-04-26T21:59:34.379 INFO:teuthology.orchestra.run.smithi164.stdout: "op_send": 1050, 2024-04-26T21:59:34.379 INFO:teuthology.orchestra.run.smithi164.stdout: "op_send_bytes": 21, 2024-04-26T21:59:34.379 INFO:teuthology.orchestra.run.smithi164.stdout: "op_resend": 3, 2024-04-26T21:59:34.379 INFO:teuthology.orchestra.run.smithi164.stdout: "op_reply": 1050, 2024-04-26T21:59:34.379 INFO:teuthology.orchestra.run.smithi164.stdout: "op_latency": { 2024-04-26T21:59:34.379 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 1050, 2024-04-26T21:59:34.379 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 1.407975846, 2024-04-26T21:59:34.380 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.001340929 2024-04-26T21:59:34.380 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:34.380 INFO:teuthology.orchestra.run.smithi164.stdout: "op_inflight": 0, 2024-04-26T21:59:34.380 INFO:teuthology.orchestra.run.smithi164.stdout: "oplen_avg": { 2024-04-26T21:59:34.380 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 1050, 2024-04-26T21:59:34.380 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 1051 2024-04-26T21:59:34.380 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:34.380 INFO:teuthology.orchestra.run.smithi164.stdout: "op": 1050, 2024-04-26T21:59:34.380 INFO:teuthology.orchestra.run.smithi164.stdout: "op_r": 1050, 2024-04-26T21:59:34.380 INFO:teuthology.orchestra.run.smithi164.stdout: "op_w": 0, 2024-04-26T21:59:34.380 INFO:teuthology.orchestra.run.smithi164.stdout: "op_rmw": 0, 2024-04-26T21:59:34.380 INFO:teuthology.orchestra.run.smithi164.stdout: "op_pg": 0, 2024-04-26T21:59:34.380 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_stat": 648, 2024-04-26T21:59:34.380 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_create": 0, 2024-04-26T21:59:34.381 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_read": 400, 2024-04-26T21:59:34.381 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_write": 0, 2024-04-26T21:59:34.381 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_writefull": 0, 2024-04-26T21:59:34.381 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_writesame": 0, 2024-04-26T21:59:34.381 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_append": 0, 2024-04-26T21:59:34.381 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_zero": 0, 2024-04-26T21:59:34.381 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_truncate": 0, 2024-04-26T21:59:34.381 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_delete": 0, 2024-04-26T21:59:34.381 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_mapext": 0, 2024-04-26T21:59:34.381 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_sparse_read": 0, 2024-04-26T21:59:34.381 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_clonerange": 0, 2024-04-26T21:59:34.381 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_getxattr": 1, 2024-04-26T21:59:34.381 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_setxattr": 0, 2024-04-26T21:59:34.381 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_cmpxattr": 0, 2024-04-26T21:59:34.381 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_rmxattr": 0, 2024-04-26T21:59:34.381 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_resetxattrs": 0, 2024-04-26T21:59:34.382 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_call": 0, 2024-04-26T21:59:34.382 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_watch": 0, 2024-04-26T21:59:34.382 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_notify": 0, 2024-04-26T21:59:34.382 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_src_cmpxattr": 0, 2024-04-26T21:59:34.382 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_pgls": 0, 2024-04-26T21:59:34.382 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_pgls_filter": 0, 2024-04-26T21:59:34.382 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_other": 0, 2024-04-26T21:59:34.382 INFO:teuthology.orchestra.run.smithi164.stdout: "linger_active": 0, 2024-04-26T21:59:34.382 INFO:teuthology.orchestra.run.smithi164.stdout: "linger_send": 0, 2024-04-26T21:59:34.382 INFO:teuthology.orchestra.run.smithi164.stdout: "linger_resend": 0, 2024-04-26T21:59:34.382 INFO:teuthology.orchestra.run.smithi164.stdout: "linger_ping": 0, 2024-04-26T21:59:34.382 INFO:teuthology.orchestra.run.smithi164.stdout: "poolop_active": 0, 2024-04-26T21:59:34.382 INFO:teuthology.orchestra.run.smithi164.stdout: "poolop_send": 0, 2024-04-26T21:59:34.382 INFO:teuthology.orchestra.run.smithi164.stdout: "poolop_resend": 0, 2024-04-26T21:59:34.383 INFO:teuthology.orchestra.run.smithi164.stdout: "poolstat_active": 0, 2024-04-26T21:59:34.383 INFO:teuthology.orchestra.run.smithi164.stdout: "poolstat_send": 0, 2024-04-26T21:59:34.383 INFO:teuthology.orchestra.run.smithi164.stdout: "poolstat_resend": 0, 2024-04-26T21:59:34.383 INFO:teuthology.orchestra.run.smithi164.stdout: "statfs_active": 0, 2024-04-26T21:59:34.383 INFO:teuthology.orchestra.run.smithi164.stdout: "statfs_send": 0, 2024-04-26T21:59:34.383 INFO:teuthology.orchestra.run.smithi164.stdout: "statfs_resend": 0, 2024-04-26T21:59:34.383 INFO:teuthology.orchestra.run.smithi164.stdout: "command_active": 0, 2024-04-26T21:59:34.383 INFO:teuthology.orchestra.run.smithi164.stdout: "command_send": 0, 2024-04-26T21:59:34.383 INFO:teuthology.orchestra.run.smithi164.stdout: "command_resend": 0, 2024-04-26T21:59:34.383 INFO:teuthology.orchestra.run.smithi164.stdout: "map_epoch": 91, 2024-04-26T21:59:34.383 INFO:teuthology.orchestra.run.smithi164.stdout: "map_full": 0, 2024-04-26T21:59:34.383 INFO:teuthology.orchestra.run.smithi164.stdout: "map_inc": 4, 2024-04-26T21:59:34.383 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_sessions": 4, 2024-04-26T21:59:34.384 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_session_open": 4, 2024-04-26T21:59:34.384 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_session_close": 0, 2024-04-26T21:59:34.384 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_laggy": 0, 2024-04-26T21:59:34.384 INFO:teuthology.orchestra.run.smithi164.stdout: "omap_wr": 0, 2024-04-26T21:59:34.384 INFO:teuthology.orchestra.run.smithi164.stdout: "omap_rd": 2, 2024-04-26T21:59:34.384 INFO:teuthology.orchestra.run.smithi164.stdout: "omap_del": 0 2024-04-26T21:59:34.384 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:34.384 INFO:teuthology.orchestra.run.smithi164.stdout: "oft": { 2024-04-26T21:59:34.385 INFO:teuthology.orchestra.run.smithi164.stdout: "omap_total_objs": 0, 2024-04-26T21:59:34.385 INFO:teuthology.orchestra.run.smithi164.stdout: "omap_total_kv_pairs": 0, 2024-04-26T21:59:34.385 INFO:teuthology.orchestra.run.smithi164.stdout: "omap_total_updates": 0, 2024-04-26T21:59:34.385 INFO:teuthology.orchestra.run.smithi164.stdout: "omap_total_removes": 0 2024-04-26T21:59:34.385 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:34.385 INFO:teuthology.orchestra.run.smithi164.stdout: "osd-slow-ops": { 2024-04-26T21:59:34.385 INFO:teuthology.orchestra.run.smithi164.stdout: "slow_ops_count": 0 2024-04-26T21:59:34.385 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:34.385 INFO:teuthology.orchestra.run.smithi164.stdout: "purge_queue": { 2024-04-26T21:59:34.385 INFO:teuthology.orchestra.run.smithi164.stdout: "pq_executing_ops": 0, 2024-04-26T21:59:34.385 INFO:teuthology.orchestra.run.smithi164.stdout: "pq_executing_ops_high_water": 0, 2024-04-26T21:59:34.385 INFO:teuthology.orchestra.run.smithi164.stdout: "pq_executing": 0, 2024-04-26T21:59:34.385 INFO:teuthology.orchestra.run.smithi164.stdout: "pq_executing_high_water": 0, 2024-04-26T21:59:34.386 INFO:teuthology.orchestra.run.smithi164.stdout: "pq_executed_ops": 0, 2024-04-26T21:59:34.386 INFO:teuthology.orchestra.run.smithi164.stdout: "pq_executed": 0, 2024-04-26T21:59:34.386 INFO:teuthology.orchestra.run.smithi164.stdout: "pq_item_in_journal": 0 2024-04-26T21:59:34.386 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:34.386 INFO:teuthology.orchestra.run.smithi164.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-26T21:59:34.386 INFO:teuthology.orchestra.run.smithi164.stdout: "val": 0, 2024-04-26T21:59:34.386 INFO:teuthology.orchestra.run.smithi164.stdout: "max": 104857600, 2024-04-26T21:59:34.386 INFO:teuthology.orchestra.run.smithi164.stdout: "get_started": 0, 2024-04-26T21:59:34.386 INFO:teuthology.orchestra.run.smithi164.stdout: "get": 1183, 2024-04-26T21:59:34.386 INFO:teuthology.orchestra.run.smithi164.stdout: "get_sum": 527523, 2024-04-26T21:59:34.386 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_fail": 0, 2024-04-26T21:59:34.386 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_success": 1183, 2024-04-26T21:59:34.386 INFO:teuthology.orchestra.run.smithi164.stdout: "take": 0, 2024-04-26T21:59:34.387 INFO:teuthology.orchestra.run.smithi164.stdout: "take_sum": 0, 2024-04-26T21:59:34.387 INFO:teuthology.orchestra.run.smithi164.stdout: "put": 1183, 2024-04-26T21:59:34.387 INFO:teuthology.orchestra.run.smithi164.stdout: "put_sum": 527523, 2024-04-26T21:59:34.387 INFO:teuthology.orchestra.run.smithi164.stdout: "wait": { 2024-04-26T21:59:34.387 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-26T21:59:34.387 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-26T21:59:34.387 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-26T21:59:34.387 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-04-26T21:59:34.387 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:34.387 INFO:teuthology.orchestra.run.smithi164.stdout: "throttle-objecter_bytes": { 2024-04-26T21:59:34.387 INFO:teuthology.orchestra.run.smithi164.stdout: "val": 0, 2024-04-26T21:59:34.387 INFO:teuthology.orchestra.run.smithi164.stdout: "max": 104857600, 2024-04-26T21:59:34.387 INFO:teuthology.orchestra.run.smithi164.stdout: "get_started": 0, 2024-04-26T21:59:34.388 INFO:teuthology.orchestra.run.smithi164.stdout: "get": 0, 2024-04-26T21:59:34.388 INFO:teuthology.orchestra.run.smithi164.stdout: "get_sum": 0, 2024-04-26T21:59:34.388 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_fail": 0, 2024-04-26T21:59:34.388 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_success": 0, 2024-04-26T21:59:34.388 INFO:teuthology.orchestra.run.smithi164.stdout: "take": 1050, 2024-04-26T21:59:34.388 INFO:teuthology.orchestra.run.smithi164.stdout: "take_sum": 1086, 2024-04-26T21:59:34.388 INFO:teuthology.orchestra.run.smithi164.stdout: "put": 1050, 2024-04-26T21:59:34.388 INFO:teuthology.orchestra.run.smithi164.stdout: "put_sum": 1086, 2024-04-26T21:59:34.388 INFO:teuthology.orchestra.run.smithi164.stdout: "wait": { 2024-04-26T21:59:34.388 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-26T21:59:34.388 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-26T21:59:34.388 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-26T21:59:34.388 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-04-26T21:59:34.388 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:34.388 INFO:teuthology.orchestra.run.smithi164.stdout: "throttle-objecter_ops": { 2024-04-26T21:59:34.388 INFO:teuthology.orchestra.run.smithi164.stdout: "val": 0, 2024-04-26T21:59:34.389 INFO:teuthology.orchestra.run.smithi164.stdout: "max": 1024, 2024-04-26T21:59:34.389 INFO:teuthology.orchestra.run.smithi164.stdout: "get_started": 0, 2024-04-26T21:59:34.389 INFO:teuthology.orchestra.run.smithi164.stdout: "get": 0, 2024-04-26T21:59:34.389 INFO:teuthology.orchestra.run.smithi164.stdout: "get_sum": 0, 2024-04-26T21:59:34.389 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_fail": 0, 2024-04-26T21:59:34.389 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_success": 0, 2024-04-26T21:59:34.389 INFO:teuthology.orchestra.run.smithi164.stdout: "take": 1050, 2024-04-26T21:59:34.389 INFO:teuthology.orchestra.run.smithi164.stdout: "take_sum": 1050, 2024-04-26T21:59:34.389 INFO:teuthology.orchestra.run.smithi164.stdout: "put": 1050, 2024-04-26T21:59:34.389 INFO:teuthology.orchestra.run.smithi164.stdout: "put_sum": 1050, 2024-04-26T21:59:34.389 INFO:teuthology.orchestra.run.smithi164.stdout: "wait": { 2024-04-26T21:59:34.389 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-26T21:59:34.389 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-26T21:59:34.389 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-26T21:59:34.389 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-04-26T21:59:34.389 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:34.390 INFO:teuthology.orchestra.run.smithi164.stdout: "throttle-write_buf_throttle": { 2024-04-26T21:59:34.390 INFO:teuthology.orchestra.run.smithi164.stdout: "val": 0, 2024-04-26T21:59:34.390 INFO:teuthology.orchestra.run.smithi164.stdout: "max": 3758096384, 2024-04-26T21:59:34.390 INFO:teuthology.orchestra.run.smithi164.stdout: "get_started": 0, 2024-04-26T21:59:34.390 INFO:teuthology.orchestra.run.smithi164.stdout: "get": 0, 2024-04-26T21:59:34.390 INFO:teuthology.orchestra.run.smithi164.stdout: "get_sum": 0, 2024-04-26T21:59:34.390 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_fail": 0, 2024-04-26T21:59:34.390 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_success": 0, 2024-04-26T21:59:34.390 INFO:teuthology.orchestra.run.smithi164.stdout: "take": 0, 2024-04-26T21:59:34.390 INFO:teuthology.orchestra.run.smithi164.stdout: "take_sum": 0, 2024-04-26T21:59:34.390 INFO:teuthology.orchestra.run.smithi164.stdout: "put": 0, 2024-04-26T21:59:34.391 INFO:teuthology.orchestra.run.smithi164.stdout: "put_sum": 0, 2024-04-26T21:59:34.391 INFO:teuthology.orchestra.run.smithi164.stdout: "wait": { 2024-04-26T21:59:34.391 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-26T21:59:34.391 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-26T21:59:34.391 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-26T21:59:34.391 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-04-26T21:59:34.391 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:34.391 INFO:teuthology.orchestra.run.smithi164.stdout: "throttle-write_buf_throttle-0x55fc8f300780": { 2024-04-26T21:59:34.391 INFO:teuthology.orchestra.run.smithi164.stdout: "val": 0, 2024-04-26T21:59:34.391 INFO:teuthology.orchestra.run.smithi164.stdout: "max": 3758096384, 2024-04-26T21:59:34.391 INFO:teuthology.orchestra.run.smithi164.stdout: "get_started": 0, 2024-04-26T21:59:34.391 INFO:teuthology.orchestra.run.smithi164.stdout: "get": 0, 2024-04-26T21:59:34.391 INFO:teuthology.orchestra.run.smithi164.stdout: "get_sum": 0, 2024-04-26T21:59:34.391 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_fail": 0, 2024-04-26T21:59:34.391 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_success": 0, 2024-04-26T21:59:34.392 INFO:teuthology.orchestra.run.smithi164.stdout: "take": 0, 2024-04-26T21:59:34.392 INFO:teuthology.orchestra.run.smithi164.stdout: "take_sum": 0, 2024-04-26T21:59:34.392 INFO:teuthology.orchestra.run.smithi164.stdout: "put": 0, 2024-04-26T21:59:34.392 INFO:teuthology.orchestra.run.smithi164.stdout: "put_sum": 0, 2024-04-26T21:59:34.392 INFO:teuthology.orchestra.run.smithi164.stdout: "wait": { 2024-04-26T21:59:34.392 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-26T21:59:34.392 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-26T21:59:34.392 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-26T21:59:34.392 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-04-26T21:59:34.392 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-04-26T21:59:34.392 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-04-26T21:59:35.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:59:35 smithi076 ceph-mon[31823]: pgmap v573: 97 pgs: 97 active+clean; 1.1 MiB data, 656 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:59:35.316 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:59:35 smithi164 ceph-mon[31916]: pgmap v573: 97 pgs: 97 active+clean; 1.1 MiB data, 656 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:59:35.320 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-04-26T21:59:35.320 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-04-26T21:59:35.321 DEBUG:tasks.check_counter:Getting stats from i 2024-04-26T21:59:35.321 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid a1780950-0415-11ef-bc93-c7b262605968 -- ceph daemon mds.i perf dump 2024-04-26T21:59:35.321 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:59:35 smithi072 ceph-mon[28138]: pgmap v573: 97 pgs: 97 active+clean; 1.1 MiB data, 656 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T21:59:35.772 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/a1780950-0415-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T21:59:37.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:59:37 smithi164 ceph-mon[31916]: pgmap v574: 97 pgs: 97 active+clean; 1.1 MiB data, 656 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-26T21:59:37.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:59:37 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:59:37.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:59:37 smithi076 ceph-mon[31823]: pgmap v574: 97 pgs: 97 active+clean; 1.1 MiB data, 656 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-26T21:59:37.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:59:37 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:59:37.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:59:37 smithi072 ceph-mon[28138]: pgmap v574: 97 pgs: 97 active+clean; 1.1 MiB data, 656 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-26T21:59:37.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:59:37 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:59:38.106 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-04-26T21:59:38.106 INFO:teuthology.orchestra.run.smithi164.stdout: "AsyncMessenger::Worker-0": { 2024-04-26T21:59:38.106 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_messages": 348, 2024-04-26T21:59:38.106 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_messages": 446, 2024-04-26T21:59:38.107 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_bytes": 402025, 2024-04-26T21:59:38.107 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_bytes": 186202, 2024-04-26T21:59:38.107 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_created_connections": 9, 2024-04-26T21:59:38.107 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_active_connections": 3, 2024-04-26T21:59:38.107 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_total_time": 0.112900317, 2024-04-26T21:59:38.107 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_send_time": 0.041690362, 2024-04-26T21:59:38.107 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_recv_time": 0.059583167, 2024-04-26T21:59:38.107 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_fast_dispatch_time": 0.009468523, 2024-04-26T21:59:38.107 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T21:59:38.107 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 445, 2024-04-26T21:59:38.107 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.092657257, 2024-04-26T21:59:38.107 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000208218 2024-04-26T21:59:38.107 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:38.107 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_handle_ack_lat": { 2024-04-26T21:59:38.108 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 442, 2024-04-26T21:59:38.108 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000319516, 2024-04-26T21:59:38.108 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000722 2024-04-26T21:59:38.108 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:38.108 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_encrypted_bytes": 346544, 2024-04-26T21:59:38.108 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_encrypted_bytes": 78064 2024-04-26T21:59:38.108 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:38.108 INFO:teuthology.orchestra.run.smithi164.stdout: "AsyncMessenger::Worker-1": { 2024-04-26T21:59:38.108 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_messages": 41, 2024-04-26T21:59:38.108 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_messages": 129, 2024-04-26T21:59:38.108 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_bytes": 16952, 2024-04-26T21:59:38.108 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_bytes": 272382, 2024-04-26T21:59:38.108 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_created_connections": 13, 2024-04-26T21:59:38.108 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_active_connections": 5, 2024-04-26T21:59:38.109 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_total_time": 0.027853490, 2024-04-26T21:59:38.109 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_send_time": 0.011345272, 2024-04-26T21:59:38.109 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_recv_time": 0.014214426, 2024-04-26T21:59:38.109 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_fast_dispatch_time": 0.000185562, 2024-04-26T21:59:38.109 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T21:59:38.109 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 127, 2024-04-26T21:59:38.109 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.050636783, 2024-04-26T21:59:38.109 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000398714 2024-04-26T21:59:38.109 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:38.109 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_handle_ack_lat": { 2024-04-26T21:59:38.109 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 48, 2024-04-26T21:59:38.109 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000020138, 2024-04-26T21:59:38.109 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000419 2024-04-26T21:59:38.110 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:38.110 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_encrypted_bytes": 160, 2024-04-26T21:59:38.110 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_encrypted_bytes": 201648 2024-04-26T21:59:38.110 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:38.110 INFO:teuthology.orchestra.run.smithi164.stdout: "AsyncMessenger::Worker-2": { 2024-04-26T21:59:38.110 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_messages": 56, 2024-04-26T21:59:38.110 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_messages": 56, 2024-04-26T21:59:38.110 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_bytes": 22200, 2024-04-26T21:59:38.110 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_bytes": 162352, 2024-04-26T21:59:38.110 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_created_connections": 20, 2024-04-26T21:59:38.110 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_active_connections": 5, 2024-04-26T21:59:38.110 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_total_time": 0.025874031, 2024-04-26T21:59:38.110 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_send_time": 0.004085592, 2024-04-26T21:59:38.110 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_recv_time": 0.019989486, 2024-04-26T21:59:38.111 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_fast_dispatch_time": 0.000623891, 2024-04-26T21:59:38.111 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T21:59:38.111 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 55, 2024-04-26T21:59:38.111 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.051051750, 2024-04-26T21:59:38.111 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000928213 2024-04-26T21:59:38.111 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:38.111 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_handle_ack_lat": { 2024-04-26T21:59:38.111 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 48, 2024-04-26T21:59:38.111 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000017868, 2024-04-26T21:59:38.111 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000372 2024-04-26T21:59:38.111 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:38.111 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_encrypted_bytes": 2912, 2024-04-26T21:59:38.111 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_encrypted_bytes": 448 2024-04-26T21:59:38.111 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:38.112 INFO:teuthology.orchestra.run.smithi164.stdout: "cct": { 2024-04-26T21:59:38.112 INFO:teuthology.orchestra.run.smithi164.stdout: "total_workers": 1, 2024-04-26T21:59:38.112 INFO:teuthology.orchestra.run.smithi164.stdout: "unhealthy_workers": 0 2024-04-26T21:59:38.112 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:38.112 INFO:teuthology.orchestra.run.smithi164.stdout: "finisher-MDSRank": { 2024-04-26T21:59:38.112 INFO:teuthology.orchestra.run.smithi164.stdout: "queue_len": 0, 2024-04-26T21:59:38.112 INFO:teuthology.orchestra.run.smithi164.stdout: "complete_latency": { 2024-04-26T21:59:38.112 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 48, 2024-04-26T21:59:38.112 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.001795691, 2024-04-26T21:59:38.112 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000037410 2024-04-26T21:59:38.112 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-04-26T21:59:38.112 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:38.112 INFO:teuthology.orchestra.run.smithi164.stdout: "finisher-PurgeQueue": { 2024-04-26T21:59:38.112 INFO:teuthology.orchestra.run.smithi164.stdout: "queue_len": 0, 2024-04-26T21:59:38.113 INFO:teuthology.orchestra.run.smithi164.stdout: "complete_latency": { 2024-04-26T21:59:38.113 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 2, 2024-04-26T21:59:38.113 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000038563, 2024-04-26T21:59:38.113 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000019281 2024-04-26T21:59:38.113 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-04-26T21:59:38.113 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:38.113 INFO:teuthology.orchestra.run.smithi164.stdout: "mds": { 2024-04-26T21:59:38.113 INFO:teuthology.orchestra.run.smithi164.stdout: "request": 0, 2024-04-26T21:59:38.113 INFO:teuthology.orchestra.run.smithi164.stdout: "reply": 0, 2024-04-26T21:59:38.113 INFO:teuthology.orchestra.run.smithi164.stdout: "reply_latency": { 2024-04-26T21:59:38.113 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-26T21:59:38.114 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-26T21:59:38.114 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-26T21:59:38.114 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:38.114 INFO:teuthology.orchestra.run.smithi164.stdout: "slow_reply": 0, 2024-04-26T21:59:38.114 INFO:teuthology.orchestra.run.smithi164.stdout: "forward": 0, 2024-04-26T21:59:38.114 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_fetch_complete": 0, 2024-04-26T21:59:38.114 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_fetch_keys": 0, 2024-04-26T21:59:38.114 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_commit": 11, 2024-04-26T21:59:38.114 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_split": 0, 2024-04-26T21:59:38.114 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_merge": 0, 2024-04-26T21:59:38.114 INFO:teuthology.orchestra.run.smithi164.stdout: "inodes": 10, 2024-04-26T21:59:38.114 INFO:teuthology.orchestra.run.smithi164.stdout: "inodes_top": 0, 2024-04-26T21:59:38.114 INFO:teuthology.orchestra.run.smithi164.stdout: "inodes_bottom": 10, 2024-04-26T21:59:38.114 INFO:teuthology.orchestra.run.smithi164.stdout: "inodes_pin_tail": 0, 2024-04-26T21:59:38.115 INFO:teuthology.orchestra.run.smithi164.stdout: "inodes_pinned": 10, 2024-04-26T21:59:38.115 INFO:teuthology.orchestra.run.smithi164.stdout: "inodes_expired": 0, 2024-04-26T21:59:38.115 INFO:teuthology.orchestra.run.smithi164.stdout: "inodes_with_caps": 0, 2024-04-26T21:59:38.115 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": 0, 2024-04-26T21:59:38.115 INFO:teuthology.orchestra.run.smithi164.stdout: "subtrees": 1, 2024-04-26T21:59:38.115 INFO:teuthology.orchestra.run.smithi164.stdout: "traverse": 0, 2024-04-26T21:59:38.115 INFO:teuthology.orchestra.run.smithi164.stdout: "traverse_hit": 0, 2024-04-26T21:59:38.115 INFO:teuthology.orchestra.run.smithi164.stdout: "traverse_forward": 0, 2024-04-26T21:59:38.115 INFO:teuthology.orchestra.run.smithi164.stdout: "traverse_discover": 0, 2024-04-26T21:59:38.116 INFO:teuthology.orchestra.run.smithi164.stdout: "traverse_dir_fetch": 0, 2024-04-26T21:59:38.116 INFO:teuthology.orchestra.run.smithi164.stdout: "traverse_remote_ino": 0, 2024-04-26T21:59:38.116 INFO:teuthology.orchestra.run.smithi164.stdout: "traverse_lock": 0, 2024-04-26T21:59:38.116 INFO:teuthology.orchestra.run.smithi164.stdout: "load_cent": 0, 2024-04-26T21:59:38.116 INFO:teuthology.orchestra.run.smithi164.stdout: "q": 0, 2024-04-26T21:59:38.116 INFO:teuthology.orchestra.run.smithi164.stdout: "exported": 0, 2024-04-26T21:59:38.116 INFO:teuthology.orchestra.run.smithi164.stdout: "exported_inodes": 0, 2024-04-26T21:59:38.116 INFO:teuthology.orchestra.run.smithi164.stdout: "imported": 0, 2024-04-26T21:59:38.116 INFO:teuthology.orchestra.run.smithi164.stdout: "imported_inodes": 0, 2024-04-26T21:59:38.116 INFO:teuthology.orchestra.run.smithi164.stdout: "openino_dir_fetch": 0, 2024-04-26T21:59:38.116 INFO:teuthology.orchestra.run.smithi164.stdout: "openino_backtrace_fetch": 0, 2024-04-26T21:59:38.116 INFO:teuthology.orchestra.run.smithi164.stdout: "openino_peer_discover": 0, 2024-04-26T21:59:38.116 INFO:teuthology.orchestra.run.smithi164.stdout: "root_rfiles": 8, 2024-04-26T21:59:38.116 INFO:teuthology.orchestra.run.smithi164.stdout: "root_rbytes": 195290530, 2024-04-26T21:59:38.116 INFO:teuthology.orchestra.run.smithi164.stdout: "root_rsnaps": 0, 2024-04-26T21:59:38.117 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_backtrace_fetch": 0, 2024-04-26T21:59:38.117 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_set_tag": 0, 2024-04-26T21:59:38.117 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_backtrace_repaired": 0, 2024-04-26T21:59:38.117 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_inotable_repaired": 0, 2024-04-26T21:59:38.117 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_dir_inodes": 0, 2024-04-26T21:59:38.117 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_dir_base_inodes": 0, 2024-04-26T21:59:38.117 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_dirfrag_rstats": 0, 2024-04-26T21:59:38.117 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_file_inodes": 0, 2024-04-26T21:59:38.117 INFO:teuthology.orchestra.run.smithi164.stdout: "handle_inode_file_caps": 0, 2024-04-26T21:59:38.117 INFO:teuthology.orchestra.run.smithi164.stdout: "ceph_cap_op_revoke": 0, 2024-04-26T21:59:38.117 INFO:teuthology.orchestra.run.smithi164.stdout: "ceph_cap_op_grant": 0, 2024-04-26T21:59:38.117 INFO:teuthology.orchestra.run.smithi164.stdout: "ceph_cap_op_trunc": 0, 2024-04-26T21:59:38.117 INFO:teuthology.orchestra.run.smithi164.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-26T21:59:38.117 INFO:teuthology.orchestra.run.smithi164.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-26T21:59:38.117 INFO:teuthology.orchestra.run.smithi164.stdout: "handle_client_caps": 0, 2024-04-26T21:59:38.118 INFO:teuthology.orchestra.run.smithi164.stdout: "handle_client_caps_dirty": 0, 2024-04-26T21:59:38.118 INFO:teuthology.orchestra.run.smithi164.stdout: "handle_client_cap_release": 0, 2024-04-26T21:59:38.118 INFO:teuthology.orchestra.run.smithi164.stdout: "process_request_cap_release": 0 2024-04-26T21:59:38.118 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:38.118 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_cache": { 2024-04-26T21:59:38.118 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_update": 0, 2024-04-26T21:59:38.118 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_update_receipt": 0, 2024-04-26T21:59:38.118 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_try_discover": 0, 2024-04-26T21:59:38.118 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_send_discover": 1, 2024-04-26T21:59:38.118 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_handle_discover": 0, 2024-04-26T21:59:38.118 INFO:teuthology.orchestra.run.smithi164.stdout: "num_strays": 0, 2024-04-26T21:59:38.118 INFO:teuthology.orchestra.run.smithi164.stdout: "num_strays_delayed": 0, 2024-04-26T21:59:38.118 INFO:teuthology.orchestra.run.smithi164.stdout: "num_strays_enqueuing": 0, 2024-04-26T21:59:38.119 INFO:teuthology.orchestra.run.smithi164.stdout: "strays_created": 0, 2024-04-26T21:59:38.119 INFO:teuthology.orchestra.run.smithi164.stdout: "strays_enqueued": 0, 2024-04-26T21:59:38.119 INFO:teuthology.orchestra.run.smithi164.stdout: "strays_reintegrated": 0, 2024-04-26T21:59:38.119 INFO:teuthology.orchestra.run.smithi164.stdout: "strays_migrated": 0, 2024-04-26T21:59:38.119 INFO:teuthology.orchestra.run.smithi164.stdout: "num_recovering_processing": 0, 2024-04-26T21:59:38.119 INFO:teuthology.orchestra.run.smithi164.stdout: "num_recovering_enqueued": 0, 2024-04-26T21:59:38.119 INFO:teuthology.orchestra.run.smithi164.stdout: "num_recovering_prioritized": 0, 2024-04-26T21:59:38.119 INFO:teuthology.orchestra.run.smithi164.stdout: "recovery_started": 0, 2024-04-26T21:59:38.119 INFO:teuthology.orchestra.run.smithi164.stdout: "recovery_completed": 0, 2024-04-26T21:59:38.119 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_quiesce_path": 0, 2024-04-26T21:59:38.119 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_quiesce_inode": 0, 2024-04-26T21:59:38.119 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_enqueue_scrub": 0, 2024-04-26T21:59:38.119 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_exportdir": 0, 2024-04-26T21:59:38.119 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_flush": 0, 2024-04-26T21:59:38.120 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_fragmentdir": 0, 2024-04-26T21:59:38.120 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_fragstats": 0, 2024-04-26T21:59:38.120 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_inodestats": 0 2024-04-26T21:59:38.120 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:38.120 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_log": { 2024-04-26T21:59:38.120 INFO:teuthology.orchestra.run.smithi164.stdout: "evlrg": 0, 2024-04-26T21:59:38.120 INFO:teuthology.orchestra.run.smithi164.stdout: "evadd": 3, 2024-04-26T21:59:38.120 INFO:teuthology.orchestra.run.smithi164.stdout: "evex": 0, 2024-04-26T21:59:38.120 INFO:teuthology.orchestra.run.smithi164.stdout: "evtrm": 0, 2024-04-26T21:59:38.121 INFO:teuthology.orchestra.run.smithi164.stdout: "ev": 3, 2024-04-26T21:59:38.121 INFO:teuthology.orchestra.run.smithi164.stdout: "evexg": 0, 2024-04-26T21:59:38.121 INFO:teuthology.orchestra.run.smithi164.stdout: "evexd": 0, 2024-04-26T21:59:38.121 INFO:teuthology.orchestra.run.smithi164.stdout: "segadd": 2, 2024-04-26T21:59:38.121 INFO:teuthology.orchestra.run.smithi164.stdout: "segex": 0, 2024-04-26T21:59:38.121 INFO:teuthology.orchestra.run.smithi164.stdout: "segtrm": 0, 2024-04-26T21:59:38.121 INFO:teuthology.orchestra.run.smithi164.stdout: "seg": 2, 2024-04-26T21:59:38.121 INFO:teuthology.orchestra.run.smithi164.stdout: "segmjr": 2, 2024-04-26T21:59:38.121 INFO:teuthology.orchestra.run.smithi164.stdout: "segexg": 0, 2024-04-26T21:59:38.121 INFO:teuthology.orchestra.run.smithi164.stdout: "segexd": 0, 2024-04-26T21:59:38.121 INFO:teuthology.orchestra.run.smithi164.stdout: "expos": 4194304, 2024-04-26T21:59:38.121 INFO:teuthology.orchestra.run.smithi164.stdout: "wrpos": 4195385, 2024-04-26T21:59:38.121 INFO:teuthology.orchestra.run.smithi164.stdout: "rdpos": 0, 2024-04-26T21:59:38.121 INFO:teuthology.orchestra.run.smithi164.stdout: "jlat": { 2024-04-26T21:59:38.122 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 2, 2024-04-26T21:59:38.122 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.007233276, 2024-04-26T21:59:38.122 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.003616638 2024-04-26T21:59:38.122 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:38.122 INFO:teuthology.orchestra.run.smithi164.stdout: "replayed": 0 2024-04-26T21:59:38.122 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:38.122 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_mem": { 2024-04-26T21:59:38.122 INFO:teuthology.orchestra.run.smithi164.stdout: "ino": 13, 2024-04-26T21:59:38.122 INFO:teuthology.orchestra.run.smithi164.stdout: "ino+": 13, 2024-04-26T21:59:38.122 INFO:teuthology.orchestra.run.smithi164.stdout: "ino-": 0, 2024-04-26T21:59:38.122 INFO:teuthology.orchestra.run.smithi164.stdout: "dir": 11, 2024-04-26T21:59:38.122 INFO:teuthology.orchestra.run.smithi164.stdout: "dir+": 12, 2024-04-26T21:59:38.122 INFO:teuthology.orchestra.run.smithi164.stdout: "dir-": 1, 2024-04-26T21:59:38.122 INFO:teuthology.orchestra.run.smithi164.stdout: "dn": 10, 2024-04-26T21:59:38.123 INFO:teuthology.orchestra.run.smithi164.stdout: "dn+": 10, 2024-04-26T21:59:38.123 INFO:teuthology.orchestra.run.smithi164.stdout: "dn-": 0, 2024-04-26T21:59:38.123 INFO:teuthology.orchestra.run.smithi164.stdout: "cap": 0, 2024-04-26T21:59:38.123 INFO:teuthology.orchestra.run.smithi164.stdout: "cap+": 0, 2024-04-26T21:59:38.123 INFO:teuthology.orchestra.run.smithi164.stdout: "cap-": 0, 2024-04-26T21:59:38.123 INFO:teuthology.orchestra.run.smithi164.stdout: "rss": 49792, 2024-04-26T21:59:38.123 INFO:teuthology.orchestra.run.smithi164.stdout: "heap": 223608 2024-04-26T21:59:38.123 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:38.123 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_server": { 2024-04-26T21:59:38.123 INFO:teuthology.orchestra.run.smithi164.stdout: "dispatch_client_request": 0, 2024-04-26T21:59:38.123 INFO:teuthology.orchestra.run.smithi164.stdout: "dispatch_server_request": 0, 2024-04-26T21:59:38.123 INFO:teuthology.orchestra.run.smithi164.stdout: "handle_client_request": 0, 2024-04-26T21:59:38.123 INFO:teuthology.orchestra.run.smithi164.stdout: "handle_client_session": 5, 2024-04-26T21:59:38.123 INFO:teuthology.orchestra.run.smithi164.stdout: "handle_peer_request": 0, 2024-04-26T21:59:38.123 INFO:teuthology.orchestra.run.smithi164.stdout: "req_create_latency": { 2024-04-26T21:59:38.123 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-26T21:59:38.124 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-26T21:59:38.124 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-26T21:59:38.124 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:38.124 INFO:teuthology.orchestra.run.smithi164.stdout: "req_getattr_latency": { 2024-04-26T21:59:38.124 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-26T21:59:38.124 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-26T21:59:38.124 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-26T21:59:38.124 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:38.124 INFO:teuthology.orchestra.run.smithi164.stdout: "req_getfilelock_latency": { 2024-04-26T21:59:38.124 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-26T21:59:38.124 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-26T21:59:38.124 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-26T21:59:38.124 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:38.124 INFO:teuthology.orchestra.run.smithi164.stdout: "req_link_latency": { 2024-04-26T21:59:38.125 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-26T21:59:38.125 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-26T21:59:38.125 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-26T21:59:38.125 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:38.125 INFO:teuthology.orchestra.run.smithi164.stdout: "req_lookup_latency": { 2024-04-26T21:59:38.125 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-26T21:59:38.125 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-26T21:59:38.125 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-26T21:59:38.125 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:38.125 INFO:teuthology.orchestra.run.smithi164.stdout: "req_lookuphash_latency": { 2024-04-26T21:59:38.125 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-26T21:59:38.125 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-26T21:59:38.125 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-26T21:59:38.126 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:38.126 INFO:teuthology.orchestra.run.smithi164.stdout: "req_lookupino_latency": { 2024-04-26T21:59:38.126 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-26T21:59:38.126 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-26T21:59:38.126 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-26T21:59:38.126 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:38.126 INFO:teuthology.orchestra.run.smithi164.stdout: "req_lookupname_latency": { 2024-04-26T21:59:38.127 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-26T21:59:38.127 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-26T21:59:38.127 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-26T21:59:38.127 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:38.127 INFO:teuthology.orchestra.run.smithi164.stdout: "req_lookupparent_latency": { 2024-04-26T21:59:38.127 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-26T21:59:38.127 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-26T21:59:38.128 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-26T21:59:38.128 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:38.128 INFO:teuthology.orchestra.run.smithi164.stdout: "req_lookupsnap_latency": { 2024-04-26T21:59:38.128 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-26T21:59:38.128 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-26T21:59:38.128 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-26T21:59:38.128 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:38.128 INFO:teuthology.orchestra.run.smithi164.stdout: "req_lssnap_latency": { 2024-04-26T21:59:38.128 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-26T21:59:38.128 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-26T21:59:38.128 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-26T21:59:38.128 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:38.128 INFO:teuthology.orchestra.run.smithi164.stdout: "req_mkdir_latency": { 2024-04-26T21:59:38.129 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-26T21:59:38.129 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-26T21:59:38.129 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-26T21:59:38.130 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:38.130 INFO:teuthology.orchestra.run.smithi164.stdout: "req_mknod_latency": { 2024-04-26T21:59:38.130 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-26T21:59:38.130 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-26T21:59:38.130 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-26T21:59:38.130 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:38.130 INFO:teuthology.orchestra.run.smithi164.stdout: "req_mksnap_latency": { 2024-04-26T21:59:38.130 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-26T21:59:38.130 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-26T21:59:38.130 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-26T21:59:38.130 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:38.130 INFO:teuthology.orchestra.run.smithi164.stdout: "req_open_latency": { 2024-04-26T21:59:38.130 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-26T21:59:38.131 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-26T21:59:38.131 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-26T21:59:38.131 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:38.131 INFO:teuthology.orchestra.run.smithi164.stdout: "req_readdir_latency": { 2024-04-26T21:59:38.131 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-26T21:59:38.131 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-26T21:59:38.131 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-26T21:59:38.131 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:38.131 INFO:teuthology.orchestra.run.smithi164.stdout: "req_rename_latency": { 2024-04-26T21:59:38.131 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-26T21:59:38.131 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-26T21:59:38.131 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-26T21:59:38.131 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:38.131 INFO:teuthology.orchestra.run.smithi164.stdout: "req_renamesnap_latency": { 2024-04-26T21:59:38.131 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-26T21:59:38.132 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-26T21:59:38.132 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-26T21:59:38.132 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:38.132 INFO:teuthology.orchestra.run.smithi164.stdout: "req_snapdiff_latency": { 2024-04-26T21:59:38.132 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-26T21:59:38.132 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-26T21:59:38.132 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-26T21:59:38.132 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:38.132 INFO:teuthology.orchestra.run.smithi164.stdout: "req_rmdir_latency": { 2024-04-26T21:59:38.132 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-26T21:59:38.132 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-26T21:59:38.132 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-26T21:59:38.132 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:38.132 INFO:teuthology.orchestra.run.smithi164.stdout: "req_rmsnap_latency": { 2024-04-26T21:59:38.132 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-26T21:59:38.133 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-26T21:59:38.133 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-26T21:59:38.133 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:38.133 INFO:teuthology.orchestra.run.smithi164.stdout: "req_rmxattr_latency": { 2024-04-26T21:59:38.133 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-26T21:59:38.133 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-26T21:59:38.133 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-26T21:59:38.133 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:38.133 INFO:teuthology.orchestra.run.smithi164.stdout: "req_setattr_latency": { 2024-04-26T21:59:38.133 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-26T21:59:38.133 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-26T21:59:38.133 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-26T21:59:38.133 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:38.133 INFO:teuthology.orchestra.run.smithi164.stdout: "req_setdirlayout_latency": { 2024-04-26T21:59:38.134 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-26T21:59:38.134 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-26T21:59:38.134 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-26T21:59:38.134 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:38.134 INFO:teuthology.orchestra.run.smithi164.stdout: "req_setfilelock_latency": { 2024-04-26T21:59:38.134 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-26T21:59:38.134 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-26T21:59:38.134 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-26T21:59:38.134 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:38.134 INFO:teuthology.orchestra.run.smithi164.stdout: "req_setlayout_latency": { 2024-04-26T21:59:38.134 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-26T21:59:38.134 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-26T21:59:38.134 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-26T21:59:38.135 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:38.135 INFO:teuthology.orchestra.run.smithi164.stdout: "req_setxattr_latency": { 2024-04-26T21:59:38.135 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-26T21:59:38.135 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-26T21:59:38.135 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-26T21:59:38.135 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:38.135 INFO:teuthology.orchestra.run.smithi164.stdout: "req_symlink_latency": { 2024-04-26T21:59:38.135 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-26T21:59:38.135 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-26T21:59:38.135 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-26T21:59:38.135 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:38.135 INFO:teuthology.orchestra.run.smithi164.stdout: "req_unlink_latency": { 2024-04-26T21:59:38.135 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-26T21:59:38.135 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-26T21:59:38.136 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-26T21:59:38.136 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:38.136 INFO:teuthology.orchestra.run.smithi164.stdout: "cap_revoke_eviction": 0, 2024-04-26T21:59:38.136 INFO:teuthology.orchestra.run.smithi164.stdout: "cap_acquisition_throttle": 0, 2024-04-26T21:59:38.136 INFO:teuthology.orchestra.run.smithi164.stdout: "req_getvxattr_latency": { 2024-04-26T21:59:38.136 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-26T21:59:38.136 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-26T21:59:38.136 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-26T21:59:38.137 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-04-26T21:59:38.137 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:38.137 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_sessions": { 2024-04-26T21:59:38.137 INFO:teuthology.orchestra.run.smithi164.stdout: "session_count": 1, 2024-04-26T21:59:38.137 INFO:teuthology.orchestra.run.smithi164.stdout: "session_add": 1, 2024-04-26T21:59:38.137 INFO:teuthology.orchestra.run.smithi164.stdout: "session_remove": 0, 2024-04-26T21:59:38.137 INFO:teuthology.orchestra.run.smithi164.stdout: "sessions_open": 1, 2024-04-26T21:59:38.137 INFO:teuthology.orchestra.run.smithi164.stdout: "sessions_stale": 0, 2024-04-26T21:59:38.137 INFO:teuthology.orchestra.run.smithi164.stdout: "total_load": 0, 2024-04-26T21:59:38.137 INFO:teuthology.orchestra.run.smithi164.stdout: "average_load": 0, 2024-04-26T21:59:38.137 INFO:teuthology.orchestra.run.smithi164.stdout: "avg_session_uptime": 256, 2024-04-26T21:59:38.138 INFO:teuthology.orchestra.run.smithi164.stdout: "mdthresh_evicted": 0 2024-04-26T21:59:38.138 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:38.138 INFO:teuthology.orchestra.run.smithi164.stdout: "mempool": { 2024-04-26T21:59:38.138 INFO:teuthology.orchestra.run.smithi164.stdout: "bloom_filter_bytes": 0, 2024-04-26T21:59:38.138 INFO:teuthology.orchestra.run.smithi164.stdout: "bloom_filter_items": 0, 2024-04-26T21:59:38.138 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_alloc_bytes": 0, 2024-04-26T21:59:38.138 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_alloc_items": 0, 2024-04-26T21:59:38.138 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_data_bytes": 0, 2024-04-26T21:59:38.138 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_data_items": 0, 2024-04-26T21:59:38.138 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-26T21:59:38.139 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_onode_items": 0, 2024-04-26T21:59:38.139 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-26T21:59:38.139 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_meta_items": 0, 2024-04-26T21:59:38.139 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_other_bytes": 0, 2024-04-26T21:59:38.139 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_other_items": 0, 2024-04-26T21:59:38.139 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-26T21:59:38.139 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_buffer_items": 0, 2024-04-26T21:59:38.139 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_extent_bytes": 0, 2024-04-26T21:59:38.139 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_extent_items": 0, 2024-04-26T21:59:38.139 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_blob_bytes": 0, 2024-04-26T21:59:38.140 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_blob_items": 0, 2024-04-26T21:59:38.140 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-26T21:59:38.140 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_shared_blob_items": 0, 2024-04-26T21:59:38.140 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-26T21:59:38.140 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_inline_bl_items": 0, 2024-04-26T21:59:38.140 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_fsck_bytes": 0, 2024-04-26T21:59:38.140 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_fsck_items": 0, 2024-04-26T21:59:38.140 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_txc_bytes": 0, 2024-04-26T21:59:38.140 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_txc_items": 0, 2024-04-26T21:59:38.140 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-26T21:59:38.140 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_writing_deferred_items": 0, 2024-04-26T21:59:38.140 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_writing_bytes": 0, 2024-04-26T21:59:38.140 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_writing_items": 0, 2024-04-26T21:59:38.140 INFO:teuthology.orchestra.run.smithi164.stdout: "bluefs_bytes": 0, 2024-04-26T21:59:38.141 INFO:teuthology.orchestra.run.smithi164.stdout: "bluefs_items": 0, 2024-04-26T21:59:38.141 INFO:teuthology.orchestra.run.smithi164.stdout: "bluefs_file_reader_bytes": 0, 2024-04-26T21:59:38.141 INFO:teuthology.orchestra.run.smithi164.stdout: "bluefs_file_reader_items": 0, 2024-04-26T21:59:38.141 INFO:teuthology.orchestra.run.smithi164.stdout: "bluefs_file_writer_bytes": 0, 2024-04-26T21:59:38.141 INFO:teuthology.orchestra.run.smithi164.stdout: "bluefs_file_writer_items": 0, 2024-04-26T21:59:38.141 INFO:teuthology.orchestra.run.smithi164.stdout: "buffer_anon_bytes": 262151, 2024-04-26T21:59:38.141 INFO:teuthology.orchestra.run.smithi164.stdout: "buffer_anon_items": 86, 2024-04-26T21:59:38.141 INFO:teuthology.orchestra.run.smithi164.stdout: "buffer_meta_bytes": 0, 2024-04-26T21:59:38.141 INFO:teuthology.orchestra.run.smithi164.stdout: "buffer_meta_items": 0, 2024-04-26T21:59:38.141 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_bytes": 0, 2024-04-26T21:59:38.141 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_items": 0, 2024-04-26T21:59:38.141 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_mapbl_bytes": 0, 2024-04-26T21:59:38.141 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_mapbl_items": 0, 2024-04-26T21:59:38.141 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_pglog_bytes": 0, 2024-04-26T21:59:38.142 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_pglog_items": 0, 2024-04-26T21:59:38.142 INFO:teuthology.orchestra.run.smithi164.stdout: "osdmap_bytes": 8008, 2024-04-26T21:59:38.142 INFO:teuthology.orchestra.run.smithi164.stdout: "osdmap_items": 168, 2024-04-26T21:59:38.142 INFO:teuthology.orchestra.run.smithi164.stdout: "osdmap_mapping_bytes": 0, 2024-04-26T21:59:38.142 INFO:teuthology.orchestra.run.smithi164.stdout: "osdmap_mapping_items": 0, 2024-04-26T21:59:38.142 INFO:teuthology.orchestra.run.smithi164.stdout: "pgmap_bytes": 0, 2024-04-26T21:59:38.142 INFO:teuthology.orchestra.run.smithi164.stdout: "pgmap_items": 0, 2024-04-26T21:59:38.142 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_co_bytes": 50872, 2024-04-26T21:59:38.142 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_co_items": 198, 2024-04-26T21:59:38.142 INFO:teuthology.orchestra.run.smithi164.stdout: "unittest_1_bytes": 0, 2024-04-26T21:59:38.142 INFO:teuthology.orchestra.run.smithi164.stdout: "unittest_1_items": 0, 2024-04-26T21:59:38.142 INFO:teuthology.orchestra.run.smithi164.stdout: "unittest_2_bytes": 0, 2024-04-26T21:59:38.142 INFO:teuthology.orchestra.run.smithi164.stdout: "unittest_2_items": 0 2024-04-26T21:59:38.142 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:38.143 INFO:teuthology.orchestra.run.smithi164.stdout: "objecter": { 2024-04-26T21:59:38.143 INFO:teuthology.orchestra.run.smithi164.stdout: "op_active": 0, 2024-04-26T21:59:38.143 INFO:teuthology.orchestra.run.smithi164.stdout: "op_laggy": 0, 2024-04-26T21:59:38.143 INFO:teuthology.orchestra.run.smithi164.stdout: "op_send": 37, 2024-04-26T21:59:38.143 INFO:teuthology.orchestra.run.smithi164.stdout: "op_send_bytes": 11293, 2024-04-26T21:59:38.143 INFO:teuthology.orchestra.run.smithi164.stdout: "op_resend": 33, 2024-04-26T21:59:38.143 INFO:teuthology.orchestra.run.smithi164.stdout: "op_reply": 37, 2024-04-26T21:59:38.143 INFO:teuthology.orchestra.run.smithi164.stdout: "op_latency": { 2024-04-26T21:59:38.143 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 37, 2024-04-26T21:59:38.143 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.453992213, 2024-04-26T21:59:38.143 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.012270059 2024-04-26T21:59:38.143 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:38.144 INFO:teuthology.orchestra.run.smithi164.stdout: "op_inflight": 0, 2024-04-26T21:59:38.144 INFO:teuthology.orchestra.run.smithi164.stdout: "oplen_avg": { 2024-04-26T21:59:38.144 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 37, 2024-04-26T21:59:38.144 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 59 2024-04-26T21:59:38.144 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:38.144 INFO:teuthology.orchestra.run.smithi164.stdout: "op": 37, 2024-04-26T21:59:38.145 INFO:teuthology.orchestra.run.smithi164.stdout: "op_r": 0, 2024-04-26T21:59:38.145 INFO:teuthology.orchestra.run.smithi164.stdout: "op_w": 37, 2024-04-26T21:59:38.145 INFO:teuthology.orchestra.run.smithi164.stdout: "op_rmw": 0, 2024-04-26T21:59:38.145 INFO:teuthology.orchestra.run.smithi164.stdout: "op_pg": 0, 2024-04-26T21:59:38.145 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_stat": 0, 2024-04-26T21:59:38.145 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_create": 10, 2024-04-26T21:59:38.145 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_read": 0, 2024-04-26T21:59:38.146 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_write": 2, 2024-04-26T21:59:38.146 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_writefull": 6, 2024-04-26T21:59:38.146 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_writesame": 0, 2024-04-26T21:59:38.146 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_append": 0, 2024-04-26T21:59:38.146 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_zero": 0, 2024-04-26T21:59:38.146 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_truncate": 0, 2024-04-26T21:59:38.147 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_delete": 6, 2024-04-26T21:59:38.147 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_mapext": 0, 2024-04-26T21:59:38.147 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_sparse_read": 0, 2024-04-26T21:59:38.147 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_clonerange": 0, 2024-04-26T21:59:38.147 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_getxattr": 0, 2024-04-26T21:59:38.147 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_setxattr": 20, 2024-04-26T21:59:38.148 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_cmpxattr": 0, 2024-04-26T21:59:38.148 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_rmxattr": 0, 2024-04-26T21:59:38.148 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_resetxattrs": 0, 2024-04-26T21:59:38.148 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_call": 0, 2024-04-26T21:59:38.148 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_watch": 0, 2024-04-26T21:59:38.148 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_notify": 0, 2024-04-26T21:59:38.148 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_src_cmpxattr": 0, 2024-04-26T21:59:38.148 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_pgls": 0, 2024-04-26T21:59:38.148 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_pgls_filter": 0, 2024-04-26T21:59:38.148 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_other": 0, 2024-04-26T21:59:38.148 INFO:teuthology.orchestra.run.smithi164.stdout: "linger_active": 0, 2024-04-26T21:59:38.148 INFO:teuthology.orchestra.run.smithi164.stdout: "linger_send": 0, 2024-04-26T21:59:38.148 INFO:teuthology.orchestra.run.smithi164.stdout: "linger_resend": 0, 2024-04-26T21:59:38.148 INFO:teuthology.orchestra.run.smithi164.stdout: "linger_ping": 0, 2024-04-26T21:59:38.148 INFO:teuthology.orchestra.run.smithi164.stdout: "poolop_active": 0, 2024-04-26T21:59:38.149 INFO:teuthology.orchestra.run.smithi164.stdout: "poolop_send": 0, 2024-04-26T21:59:38.149 INFO:teuthology.orchestra.run.smithi164.stdout: "poolop_resend": 0, 2024-04-26T21:59:38.149 INFO:teuthology.orchestra.run.smithi164.stdout: "poolstat_active": 0, 2024-04-26T21:59:38.149 INFO:teuthology.orchestra.run.smithi164.stdout: "poolstat_send": 0, 2024-04-26T21:59:38.149 INFO:teuthology.orchestra.run.smithi164.stdout: "poolstat_resend": 0, 2024-04-26T21:59:38.149 INFO:teuthology.orchestra.run.smithi164.stdout: "statfs_active": 0, 2024-04-26T21:59:38.149 INFO:teuthology.orchestra.run.smithi164.stdout: "statfs_send": 0, 2024-04-26T21:59:38.149 INFO:teuthology.orchestra.run.smithi164.stdout: "statfs_resend": 0, 2024-04-26T21:59:38.149 INFO:teuthology.orchestra.run.smithi164.stdout: "command_active": 0, 2024-04-26T21:59:38.149 INFO:teuthology.orchestra.run.smithi164.stdout: "command_send": 0, 2024-04-26T21:59:38.149 INFO:teuthology.orchestra.run.smithi164.stdout: "command_resend": 0, 2024-04-26T21:59:38.149 INFO:teuthology.orchestra.run.smithi164.stdout: "map_epoch": 91, 2024-04-26T21:59:38.149 INFO:teuthology.orchestra.run.smithi164.stdout: "map_full": 0, 2024-04-26T21:59:38.149 INFO:teuthology.orchestra.run.smithi164.stdout: "map_inc": 4, 2024-04-26T21:59:38.150 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_sessions": 9, 2024-04-26T21:59:38.150 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_session_open": 9, 2024-04-26T21:59:38.150 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_session_close": 0, 2024-04-26T21:59:38.150 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_laggy": 0, 2024-04-26T21:59:38.150 INFO:teuthology.orchestra.run.smithi164.stdout: "omap_wr": 14, 2024-04-26T21:59:38.150 INFO:teuthology.orchestra.run.smithi164.stdout: "omap_rd": 0, 2024-04-26T21:59:38.150 INFO:teuthology.orchestra.run.smithi164.stdout: "omap_del": 1 2024-04-26T21:59:38.150 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:38.150 INFO:teuthology.orchestra.run.smithi164.stdout: "oft": { 2024-04-26T21:59:38.150 INFO:teuthology.orchestra.run.smithi164.stdout: "omap_total_objs": 1, 2024-04-26T21:59:38.150 INFO:teuthology.orchestra.run.smithi164.stdout: "omap_total_kv_pairs": 0, 2024-04-26T21:59:38.150 INFO:teuthology.orchestra.run.smithi164.stdout: "omap_total_updates": 0, 2024-04-26T21:59:38.150 INFO:teuthology.orchestra.run.smithi164.stdout: "omap_total_removes": 0 2024-04-26T21:59:38.150 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:38.151 INFO:teuthology.orchestra.run.smithi164.stdout: "osd-slow-ops": { 2024-04-26T21:59:38.151 INFO:teuthology.orchestra.run.smithi164.stdout: "slow_ops_count": 0 2024-04-26T21:59:38.151 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:38.151 INFO:teuthology.orchestra.run.smithi164.stdout: "purge_queue": { 2024-04-26T21:59:38.151 INFO:teuthology.orchestra.run.smithi164.stdout: "pq_executing_ops": 0, 2024-04-26T21:59:38.151 INFO:teuthology.orchestra.run.smithi164.stdout: "pq_executing_ops_high_water": 0, 2024-04-26T21:59:38.151 INFO:teuthology.orchestra.run.smithi164.stdout: "pq_executing": 0, 2024-04-26T21:59:38.151 INFO:teuthology.orchestra.run.smithi164.stdout: "pq_executing_high_water": 0, 2024-04-26T21:59:38.151 INFO:teuthology.orchestra.run.smithi164.stdout: "pq_executed_ops": 0, 2024-04-26T21:59:38.151 INFO:teuthology.orchestra.run.smithi164.stdout: "pq_executed": 0, 2024-04-26T21:59:38.151 INFO:teuthology.orchestra.run.smithi164.stdout: "pq_item_in_journal": 0 2024-04-26T21:59:38.151 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:38.151 INFO:teuthology.orchestra.run.smithi164.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-26T21:59:38.151 INFO:teuthology.orchestra.run.smithi164.stdout: "val": 0, 2024-04-26T21:59:38.151 INFO:teuthology.orchestra.run.smithi164.stdout: "max": 104857600, 2024-04-26T21:59:38.152 INFO:teuthology.orchestra.run.smithi164.stdout: "get_started": 0, 2024-04-26T21:59:38.152 INFO:teuthology.orchestra.run.smithi164.stdout: "get": 441, 2024-04-26T21:59:38.152 INFO:teuthology.orchestra.run.smithi164.stdout: "get_sum": 390476, 2024-04-26T21:59:38.152 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_fail": 0, 2024-04-26T21:59:38.152 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_success": 441, 2024-04-26T21:59:38.152 INFO:teuthology.orchestra.run.smithi164.stdout: "take": 0, 2024-04-26T21:59:38.152 INFO:teuthology.orchestra.run.smithi164.stdout: "take_sum": 0, 2024-04-26T21:59:38.152 INFO:teuthology.orchestra.run.smithi164.stdout: "put": 441, 2024-04-26T21:59:38.152 INFO:teuthology.orchestra.run.smithi164.stdout: "put_sum": 390476, 2024-04-26T21:59:38.152 INFO:teuthology.orchestra.run.smithi164.stdout: "wait": { 2024-04-26T21:59:38.152 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-26T21:59:38.152 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-26T21:59:38.153 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-26T21:59:38.153 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-04-26T21:59:38.153 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:38.153 INFO:teuthology.orchestra.run.smithi164.stdout: "throttle-objecter_bytes": { 2024-04-26T21:59:38.153 INFO:teuthology.orchestra.run.smithi164.stdout: "val": 0, 2024-04-26T21:59:38.153 INFO:teuthology.orchestra.run.smithi164.stdout: "max": 104857600, 2024-04-26T21:59:38.153 INFO:teuthology.orchestra.run.smithi164.stdout: "get_started": 0, 2024-04-26T21:59:38.153 INFO:teuthology.orchestra.run.smithi164.stdout: "get": 0, 2024-04-26T21:59:38.153 INFO:teuthology.orchestra.run.smithi164.stdout: "get_sum": 0, 2024-04-26T21:59:38.153 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_fail": 0, 2024-04-26T21:59:38.153 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_success": 0, 2024-04-26T21:59:38.153 INFO:teuthology.orchestra.run.smithi164.stdout: "take": 37, 2024-04-26T21:59:38.153 INFO:teuthology.orchestra.run.smithi164.stdout: "take_sum": 11293, 2024-04-26T21:59:38.153 INFO:teuthology.orchestra.run.smithi164.stdout: "put": 37, 2024-04-26T21:59:38.153 INFO:teuthology.orchestra.run.smithi164.stdout: "put_sum": 11293, 2024-04-26T21:59:38.154 INFO:teuthology.orchestra.run.smithi164.stdout: "wait": { 2024-04-26T21:59:38.154 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-26T21:59:38.154 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-26T21:59:38.154 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-26T21:59:38.154 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-04-26T21:59:38.154 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:38.154 INFO:teuthology.orchestra.run.smithi164.stdout: "throttle-objecter_ops": { 2024-04-26T21:59:38.154 INFO:teuthology.orchestra.run.smithi164.stdout: "val": 0, 2024-04-26T21:59:38.154 INFO:teuthology.orchestra.run.smithi164.stdout: "max": 1024, 2024-04-26T21:59:38.154 INFO:teuthology.orchestra.run.smithi164.stdout: "get_started": 0, 2024-04-26T21:59:38.154 INFO:teuthology.orchestra.run.smithi164.stdout: "get": 0, 2024-04-26T21:59:38.154 INFO:teuthology.orchestra.run.smithi164.stdout: "get_sum": 0, 2024-04-26T21:59:38.154 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_fail": 0, 2024-04-26T21:59:38.155 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_success": 0, 2024-04-26T21:59:38.155 INFO:teuthology.orchestra.run.smithi164.stdout: "take": 37, 2024-04-26T21:59:38.155 INFO:teuthology.orchestra.run.smithi164.stdout: "take_sum": 37, 2024-04-26T21:59:38.155 INFO:teuthology.orchestra.run.smithi164.stdout: "put": 37, 2024-04-26T21:59:38.155 INFO:teuthology.orchestra.run.smithi164.stdout: "put_sum": 37, 2024-04-26T21:59:38.155 INFO:teuthology.orchestra.run.smithi164.stdout: "wait": { 2024-04-26T21:59:38.155 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-26T21:59:38.155 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-26T21:59:38.155 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-26T21:59:38.155 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-04-26T21:59:38.155 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:38.155 INFO:teuthology.orchestra.run.smithi164.stdout: "throttle-write_buf_throttle": { 2024-04-26T21:59:38.155 INFO:teuthology.orchestra.run.smithi164.stdout: "val": 0, 2024-04-26T21:59:38.155 INFO:teuthology.orchestra.run.smithi164.stdout: "max": 3758096384, 2024-04-26T21:59:38.156 INFO:teuthology.orchestra.run.smithi164.stdout: "get_started": 0, 2024-04-26T21:59:38.156 INFO:teuthology.orchestra.run.smithi164.stdout: "get": 0, 2024-04-26T21:59:38.156 INFO:teuthology.orchestra.run.smithi164.stdout: "get_sum": 0, 2024-04-26T21:59:38.156 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_fail": 0, 2024-04-26T21:59:38.156 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_success": 0, 2024-04-26T21:59:38.156 INFO:teuthology.orchestra.run.smithi164.stdout: "take": 0, 2024-04-26T21:59:38.156 INFO:teuthology.orchestra.run.smithi164.stdout: "take_sum": 0, 2024-04-26T21:59:38.156 INFO:teuthology.orchestra.run.smithi164.stdout: "put": 0, 2024-04-26T21:59:38.156 INFO:teuthology.orchestra.run.smithi164.stdout: "put_sum": 0, 2024-04-26T21:59:38.156 INFO:teuthology.orchestra.run.smithi164.stdout: "wait": { 2024-04-26T21:59:38.156 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-26T21:59:38.156 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-26T21:59:38.156 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-26T21:59:38.156 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-04-26T21:59:38.156 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:38.157 INFO:teuthology.orchestra.run.smithi164.stdout: "throttle-write_buf_throttle-0x564cd6eae500": { 2024-04-26T21:59:38.157 INFO:teuthology.orchestra.run.smithi164.stdout: "val": 0, 2024-04-26T21:59:38.157 INFO:teuthology.orchestra.run.smithi164.stdout: "max": 3758096384, 2024-04-26T21:59:38.157 INFO:teuthology.orchestra.run.smithi164.stdout: "get_started": 0, 2024-04-26T21:59:38.157 INFO:teuthology.orchestra.run.smithi164.stdout: "get": 3, 2024-04-26T21:59:38.157 INFO:teuthology.orchestra.run.smithi164.stdout: "get_sum": 1081, 2024-04-26T21:59:38.157 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_fail": 0, 2024-04-26T21:59:38.157 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_success": 3, 2024-04-26T21:59:38.157 INFO:teuthology.orchestra.run.smithi164.stdout: "take": 0, 2024-04-26T21:59:38.157 INFO:teuthology.orchestra.run.smithi164.stdout: "take_sum": 0, 2024-04-26T21:59:38.157 INFO:teuthology.orchestra.run.smithi164.stdout: "put": 2, 2024-04-26T21:59:38.157 INFO:teuthology.orchestra.run.smithi164.stdout: "put_sum": 1081, 2024-04-26T21:59:38.157 INFO:teuthology.orchestra.run.smithi164.stdout: "wait": { 2024-04-26T21:59:38.157 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-26T21:59:38.157 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-26T21:59:38.158 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-26T21:59:38.158 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-04-26T21:59:38.158 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-04-26T21:59:38.158 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-04-26T21:59:38.159 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:59:38 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:59:38.159 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:59:38 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:59:38.159 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:59:38 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:59:38.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:59:38 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:59:38.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:59:38 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:59:38.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:59:38 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:59:38.321 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:59:38 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:59:38.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:59:38 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:59:38.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:59:38 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' 2024-04-26T21:59:39.185 INFO:tasks.check_counter:Daemon mds.i mds.exported=0 2024-04-26T21:59:39.185 INFO:tasks.check_counter:Daemon mds.i mds.imported=0 2024-04-26T21:59:39.185 DEBUG:tasks.check_counter:Getting stats from l 2024-04-26T21:59:39.186 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid a1780950-0415-11ef-bc93-c7b262605968 -- ceph daemon mds.l perf dump 2024-04-26T21:59:39.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:59:39 smithi076 ceph-mon[31823]: pgmap v575: 97 pgs: 97 active+clean; 1.1 MiB data, 656 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:59:39.321 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:59:39 smithi072 ceph-mon[28138]: pgmap v575: 97 pgs: 97 active+clean; 1.1 MiB data, 656 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:59:39.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:59:39 smithi164 ceph-mon[31916]: pgmap v575: 97 pgs: 97 active+clean; 1.1 MiB data, 656 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:59:39.633 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/a1780950-0415-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T21:59:41.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:59:41 smithi076 ceph-mon[31823]: pgmap v576: 97 pgs: 97 active+clean; 1.1 MiB data, 656 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-26T21:59:41.321 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:59:41 smithi072 ceph-mon[28138]: pgmap v576: 97 pgs: 97 active+clean; 1.1 MiB data, 656 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-26T21:59:41.382 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:59:41 smithi164 ceph-mon[31916]: pgmap v576: 97 pgs: 97 active+clean; 1.1 MiB data, 656 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-26T21:59:41.954 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-04-26T21:59:41.954 INFO:teuthology.orchestra.run.smithi164.stdout: "AsyncMessenger::Worker-0": { 2024-04-26T21:59:41.954 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_messages": 408, 2024-04-26T21:59:41.954 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_messages": 410, 2024-04-26T21:59:41.954 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_bytes": 108584, 2024-04-26T21:59:41.954 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_bytes": 129808, 2024-04-26T21:59:41.954 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_created_connections": 6, 2024-04-26T21:59:41.954 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_active_connections": 2, 2024-04-26T21:59:41.955 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_total_time": 0.085483092, 2024-04-26T21:59:41.955 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_send_time": 0.027966025, 2024-04-26T21:59:41.955 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_recv_time": 0.022822632, 2024-04-26T21:59:41.955 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_fast_dispatch_time": 0.022645318, 2024-04-26T21:59:41.955 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T21:59:41.955 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 408, 2024-04-26T21:59:41.955 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.049794263, 2024-04-26T21:59:41.955 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000122044 2024-04-26T21:59:41.955 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:41.955 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_handle_ack_lat": { 2024-04-26T21:59:41.955 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-26T21:59:41.955 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-26T21:59:41.955 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-26T21:59:41.956 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:41.956 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-26T21:59:41.956 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_encrypted_bytes": 0 2024-04-26T21:59:41.956 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:41.956 INFO:teuthology.orchestra.run.smithi164.stdout: "AsyncMessenger::Worker-1": { 2024-04-26T21:59:41.956 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_messages": 268, 2024-04-26T21:59:41.956 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_messages": 361, 2024-04-26T21:59:41.956 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_bytes": 67792, 2024-04-26T21:59:41.956 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_bytes": 286967, 2024-04-26T21:59:41.956 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_created_connections": 5, 2024-04-26T21:59:41.956 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_active_connections": 4, 2024-04-26T21:59:41.956 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_total_time": 0.068347967, 2024-04-26T21:59:41.956 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_send_time": 0.025052197, 2024-04-26T21:59:41.957 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_recv_time": 0.016785887, 2024-04-26T21:59:41.957 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_fast_dispatch_time": 0.018549700, 2024-04-26T21:59:41.957 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T21:59:41.957 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 360, 2024-04-26T21:59:41.957 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.036588376, 2024-04-26T21:59:41.957 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000101634 2024-04-26T21:59:41.957 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:41.957 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_handle_ack_lat": { 2024-04-26T21:59:41.957 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-26T21:59:41.957 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-26T21:59:41.957 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-26T21:59:41.957 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:41.957 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_encrypted_bytes": 3072, 2024-04-26T21:59:41.957 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_encrypted_bytes": 203328 2024-04-26T21:59:41.958 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:41.958 INFO:teuthology.orchestra.run.smithi164.stdout: "AsyncMessenger::Worker-2": { 2024-04-26T21:59:41.958 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_messages": 541, 2024-04-26T21:59:41.958 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_messages": 538, 2024-04-26T21:59:41.958 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_bytes": 475504, 2024-04-26T21:59:41.958 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_bytes": 206538, 2024-04-26T21:59:41.958 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_created_connections": 3, 2024-04-26T21:59:41.958 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_active_connections": 2, 2024-04-26T21:59:41.958 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_total_time": 0.137096785, 2024-04-26T21:59:41.958 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_send_time": 0.049855191, 2024-04-26T21:59:41.958 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_recv_time": 0.049334662, 2024-04-26T21:59:41.958 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_fast_dispatch_time": 0.031078732, 2024-04-26T21:59:41.958 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T21:59:41.958 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 537, 2024-04-26T21:59:41.959 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.058916416, 2024-04-26T21:59:41.959 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000109713 2024-04-26T21:59:41.959 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:41.959 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_handle_ack_lat": { 2024-04-26T21:59:41.959 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-26T21:59:41.959 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-26T21:59:41.959 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-26T21:59:41.959 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:41.959 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_encrypted_bytes": 340096, 2024-04-26T21:59:41.959 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_encrypted_bytes": 78384 2024-04-26T21:59:41.959 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:41.959 INFO:teuthology.orchestra.run.smithi164.stdout: "cct": { 2024-04-26T21:59:41.959 INFO:teuthology.orchestra.run.smithi164.stdout: "total_workers": 1, 2024-04-26T21:59:41.959 INFO:teuthology.orchestra.run.smithi164.stdout: "unhealthy_workers": 0 2024-04-26T21:59:41.960 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:41.960 INFO:teuthology.orchestra.run.smithi164.stdout: "finisher-MDSRank": { 2024-04-26T21:59:41.960 INFO:teuthology.orchestra.run.smithi164.stdout: "queue_len": 0, 2024-04-26T21:59:41.960 INFO:teuthology.orchestra.run.smithi164.stdout: "complete_latency": { 2024-04-26T21:59:41.960 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 1826, 2024-04-26T21:59:41.960 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.097713698, 2024-04-26T21:59:41.960 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000053512 2024-04-26T21:59:41.960 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-04-26T21:59:41.960 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:41.960 INFO:teuthology.orchestra.run.smithi164.stdout: "finisher-PurgeQueue": { 2024-04-26T21:59:41.960 INFO:teuthology.orchestra.run.smithi164.stdout: "queue_len": 0, 2024-04-26T21:59:41.960 INFO:teuthology.orchestra.run.smithi164.stdout: "complete_latency": { 2024-04-26T21:59:41.960 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-26T21:59:41.960 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-26T21:59:41.960 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-26T21:59:41.961 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-04-26T21:59:41.961 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:41.961 INFO:teuthology.orchestra.run.smithi164.stdout: "mds": { 2024-04-26T21:59:41.961 INFO:teuthology.orchestra.run.smithi164.stdout: "request": 0, 2024-04-26T21:59:41.961 INFO:teuthology.orchestra.run.smithi164.stdout: "reply": 0, 2024-04-26T21:59:41.961 INFO:teuthology.orchestra.run.smithi164.stdout: "reply_latency": { 2024-04-26T21:59:41.961 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-26T21:59:41.961 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-26T21:59:41.961 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-26T21:59:41.961 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:41.961 INFO:teuthology.orchestra.run.smithi164.stdout: "slow_reply": 0, 2024-04-26T21:59:41.961 INFO:teuthology.orchestra.run.smithi164.stdout: "forward": 0, 2024-04-26T21:59:41.961 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_fetch_complete": 0, 2024-04-26T21:59:41.961 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_fetch_keys": 0, 2024-04-26T21:59:41.962 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_commit": 0, 2024-04-26T21:59:41.962 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_split": 0, 2024-04-26T21:59:41.962 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_merge": 0, 2024-04-26T21:59:41.962 INFO:teuthology.orchestra.run.smithi164.stdout: "inodes": 0, 2024-04-26T21:59:41.962 INFO:teuthology.orchestra.run.smithi164.stdout: "inodes_top": 0, 2024-04-26T21:59:41.962 INFO:teuthology.orchestra.run.smithi164.stdout: "inodes_bottom": 0, 2024-04-26T21:59:41.962 INFO:teuthology.orchestra.run.smithi164.stdout: "inodes_pin_tail": 0, 2024-04-26T21:59:41.962 INFO:teuthology.orchestra.run.smithi164.stdout: "inodes_pinned": 0, 2024-04-26T21:59:41.962 INFO:teuthology.orchestra.run.smithi164.stdout: "inodes_expired": 0, 2024-04-26T21:59:41.962 INFO:teuthology.orchestra.run.smithi164.stdout: "inodes_with_caps": 0, 2024-04-26T21:59:41.962 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": 0, 2024-04-26T21:59:41.962 INFO:teuthology.orchestra.run.smithi164.stdout: "subtrees": 1, 2024-04-26T21:59:41.962 INFO:teuthology.orchestra.run.smithi164.stdout: "traverse": 0, 2024-04-26T21:59:41.962 INFO:teuthology.orchestra.run.smithi164.stdout: "traverse_hit": 0, 2024-04-26T21:59:41.963 INFO:teuthology.orchestra.run.smithi164.stdout: "traverse_forward": 0, 2024-04-26T21:59:41.963 INFO:teuthology.orchestra.run.smithi164.stdout: "traverse_discover": 0, 2024-04-26T21:59:41.963 INFO:teuthology.orchestra.run.smithi164.stdout: "traverse_dir_fetch": 0, 2024-04-26T21:59:41.963 INFO:teuthology.orchestra.run.smithi164.stdout: "traverse_remote_ino": 0, 2024-04-26T21:59:41.963 INFO:teuthology.orchestra.run.smithi164.stdout: "traverse_lock": 0, 2024-04-26T21:59:41.963 INFO:teuthology.orchestra.run.smithi164.stdout: "load_cent": 0, 2024-04-26T21:59:41.963 INFO:teuthology.orchestra.run.smithi164.stdout: "q": 0, 2024-04-26T21:59:41.963 INFO:teuthology.orchestra.run.smithi164.stdout: "exported": 0, 2024-04-26T21:59:41.963 INFO:teuthology.orchestra.run.smithi164.stdout: "exported_inodes": 0, 2024-04-26T21:59:41.963 INFO:teuthology.orchestra.run.smithi164.stdout: "imported": 0, 2024-04-26T21:59:41.963 INFO:teuthology.orchestra.run.smithi164.stdout: "imported_inodes": 0, 2024-04-26T21:59:41.963 INFO:teuthology.orchestra.run.smithi164.stdout: "openino_dir_fetch": 0, 2024-04-26T21:59:41.963 INFO:teuthology.orchestra.run.smithi164.stdout: "openino_backtrace_fetch": 0, 2024-04-26T21:59:41.963 INFO:teuthology.orchestra.run.smithi164.stdout: "openino_peer_discover": 0, 2024-04-26T21:59:41.963 INFO:teuthology.orchestra.run.smithi164.stdout: "root_rfiles": 0, 2024-04-26T21:59:41.964 INFO:teuthology.orchestra.run.smithi164.stdout: "root_rbytes": 0, 2024-04-26T21:59:41.964 INFO:teuthology.orchestra.run.smithi164.stdout: "root_rsnaps": 0, 2024-04-26T21:59:41.964 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_backtrace_fetch": 0, 2024-04-26T21:59:41.964 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_set_tag": 0, 2024-04-26T21:59:41.964 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_backtrace_repaired": 0, 2024-04-26T21:59:41.964 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_inotable_repaired": 0, 2024-04-26T21:59:41.964 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_dir_inodes": 0, 2024-04-26T21:59:41.964 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_dir_base_inodes": 0, 2024-04-26T21:59:41.964 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_dirfrag_rstats": 0, 2024-04-26T21:59:41.964 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_file_inodes": 0, 2024-04-26T21:59:41.964 INFO:teuthology.orchestra.run.smithi164.stdout: "handle_inode_file_caps": 0, 2024-04-26T21:59:41.964 INFO:teuthology.orchestra.run.smithi164.stdout: "ceph_cap_op_revoke": 0, 2024-04-26T21:59:41.964 INFO:teuthology.orchestra.run.smithi164.stdout: "ceph_cap_op_grant": 0, 2024-04-26T21:59:41.964 INFO:teuthology.orchestra.run.smithi164.stdout: "ceph_cap_op_trunc": 0, 2024-04-26T21:59:41.965 INFO:teuthology.orchestra.run.smithi164.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-26T21:59:41.965 INFO:teuthology.orchestra.run.smithi164.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-26T21:59:41.965 INFO:teuthology.orchestra.run.smithi164.stdout: "handle_client_caps": 0, 2024-04-26T21:59:41.965 INFO:teuthology.orchestra.run.smithi164.stdout: "handle_client_caps_dirty": 0, 2024-04-26T21:59:41.965 INFO:teuthology.orchestra.run.smithi164.stdout: "handle_client_cap_release": 0, 2024-04-26T21:59:41.965 INFO:teuthology.orchestra.run.smithi164.stdout: "process_request_cap_release": 0 2024-04-26T21:59:41.965 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:41.965 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_cache": { 2024-04-26T21:59:41.965 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_update": 0, 2024-04-26T21:59:41.965 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_update_receipt": 0, 2024-04-26T21:59:41.965 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_try_discover": 0, 2024-04-26T21:59:41.965 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_send_discover": 0, 2024-04-26T21:59:41.965 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_handle_discover": 0, 2024-04-26T21:59:41.965 INFO:teuthology.orchestra.run.smithi164.stdout: "num_strays": 0, 2024-04-26T21:59:41.965 INFO:teuthology.orchestra.run.smithi164.stdout: "num_strays_delayed": 0, 2024-04-26T21:59:41.966 INFO:teuthology.orchestra.run.smithi164.stdout: "num_strays_enqueuing": 0, 2024-04-26T21:59:41.966 INFO:teuthology.orchestra.run.smithi164.stdout: "strays_created": 0, 2024-04-26T21:59:41.966 INFO:teuthology.orchestra.run.smithi164.stdout: "strays_enqueued": 0, 2024-04-26T21:59:41.966 INFO:teuthology.orchestra.run.smithi164.stdout: "strays_reintegrated": 0, 2024-04-26T21:59:41.966 INFO:teuthology.orchestra.run.smithi164.stdout: "strays_migrated": 0, 2024-04-26T21:59:41.966 INFO:teuthology.orchestra.run.smithi164.stdout: "num_recovering_processing": 0, 2024-04-26T21:59:41.966 INFO:teuthology.orchestra.run.smithi164.stdout: "num_recovering_enqueued": 0, 2024-04-26T21:59:41.966 INFO:teuthology.orchestra.run.smithi164.stdout: "num_recovering_prioritized": 0, 2024-04-26T21:59:41.966 INFO:teuthology.orchestra.run.smithi164.stdout: "recovery_started": 0, 2024-04-26T21:59:41.966 INFO:teuthology.orchestra.run.smithi164.stdout: "recovery_completed": 0, 2024-04-26T21:59:41.966 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_quiesce_path": 0, 2024-04-26T21:59:41.966 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_quiesce_inode": 0, 2024-04-26T21:59:41.966 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_enqueue_scrub": 0, 2024-04-26T21:59:41.967 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_exportdir": 0, 2024-04-26T21:59:41.967 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_flush": 0, 2024-04-26T21:59:41.967 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_fragmentdir": 0, 2024-04-26T21:59:41.967 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_fragstats": 0, 2024-04-26T21:59:41.967 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_inodestats": 0 2024-04-26T21:59:41.967 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:41.967 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_log": { 2024-04-26T21:59:41.967 INFO:teuthology.orchestra.run.smithi164.stdout: "evlrg": 0, 2024-04-26T21:59:41.967 INFO:teuthology.orchestra.run.smithi164.stdout: "evadd": 0, 2024-04-26T21:59:41.967 INFO:teuthology.orchestra.run.smithi164.stdout: "evex": 0, 2024-04-26T21:59:41.967 INFO:teuthology.orchestra.run.smithi164.stdout: "evtrm": 0, 2024-04-26T21:59:41.967 INFO:teuthology.orchestra.run.smithi164.stdout: "ev": 3, 2024-04-26T21:59:41.967 INFO:teuthology.orchestra.run.smithi164.stdout: "evexg": 0, 2024-04-26T21:59:41.967 INFO:teuthology.orchestra.run.smithi164.stdout: "evexd": 0, 2024-04-26T21:59:41.967 INFO:teuthology.orchestra.run.smithi164.stdout: "segadd": 0, 2024-04-26T21:59:41.968 INFO:teuthology.orchestra.run.smithi164.stdout: "segex": 0, 2024-04-26T21:59:41.968 INFO:teuthology.orchestra.run.smithi164.stdout: "segtrm": 0, 2024-04-26T21:59:41.968 INFO:teuthology.orchestra.run.smithi164.stdout: "seg": 2, 2024-04-26T21:59:41.968 INFO:teuthology.orchestra.run.smithi164.stdout: "segmjr": 2, 2024-04-26T21:59:41.968 INFO:teuthology.orchestra.run.smithi164.stdout: "segexg": 0, 2024-04-26T21:59:41.968 INFO:teuthology.orchestra.run.smithi164.stdout: "segexd": 0, 2024-04-26T21:59:41.968 INFO:teuthology.orchestra.run.smithi164.stdout: "expos": 4194304, 2024-04-26T21:59:41.968 INFO:teuthology.orchestra.run.smithi164.stdout: "wrpos": 4195385, 2024-04-26T21:59:41.968 INFO:teuthology.orchestra.run.smithi164.stdout: "rdpos": 4194892, 2024-04-26T21:59:41.968 INFO:teuthology.orchestra.run.smithi164.stdout: "jlat": { 2024-04-26T21:59:41.968 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-26T21:59:41.968 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-26T21:59:41.968 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-26T21:59:41.968 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:41.968 INFO:teuthology.orchestra.run.smithi164.stdout: "replayed": 3 2024-04-26T21:59:41.969 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:41.969 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_mem": { 2024-04-26T21:59:41.969 INFO:teuthology.orchestra.run.smithi164.stdout: "ino": 3, 2024-04-26T21:59:41.969 INFO:teuthology.orchestra.run.smithi164.stdout: "ino+": 3, 2024-04-26T21:59:41.969 INFO:teuthology.orchestra.run.smithi164.stdout: "ino-": 0, 2024-04-26T21:59:41.969 INFO:teuthology.orchestra.run.smithi164.stdout: "dir": 1, 2024-04-26T21:59:41.969 INFO:teuthology.orchestra.run.smithi164.stdout: "dir+": 1, 2024-04-26T21:59:41.969 INFO:teuthology.orchestra.run.smithi164.stdout: "dir-": 0, 2024-04-26T21:59:41.969 INFO:teuthology.orchestra.run.smithi164.stdout: "dn": 0, 2024-04-26T21:59:41.969 INFO:teuthology.orchestra.run.smithi164.stdout: "dn+": 0, 2024-04-26T21:59:41.969 INFO:teuthology.orchestra.run.smithi164.stdout: "dn-": 0, 2024-04-26T21:59:41.969 INFO:teuthology.orchestra.run.smithi164.stdout: "cap": 0, 2024-04-26T21:59:41.969 INFO:teuthology.orchestra.run.smithi164.stdout: "cap+": 0, 2024-04-26T21:59:41.969 INFO:teuthology.orchestra.run.smithi164.stdout: "cap-": 0, 2024-04-26T21:59:41.970 INFO:teuthology.orchestra.run.smithi164.stdout: "rss": 48128, 2024-04-26T21:59:41.970 INFO:teuthology.orchestra.run.smithi164.stdout: "heap": 215416 2024-04-26T21:59:41.970 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:41.970 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_server": { 2024-04-26T21:59:41.970 INFO:teuthology.orchestra.run.smithi164.stdout: "dispatch_client_request": 0, 2024-04-26T21:59:41.970 INFO:teuthology.orchestra.run.smithi164.stdout: "dispatch_server_request": 0, 2024-04-26T21:59:41.970 INFO:teuthology.orchestra.run.smithi164.stdout: "handle_client_request": 0, 2024-04-26T21:59:41.970 INFO:teuthology.orchestra.run.smithi164.stdout: "handle_client_session": 0, 2024-04-26T21:59:41.970 INFO:teuthology.orchestra.run.smithi164.stdout: "handle_peer_request": 0, 2024-04-26T21:59:41.970 INFO:teuthology.orchestra.run.smithi164.stdout: "req_create_latency": { 2024-04-26T21:59:41.970 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-26T21:59:41.970 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-26T21:59:41.970 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-26T21:59:41.970 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:41.970 INFO:teuthology.orchestra.run.smithi164.stdout: "req_getattr_latency": { 2024-04-26T21:59:41.971 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-26T21:59:41.971 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-26T21:59:41.971 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-26T21:59:41.971 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:41.971 INFO:teuthology.orchestra.run.smithi164.stdout: "req_getfilelock_latency": { 2024-04-26T21:59:41.971 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-26T21:59:41.971 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-26T21:59:41.971 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-26T21:59:41.971 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:41.971 INFO:teuthology.orchestra.run.smithi164.stdout: "req_link_latency": { 2024-04-26T21:59:41.971 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-26T21:59:41.971 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-26T21:59:41.971 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-26T21:59:41.971 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:41.971 INFO:teuthology.orchestra.run.smithi164.stdout: "req_lookup_latency": { 2024-04-26T21:59:41.972 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-26T21:59:41.972 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-26T21:59:41.972 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-26T21:59:41.972 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:41.972 INFO:teuthology.orchestra.run.smithi164.stdout: "req_lookuphash_latency": { 2024-04-26T21:59:41.972 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-26T21:59:41.972 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-26T21:59:41.972 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-26T21:59:41.972 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:41.972 INFO:teuthology.orchestra.run.smithi164.stdout: "req_lookupino_latency": { 2024-04-26T21:59:41.972 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-26T21:59:41.972 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-26T21:59:41.972 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-26T21:59:41.972 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:41.972 INFO:teuthology.orchestra.run.smithi164.stdout: "req_lookupname_latency": { 2024-04-26T21:59:41.973 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-26T21:59:41.973 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-26T21:59:41.973 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-26T21:59:41.973 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:41.973 INFO:teuthology.orchestra.run.smithi164.stdout: "req_lookupparent_latency": { 2024-04-26T21:59:41.973 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-26T21:59:41.973 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-26T21:59:41.973 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-26T21:59:41.973 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:41.973 INFO:teuthology.orchestra.run.smithi164.stdout: "req_lookupsnap_latency": { 2024-04-26T21:59:41.973 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-26T21:59:41.973 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-26T21:59:41.973 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-26T21:59:41.973 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:41.973 INFO:teuthology.orchestra.run.smithi164.stdout: "req_lssnap_latency": { 2024-04-26T21:59:41.974 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-26T21:59:41.974 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-26T21:59:41.974 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-26T21:59:41.974 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:41.974 INFO:teuthology.orchestra.run.smithi164.stdout: "req_mkdir_latency": { 2024-04-26T21:59:41.974 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-26T21:59:41.975 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-26T21:59:41.975 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-26T21:59:41.975 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:41.975 INFO:teuthology.orchestra.run.smithi164.stdout: "req_mknod_latency": { 2024-04-26T21:59:41.975 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-26T21:59:41.975 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-26T21:59:41.975 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-26T21:59:41.975 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:41.975 INFO:teuthology.orchestra.run.smithi164.stdout: "req_mksnap_latency": { 2024-04-26T21:59:41.975 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-26T21:59:41.975 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-26T21:59:41.975 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-26T21:59:41.975 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:41.976 INFO:teuthology.orchestra.run.smithi164.stdout: "req_open_latency": { 2024-04-26T21:59:41.976 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-26T21:59:41.976 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-26T21:59:41.976 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-26T21:59:41.976 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:41.976 INFO:teuthology.orchestra.run.smithi164.stdout: "req_readdir_latency": { 2024-04-26T21:59:41.976 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-26T21:59:41.976 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-26T21:59:41.976 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-26T21:59:41.976 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:41.976 INFO:teuthology.orchestra.run.smithi164.stdout: "req_rename_latency": { 2024-04-26T21:59:41.976 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-26T21:59:41.976 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-26T21:59:41.976 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-26T21:59:41.977 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:41.977 INFO:teuthology.orchestra.run.smithi164.stdout: "req_renamesnap_latency": { 2024-04-26T21:59:41.977 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-26T21:59:41.977 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-26T21:59:41.977 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-26T21:59:41.977 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:41.977 INFO:teuthology.orchestra.run.smithi164.stdout: "req_snapdiff_latency": { 2024-04-26T21:59:41.977 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-26T21:59:41.977 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-26T21:59:41.977 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-26T21:59:41.977 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:41.977 INFO:teuthology.orchestra.run.smithi164.stdout: "req_rmdir_latency": { 2024-04-26T21:59:41.977 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-26T21:59:41.977 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-26T21:59:41.978 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-26T21:59:41.978 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:41.978 INFO:teuthology.orchestra.run.smithi164.stdout: "req_rmsnap_latency": { 2024-04-26T21:59:41.978 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-26T21:59:41.978 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-26T21:59:41.978 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-26T21:59:41.978 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:41.978 INFO:teuthology.orchestra.run.smithi164.stdout: "req_rmxattr_latency": { 2024-04-26T21:59:41.978 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-26T21:59:41.978 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-26T21:59:41.978 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-26T21:59:41.978 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:41.978 INFO:teuthology.orchestra.run.smithi164.stdout: "req_setattr_latency": { 2024-04-26T21:59:41.978 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-26T21:59:41.979 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-26T21:59:41.979 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-26T21:59:41.979 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:41.979 INFO:teuthology.orchestra.run.smithi164.stdout: "req_setdirlayout_latency": { 2024-04-26T21:59:41.979 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-26T21:59:41.979 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-26T21:59:41.979 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-26T21:59:41.979 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:41.979 INFO:teuthology.orchestra.run.smithi164.stdout: "req_setfilelock_latency": { 2024-04-26T21:59:41.979 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-26T21:59:41.979 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-26T21:59:41.979 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-26T21:59:41.979 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:41.979 INFO:teuthology.orchestra.run.smithi164.stdout: "req_setlayout_latency": { 2024-04-26T21:59:41.979 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-26T21:59:41.980 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-26T21:59:41.980 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-26T21:59:41.980 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:41.980 INFO:teuthology.orchestra.run.smithi164.stdout: "req_setxattr_latency": { 2024-04-26T21:59:41.980 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-26T21:59:41.980 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-26T21:59:41.980 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-26T21:59:41.980 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:41.980 INFO:teuthology.orchestra.run.smithi164.stdout: "req_symlink_latency": { 2024-04-26T21:59:41.980 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-26T21:59:41.980 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-26T21:59:41.980 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-26T21:59:41.980 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:41.980 INFO:teuthology.orchestra.run.smithi164.stdout: "req_unlink_latency": { 2024-04-26T21:59:41.981 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-26T21:59:41.981 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-26T21:59:41.981 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-26T21:59:41.981 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:41.981 INFO:teuthology.orchestra.run.smithi164.stdout: "cap_revoke_eviction": 0, 2024-04-26T21:59:41.981 INFO:teuthology.orchestra.run.smithi164.stdout: "cap_acquisition_throttle": 0, 2024-04-26T21:59:41.981 INFO:teuthology.orchestra.run.smithi164.stdout: "req_getvxattr_latency": { 2024-04-26T21:59:41.981 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-26T21:59:41.981 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-26T21:59:41.981 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-26T21:59:41.981 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-04-26T21:59:41.981 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:41.981 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_sessions": { 2024-04-26T21:59:41.981 INFO:teuthology.orchestra.run.smithi164.stdout: "session_count": 1, 2024-04-26T21:59:41.982 INFO:teuthology.orchestra.run.smithi164.stdout: "session_add": 1, 2024-04-26T21:59:41.982 INFO:teuthology.orchestra.run.smithi164.stdout: "session_remove": 0, 2024-04-26T21:59:41.982 INFO:teuthology.orchestra.run.smithi164.stdout: "sessions_open": 1, 2024-04-26T21:59:41.982 INFO:teuthology.orchestra.run.smithi164.stdout: "sessions_stale": 0, 2024-04-26T21:59:41.982 INFO:teuthology.orchestra.run.smithi164.stdout: "total_load": 0, 2024-04-26T21:59:41.982 INFO:teuthology.orchestra.run.smithi164.stdout: "average_load": 0, 2024-04-26T21:59:41.982 INFO:teuthology.orchestra.run.smithi164.stdout: "avg_session_uptime": 1695, 2024-04-26T21:59:41.982 INFO:teuthology.orchestra.run.smithi164.stdout: "mdthresh_evicted": 0 2024-04-26T21:59:41.982 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:41.982 INFO:teuthology.orchestra.run.smithi164.stdout: "mempool": { 2024-04-26T21:59:41.982 INFO:teuthology.orchestra.run.smithi164.stdout: "bloom_filter_bytes": 0, 2024-04-26T21:59:41.982 INFO:teuthology.orchestra.run.smithi164.stdout: "bloom_filter_items": 0, 2024-04-26T21:59:41.983 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_alloc_bytes": 0, 2024-04-26T21:59:41.983 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_alloc_items": 0, 2024-04-26T21:59:41.983 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_data_bytes": 0, 2024-04-26T21:59:41.983 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_data_items": 0, 2024-04-26T21:59:41.983 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-26T21:59:41.983 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_onode_items": 0, 2024-04-26T21:59:41.983 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-26T21:59:41.983 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_meta_items": 0, 2024-04-26T21:59:41.983 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_other_bytes": 0, 2024-04-26T21:59:41.983 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_other_items": 0, 2024-04-26T21:59:41.983 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-26T21:59:41.984 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_buffer_items": 0, 2024-04-26T21:59:41.984 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_extent_bytes": 0, 2024-04-26T21:59:41.984 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_extent_items": 0, 2024-04-26T21:59:41.984 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_blob_bytes": 0, 2024-04-26T21:59:41.984 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_blob_items": 0, 2024-04-26T21:59:41.984 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-26T21:59:41.984 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_shared_blob_items": 0, 2024-04-26T21:59:41.984 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-26T21:59:41.984 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_inline_bl_items": 0, 2024-04-26T21:59:41.984 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_fsck_bytes": 0, 2024-04-26T21:59:41.984 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_fsck_items": 0, 2024-04-26T21:59:41.984 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_txc_bytes": 0, 2024-04-26T21:59:41.984 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_txc_items": 0, 2024-04-26T21:59:41.984 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-26T21:59:41.985 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_writing_deferred_items": 0, 2024-04-26T21:59:41.985 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_writing_bytes": 0, 2024-04-26T21:59:41.985 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_writing_items": 0, 2024-04-26T21:59:41.985 INFO:teuthology.orchestra.run.smithi164.stdout: "bluefs_bytes": 0, 2024-04-26T21:59:41.985 INFO:teuthology.orchestra.run.smithi164.stdout: "bluefs_items": 0, 2024-04-26T21:59:41.985 INFO:teuthology.orchestra.run.smithi164.stdout: "bluefs_file_reader_bytes": 0, 2024-04-26T21:59:41.985 INFO:teuthology.orchestra.run.smithi164.stdout: "bluefs_file_reader_items": 0, 2024-04-26T21:59:41.985 INFO:teuthology.orchestra.run.smithi164.stdout: "bluefs_file_writer_bytes": 0, 2024-04-26T21:59:41.985 INFO:teuthology.orchestra.run.smithi164.stdout: "bluefs_file_writer_items": 0, 2024-04-26T21:59:41.985 INFO:teuthology.orchestra.run.smithi164.stdout: "buffer_anon_bytes": 190023, 2024-04-26T21:59:41.985 INFO:teuthology.orchestra.run.smithi164.stdout: "buffer_anon_items": 50, 2024-04-26T21:59:41.985 INFO:teuthology.orchestra.run.smithi164.stdout: "buffer_meta_bytes": 0, 2024-04-26T21:59:41.985 INFO:teuthology.orchestra.run.smithi164.stdout: "buffer_meta_items": 0, 2024-04-26T21:59:41.985 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_bytes": 0, 2024-04-26T21:59:41.986 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_items": 0, 2024-04-26T21:59:41.986 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_mapbl_bytes": 0, 2024-04-26T21:59:41.986 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_mapbl_items": 0, 2024-04-26T21:59:41.986 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_pglog_bytes": 0, 2024-04-26T21:59:41.986 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_pglog_items": 0, 2024-04-26T21:59:41.986 INFO:teuthology.orchestra.run.smithi164.stdout: "osdmap_bytes": 8008, 2024-04-26T21:59:41.986 INFO:teuthology.orchestra.run.smithi164.stdout: "osdmap_items": 168, 2024-04-26T21:59:41.986 INFO:teuthology.orchestra.run.smithi164.stdout: "osdmap_mapping_bytes": 0, 2024-04-26T21:59:41.986 INFO:teuthology.orchestra.run.smithi164.stdout: "osdmap_mapping_items": 0, 2024-04-26T21:59:41.986 INFO:teuthology.orchestra.run.smithi164.stdout: "pgmap_bytes": 0, 2024-04-26T21:59:41.986 INFO:teuthology.orchestra.run.smithi164.stdout: "pgmap_items": 0, 2024-04-26T21:59:41.986 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_co_bytes": 8512, 2024-04-26T21:59:41.986 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_co_items": 13, 2024-04-26T21:59:41.987 INFO:teuthology.orchestra.run.smithi164.stdout: "unittest_1_bytes": 0, 2024-04-26T21:59:41.987 INFO:teuthology.orchestra.run.smithi164.stdout: "unittest_1_items": 0, 2024-04-26T21:59:41.987 INFO:teuthology.orchestra.run.smithi164.stdout: "unittest_2_bytes": 0, 2024-04-26T21:59:41.987 INFO:teuthology.orchestra.run.smithi164.stdout: "unittest_2_items": 0 2024-04-26T21:59:41.987 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:41.987 INFO:teuthology.orchestra.run.smithi164.stdout: "objecter": { 2024-04-26T21:59:41.987 INFO:teuthology.orchestra.run.smithi164.stdout: "op_active": 0, 2024-04-26T21:59:41.987 INFO:teuthology.orchestra.run.smithi164.stdout: "op_laggy": 0, 2024-04-26T21:59:41.987 INFO:teuthology.orchestra.run.smithi164.stdout: "op_send": 1074, 2024-04-26T21:59:41.987 INFO:teuthology.orchestra.run.smithi164.stdout: "op_send_bytes": 21, 2024-04-26T21:59:41.987 INFO:teuthology.orchestra.run.smithi164.stdout: "op_resend": 3, 2024-04-26T21:59:41.987 INFO:teuthology.orchestra.run.smithi164.stdout: "op_reply": 1074, 2024-04-26T21:59:41.988 INFO:teuthology.orchestra.run.smithi164.stdout: "op_latency": { 2024-04-26T21:59:41.988 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 1074, 2024-04-26T21:59:41.988 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 1.379976330, 2024-04-26T21:59:41.988 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.001284894 2024-04-26T21:59:41.988 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:41.988 INFO:teuthology.orchestra.run.smithi164.stdout: "op_inflight": 0, 2024-04-26T21:59:41.988 INFO:teuthology.orchestra.run.smithi164.stdout: "oplen_avg": { 2024-04-26T21:59:41.988 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 1074, 2024-04-26T21:59:41.988 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 1075 2024-04-26T21:59:41.988 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:41.988 INFO:teuthology.orchestra.run.smithi164.stdout: "op": 1074, 2024-04-26T21:59:41.988 INFO:teuthology.orchestra.run.smithi164.stdout: "op_r": 1074, 2024-04-26T21:59:41.988 INFO:teuthology.orchestra.run.smithi164.stdout: "op_w": 0, 2024-04-26T21:59:41.988 INFO:teuthology.orchestra.run.smithi164.stdout: "op_rmw": 0, 2024-04-26T21:59:41.989 INFO:teuthology.orchestra.run.smithi164.stdout: "op_pg": 0, 2024-04-26T21:59:41.989 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_stat": 664, 2024-04-26T21:59:41.989 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_create": 0, 2024-04-26T21:59:41.989 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_read": 408, 2024-04-26T21:59:41.989 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_write": 0, 2024-04-26T21:59:41.989 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_writefull": 0, 2024-04-26T21:59:41.989 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_writesame": 0, 2024-04-26T21:59:41.989 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_append": 0, 2024-04-26T21:59:41.989 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_zero": 0, 2024-04-26T21:59:41.989 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_truncate": 0, 2024-04-26T21:59:41.989 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_delete": 0, 2024-04-26T21:59:41.989 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_mapext": 0, 2024-04-26T21:59:41.989 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_sparse_read": 0, 2024-04-26T21:59:41.989 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_clonerange": 0, 2024-04-26T21:59:41.990 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_getxattr": 1, 2024-04-26T21:59:41.990 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_setxattr": 0, 2024-04-26T21:59:41.990 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_cmpxattr": 0, 2024-04-26T21:59:41.990 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_rmxattr": 0, 2024-04-26T21:59:41.990 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_resetxattrs": 0, 2024-04-26T21:59:41.990 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_call": 0, 2024-04-26T21:59:41.990 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_watch": 0, 2024-04-26T21:59:41.990 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_notify": 0, 2024-04-26T21:59:41.990 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_src_cmpxattr": 0, 2024-04-26T21:59:41.990 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_pgls": 0, 2024-04-26T21:59:41.990 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_pgls_filter": 0, 2024-04-26T21:59:41.990 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_other": 0, 2024-04-26T21:59:41.990 INFO:teuthology.orchestra.run.smithi164.stdout: "linger_active": 0, 2024-04-26T21:59:41.991 INFO:teuthology.orchestra.run.smithi164.stdout: "linger_send": 0, 2024-04-26T21:59:41.991 INFO:teuthology.orchestra.run.smithi164.stdout: "linger_resend": 0, 2024-04-26T21:59:41.991 INFO:teuthology.orchestra.run.smithi164.stdout: "linger_ping": 0, 2024-04-26T21:59:41.991 INFO:teuthology.orchestra.run.smithi164.stdout: "poolop_active": 0, 2024-04-26T21:59:41.991 INFO:teuthology.orchestra.run.smithi164.stdout: "poolop_send": 0, 2024-04-26T21:59:41.991 INFO:teuthology.orchestra.run.smithi164.stdout: "poolop_resend": 0, 2024-04-26T21:59:41.991 INFO:teuthology.orchestra.run.smithi164.stdout: "poolstat_active": 0, 2024-04-26T21:59:41.991 INFO:teuthology.orchestra.run.smithi164.stdout: "poolstat_send": 0, 2024-04-26T21:59:41.991 INFO:teuthology.orchestra.run.smithi164.stdout: "poolstat_resend": 0, 2024-04-26T21:59:41.991 INFO:teuthology.orchestra.run.smithi164.stdout: "statfs_active": 0, 2024-04-26T21:59:41.991 INFO:teuthology.orchestra.run.smithi164.stdout: "statfs_send": 0, 2024-04-26T21:59:41.991 INFO:teuthology.orchestra.run.smithi164.stdout: "statfs_resend": 0, 2024-04-26T21:59:41.991 INFO:teuthology.orchestra.run.smithi164.stdout: "command_active": 0, 2024-04-26T21:59:41.991 INFO:teuthology.orchestra.run.smithi164.stdout: "command_send": 0, 2024-04-26T21:59:41.991 INFO:teuthology.orchestra.run.smithi164.stdout: "command_resend": 0, 2024-04-26T21:59:41.992 INFO:teuthology.orchestra.run.smithi164.stdout: "map_epoch": 91, 2024-04-26T21:59:41.992 INFO:teuthology.orchestra.run.smithi164.stdout: "map_full": 0, 2024-04-26T21:59:41.992 INFO:teuthology.orchestra.run.smithi164.stdout: "map_inc": 4, 2024-04-26T21:59:41.992 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_sessions": 5, 2024-04-26T21:59:41.992 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_session_open": 5, 2024-04-26T21:59:41.992 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_session_close": 0, 2024-04-26T21:59:41.992 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_laggy": 0, 2024-04-26T21:59:41.992 INFO:teuthology.orchestra.run.smithi164.stdout: "omap_wr": 0, 2024-04-26T21:59:41.992 INFO:teuthology.orchestra.run.smithi164.stdout: "omap_rd": 2, 2024-04-26T21:59:41.992 INFO:teuthology.orchestra.run.smithi164.stdout: "omap_del": 0 2024-04-26T21:59:41.992 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:41.992 INFO:teuthology.orchestra.run.smithi164.stdout: "oft": { 2024-04-26T21:59:41.992 INFO:teuthology.orchestra.run.smithi164.stdout: "omap_total_objs": 0, 2024-04-26T21:59:41.992 INFO:teuthology.orchestra.run.smithi164.stdout: "omap_total_kv_pairs": 0, 2024-04-26T21:59:41.993 INFO:teuthology.orchestra.run.smithi164.stdout: "omap_total_updates": 0, 2024-04-26T21:59:41.993 INFO:teuthology.orchestra.run.smithi164.stdout: "omap_total_removes": 0 2024-04-26T21:59:41.993 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:41.993 INFO:teuthology.orchestra.run.smithi164.stdout: "osd-slow-ops": { 2024-04-26T21:59:41.993 INFO:teuthology.orchestra.run.smithi164.stdout: "slow_ops_count": 0 2024-04-26T21:59:41.993 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:41.993 INFO:teuthology.orchestra.run.smithi164.stdout: "purge_queue": { 2024-04-26T21:59:41.993 INFO:teuthology.orchestra.run.smithi164.stdout: "pq_executing_ops": 0, 2024-04-26T21:59:41.993 INFO:teuthology.orchestra.run.smithi164.stdout: "pq_executing_ops_high_water": 0, 2024-04-26T21:59:41.993 INFO:teuthology.orchestra.run.smithi164.stdout: "pq_executing": 0, 2024-04-26T21:59:41.993 INFO:teuthology.orchestra.run.smithi164.stdout: "pq_executing_high_water": 0, 2024-04-26T21:59:41.993 INFO:teuthology.orchestra.run.smithi164.stdout: "pq_executed_ops": 0, 2024-04-26T21:59:41.993 INFO:teuthology.orchestra.run.smithi164.stdout: "pq_executed": 0, 2024-04-26T21:59:41.993 INFO:teuthology.orchestra.run.smithi164.stdout: "pq_item_in_journal": 0 2024-04-26T21:59:41.994 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:41.994 INFO:teuthology.orchestra.run.smithi164.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-26T21:59:41.994 INFO:teuthology.orchestra.run.smithi164.stdout: "val": 0, 2024-04-26T21:59:41.994 INFO:teuthology.orchestra.run.smithi164.stdout: "max": 104857600, 2024-04-26T21:59:41.994 INFO:teuthology.orchestra.run.smithi164.stdout: "get_started": 0, 2024-04-26T21:59:41.994 INFO:teuthology.orchestra.run.smithi164.stdout: "get": 1213, 2024-04-26T21:59:41.994 INFO:teuthology.orchestra.run.smithi164.stdout: "get_sum": 532713, 2024-04-26T21:59:41.994 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_fail": 0, 2024-04-26T21:59:41.994 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_success": 1213, 2024-04-26T21:59:41.994 INFO:teuthology.orchestra.run.smithi164.stdout: "take": 0, 2024-04-26T21:59:41.994 INFO:teuthology.orchestra.run.smithi164.stdout: "take_sum": 0, 2024-04-26T21:59:41.994 INFO:teuthology.orchestra.run.smithi164.stdout: "put": 1213, 2024-04-26T21:59:41.994 INFO:teuthology.orchestra.run.smithi164.stdout: "put_sum": 532713, 2024-04-26T21:59:41.994 INFO:teuthology.orchestra.run.smithi164.stdout: "wait": { 2024-04-26T21:59:41.995 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-26T21:59:41.995 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-26T21:59:41.995 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-26T21:59:41.995 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-04-26T21:59:41.995 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:41.995 INFO:teuthology.orchestra.run.smithi164.stdout: "throttle-objecter_bytes": { 2024-04-26T21:59:41.995 INFO:teuthology.orchestra.run.smithi164.stdout: "val": 0, 2024-04-26T21:59:41.995 INFO:teuthology.orchestra.run.smithi164.stdout: "max": 104857600, 2024-04-26T21:59:41.995 INFO:teuthology.orchestra.run.smithi164.stdout: "get_started": 0, 2024-04-26T21:59:41.995 INFO:teuthology.orchestra.run.smithi164.stdout: "get": 0, 2024-04-26T21:59:41.995 INFO:teuthology.orchestra.run.smithi164.stdout: "get_sum": 0, 2024-04-26T21:59:41.995 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_fail": 0, 2024-04-26T21:59:41.995 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_success": 0, 2024-04-26T21:59:41.995 INFO:teuthology.orchestra.run.smithi164.stdout: "take": 1074, 2024-04-26T21:59:41.995 INFO:teuthology.orchestra.run.smithi164.stdout: "take_sum": 1086, 2024-04-26T21:59:41.996 INFO:teuthology.orchestra.run.smithi164.stdout: "put": 1074, 2024-04-26T21:59:41.996 INFO:teuthology.orchestra.run.smithi164.stdout: "put_sum": 1086, 2024-04-26T21:59:41.996 INFO:teuthology.orchestra.run.smithi164.stdout: "wait": { 2024-04-26T21:59:41.996 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-26T21:59:41.996 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-26T21:59:41.996 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-26T21:59:41.996 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-04-26T21:59:41.996 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:41.996 INFO:teuthology.orchestra.run.smithi164.stdout: "throttle-objecter_ops": { 2024-04-26T21:59:41.996 INFO:teuthology.orchestra.run.smithi164.stdout: "val": 0, 2024-04-26T21:59:41.996 INFO:teuthology.orchestra.run.smithi164.stdout: "max": 1024, 2024-04-26T21:59:41.996 INFO:teuthology.orchestra.run.smithi164.stdout: "get_started": 0, 2024-04-26T21:59:41.996 INFO:teuthology.orchestra.run.smithi164.stdout: "get": 0, 2024-04-26T21:59:41.996 INFO:teuthology.orchestra.run.smithi164.stdout: "get_sum": 0, 2024-04-26T21:59:41.997 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_fail": 0, 2024-04-26T21:59:41.997 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_success": 0, 2024-04-26T21:59:41.997 INFO:teuthology.orchestra.run.smithi164.stdout: "take": 1074, 2024-04-26T21:59:41.997 INFO:teuthology.orchestra.run.smithi164.stdout: "take_sum": 1074, 2024-04-26T21:59:41.997 INFO:teuthology.orchestra.run.smithi164.stdout: "put": 1074, 2024-04-26T21:59:41.997 INFO:teuthology.orchestra.run.smithi164.stdout: "put_sum": 1074, 2024-04-26T21:59:41.997 INFO:teuthology.orchestra.run.smithi164.stdout: "wait": { 2024-04-26T21:59:41.997 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-26T21:59:41.997 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-26T21:59:41.997 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-26T21:59:41.997 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-04-26T21:59:41.997 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:41.997 INFO:teuthology.orchestra.run.smithi164.stdout: "throttle-write_buf_throttle": { 2024-04-26T21:59:41.997 INFO:teuthology.orchestra.run.smithi164.stdout: "val": 0, 2024-04-26T21:59:41.997 INFO:teuthology.orchestra.run.smithi164.stdout: "max": 3758096384, 2024-04-26T21:59:41.998 INFO:teuthology.orchestra.run.smithi164.stdout: "get_started": 0, 2024-04-26T21:59:41.998 INFO:teuthology.orchestra.run.smithi164.stdout: "get": 0, 2024-04-26T21:59:41.998 INFO:teuthology.orchestra.run.smithi164.stdout: "get_sum": 0, 2024-04-26T21:59:41.998 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_fail": 0, 2024-04-26T21:59:41.998 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_success": 0, 2024-04-26T21:59:41.998 INFO:teuthology.orchestra.run.smithi164.stdout: "take": 0, 2024-04-26T21:59:41.998 INFO:teuthology.orchestra.run.smithi164.stdout: "take_sum": 0, 2024-04-26T21:59:41.998 INFO:teuthology.orchestra.run.smithi164.stdout: "put": 0, 2024-04-26T21:59:41.998 INFO:teuthology.orchestra.run.smithi164.stdout: "put_sum": 0, 2024-04-26T21:59:41.998 INFO:teuthology.orchestra.run.smithi164.stdout: "wait": { 2024-04-26T21:59:41.998 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-26T21:59:41.998 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-26T21:59:41.998 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-26T21:59:41.998 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-04-26T21:59:41.998 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T21:59:41.999 INFO:teuthology.orchestra.run.smithi164.stdout: "throttle-write_buf_throttle-0x561b0a7be780": { 2024-04-26T21:59:41.999 INFO:teuthology.orchestra.run.smithi164.stdout: "val": 0, 2024-04-26T21:59:41.999 INFO:teuthology.orchestra.run.smithi164.stdout: "max": 3758096384, 2024-04-26T21:59:41.999 INFO:teuthology.orchestra.run.smithi164.stdout: "get_started": 0, 2024-04-26T21:59:41.999 INFO:teuthology.orchestra.run.smithi164.stdout: "get": 0, 2024-04-26T21:59:41.999 INFO:teuthology.orchestra.run.smithi164.stdout: "get_sum": 0, 2024-04-26T21:59:41.999 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_fail": 0, 2024-04-26T21:59:41.999 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_success": 0, 2024-04-26T21:59:41.999 INFO:teuthology.orchestra.run.smithi164.stdout: "take": 0, 2024-04-26T21:59:41.999 INFO:teuthology.orchestra.run.smithi164.stdout: "take_sum": 0, 2024-04-26T21:59:41.999 INFO:teuthology.orchestra.run.smithi164.stdout: "put": 0, 2024-04-26T21:59:41.999 INFO:teuthology.orchestra.run.smithi164.stdout: "put_sum": 0, 2024-04-26T21:59:41.999 INFO:teuthology.orchestra.run.smithi164.stdout: "wait": { 2024-04-26T21:59:41.999 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-04-26T21:59:41.999 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-04-26T21:59:42.000 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-04-26T21:59:42.000 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-04-26T21:59:42.000 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-04-26T21:59:42.000 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-04-26T21:59:42.894 INFO:tasks.check_counter:Daemon mds.l mds.exported=0 2024-04-26T21:59:42.895 INFO:tasks.check_counter:Daemon mds.l mds.imported=0 2024-04-26T21:59:42.895 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-04-26T21:59:42.911 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-04-26T21:59:42.912 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2024-04-26T21:59:42.912 DEBUG:teuthology.orchestra.run.smithi072:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:59:42.970 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2024-04-26T21:59:42.970 DEBUG:teuthology.orchestra.run.smithi072:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:59:43.022 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:59:43.064 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:59:43 smithi076 ceph-mon[31823]: pgmap v577: 97 pgs: 97 active+clean; 1.1 MiB data, 656 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:59:43.190 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:59:43 smithi164 ceph-mon[31916]: pgmap v577: 97 pgs: 97 active+clean; 1.1 MiB data, 656 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:59:43.231 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:43.230+0000 7fe5ff200640 1 -- 172.21.15.72:0/348009766 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe5f8078530 msgr2=0x7fe5f8078990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:43.231 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:43.230+0000 7fe5ff200640 1 --2- 172.21.15.72:0/348009766 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe5f8078530 0x7fe5f8078990 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fe5e8009900 tx=0x7fe5e802a3f0 comp rx=0 tx=0).stop 2024-04-26T21:59:43.231 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:43.230+0000 7fe5ff200640 1 -- 172.21.15.72:0/348009766 shutdown_connections 2024-04-26T21:59:43.232 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:43.231+0000 7fe5ff200640 1 --2- 172.21.15.72:0/348009766 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe5f8078ed0 0x7fe5f8079770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:43.232 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:43.231+0000 7fe5ff200640 1 --2- 172.21.15.72:0/348009766 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe5f8078530 0x7fe5f8078990 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:43.232 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:43.231+0000 7fe5ff200640 1 --2- 172.21.15.72:0/348009766 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe5f80772f0 0x7fe5f8077710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:43.232 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:43.231+0000 7fe5ff200640 1 -- 172.21.15.72:0/348009766 >> 172.21.15.72:0/348009766 conn(0x7fe5f80ffeb0 msgr2=0x7fe5f8102310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:43.232 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:43.231+0000 7fe5ff200640 1 -- 172.21.15.72:0/348009766 shutdown_connections 2024-04-26T21:59:43.232 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:43.231+0000 7fe5ff200640 1 -- 172.21.15.72:0/348009766 wait complete. 2024-04-26T21:59:43.234 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:43.232+0000 7fe5ff200640 1 Processor -- start 2024-04-26T21:59:43.234 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:43.232+0000 7fe5ff200640 1 -- start start 2024-04-26T21:59:43.234 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:43.232+0000 7fe5ff200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe5f80772f0 0x7fe5f819eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:43.234 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:43.232+0000 7fe5ff200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe5f8078530 0x7fe5f819f430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:43.235 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:43.233+0000 7fe5ff200640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe5f8078ed0 0x7fe5f81a64a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:43.235 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:43.233+0000 7fe5ff200640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fe5f8114750 con 0x7fe5f8078ed0 2024-04-26T21:59:43.235 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:43.233+0000 7fe5ff200640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fe5f81145d0 con 0x7fe5f80772f0 2024-04-26T21:59:43.235 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:43.233+0000 7fe5ff200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe5f81148d0 con 0x7fe5f8078530 2024-04-26T21:59:43.235 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:43.233+0000 7fe5fe800640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe5f8078ed0 0x7fe5f81a64a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:43.235 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:43.233+0000 7fe5fe800640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe5f8078ed0 0x7fe5f81a64a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:36890/0 (socket says 172.21.15.72:36890) 2024-04-26T21:59:43.235 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:43.233+0000 7fe5fe800640 1 -- 172.21.15.72:0/3457111094 learned_addr learned my addr 172.21.15.72:0/3457111094 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:59:43.235 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:43.233+0000 7fe5fde00640 1 --2- 172.21.15.72:0/3457111094 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe5f80772f0 0x7fe5f819eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:43.235 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:43.233+0000 7fe5fd400640 1 --2- 172.21.15.72:0/3457111094 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe5f8078530 0x7fe5f819f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:43.235 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:43.233+0000 7fe5fe800640 1 -- 172.21.15.72:0/3457111094 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe5f8078530 msgr2=0x7fe5f819f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:43.235 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:43.233+0000 7fe5fe800640 1 --2- 172.21.15.72:0/3457111094 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe5f8078530 0x7fe5f819f430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:43.235 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:43.233+0000 7fe5fe800640 1 -- 172.21.15.72:0/3457111094 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe5f80772f0 msgr2=0x7fe5f819eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:43.235 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:43.233+0000 7fe5fe800640 1 --2- 172.21.15.72:0/3457111094 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe5f80772f0 0x7fe5f819eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:43.235 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:43.233+0000 7fe5fe800640 1 -- 172.21.15.72:0/3457111094 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5f81a6b10 con 0x7fe5f8078ed0 2024-04-26T21:59:43.236 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:43.234+0000 7fe5fe800640 1 --2- 172.21.15.72:0/3457111094 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe5f8078ed0 0x7fe5f81a64a0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fe5f400eaa0 tx=0x7fe5f400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:43.236 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:43.234+0000 7fe5f2a00640 1 -- 172.21.15.72:0/3457111094 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5f400cdb0 con 0x7fe5f8078ed0 2024-04-26T21:59:43.236 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:43.234+0000 7fe5f2a00640 1 -- 172.21.15.72:0/3457111094 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe5f4005020 con 0x7fe5f8078ed0 2024-04-26T21:59:43.236 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:43.234+0000 7fe5f2a00640 1 -- 172.21.15.72:0/3457111094 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5f40116d0 con 0x7fe5f8078ed0 2024-04-26T21:59:43.236 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:43.234+0000 7fe5ff200640 1 -- 172.21.15.72:0/3457111094 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5f81a6e00 con 0x7fe5f8078ed0 2024-04-26T21:59:43.236 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:43.234+0000 7fe5ff200640 1 -- 172.21.15.72:0/3457111094 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5f81a7270 con 0x7fe5f8078ed0 2024-04-26T21:59:43.236 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:43.235+0000 7fe5ff200640 1 -- 172.21.15.72:0/3457111094 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5f81112d0 con 0x7fe5f8078ed0 2024-04-26T21:59:43.237 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:43.237+0000 7fe5f2a00640 1 -- 172.21.15.72:0/3457111094 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fe5f40051d0 con 0x7fe5f8078ed0 2024-04-26T21:59:43.239 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:43.239+0000 7fe5f2a00640 1 --2- 172.21.15.72:0/3457111094 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fe5e00a3cc0 0x7fe5e00a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:43.239 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:43.239+0000 7fe5f2a00640 1 -- 172.21.15.72:0/3457111094 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7fe5f40d0840 con 0x7fe5f8078ed0 2024-04-26T21:59:43.239 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:43.239+0000 7fe5fde00640 1 --2- 172.21.15.72:0/3457111094 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fe5e00a3cc0 0x7fe5e00a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:43.240 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:43.240+0000 7fe5fde00640 1 --2- 172.21.15.72:0/3457111094 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fe5e00a3cc0 0x7fe5e00a6180 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fe5ec005fd0 tx=0x7fe5ec005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:43.243 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:43.244+0000 7fe5f2a00640 1 -- 172.21.15.72:0/3457111094 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fe5f4016030 con 0x7fe5f8078ed0 2024-04-26T21:59:43.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:59:43 smithi072 ceph-mon[28138]: pgmap v577: 97 pgs: 97 active+clean; 1.1 MiB data, 656 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T21:59:43.448 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:43.448+0000 7fe5ff200640 1 -- 172.21.15.72:0/3457111094 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe5f802dcf0 con 0x7fe5f8078ed0 2024-04-26T21:59:43.452 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:43.452+0000 7fe5f2a00640 1 -- 172.21.15.72:0/3457111094 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+22783 (secure 0 0 0) 0x7fe5f409cfe0 con 0x7fe5f8078ed0 2024-04-26T21:59:43.453 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T21:59:43.453 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":91,"fsid":"a1780950-0415-11ef-bc93-c7b262605968","created":"2024-04-26T21:42:01.504173+0000","modified":"2024-04-26T21:54:11.892872+0000","last_up_change":"2024-04-26T21:51:43.482437+0000","last_in_change":"2024-04-26T21:51:18.468587+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:46:29.785427+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:52:48.320391+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T21:52:49.517493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"04e06884-b355-4762-9b29-9034b7201e9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6802","nonce":1416765179},{"type":"v1","addr":"172.21.15.72:6803","nonce":1416765179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6804","nonce":1416765179},{"type":"v1","addr":"172.21.15.72:6805","nonce":1416765179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6808","nonce":1416765179},{"type":"v1","addr":"172.21.15.72:6809","nonce":1416765179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6806","nonce":1416765179},{"type":"v1","addr":"172.21.15.72:6807","nonce":1416765179}]},"public_addr":"172.21.15.72:6803/1416765179","cluster_addr":"172.21.15.72:6805/1416765179","heartbeat_back_addr":"172.21.15.72:6809/1416765179","heartbeat_front_addr":"172.21.15.72:6807/1416765179","state":["exists","up"]},{"osd":1,"uuid":"d75a479a-4a2e-48e9-a01e-2b1c9617416a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":755426949},{"type":"v1","addr":"172.21.15.76:6801","nonce":755426949}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":755426949},{"type":"v1","addr":"172.21.15.76:6803","nonce":755426949}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":755426949},{"type":"v1","addr":"172.21.15.76:6807","nonce":755426949}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":755426949},{"type":"v1","addr":"172.21.15.76:6805","nonce":755426949}]},"public_addr":"172.21.15.76:6801/755426949","cluster_addr":"172.21.15.76:6803/755426949","heartbeat_back_addr":"172.21.15.76:6807/755426949","heartbeat_front_addr":"172.21.15.76:6805/755426949","state":["exists","up"]},{"osd":2,"uuid":"b6f8da58-4428-4e8c-a944-7ea5645d6d92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":852265973},{"type":"v1","addr":"172.21.15.164:6801","nonce":852265973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":852265973},{"type":"v1","addr":"172.21.15.164:6803","nonce":852265973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":852265973},{"type":"v1","addr":"172.21.15.164:6807","nonce":852265973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":852265973},{"type":"v1","addr":"172.21.15.164:6805","nonce":852265973}]},"public_addr":"172.21.15.164:6801/852265973","cluster_addr":"172.21.15.164:6803/852265973","heartbeat_back_addr":"172.21.15.164:6807/852265973","heartbeat_front_addr":"172.21.15.164:6805/852265973","state":["exists","up"]},{"osd":3,"uuid":"76d83409-649e-41e5-b4f4-43bbd5b00ea5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6810","nonce":4027763789},{"type":"v1","addr":"172.21.15.72:6811","nonce":4027763789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6812","nonce":4027763789},{"type":"v1","addr":"172.21.15.72:6813","nonce":4027763789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6816","nonce":4027763789},{"type":"v1","addr":"172.21.15.72:6817","nonce":4027763789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6814","nonce":4027763789},{"type":"v1","addr":"172.21.15.72:6815","nonce":4027763789}]},"public_addr":"172.21.15.72:6811/4027763789","cluster_addr":"172.21.15.72:6813/4027763789","heartbeat_back_addr":"172.21.15.72:6817/4027763789","heartbeat_front_addr":"172.21.15.72:6815/4027763789","state":["exists","up"]},{"osd":4,"uuid":"d2cfeea7-1661-4156-bdcd-f770a8d1a878","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":3718867827},{"type":"v1","addr":"172.21.15.76:6809","nonce":3718867827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":3718867827},{"type":"v1","addr":"172.21.15.76:6811","nonce":3718867827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":3718867827},{"type":"v1","addr":"172.21.15.76:6815","nonce":3718867827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":3718867827},{"type":"v1","addr":"172.21.15.76:6813","nonce":3718867827}]},"public_addr":"172.21.15.76:6809/3718867827","cluster_addr":"172.21.15.76:6811/3718867827","heartbeat_back_addr":"172.21.15.76:6815/3718867827","heartbeat_front_addr":"172.21.15.76:6813/3718867827","state":["exists","up"]},{"osd":5,"uuid":"f3896ad6-a040-4ea3-a431-ff9b7fc05c36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":2543954772},{"type":"v1","addr":"172.21.15.164:6809","nonce":2543954772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":2543954772},{"type":"v1","addr":"172.21.15.164:6811","nonce":2543954772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":2543954772},{"type":"v1","addr":"172.21.15.164:6815","nonce":2543954772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":2543954772},{"type":"v1","addr":"172.21.15.164:6813","nonce":2543954772}]},"public_addr":"172.21.15.164:6809/2543954772","cluster_addr":"172.21.15.164:6811/2543954772","heartbeat_back_addr":"172.21.15.164:6815/2543954772","heartbeat_front_addr":"172.21.15.164:6813/2543954772","state":["exists","up"]},{"osd":6,"uuid":"d5d56650-be7e-4ba5-ba3a-d7d473294dab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6818","nonce":2080659027},{"type":"v1","addr":"172.21.15.72:6819","nonce":2080659027}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6820","nonce":2080659027},{"type":"v1","addr":"172.21.15.72:6821","nonce":2080659027}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6824","nonce":2080659027},{"type":"v1","addr":"172.21.15.72:6825","nonce":2080659027}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6822","nonce":2080659027},{"type":"v1","addr":"172.21.15.72:6823","nonce":2080659027}]},"public_addr":"172.21.15.72:6819/2080659027","cluster_addr":"172.21.15.72:6821/2080659027","heartbeat_back_addr":"172.21.15.72:6825/2080659027","heartbeat_front_addr":"172.21.15.72:6823/2080659027","state":["exists","up"]},{"osd":7,"uuid":"18975b5a-9309-4a0f-9f37-3ce09c2e745d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":3144631323},{"type":"v1","addr":"172.21.15.76:6817","nonce":3144631323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":3144631323},{"type":"v1","addr":"172.21.15.76:6819","nonce":3144631323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":3144631323},{"type":"v1","addr":"172.21.15.76:6823","nonce":3144631323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":3144631323},{"type":"v1","addr":"172.21.15.76:6821","nonce":3144631323}]},"public_addr":"172.21.15.76:6817/3144631323","cluster_addr":"172.21.15.76:6819/3144631323","heartbeat_back_addr":"172.21.15.76:6823/3144631323","heartbeat_front_addr":"172.21.15.76:6821/3144631323","state":["exists","up"]},{"osd":8,"uuid":"779e9c1e-8e97-48d4-a854-68ca13f38e0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":627888883},{"type":"v1","addr":"172.21.15.164:6817","nonce":627888883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":627888883},{"type":"v1","addr":"172.21.15.164:6819","nonce":627888883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":627888883},{"type":"v1","addr":"172.21.15.164:6823","nonce":627888883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":627888883},{"type":"v1","addr":"172.21.15.164:6821","nonce":627888883}]},"public_addr":"172.21.15.164:6817/627888883","cluster_addr":"172.21.15.164:6819/627888883","heartbeat_back_addr":"172.21.15.164:6823/627888883","heartbeat_front_addr":"172.21.15.164:6821/627888883","state":["exists","up"]},{"osd":9,"uuid":"d0317850-7b2d-4850-b2cf-0b919505c1fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6826","nonce":1273118485},{"type":"v1","addr":"172.21.15.72:6827","nonce":1273118485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6828","nonce":1273118485},{"type":"v1","addr":"172.21.15.72:6829","nonce":1273118485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6832","nonce":1273118485},{"type":"v1","addr":"172.21.15.72:6833","nonce":1273118485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6830","nonce":1273118485},{"type":"v1","addr":"172.21.15.72:6831","nonce":1273118485}]},"public_addr":"172.21.15.72:6827/1273118485","cluster_addr":"172.21.15.72:6829/1273118485","heartbeat_back_addr":"172.21.15.72:6833/1273118485","heartbeat_front_addr":"172.21.15.72:6831/1273118485","state":["exists","up"]},{"osd":10,"uuid":"e0aa38d8-6011-4d2a-b67b-5bff6d999e47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":2808954459},{"type":"v1","addr":"172.21.15.76:6825","nonce":2808954459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":2808954459},{"type":"v1","addr":"172.21.15.76:6827","nonce":2808954459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":2808954459},{"type":"v1","addr":"172.21.15.76:6831","nonce":2808954459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":2808954459},{"type":"v1","addr":"172.21.15.76:6829","nonce":2808954459}]},"public_addr":"172.21.15.76:6825/2808954459","cluster_addr":"172.21.15.76:6827/2808954459","heartbeat_back_addr":"172.21.15.76:6831/2808954459","heartbeat_front_addr":"172.21.15.76:6829/2808954459","state":["exists","up"]},{"osd":11,"uuid":"981b3d4d-916e-4937-9916-7cc21a891e43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":2354951784},{"type":"v1","addr":"172.21.15.164:6825","nonce":2354951784}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":2354951784},{"type":"v1","addr":"172.21.15.164:6827","nonce":2354951784}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":2354951784},{"type":"v1","addr":"172.21.15.164:6831","nonce":2354951784}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":2354951784},{"type":"v1","addr":"172.21.15.164:6829","nonce":2354951784}]},"public_addr":"172.21.15.164:6825/2354951784","cluster_addr":"172.21.15.164:6827/2354951784","heartbeat_back_addr":"172.21.15.164:6831/2354951784","heartbeat_front_addr":"172.21.15.164:6829/2354951784","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:18.997924+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:52.439081+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:46:25.893288+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:47:00.112807+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:47:35.116874+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:48:09.216344+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:48:45.028853+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:49:19.905397+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:49:55.030536+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:50:30.767662+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:51:06.423267+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:51:41.631211+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.1a","mappings":[{"from":5,"to":7}]},{"pgid":"3.1b","mappings":[{"from":4,"to":6}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.28","mappings":[{"from":5,"to":6}]},{"pgid":"3.2a","mappings":[{"from":3,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.72:0/416277784":"2024-04-27T21:43:09.602791+0000","172.21.15.72:6800/4010358672":"2024-04-27T21:43:09.602791+0000","172.21.15.72:6801/1012617371":"2024-04-27T21:42:34.684837+0000","172.21.15.72:6801/4010358672":"2024-04-27T21:43:09.602791+0000","172.21.15.72:0/3555448179":"2024-04-27T21:43:09.602791+0000","172.21.15.72:6800/1012617371":"2024-04-27T21:42:34.684837+0000","172.21.15.72:0/2920378761":"2024-04-27T21:42:34.684837+0000","172.21.15.72:0/1933740041":"2024-04-27T21:43:09.602791+0000","172.21.15.72:0/3225295775":"2024-04-27T21:42:34.684837+0000","172.21.15.72:0/2326016180":"2024-04-27T21:42:34.684837+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:59:43.455 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:43.455+0000 7fe5ff200640 1 -- 172.21.15.72:0/3457111094 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fe5e00a3cc0 msgr2=0x7fe5e00a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:43.455 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:43.455+0000 7fe5ff200640 1 --2- 172.21.15.72:0/3457111094 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fe5e00a3cc0 0x7fe5e00a6180 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fe5ec005fd0 tx=0x7fe5ec005850 comp rx=0 tx=0).stop 2024-04-26T21:59:43.455 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:43.455+0000 7fe5ff200640 1 -- 172.21.15.72:0/3457111094 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe5f8078ed0 msgr2=0x7fe5f81a64a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:43.455 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:43.455+0000 7fe5ff200640 1 --2- 172.21.15.72:0/3457111094 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe5f8078ed0 0x7fe5f81a64a0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fe5f400eaa0 tx=0x7fe5f400ef60 comp rx=0 tx=0).stop 2024-04-26T21:59:43.455 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:43.455+0000 7fe5ff200640 1 -- 172.21.15.72:0/3457111094 shutdown_connections 2024-04-26T21:59:43.455 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:43.455+0000 7fe5ff200640 1 --2- 172.21.15.72:0/3457111094 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe5f8078ed0 0x7fe5f81a64a0 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:43.455 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:43.455+0000 7fe5ff200640 1 --2- 172.21.15.72:0/3457111094 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fe5e00a3cc0 0x7fe5e00a6180 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:43.455 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:43.455+0000 7fe5ff200640 1 --2- 172.21.15.72:0/3457111094 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe5f8078530 0x7fe5f819f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:43.455 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:43.455+0000 7fe5ff200640 1 --2- 172.21.15.72:0/3457111094 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe5f80772f0 0x7fe5f819eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:43.456 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:43.455+0000 7fe5ff200640 1 -- 172.21.15.72:0/3457111094 >> 172.21.15.72:0/3457111094 conn(0x7fe5f80ffeb0 msgr2=0x7fe5f81019e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:43.456 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:43.455+0000 7fe5ff200640 1 -- 172.21.15.72:0/3457111094 shutdown_connections 2024-04-26T21:59:43.456 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:43.455+0000 7fe5ff200640 1 -- 172.21.15.72:0/3457111094 wait complete. 2024-04-26T21:59:43.474 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:46:29.785427+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T21:52:48.320391+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T21:52:49.517493+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:59:43.474 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:59:43.682 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:43.682+0000 7fdc92400640 1 -- 172.21.15.72:0/1608402044 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdc8c108f90 msgr2=0x7fdc8c1093b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:43.682 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:43.682+0000 7fdc92400640 1 --2- 172.21.15.72:0/1608402044 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdc8c108f90 0x7fdc8c1093b0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fdc7c009900 tx=0x7fdc7c02a410 comp rx=0 tx=0).stop 2024-04-26T21:59:43.683 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:43.682+0000 7fdc92400640 1 -- 172.21.15.72:0/1608402044 shutdown_connections 2024-04-26T21:59:43.683 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:43.682+0000 7fdc92400640 1 --2- 172.21.15.72:0/1608402044 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdc8c10ab40 0x7fdc8c1113d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:43.683 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:43.682+0000 7fdc92400640 1 --2- 172.21.15.72:0/1608402044 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdc8c10a180 0x7fdc8c10a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:43.683 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:43.682+0000 7fdc92400640 1 --2- 172.21.15.72:0/1608402044 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdc8c108f90 0x7fdc8c1093b0 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:43.683 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:43.682+0000 7fdc92400640 1 -- 172.21.15.72:0/1608402044 >> 172.21.15.72:0/1608402044 conn(0x7fdc8c077f30 msgr2=0x7fdc8c07a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:43.683 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:43.682+0000 7fdc92400640 1 -- 172.21.15.72:0/1608402044 shutdown_connections 2024-04-26T21:59:43.683 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:43.682+0000 7fdc92400640 1 -- 172.21.15.72:0/1608402044 wait complete. 2024-04-26T21:59:43.685 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:43.683+0000 7fdc92400640 1 Processor -- start 2024-04-26T21:59:43.686 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:43.683+0000 7fdc92400640 1 -- start start 2024-04-26T21:59:43.686 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:43.684+0000 7fdc92400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdc8c108f90 0x7fdc8c19f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:43.686 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:43.684+0000 7fdc92400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdc8c10a180 0x7fdc8c19f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:43.686 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:43.684+0000 7fdc92400640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdc8c10ab40 0x7fdc8c1a66e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:43.686 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:43.684+0000 7fdc92400640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fdc8c114980 con 0x7fdc8c108f90 2024-04-26T21:59:43.686 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:43.684+0000 7fdc92400640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fdc8c114800 con 0x7fdc8c10ab40 2024-04-26T21:59:43.686 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:43.684+0000 7fdc92400640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fdc8c114b00 con 0x7fdc8c10a180 2024-04-26T21:59:43.686 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:43.684+0000 7fdc8b400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdc8c108f90 0x7fdc8c19f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:43.686 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:43.684+0000 7fdc8b400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdc8c108f90 0x7fdc8c19f180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:36910/0 (socket says 172.21.15.72:36910) 2024-04-26T21:59:43.686 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:43.684+0000 7fdc8b400640 1 -- 172.21.15.72:0/2356424146 learned_addr learned my addr 172.21.15.72:0/2356424146 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:59:43.686 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:43.684+0000 7fdc8be00640 1 --2- 172.21.15.72:0/2356424146 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdc8c10ab40 0x7fdc8c1a66e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:43.686 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:43.684+0000 7fdc83e00640 1 --2- 172.21.15.72:0/2356424146 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdc8c10a180 0x7fdc8c19f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:43.686 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:43.685+0000 7fdc8b400640 1 -- 172.21.15.72:0/2356424146 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdc8c10a180 msgr2=0x7fdc8c19f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:43.687 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:43.685+0000 7fdc8b400640 1 --2- 172.21.15.72:0/2356424146 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdc8c10a180 0x7fdc8c19f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:43.687 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:43.685+0000 7fdc8b400640 1 -- 172.21.15.72:0/2356424146 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdc8c10ab40 msgr2=0x7fdc8c1a66e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:43.687 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:43.685+0000 7fdc8b400640 1 --2- 172.21.15.72:0/2356424146 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdc8c10ab40 0x7fdc8c1a66e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:43.687 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:43.685+0000 7fdc8b400640 1 -- 172.21.15.72:0/2356424146 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc8c1a6d50 con 0x7fdc8c108f90 2024-04-26T21:59:43.687 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:43.685+0000 7fdc8b400640 1 --2- 172.21.15.72:0/2356424146 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdc8c108f90 0x7fdc8c19f180 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fdc7c005ab0 tx=0x7fdc7c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:43.687 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:43.685+0000 7fdc88c00640 1 -- 172.21.15.72:0/2356424146 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc7c03f030 con 0x7fdc8c108f90 2024-04-26T21:59:43.687 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:43.685+0000 7fdc88c00640 1 -- 172.21.15.72:0/2356424146 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdc7c003c90 con 0x7fdc8c108f90 2024-04-26T21:59:43.687 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:43.685+0000 7fdc92400640 1 -- 172.21.15.72:0/2356424146 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc8c1a6fe0 con 0x7fdc8c108f90 2024-04-26T21:59:43.687 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:43.685+0000 7fdc88c00640 1 -- 172.21.15.72:0/2356424146 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc7c035070 con 0x7fdc8c108f90 2024-04-26T21:59:43.687 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:43.686+0000 7fdc92400640 1 -- 172.21.15.72:0/2356424146 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc8c1a73f0 con 0x7fdc8c108f90 2024-04-26T21:59:43.687 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:43.687+0000 7fdc92400640 1 -- 172.21.15.72:0/2356424146 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc8c110470 con 0x7fdc8c108f90 2024-04-26T21:59:43.688 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:43.689+0000 7fdc88c00640 1 -- 172.21.15.72:0/2356424146 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fdc7c042030 con 0x7fdc8c108f90 2024-04-26T21:59:43.691 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:43.691+0000 7fdc88c00640 1 --2- 172.21.15.72:0/2356424146 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fdc6c0a3c40 0x7fdc6c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:43.691 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:43.691+0000 7fdc88c00640 1 -- 172.21.15.72:0/2356424146 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7fdc7c0f1090 con 0x7fdc8c108f90 2024-04-26T21:59:43.691 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:43.691+0000 7fdc83e00640 1 --2- 172.21.15.72:0/2356424146 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fdc6c0a3c40 0x7fdc6c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:43.694 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:43.692+0000 7fdc83e00640 1 --2- 172.21.15.72:0/2356424146 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fdc6c0a3c40 0x7fdc6c0a6100 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fdc74005d00 tx=0x7fdc74005c50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:43.695 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:43.696+0000 7fdc88c00640 1 -- 172.21.15.72:0/2356424146 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fdc7c0bd830 con 0x7fdc8c108f90 2024-04-26T21:59:43.895 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:43.895+0000 7fdc92400640 1 -- 172.21.15.72:0/2356424146 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fdc8c02dd40 con 0x7fdc8c108f90 2024-04-26T21:59:43.896 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:43.896+0000 7fdc88c00640 1 -- 172.21.15.72:0/2356424146 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) ==== 93+0+10 (secure 0 0 0) 0x7fdc7c0bd130 con 0x7fdc8c108f90 2024-04-26T21:59:43.897 INFO:teuthology.orchestra.run.smithi072.stdout:pg_num: 1 2024-04-26T21:59:43.902 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:43.899+0000 7fdc92400640 1 -- 172.21.15.72:0/2356424146 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fdc6c0a3c40 msgr2=0x7fdc6c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:43.903 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:43.899+0000 7fdc92400640 1 --2- 172.21.15.72:0/2356424146 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fdc6c0a3c40 0x7fdc6c0a6100 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fdc74005d00 tx=0x7fdc74005c50 comp rx=0 tx=0).stop 2024-04-26T21:59:43.903 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:43.899+0000 7fdc92400640 1 -- 172.21.15.72:0/2356424146 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdc8c108f90 msgr2=0x7fdc8c19f180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:43.903 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:43.899+0000 7fdc92400640 1 --2- 172.21.15.72:0/2356424146 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdc8c108f90 0x7fdc8c19f180 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fdc7c005ab0 tx=0x7fdc7c0042f0 comp rx=0 tx=0).stop 2024-04-26T21:59:43.903 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:43.899+0000 7fdc92400640 1 -- 172.21.15.72:0/2356424146 shutdown_connections 2024-04-26T21:59:43.903 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:43.899+0000 7fdc92400640 1 --2- 172.21.15.72:0/2356424146 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdc8c10ab40 0x7fdc8c1a66e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:43.903 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:43.899+0000 7fdc92400640 1 --2- 172.21.15.72:0/2356424146 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fdc6c0a3c40 0x7fdc6c0a6100 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:43.903 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:43.899+0000 7fdc92400640 1 --2- 172.21.15.72:0/2356424146 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdc8c10a180 0x7fdc8c19f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:43.903 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:43.899+0000 7fdc92400640 1 --2- 172.21.15.72:0/2356424146 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdc8c108f90 0x7fdc8c19f180 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:43.903 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:43.899+0000 7fdc92400640 1 -- 172.21.15.72:0/2356424146 >> 172.21.15.72:0/2356424146 conn(0x7fdc8c077f30 msgr2=0x7fdc8c10d640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:43.903 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:43.899+0000 7fdc92400640 1 -- 172.21.15.72:0/2356424146 shutdown_connections 2024-04-26T21:59:43.903 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:43.899+0000 7fdc92400640 1 -- 172.21.15.72:0/2356424146 wait complete. 2024-04-26T21:59:43.918 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T21:59:44.126 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:44.126+0000 7f6b42200640 1 -- 172.21.15.72:0/2365221680 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6b3c1043f0 msgr2=0x7f6b3c10ac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:44.126 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:44.126+0000 7f6b42200640 1 --2- 172.21.15.72:0/2365221680 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6b3c1043f0 0x7f6b3c10ac80 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f6b34009ca0 tx=0x7f6b3402a8c0 comp rx=0 tx=0).stop 2024-04-26T21:59:44.126 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:44.126+0000 7f6b42200640 1 -- 172.21.15.72:0/2365221680 shutdown_connections 2024-04-26T21:59:44.126 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:44.126+0000 7f6b42200640 1 --2- 172.21.15.72:0/2365221680 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6b3c1043f0 0x7f6b3c10ac80 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:44.126 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:44.126+0000 7f6b42200640 1 --2- 172.21.15.72:0/2365221680 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6b3c103a30 0x7f6b3c103eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:44.126 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:44.126+0000 7f6b42200640 1 --2- 172.21.15.72:0/2365221680 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6b3c102840 0x7f6b3c102c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:44.127 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:44.126+0000 7f6b42200640 1 -- 172.21.15.72:0/2365221680 >> 172.21.15.72:0/2365221680 conn(0x7f6b3c0fddd0 msgr2=0x7f6b3c1001f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:44.127 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:44.126+0000 7f6b42200640 1 -- 172.21.15.72:0/2365221680 shutdown_connections 2024-04-26T21:59:44.127 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:44.126+0000 7f6b42200640 1 -- 172.21.15.72:0/2365221680 wait complete. 2024-04-26T21:59:44.129 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:44.127+0000 7f6b42200640 1 Processor -- start 2024-04-26T21:59:44.129 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:44.127+0000 7f6b42200640 1 -- start start 2024-04-26T21:59:44.129 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:44.127+0000 7f6b42200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6b3c102840 0x7f6b3c19ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:44.129 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:44.127+0000 7f6b42200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6b3c103a30 0x7f6b3c19b090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:44.129 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:44.127+0000 7f6b42200640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6b3c1043f0 0x7f6b3c1a2100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:44.129 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:44.127+0000 7f6b42200640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f6b3c10e270 con 0x7f6b3c1043f0 2024-04-26T21:59:44.129 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:44.127+0000 7f6b42200640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f6b3c10e0f0 con 0x7f6b3c102840 2024-04-26T21:59:44.129 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:44.127+0000 7f6b42200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f6b3c10e3f0 con 0x7f6b3c103a30 2024-04-26T21:59:44.129 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:44.128+0000 7f6b41800640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6b3c1043f0 0x7f6b3c1a2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:44.129 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:44.128+0000 7f6b41800640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6b3c1043f0 0x7f6b3c1a2100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:36928/0 (socket says 172.21.15.72:36928) 2024-04-26T21:59:44.129 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:44.128+0000 7f6b41800640 1 -- 172.21.15.72:0/622427786 learned_addr learned my addr 172.21.15.72:0/622427786 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:59:44.130 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:44.128+0000 7f6b40e00640 1 --2- 172.21.15.72:0/622427786 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6b3c102840 0x7f6b3c19ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:44.130 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:44.128+0000 7f6b3be00640 1 --2- 172.21.15.72:0/622427786 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6b3c103a30 0x7f6b3c19b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:44.130 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:44.128+0000 7f6b41800640 1 -- 172.21.15.72:0/622427786 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6b3c103a30 msgr2=0x7f6b3c19b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:44.130 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:44.128+0000 7f6b41800640 1 --2- 172.21.15.72:0/622427786 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6b3c103a30 0x7f6b3c19b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:44.130 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:44.128+0000 7f6b41800640 1 -- 172.21.15.72:0/622427786 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6b3c102840 msgr2=0x7f6b3c19ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:44.130 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:44.128+0000 7f6b41800640 1 --2- 172.21.15.72:0/622427786 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6b3c102840 0x7f6b3c19ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:44.130 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:44.128+0000 7f6b41800640 1 -- 172.21.15.72:0/622427786 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b3c1a2770 con 0x7f6b3c1043f0 2024-04-26T21:59:44.130 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:44.128+0000 7f6b3be00640 1 --2- 172.21.15.72:0/622427786 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6b3c103a30 0x7f6b3c19b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:59:44.130 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:44.128+0000 7f6b41800640 1 --2- 172.21.15.72:0/622427786 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6b3c1043f0 0x7f6b3c1a2100 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f6b3402aeb0 tx=0x7f6b34003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:44.130 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:44.129+0000 7f6b39600640 1 -- 172.21.15.72:0/622427786 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b34032ed0 con 0x7f6b3c1043f0 2024-04-26T21:59:44.130 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:44.129+0000 7f6b39600640 1 -- 172.21.15.72:0/622427786 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6b34003e10 con 0x7f6b3c1043f0 2024-04-26T21:59:44.130 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:44.129+0000 7f6b39600640 1 -- 172.21.15.72:0/622427786 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b34030cd0 con 0x7f6b3c1043f0 2024-04-26T21:59:44.130 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:44.129+0000 7f6b42200640 1 -- 172.21.15.72:0/622427786 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b3c1a2a00 con 0x7f6b3c1043f0 2024-04-26T21:59:44.131 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:44.129+0000 7f6b42200640 1 -- 172.21.15.72:0/622427786 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b3c1a2f50 con 0x7f6b3c1043f0 2024-04-26T21:59:44.131 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:44.130+0000 7f6b42200640 1 -- 172.21.15.72:0/622427786 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b3c1a3180 con 0x7f6b3c1043f0 2024-04-26T21:59:44.132 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:44.132+0000 7f6b39600640 1 -- 172.21.15.72:0/622427786 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f6b34004240 con 0x7f6b3c1043f0 2024-04-26T21:59:44.134 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:44.134+0000 7f6b39600640 1 --2- 172.21.15.72:0/622427786 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f6b240a3c40 0x7f6b240a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:44.135 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:44.135+0000 7f6b39600640 1 -- 172.21.15.72:0/622427786 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7f6b340f1cd0 con 0x7f6b3c1043f0 2024-04-26T21:59:44.135 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:44.135+0000 7f6b40e00640 1 --2- 172.21.15.72:0/622427786 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f6b240a3c40 0x7f6b240a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:44.135 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:44.135+0000 7f6b40e00640 1 --2- 172.21.15.72:0/622427786 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f6b240a3c40 0x7f6b240a6100 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f6b30005d00 tx=0x7f6b30005c50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:44.139 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:44.139+0000 7f6b39600640 1 -- 172.21.15.72:0/622427786 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f6b340be580 con 0x7f6b3c1043f0 2024-04-26T21:59:44.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:59:44 smithi076 ceph-mon[31823]: from='client.? 172.21.15.72:0/3457111094' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:59:44.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:59:44 smithi076 ceph-mon[31823]: from='client.? 172.21.15.72:0/2356424146' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T21:59:44.321 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:59:44 smithi072 ceph-mon[28138]: from='client.? 172.21.15.72:0/3457111094' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:59:44.321 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:59:44 smithi072 ceph-mon[28138]: from='client.? 172.21.15.72:0/2356424146' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T21:59:44.339 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:44.340+0000 7f6b42200640 1 -- 172.21.15.72:0/622427786 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f6b3c1a3470 con 0x7f6b3c1043f0 2024-04-26T21:59:44.340 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:44.340+0000 7f6b39600640 1 -- 172.21.15.72:0/622427786 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v91) ==== 104+0+11 (secure 0 0 0) 0x7f6b340bde80 con 0x7f6b3c1043f0 2024-04-26T21:59:44.340 INFO:teuthology.orchestra.run.smithi072.stdout:pg_num: 32 2024-04-26T21:59:44.343 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:44.343+0000 7f6b42200640 1 -- 172.21.15.72:0/622427786 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f6b240a3c40 msgr2=0x7f6b240a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:44.343 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:44.343+0000 7f6b42200640 1 --2- 172.21.15.72:0/622427786 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f6b240a3c40 0x7f6b240a6100 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f6b30005d00 tx=0x7f6b30005c50 comp rx=0 tx=0).stop 2024-04-26T21:59:44.344 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:44.343+0000 7f6b42200640 1 -- 172.21.15.72:0/622427786 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6b3c1043f0 msgr2=0x7f6b3c1a2100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:44.344 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:44.343+0000 7f6b42200640 1 --2- 172.21.15.72:0/622427786 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6b3c1043f0 0x7f6b3c1a2100 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f6b3402aeb0 tx=0x7f6b34003c20 comp rx=0 tx=0).stop 2024-04-26T21:59:44.344 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:44.343+0000 7f6b42200640 1 -- 172.21.15.72:0/622427786 shutdown_connections 2024-04-26T21:59:44.344 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:44.343+0000 7f6b42200640 1 --2- 172.21.15.72:0/622427786 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6b3c1043f0 0x7f6b3c1a2100 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:44.344 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:44.343+0000 7f6b42200640 1 --2- 172.21.15.72:0/622427786 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f6b240a3c40 0x7f6b240a6100 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:44.344 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:44.343+0000 7f6b42200640 1 --2- 172.21.15.72:0/622427786 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6b3c103a30 0x7f6b3c19b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:44.344 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:44.343+0000 7f6b42200640 1 --2- 172.21.15.72:0/622427786 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6b3c102840 0x7f6b3c19ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:44.344 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:44.343+0000 7f6b42200640 1 -- 172.21.15.72:0/622427786 >> 172.21.15.72:0/622427786 conn(0x7f6b3c0fddd0 msgr2=0x7f6b3c074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:44.344 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:44.343+0000 7f6b42200640 1 -- 172.21.15.72:0/622427786 shutdown_connections 2024-04-26T21:59:44.344 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:44.344+0000 7f6b42200640 1 -- 172.21.15.72:0/622427786 wait complete. 2024-04-26T21:59:44.362 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T21:59:44.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:59:44 smithi164 ceph-mon[31916]: from='client.? 172.21.15.72:0/3457111094' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:59:44.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:59:44 smithi164 ceph-mon[31916]: from='client.? 172.21.15.72:0/2356424146' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T21:59:44.567 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:44.567+0000 7fae3ea00640 1 -- 172.21.15.72:0/3172507421 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fae38102750 msgr2=0x7fae38102b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:44.568 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:44.567+0000 7fae3ea00640 1 --2- 172.21.15.72:0/3172507421 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fae38102750 0x7fae38102b70 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fae2c009900 tx=0x7fae2c02a410 comp rx=0 tx=0).stop 2024-04-26T21:59:44.568 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:44.567+0000 7fae3ea00640 1 -- 172.21.15.72:0/3172507421 shutdown_connections 2024-04-26T21:59:44.568 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:44.567+0000 7fae3ea00640 1 --2- 172.21.15.72:0/3172507421 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fae38104300 0x7fae3810ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:44.568 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:44.567+0000 7fae3ea00640 1 --2- 172.21.15.72:0/3172507421 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fae38103940 0x7fae38103dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:44.568 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:44.567+0000 7fae3ea00640 1 --2- 172.21.15.72:0/3172507421 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fae38102750 0x7fae38102b70 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:44.568 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:44.567+0000 7fae3ea00640 1 -- 172.21.15.72:0/3172507421 >> 172.21.15.72:0/3172507421 conn(0x7fae380fdcc0 msgr2=0x7fae38100100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:44.568 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:44.567+0000 7fae3ea00640 1 -- 172.21.15.72:0/3172507421 shutdown_connections 2024-04-26T21:59:44.568 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:44.567+0000 7fae3ea00640 1 -- 172.21.15.72:0/3172507421 wait complete. 2024-04-26T21:59:44.570 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:44.568+0000 7fae3ea00640 1 Processor -- start 2024-04-26T21:59:44.571 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:44.569+0000 7fae3ea00640 1 -- start start 2024-04-26T21:59:44.571 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:44.569+0000 7fae3ea00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fae38102750 0x7fae3819aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:44.571 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:44.569+0000 7fae3ea00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fae38103940 0x7fae3819aff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:44.571 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:44.569+0000 7fae3ea00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fae38104300 0x7fae381a2060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:44.571 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:44.569+0000 7fae3ea00640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fae3810e100 con 0x7fae38102750 2024-04-26T21:59:44.571 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:44.569+0000 7fae3ea00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fae3810df80 con 0x7fae38103940 2024-04-26T21:59:44.571 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:44.569+0000 7fae3ea00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fae3810e280 con 0x7fae38104300 2024-04-26T21:59:44.571 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:44.569+0000 7fae3d600640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fae38102750 0x7fae3819aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:44.571 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:44.569+0000 7fae3d600640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fae38102750 0x7fae3819aab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:36936/0 (socket says 172.21.15.72:36936) 2024-04-26T21:59:44.571 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:44.569+0000 7fae3d600640 1 -- 172.21.15.72:0/3220427210 learned_addr learned my addr 172.21.15.72:0/3220427210 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:59:44.571 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:44.570+0000 7fae3d600640 1 -- 172.21.15.72:0/3220427210 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fae38104300 msgr2=0x7fae381a2060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:44.572 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:44.570+0000 7fae3d600640 1 --2- 172.21.15.72:0/3220427210 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fae38104300 0x7fae381a2060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:44.572 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:44.570+0000 7fae3d600640 1 -- 172.21.15.72:0/3220427210 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fae38103940 msgr2=0x7fae3819aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:44.572 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:44.570+0000 7fae3d600640 1 --2- 172.21.15.72:0/3220427210 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fae38103940 0x7fae3819aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:44.572 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:44.570+0000 7fae3d600640 1 -- 172.21.15.72:0/3220427210 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae381a26d0 con 0x7fae38102750 2024-04-26T21:59:44.572 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:44.570+0000 7fae3d600640 1 --2- 172.21.15.72:0/3220427210 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fae38102750 0x7fae3819aab0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fae2c005ab0 tx=0x7fae2c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:44.572 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:44.570+0000 7fae36000640 1 -- 172.21.15.72:0/3220427210 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae2c03f030 con 0x7fae38102750 2024-04-26T21:59:44.572 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:44.570+0000 7fae36000640 1 -- 172.21.15.72:0/3220427210 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fae2c003c90 con 0x7fae38102750 2024-04-26T21:59:44.572 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:44.570+0000 7fae36000640 1 -- 172.21.15.72:0/3220427210 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae2c035070 con 0x7fae38102750 2024-04-26T21:59:44.572 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:44.570+0000 7fae3ea00640 1 -- 172.21.15.72:0/3220427210 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae381a2960 con 0x7fae38102750 2024-04-26T21:59:44.572 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:44.570+0000 7fae3ea00640 1 -- 172.21.15.72:0/3220427210 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae381a2ea0 con 0x7fae38102750 2024-04-26T21:59:44.573 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:44.571+0000 7fae3ea00640 1 -- 172.21.15.72:0/3220427210 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae38194ef0 con 0x7fae38102750 2024-04-26T21:59:44.573 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:44.573+0000 7fae36000640 1 -- 172.21.15.72:0/3220427210 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fae2c042030 con 0x7fae38102750 2024-04-26T21:59:44.575 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:44.575+0000 7fae36000640 1 --2- 172.21.15.72:0/3220427210 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fae200a3cc0 0x7fae200a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:44.575 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:44.575+0000 7fae36000640 1 -- 172.21.15.72:0/3220427210 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7fae2c0f0ff0 con 0x7fae38102750 2024-04-26T21:59:44.575 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:44.575+0000 7fae3cc00640 1 --2- 172.21.15.72:0/3220427210 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fae200a3cc0 0x7fae200a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:44.576 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:44.576+0000 7fae3cc00640 1 --2- 172.21.15.72:0/3220427210 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fae200a3cc0 0x7fae200a6180 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fae24005fd0 tx=0x7fae24005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:44.579 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:44.579+0000 7fae36000640 1 -- 172.21.15.72:0/3220427210 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fae2c0bd7b0 con 0x7fae38102750 2024-04-26T21:59:44.781 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:44.781+0000 7fae3ea00640 1 -- 172.21.15.72:0/3220427210 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fae3802dd50 con 0x7fae38102750 2024-04-26T21:59:44.782 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:44.783+0000 7fae36000640 1 -- 172.21.15.72:0/3220427210 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v91) ==== 100+0+11 (secure 0 0 0) 0x7fae2c0bd0b0 con 0x7fae38102750 2024-04-26T21:59:44.783 INFO:teuthology.orchestra.run.smithi072.stdout:pg_num: 64 2024-04-26T21:59:44.785 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:44.785+0000 7fae3ea00640 1 -- 172.21.15.72:0/3220427210 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fae200a3cc0 msgr2=0x7fae200a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:44.785 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:44.785+0000 7fae3ea00640 1 --2- 172.21.15.72:0/3220427210 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fae200a3cc0 0x7fae200a6180 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fae24005fd0 tx=0x7fae24005850 comp rx=0 tx=0).stop 2024-04-26T21:59:44.785 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:44.785+0000 7fae3ea00640 1 -- 172.21.15.72:0/3220427210 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fae38102750 msgr2=0x7fae3819aab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:44.785 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:44.785+0000 7fae3ea00640 1 --2- 172.21.15.72:0/3220427210 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fae38102750 0x7fae3819aab0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fae2c005ab0 tx=0x7fae2c0042f0 comp rx=0 tx=0).stop 2024-04-26T21:59:44.786 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:44.785+0000 7fae3ea00640 1 -- 172.21.15.72:0/3220427210 shutdown_connections 2024-04-26T21:59:44.786 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:44.785+0000 7fae3ea00640 1 --2- 172.21.15.72:0/3220427210 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fae200a3cc0 0x7fae200a6180 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:44.786 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:44.785+0000 7fae3ea00640 1 --2- 172.21.15.72:0/3220427210 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fae38104300 0x7fae381a2060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:44.786 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:44.785+0000 7fae3ea00640 1 --2- 172.21.15.72:0/3220427210 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fae38103940 0x7fae3819aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:44.786 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:44.785+0000 7fae3ea00640 1 --2- 172.21.15.72:0/3220427210 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fae38102750 0x7fae3819aab0 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:44.786 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:44.785+0000 7fae3ea00640 1 -- 172.21.15.72:0/3220427210 >> 172.21.15.72:0/3220427210 conn(0x7fae380fdcc0 msgr2=0x7fae38109200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:44.786 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:44.785+0000 7fae3ea00640 1 -- 172.21.15.72:0/3220427210 shutdown_connections 2024-04-26T21:59:44.786 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:44.785+0000 7fae3ea00640 1 -- 172.21.15.72:0/3220427210 wait complete. 2024-04-26T21:59:44.805 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-26T21:59:45.009 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:45.008+0000 7f9579e00640 1 -- 172.21.15.72:0/1625587014 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9574076970 msgr2=0x7f9574074dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:45.009 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:45.008+0000 7f9579e00640 1 --2- 172.21.15.72:0/1625587014 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9574076970 0x7f9574074dc0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f9568009930 tx=0x7f956802a420 comp rx=0 tx=0).stop 2024-04-26T21:59:45.009 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:45.009+0000 7f9579e00640 1 -- 172.21.15.72:0/1625587014 shutdown_connections 2024-04-26T21:59:45.009 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:45.009+0000 7f9579e00640 1 --2- 172.21.15.72:0/1625587014 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f957410ed40 0x7f9574111130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:45.009 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:45.009+0000 7f9579e00640 1 --2- 172.21.15.72:0/1625587014 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9574075390 0x7f9574075810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:45.009 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:45.009+0000 7f9579e00640 1 --2- 172.21.15.72:0/1625587014 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9574076970 0x7f9574074dc0 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:45.010 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:45.009+0000 7f9579e00640 1 -- 172.21.15.72:0/1625587014 >> 172.21.15.72:0/1625587014 conn(0x7f95740fdce0 msgr2=0x7f9574100120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:45.010 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:45.009+0000 7f9579e00640 1 -- 172.21.15.72:0/1625587014 shutdown_connections 2024-04-26T21:59:45.010 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:45.009+0000 7f9579e00640 1 -- 172.21.15.72:0/1625587014 wait complete. 2024-04-26T21:59:45.012 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:45.010+0000 7f9579e00640 1 Processor -- start 2024-04-26T21:59:45.012 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:45.010+0000 7f9579e00640 1 -- start start 2024-04-26T21:59:45.012 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:45.010+0000 7f9579e00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9574075390 0x7f957419ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:45.012 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:45.010+0000 7f9579e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9574076970 0x7f957419f490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:45.012 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:45.010+0000 7f9579e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f957410ed40 0x7f95741a6500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:45.012 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:45.010+0000 7f9579e00640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f95741147f0 con 0x7f9574076970 2024-04-26T21:59:45.012 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:45.010+0000 7f9579e00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f9574114670 con 0x7f9574075390 2024-04-26T21:59:45.012 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:45.010+0000 7f9579e00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f9574114970 con 0x7f957410ed40 2024-04-26T21:59:45.013 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:45.011+0000 7f9573e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9574076970 0x7f957419f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:45.013 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:45.011+0000 7f9573e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9574076970 0x7f957419f490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:36948/0 (socket says 172.21.15.72:36948) 2024-04-26T21:59:45.013 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:45.011+0000 7f9573e00640 1 -- 172.21.15.72:0/387179998 learned_addr learned my addr 172.21.15.72:0/387179998 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:59:45.013 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:45.011+0000 7f9578a00640 1 --2- 172.21.15.72:0/387179998 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9574075390 0x7f957419ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:45.013 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:45.011+0000 7f9579400640 1 --2- 172.21.15.72:0/387179998 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f957410ed40 0x7f95741a6500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:45.013 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:45.011+0000 7f9573e00640 1 -- 172.21.15.72:0/387179998 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f957410ed40 msgr2=0x7f95741a6500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:45.013 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:45.011+0000 7f9573e00640 1 --2- 172.21.15.72:0/387179998 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f957410ed40 0x7f95741a6500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:45.013 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:45.011+0000 7f9573e00640 1 -- 172.21.15.72:0/387179998 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9574075390 msgr2=0x7f957419ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:45.013 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:45.011+0000 7f9573e00640 1 --2- 172.21.15.72:0/387179998 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9574075390 0x7f957419ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:45.013 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:45.011+0000 7f9573e00640 1 -- 172.21.15.72:0/387179998 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95741a6b70 con 0x7f9574076970 2024-04-26T21:59:45.013 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:45.011+0000 7f9579400640 1 --2- 172.21.15.72:0/387179998 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f957410ed40 0x7f95741a6500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:59:45.013 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:45.012+0000 7f9573e00640 1 --2- 172.21.15.72:0/387179998 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9574076970 0x7f957419f490 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f956000d950 tx=0x7f956000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:45.013 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:45.012+0000 7f9571600640 1 -- 172.21.15.72:0/387179998 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9560017070 con 0x7f9574076970 2024-04-26T21:59:45.013 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:45.012+0000 7f9571600640 1 -- 172.21.15.72:0/387179998 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9560009c30 con 0x7f9574076970 2024-04-26T21:59:45.013 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:45.012+0000 7f9571600640 1 -- 172.21.15.72:0/387179998 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f956000f620 con 0x7f9574076970 2024-04-26T21:59:45.014 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:45.012+0000 7f9579e00640 1 -- 172.21.15.72:0/387179998 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95741a6e60 con 0x7f9574076970 2024-04-26T21:59:45.014 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:45.012+0000 7f9579e00640 1 -- 172.21.15.72:0/387179998 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95741a73f0 con 0x7f9574076970 2024-04-26T21:59:45.014 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:45.013+0000 7f9579e00640 1 -- 172.21.15.72:0/387179998 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9574104f10 con 0x7f9574076970 2024-04-26T21:59:45.015 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:45.015+0000 7f9571600640 1 -- 172.21.15.72:0/387179998 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f9560010460 con 0x7f9574076970 2024-04-26T21:59:45.017 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:45.017+0000 7f9571600640 1 --2- 172.21.15.72:0/387179998 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f955c0a3cc0 0x7f955c0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:45.017 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:45.017+0000 7f9571600640 1 -- 172.21.15.72:0/387179998 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7f95600d02f0 con 0x7f9574076970 2024-04-26T21:59:45.017 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:45.017+0000 7f9578a00640 1 --2- 172.21.15.72:0/387179998 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f955c0a3cc0 0x7f955c0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:45.018 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:45.018+0000 7f9578a00640 1 --2- 172.21.15.72:0/387179998 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f955c0a3cc0 0x7f955c0a6180 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f95680057c0 tx=0x7f9568000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:45.021 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:45.021+0000 7f9571600640 1 -- 172.21.15.72:0/387179998 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f956009ca90 con 0x7f9574076970 2024-04-26T21:59:45.071 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:59:45 smithi072 ceph-mon[28138]: pgmap v578: 97 pgs: 97 active+clean; 1.1 MiB data, 656 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s 2024-04-26T21:59:45.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:59:45 smithi072 ceph-mon[28138]: from='client.? 172.21.15.72:0/622427786' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T21:59:45.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:59:45 smithi072 ceph-mon[28138]: from='client.? 172.21.15.72:0/3220427210' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T21:59:45.237 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:45.237+0000 7f9579e00640 1 -- 172.21.15.72:0/387179998 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f957402dcf0 con 0x7f9574076970 2024-04-26T21:59:45.238 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:45.238+0000 7f9571600640 1 -- 172.21.15.72:0/387179998 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v91) ==== 81+0+591 (secure 0 0 0) 0x7f9560009de0 con 0x7f9574076970 2024-04-26T21:59:45.238 INFO:teuthology.orchestra.run.smithi072.stdout:172.21.15.72:0/416277784 2024-04-27T21:43:09.602791+0000 2024-04-26T21:59:45.239 INFO:teuthology.orchestra.run.smithi072.stdout:172.21.15.72:6800/4010358672 2024-04-27T21:43:09.602791+0000 2024-04-26T21:59:45.239 INFO:teuthology.orchestra.run.smithi072.stdout:172.21.15.72:6801/1012617371 2024-04-27T21:42:34.684837+0000 2024-04-26T21:59:45.239 INFO:teuthology.orchestra.run.smithi072.stdout:172.21.15.72:6801/4010358672 2024-04-27T21:43:09.602791+0000 2024-04-26T21:59:45.239 INFO:teuthology.orchestra.run.smithi072.stdout:172.21.15.72:0/3555448179 2024-04-27T21:43:09.602791+0000 2024-04-26T21:59:45.239 INFO:teuthology.orchestra.run.smithi072.stdout:172.21.15.72:6800/1012617371 2024-04-27T21:42:34.684837+0000 2024-04-26T21:59:45.239 INFO:teuthology.orchestra.run.smithi072.stdout:172.21.15.72:0/2920378761 2024-04-27T21:42:34.684837+0000 2024-04-26T21:59:45.239 INFO:teuthology.orchestra.run.smithi072.stdout:172.21.15.72:0/1933740041 2024-04-27T21:43:09.602791+0000 2024-04-26T21:59:45.239 INFO:teuthology.orchestra.run.smithi072.stdout:172.21.15.72:0/3225295775 2024-04-27T21:42:34.684837+0000 2024-04-26T21:59:45.239 INFO:teuthology.orchestra.run.smithi072.stdout:172.21.15.72:0/2326016180 2024-04-27T21:42:34.684837+0000 2024-04-26T21:59:45.239 INFO:teuthology.orchestra.run.smithi072.stderr:listed 10 entries 2024-04-26T21:59:45.241 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:45.241+0000 7f9579e00640 1 -- 172.21.15.72:0/387179998 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f955c0a3cc0 msgr2=0x7f955c0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:45.241 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:45.241+0000 7f9579e00640 1 --2- 172.21.15.72:0/387179998 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f955c0a3cc0 0x7f955c0a6180 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f95680057c0 tx=0x7f9568000bc0 comp rx=0 tx=0).stop 2024-04-26T21:59:45.241 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:45.241+0000 7f9579e00640 1 -- 172.21.15.72:0/387179998 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9574076970 msgr2=0x7f957419f490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:45.241 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:45.241+0000 7f9579e00640 1 --2- 172.21.15.72:0/387179998 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9574076970 0x7f957419f490 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f956000d950 tx=0x7f956000de10 comp rx=0 tx=0).stop 2024-04-26T21:59:45.241 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:45.241+0000 7f9579e00640 1 -- 172.21.15.72:0/387179998 shutdown_connections 2024-04-26T21:59:45.242 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:45.241+0000 7f9579e00640 1 --2- 172.21.15.72:0/387179998 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f955c0a3cc0 0x7f955c0a6180 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:45.242 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:45.241+0000 7f9579e00640 1 --2- 172.21.15.72:0/387179998 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f957410ed40 0x7f95741a6500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:45.242 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:45.241+0000 7f9579e00640 1 --2- 172.21.15.72:0/387179998 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9574076970 0x7f957419f490 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:45.242 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:45.241+0000 7f9579e00640 1 --2- 172.21.15.72:0/387179998 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9574075390 0x7f957419ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:45.242 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:45.241+0000 7f9579e00640 1 -- 172.21.15.72:0/387179998 >> 172.21.15.72:0/387179998 conn(0x7f95740fdce0 msgr2=0x7f95741020e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:45.242 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:45.241+0000 7f9579e00640 1 -- 172.21.15.72:0/387179998 shutdown_connections 2024-04-26T21:59:45.242 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:45.241+0000 7f9579e00640 1 -- 172.21.15.72:0/387179998 wait complete. 2024-04-26T21:59:45.260 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2024-04-26T21:59:45.260 DEBUG:teuthology.orchestra.run.smithi072:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:59:45.314 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:59:45.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:59:45 smithi076 ceph-mon[31823]: pgmap v578: 97 pgs: 97 active+clean; 1.1 MiB data, 656 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s 2024-04-26T21:59:45.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:59:45 smithi076 ceph-mon[31823]: from='client.? 172.21.15.72:0/622427786' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T21:59:45.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:59:45 smithi076 ceph-mon[31823]: from='client.? 172.21.15.72:0/3220427210' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T21:59:45.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:59:45 smithi164 ceph-mon[31916]: pgmap v578: 97 pgs: 97 active+clean; 1.1 MiB data, 656 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s 2024-04-26T21:59:45.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:59:45 smithi164 ceph-mon[31916]: from='client.? 172.21.15.72:0/622427786' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T21:59:45.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:59:45 smithi164 ceph-mon[31916]: from='client.? 172.21.15.72:0/3220427210' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T21:59:45.505 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:45.505+0000 7f1097200640 1 -- 172.21.15.72:0/3547327118 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1098105b80 msgr2=0x7f1098106000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:45.506 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:45.505+0000 7f1097200640 1 --2- 172.21.15.72:0/3547327118 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1098105b80 0x7f1098106000 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f1084009900 tx=0x7f108402a410 comp rx=0 tx=0).stop 2024-04-26T21:59:45.506 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:45.505+0000 7f1097200640 1 -- 172.21.15.72:0/3547327118 shutdown_connections 2024-04-26T21:59:45.506 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:45.505+0000 7f1097200640 1 --2- 172.21.15.72:0/3547327118 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1098106540 0x7f109810cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:45.506 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:45.505+0000 7f1097200640 1 --2- 172.21.15.72:0/3547327118 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1098105b80 0x7f1098106000 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:45.506 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:45.505+0000 7f1097200640 1 --2- 172.21.15.72:0/3547327118 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1098104990 0x7f1098104db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:45.506 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:45.505+0000 7f1097200640 1 -- 172.21.15.72:0/3547327118 >> 172.21.15.72:0/3547327118 conn(0x7f10980fff00 msgr2=0x7f1098102340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:45.506 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:45.505+0000 7f1097200640 1 -- 172.21.15.72:0/3547327118 shutdown_connections 2024-04-26T21:59:45.506 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:45.505+0000 7f1097200640 1 -- 172.21.15.72:0/3547327118 wait complete. 2024-04-26T21:59:45.508 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:45.506+0000 7f1097200640 1 Processor -- start 2024-04-26T21:59:45.508 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:45.507+0000 7f1097200640 1 -- start start 2024-04-26T21:59:45.508 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:45.507+0000 7f1097200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1098104990 0x7f109819aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:45.508 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:45.507+0000 7f1097200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1098105b80 0x7f109819b000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:45.508 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:45.507+0000 7f1097200640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1098106540 0x7f10981a2070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:45.508 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:45.507+0000 7f1097200640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f10981103c0 con 0x7f1098106540 2024-04-26T21:59:45.508 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:45.507+0000 7f1097200640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f1098110240 con 0x7f1098105b80 2024-04-26T21:59:45.508 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:45.507+0000 7f1097200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f1098110540 con 0x7f1098104990 2024-04-26T21:59:45.508 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:45.507+0000 7f1096800640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1098106540 0x7f10981a2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:45.508 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:45.507+0000 7f1096800640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1098106540 0x7f10981a2070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:36976/0 (socket says 172.21.15.72:36976) 2024-04-26T21:59:45.508 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:45.507+0000 7f1096800640 1 -- 172.21.15.72:0/894122017 learned_addr learned my addr 172.21.15.72:0/894122017 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:59:45.509 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:45.508+0000 7f1095e00640 1 --2- 172.21.15.72:0/894122017 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1098104990 0x7f109819aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:45.509 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:45.508+0000 7f1095400640 1 --2- 172.21.15.72:0/894122017 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1098105b80 0x7f109819b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:45.509 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:45.508+0000 7f1096800640 1 -- 172.21.15.72:0/894122017 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1098104990 msgr2=0x7f109819aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:45.509 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:45.508+0000 7f1096800640 1 --2- 172.21.15.72:0/894122017 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1098104990 0x7f109819aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:45.509 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:45.508+0000 7f1096800640 1 -- 172.21.15.72:0/894122017 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1098105b80 msgr2=0x7f109819b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:45.509 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:45.508+0000 7f1096800640 1 --2- 172.21.15.72:0/894122017 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1098105b80 0x7f109819b000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:45.509 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:45.508+0000 7f1096800640 1 -- 172.21.15.72:0/894122017 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10981a26e0 con 0x7f1098106540 2024-04-26T21:59:45.509 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:45.508+0000 7f1096800640 1 --2- 172.21.15.72:0/894122017 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1098106540 0x7f10981a2070 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f109000eaa0 tx=0x7f109000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:45.510 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:45.508+0000 7f108ea00640 1 -- 172.21.15.72:0/894122017 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f109000cdb0 con 0x7f1098106540 2024-04-26T21:59:45.510 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:45.508+0000 7f108ea00640 1 -- 172.21.15.72:0/894122017 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1090005020 con 0x7f1098106540 2024-04-26T21:59:45.510 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:45.508+0000 7f1097200640 1 -- 172.21.15.72:0/894122017 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10981a29d0 con 0x7f1098106540 2024-04-26T21:59:45.510 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:45.509+0000 7f108ea00640 1 -- 172.21.15.72:0/894122017 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10900116d0 con 0x7f1098106540 2024-04-26T21:59:45.510 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:45.509+0000 7f1097200640 1 -- 172.21.15.72:0/894122017 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10981a2ea0 con 0x7f1098106540 2024-04-26T21:59:45.510 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:45.510+0000 7f1097200640 1 -- 172.21.15.72:0/894122017 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f109810bd20 con 0x7f1098106540 2024-04-26T21:59:45.512 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:45.512+0000 7f108ea00640 1 -- 172.21.15.72:0/894122017 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f10900051d0 con 0x7f1098106540 2024-04-26T21:59:45.514 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:45.513+0000 7f108ea00640 1 --2- 172.21.15.72:0/894122017 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f107c0a3c40 0x7f107c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:45.514 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:45.514+0000 7f108ea00640 1 -- 172.21.15.72:0/894122017 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7f10900d0c70 con 0x7f1098106540 2024-04-26T21:59:45.514 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:45.514+0000 7f1095e00640 1 --2- 172.21.15.72:0/894122017 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f107c0a3c40 0x7f107c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:45.514 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:45.514+0000 7f1095e00640 1 --2- 172.21.15.72:0/894122017 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f107c0a3c40 0x7f107c0a6100 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f1088005fd0 tx=0x7f1088005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:45.516 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:45.516+0000 7f108ea00640 1 -- 172.21.15.72:0/894122017 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f1090016030 con 0x7f1098106540 2024-04-26T21:59:45.720 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:45.720+0000 7f1097200640 1 -- 172.21.15.72:0/894122017 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f109802dad0 con 0x7f1098106540 2024-04-26T21:59:45.722 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:45.722+0000 7f108ea00640 1 -- 172.21.15.72:0/894122017 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+22783 (secure 0 0 0) 0x7f109009d410 con 0x7f1098106540 2024-04-26T21:59:45.723 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T21:59:45.723 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":91,"fsid":"a1780950-0415-11ef-bc93-c7b262605968","created":"2024-04-26T21:42:01.504173+0000","modified":"2024-04-26T21:54:11.892872+0000","last_up_change":"2024-04-26T21:51:43.482437+0000","last_in_change":"2024-04-26T21:51:18.468587+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:46:29.785427+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:52:48.320391+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T21:52:49.517493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"04e06884-b355-4762-9b29-9034b7201e9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6802","nonce":1416765179},{"type":"v1","addr":"172.21.15.72:6803","nonce":1416765179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6804","nonce":1416765179},{"type":"v1","addr":"172.21.15.72:6805","nonce":1416765179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6808","nonce":1416765179},{"type":"v1","addr":"172.21.15.72:6809","nonce":1416765179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6806","nonce":1416765179},{"type":"v1","addr":"172.21.15.72:6807","nonce":1416765179}]},"public_addr":"172.21.15.72:6803/1416765179","cluster_addr":"172.21.15.72:6805/1416765179","heartbeat_back_addr":"172.21.15.72:6809/1416765179","heartbeat_front_addr":"172.21.15.72:6807/1416765179","state":["exists","up"]},{"osd":1,"uuid":"d75a479a-4a2e-48e9-a01e-2b1c9617416a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":755426949},{"type":"v1","addr":"172.21.15.76:6801","nonce":755426949}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":755426949},{"type":"v1","addr":"172.21.15.76:6803","nonce":755426949}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":755426949},{"type":"v1","addr":"172.21.15.76:6807","nonce":755426949}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":755426949},{"type":"v1","addr":"172.21.15.76:6805","nonce":755426949}]},"public_addr":"172.21.15.76:6801/755426949","cluster_addr":"172.21.15.76:6803/755426949","heartbeat_back_addr":"172.21.15.76:6807/755426949","heartbeat_front_addr":"172.21.15.76:6805/755426949","state":["exists","up"]},{"osd":2,"uuid":"b6f8da58-4428-4e8c-a944-7ea5645d6d92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":852265973},{"type":"v1","addr":"172.21.15.164:6801","nonce":852265973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":852265973},{"type":"v1","addr":"172.21.15.164:6803","nonce":852265973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":852265973},{"type":"v1","addr":"172.21.15.164:6807","nonce":852265973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":852265973},{"type":"v1","addr":"172.21.15.164:6805","nonce":852265973}]},"public_addr":"172.21.15.164:6801/852265973","cluster_addr":"172.21.15.164:6803/852265973","heartbeat_back_addr":"172.21.15.164:6807/852265973","heartbeat_front_addr":"172.21.15.164:6805/852265973","state":["exists","up"]},{"osd":3,"uuid":"76d83409-649e-41e5-b4f4-43bbd5b00ea5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6810","nonce":4027763789},{"type":"v1","addr":"172.21.15.72:6811","nonce":4027763789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6812","nonce":4027763789},{"type":"v1","addr":"172.21.15.72:6813","nonce":4027763789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6816","nonce":4027763789},{"type":"v1","addr":"172.21.15.72:6817","nonce":4027763789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6814","nonce":4027763789},{"type":"v1","addr":"172.21.15.72:6815","nonce":4027763789}]},"public_addr":"172.21.15.72:6811/4027763789","cluster_addr":"172.21.15.72:6813/4027763789","heartbeat_back_addr":"172.21.15.72:6817/4027763789","heartbeat_front_addr":"172.21.15.72:6815/4027763789","state":["exists","up"]},{"osd":4,"uuid":"d2cfeea7-1661-4156-bdcd-f770a8d1a878","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":3718867827},{"type":"v1","addr":"172.21.15.76:6809","nonce":3718867827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":3718867827},{"type":"v1","addr":"172.21.15.76:6811","nonce":3718867827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":3718867827},{"type":"v1","addr":"172.21.15.76:6815","nonce":3718867827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":3718867827},{"type":"v1","addr":"172.21.15.76:6813","nonce":3718867827}]},"public_addr":"172.21.15.76:6809/3718867827","cluster_addr":"172.21.15.76:6811/3718867827","heartbeat_back_addr":"172.21.15.76:6815/3718867827","heartbeat_front_addr":"172.21.15.76:6813/3718867827","state":["exists","up"]},{"osd":5,"uuid":"f3896ad6-a040-4ea3-a431-ff9b7fc05c36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":2543954772},{"type":"v1","addr":"172.21.15.164:6809","nonce":2543954772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":2543954772},{"type":"v1","addr":"172.21.15.164:6811","nonce":2543954772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":2543954772},{"type":"v1","addr":"172.21.15.164:6815","nonce":2543954772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":2543954772},{"type":"v1","addr":"172.21.15.164:6813","nonce":2543954772}]},"public_addr":"172.21.15.164:6809/2543954772","cluster_addr":"172.21.15.164:6811/2543954772","heartbeat_back_addr":"172.21.15.164:6815/2543954772","heartbeat_front_addr":"172.21.15.164:6813/2543954772","state":["exists","up"]},{"osd":6,"uuid":"d5d56650-be7e-4ba5-ba3a-d7d473294dab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6818","nonce":2080659027},{"type":"v1","addr":"172.21.15.72:6819","nonce":2080659027}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6820","nonce":2080659027},{"type":"v1","addr":"172.21.15.72:6821","nonce":2080659027}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6824","nonce":2080659027},{"type":"v1","addr":"172.21.15.72:6825","nonce":2080659027}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6822","nonce":2080659027},{"type":"v1","addr":"172.21.15.72:6823","nonce":2080659027}]},"public_addr":"172.21.15.72:6819/2080659027","cluster_addr":"172.21.15.72:6821/2080659027","heartbeat_back_addr":"172.21.15.72:6825/2080659027","heartbeat_front_addr":"172.21.15.72:6823/2080659027","state":["exists","up"]},{"osd":7,"uuid":"18975b5a-9309-4a0f-9f37-3ce09c2e745d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":3144631323},{"type":"v1","addr":"172.21.15.76:6817","nonce":3144631323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":3144631323},{"type":"v1","addr":"172.21.15.76:6819","nonce":3144631323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":3144631323},{"type":"v1","addr":"172.21.15.76:6823","nonce":3144631323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":3144631323},{"type":"v1","addr":"172.21.15.76:6821","nonce":3144631323}]},"public_addr":"172.21.15.76:6817/3144631323","cluster_addr":"172.21.15.76:6819/3144631323","heartbeat_back_addr":"172.21.15.76:6823/3144631323","heartbeat_front_addr":"172.21.15.76:6821/3144631323","state":["exists","up"]},{"osd":8,"uuid":"779e9c1e-8e97-48d4-a854-68ca13f38e0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":627888883},{"type":"v1","addr":"172.21.15.164:6817","nonce":627888883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":627888883},{"type":"v1","addr":"172.21.15.164:6819","nonce":627888883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":627888883},{"type":"v1","addr":"172.21.15.164:6823","nonce":627888883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":627888883},{"type":"v1","addr":"172.21.15.164:6821","nonce":627888883}]},"public_addr":"172.21.15.164:6817/627888883","cluster_addr":"172.21.15.164:6819/627888883","heartbeat_back_addr":"172.21.15.164:6823/627888883","heartbeat_front_addr":"172.21.15.164:6821/627888883","state":["exists","up"]},{"osd":9,"uuid":"d0317850-7b2d-4850-b2cf-0b919505c1fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6826","nonce":1273118485},{"type":"v1","addr":"172.21.15.72:6827","nonce":1273118485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6828","nonce":1273118485},{"type":"v1","addr":"172.21.15.72:6829","nonce":1273118485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6832","nonce":1273118485},{"type":"v1","addr":"172.21.15.72:6833","nonce":1273118485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6830","nonce":1273118485},{"type":"v1","addr":"172.21.15.72:6831","nonce":1273118485}]},"public_addr":"172.21.15.72:6827/1273118485","cluster_addr":"172.21.15.72:6829/1273118485","heartbeat_back_addr":"172.21.15.72:6833/1273118485","heartbeat_front_addr":"172.21.15.72:6831/1273118485","state":["exists","up"]},{"osd":10,"uuid":"e0aa38d8-6011-4d2a-b67b-5bff6d999e47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":2808954459},{"type":"v1","addr":"172.21.15.76:6825","nonce":2808954459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":2808954459},{"type":"v1","addr":"172.21.15.76:6827","nonce":2808954459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":2808954459},{"type":"v1","addr":"172.21.15.76:6831","nonce":2808954459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":2808954459},{"type":"v1","addr":"172.21.15.76:6829","nonce":2808954459}]},"public_addr":"172.21.15.76:6825/2808954459","cluster_addr":"172.21.15.76:6827/2808954459","heartbeat_back_addr":"172.21.15.76:6831/2808954459","heartbeat_front_addr":"172.21.15.76:6829/2808954459","state":["exists","up"]},{"osd":11,"uuid":"981b3d4d-916e-4937-9916-7cc21a891e43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":2354951784},{"type":"v1","addr":"172.21.15.164:6825","nonce":2354951784}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":2354951784},{"type":"v1","addr":"172.21.15.164:6827","nonce":2354951784}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":2354951784},{"type":"v1","addr":"172.21.15.164:6831","nonce":2354951784}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":2354951784},{"type":"v1","addr":"172.21.15.164:6829","nonce":2354951784}]},"public_addr":"172.21.15.164:6825/2354951784","cluster_addr":"172.21.15.164:6827/2354951784","heartbeat_back_addr":"172.21.15.164:6831/2354951784","heartbeat_front_addr":"172.21.15.164:6829/2354951784","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:18.997924+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:52.439081+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:46:25.893288+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:47:00.112807+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:47:35.116874+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:48:09.216344+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:48:45.028853+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:49:19.905397+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:49:55.030536+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:50:30.767662+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:51:06.423267+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:51:41.631211+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.1a","mappings":[{"from":5,"to":7}]},{"pgid":"3.1b","mappings":[{"from":4,"to":6}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.28","mappings":[{"from":5,"to":6}]},{"pgid":"3.2a","mappings":[{"from":3,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.72:0/416277784":"2024-04-27T21:43:09.602791+0000","172.21.15.72:6800/4010358672":"2024-04-27T21:43:09.602791+0000","172.21.15.72:6801/1012617371":"2024-04-27T21:42:34.684837+0000","172.21.15.72:6801/4010358672":"2024-04-27T21:43:09.602791+0000","172.21.15.72:0/3555448179":"2024-04-27T21:43:09.602791+0000","172.21.15.72:6800/1012617371":"2024-04-27T21:42:34.684837+0000","172.21.15.72:0/2920378761":"2024-04-27T21:42:34.684837+0000","172.21.15.72:0/1933740041":"2024-04-27T21:43:09.602791+0000","172.21.15.72:0/3225295775":"2024-04-27T21:42:34.684837+0000","172.21.15.72:0/2326016180":"2024-04-27T21:42:34.684837+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:59:45.725 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:45.725+0000 7f1097200640 1 -- 172.21.15.72:0/894122017 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f107c0a3c40 msgr2=0x7f107c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:45.725 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:45.725+0000 7f1097200640 1 --2- 172.21.15.72:0/894122017 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f107c0a3c40 0x7f107c0a6100 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f1088005fd0 tx=0x7f1088005850 comp rx=0 tx=0).stop 2024-04-26T21:59:45.725 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:45.725+0000 7f1097200640 1 -- 172.21.15.72:0/894122017 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1098106540 msgr2=0x7f10981a2070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:45.725 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:45.725+0000 7f1097200640 1 --2- 172.21.15.72:0/894122017 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1098106540 0x7f10981a2070 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f109000eaa0 tx=0x7f109000ef60 comp rx=0 tx=0).stop 2024-04-26T21:59:45.725 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:45.725+0000 7f1097200640 1 -- 172.21.15.72:0/894122017 shutdown_connections 2024-04-26T21:59:45.725 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:45.725+0000 7f1097200640 1 --2- 172.21.15.72:0/894122017 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1098106540 0x7f10981a2070 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:45.725 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:45.725+0000 7f1097200640 1 --2- 172.21.15.72:0/894122017 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1098105b80 0x7f109819b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:45.725 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:45.725+0000 7f1097200640 1 --2- 172.21.15.72:0/894122017 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f107c0a3c40 0x7f107c0a6100 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:45.725 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:45.725+0000 7f1097200640 1 --2- 172.21.15.72:0/894122017 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1098104990 0x7f109819aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:45.725 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:45.725+0000 7f1097200640 1 -- 172.21.15.72:0/894122017 >> 172.21.15.72:0/894122017 conn(0x7f10980fff00 msgr2=0x7f1098108ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:45.726 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:45.725+0000 7f1097200640 1 -- 172.21.15.72:0/894122017 shutdown_connections 2024-04-26T21:59:45.726 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:45.725+0000 7f1097200640 1 -- 172.21.15.72:0/894122017 wait complete. 2024-04-26T21:59:45.744 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:46:29.785427+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T21:52:48.320391+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T21:52:49.517493+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:59:45.745 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:59:45.953 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:45.953+0000 7f58a8a00640 1 -- 172.21.15.72:0/2208543329 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f58a4076970 msgr2=0x7f58a4074dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:45.953 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:45.953+0000 7f58a8a00640 1 --2- 172.21.15.72:0/2208543329 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f58a4076970 0x7f58a4074dc0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f5894009900 tx=0x7f589402a410 comp rx=0 tx=0).stop 2024-04-26T21:59:45.954 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:45.953+0000 7f58a8a00640 1 -- 172.21.15.72:0/2208543329 shutdown_connections 2024-04-26T21:59:45.954 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:45.953+0000 7f58a8a00640 1 --2- 172.21.15.72:0/2208543329 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f58a410a780 0x7f58a410cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:45.954 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:45.953+0000 7f58a8a00640 1 --2- 172.21.15.72:0/2208543329 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f58a4075390 0x7f58a4075810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:45.954 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:45.953+0000 7f58a8a00640 1 --2- 172.21.15.72:0/2208543329 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f58a4076970 0x7f58a4074dc0 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:45.954 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:45.953+0000 7f58a8a00640 1 -- 172.21.15.72:0/2208543329 >> 172.21.15.72:0/2208543329 conn(0x7f58a40fde30 msgr2=0x7f58a4100270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:45.954 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:45.953+0000 7f58a8a00640 1 -- 172.21.15.72:0/2208543329 shutdown_connections 2024-04-26T21:59:45.954 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:45.953+0000 7f58a8a00640 1 -- 172.21.15.72:0/2208543329 wait complete. 2024-04-26T21:59:45.956 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:45.954+0000 7f58a8a00640 1 Processor -- start 2024-04-26T21:59:45.956 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:45.954+0000 7f58a8a00640 1 -- start start 2024-04-26T21:59:45.956 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:45.954+0000 7f58a8a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f58a4075390 0x7f58a419abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:45.956 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:45.954+0000 7f58a8a00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f58a4076970 0x7f58a419b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:45.957 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:45.954+0000 7f58a8a00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f58a410a780 0x7f58a41a2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:45.957 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:45.954+0000 7f58a8a00640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f58a4110460 con 0x7f58a4076970 2024-04-26T21:59:45.957 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:45.954+0000 7f58a8a00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f58a41102e0 con 0x7f58a410a780 2024-04-26T21:59:45.957 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:45.954+0000 7f58a8a00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f58a41105e0 con 0x7f58a4075390 2024-04-26T21:59:45.957 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:45.955+0000 7f58a1e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f58a4076970 0x7f58a419b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:45.957 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:45.955+0000 7f58a1e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f58a4076970 0x7f58a419b100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:36988/0 (socket says 172.21.15.72:36988) 2024-04-26T21:59:45.957 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:45.955+0000 7f58a1e00640 1 -- 172.21.15.72:0/66295150 learned_addr learned my addr 172.21.15.72:0/66295150 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:59:45.957 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:45.955+0000 7f58a2800640 1 --2- 172.21.15.72:0/66295150 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f58a4075390 0x7f58a419abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:45.957 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:45.955+0000 7f58a3200640 1 --2- 172.21.15.72:0/66295150 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f58a410a780 0x7f58a41a2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:45.957 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:45.955+0000 7f58a1e00640 1 -- 172.21.15.72:0/66295150 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f58a4075390 msgr2=0x7f58a419abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:45.957 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:45.955+0000 7f58a1e00640 1 --2- 172.21.15.72:0/66295150 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f58a4075390 0x7f58a419abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:45.957 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:45.955+0000 7f58a1e00640 1 -- 172.21.15.72:0/66295150 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f58a410a780 msgr2=0x7f58a41a2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:45.957 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:45.955+0000 7f58a1e00640 1 --2- 172.21.15.72:0/66295150 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f58a410a780 0x7f58a41a2170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:45.957 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:45.955+0000 7f58a1e00640 1 -- 172.21.15.72:0/66295150 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58a41a27e0 con 0x7f58a4076970 2024-04-26T21:59:45.958 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:45.956+0000 7f58a1e00640 1 --2- 172.21.15.72:0/66295150 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f58a4076970 0x7f58a419b100 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f588c00eab0 tx=0x7f588c00ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:45.958 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:45.956+0000 7f589b400640 1 -- 172.21.15.72:0/66295150 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f588c00ce40 con 0x7f58a4076970 2024-04-26T21:59:45.958 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:45.956+0000 7f589b400640 1 -- 172.21.15.72:0/66295150 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f588c011900 con 0x7f58a4076970 2024-04-26T21:59:45.958 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:45.956+0000 7f589b400640 1 -- 172.21.15.72:0/66295150 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f588c010990 con 0x7f58a4076970 2024-04-26T21:59:45.958 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:45.956+0000 7f58a8a00640 1 -- 172.21.15.72:0/66295150 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58a41a2ad0 con 0x7f58a4076970 2024-04-26T21:59:45.958 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:45.956+0000 7f58a8a00640 1 -- 172.21.15.72:0/66295150 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58a41a2fd0 con 0x7f58a4076970 2024-04-26T21:59:45.958 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:45.957+0000 7f58a8a00640 1 -- 172.21.15.72:0/66295150 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58a41094b0 con 0x7f58a4076970 2024-04-26T21:59:45.959 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:45.959+0000 7f589b400640 1 -- 172.21.15.72:0/66295150 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f588c011470 con 0x7f58a4076970 2024-04-26T21:59:45.961 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:45.961+0000 7f589b400640 1 --2- 172.21.15.72:0/66295150 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f58840a3c40 0x7f58840a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:45.962 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:45.962+0000 7f589b400640 1 -- 172.21.15.72:0/66295150 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7f588c0d0d50 con 0x7f58a4076970 2024-04-26T21:59:45.962 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:45.962+0000 7f58a2800640 1 --2- 172.21.15.72:0/66295150 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f58840a3c40 0x7f58840a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:45.962 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:45.962+0000 7f58a2800640 1 --2- 172.21.15.72:0/66295150 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f58840a3c40 0x7f58840a6100 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f5894006010 tx=0x7f5894031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:45.966 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:45.966+0000 7f589b400640 1 -- 172.21.15.72:0/66295150 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f588c016030 con 0x7f58a4076970 2024-04-26T21:59:46.172 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:46.172+0000 7f58a8a00640 1 -- 172.21.15.72:0/66295150 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f58a402dce0 con 0x7f58a4076970 2024-04-26T21:59:46.173 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:46.173+0000 7f589b400640 1 -- 172.21.15.72:0/66295150 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) ==== 93+0+10 (secure 0 0 0) 0x7f588c09d4f0 con 0x7f58a4076970 2024-04-26T21:59:46.173 INFO:teuthology.orchestra.run.smithi072.stdout:pg_num: 1 2024-04-26T21:59:46.176 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:46.176+0000 7f58a8a00640 1 -- 172.21.15.72:0/66295150 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f58840a3c40 msgr2=0x7f58840a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:46.176 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:46.176+0000 7f58a8a00640 1 --2- 172.21.15.72:0/66295150 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f58840a3c40 0x7f58840a6100 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f5894006010 tx=0x7f5894031400 comp rx=0 tx=0).stop 2024-04-26T21:59:46.176 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:46.176+0000 7f58a8a00640 1 -- 172.21.15.72:0/66295150 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f58a4076970 msgr2=0x7f58a419b100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:46.176 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:46.176+0000 7f58a8a00640 1 --2- 172.21.15.72:0/66295150 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f58a4076970 0x7f58a419b100 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f588c00eab0 tx=0x7f588c00ef70 comp rx=0 tx=0).stop 2024-04-26T21:59:46.176 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:46.176+0000 7f58a8a00640 1 -- 172.21.15.72:0/66295150 shutdown_connections 2024-04-26T21:59:46.176 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:46.176+0000 7f58a8a00640 1 --2- 172.21.15.72:0/66295150 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f58a410a780 0x7f58a41a2170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:46.176 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:46.176+0000 7f58a8a00640 1 --2- 172.21.15.72:0/66295150 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f58a4076970 0x7f58a419b100 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:46.176 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:46.176+0000 7f58a8a00640 1 --2- 172.21.15.72:0/66295150 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f58840a3c40 0x7f58840a6100 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:46.177 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:46.176+0000 7f58a8a00640 1 --2- 172.21.15.72:0/66295150 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f58a4075390 0x7f58a419abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:46.177 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:46.176+0000 7f58a8a00640 1 -- 172.21.15.72:0/66295150 >> 172.21.15.72:0/66295150 conn(0x7f58a40fde30 msgr2=0x7f58a4106680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:46.177 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:46.176+0000 7f58a8a00640 1 -- 172.21.15.72:0/66295150 shutdown_connections 2024-04-26T21:59:46.177 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:46.176+0000 7f58a8a00640 1 -- 172.21.15.72:0/66295150 wait complete. 2024-04-26T21:59:46.194 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T21:59:46.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:59:46 smithi076 ceph-mon[31823]: from='client.? 172.21.15.72:0/387179998' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-26T21:59:46.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:59:46 smithi076 ceph-mon[31823]: from='client.? 172.21.15.72:0/894122017' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:59:46.321 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:59:46 smithi072 ceph-mon[28138]: from='client.? 172.21.15.72:0/387179998' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-26T21:59:46.321 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:59:46 smithi072 ceph-mon[28138]: from='client.? 172.21.15.72:0/894122017' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:59:46.385 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:46.385+0000 7ffb07000640 1 -- 172.21.15.72:0/2001149850 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ffb0010a980 msgr2=0x7ffb001112a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:46.385 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:46.385+0000 7ffb07000640 1 --2- 172.21.15.72:0/2001149850 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ffb0010a980 0x7ffb001112a0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7ffaf8009ca0 tx=0x7ffaf802a8c0 comp rx=0 tx=0).stop 2024-04-26T21:59:46.386 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:46.385+0000 7ffb07000640 1 -- 172.21.15.72:0/2001149850 shutdown_connections 2024-04-26T21:59:46.386 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:46.385+0000 7ffb07000640 1 --2- 172.21.15.72:0/2001149850 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ffb0010a980 0x7ffb001112a0 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:46.386 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:46.385+0000 7ffb07000640 1 --2- 172.21.15.72:0/2001149850 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffb00109fc0 0x7ffb0010a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:46.386 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:46.385+0000 7ffb07000640 1 --2- 172.21.15.72:0/2001149850 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffb00108e70 0x7ffb00109290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:46.386 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:46.385+0000 7ffb07000640 1 -- 172.21.15.72:0/2001149850 >> 172.21.15.72:0/2001149850 conn(0x7ffb00077e00 msgr2=0x7ffb0007a240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:46.386 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:46.385+0000 7ffb07000640 1 -- 172.21.15.72:0/2001149850 shutdown_connections 2024-04-26T21:59:46.386 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:46.385+0000 7ffb07000640 1 -- 172.21.15.72:0/2001149850 wait complete. 2024-04-26T21:59:46.388 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:46.386+0000 7ffb07000640 1 Processor -- start 2024-04-26T21:59:46.388 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:46.386+0000 7ffb07000640 1 -- start start 2024-04-26T21:59:46.388 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:46.386+0000 7ffb07000640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ffb00108e70 0x7ffb0019ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:46.388 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:46.386+0000 7ffb07000640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffb00109fc0 0x7ffb0019f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:46.389 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:46.386+0000 7ffb07000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffb0010a980 0x7ffb001a6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:46.389 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:46.386+0000 7ffb07000640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7ffb001148a0 con 0x7ffb00108e70 2024-04-26T21:59:46.389 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:46.386+0000 7ffb07000640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7ffb00114720 con 0x7ffb00109fc0 2024-04-26T21:59:46.389 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:46.386+0000 7ffb07000640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ffb00114a20 con 0x7ffb0010a980 2024-04-26T21:59:46.389 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:46.387+0000 7ffb05c00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ffb00108e70 0x7ffb0019ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:46.389 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:46.387+0000 7ffb05c00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ffb00108e70 0x7ffb0019ee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:36998/0 (socket says 172.21.15.72:36998) 2024-04-26T21:59:46.389 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:46.387+0000 7ffb05c00640 1 -- 172.21.15.72:0/1794679106 learned_addr learned my addr 172.21.15.72:0/1794679106 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:59:46.389 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:46.387+0000 7ffb05200640 1 --2- 172.21.15.72:0/1794679106 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffb00109fc0 0x7ffb0019f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:46.389 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:46.387+0000 7ffb06600640 1 --2- 172.21.15.72:0/1794679106 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffb0010a980 0x7ffb001a6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:46.389 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:46.387+0000 7ffb05c00640 1 -- 172.21.15.72:0/1794679106 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffb0010a980 msgr2=0x7ffb001a6410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:46.389 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:46.387+0000 7ffb05c00640 1 --2- 172.21.15.72:0/1794679106 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffb0010a980 0x7ffb001a6410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:46.389 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:46.387+0000 7ffb05c00640 1 -- 172.21.15.72:0/1794679106 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffb00109fc0 msgr2=0x7ffb0019f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:46.389 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:46.387+0000 7ffb05c00640 1 --2- 172.21.15.72:0/1794679106 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffb00109fc0 0x7ffb0019f3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:46.389 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:46.387+0000 7ffb05c00640 1 -- 172.21.15.72:0/1794679106 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb001a6a80 con 0x7ffb00108e70 2024-04-26T21:59:46.389 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:46.387+0000 7ffb05200640 1 --2- 172.21.15.72:0/1794679106 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffb00109fc0 0x7ffb0019f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:59:46.390 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:46.388+0000 7ffb05c00640 1 --2- 172.21.15.72:0/1794679106 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ffb00108e70 0x7ffb0019ee60 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7ffaf400d950 tx=0x7ffaf400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:46.390 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:46.388+0000 7ffafe000640 1 -- 172.21.15.72:0/1794679106 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaf4017070 con 0x7ffb00108e70 2024-04-26T21:59:46.390 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:46.388+0000 7ffb07000640 1 -- 172.21.15.72:0/1794679106 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb001a6d70 con 0x7ffb00108e70 2024-04-26T21:59:46.390 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:46.388+0000 7ffafe000640 1 -- 172.21.15.72:0/1794679106 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ffaf4009c30 con 0x7ffb00108e70 2024-04-26T21:59:46.390 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:46.388+0000 7ffafe000640 1 -- 172.21.15.72:0/1794679106 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaf400f620 con 0x7ffb00108e70 2024-04-26T21:59:46.390 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:46.388+0000 7ffb07000640 1 -- 172.21.15.72:0/1794679106 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb001a7200 con 0x7ffb00108e70 2024-04-26T21:59:46.390 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:46.389+0000 7ffb07000640 1 -- 172.21.15.72:0/1794679106 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb001100b0 con 0x7ffb00108e70 2024-04-26T21:59:46.391 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:46.391+0000 7ffafe000640 1 -- 172.21.15.72:0/1794679106 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7ffaf4010460 con 0x7ffb00108e70 2024-04-26T21:59:46.392 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:46.392+0000 7ffafe000640 1 --2- 172.21.15.72:0/1794679106 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7ffae80a3c40 0x7ffae80a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:46.392 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:46.392+0000 7ffafe000640 1 -- 172.21.15.72:0/1794679106 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7ffaf40d0330 con 0x7ffb00108e70 2024-04-26T21:59:46.392 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:46.393+0000 7ffb05200640 1 --2- 172.21.15.72:0/1794679106 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7ffae80a3c40 0x7ffae80a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:46.393 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:46.393+0000 7ffb05200640 1 --2- 172.21.15.72:0/1794679106 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7ffae80a3c40 0x7ffae80a6100 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7ffb001a0370 tx=0x7ffaec00a5e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:46.397 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:46.397+0000 7ffafe000640 1 -- 172.21.15.72:0/1794679106 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7ffaf409cad0 con 0x7ffb00108e70 2024-04-26T21:59:46.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:59:46 smithi164 ceph-mon[31916]: from='client.? 172.21.15.72:0/387179998' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-26T21:59:46.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:59:46 smithi164 ceph-mon[31916]: from='client.? 172.21.15.72:0/894122017' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:59:46.583 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:46.583+0000 7ffb07000640 1 -- 172.21.15.72:0/1794679106 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7ffb000635d0 con 0x7ffb00108e70 2024-04-26T21:59:46.583 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:46.584+0000 7ffafe000640 1 -- 172.21.15.72:0/1794679106 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v91) ==== 104+0+11 (secure 0 0 0) 0x7ffaf4009de0 con 0x7ffb00108e70 2024-04-26T21:59:46.584 INFO:teuthology.orchestra.run.smithi072.stdout:pg_num: 32 2024-04-26T21:59:46.586 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:46.586+0000 7ffb07000640 1 -- 172.21.15.72:0/1794679106 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7ffae80a3c40 msgr2=0x7ffae80a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:46.586 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:46.586+0000 7ffb07000640 1 --2- 172.21.15.72:0/1794679106 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7ffae80a3c40 0x7ffae80a6100 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7ffb001a0370 tx=0x7ffaec00a5e0 comp rx=0 tx=0).stop 2024-04-26T21:59:46.586 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:46.586+0000 7ffb07000640 1 -- 172.21.15.72:0/1794679106 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ffb00108e70 msgr2=0x7ffb0019ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:46.586 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:46.586+0000 7ffb07000640 1 --2- 172.21.15.72:0/1794679106 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ffb00108e70 0x7ffb0019ee60 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7ffaf400d950 tx=0x7ffaf400de10 comp rx=0 tx=0).stop 2024-04-26T21:59:46.586 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:46.586+0000 7ffb07000640 1 -- 172.21.15.72:0/1794679106 shutdown_connections 2024-04-26T21:59:46.586 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:46.586+0000 7ffb07000640 1 --2- 172.21.15.72:0/1794679106 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7ffae80a3c40 0x7ffae80a6100 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:46.587 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:46.586+0000 7ffb07000640 1 --2- 172.21.15.72:0/1794679106 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffb0010a980 0x7ffb001a6410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:46.587 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:46.586+0000 7ffb07000640 1 --2- 172.21.15.72:0/1794679106 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffb00109fc0 0x7ffb0019f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:46.587 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:46.586+0000 7ffb07000640 1 --2- 172.21.15.72:0/1794679106 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ffb00108e70 0x7ffb0019ee60 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:46.587 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:46.586+0000 7ffb07000640 1 -- 172.21.15.72:0/1794679106 >> 172.21.15.72:0/1794679106 conn(0x7ffb00077e00 msgr2=0x7ffb0010d280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:46.587 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:46.586+0000 7ffb07000640 1 -- 172.21.15.72:0/1794679106 shutdown_connections 2024-04-26T21:59:46.587 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:46.586+0000 7ffb07000640 1 -- 172.21.15.72:0/1794679106 wait complete. 2024-04-26T21:59:46.604 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T21:59:46.805 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:46.805+0000 7effad200640 1 -- 172.21.15.72:0/1073221422 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7effa81022e0 msgr2=0x7effa8102760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:46.805 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:46.805+0000 7effad200640 1 --2- 172.21.15.72:0/1073221422 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7effa81022e0 0x7effa8102760 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7effa0009ca0 tx=0x7effa002a8c0 comp rx=0 tx=0).stop 2024-04-26T21:59:46.805 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:46.805+0000 7effad200640 1 -- 172.21.15.72:0/1073221422 shutdown_connections 2024-04-26T21:59:46.805 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:46.805+0000 7effad200640 1 --2- 172.21.15.72:0/1073221422 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7effa81022e0 0x7effa8102760 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:46.805 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:46.805+0000 7effad200640 1 --2- 172.21.15.72:0/1073221422 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7effa8101940 0x7effa8101da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:46.806 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:46.805+0000 7effad200640 1 --2- 172.21.15.72:0/1073221422 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7effa8100fe0 0x7effa8101400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:46.806 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:46.805+0000 7effad200640 1 -- 172.21.15.72:0/1073221422 >> 172.21.15.72:0/1073221422 conn(0x7effa80fcc20 msgr2=0x7effa80ff040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:46.806 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:46.805+0000 7effad200640 1 -- 172.21.15.72:0/1073221422 shutdown_connections 2024-04-26T21:59:46.806 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:46.805+0000 7effad200640 1 -- 172.21.15.72:0/1073221422 wait complete. 2024-04-26T21:59:46.807 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:46.806+0000 7effad200640 1 Processor -- start 2024-04-26T21:59:46.807 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:46.806+0000 7effad200640 1 -- start start 2024-04-26T21:59:46.807 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:46.806+0000 7effad200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7effa8100fe0 0x7effa819a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:46.807 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:46.806+0000 7effad200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7effa8101940 0x7effa819af20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:46.807 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:46.806+0000 7effad200640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7effa81022e0 0x7effa81a1f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:46.808 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:46.806+0000 7effad200640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7effa8110370 con 0x7effa81022e0 2024-04-26T21:59:46.808 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:46.806+0000 7effad200640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7effa81101f0 con 0x7effa8101940 2024-04-26T21:59:46.808 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:46.806+0000 7effad200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7effa81104f0 con 0x7effa8100fe0 2024-04-26T21:59:46.808 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:46.807+0000 7effa7e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7effa81022e0 0x7effa81a1f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:46.808 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:46.807+0000 7effa7e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7effa81022e0 0x7effa81a1f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:37018/0 (socket says 172.21.15.72:37018) 2024-04-26T21:59:46.808 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:46.807+0000 7effa7e00640 1 -- 172.21.15.72:0/1442590454 learned_addr learned my addr 172.21.15.72:0/1442590454 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:59:46.808 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:46.807+0000 7effa7400640 1 --2- 172.21.15.72:0/1442590454 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7effa8100fe0 0x7effa819a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:46.808 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:46.807+0000 7effa6a00640 1 --2- 172.21.15.72:0/1442590454 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7effa8101940 0x7effa819af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:46.808 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:46.807+0000 7effa7e00640 1 -- 172.21.15.72:0/1442590454 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7effa8100fe0 msgr2=0x7effa819a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:46.808 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:46.807+0000 7effa7e00640 1 --2- 172.21.15.72:0/1442590454 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7effa8100fe0 0x7effa819a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:46.808 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:46.807+0000 7effa7e00640 1 -- 172.21.15.72:0/1442590454 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7effa8101940 msgr2=0x7effa819af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:46.808 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:46.807+0000 7effa7e00640 1 --2- 172.21.15.72:0/1442590454 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7effa8101940 0x7effa819af20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:46.809 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:46.807+0000 7effa7e00640 1 -- 172.21.15.72:0/1442590454 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effa81a2600 con 0x7effa81022e0 2024-04-26T21:59:46.810 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:46.808+0000 7effa7e00640 1 --2- 172.21.15.72:0/1442590454 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7effa81022e0 0x7effa81a1f90 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7effa0031040 tx=0x7effa0003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:46.810 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:46.808+0000 7eff9fe00640 1 -- 172.21.15.72:0/1442590454 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effa0004040 con 0x7effa81022e0 2024-04-26T21:59:46.810 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:46.808+0000 7eff9fe00640 1 -- 172.21.15.72:0/1442590454 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7effa0004670 con 0x7effa81022e0 2024-04-26T21:59:46.810 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:46.808+0000 7eff9fe00640 1 -- 172.21.15.72:0/1442590454 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effa0030d20 con 0x7effa81022e0 2024-04-26T21:59:46.810 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:46.808+0000 7effad200640 1 -- 172.21.15.72:0/1442590454 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effa81a2890 con 0x7effa81022e0 2024-04-26T21:59:46.810 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:46.808+0000 7effad200640 1 -- 172.21.15.72:0/1442590454 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effa81a2d90 con 0x7effa81022e0 2024-04-26T21:59:46.810 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:46.810+0000 7effad200640 1 -- 172.21.15.72:0/1442590454 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7effa810bd30 con 0x7effa81022e0 2024-04-26T21:59:46.811 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:46.812+0000 7eff9fe00640 1 -- 172.21.15.72:0/1442590454 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7effa00305c0 con 0x7effa81022e0 2024-04-26T21:59:46.813 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:46.813+0000 7eff9fe00640 1 --2- 172.21.15.72:0/1442590454 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7eff8c0a3cc0 0x7eff8c0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:46.813 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:46.813+0000 7eff9fe00640 1 -- 172.21.15.72:0/1442590454 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7effa00f1cb0 con 0x7effa81022e0 2024-04-26T21:59:46.813 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:46.814+0000 7effa7400640 1 --2- 172.21.15.72:0/1442590454 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7eff8c0a3cc0 0x7eff8c0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:46.814 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:46.814+0000 7effa7400640 1 --2- 172.21.15.72:0/1442590454 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7eff8c0a3cc0 0x7eff8c0a6180 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7eff98005fd0 tx=0x7eff98005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:46.818 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:46.818+0000 7eff9fe00640 1 -- 172.21.15.72:0/1442590454 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7effa00be560 con 0x7effa81022e0 2024-04-26T21:59:47.022 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:47.022+0000 7effad200640 1 -- 172.21.15.72:0/1442590454 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7effa802dd20 con 0x7effa81022e0 2024-04-26T21:59:47.023 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:47.023+0000 7eff9fe00640 1 -- 172.21.15.72:0/1442590454 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v91) ==== 100+0+11 (secure 0 0 0) 0x7effa00bde60 con 0x7effa81022e0 2024-04-26T21:59:47.023 INFO:teuthology.orchestra.run.smithi072.stdout:pg_num: 64 2024-04-26T21:59:47.025 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:47.025+0000 7effad200640 1 -- 172.21.15.72:0/1442590454 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7eff8c0a3cc0 msgr2=0x7eff8c0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:47.026 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:47.025+0000 7effad200640 1 --2- 172.21.15.72:0/1442590454 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7eff8c0a3cc0 0x7eff8c0a6180 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7eff98005fd0 tx=0x7eff98005850 comp rx=0 tx=0).stop 2024-04-26T21:59:47.026 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:47.025+0000 7effad200640 1 -- 172.21.15.72:0/1442590454 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7effa81022e0 msgr2=0x7effa81a1f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:47.026 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:47.025+0000 7effad200640 1 --2- 172.21.15.72:0/1442590454 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7effa81022e0 0x7effa81a1f90 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7effa0031040 tx=0x7effa0003ba0 comp rx=0 tx=0).stop 2024-04-26T21:59:47.026 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:47.025+0000 7effad200640 1 -- 172.21.15.72:0/1442590454 shutdown_connections 2024-04-26T21:59:47.026 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:47.025+0000 7effad200640 1 --2- 172.21.15.72:0/1442590454 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7effa81022e0 0x7effa81a1f90 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:47.026 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:47.025+0000 7effad200640 1 --2- 172.21.15.72:0/1442590454 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7effa8101940 0x7effa819af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:47.026 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:47.025+0000 7effad200640 1 --2- 172.21.15.72:0/1442590454 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7eff8c0a3cc0 0x7eff8c0a6180 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:47.026 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:47.025+0000 7effad200640 1 --2- 172.21.15.72:0/1442590454 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7effa8100fe0 0x7effa819a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:47.026 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:47.025+0000 7effad200640 1 -- 172.21.15.72:0/1442590454 >> 172.21.15.72:0/1442590454 conn(0x7effa80fcc20 msgr2=0x7effa80fe720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:47.027 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:47.025+0000 7effad200640 1 -- 172.21.15.72:0/1442590454 shutdown_connections 2024-04-26T21:59:47.027 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:47.026+0000 7effad200640 1 -- 172.21.15.72:0/1442590454 wait complete. 2024-04-26T21:59:47.045 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-26T21:59:47.246 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:47.245+0000 7fd2e3e00640 1 -- 172.21.15.72:0/1406820170 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd2e4108440 msgr2=0x7fd2e410a890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:47.246 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:47.245+0000 7fd2e3e00640 1 --2- 172.21.15.72:0/1406820170 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd2e4108440 0x7fd2e410a890 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fd2dc009ca0 tx=0x7fd2dc02a8c0 comp rx=0 tx=0).stop 2024-04-26T21:59:47.246 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:47.245+0000 7fd2e3e00640 1 -- 172.21.15.72:0/1406820170 shutdown_connections 2024-04-26T21:59:47.246 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:47.245+0000 7fd2e3e00640 1 --2- 172.21.15.72:0/1406820170 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd2e4108440 0x7fd2e410a890 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:47.246 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:47.245+0000 7fd2e3e00640 1 --2- 172.21.15.72:0/1406820170 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd2e4105b10 0x7fd2e4107f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:47.246 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:47.245+0000 7fd2e3e00640 1 --2- 172.21.15.72:0/1406820170 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd2e41031e0 0x7fd2e41055d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:47.246 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:47.245+0000 7fd2e3e00640 1 -- 172.21.15.72:0/1406820170 >> 172.21.15.72:0/1406820170 conn(0x7fd2e40fcbb0 msgr2=0x7fd2e40ff010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:47.246 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:47.245+0000 7fd2e3e00640 1 -- 172.21.15.72:0/1406820170 shutdown_connections 2024-04-26T21:59:47.246 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:47.245+0000 7fd2e3e00640 1 -- 172.21.15.72:0/1406820170 wait complete. 2024-04-26T21:59:47.249 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:47.246+0000 7fd2e3e00640 1 Processor -- start 2024-04-26T21:59:47.250 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:47.246+0000 7fd2e3e00640 1 -- start start 2024-04-26T21:59:47.250 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:47.247+0000 7fd2e3e00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd2e41031e0 0x7fd2e4102b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:47.250 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:47.247+0000 7fd2e3e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd2e4105b10 0x7fd2e4101190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:47.250 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:47.247+0000 7fd2e3e00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd2e4108440 0x7fd2e41016d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:47.250 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:47.247+0000 7fd2e3e00640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fd2e410e1b0 con 0x7fd2e4108440 2024-04-26T21:59:47.250 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:47.247+0000 7fd2e3e00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fd2e410e030 con 0x7fd2e41031e0 2024-04-26T21:59:47.250 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:47.247+0000 7fd2e3e00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd2e410e330 con 0x7fd2e4105b10 2024-04-26T21:59:47.250 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:47.247+0000 7fd2e3400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd2e4108440 0x7fd2e41016d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:47.250 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:47.247+0000 7fd2e3400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd2e4108440 0x7fd2e41016d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:37038/0 (socket says 172.21.15.72:37038) 2024-04-26T21:59:47.250 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:47.247+0000 7fd2e3400640 1 -- 172.21.15.72:0/3198121965 learned_addr learned my addr 172.21.15.72:0/3198121965 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:59:47.251 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:47.247+0000 7fd2e2a00640 1 --2- 172.21.15.72:0/3198121965 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd2e41031e0 0x7fd2e4102b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:47.251 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:47.247+0000 7fd2e3400640 1 -- 172.21.15.72:0/3198121965 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd2e4105b10 msgr2=0x7fd2e4101190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:47.251 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:47.247+0000 7fd2e3400640 1 --2- 172.21.15.72:0/3198121965 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd2e4105b10 0x7fd2e4101190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:47.251 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:47.247+0000 7fd2e3400640 1 -- 172.21.15.72:0/3198121965 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd2e41031e0 msgr2=0x7fd2e4102b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:47.251 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:47.247+0000 7fd2e3400640 1 --2- 172.21.15.72:0/3198121965 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd2e41031e0 0x7fd2e4102b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:47.251 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:47.247+0000 7fd2e3400640 1 -- 172.21.15.72:0/3198121965 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2e4101f60 con 0x7fd2e4108440 2024-04-26T21:59:47.251 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:47.247+0000 7fd2e2a00640 1 --2- 172.21.15.72:0/3198121965 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd2e41031e0 0x7fd2e4102b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:59:47.251 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:47.248+0000 7fd2e3400640 1 --2- 172.21.15.72:0/3198121965 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd2e4108440 0x7fd2e41016d0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fd2dc02aeb0 tx=0x7fd2dc003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:47.251 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:47.248+0000 7fd2dbe00640 1 -- 172.21.15.72:0/3198121965 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2dc032ed0 con 0x7fd2e4108440 2024-04-26T21:59:47.251 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:47.248+0000 7fd2dbe00640 1 -- 172.21.15.72:0/3198121965 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd2dc003e10 con 0x7fd2e4108440 2024-04-26T21:59:47.251 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:47.248+0000 7fd2e3e00640 1 -- 172.21.15.72:0/3198121965 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2e41a2a00 con 0x7fd2e4108440 2024-04-26T21:59:47.251 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:47.248+0000 7fd2dbe00640 1 -- 172.21.15.72:0/3198121965 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2dc030cd0 con 0x7fd2e4108440 2024-04-26T21:59:47.252 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:47.248+0000 7fd2e3e00640 1 -- 172.21.15.72:0/3198121965 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2e41a2f50 con 0x7fd2e4108440 2024-04-26T21:59:47.252 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:47.249+0000 7fd2e3e00640 1 -- 172.21.15.72:0/3198121965 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2e40fe6a0 con 0x7fd2e4108440 2024-04-26T21:59:47.252 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:47.251+0000 7fd2dbe00640 1 -- 172.21.15.72:0/3198121965 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fd2dc004240 con 0x7fd2e4108440 2024-04-26T21:59:47.253 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:47.252+0000 7fd2dbe00640 1 --2- 172.21.15.72:0/3198121965 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fd2c80a3c40 0x7fd2c80a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:47.253 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:47.252+0000 7fd2dbe00640 1 -- 172.21.15.72:0/3198121965 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7fd2dc0f1c60 con 0x7fd2e4108440 2024-04-26T21:59:47.253 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:47.253+0000 7fd2e2a00640 1 --2- 172.21.15.72:0/3198121965 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fd2c80a3c40 0x7fd2c80a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:47.253 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:47.253+0000 7fd2e2a00640 1 --2- 172.21.15.72:0/3198121965 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fd2c80a3c40 0x7fd2c80a6100 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fd2d4009b40 tx=0x7fd2d4005e50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:47.256 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:47.256+0000 7fd2dbe00640 1 -- 172.21.15.72:0/3198121965 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fd2dc0be570 con 0x7fd2e4108440 2024-04-26T21:59:47.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:59:47 smithi076 ceph-mon[31823]: pgmap v579: 97 pgs: 97 active+clean; 1.1 MiB data, 656 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-04-26T21:59:47.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:59:47 smithi076 ceph-mon[31823]: from='client.? 172.21.15.72:0/66295150' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T21:59:47.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:59:47 smithi076 ceph-mon[31823]: from='client.? 172.21.15.72:0/1794679106' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T21:59:47.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:59:47 smithi076 ceph-mon[31823]: from='client.? 172.21.15.72:0/1442590454' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T21:59:47.321 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:59:47 smithi072 ceph-mon[28138]: pgmap v579: 97 pgs: 97 active+clean; 1.1 MiB data, 656 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-04-26T21:59:47.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:59:47 smithi072 ceph-mon[28138]: from='client.? 172.21.15.72:0/66295150' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T21:59:47.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:59:47 smithi072 ceph-mon[28138]: from='client.? 172.21.15.72:0/1794679106' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T21:59:47.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:59:47 smithi072 ceph-mon[28138]: from='client.? 172.21.15.72:0/1442590454' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T21:59:47.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:59:47 smithi164 ceph-mon[31916]: pgmap v579: 97 pgs: 97 active+clean; 1.1 MiB data, 656 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-04-26T21:59:47.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:59:47 smithi164 ceph-mon[31916]: from='client.? 172.21.15.72:0/66295150' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T21:59:47.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:59:47 smithi164 ceph-mon[31916]: from='client.? 172.21.15.72:0/1794679106' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T21:59:47.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:59:47 smithi164 ceph-mon[31916]: from='client.? 172.21.15.72:0/1442590454' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T21:59:47.471 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:47.471+0000 7fd2e3e00640 1 -- 172.21.15.72:0/3198121965 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fd2e402dad0 con 0x7fd2e4108440 2024-04-26T21:59:47.472 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:47.472+0000 7fd2dbe00640 1 -- 172.21.15.72:0/3198121965 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v91) ==== 81+0+591 (secure 0 0 0) 0x7fd2dc0bde70 con 0x7fd2e4108440 2024-04-26T21:59:47.472 INFO:teuthology.orchestra.run.smithi072.stdout:172.21.15.72:0/416277784 2024-04-27T21:43:09.602791+0000 2024-04-26T21:59:47.472 INFO:teuthology.orchestra.run.smithi072.stdout:172.21.15.72:6800/4010358672 2024-04-27T21:43:09.602791+0000 2024-04-26T21:59:47.472 INFO:teuthology.orchestra.run.smithi072.stdout:172.21.15.72:6801/1012617371 2024-04-27T21:42:34.684837+0000 2024-04-26T21:59:47.472 INFO:teuthology.orchestra.run.smithi072.stdout:172.21.15.72:6801/4010358672 2024-04-27T21:43:09.602791+0000 2024-04-26T21:59:47.472 INFO:teuthology.orchestra.run.smithi072.stdout:172.21.15.72:0/3555448179 2024-04-27T21:43:09.602791+0000 2024-04-26T21:59:47.473 INFO:teuthology.orchestra.run.smithi072.stdout:172.21.15.72:6800/1012617371 2024-04-27T21:42:34.684837+0000 2024-04-26T21:59:47.473 INFO:teuthology.orchestra.run.smithi072.stdout:172.21.15.72:0/2920378761 2024-04-27T21:42:34.684837+0000 2024-04-26T21:59:47.473 INFO:teuthology.orchestra.run.smithi072.stdout:172.21.15.72:0/1933740041 2024-04-27T21:43:09.602791+0000 2024-04-26T21:59:47.473 INFO:teuthology.orchestra.run.smithi072.stdout:172.21.15.72:0/3225295775 2024-04-27T21:42:34.684837+0000 2024-04-26T21:59:47.473 INFO:teuthology.orchestra.run.smithi072.stdout:172.21.15.72:0/2326016180 2024-04-27T21:42:34.684837+0000 2024-04-26T21:59:47.473 INFO:teuthology.orchestra.run.smithi072.stderr:listed 10 entries 2024-04-26T21:59:47.475 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:47.475+0000 7fd2e3e00640 1 -- 172.21.15.72:0/3198121965 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fd2c80a3c40 msgr2=0x7fd2c80a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:47.475 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:47.475+0000 7fd2e3e00640 1 --2- 172.21.15.72:0/3198121965 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fd2c80a3c40 0x7fd2c80a6100 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fd2d4009b40 tx=0x7fd2d4005e50 comp rx=0 tx=0).stop 2024-04-26T21:59:47.475 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:47.475+0000 7fd2e3e00640 1 -- 172.21.15.72:0/3198121965 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd2e4108440 msgr2=0x7fd2e41016d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:47.475 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:47.475+0000 7fd2e3e00640 1 --2- 172.21.15.72:0/3198121965 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd2e4108440 0x7fd2e41016d0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fd2dc02aeb0 tx=0x7fd2dc003c20 comp rx=0 tx=0).stop 2024-04-26T21:59:47.475 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:47.475+0000 7fd2e3e00640 1 -- 172.21.15.72:0/3198121965 shutdown_connections 2024-04-26T21:59:47.475 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:47.475+0000 7fd2e3e00640 1 --2- 172.21.15.72:0/3198121965 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd2e4108440 0x7fd2e41016d0 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:47.475 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:47.475+0000 7fd2e3e00640 1 --2- 172.21.15.72:0/3198121965 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fd2c80a3c40 0x7fd2c80a6100 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:47.475 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:47.475+0000 7fd2e3e00640 1 --2- 172.21.15.72:0/3198121965 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd2e4105b10 0x7fd2e4101190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:47.476 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:47.475+0000 7fd2e3e00640 1 --2- 172.21.15.72:0/3198121965 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd2e41031e0 0x7fd2e4102b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:47.476 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:47.475+0000 7fd2e3e00640 1 -- 172.21.15.72:0/3198121965 >> 172.21.15.72:0/3198121965 conn(0x7fd2e40fcbb0 msgr2=0x7fd2e4103b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:47.476 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:47.475+0000 7fd2e3e00640 1 -- 172.21.15.72:0/3198121965 shutdown_connections 2024-04-26T21:59:47.476 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:47.475+0000 7fd2e3e00640 1 -- 172.21.15.72:0/3198121965 wait complete. 2024-04-26T21:59:47.495 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi072.front.sepia.ceph.com... 2024-04-26T21:59:47.495 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:59:47.495 DEBUG:teuthology.orchestra.run.smithi072:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-26T21:59:47.572 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T21:59:47.579 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi072.stderr:ceph-fuse[81138]: fuse finished with error 0 and tester_r 0 2024-04-26T21:59:48.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:59:48 smithi072 ceph-mon[28138]: from='client.? 172.21.15.72:0/3198121965' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-26T21:59:48.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:59:48 smithi072 ceph-mon[28138]: pgmap v580: 97 pgs: 97 active+clean; 1.2 MiB data, 656 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-26T21:59:48.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:59:48 smithi164 ceph-mon[31916]: from='client.? 172.21.15.72:0/3198121965' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-26T21:59:48.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:59:48 smithi164 ceph-mon[31916]: pgmap v580: 97 pgs: 97 active+clean; 1.2 MiB data, 656 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-26T21:59:48.564 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:59:48 smithi076 ceph-mon[31823]: from='client.? 172.21.15.72:0/3198121965' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-26T21:59:48.564 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:59:48 smithi076 ceph-mon[31823]: pgmap v580: 97 pgs: 97 active+clean; 1.2 MiB data, 656 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-26T21:59:51.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:59:51 smithi076 ceph-mon[31823]: pgmap v581: 97 pgs: 97 active+clean; 1.2 MiB data, 656 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 1023 B/s wr, 13 op/s 2024-04-26T21:59:51.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:59:51 smithi072 ceph-mon[28138]: pgmap v581: 97 pgs: 97 active+clean; 1.2 MiB data, 656 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 1023 B/s wr, 13 op/s 2024-04-26T21:59:51.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:59:51 smithi164 ceph-mon[31916]: pgmap v581: 97 pgs: 97 active+clean; 1.2 MiB data, 656 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 1023 B/s wr, 13 op/s 2024-04-26T21:59:53.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:59:53 smithi076 ceph-mon[31823]: pgmap v582: 97 pgs: 97 active+clean; 1.2 MiB data, 656 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1023 B/s wr, 9 op/s 2024-04-26T21:59:53.321 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:59:53 smithi072 ceph-mon[28138]: pgmap v582: 97 pgs: 97 active+clean; 1.2 MiB data, 656 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1023 B/s wr, 9 op/s 2024-04-26T21:59:53.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:59:53 smithi164 ceph-mon[31916]: pgmap v582: 97 pgs: 97 active+clean; 1.2 MiB data, 656 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1023 B/s wr, 9 op/s 2024-04-26T21:59:53.674 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi072.front.sepia.ceph.com 2024-04-26T21:59:53.674 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:59:53.675 DEBUG:teuthology.orchestra.run.smithi072:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-26T21:59:53.738 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:59:53.738 DEBUG:teuthology.orchestra.run.smithi072:> ip netns list 2024-04-26T21:59:53.791 INFO:teuthology.orchestra.run.smithi072.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-26T21:59:53.792 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:59:53.793 DEBUG:teuthology.orchestra.run.smithi072:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T21:59:53.882 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:59:53.882 DEBUG:teuthology.orchestra.run.smithi072:> sudo ip link delete ceph-brx 2024-04-26T21:59:54.003 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-04-26T21:59:54.014 INFO:tasks.cephadm:Teardown begin 2024-04-26T21:59:54.015 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-26T21:59:54.015 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:59:54.199 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:54.198+0000 7f4bb0000640 1 -- 172.21.15.72:0/1605367474 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4ba810ab40 msgr2=0x7f4ba81113d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:54.199 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:54.198+0000 7f4bb0000640 1 --2- 172.21.15.72:0/1605367474 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4ba810ab40 0x7f4ba81113d0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f4b98009900 tx=0x7f4b9802a3f0 comp rx=0 tx=0).stop 2024-04-26T21:59:54.199 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:54.199+0000 7f4bb0000640 1 -- 172.21.15.72:0/1605367474 shutdown_connections 2024-04-26T21:59:54.199 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:54.199+0000 7f4bb0000640 1 --2- 172.21.15.72:0/1605367474 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4ba810ab40 0x7f4ba81113d0 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:54.199 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:54.199+0000 7f4bb0000640 1 --2- 172.21.15.72:0/1605367474 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4ba810a180 0x7f4ba810a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:54.199 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:54.199+0000 7f4bb0000640 1 --2- 172.21.15.72:0/1605367474 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4ba8108f90 0x7f4ba81093b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:54.199 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:54.199+0000 7f4bb0000640 1 -- 172.21.15.72:0/1605367474 >> 172.21.15.72:0/1605367474 conn(0x7f4ba8077f40 msgr2=0x7f4ba807a360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:54.200 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:54.199+0000 7f4bb0000640 1 -- 172.21.15.72:0/1605367474 shutdown_connections 2024-04-26T21:59:54.200 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:54.199+0000 7f4bb0000640 1 -- 172.21.15.72:0/1605367474 wait complete. 2024-04-26T21:59:54.202 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:54.200+0000 7f4bb0000640 1 Processor -- start 2024-04-26T21:59:54.202 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:54.200+0000 7f4bb0000640 1 -- start start 2024-04-26T21:59:54.202 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:54.200+0000 7f4bb0000640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4ba8108f90 0x7f4ba819f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:54.202 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:54.200+0000 7f4bb0000640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4ba810a180 0x7f4ba819f570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:54.202 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:54.200+0000 7f4bb0000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4ba810ab40 0x7f4ba81a65e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:54.202 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:54.200+0000 7f4bb0000640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f4ba8114a10 con 0x7f4ba810a180 2024-04-26T21:59:54.202 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:54.200+0000 7f4bb0000640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f4ba8114890 con 0x7f4ba8108f90 2024-04-26T21:59:54.202 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:54.200+0000 7f4bb0000640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f4ba8114b90 con 0x7f4ba810ab40 2024-04-26T21:59:54.203 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:54.200+0000 7f4bacc00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4ba810a180 0x7f4ba819f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:54.203 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:54.200+0000 7f4bacc00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4ba810a180 0x7f4ba819f570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:48122/0 (socket says 172.21.15.72:48122) 2024-04-26T21:59:54.203 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:54.200+0000 7f4bacc00640 1 -- 172.21.15.72:0/4079284125 learned_addr learned my addr 172.21.15.72:0/4079284125 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:59:54.203 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:54.201+0000 7f4bae000640 1 --2- 172.21.15.72:0/4079284125 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4ba810ab40 0x7f4ba81a65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:54.203 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:54.201+0000 7f4bacc00640 1 -- 172.21.15.72:0/4079284125 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4ba810ab40 msgr2=0x7f4ba81a65e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:54.203 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:54.201+0000 7f4bacc00640 1 --2- 172.21.15.72:0/4079284125 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4ba810ab40 0x7f4ba81a65e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:54.203 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:54.201+0000 7f4bad600640 1 --2- 172.21.15.72:0/4079284125 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4ba8108f90 0x7f4ba819f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:54.203 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:54.201+0000 7f4bacc00640 1 -- 172.21.15.72:0/4079284125 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4ba8108f90 msgr2=0x7f4ba819f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:54.203 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:54.201+0000 7f4bacc00640 1 --2- 172.21.15.72:0/4079284125 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4ba8108f90 0x7f4ba819f030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:54.203 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:54.201+0000 7f4bacc00640 1 -- 172.21.15.72:0/4079284125 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ba81a6c50 con 0x7f4ba810a180 2024-04-26T21:59:54.203 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:54.201+0000 7f4bacc00640 1 --2- 172.21.15.72:0/4079284125 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4ba810a180 0x7f4ba819f570 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f4b9400b4a0 tx=0x7f4b9400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:54.204 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:54.201+0000 7f4ba6000640 1 -- 172.21.15.72:0/4079284125 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b94014070 con 0x7f4ba810a180 2024-04-26T21:59:54.204 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:54.201+0000 7f4ba6000640 1 -- 172.21.15.72:0/4079284125 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4b94010aa0 con 0x7f4ba810a180 2024-04-26T21:59:54.204 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:54.201+0000 7f4bb0000640 1 -- 172.21.15.72:0/4079284125 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ba81a6f40 con 0x7f4ba810a180 2024-04-26T21:59:54.204 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:54.201+0000 7f4ba6000640 1 -- 172.21.15.72:0/4079284125 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b94017b70 con 0x7f4ba810a180 2024-04-26T21:59:54.204 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:54.201+0000 7f4bb0000640 1 -- 172.21.15.72:0/4079284125 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ba81a7480 con 0x7f4ba810a180 2024-04-26T21:59:54.204 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:54.203+0000 7f4bb0000640 1 -- 172.21.15.72:0/4079284125 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ba81102b0 con 0x7f4ba810a180 2024-04-26T21:59:54.205 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:54.205+0000 7f4ba6000640 1 -- 172.21.15.72:0/4079284125 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f4b94017070 con 0x7f4ba810a180 2024-04-26T21:59:54.206 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:54.207+0000 7f4ba6000640 1 --2- 172.21.15.72:0/4079284125 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f4b8c0a3c40 0x7f4b8c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:54.207 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:54.207+0000 7f4ba6000640 1 -- 172.21.15.72:0/4079284125 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7f4b940cfcf0 con 0x7f4ba810a180 2024-04-26T21:59:54.207 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:54.207+0000 7f4bad600640 1 --2- 172.21.15.72:0/4079284125 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f4b8c0a3c40 0x7f4b8c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:54.207 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:54.208+0000 7f4bad600640 1 --2- 172.21.15.72:0/4079284125 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f4b8c0a3c40 0x7f4b8c0a6100 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f4ba8109fe0 tx=0x7f4ba00082b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:54.212 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:54.212+0000 7f4ba6000640 1 -- 172.21.15.72:0/4079284125 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f4b9409c5a0 con 0x7f4ba810a180 2024-04-26T21:59:54.456 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:54.456+0000 7f4bb0000640 1 -- 172.21.15.72:0/4079284125 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4ba802dad0 con 0x7f4ba810a180 2024-04-26T21:59:54.458 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:54.458+0000 7f4ba6000640 1 -- 172.21.15.72:0/4079284125 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11931 (secure 0 0 0) 0x7f4b9409bea0 con 0x7f4ba810a180 2024-04-26T21:59:54.458 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T21:59:54.458 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6835/1320381959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6834","nonce":1320381959},{"type":"v1","addr":"172.21.15.72:6835","nonce":1320381959}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6837/2142200512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6836","nonce":2142200512},{"type":"v1","addr":"172.21.15.72:6837","nonce":2142200512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T21:52:50.567857+0000","modified":"2024-04-26T21:55:22.894383+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34517,"mds_1":34472,"mds_2":34442,"mds_3":34406,"mds_4":24463},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14652":{"gid":14652,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.72:6839/1312898281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6838","nonce":1312898281},{"type":"v1","addr":"172.21.15.72:6839","nonce":1312898281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14664":{"gid":14664,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.72:6841/3257692339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6840","nonce":3257692339},{"type":"v1","addr":"172.21.15.72:6841","nonce":3257692339}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24388":{"gid":24388,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6833/4236668465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":4236668465},{"type":"v1","addr":"172.21.15.164:6833","nonce":4236668465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6835/2391114169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":2391114169},{"type":"v1","addr":"172.21.15.164:6835","nonce":2391114169}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6837/3585829992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":3585829992},{"type":"v1","addr":"172.21.15.164:6837","nonce":3585829992}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.164:6839/1636226820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":1636226820},{"type":"v1","addr":"172.21.15.164:6839","nonce":1636226820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34406":{"gid":34406,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":15,"addr":"172.21.15.76:6833/3869483750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":3869483750},{"type":"v1","addr":"172.21.15.76:6833","nonce":3869483750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34442":{"gid":34442,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.76:6835/3517429435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":3517429435},{"type":"v1","addr":"172.21.15.76:6835","nonce":3517429435}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34472":{"gid":34472,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.76:6837/1150678991","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":1150678991},{"type":"v1","addr":"172.21.15.76:6837","nonce":1150678991}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34517":{"gid":34517,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.76:6839/327145636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6838","nonce":327145636},{"type":"v1","addr":"172.21.15.76:6839","nonce":327145636}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34517,"qdb_cluster":[34472,34517,34442,34406,24463]},"id":1}]} 2024-04-26T21:59:54.459 INFO:teuthology.orchestra.run.smithi072.stderr:dumped fsmap epoch 27 2024-04-26T21:59:54.460 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:54.460+0000 7f4bb0000640 1 -- 172.21.15.72:0/4079284125 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f4b8c0a3c40 msgr2=0x7f4b8c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:54.460 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:54.460+0000 7f4bb0000640 1 --2- 172.21.15.72:0/4079284125 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f4b8c0a3c40 0x7f4b8c0a6100 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f4ba8109fe0 tx=0x7f4ba00082b0 comp rx=0 tx=0).stop 2024-04-26T21:59:54.460 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:54.460+0000 7f4bb0000640 1 -- 172.21.15.72:0/4079284125 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4ba810a180 msgr2=0x7f4ba819f570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:54.461 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:54.460+0000 7f4bb0000640 1 --2- 172.21.15.72:0/4079284125 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4ba810a180 0x7f4ba819f570 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f4b9400b4a0 tx=0x7f4b9400b960 comp rx=0 tx=0).stop 2024-04-26T21:59:54.461 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:54.460+0000 7f4bb0000640 1 -- 172.21.15.72:0/4079284125 shutdown_connections 2024-04-26T21:59:54.461 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:54.460+0000 7f4bb0000640 1 --2- 172.21.15.72:0/4079284125 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f4b8c0a3c40 0x7f4b8c0a6100 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:54.461 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:54.460+0000 7f4bb0000640 1 --2- 172.21.15.72:0/4079284125 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4ba810ab40 0x7f4ba81a65e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:54.461 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:54.460+0000 7f4bb0000640 1 --2- 172.21.15.72:0/4079284125 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4ba810a180 0x7f4ba819f570 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:54.461 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:54.460+0000 7f4bb0000640 1 --2- 172.21.15.72:0/4079284125 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4ba8108f90 0x7f4ba819f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:54.461 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:54.460+0000 7f4bb0000640 1 -- 172.21.15.72:0/4079284125 >> 172.21.15.72:0/4079284125 conn(0x7f4ba8077f40 msgr2=0x7f4ba810d480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:54.461 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:54.460+0000 7f4bb0000640 1 -- 172.21.15.72:0/4079284125 shutdown_connections 2024-04-26T21:59:54.461 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:54.460+0000 7f4bb0000640 1 -- 172.21.15.72:0/4079284125 wait complete. 2024-04-26T21:59:54.481 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:59:54.687 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:54.687+0000 7f5a06200640 1 -- 172.21.15.72:0/2327961543 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5a00104530 msgr2=0x7f5a0010adc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:54.687 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:54.687+0000 7f5a06200640 1 --2- 172.21.15.72:0/2327961543 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5a00104530 0x7f5a0010adc0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f59f0009900 tx=0x7f59f002a410 comp rx=0 tx=0).stop 2024-04-26T21:59:54.688 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:54.687+0000 7f5a06200640 1 -- 172.21.15.72:0/2327961543 shutdown_connections 2024-04-26T21:59:54.688 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:54.687+0000 7f5a06200640 1 --2- 172.21.15.72:0/2327961543 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5a00104530 0x7f5a0010adc0 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:54.688 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:54.687+0000 7f5a06200640 1 --2- 172.21.15.72:0/2327961543 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5a00103b70 0x7f5a00103ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:54.688 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:54.687+0000 7f5a06200640 1 --2- 172.21.15.72:0/2327961543 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5a00102980 0x7f5a00102da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:54.688 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:54.687+0000 7f5a06200640 1 -- 172.21.15.72:0/2327961543 >> 172.21.15.72:0/2327961543 conn(0x7f5a000fdf10 msgr2=0x7f5a00100330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:54.688 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:54.687+0000 7f5a06200640 1 -- 172.21.15.72:0/2327961543 shutdown_connections 2024-04-26T21:59:54.688 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:54.687+0000 7f5a06200640 1 -- 172.21.15.72:0/2327961543 wait complete. 2024-04-26T21:59:54.690 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:54.688+0000 7f5a06200640 1 Processor -- start 2024-04-26T21:59:54.690 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:54.688+0000 7f5a06200640 1 -- start start 2024-04-26T21:59:54.690 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:54.688+0000 7f5a06200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5a00102980 0x7f5a000766e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:54.690 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:54.689+0000 7f5a06200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5a00103b70 0x7f5a00074d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:54.690 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:54.689+0000 7f5a06200640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5a00104530 0x7f5a00075260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:54.691 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:54.689+0000 7f5a06200640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f5a0010e340 con 0x7f5a00104530 2024-04-26T21:59:54.691 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:54.689+0000 7f5a06200640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f5a0010e1c0 con 0x7f5a00103b70 2024-04-26T21:59:54.691 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:54.689+0000 7f5a06200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f5a0010e4c0 con 0x7f5a00102980 2024-04-26T21:59:54.691 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:54.689+0000 7f59ffe00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5a00104530 0x7f5a00075260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:54.691 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:54.689+0000 7f59ffe00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5a00104530 0x7f5a00075260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:48138/0 (socket says 172.21.15.72:48138) 2024-04-26T21:59:54.691 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:54.689+0000 7f59ffe00640 1 -- 172.21.15.72:0/2529063980 learned_addr learned my addr 172.21.15.72:0/2529063980 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:59:54.691 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:54.689+0000 7f59fea00640 1 --2- 172.21.15.72:0/2529063980 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5a00103b70 0x7f5a00074d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:54.691 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:54.689+0000 7f59ff400640 1 --2- 172.21.15.72:0/2529063980 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5a00102980 0x7f5a000766e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:54.691 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:54.689+0000 7f59ffe00640 1 -- 172.21.15.72:0/2529063980 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5a00102980 msgr2=0x7f5a000766e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:54.691 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:54.689+0000 7f59ffe00640 1 --2- 172.21.15.72:0/2529063980 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5a00102980 0x7f5a000766e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:54.691 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:54.689+0000 7f59ffe00640 1 -- 172.21.15.72:0/2529063980 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5a00103b70 msgr2=0x7f5a00074d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:54.691 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:54.689+0000 7f59ffe00640 1 --2- 172.21.15.72:0/2529063980 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5a00103b70 0x7f5a00074d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:54.691 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:54.689+0000 7f59ffe00640 1 -- 172.21.15.72:0/2529063980 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a00075af0 con 0x7f5a00104530 2024-04-26T21:59:54.691 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:54.690+0000 7f59ffe00640 1 --2- 172.21.15.72:0/2529063980 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5a00104530 0x7f5a00075260 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f59f0005940 tx=0x7f59f00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:54.692 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:54.690+0000 7f59f7e00640 1 -- 172.21.15.72:0/2529063980 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59f003f030 con 0x7f5a00104530 2024-04-26T21:59:54.692 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:54.690+0000 7f5a06200640 1 -- 172.21.15.72:0/2529063980 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a001a2aa0 con 0x7f5a00104530 2024-04-26T21:59:54.692 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:54.690+0000 7f59f7e00640 1 -- 172.21.15.72:0/2529063980 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f59f0003c90 con 0x7f5a00104530 2024-04-26T21:59:54.692 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:54.690+0000 7f59f7e00640 1 -- 172.21.15.72:0/2529063980 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59f0035070 con 0x7f5a00104530 2024-04-26T21:59:54.692 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:54.690+0000 7f5a06200640 1 -- 172.21.15.72:0/2529063980 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a001a3040 con 0x7f5a00104530 2024-04-26T21:59:54.692 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:54.691+0000 7f5a06200640 1 -- 172.21.15.72:0/2529063980 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a00109d90 con 0x7f5a00104530 2024-04-26T21:59:54.693 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:54.693+0000 7f59f7e00640 1 -- 172.21.15.72:0/2529063980 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f59f0042030 con 0x7f5a00104530 2024-04-26T21:59:54.694 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:54.695+0000 7f59f7e00640 1 --2- 172.21.15.72:0/2529063980 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f59e40a3cc0 0x7f59e40a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:54.695 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:54.695+0000 7f59f7e00640 1 -- 172.21.15.72:0/2529063980 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7f59f00f0ed0 con 0x7f5a00104530 2024-04-26T21:59:54.695 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:54.695+0000 7f59ff400640 1 --2- 172.21.15.72:0/2529063980 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f59e40a3cc0 0x7f59e40a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:54.695 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:54.696+0000 7f59ff400640 1 --2- 172.21.15.72:0/2529063980 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f59e40a3cc0 0x7f59e40a6180 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f59f800a300 tx=0x7f59f800a290 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:54.699 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:54.699+0000 7f59f7e00640 1 -- 172.21.15.72:0/2529063980 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f59f00bd670 con 0x7f5a00104530 2024-04-26T21:59:54.948 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:54.948+0000 7f5a06200640 1 -- 172.21.15.72:0/2529063980 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5a00063530 con 0x7f5a00104530 2024-04-26T21:59:54.950 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:54.950+0000 7f59f7e00640 1 -- 172.21.15.72:0/2529063980 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11931 (secure 0 0 0) 0x7f59f00bcf70 con 0x7f5a00104530 2024-04-26T21:59:54.950 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T21:59:54.951 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6835/1320381959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6834","nonce":1320381959},{"type":"v1","addr":"172.21.15.72:6835","nonce":1320381959}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6837/2142200512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6836","nonce":2142200512},{"type":"v1","addr":"172.21.15.72:6837","nonce":2142200512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T21:52:50.567857+0000","modified":"2024-04-26T21:55:22.894383+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34517,"mds_1":34472,"mds_2":34442,"mds_3":34406,"mds_4":24463},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14652":{"gid":14652,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.72:6839/1312898281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6838","nonce":1312898281},{"type":"v1","addr":"172.21.15.72:6839","nonce":1312898281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14664":{"gid":14664,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.72:6841/3257692339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6840","nonce":3257692339},{"type":"v1","addr":"172.21.15.72:6841","nonce":3257692339}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24388":{"gid":24388,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6833/4236668465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":4236668465},{"type":"v1","addr":"172.21.15.164:6833","nonce":4236668465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6835/2391114169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":2391114169},{"type":"v1","addr":"172.21.15.164:6835","nonce":2391114169}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6837/3585829992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":3585829992},{"type":"v1","addr":"172.21.15.164:6837","nonce":3585829992}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.164:6839/1636226820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":1636226820},{"type":"v1","addr":"172.21.15.164:6839","nonce":1636226820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34406":{"gid":34406,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":15,"addr":"172.21.15.76:6833/3869483750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":3869483750},{"type":"v1","addr":"172.21.15.76:6833","nonce":3869483750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34442":{"gid":34442,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.76:6835/3517429435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":3517429435},{"type":"v1","addr":"172.21.15.76:6835","nonce":3517429435}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34472":{"gid":34472,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.76:6837/1150678991","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":1150678991},{"type":"v1","addr":"172.21.15.76:6837","nonce":1150678991}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34517":{"gid":34517,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.76:6839/327145636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6838","nonce":327145636},{"type":"v1","addr":"172.21.15.76:6839","nonce":327145636}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34517,"qdb_cluster":[34472,34517,34442,34406,24463]},"id":1}]} 2024-04-26T21:59:54.951 INFO:teuthology.orchestra.run.smithi072.stderr:dumped fsmap epoch 27 2024-04-26T21:59:54.953 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:54.953+0000 7f5a06200640 1 -- 172.21.15.72:0/2529063980 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f59e40a3cc0 msgr2=0x7f59e40a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:54.953 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:54.953+0000 7f5a06200640 1 --2- 172.21.15.72:0/2529063980 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f59e40a3cc0 0x7f59e40a6180 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f59f800a300 tx=0x7f59f800a290 comp rx=0 tx=0).stop 2024-04-26T21:59:54.953 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:54.953+0000 7f5a06200640 1 -- 172.21.15.72:0/2529063980 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5a00104530 msgr2=0x7f5a00075260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:54.954 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:54.953+0000 7f5a06200640 1 --2- 172.21.15.72:0/2529063980 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5a00104530 0x7f5a00075260 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f59f0005940 tx=0x7f59f00042f0 comp rx=0 tx=0).stop 2024-04-26T21:59:54.954 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:54.953+0000 7f5a06200640 1 -- 172.21.15.72:0/2529063980 shutdown_connections 2024-04-26T21:59:54.954 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:54.953+0000 7f5a06200640 1 --2- 172.21.15.72:0/2529063980 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5a00104530 0x7f5a00075260 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:54.954 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:54.953+0000 7f5a06200640 1 --2- 172.21.15.72:0/2529063980 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5a00103b70 0x7f5a00074d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:54.954 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:54.953+0000 7f5a06200640 1 --2- 172.21.15.72:0/2529063980 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f59e40a3cc0 0x7f59e40a6180 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:54.954 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:54.953+0000 7f5a06200640 1 --2- 172.21.15.72:0/2529063980 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5a00102980 0x7f5a000766e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:54.954 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:54.953+0000 7f5a06200640 1 -- 172.21.15.72:0/2529063980 >> 172.21.15.72:0/2529063980 conn(0x7f5a000fdf10 msgr2=0x7f5a00106f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:54.954 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:54.953+0000 7f5a06200640 1 -- 172.21.15.72:0/2529063980 shutdown_connections 2024-04-26T21:59:54.954 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:54.953+0000 7f5a06200640 1 -- 172.21.15.72:0/2529063980 wait complete. 2024-04-26T21:59:54.974 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:59:55.175 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:55.174+0000 7f57f9600640 1 -- 172.21.15.72:0/585462815 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f57f41034a0 msgr2=0x7f57f4105890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:55.175 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:55.174+0000 7f57f9600640 1 --2- 172.21.15.72:0/585462815 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f57f41034a0 0x7f57f4105890 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f57e4009900 tx=0x7f57e402a3f0 comp rx=0 tx=0).stop 2024-04-26T21:59:55.175 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:55.175+0000 7f57f9600640 1 -- 172.21.15.72:0/585462815 shutdown_connections 2024-04-26T21:59:55.175 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:55.175+0000 7f57f9600640 1 --2- 172.21.15.72:0/585462815 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f57f4108700 0x7f57f410aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:55.175 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:55.175+0000 7f57f9600640 1 --2- 172.21.15.72:0/585462815 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f57f4105dd0 0x7f57f41081c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:55.175 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:55.175+0000 7f57f9600640 1 --2- 172.21.15.72:0/585462815 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f57f41034a0 0x7f57f4105890 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:55.175 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:55.175+0000 7f57f9600640 1 -- 172.21.15.72:0/585462815 >> 172.21.15.72:0/585462815 conn(0x7f57f40fcdf0 msgr2=0x7f57f40ff210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:55.176 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:55.175+0000 7f57f9600640 1 -- 172.21.15.72:0/585462815 shutdown_connections 2024-04-26T21:59:55.176 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:55.175+0000 7f57f9600640 1 -- 172.21.15.72:0/585462815 wait complete. 2024-04-26T21:59:55.177 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:55.176+0000 7f57f9600640 1 Processor -- start 2024-04-26T21:59:55.177 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:55.176+0000 7f57f9600640 1 -- start start 2024-04-26T21:59:55.177 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:55.176+0000 7f57f9600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f57f41034a0 0x7f57f419ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:55.178 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:55.176+0000 7f57f9600640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f57f4105dd0 0x7f57f419b290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:55.178 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:55.176+0000 7f57f9600640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f57f4108700 0x7f57f41a2300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:55.178 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:55.176+0000 7f57f9600640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f57f410e410 con 0x7f57f4108700 2024-04-26T21:59:55.178 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:55.176+0000 7f57f9600640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f57f410e290 con 0x7f57f4105dd0 2024-04-26T21:59:55.178 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:55.176+0000 7f57f9600640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f57f410e590 con 0x7f57f41034a0 2024-04-26T21:59:55.178 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:55.177+0000 7f57f3400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f57f4108700 0x7f57f41a2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:55.178 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:55.177+0000 7f57f3400640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f57f4108700 0x7f57f41a2300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:48158/0 (socket says 172.21.15.72:48158) 2024-04-26T21:59:55.178 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:55.177+0000 7f57f3400640 1 -- 172.21.15.72:0/3590697627 learned_addr learned my addr 172.21.15.72:0/3590697627 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:59:55.178 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:55.177+0000 7f57f2a00640 1 --2- 172.21.15.72:0/3590697627 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f57f41034a0 0x7f57f419ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:55.178 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:55.177+0000 7f57f2000640 1 --2- 172.21.15.72:0/3590697627 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f57f4105dd0 0x7f57f419b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:55.178 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:55.177+0000 7f57f3400640 1 -- 172.21.15.72:0/3590697627 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f57f41034a0 msgr2=0x7f57f419ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:55.179 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:55.177+0000 7f57f3400640 1 --2- 172.21.15.72:0/3590697627 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f57f41034a0 0x7f57f419ad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:55.179 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:55.177+0000 7f57f3400640 1 -- 172.21.15.72:0/3590697627 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f57f4105dd0 msgr2=0x7f57f419b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:55.179 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:55.177+0000 7f57f3400640 1 --2- 172.21.15.72:0/3590697627 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f57f4105dd0 0x7f57f419b290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:55.179 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:55.177+0000 7f57f3400640 1 -- 172.21.15.72:0/3590697627 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57f41a2970 con 0x7f57f4108700 2024-04-26T21:59:55.179 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:55.178+0000 7f57f3400640 1 --2- 172.21.15.72:0/3590697627 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f57f4108700 0x7f57f41a2300 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f57ec00b970 tx=0x7f57ec00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:55.180 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:55.178+0000 7f57eb400640 1 -- 172.21.15.72:0/3590697627 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57ec019070 con 0x7f57f4108700 2024-04-26T21:59:55.180 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:55.178+0000 7f57eb400640 1 -- 172.21.15.72:0/3590697627 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f57ec014440 con 0x7f57f4108700 2024-04-26T21:59:55.180 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:55.178+0000 7f57f9600640 1 -- 172.21.15.72:0/3590697627 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57f41a2c60 con 0x7f57f4108700 2024-04-26T21:59:55.180 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:55.178+0000 7f57eb400640 1 -- 172.21.15.72:0/3590697627 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57ec0075d0 con 0x7f57f4108700 2024-04-26T21:59:55.180 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:55.178+0000 7f57f9600640 1 -- 172.21.15.72:0/3590697627 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57f41a3160 con 0x7f57f4108700 2024-04-26T21:59:55.180 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:55.180+0000 7f57f9600640 1 -- 172.21.15.72:0/3590697627 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57f410a1f0 con 0x7f57f4108700 2024-04-26T21:59:55.181 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:55.181+0000 7f57eb400640 1 -- 172.21.15.72:0/3590697627 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f57ec01c070 con 0x7f57f4108700 2024-04-26T21:59:55.182 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:55.182+0000 7f57eb400640 1 --2- 172.21.15.72:0/3590697627 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f57d80a3c40 0x7f57d80a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:55.182 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:55.183+0000 7f57eb400640 1 -- 172.21.15.72:0/3590697627 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7f57ec0cfab0 con 0x7f57f4108700 2024-04-26T21:59:55.183 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:55.183+0000 7f57f2a00640 1 --2- 172.21.15.72:0/3590697627 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f57d80a3c40 0x7f57d80a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:55.183 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:55.183+0000 7f57f2a00640 1 --2- 172.21.15.72:0/3590697627 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f57d80a3c40 0x7f57d80a6100 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f57e4005ab0 tx=0x7f57e4005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:55.186 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:55.187+0000 7f57eb400640 1 -- 172.21.15.72:0/3590697627 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f57ec09bd10 con 0x7f57f4108700 2024-04-26T21:59:55.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:59:55 smithi076 ceph-mon[31823]: pgmap v583: 97 pgs: 97 active+clean; 1.2 MiB data, 656 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1023 B/s wr, 12 op/s 2024-04-26T21:59:55.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:59:55 smithi076 ceph-mon[31823]: from='client.? 172.21.15.72:0/4079284125' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:59:55.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:59:55 smithi076 ceph-mon[31823]: from='client.? 172.21.15.72:0/2529063980' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:59:55.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:59:55 smithi072 ceph-mon[28138]: pgmap v583: 97 pgs: 97 active+clean; 1.2 MiB data, 656 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1023 B/s wr, 12 op/s 2024-04-26T21:59:55.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:59:55 smithi072 ceph-mon[28138]: from='client.? 172.21.15.72:0/4079284125' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:59:55.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:59:55 smithi072 ceph-mon[28138]: from='client.? 172.21.15.72:0/2529063980' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:59:55.419 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:55.419+0000 7f57f9600640 1 -- 172.21.15.72:0/3590697627 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f57f402dc50 con 0x7f57f4108700 2024-04-26T21:59:55.422 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:55.421+0000 7f57eb400640 1 -- 172.21.15.72:0/3590697627 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+22783 (secure 0 0 0) 0x7f57ec0bae20 con 0x7f57f4108700 2024-04-26T21:59:55.422 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T21:59:55.423 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":91,"fsid":"a1780950-0415-11ef-bc93-c7b262605968","created":"2024-04-26T21:42:01.504173+0000","modified":"2024-04-26T21:54:11.892872+0000","last_up_change":"2024-04-26T21:51:43.482437+0000","last_in_change":"2024-04-26T21:51:18.468587+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:46:29.785427+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:52:48.320391+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T21:52:49.517493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"04e06884-b355-4762-9b29-9034b7201e9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6802","nonce":1416765179},{"type":"v1","addr":"172.21.15.72:6803","nonce":1416765179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6804","nonce":1416765179},{"type":"v1","addr":"172.21.15.72:6805","nonce":1416765179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6808","nonce":1416765179},{"type":"v1","addr":"172.21.15.72:6809","nonce":1416765179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6806","nonce":1416765179},{"type":"v1","addr":"172.21.15.72:6807","nonce":1416765179}]},"public_addr":"172.21.15.72:6803/1416765179","cluster_addr":"172.21.15.72:6805/1416765179","heartbeat_back_addr":"172.21.15.72:6809/1416765179","heartbeat_front_addr":"172.21.15.72:6807/1416765179","state":["exists","up"]},{"osd":1,"uuid":"d75a479a-4a2e-48e9-a01e-2b1c9617416a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":755426949},{"type":"v1","addr":"172.21.15.76:6801","nonce":755426949}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":755426949},{"type":"v1","addr":"172.21.15.76:6803","nonce":755426949}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":755426949},{"type":"v1","addr":"172.21.15.76:6807","nonce":755426949}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":755426949},{"type":"v1","addr":"172.21.15.76:6805","nonce":755426949}]},"public_addr":"172.21.15.76:6801/755426949","cluster_addr":"172.21.15.76:6803/755426949","heartbeat_back_addr":"172.21.15.76:6807/755426949","heartbeat_front_addr":"172.21.15.76:6805/755426949","state":["exists","up"]},{"osd":2,"uuid":"b6f8da58-4428-4e8c-a944-7ea5645d6d92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":852265973},{"type":"v1","addr":"172.21.15.164:6801","nonce":852265973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":852265973},{"type":"v1","addr":"172.21.15.164:6803","nonce":852265973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":852265973},{"type":"v1","addr":"172.21.15.164:6807","nonce":852265973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":852265973},{"type":"v1","addr":"172.21.15.164:6805","nonce":852265973}]},"public_addr":"172.21.15.164:6801/852265973","cluster_addr":"172.21.15.164:6803/852265973","heartbeat_back_addr":"172.21.15.164:6807/852265973","heartbeat_front_addr":"172.21.15.164:6805/852265973","state":["exists","up"]},{"osd":3,"uuid":"76d83409-649e-41e5-b4f4-43bbd5b00ea5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6810","nonce":4027763789},{"type":"v1","addr":"172.21.15.72:6811","nonce":4027763789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6812","nonce":4027763789},{"type":"v1","addr":"172.21.15.72:6813","nonce":4027763789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6816","nonce":4027763789},{"type":"v1","addr":"172.21.15.72:6817","nonce":4027763789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6814","nonce":4027763789},{"type":"v1","addr":"172.21.15.72:6815","nonce":4027763789}]},"public_addr":"172.21.15.72:6811/4027763789","cluster_addr":"172.21.15.72:6813/4027763789","heartbeat_back_addr":"172.21.15.72:6817/4027763789","heartbeat_front_addr":"172.21.15.72:6815/4027763789","state":["exists","up"]},{"osd":4,"uuid":"d2cfeea7-1661-4156-bdcd-f770a8d1a878","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":3718867827},{"type":"v1","addr":"172.21.15.76:6809","nonce":3718867827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":3718867827},{"type":"v1","addr":"172.21.15.76:6811","nonce":3718867827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":3718867827},{"type":"v1","addr":"172.21.15.76:6815","nonce":3718867827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":3718867827},{"type":"v1","addr":"172.21.15.76:6813","nonce":3718867827}]},"public_addr":"172.21.15.76:6809/3718867827","cluster_addr":"172.21.15.76:6811/3718867827","heartbeat_back_addr":"172.21.15.76:6815/3718867827","heartbeat_front_addr":"172.21.15.76:6813/3718867827","state":["exists","up"]},{"osd":5,"uuid":"f3896ad6-a040-4ea3-a431-ff9b7fc05c36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":2543954772},{"type":"v1","addr":"172.21.15.164:6809","nonce":2543954772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":2543954772},{"type":"v1","addr":"172.21.15.164:6811","nonce":2543954772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":2543954772},{"type":"v1","addr":"172.21.15.164:6815","nonce":2543954772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":2543954772},{"type":"v1","addr":"172.21.15.164:6813","nonce":2543954772}]},"public_addr":"172.21.15.164:6809/2543954772","cluster_addr":"172.21.15.164:6811/2543954772","heartbeat_back_addr":"172.21.15.164:6815/2543954772","heartbeat_front_addr":"172.21.15.164:6813/2543954772","state":["exists","up"]},{"osd":6,"uuid":"d5d56650-be7e-4ba5-ba3a-d7d473294dab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6818","nonce":2080659027},{"type":"v1","addr":"172.21.15.72:6819","nonce":2080659027}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6820","nonce":2080659027},{"type":"v1","addr":"172.21.15.72:6821","nonce":2080659027}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6824","nonce":2080659027},{"type":"v1","addr":"172.21.15.72:6825","nonce":2080659027}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6822","nonce":2080659027},{"type":"v1","addr":"172.21.15.72:6823","nonce":2080659027}]},"public_addr":"172.21.15.72:6819/2080659027","cluster_addr":"172.21.15.72:6821/2080659027","heartbeat_back_addr":"172.21.15.72:6825/2080659027","heartbeat_front_addr":"172.21.15.72:6823/2080659027","state":["exists","up"]},{"osd":7,"uuid":"18975b5a-9309-4a0f-9f37-3ce09c2e745d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":3144631323},{"type":"v1","addr":"172.21.15.76:6817","nonce":3144631323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":3144631323},{"type":"v1","addr":"172.21.15.76:6819","nonce":3144631323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":3144631323},{"type":"v1","addr":"172.21.15.76:6823","nonce":3144631323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":3144631323},{"type":"v1","addr":"172.21.15.76:6821","nonce":3144631323}]},"public_addr":"172.21.15.76:6817/3144631323","cluster_addr":"172.21.15.76:6819/3144631323","heartbeat_back_addr":"172.21.15.76:6823/3144631323","heartbeat_front_addr":"172.21.15.76:6821/3144631323","state":["exists","up"]},{"osd":8,"uuid":"779e9c1e-8e97-48d4-a854-68ca13f38e0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":627888883},{"type":"v1","addr":"172.21.15.164:6817","nonce":627888883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":627888883},{"type":"v1","addr":"172.21.15.164:6819","nonce":627888883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":627888883},{"type":"v1","addr":"172.21.15.164:6823","nonce":627888883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":627888883},{"type":"v1","addr":"172.21.15.164:6821","nonce":627888883}]},"public_addr":"172.21.15.164:6817/627888883","cluster_addr":"172.21.15.164:6819/627888883","heartbeat_back_addr":"172.21.15.164:6823/627888883","heartbeat_front_addr":"172.21.15.164:6821/627888883","state":["exists","up"]},{"osd":9,"uuid":"d0317850-7b2d-4850-b2cf-0b919505c1fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6826","nonce":1273118485},{"type":"v1","addr":"172.21.15.72:6827","nonce":1273118485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6828","nonce":1273118485},{"type":"v1","addr":"172.21.15.72:6829","nonce":1273118485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6832","nonce":1273118485},{"type":"v1","addr":"172.21.15.72:6833","nonce":1273118485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6830","nonce":1273118485},{"type":"v1","addr":"172.21.15.72:6831","nonce":1273118485}]},"public_addr":"172.21.15.72:6827/1273118485","cluster_addr":"172.21.15.72:6829/1273118485","heartbeat_back_addr":"172.21.15.72:6833/1273118485","heartbeat_front_addr":"172.21.15.72:6831/1273118485","state":["exists","up"]},{"osd":10,"uuid":"e0aa38d8-6011-4d2a-b67b-5bff6d999e47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":2808954459},{"type":"v1","addr":"172.21.15.76:6825","nonce":2808954459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":2808954459},{"type":"v1","addr":"172.21.15.76:6827","nonce":2808954459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":2808954459},{"type":"v1","addr":"172.21.15.76:6831","nonce":2808954459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":2808954459},{"type":"v1","addr":"172.21.15.76:6829","nonce":2808954459}]},"public_addr":"172.21.15.76:6825/2808954459","cluster_addr":"172.21.15.76:6827/2808954459","heartbeat_back_addr":"172.21.15.76:6831/2808954459","heartbeat_front_addr":"172.21.15.76:6829/2808954459","state":["exists","up"]},{"osd":11,"uuid":"981b3d4d-916e-4937-9916-7cc21a891e43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":2354951784},{"type":"v1","addr":"172.21.15.164:6825","nonce":2354951784}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":2354951784},{"type":"v1","addr":"172.21.15.164:6827","nonce":2354951784}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":2354951784},{"type":"v1","addr":"172.21.15.164:6831","nonce":2354951784}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":2354951784},{"type":"v1","addr":"172.21.15.164:6829","nonce":2354951784}]},"public_addr":"172.21.15.164:6825/2354951784","cluster_addr":"172.21.15.164:6827/2354951784","heartbeat_back_addr":"172.21.15.164:6831/2354951784","heartbeat_front_addr":"172.21.15.164:6829/2354951784","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:18.997924+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:52.439081+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:46:25.893288+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:47:00.112807+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:47:35.116874+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:48:09.216344+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:48:45.028853+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:49:19.905397+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:49:55.030536+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:50:30.767662+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:51:06.423267+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:51:41.631211+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.1a","mappings":[{"from":5,"to":7}]},{"pgid":"3.1b","mappings":[{"from":4,"to":6}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.28","mappings":[{"from":5,"to":6}]},{"pgid":"3.2a","mappings":[{"from":3,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.72:0/416277784":"2024-04-27T21:43:09.602791+0000","172.21.15.72:6800/4010358672":"2024-04-27T21:43:09.602791+0000","172.21.15.72:6801/1012617371":"2024-04-27T21:42:34.684837+0000","172.21.15.72:6801/4010358672":"2024-04-27T21:43:09.602791+0000","172.21.15.72:0/3555448179":"2024-04-27T21:43:09.602791+0000","172.21.15.72:6800/1012617371":"2024-04-27T21:42:34.684837+0000","172.21.15.72:0/2920378761":"2024-04-27T21:42:34.684837+0000","172.21.15.72:0/1933740041":"2024-04-27T21:43:09.602791+0000","172.21.15.72:0/3225295775":"2024-04-27T21:42:34.684837+0000","172.21.15.72:0/2326016180":"2024-04-27T21:42:34.684837+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:59:55.424 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:55.424+0000 7f57f9600640 1 -- 172.21.15.72:0/3590697627 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f57d80a3c40 msgr2=0x7f57d80a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:55.425 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:55.424+0000 7f57f9600640 1 --2- 172.21.15.72:0/3590697627 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f57d80a3c40 0x7f57d80a6100 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f57e4005ab0 tx=0x7f57e4005a20 comp rx=0 tx=0).stop 2024-04-26T21:59:55.425 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:55.424+0000 7f57f9600640 1 -- 172.21.15.72:0/3590697627 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f57f4108700 msgr2=0x7f57f41a2300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:55.425 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:55.424+0000 7f57f9600640 1 --2- 172.21.15.72:0/3590697627 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f57f4108700 0x7f57f41a2300 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f57ec00b970 tx=0x7f57ec00be30 comp rx=0 tx=0).stop 2024-04-26T21:59:55.425 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:55.424+0000 7f57f9600640 1 -- 172.21.15.72:0/3590697627 shutdown_connections 2024-04-26T21:59:55.425 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:55.424+0000 7f57f9600640 1 --2- 172.21.15.72:0/3590697627 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f57f4108700 0x7f57f41a2300 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:55.425 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:55.424+0000 7f57f9600640 1 --2- 172.21.15.72:0/3590697627 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f57f4105dd0 0x7f57f419b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:55.425 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:55.424+0000 7f57f9600640 1 --2- 172.21.15.72:0/3590697627 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f57d80a3c40 0x7f57d80a6100 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:55.425 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:55.424+0000 7f57f9600640 1 --2- 172.21.15.72:0/3590697627 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f57f41034a0 0x7f57f419ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:55.425 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:55.424+0000 7f57f9600640 1 -- 172.21.15.72:0/3590697627 >> 172.21.15.72:0/3590697627 conn(0x7f57f40fcdf0 msgr2=0x7f57f4102bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:55.426 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:55.425+0000 7f57f9600640 1 -- 172.21.15.72:0/3590697627 shutdown_connections 2024-04-26T21:59:55.426 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:55.425+0000 7f57f9600640 1 -- 172.21.15.72:0/3590697627 wait complete. 2024-04-26T21:59:55.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:59:55 smithi164 ceph-mon[31916]: pgmap v583: 97 pgs: 97 active+clean; 1.2 MiB data, 656 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1023 B/s wr, 12 op/s 2024-04-26T21:59:55.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:59:55 smithi164 ceph-mon[31916]: from='client.? 172.21.15.72:0/4079284125' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:59:55.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:59:55 smithi164 ceph-mon[31916]: from='client.? 172.21.15.72:0/2529063980' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:59:55.444 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-26T21:59:55.642 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:55.641+0000 7fa2aca00640 1 -- 172.21.15.72:0/2138806551 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa2a8075310 msgr2=0x7fa2a8075770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:55.642 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:55.641+0000 7fa2aca00640 1 --2- 172.21.15.72:0/2138806551 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa2a8075310 0x7fa2a8075770 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fa294009900 tx=0x7fa29402a480 comp rx=0 tx=0).stop 2024-04-26T21:59:55.642 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:55.642+0000 7fa2aca00640 1 -- 172.21.15.72:0/2138806551 shutdown_connections 2024-04-26T21:59:55.642 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:55.642+0000 7fa2aca00640 1 --2- 172.21.15.72:0/2138806551 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa2a810a660 0x7fa2a810caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:55.642 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:55.642+0000 7fa2aca00640 1 --2- 172.21.15.72:0/2138806551 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa2a8075310 0x7fa2a8075770 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:55.642 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:55.642+0000 7fa2aca00640 1 --2- 172.21.15.72:0/2138806551 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa2a80768f0 0x7fa2a8074d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:55.643 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:55.642+0000 7fa2aca00640 1 -- 172.21.15.72:0/2138806551 >> 172.21.15.72:0/2138806551 conn(0x7fa2a80fdd40 msgr2=0x7fa2a8100160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:55.643 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:55.642+0000 7fa2aca00640 1 -- 172.21.15.72:0/2138806551 shutdown_connections 2024-04-26T21:59:55.643 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:55.642+0000 7fa2aca00640 1 -- 172.21.15.72:0/2138806551 wait complete. 2024-04-26T21:59:55.645 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:55.643+0000 7fa2aca00640 1 Processor -- start 2024-04-26T21:59:55.645 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:55.643+0000 7fa2aca00640 1 -- start start 2024-04-26T21:59:55.645 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:55.643+0000 7fa2aca00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa2a8075310 0x7fa2a819aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:55.646 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:55.643+0000 7fa2aca00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa2a80768f0 0x7fa2a819af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:55.646 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:55.643+0000 7fa2aca00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa2a810a660 0x7fa2a81a1fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:55.646 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:55.643+0000 7fa2aca00640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a8110370 con 0x7fa2a8075310 2024-04-26T21:59:55.646 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:55.643+0000 7fa2aca00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a81101f0 con 0x7fa2a80768f0 2024-04-26T21:59:55.646 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:55.643+0000 7fa2aca00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a81104f0 con 0x7fa2a810a660 2024-04-26T21:59:55.646 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:55.643+0000 7fa2a6800640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa2a8075310 0x7fa2a819aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:55.646 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:55.643+0000 7fa2a6800640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa2a8075310 0x7fa2a819aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:48182/0 (socket says 172.21.15.72:48182) 2024-04-26T21:59:55.646 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:55.643+0000 7fa2a6800640 1 -- 172.21.15.72:0/1175899423 learned_addr learned my addr 172.21.15.72:0/1175899423 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:59:55.646 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:55.644+0000 7fa2a5e00640 1 --2- 172.21.15.72:0/1175899423 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa2a80768f0 0x7fa2a819af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:55.646 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:55.644+0000 7fa2a7200640 1 --2- 172.21.15.72:0/1175899423 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa2a810a660 0x7fa2a81a1fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:55.646 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:55.644+0000 7fa2a6800640 1 -- 172.21.15.72:0/1175899423 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa2a810a660 msgr2=0x7fa2a81a1fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:55.646 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:55.644+0000 7fa2a6800640 1 --2- 172.21.15.72:0/1175899423 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa2a810a660 0x7fa2a81a1fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:55.647 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:55.644+0000 7fa2a6800640 1 -- 172.21.15.72:0/1175899423 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa2a80768f0 msgr2=0x7fa2a819af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:55.647 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:55.644+0000 7fa2a6800640 1 --2- 172.21.15.72:0/1175899423 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa2a80768f0 0x7fa2a819af60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:55.647 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:55.644+0000 7fa2a6800640 1 -- 172.21.15.72:0/1175899423 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2a81a2640 con 0x7fa2a8075310 2024-04-26T21:59:55.647 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:55.644+0000 7fa2a5e00640 1 --2- 172.21.15.72:0/1175899423 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa2a80768f0 0x7fa2a819af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:59:55.647 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:55.644+0000 7fa2a7200640 1 --2- 172.21.15.72:0/1175899423 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa2a810a660 0x7fa2a81a1fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:59:55.647 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:55.644+0000 7fa2a6800640 1 --2- 172.21.15.72:0/1175899423 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa2a8075310 0x7fa2a819aa20 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fa29800d950 tx=0x7fa29800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:55.647 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:55.644+0000 7fa29f400640 1 -- 172.21.15.72:0/1175899423 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa298017070 con 0x7fa2a8075310 2024-04-26T21:59:55.647 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:55.645+0000 7fa29f400640 1 -- 172.21.15.72:0/1175899423 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa298009c30 con 0x7fa2a8075310 2024-04-26T21:59:55.647 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:55.645+0000 7fa29f400640 1 -- 172.21.15.72:0/1175899423 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa29800f620 con 0x7fa2a8075310 2024-04-26T21:59:55.647 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:55.645+0000 7fa2aca00640 1 -- 172.21.15.72:0/1175899423 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2a81a2930 con 0x7fa2a8075310 2024-04-26T21:59:55.647 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:55.645+0000 7fa2aca00640 1 -- 172.21.15.72:0/1175899423 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2a81a2e80 con 0x7fa2a8075310 2024-04-26T21:59:55.648 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:55.646+0000 7fa2aca00640 1 -- 172.21.15.72:0/1175899423 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2a8109420 con 0x7fa2a8075310 2024-04-26T21:59:55.648 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:55.649+0000 7fa29f400640 1 -- 172.21.15.72:0/1175899423 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fa298010460 con 0x7fa2a8075310 2024-04-26T21:59:55.650 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:55.650+0000 7fa29f400640 1 --2- 172.21.15.72:0/1175899423 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fa28c0a3c40 0x7fa28c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:55.650 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:55.650+0000 7fa29f400640 1 -- 172.21.15.72:0/1175899423 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8356+0+0 (secure 0 0 0) 0x7fa2980d0310 con 0x7fa2a8075310 2024-04-26T21:59:55.650 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:55.650+0000 7fa2a5e00640 1 --2- 172.21.15.72:0/1175899423 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fa28c0a3c40 0x7fa28c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:55.651 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:55.651+0000 7fa2a5e00640 1 --2- 172.21.15.72:0/1175899423 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fa28c0a3c40 0x7fa28c0a6100 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fa2940096c0 tx=0x7fa294009650 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:55.656 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:55.656+0000 7fa29f400640 1 -- 172.21.15.72:0/1175899423 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fa29809cab0 con 0x7fa2a8075310 2024-04-26T21:59:55.899 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:55.899+0000 7fa2aca00640 1 -- 172.21.15.72:0/1175899423 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fa2a802dca0 con 0x7fa2a8075310 2024-04-26T21:59:55.903 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:55.903+0000 7fa29f400640 1 -- 172.21.15.72:0/1175899423 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v28) ==== 161+0+0 (secure 0 0 0) 0x7fa298009de0 con 0x7fa2a8075310 2024-04-26T21:59:55.903 INFO:teuthology.orchestra.run.smithi072.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-26T21:59:55.907 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:55.907+0000 7fa2aca00640 1 -- 172.21.15.72:0/1175899423 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fa28c0a3c40 msgr2=0x7fa28c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:55.907 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:55.907+0000 7fa2aca00640 1 --2- 172.21.15.72:0/1175899423 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fa28c0a3c40 0x7fa28c0a6100 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fa2940096c0 tx=0x7fa294009650 comp rx=0 tx=0).stop 2024-04-26T21:59:55.907 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:55.907+0000 7fa2aca00640 1 -- 172.21.15.72:0/1175899423 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa2a8075310 msgr2=0x7fa2a819aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:55.907 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:55.907+0000 7fa2aca00640 1 --2- 172.21.15.72:0/1175899423 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa2a8075310 0x7fa2a819aa20 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fa29800d950 tx=0x7fa29800de10 comp rx=0 tx=0).stop 2024-04-26T21:59:55.907 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:55.907+0000 7fa2aca00640 1 -- 172.21.15.72:0/1175899423 shutdown_connections 2024-04-26T21:59:55.907 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:55.907+0000 7fa2aca00640 1 --2- 172.21.15.72:0/1175899423 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fa28c0a3c40 0x7fa28c0a6100 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:55.907 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:55.907+0000 7fa2aca00640 1 --2- 172.21.15.72:0/1175899423 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa2a810a660 0x7fa2a81a1fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:55.907 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:55.907+0000 7fa2aca00640 1 --2- 172.21.15.72:0/1175899423 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa2a80768f0 0x7fa2a819af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:55.908 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:55.907+0000 7fa2aca00640 1 --2- 172.21.15.72:0/1175899423 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa2a8075310 0x7fa2a819aa20 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:55.908 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:55.907+0000 7fa2aca00640 1 -- 172.21.15.72:0/1175899423 >> 172.21.15.72:0/1175899423 conn(0x7fa2a80fdd40 msgr2=0x7fa2a81065f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:55.908 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:55.907+0000 7fa2aca00640 1 -- 172.21.15.72:0/1175899423 shutdown_connections 2024-04-26T21:59:55.908 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:55.907+0000 7fa2aca00640 1 -- 172.21.15.72:0/1175899423 wait complete. 2024-04-26T21:59:55.936 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-26T21:59:56.002 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:59:55 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-mon-a[28134]: 2024-04-26T21:59:55.900+0000 7f1656600700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-26T21:59:56.002 INFO:journalctl@ceph.mds.a.smithi072.stdout:Apr 26 21:59:55 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-mds-a[70549]: ignoring --setuser ceph since I am not root 2024-04-26T21:59:56.002 INFO:journalctl@ceph.mds.a.smithi072.stdout:Apr 26 21:59:55 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-mds-a[70549]: ignoring --setgroup ceph since I am not root 2024-04-26T21:59:56.002 INFO:journalctl@ceph.mds.a.smithi072.stdout:Apr 26 21:59:55 smithi072 ceph-mds[70553]: -- 172.21.15.72:0/1687284392 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5652cb1024e0 con 0x5652cb0d8c00 2024-04-26T21:59:56.002 INFO:journalctl@ceph.mds.a.smithi072.stdout:Apr 26 21:59:55 smithi072 ceph-mds[70553]: -- 172.21.15.72:0/1687284392 <== mon.0 v2:172.21.15.72:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5652cb105a40 con 0x5652cb0d8c00 2024-04-26T21:59:56.003 INFO:journalctl@ceph.mds.a.smithi072.stdout:Apr 26 21:59:55 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-mds-a[70549]: starting mds.a at 2024-04-26T21:59:56.033 INFO:journalctl@ceph.mds.k.smithi076.stdout:Apr 26 21:59:56 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-mds-k[60327]: ignoring --setuser ceph since I am not root 2024-04-26T21:59:56.034 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:59:56 smithi076 ceph-mon[31823]: from='client.? 172.21.15.72:0/3590697627' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:59:56.034 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:59:56 smithi076 ceph-mon[31823]: from='client.? 172.21.15.72:0/1175899423' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-26T21:59:56.035 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:59:56 smithi076 ceph-mon[31823]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-26T21:59:56.035 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:59:56 smithi076 ceph-mon[31823]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-26T21:59:56.035 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:59:56 smithi076 ceph-mon[31823]: from='client.? 172.21.15.72:0/1175899423' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-26T21:59:56.139 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:56.136+0000 7fd8c4c00640 1 -- 172.21.15.72:0/797014149 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd8c0078ed0 msgr2=0x7fd8c0079770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:56.140 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:56.136+0000 7fd8c4c00640 1 --2- 172.21.15.72:0/797014149 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd8c0078ed0 0x7fd8c0079770 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fd8b8009900 tx=0x7fd8b802a3f0 comp rx=0 tx=0).stop 2024-04-26T21:59:56.140 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:56.137+0000 7fd8c4c00640 1 -- 172.21.15.72:0/797014149 shutdown_connections 2024-04-26T21:59:56.140 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:56.137+0000 7fd8c4c00640 1 --2- 172.21.15.72:0/797014149 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd8c0078ed0 0x7fd8c0079770 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:56.140 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:56.137+0000 7fd8c4c00640 1 --2- 172.21.15.72:0/797014149 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd8c0078530 0x7fd8c0078990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:56.140 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:56.137+0000 7fd8c4c00640 1 --2- 172.21.15.72:0/797014149 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd8c00772f0 0x7fd8c0077710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:56.140 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:56.137+0000 7fd8c4c00640 1 -- 172.21.15.72:0/797014149 >> 172.21.15.72:0/797014149 conn(0x7fd8c0100120 msgr2=0x7fd8c0102560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:56.140 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:56.137+0000 7fd8c4c00640 1 -- 172.21.15.72:0/797014149 shutdown_connections 2024-04-26T21:59:56.140 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:56.137+0000 7fd8c4c00640 1 -- 172.21.15.72:0/797014149 wait complete. 2024-04-26T21:59:56.140 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:56.138+0000 7fd8c4c00640 1 Processor -- start 2024-04-26T21:59:56.140 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:56.138+0000 7fd8c4c00640 1 -- start start 2024-04-26T21:59:56.140 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:56.138+0000 7fd8c4c00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd8c00772f0 0x7fd8c019f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:56.140 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:56.138+0000 7fd8c4c00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd8c0078530 0x7fd8c019f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:56.141 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:56.138+0000 7fd8c4c00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd8c0078ed0 0x7fd8c01a6670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:56.141 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:56.138+0000 7fd8c4c00640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fd8c01149a0 con 0x7fd8c0078ed0 2024-04-26T21:59:56.141 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:56.138+0000 7fd8c4c00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fd8c0114820 con 0x7fd8c00772f0 2024-04-26T21:59:56.141 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:56.138+0000 7fd8c4c00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd8c0114b20 con 0x7fd8c0078530 2024-04-26T21:59:56.141 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:56.138+0000 7fd8be800640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd8c0078ed0 0x7fd8c01a6670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:56.141 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:56.138+0000 7fd8be800640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd8c0078ed0 0x7fd8c01a6670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:48222/0 (socket says 172.21.15.72:48222) 2024-04-26T21:59:56.141 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:56.138+0000 7fd8be800640 1 -- 172.21.15.72:0/104576712 learned_addr learned my addr 172.21.15.72:0/104576712 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:59:56.141 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:56.139+0000 7fd8bde00640 1 --2- 172.21.15.72:0/104576712 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd8c00772f0 0x7fd8c019f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:56.141 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:56.139+0000 7fd8b5400640 1 --2- 172.21.15.72:0/104576712 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd8c0078530 0x7fd8c019f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:56.141 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:56.139+0000 7fd8be800640 1 -- 172.21.15.72:0/104576712 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd8c0078530 msgr2=0x7fd8c019f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:56.141 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:56.139+0000 7fd8be800640 1 --2- 172.21.15.72:0/104576712 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd8c0078530 0x7fd8c019f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:56.141 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:56.139+0000 7fd8be800640 1 -- 172.21.15.72:0/104576712 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd8c00772f0 msgr2=0x7fd8c019f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:56.142 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:56.139+0000 7fd8be800640 1 --2- 172.21.15.72:0/104576712 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd8c00772f0 0x7fd8c019f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:56.142 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:56.139+0000 7fd8be800640 1 -- 172.21.15.72:0/104576712 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8c01a6ce0 con 0x7fd8c0078ed0 2024-04-26T21:59:56.142 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:56.139+0000 7fd8b5400640 1 --2- 172.21.15.72:0/104576712 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd8c0078530 0x7fd8c019f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:59:56.142 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:56.139+0000 7fd8be800640 1 --2- 172.21.15.72:0/104576712 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd8c0078ed0 0x7fd8c01a6670 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fd8b8004770 tx=0x7fd8b80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:56.142 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:56.139+0000 7fd8b7400640 1 -- 172.21.15.72:0/104576712 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8b803f030 con 0x7fd8c0078ed0 2024-04-26T21:59:56.142 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:56.139+0000 7fd8b7400640 1 -- 172.21.15.72:0/104576712 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd8b8003c90 con 0x7fd8c0078ed0 2024-04-26T21:59:56.142 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:56.139+0000 7fd8c4c00640 1 -- 172.21.15.72:0/104576712 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8c01a6f70 con 0x7fd8c0078ed0 2024-04-26T21:59:56.142 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:56.139+0000 7fd8b7400640 1 -- 172.21.15.72:0/104576712 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8b8035070 con 0x7fd8c0078ed0 2024-04-26T21:59:56.143 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:56.139+0000 7fd8c4c00640 1 -- 172.21.15.72:0/104576712 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8c01a7430 con 0x7fd8c0078ed0 2024-04-26T21:59:56.143 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:56.141+0000 7fd8c4c00640 1 -- 172.21.15.72:0/104576712 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8c0111520 con 0x7fd8c0078ed0 2024-04-26T21:59:56.143 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:56.143+0000 7fd8b7400640 1 -- 172.21.15.72:0/104576712 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fd8b8042030 con 0x7fd8c0078ed0 2024-04-26T21:59:56.145 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:56.145+0000 7fd8b7400640 1 --2- 172.21.15.72:0/104576712 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fd8a40a3cc0 0x7fd8a40a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:56.145 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:56.145+0000 7fd8b7400640 1 -- 172.21.15.72:0/104576712 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8786+0+0 (secure 0 0 0) 0x7fd8b80f0f10 con 0x7fd8c0078ed0 2024-04-26T21:59:56.145 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:56.145+0000 7fd8bde00640 1 --2- 172.21.15.72:0/104576712 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fd8a40a3cc0 0x7fd8a40a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:56.145 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:56.146+0000 7fd8bde00640 1 --2- 172.21.15.72:0/104576712 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fd8a40a3cc0 0x7fd8a40a6180 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fd8b0005fd0 tx=0x7fd8b0009870 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:56.150 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:56.150+0000 7fd8b7400640 1 -- 172.21.15.72:0/104576712 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fd8b80bd500 con 0x7fd8c0078ed0 2024-04-26T21:59:56.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:59:56 smithi076 ceph-mon[31823]: osdmap e92: 12 total, 12 up, 12 in 2024-04-26T21:59:56.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:59:56 smithi076 ceph-mon[31823]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-04-26T21:59:56.315 INFO:journalctl@ceph.mds.k.smithi076.stdout:Apr 26 21:59:56 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-mds-k[60327]: ignoring --setgroup ceph since I am not root 2024-04-26T21:59:56.315 INFO:journalctl@ceph.mds.k.smithi076.stdout:Apr 26 21:59:56 smithi076 ceph-mds[60333]: -- 172.21.15.76:0/2092389501 <== mon.0 v2:172.21.15.72:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d59c4a7a40 con 0x55d59c47ac00 2024-04-26T21:59:56.315 INFO:journalctl@ceph.mds.k.smithi076.stdout:Apr 26 21:59:56 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-mds-k[60327]: starting mds.k at 2024-04-26T21:59:56.315 INFO:journalctl@ceph.mds.h.smithi076.stdout:Apr 26 21:59:56 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-mds-h[60934]: ignoring --setuser ceph since I am not root 2024-04-26T21:59:56.316 INFO:journalctl@ceph.mds.h.smithi076.stdout:Apr 26 21:59:56 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-mds-h[60934]: ignoring --setgroup ceph since I am not root 2024-04-26T21:59:56.316 INFO:journalctl@ceph.mds.h.smithi076.stdout:Apr 26 21:59:56 smithi076 ceph-mds[60938]: -- 172.21.15.76:0/1352297252 <== mon.2 v2:172.21.15.76:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x559315d43a40 con 0x559315d16800 2024-04-26T21:59:56.316 INFO:journalctl@ceph.mds.h.smithi076.stdout:Apr 26 21:59:56 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-mds-h[60934]: starting mds.h at 2024-04-26T21:59:56.316 INFO:journalctl@ceph.mds.b.smithi076.stdout:Apr 26 21:59:56 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-mds-b[61253]: ignoring --setuser ceph since I am not root 2024-04-26T21:59:56.316 INFO:journalctl@ceph.mds.b.smithi076.stdout:Apr 26 21:59:56 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-mds-b[61253]: ignoring --setgroup ceph since I am not root 2024-04-26T21:59:56.316 INFO:journalctl@ceph.mds.b.smithi076.stdout:Apr 26 21:59:56 smithi076 ceph-mds[61257]: -- 172.21.15.76:0/346595247 <== mon.2 v2:172.21.15.76:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x559492ca1a40 con 0x559492c75000 2024-04-26T21:59:56.316 INFO:journalctl@ceph.mds.b.smithi076.stdout:Apr 26 21:59:56 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-mds-b[61253]: starting mds.b at 2024-04-26T21:59:56.317 INFO:journalctl@ceph.mds.e.smithi076.stdout:Apr 26 21:59:56 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-mds-e[60629]: ignoring --setuser ceph since I am not root 2024-04-26T21:59:56.317 INFO:journalctl@ceph.mds.e.smithi076.stdout:Apr 26 21:59:56 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-mds-e[60629]: ignoring --setgroup ceph since I am not root 2024-04-26T21:59:56.317 INFO:journalctl@ceph.mds.e.smithi076.stdout:Apr 26 21:59:56 smithi076 ceph-mds[60633]: -- 172.21.15.76:0/584204003 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x560f596604e0 con 0x560f59637000 2024-04-26T21:59:56.317 INFO:journalctl@ceph.mds.e.smithi076.stdout:Apr 26 21:59:56 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-mds-e[60629]: starting mds.e at 2024-04-26T21:59:56.322 INFO:journalctl@ceph.mds.g.smithi072.stdout:Apr 26 21:59:56 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-mds-g[70185]: ignoring --setuser ceph since I am not root 2024-04-26T21:59:56.322 INFO:journalctl@ceph.mds.g.smithi072.stdout:Apr 26 21:59:56 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-mds-g[70185]: ignoring --setgroup ceph since I am not root 2024-04-26T21:59:56.322 INFO:journalctl@ceph.mds.g.smithi072.stdout:Apr 26 21:59:56 smithi072 ceph-mds[70207]: -- 172.21.15.72:0/2536034863 <== mon.0 v2:172.21.15.72:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x562c8865ba40 con 0x562c8862e800 2024-04-26T21:59:56.322 INFO:journalctl@ceph.mds.g.smithi072.stdout:Apr 26 21:59:56 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-mds-g[70185]: starting mds.g at 2024-04-26T21:59:56.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:59:56 smithi072 ceph-mon[28138]: from='client.? 172.21.15.72:0/3590697627' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:59:56.323 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:59:56 smithi072 ceph-mon[28138]: from='client.? 172.21.15.72:0/1175899423' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-26T21:59:56.323 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:59:56 smithi072 ceph-mon[28138]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-26T21:59:56.323 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:59:56 smithi072 ceph-mon[28138]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-26T21:59:56.323 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:59:56 smithi072 ceph-mon[28138]: from='client.? 172.21.15.72:0/1175899423' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-26T21:59:56.323 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:59:56 smithi072 ceph-mon[28138]: osdmap e92: 12 total, 12 up, 12 in 2024-04-26T21:59:56.323 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:59:56 smithi072 ceph-mon[28138]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-04-26T21:59:56.388 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:56.388+0000 7fd8c4c00640 1 -- 172.21.15.72:0/104576712 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fd8c002dd20 con 0x7fd8c0078ed0 2024-04-26T21:59:56.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:59:56 smithi164 ceph-mon[31916]: from='client.? 172.21.15.72:0/3590697627' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:59:56.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:59:56 smithi164 ceph-mon[31916]: from='client.? 172.21.15.72:0/1175899423' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-26T21:59:56.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:59:56 smithi164 ceph-mon[31916]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-26T21:59:56.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:59:56 smithi164 ceph-mon[31916]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-26T21:59:56.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:59:56 smithi164 ceph-mon[31916]: from='client.? 172.21.15.72:0/1175899423' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-26T21:59:56.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:59:56 smithi164 ceph-mon[31916]: osdmap e92: 12 total, 12 up, 12 in 2024-04-26T21:59:56.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:59:56 smithi164 ceph-mon[31916]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-04-26T21:59:56.442 INFO:journalctl@ceph.mds.c.smithi164.stdout:Apr 26 21:59:56 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-mds-c[62363]: ignoring --setuser ceph since I am not root 2024-04-26T21:59:56.442 INFO:journalctl@ceph.mds.c.smithi164.stdout:Apr 26 21:59:56 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-mds-c[62363]: ignoring --setgroup ceph since I am not root 2024-04-26T21:59:56.442 INFO:journalctl@ceph.mds.c.smithi164.stdout:Apr 26 21:59:56 smithi164 ceph-mds[62367]: -- 172.21.15.164:0/708924050 <== mon.0 v2:172.21.15.72:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55eaee101a40 con 0x55eaee0d4800 2024-04-26T21:59:56.442 INFO:journalctl@ceph.mds.c.smithi164.stdout:Apr 26 21:59:56 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-mds-c[62363]: starting mds.c at 2024-04-26T21:59:56.442 INFO:journalctl@ceph.mds.f.smithi164.stdout:Apr 26 21:59:56 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-mds-f[62048]: ignoring --setuser ceph since I am not root 2024-04-26T21:59:56.442 INFO:journalctl@ceph.mds.f.smithi164.stdout:Apr 26 21:59:56 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-mds-f[62048]: ignoring --setgroup ceph since I am not root 2024-04-26T21:59:56.443 INFO:journalctl@ceph.mds.f.smithi164.stdout:Apr 26 21:59:56 smithi164 ceph-mds[62054]: -- 172.21.15.164:0/1076402937 <== mon.2 v2:172.21.15.76:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b4cfb33a40 con 0x55b4cfb07000 2024-04-26T21:59:56.443 INFO:journalctl@ceph.mds.f.smithi164.stdout:Apr 26 21:59:56 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-mds-f[62048]: starting mds.f at 2024-04-26T21:59:56.443 INFO:journalctl@ceph.mds.l.smithi164.stdout:Apr 26 21:59:56 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-mds-l[61745]: ignoring --setuser ceph since I am not root 2024-04-26T21:59:56.443 INFO:journalctl@ceph.mds.l.smithi164.stdout:Apr 26 21:59:56 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-mds-l[61745]: ignoring --setgroup ceph since I am not root 2024-04-26T21:59:56.443 INFO:journalctl@ceph.mds.l.smithi164.stdout:Apr 26 21:59:56 smithi164 ceph-mds[61750]: -- 172.21.15.164:0/2068110332 <== mon.1 v2:172.21.15.164:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a1db323a40 con 0x55a1db2f7000 2024-04-26T21:59:56.443 INFO:journalctl@ceph.mds.l.smithi164.stdout:Apr 26 21:59:56 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-mds-l[61745]: starting mds.l at 2024-04-26T21:59:56.443 INFO:journalctl@ceph.mds.i.smithi164.stdout:Apr 26 21:59:56 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-mds-i[62668]: ignoring --setuser ceph since I am not root 2024-04-26T21:59:56.444 INFO:journalctl@ceph.mds.i.smithi164.stdout:Apr 26 21:59:56 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-mds-i[62668]: ignoring --setgroup ceph since I am not root 2024-04-26T21:59:56.444 INFO:journalctl@ceph.mds.i.smithi164.stdout:Apr 26 21:59:56 smithi164 ceph-mds[62672]: -- 172.21.15.164:0/4293005157 <== mon.0 v2:172.21.15.72:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56162b9f7a40 con 0x56162b9ca800 2024-04-26T21:59:56.444 INFO:journalctl@ceph.mds.i.smithi164.stdout:Apr 26 21:59:56 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-mds-i[62668]: starting mds.i at 2024-04-26T21:59:56.910 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:56.910+0000 7fd8b7400640 1 -- 172.21.15.72:0/104576712 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v29) ==== 104+0+0 (secure 0 0 0) 0x7fd8b80bce00 con 0x7fd8c0078ed0 2024-04-26T21:59:56.914 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:56.913+0000 7fd8c4c00640 1 -- 172.21.15.72:0/104576712 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fd8a40a3cc0 msgr2=0x7fd8a40a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:56.914 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:56.913+0000 7fd8c4c00640 1 --2- 172.21.15.72:0/104576712 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fd8a40a3cc0 0x7fd8a40a6180 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fd8b0005fd0 tx=0x7fd8b0009870 comp rx=0 tx=0).stop 2024-04-26T21:59:56.914 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:56.913+0000 7fd8c4c00640 1 -- 172.21.15.72:0/104576712 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd8c0078ed0 msgr2=0x7fd8c01a6670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:56.914 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:56.913+0000 7fd8c4c00640 1 --2- 172.21.15.72:0/104576712 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd8c0078ed0 0x7fd8c01a6670 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fd8b8004770 tx=0x7fd8b80042f0 comp rx=0 tx=0).stop 2024-04-26T21:59:56.914 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:56.913+0000 7fd8c4c00640 1 -- 172.21.15.72:0/104576712 shutdown_connections 2024-04-26T21:59:56.914 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:56.913+0000 7fd8c4c00640 1 --2- 172.21.15.72:0/104576712 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd8c0078ed0 0x7fd8c01a6670 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:56.914 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:56.913+0000 7fd8c4c00640 1 --2- 172.21.15.72:0/104576712 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7fd8a40a3cc0 0x7fd8a40a6180 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:56.914 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:56.913+0000 7fd8c4c00640 1 --2- 172.21.15.72:0/104576712 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd8c0078530 0x7fd8c019f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:56.914 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:56.913+0000 7fd8c4c00640 1 --2- 172.21.15.72:0/104576712 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd8c00772f0 0x7fd8c019f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:56.914 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:56.913+0000 7fd8c4c00640 1 -- 172.21.15.72:0/104576712 >> 172.21.15.72:0/104576712 conn(0x7fd8c0100120 msgr2=0x7fd8c0103480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:56.914 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:56.914+0000 7fd8c4c00640 1 -- 172.21.15.72:0/104576712 shutdown_connections 2024-04-26T21:59:56.915 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:56.914+0000 7fd8c4c00640 1 -- 172.21.15.72:0/104576712 wait complete. 2024-04-26T21:59:56.941 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-26T21:59:57.124 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:57.124+0000 7f1134c00640 1 -- 172.21.15.72:0/1424124567 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1130075390 msgr2=0x7f1130075810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:57.124 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:57.124+0000 7f1134c00640 1 --2- 172.21.15.72:0/1424124567 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1130075390 0x7f1130075810 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f1118009930 tx=0x7f111802a420 comp rx=0 tx=0).stop 2024-04-26T21:59:57.124 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:57.124+0000 7f1134c00640 1 -- 172.21.15.72:0/1424124567 shutdown_connections 2024-04-26T21:59:57.125 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:57.124+0000 7f1134c00640 1 --2- 172.21.15.72:0/1424124567 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f113010ee00 0x7f11301111f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:57.125 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:57.124+0000 7f1134c00640 1 --2- 172.21.15.72:0/1424124567 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1130075390 0x7f1130075810 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:57.125 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:57.124+0000 7f1134c00640 1 --2- 172.21.15.72:0/1424124567 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1130076970 0x7f1130074dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:57.125 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:57.124+0000 7f1134c00640 1 -- 172.21.15.72:0/1424124567 >> 172.21.15.72:0/1424124567 conn(0x7f11300fdda0 msgr2=0x7f11301001e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:57.125 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:57.124+0000 7f1134c00640 1 -- 172.21.15.72:0/1424124567 shutdown_connections 2024-04-26T21:59:57.125 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:57.124+0000 7f1134c00640 1 -- 172.21.15.72:0/1424124567 wait complete. 2024-04-26T21:59:57.128 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:57.125+0000 7f1134c00640 1 Processor -- start 2024-04-26T21:59:57.128 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:57.125+0000 7f1134c00640 1 -- start start 2024-04-26T21:59:57.128 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:57.125+0000 7f1134c00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1130075390 0x7f113019eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:57.128 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:57.125+0000 7f1134c00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1130076970 0x7f113019f530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:57.128 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:57.125+0000 7f1134c00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f113010ee00 0x7f11301a65a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:57.128 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:57.125+0000 7f1134c00640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f1130114830 con 0x7f113010ee00 2024-04-26T21:59:57.128 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:57.125+0000 7f1134c00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f11301146b0 con 0x7f1130075390 2024-04-26T21:59:57.128 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:57.125+0000 7f1134c00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f11301149b0 con 0x7f1130076970 2024-04-26T21:59:57.129 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:57.126+0000 7f112fe00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f113010ee00 0x7f11301a65a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:57.129 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:57.126+0000 7f112fe00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f113010ee00 0x7f11301a65a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:48254/0 (socket says 172.21.15.72:48254) 2024-04-26T21:59:57.129 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:57.126+0000 7f112fe00640 1 -- 172.21.15.72:0/1206930978 learned_addr learned my addr 172.21.15.72:0/1206930978 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:59:57.129 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:57.126+0000 7f112f400640 1 --2- 172.21.15.72:0/1206930978 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1130075390 0x7f113019eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:57.129 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:57.126+0000 7f1126a00640 1 --2- 172.21.15.72:0/1206930978 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1130076970 0x7f113019f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:57.129 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:57.126+0000 7f112fe00640 1 -- 172.21.15.72:0/1206930978 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1130076970 msgr2=0x7f113019f530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:57.129 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:57.126+0000 7f112fe00640 1 --2- 172.21.15.72:0/1206930978 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1130076970 0x7f113019f530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:57.129 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:57.126+0000 7f112fe00640 1 -- 172.21.15.72:0/1206930978 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1130075390 msgr2=0x7f113019eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:57.129 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:57.126+0000 7f112fe00640 1 --2- 172.21.15.72:0/1206930978 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1130075390 0x7f113019eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:57.129 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:57.126+0000 7f112fe00640 1 -- 172.21.15.72:0/1206930978 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11301a6c10 con 0x7f113010ee00 2024-04-26T21:59:57.129 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:57.126+0000 7f1126a00640 1 --2- 172.21.15.72:0/1206930978 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1130076970 0x7f113019f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:59:57.129 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:57.126+0000 7f112fe00640 1 --2- 172.21.15.72:0/1206930978 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f113010ee00 0x7f11301a65a0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f112800eaa0 tx=0x7f112800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:57.129 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:57.127+0000 7f112cc00640 1 -- 172.21.15.72:0/1206930978 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f112800cdb0 con 0x7f113010ee00 2024-04-26T21:59:57.130 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:57.127+0000 7f112cc00640 1 -- 172.21.15.72:0/1206930978 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1128005020 con 0x7f113010ee00 2024-04-26T21:59:57.130 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:57.127+0000 7f1134c00640 1 -- 172.21.15.72:0/1206930978 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11301a6f00 con 0x7f113010ee00 2024-04-26T21:59:57.130 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:57.127+0000 7f112cc00640 1 -- 172.21.15.72:0/1206930978 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11280116d0 con 0x7f113010ee00 2024-04-26T21:59:57.130 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:57.127+0000 7f1134c00640 1 -- 172.21.15.72:0/1206930978 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11301a7410 con 0x7f113010ee00 2024-04-26T21:59:57.130 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:57.128+0000 7f1134c00640 1 -- 172.21.15.72:0/1206930978 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1130104fd0 con 0x7f113010ee00 2024-04-26T21:59:57.130 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:57.130+0000 7f112cc00640 1 -- 172.21.15.72:0/1206930978 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f11280051d0 con 0x7f113010ee00 2024-04-26T21:59:57.131 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:57.131+0000 7f112cc00640 1 --2- 172.21.15.72:0/1206930978 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f11140a3cc0 0x7f11140a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:57.131 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:57.131+0000 7f112cc00640 1 -- 172.21.15.72:0/1206930978 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8786+0+0 (secure 0 0 0) 0x7f11280d09f0 con 0x7f113010ee00 2024-04-26T21:59:57.131 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:57.131+0000 7f112f400640 1 --2- 172.21.15.72:0/1206930978 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f11140a3cc0 0x7f11140a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:57.132 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:57.132+0000 7f112f400640 1 --2- 172.21.15.72:0/1206930978 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f11140a3cc0 0x7f11140a6180 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f1120005fd0 tx=0x7f1120005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:57.136 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:57.136+0000 7f112cc00640 1 -- 172.21.15.72:0/1206930978 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f1128016030 con 0x7f113010ee00 2024-04-26T21:59:57.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:59:57 smithi076 ceph-mon[31823]: pgmap v585: 97 pgs: 97 active+clean; 1.2 MiB data, 656 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-04-26T21:59:57.316 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:59:57 smithi076 ceph-mon[31823]: from='client.? 172.21.15.72:0/104576712' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-26T21:59:57.316 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:59:57 smithi076 ceph-mon[31823]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-26T21:59:57.316 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:59:57 smithi076 ceph-mon[31823]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-26T21:59:57.316 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:59:57 smithi076 ceph-mon[31823]: Cluster is now healthy 2024-04-26T21:59:57.316 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:59:57 smithi076 ceph-mon[31823]: from='client.? 172.21.15.72:0/104576712' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-26T21:59:57.316 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:59:57 smithi076 ceph-mon[31823]: mds.? [v2:172.21.15.72:6838/1924586494,v1:172.21.15.72:6839/1924586494] up:boot 2024-04-26T21:59:57.317 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:59:57 smithi076 ceph-mon[31823]: mds.? [v2:172.21.15.72:6840/3051299639,v1:172.21.15.72:6841/3051299639] up:boot 2024-04-26T21:59:57.317 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:59:57 smithi076 ceph-mon[31823]: mds.? [v2:172.21.15.76:6832/1656509929,v1:172.21.15.76:6834/1656509929] up:boot 2024-04-26T21:59:57.317 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:59:57 smithi076 ceph-mon[31823]: mds.? [v2:172.21.15.76:6833/2186164714,v1:172.21.15.76:6835/2186164714] up:boot 2024-04-26T21:59:57.317 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:59:57 smithi076 ceph-mon[31823]: mds.? [v2:172.21.15.164:6832/3127809683,v1:172.21.15.164:6833/3127809683] up:boot 2024-04-26T21:59:57.317 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:59:57 smithi076 ceph-mon[31823]: mds.? [v2:172.21.15.76:6837/1050676780,v1:172.21.15.76:6839/1050676780] up:boot 2024-04-26T21:59:57.317 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:59:57 smithi076 ceph-mon[31823]: mds.? [v2:172.21.15.164:6834/2505705409,v1:172.21.15.164:6835/2505705409] up:boot 2024-04-26T21:59:57.317 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:59:57 smithi076 ceph-mon[31823]: mds.? [v2:172.21.15.76:6836/3799954022,v1:172.21.15.76:6838/3799954022] up:boot 2024-04-26T21:59:57.317 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:59:57 smithi076 ceph-mon[31823]: mds.? [v2:172.21.15.164:6836/3079558637,v1:172.21.15.164:6838/3079558637] up:boot 2024-04-26T21:59:57.317 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:59:57 smithi076 ceph-mon[31823]: mds.? [v2:172.21.15.164:6837/1003974018,v1:172.21.15.164:6839/1003974018] up:boot 2024-04-26T21:59:57.318 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:59:57 smithi076 ceph-mon[31823]: fsmap 12 up:standby 2024-04-26T21:59:57.318 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:59:57 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-26T21:59:57.318 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:59:57 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-26T21:59:57.318 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:59:57 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-26T21:59:57.318 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:59:57 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-26T21:59:57.318 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:59:57 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-26T21:59:57.318 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:59:57 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-26T21:59:57.318 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:59:57 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-26T21:59:57.318 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:59:57 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-26T21:59:57.318 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:59:57 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-26T21:59:57.318 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:59:57 smithi076 ceph-mon[31823]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-26T21:59:57.321 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:57.321+0000 7f1134c00640 1 -- 172.21.15.72:0/1206930978 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f113002dad0 con 0x7f113010ee00 2024-04-26T21:59:57.323 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:59:57 smithi072 ceph-mon[28138]: pgmap v585: 97 pgs: 97 active+clean; 1.2 MiB data, 656 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-04-26T21:59:57.323 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:59:57 smithi072 ceph-mon[28138]: from='client.? 172.21.15.72:0/104576712' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-26T21:59:57.323 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:59:57 smithi072 ceph-mon[28138]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-26T21:59:57.323 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:59:57 smithi072 ceph-mon[28138]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-26T21:59:57.323 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:59:57 smithi072 ceph-mon[28138]: Cluster is now healthy 2024-04-26T21:59:57.323 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:59:57 smithi072 ceph-mon[28138]: from='client.? 172.21.15.72:0/104576712' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-26T21:59:57.323 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:59:57 smithi072 ceph-mon[28138]: mds.? [v2:172.21.15.72:6838/1924586494,v1:172.21.15.72:6839/1924586494] up:boot 2024-04-26T21:59:57.323 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:59:57 smithi072 ceph-mon[28138]: mds.? [v2:172.21.15.72:6840/3051299639,v1:172.21.15.72:6841/3051299639] up:boot 2024-04-26T21:59:57.323 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:59:57 smithi072 ceph-mon[28138]: mds.? [v2:172.21.15.76:6832/1656509929,v1:172.21.15.76:6834/1656509929] up:boot 2024-04-26T21:59:57.323 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:59:57 smithi072 ceph-mon[28138]: mds.? [v2:172.21.15.76:6833/2186164714,v1:172.21.15.76:6835/2186164714] up:boot 2024-04-26T21:59:57.323 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:59:57 smithi072 ceph-mon[28138]: mds.? [v2:172.21.15.164:6832/3127809683,v1:172.21.15.164:6833/3127809683] up:boot 2024-04-26T21:59:57.324 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:59:57 smithi072 ceph-mon[28138]: mds.? [v2:172.21.15.76:6837/1050676780,v1:172.21.15.76:6839/1050676780] up:boot 2024-04-26T21:59:57.324 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:59:57 smithi072 ceph-mon[28138]: mds.? [v2:172.21.15.164:6834/2505705409,v1:172.21.15.164:6835/2505705409] up:boot 2024-04-26T21:59:57.324 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:59:57 smithi072 ceph-mon[28138]: mds.? [v2:172.21.15.76:6836/3799954022,v1:172.21.15.76:6838/3799954022] up:boot 2024-04-26T21:59:57.324 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:59:57 smithi072 ceph-mon[28138]: mds.? [v2:172.21.15.164:6836/3079558637,v1:172.21.15.164:6838/3079558637] up:boot 2024-04-26T21:59:57.324 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:59:57 smithi072 ceph-mon[28138]: mds.? [v2:172.21.15.164:6837/1003974018,v1:172.21.15.164:6839/1003974018] up:boot 2024-04-26T21:59:57.324 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:59:57 smithi072 ceph-mon[28138]: fsmap 12 up:standby 2024-04-26T21:59:57.324 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:59:57 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-26T21:59:57.324 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:59:57 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-26T21:59:57.324 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:59:57 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-26T21:59:57.324 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:59:57 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-26T21:59:57.324 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:59:57 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-26T21:59:57.324 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:59:57 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-26T21:59:57.324 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:59:57 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-26T21:59:57.324 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:59:57 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-26T21:59:57.325 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:59:57 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-26T21:59:57.325 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:59:57 smithi072 ceph-mon[28138]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-26T21:59:57.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:59:57 smithi164 ceph-mon[31916]: pgmap v585: 97 pgs: 97 active+clean; 1.2 MiB data, 656 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-04-26T21:59:57.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:59:57 smithi164 ceph-mon[31916]: from='client.? 172.21.15.72:0/104576712' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-26T21:59:57.442 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:59:57 smithi164 ceph-mon[31916]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-26T21:59:57.442 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:59:57 smithi164 ceph-mon[31916]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-26T21:59:57.442 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:59:57 smithi164 ceph-mon[31916]: Cluster is now healthy 2024-04-26T21:59:57.442 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:59:57 smithi164 ceph-mon[31916]: from='client.? 172.21.15.72:0/104576712' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-26T21:59:57.442 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:59:57 smithi164 ceph-mon[31916]: mds.? [v2:172.21.15.72:6838/1924586494,v1:172.21.15.72:6839/1924586494] up:boot 2024-04-26T21:59:57.442 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:59:57 smithi164 ceph-mon[31916]: mds.? [v2:172.21.15.72:6840/3051299639,v1:172.21.15.72:6841/3051299639] up:boot 2024-04-26T21:59:57.442 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:59:57 smithi164 ceph-mon[31916]: mds.? [v2:172.21.15.76:6832/1656509929,v1:172.21.15.76:6834/1656509929] up:boot 2024-04-26T21:59:57.442 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:59:57 smithi164 ceph-mon[31916]: mds.? [v2:172.21.15.76:6833/2186164714,v1:172.21.15.76:6835/2186164714] up:boot 2024-04-26T21:59:57.442 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:59:57 smithi164 ceph-mon[31916]: mds.? [v2:172.21.15.164:6832/3127809683,v1:172.21.15.164:6833/3127809683] up:boot 2024-04-26T21:59:57.442 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:59:57 smithi164 ceph-mon[31916]: mds.? [v2:172.21.15.76:6837/1050676780,v1:172.21.15.76:6839/1050676780] up:boot 2024-04-26T21:59:57.442 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:59:57 smithi164 ceph-mon[31916]: mds.? [v2:172.21.15.164:6834/2505705409,v1:172.21.15.164:6835/2505705409] up:boot 2024-04-26T21:59:57.442 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:59:57 smithi164 ceph-mon[31916]: mds.? [v2:172.21.15.76:6836/3799954022,v1:172.21.15.76:6838/3799954022] up:boot 2024-04-26T21:59:57.442 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:59:57 smithi164 ceph-mon[31916]: mds.? [v2:172.21.15.164:6836/3079558637,v1:172.21.15.164:6838/3079558637] up:boot 2024-04-26T21:59:57.443 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:59:57 smithi164 ceph-mon[31916]: mds.? [v2:172.21.15.164:6837/1003974018,v1:172.21.15.164:6839/1003974018] up:boot 2024-04-26T21:59:57.443 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:59:57 smithi164 ceph-mon[31916]: fsmap 12 up:standby 2024-04-26T21:59:57.443 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:59:57 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-26T21:59:57.443 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:59:57 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-26T21:59:57.443 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:59:57 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-26T21:59:57.443 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:59:57 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-26T21:59:57.443 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:59:57 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-26T21:59:57.443 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:59:57 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-26T21:59:57.443 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:59:57 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-26T21:59:57.443 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:59:57 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-26T21:59:57.443 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:59:57 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-26T21:59:57.443 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:59:57 smithi164 ceph-mon[31916]: from='mgr.14152 172.21.15.72:0/2570993505' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-26T21:59:58.044 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:58.044+0000 7f112cc00640 1 -- 172.21.15.72:0/1206930978 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v93) ==== 181+0+0 (secure 0 0 0) 0x7f112809cfe0 con 0x7f113010ee00 2024-04-26T21:59:58.044 INFO:teuthology.orchestra.run.smithi072.stderr:pool 'cephfs_metadata' removed 2024-04-26T21:59:58.047 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:58.048+0000 7f1134c00640 1 -- 172.21.15.72:0/1206930978 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f11140a3cc0 msgr2=0x7f11140a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:58.047 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:58.048+0000 7f1134c00640 1 --2- 172.21.15.72:0/1206930978 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f11140a3cc0 0x7f11140a6180 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f1120005fd0 tx=0x7f1120005850 comp rx=0 tx=0).stop 2024-04-26T21:59:58.048 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:58.048+0000 7f1134c00640 1 -- 172.21.15.72:0/1206930978 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f113010ee00 msgr2=0x7f11301a65a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:58.048 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:58.048+0000 7f1134c00640 1 --2- 172.21.15.72:0/1206930978 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f113010ee00 0x7f11301a65a0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f112800eaa0 tx=0x7f112800ef60 comp rx=0 tx=0).stop 2024-04-26T21:59:58.048 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:58.048+0000 7f1134c00640 1 -- 172.21.15.72:0/1206930978 shutdown_connections 2024-04-26T21:59:58.048 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:58.048+0000 7f1134c00640 1 --2- 172.21.15.72:0/1206930978 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f113010ee00 0x7f11301a65a0 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:58.048 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:58.048+0000 7f1134c00640 1 --2- 172.21.15.72:0/1206930978 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f11140a3cc0 0x7f11140a6180 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:58.049 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:58.048+0000 7f1134c00640 1 --2- 172.21.15.72:0/1206930978 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1130076970 0x7f113019f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:58.049 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:58.048+0000 7f1134c00640 1 --2- 172.21.15.72:0/1206930978 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1130075390 0x7f113019eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:58.049 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:58.048+0000 7f1134c00640 1 -- 172.21.15.72:0/1206930978 >> 172.21.15.72:0/1206930978 conn(0x7f11300fdda0 msgr2=0x7f11301021a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:58.049 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:58.048+0000 7f1134c00640 1 -- 172.21.15.72:0/1206930978 shutdown_connections 2024-04-26T21:59:58.049 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:58.048+0000 7f1134c00640 1 -- 172.21.15.72:0/1206930978 wait complete. 2024-04-26T21:59:58.072 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-26T21:59:58.270 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:58.270+0000 7f2174a00640 1 -- 172.21.15.72:0/3342745484 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f21701032c0 msgr2=0x7f21701056b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:58.271 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:58.270+0000 7f2174a00640 1 --2- 172.21.15.72:0/3342745484 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f21701032c0 0x7f21701056b0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f2160009900 tx=0x7f216002a410 comp rx=0 tx=0).stop 2024-04-26T21:59:58.271 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:58.270+0000 7f2174a00640 1 -- 172.21.15.72:0/3342745484 shutdown_connections 2024-04-26T21:59:58.271 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:58.270+0000 7f2174a00640 1 --2- 172.21.15.72:0/3342745484 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2170108520 0x7f217010a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:58.271 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:58.270+0000 7f2174a00640 1 --2- 172.21.15.72:0/3342745484 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2170105bf0 0x7f2170107fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:58.271 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:58.270+0000 7f2174a00640 1 --2- 172.21.15.72:0/3342745484 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f21701032c0 0x7f21701056b0 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:58.271 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:58.270+0000 7f2174a00640 1 -- 172.21.15.72:0/3342745484 >> 172.21.15.72:0/3342745484 conn(0x7f21700fccd0 msgr2=0x7f21700ff0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:58.271 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:58.270+0000 7f2174a00640 1 -- 172.21.15.72:0/3342745484 shutdown_connections 2024-04-26T21:59:58.271 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:58.270+0000 7f2174a00640 1 -- 172.21.15.72:0/3342745484 wait complete. 2024-04-26T21:59:58.274 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:58.271+0000 7f2174a00640 1 Processor -- start 2024-04-26T21:59:58.274 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:58.271+0000 7f2174a00640 1 -- start start 2024-04-26T21:59:58.274 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:58.272+0000 7f2174a00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f21701032c0 0x7f2170198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:58.275 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:58.272+0000 7f2174a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2170105bf0 0x7f2170198f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:58.275 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:58.272+0000 7f2174a00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2170108520 0x7f217019ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:58.275 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:58.272+0000 7f2174a00640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f217010e2a0 con 0x7f2170108520 2024-04-26T21:59:58.275 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:58.272+0000 7f2174a00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f217010e120 con 0x7f21701032c0 2024-04-26T21:59:58.275 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:58.272+0000 7f2174a00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f217010e420 con 0x7f2170105bf0 2024-04-26T21:59:58.275 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:58.272+0000 7f216fe00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2170108520 0x7f217019ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:58.275 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:58.272+0000 7f216fe00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2170108520 0x7f217019ffa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:48266/0 (socket says 172.21.15.72:48266) 2024-04-26T21:59:58.275 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:58.272+0000 7f216fe00640 1 -- 172.21.15.72:0/2977590072 learned_addr learned my addr 172.21.15.72:0/2977590072 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-04-26T21:59:58.275 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:58.272+0000 7f216f400640 1 --2- 172.21.15.72:0/2977590072 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f21701032c0 0x7f2170198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:58.276 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:58.272+0000 7f216ea00640 1 --2- 172.21.15.72:0/2977590072 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2170105bf0 0x7f2170198f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:58.276 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:58.272+0000 7f216fe00640 1 -- 172.21.15.72:0/2977590072 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2170105bf0 msgr2=0x7f2170198f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:58.276 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:58.272+0000 7f216fe00640 1 --2- 172.21.15.72:0/2977590072 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2170105bf0 0x7f2170198f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:58.276 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:58.273+0000 7f216fe00640 1 -- 172.21.15.72:0/2977590072 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f21701032c0 msgr2=0x7f2170198a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:58.276 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:58.273+0000 7f216fe00640 1 --2- 172.21.15.72:0/2977590072 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f21701032c0 0x7f2170198a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:58.276 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:58.273+0000 7f216fe00640 1 -- 172.21.15.72:0/2977590072 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21701a0610 con 0x7f2170108520 2024-04-26T21:59:58.276 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:58.273+0000 7f216ea00640 1 --2- 172.21.15.72:0/2977590072 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2170105bf0 0x7f2170198f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:59:58.276 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:58.273+0000 7f216f400640 1 --2- 172.21.15.72:0/2977590072 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f21701032c0 0x7f2170198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:59:58.276 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:58.273+0000 7f216fe00640 1 --2- 172.21.15.72:0/2977590072 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2170108520 0x7f217019ffa0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f216800eaa0 tx=0x7f216800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:58.276 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:58.273+0000 7f2167e00640 1 -- 172.21.15.72:0/2977590072 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f216800cdb0 con 0x7f2170108520 2024-04-26T21:59:58.276 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:58.273+0000 7f2167e00640 1 -- 172.21.15.72:0/2977590072 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2168005020 con 0x7f2170108520 2024-04-26T21:59:58.276 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:58.273+0000 7f2174a00640 1 -- 172.21.15.72:0/2977590072 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21701a0900 con 0x7f2170108520 2024-04-26T21:59:58.276 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:58.273+0000 7f2167e00640 1 -- 172.21.15.72:0/2977590072 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21680116d0 con 0x7f2170108520 2024-04-26T21:59:58.277 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:58.273+0000 7f2174a00640 1 -- 172.21.15.72:0/2977590072 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21701a0e00 con 0x7f2170108520 2024-04-26T21:59:58.277 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:58.275+0000 7f2174a00640 1 -- 172.21.15.72:0/2977590072 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21700fe7c0 con 0x7f2170108520 2024-04-26T21:59:58.277 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:58.276+0000 7f2167e00640 1 -- 172.21.15.72:0/2977590072 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f21680051d0 con 0x7f2170108520 2024-04-26T21:59:58.277 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:58.278+0000 7f2167e00640 1 --2- 172.21.15.72:0/2977590072 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f21540a3c40 0x7f21540a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:58.278 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:58.278+0000 7f2167e00640 1 -- 172.21.15.72:0/2977590072 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8203+0+0 (secure 0 0 0) 0x7f21680d08c0 con 0x7f2170108520 2024-04-26T21:59:58.278 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:58.278+0000 7f216f400640 1 --2- 172.21.15.72:0/2977590072 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f21540a3c40 0x7f21540a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:58.278 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:58.279+0000 7f216f400640 1 --2- 172.21.15.72:0/2977590072 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f21540a3c40 0x7f21540a6100 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f216002a930 tx=0x7f2160000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:58.281 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:58.282+0000 7f2167e00640 1 -- 172.21.15.72:0/2977590072 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f2168016030 con 0x7f2170108520 2024-04-26T21:59:58.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:59:58 smithi076 ceph-mon[31823]: from='client.? 172.21.15.72:0/1206930978' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-26T21:59:58.321 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:59:58 smithi072 ceph-mon[28138]: from='client.? 172.21.15.72:0/1206930978' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-26T21:59:58.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:59:58 smithi164 ceph-mon[31916]: from='client.? 172.21.15.72:0/1206930978' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-26T21:59:58.483 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:58.483+0000 7f2174a00640 1 -- 172.21.15.72:0/2977590072 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f2170063530 con 0x7f2170108520 2024-04-26T21:59:59.059 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:59.059+0000 7f2167e00640 1 -- 172.21.15.72:0/2977590072 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v94) ==== 169+0+0 (secure 0 0 0) 0x7f216809d100 con 0x7f2170108520 2024-04-26T21:59:59.059 INFO:teuthology.orchestra.run.smithi072.stderr:pool 'cephfs_data' removed 2024-04-26T21:59:59.062 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:59.063+0000 7f2174a00640 1 -- 172.21.15.72:0/2977590072 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f21540a3c40 msgr2=0x7f21540a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:59.063 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:59.063+0000 7f2174a00640 1 --2- 172.21.15.72:0/2977590072 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f21540a3c40 0x7f21540a6100 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f216002a930 tx=0x7f2160000bc0 comp rx=0 tx=0).stop 2024-04-26T21:59:59.063 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:59.063+0000 7f2174a00640 1 -- 172.21.15.72:0/2977590072 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2170108520 msgr2=0x7f217019ffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:59.063 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:59.063+0000 7f2174a00640 1 --2- 172.21.15.72:0/2977590072 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2170108520 0x7f217019ffa0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f216800eaa0 tx=0x7f216800ef60 comp rx=0 tx=0).stop 2024-04-26T21:59:59.063 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:59.063+0000 7f2174a00640 1 -- 172.21.15.72:0/2977590072 shutdown_connections 2024-04-26T21:59:59.064 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:59.063+0000 7f2174a00640 1 --2- 172.21.15.72:0/2977590072 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2170108520 0x7f217019ffa0 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:59.064 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:59.063+0000 7f2174a00640 1 --2- 172.21.15.72:0/2977590072 >> [v2:172.21.15.72:6800/588150172,v1:172.21.15.72:6801/588150172] conn(0x7f21540a3c40 0x7f21540a6100 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:59.064 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:59.063+0000 7f2174a00640 1 --2- 172.21.15.72:0/2977590072 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2170105bf0 0x7f2170198f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:59.064 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:59.063+0000 7f2174a00640 1 --2- 172.21.15.72:0/2977590072 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f21701032c0 0x7f2170198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:59.064 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:59.063+0000 7f2174a00640 1 -- 172.21.15.72:0/2977590072 >> 172.21.15.72:0/2977590072 conn(0x7f21700fccd0 msgr2=0x7f2170103de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:59.064 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:59.063+0000 7f2174a00640 1 -- 172.21.15.72:0/2977590072 shutdown_connections 2024-04-26T21:59:59.064 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T21:59:59.064+0000 7f2174a00640 1 -- 172.21.15.72:0/2977590072 wait complete. 2024-04-26T21:59:59.088 DEBUG:teuthology.orchestra.run.smithi072:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-26T21:59:59.155 DEBUG:teuthology.orchestra.run.smithi076:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-26T21:59:59.253 DEBUG:teuthology.orchestra.run.smithi164:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-26T21:59:59.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:59:59 smithi076 ceph-mon[31823]: pgmap v586: 97 pgs: 97 active+clean; 1.2 MiB data, 656 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.1 KiB/s wr, 12 op/s 2024-04-26T21:59:59.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:59:59 smithi076 ceph-mon[31823]: from='client.? 172.21.15.72:0/1206930978' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-26T21:59:59.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:59:59 smithi076 ceph-mon[31823]: osdmap e93: 12 total, 12 up, 12 in 2024-04-26T21:59:59.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 21:59:59 smithi076 ceph-mon[31823]: from='client.? 172.21.15.72:0/2977590072' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-26T21:59:59.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:59:59 smithi072 ceph-mon[28138]: pgmap v586: 97 pgs: 97 active+clean; 1.2 MiB data, 656 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.1 KiB/s wr, 12 op/s 2024-04-26T21:59:59.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:59:59 smithi072 ceph-mon[28138]: from='client.? 172.21.15.72:0/1206930978' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-26T21:59:59.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:59:59 smithi072 ceph-mon[28138]: osdmap e93: 12 total, 12 up, 12 in 2024-04-26T21:59:59.322 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:59:59 smithi072 ceph-mon[28138]: from='client.? 172.21.15.72:0/2977590072' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-26T21:59:59.326 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:59:59 smithi164 ceph-mon[31916]: pgmap v586: 97 pgs: 97 active+clean; 1.2 MiB data, 656 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.1 KiB/s wr, 12 op/s 2024-04-26T21:59:59.326 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:59:59 smithi164 ceph-mon[31916]: from='client.? 172.21.15.72:0/1206930978' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-26T21:59:59.326 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:59:59 smithi164 ceph-mon[31916]: osdmap e93: 12 total, 12 up, 12 in 2024-04-26T21:59:59.326 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 21:59:59 smithi164 ceph-mon[31916]: from='client.? 172.21.15.72:0/2977590072' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-26T21:59:59.340 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-04-26T21:59:59.340 DEBUG:teuthology.orchestra.run.smithi072:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-26T21:59:59.393 DEBUG:teuthology.orchestra.run.smithi076:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-26T21:59:59.452 DEBUG:teuthology.orchestra.run.smithi164:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-26T21:59:59.508 INFO:tasks.cephadm:Stopping all daemons... 2024-04-26T21:59:59.508 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-04-26T21:59:59.508 DEBUG:teuthology.orchestra.run.smithi072:> sudo systemctl stop ceph-a1780950-0415-11ef-bc93-c7b262605968@mon.a 2024-04-26T22:00:00.072 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 21:59:59 smithi072 systemd[1]: Stopping Ceph mon.a for a1780950-0415-11ef-bc93-c7b262605968... 2024-04-26T22:00:00.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 22:00:00 smithi164 ceph-mon[31916]: from='client.? 172.21.15.72:0/2977590072' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-04-26T22:00:00.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 22:00:00 smithi164 ceph-mon[31916]: osdmap e94: 12 total, 12 up, 12 in 2024-04-26T22:00:00.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 22:00:00 smithi164 ceph-mon[31916]: overall HEALTH_OK 2024-04-26T22:00:00.529 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 22:00:00 smithi072 ceph-mon[28138]: from='client.? 172.21.15.72:0/2977590072' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-04-26T22:00:00.530 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 22:00:00 smithi072 ceph-mon[28138]: osdmap e94: 12 total, 12 up, 12 in 2024-04-26T22:00:00.530 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 22:00:00 smithi072 ceph-mon[28138]: overall HEALTH_OK 2024-04-26T22:00:00.530 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 22:00:00 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-mon-a[28134]: 2024-04-26T22:00:00.094+0000 7f165d400700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T22:00:00.530 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 22:00:00 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-mon-a[28134]: 2024-04-26T22:00:00.094+0000 7f165d400700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-04-26T22:00:00.530 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 22:00:00 smithi072 podman[88460]: 2024-04-26 22:00:00.156781419 +0000 UTC m=+0.214556251 container died bf7f10eb1c0959b58dac995a0b937d5e3e743d459deadcea0d4e15c595fa54bf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, org.label-schema.build-date=20240424) 2024-04-26T22:00:00.564 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 22:00:00 smithi076 ceph-mon[31823]: from='client.? 172.21.15.72:0/2977590072' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-04-26T22:00:00.564 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 22:00:00 smithi076 ceph-mon[31823]: osdmap e94: 12 total, 12 up, 12 in 2024-04-26T22:00:00.564 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 22:00:00 smithi076 ceph-mon[31823]: overall HEALTH_OK 2024-04-26T22:00:00.822 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 22:00:00 smithi072 podman[88460]: 2024-04-26 22:00:00.725489619 +0000 UTC m=+0.783264478 container cleanup bf7f10eb1c0959b58dac995a0b937d5e3e743d459deadcea0d4e15c595fa54bf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-mon-a, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-04-26T22:00:00.822 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 22:00:00 smithi072 bash[88460]: ceph-a1780950-0415-11ef-bc93-c7b262605968-mon-a 2024-04-26T22:00:01.333 DEBUG:teuthology.orchestra.run.smithi072:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a1780950-0415-11ef-bc93-c7b262605968@mon.a.service' 2024-04-26T22:00:01.407 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 22:00:01 smithi072 podman[88473]: 2024-04-26 22:00:01.103735585 +0000 UTC m=+0.940399498 container remove bf7f10eb1c0959b58dac995a0b937d5e3e743d459deadcea0d4e15c595fa54bf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-mon-a, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, org.label-schema.build-date=20240424, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.schema-version=1.0) 2024-04-26T22:00:01.407 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 22:00:01 smithi072 systemd[1]: ceph-a1780950-0415-11ef-bc93-c7b262605968@mon.a.service: Deactivated successfully. 2024-04-26T22:00:01.408 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 22:00:01 smithi072 systemd[1]: Stopped Ceph mon.a for a1780950-0415-11ef-bc93-c7b262605968. 2024-04-26T22:00:01.408 INFO:journalctl@ceph.mon.a.smithi072.stdout:Apr 26 22:00:01 smithi072 systemd[1]: ceph-a1780950-0415-11ef-bc93-c7b262605968@mon.a.service: Consumed 33.551s CPU time. 2024-04-26T22:00:01.461 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T22:00:01.462 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-04-26T22:00:01.462 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-04-26T22:00:01.462 DEBUG:teuthology.orchestra.run.smithi076:> sudo systemctl stop ceph-a1780950-0415-11ef-bc93-c7b262605968@mon.b 2024-04-26T22:00:01.814 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 22:00:01 smithi076 systemd[1]: Stopping Ceph mon.b for a1780950-0415-11ef-bc93-c7b262605968... 2024-04-26T22:00:02.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 22:00:02 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-mon-b[31819]: 2024-04-26T22:00:02.034+0000 7ffb93600700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T22:00:02.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 22:00:02 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-mon-b[31819]: 2024-04-26T22:00:02.034+0000 7ffb93600700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-04-26T22:00:02.315 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 22:00:02 smithi076 podman[64467]: 2024-04-26 22:00:02.148805456 +0000 UTC m=+0.258152537 container died 3c5a685d42be72f303a45f78f0af0fd7e7589cd7a8b21861f4f61ff00a8660e8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-mon-b, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T22:00:02.815 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 22:00:02 smithi076 podman[64467]: 2024-04-26 22:00:02.577165893 +0000 UTC m=+0.686512928 container cleanup 3c5a685d42be72f303a45f78f0af0fd7e7589cd7a8b21861f4f61ff00a8660e8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-mon-b, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2) 2024-04-26T22:00:02.815 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 22:00:02 smithi076 bash[64467]: ceph-a1780950-0415-11ef-bc93-c7b262605968-mon-b 2024-04-26T22:00:03.314 INFO:journalctl@ceph.mon.b.smithi076.stdout:Apr 26 22:00:02 smithi076 podman[64481]: 2024-04-26 22:00:02.920117942 +0000 UTC m=+0.752907996 container remove 3c5a685d42be72f303a45f78f0af0fd7e7589cd7a8b21861f4f61ff00a8660e8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-mon-b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T22:00:03.514 DEBUG:teuthology.orchestra.run.smithi076:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a1780950-0415-11ef-bc93-c7b262605968@mon.b.service' 2024-04-26T22:00:03.642 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T22:00:03.643 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-04-26T22:00:03.643 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-04-26T22:00:03.643 DEBUG:teuthology.orchestra.run.smithi164:> sudo systemctl stop ceph-a1780950-0415-11ef-bc93-c7b262605968@mon.c 2024-04-26T22:00:04.045 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 22:00:03 smithi164 systemd[1]: Stopping Ceph mon.c for a1780950-0415-11ef-bc93-c7b262605968... 2024-04-26T22:00:04.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 22:00:04 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-mon-c[31908]: 2024-04-26T22:00:04.199+0000 7f1ffa000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T22:00:04.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 22:00:04 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-mon-c[31908]: 2024-04-26T22:00:04.199+0000 7f1ffa000700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-04-26T22:00:04.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 22:00:04 smithi164 podman[65979]: 2024-04-26 22:00:04.240190361 +0000 UTC m=+0.160736749 container died acf9ab0946806dac3b7a628e7e123c99f2530d3e593a61757b9b660f139eaf31 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-mon-c, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-04-26T22:00:04.826 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 22:00:04 smithi164 podman[65979]: 2024-04-26 22:00:04.6757213 +0000 UTC m=+0.596267696 container cleanup acf9ab0946806dac3b7a628e7e123c99f2530d3e593a61757b9b660f139eaf31 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-mon-c, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True) 2024-04-26T22:00:04.826 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 22:00:04 smithi164 bash[65979]: ceph-a1780950-0415-11ef-bc93-c7b262605968-mon-c 2024-04-26T22:00:05.158 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 22:00:04 smithi164 podman[65993]: 2024-04-26 22:00:04.943841904 +0000 UTC m=+0.688254574 container remove acf9ab0946806dac3b7a628e7e123c99f2530d3e593a61757b9b660f139eaf31 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-mon-c, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T22:00:05.335 DEBUG:teuthology.orchestra.run.smithi164:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a1780950-0415-11ef-bc93-c7b262605968@mon.c.service' 2024-04-26T22:00:05.440 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 22:00:05 smithi164 systemd[1]: ceph-a1780950-0415-11ef-bc93-c7b262605968@mon.c.service: Deactivated successfully. 2024-04-26T22:00:05.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 22:00:05 smithi164 systemd[1]: Stopped Ceph mon.c for a1780950-0415-11ef-bc93-c7b262605968. 2024-04-26T22:00:05.441 INFO:journalctl@ceph.mon.c.smithi164.stdout:Apr 26 22:00:05 smithi164 systemd[1]: ceph-a1780950-0415-11ef-bc93-c7b262605968@mon.c.service: Consumed 21.276s CPU time. 2024-04-26T22:00:05.448 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T22:00:05.448 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-04-26T22:00:05.448 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-04-26T22:00:05.448 DEBUG:teuthology.orchestra.run.smithi072:> sudo systemctl stop ceph-a1780950-0415-11ef-bc93-c7b262605968@mgr.x 2024-04-26T22:00:05.822 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 26 22:00:05 smithi072 systemd[1]: Stopping Ceph mgr.x for a1780950-0415-11ef-bc93-c7b262605968... 2024-04-26T22:00:06.322 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 26 22:00:05 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-mgr-x[28363]: 2024-04-26T22:00:05.973+0000 7fb1a9000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T22:00:06.322 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 26 22:00:06 smithi072 podman[88584]: 2024-04-26 22:00:06.066509296 +0000 UTC m=+0.211923443 container died dd3246b2bcf10b1d2d10f3c6483838fea328137bfe9dd2dcfd4189150d4fa274 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-mgr-x, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T22:00:06.645 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 26 22:00:06 smithi072 podman[88584]: 2024-04-26 22:00:06.621808757 +0000 UTC m=+0.767222950 container cleanup dd3246b2bcf10b1d2d10f3c6483838fea328137bfe9dd2dcfd4189150d4fa274 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-mgr-x, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-04-26T22:00:06.646 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 26 22:00:06 smithi072 bash[88584]: ceph-a1780950-0415-11ef-bc93-c7b262605968-mgr-x 2024-04-26T22:00:07.310 INFO:journalctl@ceph.mgr.x.smithi072.stdout:Apr 26 22:00:06 smithi072 podman[88598]: 2024-04-26 22:00:06.874541664 +0000 UTC m=+0.793099298 container remove dd3246b2bcf10b1d2d10f3c6483838fea328137bfe9dd2dcfd4189150d4fa274 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-mgr-x, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0) 2024-04-26T22:00:07.335 DEBUG:teuthology.orchestra.run.smithi072:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a1780950-0415-11ef-bc93-c7b262605968@mgr.x.service' 2024-04-26T22:00:07.440 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T22:00:07.440 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-04-26T22:00:07.440 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-04-26T22:00:07.441 DEBUG:teuthology.orchestra.run.smithi076:> sudo systemctl stop ceph-a1780950-0415-11ef-bc93-c7b262605968@mgr.y 2024-04-26T22:00:07.814 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Apr 26 22:00:07 smithi076 systemd[1]: Stopping Ceph mgr.y for a1780950-0415-11ef-bc93-c7b262605968... 2024-04-26T22:00:08.314 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Apr 26 22:00:08 smithi076 podman[64599]: 2024-04-26 22:00:08.059608174 +0000 UTC m=+0.198533610 container died 94ad75b255dbab5cf932bef15ed8ed469ad7902a52046655778c28c29b7f271c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-mgr-y, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-04-26T22:00:08.720 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Apr 26 22:00:08 smithi076 podman[64599]: 2024-04-26 22:00:08.508122687 +0000 UTC m=+0.647048173 container cleanup 94ad75b255dbab5cf932bef15ed8ed469ad7902a52046655778c28c29b7f271c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-mgr-y, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-04-26T22:00:08.720 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Apr 26 22:00:08 smithi076 bash[64599]: ceph-a1780950-0415-11ef-bc93-c7b262605968-mgr-y 2024-04-26T22:00:09.064 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Apr 26 22:00:08 smithi076 podman[64611]: 2024-04-26 22:00:08.717346899 +0000 UTC m=+0.644149406 container remove 94ad75b255dbab5cf932bef15ed8ed469ad7902a52046655778c28c29b7f271c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-mgr-y, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T22:00:09.064 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Apr 26 22:00:08 smithi076 systemd[1]: ceph-a1780950-0415-11ef-bc93-c7b262605968@mgr.y.service: Main process exited, code=exited, status=143/n/a 2024-04-26T22:00:09.212 DEBUG:teuthology.orchestra.run.smithi076:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a1780950-0415-11ef-bc93-c7b262605968@mgr.y.service' 2024-04-26T22:00:09.324 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T22:00:09.324 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-04-26T22:00:09.324 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-04-26T22:00:09.324 DEBUG:teuthology.orchestra.run.smithi164:> sudo systemctl stop ceph-a1780950-0415-11ef-bc93-c7b262605968@mgr.z 2024-04-26T22:00:09.690 INFO:journalctl@ceph.mgr.z.smithi164.stdout:Apr 26 22:00:09 smithi164 systemd[1]: Stopping Ceph mgr.z for a1780950-0415-11ef-bc93-c7b262605968... 2024-04-26T22:00:10.346 INFO:journalctl@ceph.mgr.z.smithi164.stdout:Apr 26 22:00:09 smithi164 podman[66111]: 2024-04-26 22:00:09.99431545 +0000 UTC m=+0.250981950 container died 28ad3af3bba17086d3d40a6b79cc6e1b84de81f57839e8bc7ce787885a1e40c0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-mgr-z, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.license=GPLv2) 2024-04-26T22:00:10.690 INFO:journalctl@ceph.mgr.z.smithi164.stdout:Apr 26 22:00:10 smithi164 podman[66111]: 2024-04-26 22:00:10.504956995 +0000 UTC m=+0.761623447 container cleanup 28ad3af3bba17086d3d40a6b79cc6e1b84de81f57839e8bc7ce787885a1e40c0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-mgr-z, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240424, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-04-26T22:00:10.690 INFO:journalctl@ceph.mgr.z.smithi164.stdout:Apr 26 22:00:10 smithi164 bash[66111]: ceph-a1780950-0415-11ef-bc93-c7b262605968-mgr-z 2024-04-26T22:00:11.190 INFO:journalctl@ceph.mgr.z.smithi164.stdout:Apr 26 22:00:10 smithi164 podman[66125]: 2024-04-26 22:00:10.748261225 +0000 UTC m=+0.734201780 container remove 28ad3af3bba17086d3d40a6b79cc6e1b84de81f57839e8bc7ce787885a1e40c0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-mgr-z, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True) 2024-04-26T22:00:11.191 INFO:journalctl@ceph.mgr.z.smithi164.stdout:Apr 26 22:00:10 smithi164 systemd[1]: ceph-a1780950-0415-11ef-bc93-c7b262605968@mgr.z.service: Main process exited, code=exited, status=143/n/a 2024-04-26T22:00:11.246 DEBUG:teuthology.orchestra.run.smithi164:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a1780950-0415-11ef-bc93-c7b262605968@mgr.z.service' 2024-04-26T22:00:11.359 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T22:00:11.360 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-04-26T22:00:11.360 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-04-26T22:00:11.360 DEBUG:teuthology.orchestra.run.smithi072:> sudo systemctl stop ceph-a1780950-0415-11ef-bc93-c7b262605968@osd.0 2024-04-26T22:00:11.821 INFO:journalctl@ceph.osd.0.smithi072.stdout:Apr 26 22:00:11 smithi072 systemd[1]: Stopping Ceph osd.0 for a1780950-0415-11ef-bc93-c7b262605968... 2024-04-26T22:00:12.322 INFO:journalctl@ceph.osd.0.smithi072.stdout:Apr 26 22:00:11 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-0[43226]: 2024-04-26T22:00:11.960+0000 7fa86fa00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T22:00:12.322 INFO:journalctl@ceph.osd.0.smithi072.stdout:Apr 26 22:00:11 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-0[43226]: 2024-04-26T22:00:11.960+0000 7fa86fa00700 -1 osd.0 94 *** Got signal Terminated *** 2024-04-26T22:00:12.322 INFO:journalctl@ceph.osd.0.smithi072.stdout:Apr 26 22:00:11 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-0[43226]: 2024-04-26T22:00:11.960+0000 7fa86fa00700 -1 osd.0 94 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-26T22:00:17.764 INFO:journalctl@ceph.osd.0.smithi072.stdout:Apr 26 22:00:17 smithi072 podman[88703]: 2024-04-26 22:00:17.422445065 +0000 UTC m=+5.610882696 container died 4cb4f728f75d1b0a95fe42052afc71fda2772bd10aa5fc64eaf199ad292771f4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-0, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff) 2024-04-26T22:00:18.071 INFO:journalctl@ceph.osd.0.smithi072.stdout:Apr 26 22:00:17 smithi072 podman[88703]: 2024-04-26 22:00:17.957219799 +0000 UTC m=+6.145657459 container cleanup 4cb4f728f75d1b0a95fe42052afc71fda2772bd10aa5fc64eaf199ad292771f4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-0, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-04-26T22:00:18.072 INFO:journalctl@ceph.osd.0.smithi072.stdout:Apr 26 22:00:17 smithi072 bash[88703]: ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-0 2024-04-26T22:00:18.571 INFO:journalctl@ceph.osd.0.smithi072.stdout:Apr 26 22:00:18 smithi072 podman[88723]: 2024-04-26 22:00:18.236859156 +0000 UTC m=+0.798587236 container remove 4cb4f728f75d1b0a95fe42052afc71fda2772bd10aa5fc64eaf199ad292771f4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-04-26T22:00:19.319 INFO:journalctl@ceph.osd.0.smithi072.stdout:Apr 26 22:00:19 smithi072 podman[88813]: 2024-04-26 22:00:18.956097373 +0000 UTC m=+0.037485717 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-26T22:00:19.319 INFO:journalctl@ceph.osd.0.smithi072.stdout:Apr 26 22:00:19 smithi072 podman[88813]: 2024-04-26 22:00:19.055326928 +0000 UTC m=+0.136715309 container create df3229dc80faf96bab8bf0c9174a40e25a9e7dfddd202a8e181e73e0d84a195a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-0-deactivate, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-04-26T22:00:19.319 INFO:journalctl@ceph.osd.0.smithi072.stdout:Apr 26 22:00:19 smithi072 podman[88813]: 2024-04-26 22:00:19.318979668 +0000 UTC m=+0.400368055 container init df3229dc80faf96bab8bf0c9174a40e25a9e7dfddd202a8e181e73e0d84a195a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-0-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD) 2024-04-26T22:00:19.571 INFO:journalctl@ceph.osd.0.smithi072.stdout:Apr 26 22:00:19 smithi072 podman[88813]: 2024-04-26 22:00:19.331191601 +0000 UTC m=+0.412579954 container start df3229dc80faf96bab8bf0c9174a40e25a9e7dfddd202a8e181e73e0d84a195a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-0-deactivate, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True) 2024-04-26T22:00:19.572 INFO:journalctl@ceph.osd.0.smithi072.stdout:Apr 26 22:00:19 smithi072 podman[88813]: 2024-04-26 22:00:19.409507549 +0000 UTC m=+0.490895937 container attach df3229dc80faf96bab8bf0c9174a40e25a9e7dfddd202a8e181e73e0d84a195a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-0-deactivate, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_BRANCH=HEAD) 2024-04-26T22:00:20.072 INFO:journalctl@ceph.osd.0.smithi072.stdout:Apr 26 22:00:19 smithi072 podman[88813]: 2024-04-26 22:00:19.78352516 +0000 UTC m=+0.864913549 container died df3229dc80faf96bab8bf0c9174a40e25a9e7dfddd202a8e181e73e0d84a195a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-0-deactivate, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T22:00:20.608 DEBUG:teuthology.orchestra.run.smithi072:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a1780950-0415-11ef-bc93-c7b262605968@osd.0.service' 2024-04-26T22:00:20.720 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T22:00:20.720 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-04-26T22:00:20.721 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-04-26T22:00:20.721 DEBUG:teuthology.orchestra.run.smithi076:> sudo systemctl stop ceph-a1780950-0415-11ef-bc93-c7b262605968@osd.1 2024-04-26T22:00:21.065 INFO:journalctl@ceph.osd.1.smithi076.stdout:Apr 26 22:00:20 smithi076 systemd[1]: Stopping Ceph osd.1 for a1780950-0415-11ef-bc93-c7b262605968... 2024-04-26T22:00:21.815 INFO:journalctl@ceph.osd.1.smithi076.stdout:Apr 26 22:00:21 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-1[36420]: 2024-04-26T22:00:21.318+0000 7feaacc00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T22:00:21.815 INFO:journalctl@ceph.osd.1.smithi076.stdout:Apr 26 22:00:21 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-1[36420]: 2024-04-26T22:00:21.318+0000 7feaacc00700 -1 osd.1 94 *** Got signal Terminated *** 2024-04-26T22:00:21.815 INFO:journalctl@ceph.osd.1.smithi076.stdout:Apr 26 22:00:21 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-1[36420]: 2024-04-26T22:00:21.318+0000 7feaacc00700 -1 osd.1 94 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-26T22:00:27.065 INFO:journalctl@ceph.osd.1.smithi076.stdout:Apr 26 22:00:26 smithi076 podman[64723]: 2024-04-26 22:00:26.713135085 +0000 UTC m=+5.542592951 container died fbed269c3f5727ba59003e8f16fcb76f6af66fe54fa47335f8bd40dbef7a548e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-1, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240424) 2024-04-26T22:00:27.429 INFO:journalctl@ceph.osd.1.smithi076.stdout:Apr 26 22:00:27 smithi076 podman[64723]: 2024-04-26 22:00:27.187436756 +0000 UTC m=+6.016894679 container cleanup fbed269c3f5727ba59003e8f16fcb76f6af66fe54fa47335f8bd40dbef7a548e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-1, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T22:00:27.429 INFO:journalctl@ceph.osd.1.smithi076.stdout:Apr 26 22:00:27 smithi076 bash[64723]: ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-1 2024-04-26T22:00:27.814 INFO:journalctl@ceph.osd.1.smithi076.stdout:Apr 26 22:00:27 smithi076 podman[64740]: 2024-04-26 22:00:27.429045195 +0000 UTC m=+0.702358115 container remove fbed269c3f5727ba59003e8f16fcb76f6af66fe54fa47335f8bd40dbef7a548e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-1, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5) 2024-04-26T22:00:28.564 INFO:journalctl@ceph.osd.1.smithi076.stdout:Apr 26 22:00:28 smithi076 podman[64829]: 2024-04-26 22:00:28.202041528 +0000 UTC m=+0.035099597 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-26T22:00:28.565 INFO:journalctl@ceph.osd.1.smithi076.stdout:Apr 26 22:00:28 smithi076 podman[64829]: 2024-04-26 22:00:28.329964675 +0000 UTC m=+0.163022753 container create 58453b1fa45a38523789cdc0c18431542967a3b95b452f1db86045bb6e796ce5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-1-deactivate, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-26T22:00:29.064 INFO:journalctl@ceph.osd.1.smithi076.stdout:Apr 26 22:00:28 smithi076 podman[64829]: 2024-04-26 22:00:28.617600173 +0000 UTC m=+0.450658265 container init 58453b1fa45a38523789cdc0c18431542967a3b95b452f1db86045bb6e796ce5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-1-deactivate, org.label-schema.build-date=20240424, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-04-26T22:00:29.064 INFO:journalctl@ceph.osd.1.smithi076.stdout:Apr 26 22:00:28 smithi076 podman[64829]: 2024-04-26 22:00:28.631922757 +0000 UTC m=+0.464980898 container start 58453b1fa45a38523789cdc0c18431542967a3b95b452f1db86045bb6e796ce5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-1-deactivate, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff) 2024-04-26T22:00:29.064 INFO:journalctl@ceph.osd.1.smithi076.stdout:Apr 26 22:00:28 smithi076 podman[64829]: 2024-04-26 22:00:28.70681641 +0000 UTC m=+0.539874544 container attach 58453b1fa45a38523789cdc0c18431542967a3b95b452f1db86045bb6e796ce5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-1-deactivate, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, GIT_CLEAN=True) 2024-04-26T22:00:29.416 INFO:journalctl@ceph.osd.1.smithi076.stdout:Apr 26 22:00:29 smithi076 podman[64829]: 2024-04-26 22:00:29.092911683 +0000 UTC m=+0.925969767 container died 58453b1fa45a38523789cdc0c18431542967a3b95b452f1db86045bb6e796ce5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-1-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-04-26T22:00:29.754 DEBUG:teuthology.orchestra.run.smithi076:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a1780950-0415-11ef-bc93-c7b262605968@osd.1.service' 2024-04-26T22:00:29.857 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T22:00:29.857 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-04-26T22:00:29.857 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-04-26T22:00:29.857 DEBUG:teuthology.orchestra.run.smithi164:> sudo systemctl stop ceph-a1780950-0415-11ef-bc93-c7b262605968@osd.2 2024-04-26T22:00:30.190 INFO:journalctl@ceph.osd.2.smithi164.stdout:Apr 26 22:00:29 smithi164 systemd[1]: Stopping Ceph osd.2 for a1780950-0415-11ef-bc93-c7b262605968... 2024-04-26T22:00:30.690 INFO:journalctl@ceph.osd.2.smithi164.stdout:Apr 26 22:00:30 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-2[36918]: 2024-04-26T22:00:30.404+0000 7feec0600700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T22:00:30.690 INFO:journalctl@ceph.osd.2.smithi164.stdout:Apr 26 22:00:30 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-2[36918]: 2024-04-26T22:00:30.404+0000 7feec0600700 -1 osd.2 94 *** Got signal Terminated *** 2024-04-26T22:00:30.691 INFO:journalctl@ceph.osd.2.smithi164.stdout:Apr 26 22:00:30 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-2[36918]: 2024-04-26T22:00:30.404+0000 7feec0600700 -1 osd.2 94 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-26T22:00:36.313 INFO:journalctl@ceph.osd.2.smithi164.stdout:Apr 26 22:00:35 smithi164 podman[66234]: 2024-04-26 22:00:35.9989329 +0000 UTC m=+5.729205567 container died 60a3c1cb6581dd16380a9c6f3c6c269166e561cd87d88e4da7485bcaf58f13a5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-2, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5) 2024-04-26T22:00:36.690 INFO:journalctl@ceph.osd.2.smithi164.stdout:Apr 26 22:00:36 smithi164 podman[66234]: 2024-04-26 22:00:36.458119169 +0000 UTC m=+6.188391873 container cleanup 60a3c1cb6581dd16380a9c6f3c6c269166e561cd87d88e4da7485bcaf58f13a5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-04-26T22:00:36.691 INFO:journalctl@ceph.osd.2.smithi164.stdout:Apr 26 22:00:36 smithi164 bash[66234]: ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-2 2024-04-26T22:00:37.190 INFO:journalctl@ceph.osd.2.smithi164.stdout:Apr 26 22:00:36 smithi164 podman[66250]: 2024-04-26 22:00:36.71187066 +0000 UTC m=+0.699717774 container remove 60a3c1cb6581dd16380a9c6f3c6c269166e561cd87d88e4da7485bcaf58f13a5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.build-date=20240424) 2024-04-26T22:00:37.787 INFO:journalctl@ceph.osd.2.smithi164.stdout:Apr 26 22:00:37 smithi164 podman[66348]: 2024-04-26 22:00:37.434932039 +0000 UTC m=+0.035484833 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-26T22:00:37.787 INFO:journalctl@ceph.osd.2.smithi164.stdout:Apr 26 22:00:37 smithi164 podman[66348]: 2024-04-26 22:00:37.56860458 +0000 UTC m=+0.169157412 container create a63903f8f91be29302d1d701de1dbd98bddf22e0deeb573d37306c61852a9b24 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-2-deactivate, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, maintainer=Guillaume Abrioux ) 2024-04-26T22:00:38.190 INFO:journalctl@ceph.osd.2.smithi164.stdout:Apr 26 22:00:37 smithi164 podman[66348]: 2024-04-26 22:00:37.840401666 +0000 UTC m=+0.440954531 container init a63903f8f91be29302d1d701de1dbd98bddf22e0deeb573d37306c61852a9b24 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-2-deactivate, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T22:00:38.190 INFO:journalctl@ceph.osd.2.smithi164.stdout:Apr 26 22:00:37 smithi164 podman[66348]: 2024-04-26 22:00:37.851786005 +0000 UTC m=+0.452338867 container start a63903f8f91be29302d1d701de1dbd98bddf22e0deeb573d37306c61852a9b24 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-2-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff) 2024-04-26T22:00:38.190 INFO:journalctl@ceph.osd.2.smithi164.stdout:Apr 26 22:00:37 smithi164 podman[66348]: 2024-04-26 22:00:37.929251437 +0000 UTC m=+0.529804275 container attach a63903f8f91be29302d1d701de1dbd98bddf22e0deeb573d37306c61852a9b24 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-2-deactivate, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-04-26T22:00:38.691 INFO:journalctl@ceph.osd.2.smithi164.stdout:Apr 26 22:00:38 smithi164 podman[66348]: 2024-04-26 22:00:38.310465263 +0000 UTC m=+0.911018105 container died a63903f8f91be29302d1d701de1dbd98bddf22e0deeb573d37306c61852a9b24 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-2-deactivate, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, ceph=True) 2024-04-26T22:00:39.033 INFO:journalctl@ceph.osd.5.smithi164.stdout:Apr 26 22:00:38 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-5[43528]: 2024-04-26T22:00:38.734+0000 7fb90c200700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:15.638228+0000 front 2024-04-26T22:00:15.637988+0000 (oldest deadline 2024-04-26T22:00:38.538212+0000) 2024-04-26T22:00:39.080 DEBUG:teuthology.orchestra.run.smithi164:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a1780950-0415-11ef-bc93-c7b262605968@osd.2.service' 2024-04-26T22:00:39.188 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T22:00:39.188 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-04-26T22:00:39.188 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-04-26T22:00:39.188 DEBUG:teuthology.orchestra.run.smithi072:> sudo systemctl stop ceph-a1780950-0415-11ef-bc93-c7b262605968@osd.3 2024-04-26T22:00:39.440 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:00:39 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:00:39.051+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:14.343204+0000 front 2024-04-26T22:00:14.342815+0000 (oldest deadline 2024-04-26T22:00:39.042910+0000) 2024-04-26T22:00:39.572 INFO:journalctl@ceph.osd.3.smithi072.stdout:Apr 26 22:00:39 smithi072 systemd[1]: Stopping Ceph osd.3 for a1780950-0415-11ef-bc93-c7b262605968... 2024-04-26T22:00:40.036 INFO:journalctl@ceph.osd.5.smithi164.stdout:Apr 26 22:00:39 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-5[43528]: 2024-04-26T22:00:39.725+0000 7fb90c200700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:15.638228+0000 front 2024-04-26T22:00:15.637988+0000 (oldest deadline 2024-04-26T22:00:38.538212+0000) 2024-04-26T22:00:40.072 INFO:journalctl@ceph.osd.3.smithi072.stdout:Apr 26 22:00:39 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-3[50018]: 2024-04-26T22:00:39.747+0000 7f6044000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T22:00:40.072 INFO:journalctl@ceph.osd.3.smithi072.stdout:Apr 26 22:00:39 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-3[50018]: 2024-04-26T22:00:39.747+0000 7f6044000700 -1 osd.3 94 *** Got signal Terminated *** 2024-04-26T22:00:40.072 INFO:journalctl@ceph.osd.3.smithi072.stdout:Apr 26 22:00:39 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-3[50018]: 2024-04-26T22:00:39.747+0000 7f6044000700 -1 osd.3 94 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-26T22:00:40.440 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:00:40 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:00:40.035+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:14.343204+0000 front 2024-04-26T22:00:14.342815+0000 (oldest deadline 2024-04-26T22:00:39.042910+0000) 2024-04-26T22:00:40.821 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:00:40 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:00:40.391+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:16.595921+0000 front 2024-04-26T22:00:16.596275+0000 (oldest deadline 2024-04-26T22:00:39.495420+0000) 2024-04-26T22:00:41.065 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:00:40 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:00:40.760+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:11.890629+0000 front 2024-04-26T22:00:11.890849+0000 (oldest deadline 2024-04-26T22:00:40.692043+0000) 2024-04-26T22:00:41.076 INFO:journalctl@ceph.osd.5.smithi164.stdout:Apr 26 22:00:40 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-5[43528]: 2024-04-26T22:00:40.768+0000 7fb90c200700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:15.638228+0000 front 2024-04-26T22:00:15.637988+0000 (oldest deadline 2024-04-26T22:00:38.538212+0000) 2024-04-26T22:00:41.440 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:00:41 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:00:41.076+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:14.343204+0000 front 2024-04-26T22:00:14.342815+0000 (oldest deadline 2024-04-26T22:00:39.042910+0000) 2024-04-26T22:00:41.571 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:00:41 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:00:41.411+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:16.595921+0000 front 2024-04-26T22:00:16.596275+0000 (oldest deadline 2024-04-26T22:00:39.495420+0000) 2024-04-26T22:00:42.064 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:00:41 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:00:41.726+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:11.890629+0000 front 2024-04-26T22:00:11.890849+0000 (oldest deadline 2024-04-26T22:00:40.692043+0000) 2024-04-26T22:00:42.100 INFO:journalctl@ceph.osd.5.smithi164.stdout:Apr 26 22:00:41 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-5[43528]: 2024-04-26T22:00:41.768+0000 7fb90c200700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:15.638228+0000 front 2024-04-26T22:00:15.637988+0000 (oldest deadline 2024-04-26T22:00:38.538212+0000) 2024-04-26T22:00:42.219 INFO:journalctl@ceph.osd.6.smithi072.stdout:Apr 26 22:00:41 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-6[57091]: 2024-04-26T22:00:41.944+0000 7fb154800700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:16.543358+0000 front 2024-04-26T22:00:16.543710+0000 (oldest deadline 2024-04-26T22:00:41.744798+0000) 2024-04-26T22:00:42.440 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:00:42 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:00:42.099+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:14.343204+0000 front 2024-04-26T22:00:14.342815+0000 (oldest deadline 2024-04-26T22:00:39.042910+0000) 2024-04-26T22:00:42.571 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:00:42 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:00:42.437+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:16.595921+0000 front 2024-04-26T22:00:16.596275+0000 (oldest deadline 2024-04-26T22:00:39.495420+0000) 2024-04-26T22:00:43.064 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:00:42 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:00:42.708+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:11.890629+0000 front 2024-04-26T22:00:11.890849+0000 (oldest deadline 2024-04-26T22:00:40.692043+0000) 2024-04-26T22:00:43.083 INFO:journalctl@ceph.osd.5.smithi164.stdout:Apr 26 22:00:42 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-5[43528]: 2024-04-26T22:00:42.782+0000 7fb90c200700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:15.638228+0000 front 2024-04-26T22:00:15.637988+0000 (oldest deadline 2024-04-26T22:00:38.538212+0000) 2024-04-26T22:00:43.321 INFO:journalctl@ceph.osd.6.smithi072.stdout:Apr 26 22:00:42 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-6[57091]: 2024-04-26T22:00:42.990+0000 7fb154800700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:16.543358+0000 front 2024-04-26T22:00:16.543710+0000 (oldest deadline 2024-04-26T22:00:41.744798+0000) 2024-04-26T22:00:43.440 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:00:43 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:00:43.083+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:14.343204+0000 front 2024-04-26T22:00:14.342815+0000 (oldest deadline 2024-04-26T22:00:39.042910+0000) 2024-04-26T22:00:43.821 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:00:43 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:00:43.472+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:16.595921+0000 front 2024-04-26T22:00:16.596275+0000 (oldest deadline 2024-04-26T22:00:39.495420+0000) 2024-04-26T22:00:44.059 INFO:journalctl@ceph.osd.5.smithi164.stdout:Apr 26 22:00:43 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-5[43528]: 2024-04-26T22:00:43.748+0000 7fb90c200700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:15.638228+0000 front 2024-04-26T22:00:15.637988+0000 (oldest deadline 2024-04-26T22:00:38.538212+0000) 2024-04-26T22:00:44.064 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:00:43 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:00:43.734+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:11.890629+0000 front 2024-04-26T22:00:11.890849+0000 (oldest deadline 2024-04-26T22:00:40.692043+0000) 2024-04-26T22:00:44.321 INFO:journalctl@ceph.osd.6.smithi072.stdout:Apr 26 22:00:44 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-6[57091]: 2024-04-26T22:00:44.016+0000 7fb154800700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:16.543358+0000 front 2024-04-26T22:00:16.543710+0000 (oldest deadline 2024-04-26T22:00:41.744798+0000) 2024-04-26T22:00:44.440 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:00:44 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:00:44.059+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:14.343204+0000 front 2024-04-26T22:00:14.342815+0000 (oldest deadline 2024-04-26T22:00:39.042910+0000) 2024-04-26T22:00:44.821 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:00:44 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:00:44.453+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:16.595921+0000 front 2024-04-26T22:00:16.596275+0000 (oldest deadline 2024-04-26T22:00:39.495420+0000) 2024-04-26T22:00:45.064 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:00:44 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:00:44.715+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:11.890629+0000 front 2024-04-26T22:00:11.890849+0000 (oldest deadline 2024-04-26T22:00:40.692043+0000) 2024-04-26T22:00:45.067 INFO:journalctl@ceph.osd.5.smithi164.stdout:Apr 26 22:00:44 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-5[43528]: 2024-04-26T22:00:44.778+0000 7fb90c200700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:15.638228+0000 front 2024-04-26T22:00:15.637988+0000 (oldest deadline 2024-04-26T22:00:38.538212+0000) 2024-04-26T22:00:45.322 INFO:journalctl@ceph.osd.6.smithi072.stdout:Apr 26 22:00:45 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-6[57091]: 2024-04-26T22:00:45.060+0000 7fb154800700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:16.543358+0000 front 2024-04-26T22:00:16.543710+0000 (oldest deadline 2024-04-26T22:00:41.744798+0000) 2024-04-26T22:00:45.440 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:00:45 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:00:45.066+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:14.343204+0000 front 2024-04-26T22:00:14.342815+0000 (oldest deadline 2024-04-26T22:00:39.042910+0000) 2024-04-26T22:00:45.649 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:00:45 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:00:45.503+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:16.595921+0000 front 2024-04-26T22:00:16.596275+0000 (oldest deadline 2024-04-26T22:00:39.495420+0000) 2024-04-26T22:00:45.649 INFO:journalctl@ceph.osd.3.smithi072.stdout:Apr 26 22:00:45 smithi072 podman[88917]: 2024-04-26 22:00:45.360354364 +0000 UTC m=+5.747868841 container died b0c2882cf0524ce5bd4803a39ee0c2c6356ba8c68c9fd6c6754abebf5ef7fb82 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-3, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-04-26T22:00:45.966 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:00:45 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:00:45.715+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:11.890629+0000 front 2024-04-26T22:00:11.890849+0000 (oldest deadline 2024-04-26T22:00:40.692043+0000) 2024-04-26T22:00:45.996 INFO:journalctl@ceph.osd.3.smithi072.stdout:Apr 26 22:00:45 smithi072 podman[88917]: 2024-04-26 22:00:45.80085588 +0000 UTC m=+6.188370421 container cleanup b0c2882cf0524ce5bd4803a39ee0c2c6356ba8c68c9fd6c6754abebf5ef7fb82 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-3, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-04-26T22:00:45.996 INFO:journalctl@ceph.osd.3.smithi072.stdout:Apr 26 22:00:45 smithi072 bash[88917]: ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-3 2024-04-26T22:00:46.190 INFO:journalctl@ceph.osd.5.smithi164.stdout:Apr 26 22:00:45 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-5[43528]: 2024-04-26T22:00:45.828+0000 7fb90c200700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:15.638228+0000 front 2024-04-26T22:00:15.637988+0000 (oldest deadline 2024-04-26T22:00:38.538212+0000) 2024-04-26T22:00:46.190 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:00:46 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:00:46.027+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:14.343204+0000 front 2024-04-26T22:00:14.342815+0000 (oldest deadline 2024-04-26T22:00:39.042910+0000) 2024-04-26T22:00:46.321 INFO:journalctl@ceph.osd.3.smithi072.stdout:Apr 26 22:00:45 smithi072 podman[88947]: 2024-04-26 22:00:45.996024508 +0000 UTC m=+0.620937822 container remove b0c2882cf0524ce5bd4803a39ee0c2c6356ba8c68c9fd6c6754abebf5ef7fb82 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-3, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-04-26T22:00:46.322 INFO:journalctl@ceph.osd.6.smithi072.stdout:Apr 26 22:00:46 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-6[57091]: 2024-04-26T22:00:46.093+0000 7fb154800700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:16.543358+0000 front 2024-04-26T22:00:16.543710+0000 (oldest deadline 2024-04-26T22:00:41.744798+0000) 2024-04-26T22:00:46.813 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:00:46 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:00:46.545+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:16.595921+0000 front 2024-04-26T22:00:16.596275+0000 (oldest deadline 2024-04-26T22:00:39.495420+0000) 2024-04-26T22:00:47.064 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:00:46 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:00:46.744+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:11.890629+0000 front 2024-04-26T22:00:11.890849+0000 (oldest deadline 2024-04-26T22:00:40.692043+0000) 2024-04-26T22:00:47.071 INFO:journalctl@ceph.osd.3.smithi072.stdout:Apr 26 22:00:46 smithi072 podman[89033]: 2024-04-26 22:00:46.718571885 +0000 UTC m=+0.032152905 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-26T22:00:47.072 INFO:journalctl@ceph.osd.3.smithi072.stdout:Apr 26 22:00:46 smithi072 podman[89033]: 2024-04-26 22:00:46.831237228 +0000 UTC m=+0.144818267 container create 7dd77c259efa469672a2bcb96a0c7d964327d1ea7460cd4bbfd5ebbf642e9b08 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-3-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-04-26T22:00:47.165 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:00:47 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:00:47.021+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:14.343204+0000 front 2024-04-26T22:00:14.342815+0000 (oldest deadline 2024-04-26T22:00:39.042910+0000) 2024-04-26T22:00:47.166 INFO:journalctl@ceph.osd.5.smithi164.stdout:Apr 26 22:00:46 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-5[43528]: 2024-04-26T22:00:46.828+0000 7fb90c200700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:15.638228+0000 front 2024-04-26T22:00:15.637988+0000 (oldest deadline 2024-04-26T22:00:38.538212+0000) 2024-04-26T22:00:47.523 INFO:journalctl@ceph.osd.7.smithi076.stdout:Apr 26 22:00:47 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-7[49518]: 2024-04-26T22:00:47.166+0000 7fcc04a00700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:21.667782+0000 front 2024-04-26T22:00:21.668606+0000 (oldest deadline 2024-04-26T22:00:46.967821+0000) 2024-04-26T22:00:47.524 INFO:journalctl@ceph.osd.6.smithi072.stdout:Apr 26 22:00:47 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-6[57091]: 2024-04-26T22:00:47.130+0000 7fb154800700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:16.543358+0000 front 2024-04-26T22:00:16.543710+0000 (oldest deadline 2024-04-26T22:00:41.744798+0000) 2024-04-26T22:00:47.524 INFO:journalctl@ceph.osd.3.smithi072.stdout:Apr 26 22:00:47 smithi072 podman[89033]: 2024-04-26 22:00:47.121499445 +0000 UTC m=+0.435080597 container init 7dd77c259efa469672a2bcb96a0c7d964327d1ea7460cd4bbfd5ebbf642e9b08 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-3-deactivate, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-04-26T22:00:47.524 INFO:journalctl@ceph.osd.3.smithi072.stdout:Apr 26 22:00:47 smithi072 podman[89033]: 2024-04-26 22:00:47.134578019 +0000 UTC m=+0.448159101 container start 7dd77c259efa469672a2bcb96a0c7d964327d1ea7460cd4bbfd5ebbf642e9b08 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-3-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-26T22:00:47.525 INFO:journalctl@ceph.osd.3.smithi072.stdout:Apr 26 22:00:47 smithi072 podman[89033]: 2024-04-26 22:00:47.218731343 +0000 UTC m=+0.532312380 container attach 7dd77c259efa469672a2bcb96a0c7d964327d1ea7460cd4bbfd5ebbf642e9b08 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-3-deactivate, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T22:00:47.814 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:00:47 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:00:47.713+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:11.890629+0000 front 2024-04-26T22:00:11.890849+0000 (oldest deadline 2024-04-26T22:00:40.692043+0000) 2024-04-26T22:00:47.821 INFO:journalctl@ceph.osd.3.smithi072.stdout:Apr 26 22:00:47 smithi072 podman[89033]: 2024-04-26 22:00:47.584401244 +0000 UTC m=+0.897982326 container died 7dd77c259efa469672a2bcb96a0c7d964327d1ea7460cd4bbfd5ebbf642e9b08 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-3-deactivate, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-04-26T22:00:47.822 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:00:47 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:00:47.523+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:16.595921+0000 front 2024-04-26T22:00:16.596275+0000 (oldest deadline 2024-04-26T22:00:39.495420+0000) 2024-04-26T22:00:48.190 INFO:journalctl@ceph.osd.5.smithi164.stdout:Apr 26 22:00:47 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-5[43528]: 2024-04-26T22:00:47.816+0000 7fb90c200700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:15.638228+0000 front 2024-04-26T22:00:15.637988+0000 (oldest deadline 2024-04-26T22:00:38.538212+0000) 2024-04-26T22:00:48.191 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:00:48 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:00:48.027+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:14.343204+0000 front 2024-04-26T22:00:14.342815+0000 (oldest deadline 2024-04-26T22:00:39.042910+0000) 2024-04-26T22:00:48.191 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:00:48 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:00:48.027+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:24.945823+0000 front 2024-04-26T22:00:24.945339+0000 (oldest deadline 2024-04-26T22:00:47.245223+0000) 2024-04-26T22:00:48.468 INFO:journalctl@ceph.osd.6.smithi072.stdout:Apr 26 22:00:48 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-6[57091]: 2024-04-26T22:00:48.144+0000 7fb154800700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:16.543358+0000 front 2024-04-26T22:00:16.543710+0000 (oldest deadline 2024-04-26T22:00:41.744798+0000) 2024-04-26T22:00:48.468 INFO:journalctl@ceph.osd.6.smithi072.stdout:Apr 26 22:00:48 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-6[57091]: 2024-04-26T22:00:48.144+0000 7fb154800700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:21.746250+0000 front 2024-04-26T22:00:21.745942+0000 (oldest deadline 2024-04-26T22:00:47.645522+0000) 2024-04-26T22:00:48.515 DEBUG:teuthology.orchestra.run.smithi072:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a1780950-0415-11ef-bc93-c7b262605968@osd.3.service' 2024-04-26T22:00:48.564 INFO:journalctl@ceph.osd.7.smithi076.stdout:Apr 26 22:00:48 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-7[49518]: 2024-04-26T22:00:48.118+0000 7fcc04a00700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:21.667782+0000 front 2024-04-26T22:00:21.668606+0000 (oldest deadline 2024-04-26T22:00:46.967821+0000) 2024-04-26T22:00:48.625 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T22:00:48.625 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-04-26T22:00:48.625 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-04-26T22:00:48.626 DEBUG:teuthology.orchestra.run.smithi076:> sudo systemctl stop ceph-a1780950-0415-11ef-bc93-c7b262605968@osd.4 2024-04-26T22:00:48.822 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:00:48 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:00:48.568+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:16.595921+0000 front 2024-04-26T22:00:16.596275+0000 (oldest deadline 2024-04-26T22:00:39.495420+0000) 2024-04-26T22:00:48.822 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:00:48 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:00:48.568+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:24.600976+0000 front 2024-04-26T22:00:24.600553+0000 (oldest deadline 2024-04-26T22:00:48.100712+0000) 2024-04-26T22:00:49.064 INFO:journalctl@ceph.osd.4.smithi076.stdout:Apr 26 22:00:48 smithi076 systemd[1]: Stopping Ceph osd.4 for a1780950-0415-11ef-bc93-c7b262605968... 2024-04-26T22:00:49.064 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:00:48 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:00:48.758+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:11.890629+0000 front 2024-04-26T22:00:11.890849+0000 (oldest deadline 2024-04-26T22:00:40.692043+0000) 2024-04-26T22:00:49.190 INFO:journalctl@ceph.osd.5.smithi164.stdout:Apr 26 22:00:48 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-5[43528]: 2024-04-26T22:00:48.789+0000 7fb90c200700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:15.638228+0000 front 2024-04-26T22:00:15.637988+0000 (oldest deadline 2024-04-26T22:00:38.538212+0000) 2024-04-26T22:00:49.191 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:00:49 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:00:49.037+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:14.343204+0000 front 2024-04-26T22:00:14.342815+0000 (oldest deadline 2024-04-26T22:00:39.042910+0000) 2024-04-26T22:00:49.191 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:00:49 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:00:49.037+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:24.945823+0000 front 2024-04-26T22:00:24.945339+0000 (oldest deadline 2024-04-26T22:00:47.245223+0000) 2024-04-26T22:00:49.566 INFO:journalctl@ceph.osd.7.smithi076.stdout:Apr 26 22:00:49 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-7[49518]: 2024-04-26T22:00:49.097+0000 7fcc04a00700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:21.667782+0000 front 2024-04-26T22:00:21.668606+0000 (oldest deadline 2024-04-26T22:00:46.967821+0000) 2024-04-26T22:00:49.566 INFO:journalctl@ceph.osd.4.smithi076.stdout:Apr 26 22:00:49 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-4[42901]: 2024-04-26T22:00:49.211+0000 7f805f800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T22:00:49.566 INFO:journalctl@ceph.osd.4.smithi076.stdout:Apr 26 22:00:49 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-4[42901]: 2024-04-26T22:00:49.211+0000 7f805f800700 -1 osd.4 94 *** Got signal Terminated *** 2024-04-26T22:00:49.566 INFO:journalctl@ceph.osd.4.smithi076.stdout:Apr 26 22:00:49 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-4[42901]: 2024-04-26T22:00:49.211+0000 7f805f800700 -1 osd.4 94 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-26T22:00:49.571 INFO:journalctl@ceph.osd.6.smithi072.stdout:Apr 26 22:00:49 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-6[57091]: 2024-04-26T22:00:49.118+0000 7fb154800700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:16.543358+0000 front 2024-04-26T22:00:16.543710+0000 (oldest deadline 2024-04-26T22:00:41.744798+0000) 2024-04-26T22:00:49.571 INFO:journalctl@ceph.osd.6.smithi072.stdout:Apr 26 22:00:49 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-6[57091]: 2024-04-26T22:00:49.118+0000 7fb154800700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:21.746250+0000 front 2024-04-26T22:00:21.745942+0000 (oldest deadline 2024-04-26T22:00:47.645522+0000) 2024-04-26T22:00:50.064 INFO:journalctl@ceph.osd.4.smithi076.stdout:Apr 26 22:00:49 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-4[42901]: 2024-04-26T22:00:49.769+0000 7f8056000700 -1 osd.4 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:26.221859+0000 front 2024-04-26T22:00:26.222154+0000 (oldest deadline 2024-04-26T22:00:49.722084+0000) 2024-04-26T22:00:50.064 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:00:49 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:00:49.782+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:11.890629+0000 front 2024-04-26T22:00:11.890849+0000 (oldest deadline 2024-04-26T22:00:40.692043+0000) 2024-04-26T22:00:50.065 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:00:49 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:00:49.782+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:24.194239+0000 front 2024-04-26T22:00:24.194095+0000 (oldest deadline 2024-04-26T22:00:48.893996+0000) 2024-04-26T22:00:50.071 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:00:49 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:00:49.583+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:16.595921+0000 front 2024-04-26T22:00:16.596275+0000 (oldest deadline 2024-04-26T22:00:39.495420+0000) 2024-04-26T22:00:50.071 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:00:49 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:00:49.583+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:24.600976+0000 front 2024-04-26T22:00:24.600553+0000 (oldest deadline 2024-04-26T22:00:48.100712+0000) 2024-04-26T22:00:50.086 INFO:journalctl@ceph.osd.5.smithi164.stdout:Apr 26 22:00:49 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-5[43528]: 2024-04-26T22:00:49.744+0000 7fb90c200700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:15.638228+0000 front 2024-04-26T22:00:15.637988+0000 (oldest deadline 2024-04-26T22:00:38.538212+0000) 2024-04-26T22:00:50.086 INFO:journalctl@ceph.osd.5.smithi164.stdout:Apr 26 22:00:49 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-5[43528]: 2024-04-26T22:00:49.744+0000 7fb90c200700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:23.141714+0000 front 2024-04-26T22:00:23.141785+0000 (oldest deadline 2024-04-26T22:00:49.041949+0000) 2024-04-26T22:00:50.440 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:00:50 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:00:50.085+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:14.343204+0000 front 2024-04-26T22:00:14.342815+0000 (oldest deadline 2024-04-26T22:00:39.042910+0000) 2024-04-26T22:00:50.440 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:00:50 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:00:50.085+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:24.945823+0000 front 2024-04-26T22:00:24.945339+0000 (oldest deadline 2024-04-26T22:00:47.245223+0000) 2024-04-26T22:00:50.564 INFO:journalctl@ceph.osd.7.smithi076.stdout:Apr 26 22:00:50 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-7[49518]: 2024-04-26T22:00:50.071+0000 7fcc04a00700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:21.667782+0000 front 2024-04-26T22:00:21.668606+0000 (oldest deadline 2024-04-26T22:00:46.967821+0000) 2024-04-26T22:00:50.571 INFO:journalctl@ceph.osd.6.smithi072.stdout:Apr 26 22:00:50 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-6[57091]: 2024-04-26T22:00:50.137+0000 7fb154800700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:16.543358+0000 front 2024-04-26T22:00:16.543710+0000 (oldest deadline 2024-04-26T22:00:41.744798+0000) 2024-04-26T22:00:50.571 INFO:journalctl@ceph.osd.6.smithi072.stdout:Apr 26 22:00:50 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-6[57091]: 2024-04-26T22:00:50.137+0000 7fb154800700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:21.746250+0000 front 2024-04-26T22:00:21.745942+0000 (oldest deadline 2024-04-26T22:00:47.645522+0000) 2024-04-26T22:00:51.064 INFO:journalctl@ceph.osd.4.smithi076.stdout:Apr 26 22:00:50 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-4[42901]: 2024-04-26T22:00:50.760+0000 7f8056000700 -1 osd.4 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:26.221859+0000 front 2024-04-26T22:00:26.222154+0000 (oldest deadline 2024-04-26T22:00:49.722084+0000) 2024-04-26T22:00:51.064 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:00:50 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:00:50.757+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:11.890629+0000 front 2024-04-26T22:00:11.890849+0000 (oldest deadline 2024-04-26T22:00:40.692043+0000) 2024-04-26T22:00:51.064 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:00:50 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:00:50.757+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:24.194239+0000 front 2024-04-26T22:00:24.194095+0000 (oldest deadline 2024-04-26T22:00:48.893996+0000) 2024-04-26T22:00:51.071 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:00:50 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:00:50.599+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:16.595921+0000 front 2024-04-26T22:00:16.596275+0000 (oldest deadline 2024-04-26T22:00:39.495420+0000) 2024-04-26T22:00:51.071 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:00:50 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:00:50.599+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:24.600976+0000 front 2024-04-26T22:00:24.600553+0000 (oldest deadline 2024-04-26T22:00:48.100712+0000) 2024-04-26T22:00:51.098 INFO:journalctl@ceph.osd.5.smithi164.stdout:Apr 26 22:00:50 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-5[43528]: 2024-04-26T22:00:50.730+0000 7fb90c200700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:15.638228+0000 front 2024-04-26T22:00:15.637988+0000 (oldest deadline 2024-04-26T22:00:38.538212+0000) 2024-04-26T22:00:51.098 INFO:journalctl@ceph.osd.5.smithi164.stdout:Apr 26 22:00:50 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-5[43528]: 2024-04-26T22:00:50.730+0000 7fb90c200700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:23.141714+0000 front 2024-04-26T22:00:23.141785+0000 (oldest deadline 2024-04-26T22:00:49.041949+0000) 2024-04-26T22:00:51.314 INFO:journalctl@ceph.osd.7.smithi076.stdout:Apr 26 22:00:51 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-7[49518]: 2024-04-26T22:00:51.063+0000 7fcc04a00700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:21.667782+0000 front 2024-04-26T22:00:21.668606+0000 (oldest deadline 2024-04-26T22:00:46.967821+0000) 2024-04-26T22:00:51.440 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:00:51 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:00:51.097+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:14.343204+0000 front 2024-04-26T22:00:14.342815+0000 (oldest deadline 2024-04-26T22:00:39.042910+0000) 2024-04-26T22:00:51.440 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:00:51 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:00:51.097+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:24.945823+0000 front 2024-04-26T22:00:24.945339+0000 (oldest deadline 2024-04-26T22:00:47.245223+0000) 2024-04-26T22:00:51.441 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 26 22:00:51 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-8[50342]: 2024-04-26T22:00:51.295+0000 7f152e400700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:25.173410+0000 front 2024-04-26T22:00:25.173841+0000 (oldest deadline 2024-04-26T22:00:50.473747+0000) 2024-04-26T22:00:51.559 INFO:journalctl@ceph.osd.6.smithi072.stdout:Apr 26 22:00:51 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-6[57091]: 2024-04-26T22:00:51.150+0000 7fb154800700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:16.543358+0000 front 2024-04-26T22:00:16.543710+0000 (oldest deadline 2024-04-26T22:00:41.744798+0000) 2024-04-26T22:00:51.560 INFO:journalctl@ceph.osd.6.smithi072.stdout:Apr 26 22:00:51 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-6[57091]: 2024-04-26T22:00:51.150+0000 7fb154800700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:21.746250+0000 front 2024-04-26T22:00:21.745942+0000 (oldest deadline 2024-04-26T22:00:47.645522+0000) 2024-04-26T22:00:51.821 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:00:51 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:00:51.559+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:16.595921+0000 front 2024-04-26T22:00:16.596275+0000 (oldest deadline 2024-04-26T22:00:39.495420+0000) 2024-04-26T22:00:51.822 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:00:51 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:00:51.559+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:24.600976+0000 front 2024-04-26T22:00:24.600553+0000 (oldest deadline 2024-04-26T22:00:48.100712+0000) 2024-04-26T22:00:52.039 INFO:journalctl@ceph.osd.4.smithi076.stdout:Apr 26 22:00:51 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-4[42901]: 2024-04-26T22:00:51.758+0000 7f8056000700 -1 osd.4 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:26.221859+0000 front 2024-04-26T22:00:26.222154+0000 (oldest deadline 2024-04-26T22:00:49.722084+0000) 2024-04-26T22:00:52.039 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:00:51 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:00:51.748+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:11.890629+0000 front 2024-04-26T22:00:11.890849+0000 (oldest deadline 2024-04-26T22:00:40.692043+0000) 2024-04-26T22:00:52.039 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:00:51 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:00:51.748+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:24.194239+0000 front 2024-04-26T22:00:24.194095+0000 (oldest deadline 2024-04-26T22:00:48.893996+0000) 2024-04-26T22:00:52.081 INFO:journalctl@ceph.osd.5.smithi164.stdout:Apr 26 22:00:51 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-5[43528]: 2024-04-26T22:00:51.711+0000 7fb90c200700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:15.638228+0000 front 2024-04-26T22:00:15.637988+0000 (oldest deadline 2024-04-26T22:00:38.538212+0000) 2024-04-26T22:00:52.081 INFO:journalctl@ceph.osd.5.smithi164.stdout:Apr 26 22:00:51 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-5[43528]: 2024-04-26T22:00:51.711+0000 7fb90c200700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:23.141714+0000 front 2024-04-26T22:00:23.141785+0000 (oldest deadline 2024-04-26T22:00:49.041949+0000) 2024-04-26T22:00:52.314 INFO:journalctl@ceph.osd.7.smithi076.stdout:Apr 26 22:00:52 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-7[49518]: 2024-04-26T22:00:52.038+0000 7fcc04a00700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:21.667782+0000 front 2024-04-26T22:00:21.668606+0000 (oldest deadline 2024-04-26T22:00:46.967821+0000) 2024-04-26T22:00:52.337 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:00:52 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:00:52.080+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:14.343204+0000 front 2024-04-26T22:00:14.342815+0000 (oldest deadline 2024-04-26T22:00:39.042910+0000) 2024-04-26T22:00:52.337 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:00:52 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:00:52.080+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:24.945823+0000 front 2024-04-26T22:00:24.945339+0000 (oldest deadline 2024-04-26T22:00:47.245223+0000) 2024-04-26T22:00:52.540 INFO:journalctl@ceph.osd.6.smithi072.stdout:Apr 26 22:00:52 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-6[57091]: 2024-04-26T22:00:52.121+0000 7fb154800700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:16.543358+0000 front 2024-04-26T22:00:16.543710+0000 (oldest deadline 2024-04-26T22:00:41.744798+0000) 2024-04-26T22:00:52.541 INFO:journalctl@ceph.osd.6.smithi072.stdout:Apr 26 22:00:52 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-6[57091]: 2024-04-26T22:00:52.121+0000 7fb154800700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:21.746250+0000 front 2024-04-26T22:00:21.745942+0000 (oldest deadline 2024-04-26T22:00:47.645522+0000) 2024-04-26T22:00:52.690 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 26 22:00:52 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-8[50342]: 2024-04-26T22:00:52.337+0000 7f152e400700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:25.173410+0000 front 2024-04-26T22:00:25.173841+0000 (oldest deadline 2024-04-26T22:00:50.473747+0000) 2024-04-26T22:00:52.821 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:00:52 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:00:52.540+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:16.595921+0000 front 2024-04-26T22:00:16.596275+0000 (oldest deadline 2024-04-26T22:00:39.495420+0000) 2024-04-26T22:00:52.822 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:00:52 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:00:52.540+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:24.600976+0000 front 2024-04-26T22:00:24.600553+0000 (oldest deadline 2024-04-26T22:00:48.100712+0000) 2024-04-26T22:00:53.064 INFO:journalctl@ceph.osd.4.smithi076.stdout:Apr 26 22:00:52 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-4[42901]: 2024-04-26T22:00:52.770+0000 7f8056000700 -1 osd.4 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:26.221859+0000 front 2024-04-26T22:00:26.222154+0000 (oldest deadline 2024-04-26T22:00:49.722084+0000) 2024-04-26T22:00:53.064 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:00:52 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:00:52.787+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:11.890629+0000 front 2024-04-26T22:00:11.890849+0000 (oldest deadline 2024-04-26T22:00:40.692043+0000) 2024-04-26T22:00:53.064 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:00:52 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:00:52.787+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:24.194239+0000 front 2024-04-26T22:00:24.194095+0000 (oldest deadline 2024-04-26T22:00:48.893996+0000) 2024-04-26T22:00:53.065 INFO:journalctl@ceph.osd.5.smithi164.stdout:Apr 26 22:00:52 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-5[43528]: 2024-04-26T22:00:52.727+0000 7fb90c200700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:15.638228+0000 front 2024-04-26T22:00:15.637988+0000 (oldest deadline 2024-04-26T22:00:38.538212+0000) 2024-04-26T22:00:53.065 INFO:journalctl@ceph.osd.5.smithi164.stdout:Apr 26 22:00:52 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-5[43528]: 2024-04-26T22:00:52.727+0000 7fb90c200700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:23.141714+0000 front 2024-04-26T22:00:23.141785+0000 (oldest deadline 2024-04-26T22:00:49.041949+0000) 2024-04-26T22:00:53.440 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 26 22:00:53 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-8[50342]: 2024-04-26T22:00:53.300+0000 7f152e400700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:25.173410+0000 front 2024-04-26T22:00:25.173841+0000 (oldest deadline 2024-04-26T22:00:50.473747+0000) 2024-04-26T22:00:53.441 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:00:53 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:00:53.065+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:14.343204+0000 front 2024-04-26T22:00:14.342815+0000 (oldest deadline 2024-04-26T22:00:39.042910+0000) 2024-04-26T22:00:53.441 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:00:53 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:00:53.065+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:24.945823+0000 front 2024-04-26T22:00:24.945339+0000 (oldest deadline 2024-04-26T22:00:47.245223+0000) 2024-04-26T22:00:53.564 INFO:journalctl@ceph.osd.7.smithi076.stdout:Apr 26 22:00:53 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-7[49518]: 2024-04-26T22:00:53.069+0000 7fcc04a00700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:21.667782+0000 front 2024-04-26T22:00:21.668606+0000 (oldest deadline 2024-04-26T22:00:46.967821+0000) 2024-04-26T22:00:53.567 INFO:journalctl@ceph.osd.6.smithi072.stdout:Apr 26 22:00:53 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-6[57091]: 2024-04-26T22:00:53.073+0000 7fb154800700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:16.543358+0000 front 2024-04-26T22:00:16.543710+0000 (oldest deadline 2024-04-26T22:00:41.744798+0000) 2024-04-26T22:00:53.567 INFO:journalctl@ceph.osd.6.smithi072.stdout:Apr 26 22:00:53 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-6[57091]: 2024-04-26T22:00:53.073+0000 7fb154800700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:21.746250+0000 front 2024-04-26T22:00:21.745942+0000 (oldest deadline 2024-04-26T22:00:47.645522+0000) 2024-04-26T22:00:53.821 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:00:53 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:00:53.567+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:16.595921+0000 front 2024-04-26T22:00:16.596275+0000 (oldest deadline 2024-04-26T22:00:39.495420+0000) 2024-04-26T22:00:53.822 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:00:53 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:00:53.567+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:24.600976+0000 front 2024-04-26T22:00:24.600553+0000 (oldest deadline 2024-04-26T22:00:48.100712+0000) 2024-04-26T22:00:54.064 INFO:journalctl@ceph.osd.4.smithi076.stdout:Apr 26 22:00:53 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-4[42901]: 2024-04-26T22:00:53.766+0000 7f8056000700 -1 osd.4 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:26.221859+0000 front 2024-04-26T22:00:26.222154+0000 (oldest deadline 2024-04-26T22:00:49.722084+0000) 2024-04-26T22:00:54.064 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:00:53 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:00:53.773+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:11.890629+0000 front 2024-04-26T22:00:11.890849+0000 (oldest deadline 2024-04-26T22:00:40.692043+0000) 2024-04-26T22:00:54.065 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:00:53 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:00:53.773+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:24.194239+0000 front 2024-04-26T22:00:24.194095+0000 (oldest deadline 2024-04-26T22:00:48.893996+0000) 2024-04-26T22:00:54.072 INFO:journalctl@ceph.osd.5.smithi164.stdout:Apr 26 22:00:53 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-5[43528]: 2024-04-26T22:00:53.693+0000 7fb90c200700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:15.638228+0000 front 2024-04-26T22:00:15.637988+0000 (oldest deadline 2024-04-26T22:00:38.538212+0000) 2024-04-26T22:00:54.073 INFO:journalctl@ceph.osd.5.smithi164.stdout:Apr 26 22:00:53 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-5[43528]: 2024-04-26T22:00:53.693+0000 7fb90c200700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:23.141714+0000 front 2024-04-26T22:00:23.141785+0000 (oldest deadline 2024-04-26T22:00:49.041949+0000) 2024-04-26T22:00:54.440 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 26 22:00:54 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-8[50342]: 2024-04-26T22:00:54.253+0000 7f152e400700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:25.173410+0000 front 2024-04-26T22:00:25.173841+0000 (oldest deadline 2024-04-26T22:00:50.473747+0000) 2024-04-26T22:00:54.440 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:00:54 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:00:54.072+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:14.343204+0000 front 2024-04-26T22:00:14.342815+0000 (oldest deadline 2024-04-26T22:00:39.042910+0000) 2024-04-26T22:00:54.441 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:00:54 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:00:54.072+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:24.945823+0000 front 2024-04-26T22:00:24.945339+0000 (oldest deadline 2024-04-26T22:00:47.245223+0000) 2024-04-26T22:00:54.560 INFO:journalctl@ceph.osd.6.smithi072.stdout:Apr 26 22:00:54 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-6[57091]: 2024-04-26T22:00:54.093+0000 7fb154800700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:16.543358+0000 front 2024-04-26T22:00:16.543710+0000 (oldest deadline 2024-04-26T22:00:41.744798+0000) 2024-04-26T22:00:54.560 INFO:journalctl@ceph.osd.6.smithi072.stdout:Apr 26 22:00:54 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-6[57091]: 2024-04-26T22:00:54.093+0000 7fb154800700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:21.746250+0000 front 2024-04-26T22:00:21.745942+0000 (oldest deadline 2024-04-26T22:00:47.645522+0000) 2024-04-26T22:00:54.564 INFO:journalctl@ceph.osd.7.smithi076.stdout:Apr 26 22:00:54 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-7[49518]: 2024-04-26T22:00:54.111+0000 7fcc04a00700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:21.667782+0000 front 2024-04-26T22:00:21.668606+0000 (oldest deadline 2024-04-26T22:00:46.967821+0000) 2024-04-26T22:00:54.821 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:00:54 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:00:54.560+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:16.595921+0000 front 2024-04-26T22:00:16.596275+0000 (oldest deadline 2024-04-26T22:00:39.495420+0000) 2024-04-26T22:00:54.822 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:00:54 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:00:54.560+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:24.600976+0000 front 2024-04-26T22:00:24.600553+0000 (oldest deadline 2024-04-26T22:00:48.100712+0000) 2024-04-26T22:00:55.064 INFO:journalctl@ceph.osd.4.smithi076.stdout:Apr 26 22:00:54 smithi076 podman[64946]: 2024-04-26 22:00:54.802420873 +0000 UTC m=+5.718246701 container died 690a9f400b41c3947a6b5d8c111a5b1fc7345c634aeaf0779c47337774d9f826 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-4, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.build-date=20240424) 2024-04-26T22:00:55.065 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:00:54 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:00:54.786+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:11.890629+0000 front 2024-04-26T22:00:11.890849+0000 (oldest deadline 2024-04-26T22:00:40.692043+0000) 2024-04-26T22:00:55.065 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:00:54 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:00:54.786+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:24.194239+0000 front 2024-04-26T22:00:24.194095+0000 (oldest deadline 2024-04-26T22:00:48.893996+0000) 2024-04-26T22:00:55.081 INFO:journalctl@ceph.osd.5.smithi164.stdout:Apr 26 22:00:54 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-5[43528]: 2024-04-26T22:00:54.713+0000 7fb90c200700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:15.638228+0000 front 2024-04-26T22:00:15.637988+0000 (oldest deadline 2024-04-26T22:00:38.538212+0000) 2024-04-26T22:00:55.081 INFO:journalctl@ceph.osd.5.smithi164.stdout:Apr 26 22:00:54 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-5[43528]: 2024-04-26T22:00:54.713+0000 7fb90c200700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:23.141714+0000 front 2024-04-26T22:00:23.141785+0000 (oldest deadline 2024-04-26T22:00:49.041949+0000) 2024-04-26T22:00:55.396 INFO:journalctl@ceph.osd.7.smithi076.stdout:Apr 26 22:00:55 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-7[49518]: 2024-04-26T22:00:55.147+0000 7fcc04a00700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:21.667782+0000 front 2024-04-26T22:00:21.668606+0000 (oldest deadline 2024-04-26T22:00:46.967821+0000) 2024-04-26T22:00:55.396 INFO:journalctl@ceph.osd.4.smithi076.stdout:Apr 26 22:00:55 smithi076 podman[64946]: 2024-04-26 22:00:55.211544669 +0000 UTC m=+6.127370557 container cleanup 690a9f400b41c3947a6b5d8c111a5b1fc7345c634aeaf0779c47337774d9f826 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-4, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux ) 2024-04-26T22:00:55.396 INFO:journalctl@ceph.osd.4.smithi076.stdout:Apr 26 22:00:55 smithi076 bash[64946]: ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-4 2024-04-26T22:00:55.440 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 26 22:00:55 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-8[50342]: 2024-04-26T22:00:55.218+0000 7f152e400700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:25.173410+0000 front 2024-04-26T22:00:25.173841+0000 (oldest deadline 2024-04-26T22:00:50.473747+0000) 2024-04-26T22:00:55.440 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:00:55 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:00:55.081+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:14.343204+0000 front 2024-04-26T22:00:14.342815+0000 (oldest deadline 2024-04-26T22:00:39.042910+0000) 2024-04-26T22:00:55.440 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:00:55 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:00:55.081+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:24.945823+0000 front 2024-04-26T22:00:24.945339+0000 (oldest deadline 2024-04-26T22:00:47.245223+0000) 2024-04-26T22:00:55.567 INFO:journalctl@ceph.osd.6.smithi072.stdout:Apr 26 22:00:55 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-6[57091]: 2024-04-26T22:00:55.119+0000 7fb154800700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:16.543358+0000 front 2024-04-26T22:00:16.543710+0000 (oldest deadline 2024-04-26T22:00:41.744798+0000) 2024-04-26T22:00:55.567 INFO:journalctl@ceph.osd.6.smithi072.stdout:Apr 26 22:00:55 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-6[57091]: 2024-04-26T22:00:55.119+0000 7fb154800700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:21.746250+0000 front 2024-04-26T22:00:21.745942+0000 (oldest deadline 2024-04-26T22:00:47.645522+0000) 2024-04-26T22:00:55.761 INFO:journalctl@ceph.osd.4.smithi076.stdout:Apr 26 22:00:55 smithi076 podman[64965]: 2024-04-26 22:00:55.395822742 +0000 UTC m=+0.586123745 container remove 690a9f400b41c3947a6b5d8c111a5b1fc7345c634aeaf0779c47337774d9f826 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-4, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux ) 2024-04-26T22:00:55.822 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:00:55 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:00:55.566+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:16.595921+0000 front 2024-04-26T22:00:16.596275+0000 (oldest deadline 2024-04-26T22:00:39.495420+0000) 2024-04-26T22:00:55.822 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:00:55 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:00:55.566+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:24.600976+0000 front 2024-04-26T22:00:24.600553+0000 (oldest deadline 2024-04-26T22:00:48.100712+0000) 2024-04-26T22:00:56.064 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:00:55 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:00:55.761+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:11.890629+0000 front 2024-04-26T22:00:11.890849+0000 (oldest deadline 2024-04-26T22:00:40.692043+0000) 2024-04-26T22:00:56.064 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:00:55 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:00:55.761+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:24.194239+0000 front 2024-04-26T22:00:24.194095+0000 (oldest deadline 2024-04-26T22:00:48.893996+0000) 2024-04-26T22:00:56.131 INFO:journalctl@ceph.osd.5.smithi164.stdout:Apr 26 22:00:55 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-5[43528]: 2024-04-26T22:00:55.703+0000 7fb90c200700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:15.638228+0000 front 2024-04-26T22:00:15.637988+0000 (oldest deadline 2024-04-26T22:00:38.538212+0000) 2024-04-26T22:00:56.131 INFO:journalctl@ceph.osd.5.smithi164.stdout:Apr 26 22:00:55 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-5[43528]: 2024-04-26T22:00:55.703+0000 7fb90c200700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:23.141714+0000 front 2024-04-26T22:00:23.141785+0000 (oldest deadline 2024-04-26T22:00:49.041949+0000) 2024-04-26T22:00:56.355 INFO:journalctl@ceph.osd.4.smithi076.stdout:Apr 26 22:00:56 smithi076 podman[65052]: 2024-04-26 22:00:56.066864799 +0000 UTC m=+0.034384231 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-26T22:00:56.355 INFO:journalctl@ceph.osd.4.smithi076.stdout:Apr 26 22:00:56 smithi076 podman[65052]: 2024-04-26 22:00:56.188304161 +0000 UTC m=+0.155823630 container create ae1b95d2d2b31bc0b2cc6a71af53d2d55480e0d3481fd11363429b0202500814 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-4-deactivate, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-04-26T22:00:56.355 INFO:journalctl@ceph.osd.7.smithi076.stdout:Apr 26 22:00:56 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-7[49518]: 2024-04-26T22:00:56.102+0000 7fcc04a00700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:21.667782+0000 front 2024-04-26T22:00:21.668606+0000 (oldest deadline 2024-04-26T22:00:46.967821+0000) 2024-04-26T22:00:56.440 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 26 22:00:56 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-8[50342]: 2024-04-26T22:00:56.252+0000 7f152e400700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:25.173410+0000 front 2024-04-26T22:00:25.173841+0000 (oldest deadline 2024-04-26T22:00:50.473747+0000) 2024-04-26T22:00:56.440 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:00:56 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:00:56.131+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:14.343204+0000 front 2024-04-26T22:00:14.342815+0000 (oldest deadline 2024-04-26T22:00:39.042910+0000) 2024-04-26T22:00:56.440 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:00:56 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:00:56.131+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:24.945823+0000 front 2024-04-26T22:00:24.945339+0000 (oldest deadline 2024-04-26T22:00:47.245223+0000) 2024-04-26T22:00:56.571 INFO:journalctl@ceph.osd.6.smithi072.stdout:Apr 26 22:00:56 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-6[57091]: 2024-04-26T22:00:56.081+0000 7fb154800700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:16.543358+0000 front 2024-04-26T22:00:16.543710+0000 (oldest deadline 2024-04-26T22:00:41.744798+0000) 2024-04-26T22:00:56.572 INFO:journalctl@ceph.osd.6.smithi072.stdout:Apr 26 22:00:56 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-6[57091]: 2024-04-26T22:00:56.081+0000 7fb154800700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:21.746250+0000 front 2024-04-26T22:00:21.745942+0000 (oldest deadline 2024-04-26T22:00:47.645522+0000) 2024-04-26T22:00:56.724 INFO:journalctl@ceph.osd.4.smithi076.stdout:Apr 26 22:00:56 smithi076 podman[65052]: 2024-04-26 22:00:56.50000882 +0000 UTC m=+0.467528295 container init ae1b95d2d2b31bc0b2cc6a71af53d2d55480e0d3481fd11363429b0202500814 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-4-deactivate, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux ) 2024-04-26T22:00:56.724 INFO:journalctl@ceph.osd.4.smithi076.stdout:Apr 26 22:00:56 smithi076 podman[65052]: 2024-04-26 22:00:56.511897936 +0000 UTC m=+0.479417411 container start ae1b95d2d2b31bc0b2cc6a71af53d2d55480e0d3481fd11363429b0202500814 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-4-deactivate, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-04-26T22:00:56.724 INFO:journalctl@ceph.osd.4.smithi076.stdout:Apr 26 22:00:56 smithi076 podman[65052]: 2024-04-26 22:00:56.590398909 +0000 UTC m=+0.557918418 container attach ae1b95d2d2b31bc0b2cc6a71af53d2d55480e0d3481fd11363429b0202500814 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-4-deactivate, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T22:00:56.940 INFO:journalctl@ceph.osd.5.smithi164.stdout:Apr 26 22:00:56 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-5[43528]: 2024-04-26T22:00:56.688+0000 7fb90c200700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:15.638228+0000 front 2024-04-26T22:00:15.637988+0000 (oldest deadline 2024-04-26T22:00:38.538212+0000) 2024-04-26T22:00:56.940 INFO:journalctl@ceph.osd.5.smithi164.stdout:Apr 26 22:00:56 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-5[43528]: 2024-04-26T22:00:56.688+0000 7fb90c200700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:23.141714+0000 front 2024-04-26T22:00:23.141785+0000 (oldest deadline 2024-04-26T22:00:49.041949+0000) 2024-04-26T22:00:57.064 INFO:journalctl@ceph.osd.4.smithi076.stdout:Apr 26 22:00:56 smithi076 podman[65052]: 2024-04-26 22:00:56.971204224 +0000 UTC m=+0.938723699 container died ae1b95d2d2b31bc0b2cc6a71af53d2d55480e0d3481fd11363429b0202500814 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-4-deactivate, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T22:00:57.065 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:00:56 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:00:56.723+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:11.890629+0000 front 2024-04-26T22:00:11.890849+0000 (oldest deadline 2024-04-26T22:00:40.692043+0000) 2024-04-26T22:00:57.065 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:00:56 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:00:56.723+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:24.194239+0000 front 2024-04-26T22:00:24.194095+0000 (oldest deadline 2024-04-26T22:00:48.893996+0000) 2024-04-26T22:00:57.065 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:00:56 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:00:56.723+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:34.096788+0000 front 2024-04-26T22:00:34.096884+0000 (oldest deadline 2024-04-26T22:00:56.396862+0000) 2024-04-26T22:00:57.071 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:00:56 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:00:56.602+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:16.595921+0000 front 2024-04-26T22:00:16.596275+0000 (oldest deadline 2024-04-26T22:00:39.495420+0000) 2024-04-26T22:00:57.071 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:00:56 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:00:56.602+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:24.600976+0000 front 2024-04-26T22:00:24.600553+0000 (oldest deadline 2024-04-26T22:00:48.100712+0000) 2024-04-26T22:00:57.335 INFO:journalctl@ceph.osd.7.smithi076.stdout:Apr 26 22:00:57 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-7[49518]: 2024-04-26T22:00:57.079+0000 7fcc04a00700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:21.667782+0000 front 2024-04-26T22:00:21.668606+0000 (oldest deadline 2024-04-26T22:00:46.967821+0000) 2024-04-26T22:00:57.440 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 26 22:00:57 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-8[50342]: 2024-04-26T22:00:57.273+0000 7f152e400700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:25.173410+0000 front 2024-04-26T22:00:25.173841+0000 (oldest deadline 2024-04-26T22:00:50.473747+0000) 2024-04-26T22:00:57.440 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 26 22:00:57 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-8[50342]: 2024-04-26T22:00:57.273+0000 7f152e400700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:30.474905+0000 front 2024-04-26T22:00:30.475573+0000 (oldest deadline 2024-04-26T22:00:56.375141+0000) 2024-04-26T22:00:57.441 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:00:57 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:00:57.110+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:14.343204+0000 front 2024-04-26T22:00:14.342815+0000 (oldest deadline 2024-04-26T22:00:39.042910+0000) 2024-04-26T22:00:57.441 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:00:57 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:00:57.110+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:24.945823+0000 front 2024-04-26T22:00:24.945339+0000 (oldest deadline 2024-04-26T22:00:47.245223+0000) 2024-04-26T22:00:57.441 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:00:57 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:00:57.110+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:31.947744+0000 front 2024-04-26T22:00:31.948080+0000 (oldest deadline 2024-04-26T22:00:56.647838+0000) 2024-04-26T22:00:57.559 INFO:journalctl@ceph.osd.6.smithi072.stdout:Apr 26 22:00:57 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-6[57091]: 2024-04-26T22:00:57.117+0000 7fb154800700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:16.543358+0000 front 2024-04-26T22:00:16.543710+0000 (oldest deadline 2024-04-26T22:00:41.744798+0000) 2024-04-26T22:00:57.559 INFO:journalctl@ceph.osd.6.smithi072.stdout:Apr 26 22:00:57 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-6[57091]: 2024-04-26T22:00:57.117+0000 7fb154800700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:21.746250+0000 front 2024-04-26T22:00:21.745942+0000 (oldest deadline 2024-04-26T22:00:47.645522+0000) 2024-04-26T22:00:57.559 INFO:journalctl@ceph.osd.6.smithi072.stdout:Apr 26 22:00:57 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-6[57091]: 2024-04-26T22:00:57.117+0000 7fb154800700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:31.649534+0000 front 2024-04-26T22:00:31.649342+0000 (oldest deadline 2024-04-26T22:00:56.949491+0000) 2024-04-26T22:00:57.711 DEBUG:teuthology.orchestra.run.smithi076:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a1780950-0415-11ef-bc93-c7b262605968@osd.4.service' 2024-04-26T22:00:57.816 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T22:00:57.816 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-04-26T22:00:57.816 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-04-26T22:00:57.817 DEBUG:teuthology.orchestra.run.smithi164:> sudo systemctl stop ceph-a1780950-0415-11ef-bc93-c7b262605968@osd.5 2024-04-26T22:00:57.821 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:00:57 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:00:57.559+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:16.595921+0000 front 2024-04-26T22:00:16.596275+0000 (oldest deadline 2024-04-26T22:00:39.495420+0000) 2024-04-26T22:00:57.821 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:00:57 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:00:57.559+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:24.600976+0000 front 2024-04-26T22:00:24.600553+0000 (oldest deadline 2024-04-26T22:00:48.100712+0000) 2024-04-26T22:00:57.932 INFO:journalctl@ceph.osd.5.smithi164.stdout:Apr 26 22:00:57 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-5[43528]: 2024-04-26T22:00:57.674+0000 7fb90c200700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:15.638228+0000 front 2024-04-26T22:00:15.637988+0000 (oldest deadline 2024-04-26T22:00:38.538212+0000) 2024-04-26T22:00:57.932 INFO:journalctl@ceph.osd.5.smithi164.stdout:Apr 26 22:00:57 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-5[43528]: 2024-04-26T22:00:57.674+0000 7fb90c200700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:23.141714+0000 front 2024-04-26T22:00:23.141785+0000 (oldest deadline 2024-04-26T22:00:49.041949+0000) 2024-04-26T22:00:58.064 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:00:57 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:00:57.714+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:11.890629+0000 front 2024-04-26T22:00:11.890849+0000 (oldest deadline 2024-04-26T22:00:40.692043+0000) 2024-04-26T22:00:58.065 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:00:57 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:00:57.714+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:24.194239+0000 front 2024-04-26T22:00:24.194095+0000 (oldest deadline 2024-04-26T22:00:48.893996+0000) 2024-04-26T22:00:58.065 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:00:57 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:00:57.714+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:34.096788+0000 front 2024-04-26T22:00:34.096884+0000 (oldest deadline 2024-04-26T22:00:56.396862+0000) 2024-04-26T22:00:58.190 INFO:journalctl@ceph.osd.5.smithi164.stdout:Apr 26 22:00:57 smithi164 systemd[1]: Stopping Ceph osd.5 for a1780950-0415-11ef-bc93-c7b262605968... 2024-04-26T22:00:58.191 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:00:58 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:00:58.108+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:14.343204+0000 front 2024-04-26T22:00:14.342815+0000 (oldest deadline 2024-04-26T22:00:39.042910+0000) 2024-04-26T22:00:58.191 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:00:58 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:00:58.108+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:24.945823+0000 front 2024-04-26T22:00:24.945339+0000 (oldest deadline 2024-04-26T22:00:47.245223+0000) 2024-04-26T22:00:58.191 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:00:58 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:00:58.108+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:31.947744+0000 front 2024-04-26T22:00:31.948080+0000 (oldest deadline 2024-04-26T22:00:56.647838+0000) 2024-04-26T22:00:58.564 INFO:journalctl@ceph.osd.7.smithi076.stdout:Apr 26 22:00:58 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-7[49518]: 2024-04-26T22:00:58.075+0000 7fcc04a00700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:21.667782+0000 front 2024-04-26T22:00:21.668606+0000 (oldest deadline 2024-04-26T22:00:46.967821+0000) 2024-04-26T22:00:58.564 INFO:journalctl@ceph.osd.7.smithi076.stdout:Apr 26 22:00:58 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-7[49518]: 2024-04-26T22:00:58.075+0000 7fcc04a00700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:34.571664+0000 front 2024-04-26T22:00:34.572066+0000 (oldest deadline 2024-04-26T22:00:57.471976+0000) 2024-04-26T22:00:58.572 INFO:journalctl@ceph.osd.6.smithi072.stdout:Apr 26 22:00:58 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-6[57091]: 2024-04-26T22:00:58.156+0000 7fb154800700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:16.543358+0000 front 2024-04-26T22:00:16.543710+0000 (oldest deadline 2024-04-26T22:00:41.744798+0000) 2024-04-26T22:00:58.572 INFO:journalctl@ceph.osd.6.smithi072.stdout:Apr 26 22:00:58 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-6[57091]: 2024-04-26T22:00:58.156+0000 7fb154800700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:21.746250+0000 front 2024-04-26T22:00:21.745942+0000 (oldest deadline 2024-04-26T22:00:47.645522+0000) 2024-04-26T22:00:58.572 INFO:journalctl@ceph.osd.6.smithi072.stdout:Apr 26 22:00:58 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-6[57091]: 2024-04-26T22:00:58.156+0000 7fb154800700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:31.649534+0000 front 2024-04-26T22:00:31.649342+0000 (oldest deadline 2024-04-26T22:00:56.949491+0000) 2024-04-26T22:00:58.671 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 26 22:00:58 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-8[50342]: 2024-04-26T22:00:58.290+0000 7f152e400700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:25.173410+0000 front 2024-04-26T22:00:25.173841+0000 (oldest deadline 2024-04-26T22:00:50.473747+0000) 2024-04-26T22:00:58.671 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 26 22:00:58 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-8[50342]: 2024-04-26T22:00:58.290+0000 7f152e400700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:30.474905+0000 front 2024-04-26T22:00:30.475573+0000 (oldest deadline 2024-04-26T22:00:56.375141+0000) 2024-04-26T22:00:58.671 INFO:journalctl@ceph.osd.5.smithi164.stdout:Apr 26 22:00:58 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-5[43528]: 2024-04-26T22:00:58.409+0000 7fb915a00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T22:00:58.671 INFO:journalctl@ceph.osd.5.smithi164.stdout:Apr 26 22:00:58 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-5[43528]: 2024-04-26T22:00:58.409+0000 7fb915a00700 -1 osd.5 94 *** Got signal Terminated *** 2024-04-26T22:00:58.671 INFO:journalctl@ceph.osd.5.smithi164.stdout:Apr 26 22:00:58 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-5[43528]: 2024-04-26T22:00:58.409+0000 7fb915a00700 -1 osd.5 94 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-26T22:00:58.671 INFO:journalctl@ceph.osd.5.smithi164.stdout:Apr 26 22:00:58 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-5[43528]: 2024-04-26T22:00:58.670+0000 7fb90c200700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:15.638228+0000 front 2024-04-26T22:00:15.637988+0000 (oldest deadline 2024-04-26T22:00:38.538212+0000) 2024-04-26T22:00:58.672 INFO:journalctl@ceph.osd.5.smithi164.stdout:Apr 26 22:00:58 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-5[43528]: 2024-04-26T22:00:58.670+0000 7fb90c200700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:23.141714+0000 front 2024-04-26T22:00:23.141785+0000 (oldest deadline 2024-04-26T22:00:49.041949+0000) 2024-04-26T22:00:59.064 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:00:58 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:00:58.749+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:11.890629+0000 front 2024-04-26T22:00:11.890849+0000 (oldest deadline 2024-04-26T22:00:40.692043+0000) 2024-04-26T22:00:59.065 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:00:58 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:00:58.749+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:24.194239+0000 front 2024-04-26T22:00:24.194095+0000 (oldest deadline 2024-04-26T22:00:48.893996+0000) 2024-04-26T22:00:59.065 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:00:58 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:00:58.749+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:34.096788+0000 front 2024-04-26T22:00:34.096884+0000 (oldest deadline 2024-04-26T22:00:56.396862+0000) 2024-04-26T22:00:59.071 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:00:58 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:00:58.598+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:16.595921+0000 front 2024-04-26T22:00:16.596275+0000 (oldest deadline 2024-04-26T22:00:39.495420+0000) 2024-04-26T22:00:59.072 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:00:58 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:00:58.598+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:24.600976+0000 front 2024-04-26T22:00:24.600553+0000 (oldest deadline 2024-04-26T22:00:48.100712+0000) 2024-04-26T22:00:59.072 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:00:58 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:00:58.598+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:34.002793+0000 front 2024-04-26T22:00:34.003098+0000 (oldest deadline 2024-04-26T22:00:58.103086+0000) 2024-04-26T22:00:59.440 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 26 22:00:59 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-8[50342]: 2024-04-26T22:00:59.307+0000 7f152e400700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:25.173410+0000 front 2024-04-26T22:00:25.173841+0000 (oldest deadline 2024-04-26T22:00:50.473747+0000) 2024-04-26T22:00:59.440 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 26 22:00:59 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-8[50342]: 2024-04-26T22:00:59.307+0000 7f152e400700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:30.474905+0000 front 2024-04-26T22:00:30.475573+0000 (oldest deadline 2024-04-26T22:00:56.375141+0000) 2024-04-26T22:00:59.441 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:00:59 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:00:59.152+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:14.343204+0000 front 2024-04-26T22:00:14.342815+0000 (oldest deadline 2024-04-26T22:00:39.042910+0000) 2024-04-26T22:00:59.441 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:00:59 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:00:59.152+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:24.945823+0000 front 2024-04-26T22:00:24.945339+0000 (oldest deadline 2024-04-26T22:00:47.245223+0000) 2024-04-26T22:00:59.441 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:00:59 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:00:59.152+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:31.947744+0000 front 2024-04-26T22:00:31.948080+0000 (oldest deadline 2024-04-26T22:00:56.647838+0000) 2024-04-26T22:00:59.553 INFO:journalctl@ceph.osd.6.smithi072.stdout:Apr 26 22:00:59 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-6[57091]: 2024-04-26T22:00:59.165+0000 7fb154800700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:16.543358+0000 front 2024-04-26T22:00:16.543710+0000 (oldest deadline 2024-04-26T22:00:41.744798+0000) 2024-04-26T22:00:59.553 INFO:journalctl@ceph.osd.6.smithi072.stdout:Apr 26 22:00:59 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-6[57091]: 2024-04-26T22:00:59.165+0000 7fb154800700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:21.746250+0000 front 2024-04-26T22:00:21.745942+0000 (oldest deadline 2024-04-26T22:00:47.645522+0000) 2024-04-26T22:00:59.553 INFO:journalctl@ceph.osd.6.smithi072.stdout:Apr 26 22:00:59 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-6[57091]: 2024-04-26T22:00:59.165+0000 7fb154800700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:31.649534+0000 front 2024-04-26T22:00:31.649342+0000 (oldest deadline 2024-04-26T22:00:56.949491+0000) 2024-04-26T22:00:59.564 INFO:journalctl@ceph.osd.7.smithi076.stdout:Apr 26 22:00:59 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-7[49518]: 2024-04-26T22:00:59.069+0000 7fcc04a00700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:21.667782+0000 front 2024-04-26T22:00:21.668606+0000 (oldest deadline 2024-04-26T22:00:46.967821+0000) 2024-04-26T22:00:59.564 INFO:journalctl@ceph.osd.7.smithi076.stdout:Apr 26 22:00:59 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-7[49518]: 2024-04-26T22:00:59.069+0000 7fcc04a00700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:34.571664+0000 front 2024-04-26T22:00:34.572066+0000 (oldest deadline 2024-04-26T22:00:57.471976+0000) 2024-04-26T22:00:59.821 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:00:59 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:00:59.552+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:16.595921+0000 front 2024-04-26T22:00:16.596275+0000 (oldest deadline 2024-04-26T22:00:39.495420+0000) 2024-04-26T22:00:59.822 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:00:59 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:00:59.552+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:24.600976+0000 front 2024-04-26T22:00:24.600553+0000 (oldest deadline 2024-04-26T22:00:48.100712+0000) 2024-04-26T22:00:59.822 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:00:59 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:00:59.552+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:34.002793+0000 front 2024-04-26T22:00:34.003098+0000 (oldest deadline 2024-04-26T22:00:58.103086+0000) 2024-04-26T22:01:00.051 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:00:59 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:00:59.754+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:11.890629+0000 front 2024-04-26T22:00:11.890849+0000 (oldest deadline 2024-04-26T22:00:40.692043+0000) 2024-04-26T22:01:00.052 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:00:59 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:00:59.754+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:24.194239+0000 front 2024-04-26T22:00:24.194095+0000 (oldest deadline 2024-04-26T22:00:48.893996+0000) 2024-04-26T22:01:00.052 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:00:59 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:00:59.754+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:34.096788+0000 front 2024-04-26T22:00:34.096884+0000 (oldest deadline 2024-04-26T22:00:56.396862+0000) 2024-04-26T22:01:00.190 INFO:journalctl@ceph.osd.5.smithi164.stdout:Apr 26 22:00:59 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-5[43528]: 2024-04-26T22:00:59.716+0000 7fb90c200700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:15.638228+0000 front 2024-04-26T22:00:15.637988+0000 (oldest deadline 2024-04-26T22:00:38.538212+0000) 2024-04-26T22:01:00.190 INFO:journalctl@ceph.osd.5.smithi164.stdout:Apr 26 22:00:59 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-5[43528]: 2024-04-26T22:00:59.716+0000 7fb90c200700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:23.141714+0000 front 2024-04-26T22:00:23.141785+0000 (oldest deadline 2024-04-26T22:00:49.041949+0000) 2024-04-26T22:01:00.314 INFO:journalctl@ceph.osd.7.smithi076.stdout:Apr 26 22:01:00 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-7[49518]: 2024-04-26T22:01:00.051+0000 7fcc04a00700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:21.667782+0000 front 2024-04-26T22:00:21.668606+0000 (oldest deadline 2024-04-26T22:00:46.967821+0000) 2024-04-26T22:01:00.314 INFO:journalctl@ceph.osd.7.smithi076.stdout:Apr 26 22:01:00 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-7[49518]: 2024-04-26T22:01:00.051+0000 7fcc04a00700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:34.571664+0000 front 2024-04-26T22:00:34.572066+0000 (oldest deadline 2024-04-26T22:00:57.471976+0000) 2024-04-26T22:01:00.572 INFO:journalctl@ceph.osd.6.smithi072.stdout:Apr 26 22:01:00 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-6[57091]: 2024-04-26T22:01:00.130+0000 7fb154800700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:16.543358+0000 front 2024-04-26T22:00:16.543710+0000 (oldest deadline 2024-04-26T22:00:41.744798+0000) 2024-04-26T22:01:00.572 INFO:journalctl@ceph.osd.6.smithi072.stdout:Apr 26 22:01:00 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-6[57091]: 2024-04-26T22:01:00.130+0000 7fb154800700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:21.746250+0000 front 2024-04-26T22:00:21.745942+0000 (oldest deadline 2024-04-26T22:00:47.645522+0000) 2024-04-26T22:01:00.572 INFO:journalctl@ceph.osd.6.smithi072.stdout:Apr 26 22:01:00 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-6[57091]: 2024-04-26T22:01:00.130+0000 7fb154800700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:31.649534+0000 front 2024-04-26T22:00:31.649342+0000 (oldest deadline 2024-04-26T22:00:56.949491+0000) 2024-04-26T22:01:00.690 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 26 22:01:00 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-8[50342]: 2024-04-26T22:01:00.271+0000 7f152e400700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:25.173410+0000 front 2024-04-26T22:00:25.173841+0000 (oldest deadline 2024-04-26T22:00:50.473747+0000) 2024-04-26T22:01:00.691 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 26 22:01:00 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-8[50342]: 2024-04-26T22:01:00.271+0000 7f152e400700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:30.474905+0000 front 2024-04-26T22:00:30.475573+0000 (oldest deadline 2024-04-26T22:00:56.375141+0000) 2024-04-26T22:01:00.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:00 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:00.197+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:14.343204+0000 front 2024-04-26T22:00:14.342815+0000 (oldest deadline 2024-04-26T22:00:39.042910+0000) 2024-04-26T22:01:00.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:00 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:00.197+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:24.945823+0000 front 2024-04-26T22:00:24.945339+0000 (oldest deadline 2024-04-26T22:00:47.245223+0000) 2024-04-26T22:01:00.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:00 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:00.197+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:31.947744+0000 front 2024-04-26T22:00:31.948080+0000 (oldest deadline 2024-04-26T22:00:56.647838+0000) 2024-04-26T22:01:01.026 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:00 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:00.764+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:11.890629+0000 front 2024-04-26T22:00:11.890849+0000 (oldest deadline 2024-04-26T22:00:40.692043+0000) 2024-04-26T22:01:01.026 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:00 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:00.764+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:24.194239+0000 front 2024-04-26T22:00:24.194095+0000 (oldest deadline 2024-04-26T22:00:48.893996+0000) 2024-04-26T22:01:01.026 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:00 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:00.764+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:34.096788+0000 front 2024-04-26T22:00:34.096884+0000 (oldest deadline 2024-04-26T22:00:56.396862+0000) 2024-04-26T22:01:01.072 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:01:00 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:01:00.581+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:16.595921+0000 front 2024-04-26T22:00:16.596275+0000 (oldest deadline 2024-04-26T22:00:39.495420+0000) 2024-04-26T22:01:01.072 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:01:00 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:01:00.581+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:24.600976+0000 front 2024-04-26T22:00:24.600553+0000 (oldest deadline 2024-04-26T22:00:48.100712+0000) 2024-04-26T22:01:01.072 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:01:00 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:01:00.581+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:34.002793+0000 front 2024-04-26T22:00:34.003098+0000 (oldest deadline 2024-04-26T22:00:58.103086+0000) 2024-04-26T22:01:01.190 INFO:journalctl@ceph.osd.5.smithi164.stdout:Apr 26 22:01:00 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-5[43528]: 2024-04-26T22:01:00.717+0000 7fb90c200700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:15.638228+0000 front 2024-04-26T22:00:15.637988+0000 (oldest deadline 2024-04-26T22:00:38.538212+0000) 2024-04-26T22:01:01.190 INFO:journalctl@ceph.osd.5.smithi164.stdout:Apr 26 22:01:00 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-5[43528]: 2024-04-26T22:01:00.717+0000 7fb90c200700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:23.141714+0000 front 2024-04-26T22:00:23.141785+0000 (oldest deadline 2024-04-26T22:00:49.041949+0000) 2024-04-26T22:01:01.190 INFO:journalctl@ceph.osd.5.smithi164.stdout:Apr 26 22:01:00 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-5[43528]: 2024-04-26T22:01:00.717+0000 7fb90c200700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:35.247685+0000 front 2024-04-26T22:00:35.247753+0000 (oldest deadline 2024-04-26T22:01:00.547904+0000) 2024-04-26T22:01:01.314 INFO:journalctl@ceph.osd.7.smithi076.stdout:Apr 26 22:01:01 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-7[49518]: 2024-04-26T22:01:01.026+0000 7fcc04a00700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:21.667782+0000 front 2024-04-26T22:00:21.668606+0000 (oldest deadline 2024-04-26T22:00:46.967821+0000) 2024-04-26T22:01:01.314 INFO:journalctl@ceph.osd.7.smithi076.stdout:Apr 26 22:01:01 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-7[49518]: 2024-04-26T22:01:01.026+0000 7fcc04a00700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:34.571664+0000 front 2024-04-26T22:00:34.572066+0000 (oldest deadline 2024-04-26T22:00:57.471976+0000) 2024-04-26T22:01:01.494 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 26 22:01:01 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-8[50342]: 2024-04-26T22:01:01.306+0000 7f152e400700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:25.173410+0000 front 2024-04-26T22:00:25.173841+0000 (oldest deadline 2024-04-26T22:00:50.473747+0000) 2024-04-26T22:01:01.494 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 26 22:01:01 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-8[50342]: 2024-04-26T22:01:01.306+0000 7f152e400700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:30.474905+0000 front 2024-04-26T22:00:30.475573+0000 (oldest deadline 2024-04-26T22:00:56.375141+0000) 2024-04-26T22:01:01.495 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:01 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:01.227+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:14.343204+0000 front 2024-04-26T22:00:14.342815+0000 (oldest deadline 2024-04-26T22:00:39.042910+0000) 2024-04-26T22:01:01.495 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:01 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:01.227+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:24.945823+0000 front 2024-04-26T22:00:24.945339+0000 (oldest deadline 2024-04-26T22:00:47.245223+0000) 2024-04-26T22:01:01.495 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:01 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:01.227+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:31.947744+0000 front 2024-04-26T22:00:31.948080+0000 (oldest deadline 2024-04-26T22:00:56.647838+0000) 2024-04-26T22:01:01.568 INFO:journalctl@ceph.osd.6.smithi072.stdout:Apr 26 22:01:01 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-6[57091]: 2024-04-26T22:01:01.163+0000 7fb154800700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:16.543358+0000 front 2024-04-26T22:00:16.543710+0000 (oldest deadline 2024-04-26T22:00:41.744798+0000) 2024-04-26T22:01:01.568 INFO:journalctl@ceph.osd.6.smithi072.stdout:Apr 26 22:01:01 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-6[57091]: 2024-04-26T22:01:01.163+0000 7fb154800700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:21.746250+0000 front 2024-04-26T22:00:21.745942+0000 (oldest deadline 2024-04-26T22:00:47.645522+0000) 2024-04-26T22:01:01.568 INFO:journalctl@ceph.osd.6.smithi072.stdout:Apr 26 22:01:01 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-6[57091]: 2024-04-26T22:01:01.163+0000 7fb154800700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:31.649534+0000 front 2024-04-26T22:00:31.649342+0000 (oldest deadline 2024-04-26T22:00:56.949491+0000) 2024-04-26T22:01:01.821 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:01:01 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:01:01.567+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:16.595921+0000 front 2024-04-26T22:00:16.596275+0000 (oldest deadline 2024-04-26T22:00:39.495420+0000) 2024-04-26T22:01:01.822 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:01:01 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:01:01.567+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:24.600976+0000 front 2024-04-26T22:00:24.600553+0000 (oldest deadline 2024-04-26T22:00:48.100712+0000) 2024-04-26T22:01:01.822 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:01:01 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:01:01.567+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:34.002793+0000 front 2024-04-26T22:00:34.003098+0000 (oldest deadline 2024-04-26T22:00:58.103086+0000) 2024-04-26T22:01:01.940 INFO:journalctl@ceph.osd.5.smithi164.stdout:Apr 26 22:01:01 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-5[43528]: 2024-04-26T22:01:01.729+0000 7fb90c200700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:15.638228+0000 front 2024-04-26T22:00:15.637988+0000 (oldest deadline 2024-04-26T22:00:38.538212+0000) 2024-04-26T22:01:01.940 INFO:journalctl@ceph.osd.5.smithi164.stdout:Apr 26 22:01:01 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-5[43528]: 2024-04-26T22:01:01.729+0000 7fb90c200700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:23.141714+0000 front 2024-04-26T22:00:23.141785+0000 (oldest deadline 2024-04-26T22:00:49.041949+0000) 2024-04-26T22:01:01.940 INFO:journalctl@ceph.osd.5.smithi164.stdout:Apr 26 22:01:01 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-5[43528]: 2024-04-26T22:01:01.729+0000 7fb90c200700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:35.247685+0000 front 2024-04-26T22:00:35.247753+0000 (oldest deadline 2024-04-26T22:01:00.547904+0000) 2024-04-26T22:01:01.988 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:01 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:01.776+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:11.890629+0000 front 2024-04-26T22:00:11.890849+0000 (oldest deadline 2024-04-26T22:00:40.692043+0000) 2024-04-26T22:01:01.989 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:01 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:01.776+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:24.194239+0000 front 2024-04-26T22:00:24.194095+0000 (oldest deadline 2024-04-26T22:00:48.893996+0000) 2024-04-26T22:01:01.989 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:01 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:01.776+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:34.096788+0000 front 2024-04-26T22:00:34.096884+0000 (oldest deadline 2024-04-26T22:00:56.396862+0000) 2024-04-26T22:01:02.314 INFO:journalctl@ceph.osd.7.smithi076.stdout:Apr 26 22:01:01 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-7[49518]: 2024-04-26T22:01:01.988+0000 7fcc04a00700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:21.667782+0000 front 2024-04-26T22:00:21.668606+0000 (oldest deadline 2024-04-26T22:00:46.967821+0000) 2024-04-26T22:01:02.314 INFO:journalctl@ceph.osd.7.smithi076.stdout:Apr 26 22:01:01 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-7[49518]: 2024-04-26T22:01:01.988+0000 7fcc04a00700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:34.571664+0000 front 2024-04-26T22:00:34.572066+0000 (oldest deadline 2024-04-26T22:00:57.471976+0000) 2024-04-26T22:01:02.548 INFO:journalctl@ceph.osd.6.smithi072.stdout:Apr 26 22:01:02 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-6[57091]: 2024-04-26T22:01:02.136+0000 7fb154800700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:16.543358+0000 front 2024-04-26T22:00:16.543710+0000 (oldest deadline 2024-04-26T22:00:41.744798+0000) 2024-04-26T22:01:02.548 INFO:journalctl@ceph.osd.6.smithi072.stdout:Apr 26 22:01:02 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-6[57091]: 2024-04-26T22:01:02.136+0000 7fb154800700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:21.746250+0000 front 2024-04-26T22:00:21.745942+0000 (oldest deadline 2024-04-26T22:00:47.645522+0000) 2024-04-26T22:01:02.548 INFO:journalctl@ceph.osd.6.smithi072.stdout:Apr 26 22:01:02 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-6[57091]: 2024-04-26T22:01:02.136+0000 7fb154800700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:31.649534+0000 front 2024-04-26T22:00:31.649342+0000 (oldest deadline 2024-04-26T22:00:56.949491+0000) 2024-04-26T22:01:02.690 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 26 22:01:02 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-8[50342]: 2024-04-26T22:01:02.350+0000 7f152e400700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:25.173410+0000 front 2024-04-26T22:00:25.173841+0000 (oldest deadline 2024-04-26T22:00:50.473747+0000) 2024-04-26T22:01:02.690 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 26 22:01:02 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-8[50342]: 2024-04-26T22:01:02.350+0000 7f152e400700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:30.474905+0000 front 2024-04-26T22:00:30.475573+0000 (oldest deadline 2024-04-26T22:00:56.375141+0000) 2024-04-26T22:01:02.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:02 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:02.217+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:14.343204+0000 front 2024-04-26T22:00:14.342815+0000 (oldest deadline 2024-04-26T22:00:39.042910+0000) 2024-04-26T22:01:02.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:02 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:02.217+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:24.945823+0000 front 2024-04-26T22:00:24.945339+0000 (oldest deadline 2024-04-26T22:00:47.245223+0000) 2024-04-26T22:01:02.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:02 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:02.217+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:31.947744+0000 front 2024-04-26T22:00:31.948080+0000 (oldest deadline 2024-04-26T22:00:56.647838+0000) 2024-04-26T22:01:02.821 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:01:02 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:01:02.548+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:16.595921+0000 front 2024-04-26T22:00:16.596275+0000 (oldest deadline 2024-04-26T22:00:39.495420+0000) 2024-04-26T22:01:02.822 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:01:02 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:01:02.548+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:24.600976+0000 front 2024-04-26T22:00:24.600553+0000 (oldest deadline 2024-04-26T22:00:48.100712+0000) 2024-04-26T22:01:02.822 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:01:02 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:01:02.548+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:34.002793+0000 front 2024-04-26T22:00:34.003098+0000 (oldest deadline 2024-04-26T22:00:58.103086+0000) 2024-04-26T22:01:03.006 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:02 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:02.748+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:11.890629+0000 front 2024-04-26T22:00:11.890849+0000 (oldest deadline 2024-04-26T22:00:40.692043+0000) 2024-04-26T22:01:03.006 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:02 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:02.748+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:24.194239+0000 front 2024-04-26T22:00:24.194095+0000 (oldest deadline 2024-04-26T22:00:48.893996+0000) 2024-04-26T22:01:03.006 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:02 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:02.748+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:34.096788+0000 front 2024-04-26T22:00:34.096884+0000 (oldest deadline 2024-04-26T22:00:56.396862+0000) 2024-04-26T22:01:03.190 INFO:journalctl@ceph.osd.5.smithi164.stdout:Apr 26 22:01:02 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-5[43528]: 2024-04-26T22:01:02.691+0000 7fb90c200700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:15.638228+0000 front 2024-04-26T22:00:15.637988+0000 (oldest deadline 2024-04-26T22:00:38.538212+0000) 2024-04-26T22:01:03.190 INFO:journalctl@ceph.osd.5.smithi164.stdout:Apr 26 22:01:02 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-5[43528]: 2024-04-26T22:01:02.691+0000 7fb90c200700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:23.141714+0000 front 2024-04-26T22:00:23.141785+0000 (oldest deadline 2024-04-26T22:00:49.041949+0000) 2024-04-26T22:01:03.191 INFO:journalctl@ceph.osd.5.smithi164.stdout:Apr 26 22:01:02 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-5[43528]: 2024-04-26T22:01:02.691+0000 7fb90c200700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:35.247685+0000 front 2024-04-26T22:00:35.247753+0000 (oldest deadline 2024-04-26T22:01:00.547904+0000) 2024-04-26T22:01:03.314 INFO:journalctl@ceph.osd.7.smithi076.stdout:Apr 26 22:01:03 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-7[49518]: 2024-04-26T22:01:03.005+0000 7fcc04a00700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:21.667782+0000 front 2024-04-26T22:00:21.668606+0000 (oldest deadline 2024-04-26T22:00:46.967821+0000) 2024-04-26T22:01:03.314 INFO:journalctl@ceph.osd.7.smithi076.stdout:Apr 26 22:01:03 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-7[49518]: 2024-04-26T22:01:03.005+0000 7fcc04a00700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:34.571664+0000 front 2024-04-26T22:00:34.572066+0000 (oldest deadline 2024-04-26T22:00:57.471976+0000) 2024-04-26T22:01:03.540 INFO:journalctl@ceph.osd.6.smithi072.stdout:Apr 26 22:01:03 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-6[57091]: 2024-04-26T22:01:03.186+0000 7fb154800700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:16.543358+0000 front 2024-04-26T22:00:16.543710+0000 (oldest deadline 2024-04-26T22:00:41.744798+0000) 2024-04-26T22:01:03.540 INFO:journalctl@ceph.osd.6.smithi072.stdout:Apr 26 22:01:03 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-6[57091]: 2024-04-26T22:01:03.186+0000 7fb154800700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:21.746250+0000 front 2024-04-26T22:00:21.745942+0000 (oldest deadline 2024-04-26T22:00:47.645522+0000) 2024-04-26T22:01:03.540 INFO:journalctl@ceph.osd.6.smithi072.stdout:Apr 26 22:01:03 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-6[57091]: 2024-04-26T22:01:03.186+0000 7fb154800700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:31.649534+0000 front 2024-04-26T22:00:31.649342+0000 (oldest deadline 2024-04-26T22:00:56.949491+0000) 2024-04-26T22:01:03.690 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 26 22:01:03 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-8[50342]: 2024-04-26T22:01:03.358+0000 7f152e400700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:25.173410+0000 front 2024-04-26T22:00:25.173841+0000 (oldest deadline 2024-04-26T22:00:50.473747+0000) 2024-04-26T22:01:03.691 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 26 22:01:03 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-8[50342]: 2024-04-26T22:01:03.358+0000 7f152e400700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:30.474905+0000 front 2024-04-26T22:00:30.475573+0000 (oldest deadline 2024-04-26T22:00:56.375141+0000) 2024-04-26T22:01:03.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:03 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:03.267+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:14.343204+0000 front 2024-04-26T22:00:14.342815+0000 (oldest deadline 2024-04-26T22:00:39.042910+0000) 2024-04-26T22:01:03.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:03 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:03.267+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:24.945823+0000 front 2024-04-26T22:00:24.945339+0000 (oldest deadline 2024-04-26T22:00:47.245223+0000) 2024-04-26T22:01:03.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:03 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:03.267+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:31.947744+0000 front 2024-04-26T22:00:31.948080+0000 (oldest deadline 2024-04-26T22:00:56.647838+0000) 2024-04-26T22:01:03.821 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:01:03 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:01:03.539+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:16.595921+0000 front 2024-04-26T22:00:16.596275+0000 (oldest deadline 2024-04-26T22:00:39.495420+0000) 2024-04-26T22:01:03.821 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:01:03 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:01:03.539+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:24.600976+0000 front 2024-04-26T22:00:24.600553+0000 (oldest deadline 2024-04-26T22:00:48.100712+0000) 2024-04-26T22:01:03.822 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:01:03 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:01:03.539+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:34.002793+0000 front 2024-04-26T22:00:34.003098+0000 (oldest deadline 2024-04-26T22:00:58.103086+0000) 2024-04-26T22:01:04.064 INFO:journalctl@ceph.osd.7.smithi076.stdout:Apr 26 22:01:03 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-7[49518]: 2024-04-26T22:01:03.973+0000 7fcc04a00700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:21.667782+0000 front 2024-04-26T22:00:21.668606+0000 (oldest deadline 2024-04-26T22:00:46.967821+0000) 2024-04-26T22:01:04.064 INFO:journalctl@ceph.osd.7.smithi076.stdout:Apr 26 22:01:03 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-7[49518]: 2024-04-26T22:01:03.973+0000 7fcc04a00700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:34.571664+0000 front 2024-04-26T22:00:34.572066+0000 (oldest deadline 2024-04-26T22:00:57.471976+0000) 2024-04-26T22:01:04.065 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:03 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:03.766+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:11.890629+0000 front 2024-04-26T22:00:11.890849+0000 (oldest deadline 2024-04-26T22:00:40.692043+0000) 2024-04-26T22:01:04.065 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:03 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:03.766+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:24.194239+0000 front 2024-04-26T22:00:24.194095+0000 (oldest deadline 2024-04-26T22:00:48.893996+0000) 2024-04-26T22:01:04.065 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:03 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:03.766+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:34.096788+0000 front 2024-04-26T22:00:34.096884+0000 (oldest deadline 2024-04-26T22:00:56.396862+0000) 2024-04-26T22:01:04.350 INFO:journalctl@ceph.osd.5.smithi164.stdout:Apr 26 22:01:04 smithi164 podman[66454]: 2024-04-26 22:01:04.004155614 +0000 UTC m=+5.708895624 container died e91b532bd9683f9871a8d1ea9bfa4ac92540043b46ec8e983102cde21b80c08f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-5, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-04-26T22:01:04.351 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:04 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:04.249+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:14.343204+0000 front 2024-04-26T22:00:14.342815+0000 (oldest deadline 2024-04-26T22:00:39.042910+0000) 2024-04-26T22:01:04.351 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:04 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:04.249+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:24.945823+0000 front 2024-04-26T22:00:24.945339+0000 (oldest deadline 2024-04-26T22:00:47.245223+0000) 2024-04-26T22:01:04.351 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:04 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:04.249+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:31.947744+0000 front 2024-04-26T22:00:31.948080+0000 (oldest deadline 2024-04-26T22:00:56.647838+0000) 2024-04-26T22:01:04.548 INFO:journalctl@ceph.osd.6.smithi072.stdout:Apr 26 22:01:04 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-6[57091]: 2024-04-26T22:01:04.157+0000 7fb154800700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:16.543358+0000 front 2024-04-26T22:00:16.543710+0000 (oldest deadline 2024-04-26T22:00:41.744798+0000) 2024-04-26T22:01:04.548 INFO:journalctl@ceph.osd.6.smithi072.stdout:Apr 26 22:01:04 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-6[57091]: 2024-04-26T22:01:04.157+0000 7fb154800700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:21.746250+0000 front 2024-04-26T22:00:21.745942+0000 (oldest deadline 2024-04-26T22:00:47.645522+0000) 2024-04-26T22:01:04.548 INFO:journalctl@ceph.osd.6.smithi072.stdout:Apr 26 22:01:04 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-6[57091]: 2024-04-26T22:01:04.157+0000 7fb154800700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:31.649534+0000 front 2024-04-26T22:00:31.649342+0000 (oldest deadline 2024-04-26T22:00:56.949491+0000) 2024-04-26T22:01:04.690 INFO:journalctl@ceph.osd.5.smithi164.stdout:Apr 26 22:01:04 smithi164 podman[66454]: 2024-04-26 22:01:04.50948074 +0000 UTC m=+6.214220784 container cleanup e91b532bd9683f9871a8d1ea9bfa4ac92540043b46ec8e983102cde21b80c08f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-5, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True) 2024-04-26T22:01:04.691 INFO:journalctl@ceph.osd.5.smithi164.stdout:Apr 26 22:01:04 smithi164 bash[66454]: ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-5 2024-04-26T22:01:04.691 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 26 22:01:04 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-8[50342]: 2024-04-26T22:01:04.376+0000 7f152e400700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:25.173410+0000 front 2024-04-26T22:00:25.173841+0000 (oldest deadline 2024-04-26T22:00:50.473747+0000) 2024-04-26T22:01:04.691 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 26 22:01:04 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-8[50342]: 2024-04-26T22:01:04.376+0000 7f152e400700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:30.474905+0000 front 2024-04-26T22:00:30.475573+0000 (oldest deadline 2024-04-26T22:00:56.375141+0000) 2024-04-26T22:01:04.822 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:01:04 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:01:04.548+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:16.595921+0000 front 2024-04-26T22:00:16.596275+0000 (oldest deadline 2024-04-26T22:00:39.495420+0000) 2024-04-26T22:01:04.822 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:01:04 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:01:04.548+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:24.600976+0000 front 2024-04-26T22:00:24.600553+0000 (oldest deadline 2024-04-26T22:00:48.100712+0000) 2024-04-26T22:01:04.822 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:01:04 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:01:04.548+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:34.002793+0000 front 2024-04-26T22:00:34.003098+0000 (oldest deadline 2024-04-26T22:00:58.103086+0000) 2024-04-26T22:01:05.064 INFO:journalctl@ceph.osd.7.smithi076.stdout:Apr 26 22:01:04 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-7[49518]: 2024-04-26T22:01:04.938+0000 7fcc04a00700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:21.667782+0000 front 2024-04-26T22:00:21.668606+0000 (oldest deadline 2024-04-26T22:00:46.967821+0000) 2024-04-26T22:01:05.065 INFO:journalctl@ceph.osd.7.smithi076.stdout:Apr 26 22:01:04 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-7[49518]: 2024-04-26T22:01:04.938+0000 7fcc04a00700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:34.571664+0000 front 2024-04-26T22:00:34.572066+0000 (oldest deadline 2024-04-26T22:00:57.471976+0000) 2024-04-26T22:01:05.065 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:04 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:04.760+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:11.890629+0000 front 2024-04-26T22:00:11.890849+0000 (oldest deadline 2024-04-26T22:00:40.692043+0000) 2024-04-26T22:01:05.065 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:04 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:04.760+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:24.194239+0000 front 2024-04-26T22:00:24.194095+0000 (oldest deadline 2024-04-26T22:00:48.893996+0000) 2024-04-26T22:01:05.065 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:04 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:04.760+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:34.096788+0000 front 2024-04-26T22:00:34.096884+0000 (oldest deadline 2024-04-26T22:00:56.396862+0000) 2024-04-26T22:01:05.190 INFO:journalctl@ceph.osd.5.smithi164.stdout:Apr 26 22:01:04 smithi164 podman[66489]: 2024-04-26 22:01:04.693851197 +0000 UTC m=+0.675881333 container remove e91b532bd9683f9871a8d1ea9bfa4ac92540043b46ec8e983102cde21b80c08f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-5, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True) 2024-04-26T22:01:05.540 INFO:journalctl@ceph.osd.6.smithi072.stdout:Apr 26 22:01:05 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-6[57091]: 2024-04-26T22:01:05.205+0000 7fb154800700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:16.543358+0000 front 2024-04-26T22:00:16.543710+0000 (oldest deadline 2024-04-26T22:00:41.744798+0000) 2024-04-26T22:01:05.541 INFO:journalctl@ceph.osd.6.smithi072.stdout:Apr 26 22:01:05 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-6[57091]: 2024-04-26T22:01:05.205+0000 7fb154800700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:21.746250+0000 front 2024-04-26T22:00:21.745942+0000 (oldest deadline 2024-04-26T22:00:47.645522+0000) 2024-04-26T22:01:05.541 INFO:journalctl@ceph.osd.6.smithi072.stdout:Apr 26 22:01:05 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-6[57091]: 2024-04-26T22:01:05.205+0000 7fb154800700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:31.649534+0000 front 2024-04-26T22:00:31.649342+0000 (oldest deadline 2024-04-26T22:00:56.949491+0000) 2024-04-26T22:01:05.569 INFO:journalctl@ceph.osd.5.smithi164.stdout:Apr 26 22:01:05 smithi164 podman[66579]: 2024-04-26 22:01:05.405574235 +0000 UTC m=+0.033997305 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-26T22:01:05.570 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 26 22:01:05 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-8[50342]: 2024-04-26T22:01:05.396+0000 7f152e400700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:25.173410+0000 front 2024-04-26T22:00:25.173841+0000 (oldest deadline 2024-04-26T22:00:50.473747+0000) 2024-04-26T22:01:05.570 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 26 22:01:05 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-8[50342]: 2024-04-26T22:01:05.396+0000 7f152e400700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:30.474905+0000 front 2024-04-26T22:00:30.475573+0000 (oldest deadline 2024-04-26T22:00:56.375141+0000) 2024-04-26T22:01:05.570 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:05 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:05.261+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:14.343204+0000 front 2024-04-26T22:00:14.342815+0000 (oldest deadline 2024-04-26T22:00:39.042910+0000) 2024-04-26T22:01:05.570 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:05 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:05.261+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:24.945823+0000 front 2024-04-26T22:00:24.945339+0000 (oldest deadline 2024-04-26T22:00:47.245223+0000) 2024-04-26T22:01:05.570 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:05 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:05.261+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:31.947744+0000 front 2024-04-26T22:00:31.948080+0000 (oldest deadline 2024-04-26T22:00:56.647838+0000) 2024-04-26T22:01:05.821 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:01:05 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:01:05.540+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:16.595921+0000 front 2024-04-26T22:00:16.596275+0000 (oldest deadline 2024-04-26T22:00:39.495420+0000) 2024-04-26T22:01:05.822 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:01:05 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:01:05.540+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:24.600976+0000 front 2024-04-26T22:00:24.600553+0000 (oldest deadline 2024-04-26T22:00:48.100712+0000) 2024-04-26T22:01:05.822 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:01:05 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:01:05.540+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:34.002793+0000 front 2024-04-26T22:00:34.003098+0000 (oldest deadline 2024-04-26T22:00:58.103086+0000) 2024-04-26T22:01:05.844 INFO:journalctl@ceph.osd.5.smithi164.stdout:Apr 26 22:01:05 smithi164 podman[66579]: 2024-04-26 22:01:05.570091095 +0000 UTC m=+0.198514201 container create 658110f086e1c000f0b392fc1ebf876e2a3026dbe19ec895bab18f92250a2601 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-5-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.schema-version=1.0) 2024-04-26T22:01:05.844 INFO:journalctl@ceph.osd.5.smithi164.stdout:Apr 26 22:01:05 smithi164 podman[66579]: 2024-04-26 22:01:05.843784991 +0000 UTC m=+0.472208104 container init 658110f086e1c000f0b392fc1ebf876e2a3026dbe19ec895bab18f92250a2601 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-5-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, ceph=True) 2024-04-26T22:01:06.064 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:05 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:05.800+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:11.890629+0000 front 2024-04-26T22:00:11.890849+0000 (oldest deadline 2024-04-26T22:00:40.692043+0000) 2024-04-26T22:01:06.064 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:05 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:05.800+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:24.194239+0000 front 2024-04-26T22:00:24.194095+0000 (oldest deadline 2024-04-26T22:00:48.893996+0000) 2024-04-26T22:01:06.064 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:05 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:05.800+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:34.096788+0000 front 2024-04-26T22:00:34.096884+0000 (oldest deadline 2024-04-26T22:00:56.396862+0000) 2024-04-26T22:01:06.064 INFO:journalctl@ceph.osd.7.smithi076.stdout:Apr 26 22:01:05 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-7[49518]: 2024-04-26T22:01:05.891+0000 7fcc04a00700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:21.667782+0000 front 2024-04-26T22:00:21.668606+0000 (oldest deadline 2024-04-26T22:00:46.967821+0000) 2024-04-26T22:01:06.065 INFO:journalctl@ceph.osd.7.smithi076.stdout:Apr 26 22:01:05 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-7[49518]: 2024-04-26T22:01:05.891+0000 7fcc04a00700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:34.571664+0000 front 2024-04-26T22:00:34.572066+0000 (oldest deadline 2024-04-26T22:00:57.471976+0000) 2024-04-26T22:01:06.065 INFO:journalctl@ceph.osd.7.smithi076.stdout:Apr 26 22:01:05 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-7[49518]: 2024-04-26T22:01:05.891+0000 7fcc04a00700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-26T22:00:41.474549+0000 front 2024-04-26T22:00:41.475097+0000 (oldest deadline 2024-04-26T22:01:05.574631+0000) 2024-04-26T22:01:06.190 INFO:journalctl@ceph.osd.5.smithi164.stdout:Apr 26 22:01:05 smithi164 podman[66579]: 2024-04-26 22:01:05.855834253 +0000 UTC m=+0.484257328 container start 658110f086e1c000f0b392fc1ebf876e2a3026dbe19ec895bab18f92250a2601 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-5-deactivate, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-04-26T22:01:06.191 INFO:journalctl@ceph.osd.5.smithi164.stdout:Apr 26 22:01:05 smithi164 podman[66579]: 2024-04-26 22:01:05.930697258 +0000 UTC m=+0.559120364 container attach 658110f086e1c000f0b392fc1ebf876e2a3026dbe19ec895bab18f92250a2601 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-5-deactivate, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff) 2024-04-26T22:01:06.541 INFO:journalctl@ceph.osd.6.smithi072.stdout:Apr 26 22:01:06 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-6[57091]: 2024-04-26T22:01:06.211+0000 7fb154800700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:16.543358+0000 front 2024-04-26T22:00:16.543710+0000 (oldest deadline 2024-04-26T22:00:41.744798+0000) 2024-04-26T22:01:06.541 INFO:journalctl@ceph.osd.6.smithi072.stdout:Apr 26 22:01:06 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-6[57091]: 2024-04-26T22:01:06.211+0000 7fb154800700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:21.746250+0000 front 2024-04-26T22:00:21.745942+0000 (oldest deadline 2024-04-26T22:00:47.645522+0000) 2024-04-26T22:01:06.541 INFO:journalctl@ceph.osd.6.smithi072.stdout:Apr 26 22:01:06 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-6[57091]: 2024-04-26T22:01:06.211+0000 7fb154800700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:31.649534+0000 front 2024-04-26T22:00:31.649342+0000 (oldest deadline 2024-04-26T22:00:56.949491+0000) 2024-04-26T22:01:06.548 INFO:journalctl@ceph.osd.5.smithi164.stdout:Apr 26 22:01:06 smithi164 podman[66579]: 2024-04-26 22:01:06.316731631 +0000 UTC m=+0.945154780 container died 658110f086e1c000f0b392fc1ebf876e2a3026dbe19ec895bab18f92250a2601 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-5-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-04-26T22:01:06.549 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 26 22:01:06 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-8[50342]: 2024-04-26T22:01:06.388+0000 7f152e400700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:25.173410+0000 front 2024-04-26T22:00:25.173841+0000 (oldest deadline 2024-04-26T22:00:50.473747+0000) 2024-04-26T22:01:06.549 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 26 22:01:06 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-8[50342]: 2024-04-26T22:01:06.388+0000 7f152e400700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:30.474905+0000 front 2024-04-26T22:00:30.475573+0000 (oldest deadline 2024-04-26T22:00:56.375141+0000) 2024-04-26T22:01:06.549 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 26 22:01:06 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-8[50342]: 2024-04-26T22:01:06.388+0000 7f152e400700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-26T22:00:43.977887+0000 front 2024-04-26T22:00:43.978065+0000 (oldest deadline 2024-04-26T22:01:06.277746+0000) 2024-04-26T22:01:06.549 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:06 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:06.294+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:14.343204+0000 front 2024-04-26T22:00:14.342815+0000 (oldest deadline 2024-04-26T22:00:39.042910+0000) 2024-04-26T22:01:06.549 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:06 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:06.294+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:24.945823+0000 front 2024-04-26T22:00:24.945339+0000 (oldest deadline 2024-04-26T22:00:47.245223+0000) 2024-04-26T22:01:06.549 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:06 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:06.294+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:31.947744+0000 front 2024-04-26T22:00:31.948080+0000 (oldest deadline 2024-04-26T22:00:56.647838+0000) 2024-04-26T22:01:06.822 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:01:06 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:01:06.541+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:16.595921+0000 front 2024-04-26T22:00:16.596275+0000 (oldest deadline 2024-04-26T22:00:39.495420+0000) 2024-04-26T22:01:06.822 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:01:06 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:01:06.541+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:24.600976+0000 front 2024-04-26T22:00:24.600553+0000 (oldest deadline 2024-04-26T22:00:48.100712+0000) 2024-04-26T22:01:06.822 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:01:06 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:01:06.541+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:34.002793+0000 front 2024-04-26T22:00:34.003098+0000 (oldest deadline 2024-04-26T22:00:58.103086+0000) 2024-04-26T22:01:07.048 DEBUG:teuthology.orchestra.run.smithi164:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a1780950-0415-11ef-bc93-c7b262605968@osd.5.service' 2024-04-26T22:01:07.114 INFO:journalctl@ceph.osd.5.smithi164.stdout:Apr 26 22:01:07 smithi164 podman[66606]: 2024-04-26 22:01:07.017478045 +0000 UTC m=+0.685706435 container remove 658110f086e1c000f0b392fc1ebf876e2a3026dbe19ec895bab18f92250a2601 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-5-deactivate, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240424, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-04-26T22:01:07.114 INFO:journalctl@ceph.osd.5.smithi164.stdout:Apr 26 22:01:07 smithi164 systemd[1]: ceph-a1780950-0415-11ef-bc93-c7b262605968@osd.5.service: Deactivated successfully. 2024-04-26T22:01:07.114 INFO:journalctl@ceph.osd.5.smithi164.stdout:Apr 26 22:01:07 smithi164 systemd[1]: Stopped Ceph osd.5 for a1780950-0415-11ef-bc93-c7b262605968. 2024-04-26T22:01:07.114 INFO:journalctl@ceph.osd.5.smithi164.stdout:Apr 26 22:01:07 smithi164 systemd[1]: ceph-a1780950-0415-11ef-bc93-c7b262605968@osd.5.service: Consumed 1min 9.864s CPU time. 2024-04-26T22:01:07.145 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T22:01:07.145 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-04-26T22:01:07.146 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-04-26T22:01:07.146 DEBUG:teuthology.orchestra.run.smithi072:> sudo systemctl stop ceph-a1780950-0415-11ef-bc93-c7b262605968@osd.6 2024-04-26T22:01:07.315 INFO:journalctl@ceph.osd.7.smithi076.stdout:Apr 26 22:01:06 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-7[49518]: 2024-04-26T22:01:06.904+0000 7fcc04a00700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:21.667782+0000 front 2024-04-26T22:00:21.668606+0000 (oldest deadline 2024-04-26T22:00:46.967821+0000) 2024-04-26T22:01:07.315 INFO:journalctl@ceph.osd.7.smithi076.stdout:Apr 26 22:01:06 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-7[49518]: 2024-04-26T22:01:06.904+0000 7fcc04a00700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:34.571664+0000 front 2024-04-26T22:00:34.572066+0000 (oldest deadline 2024-04-26T22:00:57.471976+0000) 2024-04-26T22:01:07.315 INFO:journalctl@ceph.osd.7.smithi076.stdout:Apr 26 22:01:06 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-7[49518]: 2024-04-26T22:01:06.904+0000 7fcc04a00700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-26T22:00:41.474549+0000 front 2024-04-26T22:00:41.475097+0000 (oldest deadline 2024-04-26T22:01:05.574631+0000) 2024-04-26T22:01:07.315 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:06 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:06.842+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:11.890629+0000 front 2024-04-26T22:00:11.890849+0000 (oldest deadline 2024-04-26T22:00:40.692043+0000) 2024-04-26T22:01:07.315 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:06 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:06.842+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:24.194239+0000 front 2024-04-26T22:00:24.194095+0000 (oldest deadline 2024-04-26T22:00:48.893996+0000) 2024-04-26T22:01:07.315 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:06 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:06.842+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:34.096788+0000 front 2024-04-26T22:00:34.096884+0000 (oldest deadline 2024-04-26T22:00:56.396862+0000) 2024-04-26T22:01:07.315 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:06 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:06.842+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-26T22:00:42.299288+0000 front 2024-04-26T22:00:42.299033+0000 (oldest deadline 2024-04-26T22:01:06.398950+0000) 2024-04-26T22:01:07.441 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 26 22:01:07 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-8[50342]: 2024-04-26T22:01:07.341+0000 7f152e400700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:25.173410+0000 front 2024-04-26T22:00:25.173841+0000 (oldest deadline 2024-04-26T22:00:50.473747+0000) 2024-04-26T22:01:07.441 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 26 22:01:07 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-8[50342]: 2024-04-26T22:01:07.341+0000 7f152e400700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:30.474905+0000 front 2024-04-26T22:00:30.475573+0000 (oldest deadline 2024-04-26T22:00:56.375141+0000) 2024-04-26T22:01:07.441 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 26 22:01:07 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-8[50342]: 2024-04-26T22:01:07.341+0000 7f152e400700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-26T22:00:43.977887+0000 front 2024-04-26T22:00:43.978065+0000 (oldest deadline 2024-04-26T22:01:06.277746+0000) 2024-04-26T22:01:07.441 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:07 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:07.286+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:14.343204+0000 front 2024-04-26T22:00:14.342815+0000 (oldest deadline 2024-04-26T22:00:39.042910+0000) 2024-04-26T22:01:07.441 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:07 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:07.286+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:24.945823+0000 front 2024-04-26T22:00:24.945339+0000 (oldest deadline 2024-04-26T22:00:47.245223+0000) 2024-04-26T22:01:07.442 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:07 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:07.286+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:31.947744+0000 front 2024-04-26T22:00:31.948080+0000 (oldest deadline 2024-04-26T22:00:56.647838+0000) 2024-04-26T22:01:07.560 INFO:journalctl@ceph.osd.6.smithi072.stdout:Apr 26 22:01:07 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-6[57091]: 2024-04-26T22:01:07.215+0000 7fb154800700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:16.543358+0000 front 2024-04-26T22:00:16.543710+0000 (oldest deadline 2024-04-26T22:00:41.744798+0000) 2024-04-26T22:01:07.560 INFO:journalctl@ceph.osd.6.smithi072.stdout:Apr 26 22:01:07 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-6[57091]: 2024-04-26T22:01:07.216+0000 7fb154800700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:21.746250+0000 front 2024-04-26T22:00:21.745942+0000 (oldest deadline 2024-04-26T22:00:47.645522+0000) 2024-04-26T22:01:07.560 INFO:journalctl@ceph.osd.6.smithi072.stdout:Apr 26 22:01:07 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-6[57091]: 2024-04-26T22:01:07.216+0000 7fb154800700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:31.649534+0000 front 2024-04-26T22:00:31.649342+0000 (oldest deadline 2024-04-26T22:00:56.949491+0000) 2024-04-26T22:01:07.560 INFO:journalctl@ceph.osd.6.smithi072.stdout:Apr 26 22:01:07 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-6[57091]: 2024-04-26T22:01:07.216+0000 7fb154800700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-26T22:00:42.851796+0000 front 2024-04-26T22:00:42.852813+0000 (oldest deadline 2024-04-26T22:01:06.351702+0000) 2024-04-26T22:01:07.560 INFO:journalctl@ceph.osd.6.smithi072.stdout:Apr 26 22:01:07 smithi072 systemd[1]: Stopping Ceph osd.6 for a1780950-0415-11ef-bc93-c7b262605968... 2024-04-26T22:01:07.822 INFO:journalctl@ceph.osd.6.smithi072.stdout:Apr 26 22:01:07 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-6[57091]: 2024-04-26T22:01:07.722+0000 7fb15e000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T22:01:07.822 INFO:journalctl@ceph.osd.6.smithi072.stdout:Apr 26 22:01:07 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-6[57091]: 2024-04-26T22:01:07.722+0000 7fb15e000700 -1 osd.6 94 *** Got signal Terminated *** 2024-04-26T22:01:07.822 INFO:journalctl@ceph.osd.6.smithi072.stdout:Apr 26 22:01:07 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-6[57091]: 2024-04-26T22:01:07.722+0000 7fb15e000700 -1 osd.6 94 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-26T22:01:07.822 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:01:07 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:01:07.559+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:16.595921+0000 front 2024-04-26T22:00:16.596275+0000 (oldest deadline 2024-04-26T22:00:39.495420+0000) 2024-04-26T22:01:07.822 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:01:07 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:01:07.559+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:24.600976+0000 front 2024-04-26T22:00:24.600553+0000 (oldest deadline 2024-04-26T22:00:48.100712+0000) 2024-04-26T22:01:07.822 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:01:07 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:01:07.559+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:34.002793+0000 front 2024-04-26T22:00:34.003098+0000 (oldest deadline 2024-04-26T22:00:58.103086+0000) 2024-04-26T22:01:08.314 INFO:journalctl@ceph.osd.7.smithi076.stdout:Apr 26 22:01:07 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-7[49518]: 2024-04-26T22:01:07.892+0000 7fcc04a00700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:21.667782+0000 front 2024-04-26T22:00:21.668606+0000 (oldest deadline 2024-04-26T22:00:46.967821+0000) 2024-04-26T22:01:08.314 INFO:journalctl@ceph.osd.7.smithi076.stdout:Apr 26 22:01:07 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-7[49518]: 2024-04-26T22:01:07.892+0000 7fcc04a00700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:34.571664+0000 front 2024-04-26T22:00:34.572066+0000 (oldest deadline 2024-04-26T22:00:57.471976+0000) 2024-04-26T22:01:08.315 INFO:journalctl@ceph.osd.7.smithi076.stdout:Apr 26 22:01:07 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-7[49518]: 2024-04-26T22:01:07.892+0000 7fcc04a00700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-26T22:00:41.474549+0000 front 2024-04-26T22:00:41.475097+0000 (oldest deadline 2024-04-26T22:01:05.574631+0000) 2024-04-26T22:01:08.315 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:07 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:07.866+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:11.890629+0000 front 2024-04-26T22:00:11.890849+0000 (oldest deadline 2024-04-26T22:00:40.692043+0000) 2024-04-26T22:01:08.315 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:07 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:07.866+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:24.194239+0000 front 2024-04-26T22:00:24.194095+0000 (oldest deadline 2024-04-26T22:00:48.893996+0000) 2024-04-26T22:01:08.315 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:07 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:07.866+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:34.096788+0000 front 2024-04-26T22:00:34.096884+0000 (oldest deadline 2024-04-26T22:00:56.396862+0000) 2024-04-26T22:01:08.315 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:07 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:07.866+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-26T22:00:42.299288+0000 front 2024-04-26T22:00:42.299033+0000 (oldest deadline 2024-04-26T22:01:06.398950+0000) 2024-04-26T22:01:08.440 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 26 22:01:08 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-8[50342]: 2024-04-26T22:01:08.301+0000 7f152e400700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:25.173410+0000 front 2024-04-26T22:00:25.173841+0000 (oldest deadline 2024-04-26T22:00:50.473747+0000) 2024-04-26T22:01:08.441 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 26 22:01:08 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-8[50342]: 2024-04-26T22:01:08.301+0000 7f152e400700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:30.474905+0000 front 2024-04-26T22:00:30.475573+0000 (oldest deadline 2024-04-26T22:00:56.375141+0000) 2024-04-26T22:01:08.441 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 26 22:01:08 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-8[50342]: 2024-04-26T22:01:08.301+0000 7f152e400700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-26T22:00:43.977887+0000 front 2024-04-26T22:00:43.978065+0000 (oldest deadline 2024-04-26T22:01:06.277746+0000) 2024-04-26T22:01:08.441 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:08 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:08.321+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:14.343204+0000 front 2024-04-26T22:00:14.342815+0000 (oldest deadline 2024-04-26T22:00:39.042910+0000) 2024-04-26T22:01:08.441 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:08 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:08.321+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:24.945823+0000 front 2024-04-26T22:00:24.945339+0000 (oldest deadline 2024-04-26T22:00:47.245223+0000) 2024-04-26T22:01:08.441 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:08 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:08.321+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:31.947744+0000 front 2024-04-26T22:00:31.948080+0000 (oldest deadline 2024-04-26T22:00:56.647838+0000) 2024-04-26T22:01:08.441 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:08 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:08.321+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-26T22:00:42.953004+0000 front 2024-04-26T22:00:42.952908+0000 (oldest deadline 2024-04-26T22:01:07.652653+0000) 2024-04-26T22:01:08.542 INFO:journalctl@ceph.osd.6.smithi072.stdout:Apr 26 22:01:08 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-6[57091]: 2024-04-26T22:01:08.203+0000 7fb154800700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:16.543358+0000 front 2024-04-26T22:00:16.543710+0000 (oldest deadline 2024-04-26T22:00:41.744798+0000) 2024-04-26T22:01:08.542 INFO:journalctl@ceph.osd.6.smithi072.stdout:Apr 26 22:01:08 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-6[57091]: 2024-04-26T22:01:08.203+0000 7fb154800700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:21.746250+0000 front 2024-04-26T22:00:21.745942+0000 (oldest deadline 2024-04-26T22:00:47.645522+0000) 2024-04-26T22:01:08.542 INFO:journalctl@ceph.osd.6.smithi072.stdout:Apr 26 22:01:08 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-6[57091]: 2024-04-26T22:01:08.203+0000 7fb154800700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:31.649534+0000 front 2024-04-26T22:00:31.649342+0000 (oldest deadline 2024-04-26T22:00:56.949491+0000) 2024-04-26T22:01:08.542 INFO:journalctl@ceph.osd.6.smithi072.stdout:Apr 26 22:01:08 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-6[57091]: 2024-04-26T22:01:08.203+0000 7fb154800700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-26T22:00:42.851796+0000 front 2024-04-26T22:00:42.852813+0000 (oldest deadline 2024-04-26T22:01:06.351702+0000) 2024-04-26T22:01:08.822 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:01:08 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:01:08.541+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:16.595921+0000 front 2024-04-26T22:00:16.596275+0000 (oldest deadline 2024-04-26T22:00:39.495420+0000) 2024-04-26T22:01:08.822 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:01:08 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:01:08.541+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:24.600976+0000 front 2024-04-26T22:00:24.600553+0000 (oldest deadline 2024-04-26T22:00:48.100712+0000) 2024-04-26T22:01:08.822 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:01:08 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:01:08.541+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:34.002793+0000 front 2024-04-26T22:00:34.003098+0000 (oldest deadline 2024-04-26T22:00:58.103086+0000) 2024-04-26T22:01:08.822 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:01:08 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:01:08.541+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-26T22:00:41.004997+0000 front 2024-04-26T22:00:41.005270+0000 (oldest deadline 2024-04-26T22:01:08.006242+0000) 2024-04-26T22:01:09.314 INFO:journalctl@ceph.osd.7.smithi076.stdout:Apr 26 22:01:08 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-7[49518]: 2024-04-26T22:01:08.867+0000 7fcc04a00700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:21.667782+0000 front 2024-04-26T22:00:21.668606+0000 (oldest deadline 2024-04-26T22:00:46.967821+0000) 2024-04-26T22:01:09.314 INFO:journalctl@ceph.osd.7.smithi076.stdout:Apr 26 22:01:08 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-7[49518]: 2024-04-26T22:01:08.867+0000 7fcc04a00700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:34.571664+0000 front 2024-04-26T22:00:34.572066+0000 (oldest deadline 2024-04-26T22:00:57.471976+0000) 2024-04-26T22:01:09.315 INFO:journalctl@ceph.osd.7.smithi076.stdout:Apr 26 22:01:08 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-7[49518]: 2024-04-26T22:01:08.867+0000 7fcc04a00700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-26T22:00:41.474549+0000 front 2024-04-26T22:00:41.475097+0000 (oldest deadline 2024-04-26T22:01:05.574631+0000) 2024-04-26T22:01:09.315 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:08 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:08.864+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:11.890629+0000 front 2024-04-26T22:00:11.890849+0000 (oldest deadline 2024-04-26T22:00:40.692043+0000) 2024-04-26T22:01:09.315 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:08 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:08.864+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:24.194239+0000 front 2024-04-26T22:00:24.194095+0000 (oldest deadline 2024-04-26T22:00:48.893996+0000) 2024-04-26T22:01:09.315 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:08 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:08.864+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:34.096788+0000 front 2024-04-26T22:00:34.096884+0000 (oldest deadline 2024-04-26T22:00:56.396862+0000) 2024-04-26T22:01:09.315 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:08 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:08.864+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-26T22:00:42.299288+0000 front 2024-04-26T22:00:42.299033+0000 (oldest deadline 2024-04-26T22:01:06.398950+0000) 2024-04-26T22:01:09.525 INFO:journalctl@ceph.osd.6.smithi072.stdout:Apr 26 22:01:09 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-6[57091]: 2024-04-26T22:01:09.174+0000 7fb154800700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:16.543358+0000 front 2024-04-26T22:00:16.543710+0000 (oldest deadline 2024-04-26T22:00:41.744798+0000) 2024-04-26T22:01:09.526 INFO:journalctl@ceph.osd.6.smithi072.stdout:Apr 26 22:01:09 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-6[57091]: 2024-04-26T22:01:09.174+0000 7fb154800700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:21.746250+0000 front 2024-04-26T22:00:21.745942+0000 (oldest deadline 2024-04-26T22:00:47.645522+0000) 2024-04-26T22:01:09.526 INFO:journalctl@ceph.osd.6.smithi072.stdout:Apr 26 22:01:09 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-6[57091]: 2024-04-26T22:01:09.174+0000 7fb154800700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:31.649534+0000 front 2024-04-26T22:00:31.649342+0000 (oldest deadline 2024-04-26T22:00:56.949491+0000) 2024-04-26T22:01:09.526 INFO:journalctl@ceph.osd.6.smithi072.stdout:Apr 26 22:01:09 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-6[57091]: 2024-04-26T22:01:09.174+0000 7fb154800700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-26T22:00:42.851796+0000 front 2024-04-26T22:00:42.852813+0000 (oldest deadline 2024-04-26T22:01:06.351702+0000) 2024-04-26T22:01:09.690 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 26 22:01:09 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-8[50342]: 2024-04-26T22:01:09.310+0000 7f152e400700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:25.173410+0000 front 2024-04-26T22:00:25.173841+0000 (oldest deadline 2024-04-26T22:00:50.473747+0000) 2024-04-26T22:01:09.691 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 26 22:01:09 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-8[50342]: 2024-04-26T22:01:09.310+0000 7f152e400700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:30.474905+0000 front 2024-04-26T22:00:30.475573+0000 (oldest deadline 2024-04-26T22:00:56.375141+0000) 2024-04-26T22:01:09.691 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 26 22:01:09 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-8[50342]: 2024-04-26T22:01:09.310+0000 7f152e400700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-26T22:00:43.977887+0000 front 2024-04-26T22:00:43.978065+0000 (oldest deadline 2024-04-26T22:01:06.277746+0000) 2024-04-26T22:01:09.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:09 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:09.319+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:14.343204+0000 front 2024-04-26T22:00:14.342815+0000 (oldest deadline 2024-04-26T22:00:39.042910+0000) 2024-04-26T22:01:09.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:09 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:09.319+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:24.945823+0000 front 2024-04-26T22:00:24.945339+0000 (oldest deadline 2024-04-26T22:00:47.245223+0000) 2024-04-26T22:01:09.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:09 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:09.319+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:31.947744+0000 front 2024-04-26T22:00:31.948080+0000 (oldest deadline 2024-04-26T22:00:56.647838+0000) 2024-04-26T22:01:09.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:09 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:09.319+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-26T22:00:42.953004+0000 front 2024-04-26T22:00:42.952908+0000 (oldest deadline 2024-04-26T22:01:07.652653+0000) 2024-04-26T22:01:09.821 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:01:09 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:01:09.524+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:16.595921+0000 front 2024-04-26T22:00:16.596275+0000 (oldest deadline 2024-04-26T22:00:39.495420+0000) 2024-04-26T22:01:09.822 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:01:09 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:01:09.524+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:24.600976+0000 front 2024-04-26T22:00:24.600553+0000 (oldest deadline 2024-04-26T22:00:48.100712+0000) 2024-04-26T22:01:09.822 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:01:09 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:01:09.524+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:34.002793+0000 front 2024-04-26T22:00:34.003098+0000 (oldest deadline 2024-04-26T22:00:58.103086+0000) 2024-04-26T22:01:09.822 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:01:09 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:01:09.524+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-26T22:00:41.004997+0000 front 2024-04-26T22:00:41.005270+0000 (oldest deadline 2024-04-26T22:01:08.006242+0000) 2024-04-26T22:01:10.314 INFO:journalctl@ceph.osd.7.smithi076.stdout:Apr 26 22:01:09 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-7[49518]: 2024-04-26T22:01:09.838+0000 7fcc04a00700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:21.667782+0000 front 2024-04-26T22:00:21.668606+0000 (oldest deadline 2024-04-26T22:00:46.967821+0000) 2024-04-26T22:01:10.315 INFO:journalctl@ceph.osd.7.smithi076.stdout:Apr 26 22:01:09 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-7[49518]: 2024-04-26T22:01:09.838+0000 7fcc04a00700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:34.571664+0000 front 2024-04-26T22:00:34.572066+0000 (oldest deadline 2024-04-26T22:00:57.471976+0000) 2024-04-26T22:01:10.315 INFO:journalctl@ceph.osd.7.smithi076.stdout:Apr 26 22:01:09 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-7[49518]: 2024-04-26T22:01:09.838+0000 7fcc04a00700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-26T22:00:41.474549+0000 front 2024-04-26T22:00:41.475097+0000 (oldest deadline 2024-04-26T22:01:05.574631+0000) 2024-04-26T22:01:10.315 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:09 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:09.833+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:11.890629+0000 front 2024-04-26T22:00:11.890849+0000 (oldest deadline 2024-04-26T22:00:40.692043+0000) 2024-04-26T22:01:10.315 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:09 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:09.833+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:24.194239+0000 front 2024-04-26T22:00:24.194095+0000 (oldest deadline 2024-04-26T22:00:48.893996+0000) 2024-04-26T22:01:10.315 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:09 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:09.833+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:34.096788+0000 front 2024-04-26T22:00:34.096884+0000 (oldest deadline 2024-04-26T22:00:56.396862+0000) 2024-04-26T22:01:10.315 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:09 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:09.833+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-26T22:00:42.299288+0000 front 2024-04-26T22:00:42.299033+0000 (oldest deadline 2024-04-26T22:01:06.398950+0000) 2024-04-26T22:01:10.533 INFO:journalctl@ceph.osd.6.smithi072.stdout:Apr 26 22:01:10 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-6[57091]: 2024-04-26T22:01:10.166+0000 7fb154800700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:16.543358+0000 front 2024-04-26T22:00:16.543710+0000 (oldest deadline 2024-04-26T22:00:41.744798+0000) 2024-04-26T22:01:10.533 INFO:journalctl@ceph.osd.6.smithi072.stdout:Apr 26 22:01:10 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-6[57091]: 2024-04-26T22:01:10.166+0000 7fb154800700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:21.746250+0000 front 2024-04-26T22:00:21.745942+0000 (oldest deadline 2024-04-26T22:00:47.645522+0000) 2024-04-26T22:01:10.533 INFO:journalctl@ceph.osd.6.smithi072.stdout:Apr 26 22:01:10 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-6[57091]: 2024-04-26T22:01:10.166+0000 7fb154800700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:31.649534+0000 front 2024-04-26T22:00:31.649342+0000 (oldest deadline 2024-04-26T22:00:56.949491+0000) 2024-04-26T22:01:10.533 INFO:journalctl@ceph.osd.6.smithi072.stdout:Apr 26 22:01:10 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-6[57091]: 2024-04-26T22:01:10.166+0000 7fb154800700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-26T22:00:42.851796+0000 front 2024-04-26T22:00:42.852813+0000 (oldest deadline 2024-04-26T22:01:06.351702+0000) 2024-04-26T22:01:10.690 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 26 22:01:10 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-8[50342]: 2024-04-26T22:01:10.293+0000 7f152e400700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:25.173410+0000 front 2024-04-26T22:00:25.173841+0000 (oldest deadline 2024-04-26T22:00:50.473747+0000) 2024-04-26T22:01:10.691 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 26 22:01:10 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-8[50342]: 2024-04-26T22:01:10.293+0000 7f152e400700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:30.474905+0000 front 2024-04-26T22:00:30.475573+0000 (oldest deadline 2024-04-26T22:00:56.375141+0000) 2024-04-26T22:01:10.691 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 26 22:01:10 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-8[50342]: 2024-04-26T22:01:10.293+0000 7f152e400700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-26T22:00:43.977887+0000 front 2024-04-26T22:00:43.978065+0000 (oldest deadline 2024-04-26T22:01:06.277746+0000) 2024-04-26T22:01:10.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:10 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:10.294+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:14.343204+0000 front 2024-04-26T22:00:14.342815+0000 (oldest deadline 2024-04-26T22:00:39.042910+0000) 2024-04-26T22:01:10.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:10 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:10.294+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:24.945823+0000 front 2024-04-26T22:00:24.945339+0000 (oldest deadline 2024-04-26T22:00:47.245223+0000) 2024-04-26T22:01:10.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:10 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:10.294+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:31.947744+0000 front 2024-04-26T22:00:31.948080+0000 (oldest deadline 2024-04-26T22:00:56.647838+0000) 2024-04-26T22:01:10.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:10 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:10.294+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-26T22:00:42.953004+0000 front 2024-04-26T22:00:42.952908+0000 (oldest deadline 2024-04-26T22:01:07.652653+0000) 2024-04-26T22:01:10.821 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:01:10 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:01:10.531+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:16.595921+0000 front 2024-04-26T22:00:16.596275+0000 (oldest deadline 2024-04-26T22:00:39.495420+0000) 2024-04-26T22:01:10.822 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:01:10 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:01:10.531+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:24.600976+0000 front 2024-04-26T22:00:24.600553+0000 (oldest deadline 2024-04-26T22:00:48.100712+0000) 2024-04-26T22:01:10.822 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:01:10 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:01:10.531+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:34.002793+0000 front 2024-04-26T22:00:34.003098+0000 (oldest deadline 2024-04-26T22:00:58.103086+0000) 2024-04-26T22:01:10.822 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:01:10 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:01:10.531+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-26T22:00:41.004997+0000 front 2024-04-26T22:00:41.005270+0000 (oldest deadline 2024-04-26T22:01:08.006242+0000) 2024-04-26T22:01:11.314 INFO:journalctl@ceph.osd.7.smithi076.stdout:Apr 26 22:01:10 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-7[49518]: 2024-04-26T22:01:10.822+0000 7fcc04a00700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:21.667782+0000 front 2024-04-26T22:00:21.668606+0000 (oldest deadline 2024-04-26T22:00:46.967821+0000) 2024-04-26T22:01:11.315 INFO:journalctl@ceph.osd.7.smithi076.stdout:Apr 26 22:01:10 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-7[49518]: 2024-04-26T22:01:10.822+0000 7fcc04a00700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:34.571664+0000 front 2024-04-26T22:00:34.572066+0000 (oldest deadline 2024-04-26T22:00:57.471976+0000) 2024-04-26T22:01:11.315 INFO:journalctl@ceph.osd.7.smithi076.stdout:Apr 26 22:01:10 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-7[49518]: 2024-04-26T22:01:10.822+0000 7fcc04a00700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-26T22:00:41.474549+0000 front 2024-04-26T22:00:41.475097+0000 (oldest deadline 2024-04-26T22:01:05.574631+0000) 2024-04-26T22:01:11.315 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:10 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:10.852+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:11.890629+0000 front 2024-04-26T22:00:11.890849+0000 (oldest deadline 2024-04-26T22:00:40.692043+0000) 2024-04-26T22:01:11.315 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:10 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:10.852+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:24.194239+0000 front 2024-04-26T22:00:24.194095+0000 (oldest deadline 2024-04-26T22:00:48.893996+0000) 2024-04-26T22:01:11.315 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:10 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:10.852+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:34.096788+0000 front 2024-04-26T22:00:34.096884+0000 (oldest deadline 2024-04-26T22:00:56.396862+0000) 2024-04-26T22:01:11.315 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:10 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:10.852+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-26T22:00:42.299288+0000 front 2024-04-26T22:00:42.299033+0000 (oldest deadline 2024-04-26T22:01:06.398950+0000) 2024-04-26T22:01:11.528 INFO:journalctl@ceph.osd.6.smithi072.stdout:Apr 26 22:01:11 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-6[57091]: 2024-04-26T22:01:11.211+0000 7fb154800700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:16.543358+0000 front 2024-04-26T22:00:16.543710+0000 (oldest deadline 2024-04-26T22:00:41.744798+0000) 2024-04-26T22:01:11.528 INFO:journalctl@ceph.osd.6.smithi072.stdout:Apr 26 22:01:11 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-6[57091]: 2024-04-26T22:01:11.211+0000 7fb154800700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:21.746250+0000 front 2024-04-26T22:00:21.745942+0000 (oldest deadline 2024-04-26T22:00:47.645522+0000) 2024-04-26T22:01:11.528 INFO:journalctl@ceph.osd.6.smithi072.stdout:Apr 26 22:01:11 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-6[57091]: 2024-04-26T22:01:11.211+0000 7fb154800700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:31.649534+0000 front 2024-04-26T22:00:31.649342+0000 (oldest deadline 2024-04-26T22:00:56.949491+0000) 2024-04-26T22:01:11.528 INFO:journalctl@ceph.osd.6.smithi072.stdout:Apr 26 22:01:11 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-6[57091]: 2024-04-26T22:01:11.211+0000 7fb154800700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-26T22:00:42.851796+0000 front 2024-04-26T22:00:42.852813+0000 (oldest deadline 2024-04-26T22:01:06.351702+0000) 2024-04-26T22:01:11.690 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 26 22:01:11 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-8[50342]: 2024-04-26T22:01:11.262+0000 7f152e400700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:25.173410+0000 front 2024-04-26T22:00:25.173841+0000 (oldest deadline 2024-04-26T22:00:50.473747+0000) 2024-04-26T22:01:11.690 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 26 22:01:11 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-8[50342]: 2024-04-26T22:01:11.262+0000 7f152e400700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:30.474905+0000 front 2024-04-26T22:00:30.475573+0000 (oldest deadline 2024-04-26T22:00:56.375141+0000) 2024-04-26T22:01:11.691 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 26 22:01:11 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-8[50342]: 2024-04-26T22:01:11.262+0000 7f152e400700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-26T22:00:43.977887+0000 front 2024-04-26T22:00:43.978065+0000 (oldest deadline 2024-04-26T22:01:06.277746+0000) 2024-04-26T22:01:11.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:11 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:11.310+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:14.343204+0000 front 2024-04-26T22:00:14.342815+0000 (oldest deadline 2024-04-26T22:00:39.042910+0000) 2024-04-26T22:01:11.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:11 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:11.310+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:24.945823+0000 front 2024-04-26T22:00:24.945339+0000 (oldest deadline 2024-04-26T22:00:47.245223+0000) 2024-04-26T22:01:11.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:11 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:11.310+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:31.947744+0000 front 2024-04-26T22:00:31.948080+0000 (oldest deadline 2024-04-26T22:00:56.647838+0000) 2024-04-26T22:01:11.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:11 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:11.310+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-26T22:00:42.953004+0000 front 2024-04-26T22:00:42.952908+0000 (oldest deadline 2024-04-26T22:01:07.652653+0000) 2024-04-26T22:01:11.821 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:01:11 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:01:11.526+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:16.595921+0000 front 2024-04-26T22:00:16.596275+0000 (oldest deadline 2024-04-26T22:00:39.495420+0000) 2024-04-26T22:01:11.822 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:01:11 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:01:11.526+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:24.600976+0000 front 2024-04-26T22:00:24.600553+0000 (oldest deadline 2024-04-26T22:00:48.100712+0000) 2024-04-26T22:01:11.822 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:01:11 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:01:11.526+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:34.002793+0000 front 2024-04-26T22:00:34.003098+0000 (oldest deadline 2024-04-26T22:00:58.103086+0000) 2024-04-26T22:01:11.822 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:01:11 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:01:11.526+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-26T22:00:41.004997+0000 front 2024-04-26T22:00:41.005270+0000 (oldest deadline 2024-04-26T22:01:08.006242+0000) 2024-04-26T22:01:12.064 INFO:journalctl@ceph.osd.7.smithi076.stdout:Apr 26 22:01:11 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-7[49518]: 2024-04-26T22:01:11.775+0000 7fcc04a00700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:21.667782+0000 front 2024-04-26T22:00:21.668606+0000 (oldest deadline 2024-04-26T22:00:46.967821+0000) 2024-04-26T22:01:12.064 INFO:journalctl@ceph.osd.7.smithi076.stdout:Apr 26 22:01:11 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-7[49518]: 2024-04-26T22:01:11.775+0000 7fcc04a00700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:34.571664+0000 front 2024-04-26T22:00:34.572066+0000 (oldest deadline 2024-04-26T22:00:57.471976+0000) 2024-04-26T22:01:12.065 INFO:journalctl@ceph.osd.7.smithi076.stdout:Apr 26 22:01:11 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-7[49518]: 2024-04-26T22:01:11.775+0000 7fcc04a00700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-26T22:00:41.474549+0000 front 2024-04-26T22:00:41.475097+0000 (oldest deadline 2024-04-26T22:01:05.574631+0000) 2024-04-26T22:01:12.065 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:11 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:11.848+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:11.890629+0000 front 2024-04-26T22:00:11.890849+0000 (oldest deadline 2024-04-26T22:00:40.692043+0000) 2024-04-26T22:01:12.065 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:11 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:11.848+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:24.194239+0000 front 2024-04-26T22:00:24.194095+0000 (oldest deadline 2024-04-26T22:00:48.893996+0000) 2024-04-26T22:01:12.065 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:11 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:11.848+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:34.096788+0000 front 2024-04-26T22:00:34.096884+0000 (oldest deadline 2024-04-26T22:00:56.396862+0000) 2024-04-26T22:01:12.065 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:11 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:11.848+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-26T22:00:42.299288+0000 front 2024-04-26T22:00:42.299033+0000 (oldest deadline 2024-04-26T22:01:06.398950+0000) 2024-04-26T22:01:12.500 INFO:journalctl@ceph.osd.6.smithi072.stdout:Apr 26 22:01:12 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-6[57091]: 2024-04-26T22:01:12.181+0000 7fb154800700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:16.543358+0000 front 2024-04-26T22:00:16.543710+0000 (oldest deadline 2024-04-26T22:00:41.744798+0000) 2024-04-26T22:01:12.501 INFO:journalctl@ceph.osd.6.smithi072.stdout:Apr 26 22:01:12 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-6[57091]: 2024-04-26T22:01:12.181+0000 7fb154800700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:21.746250+0000 front 2024-04-26T22:00:21.745942+0000 (oldest deadline 2024-04-26T22:00:47.645522+0000) 2024-04-26T22:01:12.501 INFO:journalctl@ceph.osd.6.smithi072.stdout:Apr 26 22:01:12 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-6[57091]: 2024-04-26T22:01:12.181+0000 7fb154800700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:31.649534+0000 front 2024-04-26T22:00:31.649342+0000 (oldest deadline 2024-04-26T22:00:56.949491+0000) 2024-04-26T22:01:12.501 INFO:journalctl@ceph.osd.6.smithi072.stdout:Apr 26 22:01:12 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-6[57091]: 2024-04-26T22:01:12.181+0000 7fb154800700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-26T22:00:42.851796+0000 front 2024-04-26T22:00:42.852813+0000 (oldest deadline 2024-04-26T22:01:06.351702+0000) 2024-04-26T22:01:12.690 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 26 22:01:12 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-8[50342]: 2024-04-26T22:01:12.274+0000 7f152e400700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:25.173410+0000 front 2024-04-26T22:00:25.173841+0000 (oldest deadline 2024-04-26T22:00:50.473747+0000) 2024-04-26T22:01:12.691 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 26 22:01:12 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-8[50342]: 2024-04-26T22:01:12.274+0000 7f152e400700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:30.474905+0000 front 2024-04-26T22:00:30.475573+0000 (oldest deadline 2024-04-26T22:00:56.375141+0000) 2024-04-26T22:01:12.691 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 26 22:01:12 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-8[50342]: 2024-04-26T22:01:12.274+0000 7f152e400700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-26T22:00:43.977887+0000 front 2024-04-26T22:00:43.978065+0000 (oldest deadline 2024-04-26T22:01:06.277746+0000) 2024-04-26T22:01:12.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:12 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:12.296+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:14.343204+0000 front 2024-04-26T22:00:14.342815+0000 (oldest deadline 2024-04-26T22:00:39.042910+0000) 2024-04-26T22:01:12.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:12 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:12.296+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:24.945823+0000 front 2024-04-26T22:00:24.945339+0000 (oldest deadline 2024-04-26T22:00:47.245223+0000) 2024-04-26T22:01:12.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:12 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:12.296+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:31.947744+0000 front 2024-04-26T22:00:31.948080+0000 (oldest deadline 2024-04-26T22:00:56.647838+0000) 2024-04-26T22:01:12.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:12 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:12.296+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-26T22:00:42.953004+0000 front 2024-04-26T22:00:42.952908+0000 (oldest deadline 2024-04-26T22:01:07.652653+0000) 2024-04-26T22:01:12.822 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:01:12 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:01:12.499+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:16.595921+0000 front 2024-04-26T22:00:16.596275+0000 (oldest deadline 2024-04-26T22:00:39.495420+0000) 2024-04-26T22:01:12.822 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:01:12 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:01:12.499+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:24.600976+0000 front 2024-04-26T22:00:24.600553+0000 (oldest deadline 2024-04-26T22:00:48.100712+0000) 2024-04-26T22:01:12.822 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:01:12 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:01:12.499+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:34.002793+0000 front 2024-04-26T22:00:34.003098+0000 (oldest deadline 2024-04-26T22:00:58.103086+0000) 2024-04-26T22:01:12.822 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:01:12 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:01:12.499+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-26T22:00:41.004997+0000 front 2024-04-26T22:00:41.005270+0000 (oldest deadline 2024-04-26T22:01:08.006242+0000) 2024-04-26T22:01:13.064 INFO:journalctl@ceph.osd.7.smithi076.stdout:Apr 26 22:01:12 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-7[49518]: 2024-04-26T22:01:12.821+0000 7fcc04a00700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:21.667782+0000 front 2024-04-26T22:00:21.668606+0000 (oldest deadline 2024-04-26T22:00:46.967821+0000) 2024-04-26T22:01:13.065 INFO:journalctl@ceph.osd.7.smithi076.stdout:Apr 26 22:01:12 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-7[49518]: 2024-04-26T22:01:12.821+0000 7fcc04a00700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:34.571664+0000 front 2024-04-26T22:00:34.572066+0000 (oldest deadline 2024-04-26T22:00:57.471976+0000) 2024-04-26T22:01:13.065 INFO:journalctl@ceph.osd.7.smithi076.stdout:Apr 26 22:01:12 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-7[49518]: 2024-04-26T22:01:12.821+0000 7fcc04a00700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-26T22:00:41.474549+0000 front 2024-04-26T22:00:41.475097+0000 (oldest deadline 2024-04-26T22:01:05.574631+0000) 2024-04-26T22:01:13.065 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:12 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:12.804+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:11.890629+0000 front 2024-04-26T22:00:11.890849+0000 (oldest deadline 2024-04-26T22:00:40.692043+0000) 2024-04-26T22:01:13.065 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:12 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:12.804+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:24.194239+0000 front 2024-04-26T22:00:24.194095+0000 (oldest deadline 2024-04-26T22:00:48.893996+0000) 2024-04-26T22:01:13.065 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:12 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:12.804+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:34.096788+0000 front 2024-04-26T22:00:34.096884+0000 (oldest deadline 2024-04-26T22:00:56.396862+0000) 2024-04-26T22:01:13.065 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:12 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:12.804+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-26T22:00:42.299288+0000 front 2024-04-26T22:00:42.299033+0000 (oldest deadline 2024-04-26T22:01:06.398950+0000) 2024-04-26T22:01:13.612 INFO:journalctl@ceph.osd.6.smithi072.stdout:Apr 26 22:01:13 smithi072 podman[89152]: 2024-04-26 22:01:13.336991179 +0000 UTC m=+5.744427354 container died 617ccd1509f1b02b71a8c517f5428b9f731ee6f337b046a436ca089463026a3b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-6, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-04-26T22:01:13.613 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:01:13 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:01:13.453+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:16.595921+0000 front 2024-04-26T22:00:16.596275+0000 (oldest deadline 2024-04-26T22:00:39.495420+0000) 2024-04-26T22:01:13.613 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:01:13 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:01:13.453+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:24.600976+0000 front 2024-04-26T22:00:24.600553+0000 (oldest deadline 2024-04-26T22:00:48.100712+0000) 2024-04-26T22:01:13.613 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:01:13 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:01:13.453+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:34.002793+0000 front 2024-04-26T22:00:34.003098+0000 (oldest deadline 2024-04-26T22:00:58.103086+0000) 2024-04-26T22:01:13.613 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:01:13 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:01:13.453+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-26T22:00:41.004997+0000 front 2024-04-26T22:00:41.005270+0000 (oldest deadline 2024-04-26T22:01:08.006242+0000) 2024-04-26T22:01:13.690 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 26 22:01:13 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-8[50342]: 2024-04-26T22:01:13.233+0000 7f152e400700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:25.173410+0000 front 2024-04-26T22:00:25.173841+0000 (oldest deadline 2024-04-26T22:00:50.473747+0000) 2024-04-26T22:01:13.690 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 26 22:01:13 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-8[50342]: 2024-04-26T22:01:13.233+0000 7f152e400700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:30.474905+0000 front 2024-04-26T22:00:30.475573+0000 (oldest deadline 2024-04-26T22:00:56.375141+0000) 2024-04-26T22:01:13.691 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 26 22:01:13 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-8[50342]: 2024-04-26T22:01:13.233+0000 7f152e400700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-26T22:00:43.977887+0000 front 2024-04-26T22:00:43.978065+0000 (oldest deadline 2024-04-26T22:01:06.277746+0000) 2024-04-26T22:01:13.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:13 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:13.299+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:14.343204+0000 front 2024-04-26T22:00:14.342815+0000 (oldest deadline 2024-04-26T22:00:39.042910+0000) 2024-04-26T22:01:13.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:13 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:13.299+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:24.945823+0000 front 2024-04-26T22:00:24.945339+0000 (oldest deadline 2024-04-26T22:00:47.245223+0000) 2024-04-26T22:01:13.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:13 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:13.299+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:31.947744+0000 front 2024-04-26T22:00:31.948080+0000 (oldest deadline 2024-04-26T22:00:56.647838+0000) 2024-04-26T22:01:13.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:13 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:13.299+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-26T22:00:42.953004+0000 front 2024-04-26T22:00:42.952908+0000 (oldest deadline 2024-04-26T22:01:07.652653+0000) 2024-04-26T22:01:13.960 INFO:journalctl@ceph.osd.6.smithi072.stdout:Apr 26 22:01:13 smithi072 podman[89152]: 2024-04-26 22:01:13.723383098 +0000 UTC m=+6.130819251 container cleanup 617ccd1509f1b02b71a8c517f5428b9f731ee6f337b046a436ca089463026a3b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-6, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T22:01:13.960 INFO:journalctl@ceph.osd.6.smithi072.stdout:Apr 26 22:01:13 smithi072 bash[89152]: ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-6 2024-04-26T22:01:14.064 INFO:journalctl@ceph.osd.7.smithi076.stdout:Apr 26 22:01:13 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-7[49518]: 2024-04-26T22:01:13.774+0000 7fcc04a00700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:21.667782+0000 front 2024-04-26T22:00:21.668606+0000 (oldest deadline 2024-04-26T22:00:46.967821+0000) 2024-04-26T22:01:14.064 INFO:journalctl@ceph.osd.7.smithi076.stdout:Apr 26 22:01:13 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-7[49518]: 2024-04-26T22:01:13.774+0000 7fcc04a00700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:34.571664+0000 front 2024-04-26T22:00:34.572066+0000 (oldest deadline 2024-04-26T22:00:57.471976+0000) 2024-04-26T22:01:14.064 INFO:journalctl@ceph.osd.7.smithi076.stdout:Apr 26 22:01:13 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-7[49518]: 2024-04-26T22:01:13.774+0000 7fcc04a00700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-26T22:00:41.474549+0000 front 2024-04-26T22:00:41.475097+0000 (oldest deadline 2024-04-26T22:01:05.574631+0000) 2024-04-26T22:01:14.065 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:13 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:13.774+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:11.890629+0000 front 2024-04-26T22:00:11.890849+0000 (oldest deadline 2024-04-26T22:00:40.692043+0000) 2024-04-26T22:01:14.065 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:13 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:13.774+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:24.194239+0000 front 2024-04-26T22:00:24.194095+0000 (oldest deadline 2024-04-26T22:00:48.893996+0000) 2024-04-26T22:01:14.065 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:13 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:13.774+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:34.096788+0000 front 2024-04-26T22:00:34.096884+0000 (oldest deadline 2024-04-26T22:00:56.396862+0000) 2024-04-26T22:01:14.065 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:13 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:13.774+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-26T22:00:42.299288+0000 front 2024-04-26T22:00:42.299033+0000 (oldest deadline 2024-04-26T22:01:06.398950+0000) 2024-04-26T22:01:14.321 INFO:journalctl@ceph.osd.6.smithi072.stdout:Apr 26 22:01:13 smithi072 podman[89184]: 2024-04-26 22:01:13.959130036 +0000 UTC m=+0.607400179 container remove 617ccd1509f1b02b71a8c517f5428b9f731ee6f337b046a436ca089463026a3b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-6, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0) 2024-04-26T22:01:14.690 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 26 22:01:14 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-8[50342]: 2024-04-26T22:01:14.246+0000 7f152e400700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:25.173410+0000 front 2024-04-26T22:00:25.173841+0000 (oldest deadline 2024-04-26T22:00:50.473747+0000) 2024-04-26T22:01:14.690 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 26 22:01:14 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-8[50342]: 2024-04-26T22:01:14.246+0000 7f152e400700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:30.474905+0000 front 2024-04-26T22:00:30.475573+0000 (oldest deadline 2024-04-26T22:00:56.375141+0000) 2024-04-26T22:01:14.691 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 26 22:01:14 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-8[50342]: 2024-04-26T22:01:14.246+0000 7f152e400700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-26T22:00:43.977887+0000 front 2024-04-26T22:00:43.978065+0000 (oldest deadline 2024-04-26T22:01:06.277746+0000) 2024-04-26T22:01:14.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:14 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:14.337+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:14.343204+0000 front 2024-04-26T22:00:14.342815+0000 (oldest deadline 2024-04-26T22:00:39.042910+0000) 2024-04-26T22:01:14.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:14 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:14.337+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:24.945823+0000 front 2024-04-26T22:00:24.945339+0000 (oldest deadline 2024-04-26T22:00:47.245223+0000) 2024-04-26T22:01:14.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:14 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:14.337+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:31.947744+0000 front 2024-04-26T22:00:31.948080+0000 (oldest deadline 2024-04-26T22:00:56.647838+0000) 2024-04-26T22:01:14.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:14 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:14.337+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-26T22:00:42.953004+0000 front 2024-04-26T22:00:42.952908+0000 (oldest deadline 2024-04-26T22:01:07.652653+0000) 2024-04-26T22:01:14.776 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:01:14 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:01:14.429+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:16.595921+0000 front 2024-04-26T22:00:16.596275+0000 (oldest deadline 2024-04-26T22:00:39.495420+0000) 2024-04-26T22:01:14.776 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:01:14 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:01:14.429+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:24.600976+0000 front 2024-04-26T22:00:24.600553+0000 (oldest deadline 2024-04-26T22:00:48.100712+0000) 2024-04-26T22:01:14.776 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:01:14 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:01:14.429+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:34.002793+0000 front 2024-04-26T22:00:34.003098+0000 (oldest deadline 2024-04-26T22:00:58.103086+0000) 2024-04-26T22:01:14.777 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:01:14 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:01:14.429+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-26T22:00:41.004997+0000 front 2024-04-26T22:00:41.005270+0000 (oldest deadline 2024-04-26T22:01:08.006242+0000) 2024-04-26T22:01:15.064 INFO:journalctl@ceph.osd.7.smithi076.stdout:Apr 26 22:01:14 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-7[49518]: 2024-04-26T22:01:14.791+0000 7fcc04a00700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:21.667782+0000 front 2024-04-26T22:00:21.668606+0000 (oldest deadline 2024-04-26T22:00:46.967821+0000) 2024-04-26T22:01:15.065 INFO:journalctl@ceph.osd.7.smithi076.stdout:Apr 26 22:01:14 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-7[49518]: 2024-04-26T22:01:14.791+0000 7fcc04a00700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:34.571664+0000 front 2024-04-26T22:00:34.572066+0000 (oldest deadline 2024-04-26T22:00:57.471976+0000) 2024-04-26T22:01:15.065 INFO:journalctl@ceph.osd.7.smithi076.stdout:Apr 26 22:01:14 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-7[49518]: 2024-04-26T22:01:14.791+0000 7fcc04a00700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-26T22:00:41.474549+0000 front 2024-04-26T22:00:41.475097+0000 (oldest deadline 2024-04-26T22:01:05.574631+0000) 2024-04-26T22:01:15.065 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:14 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:14.749+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:11.890629+0000 front 2024-04-26T22:00:11.890849+0000 (oldest deadline 2024-04-26T22:00:40.692043+0000) 2024-04-26T22:01:15.065 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:14 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:14.749+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:24.194239+0000 front 2024-04-26T22:00:24.194095+0000 (oldest deadline 2024-04-26T22:00:48.893996+0000) 2024-04-26T22:01:15.065 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:14 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:14.749+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:34.096788+0000 front 2024-04-26T22:00:34.096884+0000 (oldest deadline 2024-04-26T22:00:56.396862+0000) 2024-04-26T22:01:15.065 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:14 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:14.749+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-26T22:00:42.299288+0000 front 2024-04-26T22:00:42.299033+0000 (oldest deadline 2024-04-26T22:01:06.398950+0000) 2024-04-26T22:01:15.071 INFO:journalctl@ceph.osd.6.smithi072.stdout:Apr 26 22:01:14 smithi072 podman[89275]: 2024-04-26 22:01:14.682778754 +0000 UTC m=+0.033580759 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-26T22:01:15.072 INFO:journalctl@ceph.osd.6.smithi072.stdout:Apr 26 22:01:14 smithi072 podman[89275]: 2024-04-26 22:01:14.794107657 +0000 UTC m=+0.144909668 container create 1e7725195b834b53ef73bbc11df68c3b17930c6fd4814c484e5769c46ca80450 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-6-deactivate, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, io.buildah.version=1.33.5) 2024-04-26T22:01:15.451 INFO:journalctl@ceph.osd.6.smithi072.stdout:Apr 26 22:01:15 smithi072 podman[89275]: 2024-04-26 22:01:15.077721999 +0000 UTC m=+0.428524064 container init 1e7725195b834b53ef73bbc11df68c3b17930c6fd4814c484e5769c46ca80450 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-6-deactivate, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-04-26T22:01:15.451 INFO:journalctl@ceph.osd.6.smithi072.stdout:Apr 26 22:01:15 smithi072 podman[89275]: 2024-04-26 22:01:15.090061614 +0000 UTC m=+0.440863614 container start 1e7725195b834b53ef73bbc11df68c3b17930c6fd4814c484e5769c46ca80450 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-6-deactivate, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424) 2024-04-26T22:01:15.451 INFO:journalctl@ceph.osd.6.smithi072.stdout:Apr 26 22:01:15 smithi072 podman[89275]: 2024-04-26 22:01:15.241551111 +0000 UTC m=+0.592353121 container attach 1e7725195b834b53ef73bbc11df68c3b17930c6fd4814c484e5769c46ca80450 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-6-deactivate, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240424, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-04-26T22:01:15.690 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 26 22:01:15 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-8[50342]: 2024-04-26T22:01:15.228+0000 7f152e400700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:25.173410+0000 front 2024-04-26T22:00:25.173841+0000 (oldest deadline 2024-04-26T22:00:50.473747+0000) 2024-04-26T22:01:15.690 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 26 22:01:15 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-8[50342]: 2024-04-26T22:01:15.228+0000 7f152e400700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:30.474905+0000 front 2024-04-26T22:00:30.475573+0000 (oldest deadline 2024-04-26T22:00:56.375141+0000) 2024-04-26T22:01:15.691 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 26 22:01:15 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-8[50342]: 2024-04-26T22:01:15.228+0000 7f152e400700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-26T22:00:43.977887+0000 front 2024-04-26T22:00:43.978065+0000 (oldest deadline 2024-04-26T22:01:06.277746+0000) 2024-04-26T22:01:15.691 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 26 22:01:15 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-8[50342]: 2024-04-26T22:01:15.228+0000 7f152e400700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-04-26T22:00:52.581936+0000 front 2024-04-26T22:00:52.581907+0000 (oldest deadline 2024-04-26T22:01:14.881658+0000) 2024-04-26T22:01:15.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:15 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:15.294+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:14.343204+0000 front 2024-04-26T22:00:14.342815+0000 (oldest deadline 2024-04-26T22:00:39.042910+0000) 2024-04-26T22:01:15.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:15 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:15.294+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:24.945823+0000 front 2024-04-26T22:00:24.945339+0000 (oldest deadline 2024-04-26T22:00:47.245223+0000) 2024-04-26T22:01:15.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:15 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:15.294+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:31.947744+0000 front 2024-04-26T22:00:31.948080+0000 (oldest deadline 2024-04-26T22:00:56.647838+0000) 2024-04-26T22:01:15.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:15 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:15.294+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-26T22:00:42.953004+0000 front 2024-04-26T22:00:42.952908+0000 (oldest deadline 2024-04-26T22:01:07.652653+0000) 2024-04-26T22:01:15.822 INFO:journalctl@ceph.osd.6.smithi072.stdout:Apr 26 22:01:15 smithi072 podman[89275]: 2024-04-26 22:01:15.54474438 +0000 UTC m=+0.895546448 container died 1e7725195b834b53ef73bbc11df68c3b17930c6fd4814c484e5769c46ca80450 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-6-deactivate, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T22:01:15.823 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:01:15 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:01:15.450+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:16.595921+0000 front 2024-04-26T22:00:16.596275+0000 (oldest deadline 2024-04-26T22:00:39.495420+0000) 2024-04-26T22:01:15.823 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:01:15 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:01:15.450+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:24.600976+0000 front 2024-04-26T22:00:24.600553+0000 (oldest deadline 2024-04-26T22:00:48.100712+0000) 2024-04-26T22:01:15.823 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:01:15 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:01:15.450+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:34.002793+0000 front 2024-04-26T22:00:34.003098+0000 (oldest deadline 2024-04-26T22:00:58.103086+0000) 2024-04-26T22:01:15.823 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:01:15 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:01:15.450+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-26T22:00:41.004997+0000 front 2024-04-26T22:00:41.005270+0000 (oldest deadline 2024-04-26T22:01:08.006242+0000) 2024-04-26T22:01:16.064 INFO:journalctl@ceph.osd.7.smithi076.stdout:Apr 26 22:01:15 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-7[49518]: 2024-04-26T22:01:15.802+0000 7fcc04a00700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:21.667782+0000 front 2024-04-26T22:00:21.668606+0000 (oldest deadline 2024-04-26T22:00:46.967821+0000) 2024-04-26T22:01:16.064 INFO:journalctl@ceph.osd.7.smithi076.stdout:Apr 26 22:01:15 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-7[49518]: 2024-04-26T22:01:15.802+0000 7fcc04a00700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:34.571664+0000 front 2024-04-26T22:00:34.572066+0000 (oldest deadline 2024-04-26T22:00:57.471976+0000) 2024-04-26T22:01:16.065 INFO:journalctl@ceph.osd.7.smithi076.stdout:Apr 26 22:01:15 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-7[49518]: 2024-04-26T22:01:15.802+0000 7fcc04a00700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-26T22:00:41.474549+0000 front 2024-04-26T22:00:41.475097+0000 (oldest deadline 2024-04-26T22:01:05.574631+0000) 2024-04-26T22:01:16.065 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:15 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:15.766+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:11.890629+0000 front 2024-04-26T22:00:11.890849+0000 (oldest deadline 2024-04-26T22:00:40.692043+0000) 2024-04-26T22:01:16.065 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:15 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:15.766+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:24.194239+0000 front 2024-04-26T22:00:24.194095+0000 (oldest deadline 2024-04-26T22:00:48.893996+0000) 2024-04-26T22:01:16.065 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:15 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:15.766+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:34.096788+0000 front 2024-04-26T22:00:34.096884+0000 (oldest deadline 2024-04-26T22:00:56.396862+0000) 2024-04-26T22:01:16.065 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:15 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:15.766+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-26T22:00:42.299288+0000 front 2024-04-26T22:00:42.299033+0000 (oldest deadline 2024-04-26T22:01:06.398950+0000) 2024-04-26T22:01:16.268 DEBUG:teuthology.orchestra.run.smithi072:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a1780950-0415-11ef-bc93-c7b262605968@osd.6.service' 2024-04-26T22:01:16.376 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T22:01:16.376 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-04-26T22:01:16.377 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-04-26T22:01:16.377 DEBUG:teuthology.orchestra.run.smithi076:> sudo systemctl stop ceph-a1780950-0415-11ef-bc93-c7b262605968@osd.7 2024-04-26T22:01:16.440 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:16 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:16.298+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:14.343204+0000 front 2024-04-26T22:00:14.342815+0000 (oldest deadline 2024-04-26T22:00:39.042910+0000) 2024-04-26T22:01:16.440 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:16 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:16.298+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:24.945823+0000 front 2024-04-26T22:00:24.945339+0000 (oldest deadline 2024-04-26T22:00:47.245223+0000) 2024-04-26T22:01:16.441 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:16 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:16.298+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:31.947744+0000 front 2024-04-26T22:00:31.948080+0000 (oldest deadline 2024-04-26T22:00:56.647838+0000) 2024-04-26T22:01:16.441 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:16 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:16.298+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-26T22:00:42.953004+0000 front 2024-04-26T22:00:42.952908+0000 (oldest deadline 2024-04-26T22:01:07.652653+0000) 2024-04-26T22:01:16.441 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 26 22:01:16 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-8[50342]: 2024-04-26T22:01:16.185+0000 7f152e400700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:25.173410+0000 front 2024-04-26T22:00:25.173841+0000 (oldest deadline 2024-04-26T22:00:50.473747+0000) 2024-04-26T22:01:16.441 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 26 22:01:16 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-8[50342]: 2024-04-26T22:01:16.185+0000 7f152e400700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:30.474905+0000 front 2024-04-26T22:00:30.475573+0000 (oldest deadline 2024-04-26T22:00:56.375141+0000) 2024-04-26T22:01:16.441 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 26 22:01:16 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-8[50342]: 2024-04-26T22:01:16.185+0000 7f152e400700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-26T22:00:43.977887+0000 front 2024-04-26T22:00:43.978065+0000 (oldest deadline 2024-04-26T22:01:06.277746+0000) 2024-04-26T22:01:16.441 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 26 22:01:16 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-8[50342]: 2024-04-26T22:01:16.185+0000 7f152e400700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-04-26T22:00:52.581936+0000 front 2024-04-26T22:00:52.581907+0000 (oldest deadline 2024-04-26T22:01:14.881658+0000) 2024-04-26T22:01:16.746 INFO:journalctl@ceph.osd.7.smithi076.stdout:Apr 26 22:01:16 smithi076 systemd[1]: Stopping Ceph osd.7 for a1780950-0415-11ef-bc93-c7b262605968... 2024-04-26T22:01:16.822 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:01:16 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:01:16.487+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:16.595921+0000 front 2024-04-26T22:00:16.596275+0000 (oldest deadline 2024-04-26T22:00:39.495420+0000) 2024-04-26T22:01:16.822 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:01:16 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:01:16.487+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:24.600976+0000 front 2024-04-26T22:00:24.600553+0000 (oldest deadline 2024-04-26T22:00:48.100712+0000) 2024-04-26T22:01:16.822 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:01:16 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:01:16.487+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:34.002793+0000 front 2024-04-26T22:00:34.003098+0000 (oldest deadline 2024-04-26T22:00:58.103086+0000) 2024-04-26T22:01:16.822 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:01:16 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:01:16.487+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-26T22:00:41.004997+0000 front 2024-04-26T22:00:41.005270+0000 (oldest deadline 2024-04-26T22:01:08.006242+0000) 2024-04-26T22:01:17.064 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:16 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:16.746+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:11.890629+0000 front 2024-04-26T22:00:11.890849+0000 (oldest deadline 2024-04-26T22:00:40.692043+0000) 2024-04-26T22:01:17.064 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:16 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:16.746+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:24.194239+0000 front 2024-04-26T22:00:24.194095+0000 (oldest deadline 2024-04-26T22:00:48.893996+0000) 2024-04-26T22:01:17.064 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:16 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:16.746+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:34.096788+0000 front 2024-04-26T22:00:34.096884+0000 (oldest deadline 2024-04-26T22:00:56.396862+0000) 2024-04-26T22:01:17.065 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:16 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:16.746+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-26T22:00:42.299288+0000 front 2024-04-26T22:00:42.299033+0000 (oldest deadline 2024-04-26T22:01:06.398950+0000) 2024-04-26T22:01:17.065 INFO:journalctl@ceph.osd.7.smithi076.stdout:Apr 26 22:01:16 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-7[49518]: 2024-04-26T22:01:16.789+0000 7fcc04a00700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:21.667782+0000 front 2024-04-26T22:00:21.668606+0000 (oldest deadline 2024-04-26T22:00:46.967821+0000) 2024-04-26T22:01:17.065 INFO:journalctl@ceph.osd.7.smithi076.stdout:Apr 26 22:01:16 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-7[49518]: 2024-04-26T22:01:16.789+0000 7fcc04a00700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:34.571664+0000 front 2024-04-26T22:00:34.572066+0000 (oldest deadline 2024-04-26T22:00:57.471976+0000) 2024-04-26T22:01:17.065 INFO:journalctl@ceph.osd.7.smithi076.stdout:Apr 26 22:01:16 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-7[49518]: 2024-04-26T22:01:16.789+0000 7fcc04a00700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-26T22:00:41.474549+0000 front 2024-04-26T22:00:41.475097+0000 (oldest deadline 2024-04-26T22:01:05.574631+0000) 2024-04-26T22:01:17.065 INFO:journalctl@ceph.osd.7.smithi076.stdout:Apr 26 22:01:16 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-7[49518]: 2024-04-26T22:01:16.789+0000 7fcc04a00700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-04-26T22:00:51.476654+0000 front 2024-04-26T22:00:51.476706+0000 (oldest deadline 2024-04-26T22:01:16.776822+0000) 2024-04-26T22:01:17.065 INFO:journalctl@ceph.osd.7.smithi076.stdout:Apr 26 22:01:16 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-7[49518]: 2024-04-26T22:01:16.927+0000 7fcc09200700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T22:01:17.065 INFO:journalctl@ceph.osd.7.smithi076.stdout:Apr 26 22:01:16 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-7[49518]: 2024-04-26T22:01:16.927+0000 7fcc09200700 -1 osd.7 94 *** Got signal Terminated *** 2024-04-26T22:01:17.065 INFO:journalctl@ceph.osd.7.smithi076.stdout:Apr 26 22:01:16 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-7[49518]: 2024-04-26T22:01:16.927+0000 7fcc09200700 -1 osd.7 94 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-26T22:01:17.691 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 26 22:01:17 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-8[50342]: 2024-04-26T22:01:17.206+0000 7f152e400700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:25.173410+0000 front 2024-04-26T22:00:25.173841+0000 (oldest deadline 2024-04-26T22:00:50.473747+0000) 2024-04-26T22:01:17.691 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 26 22:01:17 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-8[50342]: 2024-04-26T22:01:17.206+0000 7f152e400700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:30.474905+0000 front 2024-04-26T22:00:30.475573+0000 (oldest deadline 2024-04-26T22:00:56.375141+0000) 2024-04-26T22:01:17.691 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 26 22:01:17 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-8[50342]: 2024-04-26T22:01:17.206+0000 7f152e400700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-26T22:00:43.977887+0000 front 2024-04-26T22:00:43.978065+0000 (oldest deadline 2024-04-26T22:01:06.277746+0000) 2024-04-26T22:01:17.691 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 26 22:01:17 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-8[50342]: 2024-04-26T22:01:17.206+0000 7f152e400700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-04-26T22:00:52.581936+0000 front 2024-04-26T22:00:52.581907+0000 (oldest deadline 2024-04-26T22:01:14.881658+0000) 2024-04-26T22:01:17.692 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:17 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:17.326+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:14.343204+0000 front 2024-04-26T22:00:14.342815+0000 (oldest deadline 2024-04-26T22:00:39.042910+0000) 2024-04-26T22:01:17.692 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:17 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:17.326+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:24.945823+0000 front 2024-04-26T22:00:24.945339+0000 (oldest deadline 2024-04-26T22:00:47.245223+0000) 2024-04-26T22:01:17.692 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:17 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:17.326+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:31.947744+0000 front 2024-04-26T22:00:31.948080+0000 (oldest deadline 2024-04-26T22:00:56.647838+0000) 2024-04-26T22:01:17.692 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:17 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:17.326+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-26T22:00:42.953004+0000 front 2024-04-26T22:00:42.952908+0000 (oldest deadline 2024-04-26T22:01:07.652653+0000) 2024-04-26T22:01:17.822 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:01:17 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:01:17.516+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:16.595921+0000 front 2024-04-26T22:00:16.596275+0000 (oldest deadline 2024-04-26T22:00:39.495420+0000) 2024-04-26T22:01:17.822 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:01:17 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:01:17.516+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:24.600976+0000 front 2024-04-26T22:00:24.600553+0000 (oldest deadline 2024-04-26T22:00:48.100712+0000) 2024-04-26T22:01:17.822 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:01:17 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:01:17.516+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:34.002793+0000 front 2024-04-26T22:00:34.003098+0000 (oldest deadline 2024-04-26T22:00:58.103086+0000) 2024-04-26T22:01:17.822 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:01:17 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:01:17.516+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-26T22:00:41.004997+0000 front 2024-04-26T22:00:41.005270+0000 (oldest deadline 2024-04-26T22:01:08.006242+0000) 2024-04-26T22:01:18.065 INFO:journalctl@ceph.osd.7.smithi076.stdout:Apr 26 22:01:17 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-7[49518]: 2024-04-26T22:01:17.820+0000 7fcc04a00700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:21.667782+0000 front 2024-04-26T22:00:21.668606+0000 (oldest deadline 2024-04-26T22:00:46.967821+0000) 2024-04-26T22:01:18.065 INFO:journalctl@ceph.osd.7.smithi076.stdout:Apr 26 22:01:17 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-7[49518]: 2024-04-26T22:01:17.820+0000 7fcc04a00700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:34.571664+0000 front 2024-04-26T22:00:34.572066+0000 (oldest deadline 2024-04-26T22:00:57.471976+0000) 2024-04-26T22:01:18.065 INFO:journalctl@ceph.osd.7.smithi076.stdout:Apr 26 22:01:17 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-7[49518]: 2024-04-26T22:01:17.820+0000 7fcc04a00700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-26T22:00:41.474549+0000 front 2024-04-26T22:00:41.475097+0000 (oldest deadline 2024-04-26T22:01:05.574631+0000) 2024-04-26T22:01:18.065 INFO:journalctl@ceph.osd.7.smithi076.stdout:Apr 26 22:01:17 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-7[49518]: 2024-04-26T22:01:17.820+0000 7fcc04a00700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-04-26T22:00:51.476654+0000 front 2024-04-26T22:00:51.476706+0000 (oldest deadline 2024-04-26T22:01:16.776822+0000) 2024-04-26T22:01:18.065 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:17 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:17.747+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:11.890629+0000 front 2024-04-26T22:00:11.890849+0000 (oldest deadline 2024-04-26T22:00:40.692043+0000) 2024-04-26T22:01:18.065 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:17 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:17.747+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:24.194239+0000 front 2024-04-26T22:00:24.194095+0000 (oldest deadline 2024-04-26T22:00:48.893996+0000) 2024-04-26T22:01:18.065 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:17 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:17.747+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:34.096788+0000 front 2024-04-26T22:00:34.096884+0000 (oldest deadline 2024-04-26T22:00:56.396862+0000) 2024-04-26T22:01:18.066 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:17 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:17.747+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-26T22:00:42.299288+0000 front 2024-04-26T22:00:42.299033+0000 (oldest deadline 2024-04-26T22:01:06.398950+0000) 2024-04-26T22:01:18.440 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 26 22:01:18 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-8[50342]: 2024-04-26T22:01:18.170+0000 7f152e400700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:25.173410+0000 front 2024-04-26T22:00:25.173841+0000 (oldest deadline 2024-04-26T22:00:50.473747+0000) 2024-04-26T22:01:18.441 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 26 22:01:18 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-8[50342]: 2024-04-26T22:01:18.170+0000 7f152e400700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:30.474905+0000 front 2024-04-26T22:00:30.475573+0000 (oldest deadline 2024-04-26T22:00:56.375141+0000) 2024-04-26T22:01:18.441 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 26 22:01:18 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-8[50342]: 2024-04-26T22:01:18.170+0000 7f152e400700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-26T22:00:43.977887+0000 front 2024-04-26T22:00:43.978065+0000 (oldest deadline 2024-04-26T22:01:06.277746+0000) 2024-04-26T22:01:18.441 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 26 22:01:18 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-8[50342]: 2024-04-26T22:01:18.170+0000 7f152e400700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-04-26T22:00:52.581936+0000 front 2024-04-26T22:00:52.581907+0000 (oldest deadline 2024-04-26T22:01:14.881658+0000) 2024-04-26T22:01:18.441 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:18 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:18.300+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:14.343204+0000 front 2024-04-26T22:00:14.342815+0000 (oldest deadline 2024-04-26T22:00:39.042910+0000) 2024-04-26T22:01:18.441 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:18 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:18.300+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:24.945823+0000 front 2024-04-26T22:00:24.945339+0000 (oldest deadline 2024-04-26T22:00:47.245223+0000) 2024-04-26T22:01:18.441 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:18 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:18.300+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:31.947744+0000 front 2024-04-26T22:00:31.948080+0000 (oldest deadline 2024-04-26T22:00:56.647838+0000) 2024-04-26T22:01:18.441 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:18 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:18.300+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-26T22:00:42.953004+0000 front 2024-04-26T22:00:42.952908+0000 (oldest deadline 2024-04-26T22:01:07.652653+0000) 2024-04-26T22:01:18.821 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:01:18 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:01:18.539+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:16.595921+0000 front 2024-04-26T22:00:16.596275+0000 (oldest deadline 2024-04-26T22:00:39.495420+0000) 2024-04-26T22:01:18.822 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:01:18 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:01:18.539+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:24.600976+0000 front 2024-04-26T22:00:24.600553+0000 (oldest deadline 2024-04-26T22:00:48.100712+0000) 2024-04-26T22:01:18.822 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:01:18 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:01:18.539+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:34.002793+0000 front 2024-04-26T22:00:34.003098+0000 (oldest deadline 2024-04-26T22:00:58.103086+0000) 2024-04-26T22:01:18.822 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:01:18 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:01:18.539+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-26T22:00:41.004997+0000 front 2024-04-26T22:00:41.005270+0000 (oldest deadline 2024-04-26T22:01:08.006242+0000) 2024-04-26T22:01:19.065 INFO:journalctl@ceph.osd.7.smithi076.stdout:Apr 26 22:01:18 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-7[49518]: 2024-04-26T22:01:18.833+0000 7fcc04a00700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:21.667782+0000 front 2024-04-26T22:00:21.668606+0000 (oldest deadline 2024-04-26T22:00:46.967821+0000) 2024-04-26T22:01:19.065 INFO:journalctl@ceph.osd.7.smithi076.stdout:Apr 26 22:01:18 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-7[49518]: 2024-04-26T22:01:18.833+0000 7fcc04a00700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:34.571664+0000 front 2024-04-26T22:00:34.572066+0000 (oldest deadline 2024-04-26T22:00:57.471976+0000) 2024-04-26T22:01:19.065 INFO:journalctl@ceph.osd.7.smithi076.stdout:Apr 26 22:01:18 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-7[49518]: 2024-04-26T22:01:18.833+0000 7fcc04a00700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-26T22:00:41.474549+0000 front 2024-04-26T22:00:41.475097+0000 (oldest deadline 2024-04-26T22:01:05.574631+0000) 2024-04-26T22:01:19.065 INFO:journalctl@ceph.osd.7.smithi076.stdout:Apr 26 22:01:18 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-7[49518]: 2024-04-26T22:01:18.833+0000 7fcc04a00700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-04-26T22:00:51.476654+0000 front 2024-04-26T22:00:51.476706+0000 (oldest deadline 2024-04-26T22:01:16.776822+0000) 2024-04-26T22:01:19.065 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:18 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:18.734+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:11.890629+0000 front 2024-04-26T22:00:11.890849+0000 (oldest deadline 2024-04-26T22:00:40.692043+0000) 2024-04-26T22:01:19.065 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:18 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:18.734+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:24.194239+0000 front 2024-04-26T22:00:24.194095+0000 (oldest deadline 2024-04-26T22:00:48.893996+0000) 2024-04-26T22:01:19.065 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:18 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:18.734+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:34.096788+0000 front 2024-04-26T22:00:34.096884+0000 (oldest deadline 2024-04-26T22:00:56.396862+0000) 2024-04-26T22:01:19.065 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:18 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:18.734+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-26T22:00:42.299288+0000 front 2024-04-26T22:00:42.299033+0000 (oldest deadline 2024-04-26T22:01:06.398950+0000) 2024-04-26T22:01:19.066 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:18 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:18.734+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-04-26T22:00:51.700557+0000 front 2024-04-26T22:00:51.700671+0000 (oldest deadline 2024-04-26T22:01:18.701605+0000) 2024-04-26T22:01:19.690 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 26 22:01:19 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-8[50342]: 2024-04-26T22:01:19.206+0000 7f152e400700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:25.173410+0000 front 2024-04-26T22:00:25.173841+0000 (oldest deadline 2024-04-26T22:00:50.473747+0000) 2024-04-26T22:01:19.691 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 26 22:01:19 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-8[50342]: 2024-04-26T22:01:19.206+0000 7f152e400700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:30.474905+0000 front 2024-04-26T22:00:30.475573+0000 (oldest deadline 2024-04-26T22:00:56.375141+0000) 2024-04-26T22:01:19.691 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 26 22:01:19 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-8[50342]: 2024-04-26T22:01:19.206+0000 7f152e400700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-26T22:00:43.977887+0000 front 2024-04-26T22:00:43.978065+0000 (oldest deadline 2024-04-26T22:01:06.277746+0000) 2024-04-26T22:01:19.691 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 26 22:01:19 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-8[50342]: 2024-04-26T22:01:19.206+0000 7f152e400700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-04-26T22:00:52.581936+0000 front 2024-04-26T22:00:52.581907+0000 (oldest deadline 2024-04-26T22:01:14.881658+0000) 2024-04-26T22:01:19.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:19 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:19.281+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:14.343204+0000 front 2024-04-26T22:00:14.342815+0000 (oldest deadline 2024-04-26T22:00:39.042910+0000) 2024-04-26T22:01:19.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:19 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:19.281+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:24.945823+0000 front 2024-04-26T22:00:24.945339+0000 (oldest deadline 2024-04-26T22:00:47.245223+0000) 2024-04-26T22:01:19.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:19 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:19.281+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:31.947744+0000 front 2024-04-26T22:00:31.948080+0000 (oldest deadline 2024-04-26T22:00:56.647838+0000) 2024-04-26T22:01:19.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:19 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:19.281+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-26T22:00:42.953004+0000 front 2024-04-26T22:00:42.952908+0000 (oldest deadline 2024-04-26T22:01:07.652653+0000) 2024-04-26T22:01:19.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:19 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:19.281+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-04-26T22:00:53.256824+0000 front 2024-04-26T22:00:53.256989+0000 (oldest deadline 2024-04-26T22:01:18.556986+0000) 2024-04-26T22:01:19.821 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:01:19 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:01:19.511+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:16.595921+0000 front 2024-04-26T22:00:16.596275+0000 (oldest deadline 2024-04-26T22:00:39.495420+0000) 2024-04-26T22:01:19.822 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:01:19 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:01:19.511+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:24.600976+0000 front 2024-04-26T22:00:24.600553+0000 (oldest deadline 2024-04-26T22:00:48.100712+0000) 2024-04-26T22:01:19.822 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:01:19 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:01:19.511+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:34.002793+0000 front 2024-04-26T22:00:34.003098+0000 (oldest deadline 2024-04-26T22:00:58.103086+0000) 2024-04-26T22:01:19.822 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:01:19 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:01:19.511+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-26T22:00:41.004997+0000 front 2024-04-26T22:00:41.005270+0000 (oldest deadline 2024-04-26T22:01:08.006242+0000) 2024-04-26T22:01:20.064 INFO:journalctl@ceph.osd.7.smithi076.stdout:Apr 26 22:01:19 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-7[49518]: 2024-04-26T22:01:19.870+0000 7fcc04a00700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:21.667782+0000 front 2024-04-26T22:00:21.668606+0000 (oldest deadline 2024-04-26T22:00:46.967821+0000) 2024-04-26T22:01:20.065 INFO:journalctl@ceph.osd.7.smithi076.stdout:Apr 26 22:01:19 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-7[49518]: 2024-04-26T22:01:19.870+0000 7fcc04a00700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:34.571664+0000 front 2024-04-26T22:00:34.572066+0000 (oldest deadline 2024-04-26T22:00:57.471976+0000) 2024-04-26T22:01:20.065 INFO:journalctl@ceph.osd.7.smithi076.stdout:Apr 26 22:01:19 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-7[49518]: 2024-04-26T22:01:19.870+0000 7fcc04a00700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-26T22:00:41.474549+0000 front 2024-04-26T22:00:41.475097+0000 (oldest deadline 2024-04-26T22:01:05.574631+0000) 2024-04-26T22:01:20.065 INFO:journalctl@ceph.osd.7.smithi076.stdout:Apr 26 22:01:19 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-7[49518]: 2024-04-26T22:01:19.870+0000 7fcc04a00700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-04-26T22:00:51.476654+0000 front 2024-04-26T22:00:51.476706+0000 (oldest deadline 2024-04-26T22:01:16.776822+0000) 2024-04-26T22:01:20.065 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:19 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:19.699+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:11.890629+0000 front 2024-04-26T22:00:11.890849+0000 (oldest deadline 2024-04-26T22:00:40.692043+0000) 2024-04-26T22:01:20.065 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:19 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:19.699+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:24.194239+0000 front 2024-04-26T22:00:24.194095+0000 (oldest deadline 2024-04-26T22:00:48.893996+0000) 2024-04-26T22:01:20.065 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:19 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:19.699+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:34.096788+0000 front 2024-04-26T22:00:34.096884+0000 (oldest deadline 2024-04-26T22:00:56.396862+0000) 2024-04-26T22:01:20.065 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:19 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:19.699+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-26T22:00:42.299288+0000 front 2024-04-26T22:00:42.299033+0000 (oldest deadline 2024-04-26T22:01:06.398950+0000) 2024-04-26T22:01:20.065 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:19 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:19.699+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-04-26T22:00:51.700557+0000 front 2024-04-26T22:00:51.700671+0000 (oldest deadline 2024-04-26T22:01:18.701605+0000) 2024-04-26T22:01:20.690 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 26 22:01:20 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-8[50342]: 2024-04-26T22:01:20.235+0000 7f152e400700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:25.173410+0000 front 2024-04-26T22:00:25.173841+0000 (oldest deadline 2024-04-26T22:00:50.473747+0000) 2024-04-26T22:01:20.691 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 26 22:01:20 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-8[50342]: 2024-04-26T22:01:20.235+0000 7f152e400700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:30.474905+0000 front 2024-04-26T22:00:30.475573+0000 (oldest deadline 2024-04-26T22:00:56.375141+0000) 2024-04-26T22:01:20.691 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 26 22:01:20 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-8[50342]: 2024-04-26T22:01:20.235+0000 7f152e400700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-26T22:00:43.977887+0000 front 2024-04-26T22:00:43.978065+0000 (oldest deadline 2024-04-26T22:01:06.277746+0000) 2024-04-26T22:01:20.691 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 26 22:01:20 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-8[50342]: 2024-04-26T22:01:20.235+0000 7f152e400700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-04-26T22:00:52.581936+0000 front 2024-04-26T22:00:52.581907+0000 (oldest deadline 2024-04-26T22:01:14.881658+0000) 2024-04-26T22:01:20.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:20 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:20.288+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:14.343204+0000 front 2024-04-26T22:00:14.342815+0000 (oldest deadline 2024-04-26T22:00:39.042910+0000) 2024-04-26T22:01:20.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:20 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:20.288+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:24.945823+0000 front 2024-04-26T22:00:24.945339+0000 (oldest deadline 2024-04-26T22:00:47.245223+0000) 2024-04-26T22:01:20.692 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:20 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:20.288+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:31.947744+0000 front 2024-04-26T22:00:31.948080+0000 (oldest deadline 2024-04-26T22:00:56.647838+0000) 2024-04-26T22:01:20.692 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:20 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:20.288+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-26T22:00:42.953004+0000 front 2024-04-26T22:00:42.952908+0000 (oldest deadline 2024-04-26T22:01:07.652653+0000) 2024-04-26T22:01:20.692 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:20 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:20.288+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-04-26T22:00:53.256824+0000 front 2024-04-26T22:00:53.256989+0000 (oldest deadline 2024-04-26T22:01:18.556986+0000) 2024-04-26T22:01:20.822 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:01:20 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:01:20.513+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:16.595921+0000 front 2024-04-26T22:00:16.596275+0000 (oldest deadline 2024-04-26T22:00:39.495420+0000) 2024-04-26T22:01:20.822 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:01:20 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:01:20.513+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:24.600976+0000 front 2024-04-26T22:00:24.600553+0000 (oldest deadline 2024-04-26T22:00:48.100712+0000) 2024-04-26T22:01:20.822 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:01:20 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:01:20.513+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:34.002793+0000 front 2024-04-26T22:00:34.003098+0000 (oldest deadline 2024-04-26T22:00:58.103086+0000) 2024-04-26T22:01:20.822 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:01:20 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:01:20.513+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-26T22:00:41.004997+0000 front 2024-04-26T22:00:41.005270+0000 (oldest deadline 2024-04-26T22:01:08.006242+0000) 2024-04-26T22:01:21.065 INFO:journalctl@ceph.osd.7.smithi076.stdout:Apr 26 22:01:20 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-7[49518]: 2024-04-26T22:01:20.889+0000 7fcc04a00700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:21.667782+0000 front 2024-04-26T22:00:21.668606+0000 (oldest deadline 2024-04-26T22:00:46.967821+0000) 2024-04-26T22:01:21.065 INFO:journalctl@ceph.osd.7.smithi076.stdout:Apr 26 22:01:20 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-7[49518]: 2024-04-26T22:01:20.889+0000 7fcc04a00700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:34.571664+0000 front 2024-04-26T22:00:34.572066+0000 (oldest deadline 2024-04-26T22:00:57.471976+0000) 2024-04-26T22:01:21.065 INFO:journalctl@ceph.osd.7.smithi076.stdout:Apr 26 22:01:20 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-7[49518]: 2024-04-26T22:01:20.889+0000 7fcc04a00700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-26T22:00:41.474549+0000 front 2024-04-26T22:00:41.475097+0000 (oldest deadline 2024-04-26T22:01:05.574631+0000) 2024-04-26T22:01:21.065 INFO:journalctl@ceph.osd.7.smithi076.stdout:Apr 26 22:01:20 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-7[49518]: 2024-04-26T22:01:20.889+0000 7fcc04a00700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-04-26T22:00:51.476654+0000 front 2024-04-26T22:00:51.476706+0000 (oldest deadline 2024-04-26T22:01:16.776822+0000) 2024-04-26T22:01:21.065 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:20 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:20.728+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:11.890629+0000 front 2024-04-26T22:00:11.890849+0000 (oldest deadline 2024-04-26T22:00:40.692043+0000) 2024-04-26T22:01:21.065 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:20 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:20.728+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:24.194239+0000 front 2024-04-26T22:00:24.194095+0000 (oldest deadline 2024-04-26T22:00:48.893996+0000) 2024-04-26T22:01:21.065 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:20 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:20.728+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:34.096788+0000 front 2024-04-26T22:00:34.096884+0000 (oldest deadline 2024-04-26T22:00:56.396862+0000) 2024-04-26T22:01:21.065 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:20 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:20.728+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-26T22:00:42.299288+0000 front 2024-04-26T22:00:42.299033+0000 (oldest deadline 2024-04-26T22:01:06.398950+0000) 2024-04-26T22:01:21.065 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:20 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:20.728+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-04-26T22:00:51.700557+0000 front 2024-04-26T22:00:51.700671+0000 (oldest deadline 2024-04-26T22:01:18.701605+0000) 2024-04-26T22:01:21.690 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 26 22:01:21 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-8[50342]: 2024-04-26T22:01:21.203+0000 7f152e400700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:25.173410+0000 front 2024-04-26T22:00:25.173841+0000 (oldest deadline 2024-04-26T22:00:50.473747+0000) 2024-04-26T22:01:21.691 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 26 22:01:21 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-8[50342]: 2024-04-26T22:01:21.203+0000 7f152e400700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:30.474905+0000 front 2024-04-26T22:00:30.475573+0000 (oldest deadline 2024-04-26T22:00:56.375141+0000) 2024-04-26T22:01:21.691 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 26 22:01:21 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-8[50342]: 2024-04-26T22:01:21.203+0000 7f152e400700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-26T22:00:43.977887+0000 front 2024-04-26T22:00:43.978065+0000 (oldest deadline 2024-04-26T22:01:06.277746+0000) 2024-04-26T22:01:21.691 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 26 22:01:21 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-8[50342]: 2024-04-26T22:01:21.203+0000 7f152e400700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-04-26T22:00:52.581936+0000 front 2024-04-26T22:00:52.581907+0000 (oldest deadline 2024-04-26T22:01:14.881658+0000) 2024-04-26T22:01:21.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:21 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:21.295+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:14.343204+0000 front 2024-04-26T22:00:14.342815+0000 (oldest deadline 2024-04-26T22:00:39.042910+0000) 2024-04-26T22:01:21.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:21 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:21.295+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:24.945823+0000 front 2024-04-26T22:00:24.945339+0000 (oldest deadline 2024-04-26T22:00:47.245223+0000) 2024-04-26T22:01:21.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:21 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:21.295+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:31.947744+0000 front 2024-04-26T22:00:31.948080+0000 (oldest deadline 2024-04-26T22:00:56.647838+0000) 2024-04-26T22:01:21.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:21 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:21.295+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-26T22:00:42.953004+0000 front 2024-04-26T22:00:42.952908+0000 (oldest deadline 2024-04-26T22:01:07.652653+0000) 2024-04-26T22:01:21.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:21 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:21.295+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-04-26T22:00:53.256824+0000 front 2024-04-26T22:00:53.256989+0000 (oldest deadline 2024-04-26T22:01:18.556986+0000) 2024-04-26T22:01:21.822 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:01:21 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:01:21.545+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:16.595921+0000 front 2024-04-26T22:00:16.596275+0000 (oldest deadline 2024-04-26T22:00:39.495420+0000) 2024-04-26T22:01:21.822 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:01:21 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:01:21.545+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:24.600976+0000 front 2024-04-26T22:00:24.600553+0000 (oldest deadline 2024-04-26T22:00:48.100712+0000) 2024-04-26T22:01:21.822 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:01:21 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:01:21.545+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:34.002793+0000 front 2024-04-26T22:00:34.003098+0000 (oldest deadline 2024-04-26T22:00:58.103086+0000) 2024-04-26T22:01:21.822 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:01:21 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:01:21.545+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-26T22:00:41.004997+0000 front 2024-04-26T22:00:41.005270+0000 (oldest deadline 2024-04-26T22:01:08.006242+0000) 2024-04-26T22:01:22.064 INFO:journalctl@ceph.osd.7.smithi076.stdout:Apr 26 22:01:21 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-7[49518]: 2024-04-26T22:01:21.918+0000 7fcc04a00700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:21.667782+0000 front 2024-04-26T22:00:21.668606+0000 (oldest deadline 2024-04-26T22:00:46.967821+0000) 2024-04-26T22:01:22.065 INFO:journalctl@ceph.osd.7.smithi076.stdout:Apr 26 22:01:21 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-7[49518]: 2024-04-26T22:01:21.918+0000 7fcc04a00700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:34.571664+0000 front 2024-04-26T22:00:34.572066+0000 (oldest deadline 2024-04-26T22:00:57.471976+0000) 2024-04-26T22:01:22.065 INFO:journalctl@ceph.osd.7.smithi076.stdout:Apr 26 22:01:21 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-7[49518]: 2024-04-26T22:01:21.918+0000 7fcc04a00700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-26T22:00:41.474549+0000 front 2024-04-26T22:00:41.475097+0000 (oldest deadline 2024-04-26T22:01:05.574631+0000) 2024-04-26T22:01:22.065 INFO:journalctl@ceph.osd.7.smithi076.stdout:Apr 26 22:01:21 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-7[49518]: 2024-04-26T22:01:21.918+0000 7fcc04a00700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-04-26T22:00:51.476654+0000 front 2024-04-26T22:00:51.476706+0000 (oldest deadline 2024-04-26T22:01:16.776822+0000) 2024-04-26T22:01:22.065 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:21 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:21.701+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:11.890629+0000 front 2024-04-26T22:00:11.890849+0000 (oldest deadline 2024-04-26T22:00:40.692043+0000) 2024-04-26T22:01:22.065 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:21 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:21.701+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:24.194239+0000 front 2024-04-26T22:00:24.194095+0000 (oldest deadline 2024-04-26T22:00:48.893996+0000) 2024-04-26T22:01:22.065 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:21 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:21.701+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:34.096788+0000 front 2024-04-26T22:00:34.096884+0000 (oldest deadline 2024-04-26T22:00:56.396862+0000) 2024-04-26T22:01:22.065 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:21 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:21.701+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-26T22:00:42.299288+0000 front 2024-04-26T22:00:42.299033+0000 (oldest deadline 2024-04-26T22:01:06.398950+0000) 2024-04-26T22:01:22.065 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:21 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:21.701+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-04-26T22:00:51.700557+0000 front 2024-04-26T22:00:51.700671+0000 (oldest deadline 2024-04-26T22:01:18.701605+0000) 2024-04-26T22:01:22.690 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 26 22:01:22 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-8[50342]: 2024-04-26T22:01:22.198+0000 7f152e400700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:25.173410+0000 front 2024-04-26T22:00:25.173841+0000 (oldest deadline 2024-04-26T22:00:50.473747+0000) 2024-04-26T22:01:22.691 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 26 22:01:22 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-8[50342]: 2024-04-26T22:01:22.198+0000 7f152e400700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:30.474905+0000 front 2024-04-26T22:00:30.475573+0000 (oldest deadline 2024-04-26T22:00:56.375141+0000) 2024-04-26T22:01:22.691 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 26 22:01:22 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-8[50342]: 2024-04-26T22:01:22.198+0000 7f152e400700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-26T22:00:43.977887+0000 front 2024-04-26T22:00:43.978065+0000 (oldest deadline 2024-04-26T22:01:06.277746+0000) 2024-04-26T22:01:22.691 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 26 22:01:22 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-8[50342]: 2024-04-26T22:01:22.198+0000 7f152e400700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-04-26T22:00:52.581936+0000 front 2024-04-26T22:00:52.581907+0000 (oldest deadline 2024-04-26T22:01:14.881658+0000) 2024-04-26T22:01:22.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:22 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:22.333+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:14.343204+0000 front 2024-04-26T22:00:14.342815+0000 (oldest deadline 2024-04-26T22:00:39.042910+0000) 2024-04-26T22:01:22.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:22 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:22.333+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:24.945823+0000 front 2024-04-26T22:00:24.945339+0000 (oldest deadline 2024-04-26T22:00:47.245223+0000) 2024-04-26T22:01:22.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:22 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:22.333+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:31.947744+0000 front 2024-04-26T22:00:31.948080+0000 (oldest deadline 2024-04-26T22:00:56.647838+0000) 2024-04-26T22:01:22.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:22 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:22.333+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-26T22:00:42.953004+0000 front 2024-04-26T22:00:42.952908+0000 (oldest deadline 2024-04-26T22:01:07.652653+0000) 2024-04-26T22:01:22.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:22 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:22.333+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-04-26T22:00:53.256824+0000 front 2024-04-26T22:00:53.256989+0000 (oldest deadline 2024-04-26T22:01:18.556986+0000) 2024-04-26T22:01:22.729 INFO:journalctl@ceph.osd.7.smithi076.stdout:Apr 26 22:01:22 smithi076 podman[65172]: 2024-04-26 22:01:22.427074092 +0000 UTC m=+5.680516872 container died ba938167c1fdd8ff340020ed5b5f12a75bb2ab1493147b0a029ec464acc31996 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-7, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff) 2024-04-26T22:01:22.822 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:01:22 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:01:22.501+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:16.595921+0000 front 2024-04-26T22:00:16.596275+0000 (oldest deadline 2024-04-26T22:00:39.495420+0000) 2024-04-26T22:01:22.822 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:01:22 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:01:22.501+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:24.600976+0000 front 2024-04-26T22:00:24.600553+0000 (oldest deadline 2024-04-26T22:00:48.100712+0000) 2024-04-26T22:01:22.822 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:01:22 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:01:22.501+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:34.002793+0000 front 2024-04-26T22:00:34.003098+0000 (oldest deadline 2024-04-26T22:00:58.103086+0000) 2024-04-26T22:01:22.822 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:01:22 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:01:22.501+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-26T22:00:41.004997+0000 front 2024-04-26T22:00:41.005270+0000 (oldest deadline 2024-04-26T22:01:08.006242+0000) 2024-04-26T22:01:23.065 INFO:journalctl@ceph.osd.7.smithi076.stdout:Apr 26 22:01:22 smithi076 podman[65172]: 2024-04-26 22:01:22.880360314 +0000 UTC m=+6.133803133 container cleanup ba938167c1fdd8ff340020ed5b5f12a75bb2ab1493147b0a029ec464acc31996 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-7, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240424, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.schema-version=1.0) 2024-04-26T22:01:23.065 INFO:journalctl@ceph.osd.7.smithi076.stdout:Apr 26 22:01:22 smithi076 bash[65172]: ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-7 2024-04-26T22:01:23.065 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:22 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:22.728+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:11.890629+0000 front 2024-04-26T22:00:11.890849+0000 (oldest deadline 2024-04-26T22:00:40.692043+0000) 2024-04-26T22:01:23.065 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:22 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:22.728+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:24.194239+0000 front 2024-04-26T22:00:24.194095+0000 (oldest deadline 2024-04-26T22:00:48.893996+0000) 2024-04-26T22:01:23.065 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:22 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:22.728+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:34.096788+0000 front 2024-04-26T22:00:34.096884+0000 (oldest deadline 2024-04-26T22:00:56.396862+0000) 2024-04-26T22:01:23.065 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:22 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:22.728+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-26T22:00:42.299288+0000 front 2024-04-26T22:00:42.299033+0000 (oldest deadline 2024-04-26T22:01:06.398950+0000) 2024-04-26T22:01:23.065 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:22 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:22.728+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-04-26T22:00:51.700557+0000 front 2024-04-26T22:00:51.700671+0000 (oldest deadline 2024-04-26T22:01:18.701605+0000) 2024-04-26T22:01:23.564 INFO:journalctl@ceph.osd.7.smithi076.stdout:Apr 26 22:01:23 smithi076 podman[65201]: 2024-04-26 22:01:23.14852384 +0000 UTC m=+0.707684418 container remove ba938167c1fdd8ff340020ed5b5f12a75bb2ab1493147b0a029ec464acc31996 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-7, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240424, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T22:01:23.690 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 26 22:01:23 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-8[50342]: 2024-04-26T22:01:23.223+0000 7f152e400700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:25.173410+0000 front 2024-04-26T22:00:25.173841+0000 (oldest deadline 2024-04-26T22:00:50.473747+0000) 2024-04-26T22:01:23.691 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 26 22:01:23 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-8[50342]: 2024-04-26T22:01:23.223+0000 7f152e400700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:30.474905+0000 front 2024-04-26T22:00:30.475573+0000 (oldest deadline 2024-04-26T22:00:56.375141+0000) 2024-04-26T22:01:23.691 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 26 22:01:23 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-8[50342]: 2024-04-26T22:01:23.223+0000 7f152e400700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-26T22:00:43.977887+0000 front 2024-04-26T22:00:43.978065+0000 (oldest deadline 2024-04-26T22:01:06.277746+0000) 2024-04-26T22:01:23.691 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 26 22:01:23 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-8[50342]: 2024-04-26T22:01:23.223+0000 7f152e400700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-04-26T22:00:52.581936+0000 front 2024-04-26T22:00:52.581907+0000 (oldest deadline 2024-04-26T22:01:14.881658+0000) 2024-04-26T22:01:23.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:23 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:23.383+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:14.343204+0000 front 2024-04-26T22:00:14.342815+0000 (oldest deadline 2024-04-26T22:00:39.042910+0000) 2024-04-26T22:01:23.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:23 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:23.383+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:24.945823+0000 front 2024-04-26T22:00:24.945339+0000 (oldest deadline 2024-04-26T22:00:47.245223+0000) 2024-04-26T22:01:23.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:23 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:23.383+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:31.947744+0000 front 2024-04-26T22:00:31.948080+0000 (oldest deadline 2024-04-26T22:00:56.647838+0000) 2024-04-26T22:01:23.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:23 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:23.383+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-26T22:00:42.953004+0000 front 2024-04-26T22:00:42.952908+0000 (oldest deadline 2024-04-26T22:01:07.652653+0000) 2024-04-26T22:01:23.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:23 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:23.383+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-04-26T22:00:53.256824+0000 front 2024-04-26T22:00:53.256989+0000 (oldest deadline 2024-04-26T22:01:18.556986+0000) 2024-04-26T22:01:23.821 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:01:23 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:01:23.456+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:16.595921+0000 front 2024-04-26T22:00:16.596275+0000 (oldest deadline 2024-04-26T22:00:39.495420+0000) 2024-04-26T22:01:23.822 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:01:23 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:01:23.456+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:24.600976+0000 front 2024-04-26T22:00:24.600553+0000 (oldest deadline 2024-04-26T22:00:48.100712+0000) 2024-04-26T22:01:23.822 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:01:23 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:01:23.456+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:34.002793+0000 front 2024-04-26T22:00:34.003098+0000 (oldest deadline 2024-04-26T22:00:58.103086+0000) 2024-04-26T22:01:23.822 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:01:23 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:01:23.456+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-26T22:00:41.004997+0000 front 2024-04-26T22:00:41.005270+0000 (oldest deadline 2024-04-26T22:01:08.006242+0000) 2024-04-26T22:01:23.984 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:23 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:23.715+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:11.890629+0000 front 2024-04-26T22:00:11.890849+0000 (oldest deadline 2024-04-26T22:00:40.692043+0000) 2024-04-26T22:01:23.984 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:23 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:23.715+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:24.194239+0000 front 2024-04-26T22:00:24.194095+0000 (oldest deadline 2024-04-26T22:00:48.893996+0000) 2024-04-26T22:01:23.984 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:23 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:23.715+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:34.096788+0000 front 2024-04-26T22:00:34.096884+0000 (oldest deadline 2024-04-26T22:00:56.396862+0000) 2024-04-26T22:01:23.984 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:23 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:23.715+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-26T22:00:42.299288+0000 front 2024-04-26T22:00:42.299033+0000 (oldest deadline 2024-04-26T22:01:06.398950+0000) 2024-04-26T22:01:23.985 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:23 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:23.715+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-04-26T22:00:51.700557+0000 front 2024-04-26T22:00:51.700671+0000 (oldest deadline 2024-04-26T22:01:18.701605+0000) 2024-04-26T22:01:24.314 INFO:journalctl@ceph.osd.7.smithi076.stdout:Apr 26 22:01:23 smithi076 podman[65288]: 2024-04-26 22:01:23.889952776 +0000 UTC m=+0.033349658 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-26T22:01:24.315 INFO:journalctl@ceph.osd.7.smithi076.stdout:Apr 26 22:01:24 smithi076 podman[65288]: 2024-04-26 22:01:24.013877664 +0000 UTC m=+0.157274544 container create 55e3bdd9362b0397144498c44758e02ae3c3c357aa63840644444faaebc79467 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-7-deactivate, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424) 2024-04-26T22:01:24.690 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 26 22:01:24 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-8[50342]: 2024-04-26T22:01:24.201+0000 7f152e400700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:25.173410+0000 front 2024-04-26T22:00:25.173841+0000 (oldest deadline 2024-04-26T22:00:50.473747+0000) 2024-04-26T22:01:24.691 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 26 22:01:24 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-8[50342]: 2024-04-26T22:01:24.201+0000 7f152e400700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:30.474905+0000 front 2024-04-26T22:00:30.475573+0000 (oldest deadline 2024-04-26T22:00:56.375141+0000) 2024-04-26T22:01:24.691 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 26 22:01:24 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-8[50342]: 2024-04-26T22:01:24.201+0000 7f152e400700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-26T22:00:43.977887+0000 front 2024-04-26T22:00:43.978065+0000 (oldest deadline 2024-04-26T22:01:06.277746+0000) 2024-04-26T22:01:24.691 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 26 22:01:24 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-8[50342]: 2024-04-26T22:01:24.201+0000 7f152e400700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-04-26T22:00:52.581936+0000 front 2024-04-26T22:00:52.581907+0000 (oldest deadline 2024-04-26T22:01:14.881658+0000) 2024-04-26T22:01:24.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:24 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:24.418+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:14.343204+0000 front 2024-04-26T22:00:14.342815+0000 (oldest deadline 2024-04-26T22:00:39.042910+0000) 2024-04-26T22:01:24.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:24 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:24.418+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:24.945823+0000 front 2024-04-26T22:00:24.945339+0000 (oldest deadline 2024-04-26T22:00:47.245223+0000) 2024-04-26T22:01:24.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:24 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:24.418+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:31.947744+0000 front 2024-04-26T22:00:31.948080+0000 (oldest deadline 2024-04-26T22:00:56.647838+0000) 2024-04-26T22:01:24.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:24 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:24.418+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-26T22:00:42.953004+0000 front 2024-04-26T22:00:42.952908+0000 (oldest deadline 2024-04-26T22:01:07.652653+0000) 2024-04-26T22:01:24.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:24 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:24.418+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-04-26T22:00:53.256824+0000 front 2024-04-26T22:00:53.256989+0000 (oldest deadline 2024-04-26T22:01:18.556986+0000) 2024-04-26T22:01:24.739 INFO:journalctl@ceph.osd.7.smithi076.stdout:Apr 26 22:01:24 smithi076 podman[65288]: 2024-04-26 22:01:24.31995084 +0000 UTC m=+0.463347765 container init 55e3bdd9362b0397144498c44758e02ae3c3c357aa63840644444faaebc79467 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-7-deactivate, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff) 2024-04-26T22:01:24.739 INFO:journalctl@ceph.osd.7.smithi076.stdout:Apr 26 22:01:24 smithi076 podman[65288]: 2024-04-26 22:01:24.331040845 +0000 UTC m=+0.474437794 container start 55e3bdd9362b0397144498c44758e02ae3c3c357aa63840644444faaebc79467 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-7-deactivate, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff) 2024-04-26T22:01:24.739 INFO:journalctl@ceph.osd.7.smithi076.stdout:Apr 26 22:01:24 smithi076 podman[65288]: 2024-04-26 22:01:24.410387908 +0000 UTC m=+0.553784833 container attach 55e3bdd9362b0397144498c44758e02ae3c3c357aa63840644444faaebc79467 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-7-deactivate, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True) 2024-04-26T22:01:24.821 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:01:24 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:01:24.433+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:16.595921+0000 front 2024-04-26T22:00:16.596275+0000 (oldest deadline 2024-04-26T22:00:39.495420+0000) 2024-04-26T22:01:24.822 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:01:24 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:01:24.433+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:24.600976+0000 front 2024-04-26T22:00:24.600553+0000 (oldest deadline 2024-04-26T22:00:48.100712+0000) 2024-04-26T22:01:24.822 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:01:24 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:01:24.433+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:34.002793+0000 front 2024-04-26T22:00:34.003098+0000 (oldest deadline 2024-04-26T22:00:58.103086+0000) 2024-04-26T22:01:24.822 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:01:24 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:01:24.433+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-26T22:00:41.004997+0000 front 2024-04-26T22:00:41.005270+0000 (oldest deadline 2024-04-26T22:01:08.006242+0000) 2024-04-26T22:01:25.065 INFO:journalctl@ceph.osd.7.smithi076.stdout:Apr 26 22:01:24 smithi076 podman[65288]: 2024-04-26 22:01:24.79097054 +0000 UTC m=+0.934367452 container died 55e3bdd9362b0397144498c44758e02ae3c3c357aa63840644444faaebc79467 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-7-deactivate, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-26T22:01:25.065 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:24 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:24.738+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:11.890629+0000 front 2024-04-26T22:00:11.890849+0000 (oldest deadline 2024-04-26T22:00:40.692043+0000) 2024-04-26T22:01:25.065 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:24 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:24.738+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:24.194239+0000 front 2024-04-26T22:00:24.194095+0000 (oldest deadline 2024-04-26T22:00:48.893996+0000) 2024-04-26T22:01:25.065 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:24 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:24.738+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:34.096788+0000 front 2024-04-26T22:00:34.096884+0000 (oldest deadline 2024-04-26T22:00:56.396862+0000) 2024-04-26T22:01:25.065 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:24 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:24.738+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-26T22:00:42.299288+0000 front 2024-04-26T22:00:42.299033+0000 (oldest deadline 2024-04-26T22:01:06.398950+0000) 2024-04-26T22:01:25.065 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:24 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:24.738+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-04-26T22:00:51.700557+0000 front 2024-04-26T22:00:51.700671+0000 (oldest deadline 2024-04-26T22:01:18.701605+0000) 2024-04-26T22:01:25.585 DEBUG:teuthology.orchestra.run.smithi076:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a1780950-0415-11ef-bc93-c7b262605968@osd.7.service' 2024-04-26T22:01:25.688 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T22:01:25.689 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-04-26T22:01:25.689 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-04-26T22:01:25.689 DEBUG:teuthology.orchestra.run.smithi164:> sudo systemctl stop ceph-a1780950-0415-11ef-bc93-c7b262605968@osd.8 2024-04-26T22:01:25.690 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 26 22:01:25 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-8[50342]: 2024-04-26T22:01:25.197+0000 7f152e400700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:25.173410+0000 front 2024-04-26T22:00:25.173841+0000 (oldest deadline 2024-04-26T22:00:50.473747+0000) 2024-04-26T22:01:25.690 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 26 22:01:25 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-8[50342]: 2024-04-26T22:01:25.197+0000 7f152e400700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:30.474905+0000 front 2024-04-26T22:00:30.475573+0000 (oldest deadline 2024-04-26T22:00:56.375141+0000) 2024-04-26T22:01:25.691 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 26 22:01:25 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-8[50342]: 2024-04-26T22:01:25.197+0000 7f152e400700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-26T22:00:43.977887+0000 front 2024-04-26T22:00:43.978065+0000 (oldest deadline 2024-04-26T22:01:06.277746+0000) 2024-04-26T22:01:25.691 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 26 22:01:25 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-8[50342]: 2024-04-26T22:01:25.197+0000 7f152e400700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-04-26T22:00:52.581936+0000 front 2024-04-26T22:00:52.581907+0000 (oldest deadline 2024-04-26T22:01:14.881658+0000) 2024-04-26T22:01:25.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:25 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:25.393+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:14.343204+0000 front 2024-04-26T22:00:14.342815+0000 (oldest deadline 2024-04-26T22:00:39.042910+0000) 2024-04-26T22:01:25.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:25 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:25.393+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:24.945823+0000 front 2024-04-26T22:00:24.945339+0000 (oldest deadline 2024-04-26T22:00:47.245223+0000) 2024-04-26T22:01:25.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:25 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:25.393+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:31.947744+0000 front 2024-04-26T22:00:31.948080+0000 (oldest deadline 2024-04-26T22:00:56.647838+0000) 2024-04-26T22:01:25.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:25 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:25.393+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-26T22:00:42.953004+0000 front 2024-04-26T22:00:42.952908+0000 (oldest deadline 2024-04-26T22:01:07.652653+0000) 2024-04-26T22:01:25.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:25 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:25.393+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-04-26T22:00:53.256824+0000 front 2024-04-26T22:00:53.256989+0000 (oldest deadline 2024-04-26T22:01:18.556986+0000) 2024-04-26T22:01:25.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:25 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:25.393+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-04-26T22:00:58.557938+0000 front 2024-04-26T22:00:58.558225+0000 (oldest deadline 2024-04-26T22:01:24.457775+0000) 2024-04-26T22:01:25.815 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:25 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:25.736+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:11.890629+0000 front 2024-04-26T22:00:11.890849+0000 (oldest deadline 2024-04-26T22:00:40.692043+0000) 2024-04-26T22:01:25.815 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:25 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:25.736+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:24.194239+0000 front 2024-04-26T22:00:24.194095+0000 (oldest deadline 2024-04-26T22:00:48.893996+0000) 2024-04-26T22:01:25.815 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:25 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:25.736+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:34.096788+0000 front 2024-04-26T22:00:34.096884+0000 (oldest deadline 2024-04-26T22:00:56.396862+0000) 2024-04-26T22:01:25.815 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:25 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:25.736+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-26T22:00:42.299288+0000 front 2024-04-26T22:00:42.299033+0000 (oldest deadline 2024-04-26T22:01:06.398950+0000) 2024-04-26T22:01:25.815 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:25 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:25.736+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-04-26T22:00:51.700557+0000 front 2024-04-26T22:00:51.700671+0000 (oldest deadline 2024-04-26T22:01:18.701605+0000) 2024-04-26T22:01:25.821 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:01:25 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:01:25.387+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:16.595921+0000 front 2024-04-26T22:00:16.596275+0000 (oldest deadline 2024-04-26T22:00:39.495420+0000) 2024-04-26T22:01:25.822 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:01:25 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:01:25.387+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:24.600976+0000 front 2024-04-26T22:00:24.600553+0000 (oldest deadline 2024-04-26T22:00:48.100712+0000) 2024-04-26T22:01:25.822 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:01:25 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:01:25.387+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:34.002793+0000 front 2024-04-26T22:00:34.003098+0000 (oldest deadline 2024-04-26T22:00:58.103086+0000) 2024-04-26T22:01:25.822 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:01:25 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:01:25.387+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-26T22:00:41.004997+0000 front 2024-04-26T22:00:41.005270+0000 (oldest deadline 2024-04-26T22:01:08.006242+0000) 2024-04-26T22:01:26.185 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 26 22:01:25 smithi164 systemd[1]: Stopping Ceph osd.8 for a1780950-0415-11ef-bc93-c7b262605968... 2024-04-26T22:01:26.441 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:26 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:26.378+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:14.343204+0000 front 2024-04-26T22:00:14.342815+0000 (oldest deadline 2024-04-26T22:00:39.042910+0000) 2024-04-26T22:01:26.441 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:26 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:26.378+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:24.945823+0000 front 2024-04-26T22:00:24.945339+0000 (oldest deadline 2024-04-26T22:00:47.245223+0000) 2024-04-26T22:01:26.441 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:26 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:26.378+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:31.947744+0000 front 2024-04-26T22:00:31.948080+0000 (oldest deadline 2024-04-26T22:00:56.647838+0000) 2024-04-26T22:01:26.441 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:26 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:26.378+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-26T22:00:42.953004+0000 front 2024-04-26T22:00:42.952908+0000 (oldest deadline 2024-04-26T22:01:07.652653+0000) 2024-04-26T22:01:26.441 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:26 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:26.378+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-04-26T22:00:53.256824+0000 front 2024-04-26T22:00:53.256989+0000 (oldest deadline 2024-04-26T22:01:18.556986+0000) 2024-04-26T22:01:26.441 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:26 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:26.378+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-04-26T22:00:58.557938+0000 front 2024-04-26T22:00:58.558225+0000 (oldest deadline 2024-04-26T22:01:24.457775+0000) 2024-04-26T22:01:26.441 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 26 22:01:26 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-8[50342]: 2024-04-26T22:01:26.183+0000 7f1537c00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T22:01:26.441 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 26 22:01:26 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-8[50342]: 2024-04-26T22:01:26.183+0000 7f1537c00700 -1 osd.8 94 *** Got signal Terminated *** 2024-04-26T22:01:26.441 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 26 22:01:26 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-8[50342]: 2024-04-26T22:01:26.183+0000 7f1537c00700 -1 osd.8 94 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-26T22:01:26.442 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 26 22:01:26 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-8[50342]: 2024-04-26T22:01:26.244+0000 7f152e400700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:25.173410+0000 front 2024-04-26T22:00:25.173841+0000 (oldest deadline 2024-04-26T22:00:50.473747+0000) 2024-04-26T22:01:26.442 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 26 22:01:26 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-8[50342]: 2024-04-26T22:01:26.244+0000 7f152e400700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:30.474905+0000 front 2024-04-26T22:00:30.475573+0000 (oldest deadline 2024-04-26T22:00:56.375141+0000) 2024-04-26T22:01:26.442 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 26 22:01:26 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-8[50342]: 2024-04-26T22:01:26.244+0000 7f152e400700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-26T22:00:43.977887+0000 front 2024-04-26T22:00:43.978065+0000 (oldest deadline 2024-04-26T22:01:06.277746+0000) 2024-04-26T22:01:26.442 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 26 22:01:26 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-8[50342]: 2024-04-26T22:01:26.244+0000 7f152e400700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-04-26T22:00:52.581936+0000 front 2024-04-26T22:00:52.581907+0000 (oldest deadline 2024-04-26T22:01:14.881658+0000) 2024-04-26T22:01:26.822 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:01:26 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:01:26.408+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:16.595921+0000 front 2024-04-26T22:00:16.596275+0000 (oldest deadline 2024-04-26T22:00:39.495420+0000) 2024-04-26T22:01:26.822 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:01:26 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:01:26.408+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:24.600976+0000 front 2024-04-26T22:00:24.600553+0000 (oldest deadline 2024-04-26T22:00:48.100712+0000) 2024-04-26T22:01:26.822 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:01:26 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:01:26.408+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:34.002793+0000 front 2024-04-26T22:00:34.003098+0000 (oldest deadline 2024-04-26T22:00:58.103086+0000) 2024-04-26T22:01:26.822 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:01:26 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:01:26.408+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-26T22:00:41.004997+0000 front 2024-04-26T22:00:41.005270+0000 (oldest deadline 2024-04-26T22:01:08.006242+0000) 2024-04-26T22:01:26.822 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:01:26 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:01:26.408+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-04-26T22:00:59.511853+0000 front 2024-04-26T22:00:59.511966+0000 (oldest deadline 2024-04-26T22:01:25.411971+0000) 2024-04-26T22:01:27.064 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:26 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:26.704+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:11.890629+0000 front 2024-04-26T22:00:11.890849+0000 (oldest deadline 2024-04-26T22:00:40.692043+0000) 2024-04-26T22:01:27.065 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:26 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:26.704+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:24.194239+0000 front 2024-04-26T22:00:24.194095+0000 (oldest deadline 2024-04-26T22:00:48.893996+0000) 2024-04-26T22:01:27.065 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:26 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:26.704+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:34.096788+0000 front 2024-04-26T22:00:34.096884+0000 (oldest deadline 2024-04-26T22:00:56.396862+0000) 2024-04-26T22:01:27.065 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:26 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:26.704+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-26T22:00:42.299288+0000 front 2024-04-26T22:00:42.299033+0000 (oldest deadline 2024-04-26T22:01:06.398950+0000) 2024-04-26T22:01:27.065 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:26 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:26.704+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-04-26T22:00:51.700557+0000 front 2024-04-26T22:00:51.700671+0000 (oldest deadline 2024-04-26T22:01:18.701605+0000) 2024-04-26T22:01:27.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:27 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:27.368+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:14.343204+0000 front 2024-04-26T22:00:14.342815+0000 (oldest deadline 2024-04-26T22:00:39.042910+0000) 2024-04-26T22:01:27.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:27 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:27.368+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:24.945823+0000 front 2024-04-26T22:00:24.945339+0000 (oldest deadline 2024-04-26T22:00:47.245223+0000) 2024-04-26T22:01:27.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:27 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:27.368+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:31.947744+0000 front 2024-04-26T22:00:31.948080+0000 (oldest deadline 2024-04-26T22:00:56.647838+0000) 2024-04-26T22:01:27.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:27 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:27.368+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-26T22:00:42.953004+0000 front 2024-04-26T22:00:42.952908+0000 (oldest deadline 2024-04-26T22:01:07.652653+0000) 2024-04-26T22:01:27.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:27 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:27.368+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-04-26T22:00:53.256824+0000 front 2024-04-26T22:00:53.256989+0000 (oldest deadline 2024-04-26T22:01:18.556986+0000) 2024-04-26T22:01:27.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:27 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:27.368+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-04-26T22:00:58.557938+0000 front 2024-04-26T22:00:58.558225+0000 (oldest deadline 2024-04-26T22:01:24.457775+0000) 2024-04-26T22:01:27.691 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 26 22:01:27 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-8[50342]: 2024-04-26T22:01:27.241+0000 7f152e400700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:25.173410+0000 front 2024-04-26T22:00:25.173841+0000 (oldest deadline 2024-04-26T22:00:50.473747+0000) 2024-04-26T22:01:27.691 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 26 22:01:27 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-8[50342]: 2024-04-26T22:01:27.241+0000 7f152e400700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:30.474905+0000 front 2024-04-26T22:00:30.475573+0000 (oldest deadline 2024-04-26T22:00:56.375141+0000) 2024-04-26T22:01:27.691 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 26 22:01:27 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-8[50342]: 2024-04-26T22:01:27.241+0000 7f152e400700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-26T22:00:43.977887+0000 front 2024-04-26T22:00:43.978065+0000 (oldest deadline 2024-04-26T22:01:06.277746+0000) 2024-04-26T22:01:27.692 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 26 22:01:27 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-8[50342]: 2024-04-26T22:01:27.242+0000 7f152e400700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-04-26T22:00:52.581936+0000 front 2024-04-26T22:00:52.581907+0000 (oldest deadline 2024-04-26T22:01:14.881658+0000) 2024-04-26T22:01:27.692 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 26 22:01:27 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-8[50342]: 2024-04-26T22:01:27.242+0000 7f152e400700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-04-26T22:01:00.585394+0000 front 2024-04-26T22:01:00.585275+0000 (oldest deadline 2024-04-26T22:01:26.485192+0000) 2024-04-26T22:01:27.822 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:01:27 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:01:27.391+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:16.595921+0000 front 2024-04-26T22:00:16.596275+0000 (oldest deadline 2024-04-26T22:00:39.495420+0000) 2024-04-26T22:01:27.822 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:01:27 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:01:27.391+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:24.600976+0000 front 2024-04-26T22:00:24.600553+0000 (oldest deadline 2024-04-26T22:00:48.100712+0000) 2024-04-26T22:01:27.822 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:01:27 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:01:27.391+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:34.002793+0000 front 2024-04-26T22:00:34.003098+0000 (oldest deadline 2024-04-26T22:00:58.103086+0000) 2024-04-26T22:01:27.822 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:01:27 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:01:27.391+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-26T22:00:41.004997+0000 front 2024-04-26T22:00:41.005270+0000 (oldest deadline 2024-04-26T22:01:08.006242+0000) 2024-04-26T22:01:27.822 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:01:27 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:01:27.391+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-04-26T22:00:59.511853+0000 front 2024-04-26T22:00:59.511966+0000 (oldest deadline 2024-04-26T22:01:25.411971+0000) 2024-04-26T22:01:28.064 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:27 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:27.724+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:11.890629+0000 front 2024-04-26T22:00:11.890849+0000 (oldest deadline 2024-04-26T22:00:40.692043+0000) 2024-04-26T22:01:28.065 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:27 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:27.724+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:24.194239+0000 front 2024-04-26T22:00:24.194095+0000 (oldest deadline 2024-04-26T22:00:48.893996+0000) 2024-04-26T22:01:28.065 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:27 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:27.724+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:34.096788+0000 front 2024-04-26T22:00:34.096884+0000 (oldest deadline 2024-04-26T22:00:56.396862+0000) 2024-04-26T22:01:28.065 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:27 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:27.724+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-26T22:00:42.299288+0000 front 2024-04-26T22:00:42.299033+0000 (oldest deadline 2024-04-26T22:01:06.398950+0000) 2024-04-26T22:01:28.065 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:27 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:27.724+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-04-26T22:00:51.700557+0000 front 2024-04-26T22:00:51.700671+0000 (oldest deadline 2024-04-26T22:01:18.701605+0000) 2024-04-26T22:01:28.065 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:27 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:27.724+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-04-26T22:01:02.104474+0000 front 2024-04-26T22:01:02.104287+0000 (oldest deadline 2024-04-26T22:01:26.803970+0000) 2024-04-26T22:01:28.690 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 26 22:01:28 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-8[50342]: 2024-04-26T22:01:28.278+0000 7f152e400700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:25.173410+0000 front 2024-04-26T22:00:25.173841+0000 (oldest deadline 2024-04-26T22:00:50.473747+0000) 2024-04-26T22:01:28.691 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 26 22:01:28 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-8[50342]: 2024-04-26T22:01:28.278+0000 7f152e400700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:30.474905+0000 front 2024-04-26T22:00:30.475573+0000 (oldest deadline 2024-04-26T22:00:56.375141+0000) 2024-04-26T22:01:28.691 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 26 22:01:28 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-8[50342]: 2024-04-26T22:01:28.278+0000 7f152e400700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-26T22:00:43.977887+0000 front 2024-04-26T22:00:43.978065+0000 (oldest deadline 2024-04-26T22:01:06.277746+0000) 2024-04-26T22:01:28.691 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 26 22:01:28 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-8[50342]: 2024-04-26T22:01:28.278+0000 7f152e400700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-04-26T22:00:52.581936+0000 front 2024-04-26T22:00:52.581907+0000 (oldest deadline 2024-04-26T22:01:14.881658+0000) 2024-04-26T22:01:28.691 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 26 22:01:28 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-8[50342]: 2024-04-26T22:01:28.278+0000 7f152e400700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-04-26T22:01:00.585394+0000 front 2024-04-26T22:01:00.585275+0000 (oldest deadline 2024-04-26T22:01:26.485192+0000) 2024-04-26T22:01:28.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:28 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:28.415+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:14.343204+0000 front 2024-04-26T22:00:14.342815+0000 (oldest deadline 2024-04-26T22:00:39.042910+0000) 2024-04-26T22:01:28.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:28 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:28.415+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:24.945823+0000 front 2024-04-26T22:00:24.945339+0000 (oldest deadline 2024-04-26T22:00:47.245223+0000) 2024-04-26T22:01:28.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:28 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:28.415+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:31.947744+0000 front 2024-04-26T22:00:31.948080+0000 (oldest deadline 2024-04-26T22:00:56.647838+0000) 2024-04-26T22:01:28.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:28 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:28.415+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-26T22:00:42.953004+0000 front 2024-04-26T22:00:42.952908+0000 (oldest deadline 2024-04-26T22:01:07.652653+0000) 2024-04-26T22:01:28.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:28 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:28.415+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-04-26T22:00:53.256824+0000 front 2024-04-26T22:00:53.256989+0000 (oldest deadline 2024-04-26T22:01:18.556986+0000) 2024-04-26T22:01:28.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:28 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:28.415+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-04-26T22:00:58.557938+0000 front 2024-04-26T22:00:58.558225+0000 (oldest deadline 2024-04-26T22:01:24.457775+0000) 2024-04-26T22:01:28.822 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:01:28 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:01:28.364+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:16.595921+0000 front 2024-04-26T22:00:16.596275+0000 (oldest deadline 2024-04-26T22:00:39.495420+0000) 2024-04-26T22:01:28.822 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:01:28 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:01:28.364+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:24.600976+0000 front 2024-04-26T22:00:24.600553+0000 (oldest deadline 2024-04-26T22:00:48.100712+0000) 2024-04-26T22:01:28.822 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:01:28 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:01:28.364+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:34.002793+0000 front 2024-04-26T22:00:34.003098+0000 (oldest deadline 2024-04-26T22:00:58.103086+0000) 2024-04-26T22:01:28.822 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:01:28 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:01:28.364+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-26T22:00:41.004997+0000 front 2024-04-26T22:00:41.005270+0000 (oldest deadline 2024-04-26T22:01:08.006242+0000) 2024-04-26T22:01:28.822 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:01:28 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:01:28.364+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-04-26T22:00:59.511853+0000 front 2024-04-26T22:00:59.511966+0000 (oldest deadline 2024-04-26T22:01:25.411971+0000) 2024-04-26T22:01:29.064 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:28 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:28.696+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:11.890629+0000 front 2024-04-26T22:00:11.890849+0000 (oldest deadline 2024-04-26T22:00:40.692043+0000) 2024-04-26T22:01:29.065 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:28 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:28.696+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:24.194239+0000 front 2024-04-26T22:00:24.194095+0000 (oldest deadline 2024-04-26T22:00:48.893996+0000) 2024-04-26T22:01:29.065 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:28 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:28.696+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:34.096788+0000 front 2024-04-26T22:00:34.096884+0000 (oldest deadline 2024-04-26T22:00:56.396862+0000) 2024-04-26T22:01:29.065 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:28 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:28.696+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-26T22:00:42.299288+0000 front 2024-04-26T22:00:42.299033+0000 (oldest deadline 2024-04-26T22:01:06.398950+0000) 2024-04-26T22:01:29.065 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:28 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:28.696+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-04-26T22:00:51.700557+0000 front 2024-04-26T22:00:51.700671+0000 (oldest deadline 2024-04-26T22:01:18.701605+0000) 2024-04-26T22:01:29.065 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:28 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:28.696+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-04-26T22:01:02.104474+0000 front 2024-04-26T22:01:02.104287+0000 (oldest deadline 2024-04-26T22:01:26.803970+0000) 2024-04-26T22:01:29.690 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 26 22:01:29 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-8[50342]: 2024-04-26T22:01:29.276+0000 7f152e400700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:25.173410+0000 front 2024-04-26T22:00:25.173841+0000 (oldest deadline 2024-04-26T22:00:50.473747+0000) 2024-04-26T22:01:29.691 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 26 22:01:29 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-8[50342]: 2024-04-26T22:01:29.276+0000 7f152e400700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:30.474905+0000 front 2024-04-26T22:00:30.475573+0000 (oldest deadline 2024-04-26T22:00:56.375141+0000) 2024-04-26T22:01:29.691 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 26 22:01:29 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-8[50342]: 2024-04-26T22:01:29.276+0000 7f152e400700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-26T22:00:43.977887+0000 front 2024-04-26T22:00:43.978065+0000 (oldest deadline 2024-04-26T22:01:06.277746+0000) 2024-04-26T22:01:29.691 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 26 22:01:29 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-8[50342]: 2024-04-26T22:01:29.276+0000 7f152e400700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-04-26T22:00:52.581936+0000 front 2024-04-26T22:00:52.581907+0000 (oldest deadline 2024-04-26T22:01:14.881658+0000) 2024-04-26T22:01:29.691 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 26 22:01:29 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-8[50342]: 2024-04-26T22:01:29.276+0000 7f152e400700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-04-26T22:01:00.585394+0000 front 2024-04-26T22:01:00.585275+0000 (oldest deadline 2024-04-26T22:01:26.485192+0000) 2024-04-26T22:01:29.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:29 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:29.430+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:14.343204+0000 front 2024-04-26T22:00:14.342815+0000 (oldest deadline 2024-04-26T22:00:39.042910+0000) 2024-04-26T22:01:29.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:29 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:29.430+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:24.945823+0000 front 2024-04-26T22:00:24.945339+0000 (oldest deadline 2024-04-26T22:00:47.245223+0000) 2024-04-26T22:01:29.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:29 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:29.430+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:31.947744+0000 front 2024-04-26T22:00:31.948080+0000 (oldest deadline 2024-04-26T22:00:56.647838+0000) 2024-04-26T22:01:29.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:29 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:29.430+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-26T22:00:42.953004+0000 front 2024-04-26T22:00:42.952908+0000 (oldest deadline 2024-04-26T22:01:07.652653+0000) 2024-04-26T22:01:29.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:29 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:29.430+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-04-26T22:00:53.256824+0000 front 2024-04-26T22:00:53.256989+0000 (oldest deadline 2024-04-26T22:01:18.556986+0000) 2024-04-26T22:01:29.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:29 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:29.430+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-04-26T22:00:58.557938+0000 front 2024-04-26T22:00:58.558225+0000 (oldest deadline 2024-04-26T22:01:24.457775+0000) 2024-04-26T22:01:29.822 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:01:29 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:01:29.335+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:16.595921+0000 front 2024-04-26T22:00:16.596275+0000 (oldest deadline 2024-04-26T22:00:39.495420+0000) 2024-04-26T22:01:29.822 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:01:29 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:01:29.335+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:24.600976+0000 front 2024-04-26T22:00:24.600553+0000 (oldest deadline 2024-04-26T22:00:48.100712+0000) 2024-04-26T22:01:29.822 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:01:29 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:01:29.335+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:34.002793+0000 front 2024-04-26T22:00:34.003098+0000 (oldest deadline 2024-04-26T22:00:58.103086+0000) 2024-04-26T22:01:29.822 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:01:29 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:01:29.335+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-26T22:00:41.004997+0000 front 2024-04-26T22:00:41.005270+0000 (oldest deadline 2024-04-26T22:01:08.006242+0000) 2024-04-26T22:01:29.822 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:01:29 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:01:29.335+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-04-26T22:00:59.511853+0000 front 2024-04-26T22:00:59.511966+0000 (oldest deadline 2024-04-26T22:01:25.411971+0000) 2024-04-26T22:01:30.065 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:29 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:29.651+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:11.890629+0000 front 2024-04-26T22:00:11.890849+0000 (oldest deadline 2024-04-26T22:00:40.692043+0000) 2024-04-26T22:01:30.065 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:29 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:29.651+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:24.194239+0000 front 2024-04-26T22:00:24.194095+0000 (oldest deadline 2024-04-26T22:00:48.893996+0000) 2024-04-26T22:01:30.065 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:29 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:29.651+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:34.096788+0000 front 2024-04-26T22:00:34.096884+0000 (oldest deadline 2024-04-26T22:00:56.396862+0000) 2024-04-26T22:01:30.065 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:29 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:29.651+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-26T22:00:42.299288+0000 front 2024-04-26T22:00:42.299033+0000 (oldest deadline 2024-04-26T22:01:06.398950+0000) 2024-04-26T22:01:30.065 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:29 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:29.651+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-04-26T22:00:51.700557+0000 front 2024-04-26T22:00:51.700671+0000 (oldest deadline 2024-04-26T22:01:18.701605+0000) 2024-04-26T22:01:30.065 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:29 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:29.651+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-04-26T22:01:02.104474+0000 front 2024-04-26T22:01:02.104287+0000 (oldest deadline 2024-04-26T22:01:26.803970+0000) 2024-04-26T22:01:30.572 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:01:30 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:01:30.313+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:16.595921+0000 front 2024-04-26T22:00:16.596275+0000 (oldest deadline 2024-04-26T22:00:39.495420+0000) 2024-04-26T22:01:30.572 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:01:30 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:01:30.313+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:24.600976+0000 front 2024-04-26T22:00:24.600553+0000 (oldest deadline 2024-04-26T22:00:48.100712+0000) 2024-04-26T22:01:30.572 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:01:30 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:01:30.313+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:34.002793+0000 front 2024-04-26T22:00:34.003098+0000 (oldest deadline 2024-04-26T22:00:58.103086+0000) 2024-04-26T22:01:30.572 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:01:30 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:01:30.313+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-26T22:00:41.004997+0000 front 2024-04-26T22:00:41.005270+0000 (oldest deadline 2024-04-26T22:01:08.006242+0000) 2024-04-26T22:01:30.572 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:01:30 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:01:30.313+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-04-26T22:00:59.511853+0000 front 2024-04-26T22:00:59.511966+0000 (oldest deadline 2024-04-26T22:01:25.411971+0000) 2024-04-26T22:01:30.691 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 26 22:01:30 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-8[50342]: 2024-04-26T22:01:30.243+0000 7f152e400700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:25.173410+0000 front 2024-04-26T22:00:25.173841+0000 (oldest deadline 2024-04-26T22:00:50.473747+0000) 2024-04-26T22:01:30.691 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 26 22:01:30 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-8[50342]: 2024-04-26T22:01:30.243+0000 7f152e400700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:30.474905+0000 front 2024-04-26T22:00:30.475573+0000 (oldest deadline 2024-04-26T22:00:56.375141+0000) 2024-04-26T22:01:30.691 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 26 22:01:30 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-8[50342]: 2024-04-26T22:01:30.243+0000 7f152e400700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-26T22:00:43.977887+0000 front 2024-04-26T22:00:43.978065+0000 (oldest deadline 2024-04-26T22:01:06.277746+0000) 2024-04-26T22:01:30.691 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 26 22:01:30 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-8[50342]: 2024-04-26T22:01:30.243+0000 7f152e400700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-04-26T22:00:52.581936+0000 front 2024-04-26T22:00:52.581907+0000 (oldest deadline 2024-04-26T22:01:14.881658+0000) 2024-04-26T22:01:30.691 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 26 22:01:30 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-8[50342]: 2024-04-26T22:01:30.243+0000 7f152e400700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-04-26T22:01:00.585394+0000 front 2024-04-26T22:01:00.585275+0000 (oldest deadline 2024-04-26T22:01:26.485192+0000) 2024-04-26T22:01:30.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:30 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:30.381+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:14.343204+0000 front 2024-04-26T22:00:14.342815+0000 (oldest deadline 2024-04-26T22:00:39.042910+0000) 2024-04-26T22:01:30.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:30 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:30.381+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:24.945823+0000 front 2024-04-26T22:00:24.945339+0000 (oldest deadline 2024-04-26T22:00:47.245223+0000) 2024-04-26T22:01:30.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:30 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:30.381+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:31.947744+0000 front 2024-04-26T22:00:31.948080+0000 (oldest deadline 2024-04-26T22:00:56.647838+0000) 2024-04-26T22:01:30.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:30 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:30.381+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-26T22:00:42.953004+0000 front 2024-04-26T22:00:42.952908+0000 (oldest deadline 2024-04-26T22:01:07.652653+0000) 2024-04-26T22:01:30.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:30 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:30.381+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-04-26T22:00:53.256824+0000 front 2024-04-26T22:00:53.256989+0000 (oldest deadline 2024-04-26T22:01:18.556986+0000) 2024-04-26T22:01:30.693 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:30 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:30.381+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-04-26T22:00:58.557938+0000 front 2024-04-26T22:00:58.558225+0000 (oldest deadline 2024-04-26T22:01:24.457775+0000) 2024-04-26T22:01:31.064 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:30 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:30.615+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:11.890629+0000 front 2024-04-26T22:00:11.890849+0000 (oldest deadline 2024-04-26T22:00:40.692043+0000) 2024-04-26T22:01:31.065 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:30 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:30.615+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:24.194239+0000 front 2024-04-26T22:00:24.194095+0000 (oldest deadline 2024-04-26T22:00:48.893996+0000) 2024-04-26T22:01:31.065 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:30 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:30.615+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:34.096788+0000 front 2024-04-26T22:00:34.096884+0000 (oldest deadline 2024-04-26T22:00:56.396862+0000) 2024-04-26T22:01:31.065 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:30 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:30.615+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-26T22:00:42.299288+0000 front 2024-04-26T22:00:42.299033+0000 (oldest deadline 2024-04-26T22:01:06.398950+0000) 2024-04-26T22:01:31.065 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:30 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:30.615+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-04-26T22:00:51.700557+0000 front 2024-04-26T22:00:51.700671+0000 (oldest deadline 2024-04-26T22:01:18.701605+0000) 2024-04-26T22:01:31.065 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:30 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:30.615+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-04-26T22:01:02.104474+0000 front 2024-04-26T22:01:02.104287+0000 (oldest deadline 2024-04-26T22:01:26.803970+0000) 2024-04-26T22:01:31.572 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:01:31 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:01:31.312+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:16.595921+0000 front 2024-04-26T22:00:16.596275+0000 (oldest deadline 2024-04-26T22:00:39.495420+0000) 2024-04-26T22:01:31.572 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:01:31 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:01:31.312+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:24.600976+0000 front 2024-04-26T22:00:24.600553+0000 (oldest deadline 2024-04-26T22:00:48.100712+0000) 2024-04-26T22:01:31.572 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:01:31 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:01:31.312+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:34.002793+0000 front 2024-04-26T22:00:34.003098+0000 (oldest deadline 2024-04-26T22:00:58.103086+0000) 2024-04-26T22:01:31.572 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:01:31 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:01:31.312+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-26T22:00:41.004997+0000 front 2024-04-26T22:00:41.005270+0000 (oldest deadline 2024-04-26T22:01:08.006242+0000) 2024-04-26T22:01:31.572 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:01:31 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:01:31.312+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-04-26T22:00:59.511853+0000 front 2024-04-26T22:00:59.511966+0000 (oldest deadline 2024-04-26T22:01:25.411971+0000) 2024-04-26T22:01:31.690 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:31 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:31.403+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:14.343204+0000 front 2024-04-26T22:00:14.342815+0000 (oldest deadline 2024-04-26T22:00:39.042910+0000) 2024-04-26T22:01:31.690 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:31 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:31.403+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:24.945823+0000 front 2024-04-26T22:00:24.945339+0000 (oldest deadline 2024-04-26T22:00:47.245223+0000) 2024-04-26T22:01:31.690 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:31 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:31.403+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:31.947744+0000 front 2024-04-26T22:00:31.948080+0000 (oldest deadline 2024-04-26T22:00:56.647838+0000) 2024-04-26T22:01:31.690 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:31 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:31.403+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-26T22:00:42.953004+0000 front 2024-04-26T22:00:42.952908+0000 (oldest deadline 2024-04-26T22:01:07.652653+0000) 2024-04-26T22:01:31.690 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:31 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:31.403+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-04-26T22:00:53.256824+0000 front 2024-04-26T22:00:53.256989+0000 (oldest deadline 2024-04-26T22:01:18.556986+0000) 2024-04-26T22:01:31.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:31 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:31.403+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-04-26T22:00:58.557938+0000 front 2024-04-26T22:00:58.558225+0000 (oldest deadline 2024-04-26T22:01:24.457775+0000) 2024-04-26T22:01:31.940 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 26 22:01:31 smithi164 podman[66697]: 2024-04-26 22:01:31.689132925 +0000 UTC m=+5.608690706 container died 54bf3f3060a008a0c42b977b0ab1e8c41fc44f9b91719c96bc53f58b957a2aa0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-8, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T22:01:32.064 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:31 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:31.664+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:11.890629+0000 front 2024-04-26T22:00:11.890849+0000 (oldest deadline 2024-04-26T22:00:40.692043+0000) 2024-04-26T22:01:32.065 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:31 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:31.664+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:24.194239+0000 front 2024-04-26T22:00:24.194095+0000 (oldest deadline 2024-04-26T22:00:48.893996+0000) 2024-04-26T22:01:32.065 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:31 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:31.664+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:34.096788+0000 front 2024-04-26T22:00:34.096884+0000 (oldest deadline 2024-04-26T22:00:56.396862+0000) 2024-04-26T22:01:32.065 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:31 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:31.664+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-26T22:00:42.299288+0000 front 2024-04-26T22:00:42.299033+0000 (oldest deadline 2024-04-26T22:01:06.398950+0000) 2024-04-26T22:01:32.065 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:31 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:31.664+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-04-26T22:00:51.700557+0000 front 2024-04-26T22:00:51.700671+0000 (oldest deadline 2024-04-26T22:01:18.701605+0000) 2024-04-26T22:01:32.065 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:31 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:31.664+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-04-26T22:01:02.104474+0000 front 2024-04-26T22:01:02.104287+0000 (oldest deadline 2024-04-26T22:01:26.803970+0000) 2024-04-26T22:01:32.353 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 26 22:01:32 smithi164 podman[66697]: 2024-04-26 22:01:32.177640715 +0000 UTC m=+6.097198579 container cleanup 54bf3f3060a008a0c42b977b0ab1e8c41fc44f9b91719c96bc53f58b957a2aa0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-8, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5) 2024-04-26T22:01:32.353 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 26 22:01:32 smithi164 bash[66697]: ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-8 2024-04-26T22:01:32.572 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:01:32 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:01:32.287+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:16.595921+0000 front 2024-04-26T22:00:16.596275+0000 (oldest deadline 2024-04-26T22:00:39.495420+0000) 2024-04-26T22:01:32.572 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:01:32 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:01:32.287+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:24.600976+0000 front 2024-04-26T22:00:24.600553+0000 (oldest deadline 2024-04-26T22:00:48.100712+0000) 2024-04-26T22:01:32.572 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:01:32 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:01:32.287+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:34.002793+0000 front 2024-04-26T22:00:34.003098+0000 (oldest deadline 2024-04-26T22:00:58.103086+0000) 2024-04-26T22:01:32.572 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:01:32 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:01:32.287+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-26T22:00:41.004997+0000 front 2024-04-26T22:00:41.005270+0000 (oldest deadline 2024-04-26T22:01:08.006242+0000) 2024-04-26T22:01:32.572 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:01:32 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:01:32.287+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-04-26T22:00:59.511853+0000 front 2024-04-26T22:00:59.511966+0000 (oldest deadline 2024-04-26T22:01:25.411971+0000) 2024-04-26T22:01:32.690 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 26 22:01:32 smithi164 podman[66716]: 2024-04-26 22:01:32.353184404 +0000 UTC m=+0.653110509 container remove 54bf3f3060a008a0c42b977b0ab1e8c41fc44f9b91719c96bc53f58b957a2aa0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-8, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424) 2024-04-26T22:01:32.690 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:32 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:32.379+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:14.343204+0000 front 2024-04-26T22:00:14.342815+0000 (oldest deadline 2024-04-26T22:00:39.042910+0000) 2024-04-26T22:01:32.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:32 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:32.379+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:24.945823+0000 front 2024-04-26T22:00:24.945339+0000 (oldest deadline 2024-04-26T22:00:47.245223+0000) 2024-04-26T22:01:32.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:32 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:32.379+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:31.947744+0000 front 2024-04-26T22:00:31.948080+0000 (oldest deadline 2024-04-26T22:00:56.647838+0000) 2024-04-26T22:01:32.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:32 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:32.379+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-26T22:00:42.953004+0000 front 2024-04-26T22:00:42.952908+0000 (oldest deadline 2024-04-26T22:01:07.652653+0000) 2024-04-26T22:01:32.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:32 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:32.379+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-04-26T22:00:53.256824+0000 front 2024-04-26T22:00:53.256989+0000 (oldest deadline 2024-04-26T22:01:18.556986+0000) 2024-04-26T22:01:32.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:32 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:32.379+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-04-26T22:00:58.557938+0000 front 2024-04-26T22:00:58.558225+0000 (oldest deadline 2024-04-26T22:01:24.457775+0000) 2024-04-26T22:01:33.065 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:32 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:32.680+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:11.890629+0000 front 2024-04-26T22:00:11.890849+0000 (oldest deadline 2024-04-26T22:00:40.692043+0000) 2024-04-26T22:01:33.065 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:32 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:32.680+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:24.194239+0000 front 2024-04-26T22:00:24.194095+0000 (oldest deadline 2024-04-26T22:00:48.893996+0000) 2024-04-26T22:01:33.065 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:32 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:32.680+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:34.096788+0000 front 2024-04-26T22:00:34.096884+0000 (oldest deadline 2024-04-26T22:00:56.396862+0000) 2024-04-26T22:01:33.065 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:32 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:32.680+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-26T22:00:42.299288+0000 front 2024-04-26T22:00:42.299033+0000 (oldest deadline 2024-04-26T22:01:06.398950+0000) 2024-04-26T22:01:33.065 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:32 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:32.680+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-04-26T22:00:51.700557+0000 front 2024-04-26T22:00:51.700671+0000 (oldest deadline 2024-04-26T22:01:18.701605+0000) 2024-04-26T22:01:33.066 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:32 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:32.680+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-04-26T22:01:02.104474+0000 front 2024-04-26T22:01:02.104287+0000 (oldest deadline 2024-04-26T22:01:26.803970+0000) 2024-04-26T22:01:33.441 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 26 22:01:33 smithi164 podman[66806]: 2024-04-26 22:01:33.052869569 +0000 UTC m=+0.033420476 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-26T22:01:33.441 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 26 22:01:33 smithi164 podman[66806]: 2024-04-26 22:01:33.212710924 +0000 UTC m=+0.193261893 container create 7ede0cd7228c82af0094db17e0b8c34e212a75e2c82166ee468aab64ac15209f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-8-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-04-26T22:01:33.441 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:33 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:33.363+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:14.343204+0000 front 2024-04-26T22:00:14.342815+0000 (oldest deadline 2024-04-26T22:00:39.042910+0000) 2024-04-26T22:01:33.441 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:33 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:33.363+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:24.945823+0000 front 2024-04-26T22:00:24.945339+0000 (oldest deadline 2024-04-26T22:00:47.245223+0000) 2024-04-26T22:01:33.442 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:33 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:33.363+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:31.947744+0000 front 2024-04-26T22:00:31.948080+0000 (oldest deadline 2024-04-26T22:00:56.647838+0000) 2024-04-26T22:01:33.442 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:33 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:33.363+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-26T22:00:42.953004+0000 front 2024-04-26T22:00:42.952908+0000 (oldest deadline 2024-04-26T22:01:07.652653+0000) 2024-04-26T22:01:33.442 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:33 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:33.363+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-04-26T22:00:53.256824+0000 front 2024-04-26T22:00:53.256989+0000 (oldest deadline 2024-04-26T22:01:18.556986+0000) 2024-04-26T22:01:33.442 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:33 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:33.363+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-04-26T22:00:58.557938+0000 front 2024-04-26T22:00:58.558225+0000 (oldest deadline 2024-04-26T22:01:24.457775+0000) 2024-04-26T22:01:33.572 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:01:33 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:01:33.262+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:16.595921+0000 front 2024-04-26T22:00:16.596275+0000 (oldest deadline 2024-04-26T22:00:39.495420+0000) 2024-04-26T22:01:33.572 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:01:33 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:01:33.262+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:24.600976+0000 front 2024-04-26T22:00:24.600553+0000 (oldest deadline 2024-04-26T22:00:48.100712+0000) 2024-04-26T22:01:33.572 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:01:33 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:01:33.262+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:34.002793+0000 front 2024-04-26T22:00:34.003098+0000 (oldest deadline 2024-04-26T22:00:58.103086+0000) 2024-04-26T22:01:33.572 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:01:33 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:01:33.262+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-26T22:00:41.004997+0000 front 2024-04-26T22:00:41.005270+0000 (oldest deadline 2024-04-26T22:01:08.006242+0000) 2024-04-26T22:01:33.573 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:01:33 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:01:33.262+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-04-26T22:00:59.511853+0000 front 2024-04-26T22:00:59.511966+0000 (oldest deadline 2024-04-26T22:01:25.411971+0000) 2024-04-26T22:01:33.941 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 26 22:01:33 smithi164 podman[66806]: 2024-04-26 22:01:33.483624456 +0000 UTC m=+0.464175435 container init 7ede0cd7228c82af0094db17e0b8c34e212a75e2c82166ee468aab64ac15209f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-8-deactivate, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-04-26T22:01:33.941 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 26 22:01:33 smithi164 podman[66806]: 2024-04-26 22:01:33.496175892 +0000 UTC m=+0.476726837 container start 7ede0cd7228c82af0094db17e0b8c34e212a75e2c82166ee468aab64ac15209f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-8-deactivate, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T22:01:33.941 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 26 22:01:33 smithi164 podman[66806]: 2024-04-26 22:01:33.606581976 +0000 UTC m=+0.587132913 container attach 7ede0cd7228c82af0094db17e0b8c34e212a75e2c82166ee468aab64ac15209f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-8-deactivate, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T22:01:34.064 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:33 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:33.635+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:11.890629+0000 front 2024-04-26T22:00:11.890849+0000 (oldest deadline 2024-04-26T22:00:40.692043+0000) 2024-04-26T22:01:34.065 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:33 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:33.635+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:24.194239+0000 front 2024-04-26T22:00:24.194095+0000 (oldest deadline 2024-04-26T22:00:48.893996+0000) 2024-04-26T22:01:34.065 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:33 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:33.635+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:34.096788+0000 front 2024-04-26T22:00:34.096884+0000 (oldest deadline 2024-04-26T22:00:56.396862+0000) 2024-04-26T22:01:34.065 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:33 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:33.635+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-26T22:00:42.299288+0000 front 2024-04-26T22:00:42.299033+0000 (oldest deadline 2024-04-26T22:01:06.398950+0000) 2024-04-26T22:01:34.065 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:33 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:33.635+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-04-26T22:00:51.700557+0000 front 2024-04-26T22:00:51.700671+0000 (oldest deadline 2024-04-26T22:01:18.701605+0000) 2024-04-26T22:01:34.065 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:33 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:33.635+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-04-26T22:01:02.104474+0000 front 2024-04-26T22:01:02.104287+0000 (oldest deadline 2024-04-26T22:01:26.803970+0000) 2024-04-26T22:01:34.296 INFO:journalctl@ceph.osd.8.smithi164.stdout:Apr 26 22:01:33 smithi164 podman[66806]: 2024-04-26 22:01:33.953371049 +0000 UTC m=+0.933922003 container died 7ede0cd7228c82af0094db17e0b8c34e212a75e2c82166ee468aab64ac15209f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-8-deactivate, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T22:01:34.572 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:01:34 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:01:34.305+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:16.595921+0000 front 2024-04-26T22:00:16.596275+0000 (oldest deadline 2024-04-26T22:00:39.495420+0000) 2024-04-26T22:01:34.572 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:01:34 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:01:34.305+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:24.600976+0000 front 2024-04-26T22:00:24.600553+0000 (oldest deadline 2024-04-26T22:00:48.100712+0000) 2024-04-26T22:01:34.572 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:01:34 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:01:34.305+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:34.002793+0000 front 2024-04-26T22:00:34.003098+0000 (oldest deadline 2024-04-26T22:00:58.103086+0000) 2024-04-26T22:01:34.572 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:01:34 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:01:34.305+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-26T22:00:41.004997+0000 front 2024-04-26T22:00:41.005270+0000 (oldest deadline 2024-04-26T22:01:08.006242+0000) 2024-04-26T22:01:34.572 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:01:34 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:01:34.305+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-04-26T22:00:59.511853+0000 front 2024-04-26T22:00:59.511966+0000 (oldest deadline 2024-04-26T22:01:25.411971+0000) 2024-04-26T22:01:34.635 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:34 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:34.400+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:14.343204+0000 front 2024-04-26T22:00:14.342815+0000 (oldest deadline 2024-04-26T22:00:39.042910+0000) 2024-04-26T22:01:34.635 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:34 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:34.400+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:24.945823+0000 front 2024-04-26T22:00:24.945339+0000 (oldest deadline 2024-04-26T22:00:47.245223+0000) 2024-04-26T22:01:34.635 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:34 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:34.400+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:31.947744+0000 front 2024-04-26T22:00:31.948080+0000 (oldest deadline 2024-04-26T22:00:56.647838+0000) 2024-04-26T22:01:34.636 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:34 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:34.400+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-26T22:00:42.953004+0000 front 2024-04-26T22:00:42.952908+0000 (oldest deadline 2024-04-26T22:01:07.652653+0000) 2024-04-26T22:01:34.636 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:34 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:34.400+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-04-26T22:00:53.256824+0000 front 2024-04-26T22:00:53.256989+0000 (oldest deadline 2024-04-26T22:01:18.556986+0000) 2024-04-26T22:01:34.636 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:34 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:34.400+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-04-26T22:00:58.557938+0000 front 2024-04-26T22:00:58.558225+0000 (oldest deadline 2024-04-26T22:01:24.457775+0000) 2024-04-26T22:01:34.686 DEBUG:teuthology.orchestra.run.smithi164:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a1780950-0415-11ef-bc93-c7b262605968@osd.8.service' 2024-04-26T22:01:34.797 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T22:01:34.797 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-04-26T22:01:34.798 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-04-26T22:01:34.798 DEBUG:teuthology.orchestra.run.smithi072:> sudo systemctl stop ceph-a1780950-0415-11ef-bc93-c7b262605968@osd.9 2024-04-26T22:01:35.064 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:34 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:34.614+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:11.890629+0000 front 2024-04-26T22:00:11.890849+0000 (oldest deadline 2024-04-26T22:00:40.692043+0000) 2024-04-26T22:01:35.065 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:34 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:34.614+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:24.194239+0000 front 2024-04-26T22:00:24.194095+0000 (oldest deadline 2024-04-26T22:00:48.893996+0000) 2024-04-26T22:01:35.065 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:34 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:34.614+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:34.096788+0000 front 2024-04-26T22:00:34.096884+0000 (oldest deadline 2024-04-26T22:00:56.396862+0000) 2024-04-26T22:01:35.065 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:34 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:34.614+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-26T22:00:42.299288+0000 front 2024-04-26T22:00:42.299033+0000 (oldest deadline 2024-04-26T22:01:06.398950+0000) 2024-04-26T22:01:35.065 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:34 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:34.614+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-04-26T22:00:51.700557+0000 front 2024-04-26T22:00:51.700671+0000 (oldest deadline 2024-04-26T22:01:18.701605+0000) 2024-04-26T22:01:35.065 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:34 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:34.614+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-04-26T22:01:02.104474+0000 front 2024-04-26T22:01:02.104287+0000 (oldest deadline 2024-04-26T22:01:26.803970+0000) 2024-04-26T22:01:35.322 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:01:34 smithi072 systemd[1]: Stopping Ceph osd.9 for a1780950-0415-11ef-bc93-c7b262605968... 2024-04-26T22:01:35.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:35 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:35.356+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:14.343204+0000 front 2024-04-26T22:00:14.342815+0000 (oldest deadline 2024-04-26T22:00:39.042910+0000) 2024-04-26T22:01:35.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:35 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:35.356+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:24.945823+0000 front 2024-04-26T22:00:24.945339+0000 (oldest deadline 2024-04-26T22:00:47.245223+0000) 2024-04-26T22:01:35.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:35 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:35.356+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:31.947744+0000 front 2024-04-26T22:00:31.948080+0000 (oldest deadline 2024-04-26T22:00:56.647838+0000) 2024-04-26T22:01:35.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:35 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:35.356+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-26T22:00:42.953004+0000 front 2024-04-26T22:00:42.952908+0000 (oldest deadline 2024-04-26T22:01:07.652653+0000) 2024-04-26T22:01:35.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:35 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:35.356+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-04-26T22:00:53.256824+0000 front 2024-04-26T22:00:53.256989+0000 (oldest deadline 2024-04-26T22:01:18.556986+0000) 2024-04-26T22:01:35.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:35 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:35.356+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-04-26T22:00:58.557938+0000 front 2024-04-26T22:00:58.558225+0000 (oldest deadline 2024-04-26T22:01:24.457775+0000) 2024-04-26T22:01:35.822 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:01:35 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:01:35.343+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:16.595921+0000 front 2024-04-26T22:00:16.596275+0000 (oldest deadline 2024-04-26T22:00:39.495420+0000) 2024-04-26T22:01:35.822 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:01:35 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:01:35.343+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:24.600976+0000 front 2024-04-26T22:00:24.600553+0000 (oldest deadline 2024-04-26T22:00:48.100712+0000) 2024-04-26T22:01:35.822 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:01:35 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:01:35.343+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:34.002793+0000 front 2024-04-26T22:00:34.003098+0000 (oldest deadline 2024-04-26T22:00:58.103086+0000) 2024-04-26T22:01:35.822 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:01:35 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:01:35.343+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-26T22:00:41.004997+0000 front 2024-04-26T22:00:41.005270+0000 (oldest deadline 2024-04-26T22:01:08.006242+0000) 2024-04-26T22:01:35.822 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:01:35 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:01:35.343+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-04-26T22:00:59.511853+0000 front 2024-04-26T22:00:59.511966+0000 (oldest deadline 2024-04-26T22:01:25.411971+0000) 2024-04-26T22:01:35.823 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:01:35 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:01:35.379+0000 7f7860e00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T22:01:35.823 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:01:35 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:01:35.379+0000 7f7860e00700 -1 osd.9 94 *** Got signal Terminated *** 2024-04-26T22:01:35.823 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:01:35 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:01:35.379+0000 7f7860e00700 -1 osd.9 94 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-26T22:01:36.064 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:35 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:35.636+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:11.890629+0000 front 2024-04-26T22:00:11.890849+0000 (oldest deadline 2024-04-26T22:00:40.692043+0000) 2024-04-26T22:01:36.065 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:35 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:35.636+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:24.194239+0000 front 2024-04-26T22:00:24.194095+0000 (oldest deadline 2024-04-26T22:00:48.893996+0000) 2024-04-26T22:01:36.065 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:35 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:35.636+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:34.096788+0000 front 2024-04-26T22:00:34.096884+0000 (oldest deadline 2024-04-26T22:00:56.396862+0000) 2024-04-26T22:01:36.065 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:35 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:35.636+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-26T22:00:42.299288+0000 front 2024-04-26T22:00:42.299033+0000 (oldest deadline 2024-04-26T22:01:06.398950+0000) 2024-04-26T22:01:36.065 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:35 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:35.636+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-04-26T22:00:51.700557+0000 front 2024-04-26T22:00:51.700671+0000 (oldest deadline 2024-04-26T22:01:18.701605+0000) 2024-04-26T22:01:36.065 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:35 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:35.636+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-04-26T22:01:02.104474+0000 front 2024-04-26T22:01:02.104287+0000 (oldest deadline 2024-04-26T22:01:26.803970+0000) 2024-04-26T22:01:36.065 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:35 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:35.636+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.72:6822 osd.6 since back 2024-04-26T22:01:12.606658+0000 front 2024-04-26T22:01:12.606850+0000 (oldest deadline 2024-04-26T22:01:34.906684+0000) 2024-04-26T22:01:36.594 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:36 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:36.334+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:14.343204+0000 front 2024-04-26T22:00:14.342815+0000 (oldest deadline 2024-04-26T22:00:39.042910+0000) 2024-04-26T22:01:36.594 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:36 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:36.334+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:24.945823+0000 front 2024-04-26T22:00:24.945339+0000 (oldest deadline 2024-04-26T22:00:47.245223+0000) 2024-04-26T22:01:36.594 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:36 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:36.334+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:31.947744+0000 front 2024-04-26T22:00:31.948080+0000 (oldest deadline 2024-04-26T22:00:56.647838+0000) 2024-04-26T22:01:36.594 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:36 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:36.334+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-26T22:00:42.953004+0000 front 2024-04-26T22:00:42.952908+0000 (oldest deadline 2024-04-26T22:01:07.652653+0000) 2024-04-26T22:01:36.594 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:36 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:36.334+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-04-26T22:00:53.256824+0000 front 2024-04-26T22:00:53.256989+0000 (oldest deadline 2024-04-26T22:01:18.556986+0000) 2024-04-26T22:01:36.594 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:36 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:36.334+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-04-26T22:00:58.557938+0000 front 2024-04-26T22:00:58.558225+0000 (oldest deadline 2024-04-26T22:01:24.457775+0000) 2024-04-26T22:01:36.822 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:01:36 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:01:36.377+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:16.595921+0000 front 2024-04-26T22:00:16.596275+0000 (oldest deadline 2024-04-26T22:00:39.495420+0000) 2024-04-26T22:01:36.822 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:01:36 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:01:36.377+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:24.600976+0000 front 2024-04-26T22:00:24.600553+0000 (oldest deadline 2024-04-26T22:00:48.100712+0000) 2024-04-26T22:01:36.822 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:01:36 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:01:36.377+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:34.002793+0000 front 2024-04-26T22:00:34.003098+0000 (oldest deadline 2024-04-26T22:00:58.103086+0000) 2024-04-26T22:01:36.822 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:01:36 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:01:36.377+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-26T22:00:41.004997+0000 front 2024-04-26T22:00:41.005270+0000 (oldest deadline 2024-04-26T22:01:08.006242+0000) 2024-04-26T22:01:36.822 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:01:36 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:01:36.377+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-04-26T22:00:59.511853+0000 front 2024-04-26T22:00:59.511966+0000 (oldest deadline 2024-04-26T22:01:25.411971+0000) 2024-04-26T22:01:36.822 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:01:36 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:01:36.377+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.72:6822 osd.6 since back 2024-04-26T22:01:10.614902+0000 front 2024-04-26T22:01:10.614415+0000 (oldest deadline 2024-04-26T22:01:35.914143+0000) 2024-04-26T22:01:37.065 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:36 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:36.631+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:11.890629+0000 front 2024-04-26T22:00:11.890849+0000 (oldest deadline 2024-04-26T22:00:40.692043+0000) 2024-04-26T22:01:37.065 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:36 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:36.631+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:24.194239+0000 front 2024-04-26T22:00:24.194095+0000 (oldest deadline 2024-04-26T22:00:48.893996+0000) 2024-04-26T22:01:37.065 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:36 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:36.631+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:34.096788+0000 front 2024-04-26T22:00:34.096884+0000 (oldest deadline 2024-04-26T22:00:56.396862+0000) 2024-04-26T22:01:37.065 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:36 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:36.631+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-26T22:00:42.299288+0000 front 2024-04-26T22:00:42.299033+0000 (oldest deadline 2024-04-26T22:01:06.398950+0000) 2024-04-26T22:01:37.065 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:36 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:36.631+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-04-26T22:00:51.700557+0000 front 2024-04-26T22:00:51.700671+0000 (oldest deadline 2024-04-26T22:01:18.701605+0000) 2024-04-26T22:01:37.065 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:36 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:36.631+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-04-26T22:01:02.104474+0000 front 2024-04-26T22:01:02.104287+0000 (oldest deadline 2024-04-26T22:01:26.803970+0000) 2024-04-26T22:01:37.065 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:36 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:36.631+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.72:6822 osd.6 since back 2024-04-26T22:01:12.606658+0000 front 2024-04-26T22:01:12.606850+0000 (oldest deadline 2024-04-26T22:01:34.906684+0000) 2024-04-26T22:01:37.617 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:37 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:37.346+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:14.343204+0000 front 2024-04-26T22:00:14.342815+0000 (oldest deadline 2024-04-26T22:00:39.042910+0000) 2024-04-26T22:01:37.618 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:37 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:37.346+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:24.945823+0000 front 2024-04-26T22:00:24.945339+0000 (oldest deadline 2024-04-26T22:00:47.245223+0000) 2024-04-26T22:01:37.618 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:37 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:37.346+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:31.947744+0000 front 2024-04-26T22:00:31.948080+0000 (oldest deadline 2024-04-26T22:00:56.647838+0000) 2024-04-26T22:01:37.618 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:37 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:37.346+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-26T22:00:42.953004+0000 front 2024-04-26T22:00:42.952908+0000 (oldest deadline 2024-04-26T22:01:07.652653+0000) 2024-04-26T22:01:37.618 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:37 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:37.346+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-04-26T22:00:53.256824+0000 front 2024-04-26T22:00:53.256989+0000 (oldest deadline 2024-04-26T22:01:18.556986+0000) 2024-04-26T22:01:37.618 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:37 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:37.346+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-04-26T22:00:58.557938+0000 front 2024-04-26T22:00:58.558225+0000 (oldest deadline 2024-04-26T22:01:24.457775+0000) 2024-04-26T22:01:37.822 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:01:37 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:01:37.379+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:16.595921+0000 front 2024-04-26T22:00:16.596275+0000 (oldest deadline 2024-04-26T22:00:39.495420+0000) 2024-04-26T22:01:37.822 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:01:37 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:01:37.379+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:24.600976+0000 front 2024-04-26T22:00:24.600553+0000 (oldest deadline 2024-04-26T22:00:48.100712+0000) 2024-04-26T22:01:37.822 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:01:37 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:01:37.379+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:34.002793+0000 front 2024-04-26T22:00:34.003098+0000 (oldest deadline 2024-04-26T22:00:58.103086+0000) 2024-04-26T22:01:37.822 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:01:37 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:01:37.379+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-26T22:00:41.004997+0000 front 2024-04-26T22:00:41.005270+0000 (oldest deadline 2024-04-26T22:01:08.006242+0000) 2024-04-26T22:01:37.822 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:01:37 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:01:37.379+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-04-26T22:00:59.511853+0000 front 2024-04-26T22:00:59.511966+0000 (oldest deadline 2024-04-26T22:01:25.411971+0000) 2024-04-26T22:01:37.822 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:01:37 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:01:37.379+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.72:6822 osd.6 since back 2024-04-26T22:01:10.614902+0000 front 2024-04-26T22:01:10.614415+0000 (oldest deadline 2024-04-26T22:01:35.914143+0000) 2024-04-26T22:01:38.065 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:37 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:37.621+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:11.890629+0000 front 2024-04-26T22:00:11.890849+0000 (oldest deadline 2024-04-26T22:00:40.692043+0000) 2024-04-26T22:01:38.065 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:37 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:37.621+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:24.194239+0000 front 2024-04-26T22:00:24.194095+0000 (oldest deadline 2024-04-26T22:00:48.893996+0000) 2024-04-26T22:01:38.065 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:37 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:37.621+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:34.096788+0000 front 2024-04-26T22:00:34.096884+0000 (oldest deadline 2024-04-26T22:00:56.396862+0000) 2024-04-26T22:01:38.065 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:37 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:37.621+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-26T22:00:42.299288+0000 front 2024-04-26T22:00:42.299033+0000 (oldest deadline 2024-04-26T22:01:06.398950+0000) 2024-04-26T22:01:38.065 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:37 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:37.621+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-04-26T22:00:51.700557+0000 front 2024-04-26T22:00:51.700671+0000 (oldest deadline 2024-04-26T22:01:18.701605+0000) 2024-04-26T22:01:38.065 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:37 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:37.621+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-04-26T22:01:02.104474+0000 front 2024-04-26T22:01:02.104287+0000 (oldest deadline 2024-04-26T22:01:26.803970+0000) 2024-04-26T22:01:38.065 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:37 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:37.621+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.72:6822 osd.6 since back 2024-04-26T22:01:12.606658+0000 front 2024-04-26T22:01:12.606850+0000 (oldest deadline 2024-04-26T22:01:34.906684+0000) 2024-04-26T22:01:38.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:38 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:38.302+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:14.343204+0000 front 2024-04-26T22:00:14.342815+0000 (oldest deadline 2024-04-26T22:00:39.042910+0000) 2024-04-26T22:01:38.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:38 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:38.302+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:24.945823+0000 front 2024-04-26T22:00:24.945339+0000 (oldest deadline 2024-04-26T22:00:47.245223+0000) 2024-04-26T22:01:38.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:38 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:38.302+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:31.947744+0000 front 2024-04-26T22:00:31.948080+0000 (oldest deadline 2024-04-26T22:00:56.647838+0000) 2024-04-26T22:01:38.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:38 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:38.302+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-26T22:00:42.953004+0000 front 2024-04-26T22:00:42.952908+0000 (oldest deadline 2024-04-26T22:01:07.652653+0000) 2024-04-26T22:01:38.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:38 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:38.302+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-04-26T22:00:53.256824+0000 front 2024-04-26T22:00:53.256989+0000 (oldest deadline 2024-04-26T22:01:18.556986+0000) 2024-04-26T22:01:38.692 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:38 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:38.302+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-04-26T22:00:58.557938+0000 front 2024-04-26T22:00:58.558225+0000 (oldest deadline 2024-04-26T22:01:24.457775+0000) 2024-04-26T22:01:38.822 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:01:38 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:01:38.414+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:16.595921+0000 front 2024-04-26T22:00:16.596275+0000 (oldest deadline 2024-04-26T22:00:39.495420+0000) 2024-04-26T22:01:38.822 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:01:38 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:01:38.414+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:24.600976+0000 front 2024-04-26T22:00:24.600553+0000 (oldest deadline 2024-04-26T22:00:48.100712+0000) 2024-04-26T22:01:38.822 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:01:38 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:01:38.414+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:34.002793+0000 front 2024-04-26T22:00:34.003098+0000 (oldest deadline 2024-04-26T22:00:58.103086+0000) 2024-04-26T22:01:38.822 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:01:38 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:01:38.414+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-26T22:00:41.004997+0000 front 2024-04-26T22:00:41.005270+0000 (oldest deadline 2024-04-26T22:01:08.006242+0000) 2024-04-26T22:01:38.822 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:01:38 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:01:38.414+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-04-26T22:00:59.511853+0000 front 2024-04-26T22:00:59.511966+0000 (oldest deadline 2024-04-26T22:01:25.411971+0000) 2024-04-26T22:01:38.822 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:01:38 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:01:38.414+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.72:6822 osd.6 since back 2024-04-26T22:01:10.614902+0000 front 2024-04-26T22:01:10.614415+0000 (oldest deadline 2024-04-26T22:01:35.914143+0000) 2024-04-26T22:01:39.064 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:38 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:38.626+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:11.890629+0000 front 2024-04-26T22:00:11.890849+0000 (oldest deadline 2024-04-26T22:00:40.692043+0000) 2024-04-26T22:01:39.065 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:38 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:38.626+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:24.194239+0000 front 2024-04-26T22:00:24.194095+0000 (oldest deadline 2024-04-26T22:00:48.893996+0000) 2024-04-26T22:01:39.065 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:38 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:38.626+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:34.096788+0000 front 2024-04-26T22:00:34.096884+0000 (oldest deadline 2024-04-26T22:00:56.396862+0000) 2024-04-26T22:01:39.065 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:38 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:38.626+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-26T22:00:42.299288+0000 front 2024-04-26T22:00:42.299033+0000 (oldest deadline 2024-04-26T22:01:06.398950+0000) 2024-04-26T22:01:39.065 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:38 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:38.626+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-04-26T22:00:51.700557+0000 front 2024-04-26T22:00:51.700671+0000 (oldest deadline 2024-04-26T22:01:18.701605+0000) 2024-04-26T22:01:39.065 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:38 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:38.626+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-04-26T22:01:02.104474+0000 front 2024-04-26T22:01:02.104287+0000 (oldest deadline 2024-04-26T22:01:26.803970+0000) 2024-04-26T22:01:39.065 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:38 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:38.626+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.72:6822 osd.6 since back 2024-04-26T22:01:12.606658+0000 front 2024-04-26T22:01:12.606850+0000 (oldest deadline 2024-04-26T22:01:34.906684+0000) 2024-04-26T22:01:39.441 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:39 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:39.293+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:14.343204+0000 front 2024-04-26T22:00:14.342815+0000 (oldest deadline 2024-04-26T22:00:39.042910+0000) 2024-04-26T22:01:39.441 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:39 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:39.293+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:24.945823+0000 front 2024-04-26T22:00:24.945339+0000 (oldest deadline 2024-04-26T22:00:47.245223+0000) 2024-04-26T22:01:39.441 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:39 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:39.293+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:31.947744+0000 front 2024-04-26T22:00:31.948080+0000 (oldest deadline 2024-04-26T22:00:56.647838+0000) 2024-04-26T22:01:39.441 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:39 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:39.293+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-26T22:00:42.953004+0000 front 2024-04-26T22:00:42.952908+0000 (oldest deadline 2024-04-26T22:01:07.652653+0000) 2024-04-26T22:01:39.441 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:39 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:39.293+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-04-26T22:00:53.256824+0000 front 2024-04-26T22:00:53.256989+0000 (oldest deadline 2024-04-26T22:01:18.556986+0000) 2024-04-26T22:01:39.441 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:39 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:39.293+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-04-26T22:00:58.557938+0000 front 2024-04-26T22:00:58.558225+0000 (oldest deadline 2024-04-26T22:01:24.457775+0000) 2024-04-26T22:01:39.724 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:01:39 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:01:39.449+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:16.595921+0000 front 2024-04-26T22:00:16.596275+0000 (oldest deadline 2024-04-26T22:00:39.495420+0000) 2024-04-26T22:01:39.724 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:01:39 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:01:39.449+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:24.600976+0000 front 2024-04-26T22:00:24.600553+0000 (oldest deadline 2024-04-26T22:00:48.100712+0000) 2024-04-26T22:01:39.724 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:01:39 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:01:39.449+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:34.002793+0000 front 2024-04-26T22:00:34.003098+0000 (oldest deadline 2024-04-26T22:00:58.103086+0000) 2024-04-26T22:01:39.724 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:01:39 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:01:39.449+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-26T22:00:41.004997+0000 front 2024-04-26T22:00:41.005270+0000 (oldest deadline 2024-04-26T22:01:08.006242+0000) 2024-04-26T22:01:39.724 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:01:39 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:01:39.449+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-04-26T22:00:59.511853+0000 front 2024-04-26T22:00:59.511966+0000 (oldest deadline 2024-04-26T22:01:25.411971+0000) 2024-04-26T22:01:39.724 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:01:39 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9[63797]: 2024-04-26T22:01:39.449+0000 7f7857600700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.72:6822 osd.6 since back 2024-04-26T22:01:10.614902+0000 front 2024-04-26T22:01:10.614415+0000 (oldest deadline 2024-04-26T22:01:35.914143+0000) 2024-04-26T22:01:40.064 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:39 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:39.660+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:11.890629+0000 front 2024-04-26T22:00:11.890849+0000 (oldest deadline 2024-04-26T22:00:40.692043+0000) 2024-04-26T22:01:40.064 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:39 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:39.660+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:24.194239+0000 front 2024-04-26T22:00:24.194095+0000 (oldest deadline 2024-04-26T22:00:48.893996+0000) 2024-04-26T22:01:40.065 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:39 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:39.660+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:34.096788+0000 front 2024-04-26T22:00:34.096884+0000 (oldest deadline 2024-04-26T22:00:56.396862+0000) 2024-04-26T22:01:40.065 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:39 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:39.660+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-26T22:00:42.299288+0000 front 2024-04-26T22:00:42.299033+0000 (oldest deadline 2024-04-26T22:01:06.398950+0000) 2024-04-26T22:01:40.065 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:39 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:39.660+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-04-26T22:00:51.700557+0000 front 2024-04-26T22:00:51.700671+0000 (oldest deadline 2024-04-26T22:01:18.701605+0000) 2024-04-26T22:01:40.065 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:39 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:39.660+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-04-26T22:01:02.104474+0000 front 2024-04-26T22:01:02.104287+0000 (oldest deadline 2024-04-26T22:01:26.803970+0000) 2024-04-26T22:01:40.065 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:39 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:39.660+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.72:6822 osd.6 since back 2024-04-26T22:01:12.606658+0000 front 2024-04-26T22:01:12.606850+0000 (oldest deadline 2024-04-26T22:01:34.906684+0000) 2024-04-26T22:01:40.690 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:40 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:40.313+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:14.343204+0000 front 2024-04-26T22:00:14.342815+0000 (oldest deadline 2024-04-26T22:00:39.042910+0000) 2024-04-26T22:01:40.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:40 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:40.313+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:24.945823+0000 front 2024-04-26T22:00:24.945339+0000 (oldest deadline 2024-04-26T22:00:47.245223+0000) 2024-04-26T22:01:40.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:40 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:40.313+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:31.947744+0000 front 2024-04-26T22:00:31.948080+0000 (oldest deadline 2024-04-26T22:00:56.647838+0000) 2024-04-26T22:01:40.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:40 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:40.313+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-26T22:00:42.953004+0000 front 2024-04-26T22:00:42.952908+0000 (oldest deadline 2024-04-26T22:01:07.652653+0000) 2024-04-26T22:01:40.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:40 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:40.313+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-04-26T22:00:53.256824+0000 front 2024-04-26T22:00:53.256989+0000 (oldest deadline 2024-04-26T22:01:18.556986+0000) 2024-04-26T22:01:40.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:40 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:40.313+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-04-26T22:00:58.557938+0000 front 2024-04-26T22:00:58.558225+0000 (oldest deadline 2024-04-26T22:01:24.457775+0000) 2024-04-26T22:01:41.064 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:40 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:40.681+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:11.890629+0000 front 2024-04-26T22:00:11.890849+0000 (oldest deadline 2024-04-26T22:00:40.692043+0000) 2024-04-26T22:01:41.065 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:40 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:40.681+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:24.194239+0000 front 2024-04-26T22:00:24.194095+0000 (oldest deadline 2024-04-26T22:00:48.893996+0000) 2024-04-26T22:01:41.065 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:40 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:40.681+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:34.096788+0000 front 2024-04-26T22:00:34.096884+0000 (oldest deadline 2024-04-26T22:00:56.396862+0000) 2024-04-26T22:01:41.065 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:40 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:40.681+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-26T22:00:42.299288+0000 front 2024-04-26T22:00:42.299033+0000 (oldest deadline 2024-04-26T22:01:06.398950+0000) 2024-04-26T22:01:41.065 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:40 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:40.681+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-04-26T22:00:51.700557+0000 front 2024-04-26T22:00:51.700671+0000 (oldest deadline 2024-04-26T22:01:18.701605+0000) 2024-04-26T22:01:41.065 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:40 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:40.681+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-04-26T22:01:02.104474+0000 front 2024-04-26T22:01:02.104287+0000 (oldest deadline 2024-04-26T22:01:26.803970+0000) 2024-04-26T22:01:41.065 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:40 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:40.681+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.72:6822 osd.6 since back 2024-04-26T22:01:12.606658+0000 front 2024-04-26T22:01:12.606850+0000 (oldest deadline 2024-04-26T22:01:34.906684+0000) 2024-04-26T22:01:41.220 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:01:40 smithi072 podman[89383]: 2024-04-26 22:01:40.919769614 +0000 UTC m=+5.681508096 container died 9741ab8d063bd8ed5bb5d953fe8aac1bb4687c09313207ae719165145d1de364 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True) 2024-04-26T22:01:41.572 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:01:41 smithi072 podman[89383]: 2024-04-26 22:01:41.342031396 +0000 UTC m=+6.103769928 container cleanup 9741ab8d063bd8ed5bb5d953fe8aac1bb4687c09313207ae719165145d1de364 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, io.buildah.version=1.33.5) 2024-04-26T22:01:41.572 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:01:41 smithi072 bash[89383]: ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9 2024-04-26T22:01:41.690 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:41 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:41.267+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:14.343204+0000 front 2024-04-26T22:00:14.342815+0000 (oldest deadline 2024-04-26T22:00:39.042910+0000) 2024-04-26T22:01:41.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:41 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:41.267+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:24.945823+0000 front 2024-04-26T22:00:24.945339+0000 (oldest deadline 2024-04-26T22:00:47.245223+0000) 2024-04-26T22:01:41.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:41 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:41.267+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:31.947744+0000 front 2024-04-26T22:00:31.948080+0000 (oldest deadline 2024-04-26T22:00:56.647838+0000) 2024-04-26T22:01:41.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:41 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:41.267+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-26T22:00:42.953004+0000 front 2024-04-26T22:00:42.952908+0000 (oldest deadline 2024-04-26T22:01:07.652653+0000) 2024-04-26T22:01:41.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:41 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:41.267+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-04-26T22:00:53.256824+0000 front 2024-04-26T22:00:53.256989+0000 (oldest deadline 2024-04-26T22:01:18.556986+0000) 2024-04-26T22:01:41.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:41 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:41.267+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-04-26T22:00:58.557938+0000 front 2024-04-26T22:00:58.558225+0000 (oldest deadline 2024-04-26T22:01:24.457775+0000) 2024-04-26T22:01:42.064 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:41 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:41.690+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:11.890629+0000 front 2024-04-26T22:00:11.890849+0000 (oldest deadline 2024-04-26T22:00:40.692043+0000) 2024-04-26T22:01:42.065 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:41 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:41.690+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:24.194239+0000 front 2024-04-26T22:00:24.194095+0000 (oldest deadline 2024-04-26T22:00:48.893996+0000) 2024-04-26T22:01:42.065 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:41 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:41.690+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:34.096788+0000 front 2024-04-26T22:00:34.096884+0000 (oldest deadline 2024-04-26T22:00:56.396862+0000) 2024-04-26T22:01:42.065 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:41 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:41.690+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-26T22:00:42.299288+0000 front 2024-04-26T22:00:42.299033+0000 (oldest deadline 2024-04-26T22:01:06.398950+0000) 2024-04-26T22:01:42.065 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:41 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:41.690+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-04-26T22:00:51.700557+0000 front 2024-04-26T22:00:51.700671+0000 (oldest deadline 2024-04-26T22:01:18.701605+0000) 2024-04-26T22:01:42.065 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:41 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:41.690+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-04-26T22:01:02.104474+0000 front 2024-04-26T22:01:02.104287+0000 (oldest deadline 2024-04-26T22:01:26.803970+0000) 2024-04-26T22:01:42.065 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:41 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:41.690+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.72:6822 osd.6 since back 2024-04-26T22:01:12.606658+0000 front 2024-04-26T22:01:12.606850+0000 (oldest deadline 2024-04-26T22:01:34.906684+0000) 2024-04-26T22:01:42.071 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:01:41 smithi072 podman[89401]: 2024-04-26 22:01:41.677429815 +0000 UTC m=+0.749074625 container remove 9741ab8d063bd8ed5bb5d953fe8aac1bb4687c09313207ae719165145d1de364 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240424, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-04-26T22:01:42.690 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:42 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:42.240+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:14.343204+0000 front 2024-04-26T22:00:14.342815+0000 (oldest deadline 2024-04-26T22:00:39.042910+0000) 2024-04-26T22:01:42.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:42 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:42.240+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:24.945823+0000 front 2024-04-26T22:00:24.945339+0000 (oldest deadline 2024-04-26T22:00:47.245223+0000) 2024-04-26T22:01:42.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:42 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:42.240+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:31.947744+0000 front 2024-04-26T22:00:31.948080+0000 (oldest deadline 2024-04-26T22:00:56.647838+0000) 2024-04-26T22:01:42.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:42 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:42.240+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-26T22:00:42.953004+0000 front 2024-04-26T22:00:42.952908+0000 (oldest deadline 2024-04-26T22:01:07.652653+0000) 2024-04-26T22:01:42.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:42 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:42.240+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-04-26T22:00:53.256824+0000 front 2024-04-26T22:00:53.256989+0000 (oldest deadline 2024-04-26T22:01:18.556986+0000) 2024-04-26T22:01:42.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:42 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:42.240+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-04-26T22:00:58.557938+0000 front 2024-04-26T22:00:58.558225+0000 (oldest deadline 2024-04-26T22:01:24.457775+0000) 2024-04-26T22:01:42.822 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:01:42 smithi072 podman[89493]: 2024-04-26 22:01:42.460261417 +0000 UTC m=+0.036439778 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-26T22:01:42.822 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:01:42 smithi072 podman[89493]: 2024-04-26 22:01:42.58737615 +0000 UTC m=+0.163554529 container create 2f34266078fbba9fe6340cd3684cb9cfd59db75e93927e9e41d49888d5e0d784 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9-deactivate, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424) 2024-04-26T22:01:43.064 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:42 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:42.731+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:11.890629+0000 front 2024-04-26T22:00:11.890849+0000 (oldest deadline 2024-04-26T22:00:40.692043+0000) 2024-04-26T22:01:43.065 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:42 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:42.731+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:24.194239+0000 front 2024-04-26T22:00:24.194095+0000 (oldest deadline 2024-04-26T22:00:48.893996+0000) 2024-04-26T22:01:43.065 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:42 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:42.731+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:34.096788+0000 front 2024-04-26T22:00:34.096884+0000 (oldest deadline 2024-04-26T22:00:56.396862+0000) 2024-04-26T22:01:43.065 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:42 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:42.731+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-26T22:00:42.299288+0000 front 2024-04-26T22:00:42.299033+0000 (oldest deadline 2024-04-26T22:01:06.398950+0000) 2024-04-26T22:01:43.065 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:42 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:42.731+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-04-26T22:00:51.700557+0000 front 2024-04-26T22:00:51.700671+0000 (oldest deadline 2024-04-26T22:01:18.701605+0000) 2024-04-26T22:01:43.065 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:42 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:42.731+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-04-26T22:01:02.104474+0000 front 2024-04-26T22:01:02.104287+0000 (oldest deadline 2024-04-26T22:01:26.803970+0000) 2024-04-26T22:01:43.065 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:42 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:42.731+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.72:6822 osd.6 since back 2024-04-26T22:01:12.606658+0000 front 2024-04-26T22:01:12.606850+0000 (oldest deadline 2024-04-26T22:01:34.906684+0000) 2024-04-26T22:01:43.322 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:01:42 smithi072 podman[89493]: 2024-04-26 22:01:42.85942006 +0000 UTC m=+0.435598492 container init 2f34266078fbba9fe6340cd3684cb9cfd59db75e93927e9e41d49888d5e0d784 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9-deactivate, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, CEPH_POINT_RELEASE=) 2024-04-26T22:01:43.322 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:01:42 smithi072 podman[89493]: 2024-04-26 22:01:42.870415773 +0000 UTC m=+0.446594161 container start 2f34266078fbba9fe6340cd3684cb9cfd59db75e93927e9e41d49888d5e0d784 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9-deactivate, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , ceph=True) 2024-04-26T22:01:43.322 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:01:42 smithi072 podman[89493]: 2024-04-26 22:01:42.960032168 +0000 UTC m=+0.536210541 container attach 2f34266078fbba9fe6340cd3684cb9cfd59db75e93927e9e41d49888d5e0d784 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9-deactivate, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T22:01:43.669 INFO:journalctl@ceph.osd.9.smithi072.stdout:Apr 26 22:01:43 smithi072 podman[89493]: 2024-04-26 22:01:43.327744377 +0000 UTC m=+0.903922759 container died 2f34266078fbba9fe6340cd3684cb9cfd59db75e93927e9e41d49888d5e0d784 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-9-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-04-26T22:01:43.690 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:43 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:43.280+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:14.343204+0000 front 2024-04-26T22:00:14.342815+0000 (oldest deadline 2024-04-26T22:00:39.042910+0000) 2024-04-26T22:01:43.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:43 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:43.280+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:24.945823+0000 front 2024-04-26T22:00:24.945339+0000 (oldest deadline 2024-04-26T22:00:47.245223+0000) 2024-04-26T22:01:43.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:43 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:43.280+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:31.947744+0000 front 2024-04-26T22:00:31.948080+0000 (oldest deadline 2024-04-26T22:00:56.647838+0000) 2024-04-26T22:01:43.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:43 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:43.280+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-26T22:00:42.953004+0000 front 2024-04-26T22:00:42.952908+0000 (oldest deadline 2024-04-26T22:01:07.652653+0000) 2024-04-26T22:01:43.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:43 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:43.280+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-04-26T22:00:53.256824+0000 front 2024-04-26T22:00:53.256989+0000 (oldest deadline 2024-04-26T22:01:18.556986+0000) 2024-04-26T22:01:43.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:43 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:43.280+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-04-26T22:00:58.557938+0000 front 2024-04-26T22:00:58.558225+0000 (oldest deadline 2024-04-26T22:01:24.457775+0000) 2024-04-26T22:01:44.064 DEBUG:teuthology.orchestra.run.smithi072:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a1780950-0415-11ef-bc93-c7b262605968@osd.9.service' 2024-04-26T22:01:44.065 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:43 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:43.714+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:11.890629+0000 front 2024-04-26T22:00:11.890849+0000 (oldest deadline 2024-04-26T22:00:40.692043+0000) 2024-04-26T22:01:44.065 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:43 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:43.714+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:24.194239+0000 front 2024-04-26T22:00:24.194095+0000 (oldest deadline 2024-04-26T22:00:48.893996+0000) 2024-04-26T22:01:44.065 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:43 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:43.714+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:34.096788+0000 front 2024-04-26T22:00:34.096884+0000 (oldest deadline 2024-04-26T22:00:56.396862+0000) 2024-04-26T22:01:44.066 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:43 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:43.714+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-26T22:00:42.299288+0000 front 2024-04-26T22:00:42.299033+0000 (oldest deadline 2024-04-26T22:01:06.398950+0000) 2024-04-26T22:01:44.066 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:43 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:43.715+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-04-26T22:00:51.700557+0000 front 2024-04-26T22:00:51.700671+0000 (oldest deadline 2024-04-26T22:01:18.701605+0000) 2024-04-26T22:01:44.066 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:43 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:43.715+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-04-26T22:01:02.104474+0000 front 2024-04-26T22:01:02.104287+0000 (oldest deadline 2024-04-26T22:01:26.803970+0000) 2024-04-26T22:01:44.066 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:43 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:43.715+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.72:6822 osd.6 since back 2024-04-26T22:01:12.606658+0000 front 2024-04-26T22:01:12.606850+0000 (oldest deadline 2024-04-26T22:01:34.906684+0000) 2024-04-26T22:01:44.177 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T22:01:44.178 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-04-26T22:01:44.178 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-04-26T22:01:44.178 DEBUG:teuthology.orchestra.run.smithi076:> sudo systemctl stop ceph-a1780950-0415-11ef-bc93-c7b262605968@osd.10 2024-04-26T22:01:44.564 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:44 smithi076 systemd[1]: Stopping Ceph osd.10 for a1780950-0415-11ef-bc93-c7b262605968... 2024-04-26T22:01:44.690 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:44 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:44.258+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:14.343204+0000 front 2024-04-26T22:00:14.342815+0000 (oldest deadline 2024-04-26T22:00:39.042910+0000) 2024-04-26T22:01:44.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:44 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:44.258+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:24.945823+0000 front 2024-04-26T22:00:24.945339+0000 (oldest deadline 2024-04-26T22:00:47.245223+0000) 2024-04-26T22:01:44.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:44 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:44.258+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:31.947744+0000 front 2024-04-26T22:00:31.948080+0000 (oldest deadline 2024-04-26T22:00:56.647838+0000) 2024-04-26T22:01:44.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:44 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:44.258+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-26T22:00:42.953004+0000 front 2024-04-26T22:00:42.952908+0000 (oldest deadline 2024-04-26T22:01:07.652653+0000) 2024-04-26T22:01:44.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:44 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:44.258+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-04-26T22:00:53.256824+0000 front 2024-04-26T22:00:53.256989+0000 (oldest deadline 2024-04-26T22:01:18.556986+0000) 2024-04-26T22:01:44.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:44 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:44.258+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-04-26T22:00:58.557938+0000 front 2024-04-26T22:00:58.558225+0000 (oldest deadline 2024-04-26T22:01:24.457775+0000) 2024-04-26T22:01:44.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:44 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:44.258+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.76:6820 osd.7 since back 2024-04-26T22:01:18.561488+0000 front 2024-04-26T22:01:18.561390+0000 (oldest deadline 2024-04-26T22:01:43.861188+0000) 2024-04-26T22:01:45.065 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:44 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:44.674+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:11.890629+0000 front 2024-04-26T22:00:11.890849+0000 (oldest deadline 2024-04-26T22:00:40.692043+0000) 2024-04-26T22:01:45.066 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:44 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:44.674+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:24.194239+0000 front 2024-04-26T22:00:24.194095+0000 (oldest deadline 2024-04-26T22:00:48.893996+0000) 2024-04-26T22:01:45.066 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:44 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:44.674+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:34.096788+0000 front 2024-04-26T22:00:34.096884+0000 (oldest deadline 2024-04-26T22:00:56.396862+0000) 2024-04-26T22:01:45.066 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:44 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:44.674+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-26T22:00:42.299288+0000 front 2024-04-26T22:00:42.299033+0000 (oldest deadline 2024-04-26T22:01:06.398950+0000) 2024-04-26T22:01:45.066 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:44 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:44.674+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-04-26T22:00:51.700557+0000 front 2024-04-26T22:00:51.700671+0000 (oldest deadline 2024-04-26T22:01:18.701605+0000) 2024-04-26T22:01:45.066 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:44 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:44.674+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-04-26T22:01:02.104474+0000 front 2024-04-26T22:01:02.104287+0000 (oldest deadline 2024-04-26T22:01:26.803970+0000) 2024-04-26T22:01:45.066 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:44 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:44.674+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.72:6822 osd.6 since back 2024-04-26T22:01:12.606658+0000 front 2024-04-26T22:01:12.606850+0000 (oldest deadline 2024-04-26T22:01:34.906684+0000) 2024-04-26T22:01:45.066 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:44 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:44.726+0000 7fa3c1e00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T22:01:45.066 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:44 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:44.726+0000 7fa3c1e00700 -1 osd.10 94 *** Got signal Terminated *** 2024-04-26T22:01:45.066 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:44 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:44.726+0000 7fa3c1e00700 -1 osd.10 94 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-26T22:01:45.690 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:45 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:45.255+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:14.343204+0000 front 2024-04-26T22:00:14.342815+0000 (oldest deadline 2024-04-26T22:00:39.042910+0000) 2024-04-26T22:01:45.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:45 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:45.255+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:24.945823+0000 front 2024-04-26T22:00:24.945339+0000 (oldest deadline 2024-04-26T22:00:47.245223+0000) 2024-04-26T22:01:45.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:45 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:45.255+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:31.947744+0000 front 2024-04-26T22:00:31.948080+0000 (oldest deadline 2024-04-26T22:00:56.647838+0000) 2024-04-26T22:01:45.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:45 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:45.255+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-26T22:00:42.953004+0000 front 2024-04-26T22:00:42.952908+0000 (oldest deadline 2024-04-26T22:01:07.652653+0000) 2024-04-26T22:01:45.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:45 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:45.255+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-04-26T22:00:53.256824+0000 front 2024-04-26T22:00:53.256989+0000 (oldest deadline 2024-04-26T22:01:18.556986+0000) 2024-04-26T22:01:45.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:45 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:45.255+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-04-26T22:00:58.557938+0000 front 2024-04-26T22:00:58.558225+0000 (oldest deadline 2024-04-26T22:01:24.457775+0000) 2024-04-26T22:01:45.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:45 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:45.255+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.76:6820 osd.7 since back 2024-04-26T22:01:18.561488+0000 front 2024-04-26T22:01:18.561390+0000 (oldest deadline 2024-04-26T22:01:43.861188+0000) 2024-04-26T22:01:46.064 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:45 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:45.699+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:11.890629+0000 front 2024-04-26T22:00:11.890849+0000 (oldest deadline 2024-04-26T22:00:40.692043+0000) 2024-04-26T22:01:46.065 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:45 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:45.699+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:24.194239+0000 front 2024-04-26T22:00:24.194095+0000 (oldest deadline 2024-04-26T22:00:48.893996+0000) 2024-04-26T22:01:46.065 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:45 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:45.699+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:34.096788+0000 front 2024-04-26T22:00:34.096884+0000 (oldest deadline 2024-04-26T22:00:56.396862+0000) 2024-04-26T22:01:46.065 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:45 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:45.699+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-26T22:00:42.299288+0000 front 2024-04-26T22:00:42.299033+0000 (oldest deadline 2024-04-26T22:01:06.398950+0000) 2024-04-26T22:01:46.065 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:45 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:45.699+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-04-26T22:00:51.700557+0000 front 2024-04-26T22:00:51.700671+0000 (oldest deadline 2024-04-26T22:01:18.701605+0000) 2024-04-26T22:01:46.065 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:45 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:45.699+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-04-26T22:01:02.104474+0000 front 2024-04-26T22:01:02.104287+0000 (oldest deadline 2024-04-26T22:01:26.803970+0000) 2024-04-26T22:01:46.065 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:45 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:45.699+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.72:6822 osd.6 since back 2024-04-26T22:01:12.606658+0000 front 2024-04-26T22:01:12.606850+0000 (oldest deadline 2024-04-26T22:01:34.906684+0000) 2024-04-26T22:01:46.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:46 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:46.264+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:14.343204+0000 front 2024-04-26T22:00:14.342815+0000 (oldest deadline 2024-04-26T22:00:39.042910+0000) 2024-04-26T22:01:46.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:46 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:46.264+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:24.945823+0000 front 2024-04-26T22:00:24.945339+0000 (oldest deadline 2024-04-26T22:00:47.245223+0000) 2024-04-26T22:01:46.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:46 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:46.264+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:31.947744+0000 front 2024-04-26T22:00:31.948080+0000 (oldest deadline 2024-04-26T22:00:56.647838+0000) 2024-04-26T22:01:46.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:46 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:46.264+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-26T22:00:42.953004+0000 front 2024-04-26T22:00:42.952908+0000 (oldest deadline 2024-04-26T22:01:07.652653+0000) 2024-04-26T22:01:46.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:46 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:46.264+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-04-26T22:00:53.256824+0000 front 2024-04-26T22:00:53.256989+0000 (oldest deadline 2024-04-26T22:01:18.556986+0000) 2024-04-26T22:01:46.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:46 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:46.264+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-04-26T22:00:58.557938+0000 front 2024-04-26T22:00:58.558225+0000 (oldest deadline 2024-04-26T22:01:24.457775+0000) 2024-04-26T22:01:46.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:46 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:46.264+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.76:6820 osd.7 since back 2024-04-26T22:01:18.561488+0000 front 2024-04-26T22:01:18.561390+0000 (oldest deadline 2024-04-26T22:01:43.861188+0000) 2024-04-26T22:01:46.763 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:46 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:46.668+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:11.890629+0000 front 2024-04-26T22:00:11.890849+0000 (oldest deadline 2024-04-26T22:00:40.692043+0000) 2024-04-26T22:01:46.763 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:46 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:46.668+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:24.194239+0000 front 2024-04-26T22:00:24.194095+0000 (oldest deadline 2024-04-26T22:00:48.893996+0000) 2024-04-26T22:01:46.763 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:46 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:46.668+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:34.096788+0000 front 2024-04-26T22:00:34.096884+0000 (oldest deadline 2024-04-26T22:00:56.396862+0000) 2024-04-26T22:01:46.763 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:46 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:46.668+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-26T22:00:42.299288+0000 front 2024-04-26T22:00:42.299033+0000 (oldest deadline 2024-04-26T22:01:06.398950+0000) 2024-04-26T22:01:46.763 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:46 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:46.668+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-04-26T22:00:51.700557+0000 front 2024-04-26T22:00:51.700671+0000 (oldest deadline 2024-04-26T22:01:18.701605+0000) 2024-04-26T22:01:46.764 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:46 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:46.668+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-04-26T22:01:02.104474+0000 front 2024-04-26T22:01:02.104287+0000 (oldest deadline 2024-04-26T22:01:26.803970+0000) 2024-04-26T22:01:46.764 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:46 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:46.668+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.72:6822 osd.6 since back 2024-04-26T22:01:12.606658+0000 front 2024-04-26T22:01:12.606850+0000 (oldest deadline 2024-04-26T22:01:34.906684+0000) 2024-04-26T22:01:47.690 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:47 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:47.277+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:14.343204+0000 front 2024-04-26T22:00:14.342815+0000 (oldest deadline 2024-04-26T22:00:39.042910+0000) 2024-04-26T22:01:47.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:47 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:47.277+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:24.945823+0000 front 2024-04-26T22:00:24.945339+0000 (oldest deadline 2024-04-26T22:00:47.245223+0000) 2024-04-26T22:01:47.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:47 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:47.277+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:31.947744+0000 front 2024-04-26T22:00:31.948080+0000 (oldest deadline 2024-04-26T22:00:56.647838+0000) 2024-04-26T22:01:47.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:47 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:47.277+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-26T22:00:42.953004+0000 front 2024-04-26T22:00:42.952908+0000 (oldest deadline 2024-04-26T22:01:07.652653+0000) 2024-04-26T22:01:47.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:47 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:47.277+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-04-26T22:00:53.256824+0000 front 2024-04-26T22:00:53.256989+0000 (oldest deadline 2024-04-26T22:01:18.556986+0000) 2024-04-26T22:01:47.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:47 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:47.277+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-04-26T22:00:58.557938+0000 front 2024-04-26T22:00:58.558225+0000 (oldest deadline 2024-04-26T22:01:24.457775+0000) 2024-04-26T22:01:47.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:47 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:47.277+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.76:6820 osd.7 since back 2024-04-26T22:01:18.561488+0000 front 2024-04-26T22:01:18.561390+0000 (oldest deadline 2024-04-26T22:01:43.861188+0000) 2024-04-26T22:01:48.064 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:47 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:47.636+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:11.890629+0000 front 2024-04-26T22:00:11.890849+0000 (oldest deadline 2024-04-26T22:00:40.692043+0000) 2024-04-26T22:01:48.065 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:47 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:47.636+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:24.194239+0000 front 2024-04-26T22:00:24.194095+0000 (oldest deadline 2024-04-26T22:00:48.893996+0000) 2024-04-26T22:01:48.065 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:47 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:47.636+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:34.096788+0000 front 2024-04-26T22:00:34.096884+0000 (oldest deadline 2024-04-26T22:00:56.396862+0000) 2024-04-26T22:01:48.065 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:47 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:47.636+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-26T22:00:42.299288+0000 front 2024-04-26T22:00:42.299033+0000 (oldest deadline 2024-04-26T22:01:06.398950+0000) 2024-04-26T22:01:48.065 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:47 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:47.636+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-04-26T22:00:51.700557+0000 front 2024-04-26T22:00:51.700671+0000 (oldest deadline 2024-04-26T22:01:18.701605+0000) 2024-04-26T22:01:48.065 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:47 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:47.636+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-04-26T22:01:02.104474+0000 front 2024-04-26T22:01:02.104287+0000 (oldest deadline 2024-04-26T22:01:26.803970+0000) 2024-04-26T22:01:48.065 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:47 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:47.636+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.72:6822 osd.6 since back 2024-04-26T22:01:12.606658+0000 front 2024-04-26T22:01:12.606850+0000 (oldest deadline 2024-04-26T22:01:34.906684+0000) 2024-04-26T22:01:48.639 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:48 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:48.300+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:14.343204+0000 front 2024-04-26T22:00:14.342815+0000 (oldest deadline 2024-04-26T22:00:39.042910+0000) 2024-04-26T22:01:48.640 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:48 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:48.300+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:24.945823+0000 front 2024-04-26T22:00:24.945339+0000 (oldest deadline 2024-04-26T22:00:47.245223+0000) 2024-04-26T22:01:48.640 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:48 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:48.300+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:31.947744+0000 front 2024-04-26T22:00:31.948080+0000 (oldest deadline 2024-04-26T22:00:56.647838+0000) 2024-04-26T22:01:48.640 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:48 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:48.300+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-26T22:00:42.953004+0000 front 2024-04-26T22:00:42.952908+0000 (oldest deadline 2024-04-26T22:01:07.652653+0000) 2024-04-26T22:01:48.640 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:48 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:48.300+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-04-26T22:00:53.256824+0000 front 2024-04-26T22:00:53.256989+0000 (oldest deadline 2024-04-26T22:01:18.556986+0000) 2024-04-26T22:01:48.640 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:48 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:48.300+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-04-26T22:00:58.557938+0000 front 2024-04-26T22:00:58.558225+0000 (oldest deadline 2024-04-26T22:01:24.457775+0000) 2024-04-26T22:01:48.640 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:48 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:48.300+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.76:6820 osd.7 since back 2024-04-26T22:01:18.561488+0000 front 2024-04-26T22:01:18.561390+0000 (oldest deadline 2024-04-26T22:01:43.861188+0000) 2024-04-26T22:01:49.064 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:48 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:48.590+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:11.890629+0000 front 2024-04-26T22:00:11.890849+0000 (oldest deadline 2024-04-26T22:00:40.692043+0000) 2024-04-26T22:01:49.064 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:48 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:48.590+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:24.194239+0000 front 2024-04-26T22:00:24.194095+0000 (oldest deadline 2024-04-26T22:00:48.893996+0000) 2024-04-26T22:01:49.065 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:48 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:48.590+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:34.096788+0000 front 2024-04-26T22:00:34.096884+0000 (oldest deadline 2024-04-26T22:00:56.396862+0000) 2024-04-26T22:01:49.065 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:48 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:48.590+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-26T22:00:42.299288+0000 front 2024-04-26T22:00:42.299033+0000 (oldest deadline 2024-04-26T22:01:06.398950+0000) 2024-04-26T22:01:49.065 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:48 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:48.590+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-04-26T22:00:51.700557+0000 front 2024-04-26T22:00:51.700671+0000 (oldest deadline 2024-04-26T22:01:18.701605+0000) 2024-04-26T22:01:49.065 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:48 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:48.590+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-04-26T22:01:02.104474+0000 front 2024-04-26T22:01:02.104287+0000 (oldest deadline 2024-04-26T22:01:26.803970+0000) 2024-04-26T22:01:49.065 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:48 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:48.590+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.72:6822 osd.6 since back 2024-04-26T22:01:12.606658+0000 front 2024-04-26T22:01:12.606850+0000 (oldest deadline 2024-04-26T22:01:34.906684+0000) 2024-04-26T22:01:49.690 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:49 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:49.303+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:14.343204+0000 front 2024-04-26T22:00:14.342815+0000 (oldest deadline 2024-04-26T22:00:39.042910+0000) 2024-04-26T22:01:49.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:49 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:49.303+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:24.945823+0000 front 2024-04-26T22:00:24.945339+0000 (oldest deadline 2024-04-26T22:00:47.245223+0000) 2024-04-26T22:01:49.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:49 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:49.303+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:31.947744+0000 front 2024-04-26T22:00:31.948080+0000 (oldest deadline 2024-04-26T22:00:56.647838+0000) 2024-04-26T22:01:49.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:49 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:49.303+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-26T22:00:42.953004+0000 front 2024-04-26T22:00:42.952908+0000 (oldest deadline 2024-04-26T22:01:07.652653+0000) 2024-04-26T22:01:49.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:49 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:49.303+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-04-26T22:00:53.256824+0000 front 2024-04-26T22:00:53.256989+0000 (oldest deadline 2024-04-26T22:01:18.556986+0000) 2024-04-26T22:01:49.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:49 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:49.303+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-04-26T22:00:58.557938+0000 front 2024-04-26T22:00:58.558225+0000 (oldest deadline 2024-04-26T22:01:24.457775+0000) 2024-04-26T22:01:49.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:49 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:49.303+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.76:6820 osd.7 since back 2024-04-26T22:01:18.561488+0000 front 2024-04-26T22:01:18.561390+0000 (oldest deadline 2024-04-26T22:01:43.861188+0000) 2024-04-26T22:01:50.064 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:49 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:49.583+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:11.890629+0000 front 2024-04-26T22:00:11.890849+0000 (oldest deadline 2024-04-26T22:00:40.692043+0000) 2024-04-26T22:01:50.065 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:49 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:49.583+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:24.194239+0000 front 2024-04-26T22:00:24.194095+0000 (oldest deadline 2024-04-26T22:00:48.893996+0000) 2024-04-26T22:01:50.065 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:49 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:49.583+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:34.096788+0000 front 2024-04-26T22:00:34.096884+0000 (oldest deadline 2024-04-26T22:00:56.396862+0000) 2024-04-26T22:01:50.065 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:49 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:49.583+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-26T22:00:42.299288+0000 front 2024-04-26T22:00:42.299033+0000 (oldest deadline 2024-04-26T22:01:06.398950+0000) 2024-04-26T22:01:50.065 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:49 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:49.583+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-04-26T22:00:51.700557+0000 front 2024-04-26T22:00:51.700671+0000 (oldest deadline 2024-04-26T22:01:18.701605+0000) 2024-04-26T22:01:50.065 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:49 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:49.583+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-04-26T22:01:02.104474+0000 front 2024-04-26T22:01:02.104287+0000 (oldest deadline 2024-04-26T22:01:26.803970+0000) 2024-04-26T22:01:50.065 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:49 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10[56067]: 2024-04-26T22:01:49.583+0000 7fa3bd600700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.72:6822 osd.6 since back 2024-04-26T22:01:12.606658+0000 front 2024-04-26T22:01:12.606850+0000 (oldest deadline 2024-04-26T22:01:34.906684+0000) 2024-04-26T22:01:50.634 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:50 smithi076 podman[65393]: 2024-04-26 22:01:50.339040369 +0000 UTC m=+5.774914740 container died 4fce955adb6c35488684495d52772baa8e8d860cd8b3f43d5091a002a90b9591 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T22:01:50.690 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:50 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:50.277+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:14.343204+0000 front 2024-04-26T22:00:14.342815+0000 (oldest deadline 2024-04-26T22:00:39.042910+0000) 2024-04-26T22:01:50.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:50 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:50.277+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:24.945823+0000 front 2024-04-26T22:00:24.945339+0000 (oldest deadline 2024-04-26T22:00:47.245223+0000) 2024-04-26T22:01:50.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:50 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:50.277+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:31.947744+0000 front 2024-04-26T22:00:31.948080+0000 (oldest deadline 2024-04-26T22:00:56.647838+0000) 2024-04-26T22:01:50.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:50 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:50.277+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-26T22:00:42.953004+0000 front 2024-04-26T22:00:42.952908+0000 (oldest deadline 2024-04-26T22:01:07.652653+0000) 2024-04-26T22:01:50.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:50 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:50.277+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-04-26T22:00:53.256824+0000 front 2024-04-26T22:00:53.256989+0000 (oldest deadline 2024-04-26T22:01:18.556986+0000) 2024-04-26T22:01:50.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:50 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:50.277+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-04-26T22:00:58.557938+0000 front 2024-04-26T22:00:58.558225+0000 (oldest deadline 2024-04-26T22:01:24.457775+0000) 2024-04-26T22:01:50.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:50 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:50.277+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.76:6820 osd.7 since back 2024-04-26T22:01:18.561488+0000 front 2024-04-26T22:01:18.561390+0000 (oldest deadline 2024-04-26T22:01:43.861188+0000) 2024-04-26T22:01:51.064 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:50 smithi076 podman[65393]: 2024-04-26 22:01:50.743917238 +0000 UTC m=+6.179791646 container cleanup 4fce955adb6c35488684495d52772baa8e8d860cd8b3f43d5091a002a90b9591 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-26T22:01:51.064 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:50 smithi076 bash[65393]: ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10 2024-04-26T22:01:51.564 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:51 smithi076 podman[65425]: 2024-04-26 22:01:51.069367859 +0000 UTC m=+0.709578161 container remove 4fce955adb6c35488684495d52772baa8e8d860cd8b3f43d5091a002a90b9591 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-04-26T22:01:51.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:51 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:51.325+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:14.343204+0000 front 2024-04-26T22:00:14.342815+0000 (oldest deadline 2024-04-26T22:00:39.042910+0000) 2024-04-26T22:01:51.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:51 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:51.325+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:24.945823+0000 front 2024-04-26T22:00:24.945339+0000 (oldest deadline 2024-04-26T22:00:47.245223+0000) 2024-04-26T22:01:51.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:51 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:51.325+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:31.947744+0000 front 2024-04-26T22:00:31.948080+0000 (oldest deadline 2024-04-26T22:00:56.647838+0000) 2024-04-26T22:01:51.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:51 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:51.325+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-26T22:00:42.953004+0000 front 2024-04-26T22:00:42.952908+0000 (oldest deadline 2024-04-26T22:01:07.652653+0000) 2024-04-26T22:01:51.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:51 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:51.326+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-04-26T22:00:53.256824+0000 front 2024-04-26T22:00:53.256989+0000 (oldest deadline 2024-04-26T22:01:18.556986+0000) 2024-04-26T22:01:51.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:51 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:51.326+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-04-26T22:00:58.557938+0000 front 2024-04-26T22:00:58.558225+0000 (oldest deadline 2024-04-26T22:01:24.457775+0000) 2024-04-26T22:01:51.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:51 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:51.326+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.76:6820 osd.7 since back 2024-04-26T22:01:18.561488+0000 front 2024-04-26T22:01:18.561390+0000 (oldest deadline 2024-04-26T22:01:43.861188+0000) 2024-04-26T22:01:51.925 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:51 smithi076 podman[65513]: 2024-04-26 22:01:51.550262666 +0000 UTC m=+0.037918053 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-26T22:01:51.926 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:51 smithi076 podman[65513]: 2024-04-26 22:01:51.661933563 +0000 UTC m=+0.149589011 container create ff3061dd65da35bb56ea70f38a096d07f9b2d2adb34989baa7ef9c5e208c524d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10-deactivate, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff) 2024-04-26T22:01:52.314 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:52 smithi076 podman[65513]: 2024-04-26 22:01:52.024017428 +0000 UTC m=+0.511672850 container init ff3061dd65da35bb56ea70f38a096d07f9b2d2adb34989baa7ef9c5e208c524d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10-deactivate, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff) 2024-04-26T22:01:52.314 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:52 smithi076 podman[65513]: 2024-04-26 22:01:52.035348623 +0000 UTC m=+0.523004072 container start ff3061dd65da35bb56ea70f38a096d07f9b2d2adb34989baa7ef9c5e208c524d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10-deactivate, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_CLEAN=True) 2024-04-26T22:01:52.314 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:52 smithi076 podman[65513]: 2024-04-26 22:01:52.155624189 +0000 UTC m=+0.643279600 container attach ff3061dd65da35bb56ea70f38a096d07f9b2d2adb34989baa7ef9c5e208c524d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-04-26T22:01:52.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:52 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:52.374+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:14.343204+0000 front 2024-04-26T22:00:14.342815+0000 (oldest deadline 2024-04-26T22:00:39.042910+0000) 2024-04-26T22:01:52.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:52 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:52.374+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:24.945823+0000 front 2024-04-26T22:00:24.945339+0000 (oldest deadline 2024-04-26T22:00:47.245223+0000) 2024-04-26T22:01:52.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:52 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:52.374+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:31.947744+0000 front 2024-04-26T22:00:31.948080+0000 (oldest deadline 2024-04-26T22:00:56.647838+0000) 2024-04-26T22:01:52.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:52 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:52.374+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-26T22:00:42.953004+0000 front 2024-04-26T22:00:42.952908+0000 (oldest deadline 2024-04-26T22:01:07.652653+0000) 2024-04-26T22:01:52.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:52 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:52.374+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-04-26T22:00:53.256824+0000 front 2024-04-26T22:00:53.256989+0000 (oldest deadline 2024-04-26T22:01:18.556986+0000) 2024-04-26T22:01:52.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:52 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:52.374+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-04-26T22:00:58.557938+0000 front 2024-04-26T22:00:58.558225+0000 (oldest deadline 2024-04-26T22:01:24.457775+0000) 2024-04-26T22:01:52.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:52 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:52.374+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.76:6820 osd.7 since back 2024-04-26T22:01:18.561488+0000 front 2024-04-26T22:01:18.561390+0000 (oldest deadline 2024-04-26T22:01:43.861188+0000) 2024-04-26T22:01:52.814 INFO:journalctl@ceph.osd.10.smithi076.stdout:Apr 26 22:01:52 smithi076 podman[65513]: 2024-04-26 22:01:52.505357502 +0000 UTC m=+0.993012889 container died ff3061dd65da35bb56ea70f38a096d07f9b2d2adb34989baa7ef9c5e208c524d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-10-deactivate, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-04-26T22:01:53.224 DEBUG:teuthology.orchestra.run.smithi076:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a1780950-0415-11ef-bc93-c7b262605968@osd.10.service' 2024-04-26T22:01:53.329 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T22:01:53.329 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-04-26T22:01:53.330 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-04-26T22:01:53.330 DEBUG:teuthology.orchestra.run.smithi164:> sudo systemctl stop ceph-a1780950-0415-11ef-bc93-c7b262605968@osd.11 2024-04-26T22:01:53.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:53 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:53.379+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:14.343204+0000 front 2024-04-26T22:00:14.342815+0000 (oldest deadline 2024-04-26T22:00:39.042910+0000) 2024-04-26T22:01:53.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:53 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:53.379+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:24.945823+0000 front 2024-04-26T22:00:24.945339+0000 (oldest deadline 2024-04-26T22:00:47.245223+0000) 2024-04-26T22:01:53.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:53 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:53.379+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:31.947744+0000 front 2024-04-26T22:00:31.948080+0000 (oldest deadline 2024-04-26T22:00:56.647838+0000) 2024-04-26T22:01:53.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:53 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:53.379+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-26T22:00:42.953004+0000 front 2024-04-26T22:00:42.952908+0000 (oldest deadline 2024-04-26T22:01:07.652653+0000) 2024-04-26T22:01:53.692 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:53 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:53.379+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-04-26T22:00:53.256824+0000 front 2024-04-26T22:00:53.256989+0000 (oldest deadline 2024-04-26T22:01:18.556986+0000) 2024-04-26T22:01:53.692 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:53 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:53.379+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-04-26T22:00:58.557938+0000 front 2024-04-26T22:00:58.558225+0000 (oldest deadline 2024-04-26T22:01:24.457775+0000) 2024-04-26T22:01:53.692 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:53 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:53.379+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.76:6820 osd.7 since back 2024-04-26T22:01:18.561488+0000 front 2024-04-26T22:01:18.561390+0000 (oldest deadline 2024-04-26T22:01:43.861188+0000) 2024-04-26T22:01:53.692 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:53 smithi164 systemd[1]: Stopping Ceph osd.11 for a1780950-0415-11ef-bc93-c7b262605968... 2024-04-26T22:01:54.341 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:53 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:53.989+0000 7f9801400700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T22:01:54.342 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:53 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:53.989+0000 7f9801400700 -1 osd.11 94 *** Got signal Terminated *** 2024-04-26T22:01:54.342 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:53 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:53.989+0000 7f9801400700 -1 osd.11 94 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-26T22:01:54.342 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:54 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:54.340+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:14.343204+0000 front 2024-04-26T22:00:14.342815+0000 (oldest deadline 2024-04-26T22:00:39.042910+0000) 2024-04-26T22:01:54.342 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:54 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:54.341+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:24.945823+0000 front 2024-04-26T22:00:24.945339+0000 (oldest deadline 2024-04-26T22:00:47.245223+0000) 2024-04-26T22:01:54.342 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:54 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:54.341+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:31.947744+0000 front 2024-04-26T22:00:31.948080+0000 (oldest deadline 2024-04-26T22:00:56.647838+0000) 2024-04-26T22:01:54.690 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:54 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:54.341+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-26T22:00:42.953004+0000 front 2024-04-26T22:00:42.952908+0000 (oldest deadline 2024-04-26T22:01:07.652653+0000) 2024-04-26T22:01:54.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:54 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:54.341+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-04-26T22:00:53.256824+0000 front 2024-04-26T22:00:53.256989+0000 (oldest deadline 2024-04-26T22:01:18.556986+0000) 2024-04-26T22:01:54.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:54 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:54.341+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-04-26T22:00:58.557938+0000 front 2024-04-26T22:00:58.558225+0000 (oldest deadline 2024-04-26T22:01:24.457775+0000) 2024-04-26T22:01:54.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:54 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:54.341+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.76:6820 osd.7 since back 2024-04-26T22:01:18.561488+0000 front 2024-04-26T22:01:18.561390+0000 (oldest deadline 2024-04-26T22:01:43.861188+0000) 2024-04-26T22:01:54.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:54 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:54.341+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6820 osd.8 since back 2024-04-26T22:01:30.065068+0000 front 2024-04-26T22:01:30.065209+0000 (oldest deadline 2024-04-26T22:01:53.564918+0000) 2024-04-26T22:01:55.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:55 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:55.331+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:14.343204+0000 front 2024-04-26T22:00:14.342815+0000 (oldest deadline 2024-04-26T22:00:39.042910+0000) 2024-04-26T22:01:55.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:55 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:55.331+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:24.945823+0000 front 2024-04-26T22:00:24.945339+0000 (oldest deadline 2024-04-26T22:00:47.245223+0000) 2024-04-26T22:01:55.692 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:55 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:55.331+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:31.947744+0000 front 2024-04-26T22:00:31.948080+0000 (oldest deadline 2024-04-26T22:00:56.647838+0000) 2024-04-26T22:01:55.692 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:55 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:55.331+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-26T22:00:42.953004+0000 front 2024-04-26T22:00:42.952908+0000 (oldest deadline 2024-04-26T22:01:07.652653+0000) 2024-04-26T22:01:55.692 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:55 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:55.331+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-04-26T22:00:53.256824+0000 front 2024-04-26T22:00:53.256989+0000 (oldest deadline 2024-04-26T22:01:18.556986+0000) 2024-04-26T22:01:55.692 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:55 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:55.331+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-04-26T22:00:58.557938+0000 front 2024-04-26T22:00:58.558225+0000 (oldest deadline 2024-04-26T22:01:24.457775+0000) 2024-04-26T22:01:55.692 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:55 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:55.331+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.76:6820 osd.7 since back 2024-04-26T22:01:18.561488+0000 front 2024-04-26T22:01:18.561390+0000 (oldest deadline 2024-04-26T22:01:43.861188+0000) 2024-04-26T22:01:55.692 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:55 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:55.331+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6820 osd.8 since back 2024-04-26T22:01:30.065068+0000 front 2024-04-26T22:01:30.065209+0000 (oldest deadline 2024-04-26T22:01:53.564918+0000) 2024-04-26T22:01:56.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:56 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:56.343+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:14.343204+0000 front 2024-04-26T22:00:14.342815+0000 (oldest deadline 2024-04-26T22:00:39.042910+0000) 2024-04-26T22:01:56.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:56 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:56.343+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:24.945823+0000 front 2024-04-26T22:00:24.945339+0000 (oldest deadline 2024-04-26T22:00:47.245223+0000) 2024-04-26T22:01:56.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:56 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:56.343+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:31.947744+0000 front 2024-04-26T22:00:31.948080+0000 (oldest deadline 2024-04-26T22:00:56.647838+0000) 2024-04-26T22:01:56.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:56 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:56.343+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-26T22:00:42.953004+0000 front 2024-04-26T22:00:42.952908+0000 (oldest deadline 2024-04-26T22:01:07.652653+0000) 2024-04-26T22:01:56.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:56 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:56.343+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-04-26T22:00:53.256824+0000 front 2024-04-26T22:00:53.256989+0000 (oldest deadline 2024-04-26T22:01:18.556986+0000) 2024-04-26T22:01:56.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:56 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:56.343+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-04-26T22:00:58.557938+0000 front 2024-04-26T22:00:58.558225+0000 (oldest deadline 2024-04-26T22:01:24.457775+0000) 2024-04-26T22:01:56.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:56 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:56.343+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.76:6820 osd.7 since back 2024-04-26T22:01:18.561488+0000 front 2024-04-26T22:01:18.561390+0000 (oldest deadline 2024-04-26T22:01:43.861188+0000) 2024-04-26T22:01:56.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:56 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:56.343+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6820 osd.8 since back 2024-04-26T22:01:30.065068+0000 front 2024-04-26T22:01:30.065209+0000 (oldest deadline 2024-04-26T22:01:53.564918+0000) 2024-04-26T22:01:57.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:57 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:57.317+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:14.343204+0000 front 2024-04-26T22:00:14.342815+0000 (oldest deadline 2024-04-26T22:00:39.042910+0000) 2024-04-26T22:01:57.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:57 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:57.317+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:24.945823+0000 front 2024-04-26T22:00:24.945339+0000 (oldest deadline 2024-04-26T22:00:47.245223+0000) 2024-04-26T22:01:57.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:57 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:57.317+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:31.947744+0000 front 2024-04-26T22:00:31.948080+0000 (oldest deadline 2024-04-26T22:00:56.647838+0000) 2024-04-26T22:01:57.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:57 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:57.317+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-26T22:00:42.953004+0000 front 2024-04-26T22:00:42.952908+0000 (oldest deadline 2024-04-26T22:01:07.652653+0000) 2024-04-26T22:01:57.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:57 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:57.317+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-04-26T22:00:53.256824+0000 front 2024-04-26T22:00:53.256989+0000 (oldest deadline 2024-04-26T22:01:18.556986+0000) 2024-04-26T22:01:57.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:57 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:57.317+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-04-26T22:00:58.557938+0000 front 2024-04-26T22:00:58.558225+0000 (oldest deadline 2024-04-26T22:01:24.457775+0000) 2024-04-26T22:01:57.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:57 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:57.317+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.76:6820 osd.7 since back 2024-04-26T22:01:18.561488+0000 front 2024-04-26T22:01:18.561390+0000 (oldest deadline 2024-04-26T22:01:43.861188+0000) 2024-04-26T22:01:57.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:57 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:57.317+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6820 osd.8 since back 2024-04-26T22:01:30.065068+0000 front 2024-04-26T22:01:30.065209+0000 (oldest deadline 2024-04-26T22:01:53.564918+0000) 2024-04-26T22:01:58.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:58 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:58.276+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.72:6806 osd.0 since back 2024-04-26T22:00:14.343204+0000 front 2024-04-26T22:00:14.342815+0000 (oldest deadline 2024-04-26T22:00:39.042910+0000) 2024-04-26T22:01:58.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:58 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:58.276+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-04-26T22:00:24.945823+0000 front 2024-04-26T22:00:24.945339+0000 (oldest deadline 2024-04-26T22:00:47.245223+0000) 2024-04-26T22:01:58.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:58 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:58.276+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2024-04-26T22:00:31.947744+0000 front 2024-04-26T22:00:31.948080+0000 (oldest deadline 2024-04-26T22:00:56.647838+0000) 2024-04-26T22:01:58.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:58 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:58.276+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.72:6814 osd.3 since back 2024-04-26T22:00:42.953004+0000 front 2024-04-26T22:00:42.952908+0000 (oldest deadline 2024-04-26T22:01:07.652653+0000) 2024-04-26T22:01:58.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:58 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:58.276+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-04-26T22:00:53.256824+0000 front 2024-04-26T22:00:53.256989+0000 (oldest deadline 2024-04-26T22:01:18.556986+0000) 2024-04-26T22:01:58.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:58 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:58.276+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6812 osd.5 since back 2024-04-26T22:00:58.557938+0000 front 2024-04-26T22:00:58.558225+0000 (oldest deadline 2024-04-26T22:01:24.457775+0000) 2024-04-26T22:01:58.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:58 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:58.276+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.76:6820 osd.7 since back 2024-04-26T22:01:18.561488+0000 front 2024-04-26T22:01:18.561390+0000 (oldest deadline 2024-04-26T22:01:43.861188+0000) 2024-04-26T22:01:58.691 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:58 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11[57053]: 2024-04-26T22:01:58.276+0000 7f97fcc00700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6820 osd.8 since back 2024-04-26T22:01:30.065068+0000 front 2024-04-26T22:01:30.065209+0000 (oldest deadline 2024-04-26T22:01:53.564918+0000) 2024-04-26T22:01:59.810 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:59 smithi164 podman[66923]: 2024-04-26 22:01:59.523276054 +0000 UTC m=+5.721651475 container died ff2513c70c06e5fc81f5d41b6a27b40271c35bb1fa8e40e8af50ac0b4dddbde8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.build-date=20240424) 2024-04-26T22:02:00.129 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:59 smithi164 podman[66923]: 2024-04-26 22:01:59.936084571 +0000 UTC m=+6.134460009 container cleanup ff2513c70c06e5fc81f5d41b6a27b40271c35bb1fa8e40e8af50ac0b4dddbde8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T22:02:00.130 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:01:59 smithi164 bash[66923]: ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11 2024-04-26T22:02:00.440 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:02:00 smithi164 podman[66943]: 2024-04-26 22:02:00.128805125 +0000 UTC m=+0.586373594 container remove ff2513c70c06e5fc81f5d41b6a27b40271c35bb1fa8e40e8af50ac0b4dddbde8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-04-26T22:02:01.309 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:02:00 smithi164 podman[67030]: 2024-04-26 22:02:00.874083619 +0000 UTC m=+0.036634775 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-26T22:02:01.309 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:02:01 smithi164 podman[67030]: 2024-04-26 22:02:01.012995932 +0000 UTC m=+0.175547068 container create a20c2344d3826e2cd7f304b4fa75a1168b5086cd4debd19c8f0bc520071edf90 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11-deactivate, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True) 2024-04-26T22:02:01.309 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:02:01 smithi164 podman[67030]: 2024-04-26 22:02:01.308661321 +0000 UTC m=+0.471212494 container init a20c2344d3826e2cd7f304b4fa75a1168b5086cd4debd19c8f0bc520071edf90 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.build-date=20240424) 2024-04-26T22:02:01.690 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:02:01 smithi164 podman[67030]: 2024-04-26 22:02:01.319599776 +0000 UTC m=+0.482150914 container start a20c2344d3826e2cd7f304b4fa75a1168b5086cd4debd19c8f0bc520071edf90 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11-deactivate, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2) 2024-04-26T22:02:01.690 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:02:01 smithi164 podman[67030]: 2024-04-26 22:02:01.406948799 +0000 UTC m=+0.569500018 container attach a20c2344d3826e2cd7f304b4fa75a1168b5086cd4debd19c8f0bc520071edf90 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-26T22:02:02.098 INFO:journalctl@ceph.osd.11.smithi164.stdout:Apr 26 22:02:01 smithi164 podman[67030]: 2024-04-26 22:02:01.763745899 +0000 UTC m=+0.926297030 container died a20c2344d3826e2cd7f304b4fa75a1168b5086cd4debd19c8f0bc520071edf90 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-osd-11-deactivate, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T22:02:02.478 DEBUG:teuthology.orchestra.run.smithi164:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a1780950-0415-11ef-bc93-c7b262605968@osd.11.service' 2024-04-26T22:02:02.572 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T22:02:02.572 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-04-26T22:02:02.573 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-04-26T22:02:02.573 DEBUG:teuthology.orchestra.run.smithi072:> sudo systemctl stop ceph-a1780950-0415-11ef-bc93-c7b262605968@mds.a 2024-04-26T22:02:03.071 INFO:journalctl@ceph.mds.a.smithi072.stdout:Apr 26 22:02:02 smithi072 systemd[1]: Stopping Ceph mds.a for a1780950-0415-11ef-bc93-c7b262605968... 2024-04-26T22:02:03.571 INFO:journalctl@ceph.mds.a.smithi072.stdout:Apr 26 22:02:03 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-mds-a[70549]: 2024-04-26T22:02:03.234+0000 7fd4e2200700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T22:02:03.572 INFO:journalctl@ceph.mds.a.smithi072.stdout:Apr 26 22:02:03 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-mds-a[70549]: 2024-04-26T22:02:03.234+0000 7fd4e2200700 -1 mds.a *** got signal Terminated *** 2024-04-26T22:02:07.528 INFO:journalctl@ceph.mds.a.smithi072.stdout:Apr 26 22:02:07 smithi072 podman[89597]: 2024-04-26 22:02:07.251305944 +0000 UTC m=+4.225681928 container died fb63d88d4bc44337f9e673b040c52b6bd3b30eeeb0b347ddd09d985b0945f214 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-mds-a, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, io.buildah.version=1.33.5) 2024-04-26T22:02:07.821 INFO:journalctl@ceph.mds.a.smithi072.stdout:Apr 26 22:02:07 smithi072 podman[89597]: 2024-04-26 22:02:07.646451426 +0000 UTC m=+4.620827440 container cleanup fb63d88d4bc44337f9e673b040c52b6bd3b30eeeb0b347ddd09d985b0945f214 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-mds-a, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T22:02:07.821 INFO:journalctl@ceph.mds.a.smithi072.stdout:Apr 26 22:02:07 smithi072 bash[89597]: ceph-a1780950-0415-11ef-bc93-c7b262605968-mds-a 2024-04-26T22:02:08.321 INFO:journalctl@ceph.mds.a.smithi072.stdout:Apr 26 22:02:07 smithi072 podman[89611]: 2024-04-26 22:02:07.901699362 +0000 UTC m=+0.635232599 container remove fb63d88d4bc44337f9e673b040c52b6bd3b30eeeb0b347ddd09d985b0945f214 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-mds-a, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True) 2024-04-26T22:02:08.466 DEBUG:teuthology.orchestra.run.smithi072:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a1780950-0415-11ef-bc93-c7b262605968@mds.a.service' 2024-04-26T22:02:08.567 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T22:02:08.567 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-04-26T22:02:08.567 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-04-26T22:02:08.567 DEBUG:teuthology.orchestra.run.smithi072:> sudo systemctl stop ceph-a1780950-0415-11ef-bc93-c7b262605968@mds.d 2024-04-26T22:02:09.071 INFO:journalctl@ceph.mds.d.smithi072.stdout:Apr 26 22:02:08 smithi072 systemd[1]: Stopping Ceph mds.d for a1780950-0415-11ef-bc93-c7b262605968... 2024-04-26T22:02:09.571 INFO:journalctl@ceph.mds.d.smithi072.stdout:Apr 26 22:02:09 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-mds-d[69480]: 2024-04-26T22:02:09.159+0000 7f5d22800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T22:02:09.571 INFO:journalctl@ceph.mds.d.smithi072.stdout:Apr 26 22:02:09 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-mds-d[69480]: 2024-04-26T22:02:09.159+0000 7f5d22800700 -1 mds.d *** got signal Terminated *** 2024-04-26T22:02:12.754 INFO:journalctl@ceph.mds.d.smithi072.stdout:Apr 26 22:02:12 smithi072 podman[89718]: 2024-04-26 22:02:12.47608238 +0000 UTC m=+3.467948432 container died d54a4c15c6839e891b7fb18622148741d1f880042430b03270006bc1d86a539b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-mds-d, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-04-26T22:02:13.071 INFO:journalctl@ceph.mds.d.smithi072.stdout:Apr 26 22:02:12 smithi072 podman[89718]: 2024-04-26 22:02:12.897129649 +0000 UTC m=+3.888995728 container cleanup d54a4c15c6839e891b7fb18622148741d1f880042430b03270006bc1d86a539b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-mds-d, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff) 2024-04-26T22:02:13.072 INFO:journalctl@ceph.mds.d.smithi072.stdout:Apr 26 22:02:12 smithi072 bash[89718]: ceph-a1780950-0415-11ef-bc93-c7b262605968-mds-d 2024-04-26T22:02:13.572 INFO:journalctl@ceph.mds.d.smithi072.stdout:Apr 26 22:02:13 smithi072 podman[89744]: 2024-04-26 22:02:13.191062956 +0000 UTC m=+0.695184503 container remove d54a4c15c6839e891b7fb18622148741d1f880042430b03270006bc1d86a539b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-mds-d, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff) 2024-04-26T22:02:13.749 DEBUG:teuthology.orchestra.run.smithi072:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a1780950-0415-11ef-bc93-c7b262605968@mds.d.service' 2024-04-26T22:02:13.856 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T22:02:13.857 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-04-26T22:02:13.857 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-04-26T22:02:13.857 DEBUG:teuthology.orchestra.run.smithi072:> sudo systemctl stop ceph-a1780950-0415-11ef-bc93-c7b262605968@mds.g 2024-04-26T22:02:14.321 INFO:journalctl@ceph.mds.g.smithi072.stdout:Apr 26 22:02:14 smithi072 systemd[1]: Stopping Ceph mds.g for a1780950-0415-11ef-bc93-c7b262605968... 2024-04-26T22:02:14.821 INFO:journalctl@ceph.mds.g.smithi072.stdout:Apr 26 22:02:14 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-mds-g[70185]: 2024-04-26T22:02:14.445+0000 7fcf21000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T22:02:14.822 INFO:journalctl@ceph.mds.g.smithi072.stdout:Apr 26 22:02:14 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-mds-g[70185]: 2024-04-26T22:02:14.445+0000 7fcf21000700 -1 mds.g *** got signal Terminated *** 2024-04-26T22:02:16.321 INFO:journalctl@ceph.mds.g.smithi072.stdout:Apr 26 22:02:16 smithi072 podman[89849]: 2024-04-26 22:02:16.043277768 +0000 UTC m=+1.717793742 container died e193c142ef16960efaad05391a0c095bb14cfe94819f4a28dd03555c6535a50f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-mds-g, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-04-26T22:02:16.775 INFO:journalctl@ceph.mds.g.smithi072.stdout:Apr 26 22:02:16 smithi072 podman[89849]: 2024-04-26 22:02:16.447654941 +0000 UTC m=+2.122170912 container cleanup e193c142ef16960efaad05391a0c095bb14cfe94819f4a28dd03555c6535a50f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-mds-g, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, ceph=True, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-04-26T22:02:16.775 INFO:journalctl@ceph.mds.g.smithi072.stdout:Apr 26 22:02:16 smithi072 bash[89849]: ceph-a1780950-0415-11ef-bc93-c7b262605968-mds-g 2024-04-26T22:02:17.071 INFO:journalctl@ceph.mds.g.smithi072.stdout:Apr 26 22:02:16 smithi072 podman[89863]: 2024-04-26 22:02:16.77499262 +0000 UTC m=+0.718108543 container remove e193c142ef16960efaad05391a0c095bb14cfe94819f4a28dd03555c6535a50f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-mds-g, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff) 2024-04-26T22:02:17.381 DEBUG:teuthology.orchestra.run.smithi072:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a1780950-0415-11ef-bc93-c7b262605968@mds.g.service' 2024-04-26T22:02:17.489 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T22:02:17.489 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-04-26T22:02:17.490 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-04-26T22:02:17.490 DEBUG:teuthology.orchestra.run.smithi072:> sudo systemctl stop ceph-a1780950-0415-11ef-bc93-c7b262605968@mds.j 2024-04-26T22:02:18.071 INFO:journalctl@ceph.mds.j.smithi072.stdout:Apr 26 22:02:17 smithi072 systemd[1]: Stopping Ceph mds.j for a1780950-0415-11ef-bc93-c7b262605968... 2024-04-26T22:02:18.571 INFO:journalctl@ceph.mds.j.smithi072.stdout:Apr 26 22:02:18 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-mds-j[69091]: 2024-04-26T22:02:18.108+0000 7fd810800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T22:02:18.572 INFO:journalctl@ceph.mds.j.smithi072.stdout:Apr 26 22:02:18 smithi072 ceph-a1780950-0415-11ef-bc93-c7b262605968-mds-j[69091]: 2024-04-26T22:02:18.108+0000 7fd810800700 -1 mds.j *** got signal Terminated *** 2024-04-26T22:02:20.918 INFO:journalctl@ceph.mds.j.smithi072.stdout:Apr 26 22:02:20 smithi072 podman[89969]: 2024-04-26 22:02:20.626303518 +0000 UTC m=+2.636484390 container died efaf9a1d4cd2e9d3bdbb79292b0f581e3d95744bdf4ed9961658e3d0f3d7d8ff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-mds-j, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240424, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-04-26T22:02:21.322 INFO:journalctl@ceph.mds.j.smithi072.stdout:Apr 26 22:02:21 smithi072 podman[89969]: 2024-04-26 22:02:21.022849798 +0000 UTC m=+3.033030667 container cleanup efaf9a1d4cd2e9d3bdbb79292b0f581e3d95744bdf4ed9961658e3d0f3d7d8ff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-mds-j, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T22:02:21.322 INFO:journalctl@ceph.mds.j.smithi072.stdout:Apr 26 22:02:21 smithi072 bash[89969]: ceph-a1780950-0415-11ef-bc93-c7b262605968-mds-j 2024-04-26T22:02:21.740 INFO:journalctl@ceph.mds.j.smithi072.stdout:Apr 26 22:02:21 smithi072 podman[89983]: 2024-04-26 22:02:21.324822776 +0000 UTC m=+0.676704380 container remove efaf9a1d4cd2e9d3bdbb79292b0f581e3d95744bdf4ed9961658e3d0f3d7d8ff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-mds-j, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424) 2024-04-26T22:02:21.782 DEBUG:teuthology.orchestra.run.smithi072:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a1780950-0415-11ef-bc93-c7b262605968@mds.j.service' 2024-04-26T22:02:21.883 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T22:02:21.883 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-04-26T22:02:21.883 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-04-26T22:02:21.883 DEBUG:teuthology.orchestra.run.smithi076:> sudo systemctl stop ceph-a1780950-0415-11ef-bc93-c7b262605968@mds.b 2024-04-26T22:02:22.314 INFO:journalctl@ceph.mds.b.smithi076.stdout:Apr 26 22:02:21 smithi076 systemd[1]: Stopping Ceph mds.b for a1780950-0415-11ef-bc93-c7b262605968... 2024-04-26T22:02:22.662 INFO:journalctl@ceph.mds.b.smithi076.stdout:Apr 26 22:02:22 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-mds-b[61253]: 2024-04-26T22:02:22.455+0000 7f287aa00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T22:02:22.663 INFO:journalctl@ceph.mds.b.smithi076.stdout:Apr 26 22:02:22 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-mds-b[61253]: 2024-04-26T22:02:22.455+0000 7f287aa00700 -1 mds.b *** got signal Terminated *** 2024-04-26T22:02:24.314 INFO:journalctl@ceph.mds.b.smithi076.stdout:Apr 26 22:02:24 smithi076 podman[65619]: 2024-04-26 22:02:24.069540406 +0000 UTC m=+1.774562488 container died 3e314c4fe51de52f0d1edcd98ca3f8fb2cce235400c8ba2daebce9d756a10eb7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-mds-b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424) 2024-04-26T22:02:24.659 INFO:journalctl@ceph.mds.b.smithi076.stdout:Apr 26 22:02:24 smithi076 podman[65619]: 2024-04-26 22:02:24.475099596 +0000 UTC m=+2.180121719 container cleanup 3e314c4fe51de52f0d1edcd98ca3f8fb2cce235400c8ba2daebce9d756a10eb7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-mds-b, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=) 2024-04-26T22:02:24.659 INFO:journalctl@ceph.mds.b.smithi076.stdout:Apr 26 22:02:24 smithi076 bash[65619]: ceph-a1780950-0415-11ef-bc93-c7b262605968-mds-b 2024-04-26T22:02:25.064 INFO:journalctl@ceph.mds.b.smithi076.stdout:Apr 26 22:02:24 smithi076 podman[65645]: 2024-04-26 22:02:24.659162358 +0000 UTC m=+0.575285785 container remove 3e314c4fe51de52f0d1edcd98ca3f8fb2cce235400c8ba2daebce9d756a10eb7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-mds-b, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=) 2024-04-26T22:02:25.212 DEBUG:teuthology.orchestra.run.smithi076:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a1780950-0415-11ef-bc93-c7b262605968@mds.b.service' 2024-04-26T22:02:25.325 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T22:02:25.325 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-04-26T22:02:25.325 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-04-26T22:02:25.325 DEBUG:teuthology.orchestra.run.smithi076:> sudo systemctl stop ceph-a1780950-0415-11ef-bc93-c7b262605968@mds.e 2024-04-26T22:02:25.814 INFO:journalctl@ceph.mds.e.smithi076.stdout:Apr 26 22:02:25 smithi076 systemd[1]: Stopping Ceph mds.e for a1780950-0415-11ef-bc93-c7b262605968... 2024-04-26T22:02:26.314 INFO:journalctl@ceph.mds.e.smithi076.stdout:Apr 26 22:02:25 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-mds-e[60629]: 2024-04-26T22:02:25.910+0000 7fb9f1000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T22:02:26.314 INFO:journalctl@ceph.mds.e.smithi076.stdout:Apr 26 22:02:25 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-mds-e[60629]: 2024-04-26T22:02:25.910+0000 7fb9f1000700 -1 mds.e *** got signal Terminated *** 2024-04-26T22:02:28.314 INFO:journalctl@ceph.mds.e.smithi076.stdout:Apr 26 22:02:28 smithi076 podman[65750]: 2024-04-26 22:02:28.056240801 +0000 UTC m=+2.272435301 container died 5c97ebe3d413c16b14c3e0412f42bd87db1427dc328470a7dcd9281bce9c06f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-mds-e, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-04-26T22:02:28.662 INFO:journalctl@ceph.mds.e.smithi076.stdout:Apr 26 22:02:28 smithi076 podman[65750]: 2024-04-26 22:02:28.428036961 +0000 UTC m=+2.644231470 container cleanup 5c97ebe3d413c16b14c3e0412f42bd87db1427dc328470a7dcd9281bce9c06f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-mds-e, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-04-26T22:02:28.662 INFO:journalctl@ceph.mds.e.smithi076.stdout:Apr 26 22:02:28 smithi076 bash[65750]: ceph-a1780950-0415-11ef-bc93-c7b262605968-mds-e 2024-04-26T22:02:29.064 INFO:journalctl@ceph.mds.e.smithi076.stdout:Apr 26 22:02:28 smithi076 podman[65765]: 2024-04-26 22:02:28.662339511 +0000 UTC m=+0.586169740 container remove 5c97ebe3d413c16b14c3e0412f42bd87db1427dc328470a7dcd9281bce9c06f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-mds-e, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.license=GPLv2) 2024-04-26T22:02:29.131 DEBUG:teuthology.orchestra.run.smithi076:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a1780950-0415-11ef-bc93-c7b262605968@mds.e.service' 2024-04-26T22:02:29.236 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T22:02:29.236 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-04-26T22:02:29.236 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-04-26T22:02:29.236 DEBUG:teuthology.orchestra.run.smithi076:> sudo systemctl stop ceph-a1780950-0415-11ef-bc93-c7b262605968@mds.h 2024-04-26T22:02:29.564 INFO:journalctl@ceph.mds.h.smithi076.stdout:Apr 26 22:02:29 smithi076 systemd[1]: Stopping Ceph mds.h for a1780950-0415-11ef-bc93-c7b262605968... 2024-04-26T22:02:30.064 INFO:journalctl@ceph.mds.h.smithi076.stdout:Apr 26 22:02:29 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-mds-h[60934]: 2024-04-26T22:02:29.767+0000 7ff0d2800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T22:02:30.064 INFO:journalctl@ceph.mds.h.smithi076.stdout:Apr 26 22:02:29 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-mds-h[60934]: 2024-04-26T22:02:29.767+0000 7ff0d2800700 -1 mds.h *** got signal Terminated *** 2024-04-26T22:02:32.372 INFO:journalctl@ceph.mds.h.smithi076.stdout:Apr 26 22:02:32 smithi076 podman[65872]: 2024-04-26 22:02:32.07496208 +0000 UTC m=+2.453958755 container died 13e97c00bd326abdb7bfff9390f0d2c68c2d6ee147a64418377b978067ab513c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-mds-h, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240424, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-04-26T22:02:32.774 INFO:journalctl@ceph.mds.h.smithi076.stdout:Apr 26 22:02:32 smithi076 podman[65872]: 2024-04-26 22:02:32.481368039 +0000 UTC m=+2.860364759 container cleanup 13e97c00bd326abdb7bfff9390f0d2c68c2d6ee147a64418377b978067ab513c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-mds-h, org.label-schema.build-date=20240424, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.vendor=CentOS) 2024-04-26T22:02:32.775 INFO:journalctl@ceph.mds.h.smithi076.stdout:Apr 26 22:02:32 smithi076 bash[65872]: ceph-a1780950-0415-11ef-bc93-c7b262605968-mds-h 2024-04-26T22:02:33.044 INFO:journalctl@ceph.mds.h.smithi076.stdout:Apr 26 22:02:32 smithi076 podman[65886]: 2024-04-26 22:02:32.773914353 +0000 UTC m=+0.683731828 container remove 13e97c00bd326abdb7bfff9390f0d2c68c2d6ee147a64418377b978067ab513c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-mds-h, org.label-schema.build-date=20240424, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-04-26T22:02:33.058 DEBUG:teuthology.orchestra.run.smithi076:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a1780950-0415-11ef-bc93-c7b262605968@mds.h.service' 2024-04-26T22:02:33.168 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T22:02:33.168 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-04-26T22:02:33.168 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-04-26T22:02:33.168 DEBUG:teuthology.orchestra.run.smithi076:> sudo systemctl stop ceph-a1780950-0415-11ef-bc93-c7b262605968@mds.k 2024-04-26T22:02:33.767 INFO:journalctl@ceph.mds.k.smithi076.stdout:Apr 26 22:02:33 smithi076 systemd[1]: Stopping Ceph mds.k for a1780950-0415-11ef-bc93-c7b262605968... 2024-04-26T22:02:34.064 INFO:journalctl@ceph.mds.k.smithi076.stdout:Apr 26 22:02:33 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-mds-k[60327]: 2024-04-26T22:02:33.766+0000 7fa41d800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T22:02:34.065 INFO:journalctl@ceph.mds.k.smithi076.stdout:Apr 26 22:02:33 smithi076 ceph-a1780950-0415-11ef-bc93-c7b262605968-mds-k[60327]: 2024-04-26T22:02:33.766+0000 7fa41d800700 -1 mds.k *** got signal Terminated *** 2024-04-26T22:02:36.314 INFO:journalctl@ceph.mds.k.smithi076.stdout:Apr 26 22:02:36 smithi076 podman[65991]: 2024-04-26 22:02:36.062342253 +0000 UTC m=+2.439978766 container died c0523893e3a14c459a8b80fb3582cad49ca432e316067808b57da99afeb40f30 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-mds-k, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=) 2024-04-26T22:02:36.731 INFO:journalctl@ceph.mds.k.smithi076.stdout:Apr 26 22:02:36 smithi076 podman[65991]: 2024-04-26 22:02:36.501330629 +0000 UTC m=+2.878967136 container cleanup c0523893e3a14c459a8b80fb3582cad49ca432e316067808b57da99afeb40f30 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-mds-k, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T22:02:36.731 INFO:journalctl@ceph.mds.k.smithi076.stdout:Apr 26 22:02:36 smithi076 bash[65991]: ceph-a1780950-0415-11ef-bc93-c7b262605968-mds-k 2024-04-26T22:02:37.064 INFO:journalctl@ceph.mds.k.smithi076.stdout:Apr 26 22:02:36 smithi076 podman[66004]: 2024-04-26 22:02:36.731075975 +0000 UTC m=+0.649125858 container remove c0523893e3a14c459a8b80fb3582cad49ca432e316067808b57da99afeb40f30 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-mds-k, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-04-26T22:02:37.259 DEBUG:teuthology.orchestra.run.smithi076:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a1780950-0415-11ef-bc93-c7b262605968@mds.k.service' 2024-04-26T22:02:37.360 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T22:02:37.360 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-04-26T22:02:37.361 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-04-26T22:02:37.361 DEBUG:teuthology.orchestra.run.smithi164:> sudo systemctl stop ceph-a1780950-0415-11ef-bc93-c7b262605968@mds.c 2024-04-26T22:02:37.690 INFO:journalctl@ceph.mds.c.smithi164.stdout:Apr 26 22:02:37 smithi164 systemd[1]: Stopping Ceph mds.c for a1780950-0415-11ef-bc93-c7b262605968... 2024-04-26T22:02:38.440 INFO:journalctl@ceph.mds.c.smithi164.stdout:Apr 26 22:02:37 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-mds-c[62363]: 2024-04-26T22:02:37.973+0000 7fd685400700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T22:02:38.441 INFO:journalctl@ceph.mds.c.smithi164.stdout:Apr 26 22:02:37 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-mds-c[62363]: 2024-04-26T22:02:37.973+0000 7fd685400700 -1 mds.c *** got signal Terminated *** 2024-04-26T22:02:40.398 INFO:journalctl@ceph.mds.c.smithi164.stdout:Apr 26 22:02:40 smithi164 podman[67151]: 2024-04-26 22:02:40.084280445 +0000 UTC m=+2.268749227 container died a2fd1d91709b00c4056662cb6db264689d23b313584591ce5719c5cbbb63db3b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-mds-c, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-04-26T22:02:40.690 INFO:journalctl@ceph.mds.c.smithi164.stdout:Apr 26 22:02:40 smithi164 podman[67151]: 2024-04-26 22:02:40.50949089 +0000 UTC m=+2.693959699 container cleanup a2fd1d91709b00c4056662cb6db264689d23b313584591ce5719c5cbbb63db3b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-mds-c, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-04-26T22:02:40.690 INFO:journalctl@ceph.mds.c.smithi164.stdout:Apr 26 22:02:40 smithi164 bash[67151]: ceph-a1780950-0415-11ef-bc93-c7b262605968-mds-c 2024-04-26T22:02:41.190 INFO:journalctl@ceph.mds.c.smithi164.stdout:Apr 26 22:02:40 smithi164 podman[67166]: 2024-04-26 22:02:40.810787918 +0000 UTC m=+0.708878631 container remove a2fd1d91709b00c4056662cb6db264689d23b313584591ce5719c5cbbb63db3b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-mds-c, org.label-schema.build-date=20240424, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-04-26T22:02:41.370 DEBUG:teuthology.orchestra.run.smithi164:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a1780950-0415-11ef-bc93-c7b262605968@mds.c.service' 2024-04-26T22:02:41.476 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T22:02:41.477 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-04-26T22:02:41.477 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-04-26T22:02:41.477 DEBUG:teuthology.orchestra.run.smithi164:> sudo systemctl stop ceph-a1780950-0415-11ef-bc93-c7b262605968@mds.f 2024-04-26T22:02:41.691 INFO:journalctl@ceph.mds.f.smithi164.stdout:Apr 26 22:02:41 smithi164 systemd[1]: Stopping Ceph mds.f for a1780950-0415-11ef-bc93-c7b262605968... 2024-04-26T22:02:42.440 INFO:journalctl@ceph.mds.f.smithi164.stdout:Apr 26 22:02:42 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-mds-f[62048]: 2024-04-26T22:02:42.088+0000 7fb896c00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T22:02:42.440 INFO:journalctl@ceph.mds.f.smithi164.stdout:Apr 26 22:02:42 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-mds-f[62048]: 2024-04-26T22:02:42.088+0000 7fb896c00700 -1 mds.f *** got signal Terminated *** 2024-04-26T22:02:44.417 INFO:journalctl@ceph.mds.f.smithi164.stdout:Apr 26 22:02:44 smithi164 podman[67273]: 2024-04-26 22:02:44.090485623 +0000 UTC m=+2.150828320 container died 3bb4b5de753c8267a2dbb8e1da0e0900591c523a288b966f9b9e160414c7c944 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-mds-f, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-04-26T22:02:44.690 INFO:journalctl@ceph.mds.f.smithi164.stdout:Apr 26 22:02:44 smithi164 podman[67273]: 2024-04-26 22:02:44.587982338 +0000 UTC m=+2.648325065 container cleanup 3bb4b5de753c8267a2dbb8e1da0e0900591c523a288b966f9b9e160414c7c944 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-mds-f, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff) 2024-04-26T22:02:44.690 INFO:journalctl@ceph.mds.f.smithi164.stdout:Apr 26 22:02:44 smithi164 bash[67273]: ceph-a1780950-0415-11ef-bc93-c7b262605968-mds-f 2024-04-26T22:02:45.440 INFO:journalctl@ceph.mds.f.smithi164.stdout:Apr 26 22:02:44 smithi164 podman[67287]: 2024-04-26 22:02:44.947666432 +0000 UTC m=+0.846518348 container remove 3bb4b5de753c8267a2dbb8e1da0e0900591c523a288b966f9b9e160414c7c944 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-mds-f, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-04-26T22:02:45.492 DEBUG:teuthology.orchestra.run.smithi164:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a1780950-0415-11ef-bc93-c7b262605968@mds.f.service' 2024-04-26T22:02:45.597 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T22:02:45.597 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-04-26T22:02:45.597 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-04-26T22:02:45.598 DEBUG:teuthology.orchestra.run.smithi164:> sudo systemctl stop ceph-a1780950-0415-11ef-bc93-c7b262605968@mds.i 2024-04-26T22:02:46.172 INFO:journalctl@ceph.mds.i.smithi164.stdout:Apr 26 22:02:45 smithi164 systemd[1]: Stopping Ceph mds.i for a1780950-0415-11ef-bc93-c7b262605968... 2024-04-26T22:02:46.440 INFO:journalctl@ceph.mds.i.smithi164.stdout:Apr 26 22:02:46 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-mds-i[62668]: 2024-04-26T22:02:46.171+0000 7ff1fd200700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T22:02:46.440 INFO:journalctl@ceph.mds.i.smithi164.stdout:Apr 26 22:02:46 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-mds-i[62668]: 2024-04-26T22:02:46.171+0000 7ff1fd200700 -1 mds.i *** got signal Terminated *** 2024-04-26T22:02:48.408 INFO:journalctl@ceph.mds.i.smithi164.stdout:Apr 26 22:02:48 smithi164 podman[67392]: 2024-04-26 22:02:48.070338753 +0000 UTC m=+2.019093341 container died 9b322e4f6c6d0b46dcc2efe226853e8a0fbed9c0b8a43754e95374190ce124eb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-mds-i, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424) 2024-04-26T22:02:48.409 INFO:journalctl@ceph.mds.i.smithi164.stdout:Apr 26 22:02:48 smithi164 podman[67392]: 2024-04-26 22:02:48.408128598 +0000 UTC m=+2.356883202 container cleanup 9b322e4f6c6d0b46dcc2efe226853e8a0fbed9c0b8a43754e95374190ce124eb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-mds-i, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, GIT_BRANCH=HEAD) 2024-04-26T22:02:48.690 INFO:journalctl@ceph.mds.i.smithi164.stdout:Apr 26 22:02:48 smithi164 bash[67392]: ceph-a1780950-0415-11ef-bc93-c7b262605968-mds-i 2024-04-26T22:02:48.690 INFO:journalctl@ceph.mds.i.smithi164.stdout:Apr 26 22:02:48 smithi164 podman[67406]: 2024-04-26 22:02:48.567446281 +0000 UTC m=+0.475318001 container remove 9b322e4f6c6d0b46dcc2efe226853e8a0fbed9c0b8a43754e95374190ce124eb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-mds-i, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424) 2024-04-26T22:02:49.069 DEBUG:teuthology.orchestra.run.smithi164:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a1780950-0415-11ef-bc93-c7b262605968@mds.i.service' 2024-04-26T22:02:49.175 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T22:02:49.175 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-04-26T22:02:49.175 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-04-26T22:02:49.175 DEBUG:teuthology.orchestra.run.smithi164:> sudo systemctl stop ceph-a1780950-0415-11ef-bc93-c7b262605968@mds.l 2024-04-26T22:02:49.690 INFO:journalctl@ceph.mds.l.smithi164.stdout:Apr 26 22:02:49 smithi164 systemd[1]: Stopping Ceph mds.l for a1780950-0415-11ef-bc93-c7b262605968... 2024-04-26T22:02:50.190 INFO:journalctl@ceph.mds.l.smithi164.stdout:Apr 26 22:02:49 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-mds-l[61745]: 2024-04-26T22:02:49.801+0000 7f6ec1000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T22:02:50.191 INFO:journalctl@ceph.mds.l.smithi164.stdout:Apr 26 22:02:49 smithi164 ceph-a1780950-0415-11ef-bc93-c7b262605968-mds-l[61745]: 2024-04-26T22:02:49.801+0000 7f6ec1000700 -1 mds.l *** got signal Terminated *** 2024-04-26T22:02:52.399 INFO:journalctl@ceph.mds.l.smithi164.stdout:Apr 26 22:02:52 smithi164 podman[67512]: 2024-04-26 22:02:52.071480047 +0000 UTC m=+2.441931520 container died 781cee5595e7aad29cb0722224a71255015944254d7c489e40a3affbb709e289 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-mds-l, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-04-26T22:02:52.690 INFO:journalctl@ceph.mds.l.smithi164.stdout:Apr 26 22:02:52 smithi164 podman[67512]: 2024-04-26 22:02:52.561696505 +0000 UTC m=+2.932147972 container cleanup 781cee5595e7aad29cb0722224a71255015944254d7c489e40a3affbb709e289 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-mds-l, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-04-26T22:02:52.691 INFO:journalctl@ceph.mds.l.smithi164.stdout:Apr 26 22:02:52 smithi164 bash[67512]: ceph-a1780950-0415-11ef-bc93-c7b262605968-mds-l 2024-04-26T22:02:53.190 INFO:journalctl@ceph.mds.l.smithi164.stdout:Apr 26 22:02:52 smithi164 podman[67526]: 2024-04-26 22:02:52.745891954 +0000 UTC m=+0.653405286 container remove 781cee5595e7aad29cb0722224a71255015944254d7c489e40a3affbb709e289 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-a1780950-0415-11ef-bc93-c7b262605968-mds-l, org.label-schema.build-date=20240424, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-04-26T22:02:53.292 DEBUG:teuthology.orchestra.run.smithi164:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a1780950-0415-11ef-bc93-c7b262605968@mds.l.service' 2024-04-26T22:02:53.396 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T22:02:53.396 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-04-26T22:02:53.396 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid a1780950-0415-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-26T22:02:53.712 INFO:teuthology.orchestra.run.smithi072.stdout:Deleting cluster with fsid: a1780950-0415-11ef-bc93-c7b262605968 2024-04-26T22:02:58.566 DEBUG:teuthology.orchestra.run.smithi076:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid a1780950-0415-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-26T22:02:58.884 INFO:teuthology.orchestra.run.smithi076.stdout:Deleting cluster with fsid: a1780950-0415-11ef-bc93-c7b262605968 2024-04-26T22:03:03.767 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid a1780950-0415-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-26T22:03:04.084 INFO:teuthology.orchestra.run.smithi164.stdout:Deleting cluster with fsid: a1780950-0415-11ef-bc93-c7b262605968 2024-04-26T22:03:09.073 DEBUG:teuthology.orchestra.run.smithi072:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-26T22:03:09.165 DEBUG:teuthology.orchestra.run.smithi076:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-26T22:03:09.260 DEBUG:teuthology.orchestra.run.smithi164:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-26T22:03:09.322 INFO:tasks.cephadm:Archiving crash dumps... 2024-04-26T22:03:09.324 DEBUG:teuthology.misc:Transferring archived files from smithi072:/var/lib/ceph/a1780950-0415-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675222/remote/smithi072/crash 2024-04-26T22:03:09.325 DEBUG:teuthology.orchestra.run.smithi072:> sudo tar c -f - -C /var/lib/ceph/a1780950-0415-11ef-bc93-c7b262605968/crash -- . 2024-04-26T22:03:09.429 INFO:teuthology.orchestra.run.smithi072.stderr:tar: /var/lib/ceph/a1780950-0415-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-26T22:03:09.430 INFO:teuthology.orchestra.run.smithi072.stderr:tar: Error is not recoverable: exiting now 2024-04-26T22:03:09.434 DEBUG:teuthology.misc:Transferring archived files from smithi076:/var/lib/ceph/a1780950-0415-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675222/remote/smithi076/crash 2024-04-26T22:03:09.435 DEBUG:teuthology.orchestra.run.smithi076:> sudo tar c -f - -C /var/lib/ceph/a1780950-0415-11ef-bc93-c7b262605968/crash -- . 2024-04-26T22:03:09.541 INFO:teuthology.orchestra.run.smithi076.stderr:tar: /var/lib/ceph/a1780950-0415-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-26T22:03:09.541 INFO:teuthology.orchestra.run.smithi076.stderr:tar: Error is not recoverable: exiting now 2024-04-26T22:03:09.545 DEBUG:teuthology.misc:Transferring archived files from smithi164:/var/lib/ceph/a1780950-0415-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675222/remote/smithi164/crash 2024-04-26T22:03:09.546 DEBUG:teuthology.orchestra.run.smithi164:> sudo tar c -f - -C /var/lib/ceph/a1780950-0415-11ef-bc93-c7b262605968/crash -- . 2024-04-26T22:03:09.638 INFO:teuthology.orchestra.run.smithi164.stderr:tar: /var/lib/ceph/a1780950-0415-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-26T22:03:09.638 INFO:teuthology.orchestra.run.smithi164.stderr:tar: Error is not recoverable: exiting now 2024-04-26T22:03:09.641 INFO:tasks.cephadm:Checking cluster log for badness... 2024-04-26T22:03:09.641 DEBUG:teuthology.orchestra.run.smithi072:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/a1780950-0415-11ef-bc93-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-04-26T22:03:09.747 INFO:tasks.cephadm:Compressing logs... 2024-04-26T22:03:09.747 DEBUG:teuthology.orchestra.run.smithi072:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-26T22:03:09.751 DEBUG:teuthology.orchestra.run.smithi076:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-26T22:03:09.758 DEBUG:teuthology.orchestra.run.smithi164:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-26T22:03:09.807 INFO:teuthology.orchestra.run.smithi072.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-04-26T22:03:09.807 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.log 2024-04-26T22:03:09.808 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/a1780950-0415-11ef-bc93-c7b262605968/ceph.log 2024-04-26T22:03:09.809 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.0.log: gzip -5 --verbose -- /var/log/ceph/a1780950-0415-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-26T22:03:09.809 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/a1780950-0415-11ef-bc93-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/a1780950-0415-11ef-bc93-c7b262605968/ceph-mds.g.log 2024-04-26T22:03:09.811 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/a1780950-0415-11ef-bc93-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/a1780950-0415-11ef-bc93-c7b262605968/ceph-mds.j.log 2024-04-26T22:03:09.812 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/a1780950-0415-11ef-bc93-c7b262605968/ceph-mds.g.log: 86.9% -- replaced with /var/log/ceph/a1780950-0415-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-26T22:03:09.812 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/a1780950-0415-11ef-bc93-c7b262605968/ceph-osd.3.log 2024-04-26T22:03:09.813 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/a1780950-0415-11ef-bc93-c7b262605968/ceph-osd.0.log 2024-04-26T22:03:09.814 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/a1780950-0415-11ef-bc93-c7b262605968/ceph-mds.j.log: /var/log/ceph/a1780950-0415-11ef-bc93-c7b262605968/ceph-osd.3.log: 90.2% -- replaced with /var/log/ceph/a1780950-0415-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-26T22:03:09.814 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/a1780950-0415-11ef-bc93-c7b262605968/ceph-mgr.x.log 2024-04-26T22:03:09.815 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/a1780950-0415-11ef-bc93-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/a1780950-0415-11ef-bc93-c7b262605968/ceph-mon.a.log 2024-04-26T22:03:09.817 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/a1780950-0415-11ef-bc93-c7b262605968/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/a1780950-0415-11ef-bc93-c7b262605968/ceph-osd.6.log 2024-04-26T22:03:09.817 INFO:teuthology.orchestra.run.smithi072.stderr: 94.7% -- replaced with /var/log/ceph/a1780950-0415-11ef-bc93-c7b262605968/ceph-mds.j.log.gz 2024-04-26T22:03:09.818 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/a1780950-0415-11ef-bc93-c7b262605968/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/a1780950-0415-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-26T22:03:09.819 INFO:teuthology.orchestra.run.smithi164.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-04-26T22:03:09.821 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/a1780950-0415-11ef-bc93-c7b262605968/ceph-osd.6.log: /var/log/ceph/a1780950-0415-11ef-bc93-c7b262605968/ceph.cephadm.log: 83.0% -- replaced with /var/log/ceph/a1780950-0415-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-26T22:03:09.822 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/a1780950-0415-11ef-bc93-c7b262605968/ceph-mds.a.log 2024-04-26T22:03:09.824 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/a1780950-0415-11ef-bc93-c7b262605968/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/a1780950-0415-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-26T22:03:09.827 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/a1780950-0415-11ef-bc93-c7b262605968/ceph.log 2024-04-26T22:03:09.828 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/a1780950-0415-11ef-bc93-c7b262605968/ceph-osd.2.log 2024-04-26T22:03:09.829 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/a1780950-0415-11ef-bc93-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/a1780950-0415-11ef-bc93-c7b262605968/ceph-mds.d.log 2024-04-26T22:03:09.829 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/a1780950-0415-11ef-bc93-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/a1780950-0415-11ef-bc93-c7b262605968/ceph-mds.f.log 2024-04-26T22:03:09.830 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/a1780950-0415-11ef-bc93-c7b262605968/ceph-osd.9.log 2024-04-26T22:03:09.831 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/a1780950-0415-11ef-bc93-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/a1780950-0415-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-26T22:03:09.831 INFO:teuthology.orchestra.run.smithi164.stderr: 86.9% -- replaced with /var/log/ceph/a1780950-0415-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-26T22:03:09.832 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/a1780950-0415-11ef-bc93-c7b262605968/ceph-mds.f.log: gzip -5 --verbose -- /var/log/ceph/a1780950-0415-11ef-bc93-c7b262605968/ceph-osd.5.log 2024-04-26T22:03:09.833 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/a1780950-0415-11ef-bc93-c7b262605968/ceph-mds.c.log 2024-04-26T22:03:09.834 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/a1780950-0415-11ef-bc93-c7b262605968/ceph.audit.log: /var/log/ceph/a1780950-0415-11ef-bc93-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/a1780950-0415-11ef-bc93-c7b262605968/ceph-mon.c.log 2024-04-26T22:03:09.835 INFO:teuthology.orchestra.run.smithi076.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-04-26T22:03:09.835 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/a1780950-0415-11ef-bc93-c7b262605968/ceph.log 2024-04-26T22:03:09.836 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/a1780950-0415-11ef-bc93-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/a1780950-0415-11ef-bc93-c7b262605968/ceph-mgr.z.log 2024-04-26T22:03:09.836 INFO:teuthology.orchestra.run.smithi164.stderr: 90.4% -- replaced with /var/log/ceph/a1780950-0415-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-26T22:03:09.837 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/a1780950-0415-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-26T22:03:09.837 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/a1780950-0415-11ef-bc93-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/a1780950-0415-11ef-bc93-c7b262605968/ceph-osd.7.log 2024-04-26T22:03:09.838 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/a1780950-0415-11ef-bc93-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/a1780950-0415-11ef-bc93-c7b262605968/ceph-mds.b.log 2024-04-26T22:03:09.840 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/a1780950-0415-11ef-bc93-c7b262605968/ceph-osd.7.log: 86.9% -- replaced with /var/log/ceph/a1780950-0415-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-26T22:03:09.840 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/a1780950-0415-11ef-bc93-c7b262605968/ceph-osd.4.log 2024-04-26T22:03:09.840 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/a1780950-0415-11ef-bc93-c7b262605968/ceph-mds.d.log: /var/log/ceph/a1780950-0415-11ef-bc93-c7b262605968/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-26T22:03:09.840 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/a1780950-0415-11ef-bc93-c7b262605968/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/a1780950-0415-11ef-bc93-c7b262605968/ceph-osd.1.log 2024-04-26T22:03:09.841 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/a1780950-0415-11ef-bc93-c7b262605968/ceph-osd.4.log: 90.4% -- replaced with /var/log/ceph/a1780950-0415-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-26T22:03:09.841 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/a1780950-0415-11ef-bc93-c7b262605968/ceph-mgr.y.log 2024-04-26T22:03:09.843 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/a1780950-0415-11ef-bc93-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/a1780950-0415-11ef-bc93-c7b262605968/ceph-mds.h.log 2024-04-26T22:03:09.844 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/a1780950-0415-11ef-bc93-c7b262605968/ceph-mon.c.log: /var/log/ceph/a1780950-0415-11ef-bc93-c7b262605968/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/a1780950-0415-11ef-bc93-c7b262605968/ceph-mds.i.log 2024-04-26T22:03:09.844 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/a1780950-0415-11ef-bc93-c7b262605968/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/a1780950-0415-11ef-bc93-c7b262605968/ceph-osd.10.log 2024-04-26T22:03:09.845 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/a1780950-0415-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-26T22:03:09.846 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/cephadm.log: 94.6% -- replaced with /var/log/ceph/a1780950-0415-11ef-bc93-c7b262605968/ceph-mds.d.log.gz 2024-04-26T22:03:09.846 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/a1780950-0415-11ef-bc93-c7b262605968/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/a1780950-0415-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-26T22:03:09.847 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/a1780950-0415-11ef-bc93-c7b262605968/ceph-mds.i.log: 92.6% -- replaced with /var/log/ceph/a1780950-0415-11ef-bc93-c7b262605968/ceph-mgr.z.log.gz 2024-04-26T22:03:09.849 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/a1780950-0415-11ef-bc93-c7b262605968/ceph-osd.11.log 2024-04-26T22:03:09.849 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/a1780950-0415-11ef-bc93-c7b262605968/ceph.cephadm.log: 82.6% -- replaced with /var/log/ceph/a1780950-0415-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-26T22:03:09.851 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/a1780950-0415-11ef-bc93-c7b262605968/ceph-osd.10.log: /var/log/ceph/a1780950-0415-11ef-bc93-c7b262605968/ceph.cephadm.log: 82.7% -- replaced with /var/log/ceph/a1780950-0415-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-26T22:03:09.851 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/a1780950-0415-11ef-bc93-c7b262605968/ceph-mds.e.log 2024-04-26T22:03:09.852 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/a1780950-0415-11ef-bc93-c7b262605968/ceph-osd.8.log 2024-04-26T22:03:09.853 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/a1780950-0415-11ef-bc93-c7b262605968/ceph-mds.e.log: gzip -5 --verbose -- /var/log/ceph/a1780950-0415-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-26T22:03:09.853 INFO:teuthology.orchestra.run.smithi076.stderr: 92.4% -- replaced with /var/log/ceph/a1780950-0415-11ef-bc93-c7b262605968/ceph-mgr.y.log.gz 2024-04-26T22:03:09.855 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/a1780950-0415-11ef-bc93-c7b262605968/ceph-mon.b.log 2024-04-26T22:03:09.857 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/a1780950-0415-11ef-bc93-c7b262605968/ceph-osd.11.log: /var/log/ceph/a1780950-0415-11ef-bc93-c7b262605968/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/a1780950-0415-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-26T22:03:09.861 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/a1780950-0415-11ef-bc93-c7b262605968/ceph-mds.l.log 2024-04-26T22:03:09.862 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/a1780950-0415-11ef-bc93-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/a1780950-0415-11ef-bc93-c7b262605968/ceph-mds.k.log 2024-04-26T22:03:09.862 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/a1780950-0415-11ef-bc93-c7b262605968/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-26T22:03:09.862 INFO:teuthology.orchestra.run.smithi072.stderr: 93.2% 94.1% -- replaced with /var/log/ceph/a1780950-0415-11ef-bc93-c7b262605968/ceph-mds.g.log.gz 2024-04-26T22:03:09.864 INFO:teuthology.orchestra.run.smithi072.stderr: -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-26T22:03:09.865 INFO:teuthology.orchestra.run.smithi072.stderr: 90.9% -- replaced with /var/log/ceph/a1780950-0415-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-26T22:03:09.866 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/a1780950-0415-11ef-bc93-c7b262605968/ceph-volume.log: /var/log/ceph/a1780950-0415-11ef-bc93-c7b262605968/ceph-mds.l.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-26T22:03:09.872 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/a1780950-0415-11ef-bc93-c7b262605968/ceph-mds.k.log: /var/log/ceph/cephadm.log: 92.0% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-26T22:03:09.875 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/cephadm.log: 92.3% 94.2% -- replaced with /var/log/ceph/a1780950-0415-11ef-bc93-c7b262605968/ceph-mds.f.log.gz 2024-04-26T22:03:09.875 INFO:teuthology.orchestra.run.smithi164.stderr: -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-26T22:03:09.876 INFO:teuthology.orchestra.run.smithi076.stderr: 90.9% -- replaced with /var/log/ceph/a1780950-0415-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-26T22:03:09.885 INFO:teuthology.orchestra.run.smithi164.stderr: 90.9% -- replaced with /var/log/ceph/a1780950-0415-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-26T22:03:09.898 INFO:teuthology.orchestra.run.smithi072.stderr: 94.1% -- replaced with /var/log/ceph/a1780950-0415-11ef-bc93-c7b262605968/ceph-mds.a.log.gz 2024-04-26T22:03:09.904 INFO:teuthology.orchestra.run.smithi164.stderr: 92.4% -- replaced with /var/log/ceph/a1780950-0415-11ef-bc93-c7b262605968/ceph-mds.i.log.gz 2024-04-26T22:03:09.904 INFO:teuthology.orchestra.run.smithi076.stderr: 92.3% -- replaced with /var/log/ceph/a1780950-0415-11ef-bc93-c7b262605968/ceph-mds.e.log.gz 2024-04-26T22:03:09.907 INFO:teuthology.orchestra.run.smithi164.stderr: 94.1% -- replaced with /var/log/ceph/a1780950-0415-11ef-bc93-c7b262605968/ceph-mds.l.log.gz 2024-04-26T22:03:09.909 INFO:teuthology.orchestra.run.smithi164.stderr: 93.6% -- replaced with /var/log/ceph/a1780950-0415-11ef-bc93-c7b262605968/ceph-mds.c.log.gz 2024-04-26T22:03:09.911 INFO:teuthology.orchestra.run.smithi076.stderr: 92.3% -- replaced with /var/log/ceph/a1780950-0415-11ef-bc93-c7b262605968/ceph-mds.h.log.gz 2024-04-26T22:03:09.918 INFO:teuthology.orchestra.run.smithi076.stderr: 92.2% -- replaced with /var/log/ceph/a1780950-0415-11ef-bc93-c7b262605968/ceph-mds.k.log.gz 2024-04-26T22:03:10.104 INFO:teuthology.orchestra.run.smithi072.stderr: 89.8% -- replaced with /var/log/ceph/a1780950-0415-11ef-bc93-c7b262605968/ceph-mgr.x.log.gz 2024-04-26T22:03:10.178 INFO:teuthology.orchestra.run.smithi076.stderr: 92.6% -- replaced with /var/log/ceph/a1780950-0415-11ef-bc93-c7b262605968/ceph-mds.b.log.gz 2024-04-26T22:03:10.183 INFO:teuthology.orchestra.run.smithi164.stderr: 91.9% -- replaced with /var/log/ceph/a1780950-0415-11ef-bc93-c7b262605968/ceph-mon.c.log.gz 2024-04-26T22:03:10.215 INFO:teuthology.orchestra.run.smithi076.stderr: 92.0% -- replaced with /var/log/ceph/a1780950-0415-11ef-bc93-c7b262605968/ceph-mon.b.log.gz 2024-04-26T22:03:10.653 INFO:teuthology.orchestra.run.smithi072.stderr: 93.5% -- replaced with /var/log/ceph/a1780950-0415-11ef-bc93-c7b262605968/ceph-osd.9.log.gz 2024-04-26T22:03:10.775 INFO:teuthology.orchestra.run.smithi072.stderr: 90.5% -- replaced with /var/log/ceph/a1780950-0415-11ef-bc93-c7b262605968/ceph-mon.a.log.gz 2024-04-26T22:03:10.895 INFO:teuthology.orchestra.run.smithi164.stderr: 93.3% -- replaced with /var/log/ceph/a1780950-0415-11ef-bc93-c7b262605968/ceph-osd.8.log.gz 2024-04-26T22:03:11.234 INFO:teuthology.orchestra.run.smithi164.stderr: 93.4% -- replaced with /var/log/ceph/a1780950-0415-11ef-bc93-c7b262605968/ceph-osd.2.log.gz 2024-04-26T22:03:11.425 INFO:teuthology.orchestra.run.smithi164.stderr: 93.4% -- replaced with /var/log/ceph/a1780950-0415-11ef-bc93-c7b262605968/ceph-osd.5.log.gz 2024-04-26T22:03:11.455 INFO:teuthology.orchestra.run.smithi076.stderr: 93.5% -- replaced with /var/log/ceph/a1780950-0415-11ef-bc93-c7b262605968/ceph-osd.4.log.gz 2024-04-26T22:03:11.966 INFO:teuthology.orchestra.run.smithi076.stderr: 93.4% -- replaced with /var/log/ceph/a1780950-0415-11ef-bc93-c7b262605968/ceph-osd.10.log.gz 2024-04-26T22:03:12.012 INFO:teuthology.orchestra.run.smithi072.stderr: 93.6% -- replaced with /var/log/ceph/a1780950-0415-11ef-bc93-c7b262605968/ceph-osd.3.log.gz 2024-04-26T22:03:12.304 INFO:teuthology.orchestra.run.smithi164.stderr: 93.6% -- replaced with /var/log/ceph/a1780950-0415-11ef-bc93-c7b262605968/ceph-osd.11.log.gz 2024-04-26T22:03:12.308 INFO:teuthology.orchestra.run.smithi164.stderr: 2024-04-26T22:03:12.308 INFO:teuthology.orchestra.run.smithi164.stderr:real 0m2.512s 2024-04-26T22:03:12.308 INFO:teuthology.orchestra.run.smithi164.stderr:user 0m6.426s 2024-04-26T22:03:12.308 INFO:teuthology.orchestra.run.smithi164.stderr:sys 0m0.653s 2024-04-26T22:03:12.493 INFO:teuthology.orchestra.run.smithi072.stderr: 93.5% -- replaced with /var/log/ceph/a1780950-0415-11ef-bc93-c7b262605968/ceph-osd.0.log.gz 2024-04-26T22:03:12.708 INFO:teuthology.orchestra.run.smithi076.stderr: 93.6% -- replaced with /var/log/ceph/a1780950-0415-11ef-bc93-c7b262605968/ceph-osd.7.log.gz 2024-04-26T22:03:12.723 INFO:teuthology.orchestra.run.smithi072.stderr: 93.5% -- replaced with /var/log/ceph/a1780950-0415-11ef-bc93-c7b262605968/ceph-osd.6.log.gz 2024-04-26T22:03:13.263 INFO:teuthology.orchestra.run.smithi076.stderr: 93.7% -- replaced with /var/log/ceph/a1780950-0415-11ef-bc93-c7b262605968/ceph-osd.1.log.gz 2024-04-26T22:03:13.267 INFO:teuthology.orchestra.run.smithi076.stderr: 2024-04-26T22:03:13.267 INFO:teuthology.orchestra.run.smithi076.stderr:real 0m3.462s 2024-04-26T22:03:13.267 INFO:teuthology.orchestra.run.smithi076.stderr:user 0m10.067s 2024-04-26T22:03:13.267 INFO:teuthology.orchestra.run.smithi076.stderr:sys 0m0.889s 2024-04-26T22:03:16.855 INFO:teuthology.orchestra.run.smithi072.stderr: 96.0% -- replaced with /var/log/ceph/ceph-client.0.log.gz 2024-04-26T22:03:16.859 INFO:teuthology.orchestra.run.smithi072.stderr: 2024-04-26T22:03:16.859 INFO:teuthology.orchestra.run.smithi072.stderr:real 0m7.076s 2024-04-26T22:03:16.859 INFO:teuthology.orchestra.run.smithi072.stderr:user 0m15.552s 2024-04-26T22:03:16.859 INFO:teuthology.orchestra.run.smithi072.stderr:sys 0m1.391s 2024-04-26T22:03:16.860 INFO:tasks.cephadm:Archiving logs... 2024-04-26T22:03:16.860 DEBUG:teuthology.misc:Transferring archived files from smithi072:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675222/remote/smithi072/log 2024-04-26T22:03:16.861 DEBUG:teuthology.orchestra.run.smithi072:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-26T22:03:18.980 DEBUG:teuthology.misc:Transferring archived files from smithi076:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675222/remote/smithi076/log 2024-04-26T22:03:18.981 DEBUG:teuthology.orchestra.run.smithi076:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-26T22:03:20.429 DEBUG:teuthology.misc:Transferring archived files from smithi164:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675222/remote/smithi164/log 2024-04-26T22:03:20.430 DEBUG:teuthology.orchestra.run.smithi164:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-26T22:03:21.476 INFO:tasks.cephadm:Removing cluster... 2024-04-26T22:03:21.476 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid a1780950-0415-11ef-bc93-c7b262605968 --force 2024-04-26T22:03:21.797 INFO:teuthology.orchestra.run.smithi072.stdout:Deleting cluster with fsid: a1780950-0415-11ef-bc93-c7b262605968 2024-04-26T22:03:22.437 DEBUG:teuthology.orchestra.run.smithi076:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid a1780950-0415-11ef-bc93-c7b262605968 --force 2024-04-26T22:03:22.762 INFO:teuthology.orchestra.run.smithi076.stdout:Deleting cluster with fsid: a1780950-0415-11ef-bc93-c7b262605968 2024-04-26T22:03:23.336 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid a1780950-0415-11ef-bc93-c7b262605968 --force 2024-04-26T22:03:23.657 INFO:teuthology.orchestra.run.smithi164.stdout:Deleting cluster with fsid: a1780950-0415-11ef-bc93-c7b262605968 2024-04-26T22:03:24.168 INFO:tasks.cephadm:Removing cephadm ... 2024-04-26T22:03:24.169 DEBUG:teuthology.orchestra.run.smithi072:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-26T22:03:24.226 DEBUG:teuthology.orchestra.run.smithi076:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-26T22:03:24.283 DEBUG:teuthology.orchestra.run.smithi164:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-26T22:03:24.336 INFO:tasks.cephadm:Teardown complete 2024-04-26T22:03:24.336 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-26T22:03:24.350 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-26T22:03:24.350 DEBUG:teuthology.orchestra.run.smithi072:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-26T22:03:24.358 DEBUG:teuthology.orchestra.run.smithi076:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-26T22:03:24.366 DEBUG:teuthology.orchestra.run.smithi164:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-26T22:03:24.526 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-26T22:03:24.537 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-26T22:03:24.537 DEBUG:teuthology.orchestra.run.smithi072:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-26T22:03:24.544 DEBUG:teuthology.orchestra.run.smithi076:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-26T22:03:24.552 DEBUG:teuthology.orchestra.run.smithi164:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-26T22:03:24.590 INFO:teuthology.orchestra.run.smithi072.stderr:bash: line 1: ntpq: command not found 2024-04-26T22:03:24.599 INFO:teuthology.orchestra.run.smithi076.stderr:bash: line 1: ntpq: command not found 2024-04-26T22:03:24.600 INFO:teuthology.orchestra.run.smithi072.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-26T22:03:24.600 INFO:teuthology.orchestra.run.smithi072.stdout:=============================================================================== 2024-04-26T22:03:24.600 INFO:teuthology.orchestra.run.smithi072.stdout:^* hv01.front.sepia.ceph.com 3 7 377 115 +1477us[+1502us] +/- 62ms 2024-04-26T22:03:24.600 INFO:teuthology.orchestra.run.smithi072.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 53 -192us[ -192us] +/- 67ms 2024-04-26T22:03:24.600 INFO:teuthology.orchestra.run.smithi072.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 58 -1948us[-1948us] +/- 83ms 2024-04-26T22:03:24.600 INFO:teuthology.orchestra.run.smithi072.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T22:03:24.607 INFO:teuthology.orchestra.run.smithi164.stderr:bash: line 1: ntpq: command not found 2024-04-26T22:03:24.610 INFO:teuthology.orchestra.run.smithi076.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-26T22:03:24.610 INFO:teuthology.orchestra.run.smithi076.stdout:=============================================================================== 2024-04-26T22:03:24.610 INFO:teuthology.orchestra.run.smithi076.stdout:^* hv01.front.sepia.ceph.com 3 6 377 53 +1475us[+1470us] +/- 63ms 2024-04-26T22:03:24.610 INFO:teuthology.orchestra.run.smithi076.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 50 -193us[ -193us] +/- 67ms 2024-04-26T22:03:24.610 INFO:teuthology.orchestra.run.smithi076.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 51 -2020us[-2020us] +/- 83ms 2024-04-26T22:03:24.610 INFO:teuthology.orchestra.run.smithi076.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T22:03:24.617 INFO:teuthology.orchestra.run.smithi164.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-26T22:03:24.617 INFO:teuthology.orchestra.run.smithi164.stdout:=============================================================================== 2024-04-26T22:03:24.617 INFO:teuthology.orchestra.run.smithi164.stdout:^* hv01.front.sepia.ceph.com 3 6 377 53 +1544us[+1531us] +/- 63ms 2024-04-26T22:03:24.617 INFO:teuthology.orchestra.run.smithi164.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 55 -78us[ -91us] +/- 67ms 2024-04-26T22:03:24.618 INFO:teuthology.orchestra.run.smithi164.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 51 -1915us[-1915us] +/- 83ms 2024-04-26T22:03:24.618 INFO:teuthology.orchestra.run.smithi164.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T22:03:24.619 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-26T22:03:24.630 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-26T22:03:24.631 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-26T22:03:24.664 DEBUG:teuthology.orchestra.run.smithi072:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-26T22:03:24.861 DEBUG:teuthology.orchestra.run.smithi076:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-26T22:03:25.039 DEBUG:teuthology.orchestra.run.smithi164:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-26T22:03:25.240 DEBUG:teuthology.orchestra.run.smithi072:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-26T22:03:25.317 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T22:03:25.317 DEBUG:teuthology.orchestra.run.smithi076:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-26T22:03:25.408 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T22:03:25.409 DEBUG:teuthology.orchestra.run.smithi164:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-26T22:03:25.503 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T22:03:25.504 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-26T22:03:25.515 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-26T22:03:25.525 INFO:teuthology.task.internal:Duration was 1857.613336 seconds 2024-04-26T22:03:25.526 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-26T22:03:25.536 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-26T22:03:25.536 DEBUG:teuthology.orchestra.run.smithi072:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-26T22:03:25.544 DEBUG:teuthology.orchestra.run.smithi076:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-26T22:03:25.551 DEBUG:teuthology.orchestra.run.smithi164:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-26T22:03:25.653 INFO:teuthology.orchestra.run.smithi072.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-26T22:03:25.677 INFO:teuthology.orchestra.run.smithi076.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-26T22:03:25.698 INFO:teuthology.orchestra.run.smithi164.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-26T22:03:26.160 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-26T22:03:26.160 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi072.front.sepia.ceph.com 2024-04-26T22:03:26.160 DEBUG:teuthology.orchestra.run.smithi072:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-26T22:03:26.218 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi076.front.sepia.ceph.com 2024-04-26T22:03:26.219 DEBUG:teuthology.orchestra.run.smithi076:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-26T22:03:26.286 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi164.front.sepia.ceph.com 2024-04-26T22:03:26.286 DEBUG:teuthology.orchestra.run.smithi164:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-26T22:03:26.366 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-26T22:03:26.366 DEBUG:teuthology.orchestra.run.smithi072:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-26T22:03:26.374 DEBUG:teuthology.orchestra.run.smithi076:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-26T22:03:26.382 DEBUG:teuthology.orchestra.run.smithi164:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-26T22:03:26.466 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-26T22:03:26.466 DEBUG:teuthology.orchestra.run.smithi072:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-26T22:03:26.471 DEBUG:teuthology.orchestra.run.smithi076:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-26T22:03:26.476 DEBUG:teuthology.orchestra.run.smithi164:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-26T22:03:26.968 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-26T22:03:26.979 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-26T22:03:26.979 DEBUG:teuthology.orchestra.run.smithi072:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-26T22:03:27.062 DEBUG:teuthology.orchestra.run.smithi076:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-26T22:03:27.153 DEBUG:teuthology.orchestra.run.smithi164:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-26T22:03:27.243 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-26T22:03:27.255 DEBUG:teuthology.orchestra.run.smithi072:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-26T22:03:27.262 DEBUG:teuthology.orchestra.run.smithi076:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-26T22:03:27.270 DEBUG:teuthology.orchestra.run.smithi164:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-26T22:03:27.356 INFO:teuthology.orchestra.run.smithi072.stdout:kernel.core_pattern = core 2024-04-26T22:03:27.364 INFO:teuthology.orchestra.run.smithi076.stdout:kernel.core_pattern = core 2024-04-26T22:03:27.370 INFO:teuthology.orchestra.run.smithi164.stdout:kernel.core_pattern = core 2024-04-26T22:03:27.430 DEBUG:teuthology.orchestra.run.smithi072:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-26T22:03:27.483 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T22:03:27.484 DEBUG:teuthology.orchestra.run.smithi076:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-26T22:03:27.539 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T22:03:27.539 DEBUG:teuthology.orchestra.run.smithi164:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-26T22:03:27.594 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T22:03:27.595 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-26T22:03:27.607 INFO:teuthology.task.internal:Transferring archived files... 2024-04-26T22:03:27.608 DEBUG:teuthology.misc:Transferring archived files from smithi072:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675222/remote/smithi072 2024-04-26T22:03:27.608 DEBUG:teuthology.orchestra.run.smithi072:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-26T22:03:27.712 DEBUG:teuthology.misc:Transferring archived files from smithi076:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675222/remote/smithi076 2024-04-26T22:03:27.712 DEBUG:teuthology.orchestra.run.smithi076:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-26T22:03:27.811 DEBUG:teuthology.misc:Transferring archived files from smithi164:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675222/remote/smithi164 2024-04-26T22:03:27.811 DEBUG:teuthology.orchestra.run.smithi164:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-26T22:03:27.910 INFO:teuthology.task.internal:Removing archive directory... 2024-04-26T22:03:27.910 DEBUG:teuthology.orchestra.run.smithi072:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-26T22:03:27.918 DEBUG:teuthology.orchestra.run.smithi076:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-26T22:03:27.926 DEBUG:teuthology.orchestra.run.smithi164:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-26T22:03:27.983 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-26T22:03:27.995 INFO:teuthology.task.internal:Not uploading archives. 2024-04-26T22:03:27.996 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-26T22:03:28.006 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-26T22:03:28.007 DEBUG:teuthology.orchestra.run.smithi072:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-26T22:03:28.012 DEBUG:teuthology.orchestra.run.smithi076:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-26T22:03:28.019 DEBUG:teuthology.orchestra.run.smithi164:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-26T22:03:28.059 INFO:teuthology.orchestra.run.smithi072.stdout: 393229 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 26 22:03 /home/ubuntu/cephtest 2024-04-26T22:03:28.068 INFO:teuthology.orchestra.run.smithi076.stdout: 393229 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 26 22:03 /home/ubuntu/cephtest 2024-04-26T22:03:28.072 INFO:teuthology.orchestra.run.smithi164.stdout: 393229 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 26 22:03 /home/ubuntu/cephtest 2024-04-26T22:03:28.075 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-26T22:03:28.088 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-26T22:03:28.117 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/fuse objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/no 5-workunit/suites/fsync-tester}} duration: 1857.6133360862732 flavor: default mon.a-kernel-sha1: 806ec36b21129f113ce356c007dd59694d340b37 mon.b-kernel-sha1: 806ec36b21129f113ce356c007dd59694d340b37 mon.c-kernel-sha1: 806ec36b21129f113ce356c007dd59694d340b37 owner: scheduled_rishabh@teuthology success: true 2024-04-26T22:03:28.117 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-26T22:03:28.229 INFO:teuthology.run:pass